2024-09-10T08:18:01.921 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T08:18:01.948 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T08:18:02.046 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898917 branch: wip-rishabh-testing-20240909.165402-squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} email: null first_in_suite: false job_id: '7898917' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240909.165402-squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240909.165402-squid sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9893 sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b sleep_before_teardown: 0 subset: 197/1024 suite: fs suite_branch: wip-rishabh-testing-20240909.165402-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi089.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCxr8tgq7oMY1hGMOTo3MDhMKxRYK2ubULlbyUVoCcLcR0nA31H14++bnlqXY+QMxm5UvW1DZ6lnC/dRWSdC+5s= smithi111.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD7gdJBPMpRxVcvTbJb4EcMcBzasBstnP4vZlB751pyMpFfRDfp8SEahQRR/GuZIiJ93DHTWAk8DYwBq+J7TNR8= smithi149.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMUp+G9qbTLf06ayyyv31VY36CqiqMrav74mT+1PSvZDsVuQvGUkYVZlqFIDqrNEndNNWNikeeIzxnV42zGzn2U= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_06:58:10 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T08:18:02.046 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa; will attempt to use it 2024-09-10T08:18:02.047 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks 2024-09-10T08:18:02.047 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T08:18:02.049 INFO:teuthology.task.internal:Checking packages... 2024-09-10T08:18:02.070 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1cf819b7212a6048e92fc78c4eff963344f6d56b' 2024-09-10T08:18:02.070 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T08:18:02.070 INFO:teuthology.packaging:ref: None 2024-09-10T08:18:02.070 INFO:teuthology.packaging:tag: None 2024-09-10T08:18:02.070 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240909.165402-squid 2024-09-10T08:18:02.070 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:18:02.071 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240909.165402-squid 2024-09-10T08:18:02.221 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-305.g1cf819b7 2024-09-10T08:18:02.222 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T08:18:02.234 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T08:18:02.234 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T08:18:02.247 INFO:teuthology.task.internal:Saving configuration 2024-09-10T08:18:02.265 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T08:18:02.277 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T08:18:02.303 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898917', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 08:12:30.209303', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCxr8tgq7oMY1hGMOTo3MDhMKxRYK2ubULlbyUVoCcLcR0nA31H14++bnlqXY+QMxm5UvW1DZ6lnC/dRWSdC+5s='} 2024-09-10T08:18:02.323 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898917', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 08:12:30.207535', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD7gdJBPMpRxVcvTbJb4EcMcBzasBstnP4vZlB751pyMpFfRDfp8SEahQRR/GuZIiJ93DHTWAk8DYwBq+J7TNR8='} 2024-09-10T08:18:02.349 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898917', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 08:12:30.210443', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMUp+G9qbTLf06ayyyv31VY36CqiqMrav74mT+1PSvZDsVuQvGUkYVZlqFIDqrNEndNNWNikeeIzxnV42zGzn2U='} 2024-09-10T08:18:02.349 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T08:18:02.361 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-10T08:18:02.361 INFO:teuthology.task.internal:roles: ubuntu@smithi111.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-10T08:18:02.361 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-10T08:18:02.361 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T08:18:02.447 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbdf2a5bbe0>, signals=[15]) 2024-09-10T08:18:02.447 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T08:18:02.458 INFO:teuthology.task.internal:Opening connections... 2024-09-10T08:18:02.458 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi089.front.sepia.ceph.com 2024-09-10T08:18:02.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:18:02.535 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi111.front.sepia.ceph.com 2024-09-10T08:18:02.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:18:02.610 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi149.front.sepia.ceph.com 2024-09-10T08:18:02.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:18:02.683 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T08:18:02.695 DEBUG:teuthology.orchestra.run.smithi089:> uname -m 2024-09-10T08:18:02.714 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2024-09-10T08:18:02.714 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2024-09-10T08:18:02.772 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Stream" 2024-09-10T08:18:02.773 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="9" 2024-09-10T08:18:02.773 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos" 2024-09-10T08:18:02.773 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora" 2024-09-10T08:18:02.773 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="9" 2024-09-10T08:18:02.773 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el9" 2024-09-10T08:18:02.773 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T08:18:02.773 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2024-09-10T08:18:02.773 INFO:teuthology.orchestra.run.smithi089.stdout:LOGO="fedora-logo-icon" 2024-09-10T08:18:02.773 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T08:18:02.773 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://centos.org/" 2024-09-10T08:18:02.773 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T08:18:02.773 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T08:18:02.773 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T08:18:02.774 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2024-09-10T08:18:02.789 DEBUG:teuthology.orchestra.run.smithi111:> uname -m 2024-09-10T08:18:02.808 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64 2024-09-10T08:18:02.809 DEBUG:teuthology.orchestra.run.smithi111:> cat /etc/os-release 2024-09-10T08:18:02.866 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="CentOS Stream" 2024-09-10T08:18:02.867 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="9" 2024-09-10T08:18:02.867 INFO:teuthology.orchestra.run.smithi111.stdout:ID="centos" 2024-09-10T08:18:02.867 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="rhel fedora" 2024-09-10T08:18:02.867 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="9" 2024-09-10T08:18:02.867 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el9" 2024-09-10T08:18:02.867 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T08:18:02.867 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31" 2024-09-10T08:18:02.867 INFO:teuthology.orchestra.run.smithi111.stdout:LOGO="fedora-logo-icon" 2024-09-10T08:18:02.867 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T08:18:02.867 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://centos.org/" 2024-09-10T08:18:02.867 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T08:18:02.867 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T08:18:02.868 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T08:18:02.868 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2024-09-10T08:18:02.889 DEBUG:teuthology.orchestra.run.smithi149:> uname -m 2024-09-10T08:18:02.908 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2024-09-10T08:18:02.908 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2024-09-10T08:18:02.966 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream" 2024-09-10T08:18:02.966 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9" 2024-09-10T08:18:02.967 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos" 2024-09-10T08:18:02.967 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora" 2024-09-10T08:18:02.967 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9" 2024-09-10T08:18:02.967 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9" 2024-09-10T08:18:02.967 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T08:18:02.967 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2024-09-10T08:18:02.967 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon" 2024-09-10T08:18:02.967 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T08:18:02.967 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/" 2024-09-10T08:18:02.967 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T08:18:02.967 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T08:18:02.967 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T08:18:02.968 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2024-09-10T08:18:03.131 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T08:18:03.144 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T08:18:03.155 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T08:18:03.155 DEBUG:teuthology.orchestra.run.smithi089:> test '!' -e /home/ubuntu/cephtest 2024-09-10T08:18:03.158 DEBUG:teuthology.orchestra.run.smithi111:> test '!' -e /home/ubuntu/cephtest 2024-09-10T08:18:03.160 DEBUG:teuthology.orchestra.run.smithi149:> test '!' -e /home/ubuntu/cephtest 2024-09-10T08:18:03.177 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T08:18:03.183 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T08:18:03.183 DEBUG:teuthology.orchestra.run.smithi089:> test -z $(ls -A /var/lib/ceph) 2024-09-10T08:18:03.215 DEBUG:teuthology.orchestra.run.smithi111:> test -z $(ls -A /var/lib/ceph) 2024-09-10T08:18:03.217 DEBUG:teuthology.orchestra.run.smithi149:> test -z $(ls -A /var/lib/ceph) 2024-09-10T08:18:03.456 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T08:18:03.522 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T08:18:03.539 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-10T08:18:03.539 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-10T08:18:03.539 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-10T08:18:03.539 DEBUG:teuthology.orchestra.run.smithi089:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T08:18:03.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:18:03.557 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-10T08:18:03.773 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T08:18:03.773 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T08:18:03.773 DEBUG:teuthology.orchestra.run.smithi089:> uname -r 2024-09-10T08:18:03.791 INFO:teuthology.orchestra.run.smithi089.stdout:5.14.0-503.el9.x86_64 2024-09-10T08:18:03.791 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T08:18:03.791 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T08:18:03.792 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum upgrade -y linux-firmware 2024-09-10T08:18:07.688 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.2 MB 00:02 2024-09-10T08:18:11.173 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - AppStream 8.8 MB/s | 20 MB 00:02 2024-09-10T08:18:16.881 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.5 MB 00:01 2024-09-10T08:18:18.366 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - Extras packages 86 kB/s | 19 kB 00:00 2024-09-10T08:18:19.187 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-10T08:18:24.801 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-10T08:18:26.915 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-09-10T08:18:26.916 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2024-09-10T08:18:26.916 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-09-10T08:18:27.008 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T08:18:27.008 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T08:18:27.165 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-10T08:18:27.165 DEBUG:teuthology.orchestra.run.smithi089:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-10T08:18:27.168 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi089.front.sepia.ceph.com to complete... 2024-09-10T08:18:33.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10 08:18:33 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-10T08:18:33.462 INFO:teuthology.orchestra.run.smithi089.stderr:FINISHED --2024-09-10 08:18:33-- 2024-09-10T08:18:33.462 INFO:teuthology.orchestra.run.smithi089.stderr:Total wall clock time: 6.3s 2024-09-10T08:18:33.462 INFO:teuthology.orchestra.run.smithi089.stderr:Downloaded: 1 files, 343M in 6.0s (57.2 MB/s) 2024-09-10T08:18:33.465 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T08:18:33.465 DEBUG:teuthology.orchestra.run.smithi089:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-10T08:18:35.569 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying... ######################################## 2024-09-10T08:18:35.788 INFO:teuthology.orchestra.run.smithi089.stdout:Preparing... ######################################## 2024-09-10T08:18:35.903 INFO:teuthology.orchestra.run.smithi089.stdout:Updating / installing... 2024-09-10T08:18:43.845 INFO:teuthology.orchestra.run.smithi089.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-10T08:19:24.516 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi089.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-10T08:19:24.517 DEBUG:teuthology.orchestra.run.smithi089:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-10T08:19:24.603 INFO:teuthology.orchestra.run.smithi089.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T08:19:24.603 INFO:teuthology.orchestra.run.smithi089.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T08:19:24.603 INFO:teuthology.orchestra.run.smithi089.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T08:19:24.603 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-10T08:19:24.603 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-10T08:19:24.603 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-10T08:19:24.603 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-10T08:19:24.603 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-10T08:19:24.603 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-10T08:19:24.603 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-10T08:19:24.603 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T08:19:24.604 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T08:19:24.604 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T08:19:24.604 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T08:19:24.604 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T08:19:24.604 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T08:19:24.604 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T08:19:24.604 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-10T08:19:24.604 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T08:19:24.604 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T08:19:24.604 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T08:19:24.604 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T08:19:24.604 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T08:19:24.605 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T08:19:24.605 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T08:19:24.605 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T08:19:24.605 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T08:19:24.605 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T08:19:24.605 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T08:19:24.605 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T08:19:24.605 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-10T08:19:24.605 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-10T08:19:24.605 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-10T08:19:24.605 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T08:19:24.605 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-10T08:19:24.605 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T08:19:24.606 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T08:19:24.606 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T08:19:24.606 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-10T08:19:24.606 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T08:19:24.606 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T08:19:24.606 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T08:19:24.606 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-10T08:19:24.606 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T08:19:24.606 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T08:19:24.606 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T08:19:24.606 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T08:19:24.606 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T08:19:24.606 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T08:19:24.607 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T08:19:24.607 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T08:19:24.607 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T08:19:24.607 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T08:19:24.607 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T08:19:24.607 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T08:19:24.607 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T08:19:24.607 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T08:19:24.607 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T08:19:24.607 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T08:19:24.607 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T08:19:24.607 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T08:19:24.607 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T08:19:24.607 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T08:19:24.608 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T08:19:24.608 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T08:19:24.608 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T08:19:24.608 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-10T08:19:24.608 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-10T08:19:24.608 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T08:19:24.608 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T08:19:24.608 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T08:19:24.608 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T08:19:24.608 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T08:19:24.608 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T08:19:24.608 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T08:19:24.608 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T08:19:24.609 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T08:19:24.609 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T08:19:24.609 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T08:19:24.609 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T08:19:24.609 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T08:19:24.609 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T08:19:24.609 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T08:19:24.609 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T08:19:24.609 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T08:19:24.609 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T08:19:24.609 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T08:19:24.609 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T08:19:24.609 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T08:19:24.609 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T08:19:24.610 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-10T08:19:24.610 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-10T08:19:24.610 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T08:19:24.610 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T08:19:24.610 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T08:19:24.610 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T08:19:24.610 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T08:19:24.610 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T08:19:24.610 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T08:19:24.610 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-10T08:19:24.610 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T08:19:24.610 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T08:19:24.610 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-10T08:19:24.611 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T08:19:24.611 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T08:19:24.611 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T08:19:24.611 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T08:19:24.611 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T08:19:24.611 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T08:19:24.611 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T08:19:24.611 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T08:19:24.611 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T08:19:24.611 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T08:19:24.611 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T08:19:24.611 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T08:19:24.611 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T08:19:24.611 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T08:19:24.612 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T08:19:24.612 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T08:19:24.612 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T08:19:24.612 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T08:19:24.612 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T08:19:24.612 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T08:19:24.612 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T08:19:24.612 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T08:19:24.612 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T08:19:24.612 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T08:19:24.612 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T08:19:24.612 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T08:19:24.612 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T08:19:24.613 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T08:19:24.613 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T08:19:24.613 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T08:19:24.613 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T08:19:24.613 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T08:19:24.613 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T08:19:24.613 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T08:19:24.613 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T08:19:24.613 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-10T08:19:24.613 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T08:19:24.613 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T08:19:24.613 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T08:19:24.613 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T08:19:24.613 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T08:19:24.614 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T08:19:24.614 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T08:19:24.614 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T08:19:24.614 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T08:19:24.614 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T08:19:24.614 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T08:19:24.614 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T08:19:24.614 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T08:19:24.614 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T08:19:24.614 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T08:19:24.614 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T08:19:24.614 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T08:19:24.614 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-10T08:19:24.614 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T08:19:24.615 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-10T08:19:24.615 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-10T08:19:24.615 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-10T08:19:24.615 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-10T08:19:24.615 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-10T08:19:24.615 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T08:19:24.615 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-10T08:19:24.615 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-10T08:19:24.615 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T08:19:24.615 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-10T08:19:24.615 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-10T08:19:24.615 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T08:19:24.615 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T08:19:24.615 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-10T08:19:24.616 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-10T08:19:24.616 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T08:19:24.616 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T08:19:24.616 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-10T08:19:24.616 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T08:19:24.616 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T08:19:24.616 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T08:19:24.616 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-10T08:19:24.616 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-10T08:19:24.616 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T08:19:24.616 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-10T08:19:24.616 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-10T08:19:24.616 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T08:19:24.616 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-10T08:19:24.617 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T08:19:24.617 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-10T08:19:24.617 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T08:19:24.617 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T08:19:24.617 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T08:19:24.617 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T08:19:24.617 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T08:19:24.617 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-10T08:19:24.617 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T08:19:24.617 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T08:19:24.617 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T08:19:24.617 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T08:19:24.617 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T08:19:24.617 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T08:19:24.618 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T08:19:24.618 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-10T08:19:24.618 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T08:19:24.618 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T08:19:24.618 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T08:19:24.618 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-10T08:19:24.618 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-10T08:19:24.618 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T08:19:24.618 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T08:19:24.618 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T08:19:24.618 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-10T08:19:24.618 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-10T08:19:24.618 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-10T08:19:24.619 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-10T08:19:24.619 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-10T08:19:24.619 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-10T08:19:24.619 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T08:19:24.619 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T08:19:24.619 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-10T08:19:24.619 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-10T08:19:24.619 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T08:19:24.619 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-10T08:19:24.619 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-10T08:19:24.619 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T08:19:24.619 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T08:19:24.619 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-10T08:19:24.619 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-10T08:19:24.620 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-10T08:19:24.620 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T08:19:24.620 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-10T08:19:24.620 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T08:19:24.620 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T08:19:24.620 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T08:19:24.620 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-10T08:19:24.620 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T08:19:24.620 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-10T08:19:24.620 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T08:19:24.620 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-10T08:19:24.620 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-10T08:19:24.620 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-10T08:19:24.620 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-10T08:19:24.621 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T08:19:24.621 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-10T08:19:24.621 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T08:19:24.621 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-10T08:19:24.621 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T08:19:24.621 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T08:19:24.621 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T08:19:24.621 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T08:19:24.621 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T08:19:24.621 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T08:19:24.621 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T08:19:24.621 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T08:19:24.621 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T08:19:24.622 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T08:19:24.622 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T08:19:24.622 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T08:19:24.622 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T08:19:24.622 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-10T08:19:24.622 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-10T08:19:24.622 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T08:19:24.622 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-10T08:19:24.622 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T08:19:24.622 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T08:19:24.622 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-10T08:19:24.622 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T08:19:24.622 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T08:19:24.622 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T08:19:24.623 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T08:19:24.623 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T08:19:24.623 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T08:19:24.623 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T08:19:24.623 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T08:19:24.623 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T08:19:24.623 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T08:19:24.623 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T08:19:24.623 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T08:19:24.623 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T08:19:24.623 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T08:19:24.623 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T08:19:24.623 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T08:19:24.623 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T08:19:24.624 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T08:19:24.624 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T08:19:24.624 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T08:19:24.624 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T08:19:24.624 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T08:19:24.624 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T08:19:24.624 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T08:19:24.624 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T08:19:24.624 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T08:19:24.624 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T08:19:24.624 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T08:19:24.624 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T08:19:24.624 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T08:19:24.624 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T08:19:24.625 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T08:19:24.625 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T08:19:24.625 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T08:19:24.625 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T08:19:24.625 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T08:19:24.625 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T08:19:24.625 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T08:19:24.625 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T08:19:24.625 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T08:19:24.625 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T08:19:24.625 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T08:19:24.625 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T08:19:24.625 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T08:19:24.625 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T08:19:24.626 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T08:19:24.626 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T08:19:24.626 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T08:19:24.626 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T08:19:24.626 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T08:19:24.626 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T08:19:24.626 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T08:19:24.626 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T08:19:24.626 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T08:19:24.626 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T08:19:24.626 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T08:19:24.626 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T08:19:24.626 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T08:19:24.626 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T08:19:24.627 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T08:19:24.627 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T08:19:24.627 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T08:19:24.627 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T08:19:24.627 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T08:19:24.627 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-10T08:19:24.627 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T08:19:24.627 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T08:19:24.627 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T08:19:24.627 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-10T08:19:24.627 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T08:19:24.627 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T08:19:24.627 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T08:19:24.627 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T08:19:24.628 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T08:19:24.628 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T08:19:24.628 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T08:19:24.628 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T08:19:24.628 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T08:19:24.628 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T08:19:24.628 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T08:19:24.628 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T08:19:24.628 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T08:19:24.628 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T08:19:24.628 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T08:19:24.628 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T08:19:24.628 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T08:19:24.628 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T08:19:24.629 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T08:19:24.629 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T08:19:24.629 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T08:19:24.629 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T08:19:24.629 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T08:19:24.629 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T08:19:24.629 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T08:19:24.629 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T08:19:24.629 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T08:19:24.629 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T08:19:24.629 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T08:19:24.629 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T08:19:24.629 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T08:19:24.630 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T08:19:24.630 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T08:19:24.630 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T08:19:24.630 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T08:19:24.630 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T08:19:24.630 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T08:19:24.630 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T08:19:24.630 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T08:19:24.630 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T08:19:24.630 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T08:19:24.630 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T08:19:24.630 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T08:19:24.630 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T08:19:24.631 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T08:19:24.631 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T08:19:24.631 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T08:19:24.631 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T08:19:24.631 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T08:19:24.631 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T08:19:24.631 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T08:19:24.631 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T08:19:24.631 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T08:19:24.631 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T08:19:24.631 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T08:19:24.631 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T08:19:24.631 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T08:19:24.631 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T08:19:24.632 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T08:19:24.632 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T08:19:24.632 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T08:19:24.632 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T08:19:24.632 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T08:19:24.632 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T08:19:24.632 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T08:19:24.632 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T08:19:24.632 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T08:19:24.632 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T08:19:24.632 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T08:19:24.632 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T08:19:24.632 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T08:19:24.632 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T08:19:24.633 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T08:19:24.633 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T08:19:24.633 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T08:19:24.633 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T08:19:24.633 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T08:19:24.633 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T08:19:24.633 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T08:19:24.633 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T08:19:24.633 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T08:19:24.633 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T08:19:24.633 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T08:19:24.633 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T08:19:24.633 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T08:19:24.633 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T08:19:24.634 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T08:19:24.634 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T08:19:24.634 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-10T08:19:24.634 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T08:19:24.634 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T08:19:24.634 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T08:19:24.634 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T08:19:24.634 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T08:19:24.634 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T08:19:24.634 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T08:19:24.634 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T08:19:24.634 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T08:19:24.634 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T08:19:24.634 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T08:19:24.635 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T08:19:24.635 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T08:19:24.635 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T08:19:24.635 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T08:19:24.635 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T08:19:24.635 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T08:19:24.635 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T08:19:24.635 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T08:19:24.635 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T08:19:24.635 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T08:19:24.635 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T08:19:24.635 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T08:19:24.635 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T08:19:24.635 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T08:19:24.636 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T08:19:24.636 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T08:19:24.636 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T08:19:24.636 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T08:19:24.636 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T08:19:24.636 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T08:19:24.636 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T08:19:24.636 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T08:19:24.636 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T08:19:24.636 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T08:19:24.636 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T08:19:24.636 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T08:19:24.636 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T08:19:24.636 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T08:19:24.636 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T08:19:24.637 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T08:19:24.637 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-10T08:19:24.637 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-10T08:19:24.637 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T08:19:24.637 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T08:19:24.637 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-10T08:19:24.637 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T08:19:24.637 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T08:19:24.637 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T08:19:24.637 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T08:19:24.637 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T08:19:24.637 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T08:19:24.637 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T08:19:24.638 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T08:19:24.638 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T08:19:24.638 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T08:19:24.638 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T08:19:24.638 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T08:19:24.638 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T08:19:24.638 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T08:19:24.638 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T08:19:24.638 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T08:19:24.638 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T08:19:24.638 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T08:19:24.638 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T08:19:24.638 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T08:19:24.638 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T08:19:24.639 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T08:19:24.639 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T08:19:24.639 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-10T08:19:24.639 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T08:19:24.639 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-10T08:19:24.639 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T08:19:24.639 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T08:19:24.639 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-10T08:19:24.639 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-10T08:19:24.639 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T08:19:24.639 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T08:19:24.639 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T08:19:24.639 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T08:19:24.639 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T08:19:24.640 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T08:19:24.640 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T08:19:24.640 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-10T08:19:24.640 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-10T08:19:24.640 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T08:19:24.640 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-10T08:19:24.640 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T08:19:24.640 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-10T08:19:24.640 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T08:19:24.640 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-10T08:19:24.640 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T08:19:24.640 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-10T08:19:24.640 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-10T08:19:24.640 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T08:19:24.641 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-10T08:19:24.641 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T08:19:24.641 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-10T08:19:24.641 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T08:19:24.641 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-10T08:19:24.641 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T08:19:24.641 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-10T08:19:24.641 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T08:19:24.641 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-10T08:19:24.641 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-10T08:19:24.641 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T08:19:24.641 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T08:19:24.641 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T08:19:24.642 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T08:19:24.642 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-10T08:19:24.642 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T08:19:24.642 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-10T08:19:24.642 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T08:19:24.642 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T08:19:24.642 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T08:19:24.642 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T08:19:24.642 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T08:19:24.642 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T08:19:24.642 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T08:19:24.642 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T08:19:24.642 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-10T08:19:24.642 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T08:19:24.643 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-10T08:19:24.643 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T08:19:24.643 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T08:19:24.643 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T08:19:24.643 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T08:19:24.643 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T08:19:24.643 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-10T08:19:24.643 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T08:19:24.643 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T08:19:24.643 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T08:19:24.643 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T08:19:24.643 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-10T08:19:24.643 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T08:19:24.643 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T08:19:24.644 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T08:19:24.644 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T08:19:24.644 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T08:19:24.644 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T08:19:24.644 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T08:19:24.644 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T08:19:24.644 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-10T08:19:24.644 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-10T08:19:24.644 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T08:19:24.644 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T08:19:24.644 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T08:19:24.644 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T08:19:24.644 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T08:19:24.645 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T08:19:24.645 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T08:19:24.645 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T08:19:24.645 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T08:19:24.645 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T08:19:24.645 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T08:19:24.645 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T08:19:24.645 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T08:19:24.645 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T08:19:24.645 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T08:19:24.645 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T08:19:24.645 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-10T08:19:24.645 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T08:19:24.645 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T08:19:24.646 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T08:19:24.646 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T08:19:24.646 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T08:19:24.646 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-10T08:19:24.646 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-10T08:19:24.646 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T08:19:24.646 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T08:19:24.646 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T08:19:24.646 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T08:19:24.646 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T08:19:24.646 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T08:19:24.646 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T08:19:24.646 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-10T08:19:24.646 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T08:19:24.647 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T08:19:24.647 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-10T08:19:24.647 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T08:19:24.647 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T08:19:24.647 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T08:19:24.647 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T08:19:24.647 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-10T08:19:24.647 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-10T08:19:24.647 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T08:19:24.647 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T08:19:24.647 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T08:19:24.647 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T08:19:24.647 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-10T08:19:24.647 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T08:19:24.648 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-10T08:19:24.648 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T08:19:24.648 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T08:19:24.648 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T08:19:24.648 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T08:19:24.648 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T08:19:24.648 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T08:19:24.648 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T08:19:24.648 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-10T08:19:24.648 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T08:19:24.648 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T08:19:24.648 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-10T08:19:24.648 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T08:19:24.648 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T08:19:24.649 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T08:19:24.649 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-10T08:19:24.649 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-10T08:19:24.649 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-10T08:19:24.649 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-10T08:19:24.649 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-10T08:19:24.649 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-10T08:19:24.649 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T08:19:24.649 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T08:19:24.649 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T08:19:24.649 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T08:19:24.649 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-10T08:19:24.649 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T08:19:24.649 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-10T08:19:24.650 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-10T08:19:24.650 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T08:19:24.650 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-10T08:19:24.650 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-10T08:19:24.650 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T08:19:24.650 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T08:19:24.650 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T08:19:24.650 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-10T08:19:24.650 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T08:19:24.650 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T08:19:24.650 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T08:19:24.650 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-10T08:19:24.650 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-10T08:19:24.650 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T08:19:24.650 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T08:19:24.651 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T08:19:24.651 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T08:19:24.651 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-10T08:19:24.651 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T08:19:24.651 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T08:19:24.651 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T08:19:24.651 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T08:19:24.651 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-10T08:19:24.651 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T08:19:24.651 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-10T08:19:24.651 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T08:19:24.651 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-10T08:19:24.651 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T08:19:24.652 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-10T08:19:24.652 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T08:19:24.652 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-10T08:19:24.652 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T08:19:24.652 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T08:19:24.652 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T08:19:24.652 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-10T08:19:24.652 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-10T08:19:24.652 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T08:19:24.652 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-10T08:19:24.652 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T08:19:24.652 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-10T08:19:24.652 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T08:19:24.652 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-10T08:19:24.653 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T08:19:24.653 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T08:19:24.653 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-10T08:19:24.653 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T08:19:24.653 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T08:19:24.653 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-10T08:19:24.653 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T08:19:24.653 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T08:19:24.653 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-10T08:19:24.653 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T08:19:24.653 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T08:19:24.653 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-10T08:19:24.653 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-10T08:19:24.653 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T08:19:24.654 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-10T08:19:24.654 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-10T08:19:24.654 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T08:19:24.654 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-10T08:19:24.654 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T08:19:24.654 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-10T08:19:24.654 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T08:19:24.654 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-10T08:19:24.654 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T08:19:24.654 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-10T08:19:24.654 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T08:19:24.654 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-10T08:19:24.654 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-10T08:19:24.655 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T08:19:24.655 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-10T08:19:24.655 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-10T08:19:24.655 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T08:19:24.655 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T08:19:24.655 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T08:19:24.655 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T08:19:24.655 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T08:19:24.655 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T08:19:24.655 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-10T08:19:24.655 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T08:19:24.655 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T08:19:24.655 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T08:19:24.656 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-10T08:19:24.656 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-10T08:19:24.656 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T08:19:24.656 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T08:19:24.656 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T08:19:24.656 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-10T08:19:24.656 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T08:19:24.656 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-10T08:19:24.656 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T08:19:24.656 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-10T08:19:24.656 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T08:19:24.656 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-10T08:19:24.656 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T08:19:24.656 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-10T08:19:24.657 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T08:19:24.657 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-10T08:19:24.657 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T08:19:24.657 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-10T08:19:24.657 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T08:19:24.657 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-10T08:19:24.657 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T08:19:24.657 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-10T08:19:24.657 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T08:19:24.657 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-10T08:19:24.657 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T08:19:24.657 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-10T08:19:24.657 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T08:19:24.657 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-10T08:19:24.658 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T08:19:24.658 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T08:19:24.658 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-10T08:19:24.658 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T08:19:24.658 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T08:19:24.658 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T08:19:24.658 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-10T08:19:24.658 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-10T08:19:24.658 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T08:19:24.658 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T08:19:24.658 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-10T08:19:24.658 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-10T08:19:24.658 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T08:19:24.658 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-10T08:19:24.659 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T08:19:24.659 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-10T08:19:24.659 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-10T08:19:24.659 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T08:19:24.659 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-10T08:19:24.659 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T08:19:24.659 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T08:19:24.659 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-10T08:19:24.659 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T08:19:24.659 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-10T08:19:24.659 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T08:19:24.659 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-10T08:19:24.659 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T08:19:24.659 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T08:19:24.660 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-10T08:19:24.660 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-10T08:19:24.660 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T08:19:24.660 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T08:19:24.660 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-10T08:19:24.660 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T08:19:24.660 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-10T08:19:24.660 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T08:19:24.660 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-10T08:19:24.660 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T08:19:24.660 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T08:19:24.660 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T08:19:24.660 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T08:19:24.660 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-10T08:19:24.661 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-10T08:19:24.661 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-10T08:19:24.661 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T08:19:24.661 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-10T08:19:24.661 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T08:19:24.661 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-10T08:19:24.661 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T08:19:24.661 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T08:19:24.661 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-10T08:19:24.661 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T08:19:24.661 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T08:19:24.661 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T08:19:24.661 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T08:19:24.661 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-10T08:19:24.662 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-10T08:19:24.662 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T08:19:24.662 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T08:19:24.662 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-10T08:19:24.662 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T08:19:24.662 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T08:19:24.662 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T08:19:24.662 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T08:19:24.662 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-10T08:19:24.662 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T08:19:24.662 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-10T08:19:24.662 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T08:19:24.662 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-10T08:19:24.662 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T08:19:24.663 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T08:19:24.663 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-10T08:19:24.663 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T08:19:24.663 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T08:19:24.663 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-10T08:19:24.663 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T08:19:24.663 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T08:19:24.663 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-10T08:19:24.663 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T08:19:24.663 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-10T08:19:24.663 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T08:19:24.663 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T08:19:24.663 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-10T08:19:24.664 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T08:19:24.664 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T08:19:24.664 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T08:19:24.664 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-10T08:19:24.664 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T08:19:24.664 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T08:19:24.664 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T08:19:24.664 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T08:19:24.664 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T08:19:24.664 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-10T08:19:24.664 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T08:19:24.664 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T08:19:24.664 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-10T08:19:24.665 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T08:19:24.665 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T08:19:24.665 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-10T08:19:24.665 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T08:19:24.665 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T08:19:24.665 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-10T08:19:24.665 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T08:19:24.665 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T08:19:24.665 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T08:19:24.665 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T08:19:24.665 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T08:19:24.665 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-10T08:19:24.665 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T08:19:24.665 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T08:19:24.666 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T08:19:24.666 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T08:19:24.666 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T08:19:24.666 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T08:19:24.666 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T08:19:24.666 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T08:19:24.666 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T08:19:24.666 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T08:19:24.666 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T08:19:24.666 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T08:19:24.666 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-10T08:19:24.666 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T08:19:24.666 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T08:19:24.666 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T08:19:24.667 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-10T08:19:24.667 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T08:19:24.667 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T08:19:24.667 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T08:19:24.667 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T08:19:24.667 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T08:19:24.667 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-10T08:19:24.667 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-10T08:19:24.667 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T08:19:24.667 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T08:19:24.667 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T08:19:24.667 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-10T08:19:24.667 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T08:19:24.668 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T08:19:24.668 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T08:19:24.668 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T08:19:24.668 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T08:19:24.668 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T08:19:24.668 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-10T08:19:24.668 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T08:19:24.668 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-10T08:19:24.668 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-10T08:19:24.668 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T08:19:24.668 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-10T08:19:24.668 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T08:19:24.668 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-10T08:19:24.668 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T08:19:24.669 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-10T08:19:24.669 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-10T08:19:24.669 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T08:19:24.669 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-10T08:19:24.669 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-10T08:19:24.669 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T08:19:24.669 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T08:19:24.669 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-10T08:19:24.669 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T08:19:24.669 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T08:19:24.669 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T08:19:24.669 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T08:19:24.669 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-10T08:19:24.669 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-10T08:19:24.669 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T08:19:24.670 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-10T08:19:24.670 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-10T08:19:24.670 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T08:19:24.670 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-10T08:19:24.670 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T08:19:24.670 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T08:19:24.670 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T08:19:24.670 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-10T08:19:24.670 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-10T08:19:24.670 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T08:19:24.670 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-10T08:19:24.670 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-10T08:19:24.670 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T08:19:24.670 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T08:19:24.671 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T08:19:24.671 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T08:19:24.671 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T08:19:24.671 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T08:19:24.671 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T08:19:24.671 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T08:19:24.671 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T08:19:24.671 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T08:19:24.671 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T08:19:24.671 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T08:19:24.671 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-10T08:19:24.671 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T08:19:24.671 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T08:19:24.671 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T08:19:24.672 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T08:19:24.672 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T08:19:24.672 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T08:19:24.672 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T08:19:24.672 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T08:19:24.672 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T08:19:24.672 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T08:19:24.672 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T08:19:24.672 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T08:19:24.672 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-10T08:19:24.672 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T08:19:24.672 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T08:19:24.672 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-10T08:19:24.672 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-10T08:19:24.673 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T08:19:24.673 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T08:19:24.673 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T08:19:24.673 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T08:19:24.673 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-10T08:19:24.673 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T08:19:24.673 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T08:19:24.673 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-10T08:19:24.673 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-10T08:19:24.673 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-10T08:19:24.673 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T08:19:24.673 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T08:19:24.673 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-10T08:19:24.674 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T08:19:24.674 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T08:19:24.674 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T08:19:24.674 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T08:19:24.674 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T08:19:24.674 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T08:19:24.674 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T08:19:24.674 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T08:19:24.674 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T08:19:24.674 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T08:19:24.674 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T08:19:24.674 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T08:19:24.674 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T08:19:24.674 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T08:19:24.675 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T08:19:24.675 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T08:19:24.675 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T08:19:24.675 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T08:19:24.675 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T08:19:24.675 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T08:19:24.675 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T08:19:24.675 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T08:19:24.675 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T08:19:24.675 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T08:19:24.675 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T08:19:24.675 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T08:19:24.675 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T08:19:24.675 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T08:19:24.676 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T08:19:24.676 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T08:19:24.676 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T08:19:24.676 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-10T08:19:24.676 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T08:19:24.676 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T08:19:24.676 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T08:19:24.676 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T08:19:24.676 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T08:19:24.676 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-10T08:19:24.676 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T08:19:24.676 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T08:19:24.676 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-10T08:19:24.677 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T08:19:24.677 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T08:19:24.677 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T08:19:24.677 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-10T08:19:24.677 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T08:19:24.677 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T08:19:24.677 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-10T08:19:24.677 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T08:19:24.677 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-10T08:19:24.677 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T08:19:24.677 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-10T08:19:24.677 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T08:19:24.677 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-10T08:19:24.677 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T08:19:24.678 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T08:19:24.678 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-10T08:19:24.678 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T08:19:24.678 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-10T08:19:24.678 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-10T08:19:24.678 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T08:19:24.678 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-10T08:19:24.678 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T08:19:24.678 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T08:19:24.678 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T08:19:24.678 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-10T08:19:24.678 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-10T08:19:24.678 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-10T08:19:24.678 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-10T08:19:24.679 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-10T08:19:24.679 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T08:19:24.679 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-10T08:19:24.679 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T08:19:24.679 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T08:19:24.679 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-10T08:19:24.679 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T08:19:24.679 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T08:19:24.679 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-10T08:19:24.679 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T08:19:24.679 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T08:19:24.679 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T08:19:24.679 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T08:19:24.679 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T08:19:24.680 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T08:19:24.680 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T08:19:24.680 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-10T08:19:24.680 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T08:19:24.680 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T08:19:24.680 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T08:19:24.680 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-10T08:19:24.680 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T08:19:24.680 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T08:19:24.680 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T08:19:24.680 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-10T08:19:24.680 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T08:19:24.680 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-10T08:19:24.681 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T08:19:24.681 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-10T08:19:24.681 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T08:19:24.681 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T08:19:24.681 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T08:19:24.681 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T08:19:24.681 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-10T08:19:24.681 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T08:19:24.681 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-10T08:19:24.681 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T08:19:24.681 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T08:19:24.681 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-10T08:19:24.681 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T08:19:24.681 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T08:19:24.682 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T08:19:24.682 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-10T08:19:24.682 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T08:19:24.682 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T08:19:24.682 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-10T08:19:24.682 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T08:19:24.682 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T08:19:24.682 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T08:19:24.682 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T08:19:24.682 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T08:19:24.682 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T08:19:24.682 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T08:19:24.682 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T08:19:24.682 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T08:19:24.683 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T08:19:24.683 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T08:19:24.683 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-10T08:19:24.683 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T08:19:24.683 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T08:19:24.683 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T08:19:24.683 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T08:19:24.683 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-10T08:19:24.683 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T08:19:24.683 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-10T08:19:24.683 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-10T08:19:24.683 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T08:19:24.683 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-10T08:19:24.683 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T08:19:24.684 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T08:19:24.684 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T08:19:24.684 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T08:19:24.684 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T08:19:24.684 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T08:19:24.684 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-10T08:19:24.684 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T08:19:24.684 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-10T08:19:24.684 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-10T08:19:24.684 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-10T08:19:24.684 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T08:19:24.684 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T08:19:24.685 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T08:19:24.685 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T08:19:24.685 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T08:19:24.685 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T08:19:24.685 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T08:19:24.685 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T08:19:24.685 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T08:19:24.685 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T08:19:24.685 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T08:19:24.685 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T08:19:24.685 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T08:19:24.685 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T08:19:24.685 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T08:19:24.686 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T08:19:24.686 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T08:19:24.686 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T08:19:24.686 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-10T08:19:24.686 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-10T08:19:24.686 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T08:19:24.686 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T08:19:24.686 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T08:19:24.686 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T08:19:24.686 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-10T08:19:24.686 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-10T08:19:24.686 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T08:19:24.687 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T08:19:24.687 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-10T08:19:24.687 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T08:19:24.687 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-10T08:19:24.687 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-10T08:19:24.687 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T08:19:24.687 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-10T08:19:24.687 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T08:19:24.687 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-10T08:19:24.687 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T08:19:24.687 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T08:19:24.687 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T08:19:24.687 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T08:19:24.687 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T08:19:24.688 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-10T08:19:24.688 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-10T08:19:24.688 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T08:19:24.688 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T08:19:24.688 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T08:19:24.688 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T08:19:24.688 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-10T08:19:24.688 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T08:19:24.688 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-10T08:19:24.688 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T08:19:24.688 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T08:19:24.688 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-10T08:19:24.688 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T08:19:24.689 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T08:19:24.689 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T08:19:24.689 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T08:19:24.689 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T08:19:24.689 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T08:19:24.689 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T08:19:24.689 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T08:19:24.689 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T08:19:24.689 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T08:19:24.689 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-10T08:19:24.689 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T08:19:24.689 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T08:19:24.689 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T08:19:24.689 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T08:19:24.690 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T08:19:24.690 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T08:19:24.690 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T08:19:24.690 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-10T08:19:24.690 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T08:19:24.690 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-10T08:19:24.690 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T08:19:24.690 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T08:19:24.690 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T08:19:24.690 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T08:19:24.690 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T08:19:24.690 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T08:19:24.690 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T08:19:24.690 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T08:19:24.691 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T08:19:24.691 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T08:19:24.691 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T08:19:24.691 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T08:19:24.691 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T08:19:24.691 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T08:19:24.691 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T08:19:24.691 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T08:19:24.691 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T08:19:24.691 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T08:19:24.691 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T08:19:24.691 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T08:19:24.691 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T08:19:24.691 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T08:19:24.692 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T08:19:24.692 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T08:19:24.692 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T08:19:24.692 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T08:19:24.692 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T08:19:24.692 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T08:19:24.692 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T08:19:24.692 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T08:19:24.692 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T08:19:24.692 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T08:19:24.692 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T08:19:24.692 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T08:19:24.692 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T08:19:24.692 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T08:19:24.693 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T08:19:24.693 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T08:19:24.693 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T08:19:24.693 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T08:19:24.693 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T08:19:24.693 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T08:19:24.693 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T08:19:24.693 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T08:19:24.693 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T08:19:24.693 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-10T08:19:24.693 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T08:19:24.693 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T08:19:24.693 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T08:19:24.693 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T08:19:24.694 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T08:19:24.694 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T08:19:24.694 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T08:19:24.694 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T08:19:24.694 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T08:19:24.694 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T08:19:24.694 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T08:19:24.694 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T08:19:24.694 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T08:19:24.694 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T08:19:24.694 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T08:19:24.694 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-10T08:19:24.694 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T08:19:24.695 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T08:19:24.695 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T08:19:24.695 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-10T08:19:24.695 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-10T08:19:24.695 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T08:19:24.695 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T08:19:24.695 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T08:19:24.695 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T08:19:24.695 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-10T08:19:24.695 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T08:19:24.695 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T08:19:24.695 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T08:19:24.695 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T08:19:24.696 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-10T08:19:24.696 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-10T08:19:24.696 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T08:19:24.696 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T08:19:24.696 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T08:19:24.696 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-10T08:19:24.696 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-10T08:19:24.696 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-10T08:19:24.696 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-10T08:19:24.696 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-10T08:19:24.696 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-10T08:19:24.696 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-10T08:19:24.697 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-10T08:19:24.697 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-10T08:19:24.697 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-10T08:19:24.697 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T08:19:24.697 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T08:19:24.697 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-10T08:19:24.697 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T08:19:24.697 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T08:19:24.697 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T08:19:24.697 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T08:19:24.697 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T08:19:24.697 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T08:19:24.697 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T08:19:24.697 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T08:19:24.697 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T08:19:24.698 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T08:19:24.698 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T08:19:24.698 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T08:19:24.698 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-10T08:19:24.698 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T08:19:24.698 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T08:19:24.698 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T08:19:24.698 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T08:19:24.698 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T08:19:24.698 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T08:19:24.698 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T08:19:24.698 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T08:19:24.698 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T08:19:24.698 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T08:19:24.699 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T08:19:24.699 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T08:19:24.699 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T08:19:24.699 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T08:19:24.699 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T08:19:24.699 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T08:19:24.699 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T08:19:24.699 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T08:19:24.699 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T08:19:24.699 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T08:19:24.699 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T08:19:24.699 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T08:19:24.699 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T08:19:24.700 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T08:19:24.700 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T08:19:24.700 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T08:19:24.700 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-10T08:19:24.700 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-10T08:19:24.700 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T08:19:24.700 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T08:19:24.700 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T08:19:24.700 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-10T08:19:24.700 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T08:19:24.700 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T08:19:24.700 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-10T08:19:24.700 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T08:19:24.700 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-10T08:19:24.701 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-10T08:19:24.701 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T08:19:24.701 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-10T08:19:24.701 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T08:19:24.701 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-10T08:19:24.701 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T08:19:24.701 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T08:19:24.701 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-10T08:19:24.701 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T08:19:24.701 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-10T08:19:24.701 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T08:19:24.701 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-10T08:19:24.701 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T08:19:24.701 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-10T08:19:24.702 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T08:19:24.702 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-10T08:19:24.702 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T08:19:24.702 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-10T08:19:24.702 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T08:19:24.702 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-10T08:19:24.702 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-10T08:19:24.702 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-10T08:19:24.702 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T08:19:24.702 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-10T08:19:24.703 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-10T08:19:24.703 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T08:19:24.703 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-10T08:19:24.703 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-10T08:19:24.703 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T08:19:24.703 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-10T08:19:24.703 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-10T08:19:24.703 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T08:19:24.703 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-10T08:19:24.703 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T08:19:24.703 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-10T08:19:24.703 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T08:19:24.703 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-10T08:19:24.703 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T08:19:24.704 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-10T08:19:24.704 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T08:19:24.704 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-10T08:19:24.704 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T08:19:24.704 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-10T08:19:24.704 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T08:19:24.704 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-10T08:19:24.704 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T08:19:24.704 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-10T08:19:24.704 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-10T08:19:24.704 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T08:19:24.704 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-10T08:19:24.704 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T08:19:24.704 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-10T08:19:24.705 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T08:19:24.705 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T08:19:24.705 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T08:19:24.705 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T08:19:24.705 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-10T08:19:24.705 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T08:19:24.705 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T08:19:24.705 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-10T08:19:24.705 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T08:19:24.705 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-10T08:19:24.705 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T08:19:24.705 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-10T08:19:24.705 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T08:19:24.706 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T08:19:24.706 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T08:19:24.706 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T08:19:24.706 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T08:19:24.706 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T08:19:24.706 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T08:19:24.706 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T08:19:24.706 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T08:19:24.706 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T08:19:24.706 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T08:19:24.706 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T08:19:24.706 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T08:19:24.706 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T08:19:24.706 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T08:19:24.706 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T08:19:24.707 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T08:19:24.707 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T08:19:24.707 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T08:19:24.707 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T08:19:24.707 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T08:19:24.707 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T08:19:24.707 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T08:19:24.707 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T08:19:24.707 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T08:19:24.707 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T08:19:24.707 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T08:19:24.707 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T08:19:24.707 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T08:19:24.707 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T08:19:24.708 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T08:19:24.708 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T08:19:24.708 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T08:19:24.708 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T08:19:24.708 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T08:19:24.708 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T08:19:24.708 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T08:19:24.708 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T08:19:24.708 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T08:19:24.708 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T08:19:24.708 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T08:19:24.708 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T08:19:24.708 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T08:19:24.708 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T08:19:24.709 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T08:19:24.709 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T08:19:24.709 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T08:19:24.709 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T08:19:24.709 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T08:19:24.709 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T08:19:24.709 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-10T08:19:24.709 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-10T08:19:24.709 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T08:19:24.709 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-10T08:19:24.709 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T08:19:24.709 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-10T08:19:24.709 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T08:19:24.710 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T08:19:24.710 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T08:19:24.710 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T08:19:24.710 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T08:19:24.710 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-10T08:19:24.710 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T08:19:24.710 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-10T08:19:24.710 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T08:19:24.710 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-10T08:19:24.710 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T08:19:24.710 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-10T08:19:24.710 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T08:19:24.710 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-10T08:19:24.710 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-10T08:19:24.711 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T08:19:24.711 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-10T08:19:24.711 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T08:19:24.711 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T08:19:24.711 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-10T08:19:24.711 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T08:19:24.711 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-10T08:19:24.711 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T08:19:24.711 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-10T08:19:24.711 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T08:19:24.711 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-10T08:19:24.711 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T08:19:24.711 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-10T08:19:24.711 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T08:19:24.712 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-10T08:19:24.712 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-10T08:19:24.712 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-10T08:19:24.712 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T08:19:24.712 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-10T08:19:24.712 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-10T08:19:24.712 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-10T08:19:24.712 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-10T08:19:24.712 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T08:19:24.712 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T08:19:24.712 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-10T08:19:24.712 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-10T08:19:24.712 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T08:19:24.712 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T08:19:24.713 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T08:19:24.713 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T08:19:24.713 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-10T08:19:24.713 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T08:19:24.713 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T08:19:24.713 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-10T08:19:24.713 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T08:19:24.713 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-10T08:19:24.713 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T08:19:24.713 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-10T08:19:24.713 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T08:19:24.713 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T08:19:24.713 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T08:19:24.713 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T08:19:24.714 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-10T08:19:24.714 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-10T08:19:24.714 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-10T08:19:24.714 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T08:19:24.714 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-10T08:19:24.714 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-10T08:19:24.714 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T08:19:24.714 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T08:19:24.714 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T08:19:24.714 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T08:19:24.714 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T08:19:24.714 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T08:19:24.714 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T08:19:24.714 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T08:19:24.715 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-10T08:19:24.715 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T08:19:24.715 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T08:19:24.715 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T08:19:24.715 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T08:19:24.715 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T08:19:24.715 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-10T08:19:24.715 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T08:19:24.715 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-10T08:19:24.715 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T08:19:24.715 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T08:19:24.715 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T08:19:24.715 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T08:19:24.715 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-10T08:19:24.716 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T08:19:24.716 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-10T08:19:24.716 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T08:19:24.716 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T08:19:24.716 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-10T08:19:24.716 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T08:19:24.716 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T08:19:24.716 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T08:19:24.716 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T08:19:24.716 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T08:19:24.716 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T08:19:24.716 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T08:19:24.716 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T08:19:24.716 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T08:19:24.717 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T08:19:24.717 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T08:19:24.717 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T08:19:24.717 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T08:19:24.717 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T08:19:24.717 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T08:19:24.717 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T08:19:24.717 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T08:19:24.717 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T08:19:24.717 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T08:19:24.717 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T08:19:24.717 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T08:19:24.717 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-10T08:19:24.717 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T08:19:24.718 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T08:19:24.718 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-10T08:19:24.718 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T08:19:24.718 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-10T08:19:24.718 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-10T08:19:24.718 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T08:19:24.718 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-10T08:19:24.718 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T08:19:24.718 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T08:19:24.718 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T08:19:24.718 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T08:19:24.718 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-10T08:19:24.718 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-10T08:19:24.718 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-10T08:19:24.719 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T08:19:24.719 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T08:19:24.719 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T08:19:24.719 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T08:19:24.719 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T08:19:24.719 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-10T08:19:24.719 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-10T08:19:24.719 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T08:19:24.719 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T08:19:24.719 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T08:19:24.719 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-10T08:19:24.719 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T08:19:24.719 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T08:19:24.719 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T08:19:24.720 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T08:19:24.720 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T08:19:24.720 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T08:19:24.720 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T08:19:24.720 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T08:19:24.720 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T08:19:24.720 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T08:19:24.720 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-10T08:19:24.720 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T08:19:24.720 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T08:19:24.720 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T08:19:24.720 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T08:19:24.720 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T08:19:24.720 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T08:19:24.721 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T08:19:24.721 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T08:19:24.721 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T08:19:24.721 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T08:19:24.721 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T08:19:24.721 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T08:19:24.721 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T08:19:24.721 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T08:19:24.721 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T08:19:24.721 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T08:19:24.721 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T08:19:24.721 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T08:19:24.721 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T08:19:24.722 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T08:19:24.722 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T08:19:24.722 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T08:19:24.722 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T08:19:24.722 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T08:19:24.722 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T08:19:24.722 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T08:19:24.722 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T08:19:24.722 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T08:19:24.722 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T08:19:24.722 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T08:19:24.722 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T08:19:24.722 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T08:19:24.722 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T08:19:24.723 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T08:19:24.723 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T08:19:24.723 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T08:19:24.723 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T08:19:24.723 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T08:19:24.723 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-10T08:19:24.723 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T08:19:24.723 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T08:19:24.723 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T08:19:24.723 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T08:19:24.723 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T08:19:24.723 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T08:19:24.723 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T08:19:24.723 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-10T08:19:24.724 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T08:19:24.724 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T08:19:24.724 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T08:19:24.724 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T08:19:24.724 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T08:19:24.724 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T08:19:24.724 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T08:19:24.724 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T08:19:24.724 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T08:19:24.724 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T08:19:24.724 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T08:19:24.724 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T08:19:24.724 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T08:19:24.725 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T08:19:24.725 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T08:19:24.725 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T08:19:24.725 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T08:19:24.725 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T08:19:24.725 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T08:19:24.725 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T08:19:24.725 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T08:19:24.725 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T08:19:24.725 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T08:19:24.725 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T08:19:24.725 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T08:19:24.725 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T08:19:24.726 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T08:19:24.726 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-10T08:19:24.726 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T08:19:24.726 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-10T08:19:24.726 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T08:19:24.726 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T08:19:24.726 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T08:19:24.726 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T08:19:24.726 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T08:19:24.726 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T08:19:24.726 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T08:19:24.726 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-10T08:19:24.726 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T08:19:24.726 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-10T08:19:24.727 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T08:19:24.727 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-10T08:19:24.727 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T08:19:24.727 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-10T08:19:24.727 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-10T08:19:24.727 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T08:19:24.727 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T08:19:24.727 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T08:19:24.727 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T08:19:24.727 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T08:19:24.727 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T08:19:24.727 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T08:19:24.727 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T08:19:24.727 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T08:19:24.728 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T08:19:24.728 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T08:19:24.728 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T08:19:24.728 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T08:19:24.728 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T08:19:24.728 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T08:19:24.728 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T08:19:24.728 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-10T08:19:24.728 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T08:19:24.728 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T08:19:24.728 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T08:19:24.728 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T08:19:24.729 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T08:19:24.729 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T08:19:24.729 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T08:19:24.729 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T08:19:24.729 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T08:19:24.729 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T08:19:24.729 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T08:19:24.729 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T08:19:24.729 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T08:19:24.729 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T08:19:24.729 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T08:19:24.729 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T08:19:24.729 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T08:19:24.730 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T08:19:24.730 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T08:19:24.730 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T08:19:24.730 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T08:19:24.730 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T08:19:24.730 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T08:19:24.730 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T08:19:24.730 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T08:19:24.730 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T08:19:24.730 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T08:19:24.730 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T08:19:24.730 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T08:19:24.731 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T08:19:24.731 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T08:19:24.731 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T08:19:24.731 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T08:19:24.731 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T08:19:24.731 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T08:19:24.731 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T08:19:24.731 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T08:19:24.731 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T08:19:24.731 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T08:19:24.731 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T08:19:24.731 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T08:19:24.731 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T08:19:24.732 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T08:19:24.732 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T08:19:24.732 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T08:19:24.732 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T08:19:24.732 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T08:19:24.732 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T08:19:24.732 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T08:19:24.732 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T08:19:24.732 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T08:19:24.732 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T08:19:24.732 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T08:19:24.732 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T08:19:24.732 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T08:19:24.732 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T08:19:24.733 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T08:19:24.733 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T08:19:24.733 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-10T08:19:24.733 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T08:19:24.733 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T08:19:24.733 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T08:19:24.733 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T08:19:24.733 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-10T08:19:24.733 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T08:19:24.733 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T08:19:24.733 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T08:19:24.733 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T08:19:24.733 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T08:19:24.734 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T08:19:24.734 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T08:19:24.734 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T08:19:24.734 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T08:19:24.734 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T08:19:24.734 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T08:19:24.734 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T08:19:24.734 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T08:19:24.734 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T08:19:24.734 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T08:19:24.734 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T08:19:24.734 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T08:19:24.734 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T08:19:24.734 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T08:19:24.735 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T08:19:24.735 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T08:19:24.735 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T08:19:24.735 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T08:19:24.735 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T08:19:24.735 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T08:19:24.735 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T08:19:24.735 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T08:19:24.735 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T08:19:24.735 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T08:19:24.735 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T08:19:24.735 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T08:19:24.735 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T08:19:24.735 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T08:19:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T08:19:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T08:19:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T08:19:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T08:19:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T08:19:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T08:19:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T08:19:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T08:19:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T08:19:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T08:19:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T08:19:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T08:19:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T08:19:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T08:19:24.737 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T08:19:24.737 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T08:19:24.737 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T08:19:24.737 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T08:19:24.737 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T08:19:24.737 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T08:19:24.737 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T08:19:24.737 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T08:19:24.737 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T08:19:24.737 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T08:19:24.737 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T08:19:24.737 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T08:19:24.737 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T08:19:24.737 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T08:19:24.738 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T08:19:24.738 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T08:19:24.738 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T08:19:24.738 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T08:19:24.738 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T08:19:24.738 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T08:19:24.738 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T08:19:24.738 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T08:19:24.738 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T08:19:24.738 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T08:19:24.738 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-10T08:19:24.738 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T08:19:24.739 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-10T08:19:24.739 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T08:19:24.739 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-10T08:19:24.739 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T08:19:24.739 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T08:19:24.739 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T08:19:24.739 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T08:19:24.739 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-10T08:19:24.739 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T08:19:24.739 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-10T08:19:24.739 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T08:19:24.739 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T08:19:24.739 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T08:19:24.739 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-10T08:19:24.740 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T08:19:24.740 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T08:19:24.740 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T08:19:24.740 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T08:19:24.740 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T08:19:24.740 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T08:19:24.740 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T08:19:24.740 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T08:19:24.740 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T08:19:24.740 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T08:19:24.740 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T08:19:24.740 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T08:19:24.740 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T08:19:24.740 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T08:19:24.741 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T08:19:24.741 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T08:19:24.741 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T08:19:24.741 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T08:19:24.741 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T08:19:24.741 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T08:19:24.741 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T08:19:24.741 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T08:19:24.741 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T08:19:24.741 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T08:19:24.741 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T08:19:24.741 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T08:19:24.741 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T08:19:24.741 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T08:19:24.742 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T08:19:24.742 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T08:19:24.742 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T08:19:24.742 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T08:19:24.742 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T08:19:24.742 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T08:19:24.742 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T08:19:24.742 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T08:19:24.742 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T08:19:24.742 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T08:19:24.742 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T08:19:24.742 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T08:19:24.742 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T08:19:24.742 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T08:19:24.743 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T08:19:24.743 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-10T08:19:24.743 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T08:19:24.743 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T08:19:24.743 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-10T08:19:24.743 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-10T08:19:24.743 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T08:19:24.743 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T08:19:24.743 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T08:19:24.743 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-10T08:19:24.743 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T08:19:24.743 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-10T08:19:24.743 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T08:19:24.743 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T08:19:24.744 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-10T08:19:24.744 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T08:19:24.744 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-10T08:19:24.744 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T08:19:24.744 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T08:19:24.744 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T08:19:24.744 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T08:19:24.744 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T08:19:24.744 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-10T08:19:24.744 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T08:19:24.744 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-10T08:19:24.744 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-10T08:19:24.744 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-10T08:19:24.744 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-10T08:19:24.745 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-10T08:19:24.745 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-10T08:19:24.745 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T08:19:24.745 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T08:19:24.745 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T08:19:24.745 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T08:19:24.745 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T08:19:24.745 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T08:19:24.745 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T08:19:24.745 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T08:19:24.745 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T08:19:24.745 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T08:19:24.745 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T08:19:24.745 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T08:19:24.746 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T08:19:24.746 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-10T08:19:24.746 INFO:teuthology.orchestra.run.smithi089.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T08:19:24.750 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T08:19:24.750 DEBUG:teuthology.orchestra.run.smithi089:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-10T08:19:24.773 INFO:teuthology.orchestra.run.smithi089.stdout:postinstall scriptlet (using /bin/sh): 2024-09-10T08:19:24.773 INFO:teuthology.orchestra.run.smithi089.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T08:19:24.773 INFO:teuthology.orchestra.run.smithi089.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T08:19:24.773 INFO:teuthology.orchestra.run.smithi089.stdout:fi 2024-09-10T08:19:24.773 INFO:teuthology.orchestra.run.smithi089.stdout:for file in vmlinuz System.map config; do 2024-09-10T08:19:24.774 INFO:teuthology.orchestra.run.smithi089.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-10T08:19:24.774 INFO:teuthology.orchestra.run.smithi089.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T08:19:24.774 INFO:teuthology.orchestra.run.smithi089.stdout: fi 2024-09-10T08:19:24.774 INFO:teuthology.orchestra.run.smithi089.stdout:done 2024-09-10T08:19:24.774 INFO:teuthology.orchestra.run.smithi089.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-10T08:19:24.774 INFO:teuthology.orchestra.run.smithi089.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-10T08:19:24.774 INFO:teuthology.orchestra.run.smithi089.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T08:19:24.774 INFO:teuthology.orchestra.run.smithi089.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T08:19:24.774 INFO:teuthology.orchestra.run.smithi089.stdout:fi 2024-09-10T08:19:24.774 INFO:teuthology.orchestra.run.smithi089.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-10T08:19:24.774 INFO:teuthology.orchestra.run.smithi089.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T08:19:24.774 INFO:teuthology.orchestra.run.smithi089.stdout:fi 2024-09-10T08:19:24.774 INFO:teuthology.orchestra.run.smithi089.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-10T08:19:24.775 INFO:teuthology.orchestra.run.smithi089.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T08:19:24.775 INFO:teuthology.orchestra.run.smithi089.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T08:19:24.775 INFO:teuthology.orchestra.run.smithi089.stdout:fi 2024-09-10T08:19:24.776 DEBUG:teuthology.orchestra.run.smithi089:> sudo rpm -qi grub2-tools 2024-09-10T08:19:24.858 INFO:teuthology.orchestra.run.smithi089.stdout:Name : grub2-tools 2024-09-10T08:19:24.859 INFO:teuthology.orchestra.run.smithi089.stdout:Epoch : 1 2024-09-10T08:19:24.859 INFO:teuthology.orchestra.run.smithi089.stdout:Version : 2.06 2024-09-10T08:19:24.859 INFO:teuthology.orchestra.run.smithi089.stdout:Release : 82.el9 2024-09-10T08:19:24.859 INFO:teuthology.orchestra.run.smithi089.stdout:Architecture: x86_64 2024-09-10T08:19:24.859 INFO:teuthology.orchestra.run.smithi089.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T08:19:24.859 INFO:teuthology.orchestra.run.smithi089.stdout:Group : Unspecified 2024-09-10T08:19:24.859 INFO:teuthology.orchestra.run.smithi089.stdout:Size : 8274670 2024-09-10T08:19:24.859 INFO:teuthology.orchestra.run.smithi089.stdout:License : GPLv3+ 2024-09-10T08:19:24.859 INFO:teuthology.orchestra.run.smithi089.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T08:19:24.860 INFO:teuthology.orchestra.run.smithi089.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T08:19:24.860 INFO:teuthology.orchestra.run.smithi089.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T08:19:24.860 INFO:teuthology.orchestra.run.smithi089.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T08:19:24.860 INFO:teuthology.orchestra.run.smithi089.stdout:Packager : builder@centos.org 2024-09-10T08:19:24.860 INFO:teuthology.orchestra.run.smithi089.stdout:Vendor : CentOS 2024-09-10T08:19:24.860 INFO:teuthology.orchestra.run.smithi089.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T08:19:24.860 INFO:teuthology.orchestra.run.smithi089.stdout:Summary : Support tools for GRUB. 2024-09-10T08:19:24.860 INFO:teuthology.orchestra.run.smithi089.stdout:Description : 2024-09-10T08:19:24.860 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:19:24.860 INFO:teuthology.orchestra.run.smithi089.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T08:19:24.861 INFO:teuthology.orchestra.run.smithi089.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T08:19:24.861 INFO:teuthology.orchestra.run.smithi089.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T08:19:24.861 INFO:teuthology.orchestra.run.smithi089.stdout:hardware devices. 2024-09-10T08:19:24.861 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:19:24.861 INFO:teuthology.orchestra.run.smithi089.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T08:19:24.867 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T08:19:24.868 INFO:teuthology.task.kernel:Updating grub on smithi089 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T08:19:24.868 DEBUG:teuthology.orchestra.run.smithi089:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T08:19:25.415 INFO:teuthology.orchestra.run.smithi089.stderr:Generating grub configuration file ... 2024-09-10T08:19:26.675 INFO:teuthology.orchestra.run.smithi089.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T08:19:26.715 INFO:teuthology.orchestra.run.smithi089.stderr:done 2024-09-10T08:19:26.717 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2024-09-10T08:19:26.733 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.Tw07m90Ww0 2024-09-10T08:19:26.733 DEBUG:teuthology.orchestra.run.smithi089:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Tw07m90Ww0 2024-09-10T08:19:26.799 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.Tw07m90Ww0 2024-09-10T08:19:26.942 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.Tw07m90Ww0 is 6KB 2024-09-10T08:19:26.990 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.Tw07m90Ww0 2024-09-10T08:19:27.005 DEBUG:teuthology.orchestra.run.smithi089:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T08:19:27.080 INFO:teuthology.orchestra.run.smithi089.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T08:19:27.081 INFO:teuthology.orchestra.run.smithi089.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T08:19:27.081 INFO:teuthology.orchestra.run.smithi089.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T08:19:27.081 INFO:teuthology.orchestra.run.smithi089.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-10T08:19:27.082 DEBUG:teuthology.orchestra.run.smithi089:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T08:19:27.258 DEBUG:teuthology.orchestra.run.smithi089:> sudo shutdown -r now 2024-09-10T08:19:57.260 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-10T08:19:57.261 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-10T08:19:57.262 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:20:15.805 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-09-10T08:20:24.807 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-10T08:20:24.808 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:20:27.869 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-09-10T08:20:39.869 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-10T08:20:39.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:21:11.746 DEBUG:teuthology.orchestra.run.smithi089:> true 2024-09-10T08:21:12.242 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-10T08:21:12.242 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T08:21:12.243 DEBUG:teuthology.orchestra.run.smithi089:> uname -r 2024-09-10T08:21:12.278 INFO:teuthology.orchestra.run.smithi089.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T08:21:12.278 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T08:21:12.279 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T08:21:12.279 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T08:21:13.280 DEBUG:teuthology.parallel:result is None 2024-09-10T08:21:13.280 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T08:21:13.289 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T08:21:13.290 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T08:21:13.294 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T08:21:13.297 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T08:21:13.326 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T08:21:13.332 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T08:21:13.348 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T08:21:13.349 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T08:21:13.374 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T08:21:13.377 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T08:21:13.576 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T08:21:13.582 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T08:21:13.582 DEBUG:teuthology.orchestra.run.smithi111:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T08:21:13.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:21:13.600 DEBUG:teuthology.orchestra.run.smithi149:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T08:21:13.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:21:13.621 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T08:21:13.625 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T08:21:13.642 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T08:21:13.698 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T08:21:13.703 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T08:21:13.712 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T08:21:13.722 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T08:21:13.796 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T08:21:13.926 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T08:21:13.929 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T08:21:13.936 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T08:21:13.936 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T08:21:13.940 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T08:21:13.942 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T08:21:13.995 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T08:21:14.004 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T08:21:14.004 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T08:21:14.008 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T08:21:14.011 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T08:21:14.038 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T08:21:14.146 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T08:21:14.237 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T08:21:14.296 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T08:21:14.378 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:21:14.378 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T08:21:14.475 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T08:21:14.518 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T08:21:14.582 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T08:21:14.623 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T08:21:14.671 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-10T08:21:14.671 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T08:21:14.740 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T08:21:14.782 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T08:21:14.839 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T08:21:14.917 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T08:21:14.966 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T08:21:14.966 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T08:21:15.032 DEBUG:teuthology.orchestra.run.smithi089:> sudo service rsyslog restart 2024-09-10T08:21:15.036 DEBUG:teuthology.orchestra.run.smithi111:> sudo service rsyslog restart 2024-09-10T08:21:15.038 DEBUG:teuthology.orchestra.run.smithi149:> sudo service rsyslog restart 2024-09-10T08:21:15.076 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T08:21:15.115 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T08:21:15.124 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T08:21:15.581 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T08:21:15.589 INFO:teuthology.task.internal:Starting timer... 2024-09-10T08:21:15.589 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T08:21:15.606 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T08:21:15.615 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-10T08:21:15.616 DEBUG:teuthology.orchestra.run.smithi089:> sudo service auditd rotate 2024-09-10T08:21:15.709 INFO:teuthology.orchestra.run.smithi089.stdout:Rotating logs: 2024-09-10T08:21:15.712 DEBUG:teuthology.orchestra.run.smithi111:> sudo service auditd rotate 2024-09-10T08:21:15.774 INFO:teuthology.orchestra.run.smithi111.stdout:Rotating logs: 2024-09-10T08:21:15.776 DEBUG:teuthology.orchestra.run.smithi149:> sudo service auditd rotate 2024-09-10T08:21:15.832 INFO:teuthology.orchestra.run.smithi149.stdout:Rotating logs: 2024-09-10T08:21:15.834 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T08:21:15.834 DEBUG:teuthology.orchestra.run.smithi089:> /usr/sbin/getenforce 2024-09-10T08:21:15.875 INFO:teuthology.orchestra.run.smithi089.stdout:Permissive 2024-09-10T08:21:15.875 DEBUG:teuthology.orchestra.run.smithi111:> /usr/sbin/getenforce 2024-09-10T08:21:15.904 INFO:teuthology.orchestra.run.smithi111.stdout:Permissive 2024-09-10T08:21:15.904 DEBUG:teuthology.orchestra.run.smithi149:> /usr/sbin/getenforce 2024-09-10T08:21:15.935 INFO:teuthology.orchestra.run.smithi149.stdout:Permissive 2024-09-10T08:21:15.935 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi089.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi111.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi149.front.sepia.ceph.com': 'permissive'} 2024-09-10T08:21:15.935 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T08:21:15.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:21:15.992 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T08:21:16.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:21:16.020 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T08:21:16.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:21:16.050 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T08:21:16.051 DEBUG:teuthology.orchestra.run.smithi089:> sudo /usr/sbin/setenforce permissive 2024-09-10T08:21:16.117 DEBUG:teuthology.orchestra.run.smithi111:> sudo /usr/sbin/setenforce permissive 2024-09-10T08:21:16.150 DEBUG:teuthology.orchestra.run.smithi149:> sudo /usr/sbin/setenforce permissive 2024-09-10T08:21:16.185 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T08:21:16.198 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T08:21:16.212 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T08:21:16.213 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi089.front.sepia.ceph.com,smithi111.front.sepia.ceph.com,smithi149.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T08:25:47.651 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi089.front.sepia.ceph.com'), Remote(name='ubuntu@smithi111.front.sepia.ceph.com'), Remote(name='ubuntu@smithi149.front.sepia.ceph.com')] 2024-09-10T08:25:47.653 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-10T08:25:47.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:25:47.743 DEBUG:teuthology.orchestra.run.smithi089:> true 2024-09-10T08:25:47.855 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-10T08:25:47.855 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-10T08:25:47.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:25:47.932 DEBUG:teuthology.orchestra.run.smithi111:> true 2024-09-10T08:25:48.012 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-10T08:25:48.013 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-09-10T08:25:48.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:25:48.089 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-09-10T08:25:48.165 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-09-10T08:25:48.166 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T08:25:48.177 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T08:25:48.177 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T08:25:48.178 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T08:25:48.183 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T08:25:48.183 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T08:25:48.185 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T08:25:48.186 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T08:25:48.213 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T08:25:48.228 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T08:25:48.235 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T08:25:48.239 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T08:25:48.251 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T08:25:48.252 INFO:teuthology.orchestra.run.smithi111.stderr:sudo: ntpd: command not found 2024-09-10T08:25:48.263 INFO:teuthology.orchestra.run.smithi111.stdout:506 Cannot talk to daemon 2024-09-10T08:25:48.268 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T08:25:48.277 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T08:25:48.278 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: ntpd: command not found 2024-09-10T08:25:48.289 INFO:teuthology.orchestra.run.smithi149.stdout:506 Cannot talk to daemon 2024-09-10T08:25:48.291 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T08:25:48.303 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T08:25:48.317 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T08:25:48.318 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: ntpd: command not found 2024-09-10T08:25:48.342 INFO:teuthology.orchestra.run.smithi089.stdout:506 Cannot talk to daemon 2024-09-10T08:25:48.346 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-09-10T08:25:48.350 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T08:25:48.350 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-09-10T08:25:48.350 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:25:48.350 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:25:48.350 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:25:48.350 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:25:48.371 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T08:25:48.372 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-09-10T08:25:48.375 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T08:25:48.375 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-09-10T08:25:48.376 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:25:48.376 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:25:48.376 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:25:48.376 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:25:48.401 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T08:25:48.497 INFO:teuthology.orchestra.run.smithi089.stderr:bash: line 1: ntpq: command not found 2024-09-10T08:25:48.503 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T08:25:48.503 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2024-09-10T08:25:48.503 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:25:48.503 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:25:48.503 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:25:48.503 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:25:48.504 INFO:teuthology.run_tasks:Running task install... 2024-09-10T08:25:48.515 DEBUG:teuthology.task.install:project ceph 2024-09-10T08:25:48.516 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}} 2024-09-10T08:25:48.516 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'} 2024-09-10T08:25:48.516 INFO:teuthology.task.install:Using flavor: default 2024-09-10T08:25:48.523 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T08:25:48.523 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T08:25:48.524 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T08:25:48.525 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:25:48.528 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T08:25:48.528 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:25:48.529 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T08:25:48.530 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:25:48.682 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T08:25:48.682 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T08:25:48.690 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T08:25:48.691 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T08:25:48.696 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T08:25:48.697 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T08:25:48.791 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T08:25:48.792 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:25:48.792 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T08:25:48.797 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T08:25:48.798 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T08:25:48.798 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T08:25:48.806 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T08:25:48.806 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-10T08:25:48.806 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T08:25:48.824 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T08:25:48.824 DEBUG:teuthology.orchestra.run.smithi149:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T08:25:48.831 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T08:25:48.831 DEBUG:teuthology.orchestra.run.smithi111:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T08:25:48.848 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T08:25:48.848 DEBUG:teuthology.orchestra.run.smithi089:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T08:25:48.890 DEBUG:teuthology.orchestra.run.smithi149:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T08:25:48.897 DEBUG:teuthology.orchestra.run.smithi111:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T08:25:48.945 DEBUG:teuthology.orchestra.run.smithi089:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T08:25:48.965 DEBUG:teuthology.orchestra.run.smithi149:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T08:25:48.972 DEBUG:teuthology.orchestra.run.smithi111:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T08:25:49.023 DEBUG:teuthology.orchestra.run.smithi089:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T08:25:49.029 INFO:teuthology.orchestra.run.smithi149.stdout:check_obsoletes = 1 2024-09-10T08:25:49.031 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum clean all 2024-09-10T08:25:49.036 INFO:teuthology.orchestra.run.smithi111.stdout:check_obsoletes = 1 2024-09-10T08:25:49.038 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum clean all 2024-09-10T08:25:49.078 INFO:teuthology.orchestra.run.smithi089.stdout:check_obsoletes = 1 2024-09-10T08:25:49.081 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum clean all 2024-09-10T08:25:49.257 INFO:teuthology.orchestra.run.smithi149.stdout:45 files removed 2024-09-10T08:25:49.264 INFO:teuthology.orchestra.run.smithi111.stdout:45 files removed 2024-09-10T08:25:49.285 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T08:25:49.290 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T08:25:49.345 INFO:teuthology.orchestra.run.smithi089.stdout:45 files removed 2024-09-10T08:25:49.378 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T08:25:49.834 INFO:teuthology.orchestra.run.smithi111.stdout:ceph packages for x86_64 310 kB/s | 84 kB 00:00 2024-09-10T08:25:49.837 INFO:teuthology.orchestra.run.smithi149.stdout:ceph packages for x86_64 309 kB/s | 84 kB 00:00 2024-09-10T08:25:50.059 INFO:teuthology.orchestra.run.smithi089.stdout:ceph packages for x86_64 291 kB/s | 84 kB 00:00 2024-09-10T08:25:50.102 INFO:teuthology.orchestra.run.smithi111.stdout:ceph noarch packages 52 kB/s | 12 kB 00:00 2024-09-10T08:25:50.121 INFO:teuthology.orchestra.run.smithi149.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-10T08:25:50.359 INFO:teuthology.orchestra.run.smithi089.stdout:ceph noarch packages 46 kB/s | 12 kB 00:00 2024-09-10T08:25:50.360 INFO:teuthology.orchestra.run.smithi111.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-10T08:25:50.389 INFO:teuthology.orchestra.run.smithi149.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-10T08:25:50.635 INFO:teuthology.orchestra.run.smithi089.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-10T08:25:51.213 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-10T08:25:52.022 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.2 MB 00:01 2024-09-10T08:25:52.177 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.2 MB 00:01 2024-09-10T08:25:53.104 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-10T08:25:53.899 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-10T08:25:53.989 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-10T08:25:59.055 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-09-10T08:25:59.306 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.5 MB 00:01 2024-09-10T08:25:59.574 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-10T08:26:00.706 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 19 kB 00:00 2024-09-10T08:26:00.892 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 19 kB 00:00 2024-09-10T08:26:01.192 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 19 kB 00:00 2024-09-10T08:26:01.794 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-10T08:26:01.985 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-10T08:26:02.145 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-10T08:26:07.150 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-10T08:26:07.635 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-10T08:26:08.349 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-10T08:26:09.427 INFO:teuthology.orchestra.run.smithi111.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T08:26:09.427 INFO:teuthology.orchestra.run.smithi111.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T08:26:09.438 INFO:teuthology.orchestra.run.smithi111.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T08:26:09.439 INFO:teuthology.orchestra.run.smithi111.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T08:26:09.440 INFO:teuthology.orchestra.run.smithi111.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T08:26:09.441 INFO:teuthology.orchestra.run.smithi111.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T08:26:09.442 INFO:teuthology.orchestra.run.smithi111.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T08:26:09.445 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T08:26:09.445 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T08:26:09.507 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-10T08:26:09.519 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================== 2024-09-10T08:26:09.519 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-09-10T08:26:09.519 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================== 2024-09-10T08:26:09.519 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-10T08:26:09.519 INFO:teuthology.orchestra.run.smithi111.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T08:26:09.520 INFO:teuthology.orchestra.run.smithi111.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T08:26:09.520 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T08:26:09.520 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T08:26:09.520 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T08:26:09.520 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T08:26:09.520 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T08:26:09.520 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T08:26:09.520 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T08:26:09.520 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T08:26:09.520 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T08:26:09.521 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T08:26:09.521 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T08:26:09.521 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T08:26:09.521 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T08:26:09.521 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T08:26:09.521 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T08:26:09.521 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T08:26:09.521 INFO:teuthology.orchestra.run.smithi111.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T08:26:09.521 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T08:26:09.521 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T08:26:09.521 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T08:26:09.521 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T08:26:09.522 INFO:teuthology.orchestra.run.smithi111.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T08:26:09.522 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T08:26:09.522 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T08:26:09.522 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T08:26:09.522 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T08:26:09.522 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T08:26:09.522 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T08:26:09.522 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T08:26:09.522 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T08:26:09.522 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T08:26:09.522 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T08:26:09.523 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T08:26:09.523 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T08:26:09.523 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T08:26:09.523 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrading: 2024-09-10T08:26:09.523 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T08:26:09.523 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T08:26:09.523 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T08:26:09.523 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T08:26:09.523 INFO:teuthology.orchestra.run.smithi111.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T08:26:09.523 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T08:26:09.523 INFO:teuthology.orchestra.run.smithi111.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T08:26:09.523 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T08:26:09.524 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-09-10T08:26:09.524 INFO:teuthology.orchestra.run.smithi111.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T08:26:09.524 INFO:teuthology.orchestra.run.smithi111.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T08:26:09.524 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T08:26:09.524 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T08:26:09.524 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T08:26:09.524 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T08:26:09.524 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T08:26:09.524 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T08:26:09.524 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T08:26:09.524 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T08:26:09.525 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T08:26:09.525 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T08:26:09.525 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T08:26:09.525 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T08:26:09.525 INFO:teuthology.orchestra.run.smithi111.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T08:26:09.525 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T08:26:09.525 INFO:teuthology.orchestra.run.smithi111.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T08:26:09.525 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T08:26:09.525 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T08:26:09.525 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T08:26:09.525 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T08:26:09.526 INFO:teuthology.orchestra.run.smithi111.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T08:26:09.526 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T08:26:09.526 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T08:26:09.526 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T08:26:09.526 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T08:26:09.526 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T08:26:09.526 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T08:26:09.526 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T08:26:09.526 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T08:26:09.526 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T08:26:09.527 INFO:teuthology.orchestra.run.smithi111.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T08:26:09.527 INFO:teuthology.orchestra.run.smithi111.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T08:26:09.527 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T08:26:09.527 INFO:teuthology.orchestra.run.smithi111.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T08:26:09.527 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T08:26:09.527 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T08:26:09.527 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T08:26:09.527 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T08:26:09.527 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T08:26:09.527 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T08:26:09.527 INFO:teuthology.orchestra.run.smithi111.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T08:26:09.527 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T08:26:09.527 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T08:26:09.528 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T08:26:09.528 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T08:26:09.528 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T08:26:09.528 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T08:26:09.528 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T08:26:09.528 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T08:26:09.528 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T08:26:09.528 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T08:26:09.528 INFO:teuthology.orchestra.run.smithi111.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T08:26:09.528 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T08:26:09.529 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T08:26:09.529 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T08:26:09.529 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T08:26:09.529 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T08:26:09.529 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T08:26:09.529 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T08:26:09.529 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T08:26:09.529 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T08:26:09.529 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T08:26:09.529 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T08:26:09.529 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T08:26:09.529 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T08:26:09.529 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T08:26:09.530 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T08:26:09.530 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T08:26:09.530 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T08:26:09.530 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T08:26:09.530 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T08:26:09.530 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T08:26:09.530 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T08:26:09.530 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T08:26:09.530 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T08:26:09.530 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T08:26:09.530 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T08:26:09.531 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T08:26:09.531 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T08:26:09.531 INFO:teuthology.orchestra.run.smithi111.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T08:26:09.531 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T08:26:09.531 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T08:26:09.531 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T08:26:09.531 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T08:26:09.531 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T08:26:09.531 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T08:26:09.531 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T08:26:09.532 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T08:26:09.532 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T08:26:09.532 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T08:26:09.532 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T08:26:09.532 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T08:26:09.532 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T08:26:09.532 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T08:26:09.532 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T08:26:09.532 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T08:26:09.532 INFO:teuthology.orchestra.run.smithi111.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T08:26:09.532 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T08:26:09.532 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T08:26:09.533 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T08:26:09.533 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T08:26:09.533 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T08:26:09.533 INFO:teuthology.orchestra.run.smithi111.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T08:26:09.533 INFO:teuthology.orchestra.run.smithi111.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T08:26:09.533 INFO:teuthology.orchestra.run.smithi111.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T08:26:09.533 INFO:teuthology.orchestra.run.smithi111.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T08:26:09.533 INFO:teuthology.orchestra.run.smithi111.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T08:26:09.533 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T08:26:09.533 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T08:26:09.533 INFO:teuthology.orchestra.run.smithi111.stdout:Installing weak dependencies: 2024-09-10T08:26:09.533 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T08:26:09.534 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-10T08:26:09.534 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-10T08:26:09.534 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================== 2024-09-10T08:26:09.534 INFO:teuthology.orchestra.run.smithi111.stdout:Install 148 Packages 2024-09-10T08:26:09.534 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrade 8 Packages 2024-09-10T08:26:09.534 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-10T08:26:09.534 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 226 M 2024-09-10T08:26:09.535 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-10T08:26:09.928 INFO:teuthology.orchestra.run.smithi149.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T08:26:09.929 INFO:teuthology.orchestra.run.smithi149.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T08:26:09.940 INFO:teuthology.orchestra.run.smithi149.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T08:26:09.941 INFO:teuthology.orchestra.run.smithi149.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T08:26:09.942 INFO:teuthology.orchestra.run.smithi149.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T08:26:09.942 INFO:teuthology.orchestra.run.smithi149.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T08:26:09.944 INFO:teuthology.orchestra.run.smithi149.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T08:26:09.946 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T08:26:09.947 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T08:26:10.009 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-10T08:26:10.027 INFO:teuthology.orchestra.run.smithi149.stdout:====================================================================================== 2024-09-10T08:26:10.027 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-09-10T08:26:10.027 INFO:teuthology.orchestra.run.smithi149.stdout:====================================================================================== 2024-09-10T08:26:10.028 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-10T08:26:10.028 INFO:teuthology.orchestra.run.smithi149.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T08:26:10.028 INFO:teuthology.orchestra.run.smithi149.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T08:26:10.028 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T08:26:10.028 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T08:26:10.028 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T08:26:10.028 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T08:26:10.028 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T08:26:10.028 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T08:26:10.028 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T08:26:10.028 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T08:26:10.028 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T08:26:10.029 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T08:26:10.029 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T08:26:10.029 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T08:26:10.029 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T08:26:10.029 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T08:26:10.029 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T08:26:10.029 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T08:26:10.029 INFO:teuthology.orchestra.run.smithi149.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T08:26:10.029 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T08:26:10.029 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T08:26:10.029 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T08:26:10.029 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T08:26:10.029 INFO:teuthology.orchestra.run.smithi149.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T08:26:10.030 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T08:26:10.030 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T08:26:10.030 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T08:26:10.030 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T08:26:10.030 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T08:26:10.030 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T08:26:10.030 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T08:26:10.030 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T08:26:10.030 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T08:26:10.030 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T08:26:10.030 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T08:26:10.030 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T08:26:10.031 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T08:26:10.031 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrading: 2024-09-10T08:26:10.031 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T08:26:10.031 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T08:26:10.031 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T08:26:10.031 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T08:26:10.031 INFO:teuthology.orchestra.run.smithi149.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T08:26:10.031 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T08:26:10.031 INFO:teuthology.orchestra.run.smithi149.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T08:26:10.031 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T08:26:10.031 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-09-10T08:26:10.031 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T08:26:10.031 INFO:teuthology.orchestra.run.smithi149.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T08:26:10.031 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T08:26:10.032 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T08:26:10.032 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T08:26:10.032 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T08:26:10.032 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T08:26:10.032 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T08:26:10.032 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T08:26:10.032 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T08:26:10.032 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T08:26:10.032 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T08:26:10.032 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T08:26:10.032 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T08:26:10.032 INFO:teuthology.orchestra.run.smithi149.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T08:26:10.032 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T08:26:10.033 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T08:26:10.033 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T08:26:10.033 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T08:26:10.033 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T08:26:10.033 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T08:26:10.033 INFO:teuthology.orchestra.run.smithi149.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T08:26:10.033 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T08:26:10.033 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T08:26:10.033 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T08:26:10.033 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T08:26:10.033 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T08:26:10.033 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T08:26:10.033 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T08:26:10.034 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T08:26:10.034 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T08:26:10.034 INFO:teuthology.orchestra.run.smithi149.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T08:26:10.034 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T08:26:10.034 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T08:26:10.034 INFO:teuthology.orchestra.run.smithi149.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T08:26:10.034 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T08:26:10.034 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T08:26:10.034 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T08:26:10.034 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T08:26:10.034 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T08:26:10.034 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T08:26:10.034 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T08:26:10.035 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T08:26:10.035 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T08:26:10.035 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T08:26:10.035 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T08:26:10.035 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T08:26:10.035 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T08:26:10.035 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T08:26:10.035 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T08:26:10.035 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T08:26:10.035 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T08:26:10.035 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T08:26:10.035 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T08:26:10.036 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T08:26:10.036 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T08:26:10.036 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T08:26:10.036 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T08:26:10.036 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T08:26:10.036 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T08:26:10.036 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T08:26:10.036 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T08:26:10.036 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T08:26:10.036 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T08:26:10.036 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T08:26:10.036 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T08:26:10.036 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T08:26:10.036 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T08:26:10.037 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T08:26:10.037 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T08:26:10.037 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T08:26:10.037 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T08:26:10.037 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T08:26:10.037 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T08:26:10.037 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T08:26:10.037 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T08:26:10.037 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T08:26:10.037 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T08:26:10.037 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T08:26:10.037 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T08:26:10.038 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T08:26:10.038 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T08:26:10.038 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T08:26:10.038 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T08:26:10.038 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T08:26:10.038 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T08:26:10.038 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T08:26:10.038 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T08:26:10.038 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T08:26:10.038 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T08:26:10.038 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T08:26:10.038 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T08:26:10.038 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T08:26:10.039 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T08:26:10.039 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T08:26:10.039 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T08:26:10.039 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T08:26:10.039 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T08:26:10.039 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T08:26:10.039 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T08:26:10.039 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T08:26:10.039 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T08:26:10.039 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T08:26:10.039 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T08:26:10.039 INFO:teuthology.orchestra.run.smithi149.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T08:26:10.039 INFO:teuthology.orchestra.run.smithi149.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T08:26:10.040 INFO:teuthology.orchestra.run.smithi149.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T08:26:10.040 INFO:teuthology.orchestra.run.smithi149.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T08:26:10.040 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T08:26:10.040 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T08:26:10.040 INFO:teuthology.orchestra.run.smithi149.stdout:Installing weak dependencies: 2024-09-10T08:26:10.040 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T08:26:10.040 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:26:10.040 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-10T08:26:10.040 INFO:teuthology.orchestra.run.smithi149.stdout:====================================================================================== 2024-09-10T08:26:10.040 INFO:teuthology.orchestra.run.smithi149.stdout:Install 148 Packages 2024-09-10T08:26:10.040 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrade 8 Packages 2024-09-10T08:26:10.040 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:26:10.041 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 226 M 2024-09-10T08:26:10.041 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-10T08:26:10.093 INFO:teuthology.orchestra.run.smithi111.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 52 kB/s | 6.5 kB 00:00 2024-09-10T08:26:10.335 INFO:teuthology.orchestra.run.smithi111.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 3.6 MB/s | 897 kB 00:00 2024-09-10T08:26:10.418 INFO:teuthology.orchestra.run.smithi111.stdout:(3/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-10T08:26:10.444 INFO:teuthology.orchestra.run.smithi111.stdout:(4/156): ceph-immutable-object-cache-19.1.1-305 1.3 MB/s | 145 kB 00:00 2024-09-10T08:26:10.453 INFO:teuthology.orchestra.run.smithi149.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 35 kB/s | 6.5 kB 00:00 2024-09-10T08:26:10.553 INFO:teuthology.orchestra.run.smithi111.stdout:(5/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 18 MB/s | 2.4 MB 00:00 2024-09-10T08:26:10.611 INFO:teuthology.orchestra.run.smithi111.stdout:(6/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 9.1 MB/s | 1.5 MB 00:00 2024-09-10T08:26:10.845 INFO:teuthology.orchestra.run.smithi111.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 16 MB/s | 4.7 MB 00:00 2024-09-10T08:26:10.897 INFO:teuthology.orchestra.run.smithi089.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T08:26:10.898 INFO:teuthology.orchestra.run.smithi089.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T08:26:10.911 INFO:teuthology.orchestra.run.smithi089.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T08:26:10.912 INFO:teuthology.orchestra.run.smithi089.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T08:26:10.913 INFO:teuthology.orchestra.run.smithi089.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T08:26:10.914 INFO:teuthology.orchestra.run.smithi089.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T08:26:10.916 INFO:teuthology.orchestra.run.smithi089.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T08:26:10.919 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T08:26:10.920 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T08:26:10.985 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-09-10T08:26:10.999 INFO:teuthology.orchestra.run.smithi089.stdout:====================================================================================== 2024-09-10T08:26:10.999 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2024-09-10T08:26:10.999 INFO:teuthology.orchestra.run.smithi089.stdout:====================================================================================== 2024-09-10T08:26:10.999 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-09-10T08:26:10.999 INFO:teuthology.orchestra.run.smithi089.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T08:26:10.999 INFO:teuthology.orchestra.run.smithi089.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T08:26:10.999 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T08:26:11.000 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T08:26:11.000 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T08:26:11.000 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T08:26:11.000 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T08:26:11.000 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T08:26:11.000 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T08:26:11.000 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T08:26:11.000 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T08:26:11.000 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T08:26:11.000 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T08:26:11.000 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T08:26:11.000 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T08:26:11.001 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T08:26:11.001 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T08:26:11.001 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T08:26:11.001 INFO:teuthology.orchestra.run.smithi089.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T08:26:11.001 INFO:teuthology.orchestra.run.smithi089.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T08:26:11.001 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T08:26:11.001 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T08:26:11.001 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T08:26:11.001 INFO:teuthology.orchestra.run.smithi089.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T08:26:11.001 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T08:26:11.001 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T08:26:11.001 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T08:26:11.001 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T08:26:11.002 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T08:26:11.002 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T08:26:11.002 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T08:26:11.002 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T08:26:11.002 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T08:26:11.002 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T08:26:11.002 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T08:26:11.002 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T08:26:11.002 INFO:teuthology.orchestra.run.smithi089.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T08:26:11.002 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrading: 2024-09-10T08:26:11.002 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T08:26:11.002 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T08:26:11.002 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T08:26:11.003 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T08:26:11.003 INFO:teuthology.orchestra.run.smithi089.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T08:26:11.003 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T08:26:11.003 INFO:teuthology.orchestra.run.smithi089.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T08:26:11.003 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T08:26:11.003 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2024-09-10T08:26:11.003 INFO:teuthology.orchestra.run.smithi089.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T08:26:11.003 INFO:teuthology.orchestra.run.smithi089.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T08:26:11.003 INFO:teuthology.orchestra.run.smithi089.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T08:26:11.003 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T08:26:11.003 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T08:26:11.003 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T08:26:11.003 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T08:26:11.003 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T08:26:11.004 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T08:26:11.004 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T08:26:11.004 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T08:26:11.004 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T08:26:11.004 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T08:26:11.004 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T08:26:11.004 INFO:teuthology.orchestra.run.smithi089.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T08:26:11.004 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T08:26:11.004 INFO:teuthology.orchestra.run.smithi089.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T08:26:11.004 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T08:26:11.004 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T08:26:11.004 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T08:26:11.004 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T08:26:11.005 INFO:teuthology.orchestra.run.smithi089.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T08:26:11.005 INFO:teuthology.orchestra.run.smithi089.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T08:26:11.005 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T08:26:11.005 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T08:26:11.005 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T08:26:11.005 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T08:26:11.005 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T08:26:11.005 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T08:26:11.005 INFO:teuthology.orchestra.run.smithi089.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T08:26:11.005 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T08:26:11.005 INFO:teuthology.orchestra.run.smithi089.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T08:26:11.005 INFO:teuthology.orchestra.run.smithi089.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T08:26:11.005 INFO:teuthology.orchestra.run.smithi089.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T08:26:11.006 INFO:teuthology.orchestra.run.smithi089.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T08:26:11.006 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T08:26:11.006 INFO:teuthology.orchestra.run.smithi089.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T08:26:11.006 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T08:26:11.006 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T08:26:11.006 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T08:26:11.006 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T08:26:11.006 INFO:teuthology.orchestra.run.smithi089.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T08:26:11.006 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T08:26:11.006 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T08:26:11.006 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T08:26:11.006 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T08:26:11.006 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T08:26:11.006 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T08:26:11.007 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T08:26:11.007 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T08:26:11.007 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T08:26:11.007 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T08:26:11.007 INFO:teuthology.orchestra.run.smithi089.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T08:26:11.007 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T08:26:11.007 INFO:teuthology.orchestra.run.smithi089.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T08:26:11.007 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T08:26:11.007 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T08:26:11.007 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T08:26:11.007 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T08:26:11.007 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T08:26:11.007 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T08:26:11.008 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T08:26:11.008 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T08:26:11.008 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T08:26:11.008 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T08:26:11.008 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T08:26:11.008 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T08:26:11.008 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T08:26:11.008 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T08:26:11.008 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T08:26:11.008 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T08:26:11.008 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T08:26:11.008 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T08:26:11.008 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T08:26:11.008 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T08:26:11.009 INFO:teuthology.orchestra.run.smithi089.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T08:26:11.009 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T08:26:11.009 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T08:26:11.009 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T08:26:11.009 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T08:26:11.009 INFO:teuthology.orchestra.run.smithi089.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T08:26:11.009 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T08:26:11.009 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T08:26:11.009 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T08:26:11.009 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T08:26:11.009 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T08:26:11.009 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T08:26:11.009 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T08:26:11.010 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T08:26:11.010 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T08:26:11.010 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T08:26:11.010 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T08:26:11.010 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T08:26:11.010 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T08:26:11.010 INFO:teuthology.orchestra.run.smithi089.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T08:26:11.010 INFO:teuthology.orchestra.run.smithi089.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T08:26:11.010 INFO:teuthology.orchestra.run.smithi089.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T08:26:11.010 INFO:teuthology.orchestra.run.smithi089.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T08:26:11.010 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T08:26:11.010 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T08:26:11.010 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T08:26:11.011 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T08:26:11.011 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T08:26:11.011 INFO:teuthology.orchestra.run.smithi089.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T08:26:11.011 INFO:teuthology.orchestra.run.smithi089.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T08:26:11.011 INFO:teuthology.orchestra.run.smithi089.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T08:26:11.011 INFO:teuthology.orchestra.run.smithi089.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T08:26:11.011 INFO:teuthology.orchestra.run.smithi089.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T08:26:11.011 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T08:26:11.011 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T08:26:11.011 INFO:teuthology.orchestra.run.smithi089.stdout:Installing weak dependencies: 2024-09-10T08:26:11.011 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T08:26:11.011 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:26:11.011 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-09-10T08:26:11.012 INFO:teuthology.orchestra.run.smithi089.stdout:====================================================================================== 2024-09-10T08:26:11.012 INFO:teuthology.orchestra.run.smithi089.stdout:Install 148 Packages 2024-09-10T08:26:11.012 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrade 8 Packages 2024-09-10T08:26:11.012 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:26:11.012 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 226 M 2024-09-10T08:26:11.012 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-09-10T08:26:11.237 INFO:teuthology.orchestra.run.smithi149.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 1.1 MB/s | 897 kB 00:00 2024-09-10T08:26:11.353 INFO:teuthology.orchestra.run.smithi149.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.2 MB/s | 145 kB 00:00 2024-09-10T08:26:11.585 INFO:teuthology.orchestra.run.smithi089.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 51 kB/s | 6.5 kB 00:00 2024-09-10T08:26:11.595 INFO:teuthology.orchestra.run.smithi111.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 13 MB/s | 21 MB 00:01 2024-09-10T08:26:11.704 INFO:teuthology.orchestra.run.smithi111.stdout:(9/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9. 12 MB/s | 11 MB 00:00 2024-09-10T08:26:11.720 INFO:teuthology.orchestra.run.smithi111.stdout:(10/156): ceph-selinux-19.1.1-305.g1cf819b7.el9 201 kB/s | 25 kB 00:00 2024-09-10T08:26:11.751 INFO:teuthology.orchestra.run.smithi089.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 5.3 MB/s | 897 kB 00:00 2024-09-10T08:26:11.796 INFO:teuthology.orchestra.run.smithi111.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 3.0 MB/s | 228 kB 00:00 2024-09-10T08:26:11.843 INFO:teuthology.orchestra.run.smithi089.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.5 MB/s | 145 kB 00:00 2024-09-10T08:26:11.920 INFO:teuthology.orchestra.run.smithi111.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 219 kB/s | 27 kB 00:00 2024-09-10T08:26:12.274 INFO:teuthology.orchestra.run.smithi089.stdout:(4/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 5.6 MB/s | 2.4 MB 00:00 2024-09-10T08:26:12.380 INFO:teuthology.orchestra.run.smithi111.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 69 kB/s | 32 kB 00:00 2024-09-10T08:26:12.385 INFO:teuthology.orchestra.run.smithi089.stdout:(5/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 5.8 MB/s | 5.4 MB 00:00 2024-09-10T08:26:12.519 INFO:teuthology.orchestra.run.smithi089.stdout:(6/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 6.2 MB/s | 1.5 MB 00:00 2024-09-10T08:26:12.588 INFO:teuthology.orchestra.run.smithi111.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 3.5 MB/s | 751 kB 00:00 2024-09-10T08:26:12.663 INFO:teuthology.orchestra.run.smithi111.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.1 MB/s | 164 kB 00:00 2024-09-10T08:26:12.695 INFO:teuthology.orchestra.run.smithi149.stdout:(4/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 1.8 MB/s | 2.4 MB 00:01 2024-09-10T08:26:12.812 INFO:teuthology.orchestra.run.smithi149.stdout:(5/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 2.1 MB/s | 5.4 MB 00:02 2024-09-10T08:26:12.855 INFO:teuthology.orchestra.run.smithi111.stdout:(16/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 7.5 MB/s | 17 MB 00:02 2024-09-10T08:26:12.881 INFO:teuthology.orchestra.run.smithi111.stdout:(17/156): librados-devel-19.1.1-305.g1cf819b7.e 584 kB/s | 127 kB 00:00 2024-09-10T08:26:12.894 INFO:teuthology.orchestra.run.smithi089.stdout:(7/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 15 MB/s | 21 MB 00:01 2024-09-10T08:26:12.939 INFO:teuthology.orchestra.run.smithi111.stdout:(18/156): libradosstriper1-19.1.1-305.g1cf819b7 5.9 MB/s | 506 kB 00:00 2024-09-10T08:26:13.006 INFO:teuthology.orchestra.run.smithi111.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 679 kB/s | 45 kB 00:00 2024-09-10T08:26:13.062 INFO:teuthology.orchestra.run.smithi089.stdout:(8/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 7.0 MB/s | 4.7 MB 00:00 2024-09-10T08:26:13.065 INFO:teuthology.orchestra.run.smithi111.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 2.3 MB/s | 136 kB 00:00 2024-09-10T08:26:13.137 INFO:teuthology.orchestra.run.smithi089.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 336 kB/s | 25 kB 00:00 2024-09-10T08:26:13.156 INFO:teuthology.orchestra.run.smithi111.stdout:(21/156): python3-cephfs-19.1.1-305.g1cf819b7.e 1.7 MB/s | 161 kB 00:00 2024-09-10T08:26:13.265 INFO:teuthology.orchestra.run.smithi111.stdout:(22/156): python3-rados-19.1.1-305.g1cf819b7.el 2.9 MB/s | 320 kB 00:00 2024-09-10T08:26:13.369 INFO:teuthology.orchestra.run.smithi111.stdout:(23/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 2.9 MB/s | 301 kB 00:00 2024-09-10T08:26:13.457 INFO:teuthology.orchestra.run.smithi111.stdout:(24/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.1 MB/s | 99 kB 00:00 2024-09-10T08:26:13.516 INFO:teuthology.orchestra.run.smithi111.stdout:(25/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-10T08:26:13.573 INFO:teuthology.orchestra.run.smithi089.stdout:(10/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 16 MB/s | 11 MB 00:00 2024-09-10T08:26:13.654 INFO:teuthology.orchestra.run.smithi089.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 2.7 MB/s | 228 kB 00:00 2024-09-10T08:26:13.695 INFO:teuthology.orchestra.run.smithi149.stdout:(6/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 1.5 MB/s | 1.5 MB 00:00 2024-09-10T08:26:13.738 INFO:teuthology.orchestra.run.smithi089.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 324 kB/s | 27 kB 00:00 2024-09-10T08:26:13.813 INFO:teuthology.orchestra.run.smithi089.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 423 kB/s | 32 kB 00:00 2024-09-10T08:26:13.850 INFO:teuthology.orchestra.run.smithi111.stdout:(26/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 5.5 MB/s | 5.3 MB 00:00 2024-09-10T08:26:13.905 INFO:teuthology.orchestra.run.smithi089.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 8.0 MB/s | 751 kB 00:00 2024-09-10T08:26:13.942 INFO:teuthology.orchestra.run.smithi111.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 1.8 MB/s | 170 kB 00:00 2024-09-10T08:26:13.981 INFO:teuthology.orchestra.run.smithi089.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.1 MB/s | 164 kB 00:00 2024-09-10T08:26:14.050 INFO:teuthology.orchestra.run.smithi111.stdout:(28/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 5.8 MB/s | 3.1 MB 00:00 2024-09-10T08:26:14.076 INFO:teuthology.orchestra.run.smithi111.stdout:(29/156): ceph-grafana-dashboards-19.1.1-305.g1 238 kB/s | 32 kB 00:00 2024-09-10T08:26:14.101 INFO:teuthology.orchestra.run.smithi111.stdout:(30/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.8 MB/s | 146 kB 00:00 2024-09-10T08:26:14.229 INFO:teuthology.orchestra.run.smithi089.stdout:(16/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 9.9 MB/s | 17 MB 00:01 2024-09-10T08:26:14.247 INFO:teuthology.orchestra.run.smithi089.stdout:(17/156): librados-devel-19.1.1-305.g1cf819b7.e 475 kB/s | 127 kB 00:00 2024-09-10T08:26:14.306 INFO:teuthology.orchestra.run.smithi089.stdout:(18/156): libradosstriper1-19.1.1-305.g1cf819b7 6.4 MB/s | 506 kB 00:00 2024-09-10T08:26:14.399 INFO:teuthology.orchestra.run.smithi089.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 492 kB/s | 45 kB 00:00 2024-09-10T08:26:14.499 INFO:teuthology.orchestra.run.smithi089.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 1.3 MB/s | 136 kB 00:00 2024-09-10T08:26:14.513 INFO:teuthology.orchestra.run.smithi149.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 2.8 MB/s | 4.7 MB 00:01 2024-09-10T08:26:14.640 INFO:teuthology.orchestra.run.smithi089.stdout:(21/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 14 MB/s | 5.3 MB 00:00 2024-09-10T08:26:14.643 INFO:teuthology.orchestra.run.smithi111.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 6.3 MB/s | 3.6 MB 00:00 2024-09-10T08:26:14.666 INFO:teuthology.orchestra.run.smithi089.stdout:(22/156): python3-cephfs-19.1.1-305.g1cf819b7.e 968 kB/s | 161 kB 00:00 2024-09-10T08:26:14.701 INFO:teuthology.orchestra.run.smithi111.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 4.2 MB/s | 249 kB 00:00 2024-09-10T08:26:14.708 INFO:teuthology.orchestra.run.smithi089.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 4.7 MB/s | 320 kB 00:00 2024-09-10T08:26:14.741 INFO:teuthology.orchestra.run.smithi089.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 3.9 MB/s | 301 kB 00:00 2024-09-10T08:26:14.768 INFO:teuthology.orchestra.run.smithi111.stdout:(33/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 741 kB/s | 49 kB 00:00 2024-09-10T08:26:14.775 INFO:teuthology.orchestra.run.smithi089.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.4 MB/s | 99 kB 00:00 2024-09-10T08:26:14.809 INFO:teuthology.orchestra.run.smithi089.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-10T08:26:14.827 INFO:teuthology.orchestra.run.smithi111.stdout:(34/156): ceph-prometheus-alerts-19.1.1-305.g1c 280 kB/s | 16 kB 00:00 2024-09-10T08:26:14.892 INFO:teuthology.orchestra.run.smithi089.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 2.0 MB/s | 170 kB 00:00 2024-09-10T08:26:14.927 INFO:teuthology.orchestra.run.smithi111.stdout:(35/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 2.8 MB/s | 285 kB 00:00 2024-09-10T08:26:14.976 INFO:teuthology.orchestra.run.smithi089.stdout:(28/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 15 MB/s | 3.1 MB 00:00 2024-09-10T08:26:15.018 INFO:teuthology.orchestra.run.smithi089.stdout:(29/156): ceph-grafana-dashboards-19.1.1-305.g1 255 kB/s | 32 kB 00:00 2024-09-10T08:26:15.052 INFO:teuthology.orchestra.run.smithi089.stdout:(30/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 1.9 MB/s | 146 kB 00:00 2024-09-10T08:26:15.086 INFO:teuthology.orchestra.run.smithi111.stdout:(36/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 4.7 MB/s | 766 kB 00:00 2024-09-10T08:26:15.236 INFO:teuthology.orchestra.run.smithi111.stdout:(37/156): ceph-mgr-diskprediction-local-19.1.1- 6.5 MB/s | 7.4 MB 00:01 2024-09-10T08:26:15.253 INFO:teuthology.orchestra.run.smithi111.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 109 kB/s | 18 kB 00:00 2024-09-10T08:26:15.394 INFO:teuthology.orchestra.run.smithi089.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 9.5 MB/s | 3.6 MB 00:00 2024-09-10T08:26:15.494 INFO:teuthology.orchestra.run.smithi089.stdout:(32/156): ceph-mgr-diskprediction-local-19.1.1- 17 MB/s | 7.4 MB 00:00 2024-09-10T08:26:15.519 INFO:teuthology.orchestra.run.smithi089.stdout:(33/156): ceph-mgr-modules-core-19.1.1-305.g1cf 1.9 MB/s | 249 kB 00:00 2024-09-10T08:26:15.561 INFO:teuthology.orchestra.run.smithi089.stdout:(34/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 737 kB/s | 49 kB 00:00 2024-09-10T08:26:15.603 INFO:teuthology.orchestra.run.smithi089.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 197 kB/s | 16 kB 00:00 2024-09-10T08:26:15.629 INFO:teuthology.orchestra.run.smithi089.stdout:(36/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 4.2 MB/s | 285 kB 00:00 2024-09-10T08:26:15.678 INFO:teuthology.orchestra.run.smithi111.stdout:(39/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 12 MB/s | 49 MB 00:03 2024-09-10T08:26:15.679 INFO:teuthology.orchestra.run.smithi089.stdout:(37/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 10 MB/s | 766 kB 00:00 2024-09-10T08:26:15.695 INFO:teuthology.orchestra.run.smithi089.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 274 kB/s | 18 kB 00:00 2024-09-10T08:26:15.779 INFO:teuthology.orchestra.run.smithi111.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 147 kB/s | 80 kB 00:00 2024-09-10T08:26:15.796 INFO:teuthology.orchestra.run.smithi089.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 682 kB/s | 80 kB 00:00 2024-09-10T08:26:15.821 INFO:teuthology.orchestra.run.smithi111.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-10T08:26:15.855 INFO:teuthology.orchestra.run.smithi089.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-10T08:26:15.879 INFO:teuthology.orchestra.run.smithi111.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-10T08:26:15.897 INFO:teuthology.orchestra.run.smithi089.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.9 MB/s | 801 kB 00:00 2024-09-10T08:26:15.904 INFO:teuthology.orchestra.run.smithi111.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 801 kB 00:00 2024-09-10T08:26:15.922 INFO:teuthology.orchestra.run.smithi089.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-10T08:26:15.929 INFO:teuthology.orchestra.run.smithi111.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-10T08:26:15.955 INFO:teuthology.orchestra.run.smithi111.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-10T08:26:15.964 INFO:teuthology.orchestra.run.smithi089.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-10T08:26:15.980 INFO:teuthology.orchestra.run.smithi111.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-10T08:26:16.005 INFO:teuthology.orchestra.run.smithi111.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-10T08:26:16.031 INFO:teuthology.orchestra.run.smithi089.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-10T08:26:16.039 INFO:teuthology.orchestra.run.smithi111.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 5.6 MB/s | 1.2 MB 00:00 2024-09-10T08:26:16.073 INFO:teuthology.orchestra.run.smithi111.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 7.3 MB/s | 689 kB 00:00 2024-09-10T08:26:16.098 INFO:teuthology.orchestra.run.smithi111.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-10T08:26:16.115 INFO:teuthology.orchestra.run.smithi089.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-10T08:26:16.115 INFO:teuthology.orchestra.run.smithi111.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 520 kB/s | 22 kB 00:00 2024-09-10T08:26:16.140 INFO:teuthology.orchestra.run.smithi111.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 313 kB 00:00 2024-09-10T08:26:16.165 INFO:teuthology.orchestra.run.smithi111.stdout:(53/156): bison-3.7.4-5.el9.x86_64.rpm 5.8 MB/s | 943 kB 00:00 2024-09-10T08:26:16.182 INFO:teuthology.orchestra.run.smithi111.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-09-10T08:26:16.207 INFO:teuthology.orchestra.run.smithi089.stdout:(46/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-10T08:26:16.216 INFO:teuthology.orchestra.run.smithi111.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 335 kB/s | 11 kB 00:00 2024-09-10T08:26:16.233 INFO:teuthology.orchestra.run.smithi111.stdout:(56/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-10T08:26:16.249 INFO:teuthology.orchestra.run.smithi089.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 3.5 MB/s | 1.2 MB 00:00 2024-09-10T08:26:16.258 INFO:teuthology.orchestra.run.smithi111.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-09-10T08:26:16.283 INFO:teuthology.orchestra.run.smithi111.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-09-10T08:26:16.317 INFO:teuthology.orchestra.run.smithi111.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-10T08:26:16.367 INFO:teuthology.orchestra.run.smithi111.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-09-10T08:26:16.393 INFO:teuthology.orchestra.run.smithi111.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 5.2 MB/s | 581 kB 00:00 2024-09-10T08:26:16.417 INFO:teuthology.orchestra.run.smithi111.stdout:(62/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 243 kB/s | 24 kB 00:00 2024-09-10T08:26:16.443 INFO:teuthology.orchestra.run.smithi111.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-09-10T08:26:16.454 INFO:teuthology.orchestra.run.smithi149.stdout:(8/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9. 5.5 MB/s | 11 MB 00:01 2024-09-10T08:26:16.468 INFO:teuthology.orchestra.run.smithi111.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T08:26:16.493 INFO:teuthology.orchestra.run.smithi111.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-10T08:26:16.527 INFO:teuthology.orchestra.run.smithi111.stdout:(66/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-10T08:26:16.669 INFO:teuthology.orchestra.run.smithi111.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-09-10T08:26:16.671 INFO:teuthology.orchestra.run.smithi149.stdout:(9/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 3.3 MB/s | 21 MB 00:06 2024-09-10T08:26:16.696 INFO:teuthology.orchestra.run.smithi149.stdout:(10/156): ceph-selinux-19.1.1-305.g1cf819b7.el9 104 kB/s | 25 kB 00:00 2024-09-10T08:26:16.711 INFO:teuthology.orchestra.run.smithi111.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-09-10T08:26:16.778 INFO:teuthology.orchestra.run.smithi111.stdout:(69/156): openblas-openmp-0.3.26-2.el9.x86_64.r 16 MB/s | 4.9 MB 00:00 2024-09-10T08:26:16.780 INFO:teuthology.orchestra.run.smithi149.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 2.7 MB/s | 228 kB 00:00 2024-09-10T08:26:16.812 INFO:teuthology.orchestra.run.smithi111.stdout:(70/156): postgresql-contrib-13.16-1.el9.x86_64 6.3 MB/s | 910 kB 00:00 2024-09-10T08:26:16.822 INFO:teuthology.orchestra.run.smithi149.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 648 kB/s | 27 kB 00:00 2024-09-10T08:26:16.837 INFO:teuthology.orchestra.run.smithi111.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-10T08:26:16.872 INFO:teuthology.orchestra.run.smithi149.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 636 kB/s | 32 kB 00:00 2024-09-10T08:26:16.879 INFO:teuthology.orchestra.run.smithi111.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-10T08:26:16.904 INFO:teuthology.orchestra.run.smithi111.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-10T08:26:16.946 INFO:teuthology.orchestra.run.smithi111.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-10T08:26:16.981 INFO:teuthology.orchestra.run.smithi149.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 6.8 MB/s | 751 kB 00:00 2024-09-10T08:26:17.021 INFO:teuthology.orchestra.run.smithi111.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-09-10T08:26:17.031 INFO:teuthology.orchestra.run.smithi149.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 3.2 MB/s | 164 kB 00:00 2024-09-10T08:26:17.046 INFO:teuthology.orchestra.run.smithi111.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-10T08:26:17.072 INFO:teuthology.orchestra.run.smithi111.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-10T08:26:17.081 INFO:teuthology.orchestra.run.smithi149.stdout:(16/156): librados-devel-19.1.1-305.g1cf819b7.e 2.5 MB/s | 127 kB 00:00 2024-09-10T08:26:17.089 INFO:teuthology.orchestra.run.smithi111.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 611 kB/s | 26 kB 00:00 2024-09-10T08:26:17.093 INFO:teuthology.orchestra.run.smithi089.stdout:(48/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 12 MB/s | 49 MB 00:03 2024-09-10T08:26:17.106 INFO:teuthology.orchestra.run.smithi111.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 571 kB/s | 19 kB 00:00 2024-09-10T08:26:17.131 INFO:teuthology.orchestra.run.smithi111.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-09-10T08:26:17.133 INFO:teuthology.orchestra.run.smithi089.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 744 kB/s | 689 kB 00:00 2024-09-10T08:26:17.148 INFO:teuthology.orchestra.run.smithi111.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 831 kB/s | 35 kB 00:00 2024-09-10T08:26:17.166 INFO:teuthology.orchestra.run.smithi089.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-09-10T08:26:17.173 INFO:teuthology.orchestra.run.smithi149.stdout:(17/156): libradosstriper1-19.1.1-305.g1cf819b7 5.4 MB/s | 506 kB 00:00 2024-09-10T08:26:17.223 INFO:teuthology.orchestra.run.smithi111.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-10T08:26:17.242 INFO:teuthology.orchestra.run.smithi089.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 200 kB/s | 22 kB 00:00 2024-09-10T08:26:17.248 INFO:teuthology.orchestra.run.smithi111.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-10T08:26:17.274 INFO:teuthology.orchestra.run.smithi111.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-10T08:26:17.284 INFO:teuthology.orchestra.run.smithi089.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-10T08:26:17.299 INFO:teuthology.orchestra.run.smithi111.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-10T08:26:17.316 INFO:teuthology.orchestra.run.smithi111.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-10T08:26:17.350 INFO:teuthology.orchestra.run.smithi111.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-10T08:26:17.350 INFO:teuthology.orchestra.run.smithi089.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-09-10T08:26:17.391 INFO:teuthology.orchestra.run.smithi111.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-10T08:26:17.417 INFO:teuthology.orchestra.run.smithi111.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-09-10T08:26:17.434 INFO:teuthology.orchestra.run.smithi111.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-10T08:26:17.459 INFO:teuthology.orchestra.run.smithi089.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-09-10T08:26:17.468 INFO:teuthology.orchestra.run.smithi111.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-10T08:26:17.518 INFO:teuthology.orchestra.run.smithi089.stdout:(55/156): boost-program-options-1.75.0-8.el9.x8 252 kB/s | 107 kB 00:00 2024-09-10T08:26:17.551 INFO:teuthology.orchestra.run.smithi111.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-10T08:26:17.568 INFO:teuthology.orchestra.run.smithi111.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-09-10T08:26:17.585 INFO:teuthology.orchestra.run.smithi111.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-09-10T08:26:17.610 INFO:teuthology.orchestra.run.smithi089.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 74 kB/s | 11 kB 00:00 2024-09-10T08:26:17.610 INFO:teuthology.orchestra.run.smithi111.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 5.1 MB/s | 221 kB 00:00 2024-09-10T08:26:17.652 INFO:teuthology.orchestra.run.smithi111.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-10T08:26:17.694 INFO:teuthology.orchestra.run.smithi111.stdout:(97/156): socat-1.7.4.1-6.el9.x86_64.rpm 7.1 MB/s | 304 kB 00:00 2024-09-10T08:26:17.719 INFO:teuthology.orchestra.run.smithi089.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 418 kB/s | 45 kB 00:00 2024-09-10T08:26:17.736 INFO:teuthology.orchestra.run.smithi111.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-10T08:26:17.777 INFO:teuthology.orchestra.run.smithi089.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.1 MB/s | 3.0 MB 00:00 2024-09-10T08:26:17.803 INFO:teuthology.orchestra.run.smithi089.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 564 kB/s | 160 kB 00:00 2024-09-10T08:26:17.815 INFO:teuthology.orchestra.run.smithi149.stdout:(18/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 4.1 MB/s | 17 MB 00:04 2024-09-10T08:26:17.857 INFO:teuthology.orchestra.run.smithi149.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 1.1 MB/s | 45 kB 00:00 2024-09-10T08:26:17.895 INFO:teuthology.orchestra.run.smithi111.stdout:(99/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-09-10T08:26:17.911 INFO:teuthology.orchestra.run.smithi089.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 226 kB/s | 24 kB 00:00 2024-09-10T08:26:17.912 INFO:teuthology.orchestra.run.smithi111.stdout:(100/156): qatlib-service-24.02.0-1.el9.x86_64. 110 kB/s | 36 kB 00:00 2024-09-10T08:26:17.932 INFO:teuthology.orchestra.run.smithi149.stdout:(20/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 7.0 MB/s | 5.3 MB 00:00 2024-09-10T08:26:17.937 INFO:teuthology.orchestra.run.smithi111.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 477 kB/s | 96 kB 00:00 2024-09-10T08:26:17.958 INFO:teuthology.orchestra.run.smithi149.stdout:(21/156): python3-ceph-common-19.1.1-305.g1cf81 1.3 MB/s | 136 kB 00:00 2024-09-10T08:26:17.962 INFO:teuthology.orchestra.run.smithi111.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 951 kB/s | 64 kB 00:00 2024-09-10T08:26:17.983 INFO:teuthology.orchestra.run.smithi149.stdout:(22/156): python3-cephfs-19.1.1-305.g1cf819b7.e 3.2 MB/s | 161 kB 00:00 2024-09-10T08:26:17.986 INFO:teuthology.orchestra.run.smithi089.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.4 MB/s | 662 kB 00:00 2024-09-10T08:26:17.988 INFO:teuthology.orchestra.run.smithi111.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 593 kB/s | 45 kB 00:00 2024-09-10T08:26:18.033 INFO:teuthology.orchestra.run.smithi149.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 4.2 MB/s | 320 kB 00:00 2024-09-10T08:26:18.037 INFO:teuthology.orchestra.run.smithi089.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 377 kB/s | 47 kB 00:00 2024-09-10T08:26:18.038 INFO:teuthology.orchestra.run.smithi111.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-10T08:26:18.063 INFO:teuthology.orchestra.run.smithi111.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-09-10T08:26:18.084 INFO:teuthology.orchestra.run.smithi149.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 2.9 MB/s | 301 kB 00:00 2024-09-10T08:26:18.088 INFO:teuthology.orchestra.run.smithi111.stdout:(106/156): lua-devel-5.4.4-4.el9.x86_64.rpm 148 kB/s | 22 kB 00:00 2024-09-10T08:26:18.095 INFO:teuthology.orchestra.run.smithi089.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-10T08:26:18.109 INFO:teuthology.orchestra.run.smithi149.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.3 MB/s | 99 kB 00:00 2024-09-10T08:26:18.114 INFO:teuthology.orchestra.run.smithi111.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 388 kB/s | 19 kB 00:00 2024-09-10T08:26:18.134 INFO:teuthology.orchestra.run.smithi149.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-10T08:26:18.146 INFO:teuthology.orchestra.run.smithi089.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-10T08:26:18.148 INFO:teuthology.orchestra.run.smithi111.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T08:26:18.173 INFO:teuthology.orchestra.run.smithi111.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-10T08:26:18.184 INFO:teuthology.orchestra.run.smithi149.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 3.3 MB/s | 170 kB 00:00 2024-09-10T08:26:18.198 INFO:teuthology.orchestra.run.smithi111.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-10T08:26:18.204 INFO:teuthology.orchestra.run.smithi089.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 355 kB/s | 38 kB 00:00 2024-09-10T08:26:18.223 INFO:teuthology.orchestra.run.smithi111.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-10T08:26:18.243 INFO:teuthology.orchestra.run.smithi149.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 543 kB/s | 32 kB 00:00 2024-09-10T08:26:18.273 INFO:teuthology.orchestra.run.smithi111.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-10T08:26:18.293 INFO:teuthology.orchestra.run.smithi149.stdout:(29/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.9 MB/s | 146 kB 00:00 2024-09-10T08:26:18.307 INFO:teuthology.orchestra.run.smithi111.stdout:(113/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-09-10T08:26:18.349 INFO:teuthology.orchestra.run.smithi111.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-09-10T08:26:18.375 INFO:teuthology.orchestra.run.smithi111.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.4 MB/s | 32 kB 00:00 2024-09-10T08:26:18.400 INFO:teuthology.orchestra.run.smithi111.stdout:(116/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 472 kB/s | 43 kB 00:00 2024-09-10T08:26:18.421 INFO:teuthology.orchestra.run.smithi089.stdout:(66/156): libtool-2.4.6-46.el9.x86_64.rpm 903 kB/s | 581 kB 00:00 2024-09-10T08:26:18.425 INFO:teuthology.orchestra.run.smithi111.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-10T08:26:18.450 INFO:teuthology.orchestra.run.smithi111.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-10T08:26:18.476 INFO:teuthology.orchestra.run.smithi111.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-10T08:26:18.501 INFO:teuthology.orchestra.run.smithi111.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-10T08:26:18.526 INFO:teuthology.orchestra.run.smithi111.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T08:26:18.551 INFO:teuthology.orchestra.run.smithi111.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-10T08:26:18.577 INFO:teuthology.orchestra.run.smithi111.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-10T08:26:18.577 INFO:teuthology.orchestra.run.smithi149.stdout:(30/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 6.6 MB/s | 3.1 MB 00:00 2024-09-10T08:26:18.602 INFO:teuthology.orchestra.run.smithi111.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 713 kB/s | 18 kB 00:00 2024-09-10T08:26:18.627 INFO:teuthology.orchestra.run.smithi111.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 935 kB/s | 23 kB 00:00 2024-09-10T08:26:18.652 INFO:teuthology.orchestra.run.smithi111.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 782 kB/s | 19 kB 00:00 2024-09-10T08:26:18.678 INFO:teuthology.orchestra.run.smithi111.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 784 kB/s | 20 kB 00:00 2024-09-10T08:26:18.744 INFO:teuthology.orchestra.run.smithi149.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 7.9 MB/s | 3.6 MB 00:00 2024-09-10T08:26:18.745 INFO:teuthology.orchestra.run.smithi111.stdout:(128/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.7 MB/s | 4.4 MB 00:00 2024-09-10T08:26:18.770 INFO:teuthology.orchestra.run.smithi111.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T08:26:18.795 INFO:teuthology.orchestra.run.smithi111.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.0 MB/s | 79 kB 00:00 2024-09-10T08:26:18.811 INFO:teuthology.orchestra.run.smithi149.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 3.7 MB/s | 249 kB 00:00 2024-09-10T08:26:18.813 INFO:teuthology.orchestra.run.smithi089.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 7.4 MB/s | 4.9 MB 00:00 2024-09-10T08:26:18.821 INFO:teuthology.orchestra.run.smithi111.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T08:26:18.861 INFO:teuthology.orchestra.run.smithi149.stdout:(33/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 992 kB/s | 49 kB 00:00 2024-09-10T08:26:18.871 INFO:teuthology.orchestra.run.smithi111.stdout:(132/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.5 MB/s | 2.0 MB 00:00 2024-09-10T08:26:18.903 INFO:teuthology.orchestra.run.smithi149.stdout:(34/156): ceph-prometheus-alerts-19.1.1-305.g1c 396 kB/s | 16 kB 00:00 2024-09-10T08:26:18.904 INFO:teuthology.orchestra.run.smithi111.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 495 kB/s | 16 kB 00:00 2024-09-10T08:26:18.930 INFO:teuthology.orchestra.run.smithi111.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-09-10T08:26:18.930 INFO:teuthology.orchestra.run.smithi089.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 3.1 MB/s | 1.6 MB 00:00 2024-09-10T08:26:18.963 INFO:teuthology.orchestra.run.smithi111.stdout:(135/156): python3-kubernetes-26.1.0-2.el9.noar 3.6 MB/s | 1.0 MB 00:00 2024-09-10T08:26:18.970 INFO:teuthology.orchestra.run.smithi149.stdout:(35/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 4.2 MB/s | 285 kB 00:00 2024-09-10T08:26:18.989 INFO:teuthology.orchestra.run.smithi111.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-09-10T08:26:18.990 INFO:teuthology.orchestra.run.smithi089.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 5.1 MB/s | 910 kB 00:00 2024-09-10T08:26:19.005 INFO:teuthology.orchestra.run.smithi111.stdout:(137/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 894 kB/s | 90 kB 00:00 2024-09-10T08:26:19.031 INFO:teuthology.orchestra.run.smithi111.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 459 kB/s | 31 kB 00:00 2024-09-10T08:26:19.056 INFO:teuthology.orchestra.run.smithi111.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-10T08:26:19.081 INFO:teuthology.orchestra.run.smithi111.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-10T08:26:19.087 INFO:teuthology.orchestra.run.smithi149.stdout:(36/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 6.4 MB/s | 766 kB 00:00 2024-09-10T08:26:19.107 INFO:teuthology.orchestra.run.smithi111.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-10T08:26:19.131 INFO:teuthology.orchestra.run.smithi089.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 686 kB/s | 137 kB 00:00 2024-09-10T08:26:19.132 INFO:teuthology.orchestra.run.smithi111.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-09-10T08:26:19.137 INFO:teuthology.orchestra.run.smithi149.stdout:(37/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 366 kB/s | 18 kB 00:00 2024-09-10T08:26:19.149 INFO:teuthology.orchestra.run.smithi111.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-09-10T08:26:19.174 INFO:teuthology.orchestra.run.smithi111.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-10T08:26:19.191 INFO:teuthology.orchestra.run.smithi111.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 475 kB/s | 20 kB 00:00 2024-09-10T08:26:19.215 INFO:teuthology.orchestra.run.smithi089.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.4 MB/s | 4.4 MB 00:01 2024-09-10T08:26:19.216 INFO:teuthology.orchestra.run.smithi111.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-10T08:26:19.242 INFO:teuthology.orchestra.run.smithi111.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-09-10T08:26:19.288 INFO:teuthology.orchestra.run.smithi149.stdout:(38/156): fuse-2.9.9-16.el9.x86_64.rpm 532 kB/s | 80 kB 00:00 2024-09-10T08:26:19.292 INFO:teuthology.orchestra.run.smithi111.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-10T08:26:19.323 INFO:teuthology.orchestra.run.smithi089.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 809 kB/s | 87 kB 00:00 2024-09-10T08:26:19.442 INFO:teuthology.orchestra.run.smithi111.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 15 MB/s | 2.3 MB 00:00 2024-09-10T08:26:19.509 INFO:teuthology.orchestra.run.smithi111.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-09-10T08:26:19.515 INFO:teuthology.orchestra.run.smithi089.stdout:(73/156): postgresql-server-13.16-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-09-10T08:26:19.563 INFO:teuthology.orchestra.run.smithi149.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 7.5 MB/s | 7.4 MB 00:00 2024-09-10T08:26:19.567 INFO:teuthology.orchestra.run.smithi111.stdout:(151/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 9.7 MB/s | 3.2 MB 00:00 2024-09-10T08:26:19.588 INFO:teuthology.orchestra.run.smithi149.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.6 MB/s | 801 kB 00:00 2024-09-10T08:26:19.593 INFO:teuthology.orchestra.run.smithi111.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 488 kB/s | 41 kB 00:00 2024-09-10T08:26:19.624 INFO:teuthology.orchestra.run.smithi089.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 567 kB/s | 61 kB 00:00 2024-09-10T08:26:19.668 INFO:teuthology.orchestra.run.smithi111.stdout:(153/156): librados2-19.1.1-305.g1cf819b7.el9.x 7.6 MB/s | 3.4 MB 00:00 2024-09-10T08:26:19.689 INFO:teuthology.orchestra.run.smithi149.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-10T08:26:19.693 INFO:teuthology.orchestra.run.smithi111.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 1.9 MB/s | 249 kB 00:00 2024-09-10T08:26:19.735 INFO:teuthology.orchestra.run.smithi111.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 9.5 MB/s | 1.4 MB 00:00 2024-09-10T08:26:19.764 INFO:teuthology.orchestra.run.smithi149.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 955 kB/s | 191 kB 00:00 2024-09-10T08:26:19.814 INFO:teuthology.orchestra.run.smithi149.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-10T08:26:19.835 INFO:teuthology.orchestra.run.smithi111.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-09-10T08:26:19.842 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-10T08:26:19.842 INFO:teuthology.orchestra.run.smithi111.stdout:Total 22 MB/s | 226 MB 00:10 2024-09-10T08:26:19.864 INFO:teuthology.orchestra.run.smithi149.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-09-10T08:26:19.923 INFO:teuthology.orchestra.run.smithi149.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-10T08:26:19.973 INFO:teuthology.orchestra.run.smithi149.stdout:(46/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-10T08:26:19.974 INFO:teuthology.orchestra.run.smithi089.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 699 kB/s | 245 kB 00:00 2024-09-10T08:26:20.132 INFO:teuthology.orchestra.run.smithi149.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 2.8 MB/s | 1.2 MB 00:00 2024-09-10T08:26:20.157 INFO:teuthology.orchestra.run.smithi149.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-09-10T08:26:20.199 INFO:teuthology.orchestra.run.smithi149.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-09-10T08:26:20.241 INFO:teuthology.orchestra.run.smithi149.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 522 kB/s | 22 kB 00:00 2024-09-10T08:26:20.241 INFO:teuthology.orchestra.run.smithi089.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 932 kB/s | 248 kB 00:00 2024-09-10T08:26:20.283 INFO:teuthology.orchestra.run.smithi149.stdout:(51/156): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-09-10T08:26:20.325 INFO:teuthology.orchestra.run.smithi149.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 786 kB/s | 33 kB 00:00 2024-09-10T08:26:20.359 INFO:teuthology.orchestra.run.smithi149.stdout:(53/156): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-09-10T08:26:20.366 INFO:teuthology.orchestra.run.smithi089.stdout:(77/156): protobuf-3.14.0-13.el9.x86_64.rpm 836 kB/s | 1.0 MB 00:01 2024-09-10T08:26:20.392 INFO:teuthology.orchestra.run.smithi089.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 170 kB/s | 26 kB 00:00 2024-09-10T08:26:20.400 INFO:teuthology.orchestra.run.smithi149.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 439 kB/s | 18 kB 00:00 2024-09-10T08:26:20.442 INFO:teuthology.orchestra.run.smithi149.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 269 kB/s | 11 kB 00:00 2024-09-10T08:26:20.500 INFO:teuthology.orchestra.run.smithi089.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 144 kB/s | 19 kB 00:00 2024-09-10T08:26:20.518 INFO:teuthology.orchestra.run.smithi149.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-09-10T08:26:20.560 INFO:teuthology.orchestra.run.smithi149.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-10T08:26:20.634 INFO:teuthology.orchestra.run.smithi089.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 261 kB/s | 35 kB 00:00 2024-09-10T08:26:20.659 INFO:teuthology.orchestra.run.smithi089.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 645 kB/s | 172 kB 00:00 2024-09-10T08:26:20.752 INFO:teuthology.orchestra.run.smithi149.stdout:(58/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-09-10T08:26:20.936 INFO:teuthology.orchestra.run.smithi149.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 3.1 MB/s | 581 kB 00:00 2024-09-10T08:26:20.978 INFO:teuthology.orchestra.run.smithi149.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 591 kB/s | 24 kB 00:00 2024-09-10T08:26:21.009 INFO:teuthology.orchestra.run.smithi089.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 716 kB/s | 250 kB 00:00 2024-09-10T08:26:21.019 INFO:teuthology.orchestra.run.smithi149.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-10T08:26:21.033 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-10T08:26:21.122 INFO:teuthology.orchestra.run.smithi149.stdout:(62/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-10T08:26:21.183 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-10T08:26:21.183 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-10T08:26:21.197 INFO:teuthology.orchestra.run.smithi149.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-09-10T08:26:21.265 INFO:teuthology.orchestra.run.smithi149.stdout:(64/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.2 MB/s | 3.0 MB 00:00 2024-09-10T08:26:21.276 INFO:teuthology.orchestra.run.smithi089.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 833 kB/s | 222 kB 00:00 2024-09-10T08:26:21.290 INFO:teuthology.orchestra.run.smithi149.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 419 kB/s | 38 kB 00:00 2024-09-10T08:26:21.415 INFO:teuthology.orchestra.run.smithi149.stdout:(66/156): openblas-openmp-0.3.26-2.el9.x86_64.r 33 MB/s | 4.9 MB 00:00 2024-09-10T08:26:21.477 INFO:teuthology.orchestra.run.smithi089.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 386 kB/s | 77 kB 00:00 2024-09-10T08:26:21.482 INFO:teuthology.orchestra.run.smithi149.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-09-10T08:26:21.532 INFO:teuthology.orchestra.run.smithi149.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 18 MB/s | 910 kB 00:00 2024-09-10T08:26:21.574 INFO:teuthology.orchestra.run.smithi149.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 3.2 MB/s | 137 kB 00:00 2024-09-10T08:26:21.610 INFO:teuthology.orchestra.run.smithi089.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 317 kB/s | 42 kB 00:00 2024-09-10T08:26:21.741 INFO:teuthology.orchestra.run.smithi149.stdout:(70/156): postgresql-server-13.16-1.el9.x86_64. 35 MB/s | 5.8 MB 00:00 2024-09-10T08:26:21.800 INFO:teuthology.orchestra.run.smithi149.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-09-10T08:26:21.875 INFO:teuthology.orchestra.run.smithi149.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-09-10T08:26:22.044 INFO:teuthology.orchestra.run.smithi089.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 622 kB/s | 269 kB 00:00 2024-09-10T08:26:22.311 INFO:teuthology.orchestra.run.smithi089.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 595 kB/s | 159 kB 00:00 2024-09-10T08:26:22.325 INFO:teuthology.orchestra.run.smithi149.stdout:(73/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 8.6 MB/s | 49 MB 00:05 2024-09-10T08:26:22.418 INFO:teuthology.orchestra.run.smithi149.stdout:(74/156): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-09-10T08:26:22.419 INFO:teuthology.orchestra.run.smithi089.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-09-10T08:26:22.442 INFO:teuthology.orchestra.run.smithi149.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 525 kB/s | 61 kB 00:00 2024-09-10T08:26:22.468 INFO:teuthology.orchestra.run.smithi149.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-10T08:26:22.510 INFO:teuthology.orchestra.run.smithi149.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 616 kB/s | 26 kB 00:00 2024-09-10T08:26:22.528 INFO:teuthology.orchestra.run.smithi089.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 472 kB/s | 51 kB 00:00 2024-09-10T08:26:22.544 INFO:teuthology.orchestra.run.smithi149.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-09-10T08:26:22.569 INFO:teuthology.orchestra.run.smithi149.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-09-10T08:26:22.594 INFO:teuthology.orchestra.run.smithi149.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-10T08:26:22.628 INFO:teuthology.orchestra.run.smithi149.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 591 kB/s | 35 kB 00:00 2024-09-10T08:26:22.645 INFO:teuthology.orchestra.run.smithi089.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 460 kB/s | 54 kB 00:00 2024-09-10T08:26:22.670 INFO:teuthology.orchestra.run.smithi149.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-09-10T08:26:22.712 INFO:teuthology.orchestra.run.smithi149.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 5.2 MB/s | 222 kB 00:00 2024-09-10T08:26:22.754 INFO:teuthology.orchestra.run.smithi149.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-09-10T08:26:22.796 INFO:teuthology.orchestra.run.smithi149.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-10T08:26:22.846 INFO:teuthology.orchestra.run.smithi149.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-10T08:26:22.913 INFO:teuthology.orchestra.run.smithi149.stdout:(87/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-09-10T08:26:22.938 INFO:teuthology.orchestra.run.smithi149.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-09-10T08:26:22.964 INFO:teuthology.orchestra.run.smithi149.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-10T08:26:22.997 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-10T08:26:22.997 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-10T08:26:23.022 INFO:teuthology.orchestra.run.smithi149.stdout:(90/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.6 MB/s | 4.4 MB 00:01 2024-09-10T08:26:23.048 INFO:teuthology.orchestra.run.smithi149.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 469 kB/s | 51 kB 00:00 2024-09-10T08:26:23.073 INFO:teuthology.orchestra.run.smithi149.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 493 kB/s | 54 kB 00:00 2024-09-10T08:26:23.098 INFO:teuthology.orchestra.run.smithi149.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 827 kB/s | 42 kB 00:00 2024-09-10T08:26:23.124 INFO:teuthology.orchestra.run.smithi149.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 877 kB/s | 44 kB 00:00 2024-09-10T08:26:23.165 INFO:teuthology.orchestra.run.smithi149.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 867 kB/s | 36 kB 00:00 2024-09-10T08:26:23.207 INFO:teuthology.orchestra.run.smithi149.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-09-10T08:26:23.249 INFO:teuthology.orchestra.run.smithi149.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 568 kB/s | 47 kB 00:00 2024-09-10T08:26:23.291 INFO:teuthology.orchestra.run.smithi149.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-10T08:26:23.333 INFO:teuthology.orchestra.run.smithi149.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 2.2 MB/s | 96 kB 00:00 2024-09-10T08:26:23.413 INFO:teuthology.orchestra.run.smithi149.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-10T08:26:23.442 INFO:teuthology.orchestra.run.smithi149.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 586 kB/s | 64 kB 00:00 2024-09-10T08:26:23.467 INFO:teuthology.orchestra.run.smithi149.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 832 kB/s | 45 kB 00:00 2024-09-10T08:26:23.626 INFO:teuthology.orchestra.run.smithi149.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-10T08:26:23.677 INFO:teuthology.orchestra.run.smithi149.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-10T08:26:23.702 INFO:teuthology.orchestra.run.smithi149.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-09-10T08:26:23.727 INFO:teuthology.orchestra.run.smithi149.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 786 kB/s | 19 kB 00:00 2024-09-10T08:26:23.827 INFO:teuthology.orchestra.run.smithi149.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-09-10T08:26:23.861 INFO:teuthology.orchestra.run.smithi149.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.4 MB/s | 26 kB 00:00 2024-09-10T08:26:23.878 INFO:teuthology.orchestra.run.smithi149.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.8 MB/s | 49 kB 00:00 2024-09-10T08:26:23.903 INFO:teuthology.orchestra.run.smithi149.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-10T08:26:23.928 INFO:teuthology.orchestra.run.smithi149.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-10T08:26:23.962 INFO:teuthology.orchestra.run.smithi149.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 26 MB/s | 839 kB 00:00 2024-09-10T08:26:24.004 INFO:teuthology.orchestra.run.smithi149.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-10T08:26:24.029 INFO:teuthology.orchestra.run.smithi149.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T08:26:24.055 INFO:teuthology.orchestra.run.smithi149.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-10T08:26:24.080 INFO:teuthology.orchestra.run.smithi149.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 568 kB/s | 14 kB 00:00 2024-09-10T08:26:24.105 INFO:teuthology.orchestra.run.smithi149.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-10T08:26:24.138 INFO:teuthology.orchestra.run.smithi149.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 10 MB/s | 358 kB 00:00 2024-09-10T08:26:24.172 INFO:teuthology.orchestra.run.smithi149.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-10T08:26:24.197 INFO:teuthology.orchestra.run.smithi149.stdout:(120/156): lua-devel-5.4.4-4.el9.x86_64.rpm 29 kB/s | 22 kB 00:00 2024-09-10T08:26:24.231 INFO:teuthology.orchestra.run.smithi149.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-10T08:26:24.256 INFO:teuthology.orchestra.run.smithi149.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 6.1 MB/s | 144 kB 00:00 2024-09-10T08:26:24.282 INFO:teuthology.orchestra.run.smithi149.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-10T08:26:24.307 INFO:teuthology.orchestra.run.smithi149.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 706 kB/s | 18 kB 00:00 2024-09-10T08:26:24.333 INFO:teuthology.orchestra.run.smithi149.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 930 kB/s | 23 kB 00:00 2024-09-10T08:26:24.358 INFO:teuthology.orchestra.run.smithi149.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-09-10T08:26:24.383 INFO:teuthology.orchestra.run.smithi149.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 794 kB/s | 20 kB 00:00 2024-09-10T08:26:24.433 INFO:teuthology.orchestra.run.smithi149.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 20 MB/s | 1.0 MB 00:00 2024-09-10T08:26:24.459 INFO:teuthology.orchestra.run.smithi149.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T08:26:24.484 INFO:teuthology.orchestra.run.smithi149.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T08:26:24.509 INFO:teuthology.orchestra.run.smithi149.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T08:26:24.559 INFO:teuthology.orchestra.run.smithi149.stdout:(132/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.6 MB/s | 2.0 MB 00:00 2024-09-10T08:26:24.585 INFO:teuthology.orchestra.run.smithi149.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-10T08:26:24.610 INFO:teuthology.orchestra.run.smithi149.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-09-10T08:26:24.636 INFO:teuthology.orchestra.run.smithi149.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-10T08:26:24.652 INFO:teuthology.orchestra.run.smithi149.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 673 kB/s | 28 kB 00:00 2024-09-10T08:26:24.669 INFO:teuthology.orchestra.run.smithi149.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 924 kB/s | 31 kB 00:00 2024-09-10T08:26:24.711 INFO:teuthology.orchestra.run.smithi149.stdout:(138/156): protobuf-compiler-3.14.0-13.el9.x86_ 694 kB/s | 863 kB 00:01 2024-09-10T08:26:24.728 INFO:teuthology.orchestra.run.smithi149.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-10T08:26:24.762 INFO:teuthology.orchestra.run.smithi149.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-09-10T08:26:24.788 INFO:teuthology.orchestra.run.smithi149.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-10T08:26:24.813 INFO:teuthology.orchestra.run.smithi149.stdout:(142/156): python3-tempora-5.0.0-2.el9.noarch.r 354 kB/s | 36 kB 00:00 2024-09-10T08:26:24.838 INFO:teuthology.orchestra.run.smithi149.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-10T08:26:24.863 INFO:teuthology.orchestra.run.smithi149.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 796 kB/s | 20 kB 00:00 2024-09-10T08:26:24.888 INFO:teuthology.orchestra.run.smithi149.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-10T08:26:24.914 INFO:teuthology.orchestra.run.smithi149.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T08:26:24.939 INFO:teuthology.orchestra.run.smithi149.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-10T08:26:24.989 INFO:teuthology.orchestra.run.smithi149.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-10T08:26:25.365 INFO:teuthology.orchestra.run.smithi149.stdout:(149/156): librados2-19.1.1-305.g1cf819b7.el9.x 7.6 MB/s | 3.4 MB 00:00 2024-09-10T08:26:25.415 INFO:teuthology.orchestra.run.smithi149.stdout:(150/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 6.6 MB/s | 3.2 MB 00:00 2024-09-10T08:26:25.523 INFO:teuthology.orchestra.run.smithi149.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 377 kB/s | 41 kB 00:00 2024-09-10T08:26:25.565 INFO:teuthology.orchestra.run.smithi149.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 4.0 MB/s | 2.3 MB 00:00 2024-09-10T08:26:25.632 INFO:teuthology.orchestra.run.smithi149.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 2.2 MB/s | 249 kB 00:00 2024-09-10T08:26:25.733 INFO:teuthology.orchestra.run.smithi149.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 4.9 MB/s | 1.8 MB 00:00 2024-09-10T08:26:25.816 INFO:teuthology.orchestra.run.smithi149.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 5.4 MB/s | 1.4 MB 00:00 2024-09-10T08:26:26.058 INFO:teuthology.orchestra.run.smithi149.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 5.6 MB/s | 2.4 MB 00:00 2024-09-10T08:26:26.065 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-10T08:26:26.065 INFO:teuthology.orchestra.run.smithi149.stdout:Total 14 MB/s | 226 MB 00:16 2024-09-10T08:26:26.762 INFO:teuthology.orchestra.run.smithi089.stdout:(91/156): python3-babel-2.9.1-2.el9.noarch.rpm 821 kB/s | 6.0 MB 00:07 2024-09-10T08:26:26.879 INFO:teuthology.orchestra.run.smithi089.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 836 kB/s | 5.1 MB 00:06 2024-09-10T08:26:26.904 INFO:teuthology.orchestra.run.smithi089.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 294 kB/s | 42 kB 00:00 2024-09-10T08:26:26.988 INFO:teuthology.orchestra.run.smithi089.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 407 kB/s | 44 kB 00:00 2024-09-10T08:26:27.097 INFO:teuthology.orchestra.run.smithi089.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 221 kB 00:00 2024-09-10T08:26:27.122 INFO:teuthology.orchestra.run.smithi089.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 268 kB/s | 36 kB 00:00 2024-09-10T08:26:27.205 INFO:teuthology.orchestra.run.smithi089.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-09-10T08:26:27.314 INFO:teuthology.orchestra.run.smithi089.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 534 kB/s | 58 kB 00:00 2024-09-10T08:26:27.324 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-10T08:26:27.389 INFO:teuthology.orchestra.run.smithi089.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 1.1 MB/s | 304 kB 00:00 2024-09-10T08:26:27.423 INFO:teuthology.orchestra.run.smithi089.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 883 kB/s | 96 kB 00:00 2024-09-10T08:26:27.476 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-10T08:26:27.476 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-10T08:26:27.498 INFO:teuthology.orchestra.run.smithi089.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 587 kB/s | 64 kB 00:00 2024-09-10T08:26:27.524 INFO:teuthology.orchestra.run.smithi089.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 446 kB/s | 45 kB 00:00 2024-09-10T08:26:28.074 INFO:teuthology.orchestra.run.smithi089.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 39 kB/s | 22 kB 00:00 2024-09-10T08:26:28.133 INFO:teuthology.orchestra.run.smithi089.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.3 MB/s | 548 kB 00:00 2024-09-10T08:26:28.166 INFO:teuthology.orchestra.run.smithi089.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-10T08:26:28.191 INFO:teuthology.orchestra.run.smithi089.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 781 kB/s | 19 kB 00:00 2024-09-10T08:26:28.309 INFO:teuthology.orchestra.run.smithi089.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-10T08:26:28.334 INFO:teuthology.orchestra.run.smithi089.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T08:26:28.359 INFO:teuthology.orchestra.run.smithi089.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T08:26:28.384 INFO:teuthology.orchestra.run.smithi089.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-10T08:26:28.409 INFO:teuthology.orchestra.run.smithi089.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-10T08:26:28.452 INFO:teuthology.orchestra.run.smithi089.stdout:(112/156): protobuf-compiler-3.14.0-13.el9.x86_ 931 kB/s | 863 kB 00:00 2024-09-10T08:26:28.494 INFO:teuthology.orchestra.run.smithi089.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-09-10T08:26:28.695 INFO:teuthology.orchestra.run.smithi089.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 221 kB/s | 43 kB 00:00 2024-09-10T08:26:28.721 INFO:teuthology.orchestra.run.smithi089.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-10T08:26:28.745 INFO:teuthology.orchestra.run.smithi089.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-10T08:26:28.770 INFO:teuthology.orchestra.run.smithi089.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-09-10T08:26:28.804 INFO:teuthology.orchestra.run.smithi089.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 1.5 MB/s | 548 kB 00:00 2024-09-10T08:26:28.837 INFO:teuthology.orchestra.run.smithi089.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-10T08:26:28.863 INFO:teuthology.orchestra.run.smithi089.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-10T08:26:28.896 INFO:teuthology.orchestra.run.smithi089.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-10T08:26:28.924 INFO:teuthology.orchestra.run.smithi089.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-10T08:26:28.947 INFO:teuthology.orchestra.run.smithi089.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-10T08:26:28.973 INFO:teuthology.orchestra.run.smithi089.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-09-10T08:26:28.998 INFO:teuthology.orchestra.run.smithi089.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 939 kB/s | 23 kB 00:00 2024-09-10T08:26:29.023 INFO:teuthology.orchestra.run.smithi089.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 786 kB/s | 19 kB 00:00 2024-09-10T08:26:29.049 INFO:teuthology.orchestra.run.smithi089.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-09-10T08:26:29.099 INFO:teuthology.orchestra.run.smithi089.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 21 MB/s | 1.0 MB 00:00 2024-09-10T08:26:29.149 INFO:teuthology.orchestra.run.smithi089.stdout:(129/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.1 MB/s | 2.0 MB 00:00 2024-09-10T08:26:29.174 INFO:teuthology.orchestra.run.smithi089.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 616 kB/s | 46 kB 00:00 2024-09-10T08:26:29.199 INFO:teuthology.orchestra.run.smithi089.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-09-10T08:26:29.213 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-10T08:26:29.225 INFO:teuthology.orchestra.run.smithi089.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-10T08:26:29.250 INFO:teuthology.orchestra.run.smithi089.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 663 kB/s | 16 kB 00:00 2024-09-10T08:26:29.275 INFO:teuthology.orchestra.run.smithi089.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-10T08:26:29.276 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-10T08:26:29.276 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-10T08:26:29.301 INFO:teuthology.orchestra.run.smithi089.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-10T08:26:29.326 INFO:teuthology.orchestra.run.smithi089.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-09-10T08:26:29.348 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T08:26:29.351 INFO:teuthology.orchestra.run.smithi089.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-10T08:26:29.385 INFO:teuthology.orchestra.run.smithi089.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2024-09-10T08:26:29.402 INFO:teuthology.orchestra.run.smithi089.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 2.1 MB/s | 36 kB 00:00 2024-09-10T08:26:29.427 INFO:teuthology.orchestra.run.smithi089.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-10T08:26:29.453 INFO:teuthology.orchestra.run.smithi089.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-10T08:26:29.470 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T08:26:29.478 INFO:teuthology.orchestra.run.smithi089.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T08:26:29.503 INFO:teuthology.orchestra.run.smithi089.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T08:26:29.528 INFO:teuthology.orchestra.run.smithi089.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 807 kB/s | 20 kB 00:00 2024-09-10T08:26:29.554 INFO:teuthology.orchestra.run.smithi089.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-10T08:26:29.579 INFO:teuthology.orchestra.run.smithi089.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-10T08:26:29.624 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T08:26:29.646 INFO:teuthology.orchestra.run.smithi089.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-10T08:26:29.665 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T08:26:29.690 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T08:26:29.733 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T08:26:29.855 INFO:teuthology.orchestra.run.smithi089.stdout:(148/156): librados2-19.1.1-305.g1cf819b7.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-10T08:26:29.964 INFO:teuthology.orchestra.run.smithi089.stdout:(149/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 10 MB/s | 3.2 MB 00:00 2024-09-10T08:26:30.072 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T08:26:30.175 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T08:26:30.288 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T08:26:30.317 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T08:26:30.399 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T08:26:30.422 INFO:teuthology.orchestra.run.smithi089.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 4.1 MB/s | 2.3 MB 00:00 2024-09-10T08:26:30.429 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T08:26:30.464 INFO:teuthology.orchestra.run.smithi089.stdout:(151/156): NetworkManager-libnm-1.49.90-1.el9.x 3.6 MB/s | 1.8 MB 00:00 2024-09-10T08:26:30.488 INFO:teuthology.orchestra.run.smithi089.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 626 kB/s | 41 kB 00:00 2024-09-10T08:26:30.513 INFO:teuthology.orchestra.run.smithi089.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 5.0 MB/s | 249 kB 00:00 2024-09-10T08:26:30.578 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T08:26:30.635 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T08:26:30.663 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T08:26:30.775 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T08:26:30.797 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T08:26:30.860 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T08:26:30.872 INFO:teuthology.orchestra.run.smithi089.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 3.5 MB/s | 1.4 MB 00:00 2024-09-10T08:26:30.882 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T08:26:30.906 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T08:26:31.039 INFO:teuthology.orchestra.run.smithi089.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 4.6 MB/s | 2.4 MB 00:00 2024-09-10T08:26:31.090 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T08:26:31.230 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T08:26:31.380 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T08:26:31.480 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T08:26:31.548 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T08:26:31.770 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T08:26:31.880 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T08:26:31.975 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T08:26:32.020 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T08:26:32.054 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T08:26:32.118 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T08:26:32.163 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T08:26:32.195 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T08:26:32.239 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T08:26:32.271 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T08:26:32.299 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T08:26:32.359 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T08:26:32.413 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T08:26:32.444 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T08:26:32.488 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T08:26:32.562 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T08:26:32.598 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T08:26:32.641 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T08:26:32.670 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T08:26:32.738 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T08:26:32.793 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T08:26:32.866 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T08:26:32.997 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T08:26:33.045 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T08:26:33.567 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T08:26:33.649 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T08:26:33.678 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T08:26:33.714 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T08:26:33.752 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T08:26:33.793 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T08:26:33.943 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T08:26:33.999 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T08:26:34.053 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T08:26:34.126 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T08:26:34.158 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T08:26:34.194 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T08:26:34.241 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T08:26:34.277 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T08:26:34.320 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T08:26:34.362 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T08:26:34.498 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T08:26:34.547 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T08:26:34.588 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T08:26:34.690 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T08:26:34.731 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T08:26:34.770 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T08:26:34.796 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T08:26:34.821 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T08:26:34.895 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T08:26:35.021 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T08:26:35.253 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T08:26:35.291 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T08:26:35.325 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T08:26:35.364 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T08:26:35.395 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T08:26:35.425 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T08:26:35.459 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T08:26:35.493 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T08:26:35.535 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T08:26:35.569 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T08:26:35.655 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T08:26:35.673 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-10T08:26:35.699 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T08:26:35.730 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T08:26:35.795 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T08:26:35.832 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T08:26:35.889 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T08:26:35.923 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T08:26:36.132 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T08:26:36.172 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T08:26:36.197 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T08:26:36.240 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T08:26:36.570 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T08:26:36.603 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T08:26:36.653 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T08:26:36.681 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T08:26:36.687 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T08:26:36.786 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T08:26:36.794 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T08:26:36.823 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T08:26:36.850 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T08:26:36.895 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T08:26:36.914 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T08:26:36.944 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T08:26:36.981 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T08:26:37.014 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T08:26:37.091 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T08:26:37.150 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T08:26:37.178 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T08:26:37.199 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T08:26:37.229 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T08:26:37.281 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T08:26:37.304 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T08:26:37.314 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T08:26:37.366 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T08:26:37.396 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T08:26:37.420 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T08:26:37.564 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T08:26:37.710 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T08:26:37.837 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T08:26:37.939 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T08:26:37.942 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T08:26:38.038 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T08:26:38.058 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T08:26:38.259 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T08:26:38.372 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T08:26:38.473 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T08:26:38.502 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T08:26:38.545 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T08:26:38.610 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T08:26:38.654 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T08:26:38.686 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T08:26:38.730 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T08:26:38.762 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T08:26:38.790 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T08:26:38.860 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T08:26:38.912 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T08:26:38.943 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T08:26:38.978 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T08:26:39.054 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T08:26:39.089 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T08:26:39.132 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T08:26:39.161 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T08:26:39.230 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T08:26:39.263 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T08:26:39.279 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T08:26:39.353 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T08:26:39.357 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T08:26:39.385 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T08:26:39.418 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T08:26:39.485 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T08:26:39.531 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T08:26:39.731 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T08:26:39.773 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T08:26:39.862 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T08:26:39.894 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T08:26:39.930 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T08:26:40.049 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T08:26:40.127 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T08:26:40.157 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T08:26:40.199 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T08:26:40.237 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T08:26:40.304 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T08:26:40.414 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T08:26:40.459 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T08:26:40.474 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T08:26:40.513 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T08:26:40.524 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T08:26:40.536 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T08:26:40.578 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T08:26:40.651 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T08:26:40.683 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T08:26:40.719 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T08:26:40.766 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T08:26:40.802 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T08:26:40.837 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T08:26:40.879 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T08:26:41.014 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T08:26:41.056 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T08:26:41.100 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T08:26:41.201 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T08:26:41.233 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T08:26:41.273 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T08:26:41.391 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T08:26:41.415 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T08:26:41.492 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T08:26:41.617 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T08:26:41.647 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T08:26:41.667 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T08:26:41.701 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T08:26:41.740 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T08:26:41.772 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T08:26:41.802 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T08:26:41.837 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T08:26:41.870 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T08:26:41.911 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T08:26:41.946 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T08:26:42.032 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T08:26:42.085 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T08:26:42.116 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T08:26:42.171 INFO:teuthology.orchestra.run.smithi089.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 852 kB/s | 16 MB 00:19 2024-09-10T08:26:42.177 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-09-10T08:26:42.178 INFO:teuthology.orchestra.run.smithi089.stdout:Total 7.3 MB/s | 226 MB 00:31 2024-09-10T08:26:42.218 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T08:26:42.266 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T08:26:42.846 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T08:26:42.987 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T08:26:43.039 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T08:26:43.055 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T08:26:43.073 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T08:26:43.174 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T08:26:43.227 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T08:26:43.272 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T08:26:43.352 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T08:26:43.392 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T08:26:43.578 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T08:26:43.599 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T08:26:43.607 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T08:26:43.635 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T08:26:43.684 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T08:26:43.731 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T08:26:43.732 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T08:26:43.732 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T08:26:43.732 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-10T08:26:43.762 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T08:26:44.237 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-09-10T08:26:44.341 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T08:26:44.420 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-09-10T08:26:44.420 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-09-10T08:26:44.454 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T08:26:45.672 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T08:26:45.755 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T08:26:45.783 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T08:26:45.816 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T08:26:46.128 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T08:26:46.171 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T08:26:46.244 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T08:26:46.276 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T08:26:46.311 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T08:26:46.759 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-09-10T08:26:46.760 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-09-10T08:26:46.795 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T08:26:46.832 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T08:26:46.903 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T08:26:46.933 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T08:26:49.287 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T08:26:49.322 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T08:26:49.869 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T08:26:49.909 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T08:26:50.002 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T08:26:50.003 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T08:26:50.003 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T08:26:50.003 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:26:50.027 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T08:26:53.375 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-09-10T08:26:53.510 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T08:26:53.576 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T08:26:53.667 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T08:26:53.712 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T08:26:53.744 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T08:26:53.810 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T08:26:54.181 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T08:26:54.295 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T08:26:54.447 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T08:26:54.487 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T08:26:54.609 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T08:26:54.648 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T08:26:54.849 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T08:26:54.907 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T08:26:54.934 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T08:26:55.051 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T08:26:55.077 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T08:26:55.159 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T08:26:55.186 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T08:26:55.226 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T08:26:55.463 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T08:26:55.637 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T08:26:55.657 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T08:26:55.657 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /sys 2024-09-10T08:26:55.658 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /proc 2024-09-10T08:26:55.658 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /mnt 2024-09-10T08:26:55.658 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /var/tmp 2024-09-10T08:26:55.658 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /home 2024-09-10T08:26:55.658 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /root 2024-09-10T08:26:55.658 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /tmp 2024-09-10T08:26:55.658 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-10T08:26:55.819 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T08:26:55.833 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T08:26:55.849 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T08:26:55.849 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:26:55.849 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T08:26:55.849 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-10T08:26:55.954 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T08:26:56.046 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T08:26:56.334 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T08:26:56.468 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T08:26:56.603 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T08:26:56.660 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T08:26:56.705 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T08:26:56.787 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T08:26:56.849 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T08:26:56.888 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T08:26:56.943 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T08:26:56.981 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T08:26:57.015 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T08:26:57.104 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T08:26:57.176 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T08:26:57.211 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T08:26:57.258 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T08:26:57.336 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T08:26:57.346 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T08:26:57.362 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T08:26:57.363 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:26:57.363 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T08:26:57.363 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T08:26:57.363 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T08:26:57.363 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-10T08:26:57.390 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T08:26:57.439 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T08:26:57.487 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T08:26:57.575 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T08:26:57.595 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T08:26:57.618 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T08:26:57.618 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:26:57.618 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T08:26:57.618 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T08:26:57.619 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T08:26:57.619 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-10T08:26:57.639 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T08:26:57.730 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T08:26:57.888 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T08:26:57.951 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T08:26:58.054 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T08:26:58.077 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T08:26:58.077 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:26:58.077 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T08:26:58.077 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T08:26:58.077 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T08:26:58.077 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-10T08:26:58.184 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T08:26:58.249 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T08:26:58.313 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T08:26:58.498 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T08:26:58.538 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T08:26:58.601 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T08:26:58.616 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T08:26:58.702 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T08:26:58.712 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T08:26:58.782 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T08:26:58.828 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T08:26:58.851 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T08:26:58.863 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T08:26:58.905 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T08:26:58.924 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T08:26:58.936 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T08:26:58.964 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T08:26:59.115 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T08:26:59.209 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T08:26:59.268 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T08:26:59.353 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T08:26:59.377 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T08:26:59.401 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T08:26:59.405 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T08:26:59.436 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T08:26:59.438 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T08:26:59.496 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T08:26:59.498 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T08:26:59.528 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T08:26:59.545 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T08:26:59.591 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T08:26:59.641 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T08:26:59.811 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T08:26:59.858 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T08:26:59.901 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T08:27:00.019 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T08:27:00.059 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T08:27:00.100 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T08:27:00.139 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T08:27:00.164 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T08:27:00.270 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T08:27:00.455 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T08:27:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T08:27:00.538 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T08:27:00.567 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T08:27:00.591 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T08:27:00.622 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T08:27:00.638 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T08:27:00.678 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T08:27:00.715 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T08:27:00.750 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T08:27:00.816 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T08:27:00.859 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T08:27:00.892 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T08:27:00.991 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T08:27:01.043 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T08:27:01.097 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T08:27:01.218 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T08:27:01.293 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T08:27:01.587 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T08:27:01.613 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T08:27:01.714 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T08:27:01.875 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T08:27:01.899 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T08:27:01.922 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T08:27:01.922 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:01.922 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T08:27:01.922 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T08:27:01.922 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T08:27:01.922 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-10T08:27:01.966 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T08:27:01.977 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T08:27:02.068 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T08:27:02.122 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T08:27:02.167 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T08:27:02.228 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T08:27:02.271 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T08:27:02.383 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T08:27:02.451 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T08:27:02.488 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T08:27:02.488 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /sys 2024-09-10T08:27:02.488 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /proc 2024-09-10T08:27:02.488 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /mnt 2024-09-10T08:27:02.489 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /var/tmp 2024-09-10T08:27:02.489 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /home 2024-09-10T08:27:02.489 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /root 2024-09-10T08:27:02.489 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /tmp 2024-09-10T08:27:02.489 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:27:02.506 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T08:27:02.601 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T08:27:02.638 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T08:27:02.682 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T08:27:02.697 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T08:27:02.698 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:02.698 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T08:27:02.698 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:27:02.872 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T08:27:03.048 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T08:27:03.077 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T08:27:03.101 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T08:27:03.101 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:03.101 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T08:27:03.101 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T08:27:03.101 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T08:27:03.101 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-10T08:27:03.111 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T08:27:03.144 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T08:27:03.167 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T08:27:03.167 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:03.167 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T08:27:03.167 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-10T08:27:03.211 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T08:27:03.213 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T08:27:03.234 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T08:27:03.234 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:03.234 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T08:27:03.234 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T08:27:03.234 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T08:27:03.235 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-10T08:27:03.545 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T08:27:03.568 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T08:27:03.568 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:03.568 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T08:27:03.568 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T08:27:03.568 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T08:27:03.569 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-10T08:27:04.022 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T08:27:04.158 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T08:27:04.168 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T08:27:04.195 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T08:27:04.195 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:04.195 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T08:27:04.196 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T08:27:04.196 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T08:27:04.196 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:27:04.418 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T08:27:04.442 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T08:27:04.443 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:04.443 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T08:27:04.443 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T08:27:04.443 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T08:27:04.443 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:27:04.878 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T08:27:04.901 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T08:27:04.901 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:04.901 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T08:27:04.902 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T08:27:04.902 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T08:27:04.902 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:27:05.007 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T08:27:05.064 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T08:27:05.119 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T08:27:05.312 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T08:27:05.362 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T08:27:05.430 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T08:27:05.519 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T08:27:05.649 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T08:27:05.695 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T08:27:05.711 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T08:27:05.727 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T08:27:05.755 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T08:27:05.819 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T08:27:05.855 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T08:27:05.889 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T08:27:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T08:27:06.232 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T08:27:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T08:27:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T08:27:06.276 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T08:27:06.314 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T08:27:06.343 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T08:27:06.395 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T08:27:06.432 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T08:27:06.480 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T08:27:06.997 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T08:27:07.071 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T08:27:07.161 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T08:27:07.189 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T08:27:07.387 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T08:27:07.428 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T08:27:08.091 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T08:27:08.132 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T08:27:08.173 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T08:27:08.197 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T08:27:08.224 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T08:27:08.392 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T08:27:08.403 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T08:27:08.420 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T08:27:08.446 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T08:27:08.476 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T08:27:08.506 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T08:27:08.524 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T08:27:08.552 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T08:27:08.683 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T08:27:08.714 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T08:27:08.738 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T08:27:08.739 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:08.739 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T08:27:08.739 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T08:27:08.739 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T08:27:08.739 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:27:08.790 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T08:27:08.802 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T08:27:08.899 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T08:27:08.955 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T08:27:09.162 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T08:27:09.200 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T08:27:09.219 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T08:27:09.701 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T08:27:09.753 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T08:27:09.877 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T08:27:09.918 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T08:27:09.942 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T08:27:09.942 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:09.942 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T08:27:09.942 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T08:27:09.942 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T08:27:09.942 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:27:09.985 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T08:27:10.008 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T08:27:10.008 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:10.009 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T08:27:10.009 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:27:10.041 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T08:27:10.060 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T08:27:10.085 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T08:27:10.085 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:10.085 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T08:27:10.085 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T08:27:10.085 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T08:27:10.085 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:27:10.129 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T08:27:10.170 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T08:27:10.251 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T08:27:10.299 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T08:27:10.349 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T08:27:10.375 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T08:27:10.375 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T08:27:10.377 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T08:27:10.394 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T08:27:10.418 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T08:27:10.419 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:10.419 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T08:27:10.419 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T08:27:10.419 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T08:27:10.419 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:27:10.423 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T08:27:10.459 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T08:27:10.459 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T08:27:10.543 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T08:27:10.553 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T08:27:10.554 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T08:27:10.554 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T08:27:10.554 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:27:10.568 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T08:27:10.573 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T08:27:10.573 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T08:27:10.591 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T08:27:10.754 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T08:27:10.754 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T08:27:10.838 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T08:27:10.901 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T08:27:14.957 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T08:27:15.002 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T08:27:15.039 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T08:27:15.072 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T08:27:15.099 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T08:27:15.277 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T08:27:15.329 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T08:27:15.359 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T08:27:15.389 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T08:27:15.435 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T08:27:16.009 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T08:27:16.050 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T08:27:16.070 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T08:27:16.453 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T08:27:16.453 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T08:27:16.453 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T08:27:16.454 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T08:27:16.454 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T08:27:16.454 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T08:27:16.454 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T08:27:16.454 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T08:27:16.454 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T08:27:16.454 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T08:27:16.454 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T08:27:16.454 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T08:27:16.454 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T08:27:16.454 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T08:27:16.454 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T08:27:16.454 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T08:27:16.455 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T08:27:16.455 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T08:27:16.455 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T08:27:16.455 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T08:27:16.455 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T08:27:16.455 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T08:27:16.455 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T08:27:16.455 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T08:27:16.455 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T08:27:16.455 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T08:27:16.455 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T08:27:16.455 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T08:27:16.467 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T08:27:16.467 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T08:27:16.467 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T08:27:16.467 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T08:27:16.468 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T08:27:16.468 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T08:27:16.468 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T08:27:16.468 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T08:27:16.468 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T08:27:16.468 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T08:27:16.468 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T08:27:16.468 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T08:27:16.468 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T08:27:16.468 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T08:27:16.468 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T08:27:16.468 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T08:27:16.469 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T08:27:16.469 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T08:27:16.469 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T08:27:16.469 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T08:27:16.469 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T08:27:16.469 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T08:27:16.469 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T08:27:16.469 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T08:27:16.469 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T08:27:16.469 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T08:27:16.469 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T08:27:16.469 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T08:27:16.469 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T08:27:16.469 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T08:27:16.470 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T08:27:16.470 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T08:27:16.470 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T08:27:16.470 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T08:27:16.470 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T08:27:16.470 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T08:27:16.470 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T08:27:16.470 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T08:27:16.470 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T08:27:16.470 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T08:27:16.470 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T08:27:16.470 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T08:27:16.470 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T08:27:16.471 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T08:27:16.471 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T08:27:16.471 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T08:27:16.471 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T08:27:16.471 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T08:27:16.471 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T08:27:16.471 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T08:27:16.471 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T08:27:16.471 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T08:27:16.471 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T08:27:16.471 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T08:27:16.471 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T08:27:16.471 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T08:27:16.472 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T08:27:16.472 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T08:27:16.472 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T08:27:16.472 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T08:27:16.472 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T08:27:16.472 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T08:27:16.472 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T08:27:16.472 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T08:27:16.472 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T08:27:16.472 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T08:27:16.472 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T08:27:16.472 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T08:27:16.472 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T08:27:16.473 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T08:27:16.473 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T08:27:16.473 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T08:27:16.473 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T08:27:16.473 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T08:27:16.473 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T08:27:16.473 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T08:27:16.473 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T08:27:16.473 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T08:27:16.473 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T08:27:16.473 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T08:27:16.473 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T08:27:16.473 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T08:27:16.473 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T08:27:16.474 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T08:27:16.474 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T08:27:16.474 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T08:27:16.474 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T08:27:16.474 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T08:27:16.474 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T08:27:16.474 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T08:27:16.474 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T08:27:16.474 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T08:27:16.474 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T08:27:16.474 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T08:27:16.474 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T08:27:16.474 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T08:27:16.475 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T08:27:16.475 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T08:27:16.475 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T08:27:16.475 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T08:27:16.475 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T08:27:16.475 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T08:27:16.475 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T08:27:16.475 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T08:27:16.475 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T08:27:16.475 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T08:27:16.475 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T08:27:16.475 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T08:27:16.475 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T08:27:16.475 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T08:27:16.476 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T08:27:16.476 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T08:27:16.476 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T08:27:16.476 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T08:27:16.476 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T08:27:16.476 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T08:27:16.476 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T08:27:16.476 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T08:27:16.476 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T08:27:16.476 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T08:27:16.476 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T08:27:16.476 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T08:27:16.476 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T08:27:16.476 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T08:27:16.477 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T08:27:16.477 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T08:27:16.477 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T08:27:16.477 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T08:27:16.477 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T08:27:16.477 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T08:27:16.477 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T08:27:16.477 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T08:27:16.477 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T08:27:16.477 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T08:27:16.477 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T08:27:16.477 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T08:27:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T08:27:16.971 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T08:27:17.012 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T08:27:17.093 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T08:27:17.154 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T08:27:17.183 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T08:27:17.210 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T08:27:17.210 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T08:27:17.294 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T08:27:17.294 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T08:27:17.378 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T08:27:17.403 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T08:27:17.408 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T08:27:17.409 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T08:27:17.588 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T08:27:17.588 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T08:27:17.672 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T08:27:17.735 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T08:27:19.184 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T08:27:19.184 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-10T08:27:19.184 INFO:teuthology.orchestra.run.smithi111.stdout:Upgraded: 2024-09-10T08:27:19.184 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T08:27:19.184 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T08:27:19.185 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T08:27:19.185 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T08:27:19.185 INFO:teuthology.orchestra.run.smithi111.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.185 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.185 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T08:27:19.185 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T08:27:19.185 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-10T08:27:19.185 INFO:teuthology.orchestra.run.smithi111.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T08:27:19.185 INFO:teuthology.orchestra.run.smithi111.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T08:27:19.185 INFO:teuthology.orchestra.run.smithi111.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T08:27:19.185 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T08:27:19.185 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.186 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.186 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.186 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.186 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:19.186 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.186 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.186 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.186 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:19.186 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:19.186 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:19.186 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:19.186 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:19.186 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.187 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.187 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:19.187 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.187 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.187 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.187 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:19.187 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:19.187 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.187 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.187 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:19.187 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T08:27:19.187 INFO:teuthology.orchestra.run.smithi111.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T08:27:19.187 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T08:27:19.188 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T08:27:19.188 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T08:27:19.188 INFO:teuthology.orchestra.run.smithi111.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T08:27:19.188 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T08:27:19.188 INFO:teuthology.orchestra.run.smithi111.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T08:27:19.188 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T08:27:19.188 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T08:27:19.188 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T08:27:19.188 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.188 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.188 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.188 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T08:27:19.189 INFO:teuthology.orchestra.run.smithi111.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T08:27:19.189 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T08:27:19.189 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T08:27:19.189 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T08:27:19.189 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.189 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.189 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T08:27:19.189 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.189 INFO:teuthology.orchestra.run.smithi111.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T08:27:19.189 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T08:27:19.189 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T08:27:19.189 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T08:27:19.189 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T08:27:19.189 INFO:teuthology.orchestra.run.smithi111.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T08:27:19.190 INFO:teuthology.orchestra.run.smithi111.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T08:27:19.190 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T08:27:19.190 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T08:27:19.190 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T08:27:19.190 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T08:27:19.190 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T08:27:19.190 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T08:27:19.190 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T08:27:19.190 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T08:27:19.190 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T08:27:19.190 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T08:27:19.190 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T08:27:19.190 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T08:27:19.191 INFO:teuthology.orchestra.run.smithi111.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T08:27:19.191 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T08:27:19.191 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T08:27:19.191 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T08:27:19.191 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.191 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.191 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.191 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T08:27:19.191 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T08:27:19.191 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T08:27:19.191 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T08:27:19.191 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T08:27:19.191 INFO:teuthology.orchestra.run.smithi111.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T08:27:19.192 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T08:27:19.192 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T08:27:19.192 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T08:27:19.192 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T08:27:19.192 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T08:27:19.192 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T08:27:19.192 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T08:27:19.192 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T08:27:19.192 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T08:27:19.192 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T08:27:19.192 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T08:27:19.192 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T08:27:19.192 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T08:27:19.193 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T08:27:19.193 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T08:27:19.193 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T08:27:19.193 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T08:27:19.193 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T08:27:19.193 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T08:27:19.193 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T08:27:19.193 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T08:27:19.193 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T08:27:19.193 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T08:27:19.193 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T08:27:19.193 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T08:27:19.193 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T08:27:19.194 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T08:27:19.194 INFO:teuthology.orchestra.run.smithi111.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T08:27:19.194 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T08:27:19.194 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T08:27:19.194 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T08:27:19.194 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T08:27:19.194 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T08:27:19.194 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T08:27:19.194 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.194 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.194 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T08:27:19.194 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T08:27:19.194 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T08:27:19.194 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.195 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T08:27:19.195 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T08:27:19.195 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T08:27:19.195 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T08:27:19.195 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T08:27:19.195 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T08:27:19.195 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T08:27:19.195 INFO:teuthology.orchestra.run.smithi111.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T08:27:19.195 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T08:27:19.195 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T08:27:19.195 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T08:27:19.195 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T08:27:19.195 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T08:27:19.196 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T08:27:19.196 INFO:teuthology.orchestra.run.smithi111.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T08:27:19.196 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.196 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.196 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:19.196 INFO:teuthology.orchestra.run.smithi111.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T08:27:19.196 INFO:teuthology.orchestra.run.smithi111.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T08:27:19.196 INFO:teuthology.orchestra.run.smithi111.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T08:27:19.196 INFO:teuthology.orchestra.run.smithi111.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T08:27:19.196 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T08:27:19.196 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T08:27:19.197 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T08:27:19.197 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-10T08:27:19.197 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-10T08:27:19.592 DEBUG:teuthology.parallel:result is None 2024-09-10T08:27:23.394 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T08:27:23.395 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /sys 2024-09-10T08:27:23.395 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /proc 2024-09-10T08:27:23.395 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /mnt 2024-09-10T08:27:23.395 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /var/tmp 2024-09-10T08:27:23.395 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /home 2024-09-10T08:27:23.395 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /root 2024-09-10T08:27:23.395 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /tmp 2024-09-10T08:27:23.395 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:27:23.412 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T08:27:23.412 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T08:27:23.412 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T08:27:23.412 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T08:27:23.412 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T08:27:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T08:27:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T08:27:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T08:27:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T08:27:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T08:27:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T08:27:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T08:27:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T08:27:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T08:27:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T08:27:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T08:27:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T08:27:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T08:27:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T08:27:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T08:27:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T08:27:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T08:27:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T08:27:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T08:27:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T08:27:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T08:27:23.415 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T08:27:23.415 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T08:27:23.415 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T08:27:23.415 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T08:27:23.415 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T08:27:23.432 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T08:27:23.432 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T08:27:23.432 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T08:27:23.432 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T08:27:23.432 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T08:27:23.432 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T08:27:23.432 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T08:27:23.433 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T08:27:23.433 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T08:27:23.433 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T08:27:23.433 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T08:27:23.433 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T08:27:23.433 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T08:27:23.433 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T08:27:23.433 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T08:27:23.433 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T08:27:23.433 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T08:27:23.433 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T08:27:23.433 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T08:27:23.434 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T08:27:23.434 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T08:27:23.434 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T08:27:23.434 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T08:27:23.434 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T08:27:23.434 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T08:27:23.434 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T08:27:23.434 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T08:27:23.434 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T08:27:23.434 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T08:27:23.434 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T08:27:23.434 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T08:27:23.435 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T08:27:23.435 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T08:27:23.435 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T08:27:23.435 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T08:27:23.435 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T08:27:23.435 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T08:27:23.435 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T08:27:23.435 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T08:27:23.435 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T08:27:23.435 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T08:27:23.435 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T08:27:23.435 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T08:27:23.435 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T08:27:23.436 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T08:27:23.436 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T08:27:23.436 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T08:27:23.436 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T08:27:23.436 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T08:27:23.436 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T08:27:23.436 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T08:27:23.436 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T08:27:23.436 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T08:27:23.436 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T08:27:23.436 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T08:27:23.436 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T08:27:23.437 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T08:27:23.437 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T08:27:23.437 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T08:27:23.437 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T08:27:23.437 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T08:27:23.437 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T08:27:23.437 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T08:27:23.437 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T08:27:23.437 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T08:27:23.438 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T08:27:23.438 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T08:27:23.438 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T08:27:23.438 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T08:27:23.438 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T08:27:23.438 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T08:27:23.438 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T08:27:23.438 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T08:27:23.438 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T08:27:23.438 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T08:27:23.439 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T08:27:23.439 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T08:27:23.439 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T08:27:23.439 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T08:27:23.439 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T08:27:23.439 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T08:27:23.439 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T08:27:23.439 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T08:27:23.439 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T08:27:23.440 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T08:27:23.440 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T08:27:23.440 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T08:27:23.440 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T08:27:23.440 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T08:27:23.440 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T08:27:23.440 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T08:27:23.440 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T08:27:23.440 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T08:27:23.440 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T08:27:23.440 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T08:27:23.440 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T08:27:23.441 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T08:27:23.441 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T08:27:23.441 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T08:27:23.441 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T08:27:23.441 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T08:27:23.441 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T08:27:23.441 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T08:27:23.441 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T08:27:23.441 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T08:27:23.441 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T08:27:23.441 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T08:27:23.441 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T08:27:23.442 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T08:27:23.442 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T08:27:23.442 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T08:27:23.442 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T08:27:23.442 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T08:27:23.442 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T08:27:23.442 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T08:27:23.442 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T08:27:23.442 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T08:27:23.442 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T08:27:23.442 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T08:27:23.442 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T08:27:23.443 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T08:27:23.443 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T08:27:23.443 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T08:27:23.443 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T08:27:23.443 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T08:27:23.443 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T08:27:23.443 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T08:27:23.443 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T08:27:23.443 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T08:27:23.443 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T08:27:23.443 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T08:27:23.443 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T08:27:23.443 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T08:27:23.599 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T08:27:23.637 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T08:27:23.637 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:23.637 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T08:27:23.638 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:27:25.254 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T08:27:25.313 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T08:27:25.313 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:25.313 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T08:27:25.314 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T08:27:25.314 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T08:27:25.314 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:27:25.571 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T08:27:25.623 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T08:27:25.623 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:25.623 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T08:27:25.623 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T08:27:25.623 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T08:27:25.624 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:27:25.628 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T08:27:25.628 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:27:25.629 INFO:teuthology.orchestra.run.smithi149.stdout:Upgraded: 2024-09-10T08:27:25.629 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T08:27:25.629 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T08:27:25.629 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T08:27:25.629 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T08:27:25.629 INFO:teuthology.orchestra.run.smithi149.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.629 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.629 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T08:27:25.629 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T08:27:25.630 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-10T08:27:25.630 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T08:27:25.630 INFO:teuthology.orchestra.run.smithi149.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T08:27:25.630 INFO:teuthology.orchestra.run.smithi149.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T08:27:25.630 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T08:27:25.630 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.630 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.630 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.630 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.630 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:25.630 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.631 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.631 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.631 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:25.631 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:25.631 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:25.631 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:25.631 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:25.631 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.631 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.631 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:25.631 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.632 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.632 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.632 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:25.632 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:25.632 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.632 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.632 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:25.632 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T08:27:25.633 INFO:teuthology.orchestra.run.smithi149.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T08:27:25.633 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T08:27:25.633 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T08:27:25.633 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T08:27:25.633 INFO:teuthology.orchestra.run.smithi149.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T08:27:25.633 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T08:27:25.634 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T08:27:25.634 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T08:27:25.634 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T08:27:25.634 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T08:27:25.634 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.634 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.634 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.634 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T08:27:25.634 INFO:teuthology.orchestra.run.smithi149.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T08:27:25.634 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T08:27:25.634 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T08:27:25.634 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T08:27:25.634 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.635 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.635 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T08:27:25.635 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.635 INFO:teuthology.orchestra.run.smithi149.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T08:27:25.635 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T08:27:25.635 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T08:27:25.635 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T08:27:25.635 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T08:27:25.635 INFO:teuthology.orchestra.run.smithi149.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T08:27:25.635 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T08:27:25.635 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T08:27:25.635 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T08:27:25.635 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T08:27:25.636 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T08:27:25.636 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T08:27:25.636 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T08:27:25.636 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T08:27:25.636 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T08:27:25.636 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T08:27:25.636 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T08:27:25.636 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T08:27:25.636 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T08:27:25.636 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T08:27:25.636 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T08:27:25.636 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T08:27:25.636 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T08:27:25.637 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.637 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.637 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.637 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T08:27:25.637 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T08:27:25.637 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T08:27:25.637 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T08:27:25.637 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T08:27:25.637 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T08:27:25.637 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T08:27:25.637 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T08:27:25.637 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T08:27:25.638 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T08:27:25.638 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T08:27:25.638 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T08:27:25.638 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T08:27:25.638 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T08:27:25.638 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T08:27:25.638 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T08:27:25.638 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T08:27:25.638 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T08:27:25.638 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T08:27:25.638 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T08:27:25.638 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T08:27:25.639 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T08:27:25.639 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T08:27:25.639 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T08:27:25.639 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T08:27:25.639 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T08:27:25.639 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T08:27:25.639 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T08:27:25.639 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T08:27:25.639 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T08:27:25.639 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T08:27:25.639 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T08:27:25.639 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T08:27:25.639 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T08:27:25.640 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T08:27:25.640 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T08:27:25.640 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T08:27:25.640 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T08:27:25.640 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T08:27:25.640 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T08:27:25.640 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.640 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.640 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T08:27:25.640 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T08:27:25.640 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T08:27:25.640 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.640 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T08:27:25.641 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T08:27:25.641 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T08:27:25.641 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T08:27:25.641 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T08:27:25.641 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T08:27:25.641 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T08:27:25.641 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T08:27:25.641 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T08:27:25.641 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T08:27:25.641 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T08:27:25.641 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T08:27:25.641 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T08:27:25.642 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T08:27:25.642 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T08:27:25.642 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.642 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.642 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:25.642 INFO:teuthology.orchestra.run.smithi149.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T08:27:25.642 INFO:teuthology.orchestra.run.smithi149.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T08:27:25.642 INFO:teuthology.orchestra.run.smithi149.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T08:27:25.642 INFO:teuthology.orchestra.run.smithi149.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T08:27:25.642 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T08:27:25.642 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T08:27:25.643 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T08:27:25.643 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:27:25.643 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-10T08:27:26.019 DEBUG:teuthology.parallel:result is None 2024-09-10T08:27:26.105 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T08:27:26.156 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T08:27:26.156 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:26.156 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T08:27:26.157 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T08:27:26.157 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T08:27:26.157 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:27:26.280 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T08:27:26.339 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T08:27:26.405 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T08:27:26.628 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T08:27:26.684 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T08:27:26.769 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T08:27:26.871 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T08:27:27.053 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T08:27:27.102 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T08:27:27.149 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T08:27:27.175 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T08:27:27.665 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T08:27:27.700 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T08:27:27.733 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T08:27:27.800 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T08:27:27.863 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T08:27:29.074 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T08:27:29.125 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T08:27:30.208 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T08:27:30.242 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T08:27:30.379 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T08:27:30.554 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T08:27:30.594 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T08:27:30.645 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T08:27:30.645 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:30.645 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T08:27:30.645 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T08:27:30.645 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T08:27:30.645 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:27:30.703 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T08:27:30.730 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T08:27:30.844 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T08:27:30.911 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T08:27:31.902 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T08:27:31.939 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T08:27:31.990 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T08:27:31.990 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:31.990 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T08:27:31.990 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T08:27:31.991 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T08:27:31.991 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:27:32.048 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T08:27:32.098 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T08:27:32.098 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:32.098 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T08:27:32.098 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:27:32.149 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T08:27:32.201 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T08:27:32.201 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:32.201 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T08:27:32.201 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T08:27:32.201 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T08:27:32.201 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:27:32.542 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T08:27:32.594 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T08:27:32.594 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:27:32.594 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T08:27:32.594 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T08:27:32.594 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T08:27:32.594 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:27:37.555 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T08:27:37.598 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T08:27:37.641 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T08:27:37.670 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T08:27:37.706 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T08:27:37.934 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T08:27:37.987 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T08:27:38.016 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T08:27:38.048 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T08:27:38.084 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T08:27:38.856 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T08:27:38.916 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T08:27:38.958 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T08:27:40.225 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T08:27:40.358 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T08:27:40.591 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T08:27:40.591 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T08:27:40.591 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T08:27:40.592 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T08:27:40.630 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T08:27:40.630 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T08:27:40.737 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T08:27:40.737 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T08:27:40.854 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T08:27:40.879 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T08:27:40.895 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T08:27:40.895 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T08:27:41.140 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T08:27:41.140 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T08:27:41.241 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T08:27:41.340 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T08:27:41.543 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T08:27:41.543 INFO:teuthology.orchestra.run.smithi089.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-10T08:27:41.543 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:27:47.230 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T08:27:47.230 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T08:27:47.230 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T08:27:47.230 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T08:27:47.231 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T08:27:47.231 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T08:27:47.231 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T08:27:47.231 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T08:27:47.231 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T08:27:47.231 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T08:27:47.231 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T08:27:47.231 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T08:27:47.231 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T08:27:47.231 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T08:27:47.231 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T08:27:47.231 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T08:27:47.232 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T08:27:47.232 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T08:27:47.232 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T08:27:47.232 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T08:27:47.232 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T08:27:47.232 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T08:27:47.232 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T08:27:47.232 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T08:27:47.232 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T08:27:47.232 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T08:27:47.232 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T08:27:47.232 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T08:27:47.232 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T08:27:47.233 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T08:27:47.233 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T08:27:47.233 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T08:27:47.233 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T08:27:47.233 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T08:27:47.233 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T08:27:47.233 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T08:27:47.233 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T08:27:47.233 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T08:27:47.233 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T08:27:47.233 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T08:27:47.234 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T08:27:47.234 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T08:27:47.234 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T08:27:47.234 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T08:27:47.234 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T08:27:47.234 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T08:27:47.234 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T08:27:47.234 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T08:27:47.234 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T08:27:47.234 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T08:27:47.234 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T08:27:47.234 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T08:27:47.235 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T08:27:47.235 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T08:27:47.235 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T08:27:47.235 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T08:27:47.235 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T08:27:47.235 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T08:27:47.235 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T08:27:47.235 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T08:27:47.235 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T08:27:47.235 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T08:27:47.235 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T08:27:47.235 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T08:27:47.236 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T08:27:47.236 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T08:27:47.236 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T08:27:47.236 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T08:27:47.236 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T08:27:47.236 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T08:27:47.236 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T08:27:47.236 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T08:27:47.236 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T08:27:47.236 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T08:27:47.236 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T08:27:47.237 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T08:27:47.237 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T08:27:47.237 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T08:27:47.237 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T08:27:47.237 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T08:27:47.237 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T08:27:47.237 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T08:27:47.237 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T08:27:47.237 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T08:27:47.237 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T08:27:47.237 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T08:27:47.237 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T08:27:47.238 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T08:27:47.238 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T08:27:47.238 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T08:27:47.238 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T08:27:47.238 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T08:27:47.238 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T08:27:47.238 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T08:27:47.238 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T08:27:47.238 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T08:27:47.238 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T08:27:47.238 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T08:27:47.238 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T08:27:47.238 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T08:27:47.238 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T08:27:47.239 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T08:27:47.239 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T08:27:47.239 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T08:27:47.239 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T08:27:47.239 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T08:27:47.239 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T08:27:47.239 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T08:27:47.239 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T08:27:47.239 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T08:27:47.240 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T08:27:47.240 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T08:27:47.240 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T08:27:47.240 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T08:27:47.240 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T08:27:47.240 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T08:27:47.240 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T08:27:47.240 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T08:27:47.240 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T08:27:47.240 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T08:27:47.241 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T08:27:47.241 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T08:27:47.241 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T08:27:47.241 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T08:27:47.241 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T08:27:47.241 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T08:27:47.241 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T08:27:47.241 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T08:27:47.241 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T08:27:47.241 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T08:27:47.241 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T08:27:47.241 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T08:27:47.242 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T08:27:47.242 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T08:27:47.242 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T08:27:47.242 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T08:27:47.242 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T08:27:47.242 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T08:27:47.242 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T08:27:47.242 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T08:27:47.242 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T08:27:47.242 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T08:27:47.242 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T08:27:47.242 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T08:27:47.242 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T08:27:47.242 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T08:27:47.243 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T08:27:47.243 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T08:27:47.243 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T08:27:47.243 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T08:27:47.243 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T08:27:47.243 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T08:27:47.243 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T08:27:47.243 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T08:27:47.243 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T08:27:47.243 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T08:27:47.243 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T08:27:47.244 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T08:27:47.244 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T08:27:47.244 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T08:27:47.244 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T08:27:47.244 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T08:27:47.244 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T08:27:49.485 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T08:27:49.486 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:27:49.486 INFO:teuthology.orchestra.run.smithi089.stdout:Upgraded: 2024-09-10T08:27:49.486 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T08:27:49.486 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T08:27:49.486 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T08:27:49.486 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T08:27:49.486 INFO:teuthology.orchestra.run.smithi089.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.486 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.486 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T08:27:49.486 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T08:27:49.487 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-09-10T08:27:49.487 INFO:teuthology.orchestra.run.smithi089.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T08:27:49.487 INFO:teuthology.orchestra.run.smithi089.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T08:27:49.487 INFO:teuthology.orchestra.run.smithi089.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T08:27:49.487 INFO:teuthology.orchestra.run.smithi089.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T08:27:49.487 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.487 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.487 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.487 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.487 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:49.487 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.488 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.488 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.488 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:49.488 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:49.488 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:49.488 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:49.488 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:49.488 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.488 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.488 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:49.489 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.489 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.489 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.489 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:49.489 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:49.489 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.489 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.489 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:27:49.489 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T08:27:49.489 INFO:teuthology.orchestra.run.smithi089.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T08:27:49.490 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T08:27:49.490 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T08:27:49.490 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T08:27:49.490 INFO:teuthology.orchestra.run.smithi089.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T08:27:49.490 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T08:27:49.490 INFO:teuthology.orchestra.run.smithi089.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T08:27:49.490 INFO:teuthology.orchestra.run.smithi089.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T08:27:49.490 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T08:27:49.490 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T08:27:49.491 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.491 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.491 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.491 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T08:27:49.491 INFO:teuthology.orchestra.run.smithi089.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T08:27:49.491 INFO:teuthology.orchestra.run.smithi089.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T08:27:49.491 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T08:27:49.491 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T08:27:49.491 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.491 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.491 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T08:27:49.491 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.491 INFO:teuthology.orchestra.run.smithi089.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T08:27:49.492 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T08:27:49.492 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T08:27:49.492 INFO:teuthology.orchestra.run.smithi089.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T08:27:49.492 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T08:27:49.492 INFO:teuthology.orchestra.run.smithi089.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T08:27:49.492 INFO:teuthology.orchestra.run.smithi089.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T08:27:49.492 INFO:teuthology.orchestra.run.smithi089.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T08:27:49.492 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T08:27:49.492 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T08:27:49.493 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T08:27:49.493 INFO:teuthology.orchestra.run.smithi089.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T08:27:49.493 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T08:27:49.493 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T08:27:49.493 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T08:27:49.493 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T08:27:49.493 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T08:27:49.493 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T08:27:49.493 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T08:27:49.493 INFO:teuthology.orchestra.run.smithi089.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T08:27:49.493 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T08:27:49.493 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T08:27:49.493 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T08:27:49.494 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.494 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.494 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.494 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T08:27:49.494 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T08:27:49.495 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T08:27:49.495 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T08:27:49.495 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T08:27:49.495 INFO:teuthology.orchestra.run.smithi089.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T08:27:49.495 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T08:27:49.495 INFO:teuthology.orchestra.run.smithi089.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T08:27:49.495 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T08:27:49.495 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T08:27:49.495 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T08:27:49.496 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T08:27:49.496 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T08:27:49.496 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T08:27:49.496 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T08:27:49.496 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T08:27:49.496 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T08:27:49.496 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T08:27:49.496 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T08:27:49.496 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T08:27:49.496 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T08:27:49.496 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T08:27:49.496 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T08:27:49.497 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T08:27:49.497 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T08:27:49.497 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T08:27:49.497 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T08:27:49.497 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T08:27:49.497 INFO:teuthology.orchestra.run.smithi089.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T08:27:49.497 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T08:27:49.497 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T08:27:49.497 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T08:27:49.497 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T08:27:49.497 INFO:teuthology.orchestra.run.smithi089.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T08:27:49.497 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T08:27:49.497 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T08:27:49.498 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T08:27:49.498 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T08:27:49.498 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T08:27:49.498 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T08:27:49.498 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.498 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.498 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T08:27:49.498 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T08:27:49.498 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T08:27:49.498 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.498 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T08:27:49.498 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T08:27:49.498 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T08:27:49.498 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T08:27:49.499 INFO:teuthology.orchestra.run.smithi089.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T08:27:49.499 INFO:teuthology.orchestra.run.smithi089.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T08:27:49.499 INFO:teuthology.orchestra.run.smithi089.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T08:27:49.499 INFO:teuthology.orchestra.run.smithi089.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T08:27:49.499 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T08:27:49.499 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T08:27:49.499 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T08:27:49.499 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T08:27:49.499 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T08:27:49.499 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T08:27:49.499 INFO:teuthology.orchestra.run.smithi089.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T08:27:49.500 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.500 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.500 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:27:49.500 INFO:teuthology.orchestra.run.smithi089.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T08:27:49.500 INFO:teuthology.orchestra.run.smithi089.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T08:27:49.500 INFO:teuthology.orchestra.run.smithi089.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T08:27:49.500 INFO:teuthology.orchestra.run.smithi089.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T08:27:49.500 INFO:teuthology.orchestra.run.smithi089.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T08:27:49.500 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T08:27:49.501 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T08:27:49.501 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:27:49.501 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-09-10T08:27:50.082 DEBUG:teuthology.parallel:result is None 2024-09-10T08:27:50.082 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:27:50.235 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T08:27:50.280 INFO:teuthology.orchestra.run.smithi089.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T08:27:50.280 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T08:27:50.281 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T08:27:50.283 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:27:50.425 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T08:27:50.452 INFO:teuthology.orchestra.run.smithi111.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T08:27:50.453 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T08:27:50.453 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T08:27:50.455 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:27:50.593 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T08:27:50.620 INFO:teuthology.orchestra.run.smithi149.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T08:27:50.620 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T08:27:50.620 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T08:27:50.622 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T08:27:50.622 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:27:50.623 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T08:27:50.682 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-10T08:27:50.683 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T08:27:50.715 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T08:27:50.715 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T08:27:50.747 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T08:27:50.747 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:27:50.747 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T08:27:50.803 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T08:27:50.907 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-10T08:27:50.908 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T08:27:50.938 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T08:27:51.007 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T08:27:51.007 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T08:27:51.037 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T08:27:51.105 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T08:27:51.106 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:27:51.106 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T08:27:51.163 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T08:27:51.258 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-10T08:27:51.258 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T08:27:51.284 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T08:27:51.350 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T08:27:51.350 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T08:27:51.375 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T08:27:51.441 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T08:27:51.441 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:27:51.442 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T08:27:51.500 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T08:27:51.597 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-10T08:27:51.598 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T08:27:51.624 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T08:27:51.689 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T08:27:51.689 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T08:27:51.714 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T08:27:51.779 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T08:27:51.880 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-10T08:27:51.880 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:27:51.881 INFO:tasks.cephadm:Cluster fsid is 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:27:51.881 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T08:27:51.881 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.89', 'mon.b': '172.21.15.111', 'mon.c': '172.21.15.149'} 2024-09-10T08:27:51.881 INFO:tasks.cephadm:First mon is mon.a on smithi089 2024-09-10T08:27:51.881 INFO:tasks.cephadm:First mgr is x 2024-09-10T08:27:51.881 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T08:27:51.881 DEBUG:teuthology.orchestra.run.smithi089:> sudo hostname $(hostname -s) 2024-09-10T08:27:51.943 DEBUG:teuthology.orchestra.run.smithi111:> sudo hostname $(hostname -s) 2024-09-10T08:27:51.970 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname $(hostname -s) 2024-09-10T08:27:51.996 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-10T08:27:51.996 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:27:52.138 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82914/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-305-g1cf819b7', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-305.g1cf819b7'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 22:14:38.445020', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240909.165402-squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82912/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-305-g1cf819b7', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-305.g1cf819b7'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 22:05:03.996701', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240909.165402-squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-10T08:27:52.277 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240909.165402-squid, sha1 1cf819b7212a6048e92fc78c4eff963344f6d56b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:27:52.279 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm 2024-09-10T08:27:52.279 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm 2024-09-10T08:27:52.279 DEBUG:teuthology.orchestra.run.smithi089:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T08:27:52.689 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 08:27 /home/ubuntu/cephtest/cephadm 2024-09-10T08:27:52.690 DEBUG:teuthology.orchestra.run.smithi111:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T08:27:53.087 INFO:teuthology.orchestra.run.smithi111.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 08:27 /home/ubuntu/cephtest/cephadm 2024-09-10T08:27:53.087 DEBUG:teuthology.orchestra.run.smithi149:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T08:27:53.751 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 08:27 /home/ubuntu/cephtest/cephadm 2024-09-10T08:27:53.752 DEBUG:teuthology.orchestra.run.smithi089:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T08:27:53.789 DEBUG:teuthology.orchestra.run.smithi111:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T08:27:53.808 DEBUG:teuthology.orchestra.run.smithi149:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T08:27:53.847 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b on all hosts... 2024-09-10T08:27:53.848 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T08:27:53.853 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T08:27:53.856 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T08:27:55.296 INFO:teuthology.orchestra.run.smithi111.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T08:27:55.382 INFO:teuthology.orchestra.run.smithi149.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T08:27:55.917 INFO:teuthology.orchestra.run.smithi089.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T08:28:22.810 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-09-10T08:28:22.811 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T08:28:22.811 INFO:teuthology.orchestra.run.smithi149.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T08:28:22.811 INFO:teuthology.orchestra.run.smithi149.stdout: "repo_digests": [ 2024-09-10T08:28:22.812 INFO:teuthology.orchestra.run.smithi149.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T08:28:22.812 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-09-10T08:28:22.812 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-09-10T08:28:28.139 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-10T08:28:28.139 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T08:28:28.139 INFO:teuthology.orchestra.run.smithi111.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T08:28:28.139 INFO:teuthology.orchestra.run.smithi111.stdout: "repo_digests": [ 2024-09-10T08:28:28.139 INFO:teuthology.orchestra.run.smithi111.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T08:28:28.139 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-10T08:28:28.139 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-10T08:28:28.361 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:28:28.361 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T08:28:28.361 INFO:teuthology.orchestra.run.smithi089.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T08:28:28.361 INFO:teuthology.orchestra.run.smithi089.stdout: "repo_digests": [ 2024-09-10T08:28:28.361 INFO:teuthology.orchestra.run.smithi089.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T08:28:28.362 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-10T08:28:28.362 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:28:28.391 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /etc/ceph 2024-09-10T08:28:28.476 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /etc/ceph 2024-09-10T08:28:28.508 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph 2024-09-10T08:28:28.542 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 777 /etc/ceph 2024-09-10T08:28:28.597 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 777 /etc/ceph 2024-09-10T08:28:28.626 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 777 /etc/ceph 2024-09-10T08:28:28.655 INFO:tasks.cephadm:Writing seed config... 2024-09-10T08:28:28.656 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-10T08:28:28.656 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-10T08:28:28.656 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-10T08:28:28.656 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-10T08:28:28.656 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-10T08:28:28.656 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-10T08:28:28.657 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-10T08:28:28.657 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-10T08:28:28.657 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-10T08:28:28.657 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-10T08:28:28.657 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-10T08:28:28.657 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-10T08:28:28.657 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-10T08:28:28.657 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-10T08:28:28.657 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-10T08:28:28.657 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-10T08:28:28.657 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-10T08:28:28.657 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-10T08:28:28.657 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-10T08:28:28.657 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-10T08:28:28.658 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-10T08:28:28.658 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-10T08:28:28.658 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-10T08:28:28.658 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T08:28:28.658 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T08:28:28.658 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T08:28:28.658 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T08:28:28.658 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T08:28:28.658 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-10T08:28:28.658 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-10T08:28:28.658 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-10T08:28:28.658 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-10T08:28:28.658 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-10T08:28:28.658 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-10T08:28:28.659 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-10T08:28:28.659 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-10T08:28:28.659 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T08:28:28.659 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T08:28:28.659 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-10T08:28:28.659 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-10T08:28:28.659 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-10T08:28:28.659 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-10T08:28:28.659 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-10T08:28:28.659 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-10T08:28:28.659 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-10T08:28:28.659 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T08:28:28.659 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-10T08:28:28.660 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:28:28.660 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T08:28:28.695 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 91f747bc-6f4e-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-10T08:28:28.695 DEBUG:teuthology.orchestra.run.smithi089:mon.a> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T08:28:28.739 DEBUG:teuthology.orchestra.run.smithi089:mgr.x> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T08:28:28.783 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T08:28:28.783 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b -v bootstrap --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.89 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T08:28:29.039 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-09-10T08:28:29.039 INFO:teuthology.orchestra.run.smithi089.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b', '-v', 'bootstrap', '--fsid', '91f747bc-6f4e-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.89', '--skip-admin-label'] 2024-09-10T08:28:29.040 INFO:teuthology.orchestra.run.smithi089.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T08:28:29.040 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying podman|docker is present... 2024-09-10T08:28:29.083 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stdout 5.2.2 2024-09-10T08:28:29.083 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying lvm2 is present... 2024-09-10T08:28:29.083 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying time synchronization is in place... 2024-09-10T08:28:29.097 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T08:28:29.097 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T08:28:29.109 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T08:28:29.110 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout inactive 2024-09-10T08:28:29.123 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout enabled 2024-09-10T08:28:29.136 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout active 2024-09-10T08:28:29.136 INFO:teuthology.orchestra.run.smithi089.stdout:Unit chronyd.service is enabled and running 2024-09-10T08:28:29.136 INFO:teuthology.orchestra.run.smithi089.stdout:Repeating the final host check... 2024-09-10T08:28:29.178 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stdout 5.2.2 2024-09-10T08:28:29.178 INFO:teuthology.orchestra.run.smithi089.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T08:28:29.178 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl is present 2024-09-10T08:28:29.178 INFO:teuthology.orchestra.run.smithi089.stdout:lvcreate is present 2024-09-10T08:28:29.191 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T08:28:29.191 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T08:28:29.203 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T08:28:29.203 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout inactive 2024-09-10T08:28:29.216 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout enabled 2024-09-10T08:28:29.229 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout active 2024-09-10T08:28:29.229 INFO:teuthology.orchestra.run.smithi089.stdout:Unit chronyd.service is enabled and running 2024-09-10T08:28:29.229 INFO:teuthology.orchestra.run.smithi089.stdout:Host looks OK 2024-09-10T08:28:29.229 INFO:teuthology.orchestra.run.smithi089.stdout:Cluster fsid: 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:28:29.229 INFO:teuthology.orchestra.run.smithi089.stdout:Acquiring lock 140057522205552 on /run/cephadm/91f747bc-6f4e-11ef-bcea-c7b262605968.lock 2024-09-10T08:28:29.229 INFO:teuthology.orchestra.run.smithi089.stdout:Lock 140057522205552 acquired on /run/cephadm/91f747bc-6f4e-11ef-bcea-c7b262605968.lock 2024-09-10T08:28:29.230 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying IP 172.21.15.89 port 3300 ... 2024-09-10T08:28:29.230 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying IP 172.21.15.89 port 6789 ... 2024-09-10T08:28:29.230 INFO:teuthology.orchestra.run.smithi089.stdout:Base mon IP(s) is [172.21.15.89:3300, 172.21.15.89:6789], mon addrv is [v2:172.21.15.89:3300,v1:172.21.15.89:6789] 2024-09-10T08:28:29.237 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.89 metric 100 2024-09-10T08:28:29.237 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.89 metric 100 2024-09-10T08:28:29.243 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T08:28:29.243 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-10T08:28:29.243 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-10T08:28:29.249 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T08:28:29.249 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T08:28:29.249 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T08:28:29.249 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-10T08:28:29.249 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d501/64 scope link noprefixroute 2024-09-10T08:28:29.249 INFO:teuthology.orchestra.run.smithi089.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T08:28:29.250 INFO:teuthology.orchestra.run.smithi089.stdout:Mon IP `172.21.15.89` is in CIDR network `172.21.0.0/20` 2024-09-10T08:28:29.250 INFO:teuthology.orchestra.run.smithi089.stdout:Mon IP `172.21.15.89` is in CIDR network `172.21.0.0/20` 2024-09-10T08:28:29.250 INFO:teuthology.orchestra.run.smithi089.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T08:28:29.250 INFO:teuthology.orchestra.run.smithi089.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T08:28:29.251 INFO:teuthology.orchestra.run.smithi089.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T08:28:29.929 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stdout 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 2024-09-10T08:28:29.929 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T08:28:29.929 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T08:28:29.930 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stderr Copying blob sha256:3965ca83761b6e221cd35b61f2ebac0e3b1f9d68a1ce39c1ada69153a1d73116 2024-09-10T08:28:29.930 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T08:28:29.930 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stderr Copying config sha256:41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 2024-09-10T08:28:29.930 INFO:teuthology.orchestra.run.smithi089.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T08:28:30.991 INFO:teuthology.orchestra.run.smithi089.stdout:ceph: stdout ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T08:28:30.992 INFO:teuthology.orchestra.run.smithi089.stdout:Ceph version: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T08:28:30.992 INFO:teuthology.orchestra.run.smithi089.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T08:28:31.906 INFO:teuthology.orchestra.run.smithi089.stdout:stat: stdout 167 167 2024-09-10T08:28:31.906 INFO:teuthology.orchestra.run.smithi089.stdout:Creating initial keys... 2024-09-10T08:28:37.240 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph-authtool: stdout AQAwA+BmgAW+EBAAzUaTsfdaZpQQA3LjEI1MMA== 2024-09-10T08:28:44.495 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph-authtool: stdout AQA7A+BmWrHrNRAAo0OXao4TFcU6zXz6+dufKQ== 2024-09-10T08:28:45.551 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph-authtool: stdout AQA8A+BmBBDoNBAAqeFOOAM/mIdmksdEzLDgeg== 2024-09-10T08:28:45.552 INFO:teuthology.orchestra.run.smithi089.stdout:Creating initial monmap... 2024-09-10T08:28:46.557 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T08:28:46.557 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-10T08:28:46.557 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:28:46.558 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T08:28:46.558 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool for a [v2:172.21.15.89:3300,v1:172.21.15.89:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T08:28:46.558 INFO:teuthology.orchestra.run.smithi089.stdout:setting min_mon_release = quincy 2024-09-10T08:28:46.558 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/monmaptool: set fsid to 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:28:46.558 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T08:28:46.558 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:28:46.558 INFO:teuthology.orchestra.run.smithi089.stdout:Creating mon... 2024-09-10T08:28:47.578 INFO:teuthology.orchestra.run.smithi089.stdout:create mon.a on 2024-09-10T08:28:47.895 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T08:28:48.114 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T08:28:48.358 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-91f747bc-6f4e-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-91f747bc-6f4e-11ef-bcea-c7b262605968.target. 2024-09-10T08:28:48.358 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-91f747bc-6f4e-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-91f747bc-6f4e-11ef-bcea-c7b262605968.target. 2024-09-10T08:28:48.718 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mon.a 2024-09-10T08:28:48.719 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Failed to reset failed state of unit ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T08:28:48.992 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-91f747bc-6f4e-11ef-bcea-c7b262605968.target.wants/ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@.service. 2024-09-10T08:28:49.629 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:49 smithi089 podman[30377]: 2024-09-10 08:28:49.410620697 +0000 UTC m=+0.186690909 container create 4fba48dd91cd63711619c5de0fb66fd3cc1d548e71c4dd3d6daef131e7ddc21e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:28:49.629 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:49 smithi089 podman[30377]: 2024-09-10 08:28:49.629979786 +0000 UTC m=+0.406050005 container init 4fba48dd91cd63711619c5de0fb66fd3cc1d548e71c4dd3d6daef131e7ddc21e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T08:28:49.738 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T08:28:49.738 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout disabled 2024-09-10T08:28:49.752 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T08:28:49.752 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout inactive 2024-09-10T08:28:49.752 INFO:teuthology.orchestra.run.smithi089.stdout:firewalld.service is not enabled 2024-09-10T08:28:49.752 INFO:teuthology.orchestra.run.smithi089.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T08:28:49.753 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for mon to start... 2024-09-10T08:28:49.753 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for mon... 2024-09-10T08:28:49.894 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:49 smithi089 podman[30377]: 2024-09-10 08:28:49.636691954 +0000 UTC m=+0.412762167 container start 4fba48dd91cd63711619c5de0fb66fd3cc1d548e71c4dd3d6daef131e7ddc21e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:28:49.894 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:49 smithi089 ceph-mon[30396]: mkfs 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:28:49.894 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:49 smithi089 ceph-mon[30396]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T08:28:49.894 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:49 smithi089 bash[30377]: 4fba48dd91cd63711619c5de0fb66fd3cc1d548e71c4dd3d6daef131e7ddc21e 2024-09-10T08:28:49.894 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:49 smithi089 systemd[1]: Started Ceph mon.a for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:28:50.944 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:50 smithi089 ceph-mon[30396]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T08:28:50.944 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:50 smithi089 ceph-mon[30396]: monmap epoch 1 2024-09-10T08:28:50.944 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:50 smithi089 ceph-mon[30396]: fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:28:50.944 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:50 smithi089 ceph-mon[30396]: last_changed 2024-09-10T08:28:45.922310+0000 2024-09-10T08:28:50.944 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:50 smithi089 ceph-mon[30396]: created 2024-09-10T08:28:45.922310+0000 2024-09-10T08:28:50.944 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:50 smithi089 ceph-mon[30396]: min_mon_release 19 (squid) 2024-09-10T08:28:50.944 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:50 smithi089 ceph-mon[30396]: election_strategy: 1 2024-09-10T08:28:50.944 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:50 smithi089 ceph-mon[30396]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-10T08:28:50.944 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:50 smithi089 ceph-mon[30396]: fsmap 2024-09-10T08:28:50.944 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:50 smithi089 ceph-mon[30396]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T08:28:50.945 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:50 smithi089 ceph-mon[30396]: mgrmap e1: no daemons active 2024-09-10T08:28:50.945 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:50 smithi089 ceph-mon[30396]: from='client.? 172.21.15.89:0/3608572486' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-10T08:28:50.955 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T08:28:50.955 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout id: 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:28:50.955 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T08:28:50.955 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-10T08:28:50.955 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout services: 2024-09-10T08:28:50.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.643079s) 2024-09-10T08:28:50.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T08:28:50.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T08:28:50.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-10T08:28:50.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout data: 2024-09-10T08:28:50.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T08:28:50.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T08:28:50.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T08:28:50.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T08:28:50.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-10T08:28:50.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.262+0000 7fda13e00640 1 Processor -- start 2024-09-10T08:28:50.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.262+0000 7fda13e00640 1 -- start start 2024-09-10T08:28:50.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.262+0000 7fda13e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0c07dde0 0x7fda0c07c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:28:50.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.262+0000 7fda13e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c07c820 con 0x7fda0c07dde0 2024-09-10T08:28:50.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.263+0000 7fda11400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0c07dde0 0x7fda0c07c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:28:50.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.263+0000 7fda11400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0c07dde0 0x7fda0c07c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34178/0 (socket says 172.21.15.89:34178) 2024-09-10T08:28:50.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.263+0000 7fda11400640 1 -- 172.21.15.89:0/4002452130 learned_addr learned my addr 172.21.15.89:0/4002452130 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:28:50.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.264+0000 7fda11400640 1 -- 172.21.15.89:0/4002452130 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c07c9a0 con 0x7fda0c07dde0 2024-09-10T08:28:50.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.266+0000 7fda11400640 1 --2- 172.21.15.89:0/4002452130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0c07dde0 0x7fda0c07c2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fda00009820 tx=0x7fda0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e2feb3bdf62ea8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:28:50.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.266+0000 7fda0be00640 1 -- 172.21.15.89:0/4002452130 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda00037070 con 0x7fda0c07dde0 2024-09-10T08:28:50.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.266+0000 7fda0be00640 1 -- 172.21.15.89:0/4002452130 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda00032440 con 0x7fda0c07dde0 2024-09-10T08:28:50.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.266+0000 7fda0be00640 1 -- 172.21.15.89:0/4002452130 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda00032740 con 0x7fda0c07dde0 2024-09-10T08:28:50.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.267+0000 7fda13e00640 1 -- 172.21.15.89:0/4002452130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0c07dde0 msgr2=0x7fda0c07c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:28:50.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.267+0000 7fda13e00640 1 --2- 172.21.15.89:0/4002452130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0c07dde0 0x7fda0c07c2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fda00009820 tx=0x7fda0002a270 comp rx=0 tx=0).stop 2024-09-10T08:28:50.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.267+0000 7fda13e00640 1 -- 172.21.15.89:0/4002452130 shutdown_connections 2024-09-10T08:28:50.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.267+0000 7fda13e00640 1 --2- 172.21.15.89:0/4002452130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0c07dde0 0x7fda0c07c2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:28:50.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.267+0000 7fda13e00640 1 -- 172.21.15.89:0/4002452130 >> 172.21.15.89:0/4002452130 conn(0x7fda0c102e10 msgr2=0x7fda0c105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:28:50.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.267+0000 7fda13e00640 1 -- 172.21.15.89:0/4002452130 shutdown_connections 2024-09-10T08:28:50.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.267+0000 7fda13e00640 1 -- 172.21.15.89:0/4002452130 wait complete. 2024-09-10T08:28:50.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.268+0000 7fda13e00640 1 Processor -- start 2024-09-10T08:28:50.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.268+0000 7fda13e00640 1 -- start start 2024-09-10T08:28:50.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.268+0000 7fda13e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0c07dde0 0x7fda0c19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:28:50.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.268+0000 7fda13e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c07cf40 con 0x7fda0c07dde0 2024-09-10T08:28:50.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.269+0000 7fda11400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0c07dde0 0x7fda0c19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:28:50.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.269+0000 7fda11400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0c07dde0 0x7fda0c19cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34184/0 (socket says 172.21.15.89:34184) 2024-09-10T08:28:50.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.269+0000 7fda11400640 1 -- 172.21.15.89:0/3608572486 learned_addr learned my addr 172.21.15.89:0/3608572486 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:28:50.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.269+0000 7fda11400640 1 -- 172.21.15.89:0/3608572486 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c19d500 con 0x7fda0c07dde0 2024-09-10T08:28:50.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.269+0000 7fda11400640 1 --2- 172.21.15.89:0/3608572486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0c07dde0 0x7fda0c19cfc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fda000090b0 tx=0x7fda000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:28:50.959 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.269+0000 7fda0a000640 1 -- 172.21.15.89:0/3608572486 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda00041070 con 0x7fda0c07dde0 2024-09-10T08:28:50.959 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.270+0000 7fda0a000640 1 -- 172.21.15.89:0/3608572486 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda0003c7a0 con 0x7fda0c07dde0 2024-09-10T08:28:50.959 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.270+0000 7fda0a000640 1 -- 172.21.15.89:0/3608572486 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda00037040 con 0x7fda0c07dde0 2024-09-10T08:28:50.959 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.270+0000 7fda13e00640 1 -- 172.21.15.89:0/3608572486 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda0c19d790 con 0x7fda0c07dde0 2024-09-10T08:28:50.959 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.270+0000 7fda13e00640 1 -- 172.21.15.89:0/3608572486 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda0c1a0480 con 0x7fda0c07dde0 2024-09-10T08:28:50.959 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.270+0000 7fda0a000640 1 -- 172.21.15.89:0/3608572486 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fda00044070 con 0x7fda0c07dde0 2024-09-10T08:28:50.959 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.270+0000 7fda0a000640 1 -- 172.21.15.89:0/3608572486 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fda0003d3f0 con 0x7fda0c07dde0 2024-09-10T08:28:50.959 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.271+0000 7fda13e00640 1 -- 172.21.15.89:0/3608572486 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda0c07c2e0 con 0x7fda0c07dde0 2024-09-10T08:28:50.959 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.274+0000 7fda0a000640 1 -- 172.21.15.89:0/3608572486 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fda00046450 con 0x7fda0c07dde0 2024-09-10T08:28:50.959 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.337+0000 7fda13e00640 1 -- 172.21.15.89:0/3608572486 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fda0c1a09d0 con 0x7fda0c07dde0 2024-09-10T08:28:50.959 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.337+0000 7fda0a000640 1 -- 172.21.15.89:0/3608572486 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fda00046630 con 0x7fda0c07dde0 2024-09-10T08:28:50.959 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.338+0000 7fda13e00640 1 -- 172.21.15.89:0/3608572486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0c07dde0 msgr2=0x7fda0c19cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:28:50.960 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.338+0000 7fda13e00640 1 --2- 172.21.15.89:0/3608572486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0c07dde0 0x7fda0c19cfc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fda000090b0 tx=0x7fda000046c0 comp rx=0 tx=0).stop 2024-09-10T08:28:50.960 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.338+0000 7fda13e00640 1 -- 172.21.15.89:0/3608572486 shutdown_connections 2024-09-10T08:28:50.960 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.338+0000 7fda13e00640 1 --2- 172.21.15.89:0/3608572486 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fda0c07dde0 0x7fda0c19cfc0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:28:50.960 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.338+0000 7fda13e00640 1 -- 172.21.15.89:0/3608572486 >> 172.21.15.89:0/3608572486 conn(0x7fda0c102e10 msgr2=0x7fda0c1051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:28:50.960 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.338+0000 7fda13e00640 1 -- 172.21.15.89:0/3608572486 shutdown_connections 2024-09-10T08:28:50.960 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:50.338+0000 7fda13e00640 1 -- 172.21.15.89:0/3608572486 wait complete. 2024-09-10T08:28:50.960 INFO:teuthology.orchestra.run.smithi089.stdout:mon is available 2024-09-10T08:28:50.960 INFO:teuthology.orchestra.run.smithi089.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T08:28:51.897 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:51 smithi089 ceph-mon[30396]: from='client.? 172.21.15.89:0/3865460425' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T08:28:51.897 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:51 smithi089 ceph-mon[30396]: from='client.? 172.21.15.89:0/3865460425' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T08:28:52.203 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-10T08:28:52.203 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T08:28:52.203 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout fsid = 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:28:52.203 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.89:3300,v1:172.21.15.89:6789] 2024-09-10T08:28:52.203 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T08:28:52.203 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T08:28:52.204 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T08:28:52.204 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T08:28:52.204 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-10T08:28:52.204 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T08:28:52.204 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T08:28:52.204 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-10T08:28:52.204 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T08:28:52.204 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T08:28:52.204 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T08:28:52.204 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T08:28:52.204 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T08:28:52.204 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T08:28:52.204 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.484+0000 7f5f35e00640 1 Processor -- start 2024-09-10T08:28:52.205 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.485+0000 7f5f35e00640 1 -- start start 2024-09-10T08:28:52.205 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.485+0000 7f5f35e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f30109870 0x7f5f30109c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:28:52.205 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.485+0000 7f5f35e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3010a240 con 0x7f5f30109870 2024-09-10T08:28:52.205 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.486+0000 7f5f2ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f30109870 0x7f5f30109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:28:52.205 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.486+0000 7f5f2ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f30109870 0x7f5f30109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34186/0 (socket says 172.21.15.89:34186) 2024-09-10T08:28:52.205 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.486+0000 7f5f2ea00640 1 -- 172.21.15.89:0/4071789680 learned_addr learned my addr 172.21.15.89:0/4071789680 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:28:52.205 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.486+0000 7f5f2ea00640 1 -- 172.21.15.89:0/4071789680 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3010aa10 con 0x7f5f30109870 2024-09-10T08:28:52.205 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.487+0000 7f5f2ea00640 1 --2- 172.21.15.89:0/4071789680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f30109870 0x7f5f30109c70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009e80 tx=0x7f5f2002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399870d2a7cd545f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:28:52.205 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.487+0000 7f5f2e000640 1 -- 172.21.15.89:0/4071789680 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f20036070 con 0x7f5f30109870 2024-09-10T08:28:52.205 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.487+0000 7f5f2e000640 1 -- 172.21.15.89:0/4071789680 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f20031400 con 0x7f5f30109870 2024-09-10T08:28:52.206 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.487+0000 7f5f2e000640 1 -- 172.21.15.89:0/4071789680 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f20031700 con 0x7f5f30109870 2024-09-10T08:28:52.206 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.488+0000 7f5f35e00640 1 -- 172.21.15.89:0/4071789680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f30109870 msgr2=0x7f5f30109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:28:52.206 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.488+0000 7f5f35e00640 1 --2- 172.21.15.89:0/4071789680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f30109870 0x7f5f30109c70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009e80 tx=0x7f5f2002a7a0 comp rx=0 tx=0).stop 2024-09-10T08:28:52.206 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.488+0000 7f5f35e00640 1 -- 172.21.15.89:0/4071789680 shutdown_connections 2024-09-10T08:28:52.206 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.488+0000 7f5f35e00640 1 --2- 172.21.15.89:0/4071789680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f30109870 0x7f5f30109c70 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:28:52.206 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.488+0000 7f5f35e00640 1 -- 172.21.15.89:0/4071789680 >> 172.21.15.89:0/4071789680 conn(0x7f5f3007cf40 msgr2=0x7f5f3007d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:28:52.206 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.488+0000 7f5f35e00640 1 -- 172.21.15.89:0/4071789680 shutdown_connections 2024-09-10T08:28:52.207 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.488+0000 7f5f35e00640 1 -- 172.21.15.89:0/4071789680 wait complete. 2024-09-10T08:28:52.207 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.489+0000 7f5f35e00640 1 Processor -- start 2024-09-10T08:28:52.207 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.489+0000 7f5f35e00640 1 -- start start 2024-09-10T08:28:52.207 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.489+0000 7f5f35e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f30109870 0x7f5f3019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:28:52.207 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.489+0000 7f5f35e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3010afb0 con 0x7f5f30109870 2024-09-10T08:28:52.207 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.489+0000 7f5f2ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f30109870 0x7f5f3019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:28:52.207 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.489+0000 7f5f2ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f30109870 0x7f5f3019cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34194/0 (socket says 172.21.15.89:34194) 2024-09-10T08:28:52.207 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.489+0000 7f5f2ea00640 1 -- 172.21.15.89:0/3865460425 learned_addr learned my addr 172.21.15.89:0/3865460425 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:28:52.207 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.490+0000 7f5f2ea00640 1 -- 172.21.15.89:0/3865460425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3019d500 con 0x7f5f30109870 2024-09-10T08:28:52.207 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.490+0000 7f5f2ea00640 1 --2- 172.21.15.89:0/3865460425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f30109870 0x7f5f3019cfc0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5f20034040 tx=0x7f5f20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:28:52.207 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.490+0000 7f5f34a00640 1 -- 172.21.15.89:0/3865460425 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f20041070 con 0x7f5f30109870 2024-09-10T08:28:52.207 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.490+0000 7f5f34a00640 1 -- 172.21.15.89:0/3865460425 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2003c650 con 0x7f5f30109870 2024-09-10T08:28:52.208 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.490+0000 7f5f34a00640 1 -- 172.21.15.89:0/3865460425 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f20036040 con 0x7f5f30109870 2024-09-10T08:28:52.208 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.490+0000 7f5f35e00640 1 -- 172.21.15.89:0/3865460425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f3019d790 con 0x7f5f30109870 2024-09-10T08:28:52.208 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.491+0000 7f5f35e00640 1 -- 172.21.15.89:0/3865460425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f301a0480 con 0x7f5f30109870 2024-09-10T08:28:52.208 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.491+0000 7f5f34a00640 1 -- 172.21.15.89:0/3865460425 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5f20044070 con 0x7f5f30109870 2024-09-10T08:28:52.208 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.491+0000 7f5f34a00640 1 -- 172.21.15.89:0/3865460425 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5f20046c10 con 0x7f5f30109870 2024-09-10T08:28:52.208 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.492+0000 7f5f35e00640 1 -- 172.21.15.89:0/3865460425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f3010b620 con 0x7f5f30109870 2024-09-10T08:28:52.208 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.494+0000 7f5f34a00640 1 -- 172.21.15.89:0/3865460425 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f5f2003c950 con 0x7f5f30109870 2024-09-10T08:28:52.208 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.554+0000 7f5f35e00640 1 -- 172.21.15.89:0/3865460425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f5f301a1ba0 con 0x7f5f30109870 2024-09-10T08:28:52.208 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.559+0000 7f5f34a00640 1 -- 172.21.15.89:0/3865460425 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f5f2003cb30 con 0x7f5f30109870 2024-09-10T08:28:52.208 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.560+0000 7f5f34a00640 1 -- 172.21.15.89:0/3865460425 <== mon.0 v2:172.21.15.89:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5f20031400 con 0x7f5f30109870 2024-09-10T08:28:52.208 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.560+0000 7f5f35e00640 1 -- 172.21.15.89:0/3865460425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f30109870 msgr2=0x7f5f3019cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:28:52.208 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.560+0000 7f5f35e00640 1 --2- 172.21.15.89:0/3865460425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f30109870 0x7f5f3019cfc0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5f20034040 tx=0x7f5f20004a40 comp rx=0 tx=0).stop 2024-09-10T08:28:52.209 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.560+0000 7f5f35e00640 1 -- 172.21.15.89:0/3865460425 shutdown_connections 2024-09-10T08:28:52.209 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.560+0000 7f5f35e00640 1 --2- 172.21.15.89:0/3865460425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f30109870 0x7f5f3019cfc0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:28:52.209 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.560+0000 7f5f35e00640 1 -- 172.21.15.89:0/3865460425 >> 172.21.15.89:0/3865460425 conn(0x7f5f3007cf40 msgr2=0x7f5f301912c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:28:52.209 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.560+0000 7f5f35e00640 1 -- 172.21.15.89:0/3865460425 shutdown_connections 2024-09-10T08:28:52.209 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:51.560+0000 7f5f35e00640 1 -- 172.21.15.89:0/3865460425 wait complete. 2024-09-10T08:28:52.209 INFO:teuthology.orchestra.run.smithi089.stdout:Generating new minimal ceph.conf... 2024-09-10T08:28:53.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:52 smithi089 ceph-mon[30396]: from='client.? 172.21.15.89:0/2262486855' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:28:53.379 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.683+0000 7f2c44600640 1 Processor -- start 2024-09-10T08:28:53.379 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.683+0000 7f2c44600640 1 -- start start 2024-09-10T08:28:53.380 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.684+0000 7f2c44600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c3c07de00 0x7f2c3c07c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:28:53.380 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.684+0000 7f2c44600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c07c840 con 0x7f2c3c07de00 2024-09-10T08:28:53.380 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.684+0000 7f2c41c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c3c07de00 0x7f2c3c07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:28:53.380 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.684+0000 7f2c41c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c3c07de00 0x7f2c3c07c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34200/0 (socket says 172.21.15.89:34200) 2024-09-10T08:28:53.380 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.684+0000 7f2c41c00640 1 -- 172.21.15.89:0/3905595086 learned_addr learned my addr 172.21.15.89:0/3905595086 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:28:53.380 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.684+0000 7f2c41c00640 1 -- 172.21.15.89:0/3905595086 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c07c9c0 con 0x7f2c3c07de00 2024-09-10T08:28:53.380 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.685+0000 7f2c41c00640 1 --2- 172.21.15.89:0/3905595086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c3c07de00 0x7f2c3c07c300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009e80 tx=0x7f2c3002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f62eac210f4b05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:28:53.380 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.686+0000 7f2c3be00640 1 -- 172.21.15.89:0/3905595086 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c30036070 con 0x7f2c3c07de00 2024-09-10T08:28:53.386 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.686+0000 7f2c3be00640 1 -- 172.21.15.89:0/3905595086 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2c300318c0 con 0x7f2c3c07de00 2024-09-10T08:28:53.386 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.686+0000 7f2c3be00640 1 -- 172.21.15.89:0/3905595086 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c30007ab0 con 0x7f2c3c07de00 2024-09-10T08:28:53.386 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.686+0000 7f2c44600640 1 -- 172.21.15.89:0/3905595086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c3c07de00 msgr2=0x7f2c3c07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:28:53.386 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.686+0000 7f2c44600640 1 --2- 172.21.15.89:0/3905595086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c3c07de00 0x7f2c3c07c300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009e80 tx=0x7f2c3002a7a0 comp rx=0 tx=0).stop 2024-09-10T08:28:53.387 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.686+0000 7f2c44600640 1 -- 172.21.15.89:0/3905595086 shutdown_connections 2024-09-10T08:28:53.392 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.686+0000 7f2c44600640 1 --2- 172.21.15.89:0/3905595086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c3c07de00 0x7f2c3c07c300 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:28:53.392 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.686+0000 7f2c44600640 1 -- 172.21.15.89:0/3905595086 >> 172.21.15.89:0/3905595086 conn(0x7f2c3c102e10 msgr2=0x7f2c3c105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:28:53.392 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.686+0000 7f2c44600640 1 -- 172.21.15.89:0/3905595086 shutdown_connections 2024-09-10T08:28:53.392 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.686+0000 7f2c44600640 1 -- 172.21.15.89:0/3905595086 wait complete. 2024-09-10T08:28:53.392 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.687+0000 7f2c44600640 1 Processor -- start 2024-09-10T08:28:53.392 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.687+0000 7f2c44600640 1 -- start start 2024-09-10T08:28:53.393 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.688+0000 7f2c44600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c3c07de00 0x7f2c3c19d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:28:53.393 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.688+0000 7f2c44600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c10dc30 con 0x7f2c3c07de00 2024-09-10T08:28:53.393 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.688+0000 7f2c41c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c3c07de00 0x7f2c3c19d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:28:53.393 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.688+0000 7f2c41c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c3c07de00 0x7f2c3c19d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34202/0 (socket says 172.21.15.89:34202) 2024-09-10T08:28:53.393 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.688+0000 7f2c41c00640 1 -- 172.21.15.89:0/2262486855 learned_addr learned my addr 172.21.15.89:0/2262486855 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:28:53.393 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.688+0000 7f2c41c00640 1 -- 172.21.15.89:0/2262486855 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c19dd40 con 0x7f2c3c07de00 2024-09-10T08:28:53.393 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.689+0000 7f2c41c00640 1 --2- 172.21.15.89:0/2262486855 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c3c07de00 0x7f2c3c19d800 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2c3002acc0 tx=0x7f2c30031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:28:53.393 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.689+0000 7f2c3a000640 1 -- 172.21.15.89:0/2262486855 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c30036050 con 0x7f2c3c07de00 2024-09-10T08:28:53.394 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.689+0000 7f2c3a000640 1 -- 172.21.15.89:0/2262486855 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2c30037040 con 0x7f2c3c07de00 2024-09-10T08:28:53.394 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.689+0000 7f2c44600640 1 -- 172.21.15.89:0/2262486855 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c3c19dfd0 con 0x7f2c3c07de00 2024-09-10T08:28:53.394 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.689+0000 7f2c3a000640 1 -- 172.21.15.89:0/2262486855 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c3003d920 con 0x7f2c3c07de00 2024-09-10T08:28:53.394 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.689+0000 7f2c44600640 1 -- 172.21.15.89:0/2262486855 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c3c1a0cc0 con 0x7f2c3c07de00 2024-09-10T08:28:53.394 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.689+0000 7f2c3a000640 1 -- 172.21.15.89:0/2262486855 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2c30044020 con 0x7f2c3c07de00 2024-09-10T08:28:53.394 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.689+0000 7f2c3a000640 1 -- 172.21.15.89:0/2262486855 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2c30042020 con 0x7f2c3c07de00 2024-09-10T08:28:53.394 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.690+0000 7f2c44600640 1 -- 172.21.15.89:0/2262486855 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c3c07c380 con 0x7f2c3c07de00 2024-09-10T08:28:53.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.693+0000 7f2c3a000640 1 -- 172.21.15.89:0/2262486855 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f2c3004b020 con 0x7f2c3c07de00 2024-09-10T08:28:53.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.753+0000 7f2c44600640 1 -- 172.21.15.89:0/2262486855 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f2c3c1a1230 con 0x7f2c3c07de00 2024-09-10T08:28:53.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.753+0000 7f2c3a000640 1 -- 172.21.15.89:0/2262486855 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f2c30046410 con 0x7f2c3c07de00 2024-09-10T08:28:53.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.754+0000 7f2c44600640 1 -- 172.21.15.89:0/2262486855 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c3c07de00 msgr2=0x7f2c3c19d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:28:53.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.754+0000 7f2c44600640 1 --2- 172.21.15.89:0/2262486855 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c3c07de00 0x7f2c3c19d800 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2c3002acc0 tx=0x7f2c30031ca0 comp rx=0 tx=0).stop 2024-09-10T08:28:53.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.754+0000 7f2c44600640 1 -- 172.21.15.89:0/2262486855 shutdown_connections 2024-09-10T08:28:53.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.754+0000 7f2c44600640 1 --2- 172.21.15.89:0/2262486855 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c3c07de00 0x7f2c3c19d800 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:28:53.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.754+0000 7f2c44600640 1 -- 172.21.15.89:0/2262486855 >> 172.21.15.89:0/2262486855 conn(0x7f2c3c102e10 msgr2=0x7f2c3c1039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:28:53.396 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.754+0000 7f2c44600640 1 -- 172.21.15.89:0/2262486855 shutdown_connections 2024-09-10T08:28:53.396 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:52.754+0000 7f2c44600640 1 -- 172.21.15.89:0/2262486855 wait complete. 2024-09-10T08:28:53.396 INFO:teuthology.orchestra.run.smithi089.stdout:Restarting the monitor... 2024-09-10T08:28:53.730 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:53 smithi089 systemd[1]: Stopping Ceph mon.a for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:28:53.986 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:53 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-a[30392]: 2024-09-10T08:28:53.729+0000 7f4de1c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:28:53.986 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:53 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-a[30392]: 2024-09-10T08:28:53.729+0000 7f4de1c00640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T08:28:54.276 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:53 smithi089 podman[30637]: 2024-09-10 08:28:53.987017624 +0000 UTC m=+0.368684284 container died 4fba48dd91cd63711619c5de0fb66fd3cc1d548e71c4dd3d6daef131e7ddc21e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T08:28:54.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:54 smithi089 podman[30637]: 2024-09-10 08:28:54.384506069 +0000 UTC m=+0.766172733 container cleanup 4fba48dd91cd63711619c5de0fb66fd3cc1d548e71c4dd3d6daef131e7ddc21e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T08:28:54.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:54 smithi089 bash[30637]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-a 2024-09-10T08:28:55.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:54 smithi089 podman[30649]: 2024-09-10 08:28:54.66090294 +0000 UTC m=+0.659489487 container remove 4fba48dd91cd63711619c5de0fb66fd3cc1d548e71c4dd3d6daef131e7ddc21e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T08:28:55.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:54 smithi089 systemd[1]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T08:28:55.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:54 smithi089 systemd[1]: Stopped Ceph mon.a for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:28:55.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:54 smithi089 systemd[1]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mon.a.service: Consumed 1.188s CPU time. 2024-09-10T08:28:55.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:54 smithi089 systemd[1]: Starting Ceph mon.a for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:28:55.432 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 podman[30739]: 2024-09-10 08:28:55.087857216 +0000 UTC m=+0.027531732 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:28:55.432 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 podman[30739]: 2024-09-10 08:28:55.195197856 +0000 UTC m=+0.134872410 container create 975872fc80f0340371cdff7e9073f5d8c75ef9b60c8378582473de8625f73fc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T08:28:55.432 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 podman[30739]: 2024-09-10 08:28:55.389063873 +0000 UTC m=+0.328738408 container init 975872fc80f0340371cdff7e9073f5d8c75ef9b60c8378582473de8625f73fc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:28:55.432 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 podman[30739]: 2024-09-10 08:28:55.395861649 +0000 UTC m=+0.335536165 container start 975872fc80f0340371cdff7e9073f5d8c75ef9b60c8378582473de8625f73fc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-10T08:28:55.432 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T08:28:55.432 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable), process ceph-mon, pid 2 2024-09-10T08:28:55.433 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: pidfile_write: ignore empty --pid-file 2024-09-10T08:28:55.433 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: load: jerasure load: lrc 2024-09-10T08:28:55.488 INFO:teuthology.orchestra.run.smithi089.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T08:28:55.727 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: RocksDB version: 7.9.2 2024-09-10T08:28:55.727 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Git sha 0 2024-09-10T08:28:55.727 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Compile date 2024-09-09 20:43:47 2024-09-10T08:28:55.727 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: DB SUMMARY 2024-09-10T08:28:55.727 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: DB Session ID: WJ7TYG7OH7K315LFWGIV 2024-09-10T08:28:55.727 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: CURRENT file: CURRENT 2024-09-10T08:28:55.727 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T08:28:55.727 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-10T08:28:55.727 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T08:28:55.728 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105422 ; 2024-09-10T08:28:55.728 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.error_if_exists: 0 2024-09-10T08:28:55.728 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.create_if_missing: 0 2024-09-10T08:28:55.728 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.paranoid_checks: 1 2024-09-10T08:28:55.728 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T08:28:55.728 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T08:28:55.728 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T08:28:55.728 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.env: 0x56148fe32940 2024-09-10T08:28:55.728 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T08:28:55.728 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.info_log: 0x5614a771d6a0 2024-09-10T08:28:55.728 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T08:28:55.728 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.statistics: (nil) 2024-09-10T08:28:55.728 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.use_fsync: 0 2024-09-10T08:28:55.729 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_log_file_size: 0 2024-09-10T08:28:55.729 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T08:28:55.729 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T08:28:55.729 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T08:28:55.729 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T08:28:55.729 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.allow_fallocate: 1 2024-09-10T08:28:55.729 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T08:28:55.729 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T08:28:55.729 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.use_direct_reads: 0 2024-09-10T08:28:55.729 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T08:28:55.729 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T08:28:55.729 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.db_log_dir: 2024-09-10T08:28:55.729 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.wal_dir: 2024-09-10T08:28:55.730 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T08:28:55.730 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T08:28:55.730 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T08:28:55.730 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T08:28:55.730 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T08:28:55.730 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T08:28:55.730 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T08:28:55.730 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T08:28:55.730 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.write_buffer_manager: 0x5614a772cdc0 2024-09-10T08:28:55.730 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T08:28:55.730 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T08:28:55.730 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T08:28:55.730 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T08:28:55.731 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T08:28:55.731 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T08:28:55.731 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T08:28:55.731 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T08:28:55.731 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.unordered_write: 0 2024-09-10T08:28:55.731 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T08:28:55.731 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T08:28:55.731 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T08:28:55.731 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T08:28:55.731 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.row_cache: None 2024-09-10T08:28:55.731 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.wal_filter: None 2024-09-10T08:28:55.731 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T08:28:55.731 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T08:28:55.732 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.two_write_queues: 0 2024-09-10T08:28:55.732 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T08:28:55.732 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.wal_compression: 0 2024-09-10T08:28:55.732 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.atomic_flush: 0 2024-09-10T08:28:55.732 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T08:28:55.732 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T08:28:55.732 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T08:28:55.732 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.log_readahead_size: 0 2024-09-10T08:28:55.732 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T08:28:55.732 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T08:28:55.732 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T08:28:55.732 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T08:28:55.732 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T08:28:55.732 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T08:28:55.733 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T08:28:55.733 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_background_jobs: 2 2024-09-10T08:28:55.733 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_background_compactions: -1 2024-09-10T08:28:55.733 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_subcompactions: 1 2024-09-10T08:28:55.734 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T08:28:55.734 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T08:28:55.734 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T08:28:55.734 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T08:28:55.734 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T08:28:55.734 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T08:28:55.734 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T08:28:55.735 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T08:28:55.735 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_open_files: -1 2024-09-10T08:28:55.735 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T08:28:55.735 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T08:28:55.735 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T08:28:55.735 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T08:28:55.735 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_background_flushes: -1 2024-09-10T08:28:55.735 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Compression algorithms supported: 2024-09-10T08:28:55.735 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: kZSTD supported: 0 2024-09-10T08:28:55.735 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: kXpressCompression supported: 0 2024-09-10T08:28:55.735 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: kBZip2Compression supported: 0 2024-09-10T08:28:55.735 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T08:28:55.735 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: kLZ4Compression supported: 1 2024-09-10T08:28:55.736 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: kZlibCompression supported: 1 2024-09-10T08:28:55.736 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T08:28:55.736 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: kSnappyCompression supported: 1 2024-09-10T08:28:55.736 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T08:28:55.736 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T08:28:55.736 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-10T08:28:55.736 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T08:28:55.736 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T08:28:55.736 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.merge_operator: 2024-09-10T08:28:55.736 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compaction_filter: None 2024-09-10T08:28:55.736 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T08:28:55.736 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T08:28:55.736 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T08:28:55.737 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T08:28:55.737 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5614a771d300) 2024-09-10T08:28:55.737 INFO:journalctl@ceph.mon.a.smithi089.stdout: cache_index_and_filter_blocks: 1 2024-09-10T08:28:55.737 INFO:journalctl@ceph.mon.a.smithi089.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T08:28:55.737 INFO:journalctl@ceph.mon.a.smithi089.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T08:28:55.737 INFO:journalctl@ceph.mon.a.smithi089.stdout: pin_top_level_index_and_filter: 1 2024-09-10T08:28:55.737 INFO:journalctl@ceph.mon.a.smithi089.stdout: index_type: 0 2024-09-10T08:28:55.737 INFO:journalctl@ceph.mon.a.smithi089.stdout: data_block_index_type: 0 2024-09-10T08:28:55.737 INFO:journalctl@ceph.mon.a.smithi089.stdout: index_shortening: 1 2024-09-10T08:28:55.737 INFO:journalctl@ceph.mon.a.smithi089.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T08:28:55.737 INFO:journalctl@ceph.mon.a.smithi089.stdout: checksum: 4 2024-09-10T08:28:55.737 INFO:journalctl@ceph.mon.a.smithi089.stdout: no_block_cache: 0 2024-09-10T08:28:55.737 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_cache: 0x5614a77111f0 2024-09-10T08:28:55.737 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_cache_name: BinnedLRUCache 2024-09-10T08:28:55.738 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_cache_options: 2024-09-10T08:28:55.738 INFO:journalctl@ceph.mon.a.smithi089.stdout: capacity : 536870912 2024-09-10T08:28:55.738 INFO:journalctl@ceph.mon.a.smithi089.stdout: num_shard_bits : 4 2024-09-10T08:28:55.738 INFO:journalctl@ceph.mon.a.smithi089.stdout: strict_capacity_limit : 0 2024-09-10T08:28:55.738 INFO:journalctl@ceph.mon.a.smithi089.stdout: high_pri_pool_ratio: 0.000 2024-09-10T08:28:55.738 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_cache_compressed: (nil) 2024-09-10T08:28:55.738 INFO:journalctl@ceph.mon.a.smithi089.stdout: persistent_cache: (nil) 2024-09-10T08:28:55.738 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_size: 4096 2024-09-10T08:28:55.738 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_size_deviation: 10 2024-09-10T08:28:55.738 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_restart_interval: 16 2024-09-10T08:28:55.738 INFO:journalctl@ceph.mon.a.smithi089.stdout: index_block_restart_interval: 1 2024-09-10T08:28:55.738 INFO:journalctl@ceph.mon.a.smithi089.stdout: metadata_block_size: 4096 2024-09-10T08:28:55.738 INFO:journalctl@ceph.mon.a.smithi089.stdout: partition_filters: 0 2024-09-10T08:28:55.739 INFO:journalctl@ceph.mon.a.smithi089.stdout: use_delta_encoding: 1 2024-09-10T08:28:55.739 INFO:journalctl@ceph.mon.a.smithi089.stdout: filter_policy: bloomfilter 2024-09-10T08:28:55.739 INFO:journalctl@ceph.mon.a.smithi089.stdout: whole_key_filtering: 1 2024-09-10T08:28:55.739 INFO:journalctl@ceph.mon.a.smithi089.stdout: verify_compression: 0 2024-09-10T08:28:55.739 INFO:journalctl@ceph.mon.a.smithi089.stdout: read_amp_bytes_per_bit: 0 2024-09-10T08:28:55.739 INFO:journalctl@ceph.mon.a.smithi089.stdout: format_version: 5 2024-09-10T08:28:55.739 INFO:journalctl@ceph.mon.a.smithi089.stdout: enable_index_compression: 1 2024-09-10T08:28:55.739 INFO:journalctl@ceph.mon.a.smithi089.stdout: block_align: 0 2024-09-10T08:28:55.739 INFO:journalctl@ceph.mon.a.smithi089.stdout: max_auto_readahead_size: 262144 2024-09-10T08:28:55.739 INFO:journalctl@ceph.mon.a.smithi089.stdout: prepopulate_block_cache: 0 2024-09-10T08:28:55.739 INFO:journalctl@ceph.mon.a.smithi089.stdout: initial_auto_readahead_size: 8192 2024-09-10T08:28:55.739 INFO:journalctl@ceph.mon.a.smithi089.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T08:28:55.739 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T08:28:55.739 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T08:28:55.740 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compression: NoCompression 2024-09-10T08:28:55.740 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T08:28:55.740 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T08:28:55.740 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T08:28:55.740 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.num_levels: 7 2024-09-10T08:28:55.740 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T08:28:55.740 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T08:28:55.740 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T08:28:55.740 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T08:28:55.740 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T08:28:55.740 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T08:28:55.740 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T08:28:55.740 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T08:28:55.741 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T08:28:55.741 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T08:28:55.741 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T08:28:55.741 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T08:28:55.741 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T08:28:55.741 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T08:28:55.741 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T08:28:55.741 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T08:28:55.741 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T08:28:55.741 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T08:28:55.741 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T08:28:55.741 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T08:28:55.741 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T08:28:55.741 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T08:28:55.742 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T08:28:55.742 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T08:28:55.742 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T08:28:55.742 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T08:28:55.742 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T08:28:55.742 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T08:28:55.742 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T08:28:55.742 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T08:28:55.742 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T08:28:55.742 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T08:28:55.742 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T08:28:55.742 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T08:28:55.742 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T08:28:55.743 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T08:28:55.743 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T08:28:55.743 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T08:28:55.743 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T08:28:55.743 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T08:28:55.743 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T08:28:55.743 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T08:28:55.743 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T08:28:55.743 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T08:28:55.743 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T08:28:55.743 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T08:28:55.743 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T08:28:55.743 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T08:28:55.744 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T08:28:55.744 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T08:28:55.744 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T08:28:55.744 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T08:28:55.744 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T08:28:55.744 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T08:28:55.744 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.inplace_update_support: 0 2024-09-10T08:28:55.744 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T08:28:55.744 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T08:28:55.744 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T08:28:55.744 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T08:28:55.744 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.bloom_locality: 0 2024-09-10T08:28:55.744 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.max_successive_merges: 0 2024-09-10T08:28:55.744 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T08:28:55.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T08:28:55.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T08:28:55.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T08:28:55.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.ttl: 2592000 2024-09-10T08:28:55.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T08:28:55.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T08:28:55.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T08:28:55.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.enable_blob_files: false 2024-09-10T08:28:55.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.min_blob_size: 0 2024-09-10T08:28:55.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T08:28:55.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T08:28:55.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T08:28:55.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T08:28:55.746 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T08:28:55.746 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T08:28:55.746 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T08:28:55.746 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T08:28:55.746 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-10T08:28:55.746 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-10T08:28:55.746 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: af66ee13-6239-4f4e-ad33-a5c9e2d4422f 2024-09-10T08:28:55.746 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725956935437155, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-10T08:28:55.746 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-10T08:28:55.746 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725956935439236, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100470, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98364, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14320, "raw_average_key_size": 49, "raw_value_size": 90617, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725956935, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "af66ee13-6239-4f4e-ad33-a5c9e2d4422f", "db_session_id": "WJ7TYG7OH7K315LFWGIV", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-10T08:28:55.746 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725956935439412, "job": 1, "event": "recovery_finished"} 2024-09-10T08:28:55.746 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-10T08:28:55.746 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T08:28:55.747 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5614a7740e00 2024-09-10T08:28:55.747 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: DB pointer 0x5614a783c000 2024-09-10T08:28:55.747 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T08:28:55.747 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T08:28:55.747 INFO:journalctl@ceph.mon.a.smithi089.stdout: ** DB Stats ** 2024-09-10T08:28:55.747 INFO:journalctl@ceph.mon.a.smithi089.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T08:28:55.747 INFO:journalctl@ceph.mon.a.smithi089.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T08:28:55.747 INFO:journalctl@ceph.mon.a.smithi089.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T08:28:55.747 INFO:journalctl@ceph.mon.a.smithi089.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T08:28:55.747 INFO:journalctl@ceph.mon.a.smithi089.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T08:28:55.747 INFO:journalctl@ceph.mon.a.smithi089.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T08:28:55.747 INFO:journalctl@ceph.mon.a.smithi089.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T08:28:55.747 INFO:journalctl@ceph.mon.a.smithi089.stdout: 2024-09-10T08:28:55.747 INFO:journalctl@ceph.mon.a.smithi089.stdout: ** Compaction Stats [default] ** 2024-09-10T08:28:55.748 INFO:journalctl@ceph.mon.a.smithi089.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T08:28:55.748 INFO:journalctl@ceph.mon.a.smithi089.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T08:28:55.748 INFO:journalctl@ceph.mon.a.smithi089.stdout: L0 2/0 99.98 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T08:28:55.748 INFO:journalctl@ceph.mon.a.smithi089.stdout: Sum 2/0 99.98 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T08:28:55.748 INFO:journalctl@ceph.mon.a.smithi089.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T08:28:55.748 INFO:journalctl@ceph.mon.a.smithi089.stdout: 2024-09-10T08:28:55.748 INFO:journalctl@ceph.mon.a.smithi089.stdout: ** Compaction Stats [default] ** 2024-09-10T08:28:55.748 INFO:journalctl@ceph.mon.a.smithi089.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T08:28:55.748 INFO:journalctl@ceph.mon.a.smithi089.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T08:28:55.748 INFO:journalctl@ceph.mon.a.smithi089.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 73.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T08:28:55.748 INFO:journalctl@ceph.mon.a.smithi089.stdout: 2024-09-10T08:28:55.748 INFO:journalctl@ceph.mon.a.smithi089.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T08:28:55.749 INFO:journalctl@ceph.mon.a.smithi089.stdout: 2024-09-10T08:28:55.749 INFO:journalctl@ceph.mon.a.smithi089.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T08:28:55.749 INFO:journalctl@ceph.mon.a.smithi089.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T08:28:55.749 INFO:journalctl@ceph.mon.a.smithi089.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T08:28:55.749 INFO:journalctl@ceph.mon.a.smithi089.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T08:28:55.749 INFO:journalctl@ceph.mon.a.smithi089.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T08:28:55.749 INFO:journalctl@ceph.mon.a.smithi089.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T08:28:55.749 INFO:journalctl@ceph.mon.a.smithi089.stdout: Cumulative compaction: 0.00 GB write, 15.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T08:28:55.749 INFO:journalctl@ceph.mon.a.smithi089.stdout: Interval compaction: 0.00 GB write, 15.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T08:28:55.749 INFO:journalctl@ceph.mon.a.smithi089.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T08:28:55.749 INFO:journalctl@ceph.mon.a.smithi089.stdout: Block cache BinnedLRUCache@0x5614a77111f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-10T08:28:55.749 INFO:journalctl@ceph.mon.a.smithi089.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-10T08:28:55.749 INFO:journalctl@ceph.mon.a.smithi089.stdout: 2024-09-10T08:28:55.750 INFO:journalctl@ceph.mon.a.smithi089.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T08:28:55.750 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: starting mon.a rank 0 at public addrs [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] at bind addrs [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:28:55.750 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: mon.a@-1(???) e1 preinit fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:28:55.750 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: mon.a@-1(???).mds e1 new map 2024-09-10T08:28:55.750 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: mon.a@-1(???).mds e1 print_map 2024-09-10T08:28:55.750 INFO:journalctl@ceph.mon.a.smithi089.stdout: e1 2024-09-10T08:28:55.750 INFO:journalctl@ceph.mon.a.smithi089.stdout: btime 2024-09-10T08:28:49:696758+0000 2024-09-10T08:28:55.750 INFO:journalctl@ceph.mon.a.smithi089.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T08:28:55.750 INFO:journalctl@ceph.mon.a.smithi089.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T08:28:55.750 INFO:journalctl@ceph.mon.a.smithi089.stdout: legacy client fscid: -1 2024-09-10T08:28:55.750 INFO:journalctl@ceph.mon.a.smithi089.stdout: 2024-09-10T08:28:55.750 INFO:journalctl@ceph.mon.a.smithi089.stdout: No filesystems configured 2024-09-10T08:28:55.750 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T08:28:55.750 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T08:28:55.751 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T08:28:55.751 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T08:28:55.751 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T08:28:55.751 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 bash[30739]: 975872fc80f0340371cdff7e9073f5d8c75ef9b60c8378582473de8625f73fc5 2024-09-10T08:28:55.751 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 systemd[1]: Started Ceph mon.a for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:28:55.751 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T08:28:55.751 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: monmap epoch 1 2024-09-10T08:28:55.751 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:28:55.751 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: last_changed 2024-09-10T08:28:45.922310+0000 2024-09-10T08:28:55.751 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: created 2024-09-10T08:28:45.922310+0000 2024-09-10T08:28:55.751 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: min_mon_release 19 (squid) 2024-09-10T08:28:55.751 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: election_strategy: 1 2024-09-10T08:28:55.751 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-10T08:28:55.751 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: fsmap 2024-09-10T08:28:55.752 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T08:28:55.752 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:55 smithi089 ceph-mon[30757]: mgrmap e1: no daemons active 2024-09-10T08:28:56.621 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.950+0000 7f9f3d000640 1 Processor -- start 2024-09-10T08:28:56.621 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.950+0000 7f9f3d000640 1 -- start start 2024-09-10T08:28:56.621 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.950+0000 7f9f3d000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f38107440 0x7f9f38107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:28:56.621 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.950+0000 7f9f3d000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9f38107e10 con 0x7f9f38107440 2024-09-10T08:28:56.621 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.951+0000 7f9f37400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f38107440 0x7f9f38107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:28:56.621 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.951+0000 7f9f37400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f38107440 0x7f9f38107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52066/0 (socket says 172.21.15.89:52066) 2024-09-10T08:28:56.621 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.951+0000 7f9f37400640 1 -- 172.21.15.89:0/1514884060 learned_addr learned my addr 172.21.15.89:0/1514884060 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:28:56.621 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.951+0000 7f9f37400640 1 -- 172.21.15.89:0/1514884060 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f38108640 con 0x7f9f38107440 2024-09-10T08:28:56.621 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.954+0000 7f9f37400640 1 --2- 172.21.15.89:0/1514884060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f38107440 0x7f9f38107840 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9f28009820 tx=0x7f9f2802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bec393b9393b4a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:28:56.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.954+0000 7f9f36000640 1 -- 172.21.15.89:0/1514884060 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f28037070 con 0x7f9f38107440 2024-09-10T08:28:56.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.954+0000 7f9f36000640 1 -- 172.21.15.89:0/1514884060 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9f28032900 con 0x7f9f38107440 2024-09-10T08:28:56.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.954+0000 7f9f36000640 1 -- 172.21.15.89:0/1514884060 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f28030850 con 0x7f9f38107440 2024-09-10T08:28:56.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.955+0000 7f9f3d000640 1 -- 172.21.15.89:0/1514884060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f38107440 msgr2=0x7f9f38107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:28:56.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.955+0000 7f9f3d000640 1 --2- 172.21.15.89:0/1514884060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f38107440 0x7f9f38107840 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9f28009820 tx=0x7f9f2802a270 comp rx=0 tx=0).stop 2024-09-10T08:28:56.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.955+0000 7f9f3d000640 1 -- 172.21.15.89:0/1514884060 shutdown_connections 2024-09-10T08:28:56.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.955+0000 7f9f3d000640 1 --2- 172.21.15.89:0/1514884060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f38107440 0x7f9f38107840 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:28:56.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.955+0000 7f9f3d000640 1 -- 172.21.15.89:0/1514884060 >> 172.21.15.89:0/1514884060 conn(0x7f9f38102c10 msgr2=0x7f9f38105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:28:56.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.955+0000 7f9f3d000640 1 -- 172.21.15.89:0/1514884060 shutdown_connections 2024-09-10T08:28:56.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.955+0000 7f9f3d000640 1 -- 172.21.15.89:0/1514884060 wait complete. 2024-09-10T08:28:56.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.956+0000 7f9f3d000640 1 Processor -- start 2024-09-10T08:28:56.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.956+0000 7f9f3d000640 1 -- start start 2024-09-10T08:28:56.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.956+0000 7f9f3d000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f38107440 0x7f9f3807db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:28:56.623 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.956+0000 7f9f3d000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3810b9f0 con 0x7f9f38107440 2024-09-10T08:28:56.623 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.956+0000 7f9f37400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f38107440 0x7f9f3807db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:28:56.623 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.956+0000 7f9f37400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f38107440 0x7f9f3807db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52076/0 (socket says 172.21.15.89:52076) 2024-09-10T08:28:56.623 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.956+0000 7f9f37400640 1 -- 172.21.15.89:0/1430107150 learned_addr learned my addr 172.21.15.89:0/1430107150 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:28:56.623 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.957+0000 7f9f37400640 1 -- 172.21.15.89:0/1430107150 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3807e0b0 con 0x7f9f38107440 2024-09-10T08:28:56.625 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.957+0000 7f9f37400640 1 --2- 172.21.15.89:0/1430107150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f38107440 0x7f9f3807db70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9f280097f0 tx=0x7f9f28030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:28:56.625 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.957+0000 7f9f2fe00640 1 -- 172.21.15.89:0/1430107150 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f28042070 con 0x7f9f38107440 2024-09-10T08:28:56.625 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.957+0000 7f9f2fe00640 1 -- 172.21.15.89:0/1430107150 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9f28032ca0 con 0x7f9f38107440 2024-09-10T08:28:56.625 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.957+0000 7f9f2fe00640 1 -- 172.21.15.89:0/1430107150 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f28037040 con 0x7f9f38107440 2024-09-10T08:28:56.625 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.957+0000 7f9f3d000640 1 -- 172.21.15.89:0/1430107150 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3807c2a0 con 0x7f9f38107440 2024-09-10T08:28:56.625 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.957+0000 7f9f3d000640 1 -- 172.21.15.89:0/1430107150 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f3807c7e0 con 0x7f9f38107440 2024-09-10T08:28:56.626 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.958+0000 7f9f2fe00640 1 -- 172.21.15.89:0/1430107150 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9f28045070 con 0x7f9f38107440 2024-09-10T08:28:56.626 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.958+0000 7f9f2fe00640 1 -- 172.21.15.89:0/1430107150 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9f28047a30 con 0x7f9f38107440 2024-09-10T08:28:56.626 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.959+0000 7f9f3d000640 1 -- 172.21.15.89:0/1430107150 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f3810bb70 con 0x7f9f38107440 2024-09-10T08:28:56.626 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:55.962+0000 7f9f2fe00640 1 -- 172.21.15.89:0/1430107150 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f9f2803da60 con 0x7f9f38107440 2024-09-10T08:28:56.626 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:56.022+0000 7f9f3d000640 1 -- 172.21.15.89:0/1430107150 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f9f381078c0 con 0x7f9f38107440 2024-09-10T08:28:56.626 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:56.024+0000 7f9f2fe00640 1 -- 172.21.15.89:0/1430107150 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f9f2803d430 con 0x7f9f38107440 2024-09-10T08:28:56.626 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:56.025+0000 7f9f3d000640 1 -- 172.21.15.89:0/1430107150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f38107440 msgr2=0x7f9f3807db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:28:56.626 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:56.025+0000 7f9f3d000640 1 --2- 172.21.15.89:0/1430107150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f38107440 0x7f9f3807db70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9f280097f0 tx=0x7f9f28030ec0 comp rx=0 tx=0).stop 2024-09-10T08:28:56.626 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:56.025+0000 7f9f3d000640 1 -- 172.21.15.89:0/1430107150 shutdown_connections 2024-09-10T08:28:56.626 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:56.025+0000 7f9f3d000640 1 --2- 172.21.15.89:0/1430107150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f38107440 0x7f9f3807db70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:28:56.626 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:56.025+0000 7f9f3d000640 1 -- 172.21.15.89:0/1430107150 >> 172.21.15.89:0/1430107150 conn(0x7f9f38102c10 msgr2=0x7f9f381037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:28:56.626 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:56.025+0000 7f9f3d000640 1 -- 172.21.15.89:0/1430107150 shutdown_connections 2024-09-10T08:28:56.626 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:56.026+0000 7f9f3d000640 1 -- 172.21.15.89:0/1430107150 wait complete. 2024-09-10T08:28:56.626 INFO:teuthology.orchestra.run.smithi089.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T08:28:56.652 INFO:teuthology.orchestra.run.smithi089.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T08:28:56.652 INFO:teuthology.orchestra.run.smithi089.stdout:Creating mgr... 2024-09-10T08:28:56.652 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T08:28:56.653 INFO:teuthology.orchestra.run.smithi089.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T08:28:56.985 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mgr.x 2024-09-10T08:28:56.985 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Failed to reset failed state of unit ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-10T08:28:57.206 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-91f747bc-6f4e-11ef-bcea-c7b262605968.target.wants/ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@.service. 2024-09-10T08:28:57.302 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:57 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1430107150' entity='client.admin' 2024-09-10T08:28:57.303 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:28:57 smithi089 systemd[1]: Starting Ceph mgr.x for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:28:57.803 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:28:57 smithi089 podman[30964]: 2024-09-10 08:28:57.454757805 +0000 UTC m=+0.026518626 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:28:57.803 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:28:57 smithi089 podman[30964]: 2024-09-10 08:28:57.563869802 +0000 UTC m=+0.135630610 container create 68b20d409ffe2c1c22c43cfc133613ccc063a40fc2d9afc32010d1bca28ff56a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:28:57.803 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:28:57 smithi089 podman[30964]: 2024-09-10 08:28:57.766139982 +0000 UTC m=+0.337900798 container init 68b20d409ffe2c1c22c43cfc133613ccc063a40fc2d9afc32010d1bca28ff56a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T08:28:57.803 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:28:57 smithi089 podman[30964]: 2024-09-10 08:28:57.772266264 +0000 UTC m=+0.344027072 container start 68b20d409ffe2c1c22c43cfc133613ccc063a40fc2d9afc32010d1bca28ff56a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD) 2024-09-10T08:28:57.867 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T08:28:57.867 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout disabled 2024-09-10T08:28:57.880 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T08:28:57.880 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout inactive 2024-09-10T08:28:57.880 INFO:teuthology.orchestra.run.smithi089.stdout:firewalld.service is not enabled 2024-09-10T08:28:57.880 INFO:teuthology.orchestra.run.smithi089.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T08:28:57.897 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T08:28:57.897 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout disabled 2024-09-10T08:28:57.911 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T08:28:57.911 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout inactive 2024-09-10T08:28:57.911 INFO:teuthology.orchestra.run.smithi089.stdout:firewalld.service is not enabled 2024-09-10T08:28:57.911 INFO:teuthology.orchestra.run.smithi089.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T08:28:57.911 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for mgr to start... 2024-09-10T08:28:57.912 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for mgr... 2024-09-10T08:28:58.180 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:28:57 smithi089 bash[30964]: 68b20d409ffe2c1c22c43cfc133613ccc063a40fc2d9afc32010d1bca28ff56a 2024-09-10T08:28:58.180 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:28:57 smithi089 systemd[1]: Started Ceph mgr.x for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:28:58.181 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:28:57 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:28:57.941+0000 7fda25e36040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T08:28:58.181 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:28:58 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:28:58.018+0000 7fda25e36040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T08:28:58.447 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:28:58 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:28:58.208+0000 7fda25e36040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T08:28:58.447 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:28:58 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:28:58.293+0000 7fda25e36040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T08:28:58.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:28:58 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1614818919' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T08:28:58.803 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:28:58 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:28:58.479+0000 7fda25e36040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T08:28:58.803 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:28:58 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:28:58.658+0000 7fda25e36040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T08:28:59.132 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:28:59 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:28:59.057+0000 7fda25e36040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T08:28:59.132 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:28:59 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:28:59.132+0000 7fda25e36040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T08:28:59.552 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:28:59 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:28:59.351+0000 7fda25e36040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T08:28:59.661 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-10T08:28:59.663 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2024-09-10T08:28:59.664 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "fsid": "91f747bc-6f4e-11ef-bcea-c7b262605968", 2024-09-10T08:28:59.664 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T08:28:59.664 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T08:28:59.664 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T08:28:59.664 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T08:28:59.664 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:28:59.664 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T08:28:59.664 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T08:28:59.664 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 0 2024-09-10T08:28:59.664 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2024-09-10T08:28:59.664 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T08:28:59.664 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T08:28:59.665 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2024-09-10T08:28:59.665 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-10T08:28:59.665 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T08:28:59.665 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:28:59.665 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T08:28:59.665 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T08:28:59.665 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:28:59.665 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T08:28:59.665 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:28:59.665 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T08:28:59.665 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T08:28:59.665 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T08:28:59.673 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T08:28:59.673 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T08:28:59.673 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T08:28:59.673 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:28:59.673 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T08:28:59.673 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T08:28:59.673 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T08:28:59.673 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T08:28:59.673 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T08:28:59.673 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T08:28:59.673 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T08:28:59.674 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T08:28:59.674 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T08:28:59.674 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:28:59.674 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T08:28:59.674 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:28:59.674 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T08:28:49:696758+0000", 2024-09-10T08:28:59.674 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T08:28:59.674 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T08:28:59.674 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:28:59.674 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T08:28:59.674 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T08:28:59.674 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T08:28:59.674 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T08:28:59.674 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T08:28:59.675 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T08:28:59.675 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T08:28:59.675 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2024-09-10T08:28:59.675 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T08:28:59.675 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:28:59.675 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T08:28:59.675 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:28:59.675 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T08:28:49.697729+0000", 2024-09-10T08:28:59.675 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T08:28:59.675 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:28:59.675 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T08:28:59.675 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2024-09-10T08:28:59.675 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.436+0000 7f58e4800640 1 Processor -- start 2024-09-10T08:28:59.675 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.436+0000 7f58e4800640 1 -- start start 2024-09-10T08:28:59.676 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.436+0000 7f58e4800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58dc109920 0x7f58dc109d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:28:59.676 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.436+0000 7f58e4800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc10a310 con 0x7f58dc109920 2024-09-10T08:28:59.676 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.437+0000 7f58e1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58dc109920 0x7f58dc109d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:28:59.676 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.437+0000 7f58e1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58dc109920 0x7f58dc109d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52100/0 (socket says 172.21.15.89:52100) 2024-09-10T08:28:59.676 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.437+0000 7f58e1e00640 1 -- 172.21.15.89:0/2053404950 learned_addr learned my addr 172.21.15.89:0/2053404950 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:28:59.676 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.438+0000 7f58e1e00640 1 -- 172.21.15.89:0/2053404950 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc10aa10 con 0x7f58dc109920 2024-09-10T08:28:59.676 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.439+0000 7f58e1e00640 1 --2- 172.21.15.89:0/2053404950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58dc109920 0x7f58dc109d40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009820 tx=0x7f58d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62fec278fb29cfc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:28:59.676 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.439+0000 7f58e0a00640 1 -- 172.21.15.89:0/2053404950 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58d0037070 con 0x7f58dc109920 2024-09-10T08:28:59.676 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.439+0000 7f58e0a00640 1 -- 172.21.15.89:0/2053404950 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f58d0032900 con 0x7f58dc109920 2024-09-10T08:28:59.676 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.439+0000 7f58e0a00640 1 -- 172.21.15.89:0/2053404950 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58d0030850 con 0x7f58dc109920 2024-09-10T08:28:59.676 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.440+0000 7f58e4800640 1 -- 172.21.15.89:0/2053404950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58dc109920 msgr2=0x7f58dc109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:28:59.676 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.440+0000 7f58e4800640 1 --2- 172.21.15.89:0/2053404950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58dc109920 0x7f58dc109d40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009820 tx=0x7f58d002a270 comp rx=0 tx=0).stop 2024-09-10T08:28:59.676 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.440+0000 7f58e4800640 1 -- 172.21.15.89:0/2053404950 shutdown_connections 2024-09-10T08:28:59.677 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.440+0000 7f58e4800640 1 --2- 172.21.15.89:0/2053404950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58dc109920 0x7f58dc109d40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:28:59.677 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.440+0000 7f58e4800640 1 -- 172.21.15.89:0/2053404950 >> 172.21.15.89:0/2053404950 conn(0x7f58dc07cf40 msgr2=0x7f58dc07d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:28:59.677 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.440+0000 7f58e4800640 1 -- 172.21.15.89:0/2053404950 shutdown_connections 2024-09-10T08:28:59.677 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.440+0000 7f58e4800640 1 -- 172.21.15.89:0/2053404950 wait complete. 2024-09-10T08:28:59.677 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.442+0000 7f58e4800640 1 Processor -- start 2024-09-10T08:28:59.677 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.442+0000 7f58e4800640 1 -- start start 2024-09-10T08:28:59.677 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.442+0000 7f58e4800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58dc109920 0x7f58dc19d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:28:59.677 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.442+0000 7f58e4800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc10ddc0 con 0x7f58dc109920 2024-09-10T08:28:59.677 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.443+0000 7f58e1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58dc109920 0x7f58dc19d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:28:59.677 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.443+0000 7f58e1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58dc109920 0x7f58dc19d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52112/0 (socket says 172.21.15.89:52112) 2024-09-10T08:28:59.677 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.443+0000 7f58e1e00640 1 -- 172.21.15.89:0/1614818919 learned_addr learned my addr 172.21.15.89:0/1614818919 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:28:59.677 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.443+0000 7f58e1e00640 1 -- 172.21.15.89:0/1614818919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc19dd70 con 0x7f58dc109920 2024-09-10T08:28:59.677 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.444+0000 7f58e1e00640 1 --2- 172.21.15.89:0/1614818919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58dc109920 0x7f58dc19d830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f58d00097f0 tx=0x7f58d0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:28:59.677 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.444+0000 7f58d6a00640 1 -- 172.21.15.89:0/1614818919 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58d0037040 con 0x7f58dc109920 2024-09-10T08:28:59.678 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.444+0000 7f58d6a00640 1 -- 172.21.15.89:0/1614818919 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f58d0040070 con 0x7f58dc109920 2024-09-10T08:28:59.678 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.444+0000 7f58d6a00640 1 -- 172.21.15.89:0/1614818919 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58d003a390 con 0x7f58dc109920 2024-09-10T08:28:59.678 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.444+0000 7f58e4800640 1 -- 172.21.15.89:0/1614818919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58dc19e000 con 0x7f58dc109920 2024-09-10T08:28:59.678 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.444+0000 7f58e4800640 1 -- 172.21.15.89:0/1614818919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58dc1a0cf0 con 0x7f58dc109920 2024-09-10T08:28:59.678 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.445+0000 7f58d6a00640 1 -- 172.21.15.89:0/1614818919 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f58d003a5a0 con 0x7f58dc109920 2024-09-10T08:28:59.678 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.445+0000 7f58d6a00640 1 -- 172.21.15.89:0/1614818919 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f58d0045540 con 0x7f58dc109920 2024-09-10T08:28:59.678 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.445+0000 7f58e4800640 1 -- 172.21.15.89:0/1614818919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58dc10df40 con 0x7f58dc109920 2024-09-10T08:28:59.678 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.448+0000 7f58d6a00640 1 -- 172.21.15.89:0/1614818919 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f58d003b430 con 0x7f58dc109920 2024-09-10T08:28:59.678 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.513+0000 7f58e4800640 1 -- 172.21.15.89:0/1614818919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f58dc1a0ec0 con 0x7f58dc109920 2024-09-10T08:28:59.678 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.514+0000 7f58d6a00640 1 -- 172.21.15.89:0/1614818919 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f58d003b7c0 con 0x7f58dc109920 2024-09-10T08:28:59.678 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.515+0000 7f58e4800640 1 -- 172.21.15.89:0/1614818919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58dc109920 msgr2=0x7f58dc19d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:28:59.678 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.515+0000 7f58e4800640 1 --2- 172.21.15.89:0/1614818919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58dc109920 0x7f58dc19d830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f58d00097f0 tx=0x7f58d0030d80 comp rx=0 tx=0).stop 2024-09-10T08:28:59.678 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.515+0000 7f58e4800640 1 -- 172.21.15.89:0/1614818919 shutdown_connections 2024-09-10T08:28:59.678 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.515+0000 7f58e4800640 1 --2- 172.21.15.89:0/1614818919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58dc109920 0x7f58dc19d830 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:28:59.679 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.515+0000 7f58e4800640 1 -- 172.21.15.89:0/1614818919 >> 172.21.15.89:0/1614818919 conn(0x7f58dc07cf40 msgr2=0x7f58dc107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:28:59.679 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.516+0000 7f58e4800640 1 -- 172.21.15.89:0/1614818919 shutdown_connections 2024-09-10T08:28:59.679 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:28:58.516+0000 7f58e4800640 1 -- 172.21.15.89:0/1614818919 wait complete. 2024-09-10T08:28:59.679 INFO:teuthology.orchestra.run.smithi089.stdout:mgr not available, waiting (1/15)... 2024-09-10T08:29:00.802 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:00 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:00.485+0000 7fda25e36040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T08:29:00.802 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:00 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:00.568+0000 7fda25e36040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T08:29:01.784 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:01 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:01.481+0000 7fda25e36040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T08:29:01.784 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:01 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:01.615+0000 7fda25e36040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T08:29:01.784 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:01 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:01.783+0000 7fda25e36040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T08:29:02.036 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:01 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:01.858+0000 7fda25e36040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T08:29:02.036 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:02 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:02.036+0000 7fda25e36040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T08:29:02.286 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:02 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:02.119+0000 7fda25e36040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T08:29:02.286 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:02 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:02.194+0000 7fda25e36040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T08:29:02.542 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:02 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1654346493' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T08:29:02.542 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:02 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:02.385+0000 7fda25e36040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T08:29:02.542 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:02 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:02.462+0000 7fda25e36040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T08:29:02.542 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:02 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:02.542+0000 7fda25e36040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T08:29:02.802 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:02 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:02.639+0000 7fda25e36040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T08:29:02.802 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:02 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:02.712+0000 7fda25e36040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T08:29:02.802 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:02 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:02.783+0000 7fda25e36040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T08:29:03.007 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-10T08:29:03.007 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2024-09-10T08:29:03.007 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "fsid": "91f747bc-6f4e-11ef-bcea-c7b262605968", 2024-09-10T08:29:03.007 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T08:29:03.007 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T08:29:03.008 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T08:29:03.008 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T08:29:03.008 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:29:03.008 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T08:29:03.008 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T08:29:03.008 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 0 2024-09-10T08:29:03.008 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2024-09-10T08:29:03.008 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T08:29:03.008 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T08:29:03.008 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2024-09-10T08:29:03.009 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-10T08:29:03.009 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T08:29:03.009 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:29:03.009 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T08:29:03.009 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T08:29:03.009 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:29:03.009 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T08:29:03.009 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:29:03.009 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T08:29:03.009 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T08:29:03.009 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T08:29:03.009 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T08:29:03.009 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T08:29:03.010 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T08:29:03.010 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:29:03.010 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T08:29:03.010 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T08:29:03.010 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T08:29:03.010 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T08:29:03.010 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T08:29:03.010 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T08:29:03.010 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T08:29:03.010 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T08:29:03.010 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T08:29:03.010 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:29:03.010 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T08:29:03.011 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:29:03.011 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T08:28:49:696758+0000", 2024-09-10T08:29:03.011 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T08:29:03.011 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T08:29:03.011 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:29:03.011 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T08:29:03.011 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T08:29:03.011 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T08:29:03.011 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T08:29:03.011 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T08:29:03.012 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T08:29:03.012 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T08:29:03.012 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2024-09-10T08:29:03.012 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T08:29:03.012 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:29:03.012 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T08:29:03.013 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:29:03.013 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T08:28:49.697729+0000", 2024-09-10T08:29:03.013 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T08:29:03.013 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:29:03.013 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T08:29:03.013 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2024-09-10T08:29:03.014 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.208+0000 7f447d800640 1 Processor -- start 2024-09-10T08:29:03.014 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.209+0000 7f447d800640 1 -- start start 2024-09-10T08:29:03.014 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.209+0000 7f447d800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4478107440 0x7f4478107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:03.014 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.209+0000 7f447d800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4478107e10 con 0x7f4478107440 2024-09-10T08:29:03.014 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.209+0000 7f4477e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4478107440 0x7f4478107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:03.015 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.209+0000 7f4477e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4478107440 0x7f4478107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52122/0 (socket says 172.21.15.89:52122) 2024-09-10T08:29:03.015 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.209+0000 7f4477e00640 1 -- 172.21.15.89:0/655356951 learned_addr learned my addr 172.21.15.89:0/655356951 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:03.015 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.210+0000 7f4477e00640 1 -- 172.21.15.89:0/655356951 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4478108640 con 0x7f4478107440 2024-09-10T08:29:03.015 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.211+0000 7f4477e00640 1 --2- 172.21.15.89:0/655356951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4478107440 0x7f4478107840 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4468009820 tx=0x7f446802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f0a809b38f0e9e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:03.015 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.211+0000 7f4476a00640 1 -- 172.21.15.89:0/655356951 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4468037070 con 0x7f4478107440 2024-09-10T08:29:03.015 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.211+0000 7f4476a00640 1 -- 172.21.15.89:0/655356951 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4468032900 con 0x7f4478107440 2024-09-10T08:29:03.016 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.211+0000 7f4476a00640 1 -- 172.21.15.89:0/655356951 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4468030850 con 0x7f4478107440 2024-09-10T08:29:03.016 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.212+0000 7f447d800640 1 -- 172.21.15.89:0/655356951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4478107440 msgr2=0x7f4478107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:03.016 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.212+0000 7f447d800640 1 --2- 172.21.15.89:0/655356951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4478107440 0x7f4478107840 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4468009820 tx=0x7f446802a270 comp rx=0 tx=0).stop 2024-09-10T08:29:03.016 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.212+0000 7f447d800640 1 -- 172.21.15.89:0/655356951 shutdown_connections 2024-09-10T08:29:03.016 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.212+0000 7f447d800640 1 --2- 172.21.15.89:0/655356951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4478107440 0x7f4478107840 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:03.016 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.212+0000 7f447d800640 1 -- 172.21.15.89:0/655356951 >> 172.21.15.89:0/655356951 conn(0x7f4478102c10 msgr2=0x7f4478105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:03.017 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.212+0000 7f447d800640 1 -- 172.21.15.89:0/655356951 shutdown_connections 2024-09-10T08:29:03.017 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.212+0000 7f447d800640 1 -- 172.21.15.89:0/655356951 wait complete. 2024-09-10T08:29:03.017 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.213+0000 7f447d800640 1 Processor -- start 2024-09-10T08:29:03.017 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.213+0000 7f447d800640 1 -- start start 2024-09-10T08:29:03.017 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.213+0000 7f447d800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4478107440 0x7f447819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:03.017 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.213+0000 7f447d800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f447810b9f0 con 0x7f4478107440 2024-09-10T08:29:03.017 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.213+0000 7f4477e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4478107440 0x7f447819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:03.018 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.214+0000 7f4477e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4478107440 0x7f447819b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52124/0 (socket says 172.21.15.89:52124) 2024-09-10T08:29:03.018 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.214+0000 7f4477e00640 1 -- 172.21.15.89:0/1654346493 learned_addr learned my addr 172.21.15.89:0/1654346493 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:03.018 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.214+0000 7f4477e00640 1 -- 172.21.15.89:0/1654346493 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447819ba20 con 0x7f4478107440 2024-09-10T08:29:03.018 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.214+0000 7f4477e00640 1 --2- 172.21.15.89:0/1654346493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4478107440 0x7f447819b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4468009940 tx=0x7f44680326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:03.018 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.214+0000 7f4474c00640 1 -- 172.21.15.89:0/1654346493 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4468042070 con 0x7f4478107440 2024-09-10T08:29:03.018 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.214+0000 7f4474c00640 1 -- 172.21.15.89:0/1654346493 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4468032be0 con 0x7f4478107440 2024-09-10T08:29:03.019 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.214+0000 7f4474c00640 1 -- 172.21.15.89:0/1654346493 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4468037040 con 0x7f4478107440 2024-09-10T08:29:03.019 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.214+0000 7f447d800640 1 -- 172.21.15.89:0/1654346493 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447819bcb0 con 0x7f4478107440 2024-09-10T08:29:03.019 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.215+0000 7f447d800640 1 -- 172.21.15.89:0/1654346493 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447819e9a0 con 0x7f4478107440 2024-09-10T08:29:03.019 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.215+0000 7f4474c00640 1 -- 172.21.15.89:0/1654346493 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4468045070 con 0x7f4478107440 2024-09-10T08:29:03.019 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.215+0000 7f4474c00640 1 -- 172.21.15.89:0/1654346493 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4468047a80 con 0x7f4478107440 2024-09-10T08:29:03.020 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.216+0000 7f447d800640 1 -- 172.21.15.89:0/1654346493 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f447810bb70 con 0x7f4478107440 2024-09-10T08:29:03.020 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.218+0000 7f4474c00640 1 -- 172.21.15.89:0/1654346493 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4468032d90 con 0x7f4478107440 2024-09-10T08:29:03.020 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.282+0000 7f447d800640 1 -- 172.21.15.89:0/1654346493 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f447819eb70 con 0x7f4478107440 2024-09-10T08:29:03.020 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.282+0000 7f4474c00640 1 -- 172.21.15.89:0/1654346493 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f446803d430 con 0x7f4478107440 2024-09-10T08:29:03.020 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.283+0000 7f447d800640 1 -- 172.21.15.89:0/1654346493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4478107440 msgr2=0x7f447819b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:03.020 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.283+0000 7f447d800640 1 --2- 172.21.15.89:0/1654346493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4478107440 0x7f447819b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4468009940 tx=0x7f44680326e0 comp rx=0 tx=0).stop 2024-09-10T08:29:03.020 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.283+0000 7f447d800640 1 -- 172.21.15.89:0/1654346493 shutdown_connections 2024-09-10T08:29:03.021 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.283+0000 7f447d800640 1 --2- 172.21.15.89:0/1654346493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4478107440 0x7f447819b4e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:03.021 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.283+0000 7f447d800640 1 -- 172.21.15.89:0/1654346493 >> 172.21.15.89:0/1654346493 conn(0x7f4478102c10 msgr2=0x7f44781037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:03.021 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.284+0000 7f447d800640 1 -- 172.21.15.89:0/1654346493 shutdown_connections 2024-09-10T08:29:03.021 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:02.284+0000 7f447d800640 1 -- 172.21.15.89:0/1654346493 wait complete. 2024-09-10T08:29:03.021 INFO:teuthology.orchestra.run.smithi089.stdout:mgr not available, waiting (2/15)... 2024-09-10T08:29:04.052 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:03 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:03.695+0000 7fda25e36040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T08:29:04.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:03 smithi089 ceph-mon[30757]: Activating manager daemon x 2024-09-10T08:29:04.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:03 smithi089 ceph-mon[30757]: mgrmap e2: x(active, starting, since 0.00421207s) 2024-09-10T08:29:04.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:03 smithi089 ceph-mon[30757]: from='mgr.14100 172.21.15.89:0/2631731724' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T08:29:04.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:03 smithi089 ceph-mon[30757]: from='mgr.14100 172.21.15.89:0/2631731724' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T08:29:04.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:03 smithi089 ceph-mon[30757]: from='mgr.14100 172.21.15.89:0/2631731724' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T08:29:04.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:03 smithi089 ceph-mon[30757]: from='mgr.14100 172.21.15.89:0/2631731724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:29:04.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:03 smithi089 ceph-mon[30757]: from='mgr.14100 172.21.15.89:0/2631731724' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T08:29:04.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:03 smithi089 ceph-mon[30757]: Manager daemon x is now available 2024-09-10T08:29:04.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:03 smithi089 ceph-mon[30757]: from='mgr.14100 172.21.15.89:0/2631731724' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T08:29:04.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:03 smithi089 ceph-mon[30757]: from='mgr.14100 172.21.15.89:0/2631731724' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T08:29:04.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:03 smithi089 ceph-mon[30757]: from='mgr.14100 172.21.15.89:0/2631731724' entity='mgr.x' 2024-09-10T08:29:04.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:03 smithi089 ceph-mon[30757]: from='mgr.14100 172.21.15.89:0/2631731724' entity='mgr.x' 2024-09-10T08:29:04.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:03 smithi089 ceph-mon[30757]: from='mgr.14100 172.21.15.89:0/2631731724' entity='mgr.x' 2024-09-10T08:29:06.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:05 smithi089 ceph-mon[30757]: mgrmap e3: x(active, since 1.00938s) 2024-09-10T08:29:06.510 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-10T08:29:06.510 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2024-09-10T08:29:06.510 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "fsid": "91f747bc-6f4e-11ef-bcea-c7b262605968", 2024-09-10T08:29:06.511 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T08:29:06.511 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T08:29:06.511 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T08:29:06.511 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T08:29:06.511 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:29:06.511 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T08:29:06.511 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T08:29:06.511 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 0 2024-09-10T08:29:06.511 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2024-09-10T08:29:06.511 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T08:29:06.511 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T08:29:06.511 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2024-09-10T08:29:06.511 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-10T08:29:06.512 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T08:29:06.512 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:29:06.512 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T08:29:06.512 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T08:29:06.512 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:29:06.512 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T08:29:06.512 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:29:06.512 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T08:29:06.512 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T08:29:06.512 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T08:29:06.512 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T08:29:06.512 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T08:29:06.512 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T08:29:06.512 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:29:06.513 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T08:29:06.513 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T08:29:06.513 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T08:29:06.513 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T08:29:06.513 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T08:29:06.513 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T08:29:06.513 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T08:29:06.513 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T08:29:06.513 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T08:29:06.513 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:29:06.513 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T08:29:06.513 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:29:06.513 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T08:28:49:696758+0000", 2024-09-10T08:29:06.513 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T08:29:06.514 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T08:29:06.514 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:29:06.514 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T08:29:06.514 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T08:29:06.514 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T08:29:06.514 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T08:29:06.514 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T08:29:06.514 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T08:29:06.514 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T08:29:06.514 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ], 2024-09-10T08:29:06.514 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T08:29:06.514 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:29:06.514 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T08:29:06.515 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:29:06.515 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T08:28:49.697729+0000", 2024-09-10T08:29:06.515 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T08:29:06.515 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:29:06.515 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T08:29:06.515 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2024-09-10T08:29:06.515 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.516+0000 7f93ba600640 1 Processor -- start 2024-09-10T08:29:06.515 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.517+0000 7f93ba600640 1 -- start start 2024-09-10T08:29:06.515 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.517+0000 7f93ba600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93b4105370 0x7f93b4105770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:06.515 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.517+0000 7f93ba600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f93b4105cb0 con 0x7f93b4105370 2024-09-10T08:29:06.515 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.517+0000 7f93b3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93b4105370 0x7f93b4105770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:06.515 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.517+0000 7f93b3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93b4105370 0x7f93b4105770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35212/0 (socket says 172.21.15.89:35212) 2024-09-10T08:29:06.515 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.517+0000 7f93b3400640 1 -- 172.21.15.89:0/2770554023 learned_addr learned my addr 172.21.15.89:0/2770554023 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:06.515 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.518+0000 7f93b3400640 1 -- 172.21.15.89:0/2770554023 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b4105e30 con 0x7f93b4105370 2024-09-10T08:29:06.516 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.519+0000 7f93b3400640 1 --2- 172.21.15.89:0/2770554023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93b4105370 0x7f93b4105770 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f93a4009820 tx=0x7f93a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2099911481a2122a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:06.516 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.519+0000 7f93b2a00640 1 -- 172.21.15.89:0/2770554023 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93a4037070 con 0x7f93b4105370 2024-09-10T08:29:06.516 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.519+0000 7f93b2a00640 1 -- 172.21.15.89:0/2770554023 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f93a4032900 con 0x7f93b4105370 2024-09-10T08:29:06.516 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.519+0000 7f93b2a00640 1 -- 172.21.15.89:0/2770554023 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93a4030850 con 0x7f93b4105370 2024-09-10T08:29:06.516 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.520+0000 7f93ba600640 1 -- 172.21.15.89:0/2770554023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93b4105370 msgr2=0x7f93b4105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:06.516 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.520+0000 7f93ba600640 1 --2- 172.21.15.89:0/2770554023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93b4105370 0x7f93b4105770 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f93a4009820 tx=0x7f93a402a270 comp rx=0 tx=0).stop 2024-09-10T08:29:06.516 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.520+0000 7f93ba600640 1 -- 172.21.15.89:0/2770554023 shutdown_connections 2024-09-10T08:29:06.516 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.520+0000 7f93ba600640 1 --2- 172.21.15.89:0/2770554023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93b4105370 0x7f93b4105770 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:06.516 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.520+0000 7f93ba600640 1 -- 172.21.15.89:0/2770554023 >> 172.21.15.89:0/2770554023 conn(0x7f93b4100fa0 msgr2=0x7f93b4103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:06.516 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.520+0000 7f93ba600640 1 -- 172.21.15.89:0/2770554023 shutdown_connections 2024-09-10T08:29:06.516 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.520+0000 7f93ba600640 1 -- 172.21.15.89:0/2770554023 wait complete. 2024-09-10T08:29:06.516 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.521+0000 7f93ba600640 1 Processor -- start 2024-09-10T08:29:06.516 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.521+0000 7f93ba600640 1 -- start start 2024-09-10T08:29:06.517 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.521+0000 7f93ba600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93b4105370 0x7f93b419d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:06.517 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.521+0000 7f93ba600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f93b410ba20 con 0x7f93b4105370 2024-09-10T08:29:06.517 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.521+0000 7f93b3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93b4105370 0x7f93b419d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:06.517 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.521+0000 7f93b3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93b4105370 0x7f93b419d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35226/0 (socket says 172.21.15.89:35226) 2024-09-10T08:29:06.517 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.521+0000 7f93b3400640 1 -- 172.21.15.89:0/825113168 learned_addr learned my addr 172.21.15.89:0/825113168 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:06.517 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.522+0000 7f93b3400640 1 -- 172.21.15.89:0/825113168 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b419ddc0 con 0x7f93b4105370 2024-09-10T08:29:06.517 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.522+0000 7f93b3400640 1 --2- 172.21.15.89:0/825113168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93b4105370 0x7f93b419d880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f93a4031000 tx=0x7f93a40326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:06.517 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.522+0000 7f93b0c00640 1 -- 172.21.15.89:0/825113168 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93a4042070 con 0x7f93b4105370 2024-09-10T08:29:06.517 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.522+0000 7f93ba600640 1 -- 172.21.15.89:0/825113168 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b419e050 con 0x7f93b4105370 2024-09-10T08:29:06.517 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.522+0000 7f93b0c00640 1 -- 172.21.15.89:0/825113168 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f93a4032be0 con 0x7f93b4105370 2024-09-10T08:29:06.517 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.522+0000 7f93b0c00640 1 -- 172.21.15.89:0/825113168 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93a4037040 con 0x7f93b4105370 2024-09-10T08:29:06.517 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.522+0000 7f93ba600640 1 -- 172.21.15.89:0/825113168 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b41a0d40 con 0x7f93b4105370 2024-09-10T08:29:06.517 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.523+0000 7f93b0c00640 1 -- 172.21.15.89:0/825113168 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7f93a4045070 con 0x7f93b4105370 2024-09-10T08:29:06.518 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.523+0000 7f93ba600640 1 -- 172.21.15.89:0/825113168 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b41057f0 con 0x7f93b4105370 2024-09-10T08:29:06.518 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.524+0000 7f93b0c00640 1 --2- 172.21.15.89:0/825113168 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f9398038e10 0x7f939803b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:06.518 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.524+0000 7f93b0c00640 1 -- 172.21.15.89:0/825113168 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f93a406bd30 con 0x7f93b4105370 2024-09-10T08:29:06.518 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.524+0000 7f93aaa00640 1 --2- 172.21.15.89:0/825113168 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f9398038e10 0x7f939803b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:06.518 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.525+0000 7f93aaa00640 1 --2- 172.21.15.89:0/825113168 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f9398038e10 0x7f939803b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f939c00ad30 tx=0x7f939c0093f0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:06.518 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.530+0000 7f93b0c00640 1 -- 172.21.15.89:0/825113168 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93a402f2a0 con 0x7f93b4105370 2024-09-10T08:29:06.518 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.708+0000 7f93b0c00640 1 -- 172.21.15.89:0/825113168 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f93a403d430 con 0x7f93b4105370 2024-09-10T08:29:06.518 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.776+0000 7f93ba600640 1 -- 172.21.15.89:0/825113168 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f93b41a0f10 con 0x7f93b4105370 2024-09-10T08:29:06.518 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.777+0000 7f93b0c00640 1 -- 172.21.15.89:0/825113168 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f93a40392e0 con 0x7f93b4105370 2024-09-10T08:29:06.518 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.779+0000 7f93ba600640 1 -- 172.21.15.89:0/825113168 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f9398038e10 msgr2=0x7f939803b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:06.518 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.779+0000 7f93ba600640 1 --2- 172.21.15.89:0/825113168 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f9398038e10 0x7f939803b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f939c00ad30 tx=0x7f939c0093f0 comp rx=0 tx=0).stop 2024-09-10T08:29:06.518 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.779+0000 7f93ba600640 1 -- 172.21.15.89:0/825113168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93b4105370 msgr2=0x7f93b419d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:06.518 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.779+0000 7f93ba600640 1 --2- 172.21.15.89:0/825113168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93b4105370 0x7f93b419d880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f93a4031000 tx=0x7f93a40326e0 comp rx=0 tx=0).stop 2024-09-10T08:29:06.519 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.780+0000 7f93ba600640 1 -- 172.21.15.89:0/825113168 shutdown_connections 2024-09-10T08:29:06.519 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.780+0000 7f93ba600640 1 --2- 172.21.15.89:0/825113168 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f9398038e10 0x7f939803b2d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:06.519 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.780+0000 7f93ba600640 1 --2- 172.21.15.89:0/825113168 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f93b4105370 0x7f93b419d880 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:06.519 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.780+0000 7f93ba600640 1 -- 172.21.15.89:0/825113168 >> 172.21.15.89:0/825113168 conn(0x7f93b4100fa0 msgr2=0x7f93b4101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:06.519 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.780+0000 7f93ba600640 1 -- 172.21.15.89:0/825113168 shutdown_connections 2024-09-10T08:29:06.519 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:05.780+0000 7f93ba600640 1 -- 172.21.15.89:0/825113168 wait complete. 2024-09-10T08:29:06.519 INFO:teuthology.orchestra.run.smithi089.stdout:mgr is available 2024-09-10T08:29:06.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:06 smithi089 ceph-mon[30757]: mgrmap e4: x(active, since 2s) 2024-09-10T08:29:06.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:06 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/825113168' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T08:29:07.958 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:07 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2891131340' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T08:29:07.971 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-10T08:29:07.971 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T08:29:07.971 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout fsid = 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:29:07.971 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.89:3300,v1:172.21.15.89:6789] 2024-09-10T08:29:07.972 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T08:29:07.972 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T08:29:07.972 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T08:29:07.972 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T08:29:07.972 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-10T08:29:07.972 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T08:29:07.972 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T08:29:07.972 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 2024-09-10T08:29:07.972 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T08:29:07.972 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T08:29:07.972 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T08:29:07.972 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T08:29:07.973 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T08:29:07.973 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T08:29:07.973 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.053+0000 7fd6c1c00640 1 Processor -- start 2024-09-10T08:29:07.973 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.053+0000 7fd6c1c00640 1 -- start start 2024-09-10T08:29:07.973 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.053+0000 7fd6c1c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc107440 0x7fd6bc107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:07.973 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.053+0000 7fd6c1c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc107e10 con 0x7fd6bc107440 2024-09-10T08:29:07.973 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.054+0000 7fd6bbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc107440 0x7fd6bc107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:07.973 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.054+0000 7fd6bbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc107440 0x7fd6bc107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35234/0 (socket says 172.21.15.89:35234) 2024-09-10T08:29:07.973 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.054+0000 7fd6bbe00640 1 -- 172.21.15.89:0/349741475 learned_addr learned my addr 172.21.15.89:0/349741475 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:07.973 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.054+0000 7fd6bbe00640 1 -- 172.21.15.89:0/349741475 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6bc108640 con 0x7fd6bc107440 2024-09-10T08:29:07.973 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.055+0000 7fd6bbe00640 1 --2- 172.21.15.89:0/349741475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc107440 0x7fd6bc107840 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac009820 tx=0x7fd6ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6985ce8bfc1635 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:07.973 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.055+0000 7fd6bb400640 1 -- 172.21.15.89:0/349741475 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6ac037070 con 0x7fd6bc107440 2024-09-10T08:29:07.974 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.055+0000 7fd6bb400640 1 -- 172.21.15.89:0/349741475 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd6ac032900 con 0x7fd6bc107440 2024-09-10T08:29:07.974 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.055+0000 7fd6bb400640 1 -- 172.21.15.89:0/349741475 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6ac030850 con 0x7fd6bc107440 2024-09-10T08:29:07.974 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.056+0000 7fd6c1c00640 1 -- 172.21.15.89:0/349741475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc107440 msgr2=0x7fd6bc107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:07.974 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.056+0000 7fd6c1c00640 1 --2- 172.21.15.89:0/349741475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc107440 0x7fd6bc107840 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac009820 tx=0x7fd6ac02a270 comp rx=0 tx=0).stop 2024-09-10T08:29:07.974 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.056+0000 7fd6c1c00640 1 -- 172.21.15.89:0/349741475 shutdown_connections 2024-09-10T08:29:07.974 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.056+0000 7fd6c1c00640 1 --2- 172.21.15.89:0/349741475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc107440 0x7fd6bc107840 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:07.974 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.056+0000 7fd6c1c00640 1 -- 172.21.15.89:0/349741475 >> 172.21.15.89:0/349741475 conn(0x7fd6bc102c10 msgr2=0x7fd6bc105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:07.974 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.056+0000 7fd6c1c00640 1 -- 172.21.15.89:0/349741475 shutdown_connections 2024-09-10T08:29:07.974 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.056+0000 7fd6c1c00640 1 -- 172.21.15.89:0/349741475 wait complete. 2024-09-10T08:29:07.974 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.057+0000 7fd6c1c00640 1 Processor -- start 2024-09-10T08:29:07.974 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.057+0000 7fd6c1c00640 1 -- start start 2024-09-10T08:29:07.974 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.057+0000 7fd6c1c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc107440 0x7fd6bc19b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:07.974 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.057+0000 7fd6c1c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc10b9f0 con 0x7fd6bc107440 2024-09-10T08:29:07.974 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.058+0000 7fd6bbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc107440 0x7fd6bc19b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:07.975 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.058+0000 7fd6bbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc107440 0x7fd6bc19b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35240/0 (socket says 172.21.15.89:35240) 2024-09-10T08:29:07.975 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.058+0000 7fd6bbe00640 1 -- 172.21.15.89:0/2891131340 learned_addr learned my addr 172.21.15.89:0/2891131340 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:07.975 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.058+0000 7fd6bbe00640 1 -- 172.21.15.89:0/2891131340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6bc19ba20 con 0x7fd6bc107440 2024-09-10T08:29:07.975 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.058+0000 7fd6bbe00640 1 --2- 172.21.15.89:0/2891131340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc107440 0x7fd6bc19b4e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0097f0 tx=0x7fd6ac030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:07.975 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.059+0000 7fd6b9600640 1 -- 172.21.15.89:0/2891131340 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6ac042070 con 0x7fd6bc107440 2024-09-10T08:29:07.975 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.059+0000 7fd6b9600640 1 -- 172.21.15.89:0/2891131340 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd6ac032ca0 con 0x7fd6bc107440 2024-09-10T08:29:07.975 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.059+0000 7fd6b9600640 1 -- 172.21.15.89:0/2891131340 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6ac037040 con 0x7fd6bc107440 2024-09-10T08:29:07.975 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.059+0000 7fd6c1c00640 1 -- 172.21.15.89:0/2891131340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6bc19bcb0 con 0x7fd6bc107440 2024-09-10T08:29:07.975 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.059+0000 7fd6c1c00640 1 -- 172.21.15.89:0/2891131340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6bc19e9a0 con 0x7fd6bc107440 2024-09-10T08:29:07.975 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.060+0000 7fd6b9600640 1 -- 172.21.15.89:0/2891131340 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fd6ac045070 con 0x7fd6bc107440 2024-09-10T08:29:07.975 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.060+0000 7fd6c1c00640 1 -- 172.21.15.89:0/2891131340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6bc10bb70 con 0x7fd6bc107440 2024-09-10T08:29:07.975 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.060+0000 7fd6b9600640 1 --2- 172.21.15.89:0/2891131340 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7fd6a0038f70 0x7fd6a003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:07.975 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.060+0000 7fd6b9600640 1 -- 172.21.15.89:0/2891131340 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd6ac06be50 con 0x7fd6bc107440 2024-09-10T08:29:07.976 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.060+0000 7fd6b3400640 1 --2- 172.21.15.89:0/2891131340 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7fd6a0038f70 0x7fd6a003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:07.976 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.061+0000 7fd6b3400640 1 --2- 172.21.15.89:0/2891131340 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7fd6a0038f70 0x7fd6a003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40098c0 tx=0x7fd6a4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:07.976 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.065+0000 7fd6b9600640 1 -- 172.21.15.89:0/2891131340 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd6ac04b170 con 0x7fd6bc107440 2024-09-10T08:29:07.976 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.238+0000 7fd6c1c00640 1 -- 172.21.15.89:0/2891131340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd6bc1078c0 con 0x7fd6bc107440 2024-09-10T08:29:07.976 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.239+0000 7fd6b9600640 1 -- 172.21.15.89:0/2891131340 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7fd6ac06b870 con 0x7fd6bc107440 2024-09-10T08:29:07.976 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.242+0000 7fd6c1c00640 1 -- 172.21.15.89:0/2891131340 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7fd6a0038f70 msgr2=0x7fd6a003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:07.976 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.242+0000 7fd6c1c00640 1 --2- 172.21.15.89:0/2891131340 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7fd6a0038f70 0x7fd6a003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40098c0 tx=0x7fd6a4006db0 comp rx=0 tx=0).stop 2024-09-10T08:29:07.976 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.242+0000 7fd6c1c00640 1 -- 172.21.15.89:0/2891131340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc107440 msgr2=0x7fd6bc19b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:07.976 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.242+0000 7fd6c1c00640 1 --2- 172.21.15.89:0/2891131340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc107440 0x7fd6bc19b4e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0097f0 tx=0x7fd6ac030ec0 comp rx=0 tx=0).stop 2024-09-10T08:29:07.976 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.243+0000 7fd6c1c00640 1 -- 172.21.15.89:0/2891131340 shutdown_connections 2024-09-10T08:29:07.976 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.243+0000 7fd6c1c00640 1 --2- 172.21.15.89:0/2891131340 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7fd6a0038f70 0x7fd6a003b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:07.976 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.243+0000 7fd6c1c00640 1 --2- 172.21.15.89:0/2891131340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6bc107440 0x7fd6bc19b4e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:07.976 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.243+0000 7fd6c1c00640 1 -- 172.21.15.89:0/2891131340 >> 172.21.15.89:0/2891131340 conn(0x7fd6bc102c10 msgr2=0x7fd6bc1037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:07.977 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.243+0000 7fd6c1c00640 1 -- 172.21.15.89:0/2891131340 shutdown_connections 2024-09-10T08:29:07.977 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:07.243+0000 7fd6c1c00640 1 -- 172.21.15.89:0/2891131340 wait complete. 2024-09-10T08:29:07.977 INFO:teuthology.orchestra.run.smithi089.stdout:Enabling cephadm module... 2024-09-10T08:29:08.998 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:08 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/4097086248' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-10T08:29:08.999 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:08 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: ignoring --setuser ceph since I am not root 2024-09-10T08:29:08.999 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:08 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: ignoring --setgroup ceph since I am not root 2024-09-10T08:29:08.999 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:08 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:08.928+0000 7ff28042d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T08:29:08.999 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:08 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:08.999+0000 7ff28042d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T08:29:09.265 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:09 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:09.185+0000 7ff28042d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T08:29:09.265 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:09 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:09.266+0000 7ff28042d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T08:29:09.429 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.454+0000 7f5f64c00640 1 Processor -- start 2024-09-10T08:29:09.430 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.455+0000 7f5f64c00640 1 -- start start 2024-09-10T08:29:09.430 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.455+0000 7f5f64c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f60105ec0 0x7f5f601062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:09.430 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.455+0000 7f5f64c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6007be30 con 0x7f5f60105ec0 2024-09-10T08:29:09.430 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.455+0000 7f5f5de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f60105ec0 0x7f5f601062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:09.430 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.455+0000 7f5f5de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f60105ec0 0x7f5f601062c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35254/0 (socket says 172.21.15.89:35254) 2024-09-10T08:29:09.430 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.455+0000 7f5f5de00640 1 -- 172.21.15.89:0/3653588896 learned_addr learned my addr 172.21.15.89:0/3653588896 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:09.430 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.456+0000 7f5f5de00640 1 -- 172.21.15.89:0/3653588896 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f60106800 con 0x7f5f60105ec0 2024-09-10T08:29:09.430 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.457+0000 7f5f5de00640 1 --2- 172.21.15.89:0/3653588896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f60105ec0 0x7f5f601062c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009820 tx=0x7f5f5002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e397de48833a99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:09.430 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.457+0000 7f5f5d400640 1 -- 172.21.15.89:0/3653588896 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f50037070 con 0x7f5f60105ec0 2024-09-10T08:29:09.430 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.457+0000 7f5f5d400640 1 -- 172.21.15.89:0/3653588896 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5f50032900 con 0x7f5f60105ec0 2024-09-10T08:29:09.430 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.457+0000 7f5f5d400640 1 -- 172.21.15.89:0/3653588896 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f50030850 con 0x7f5f60105ec0 2024-09-10T08:29:09.430 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.457+0000 7f5f64c00640 1 -- 172.21.15.89:0/3653588896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f60105ec0 msgr2=0x7f5f601062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:09.431 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.457+0000 7f5f64c00640 1 --2- 172.21.15.89:0/3653588896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f60105ec0 0x7f5f601062c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009820 tx=0x7f5f5002a270 comp rx=0 tx=0).stop 2024-09-10T08:29:09.431 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.457+0000 7f5f64c00640 1 -- 172.21.15.89:0/3653588896 shutdown_connections 2024-09-10T08:29:09.431 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.458+0000 7f5f64c00640 1 --2- 172.21.15.89:0/3653588896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f60105ec0 0x7f5f601062c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:09.431 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.458+0000 7f5f64c00640 1 -- 172.21.15.89:0/3653588896 >> 172.21.15.89:0/3653588896 conn(0x7f5f60101d50 msgr2=0x7f5f60104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:09.431 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.458+0000 7f5f64c00640 1 -- 172.21.15.89:0/3653588896 shutdown_connections 2024-09-10T08:29:09.431 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.458+0000 7f5f64c00640 1 -- 172.21.15.89:0/3653588896 wait complete. 2024-09-10T08:29:09.431 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.458+0000 7f5f64c00640 1 Processor -- start 2024-09-10T08:29:09.431 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.459+0000 7f5f64c00640 1 -- start start 2024-09-10T08:29:09.431 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.459+0000 7f5f64c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f60105ec0 0x7f5f6019d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:09.431 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.459+0000 7f5f64c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5f60107900 con 0x7f5f60105ec0 2024-09-10T08:29:09.431 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.459+0000 7f5f5de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f60105ec0 0x7f5f6019d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:09.431 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.459+0000 7f5f5de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f60105ec0 0x7f5f6019d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35264/0 (socket says 172.21.15.89:35264) 2024-09-10T08:29:09.431 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.459+0000 7f5f5de00640 1 -- 172.21.15.89:0/4097086248 learned_addr learned my addr 172.21.15.89:0/4097086248 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:09.432 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.459+0000 7f5f5de00640 1 -- 172.21.15.89:0/4097086248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6019dce0 con 0x7f5f60105ec0 2024-09-10T08:29:09.432 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.460+0000 7f5f5de00640 1 --2- 172.21.15.89:0/4097086248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f60105ec0 0x7f5f6019d7a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5f500097f0 tx=0x7f5f50030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:09.432 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.460+0000 7f5f57400640 1 -- 172.21.15.89:0/4097086248 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f50037040 con 0x7f5f60105ec0 2024-09-10T08:29:09.432 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.460+0000 7f5f57400640 1 -- 172.21.15.89:0/4097086248 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5f50040070 con 0x7f5f60105ec0 2024-09-10T08:29:09.432 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.460+0000 7f5f64c00640 1 -- 172.21.15.89:0/4097086248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6019df70 con 0x7f5f60105ec0 2024-09-10T08:29:09.432 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.460+0000 7f5f57400640 1 -- 172.21.15.89:0/4097086248 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f5003a390 con 0x7f5f60105ec0 2024-09-10T08:29:09.432 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.460+0000 7f5f64c00640 1 -- 172.21.15.89:0/4097086248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f601a0c60 con 0x7f5f60105ec0 2024-09-10T08:29:09.432 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.461+0000 7f5f57400640 1 -- 172.21.15.89:0/4097086248 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f5f5003a5e0 con 0x7f5f60105ec0 2024-09-10T08:29:09.432 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.461+0000 7f5f64c00640 1 -- 172.21.15.89:0/4097086248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f601062c0 con 0x7f5f60105ec0 2024-09-10T08:29:09.432 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.462+0000 7f5f57400640 1 --2- 172.21.15.89:0/4097086248 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f5f44038f70 0x7f5f4403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:09.432 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.462+0000 7f5f57400640 1 -- 172.21.15.89:0/4097086248 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5f5006c0e0 con 0x7f5f60105ec0 2024-09-10T08:29:09.432 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.462+0000 7f5f55400640 1 --2- 172.21.15.89:0/4097086248 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f5f44038f70 0x7f5f4403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:09.432 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.463+0000 7f5f55400640 1 --2- 172.21.15.89:0/4097086248 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f5f44038f70 0x7f5f4403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5f480098c0 tx=0x7f5f48006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:09.433 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.467+0000 7f5f57400640 1 -- 172.21.15.89:0/4097086248 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5f5003a980 con 0x7f5f60105ec0 2024-09-10T08:29:09.433 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.685+0000 7f5f64c00640 1 -- 172.21.15.89:0/4097086248 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f5f6019e3f0 con 0x7f5f60105ec0 2024-09-10T08:29:09.433 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.712+0000 7f5f57400640 1 -- 172.21.15.89:0/4097086248 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f5f5006be40 con 0x7f5f60105ec0 2024-09-10T08:29:09.433 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.713+0000 7f5f57400640 1 -- 172.21.15.89:0/4097086248 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f5f5002fd60 con 0x7f5f60105ec0 2024-09-10T08:29:09.433 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.715+0000 7f5f64c00640 1 -- 172.21.15.89:0/4097086248 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f5f44038f70 msgr2=0x7f5f4403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:09.433 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.715+0000 7f5f64c00640 1 --2- 172.21.15.89:0/4097086248 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f5f44038f70 0x7f5f4403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5f480098c0 tx=0x7f5f48006db0 comp rx=0 tx=0).stop 2024-09-10T08:29:09.433 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.715+0000 7f5f64c00640 1 -- 172.21.15.89:0/4097086248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f60105ec0 msgr2=0x7f5f6019d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:09.433 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.715+0000 7f5f64c00640 1 --2- 172.21.15.89:0/4097086248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f60105ec0 0x7f5f6019d7a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5f500097f0 tx=0x7f5f50030d80 comp rx=0 tx=0).stop 2024-09-10T08:29:09.433 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.715+0000 7f5f64c00640 1 -- 172.21.15.89:0/4097086248 shutdown_connections 2024-09-10T08:29:09.433 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.715+0000 7f5f64c00640 1 --2- 172.21.15.89:0/4097086248 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f5f44038f70 0x7f5f4403b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:09.433 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.715+0000 7f5f64c00640 1 --2- 172.21.15.89:0/4097086248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f60105ec0 0x7f5f6019d7a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:09.433 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.715+0000 7f5f64c00640 1 -- 172.21.15.89:0/4097086248 >> 172.21.15.89:0/4097086248 conn(0x7f5f60101d50 msgr2=0x7f5f60191aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:09.433 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.715+0000 7f5f64c00640 1 -- 172.21.15.89:0/4097086248 shutdown_connections 2024-09-10T08:29:09.434 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:08.715+0000 7f5f64c00640 1 -- 172.21.15.89:0/4097086248 wait complete. 2024-09-10T08:29:09.552 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:09 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:09.440+0000 7ff28042d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T08:29:09.828 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:09 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/4097086248' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T08:29:09.828 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:09 smithi089 ceph-mon[30757]: mgrmap e5: x(active, since 5s) 2024-09-10T08:29:09.828 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:09 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:09.611+0000 7ff28042d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T08:29:10.143 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:09 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:09.990+0000 7ff28042d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T08:29:10.143 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:10 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:10.063+0000 7ff28042d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T08:29:10.531 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:10 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:10.282+0000 7ff28042d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T08:29:10.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:10 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/349494927' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T08:29:10.929 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2024-09-10T08:29:10.933 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T08:29:10.933 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T08:29:10.933 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T08:29:10.933 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T08:29:10.933 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2024-09-10T08:29:10.934 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.935+0000 7f509fe00640 1 Processor -- start 2024-09-10T08:29:10.934 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.935+0000 7f509fe00640 1 -- start start 2024-09-10T08:29:10.934 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.936+0000 7f509fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50a007dea0 0x7f50a007c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:10.934 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.936+0000 7f509fe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f50a007c8d0 con 0x7f50a007dea0 2024-09-10T08:29:10.934 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.936+0000 7f509ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50a007dea0 0x7f50a007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:10.934 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.936+0000 7f509ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50a007dea0 0x7f50a007c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35282/0 (socket says 172.21.15.89:35282) 2024-09-10T08:29:10.934 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.936+0000 7f509ea00640 1 -- 172.21.15.89:0/3401937718 learned_addr learned my addr 172.21.15.89:0/3401937718 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:10.934 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.937+0000 7f509ea00640 1 -- 172.21.15.89:0/3401937718 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a007ca50 con 0x7f50a007dea0 2024-09-10T08:29:10.934 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.938+0000 7f509ea00640 1 --2- 172.21.15.89:0/3401937718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50a007dea0 0x7f50a007c300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5090009820 tx=0x7f509002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247c17e38c18fafc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:10.934 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.938+0000 7f509e000640 1 -- 172.21.15.89:0/3401937718 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5090037070 con 0x7f50a007dea0 2024-09-10T08:29:10.935 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.938+0000 7f509e000640 1 -- 172.21.15.89:0/3401937718 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5090032900 con 0x7f50a007dea0 2024-09-10T08:29:10.935 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.938+0000 7f509e000640 1 -- 172.21.15.89:0/3401937718 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5090030850 con 0x7f50a007dea0 2024-09-10T08:29:10.935 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.939+0000 7f509fe00640 1 -- 172.21.15.89:0/3401937718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50a007dea0 msgr2=0x7f50a007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:10.935 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.939+0000 7f509fe00640 1 --2- 172.21.15.89:0/3401937718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50a007dea0 0x7f50a007c300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5090009820 tx=0x7f509002a270 comp rx=0 tx=0).stop 2024-09-10T08:29:10.935 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.939+0000 7f509fe00640 1 -- 172.21.15.89:0/3401937718 shutdown_connections 2024-09-10T08:29:10.935 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.939+0000 7f509fe00640 1 --2- 172.21.15.89:0/3401937718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50a007dea0 0x7f50a007c300 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:10.935 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.939+0000 7f509fe00640 1 -- 172.21.15.89:0/3401937718 >> 172.21.15.89:0/3401937718 conn(0x7f50a0102c10 msgr2=0x7f50a0105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:10.935 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.939+0000 7f509fe00640 1 -- 172.21.15.89:0/3401937718 shutdown_connections 2024-09-10T08:29:10.935 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.939+0000 7f509fe00640 1 -- 172.21.15.89:0/3401937718 wait complete. 2024-09-10T08:29:10.935 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.940+0000 7f509fe00640 1 Processor -- start 2024-09-10T08:29:10.935 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.940+0000 7f509fe00640 1 -- start start 2024-09-10T08:29:10.936 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.940+0000 7f509fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50a007dea0 0x7f50a019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:10.936 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.940+0000 7f509fe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f50a010dab0 con 0x7f50a007dea0 2024-09-10T08:29:10.936 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.941+0000 7f509ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50a007dea0 0x7f50a019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:10.936 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.941+0000 7f509ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50a007dea0 0x7f50a019d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35288/0 (socket says 172.21.15.89:35288) 2024-09-10T08:29:10.936 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.941+0000 7f509ea00640 1 -- 172.21.15.89:0/349494927 learned_addr learned my addr 172.21.15.89:0/349494927 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:10.936 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.941+0000 7f509ea00640 1 -- 172.21.15.89:0/349494927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a019dbf0 con 0x7f50a007dea0 2024-09-10T08:29:10.936 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.941+0000 7f509ea00640 1 --2- 172.21.15.89:0/349494927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50a007dea0 0x7f50a019d6b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f50900097f0 tx=0x7f5090030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:10.936 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.941+0000 7f5097e00640 1 -- 172.21.15.89:0/349494927 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5090042070 con 0x7f50a007dea0 2024-09-10T08:29:10.936 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.942+0000 7f5097e00640 1 -- 172.21.15.89:0/349494927 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5090032ca0 con 0x7f50a007dea0 2024-09-10T08:29:10.936 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.942+0000 7f5097e00640 1 -- 172.21.15.89:0/349494927 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5090037040 con 0x7f50a007dea0 2024-09-10T08:29:10.936 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.942+0000 7f509fe00640 1 -- 172.21.15.89:0/349494927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a019de80 con 0x7f50a007dea0 2024-09-10T08:29:10.936 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.942+0000 7f509fe00640 1 -- 172.21.15.89:0/349494927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a01a0b70 con 0x7f50a007dea0 2024-09-10T08:29:10.937 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.943+0000 7f5097e00640 1 -- 172.21.15.89:0/349494927 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f5090045070 con 0x7f50a007dea0 2024-09-10T08:29:10.937 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.943+0000 7f509fe00640 1 -- 172.21.15.89:0/349494927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a007c380 con 0x7f50a007dea0 2024-09-10T08:29:10.937 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.943+0000 7f5097e00640 1 --2- 172.21.15.89:0/349494927 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f5084038fc0 0x7f508403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:10.937 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.943+0000 7f5097e00640 1 -- 172.21.15.89:0/349494927 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f509006bef0 con 0x7f50a007dea0 2024-09-10T08:29:10.937 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.943+0000 7f5096000640 1 -- 172.21.15.89:0/349494927 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f5084038fc0 msgr2=0x7f508403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:6800/4223112778 2024-09-10T08:29:10.937 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.943+0000 7f5096000640 1 --2- 172.21.15.89:0/349494927 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f5084038fc0 0x7f508403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T08:29:10.937 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:09.948+0000 7f5097e00640 1 -- 172.21.15.89:0/349494927 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5090045360 con 0x7f50a007dea0 2024-09-10T08:29:10.937 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:10.144+0000 7f5096000640 1 -- 172.21.15.89:0/349494927 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f5084038fc0 msgr2=0x7f508403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:6800/4223112778 2024-09-10T08:29:10.937 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:10.144+0000 7f5096000640 1 --2- 172.21.15.89:0/349494927 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f5084038fc0 0x7f508403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T08:29:10.937 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:10.180+0000 7f509fe00640 1 -- 172.21.15.89:0/349494927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f50a01a11f0 con 0x7f50a007dea0 2024-09-10T08:29:10.937 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:10.180+0000 7f5097e00640 1 -- 172.21.15.89:0/349494927 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f5090043260 con 0x7f50a007dea0 2024-09-10T08:29:10.937 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:10.182+0000 7f509fe00640 1 -- 172.21.15.89:0/349494927 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f5084038fc0 msgr2=0x7f508403b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:29:10.937 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:10.182+0000 7f509fe00640 1 --2- 172.21.15.89:0/349494927 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f5084038fc0 0x7f508403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:10.938 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:10.182+0000 7f509fe00640 1 -- 172.21.15.89:0/349494927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50a007dea0 msgr2=0x7f50a019d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:10.938 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:10.182+0000 7f509fe00640 1 --2- 172.21.15.89:0/349494927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50a007dea0 0x7f50a019d6b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f50900097f0 tx=0x7f5090030ec0 comp rx=0 tx=0).stop 2024-09-10T08:29:10.938 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:10.182+0000 7f509fe00640 1 -- 172.21.15.89:0/349494927 shutdown_connections 2024-09-10T08:29:10.938 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:10.182+0000 7f509fe00640 1 --2- 172.21.15.89:0/349494927 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f5084038fc0 0x7f508403b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:10.938 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:10.182+0000 7f509fe00640 1 --2- 172.21.15.89:0/349494927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50a007dea0 0x7f50a019d6b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:10.938 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:10.182+0000 7f509fe00640 1 -- 172.21.15.89:0/349494927 >> 172.21.15.89:0/349494927 conn(0x7f50a0102c10 msgr2=0x7f50a0103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:10.938 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:10.183+0000 7f509fe00640 1 -- 172.21.15.89:0/349494927 shutdown_connections 2024-09-10T08:29:10.938 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:10.183+0000 7f509fe00640 1 -- 172.21.15.89:0/349494927 wait complete. 2024-09-10T08:29:10.938 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for the mgr to restart... 2024-09-10T08:29:10.938 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for mgr epoch 5... 2024-09-10T08:29:11.802 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:11 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:11.429+0000 7ff28042d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T08:29:11.802 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:11 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:11.509+0000 7ff28042d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T08:29:12.697 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:12 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:12.405+0000 7ff28042d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T08:29:12.697 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:12 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:12.538+0000 7ff28042d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T08:29:12.697 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:12 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:12.697+0000 7ff28042d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T08:29:13.017 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:12 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:12.769+0000 7ff28042d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T08:29:13.017 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:12 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:12.938+0000 7ff28042d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T08:29:13.018 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:13 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:13.018+0000 7ff28042d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T08:29:13.302 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:13 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:13.089+0000 7ff28042d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T08:29:13.302 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:13 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:13.266+0000 7ff28042d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T08:29:13.653 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:13 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:13.342+0000 7ff28042d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T08:29:13.653 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:13 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:13.420+0000 7ff28042d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T08:29:13.653 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:13 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:13.514+0000 7ff28042d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T08:29:13.653 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:13 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:13.584+0000 7ff28042d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T08:29:13.653 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:13 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:13.653+0000 7ff28042d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T08:29:14.802 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:14 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:14.552+0000 7ff28042d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T08:29:14.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:14 smithi089 ceph-mon[30757]: Active manager daemon x restarted 2024-09-10T08:29:14.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:14 smithi089 ceph-mon[30757]: Activating manager daemon x 2024-09-10T08:29:14.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:14 smithi089 ceph-mon[30757]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T08:29:14.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:14 smithi089 ceph-mon[30757]: mgrmap e6: x(active, starting, since 0.00394038s) 2024-09-10T08:29:14.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:14 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:29:14.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:14 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T08:29:14.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:14 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T08:29:14.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:14 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T08:29:14.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:14 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T08:29:14.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:14 smithi089 ceph-mon[30757]: Manager daemon x is now available 2024-09-10T08:29:14.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:14 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' 2024-09-10T08:29:14.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:14 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' 2024-09-10T08:29:14.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:14 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:29:14.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:14 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:29:15.913 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:15 smithi089 ceph-mon[30757]: Found migration_current of "None". Setting to last migration. 2024-09-10T08:29:15.913 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:15 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T08:29:15.913 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:15 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T08:29:15.913 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:15 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' 2024-09-10T08:29:15.913 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:15 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' 2024-09-10T08:29:15.913 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:15 smithi089 ceph-mon[30757]: mgrmap e7: x(active, since 1.00924s) 2024-09-10T08:29:16.285 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2024-09-10T08:29:16.286 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T08:29:16.286 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T08:29:16.286 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2024-09-10T08:29:16.286 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.385+0000 7f99bb200640 1 Processor -- start 2024-09-10T08:29:16.286 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.385+0000 7f99bb200640 1 -- start start 2024-09-10T08:29:16.286 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.386+0000 7f99bb200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99bc107440 0x7f99bc107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:16.286 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.386+0000 7f99bb200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc107e10 con 0x7f99bc107440 2024-09-10T08:29:16.286 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.386+0000 7f99b9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99bc107440 0x7f99bc107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:16.286 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.386+0000 7f99b9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99bc107440 0x7f99bc107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35296/0 (socket says 172.21.15.89:35296) 2024-09-10T08:29:16.286 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.386+0000 7f99b9e00640 1 -- 172.21.15.89:0/1418224991 learned_addr learned my addr 172.21.15.89:0/1418224991 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:16.286 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.386+0000 7f99b9e00640 1 -- 172.21.15.89:0/1418224991 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99bc108640 con 0x7f99bc107440 2024-09-10T08:29:16.287 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.387+0000 7f99b9e00640 1 --2- 172.21.15.89:0/1418224991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99bc107440 0x7f99bc107840 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f99ac009820 tx=0x7f99ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c625e25ea9cf757 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:16.287 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.387+0000 7f99b9400640 1 -- 172.21.15.89:0/1418224991 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99ac037070 con 0x7f99bc107440 2024-09-10T08:29:16.287 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.388+0000 7f99b9400640 1 -- 172.21.15.89:0/1418224991 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f99ac032900 con 0x7f99bc107440 2024-09-10T08:29:16.287 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.388+0000 7f99b9400640 1 -- 172.21.15.89:0/1418224991 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99ac030850 con 0x7f99bc107440 2024-09-10T08:29:16.287 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.388+0000 7f99bb200640 1 -- 172.21.15.89:0/1418224991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99bc107440 msgr2=0x7f99bc107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:16.287 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.388+0000 7f99bb200640 1 --2- 172.21.15.89:0/1418224991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99bc107440 0x7f99bc107840 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f99ac009820 tx=0x7f99ac02a270 comp rx=0 tx=0).stop 2024-09-10T08:29:16.287 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.388+0000 7f99bb200640 1 -- 172.21.15.89:0/1418224991 shutdown_connections 2024-09-10T08:29:16.287 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.388+0000 7f99bb200640 1 --2- 172.21.15.89:0/1418224991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99bc107440 0x7f99bc107840 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:16.287 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.388+0000 7f99bb200640 1 -- 172.21.15.89:0/1418224991 >> 172.21.15.89:0/1418224991 conn(0x7f99bc102c10 msgr2=0x7f99bc105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:16.287 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.388+0000 7f99bb200640 1 -- 172.21.15.89:0/1418224991 shutdown_connections 2024-09-10T08:29:16.287 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.388+0000 7f99bb200640 1 -- 172.21.15.89:0/1418224991 wait complete. 2024-09-10T08:29:16.287 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.389+0000 7f99bb200640 1 Processor -- start 2024-09-10T08:29:16.288 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.389+0000 7f99bb200640 1 -- start start 2024-09-10T08:29:16.288 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.390+0000 7f99bb200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99bc107440 0x7f99bc07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:16.288 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.390+0000 7f99bb200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc10b9f0 con 0x7f99bc107440 2024-09-10T08:29:16.288 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.390+0000 7f99b9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99bc107440 0x7f99bc07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:16.288 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.390+0000 7f99b9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99bc107440 0x7f99bc07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35310/0 (socket says 172.21.15.89:35310) 2024-09-10T08:29:16.288 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.390+0000 7f99b9e00640 1 -- 172.21.15.89:0/3205033441 learned_addr learned my addr 172.21.15.89:0/3205033441 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:16.288 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.390+0000 7f99b9e00640 1 -- 172.21.15.89:0/3205033441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99bc07e0b0 con 0x7f99bc107440 2024-09-10T08:29:16.288 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.390+0000 7f99b9e00640 1 --2- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99bc107440 0x7f99bc07db70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0097f0 tx=0x7f99ac030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:16.288 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.391+0000 7f99b2a00640 1 -- 172.21.15.89:0/3205033441 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99ac042070 con 0x7f99bc107440 2024-09-10T08:29:16.288 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.391+0000 7f99b2a00640 1 -- 172.21.15.89:0/3205033441 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f99ac032ca0 con 0x7f99bc107440 2024-09-10T08:29:16.288 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.391+0000 7f99b2a00640 1 -- 172.21.15.89:0/3205033441 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99ac037040 con 0x7f99bc107440 2024-09-10T08:29:16.288 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.391+0000 7f99bb200640 1 -- 172.21.15.89:0/3205033441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99bc07c2a0 con 0x7f99bc107440 2024-09-10T08:29:16.289 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.391+0000 7f99bb200640 1 -- 172.21.15.89:0/3205033441 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99bc07c7e0 con 0x7f99bc107440 2024-09-10T08:29:16.289 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.392+0000 7f99b2a00640 1 -- 172.21.15.89:0/3205033441 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f99ac045070 con 0x7f99bc107440 2024-09-10T08:29:16.289 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.392+0000 7f99b2a00640 1 --2- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f99a0038fc0 0x7f99a003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:16.289 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.392+0000 7f99b2a00640 1 -- 172.21.15.89:0/3205033441 --> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f99a003bb90 con 0x7f99a0038fc0 2024-09-10T08:29:16.289 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.392+0000 7f99b2a00640 1 -- 172.21.15.89:0/3205033441 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f99ac06be70 con 0x7f99bc107440 2024-09-10T08:29:16.289 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.392+0000 7f99b3e00640 1 -- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f99a0038fc0 msgr2=0x7f99a003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:6800/4223112778 2024-09-10T08:29:16.289 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.392+0000 7f99b3e00640 1 --2- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f99a0038fc0 0x7f99a003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T08:29:16.289 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.593+0000 7f99b3e00640 1 -- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f99a0038fc0 msgr2=0x7f99a003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:6800/4223112778 2024-09-10T08:29:16.289 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.593+0000 7f99b3e00640 1 --2- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f99a0038fc0 0x7f99a003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T08:29:16.289 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.994+0000 7f99b3e00640 1 -- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f99a0038fc0 msgr2=0x7f99a003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:6800/4223112778 2024-09-10T08:29:16.289 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:11.994+0000 7f99b3e00640 1 --2- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f99a0038fc0 0x7f99a003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T08:29:16.290 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:12.795+0000 7f99b3e00640 1 -- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f99a0038fc0 msgr2=0x7f99a003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:6800/4223112778 2024-09-10T08:29:16.290 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:12.795+0000 7f99b3e00640 1 --2- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f99a0038fc0 0x7f99a003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T08:29:16.290 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:14.396+0000 7f99b3e00640 1 -- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f99a0038fc0 msgr2=0x7f99a003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:6800/4223112778 2024-09-10T08:29:16.290 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:14.397+0000 7f99b3e00640 1 --2- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f99a0038fc0 0x7f99a003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-10T08:29:16.290 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:14.557+0000 7f99b2a00640 1 -- 172.21.15.89:0/3205033441 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f99ac03d430 con 0x7f99bc107440 2024-09-10T08:29:16.290 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:14.557+0000 7f99b2a00640 1 -- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f99a0038fc0 msgr2=0x7f99a003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:29:16.290 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:14.557+0000 7f99b2a00640 1 --2- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f99a0038fc0 0x7f99a003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:16.290 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:15.561+0000 7f99b2a00640 1 -- 172.21.15.89:0/3205033441 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f99ac06dd50 con 0x7f99bc107440 2024-09-10T08:29:16.290 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:15.561+0000 7f99b2a00640 1 --2- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f99a003c8f0 0x7f99a003ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:16.290 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:15.561+0000 7f99b2a00640 1 -- 172.21.15.89:0/3205033441 --> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f99ac00af30 con 0x7f99a003c8f0 2024-09-10T08:29:16.290 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:15.561+0000 7f99b3e00640 1 --2- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f99a003c8f0 0x7f99a003ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:16.290 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:15.563+0000 7f99b3e00640 1 --2- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f99a003c8f0 0x7f99a003ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f99a4003980 tx=0x7f99a40092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:16.291 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:15.564+0000 7f99b2a00640 1 -- 172.21.15.89:0/3205033441 <== mgr.14118 v2:172.21.15.89:6800/3794982914 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f99ac00af30 con 0x7f99a003c8f0 2024-09-10T08:29:16.291 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:15.571+0000 7f99bb200640 1 -- 172.21.15.89:0/3205033441 --> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f99bc10bb70 con 0x7f99a003c8f0 2024-09-10T08:29:16.291 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:15.571+0000 7f99b2a00640 1 -- 172.21.15.89:0/3205033441 <== mgr.14118 v2:172.21.15.89:6800/3794982914 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f99bc10bb70 con 0x7f99a003c8f0 2024-09-10T08:29:16.291 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:15.572+0000 7f99bb200640 1 -- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f99a003c8f0 msgr2=0x7f99a003ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:16.291 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:15.572+0000 7f99bb200640 1 --2- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f99a003c8f0 0x7f99a003ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f99a4003980 tx=0x7f99a40092b0 comp rx=0 tx=0).stop 2024-09-10T08:29:16.291 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:15.572+0000 7f99bb200640 1 -- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99bc107440 msgr2=0x7f99bc07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:16.291 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:15.572+0000 7f99bb200640 1 --2- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99bc107440 0x7f99bc07db70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0097f0 tx=0x7f99ac030ec0 comp rx=0 tx=0).stop 2024-09-10T08:29:16.291 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:15.572+0000 7f99bb200640 1 -- 172.21.15.89:0/3205033441 shutdown_connections 2024-09-10T08:29:16.291 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:15.572+0000 7f99bb200640 1 --2- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f99a003c8f0 0x7f99a003ece0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:16.291 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:15.572+0000 7f99bb200640 1 --2- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:6800/4223112778,v1:172.21.15.89:6801/4223112778] conn(0x7f99a0038fc0 0x7f99a003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:16.291 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:15.572+0000 7f99bb200640 1 --2- 172.21.15.89:0/3205033441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99bc107440 0x7f99bc07db70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:16.292 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:15.572+0000 7f99bb200640 1 -- 172.21.15.89:0/3205033441 >> 172.21.15.89:0/3205033441 conn(0x7f99bc102c10 msgr2=0x7f99bc1037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:16.292 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:15.572+0000 7f99bb200640 1 -- 172.21.15.89:0/3205033441 shutdown_connections 2024-09-10T08:29:16.292 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:15.572+0000 7f99bb200640 1 -- 172.21.15.89:0/3205033441 wait complete. 2024-09-10T08:29:16.292 INFO:teuthology.orchestra.run.smithi089.stdout:mgr epoch 5 is available 2024-09-10T08:29:16.292 INFO:teuthology.orchestra.run.smithi089.stdout:Setting orchestrator backend to cephadm... 2024-09-10T08:29:16.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-mon[30757]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T08:29:16.791 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-mon[30757]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T08:29:16.791 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:16.529+0000 7ff1df600640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:08:29:16] ENGINE Error in HTTPServer.serve 2024-09-10T08:29:16.791 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: Traceback (most recent call last): 2024-09-10T08:29:16.791 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T08:29:16.791 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: self._connections.run(self.expiration_interval) 2024-09-10T08:29:16.791 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T08:29:16.791 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: self._run(expiration_interval) 2024-09-10T08:29:16.792 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T08:29:16.792 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T08:29:16.792 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T08:29:16.792 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T08:29:16.792 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T08:29:16.792 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: s = self.context.wrap_socket( 2024-09-10T08:29:16.792 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T08:29:16.792 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: return self.sslsocket_class._create( 2024-09-10T08:29:16.792 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T08:29:16.792 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: self.do_handshake() 2024-09-10T08:29:16.792 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T08:29:16.792 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: self._sslobj.do_handshake() 2024-09-10T08:29:16.792 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T08:29:16.792 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:16 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:17.703 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:17 smithi089 ceph-mon[30757]: [10/Sep/2024:08:29:16] ENGINE Bus STARTING 2024-09-10T08:29:17.703 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:17 smithi089 ceph-mon[30757]: [10/Sep/2024:08:29:16] ENGINE Serving on https://172.21.15.89:7150 2024-09-10T08:29:17.703 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:17 smithi089 ceph-mon[30757]: [10/Sep/2024:08:29:16] ENGINE Error in HTTPServer.serve 2024-09-10T08:29:17.703 INFO:journalctl@ceph.mon.a.smithi089.stdout: Traceback (most recent call last): 2024-09-10T08:29:17.703 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T08:29:17.703 INFO:journalctl@ceph.mon.a.smithi089.stdout: self._connections.run(self.expiration_interval) 2024-09-10T08:29:17.703 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T08:29:17.703 INFO:journalctl@ceph.mon.a.smithi089.stdout: self._run(expiration_interval) 2024-09-10T08:29:17.703 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T08:29:17.703 INFO:journalctl@ceph.mon.a.smithi089.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T08:29:17.704 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T08:29:17.704 INFO:journalctl@ceph.mon.a.smithi089.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T08:29:17.704 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T08:29:17.704 INFO:journalctl@ceph.mon.a.smithi089.stdout: s = self.context.wrap_socket( 2024-09-10T08:29:17.704 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T08:29:17.704 INFO:journalctl@ceph.mon.a.smithi089.stdout: return self.sslsocket_class._create( 2024-09-10T08:29:17.704 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T08:29:17.704 INFO:journalctl@ceph.mon.a.smithi089.stdout: self.do_handshake() 2024-09-10T08:29:17.704 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T08:29:17.704 INFO:journalctl@ceph.mon.a.smithi089.stdout: self._sslobj.do_handshake() 2024-09-10T08:29:17.704 INFO:journalctl@ceph.mon.a.smithi089.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T08:29:17.704 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:17 smithi089 ceph-mon[30757]: mgrmap e8: x(active, since 2s) 2024-09-10T08:29:17.704 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:17 smithi089 ceph-mon[30757]: [10/Sep/2024:08:29:16] ENGINE Serving on http://172.21.15.89:8765 2024-09-10T08:29:17.705 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:17 smithi089 ceph-mon[30757]: [10/Sep/2024:08:29:16] ENGINE Bus STARTED 2024-09-10T08:29:17.705 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:17 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:29:17.705 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:17 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' 2024-09-10T08:29:17.705 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:17 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:29:17.713 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.791+0000 7f3b5c400640 1 Processor -- start 2024-09-10T08:29:17.713 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.791+0000 7f3b5c400640 1 -- start start 2024-09-10T08:29:17.713 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.792+0000 7f3b5c400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b54107680 0x7f3b54107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:17.713 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.792+0000 7f3b5c400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3b54107fc0 con 0x7f3b54107680 2024-09-10T08:29:17.713 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.792+0000 7f3b59a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b54107680 0x7f3b54107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:17.713 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.792+0000 7f3b59a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b54107680 0x7f3b54107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38640/0 (socket says 172.21.15.89:38640) 2024-09-10T08:29:17.713 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.792+0000 7f3b59a00640 1 -- 172.21.15.89:0/1940402364 learned_addr learned my addr 172.21.15.89:0/1940402364 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:17.713 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.792+0000 7f3b59a00640 1 -- 172.21.15.89:0/1940402364 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b54108140 con 0x7f3b54107680 2024-09-10T08:29:17.714 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.793+0000 7f3b59a00640 1 --2- 172.21.15.89:0/1940402364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b54107680 0x7f3b54107a80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3b48009e80 tx=0x7f3b4802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e95b65e539a6a06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:17.714 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.794+0000 7f3b53e00640 1 -- 172.21.15.89:0/1940402364 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b48036070 con 0x7f3b54107680 2024-09-10T08:29:17.714 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.794+0000 7f3b53e00640 1 -- 172.21.15.89:0/1940402364 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3b480318c0 con 0x7f3b54107680 2024-09-10T08:29:17.714 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.794+0000 7f3b53e00640 1 -- 172.21.15.89:0/1940402364 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b48007ab0 con 0x7f3b54107680 2024-09-10T08:29:17.714 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.794+0000 7f3b5c400640 1 -- 172.21.15.89:0/1940402364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b54107680 msgr2=0x7f3b54107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:17.714 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.794+0000 7f3b5c400640 1 --2- 172.21.15.89:0/1940402364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b54107680 0x7f3b54107a80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3b48009e80 tx=0x7f3b4802a7a0 comp rx=0 tx=0).stop 2024-09-10T08:29:17.714 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.794+0000 7f3b5c400640 1 -- 172.21.15.89:0/1940402364 shutdown_connections 2024-09-10T08:29:17.714 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.794+0000 7f3b5c400640 1 --2- 172.21.15.89:0/1940402364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b54107680 0x7f3b54107a80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:17.714 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.794+0000 7f3b5c400640 1 -- 172.21.15.89:0/1940402364 >> 172.21.15.89:0/1940402364 conn(0x7f3b54102eb0 msgr2=0x7f3b541052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:17.714 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.795+0000 7f3b5c400640 1 -- 172.21.15.89:0/1940402364 shutdown_connections 2024-09-10T08:29:17.714 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.795+0000 7f3b5c400640 1 -- 172.21.15.89:0/1940402364 wait complete. 2024-09-10T08:29:17.714 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.795+0000 7f3b5c400640 1 Processor -- start 2024-09-10T08:29:17.715 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.796+0000 7f3b5c400640 1 -- start start 2024-09-10T08:29:17.715 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.796+0000 7f3b5c400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b54107680 0x7f3b5407db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:17.715 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.796+0000 7f3b5c400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5410baf0 con 0x7f3b54107680 2024-09-10T08:29:17.715 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.796+0000 7f3b59a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b54107680 0x7f3b5407db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:17.715 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.796+0000 7f3b59a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b54107680 0x7f3b5407db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38648/0 (socket says 172.21.15.89:38648) 2024-09-10T08:29:17.715 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.796+0000 7f3b59a00640 1 -- 172.21.15.89:0/4138229983 learned_addr learned my addr 172.21.15.89:0/4138229983 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:17.715 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.796+0000 7f3b59a00640 1 -- 172.21.15.89:0/4138229983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5407e0b0 con 0x7f3b54107680 2024-09-10T08:29:17.715 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.797+0000 7f3b59a00640 1 --2- 172.21.15.89:0/4138229983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b54107680 0x7f3b5407db70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3b4802acc0 tx=0x7f3b48031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:17.715 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.797+0000 7f3b52000640 1 -- 172.21.15.89:0/4138229983 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b48036050 con 0x7f3b54107680 2024-09-10T08:29:17.715 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.797+0000 7f3b52000640 1 -- 172.21.15.89:0/4138229983 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3b48037040 con 0x7f3b54107680 2024-09-10T08:29:17.715 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.797+0000 7f3b5c400640 1 -- 172.21.15.89:0/4138229983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b5407c2a0 con 0x7f3b54107680 2024-09-10T08:29:17.715 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.797+0000 7f3b52000640 1 -- 172.21.15.89:0/4138229983 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b4803d920 con 0x7f3b54107680 2024-09-10T08:29:17.716 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.797+0000 7f3b5c400640 1 -- 172.21.15.89:0/4138229983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b5407c7e0 con 0x7f3b54107680 2024-09-10T08:29:17.716 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.798+0000 7f3b5c400640 1 -- 172.21.15.89:0/4138229983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b54107b00 con 0x7f3b54107680 2024-09-10T08:29:17.716 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.798+0000 7f3b52000640 1 -- 172.21.15.89:0/4138229983 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3b48044020 con 0x7f3b54107680 2024-09-10T08:29:17.716 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.799+0000 7f3b52000640 1 --2- 172.21.15.89:0/4138229983 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f3b3c038fc0 0x7f3b3c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:17.716 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.799+0000 7f3b52000640 1 -- 172.21.15.89:0/4138229983 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3b48042020 con 0x7f3b54107680 2024-09-10T08:29:17.716 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.799+0000 7f3b59000640 1 --2- 172.21.15.89:0/4138229983 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f3b3c038fc0 0x7f3b3c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:17.716 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.800+0000 7f3b59000640 1 --2- 172.21.15.89:0/4138229983 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f3b3c038fc0 0x7f3b3c03b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3b400098c0 tx=0x7f3b40006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:17.716 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.804+0000 7f3b52000640 1 -- 172.21.15.89:0/4138229983 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3b4804b020 con 0x7f3b54107680 2024-09-10T08:29:17.716 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.991+0000 7f3b5c400640 1 -- 172.21.15.89:0/4138229983 --> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f3b5407d130 con 0x7f3b3c038fc0 2024-09-10T08:29:17.716 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:16.999+0000 7f3b52000640 1 -- 172.21.15.89:0/4138229983 <== mgr.14118 v2:172.21.15.89:6800/3794982914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3b5407d130 con 0x7f3b3c038fc0 2024-09-10T08:29:17.716 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:17.002+0000 7f3b5c400640 1 -- 172.21.15.89:0/4138229983 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f3b3c038fc0 msgr2=0x7f3b3c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:17.717 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:17.002+0000 7f3b5c400640 1 --2- 172.21.15.89:0/4138229983 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f3b3c038fc0 0x7f3b3c03b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3b400098c0 tx=0x7f3b40006db0 comp rx=0 tx=0).stop 2024-09-10T08:29:17.717 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:17.002+0000 7f3b5c400640 1 -- 172.21.15.89:0/4138229983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b54107680 msgr2=0x7f3b5407db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:17.717 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:17.002+0000 7f3b5c400640 1 --2- 172.21.15.89:0/4138229983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b54107680 0x7f3b5407db70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3b4802acc0 tx=0x7f3b48031ca0 comp rx=0 tx=0).stop 2024-09-10T08:29:17.717 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:17.002+0000 7f3b5c400640 1 -- 172.21.15.89:0/4138229983 shutdown_connections 2024-09-10T08:29:17.717 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:17.002+0000 7f3b5c400640 1 --2- 172.21.15.89:0/4138229983 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f3b3c038fc0 0x7f3b3c03b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:17.717 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:17.002+0000 7f3b5c400640 1 --2- 172.21.15.89:0/4138229983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b54107680 0x7f3b5407db70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:17.717 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:17.002+0000 7f3b5c400640 1 -- 172.21.15.89:0/4138229983 >> 172.21.15.89:0/4138229983 conn(0x7f3b54102eb0 msgr2=0x7f3b541039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:17.717 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:17.003+0000 7f3b5c400640 1 -- 172.21.15.89:0/4138229983 shutdown_connections 2024-09-10T08:29:17.717 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:17.003+0000 7f3b5c400640 1 -- 172.21.15.89:0/4138229983 wait complete. 2024-09-10T08:29:18.941 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:18 smithi089 ceph-mon[30757]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:29:18.954 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T08:29:18.954 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.186+0000 7f226a400640 1 Processor -- start 2024-09-10T08:29:18.955 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.186+0000 7f226a400640 1 -- start start 2024-09-10T08:29:18.955 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.186+0000 7f226a400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f226407dea0 0x7f226407c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:18.955 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.186+0000 7f226a400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f226407c8d0 con 0x7f226407dea0 2024-09-10T08:29:18.955 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.186+0000 7f2269000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f226407dea0 0x7f226407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:18.955 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.186+0000 7f2269000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f226407dea0 0x7f226407c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38650/0 (socket says 172.21.15.89:38650) 2024-09-10T08:29:18.955 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.186+0000 7f2269000640 1 -- 172.21.15.89:0/916931641 learned_addr learned my addr 172.21.15.89:0/916931641 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:18.955 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.187+0000 7f2269000640 1 -- 172.21.15.89:0/916931641 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226407ca50 con 0x7f226407dea0 2024-09-10T08:29:18.955 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.188+0000 7f2269000640 1 --2- 172.21.15.89:0/916931641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f226407dea0 0x7f226407c300 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2258009820 tx=0x7f225802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c3bc46ebef32ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:18.955 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.188+0000 7f2263400640 1 -- 172.21.15.89:0/916931641 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2258037070 con 0x7f226407dea0 2024-09-10T08:29:18.955 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.188+0000 7f2263400640 1 -- 172.21.15.89:0/916931641 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2258032900 con 0x7f226407dea0 2024-09-10T08:29:18.955 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.188+0000 7f2263400640 1 -- 172.21.15.89:0/916931641 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2258030850 con 0x7f226407dea0 2024-09-10T08:29:18.955 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.188+0000 7f226a400640 1 -- 172.21.15.89:0/916931641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f226407dea0 msgr2=0x7f226407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:18.955 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.188+0000 7f226a400640 1 --2- 172.21.15.89:0/916931641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f226407dea0 0x7f226407c300 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2258009820 tx=0x7f225802a270 comp rx=0 tx=0).stop 2024-09-10T08:29:18.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.188+0000 7f226a400640 1 -- 172.21.15.89:0/916931641 shutdown_connections 2024-09-10T08:29:18.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.188+0000 7f226a400640 1 --2- 172.21.15.89:0/916931641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f226407dea0 0x7f226407c300 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:18.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.188+0000 7f226a400640 1 -- 172.21.15.89:0/916931641 >> 172.21.15.89:0/916931641 conn(0x7f2264102c10 msgr2=0x7f2264105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:18.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.189+0000 7f226a400640 1 -- 172.21.15.89:0/916931641 shutdown_connections 2024-09-10T08:29:18.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.189+0000 7f226a400640 1 -- 172.21.15.89:0/916931641 wait complete. 2024-09-10T08:29:18.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.189+0000 7f226a400640 1 Processor -- start 2024-09-10T08:29:18.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.190+0000 7f226a400640 1 -- start start 2024-09-10T08:29:18.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.190+0000 7f226a400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f226407dea0 0x7f226419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:18.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.190+0000 7f226a400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f226410dab0 con 0x7f226407dea0 2024-09-10T08:29:18.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.190+0000 7f2269000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f226407dea0 0x7f226419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:18.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.190+0000 7f2269000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f226407dea0 0x7f226419d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38660/0 (socket says 172.21.15.89:38660) 2024-09-10T08:29:18.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.190+0000 7f2269000640 1 -- 172.21.15.89:0/1062990683 learned_addr learned my addr 172.21.15.89:0/1062990683 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:18.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.190+0000 7f2269000640 1 -- 172.21.15.89:0/1062990683 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226419dbf0 con 0x7f226407dea0 2024-09-10T08:29:18.956 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.191+0000 7f2269000640 1 --2- 172.21.15.89:0/1062990683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f226407dea0 0x7f226419d6b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2258031000 tx=0x7f22580326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:18.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.191+0000 7f2261600640 1 -- 172.21.15.89:0/1062990683 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2258042070 con 0x7f226407dea0 2024-09-10T08:29:18.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.191+0000 7f2261600640 1 -- 172.21.15.89:0/1062990683 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2258032be0 con 0x7f226407dea0 2024-09-10T08:29:18.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.191+0000 7f2261600640 1 -- 172.21.15.89:0/1062990683 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2258037040 con 0x7f226407dea0 2024-09-10T08:29:18.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.191+0000 7f226a400640 1 -- 172.21.15.89:0/1062990683 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f226419de80 con 0x7f226407dea0 2024-09-10T08:29:18.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.191+0000 7f226a400640 1 -- 172.21.15.89:0/1062990683 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22641a0b70 con 0x7f226407dea0 2024-09-10T08:29:18.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.192+0000 7f2261600640 1 -- 172.21.15.89:0/1062990683 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2258045070 con 0x7f226407dea0 2024-09-10T08:29:18.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.192+0000 7f226a400640 1 -- 172.21.15.89:0/1062990683 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f226407c380 con 0x7f226407dea0 2024-09-10T08:29:18.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.192+0000 7f2261600640 1 --2- 172.21.15.89:0/1062990683 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f224c038fc0 0x7f224c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:18.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.192+0000 7f2261600640 1 -- 172.21.15.89:0/1062990683 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f225806bff0 con 0x7f226407dea0 2024-09-10T08:29:18.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.193+0000 7f2263e00640 1 --2- 172.21.15.89:0/1062990683 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f224c038fc0 0x7f224c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:18.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.193+0000 7f2263e00640 1 --2- 172.21.15.89:0/1062990683 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f224c038fc0 0x7f224c03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f22500098c0 tx=0x7f2250006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:18.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.197+0000 7f2261600640 1 -- 172.21.15.89:0/1062990683 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f225802f2a0 con 0x7f226407dea0 2024-09-10T08:29:18.957 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.372+0000 7f226a400640 1 -- 172.21.15.89:0/1062990683 --> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f2264103e90 con 0x7f224c038fc0 2024-09-10T08:29:18.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.373+0000 7f2261600640 1 -- 172.21.15.89:0/1062990683 <== mgr.14118 v2:172.21.15.89:6800/3794982914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f2264103e90 con 0x7f224c038fc0 2024-09-10T08:29:18.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.375+0000 7f226a400640 1 -- 172.21.15.89:0/1062990683 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f224c038fc0 msgr2=0x7f224c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:18.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.375+0000 7f226a400640 1 --2- 172.21.15.89:0/1062990683 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f224c038fc0 0x7f224c03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f22500098c0 tx=0x7f2250006db0 comp rx=0 tx=0).stop 2024-09-10T08:29:18.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.375+0000 7f226a400640 1 -- 172.21.15.89:0/1062990683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f226407dea0 msgr2=0x7f226419d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:18.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.375+0000 7f226a400640 1 --2- 172.21.15.89:0/1062990683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f226407dea0 0x7f226419d6b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2258031000 tx=0x7f22580326e0 comp rx=0 tx=0).stop 2024-09-10T08:29:18.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.375+0000 7f226a400640 1 -- 172.21.15.89:0/1062990683 shutdown_connections 2024-09-10T08:29:18.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.375+0000 7f226a400640 1 --2- 172.21.15.89:0/1062990683 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f224c038fc0 0x7f224c03b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:18.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.375+0000 7f226a400640 1 --2- 172.21.15.89:0/1062990683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f226407dea0 0x7f226419d6b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:18.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.375+0000 7f226a400640 1 -- 172.21.15.89:0/1062990683 >> 172.21.15.89:0/1062990683 conn(0x7f2264102c10 msgr2=0x7f2264103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:18.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.376+0000 7f226a400640 1 -- 172.21.15.89:0/1062990683 shutdown_connections 2024-09-10T08:29:18.958 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:18.376+0000 7f226a400640 1 -- 172.21.15.89:0/1062990683 wait complete. 2024-09-10T08:29:18.958 INFO:teuthology.orchestra.run.smithi089.stdout:Generating ssh key... 2024-09-10T08:29:19.893 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:19 smithi089 ceph-mon[30757]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:29:20.290 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:19 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: Generating public/private rsa key pair. 2024-09-10T08:29:20.290 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:19 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: Your identification has been saved in /tmp/tmph_g40cvg/key 2024-09-10T08:29:20.290 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:19 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: Your public key has been saved in /tmp/tmph_g40cvg/key.pub 2024-09-10T08:29:20.290 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:19 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: The key fingerprint is: 2024-09-10T08:29:20.290 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:19 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: SHA256:dpkxyDtcAsYd994aGjvf6HJPuLlJe8mjdvvBFYAaSQU ceph-91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:29:20.290 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:19 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: The key's randomart image is: 2024-09-10T08:29:20.290 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:19 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: +---[RSA 3072]----+ 2024-09-10T08:29:20.290 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:19 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: | .o.oE=... | 2024-09-10T08:29:20.290 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:19 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: | ..o.= o . | 2024-09-10T08:29:20.291 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:19 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: | + * . . | 2024-09-10T08:29:20.291 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:19 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: | . = * . .| 2024-09-10T08:29:20.291 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:19 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: | S = o . .| 2024-09-10T08:29:20.291 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:19 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: | . o + +. .| 2024-09-10T08:29:20.291 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:19 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: | + +..+ | 2024-09-10T08:29:20.291 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:19 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: | .+oO* .| 2024-09-10T08:29:20.291 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:19 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: | =@*++.| 2024-09-10T08:29:20.291 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:19 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: +----[SHA256]-----+ 2024-09-10T08:29:20.592 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.440+0000 7ffb0ae00640 1 Processor -- start 2024-09-10T08:29:20.592 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.440+0000 7ffb0ae00640 1 -- start start 2024-09-10T08:29:20.592 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.440+0000 7ffb0ae00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04107680 0x7ffb04107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:20.592 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.440+0000 7ffb0ae00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ffb04107fc0 con 0x7ffb04107680 2024-09-10T08:29:20.592 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.441+0000 7ffb03e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04107680 0x7ffb04107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:20.592 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.441+0000 7ffb03e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04107680 0x7ffb04107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38670/0 (socket says 172.21.15.89:38670) 2024-09-10T08:29:20.592 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.441+0000 7ffb03e00640 1 -- 172.21.15.89:0/2554738880 learned_addr learned my addr 172.21.15.89:0/2554738880 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:20.592 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.441+0000 7ffb03e00640 1 -- 172.21.15.89:0/2554738880 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb04108140 con 0x7ffb04107680 2024-09-10T08:29:20.592 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.442+0000 7ffb03e00640 1 --2- 172.21.15.89:0/2554738880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04107680 0x7ffb04107a80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009820 tx=0x7ffaf402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e7a0827417e3b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:20.593 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.442+0000 7ffb02a00640 1 -- 172.21.15.89:0/2554738880 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffaf4037070 con 0x7ffb04107680 2024-09-10T08:29:20.593 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.442+0000 7ffb02a00640 1 -- 172.21.15.89:0/2554738880 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffaf4032900 con 0x7ffb04107680 2024-09-10T08:29:20.593 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.442+0000 7ffb02a00640 1 -- 172.21.15.89:0/2554738880 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffaf4030850 con 0x7ffb04107680 2024-09-10T08:29:20.593 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.443+0000 7ffb0ae00640 1 -- 172.21.15.89:0/2554738880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04107680 msgr2=0x7ffb04107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:20.593 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.443+0000 7ffb0ae00640 1 --2- 172.21.15.89:0/2554738880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04107680 0x7ffb04107a80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009820 tx=0x7ffaf402a270 comp rx=0 tx=0).stop 2024-09-10T08:29:20.593 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.443+0000 7ffb0ae00640 1 -- 172.21.15.89:0/2554738880 shutdown_connections 2024-09-10T08:29:20.593 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.443+0000 7ffb0ae00640 1 --2- 172.21.15.89:0/2554738880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04107680 0x7ffb04107a80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:20.593 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.443+0000 7ffb0ae00640 1 -- 172.21.15.89:0/2554738880 >> 172.21.15.89:0/2554738880 conn(0x7ffb04102eb0 msgr2=0x7ffb041052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:20.593 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.443+0000 7ffb0ae00640 1 -- 172.21.15.89:0/2554738880 shutdown_connections 2024-09-10T08:29:20.593 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.443+0000 7ffb0ae00640 1 -- 172.21.15.89:0/2554738880 wait complete. 2024-09-10T08:29:20.593 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.444+0000 7ffb0ae00640 1 Processor -- start 2024-09-10T08:29:20.593 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.444+0000 7ffb0ae00640 1 -- start start 2024-09-10T08:29:20.593 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.444+0000 7ffb0ae00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04107680 0x7ffb0419d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:20.593 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.444+0000 7ffb0ae00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0410baf0 con 0x7ffb04107680 2024-09-10T08:29:20.594 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.445+0000 7ffb03e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04107680 0x7ffb0419d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:20.594 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.445+0000 7ffb03e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04107680 0x7ffb0419d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38678/0 (socket says 172.21.15.89:38678) 2024-09-10T08:29:20.594 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.445+0000 7ffb03e00640 1 -- 172.21.15.89:0/2651970045 learned_addr learned my addr 172.21.15.89:0/2651970045 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:20.594 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.445+0000 7ffb03e00640 1 -- 172.21.15.89:0/2651970045 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb0419dde0 con 0x7ffb04107680 2024-09-10T08:29:20.594 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.445+0000 7ffb03e00640 1 --2- 172.21.15.89:0/2651970045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04107680 0x7ffb0419d8a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40097f0 tx=0x7ffaf4030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:20.594 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.445+0000 7ffb00c00640 1 -- 172.21.15.89:0/2651970045 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffaf4042070 con 0x7ffb04107680 2024-09-10T08:29:20.594 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.445+0000 7ffb00c00640 1 -- 172.21.15.89:0/2651970045 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffaf4032ca0 con 0x7ffb04107680 2024-09-10T08:29:20.594 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.445+0000 7ffb00c00640 1 -- 172.21.15.89:0/2651970045 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffaf4037040 con 0x7ffb04107680 2024-09-10T08:29:20.594 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.445+0000 7ffb0ae00640 1 -- 172.21.15.89:0/2651970045 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb0419e070 con 0x7ffb04107680 2024-09-10T08:29:20.594 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.446+0000 7ffb0ae00640 1 -- 172.21.15.89:0/2651970045 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb041a0d60 con 0x7ffb04107680 2024-09-10T08:29:20.594 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.447+0000 7ffb0ae00640 1 -- 172.21.15.89:0/2651970045 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb04107b00 con 0x7ffb04107680 2024-09-10T08:29:20.594 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.447+0000 7ffb00c00640 1 -- 172.21.15.89:0/2651970045 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ffaf4045070 con 0x7ffb04107680 2024-09-10T08:29:20.594 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.447+0000 7ffb00c00640 1 --2- 172.21.15.89:0/2651970045 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7ffae8038fc0 0x7ffae803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:20.595 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.447+0000 7ffb00c00640 1 -- 172.21.15.89:0/2651970045 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ffaf406bfc0 con 0x7ffb04107680 2024-09-10T08:29:20.595 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.448+0000 7ffb03400640 1 --2- 172.21.15.89:0/2651970045 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7ffae8038fc0 0x7ffae803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:20.595 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.448+0000 7ffb03400640 1 --2- 172.21.15.89:0/2651970045 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7ffae8038fc0 0x7ffae803b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00098c0 tx=0x7ffaf0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:20.595 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.452+0000 7ffb00c00640 1 -- 172.21.15.89:0/2651970045 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffaf4045360 con 0x7ffb04107680 2024-09-10T08:29:20.595 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.628+0000 7ffb0ae00640 1 -- 172.21.15.89:0/2651970045 --> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7ffb041040c0 con 0x7ffae8038fc0 2024-09-10T08:29:20.595 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.902+0000 7ffb00c00640 1 -- 172.21.15.89:0/2651970045 <== mgr.14118 v2:172.21.15.89:6800/3794982914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffb041040c0 con 0x7ffae8038fc0 2024-09-10T08:29:20.595 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.905+0000 7ffb0ae00640 1 -- 172.21.15.89:0/2651970045 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7ffae8038fc0 msgr2=0x7ffae803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:20.595 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.905+0000 7ffb0ae00640 1 --2- 172.21.15.89:0/2651970045 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7ffae8038fc0 0x7ffae803b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00098c0 tx=0x7ffaf0006db0 comp rx=0 tx=0).stop 2024-09-10T08:29:20.595 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.905+0000 7ffb0ae00640 1 -- 172.21.15.89:0/2651970045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04107680 msgr2=0x7ffb0419d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:20.595 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.905+0000 7ffb0ae00640 1 --2- 172.21.15.89:0/2651970045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04107680 0x7ffb0419d8a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40097f0 tx=0x7ffaf4030ec0 comp rx=0 tx=0).stop 2024-09-10T08:29:20.595 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.905+0000 7ffb0ae00640 1 -- 172.21.15.89:0/2651970045 shutdown_connections 2024-09-10T08:29:20.595 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.905+0000 7ffb0ae00640 1 --2- 172.21.15.89:0/2651970045 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7ffae8038fc0 0x7ffae803b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:20.595 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.905+0000 7ffb0ae00640 1 --2- 172.21.15.89:0/2651970045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04107680 0x7ffb0419d8a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:20.596 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.905+0000 7ffb0ae00640 1 -- 172.21.15.89:0/2651970045 >> 172.21.15.89:0/2651970045 conn(0x7ffb04102eb0 msgr2=0x7ffb041039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:20.596 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.905+0000 7ffb0ae00640 1 -- 172.21.15.89:0/2651970045 shutdown_connections 2024-09-10T08:29:20.596 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:19.905+0000 7ffb0ae00640 1 -- 172.21.15.89:0/2651970045 wait complete. 2024-09-10T08:29:20.836 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:20 smithi089 ceph-mon[30757]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:29:20.837 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:20 smithi089 ceph-mon[30757]: Generating ssh key... 2024-09-10T08:29:20.837 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:20 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' 2024-09-10T08:29:20.837 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:20 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' 2024-09-10T08:29:21.905 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:21 smithi089 ceph-mon[30757]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:29:21.914 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCR6B/hB/QR/NuoQZZkUg8qnpfBD7OJvBAGxXJJ29ZccHVUSp+dtP2/X7U3FntyqpEPVLQpkmE02A5u5UlYOckToFMvXhOBcgkjdrBukpuPWEV7JABHXX7QD6q3SzUhnPoc8Xz4Hw5zByVQkLqYLNPfdqLlmXODJtf7ilujTc44duAwXbmSN95A3ZurHDz4NUZjJipuuUZ1rqNO+R/X/PE3hSkmPTShdVg5pbtyqoHEc0hyHaNDdzOTfgMRxJcNfe6QUzG3ngf5FXSrJDdBkZrXBocps+CH/dg+GFbbh93k2GEGhKY8dPMtEW3s37NESY3yl3LdJ+/Zda7Rflkse8/Xym6onFS+eQD0e3gWb5I0KS2joQIOTl4w4FSwHY1QxaNELks88TezYwrg+L5Tj/sdsGjrdWUjQo7k8KhSQcdH1dv80VBVEkoAG2iWfuWiB0/zfuZGHNH7n03SdEqW9vbZOGQxtEip5iV/zHr5F3t7TA2rKWTE03rG27QX1MDT5lM= ceph-91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:29:21.915 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.062+0000 7f0326a00640 1 Processor -- start 2024-09-10T08:29:21.915 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.063+0000 7f0326a00640 1 -- start start 2024-09-10T08:29:21.915 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.063+0000 7f0326a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f032007dee0 0x7f032007c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:21.915 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.063+0000 7f0326a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f032007c910 con 0x7f032007dee0 2024-09-10T08:29:21.915 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.063+0000 7f0325600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f032007dee0 0x7f032007c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:21.915 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.063+0000 7f0325600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f032007dee0 0x7f032007c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38692/0 (socket says 172.21.15.89:38692) 2024-09-10T08:29:21.915 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.063+0000 7f0325600640 1 -- 172.21.15.89:0/2976509857 learned_addr learned my addr 172.21.15.89:0/2976509857 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:21.915 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.064+0000 7f0325600640 1 -- 172.21.15.89:0/2976509857 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032007ca90 con 0x7f032007dee0 2024-09-10T08:29:21.915 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.064+0000 7f0325600640 1 --2- 172.21.15.89:0/2976509857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f032007dee0 0x7f032007c340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0314009e80 tx=0x7f031402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4a46771b87e1aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:21.915 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.065+0000 7f031fe00640 1 -- 172.21.15.89:0/2976509857 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0314036070 con 0x7f032007dee0 2024-09-10T08:29:21.915 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.065+0000 7f031fe00640 1 -- 172.21.15.89:0/2976509857 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f03140318c0 con 0x7f032007dee0 2024-09-10T08:29:21.915 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.065+0000 7f031fe00640 1 -- 172.21.15.89:0/2976509857 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0314007ab0 con 0x7f032007dee0 2024-09-10T08:29:21.916 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.065+0000 7f0326a00640 1 -- 172.21.15.89:0/2976509857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f032007dee0 msgr2=0x7f032007c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:21.916 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.065+0000 7f0326a00640 1 --2- 172.21.15.89:0/2976509857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f032007dee0 0x7f032007c340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0314009e80 tx=0x7f031402a7a0 comp rx=0 tx=0).stop 2024-09-10T08:29:21.916 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.065+0000 7f0326a00640 1 -- 172.21.15.89:0/2976509857 shutdown_connections 2024-09-10T08:29:21.916 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.065+0000 7f0326a00640 1 --2- 172.21.15.89:0/2976509857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f032007dee0 0x7f032007c340 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:21.916 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.065+0000 7f0326a00640 1 -- 172.21.15.89:0/2976509857 >> 172.21.15.89:0/2976509857 conn(0x7f0320102c10 msgr2=0x7f0320105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:21.916 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.065+0000 7f0326a00640 1 -- 172.21.15.89:0/2976509857 shutdown_connections 2024-09-10T08:29:21.916 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.065+0000 7f0326a00640 1 -- 172.21.15.89:0/2976509857 wait complete. 2024-09-10T08:29:21.916 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.066+0000 7f0326a00640 1 Processor -- start 2024-09-10T08:29:21.916 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.066+0000 7f0326a00640 1 -- start start 2024-09-10T08:29:21.916 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.067+0000 7f0326a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f032007dee0 0x7f03201a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:21.916 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.067+0000 7f0326a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0320109660 con 0x7f032007dee0 2024-09-10T08:29:21.916 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.067+0000 7f0325600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f032007dee0 0x7f03201a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:21.916 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.067+0000 7f0325600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f032007dee0 0x7f03201a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38706/0 (socket says 172.21.15.89:38706) 2024-09-10T08:29:21.917 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.067+0000 7f0325600640 1 -- 172.21.15.89:0/2011553875 learned_addr learned my addr 172.21.15.89:0/2011553875 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:21.917 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.067+0000 7f0325600640 1 -- 172.21.15.89:0/2011553875 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03201a2010 con 0x7f032007dee0 2024-09-10T08:29:21.917 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.067+0000 7f0325600640 1 --2- 172.21.15.89:0/2011553875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f032007dee0 0x7f03201a1ad0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f031402acc0 tx=0x7f0314031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:21.917 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.068+0000 7f031e000640 1 -- 172.21.15.89:0/2011553875 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0314036050 con 0x7f032007dee0 2024-09-10T08:29:21.917 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.068+0000 7f031e000640 1 -- 172.21.15.89:0/2011553875 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0314037040 con 0x7f032007dee0 2024-09-10T08:29:21.917 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.068+0000 7f031e000640 1 -- 172.21.15.89:0/2011553875 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f031403d890 con 0x7f032007dee0 2024-09-10T08:29:21.917 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.068+0000 7f0326a00640 1 -- 172.21.15.89:0/2011553875 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03201a22a0 con 0x7f032007dee0 2024-09-10T08:29:21.917 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.068+0000 7f0326a00640 1 -- 172.21.15.89:0/2011553875 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03201a4f90 con 0x7f032007dee0 2024-09-10T08:29:21.917 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.069+0000 7f031e000640 1 -- 172.21.15.89:0/2011553875 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0314044020 con 0x7f032007dee0 2024-09-10T08:29:21.917 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.069+0000 7f0326a00640 1 -- 172.21.15.89:0/2011553875 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f032007c3c0 con 0x7f032007dee0 2024-09-10T08:29:21.917 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.069+0000 7f031e000640 1 --2- 172.21.15.89:0/2011553875 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f0308038fc0 0x7f030803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:21.917 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.069+0000 7f031e000640 1 -- 172.21.15.89:0/2011553875 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0314042020 con 0x7f032007dee0 2024-09-10T08:29:21.917 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.070+0000 7f0324c00640 1 --2- 172.21.15.89:0/2011553875 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f0308038fc0 0x7f030803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:21.918 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.070+0000 7f0324c00640 1 --2- 172.21.15.89:0/2011553875 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f0308038fc0 0x7f030803b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f030c0098c0 tx=0x7f030c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:21.918 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.075+0000 7f031e000640 1 -- 172.21.15.89:0/2011553875 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f031404b020 con 0x7f032007dee0 2024-09-10T08:29:21.918 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.251+0000 7f0326a00640 1 -- 172.21.15.89:0/2011553875 --> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f032002dab0 con 0x7f0308038fc0 2024-09-10T08:29:21.918 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.251+0000 7f031e000640 1 -- 172.21.15.89:0/2011553875 <== mgr.14118 v2:172.21.15.89:6800/3794982914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f032002dab0 con 0x7f0308038fc0 2024-09-10T08:29:21.918 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.254+0000 7f0326a00640 1 -- 172.21.15.89:0/2011553875 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f0308038fc0 msgr2=0x7f030803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:21.918 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.254+0000 7f0326a00640 1 --2- 172.21.15.89:0/2011553875 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f0308038fc0 0x7f030803b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f030c0098c0 tx=0x7f030c006db0 comp rx=0 tx=0).stop 2024-09-10T08:29:21.918 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.254+0000 7f0326a00640 1 -- 172.21.15.89:0/2011553875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f032007dee0 msgr2=0x7f03201a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:21.918 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.254+0000 7f0326a00640 1 --2- 172.21.15.89:0/2011553875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f032007dee0 0x7f03201a1ad0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f031402acc0 tx=0x7f0314031ca0 comp rx=0 tx=0).stop 2024-09-10T08:29:21.918 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.254+0000 7f0326a00640 1 -- 172.21.15.89:0/2011553875 shutdown_connections 2024-09-10T08:29:21.918 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.254+0000 7f0326a00640 1 --2- 172.21.15.89:0/2011553875 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f0308038fc0 0x7f030803b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:21.918 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.254+0000 7f0326a00640 1 --2- 172.21.15.89:0/2011553875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f032007dee0 0x7f03201a1ad0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:21.918 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.254+0000 7f0326a00640 1 -- 172.21.15.89:0/2011553875 >> 172.21.15.89:0/2011553875 conn(0x7f0320102c10 msgr2=0x7f0320198730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:21.918 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.254+0000 7f0326a00640 1 -- 172.21.15.89:0/2011553875 shutdown_connections 2024-09-10T08:29:21.918 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:21.254+0000 7f0326a00640 1 -- 172.21.15.89:0/2011553875 wait complete. 2024-09-10T08:29:21.919 INFO:teuthology.orchestra.run.smithi089.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T08:29:21.919 INFO:teuthology.orchestra.run.smithi089.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T08:29:21.934 INFO:teuthology.orchestra.run.smithi089.stdout:Adding host smithi089... 2024-09-10T08:29:22.915 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:22 smithi089 ceph-mon[30757]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi089", "addr": "172.21.15.89", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:29:24.176 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:23 smithi089 ceph-mon[30757]: Deploying cephadm binary to smithi089 2024-09-10T08:29:27.148 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout Added host 'smithi089' with addr '172.21.15.89' 2024-09-10T08:29:27.148 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.432+0000 7fea81a00640 1 Processor -- start 2024-09-10T08:29:27.148 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.432+0000 7fea81a00640 1 -- start start 2024-09-10T08:29:27.148 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.433+0000 7fea81a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7c107680 0x7fea7c107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:27.148 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.433+0000 7fea81a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c107fc0 con 0x7fea7c107680 2024-09-10T08:29:27.149 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.433+0000 7fea7aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7c107680 0x7fea7c107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:27.149 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.433+0000 7fea7aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7c107680 0x7fea7c107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38722/0 (socket says 172.21.15.89:38722) 2024-09-10T08:29:27.149 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.433+0000 7fea7aa00640 1 -- 172.21.15.89:0/1469317434 learned_addr learned my addr 172.21.15.89:0/1469317434 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:27.149 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.433+0000 7fea7aa00640 1 -- 172.21.15.89:0/1469317434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7c108140 con 0x7fea7c107680 2024-09-10T08:29:27.149 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.434+0000 7fea7aa00640 1 --2- 172.21.15.89:0/1469317434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7c107680 0x7fea7c107a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009820 tx=0x7fea6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e3bbc9e4eeff60 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:27.149 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.435+0000 7fea7a000640 1 -- 172.21.15.89:0/1469317434 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea6c037070 con 0x7fea7c107680 2024-09-10T08:29:27.149 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.435+0000 7fea7a000640 1 -- 172.21.15.89:0/1469317434 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fea6c032900 con 0x7fea7c107680 2024-09-10T08:29:27.149 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.435+0000 7fea7a000640 1 -- 172.21.15.89:0/1469317434 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea6c030850 con 0x7fea7c107680 2024-09-10T08:29:27.149 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.435+0000 7fea81a00640 1 -- 172.21.15.89:0/1469317434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7c107680 msgr2=0x7fea7c107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:27.149 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.435+0000 7fea81a00640 1 --2- 172.21.15.89:0/1469317434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7c107680 0x7fea7c107a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009820 tx=0x7fea6c02a270 comp rx=0 tx=0).stop 2024-09-10T08:29:27.149 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.435+0000 7fea81a00640 1 -- 172.21.15.89:0/1469317434 shutdown_connections 2024-09-10T08:29:27.149 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.435+0000 7fea81a00640 1 --2- 172.21.15.89:0/1469317434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7c107680 0x7fea7c107a80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:27.149 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.435+0000 7fea81a00640 1 -- 172.21.15.89:0/1469317434 >> 172.21.15.89:0/1469317434 conn(0x7fea7c102eb0 msgr2=0x7fea7c1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:27.149 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.436+0000 7fea81a00640 1 -- 172.21.15.89:0/1469317434 shutdown_connections 2024-09-10T08:29:27.150 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.436+0000 7fea81a00640 1 -- 172.21.15.89:0/1469317434 wait complete. 2024-09-10T08:29:27.150 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.436+0000 7fea81a00640 1 Processor -- start 2024-09-10T08:29:27.150 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.437+0000 7fea81a00640 1 -- start start 2024-09-10T08:29:27.150 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.437+0000 7fea81a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7c107680 0x7fea7c07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:27.150 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.437+0000 7fea81a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c10baf0 con 0x7fea7c107680 2024-09-10T08:29:27.150 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.437+0000 7fea7aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7c107680 0x7fea7c07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:27.150 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.437+0000 7fea7aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7c107680 0x7fea7c07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38732/0 (socket says 172.21.15.89:38732) 2024-09-10T08:29:27.150 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.437+0000 7fea7aa00640 1 -- 172.21.15.89:0/3464189256 learned_addr learned my addr 172.21.15.89:0/3464189256 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:27.150 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.437+0000 7fea7aa00640 1 -- 172.21.15.89:0/3464189256 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7c07e0b0 con 0x7fea7c107680 2024-09-10T08:29:27.150 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.438+0000 7fea7aa00640 1 --2- 172.21.15.89:0/3464189256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7c107680 0x7fea7c07db70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0097f0 tx=0x7fea6c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:27.150 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.438+0000 7fea73e00640 1 -- 172.21.15.89:0/3464189256 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea6c042070 con 0x7fea7c107680 2024-09-10T08:29:27.150 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.438+0000 7fea73e00640 1 -- 172.21.15.89:0/3464189256 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fea6c032ca0 con 0x7fea7c107680 2024-09-10T08:29:27.150 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.438+0000 7fea73e00640 1 -- 172.21.15.89:0/3464189256 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea6c037040 con 0x7fea7c107680 2024-09-10T08:29:27.150 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.438+0000 7fea81a00640 1 -- 172.21.15.89:0/3464189256 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7c07c2a0 con 0x7fea7c107680 2024-09-10T08:29:27.151 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.438+0000 7fea81a00640 1 -- 172.21.15.89:0/3464189256 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea7c07c7e0 con 0x7fea7c107680 2024-09-10T08:29:27.151 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.439+0000 7fea73e00640 1 -- 172.21.15.89:0/3464189256 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fea6c045070 con 0x7fea7c107680 2024-09-10T08:29:27.151 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.439+0000 7fea81a00640 1 -- 172.21.15.89:0/3464189256 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea7c107b00 con 0x7fea7c107680 2024-09-10T08:29:27.151 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.440+0000 7fea73e00640 1 --2- 172.21.15.89:0/3464189256 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7fea60038fc0 0x7fea6003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:27.151 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.440+0000 7fea73e00640 1 -- 172.21.15.89:0/3464189256 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fea6c06bfe0 con 0x7fea7c107680 2024-09-10T08:29:27.151 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.440+0000 7fea72000640 1 --2- 172.21.15.89:0/3464189256 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7fea60038fc0 0x7fea6003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:27.151 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.440+0000 7fea72000640 1 --2- 172.21.15.89:0/3464189256 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7fea60038fc0 0x7fea6003b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fea640098c0 tx=0x7fea64006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:27.151 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.445+0000 7fea73e00640 1 -- 172.21.15.89:0/3464189256 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fea6c045360 con 0x7fea7c107680 2024-09-10T08:29:27.151 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:22.627+0000 7fea81a00640 1 -- 172.21.15.89:0/3464189256 --> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi089", "addr": "172.21.15.89", "target": ["mon-mgr", ""]}) -- 0x7fea7c0008d0 con 0x7fea60038fc0 2024-09-10T08:29:27.151 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:26.480+0000 7fea73e00640 1 -- 172.21.15.89:0/3464189256 <== mgr.14118 v2:172.21.15.89:6800/3794982914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fea7c0008d0 con 0x7fea60038fc0 2024-09-10T08:29:27.151 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:26.483+0000 7fea81a00640 1 -- 172.21.15.89:0/3464189256 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7fea60038fc0 msgr2=0x7fea6003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:27.151 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:26.483+0000 7fea81a00640 1 --2- 172.21.15.89:0/3464189256 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7fea60038fc0 0x7fea6003b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fea640098c0 tx=0x7fea64006db0 comp rx=0 tx=0).stop 2024-09-10T08:29:27.151 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:26.483+0000 7fea81a00640 1 -- 172.21.15.89:0/3464189256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7c107680 msgr2=0x7fea7c07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:27.151 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:26.483+0000 7fea81a00640 1 --2- 172.21.15.89:0/3464189256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7c107680 0x7fea7c07db70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0097f0 tx=0x7fea6c030ec0 comp rx=0 tx=0).stop 2024-09-10T08:29:27.152 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:26.484+0000 7fea81a00640 1 -- 172.21.15.89:0/3464189256 shutdown_connections 2024-09-10T08:29:27.152 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:26.484+0000 7fea81a00640 1 --2- 172.21.15.89:0/3464189256 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7fea60038fc0 0x7fea6003b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:27.152 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:26.484+0000 7fea81a00640 1 --2- 172.21.15.89:0/3464189256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea7c107680 0x7fea7c07db70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:27.152 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:26.484+0000 7fea81a00640 1 -- 172.21.15.89:0/3464189256 >> 172.21.15.89:0/3464189256 conn(0x7fea7c102eb0 msgr2=0x7fea7c1039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:27.152 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:26.484+0000 7fea81a00640 1 -- 172.21.15.89:0/3464189256 shutdown_connections 2024-09-10T08:29:27.152 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:26.484+0000 7fea81a00640 1 -- 172.21.15.89:0/3464189256 wait complete. 2024-09-10T08:29:27.152 INFO:teuthology.orchestra.run.smithi089.stdout:Deploying unmanaged mon service... 2024-09-10T08:29:27.734 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:27 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' 2024-09-10T08:29:27.735 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:27 smithi089 ceph-mon[30757]: Added host smithi089 2024-09-10T08:29:27.735 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:27 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:29:28.824 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T08:29:28.824 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.734+0000 7f6541a00640 1 Processor -- start 2024-09-10T08:29:28.824 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.734+0000 7f6541a00640 1 -- start start 2024-09-10T08:29:28.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.734+0000 7f6541a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653c07de80 0x7f653c07c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:28.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.734+0000 7f6541a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f653c07c8b0 con 0x7f653c07de80 2024-09-10T08:29:28.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.735+0000 7f653be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653c07de80 0x7f653c07c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:28.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.735+0000 7f653be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653c07de80 0x7f653c07c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34852/0 (socket says 172.21.15.89:34852) 2024-09-10T08:29:28.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.735+0000 7f653be00640 1 -- 172.21.15.89:0/2550759095 learned_addr learned my addr 172.21.15.89:0/2550759095 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:28.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.735+0000 7f653be00640 1 -- 172.21.15.89:0/2550759095 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653c07ca30 con 0x7f653c07de80 2024-09-10T08:29:28.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.736+0000 7f653be00640 1 --2- 172.21.15.89:0/2550759095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653c07de80 0x7f653c07c2e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f652c009820 tx=0x7f652c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f99700e66ef9c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:28.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.736+0000 7f653aa00640 1 -- 172.21.15.89:0/2550759095 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f652c037070 con 0x7f653c07de80 2024-09-10T08:29:28.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.736+0000 7f653aa00640 1 -- 172.21.15.89:0/2550759095 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f652c032900 con 0x7f653c07de80 2024-09-10T08:29:28.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.736+0000 7f653aa00640 1 -- 172.21.15.89:0/2550759095 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f652c030850 con 0x7f653c07de80 2024-09-10T08:29:28.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.737+0000 7f6541a00640 1 -- 172.21.15.89:0/2550759095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653c07de80 msgr2=0x7f653c07c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:28.825 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.737+0000 7f6541a00640 1 --2- 172.21.15.89:0/2550759095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653c07de80 0x7f653c07c2e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f652c009820 tx=0x7f652c02a270 comp rx=0 tx=0).stop 2024-09-10T08:29:28.826 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.737+0000 7f6541a00640 1 -- 172.21.15.89:0/2550759095 shutdown_connections 2024-09-10T08:29:28.826 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.737+0000 7f6541a00640 1 --2- 172.21.15.89:0/2550759095 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653c07de80 0x7f653c07c2e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:28.826 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.737+0000 7f6541a00640 1 -- 172.21.15.89:0/2550759095 >> 172.21.15.89:0/2550759095 conn(0x7f653c102c10 msgr2=0x7f653c105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:28.826 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.737+0000 7f6541a00640 1 -- 172.21.15.89:0/2550759095 shutdown_connections 2024-09-10T08:29:28.826 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.737+0000 7f6541a00640 1 -- 172.21.15.89:0/2550759095 wait complete. 2024-09-10T08:29:28.826 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.738+0000 7f6541a00640 1 Processor -- start 2024-09-10T08:29:28.826 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.738+0000 7f6541a00640 1 -- start start 2024-09-10T08:29:28.826 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.738+0000 7f6541a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653c07de80 0x7f653c19d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:28.826 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.738+0000 7f6541a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f653c10da70 con 0x7f653c07de80 2024-09-10T08:29:28.826 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.739+0000 7f653be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653c07de80 0x7f653c19d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:28.826 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.739+0000 7f653be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653c07de80 0x7f653c19d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34858/0 (socket says 172.21.15.89:34858) 2024-09-10T08:29:28.826 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.739+0000 7f653be00640 1 -- 172.21.15.89:0/1090337212 learned_addr learned my addr 172.21.15.89:0/1090337212 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:28.826 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.739+0000 7f653be00640 1 -- 172.21.15.89:0/1090337212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653c19dbf0 con 0x7f653c07de80 2024-09-10T08:29:28.827 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.739+0000 7f653be00640 1 --2- 172.21.15.89:0/1090337212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653c07de80 0x7f653c19d6b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f652c0097f0 tx=0x7f652c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:28.827 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.739+0000 7f6538c00640 1 -- 172.21.15.89:0/1090337212 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f652c042070 con 0x7f653c07de80 2024-09-10T08:29:28.827 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.739+0000 7f6538c00640 1 -- 172.21.15.89:0/1090337212 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f652c032ca0 con 0x7f653c07de80 2024-09-10T08:29:28.827 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.739+0000 7f6541a00640 1 -- 172.21.15.89:0/1090337212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f653c19de80 con 0x7f653c07de80 2024-09-10T08:29:28.827 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.739+0000 7f6538c00640 1 -- 172.21.15.89:0/1090337212 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f652c037040 con 0x7f653c07de80 2024-09-10T08:29:28.827 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.740+0000 7f6541a00640 1 -- 172.21.15.89:0/1090337212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f653c1a0b70 con 0x7f653c07de80 2024-09-10T08:29:28.827 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.741+0000 7f6541a00640 1 -- 172.21.15.89:0/1090337212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f653c07c360 con 0x7f653c07de80 2024-09-10T08:29:28.827 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.741+0000 7f6538c00640 1 -- 172.21.15.89:0/1090337212 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f652c045070 con 0x7f653c07de80 2024-09-10T08:29:28.827 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.741+0000 7f6538c00640 1 --2- 172.21.15.89:0/1090337212 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f6520038f70 0x7f652003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:28.827 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.741+0000 7f6538c00640 1 -- 172.21.15.89:0/1090337212 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f652c06c040 con 0x7f653c07de80 2024-09-10T08:29:28.827 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.741+0000 7f653b400640 1 --2- 172.21.15.89:0/1090337212 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f6520038f70 0x7f652003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:28.827 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.742+0000 7f653b400640 1 --2- 172.21.15.89:0/1090337212 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f6520038f70 0x7f652003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f65280098c0 tx=0x7f6528006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:28.827 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.746+0000 7f6538c00640 1 -- 172.21.15.89:0/1090337212 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f652c045360 con 0x7f653c07de80 2024-09-10T08:29:28.828 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.955+0000 7f6541a00640 1 -- 172.21.15.89:0/1090337212 --> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f653c103f20 con 0x7f6520038f70 2024-09-10T08:29:28.828 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.960+0000 7f6538c00640 1 -- 172.21.15.89:0/1090337212 <== mgr.14118 v2:172.21.15.89:6800/3794982914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f653c103f20 con 0x7f6520038f70 2024-09-10T08:29:28.828 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.962+0000 7f6541a00640 1 -- 172.21.15.89:0/1090337212 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f6520038f70 msgr2=0x7f652003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:28.828 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.962+0000 7f6541a00640 1 --2- 172.21.15.89:0/1090337212 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f6520038f70 0x7f652003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f65280098c0 tx=0x7f6528006db0 comp rx=0 tx=0).stop 2024-09-10T08:29:28.828 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.962+0000 7f6541a00640 1 -- 172.21.15.89:0/1090337212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653c07de80 msgr2=0x7f653c19d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:28.828 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.962+0000 7f6541a00640 1 --2- 172.21.15.89:0/1090337212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653c07de80 0x7f653c19d6b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f652c0097f0 tx=0x7f652c030ec0 comp rx=0 tx=0).stop 2024-09-10T08:29:28.828 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.963+0000 7f6541a00640 1 -- 172.21.15.89:0/1090337212 shutdown_connections 2024-09-10T08:29:28.828 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.963+0000 7f6541a00640 1 --2- 172.21.15.89:0/1090337212 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f6520038f70 0x7f652003b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:28.828 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.963+0000 7f6541a00640 1 --2- 172.21.15.89:0/1090337212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f653c07de80 0x7f653c19d6b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:28.829 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.963+0000 7f6541a00640 1 -- 172.21.15.89:0/1090337212 >> 172.21.15.89:0/1090337212 conn(0x7f653c102c10 msgr2=0x7f653c103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:28.829 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.963+0000 7f6541a00640 1 -- 172.21.15.89:0/1090337212 shutdown_connections 2024-09-10T08:29:28.829 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:27.963+0000 7f6541a00640 1 -- 172.21.15.89:0/1090337212 wait complete. 2024-09-10T08:29:28.829 INFO:teuthology.orchestra.run.smithi089.stdout:Deploying unmanaged mgr service... 2024-09-10T08:29:29.142 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:28 smithi089 ceph-mon[30757]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:29:29.142 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:28 smithi089 ceph-mon[30757]: Saving service mon spec with placement count:5 2024-09-10T08:29:29.142 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:28 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' 2024-09-10T08:29:31.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:30 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' 2024-09-10T08:29:31.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:30 smithi089 ceph-mon[30757]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:29:31.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:30 smithi089 ceph-mon[30757]: Saving service mgr spec with placement count:2 2024-09-10T08:29:31.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:30 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' 2024-09-10T08:29:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:30 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' 2024-09-10T08:29:31.120 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T08:29:31.120 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.142+0000 7f7205200640 1 Processor -- start 2024-09-10T08:29:31.120 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.143+0000 7f7205200640 1 -- start start 2024-09-10T08:29:31.120 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.143+0000 7f7205200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720007de80 0x7f720007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:31.121 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.143+0000 7f7205200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f720007c8b0 con 0x7f720007de80 2024-09-10T08:29:31.121 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.143+0000 7f71ff400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720007de80 0x7f720007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:31.121 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.144+0000 7f71ff400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720007de80 0x7f720007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34868/0 (socket says 172.21.15.89:34868) 2024-09-10T08:29:31.121 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.144+0000 7f71ff400640 1 -- 172.21.15.89:0/247211776 learned_addr learned my addr 172.21.15.89:0/247211776 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:31.121 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.144+0000 7f71ff400640 1 -- 172.21.15.89:0/247211776 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720007ca30 con 0x7f720007de80 2024-09-10T08:29:31.121 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.145+0000 7f71ff400640 1 --2- 172.21.15.89:0/247211776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720007de80 0x7f720007c2e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009820 tx=0x7f71f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d8f94540496a6a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:31.121 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.145+0000 7f71fe000640 1 -- 172.21.15.89:0/247211776 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71f0037070 con 0x7f720007de80 2024-09-10T08:29:31.121 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.145+0000 7f71fe000640 1 -- 172.21.15.89:0/247211776 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f0032920 con 0x7f720007de80 2024-09-10T08:29:31.121 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.145+0000 7f71fe000640 1 -- 172.21.15.89:0/247211776 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71f0030810 con 0x7f720007de80 2024-09-10T08:29:31.121 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.146+0000 7f7205200640 1 -- 172.21.15.89:0/247211776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720007de80 msgr2=0x7f720007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:31.121 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.146+0000 7f7205200640 1 --2- 172.21.15.89:0/247211776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720007de80 0x7f720007c2e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009820 tx=0x7f71f002a270 comp rx=0 tx=0).stop 2024-09-10T08:29:31.121 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.146+0000 7f7205200640 1 -- 172.21.15.89:0/247211776 shutdown_connections 2024-09-10T08:29:31.121 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.146+0000 7f7205200640 1 --2- 172.21.15.89:0/247211776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720007de80 0x7f720007c2e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:31.122 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.146+0000 7f7205200640 1 -- 172.21.15.89:0/247211776 >> 172.21.15.89:0/247211776 conn(0x7f7200102c10 msgr2=0x7f7200105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:31.122 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.146+0000 7f7205200640 1 -- 172.21.15.89:0/247211776 shutdown_connections 2024-09-10T08:29:31.122 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.146+0000 7f7205200640 1 -- 172.21.15.89:0/247211776 wait complete. 2024-09-10T08:29:31.122 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.148+0000 7f7205200640 1 Processor -- start 2024-09-10T08:29:31.122 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.148+0000 7f7205200640 1 -- start start 2024-09-10T08:29:31.122 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.148+0000 7f7205200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720007de80 0x7f720019d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:31.122 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.148+0000 7f7205200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f720010dad0 con 0x7f720007de80 2024-09-10T08:29:31.122 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.149+0000 7f71ff400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720007de80 0x7f720019d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:31.122 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.149+0000 7f71ff400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720007de80 0x7f720019d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34880/0 (socket says 172.21.15.89:34880) 2024-09-10T08:29:31.122 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.149+0000 7f71ff400640 1 -- 172.21.15.89:0/3982376328 learned_addr learned my addr 172.21.15.89:0/3982376328 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:31.122 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.149+0000 7f71ff400640 1 -- 172.21.15.89:0/3982376328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720019dc20 con 0x7f720007de80 2024-09-10T08:29:31.122 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.149+0000 7f71ff400640 1 --2- 172.21.15.89:0/3982376328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720007de80 0x7f720019d6e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f71f0030420 tx=0x7f71f0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:31.122 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.150+0000 7f71f7e00640 1 -- 172.21.15.89:0/3982376328 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71f0037040 con 0x7f720007de80 2024-09-10T08:29:31.122 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.150+0000 7f71f7e00640 1 -- 172.21.15.89:0/3982376328 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f0040070 con 0x7f720007de80 2024-09-10T08:29:31.123 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.150+0000 7f7205200640 1 -- 172.21.15.89:0/3982376328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720019deb0 con 0x7f720007de80 2024-09-10T08:29:31.123 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.150+0000 7f71f7e00640 1 -- 172.21.15.89:0/3982376328 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71f003a6e0 con 0x7f720007de80 2024-09-10T08:29:31.123 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.150+0000 7f7205200640 1 -- 172.21.15.89:0/3982376328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72001a0ba0 con 0x7f720007de80 2024-09-10T08:29:31.123 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.151+0000 7f7205200640 1 -- 172.21.15.89:0/3982376328 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f720007c360 con 0x7f720007de80 2024-09-10T08:29:31.123 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.151+0000 7f71f7e00640 1 -- 172.21.15.89:0/3982376328 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f71f003bb20 con 0x7f720007de80 2024-09-10T08:29:31.123 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.152+0000 7f71f7e00640 1 --2- 172.21.15.89:0/3982376328 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f71e4038fc0 0x7f71e403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:31.123 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.152+0000 7f71f7e00640 1 -- 172.21.15.89:0/3982376328 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f71f006c550 con 0x7f720007de80 2024-09-10T08:29:31.123 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.152+0000 7f71fea00640 1 --2- 172.21.15.89:0/3982376328 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f71e4038fc0 0x7f71e403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:31.123 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.153+0000 7f71fea00640 1 --2- 172.21.15.89:0/3982376328 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f71e4038fc0 0x7f71e403b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0098c0 tx=0x7f71ec006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:31.123 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.157+0000 7f71f7e00640 1 -- 172.21.15.89:0/3982376328 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f71f0043360 con 0x7f720007de80 2024-09-10T08:29:31.123 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.353+0000 7f7205200640 1 -- 172.21.15.89:0/3982376328 --> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f7200103f20 con 0x7f71e4038fc0 2024-09-10T08:29:31.123 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.358+0000 7f71f7e00640 1 -- 172.21.15.89:0/3982376328 <== mgr.14118 v2:172.21.15.89:6800/3794982914 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7200103f20 con 0x7f71e4038fc0 2024-09-10T08:29:31.123 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.360+0000 7f7205200640 1 -- 172.21.15.89:0/3982376328 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f71e4038fc0 msgr2=0x7f71e403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:31.124 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.360+0000 7f7205200640 1 --2- 172.21.15.89:0/3982376328 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f71e4038fc0 0x7f71e403b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0098c0 tx=0x7f71ec006db0 comp rx=0 tx=0).stop 2024-09-10T08:29:31.124 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.361+0000 7f7205200640 1 -- 172.21.15.89:0/3982376328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720007de80 msgr2=0x7f720019d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:31.124 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.361+0000 7f7205200640 1 --2- 172.21.15.89:0/3982376328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720007de80 0x7f720019d6e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f71f0030420 tx=0x7f71f0032670 comp rx=0 tx=0).stop 2024-09-10T08:29:31.124 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.361+0000 7f7205200640 1 -- 172.21.15.89:0/3982376328 shutdown_connections 2024-09-10T08:29:31.124 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.361+0000 7f7205200640 1 --2- 172.21.15.89:0/3982376328 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f71e4038fc0 0x7f71e403b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:31.124 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.361+0000 7f7205200640 1 --2- 172.21.15.89:0/3982376328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f720007de80 0x7f720019d6e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:31.124 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.361+0000 7f7205200640 1 -- 172.21.15.89:0/3982376328 >> 172.21.15.89:0/3982376328 conn(0x7f7200102c10 msgr2=0x7f7200103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:31.124 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.362+0000 7f7205200640 1 -- 172.21.15.89:0/3982376328 shutdown_connections 2024-09-10T08:29:31.124 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:30.362+0000 7f7205200640 1 -- 172.21.15.89:0/3982376328 wait complete. 2024-09-10T08:29:32.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.606+0000 7f48ae000640 1 Processor -- start 2024-09-10T08:29:32.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.606+0000 7f48ae000640 1 -- start start 2024-09-10T08:29:32.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.607+0000 7f48ae000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48a807ddc0 0x7f48a807c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:32.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.607+0000 7f48ae000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f48a807c800 con 0x7f48a807ddc0 2024-09-10T08:29:32.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.607+0000 7f48acc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48a807ddc0 0x7f48a807c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:32.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.607+0000 7f48acc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48a807ddc0 0x7f48a807c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34892/0 (socket says 172.21.15.89:34892) 2024-09-10T08:29:32.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.607+0000 7f48acc00640 1 -- 172.21.15.89:0/282592389 learned_addr learned my addr 172.21.15.89:0/282592389 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:32.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.608+0000 7f48acc00640 1 -- 172.21.15.89:0/282592389 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a807c980 con 0x7f48a807ddc0 2024-09-10T08:29:32.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.608+0000 7f48acc00640 1 --2- 172.21.15.89:0/282592389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48a807ddc0 0x7f48a807c2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f489c009820 tx=0x7f489c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cac631660b1363 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:32.619 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.609+0000 7f48a7400640 1 -- 172.21.15.89:0/282592389 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f489c037070 con 0x7f48a807ddc0 2024-09-10T08:29:32.620 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.609+0000 7f48a7400640 1 -- 172.21.15.89:0/282592389 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f489c032920 con 0x7f48a807ddc0 2024-09-10T08:29:32.620 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.609+0000 7f48a7400640 1 -- 172.21.15.89:0/282592389 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f489c030810 con 0x7f48a807ddc0 2024-09-10T08:29:32.620 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.609+0000 7f48ae000640 1 -- 172.21.15.89:0/282592389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48a807ddc0 msgr2=0x7f48a807c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:32.620 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.609+0000 7f48ae000640 1 --2- 172.21.15.89:0/282592389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48a807ddc0 0x7f48a807c2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f489c009820 tx=0x7f489c02a270 comp rx=0 tx=0).stop 2024-09-10T08:29:32.620 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.609+0000 7f48ae000640 1 -- 172.21.15.89:0/282592389 shutdown_connections 2024-09-10T08:29:32.620 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.609+0000 7f48ae000640 1 --2- 172.21.15.89:0/282592389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48a807ddc0 0x7f48a807c2c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:32.620 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.609+0000 7f48ae000640 1 -- 172.21.15.89:0/282592389 >> 172.21.15.89:0/282592389 conn(0x7f48a8102d00 msgr2=0x7f48a8105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:32.620 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.609+0000 7f48ae000640 1 -- 172.21.15.89:0/282592389 shutdown_connections 2024-09-10T08:29:32.620 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.609+0000 7f48ae000640 1 -- 172.21.15.89:0/282592389 wait complete. 2024-09-10T08:29:32.620 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.610+0000 7f48ae000640 1 Processor -- start 2024-09-10T08:29:32.620 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.610+0000 7f48ae000640 1 -- start start 2024-09-10T08:29:32.620 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.611+0000 7f48ae000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48a807ddc0 0x7f48a819d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:32.620 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.611+0000 7f48ae000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f48a810dad0 con 0x7f48a807ddc0 2024-09-10T08:29:32.621 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.611+0000 7f48acc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48a807ddc0 0x7f48a819d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:32.621 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.611+0000 7f48acc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48a807ddc0 0x7f48a819d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34902/0 (socket says 172.21.15.89:34902) 2024-09-10T08:29:32.621 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.611+0000 7f48acc00640 1 -- 172.21.15.89:0/2828966056 learned_addr learned my addr 172.21.15.89:0/2828966056 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:32.621 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.611+0000 7f48acc00640 1 -- 172.21.15.89:0/2828966056 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a819dc10 con 0x7f48a807ddc0 2024-09-10T08:29:32.621 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.612+0000 7f48acc00640 1 --2- 172.21.15.89:0/2828966056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48a807ddc0 0x7f48a819d6d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f489c030420 tx=0x7f489c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:32.621 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.612+0000 7f48a5600640 1 -- 172.21.15.89:0/2828966056 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f489c037040 con 0x7f48a807ddc0 2024-09-10T08:29:32.621 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.612+0000 7f48a5600640 1 -- 172.21.15.89:0/2828966056 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f489c040070 con 0x7f48a807ddc0 2024-09-10T08:29:32.621 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.612+0000 7f48ae000640 1 -- 172.21.15.89:0/2828966056 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a819dea0 con 0x7f48a807ddc0 2024-09-10T08:29:32.621 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.612+0000 7f48a5600640 1 -- 172.21.15.89:0/2828966056 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f489c03a6e0 con 0x7f48a807ddc0 2024-09-10T08:29:32.621 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.612+0000 7f48ae000640 1 -- 172.21.15.89:0/2828966056 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a81a0b90 con 0x7f48a807ddc0 2024-09-10T08:29:32.621 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.613+0000 7f48a5600640 1 -- 172.21.15.89:0/2828966056 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f489c03a9e0 con 0x7f48a807ddc0 2024-09-10T08:29:32.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.613+0000 7f48ae000640 1 -- 172.21.15.89:0/2828966056 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48880052c0 con 0x7f48a807ddc0 2024-09-10T08:29:32.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.614+0000 7f48a5600640 1 --2- 172.21.15.89:0/2828966056 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f4890038f70 0x7f489003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:32.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.614+0000 7f48a5600640 1 -- 172.21.15.89:0/2828966056 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f489c06c680 con 0x7f48a807ddc0 2024-09-10T08:29:32.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.614+0000 7f48a7e00640 1 --2- 172.21.15.89:0/2828966056 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f4890038f70 0x7f489003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:32.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.615+0000 7f48a7e00640 1 --2- 172.21.15.89:0/2828966056 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f4890038f70 0x7f489003b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f48940098c0 tx=0x7f4894006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:32.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.619+0000 7f48a5600640 1 -- 172.21.15.89:0/2828966056 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f489c043940 con 0x7f48a807ddc0 2024-09-10T08:29:32.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.810+0000 7f48ae000640 1 -- 172.21.15.89:0/2828966056 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f48880055b0 con 0x7f48a807ddc0 2024-09-10T08:29:32.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.813+0000 7f48a5600640 1 -- 172.21.15.89:0/2828966056 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f489c06bfd0 con 0x7f48a807ddc0 2024-09-10T08:29:32.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.816+0000 7f48ae000640 1 -- 172.21.15.89:0/2828966056 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f4890038f70 msgr2=0x7f489003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:32.622 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.816+0000 7f48ae000640 1 --2- 172.21.15.89:0/2828966056 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f4890038f70 0x7f489003b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f48940098c0 tx=0x7f4894006db0 comp rx=0 tx=0).stop 2024-09-10T08:29:32.623 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.816+0000 7f48ae000640 1 -- 172.21.15.89:0/2828966056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48a807ddc0 msgr2=0x7f48a819d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:32.623 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.816+0000 7f48ae000640 1 --2- 172.21.15.89:0/2828966056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48a807ddc0 0x7f48a819d6d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f489c030420 tx=0x7f489c032670 comp rx=0 tx=0).stop 2024-09-10T08:29:32.623 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.816+0000 7f48ae000640 1 -- 172.21.15.89:0/2828966056 shutdown_connections 2024-09-10T08:29:32.623 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.816+0000 7f48ae000640 1 --2- 172.21.15.89:0/2828966056 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f4890038f70 0x7f489003b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:32.623 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.816+0000 7f48ae000640 1 --2- 172.21.15.89:0/2828966056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48a807ddc0 0x7f48a819d6d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:32.623 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.816+0000 7f48ae000640 1 -- 172.21.15.89:0/2828966056 >> 172.21.15.89:0/2828966056 conn(0x7f48a8102d00 msgr2=0x7f48a8103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:32.623 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.816+0000 7f48ae000640 1 -- 172.21.15.89:0/2828966056 shutdown_connections 2024-09-10T08:29:32.623 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:31.816+0000 7f48ae000640 1 -- 172.21.15.89:0/2828966056 wait complete. 2024-09-10T08:29:32.878 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:32 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2828966056' entity='client.admin' 2024-09-10T08:29:34.394 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.546+0000 7f4088a00640 1 Processor -- start 2024-09-10T08:29:34.394 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.546+0000 7f4088a00640 1 -- start start 2024-09-10T08:29:34.394 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.547+0000 7f4088a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4084107440 0x7f4084107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:34.394 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.547+0000 7f4088a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4084107e10 con 0x7f4084107440 2024-09-10T08:29:34.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.547+0000 7f4082800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4084107440 0x7f4084107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:34.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.547+0000 7f4082800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4084107440 0x7f4084107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49874/0 (socket says 172.21.15.89:49874) 2024-09-10T08:29:34.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.547+0000 7f4082800640 1 -- 172.21.15.89:0/1757992206 learned_addr learned my addr 172.21.15.89:0/1757992206 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:34.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.548+0000 7f4082800640 1 -- 172.21.15.89:0/1757992206 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4084108640 con 0x7f4084107440 2024-09-10T08:29:34.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.549+0000 7f4082800640 1 --2- 172.21.15.89:0/1757992206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4084107440 0x7f4084107840 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4074009820 tx=0x7f407402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4e86f8c7c4df1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:34.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.549+0000 7f4081400640 1 -- 172.21.15.89:0/1757992206 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4074037070 con 0x7f4084107440 2024-09-10T08:29:34.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.549+0000 7f4081400640 1 -- 172.21.15.89:0/1757992206 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4074032920 con 0x7f4084107440 2024-09-10T08:29:34.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.549+0000 7f4081400640 1 -- 172.21.15.89:0/1757992206 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4074030810 con 0x7f4084107440 2024-09-10T08:29:34.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.550+0000 7f4088a00640 1 -- 172.21.15.89:0/1757992206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4084107440 msgr2=0x7f4084107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:34.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.550+0000 7f4088a00640 1 --2- 172.21.15.89:0/1757992206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4084107440 0x7f4084107840 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4074009820 tx=0x7f407402a270 comp rx=0 tx=0).stop 2024-09-10T08:29:34.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.550+0000 7f4088a00640 1 -- 172.21.15.89:0/1757992206 shutdown_connections 2024-09-10T08:29:34.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.550+0000 7f4088a00640 1 --2- 172.21.15.89:0/1757992206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4084107440 0x7f4084107840 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:34.395 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.550+0000 7f4088a00640 1 -- 172.21.15.89:0/1757992206 >> 172.21.15.89:0/1757992206 conn(0x7f4084102c10 msgr2=0x7f4084105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:34.396 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.550+0000 7f4088a00640 1 -- 172.21.15.89:0/1757992206 shutdown_connections 2024-09-10T08:29:34.396 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.550+0000 7f4088a00640 1 -- 172.21.15.89:0/1757992206 wait complete. 2024-09-10T08:29:34.396 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.551+0000 7f4088a00640 1 Processor -- start 2024-09-10T08:29:34.396 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.551+0000 7f4088a00640 1 -- start start 2024-09-10T08:29:34.396 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.551+0000 7f4088a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4084107440 0x7f408419b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:34.396 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.551+0000 7f4088a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f408410b9f0 con 0x7f4084107440 2024-09-10T08:29:34.396 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.551+0000 7f4082800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4084107440 0x7f408419b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:34.396 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.551+0000 7f4082800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4084107440 0x7f408419b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49888/0 (socket says 172.21.15.89:49888) 2024-09-10T08:29:34.396 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.551+0000 7f4082800640 1 -- 172.21.15.89:0/2022392675 learned_addr learned my addr 172.21.15.89:0/2022392675 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:34.396 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.552+0000 7f4082800640 1 -- 172.21.15.89:0/2022392675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408419ba30 con 0x7f4084107440 2024-09-10T08:29:34.396 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.552+0000 7f4082800640 1 --2- 172.21.15.89:0/2022392675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4084107440 0x7f408419b4f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f40740097f0 tx=0x7f40740327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:34.396 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.552+0000 7f407b400640 1 -- 172.21.15.89:0/2022392675 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4074037040 con 0x7f4084107440 2024-09-10T08:29:34.396 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.552+0000 7f407b400640 1 -- 172.21.15.89:0/2022392675 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4074030c40 con 0x7f4084107440 2024-09-10T08:29:34.397 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.552+0000 7f4088a00640 1 -- 172.21.15.89:0/2022392675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408419bcc0 con 0x7f4084107440 2024-09-10T08:29:34.397 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.552+0000 7f407b400640 1 -- 172.21.15.89:0/2022392675 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f407403a5b0 con 0x7f4084107440 2024-09-10T08:29:34.397 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.552+0000 7f4088a00640 1 -- 172.21.15.89:0/2022392675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408419e9b0 con 0x7f4084107440 2024-09-10T08:29:34.397 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.553+0000 7f407b400640 1 -- 172.21.15.89:0/2022392675 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f407403a7f0 con 0x7f4084107440 2024-09-10T08:29:34.397 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.554+0000 7f4088a00640 1 -- 172.21.15.89:0/2022392675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f408410bb70 con 0x7f4084107440 2024-09-10T08:29:34.397 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.554+0000 7f407b400640 1 --2- 172.21.15.89:0/2022392675 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f4068038f70 0x7f406803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:34.397 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.554+0000 7f407b400640 1 -- 172.21.15.89:0/2022392675 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f407406c4b0 con 0x7f4084107440 2024-09-10T08:29:34.397 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.554+0000 7f4081e00640 1 --2- 172.21.15.89:0/2022392675 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f4068038f70 0x7f406803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:34.397 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.555+0000 7f4081e00640 1 --2- 172.21.15.89:0/2022392675 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f4068038f70 0x7f406803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f40700098c0 tx=0x7f4070006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:34.397 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.563+0000 7f407b400640 1 -- 172.21.15.89:0/2022392675 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40740371e0 con 0x7f4084107440 2024-09-10T08:29:34.397 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.748+0000 7f4088a00640 1 -- 172.21.15.89:0/2022392675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f40841078c0 con 0x7f4084107440 2024-09-10T08:29:34.397 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.752+0000 7f407b400640 1 -- 172.21.15.89:0/2022392675 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f407406be00 con 0x7f4084107440 2024-09-10T08:29:34.397 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.755+0000 7f4088a00640 1 -- 172.21.15.89:0/2022392675 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f4068038f70 msgr2=0x7f406803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:34.398 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.755+0000 7f4088a00640 1 --2- 172.21.15.89:0/2022392675 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f4068038f70 0x7f406803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f40700098c0 tx=0x7f4070006db0 comp rx=0 tx=0).stop 2024-09-10T08:29:34.398 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.755+0000 7f4088a00640 1 -- 172.21.15.89:0/2022392675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4084107440 msgr2=0x7f408419b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:34.398 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.755+0000 7f4088a00640 1 --2- 172.21.15.89:0/2022392675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4084107440 0x7f408419b4f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f40740097f0 tx=0x7f40740327b0 comp rx=0 tx=0).stop 2024-09-10T08:29:34.398 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.755+0000 7f4088a00640 1 -- 172.21.15.89:0/2022392675 shutdown_connections 2024-09-10T08:29:34.398 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.755+0000 7f4088a00640 1 --2- 172.21.15.89:0/2022392675 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f4068038f70 0x7f406803b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:34.398 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.755+0000 7f4088a00640 1 --2- 172.21.15.89:0/2022392675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4084107440 0x7f408419b4f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:34.398 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.755+0000 7f4088a00640 1 -- 172.21.15.89:0/2022392675 >> 172.21.15.89:0/2022392675 conn(0x7f4084102c10 msgr2=0x7f4084103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:34.398 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.755+0000 7f4088a00640 1 -- 172.21.15.89:0/2022392675 shutdown_connections 2024-09-10T08:29:34.398 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:33.756+0000 7f4088a00640 1 -- 172.21.15.89:0/2022392675 wait complete. 2024-09-10T08:29:34.398 INFO:teuthology.orchestra.run.smithi089.stdout:Enabling the dashboard module... 2024-09-10T08:29:34.661 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:34 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' 2024-09-10T08:29:34.661 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:34 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2022392675' entity='client.admin' 2024-09-10T08:29:35.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:35 smithi089 ceph-mon[30757]: from='mgr.14118 172.21.15.89:0/3311347131' entity='mgr.x' 2024-09-10T08:29:35.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:35 smithi089 ceph-mon[30757]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:29:35.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:35 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1713893112' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-10T08:29:35.803 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:35 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: ignoring --setuser ceph since I am not root 2024-09-10T08:29:35.803 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:35 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: ignoring --setgroup ceph since I am not root 2024-09-10T08:29:36.070 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:35 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:35.812+0000 7f53a5fed040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T08:29:36.070 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:35 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:35.882+0000 7f53a5fed040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T08:29:36.070 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:36 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:36.069+0000 7f53a5fed040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T08:29:36.323 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:36 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:36.149+0000 7f53a5fed040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T08:29:36.323 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:36 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:36.322+0000 7f53a5fed040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T08:29:36.613 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:36 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1713893112' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T08:29:36.613 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:36 smithi089 ceph-mon[30757]: mgrmap e9: x(active, since 21s) 2024-09-10T08:29:36.613 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:36 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:36.487+0000 7f53a5fed040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T08:29:36.889 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:36 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:36.862+0000 7f53a5fed040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T08:29:36.900 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.914+0000 7fbb21400640 1 Processor -- start 2024-09-10T08:29:36.901 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.914+0000 7fbb21400640 1 -- start start 2024-09-10T08:29:36.901 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.914+0000 7fbb21400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c07c340 0x7fbb1c07c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:36.901 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.914+0000 7fbb21400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c07e070 con 0x7fbb1c07c340 2024-09-10T08:29:36.901 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.915+0000 7fbb1a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c07c340 0x7fbb1c07c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:36.901 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.915+0000 7fbb1a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c07c340 0x7fbb1c07c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49898/0 (socket says 172.21.15.89:49898) 2024-09-10T08:29:36.901 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.915+0000 7fbb1a800640 1 -- 172.21.15.89:0/1988670400 learned_addr learned my addr 172.21.15.89:0/1988670400 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:36.901 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.915+0000 7fbb1a800640 1 -- 172.21.15.89:0/1988670400 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1c07cc80 con 0x7fbb1c07c340 2024-09-10T08:29:36.901 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.916+0000 7fbb1a800640 1 --2- 172.21.15.89:0/1988670400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c07c340 0x7fbb1c07c740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009820 tx=0x7fbb0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817391678ed24ebf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:36.901 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.916+0000 7fbb19400640 1 -- 172.21.15.89:0/1988670400 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb0c037070 con 0x7fbb1c07c340 2024-09-10T08:29:36.901 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.916+0000 7fbb19400640 1 -- 172.21.15.89:0/1988670400 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb0c032920 con 0x7fbb1c07c340 2024-09-10T08:29:36.901 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.916+0000 7fbb19400640 1 -- 172.21.15.89:0/1988670400 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb0c030810 con 0x7fbb1c07c340 2024-09-10T08:29:36.901 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.917+0000 7fbb21400640 1 -- 172.21.15.89:0/1988670400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c07c340 msgr2=0x7fbb1c07c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:36.902 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.917+0000 7fbb21400640 1 --2- 172.21.15.89:0/1988670400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c07c340 0x7fbb1c07c740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009820 tx=0x7fbb0c02a270 comp rx=0 tx=0).stop 2024-09-10T08:29:36.902 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.917+0000 7fbb21400640 1 -- 172.21.15.89:0/1988670400 shutdown_connections 2024-09-10T08:29:36.902 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.917+0000 7fbb21400640 1 --2- 172.21.15.89:0/1988670400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c07c340 0x7fbb1c07c740 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:36.902 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.917+0000 7fbb21400640 1 -- 172.21.15.89:0/1988670400 >> 172.21.15.89:0/1988670400 conn(0x7fbb1c102dc0 msgr2=0x7fbb1c1051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:36.902 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.918+0000 7fbb21400640 1 -- 172.21.15.89:0/1988670400 shutdown_connections 2024-09-10T08:29:36.902 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.918+0000 7fbb21400640 1 -- 172.21.15.89:0/1988670400 wait complete. 2024-09-10T08:29:36.902 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.919+0000 7fbb21400640 1 Processor -- start 2024-09-10T08:29:36.902 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.919+0000 7fbb21400640 1 -- start start 2024-09-10T08:29:36.902 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.919+0000 7fbb21400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c07c340 0x7fbb1c1a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:36.902 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.919+0000 7fbb21400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1099b0 con 0x7fbb1c07c340 2024-09-10T08:29:36.902 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.919+0000 7fbb1a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c07c340 0x7fbb1c1a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:36.902 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.919+0000 7fbb1a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c07c340 0x7fbb1c1a1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49910/0 (socket says 172.21.15.89:49910) 2024-09-10T08:29:36.902 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.919+0000 7fbb1a800640 1 -- 172.21.15.89:0/1713893112 learned_addr learned my addr 172.21.15.89:0/1713893112 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:36.903 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.920+0000 7fbb1a800640 1 -- 172.21.15.89:0/1713893112 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1c1a22a0 con 0x7fbb1c07c340 2024-09-10T08:29:36.903 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.920+0000 7fbb1a800640 1 --2- 172.21.15.89:0/1713893112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c07c340 0x7fbb1c1a1d60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0097f0 tx=0x7fbb0c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:36.903 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.920+0000 7fbb13400640 1 -- 172.21.15.89:0/1713893112 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb0c037040 con 0x7fbb1c07c340 2024-09-10T08:29:36.903 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.920+0000 7fbb13400640 1 -- 172.21.15.89:0/1713893112 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb0c030cc0 con 0x7fbb1c07c340 2024-09-10T08:29:36.903 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.920+0000 7fbb13400640 1 -- 172.21.15.89:0/1713893112 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb0c03a6b0 con 0x7fbb1c07c340 2024-09-10T08:29:36.903 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.920+0000 7fbb21400640 1 -- 172.21.15.89:0/1713893112 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb1c1a2530 con 0x7fbb1c07c340 2024-09-10T08:29:36.903 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.921+0000 7fbb21400640 1 -- 172.21.15.89:0/1713893112 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb1c1a5220 con 0x7fbb1c07c340 2024-09-10T08:29:36.903 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.922+0000 7fbb13400640 1 -- 172.21.15.89:0/1713893112 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbb0c03a850 con 0x7fbb1c07c340 2024-09-10T08:29:36.903 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.922+0000 7fbb21400640 1 -- 172.21.15.89:0/1713893112 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb1c07c7c0 con 0x7fbb1c07c340 2024-09-10T08:29:36.903 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.922+0000 7fbb13400640 1 --2- 172.21.15.89:0/1713893112 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7fbb00038fc0 0x7fbb0003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:36.903 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.922+0000 7fbb13400640 1 -- 172.21.15.89:0/1713893112 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbb0c06c560 con 0x7fbb1c07c340 2024-09-10T08:29:36.903 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.923+0000 7fbb19e00640 1 --2- 172.21.15.89:0/1713893112 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7fbb00038fc0 0x7fbb0003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:36.903 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.923+0000 7fbb19e00640 1 --2- 172.21.15.89:0/1713893112 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7fbb00038fc0 0x7fbb0003b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbb080098c0 tx=0x7fbb08006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:36.904 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:34.927+0000 7fbb13400640 1 -- 172.21.15.89:0/1713893112 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb0c042e10 con 0x7fbb1c07c340 2024-09-10T08:29:36.904 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:35.180+0000 7fbb21400640 1 -- 172.21.15.89:0/1713893112 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fbb1c02d8a0 con 0x7fbb1c07c340 2024-09-10T08:29:36.904 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:35.565+0000 7fbb13400640 1 -- 172.21.15.89:0/1713893112 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fbb0c06beb0 con 0x7fbb1c07c340 2024-09-10T08:29:36.904 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:35.567+0000 7fbb13400640 1 -- 172.21.15.89:0/1713893112 <== mon.0 v2:172.21.15.89:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fbb0c03b3f0 con 0x7fbb1c07c340 2024-09-10T08:29:36.904 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:35.568+0000 7fbb21400640 1 -- 172.21.15.89:0/1713893112 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7fbb00038fc0 msgr2=0x7fbb0003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:36.904 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:35.568+0000 7fbb21400640 1 --2- 172.21.15.89:0/1713893112 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7fbb00038fc0 0x7fbb0003b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbb080098c0 tx=0x7fbb08006db0 comp rx=0 tx=0).stop 2024-09-10T08:29:36.904 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:35.568+0000 7fbb21400640 1 -- 172.21.15.89:0/1713893112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c07c340 msgr2=0x7fbb1c1a1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:36.904 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:35.568+0000 7fbb21400640 1 --2- 172.21.15.89:0/1713893112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c07c340 0x7fbb1c1a1d60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0097f0 tx=0x7fbb0c0327b0 comp rx=0 tx=0).stop 2024-09-10T08:29:36.904 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:35.568+0000 7fbb21400640 1 -- 172.21.15.89:0/1713893112 shutdown_connections 2024-09-10T08:29:36.904 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:35.568+0000 7fbb21400640 1 --2- 172.21.15.89:0/1713893112 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7fbb00038fc0 0x7fbb0003b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:36.904 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:35.568+0000 7fbb21400640 1 --2- 172.21.15.89:0/1713893112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c07c340 0x7fbb1c1a1d60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:36.904 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:35.568+0000 7fbb21400640 1 -- 172.21.15.89:0/1713893112 >> 172.21.15.89:0/1713893112 conn(0x7fbb1c102dc0 msgr2=0x7fbb1c198900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:36.904 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:35.569+0000 7fbb21400640 1 -- 172.21.15.89:0/1713893112 shutdown_connections 2024-09-10T08:29:36.905 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:35.569+0000 7fbb21400640 1 -- 172.21.15.89:0/1713893112 wait complete. 2024-09-10T08:29:37.160 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:36 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:36.942+0000 7f53a5fed040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T08:29:37.160 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:37 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:37.159+0000 7f53a5fed040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T08:29:37.980 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:37 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/720948505' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T08:29:38.315 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2024-09-10T08:29:38.315 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T08:29:38.315 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T08:29:38.315 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T08:29:38.315 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T08:29:38.315 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2024-09-10T08:29:38.315 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.399+0000 7f1afb200640 1 Processor -- start 2024-09-10T08:29:38.315 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.400+0000 7f1afb200640 1 -- start start 2024-09-10T08:29:38.315 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.400+0000 7f1afb200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1afc107440 0x7f1afc107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:38.315 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.400+0000 7f1afb200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc107e10 con 0x7f1afc107440 2024-09-10T08:29:38.315 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.400+0000 7f1af9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1afc107440 0x7f1afc107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:38.316 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.400+0000 7f1af9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1afc107440 0x7f1afc107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49928/0 (socket says 172.21.15.89:49928) 2024-09-10T08:29:38.316 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.400+0000 7f1af9e00640 1 -- 172.21.15.89:0/4076735039 learned_addr learned my addr 172.21.15.89:0/4076735039 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:38.316 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.401+0000 7f1af9e00640 1 -- 172.21.15.89:0/4076735039 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1afc108640 con 0x7f1afc107440 2024-09-10T08:29:38.316 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.402+0000 7f1af9e00640 1 --2- 172.21.15.89:0/4076735039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1afc107440 0x7f1afc107840 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1aec009820 tx=0x7f1aec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1160cb008ddc431f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:38.316 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.402+0000 7f1af8a00640 1 -- 172.21.15.89:0/4076735039 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aec037070 con 0x7f1afc107440 2024-09-10T08:29:38.316 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.402+0000 7f1af8a00640 1 -- 172.21.15.89:0/4076735039 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aec032920 con 0x7f1afc107440 2024-09-10T08:29:38.316 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.402+0000 7f1af8a00640 1 -- 172.21.15.89:0/4076735039 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aec030810 con 0x7f1afc107440 2024-09-10T08:29:38.316 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.403+0000 7f1afb200640 1 -- 172.21.15.89:0/4076735039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1afc107440 msgr2=0x7f1afc107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:38.316 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.403+0000 7f1afb200640 1 --2- 172.21.15.89:0/4076735039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1afc107440 0x7f1afc107840 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1aec009820 tx=0x7f1aec02a270 comp rx=0 tx=0).stop 2024-09-10T08:29:38.316 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.403+0000 7f1afb200640 1 -- 172.21.15.89:0/4076735039 shutdown_connections 2024-09-10T08:29:38.316 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.403+0000 7f1afb200640 1 --2- 172.21.15.89:0/4076735039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1afc107440 0x7f1afc107840 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:38.317 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.403+0000 7f1afb200640 1 -- 172.21.15.89:0/4076735039 >> 172.21.15.89:0/4076735039 conn(0x7f1afc102c10 msgr2=0x7f1afc105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:38.317 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.403+0000 7f1afb200640 1 -- 172.21.15.89:0/4076735039 shutdown_connections 2024-09-10T08:29:38.317 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.403+0000 7f1afb200640 1 -- 172.21.15.89:0/4076735039 wait complete. 2024-09-10T08:29:38.317 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.404+0000 7f1afb200640 1 Processor -- start 2024-09-10T08:29:38.317 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.405+0000 7f1afb200640 1 -- start start 2024-09-10T08:29:38.317 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.405+0000 7f1afb200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1afc107440 0x7f1afc07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:38.317 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.405+0000 7f1afb200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc10b9f0 con 0x7f1afc107440 2024-09-10T08:29:38.317 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.405+0000 7f1af9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1afc107440 0x7f1afc07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:38.317 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.405+0000 7f1af9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1afc107440 0x7f1afc07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49930/0 (socket says 172.21.15.89:49930) 2024-09-10T08:29:38.317 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.405+0000 7f1af9e00640 1 -- 172.21.15.89:0/720948505 learned_addr learned my addr 172.21.15.89:0/720948505 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:38.317 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.406+0000 7f1af9e00640 1 -- 172.21.15.89:0/720948505 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1afc07e0b0 con 0x7f1afc107440 2024-09-10T08:29:38.318 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.406+0000 7f1af9e00640 1 --2- 172.21.15.89:0/720948505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1afc107440 0x7f1afc07db70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1aec030560 tx=0x7f1aec032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:38.318 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.406+0000 7f1af2a00640 1 -- 172.21.15.89:0/720948505 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aec037040 con 0x7f1afc107440 2024-09-10T08:29:38.318 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.406+0000 7f1af2a00640 1 -- 172.21.15.89:0/720948505 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aec040070 con 0x7f1afc107440 2024-09-10T08:29:38.318 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.406+0000 7f1afb200640 1 -- 172.21.15.89:0/720948505 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1afc07c2a0 con 0x7f1afc107440 2024-09-10T08:29:38.318 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.406+0000 7f1af2a00640 1 -- 172.21.15.89:0/720948505 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aec03a6d0 con 0x7f1afc107440 2024-09-10T08:29:38.318 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.406+0000 7f1afb200640 1 -- 172.21.15.89:0/720948505 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1afc07c7e0 con 0x7f1afc107440 2024-09-10T08:29:38.318 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.407+0000 7f1afb200640 1 -- 172.21.15.89:0/720948505 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1adc0052c0 con 0x7f1afc107440 2024-09-10T08:29:38.318 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.408+0000 7f1af2a00640 1 -- 172.21.15.89:0/720948505 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f1aec03a940 con 0x7f1afc107440 2024-09-10T08:29:38.318 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.408+0000 7f1af2a00640 1 --2- 172.21.15.89:0/720948505 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f1ae0038fc0 0x7f1ae003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:38.318 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.408+0000 7f1af2a00640 1 -- 172.21.15.89:0/720948505 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1aec06c620 con 0x7f1afc107440 2024-09-10T08:29:38.318 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.408+0000 7f1af9400640 1 -- 172.21.15.89:0/720948505 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f1ae0038fc0 msgr2=0x7f1ae003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:6800/3794982914 2024-09-10T08:29:38.318 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.408+0000 7f1af9400640 1 --2- 172.21.15.89:0/720948505 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f1ae0038fc0 0x7f1ae003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T08:29:38.318 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.417+0000 7f1af2a00640 1 -- 172.21.15.89:0/720948505 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1aec044c70 con 0x7f1afc107440 2024-09-10T08:29:38.318 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.609+0000 7f1af9400640 1 -- 172.21.15.89:0/720948505 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f1ae0038fc0 msgr2=0x7f1ae003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:6800/3794982914 2024-09-10T08:29:38.319 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.609+0000 7f1af9400640 1 --2- 172.21.15.89:0/720948505 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f1ae0038fc0 0x7f1ae003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T08:29:38.319 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.648+0000 7f1afb200640 1 -- 172.21.15.89:0/720948505 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f1adc005dc0 con 0x7f1afc107440 2024-09-10T08:29:38.319 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.649+0000 7f1af2a00640 1 -- 172.21.15.89:0/720948505 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f1aec0328f0 con 0x7f1afc107440 2024-09-10T08:29:38.319 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.651+0000 7f1afb200640 1 -- 172.21.15.89:0/720948505 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f1ae0038fc0 msgr2=0x7f1ae003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:29:38.319 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.651+0000 7f1afb200640 1 --2- 172.21.15.89:0/720948505 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f1ae0038fc0 0x7f1ae003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:38.319 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.651+0000 7f1afb200640 1 -- 172.21.15.89:0/720948505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1afc107440 msgr2=0x7f1afc07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:38.319 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.651+0000 7f1afb200640 1 --2- 172.21.15.89:0/720948505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1afc107440 0x7f1afc07db70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1aec030560 tx=0x7f1aec032670 comp rx=0 tx=0).stop 2024-09-10T08:29:38.319 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.651+0000 7f1afb200640 1 -- 172.21.15.89:0/720948505 shutdown_connections 2024-09-10T08:29:38.319 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.651+0000 7f1afb200640 1 --2- 172.21.15.89:0/720948505 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f1ae0038fc0 0x7f1ae003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:38.319 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.651+0000 7f1afb200640 1 --2- 172.21.15.89:0/720948505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1afc107440 0x7f1afc07db70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:38.319 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.651+0000 7f1afb200640 1 -- 172.21.15.89:0/720948505 >> 172.21.15.89:0/720948505 conn(0x7f1afc102c10 msgr2=0x7f1afc103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:38.319 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.651+0000 7f1afb200640 1 -- 172.21.15.89:0/720948505 shutdown_connections 2024-09-10T08:29:38.319 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:37.651+0000 7f1afb200640 1 -- 172.21.15.89:0/720948505 wait complete. 2024-09-10T08:29:38.320 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for the mgr to restart... 2024-09-10T08:29:38.320 INFO:teuthology.orchestra.run.smithi089.stdout:Waiting for mgr epoch 9... 2024-09-10T08:29:38.552 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:38 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:38.291+0000 7f53a5fed040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T08:29:38.552 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:38 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:38.378+0000 7f53a5fed040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T08:29:39.602 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:39 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:39.305+0000 7f53a5fed040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T08:29:39.602 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:39 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:39.440+0000 7f53a5fed040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T08:29:39.602 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:39 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:39.601+0000 7f53a5fed040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T08:29:39.923 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:39 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:39.673+0000 7f53a5fed040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T08:29:39.923 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:39 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:39.842+0000 7f53a5fed040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T08:29:39.923 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:39 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:39.923+0000 7f53a5fed040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T08:29:40.247 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:39 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:39.993+0000 7f53a5fed040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T08:29:40.247 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:40 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:40.170+0000 7f53a5fed040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T08:29:40.247 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:40 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:40.246+0000 7f53a5fed040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T08:29:40.552 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:40 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:40.324+0000 7f53a5fed040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T08:29:40.552 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:40 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:40.419+0000 7f53a5fed040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T08:29:40.552 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:40 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:40.489+0000 7f53a5fed040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T08:29:41.052 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:40 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:40.557+0000 7f53a5fed040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T08:29:41.708 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:41 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:41.468+0000 7f53a5fed040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T08:29:41.708 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:41 smithi089 ceph-mon[30757]: Active manager daemon x restarted 2024-09-10T08:29:41.708 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:41 smithi089 ceph-mon[30757]: Activating manager daemon x 2024-09-10T08:29:41.708 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:41 smithi089 ceph-mon[30757]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T08:29:41.708 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:41 smithi089 ceph-mon[30757]: mgrmap e10: x(active, starting, since 0.00407735s) 2024-09-10T08:29:41.709 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:29:41.709 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T08:29:41.709 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T08:29:41.709 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T08:29:41.709 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T08:29:41.709 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:41 smithi089 ceph-mon[30757]: Manager daemon x is now available 2024-09-10T08:29:41.709 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:29:42.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T08:29:42.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T08:29:42.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-mon[30757]: mgrmap e11: x(active, since 1.00806s) 2024-09-10T08:29:43.296 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:42.879+0000 7f5302800640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:08:29:42] ENGINE Error in HTTPServer.serve 2024-09-10T08:29:43.296 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: Traceback (most recent call last): 2024-09-10T08:29:43.296 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T08:29:43.296 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: self._connections.run(self.expiration_interval) 2024-09-10T08:29:43.296 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T08:29:43.296 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: self._run(expiration_interval) 2024-09-10T08:29:43.296 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T08:29:43.296 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T08:29:43.297 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T08:29:43.297 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T08:29:43.297 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T08:29:43.297 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: s = self.context.wrap_socket( 2024-09-10T08:29:43.297 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T08:29:43.297 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: return self.sslsocket_class._create( 2024-09-10T08:29:43.297 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T08:29:43.297 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: self.do_handshake() 2024-09-10T08:29:43.297 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T08:29:43.297 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: self._sslobj.do_handshake() 2024-09-10T08:29:43.297 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T08:29:43.297 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:29:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:29:43.307 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout { 2024-09-10T08:29:43.307 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T08:29:43.307 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T08:29:43.307 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout } 2024-09-10T08:29:43.307 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.839+0000 7f75b3e00640 1 Processor -- start 2024-09-10T08:29:43.307 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.839+0000 7f75b3e00640 1 -- start start 2024-09-10T08:29:43.307 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.839+0000 7f75b3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75b407de80 0x7f75b407c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:43.308 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.839+0000 7f75b3e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f75b407c8b0 con 0x7f75b407de80 2024-09-10T08:29:43.308 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.840+0000 7f75b2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75b407de80 0x7f75b407c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:43.308 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.840+0000 7f75b2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75b407de80 0x7f75b407c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49936/0 (socket says 172.21.15.89:49936) 2024-09-10T08:29:43.308 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.840+0000 7f75b2a00640 1 -- 172.21.15.89:0/3811525827 learned_addr learned my addr 172.21.15.89:0/3811525827 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:43.308 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.840+0000 7f75b2a00640 1 -- 172.21.15.89:0/3811525827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b407ca30 con 0x7f75b407de80 2024-09-10T08:29:43.308 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.841+0000 7f75b2a00640 1 --2- 172.21.15.89:0/3811525827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75b407de80 0x7f75b407c2e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f75a4009820 tx=0x7f75a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06f3b960ba8513d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:43.308 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.841+0000 7f75b1600640 1 -- 172.21.15.89:0/3811525827 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75a4037070 con 0x7f75b407de80 2024-09-10T08:29:43.308 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.841+0000 7f75b1600640 1 -- 172.21.15.89:0/3811525827 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a4032920 con 0x7f75b407de80 2024-09-10T08:29:43.308 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.841+0000 7f75b1600640 1 -- 172.21.15.89:0/3811525827 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75a4030810 con 0x7f75b407de80 2024-09-10T08:29:43.308 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.842+0000 7f75b3e00640 1 -- 172.21.15.89:0/3811525827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75b407de80 msgr2=0x7f75b407c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:43.308 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.842+0000 7f75b3e00640 1 --2- 172.21.15.89:0/3811525827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75b407de80 0x7f75b407c2e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f75a4009820 tx=0x7f75a402a270 comp rx=0 tx=0).stop 2024-09-10T08:29:43.308 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.842+0000 7f75b3e00640 1 -- 172.21.15.89:0/3811525827 shutdown_connections 2024-09-10T08:29:43.308 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.842+0000 7f75b3e00640 1 --2- 172.21.15.89:0/3811525827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75b407de80 0x7f75b407c2e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:43.309 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.842+0000 7f75b3e00640 1 -- 172.21.15.89:0/3811525827 >> 172.21.15.89:0/3811525827 conn(0x7f75b4102c10 msgr2=0x7f75b4105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:43.309 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.842+0000 7f75b3e00640 1 -- 172.21.15.89:0/3811525827 shutdown_connections 2024-09-10T08:29:43.309 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.842+0000 7f75b3e00640 1 -- 172.21.15.89:0/3811525827 wait complete. 2024-09-10T08:29:43.309 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.843+0000 7f75b3e00640 1 Processor -- start 2024-09-10T08:29:43.309 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.843+0000 7f75b3e00640 1 -- start start 2024-09-10T08:29:43.309 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.844+0000 7f75b3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75b407de80 0x7f75b419d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:43.309 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.844+0000 7f75b3e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f75b410dad0 con 0x7f75b407de80 2024-09-10T08:29:43.309 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.844+0000 7f75b2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75b407de80 0x7f75b419d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:43.309 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.844+0000 7f75b2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75b407de80 0x7f75b419d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49942/0 (socket says 172.21.15.89:49942) 2024-09-10T08:29:43.309 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.844+0000 7f75b2a00640 1 -- 172.21.15.89:0/2691584998 learned_addr learned my addr 172.21.15.89:0/2691584998 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:43.309 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.844+0000 7f75b2a00640 1 -- 172.21.15.89:0/2691584998 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b419dc20 con 0x7f75b407de80 2024-09-10T08:29:43.309 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.845+0000 7f75b2a00640 1 --2- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75b407de80 0x7f75b419d6e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f75a4030420 tx=0x7f75a4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:43.310 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.845+0000 7f75ab400640 1 -- 172.21.15.89:0/2691584998 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75a4037040 con 0x7f75b407de80 2024-09-10T08:29:43.310 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.845+0000 7f75ab400640 1 -- 172.21.15.89:0/2691584998 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a4040070 con 0x7f75b407de80 2024-09-10T08:29:43.310 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.845+0000 7f75ab400640 1 -- 172.21.15.89:0/2691584998 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75a403a6e0 con 0x7f75b407de80 2024-09-10T08:29:43.310 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.845+0000 7f75b3e00640 1 -- 172.21.15.89:0/2691584998 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b419deb0 con 0x7f75b407de80 2024-09-10T08:29:43.310 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.845+0000 7f75b3e00640 1 -- 172.21.15.89:0/2691584998 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b41a0ba0 con 0x7f75b407de80 2024-09-10T08:29:43.310 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.846+0000 7f75ab400640 1 -- 172.21.15.89:0/2691584998 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f75a403bb20 con 0x7f75b407de80 2024-09-10T08:29:43.310 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.847+0000 7f75ab400640 1 --2- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f7598039020 0x7f759803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:43.310 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.847+0000 7f75ab400640 1 -- 172.21.15.89:0/2691584998 --> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f759803bbf0 con 0x7f7598039020 2024-09-10T08:29:43.310 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.847+0000 7f75ab400640 1 -- 172.21.15.89:0/2691584998 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f75a406c680 con 0x7f75b407de80 2024-09-10T08:29:43.310 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.847+0000 7f75b2000640 1 -- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f7598039020 msgr2=0x7f759803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:6800/3794982914 2024-09-10T08:29:43.310 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:38.847+0000 7f75b2000640 1 --2- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f7598039020 0x7f759803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T08:29:43.310 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:39.048+0000 7f75b2000640 1 -- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f7598039020 msgr2=0x7f759803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:6800/3794982914 2024-09-10T08:29:43.310 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:39.048+0000 7f75b2000640 1 --2- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f7598039020 0x7f759803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T08:29:43.311 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:39.449+0000 7f75b2000640 1 -- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f7598039020 msgr2=0x7f759803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:6800/3794982914 2024-09-10T08:29:43.311 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:39.449+0000 7f75b2000640 1 --2- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f7598039020 0x7f759803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T08:29:43.311 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:40.250+0000 7f75b2000640 1 -- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f7598039020 msgr2=0x7f759803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.89:6800/3794982914 2024-09-10T08:29:43.311 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:40.250+0000 7f75b2000640 1 --2- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f7598039020 0x7f759803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T08:29:43.311 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:41.473+0000 7f75ab400640 1 -- 172.21.15.89:0/2691584998 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f75a406b070 con 0x7f75b407de80 2024-09-10T08:29:43.311 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:41.473+0000 7f75ab400640 1 -- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f7598039020 msgr2=0x7f759803b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:29:43.311 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:41.473+0000 7f75ab400640 1 --2- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f7598039020 0x7f759803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:43.311 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:42.476+0000 7f75ab400640 1 -- 172.21.15.89:0/2691584998 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f75a403b420 con 0x7f75b407de80 2024-09-10T08:29:43.311 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:42.476+0000 7f75ab400640 1 --2- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f759803cae0 0x7f759803eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:43.311 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:42.476+0000 7f75ab400640 1 -- 172.21.15.89:0/2691584998 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f75a403cec0 con 0x7f759803cae0 2024-09-10T08:29:43.311 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:42.477+0000 7f75b2000640 1 --2- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f759803cae0 0x7f759803eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:43.311 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:42.478+0000 7f75b2000640 1 --2- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f759803cae0 0x7f759803eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f75a0003980 tx=0x7f75a00092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:43.311 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:42.479+0000 7f75ab400640 1 -- 172.21.15.89:0/2691584998 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f75a403cec0 con 0x7f759803cae0 2024-09-10T08:29:43.312 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:42.486+0000 7f75b3e00640 1 -- 172.21.15.89:0/2691584998 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f75b407c360 con 0x7f759803cae0 2024-09-10T08:29:43.312 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:42.487+0000 7f75ab400640 1 -- 172.21.15.89:0/2691584998 <== mgr.14150 v2:172.21.15.89:6800/1633345099 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f75b407c360 con 0x7f759803cae0 2024-09-10T08:29:43.312 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:42.487+0000 7f75b3e00640 1 -- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f759803cae0 msgr2=0x7f759803eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:43.312 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:42.487+0000 7f75b3e00640 1 --2- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f759803cae0 0x7f759803eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f75a0003980 tx=0x7f75a00092b0 comp rx=0 tx=0).stop 2024-09-10T08:29:43.312 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:42.487+0000 7f75b3e00640 1 -- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75b407de80 msgr2=0x7f75b419d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:43.312 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:42.487+0000 7f75b3e00640 1 --2- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75b407de80 0x7f75b419d6e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f75a4030420 tx=0x7f75a4032670 comp rx=0 tx=0).stop 2024-09-10T08:29:43.312 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:42.487+0000 7f75b3e00640 1 -- 172.21.15.89:0/2691584998 shutdown_connections 2024-09-10T08:29:43.312 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:42.487+0000 7f75b3e00640 1 --2- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f759803cae0 0x7f759803eed0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:43.312 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:42.487+0000 7f75b3e00640 1 --2- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:6800/3794982914,v1:172.21.15.89:6801/3794982914] conn(0x7f7598039020 0x7f759803b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:43.312 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:42.487+0000 7f75b3e00640 1 --2- 172.21.15.89:0/2691584998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75b407de80 0x7f75b419d6e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:43.312 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:42.487+0000 7f75b3e00640 1 -- 172.21.15.89:0/2691584998 >> 172.21.15.89:0/2691584998 conn(0x7f75b4102c10 msgr2=0x7f75b4103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:43.312 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:42.487+0000 7f75b3e00640 1 -- 172.21.15.89:0/2691584998 shutdown_connections 2024-09-10T08:29:43.312 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:42.487+0000 7f75b3e00640 1 -- 172.21.15.89:0/2691584998 wait complete. 2024-09-10T08:29:43.312 INFO:teuthology.orchestra.run.smithi089.stdout:mgr epoch 9 is available 2024-09-10T08:29:43.312 INFO:teuthology.orchestra.run.smithi089.stdout:Generating a dashboard self-signed certificate... 2024-09-10T08:29:43.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:43 smithi089 ceph-mon[30757]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T08:29:43.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:43 smithi089 ceph-mon[30757]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T08:29:44.788 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-10T08:29:44.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.755+0000 7fb833200640 1 Processor -- start 2024-09-10T08:29:44.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.755+0000 7fb833200640 1 -- start start 2024-09-10T08:29:44.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.755+0000 7fb833200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb83407de60 0x7fb83407c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:44.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.755+0000 7fb833200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb83407c800 con 0x7fb83407de60 2024-09-10T08:29:44.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.756+0000 7fb831e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb83407de60 0x7fb83407c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:44.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.756+0000 7fb831e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb83407de60 0x7fb83407c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38166/0 (socket says 172.21.15.89:38166) 2024-09-10T08:29:44.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.756+0000 7fb831e00640 1 -- 172.21.15.89:0/770403360 learned_addr learned my addr 172.21.15.89:0/770403360 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:44.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.756+0000 7fb831e00640 1 -- 172.21.15.89:0/770403360 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83407c980 con 0x7fb83407de60 2024-09-10T08:29:44.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.757+0000 7fb831e00640 1 --2- 172.21.15.89:0/770403360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb83407de60 0x7fb83407c2c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb824009820 tx=0x7fb82402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e1b50d66e7ac55 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:44.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.757+0000 7fb831400640 1 -- 172.21.15.89:0/770403360 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb824037070 con 0x7fb83407de60 2024-09-10T08:29:44.789 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.757+0000 7fb831400640 1 -- 172.21.15.89:0/770403360 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb824032920 con 0x7fb83407de60 2024-09-10T08:29:44.790 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.757+0000 7fb831400640 1 -- 172.21.15.89:0/770403360 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb824030810 con 0x7fb83407de60 2024-09-10T08:29:44.790 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.758+0000 7fb833200640 1 -- 172.21.15.89:0/770403360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb83407de60 msgr2=0x7fb83407c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:44.790 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.758+0000 7fb833200640 1 --2- 172.21.15.89:0/770403360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb83407de60 0x7fb83407c2c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb824009820 tx=0x7fb82402a270 comp rx=0 tx=0).stop 2024-09-10T08:29:44.790 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.758+0000 7fb833200640 1 -- 172.21.15.89:0/770403360 shutdown_connections 2024-09-10T08:29:44.791 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.758+0000 7fb833200640 1 --2- 172.21.15.89:0/770403360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb83407de60 0x7fb83407c2c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:44.791 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.758+0000 7fb833200640 1 -- 172.21.15.89:0/770403360 >> 172.21.15.89:0/770403360 conn(0x7fb834102c10 msgr2=0x7fb834105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:44.791 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.758+0000 7fb833200640 1 -- 172.21.15.89:0/770403360 shutdown_connections 2024-09-10T08:29:44.791 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.758+0000 7fb833200640 1 -- 172.21.15.89:0/770403360 wait complete. 2024-09-10T08:29:44.791 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.759+0000 7fb833200640 1 Processor -- start 2024-09-10T08:29:44.792 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.759+0000 7fb833200640 1 -- start start 2024-09-10T08:29:44.792 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.759+0000 7fb833200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb83407de60 0x7fb83419d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:44.792 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.759+0000 7fb833200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb83410da90 con 0x7fb83407de60 2024-09-10T08:29:44.792 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.759+0000 7fb831e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb83407de60 0x7fb83419d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:44.793 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.759+0000 7fb831e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb83407de60 0x7fb83419d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38168/0 (socket says 172.21.15.89:38168) 2024-09-10T08:29:44.793 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.760+0000 7fb831e00640 1 -- 172.21.15.89:0/3310830377 learned_addr learned my addr 172.21.15.89:0/3310830377 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:44.793 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.760+0000 7fb831e00640 1 -- 172.21.15.89:0/3310830377 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83419dc00 con 0x7fb83407de60 2024-09-10T08:29:44.793 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.760+0000 7fb831e00640 1 --2- 172.21.15.89:0/3310830377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb83407de60 0x7fb83419d6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb8240097f0 tx=0x7fb8240327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:44.793 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.760+0000 7fb82b400640 1 -- 172.21.15.89:0/3310830377 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb824037040 con 0x7fb83407de60 2024-09-10T08:29:44.793 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.760+0000 7fb82b400640 1 -- 172.21.15.89:0/3310830377 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb824030c40 con 0x7fb83407de60 2024-09-10T08:29:44.793 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.760+0000 7fb833200640 1 -- 172.21.15.89:0/3310830377 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb83419de90 con 0x7fb83407de60 2024-09-10T08:29:44.793 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.760+0000 7fb82b400640 1 -- 172.21.15.89:0/3310830377 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb82403a5d0 con 0x7fb83407de60 2024-09-10T08:29:44.794 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.760+0000 7fb833200640 1 -- 172.21.15.89:0/3310830377 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8341a0b80 con 0x7fb83407de60 2024-09-10T08:29:44.794 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.762+0000 7fb82b400640 1 -- 172.21.15.89:0/3310830377 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fb82403b3f0 con 0x7fb83407de60 2024-09-10T08:29:44.794 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.762+0000 7fb833200640 1 -- 172.21.15.89:0/3310830377 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb83407c340 con 0x7fb83407de60 2024-09-10T08:29:44.794 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.762+0000 7fb82b400640 1 --2- 172.21.15.89:0/3310830377 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb818039020 0x7fb81803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:44.794 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.762+0000 7fb82b400640 1 -- 172.21.15.89:0/3310830377 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb82406c670 con 0x7fb83407de60 2024-09-10T08:29:44.794 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.762+0000 7fb829400640 1 --2- 172.21.15.89:0/3310830377 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb818039020 0x7fb81803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:44.794 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.763+0000 7fb829400640 1 --2- 172.21.15.89:0/3310830377 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb818039020 0x7fb81803b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0098c0 tx=0x7fb81c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:44.794 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.767+0000 7fb82b400640 1 -- 172.21.15.89:0/3310830377 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb8240371e0 con 0x7fb83407de60 2024-09-10T08:29:44.794 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:43.949+0000 7fb833200640 1 -- 172.21.15.89:0/3310830377 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fb834103ed0 con 0x7fb818039020 2024-09-10T08:29:44.794 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:44.017+0000 7fb82b400640 1 -- 172.21.15.89:0/3310830377 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fb834103ed0 con 0x7fb818039020 2024-09-10T08:29:44.794 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:44.019+0000 7fb833200640 1 -- 172.21.15.89:0/3310830377 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb818039020 msgr2=0x7fb81803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:44.794 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:44.019+0000 7fb833200640 1 --2- 172.21.15.89:0/3310830377 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb818039020 0x7fb81803b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0098c0 tx=0x7fb81c006db0 comp rx=0 tx=0).stop 2024-09-10T08:29:44.794 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:44.019+0000 7fb833200640 1 -- 172.21.15.89:0/3310830377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb83407de60 msgr2=0x7fb83419d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:44.794 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:44.019+0000 7fb833200640 1 --2- 172.21.15.89:0/3310830377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb83407de60 0x7fb83419d6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb8240097f0 tx=0x7fb8240327b0 comp rx=0 tx=0).stop 2024-09-10T08:29:44.795 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:44.019+0000 7fb833200640 1 -- 172.21.15.89:0/3310830377 shutdown_connections 2024-09-10T08:29:44.795 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:44.019+0000 7fb833200640 1 --2- 172.21.15.89:0/3310830377 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb818039020 0x7fb81803b4e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:44.795 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:44.019+0000 7fb833200640 1 --2- 172.21.15.89:0/3310830377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb83407de60 0x7fb83419d6c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:44.795 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:44.019+0000 7fb833200640 1 -- 172.21.15.89:0/3310830377 >> 172.21.15.89:0/3310830377 conn(0x7fb834102c10 msgr2=0x7fb8341037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:44.795 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:44.020+0000 7fb833200640 1 -- 172.21.15.89:0/3310830377 shutdown_connections 2024-09-10T08:29:44.795 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:44.020+0000 7fb833200640 1 -- 172.21.15.89:0/3310830377 wait complete. 2024-09-10T08:29:44.795 INFO:teuthology.orchestra.run.smithi089.stdout:Creating initial admin user... 2024-09-10T08:29:44.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:44 smithi089 ceph-mon[30757]: [10/Sep/2024:08:29:42] ENGINE Bus STARTING 2024-09-10T08:29:44.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:44 smithi089 ceph-mon[30757]: [10/Sep/2024:08:29:42] ENGINE Serving on https://172.21.15.89:7150 2024-09-10T08:29:44.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:44 smithi089 ceph-mon[30757]: [10/Sep/2024:08:29:42] ENGINE Error in HTTPServer.serve 2024-09-10T08:29:44.802 INFO:journalctl@ceph.mon.a.smithi089.stdout: Traceback (most recent call last): 2024-09-10T08:29:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T08:29:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout: self._connections.run(self.expiration_interval) 2024-09-10T08:29:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T08:29:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout: self._run(expiration_interval) 2024-09-10T08:29:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T08:29:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T08:29:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T08:29:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T08:29:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T08:29:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout: s = self.context.wrap_socket( 2024-09-10T08:29:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T08:29:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout: return self.sslsocket_class._create( 2024-09-10T08:29:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T08:29:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout: self.do_handshake() 2024-09-10T08:29:44.804 INFO:journalctl@ceph.mon.a.smithi089.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T08:29:44.804 INFO:journalctl@ceph.mon.a.smithi089.stdout: self._sslobj.do_handshake() 2024-09-10T08:29:44.804 INFO:journalctl@ceph.mon.a.smithi089.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T08:29:44.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:44 smithi089 ceph-mon[30757]: [10/Sep/2024:08:29:42] ENGINE Serving on http://172.21.15.89:8765 2024-09-10T08:29:44.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:44 smithi089 ceph-mon[30757]: [10/Sep/2024:08:29:42] ENGINE Bus STARTED 2024-09-10T08:29:44.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:44 smithi089 ceph-mon[30757]: mgrmap e12: x(active, since 2s) 2024-09-10T08:29:44.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:44 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:29:44.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:44 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:29:45.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:45 smithi089 ceph-mon[30757]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:29:45.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:45 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:29:46.452 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$w.eTqJ838G3/5Ym818z6xO/M3i6.5eFiLPbpHTyCoSEaI25wSfE4W", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725956985, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T08:29:46.452 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.318+0000 7f14fcc00640 1 Processor -- start 2024-09-10T08:29:46.453 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.318+0000 7f14fcc00640 1 -- start start 2024-09-10T08:29:46.453 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.319+0000 7f14fcc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14f8107680 0x7f14f8107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:46.453 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.319+0000 7f14fcc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f14f8108050 con 0x7f14f8107680 2024-09-10T08:29:46.453 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.319+0000 7f14f5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14f8107680 0x7f14f8107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:46.453 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.319+0000 7f14f5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14f8107680 0x7f14f8107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38172/0 (socket says 172.21.15.89:38172) 2024-09-10T08:29:46.453 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.319+0000 7f14f5e00640 1 -- 172.21.15.89:0/3179716494 learned_addr learned my addr 172.21.15.89:0/3179716494 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:46.454 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.319+0000 7f14f5e00640 1 -- 172.21.15.89:0/3179716494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f81087e0 con 0x7f14f8107680 2024-09-10T08:29:46.454 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.320+0000 7f14f5e00640 1 --2- 172.21.15.89:0/3179716494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14f8107680 0x7f14f8107a80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009e80 tx=0x7f14e802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcaf6a1d4e059a5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:46.454 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.321+0000 7f14f4a00640 1 -- 172.21.15.89:0/3179716494 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e8036070 con 0x7f14f8107680 2024-09-10T08:29:46.454 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.321+0000 7f14f4a00640 1 -- 172.21.15.89:0/3179716494 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e80318e0 con 0x7f14f8107680 2024-09-10T08:29:46.455 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.321+0000 7f14f4a00640 1 -- 172.21.15.89:0/3179716494 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e8007a00 con 0x7f14f8107680 2024-09-10T08:29:46.455 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.321+0000 7f14fcc00640 1 -- 172.21.15.89:0/3179716494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14f8107680 msgr2=0x7f14f8107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:46.455 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.321+0000 7f14fcc00640 1 --2- 172.21.15.89:0/3179716494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14f8107680 0x7f14f8107a80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009e80 tx=0x7f14e802a7a0 comp rx=0 tx=0).stop 2024-09-10T08:29:46.456 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.321+0000 7f14fcc00640 1 -- 172.21.15.89:0/3179716494 shutdown_connections 2024-09-10T08:29:46.456 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.321+0000 7f14fcc00640 1 --2- 172.21.15.89:0/3179716494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14f8107680 0x7f14f8107a80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:46.456 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.321+0000 7f14fcc00640 1 -- 172.21.15.89:0/3179716494 >> 172.21.15.89:0/3179716494 conn(0x7f14f8102e10 msgr2=0x7f14f8105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:46.456 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.321+0000 7f14fcc00640 1 -- 172.21.15.89:0/3179716494 shutdown_connections 2024-09-10T08:29:46.456 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.322+0000 7f14fcc00640 1 -- 172.21.15.89:0/3179716494 wait complete. 2024-09-10T08:29:46.456 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.322+0000 7f14fcc00640 1 Processor -- start 2024-09-10T08:29:46.457 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.323+0000 7f14fcc00640 1 -- start start 2024-09-10T08:29:46.457 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.323+0000 7f14fcc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14f8107680 0x7f14f819d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:46.457 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.323+0000 7f14fcc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f14f810bb90 con 0x7f14f8107680 2024-09-10T08:29:46.457 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.323+0000 7f14f5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14f8107680 0x7f14f819d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:46.457 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.323+0000 7f14f5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14f8107680 0x7f14f819d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38176/0 (socket says 172.21.15.89:38176) 2024-09-10T08:29:46.458 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.323+0000 7f14f5e00640 1 -- 172.21.15.89:0/4013821960 learned_addr learned my addr 172.21.15.89:0/4013821960 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:46.458 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.324+0000 7f14f5e00640 1 -- 172.21.15.89:0/4013821960 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f819dde0 con 0x7f14f8107680 2024-09-10T08:29:46.458 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.324+0000 7f14f5e00640 1 --2- 172.21.15.89:0/4013821960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14f8107680 0x7f14f819d8a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f14e802acc0 tx=0x7f14e8031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:46.458 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.324+0000 7f14eea00640 1 -- 172.21.15.89:0/4013821960 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e8036050 con 0x7f14f8107680 2024-09-10T08:29:46.459 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.324+0000 7f14eea00640 1 -- 172.21.15.89:0/4013821960 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e8037040 con 0x7f14f8107680 2024-09-10T08:29:46.459 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.324+0000 7f14eea00640 1 -- 172.21.15.89:0/4013821960 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e803b920 con 0x7f14f8107680 2024-09-10T08:29:46.459 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.324+0000 7f14fcc00640 1 -- 172.21.15.89:0/4013821960 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f819e070 con 0x7f14f8107680 2024-09-10T08:29:46.459 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.325+0000 7f14fcc00640 1 -- 172.21.15.89:0/4013821960 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f81a0d60 con 0x7f14f8107680 2024-09-10T08:29:46.459 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.326+0000 7f14fcc00640 1 -- 172.21.15.89:0/4013821960 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f810bd10 con 0x7f14f8107680 2024-09-10T08:29:46.459 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.326+0000 7f14eea00640 1 -- 172.21.15.89:0/4013821960 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f14e8044020 con 0x7f14f8107680 2024-09-10T08:29:46.460 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.327+0000 7f14eea00640 1 --2- 172.21.15.89:0/4013821960 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14dc038fc0 0x7f14dc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:46.460 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.327+0000 7f14eea00640 1 -- 172.21.15.89:0/4013821960 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f14e8042020 con 0x7f14f8107680 2024-09-10T08:29:46.460 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.327+0000 7f14f5400640 1 --2- 172.21.15.89:0/4013821960 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14dc038fc0 0x7f14dc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:46.460 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.328+0000 7f14f5400640 1 --2- 172.21.15.89:0/4013821960 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14dc038fc0 0x7f14dc03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f14e40098c0 tx=0x7f14e4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:46.460 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.334+0000 7f14eea00640 1 -- 172.21.15.89:0/4013821960 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f14e804b020 con 0x7f14f8107680 2024-09-10T08:29:46.460 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.527+0000 7f14fcc00640 1 -- 172.21.15.89:0/4013821960 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f14f81a0ff0 con 0x7f14dc038fc0 2024-09-10T08:29:46.461 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.758+0000 7f14eea00640 1 -- 172.21.15.89:0/4013821960 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f14f81a0ff0 con 0x7f14dc038fc0 2024-09-10T08:29:46.461 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.760+0000 7f14fcc00640 1 -- 172.21.15.89:0/4013821960 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14dc038fc0 msgr2=0x7f14dc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:46.461 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.760+0000 7f14fcc00640 1 --2- 172.21.15.89:0/4013821960 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14dc038fc0 0x7f14dc03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f14e40098c0 tx=0x7f14e4006db0 comp rx=0 tx=0).stop 2024-09-10T08:29:46.461 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.760+0000 7f14fcc00640 1 -- 172.21.15.89:0/4013821960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14f8107680 msgr2=0x7f14f819d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:46.461 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.760+0000 7f14fcc00640 1 --2- 172.21.15.89:0/4013821960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14f8107680 0x7f14f819d8a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f14e802acc0 tx=0x7f14e8031cc0 comp rx=0 tx=0).stop 2024-09-10T08:29:46.462 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.760+0000 7f14fcc00640 1 -- 172.21.15.89:0/4013821960 shutdown_connections 2024-09-10T08:29:46.462 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.760+0000 7f14fcc00640 1 --2- 172.21.15.89:0/4013821960 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14dc038fc0 0x7f14dc03b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:46.462 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.760+0000 7f14fcc00640 1 --2- 172.21.15.89:0/4013821960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14f8107680 0x7f14f819d8a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:46.462 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.760+0000 7f14fcc00640 1 -- 172.21.15.89:0/4013821960 >> 172.21.15.89:0/4013821960 conn(0x7f14f8102e10 msgr2=0x7f14f81039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:46.462 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.761+0000 7f14fcc00640 1 -- 172.21.15.89:0/4013821960 shutdown_connections 2024-09-10T08:29:46.462 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:45.761+0000 7f14fcc00640 1 -- 172.21.15.89:0/4013821960 wait complete. 2024-09-10T08:29:46.463 INFO:teuthology.orchestra.run.smithi089.stdout:Fetching dashboard port number... 2024-09-10T08:29:46.723 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:46 smithi089 ceph-mon[30757]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:29:46.724 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:29:46.724 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:29:47.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:47 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/939142193' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-10T08:29:48.406 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T08:29:48.407 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.972+0000 7fa285000640 1 Processor -- start 2024-09-10T08:29:48.407 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.972+0000 7fa285000640 1 -- start start 2024-09-10T08:29:48.407 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.972+0000 7fa285000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa280107f10 0x7fa28010a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:48.407 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.972+0000 7fa285000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa28007be30 con 0x7fa280107f10 2024-09-10T08:29:48.407 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.972+0000 7fa27f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa280107f10 0x7fa28010a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:48.407 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.973+0000 7fa27f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa280107f10 0x7fa28010a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38192/0 (socket says 172.21.15.89:38192) 2024-09-10T08:29:48.407 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.973+0000 7fa27f400640 1 -- 172.21.15.89:0/213494390 learned_addr learned my addr 172.21.15.89:0/213494390 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:48.407 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.973+0000 7fa27f400640 1 -- 172.21.15.89:0/213494390 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28010a840 con 0x7fa280107f10 2024-09-10T08:29:48.407 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.974+0000 7fa27f400640 1 --2- 172.21.15.89:0/213494390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa280107f10 0x7fa28010a300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa270009e80 tx=0x7fa27002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d633b0efde7f6195 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:48.407 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.974+0000 7fa27ea00640 1 -- 172.21.15.89:0/213494390 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa270036070 con 0x7fa280107f10 2024-09-10T08:29:48.407 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.974+0000 7fa27ea00640 1 -- 172.21.15.89:0/213494390 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2700318e0 con 0x7fa280107f10 2024-09-10T08:29:48.407 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.974+0000 7fa27ea00640 1 -- 172.21.15.89:0/213494390 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa270007a00 con 0x7fa280107f10 2024-09-10T08:29:48.408 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.975+0000 7fa285000640 1 -- 172.21.15.89:0/213494390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa280107f10 msgr2=0x7fa28010a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:48.408 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.975+0000 7fa285000640 1 --2- 172.21.15.89:0/213494390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa280107f10 0x7fa28010a300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa270009e80 tx=0x7fa27002a7a0 comp rx=0 tx=0).stop 2024-09-10T08:29:48.408 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.975+0000 7fa285000640 1 -- 172.21.15.89:0/213494390 shutdown_connections 2024-09-10T08:29:48.408 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.975+0000 7fa285000640 1 --2- 172.21.15.89:0/213494390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa280107f10 0x7fa28010a300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:48.408 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.975+0000 7fa285000640 1 -- 172.21.15.89:0/213494390 >> 172.21.15.89:0/213494390 conn(0x7fa280101a70 msgr2=0x7fa280103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:48.408 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.975+0000 7fa285000640 1 -- 172.21.15.89:0/213494390 shutdown_connections 2024-09-10T08:29:48.408 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.975+0000 7fa285000640 1 -- 172.21.15.89:0/213494390 wait complete. 2024-09-10T08:29:48.408 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.976+0000 7fa285000640 1 Processor -- start 2024-09-10T08:29:48.408 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.976+0000 7fa285000640 1 -- start start 2024-09-10T08:29:48.408 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.976+0000 7fa285000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa280107f10 0x7fa28019d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:48.408 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.976+0000 7fa285000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa28010b860 con 0x7fa280107f10 2024-09-10T08:29:48.408 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.977+0000 7fa27f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa280107f10 0x7fa28019d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:48.408 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.977+0000 7fa27f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa280107f10 0x7fa28019d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38194/0 (socket says 172.21.15.89:38194) 2024-09-10T08:29:48.409 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.977+0000 7fa27f400640 1 -- 172.21.15.89:0/939142193 learned_addr learned my addr 172.21.15.89:0/939142193 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:48.409 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.977+0000 7fa27f400640 1 -- 172.21.15.89:0/939142193 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28019dc20 con 0x7fa280107f10 2024-09-10T08:29:48.409 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.977+0000 7fa27f400640 1 --2- 172.21.15.89:0/939142193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa280107f10 0x7fa28019d6e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa27002acc0 tx=0x7fa270031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:48.409 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.977+0000 7fa27cc00640 1 -- 172.21.15.89:0/939142193 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa270036050 con 0x7fa280107f10 2024-09-10T08:29:48.409 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.977+0000 7fa27cc00640 1 -- 172.21.15.89:0/939142193 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa270037040 con 0x7fa280107f10 2024-09-10T08:29:48.409 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.978+0000 7fa285000640 1 -- 172.21.15.89:0/939142193 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28019deb0 con 0x7fa280107f10 2024-09-10T08:29:48.409 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.978+0000 7fa27cc00640 1 -- 172.21.15.89:0/939142193 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa27003d970 con 0x7fa280107f10 2024-09-10T08:29:48.409 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.978+0000 7fa285000640 1 -- 172.21.15.89:0/939142193 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2801a0ba0 con 0x7fa280107f10 2024-09-10T08:29:48.409 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.979+0000 7fa27cc00640 1 -- 172.21.15.89:0/939142193 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fa270044020 con 0x7fa280107f10 2024-09-10T08:29:48.409 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.979+0000 7fa285000640 1 -- 172.21.15.89:0/939142193 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa280108370 con 0x7fa280107f10 2024-09-10T08:29:48.409 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.979+0000 7fa27cc00640 1 --2- 172.21.15.89:0/939142193 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa264038fc0 0x7fa26403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:48.410 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.979+0000 7fa27cc00640 1 -- 172.21.15.89:0/939142193 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa270042020 con 0x7fa280107f10 2024-09-10T08:29:48.410 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.979+0000 7fa276a00640 1 --2- 172.21.15.89:0/939142193 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa264038fc0 0x7fa26403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:48.410 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.980+0000 7fa276a00640 1 --2- 172.21.15.89:0/939142193 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa264038fc0 0x7fa26403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa2680098c0 tx=0x7fa268006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:48.410 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:46.984+0000 7fa27cc00640 1 -- 172.21.15.89:0/939142193 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa27002eca0 con 0x7fa280107f10 2024-09-10T08:29:48.410 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:47.169+0000 7fa285000640 1 -- 172.21.15.89:0/939142193 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fa2801a1190 con 0x7fa280107f10 2024-09-10T08:29:48.410 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:47.169+0000 7fa27cc00640 1 -- 172.21.15.89:0/939142193 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fa27003bc60 con 0x7fa280107f10 2024-09-10T08:29:48.410 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:47.172+0000 7fa285000640 1 -- 172.21.15.89:0/939142193 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa264038fc0 msgr2=0x7fa26403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:48.410 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:47.172+0000 7fa285000640 1 --2- 172.21.15.89:0/939142193 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa264038fc0 0x7fa26403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa2680098c0 tx=0x7fa268006db0 comp rx=0 tx=0).stop 2024-09-10T08:29:48.410 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:47.172+0000 7fa285000640 1 -- 172.21.15.89:0/939142193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa280107f10 msgr2=0x7fa28019d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:48.410 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:47.172+0000 7fa285000640 1 --2- 172.21.15.89:0/939142193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa280107f10 0x7fa28019d6e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa27002acc0 tx=0x7fa270031cc0 comp rx=0 tx=0).stop 2024-09-10T08:29:48.410 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:47.172+0000 7fa285000640 1 -- 172.21.15.89:0/939142193 shutdown_connections 2024-09-10T08:29:48.410 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:47.172+0000 7fa285000640 1 --2- 172.21.15.89:0/939142193 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa264038fc0 0x7fa26403b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:48.410 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:47.172+0000 7fa285000640 1 --2- 172.21.15.89:0/939142193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa280107f10 0x7fa28019d6e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:48.411 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:47.172+0000 7fa285000640 1 -- 172.21.15.89:0/939142193 >> 172.21.15.89:0/939142193 conn(0x7fa280101a70 msgr2=0x7fa280102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:48.411 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:47.172+0000 7fa285000640 1 -- 172.21.15.89:0/939142193 shutdown_connections 2024-09-10T08:29:48.411 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:47.172+0000 7fa285000640 1 -- 172.21.15.89:0/939142193 wait complete. 2024-09-10T08:29:48.422 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T08:29:48.422 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout disabled 2024-09-10T08:29:48.435 INFO:teuthology.orchestra.run.smithi089.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T08:29:48.435 INFO:teuthology.orchestra.run.smithi089.stdout:systemctl: stdout inactive 2024-09-10T08:29:48.435 INFO:teuthology.orchestra.run.smithi089.stdout:firewalld.service is not enabled 2024-09-10T08:29:48.435 INFO:teuthology.orchestra.run.smithi089.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T08:29:48.438 INFO:teuthology.orchestra.run.smithi089.stdout:Ceph Dashboard is now available at: 2024-09-10T08:29:48.438 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:29:48.438 INFO:teuthology.orchestra.run.smithi089.stdout: URL: https://smithi089.front.sepia.ceph.com:8443/ 2024-09-10T08:29:48.438 INFO:teuthology.orchestra.run.smithi089.stdout: User: admin 2024-09-10T08:29:48.438 INFO:teuthology.orchestra.run.smithi089.stdout: Password: iglmawhc8a 2024-09-10T08:29:48.438 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:29:48.439 INFO:teuthology.orchestra.run.smithi089.stdout:Saving cluster configuration to /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config directory 2024-09-10T08:29:48.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:48 smithi089 ceph-mon[30757]: mgrmap e13: x(active, since 6s) 2024-09-10T08:29:50.153 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.269+0000 7fcb71a00640 1 Processor -- start 2024-09-10T08:29:50.153 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.269+0000 7fcb71a00640 1 -- start start 2024-09-10T08:29:50.153 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.269+0000 7fcb71a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c07de00 0x7fcb6c07c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:50.153 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.269+0000 7fcb71a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6c07c840 con 0x7fcb6c07de00 2024-09-10T08:29:50.153 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.270+0000 7fcb6aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c07de00 0x7fcb6c07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:50.153 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.270+0000 7fcb6aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c07de00 0x7fcb6c07c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38200/0 (socket says 172.21.15.89:38200) 2024-09-10T08:29:50.153 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.270+0000 7fcb6aa00640 1 -- 172.21.15.89:0/778283454 learned_addr learned my addr 172.21.15.89:0/778283454 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:50.153 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.270+0000 7fcb6aa00640 1 -- 172.21.15.89:0/778283454 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6c07c9c0 con 0x7fcb6c07de00 2024-09-10T08:29:50.154 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.271+0000 7fcb6aa00640 1 --2- 172.21.15.89:0/778283454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c07de00 0x7fcb6c07c300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009820 tx=0x7fcb5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f509aceec84e1792 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:50.154 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.271+0000 7fcb69600640 1 -- 172.21.15.89:0/778283454 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb5c037070 con 0x7fcb6c07de00 2024-09-10T08:29:50.154 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.271+0000 7fcb69600640 1 -- 172.21.15.89:0/778283454 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb5c032920 con 0x7fcb6c07de00 2024-09-10T08:29:50.154 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.272+0000 7fcb69600640 1 -- 172.21.15.89:0/778283454 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb5c030810 con 0x7fcb6c07de00 2024-09-10T08:29:50.154 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.272+0000 7fcb71a00640 1 -- 172.21.15.89:0/778283454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c07de00 msgr2=0x7fcb6c07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:50.154 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.272+0000 7fcb71a00640 1 --2- 172.21.15.89:0/778283454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c07de00 0x7fcb6c07c300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009820 tx=0x7fcb5c02a270 comp rx=0 tx=0).stop 2024-09-10T08:29:50.154 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.272+0000 7fcb71a00640 1 -- 172.21.15.89:0/778283454 shutdown_connections 2024-09-10T08:29:50.154 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.272+0000 7fcb71a00640 1 --2- 172.21.15.89:0/778283454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c07de00 0x7fcb6c07c300 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:50.154 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.272+0000 7fcb71a00640 1 -- 172.21.15.89:0/778283454 >> 172.21.15.89:0/778283454 conn(0x7fcb6c102e50 msgr2=0x7fcb6c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:50.154 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.272+0000 7fcb71a00640 1 -- 172.21.15.89:0/778283454 shutdown_connections 2024-09-10T08:29:50.154 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.272+0000 7fcb71a00640 1 -- 172.21.15.89:0/778283454 wait complete. 2024-09-10T08:29:50.154 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.273+0000 7fcb71a00640 1 Processor -- start 2024-09-10T08:29:50.154 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.273+0000 7fcb71a00640 1 -- start start 2024-09-10T08:29:50.155 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.274+0000 7fcb71a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c07de00 0x7fcb6c19d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:50.155 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.274+0000 7fcb71a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6c10dc50 con 0x7fcb6c07de00 2024-09-10T08:29:50.155 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.274+0000 7fcb6aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c07de00 0x7fcb6c19d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:50.155 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.274+0000 7fcb6aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c07de00 0x7fcb6c19d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38206/0 (socket says 172.21.15.89:38206) 2024-09-10T08:29:50.155 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.274+0000 7fcb6aa00640 1 -- 172.21.15.89:0/3092932259 learned_addr learned my addr 172.21.15.89:0/3092932259 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:50.155 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.274+0000 7fcb6aa00640 1 -- 172.21.15.89:0/3092932259 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6c19ddd0 con 0x7fcb6c07de00 2024-09-10T08:29:50.155 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.274+0000 7fcb6aa00640 1 --2- 172.21.15.89:0/3092932259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c07de00 0x7fcb6c19d890 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c030420 tx=0x7fcb5c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:50.155 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.275+0000 7fcb63400640 1 -- 172.21.15.89:0/3092932259 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb5c037040 con 0x7fcb6c07de00 2024-09-10T08:29:50.155 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.275+0000 7fcb63400640 1 -- 172.21.15.89:0/3092932259 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb5c040070 con 0x7fcb6c07de00 2024-09-10T08:29:50.155 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.275+0000 7fcb63400640 1 -- 172.21.15.89:0/3092932259 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb5c03a6e0 con 0x7fcb6c07de00 2024-09-10T08:29:50.156 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.275+0000 7fcb71a00640 1 -- 172.21.15.89:0/3092932259 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6c19e060 con 0x7fcb6c07de00 2024-09-10T08:29:50.156 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.275+0000 7fcb71a00640 1 -- 172.21.15.89:0/3092932259 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb6c1a0d50 con 0x7fcb6c07de00 2024-09-10T08:29:50.156 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.276+0000 7fcb63400640 1 -- 172.21.15.89:0/3092932259 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fcb5c03bb20 con 0x7fcb6c07de00 2024-09-10T08:29:50.156 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.276+0000 7fcb71a00640 1 -- 172.21.15.89:0/3092932259 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb6c07c380 con 0x7fcb6c07de00 2024-09-10T08:29:50.156 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.276+0000 7fcb63400640 1 --2- 172.21.15.89:0/3092932259 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcb500390c0 0x7fcb5003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:50.156 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.276+0000 7fcb63400640 1 -- 172.21.15.89:0/3092932259 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcb5c06c810 con 0x7fcb6c07de00 2024-09-10T08:29:50.156 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.277+0000 7fcb6a000640 1 --2- 172.21.15.89:0/3092932259 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcb500390c0 0x7fcb5003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:50.156 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.277+0000 7fcb6a000640 1 --2- 172.21.15.89:0/3092932259 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcb500390c0 0x7fcb5003b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcb580098c0 tx=0x7fcb58006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:50.156 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.281+0000 7fcb63400640 1 -- 172.21.15.89:0/3092932259 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb5c042b20 con 0x7fcb6c07de00 2024-09-10T08:29:50.156 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.544+0000 7fcb71a00640 1 -- 172.21.15.89:0/3092932259 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fcb6c1a12d0 con 0x7fcb6c07de00 2024-09-10T08:29:50.156 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.547+0000 7fcb63400640 1 -- 172.21.15.89:0/3092932259 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fcb5c043360 con 0x7fcb6c07de00 2024-09-10T08:29:50.156 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T08:29:50.156 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.549+0000 7fcb71a00640 1 -- 172.21.15.89:0/3092932259 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcb500390c0 msgr2=0x7fcb5003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:50.157 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.549+0000 7fcb71a00640 1 --2- 172.21.15.89:0/3092932259 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcb500390c0 0x7fcb5003b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcb580098c0 tx=0x7fcb58006db0 comp rx=0 tx=0).stop 2024-09-10T08:29:50.157 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.549+0000 7fcb71a00640 1 -- 172.21.15.89:0/3092932259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c07de00 msgr2=0x7fcb6c19d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:50.157 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.549+0000 7fcb71a00640 1 --2- 172.21.15.89:0/3092932259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c07de00 0x7fcb6c19d890 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c030420 tx=0x7fcb5c032670 comp rx=0 tx=0).stop 2024-09-10T08:29:50.157 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.550+0000 7fcb71a00640 1 -- 172.21.15.89:0/3092932259 shutdown_connections 2024-09-10T08:29:50.158 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.550+0000 7fcb71a00640 1 --2- 172.21.15.89:0/3092932259 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcb500390c0 0x7fcb5003b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:50.158 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.550+0000 7fcb71a00640 1 --2- 172.21.15.89:0/3092932259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb6c07de00 0x7fcb6c19d890 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:50.158 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.550+0000 7fcb71a00640 1 -- 172.21.15.89:0/3092932259 >> 172.21.15.89:0/3092932259 conn(0x7fcb6c102e50 msgr2=0x7fcb6c103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:50.158 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.550+0000 7fcb71a00640 1 -- 172.21.15.89:0/3092932259 shutdown_connections 2024-09-10T08:29:50.158 INFO:teuthology.orchestra.run.smithi089.stdout:/usr/bin/ceph: stderr 2024-09-10T08:29:49.550+0000 7fcb71a00640 1 -- 172.21.15.89:0/3092932259 wait complete. 2024-09-10T08:29:50.158 INFO:teuthology.orchestra.run.smithi089.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T08:29:50.158 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:29:50.158 INFO:teuthology.orchestra.run.smithi089.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T08:29:50.158 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:29:50.158 INFO:teuthology.orchestra.run.smithi089.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T08:29:50.158 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:29:50.158 INFO:teuthology.orchestra.run.smithi089.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T08:29:50.159 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:29:50.159 INFO:teuthology.orchestra.run.smithi089.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T08:29:50.159 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:29:50.159 INFO:teuthology.orchestra.run.smithi089.stdout: ceph telemetry on 2024-09-10T08:29:50.159 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:29:50.159 INFO:teuthology.orchestra.run.smithi089.stdout:For more information see: 2024-09-10T08:29:50.159 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:29:50.159 INFO:teuthology.orchestra.run.smithi089.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-10T08:29:50.159 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:29:50.159 INFO:teuthology.orchestra.run.smithi089.stdout:Bootstrap complete. 2024-09-10T08:29:50.206 INFO:tasks.cephadm:Fetching config... 2024-09-10T08:29:50.206 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:29:50.206 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T08:29:50.242 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T08:29:50.242 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:29:50.242 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T08:29:50.319 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T08:29:50.319 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:29:50.320 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T08:29:50.418 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T08:29:50.418 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:29:50.418 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T08:29:50.493 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T08:29:50.493 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCR6B/hB/QR/NuoQZZkUg8qnpfBD7OJvBAGxXJJ29ZccHVUSp+dtP2/X7U3FntyqpEPVLQpkmE02A5u5UlYOckToFMvXhOBcgkjdrBukpuPWEV7JABHXX7QD6q3SzUhnPoc8Xz4Hw5zByVQkLqYLNPfdqLlmXODJtf7ilujTc44duAwXbmSN95A3ZurHDz4NUZjJipuuUZ1rqNO+R/X/PE3hSkmPTShdVg5pbtyqoHEc0hyHaNDdzOTfgMRxJcNfe6QUzG3ngf5FXSrJDdBkZrXBocps+CH/dg+GFbbh93k2GEGhKY8dPMtEW3s37NESY3yl3LdJ+/Zda7Rflkse8/Xym6onFS+eQD0e3gWb5I0KS2joQIOTl4w4FSwHY1QxaNELks88TezYwrg+L5Tj/sdsGjrdWUjQo7k8KhSQcdH1dv80VBVEkoAG2iWfuWiB0/zfuZGHNH7n03SdEqW9vbZOGQxtEip5iV/zHr5F3t7TA2rKWTE03rG27QX1MDT5lM= ceph-91f747bc-6f4e-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T08:29:50.622 INFO:teuthology.orchestra.run.smithi089.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCR6B/hB/QR/NuoQZZkUg8qnpfBD7OJvBAGxXJJ29ZccHVUSp+dtP2/X7U3FntyqpEPVLQpkmE02A5u5UlYOckToFMvXhOBcgkjdrBukpuPWEV7JABHXX7QD6q3SzUhnPoc8Xz4Hw5zByVQkLqYLNPfdqLlmXODJtf7ilujTc44duAwXbmSN95A3ZurHDz4NUZjJipuuUZ1rqNO+R/X/PE3hSkmPTShdVg5pbtyqoHEc0hyHaNDdzOTfgMRxJcNfe6QUzG3ngf5FXSrJDdBkZrXBocps+CH/dg+GFbbh93k2GEGhKY8dPMtEW3s37NESY3yl3LdJ+/Zda7Rflkse8/Xym6onFS+eQD0e3gWb5I0KS2joQIOTl4w4FSwHY1QxaNELks88TezYwrg+L5Tj/sdsGjrdWUjQo7k8KhSQcdH1dv80VBVEkoAG2iWfuWiB0/zfuZGHNH7n03SdEqW9vbZOGQxtEip5iV/zHr5F3t7TA2rKWTE03rG27QX1MDT5lM= ceph-91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:29:50.650 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCR6B/hB/QR/NuoQZZkUg8qnpfBD7OJvBAGxXJJ29ZccHVUSp+dtP2/X7U3FntyqpEPVLQpkmE02A5u5UlYOckToFMvXhOBcgkjdrBukpuPWEV7JABHXX7QD6q3SzUhnPoc8Xz4Hw5zByVQkLqYLNPfdqLlmXODJtf7ilujTc44duAwXbmSN95A3ZurHDz4NUZjJipuuUZ1rqNO+R/X/PE3hSkmPTShdVg5pbtyqoHEc0hyHaNDdzOTfgMRxJcNfe6QUzG3ngf5FXSrJDdBkZrXBocps+CH/dg+GFbbh93k2GEGhKY8dPMtEW3s37NESY3yl3LdJ+/Zda7Rflkse8/Xym6onFS+eQD0e3gWb5I0KS2joQIOTl4w4FSwHY1QxaNELks88TezYwrg+L5Tj/sdsGjrdWUjQo7k8KhSQcdH1dv80VBVEkoAG2iWfuWiB0/zfuZGHNH7n03SdEqW9vbZOGQxtEip5iV/zHr5F3t7TA2rKWTE03rG27QX1MDT5lM= ceph-91f747bc-6f4e-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T08:29:50.692 INFO:teuthology.orchestra.run.smithi111.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCR6B/hB/QR/NuoQZZkUg8qnpfBD7OJvBAGxXJJ29ZccHVUSp+dtP2/X7U3FntyqpEPVLQpkmE02A5u5UlYOckToFMvXhOBcgkjdrBukpuPWEV7JABHXX7QD6q3SzUhnPoc8Xz4Hw5zByVQkLqYLNPfdqLlmXODJtf7ilujTc44duAwXbmSN95A3ZurHDz4NUZjJipuuUZ1rqNO+R/X/PE3hSkmPTShdVg5pbtyqoHEc0hyHaNDdzOTfgMRxJcNfe6QUzG3ngf5FXSrJDdBkZrXBocps+CH/dg+GFbbh93k2GEGhKY8dPMtEW3s37NESY3yl3LdJ+/Zda7Rflkse8/Xym6onFS+eQD0e3gWb5I0KS2joQIOTl4w4FSwHY1QxaNELks88TezYwrg+L5Tj/sdsGjrdWUjQo7k8KhSQcdH1dv80VBVEkoAG2iWfuWiB0/zfuZGHNH7n03SdEqW9vbZOGQxtEip5iV/zHr5F3t7TA2rKWTE03rG27QX1MDT5lM= ceph-91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:29:50.704 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCR6B/hB/QR/NuoQZZkUg8qnpfBD7OJvBAGxXJJ29ZccHVUSp+dtP2/X7U3FntyqpEPVLQpkmE02A5u5UlYOckToFMvXhOBcgkjdrBukpuPWEV7JABHXX7QD6q3SzUhnPoc8Xz4Hw5zByVQkLqYLNPfdqLlmXODJtf7ilujTc44duAwXbmSN95A3ZurHDz4NUZjJipuuUZ1rqNO+R/X/PE3hSkmPTShdVg5pbtyqoHEc0hyHaNDdzOTfgMRxJcNfe6QUzG3ngf5FXSrJDdBkZrXBocps+CH/dg+GFbbh93k2GEGhKY8dPMtEW3s37NESY3yl3LdJ+/Zda7Rflkse8/Xym6onFS+eQD0e3gWb5I0KS2joQIOTl4w4FSwHY1QxaNELks88TezYwrg+L5Tj/sdsGjrdWUjQo7k8KhSQcdH1dv80VBVEkoAG2iWfuWiB0/zfuZGHNH7n03SdEqW9vbZOGQxtEip5iV/zHr5F3t7TA2rKWTE03rG27QX1MDT5lM= ceph-91f747bc-6f4e-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T08:29:50.725 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:50 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3092932259' entity='client.admin' 2024-09-10T08:29:50.745 INFO:teuthology.orchestra.run.smithi149.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCR6B/hB/QR/NuoQZZkUg8qnpfBD7OJvBAGxXJJ29ZccHVUSp+dtP2/X7U3FntyqpEPVLQpkmE02A5u5UlYOckToFMvXhOBcgkjdrBukpuPWEV7JABHXX7QD6q3SzUhnPoc8Xz4Hw5zByVQkLqYLNPfdqLlmXODJtf7ilujTc44duAwXbmSN95A3ZurHDz4NUZjJipuuUZ1rqNO+R/X/PE3hSkmPTShdVg5pbtyqoHEc0hyHaNDdzOTfgMRxJcNfe6QUzG3ngf5FXSrJDdBkZrXBocps+CH/dg+GFbbh93k2GEGhKY8dPMtEW3s37NESY3yl3LdJ+/Zda7Rflkse8/Xym6onFS+eQD0e3gWb5I0KS2joQIOTl4w4FSwHY1QxaNELks88TezYwrg+L5Tj/sdsGjrdWUjQo7k8KhSQcdH1dv80VBVEkoAG2iWfuWiB0/zfuZGHNH7n03SdEqW9vbZOGQxtEip5iV/zHr5F3t7TA2rKWTE03rG27QX1MDT5lM= ceph-91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:29:50.758 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T08:29:51.122 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:29:51.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.678+0000 7fdde2400640 1 -- 172.21.15.89:0/1224349291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc1037f0 msgr2=0x7fdddc103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:51.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.678+0000 7fdde2400640 1 --2- 172.21.15.89:0/1224349291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc1037f0 0x7fdddc103bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009880 tx=0x7fddcc02a5d0 comp rx=0 tx=0).stop 2024-09-10T08:29:51.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.678+0000 7fdde2400640 1 -- 172.21.15.89:0/1224349291 shutdown_connections 2024-09-10T08:29:51.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.678+0000 7fdde2400640 1 --2- 172.21.15.89:0/1224349291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc1037f0 0x7fdddc103bf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:51.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.678+0000 7fdde2400640 1 -- 172.21.15.89:0/1224349291 >> 172.21.15.89:0/1224349291 conn(0x7fdddc0fefc0 msgr2=0x7fdddc101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:51.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.678+0000 7fdde2400640 1 -- 172.21.15.89:0/1224349291 shutdown_connections 2024-09-10T08:29:51.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.678+0000 7fdde2400640 1 -- 172.21.15.89:0/1224349291 wait complete. 2024-09-10T08:29:51.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.679+0000 7fdde2400640 1 Processor -- start 2024-09-10T08:29:51.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.679+0000 7fdde2400640 1 -- start start 2024-09-10T08:29:51.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.680+0000 7fdde2400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc1037f0 0x7fdddc198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:51.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.680+0000 7fdde2400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc109900 con 0x7fdddc1037f0 2024-09-10T08:29:51.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.680+0000 7fdddb400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc1037f0 0x7fdddc198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:51.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.680+0000 7fdddb400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc1037f0 0x7fdddc198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38232/0 (socket says 172.21.15.89:38232) 2024-09-10T08:29:51.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.680+0000 7fdddb400640 1 -- 172.21.15.89:0/1513691637 learned_addr learned my addr 172.21.15.89:0/1513691637 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:51.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.680+0000 7fdddb400640 1 -- 172.21.15.89:0/1513691637 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdddc199470 con 0x7fdddc1037f0 2024-09-10T08:29:51.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.681+0000 7fdddb400640 1 --2- 172.21.15.89:0/1513691637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc1037f0 0x7fdddc198f30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fddcc004f60 tx=0x7fddcc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:51.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.681+0000 7fddd3e00640 1 -- 172.21.15.89:0/1513691637 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddcc03f070 con 0x7fdddc1037f0 2024-09-10T08:29:51.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.681+0000 7fdde2400640 1 -- 172.21.15.89:0/1513691637 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdddc199700 con 0x7fdddc1037f0 2024-09-10T08:29:51.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.681+0000 7fddd3e00640 1 -- 172.21.15.89:0/1513691637 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddcc004cc0 con 0x7fdddc1037f0 2024-09-10T08:29:51.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.681+0000 7fddd3e00640 1 -- 172.21.15.89:0/1513691637 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddcc035030 con 0x7fdddc1037f0 2024-09-10T08:29:51.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.681+0000 7fdde2400640 1 -- 172.21.15.89:0/1513691637 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdddc19c3f0 con 0x7fdddc1037f0 2024-09-10T08:29:51.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.682+0000 7fdde2400640 1 -- 172.21.15.89:0/1513691637 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdddc10a080 con 0x7fdddc1037f0 2024-09-10T08:29:51.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.682+0000 7fddd3e00640 1 -- 172.21.15.89:0/1513691637 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fddcc042070 con 0x7fdddc1037f0 2024-09-10T08:29:51.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.683+0000 7fddd3e00640 1 --2- 172.21.15.89:0/1513691637 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fddc00390c0 0x7fddc003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:51.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.683+0000 7fddd3e00640 1 -- 172.21.15.89:0/1513691637 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fddcc06ce00 con 0x7fdddc1037f0 2024-09-10T08:29:51.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.683+0000 7fdddaa00640 1 --2- 172.21.15.89:0/1513691637 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fddc00390c0 0x7fddc003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:51.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.684+0000 7fdddaa00640 1 --2- 172.21.15.89:0/1513691637 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fddc00390c0 0x7fddc003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fddc80098c0 tx=0x7fddc8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:51.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.688+0000 7fddd3e00640 1 -- 172.21.15.89:0/1513691637 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fddcc03a3c0 con 0x7fdddc1037f0 2024-09-10T08:29:51.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.873+0000 7fdde2400640 1 -- 172.21.15.89:0/1513691637 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fdddc02d830 con 0x7fdddc1037f0 2024-09-10T08:29:51.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.876+0000 7fddd3e00640 1 -- 172.21.15.89:0/1513691637 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fddcc003dd0 con 0x7fdddc1037f0 2024-09-10T08:29:51.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.879+0000 7fdde2400640 1 -- 172.21.15.89:0/1513691637 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fddc00390c0 msgr2=0x7fddc003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.879+0000 7fdde2400640 1 --2- 172.21.15.89:0/1513691637 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fddc00390c0 0x7fddc003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fddc80098c0 tx=0x7fddc8006db0 comp rx=0 tx=0).stop 2024-09-10T08:29:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.879+0000 7fdde2400640 1 -- 172.21.15.89:0/1513691637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc1037f0 msgr2=0x7fdddc198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.879+0000 7fdde2400640 1 --2- 172.21.15.89:0/1513691637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc1037f0 0x7fdddc198f30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fddcc004f60 tx=0x7fddcc0048f0 comp rx=0 tx=0).stop 2024-09-10T08:29:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.879+0000 7fdde2400640 1 -- 172.21.15.89:0/1513691637 shutdown_connections 2024-09-10T08:29:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.879+0000 7fdde2400640 1 --2- 172.21.15.89:0/1513691637 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fddc00390c0 0x7fddc003b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.879+0000 7fdde2400640 1 --2- 172.21.15.89:0/1513691637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdddc1037f0 0x7fdddc198f30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.879+0000 7fdde2400640 1 -- 172.21.15.89:0/1513691637 >> 172.21.15.89:0/1513691637 conn(0x7fdddc0fefc0 msgr2=0x7fdddc0ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.879+0000 7fdde2400640 1 -- 172.21.15.89:0/1513691637 shutdown_connections 2024-09-10T08:29:51.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:51.879+0000 7fdde2400640 1 -- 172.21.15.89:0/1513691637 wait complete. 2024-09-10T08:29:52.612 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T08:29:52.613 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T08:29:53.125 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:52 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1513691637' entity='client.admin' 2024-09-10T08:29:53.223 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:29:54.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.380+0000 7f344a800640 1 -- 172.21.15.89:0/1477339568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3444101600 msgr2=0x7f3444101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:54.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.380+0000 7f344a800640 1 --2- 172.21.15.89:0/1477339568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3444101600 0x7f3444101a00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3434009880 tx=0x7f343402a5d0 comp rx=0 tx=0).stop 2024-09-10T08:29:54.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.381+0000 7f344a800640 1 -- 172.21.15.89:0/1477339568 shutdown_connections 2024-09-10T08:29:54.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.381+0000 7f344a800640 1 --2- 172.21.15.89:0/1477339568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3444101600 0x7f3444101a00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:54.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.381+0000 7f344a800640 1 -- 172.21.15.89:0/1477339568 >> 172.21.15.89:0/1477339568 conn(0x7f34440fcdb0 msgr2=0x7f34440ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:54.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.381+0000 7f344a800640 1 -- 172.21.15.89:0/1477339568 shutdown_connections 2024-09-10T08:29:54.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.381+0000 7f344a800640 1 -- 172.21.15.89:0/1477339568 wait complete. 2024-09-10T08:29:54.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.382+0000 7f344a800640 1 Processor -- start 2024-09-10T08:29:54.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.382+0000 7f344a800640 1 -- start start 2024-09-10T08:29:54.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.382+0000 7f344a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3444101600 0x7f3444198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:54.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.382+0000 7f344a800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3444107710 con 0x7f3444101600 2024-09-10T08:29:54.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.382+0000 7f3443400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3444101600 0x7f3444198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:54.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.382+0000 7f3443400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3444101600 0x7f3444198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41726/0 (socket says 172.21.15.89:41726) 2024-09-10T08:29:54.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.382+0000 7f3443400640 1 -- 172.21.15.89:0/7076469 learned_addr learned my addr 172.21.15.89:0/7076469 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:54.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.383+0000 7f3443400640 1 -- 172.21.15.89:0/7076469 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3444199510 con 0x7f3444101600 2024-09-10T08:29:54.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.383+0000 7f3443400640 1 --2- 172.21.15.89:0/7076469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3444101600 0x7f3444198fd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3434004f60 tx=0x7f34340048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:54.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.383+0000 7f343be00640 1 -- 172.21.15.89:0/7076469 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f343403f070 con 0x7f3444101600 2024-09-10T08:29:54.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.383+0000 7f343be00640 1 -- 172.21.15.89:0/7076469 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3434004d40 con 0x7f3444101600 2024-09-10T08:29:54.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.383+0000 7f344a800640 1 -- 172.21.15.89:0/7076469 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34441997a0 con 0x7f3444101600 2024-09-10T08:29:54.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.383+0000 7f343be00640 1 -- 172.21.15.89:0/7076469 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3434035030 con 0x7f3444101600 2024-09-10T08:29:54.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.383+0000 7f344a800640 1 -- 172.21.15.89:0/7076469 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f344419c490 con 0x7f3444101600 2024-09-10T08:29:54.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.384+0000 7f343be00640 1 -- 172.21.15.89:0/7076469 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3434042070 con 0x7f3444101600 2024-09-10T08:29:54.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.384+0000 7f344a800640 1 -- 172.21.15.89:0/7076469 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3444107e90 con 0x7f3444101600 2024-09-10T08:29:54.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.385+0000 7f343be00640 1 --2- 172.21.15.89:0/7076469 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3428039060 0x7f342803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:54.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.385+0000 7f343be00640 1 -- 172.21.15.89:0/7076469 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f343406cee0 con 0x7f3444101600 2024-09-10T08:29:54.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.385+0000 7f3442a00640 1 --2- 172.21.15.89:0/7076469 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3428039060 0x7f342803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:54.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.386+0000 7f3442a00640 1 --2- 172.21.15.89:0/7076469 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3428039060 0x7f342803b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f34300098c0 tx=0x7f3430006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:54.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.390+0000 7f343be00640 1 -- 172.21.15.89:0/7076469 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3434042360 con 0x7f3444101600 2024-09-10T08:29:54.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.579+0000 7f344a800640 1 -- 172.21.15.89:0/7076469 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f344402d830 con 0x7f3428039060 2024-09-10T08:29:54.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.582+0000 7f343be00640 1 -- 172.21.15.89:0/7076469 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f344402d830 con 0x7f3428039060 2024-09-10T08:29:54.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.585+0000 7f344a800640 1 -- 172.21.15.89:0/7076469 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3428039060 msgr2=0x7f342803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:54.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.585+0000 7f344a800640 1 --2- 172.21.15.89:0/7076469 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3428039060 0x7f342803b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f34300098c0 tx=0x7f3430006db0 comp rx=0 tx=0).stop 2024-09-10T08:29:54.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.585+0000 7f344a800640 1 -- 172.21.15.89:0/7076469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3444101600 msgr2=0x7f3444198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:54.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.585+0000 7f344a800640 1 --2- 172.21.15.89:0/7076469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3444101600 0x7f3444198fd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3434004f60 tx=0x7f34340048f0 comp rx=0 tx=0).stop 2024-09-10T08:29:54.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.585+0000 7f344a800640 1 -- 172.21.15.89:0/7076469 shutdown_connections 2024-09-10T08:29:54.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.585+0000 7f344a800640 1 --2- 172.21.15.89:0/7076469 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3428039060 0x7f342803b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:54.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.585+0000 7f344a800640 1 --2- 172.21.15.89:0/7076469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3444101600 0x7f3444198fd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:54.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.585+0000 7f344a800640 1 -- 172.21.15.89:0/7076469 >> 172.21.15.89:0/7076469 conn(0x7f34440fcdb0 msgr2=0x7f34440ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:54.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.585+0000 7f344a800640 1 -- 172.21.15.89:0/7076469 shutdown_connections 2024-09-10T08:29:54.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:54.585+0000 7f344a800640 1 -- 172.21.15.89:0/7076469 wait complete. 2024-09-10T08:29:55.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:55 smithi089 ceph-mon[30757]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:29:55.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:55 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:29:55.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:55 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:29:55.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:55 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:29:55.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:55 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:29:55.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:55 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:29:55.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:55 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:29:56.068 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi111 2024-09-10T08:29:56.068 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-10T08:29:56.068 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/etc/ceph/ceph.conf 2024-09-10T08:29:56.087 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-10T08:29:56.087 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T08:29:56.145 INFO:tasks.cephadm:Adding host smithi111 to orchestrator... 2024-09-10T08:29:56.146 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph orch host add smithi111 2024-09-10T08:29:56.516 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:29:56.848 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:56 smithi089 ceph-mon[30757]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-10T08:29:57.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.082+0000 7fc8f4800640 1 -- 172.21.15.89:0/1634628433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8ec1037f0 msgr2=0x7fc8ec103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:29:57.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.082+0000 7fc8f4800640 1 --2- 172.21.15.89:0/1634628433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8ec1037f0 0x7fc8ec103bf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009880 tx=0x7fc8e002a5d0 comp rx=0 tx=0).stop 2024-09-10T08:29:57.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.082+0000 7fc8f4800640 1 -- 172.21.15.89:0/1634628433 shutdown_connections 2024-09-10T08:29:57.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.082+0000 7fc8f4800640 1 --2- 172.21.15.89:0/1634628433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8ec1037f0 0x7fc8ec103bf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:29:57.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.082+0000 7fc8f4800640 1 -- 172.21.15.89:0/1634628433 >> 172.21.15.89:0/1634628433 conn(0x7fc8ec0fefc0 msgr2=0x7fc8ec101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:29:57.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.082+0000 7fc8f4800640 1 -- 172.21.15.89:0/1634628433 shutdown_connections 2024-09-10T08:29:57.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.082+0000 7fc8f4800640 1 -- 172.21.15.89:0/1634628433 wait complete. 2024-09-10T08:29:57.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.083+0000 7fc8f4800640 1 Processor -- start 2024-09-10T08:29:57.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.084+0000 7fc8f4800640 1 -- start start 2024-09-10T08:29:57.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.084+0000 7fc8f4800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8ec1037f0 0x7fc8ec198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:57.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.084+0000 7fc8f4800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec109900 con 0x7fc8ec1037f0 2024-09-10T08:29:57.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.084+0000 7fc8f1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8ec1037f0 0x7fc8ec198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:57.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.084+0000 7fc8f1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8ec1037f0 0x7fc8ec198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41750/0 (socket says 172.21.15.89:41750) 2024-09-10T08:29:57.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.084+0000 7fc8f1e00640 1 -- 172.21.15.89:0/445930949 learned_addr learned my addr 172.21.15.89:0/445930949 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:29:57.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.085+0000 7fc8f1e00640 1 -- 172.21.15.89:0/445930949 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ec199470 con 0x7fc8ec1037f0 2024-09-10T08:29:57.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.085+0000 7fc8f1e00640 1 --2- 172.21.15.89:0/445930949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8ec1037f0 0x7fc8ec198f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0004f60 tx=0x7fc8e00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:57.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.085+0000 7fc8e6a00640 1 -- 172.21.15.89:0/445930949 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8e003f070 con 0x7fc8ec1037f0 2024-09-10T08:29:57.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.085+0000 7fc8e6a00640 1 -- 172.21.15.89:0/445930949 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8e0004cc0 con 0x7fc8ec1037f0 2024-09-10T08:29:57.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.085+0000 7fc8f4800640 1 -- 172.21.15.89:0/445930949 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ec199700 con 0x7fc8ec1037f0 2024-09-10T08:29:57.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.085+0000 7fc8e6a00640 1 -- 172.21.15.89:0/445930949 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8e0035030 con 0x7fc8ec1037f0 2024-09-10T08:29:57.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.086+0000 7fc8f4800640 1 -- 172.21.15.89:0/445930949 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ec19c3f0 con 0x7fc8ec1037f0 2024-09-10T08:29:57.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.087+0000 7fc8e6a00640 1 -- 172.21.15.89:0/445930949 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc8e0042070 con 0x7fc8ec1037f0 2024-09-10T08:29:57.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.087+0000 7fc8f4800640 1 -- 172.21.15.89:0/445930949 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8ec10a080 con 0x7fc8ec1037f0 2024-09-10T08:29:57.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.087+0000 7fc8e6a00640 1 --2- 172.21.15.89:0/445930949 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc8d40390c0 0x7fc8d403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:29:57.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.087+0000 7fc8e6a00640 1 -- 172.21.15.89:0/445930949 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc8e006ce50 con 0x7fc8ec1037f0 2024-09-10T08:29:57.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.087+0000 7fc8f1400640 1 --2- 172.21.15.89:0/445930949 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc8d40390c0 0x7fc8d403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:29:57.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.088+0000 7fc8f1400640 1 --2- 172.21.15.89:0/445930949 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc8d40390c0 0x7fc8d403b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0098c0 tx=0x7fc8dc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:29:57.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.092+0000 7fc8e6a00640 1 -- 172.21.15.89:0/445930949 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc8e00433b0 con 0x7fc8ec1037f0 2024-09-10T08:29:57.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:29:57.286+0000 7fc8f4800640 1 -- 172.21.15.89:0/445930949 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi111", "target": ["mon-mgr", ""]}) -- 0x7fc8ec02d830 con 0x7fc8d40390c0 2024-09-10T08:29:57.956 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:57 smithi089 ceph-mon[30757]: Updating smithi089:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:57 smithi089 ceph-mon[30757]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2024-09-10T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:57 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:57 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:57 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:57 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:57 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:57 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:29:57.957 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:57 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:29:59.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:58 smithi089 ceph-mon[30757]: Updating smithi089:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T08:29:59.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:58 smithi089 ceph-mon[30757]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi111", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:30:00.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:29:59 smithi089 ceph-mon[30757]: Deploying cephadm binary to smithi111 2024-09-10T08:30:00.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:00.534+0000 7fc8e6a00640 1 -- 172.21.15.89:0/445930949 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fc8ec02d830 con 0x7fc8d40390c0 2024-09-10T08:30:00.535 INFO:teuthology.orchestra.run.smithi089.stdout:Added host 'smithi111' with addr '172.21.15.111' 2024-09-10T08:30:00.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:00.536+0000 7fc8f4800640 1 -- 172.21.15.89:0/445930949 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc8d40390c0 msgr2=0x7fc8d403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:00.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:00.536+0000 7fc8f4800640 1 --2- 172.21.15.89:0/445930949 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc8d40390c0 0x7fc8d403b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0098c0 tx=0x7fc8dc006db0 comp rx=0 tx=0).stop 2024-09-10T08:30:00.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:00.537+0000 7fc8f4800640 1 -- 172.21.15.89:0/445930949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8ec1037f0 msgr2=0x7fc8ec198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:00.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:00.537+0000 7fc8f4800640 1 --2- 172.21.15.89:0/445930949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8ec1037f0 0x7fc8ec198f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0004f60 tx=0x7fc8e00048f0 comp rx=0 tx=0).stop 2024-09-10T08:30:00.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:00.537+0000 7fc8f4800640 1 -- 172.21.15.89:0/445930949 shutdown_connections 2024-09-10T08:30:00.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:00.537+0000 7fc8f4800640 1 --2- 172.21.15.89:0/445930949 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc8d40390c0 0x7fc8d403b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:00.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:00.537+0000 7fc8f4800640 1 --2- 172.21.15.89:0/445930949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8ec1037f0 0x7fc8ec198f30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:00.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:00.537+0000 7fc8f4800640 1 -- 172.21.15.89:0/445930949 >> 172.21.15.89:0/445930949 conn(0x7fc8ec0fefc0 msgr2=0x7fc8ec0ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:00.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:00.537+0000 7fc8f4800640 1 -- 172.21.15.89:0/445930949 shutdown_connections 2024-09-10T08:30:00.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:00.537+0000 7fc8f4800640 1 -- 172.21.15.89:0/445930949 wait complete. 2024-09-10T08:30:01.248 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T08:30:01.606 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:30:01.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:01 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:01.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:01 smithi089 ceph-mon[30757]: Added host smithi111 2024-09-10T08:30:01.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:01 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:30:01.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:01 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:01.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:01 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.107+0000 7fc307200640 1 -- 172.21.15.89:0/407564576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc308109660 msgr2=0x7fc308109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:02.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.107+0000 7fc307200640 1 --2- 172.21.15.89:0/407564576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc308109660 0x7fc308109a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009880 tx=0x7fc2f802a5d0 comp rx=0 tx=0).stop 2024-09-10T08:30:02.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.107+0000 7fc307200640 1 -- 172.21.15.89:0/407564576 shutdown_connections 2024-09-10T08:30:02.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.107+0000 7fc307200640 1 --2- 172.21.15.89:0/407564576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc308109660 0x7fc308109a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:02.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.107+0000 7fc307200640 1 -- 172.21.15.89:0/407564576 >> 172.21.15.89:0/407564576 conn(0x7fc3080ff290 msgr2=0x7fc3081016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:02.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.107+0000 7fc307200640 1 -- 172.21.15.89:0/407564576 shutdown_connections 2024-09-10T08:30:02.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.107+0000 7fc307200640 1 -- 172.21.15.89:0/407564576 wait complete. 2024-09-10T08:30:02.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.108+0000 7fc307200640 1 Processor -- start 2024-09-10T08:30:02.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.108+0000 7fc307200640 1 -- start start 2024-09-10T08:30:02.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.108+0000 7fc307200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc308109660 0x7fc308199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:02.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.108+0000 7fc307200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc30810dc70 con 0x7fc308109660 2024-09-10T08:30:02.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.109+0000 7fc305e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc308109660 0x7fc308199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:02.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.109+0000 7fc305e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc308109660 0x7fc308199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41762/0 (socket says 172.21.15.89:41762) 2024-09-10T08:30:02.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.109+0000 7fc305e00640 1 -- 172.21.15.89:0/397902083 learned_addr learned my addr 172.21.15.89:0/397902083 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:30:02.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.109+0000 7fc305e00640 1 -- 172.21.15.89:0/397902083 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3081996b0 con 0x7fc308109660 2024-09-10T08:30:02.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.109+0000 7fc305e00640 1 --2- 172.21.15.89:0/397902083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc308109660 0x7fc308199170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009230 tx=0x7fc2f8003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:02.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.110+0000 7fc2fea00640 1 -- 172.21.15.89:0/397902083 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2f803f070 con 0x7fc308109660 2024-09-10T08:30:02.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.110+0000 7fc2fea00640 1 -- 172.21.15.89:0/397902083 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f803bc80 con 0x7fc308109660 2024-09-10T08:30:02.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.110+0000 7fc2fea00640 1 -- 172.21.15.89:0/397902083 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2f8035030 con 0x7fc308109660 2024-09-10T08:30:02.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.110+0000 7fc307200640 1 -- 172.21.15.89:0/397902083 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3081999a0 con 0x7fc308109660 2024-09-10T08:30:02.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.110+0000 7fc307200640 1 -- 172.21.15.89:0/397902083 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc30819da40 con 0x7fc308109660 2024-09-10T08:30:02.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.111+0000 7fc2fea00640 1 -- 172.21.15.89:0/397902083 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc2f8042070 con 0x7fc308109660 2024-09-10T08:30:02.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.111+0000 7fc307200640 1 -- 172.21.15.89:0/397902083 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc308104d10 con 0x7fc308109660 2024-09-10T08:30:02.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.112+0000 7fc2fea00640 1 --2- 172.21.15.89:0/397902083 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc2ec0390c0 0x7fc2ec03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:02.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.112+0000 7fc2fea00640 1 -- 172.21.15.89:0/397902083 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc2f806cd00 con 0x7fc308109660 2024-09-10T08:30:02.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.112+0000 7fc305400640 1 --2- 172.21.15.89:0/397902083 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc2ec0390c0 0x7fc2ec03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:02.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.112+0000 7fc305400640 1 --2- 172.21.15.89:0/397902083 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc2ec0390c0 0x7fc2ec03b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc2f40098c0 tx=0x7fc2f4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:02.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.117+0000 7fc2fea00640 1 -- 172.21.15.89:0/397902083 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc2f80433b0 con 0x7fc308109660 2024-09-10T08:30:02.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.295+0000 7fc307200640 1 -- 172.21.15.89:0/397902083 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc30819a4d0 con 0x7fc2ec0390c0 2024-09-10T08:30:02.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.296+0000 7fc2fea00640 1 -- 172.21.15.89:0/397902083 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7fc30819a4d0 con 0x7fc2ec0390c0 2024-09-10T08:30:02.297 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:30:02.297 INFO:teuthology.orchestra.run.smithi089.stdout:[{"addr": "172.21.15.89", "hostname": "smithi089", "labels": [], "status": ""}, {"addr": "172.21.15.111", "hostname": "smithi111", "labels": [], "status": ""}] 2024-09-10T08:30:02.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.298+0000 7fc307200640 1 -- 172.21.15.89:0/397902083 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc2ec0390c0 msgr2=0x7fc2ec03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:02.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.298+0000 7fc307200640 1 --2- 172.21.15.89:0/397902083 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc2ec0390c0 0x7fc2ec03b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc2f40098c0 tx=0x7fc2f4006db0 comp rx=0 tx=0).stop 2024-09-10T08:30:02.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.298+0000 7fc307200640 1 -- 172.21.15.89:0/397902083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc308109660 msgr2=0x7fc308199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:02.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.298+0000 7fc307200640 1 --2- 172.21.15.89:0/397902083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc308109660 0x7fc308199170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009230 tx=0x7fc2f8003d40 comp rx=0 tx=0).stop 2024-09-10T08:30:02.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.298+0000 7fc307200640 1 -- 172.21.15.89:0/397902083 shutdown_connections 2024-09-10T08:30:02.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.298+0000 7fc307200640 1 --2- 172.21.15.89:0/397902083 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc2ec0390c0 0x7fc2ec03b580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:02.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.298+0000 7fc307200640 1 --2- 172.21.15.89:0/397902083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc308109660 0x7fc308199170 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:02.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.298+0000 7fc307200640 1 -- 172.21.15.89:0/397902083 >> 172.21.15.89:0/397902083 conn(0x7fc3080ff290 msgr2=0x7fc308100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:02.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.299+0000 7fc307200640 1 -- 172.21.15.89:0/397902083 shutdown_connections 2024-09-10T08:30:02.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:02.299+0000 7fc307200640 1 -- 172.21.15.89:0/397902083 wait complete. 2024-09-10T08:30:02.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:02 smithi089 ceph-mon[30757]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:02.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:02 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:02.961 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi149 2024-09-10T08:30:02.961 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T08:30:02.961 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.conf 2024-09-10T08:30:02.982 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T08:30:02.982 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T08:30:03.040 INFO:tasks.cephadm:Adding host smithi149 to orchestrator... 2024-09-10T08:30:03.040 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph orch host add smithi149 2024-09-10T08:30:03.394 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:30:03.785 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:03 smithi089 ceph-mon[30757]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:30:03.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.898+0000 7f2cb7e00640 1 -- 172.21.15.89:0/3360179433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb01065a0 msgr2=0x7f2cb0106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:03.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.898+0000 7f2cb7e00640 1 --2- 172.21.15.89:0/3360179433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb01065a0 0x7f2cb0106980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009880 tx=0x7f2ca402a5d0 comp rx=0 tx=0).stop 2024-09-10T08:30:03.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.898+0000 7f2cb7e00640 1 -- 172.21.15.89:0/3360179433 shutdown_connections 2024-09-10T08:30:03.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.898+0000 7f2cb7e00640 1 --2- 172.21.15.89:0/3360179433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb01065a0 0x7f2cb0106980 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:03.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.898+0000 7f2cb7e00640 1 -- 172.21.15.89:0/3360179433 >> 172.21.15.89:0/3360179433 conn(0x7f2cb00fb4b0 msgr2=0x7f2cb00fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:03.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.898+0000 7f2cb7e00640 1 -- 172.21.15.89:0/3360179433 shutdown_connections 2024-09-10T08:30:03.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.898+0000 7f2cb7e00640 1 -- 172.21.15.89:0/3360179433 wait complete. 2024-09-10T08:30:03.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.899+0000 7f2cb7e00640 1 Processor -- start 2024-09-10T08:30:03.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.899+0000 7f2cb7e00640 1 -- start start 2024-09-10T08:30:03.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.899+0000 7f2cb7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb01065a0 0x7f2cb0100ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:03.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.900+0000 7f2cb7e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb010bc60 con 0x7f2cb01065a0 2024-09-10T08:30:03.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.900+0000 7f2cb5400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb01065a0 0x7f2cb0100ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:03.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.900+0000 7f2cb5400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb01065a0 0x7f2cb0100ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38978/0 (socket says 172.21.15.89:38978) 2024-09-10T08:30:03.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.900+0000 7f2cb5400640 1 -- 172.21.15.89:0/1752028426 learned_addr learned my addr 172.21.15.89:0/1752028426 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:30:03.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.900+0000 7f2cb5400640 1 -- 172.21.15.89:0/1752028426 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb0102ee0 con 0x7f2cb01065a0 2024-09-10T08:30:03.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.900+0000 7f2cb5400640 1 --2- 172.21.15.89:0/1752028426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb01065a0 0x7f2cb0100ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009230 tx=0x7f2ca4003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:03.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.901+0000 7f2cae000640 1 -- 172.21.15.89:0/1752028426 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ca403f070 con 0x7f2cb01065a0 2024-09-10T08:30:03.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.901+0000 7f2cae000640 1 -- 172.21.15.89:0/1752028426 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ca403bd40 con 0x7f2cb01065a0 2024-09-10T08:30:03.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.901+0000 7f2cae000640 1 -- 172.21.15.89:0/1752028426 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ca4035030 con 0x7f2cb01065a0 2024-09-10T08:30:03.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.901+0000 7f2cb7e00640 1 -- 172.21.15.89:0/1752028426 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb0101530 con 0x7f2cb01065a0 2024-09-10T08:30:03.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.901+0000 7f2cb7e00640 1 -- 172.21.15.89:0/1752028426 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb01019f0 con 0x7f2cb01065a0 2024-09-10T08:30:03.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.902+0000 7f2cb7e00640 1 -- 172.21.15.89:0/1752028426 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb0100aa0 con 0x7f2cb01065a0 2024-09-10T08:30:03.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.902+0000 7f2cae000640 1 -- 172.21.15.89:0/1752028426 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2ca4042070 con 0x7f2cb01065a0 2024-09-10T08:30:03.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.902+0000 7f2cae000640 1 --2- 172.21.15.89:0/1752028426 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2c980390c0 0x7f2c9803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:03.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.903+0000 7f2cae000640 1 -- 172.21.15.89:0/1752028426 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2ca406ccd0 con 0x7f2cb01065a0 2024-09-10T08:30:03.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.903+0000 7f2cb4a00640 1 --2- 172.21.15.89:0/1752028426 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2c980390c0 0x7f2c9803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:03.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.903+0000 7f2cb4a00640 1 --2- 172.21.15.89:0/1752028426 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2c980390c0 0x7f2c9803b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0098c0 tx=0x7f2c9c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:03.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:03.907+0000 7f2cae000640 1 -- 172.21.15.89:0/1752028426 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ca40412c0 con 0x7f2cb01065a0 2024-09-10T08:30:04.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:04.086+0000 7f2cb7e00640 1 -- 172.21.15.89:0/1752028426 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}) -- 0x7f2cb0102590 con 0x7f2c980390c0 2024-09-10T08:30:04.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:04 smithi089 ceph-mon[30757]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:06.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:05 smithi089 ceph-mon[30757]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:30:06.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:05 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:06.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:05 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:06.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:05 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:06.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:05 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:06.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:05 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi111", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:30:06.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:05 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:06.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:05 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:30:06.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:05 smithi089 ceph-mon[30757]: Updating smithi111:/etc/ceph/ceph.conf 2024-09-10T08:30:06.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:05 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:06.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:05 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:06.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:05 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:07.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:06 smithi089 ceph-mon[30757]: Updating smithi111:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:30:07.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:06 smithi089 ceph-mon[30757]: Deploying cephadm binary to smithi149 2024-09-10T08:30:07.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:06 smithi089 ceph-mon[30757]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2024-09-10T08:30:07.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:06 smithi089 ceph-mon[30757]: Updating smithi111:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T08:30:07.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:06 smithi089 ceph-mon[30757]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:07.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:07.424+0000 7f2cae000640 1 -- 172.21.15.89:0/1752028426 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f2cb0102590 con 0x7f2c980390c0 2024-09-10T08:30:07.425 INFO:teuthology.orchestra.run.smithi089.stdout:Added host 'smithi149' with addr '172.21.15.149' 2024-09-10T08:30:07.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:07.427+0000 7f2cb7e00640 1 -- 172.21.15.89:0/1752028426 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2c980390c0 msgr2=0x7f2c9803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:07.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:07.427+0000 7f2cb7e00640 1 --2- 172.21.15.89:0/1752028426 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2c980390c0 0x7f2c9803b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0098c0 tx=0x7f2c9c006db0 comp rx=0 tx=0).stop 2024-09-10T08:30:07.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:07.427+0000 7f2cb7e00640 1 -- 172.21.15.89:0/1752028426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb01065a0 msgr2=0x7f2cb0100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:07.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:07.427+0000 7f2cb7e00640 1 --2- 172.21.15.89:0/1752028426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb01065a0 0x7f2cb0100ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009230 tx=0x7f2ca4003d80 comp rx=0 tx=0).stop 2024-09-10T08:30:07.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:07.427+0000 7f2cb7e00640 1 -- 172.21.15.89:0/1752028426 shutdown_connections 2024-09-10T08:30:07.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:07.427+0000 7f2cb7e00640 1 --2- 172.21.15.89:0/1752028426 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2c980390c0 0x7f2c9803b580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:07.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:07.427+0000 7f2cb7e00640 1 --2- 172.21.15.89:0/1752028426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2cb01065a0 0x7f2cb0100ff0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:07.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:07.427+0000 7f2cb7e00640 1 -- 172.21.15.89:0/1752028426 >> 172.21.15.89:0/1752028426 conn(0x7f2cb00fb4b0 msgr2=0x7f2cb01092c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:07.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:07.427+0000 7f2cb7e00640 1 -- 172.21.15.89:0/1752028426 shutdown_connections 2024-09-10T08:30:07.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:07.427+0000 7f2cb7e00640 1 -- 172.21.15.89:0/1752028426 wait complete. 2024-09-10T08:30:08.132 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T08:30:08.495 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:30:08.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:08 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:08.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:08 smithi089 ceph-mon[30757]: Added host smithi149 2024-09-10T08:30:08.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:08 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:30:08.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:08 smithi089 ceph-mon[30757]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:08.746 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:08 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:08.746 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:08 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:09.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.004+0000 7ff3ce600640 1 -- 172.21.15.89:0/1437079684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3c8109660 msgr2=0x7ff3c8109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.004+0000 7ff3ce600640 1 --2- 172.21.15.89:0/1437079684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3c8109660 0x7ff3c8109a40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009880 tx=0x7ff3bc02a5d0 comp rx=0 tx=0).stop 2024-09-10T08:30:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.004+0000 7ff3ce600640 1 -- 172.21.15.89:0/1437079684 shutdown_connections 2024-09-10T08:30:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.004+0000 7ff3ce600640 1 --2- 172.21.15.89:0/1437079684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3c8109660 0x7ff3c8109a40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.004+0000 7ff3ce600640 1 -- 172.21.15.89:0/1437079684 >> 172.21.15.89:0/1437079684 conn(0x7ff3c80ff290 msgr2=0x7ff3c81016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.004+0000 7ff3ce600640 1 -- 172.21.15.89:0/1437079684 shutdown_connections 2024-09-10T08:30:09.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.004+0000 7ff3ce600640 1 -- 172.21.15.89:0/1437079684 wait complete. 2024-09-10T08:30:09.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.005+0000 7ff3ce600640 1 Processor -- start 2024-09-10T08:30:09.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.005+0000 7ff3ce600640 1 -- start start 2024-09-10T08:30:09.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.005+0000 7ff3ce600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3c8109660 0x7ff3c8199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:09.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.005+0000 7ff3ce600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c810dc70 con 0x7ff3c8109660 2024-09-10T08:30:09.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.006+0000 7ff3cd200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3c8109660 0x7ff3c8199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:09.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.006+0000 7ff3cd200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3c8109660 0x7ff3c8199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39004/0 (socket says 172.21.15.89:39004) 2024-09-10T08:30:09.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.006+0000 7ff3cd200640 1 -- 172.21.15.89:0/723912431 learned_addr learned my addr 172.21.15.89:0/723912431 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:30:09.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.006+0000 7ff3cd200640 1 -- 172.21.15.89:0/723912431 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c81996b0 con 0x7ff3c8109660 2024-09-10T08:30:09.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.006+0000 7ff3cd200640 1 --2- 172.21.15.89:0/723912431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3c8109660 0x7ff3c8199170 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009230 tx=0x7ff3bc003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:09.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.007+0000 7ff3c5600640 1 -- 172.21.15.89:0/723912431 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3bc03f070 con 0x7ff3c8109660 2024-09-10T08:30:09.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.007+0000 7ff3ce600640 1 -- 172.21.15.89:0/723912431 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c81999a0 con 0x7ff3c8109660 2024-09-10T08:30:09.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.007+0000 7ff3c5600640 1 -- 172.21.15.89:0/723912431 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3bc03bd40 con 0x7ff3c8109660 2024-09-10T08:30:09.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.007+0000 7ff3c5600640 1 -- 172.21.15.89:0/723912431 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3bc035030 con 0x7ff3c8109660 2024-09-10T08:30:09.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.007+0000 7ff3ce600640 1 -- 172.21.15.89:0/723912431 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c819da40 con 0x7ff3c8109660 2024-09-10T08:30:09.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.008+0000 7ff3ce600640 1 -- 172.21.15.89:0/723912431 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3c8104d10 con 0x7ff3c8109660 2024-09-10T08:30:09.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.008+0000 7ff3c5600640 1 -- 172.21.15.89:0/723912431 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff3bc042070 con 0x7ff3c8109660 2024-09-10T08:30:09.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.008+0000 7ff3c5600640 1 --2- 172.21.15.89:0/723912431 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3b0039060 0x7ff3b003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:09.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.008+0000 7ff3c5600640 1 -- 172.21.15.89:0/723912431 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff3bc06c9a0 con 0x7ff3c8109660 2024-09-10T08:30:09.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.009+0000 7ff3c7e00640 1 --2- 172.21.15.89:0/723912431 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3b0039060 0x7ff3b003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:09.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.009+0000 7ff3c7e00640 1 --2- 172.21.15.89:0/723912431 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3b0039060 0x7ff3b003b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40098c0 tx=0x7ff3b4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:09.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.013+0000 7ff3c5600640 1 -- 172.21.15.89:0/723912431 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff3bc044e10 con 0x7ff3c8109660 2024-09-10T08:30:09.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.194+0000 7ff3ce600640 1 -- 172.21.15.89:0/723912431 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff3c819a4d0 con 0x7ff3b0039060 2024-09-10T08:30:09.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.196+0000 7ff3c5600640 1 -- 172.21.15.89:0/723912431 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7ff3c819a4d0 con 0x7ff3b0039060 2024-09-10T08:30:09.196 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:30:09.196 INFO:teuthology.orchestra.run.smithi089.stdout:[{"addr": "172.21.15.89", "hostname": "smithi089", "labels": [], "status": ""}, {"addr": "172.21.15.111", "hostname": "smithi111", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}] 2024-09-10T08:30:09.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.198+0000 7ff3ce600640 1 -- 172.21.15.89:0/723912431 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3b0039060 msgr2=0x7ff3b003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:09.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.198+0000 7ff3ce600640 1 --2- 172.21.15.89:0/723912431 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3b0039060 0x7ff3b003b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40098c0 tx=0x7ff3b4006db0 comp rx=0 tx=0).stop 2024-09-10T08:30:09.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.198+0000 7ff3ce600640 1 -- 172.21.15.89:0/723912431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3c8109660 msgr2=0x7ff3c8199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:09.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.198+0000 7ff3ce600640 1 --2- 172.21.15.89:0/723912431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3c8109660 0x7ff3c8199170 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009230 tx=0x7ff3bc003d80 comp rx=0 tx=0).stop 2024-09-10T08:30:09.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.198+0000 7ff3ce600640 1 -- 172.21.15.89:0/723912431 shutdown_connections 2024-09-10T08:30:09.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.198+0000 7ff3ce600640 1 --2- 172.21.15.89:0/723912431 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3b0039060 0x7ff3b003b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:09.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.198+0000 7ff3ce600640 1 --2- 172.21.15.89:0/723912431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3c8109660 0x7ff3c8199170 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:09.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.198+0000 7ff3ce600640 1 -- 172.21.15.89:0/723912431 >> 172.21.15.89:0/723912431 conn(0x7ff3c80ff290 msgr2=0x7ff3c8100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:09.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.198+0000 7ff3ce600640 1 -- 172.21.15.89:0/723912431 shutdown_connections 2024-09-10T08:30:09.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:09.198+0000 7ff3ce600640 1 -- 172.21.15.89:0/723912431 wait complete. 2024-09-10T08:30:09.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:09 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:09.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:09 smithi089 ceph-mon[30757]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:30:09.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:09 smithi089 ceph-mon[30757]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:09.860 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T08:30:09.860 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T08:30:10.210 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:30:10.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.709+0000 7f7689600640 1 -- 172.21.15.89:0/3521348464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7684107580 msgr2=0x7f7684107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:10.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.709+0000 7f7689600640 1 --2- 172.21.15.89:0/3521348464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7684107580 0x7f7684107960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7674009880 tx=0x7f767402a5d0 comp rx=0 tx=0).stop 2024-09-10T08:30:10.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.709+0000 7f7689600640 1 -- 172.21.15.89:0/3521348464 shutdown_connections 2024-09-10T08:30:10.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.709+0000 7f7689600640 1 --2- 172.21.15.89:0/3521348464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7684107580 0x7f7684107960 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:10.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.709+0000 7f7689600640 1 -- 172.21.15.89:0/3521348464 >> 172.21.15.89:0/3521348464 conn(0x7f7684078270 msgr2=0x7f76840ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:10.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.710+0000 7f7689600640 1 -- 172.21.15.89:0/3521348464 shutdown_connections 2024-09-10T08:30:10.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.710+0000 7f7689600640 1 -- 172.21.15.89:0/3521348464 wait complete. 2024-09-10T08:30:10.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.710+0000 7f7689600640 1 Processor -- start 2024-09-10T08:30:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.711+0000 7f7689600640 1 -- start start 2024-09-10T08:30:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.711+0000 7f7689600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7684107580 0x7f76840744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.711+0000 7f7689600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f768410bbb0 con 0x7f7684107580 2024-09-10T08:30:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.711+0000 7f7683e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7684107580 0x7f76840744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.711+0000 7f7683e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7684107580 0x7f76840744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39010/0 (socket says 172.21.15.89:39010) 2024-09-10T08:30:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.711+0000 7f7683e00640 1 -- 172.21.15.89:0/2985102818 learned_addr learned my addr 172.21.15.89:0/2985102818 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:30:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.712+0000 7f7683e00640 1 -- 172.21.15.89:0/2985102818 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76840763e0 con 0x7f7684107580 2024-09-10T08:30:10.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.712+0000 7f7683e00640 1 --2- 172.21.15.89:0/2985102818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7684107580 0x7f76840744f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7674009230 tx=0x7f7674003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:10.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.712+0000 7f7681600640 1 -- 172.21.15.89:0/2985102818 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f767403f070 con 0x7f7684107580 2024-09-10T08:30:10.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.712+0000 7f7689600640 1 -- 172.21.15.89:0/2985102818 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7684074a30 con 0x7f7684107580 2024-09-10T08:30:10.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.712+0000 7f7681600640 1 -- 172.21.15.89:0/2985102818 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f767403bd40 con 0x7f7684107580 2024-09-10T08:30:10.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.712+0000 7f7681600640 1 -- 172.21.15.89:0/2985102818 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7674035030 con 0x7f7684107580 2024-09-10T08:30:10.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.712+0000 7f7689600640 1 -- 172.21.15.89:0/2985102818 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7684074ef0 con 0x7f7684107580 2024-09-10T08:30:10.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.713+0000 7f7681600640 1 -- 172.21.15.89:0/2985102818 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7674042070 con 0x7f7684107580 2024-09-10T08:30:10.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.713+0000 7f7689600640 1 -- 172.21.15.89:0/2985102818 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7684102ae0 con 0x7f7684107580 2024-09-10T08:30:10.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.714+0000 7f7681600640 1 --2- 172.21.15.89:0/2985102818 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7668039060 0x7f766803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:10.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.714+0000 7f7681600640 1 -- 172.21.15.89:0/2985102818 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f767406cd00 con 0x7f7684107580 2024-09-10T08:30:10.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.714+0000 7f767b400640 1 --2- 172.21.15.89:0/2985102818 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7668039060 0x7f766803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.714+0000 7f767b400640 1 --2- 172.21.15.89:0/2985102818 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7668039060 0x7f766803b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f766c0098c0 tx=0x7f766c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:10.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.719+0000 7f7681600640 1 -- 172.21.15.89:0/2985102818 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7674036df0 con 0x7f7684107580 2024-09-10T08:30:10.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.901+0000 7f7689600640 1 -- 172.21.15.89:0/2985102818 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f7684075a20 con 0x7f7684107580 2024-09-10T08:30:10.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.953+0000 7f7681600640 1 -- 172.21.15.89:0/2985102818 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f767403b860 con 0x7f7684107580 2024-09-10T08:30:10.953 INFO:teuthology.orchestra.run.smithi089.stderr:adjusted tunables profile to default 2024-09-10T08:30:10.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.955+0000 7f7689600640 1 -- 172.21.15.89:0/2985102818 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7668039060 msgr2=0x7f766803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:10.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.955+0000 7f7689600640 1 --2- 172.21.15.89:0/2985102818 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7668039060 0x7f766803b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f766c0098c0 tx=0x7f766c006db0 comp rx=0 tx=0).stop 2024-09-10T08:30:10.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.955+0000 7f7689600640 1 -- 172.21.15.89:0/2985102818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7684107580 msgr2=0x7f76840744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:10.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.955+0000 7f7689600640 1 --2- 172.21.15.89:0/2985102818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7684107580 0x7f76840744f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7674009230 tx=0x7f7674003d80 comp rx=0 tx=0).stop 2024-09-10T08:30:10.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.955+0000 7f7689600640 1 -- 172.21.15.89:0/2985102818 shutdown_connections 2024-09-10T08:30:10.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.955+0000 7f7689600640 1 --2- 172.21.15.89:0/2985102818 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7668039060 0x7f766803b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:10.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.955+0000 7f7689600640 1 --2- 172.21.15.89:0/2985102818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7684107580 0x7f76840744f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:10.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.955+0000 7f7689600640 1 -- 172.21.15.89:0/2985102818 >> 172.21.15.89:0/2985102818 conn(0x7f7684078270 msgr2=0x7f76840fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:10.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.955+0000 7f7689600640 1 -- 172.21.15.89:0/2985102818 shutdown_connections 2024-09-10T08:30:10.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:10.955+0000 7f7689600640 1 -- 172.21.15.89:0/2985102818 wait complete. 2024-09-10T08:30:11.302 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:10 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2985102818' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T08:30:11.683 INFO:tasks.cephadm:Adding mon.a on smithi089 2024-09-10T08:30:11.683 INFO:tasks.cephadm:Adding mon.b on smithi111 2024-09-10T08:30:11.684 INFO:tasks.cephadm:Adding mon.c on smithi149 2024-09-10T08:30:11.684 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi089:172.21.15.89=a;smithi111:172.21.15.111=b;smithi149:172.21.15.149=c' 2024-09-10T08:30:11.928 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T08:30:11.974 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T08:30:12.302 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:11 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2985102818' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T08:30:12.302 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:11 smithi089 ceph-mon[30757]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:30:12.302 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:11 smithi089 ceph-mon[30757]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:12.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:12.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:12.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:12.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:12.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:30:12.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:12.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:30:12.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:11 smithi089 ceph-mon[30757]: Updating smithi149:/etc/ceph/ceph.conf 2024-09-10T08:30:12.491 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.460+0000 7f145d424640 1 -- 172.21.15.149:0/3088908170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14580ffd10 msgr2=0x7f14581000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:12.491 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.460+0000 7f145d424640 1 --2- 172.21.15.149:0/3088908170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14580ffd10 0x7f14581000f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1448009880 tx=0x7f144802a5d0 comp rx=0 tx=0).stop 2024-09-10T08:30:12.491 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.460+0000 7f145d424640 1 -- 172.21.15.149:0/3088908170 shutdown_connections 2024-09-10T08:30:12.491 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.460+0000 7f145d424640 1 --2- 172.21.15.149:0/3088908170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14580ffd10 0x7f14581000f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:12.491 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.460+0000 7f145d424640 1 -- 172.21.15.149:0/3088908170 >> 172.21.15.149:0/3088908170 conn(0x7f14580fb900 msgr2=0x7f14580fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:12.491 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.460+0000 7f145d424640 1 -- 172.21.15.149:0/3088908170 shutdown_connections 2024-09-10T08:30:12.491 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.460+0000 7f145d424640 1 -- 172.21.15.149:0/3088908170 wait complete. 2024-09-10T08:30:12.491 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.461+0000 7f145d424640 1 Processor -- start 2024-09-10T08:30:12.491 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.461+0000 7f145d424640 1 -- start start 2024-09-10T08:30:12.491 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.461+0000 7f145d424640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14580ffd10 0x7f1458197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:12.492 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.461+0000 7f145d424640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f145810bc30 con 0x7f14580ffd10 2024-09-10T08:30:12.492 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.461+0000 7f1456ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14580ffd10 0x7f1458197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:12.492 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.461+0000 7f1456ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14580ffd10 0x7f1458197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.149:47272/0 (socket says 172.21.15.149:47272) 2024-09-10T08:30:12.492 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.461+0000 7f1456ffd640 1 -- 172.21.15.149:0/3772923691 learned_addr learned my addr 172.21.15.149:0/3772923691 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-09-10T08:30:12.492 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.462+0000 7f1456ffd640 1 -- 172.21.15.149:0/3772923691 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1458197640 con 0x7f14580ffd10 2024-09-10T08:30:12.492 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.462+0000 7f1456ffd640 1 --2- 172.21.15.149:0/3772923691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14580ffd10 0x7f1458197100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1448009230 tx=0x7f1448003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:12.492 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.462+0000 7f1454ff9640 1 -- 172.21.15.149:0/3772923691 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f144803f070 con 0x7f14580ffd10 2024-09-10T08:30:12.492 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.462+0000 7f145d424640 1 -- 172.21.15.149:0/3772923691 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1458197930 con 0x7f14580ffd10 2024-09-10T08:30:12.492 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.462+0000 7f145d424640 1 -- 172.21.15.149:0/3772923691 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f145819b9d0 con 0x7f14580ffd10 2024-09-10T08:30:12.492 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.462+0000 7f1454ff9640 1 -- 172.21.15.149:0/3772923691 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f144803bd40 con 0x7f14580ffd10 2024-09-10T08:30:12.492 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.462+0000 7f1454ff9640 1 -- 172.21.15.149:0/3772923691 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1448049060 con 0x7f14580ffd10 2024-09-10T08:30:12.492 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.463+0000 7f145d424640 1 -- 172.21.15.149:0/3772923691 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1438005290 con 0x7f14580ffd10 2024-09-10T08:30:12.493 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.463+0000 7f1454ff9640 1 -- 172.21.15.149:0/3772923691 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1448042070 con 0x7f14580ffd10 2024-09-10T08:30:12.493 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.463+0000 7f1454ff9640 1 --2- 172.21.15.149:0/3772923691 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14440390c0 0x7f144403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:12.493 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.463+0000 7f1454ff9640 1 -- 172.21.15.149:0/3772923691 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f144806cc00 con 0x7f14580ffd10 2024-09-10T08:30:12.493 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.464+0000 7f144ffff640 1 --2- 172.21.15.149:0/3772923691 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14440390c0 0x7f144403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:12.493 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.464+0000 7f144ffff640 1 --2- 172.21.15.149:0/3772923691 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14440390c0 0x7f144403b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f14400098c0 tx=0x7f1440006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:12.493 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.469+0000 7f1454ff9640 1 -- 172.21.15.149:0/3772923691 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1448039350 con 0x7f14580ffd10 2024-09-10T08:30:12.649 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.650+0000 7f145d424640 1 -- 172.21.15.149:0/3772923691 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi089:172.21.15.89=a;smithi111:172.21.15.111=b;smithi149:172.21.15.149=c", "target": ["mon-mgr", ""]}) -- 0x7f1438000c80 con 0x7f14440390c0 2024-09-10T08:30:12.656 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.657+0000 7f1454ff9640 1 -- 172.21.15.149:0/3772923691 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1438000c80 con 0x7f14440390c0 2024-09-10T08:30:12.656 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mon update... 2024-09-10T08:30:12.658 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.658+0000 7f145d424640 1 -- 172.21.15.149:0/3772923691 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14440390c0 msgr2=0x7f144403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:12.658 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.658+0000 7f145d424640 1 --2- 172.21.15.149:0/3772923691 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14440390c0 0x7f144403b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f14400098c0 tx=0x7f1440006db0 comp rx=0 tx=0).stop 2024-09-10T08:30:12.658 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.658+0000 7f145d424640 1 -- 172.21.15.149:0/3772923691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14580ffd10 msgr2=0x7f1458197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:12.658 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.658+0000 7f145d424640 1 --2- 172.21.15.149:0/3772923691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14580ffd10 0x7f1458197100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1448009230 tx=0x7f1448003d80 comp rx=0 tx=0).stop 2024-09-10T08:30:12.658 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.659+0000 7f145d424640 1 -- 172.21.15.149:0/3772923691 shutdown_connections 2024-09-10T08:30:12.658 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.659+0000 7f145d424640 1 --2- 172.21.15.149:0/3772923691 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14440390c0 0x7f144403b580 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:12.658 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.659+0000 7f145d424640 1 --2- 172.21.15.149:0/3772923691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14580ffd10 0x7f1458197100 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:12.658 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.659+0000 7f145d424640 1 -- 172.21.15.149:0/3772923691 >> 172.21.15.149:0/3772923691 conn(0x7f14580fb900 msgr2=0x7f1458109280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:12.659 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.659+0000 7f145d424640 1 -- 172.21.15.149:0/3772923691 shutdown_connections 2024-09-10T08:30:12.659 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:12.659+0000 7f145d424640 1 -- 172.21.15.149:0/3772923691 wait complete. 2024-09-10T08:30:13.255 DEBUG:teuthology.orchestra.run.smithi111:mon.b> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T08:30:13.258 DEBUG:teuthology.orchestra.run.smithi149:mon.c> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T08:30:13.261 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T08:30:13.261 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T08:30:13.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:13 smithi089 ceph-mon[30757]: Updating smithi149:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:30:13.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:13 smithi089 ceph-mon[30757]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-09-10T08:30:13.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:13 smithi089 ceph-mon[30757]: Updating smithi149:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T08:30:13.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:13.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:13.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:13.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:13 smithi089 ceph-mon[30757]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi089:172.21.15.89=a;smithi111:172.21.15.111=b;smithi149:172.21.15.149=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:30:13.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:13 smithi089 ceph-mon[30757]: Saving service mon spec with placement smithi089:172.21.15.89=a;smithi111:172.21.15.111=b;smithi149:172.21.15.149=c;count:3 2024-09-10T08:30:13.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:13.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:30:13.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:13.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:30:13.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:13.554 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:30:13.554 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:13.554 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:13 smithi089 ceph-mon[30757]: Deploying daemon mon.c on smithi149 2024-09-10T08:30:14.312 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:30:14.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:14 smithi089 ceph-mon[30757]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:15.348 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.337+0000 7f5366e85640 1 -- 172.21.15.149:0/1329093371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5360106540 msgr2=0x7f5360106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:15.348 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.337+0000 7f5366e85640 1 --2- 172.21.15.149:0/1329093371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5360106540 0x7f5360106920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5348009810 tx=0x7f534802a540 comp rx=0 tx=0).stop 2024-09-10T08:30:15.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.337+0000 7f5366e85640 1 -- 172.21.15.149:0/1329093371 shutdown_connections 2024-09-10T08:30:15.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.337+0000 7f5366e85640 1 --2- 172.21.15.149:0/1329093371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5360106540 0x7f5360106920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:15.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.337+0000 7f5366e85640 1 -- 172.21.15.149:0/1329093371 >> 172.21.15.149:0/1329093371 conn(0x7f5360077310 msgr2=0x7f53600fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:15.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.337+0000 7f5366e85640 1 -- 172.21.15.149:0/1329093371 shutdown_connections 2024-09-10T08:30:15.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.337+0000 7f5366e85640 1 -- 172.21.15.149:0/1329093371 wait complete. 2024-09-10T08:30:15.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.338+0000 7f5366e85640 1 Processor -- start 2024-09-10T08:30:15.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.338+0000 7f5366e85640 1 -- start start 2024-09-10T08:30:15.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.338+0000 7f5366e85640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5360106540 0x7f5360197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:15.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.338+0000 7f5366e85640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f536010ab70 con 0x7f5360106540 2024-09-10T08:30:15.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.338+0000 7f5364c21640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5360106540 0x7f5360197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:15.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.338+0000 7f5364c21640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5360106540 0x7f5360197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.149:47284/0 (socket says 172.21.15.149:47284) 2024-09-10T08:30:15.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.338+0000 7f5364c21640 1 -- 172.21.15.149:0/522404013 learned_addr learned my addr 172.21.15.149:0/522404013 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-09-10T08:30:15.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.339+0000 7f5364c21640 1 -- 172.21.15.149:0/522404013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5360198510 con 0x7f5360106540 2024-09-10T08:30:15.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.339+0000 7f5364c21640 1 --2- 172.21.15.149:0/522404013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5360106540 0x7f5360197fd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f53480045e0 tx=0x7f5348004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:15.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.339+0000 7f535dffb640 1 -- 172.21.15.149:0/522404013 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5348030e70 con 0x7f5360106540 2024-09-10T08:30:15.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.339+0000 7f5366e85640 1 -- 172.21.15.149:0/522404013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53601987a0 con 0x7f5360106540 2024-09-10T08:30:15.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.339+0000 7f535dffb640 1 -- 172.21.15.149:0/522404013 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53480318e0 con 0x7f5360106540 2024-09-10T08:30:15.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.339+0000 7f5366e85640 1 -- 172.21.15.149:0/522404013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5360198c00 con 0x7f5360106540 2024-09-10T08:30:15.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.339+0000 7f535dffb640 1 -- 172.21.15.149:0/522404013 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5348049060 con 0x7f5360106540 2024-09-10T08:30:15.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.340+0000 7f535dffb640 1 -- 172.21.15.149:0/522404013 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5348039040 con 0x7f5360106540 2024-09-10T08:30:15.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.341+0000 7f535dffb640 1 --2- 172.21.15.149:0/522404013 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f534c039060 0x7f534c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:15.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.341+0000 7f5366e85640 1 -- 172.21.15.149:0/522404013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5360101bb0 con 0x7f5360106540 2024-09-10T08:30:15.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.341+0000 7f535dffb640 1 -- 172.21.15.149:0/522404013 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f534806d620 con 0x7f5360106540 2024-09-10T08:30:15.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.341+0000 7f535ffff640 1 --2- 172.21.15.149:0/522404013 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f534c039060 0x7f534c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:15.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.342+0000 7f535ffff640 1 --2- 172.21.15.149:0/522404013 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f534c039060 0x7f534c03b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f535000ad30 tx=0x7f53500093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:15.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.346+0000 7f535dffb640 1 -- 172.21.15.149:0/522404013 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5348040030 con 0x7f5360106540 2024-09-10T08:30:15.581 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.582+0000 7f5366e85640 1 -- 172.21.15.149:0/522404013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f53601069a0 con 0x7f5360106540 2024-09-10T08:30:15.582 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.582+0000 7f535dffb640 1 -- 172.21.15.149:0/522404013 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f53480035f0 con 0x7f5360106540 2024-09-10T08:30:15.582 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:30:15.582 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"91f747bc-6f4e-11ef-bcea-c7b262605968","modified":"2024-09-10T08:28:45.922310Z","created":"2024-09-10T08:28:45.922310Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T08:30:15.582 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-09-10T08:30:15.584 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.585+0000 7f5366e85640 1 -- 172.21.15.149:0/522404013 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f534c039060 msgr2=0x7f534c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:15.584 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.585+0000 7f5366e85640 1 --2- 172.21.15.149:0/522404013 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f534c039060 0x7f534c03b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f535000ad30 tx=0x7f53500093f0 comp rx=0 tx=0).stop 2024-09-10T08:30:15.584 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.585+0000 7f5366e85640 1 -- 172.21.15.149:0/522404013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5360106540 msgr2=0x7f5360197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:15.584 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.585+0000 7f5366e85640 1 --2- 172.21.15.149:0/522404013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5360106540 0x7f5360197fd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f53480045e0 tx=0x7f5348004610 comp rx=0 tx=0).stop 2024-09-10T08:30:15.585 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.585+0000 7f5366e85640 1 -- 172.21.15.149:0/522404013 shutdown_connections 2024-09-10T08:30:15.585 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.585+0000 7f5366e85640 1 --2- 172.21.15.149:0/522404013 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f534c039060 0x7f534c03b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:15.585 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.585+0000 7f5366e85640 1 --2- 172.21.15.149:0/522404013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5360106540 0x7f5360197fd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:15.585 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.585+0000 7f5366e85640 1 -- 172.21.15.149:0/522404013 >> 172.21.15.149:0/522404013 conn(0x7f5360077310 msgr2=0x7f53600fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:15.585 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.585+0000 7f5366e85640 1 -- 172.21.15.149:0/522404013 shutdown_connections 2024-09-10T08:30:15.585 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:15.585+0000 7f5366e85640 1 -- 172.21.15.149:0/522404013 wait complete. 2024-09-10T08:30:16.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:16 smithi089 ceph-mon[30757]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:16.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:16 smithi089 ceph-mon[30757]: from='client.? 172.21.15.149:0/522404013' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T08:30:17.555 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T08:30:17.555 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T08:30:17.813 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:30:22.633 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.631+0000 7f7ec7f4e640 1 -- 172.21.15.149:0/4088210502 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7eac003530 msgr2=0x7f7eac0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:22.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.631+0000 7f7ec7f4e640 1 --2- 172.21.15.149:0/4088210502 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7eac003530 0x7f7eac0059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7ec01016a0 tx=0x7f7eb002c490 comp rx=0 tx=0).stop 2024-09-10T08:30:22.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.631+0000 7f7ec7f4e640 1 -- 172.21.15.149:0/4088210502 shutdown_connections 2024-09-10T08:30:22.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.631+0000 7f7ec7f4e640 1 --2- 172.21.15.149:0/4088210502 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7eac003530 0x7f7eac0059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:22.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.631+0000 7f7ec7f4e640 1 -- 172.21.15.149:0/4088210502 >> 172.21.15.149:0/4088210502 conn(0x7f7ec00fc220 msgr2=0x7f7ec00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:22.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.631+0000 7f7ec7f4e640 1 -- 172.21.15.149:0/4088210502 shutdown_connections 2024-09-10T08:30:22.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.631+0000 7f7ec7f4e640 1 -- 172.21.15.149:0/4088210502 wait complete. 2024-09-10T08:30:22.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.632+0000 7f7ec7f4e640 1 Processor -- start 2024-09-10T08:30:22.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.632+0000 7f7ec7f4e640 1 -- start start 2024-09-10T08:30:22.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.632+0000 7f7ec7f4e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7eac003530 0x7f7ec01972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:22.634 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.632+0000 7f7ec7f4e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ec01977e0 0x7f7ec019bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:22.635 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.632+0000 7f7ec7f4e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec0108670 con 0x7f7eac003530 2024-09-10T08:30:22.635 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.632+0000 7f7ec7f4e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec01084f0 con 0x7f7ec01977e0 2024-09-10T08:30:22.635 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.633+0000 7f7ec54e9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ec01977e0 0x7f7ec019bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:22.635 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.633+0000 7f7ec54e9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ec01977e0 0x7f7ec019bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:38376/0 (socket says 172.21.15.149:38376) 2024-09-10T08:30:22.635 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.633+0000 7f7ec54e9640 1 -- 172.21.15.149:0/1116661343 learned_addr learned my addr 172.21.15.149:0/1116661343 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-09-10T08:30:22.635 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.633+0000 7f7ec54e9640 1 -- 172.21.15.149:0/1116661343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ec01977e0 msgr2=0x7f7ec019bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-10T08:30:22.635 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.633+0000 7f7ec54e9640 1 -- 172.21.15.149:0/1116661343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ec01977e0 msgr2=0x7f7ec019bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T08:30:22.635 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.633+0000 7f7ec54e9640 1 --2- 172.21.15.149:0/1116661343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ec01977e0 0x7f7ec019bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T08:30:22.635 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.633+0000 7f7ec5cea640 1 --2- 172.21.15.149:0/1116661343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7eac003530 0x7f7ec01972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:22.635 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.633+0000 7f7ec54e9640 1 --2- 172.21.15.149:0/1116661343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ec01977e0 0x7f7ec019bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T08:30:22.637 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.633+0000 7f7ec5cea640 1 -- 172.21.15.149:0/1116661343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ec01977e0 msgr2=0x7f7ec019bb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:30:22.637 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.633+0000 7f7ec5cea640 1 --2- 172.21.15.149:0/1116661343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ec01977e0 0x7f7ec019bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:22.637 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.633+0000 7f7ec5cea640 1 -- 172.21.15.149:0/1116661343 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec0197e00 con 0x7f7eac003530 2024-09-10T08:30:22.637 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.634+0000 7f7ec5cea640 1 --2- 172.21.15.149:0/1116661343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7eac003530 0x7f7ec01972a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0005f30 tx=0x7f7eb0033680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:22.637 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.634+0000 7f7eb6ffd640 1 -- 172.21.15.149:0/1116661343 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7eb0038040 con 0x7f7eac003530 2024-09-10T08:30:22.637 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.634+0000 7f7eb6ffd640 1 -- 172.21.15.149:0/1116661343 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb0032e00 con 0x7f7eac003530 2024-09-10T08:30:22.637 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.634+0000 7f7ec7f4e640 1 -- 172.21.15.149:0/1116661343 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec019c0b0 con 0x7f7eac003530 2024-09-10T08:30:22.637 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.634+0000 7f7eb6ffd640 1 -- 172.21.15.149:0/1116661343 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7eb0004a00 con 0x7f7eac003530 2024-09-10T08:30:22.637 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.634+0000 7f7ec7f4e640 1 -- 172.21.15.149:0/1116661343 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec019c670 con 0x7f7eac003530 2024-09-10T08:30:22.637 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.635+0000 7f7eb6ffd640 1 -- 172.21.15.149:0/1116661343 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7eb0043020 con 0x7f7eac003530 2024-09-10T08:30:22.637 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.635+0000 7f7eb6ffd640 1 --2- 172.21.15.149:0/1116661343 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7ea4038e00 0x7f7ea403b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:22.637 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.635+0000 7f7eb6ffd640 1 -- 172.21.15.149:0/1116661343 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7eb006e2a0 con 0x7f7eac003530 2024-09-10T08:30:22.638 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.635+0000 7f7ec7f4e640 1 -- 172.21.15.149:0/1116661343 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ec0101ca0 con 0x7f7eac003530 2024-09-10T08:30:22.638 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.636+0000 7f7ec54e9640 1 --2- 172.21.15.149:0/1116661343 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7ea4038e00 0x7f7ea403b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:22.638 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.636+0000 7f7ec54e9640 1 --2- 172.21.15.149:0/1116661343 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7ea4038e00 0x7f7ea403b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009100 tx=0x7f7ebc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:22.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.641+0000 7f7eb6ffd640 1 -- 172.21.15.149:0/1116661343 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7eb004a020 con 0x7f7eac003530 2024-09-10T08:30:22.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:30:22.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:30:22.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: mon.a calling monitor election 2024-09-10T08:30:22.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:22.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:30:22.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:30:22.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: mon.c calling monitor election 2024-09-10T08:30:22.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:22.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:30:22.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:30:22.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:22.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:30:22.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T08:30:22.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: monmap epoch 2 2024-09-10T08:30:22.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:30:22.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: last_changed 2024-09-10T08:30:17.406059+0000 2024-09-10T08:30:22.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: created 2024-09-10T08:28:45.922310+0000 2024-09-10T08:30:22.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: min_mon_release 19 (squid) 2024-09-10T08:30:22.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: election_strategy: 1 2024-09-10T08:30:22.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-10T08:30:22.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-09-10T08:30:22.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: fsmap 2024-09-10T08:30:22.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:30:22.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: mgrmap e13: x(active, since 40s) 2024-09-10T08:30:22.805 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: overall HEALTH_OK 2024-09-10T08:30:22.805 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:22.805 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:22.805 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:22.805 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:30:22.805 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:22 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:22.876 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.876+0000 7f7ec7f4e640 1 -- 172.21.15.149:0/1116661343 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7ec019c920 con 0x7f7eac003530 2024-09-10T08:30:22.877 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.876+0000 7f7eb6ffd640 1 -- 172.21.15.149:0/1116661343 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f7eb0039250 con 0x7f7eac003530 2024-09-10T08:30:22.877 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:30:22.877 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":2,"fsid":"91f747bc-6f4e-11ef-bcea-c7b262605968","modified":"2024-09-10T08:30:17.406059Z","created":"2024-09-10T08:28:45.922310Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T08:30:22.877 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 2 2024-09-10T08:30:22.880 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.879+0000 7f7ec7f4e640 1 -- 172.21.15.149:0/1116661343 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7ea4038e00 msgr2=0x7f7ea403b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:22.880 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.879+0000 7f7ec7f4e640 1 --2- 172.21.15.149:0/1116661343 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7ea4038e00 0x7f7ea403b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009100 tx=0x7f7ebc008040 comp rx=0 tx=0).stop 2024-09-10T08:30:22.880 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.879+0000 7f7ec7f4e640 1 -- 172.21.15.149:0/1116661343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7eac003530 msgr2=0x7f7ec01972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:22.880 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.879+0000 7f7ec7f4e640 1 --2- 172.21.15.149:0/1116661343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7eac003530 0x7f7ec01972a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0005f30 tx=0x7f7eb0033680 comp rx=0 tx=0).stop 2024-09-10T08:30:22.880 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.879+0000 7f7ec7f4e640 1 -- 172.21.15.149:0/1116661343 shutdown_connections 2024-09-10T08:30:22.880 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.879+0000 7f7ec7f4e640 1 --2- 172.21.15.149:0/1116661343 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7ea4038e00 0x7f7ea403b2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:22.880 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.879+0000 7f7ec7f4e640 1 --2- 172.21.15.149:0/1116661343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ec01977e0 0x7f7ec019bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:22.880 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.879+0000 7f7ec7f4e640 1 --2- 172.21.15.149:0/1116661343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7eac003530 0x7f7ec01972a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:22.880 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.879+0000 7f7ec7f4e640 1 -- 172.21.15.149:0/1116661343 >> 172.21.15.149:0/1116661343 conn(0x7f7ec00fc220 msgr2=0x7f7ec00fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:22.880 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.879+0000 7f7ec7f4e640 1 -- 172.21.15.149:0/1116661343 shutdown_connections 2024-09-10T08:30:22.881 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:22.879+0000 7f7ec7f4e640 1 -- 172.21.15.149:0/1116661343 wait complete. 2024-09-10T08:30:23.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:23 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:30:23.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:23 smithi089 ceph-mon[30757]: Deploying daemon mon.b on smithi111 2024-09-10T08:30:23.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:23 smithi089 ceph-mon[30757]: from='client.? 172.21.15.149:0/1116661343' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T08:30:23.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:23 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:30:24.445 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T08:30:24.445 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T08:30:24.713 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:30:24.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:24 smithi089 ceph-mon[30757]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:24.803 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:30:24 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:30:24.406+0000 7f5342e00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T08:30:24.816 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:24 smithi149 ceph-mon[36167]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:25.091 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.089+0000 7f751eaca640 1 -- 172.21.15.149:0/281912465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7518106630 msgr2=0x7f7518106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:25.092 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.089+0000 7f751eaca640 1 --2- 172.21.15.149:0/281912465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7518106630 0x7f7518106a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f750c009880 tx=0x7f750c02a5d0 comp rx=0 tx=0).stop 2024-09-10T08:30:25.092 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.089+0000 7f751eaca640 1 -- 172.21.15.149:0/281912465 shutdown_connections 2024-09-10T08:30:25.092 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.089+0000 7f751eaca640 1 --2- 172.21.15.149:0/281912465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7518106630 0x7f7518106a10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:25.092 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.089+0000 7f751eaca640 1 -- 172.21.15.149:0/281912465 >> 172.21.15.149:0/281912465 conn(0x7f75180fc220 msgr2=0x7f75180fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:25.092 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.089+0000 7f751eaca640 1 -- 172.21.15.149:0/281912465 shutdown_connections 2024-09-10T08:30:25.092 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.089+0000 7f751eaca640 1 -- 172.21.15.149:0/281912465 wait complete. 2024-09-10T08:30:25.092 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.089+0000 7f751eaca640 1 Processor -- start 2024-09-10T08:30:25.092 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.089+0000 7f751eaca640 1 -- start start 2024-09-10T08:30:25.092 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.090+0000 7f751eaca640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7518106630 0x7f75181962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:25.092 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.090+0000 7f751eaca640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7518196820 0x7f751819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:25.092 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.090+0000 7f751eaca640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f751810ac60 con 0x7f7518196820 2024-09-10T08:30:25.092 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.090+0000 7f751eaca640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f751810aae0 con 0x7f7518106630 2024-09-10T08:30:25.093 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.090+0000 7f751c866640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7518106630 0x7f75181962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:25.093 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.090+0000 7f751c866640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7518106630 0x7f75181962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:38382/0 (socket says 172.21.15.149:38382) 2024-09-10T08:30:25.093 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.090+0000 7f751c866640 1 -- 172.21.15.149:0/2558760495 learned_addr learned my addr 172.21.15.149:0/2558760495 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-09-10T08:30:25.093 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.090+0000 7f751c866640 1 -- 172.21.15.149:0/2558760495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7518106630 msgr2=0x7f75181962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-10T08:30:25.093 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.090+0000 7f751c866640 1 -- 172.21.15.149:0/2558760495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7518106630 msgr2=0x7f75181962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T08:30:25.093 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.090+0000 7f751c866640 1 --2- 172.21.15.149:0/2558760495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7518106630 0x7f75181962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T08:30:25.093 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.090+0000 7f751c866640 1 --2- 172.21.15.149:0/2558760495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7518106630 0x7f75181962e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T08:30:25.093 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.090+0000 7f7517fff640 1 --2- 172.21.15.149:0/2558760495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7518196820 0x7f751819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:25.093 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.091+0000 7f7517fff640 1 -- 172.21.15.149:0/2558760495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7518106630 msgr2=0x7f75181962e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:30:25.093 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.091+0000 7f7517fff640 1 --2- 172.21.15.149:0/2558760495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7518106630 0x7f75181962e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:25.093 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.091+0000 7f7517fff640 1 -- 172.21.15.149:0/2558760495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7518196e40 con 0x7f7518196820 2024-09-10T08:30:25.093 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.091+0000 7f7517fff640 1 --2- 172.21.15.149:0/2558760495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7518196820 0x7f751819abb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f750400b3d0 tx=0x7f750400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:25.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.091+0000 7f7515ffb640 1 -- 172.21.15.149:0/2558760495 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7504011630 con 0x7f7518196820 2024-09-10T08:30:25.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.091+0000 7f751eaca640 1 -- 172.21.15.149:0/2558760495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751819b0f0 con 0x7f7518196820 2024-09-10T08:30:25.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.091+0000 7f7515ffb640 1 -- 172.21.15.149:0/2558760495 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7504011cb0 con 0x7f7518196820 2024-09-10T08:30:25.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.091+0000 7f7515ffb640 1 -- 172.21.15.149:0/2558760495 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7504010df0 con 0x7f7518196820 2024-09-10T08:30:25.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.091+0000 7f751eaca640 1 -- 172.21.15.149:0/2558760495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f751819b630 con 0x7f7518196820 2024-09-10T08:30:25.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.092+0000 7f7515ffb640 1 -- 172.21.15.149:0/2558760495 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7504018020 con 0x7f7518196820 2024-09-10T08:30:25.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.092+0000 7f751eaca640 1 -- 172.21.15.149:0/2558760495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7518101ca0 con 0x7f7518196820 2024-09-10T08:30:25.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.092+0000 7f7515ffb640 1 --2- 172.21.15.149:0/2558760495 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7508038e00 0x7f750803b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:25.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.092+0000 7f7515ffb640 1 -- 172.21.15.149:0/2558760495 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f750404bc80 con 0x7f7518196820 2024-09-10T08:30:25.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.093+0000 7f751c866640 1 --2- 172.21.15.149:0/2558760495 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7508038e00 0x7f750803b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:25.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.093+0000 7f751c866640 1 --2- 172.21.15.149:0/2558760495 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7508038e00 0x7f750803b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f750c0096c0 tx=0x7f750c005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:25.098 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.098+0000 7f7515ffb640 1 -- 172.21.15.149:0/2558760495 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7504052080 con 0x7f7518196820 2024-09-10T08:30:25.327 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.326+0000 7f751eaca640 1 -- 172.21.15.149:0/2558760495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f751819b940 con 0x7f7518196820 2024-09-10T08:30:25.328 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.327+0000 7f7515ffb640 1 -- 172.21.15.149:0/2558760495 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f750401f020 con 0x7f7518196820 2024-09-10T08:30:25.328 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:30:25.328 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":2,"fsid":"91f747bc-6f4e-11ef-bcea-c7b262605968","modified":"2024-09-10T08:30:17.406059Z","created":"2024-09-10T08:28:45.922310Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T08:30:25.328 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 2 2024-09-10T08:30:25.330 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.329+0000 7f751eaca640 1 -- 172.21.15.149:0/2558760495 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7508038e00 msgr2=0x7f750803b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:25.330 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.329+0000 7f751eaca640 1 --2- 172.21.15.149:0/2558760495 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7508038e00 0x7f750803b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f750c0096c0 tx=0x7f750c005a40 comp rx=0 tx=0).stop 2024-09-10T08:30:25.330 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.329+0000 7f751eaca640 1 -- 172.21.15.149:0/2558760495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7518196820 msgr2=0x7f751819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:25.330 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.329+0000 7f751eaca640 1 --2- 172.21.15.149:0/2558760495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7518196820 0x7f751819abb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f750400b3d0 tx=0x7f750400b890 comp rx=0 tx=0).stop 2024-09-10T08:30:25.330 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.329+0000 7f751eaca640 1 -- 172.21.15.149:0/2558760495 shutdown_connections 2024-09-10T08:30:25.330 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.329+0000 7f751eaca640 1 --2- 172.21.15.149:0/2558760495 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7508038e00 0x7f750803b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:25.330 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.329+0000 7f751eaca640 1 --2- 172.21.15.149:0/2558760495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7518196820 0x7f751819abb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:25.330 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.329+0000 7f751eaca640 1 --2- 172.21.15.149:0/2558760495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7518106630 0x7f75181962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:25.331 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.329+0000 7f751eaca640 1 -- 172.21.15.149:0/2558760495 >> 172.21.15.149:0/2558760495 conn(0x7f75180fc220 msgr2=0x7f75180fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:25.331 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.329+0000 7f751eaca640 1 -- 172.21.15.149:0/2558760495 shutdown_connections 2024-09-10T08:30:25.331 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:25.329+0000 7f751eaca640 1 -- 172.21.15.149:0/2558760495 wait complete. 2024-09-10T08:30:25.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:25 smithi149 ceph-mon[36167]: from='client.? 172.21.15.149:0/2558760495' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T08:30:25.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:25 smithi089 ceph-mon[30757]: from='client.? 172.21.15.149:0/2558760495' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T08:30:25.973 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 podman[35202]: 2024-09-10 08:30:25.952034902 +0000 UTC m=+0.339109653 container start 041932f325a20ba08b274792a283e682998dc246f9c4cd06cc9ad6a900ba823c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T08:30:25.973 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T08:30:25.973 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable), process ceph-mon, pid 2 2024-09-10T08:30:25.973 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: pidfile_write: ignore empty --pid-file 2024-09-10T08:30:26.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: load: jerasure load: lrc 2024-09-10T08:30:26.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: RocksDB version: 7.9.2 2024-09-10T08:30:26.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Git sha 0 2024-09-10T08:30:26.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Compile date 2024-09-09 20:43:47 2024-09-10T08:30:26.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: DB SUMMARY 2024-09-10T08:30:26.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: DB Session ID: 0WEDXMNVVGMV6FMO9MJE 2024-09-10T08:30:26.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: CURRENT file: CURRENT 2024-09-10T08:30:26.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T08:30:26.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-10T08:30:26.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-10T08:30:26.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-10T08:30:26.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.error_if_exists: 0 2024-09-10T08:30:26.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.create_if_missing: 0 2024-09-10T08:30:26.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.paranoid_checks: 1 2024-09-10T08:30:26.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T08:30:26.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T08:30:26.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T08:30:26.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.env: 0x55fd3d321940 2024-09-10T08:30:26.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T08:30:26.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.info_log: 0x55fd3e1b56e0 2024-09-10T08:30:26.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T08:30:26.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.statistics: (nil) 2024-09-10T08:30:26.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.use_fsync: 0 2024-09-10T08:30:26.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_log_file_size: 0 2024-09-10T08:30:26.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T08:30:26.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T08:30:26.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T08:30:26.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T08:30:26.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.allow_fallocate: 1 2024-09-10T08:30:26.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T08:30:26.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T08:30:26.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.use_direct_reads: 0 2024-09-10T08:30:26.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T08:30:26.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T08:30:26.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.db_log_dir: 2024-09-10T08:30:26.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.wal_dir: 2024-09-10T08:30:26.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T08:30:26.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T08:30:26.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T08:30:26.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T08:30:26.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T08:30:26.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T08:30:26.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T08:30:26.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T08:30:26.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.write_buffer_manager: 0x55fd3e1c4dc0 2024-09-10T08:30:26.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T08:30:26.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T08:30:26.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T08:30:26.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T08:30:26.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T08:30:26.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T08:30:26.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T08:30:26.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T08:30:26.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.unordered_write: 0 2024-09-10T08:30:26.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T08:30:26.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T08:30:26.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T08:30:26.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T08:30:26.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.row_cache: None 2024-09-10T08:30:26.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.wal_filter: None 2024-09-10T08:30:26.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T08:30:26.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T08:30:26.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.two_write_queues: 0 2024-09-10T08:30:26.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T08:30:26.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.wal_compression: 0 2024-09-10T08:30:26.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.atomic_flush: 0 2024-09-10T08:30:26.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T08:30:26.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T08:30:26.299 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T08:30:26.299 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.log_readahead_size: 0 2024-09-10T08:30:26.299 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T08:30:26.299 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T08:30:26.299 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T08:30:26.299 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T08:30:26.299 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T08:30:26.299 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T08:30:26.299 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T08:30:26.299 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_background_jobs: 2 2024-09-10T08:30:26.299 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_background_compactions: -1 2024-09-10T08:30:26.299 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_subcompactions: 1 2024-09-10T08:30:26.301 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T08:30:26.301 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T08:30:26.301 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T08:30:26.301 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T08:30:26.301 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T08:30:26.301 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T08:30:26.301 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T08:30:26.301 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T08:30:26.302 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_open_files: -1 2024-09-10T08:30:26.302 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T08:30:26.302 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T08:30:26.302 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T08:30:26.302 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T08:30:26.302 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_background_flushes: -1 2024-09-10T08:30:26.302 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Compression algorithms supported: 2024-09-10T08:30:26.302 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: kZSTD supported: 0 2024-09-10T08:30:26.302 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: kXpressCompression supported: 0 2024-09-10T08:30:26.302 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: kBZip2Compression supported: 0 2024-09-10T08:30:26.302 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T08:30:26.302 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: kLZ4Compression supported: 1 2024-09-10T08:30:26.302 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: kZlibCompression supported: 1 2024-09-10T08:30:26.302 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T08:30:26.303 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: kSnappyCompression supported: 1 2024-09-10T08:30:26.303 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T08:30:26.303 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T08:30:26.303 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-10T08:30:26.303 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T08:30:26.303 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T08:30:26.303 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.merge_operator: 2024-09-10T08:30:26.303 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compaction_filter: None 2024-09-10T08:30:26.303 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T08:30:26.303 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T08:30:26.303 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T08:30:26.303 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T08:30:26.303 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fd3e1b5320) 2024-09-10T08:30:26.303 INFO:journalctl@ceph.mon.b.smithi111.stdout: cache_index_and_filter_blocks: 1 2024-09-10T08:30:26.304 INFO:journalctl@ceph.mon.b.smithi111.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T08:30:26.304 INFO:journalctl@ceph.mon.b.smithi111.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T08:30:26.304 INFO:journalctl@ceph.mon.b.smithi111.stdout: pin_top_level_index_and_filter: 1 2024-09-10T08:30:26.304 INFO:journalctl@ceph.mon.b.smithi111.stdout: index_type: 0 2024-09-10T08:30:26.304 INFO:journalctl@ceph.mon.b.smithi111.stdout: data_block_index_type: 0 2024-09-10T08:30:26.304 INFO:journalctl@ceph.mon.b.smithi111.stdout: index_shortening: 1 2024-09-10T08:30:26.304 INFO:journalctl@ceph.mon.b.smithi111.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T08:30:26.304 INFO:journalctl@ceph.mon.b.smithi111.stdout: checksum: 4 2024-09-10T08:30:26.304 INFO:journalctl@ceph.mon.b.smithi111.stdout: no_block_cache: 0 2024-09-10T08:30:26.304 INFO:journalctl@ceph.mon.b.smithi111.stdout: block_cache: 0x55fd3e1a91f0 2024-09-10T08:30:26.304 INFO:journalctl@ceph.mon.b.smithi111.stdout: block_cache_name: BinnedLRUCache 2024-09-10T08:30:26.304 INFO:journalctl@ceph.mon.b.smithi111.stdout: block_cache_options: 2024-09-10T08:30:26.304 INFO:journalctl@ceph.mon.b.smithi111.stdout: capacity : 536870912 2024-09-10T08:30:26.305 INFO:journalctl@ceph.mon.b.smithi111.stdout: num_shard_bits : 4 2024-09-10T08:30:26.305 INFO:journalctl@ceph.mon.b.smithi111.stdout: strict_capacity_limit : 0 2024-09-10T08:30:26.305 INFO:journalctl@ceph.mon.b.smithi111.stdout: high_pri_pool_ratio: 0.000 2024-09-10T08:30:26.305 INFO:journalctl@ceph.mon.b.smithi111.stdout: block_cache_compressed: (nil) 2024-09-10T08:30:26.305 INFO:journalctl@ceph.mon.b.smithi111.stdout: persistent_cache: (nil) 2024-09-10T08:30:26.305 INFO:journalctl@ceph.mon.b.smithi111.stdout: block_size: 4096 2024-09-10T08:30:26.305 INFO:journalctl@ceph.mon.b.smithi111.stdout: block_size_deviation: 10 2024-09-10T08:30:26.305 INFO:journalctl@ceph.mon.b.smithi111.stdout: block_restart_interval: 16 2024-09-10T08:30:26.305 INFO:journalctl@ceph.mon.b.smithi111.stdout: index_block_restart_interval: 1 2024-09-10T08:30:26.305 INFO:journalctl@ceph.mon.b.smithi111.stdout: metadata_block_size: 4096 2024-09-10T08:30:26.305 INFO:journalctl@ceph.mon.b.smithi111.stdout: partition_filters: 0 2024-09-10T08:30:26.305 INFO:journalctl@ceph.mon.b.smithi111.stdout: use_delta_encoding: 1 2024-09-10T08:30:26.306 INFO:journalctl@ceph.mon.b.smithi111.stdout: filter_policy: bloomfilter 2024-09-10T08:30:26.306 INFO:journalctl@ceph.mon.b.smithi111.stdout: whole_key_filtering: 1 2024-09-10T08:30:26.306 INFO:journalctl@ceph.mon.b.smithi111.stdout: verify_compression: 0 2024-09-10T08:30:26.306 INFO:journalctl@ceph.mon.b.smithi111.stdout: read_amp_bytes_per_bit: 0 2024-09-10T08:30:26.306 INFO:journalctl@ceph.mon.b.smithi111.stdout: format_version: 5 2024-09-10T08:30:26.306 INFO:journalctl@ceph.mon.b.smithi111.stdout: enable_index_compression: 1 2024-09-10T08:30:26.306 INFO:journalctl@ceph.mon.b.smithi111.stdout: block_align: 0 2024-09-10T08:30:26.306 INFO:journalctl@ceph.mon.b.smithi111.stdout: max_auto_readahead_size: 262144 2024-09-10T08:30:26.306 INFO:journalctl@ceph.mon.b.smithi111.stdout: prepopulate_block_cache: 0 2024-09-10T08:30:26.306 INFO:journalctl@ceph.mon.b.smithi111.stdout: initial_auto_readahead_size: 8192 2024-09-10T08:30:26.306 INFO:journalctl@ceph.mon.b.smithi111.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T08:30:26.306 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T08:30:26.306 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T08:30:26.306 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compression: NoCompression 2024-09-10T08:30:26.306 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T08:30:26.307 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T08:30:26.307 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T08:30:26.307 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.num_levels: 7 2024-09-10T08:30:26.307 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T08:30:26.307 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T08:30:26.307 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T08:30:26.307 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T08:30:26.307 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T08:30:26.307 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T08:30:26.307 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T08:30:26.307 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T08:30:26.307 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T08:30:26.307 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T08:30:26.307 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T08:30:26.308 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T08:30:26.308 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T08:30:26.308 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T08:30:26.308 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T08:30:26.308 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T08:30:26.308 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T08:30:26.308 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T08:30:26.308 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T08:30:26.308 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T08:30:26.308 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T08:30:26.308 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T08:30:26.308 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T08:30:26.308 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T08:30:26.308 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T08:30:26.309 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T08:30:26.309 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T08:30:26.309 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T08:30:26.309 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T08:30:26.309 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T08:30:26.309 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T08:30:26.309 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T08:30:26.309 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T08:30:26.309 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T08:30:26.309 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T08:30:26.309 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T08:30:26.309 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T08:30:26.309 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T08:30:26.309 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T08:30:26.310 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T08:30:26.310 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T08:30:26.310 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T08:30:26.310 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T08:30:26.310 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T08:30:26.310 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T08:30:26.310 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T08:30:26.310 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T08:30:26.310 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T08:30:26.310 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T08:30:26.310 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T08:30:26.310 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T08:30:26.310 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T08:30:26.310 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T08:30:26.311 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T08:30:26.311 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.inplace_update_support: 0 2024-09-10T08:30:26.311 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T08:30:26.311 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T08:30:26.311 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T08:30:26.311 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T08:30:26.311 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.bloom_locality: 0 2024-09-10T08:30:26.311 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.max_successive_merges: 0 2024-09-10T08:30:26.311 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T08:30:26.311 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T08:30:26.311 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T08:30:26.311 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T08:30:26.311 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.ttl: 2592000 2024-09-10T08:30:26.311 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T08:30:26.312 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T08:30:26.312 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T08:30:26.312 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.enable_blob_files: false 2024-09-10T08:30:26.312 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.min_blob_size: 0 2024-09-10T08:30:26.312 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T08:30:26.312 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T08:30:26.312 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T08:30:26.312 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T08:30:26.312 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T08:30:26.312 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T08:30:26.312 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T08:30:26.312 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T08:30:26.312 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T08:30:26.312 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-10T08:30:26.313 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 148d3be6-fd50-4d10-9ef9-6ca4395be0be 2024-09-10T08:30:26.313 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725957025979366, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T08:30:26.313 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-10T08:30:26.313 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725957025979785, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725957025, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "148d3be6-fd50-4d10-9ef9-6ca4395be0be", "db_session_id": "0WEDXMNVVGMV6FMO9MJE", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-10T08:30:26.313 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725957025979886, "job": 1, "event": "recovery_finished"} 2024-09-10T08:30:26.313 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-10T08:30:26.313 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T08:30:26.313 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55fd3e1d6e00 2024-09-10T08:30:26.313 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: DB pointer 0x55fd3e2d2000 2024-09-10T08:30:26.313 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-10T08:30:26.313 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T08:30:26.313 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T08:30:26.313 INFO:journalctl@ceph.mon.b.smithi111.stdout: ** DB Stats ** 2024-09-10T08:30:26.314 INFO:journalctl@ceph.mon.b.smithi111.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T08:30:26.314 INFO:journalctl@ceph.mon.b.smithi111.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T08:30:26.314 INFO:journalctl@ceph.mon.b.smithi111.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T08:30:26.314 INFO:journalctl@ceph.mon.b.smithi111.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T08:30:26.314 INFO:journalctl@ceph.mon.b.smithi111.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T08:30:26.314 INFO:journalctl@ceph.mon.b.smithi111.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T08:30:26.314 INFO:journalctl@ceph.mon.b.smithi111.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T08:30:26.314 INFO:journalctl@ceph.mon.b.smithi111.stdout: 2024-09-10T08:30:26.314 INFO:journalctl@ceph.mon.b.smithi111.stdout: ** Compaction Stats [default] ** 2024-09-10T08:30:26.314 INFO:journalctl@ceph.mon.b.smithi111.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T08:30:26.314 INFO:journalctl@ceph.mon.b.smithi111.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T08:30:26.314 INFO:journalctl@ceph.mon.b.smithi111.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T08:30:26.314 INFO:journalctl@ceph.mon.b.smithi111.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T08:30:26.314 INFO:journalctl@ceph.mon.b.smithi111.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T08:30:26.314 INFO:journalctl@ceph.mon.b.smithi111.stdout: 2024-09-10T08:30:26.315 INFO:journalctl@ceph.mon.b.smithi111.stdout: ** Compaction Stats [default] ** 2024-09-10T08:30:26.315 INFO:journalctl@ceph.mon.b.smithi111.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T08:30:26.315 INFO:journalctl@ceph.mon.b.smithi111.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T08:30:26.315 INFO:journalctl@ceph.mon.b.smithi111.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T08:30:26.315 INFO:journalctl@ceph.mon.b.smithi111.stdout: 2024-09-10T08:30:26.315 INFO:journalctl@ceph.mon.b.smithi111.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T08:30:26.315 INFO:journalctl@ceph.mon.b.smithi111.stdout: 2024-09-10T08:30:26.315 INFO:journalctl@ceph.mon.b.smithi111.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T08:30:26.315 INFO:journalctl@ceph.mon.b.smithi111.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T08:30:26.315 INFO:journalctl@ceph.mon.b.smithi111.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T08:30:26.315 INFO:journalctl@ceph.mon.b.smithi111.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T08:30:26.315 INFO:journalctl@ceph.mon.b.smithi111.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T08:30:26.315 INFO:journalctl@ceph.mon.b.smithi111.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T08:30:26.315 INFO:journalctl@ceph.mon.b.smithi111.stdout: Cumulative compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T08:30:26.316 INFO:journalctl@ceph.mon.b.smithi111.stdout: Interval compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T08:30:26.316 INFO:journalctl@ceph.mon.b.smithi111.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T08:30:26.316 INFO:journalctl@ceph.mon.b.smithi111.stdout: Block cache BinnedLRUCache@0x55fd3e1a91f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-09-10T08:30:26.316 INFO:journalctl@ceph.mon.b.smithi111.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-10T08:30:26.316 INFO:journalctl@ceph.mon.b.smithi111.stdout: 2024-09-10T08:30:26.316 INFO:journalctl@ceph.mon.b.smithi111.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T08:30:26.316 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: using public_addr v2:172.21.15.111:0/0 -> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] 2024-09-10T08:30:26.316 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: starting mon.b rank -1 at public addrs [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] at bind addrs [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:30:26.316 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:25 smithi111 ceph-mon[35220]: mon.b@-1(???) e0 preinit fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:30:26.316 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: mon.b@-1(synchronizing).mds e1 new map 2024-09-10T08:30:26.316 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-10T08:30:26.316 INFO:journalctl@ceph.mon.b.smithi111.stdout: e1 2024-09-10T08:30:26.316 INFO:journalctl@ceph.mon.b.smithi111.stdout: btime 2024-09-10T08:28:49:696758+0000 2024-09-10T08:30:26.316 INFO:journalctl@ceph.mon.b.smithi111.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T08:30:26.316 INFO:journalctl@ceph.mon.b.smithi111.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T08:30:26.317 INFO:journalctl@ceph.mon.b.smithi111.stdout: legacy client fscid: -1 2024-09-10T08:30:26.317 INFO:journalctl@ceph.mon.b.smithi111.stdout: 2024-09-10T08:30:26.317 INFO:journalctl@ceph.mon.b.smithi111.stdout: No filesystems configured 2024-09-10T08:30:26.317 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T08:30:26.317 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T08:30:26.317 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T08:30:26.317 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T08:30:26.317 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T08:30:26.317 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T08:30:26.317 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T08:30:26.317 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T08:30:26.317 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T08:30:26.317 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T08:30:26.317 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:30:26.318 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:26.318 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:30:26.318 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.318 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.318 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.318 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.318 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi111", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:30:26.318 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:26.318 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:30:26.318 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: Updating smithi111:/etc/ceph/ceph.conf 2024-09-10T08:30:26.318 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.318 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.318 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.319 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: Updating smithi111:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:30:26.319 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: Deploying cephadm binary to smithi149 2024-09-10T08:30:26.319 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2024-09-10T08:30:26.319 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: Updating smithi111:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T08:30:26.319 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:26.319 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.319 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: Added host smithi149 2024-09-10T08:30:26.319 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:30:26.319 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:26.319 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.319 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.319 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.319 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:30:26.319 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:26.320 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2985102818' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T08:30:26.320 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2985102818' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T08:30:26.320 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:30:26.320 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:26.320 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.320 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.320 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.320 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.320 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:30:26.320 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:26.320 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:30:26.320 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: Updating smithi149:/etc/ceph/ceph.conf 2024-09-10T08:30:26.320 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: Updating smithi149:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:30:26.320 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-09-10T08:30:26.321 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: Updating smithi149:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T08:30:26.321 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.321 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.321 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.321 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi089:172.21.15.89=a;smithi111:172.21.15.111=b;smithi149:172.21.15.149=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:30:26.321 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: Saving service mon spec with placement smithi089:172.21.15.89=a;smithi111:172.21.15.111=b;smithi149:172.21.15.149=c;count:3 2024-09-10T08:30:26.321 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.321 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:30:26.321 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:26.321 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:30:26.321 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.321 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:30:26.321 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:26.322 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: Deploying daemon mon.c on smithi149 2024-09-10T08:30:26.322 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:26.322 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:26.322 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='client.? 172.21.15.149:0/522404013' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T08:30:26.322 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:30:26.322 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:30:26.322 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: mon.a calling monitor election 2024-09-10T08:30:26.322 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:26.322 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:30:26.322 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:30:26.322 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: mon.c calling monitor election 2024-09-10T08:30:26.322 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:26.322 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:30:26.323 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:30:26.323 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:26.323 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:30:26.323 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T08:30:26.323 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: monmap epoch 2 2024-09-10T08:30:26.323 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:30:26.323 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: last_changed 2024-09-10T08:30:17.406059+0000 2024-09-10T08:30:26.323 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: created 2024-09-10T08:28:45.922310+0000 2024-09-10T08:30:26.323 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: min_mon_release 19 (squid) 2024-09-10T08:30:26.323 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: election_strategy: 1 2024-09-10T08:30:26.323 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-10T08:30:26.323 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-09-10T08:30:26.323 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: fsmap 2024-09-10T08:30:26.323 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:30:26.324 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: mgrmap e13: x(active, since 40s) 2024-09-10T08:30:26.324 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: overall HEALTH_OK 2024-09-10T08:30:26.324 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.324 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.324 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:26.324 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:30:26.324 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:26.324 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: Deploying daemon mon.b on smithi111 2024-09-10T08:30:26.324 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='client.? 172.21.15.149:0/1116661343' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T08:30:26.324 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:30:26.324 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:26.324 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: from='client.? 172.21.15.149:0/2558760495' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T08:30:26.324 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 ceph-mon[35220]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T08:30:26.325 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 bash[35202]: 041932f325a20ba08b274792a283e682998dc246f9c4cd06cc9ad6a900ba823c 2024-09-10T08:30:26.325 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:26 smithi111 systemd[1]: Started Ceph mon.b for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:30:26.849 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T08:30:26.850 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T08:30:27.100 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:30:31.107 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.105+0000 7fdf2853d640 1 -- 172.21.15.149:0/23996713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c005740 msgr2=0x7fdf0c007bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:31.107 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.105+0000 7fdf2853d640 1 --2- 172.21.15.149:0/23996713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c005740 0x7fdf0c007bd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdf201016a0 tx=0x7fdf1c02c3f0 comp rx=0 tx=0).stop 2024-09-10T08:30:31.107 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.106+0000 7fdf2853d640 1 -- 172.21.15.149:0/23996713 shutdown_connections 2024-09-10T08:30:31.107 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.106+0000 7fdf2853d640 1 --2- 172.21.15.149:0/23996713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c005740 0x7fdf0c007bd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:31.107 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.106+0000 7fdf2853d640 1 -- 172.21.15.149:0/23996713 >> 172.21.15.149:0/23996713 conn(0x7fdf200fc220 msgr2=0x7fdf200fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:31.107 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.106+0000 7fdf2853d640 1 -- 172.21.15.149:0/23996713 shutdown_connections 2024-09-10T08:30:31.108 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.106+0000 7fdf2853d640 1 -- 172.21.15.149:0/23996713 wait complete. 2024-09-10T08:30:31.110 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.107+0000 7fdf2853d640 1 Processor -- start 2024-09-10T08:30:31.110 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.107+0000 7fdf2853d640 1 -- start start 2024-09-10T08:30:31.110 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.107+0000 7fdf2853d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c005740 0x7fdf20195080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:31.110 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.107+0000 7fdf2853d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf201955c0 0x7fdf20199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:31.111 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.107+0000 7fdf2853d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf20199e90 0x7fdf2019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:31.111 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.107+0000 7fdf2853d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdf201075a0 con 0x7fdf0c005740 2024-09-10T08:30:31.111 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.107+0000 7fdf2853d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdf20107420 con 0x7fdf201955c0 2024-09-10T08:30:31.111 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.107+0000 7fdf2853d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdf20107720 con 0x7fdf20199e90 2024-09-10T08:30:31.111 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.107+0000 7fdf26ada640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf20199e90 0x7fdf2019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:31.111 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.107+0000 7fdf262d9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c005740 0x7fdf20195080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:31.111 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.107+0000 7fdf262d9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c005740 0x7fdf20195080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.149:35634/0 (socket says 172.21.15.149:35634) 2024-09-10T08:30:31.111 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.107+0000 7fdf262d9640 1 -- 172.21.15.149:0/3715345622 learned_addr learned my addr 172.21.15.149:0/3715345622 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-09-10T08:30:31.111 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.107+0000 7fdf26ada640 1 --2- 172.21.15.149:0/3715345622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf20199e90 0x7fdf2019a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:39322/0 (socket says 172.21.15.149:39322) 2024-09-10T08:30:31.111 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.107+0000 7fdf25ad8640 1 --2- 172.21.15.149:0/3715345622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf201955c0 0x7fdf20199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:31.111 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.107+0000 7fdf26ada640 1 -- 172.21.15.149:0/3715345622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf20199e90 msgr2=0x7fdf2019a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-10T08:30:31.111 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.107+0000 7fdf26ada640 1 -- 172.21.15.149:0/3715345622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf20199e90 msgr2=0x7fdf2019a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T08:30:31.111 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.107+0000 7fdf26ada640 1 --2- 172.21.15.149:0/3715345622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf20199e90 0x7fdf2019a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T08:30:31.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.108+0000 7fdf25ad8640 1 -- 172.21.15.149:0/3715345622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf201955c0 msgr2=0x7fdf20199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-10T08:30:31.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.108+0000 7fdf25ad8640 1 -- 172.21.15.149:0/3715345622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf201955c0 msgr2=0x7fdf20199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T08:30:31.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.108+0000 7fdf25ad8640 1 --2- 172.21.15.149:0/3715345622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf201955c0 0x7fdf20199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T08:30:31.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.108+0000 7fdf26ada640 1 --2- 172.21.15.149:0/3715345622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf20199e90 0x7fdf2019a340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T08:30:31.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.108+0000 7fdf25ad8640 1 --2- 172.21.15.149:0/3715345622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf201955c0 0x7fdf20199950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T08:30:31.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.108+0000 7fdf262d9640 1 -- 172.21.15.149:0/3715345622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf20199e90 msgr2=0x7fdf2019a340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:30:31.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.108+0000 7fdf262d9640 1 --2- 172.21.15.149:0/3715345622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf20199e90 0x7fdf2019a340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:31.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.108+0000 7fdf262d9640 1 -- 172.21.15.149:0/3715345622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf201955c0 msgr2=0x7fdf20199950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:30:31.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.108+0000 7fdf262d9640 1 --2- 172.21.15.149:0/3715345622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf201955c0 0x7fdf20199950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:31.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.108+0000 7fdf262d9640 1 -- 172.21.15.149:0/3715345622 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf201a1c00 con 0x7fdf0c005740 2024-09-10T08:30:31.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.108+0000 7fdf262d9640 1 --2- 172.21.15.149:0/3715345622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c005740 0x7fdf20195080 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c005fd0 tx=0x7fdf1c0048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:31.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.108+0000 7fdf177fe640 1 -- 172.21.15.149:0/3715345622 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c012450 con 0x7fdf0c005740 2024-09-10T08:30:31.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.108+0000 7fdf177fe640 1 -- 172.21.15.149:0/3715345622 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf1c042070 con 0x7fdf0c005740 2024-09-10T08:30:31.113 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.108+0000 7fdf2853d640 1 -- 172.21.15.149:0/3715345622 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf201a1dd0 con 0x7fdf0c005740 2024-09-10T08:30:31.113 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.108+0000 7fdf177fe640 1 -- 172.21.15.149:0/3715345622 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c03d770 con 0x7fdf0c005740 2024-09-10T08:30:31.113 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.108+0000 7fdf2853d640 1 -- 172.21.15.149:0/3715345622 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf201a2300 con 0x7fdf0c005740 2024-09-10T08:30:31.113 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.109+0000 7fdf177fe640 1 -- 172.21.15.149:0/3715345622 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fdf1c03d990 con 0x7fdf0c005740 2024-09-10T08:30:31.113 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.109+0000 7fdf2853d640 1 -- 172.21.15.149:0/3715345622 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf20108690 con 0x7fdf0c005740 2024-09-10T08:30:31.113 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.110+0000 7fdf177fe640 1 --2- 172.21.15.149:0/3715345622 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdf040394f0 0x7fdf0403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:31.113 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.110+0000 7fdf177fe640 1 -- 172.21.15.149:0/3715345622 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdf1c06e620 con 0x7fdf0c005740 2024-09-10T08:30:31.114 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.110+0000 7fdf25ad8640 1 --2- 172.21.15.149:0/3715345622 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdf040394f0 0x7fdf0403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:31.114 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.110+0000 7fdf25ad8640 1 --2- 172.21.15.149:0/3715345622 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdf040394f0 0x7fdf0403b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdf10006fd0 tx=0x7fdf10008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:31.116 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.115+0000 7fdf177fe640 1 -- 172.21.15.149:0/3715345622 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf1c03b340 con 0x7fdf0c005740 2024-09-10T08:30:31.363 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.362+0000 7fdf2853d640 1 -- 172.21.15.149:0/3715345622 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fdf2002d7c0 con 0x7fdf0c005740 2024-09-10T08:30:31.364 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: mon.c calling monitor election 2024-09-10T08:30:31.364 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:30:31.364 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: mon.a calling monitor election 2024-09-10T08:30:31.364 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:31.364 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:30:31.364 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:31.364 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:31.364 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:31.364 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:31.364 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:31.364 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:31.365 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:31.365 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T08:30:31.365 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: monmap epoch 3 2024-09-10T08:30:31.365 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:30:31.365 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: last_changed 2024-09-10T08:30:26.006321+0000 2024-09-10T08:30:31.365 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: created 2024-09-10T08:28:45.922310+0000 2024-09-10T08:30:31.365 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: min_mon_release 19 (squid) 2024-09-10T08:30:31.365 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: election_strategy: 1 2024-09-10T08:30:31.365 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-10T08:30:31.365 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-09-10T08:30:31.365 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-09-10T08:30:31.365 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: fsmap 2024-09-10T08:30:31.366 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:30:31.366 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: mgrmap e13: x(active, since 49s) 2024-09-10T08:30:31.366 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: overall HEALTH_OK 2024-09-10T08:30:31.366 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:31.366 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:31.366 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:31.366 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:31 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:31.366 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.363+0000 7fdf177fe640 1 -- 172.21.15.149:0/3715345622 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7fdf1c04c200 con 0x7fdf0c005740 2024-09-10T08:30:31.367 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:30:31.367 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":3,"fsid":"91f747bc-6f4e-11ef-bcea-c7b262605968","modified":"2024-09-10T08:30:26.006321Z","created":"2024-09-10T08:28:45.922310Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3300","nonce":0},{"type":"v1","addr":"172.21.15.111:6789","nonce":0}]},"addr":"172.21.15.111:6789/0","public_addr":"172.21.15.111:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T08:30:31.367 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 3 2024-09-10T08:30:31.367 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.365+0000 7fdf2853d640 1 -- 172.21.15.149:0/3715345622 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdf040394f0 msgr2=0x7fdf0403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:31.368 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.365+0000 7fdf2853d640 1 --2- 172.21.15.149:0/3715345622 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdf040394f0 0x7fdf0403b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdf10006fd0 tx=0x7fdf10008040 comp rx=0 tx=0).stop 2024-09-10T08:30:31.368 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.365+0000 7fdf2853d640 1 -- 172.21.15.149:0/3715345622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c005740 msgr2=0x7fdf20195080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:31.368 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.365+0000 7fdf2853d640 1 --2- 172.21.15.149:0/3715345622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c005740 0x7fdf20195080 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c005fd0 tx=0x7fdf1c0048d0 comp rx=0 tx=0).stop 2024-09-10T08:30:31.368 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.365+0000 7fdf2853d640 1 -- 172.21.15.149:0/3715345622 shutdown_connections 2024-09-10T08:30:31.368 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.365+0000 7fdf2853d640 1 --2- 172.21.15.149:0/3715345622 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdf040394f0 0x7fdf0403b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:31.368 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.365+0000 7fdf2853d640 1 --2- 172.21.15.149:0/3715345622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf20199e90 0x7fdf2019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:31.368 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.365+0000 7fdf2853d640 1 --2- 172.21.15.149:0/3715345622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf201955c0 0x7fdf20199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:31.368 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.365+0000 7fdf2853d640 1 --2- 172.21.15.149:0/3715345622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf0c005740 0x7fdf20195080 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:31.368 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.365+0000 7fdf2853d640 1 -- 172.21.15.149:0/3715345622 >> 172.21.15.149:0/3715345622 conn(0x7fdf200fc220 msgr2=0x7fdf200fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:31.368 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.365+0000 7fdf2853d640 1 -- 172.21.15.149:0/3715345622 shutdown_connections 2024-09-10T08:30:31.368 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:31.365+0000 7fdf2853d640 1 -- 172.21.15.149:0/3715345622 wait complete. 2024-09-10T08:30:31.532 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: mon.c calling monitor election 2024-09-10T08:30:31.532 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:30:31.532 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: mon.a calling monitor election 2024-09-10T08:30:31.532 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:31.532 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:30:31.532 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:31.532 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:31.532 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:31.532 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:31.532 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:31.533 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:31.533 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:31.533 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T08:30:31.533 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: monmap epoch 3 2024-09-10T08:30:31.533 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:30:31.533 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: last_changed 2024-09-10T08:30:26.006321+0000 2024-09-10T08:30:31.533 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: created 2024-09-10T08:28:45.922310+0000 2024-09-10T08:30:31.533 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: min_mon_release 19 (squid) 2024-09-10T08:30:31.533 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: election_strategy: 1 2024-09-10T08:30:31.533 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-10T08:30:31.533 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-09-10T08:30:31.533 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-09-10T08:30:31.533 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: fsmap 2024-09-10T08:30:31.533 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:30:31.534 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: mgrmap e13: x(active, since 49s) 2024-09-10T08:30:31.534 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: overall HEALTH_OK 2024-09-10T08:30:31.534 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:31.534 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:31.534 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:31.534 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:31 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:31.964 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T08:30:31.964 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T08:30:32.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:32 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:30:32.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:32 smithi149 ceph-mon[36167]: from='client.? 172.21.15.149:0/3715345622' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T08:30:32.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:32 smithi149 ceph-mon[36167]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:32.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:32 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:32.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:32 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:32.302 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:32 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:30:32.302 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:32 smithi089 ceph-mon[30757]: from='client.? 172.21.15.149:0/3715345622' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T08:30:32.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:32 smithi089 ceph-mon[30757]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:32.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:32 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:32.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:32 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:32.361 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:30:32.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.871+0000 7f4224a00640 1 -- 172.21.15.89:0/1402150536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4220107660 msgr2=0x7f4220107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:32.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.871+0000 7f4224a00640 1 --2- 172.21.15.89:0/1402150536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4220107660 0x7f4220107a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4210009880 tx=0x7f421002a5d0 comp rx=0 tx=0).stop 2024-09-10T08:30:32.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.871+0000 7f4224a00640 1 -- 172.21.15.89:0/1402150536 shutdown_connections 2024-09-10T08:30:32.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.871+0000 7f4224a00640 1 --2- 172.21.15.89:0/1402150536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4220107660 0x7f4220107a40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:32.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.871+0000 7f4224a00640 1 -- 172.21.15.89:0/1402150536 >> 172.21.15.89:0/1402150536 conn(0x7f42200fd290 msgr2=0x7f42200ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:32.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.871+0000 7f4224a00640 1 -- 172.21.15.89:0/1402150536 shutdown_connections 2024-09-10T08:30:32.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.871+0000 7f4224a00640 1 -- 172.21.15.89:0/1402150536 wait complete. 2024-09-10T08:30:32.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.872+0000 7f4224a00640 1 Processor -- start 2024-09-10T08:30:32.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.872+0000 7f4224a00640 1 -- start start 2024-09-10T08:30:32.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.873+0000 7f4224a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4220107660 0x7f42200744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:32.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.873+0000 7f4224a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4220074a30 0x7f4220074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:32.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.873+0000 7f4224a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42201a3510 0x7f42201a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:32.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.873+0000 7f4224a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f422010bc90 con 0x7f4220107660 2024-09-10T08:30:32.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.873+0000 7f4224a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f422010bb10 con 0x7f42201a3510 2024-09-10T08:30:32.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.873+0000 7f4224a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f422010be10 con 0x7f4220074a30 2024-09-10T08:30:32.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.873+0000 7f421de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4220107660 0x7f42200744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:32.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.873+0000 7f421d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4220074a30 0x7f4220074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:32.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.873+0000 7f421de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4220107660 0x7f42200744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39718/0 (socket says 172.21.15.89:39718) 2024-09-10T08:30:32.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.873+0000 7f421de00640 1 -- 172.21.15.89:0/2691661535 learned_addr learned my addr 172.21.15.89:0/2691661535 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:30:32.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.873+0000 7f421e800640 1 --2- 172.21.15.89:0/2691661535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42201a3510 0x7f42201a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:32.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.874+0000 7f421de00640 1 -- 172.21.15.89:0/2691661535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4220074a30 msgr2=0x7f4220074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:32.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.874+0000 7f421de00640 1 --2- 172.21.15.89:0/2691661535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4220074a30 0x7f4220074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:32.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.874+0000 7f421de00640 1 -- 172.21.15.89:0/2691661535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42201a3510 msgr2=0x7f42201a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:32.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.874+0000 7f421de00640 1 --2- 172.21.15.89:0/2691661535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42201a3510 0x7f42201a5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:32.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.874+0000 7f421de00640 1 -- 172.21.15.89:0/2691661535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42201a5fd0 con 0x7f4220107660 2024-09-10T08:30:32.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.874+0000 7f421d400640 1 --2- 172.21.15.89:0/2691661535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4220074a30 0x7f4220074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:30:32.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.874+0000 7f421de00640 1 --2- 172.21.15.89:0/2691661535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4220107660 0x7f42200744f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4210006010 tx=0x7f4210003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:32.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.874+0000 7f4216a00640 1 -- 172.21.15.89:0/2691661535 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421003f070 con 0x7f4220107660 2024-09-10T08:30:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.874+0000 7f4216a00640 1 -- 172.21.15.89:0/2691661535 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f421003b460 con 0x7f4220107660 2024-09-10T08:30:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.874+0000 7f4224a00640 1 -- 172.21.15.89:0/2691661535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42201a6260 con 0x7f4220107660 2024-09-10T08:30:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.874+0000 7f4216a00640 1 -- 172.21.15.89:0/2691661535 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210035030 con 0x7f4220107660 2024-09-10T08:30:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.874+0000 7f4224a00640 1 -- 172.21.15.89:0/2691661535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42201a6790 con 0x7f4220107660 2024-09-10T08:30:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.876+0000 7f4216a00640 1 -- 172.21.15.89:0/2691661535 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4210042070 con 0x7f4220107660 2024-09-10T08:30:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.876+0000 7f4224a00640 1 -- 172.21.15.89:0/2691661535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4220107ac0 con 0x7f4220107660 2024-09-10T08:30:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.876+0000 7f4216a00640 1 --2- 172.21.15.89:0/2691661535 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f42040394f0 0x7f420403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.876+0000 7f4216a00640 1 -- 172.21.15.89:0/2691661535 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f421006cce0 con 0x7f4220107660 2024-09-10T08:30:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.877+0000 7f421d400640 1 --2- 172.21.15.89:0/2691661535 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f42040394f0 0x7f420403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:32.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.877+0000 7f421d400640 1 --2- 172.21.15.89:0/2691661535 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f42040394f0 0x7f420403b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f420c009e20 tx=0x7f420c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:32.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:32.881+0000 7f4216a00640 1 -- 172.21.15.89:0/2691661535 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4210039510 con 0x7f4220107660 2024-09-10T08:30:33.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:33.098+0000 7f4224a00640 1 -- 172.21.15.89:0/2691661535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f42201a69f0 con 0x7f4220107660 2024-09-10T08:30:33.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:33.098+0000 7f4216a00640 1 -- 172.21.15.89:0/2691661535 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f4210039770 con 0x7f4220107660 2024-09-10T08:30:33.098 INFO:teuthology.orchestra.run.smithi089.stdout:# minimal ceph.conf for 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:30:33.098 INFO:teuthology.orchestra.run.smithi089.stdout:[global] 2024-09-10T08:30:33.098 INFO:teuthology.orchestra.run.smithi089.stdout: fsid = 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:30:33.098 INFO:teuthology.orchestra.run.smithi089.stdout: mon_host = [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] 2024-09-10T08:30:33.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:33.102+0000 7f4224a00640 1 -- 172.21.15.89:0/2691661535 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f42040394f0 msgr2=0x7f420403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:33.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:33.102+0000 7f4224a00640 1 --2- 172.21.15.89:0/2691661535 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f42040394f0 0x7f420403b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f420c009e20 tx=0x7f420c009190 comp rx=0 tx=0).stop 2024-09-10T08:30:33.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:33.102+0000 7f4224a00640 1 -- 172.21.15.89:0/2691661535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4220107660 msgr2=0x7f42200744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:33.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:33.102+0000 7f4224a00640 1 --2- 172.21.15.89:0/2691661535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4220107660 0x7f42200744f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4210006010 tx=0x7f4210003c00 comp rx=0 tx=0).stop 2024-09-10T08:30:33.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:33.102+0000 7f4224a00640 1 -- 172.21.15.89:0/2691661535 shutdown_connections 2024-09-10T08:30:33.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:33.102+0000 7f4224a00640 1 --2- 172.21.15.89:0/2691661535 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f42040394f0 0x7f420403b9b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:33.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:33.102+0000 7f4224a00640 1 --2- 172.21.15.89:0/2691661535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42201a3510 0x7f42201a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:33.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:33.102+0000 7f4224a00640 1 --2- 172.21.15.89:0/2691661535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4220074a30 0x7f4220074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:33.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:33.102+0000 7f4224a00640 1 --2- 172.21.15.89:0/2691661535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4220107660 0x7f42200744f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:33.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:33.102+0000 7f4224a00640 1 -- 172.21.15.89:0/2691661535 >> 172.21.15.89:0/2691661535 conn(0x7f42200fd290 msgr2=0x7f42200fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:33.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:33.102+0000 7f4224a00640 1 -- 172.21.15.89:0/2691661535 shutdown_connections 2024-09-10T08:30:33.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:33.102+0000 7f4224a00640 1 -- 172.21.15.89:0/2691661535 wait complete. 2024-09-10T08:30:33.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: mon.c calling monitor election 2024-09-10T08:30:33.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:30:33.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: mon.a calling monitor election 2024-09-10T08:30:33.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:33.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:30:33.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:33.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:33.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:33.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:33.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:33.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:33.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:33.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T08:30:33.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: monmap epoch 3 2024-09-10T08:30:33.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:30:33.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: last_changed 2024-09-10T08:30:26.006321+0000 2024-09-10T08:30:33.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: created 2024-09-10T08:28:45.922310+0000 2024-09-10T08:30:33.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: min_mon_release 19 (squid) 2024-09-10T08:30:33.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: election_strategy: 1 2024-09-10T08:30:33.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-10T08:30:33.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-09-10T08:30:33.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-09-10T08:30:33.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: fsmap 2024-09-10T08:30:33.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:30:33.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: mgrmap e13: x(active, since 49s) 2024-09-10T08:30:33.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: overall HEALTH_OK 2024-09-10T08:30:33.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:33.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:33.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:33.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:33.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:30:33.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='client.? 172.21.15.149:0/3715345622' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T08:30:33.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:33.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:33.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:33.799 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T08:30:33.799 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:30:33.800 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T08:30:33.858 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:30:33.858 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T08:30:33.957 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-10T08:30:33.957 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T08:30:33.987 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-10T08:30:33.987 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T08:30:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: mon.b calling monitor election 2024-09-10T08:30:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-10T08:30:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: Updating smithi111:/etc/ceph/ceph.conf 2024-09-10T08:30:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: Updating smithi149:/etc/ceph/ceph.conf 2024-09-10T08:30:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: Updating smithi149:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:30:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: Updating smithi111:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:30:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: mon.c calling monitor election 2024-09-10T08:30:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: mon.b calling monitor election 2024-09-10T08:30:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: mon.a calling monitor election 2024-09-10T08:30:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T08:30:34.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: monmap epoch 3 2024-09-10T08:30:34.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:30:34.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: last_changed 2024-09-10T08:30:26.006321+0000 2024-09-10T08:30:34.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: created 2024-09-10T08:28:45.922310+0000 2024-09-10T08:30:34.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: min_mon_release 19 (squid) 2024-09-10T08:30:34.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: election_strategy: 1 2024-09-10T08:30:34.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-10T08:30:34.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-09-10T08:30:34.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-09-10T08:30:34.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: fsmap 2024-09-10T08:30:34.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:30:34.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: mgrmap e13: x(active, since 51s) 2024-09-10T08:30:34.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: overall HEALTH_OK 2024-09-10T08:30:34.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2691661535' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:34.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: Updating smithi089:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:30:34.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:34.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: mon.b calling monitor election 2024-09-10T08:30:34.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-10T08:30:34.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: Updating smithi111:/etc/ceph/ceph.conf 2024-09-10T08:30:34.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: Updating smithi149:/etc/ceph/ceph.conf 2024-09-10T08:30:34.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: Updating smithi149:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:30:34.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: Updating smithi111:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:30:34.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: mon.c calling monitor election 2024-09-10T08:30:34.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: mon.b calling monitor election 2024-09-10T08:30:34.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: mon.a calling monitor election 2024-09-10T08:30:34.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T08:30:34.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: monmap epoch 3 2024-09-10T08:30:34.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:30:34.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: last_changed 2024-09-10T08:30:26.006321+0000 2024-09-10T08:30:34.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: created 2024-09-10T08:28:45.922310+0000 2024-09-10T08:30:34.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: min_mon_release 19 (squid) 2024-09-10T08:30:34.055 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: election_strategy: 1 2024-09-10T08:30:34.055 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-10T08:30:34.055 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-09-10T08:30:34.055 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-09-10T08:30:34.055 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: fsmap 2024-09-10T08:30:34.055 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:30:34.055 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: mgrmap e13: x(active, since 51s) 2024-09-10T08:30:34.055 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: overall HEALTH_OK 2024-09-10T08:30:34.055 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2691661535' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:34.055 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.055 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.055 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.056 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.056 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: Updating smithi089:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:30:34.056 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:34.056 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.056 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.056 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:33 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.057 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T08:30:34.057 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T08:30:34.081 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: mon.b calling monitor election 2024-09-10T08:30:34.081 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-10T08:30:34.081 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: Updating smithi111:/etc/ceph/ceph.conf 2024-09-10T08:30:34.081 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: Updating smithi149:/etc/ceph/ceph.conf 2024-09-10T08:30:34.081 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: Updating smithi149:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:30:34.081 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: Updating smithi111:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:30:34.081 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: mon.c calling monitor election 2024-09-10T08:30:34.081 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: mon.b calling monitor election 2024-09-10T08:30:34.081 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: mon.a calling monitor election 2024-09-10T08:30:34.081 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T08:30:34.081 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: monmap epoch 3 2024-09-10T08:30:34.081 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:30:34.082 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: last_changed 2024-09-10T08:30:26.006321+0000 2024-09-10T08:30:34.082 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: created 2024-09-10T08:28:45.922310+0000 2024-09-10T08:30:34.082 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: min_mon_release 19 (squid) 2024-09-10T08:30:34.082 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: election_strategy: 1 2024-09-10T08:30:34.082 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: 0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-09-10T08:30:34.082 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-09-10T08:30:34.082 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-09-10T08:30:34.082 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: fsmap 2024-09-10T08:30:34.082 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:30:34.082 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: mgrmap e13: x(active, since 51s) 2024-09-10T08:30:34.082 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: overall HEALTH_OK 2024-09-10T08:30:34.082 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2691661535' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:34.082 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.082 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.083 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.083 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.083 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: Updating smithi089:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:30:34.083 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:34.083 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.083 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.083 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:33 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:34.085 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T08:30:34.085 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T08:30:34.151 INFO:tasks.cephadm:Adding mgr.x on smithi089 2024-09-10T08:30:34.152 INFO:tasks.cephadm:Adding mgr.y on smithi111 2024-09-10T08:30:34.152 INFO:tasks.cephadm:Adding mgr.z on smithi149 2024-09-10T08:30:34.152 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi089=x;smithi111=y;smithi149=z' 2024-09-10T08:30:34.430 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:30:34.861 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.858+0000 7f35d6a4f640 1 -- 172.21.15.149:0/2198676810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35d01005f0 msgr2=0x7f35d01009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:34.861 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.858+0000 7f35d6a4f640 1 --2- 172.21.15.149:0/2198676810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35d01005f0 0x7f35d01009f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009880 tx=0x7f35c002a5d0 comp rx=0 tx=0).stop 2024-09-10T08:30:34.861 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.858+0000 7f35d6a4f640 1 -- 172.21.15.149:0/2198676810 shutdown_connections 2024-09-10T08:30:34.861 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.858+0000 7f35d6a4f640 1 --2- 172.21.15.149:0/2198676810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35d01005f0 0x7f35d01009f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:34.861 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.858+0000 7f35d6a4f640 1 -- 172.21.15.149:0/2198676810 >> 172.21.15.149:0/2198676810 conn(0x7f35d00fbda0 msgr2=0x7f35d00fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:34.861 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.858+0000 7f35d6a4f640 1 -- 172.21.15.149:0/2198676810 shutdown_connections 2024-09-10T08:30:34.861 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.858+0000 7f35d6a4f640 1 -- 172.21.15.149:0/2198676810 wait complete. 2024-09-10T08:30:34.861 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.859+0000 7f35d6a4f640 1 Processor -- start 2024-09-10T08:30:34.861 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.859+0000 7f35d6a4f640 1 -- start start 2024-09-10T08:30:34.862 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.859+0000 7f35d6a4f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35d01005f0 0x7f35d0197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:34.862 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.859+0000 7f35d6a4f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35d0197560 0x7f35d019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:34.862 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.859+0000 7f35d6a4f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35d01979e0 0x7f35d019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:34.862 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.859+0000 7f35d6a4f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f35d0106700 con 0x7f35d0197560 2024-09-10T08:30:34.862 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.859+0000 7f35d6a4f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f35d0106580 con 0x7f35d01979e0 2024-09-10T08:30:34.862 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.859+0000 7f35d6a4f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f35d0106880 con 0x7f35d01005f0 2024-09-10T08:30:34.862 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.859+0000 7f35cffff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35d01005f0 0x7f35d0197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:34.862 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.859+0000 7f35cffff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35d01005f0 0x7f35d0197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:39336/0 (socket says 172.21.15.149:39336) 2024-09-10T08:30:34.862 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.859+0000 7f35cffff640 1 -- 172.21.15.149:0/2434302863 learned_addr learned my addr 172.21.15.149:0/2434302863 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-09-10T08:30:34.862 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.859+0000 7f35cf7fe640 1 --2- 172.21.15.149:0/2434302863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35d0197560 0x7f35d019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:34.862 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.859+0000 7f35d4fec640 1 --2- 172.21.15.149:0/2434302863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35d01979e0 0x7f35d019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:34.862 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.860+0000 7f35cffff640 1 -- 172.21.15.149:0/2434302863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35d01979e0 msgr2=0x7f35d019eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:34.862 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.860+0000 7f35cffff640 1 --2- 172.21.15.149:0/2434302863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35d01979e0 0x7f35d019eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:34.862 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.860+0000 7f35cffff640 1 -- 172.21.15.149:0/2434302863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35d0197560 msgr2=0x7f35d019c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:34.863 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.860+0000 7f35cffff640 1 --2- 172.21.15.149:0/2434302863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35d0197560 0x7f35d019c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:34.863 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.860+0000 7f35cffff640 1 -- 172.21.15.149:0/2434302863 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d019f3b0 con 0x7f35d01005f0 2024-09-10T08:30:34.863 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.860+0000 7f35cffff640 1 --2- 172.21.15.149:0/2434302863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35d01005f0 0x7f35d0197020 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f35c0005ab0 tx=0x7f35c0032d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:34.863 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.860+0000 7f35cd7fa640 1 -- 172.21.15.149:0/2434302863 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c003f030 con 0x7f35d01005f0 2024-09-10T08:30:34.863 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.860+0000 7f35cd7fa640 1 -- 172.21.15.149:0/2434302863 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35c0003ce0 con 0x7f35d01005f0 2024-09-10T08:30:34.863 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.860+0000 7f35d6a4f640 1 -- 172.21.15.149:0/2434302863 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d019f640 con 0x7f35d01005f0 2024-09-10T08:30:34.863 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.860+0000 7f35cd7fa640 1 -- 172.21.15.149:0/2434302863 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c0035070 con 0x7f35d01005f0 2024-09-10T08:30:34.863 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.860+0000 7f35d6a4f640 1 -- 172.21.15.149:0/2434302863 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d019fba0 con 0x7f35d01005f0 2024-09-10T08:30:34.864 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.861+0000 7f35cd7fa640 1 -- 172.21.15.149:0/2434302863 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f35c0042030 con 0x7f35d01005f0 2024-09-10T08:30:34.864 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.861+0000 7f35cd7fa640 1 --2- 172.21.15.149:0/2434302863 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f35b80394f0 0x7f35b803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:34.864 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.861+0000 7f35d6a4f640 1 -- 172.21.15.149:0/2434302863 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d0190800 con 0x7f35d01005f0 2024-09-10T08:30:34.864 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.861+0000 7f35cd7fa640 1 -- 172.21.15.149:0/2434302863 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f35c006ca20 con 0x7f35d01005f0 2024-09-10T08:30:34.864 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.861+0000 7f35cf7fe640 1 --2- 172.21.15.149:0/2434302863 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f35b80394f0 0x7f35b803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:34.865 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.862+0000 7f35cf7fe640 1 --2- 172.21.15.149:0/2434302863 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f35b80394f0 0x7f35b803b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f35d01984d0 tx=0x7f35c4009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:34.868 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:34.866+0000 7f35cd7fa640 1 -- 172.21.15.149:0/2434302863 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35c002c140 con 0x7f35d01005f0 2024-09-10T08:30:35.046 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:35.045+0000 7f35d6a4f640 1 -- 172.21.15.149:0/2434302863 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi089=x;smithi111=y;smithi149=z", "target": ["mon-mgr", ""]}) -- 0x7f35d00626c0 con 0x7f35b80394f0 2024-09-10T08:30:35.051 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:35.051+0000 7f35cd7fa640 1 -- 172.21.15.149:0/2434302863 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f35d00626c0 con 0x7f35b80394f0 2024-09-10T08:30:35.051 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mgr update... 2024-09-10T08:30:35.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:35.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:35.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:35.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:35.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:34 smithi089 ceph-mon[30757]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T08:30:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:30:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T08:30:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:34 smithi089 ceph-mon[30757]: Reconfiguring daemon mon.a on smithi089 2024-09-10T08:30:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:35.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:35.053+0000 7f35d6a4f640 1 -- 172.21.15.149:0/2434302863 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f35b80394f0 msgr2=0x7f35b803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:35.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:35.053+0000 7f35d6a4f640 1 --2- 172.21.15.149:0/2434302863 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f35b80394f0 0x7f35b803b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f35d01984d0 tx=0x7f35c4009190 comp rx=0 tx=0).stop 2024-09-10T08:30:35.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:35.053+0000 7f35d6a4f640 1 -- 172.21.15.149:0/2434302863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35d01005f0 msgr2=0x7f35d0197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:35.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:35.053+0000 7f35d6a4f640 1 --2- 172.21.15.149:0/2434302863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35d01005f0 0x7f35d0197020 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f35c0005ab0 tx=0x7f35c0032d00 comp rx=0 tx=0).stop 2024-09-10T08:30:35.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:35.053+0000 7f35d6a4f640 1 -- 172.21.15.149:0/2434302863 shutdown_connections 2024-09-10T08:30:35.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:35.053+0000 7f35d6a4f640 1 --2- 172.21.15.149:0/2434302863 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f35b80394f0 0x7f35b803b9b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:35.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:35.053+0000 7f35d6a4f640 1 --2- 172.21.15.149:0/2434302863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35d01979e0 0x7f35d019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:35.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:35.053+0000 7f35d6a4f640 1 --2- 172.21.15.149:0/2434302863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35d0197560 0x7f35d019c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:35.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:35.053+0000 7f35d6a4f640 1 --2- 172.21.15.149:0/2434302863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35d01005f0 0x7f35d0197020 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:35.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:35.053+0000 7f35d6a4f640 1 -- 172.21.15.149:0/2434302863 >> 172.21.15.149:0/2434302863 conn(0x7f35d00fbda0 msgr2=0x7f35d006a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:35.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:35.053+0000 7f35d6a4f640 1 -- 172.21.15.149:0/2434302863 shutdown_connections 2024-09-10T08:30:35.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:30:35.053+0000 7f35d6a4f640 1 -- 172.21.15.149:0/2434302863 wait complete. 2024-09-10T08:30:35.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:35.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:35.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:35.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:35.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:34 smithi149 ceph-mon[36167]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T08:30:35.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:30:35.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T08:30:35.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:35.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:34 smithi149 ceph-mon[36167]: Reconfiguring daemon mon.a on smithi089 2024-09-10T08:30:35.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:35.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:35.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:35.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:35.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:35.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:34 smithi111 ceph-mon[35220]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T08:30:35.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:30:35.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T08:30:35.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:35.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:34 smithi111 ceph-mon[35220]: Reconfiguring daemon mon.a on smithi089 2024-09-10T08:30:35.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:30:35.653 DEBUG:teuthology.orchestra.run.smithi111:mgr.y> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T08:30:35.656 DEBUG:teuthology.orchestra.run.smithi149:mgr.z> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mgr.z.service 2024-09-10T08:30:35.658 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T08:30:35.659 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:30:35.659 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T08:30:35.694 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T08:30:35.694 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_1 2024-09-10T08:30:35.770 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T08:30:35.770 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:30:35.770 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-09-10T08:30:35.770 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:30:35.770 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:30:35.770 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-10 08:29:54.439176248 +0000 2024-09-10T08:30:35.770 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-10 08:25:24.819587215 +0000 2024-09-10T08:30:35.770 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-10 08:25:24.819587215 +0000 2024-09-10T08:30:35.770 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-10 08:25:24.819587215 +0000 2024-09-10T08:30:35.771 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T08:30:35.865 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-10T08:30:35.865 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-10T08:30:35.865 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000380485 s, 1.3 MB/s 2024-09-10T08:30:35.868 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T08:30:35.906 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_2 2024-09-10T08:30:35.983 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T08:30:35.983 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:30:35.983 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-09-10T08:30:35.983 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:30:35.984 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:30:35.984 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-10 08:29:54.486175113 +0000 2024-09-10T08:30:35.984 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-10 08:25:25.288576304 +0000 2024-09-10T08:30:35.984 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-10 08:25:25.288576304 +0000 2024-09-10T08:30:35.984 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-10 08:25:25.288576304 +0000 2024-09-10T08:30:35.984 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T08:30:36.078 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-10T08:30:36.078 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-10T08:30:36.078 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000272142 s, 1.9 MB/s 2024-09-10T08:30:36.081 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T08:30:36.118 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_3 2024-09-10T08:30:36.185 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:36 smithi089 ceph-mon[30757]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi089=x;smithi111=y;smithi149=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:30:36.185 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:36 smithi089 ceph-mon[30757]: Saving service mgr spec with placement smithi089=x;smithi111=y;smithi149=z;count:3 2024-09-10T08:30:36.186 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:36.186 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:36 smithi089 ceph-mon[30757]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:36.186 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:36.186 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:36.186 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:36 smithi089 ceph-mon[30757]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T08:30:36.186 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T08:30:36.186 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T08:30:36.186 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:36.186 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:36 smithi089 ceph-mon[30757]: Reconfiguring daemon mgr.x on smithi089 2024-09-10T08:30:36.195 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T08:30:36.195 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:30:36.195 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-09-10T08:30:36.195 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:30:36.195 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:30:36.195 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-10 08:29:54.532174003 +0000 2024-09-10T08:30:36.195 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-10 08:25:25.767565161 +0000 2024-09-10T08:30:36.196 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-10 08:25:25.767565161 +0000 2024-09-10T08:30:36.196 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-10 08:25:25.767565161 +0000 2024-09-10T08:30:36.196 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T08:30:36.289 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-10T08:30:36.289 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-10T08:30:36.289 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000278711 s, 1.8 MB/s 2024-09-10T08:30:36.291 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T08:30:36.328 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_4 2024-09-10T08:30:36.407 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T08:30:36.407 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:30:36.407 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-09-10T08:30:36.407 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:30:36.407 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:30:36.407 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-10 08:29:54.577172916 +0000 2024-09-10T08:30:36.407 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-10 08:25:26.289553017 +0000 2024-09-10T08:30:36.407 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-10 08:25:26.289553017 +0000 2024-09-10T08:30:36.407 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-10 08:25:26.289553017 +0000 2024-09-10T08:30:36.408 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T08:30:36.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:36 smithi149 ceph-mon[36167]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi089=x;smithi111=y;smithi149=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:30:36.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:36 smithi149 ceph-mon[36167]: Saving service mgr spec with placement smithi089=x;smithi111=y;smithi149=z;count:3 2024-09-10T08:30:36.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:36.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:36 smithi149 ceph-mon[36167]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:36.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:36.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:36.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:36 smithi149 ceph-mon[36167]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T08:30:36.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T08:30:36.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T08:30:36.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:36.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:36 smithi149 ceph-mon[36167]: Reconfiguring daemon mgr.x on smithi089 2024-09-10T08:30:36.500 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-10T08:30:36.500 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-10T08:30:36.500 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.00024223 s, 2.1 MB/s 2024-09-10T08:30:36.504 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T08:30:36.541 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-10T08:30:36.541 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T08:30:36.542 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:36 smithi111 ceph-mon[35220]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi089=x;smithi111=y;smithi149=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:30:36.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:36 smithi111 ceph-mon[35220]: Saving service mgr spec with placement smithi089=x;smithi111=y;smithi149=z;count:3 2024-09-10T08:30:36.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:36.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:36 smithi111 ceph-mon[35220]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:36.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:36.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:36.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:36 smithi111 ceph-mon[35220]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T08:30:36.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T08:30:36.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T08:30:36.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:36.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:36 smithi111 ceph-mon[35220]: Reconfiguring daemon mgr.x on smithi089 2024-09-10T08:30:36.559 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T08:30:36.559 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_1 2024-09-10T08:30:36.615 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T08:30:36.616 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:30:36.616 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-10T08:30:36.616 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:30:36.616 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:30:36.616 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-10 08:30:25.427236796 +0000 2024-09-10T08:30:36.616 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-10 08:25:03.732648304 +0000 2024-09-10T08:30:36.616 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-10 08:25:03.732648304 +0000 2024-09-10T08:30:36.616 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-10 08:25:03.732648304 +0000 2024-09-10T08:30:36.616 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T08:30:36.681 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-10T08:30:36.681 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-10T08:30:36.681 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000195883 s, 2.6 MB/s 2024-09-10T08:30:36.682 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T08:30:36.739 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_2 2024-09-10T08:30:36.796 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T08:30:36.797 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:30:36.797 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-10T08:30:36.797 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:30:36.797 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:30:36.797 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-10 08:30:25.428236777 +0000 2024-09-10T08:30:36.797 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-10 08:25:04.054642347 +0000 2024-09-10T08:30:36.797 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-10 08:25:04.054642347 +0000 2024-09-10T08:30:36.797 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-10 08:25:04.054642347 +0000 2024-09-10T08:30:36.797 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T08:30:36.860 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-10T08:30:36.860 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-10T08:30:36.860 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000200653 s, 2.6 MB/s 2024-09-10T08:30:36.862 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T08:30:36.919 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_3 2024-09-10T08:30:36.975 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T08:30:36.975 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:30:36.975 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-10T08:30:36.975 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:30:36.975 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:30:36.975 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-10 08:30:25.428236777 +0000 2024-09-10T08:30:36.975 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-10 08:25:04.370636500 +0000 2024-09-10T08:30:36.975 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-10 08:25:04.370636500 +0000 2024-09-10T08:30:36.975 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-10 08:25:04.370636500 +0000 2024-09-10T08:30:36.976 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T08:30:37.038 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-10T08:30:37.039 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-10T08:30:37.039 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000227013 s, 2.3 MB/s 2024-09-10T08:30:37.040 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T08:30:37.096 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_4 2024-09-10T08:30:37.152 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T08:30:37.153 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:30:37.153 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-10T08:30:37.153 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:30:37.153 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:30:37.153 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-10 08:30:25.428236777 +0000 2024-09-10T08:30:37.153 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-10 08:25:04.695630488 +0000 2024-09-10T08:30:37.153 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-10 08:25:04.695630488 +0000 2024-09-10T08:30:37.153 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-10 08:25:04.695630488 +0000 2024-09-10T08:30:37.153 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T08:30:37.216 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-10T08:30:37.216 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-10T08:30:37.216 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000215494 s, 2.4 MB/s 2024-09-10T08:30:37.218 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T08:30:37.274 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-10T08:30:37.274 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T08:30:37.294 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T08:30:37.294 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_1 2024-09-10T08:30:37.353 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T08:30:37.353 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:30:37.354 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 803 Links: 1 2024-09-10T08:30:37.354 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:30:37.354 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:30:37.354 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-09-10 08:30:16.855111718 +0000 2024-09-10T08:30:37.354 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-09-10 08:25:03.727346976 +0000 2024-09-10T08:30:37.354 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-09-10 08:25:03.727346976 +0000 2024-09-10T08:30:37.354 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-09-10 08:25:03.727346976 +0000 2024-09-10T08:30:37.354 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T08:30:37.422 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-09-10T08:30:37.422 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-09-10T08:30:37.422 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000240346 s, 2.1 MB/s 2024-09-10T08:30:37.424 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T08:30:37.482 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_2 2024-09-10T08:30:37.540 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T08:30:37.540 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:30:37.541 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 825 Links: 1 2024-09-10T08:30:37.541 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:30:37.541 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:30:37.541 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-09-10 08:30:16.856111528 +0000 2024-09-10T08:30:37.541 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-09-10 08:25:04.050339879 +0000 2024-09-10T08:30:37.541 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-09-10 08:25:04.050339879 +0000 2024-09-10T08:30:37.541 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-09-10 08:25:04.050339879 +0000 2024-09-10T08:30:37.541 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T08:30:37.607 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-09-10T08:30:37.607 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-09-10T08:30:37.607 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.00023997 s, 2.1 MB/s 2024-09-10T08:30:37.608 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T08:30:37.666 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_3 2024-09-10T08:30:37.724 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T08:30:37.724 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:30:37.724 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 842 Links: 1 2024-09-10T08:30:37.724 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:30:37.724 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:30:37.724 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-09-10 08:30:16.856111528 +0000 2024-09-10T08:30:37.724 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-09-10 08:25:04.370332847 +0000 2024-09-10T08:30:37.724 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-09-10 08:25:04.370332847 +0000 2024-09-10T08:30:37.724 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-09-10 08:25:04.370332847 +0000 2024-09-10T08:30:37.725 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T08:30:37.790 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-09-10T08:30:37.790 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-09-10T08:30:37.790 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.00019819 s, 2.6 MB/s 2024-09-10T08:30:37.791 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T08:30:37.849 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_4 2024-09-10T08:30:37.905 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T08:30:37.905 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:30:37.905 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 861 Links: 1 2024-09-10T08:30:37.905 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:30:37.905 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:30:37.905 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-09-10 08:30:16.856111528 +0000 2024-09-10T08:30:37.906 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-09-10 08:25:04.688325859 +0000 2024-09-10T08:30:37.906 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-09-10 08:25:04.688325859 +0000 2024-09-10T08:30:37.906 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-09-10 08:25:04.688325859 +0000 2024-09-10T08:30:37.906 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T08:30:37.970 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-09-10T08:30:37.970 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-09-10T08:30:37.970 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000239435 s, 2.1 MB/s 2024-09-10T08:30:37.971 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T08:30:38.028 INFO:tasks.cephadm:Deploying osd.0 on smithi089 with /dev/vg_nvme/lv_4... 2024-09-10T08:30:38.028 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T08:30:38.387 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:30:38.527 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:38 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:38.527 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:38 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:38.528 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:38 smithi089 ceph-mon[30757]: Reconfiguring mon.b (monmap changed)... 2024-09-10T08:30:38.528 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:38 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:30:38.528 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:38 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T08:30:38.528 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:38 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:38.528 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:38 smithi089 ceph-mon[30757]: Reconfiguring daemon mon.b on smithi111 2024-09-10T08:30:38.528 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:38 smithi089 ceph-mon[30757]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:38.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:38 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:38.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:38 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:38.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:38 smithi111 ceph-mon[35220]: Reconfiguring mon.b (monmap changed)... 2024-09-10T08:30:38.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:38 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:30:38.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:38 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T08:30:38.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:38 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:38.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:38 smithi111 ceph-mon[35220]: Reconfiguring daemon mon.b on smithi111 2024-09-10T08:30:38.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:38 smithi111 ceph-mon[35220]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:38.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:38 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:38.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:38 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:38.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:38 smithi149 ceph-mon[36167]: Reconfiguring mon.b (monmap changed)... 2024-09-10T08:30:38.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:38 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:30:38.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:38 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T08:30:38.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:38 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:38.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:38 smithi149 ceph-mon[36167]: Reconfiguring daemon mon.b on smithi111 2024-09-10T08:30:38.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:38 smithi149 ceph-mon[36167]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:39.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:39 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:39.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:39 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:39.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:39 smithi089 ceph-mon[30757]: Reconfiguring mon.c (monmap changed)... 2024-09-10T08:30:39.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:39 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:30:39.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:39 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T08:30:39.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:39 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:39.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:39 smithi089 ceph-mon[30757]: Reconfiguring daemon mon.c on smithi149 2024-09-10T08:30:39.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:39 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:39.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:39 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:39.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:39 smithi149 ceph-mon[36167]: Reconfiguring mon.c (monmap changed)... 2024-09-10T08:30:39.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:39 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:30:39.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:39 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T08:30:39.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:39 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:39.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:39 smithi149 ceph-mon[36167]: Reconfiguring daemon mon.c on smithi149 2024-09-10T08:30:39.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:39 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:39.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:39 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:39.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:39 smithi111 ceph-mon[35220]: Reconfiguring mon.c (monmap changed)... 2024-09-10T08:30:39.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:39 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:30:39.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:39 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T08:30:39.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:39 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:39.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:39 smithi111 ceph-mon[35220]: Reconfiguring daemon mon.c on smithi149 2024-09-10T08:30:40.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:40 smithi149 ceph-mon[36167]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:40.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:40 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:40.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:40 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:40.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:40 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:30:40.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:40 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:40.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:40 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:30:40.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:40 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:40.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:40 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T08:30:40.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:40 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T08:30:40.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:40 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T08:30:40.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:40 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:40.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:40 smithi111 ceph-mon[35220]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:40.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:40 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:40.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:40 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:40.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:40 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:30:40.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:40 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:40.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:40 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:30:40.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:40 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:40.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:40 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T08:30:40.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:40 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T08:30:40.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:40 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T08:30:40.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:40 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:40.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:40 smithi089 ceph-mon[30757]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:40.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:40 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:40.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:40 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:40.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:40 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:30:40.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:40 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:40.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:40 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:30:40.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:40 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:40.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:40 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T08:30:40.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:40 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T08:30:40.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:40 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T08:30:40.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:40 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:41.148 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:30:41.182 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi089:vg_nvme/lv_4 2024-09-10T08:30:41.544 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:30:41.676 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:41 smithi149 ceph-mon[36167]: Deploying daemon mgr.z on smithi149 2024-09-10T08:30:41.677 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:41 smithi149 systemd[1]: Starting Ceph mgr.z for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:30:41.681 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:41 smithi089 ceph-mon[30757]: Deploying daemon mgr.z on smithi149 2024-09-10T08:30:41.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:41 smithi111 ceph-mon[35220]: Deploying daemon mgr.z on smithi149 2024-09-10T08:30:41.937 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:41 smithi149 podman[37959]: 2024-09-10 08:30:41.67666004 +0000 UTC m=+0.111635995 container create ee474161c8b215bccd2e3e1b2ec102c91dcfd6b7ce0ee4a88436cc86da5e245b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T08:30:41.937 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:41 smithi149 podman[37959]: 2024-09-10 08:30:41.58115206 +0000 UTC m=+0.016128008 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:30:41.937 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:41 smithi149 podman[37959]: 2024-09-10 08:30:41.870957024 +0000 UTC m=+0.305932971 container init ee474161c8b215bccd2e3e1b2ec102c91dcfd6b7ce0ee4a88436cc86da5e245b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:30:41.937 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:41 smithi149 podman[37959]: 2024-09-10 08:30:41.873681804 +0000 UTC m=+0.308657761 container start ee474161c8b215bccd2e3e1b2ec102c91dcfd6b7ce0ee4a88436cc86da5e245b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T08:30:41.937 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:41 smithi149 ceph-mgr[37978]: -- 172.21.15.149:0/3482631854 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5584f5d390e0 con 0x5584f5d17000 2024-09-10T08:30:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.128+0000 7f1975200640 1 -- 172.21.15.89:0/105876188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1970101c00 msgr2=0x7f1970102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:42.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.128+0000 7f1975200640 1 --2- 172.21.15.89:0/105876188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1970101c00 0x7f1970102000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1960009900 tx=0x7f196002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:30:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.128+0000 7f1975200640 1 -- 172.21.15.89:0/105876188 shutdown_connections 2024-09-10T08:30:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.128+0000 7f1975200640 1 --2- 172.21.15.89:0/105876188 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19701037c0 0x7f197010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.128+0000 7f1975200640 1 --2- 172.21.15.89:0/105876188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1970102e00 0x7f1970103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.128+0000 7f1975200640 1 --2- 172.21.15.89:0/105876188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1970101c00 0x7f1970102000 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.128+0000 7f1975200640 1 -- 172.21.15.89:0/105876188 >> 172.21.15.89:0/105876188 conn(0x7f19700fd390 msgr2=0x7f19700ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:30:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.129+0000 7f1975200640 1 -- 172.21.15.89:0/105876188 shutdown_connections 2024-09-10T08:30:42.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.129+0000 7f1975200640 1 -- 172.21.15.89:0/105876188 wait complete. 2024-09-10T08:30:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.129+0000 7f1975200640 1 Processor -- start 2024-09-10T08:30:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.130+0000 7f1975200640 1 -- start start 2024-09-10T08:30:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.130+0000 7f1975200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1970101c00 0x7f1970075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.130+0000 7f1975200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1970102e00 0x7f19700745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.130+0000 7f1975200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19701037c0 0x7f1970074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.130+0000 7f1975200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f197010d8d0 con 0x7f19701037c0 2024-09-10T08:30:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.130+0000 7f1975200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f197010d750 con 0x7f1970102e00 2024-09-10T08:30:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.130+0000 7f1975200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f197010da50 con 0x7f1970101c00 2024-09-10T08:30:42.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.130+0000 7f196de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1970101c00 0x7f1970075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.130+0000 7f196e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19701037c0 0x7f1970074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.130+0000 7f196de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1970101c00 0x7f1970075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:56108/0 (socket says 172.21.15.89:56108) 2024-09-10T08:30:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.130+0000 7f196de00640 1 -- 172.21.15.89:0/1364320565 learned_addr learned my addr 172.21.15.89:0/1364320565 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:30:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.130+0000 7f196d400640 1 --2- 172.21.15.89:0/1364320565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1970102e00 0x7f19700745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.131+0000 7f196de00640 1 -- 172.21.15.89:0/1364320565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1970102e00 msgr2=0x7f19700745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.131+0000 7f196de00640 1 --2- 172.21.15.89:0/1364320565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1970102e00 0x7f19700745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.131+0000 7f196de00640 1 -- 172.21.15.89:0/1364320565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19701037c0 msgr2=0x7f1970074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:30:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.131+0000 7f196de00640 1 --2- 172.21.15.89:0/1364320565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19701037c0 0x7f1970074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:30:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.131+0000 7f196de00640 1 -- 172.21.15.89:0/1364320565 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19700753e0 con 0x7f1970101c00 2024-09-10T08:30:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.131+0000 7f196d400640 1 --2- 172.21.15.89:0/1364320565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1970102e00 0x7f19700745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:30:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.131+0000 7f196de00640 1 --2- 172.21.15.89:0/1364320565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1970101c00 0x7f1970075f90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1960031440 tx=0x7f1960032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:42.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.131+0000 7f1966a00640 1 -- 172.21.15.89:0/1364320565 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196003f030 con 0x7f1970101c00 2024-09-10T08:30:42.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.131+0000 7f1966a00640 1 -- 172.21.15.89:0/1364320565 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1960003ce0 con 0x7f1970101c00 2024-09-10T08:30:42.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.131+0000 7f1975200640 1 -- 172.21.15.89:0/1364320565 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19701a2080 con 0x7f1970101c00 2024-09-10T08:30:42.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.131+0000 7f1966a00640 1 -- 172.21.15.89:0/1364320565 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960035070 con 0x7f1970101c00 2024-09-10T08:30:42.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.131+0000 7f1975200640 1 -- 172.21.15.89:0/1364320565 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19701a2580 con 0x7f1970101c00 2024-09-10T08:30:42.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.132+0000 7f1966a00640 1 -- 172.21.15.89:0/1364320565 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1960042030 con 0x7f1970101c00 2024-09-10T08:30:42.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.132+0000 7f1975200640 1 -- 172.21.15.89:0/1364320565 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1970109020 con 0x7f1970101c00 2024-09-10T08:30:42.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.133+0000 7f1966a00640 1 --2- 172.21.15.89:0/1364320565 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1954039290 0x7f195403b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:30:42.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.133+0000 7f1966a00640 1 -- 172.21.15.89:0/1364320565 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f196006cc90 con 0x7f1970101c00 2024-09-10T08:30:42.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.133+0000 7f196d400640 1 --2- 172.21.15.89:0/1364320565 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1954039290 0x7f195403b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:30:42.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.134+0000 7f196d400640 1 --2- 172.21.15.89:0/1364320565 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1954039290 0x7f195403b750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1970075d10 tx=0x7f195c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:30:42.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.138+0000 7f1966a00640 1 -- 172.21.15.89:0/1364320565 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1960040270 con 0x7f1970101c00 2024-09-10T08:30:42.207 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:41 smithi149 bash[37959]: ee474161c8b215bccd2e3e1b2ec102c91dcfd6b7ce0ee4a88436cc86da5e245b 2024-09-10T08:30:42.208 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:41 smithi149 systemd[1]: Started Ceph mgr.z for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:30:42.208 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:41 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:41.997+0000 7fc4312d9040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T08:30:42.208 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:42 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:42.059+0000 7fc4312d9040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T08:30:42.208 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:42 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:42.207+0000 7fc4312d9040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T08:30:42.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:42.324+0000 7f1975200640 1 -- 172.21.15.89:0/1364320565 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1970062630 con 0x7f1954039290 2024-09-10T08:30:42.470 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:42 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:42.274+0000 7fc4312d9040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T08:30:42.470 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:42 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:42.422+0000 7fc4312d9040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T08:30:42.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:42 smithi149 ceph-mon[36167]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:42.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:42.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:42.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:42.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T08:30:42.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T08:30:42.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T08:30:42.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:42.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:30:42.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:30:42.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:42.677 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:42 smithi111 ceph-mon[35220]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:42.677 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:42.677 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:42.677 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:42.677 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T08:30:42.677 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T08:30:42.677 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T08:30:42.677 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:42.677 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:30:42.677 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:30:42.677 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:42.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:42 smithi089 ceph-mon[30757]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:42.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:42.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:42.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:42.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T08:30:42.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T08:30:42.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T08:30:42.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:42.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:30:42.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:30:42.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:42.877 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:42 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:42.563+0000 7fc4312d9040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T08:30:42.877 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:42 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:42.877+0000 7fc4312d9040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T08:30:43.220 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:42 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:42.938+0000 7fc4312d9040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T08:30:43.220 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:43 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:43.121+0000 7fc4312d9040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T08:30:43.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:43 smithi089 ceph-mon[30757]: Deploying daemon mgr.y on smithi111 2024-09-10T08:30:43.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:43 smithi089 ceph-mon[30757]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:30:43.667 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:43 smithi111 ceph-mon[35220]: Deploying daemon mgr.y on smithi111 2024-09-10T08:30:43.668 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:43 smithi111 ceph-mon[35220]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:30:43.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:43 smithi149 ceph-mon[36167]: Deploying daemon mgr.y on smithi111 2024-09-10T08:30:43.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:43 smithi149 ceph-mon[36167]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:30:44.041 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:43 smithi111 systemd[1]: Starting Ceph mgr.y for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:30:44.293 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:44 smithi111 podman[36566]: 2024-09-10 08:30:44.041541966 +0000 UTC m=+0.105456734 container create 1e5793fe7519abbc91dc1e5f6d50e61424bc9dacc6ac5bf1fccdece156ed8844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0) 2024-09-10T08:30:44.294 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:44 smithi111 podman[36566]: 2024-09-10 08:30:43.951626363 +0000 UTC m=+0.015541137 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:30:44.294 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:44 smithi111 podman[36566]: 2024-09-10 08:30:44.260897521 +0000 UTC m=+0.324812281 container init 1e5793fe7519abbc91dc1e5f6d50e61424bc9dacc6ac5bf1fccdece156ed8844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T08:30:44.294 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:44 smithi111 podman[36566]: 2024-09-10 08:30:44.263747399 +0000 UTC m=+0.327662169 container start 1e5793fe7519abbc91dc1e5f6d50e61424bc9dacc6ac5bf1fccdece156ed8844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:30:44.294 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:44 smithi111 ceph-mgr[36585]: -- 172.21.15.111:0/497228287 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5604d1e670e0 con 0x5604d1e45400 2024-09-10T08:30:44.428 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:44 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:44.004+0000 7fc4312d9040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T08:30:44.428 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:44 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:44.070+0000 7fc4312d9040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T08:30:44.593 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:44 smithi089 ceph-mon[30757]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:44.593 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:44 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:44.593 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:44 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:44.593 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:44 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:44.593 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:44 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:44.604 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:44 smithi111 ceph-mon[35220]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:44.604 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:44 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:44.604 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:44 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:44.604 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:44 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:44.604 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:44 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:44.604 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:44 smithi111 bash[36566]: 1e5793fe7519abbc91dc1e5f6d50e61424bc9dacc6ac5bf1fccdece156ed8844 2024-09-10T08:30:44.604 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:44 smithi111 systemd[1]: Started Ceph mgr.y for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:30:44.605 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:44 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:44.384+0000 7fc74bed3040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T08:30:44.605 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:44 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:44.449+0000 7fc74bed3040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T08:30:44.605 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:44 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:44.603+0000 7fc74bed3040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T08:30:44.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:44 smithi149 ceph-mon[36167]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:44.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:44 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:44.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:44 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:44.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:44 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:44.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:44 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:44.988 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:44 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:44.673+0000 7fc74bed3040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T08:30:44.989 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:44 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:44.830+0000 7fc74bed3040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T08:30:44.989 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:44 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:44.988+0000 7fc74bed3040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T08:30:45.103 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:44 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:44.815+0000 7fc4312d9040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T08:30:45.103 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:44 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:44.953+0000 7fc4312d9040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T08:30:45.103 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:45 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:45.103+0000 7fc4312d9040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T08:30:45.396 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:45 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:45.163+0000 7fc4312d9040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T08:30:45.396 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:45 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:45.327+0000 7fc4312d9040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T08:30:45.396 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:45 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:45.396+0000 7fc4312d9040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T08:30:45.543 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:45 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:45.318+0000 7fc74bed3040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T08:30:45.543 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:45 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:45.380+0000 7fc74bed3040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T08:30:45.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:45 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:30:45.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:45 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/38697379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3b55192-bd95-4497-89ea-dc017a25c0cc"}]: dispatch 2024-09-10T08:30:45.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:45 smithi111 ceph-mon[35220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3b55192-bd95-4497-89ea-dc017a25c0cc"}]: dispatch 2024-09-10T08:30:45.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:45 smithi111 ceph-mon[35220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3b55192-bd95-4497-89ea-dc017a25c0cc"}]': finished 2024-09-10T08:30:45.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:45 smithi111 ceph-mon[35220]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T08:30:45.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:45 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:30:45.693 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:45 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:30:45.693 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:45 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/38697379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3b55192-bd95-4497-89ea-dc017a25c0cc"}]: dispatch 2024-09-10T08:30:45.693 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:45 smithi149 ceph-mon[36167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3b55192-bd95-4497-89ea-dc017a25c0cc"}]: dispatch 2024-09-10T08:30:45.693 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:45 smithi149 ceph-mon[36167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3b55192-bd95-4497-89ea-dc017a25c0cc"}]': finished 2024-09-10T08:30:45.694 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:45 smithi149 ceph-mon[36167]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T08:30:45.694 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:45 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:30:45.696 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:45 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:45.458+0000 7fc4312d9040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T08:30:45.696 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:45 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:45.627+0000 7fc4312d9040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T08:30:45.696 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:45 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:45.693+0000 7fc4312d9040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T08:30:45.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:45 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:30:45.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:45 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/38697379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3b55192-bd95-4497-89ea-dc017a25c0cc"}]: dispatch 2024-09-10T08:30:45.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:45 smithi089 ceph-mon[30757]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3b55192-bd95-4497-89ea-dc017a25c0cc"}]: dispatch 2024-09-10T08:30:45.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:45 smithi089 ceph-mon[30757]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3b55192-bd95-4497-89ea-dc017a25c0cc"}]': finished 2024-09-10T08:30:45.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:45 smithi089 ceph-mon[30757]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T08:30:45.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:45 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:30:45.967 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:45 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:45.765+0000 7fc4312d9040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T08:30:45.967 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:45 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:45.849+0000 7fc4312d9040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T08:30:45.967 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:45 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:45.908+0000 7fc4312d9040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T08:30:45.967 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:45 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:45.967+0000 7fc4312d9040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T08:30:46.042 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:45 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:45.571+0000 7fc74bed3040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T08:30:46.793 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:46 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:46.468+0000 7fc74bed3040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T08:30:46.793 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:46 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:46.537+0000 7fc74bed3040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T08:30:46.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:46 smithi111 ceph-mon[35220]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:46.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:46.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:46.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:46.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:46.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:46.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:30:46.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:46.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:30:46.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:46 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3749342282' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:30:46.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:46.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:30:46.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:46.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:46.970 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:30:46 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z[37974]: 2024-09-10T08:30:46.758+0000 7fc4312d9040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T08:30:46.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:46 smithi149 ceph-mon[36167]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:46.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:46.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:46.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:46.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:46.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:46.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:30:46.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:46.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:30:46.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:46 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3749342282' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:30:46.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:46.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:30:46.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:46.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:46 smithi089 ceph-mon[30757]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:30:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:30:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:46 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3749342282' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:30:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:47.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:30:47.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:47.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:30:47.506 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:47 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:47.250+0000 7fc74bed3040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T08:30:47.506 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:47 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:47.365+0000 7fc74bed3040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T08:30:47.506 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:47 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:47.505+0000 7fc74bed3040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T08:30:47.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:47.587+0000 7f1966a00640 1 -- 172.21.15.89:0/1364320565 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f1960003800 con 0x7f1970101c00 2024-09-10T08:30:47.784 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:47 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:47.565+0000 7fc74bed3040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T08:30:47.784 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:47 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:47.716+0000 7fc74bed3040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T08:30:47.784 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:47 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:47.783+0000 7fc74bed3040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T08:30:47.784 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:47 smithi111 ceph-mon[35220]: Standby manager daemon z started 2024-09-10T08:30:47.784 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:47 smithi111 ceph-mon[35220]: from='mgr.? 172.21.15.149:0/1532587325' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T08:30:47.784 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:47 smithi111 ceph-mon[35220]: from='mgr.? 172.21.15.149:0/1532587325' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T08:30:47.784 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:47 smithi111 ceph-mon[35220]: from='mgr.? 172.21.15.149:0/1532587325' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T08:30:47.784 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:47 smithi111 ceph-mon[35220]: from='mgr.? 172.21.15.149:0/1532587325' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T08:30:47.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:47 smithi149 ceph-mon[36167]: Standby manager daemon z started 2024-09-10T08:30:47.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:47 smithi149 ceph-mon[36167]: from='mgr.? 172.21.15.149:0/1532587325' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T08:30:47.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:47 smithi149 ceph-mon[36167]: from='mgr.? 172.21.15.149:0/1532587325' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T08:30:47.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:47 smithi149 ceph-mon[36167]: from='mgr.? 172.21.15.149:0/1532587325' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T08:30:47.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:47 smithi149 ceph-mon[36167]: from='mgr.? 172.21.15.149:0/1532587325' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T08:30:48.042 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:47 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:47.843+0000 7fc74bed3040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T08:30:48.043 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:48 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:47.999+0000 7fc74bed3040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T08:30:48.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:47 smithi089 ceph-mon[30757]: Standby manager daemon z started 2024-09-10T08:30:48.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:47 smithi089 ceph-mon[30757]: from='mgr.? 172.21.15.149:0/1532587325' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T08:30:48.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:47 smithi089 ceph-mon[30757]: from='mgr.? 172.21.15.149:0/1532587325' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T08:30:48.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:47 smithi089 ceph-mon[30757]: from='mgr.? 172.21.15.149:0/1532587325' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T08:30:48.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:47 smithi089 ceph-mon[30757]: from='mgr.? 172.21.15.149:0/1532587325' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T08:30:48.327 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:48 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:48.064+0000 7fc74bed3040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T08:30:48.327 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:48 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:48.129+0000 7fc74bed3040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T08:30:48.327 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:48 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:48.210+0000 7fc74bed3040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T08:30:48.327 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:48 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:48.268+0000 7fc74bed3040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T08:30:48.327 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:48 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:48.326+0000 7fc74bed3040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T08:30:48.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:48 smithi149 ceph-mon[36167]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:48.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:48 smithi149 ceph-mon[36167]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-10T08:30:48.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:48 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T08:30:49.042 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:48 smithi111 ceph-mon[35220]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:49.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:48 smithi111 ceph-mon[35220]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-10T08:30:49.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:48 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T08:30:49.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:48 smithi089 ceph-mon[30757]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:49.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:48 smithi089 ceph-mon[30757]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-10T08:30:49.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:48 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T08:30:49.543 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:30:49 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y[36581]: 2024-09-10T08:30:49.108+0000 7fc74bed3040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T08:30:49.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:30:49.598+0000 7f1966a00640 1 -- 172.21.15.89:0/1364320565 <== mon.1 v2:172.21.15.149:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f196003bcc0 con 0x7f1970101c00 2024-09-10T08:30:49.965 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:49 smithi089 ceph-mon[30757]: Standby manager daemon y started 2024-09-10T08:30:49.965 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:49 smithi089 ceph-mon[30757]: from='mgr.? 172.21.15.111:0/1008216775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T08:30:49.965 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:49 smithi089 ceph-mon[30757]: from='mgr.? 172.21.15.111:0/1008216775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T08:30:49.965 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:49 smithi089 ceph-mon[30757]: from='mgr.? 172.21.15.111:0/1008216775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T08:30:49.965 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:49 smithi089 ceph-mon[30757]: from='mgr.? 172.21.15.111:0/1008216775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T08:30:49.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:49 smithi149 ceph-mon[36167]: Standby manager daemon y started 2024-09-10T08:30:49.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:49 smithi149 ceph-mon[36167]: from='mgr.? 172.21.15.111:0/1008216775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T08:30:49.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:49 smithi149 ceph-mon[36167]: from='mgr.? 172.21.15.111:0/1008216775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T08:30:49.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:49 smithi149 ceph-mon[36167]: from='mgr.? 172.21.15.111:0/1008216775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T08:30:49.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:49 smithi149 ceph-mon[36167]: from='mgr.? 172.21.15.111:0/1008216775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T08:30:50.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:49 smithi111 ceph-mon[35220]: Standby manager daemon y started 2024-09-10T08:30:50.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:49 smithi111 ceph-mon[35220]: from='mgr.? 172.21.15.111:0/1008216775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T08:30:50.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:49 smithi111 ceph-mon[35220]: from='mgr.? 172.21.15.111:0/1008216775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T08:30:50.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:49 smithi111 ceph-mon[35220]: from='mgr.? 172.21.15.111:0/1008216775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T08:30:50.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:49 smithi111 ceph-mon[35220]: from='mgr.? 172.21.15.111:0/1008216775' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T08:30:50.701 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:50 smithi089 ceph-mon[30757]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:50.701 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:50 smithi089 ceph-mon[30757]: mgrmap e15: x(active, since 68s), standbys: z, y 2024-09-10T08:30:50.701 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T08:30:50.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:50 smithi149 ceph-mon[36167]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:50.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:50 smithi149 ceph-mon[36167]: mgrmap e15: x(active, since 68s), standbys: z, y 2024-09-10T08:30:50.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T08:30:51.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:50 smithi111 ceph-mon[35220]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:51.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:50 smithi111 ceph-mon[35220]: mgrmap e15: x(active, since 68s), standbys: z, y 2024-09-10T08:30:51.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T08:30:52.853 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:52 smithi089 ceph-mon[30757]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:52.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:52 smithi149 ceph-mon[36167]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:53.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:52 smithi111 ceph-mon[35220]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:54.760 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:54 smithi089 ceph-mon[30757]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:54.761 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T08:30:54.761 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:54.858 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:54 smithi111 ceph-mon[35220]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:54.859 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T08:30:54.859 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:54.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:54 smithi149 ceph-mon[36167]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:54.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T08:30:54.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:30:55.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:55 smithi089 ceph-mon[30757]: Deploying daemon osd.0 on smithi089 2024-09-10T08:30:55.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:55 smithi149 ceph-mon[36167]: Deploying daemon osd.0 on smithi089 2024-09-10T08:30:56.042 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:55 smithi111 ceph-mon[35220]: Deploying daemon osd.0 on smithi089 2024-09-10T08:30:56.920 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:56 smithi089 ceph-mon[30757]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:56.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:56 smithi149 ceph-mon[36167]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:57.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:56 smithi111 ceph-mon[35220]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:58.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:30:58 smithi149 ceph-mon[36167]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:59.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:30:58 smithi111 ceph-mon[35220]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:30:59.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:30:58 smithi089 ceph-mon[30757]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:00.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:00 smithi149 ceph-mon[36167]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:01.027 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:00 smithi089 ceph-mon[30757]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:01.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:00 smithi111 ceph-mon[35220]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:02.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:02 smithi149 ceph-mon[36167]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:02.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:02 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:31:02.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:02 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:02.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:02 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:02 smithi111 ceph-mon[35220]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:02 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:31:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:02 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:02 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:03.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:02 smithi089 ceph-mon[30757]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:03.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:02 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:31:03.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:02 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:03.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:02 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:04.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:04 smithi149 ceph-mon[36167]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:04.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:04 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:04.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:04 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:04.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:04 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:04.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:04 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:31:04.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:04 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:05.042 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:04 smithi111 ceph-mon[35220]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:04 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:04 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:04 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:04 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:31:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:04 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:05.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:04 smithi089 ceph-mon[30757]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:05.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:04 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:05.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:04 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:05.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:04 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:05.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:04 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:31:05.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:04 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:06.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:31:06.576+0000 7f1966a00640 1 -- 172.21.15.89:0/1364320565 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1970062630 con 0x7f1954039290 2024-09-10T08:31:06.577 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 0 on host 'smithi089' 2024-09-10T08:31:06.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:31:06.579+0000 7f1975200640 1 -- 172.21.15.89:0/1364320565 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1954039290 msgr2=0x7f195403b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:31:06.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:31:06.579+0000 7f1975200640 1 --2- 172.21.15.89:0/1364320565 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1954039290 0x7f195403b750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1970075d10 tx=0x7f195c005f90 comp rx=0 tx=0).stop 2024-09-10T08:31:06.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:31:06.579+0000 7f1975200640 1 -- 172.21.15.89:0/1364320565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1970101c00 msgr2=0x7f1970075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:31:06.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:31:06.579+0000 7f1975200640 1 --2- 172.21.15.89:0/1364320565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1970101c00 0x7f1970075f90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1960031440 tx=0x7f1960032ce0 comp rx=0 tx=0).stop 2024-09-10T08:31:06.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:31:06.579+0000 7f1975200640 1 -- 172.21.15.89:0/1364320565 shutdown_connections 2024-09-10T08:31:06.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:31:06.579+0000 7f1975200640 1 --2- 172.21.15.89:0/1364320565 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1954039290 0x7f195403b750 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:06.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:31:06.579+0000 7f1975200640 1 --2- 172.21.15.89:0/1364320565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19701037c0 0x7f1970074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:06.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:31:06.579+0000 7f1975200640 1 --2- 172.21.15.89:0/1364320565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1970102e00 0x7f19700745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:06.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:31:06.579+0000 7f1975200640 1 --2- 172.21.15.89:0/1364320565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1970101c00 0x7f1970075f90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:06.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:31:06.579+0000 7f1975200640 1 -- 172.21.15.89:0/1364320565 >> 172.21.15.89:0/1364320565 conn(0x7f19700fd390 msgr2=0x7f19701061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:31:06.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:31:06.579+0000 7f1975200640 1 -- 172.21.15.89:0/1364320565 shutdown_connections 2024-09-10T08:31:06.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:31:06.579+0000 7f1975200640 1 -- 172.21.15.89:0/1364320565 wait complete. 2024-09-10T08:31:06.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:06 smithi089 ceph-mon[30757]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:06.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:06 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:31:06.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:06 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:06.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:06 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:06.974 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:06 smithi149 ceph-mon[36167]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:06.974 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:06 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:31:06.974 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:06 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:06.974 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:06 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:07.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:06 smithi111 ceph-mon[35220]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:07.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:06 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:31:07.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:06 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:07.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:06 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:07.279 DEBUG:teuthology.orchestra.run.smithi089:osd.0> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T08:31:07.283 INFO:tasks.cephadm:Deploying osd.1 on smithi111 with /dev/vg_nvme/lv_4... 2024-09-10T08:31:07.283 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T08:31:07.532 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:31:08.053 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:31:07 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-0[44616]: 2024-09-10T08:31:07.985+0000 7f7b64c38640 -1 osd.0 0 log_to_monitors true 2024-09-10T08:31:08.829 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:08 smithi111 ceph-mon[35220]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:08.830 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:08 smithi111 ceph-mon[35220]: from='osd.0 [v2:172.21.15.89:6802/1760523284,v1:172.21.15.89:6803/1760523284]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T08:31:08.830 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:08 smithi111 ceph-mon[35220]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T08:31:08.866 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:08 smithi089 ceph-mon[30757]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:08.866 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:08 smithi089 ceph-mon[30757]: from='osd.0 [v2:172.21.15.89:6802/1760523284,v1:172.21.15.89:6803/1760523284]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T08:31:08.866 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:08 smithi089 ceph-mon[30757]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T08:31:08.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:08 smithi149 ceph-mon[36167]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:08.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:08 smithi149 ceph-mon[36167]: from='osd.0 [v2:172.21.15.89:6802/1760523284,v1:172.21.15.89:6803/1760523284]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T08:31:08.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:08 smithi149 ceph-mon[36167]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T08:31:09.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:09 smithi111 ceph-mon[35220]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T08:31:09.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:09 smithi111 ceph-mon[35220]: from='osd.0 [v2:172.21.15.89:6802/1760523284,v1:172.21.15.89:6803/1760523284]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-10T08:31:09.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:09 smithi111 ceph-mon[35220]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T08:31:09.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:09 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:31:09.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:09 smithi111 ceph-mon[35220]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-10T08:31:09.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:09 smithi111 ceph-mon[35220]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-10T08:31:09.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:09 smithi111 ceph-mon[35220]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T08:31:09.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:09 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:31:09.879 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-10T08:31:09.905 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_4 2024-09-10T08:31:09.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:09 smithi149 ceph-mon[36167]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T08:31:09.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:09 smithi149 ceph-mon[36167]: from='osd.0 [v2:172.21.15.89:6802/1760523284,v1:172.21.15.89:6803/1760523284]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-10T08:31:09.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:09 smithi149 ceph-mon[36167]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T08:31:09.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:09 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:31:09.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:09 smithi149 ceph-mon[36167]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-10T08:31:09.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:09 smithi149 ceph-mon[36167]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-10T08:31:09.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:09 smithi149 ceph-mon[36167]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T08:31:09.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:09 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:31:10.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:09 smithi089 ceph-mon[30757]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T08:31:10.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:09 smithi089 ceph-mon[30757]: from='osd.0 [v2:172.21.15.89:6802/1760523284,v1:172.21.15.89:6803/1760523284]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-10T08:31:10.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:09 smithi089 ceph-mon[30757]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T08:31:10.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:09 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:31:10.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:09 smithi089 ceph-mon[30757]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-10T08:31:10.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:09 smithi089 ceph-mon[30757]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-10T08:31:10.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:09 smithi089 ceph-mon[30757]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T08:31:10.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:09 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:31:10.153 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:31:10.509 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:31:10 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-0[44616]: 2024-09-10T08:31:10.163+0000 7f7b5f400640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T08:31:10.509 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:31:10 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-0[44616]: 2024-09-10T08:31:10.182+0000 7f7b55400640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:31:10.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.522+0000 7f7ca093e640 1 -- 172.21.15.111:0/2376607569 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c9c105030 msgr2=0x7f7c9c107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:31:10.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.522+0000 7f7ca093e640 1 --2- 172.21.15.111:0/2376607569 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c9c105030 0x7f7c9c107420 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009900 tx=0x7f7c8402a410 comp rx=0 tx=0).stop 2024-09-10T08:31:10.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.522+0000 7f7ca093e640 1 -- 172.21.15.111:0/2376607569 shutdown_connections 2024-09-10T08:31:10.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.522+0000 7f7ca093e640 1 --2- 172.21.15.111:0/2376607569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c9c107960 0x7f7c9c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:10.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.522+0000 7f7ca093e640 1 --2- 172.21.15.111:0/2376607569 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c9c105030 0x7f7c9c107420 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:10.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.522+0000 7f7ca093e640 1 --2- 172.21.15.111:0/2376607569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c9c102700 0x7f7c9c104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:10.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.522+0000 7f7ca093e640 1 -- 172.21.15.111:0/2376607569 >> 172.21.15.111:0/2376607569 conn(0x7f7c9c0fc2a0 msgr2=0x7f7c9c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:31:10.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.522+0000 7f7ca093e640 1 -- 172.21.15.111:0/2376607569 shutdown_connections 2024-09-10T08:31:10.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.522+0000 7f7ca093e640 1 -- 172.21.15.111:0/2376607569 wait complete. 2024-09-10T08:31:10.529 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.522+0000 7f7ca093e640 1 Processor -- start 2024-09-10T08:31:10.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.523+0000 7f7ca093e640 1 -- start start 2024-09-10T08:31:10.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.523+0000 7f7ca093e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c9c102700 0x7f7c9c1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:31:10.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.523+0000 7f7ca093e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c9c105030 0x7f7c9c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:31:10.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.523+0000 7f7ca093e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c9c107960 0x7f7c9c19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:31:10.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.523+0000 7f7ca093e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c10d9e0 con 0x7f7c9c107960 2024-09-10T08:31:10.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.523+0000 7f7ca093e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c10d860 con 0x7f7c9c102700 2024-09-10T08:31:10.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.523+0000 7f7ca093e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c10db60 con 0x7f7c9c105030 2024-09-10T08:31:10.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.523+0000 7f7c9a59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c9c102700 0x7f7c9c1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:31:10.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.523+0000 7f7c9a59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c9c102700 0x7f7c9c1980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35524/0 (socket says 172.21.15.111:35524) 2024-09-10T08:31:10.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.523+0000 7f7c9a59c640 1 -- 172.21.15.111:0/3763202933 learned_addr learned my addr 172.21.15.111:0/3763202933 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-10T08:31:10.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.523+0000 7f7c99d9b640 1 --2- 172.21.15.111:0/3763202933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c9c105030 0x7f7c9c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:31:10.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.523+0000 7f7c9a59c640 1 -- 172.21.15.111:0/3763202933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c9c105030 msgr2=0x7f7c9c198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:31:10.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.523+0000 7f7c9a59c640 1 --2- 172.21.15.111:0/3763202933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c9c105030 0x7f7c9c198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:10.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.523+0000 7f7c9a59c640 1 -- 172.21.15.111:0/3763202933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c9c107960 msgr2=0x7f7c9c19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:31:10.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.523+0000 7f7c9a59c640 1 --2- 172.21.15.111:0/3763202933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c9c107960 0x7f7c9c19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:10.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.523+0000 7f7c9a59c640 1 -- 172.21.15.111:0/3763202933 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9c19fd20 con 0x7f7c9c102700 2024-09-10T08:31:10.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.523+0000 7f7c9a59c640 1 --2- 172.21.15.111:0/3763202933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c9c102700 0x7f7c9c1980f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00b4a0 tx=0x7f7c8c00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:31:10.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.524+0000 7f7c937fe640 1 -- 172.21.15.111:0/3763202933 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c014070 con 0x7f7c9c102700 2024-09-10T08:31:10.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.524+0000 7f7ca093e640 1 -- 172.21.15.111:0/3763202933 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9c19ffb0 con 0x7f7c9c102700 2024-09-10T08:31:10.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.524+0000 7f7c937fe640 1 -- 172.21.15.111:0/3763202933 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c8c010af0 con 0x7f7c9c102700 2024-09-10T08:31:10.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.524+0000 7f7c937fe640 1 -- 172.21.15.111:0/3763202933 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c017d10 con 0x7f7c9c102700 2024-09-10T08:31:10.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.524+0000 7f7ca093e640 1 -- 172.21.15.111:0/3763202933 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c9c1a0530 con 0x7f7c9c102700 2024-09-10T08:31:10.531 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.525+0000 7f7ca093e640 1 -- 172.21.15.111:0/3763202933 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c9c0fdd90 con 0x7f7c9c102700 2024-09-10T08:31:10.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.526+0000 7f7c937fe640 1 -- 172.21.15.111:0/3763202933 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f7c8c017070 con 0x7f7c9c102700 2024-09-10T08:31:10.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.526+0000 7f7c937fe640 1 --2- 172.21.15.111:0/3763202933 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7c880a2b20 0x7f7c880a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:31:10.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.527+0000 7f7c937fe640 1 -- 172.21.15.111:0/3763202933 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f7c8c0cc240 con 0x7f7c9c102700 2024-09-10T08:31:10.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.531+0000 7f7c937fe640 1 -- 172.21.15.111:0/3763202933 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7c8c09ad30 con 0x7f7c9c102700 2024-09-10T08:31:10.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.533+0000 7f7c99d9b640 1 --2- 172.21.15.111:0/3763202933 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7c880a2b20 0x7f7c880a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:31:10.533 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.534+0000 7f7c99d9b640 1 --2- 172.21.15.111:0/3763202933 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7c880a2b20 0x7f7c880a4fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7c8402aa00 tx=0x7f7c84031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:31:10.705 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:10.705+0000 7f7ca093e640 1 -- 172.21.15.111:0/3763202933 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7c9c062630 con 0x7f7c880a2b20 2024-09-10T08:31:10.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:10 smithi111 ceph-mon[35220]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:10.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:31:10.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:10 smithi111 ceph-mon[35220]: from='osd.0 ' entity='osd.0' 2024-09-10T08:31:10.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:10 smithi089 ceph-mon[30757]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:10.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:31:10.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:10 smithi089 ceph-mon[30757]: from='osd.0 ' entity='osd.0' 2024-09-10T08:31:10.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:10 smithi149 ceph-mon[36167]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:10.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:31:10.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:10 smithi149 ceph-mon[36167]: from='osd.0 ' entity='osd.0' 2024-09-10T08:31:11.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:11 smithi149 ceph-mon[36167]: purged_snaps scrub starts 2024-09-10T08:31:11.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:11 smithi149 ceph-mon[36167]: purged_snaps scrub ok 2024-09-10T08:31:11.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:11 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:31:11.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:11 smithi149 ceph-mon[36167]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:31:11.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:11 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:31:11.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:11 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:31:11.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:11 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:11.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:11 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:11.972 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:11 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:11.972 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:11 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:31:11.972 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:11 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:11.972 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:11 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:11.972 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:11 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:31:11.972 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:11 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:11.973 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:11 smithi149 ceph-mon[36167]: osd.0 [v2:172.21.15.89:6802/1760523284,v1:172.21.15.89:6803/1760523284] boot 2024-09-10T08:31:11.973 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:11 smithi149 ceph-mon[36167]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T08:31:11.973 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:11 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:31:12.019 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:11 smithi111 ceph-mon[35220]: purged_snaps scrub starts 2024-09-10T08:31:12.019 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:11 smithi111 ceph-mon[35220]: purged_snaps scrub ok 2024-09-10T08:31:12.019 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:11 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:31:12.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:11 smithi111 ceph-mon[35220]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:31:12.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:11 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:31:12.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:11 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:31:12.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:11 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:12.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:11 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:12.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:11 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:12.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:11 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:31:12.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:11 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:12.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:11 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:12.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:11 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:31:12.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:11 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:12.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:11 smithi111 ceph-mon[35220]: osd.0 [v2:172.21.15.89:6802/1760523284,v1:172.21.15.89:6803/1760523284] boot 2024-09-10T08:31:12.021 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:11 smithi111 ceph-mon[35220]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T08:31:12.021 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:11 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:31:12.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:11 smithi089 ceph-mon[30757]: purged_snaps scrub starts 2024-09-10T08:31:12.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:11 smithi089 ceph-mon[30757]: purged_snaps scrub ok 2024-09-10T08:31:12.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:31:12.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:11 smithi089 ceph-mon[30757]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:31:12.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:31:12.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:31:12.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:12.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:12.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:12.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:31:12.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:12.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:12.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:31:12.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:12.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:11 smithi089 ceph-mon[30757]: osd.0 [v2:172.21.15.89:6802/1760523284,v1:172.21.15.89:6803/1760523284] boot 2024-09-10T08:31:12.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:11 smithi089 ceph-mon[30757]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T08:31:12.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:31:12.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:12 smithi149 ceph-mon[36167]: Detected new or changed devices on smithi089 2024-09-10T08:31:12.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:12 smithi149 ceph-mon[36167]: Adjusting osd_memory_target on smithi089 to 17089M 2024-09-10T08:31:12.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:12 smithi149 ceph-mon[36167]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:13.023 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:12 smithi111 ceph-mon[35220]: Detected new or changed devices on smithi089 2024-09-10T08:31:13.023 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:12 smithi111 ceph-mon[35220]: Adjusting osd_memory_target on smithi089 to 17089M 2024-09-10T08:31:13.023 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:12 smithi111 ceph-mon[35220]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:13.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:12 smithi089 ceph-mon[30757]: Detected new or changed devices on smithi089 2024-09-10T08:31:13.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:12 smithi089 ceph-mon[30757]: Adjusting osd_memory_target on smithi089 to 17089M 2024-09-10T08:31:13.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:12 smithi089 ceph-mon[30757]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:31:13.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:13 smithi149 ceph-mon[36167]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T08:31:13.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:13 smithi149 ceph-mon[36167]: from='client.? 172.21.15.111:0/1318516714' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60aae117-cb52-4034-a700-11693171cd52"}]: dispatch 2024-09-10T08:31:13.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:13 smithi149 ceph-mon[36167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60aae117-cb52-4034-a700-11693171cd52"}]: dispatch 2024-09-10T08:31:13.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:13 smithi149 ceph-mon[36167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60aae117-cb52-4034-a700-11693171cd52"}]': finished 2024-09-10T08:31:13.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:13 smithi149 ceph-mon[36167]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T08:31:13.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:13 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:31:13.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:13 smithi149 ceph-mon[36167]: from='client.? 172.21.15.111:0/765014214' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:31:14.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:13 smithi111 ceph-mon[35220]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T08:31:14.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:13 smithi111 ceph-mon[35220]: from='client.? 172.21.15.111:0/1318516714' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60aae117-cb52-4034-a700-11693171cd52"}]: dispatch 2024-09-10T08:31:14.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:13 smithi111 ceph-mon[35220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60aae117-cb52-4034-a700-11693171cd52"}]: dispatch 2024-09-10T08:31:14.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:13 smithi111 ceph-mon[35220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60aae117-cb52-4034-a700-11693171cd52"}]': finished 2024-09-10T08:31:14.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:13 smithi111 ceph-mon[35220]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T08:31:14.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:13 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:31:14.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:13 smithi111 ceph-mon[35220]: from='client.? 172.21.15.111:0/765014214' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:31:14.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:13 smithi089 ceph-mon[30757]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T08:31:14.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:13 smithi089 ceph-mon[30757]: from='client.? 172.21.15.111:0/1318516714' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60aae117-cb52-4034-a700-11693171cd52"}]: dispatch 2024-09-10T08:31:14.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:13 smithi089 ceph-mon[30757]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60aae117-cb52-4034-a700-11693171cd52"}]: dispatch 2024-09-10T08:31:14.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:13 smithi089 ceph-mon[30757]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60aae117-cb52-4034-a700-11693171cd52"}]': finished 2024-09-10T08:31:14.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:13 smithi089 ceph-mon[30757]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T08:31:14.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:31:14.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:13 smithi089 ceph-mon[30757]: from='client.? 172.21.15.111:0/765014214' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:31:14.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:14 smithi089 ceph-mon[30757]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:14.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:14 smithi149 ceph-mon[36167]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:15.042 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:14 smithi111 ceph-mon[35220]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:16.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:16 smithi149 ceph-mon[36167]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:17.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:16 smithi111 ceph-mon[35220]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:17.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:16 smithi089 ceph-mon[30757]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:18.713 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:18 smithi111 ceph-mon[35220]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:18.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:18 smithi149 ceph-mon[36167]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:19.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:18 smithi089 ceph-mon[30757]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:20.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:20 smithi149 ceph-mon[36167]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:20.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:20 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T08:31:20.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:20 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:20.992 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:20 smithi111 ceph-mon[35220]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:20.992 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:20 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T08:31:20.992 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:20 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:21.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:20 smithi089 ceph-mon[30757]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:21.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:20 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T08:31:21.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:20 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:21.948 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:21 smithi111 ceph-mon[35220]: Deploying daemon osd.1 on smithi111 2024-09-10T08:31:21.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:21 smithi149 ceph-mon[36167]: Deploying daemon osd.1 on smithi111 2024-09-10T08:31:22.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:21 smithi089 ceph-mon[30757]: Deploying daemon osd.1 on smithi111 2024-09-10T08:31:22.926 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:22 smithi111 ceph-mon[35220]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:22.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:22 smithi149 ceph-mon[36167]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:23.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:22 smithi089 ceph-mon[30757]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:24.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:24 smithi149 ceph-mon[36167]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:25.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:24 smithi111 ceph-mon[35220]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:25.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:24 smithi089 ceph-mon[30757]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:26.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:26 smithi111 ceph-mon[35220]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:26.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:31:26.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:26.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:26.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:26 smithi149 ceph-mon[36167]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:26.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:26 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:31:26.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:26 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:26.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:26 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:27.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:26 smithi089 ceph-mon[30757]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:27.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:26 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:31:27.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:26 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:27.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:26 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:28.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:28 smithi149 ceph-mon[36167]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:28.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:28 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:28.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:28 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:28.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:28 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:28.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:28 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:31:28.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:28 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:28.985 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:28 smithi111 ceph-mon[35220]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:28.985 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:28 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:28.985 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:28 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:28.985 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:28 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:28.985 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:28 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:31:28.985 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:28 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:29.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:28 smithi089 ceph-mon[30757]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:29.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:28 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:29.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:28 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:29.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:28 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:29.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:28 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:31:29.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:28 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:30.628 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:30.625+0000 7f7c937fe640 1 -- 172.21.15.111:0/3763202933 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7c9c062630 con 0x7f7c880a2b20 2024-09-10T08:31:30.629 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 1 on host 'smithi111' 2024-09-10T08:31:30.629 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:30.628+0000 7f7ca093e640 1 -- 172.21.15.111:0/3763202933 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7c880a2b20 msgr2=0x7f7c880a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:31:30.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:30.628+0000 7f7ca093e640 1 --2- 172.21.15.111:0/3763202933 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7c880a2b20 0x7f7c880a4fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7c8402aa00 tx=0x7f7c84031400 comp rx=0 tx=0).stop 2024-09-10T08:31:30.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:30.628+0000 7f7ca093e640 1 -- 172.21.15.111:0/3763202933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c9c102700 msgr2=0x7f7c9c1980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:31:30.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:30.628+0000 7f7ca093e640 1 --2- 172.21.15.111:0/3763202933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c9c102700 0x7f7c9c1980f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c00b4a0 tx=0x7f7c8c00b960 comp rx=0 tx=0).stop 2024-09-10T08:31:30.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:30.628+0000 7f7ca093e640 1 -- 172.21.15.111:0/3763202933 shutdown_connections 2024-09-10T08:31:30.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:30.628+0000 7f7ca093e640 1 --2- 172.21.15.111:0/3763202933 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7c880a2b20 0x7f7c880a4fe0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:30.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:30.628+0000 7f7ca093e640 1 --2- 172.21.15.111:0/3763202933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c9c107960 0x7f7c9c19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:30.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:30.628+0000 7f7ca093e640 1 --2- 172.21.15.111:0/3763202933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c9c105030 0x7f7c9c198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:30.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:30.628+0000 7f7ca093e640 1 --2- 172.21.15.111:0/3763202933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c9c102700 0x7f7c9c1980f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:30.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:30.628+0000 7f7ca093e640 1 -- 172.21.15.111:0/3763202933 >> 172.21.15.111:0/3763202933 conn(0x7f7c9c0fc2a0 msgr2=0x7f7c9c103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:31:30.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:30.628+0000 7f7ca093e640 1 -- 172.21.15.111:0/3763202933 shutdown_connections 2024-09-10T08:31:30.630 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:31:30.628+0000 7f7ca093e640 1 -- 172.21.15.111:0/3763202933 wait complete. 2024-09-10T08:31:30.924 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:30 smithi111 ceph-mon[35220]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:30.924 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:31:30.924 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:30.924 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:30.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:30 smithi149 ceph-mon[36167]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:30.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:31:30.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:30.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:31.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:30 smithi089 ceph-mon[30757]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:31:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:31.234 DEBUG:teuthology.orchestra.run.smithi111:osd.1> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T08:31:31.236 INFO:tasks.cephadm:Deploying osd.2 on smithi149 with /dev/vg_nvme/lv_4... 2024-09-10T08:31:31.236 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T08:31:31.487 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:31:32.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:32 smithi111 ceph-mon[35220]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:32.793 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:31:32 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-1[39687]: 2024-09-10T08:31:32.705+0000 7f349d2bb640 -1 osd.1 0 log_to_monitors true 2024-09-10T08:31:32.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:32 smithi149 ceph-mon[36167]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:33.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:32 smithi089 ceph-mon[30757]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:33.669 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:31:33.695 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_4 2024-09-10T08:31:33.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:33 smithi111 ceph-mon[35220]: from='osd.1 [v2:172.21.15.111:6800/2369291742,v1:172.21.15.111:6801/2369291742]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T08:31:33.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:33 smithi111 ceph-mon[35220]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T08:31:33.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:33 smithi089 ceph-mon[30757]: from='osd.1 [v2:172.21.15.111:6800/2369291742,v1:172.21.15.111:6801/2369291742]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T08:31:33.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:33 smithi089 ceph-mon[30757]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T08:31:33.937 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:31:33.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:33 smithi149 ceph-mon[36167]: from='osd.1 [v2:172.21.15.111:6800/2369291742,v1:172.21.15.111:6801/2369291742]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T08:31:33.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:33 smithi149 ceph-mon[36167]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T08:31:34.379 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.377+0000 7fbb2712a640 1 -- 172.21.15.149:0/2794956420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb20100ad0 msgr2=0x7fbb20100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:31:34.379 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.377+0000 7fbb2712a640 1 --2- 172.21.15.149:0/2794956420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb20100ad0 0x7fbb20100ed0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009900 tx=0x7fbb1002a410 comp rx=0 tx=0).stop 2024-09-10T08:31:34.379 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.377+0000 7fbb2712a640 1 -- 172.21.15.149:0/2794956420 shutdown_connections 2024-09-10T08:31:34.379 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.377+0000 7fbb2712a640 1 --2- 172.21.15.149:0/2794956420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb20102690 0x7fbb20108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:34.379 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.377+0000 7fbb2712a640 1 --2- 172.21.15.149:0/2794956420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb20101cd0 0x7fbb20102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:34.379 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.377+0000 7fbb2712a640 1 --2- 172.21.15.149:0/2794956420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb20100ad0 0x7fbb20100ed0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:34.379 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.377+0000 7fbb2712a640 1 -- 172.21.15.149:0/2794956420 >> 172.21.15.149:0/2794956420 conn(0x7fbb200fc280 msgr2=0x7fbb200fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:31:34.379 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.377+0000 7fbb2712a640 1 -- 172.21.15.149:0/2794956420 shutdown_connections 2024-09-10T08:31:34.380 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.377+0000 7fbb2712a640 1 -- 172.21.15.149:0/2794956420 wait complete. 2024-09-10T08:31:34.380 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.377+0000 7fbb2712a640 1 Processor -- start 2024-09-10T08:31:34.380 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.378+0000 7fbb2712a640 1 -- start start 2024-09-10T08:31:34.380 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.378+0000 7fbb2712a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb20100ad0 0x7fbb2006aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:31:34.380 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.378+0000 7fbb2712a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb20101cd0 0x7fbb2019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:31:34.380 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.378+0000 7fbb2712a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb20102690 0x7fbb2019f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:31:34.380 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.378+0000 7fbb2712a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2010c7b0 con 0x7fbb20100ad0 2024-09-10T08:31:34.380 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.378+0000 7fbb2712a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2010c630 con 0x7fbb20101cd0 2024-09-10T08:31:34.380 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.378+0000 7fbb2712a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2010c930 con 0x7fbb20102690 2024-09-10T08:31:34.380 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.378+0000 7fbb256c7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb20102690 0x7fbb2019f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:31:34.380 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.378+0000 7fbb256c7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb20102690 0x7fbb2019f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:59894/0 (socket says 172.21.15.149:59894) 2024-09-10T08:31:34.380 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.378+0000 7fbb256c7640 1 -- 172.21.15.149:0/1978040282 learned_addr learned my addr 172.21.15.149:0/1978040282 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-09-10T08:31:34.380 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.378+0000 7fbb1ffff640 1 --2- 172.21.15.149:0/1978040282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb20101cd0 0x7fbb2019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:31:34.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.378+0000 7fbb24ec6640 1 --2- 172.21.15.149:0/1978040282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb20100ad0 0x7fbb2006aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:31:34.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.378+0000 7fbb256c7640 1 -- 172.21.15.149:0/1978040282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb20101cd0 msgr2=0x7fbb2019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:31:34.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.378+0000 7fbb256c7640 1 --2- 172.21.15.149:0/1978040282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb20101cd0 0x7fbb2019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:34.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.378+0000 7fbb256c7640 1 -- 172.21.15.149:0/1978040282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb20100ad0 msgr2=0x7fbb2006aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:31:34.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.378+0000 7fbb256c7640 1 --2- 172.21.15.149:0/1978040282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb20100ad0 0x7fbb2006aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:34.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.378+0000 7fbb256c7640 1 -- 172.21.15.149:0/1978040282 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2019fc00 con 0x7fbb20102690 2024-09-10T08:31:34.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.378+0000 7fbb256c7640 1 --2- 172.21.15.149:0/1978040282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb20102690 0x7fbb2019f2f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800c970 tx=0x7fbb1800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:31:34.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.379+0000 7fbb1dffb640 1 -- 172.21.15.149:0/1978040282 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18007bb0 con 0x7fbb20102690 2024-09-10T08:31:34.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.379+0000 7fbb1dffb640 1 -- 172.21.15.149:0/1978040282 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb18007d50 con 0x7fbb20102690 2024-09-10T08:31:34.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.379+0000 7fbb2712a640 1 -- 172.21.15.149:0/1978040282 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb2019fe90 con 0x7fbb20102690 2024-09-10T08:31:34.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.379+0000 7fbb1dffb640 1 -- 172.21.15.149:0/1978040282 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1800f450 con 0x7fbb20102690 2024-09-10T08:31:34.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.379+0000 7fbb2712a640 1 -- 172.21.15.149:0/1978040282 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb201a0420 con 0x7fbb20102690 2024-09-10T08:31:34.382 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.380+0000 7fbb2712a640 1 -- 172.21.15.149:0/1978040282 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb20107f20 con 0x7fbb20102690 2024-09-10T08:31:34.382 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.381+0000 7fbb1dffb640 1 -- 172.21.15.149:0/1978040282 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fbb18004f60 con 0x7fbb20102690 2024-09-10T08:31:34.382 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.381+0000 7fbb1dffb640 1 --2- 172.21.15.149:0/1978040282 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbb080a2b70 0x7fbb080a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:31:34.382 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.382+0000 7fbb1dffb640 1 -- 172.21.15.149:0/1978040282 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 1903+0+0 (secure 0 0 0) 0x7fbb180cd2a0 con 0x7fbb20102690 2024-09-10T08:31:34.383 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.382+0000 7fbb24ec6640 1 --2- 172.21.15.149:0/1978040282 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbb080a2b70 0x7fbb080a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:31:34.383 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.382+0000 7fbb24ec6640 1 --2- 172.21.15.149:0/1978040282 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbb080a2b70 0x7fbb080a5030 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbb10006010 tx=0x7fbb10000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:31:34.386 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.386+0000 7fbb1dffb640 1 -- 172.21.15.149:0/1978040282 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb18014030 con 0x7fbb20102690 2024-09-10T08:31:34.560 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:34.560+0000 7fbb2712a640 1 -- 172.21.15.149:0/1978040282 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbb20062630 con 0x7fbb080a2b70 2024-09-10T08:31:34.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:34 smithi149 ceph-mon[36167]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:34.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:34 smithi149 ceph-mon[36167]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T08:31:34.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:34 smithi149 ceph-mon[36167]: from='osd.1 [v2:172.21.15.111:6800/2369291742,v1:172.21.15.111:6801/2369291742]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:31:34.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:34 smithi149 ceph-mon[36167]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T08:31:34.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:31:34.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:34 smithi149 ceph-mon[36167]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:31:34.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:34.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:34.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:31:34.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:34.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:34.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:31:34.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:34.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:31:34.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:31:34.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:34 smithi111 ceph-mon[35220]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:34 smithi111 ceph-mon[35220]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T08:31:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:34 smithi111 ceph-mon[35220]: from='osd.1 [v2:172.21.15.111:6800/2369291742,v1:172.21.15.111:6801/2369291742]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:31:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:34 smithi111 ceph-mon[35220]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T08:31:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:31:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:34 smithi111 ceph-mon[35220]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:31:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:31:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:31:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:31:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:31:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:35.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:34 smithi089 ceph-mon[30757]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:35.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:34 smithi089 ceph-mon[30757]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T08:31:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:34 smithi089 ceph-mon[30757]: from='osd.1 [v2:172.21.15.111:6800/2369291742,v1:172.21.15.111:6801/2369291742]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:31:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:34 smithi089 ceph-mon[30757]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T08:31:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:31:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:34 smithi089 ceph-mon[30757]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:31:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:31:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:31:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:31:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:31:35.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:35.543 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:31:35 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-1[39687]: 2024-09-10T08:31:35.059+0000 7f349923e640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T08:31:35.543 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:31:35 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-1[39687]: 2024-09-10T08:31:35.061+0000 7f349105e640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:31:35.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:35 smithi149 ceph-mon[36167]: Detected new or changed devices on smithi111 2024-09-10T08:31:35.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:35 smithi149 ceph-mon[36167]: Adjusting osd_memory_target on smithi111 to 17088M 2024-09-10T08:31:35.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:35 smithi149 ceph-mon[36167]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:31:35.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:35 smithi149 ceph-mon[36167]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-10T08:31:35.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:35 smithi149 ceph-mon[36167]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T08:31:35.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:35 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:31:35.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:35 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:31:35.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:35 smithi149 ceph-mon[36167]: from='osd.1 ' entity='osd.1' 2024-09-10T08:31:36.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:35 smithi111 ceph-mon[35220]: Detected new or changed devices on smithi111 2024-09-10T08:31:36.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:35 smithi111 ceph-mon[35220]: Adjusting osd_memory_target on smithi111 to 17088M 2024-09-10T08:31:36.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:35 smithi111 ceph-mon[35220]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:31:36.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:35 smithi111 ceph-mon[35220]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-10T08:31:36.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:35 smithi111 ceph-mon[35220]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T08:31:36.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:35 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:31:36.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:35 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:31:36.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:35 smithi111 ceph-mon[35220]: from='osd.1 ' entity='osd.1' 2024-09-10T08:31:36.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:35 smithi089 ceph-mon[30757]: Detected new or changed devices on smithi111 2024-09-10T08:31:36.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:35 smithi089 ceph-mon[30757]: Adjusting osd_memory_target on smithi111 to 17088M 2024-09-10T08:31:36.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:35 smithi089 ceph-mon[30757]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:31:36.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:35 smithi089 ceph-mon[30757]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-10T08:31:36.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:35 smithi089 ceph-mon[30757]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T08:31:36.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:35 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:31:36.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:35 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:31:36.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:35 smithi089 ceph-mon[30757]: from='osd.1 ' entity='osd.1' 2024-09-10T08:31:36.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:36 smithi149 ceph-mon[36167]: purged_snaps scrub starts 2024-09-10T08:31:36.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:36 smithi149 ceph-mon[36167]: purged_snaps scrub ok 2024-09-10T08:31:36.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:36 smithi149 ceph-mon[36167]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:36.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:31:36.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:36 smithi149 ceph-mon[36167]: osd.1 [v2:172.21.15.111:6800/2369291742,v1:172.21.15.111:6801/2369291742] boot 2024-09-10T08:31:36.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:36 smithi149 ceph-mon[36167]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T08:31:36.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:31:36.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:36 smithi149 ceph-mon[36167]: from='client.? 172.21.15.149:0/3457916561' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a305a019-0870-437c-a298-4f923ec54e77"}]: dispatch 2024-09-10T08:31:36.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:36 smithi149 ceph-mon[36167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a305a019-0870-437c-a298-4f923ec54e77"}]: dispatch 2024-09-10T08:31:36.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:36 smithi149 ceph-mon[36167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a305a019-0870-437c-a298-4f923ec54e77"}]': finished 2024-09-10T08:31:36.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:36 smithi149 ceph-mon[36167]: osdmap e14: 3 total, 2 up, 3 in 2024-09-10T08:31:36.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:31:37.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:36 smithi111 ceph-mon[35220]: purged_snaps scrub starts 2024-09-10T08:31:37.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:36 smithi111 ceph-mon[35220]: purged_snaps scrub ok 2024-09-10T08:31:37.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:36 smithi111 ceph-mon[35220]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:37.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:31:37.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:36 smithi111 ceph-mon[35220]: osd.1 [v2:172.21.15.111:6800/2369291742,v1:172.21.15.111:6801/2369291742] boot 2024-09-10T08:31:37.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:36 smithi111 ceph-mon[35220]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T08:31:37.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:31:37.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:36 smithi111 ceph-mon[35220]: from='client.? 172.21.15.149:0/3457916561' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a305a019-0870-437c-a298-4f923ec54e77"}]: dispatch 2024-09-10T08:31:37.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:36 smithi111 ceph-mon[35220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a305a019-0870-437c-a298-4f923ec54e77"}]: dispatch 2024-09-10T08:31:37.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:36 smithi111 ceph-mon[35220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a305a019-0870-437c-a298-4f923ec54e77"}]': finished 2024-09-10T08:31:37.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:36 smithi111 ceph-mon[35220]: osdmap e14: 3 total, 2 up, 3 in 2024-09-10T08:31:37.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:31:37.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:36 smithi089 ceph-mon[30757]: purged_snaps scrub starts 2024-09-10T08:31:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:36 smithi089 ceph-mon[30757]: purged_snaps scrub ok 2024-09-10T08:31:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:36 smithi089 ceph-mon[30757]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:31:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:31:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:36 smithi089 ceph-mon[30757]: osd.1 [v2:172.21.15.111:6800/2369291742,v1:172.21.15.111:6801/2369291742] boot 2024-09-10T08:31:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:36 smithi089 ceph-mon[30757]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T08:31:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:31:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:36 smithi089 ceph-mon[30757]: from='client.? 172.21.15.149:0/3457916561' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a305a019-0870-437c-a298-4f923ec54e77"}]: dispatch 2024-09-10T08:31:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:36 smithi089 ceph-mon[30757]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a305a019-0870-437c-a298-4f923ec54e77"}]: dispatch 2024-09-10T08:31:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:36 smithi089 ceph-mon[30757]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a305a019-0870-437c-a298-4f923ec54e77"}]': finished 2024-09-10T08:31:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:36 smithi089 ceph-mon[30757]: osdmap e14: 3 total, 2 up, 3 in 2024-09-10T08:31:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:31:37.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:37 smithi149 ceph-mon[36167]: from='client.? 172.21.15.149:0/3101073946' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:31:38.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:37 smithi111 ceph-mon[35220]: from='client.? 172.21.15.149:0/3101073946' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:31:38.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:37 smithi089 ceph-mon[30757]: from='client.? 172.21.15.149:0/3101073946' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:31:38.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:38 smithi149 ceph-mon[36167]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:38.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:38 smithi149 ceph-mon[36167]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T08:31:38.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:38 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:31:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:38 smithi111 ceph-mon[35220]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:38 smithi111 ceph-mon[35220]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T08:31:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:38 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:31:39.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:38 smithi089 ceph-mon[30757]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:39.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:38 smithi089 ceph-mon[30757]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T08:31:39.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:38 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:31:40.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:39 smithi111 ceph-mon[35220]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:40.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:39 smithi089 ceph-mon[30757]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:40.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:39 smithi149 ceph-mon[36167]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:42.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:42 smithi089 ceph-mon[30757]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:42.947 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:42 smithi149 ceph-mon[36167]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:43.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:42 smithi111 ceph-mon[35220]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:44.668 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:44 smithi149 ceph-mon[36167]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:44.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:44 smithi089 ceph-mon[30757]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:45.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:44 smithi111 ceph-mon[35220]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:45.688 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:45 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T08:31:45.689 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:45 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:46.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:45 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T08:31:46.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:45 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:46.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:45 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T08:31:46.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:45 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:46.759 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:46 smithi149 ceph-mon[36167]: Deploying daemon osd.2 on smithi149 2024-09-10T08:31:46.759 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:46 smithi149 ceph-mon[36167]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:47.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:46 smithi111 ceph-mon[35220]: Deploying daemon osd.2 on smithi149 2024-09-10T08:31:47.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:46 smithi111 ceph-mon[35220]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:47.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:46 smithi089 ceph-mon[30757]: Deploying daemon osd.2 on smithi149 2024-09-10T08:31:47.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:46 smithi089 ceph-mon[30757]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:48.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:48 smithi149 ceph-mon[36167]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:49.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:48 smithi111 ceph-mon[35220]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:49.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:48 smithi089 ceph-mon[30757]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:50.665 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:50 smithi149 ceph-mon[36167]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:51.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:50 smithi111 ceph-mon[35220]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:51.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:50 smithi089 ceph-mon[30757]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:51.871 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:51 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:31:51.871 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:51 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:51.871 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:51 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:52.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:51 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:31:52.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:51 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:52.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:51 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:52.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:51 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:31:52.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:51 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:52.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:51 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:52.889 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:52 smithi149 ceph-mon[36167]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:53.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:52 smithi111 ceph-mon[35220]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:53.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:52 smithi089 ceph-mon[30757]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:54.829 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:54 smithi149 ceph-mon[36167]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:54.829 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:54.829 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:54.829 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:54.829 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:31:54.829 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:54.845 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:54 smithi111 ceph-mon[35220]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:54.845 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:54.845 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:54.845 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:54.845 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:31:54.846 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:55.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:54 smithi089 ceph-mon[30757]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:55.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:55.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:55.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:55.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:31:55.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:55.410 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:55.409+0000 7fbb1dffb640 1 -- 172.21.15.149:0/1978040282 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbb20062630 con 0x7fbb080a2b70 2024-09-10T08:31:55.410 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 2 on host 'smithi149' 2024-09-10T08:31:55.412 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:55.411+0000 7fbb2712a640 1 -- 172.21.15.149:0/1978040282 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbb080a2b70 msgr2=0x7fbb080a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:31:55.413 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:55.411+0000 7fbb2712a640 1 --2- 172.21.15.149:0/1978040282 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbb080a2b70 0x7fbb080a5030 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbb10006010 tx=0x7fbb10000bc0 comp rx=0 tx=0).stop 2024-09-10T08:31:55.413 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:55.412+0000 7fbb2712a640 1 -- 172.21.15.149:0/1978040282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb20102690 msgr2=0x7fbb2019f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:31:55.413 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:55.412+0000 7fbb2712a640 1 --2- 172.21.15.149:0/1978040282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb20102690 0x7fbb2019f2f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800c970 tx=0x7fbb1800ce30 comp rx=0 tx=0).stop 2024-09-10T08:31:55.413 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:55.412+0000 7fbb2712a640 1 -- 172.21.15.149:0/1978040282 shutdown_connections 2024-09-10T08:31:55.413 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:55.412+0000 7fbb2712a640 1 --2- 172.21.15.149:0/1978040282 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbb080a2b70 0x7fbb080a5030 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:55.413 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:55.412+0000 7fbb2712a640 1 --2- 172.21.15.149:0/1978040282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb20102690 0x7fbb2019f2f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:55.413 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:55.412+0000 7fbb2712a640 1 --2- 172.21.15.149:0/1978040282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb20101cd0 0x7fbb2019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:55.413 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:55.412+0000 7fbb2712a640 1 --2- 172.21.15.149:0/1978040282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb20100ad0 0x7fbb2006aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:31:55.413 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:55.412+0000 7fbb2712a640 1 -- 172.21.15.149:0/1978040282 >> 172.21.15.149:0/1978040282 conn(0x7fbb200fc280 msgr2=0x7fbb201050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:31:55.413 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:55.412+0000 7fbb2712a640 1 -- 172.21.15.149:0/1978040282 shutdown_connections 2024-09-10T08:31:55.413 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:31:55.412+0000 7fbb2712a640 1 -- 172.21.15.149:0/1978040282 wait complete. 2024-09-10T08:31:55.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:55 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:31:55.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:55 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:55.704 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:55 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:55.861 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:55 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:31:55.862 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:55 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:55.862 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:55 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:56.014 DEBUG:teuthology.orchestra.run.smithi149:osd.2> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T08:31:56.017 INFO:tasks.cephadm:Deploying osd.3 on smithi089 with /dev/vg_nvme/lv_3... 2024-09-10T08:31:56.017 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T08:31:56.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:55 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:31:56.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:55 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:56.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:55 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:56.385 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:31:56.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:56 smithi089 ceph-mon[30757]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:56.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:56 smithi149 ceph-mon[36167]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:57.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:56 smithi111 ceph-mon[35220]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:57.370 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:31:57 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-2[41115]: 2024-09-10T08:31:57.087+0000 7fee0047f640 -1 osd.2 0 log_to_monitors true 2024-09-10T08:31:57.638 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:57 smithi149 ceph-mon[36167]: from='osd.2 [v2:172.21.15.149:6800/1092261395,v1:172.21.15.149:6801/1092261395]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T08:31:57.638 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:57 smithi149 ceph-mon[36167]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T08:31:57.717 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:57 smithi089 ceph-mon[30757]: from='osd.2 [v2:172.21.15.149:6800/1092261395,v1:172.21.15.149:6801/1092261395]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T08:31:57.718 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:57 smithi089 ceph-mon[30757]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T08:31:58.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:57 smithi111 ceph-mon[35220]: from='osd.2 [v2:172.21.15.149:6800/1092261395,v1:172.21.15.149:6801/1092261395]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T08:31:58.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:57 smithi111 ceph-mon[35220]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T08:31:58.851 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:58 smithi089 ceph-mon[30757]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:58.851 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:58 smithi089 ceph-mon[30757]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T08:31:58.852 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:58 smithi089 ceph-mon[30757]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T08:31:58.852 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:58 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:31:58.852 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:58 smithi089 ceph-mon[30757]: from='osd.2 [v2:172.21.15.149:6800/1092261395,v1:172.21.15.149:6801/1092261395]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:31:58.852 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:58 smithi089 ceph-mon[30757]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:31:58.884 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:58 smithi149 ceph-mon[36167]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:58.884 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:58 smithi149 ceph-mon[36167]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T08:31:58.884 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:58 smithi149 ceph-mon[36167]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T08:31:58.884 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:58 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:31:58.884 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:58 smithi149 ceph-mon[36167]: from='osd.2 [v2:172.21.15.149:6800/1092261395,v1:172.21.15.149:6801/1092261395]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:31:58.884 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:58 smithi149 ceph-mon[36167]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:31:58.884 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:31:58 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-2[41115]: 2024-09-10T08:31:58.599+0000 7fedfc402640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T08:31:58.884 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:31:58 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-2[41115]: 2024-09-10T08:31:58.602+0000 7fedf822a640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:31:59.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:58 smithi111 ceph-mon[35220]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:31:59.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:58 smithi111 ceph-mon[35220]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T08:31:59.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:58 smithi111 ceph-mon[35220]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T08:31:59.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:58 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:31:59.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:58 smithi111 ceph-mon[35220]: from='osd.2 [v2:172.21.15.149:6800/1092261395,v1:172.21.15.149:6801/1092261395]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:31:59.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:58 smithi111 ceph-mon[35220]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:31:59.407 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:31:59.441 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi089:vg_nvme/lv_3 2024-09-10T08:31:59.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:59 smithi089 ceph-mon[30757]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T08:31:59.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:59 smithi089 ceph-mon[30757]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T08:31:59.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:31:59.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:31:59.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:59.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:59.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:31:59.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:59.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:59.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:31:59.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:31:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:59.847 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:31:59.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:59 smithi149 ceph-mon[36167]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T08:31:59.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:59 smithi149 ceph-mon[36167]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T08:31:59.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:31:59.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:31:59.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:59.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:59.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:31:59.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:31:59.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:31:59.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:31:59.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:31:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:00.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:59 smithi111 ceph-mon[35220]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T08:32:00.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:59 smithi111 ceph-mon[35220]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T08:32:00.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:32:00.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:32:00.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:00.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:00.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:32:00.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:00.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:00.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:32:00.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:31:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:00.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.392+0000 7f84f6000640 1 -- 172.21.15.89:0/3494119247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f0102e00 msgr2=0x7f84f0103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:32:00.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.392+0000 7f84f6000640 1 --2- 172.21.15.89:0/3494119247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f0102e00 0x7f84f0103280 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009860 tx=0x7f84dc02a340 comp rx=0 tx=0).stop 2024-09-10T08:32:00.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.392+0000 7f84f6000640 1 -- 172.21.15.89:0/3494119247 shutdown_connections 2024-09-10T08:32:00.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.392+0000 7f84f6000640 1 --2- 172.21.15.89:0/3494119247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f01037c0 0x7f84f010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:00.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.392+0000 7f84f6000640 1 --2- 172.21.15.89:0/3494119247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f0102e00 0x7f84f0103280 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:00.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.392+0000 7f84f6000640 1 --2- 172.21.15.89:0/3494119247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f0101c00 0x7f84f0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:00.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.392+0000 7f84f6000640 1 -- 172.21.15.89:0/3494119247 >> 172.21.15.89:0/3494119247 conn(0x7f84f00fd390 msgr2=0x7f84f00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:32:00.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.392+0000 7f84f6000640 1 -- 172.21.15.89:0/3494119247 shutdown_connections 2024-09-10T08:32:00.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.392+0000 7f84f6000640 1 -- 172.21.15.89:0/3494119247 wait complete. 2024-09-10T08:32:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.393+0000 7f84f6000640 1 Processor -- start 2024-09-10T08:32:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.393+0000 7f84f6000640 1 -- start start 2024-09-10T08:32:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.394+0000 7f84f6000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f0101c00 0x7f84f0197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:32:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.394+0000 7f84f6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f0102e00 0x7f84f0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:32:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.394+0000 7f84f6000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f01037c0 0x7f84f019f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:32:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.394+0000 7f84f6000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f84f010d860 con 0x7f84f01037c0 2024-09-10T08:32:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.394+0000 7f84f6000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f84f010d6e0 con 0x7f84f0102e00 2024-09-10T08:32:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.394+0000 7f84f6000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f84f010d9e0 con 0x7f84f0101c00 2024-09-10T08:32:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.394+0000 7f84eea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f0102e00 0x7f84f0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:32:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.394+0000 7f84eea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f0102e00 0x7f84f0198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:41596/0 (socket says 172.21.15.89:41596) 2024-09-10T08:32:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.394+0000 7f84eea00640 1 -- 172.21.15.89:0/323166058 learned_addr learned my addr 172.21.15.89:0/323166058 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:32:00.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.394+0000 7f84efe00640 1 --2- 172.21.15.89:0/323166058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f01037c0 0x7f84f019f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:32:00.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.394+0000 7f84ef400640 1 --2- 172.21.15.89:0/323166058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f0101c00 0x7f84f0197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:32:00.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.394+0000 7f84eea00640 1 -- 172.21.15.89:0/323166058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f0101c00 msgr2=0x7f84f0197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:32:00.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.394+0000 7f84eea00640 1 --2- 172.21.15.89:0/323166058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f0101c00 0x7f84f0197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:00.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.394+0000 7f84eea00640 1 -- 172.21.15.89:0/323166058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f01037c0 msgr2=0x7f84f019f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:32:00.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.394+0000 7f84eea00640 1 --2- 172.21.15.89:0/323166058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f01037c0 0x7f84f019f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:00.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.394+0000 7f84eea00640 1 -- 172.21.15.89:0/323166058 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f019fb20 con 0x7f84f0102e00 2024-09-10T08:32:00.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.395+0000 7f84eea00640 1 --2- 172.21.15.89:0/323166058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f0102e00 0x7f84f0198430 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f84dc035040 tx=0x7f84dc033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:32:00.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.395+0000 7f84e7e00640 1 -- 172.21.15.89:0/323166058 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc037040 con 0x7f84f0102e00 2024-09-10T08:32:00.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.395+0000 7f84e7e00640 1 -- 172.21.15.89:0/323166058 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84dc030850 con 0x7f84f0102e00 2024-09-10T08:32:00.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.395+0000 7f84e7e00640 1 -- 172.21.15.89:0/323166058 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc004920 con 0x7f84f0102e00 2024-09-10T08:32:00.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.395+0000 7f84f6000640 1 -- 172.21.15.89:0/323166058 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f019fdb0 con 0x7f84f0102e00 2024-09-10T08:32:00.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.395+0000 7f84f6000640 1 -- 172.21.15.89:0/323166058 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f01a01e0 con 0x7f84f0102e00 2024-09-10T08:32:00.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.396+0000 7f84f6000640 1 -- 172.21.15.89:0/323166058 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f0108ff0 con 0x7f84f0102e00 2024-09-10T08:32:00.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.398+0000 7f84e7e00640 1 -- 172.21.15.89:0/323166058 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f84dc004050 con 0x7f84f0102e00 2024-09-10T08:32:00.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.399+0000 7f84e7e00640 1 --2- 172.21.15.89:0/323166058 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f84d40a2b20 0x7f84d40a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:32:00.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.400+0000 7f84e7e00640 1 -- 172.21.15.89:0/323166058 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7f84dc0edcf0 con 0x7f84f0102e00 2024-09-10T08:32:00.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.400+0000 7f84ef400640 1 --2- 172.21.15.89:0/323166058 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f84d40a2b20 0x7f84d40a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:32:00.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.400+0000 7f84ef400640 1 --2- 172.21.15.89:0/323166058 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f84d40a2b20 0x7f84d40a4fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f84e0005d00 tx=0x7f84e0005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:32:00.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.402+0000 7f84e7e00640 1 -- 172.21.15.89:0/323166058 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f84dc0bc1e0 con 0x7f84f0102e00 2024-09-10T08:32:00.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:00.586+0000 7f84f6000640 1 -- 172.21.15.89:0/323166058 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f84f0062630 con 0x7f84d40a2b20 2024-09-10T08:32:00.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:00 smithi149 ceph-mon[36167]: purged_snaps scrub starts 2024-09-10T08:32:00.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:00 smithi149 ceph-mon[36167]: purged_snaps scrub ok 2024-09-10T08:32:00.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:00 smithi149 ceph-mon[36167]: Detected new or changed devices on smithi149 2024-09-10T08:32:00.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:00 smithi149 ceph-mon[36167]: Adjusting osd_memory_target on smithi149 to 5815M 2024-09-10T08:32:00.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:00 smithi149 ceph-mon[36167]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:32:00.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:00 smithi149 ceph-mon[36167]: osd.2 [v2:172.21.15.149:6800/1092261395,v1:172.21.15.149:6801/1092261395] boot 2024-09-10T08:32:00.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:00 smithi149 ceph-mon[36167]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T08:32:00.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:00 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:32:00.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:00 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:32:00.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:00 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:32:00.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:00 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:01.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:00 smithi111 ceph-mon[35220]: purged_snaps scrub starts 2024-09-10T08:32:01.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:00 smithi111 ceph-mon[35220]: purged_snaps scrub ok 2024-09-10T08:32:01.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:00 smithi111 ceph-mon[35220]: Detected new or changed devices on smithi149 2024-09-10T08:32:01.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:00 smithi111 ceph-mon[35220]: Adjusting osd_memory_target on smithi149 to 5815M 2024-09-10T08:32:01.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:00 smithi111 ceph-mon[35220]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:32:01.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:00 smithi111 ceph-mon[35220]: osd.2 [v2:172.21.15.149:6800/1092261395,v1:172.21.15.149:6801/1092261395] boot 2024-09-10T08:32:01.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:00 smithi111 ceph-mon[35220]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T08:32:01.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:00 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:32:01.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:00 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:32:01.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:00 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:32:01.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:00 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:01.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:00 smithi089 ceph-mon[30757]: purged_snaps scrub starts 2024-09-10T08:32:01.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:00 smithi089 ceph-mon[30757]: purged_snaps scrub ok 2024-09-10T08:32:01.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:00 smithi089 ceph-mon[30757]: Detected new or changed devices on smithi149 2024-09-10T08:32:01.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:00 smithi089 ceph-mon[30757]: Adjusting osd_memory_target on smithi149 to 5815M 2024-09-10T08:32:01.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:00 smithi089 ceph-mon[30757]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:32:01.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:00 smithi089 ceph-mon[30757]: osd.2 [v2:172.21.15.149:6800/1092261395,v1:172.21.15.149:6801/1092261395] boot 2024-09-10T08:32:01.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:00 smithi089 ceph-mon[30757]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T08:32:01.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:00 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:32:01.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:00 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:32:01.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:00 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:32:01.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:00 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:02.236 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:02 smithi089 ceph-mon[30757]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:32:02.236 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:02 smithi089 ceph-mon[30757]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T08:32:02.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:02 smithi149 ceph-mon[36167]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:32:02.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:02 smithi149 ceph-mon[36167]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T08:32:02.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:02 smithi111 ceph-mon[35220]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:32:02.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:02 smithi111 ceph-mon[35220]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T08:32:03.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:03 smithi149 ceph-mon[36167]: pgmap v79: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:03.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:03 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:32:03.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:03 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T08:32:03.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:03 smithi149 ceph-mon[36167]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T08:32:03.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:03 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:32:03.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:03 smithi111 ceph-mon[35220]: pgmap v79: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:03.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:03 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:32:03.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:03 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T08:32:03.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:03 smithi111 ceph-mon[35220]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T08:32:03.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:03 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:32:03.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:03 smithi089 ceph-mon[30757]: pgmap v79: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:03.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:03 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:32:03.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:03 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T08:32:03.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:03 smithi089 ceph-mon[30757]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T08:32:03.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:03 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:32:04.227 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:04 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T08:32:04.227 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:04 smithi089 ceph-mon[30757]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T08:32:04.227 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:04 smithi089 ceph-mon[30757]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:04.228 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:04 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1963865724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d02f54c1-f883-40f6-9341-8d9fc4b712ac"}]: dispatch 2024-09-10T08:32:04.228 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:04 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1963865724' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d02f54c1-f883-40f6-9341-8d9fc4b712ac"}]': finished 2024-09-10T08:32:04.228 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:04 smithi089 ceph-mon[30757]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T08:32:04.228 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:04 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:32:04.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:04 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T08:32:04.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:04 smithi149 ceph-mon[36167]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T08:32:04.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:04 smithi149 ceph-mon[36167]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:04.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:04 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1963865724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d02f54c1-f883-40f6-9341-8d9fc4b712ac"}]: dispatch 2024-09-10T08:32:04.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:04 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1963865724' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d02f54c1-f883-40f6-9341-8d9fc4b712ac"}]': finished 2024-09-10T08:32:04.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:04 smithi149 ceph-mon[36167]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T08:32:04.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:04 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:32:04.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:04 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T08:32:04.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:04 smithi111 ceph-mon[35220]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T08:32:04.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:04 smithi111 ceph-mon[35220]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:04.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:04 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1963865724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d02f54c1-f883-40f6-9341-8d9fc4b712ac"}]: dispatch 2024-09-10T08:32:04.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:04 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1963865724' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d02f54c1-f883-40f6-9341-8d9fc4b712ac"}]': finished 2024-09-10T08:32:04.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:04 smithi111 ceph-mon[35220]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T08:32:04.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:04 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:32:05.052 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:32:04 smithi089 sudo[49101]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T08:32:05.052 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:32:04 smithi089 sudo[49101]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:32:05.052 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:32:04 smithi089 sudo[49101]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:32:05.053 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:32:04 smithi089 sudo[49101]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:32:05.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:05 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3008016972' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:32:05.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:05 smithi149 ceph-mon[36167]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T08:32:05.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:05 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:32:05.470 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:05 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3008016972' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:32:05.470 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:05 smithi111 ceph-mon[35220]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T08:32:05.470 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:05 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:32:05.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:05 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3008016972' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:32:05.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:05 smithi089 ceph-mon[30757]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T08:32:05.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:05 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:32:05.553 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:32:05 smithi089 sudo[49109]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T08:32:05.553 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:32:05 smithi089 sudo[49109]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:32:05.553 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:32:05 smithi089 sudo[49109]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:32:05.553 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:32:05 smithi089 sudo[49109]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:32:05.723 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:32:05 smithi111 sudo[43395]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T08:32:05.723 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:32:05 smithi111 sudo[43395]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:32:05.723 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:32:05 smithi111 sudo[43395]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:32:05.723 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:32:05 smithi111 sudo[43395]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:32:06.043 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:32:05 smithi111 sudo[43398]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T08:32:06.043 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:32:05 smithi111 sudo[43398]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:32:06.043 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:32:05 smithi111 sudo[43398]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:32:06.043 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:32:05 smithi111 sudo[43398]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:32:06.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:06.241+0000 7f84e7e00640 1 -- 172.21.15.89:0/323166058 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f84dc0b8150 con 0x7f84f0102e00 2024-09-10T08:32:06.253 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:06 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:06.253 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:06 smithi149 ceph-mon[36167]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:06.253 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:06 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:06.254 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:32:05 smithi149 sudo[45022]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T08:32:06.254 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:32:05 smithi149 sudo[45022]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:32:06.254 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:32:05 smithi149 sudo[45022]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:32:06.254 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:32:06 smithi149 sudo[45022]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:32:06.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:06 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:06.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:06 smithi111 ceph-mon[35220]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:06.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:06 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:06.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:06 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:06.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:06 smithi089 ceph-mon[30757]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:06.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:06 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:06.720 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:32:06 smithi149 sudo[45025]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T08:32:06.720 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:32:06 smithi149 sudo[45025]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:32:06.720 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:32:06 smithi149 sudo[45025]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:32:06.720 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:32:06 smithi149 sudo[45025]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:32:07.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:06 smithi089 sudo[49676]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T08:32:07.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:06 smithi089 sudo[49676]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:32:07.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:06 smithi089 sudo[49676]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:32:07.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:06 smithi089 sudo[49676]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:32:07.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:07 smithi111 ceph-mon[35220]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T08:32:07.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:07 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:07.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:07 smithi111 ceph-mon[35220]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:32:07.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:07 smithi149 ceph-mon[36167]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T08:32:07.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:07 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:07.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:07 smithi149 ceph-mon[36167]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:32:07.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:07 smithi149 sudo[45028]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T08:32:07.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:07 smithi149 sudo[45028]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:32:07.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:07 smithi149 sudo[45028]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:32:07.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:07 smithi149 sudo[45028]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:32:07.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:07 smithi089 sudo[49866]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T08:32:07.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:07 smithi089 sudo[49866]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:32:07.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:07 smithi089 sudo[49866]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:32:07.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:07 smithi089 sudo[49866]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:32:07.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:07 smithi089 ceph-mon[30757]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T08:32:07.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:07 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:07.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:07 smithi089 ceph-mon[30757]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:32:07.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:07 smithi149 sudo[45031]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T08:32:07.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:07 smithi149 sudo[45031]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:32:07.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:07 smithi149 sudo[45031]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:32:07.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:07 smithi149 sudo[45031]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:32:08.351 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 sudo[43401]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T08:32:08.352 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 sudo[43401]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:32:08.352 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 sudo[43401]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:32:08.352 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 sudo[43401]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:32:08.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:08 smithi149 ceph-mon[36167]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:32:08.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:08 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:32:08.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:08 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:32:08.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:08 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:32:08.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:08 smithi149 ceph-mon[36167]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:32:08.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:08 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:32:08.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:08 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:32:08.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:08 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:32:08.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:08 smithi149 ceph-mon[36167]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:08.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:08 smithi149 ceph-mon[36167]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:32:08.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:08 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:32:08.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:08 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:32:08.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:08 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:32:08.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:08 smithi089 ceph-mon[30757]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:32:08.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:08 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:32:08.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:08 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:32:08.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:08 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:32:08.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:08 smithi089 ceph-mon[30757]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:32:08.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:08 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:32:08.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:08 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:32:08.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:08 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:32:08.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:08 smithi089 ceph-mon[30757]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:08.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:08 smithi089 ceph-mon[30757]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:32:08.745 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:08 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:32:08.746 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:08 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:32:08.746 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:08 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:32:08.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 sudo[43404]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T08:32:08.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 sudo[43404]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:32:08.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 sudo[43404]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:32:08.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 sudo[43404]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:32:08.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 ceph-mon[35220]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:32:08.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:32:08.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:32:08.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:32:08.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 ceph-mon[35220]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:32:08.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:32:08.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:32:08.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:32:08.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 ceph-mon[35220]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:08.795 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 ceph-mon[35220]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:32:08.795 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:32:08.795 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:32:08.795 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:08 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:32:09.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:09 smithi149 ceph-mon[36167]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:32:09.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:09 smithi149 ceph-mon[36167]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:32:09.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:09 smithi111 ceph-mon[35220]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:32:09.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:09 smithi111 ceph-mon[35220]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:32:09.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:09 smithi089 ceph-mon[30757]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:32:09.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:09 smithi089 ceph-mon[30757]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:32:10.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:10 smithi149 ceph-mon[36167]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:10.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:10 smithi111 ceph-mon[35220]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:10.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:10 smithi089 ceph-mon[30757]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:12.700 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:12 smithi089 ceph-mon[30757]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:12.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:12 smithi149 ceph-mon[36167]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:13.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:12 smithi111 ceph-mon[35220]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:13.671 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T08:32:13.672 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:13.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:13 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T08:32:13.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:13 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:14.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:13 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T08:32:14.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:13 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:14.657 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:14 smithi089 ceph-mon[30757]: Deploying daemon osd.3 on smithi089 2024-09-10T08:32:14.657 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:14 smithi089 ceph-mon[30757]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-10T08:32:14.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:14 smithi149 ceph-mon[36167]: Deploying daemon osd.3 on smithi089 2024-09-10T08:32:14.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:14 smithi149 ceph-mon[36167]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-10T08:32:15.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:14 smithi111 ceph-mon[35220]: Deploying daemon osd.3 on smithi089 2024-09-10T08:32:15.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:14 smithi111 ceph-mon[35220]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-10T08:32:16.681 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:16 smithi089 ceph-mon[30757]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-10T08:32:16.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:16 smithi149 ceph-mon[36167]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-10T08:32:17.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:16 smithi111 ceph-mon[35220]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-10T08:32:18.867 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:18 smithi089 ceph-mon[30757]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T08:32:18.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:18 smithi149 ceph-mon[36167]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T08:32:19.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:18 smithi111 ceph-mon[35220]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T08:32:20.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:20 smithi089 ceph-mon[30757]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T08:32:20.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:20 smithi149 ceph-mon[36167]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T08:32:21.042 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:20 smithi111 ceph-mon[35220]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T08:32:21.672 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:21 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:32:21.672 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:21 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:21.672 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:21 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:21.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:21 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:32:21.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:21 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:21.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:21 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:22.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:21 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:32:22.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:21 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:22.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:21 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:22.871 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:22 smithi149 ceph-mon[36167]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T08:32:23.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:22 smithi111 ceph-mon[35220]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T08:32:23.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:22 smithi089 ceph-mon[30757]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T08:32:24.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:24 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:24.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:24 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:24.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:24 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:24.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:24 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:32:24.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:24 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:24.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:24 smithi149 ceph-mon[36167]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T08:32:24.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:24 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:24.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:24 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:24.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:24 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:24.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:24 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:32:24.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:24 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:24.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:24 smithi111 ceph-mon[35220]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T08:32:24.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:24 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:24.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:24 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:24.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:24 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:24.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:24 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:32:24.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:24 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:24.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:24 smithi089 ceph-mon[30757]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-10T08:32:25.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:25.524+0000 7f84e7e00640 1 -- 172.21.15.89:0/323166058 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f84f0062630 con 0x7f84d40a2b20 2024-09-10T08:32:25.525 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 3 on host 'smithi089' 2024-09-10T08:32:25.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:25.527+0000 7f84f6000640 1 -- 172.21.15.89:0/323166058 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f84d40a2b20 msgr2=0x7f84d40a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:32:25.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:25.527+0000 7f84f6000640 1 --2- 172.21.15.89:0/323166058 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f84d40a2b20 0x7f84d40a4fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f84e0005d00 tx=0x7f84e0005c50 comp rx=0 tx=0).stop 2024-09-10T08:32:25.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:25.527+0000 7f84f6000640 1 -- 172.21.15.89:0/323166058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f0102e00 msgr2=0x7f84f0198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:32:25.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:25.527+0000 7f84f6000640 1 --2- 172.21.15.89:0/323166058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f0102e00 0x7f84f0198430 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f84dc035040 tx=0x7f84dc033d90 comp rx=0 tx=0).stop 2024-09-10T08:32:25.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:25.527+0000 7f84f6000640 1 -- 172.21.15.89:0/323166058 shutdown_connections 2024-09-10T08:32:25.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:25.527+0000 7f84f6000640 1 --2- 172.21.15.89:0/323166058 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f84d40a2b20 0x7f84d40a4fe0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:25.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:25.527+0000 7f84f6000640 1 --2- 172.21.15.89:0/323166058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84f01037c0 0x7f84f019f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:25.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:25.527+0000 7f84f6000640 1 --2- 172.21.15.89:0/323166058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f0102e00 0x7f84f0198430 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:25.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:25.527+0000 7f84f6000640 1 --2- 172.21.15.89:0/323166058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f0101c00 0x7f84f0197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:25.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:25.527+0000 7f84f6000640 1 -- 172.21.15.89:0/323166058 >> 172.21.15.89:0/323166058 conn(0x7f84f00fd390 msgr2=0x7f84f01061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:32:25.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:25.527+0000 7f84f6000640 1 -- 172.21.15.89:0/323166058 shutdown_connections 2024-09-10T08:32:25.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:32:25.527+0000 7f84f6000640 1 -- 172.21.15.89:0/323166058 wait complete. 2024-09-10T08:32:25.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:25 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:32:25.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:25 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:25.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:25 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:25.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:25 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:32:25.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:25 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:25.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:25 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:26.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:25 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:32:26.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:25 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:26.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:25 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:26.276 DEBUG:teuthology.orchestra.run.smithi089:osd.3> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T08:32:26.280 INFO:tasks.cephadm:Deploying osd.4 on smithi111 with /dev/vg_nvme/lv_3... 2024-09-10T08:32:26.280 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T08:32:26.532 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:32:26.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:26 smithi089 ceph-mon[30757]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:26.803 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 10 08:32:26 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-3[51082]: 2024-09-10T08:32:26.627+0000 7f607af61640 -1 osd.3 0 log_to_monitors true 2024-09-10T08:32:26.871 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:26 smithi111 ceph-mon[35220]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:26.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:26 smithi149 ceph-mon[36167]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:27.665 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:27 smithi089 ceph-mon[30757]: from='osd.3 [v2:172.21.15.89:6810/1405773950,v1:172.21.15.89:6811/1405773950]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T08:32:27.665 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:27 smithi089 ceph-mon[30757]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T08:32:27.950 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:27 smithi111 ceph-mon[35220]: from='osd.3 [v2:172.21.15.89:6810/1405773950,v1:172.21.15.89:6811/1405773950]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T08:32:27.950 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:27 smithi111 ceph-mon[35220]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T08:32:27.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:27 smithi149 ceph-mon[36167]: from='osd.3 [v2:172.21.15.89:6810/1405773950,v1:172.21.15.89:6811/1405773950]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T08:32:27.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:27 smithi149 ceph-mon[36167]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T08:32:28.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:28 smithi149 ceph-mon[36167]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:28.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:28 smithi149 ceph-mon[36167]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T08:32:28.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:28 smithi149 ceph-mon[36167]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T08:32:28.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:28 smithi149 ceph-mon[36167]: from='osd.3 [v2:172.21.15.89:6810/1405773950,v1:172.21.15.89:6811/1405773950]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-10T08:32:28.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:28 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:32:28.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:28 smithi149 ceph-mon[36167]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-10T08:32:29.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:28 smithi111 ceph-mon[35220]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:29.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:28 smithi111 ceph-mon[35220]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T08:32:29.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:28 smithi111 ceph-mon[35220]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T08:32:29.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:28 smithi111 ceph-mon[35220]: from='osd.3 [v2:172.21.15.89:6810/1405773950,v1:172.21.15.89:6811/1405773950]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-10T08:32:29.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:28 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:32:29.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:28 smithi111 ceph-mon[35220]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-10T08:32:29.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:28 smithi089 ceph-mon[30757]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:29.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:28 smithi089 ceph-mon[30757]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T08:32:29.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:28 smithi089 ceph-mon[30757]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T08:32:29.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:28 smithi089 ceph-mon[30757]: from='osd.3 [v2:172.21.15.89:6810/1405773950,v1:172.21.15.89:6811/1405773950]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-10T08:32:29.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:28 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:32:29.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:28 smithi089 ceph-mon[30757]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-10T08:32:29.094 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-10T08:32:29.120 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_3 2024-09-10T08:32:29.376 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:32:29.553 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 10 08:32:29 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-3[51082]: 2024-09-10T08:32:29.192+0000 7f6075e00640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T08:32:29.553 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 10 08:32:29 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-3[51082]: 2024-09-10T08:32:29.214+0000 7f606be00640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:32:29.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:29 smithi111 ceph-mon[35220]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-10T08:32:29.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:29 smithi111 ceph-mon[35220]: osdmap e25: 4 total, 3 up, 4 in 2024-09-10T08:32:29.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:29 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:32:29.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:29 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:32:29.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:29 smithi111 ceph-mon[35220]: from='osd.3 ' entity='osd.3' 2024-09-10T08:32:29.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:29 smithi111 ceph-mon[35220]: osd.3 [v2:172.21.15.89:6810/1405773950,v1:172.21.15.89:6811/1405773950] boot 2024-09-10T08:32:29.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:29 smithi111 ceph-mon[35220]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T08:32:29.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:29 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:32:29.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.811+0000 7f5c8fd86640 1 -- 172.21.15.111:0/1077956876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c88102120 msgr2=0x7f5c88102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:32:29.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.811+0000 7f5c8fd86640 1 --2- 172.21.15.111:0/1077956876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c88102120 0x7f5c88102520 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009900 tx=0x7f5c8402a410 comp rx=0 tx=0).stop 2024-09-10T08:32:29.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.811+0000 7f5c8fd86640 1 -- 172.21.15.111:0/1077956876 shutdown_connections 2024-09-10T08:32:29.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.811+0000 7f5c8fd86640 1 --2- 172.21.15.111:0/1077956876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c88101740 0x7f5c88108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:29.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.811+0000 7f5c8fd86640 1 --2- 172.21.15.111:0/1077956876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c88100d80 0x7f5c88101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:29.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.811+0000 7f5c8fd86640 1 --2- 172.21.15.111:0/1077956876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c88102120 0x7f5c88102520 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:29.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.811+0000 7f5c8fd86640 1 -- 172.21.15.111:0/1077956876 >> 172.21.15.111:0/1077956876 conn(0x7f5c880fba40 msgr2=0x7f5c880fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:32:29.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.812+0000 7f5c8fd86640 1 -- 172.21.15.111:0/1077956876 shutdown_connections 2024-09-10T08:32:29.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.812+0000 7f5c8fd86640 1 -- 172.21.15.111:0/1077956876 wait complete. 2024-09-10T08:32:29.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.812+0000 7f5c8fd86640 1 Processor -- start 2024-09-10T08:32:29.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.812+0000 7f5c8fd86640 1 -- start start 2024-09-10T08:32:29.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.812+0000 7f5c8fd86640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c88100d80 0x7f5c88198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:32:29.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.812+0000 7f5c8fd86640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c88101740 0x7f5c88198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:32:29.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.812+0000 7f5c8fd86640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c88102120 0x7f5c8819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:32:29.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.812+0000 7f5c8fd86640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8810c8b0 con 0x7f5c88100d80 2024-09-10T08:32:29.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.812+0000 7f5c8fd86640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8810c730 con 0x7f5c88102120 2024-09-10T08:32:29.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.812+0000 7f5c8fd86640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8810ca30 con 0x7f5c88101740 2024-09-10T08:32:29.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.812+0000 7f5c8e323640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c88102120 0x7f5c8819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:32:29.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.812+0000 7f5c8e323640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c88102120 0x7f5c8819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39322/0 (socket says 172.21.15.111:39322) 2024-09-10T08:32:29.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.812+0000 7f5c8e323640 1 -- 172.21.15.111:0/1458933464 learned_addr learned my addr 172.21.15.111:0/1458933464 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-10T08:32:29.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.813+0000 7f5c8db22640 1 --2- 172.21.15.111:0/1458933464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c88100d80 0x7f5c88198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:32:29.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.813+0000 7f5c8d321640 1 --2- 172.21.15.111:0/1458933464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c88101740 0x7f5c88198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:32:29.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.813+0000 7f5c8e323640 1 -- 172.21.15.111:0/1458933464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c88101740 msgr2=0x7f5c88198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:32:29.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.813+0000 7f5c8e323640 1 --2- 172.21.15.111:0/1458933464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c88101740 0x7f5c88198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:29.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.813+0000 7f5c8e323640 1 -- 172.21.15.111:0/1458933464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c88100d80 msgr2=0x7f5c88198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:32:29.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.813+0000 7f5c8e323640 1 --2- 172.21.15.111:0/1458933464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c88100d80 0x7f5c88198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:29.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.813+0000 7f5c8e323640 1 -- 172.21.15.111:0/1458933464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c8819fd70 con 0x7f5c88102120 2024-09-10T08:32:29.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.813+0000 7f5c8db22640 1 --2- 172.21.15.111:0/1458933464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c88100d80 0x7f5c88198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:32:29.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.813+0000 7f5c8e323640 1 --2- 172.21.15.111:0/1458933464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c88102120 0x7f5c8819f700 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000b970 tx=0x7f5c8000be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:32:29.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.813+0000 7f5c7effd640 1 -- 172.21.15.111:0/1458933464 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80019070 con 0x7f5c88102120 2024-09-10T08:32:29.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.813+0000 7f5c7effd640 1 -- 172.21.15.111:0/1458933464 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c8000cb40 con 0x7f5c88102120 2024-09-10T08:32:29.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.813+0000 7f5c8fd86640 1 -- 172.21.15.111:0/1458933464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c881a0060 con 0x7f5c88102120 2024-09-10T08:32:29.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.813+0000 7f5c7effd640 1 -- 172.21.15.111:0/1458933464 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80007610 con 0x7f5c88102120 2024-09-10T08:32:29.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.813+0000 7f5c8fd86640 1 -- 172.21.15.111:0/1458933464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c881a0620 con 0x7f5c88102120 2024-09-10T08:32:29.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.814+0000 7f5c8fd86640 1 -- 172.21.15.111:0/1458933464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c881a08e0 con 0x7f5c88102120 2024-09-10T08:32:29.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.815+0000 7f5c7effd640 1 -- 172.21.15.111:0/1458933464 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5c8001c070 con 0x7f5c88102120 2024-09-10T08:32:29.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.816+0000 7f5c7effd640 1 --2- 172.21.15.111:0/1458933464 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5c740a2bb0 0x7f5c740a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:32:29.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.816+0000 7f5c7effd640 1 -- 172.21.15.111:0/1458933464 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 3702+0+0 (secure 0 0 0) 0x7f5c800cd8b0 con 0x7f5c88102120 2024-09-10T08:32:29.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.816+0000 7f5c8db22640 1 --2- 172.21.15.111:0/1458933464 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5c740a2bb0 0x7f5c740a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:32:29.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.817+0000 7f5c8db22640 1 --2- 172.21.15.111:0/1458933464 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5c740a2bb0 0x7f5c740a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c84000c00 tx=0x7f5c840315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:32:29.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.819+0000 7f5c7effd640 1 -- 172.21.15.111:0/1458933464 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c8009ba10 con 0x7f5c88102120 2024-09-10T08:32:29.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:29 smithi149 ceph-mon[36167]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-10T08:32:29.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:29 smithi149 ceph-mon[36167]: osdmap e25: 4 total, 3 up, 4 in 2024-09-10T08:32:29.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:29 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:32:29.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:29 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:32:29.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:29 smithi149 ceph-mon[36167]: from='osd.3 ' entity='osd.3' 2024-09-10T08:32:29.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:29 smithi149 ceph-mon[36167]: osd.3 [v2:172.21.15.89:6810/1405773950,v1:172.21.15.89:6811/1405773950] boot 2024-09-10T08:32:29.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:29 smithi149 ceph-mon[36167]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T08:32:29.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:29 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:32:29.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:29.994+0000 7f5c8fd86640 1 -- 172.21.15.111:0/1458933464 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f5c881a0ab0 con 0x7f5c740a2bb0 2024-09-10T08:32:30.032 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:29 smithi089 ceph-mon[30757]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-10T08:32:30.032 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:29 smithi089 ceph-mon[30757]: osdmap e25: 4 total, 3 up, 4 in 2024-09-10T08:32:30.032 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:29 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:32:30.032 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:29 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:32:30.032 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:29 smithi089 ceph-mon[30757]: from='osd.3 ' entity='osd.3' 2024-09-10T08:32:30.032 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:29 smithi089 ceph-mon[30757]: osd.3 [v2:172.21.15.89:6810/1405773950,v1:172.21.15.89:6811/1405773950] boot 2024-09-10T08:32:30.032 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:29 smithi089 ceph-mon[30757]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T08:32:30.032 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:29 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:32:30.667 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:30 smithi111 ceph-mon[35220]: purged_snaps scrub starts 2024-09-10T08:32:30.667 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:30 smithi111 ceph-mon[35220]: purged_snaps scrub ok 2024-09-10T08:32:30.667 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:30 smithi111 ceph-mon[35220]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:30.668 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:32:30.668 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:32:30.668 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:30.668 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:30.668 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:30.668 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:32:30.668 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:32:30.668 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:30.668 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:30.668 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:32:30.668 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:30.668 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:30 smithi111 ceph-mon[35220]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T08:32:30.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:30 smithi149 ceph-mon[36167]: purged_snaps scrub starts 2024-09-10T08:32:30.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:30 smithi149 ceph-mon[36167]: purged_snaps scrub ok 2024-09-10T08:32:30.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:30 smithi149 ceph-mon[36167]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:30.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:32:30.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:32:30.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:30.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:30.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:30.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:32:30.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:32:30.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:30.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:30.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:32:30.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:30.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:30 smithi149 ceph-mon[36167]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T08:32:31.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:30 smithi089 ceph-mon[30757]: purged_snaps scrub starts 2024-09-10T08:32:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:30 smithi089 ceph-mon[30757]: purged_snaps scrub ok 2024-09-10T08:32:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:30 smithi089 ceph-mon[30757]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:32:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:32:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:32:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:32:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:32:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:32:31.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:31.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:30 smithi089 ceph-mon[30757]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T08:32:31.663 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:31 smithi111 ceph-mon[35220]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:32:31.663 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:31 smithi111 ceph-mon[35220]: Detected new or changed devices on smithi089 2024-09-10T08:32:31.664 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:31 smithi111 ceph-mon[35220]: Adjusting osd_memory_target on smithi089 to 8544M 2024-09-10T08:32:31.664 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:31 smithi111 ceph-mon[35220]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T08:32:31.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:31 smithi149 ceph-mon[36167]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:32:31.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:31 smithi149 ceph-mon[36167]: Detected new or changed devices on smithi089 2024-09-10T08:32:31.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:31 smithi149 ceph-mon[36167]: Adjusting osd_memory_target on smithi089 to 8544M 2024-09-10T08:32:31.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:31 smithi149 ceph-mon[36167]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T08:32:32.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:31 smithi089 ceph-mon[30757]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:32:32.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:31 smithi089 ceph-mon[30757]: Detected new or changed devices on smithi089 2024-09-10T08:32:32.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:31 smithi089 ceph-mon[30757]: Adjusting osd_memory_target on smithi089 to 8544M 2024-09-10T08:32:32.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:31 smithi089 ceph-mon[30757]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T08:32:32.954 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:32 smithi089 ceph-mon[30757]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:32.954 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:32 smithi089 ceph-mon[30757]: from='client.? 172.21.15.111:0/457643059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23cff46b-88dd-4344-be06-b35db6014f7f"}]: dispatch 2024-09-10T08:32:32.954 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:32 smithi089 ceph-mon[30757]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23cff46b-88dd-4344-be06-b35db6014f7f"}]: dispatch 2024-09-10T08:32:32.954 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:32 smithi089 ceph-mon[30757]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23cff46b-88dd-4344-be06-b35db6014f7f"}]': finished 2024-09-10T08:32:32.955 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:32 smithi089 ceph-mon[30757]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T08:32:32.955 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:32 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:32.965 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:32 smithi111 ceph-mon[35220]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:32.966 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:32 smithi111 ceph-mon[35220]: from='client.? 172.21.15.111:0/457643059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23cff46b-88dd-4344-be06-b35db6014f7f"}]: dispatch 2024-09-10T08:32:32.966 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:32 smithi111 ceph-mon[35220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23cff46b-88dd-4344-be06-b35db6014f7f"}]: dispatch 2024-09-10T08:32:32.966 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:32 smithi111 ceph-mon[35220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23cff46b-88dd-4344-be06-b35db6014f7f"}]': finished 2024-09-10T08:32:32.966 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:32 smithi111 ceph-mon[35220]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T08:32:32.966 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:32 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:32.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:32 smithi149 ceph-mon[36167]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:32.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:32 smithi149 ceph-mon[36167]: from='client.? 172.21.15.111:0/457643059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23cff46b-88dd-4344-be06-b35db6014f7f"}]: dispatch 2024-09-10T08:32:32.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:32 smithi149 ceph-mon[36167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23cff46b-88dd-4344-be06-b35db6014f7f"}]: dispatch 2024-09-10T08:32:32.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:32 smithi149 ceph-mon[36167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23cff46b-88dd-4344-be06-b35db6014f7f"}]': finished 2024-09-10T08:32:32.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:32 smithi149 ceph-mon[36167]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T08:32:32.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:32 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:33.592 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:33 smithi149 ceph-mon[36167]: from='client.? 172.21.15.111:0/3170390594' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:32:33.592 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:33 smithi149 ceph-mon[36167]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T08:32:33.592 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:33 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:33.967 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:33 smithi089 ceph-mon[30757]: from='client.? 172.21.15.111:0/3170390594' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:32:33.967 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:33 smithi089 ceph-mon[30757]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T08:32:33.967 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:33 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:33 smithi111 ceph-mon[35220]: from='client.? 172.21.15.111:0/3170390594' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:32:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:33 smithi111 ceph-mon[35220]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T08:32:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:34.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:34 smithi149 ceph-mon[36167]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-09-10T08:32:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:34 smithi111 ceph-mon[35220]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-09-10T08:32:35.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:34 smithi089 ceph-mon[30757]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-09-10T08:32:36.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:36 smithi149 ceph-mon[36167]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-10T08:32:37.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:36 smithi111 ceph-mon[35220]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-10T08:32:37.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:36 smithi089 ceph-mon[30757]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-10T08:32:38.891 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:38 smithi111 ceph-mon[35220]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T08:32:38.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:38 smithi149 ceph-mon[36167]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T08:32:39.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:38 smithi089 ceph-mon[30757]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T08:32:40.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:40 smithi149 ceph-mon[36167]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T08:32:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:40 smithi111 ceph-mon[35220]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T08:32:41.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:40 smithi089 ceph-mon[30757]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T08:32:41.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:41 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T08:32:41.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:41 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:41.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:41 smithi111 ceph-mon[35220]: Deploying daemon osd.4 on smithi111 2024-09-10T08:32:41.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:41 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T08:32:41.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:41 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:41.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:41 smithi149 ceph-mon[36167]: Deploying daemon osd.4 on smithi111 2024-09-10T08:32:42.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T08:32:42.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:42.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:41 smithi089 ceph-mon[30757]: Deploying daemon osd.4 on smithi111 2024-09-10T08:32:42.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:42 smithi111 ceph-mon[35220]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:42.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:42 smithi149 ceph-mon[36167]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:43.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:42 smithi089 ceph-mon[30757]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:44.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:44 smithi149 ceph-mon[36167]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:45.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:44 smithi111 ceph-mon[35220]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:45.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:44 smithi089 ceph-mon[30757]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:46.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:46 smithi149 ceph-mon[36167]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:47.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:46 smithi111 ceph-mon[35220]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:47.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:46 smithi089 ceph-mon[30757]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:47.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:47 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:32:47.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:47 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:47.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:47 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:47.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:47 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:32:47.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:47 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:47.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:47 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:48.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:47 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:32:48.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:47 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:48.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:47 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:48.880 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:48 smithi111 ceph-mon[35220]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:48.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:48 smithi149 ceph-mon[36167]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:49.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:48 smithi089 ceph-mon[30757]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:50.285 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 4 on host 'smithi111' 2024-09-10T08:32:50.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:50.285+0000 7f5c7effd640 1 -- 172.21.15.111:0/1458933464 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5c881a0ab0 con 0x7f5c740a2bb0 2024-09-10T08:32:50.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:50.287+0000 7f5c8fd86640 1 -- 172.21.15.111:0/1458933464 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5c740a2bb0 msgr2=0x7f5c740a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:32:50.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:50.287+0000 7f5c8fd86640 1 --2- 172.21.15.111:0/1458933464 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5c740a2bb0 0x7f5c740a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c84000c00 tx=0x7f5c840315b0 comp rx=0 tx=0).stop 2024-09-10T08:32:50.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:50.287+0000 7f5c8fd86640 1 -- 172.21.15.111:0/1458933464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c88102120 msgr2=0x7f5c8819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:32:50.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:50.287+0000 7f5c8fd86640 1 --2- 172.21.15.111:0/1458933464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c88102120 0x7f5c8819f700 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000b970 tx=0x7f5c8000be30 comp rx=0 tx=0).stop 2024-09-10T08:32:50.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:50.288+0000 7f5c8fd86640 1 -- 172.21.15.111:0/1458933464 shutdown_connections 2024-09-10T08:32:50.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:50.288+0000 7f5c8fd86640 1 --2- 172.21.15.111:0/1458933464 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5c740a2bb0 0x7f5c740a5070 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:50.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:50.288+0000 7f5c8fd86640 1 --2- 172.21.15.111:0/1458933464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c88102120 0x7f5c8819f700 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:50.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:50.288+0000 7f5c8fd86640 1 --2- 172.21.15.111:0/1458933464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c88101740 0x7f5c88198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:50.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:50.288+0000 7f5c8fd86640 1 --2- 172.21.15.111:0/1458933464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c88100d80 0x7f5c88198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:50.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:50.288+0000 7f5c8fd86640 1 -- 172.21.15.111:0/1458933464 >> 172.21.15.111:0/1458933464 conn(0x7f5c880fba40 msgr2=0x7f5c880ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:32:50.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:50.288+0000 7f5c8fd86640 1 -- 172.21.15.111:0/1458933464 shutdown_connections 2024-09-10T08:32:50.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:32:50.288+0000 7f5c8fd86640 1 -- 172.21.15.111:0/1458933464 wait complete. 2024-09-10T08:32:50.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:50.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:50.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:50.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:32:50.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:50.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:50 smithi111 ceph-mon[35220]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:50.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:32:50.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:50.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:50.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:50.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:50.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:50.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:32:50.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:50.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:50 smithi149 ceph-mon[36167]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:50.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:32:50.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:50.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:50.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:50.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:50.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:50.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:32:50.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:50.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:50 smithi089 ceph-mon[30757]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:50.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:32:50.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:50.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:50.875 DEBUG:teuthology.orchestra.run.smithi111:osd.4> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T08:32:50.877 INFO:tasks.cephadm:Deploying osd.5 on smithi149 with /dev/vg_nvme/lv_3... 2024-09-10T08:32:50.877 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T08:32:51.131 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:32:52.358 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:32:52 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-4[46171]: 2024-09-10T08:32:52.271+0000 7f7cff157640 -1 osd.4 0 log_to_monitors true 2024-09-10T08:32:52.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:52 smithi111 ceph-mon[35220]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:52.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:52 smithi111 ceph-mon[35220]: from='osd.4 [v2:172.21.15.111:6808/316717323,v1:172.21.15.111:6809/316717323]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T08:32:52.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:52 smithi111 ceph-mon[35220]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T08:32:52.878 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:52 smithi149 ceph-mon[36167]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:52.878 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:52 smithi149 ceph-mon[36167]: from='osd.4 [v2:172.21.15.111:6808/316717323,v1:172.21.15.111:6809/316717323]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T08:32:52.878 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:52 smithi149 ceph-mon[36167]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T08:32:53.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:52 smithi089 ceph-mon[30757]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:53.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:52 smithi089 ceph-mon[30757]: from='osd.4 [v2:172.21.15.111:6808/316717323,v1:172.21.15.111:6809/316717323]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T08:32:53.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:52 smithi089 ceph-mon[30757]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T08:32:53.530 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:32:53.557 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_3 2024-09-10T08:32:53.704 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:53 smithi111 ceph-mon[35220]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T08:32:53.705 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:53 smithi111 ceph-mon[35220]: from='osd.4 [v2:172.21.15.111:6808/316717323,v1:172.21.15.111:6809/316717323]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:32:53.705 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:53 smithi111 ceph-mon[35220]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T08:32:53.705 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:53 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:53.705 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:53 smithi111 ceph-mon[35220]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:32:53.705 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:53 smithi111 ceph-mon[35220]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-10T08:32:53.705 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:53 smithi111 ceph-mon[35220]: osdmap e32: 5 total, 4 up, 5 in 2024-09-10T08:32:53.810 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:32:53.931 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:53 smithi149 ceph-mon[36167]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T08:32:53.932 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:53 smithi149 ceph-mon[36167]: from='osd.4 [v2:172.21.15.111:6808/316717323,v1:172.21.15.111:6809/316717323]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:32:53.932 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:53 smithi149 ceph-mon[36167]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T08:32:53.932 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:53 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:53.932 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:53 smithi149 ceph-mon[36167]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:32:53.932 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:53 smithi149 ceph-mon[36167]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-10T08:32:53.932 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:53 smithi149 ceph-mon[36167]: osdmap e32: 5 total, 4 up, 5 in 2024-09-10T08:32:54.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:53 smithi089 ceph-mon[30757]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T08:32:54.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:53 smithi089 ceph-mon[30757]: from='osd.4 [v2:172.21.15.111:6808/316717323,v1:172.21.15.111:6809/316717323]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:32:54.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:53 smithi089 ceph-mon[30757]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T08:32:54.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:53 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:54.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:53 smithi089 ceph-mon[30757]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:32:54.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:53 smithi089 ceph-mon[30757]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-10T08:32:54.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:53 smithi089 ceph-mon[30757]: osdmap e32: 5 total, 4 up, 5 in 2024-09-10T08:32:54.262 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.254+0000 7f7fde595640 1 -- 172.21.15.149:0/3870544273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fd8102690 msgr2=0x7f7fd8108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:32:54.262 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.254+0000 7f7fde595640 1 --2- 172.21.15.149:0/3870544273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fd8102690 0x7f7fd8108f20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009ca0 tx=0x7f7fd002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:32:54.262 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.254+0000 7f7fde595640 1 -- 172.21.15.149:0/3870544273 shutdown_connections 2024-09-10T08:32:54.262 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.254+0000 7f7fde595640 1 --2- 172.21.15.149:0/3870544273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fd8102690 0x7f7fd8108f20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:54.262 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.254+0000 7f7fde595640 1 --2- 172.21.15.149:0/3870544273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fd8101cd0 0x7f7fd8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:54.262 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.254+0000 7f7fde595640 1 --2- 172.21.15.149:0/3870544273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fd8100ad0 0x7f7fd8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:54.263 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.254+0000 7f7fde595640 1 -- 172.21.15.149:0/3870544273 >> 172.21.15.149:0/3870544273 conn(0x7f7fd80fc280 msgr2=0x7f7fd80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:32:54.263 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.254+0000 7f7fde595640 1 -- 172.21.15.149:0/3870544273 shutdown_connections 2024-09-10T08:32:54.263 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.254+0000 7f7fde595640 1 -- 172.21.15.149:0/3870544273 wait complete. 2024-09-10T08:32:54.263 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.255+0000 7f7fde595640 1 Processor -- start 2024-09-10T08:32:54.263 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.255+0000 7f7fde595640 1 -- start start 2024-09-10T08:32:54.263 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.255+0000 7f7fde595640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fd8100ad0 0x7f7fd8196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:32:54.263 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.255+0000 7f7fde595640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fd8101cd0 0x7f7fd8197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:32:54.263 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.255+0000 7f7fde595640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fd8102690 0x7f7fd819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:32:54.263 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.255+0000 7f7fde595640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd810c800 con 0x7f7fd8100ad0 2024-09-10T08:32:54.263 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.255+0000 7f7fde595640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd810c680 con 0x7f7fd8102690 2024-09-10T08:32:54.263 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.255+0000 7f7fde595640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd810c980 con 0x7f7fd8101cd0 2024-09-10T08:32:54.263 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.256+0000 7f7fd77fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fd8101cd0 0x7f7fd8197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:32:54.264 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.256+0000 7f7fd77fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fd8101cd0 0x7f7fd8197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:38098/0 (socket says 172.21.15.149:38098) 2024-09-10T08:32:54.264 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.256+0000 7f7fd77fe640 1 -- 172.21.15.149:0/961824348 learned_addr learned my addr 172.21.15.149:0/961824348 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-09-10T08:32:54.264 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.256+0000 7f7fdcb32640 1 --2- 172.21.15.149:0/961824348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fd8102690 0x7f7fd819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:32:54.264 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.256+0000 7f7fd77fe640 1 -- 172.21.15.149:0/961824348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fd8102690 msgr2=0x7f7fd819e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:32:54.264 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.256+0000 7f7fd77fe640 1 --2- 172.21.15.149:0/961824348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fd8102690 0x7f7fd819e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:54.264 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.256+0000 7f7fd77fe640 1 -- 172.21.15.149:0/961824348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fd8100ad0 msgr2=0x7f7fd8196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:32:54.264 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.256+0000 7f7fd77fe640 1 --2- 172.21.15.149:0/961824348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fd8100ad0 0x7f7fd8196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:32:54.264 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.256+0000 7f7fd77fe640 1 -- 172.21.15.149:0/961824348 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd819ea50 con 0x7f7fd8101cd0 2024-09-10T08:32:54.264 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.256+0000 7f7fd77fe640 1 --2- 172.21.15.149:0/961824348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fd8101cd0 0x7f7fd8197360 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00e9f0 tx=0x7f7fcc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:32:54.264 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.256+0000 7f7fd57fa640 1 -- 172.21.15.149:0/961824348 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc00cd80 con 0x7f7fd8101cd0 2024-09-10T08:32:54.264 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.256+0000 7f7fde595640 1 -- 172.21.15.149:0/961824348 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd819ed40 con 0x7f7fd8101cd0 2024-09-10T08:32:54.265 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.256+0000 7f7fd57fa640 1 -- 172.21.15.149:0/961824348 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fcc005500 con 0x7f7fd8101cd0 2024-09-10T08:32:54.265 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.256+0000 7f7fd57fa640 1 -- 172.21.15.149:0/961824348 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc010690 con 0x7f7fd8101cd0 2024-09-10T08:32:54.265 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.256+0000 7f7fde595640 1 -- 172.21.15.149:0/961824348 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd819f2a0 con 0x7f7fd8101cd0 2024-09-10T08:32:54.265 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.257+0000 7f7fde595640 1 -- 172.21.15.149:0/961824348 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd8107e80 con 0x7f7fd8101cd0 2024-09-10T08:32:54.265 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.258+0000 7f7fd57fa640 1 -- 172.21.15.149:0/961824348 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7fcc0046c0 con 0x7f7fd8101cd0 2024-09-10T08:32:54.265 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.259+0000 7f7fd57fa640 1 --2- 172.21.15.149:0/961824348 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7fc00a2c30 0x7f7fc00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:32:54.265 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.259+0000 7f7fd57fa640 1 -- 172.21.15.149:0/961824348 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 3821+0+0 (secure 0 0 0) 0x7f7fcc0ce2a0 con 0x7f7fd8101cd0 2024-09-10T08:32:54.265 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.260+0000 7f7fd7fff640 1 --2- 172.21.15.149:0/961824348 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7fc00a2c30 0x7f7fc00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:32:54.265 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.260+0000 7f7fd7fff640 1 --2- 172.21.15.149:0/961824348 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7fc00a2c30 0x7f7fc00a50f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8101b30 tx=0x7f7fc8006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:32:54.265 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.264+0000 7f7fd57fa640 1 -- 172.21.15.149:0/961824348 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7fcc09c390 con 0x7f7fd8101cd0 2024-09-10T08:32:54.293 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:32:53 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-4[46171]: 2024-09-10T08:32:53.968+0000 7f7cfb0da640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T08:32:54.293 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:32:53 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-4[46171]: 2024-09-10T08:32:53.975+0000 7f7cf2efa640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:32:54.441 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:32:54.440+0000 7f7fde595640 1 -- 172.21.15.149:0/961824348 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f7fd8062630 con 0x7f7fc00a2c30 2024-09-10T08:32:54.852 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:54 smithi149 ceph-mon[36167]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:54.852 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:54.852 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:54.852 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:54.852 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:54.852 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:32:54.852 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:32:54.852 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:54.852 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:54.853 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:32:54.853 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:54.853 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:54 smithi149 ceph-mon[36167]: from='osd.4 ' entity='osd.4' 2024-09-10T08:32:54.853 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:32:54.853 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:32:54.853 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:54.857 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:54 smithi089 ceph-mon[30757]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:54.857 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:54.857 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:54.857 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:54.857 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:54.857 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:32:54.857 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:32:54.857 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:54.857 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:54.857 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:32:54.857 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:54.858 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:54 smithi089 ceph-mon[30757]: from='osd.4 ' entity='osd.4' 2024-09-10T08:32:54.858 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:32:54.858 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:32:54.858 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:55.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:54 smithi111 ceph-mon[35220]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:55.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:55.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:55.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:55.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:55.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:32:55.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:32:55.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:55.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:55.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:32:55.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:32:55.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:54 smithi111 ceph-mon[35220]: from='osd.4 ' entity='osd.4' 2024-09-10T08:32:55.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:32:55.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:32:55.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:32:55.844 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:55 smithi149 ceph-mon[36167]: purged_snaps scrub starts 2024-09-10T08:32:55.844 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:55 smithi149 ceph-mon[36167]: purged_snaps scrub ok 2024-09-10T08:32:55.844 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:55 smithi149 ceph-mon[36167]: Detected new or changed devices on smithi111 2024-09-10T08:32:55.844 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:55 smithi149 ceph-mon[36167]: Adjusting osd_memory_target on smithi111 to 8544M 2024-09-10T08:32:55.844 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:55 smithi149 ceph-mon[36167]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:32:55.844 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:55 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:55.844 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:55 smithi149 ceph-mon[36167]: osd.4 [v2:172.21.15.111:6808/316717323,v1:172.21.15.111:6809/316717323] boot 2024-09-10T08:32:55.845 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:55 smithi149 ceph-mon[36167]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T08:32:55.845 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:55 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:56.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:55 smithi111 ceph-mon[35220]: purged_snaps scrub starts 2024-09-10T08:32:56.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:55 smithi111 ceph-mon[35220]: purged_snaps scrub ok 2024-09-10T08:32:56.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:55 smithi111 ceph-mon[35220]: Detected new or changed devices on smithi111 2024-09-10T08:32:56.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:55 smithi111 ceph-mon[35220]: Adjusting osd_memory_target on smithi111 to 8544M 2024-09-10T08:32:56.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:55 smithi111 ceph-mon[35220]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:32:56.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:55 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:56.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:55 smithi111 ceph-mon[35220]: osd.4 [v2:172.21.15.111:6808/316717323,v1:172.21.15.111:6809/316717323] boot 2024-09-10T08:32:56.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:55 smithi111 ceph-mon[35220]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T08:32:56.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:55 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:56.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:55 smithi089 ceph-mon[30757]: purged_snaps scrub starts 2024-09-10T08:32:56.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:55 smithi089 ceph-mon[30757]: purged_snaps scrub ok 2024-09-10T08:32:56.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:55 smithi089 ceph-mon[30757]: Detected new or changed devices on smithi111 2024-09-10T08:32:56.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:55 smithi089 ceph-mon[30757]: Adjusting osd_memory_target on smithi111 to 8544M 2024-09-10T08:32:56.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:55 smithi089 ceph-mon[30757]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:32:56.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:55 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:56.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:55 smithi089 ceph-mon[30757]: osd.4 [v2:172.21.15.111:6808/316717323,v1:172.21.15.111:6809/316717323] boot 2024-09-10T08:32:56.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:55 smithi089 ceph-mon[30757]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T08:32:56.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:55 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:32:56.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:56 smithi111 ceph-mon[35220]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:56.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:56 smithi111 ceph-mon[35220]: osdmap e34: 5 total, 5 up, 5 in 2024-09-10T08:32:56.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:56 smithi149 ceph-mon[36167]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:56.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:56 smithi149 ceph-mon[36167]: osdmap e34: 5 total, 5 up, 5 in 2024-09-10T08:32:57.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:56 smithi089 ceph-mon[30757]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:32:57.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:56 smithi089 ceph-mon[30757]: osdmap e34: 5 total, 5 up, 5 in 2024-09-10T08:32:57.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:57 smithi149 ceph-mon[36167]: from='client.? 172.21.15.149:0/1591553853' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e516e8d-e7c6-4ebc-80a5-29932161df4c"}]: dispatch 2024-09-10T08:32:57.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:57 smithi149 ceph-mon[36167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e516e8d-e7c6-4ebc-80a5-29932161df4c"}]: dispatch 2024-09-10T08:32:57.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:57 smithi149 ceph-mon[36167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e516e8d-e7c6-4ebc-80a5-29932161df4c"}]': finished 2024-09-10T08:32:57.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:57 smithi149 ceph-mon[36167]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T08:32:57.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:57 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:32:57.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:57 smithi149 ceph-mon[36167]: from='client.? 172.21.15.149:0/237755087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:32:58.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:57 smithi111 ceph-mon[35220]: from='client.? 172.21.15.149:0/1591553853' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e516e8d-e7c6-4ebc-80a5-29932161df4c"}]: dispatch 2024-09-10T08:32:58.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:57 smithi111 ceph-mon[35220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e516e8d-e7c6-4ebc-80a5-29932161df4c"}]: dispatch 2024-09-10T08:32:58.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:57 smithi111 ceph-mon[35220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e516e8d-e7c6-4ebc-80a5-29932161df4c"}]': finished 2024-09-10T08:32:58.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:57 smithi111 ceph-mon[35220]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T08:32:58.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:57 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:32:58.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:57 smithi111 ceph-mon[35220]: from='client.? 172.21.15.149:0/237755087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:32:58.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:57 smithi089 ceph-mon[30757]: from='client.? 172.21.15.149:0/1591553853' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e516e8d-e7c6-4ebc-80a5-29932161df4c"}]: dispatch 2024-09-10T08:32:58.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:57 smithi089 ceph-mon[30757]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e516e8d-e7c6-4ebc-80a5-29932161df4c"}]: dispatch 2024-09-10T08:32:58.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:57 smithi089 ceph-mon[30757]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e516e8d-e7c6-4ebc-80a5-29932161df4c"}]': finished 2024-09-10T08:32:58.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:57 smithi089 ceph-mon[30757]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T08:32:58.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:57 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:32:58.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:57 smithi089 ceph-mon[30757]: from='client.? 172.21.15.149:0/237755087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:32:58.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:32:58 smithi149 ceph-mon[36167]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:32:59.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:32:58 smithi111 ceph-mon[35220]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:32:59.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:32:58 smithi089 ceph-mon[30757]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:00.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:00 smithi149 ceph-mon[36167]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:01.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:00 smithi111 ceph-mon[35220]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:01.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:00 smithi089 ceph-mon[30757]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:02.855 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:02 smithi089 ceph-mon[30757]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:02.855 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:02 smithi149 ceph-mon[36167]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:02 smithi111 ceph-mon[35220]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:04.840 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:04 smithi149 ceph-mon[36167]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:04 smithi111 ceph-mon[35220]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:05.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:04 smithi089 ceph-mon[30757]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:05.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:05 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T08:33:05.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:05 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:06.035 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:05 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T08:33:06.035 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:05 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:06.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:05 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T08:33:06.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:05 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:06.773 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:06 smithi149 ceph-mon[36167]: Deploying daemon osd.5 on smithi149 2024-09-10T08:33:06.774 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:06 smithi149 ceph-mon[36167]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:07.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:06 smithi111 ceph-mon[35220]: Deploying daemon osd.5 on smithi149 2024-09-10T08:33:07.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:06 smithi111 ceph-mon[35220]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:07.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:06 smithi089 ceph-mon[30757]: Deploying daemon osd.5 on smithi149 2024-09-10T08:33:07.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:06 smithi089 ceph-mon[30757]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:08.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:08 smithi149 ceph-mon[36167]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:09.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:08 smithi111 ceph-mon[35220]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:09.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:08 smithi089 ceph-mon[30757]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:10.824 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:10 smithi149 ceph-mon[36167]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:11.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:10 smithi111 ceph-mon[35220]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:11.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:10 smithi089 ceph-mon[30757]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:11.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:11 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:33:11.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:11 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:11.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:11 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:12.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:11 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:33:12.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:11 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:12.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:11 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:12.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:33:12.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:12.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:11 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:12.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:12 smithi149 ceph-mon[36167]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:13.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:12 smithi111 ceph-mon[35220]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:13.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:12 smithi089 ceph-mon[30757]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:14.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:14 smithi149 ceph-mon[36167]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:14.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:14 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:14.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:14 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:14.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:14 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:14.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:14 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:33:14.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:14 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:15.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:14 smithi111 ceph-mon[35220]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:15.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:14 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:15.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:14 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:15.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:14 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:15.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:14 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:33:15.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:14 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:15.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:14 smithi089 ceph-mon[30757]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:15.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:14 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:15.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:14 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:15.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:14 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:15.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:14 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:33:15.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:14 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:15.135 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:33:15.135+0000 7f7fd57fa640 1 -- 172.21.15.149:0/961824348 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7fd8062630 con 0x7f7fc00a2c30 2024-09-10T08:33:15.136 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 5 on host 'smithi149' 2024-09-10T08:33:15.138 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:33:15.138+0000 7f7fde595640 1 -- 172.21.15.149:0/961824348 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7fc00a2c30 msgr2=0x7f7fc00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:33:15.138 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:33:15.138+0000 7f7fde595640 1 --2- 172.21.15.149:0/961824348 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7fc00a2c30 0x7f7fc00a50f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8101b30 tx=0x7f7fc8006c90 comp rx=0 tx=0).stop 2024-09-10T08:33:15.138 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:33:15.138+0000 7f7fde595640 1 -- 172.21.15.149:0/961824348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fd8101cd0 msgr2=0x7f7fd8197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:33:15.138 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:33:15.138+0000 7f7fde595640 1 --2- 172.21.15.149:0/961824348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fd8101cd0 0x7f7fd8197360 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00e9f0 tx=0x7f7fcc00edf0 comp rx=0 tx=0).stop 2024-09-10T08:33:15.138 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:33:15.138+0000 7f7fde595640 1 -- 172.21.15.149:0/961824348 shutdown_connections 2024-09-10T08:33:15.138 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:33:15.138+0000 7f7fde595640 1 --2- 172.21.15.149:0/961824348 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7fc00a2c30 0x7f7fc00a50f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:33:15.138 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:33:15.138+0000 7f7fde595640 1 --2- 172.21.15.149:0/961824348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fd8102690 0x7f7fd819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:33:15.138 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:33:15.138+0000 7f7fde595640 1 --2- 172.21.15.149:0/961824348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fd8101cd0 0x7f7fd8197360 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:33:15.138 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:33:15.138+0000 7f7fde595640 1 --2- 172.21.15.149:0/961824348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fd8100ad0 0x7f7fd8196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:33:15.139 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:33:15.138+0000 7f7fde595640 1 -- 172.21.15.149:0/961824348 >> 172.21.15.149:0/961824348 conn(0x7f7fd80fc280 msgr2=0x7f7fd8105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:33:15.139 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:33:15.138+0000 7f7fde595640 1 -- 172.21.15.149:0/961824348 shutdown_connections 2024-09-10T08:33:15.139 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:33:15.138+0000 7f7fde595640 1 -- 172.21.15.149:0/961824348 wait complete. 2024-09-10T08:33:15.771 DEBUG:teuthology.orchestra.run.smithi149:osd.5> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T08:33:15.774 INFO:tasks.cephadm:Deploying osd.6 on smithi089 with /dev/vg_nvme/lv_2... 2024-09-10T08:33:15.774 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T08:33:15.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:15 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:33:15.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:15 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:15.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:15 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:16.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:15 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:33:16.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:15 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:16.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:15 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:16.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:15 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:33:16.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:15 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:16.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:15 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:16.143 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:33:16.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:16 smithi149 ceph-mon[36167]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:17.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:16 smithi111 ceph-mon[35220]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:17.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:16 smithi089 ceph-mon[30757]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:17.708 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:33:17 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:33:17.335+0000 7fd367d3d640 -1 osd.5 0 log_to_monitors true 2024-09-10T08:33:17.736 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:17 smithi089 ceph-mon[30757]: from='osd.5 [v2:172.21.15.149:6808/1256148461,v1:172.21.15.149:6809/1256148461]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T08:33:17.737 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:17 smithi089 ceph-mon[30757]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T08:33:17.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:17 smithi149 ceph-mon[36167]: from='osd.5 [v2:172.21.15.149:6808/1256148461,v1:172.21.15.149:6809/1256148461]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T08:33:17.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:17 smithi149 ceph-mon[36167]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T08:33:18.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:17 smithi111 ceph-mon[35220]: from='osd.5 [v2:172.21.15.149:6808/1256148461,v1:172.21.15.149:6809/1256148461]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T08:33:18.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:17 smithi111 ceph-mon[35220]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T08:33:18.970 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:33:18 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:33:18.727+0000 7fd363cc0640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T08:33:18.970 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:33:18 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:33:18.733+0000 7fd35fae8640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:33:18.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:18 smithi149 ceph-mon[36167]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:18.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:18 smithi149 ceph-mon[36167]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T08:33:18.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:18 smithi149 ceph-mon[36167]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T08:33:18.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:18 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:33:18.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:18 smithi149 ceph-mon[36167]: from='osd.5 [v2:172.21.15.149:6808/1256148461,v1:172.21.15.149:6809/1256148461]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:33:18.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:18 smithi149 ceph-mon[36167]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:33:18.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:18 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:18.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:18 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:18.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:18 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:33:18.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:18 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:33:18.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:18 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:18.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:18 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:18.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:18 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:33:18.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:18 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:18.974 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:18 smithi089 ceph-mon[30757]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:18.974 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:18 smithi089 ceph-mon[30757]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T08:33:18.975 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:18 smithi089 ceph-mon[30757]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T08:33:18.975 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:18 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:33:18.975 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:18 smithi089 ceph-mon[30757]: from='osd.5 [v2:172.21.15.149:6808/1256148461,v1:172.21.15.149:6809/1256148461]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:33:18.975 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:18 smithi089 ceph-mon[30757]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:33:18.975 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:18 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:18.975 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:18 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:18.975 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:18 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:33:18.975 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:18 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:33:18.975 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:18 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:18.976 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:18 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:18.976 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:18 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:33:18.976 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:18 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:19.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:18 smithi111 ceph-mon[35220]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:19.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:18 smithi111 ceph-mon[35220]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T08:33:19.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:18 smithi111 ceph-mon[35220]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T08:33:19.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:18 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:33:19.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:18 smithi111 ceph-mon[35220]: from='osd.5 [v2:172.21.15.149:6808/1256148461,v1:172.21.15.149:6809/1256148461]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:33:19.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:18 smithi111 ceph-mon[35220]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:33:19.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:18 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:19.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:18 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:19.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:18 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:33:19.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:18 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:33:19.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:18 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:19.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:18 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:19.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:18 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:33:19.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:18 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:19.309 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:33:19.339 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi089:vg_nvme/lv_2 2024-09-10T08:33:19.699 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:33:19.998 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:19 smithi089 ceph-mon[30757]: Detected new or changed devices on smithi149 2024-09-10T08:33:19.998 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:19 smithi089 ceph-mon[30757]: Adjusting osd_memory_target on smithi149 to 2907M 2024-09-10T08:33:19.998 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:19 smithi089 ceph-mon[30757]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T08:33:19.998 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:19 smithi089 ceph-mon[30757]: osdmap e37: 6 total, 5 up, 6 in 2024-09-10T08:33:19.998 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:19 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:33:19.998 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:19 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:33:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:19 smithi111 ceph-mon[35220]: Detected new or changed devices on smithi149 2024-09-10T08:33:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:19 smithi111 ceph-mon[35220]: Adjusting osd_memory_target on smithi149 to 2907M 2024-09-10T08:33:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:19 smithi111 ceph-mon[35220]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T08:33:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:19 smithi111 ceph-mon[35220]: osdmap e37: 6 total, 5 up, 6 in 2024-09-10T08:33:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:19 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:33:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:19 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:33:20.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:19 smithi149 ceph-mon[36167]: Detected new or changed devices on smithi149 2024-09-10T08:33:20.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:19 smithi149 ceph-mon[36167]: Adjusting osd_memory_target on smithi149 to 2907M 2024-09-10T08:33:20.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:19 smithi149 ceph-mon[36167]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T08:33:20.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:19 smithi149 ceph-mon[36167]: osdmap e37: 6 total, 5 up, 6 in 2024-09-10T08:33:20.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:19 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:33:20.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:19 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:33:20.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.267+0000 7fdbbb200640 1 -- 172.21.15.89:0/616688680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbbc104e50 msgr2=0x7fdbbc1052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:33:20.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.267+0000 7fdbbb200640 1 --2- 172.21.15.89:0/616688680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbbc104e50 0x7fdbbc1052d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009900 tx=0x7fdba802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:33:20.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.267+0000 7fdbbb200640 1 -- 172.21.15.89:0/616688680 shutdown_connections 2024-09-10T08:33:20.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.267+0000 7fdbbb200640 1 --2- 172.21.15.89:0/616688680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbbc105810 0x7fdbbc10c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:33:20.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.267+0000 7fdbbb200640 1 --2- 172.21.15.89:0/616688680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbbc104e50 0x7fdbbc1052d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:33:20.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.267+0000 7fdbbb200640 1 --2- 172.21.15.89:0/616688680 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbbc103c50 0x7fdbbc104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:33:20.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.267+0000 7fdbbb200640 1 -- 172.21.15.89:0/616688680 >> 172.21.15.89:0/616688680 conn(0x7fdbbc0ff3e0 msgr2=0x7fdbbc101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:33:20.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.267+0000 7fdbbb200640 1 -- 172.21.15.89:0/616688680 shutdown_connections 2024-09-10T08:33:20.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.267+0000 7fdbbb200640 1 -- 172.21.15.89:0/616688680 wait complete. 2024-09-10T08:33:20.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.268+0000 7fdbbb200640 1 Processor -- start 2024-09-10T08:33:20.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.268+0000 7fdbbb200640 1 -- start start 2024-09-10T08:33:20.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.268+0000 7fdbbb200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbbc103c50 0x7fdbbc199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:33:20.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.268+0000 7fdbbb200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbbc104e50 0x7fdbbc19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:33:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.268+0000 7fdbbb200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbbc105810 0x7fdbbc1a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:33:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.268+0000 7fdbbb200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc10f920 con 0x7fdbbc105810 2024-09-10T08:33:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.268+0000 7fdbbb200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc10f7a0 con 0x7fdbbc103c50 2024-09-10T08:33:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.268+0000 7fdbbb200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc10faa0 con 0x7fdbbc104e50 2024-09-10T08:33:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.268+0000 7fdbb9400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbbc104e50 0x7fdbbc19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:33:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.268+0000 7fdbba800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbbc105810 0x7fdbbc1a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:33:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.269+0000 7fdbb9400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbbc104e50 0x7fdbbc19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:38038/0 (socket says 172.21.15.89:38038) 2024-09-10T08:33:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.269+0000 7fdbb9400640 1 -- 172.21.15.89:0/2077482421 learned_addr learned my addr 172.21.15.89:0/2077482421 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:33:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.269+0000 7fdbb9e00640 1 --2- 172.21.15.89:0/2077482421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbbc103c50 0x7fdbbc199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:33:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.269+0000 7fdbb9400640 1 -- 172.21.15.89:0/2077482421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbbc103c50 msgr2=0x7fdbbc199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:33:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.269+0000 7fdbb9400640 1 --2- 172.21.15.89:0/2077482421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbbc103c50 0x7fdbbc199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:33:20.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.269+0000 7fdbb9400640 1 -- 172.21.15.89:0/2077482421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbbc105810 msgr2=0x7fdbbc1a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:33:20.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.269+0000 7fdbb9400640 1 --2- 172.21.15.89:0/2077482421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbbc105810 0x7fdbbc1a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:33:20.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.269+0000 7fdbb9400640 1 -- 172.21.15.89:0/2077482421 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbbc1a1b50 con 0x7fdbbc104e50 2024-09-10T08:33:20.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.269+0000 7fdbb9400640 1 --2- 172.21.15.89:0/2077482421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbbc104e50 0x7fdbbc19a460 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdba8031440 tx=0x7fdba8032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:33:20.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.270+0000 7fdbb2a00640 1 -- 172.21.15.89:0/2077482421 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba803f030 con 0x7fdbbc104e50 2024-09-10T08:33:20.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.270+0000 7fdbbb200640 1 -- 172.21.15.89:0/2077482421 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbbc1a1de0 con 0x7fdbbc104e50 2024-09-10T08:33:20.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.270+0000 7fdbb2a00640 1 -- 172.21.15.89:0/2077482421 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba8003ce0 con 0x7fdbbc104e50 2024-09-10T08:33:20.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.270+0000 7fdbb2a00640 1 -- 172.21.15.89:0/2077482421 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8035070 con 0x7fdbbc104e50 2024-09-10T08:33:20.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.270+0000 7fdbbb200640 1 -- 172.21.15.89:0/2077482421 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbbc1a2240 con 0x7fdbbc104e50 2024-09-10T08:33:20.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.271+0000 7fdbbb200640 1 -- 172.21.15.89:0/2077482421 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbbc10b020 con 0x7fdbbc104e50 2024-09-10T08:33:20.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.273+0000 7fdbb2a00640 1 -- 172.21.15.89:0/2077482421 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdba8042030 con 0x7fdbbc104e50 2024-09-10T08:33:20.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.274+0000 7fdbb2a00640 1 --2- 172.21.15.89:0/2077482421 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdba00a2bb0 0x7fdba00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:33:20.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.274+0000 7fdbb2a00640 1 -- 172.21.15.89:0/2077482421 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4566+0+0 (secure 0 0 0) 0x7fdba80ee7a0 con 0x7fdbbc104e50 2024-09-10T08:33:20.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.274+0000 7fdbb9e00640 1 --2- 172.21.15.89:0/2077482421 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdba00a2bb0 0x7fdba00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:33:20.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.275+0000 7fdbb9e00640 1 --2- 172.21.15.89:0/2077482421 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdba00a2bb0 0x7fdba00a5070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdbac005fd0 tx=0x7fdbac00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:33:20.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.276+0000 7fdbb2a00640 1 -- 172.21.15.89:0/2077482421 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdba80bbeb0 con 0x7fdbbc104e50 2024-09-10T08:33:20.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:20.459+0000 7fdbbb200640 1 -- 172.21.15.89:0/2077482421 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdbbc062630 con 0x7fdba00a2bb0 2024-09-10T08:33:21.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:20 smithi111 ceph-mon[35220]: purged_snaps scrub starts 2024-09-10T08:33:21.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:20 smithi111 ceph-mon[35220]: purged_snaps scrub ok 2024-09-10T08:33:21.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:20 smithi111 ceph-mon[35220]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:21.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:20 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:33:21.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:20 smithi111 ceph-mon[35220]: osd.5 [v2:172.21.15.149:6808/1256148461,v1:172.21.15.149:6809/1256148461] boot 2024-09-10T08:33:21.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:20 smithi111 ceph-mon[35220]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T08:33:21.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:20 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:33:21.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:20 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:33:21.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:20 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:33:21.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:20 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:21.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:20 smithi111 ceph-mon[35220]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T08:33:21.049 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:20 smithi089 ceph-mon[30757]: purged_snaps scrub starts 2024-09-10T08:33:21.049 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:20 smithi089 ceph-mon[30757]: purged_snaps scrub ok 2024-09-10T08:33:21.049 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:20 smithi089 ceph-mon[30757]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:21.049 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:20 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:33:21.049 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:20 smithi089 ceph-mon[30757]: osd.5 [v2:172.21.15.149:6808/1256148461,v1:172.21.15.149:6809/1256148461] boot 2024-09-10T08:33:21.049 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:20 smithi089 ceph-mon[30757]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T08:33:21.050 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:20 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:33:21.050 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:20 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:33:21.050 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:20 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:33:21.050 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:20 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:21.050 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:20 smithi089 ceph-mon[30757]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T08:33:21.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:20 smithi149 ceph-mon[36167]: purged_snaps scrub starts 2024-09-10T08:33:21.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:20 smithi149 ceph-mon[36167]: purged_snaps scrub ok 2024-09-10T08:33:21.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:20 smithi149 ceph-mon[36167]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:33:21.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:20 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:33:21.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:20 smithi149 ceph-mon[36167]: osd.5 [v2:172.21.15.149:6808/1256148461,v1:172.21.15.149:6809/1256148461] boot 2024-09-10T08:33:21.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:20 smithi149 ceph-mon[36167]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T08:33:21.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:20 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:33:21.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:20 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:33:21.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:20 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:33:21.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:20 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:21.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:20 smithi149 ceph-mon[36167]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T08:33:22.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:21 smithi111 ceph-mon[35220]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:33:22.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:21 smithi111 ceph-mon[35220]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T08:33:22.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:21 smithi111 ceph-mon[35220]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:22.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:21 smithi089 ceph-mon[30757]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:33:22.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:21 smithi089 ceph-mon[30757]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T08:33:22.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:21 smithi089 ceph-mon[30757]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:22.121 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:21 smithi149 ceph-mon[36167]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:33:22.122 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:21 smithi149 ceph-mon[36167]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T08:33:22.122 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:21 smithi149 ceph-mon[36167]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:24.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:23 smithi111 ceph-mon[35220]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T08:33:24.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:23 smithi111 ceph-mon[35220]: pgmap v142: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-10T08:33:24.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:23 smithi089 ceph-mon[30757]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T08:33:24.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:23 smithi089 ceph-mon[30757]: pgmap v142: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-10T08:33:24.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:23 smithi149 ceph-mon[36167]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T08:33:24.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:23 smithi149 ceph-mon[36167]: pgmap v142: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-10T08:33:25.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:24 smithi111 ceph-mon[35220]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-10T08:33:25.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:24 smithi111 ceph-mon[35220]: osdmap e42: 6 total, 6 up, 6 in 2024-09-10T08:33:25.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:24 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3534840942' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48f92472-f657-4c5a-adeb-20d4fea1ce8a"}]: dispatch 2024-09-10T08:33:25.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:24 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3534840942' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48f92472-f657-4c5a-adeb-20d4fea1ce8a"}]': finished 2024-09-10T08:33:25.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:24 smithi111 ceph-mon[35220]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T08:33:25.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:24 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:33:25.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:24 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2961938646' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:33:25.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:24 smithi089 ceph-mon[30757]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-10T08:33:25.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:24 smithi089 ceph-mon[30757]: osdmap e42: 6 total, 6 up, 6 in 2024-09-10T08:33:25.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:24 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3534840942' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48f92472-f657-4c5a-adeb-20d4fea1ce8a"}]: dispatch 2024-09-10T08:33:25.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:24 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3534840942' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48f92472-f657-4c5a-adeb-20d4fea1ce8a"}]': finished 2024-09-10T08:33:25.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:24 smithi089 ceph-mon[30757]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T08:33:25.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:24 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:33:25.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:24 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2961938646' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:33:25.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:24 smithi149 ceph-mon[36167]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-10T08:33:25.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:24 smithi149 ceph-mon[36167]: osdmap e42: 6 total, 6 up, 6 in 2024-09-10T08:33:25.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:24 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3534840942' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48f92472-f657-4c5a-adeb-20d4fea1ce8a"}]: dispatch 2024-09-10T08:33:25.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:24 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3534840942' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48f92472-f657-4c5a-adeb-20d4fea1ce8a"}]': finished 2024-09-10T08:33:25.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:24 smithi149 ceph-mon[36167]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T08:33:25.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:24 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:33:25.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:24 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2961938646' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:33:26.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:25 smithi111 ceph-mon[35220]: pgmap v145: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-10T08:33:26.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:25 smithi089 ceph-mon[30757]: pgmap v145: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-10T08:33:26.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:25 smithi149 ceph-mon[36167]: pgmap v145: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-09-10T08:33:27.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:27 smithi149 ceph-mon[36167]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-10T08:33:27.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:27 smithi149 ceph-mon[36167]: Cluster is now healthy 2024-09-10T08:33:28.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:27 smithi111 ceph-mon[35220]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-10T08:33:28.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:27 smithi111 ceph-mon[35220]: Cluster is now healthy 2024-09-10T08:33:28.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:27 smithi089 ceph-mon[30757]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-10T08:33:28.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:27 smithi089 ceph-mon[30757]: Cluster is now healthy 2024-09-10T08:33:28.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:28 smithi149 ceph-mon[36167]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T08:33:29.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:28 smithi111 ceph-mon[35220]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T08:33:29.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:28 smithi089 ceph-mon[30757]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T08:33:30.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:30 smithi089 ceph-mon[30757]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T08:33:30.972 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:30 smithi149 ceph-mon[36167]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T08:33:31.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:30 smithi111 ceph-mon[35220]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T08:33:32.945 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:32 smithi089 ceph-mon[30757]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T08:33:32.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:32 smithi149 ceph-mon[36167]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T08:33:33.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:32 smithi111 ceph-mon[35220]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T08:33:33.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:33 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T08:33:33.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:33 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:33.955 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:33 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T08:33:33.955 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:33 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T08:33:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:34.940 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:34 smithi089 ceph-mon[30757]: Deploying daemon osd.6 on smithi089 2024-09-10T08:33:34.941 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:34 smithi089 ceph-mon[30757]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T08:33:34.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:34 smithi149 ceph-mon[36167]: Deploying daemon osd.6 on smithi089 2024-09-10T08:33:34.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:34 smithi149 ceph-mon[36167]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T08:33:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:34 smithi111 ceph-mon[35220]: Deploying daemon osd.6 on smithi089 2024-09-10T08:33:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:34 smithi111 ceph-mon[35220]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T08:33:36.777 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:36 smithi089 ceph-mon[30757]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T08:33:36.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:36 smithi149 ceph-mon[36167]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T08:33:37.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:36 smithi111 ceph-mon[35220]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T08:33:38.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:38 smithi149 ceph-mon[36167]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:33:39.008 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:38 smithi089 ceph-mon[30757]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:33:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:38 smithi111 ceph-mon[35220]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:33:40.660 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:40 smithi089 ceph-mon[30757]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:40.975 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:40 smithi149 ceph-mon[36167]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:40 smithi111 ceph-mon[35220]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:41.749 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:33:41.749 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:41.749 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:41.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:41 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:33:41.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:41 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:41.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:41 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:42.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:41 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:33:42.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:41 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:42.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:41 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:42.900 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:42 smithi089 ceph-mon[30757]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:42.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:42 smithi149 ceph-mon[36167]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:43.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:42 smithi111 ceph-mon[35220]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:44.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:44 smithi089 ceph-mon[30757]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:44 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:44 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:44 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:44 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:33:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:44 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:44.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:44 smithi149 ceph-mon[36167]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:44.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:44 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:44.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:44 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:44.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:44 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:44.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:44 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:33:44.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:44 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:45.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:44 smithi111 ceph-mon[35220]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:45.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:44 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:45.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:44 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:45.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:44 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:45.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:44 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:33:45.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:44 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:45.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:45.246+0000 7fdbb2a00640 1 -- 172.21.15.89:0/2077482421 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdbbc062630 con 0x7fdba00a2bb0 2024-09-10T08:33:45.247 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 6 on host 'smithi089' 2024-09-10T08:33:45.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:45.249+0000 7fdbbb200640 1 -- 172.21.15.89:0/2077482421 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdba00a2bb0 msgr2=0x7fdba00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:33:45.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:45.249+0000 7fdbbb200640 1 --2- 172.21.15.89:0/2077482421 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdba00a2bb0 0x7fdba00a5070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdbac005fd0 tx=0x7fdbac00b410 comp rx=0 tx=0).stop 2024-09-10T08:33:45.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:45.249+0000 7fdbbb200640 1 -- 172.21.15.89:0/2077482421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbbc104e50 msgr2=0x7fdbbc19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:33:45.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:45.249+0000 7fdbbb200640 1 --2- 172.21.15.89:0/2077482421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbbc104e50 0x7fdbbc19a460 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdba8031440 tx=0x7fdba8032ce0 comp rx=0 tx=0).stop 2024-09-10T08:33:45.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:45.249+0000 7fdbbb200640 1 -- 172.21.15.89:0/2077482421 shutdown_connections 2024-09-10T08:33:45.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:45.249+0000 7fdbbb200640 1 --2- 172.21.15.89:0/2077482421 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdba00a2bb0 0x7fdba00a5070 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:33:45.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:45.249+0000 7fdbbb200640 1 --2- 172.21.15.89:0/2077482421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbbc105810 0x7fdbbc1a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:33:45.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:45.249+0000 7fdbbb200640 1 --2- 172.21.15.89:0/2077482421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbbc104e50 0x7fdbbc19a460 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:33:45.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:45.249+0000 7fdbbb200640 1 --2- 172.21.15.89:0/2077482421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbbc103c50 0x7fdbbc199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:33:45.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:45.249+0000 7fdbbb200640 1 -- 172.21.15.89:0/2077482421 >> 172.21.15.89:0/2077482421 conn(0x7fdbbc0ff3e0 msgr2=0x7fdbbc1081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:33:45.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:45.249+0000 7fdbbb200640 1 -- 172.21.15.89:0/2077482421 shutdown_connections 2024-09-10T08:33:45.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:33:45.250+0000 7fdbbb200640 1 -- 172.21.15.89:0/2077482421 wait complete. 2024-09-10T08:33:45.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:45 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:33:45.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:45 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:45.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:45 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:46.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:45 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:33:46.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:45 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:46.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:45 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:46.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:45 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:33:46.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:45 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:46.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:45 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:46.159 DEBUG:teuthology.orchestra.run.smithi089:osd.6> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T08:33:46.163 INFO:tasks.cephadm:Deploying osd.7 on smithi111 with /dev/vg_nvme/lv_2... 2024-09-10T08:33:46.164 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T08:33:46.419 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:33:46.792 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:46 smithi111 ceph-mon[35220]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:46.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:46 smithi149 ceph-mon[36167]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:46 smithi089 ceph-mon[30757]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:47.053 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:33:46 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:33:46.815+0000 7eff4ea2b640 -1 osd.6 0 log_to_monitors true 2024-09-10T08:33:47.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:47 smithi089 ceph-mon[30757]: from='osd.6 [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T08:33:47.870 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:47 smithi111 ceph-mon[35220]: from='osd.6 [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T08:33:47.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:47 smithi149 ceph-mon[36167]: from='osd.6 [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T08:33:48.960 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:48 smithi111 ceph-mon[35220]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:48.960 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:48 smithi111 ceph-mon[35220]: from='osd.6 [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T08:33:48.960 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:48 smithi111 ceph-mon[35220]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T08:33:48.960 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:48 smithi111 ceph-mon[35220]: from='osd.6 [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-10T08:33:48.960 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:48 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:33:48.961 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-10T08:33:48.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:48 smithi149 ceph-mon[36167]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:48.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:48 smithi149 ceph-mon[36167]: from='osd.6 [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T08:33:48.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:48 smithi149 ceph-mon[36167]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T08:33:48.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:48 smithi149 ceph-mon[36167]: from='osd.6 [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-10T08:33:48.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:48 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:33:48.988 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_2 2024-09-10T08:33:49.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:48 smithi089 ceph-mon[30757]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:49.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:48 smithi089 ceph-mon[30757]: from='osd.6 [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T08:33:49.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:48 smithi089 ceph-mon[30757]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T08:33:49.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:48 smithi089 ceph-mon[30757]: from='osd.6 [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-10T08:33:49.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:48 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:33:49.054 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:33:48 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:33:48.685+0000 7eff49600640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T08:33:49.054 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:33:48 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:33:48.696+0000 7eff44600640 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:33:49.239 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:33:49.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.755+0000 7f48e9236640 1 -- 172.21.15.111:0/675868185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48e4103870 msgr2=0x7f48e410a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:33:49.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.755+0000 7f48e9236640 1 --2- 172.21.15.111:0/675868185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48e4103870 0x7f48e410a100 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009ca0 tx=0x7f48dc02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:33:49.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.755+0000 7f48e9236640 1 -- 172.21.15.111:0/675868185 shutdown_connections 2024-09-10T08:33:49.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.755+0000 7f48e9236640 1 --2- 172.21.15.111:0/675868185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48e4103870 0x7f48e410a100 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:33:49.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.755+0000 7f48e9236640 1 --2- 172.21.15.111:0/675868185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e4102eb0 0x7f48e4103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:33:49.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.755+0000 7f48e9236640 1 --2- 172.21.15.111:0/675868185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e40ffed0 0x7f48e41002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:33:49.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.755+0000 7f48e9236640 1 -- 172.21.15.111:0/675868185 >> 172.21.15.111:0/675868185 conn(0x7f48e40fba40 msgr2=0x7f48e40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:33:49.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.755+0000 7f48e9236640 1 -- 172.21.15.111:0/675868185 shutdown_connections 2024-09-10T08:33:49.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.755+0000 7f48e9236640 1 -- 172.21.15.111:0/675868185 wait complete. 2024-09-10T08:33:49.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.756+0000 7f48e9236640 1 Processor -- start 2024-09-10T08:33:49.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.756+0000 7f48e9236640 1 -- start start 2024-09-10T08:33:49.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.756+0000 7f48e9236640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48e40ffed0 0x7f48e4197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:33:49.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48e9236640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e4102eb0 0x7f48e41984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:33:49.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48e9236640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e4103870 0x7f48e419f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:33:49.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48e9236640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f48e410d910 con 0x7f48e4102eb0 2024-09-10T08:33:49.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48e9236640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f48e410d790 con 0x7f48e40ffed0 2024-09-10T08:33:49.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48e9236640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f48e410da90 con 0x7f48e4103870 2024-09-10T08:33:49.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48e2d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48e40ffed0 0x7f48e4197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:33:49.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48e2d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48e40ffed0 0x7f48e4197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43978/0 (socket says 172.21.15.111:43978) 2024-09-10T08:33:49.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48e2d9d640 1 -- 172.21.15.111:0/2527774408 learned_addr learned my addr 172.21.15.111:0/2527774408 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-10T08:33:49.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48e259c640 1 --2- 172.21.15.111:0/2527774408 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e4102eb0 0x7f48e41984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:33:49.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48e2d9d640 1 -- 172.21.15.111:0/2527774408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e4103870 msgr2=0x7f48e419f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:33:49.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48e359e640 1 --2- 172.21.15.111:0/2527774408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e4103870 0x7f48e419f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:33:49.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48e2d9d640 1 --2- 172.21.15.111:0/2527774408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e4103870 0x7f48e419f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:33:49.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48e2d9d640 1 -- 172.21.15.111:0/2527774408 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e4102eb0 msgr2=0x7f48e41984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:33:49.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48e2d9d640 1 --2- 172.21.15.111:0/2527774408 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e4102eb0 0x7f48e41984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:33:49.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48e2d9d640 1 -- 172.21.15.111:0/2527774408 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e419fb90 con 0x7f48e40ffed0 2024-09-10T08:33:49.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48e259c640 1 --2- 172.21.15.111:0/2527774408 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e4102eb0 0x7f48e41984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:33:49.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48e2d9d640 1 --2- 172.21.15.111:0/2527774408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48e40ffed0 0x7f48e4197f60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f48d400d950 tx=0x7f48d400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:33:49.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48cbfff640 1 -- 172.21.15.111:0/2527774408 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4017020 con 0x7f48e40ffed0 2024-09-10T08:33:49.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48cbfff640 1 -- 172.21.15.111:0/2527774408 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48d4009c30 con 0x7f48e40ffed0 2024-09-10T08:33:49.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48e9236640 1 -- 172.21.15.111:0/2527774408 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e419fe80 con 0x7f48e40ffed0 2024-09-10T08:33:49.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48cbfff640 1 -- 172.21.15.111:0/2527774408 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d400f740 con 0x7f48e40ffed0 2024-09-10T08:33:49.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.757+0000 7f48e9236640 1 -- 172.21.15.111:0/2527774408 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e41a02d0 con 0x7f48e40ffed0 2024-09-10T08:33:49.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.758+0000 7f48e9236640 1 -- 172.21.15.111:0/2527774408 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48e4109060 con 0x7f48e40ffed0 2024-09-10T08:33:49.762 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.759+0000 7f48cbfff640 1 -- 172.21.15.111:0/2527774408 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f48d401a020 con 0x7f48e40ffed0 2024-09-10T08:33:49.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.760+0000 7f48cbfff640 1 --2- 172.21.15.111:0/2527774408 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f48cc0a2bb0 0x7f48cc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:33:49.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.760+0000 7f48cbfff640 1 -- 172.21.15.111:0/2527774408 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4965+0+0 (secure 0 0 0) 0x7f48d40cddb0 con 0x7f48e40ffed0 2024-09-10T08:33:49.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.760+0000 7f48e259c640 1 --2- 172.21.15.111:0/2527774408 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f48cc0a2bb0 0x7f48cc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:33:49.763 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.761+0000 7f48e259c640 1 --2- 172.21.15.111:0/2527774408 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f48cc0a2bb0 0x7f48cc0a5070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f48e4199480 tx=0x7f48d8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:33:49.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.764+0000 7f48cbfff640 1 -- 172.21.15.111:0/2527774408 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48d4021070 con 0x7f48e40ffed0 2024-09-10T08:33:49.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:33:49.938+0000 7f48e9236640 1 -- 172.21.15.111:0/2527774408 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f48e4062630 con 0x7f48cc0a2bb0 2024-09-10T08:33:49.938 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:49 smithi111 ceph-mon[35220]: from='osd.6 [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-10T08:33:49.938 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:49 smithi111 ceph-mon[35220]: osdmap e45: 7 total, 6 up, 7 in 2024-09-10T08:33:49.939 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:49 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:33:49.939 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:49 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:33:49.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:49 smithi149 ceph-mon[36167]: from='osd.6 [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-10T08:33:49.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:49 smithi149 ceph-mon[36167]: osdmap e45: 7 total, 6 up, 7 in 2024-09-10T08:33:49.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:49 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:33:49.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:49 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:33:50.006 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:49 smithi089 ceph-mon[30757]: from='osd.6 [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-10T08:33:50.006 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:49 smithi089 ceph-mon[30757]: osdmap e45: 7 total, 6 up, 7 in 2024-09-10T08:33:50.006 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:49 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:33:50.006 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:49 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:33:50.961 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:50 smithi111 ceph-mon[35220]: purged_snaps scrub starts 2024-09-10T08:33:50.961 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:50 smithi111 ceph-mon[35220]: purged_snaps scrub ok 2024-09-10T08:33:50.961 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:50 smithi111 ceph-mon[35220]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:50.961 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:50 smithi111 ceph-mon[35220]: osd.6 [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024] boot 2024-09-10T08:33:50.961 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:50 smithi111 ceph-mon[35220]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T08:33:50.961 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:33:50.961 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:33:50.961 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:33:50.962 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:50.962 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:50.962 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:50.962 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:33:50.962 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:33:50.962 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:33:50.962 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:50.962 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:50.962 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:33:50.962 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:50.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:50 smithi149 ceph-mon[36167]: purged_snaps scrub starts 2024-09-10T08:33:50.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:50 smithi149 ceph-mon[36167]: purged_snaps scrub ok 2024-09-10T08:33:50.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:50 smithi149 ceph-mon[36167]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:50.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:50 smithi149 ceph-mon[36167]: osd.6 [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024] boot 2024-09-10T08:33:50.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:50 smithi149 ceph-mon[36167]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T08:33:50.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:33:50.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:33:50.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:33:50.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:50.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:50.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:50.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:33:50.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:33:50.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:33:50.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:50.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:50.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:33:50.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:51.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:50 smithi089 ceph-mon[30757]: purged_snaps scrub starts 2024-09-10T08:33:51.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:50 smithi089 ceph-mon[30757]: purged_snaps scrub ok 2024-09-10T08:33:51.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:50 smithi089 ceph-mon[30757]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:33:51.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:50 smithi089 ceph-mon[30757]: osd.6 [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024] boot 2024-09-10T08:33:51.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:50 smithi089 ceph-mon[30757]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T08:33:51.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:33:51.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:33:51.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:33:51.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:51.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:51.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:51.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:33:51.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:33:51.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:33:51.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:51.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:33:51.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:33:51.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:33:51.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:51 smithi111 ceph-mon[35220]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:33:51.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:51 smithi111 ceph-mon[35220]: Detected new or changed devices on smithi089 2024-09-10T08:33:51.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:51 smithi111 ceph-mon[35220]: Adjusting osd_memory_target on smithi089 to 5696M 2024-09-10T08:33:51.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:51 smithi111 ceph-mon[35220]: osdmap e47: 7 total, 7 up, 7 in 2024-09-10T08:33:51.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:51 smithi149 ceph-mon[36167]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:33:51.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:51 smithi149 ceph-mon[36167]: Detected new or changed devices on smithi089 2024-09-10T08:33:51.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:51 smithi149 ceph-mon[36167]: Adjusting osd_memory_target on smithi089 to 5696M 2024-09-10T08:33:51.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:51 smithi149 ceph-mon[36167]: osdmap e47: 7 total, 7 up, 7 in 2024-09-10T08:33:52.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:51 smithi089 ceph-mon[30757]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:33:52.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:51 smithi089 ceph-mon[30757]: Detected new or changed devices on smithi089 2024-09-10T08:33:52.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:51 smithi089 ceph-mon[30757]: Adjusting osd_memory_target on smithi089 to 5696M 2024-09-10T08:33:52.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:51 smithi089 ceph-mon[30757]: osdmap e47: 7 total, 7 up, 7 in 2024-09-10T08:33:52.694 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:52 smithi111 ceph-mon[35220]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:33:52.695 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:52 smithi111 ceph-mon[35220]: from='client.? 172.21.15.111:0/483180426' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14a2a6f0-c4cd-406b-b637-7527fd32babe"}]: dispatch 2024-09-10T08:33:52.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:52 smithi149 ceph-mon[36167]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:33:52.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:52 smithi149 ceph-mon[36167]: from='client.? 172.21.15.111:0/483180426' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14a2a6f0-c4cd-406b-b637-7527fd32babe"}]: dispatch 2024-09-10T08:33:52.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:52 smithi149 ceph-mon[36167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14a2a6f0-c4cd-406b-b637-7527fd32babe"}]: dispatch 2024-09-10T08:33:52.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:52 smithi149 ceph-mon[36167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14a2a6f0-c4cd-406b-b637-7527fd32babe"}]': finished 2024-09-10T08:33:52.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:52 smithi149 ceph-mon[36167]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T08:33:52.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:52 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:33:53.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:52 smithi111 ceph-mon[35220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14a2a6f0-c4cd-406b-b637-7527fd32babe"}]: dispatch 2024-09-10T08:33:53.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:52 smithi111 ceph-mon[35220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14a2a6f0-c4cd-406b-b637-7527fd32babe"}]': finished 2024-09-10T08:33:53.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:52 smithi111 ceph-mon[35220]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T08:33:53.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:52 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:33:53.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:52 smithi089 ceph-mon[30757]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:33:53.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:52 smithi089 ceph-mon[30757]: from='client.? 172.21.15.111:0/483180426' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14a2a6f0-c4cd-406b-b637-7527fd32babe"}]: dispatch 2024-09-10T08:33:53.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:52 smithi089 ceph-mon[30757]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14a2a6f0-c4cd-406b-b637-7527fd32babe"}]: dispatch 2024-09-10T08:33:53.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:52 smithi089 ceph-mon[30757]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14a2a6f0-c4cd-406b-b637-7527fd32babe"}]': finished 2024-09-10T08:33:53.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:52 smithi089 ceph-mon[30757]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T08:33:53.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:52 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:33:53.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:53 smithi149 ceph-mon[36167]: from='client.? 172.21.15.111:0/3335048811' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:33:54.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:53 smithi111 ceph-mon[35220]: from='client.? 172.21.15.111:0/3335048811' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:33:54.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:53 smithi089 ceph-mon[30757]: from='client.? 172.21.15.111:0/3335048811' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:33:54.953 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:54 smithi111 ceph-mon[35220]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:33:54.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:54 smithi149 ceph-mon[36167]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:33:55.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:54 smithi089 ceph-mon[30757]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:33:56.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:56 smithi149 ceph-mon[36167]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:33:57.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:56 smithi111 ceph-mon[35220]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:33:57.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:56 smithi089 ceph-mon[30757]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:33:58.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:33:58 smithi111 ceph-mon[35220]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:33:58.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:33:58 smithi149 ceph-mon[36167]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:33:59.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:33:58 smithi089 ceph-mon[30757]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:00.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:00 smithi149 ceph-mon[36167]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:00.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:00 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T08:34:00.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:00 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:01.013 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:00 smithi111 ceph-mon[35220]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:01.014 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:00 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T08:34:01.014 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:00 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:01.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:00 smithi089 ceph-mon[30757]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:01.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:00 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T08:34:01.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:00 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:01.884 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:01 smithi111 ceph-mon[35220]: Deploying daemon osd.7 on smithi111 2024-09-10T08:34:01.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:01 smithi149 ceph-mon[36167]: Deploying daemon osd.7 on smithi111 2024-09-10T08:34:02.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:01 smithi089 ceph-mon[30757]: Deploying daemon osd.7 on smithi111 2024-09-10T08:34:02.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:02 smithi149 ceph-mon[36167]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:02 smithi111 ceph-mon[35220]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:03.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:02 smithi089 ceph-mon[30757]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:04.877 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:04 smithi111 ceph-mon[35220]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:05.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:04 smithi089 ceph-mon[30757]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:05.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:04 smithi149 ceph-mon[36167]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:07.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:06 smithi111 ceph-mon[35220]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:07.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:06 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:34:07.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:06 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:07.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:06 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:07.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:06 smithi089 ceph-mon[30757]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:07.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:06 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:34:07.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:06 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:07.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:06 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:07.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:06 smithi149 ceph-mon[36167]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:07.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:06 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:34:07.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:06 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:07.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:06 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:09.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:08 smithi111 ceph-mon[35220]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:09.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:08 smithi089 ceph-mon[30757]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:09.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:08 smithi149 ceph-mon[36167]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:09.691 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 7 on host 'smithi111' 2024-09-10T08:34:09.692 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:34:09.690+0000 7f48cbfff640 1 -- 172.21.15.111:0/2527774408 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f48e4062630 con 0x7f48cc0a2bb0 2024-09-10T08:34:09.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:34:09.693+0000 7f48e9236640 1 -- 172.21.15.111:0/2527774408 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f48cc0a2bb0 msgr2=0x7f48cc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:34:09.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:34:09.693+0000 7f48e9236640 1 --2- 172.21.15.111:0/2527774408 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f48cc0a2bb0 0x7f48cc0a5070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f48e4199480 tx=0x7f48d8008040 comp rx=0 tx=0).stop 2024-09-10T08:34:09.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:34:09.693+0000 7f48e9236640 1 -- 172.21.15.111:0/2527774408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48e40ffed0 msgr2=0x7f48e4197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:34:09.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:34:09.693+0000 7f48e9236640 1 --2- 172.21.15.111:0/2527774408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48e40ffed0 0x7f48e4197f60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f48d400d950 tx=0x7f48d400de10 comp rx=0 tx=0).stop 2024-09-10T08:34:09.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:34:09.693+0000 7f48e9236640 1 -- 172.21.15.111:0/2527774408 shutdown_connections 2024-09-10T08:34:09.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:34:09.693+0000 7f48e9236640 1 --2- 172.21.15.111:0/2527774408 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f48cc0a2bb0 0x7f48cc0a5070 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:34:09.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:34:09.693+0000 7f48e9236640 1 --2- 172.21.15.111:0/2527774408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e4103870 0x7f48e419f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:34:09.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:34:09.693+0000 7f48e9236640 1 --2- 172.21.15.111:0/2527774408 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48e4102eb0 0x7f48e41984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:34:09.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:34:09.693+0000 7f48e9236640 1 --2- 172.21.15.111:0/2527774408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48e40ffed0 0x7f48e4197f60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:34:09.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:34:09.693+0000 7f48e9236640 1 -- 172.21.15.111:0/2527774408 >> 172.21.15.111:0/2527774408 conn(0x7f48e40fba40 msgr2=0x7f48e4106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:34:09.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:34:09.693+0000 7f48e9236640 1 -- 172.21.15.111:0/2527774408 shutdown_connections 2024-09-10T08:34:09.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:34:09.693+0000 7f48e9236640 1 -- 172.21.15.111:0/2527774408 wait complete. 2024-09-10T08:34:10.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:09 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:10.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:09 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:10.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:09 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:10.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:09 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:34:10.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:09 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:10.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:09 smithi111 ceph-mon[35220]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:10.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:09 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:34:10.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:09 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:10.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:09 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:10.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:09 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:10.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:09 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:10.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:09 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:10.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:09 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:34:10.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:09 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:10.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:09 smithi089 ceph-mon[30757]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:10.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:09 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:34:10.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:09 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:10.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:09 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:10.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:09 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:10.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:09 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:10.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:09 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:10.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:09 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:34:10.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:09 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:10.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:09 smithi149 ceph-mon[36167]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:10.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:09 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:34:10.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:09 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:10.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:09 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:10.371 DEBUG:teuthology.orchestra.run.smithi111:osd.7> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T08:34:10.373 INFO:tasks.cephadm:Deploying osd.8 on smithi149 with /dev/vg_nvme/lv_2... 2024-09-10T08:34:10.374 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T08:34:10.628 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:34:12.043 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:34:11 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:34:11.847+0000 7f3212820640 -1 osd.7 0 log_to_monitors true 2024-09-10T08:34:12.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:12 smithi149 ceph-mon[36167]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:12.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:12 smithi149 ceph-mon[36167]: from='osd.7 [v2:172.21.15.111:6816/652666376,v1:172.21.15.111:6817/652666376]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T08:34:12.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:12 smithi149 ceph-mon[36167]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T08:34:12.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:12 smithi111 ceph-mon[35220]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:12.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:12 smithi111 ceph-mon[35220]: from='osd.7 [v2:172.21.15.111:6816/652666376,v1:172.21.15.111:6817/652666376]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T08:34:12.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:12 smithi111 ceph-mon[35220]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T08:34:13.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:12 smithi089 ceph-mon[30757]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:13.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:12 smithi089 ceph-mon[30757]: from='osd.7 [v2:172.21.15.111:6816/652666376,v1:172.21.15.111:6817/652666376]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T08:34:13.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:12 smithi089 ceph-mon[30757]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T08:34:13.196 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:34:13.221 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_2 2024-09-10T08:34:13.470 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:34:13.847 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:13 smithi149 ceph-mon[36167]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T08:34:13.847 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:13 smithi149 ceph-mon[36167]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T08:34:13.847 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:13 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:34:13.847 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:13 smithi149 ceph-mon[36167]: from='osd.7 [v2:172.21.15.111:6816/652666376,v1:172.21.15.111:6817/652666376]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:34:13.847 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:13 smithi149 ceph-mon[36167]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:34:13.847 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:13 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:13.847 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:13 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:13.847 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:13 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:34:13.847 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:13 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:34:13.848 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:13 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:34:13.848 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:13 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:13.848 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:13 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:13.848 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:13 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:34:13.848 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:13 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:13.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.928+0000 7f0aad15a640 1 -- 172.21.15.149:0/1640859159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0aa8107940 msgr2=0x7f0aa8109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:34:13.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.929+0000 7f0aad15a640 1 --2- 172.21.15.149:0/1640859159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0aa8107940 0x7f0aa8109d30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009ca0 tx=0x7f0aa002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:34:13.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.929+0000 7f0aad15a640 1 -- 172.21.15.149:0/1640859159 shutdown_connections 2024-09-10T08:34:13.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.929+0000 7f0aad15a640 1 --2- 172.21.15.149:0/1640859159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0aa8107940 0x7f0aa8109d30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:34:13.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.929+0000 7f0aad15a640 1 --2- 172.21.15.149:0/1640859159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa8105010 0x7f0aa8107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:34:13.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.929+0000 7f0aad15a640 1 --2- 172.21.15.149:0/1640859159 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0aa81026e0 0x7f0aa8104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:34:13.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.929+0000 7f0aad15a640 1 -- 172.21.15.149:0/1640859159 >> 172.21.15.149:0/1640859159 conn(0x7f0aa80fc280 msgr2=0x7f0aa80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:34:13.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.929+0000 7f0aad15a640 1 -- 172.21.15.149:0/1640859159 shutdown_connections 2024-09-10T08:34:13.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.929+0000 7f0aad15a640 1 -- 172.21.15.149:0/1640859159 wait complete. 2024-09-10T08:34:13.932 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.929+0000 7f0aad15a640 1 Processor -- start 2024-09-10T08:34:13.932 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.930+0000 7f0aad15a640 1 -- start start 2024-09-10T08:34:13.932 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.930+0000 7f0aad15a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0aa81026e0 0x7f0aa8198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:34:13.932 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.930+0000 7f0aad15a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0aa8105010 0x7f0aa8198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:34:13.932 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.930+0000 7f0aad15a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa8107940 0x7f0aa819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:34:13.932 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.930+0000 7f0aad15a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa810d950 con 0x7f0aa8107940 2024-09-10T08:34:13.932 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.930+0000 7f0aad15a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa810d7d0 con 0x7f0aa8105010 2024-09-10T08:34:13.932 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.930+0000 7f0aad15a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa810dad0 con 0x7f0aa81026e0 2024-09-10T08:34:13.932 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.930+0000 7f0aa6d9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0aa81026e0 0x7f0aa8198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:34:13.933 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.930+0000 7f0aa6d9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0aa81026e0 0x7f0aa8198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:54212/0 (socket says 172.21.15.149:54212) 2024-09-10T08:34:13.933 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.930+0000 7f0aa6d9d640 1 -- 172.21.15.149:0/2151039051 learned_addr learned my addr 172.21.15.149:0/2151039051 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-09-10T08:34:13.933 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.930+0000 7f0aa759e640 1 --2- 172.21.15.149:0/2151039051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa8107940 0x7f0aa819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:34:13.933 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.930+0000 7f0aa659c640 1 --2- 172.21.15.149:0/2151039051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0aa8105010 0x7f0aa8198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:34:13.933 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.930+0000 7f0aa6d9d640 1 -- 172.21.15.149:0/2151039051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0aa8105010 msgr2=0x7f0aa8198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:34:13.933 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.930+0000 7f0aa6d9d640 1 --2- 172.21.15.149:0/2151039051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0aa8105010 0x7f0aa8198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:34:13.933 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.930+0000 7f0aa6d9d640 1 -- 172.21.15.149:0/2151039051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa8107940 msgr2=0x7f0aa819f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:34:13.933 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.930+0000 7f0aa6d9d640 1 --2- 172.21.15.149:0/2151039051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa8107940 0x7f0aa819f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:34:13.933 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.930+0000 7f0aa6d9d640 1 -- 172.21.15.149:0/2151039051 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa819fc60 con 0x7f0aa81026e0 2024-09-10T08:34:13.933 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.930+0000 7f0aa759e640 1 --2- 172.21.15.149:0/2151039051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa8107940 0x7f0aa819f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:34:13.933 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.930+0000 7f0aa6d9d640 1 --2- 172.21.15.149:0/2151039051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0aa81026e0 0x7f0aa8198030 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800b4a0 tx=0x7f0a9800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:34:13.933 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.931+0000 7f0a8ffff640 1 -- 172.21.15.149:0/2151039051 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a98014070 con 0x7f0aa81026e0 2024-09-10T08:34:13.933 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.931+0000 7f0aad15a640 1 -- 172.21.15.149:0/2151039051 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa819ff50 con 0x7f0aa81026e0 2024-09-10T08:34:13.934 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.931+0000 7f0a8ffff640 1 -- 172.21.15.149:0/2151039051 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a98010af0 con 0x7f0aa81026e0 2024-09-10T08:34:13.934 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.931+0000 7f0a8ffff640 1 -- 172.21.15.149:0/2151039051 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a98017d10 con 0x7f0aa81026e0 2024-09-10T08:34:13.934 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.931+0000 7f0aad15a640 1 -- 172.21.15.149:0/2151039051 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa81a0420 con 0x7f0aa81026e0 2024-09-10T08:34:13.934 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.932+0000 7f0aad15a640 1 -- 172.21.15.149:0/2151039051 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa8105470 con 0x7f0aa81026e0 2024-09-10T08:34:13.935 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.933+0000 7f0a8ffff640 1 -- 172.21.15.149:0/2151039051 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0a98017070 con 0x7f0aa81026e0 2024-09-10T08:34:13.935 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.933+0000 7f0a8ffff640 1 --2- 172.21.15.149:0/2151039051 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0a900a2bb0 0x7f0a900a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:34:13.935 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.933+0000 7f0a8ffff640 1 -- 172.21.15.149:0/2151039051 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5117+0+0 (secure 0 0 0) 0x7f0a980cd8a0 con 0x7f0aa81026e0 2024-09-10T08:34:13.935 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.934+0000 7f0aa659c640 1 --2- 172.21.15.149:0/2151039051 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0a900a2bb0 0x7f0a900a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:34:13.935 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.934+0000 7f0aa659c640 1 --2- 172.21.15.149:0/2151039051 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0a900a2bb0 0x7f0a900a5070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8199550 tx=0x7f0a9c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:34:13.938 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:13.938+0000 7f0a8ffff640 1 -- 172.21.15.149:0/2151039051 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0a9809b480 con 0x7f0aa81026e0 2024-09-10T08:34:14.043 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:34:13 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:34:13.595+0000 7f320e7a3640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T08:34:14.043 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:34:13 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:34:13.604+0000 7f320a5cb640 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:34:14.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:13 smithi111 ceph-mon[35220]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T08:34:14.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:13 smithi111 ceph-mon[35220]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T08:34:14.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:13 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:34:14.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:13 smithi111 ceph-mon[35220]: from='osd.7 [v2:172.21.15.111:6816/652666376,v1:172.21.15.111:6817/652666376]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:34:14.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:13 smithi111 ceph-mon[35220]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:34:14.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:13 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:14.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:13 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:14.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:13 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:34:14.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:13 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:34:14.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:13 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:34:14.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:13 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:14.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:13 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:14.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:13 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:34:14.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:13 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:14.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:13 smithi089 ceph-mon[30757]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T08:34:14.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:13 smithi089 ceph-mon[30757]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T08:34:14.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:34:14.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:13 smithi089 ceph-mon[30757]: from='osd.7 [v2:172.21.15.111:6816/652666376,v1:172.21.15.111:6817/652666376]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:34:14.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:13 smithi089 ceph-mon[30757]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:34:14.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:14.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:14.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:34:14.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:34:14.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:34:14.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:14.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:14.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:34:14.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:13 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:14.115 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:14.114+0000 7f0aad15a640 1 -- 172.21.15.149:0/2151039051 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0aa8062630 con 0x7f0a900a2bb0 2024-09-10T08:34:14.850 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:14 smithi149 ceph-mon[36167]: Detected new or changed devices on smithi111 2024-09-10T08:34:14.850 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:14 smithi149 ceph-mon[36167]: Adjusting osd_memory_target on smithi111 to 5696M 2024-09-10T08:34:14.850 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:14 smithi149 ceph-mon[36167]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:14.850 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:14 smithi149 ceph-mon[36167]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-10T08:34:14.850 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:14 smithi149 ceph-mon[36167]: osdmap e50: 8 total, 7 up, 8 in 2024-09-10T08:34:14.850 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:14 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:34:14.850 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:14 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:34:14.850 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:14 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:34:14.850 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:14 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:34:14.850 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:14 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:14.851 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:14 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:34:14.917 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:14 smithi089 ceph-mon[30757]: Detected new or changed devices on smithi111 2024-09-10T08:34:14.917 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:14 smithi089 ceph-mon[30757]: Adjusting osd_memory_target on smithi111 to 5696M 2024-09-10T08:34:14.918 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:14 smithi089 ceph-mon[30757]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:14.918 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:14 smithi089 ceph-mon[30757]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-10T08:34:14.918 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:14 smithi089 ceph-mon[30757]: osdmap e50: 8 total, 7 up, 8 in 2024-09-10T08:34:14.918 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:14 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:34:14.918 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:14 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:34:14.918 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:14 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:34:14.918 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:14 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:34:14.918 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:14 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:14.918 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:14 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:34:15.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:14 smithi111 ceph-mon[35220]: Detected new or changed devices on smithi111 2024-09-10T08:34:15.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:14 smithi111 ceph-mon[35220]: Adjusting osd_memory_target on smithi111 to 5696M 2024-09-10T08:34:15.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:14 smithi111 ceph-mon[35220]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:15.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:14 smithi111 ceph-mon[35220]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-10T08:34:15.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:14 smithi111 ceph-mon[35220]: osdmap e50: 8 total, 7 up, 8 in 2024-09-10T08:34:15.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:14 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:34:15.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:14 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:34:15.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:14 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:34:15.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:14 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:34:15.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:14 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:15.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:14 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:34:15.788 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:15 smithi149 ceph-mon[36167]: purged_snaps scrub starts 2024-09-10T08:34:15.788 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:15 smithi149 ceph-mon[36167]: purged_snaps scrub ok 2024-09-10T08:34:15.788 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:15 smithi149 ceph-mon[36167]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:34:15.788 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:15 smithi149 ceph-mon[36167]: osd.7 [v2:172.21.15.111:6816/652666376,v1:172.21.15.111:6817/652666376] boot 2024-09-10T08:34:15.788 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:15 smithi149 ceph-mon[36167]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T08:34:15.789 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:15 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:34:15.789 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:15 smithi149 ceph-mon[36167]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T08:34:15.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:15 smithi089 ceph-mon[30757]: purged_snaps scrub starts 2024-09-10T08:34:15.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:15 smithi089 ceph-mon[30757]: purged_snaps scrub ok 2024-09-10T08:34:15.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:15 smithi089 ceph-mon[30757]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:34:15.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:15 smithi089 ceph-mon[30757]: osd.7 [v2:172.21.15.111:6816/652666376,v1:172.21.15.111:6817/652666376] boot 2024-09-10T08:34:15.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:15 smithi089 ceph-mon[30757]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T08:34:15.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:15 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:34:15.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:15 smithi089 ceph-mon[30757]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T08:34:16.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:15 smithi111 ceph-mon[35220]: purged_snaps scrub starts 2024-09-10T08:34:16.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:15 smithi111 ceph-mon[35220]: purged_snaps scrub ok 2024-09-10T08:34:16.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:15 smithi111 ceph-mon[35220]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:34:16.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:15 smithi111 ceph-mon[35220]: osd.7 [v2:172.21.15.111:6816/652666376,v1:172.21.15.111:6817/652666376] boot 2024-09-10T08:34:16.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:15 smithi111 ceph-mon[35220]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T08:34:16.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:15 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:34:16.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:15 smithi111 ceph-mon[35220]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T08:34:16.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:16 smithi149 ceph-mon[36167]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:16.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:16 smithi149 ceph-mon[36167]: from='client.? 172.21.15.149:0/3776257360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a54502e6-95f7-431f-8db1-54a6c7983b5d"}]: dispatch 2024-09-10T08:34:16.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:16 smithi149 ceph-mon[36167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a54502e6-95f7-431f-8db1-54a6c7983b5d"}]: dispatch 2024-09-10T08:34:16.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:16 smithi149 ceph-mon[36167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a54502e6-95f7-431f-8db1-54a6c7983b5d"}]': finished 2024-09-10T08:34:16.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:16 smithi149 ceph-mon[36167]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T08:34:16.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:16 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:17.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:16 smithi111 ceph-mon[35220]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:17.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:16 smithi111 ceph-mon[35220]: from='client.? 172.21.15.149:0/3776257360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a54502e6-95f7-431f-8db1-54a6c7983b5d"}]: dispatch 2024-09-10T08:34:17.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:16 smithi111 ceph-mon[35220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a54502e6-95f7-431f-8db1-54a6c7983b5d"}]: dispatch 2024-09-10T08:34:17.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:16 smithi111 ceph-mon[35220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a54502e6-95f7-431f-8db1-54a6c7983b5d"}]': finished 2024-09-10T08:34:17.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:16 smithi111 ceph-mon[35220]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T08:34:17.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:16 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:17.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:16 smithi089 ceph-mon[30757]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:34:17.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:16 smithi089 ceph-mon[30757]: from='client.? 172.21.15.149:0/3776257360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a54502e6-95f7-431f-8db1-54a6c7983b5d"}]: dispatch 2024-09-10T08:34:17.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:16 smithi089 ceph-mon[30757]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a54502e6-95f7-431f-8db1-54a6c7983b5d"}]: dispatch 2024-09-10T08:34:17.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:16 smithi089 ceph-mon[30757]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a54502e6-95f7-431f-8db1-54a6c7983b5d"}]': finished 2024-09-10T08:34:17.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:16 smithi089 ceph-mon[30757]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T08:34:17.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:16 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:17.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:17 smithi149 ceph-mon[36167]: from='client.? 172.21.15.149:0/1522113974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:34:18.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:17 smithi111 ceph-mon[35220]: from='client.? 172.21.15.149:0/1522113974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:34:18.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:17 smithi089 ceph-mon[30757]: from='client.? 172.21.15.149:0/1522113974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:34:18.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:18 smithi149 ceph-mon[36167]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:18.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:18 smithi149 ceph-mon[36167]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T08:34:18.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:18 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:19.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:18 smithi111 ceph-mon[35220]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:19.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:18 smithi111 ceph-mon[35220]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T08:34:19.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:18 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:19.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:18 smithi089 ceph-mon[30757]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:19.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:18 smithi089 ceph-mon[30757]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T08:34:19.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:18 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:19.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:19 smithi149 ceph-mon[36167]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T08:34:19.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:19 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:19 smithi111 ceph-mon[35220]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T08:34:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:19 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:20.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:19 smithi089 ceph-mon[30757]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T08:34:20.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:19 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:20.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:20 smithi149 ceph-mon[36167]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:20.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:20 smithi149 ceph-mon[36167]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-10T08:34:21.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:20 smithi111 ceph-mon[35220]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:21.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:20 smithi111 ceph-mon[35220]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-10T08:34:21.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:20 smithi089 ceph-mon[30757]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:21.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:20 smithi089 ceph-mon[30757]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-10T08:34:21.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:21 smithi149 ceph-mon[36167]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-10T08:34:21.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:21 smithi149 ceph-mon[36167]: Cluster is now healthy 2024-09-10T08:34:22.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:21 smithi111 ceph-mon[35220]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-10T08:34:22.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:21 smithi111 ceph-mon[35220]: Cluster is now healthy 2024-09-10T08:34:22.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:21 smithi089 ceph-mon[30757]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-10T08:34:22.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:21 smithi089 ceph-mon[30757]: Cluster is now healthy 2024-09-10T08:34:22.783 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:22 smithi149 ceph-mon[36167]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:23.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:22 smithi111 ceph-mon[35220]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:23.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:22 smithi089 ceph-mon[30757]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:24.885 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:24 smithi149 ceph-mon[36167]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:25.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:24 smithi111 ceph-mon[35220]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:25.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:24 smithi089 ceph-mon[30757]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:25.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:25 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T08:34:25.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:25 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:26.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:25 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T08:34:26.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:25 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:26.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:25 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T08:34:26.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:25 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:26.897 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:26 smithi149 ceph-mon[36167]: Deploying daemon osd.8 on smithi149 2024-09-10T08:34:26.898 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:26 smithi149 ceph-mon[36167]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:27.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:26 smithi111 ceph-mon[35220]: Deploying daemon osd.8 on smithi149 2024-09-10T08:34:27.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:26 smithi111 ceph-mon[35220]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:27.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:26 smithi089 ceph-mon[30757]: Deploying daemon osd.8 on smithi149 2024-09-10T08:34:27.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:26 smithi089 ceph-mon[30757]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:28.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:28 smithi149 ceph-mon[36167]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:29.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:28 smithi111 ceph-mon[35220]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:29.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:28 smithi089 ceph-mon[30757]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:30.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:30 smithi149 ceph-mon[36167]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:31.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:30 smithi111 ceph-mon[35220]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:31.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:30 smithi089 ceph-mon[30757]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:31.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:31 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:34:31.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:31 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:31.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:31 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:32.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:31 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:34:32.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:31 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:32.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:31 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:32.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:31 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:34:32.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:31 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:32.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:31 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:32.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:32 smithi149 ceph-mon[36167]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:33.062 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:32 smithi111 ceph-mon[35220]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:33.062 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:32 smithi089 ceph-mon[30757]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:34.830 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:34 smithi149 ceph-mon[36167]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:34.830 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:34.830 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:34.830 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:34.830 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:34:34.830 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:34.875 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:34.875+0000 7f0a8ffff640 1 -- 172.21.15.149:0/2151039051 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0aa8062630 con 0x7f0a900a2bb0 2024-09-10T08:34:34.875 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 8 on host 'smithi149' 2024-09-10T08:34:34.878 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:34.878+0000 7f0aad15a640 1 -- 172.21.15.149:0/2151039051 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0a900a2bb0 msgr2=0x7f0a900a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:34:34.878 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:34.878+0000 7f0aad15a640 1 --2- 172.21.15.149:0/2151039051 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0a900a2bb0 0x7f0a900a5070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8199550 tx=0x7f0a9c008040 comp rx=0 tx=0).stop 2024-09-10T08:34:34.878 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:34.878+0000 7f0aad15a640 1 -- 172.21.15.149:0/2151039051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0aa81026e0 msgr2=0x7f0aa8198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:34:34.878 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:34.878+0000 7f0aad15a640 1 --2- 172.21.15.149:0/2151039051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0aa81026e0 0x7f0aa8198030 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800b4a0 tx=0x7f0a9800b960 comp rx=0 tx=0).stop 2024-09-10T08:34:34.878 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:34.878+0000 7f0aad15a640 1 -- 172.21.15.149:0/2151039051 shutdown_connections 2024-09-10T08:34:34.879 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:34.878+0000 7f0aad15a640 1 --2- 172.21.15.149:0/2151039051 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0a900a2bb0 0x7f0a900a5070 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:34:34.879 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:34.878+0000 7f0aad15a640 1 --2- 172.21.15.149:0/2151039051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0aa8107940 0x7f0aa819f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:34:34.879 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:34.878+0000 7f0aad15a640 1 --2- 172.21.15.149:0/2151039051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0aa8105010 0x7f0aa8198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:34:34.879 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:34.878+0000 7f0aad15a640 1 --2- 172.21.15.149:0/2151039051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0aa81026e0 0x7f0aa8198030 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:34:34.879 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:34.878+0000 7f0aad15a640 1 -- 172.21.15.149:0/2151039051 >> 172.21.15.149:0/2151039051 conn(0x7f0aa80fc280 msgr2=0x7f0aa8103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:34:34.879 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:34.878+0000 7f0aad15a640 1 -- 172.21.15.149:0/2151039051 shutdown_connections 2024-09-10T08:34:34.879 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:34:34.878+0000 7f0aad15a640 1 -- 172.21.15.149:0/2151039051 wait complete. 2024-09-10T08:34:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:34 smithi111 ceph-mon[35220]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:34:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:35.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:34 smithi089 ceph-mon[30757]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:35.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:35.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:34:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:35.561 DEBUG:teuthology.orchestra.run.smithi149:osd.8> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.8.service 2024-09-10T08:34:35.563 INFO:tasks.cephadm:Deploying osd.9 on smithi089 with /dev/vg_nvme/lv_1... 2024-09-10T08:34:35.564 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T08:34:35.823 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:35 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:34:35.823 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:35 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:35.823 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:35 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:35.939 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:34:36.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:35 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:34:36.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:35 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:36.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:35 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:36.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:35 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:34:36.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:35 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:36.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:35 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:36.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:36 smithi149 ceph-mon[36167]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:36.970 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:34:36 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:34:36.776+0000 7fe0f55f4640 -1 osd.8 0 log_to_monitors true 2024-09-10T08:34:37.015 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:36 smithi089 ceph-mon[30757]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:37.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:36 smithi111 ceph-mon[35220]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:37.974 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:37 smithi089 ceph-mon[30757]: from='osd.8 [v2:172.21.15.149:6816/2955172869,v1:172.21.15.149:6817/2955172869]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T08:34:37.974 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:37 smithi089 ceph-mon[30757]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T08:34:38.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:37 smithi111 ceph-mon[35220]: from='osd.8 [v2:172.21.15.149:6816/2955172869,v1:172.21.15.149:6817/2955172869]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T08:34:38.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:37 smithi111 ceph-mon[35220]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T08:34:38.099 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:37 smithi149 ceph-mon[36167]: from='osd.8 [v2:172.21.15.149:6816/2955172869,v1:172.21.15.149:6817/2955172869]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T08:34:38.100 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:37 smithi149 ceph-mon[36167]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T08:34:38.972 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:38 smithi089 ceph-mon[30757]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:38.972 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:38 smithi089 ceph-mon[30757]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T08:34:38.972 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:38 smithi089 ceph-mon[30757]: osdmap e56: 9 total, 8 up, 9 in 2024-09-10T08:34:38.972 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:38 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:38.972 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:38 smithi089 ceph-mon[30757]: from='osd.8 [v2:172.21.15.149:6816/2955172869,v1:172.21.15.149:6817/2955172869]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:34:38.973 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:38 smithi089 ceph-mon[30757]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:34:38.973 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:38 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:38.973 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:38 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:38.973 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:38 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:34:38.973 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:38 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:34:38.973 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:38 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:34:38.973 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:38 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:38.973 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:38 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:38.973 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:38 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:34:38.973 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:38 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:38 smithi111 ceph-mon[35220]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:38 smithi111 ceph-mon[35220]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T08:34:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:38 smithi111 ceph-mon[35220]: osdmap e56: 9 total, 8 up, 9 in 2024-09-10T08:34:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:38 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:38 smithi111 ceph-mon[35220]: from='osd.8 [v2:172.21.15.149:6816/2955172869,v1:172.21.15.149:6817/2955172869]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:34:39.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:38 smithi111 ceph-mon[35220]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:34:39.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:38 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:39.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:38 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:39.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:38 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:34:39.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:38 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:34:39.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:38 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:34:39.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:38 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:39.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:38 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:39.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:38 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:34:39.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:38 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:39.118 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:38 smithi149 ceph-mon[36167]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:39.118 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:38 smithi149 ceph-mon[36167]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T08:34:39.118 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:38 smithi149 ceph-mon[36167]: osdmap e56: 9 total, 8 up, 9 in 2024-09-10T08:34:39.118 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:38 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:39.118 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:38 smithi149 ceph-mon[36167]: from='osd.8 [v2:172.21.15.149:6816/2955172869,v1:172.21.15.149:6817/2955172869]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:34:39.118 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:38 smithi149 ceph-mon[36167]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:34:39.118 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:38 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:39.118 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:38 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:39.119 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:38 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:34:39.119 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:38 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:34:39.119 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:38 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:34:39.119 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:38 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:39.119 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:38 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:39.119 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:38 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:34:39.119 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:38 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:34:39.345 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:34:39.378 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi089:vg_nvme/lv_1 2024-09-10T08:34:39.470 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:34:39 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:34:39.117+0000 7fe0f1577640 -1 osd.8 0 waiting for initial osdmap 2024-09-10T08:34:39.470 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:34:39 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:34:39.128+0000 7fe0e9397640 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:34:39.754 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:34:39.993 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:39 smithi089 ceph-mon[30757]: Detected new or changed devices on smithi149 2024-09-10T08:34:39.993 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:39 smithi089 ceph-mon[30757]: Adjusting osd_memory_target on smithi149 to 1938M 2024-09-10T08:34:39.993 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:39 smithi089 ceph-mon[30757]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T08:34:39.993 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:39 smithi089 ceph-mon[30757]: osdmap e57: 9 total, 8 up, 9 in 2024-09-10T08:34:39.994 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:39 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:39.994 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:39 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:39.994 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:39 smithi089 ceph-mon[30757]: from='osd.8 ' entity='osd.8' 2024-09-10T08:34:40.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:39 smithi111 ceph-mon[35220]: Detected new or changed devices on smithi149 2024-09-10T08:34:40.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:39 smithi111 ceph-mon[35220]: Adjusting osd_memory_target on smithi149 to 1938M 2024-09-10T08:34:40.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:39 smithi111 ceph-mon[35220]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T08:34:40.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:39 smithi111 ceph-mon[35220]: osdmap e57: 9 total, 8 up, 9 in 2024-09-10T08:34:40.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:39 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:40.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:39 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:40.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:39 smithi111 ceph-mon[35220]: from='osd.8 ' entity='osd.8' 2024-09-10T08:34:40.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:39 smithi149 ceph-mon[36167]: Detected new or changed devices on smithi149 2024-09-10T08:34:40.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:39 smithi149 ceph-mon[36167]: Adjusting osd_memory_target on smithi149 to 1938M 2024-09-10T08:34:40.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:39 smithi149 ceph-mon[36167]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T08:34:40.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:39 smithi149 ceph-mon[36167]: osdmap e57: 9 total, 8 up, 9 in 2024-09-10T08:34:40.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:39 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:40.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:39 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:40.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:39 smithi149 ceph-mon[36167]: from='osd.8 ' entity='osd.8' 2024-09-10T08:34:40.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.356+0000 7ff6b2000640 1 -- 172.21.15.89:0/825164786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6ac104f60 msgr2=0x7ff6ac1053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:34:40.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.356+0000 7ff6b2000640 1 --2- 172.21.15.89:0/825164786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6ac104f60 0x7ff6ac1053e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff694009900 tx=0x7ff69402a410 comp rx=0 tx=0).stop 2024-09-10T08:34:40.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.356+0000 7ff6b2000640 1 -- 172.21.15.89:0/825164786 shutdown_connections 2024-09-10T08:34:40.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.356+0000 7ff6b2000640 1 --2- 172.21.15.89:0/825164786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6ac105920 0x7ff6ac10c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:34:40.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.356+0000 7ff6b2000640 1 --2- 172.21.15.89:0/825164786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6ac104f60 0x7ff6ac1053e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:34:40.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.356+0000 7ff6b2000640 1 --2- 172.21.15.89:0/825164786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6ac103e00 0x7ff6ac104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:34:40.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.356+0000 7ff6b2000640 1 -- 172.21.15.89:0/825164786 >> 172.21.15.89:0/825164786 conn(0x7ff6ac0ff5b0 msgr2=0x7ff6ac1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:34:40.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.356+0000 7ff6b2000640 1 -- 172.21.15.89:0/825164786 shutdown_connections 2024-09-10T08:34:40.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.356+0000 7ff6b2000640 1 -- 172.21.15.89:0/825164786 wait complete. 2024-09-10T08:34:40.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.357+0000 7ff6b2000640 1 Processor -- start 2024-09-10T08:34:40.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.357+0000 7ff6b2000640 1 -- start start 2024-09-10T08:34:40.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.357+0000 7ff6b2000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6ac103e00 0x7ff6ac19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:34:40.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.357+0000 7ff6b2000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6ac104f60 0x7ff6ac19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:34:40.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.357+0000 7ff6b2000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6ac105920 0x7ff6ac1a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:34:40.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.357+0000 7ff6b2000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac10fb00 con 0x7ff6ac103e00 2024-09-10T08:34:40.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.357+0000 7ff6b2000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac10f980 con 0x7ff6ac105920 2024-09-10T08:34:40.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.357+0000 7ff6b2000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac10fc80 con 0x7ff6ac104f60 2024-09-10T08:34:40.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.357+0000 7ff6ab400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6ac103e00 0x7ff6ac19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:34:40.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.358+0000 7ff6ab400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6ac103e00 0x7ff6ac19a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35944/0 (socket says 172.21.15.89:35944) 2024-09-10T08:34:40.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.358+0000 7ff6ab400640 1 -- 172.21.15.89:0/454190727 learned_addr learned my addr 172.21.15.89:0/454190727 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:34:40.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.358+0000 7ff6a2a00640 1 --2- 172.21.15.89:0/454190727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6ac104f60 0x7ff6ac19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:34:40.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.358+0000 7ff6abe00640 1 --2- 172.21.15.89:0/454190727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6ac105920 0x7ff6ac1a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:34:40.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.358+0000 7ff6ab400640 1 -- 172.21.15.89:0/454190727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6ac104f60 msgr2=0x7ff6ac19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:34:40.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.358+0000 7ff6ab400640 1 --2- 172.21.15.89:0/454190727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6ac104f60 0x7ff6ac19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:34:40.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.358+0000 7ff6ab400640 1 -- 172.21.15.89:0/454190727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6ac105920 msgr2=0x7ff6ac1a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:34:40.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.358+0000 7ff6ab400640 1 --2- 172.21.15.89:0/454190727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6ac105920 0x7ff6ac1a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:34:40.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.358+0000 7ff6ab400640 1 -- 172.21.15.89:0/454190727 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ac1a1d30 con 0x7ff6ac103e00 2024-09-10T08:34:40.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.358+0000 7ff6abe00640 1 --2- 172.21.15.89:0/454190727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6ac105920 0x7ff6ac1a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:34:40.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.358+0000 7ff6ab400640 1 --2- 172.21.15.89:0/454190727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6ac103e00 0x7ff6ac19a100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00d950 tx=0x7ff69c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:34:40.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.358+0000 7ff6a8c00640 1 -- 172.21.15.89:0/454190727 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c017070 con 0x7ff6ac103e00 2024-09-10T08:34:40.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.358+0000 7ff6a8c00640 1 -- 172.21.15.89:0/454190727 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff69c009c30 con 0x7ff6ac103e00 2024-09-10T08:34:40.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.358+0000 7ff6a8c00640 1 -- 172.21.15.89:0/454190727 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c00f6b0 con 0x7ff6ac103e00 2024-09-10T08:34:40.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.359+0000 7ff6b2000640 1 -- 172.21.15.89:0/454190727 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6ac1a1fc0 con 0x7ff6ac103e00 2024-09-10T08:34:40.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.359+0000 7ff6b2000640 1 -- 172.21.15.89:0/454190727 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6ac1a2420 con 0x7ff6ac103e00 2024-09-10T08:34:40.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.360+0000 7ff6b2000640 1 -- 172.21.15.89:0/454190727 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6ac10b120 con 0x7ff6ac103e00 2024-09-10T08:34:40.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.362+0000 7ff6a8c00640 1 -- 172.21.15.89:0/454190727 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff69c0109d0 con 0x7ff6ac103e00 2024-09-10T08:34:40.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.363+0000 7ff6a8c00640 1 --2- 172.21.15.89:0/454190727 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff6900a2c30 0x7ff6900a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:34:40.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.363+0000 7ff6a8c00640 1 -- 172.21.15.89:0/454190727 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5862+0+0 (secure 0 0 0) 0x7ff69c0ce380 con 0x7ff6ac103e00 2024-09-10T08:34:40.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.363+0000 7ff6a2a00640 1 --2- 172.21.15.89:0/454190727 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff6900a2c30 0x7ff6900a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:34:40.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.364+0000 7ff6a2a00640 1 --2- 172.21.15.89:0/454190727 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff6900a2c30 0x7ff6900a50f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff69402aa00 tx=0x7ff694031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:34:40.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.367+0000 7ff6a8c00640 1 -- 172.21.15.89:0/454190727 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff69c009de0 con 0x7ff6ac103e00 2024-09-10T08:34:40.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:34:40.554+0000 7ff6b2000640 1 -- 172.21.15.89:0/454190727 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff6ac062630 con 0x7ff6900a2c30 2024-09-10T08:34:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:40 smithi111 ceph-mon[35220]: purged_snaps scrub starts 2024-09-10T08:34:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:40 smithi111 ceph-mon[35220]: purged_snaps scrub ok 2024-09-10T08:34:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:40 smithi111 ceph-mon[35220]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:40 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:40 smithi111 ceph-mon[35220]: osd.8 [v2:172.21.15.149:6816/2955172869,v1:172.21.15.149:6817/2955172869] boot 2024-09-10T08:34:41.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:40 smithi111 ceph-mon[35220]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T08:34:41.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:40 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:41.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:40 smithi111 ceph-mon[35220]: osdmap e59: 9 total, 9 up, 9 in 2024-09-10T08:34:41.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:40 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:34:41.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:40 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:34:41.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:40 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:41.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:40 smithi089 ceph-mon[30757]: purged_snaps scrub starts 2024-09-10T08:34:41.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:40 smithi089 ceph-mon[30757]: purged_snaps scrub ok 2024-09-10T08:34:41.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:40 smithi089 ceph-mon[30757]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:41.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:40 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:41.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:40 smithi089 ceph-mon[30757]: osd.8 [v2:172.21.15.149:6816/2955172869,v1:172.21.15.149:6817/2955172869] boot 2024-09-10T08:34:41.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:40 smithi089 ceph-mon[30757]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T08:34:41.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:40 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:41.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:40 smithi089 ceph-mon[30757]: osdmap e59: 9 total, 9 up, 9 in 2024-09-10T08:34:41.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:40 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:34:41.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:40 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:34:41.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:40 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:41.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:40 smithi149 ceph-mon[36167]: purged_snaps scrub starts 2024-09-10T08:34:41.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:40 smithi149 ceph-mon[36167]: purged_snaps scrub ok 2024-09-10T08:34:41.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:40 smithi149 ceph-mon[36167]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:34:41.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:40 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:41.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:40 smithi149 ceph-mon[36167]: osd.8 [v2:172.21.15.149:6816/2955172869,v1:172.21.15.149:6817/2955172869] boot 2024-09-10T08:34:41.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:40 smithi149 ceph-mon[36167]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T08:34:41.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:40 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:34:41.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:40 smithi149 ceph-mon[36167]: osdmap e59: 9 total, 9 up, 9 in 2024-09-10T08:34:41.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:40 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:34:41.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:40 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:34:41.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:40 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:42.029 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:41 smithi089 ceph-mon[30757]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:34:42.029 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:41 smithi089 ceph-mon[30757]: osdmap e60: 9 total, 9 up, 9 in 2024-09-10T08:34:42.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:41 smithi111 ceph-mon[35220]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:34:42.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:41 smithi111 ceph-mon[35220]: osdmap e60: 9 total, 9 up, 9 in 2024-09-10T08:34:42.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:41 smithi149 ceph-mon[36167]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:34:42.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:41 smithi149 ceph-mon[36167]: osdmap e60: 9 total, 9 up, 9 in 2024-09-10T08:34:42.751 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:42 smithi089 ceph-mon[30757]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:34:42.752 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:42 smithi089 ceph-mon[30757]: osdmap e61: 9 total, 9 up, 9 in 2024-09-10T08:34:43.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:42 smithi111 ceph-mon[35220]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:34:43.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:42 smithi111 ceph-mon[35220]: osdmap e61: 9 total, 9 up, 9 in 2024-09-10T08:34:43.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:42 smithi149 ceph-mon[36167]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:34:43.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:42 smithi149 ceph-mon[36167]: osdmap e61: 9 total, 9 up, 9 in 2024-09-10T08:34:44.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:44 smithi111 ceph-mon[35220]: osdmap e62: 9 total, 9 up, 9 in 2024-09-10T08:34:44.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:44 smithi111 ceph-mon[35220]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:34:44.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:44 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3476584023' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fe17344-5b1c-4601-a633-639cc1691478"}]: dispatch 2024-09-10T08:34:44.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:44 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3476584023' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fe17344-5b1c-4601-a633-639cc1691478"}]': finished 2024-09-10T08:34:44.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:44 smithi111 ceph-mon[35220]: osdmap e63: 10 total, 9 up, 10 in 2024-09-10T08:34:44.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:44 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:34:44.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:44 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2981248735' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:34:44.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:44 smithi089 ceph-mon[30757]: osdmap e62: 9 total, 9 up, 9 in 2024-09-10T08:34:44.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:44 smithi089 ceph-mon[30757]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:34:44.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:44 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3476584023' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fe17344-5b1c-4601-a633-639cc1691478"}]: dispatch 2024-09-10T08:34:44.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:44 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3476584023' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fe17344-5b1c-4601-a633-639cc1691478"}]': finished 2024-09-10T08:34:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:44 smithi089 ceph-mon[30757]: osdmap e63: 10 total, 9 up, 10 in 2024-09-10T08:34:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:44 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:34:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:44 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2981248735' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:34:44.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:44 smithi149 ceph-mon[36167]: osdmap e62: 9 total, 9 up, 9 in 2024-09-10T08:34:44.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:44 smithi149 ceph-mon[36167]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:34:44.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:44 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3476584023' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fe17344-5b1c-4601-a633-639cc1691478"}]: dispatch 2024-09-10T08:34:44.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:44 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3476584023' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fe17344-5b1c-4601-a633-639cc1691478"}]': finished 2024-09-10T08:34:44.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:44 smithi149 ceph-mon[36167]: osdmap e63: 10 total, 9 up, 10 in 2024-09-10T08:34:44.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:44 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:34:44.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:44 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2981248735' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:34:46.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:46 smithi149 ceph-mon[36167]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:34:47.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:46 smithi111 ceph-mon[35220]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:34:47.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:46 smithi089 ceph-mon[30757]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:34:48.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:48 smithi089 ceph-mon[30757]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T08:34:48.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:48 smithi149 ceph-mon[36167]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T08:34:49.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:48 smithi111 ceph-mon[35220]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T08:34:50.900 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:50 smithi089 ceph-mon[30757]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T08:34:50.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:50 smithi149 ceph-mon[36167]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T08:34:51.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:50 smithi111 ceph-mon[35220]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T08:34:52.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:52 smithi089 ceph-mon[30757]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T08:34:52.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:52 smithi149 ceph-mon[36167]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T08:34:53.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:52 smithi111 ceph-mon[35220]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T08:34:53.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:53 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T08:34:53.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:53 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:53.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:53 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T08:34:53.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:53 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:54.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:53 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T08:34:54.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:53 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:34:54.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:54 smithi089 ceph-mon[30757]: Deploying daemon osd.9 on smithi089 2024-09-10T08:34:54.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:54 smithi089 ceph-mon[30757]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T08:34:54.944 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:54 smithi111 ceph-mon[35220]: Deploying daemon osd.9 on smithi089 2024-09-10T08:34:54.944 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:54 smithi111 ceph-mon[35220]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T08:34:54.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:54 smithi149 ceph-mon[36167]: Deploying daemon osd.9 on smithi089 2024-09-10T08:34:54.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:54 smithi149 ceph-mon[36167]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T08:34:56.690 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:56 smithi089 ceph-mon[30757]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T08:34:56.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:56 smithi149 ceph-mon[36167]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T08:34:57.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:56 smithi111 ceph-mon[35220]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T08:34:58.960 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:34:58 smithi089 ceph-mon[30757]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:34:58.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:34:58 smithi149 ceph-mon[36167]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:34:59.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:34:58 smithi111 ceph-mon[35220]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:35:00.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:00 smithi089 ceph-mon[30757]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:35:00.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:00 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:35:00.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:00 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:00.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:00 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:00.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:00 smithi149 ceph-mon[36167]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:35:00.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:00 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:35:00.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:00 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:00.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:00 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:01.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:00 smithi111 ceph-mon[35220]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:35:01.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:00 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:35:01.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:00 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:01.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:00 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:02.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:02 smithi149 ceph-mon[36167]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:35:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:02 smithi111 ceph-mon[35220]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:35:03.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:02 smithi089 ceph-mon[30757]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:35:04.926 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:04 smithi089 ceph-mon[30757]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:35:04.926 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:04 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:04.926 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:04 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:04.927 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:04 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:04.927 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:04 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:35:04.927 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:04 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:04.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:04 smithi149 ceph-mon[36167]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:35:04.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:04 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:04.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:04 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:04.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:04 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:04.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:04 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:35:04.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:04 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:04 smithi111 ceph-mon[35220]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:35:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:04 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:04 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:04 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:04 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:35:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:04 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:05.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:05.063+0000 7ff6a8c00640 1 -- 172.21.15.89:0/454190727 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff6ac062630 con 0x7ff6900a2c30 2024-09-10T08:35:05.064 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 9 on host 'smithi089' 2024-09-10T08:35:05.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:05.066+0000 7ff6b2000640 1 -- 172.21.15.89:0/454190727 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff6900a2c30 msgr2=0x7ff6900a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:35:05.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:05.066+0000 7ff6b2000640 1 --2- 172.21.15.89:0/454190727 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff6900a2c30 0x7ff6900a50f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff69402aa00 tx=0x7ff694031400 comp rx=0 tx=0).stop 2024-09-10T08:35:05.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:05.066+0000 7ff6b2000640 1 -- 172.21.15.89:0/454190727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6ac103e00 msgr2=0x7ff6ac19a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:35:05.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:05.066+0000 7ff6b2000640 1 --2- 172.21.15.89:0/454190727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6ac103e00 0x7ff6ac19a100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00d950 tx=0x7ff69c00de10 comp rx=0 tx=0).stop 2024-09-10T08:35:05.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:05.066+0000 7ff6b2000640 1 -- 172.21.15.89:0/454190727 shutdown_connections 2024-09-10T08:35:05.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:05.066+0000 7ff6b2000640 1 --2- 172.21.15.89:0/454190727 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff6900a2c30 0x7ff6900a50f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:05.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:05.066+0000 7ff6b2000640 1 --2- 172.21.15.89:0/454190727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6ac105920 0x7ff6ac1a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:05.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:05.066+0000 7ff6b2000640 1 --2- 172.21.15.89:0/454190727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6ac104f60 0x7ff6ac19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:05.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:05.066+0000 7ff6b2000640 1 --2- 172.21.15.89:0/454190727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6ac103e00 0x7ff6ac19a100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:05.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:05.066+0000 7ff6b2000640 1 -- 172.21.15.89:0/454190727 >> 172.21.15.89:0/454190727 conn(0x7ff6ac0ff5b0 msgr2=0x7ff6ac1082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:35:05.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:05.066+0000 7ff6b2000640 1 -- 172.21.15.89:0/454190727 shutdown_connections 2024-09-10T08:35:05.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:05.067+0000 7ff6b2000640 1 -- 172.21.15.89:0/454190727 wait complete. 2024-09-10T08:35:05.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:05 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:35:05.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:05 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:05.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:05 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:05.916 DEBUG:teuthology.orchestra.run.smithi089:osd.9> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.9.service 2024-09-10T08:35:05.919 INFO:tasks.cephadm:Deploying osd.10 on smithi111 with /dev/vg_nvme/lv_1... 2024-09-10T08:35:05.920 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T08:35:05.947 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:05 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:35:05.948 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:05 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:05.948 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:05 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:05.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:05 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:35:05.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:05 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:05.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:05 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:06.173 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:35:06.720 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:06 smithi089 ceph-mon[30757]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:35:06.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:06 smithi149 ceph-mon[36167]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:35:07.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:06 smithi111 ceph-mon[35220]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:35:07.053 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:35:06 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:35:06.719+0000 7f3aa9843640 -1 osd.9 0 log_to_monitors true 2024-09-10T08:35:07.910 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:07 smithi111 ceph-mon[35220]: from='osd.9 [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T08:35:07.940 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:07 smithi089 ceph-mon[30757]: from='osd.9 [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T08:35:07.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:07 smithi149 ceph-mon[36167]: from='osd.9 [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T08:35:08.790 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:08 smithi111 ceph-mon[35220]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:35:08.790 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:08 smithi111 ceph-mon[35220]: from='osd.9 [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T08:35:08.790 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:08 smithi111 ceph-mon[35220]: osdmap e64: 10 total, 9 up, 10 in 2024-09-10T08:35:08.790 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:08 smithi111 ceph-mon[35220]: from='osd.9 [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-10T08:35:08.790 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:08 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:35:08.792 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-10T08:35:08.817 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_1 2024-09-10T08:35:08.888 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:08 smithi089 ceph-mon[30757]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:35:08.888 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:08 smithi089 ceph-mon[30757]: from='osd.9 [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T08:35:08.889 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:08 smithi089 ceph-mon[30757]: osdmap e64: 10 total, 9 up, 10 in 2024-09-10T08:35:08.889 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:08 smithi089 ceph-mon[30757]: from='osd.9 [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-10T08:35:08.889 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:08 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:35:08.889 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:35:08 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:35:08.643+0000 7f3aa4000640 -1 osd.9 0 waiting for initial osdmap 2024-09-10T08:35:08.889 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:35:08 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:35:08.656+0000 7f3a9f000640 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:35:08.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:08 smithi149 ceph-mon[36167]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:35:08.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:08 smithi149 ceph-mon[36167]: from='osd.9 [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T08:35:08.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:08 smithi149 ceph-mon[36167]: osdmap e64: 10 total, 9 up, 10 in 2024-09-10T08:35:08.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:08 smithi149 ceph-mon[36167]: from='osd.9 [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-09-10T08:35:08.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:08 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:35:09.110 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:35:09.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.584+0000 7f7744cef640 1 -- 172.21.15.111:0/3584864337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7740101bf0 msgr2=0x7f7740101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:35:09.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.584+0000 7f7744cef640 1 --2- 172.21.15.111:0/3584864337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7740101bf0 0x7f7740101ff0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7730009900 tx=0x7f773002a410 comp rx=0 tx=0).stop 2024-09-10T08:35:09.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.584+0000 7f7744cef640 1 -- 172.21.15.111:0/3584864337 shutdown_connections 2024-09-10T08:35:09.602 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.584+0000 7f7744cef640 1 --2- 172.21.15.111:0/3584864337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77401037b0 0x7f774010a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:09.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.584+0000 7f7744cef640 1 --2- 172.21.15.111:0/3584864337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7740102df0 0x7f7740103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:09.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.584+0000 7f7744cef640 1 --2- 172.21.15.111:0/3584864337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7740101bf0 0x7f7740101ff0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:09.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.584+0000 7f7744cef640 1 -- 172.21.15.111:0/3584864337 >> 172.21.15.111:0/3584864337 conn(0x7f77400fd3c0 msgr2=0x7f77400ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:35:09.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.584+0000 7f7744cef640 1 -- 172.21.15.111:0/3584864337 shutdown_connections 2024-09-10T08:35:09.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.584+0000 7f7744cef640 1 -- 172.21.15.111:0/3584864337 wait complete. 2024-09-10T08:35:09.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.585+0000 7f7744cef640 1 Processor -- start 2024-09-10T08:35:09.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.585+0000 7f7744cef640 1 -- start start 2024-09-10T08:35:09.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.585+0000 7f7744cef640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7740101bf0 0x7f7740197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:35:09.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.585+0000 7f7744cef640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7740102df0 0x7f7740198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:35:09.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.585+0000 7f7744cef640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77401037b0 0x7f774019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:35:09.603 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.585+0000 7f7744cef640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f774010d8d0 con 0x7f7740101bf0 2024-09-10T08:35:09.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.585+0000 7f7744cef640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f774010d750 con 0x7f7740102df0 2024-09-10T08:35:09.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.585+0000 7f7744cef640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f774010da50 con 0x7f77401037b0 2024-09-10T08:35:09.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.585+0000 7f773dd9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7740102df0 0x7f7740198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:35:09.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.585+0000 7f773dd9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7740102df0 0x7f7740198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47154/0 (socket says 172.21.15.111:47154) 2024-09-10T08:35:09.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.585+0000 7f773dd9b640 1 -- 172.21.15.111:0/3926278613 learned_addr learned my addr 172.21.15.111:0/3926278613 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-10T08:35:09.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.585+0000 7f773e59c640 1 --2- 172.21.15.111:0/3926278613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7740101bf0 0x7f7740197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:35:09.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.585+0000 7f773ed9d640 1 --2- 172.21.15.111:0/3926278613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77401037b0 0x7f774019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:35:09.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.585+0000 7f773dd9b640 1 -- 172.21.15.111:0/3926278613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77401037b0 msgr2=0x7f774019f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:35:09.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.585+0000 7f773dd9b640 1 --2- 172.21.15.111:0/3926278613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77401037b0 0x7f774019f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:09.604 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.585+0000 7f773dd9b640 1 -- 172.21.15.111:0/3926278613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7740101bf0 msgr2=0x7f7740197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:35:09.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.585+0000 7f773dd9b640 1 --2- 172.21.15.111:0/3926278613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7740101bf0 0x7f7740197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:09.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.585+0000 7f773dd9b640 1 -- 172.21.15.111:0/3926278613 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774019fb60 con 0x7f7740102df0 2024-09-10T08:35:09.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.585+0000 7f773e59c640 1 --2- 172.21.15.111:0/3926278613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7740101bf0 0x7f7740197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:35:09.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.586+0000 7f773dd9b640 1 --2- 172.21.15.111:0/3926278613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7740102df0 0x7f7740198470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f773400b4a0 tx=0x7f773400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:35:09.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.586+0000 7f772f7fe640 1 -- 172.21.15.111:0/3926278613 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734014070 con 0x7f7740102df0 2024-09-10T08:35:09.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.586+0000 7f7744cef640 1 -- 172.21.15.111:0/3926278613 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774019fe50 con 0x7f7740102df0 2024-09-10T08:35:09.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.586+0000 7f772f7fe640 1 -- 172.21.15.111:0/3926278613 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7734010af0 con 0x7f7740102df0 2024-09-10T08:35:09.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.586+0000 7f772f7fe640 1 -- 172.21.15.111:0/3926278613 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734017d10 con 0x7f7740102df0 2024-09-10T08:35:09.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.586+0000 7f7744cef640 1 -- 172.21.15.111:0/3926278613 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77401a03d0 con 0x7f7740102df0 2024-09-10T08:35:09.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.587+0000 7f7744cef640 1 -- 172.21.15.111:0/3926278613 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7740108ff0 con 0x7f7740102df0 2024-09-10T08:35:09.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.588+0000 7f772f7fe640 1 -- 172.21.15.111:0/3926278613 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7734017070 con 0x7f7740102df0 2024-09-10T08:35:09.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.588+0000 7f772f7fe640 1 --2- 172.21.15.111:0/3926278613 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f77280a2bb0 0x7f77280a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:35:09.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.589+0000 7f772f7fe640 1 -- 172.21.15.111:0/3926278613 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5981+0+0 (secure 0 0 0) 0x7f77340cdb20 con 0x7f7740102df0 2024-09-10T08:35:09.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.589+0000 7f773e59c640 1 --2- 172.21.15.111:0/3926278613 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f77280a2bb0 0x7f77280a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:35:09.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.589+0000 7f773e59c640 1 --2- 172.21.15.111:0/3926278613 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f77280a2bb0 0x7f77280a5070 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7730006010 tx=0x7f7730032dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:35:09.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.593+0000 7f772f7fe640 1 -- 172.21.15.111:0/3926278613 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f773409b3a0 con 0x7f7740102df0 2024-09-10T08:35:09.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:09.775+0000 7f7744cef640 1 -- 172.21.15.111:0/3926278613 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7740062630 con 0x7f77280a2bb0 2024-09-10T08:35:09.776 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:09 smithi111 ceph-mon[35220]: from='osd.9 [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-10T08:35:09.776 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:09 smithi111 ceph-mon[35220]: osdmap e65: 10 total, 9 up, 10 in 2024-09-10T08:35:09.776 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:09 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:35:09.776 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:09 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:35:09.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:09 smithi149 ceph-mon[36167]: from='osd.9 [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-10T08:35:09.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:09 smithi149 ceph-mon[36167]: osdmap e65: 10 total, 9 up, 10 in 2024-09-10T08:35:09.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:09 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:35:09.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:09 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:35:10.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:09 smithi089 ceph-mon[30757]: from='osd.9 [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-10T08:35:10.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:09 smithi089 ceph-mon[30757]: osdmap e65: 10 total, 9 up, 10 in 2024-09-10T08:35:10.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:09 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:35:10.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:09 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:35:10.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:10 smithi111 ceph-mon[35220]: purged_snaps scrub starts 2024-09-10T08:35:10.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:10 smithi111 ceph-mon[35220]: purged_snaps scrub ok 2024-09-10T08:35:10.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:10 smithi111 ceph-mon[35220]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:35:10.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:10 smithi111 ceph-mon[35220]: osd.9 [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040] boot 2024-09-10T08:35:10.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:10 smithi111 ceph-mon[35220]: osdmap e66: 10 total, 10 up, 10 in 2024-09-10T08:35:10.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:35:10.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:35:10.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:35:10.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:10.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:10.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:10.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:10.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:10.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:10.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:10.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:10.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:10.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:35:10.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:10.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:10 smithi149 ceph-mon[36167]: purged_snaps scrub starts 2024-09-10T08:35:10.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:10 smithi149 ceph-mon[36167]: purged_snaps scrub ok 2024-09-10T08:35:10.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:10 smithi149 ceph-mon[36167]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:35:10.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:10 smithi149 ceph-mon[36167]: osd.9 [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040] boot 2024-09-10T08:35:10.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:10 smithi149 ceph-mon[36167]: osdmap e66: 10 total, 10 up, 10 in 2024-09-10T08:35:10.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:35:10.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:35:10.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:35:10.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:10.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:10.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:10.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:10.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:10.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:10.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:10.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:10.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:10.972 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:35:10.972 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:11.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:10 smithi089 ceph-mon[30757]: purged_snaps scrub starts 2024-09-10T08:35:11.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:10 smithi089 ceph-mon[30757]: purged_snaps scrub ok 2024-09-10T08:35:11.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:10 smithi089 ceph-mon[30757]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:35:11.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:10 smithi089 ceph-mon[30757]: osd.9 [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040] boot 2024-09-10T08:35:11.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:10 smithi089 ceph-mon[30757]: osdmap e66: 10 total, 10 up, 10 in 2024-09-10T08:35:11.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:35:11.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:35:11.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:35:11.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:11.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:11.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:11.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:11.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:11.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:11.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:11.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:11.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:11.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:35:11.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:11.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:11 smithi111 ceph-mon[35220]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:35:11.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:11 smithi111 ceph-mon[35220]: Detected new or changed devices on smithi089 2024-09-10T08:35:11.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:11 smithi111 ceph-mon[35220]: Adjusting osd_memory_target on smithi089 to 4272M 2024-09-10T08:35:11.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:11 smithi111 ceph-mon[35220]: osdmap e67: 10 total, 10 up, 10 in 2024-09-10T08:35:11.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:11 smithi149 ceph-mon[36167]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:35:11.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:11 smithi149 ceph-mon[36167]: Detected new or changed devices on smithi089 2024-09-10T08:35:11.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:11 smithi149 ceph-mon[36167]: Adjusting osd_memory_target on smithi089 to 4272M 2024-09-10T08:35:11.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:11 smithi149 ceph-mon[36167]: osdmap e67: 10 total, 10 up, 10 in 2024-09-10T08:35:12.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:11 smithi089 ceph-mon[30757]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:35:12.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:11 smithi089 ceph-mon[30757]: Detected new or changed devices on smithi089 2024-09-10T08:35:12.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:11 smithi089 ceph-mon[30757]: Adjusting osd_memory_target on smithi089 to 4272M 2024-09-10T08:35:12.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:11 smithi089 ceph-mon[30757]: osdmap e67: 10 total, 10 up, 10 in 2024-09-10T08:35:12.775 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:12 smithi111 ceph-mon[35220]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:12.775 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:12 smithi111 ceph-mon[35220]: from='client.? 172.21.15.111:0/725514438' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8f29890-3064-4b67-b649-467a0ac86db7"}]: dispatch 2024-09-10T08:35:12.775 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:12 smithi111 ceph-mon[35220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8f29890-3064-4b67-b649-467a0ac86db7"}]: dispatch 2024-09-10T08:35:12.775 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:12 smithi111 ceph-mon[35220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8f29890-3064-4b67-b649-467a0ac86db7"}]': finished 2024-09-10T08:35:12.775 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:12 smithi111 ceph-mon[35220]: osdmap e68: 11 total, 10 up, 11 in 2024-09-10T08:35:12.775 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:12 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:35:12.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:12 smithi149 ceph-mon[36167]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:12.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:12 smithi149 ceph-mon[36167]: from='client.? 172.21.15.111:0/725514438' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8f29890-3064-4b67-b649-467a0ac86db7"}]: dispatch 2024-09-10T08:35:12.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:12 smithi149 ceph-mon[36167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8f29890-3064-4b67-b649-467a0ac86db7"}]: dispatch 2024-09-10T08:35:12.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:12 smithi149 ceph-mon[36167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8f29890-3064-4b67-b649-467a0ac86db7"}]': finished 2024-09-10T08:35:12.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:12 smithi149 ceph-mon[36167]: osdmap e68: 11 total, 10 up, 11 in 2024-09-10T08:35:12.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:12 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:35:13.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:12 smithi089 ceph-mon[30757]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:13.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:12 smithi089 ceph-mon[30757]: from='client.? 172.21.15.111:0/725514438' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8f29890-3064-4b67-b649-467a0ac86db7"}]: dispatch 2024-09-10T08:35:13.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:12 smithi089 ceph-mon[30757]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8f29890-3064-4b67-b649-467a0ac86db7"}]: dispatch 2024-09-10T08:35:13.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:12 smithi089 ceph-mon[30757]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8f29890-3064-4b67-b649-467a0ac86db7"}]': finished 2024-09-10T08:35:13.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:12 smithi089 ceph-mon[30757]: osdmap e68: 11 total, 10 up, 11 in 2024-09-10T08:35:13.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:12 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:35:13.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:13 smithi149 ceph-mon[36167]: from='client.? 172.21.15.111:0/1510364250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:35:14.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:13 smithi111 ceph-mon[35220]: from='client.? 172.21.15.111:0/1510364250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:35:14.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:13 smithi089 ceph-mon[30757]: from='client.? 172.21.15.111:0/1510364250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:35:14.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:14 smithi149 ceph-mon[36167]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:15.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:14 smithi111 ceph-mon[35220]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:15.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:14 smithi089 ceph-mon[30757]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:16.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:16 smithi149 ceph-mon[36167]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:17.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:16 smithi111 ceph-mon[35220]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:17.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:16 smithi089 ceph-mon[30757]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:18.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:18 smithi111 ceph-mon[35220]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:18.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:18 smithi149 ceph-mon[36167]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:19.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:18 smithi089 ceph-mon[30757]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:20.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:20 smithi111 ceph-mon[35220]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:20.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:20 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T08:35:20.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:20 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:20.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:20 smithi149 ceph-mon[36167]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:20.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:20 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T08:35:20.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:20 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:21.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:20 smithi089 ceph-mon[30757]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:21.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:20 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T08:35:21.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:20 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:21.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:21 smithi149 ceph-mon[36167]: Deploying daemon osd.10 on smithi111 2024-09-10T08:35:21.986 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:21 smithi111 ceph-mon[35220]: Deploying daemon osd.10 on smithi111 2024-09-10T08:35:22.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:21 smithi089 ceph-mon[30757]: Deploying daemon osd.10 on smithi111 2024-09-10T08:35:22.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:22 smithi149 ceph-mon[36167]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:22.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:22 smithi111 ceph-mon[35220]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:23.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:22 smithi089 ceph-mon[30757]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:24.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:24 smithi149 ceph-mon[36167]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:25.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:24 smithi111 ceph-mon[35220]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:25.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:24 smithi089 ceph-mon[30757]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:26.694 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:26 smithi111 ceph-mon[35220]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:26.694 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:35:26.694 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:26.694 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:26.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:26 smithi149 ceph-mon[36167]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:26.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:26 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:35:26.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:26 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:26.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:26 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:27.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:26 smithi089 ceph-mon[30757]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:27.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:26 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:35:27.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:26 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:27.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:26 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:28.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:28 smithi149 ceph-mon[36167]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:29.024 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:28 smithi111 ceph-mon[35220]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:29.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:28 smithi089 ceph-mon[30757]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:30.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:30.062+0000 7f772f7fe640 1 -- 172.21.15.111:0/3926278613 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7740062630 con 0x7f77280a2bb0 2024-09-10T08:35:30.062 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 10 on host 'smithi111' 2024-09-10T08:35:30.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:30.065+0000 7f7744cef640 1 -- 172.21.15.111:0/3926278613 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f77280a2bb0 msgr2=0x7f77280a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:35:30.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:30.065+0000 7f7744cef640 1 --2- 172.21.15.111:0/3926278613 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f77280a2bb0 0x7f77280a5070 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7730006010 tx=0x7f7730032dd0 comp rx=0 tx=0).stop 2024-09-10T08:35:30.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:30.065+0000 7f7744cef640 1 -- 172.21.15.111:0/3926278613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7740102df0 msgr2=0x7f7740198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:35:30.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:30.065+0000 7f7744cef640 1 --2- 172.21.15.111:0/3926278613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7740102df0 0x7f7740198470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f773400b4a0 tx=0x7f773400b960 comp rx=0 tx=0).stop 2024-09-10T08:35:30.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:30.065+0000 7f7744cef640 1 -- 172.21.15.111:0/3926278613 shutdown_connections 2024-09-10T08:35:30.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:30.065+0000 7f7744cef640 1 --2- 172.21.15.111:0/3926278613 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f77280a2bb0 0x7f77280a5070 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:30.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:30.065+0000 7f7744cef640 1 --2- 172.21.15.111:0/3926278613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77401037b0 0x7f774019f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:30.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:30.065+0000 7f7744cef640 1 --2- 172.21.15.111:0/3926278613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7740102df0 0x7f7740198470 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:30.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:30.065+0000 7f7744cef640 1 --2- 172.21.15.111:0/3926278613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7740101bf0 0x7f7740197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:30.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:30.065+0000 7f7744cef640 1 -- 172.21.15.111:0/3926278613 >> 172.21.15.111:0/3926278613 conn(0x7f77400fd3c0 msgr2=0x7f77401061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:35:30.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:30.065+0000 7f7744cef640 1 -- 172.21.15.111:0/3926278613 shutdown_connections 2024-09-10T08:35:30.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-10T08:35:30.065+0000 7f7744cef640 1 -- 172.21.15.111:0/3926278613 wait complete. 2024-09-10T08:35:30.373 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:30.374 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:30.374 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:30.374 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:35:30.374 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:30.374 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:30 smithi111 ceph-mon[35220]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:30.374 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:35:30.374 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:30.374 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:30.693 DEBUG:teuthology.orchestra.run.smithi111:osd.10> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.10.service 2024-09-10T08:35:30.696 INFO:tasks.cephadm:Deploying osd.11 on smithi149 with /dev/vg_nvme/lv_1... 2024-09-10T08:35:30.696 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T08:35:30.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:30.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:30.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:30.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:35:30.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:30.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:30 smithi149 ceph-mon[36167]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:30.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:35:30.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:30.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:30.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:30.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:30.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:30.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:35:30.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:30.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:30 smithi089 ceph-mon[30757]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:30.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:35:30.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:30.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:30.948 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:35:32.546 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:35:32 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:35:32.381+0000 7ff7a66ae640 -1 osd.10 0 log_to_monitors true 2024-09-10T08:35:32.842 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:32 smithi149 ceph-mon[36167]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:32.842 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:32 smithi149 ceph-mon[36167]: from='osd.10 [v2:172.21.15.111:6824/2328678744,v1:172.21.15.111:6825/2328678744]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T08:35:32.842 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:32 smithi149 ceph-mon[36167]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T08:35:32.914 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:32 smithi111 ceph-mon[35220]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:32.914 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:32 smithi111 ceph-mon[35220]: from='osd.10 [v2:172.21.15.111:6824/2328678744,v1:172.21.15.111:6825/2328678744]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T08:35:32.915 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:32 smithi111 ceph-mon[35220]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T08:35:33.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:32 smithi089 ceph-mon[30757]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:33.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:32 smithi089 ceph-mon[30757]: from='osd.10 [v2:172.21.15.111:6824/2328678744,v1:172.21.15.111:6825/2328678744]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T08:35:33.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:32 smithi089 ceph-mon[30757]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T08:35:33.567 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-10T08:35:33.594 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_1 2024-09-10T08:35:33.869 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:33 smithi149 ceph-mon[36167]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T08:35:33.870 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:33 smithi149 ceph-mon[36167]: osdmap e69: 11 total, 10 up, 11 in 2024-09-10T08:35:33.870 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:33 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:35:33.870 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:33 smithi149 ceph-mon[36167]: from='osd.10 [v2:172.21.15.111:6824/2328678744,v1:172.21.15.111:6825/2328678744]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:35:33.870 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:33 smithi149 ceph-mon[36167]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:35:33.896 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:35:33.944 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:33 smithi089 ceph-mon[30757]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T08:35:33.945 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:33 smithi089 ceph-mon[30757]: osdmap e69: 11 total, 10 up, 11 in 2024-09-10T08:35:33.945 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:33 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:35:33.945 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:33 smithi089 ceph-mon[30757]: from='osd.10 [v2:172.21.15.111:6824/2328678744,v1:172.21.15.111:6825/2328678744]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:35:33.945 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:33 smithi089 ceph-mon[30757]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:35:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:33 smithi111 ceph-mon[35220]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T08:35:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:33 smithi111 ceph-mon[35220]: osdmap e69: 11 total, 10 up, 11 in 2024-09-10T08:35:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:35:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:33 smithi111 ceph-mon[35220]: from='osd.10 [v2:172.21.15.111:6824/2328678744,v1:172.21.15.111:6825/2328678744]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:35:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:33 smithi111 ceph-mon[35220]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-10T08:35:34.044 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:35:33 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:35:33.616+0000 7ff7a2631640 -1 osd.10 0 waiting for initial osdmap 2024-09-10T08:35:34.044 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:35:33 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:35:33.627+0000 7ff79e459640 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:35:34.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.411+0000 7f51bc277640 1 -- 172.21.15.149:0/3470756337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b41026c0 msgr2=0x7f51b4108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:35:34.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.411+0000 7f51bc277640 1 --2- 172.21.15.149:0/3470756337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b41026c0 0x7f51b4108f50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009ca0 tx=0x7f51ac02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:35:34.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.412+0000 7f51bc277640 1 -- 172.21.15.149:0/3470756337 shutdown_connections 2024-09-10T08:35:34.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.412+0000 7f51bc277640 1 --2- 172.21.15.149:0/3470756337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b41026c0 0x7f51b4108f50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:34.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.412+0000 7f51bc277640 1 --2- 172.21.15.149:0/3470756337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51b4101d00 0x7f51b4102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:34.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.412+0000 7f51bc277640 1 --2- 172.21.15.149:0/3470756337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51b4100b00 0x7f51b4100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:34.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.412+0000 7f51bc277640 1 -- 172.21.15.149:0/3470756337 >> 172.21.15.149:0/3470756337 conn(0x7f51b40fc270 msgr2=0x7f51b40fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:35:34.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.412+0000 7f51bc277640 1 -- 172.21.15.149:0/3470756337 shutdown_connections 2024-09-10T08:35:34.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.412+0000 7f51bc277640 1 -- 172.21.15.149:0/3470756337 wait complete. 2024-09-10T08:35:34.414 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.412+0000 7f51bc277640 1 Processor -- start 2024-09-10T08:35:34.415 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.412+0000 7f51bc277640 1 -- start start 2024-09-10T08:35:34.415 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.412+0000 7f51bc277640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51b4100b00 0x7f51b406b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:35:34.415 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51bc277640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4101d00 0x7f51b419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:35:34.415 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51bc277640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51b41026c0 0x7f51b419f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:35:34.415 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51bc277640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f51b410c760 con 0x7f51b4100b00 2024-09-10T08:35:34.415 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51bc277640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f51b410c5e0 con 0x7f51b41026c0 2024-09-10T08:35:34.415 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51bc277640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f51b410c8e0 con 0x7f51b4101d00 2024-09-10T08:35:34.415 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51b9812640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4101d00 0x7f51b419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:35:34.415 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51b9812640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4101d00 0x7f51b419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:57822/0 (socket says 172.21.15.149:57822) 2024-09-10T08:35:34.415 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51b9812640 1 -- 172.21.15.149:0/3700726577 learned_addr learned my addr 172.21.15.149:0/3700726577 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-09-10T08:35:34.415 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51ba013640 1 --2- 172.21.15.149:0/3700726577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51b4100b00 0x7f51b406b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:35:34.416 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51ba814640 1 --2- 172.21.15.149:0/3700726577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51b41026c0 0x7f51b419f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:35:34.416 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51b9812640 1 -- 172.21.15.149:0/3700726577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51b41026c0 msgr2=0x7f51b419f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:35:34.416 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51b9812640 1 --2- 172.21.15.149:0/3700726577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51b41026c0 0x7f51b419f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:34.416 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51b9812640 1 -- 172.21.15.149:0/3700726577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51b4100b00 msgr2=0x7f51b406b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:35:34.416 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51b9812640 1 --2- 172.21.15.149:0/3700726577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51b4100b00 0x7f51b406b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:34.416 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51b9812640 1 -- 172.21.15.149:0/3700726577 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b419fb60 con 0x7f51b4101d00 2024-09-10T08:35:34.416 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51ba814640 1 --2- 172.21.15.149:0/3700726577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51b41026c0 0x7f51b419f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:35:34.416 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51b9812640 1 --2- 172.21.15.149:0/3700726577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4101d00 0x7f51b419cc30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f51a400d950 tx=0x7f51a400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:35:34.416 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51ab7fe640 1 -- 172.21.15.149:0/3700726577 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a4017070 con 0x7f51b4101d00 2024-09-10T08:35:34.416 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51bc277640 1 -- 172.21.15.149:0/3700726577 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b419fdf0 con 0x7f51b4101d00 2024-09-10T08:35:34.416 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51ab7fe640 1 -- 172.21.15.149:0/3700726577 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51a4009c30 con 0x7f51b4101d00 2024-09-10T08:35:34.416 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51ab7fe640 1 -- 172.21.15.149:0/3700726577 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a400f6b0 con 0x7f51b4101d00 2024-09-10T08:35:34.416 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.413+0000 7f51bc277640 1 -- 172.21.15.149:0/3700726577 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b41a02a0 con 0x7f51b4101d00 2024-09-10T08:35:34.417 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.415+0000 7f51bc277640 1 -- 172.21.15.149:0/3700726577 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b4107fa0 con 0x7f51b4101d00 2024-09-10T08:35:34.417 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.416+0000 7f51ab7fe640 1 -- 172.21.15.149:0/3700726577 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f51a40109d0 con 0x7f51b4101d00 2024-09-10T08:35:34.418 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.416+0000 7f51ab7fe640 1 --2- 172.21.15.149:0/3700726577 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f51a00a2b80 0x7f51a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:35:34.418 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.416+0000 7f51ab7fe640 1 -- 172.21.15.149:0/3700726577 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6414+0+0 (secure 0 0 0) 0x7f51a40ce450 con 0x7f51b4101d00 2024-09-10T08:35:34.418 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.417+0000 7f51ba013640 1 --2- 172.21.15.149:0/3700726577 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f51a00a2b80 0x7f51a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:35:34.418 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.417+0000 7f51ba013640 1 --2- 172.21.15.149:0/3700726577 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f51a00a2b80 0x7f51a00a5040 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f51b0005fd0 tx=0x7f51b000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:35:34.420 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.421+0000 7f51ab7fe640 1 -- 172.21.15.149:0/3700726577 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f51a4009de0 con 0x7f51b4101d00 2024-09-10T08:35:34.613 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:34.612+0000 7f51bc277640 1 -- 172.21.15.149:0/3700726577 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f51b4062630 con 0x7f51a00a2b80 2024-09-10T08:35:34.882 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:34.882 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-10T08:35:34.882 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: osdmap e70: 11 total, 10 up, 11 in 2024-09-10T08:35:34.883 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:35:34.883 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:35:34.883 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: Detected new or changed devices on smithi111 2024-09-10T08:35:34.883 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:34.883 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:34.883 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:34.883 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:34.883 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:34.883 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:34.883 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: Adjusting osd_memory_target on smithi111 to 4272M 2024-09-10T08:35:34.884 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:34.884 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:34.884 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:35:34.884 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:34.884 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: osd.10 [v2:172.21.15.111:6824/2328678744,v1:172.21.15.111:6825/2328678744] boot 2024-09-10T08:35:34.884 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T08:35:34.884 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:35:34.884 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:34 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:35:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-10T08:35:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: osdmap e70: 11 total, 10 up, 11 in 2024-09-10T08:35:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:35:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:35:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: Detected new or changed devices on smithi111 2024-09-10T08:35:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: Adjusting osd_memory_target on smithi111 to 4272M 2024-09-10T08:35:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:35:35.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:35.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: osd.10 [v2:172.21.15.111:6824/2328678744,v1:172.21.15.111:6825/2328678744] boot 2024-09-10T08:35:35.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T08:35:35.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:35:35.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:34 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:35:35.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:35:35.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-10T08:35:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: osdmap e70: 11 total, 10 up, 11 in 2024-09-10T08:35:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:35:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:35:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: Detected new or changed devices on smithi111 2024-09-10T08:35:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: Adjusting osd_memory_target on smithi111 to 4272M 2024-09-10T08:35:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:35.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:35:35.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:35.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: osd.10 [v2:172.21.15.111:6824/2328678744,v1:172.21.15.111:6825/2328678744] boot 2024-09-10T08:35:35.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T08:35:35.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:35:35.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:34 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:35:35.962 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:35 smithi111 ceph-mon[35220]: purged_snaps scrub starts 2024-09-10T08:35:35.962 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:35 smithi111 ceph-mon[35220]: purged_snaps scrub ok 2024-09-10T08:35:35.962 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:35 smithi111 ceph-mon[35220]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:35:35.962 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:35 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:35:35.962 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:35 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:35.962 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:35 smithi111 ceph-mon[35220]: osdmap e72: 11 total, 11 up, 11 in 2024-09-10T08:35:35.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:35 smithi149 ceph-mon[36167]: purged_snaps scrub starts 2024-09-10T08:35:35.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:35 smithi149 ceph-mon[36167]: purged_snaps scrub ok 2024-09-10T08:35:35.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:35 smithi149 ceph-mon[36167]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:35:35.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:35 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:35:35.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:35 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:35.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:35 smithi149 ceph-mon[36167]: osdmap e72: 11 total, 11 up, 11 in 2024-09-10T08:35:36.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:35 smithi089 ceph-mon[30757]: purged_snaps scrub starts 2024-09-10T08:35:36.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:35 smithi089 ceph-mon[30757]: purged_snaps scrub ok 2024-09-10T08:35:36.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:35 smithi089 ceph-mon[30757]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:35:36.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:35 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:35:36.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:35 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:36.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:35 smithi089 ceph-mon[30757]: osdmap e72: 11 total, 11 up, 11 in 2024-09-10T08:35:36.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:36 smithi149 ceph-mon[36167]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:35:36.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:36 smithi149 ceph-mon[36167]: osdmap e73: 11 total, 11 up, 11 in 2024-09-10T08:35:37.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:36 smithi111 ceph-mon[35220]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:35:37.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:36 smithi111 ceph-mon[35220]: osdmap e73: 11 total, 11 up, 11 in 2024-09-10T08:35:37.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:36 smithi089 ceph-mon[30757]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:35:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:36 smithi089 ceph-mon[30757]: osdmap e73: 11 total, 11 up, 11 in 2024-09-10T08:35:37.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:37 smithi149 ceph-mon[36167]: from='client.? 172.21.15.149:0/2218636448' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92182c73-f139-47ca-a0ef-5046ae9b2310"}]: dispatch 2024-09-10T08:35:37.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:37 smithi149 ceph-mon[36167]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92182c73-f139-47ca-a0ef-5046ae9b2310"}]: dispatch 2024-09-10T08:35:37.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:37 smithi149 ceph-mon[36167]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92182c73-f139-47ca-a0ef-5046ae9b2310"}]': finished 2024-09-10T08:35:37.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:37 smithi149 ceph-mon[36167]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T08:35:37.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:37 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:35:38.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:37 smithi111 ceph-mon[35220]: from='client.? 172.21.15.149:0/2218636448' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92182c73-f139-47ca-a0ef-5046ae9b2310"}]: dispatch 2024-09-10T08:35:38.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:37 smithi111 ceph-mon[35220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92182c73-f139-47ca-a0ef-5046ae9b2310"}]: dispatch 2024-09-10T08:35:38.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:37 smithi111 ceph-mon[35220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92182c73-f139-47ca-a0ef-5046ae9b2310"}]': finished 2024-09-10T08:35:38.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:37 smithi111 ceph-mon[35220]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T08:35:38.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:37 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:35:38.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:37 smithi089 ceph-mon[30757]: from='client.? 172.21.15.149:0/2218636448' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92182c73-f139-47ca-a0ef-5046ae9b2310"}]: dispatch 2024-09-10T08:35:38.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:37 smithi089 ceph-mon[30757]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92182c73-f139-47ca-a0ef-5046ae9b2310"}]: dispatch 2024-09-10T08:35:38.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:37 smithi089 ceph-mon[30757]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92182c73-f139-47ca-a0ef-5046ae9b2310"}]': finished 2024-09-10T08:35:38.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:37 smithi089 ceph-mon[30757]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T08:35:38.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:37 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:35:38.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:38 smithi149 ceph-mon[36167]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:35:38.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:38 smithi149 ceph-mon[36167]: from='client.? 172.21.15.149:0/3412730863' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:35:38.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:38 smithi149 ceph-mon[36167]: osdmap e75: 12 total, 11 up, 12 in 2024-09-10T08:35:38.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:38 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:35:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:38 smithi111 ceph-mon[35220]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:35:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:38 smithi111 ceph-mon[35220]: from='client.? 172.21.15.149:0/3412730863' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:35:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:38 smithi111 ceph-mon[35220]: osdmap e75: 12 total, 11 up, 12 in 2024-09-10T08:35:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:38 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:35:39.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:38 smithi089 ceph-mon[30757]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:35:39.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:38 smithi089 ceph-mon[30757]: from='client.? 172.21.15.149:0/3412730863' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:35:39.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:38 smithi089 ceph-mon[30757]: osdmap e75: 12 total, 11 up, 12 in 2024-09-10T08:35:39.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:38 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:35:40.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:40 smithi149 ceph-mon[36167]: osdmap e76: 12 total, 11 up, 12 in 2024-09-10T08:35:40.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:40 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:35:40.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:40 smithi149 ceph-mon[36167]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:35:40.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:40 smithi111 ceph-mon[35220]: osdmap e76: 12 total, 11 up, 12 in 2024-09-10T08:35:40.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:40 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:35:40.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:40 smithi111 ceph-mon[35220]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:35:40.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:40 smithi089 ceph-mon[30757]: osdmap e76: 12 total, 11 up, 12 in 2024-09-10T08:35:40.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:40 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:35:40.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:40 smithi089 ceph-mon[30757]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:35:42.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:42 smithi149 ceph-mon[36167]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-10T08:35:43.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:42 smithi111 ceph-mon[35220]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-10T08:35:43.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:42 smithi089 ceph-mon[30757]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-10T08:35:44.783 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:44 smithi149 ceph-mon[36167]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-10T08:35:45.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:44 smithi111 ceph-mon[35220]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-10T08:35:45.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:44 smithi089 ceph-mon[30757]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-10T08:35:46.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:46 smithi149 ceph-mon[36167]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T08:35:46.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T08:35:46.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:47.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:46 smithi111 ceph-mon[35220]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T08:35:47.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T08:35:47.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:46 smithi089 ceph-mon[30757]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T08:35:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T08:35:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:47.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:47 smithi149 ceph-mon[36167]: Deploying daemon osd.11 on smithi149 2024-09-10T08:35:48.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:47 smithi111 ceph-mon[35220]: Deploying daemon osd.11 on smithi149 2024-09-10T08:35:48.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:47 smithi089 ceph-mon[30757]: Deploying daemon osd.11 on smithi149 2024-09-10T08:35:48.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:48 smithi149 ceph-mon[36167]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T08:35:49.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:48 smithi111 ceph-mon[35220]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T08:35:49.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:48 smithi089 ceph-mon[30757]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T08:35:50.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:50 smithi149 ceph-mon[36167]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-10T08:35:51.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:50 smithi111 ceph-mon[35220]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-10T08:35:51.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:50 smithi089 ceph-mon[30757]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-10T08:35:52.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:52 smithi149 ceph-mon[36167]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:35:52.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:52 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:35:52.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:52 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:52.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:52 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:53.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:52 smithi111 ceph-mon[35220]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:35:53.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:52 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:35:53.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:52 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:53.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:52 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:53.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:52 smithi089 ceph-mon[30757]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:35:53.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:52 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:35:53.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:52 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:53.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:52 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:54.801 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:54 smithi149 ceph-mon[36167]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:35:55.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:54 smithi111 ceph-mon[35220]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:35:55.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:54 smithi089 ceph-mon[30757]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:35:55.516 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:55.515+0000 7f51ab7fe640 1 -- 172.21.15.149:0/3700726577 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f51b4062630 con 0x7f51a00a2b80 2024-09-10T08:35:55.517 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 11 on host 'smithi149' 2024-09-10T08:35:55.520 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:55.518+0000 7f51bc277640 1 -- 172.21.15.149:0/3700726577 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f51a00a2b80 msgr2=0x7f51a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:35:55.520 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:55.518+0000 7f51bc277640 1 --2- 172.21.15.149:0/3700726577 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f51a00a2b80 0x7f51a00a5040 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f51b0005fd0 tx=0x7f51b000b410 comp rx=0 tx=0).stop 2024-09-10T08:35:55.520 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:55.518+0000 7f51bc277640 1 -- 172.21.15.149:0/3700726577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4101d00 msgr2=0x7f51b419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:35:55.520 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:55.518+0000 7f51bc277640 1 --2- 172.21.15.149:0/3700726577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4101d00 0x7f51b419cc30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f51a400d950 tx=0x7f51a400de10 comp rx=0 tx=0).stop 2024-09-10T08:35:55.520 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:55.519+0000 7f51bc277640 1 -- 172.21.15.149:0/3700726577 shutdown_connections 2024-09-10T08:35:55.520 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:55.519+0000 7f51bc277640 1 --2- 172.21.15.149:0/3700726577 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f51a00a2b80 0x7f51a00a5040 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:55.520 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:55.519+0000 7f51bc277640 1 --2- 172.21.15.149:0/3700726577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51b41026c0 0x7f51b419f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:55.520 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:55.519+0000 7f51bc277640 1 --2- 172.21.15.149:0/3700726577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b4101d00 0x7f51b419cc30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:55.521 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:55.519+0000 7f51bc277640 1 --2- 172.21.15.149:0/3700726577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51b4100b00 0x7f51b406b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:55.521 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:55.519+0000 7f51bc277640 1 -- 172.21.15.149:0/3700726577 >> 172.21.15.149:0/3700726577 conn(0x7f51b40fc270 msgr2=0x7f51b4105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:35:55.521 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:55.519+0000 7f51bc277640 1 -- 172.21.15.149:0/3700726577 shutdown_connections 2024-09-10T08:35:55.521 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:35:55.519+0000 7f51bc277640 1 -- 172.21.15.149:0/3700726577 wait complete. 2024-09-10T08:35:56.070 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:55 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:56.070 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:55 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:56.071 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:55 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:56.071 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:55 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:35:56.071 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:55 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:56.071 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:55 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:35:56.071 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:55 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:56.071 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:55 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:56.071 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:55 smithi111 ceph-mon[35220]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:35:56.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:55 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:56.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:55 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:56.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:55 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:56.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:55 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:35:56.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:55 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:56.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:55 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:35:56.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:55 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:56.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:55 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:56.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:55 smithi149 ceph-mon[36167]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:35:56.254 DEBUG:teuthology.orchestra.run.smithi149:osd.11> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.11.service 2024-09-10T08:35:56.256 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-10T08:35:56.256 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T08:35:56.302 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:55 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:56.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:55 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:56.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:55 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:56.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:55 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:35:56.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:55 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:56.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:55 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:35:56.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:55 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:56.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:55 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:56.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:55 smithi089 ceph-mon[30757]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:35:56.638 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:35:57.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.210+0000 7f6a0aa00640 1 -- 172.21.15.89:0/4010194395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a04105810 msgr2=0x7f6a0410c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:35:57.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.210+0000 7f6a0aa00640 1 --2- 172.21.15.89:0/4010194395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a04105810 0x7f6a0410c0a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009ca0 tx=0x7f69fc02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:35:57.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.210+0000 7f6a0aa00640 1 -- 172.21.15.89:0/4010194395 shutdown_connections 2024-09-10T08:35:57.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.210+0000 7f6a0aa00640 1 --2- 172.21.15.89:0/4010194395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a04105810 0x7f6a0410c0a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:57.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.210+0000 7f6a0aa00640 1 --2- 172.21.15.89:0/4010194395 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a04104e50 0x7f6a041052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:57.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.210+0000 7f6a0aa00640 1 --2- 172.21.15.89:0/4010194395 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a04103c50 0x7f6a04104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:57.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.210+0000 7f6a0aa00640 1 -- 172.21.15.89:0/4010194395 >> 172.21.15.89:0/4010194395 conn(0x7f6a040ff3e0 msgr2=0x7f6a04101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:35:57.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.210+0000 7f6a0aa00640 1 -- 172.21.15.89:0/4010194395 shutdown_connections 2024-09-10T08:35:57.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.210+0000 7f6a0aa00640 1 -- 172.21.15.89:0/4010194395 wait complete. 2024-09-10T08:35:57.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.211+0000 7f6a0aa00640 1 Processor -- start 2024-09-10T08:35:57.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.211+0000 7f6a0aa00640 1 -- start start 2024-09-10T08:35:57.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.211+0000 7f6a0aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a04103c50 0x7f6a04199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:35:57.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.211+0000 7f6a0aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a04104e50 0x7f6a0419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:35:57.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.211+0000 7f6a0aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a04105810 0x7f6a041a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:35:57.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.211+0000 7f6a0aa00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0410f8d0 con 0x7f6a04105810 2024-09-10T08:35:57.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.211+0000 7f6a0aa00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0410f750 con 0x7f6a04104e50 2024-09-10T08:35:57.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.211+0000 7f6a0aa00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0410fa50 con 0x7f6a04103c50 2024-09-10T08:35:57.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.211+0000 7f6a09600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a04103c50 0x7f6a04199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:35:57.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.211+0000 7f6a09600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a04103c50 0x7f6a04199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:47610/0 (socket says 172.21.15.89:47610) 2024-09-10T08:35:57.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.211+0000 7f6a09600640 1 -- 172.21.15.89:0/3359977649 learned_addr learned my addr 172.21.15.89:0/3359977649 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:35:57.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.211+0000 7f6a08c00640 1 --2- 172.21.15.89:0/3359977649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a04104e50 0x7f6a0419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:35:57.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.212+0000 7f6a0a000640 1 --2- 172.21.15.89:0/3359977649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a04105810 0x7f6a041a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:35:57.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.212+0000 7f6a09600640 1 -- 172.21.15.89:0/3359977649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a04104e50 msgr2=0x7f6a0419a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:35:57.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.212+0000 7f6a09600640 1 --2- 172.21.15.89:0/3359977649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a04104e50 0x7f6a0419a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:57.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.212+0000 7f6a09600640 1 -- 172.21.15.89:0/3359977649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a04105810 msgr2=0x7f6a041a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:35:57.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.212+0000 7f6a09600640 1 --2- 172.21.15.89:0/3359977649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a04105810 0x7f6a041a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:57.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.212+0000 7f6a09600640 1 -- 172.21.15.89:0/3359977649 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a041a1ad0 con 0x7f6a04103c50 2024-09-10T08:35:57.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.212+0000 7f6a09600640 1 --2- 172.21.15.89:0/3359977649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a04103c50 0x7f6a04199ea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f69f800d950 tx=0x7f69f800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:35:57.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.212+0000 7f6a02000640 1 -- 172.21.15.89:0/3359977649 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8017020 con 0x7f6a04103c50 2024-09-10T08:35:57.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.212+0000 7f6a0aa00640 1 -- 172.21.15.89:0/3359977649 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a041a1dc0 con 0x7f6a04103c50 2024-09-10T08:35:57.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.212+0000 7f6a02000640 1 -- 172.21.15.89:0/3359977649 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69f8009c30 con 0x7f6a04103c50 2024-09-10T08:35:57.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.212+0000 7f6a02000640 1 -- 172.21.15.89:0/3359977649 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f800f6b0 con 0x7f6a04103c50 2024-09-10T08:35:57.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.213+0000 7f6a0aa00640 1 -- 172.21.15.89:0/3359977649 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a041a2220 con 0x7f6a04103c50 2024-09-10T08:35:57.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.214+0000 7f6a0aa00640 1 -- 172.21.15.89:0/3359977649 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a0410af80 con 0x7f6a04103c50 2024-09-10T08:35:57.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.215+0000 7f6a02000640 1 -- 172.21.15.89:0/3359977649 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f69f801a020 con 0x7f6a04103c50 2024-09-10T08:35:57.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.216+0000 7f6a02000640 1 --2- 172.21.15.89:0/3359977649 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f69ec0a2bb0 0x7f69ec0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:35:57.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.217+0000 7f6a02000640 1 -- 172.21.15.89:0/3359977649 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6809+0+0 (secure 0 0 0) 0x7f69f80cea50 con 0x7f6a04103c50 2024-09-10T08:35:57.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.217+0000 7f6a08c00640 1 --2- 172.21.15.89:0/3359977649 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f69ec0a2bb0 0x7f69ec0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:35:57.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.217+0000 7f6a08c00640 1 --2- 172.21.15.89:0/3359977649 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f69ec0a2bb0 0x7f69ec0a5070 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6a0419b3c0 tx=0x7f69f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:35:57.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.221+0000 7f6a02000640 1 -- 172.21.15.89:0/3359977649 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f69f8021020 con 0x7f6a04103c50 2024-09-10T08:35:57.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.397+0000 7f6a0aa00640 1 -- 172.21.15.89:0/3359977649 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f6a04062ef0 con 0x7f6a04103c50 2024-09-10T08:35:57.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.397+0000 7f6a02000640 1 -- 172.21.15.89:0/3359977649 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7f69f809bef0 con 0x7f6a04103c50 2024-09-10T08:35:57.398 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:35:57.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.400+0000 7f6a0aa00640 1 -- 172.21.15.89:0/3359977649 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f69ec0a2bb0 msgr2=0x7f69ec0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:35:57.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.400+0000 7f6a0aa00640 1 --2- 172.21.15.89:0/3359977649 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f69ec0a2bb0 0x7f69ec0a5070 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6a0419b3c0 tx=0x7f69f0005f90 comp rx=0 tx=0).stop 2024-09-10T08:35:57.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.400+0000 7f6a0aa00640 1 -- 172.21.15.89:0/3359977649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a04103c50 msgr2=0x7f6a04199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:35:57.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.400+0000 7f6a0aa00640 1 --2- 172.21.15.89:0/3359977649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a04103c50 0x7f6a04199ea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f69f800d950 tx=0x7f69f800de10 comp rx=0 tx=0).stop 2024-09-10T08:35:57.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.400+0000 7f6a0aa00640 1 -- 172.21.15.89:0/3359977649 shutdown_connections 2024-09-10T08:35:57.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.400+0000 7f6a0aa00640 1 --2- 172.21.15.89:0/3359977649 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f69ec0a2bb0 0x7f69ec0a5070 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:57.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.400+0000 7f6a0aa00640 1 --2- 172.21.15.89:0/3359977649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a04105810 0x7f6a041a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:57.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.400+0000 7f6a0aa00640 1 --2- 172.21.15.89:0/3359977649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a04104e50 0x7f6a0419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:57.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.400+0000 7f6a0aa00640 1 --2- 172.21.15.89:0/3359977649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a04103c50 0x7f6a04199ea0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:35:57.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.400+0000 7f6a0aa00640 1 -- 172.21.15.89:0/3359977649 >> 172.21.15.89:0/3359977649 conn(0x7f6a040ff3e0 msgr2=0x7f6a04108150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:35:57.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.400+0000 7f6a0aa00640 1 -- 172.21.15.89:0/3359977649 shutdown_connections 2024-09-10T08:35:57.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:35:57.400+0000 7f6a0aa00640 1 -- 172.21.15.89:0/3359977649 wait complete. 2024-09-10T08:35:57.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:57 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3359977649' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:35:57.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:57 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3359977649' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:35:57.795 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:57 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3359977649' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:35:58.295 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1725957334,"num_in_osds":12,"osd_in_since":1725957337,"num_remapped_pgs":0} 2024-09-10T08:35:58.471 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:35:57 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:35:57.993+0000 7f82586c4640 -1 osd.11 0 log_to_monitors true 2024-09-10T08:35:58.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:58 smithi111 ceph-mon[35220]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:35:58.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:58 smithi111 ceph-mon[35220]: from='osd.11 [v2:172.21.15.149:6824/3130508827,v1:172.21.15.149:6825/3130508827]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T08:35:58.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:58 smithi111 ceph-mon[35220]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T08:35:58.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:58 smithi089 ceph-mon[30757]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:35:58.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:58 smithi089 ceph-mon[30757]: from='osd.11 [v2:172.21.15.149:6824/3130508827,v1:172.21.15.149:6825/3130508827]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T08:35:58.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:58 smithi089 ceph-mon[30757]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T08:35:58.884 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:58 smithi149 ceph-mon[36167]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:35:58.885 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:58 smithi149 ceph-mon[36167]: from='osd.11 [v2:172.21.15.149:6824/3130508827,v1:172.21.15.149:6825/3130508827]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T08:35:58.885 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:58 smithi149 ceph-mon[36167]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T08:35:59.297 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T08:35:59.653 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:59 smithi089 ceph-mon[30757]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T08:35:59.653 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:59 smithi089 ceph-mon[30757]: osdmap e77: 12 total, 11 up, 12 in 2024-09-10T08:35:59.653 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:35:59.653 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:59 smithi089 ceph-mon[30757]: from='osd.11 [v2:172.21.15.149:6824/3130508827,v1:172.21.15.149:6825/3130508827]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:35:59.653 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:59 smithi089 ceph-mon[30757]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:35:59.653 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:59.653 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:59.654 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:59.654 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:59.654 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:59.654 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:59.654 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:59.654 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:59.654 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:35:59.654 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:35:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:59.688 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:35:59.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:59 smithi111 ceph-mon[35220]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T08:35:59.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:59 smithi111 ceph-mon[35220]: osdmap e77: 12 total, 11 up, 12 in 2024-09-10T08:35:59.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:35:59.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:59 smithi111 ceph-mon[35220]: from='osd.11 [v2:172.21.15.149:6824/3130508827,v1:172.21.15.149:6825/3130508827]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:35:59.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:59 smithi111 ceph-mon[35220]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:35:59.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:59.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:59.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:59.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:59.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:59.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:59.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:59.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:59.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:35:59.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:35:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:59.970 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:35:59 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:35:59.540+0000 7f8254647640 -1 osd.11 0 waiting for initial osdmap 2024-09-10T08:35:59.970 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:35:59 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:35:59.551+0000 7f825046f640 -1 osd.11 78 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:35:59.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:59 smithi149 ceph-mon[36167]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T08:35:59.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:59 smithi149 ceph-mon[36167]: osdmap e77: 12 total, 11 up, 12 in 2024-09-10T08:35:59.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:35:59.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:59 smithi149 ceph-mon[36167]: from='osd.11 [v2:172.21.15.149:6824/3130508827,v1:172.21.15.149:6825/3130508827]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:35:59.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:59 smithi149 ceph-mon[36167]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-09-10T08:35:59.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:59.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:59.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:59.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:59.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:59.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:35:59.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:35:59.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:35:59.972 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:35:59.972 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:35:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:00.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.247+0000 7f1951400640 1 -- 172.21.15.89:0/2647048190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f194c104f60 msgr2=0x7f194c1053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:00.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.247+0000 7f1951400640 1 --2- 172.21.15.89:0/2647048190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f194c104f60 0x7f194c1053e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1938009900 tx=0x7f193802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:36:00.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.247+0000 7f1951400640 1 -- 172.21.15.89:0/2647048190 shutdown_connections 2024-09-10T08:36:00.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.247+0000 7f1951400640 1 --2- 172.21.15.89:0/2647048190 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f194c105920 0x7f194c10c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:00.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.247+0000 7f1951400640 1 --2- 172.21.15.89:0/2647048190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f194c104f60 0x7f194c1053e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:00.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.248+0000 7f1951400640 1 --2- 172.21.15.89:0/2647048190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f194c103e00 0x7f194c104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:00.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.248+0000 7f1951400640 1 -- 172.21.15.89:0/2647048190 >> 172.21.15.89:0/2647048190 conn(0x7f194c0ff5b0 msgr2=0x7f194c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:00.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.248+0000 7f1951400640 1 -- 172.21.15.89:0/2647048190 shutdown_connections 2024-09-10T08:36:00.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.248+0000 7f1951400640 1 -- 172.21.15.89:0/2647048190 wait complete. 2024-09-10T08:36:00.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.249+0000 7f1951400640 1 Processor -- start 2024-09-10T08:36:00.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.249+0000 7f1951400640 1 -- start start 2024-09-10T08:36:00.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.249+0000 7f1951400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f194c103e00 0x7f194c19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:00.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.249+0000 7f1951400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f194c104f60 0x7f194c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:00.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.249+0000 7f1951400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f194c105920 0x7f194c1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.249+0000 7f1951400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f194c10fab0 con 0x7f194c103e00 2024-09-10T08:36:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.249+0000 7f1951400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f194c10f930 con 0x7f194c105920 2024-09-10T08:36:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.249+0000 7f1951400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f194c10fc30 con 0x7f194c104f60 2024-09-10T08:36:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.249+0000 7f194a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f194c103e00 0x7f194c19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.249+0000 7f194a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f194c103e00 0x7f194c19a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45828/0 (socket says 172.21.15.89:45828) 2024-09-10T08:36:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.249+0000 7f194a800640 1 -- 172.21.15.89:0/389751700 learned_addr learned my addr 172.21.15.89:0/389751700 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:36:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.249+0000 7f1949e00640 1 --2- 172.21.15.89:0/389751700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f194c104f60 0x7f194c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.250+0000 7f194b200640 1 --2- 172.21.15.89:0/389751700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f194c105920 0x7f194c1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.250+0000 7f194a800640 1 -- 172.21.15.89:0/389751700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f194c104f60 msgr2=0x7f194c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.250+0000 7f194a800640 1 --2- 172.21.15.89:0/389751700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f194c104f60 0x7f194c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.250+0000 7f194a800640 1 -- 172.21.15.89:0/389751700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f194c105920 msgr2=0x7f194c1a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.250+0000 7f194a800640 1 --2- 172.21.15.89:0/389751700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f194c105920 0x7f194c1a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:00.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.250+0000 7f194a800640 1 -- 172.21.15.89:0/389751700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194c1a1ce0 con 0x7f194c103e00 2024-09-10T08:36:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.250+0000 7f1949e00640 1 --2- 172.21.15.89:0/389751700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f194c104f60 0x7f194c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:36:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.250+0000 7f194a800640 1 --2- 172.21.15.89:0/389751700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f194c103e00 0x7f194c19a0b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f193c00ea60 tx=0x7f193c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.250+0000 7f1943400640 1 -- 172.21.15.89:0/389751700 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c00cde0 con 0x7f194c103e00 2024-09-10T08:36:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.250+0000 7f1943400640 1 -- 172.21.15.89:0/389751700 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f193c0035f0 con 0x7f194c103e00 2024-09-10T08:36:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.250+0000 7f1951400640 1 -- 172.21.15.89:0/389751700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f194c1a1fd0 con 0x7f194c103e00 2024-09-10T08:36:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.250+0000 7f1943400640 1 -- 172.21.15.89:0/389751700 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c010670 con 0x7f194c103e00 2024-09-10T08:36:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.251+0000 7f1951400640 1 -- 172.21.15.89:0/389751700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f194c1a2420 con 0x7f194c103e00 2024-09-10T08:36:00.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.252+0000 7f1951400640 1 -- 172.21.15.89:0/389751700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f194c10b1f0 con 0x7f194c103e00 2024-09-10T08:36:00.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.254+0000 7f1943400640 1 -- 172.21.15.89:0/389751700 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f193c0037a0 con 0x7f194c103e00 2024-09-10T08:36:00.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.255+0000 7f1943400640 1 --2- 172.21.15.89:0/389751700 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f19300a2bb0 0x7f19300a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:00.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.255+0000 7f1943400640 1 -- 172.21.15.89:0/389751700 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6847+0+0 (secure 0 0 0) 0x7f193c0cedf0 con 0x7f194c103e00 2024-09-10T08:36:00.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.255+0000 7f1949e00640 1 --2- 172.21.15.89:0/389751700 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f19300a2bb0 0x7f19300a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:00.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.256+0000 7f1949e00640 1 --2- 172.21.15.89:0/389751700 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f19300a2bb0 0x7f19300a5070 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1938005ab0 tx=0x7f1938000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:00.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.259+0000 7f1943400640 1 -- 172.21.15.89:0/389751700 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f193c09c270 con 0x7f194c103e00 2024-09-10T08:36:00.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.437+0000 7f1951400640 1 -- 172.21.15.89:0/389751700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f194c062e90 con 0x7f194c103e00 2024-09-10T08:36:00.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.437+0000 7f1943400640 1 -- 172.21.15.89:0/389751700 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) ==== 74+0+133 (secure 0 0 0) 0x7f193c09bb70 con 0x7f194c103e00 2024-09-10T08:36:00.438 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:36:00.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.440+0000 7f1951400640 1 -- 172.21.15.89:0/389751700 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f19300a2bb0 msgr2=0x7f19300a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:00.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.440+0000 7f1951400640 1 --2- 172.21.15.89:0/389751700 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f19300a2bb0 0x7f19300a5070 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1938005ab0 tx=0x7f1938000bc0 comp rx=0 tx=0).stop 2024-09-10T08:36:00.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.440+0000 7f1951400640 1 -- 172.21.15.89:0/389751700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f194c103e00 msgr2=0x7f194c19a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:00.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.440+0000 7f1951400640 1 --2- 172.21.15.89:0/389751700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f194c103e00 0x7f194c19a0b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f193c00ea60 tx=0x7f193c00ef20 comp rx=0 tx=0).stop 2024-09-10T08:36:00.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.440+0000 7f1951400640 1 -- 172.21.15.89:0/389751700 shutdown_connections 2024-09-10T08:36:00.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.440+0000 7f1951400640 1 --2- 172.21.15.89:0/389751700 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f19300a2bb0 0x7f19300a5070 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:00.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.440+0000 7f1951400640 1 --2- 172.21.15.89:0/389751700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f194c105920 0x7f194c1a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:00.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.440+0000 7f1951400640 1 --2- 172.21.15.89:0/389751700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f194c104f60 0x7f194c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:00.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.440+0000 7f1951400640 1 --2- 172.21.15.89:0/389751700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f194c103e00 0x7f194c19a0b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:00.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.440+0000 7f1951400640 1 -- 172.21.15.89:0/389751700 >> 172.21.15.89:0/389751700 conn(0x7f194c0ff5b0 msgr2=0x7f194c1083c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:00.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.440+0000 7f1951400640 1 -- 172.21.15.89:0/389751700 shutdown_connections 2024-09-10T08:36:00.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:00.440+0000 7f1951400640 1 -- 172.21.15.89:0/389751700 wait complete. 2024-09-10T08:36:00.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:00 smithi089 ceph-mon[30757]: Detected new or changed devices on smithi149 2024-09-10T08:36:00.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:00 smithi089 ceph-mon[30757]: Adjusting osd_memory_target on smithi149 to 1453M 2024-09-10T08:36:00.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:00 smithi089 ceph-mon[30757]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T08:36:00.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:00 smithi089 ceph-mon[30757]: osdmap e78: 12 total, 11 up, 12 in 2024-09-10T08:36:00.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:00 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:36:00.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:00 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:36:00.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:00 smithi089 ceph-mon[30757]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:36:00.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:00 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/389751700' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:36:00.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:00 smithi149 ceph-mon[36167]: Detected new or changed devices on smithi149 2024-09-10T08:36:00.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:00 smithi149 ceph-mon[36167]: Adjusting osd_memory_target on smithi149 to 1453M 2024-09-10T08:36:00.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:00 smithi149 ceph-mon[36167]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T08:36:00.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:00 smithi149 ceph-mon[36167]: osdmap e78: 12 total, 11 up, 12 in 2024-09-10T08:36:00.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:00 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:36:00.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:00 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:36:00.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:00 smithi149 ceph-mon[36167]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:36:00.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:00 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/389751700' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:36:01.049 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:00 smithi111 ceph-mon[35220]: Detected new or changed devices on smithi149 2024-09-10T08:36:01.050 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:00 smithi111 ceph-mon[35220]: Adjusting osd_memory_target on smithi149 to 1453M 2024-09-10T08:36:01.050 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:00 smithi111 ceph-mon[35220]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-09-10T08:36:01.050 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:00 smithi111 ceph-mon[35220]: osdmap e78: 12 total, 11 up, 12 in 2024-09-10T08:36:01.050 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:00 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:36:01.050 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:00 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:36:01.050 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:00 smithi111 ceph-mon[35220]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:36:01.050 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:00 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/389751700' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:36:01.337 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":78,"num_osds":12,"num_up_osds":11,"osd_up_since":1725957334,"num_in_osds":12,"osd_in_since":1725957337,"num_remapped_pgs":0} 2024-09-10T08:36:01.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:01 smithi089 ceph-mon[30757]: purged_snaps scrub starts 2024-09-10T08:36:01.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:01 smithi089 ceph-mon[30757]: purged_snaps scrub ok 2024-09-10T08:36:01.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:01 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:36:01.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:01 smithi089 ceph-mon[30757]: osd.11 [v2:172.21.15.149:6824/3130508827,v1:172.21.15.149:6825/3130508827] boot 2024-09-10T08:36:01.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:01 smithi089 ceph-mon[30757]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T08:36:01.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:01 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:36:01.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:01 smithi149 ceph-mon[36167]: purged_snaps scrub starts 2024-09-10T08:36:01.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:01 smithi149 ceph-mon[36167]: purged_snaps scrub ok 2024-09-10T08:36:01.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:01 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:36:01.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:01 smithi149 ceph-mon[36167]: osd.11 [v2:172.21.15.149:6824/3130508827,v1:172.21.15.149:6825/3130508827] boot 2024-09-10T08:36:01.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:01 smithi149 ceph-mon[36167]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T08:36:01.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:01 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:36:02.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:01 smithi111 ceph-mon[35220]: purged_snaps scrub starts 2024-09-10T08:36:02.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:01 smithi111 ceph-mon[35220]: purged_snaps scrub ok 2024-09-10T08:36:02.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:01 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:36:02.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:01 smithi111 ceph-mon[35220]: osd.11 [v2:172.21.15.149:6824/3130508827,v1:172.21.15.149:6825/3130508827] boot 2024-09-10T08:36:02.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:01 smithi111 ceph-mon[35220]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T08:36:02.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:01 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:36:02.338 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T08:36:02.704 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:02 smithi089 ceph-mon[30757]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T08:36:02.704 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:02 smithi089 ceph-mon[30757]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:02.714 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:36:02.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:02 smithi149 ceph-mon[36167]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T08:36:02.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:02 smithi149 ceph-mon[36167]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:02 smithi111 ceph-mon[35220]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T08:36:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:02 smithi111 ceph-mon[35220]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.278+0000 7f1685600640 1 -- 172.21.15.89:0/1625743098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1680105810 msgr2=0x7f168010c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.278+0000 7f1685600640 1 --2- 172.21.15.89:0/1625743098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1680105810 0x7f168010c0a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1678009ca0 tx=0x7f167802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:36:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.278+0000 7f1685600640 1 -- 172.21.15.89:0/1625743098 shutdown_connections 2024-09-10T08:36:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.278+0000 7f1685600640 1 --2- 172.21.15.89:0/1625743098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1680105810 0x7f168010c0a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.278+0000 7f1685600640 1 --2- 172.21.15.89:0/1625743098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1680104e50 0x7f16801052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.278+0000 7f1685600640 1 --2- 172.21.15.89:0/1625743098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1680103c50 0x7f1680104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.278+0000 7f1685600640 1 -- 172.21.15.89:0/1625743098 >> 172.21.15.89:0/1625743098 conn(0x7f16800ff3e0 msgr2=0x7f1680101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.278+0000 7f1685600640 1 -- 172.21.15.89:0/1625743098 shutdown_connections 2024-09-10T08:36:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.278+0000 7f1685600640 1 -- 172.21.15.89:0/1625743098 wait complete. 2024-09-10T08:36:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.279+0000 7f1685600640 1 Processor -- start 2024-09-10T08:36:03.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.279+0000 7f1685600640 1 -- start start 2024-09-10T08:36:03.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.280+0000 7f1685600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1680103c50 0x7f1680199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:03.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.280+0000 7f1685600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1680104e50 0x7f168019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:03.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.280+0000 7f1685600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1680105810 0x7f16801a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:03.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.280+0000 7f1685600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f168010f8d0 con 0x7f1680105810 2024-09-10T08:36:03.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.280+0000 7f1685600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f168010f750 con 0x7f1680104e50 2024-09-10T08:36:03.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.280+0000 7f1685600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f168010fa50 con 0x7f1680103c50 2024-09-10T08:36:03.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.280+0000 7f1684c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1680105810 0x7f16801a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:03.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.280+0000 7f167f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1680104e50 0x7f168019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:03.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.280+0000 7f1684c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1680105810 0x7f16801a1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39840/0 (socket says 172.21.15.89:39840) 2024-09-10T08:36:03.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.280+0000 7f1684c00640 1 -- 172.21.15.89:0/2142817204 learned_addr learned my addr 172.21.15.89:0/2142817204 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:36:03.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.280+0000 7f167fe00640 1 --2- 172.21.15.89:0/2142817204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1680103c50 0x7f1680199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:03.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.280+0000 7f167f400640 1 -- 172.21.15.89:0/2142817204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1680103c50 msgr2=0x7f1680199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:03.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.280+0000 7f167f400640 1 --2- 172.21.15.89:0/2142817204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1680103c50 0x7f1680199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:03.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.280+0000 7f167f400640 1 -- 172.21.15.89:0/2142817204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1680105810 msgr2=0x7f16801a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:03.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.280+0000 7f167f400640 1 --2- 172.21.15.89:0/2142817204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1680105810 0x7f16801a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:03.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.280+0000 7f167f400640 1 -- 172.21.15.89:0/2142817204 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16801a1b00 con 0x7f1680104e50 2024-09-10T08:36:03.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.281+0000 7f167f400640 1 --2- 172.21.15.89:0/2142817204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1680104e50 0x7f168019a410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f166c00e9f0 tx=0x7f166c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:03.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.281+0000 7f167cc00640 1 -- 172.21.15.89:0/2142817204 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c00ccc0 con 0x7f1680104e50 2024-09-10T08:36:03.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.281+0000 7f1685600640 1 -- 172.21.15.89:0/2142817204 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16801a1df0 con 0x7f1680104e50 2024-09-10T08:36:03.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.281+0000 7f167cc00640 1 -- 172.21.15.89:0/2142817204 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f166c005020 con 0x7f1680104e50 2024-09-10T08:36:03.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.281+0000 7f167cc00640 1 -- 172.21.15.89:0/2142817204 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c0049e0 con 0x7f1680104e50 2024-09-10T08:36:03.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.281+0000 7f1685600640 1 -- 172.21.15.89:0/2142817204 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16801a2250 con 0x7f1680104e50 2024-09-10T08:36:03.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.282+0000 7f1685600640 1 -- 172.21.15.89:0/2142817204 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f168010b050 con 0x7f1680104e50 2024-09-10T08:36:03.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.284+0000 7f167cc00640 1 -- 172.21.15.89:0/2142817204 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f166c005560 con 0x7f1680104e50 2024-09-10T08:36:03.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.285+0000 7f167cc00640 1 --2- 172.21.15.89:0/2142817204 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f16640a2bb0 0x7f16640a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:03.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.286+0000 7f167cc00640 1 -- 172.21.15.89:0/2142817204 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f166c0ceac0 con 0x7f1680104e50 2024-09-10T08:36:03.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.286+0000 7f167fe00640 1 --2- 172.21.15.89:0/2142817204 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f16640a2bb0 0x7f16640a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:03.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.286+0000 7f167fe00640 1 --2- 172.21.15.89:0/2142817204 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f16640a2bb0 0x7f16640a5070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1680104cb0 tx=0x7f1670005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:03.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.288+0000 7f167cc00640 1 -- 172.21.15.89:0/2142817204 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f166c0051d0 con 0x7f1680104e50 2024-09-10T08:36:03.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.464+0000 7f1685600640 1 -- 172.21.15.89:0/2142817204 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f168002da10 con 0x7f1680104e50 2024-09-10T08:36:03.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.465+0000 7f167cc00640 1 -- 172.21.15.89:0/2142817204 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f166c09be20 con 0x7f1680104e50 2024-09-10T08:36:03.465 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:36:03.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.467+0000 7f1685600640 1 -- 172.21.15.89:0/2142817204 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f16640a2bb0 msgr2=0x7f16640a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.467+0000 7f1685600640 1 --2- 172.21.15.89:0/2142817204 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f16640a2bb0 0x7f16640a5070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1680104cb0 tx=0x7f1670005f90 comp rx=0 tx=0).stop 2024-09-10T08:36:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.467+0000 7f1685600640 1 -- 172.21.15.89:0/2142817204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1680104e50 msgr2=0x7f168019a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.467+0000 7f1685600640 1 --2- 172.21.15.89:0/2142817204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1680104e50 0x7f168019a410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f166c00e9f0 tx=0x7f166c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:36:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.467+0000 7f1685600640 1 -- 172.21.15.89:0/2142817204 shutdown_connections 2024-09-10T08:36:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.467+0000 7f1685600640 1 --2- 172.21.15.89:0/2142817204 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f16640a2bb0 0x7f16640a5070 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.467+0000 7f1685600640 1 --2- 172.21.15.89:0/2142817204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1680105810 0x7f16801a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.467+0000 7f1685600640 1 --2- 172.21.15.89:0/2142817204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1680104e50 0x7f168019a410 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.467+0000 7f1685600640 1 --2- 172.21.15.89:0/2142817204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1680103c50 0x7f1680199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.467+0000 7f1685600640 1 -- 172.21.15.89:0/2142817204 >> 172.21.15.89:0/2142817204 conn(0x7f16800ff3e0 msgr2=0x7f1680108220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.467+0000 7f1685600640 1 -- 172.21.15.89:0/2142817204 shutdown_connections 2024-09-10T08:36:03.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:03.467+0000 7f1685600640 1 -- 172.21.15.89:0/2142817204 wait complete. 2024-09-10T08:36:03.869 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:03 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2142817204' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:36:03.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:03 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2142817204' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:36:04.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:03 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2142817204' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:36:04.360 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1725957360,"num_in_osds":12,"osd_in_since":1725957337,"num_remapped_pgs":0} 2024-09-10T08:36:04.361 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:36:04.724 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:36:04.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:04 smithi089 ceph-mon[30757]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:04.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:04 smithi149 ceph-mon[36167]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:04 smithi111 ceph-mon[35220]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:05.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.279+0000 7ffb42600640 1 -- 172.21.15.89:0/3944590362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c1079d0 msgr2=0x7ffb3c107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:05.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.279+0000 7ffb42600640 1 --2- 172.21.15.89:0/3944590362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c1079d0 0x7ffb3c107db0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009930 tx=0x7ffb3002a420 comp rx=0 tx=0).stop 2024-09-10T08:36:05.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.279+0000 7ffb42600640 1 -- 172.21.15.89:0/3944590362 shutdown_connections 2024-09-10T08:36:05.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.279+0000 7ffb42600640 1 --2- 172.21.15.89:0/3944590362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb3c102370 0x7ffb3c10e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:05.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.279+0000 7ffb42600640 1 --2- 172.21.15.89:0/3944590362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb3c1019d0 0x7ffb3c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:05.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.279+0000 7ffb42600640 1 --2- 172.21.15.89:0/3944590362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c1079d0 0x7ffb3c107db0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:05.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.279+0000 7ffb42600640 1 -- 172.21.15.89:0/3944590362 >> 172.21.15.89:0/3944590362 conn(0x7ffb3c0fd710 msgr2=0x7ffb3c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:05.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.279+0000 7ffb42600640 1 -- 172.21.15.89:0/3944590362 shutdown_connections 2024-09-10T08:36:05.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.279+0000 7ffb42600640 1 -- 172.21.15.89:0/3944590362 wait complete. 2024-09-10T08:36:05.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.280+0000 7ffb42600640 1 Processor -- start 2024-09-10T08:36:05.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.280+0000 7ffb42600640 1 -- start start 2024-09-10T08:36:05.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.280+0000 7ffb42600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb3c1019d0 0x7ffb3c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:05.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.280+0000 7ffb42600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c102370 0x7ffb3c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:05.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.281+0000 7ffb42600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb3c1079d0 0x7ffb3c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:05.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.281+0000 7ffb42600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3c111e20 con 0x7ffb3c102370 2024-09-10T08:36:05.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.281+0000 7ffb42600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3c111ca0 con 0x7ffb3c1019d0 2024-09-10T08:36:05.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.281+0000 7ffb42600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3c111fa0 con 0x7ffb3c1079d0 2024-09-10T08:36:05.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.281+0000 7ffb3be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c102370 0x7ffb3c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:05.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.281+0000 7ffb3be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c102370 0x7ffb3c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39854/0 (socket says 172.21.15.89:39854) 2024-09-10T08:36:05.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.281+0000 7ffb3be00640 1 -- 172.21.15.89:0/2972141989 learned_addr learned my addr 172.21.15.89:0/2972141989 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:36:05.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.281+0000 7ffb41200640 1 --2- 172.21.15.89:0/2972141989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb3c1019d0 0x7ffb3c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:05.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.281+0000 7ffb41c00640 1 --2- 172.21.15.89:0/2972141989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb3c1079d0 0x7ffb3c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:05.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.281+0000 7ffb3be00640 1 -- 172.21.15.89:0/2972141989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb3c1079d0 msgr2=0x7ffb3c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:05.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.281+0000 7ffb3be00640 1 --2- 172.21.15.89:0/2972141989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb3c1079d0 0x7ffb3c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:05.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.281+0000 7ffb3be00640 1 -- 172.21.15.89:0/2972141989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb3c1019d0 msgr2=0x7ffb3c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:05.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.281+0000 7ffb3be00640 1 --2- 172.21.15.89:0/2972141989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb3c1019d0 0x7ffb3c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:05.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.281+0000 7ffb3be00640 1 -- 172.21.15.89:0/2972141989 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb3c1a67c0 con 0x7ffb3c102370 2024-09-10T08:36:05.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.281+0000 7ffb3be00640 1 --2- 172.21.15.89:0/2972141989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c102370 0x7ffb3c074a30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffb2800e9f0 tx=0x7ffb2800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:05.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.282+0000 7ffb39600640 1 -- 172.21.15.89:0/2972141989 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2800cd80 con 0x7ffb3c102370 2024-09-10T08:36:05.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.282+0000 7ffb39600640 1 -- 172.21.15.89:0/2972141989 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb28005500 con 0x7ffb3c102370 2024-09-10T08:36:05.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.282+0000 7ffb42600640 1 -- 172.21.15.89:0/2972141989 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb3c1a6ab0 con 0x7ffb3c102370 2024-09-10T08:36:05.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.282+0000 7ffb39600640 1 -- 172.21.15.89:0/2972141989 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb28010690 con 0x7ffb3c102370 2024-09-10T08:36:05.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.282+0000 7ffb42600640 1 -- 172.21.15.89:0/2972141989 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb3c1a6ff0 con 0x7ffb3c102370 2024-09-10T08:36:05.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.283+0000 7ffb42600640 1 -- 172.21.15.89:0/2972141989 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb3c101e30 con 0x7ffb3c102370 2024-09-10T08:36:05.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.286+0000 7ffb39600640 1 -- 172.21.15.89:0/2972141989 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ffb280046c0 con 0x7ffb3c102370 2024-09-10T08:36:05.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.288+0000 7ffb39600640 1 --2- 172.21.15.89:0/2972141989 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffb240a2c00 0x7ffb240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:05.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.288+0000 7ffb39600640 1 -- 172.21.15.89:0/2972141989 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7ffb280cf250 con 0x7ffb3c102370 2024-09-10T08:36:05.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.289+0000 7ffb41200640 1 --2- 172.21.15.89:0/2972141989 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffb240a2c00 0x7ffb240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:05.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.289+0000 7ffb41200640 1 --2- 172.21.15.89:0/2972141989 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffb240a2c00 0x7ffb240a50c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb30006010 tx=0x7ffb30000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:05.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.290+0000 7ffb39600640 1 -- 172.21.15.89:0/2972141989 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb280541b0 con 0x7ffb3c102370 2024-09-10T08:36:05.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.466+0000 7ffb42600640 1 -- 172.21.15.89:0/2972141989 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb3c109f40 con 0x7ffb3c102370 2024-09-10T08:36:05.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.467+0000 7ffb39600640 1 -- 172.21.15.89:0/2972141989 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+18492 (secure 0 0 0) 0x7ffb2809c5b0 con 0x7ffb3c102370 2024-09-10T08:36:05.467 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:36:05.467 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":80,"fsid":"91f747bc-6f4e-11ef-bcea-c7b262605968","created":"2024-09-10T08:28:49.697035+0000","modified":"2024-09-10T08:36:01.543805+0000","last_up_change":"2024-09-10T08:36:00.535433+0000","last_in_change":"2024-09-10T08:35:37.131131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:32:02.060714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3b55192-bd95-4497-89ea-dc017a25c0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6803","nonce":1760523284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6805","nonce":1760523284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6809","nonce":1760523284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6807","nonce":1760523284}]},"public_addr":"172.21.15.89:6803/1760523284","cluster_addr":"172.21.15.89:6805/1760523284","heartbeat_back_addr":"172.21.15.89:6809/1760523284","heartbeat_front_addr":"172.21.15.89:6807/1760523284","state":["exists","up"]},{"osd":1,"uuid":"60aae117-cb52-4034-a700-11693171cd52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6801","nonce":2369291742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6803","nonce":2369291742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6807","nonce":2369291742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6805","nonce":2369291742}]},"public_addr":"172.21.15.111:6801/2369291742","cluster_addr":"172.21.15.111:6803/2369291742","heartbeat_back_addr":"172.21.15.111:6807/2369291742","heartbeat_front_addr":"172.21.15.111:6805/2369291742","state":["exists","up"]},{"osd":2,"uuid":"a305a019-0870-437c-a298-4f923ec54e77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6801","nonce":1092261395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6803","nonce":1092261395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6807","nonce":1092261395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6805","nonce":1092261395}]},"public_addr":"172.21.15.149:6801/1092261395","cluster_addr":"172.21.15.149:6803/1092261395","heartbeat_back_addr":"172.21.15.149:6807/1092261395","heartbeat_front_addr":"172.21.15.149:6805/1092261395","state":["exists","up"]},{"osd":3,"uuid":"d02f54c1-f883-40f6-9341-8d9fc4b712ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6811","nonce":1405773950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6813","nonce":1405773950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6817","nonce":1405773950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6815","nonce":1405773950}]},"public_addr":"172.21.15.89:6811/1405773950","cluster_addr":"172.21.15.89:6813/1405773950","heartbeat_back_addr":"172.21.15.89:6817/1405773950","heartbeat_front_addr":"172.21.15.89:6815/1405773950","state":["exists","up"]},{"osd":4,"uuid":"23cff46b-88dd-4344-be06-b35db6014f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6809","nonce":316717323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6811","nonce":316717323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6815","nonce":316717323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6813","nonce":316717323}]},"public_addr":"172.21.15.111:6809/316717323","cluster_addr":"172.21.15.111:6811/316717323","heartbeat_back_addr":"172.21.15.111:6815/316717323","heartbeat_front_addr":"172.21.15.111:6813/316717323","state":["exists","up"]},{"osd":5,"uuid":"8e516e8d-e7c6-4ebc-80a5-29932161df4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6809","nonce":1256148461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6811","nonce":1256148461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6815","nonce":1256148461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6813","nonce":1256148461}]},"public_addr":"172.21.15.149:6809/1256148461","cluster_addr":"172.21.15.149:6811/1256148461","heartbeat_back_addr":"172.21.15.149:6815/1256148461","heartbeat_front_addr":"172.21.15.149:6813/1256148461","state":["exists","up"]},{"osd":6,"uuid":"48f92472-f657-4c5a-adeb-20d4fea1ce8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6819","nonce":4190488024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6821","nonce":4190488024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6825","nonce":4190488024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6823","nonce":4190488024}]},"public_addr":"172.21.15.89:6819/4190488024","cluster_addr":"172.21.15.89:6821/4190488024","heartbeat_back_addr":"172.21.15.89:6825/4190488024","heartbeat_front_addr":"172.21.15.89:6823/4190488024","state":["exists","up"]},{"osd":7,"uuid":"14a2a6f0-c4cd-406b-b637-7527fd32babe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6817","nonce":652666376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6819","nonce":652666376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6823","nonce":652666376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6821","nonce":652666376}]},"public_addr":"172.21.15.111:6817/652666376","cluster_addr":"172.21.15.111:6819/652666376","heartbeat_back_addr":"172.21.15.111:6823/652666376","heartbeat_front_addr":"172.21.15.111:6821/652666376","state":["exists","up"]},{"osd":8,"uuid":"a54502e6-95f7-431f-8db1-54a6c7983b5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6817","nonce":2955172869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6819","nonce":2955172869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6823","nonce":2955172869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6821","nonce":2955172869}]},"public_addr":"172.21.15.149:6817/2955172869","cluster_addr":"172.21.15.149:6819/2955172869","heartbeat_back_addr":"172.21.15.149:6823/2955172869","heartbeat_front_addr":"172.21.15.149:6821/2955172869","state":["exists","up"]},{"osd":9,"uuid":"5fe17344-5b1c-4601-a633-639cc1691478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6827","nonce":2427467040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6829","nonce":2427467040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6833","nonce":2427467040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6831","nonce":2427467040}]},"public_addr":"172.21.15.89:6827/2427467040","cluster_addr":"172.21.15.89:6829/2427467040","heartbeat_back_addr":"172.21.15.89:6833/2427467040","heartbeat_front_addr":"172.21.15.89:6831/2427467040","state":["exists","up"]},{"osd":10,"uuid":"d8f29890-3064-4b67-b649-467a0ac86db7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6825","nonce":2328678744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6827","nonce":2328678744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6831","nonce":2328678744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6829","nonce":2328678744}]},"public_addr":"172.21.15.111:6825/2328678744","cluster_addr":"172.21.15.111:6827/2328678744","heartbeat_back_addr":"172.21.15.111:6831/2328678744","heartbeat_front_addr":"172.21.15.111:6829/2328678744","state":["exists","up"]},{"osd":11,"uuid":"92182c73-f139-47ca-a0ef-5046ae9b2310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6825","nonce":3130508827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6827","nonce":3130508827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6831","nonce":3130508827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6829","nonce":3130508827}]},"public_addr":"172.21.15.149:6825/3130508827","cluster_addr":"172.21.15.149:6827/3130508827","heartbeat_back_addr":"172.21.15.149:6831/3130508827","heartbeat_front_addr":"172.21.15.149:6829/3130508827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:09.031810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:33.695124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:58.065510+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:27.592838+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:53.275289+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:18.305677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:47.781535+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:12.889512+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:37.744193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:07.718408+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:33.391027+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:59.013250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/850669510":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/3541027428":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6801/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6800/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/2396468787":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/1609720980":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/995019144":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6801/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6800/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/2179977257":"2024-09-11T08:29:14.554869+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:36:05.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.470+0000 7ffb42600640 1 -- 172.21.15.89:0/2972141989 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffb240a2c00 msgr2=0x7ffb240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:05.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.470+0000 7ffb42600640 1 --2- 172.21.15.89:0/2972141989 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffb240a2c00 0x7ffb240a50c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb30006010 tx=0x7ffb30000bc0 comp rx=0 tx=0).stop 2024-09-10T08:36:05.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.470+0000 7ffb42600640 1 -- 172.21.15.89:0/2972141989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c102370 msgr2=0x7ffb3c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:05.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.470+0000 7ffb42600640 1 --2- 172.21.15.89:0/2972141989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c102370 0x7ffb3c074a30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffb2800e9f0 tx=0x7ffb2800edf0 comp rx=0 tx=0).stop 2024-09-10T08:36:05.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.470+0000 7ffb42600640 1 -- 172.21.15.89:0/2972141989 shutdown_connections 2024-09-10T08:36:05.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.470+0000 7ffb42600640 1 --2- 172.21.15.89:0/2972141989 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffb240a2c00 0x7ffb240a50c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:05.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.470+0000 7ffb42600640 1 --2- 172.21.15.89:0/2972141989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb3c1079d0 0x7ffb3c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:05.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.470+0000 7ffb42600640 1 --2- 172.21.15.89:0/2972141989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb3c102370 0x7ffb3c074a30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:05.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.470+0000 7ffb42600640 1 --2- 172.21.15.89:0/2972141989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb3c1019d0 0x7ffb3c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:05.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.470+0000 7ffb42600640 1 -- 172.21.15.89:0/2972141989 >> 172.21.15.89:0/2972141989 conn(0x7ffb3c0fd710 msgr2=0x7ffb3c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:05.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.471+0000 7ffb42600640 1 -- 172.21.15.89:0/2972141989 shutdown_connections 2024-09-10T08:36:05.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:05.471+0000 7ffb42600640 1 -- 172.21.15.89:0/2972141989 wait complete. 2024-09-10T08:36:05.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:05 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2972141989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:36:05.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:05 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2972141989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:36:06.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:05 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2972141989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:36:06.339 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T08:32:02.060714+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T08:36:06.339 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T08:36:06.699 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:36:06.837 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:06 smithi089 ceph-mon[30757]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:06.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:06 smithi149 ceph-mon[36167]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:07.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:06 smithi111 ceph-mon[35220]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:07.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.311+0000 7fc2f6200640 1 -- 172.21.15.89:0/617821363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2f0102d30 msgr2=0x7fc2f0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:07.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.311+0000 7fc2f6200640 1 --2- 172.21.15.89:0/617821363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2f0102d30 0x7fc2f0103110 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009ca0 tx=0x7fc2e002a950 comp rx=0 tx=0).stop 2024-09-10T08:36:07.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.311+0000 7fc2f6200640 1 -- 172.21.15.89:0/617821363 shutdown_connections 2024-09-10T08:36:07.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.311+0000 7fc2f6200640 1 --2- 172.21.15.89:0/617821363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2f010c160 0x7fc2f010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:07.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.311+0000 7fc2f6200640 1 --2- 172.21.15.89:0/617821363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2f0103650 0x7fc2f010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:07.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.311+0000 7fc2f6200640 1 --2- 172.21.15.89:0/617821363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2f0102d30 0x7fc2f0103110 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:07.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.311+0000 7fc2f6200640 1 -- 172.21.15.89:0/617821363 >> 172.21.15.89:0/617821363 conn(0x7fc2f00fc7d0 msgr2=0x7fc2f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:07.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.311+0000 7fc2f6200640 1 -- 172.21.15.89:0/617821363 shutdown_connections 2024-09-10T08:36:07.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.311+0000 7fc2f6200640 1 -- 172.21.15.89:0/617821363 wait complete. 2024-09-10T08:36:07.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.312+0000 7fc2f6200640 1 Processor -- start 2024-09-10T08:36:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.312+0000 7fc2f6200640 1 -- start start 2024-09-10T08:36:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.312+0000 7fc2f6200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2f0102d30 0x7fc2f019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.312+0000 7fc2f6200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2f0103650 0x7fc2f019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.312+0000 7fc2f6200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2f010c160 0x7fc2f019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.312+0000 7fc2f6200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0111f60 con 0x7fc2f0102d30 2024-09-10T08:36:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.312+0000 7fc2f6200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0111de0 con 0x7fc2f010c160 2024-09-10T08:36:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.312+0000 7fc2f6200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f01120e0 con 0x7fc2f0103650 2024-09-10T08:36:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.313+0000 7fc2ef400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2f0102d30 0x7fc2f019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.313+0000 7fc2eea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2f0103650 0x7fc2f019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.313+0000 7fc2ef400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2f0102d30 0x7fc2f019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39880/0 (socket says 172.21.15.89:39880) 2024-09-10T08:36:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.313+0000 7fc2ef400640 1 -- 172.21.15.89:0/2836803689 learned_addr learned my addr 172.21.15.89:0/2836803689 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:36:07.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.313+0000 7fc2efe00640 1 --2- 172.21.15.89:0/2836803689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2f010c160 0x7fc2f019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:07.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.313+0000 7fc2ef400640 1 -- 172.21.15.89:0/2836803689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2f0103650 msgr2=0x7fc2f019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:07.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.313+0000 7fc2ef400640 1 --2- 172.21.15.89:0/2836803689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2f0103650 0x7fc2f019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:07.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.313+0000 7fc2ef400640 1 -- 172.21.15.89:0/2836803689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2f010c160 msgr2=0x7fc2f019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:07.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.313+0000 7fc2ef400640 1 --2- 172.21.15.89:0/2836803689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2f010c160 0x7fc2f019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:07.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.313+0000 7fc2ef400640 1 -- 172.21.15.89:0/2836803689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f019f5e0 con 0x7fc2f0102d30 2024-09-10T08:36:07.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.313+0000 7fc2efe00640 1 --2- 172.21.15.89:0/2836803689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2f010c160 0x7fc2f019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:36:07.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.313+0000 7fc2eea00640 1 --2- 172.21.15.89:0/2836803689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2f0103650 0x7fc2f019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:36:07.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.313+0000 7fc2ef400640 1 --2- 172.21.15.89:0/2836803689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2f0102d30 0x7fc2f019a630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0005cc0 tx=0x7fc2e002f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:07.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.314+0000 7fc2e7e00640 1 -- 172.21.15.89:0/2836803689 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0034040 con 0x7fc2f0102d30 2024-09-10T08:36:07.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.314+0000 7fc2f6200640 1 -- 172.21.15.89:0/2836803689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f019f870 con 0x7fc2f0102d30 2024-09-10T08:36:07.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.314+0000 7fc2e7e00640 1 -- 172.21.15.89:0/2836803689 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2e0030920 con 0x7fc2f0102d30 2024-09-10T08:36:07.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.314+0000 7fc2f6200640 1 -- 172.21.15.89:0/2836803689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f01a71b0 con 0x7fc2f0102d30 2024-09-10T08:36:07.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.314+0000 7fc2e7e00640 1 -- 172.21.15.89:0/2836803689 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0004720 con 0x7fc2f0102d30 2024-09-10T08:36:07.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.315+0000 7fc2f6200640 1 -- 172.21.15.89:0/2836803689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f0103110 con 0x7fc2f0102d30 2024-09-10T08:36:07.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.317+0000 7fc2e7e00640 1 -- 172.21.15.89:0/2836803689 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc2e0037020 con 0x7fc2f0102d30 2024-09-10T08:36:07.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.318+0000 7fc2e7e00640 1 --2- 172.21.15.89:0/2836803689 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc2d40a2b80 0x7fc2d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:07.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.318+0000 7fc2e7e00640 1 -- 172.21.15.89:0/2836803689 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7fc2e00efb10 con 0x7fc2f0102d30 2024-09-10T08:36:07.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.318+0000 7fc2eea00640 1 --2- 172.21.15.89:0/2836803689 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc2d40a2b80 0x7fc2d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:07.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.319+0000 7fc2eea00640 1 --2- 172.21.15.89:0/2836803689 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc2d40a2b80 0x7fc2d40a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc2f010e020 tx=0x7fc2dc00b360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:07.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.322+0000 7fc2e7e00640 1 -- 172.21.15.89:0/2836803689 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc2e00bce70 con 0x7fc2f0102d30 2024-09-10T08:36:07.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.497+0000 7fc2f6200640 1 -- 172.21.15.89:0/2836803689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc2f01a7380 con 0x7fc2f0102d30 2024-09-10T08:36:07.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.497+0000 7fc2e7e00640 1 -- 172.21.15.89:0/2836803689 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7fc2e00bc770 con 0x7fc2f0102d30 2024-09-10T08:36:07.497 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2024-09-10T08:36:07.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.499+0000 7fc2f6200640 1 -- 172.21.15.89:0/2836803689 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc2d40a2b80 msgr2=0x7fc2d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:07.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.499+0000 7fc2f6200640 1 --2- 172.21.15.89:0/2836803689 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc2d40a2b80 0x7fc2d40a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc2f010e020 tx=0x7fc2dc00b360 comp rx=0 tx=0).stop 2024-09-10T08:36:07.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.500+0000 7fc2f6200640 1 -- 172.21.15.89:0/2836803689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2f0102d30 msgr2=0x7fc2f019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:07.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.500+0000 7fc2f6200640 1 --2- 172.21.15.89:0/2836803689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2f0102d30 0x7fc2f019a630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0005cc0 tx=0x7fc2e002f630 comp rx=0 tx=0).stop 2024-09-10T08:36:07.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.500+0000 7fc2f6200640 1 -- 172.21.15.89:0/2836803689 shutdown_connections 2024-09-10T08:36:07.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.500+0000 7fc2f6200640 1 --2- 172.21.15.89:0/2836803689 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc2d40a2b80 0x7fc2d40a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:07.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.500+0000 7fc2f6200640 1 --2- 172.21.15.89:0/2836803689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2f010c160 0x7fc2f019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:07.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.500+0000 7fc2f6200640 1 --2- 172.21.15.89:0/2836803689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2f0103650 0x7fc2f019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:07.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.500+0000 7fc2f6200640 1 --2- 172.21.15.89:0/2836803689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc2f0102d30 0x7fc2f019a630 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:07.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.500+0000 7fc2f6200640 1 -- 172.21.15.89:0/2836803689 >> 172.21.15.89:0/2836803689 conn(0x7fc2f00fc7d0 msgr2=0x7fc2f010d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:07.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.500+0000 7fc2f6200640 1 -- 172.21.15.89:0/2836803689 shutdown_connections 2024-09-10T08:36:07.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:07.500+0000 7fc2f6200640 1 -- 172.21.15.89:0/2836803689 wait complete. 2024-09-10T08:36:07.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:07 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2836803689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:36:07.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:07 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2836803689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:36:08.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:07 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2836803689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:36:08.372 INFO:tasks.cephadm:Adding mds.a on smithi089 2024-09-10T08:36:08.372 INFO:tasks.cephadm:Adding mds.d on smithi089 2024-09-10T08:36:08.372 INFO:tasks.cephadm:Adding mds.g on smithi089 2024-09-10T08:36:08.372 INFO:tasks.cephadm:Adding mds.j on smithi089 2024-09-10T08:36:08.372 INFO:tasks.cephadm:Adding mds.b on smithi111 2024-09-10T08:36:08.372 INFO:tasks.cephadm:Adding mds.e on smithi111 2024-09-10T08:36:08.372 INFO:tasks.cephadm:Adding mds.h on smithi111 2024-09-10T08:36:08.372 INFO:tasks.cephadm:Adding mds.k on smithi111 2024-09-10T08:36:08.372 INFO:tasks.cephadm:Adding mds.c on smithi149 2024-09-10T08:36:08.373 INFO:tasks.cephadm:Adding mds.f on smithi149 2024-09-10T08:36:08.373 INFO:tasks.cephadm:Adding mds.i on smithi149 2024-09-10T08:36:08.373 INFO:tasks.cephadm:Adding mds.l on smithi149 2024-09-10T08:36:08.373 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l' 2024-09-10T08:36:08.633 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:36:08.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:08 smithi149 ceph-mon[36167]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:08.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:08 smithi089 ceph-mon[30757]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:09.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:08 smithi111 ceph-mon[35220]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:09.156 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.119+0000 7f01e7c3d640 1 -- 172.21.15.149:0/4275749296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01e0100ad0 msgr2=0x7f01e0100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:09.156 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.119+0000 7f01e7c3d640 1 --2- 172.21.15.149:0/4275749296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01e0100ad0 0x7f01e0100ed0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009900 tx=0x7f01dc02a410 comp rx=0 tx=0).stop 2024-09-10T08:36:09.156 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.119+0000 7f01e7c3d640 1 -- 172.21.15.149:0/4275749296 shutdown_connections 2024-09-10T08:36:09.157 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.119+0000 7f01e7c3d640 1 --2- 172.21.15.149:0/4275749296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01e0102690 0x7f01e0108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:09.157 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.119+0000 7f01e7c3d640 1 --2- 172.21.15.149:0/4275749296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01e0101cd0 0x7f01e0102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:09.157 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.119+0000 7f01e7c3d640 1 --2- 172.21.15.149:0/4275749296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01e0100ad0 0x7f01e0100ed0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:09.157 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.119+0000 7f01e7c3d640 1 -- 172.21.15.149:0/4275749296 >> 172.21.15.149:0/4275749296 conn(0x7f01e00fc280 msgr2=0x7f01e00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:09.157 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.119+0000 7f01e7c3d640 1 -- 172.21.15.149:0/4275749296 shutdown_connections 2024-09-10T08:36:09.157 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.119+0000 7f01e7c3d640 1 -- 172.21.15.149:0/4275749296 wait complete. 2024-09-10T08:36:09.157 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.120+0000 7f01e7c3d640 1 Processor -- start 2024-09-10T08:36:09.157 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.120+0000 7f01e7c3d640 1 -- start start 2024-09-10T08:36:09.157 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.120+0000 7f01e7c3d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01e0100ad0 0x7f01e0197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:09.157 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.120+0000 7f01e7c3d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01e0101cd0 0x7f01e01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:09.157 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.120+0000 7f01e7c3d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01e0102690 0x7f01e019f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:09.157 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.120+0000 7f01e7c3d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f01e010c7d0 con 0x7f01e0100ad0 2024-09-10T08:36:09.158 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.120+0000 7f01e7c3d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f01e010c650 con 0x7f01e0101cd0 2024-09-10T08:36:09.158 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.120+0000 7f01e7c3d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f01e010c950 con 0x7f01e0102690 2024-09-10T08:36:09.158 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.120+0000 7f01e61da640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01e0102690 0x7f01e019f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:09.158 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.120+0000 7f01e61da640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01e0102690 0x7f01e019f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:43694/0 (socket says 172.21.15.149:43694) 2024-09-10T08:36:09.158 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.120+0000 7f01e61da640 1 -- 172.21.15.149:0/825544087 learned_addr learned my addr 172.21.15.149:0/825544087 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-09-10T08:36:09.158 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.120+0000 7f01e51d8640 1 --2- 172.21.15.149:0/825544087 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01e0101cd0 0x7f01e01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:09.158 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.120+0000 7f01e61da640 1 -- 172.21.15.149:0/825544087 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01e0101cd0 msgr2=0x7f01e01984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:09.158 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.120+0000 7f01e61da640 1 --2- 172.21.15.149:0/825544087 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01e0101cd0 0x7f01e01984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:09.158 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.120+0000 7f01e61da640 1 -- 172.21.15.149:0/825544087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01e0100ad0 msgr2=0x7f01e0197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:09.158 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.120+0000 7f01e61da640 1 --2- 172.21.15.149:0/825544087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01e0100ad0 0x7f01e0197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:09.158 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.120+0000 7f01e61da640 1 -- 172.21.15.149:0/825544087 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e019fbc0 con 0x7f01e0102690 2024-09-10T08:36:09.158 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.121+0000 7f01e61da640 1 --2- 172.21.15.149:0/825544087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01e0102690 0x7f01e019f550 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f01d800b970 tx=0x7f01d800be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:09.158 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.121+0000 7f01d6ffd640 1 -- 172.21.15.149:0/825544087 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8019070 con 0x7f01e0102690 2024-09-10T08:36:09.159 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.121+0000 7f01d6ffd640 1 -- 172.21.15.149:0/825544087 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01d800cb40 con 0x7f01e0102690 2024-09-10T08:36:09.159 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.121+0000 7f01e7c3d640 1 -- 172.21.15.149:0/825544087 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e019feb0 con 0x7f01e0102690 2024-09-10T08:36:09.159 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.121+0000 7f01d6ffd640 1 -- 172.21.15.149:0/825544087 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8007650 con 0x7f01e0102690 2024-09-10T08:36:09.159 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.121+0000 7f01e7c3d640 1 -- 172.21.15.149:0/825544087 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e01a0390 con 0x7f01e0102690 2024-09-10T08:36:09.159 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.122+0000 7f01e7c3d640 1 -- 172.21.15.149:0/825544087 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e01a0620 con 0x7f01e0102690 2024-09-10T08:36:09.159 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.123+0000 7f01d6ffd640 1 -- 172.21.15.149:0/825544087 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f01d801c070 con 0x7f01e0102690 2024-09-10T08:36:09.159 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.124+0000 7f01d6ffd640 1 --2- 172.21.15.149:0/825544087 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f01cc0a2bb0 0x7f01cc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:09.159 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.124+0000 7f01d6ffd640 1 -- 172.21.15.149:0/825544087 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f01d80ce1e0 con 0x7f01e0102690 2024-09-10T08:36:09.159 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.124+0000 7f01e59d9640 1 --2- 172.21.15.149:0/825544087 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f01cc0a2bb0 0x7f01cc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:09.159 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.125+0000 7f01e59d9640 1 --2- 172.21.15.149:0/825544087 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f01cc0a2bb0 0x7f01cc0a5070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0096c0 tx=0x7f01dc031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:09.159 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.128+0000 7f01d6ffd640 1 -- 172.21.15.149:0/825544087 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f01d809b5e0 con 0x7f01e0102690 2024-09-10T08:36:09.314 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.314+0000 7f01e7c3d640 1 -- 172.21.15.149:0/825544087 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}) -- 0x7f01e006a770 con 0x7f01cc0a2bb0 2024-09-10T08:36:09.322 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.323+0000 7f01d6ffd640 1 -- 172.21.15.149:0/825544087 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f01e006a770 con 0x7f01cc0a2bb0 2024-09-10T08:36:09.323 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mds.all update... 2024-09-10T08:36:09.325 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.325+0000 7f01e7c3d640 1 -- 172.21.15.149:0/825544087 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f01cc0a2bb0 msgr2=0x7f01cc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:09.325 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.325+0000 7f01e7c3d640 1 --2- 172.21.15.149:0/825544087 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f01cc0a2bb0 0x7f01cc0a5070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0096c0 tx=0x7f01dc031400 comp rx=0 tx=0).stop 2024-09-10T08:36:09.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.325+0000 7f01e7c3d640 1 -- 172.21.15.149:0/825544087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01e0102690 msgr2=0x7f01e019f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:09.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.325+0000 7f01e7c3d640 1 --2- 172.21.15.149:0/825544087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01e0102690 0x7f01e019f550 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f01d800b970 tx=0x7f01d800be30 comp rx=0 tx=0).stop 2024-09-10T08:36:09.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.325+0000 7f01e7c3d640 1 -- 172.21.15.149:0/825544087 shutdown_connections 2024-09-10T08:36:09.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.325+0000 7f01e7c3d640 1 --2- 172.21.15.149:0/825544087 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f01cc0a2bb0 0x7f01cc0a5070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:09.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.325+0000 7f01e7c3d640 1 --2- 172.21.15.149:0/825544087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01e0102690 0x7f01e019f550 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:09.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.325+0000 7f01e7c3d640 1 --2- 172.21.15.149:0/825544087 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01e0101cd0 0x7f01e01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:09.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.325+0000 7f01e7c3d640 1 --2- 172.21.15.149:0/825544087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01e0100ad0 0x7f01e0197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:09.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.325+0000 7f01e7c3d640 1 -- 172.21.15.149:0/825544087 >> 172.21.15.149:0/825544087 conn(0x7f01e00fc280 msgr2=0x7f01e006a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:09.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.325+0000 7f01e7c3d640 1 -- 172.21.15.149:0/825544087 shutdown_connections 2024-09-10T08:36:09.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-10T08:36:09.325+0000 7f01e7c3d640 1 -- 172.21.15.149:0/825544087 wait complete. 2024-09-10T08:36:10.017 DEBUG:teuthology.orchestra.run.smithi089:mds.a> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.a.service 2024-09-10T08:36:10.022 DEBUG:teuthology.orchestra.run.smithi089:mds.d> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.d.service 2024-09-10T08:36:10.068 DEBUG:teuthology.orchestra.run.smithi089:mds.g> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.g.service 2024-09-10T08:36:10.114 DEBUG:teuthology.orchestra.run.smithi089:mds.j> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.j.service 2024-09-10T08:36:10.159 DEBUG:teuthology.orchestra.run.smithi111:mds.b> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.b.service 2024-09-10T08:36:10.162 DEBUG:teuthology.orchestra.run.smithi111:mds.e> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.e.service 2024-09-10T08:36:10.204 DEBUG:teuthology.orchestra.run.smithi111:mds.h> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.h.service 2024-09-10T08:36:10.247 DEBUG:teuthology.orchestra.run.smithi111:mds.k> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.k.service 2024-09-10T08:36:10.290 DEBUG:teuthology.orchestra.run.smithi149:mds.c> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.c.service 2024-09-10T08:36:10.293 DEBUG:teuthology.orchestra.run.smithi149:mds.f> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.f.service 2024-09-10T08:36:10.336 DEBUG:teuthology.orchestra.run.smithi149:mds.i> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.i.service 2024-09-10T08:36:10.379 DEBUG:teuthology.orchestra.run.smithi149:mds.l> sudo journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.l.service 2024-09-10T08:36:10.422 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-10T08:36:10.422 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:36:10.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:10 smithi111 ceph-mon[35220]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:36:10.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:10 smithi111 ceph-mon[35220]: Saving service mds.all spec with placement smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2024-09-10T08:36:10.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:10.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:36:10.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:10 smithi111 ceph-mon[35220]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:10.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:10.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:36:10.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:10.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:10.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:10.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:10 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:10.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:10 smithi111 ceph-mon[35220]: Deploying daemon mds.l on smithi149 2024-09-10T08:36:10.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:10 smithi149 ceph-mon[36167]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:36:10.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:10 smithi149 ceph-mon[36167]: Saving service mds.all spec with placement smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2024-09-10T08:36:10.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:10.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:36:10.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:10 smithi149 ceph-mon[36167]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:10.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:10.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:36:10.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:10.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:10.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:10.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:10 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:10.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:10 smithi149 ceph-mon[36167]: Deploying daemon mds.l on smithi149 2024-09-10T08:36:10.780 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:36:10.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:10 smithi089 ceph-mon[30757]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:36:10.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:10 smithi089 ceph-mon[30757]: Saving service mds.all spec with placement smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2024-09-10T08:36:10.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:10.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:36:10.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:10 smithi089 ceph-mon[30757]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:10.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:10.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:36:10.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:10.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:10.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:10.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:10 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:10.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:10 smithi089 ceph-mon[30757]: Deploying daemon mds.l on smithi149 2024-09-10T08:36:11.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.330+0000 7fb3a2600640 1 -- 172.21.15.89:0/739609702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb39c1047e0 msgr2=0x7fb39c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:11.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.330+0000 7fb3a2600640 1 --2- 172.21.15.89:0/739609702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb39c1047e0 0x7fb39c108830 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb388009900 tx=0x7fb38802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:36:11.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.330+0000 7fb3a2600640 1 -- 172.21.15.89:0/739609702 shutdown_connections 2024-09-10T08:36:11.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.330+0000 7fb3a2600640 1 --2- 172.21.15.89:0/739609702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb39c108f60 0x7fb39c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:11.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.331+0000 7fb3a2600640 1 --2- 172.21.15.89:0/739609702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb39c1047e0 0x7fb39c108830 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:11.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.331+0000 7fb3a2600640 1 --2- 172.21.15.89:0/739609702 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb39c103e30 0x7fb39c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:11.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.331+0000 7fb3a2600640 1 -- 172.21.15.89:0/739609702 >> 172.21.15.89:0/739609702 conn(0x7fb39c0ff8d0 msgr2=0x7fb39c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:11.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.331+0000 7fb3a2600640 1 -- 172.21.15.89:0/739609702 shutdown_connections 2024-09-10T08:36:11.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.331+0000 7fb3a2600640 1 -- 172.21.15.89:0/739609702 wait complete. 2024-09-10T08:36:11.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.332+0000 7fb3a2600640 1 Processor -- start 2024-09-10T08:36:11.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.332+0000 7fb3a2600640 1 -- start start 2024-09-10T08:36:11.332 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.332+0000 7fb3a2600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb39c103e30 0x7fb39c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:11.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.332+0000 7fb3a2600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb39c1047e0 0x7fb39c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:11.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.332+0000 7fb3a2600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb39c108f60 0x7fb39c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:11.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.332+0000 7fb3a2600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c114010 con 0x7fb39c1047e0 2024-09-10T08:36:11.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.332+0000 7fb3a2600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c113e90 con 0x7fb39c103e30 2024-09-10T08:36:11.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.332+0000 7fb3a2600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c114190 con 0x7fb39c108f60 2024-09-10T08:36:11.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.332+0000 7fb3a1c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb39c108f60 0x7fb39c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:11.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.332+0000 7fb39be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb39c1047e0 0x7fb39c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:11.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.332+0000 7fb3a1200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb39c103e30 0x7fb39c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:11.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.332+0000 7fb3a1200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb39c103e30 0x7fb39c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:54962/0 (socket says 172.21.15.89:54962) 2024-09-10T08:36:11.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.332+0000 7fb3a1c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb39c108f60 0x7fb39c19ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:52040/0 (socket says 172.21.15.89:52040) 2024-09-10T08:36:11.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.332+0000 7fb3a1200640 1 -- 172.21.15.89:0/2220372559 learned_addr learned my addr 172.21.15.89:0/2220372559 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:36:11.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.333+0000 7fb3a1200640 1 -- 172.21.15.89:0/2220372559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb39c108f60 msgr2=0x7fb39c19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:11.333 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.333+0000 7fb3a1200640 1 --2- 172.21.15.89:0/2220372559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb39c108f60 0x7fb39c19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:11.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.333+0000 7fb3a1200640 1 -- 172.21.15.89:0/2220372559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb39c1047e0 msgr2=0x7fb39c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:11.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.333+0000 7fb3a1200640 1 --2- 172.21.15.89:0/2220372559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb39c1047e0 0x7fb39c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:11.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.333+0000 7fb3a1200640 1 -- 172.21.15.89:0/2220372559 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39c19f420 con 0x7fb39c103e30 2024-09-10T08:36:11.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.333+0000 7fb3a1200640 1 --2- 172.21.15.89:0/2220372559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb39c103e30 0x7fb39c19a470 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb39000e9f0 tx=0x7fb39000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:11.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.333+0000 7fb399600640 1 -- 172.21.15.89:0/2220372559 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39000cd80 con 0x7fb39c103e30 2024-09-10T08:36:11.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.333+0000 7fb3a2600640 1 -- 172.21.15.89:0/2220372559 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb39c19f710 con 0x7fb39c103e30 2024-09-10T08:36:11.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.333+0000 7fb399600640 1 -- 172.21.15.89:0/2220372559 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb390010910 con 0x7fb39c103e30 2024-09-10T08:36:11.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.334+0000 7fb399600640 1 -- 172.21.15.89:0/2220372559 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3900188f0 con 0x7fb39c103e30 2024-09-10T08:36:11.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.334+0000 7fb3a2600640 1 -- 172.21.15.89:0/2220372559 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb39c1a6ff0 con 0x7fb39c103e30 2024-09-10T08:36:11.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.335+0000 7fb3a2600640 1 -- 172.21.15.89:0/2220372559 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb39c104210 con 0x7fb39c103e30 2024-09-10T08:36:11.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.337+0000 7fb399600640 1 -- 172.21.15.89:0/2220372559 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb390010ac0 con 0x7fb39c103e30 2024-09-10T08:36:11.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.338+0000 7fb399600640 1 --2- 172.21.15.89:0/2220372559 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb3840a2c00 0x7fb3840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:11.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.338+0000 7fb399600640 1 -- 172.21.15.89:0/2220372559 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7fb3900cf330 con 0x7fb39c103e30 2024-09-10T08:36:11.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.338+0000 7fb39be00640 1 --2- 172.21.15.89:0/2220372559 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb3840a2c00 0x7fb3840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:11.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.339+0000 7fb39be00640 1 --2- 172.21.15.89:0/2220372559 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb3840a2c00 0x7fb3840a50c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb388000c00 tx=0x7fb388005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:11.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.340+0000 7fb399600640 1 -- 172.21.15.89:0/2220372559 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb39009c040 con 0x7fb39c103e30 2024-09-10T08:36:11.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.551+0000 7fb3a2600640 1 -- 172.21.15.89:0/2220372559 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb39c19b7d0 con 0x7fb39c103e30 2024-09-10T08:36:11.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.552+0000 7fb399600640 1 -- 172.21.15.89:0/2220372559 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fb39c19b7d0 con 0x7fb39c103e30 2024-09-10T08:36:11.551 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:36:11.552 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":1,"btime":"2024-09-10T08:28:49:696758+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-10T08:36:11.552 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 1 2024-09-10T08:36:11.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.554+0000 7fb3a2600640 1 -- 172.21.15.89:0/2220372559 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb3840a2c00 msgr2=0x7fb3840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:11.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.554+0000 7fb3a2600640 1 --2- 172.21.15.89:0/2220372559 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb3840a2c00 0x7fb3840a50c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb388000c00 tx=0x7fb388005b50 comp rx=0 tx=0).stop 2024-09-10T08:36:11.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.554+0000 7fb3a2600640 1 -- 172.21.15.89:0/2220372559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb39c103e30 msgr2=0x7fb39c19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:11.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.554+0000 7fb3a2600640 1 --2- 172.21.15.89:0/2220372559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb39c103e30 0x7fb39c19a470 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb39000e9f0 tx=0x7fb39000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:36:11.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.554+0000 7fb3a2600640 1 -- 172.21.15.89:0/2220372559 shutdown_connections 2024-09-10T08:36:11.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.554+0000 7fb3a2600640 1 --2- 172.21.15.89:0/2220372559 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb3840a2c00 0x7fb3840a50c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:11.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.554+0000 7fb3a2600640 1 --2- 172.21.15.89:0/2220372559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb39c108f60 0x7fb39c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:11.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.554+0000 7fb3a2600640 1 --2- 172.21.15.89:0/2220372559 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb39c1047e0 0x7fb39c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:11.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.554+0000 7fb3a2600640 1 --2- 172.21.15.89:0/2220372559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb39c103e30 0x7fb39c19a470 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:11.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.554+0000 7fb3a2600640 1 -- 172.21.15.89:0/2220372559 >> 172.21.15.89:0/2220372559 conn(0x7fb39c0ff8d0 msgr2=0x7fb39c100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:11.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.555+0000 7fb3a2600640 1 -- 172.21.15.89:0/2220372559 shutdown_connections 2024-09-10T08:36:11.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:11.555+0000 7fb3a2600640 1 -- 172.21.15.89:0/2220372559 wait complete. 2024-09-10T08:36:11.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:11 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2220372559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:11.909 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:11 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2220372559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:11.910 INFO:journalctl@ceph.mds.l.smithi149.stdout:Sep 10 08:36:11 smithi149 systemd[1]: Starting Ceph mds.l for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:36:12.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:11 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2220372559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:12.175 INFO:journalctl@ceph.mds.l.smithi149.stdout:Sep 10 08:36:11 smithi149 podman[65800]: 2024-09-10 08:36:11.812483925 +0000 UTC m=+0.016175785 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:36:12.175 INFO:journalctl@ceph.mds.l.smithi149.stdout:Sep 10 08:36:11 smithi149 podman[65800]: 2024-09-10 08:36:11.921259985 +0000 UTC m=+0.124951832 container create 2d0dd4a58e261b911306a522e4bfaab20e7650a1c6913caf64e5ea67e4937aae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T08:36:12.175 INFO:journalctl@ceph.mds.l.smithi149.stdout:Sep 10 08:36:12 smithi149 podman[65800]: 2024-09-10 08:36:12.174954452 +0000 UTC m=+0.378646314 container init 2d0dd4a58e261b911306a522e4bfaab20e7650a1c6913caf64e5ea67e4937aae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T08:36:12.470 INFO:journalctl@ceph.mds.l.smithi149.stdout:Sep 10 08:36:12 smithi149 podman[65800]: 2024-09-10 08:36:12.177844692 +0000 UTC m=+0.381536541 container start 2d0dd4a58e261b911306a522e4bfaab20e7650a1c6913caf64e5ea67e4937aae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T08:36:12.470 INFO:journalctl@ceph.mds.l.smithi149.stdout:Sep 10 08:36:12 smithi149 ceph-mds[65819]: -- 172.21.15.149:0/1013658258 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d7f32a7860 con 0x55d7f3285400 2024-09-10T08:36:12.470 INFO:journalctl@ceph.mds.l.smithi149.stdout:Sep 10 08:36:12 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-l[65815]: starting mds.l at 2024-09-10T08:36:12.470 INFO:journalctl@ceph.mds.l.smithi149.stdout:Sep 10 08:36:12 smithi149 bash[65800]: 2d0dd4a58e261b911306a522e4bfaab20e7650a1c6913caf64e5ea67e4937aae 2024-09-10T08:36:12.471 INFO:journalctl@ceph.mds.l.smithi149.stdout:Sep 10 08:36:12 smithi149 systemd[1]: Started Ceph mds.l for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:36:12.759 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:12 smithi089 ceph-mon[30757]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:12.759 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:12 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.149:6832/210698608,v1:172.21.15.149:6833/210698608] up:boot 2024-09-10T08:36:12.759 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:12 smithi089 ceph-mon[30757]: fsmap 1 up:standby 2024-09-10T08:36:12.759 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:12 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T08:36:12.759 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:12 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:12.760 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:12 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:12.760 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:12 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:12.760 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:12 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:12.760 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:12 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:12.760 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:12 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:12.860 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:12 smithi111 ceph-mon[35220]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:12.860 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:12 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.149:6832/210698608,v1:172.21.15.149:6833/210698608] up:boot 2024-09-10T08:36:12.860 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:12 smithi111 ceph-mon[35220]: fsmap 1 up:standby 2024-09-10T08:36:12.861 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:12 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T08:36:12.861 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:12 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:12.861 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:12 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:12.861 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:12 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:12.861 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:12 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:12.861 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:12 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:12.861 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:12 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:12.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:12 smithi149 ceph-mon[36167]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:12.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:12 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.149:6832/210698608,v1:172.21.15.149:6833/210698608] up:boot 2024-09-10T08:36:12.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:12 smithi149 ceph-mon[36167]: fsmap 1 up:standby 2024-09-10T08:36:12.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:12 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T08:36:12.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:12 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:12.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:12 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:12.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:12 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:12.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:12 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:12.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:12 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:12.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:12 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:13.906 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:13 smithi111 ceph-mon[35220]: Deploying daemon mds.k on smithi111 2024-09-10T08:36:13.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:13 smithi149 ceph-mon[36167]: Deploying daemon mds.k on smithi111 2024-09-10T08:36:14.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:13 smithi089 ceph-mon[30757]: Deploying daemon mds.k on smithi111 2024-09-10T08:36:14.285 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 10 08:36:14 smithi111 systemd[1]: Starting Ceph mds.k for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:36:14.486 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:36:14.543 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 10 08:36:14 smithi111 podman[64127]: 2024-09-10 08:36:14.397363751 +0000 UTC m=+0.015705304 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:36:14.543 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 10 08:36:14 smithi111 podman[64127]: 2024-09-10 08:36:14.50979991 +0000 UTC m=+0.128141452 container create 33aea65ed9d8d0d6bf6fc2b89d82242919909dfa693306e039402b3036e2eaf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T08:36:14.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:14 smithi089 ceph-mon[30757]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:14.860 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:36:14.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:14 smithi149 ceph-mon[36167]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:15.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:14 smithi111 ceph-mon[35220]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:15.043 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 10 08:36:14 smithi111 podman[64127]: 2024-09-10 08:36:14.746240425 +0000 UTC m=+0.364581966 container init 33aea65ed9d8d0d6bf6fc2b89d82242919909dfa693306e039402b3036e2eaf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-k, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:36:15.043 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 10 08:36:14 smithi111 podman[64127]: 2024-09-10 08:36:14.74913218 +0000 UTC m=+0.367473725 container start 33aea65ed9d8d0d6bf6fc2b89d82242919909dfa693306e039402b3036e2eaf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T08:36:15.043 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 10 08:36:14 smithi111 ceph-mds[64146]: -- 172.21.15.111:0/3683326791 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dc5951b860 con 0x55dc594f8800 2024-09-10T08:36:15.044 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 10 08:36:14 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-k[64142]: starting mds.k at 2024-09-10T08:36:15.044 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 10 08:36:14 smithi111 bash[64127]: 33aea65ed9d8d0d6bf6fc2b89d82242919909dfa693306e039402b3036e2eaf7 2024-09-10T08:36:15.044 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 10 08:36:14 smithi111 systemd[1]: Started Ceph mds.k for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:36:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.430+0000 7f271c400640 1 -- 172.21.15.89:0/4292779319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2714106f20 msgr2=0x7f271410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:15.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.430+0000 7f271c400640 1 --2- 172.21.15.89:0/4292779319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2714106f20 0x7f271410eaa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f270c009900 tx=0x7f270c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:36:15.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.430+0000 7f271c400640 1 -- 172.21.15.89:0/4292779319 shutdown_connections 2024-09-10T08:36:15.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.430+0000 7f271c400640 1 --2- 172.21.15.89:0/4292779319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2714106f20 0x7f271410eaa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:15.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.430+0000 7f271c400640 1 --2- 172.21.15.89:0/4292779319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27141027a0 0x7f27141067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:15.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.430+0000 7f271c400640 1 --2- 172.21.15.89:0/4292779319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2714101df0 0x7f27141021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:15.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.430+0000 7f271c400640 1 -- 172.21.15.89:0/4292779319 >> 172.21.15.89:0/4292779319 conn(0x7f27140fd8b0 msgr2=0x7f27140ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:15.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.431+0000 7f271c400640 1 -- 172.21.15.89:0/4292779319 shutdown_connections 2024-09-10T08:36:15.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.431+0000 7f271c400640 1 -- 172.21.15.89:0/4292779319 wait complete. 2024-09-10T08:36:15.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.432+0000 7f271c400640 1 Processor -- start 2024-09-10T08:36:15.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.432+0000 7f271c400640 1 -- start start 2024-09-10T08:36:15.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.432+0000 7f271c400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2714101df0 0x7f271419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:15.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.432+0000 7f271c400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27141027a0 0x7f271419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:15.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.432+0000 7f271c400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2714106f20 0x7f271419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.432+0000 7f271c400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2714111f80 con 0x7f2714106f20 2024-09-10T08:36:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.432+0000 7f271c400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2714111e00 con 0x7f27141027a0 2024-09-10T08:36:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.432+0000 7f271c400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2714112100 con 0x7f2714101df0 2024-09-10T08:36:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.433+0000 7f2719000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27141027a0 0x7f271419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.433+0000 7f2719000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27141027a0 0x7f271419aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:37116/0 (socket says 172.21.15.89:37116) 2024-09-10T08:36:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.433+0000 7f2719a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2714101df0 0x7f271419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.433+0000 7f2719000640 1 -- 172.21.15.89:0/907846637 learned_addr learned my addr 172.21.15.89:0/907846637 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:36:15.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.433+0000 7f271a400640 1 --2- 172.21.15.89:0/907846637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2714106f20 0x7f271419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.433+0000 7f2719000640 1 -- 172.21.15.89:0/907846637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2714101df0 msgr2=0x7f271419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.433+0000 7f2719000640 1 --2- 172.21.15.89:0/907846637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2714101df0 0x7f271419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.433+0000 7f2719000640 1 -- 172.21.15.89:0/907846637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2714106f20 msgr2=0x7f271419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.433+0000 7f2719000640 1 --2- 172.21.15.89:0/907846637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2714106f20 0x7f271419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.433+0000 7f2719000640 1 -- 172.21.15.89:0/907846637 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271419f550 con 0x7f27141027a0 2024-09-10T08:36:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.433+0000 7f2719000640 1 --2- 172.21.15.89:0/907846637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27141027a0 0x7f271419aae0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f270000e9f0 tx=0x7f270000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.434+0000 7f2712000640 1 -- 172.21.15.89:0/907846637 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270000cd80 con 0x7f27141027a0 2024-09-10T08:36:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.434+0000 7f2712000640 1 -- 172.21.15.89:0/907846637 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2700010910 con 0x7f27141027a0 2024-09-10T08:36:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.434+0000 7f271c400640 1 -- 172.21.15.89:0/907846637 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271419f840 con 0x7f27141027a0 2024-09-10T08:36:15.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.434+0000 7f2712000640 1 -- 172.21.15.89:0/907846637 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700018990 con 0x7f27141027a0 2024-09-10T08:36:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.434+0000 7f271c400640 1 -- 172.21.15.89:0/907846637 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27141a7120 con 0x7f27141027a0 2024-09-10T08:36:15.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.435+0000 7f271c400640 1 -- 172.21.15.89:0/907846637 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27141021d0 con 0x7f27141027a0 2024-09-10T08:36:15.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.437+0000 7f2712000640 1 -- 172.21.15.89:0/907846637 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2700010ac0 con 0x7f27141027a0 2024-09-10T08:36:15.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.439+0000 7f2712000640 1 --2- 172.21.15.89:0/907846637 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f26fc0a2b80 0x7f26fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:15.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.439+0000 7f2712000640 1 -- 172.21.15.89:0/907846637 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f27000ceed0 con 0x7f27141027a0 2024-09-10T08:36:15.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.439+0000 7f2719a00640 1 --2- 172.21.15.89:0/907846637 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f26fc0a2b80 0x7f26fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:15.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.440+0000 7f2719a00640 1 --2- 172.21.15.89:0/907846637 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f26fc0a2b80 0x7f26fc0a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2708005fd0 tx=0x7f2708009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:15.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.442+0000 7f2712000640 1 -- 172.21.15.89:0/907846637 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f270009bbe0 con 0x7f27141027a0 2024-09-10T08:36:15.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.665+0000 7f271c400640 1 -- 172.21.15.89:0/907846637 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27141a7380 con 0x7f27141027a0 2024-09-10T08:36:15.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.665+0000 7f2712000640 1 -- 172.21.15.89:0/907846637 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2263 (secure 0 0 0) 0x7f270009fee0 con 0x7f27141027a0 2024-09-10T08:36:15.668 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:36:15.669 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":3,"btime":"2024-09-10T08:36:15:617094+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T08:36:15.669 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 3 2024-09-10T08:36:15.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.667+0000 7f271c400640 1 -- 172.21.15.89:0/907846637 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f26fc0a2b80 msgr2=0x7f26fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:15.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.667+0000 7f271c400640 1 --2- 172.21.15.89:0/907846637 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f26fc0a2b80 0x7f26fc0a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2708005fd0 tx=0x7f2708009680 comp rx=0 tx=0).stop 2024-09-10T08:36:15.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.667+0000 7f271c400640 1 -- 172.21.15.89:0/907846637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27141027a0 msgr2=0x7f271419aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:15.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.667+0000 7f271c400640 1 --2- 172.21.15.89:0/907846637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27141027a0 0x7f271419aae0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f270000e9f0 tx=0x7f270000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:36:15.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.667+0000 7f271c400640 1 -- 172.21.15.89:0/907846637 shutdown_connections 2024-09-10T08:36:15.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.667+0000 7f271c400640 1 --2- 172.21.15.89:0/907846637 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f26fc0a2b80 0x7f26fc0a5040 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:15.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.667+0000 7f271c400640 1 --2- 172.21.15.89:0/907846637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2714106f20 0x7f271419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:15.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.667+0000 7f271c400640 1 --2- 172.21.15.89:0/907846637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27141027a0 0x7f271419aae0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:15.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.667+0000 7f271c400640 1 --2- 172.21.15.89:0/907846637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2714101df0 0x7f271419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:15.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.667+0000 7f271c400640 1 -- 172.21.15.89:0/907846637 >> 172.21.15.89:0/907846637 conn(0x7f27140fd8b0 msgr2=0x7f27140fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:15.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.668+0000 7f271c400640 1 -- 172.21.15.89:0/907846637 shutdown_connections 2024-09-10T08:36:15.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:15.668+0000 7f271c400640 1 -- 172.21.15.89:0/907846637 wait complete. 2024-09-10T08:36:16.175 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:15 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:16.175 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:15 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:16.175 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:15 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:16.175 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:15 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:16.175 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:15 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:16.175 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:15 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:16.175 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:15 smithi089 ceph-mon[30757]: Deploying daemon mds.j on smithi089 2024-09-10T08:36:16.175 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:15 smithi089 ceph-mon[30757]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:16.175 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:15 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.111:6832/2902311847,v1:172.21.15.111:6833/2902311847] up:boot 2024-09-10T08:36:16.175 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:15 smithi089 ceph-mon[30757]: fsmap 2 up:standby 2024-09-10T08:36:16.175 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:15 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T08:36:16.175 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:15 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/907846637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:16.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:15 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:16.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:15 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:16.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:15 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:16.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:15 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:16.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:15 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:16.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:15 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:16.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:15 smithi149 ceph-mon[36167]: Deploying daemon mds.j on smithi089 2024-09-10T08:36:16.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:15 smithi149 ceph-mon[36167]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:16.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:15 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.111:6832/2902311847,v1:172.21.15.111:6833/2902311847] up:boot 2024-09-10T08:36:16.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:15 smithi149 ceph-mon[36167]: fsmap 2 up:standby 2024-09-10T08:36:16.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:15 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T08:36:16.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:15 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/907846637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:16.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:15 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:16.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:15 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:16.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:15 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:16.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:15 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:16.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:15 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:16.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:15 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:16.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:15 smithi111 ceph-mon[35220]: Deploying daemon mds.j on smithi089 2024-09-10T08:36:16.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:15 smithi111 ceph-mon[35220]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:16.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:15 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.111:6832/2902311847,v1:172.21.15.111:6833/2902311847] up:boot 2024-09-10T08:36:16.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:15 smithi111 ceph-mon[35220]: fsmap 2 up:standby 2024-09-10T08:36:16.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:15 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T08:36:16.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:15 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/907846637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:17.969 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 10 08:36:17 smithi089 systemd[1]: Starting Ceph mds.j for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:36:18.246 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 10 08:36:17 smithi089 podman[70137]: 2024-09-10 08:36:17.873507745 +0000 UTC m=+0.027031317 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:36:18.246 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 10 08:36:18 smithi089 podman[70137]: 2024-09-10 08:36:18.009823487 +0000 UTC m=+0.163347054 container create 2d77cfd89db9c09b152771e783836392b8edf12456e0e6f67d082aa320748d5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T08:36:18.246 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 10 08:36:18 smithi089 podman[70137]: 2024-09-10 08:36:18.246580567 +0000 UTC m=+0.400104155 container init 2d77cfd89db9c09b152771e783836392b8edf12456e0e6f67d082aa320748d5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T08:36:18.553 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 10 08:36:18 smithi089 podman[70137]: 2024-09-10 08:36:18.252965874 +0000 UTC m=+0.406489449 container start 2d77cfd89db9c09b152771e783836392b8edf12456e0e6f67d082aa320748d5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-j, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2) 2024-09-10T08:36:18.553 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 10 08:36:18 smithi089 ceph-mds[70156]: -- 172.21.15.89:0/3890639602 <== mon.0 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fd0aedfc20 con 0x55fd0aebd400 2024-09-10T08:36:18.553 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 10 08:36:18 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-j[70152]: starting mds.j at 2024-09-10T08:36:18.553 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 10 08:36:18 smithi089 bash[70137]: 2d77cfd89db9c09b152771e783836392b8edf12456e0e6f67d082aa320748d5d 2024-09-10T08:36:18.553 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 10 08:36:18 smithi089 systemd[1]: Started Ceph mds.j for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:36:18.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:18 smithi149 ceph-mon[36167]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:18.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:18 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:18.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:18 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:18.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:18 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:18.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:18 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:18.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:18 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:18.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:18 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:19.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:18 smithi111 ceph-mon[35220]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:19.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:18 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:19.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:18 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:19.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:18 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:19.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:18 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:19.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:18 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:19.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:18 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:19.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:18 smithi089 ceph-mon[30757]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:19.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:18 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:19.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:18 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:19.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:18 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:19.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:18 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:19.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:18 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:19.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:18 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:19.373 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:36:19.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:19 smithi089 ceph-mon[30757]: Deploying daemon mds.d on smithi089 2024-09-10T08:36:19.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:19 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.89:6834/2129325722,v1:172.21.15.89:6835/2129325722] up:boot 2024-09-10T08:36:19.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:19 smithi089 ceph-mon[30757]: fsmap 3 up:standby 2024-09-10T08:36:19.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:19 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T08:36:19.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:19 smithi149 ceph-mon[36167]: Deploying daemon mds.d on smithi089 2024-09-10T08:36:19.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:19 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.89:6834/2129325722,v1:172.21.15.89:6835/2129325722] up:boot 2024-09-10T08:36:19.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:19 smithi149 ceph-mon[36167]: fsmap 3 up:standby 2024-09-10T08:36:19.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:19 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T08:36:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:19 smithi111 ceph-mon[35220]: Deploying daemon mds.d on smithi089 2024-09-10T08:36:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:19 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.89:6834/2129325722,v1:172.21.15.89:6835/2129325722] up:boot 2024-09-10T08:36:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:19 smithi111 ceph-mon[35220]: fsmap 3 up:standby 2024-09-10T08:36:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:19 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T08:36:20.304 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:36:20.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:20 smithi149 ceph-mon[36167]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:20.870 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:20 smithi089 ceph-mon[30757]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:20.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.877+0000 7fdbe9800640 1 -- 172.21.15.89:0/86136868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbe4103e30 msgr2=0x7fdbe4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:20.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.877+0000 7fdbe9800640 1 --2- 172.21.15.89:0/86136868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbe4103e30 0x7fdbe4104210 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009900 tx=0x7fdbd402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:36:20.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.877+0000 7fdbe9800640 1 -- 172.21.15.89:0/86136868 shutdown_connections 2024-09-10T08:36:20.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.877+0000 7fdbe9800640 1 --2- 172.21.15.89:0/86136868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbe4108f60 0x7fdbe4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:20.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.877+0000 7fdbe9800640 1 --2- 172.21.15.89:0/86136868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbe41047e0 0x7fdbe4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:20.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.877+0000 7fdbe9800640 1 --2- 172.21.15.89:0/86136868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbe4103e30 0x7fdbe4104210 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:20.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.877+0000 7fdbe9800640 1 -- 172.21.15.89:0/86136868 >> 172.21.15.89:0/86136868 conn(0x7fdbe40ff910 msgr2=0x7fdbe4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:20.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.877+0000 7fdbe9800640 1 -- 172.21.15.89:0/86136868 shutdown_connections 2024-09-10T08:36:20.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.877+0000 7fdbe9800640 1 -- 172.21.15.89:0/86136868 wait complete. 2024-09-10T08:36:20.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.878+0000 7fdbe9800640 1 Processor -- start 2024-09-10T08:36:20.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.878+0000 7fdbe9800640 1 -- start start 2024-09-10T08:36:20.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.879+0000 7fdbe9800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbe4103e30 0x7fdbe419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:20.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.879+0000 7fdbe9800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbe41047e0 0x7fdbe419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:20.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.879+0000 7fdbe9800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbe4108f60 0x7fdbe419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:20.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.879+0000 7fdbe9800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe4114010 con 0x7fdbe41047e0 2024-09-10T08:36:20.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.879+0000 7fdbe9800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe4113e90 con 0x7fdbe4103e30 2024-09-10T08:36:20.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.879+0000 7fdbe9800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe4114190 con 0x7fdbe4108f60 2024-09-10T08:36:20.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.879+0000 7fdbe3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbe41047e0 0x7fdbe419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:20.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.879+0000 7fdbe3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbe4103e30 0x7fdbe419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:20.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.879+0000 7fdbe3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbe4103e30 0x7fdbe419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:37160/0 (socket says 172.21.15.89:37160) 2024-09-10T08:36:20.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.879+0000 7fdbe3e00640 1 -- 172.21.15.89:0/2978131198 learned_addr learned my addr 172.21.15.89:0/2978131198 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:36:20.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.879+0000 7fdbe3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbe41047e0 0x7fdbe419aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53382/0 (socket says 172.21.15.89:53382) 2024-09-10T08:36:20.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.879+0000 7fdbe8e00640 1 --2- 172.21.15.89:0/2978131198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbe4108f60 0x7fdbe419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:20.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.879+0000 7fdbe3e00640 1 -- 172.21.15.89:0/2978131198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbe4108f60 msgr2=0x7fdbe419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:20.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.879+0000 7fdbe3e00640 1 --2- 172.21.15.89:0/2978131198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbe4108f60 0x7fdbe419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:20.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.879+0000 7fdbe3e00640 1 -- 172.21.15.89:0/2978131198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbe41047e0 msgr2=0x7fdbe419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:20.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.879+0000 7fdbe3e00640 1 --2- 172.21.15.89:0/2978131198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbe41047e0 0x7fdbe419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:20.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.879+0000 7fdbe3e00640 1 -- 172.21.15.89:0/2978131198 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe419f4a0 con 0x7fdbe4103e30 2024-09-10T08:36:20.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.880+0000 7fdbe3e00640 1 --2- 172.21.15.89:0/2978131198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbe4103e30 0x7fdbe419a4f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4004770 tx=0x7fdbd4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:20.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.880+0000 7fdbe0c00640 1 -- 172.21.15.89:0/2978131198 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd403f070 con 0x7fdbe4103e30 2024-09-10T08:36:20.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.880+0000 7fdbe9800640 1 -- 172.21.15.89:0/2978131198 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe419f730 con 0x7fdbe4103e30 2024-09-10T08:36:20.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.880+0000 7fdbe0c00640 1 -- 172.21.15.89:0/2978131198 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd4031ca0 con 0x7fdbe4103e30 2024-09-10T08:36:20.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.880+0000 7fdbe0c00640 1 -- 172.21.15.89:0/2978131198 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4035030 con 0x7fdbe4103e30 2024-09-10T08:36:20.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.880+0000 7fdbe9800640 1 -- 172.21.15.89:0/2978131198 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe41a7070 con 0x7fdbe4103e30 2024-09-10T08:36:20.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.882+0000 7fdbe9800640 1 -- 172.21.15.89:0/2978131198 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbe4105310 con 0x7fdbe4103e30 2024-09-10T08:36:20.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.883+0000 7fdbe0c00640 1 -- 172.21.15.89:0/2978131198 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdbd4042070 con 0x7fdbe4103e30 2024-09-10T08:36:20.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.885+0000 7fdbe0c00640 1 --2- 172.21.15.89:0/2978131198 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdbc80a2b80 0x7fdbc80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:20.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.885+0000 7fdbe0c00640 1 -- 172.21.15.89:0/2978131198 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7fdbd40ef220 con 0x7fdbe4103e30 2024-09-10T08:36:20.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.885+0000 7fdbe3400640 1 --2- 172.21.15.89:0/2978131198 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdbc80a2b80 0x7fdbc80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:20.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.886+0000 7fdbe3400640 1 --2- 172.21.15.89:0/2978131198 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdbc80a2b80 0x7fdbc80a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40ffd50 tx=0x7fdbd0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:20.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:20.889+0000 7fdbe0c00640 1 -- 172.21.15.89:0/2978131198 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdbd40bbff0 con 0x7fdbe4103e30 2024-09-10T08:36:21.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:20 smithi111 ceph-mon[35220]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:21.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:21.118+0000 7fdbe9800640 1 -- 172.21.15.89:0/2978131198 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbe410c140 con 0x7fdbe4103e30 2024-09-10T08:36:21.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:21.119+0000 7fdbe0c00640 1 -- 172.21.15.89:0/2978131198 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7fdbd40c02f0 con 0x7fdbe4103e30 2024-09-10T08:36:21.121 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:36:21.121 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":4,"btime":"2024-09-10T08:36:18:605178+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T08:36:21.121 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 4 2024-09-10T08:36:21.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:21.122+0000 7fdbe9800640 1 -- 172.21.15.89:0/2978131198 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdbc80a2b80 msgr2=0x7fdbc80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:21.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:21.122+0000 7fdbe9800640 1 --2- 172.21.15.89:0/2978131198 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdbc80a2b80 0x7fdbc80a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40ffd50 tx=0x7fdbd0005f90 comp rx=0 tx=0).stop 2024-09-10T08:36:21.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:21.122+0000 7fdbe9800640 1 -- 172.21.15.89:0/2978131198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbe4103e30 msgr2=0x7fdbe419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:21.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:21.122+0000 7fdbe9800640 1 --2- 172.21.15.89:0/2978131198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbe4103e30 0x7fdbe419a4f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4004770 tx=0x7fdbd4003c10 comp rx=0 tx=0).stop 2024-09-10T08:36:21.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:21.122+0000 7fdbe9800640 1 -- 172.21.15.89:0/2978131198 shutdown_connections 2024-09-10T08:36:21.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:21.122+0000 7fdbe9800640 1 --2- 172.21.15.89:0/2978131198 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdbc80a2b80 0x7fdbc80a5040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:21.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:21.122+0000 7fdbe9800640 1 --2- 172.21.15.89:0/2978131198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbe4108f60 0x7fdbe419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:21.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:21.122+0000 7fdbe9800640 1 --2- 172.21.15.89:0/2978131198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbe41047e0 0x7fdbe419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:21.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:21.122+0000 7fdbe9800640 1 --2- 172.21.15.89:0/2978131198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbe4103e30 0x7fdbe419a4f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:21.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:21.122+0000 7fdbe9800640 1 -- 172.21.15.89:0/2978131198 >> 172.21.15.89:0/2978131198 conn(0x7fdbe40ff910 msgr2=0x7fdbe4100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:21.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:21.122+0000 7fdbe9800640 1 -- 172.21.15.89:0/2978131198 shutdown_connections 2024-09-10T08:36:21.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:21.122+0000 7fdbe9800640 1 -- 172.21.15.89:0/2978131198 wait complete. 2024-09-10T08:36:21.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:21 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2978131198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:21.784 INFO:journalctl@ceph.mds.d.smithi089.stdout:Sep 10 08:36:21 smithi089 podman[70541]: 2024-09-10 08:36:21.65118771 +0000 UTC m=+0.264232464 container create 50ead59fd148974909ce07438264477e5b716981f062daf0d3a1350b6a3dbe58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T08:36:21.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:21 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2978131198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:22.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:21 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2978131198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:22.553 INFO:journalctl@ceph.mds.d.smithi089.stdout:Sep 10 08:36:22 smithi089 podman[70541]: 2024-09-10 08:36:22.215570234 +0000 UTC m=+0.828614963 container init 50ead59fd148974909ce07438264477e5b716981f062daf0d3a1350b6a3dbe58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:36:22.553 INFO:journalctl@ceph.mds.d.smithi089.stdout:Sep 10 08:36:22 smithi089 podman[70541]: 2024-09-10 08:36:22.221708091 +0000 UTC m=+0.834752805 container start 50ead59fd148974909ce07438264477e5b716981f062daf0d3a1350b6a3dbe58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-d, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T08:36:22.553 INFO:journalctl@ceph.mds.d.smithi089.stdout:Sep 10 08:36:22 smithi089 ceph-mds[70563]: -- 172.21.15.89:0/3102099431 <== mon.0 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a0dc3f5860 con 0x55a0dc3d3400 2024-09-10T08:36:22.553 INFO:journalctl@ceph.mds.d.smithi089.stdout:Sep 10 08:36:22 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-d[70559]: starting mds.d at 2024-09-10T08:36:22.553 INFO:journalctl@ceph.mds.d.smithi089.stdout:Sep 10 08:36:22 smithi089 bash[70541]: 50ead59fd148974909ce07438264477e5b716981f062daf0d3a1350b6a3dbe58 2024-09-10T08:36:22.553 INFO:journalctl@ceph.mds.d.smithi089.stdout:Sep 10 08:36:22 smithi089 systemd[1]: Started Ceph mds.d for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:36:22.885 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:22 smithi149 ceph-mon[36167]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:22.885 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:22 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:22.885 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:22 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:22.885 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:22 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:22.885 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:22 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:22.885 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:22 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:22.885 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:22 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:22.944 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:22 smithi111 ceph-mon[35220]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:22.945 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:22 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:22.945 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:22 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:22.945 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:22 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:22.945 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:22 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:22.945 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:22 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:22.945 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:22 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:23.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:22 smithi089 ceph-mon[30757]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:23.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:22 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:23.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:22 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:23.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:22 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:23.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:22 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:23.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:22 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:23.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:22 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:23.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:23 smithi111 ceph-mon[35220]: Deploying daemon mds.e on smithi111 2024-09-10T08:36:23.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:23 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.89:6836/3715416661,v1:172.21.15.89:6837/3715416661] up:boot 2024-09-10T08:36:23.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:23 smithi111 ceph-mon[35220]: fsmap 4 up:standby 2024-09-10T08:36:23.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:23 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T08:36:23.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:23 smithi149 ceph-mon[36167]: Deploying daemon mds.e on smithi111 2024-09-10T08:36:23.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:23 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.89:6836/3715416661,v1:172.21.15.89:6837/3715416661] up:boot 2024-09-10T08:36:23.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:23 smithi149 ceph-mon[36167]: fsmap 4 up:standby 2024-09-10T08:36:23.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:23 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T08:36:24.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:23 smithi089 ceph-mon[30757]: Deploying daemon mds.e on smithi111 2024-09-10T08:36:24.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:23 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.89:6836/3715416661,v1:172.21.15.89:6837/3715416661] up:boot 2024-09-10T08:36:24.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:23 smithi089 ceph-mon[30757]: fsmap 4 up:standby 2024-09-10T08:36:24.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:23 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T08:36:24.116 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:36:24.494 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:36:24.543 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 10 08:36:24 smithi111 systemd[1]: Starting Ceph mds.e for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:36:24.926 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:24 smithi111 ceph-mon[35220]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:24.926 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 10 08:36:24 smithi111 podman[64443]: 2024-09-10 08:36:24.547976403 +0000 UTC m=+0.016130672 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:36:24.926 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 10 08:36:24 smithi111 podman[64443]: 2024-09-10 08:36:24.674162864 +0000 UTC m=+0.142317135 container create c196a0b6f6fc1f368ed84eba8e1aeb85efaf874ecedca65be76b0e49754e7b3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:36:24.927 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 10 08:36:24 smithi111 podman[64443]: 2024-09-10 08:36:24.927121034 +0000 UTC m=+0.395275307 container init c196a0b6f6fc1f368ed84eba8e1aeb85efaf874ecedca65be76b0e49754e7b3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T08:36:24.945 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:24 smithi089 ceph-mon[30757]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:24.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:24 smithi149 ceph-mon[36167]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:25.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.055+0000 7fccdd000640 1 -- 172.21.15.89:0/359656525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccd8101de0 msgr2=0x7fccd81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:25.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.055+0000 7fccdd000640 1 --2- 172.21.15.89:0/359656525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccd8101de0 0x7fccd81021c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009ca0 tx=0x7fccc802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:36:25.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.055+0000 7fccdd000640 1 -- 172.21.15.89:0/359656525 shutdown_connections 2024-09-10T08:36:25.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.055+0000 7fccdd000640 1 --2- 172.21.15.89:0/359656525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccd8106f10 0x7fccd810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:25.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.055+0000 7fccdd000640 1 --2- 172.21.15.89:0/359656525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccd8102790 0x7fccd81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:25.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.055+0000 7fccdd000640 1 --2- 172.21.15.89:0/359656525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccd8101de0 0x7fccd81021c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:25.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.055+0000 7fccdd000640 1 -- 172.21.15.89:0/359656525 >> 172.21.15.89:0/359656525 conn(0x7fccd80fd8c0 msgr2=0x7fccd80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:25.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.055+0000 7fccdd000640 1 -- 172.21.15.89:0/359656525 shutdown_connections 2024-09-10T08:36:25.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.056+0000 7fccdd000640 1 -- 172.21.15.89:0/359656525 wait complete. 2024-09-10T08:36:25.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.056+0000 7fccdd000640 1 Processor -- start 2024-09-10T08:36:25.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.057+0000 7fccdd000640 1 -- start start 2024-09-10T08:36:25.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.057+0000 7fccdd000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccd8101de0 0x7fccd8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:25.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.057+0000 7fccdd000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccd8102790 0x7fccd8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:25.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.057+0000 7fccdd000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccd8106f10 0x7fccd819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:25.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.057+0000 7fccdd000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8111f80 con 0x7fccd8106f10 2024-09-10T08:36:25.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.057+0000 7fccdd000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8111e00 con 0x7fccd8102790 2024-09-10T08:36:25.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.057+0000 7fccdd000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8112100 con 0x7fccd8101de0 2024-09-10T08:36:25.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.057+0000 7fccd5400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccd8102790 0x7fccd8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:25.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.057+0000 7fccd5400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccd8102790 0x7fccd8198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:52258/0 (socket says 172.21.15.89:52258) 2024-09-10T08:36:25.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.057+0000 7fccd5400640 1 -- 172.21.15.89:0/677609880 learned_addr learned my addr 172.21.15.89:0/677609880 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:36:25.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.057+0000 7fccd6800640 1 --2- 172.21.15.89:0/677609880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccd8106f10 0x7fccd819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:25.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.057+0000 7fccd5e00640 1 --2- 172.21.15.89:0/677609880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccd8101de0 0x7fccd8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:25.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.058+0000 7fccd5400640 1 -- 172.21.15.89:0/677609880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccd8101de0 msgr2=0x7fccd8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:25.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.058+0000 7fccd5400640 1 --2- 172.21.15.89:0/677609880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccd8101de0 0x7fccd8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:25.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.058+0000 7fccd5400640 1 -- 172.21.15.89:0/677609880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccd8106f10 msgr2=0x7fccd819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:25.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.058+0000 7fccd5400640 1 --2- 172.21.15.89:0/677609880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccd8106f10 0x7fccd819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:25.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.058+0000 7fccd5400640 1 -- 172.21.15.89:0/677609880 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd819d3e0 con 0x7fccd8102790 2024-09-10T08:36:25.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.058+0000 7fccd5e00640 1 --2- 172.21.15.89:0/677609880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccd8101de0 0x7fccd8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:36:25.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.058+0000 7fccd5400640 1 --2- 172.21.15.89:0/677609880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccd8102790 0x7fccd8198970 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fccc400ea60 tx=0x7fccc400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:25.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.058+0000 7fcccea00640 1 -- 172.21.15.89:0/677609880 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc400cde0 con 0x7fccd8102790 2024-09-10T08:36:25.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.058+0000 7fccdd000640 1 -- 172.21.15.89:0/677609880 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd819d6d0 con 0x7fccd8102790 2024-09-10T08:36:25.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.058+0000 7fcccea00640 1 -- 172.21.15.89:0/677609880 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc40035f0 con 0x7fccd8102790 2024-09-10T08:36:25.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.058+0000 7fcccea00640 1 -- 172.21.15.89:0/677609880 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc4010670 con 0x7fccd8102790 2024-09-10T08:36:25.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.058+0000 7fccdd000640 1 -- 172.21.15.89:0/677609880 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd81a4fb0 con 0x7fccd8102790 2024-09-10T08:36:25.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.059+0000 7fccdd000640 1 -- 172.21.15.89:0/677609880 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd81021c0 con 0x7fccd8102790 2024-09-10T08:36:25.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.061+0000 7fcccea00640 1 -- 172.21.15.89:0/677609880 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fccc40037a0 con 0x7fccd8102790 2024-09-10T08:36:25.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.062+0000 7fcccea00640 1 --2- 172.21.15.89:0/677609880 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fccbc0a2b80 0x7fccbc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:25.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.062+0000 7fcccea00640 1 -- 172.21.15.89:0/677609880 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7fccc40cefa0 con 0x7fccd8102790 2024-09-10T08:36:25.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.062+0000 7fccd5e00640 1 --2- 172.21.15.89:0/677609880 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fccbc0a2b80 0x7fccbc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:25.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.063+0000 7fccd5e00640 1 --2- 172.21.15.89:0/677609880 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fccbc0a2b80 0x7fccbc0a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fccc8000f80 tx=0x7fccc8005fa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:25.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.066+0000 7fcccea00640 1 -- 172.21.15.89:0/677609880 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fccc409c300 con 0x7fccd8102790 2024-09-10T08:36:25.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.277+0000 7fccdd000640 1 -- 172.21.15.89:0/677609880 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccd8199790 con 0x7fccd8102790 2024-09-10T08:36:25.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.278+0000 7fcccea00640 1 -- 172.21.15.89:0/677609880 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3913 (secure 0 0 0) 0x7fccd8199790 con 0x7fccd8102790 2024-09-10T08:36:25.279 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:36:25.279 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":5,"btime":"2024-09-10T08:36:22:624038+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T08:36:25.279 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 5 2024-09-10T08:36:25.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.280+0000 7fccdd000640 1 -- 172.21.15.89:0/677609880 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fccbc0a2b80 msgr2=0x7fccbc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:25.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.280+0000 7fccdd000640 1 --2- 172.21.15.89:0/677609880 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fccbc0a2b80 0x7fccbc0a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fccc8000f80 tx=0x7fccc8005fa0 comp rx=0 tx=0).stop 2024-09-10T08:36:25.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.280+0000 7fccdd000640 1 -- 172.21.15.89:0/677609880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccd8102790 msgr2=0x7fccd8198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:25.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.280+0000 7fccdd000640 1 --2- 172.21.15.89:0/677609880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccd8102790 0x7fccd8198970 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fccc400ea60 tx=0x7fccc400ef20 comp rx=0 tx=0).stop 2024-09-10T08:36:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.281+0000 7fccdd000640 1 -- 172.21.15.89:0/677609880 shutdown_connections 2024-09-10T08:36:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.281+0000 7fccdd000640 1 --2- 172.21.15.89:0/677609880 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fccbc0a2b80 0x7fccbc0a5040 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.281+0000 7fccdd000640 1 --2- 172.21.15.89:0/677609880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccd8106f10 0x7fccd819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.281+0000 7fccdd000640 1 --2- 172.21.15.89:0/677609880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fccd8102790 0x7fccd8198970 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.281+0000 7fccdd000640 1 --2- 172.21.15.89:0/677609880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccd8101de0 0x7fccd8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.281+0000 7fccdd000640 1 -- 172.21.15.89:0/677609880 >> 172.21.15.89:0/677609880 conn(0x7fccd80fd8c0 msgr2=0x7fccd80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.281+0000 7fccdd000640 1 -- 172.21.15.89:0/677609880 shutdown_connections 2024-09-10T08:36:25.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:25.281+0000 7fccdd000640 1 -- 172.21.15.89:0/677609880 wait complete. 2024-09-10T08:36:25.293 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 10 08:36:24 smithi111 podman[64443]: 2024-09-10 08:36:24.930169987 +0000 UTC m=+0.398324266 container start c196a0b6f6fc1f368ed84eba8e1aeb85efaf874ecedca65be76b0e49754e7b3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-e, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T08:36:25.293 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 10 08:36:24 smithi111 ceph-mds[64463]: -- 172.21.15.111:0/3104091441 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c2d32eb860 con 0x55c2d32c9000 2024-09-10T08:36:25.293 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 10 08:36:24 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-e[64459]: starting mds.e at 2024-09-10T08:36:25.293 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 10 08:36:25 smithi111 bash[64443]: c196a0b6f6fc1f368ed84eba8e1aeb85efaf874ecedca65be76b0e49754e7b3f 2024-09-10T08:36:25.293 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 10 08:36:25 smithi111 systemd[1]: Started Ceph mds.e for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:36:26.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:26 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:26.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:26 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:26.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:26 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:26.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:26 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:26.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:26 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:26.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:26 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:26.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:26 smithi089 ceph-mon[30757]: Deploying daemon mds.f on smithi149 2024-09-10T08:36:26.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:26 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/677609880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:26.304 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:26 smithi089 ceph-mon[30757]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:26.304 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:26 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.111:6834/285258866,v1:172.21.15.111:6835/285258866] up:boot 2024-09-10T08:36:26.304 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:26 smithi089 ceph-mon[30757]: fsmap 5 up:standby 2024-09-10T08:36:26.304 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:26 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T08:36:26.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:26 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:26.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:26 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:26.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:26 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:26.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:26 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:26.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:26 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:26.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:26 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:26.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:26 smithi149 ceph-mon[36167]: Deploying daemon mds.f on smithi149 2024-09-10T08:36:26.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:26 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/677609880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:26.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:26 smithi149 ceph-mon[36167]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:26.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:26 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.111:6834/285258866,v1:172.21.15.111:6835/285258866] up:boot 2024-09-10T08:36:26.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:26 smithi149 ceph-mon[36167]: fsmap 5 up:standby 2024-09-10T08:36:26.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:26 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T08:36:26.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:26.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:26.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:26.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:26.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:26.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:26.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:26 smithi111 ceph-mon[35220]: Deploying daemon mds.f on smithi149 2024-09-10T08:36:26.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:26 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/677609880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:26.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:26 smithi111 ceph-mon[35220]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:26.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:26 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.111:6834/285258866,v1:172.21.15.111:6835/285258866] up:boot 2024-09-10T08:36:26.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:26 smithi111 ceph-mon[35220]: fsmap 5 up:standby 2024-09-10T08:36:26.545 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:26 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T08:36:27.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:Sep 10 08:36:27 smithi149 systemd[1]: Starting Ceph mds.f for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:36:27.509 INFO:journalctl@ceph.mds.f.smithi149.stdout:Sep 10 08:36:27 smithi149 podman[66114]: 2024-09-10 08:36:27.142099076 +0000 UTC m=+0.015834585 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:36:27.510 INFO:journalctl@ceph.mds.f.smithi149.stdout:Sep 10 08:36:27 smithi149 podman[66114]: 2024-09-10 08:36:27.267005726 +0000 UTC m=+0.140741216 container create 67b01eafeae780b66b4392179247837df6c20b3954754ea045a894f5720d9e04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-10T08:36:27.510 INFO:journalctl@ceph.mds.f.smithi149.stdout:Sep 10 08:36:27 smithi149 podman[66114]: 2024-09-10 08:36:27.509977958 +0000 UTC m=+0.383713448 container init 67b01eafeae780b66b4392179247837df6c20b3954754ea045a894f5720d9e04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-f, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:36:27.970 INFO:journalctl@ceph.mds.f.smithi149.stdout:Sep 10 08:36:27 smithi149 podman[66114]: 2024-09-10 08:36:27.512761516 +0000 UTC m=+0.386497010 container start 67b01eafeae780b66b4392179247837df6c20b3954754ea045a894f5720d9e04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-f, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-10T08:36:27.970 INFO:journalctl@ceph.mds.f.smithi149.stdout:Sep 10 08:36:27 smithi149 ceph-mds[66135]: -- 172.21.15.149:0/1326677883 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ee22be9860 con 0x55ee22bc6800 2024-09-10T08:36:27.970 INFO:journalctl@ceph.mds.f.smithi149.stdout:Sep 10 08:36:27 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-f[66131]: starting mds.f at 2024-09-10T08:36:27.970 INFO:journalctl@ceph.mds.f.smithi149.stdout:Sep 10 08:36:27 smithi149 bash[66114]: 67b01eafeae780b66b4392179247837df6c20b3954754ea045a894f5720d9e04 2024-09-10T08:36:27.971 INFO:journalctl@ceph.mds.f.smithi149.stdout:Sep 10 08:36:27 smithi149 systemd[1]: Started Ceph mds.f for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:36:28.020 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:36:28.396 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:36:28.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:28 smithi149 ceph-mon[36167]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:28.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:28 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.149:6834/881263752,v1:172.21.15.149:6835/881263752] up:boot 2024-09-10T08:36:28.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:28 smithi149 ceph-mon[36167]: fsmap 6 up:standby 2024-09-10T08:36:28.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:28 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T08:36:28.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:28 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:28.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:28 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:28.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:28 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:28.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:28 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:28.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:28 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:28.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:28 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:28.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:28 smithi149 ceph-mon[36167]: Deploying daemon mds.c on smithi149 2024-09-10T08:36:28.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:28 smithi089 ceph-mon[30757]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:28.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:28 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.149:6834/881263752,v1:172.21.15.149:6835/881263752] up:boot 2024-09-10T08:36:28.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:28 smithi089 ceph-mon[30757]: fsmap 6 up:standby 2024-09-10T08:36:28.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:28 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T08:36:28.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:28 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:28.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:28 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:28.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:28 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:28.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:28 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:28.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:28 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:28.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:28 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:28.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:28 smithi089 ceph-mon[30757]: Deploying daemon mds.c on smithi149 2024-09-10T08:36:29.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:28 smithi111 ceph-mon[35220]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:29.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:28 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.149:6834/881263752,v1:172.21.15.149:6835/881263752] up:boot 2024-09-10T08:36:29.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:28 smithi111 ceph-mon[35220]: fsmap 6 up:standby 2024-09-10T08:36:29.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:28 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T08:36:29.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:28 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:29.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:28 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:29.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:28 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:29.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:28 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:29.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:28 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:29.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:28 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:29.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:28 smithi111 ceph-mon[35220]: Deploying daemon mds.c on smithi149 2024-09-10T08:36:29.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.130+0000 7f30e7400640 1 -- 172.21.15.89:0/4292498563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30e01049a0 msgr2=0x7f30e01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:29.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.130+0000 7f30e7400640 1 --2- 172.21.15.89:0/4292498563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30e01049a0 0x7f30e01089f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f30cc009860 tx=0x7f30cc02a340 comp rx=0 tx=0).stop 2024-09-10T08:36:29.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.130+0000 7f30e7400640 1 -- 172.21.15.89:0/4292498563 shutdown_connections 2024-09-10T08:36:29.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.130+0000 7f30e7400640 1 --2- 172.21.15.89:0/4292498563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30e0109120 0x7f30e0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:29.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.130+0000 7f30e7400640 1 --2- 172.21.15.89:0/4292498563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30e01049a0 0x7f30e01089f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:29.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.130+0000 7f30e7400640 1 --2- 172.21.15.89:0/4292498563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30e0103ff0 0x7f30e01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:29.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.130+0000 7f30e7400640 1 -- 172.21.15.89:0/4292498563 >> 172.21.15.89:0/4292498563 conn(0x7f30e00ffa90 msgr2=0x7f30e0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:29.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.130+0000 7f30e7400640 1 -- 172.21.15.89:0/4292498563 shutdown_connections 2024-09-10T08:36:29.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.130+0000 7f30e7400640 1 -- 172.21.15.89:0/4292498563 wait complete. 2024-09-10T08:36:29.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.131+0000 7f30e7400640 1 Processor -- start 2024-09-10T08:36:29.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.131+0000 7f30e7400640 1 -- start start 2024-09-10T08:36:29.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.132+0000 7f30e7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30e0103ff0 0x7f30e019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:29.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.132+0000 7f30e7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30e01049a0 0x7f30e019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:29.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.132+0000 7f30e7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30e0109120 0x7f30e019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:29.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.132+0000 7f30e7400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f30e0114140 con 0x7f30e0109120 2024-09-10T08:36:29.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.132+0000 7f30e7400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f30e0113fc0 con 0x7f30e01049a0 2024-09-10T08:36:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.132+0000 7f30e7400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f30e01142c0 con 0x7f30e0103ff0 2024-09-10T08:36:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.132+0000 7f30e5400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30e0109120 0x7f30e019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.132+0000 7f30e4a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30e0103ff0 0x7f30e019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.132+0000 7f30e5400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30e0109120 0x7f30e019ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59026/0 (socket says 172.21.15.89:59026) 2024-09-10T08:36:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.132+0000 7f30e5400640 1 -- 172.21.15.89:0/4024700615 learned_addr learned my addr 172.21.15.89:0/4024700615 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:36:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.132+0000 7f30dfe00640 1 --2- 172.21.15.89:0/4024700615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30e01049a0 0x7f30e019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.132+0000 7f30e5400640 1 -- 172.21.15.89:0/4024700615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30e0103ff0 msgr2=0x7f30e019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.132+0000 7f30e5400640 1 --2- 172.21.15.89:0/4024700615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30e0103ff0 0x7f30e019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.132+0000 7f30e5400640 1 -- 172.21.15.89:0/4024700615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30e01049a0 msgr2=0x7f30e019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.132+0000 7f30e5400640 1 --2- 172.21.15.89:0/4024700615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30e01049a0 0x7f30e019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.132+0000 7f30e5400640 1 -- 172.21.15.89:0/4024700615 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e019f650 con 0x7f30e0109120 2024-09-10T08:36:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.133+0000 7f30e5400640 1 --2- 172.21.15.89:0/4024700615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30e0109120 0x7f30e019ef70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009fd0 tx=0x7f30d800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.133+0000 7f30dd600640 1 -- 172.21.15.89:0/4024700615 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d8004b20 con 0x7f30e0109120 2024-09-10T08:36:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.133+0000 7f30dd600640 1 -- 172.21.15.89:0/4024700615 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30d8003e50 con 0x7f30e0109120 2024-09-10T08:36:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.133+0000 7f30e7400640 1 -- 172.21.15.89:0/4024700615 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e019f940 con 0x7f30e0109120 2024-09-10T08:36:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.133+0000 7f30dd600640 1 -- 172.21.15.89:0/4024700615 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d800b630 con 0x7f30e0109120 2024-09-10T08:36:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.133+0000 7f30e7400640 1 -- 172.21.15.89:0/4024700615 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e01a7220 con 0x7f30e0109120 2024-09-10T08:36:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.134+0000 7f30e7400640 1 -- 172.21.15.89:0/4024700615 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30e01043d0 con 0x7f30e0109120 2024-09-10T08:36:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.136+0000 7f30dd600640 1 -- 172.21.15.89:0/4024700615 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f30d801d020 con 0x7f30e0109120 2024-09-10T08:36:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.137+0000 7f30dd600640 1 --2- 172.21.15.89:0/4024700615 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30c80a2b80 0x7f30c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.137+0000 7f30dd600640 1 -- 172.21.15.89:0/4024700615 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f30d80ceb80 con 0x7f30e0109120 2024-09-10T08:36:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.137+0000 7f30e4a00640 1 --2- 172.21.15.89:0/4024700615 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30c80a2b80 0x7f30c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.138+0000 7f30e4a00640 1 --2- 172.21.15.89:0/4024700615 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30c80a2b80 0x7f30c80a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f30d4009bd0 tx=0x7f30d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.142+0000 7f30dd600640 1 -- 172.21.15.89:0/4024700615 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30d809bee0 con 0x7f30e0109120 2024-09-10T08:36:29.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.352+0000 7f30e7400640 1 -- 172.21.15.89:0/4024700615 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30e010c360 con 0x7f30e0109120 2024-09-10T08:36:29.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.353+0000 7f30dd600640 1 -- 172.21.15.89:0/4024700615 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5563 (secure 0 0 0) 0x7f30d809b7e0 con 0x7f30e0109120 2024-09-10T08:36:29.354 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:36:29.354 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":7,"btime":"2024-09-10T08:36:27:591952+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-10T08:36:29.354 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 7 2024-09-10T08:36:29.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.355+0000 7f30e7400640 1 -- 172.21.15.89:0/4024700615 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30c80a2b80 msgr2=0x7f30c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:29.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.355+0000 7f30e7400640 1 --2- 172.21.15.89:0/4024700615 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30c80a2b80 0x7f30c80a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f30d4009bd0 tx=0x7f30d4005f90 comp rx=0 tx=0).stop 2024-09-10T08:36:29.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.355+0000 7f30e7400640 1 -- 172.21.15.89:0/4024700615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30e0109120 msgr2=0x7f30e019ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:29.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.355+0000 7f30e7400640 1 --2- 172.21.15.89:0/4024700615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30e0109120 0x7f30e019ef70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009fd0 tx=0x7f30d800c800 comp rx=0 tx=0).stop 2024-09-10T08:36:29.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.355+0000 7f30e7400640 1 -- 172.21.15.89:0/4024700615 shutdown_connections 2024-09-10T08:36:29.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.355+0000 7f30e7400640 1 --2- 172.21.15.89:0/4024700615 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30c80a2b80 0x7f30c80a5040 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:29.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.355+0000 7f30e7400640 1 --2- 172.21.15.89:0/4024700615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30e0109120 0x7f30e019ef70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:29.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.355+0000 7f30e7400640 1 --2- 172.21.15.89:0/4024700615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30e01049a0 0x7f30e019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:29.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.355+0000 7f30e7400640 1 --2- 172.21.15.89:0/4024700615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30e0103ff0 0x7f30e019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:29.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.355+0000 7f30e7400640 1 -- 172.21.15.89:0/4024700615 >> 172.21.15.89:0/4024700615 conn(0x7f30e00ffa90 msgr2=0x7f30e0100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:29.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.356+0000 7f30e7400640 1 -- 172.21.15.89:0/4024700615 shutdown_connections 2024-09-10T08:36:29.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:29.356+0000 7f30e7400640 1 -- 172.21.15.89:0/4024700615 wait complete. 2024-09-10T08:36:29.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:29 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/4024700615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:29.720 INFO:journalctl@ceph.mds.c.smithi149.stdout:Sep 10 08:36:29 smithi149 systemd[1]: Starting Ceph mds.c for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:36:30.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:29 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/4024700615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:30.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:29 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/4024700615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:30.124 INFO:journalctl@ceph.mds.c.smithi149.stdout:Sep 10 08:36:29 smithi149 podman[66420]: 2024-09-10 08:36:29.745331762 +0000 UTC m=+0.016228257 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:36:30.124 INFO:journalctl@ceph.mds.c.smithi149.stdout:Sep 10 08:36:29 smithi149 podman[66420]: 2024-09-10 08:36:29.863089471 +0000 UTC m=+0.133985959 container create a437630f0b5577af4a324f6933d761e7d69de6cdcc83606d4ec810340463c0b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-c, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:36:30.124 INFO:journalctl@ceph.mds.c.smithi149.stdout:Sep 10 08:36:30 smithi149 podman[66420]: 2024-09-10 08:36:30.124179132 +0000 UTC m=+0.395075620 container init a437630f0b5577af4a324f6933d761e7d69de6cdcc83606d4ec810340463c0b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T08:36:30.470 INFO:journalctl@ceph.mds.c.smithi149.stdout:Sep 10 08:36:30 smithi149 podman[66420]: 2024-09-10 08:36:30.126992769 +0000 UTC m=+0.397889258 container start a437630f0b5577af4a324f6933d761e7d69de6cdcc83606d4ec810340463c0b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-c, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:36:30.471 INFO:journalctl@ceph.mds.c.smithi149.stdout:Sep 10 08:36:30 smithi149 ceph-mds[66439]: -- 172.21.15.149:0/4098444581 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558dc770d860 con 0x558dc76eb000 2024-09-10T08:36:30.471 INFO:journalctl@ceph.mds.c.smithi149.stdout:Sep 10 08:36:30 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-c[66435]: starting mds.c at 2024-09-10T08:36:30.471 INFO:journalctl@ceph.mds.c.smithi149.stdout:Sep 10 08:36:30 smithi149 bash[66420]: a437630f0b5577af4a324f6933d761e7d69de6cdcc83606d4ec810340463c0b3 2024-09-10T08:36:30.471 INFO:journalctl@ceph.mds.c.smithi149.stdout:Sep 10 08:36:30 smithi149 systemd[1]: Started Ceph mds.c for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:36:30.857 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:30 smithi111 ceph-mon[35220]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:30.857 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:30.858 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:30.858 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:30.858 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:30.858 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:30.858 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:30 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:30.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:30 smithi149 ceph-mon[36167]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:30.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:30.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:30.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:30.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:30.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:30.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:30 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:30 smithi089 ceph-mon[30757]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:30 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:31.721 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:31 smithi089 ceph-mon[30757]: Deploying daemon mds.h on smithi111 2024-09-10T08:36:31.721 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:31 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.149:6836/2367754626,v1:172.21.15.149:6837/2367754626] up:boot 2024-09-10T08:36:31.721 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:31 smithi089 ceph-mon[30757]: fsmap 7 up:standby 2024-09-10T08:36:31.721 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:31 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T08:36:31.943 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:31 smithi111 ceph-mon[35220]: Deploying daemon mds.h on smithi111 2024-09-10T08:36:31.943 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:31 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.149:6836/2367754626,v1:172.21.15.149:6837/2367754626] up:boot 2024-09-10T08:36:31.943 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:31 smithi111 ceph-mon[35220]: fsmap 7 up:standby 2024-09-10T08:36:31.943 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:31 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T08:36:31.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:31 smithi149 ceph-mon[36167]: Deploying daemon mds.h on smithi111 2024-09-10T08:36:31.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:31 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.149:6836/2367754626,v1:172.21.15.149:6837/2367754626] up:boot 2024-09-10T08:36:31.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:31 smithi149 ceph-mon[36167]: fsmap 7 up:standby 2024-09-10T08:36:31.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:31 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T08:36:32.097 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:36:32.470 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:36:32.614 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:36:32 smithi111 podman[64744]: 2024-09-10 08:36:32.563501799 +0000 UTC m=+0.142611987 container create 34d380ce62f19dcd97633ac9482548a5946dfa4f9c1b6eef2dc97b1adfd3e8e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-10T08:36:32.834 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:32 smithi089 ceph-mon[30757]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:32.899 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:32 smithi111 ceph-mon[35220]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:32.899 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:36:32 smithi111 podman[64744]: 2024-09-10 08:36:32.807886498 +0000 UTC m=+0.386996644 container init 34d380ce62f19dcd97633ac9482548a5946dfa4f9c1b6eef2dc97b1adfd3e8e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:36:32.899 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:36:32 smithi111 podman[64744]: 2024-09-10 08:36:32.810646519 +0000 UTC m=+0.389756674 container start 34d380ce62f19dcd97633ac9482548a5946dfa4f9c1b6eef2dc97b1adfd3e8e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:36:32.899 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:36:32 smithi111 ceph-mds[64763]: -- 172.21.15.111:0/68782417 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c6afdbd860 con 0x55c6afd9b000 2024-09-10T08:36:32.899 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:36:32 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h[64759]: starting mds.h at 2024-09-10T08:36:32.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:32 smithi149 ceph-mon[36167]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:33.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.069+0000 7f0fdf600640 1 -- 172.21.15.89:0/2308702816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fd8101de0 msgr2=0x7f0fd81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:33.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.069+0000 7f0fdf600640 1 --2- 172.21.15.89:0/2308702816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fd8101de0 0x7f0fd81021c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009900 tx=0x7f0fcc02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:36:33.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.069+0000 7f0fdf600640 1 -- 172.21.15.89:0/2308702816 shutdown_connections 2024-09-10T08:36:33.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.069+0000 7f0fdf600640 1 --2- 172.21.15.89:0/2308702816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fd8106f10 0x7f0fd810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:33.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.069+0000 7f0fdf600640 1 --2- 172.21.15.89:0/2308702816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0fd8102790 0x7f0fd81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:33.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.069+0000 7f0fdf600640 1 --2- 172.21.15.89:0/2308702816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fd8101de0 0x7f0fd81021c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:33.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.069+0000 7f0fdf600640 1 -- 172.21.15.89:0/2308702816 >> 172.21.15.89:0/2308702816 conn(0x7f0fd80fd8c0 msgr2=0x7f0fd80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:33.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.069+0000 7f0fdf600640 1 -- 172.21.15.89:0/2308702816 shutdown_connections 2024-09-10T08:36:33.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.069+0000 7f0fdf600640 1 -- 172.21.15.89:0/2308702816 wait complete. 2024-09-10T08:36:33.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.070+0000 7f0fdf600640 1 Processor -- start 2024-09-10T08:36:33.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.070+0000 7f0fdf600640 1 -- start start 2024-09-10T08:36:33.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.070+0000 7f0fdf600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fd8101de0 0x7f0fd8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:33.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.070+0000 7f0fdf600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fd8102790 0x7f0fd81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:33.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.070+0000 7f0fdf600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0fd8106f10 0x7f0fd819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:33.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.070+0000 7f0fdf600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd8111f80 con 0x7f0fd8101de0 2024-09-10T08:36:33.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.070+0000 7f0fdf600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd8111e00 con 0x7f0fd8106f10 2024-09-10T08:36:33.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.070+0000 7f0fdf600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd8112100 con 0x7f0fd8102790 2024-09-10T08:36:33.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.071+0000 7f0fdcc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fd8101de0 0x7f0fd8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:33.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.071+0000 7f0fdcc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fd8101de0 0x7f0fd8198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36542/0 (socket says 172.21.15.89:36542) 2024-09-10T08:36:33.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.071+0000 7f0fd7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fd8102790 0x7f0fd81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:33.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.071+0000 7f0fdcc00640 1 -- 172.21.15.89:0/3393469323 learned_addr learned my addr 172.21.15.89:0/3393469323 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:36:33.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.071+0000 7f0fdd600640 1 --2- 172.21.15.89:0/3393469323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0fd8106f10 0x7f0fd819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:33.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.071+0000 7f0fdcc00640 1 -- 172.21.15.89:0/3393469323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fd8102790 msgr2=0x7f0fd81989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:33.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.071+0000 7f0fdcc00640 1 --2- 172.21.15.89:0/3393469323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fd8102790 0x7f0fd81989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:33.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.071+0000 7f0fdcc00640 1 -- 172.21.15.89:0/3393469323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0fd8106f10 msgr2=0x7f0fd819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:33.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.071+0000 7f0fdcc00640 1 --2- 172.21.15.89:0/3393469323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0fd8106f10 0x7f0fd819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:33.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.071+0000 7f0fdcc00640 1 -- 172.21.15.89:0/3393469323 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd819d410 con 0x7f0fd8101de0 2024-09-10T08:36:33.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.071+0000 7f0fdcc00640 1 --2- 172.21.15.89:0/3393469323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fd8101de0 0x7f0fd8198460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc004770 tx=0x7f0fcc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:33.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.072+0000 7f0fd5600640 1 -- 172.21.15.89:0/3393469323 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc03f070 con 0x7f0fd8101de0 2024-09-10T08:36:33.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.072+0000 7f0fd5600640 1 -- 172.21.15.89:0/3393469323 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fcc031ca0 con 0x7f0fd8101de0 2024-09-10T08:36:33.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.072+0000 7f0fd5600640 1 -- 172.21.15.89:0/3393469323 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc035030 con 0x7f0fd8101de0 2024-09-10T08:36:33.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.072+0000 7f0fdf600640 1 -- 172.21.15.89:0/3393469323 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd819d6a0 con 0x7f0fd8101de0 2024-09-10T08:36:33.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.072+0000 7f0fdf600640 1 -- 172.21.15.89:0/3393469323 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd819db30 con 0x7f0fd8101de0 2024-09-10T08:36:33.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.073+0000 7f0fdf600640 1 -- 172.21.15.89:0/3393469323 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd81021c0 con 0x7f0fd8101de0 2024-09-10T08:36:33.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.075+0000 7f0fd5600640 1 -- 172.21.15.89:0/3393469323 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0fcc042070 con 0x7f0fd8101de0 2024-09-10T08:36:33.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.076+0000 7f0fd5600640 1 --2- 172.21.15.89:0/3393469323 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0fc00a2b80 0x7f0fc00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:33.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.076+0000 7f0fd5600640 1 -- 172.21.15.89:0/3393469323 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f0fcc0ef2c0 con 0x7f0fd8101de0 2024-09-10T08:36:33.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.076+0000 7f0fd7e00640 1 --2- 172.21.15.89:0/3393469323 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0fc00a2b80 0x7f0fc00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:33.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.077+0000 7f0fd7e00640 1 --2- 172.21.15.89:0/3393469323 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0fc00a2b80 0x7f0fc00a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0fd80fdce0 tx=0x7f0fc4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:33.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.080+0000 7f0fd5600640 1 -- 172.21.15.89:0/3393469323 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0fcc0bbfd0 con 0x7f0fd8101de0 2024-09-10T08:36:33.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.287+0000 7f0fdf600640 1 -- 172.21.15.89:0/3393469323 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fd802d7c0 con 0x7f0fd8101de0 2024-09-10T08:36:33.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.287+0000 7f0fd5600640 1 -- 172.21.15.89:0/3393469323 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6391 (secure 0 0 0) 0x7f0fcc0c02d0 con 0x7f0fd8101de0 2024-09-10T08:36:33.288 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:36:33.289 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":8,"btime":"2024-09-10T08:36:30:601541+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-10T08:36:33.289 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 8 2024-09-10T08:36:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.290+0000 7f0fdf600640 1 -- 172.21.15.89:0/3393469323 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0fc00a2b80 msgr2=0x7f0fc00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.290+0000 7f0fdf600640 1 --2- 172.21.15.89:0/3393469323 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0fc00a2b80 0x7f0fc00a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0fd80fdce0 tx=0x7f0fc4005f90 comp rx=0 tx=0).stop 2024-09-10T08:36:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.290+0000 7f0fdf600640 1 -- 172.21.15.89:0/3393469323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fd8101de0 msgr2=0x7f0fd8198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.290+0000 7f0fdf600640 1 --2- 172.21.15.89:0/3393469323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fd8101de0 0x7f0fd8198460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc004770 tx=0x7f0fcc003c10 comp rx=0 tx=0).stop 2024-09-10T08:36:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.290+0000 7f0fdf600640 1 -- 172.21.15.89:0/3393469323 shutdown_connections 2024-09-10T08:36:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.290+0000 7f0fdf600640 1 --2- 172.21.15.89:0/3393469323 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0fc00a2b80 0x7f0fc00a5040 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.290+0000 7f0fdf600640 1 --2- 172.21.15.89:0/3393469323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0fd8106f10 0x7f0fd819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.290+0000 7f0fdf600640 1 --2- 172.21.15.89:0/3393469323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fd8102790 0x7f0fd81989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.290+0000 7f0fdf600640 1 --2- 172.21.15.89:0/3393469323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fd8101de0 0x7f0fd8198460 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.290+0000 7f0fdf600640 1 -- 172.21.15.89:0/3393469323 >> 172.21.15.89:0/3393469323 conn(0x7f0fd80fd8c0 msgr2=0x7f0fd80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.290+0000 7f0fdf600640 1 -- 172.21.15.89:0/3393469323 shutdown_connections 2024-09-10T08:36:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:33.290+0000 7f0fdf600640 1 -- 172.21.15.89:0/3393469323 wait complete. 2024-09-10T08:36:33.293 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:36:32 smithi111 bash[64744]: 34d380ce62f19dcd97633ac9482548a5946dfa4f9c1b6eef2dc97b1adfd3e8e6 2024-09-10T08:36:33.293 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:36:32 smithi111 systemd[1]: Started Ceph mds.h for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:36:34.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:33 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:34.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:33 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:34.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:33 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:34.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:33 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:34.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:33 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:34.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:33 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:34.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:33 smithi149 ceph-mon[36167]: Deploying daemon mds.i on smithi149 2024-09-10T08:36:34.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:33 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3393469323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:34.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:33 smithi149 ceph-mon[36167]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:34.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:33 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] up:boot 2024-09-10T08:36:34.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:33 smithi149 ceph-mon[36167]: fsmap 8 up:standby 2024-09-10T08:36:34.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:33 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T08:36:34.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:34.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:34.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:34.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:34.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:34.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:34.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:33 smithi111 ceph-mon[35220]: Deploying daemon mds.i on smithi149 2024-09-10T08:36:34.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:33 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3393469323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:34.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:33 smithi111 ceph-mon[35220]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:34.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:33 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] up:boot 2024-09-10T08:36:34.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:33 smithi111 ceph-mon[35220]: fsmap 8 up:standby 2024-09-10T08:36:34.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:33 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T08:36:34.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:33 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:34.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:33 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:34.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:33 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:34.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:33 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:34.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:33 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:34.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:33 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:34.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:33 smithi089 ceph-mon[30757]: Deploying daemon mds.i on smithi149 2024-09-10T08:36:34.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:33 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3393469323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:34.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:33 smithi089 ceph-mon[30757]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:34.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:33 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] up:boot 2024-09-10T08:36:34.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:33 smithi089 ceph-mon[30757]: fsmap 8 up:standby 2024-09-10T08:36:34.304 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:33 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T08:36:35.172 INFO:journalctl@ceph.mds.i.smithi149.stdout:Sep 10 08:36:34 smithi149 systemd[1]: Starting Ceph mds.i for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:36:35.437 INFO:journalctl@ceph.mds.i.smithi149.stdout:Sep 10 08:36:35 smithi149 podman[66723]: 2024-09-10 08:36:35.075005967 +0000 UTC m=+0.015798083 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:36:35.438 INFO:journalctl@ceph.mds.i.smithi149.stdout:Sep 10 08:36:35 smithi149 podman[66723]: 2024-09-10 08:36:35.185408448 +0000 UTC m=+0.126200557 container create f9e956b83d57675f6b352cab851977a2979ae18307985e21c5627d3a227d6890 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-10T08:36:35.438 INFO:journalctl@ceph.mds.i.smithi149.stdout:Sep 10 08:36:35 smithi149 podman[66723]: 2024-09-10 08:36:35.438057699 +0000 UTC m=+0.378849809 container init f9e956b83d57675f6b352cab851977a2979ae18307985e21c5627d3a227d6890 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-i, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T08:36:35.721 INFO:journalctl@ceph.mds.i.smithi149.stdout:Sep 10 08:36:35 smithi149 podman[66723]: 2024-09-10 08:36:35.441003851 +0000 UTC m=+0.381795953 container start f9e956b83d57675f6b352cab851977a2979ae18307985e21c5627d3a227d6890 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-i, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-10T08:36:35.721 INFO:journalctl@ceph.mds.i.smithi149.stdout:Sep 10 08:36:35 smithi149 ceph-mds[66742]: -- 172.21.15.149:0/1781805564 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aea8887860 con 0x55aea8865000 2024-09-10T08:36:35.721 INFO:journalctl@ceph.mds.i.smithi149.stdout:Sep 10 08:36:35 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-i[66738]: starting mds.i at 2024-09-10T08:36:35.721 INFO:journalctl@ceph.mds.i.smithi149.stdout:Sep 10 08:36:35 smithi149 bash[66723]: f9e956b83d57675f6b352cab851977a2979ae18307985e21c5627d3a227d6890 2024-09-10T08:36:35.722 INFO:journalctl@ceph.mds.i.smithi149.stdout:Sep 10 08:36:35 smithi149 systemd[1]: Started Ceph mds.i for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:36:36.035 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:36:36.609 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:36 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] up:boot 2024-09-10T08:36:36.609 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:36 smithi089 ceph-mon[30757]: fsmap 9 up:standby 2024-09-10T08:36:36.609 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T08:36:36.609 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:36 smithi089 ceph-mon[30757]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:36.609 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:36.609 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:36.610 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:36.610 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:36.610 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:36.610 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:36.610 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:36 smithi089 ceph-mon[30757]: Deploying daemon mds.g on smithi089 2024-09-10T08:36:36.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:36 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] up:boot 2024-09-10T08:36:36.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:36 smithi111 ceph-mon[35220]: fsmap 9 up:standby 2024-09-10T08:36:36.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T08:36:36.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:36 smithi111 ceph-mon[35220]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:36.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:36.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:36.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:36.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:36.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:36.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:36.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:36 smithi111 ceph-mon[35220]: Deploying daemon mds.g on smithi089 2024-09-10T08:36:36.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:36 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] up:boot 2024-09-10T08:36:36.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:36 smithi149 ceph-mon[36167]: fsmap 9 up:standby 2024-09-10T08:36:36.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T08:36:36.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:36 smithi149 ceph-mon[36167]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:36.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:36.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:36.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:36.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:36.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:36.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:36.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:36 smithi149 ceph-mon[36167]: Deploying daemon mds.g on smithi089 2024-09-10T08:36:37.353 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:36:38.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.051+0000 7f47bf200640 1 -- 172.21.15.89:0/3225674951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47b8103ff0 msgr2=0x7f47b81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:38.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.051+0000 7f47bf200640 1 --2- 172.21.15.89:0/3225674951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47b8103ff0 0x7f47b81043d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009900 tx=0x7f47a802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:36:38.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.051+0000 7f47bf200640 1 -- 172.21.15.89:0/3225674951 shutdown_connections 2024-09-10T08:36:38.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.051+0000 7f47bf200640 1 --2- 172.21.15.89:0/3225674951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47b8109120 0x7f47b8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:38.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.051+0000 7f47bf200640 1 --2- 172.21.15.89:0/3225674951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47b81049a0 0x7f47b81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:38.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.051+0000 7f47bf200640 1 --2- 172.21.15.89:0/3225674951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47b8103ff0 0x7f47b81043d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:38.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.051+0000 7f47bf200640 1 -- 172.21.15.89:0/3225674951 >> 172.21.15.89:0/3225674951 conn(0x7f47b80ffa90 msgr2=0x7f47b8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:38.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.051+0000 7f47bf200640 1 -- 172.21.15.89:0/3225674951 shutdown_connections 2024-09-10T08:36:38.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.051+0000 7f47bf200640 1 -- 172.21.15.89:0/3225674951 wait complete. 2024-09-10T08:36:38.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.052+0000 7f47bf200640 1 Processor -- start 2024-09-10T08:36:38.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.052+0000 7f47bf200640 1 -- start start 2024-09-10T08:36:38.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.053+0000 7f47bf200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47b8103ff0 0x7f47b819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:38.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.053+0000 7f47bf200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47b81049a0 0x7f47b819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:38.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.053+0000 7f47bf200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47b8109120 0x7f47b819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:38.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.053+0000 7f47bf200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f47b81141a0 con 0x7f47b8103ff0 2024-09-10T08:36:38.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.053+0000 7f47bf200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f47b8114020 con 0x7f47b8109120 2024-09-10T08:36:38.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.053+0000 7f47bf200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f47b8114320 con 0x7f47b81049a0 2024-09-10T08:36:38.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.053+0000 7f47b7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47b8103ff0 0x7f47b819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:38.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.053+0000 7f47b7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47b8103ff0 0x7f47b819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36554/0 (socket says 172.21.15.89:36554) 2024-09-10T08:36:38.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.053+0000 7f47b7e00640 1 -- 172.21.15.89:0/2516349236 learned_addr learned my addr 172.21.15.89:0/2516349236 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:36:38.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.053+0000 7f47b7400640 1 --2- 172.21.15.89:0/2516349236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47b81049a0 0x7f47b819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:38.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.053+0000 7f47bd200640 1 --2- 172.21.15.89:0/2516349236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47b8109120 0x7f47b819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:38.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.054+0000 7f47b7e00640 1 -- 172.21.15.89:0/2516349236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47b81049a0 msgr2=0x7f47b819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:38.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.054+0000 7f47b7e00640 1 --2- 172.21.15.89:0/2516349236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47b81049a0 0x7f47b819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:38.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.054+0000 7f47b7e00640 1 -- 172.21.15.89:0/2516349236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47b8109120 msgr2=0x7f47b819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:38.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.054+0000 7f47b7e00640 1 --2- 172.21.15.89:0/2516349236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47b8109120 0x7f47b819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:38.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.054+0000 7f47b7e00640 1 -- 172.21.15.89:0/2516349236 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b819f5d0 con 0x7f47b8103ff0 2024-09-10T08:36:38.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.054+0000 7f47b7e00640 1 --2- 172.21.15.89:0/2516349236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47b8103ff0 0x7f47b819a620 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f47a8004770 tx=0x7f47a8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:38.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.054+0000 7f47b4c00640 1 -- 172.21.15.89:0/2516349236 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a803f070 con 0x7f47b8103ff0 2024-09-10T08:36:38.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.054+0000 7f47b4c00640 1 -- 172.21.15.89:0/2516349236 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47a8031ca0 con 0x7f47b8103ff0 2024-09-10T08:36:38.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.054+0000 7f47b4c00640 1 -- 172.21.15.89:0/2516349236 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8035030 con 0x7f47b8103ff0 2024-09-10T08:36:38.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.054+0000 7f47bf200640 1 -- 172.21.15.89:0/2516349236 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b819f860 con 0x7f47b8103ff0 2024-09-10T08:36:38.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.055+0000 7f47bf200640 1 -- 172.21.15.89:0/2516349236 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b819fcf0 con 0x7f47b8103ff0 2024-09-10T08:36:38.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.056+0000 7f47bf200640 1 -- 172.21.15.89:0/2516349236 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47b81043d0 con 0x7f47b8103ff0 2024-09-10T08:36:38.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.058+0000 7f47b4c00640 1 -- 172.21.15.89:0/2516349236 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f47a8042070 con 0x7f47b8103ff0 2024-09-10T08:36:38.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.059+0000 7f47b4c00640 1 --2- 172.21.15.89:0/2516349236 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f479c0a2b80 0x7f479c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:38.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.059+0000 7f47b4c00640 1 -- 172.21.15.89:0/2516349236 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f47a80ef2c0 con 0x7f47b8103ff0 2024-09-10T08:36:38.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.059+0000 7f47b7400640 1 --2- 172.21.15.89:0/2516349236 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f479c0a2b80 0x7f479c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:38.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.060+0000 7f47b7400640 1 --2- 172.21.15.89:0/2516349236 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f479c0a2b80 0x7f479c0a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f47b80fff20 tx=0x7f47a400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:38.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.063+0000 7f47b4c00640 1 -- 172.21.15.89:0/2516349236 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f47a80bbfd0 con 0x7f47b8103ff0 2024-09-10T08:36:38.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.308+0000 7f47bf200640 1 -- 172.21.15.89:0/2516349236 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47b802d7c0 con 0x7f47b8103ff0 2024-09-10T08:36:38.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.309+0000 7f47b4c00640 1 -- 172.21.15.89:0/2516349236 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8046 (secure 0 0 0) 0x7f47a80c02d0 con 0x7f47b8103ff0 2024-09-10T08:36:38.310 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:36:38.310 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":10,"btime":"2024-09-10T08:36:35:520027+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-10T08:36:38.310 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 10 2024-09-10T08:36:38.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.311+0000 7f47bf200640 1 -- 172.21.15.89:0/2516349236 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f479c0a2b80 msgr2=0x7f479c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:38.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.311+0000 7f47bf200640 1 --2- 172.21.15.89:0/2516349236 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f479c0a2b80 0x7f479c0a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f47b80fff20 tx=0x7f47a400b410 comp rx=0 tx=0).stop 2024-09-10T08:36:38.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.311+0000 7f47bf200640 1 -- 172.21.15.89:0/2516349236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47b8103ff0 msgr2=0x7f47b819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:38.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.311+0000 7f47bf200640 1 --2- 172.21.15.89:0/2516349236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47b8103ff0 0x7f47b819a620 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f47a8004770 tx=0x7f47a8003c10 comp rx=0 tx=0).stop 2024-09-10T08:36:38.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.311+0000 7f47bf200640 1 -- 172.21.15.89:0/2516349236 shutdown_connections 2024-09-10T08:36:38.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.311+0000 7f47bf200640 1 --2- 172.21.15.89:0/2516349236 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f479c0a2b80 0x7f479c0a5040 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:38.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.311+0000 7f47bf200640 1 --2- 172.21.15.89:0/2516349236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47b8109120 0x7f47b819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:38.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.311+0000 7f47bf200640 1 --2- 172.21.15.89:0/2516349236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47b81049a0 0x7f47b819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:38.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.311+0000 7f47bf200640 1 --2- 172.21.15.89:0/2516349236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f47b8103ff0 0x7f47b819a620 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:38.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.311+0000 7f47bf200640 1 -- 172.21.15.89:0/2516349236 >> 172.21.15.89:0/2516349236 conn(0x7f47b80ffa90 msgr2=0x7f47b8100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:38.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.311+0000 7f47bf200640 1 -- 172.21.15.89:0/2516349236 shutdown_connections 2024-09-10T08:36:38.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:38.311+0000 7f47bf200640 1 -- 172.21.15.89:0/2516349236 wait complete. 2024-09-10T08:36:38.600 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 10 08:36:38 smithi089 podman[71292]: 2024-09-10 08:36:38.456697195 +0000 UTC m=+0.194967011 container create e499f39fca99fa88b5b91751749da0c787bf3067ffd04e65e5fe2dc7de93f4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-g, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:36:38.913 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:38 smithi089 ceph-mon[30757]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:38.914 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:38 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2516349236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:38.914 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 10 08:36:38 smithi089 podman[71292]: 2024-09-10 08:36:38.71850461 +0000 UTC m=+0.456774427 container init e499f39fca99fa88b5b91751749da0c787bf3067ffd04e65e5fe2dc7de93f4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T08:36:38.914 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 10 08:36:38 smithi089 podman[71292]: 2024-09-10 08:36:38.724904528 +0000 UTC m=+0.463174354 container start e499f39fca99fa88b5b91751749da0c787bf3067ffd04e65e5fe2dc7de93f4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-10T08:36:38.914 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 10 08:36:38 smithi089 ceph-mds[71323]: -- 172.21.15.89:0/3268817503 <== mon.0 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c43043bc20 con 0x55c430418800 2024-09-10T08:36:38.915 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 10 08:36:38 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-g[71319]: starting mds.g at 2024-09-10T08:36:38.915 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 10 08:36:38 smithi089 bash[71292]: e499f39fca99fa88b5b91751749da0c787bf3067ffd04e65e5fe2dc7de93f4fe 2024-09-10T08:36:38.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:38 smithi149 ceph-mon[36167]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:38.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:38 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2516349236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:38 smithi111 ceph-mon[35220]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:38 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2516349236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:39.303 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 10 08:36:38 smithi089 systemd[1]: Started Ceph mds.g for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:36:40.261 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:39 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:40.261 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:39 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:40.261 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:39 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:40.261 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:39 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:40.261 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:39 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:40.262 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:39 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:40.262 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:39 smithi089 ceph-mon[30757]: Deploying daemon mds.a on smithi089 2024-09-10T08:36:40.262 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:39 smithi089 ceph-mon[30757]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:40.262 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:39 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.89:6838/2573030321,v1:172.21.15.89:6839/2573030321] up:boot 2024-09-10T08:36:40.262 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:39 smithi089 ceph-mon[30757]: fsmap 10 up:standby 2024-09-10T08:36:40.262 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:39 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T08:36:40.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:39 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:40.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:39 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:40.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:39 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:40.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:39 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:40.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:39 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:40.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:39 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:40.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:39 smithi111 ceph-mon[35220]: Deploying daemon mds.a on smithi089 2024-09-10T08:36:40.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:39 smithi111 ceph-mon[35220]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:40.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:39 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.89:6838/2573030321,v1:172.21.15.89:6839/2573030321] up:boot 2024-09-10T08:36:40.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:39 smithi111 ceph-mon[35220]: fsmap 10 up:standby 2024-09-10T08:36:40.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:39 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T08:36:40.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:39 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:40.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:39 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:40.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:39 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:40.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:39 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:40.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:39 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:40.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:39 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:40.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:39 smithi149 ceph-mon[36167]: Deploying daemon mds.a on smithi089 2024-09-10T08:36:40.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:39 smithi149 ceph-mon[36167]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:40.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:39 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.89:6838/2573030321,v1:172.21.15.89:6839/2573030321] up:boot 2024-09-10T08:36:40.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:39 smithi149 ceph-mon[36167]: fsmap 10 up:standby 2024-09-10T08:36:40.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:39 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T08:36:41.726 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:36:42.356 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 10 08:36:42 smithi089 podman[71625]: 2024-09-10 08:36:42.120131845 +0000 UTC m=+0.152980988 container create d384f926c0a908dc9441960f2359da2bb80852212035a745de514459990de481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-10T08:36:42.357 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 10 08:36:42 smithi089 podman[71625]: 2024-09-10 08:36:42.356951934 +0000 UTC m=+0.389801077 container init d384f926c0a908dc9441960f2359da2bb80852212035a745de514459990de481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:36:42.539 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:36:42.618 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 10 08:36:42 smithi089 podman[71625]: 2024-09-10 08:36:42.363167466 +0000 UTC m=+0.396016610 container start d384f926c0a908dc9441960f2359da2bb80852212035a745de514459990de481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T08:36:42.618 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 10 08:36:42 smithi089 ceph-mds[71670]: -- 172.21.15.89:0/2364245486 <== mon.0 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b489511860 con 0x55b4894ef000 2024-09-10T08:36:42.618 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 10 08:36:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-a[71666]: starting mds.a at 2024-09-10T08:36:42.618 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 10 08:36:42 smithi089 bash[71625]: d384f926c0a908dc9441960f2359da2bb80852212035a745de514459990de481 2024-09-10T08:36:42.618 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 10 08:36:42 smithi089 systemd[1]: Started Ceph mds.a for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:36:42.877 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:42 smithi089 ceph-mon[30757]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:42.881 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:42.881 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:42.881 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:42.881 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:42.881 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:42.881 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:42.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:42 smithi149 ceph-mon[36167]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:42.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:42.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:42.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:42.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:42.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:42.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:42.992 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:42 smithi111 ceph-mon[35220]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:42.992 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:42.992 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:42.992 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:42.992 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:36:42.992 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:36:42.993 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.146+0000 7f8200a00640 1 -- 172.21.15.89:0/2506046950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81fc077040 msgr2=0x7f81fc112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.146+0000 7f8200a00640 1 --2- 172.21.15.89:0/2506046950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81fc077040 0x7f81fc112330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009900 tx=0x7f81e802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:36:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.146+0000 7f8200a00640 1 -- 172.21.15.89:0/2506046950 shutdown_connections 2024-09-10T08:36:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.146+0000 7f8200a00640 1 --2- 172.21.15.89:0/2506046950 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81fc112870 0x7f81fc114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.146+0000 7f8200a00640 1 --2- 172.21.15.89:0/2506046950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81fc077040 0x7f81fc112330 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.146+0000 7f8200a00640 1 --2- 172.21.15.89:0/2506046950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81fc076720 0x7f81fc076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.147+0000 7f8200a00640 1 -- 172.21.15.89:0/2506046950 >> 172.21.15.89:0/2506046950 conn(0x7f81fc0ff910 msgr2=0x7f81fc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.147+0000 7f8200a00640 1 -- 172.21.15.89:0/2506046950 shutdown_connections 2024-09-10T08:36:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.147+0000 7f8200a00640 1 -- 172.21.15.89:0/2506046950 wait complete. 2024-09-10T08:36:43.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.148+0000 7f8200a00640 1 Processor -- start 2024-09-10T08:36:43.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.148+0000 7f8200a00640 1 -- start start 2024-09-10T08:36:43.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.148+0000 7f8200a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81fc076720 0x7f81fc1a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:43.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.148+0000 7f8200a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81fc077040 0x7f81fc1a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:43.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.148+0000 7f8200a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81fc112870 0x7f81fc19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:43.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.148+0000 7f8200a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc118450 con 0x7f81fc076720 2024-09-10T08:36:43.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.148+0000 7f8200a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc1182d0 con 0x7f81fc112870 2024-09-10T08:36:43.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.148+0000 7f8200a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc1185d0 con 0x7f81fc077040 2024-09-10T08:36:43.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.148+0000 7f81fa800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81fc076720 0x7f81fc1a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:43.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.148+0000 7f81fa800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81fc076720 0x7f81fc1a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58714/0 (socket says 172.21.15.89:58714) 2024-09-10T08:36:43.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.148+0000 7f81fa800640 1 -- 172.21.15.89:0/1624679811 learned_addr learned my addr 172.21.15.89:0/1624679811 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:36:43.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.148+0000 7f81f9e00640 1 --2- 172.21.15.89:0/1624679811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81fc077040 0x7f81fc1a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:43.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.149+0000 7f81fb200640 1 --2- 172.21.15.89:0/1624679811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81fc112870 0x7f81fc19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:43.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.149+0000 7f81fa800640 1 -- 172.21.15.89:0/1624679811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81fc077040 msgr2=0x7f81fc1a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:43.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.149+0000 7f81fa800640 1 --2- 172.21.15.89:0/1624679811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81fc077040 0x7f81fc1a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:43.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.149+0000 7f81fa800640 1 -- 172.21.15.89:0/1624679811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81fc112870 msgr2=0x7f81fc19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:43.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.149+0000 7f81fa800640 1 --2- 172.21.15.89:0/1624679811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81fc112870 0x7f81fc19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:43.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.149+0000 7f81fa800640 1 -- 172.21.15.89:0/1624679811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc19ef20 con 0x7f81fc076720 2024-09-10T08:36:43.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.149+0000 7f81fa800640 1 --2- 172.21.15.89:0/1624679811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81fc076720 0x7f81fc1a4600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f81ec00e9f0 tx=0x7f81ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.149+0000 7f81f3400640 1 -- 172.21.15.89:0/1624679811 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec00ccc0 con 0x7f81fc076720 2024-09-10T08:36:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.150+0000 7f81f3400640 1 -- 172.21.15.89:0/1624679811 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81ec00ce60 con 0x7f81fc076720 2024-09-10T08:36:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.150+0000 7f81f3400640 1 -- 172.21.15.89:0/1624679811 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec010690 con 0x7f81fc076720 2024-09-10T08:36:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.150+0000 7f8200a00640 1 -- 172.21.15.89:0/1624679811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81fc19f210 con 0x7f81fc076720 2024-09-10T08:36:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.150+0000 7f8200a00640 1 -- 172.21.15.89:0/1624679811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81fc1ab460 con 0x7f81fc076720 2024-09-10T08:36:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.151+0000 7f8200a00640 1 -- 172.21.15.89:0/1624679811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81fc077e60 con 0x7f81fc076720 2024-09-10T08:36:43.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.153+0000 7f81f3400640 1 -- 172.21.15.89:0/1624679811 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f81ec0046c0 con 0x7f81fc076720 2024-09-10T08:36:43.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.154+0000 7f81f3400640 1 --2- 172.21.15.89:0/1624679811 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f81e00a2b80 0x7f81e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:43.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.154+0000 7f81f3400640 1 -- 172.21.15.89:0/1624679811 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f81ec0cf220 con 0x7f81fc076720 2024-09-10T08:36:43.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.155+0000 7f81f9e00640 1 --2- 172.21.15.89:0/1624679811 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f81e00a2b80 0x7f81e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:43.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.155+0000 7f81f9e00640 1 --2- 172.21.15.89:0/1624679811 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f81e00a2b80 0x7f81e00a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f81e8000c00 tx=0x7f81e8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:43.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.157+0000 7f81f3400640 1 -- 172.21.15.89:0/1624679811 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f81ec0541b0 con 0x7f81fc076720 2024-09-10T08:36:43.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.362+0000 7f8200a00640 1 -- 172.21.15.89:0/1624679811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81fc076b00 con 0x7f81fc076720 2024-09-10T08:36:43.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.363+0000 7f81f3400640 1 -- 172.21.15.89:0/1624679811 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9698 (secure 0 0 0) 0x7f81ec09c580 con 0x7f81fc076720 2024-09-10T08:36:43.364 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:36:43.364 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":12,"btime":"2024-09-10T08:36:42:610491+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-10T08:36:43.364 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 12 2024-09-10T08:36:43.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.365+0000 7f8200a00640 1 -- 172.21.15.89:0/1624679811 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f81e00a2b80 msgr2=0x7f81e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:43.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.365+0000 7f8200a00640 1 --2- 172.21.15.89:0/1624679811 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f81e00a2b80 0x7f81e00a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f81e8000c00 tx=0x7f81e8005b50 comp rx=0 tx=0).stop 2024-09-10T08:36:43.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.365+0000 7f8200a00640 1 -- 172.21.15.89:0/1624679811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81fc076720 msgr2=0x7f81fc1a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:43.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.365+0000 7f8200a00640 1 --2- 172.21.15.89:0/1624679811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81fc076720 0x7f81fc1a4600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f81ec00e9f0 tx=0x7f81ec00edf0 comp rx=0 tx=0).stop 2024-09-10T08:36:43.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.365+0000 7f8200a00640 1 -- 172.21.15.89:0/1624679811 shutdown_connections 2024-09-10T08:36:43.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.365+0000 7f8200a00640 1 --2- 172.21.15.89:0/1624679811 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f81e00a2b80 0x7f81e00a5040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:43.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.365+0000 7f8200a00640 1 --2- 172.21.15.89:0/1624679811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81fc112870 0x7f81fc19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:43.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.365+0000 7f8200a00640 1 --2- 172.21.15.89:0/1624679811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81fc077040 0x7f81fc1a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:43.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.365+0000 7f8200a00640 1 --2- 172.21.15.89:0/1624679811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81fc076720 0x7f81fc1a4600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:43.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.365+0000 7f8200a00640 1 -- 172.21.15.89:0/1624679811 >> 172.21.15.89:0/1624679811 conn(0x7f81fc0ff910 msgr2=0x7f81fc101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:43.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.366+0000 7f8200a00640 1 -- 172.21.15.89:0/1624679811 shutdown_connections 2024-09-10T08:36:43.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:43.366+0000 7f8200a00640 1 -- 172.21.15.89:0/1624679811 wait complete. 2024-09-10T08:36:43.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:43 smithi089 ceph-mon[30757]: Deploying daemon mds.b on smithi111 2024-09-10T08:36:43.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:43 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.89:6840/3524299195,v1:172.21.15.89:6841/3524299195] up:boot 2024-09-10T08:36:43.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:43 smithi089 ceph-mon[30757]: fsmap 11 up:standby 2024-09-10T08:36:43.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:43 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T08:36:43.783 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:43 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1624679811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:43.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:43 smithi149 ceph-mon[36167]: Deploying daemon mds.b on smithi111 2024-09-10T08:36:43.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:43 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.89:6840/3524299195,v1:172.21.15.89:6841/3524299195] up:boot 2024-09-10T08:36:43.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:43 smithi149 ceph-mon[36167]: fsmap 11 up:standby 2024-09-10T08:36:43.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:43 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T08:36:43.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:43 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1624679811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:44.019 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:43 smithi111 ceph-mon[35220]: Deploying daemon mds.b on smithi111 2024-09-10T08:36:44.019 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:43 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.89:6840/3524299195,v1:172.21.15.89:6841/3524299195] up:boot 2024-09-10T08:36:44.019 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:43 smithi111 ceph-mon[35220]: fsmap 11 up:standby 2024-09-10T08:36:44.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:43 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T08:36:44.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:43 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1624679811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:44.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:44 smithi111 ceph-mon[35220]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:44.794 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:36:44 smithi111 systemd[1]: Starting Ceph mds.b for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:36:44.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:44 smithi149 ceph-mon[36167]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:45.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:44 smithi089 ceph-mon[30757]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:45.105 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:36:44 smithi111 podman[65048]: 2024-09-10 08:36:44.740045124 +0000 UTC m=+0.015659218 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:36:45.105 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:36:44 smithi111 podman[65048]: 2024-09-10 08:36:44.877705066 +0000 UTC m=+0.153319148 container create d725d119605e2363ba9aa22261f90f90daa8b52afa2101b3f47d1f6213fe7e1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T08:36:45.105 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:36:45 smithi111 podman[65048]: 2024-09-10 08:36:45.10566942 +0000 UTC m=+0.381283505 container init d725d119605e2363ba9aa22261f90f90daa8b52afa2101b3f47d1f6213fe7e1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T08:36:45.543 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:36:45 smithi111 podman[65048]: 2024-09-10 08:36:45.108801446 +0000 UTC m=+0.384415541 container start d725d119605e2363ba9aa22261f90f90daa8b52afa2101b3f47d1f6213fe7e1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-b, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-10T08:36:45.543 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:36:45 smithi111 ceph-mds[65067]: -- 172.21.15.111:0/2092200876 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557fa98c9860 con 0x557fa98a7000 2024-09-10T08:36:45.543 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:36:45 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-b[65063]: starting mds.b at 2024-09-10T08:36:45.543 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:36:45 smithi111 bash[65048]: d725d119605e2363ba9aa22261f90f90daa8b52afa2101b3f47d1f6213fe7e1d 2024-09-10T08:36:45.543 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:36:45 smithi111 systemd[1]: Started Ceph mds.b for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:36:46.316 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:36:46.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:46.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:46.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:46.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:46.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:46.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:36:46.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:46 smithi149 ceph-mon[36167]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:46.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:46 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] up:boot 2024-09-10T08:36:46.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:46 smithi149 ceph-mon[36167]: fsmap 12 up:standby 2024-09-10T08:36:46.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:46 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T08:36:46.499 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:46.499 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:46.499 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:46.499 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:46.499 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:46.499 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:36:46.499 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:46 smithi111 ceph-mon[35220]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:46.500 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:46 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] up:boot 2024-09-10T08:36:46.500 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:46 smithi111 ceph-mon[35220]: fsmap 12 up:standby 2024-09-10T08:36:46.500 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:46 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T08:36:46.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:46.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:46.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:46.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:46.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:46.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:36:46.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:46 smithi089 ceph-mon[30757]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:46.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:46 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] up:boot 2024-09-10T08:36:46.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:46 smithi089 ceph-mon[30757]: fsmap 12 up:standby 2024-09-10T08:36:46.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:46 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T08:36:46.886 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:36:47.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.472+0000 7f4f95c00640 1 -- 172.21.15.89:0/3233878327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f90102b90 msgr2=0x7f4f90102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:47.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.472+0000 7f4f95c00640 1 --2- 172.21.15.89:0/3233878327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f90102b90 0x7f4f90102f70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009900 tx=0x7f4f8002a480 comp rx=0 tx=0).stop 2024-09-10T08:36:47.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.472+0000 7f4f95c00640 1 -- 172.21.15.89:0/3233878327 shutdown_connections 2024-09-10T08:36:47.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.472+0000 7f4f95c00640 1 --2- 172.21.15.89:0/3233878327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f9010bfc0 0x7f4f9010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:47.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.472+0000 7f4f95c00640 1 --2- 172.21.15.89:0/3233878327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f901034b0 0x7f4f9010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:47.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.472+0000 7f4f95c00640 1 --2- 172.21.15.89:0/3233878327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f90102b90 0x7f4f90102f70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:47.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.472+0000 7f4f95c00640 1 -- 172.21.15.89:0/3233878327 >> 172.21.15.89:0/3233878327 conn(0x7f4f900fc610 msgr2=0x7f4f900fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:47.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.472+0000 7f4f95c00640 1 -- 172.21.15.89:0/3233878327 shutdown_connections 2024-09-10T08:36:47.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.472+0000 7f4f95c00640 1 -- 172.21.15.89:0/3233878327 wait complete. 2024-09-10T08:36:47.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.473+0000 7f4f95c00640 1 Processor -- start 2024-09-10T08:36:47.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.473+0000 7f4f95c00640 1 -- start start 2024-09-10T08:36:47.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.474+0000 7f4f95c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f90102b90 0x7f4f9019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:47.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.474+0000 7f4f95c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f901034b0 0x7f4f9019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:47.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.474+0000 7f4f95c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f9010bfc0 0x7f4f9019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:47.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.474+0000 7f4f95c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4f90111dc0 con 0x7f4f901034b0 2024-09-10T08:36:47.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.474+0000 7f4f95c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4f90111c40 con 0x7f4f90102b90 2024-09-10T08:36:47.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.474+0000 7f4f95c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4f90111f40 con 0x7f4f9010bfc0 2024-09-10T08:36:47.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.474+0000 7f4f8fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f90102b90 0x7f4f9019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:47.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.474+0000 7f4f8fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f90102b90 0x7f4f9019a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:33138/0 (socket says 172.21.15.89:33138) 2024-09-10T08:36:47.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.474+0000 7f4f8fe00640 1 -- 172.21.15.89:0/3508763891 learned_addr learned my addr 172.21.15.89:0/3508763891 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:36:47.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.474+0000 7f4f8f400640 1 --2- 172.21.15.89:0/3508763891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f901034b0 0x7f4f9019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:47.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.474+0000 7f4f95200640 1 --2- 172.21.15.89:0/3508763891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f9010bfc0 0x7f4f9019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:47.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.474+0000 7f4f8fe00640 1 -- 172.21.15.89:0/3508763891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f9010bfc0 msgr2=0x7f4f9019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:47.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.474+0000 7f4f8fe00640 1 --2- 172.21.15.89:0/3508763891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f9010bfc0 0x7f4f9019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:47.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.474+0000 7f4f8fe00640 1 -- 172.21.15.89:0/3508763891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f901034b0 msgr2=0x7f4f9019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:47.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.474+0000 7f4f8fe00640 1 --2- 172.21.15.89:0/3508763891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f901034b0 0x7f4f9019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:47.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.474+0000 7f4f8fe00640 1 -- 172.21.15.89:0/3508763891 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9019f440 con 0x7f4f90102b90 2024-09-10T08:36:47.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.475+0000 7f4f8fe00640 1 --2- 172.21.15.89:0/3508763891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f90102b90 0x7f4f9019a490 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4f80006010 tx=0x7f4f80004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:47.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.475+0000 7f4f8cc00640 1 -- 172.21.15.89:0/3508763891 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8003f030 con 0x7f4f90102b90 2024-09-10T08:36:47.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.475+0000 7f4f8cc00640 1 -- 172.21.15.89:0/3508763891 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f80004440 con 0x7f4f90102b90 2024-09-10T08:36:47.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.475+0000 7f4f95c00640 1 -- 172.21.15.89:0/3508763891 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f9019f6d0 con 0x7f4f90102b90 2024-09-10T08:36:47.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.475+0000 7f4f8cc00640 1 -- 172.21.15.89:0/3508763891 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f80035070 con 0x7f4f90102b90 2024-09-10T08:36:47.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.475+0000 7f4f95c00640 1 -- 172.21.15.89:0/3508763891 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f901a7010 con 0x7f4f90102b90 2024-09-10T08:36:47.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.476+0000 7f4f95c00640 1 -- 172.21.15.89:0/3508763891 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f90102f70 con 0x7f4f90102b90 2024-09-10T08:36:47.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.478+0000 7f4f8cc00640 1 -- 172.21.15.89:0/3508763891 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4f80042030 con 0x7f4f90102b90 2024-09-10T08:36:47.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.479+0000 7f4f8cc00640 1 --2- 172.21.15.89:0/3508763891 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f4f740a2c00 0x7f4f740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:47.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.480+0000 7f4f8cc00640 1 -- 172.21.15.89:0/3508763891 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f4f800eefc0 con 0x7f4f90102b90 2024-09-10T08:36:47.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.480+0000 7f4f8f400640 1 --2- 172.21.15.89:0/3508763891 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f4f740a2c00 0x7f4f740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:47.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.480+0000 7f4f8f400640 1 --2- 172.21.15.89:0/3508763891 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f4f740a2c00 0x7f4f740a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0095f0 tx=0x7f4f7c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:47.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.484+0000 7f4f8cc00640 1 -- 172.21.15.89:0/3508763891 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f800bc320 con 0x7f4f90102b90 2024-09-10T08:36:47.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.711+0000 7f4f95c00640 1 -- 172.21.15.89:0/3508763891 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f901a71e0 con 0x7f4f90102b90 2024-09-10T08:36:47.712 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:47 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:47.712 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:47 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:47.712 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:47 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:47.712 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:47 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:47.712 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:47 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:47.714 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:36:47.714 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":13,"btime":"2024-09-10T08:36:45:625357+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-10T08:36:47.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.713+0000 7f4f8cc00640 1 -- 172.21.15.89:0/3508763891 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10527 (secure 0 0 0) 0x7f4f800bbc20 con 0x7f4f90102b90 2024-09-10T08:36:47.714 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 13 2024-09-10T08:36:47.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.716+0000 7f4f95c00640 1 -- 172.21.15.89:0/3508763891 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f4f740a2c00 msgr2=0x7f4f740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:47.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.716+0000 7f4f95c00640 1 --2- 172.21.15.89:0/3508763891 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f4f740a2c00 0x7f4f740a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0095f0 tx=0x7f4f7c005f90 comp rx=0 tx=0).stop 2024-09-10T08:36:47.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.716+0000 7f4f95c00640 1 -- 172.21.15.89:0/3508763891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f90102b90 msgr2=0x7f4f9019a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:47.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.716+0000 7f4f95c00640 1 --2- 172.21.15.89:0/3508763891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f90102b90 0x7f4f9019a490 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4f80006010 tx=0x7f4f80004040 comp rx=0 tx=0).stop 2024-09-10T08:36:47.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.716+0000 7f4f95c00640 1 -- 172.21.15.89:0/3508763891 shutdown_connections 2024-09-10T08:36:47.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.716+0000 7f4f95c00640 1 --2- 172.21.15.89:0/3508763891 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f4f740a2c00 0x7f4f740a50c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:47.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.716+0000 7f4f95c00640 1 --2- 172.21.15.89:0/3508763891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f9010bfc0 0x7f4f9019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:47.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.716+0000 7f4f95c00640 1 --2- 172.21.15.89:0/3508763891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f901034b0 0x7f4f9019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:47.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.716+0000 7f4f95c00640 1 --2- 172.21.15.89:0/3508763891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f90102b90 0x7f4f9019a490 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:47.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.716+0000 7f4f95c00640 1 -- 172.21.15.89:0/3508763891 >> 172.21.15.89:0/3508763891 conn(0x7f4f900fc610 msgr2=0x7f4f9010d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:47.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.716+0000 7f4f95c00640 1 -- 172.21.15.89:0/3508763891 shutdown_connections 2024-09-10T08:36:47.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:47.716+0000 7f4f95c00640 1 -- 172.21.15.89:0/3508763891 wait complete. 2024-09-10T08:36:47.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:47 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:47.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:47 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:47.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:47 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:47.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:47 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:47.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:47 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:47.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:47 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:47.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:47 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:47.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:47 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:47.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:47 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:47.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:47 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:48.454 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-10T08:36:48.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:48 smithi089 ceph-mon[30757]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:48.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:48 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3508763891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:48.844 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:36:48.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:48 smithi149 ceph-mon[36167]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:48.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:48 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3508763891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:49.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:48 smithi111 ceph-mon[35220]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:49.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:48 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3508763891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:36:49.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.414+0000 7f13d2c00640 1 -- 172.21.15.89:0/1299094372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13cc1047e0 msgr2=0x7f13cc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:49.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.414+0000 7f13d2c00640 1 --2- 172.21.15.89:0/1299094372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13cc1047e0 0x7f13cc108830 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f13b8009900 tx=0x7f13b802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:36:49.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.414+0000 7f13d2c00640 1 -- 172.21.15.89:0/1299094372 shutdown_connections 2024-09-10T08:36:49.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.414+0000 7f13d2c00640 1 --2- 172.21.15.89:0/1299094372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13cc108f60 0x7f13cc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:49.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.414+0000 7f13d2c00640 1 --2- 172.21.15.89:0/1299094372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13cc1047e0 0x7f13cc108830 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:49.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.414+0000 7f13d2c00640 1 --2- 172.21.15.89:0/1299094372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13cc103e30 0x7f13cc104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:49.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.414+0000 7f13d2c00640 1 -- 172.21.15.89:0/1299094372 >> 172.21.15.89:0/1299094372 conn(0x7f13cc0ff910 msgr2=0x7f13cc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:49.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.414+0000 7f13d2c00640 1 -- 172.21.15.89:0/1299094372 shutdown_connections 2024-09-10T08:36:49.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.414+0000 7f13d2c00640 1 -- 172.21.15.89:0/1299094372 wait complete. 2024-09-10T08:36:49.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.415+0000 7f13d2c00640 1 Processor -- start 2024-09-10T08:36:49.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.415+0000 7f13d2c00640 1 -- start start 2024-09-10T08:36:49.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.415+0000 7f13d2c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13cc103e30 0x7f13cc19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:49.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.415+0000 7f13d2c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13cc1047e0 0x7f13cc19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:49.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.415+0000 7f13d2c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13cc108f60 0x7f13cc19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:49.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.415+0000 7f13d2c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc113fd0 con 0x7f13cc1047e0 2024-09-10T08:36:49.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.415+0000 7f13d2c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc113e50 con 0x7f13cc108f60 2024-09-10T08:36:49.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.415+0000 7f13d2c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f13cc114150 con 0x7f13cc103e30 2024-09-10T08:36:49.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.415+0000 7f13d1800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13cc103e30 0x7f13cc19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:49.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.415+0000 7f13d1800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13cc103e30 0x7f13cc19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:55590/0 (socket says 172.21.15.89:55590) 2024-09-10T08:36:49.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.415+0000 7f13d1800640 1 -- 172.21.15.89:0/3527461816 learned_addr learned my addr 172.21.15.89:0/3527461816 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:36:49.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.415+0000 7f13d0e00640 1 --2- 172.21.15.89:0/3527461816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13cc1047e0 0x7f13cc19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:49.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.416+0000 7f13d2200640 1 --2- 172.21.15.89:0/3527461816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13cc108f60 0x7f13cc19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:49.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.416+0000 7f13d1800640 1 -- 172.21.15.89:0/3527461816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13cc108f60 msgr2=0x7f13cc19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:49.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.416+0000 7f13d1800640 1 --2- 172.21.15.89:0/3527461816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13cc108f60 0x7f13cc19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:49.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.416+0000 7f13d1800640 1 -- 172.21.15.89:0/3527461816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13cc1047e0 msgr2=0x7f13cc19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:49.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.416+0000 7f13d1800640 1 --2- 172.21.15.89:0/3527461816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13cc1047e0 0x7f13cc19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:49.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.416+0000 7f13d1800640 1 -- 172.21.15.89:0/3527461816 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13cc19f440 con 0x7f13cc103e30 2024-09-10T08:36:49.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.416+0000 7f13d0e00640 1 --2- 172.21.15.89:0/3527461816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13cc1047e0 0x7f13cc19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:36:49.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.416+0000 7f13d1800640 1 --2- 172.21.15.89:0/3527461816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13cc103e30 0x7f13cc19a490 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f13c000e9f0 tx=0x7f13c000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:49.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.417+0000 7f13ca000640 1 -- 172.21.15.89:0/3527461816 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c000cd80 con 0x7f13cc103e30 2024-09-10T08:36:49.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.417+0000 7f13ca000640 1 -- 172.21.15.89:0/3527461816 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c0010910 con 0x7f13cc103e30 2024-09-10T08:36:49.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.417+0000 7f13d2c00640 1 -- 172.21.15.89:0/3527461816 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13cc19f730 con 0x7f13cc103e30 2024-09-10T08:36:49.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.417+0000 7f13ca000640 1 -- 172.21.15.89:0/3527461816 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0018990 con 0x7f13cc103e30 2024-09-10T08:36:49.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.417+0000 7f13d2c00640 1 -- 172.21.15.89:0/3527461816 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13cc1a7010 con 0x7f13cc103e30 2024-09-10T08:36:49.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.418+0000 7f13d2c00640 1 -- 172.21.15.89:0/3527461816 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13cc105310 con 0x7f13cc103e30 2024-09-10T08:36:49.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.420+0000 7f13ca000640 1 -- 172.21.15.89:0/3527461816 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f13c0010ac0 con 0x7f13cc103e30 2024-09-10T08:36:49.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.421+0000 7f13ca000640 1 --2- 172.21.15.89:0/3527461816 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f13b40a2b80 0x7f13b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:49.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.421+0000 7f13ca000640 1 -- 172.21.15.89:0/3527461816 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f13c00cf390 con 0x7f13cc103e30 2024-09-10T08:36:49.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.421+0000 7f13d0e00640 1 --2- 172.21.15.89:0/3527461816 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f13b40a2b80 0x7f13b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:49.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.422+0000 7f13d0e00640 1 --2- 172.21.15.89:0/3527461816 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f13b40a2b80 0x7f13b40a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f13b8000c00 tx=0x7f13b8031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:49.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.423+0000 7f13ca000640 1 -- 172.21.15.89:0/3527461816 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f13c009c0a0 con 0x7f13cc103e30 2024-09-10T08:36:49.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.616+0000 7f13d2c00640 1 -- 172.21.15.89:0/3527461816 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f13cc10bbb0 con 0x7f13cc103e30 2024-09-10T08:36:49.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.617+0000 7f13ca000640 1 -- 172.21.15.89:0/3527461816 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) ==== 84+0+65 (secure 0 0 0) 0x7f13c00a03a0 con 0x7f13cc103e30 2024-09-10T08:36:49.617 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:36:49.617 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2024-09-10T08:36:49.617 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:36:49.617 INFO:teuthology.orchestra.run.smithi089.stdout: "poolnum": 1, 2024-09-10T08:36:49.617 INFO:teuthology.orchestra.run.smithi089.stdout: "poolname": ".mgr" 2024-09-10T08:36:49.617 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:36:49.617 INFO:teuthology.orchestra.run.smithi089.stdout:] 2024-09-10T08:36:49.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.619+0000 7f13d2c00640 1 -- 172.21.15.89:0/3527461816 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f13b40a2b80 msgr2=0x7f13b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.619+0000 7f13d2c00640 1 --2- 172.21.15.89:0/3527461816 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f13b40a2b80 0x7f13b40a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f13b8000c00 tx=0x7f13b8031510 comp rx=0 tx=0).stop 2024-09-10T08:36:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.620+0000 7f13d2c00640 1 -- 172.21.15.89:0/3527461816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13cc103e30 msgr2=0x7f13cc19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.620+0000 7f13d2c00640 1 --2- 172.21.15.89:0/3527461816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13cc103e30 0x7f13cc19a490 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f13c000e9f0 tx=0x7f13c000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:36:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.620+0000 7f13d2c00640 1 -- 172.21.15.89:0/3527461816 shutdown_connections 2024-09-10T08:36:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.620+0000 7f13d2c00640 1 --2- 172.21.15.89:0/3527461816 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f13b40a2b80 0x7f13b40a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.620+0000 7f13d2c00640 1 --2- 172.21.15.89:0/3527461816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13cc108f60 0x7f13cc19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.620+0000 7f13d2c00640 1 --2- 172.21.15.89:0/3527461816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13cc1047e0 0x7f13cc19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.620+0000 7f13d2c00640 1 --2- 172.21.15.89:0/3527461816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13cc103e30 0x7f13cc19a490 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.620+0000 7f13d2c00640 1 -- 172.21.15.89:0/3527461816 >> 172.21.15.89:0/3527461816 conn(0x7f13cc0ff910 msgr2=0x7f13cc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.620+0000 7f13d2c00640 1 -- 172.21.15.89:0/3527461816 shutdown_connections 2024-09-10T08:36:49.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:49.620+0000 7f13d2c00640 1 -- 172.21.15.89:0/3527461816 wait complete. 2024-09-10T08:36:49.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:49 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:49.793 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:49 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:49.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:49 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:49.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:49 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:36:49.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:49 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:49.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:49 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:36:49.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:49 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3527461816' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T08:36:49.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:49 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:49.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:49 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:49.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:49 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:49.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:49 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:36:49.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:49 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:49.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:49 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:36:49.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:49 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3527461816' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T08:36:49.981 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:49 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:49.981 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:49 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:49.981 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:49 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:49.981 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:49 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:36:49.981 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:49 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:49.981 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:49 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:36:49.981 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:49 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3527461816' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T08:36:50.352 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T08:36:50.352 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T08:36:50.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:50 smithi149 ceph-mon[36167]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:50.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:50.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:50.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:50.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:50 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:51.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:50 smithi111 ceph-mon[35220]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:51.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:51.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:51.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:51.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:50 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:51.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:50 smithi089 ceph-mon[30757]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:51.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:51.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:51.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:51.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:50 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:51.154 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:36:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.743+0000 7f673d600640 1 -- 172.21.15.89:0/3805388803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6738109140 msgr2=0x7f6738110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:51.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.743+0000 7f673d600640 1 --2- 172.21.15.89:0/3805388803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6738109140 0x7f6738110cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6730009ca0 tx=0x7f673002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:36:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.743+0000 7f673d600640 1 -- 172.21.15.89:0/3805388803 shutdown_connections 2024-09-10T08:36:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.743+0000 7f673d600640 1 --2- 172.21.15.89:0/3805388803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6738109140 0x7f6738110cc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.743+0000 7f673d600640 1 --2- 172.21.15.89:0/3805388803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67381049c0 0x7f6738108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.743+0000 7f673d600640 1 --2- 172.21.15.89:0/3805388803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6738104010 0x7f67381043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.743+0000 7f673d600640 1 -- 172.21.15.89:0/3805388803 >> 172.21.15.89:0/3805388803 conn(0x7f67380ffa50 msgr2=0x7f6738101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.743+0000 7f673d600640 1 -- 172.21.15.89:0/3805388803 shutdown_connections 2024-09-10T08:36:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.743+0000 7f673d600640 1 -- 172.21.15.89:0/3805388803 wait complete. 2024-09-10T08:36:51.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.744+0000 7f673d600640 1 Processor -- start 2024-09-10T08:36:51.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.744+0000 7f673d600640 1 -- start start 2024-09-10T08:36:51.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.745+0000 7f673d600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6738104010 0x7f673819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:51.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.745+0000 7f673d600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67381049c0 0x7f673819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:51.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.745+0000 7f673d600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6738109140 0x7f673819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:51.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.745+0000 7f673d600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6738114190 con 0x7f67381049c0 2024-09-10T08:36:51.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.745+0000 7f673d600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6738114010 con 0x7f6738104010 2024-09-10T08:36:51.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.745+0000 7f673d600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6738114310 con 0x7f6738109140 2024-09-10T08:36:51.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.745+0000 7f6736000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67381049c0 0x7f673819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:51.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.745+0000 7f6736a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6738104010 0x7f673819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:51.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.745+0000 7f6736000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67381049c0 0x7f673819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58778/0 (socket says 172.21.15.89:58778) 2024-09-10T08:36:51.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.745+0000 7f6736000640 1 -- 172.21.15.89:0/707028174 learned_addr learned my addr 172.21.15.89:0/707028174 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:36:51.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.745+0000 7f6737400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6738109140 0x7f673819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:51.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.745+0000 7f6736a00640 1 -- 172.21.15.89:0/707028174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6738109140 msgr2=0x7f673819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:51.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.745+0000 7f6736a00640 1 --2- 172.21.15.89:0/707028174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6738109140 0x7f673819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:51.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.745+0000 7f6736a00640 1 -- 172.21.15.89:0/707028174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67381049c0 msgr2=0x7f673819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:51.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.745+0000 7f6736a00640 1 --2- 172.21.15.89:0/707028174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67381049c0 0x7f673819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:51.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.745+0000 7f6736a00640 1 -- 172.21.15.89:0/707028174 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673819f5e0 con 0x7f6738104010 2024-09-10T08:36:51.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.746+0000 7f6737400640 1 --2- 172.21.15.89:0/707028174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6738109140 0x7f673819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:36:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.746+0000 7f6736a00640 1 --2- 172.21.15.89:0/707028174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6738104010 0x7f673819a630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f672800e9f0 tx=0x7f672800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.746+0000 7f672f400640 1 -- 172.21.15.89:0/707028174 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672800ccc0 con 0x7f6738104010 2024-09-10T08:36:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.746+0000 7f672f400640 1 -- 172.21.15.89:0/707028174 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6728005020 con 0x7f6738104010 2024-09-10T08:36:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.746+0000 7f673d600640 1 -- 172.21.15.89:0/707028174 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673819f8d0 con 0x7f6738104010 2024-09-10T08:36:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.746+0000 7f672f400640 1 -- 172.21.15.89:0/707028174 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67280049e0 con 0x7f6738104010 2024-09-10T08:36:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.746+0000 7f673d600640 1 -- 172.21.15.89:0/707028174 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67381a71b0 con 0x7f6738104010 2024-09-10T08:36:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.747+0000 7f673d600640 1 -- 172.21.15.89:0/707028174 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67381043f0 con 0x7f6738104010 2024-09-10T08:36:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.749+0000 7f672f400640 1 -- 172.21.15.89:0/707028174 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6728005560 con 0x7f6738104010 2024-09-10T08:36:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.751+0000 7f672f400640 1 --2- 172.21.15.89:0/707028174 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f671c0a2c00 0x7f671c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.751+0000 7f672f400640 1 -- 172.21.15.89:0/707028174 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f67280ceb10 con 0x7f6738104010 2024-09-10T08:36:51.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.751+0000 7f6736000640 1 --2- 172.21.15.89:0/707028174 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f671c0a2c00 0x7f671c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.752+0000 7f6736000640 1 --2- 172.21.15.89:0/707028174 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f671c0a2c00 0x7f671c0a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f673810ab10 tx=0x7f6724005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:51.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.755+0000 7f672f400640 1 -- 172.21.15.89:0/707028174 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f67280051d0 con 0x7f6738104010 2024-09-10T08:36:51.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:51.946+0000 7f673d600640 1 -- 172.21.15.89:0/707028174 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f673810c3e0 con 0x7f6738104010 2024-09-10T08:36:52.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:52.706+0000 7f672f400640 1 -- 172.21.15.89:0/707028174 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) ==== 138+0+0 (secure 0 0 0) 0x7f672809be70 con 0x7f6738104010 2024-09-10T08:36:52.707 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' created 2024-09-10T08:36:52.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:52.711+0000 7f673d600640 1 -- 172.21.15.89:0/707028174 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f671c0a2c00 msgr2=0x7f671c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:52.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:52.711+0000 7f673d600640 1 --2- 172.21.15.89:0/707028174 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f671c0a2c00 0x7f671c0a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f673810ab10 tx=0x7f6724005f90 comp rx=0 tx=0).stop 2024-09-10T08:36:52.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:52.711+0000 7f673d600640 1 -- 172.21.15.89:0/707028174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6738104010 msgr2=0x7f673819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:52.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:52.711+0000 7f673d600640 1 --2- 172.21.15.89:0/707028174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6738104010 0x7f673819a630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f672800e9f0 tx=0x7f672800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:36:52.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:52.711+0000 7f673d600640 1 -- 172.21.15.89:0/707028174 shutdown_connections 2024-09-10T08:36:52.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:52.711+0000 7f673d600640 1 --2- 172.21.15.89:0/707028174 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f671c0a2c00 0x7f671c0a50c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:52.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:52.711+0000 7f673d600640 1 --2- 172.21.15.89:0/707028174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6738109140 0x7f673819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:52.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:52.711+0000 7f673d600640 1 --2- 172.21.15.89:0/707028174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67381049c0 0x7f673819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:52.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:52.711+0000 7f673d600640 1 --2- 172.21.15.89:0/707028174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6738104010 0x7f673819a630 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:52.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:52.711+0000 7f673d600640 1 -- 172.21.15.89:0/707028174 >> 172.21.15.89:0/707028174 conn(0x7f67380ffa50 msgr2=0x7f6738100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:52.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:52.711+0000 7f673d600640 1 -- 172.21.15.89:0/707028174 shutdown_connections 2024-09-10T08:36:52.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:52.712+0000 7f673d600640 1 -- 172.21.15.89:0/707028174 wait complete. 2024-09-10T08:36:52.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:52 smithi149 ceph-mon[36167]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:52.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:52 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/707028174' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T08:36:52.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:52 smithi149 ceph-mon[36167]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T08:36:52.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:52 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:52.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:52 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:52.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:52 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:52.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:52 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:36:52.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:52 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:53.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:52 smithi111 ceph-mon[35220]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:53.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:52 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/707028174' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T08:36:53.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:52 smithi111 ceph-mon[35220]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T08:36:53.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:52 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:53.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:52 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:53.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:52 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:53.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:52 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:36:53.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:52 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:53.049 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:52 smithi089 ceph-mon[30757]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:53.049 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:52 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/707028174' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T08:36:53.049 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:52 smithi089 ceph-mon[30757]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T08:36:53.049 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:52 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:53.049 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:52 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:53.049 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:52 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:53.049 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:52 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:36:53.049 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:52 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:36:53.462 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-10T08:36:53.843 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:36:53.982 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:53 smithi089 ceph-mon[30757]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T08:36:53.983 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:53 smithi089 ceph-mon[30757]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T08:36:54.000 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:53 smithi149 ceph-mon[36167]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T08:36:54.000 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:53 smithi149 ceph-mon[36167]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T08:36:54.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:53 smithi111 ceph-mon[35220]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T08:36:54.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:53 smithi111 ceph-mon[35220]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T08:36:54.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.429+0000 7f8c0de00640 1 -- 172.21.15.89:0/2581036482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c08102790 msgr2=0x7f8c081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:54.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.429+0000 7f8c0de00640 1 --2- 172.21.15.89:0/2581036482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c08102790 0x7f8c081067e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009900 tx=0x7f8bf402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:36:54.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.429+0000 7f8c0de00640 1 -- 172.21.15.89:0/2581036482 shutdown_connections 2024-09-10T08:36:54.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.429+0000 7f8c0de00640 1 --2- 172.21.15.89:0/2581036482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c08106f10 0x7f8c0810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:54.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.429+0000 7f8c0de00640 1 --2- 172.21.15.89:0/2581036482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c08102790 0x7f8c081067e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:54.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.429+0000 7f8c0de00640 1 --2- 172.21.15.89:0/2581036482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c08101de0 0x7f8c081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:54.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.429+0000 7f8c0de00640 1 -- 172.21.15.89:0/2581036482 >> 172.21.15.89:0/2581036482 conn(0x7f8c080fd8c0 msgr2=0x7f8c080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:54.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.429+0000 7f8c0de00640 1 -- 172.21.15.89:0/2581036482 shutdown_connections 2024-09-10T08:36:54.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.429+0000 7f8c0de00640 1 -- 172.21.15.89:0/2581036482 wait complete. 2024-09-10T08:36:54.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.430+0000 7f8c0de00640 1 Processor -- start 2024-09-10T08:36:54.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.430+0000 7f8c0de00640 1 -- start start 2024-09-10T08:36:54.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.430+0000 7f8c0de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c08101de0 0x7f8c080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:54.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.430+0000 7f8c0de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c08102790 0x7f8c08074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:54.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.430+0000 7f8c0de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c08106f10 0x7f8c08074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:54.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.430+0000 7f8c0de00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8c08111fc0 con 0x7f8c08106f10 2024-09-10T08:36:54.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.430+0000 7f8c0de00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8c08111e40 con 0x7f8c08101de0 2024-09-10T08:36:54.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.430+0000 7f8c0de00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8c08112140 con 0x7f8c08102790 2024-09-10T08:36:54.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.431+0000 7f8c07400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c08106f10 0x7f8c08074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:54.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.431+0000 7f8c06a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c08101de0 0x7f8c080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:54.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.431+0000 7f8c06a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c08101de0 0x7f8c080744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:36610/0 (socket says 172.21.15.89:36610) 2024-09-10T08:36:54.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.431+0000 7f8c06a00640 1 -- 172.21.15.89:0/2541880763 learned_addr learned my addr 172.21.15.89:0/2541880763 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:36:54.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.431+0000 7f8c06000640 1 --2- 172.21.15.89:0/2541880763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c08102790 0x7f8c08074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:54.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.431+0000 7f8c07400640 1 -- 172.21.15.89:0/2541880763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c08102790 msgr2=0x7f8c08074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:54.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.431+0000 7f8c07400640 1 --2- 172.21.15.89:0/2541880763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c08102790 0x7f8c08074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:54.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.431+0000 7f8c07400640 1 -- 172.21.15.89:0/2541880763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c08101de0 msgr2=0x7f8c080744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:54.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.431+0000 7f8c07400640 1 --2- 172.21.15.89:0/2541880763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c08101de0 0x7f8c080744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:54.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.431+0000 7f8c07400640 1 -- 172.21.15.89:0/2541880763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c081aadf0 con 0x7f8c08106f10 2024-09-10T08:36:54.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.431+0000 7f8c06a00640 1 --2- 172.21.15.89:0/2541880763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c08101de0 0x7f8c080744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:36:54.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.431+0000 7f8c07400640 1 --2- 172.21.15.89:0/2541880763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c08106f10 0x7f8c08074f70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000e940 tx=0x7f8c0000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:54.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.432+0000 7f8c0ca00640 1 -- 172.21.15.89:0/2541880763 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00004d20 con 0x7f8c08106f10 2024-09-10T08:36:54.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.432+0000 7f8c0ca00640 1 -- 172.21.15.89:0/2541880763 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c00005020 con 0x7f8c08106f10 2024-09-10T08:36:54.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.432+0000 7f8c0de00640 1 -- 172.21.15.89:0/2541880763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c081aafc0 con 0x7f8c08106f10 2024-09-10T08:36:54.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.432+0000 7f8c0ca00640 1 -- 172.21.15.89:0/2541880763 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c000116b0 con 0x7f8c08106f10 2024-09-10T08:36:54.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.432+0000 7f8c0de00640 1 -- 172.21.15.89:0/2541880763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c081ab480 con 0x7f8c08106f10 2024-09-10T08:36:54.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.433+0000 7f8c0de00640 1 -- 172.21.15.89:0/2541880763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c081032c0 con 0x7f8c08106f10 2024-09-10T08:36:54.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.435+0000 7f8c0ca00640 1 -- 172.21.15.89:0/2541880763 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8c000054d0 con 0x7f8c08106f10 2024-09-10T08:36:54.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.436+0000 7f8c0ca00640 1 --2- 172.21.15.89:0/2541880763 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8bec0a2c00 0x7f8bec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:54.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.437+0000 7f8c0ca00640 1 -- 172.21.15.89:0/2541880763 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f8c000cf0b0 con 0x7f8c08106f10 2024-09-10T08:36:54.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.437+0000 7f8c06a00640 1 --2- 172.21.15.89:0/2541880763 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8bec0a2c00 0x7f8bec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:54.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.437+0000 7f8c06a00640 1 --2- 172.21.15.89:0/2541880763 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8bec0a2c00 0x7f8bec0a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8011550 tx=0x7f8bf8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:54.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.441+0000 7f8c0ca00640 1 -- 172.21.15.89:0/2541880763 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8c00016030 con 0x7f8c08106f10 2024-09-10T08:36:54.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:54.618+0000 7f8c0de00640 1 -- 172.21.15.89:0/2541880763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f8c0810a150 con 0x7f8c08106f10 2024-09-10T08:36:55.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:55.121+0000 7f8c0ca00640 1 -- 172.21.15.89:0/2541880763 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) ==== 171+0+0 (secure 0 0 0) 0x7f8c0009bc50 con 0x7f8c08106f10 2024-09-10T08:36:55.121 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' created 2024-09-10T08:36:55.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:55.124+0000 7f8c0de00640 1 -- 172.21.15.89:0/2541880763 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8bec0a2c00 msgr2=0x7f8bec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:55.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:55.124+0000 7f8c0de00640 1 --2- 172.21.15.89:0/2541880763 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8bec0a2c00 0x7f8bec0a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8011550 tx=0x7f8bf8005f90 comp rx=0 tx=0).stop 2024-09-10T08:36:55.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:55.124+0000 7f8c0de00640 1 -- 172.21.15.89:0/2541880763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c08106f10 msgr2=0x7f8c08074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:55.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:55.124+0000 7f8c0de00640 1 --2- 172.21.15.89:0/2541880763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c08106f10 0x7f8c08074f70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000e940 tx=0x7f8c0000ee00 comp rx=0 tx=0).stop 2024-09-10T08:36:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:55.124+0000 7f8c0de00640 1 -- 172.21.15.89:0/2541880763 shutdown_connections 2024-09-10T08:36:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:55.124+0000 7f8c0de00640 1 --2- 172.21.15.89:0/2541880763 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8bec0a2c00 0x7f8bec0a50c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:55.124+0000 7f8c0de00640 1 --2- 172.21.15.89:0/2541880763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c08106f10 0x7f8c08074f70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:55.124+0000 7f8c0de00640 1 --2- 172.21.15.89:0/2541880763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c08102790 0x7f8c08074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:55.124+0000 7f8c0de00640 1 --2- 172.21.15.89:0/2541880763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c08101de0 0x7f8c080744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:55.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:55.124+0000 7f8c0de00640 1 -- 172.21.15.89:0/2541880763 >> 172.21.15.89:0/2541880763 conn(0x7f8c080fd8c0 msgr2=0x7f8c080fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:55.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:55.125+0000 7f8c0de00640 1 -- 172.21.15.89:0/2541880763 shutdown_connections 2024-09-10T08:36:55.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:55.125+0000 7f8c0de00640 1 -- 172.21.15.89:0/2541880763 wait complete. 2024-09-10T08:36:55.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:55 smithi149 ceph-mon[36167]: pgmap v287: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:55.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:55 smithi149 ceph-mon[36167]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T08:36:55.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:55 smithi149 ceph-mon[36167]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T08:36:55.526 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:55 smithi089 ceph-mon[30757]: pgmap v287: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:55.526 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:55 smithi089 ceph-mon[30757]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T08:36:55.526 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:55 smithi089 ceph-mon[30757]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T08:36:55.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:55 smithi111 ceph-mon[35220]: pgmap v287: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:55.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:55 smithi111 ceph-mon[35220]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T08:36:55.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:55 smithi111 ceph-mon[35220]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T08:36:55.938 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T08:36:56.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:56 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2541880763' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T08:36:56.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:56 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2541880763' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T08:36:56.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:56 smithi089 ceph-mon[30757]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T08:36:56.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:56 smithi089 ceph-mon[30757]: pgmap v290: 97 pgs: 91 unknown, 6 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:56.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:56 smithi089 ceph-mon[30757]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T08:36:56.321 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:36:56.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:56 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2541880763' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T08:36:56.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:56 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2541880763' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T08:36:56.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:56 smithi149 ceph-mon[36167]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T08:36:56.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:56 smithi149 ceph-mon[36167]: pgmap v290: 97 pgs: 91 unknown, 6 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:56.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:56 smithi149 ceph-mon[36167]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T08:36:56.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:56 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2541880763' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T08:36:56.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:56 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2541880763' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T08:36:56.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:56 smithi111 ceph-mon[35220]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T08:36:56.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:56 smithi111 ceph-mon[35220]: pgmap v290: 97 pgs: 91 unknown, 6 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:56.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:56 smithi111 ceph-mon[35220]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T08:36:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.939+0000 7f7908800640 1 -- 172.21.15.89:0/3283159090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7900106f10 msgr2=0x7f790010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.939+0000 7f7908800640 1 --2- 172.21.15.89:0/3283159090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7900106f10 0x7f790010ea90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009ca0 tx=0x7f78fc02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:36:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.940+0000 7f7908800640 1 -- 172.21.15.89:0/3283159090 shutdown_connections 2024-09-10T08:36:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.940+0000 7f7908800640 1 --2- 172.21.15.89:0/3283159090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7900106f10 0x7f790010ea90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.940+0000 7f7908800640 1 --2- 172.21.15.89:0/3283159090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7900102790 0x7f79001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.940+0000 7f7908800640 1 --2- 172.21.15.89:0/3283159090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7900101de0 0x7f79001021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.940+0000 7f7908800640 1 -- 172.21.15.89:0/3283159090 >> 172.21.15.89:0/3283159090 conn(0x7f79000fd8c0 msgr2=0x7f79000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.940+0000 7f7908800640 1 -- 172.21.15.89:0/3283159090 shutdown_connections 2024-09-10T08:36:56.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.940+0000 7f7908800640 1 -- 172.21.15.89:0/3283159090 wait complete. 2024-09-10T08:36:56.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.941+0000 7f7908800640 1 Processor -- start 2024-09-10T08:36:56.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.941+0000 7f7908800640 1 -- start start 2024-09-10T08:36:56.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.941+0000 7f7908800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7900101de0 0x7f790019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:56.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.941+0000 7f7908800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7900102790 0x7f790019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:56.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.941+0000 7f7908800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7900106f10 0x7f790019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:56.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.941+0000 7f7908800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7900111fc0 con 0x7f7900102790 2024-09-10T08:36:56.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.941+0000 7f7908800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7900111e40 con 0x7f7900106f10 2024-09-10T08:36:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.941+0000 7f7908800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7900112140 con 0x7f7900101de0 2024-09-10T08:36:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.941+0000 7f7905400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7900102790 0x7f790019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.941+0000 7f7905e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7900101de0 0x7f790019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.941+0000 7f7905400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7900102790 0x7f790019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57850/0 (socket says 172.21.15.89:57850) 2024-09-10T08:36:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.941+0000 7f7905400640 1 -- 172.21.15.89:0/1991397973 learned_addr learned my addr 172.21.15.89:0/1991397973 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:36:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.942+0000 7f7906800640 1 --2- 172.21.15.89:0/1991397973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7900106f10 0x7f790019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.942+0000 7f7905e00640 1 -- 172.21.15.89:0/1991397973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7900106f10 msgr2=0x7f790019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.942+0000 7f7905e00640 1 --2- 172.21.15.89:0/1991397973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7900106f10 0x7f790019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.942+0000 7f7905e00640 1 -- 172.21.15.89:0/1991397973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7900102790 msgr2=0x7f790019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.942+0000 7f7905e00640 1 --2- 172.21.15.89:0/1991397973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7900102790 0x7f790019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.942+0000 7f7905e00640 1 -- 172.21.15.89:0/1991397973 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790019f610 con 0x7f7900101de0 2024-09-10T08:36:56.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.942+0000 7f7905e00640 1 --2- 172.21.15.89:0/1991397973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7900101de0 0x7f790019a660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f78f400e9f0 tx=0x7f78f400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.943+0000 7f78faa00640 1 -- 172.21.15.89:0/1991397973 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f400ccc0 con 0x7f7900101de0 2024-09-10T08:36:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.943+0000 7f7908800640 1 -- 172.21.15.89:0/1991397973 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790019f900 con 0x7f7900101de0 2024-09-10T08:36:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.943+0000 7f78faa00640 1 -- 172.21.15.89:0/1991397973 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78f4010910 con 0x7f7900101de0 2024-09-10T08:36:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.943+0000 7f78faa00640 1 -- 172.21.15.89:0/1991397973 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f4018960 con 0x7f7900101de0 2024-09-10T08:36:56.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.943+0000 7f7908800640 1 -- 172.21.15.89:0/1991397973 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79001a71e0 con 0x7f7900101de0 2024-09-10T08:36:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.944+0000 7f7908800640 1 -- 172.21.15.89:0/1991397973 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79001021c0 con 0x7f7900101de0 2024-09-10T08:36:56.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.946+0000 7f78faa00640 1 -- 172.21.15.89:0/1991397973 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f78f4010ac0 con 0x7f7900101de0 2024-09-10T08:36:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.947+0000 7f78faa00640 1 --2- 172.21.15.89:0/1991397973 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f78e80a2b80 0x7f78e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.947+0000 7f78faa00640 1 -- 172.21.15.89:0/1991397973 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f78f40cf640 con 0x7f7900101de0 2024-09-10T08:36:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.947+0000 7f7905400640 1 --2- 172.21.15.89:0/1991397973 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f78e80a2b80 0x7f78e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:56.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.948+0000 7f7905400640 1 --2- 172.21.15.89:0/1991397973 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f78e80a2b80 0x7f78e80a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f79000ff6a0 tx=0x7f78f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:56.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:56.949+0000 7f78faa00640 1 -- 172.21.15.89:0/1991397973 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f78f409c060 con 0x7f7900101de0 2024-09-10T08:36:57.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:57.160+0000 7f7908800640 1 -- 172.21.15.89:0/1991397973 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f79001a73e0 con 0x7f7900101de0 2024-09-10T08:36:58.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:58.149+0000 7f78faa00640 1 -- 172.21.15.89:0/1991397973 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f78f40a0360 con 0x7f7900101de0 2024-09-10T08:36:58.149 INFO:teuthology.orchestra.run.smithi089.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T08:36:58.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:58.153+0000 7f7908800640 1 -- 172.21.15.89:0/1991397973 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f78e80a2b80 msgr2=0x7f78e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:58.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:58.153+0000 7f7908800640 1 --2- 172.21.15.89:0/1991397973 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f78e80a2b80 0x7f78e80a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f79000ff6a0 tx=0x7f78f0005f90 comp rx=0 tx=0).stop 2024-09-10T08:36:58.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:58.153+0000 7f7908800640 1 -- 172.21.15.89:0/1991397973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7900101de0 msgr2=0x7f790019a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:58.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:58.153+0000 7f7908800640 1 --2- 172.21.15.89:0/1991397973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7900101de0 0x7f790019a660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f78f400e9f0 tx=0x7f78f400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:36:58.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:58.153+0000 7f7908800640 1 -- 172.21.15.89:0/1991397973 shutdown_connections 2024-09-10T08:36:58.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:58.153+0000 7f7908800640 1 --2- 172.21.15.89:0/1991397973 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f78e80a2b80 0x7f78e80a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:58.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:58.153+0000 7f7908800640 1 --2- 172.21.15.89:0/1991397973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7900106f10 0x7f790019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:58.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:58.153+0000 7f7908800640 1 --2- 172.21.15.89:0/1991397973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7900102790 0x7f790019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:58.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:58.153+0000 7f7908800640 1 --2- 172.21.15.89:0/1991397973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7900101de0 0x7f790019a660 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:58.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:58.153+0000 7f7908800640 1 -- 172.21.15.89:0/1991397973 >> 172.21.15.89:0/1991397973 conn(0x7f79000fd8c0 msgr2=0x7f79000fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:58.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:58.154+0000 7f7908800640 1 -- 172.21.15.89:0/1991397973 shutdown_connections 2024-09-10T08:36:58.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:58.154+0000 7f7908800640 1 -- 172.21.15.89:0/1991397973 wait complete. 2024-09-10T08:36:58.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:58 smithi149 ceph-mon[36167]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T08:36:58.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:58 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1991397973' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T08:36:58.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:58 smithi149 ceph-mon[36167]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T08:36:58.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:58 smithi149 ceph-mon[36167]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T08:36:58.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:58 smithi149 ceph-mon[36167]: pgmap v293: 97 pgs: 31 creating+peering, 12 unknown, 54 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:58.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:58 smithi111 ceph-mon[35220]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T08:36:58.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:58 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1991397973' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T08:36:58.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:58 smithi111 ceph-mon[35220]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T08:36:58.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:58 smithi111 ceph-mon[35220]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T08:36:58.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:58 smithi111 ceph-mon[35220]: pgmap v293: 97 pgs: 31 creating+peering, 12 unknown, 54 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:58.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:58 smithi089 ceph-mon[30757]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T08:36:58.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:58 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1991397973' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T08:36:58.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:58 smithi089 ceph-mon[30757]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T08:36:58.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:58 smithi089 ceph-mon[30757]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T08:36:58.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:58 smithi089 ceph-mon[30757]: pgmap v293: 97 pgs: 31 creating+peering, 12 unknown, 54 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:36:59.028 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-10T08:36:59.029 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-10T08:36:59.302 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:59 smithi089 ceph-mon[30757]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T08:36:59.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:59 smithi089 ceph-mon[30757]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T08:36:59.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:59 smithi089 ceph-mon[30757]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T08:36:59.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:36:59 smithi089 ceph-mon[30757]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T08:36:59.399 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:36:59.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:59 smithi149 ceph-mon[36167]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T08:36:59.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:59 smithi149 ceph-mon[36167]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T08:36:59.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:59 smithi149 ceph-mon[36167]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T08:36:59.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:36:59 smithi149 ceph-mon[36167]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T08:36:59.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:59 smithi111 ceph-mon[35220]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T08:36:59.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:59 smithi111 ceph-mon[35220]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T08:36:59.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:59 smithi111 ceph-mon[35220]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T08:36:59.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:36:59 smithi111 ceph-mon[35220]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T08:36:59.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.983+0000 7f2f8f800640 1 -- 172.21.15.89:0/1343133233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f88104010 msgr2=0x7f2f881043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:59.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.983+0000 7f2f8f800640 1 --2- 172.21.15.89:0/1343133233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f88104010 0x7f2f881043f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009900 tx=0x7f2f7c02a410 comp rx=0 tx=0).stop 2024-09-10T08:36:59.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.983+0000 7f2f8f800640 1 -- 172.21.15.89:0/1343133233 shutdown_connections 2024-09-10T08:36:59.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.983+0000 7f2f8f800640 1 --2- 172.21.15.89:0/1343133233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f88109140 0x7f2f88110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:59.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.983+0000 7f2f8f800640 1 --2- 172.21.15.89:0/1343133233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f881049c0 0x7f2f88108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:59.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.983+0000 7f2f8f800640 1 --2- 172.21.15.89:0/1343133233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f88104010 0x7f2f881043f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:59.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.983+0000 7f2f8f800640 1 -- 172.21.15.89:0/1343133233 >> 172.21.15.89:0/1343133233 conn(0x7f2f880ffa50 msgr2=0x7f2f88101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:59.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.983+0000 7f2f8f800640 1 -- 172.21.15.89:0/1343133233 shutdown_connections 2024-09-10T08:36:59.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.983+0000 7f2f8f800640 1 -- 172.21.15.89:0/1343133233 wait complete. 2024-09-10T08:36:59.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.984+0000 7f2f8f800640 1 Processor -- start 2024-09-10T08:36:59.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.984+0000 7f2f8f800640 1 -- start start 2024-09-10T08:36:59.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.985+0000 7f2f8f800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f88104010 0x7f2f88076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:59.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.985+0000 7f2f8f800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f881049c0 0x7f2f88076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:59.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.985+0000 7f2f8f800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f88109140 0x7f2f88077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:59.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.985+0000 7f2f8f800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2f88114190 con 0x7f2f88109140 2024-09-10T08:36:59.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.985+0000 7f2f8f800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2f88114010 con 0x7f2f881049c0 2024-09-10T08:36:59.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.985+0000 7f2f8f800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2f88114310 con 0x7f2f88104010 2024-09-10T08:36:59.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.985+0000 7f2f8ce00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f88104010 0x7f2f88076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:59.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.985+0000 7f2f8d800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f88109140 0x7f2f88077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:59.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.985+0000 7f2f8ce00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f88104010 0x7f2f88076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:53668/0 (socket says 172.21.15.89:53668) 2024-09-10T08:36:59.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.985+0000 7f2f8ce00640 1 -- 172.21.15.89:0/2053747973 learned_addr learned my addr 172.21.15.89:0/2053747973 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:36:59.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.985+0000 7f2f87e00640 1 --2- 172.21.15.89:0/2053747973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f881049c0 0x7f2f88076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:59.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.986+0000 7f2f8ce00640 1 -- 172.21.15.89:0/2053747973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f881049c0 msgr2=0x7f2f88076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:59.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.986+0000 7f2f8ce00640 1 --2- 172.21.15.89:0/2053747973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f881049c0 0x7f2f88076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:59.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.986+0000 7f2f8ce00640 1 -- 172.21.15.89:0/2053747973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f88109140 msgr2=0x7f2f88077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:59.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.986+0000 7f2f8ce00640 1 --2- 172.21.15.89:0/2053747973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f88109140 0x7f2f88077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:59.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.986+0000 7f2f8ce00640 1 -- 172.21.15.89:0/2053747973 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f881aaca0 con 0x7f2f88104010 2024-09-10T08:36:59.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.986+0000 7f2f87e00640 1 --2- 172.21.15.89:0/2053747973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f881049c0 0x7f2f88076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:36:59.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.986+0000 7f2f8ce00640 1 --2- 172.21.15.89:0/2053747973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f88104010 0x7f2f88076700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c02aa00 tx=0x7f2f7c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:59.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.986+0000 7f2f85600640 1 -- 172.21.15.89:0/2053747973 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c03f070 con 0x7f2f88104010 2024-09-10T08:36:59.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.986+0000 7f2f85600640 1 -- 172.21.15.89:0/2053747973 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f7c031c90 con 0x7f2f88104010 2024-09-10T08:36:59.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.987+0000 7f2f8f800640 1 -- 172.21.15.89:0/2053747973 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f881aaf30 con 0x7f2f88104010 2024-09-10T08:36:59.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.987+0000 7f2f85600640 1 -- 172.21.15.89:0/2053747973 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c035030 con 0x7f2f88104010 2024-09-10T08:36:59.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.987+0000 7f2f8f800640 1 -- 172.21.15.89:0/2053747973 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f881ab3e0 con 0x7f2f88104010 2024-09-10T08:36:59.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.988+0000 7f2f8f800640 1 -- 172.21.15.89:0/2053747973 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f881054f0 con 0x7f2f88104010 2024-09-10T08:36:59.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.990+0000 7f2f85600640 1 -- 172.21.15.89:0/2053747973 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2f7c042070 con 0x7f2f88104010 2024-09-10T08:36:59.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.991+0000 7f2f85600640 1 --2- 172.21.15.89:0/2053747973 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2f700a2c00 0x7f2f700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.991+0000 7f2f85600640 1 -- 172.21.15.89:0/2053747973 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f2f7c0efb50 con 0x7f2f88104010 2024-09-10T08:36:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.991+0000 7f2f87e00640 1 --2- 172.21.15.89:0/2053747973 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2f700a2c00 0x7f2f700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:59.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.992+0000 7f2f87e00640 1 --2- 172.21.15.89:0/2053747973 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2f700a2c00 0x7f2f700a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2f88077ec0 tx=0x7f2f7400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:59.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:36:59.993+0000 7f2f85600640 1 -- 172.21.15.89:0/2053747973 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f7c0bc510 con 0x7f2f88104010 2024-09-10T08:37:00.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:00.180+0000 7f2f8f800640 1 -- 172.21.15.89:0/2053747973 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f2f881043f0 con 0x7f2f88104010 2024-09-10T08:37:00.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:00 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] up:active 2024-09-10T08:37:00.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:00 smithi149 ceph-mon[36167]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:37:00.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:00 smithi149 ceph-mon[36167]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:37:00.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:00 smithi149 ceph-mon[36167]: pgmap v295: 97 pgs: 31 creating+peering, 12 unknown, 54 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:00.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:00 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] up:active 2024-09-10T08:37:00.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:00 smithi111 ceph-mon[35220]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:37:00.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:00 smithi111 ceph-mon[35220]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:37:00.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:00 smithi111 ceph-mon[35220]: pgmap v295: 97 pgs: 31 creating+peering, 12 unknown, 54 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:00.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:00 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] up:active 2024-09-10T08:37:00.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:00 smithi089 ceph-mon[30757]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:37:00.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:00 smithi089 ceph-mon[30757]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:37:00.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:00 smithi089 ceph-mon[30757]: pgmap v295: 97 pgs: 31 creating+peering, 12 unknown, 54 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:01.043 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:37:00 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-b[65063]: 2024-09-10T08:37:00.632+0000 7fb1456ce640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T08:37:01.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:01.164+0000 7f2f85600640 1 -- 172.21.15.89:0/2053747973 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v87) ==== 157+0+0 (secure 0 0 0) 0x7f2f7c0c0810 con 0x7f2f88104010 2024-09-10T08:37:01.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:01.166+0000 7f2f8f800640 1 -- 172.21.15.89:0/2053747973 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2f700a2c00 msgr2=0x7f2f700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:01.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:01.166+0000 7f2f8f800640 1 --2- 172.21.15.89:0/2053747973 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2f700a2c00 0x7f2f700a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2f88077ec0 tx=0x7f2f7400b410 comp rx=0 tx=0).stop 2024-09-10T08:37:01.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:01.166+0000 7f2f8f800640 1 -- 172.21.15.89:0/2053747973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f88104010 msgr2=0x7f2f88076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:01.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:01.166+0000 7f2f8f800640 1 --2- 172.21.15.89:0/2053747973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f88104010 0x7f2f88076700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c02aa00 tx=0x7f2f7c003c10 comp rx=0 tx=0).stop 2024-09-10T08:37:01.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:01.167+0000 7f2f8f800640 1 -- 172.21.15.89:0/2053747973 shutdown_connections 2024-09-10T08:37:01.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:01.167+0000 7f2f8f800640 1 --2- 172.21.15.89:0/2053747973 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2f700a2c00 0x7f2f700a50c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:01.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:01.167+0000 7f2f8f800640 1 --2- 172.21.15.89:0/2053747973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f88109140 0x7f2f88077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:01.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:01.167+0000 7f2f8f800640 1 --2- 172.21.15.89:0/2053747973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f881049c0 0x7f2f88076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:01.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:01.167+0000 7f2f8f800640 1 --2- 172.21.15.89:0/2053747973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f88104010 0x7f2f88076700 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:01.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:01.167+0000 7f2f8f800640 1 -- 172.21.15.89:0/2053747973 >> 172.21.15.89:0/2053747973 conn(0x7f2f880ffa50 msgr2=0x7f2f88100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:01.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:01.167+0000 7f2f8f800640 1 -- 172.21.15.89:0/2053747973 shutdown_connections 2024-09-10T08:37:01.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:01.167+0000 7f2f8f800640 1 -- 172.21.15.89:0/2053747973 wait complete. 2024-09-10T08:37:01.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:01 smithi149 ceph-mon[36167]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T08:37:01.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:01 smithi149 ceph-mon[36167]: Cluster is now healthy 2024-09-10T08:37:01.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:01 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2053747973' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T08:37:01.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:01 smithi149 ceph-mon[36167]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T08:37:01.489 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:01 smithi089 ceph-mon[30757]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T08:37:01.489 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:01 smithi089 ceph-mon[30757]: Cluster is now healthy 2024-09-10T08:37:01.490 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:01 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2053747973' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T08:37:01.490 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:01 smithi089 ceph-mon[30757]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T08:37:01.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:01 smithi111 ceph-mon[35220]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T08:37:01.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:01 smithi111 ceph-mon[35220]: Cluster is now healthy 2024-09-10T08:37:01.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:01 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2053747973' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T08:37:01.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:01 smithi111 ceph-mon[35220]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T08:37:02.050 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T08:37:02.422 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:02.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:02 smithi149 ceph-mon[36167]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T08:37:02.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:02 smithi149 ceph-mon[36167]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T08:37:02.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:02 smithi149 ceph-mon[36167]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-09-10T08:37:02.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:02 smithi111 ceph-mon[35220]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T08:37:02.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:02 smithi111 ceph-mon[35220]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T08:37:02.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:02 smithi111 ceph-mon[35220]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-09-10T08:37:02.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:02 smithi089 ceph-mon[30757]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T08:37:02.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:02 smithi089 ceph-mon[30757]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T08:37:02.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:02 smithi089 ceph-mon[30757]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-09-10T08:37:02.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.980+0000 7f4075000640 1 -- 172.21.15.89:0/4233014034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40701047e0 msgr2=0x7f4070108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:02.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.980+0000 7f4075000640 1 --2- 172.21.15.89:0/4233014034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40701047e0 0x7f4070108830 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4058009900 tx=0x7f405802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:37:02.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.980+0000 7f4075000640 1 -- 172.21.15.89:0/4233014034 shutdown_connections 2024-09-10T08:37:02.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.980+0000 7f4075000640 1 --2- 172.21.15.89:0/4233014034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4070108f60 0x7f4070110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:02.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.980+0000 7f4075000640 1 --2- 172.21.15.89:0/4233014034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40701047e0 0x7f4070108830 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:02.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.980+0000 7f4075000640 1 --2- 172.21.15.89:0/4233014034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4070103e30 0x7f4070104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:02.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.980+0000 7f4075000640 1 -- 172.21.15.89:0/4233014034 >> 172.21.15.89:0/4233014034 conn(0x7f40700ff8d0 msgr2=0x7f4070101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:02.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.980+0000 7f4075000640 1 -- 172.21.15.89:0/4233014034 shutdown_connections 2024-09-10T08:37:02.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.980+0000 7f4075000640 1 -- 172.21.15.89:0/4233014034 wait complete. 2024-09-10T08:37:02.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.981+0000 7f4075000640 1 Processor -- start 2024-09-10T08:37:02.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.982+0000 7f4075000640 1 -- start start 2024-09-10T08:37:02.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.982+0000 7f4075000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4070103e30 0x7f407019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:02.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.982+0000 7f4075000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40701047e0 0x7f407019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:02.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.982+0000 7f4075000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4070108f60 0x7f407019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:02.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.982+0000 7f4075000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4070114010 con 0x7f40701047e0 2024-09-10T08:37:02.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.982+0000 7f4075000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4070113e90 con 0x7f4070103e30 2024-09-10T08:37:02.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.982+0000 7f4075000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4070114190 con 0x7f4070108f60 2024-09-10T08:37:02.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.982+0000 7f4066a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40701047e0 0x7f407019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:02.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.982+0000 7f4066a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40701047e0 0x7f407019a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60850/0 (socket says 172.21.15.89:60850) 2024-09-10T08:37:02.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.982+0000 7f4066a00640 1 -- 172.21.15.89:0/2038828431 learned_addr learned my addr 172.21.15.89:0/2038828431 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:02.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.982+0000 7f406f400640 1 --2- 172.21.15.89:0/2038828431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4070103e30 0x7f407019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:02.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.982+0000 7f406fe00640 1 --2- 172.21.15.89:0/2038828431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4070108f60 0x7f407019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:02.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.983+0000 7f4066a00640 1 -- 172.21.15.89:0/2038828431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4070108f60 msgr2=0x7f407019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:02.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.983+0000 7f4066a00640 1 --2- 172.21.15.89:0/2038828431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4070108f60 0x7f407019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:02.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.983+0000 7f4066a00640 1 -- 172.21.15.89:0/2038828431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4070103e30 msgr2=0x7f407019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:02.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.983+0000 7f4066a00640 1 --2- 172.21.15.89:0/2038828431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4070103e30 0x7f407019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:02.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.983+0000 7f4066a00640 1 -- 172.21.15.89:0/2038828431 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407019f420 con 0x7f40701047e0 2024-09-10T08:37:02.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.983+0000 7f4066a00640 1 --2- 172.21.15.89:0/2038828431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40701047e0 0x7f407019a9b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4058004770 tx=0x7f4058003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:02.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.983+0000 7f406cc00640 1 -- 172.21.15.89:0/2038828431 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405803f070 con 0x7f40701047e0 2024-09-10T08:37:02.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.983+0000 7f406cc00640 1 -- 172.21.15.89:0/2038828431 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4058031ca0 con 0x7f40701047e0 2024-09-10T08:37:02.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.983+0000 7f4075000640 1 -- 172.21.15.89:0/2038828431 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f407019f6b0 con 0x7f40701047e0 2024-09-10T08:37:02.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.983+0000 7f406cc00640 1 -- 172.21.15.89:0/2038828431 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4058035030 con 0x7f40701047e0 2024-09-10T08:37:02.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.984+0000 7f4075000640 1 -- 172.21.15.89:0/2038828431 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40701a6ff0 con 0x7f40701047e0 2024-09-10T08:37:02.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.984+0000 7f4075000640 1 -- 172.21.15.89:0/2038828431 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4070105310 con 0x7f40701047e0 2024-09-10T08:37:02.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.986+0000 7f406cc00640 1 -- 172.21.15.89:0/2038828431 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4058042070 con 0x7f40701047e0 2024-09-10T08:37:02.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.989+0000 7f406cc00640 1 --2- 172.21.15.89:0/2038828431 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f40540a2c00 0x7f40540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:02.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.989+0000 7f406cc00640 1 -- 172.21.15.89:0/2038828431 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8277+0+0 (secure 0 0 0) 0x7f40580efcc0 con 0x7f40701047e0 2024-09-10T08:37:02.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.989+0000 7f406f400640 1 --2- 172.21.15.89:0/2038828431 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f40540a2c00 0x7f40540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:02.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.990+0000 7f406f400640 1 --2- 172.21.15.89:0/2038828431 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f40540a2c00 0x7f40540a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4060009bd0 tx=0x7f4060005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:02.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:02.992+0000 7f406cc00640 1 -- 172.21.15.89:0/2038828431 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40580bc540 con 0x7f40701047e0 2024-09-10T08:37:03.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:03.165+0000 7f4075000640 1 -- 172.21.15.89:0/2038828431 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f407019b7d0 con 0x7f40701047e0 2024-09-10T08:37:03.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:03 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2038828431' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T08:37:04.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:03 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2038828431' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T08:37:04.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:03 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2038828431' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T08:37:04.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:04.223+0000 7f406cc00640 1 -- 172.21.15.89:0/2038828431 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v89) ==== 228+0+0 (secure 0 0 0) 0x7f40580c0840 con 0x7f40701047e0 2024-09-10T08:37:04.224 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data_ec' created 2024-09-10T08:37:04.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:04.228+0000 7f4075000640 1 -- 172.21.15.89:0/2038828431 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f40540a2c00 msgr2=0x7f40540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:04.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:04.228+0000 7f4075000640 1 --2- 172.21.15.89:0/2038828431 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f40540a2c00 0x7f40540a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4060009bd0 tx=0x7f4060005f90 comp rx=0 tx=0).stop 2024-09-10T08:37:04.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:04.228+0000 7f4075000640 1 -- 172.21.15.89:0/2038828431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40701047e0 msgr2=0x7f407019a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:04.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:04.228+0000 7f4075000640 1 --2- 172.21.15.89:0/2038828431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40701047e0 0x7f407019a9b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4058004770 tx=0x7f4058003c10 comp rx=0 tx=0).stop 2024-09-10T08:37:04.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:04.228+0000 7f4075000640 1 -- 172.21.15.89:0/2038828431 shutdown_connections 2024-09-10T08:37:04.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:04.228+0000 7f4075000640 1 --2- 172.21.15.89:0/2038828431 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f40540a2c00 0x7f40540a50c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:04.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:04.228+0000 7f4075000640 1 --2- 172.21.15.89:0/2038828431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4070108f60 0x7f407019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:04.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:04.228+0000 7f4075000640 1 --2- 172.21.15.89:0/2038828431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40701047e0 0x7f407019a9b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:04.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:04.228+0000 7f4075000640 1 --2- 172.21.15.89:0/2038828431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4070103e30 0x7f407019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:04.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:04.228+0000 7f4075000640 1 -- 172.21.15.89:0/2038828431 >> 172.21.15.89:0/2038828431 conn(0x7f40700ff8d0 msgr2=0x7f4070100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:04.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:04.230+0000 7f4075000640 1 -- 172.21.15.89:0/2038828431 shutdown_connections 2024-09-10T08:37:04.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:04.230+0000 7f4075000640 1 -- 172.21.15.89:0/2038828431 wait complete. 2024-09-10T08:37:04.985 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:04 smithi089 ceph-mon[30757]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T08:37:04.985 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:04 smithi089 ceph-mon[30757]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T08:37:04.985 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:04 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2038828431' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T08:37:04.985 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:04 smithi089 ceph-mon[30757]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T08:37:05.022 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-10T08:37:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:04 smithi111 ceph-mon[35220]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T08:37:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:04 smithi111 ceph-mon[35220]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T08:37:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:04 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2038828431' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T08:37:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:04 smithi111 ceph-mon[35220]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T08:37:05.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:04 smithi149 ceph-mon[36167]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T08:37:05.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:04 smithi149 ceph-mon[36167]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T08:37:05.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:04 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2038828431' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T08:37:05.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:04 smithi149 ceph-mon[36167]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T08:37:05.415 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:05.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.986+0000 7f562d800640 1 -- 172.21.15.89:0/3620452891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5628074620 msgr2=0x7f5628074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:05.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.986+0000 7f562d800640 1 --2- 172.21.15.89:0/3620452891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5628074620 0x7f5628074a00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5618009900 tx=0x7f561802a480 comp rx=0 tx=0).stop 2024-09-10T08:37:05.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.986+0000 7f562d800640 1 -- 172.21.15.89:0/3620452891 shutdown_connections 2024-09-10T08:37:05.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.986+0000 7f562d800640 1 --2- 172.21.15.89:0/3620452891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f562810e3c0 0x7f5628110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:05.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.986+0000 7f562d800640 1 --2- 172.21.15.89:0/3620452891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5628074f40 0x7f562810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:05.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.986+0000 7f562d800640 1 --2- 172.21.15.89:0/3620452891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5628074620 0x7f5628074a00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:05.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.986+0000 7f562d800640 1 -- 172.21.15.89:0/3620452891 >> 172.21.15.89:0/3620452891 conn(0x7f56280fd8c0 msgr2=0x7f56280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:05.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.986+0000 7f562d800640 1 -- 172.21.15.89:0/3620452891 shutdown_connections 2024-09-10T08:37:05.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.986+0000 7f562d800640 1 -- 172.21.15.89:0/3620452891 wait complete. 2024-09-10T08:37:05.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.987+0000 7f562d800640 1 Processor -- start 2024-09-10T08:37:05.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.987+0000 7f562d800640 1 -- start start 2024-09-10T08:37:05.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.988+0000 7f562d800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5628074620 0x7f562819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:05.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.988+0000 7f562d800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5628074f40 0x7f562819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:05.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.988+0000 7f562d800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f562810e3c0 0x7f562819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:05.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.988+0000 7f562d800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5628114150 con 0x7f562810e3c0 2024-09-10T08:37:05.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.988+0000 7f562d800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5628113fd0 con 0x7f5628074620 2024-09-10T08:37:05.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.988+0000 7f562d800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f56281142d0 con 0x7f5628074f40 2024-09-10T08:37:05.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.988+0000 7f5626a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5628074620 0x7f562819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:05.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.988+0000 7f5626a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5628074620 0x7f562819a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:52090/0 (socket says 172.21.15.89:52090) 2024-09-10T08:37:05.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.988+0000 7f5626a00640 1 -- 172.21.15.89:0/1947246398 learned_addr learned my addr 172.21.15.89:0/1947246398 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:05.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.988+0000 7f5627400640 1 --2- 172.21.15.89:0/1947246398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f562810e3c0 0x7f562819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:05.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.988+0000 7f5626000640 1 --2- 172.21.15.89:0/1947246398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5628074f40 0x7f562819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:05.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.988+0000 7f5626a00640 1 -- 172.21.15.89:0/1947246398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5628074f40 msgr2=0x7f562819ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:05.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.988+0000 7f5626a00640 1 --2- 172.21.15.89:0/1947246398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5628074f40 0x7f562819ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:05.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.989+0000 7f5626a00640 1 -- 172.21.15.89:0/1947246398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f562810e3c0 msgr2=0x7f562819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:05.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.989+0000 7f5626a00640 1 --2- 172.21.15.89:0/1947246398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f562810e3c0 0x7f562819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:05.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.989+0000 7f5626a00640 1 -- 172.21.15.89:0/1947246398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562819f6c0 con 0x7f5628074620 2024-09-10T08:37:05.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.989+0000 7f5626000640 1 --2- 172.21.15.89:0/1947246398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5628074f40 0x7f562819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:37:05.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.989+0000 7f5626a00640 1 --2- 172.21.15.89:0/1947246398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5628074620 0x7f562819a710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5618006010 tx=0x7f56180042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:05.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.989+0000 7f561f400640 1 -- 172.21.15.89:0/1947246398 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561803f030 con 0x7f5628074620 2024-09-10T08:37:05.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.989+0000 7f562d800640 1 -- 172.21.15.89:0/1947246398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f562819f950 con 0x7f5628074620 2024-09-10T08:37:05.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.989+0000 7f561f400640 1 -- 172.21.15.89:0/1947246398 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5618003ce0 con 0x7f5628074620 2024-09-10T08:37:05.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.989+0000 7f561f400640 1 -- 172.21.15.89:0/1947246398 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5618035070 con 0x7f5628074620 2024-09-10T08:37:05.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.989+0000 7f562d800640 1 -- 172.21.15.89:0/1947246398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56281a7290 con 0x7f5628074620 2024-09-10T08:37:05.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.990+0000 7f562d800640 1 -- 172.21.15.89:0/1947246398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5628075cd0 con 0x7f5628074620 2024-09-10T08:37:05.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.992+0000 7f561f400640 1 -- 172.21.15.89:0/1947246398 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5618042030 con 0x7f5628074620 2024-09-10T08:37:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.994+0000 7f561f400640 1 --2- 172.21.15.89:0/1947246398 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f560c0a2b80 0x7f560c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.994+0000 7f561f400640 1 -- 172.21.15.89:0/1947246398 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8770+0+0 (secure 0 0 0) 0x7f56180efc20 con 0x7f5628074620 2024-09-10T08:37:05.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.994+0000 7f5626000640 1 --2- 172.21.15.89:0/1947246398 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f560c0a2b80 0x7f560c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:05.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.994+0000 7f5626000640 1 --2- 172.21.15.89:0/1947246398 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f560c0a2b80 0x7f560c0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f562810f8b0 tx=0x7f561400b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:05.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:05.996+0000 7f561f400640 1 -- 172.21.15.89:0/1947246398 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56180bc9c0 con 0x7f5628074620 2024-09-10T08:37:06.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:06.170+0000 7f562d800640 1 -- 172.21.15.89:0/1947246398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f5628109540 con 0x7f5628074620 2024-09-10T08:37:06.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:06.235+0000 7f561f400640 1 -- 172.21.15.89:0/1947246398 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v91) ==== 169+0+0 (secure 0 0 0) 0x7f56180bc2c0 con 0x7f5628074620 2024-09-10T08:37:06.236 INFO:teuthology.orchestra.run.smithi089.stderr:set pool 4 allow_ec_overwrites to true 2024-09-10T08:37:06.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:06.237+0000 7f562d800640 1 -- 172.21.15.89:0/1947246398 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f560c0a2b80 msgr2=0x7f560c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:06.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:06.237+0000 7f562d800640 1 --2- 172.21.15.89:0/1947246398 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f560c0a2b80 0x7f560c0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f562810f8b0 tx=0x7f561400b5f0 comp rx=0 tx=0).stop 2024-09-10T08:37:06.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:06.237+0000 7f562d800640 1 -- 172.21.15.89:0/1947246398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5628074620 msgr2=0x7f562819a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:06.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:06.237+0000 7f562d800640 1 --2- 172.21.15.89:0/1947246398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5628074620 0x7f562819a710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5618006010 tx=0x7f56180042f0 comp rx=0 tx=0).stop 2024-09-10T08:37:06.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:06.238+0000 7f562d800640 1 -- 172.21.15.89:0/1947246398 shutdown_connections 2024-09-10T08:37:06.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:06.238+0000 7f562d800640 1 --2- 172.21.15.89:0/1947246398 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f560c0a2b80 0x7f560c0a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:06.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:06.238+0000 7f562d800640 1 --2- 172.21.15.89:0/1947246398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f562810e3c0 0x7f562819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:06.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:06.238+0000 7f562d800640 1 --2- 172.21.15.89:0/1947246398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5628074f40 0x7f562819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:06.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:06.238+0000 7f562d800640 1 --2- 172.21.15.89:0/1947246398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5628074620 0x7f562819a710 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:06.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:06.238+0000 7f562d800640 1 -- 172.21.15.89:0/1947246398 >> 172.21.15.89:0/1947246398 conn(0x7f56280fd8c0 msgr2=0x7f56280fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:06.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:06.238+0000 7f562d800640 1 -- 172.21.15.89:0/1947246398 shutdown_connections 2024-09-10T08:37:06.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:06.238+0000 7f562d800640 1 -- 172.21.15.89:0/1947246398 wait complete. 2024-09-10T08:37:06.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:06 smithi111 ceph-mon[35220]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T08:37:06.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:06 smithi111 ceph-mon[35220]: pgmap v302: 129 pgs: 2 creating+peering, 30 unknown, 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:06.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:06 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1947246398' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T08:37:06.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:06 smithi111 ceph-mon[35220]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T08:37:06.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:06 smithi089 ceph-mon[30757]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T08:37:06.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:06 smithi089 ceph-mon[30757]: pgmap v302: 129 pgs: 2 creating+peering, 30 unknown, 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:06.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:06 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1947246398' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T08:37:06.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:06 smithi089 ceph-mon[30757]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T08:37:06.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:06 smithi149 ceph-mon[36167]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T08:37:06.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:06 smithi149 ceph-mon[36167]: pgmap v302: 129 pgs: 2 creating+peering, 30 unknown, 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:06.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:06 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1947246398' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T08:37:06.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:06 smithi149 ceph-mon[36167]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T08:37:07.015 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-10T08:37:07.382 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:07.520 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:07 smithi089 ceph-mon[30757]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T08:37:07.520 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:07 smithi089 ceph-mon[30757]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T08:37:07.520 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:07 smithi089 ceph-mon[30757]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T08:37:07.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:07 smithi111 ceph-mon[35220]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T08:37:07.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:07 smithi111 ceph-mon[35220]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T08:37:07.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:07 smithi111 ceph-mon[35220]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T08:37:07.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:07 smithi149 ceph-mon[36167]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T08:37:07.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:07 smithi149 ceph-mon[36167]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T08:37:07.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:07 smithi149 ceph-mon[36167]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T08:37:08.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.001+0000 7ff975000640 1 -- 172.21.15.89:0/774848745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff970112a30 msgr2=0x7ff970114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:08.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.001+0000 7ff975000640 1 --2- 172.21.15.89:0/774848745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff970112a30 0x7ff970114e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff968009ca0 tx=0x7ff96802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:37:08.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.001+0000 7ff975000640 1 -- 172.21.15.89:0/774848745 shutdown_connections 2024-09-10T08:37:08.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.001+0000 7ff975000640 1 --2- 172.21.15.89:0/774848745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff970112a30 0x7ff970114e20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:08.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.001+0000 7ff975000640 1 --2- 172.21.15.89:0/774848745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff970077040 0x7ff9701124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:08.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.001+0000 7ff975000640 1 --2- 172.21.15.89:0/774848745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff970076720 0x7ff970076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:08.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.001+0000 7ff975000640 1 -- 172.21.15.89:0/774848745 >> 172.21.15.89:0/774848745 conn(0x7ff9700ffa90 msgr2=0x7ff970101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:08.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.001+0000 7ff975000640 1 -- 172.21.15.89:0/774848745 shutdown_connections 2024-09-10T08:37:08.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.001+0000 7ff975000640 1 -- 172.21.15.89:0/774848745 wait complete. 2024-09-10T08:37:08.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.002+0000 7ff975000640 1 Processor -- start 2024-09-10T08:37:08.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.002+0000 7ff975000640 1 -- start start 2024-09-10T08:37:08.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.002+0000 7ff975000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff970076720 0x7ff9701a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:08.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.003+0000 7ff975000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff970077040 0x7ff9701a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:08.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.003+0000 7ff975000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff970112a30 0x7ff97019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:08.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.003+0000 7ff975000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff9701185c0 con 0x7ff970076720 2024-09-10T08:37:08.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.003+0000 7ff975000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff970118440 con 0x7ff970077040 2024-09-10T08:37:08.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.003+0000 7ff975000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff970118740 con 0x7ff970112a30 2024-09-10T08:37:08.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.003+0000 7ff96de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff970076720 0x7ff9701a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:08.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.003+0000 7ff96de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff970076720 0x7ff9701a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60890/0 (socket says 172.21.15.89:60890) 2024-09-10T08:37:08.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.003+0000 7ff96de00640 1 -- 172.21.15.89:0/2232669245 learned_addr learned my addr 172.21.15.89:0/2232669245 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:08.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.003+0000 7ff96d400640 1 --2- 172.21.15.89:0/2232669245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff970077040 0x7ff9701a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:08.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.003+0000 7ff96e800640 1 --2- 172.21.15.89:0/2232669245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff970112a30 0x7ff97019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:08.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.003+0000 7ff96de00640 1 -- 172.21.15.89:0/2232669245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff970112a30 msgr2=0x7ff97019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:08.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.003+0000 7ff96de00640 1 --2- 172.21.15.89:0/2232669245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff970112a30 0x7ff97019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:08.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.003+0000 7ff96de00640 1 -- 172.21.15.89:0/2232669245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff970077040 msgr2=0x7ff9701a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:08.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.003+0000 7ff96de00640 1 --2- 172.21.15.89:0/2232669245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff970077040 0x7ff9701a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:08.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.003+0000 7ff96de00640 1 -- 172.21.15.89:0/2232669245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97019efd0 con 0x7ff970076720 2024-09-10T08:37:08.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.004+0000 7ff96de00640 1 --2- 172.21.15.89:0/2232669245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff970076720 0x7ff9701a46b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff96000d950 tx=0x7ff96000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:08.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.004+0000 7ff966a00640 1 -- 172.21.15.89:0/2232669245 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960017070 con 0x7ff970076720 2024-09-10T08:37:08.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.004+0000 7ff966a00640 1 -- 172.21.15.89:0/2232669245 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff960009c30 con 0x7ff970076720 2024-09-10T08:37:08.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.004+0000 7ff966a00640 1 -- 172.21.15.89:0/2232669245 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96000f6b0 con 0x7ff970076720 2024-09-10T08:37:08.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.004+0000 7ff975000640 1 -- 172.21.15.89:0/2232669245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff97019f2c0 con 0x7ff970076720 2024-09-10T08:37:08.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.004+0000 7ff975000640 1 -- 172.21.15.89:0/2232669245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9701ab510 con 0x7ff970076720 2024-09-10T08:37:08.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.005+0000 7ff975000640 1 -- 172.21.15.89:0/2232669245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff970077e60 con 0x7ff970076720 2024-09-10T08:37:08.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.007+0000 7ff966a00640 1 -- 172.21.15.89:0/2232669245 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff9600109d0 con 0x7ff970076720 2024-09-10T08:37:08.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.009+0000 7ff966a00640 1 --2- 172.21.15.89:0/2232669245 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff9540a2c00 0x7ff9540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:08.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.009+0000 7ff966a00640 1 -- 172.21.15.89:0/2232669245 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8770+0+0 (secure 0 0 0) 0x7ff9600ced80 con 0x7ff970076720 2024-09-10T08:37:08.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.009+0000 7ff96d400640 1 --2- 172.21.15.89:0/2232669245 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff9540a2c00 0x7ff9540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:08.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.009+0000 7ff96d400640 1 --2- 172.21.15.89:0/2232669245 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff9540a2c00 0x7ff9540a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff970079220 tx=0x7ff95c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:08.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.012+0000 7ff966a00640 1 -- 172.21.15.89:0/2232669245 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff960009de0 con 0x7ff970076720 2024-09-10T08:37:08.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.223+0000 7ff975000640 1 -- 172.21.15.89:0/2232669245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7ff97019fd60 con 0x7ff970076720 2024-09-10T08:37:08.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.240+0000 7ff966a00640 1 -- 172.21.15.89:0/2232669245 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7ff97019fd60 con 0x7ff970076720 2024-09-10T08:37:08.241 INFO:teuthology.orchestra.run.smithi089.stderr:added data pool 4 to fsmap 2024-09-10T08:37:08.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.243+0000 7ff975000640 1 -- 172.21.15.89:0/2232669245 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff9540a2c00 msgr2=0x7ff9540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:08.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.243+0000 7ff975000640 1 --2- 172.21.15.89:0/2232669245 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff9540a2c00 0x7ff9540a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff970079220 tx=0x7ff95c00b410 comp rx=0 tx=0).stop 2024-09-10T08:37:08.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.243+0000 7ff975000640 1 -- 172.21.15.89:0/2232669245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff970076720 msgr2=0x7ff9701a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:08.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.243+0000 7ff975000640 1 --2- 172.21.15.89:0/2232669245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff970076720 0x7ff9701a46b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff96000d950 tx=0x7ff96000de10 comp rx=0 tx=0).stop 2024-09-10T08:37:08.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.243+0000 7ff975000640 1 -- 172.21.15.89:0/2232669245 shutdown_connections 2024-09-10T08:37:08.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.243+0000 7ff975000640 1 --2- 172.21.15.89:0/2232669245 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff9540a2c00 0x7ff9540a50c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:08.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.243+0000 7ff975000640 1 --2- 172.21.15.89:0/2232669245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff970112a30 0x7ff97019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:08.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.243+0000 7ff975000640 1 --2- 172.21.15.89:0/2232669245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff970077040 0x7ff9701a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:08.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.243+0000 7ff975000640 1 --2- 172.21.15.89:0/2232669245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff970076720 0x7ff9701a46b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:08.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.243+0000 7ff975000640 1 -- 172.21.15.89:0/2232669245 >> 172.21.15.89:0/2232669245 conn(0x7ff9700ffa90 msgr2=0x7ff970101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:08.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.243+0000 7ff975000640 1 -- 172.21.15.89:0/2232669245 shutdown_connections 2024-09-10T08:37:08.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:08.243+0000 7ff975000640 1 -- 172.21.15.89:0/2232669245 wait complete. 2024-09-10T08:37:08.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:08 smithi111 ceph-mon[35220]: pgmap v304: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:08.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:08 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2232669245' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T08:37:08.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:08 smithi089 ceph-mon[30757]: pgmap v304: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:08.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:08 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2232669245' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T08:37:08.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:08 smithi149 ceph-mon[36167]: pgmap v304: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:08.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:08 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2232669245' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T08:37:09.100 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:37:09.469 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:09.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:09 smithi111 ceph-mon[35220]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T08:37:09.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:09 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2232669245' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T08:37:09.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:09 smithi111 ceph-mon[35220]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:37:09.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:09 smithi089 ceph-mon[30757]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T08:37:09.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:09 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2232669245' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T08:37:09.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:09 smithi089 ceph-mon[30757]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:37:09.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:09 smithi149 ceph-mon[36167]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T08:37:09.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:09 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2232669245' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T08:37:09.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:09 smithi149 ceph-mon[36167]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:37:10.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.042+0000 7f3102400640 1 -- 172.21.15.89:0/4266252007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30fc1047e0 msgr2=0x7f30fc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:10.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.042+0000 7f3102400640 1 --2- 172.21.15.89:0/4266252007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30fc1047e0 0x7f30fc108830 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f30e8009860 tx=0x7f30e802a340 comp rx=0 tx=0).stop 2024-09-10T08:37:10.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.042+0000 7f3102400640 1 -- 172.21.15.89:0/4266252007 shutdown_connections 2024-09-10T08:37:10.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.042+0000 7f3102400640 1 --2- 172.21.15.89:0/4266252007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30fc108f60 0x7f30fc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:10.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.042+0000 7f3102400640 1 --2- 172.21.15.89:0/4266252007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30fc1047e0 0x7f30fc108830 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:10.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.042+0000 7f3102400640 1 --2- 172.21.15.89:0/4266252007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30fc103e30 0x7f30fc104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:10.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.042+0000 7f3102400640 1 -- 172.21.15.89:0/4266252007 >> 172.21.15.89:0/4266252007 conn(0x7f30fc0ff8d0 msgr2=0x7f30fc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:10.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.042+0000 7f3102400640 1 -- 172.21.15.89:0/4266252007 shutdown_connections 2024-09-10T08:37:10.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.042+0000 7f3102400640 1 -- 172.21.15.89:0/4266252007 wait complete. 2024-09-10T08:37:10.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.043+0000 7f3102400640 1 Processor -- start 2024-09-10T08:37:10.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.043+0000 7f3102400640 1 -- start start 2024-09-10T08:37:10.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.043+0000 7f3102400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30fc103e30 0x7f30fc19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:10.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.043+0000 7f3102400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30fc1047e0 0x7f30fc19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:10.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.044+0000 7f3102400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30fc108f60 0x7f30fc19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:10.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.044+0000 7f3102400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc113fc0 con 0x7f30fc1047e0 2024-09-10T08:37:10.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.044+0000 7f3102400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc113e40 con 0x7f30fc103e30 2024-09-10T08:37:10.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.044+0000 7f3102400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc114140 con 0x7f30fc108f60 2024-09-10T08:37:10.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.044+0000 7f30fbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30fc1047e0 0x7f30fc19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:10.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.044+0000 7f30fbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30fc1047e0 0x7f30fc19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60914/0 (socket says 172.21.15.89:60914) 2024-09-10T08:37:10.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.044+0000 7f30fbe00640 1 -- 172.21.15.89:0/482661983 learned_addr learned my addr 172.21.15.89:0/482661983 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:10.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.044+0000 7f3101a00640 1 --2- 172.21.15.89:0/482661983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30fc108f60 0x7f30fc19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:10.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.044+0000 7f3101000640 1 --2- 172.21.15.89:0/482661983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30fc103e30 0x7f30fc19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:10.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.044+0000 7f30fbe00640 1 -- 172.21.15.89:0/482661983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30fc108f60 msgr2=0x7f30fc19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:10.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.044+0000 7f30fbe00640 1 --2- 172.21.15.89:0/482661983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30fc108f60 0x7f30fc19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:10.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.044+0000 7f30fbe00640 1 -- 172.21.15.89:0/482661983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30fc103e30 msgr2=0x7f30fc19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:10.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.044+0000 7f30fbe00640 1 --2- 172.21.15.89:0/482661983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30fc103e30 0x7f30fc19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:10.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.044+0000 7f30fbe00640 1 -- 172.21.15.89:0/482661983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30fc19f450 con 0x7f30fc1047e0 2024-09-10T08:37:10.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.045+0000 7f30fbe00640 1 --2- 172.21.15.89:0/482661983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30fc1047e0 0x7f30fc19a9e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f30e8035040 tx=0x7f30e8033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:10.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.045+0000 7f30f9600640 1 -- 172.21.15.89:0/482661983 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e803f020 con 0x7f30fc1047e0 2024-09-10T08:37:10.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.045+0000 7f30f9600640 1 -- 172.21.15.89:0/482661983 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30e8032770 con 0x7f30fc1047e0 2024-09-10T08:37:10.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.045+0000 7f3102400640 1 -- 172.21.15.89:0/482661983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30fc19f6e0 con 0x7f30fc1047e0 2024-09-10T08:37:10.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.045+0000 7f30f9600640 1 -- 172.21.15.89:0/482661983 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8037040 con 0x7f30fc1047e0 2024-09-10T08:37:10.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.045+0000 7f3102400640 1 -- 172.21.15.89:0/482661983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30fc1a7020 con 0x7f30fc1047e0 2024-09-10T08:37:10.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.046+0000 7f3102400640 1 -- 172.21.15.89:0/482661983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30fc104210 con 0x7f30fc1047e0 2024-09-10T08:37:10.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.048+0000 7f30f9600640 1 -- 172.21.15.89:0/482661983 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f30e8042020 con 0x7f30fc1047e0 2024-09-10T08:37:10.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.050+0000 7f30f9600640 1 --2- 172.21.15.89:0/482661983 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30e40a2b80 0x7f30e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:10.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.050+0000 7f30f9600640 1 -- 172.21.15.89:0/482661983 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f30e80ef840 con 0x7f30fc1047e0 2024-09-10T08:37:10.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.050+0000 7f3101000640 1 --2- 172.21.15.89:0/482661983 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30e40a2b80 0x7f30e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:10.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.051+0000 7f3101000640 1 --2- 172.21.15.89:0/482661983 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30e40a2b80 0x7f30e40a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009bd0 tx=0x7f30f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:10.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.051+0000 7f30f9600640 1 -- 172.21.15.89:0/482661983 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30e8049020 con 0x7f30fc1047e0 2024-09-10T08:37:10.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.263+0000 7f3102400640 1 -- 172.21.15.89:0/482661983 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30fc19b800 con 0x7f30fc1047e0 2024-09-10T08:37:10.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.263+0000 7f30f9600640 1 -- 172.21.15.89:0/482661983 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11845 (secure 0 0 0) 0x7f30fc19b800 con 0x7f30fc1047e0 2024-09-10T08:37:10.265 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:37:10.265 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 17 2024-09-10T08:37:10.266 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":17,"btime":"2024-09-10T08:37:08:233738+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:37:08.225259+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24554},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24554,"qdb_cluster":[24554]},"id":1}]} 2024-09-10T08:37:10.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.266+0000 7f3102400640 1 -- 172.21.15.89:0/482661983 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30e40a2b80 msgr2=0x7f30e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:10.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.266+0000 7f3102400640 1 --2- 172.21.15.89:0/482661983 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30e40a2b80 0x7f30e40a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009bd0 tx=0x7f30f0005f90 comp rx=0 tx=0).stop 2024-09-10T08:37:10.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.266+0000 7f3102400640 1 -- 172.21.15.89:0/482661983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30fc1047e0 msgr2=0x7f30fc19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:10.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.266+0000 7f3102400640 1 --2- 172.21.15.89:0/482661983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30fc1047e0 0x7f30fc19a9e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f30e8035040 tx=0x7f30e8033d90 comp rx=0 tx=0).stop 2024-09-10T08:37:10.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.266+0000 7f3102400640 1 -- 172.21.15.89:0/482661983 shutdown_connections 2024-09-10T08:37:10.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.266+0000 7f3102400640 1 --2- 172.21.15.89:0/482661983 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30e40a2b80 0x7f30e40a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:10.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.266+0000 7f3102400640 1 --2- 172.21.15.89:0/482661983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30fc108f60 0x7f30fc19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:10.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.266+0000 7f3102400640 1 --2- 172.21.15.89:0/482661983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30fc1047e0 0x7f30fc19a9e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:10.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.266+0000 7f3102400640 1 --2- 172.21.15.89:0/482661983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30fc103e30 0x7f30fc19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:10.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.266+0000 7f3102400640 1 -- 172.21.15.89:0/482661983 >> 172.21.15.89:0/482661983 conn(0x7f30fc0ff8d0 msgr2=0x7f30fc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:10.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.266+0000 7f3102400640 1 -- 172.21.15.89:0/482661983 shutdown_connections 2024-09-10T08:37:10.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:10.266+0000 7f3102400640 1 -- 172.21.15.89:0/482661983 wait complete. 2024-09-10T08:37:10.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:10 smithi111 ceph-mon[35220]: pgmap v306: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:10.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:10 smithi089 ceph-mon[30757]: pgmap v306: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:10.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:10 smithi149 ceph-mon[36167]: pgmap v306: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:11.171 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:37:11.540 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:11.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:11 smithi111 ceph-mon[35220]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T08:37:11.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:11 smithi111 ceph-mon[35220]: Cluster is now healthy 2024-09-10T08:37:11.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:11 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/482661983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:37:11.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:11 smithi089 ceph-mon[30757]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T08:37:11.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:11 smithi089 ceph-mon[30757]: Cluster is now healthy 2024-09-10T08:37:11.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:11 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/482661983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:37:11.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:11 smithi149 ceph-mon[36167]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T08:37:11.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:11 smithi149 ceph-mon[36167]: Cluster is now healthy 2024-09-10T08:37:11.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:11 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/482661983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:37:12.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.148+0000 7f71b9a00640 1 -- 172.21.15.89:0/2437665833 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71b4103e30 msgr2=0x7f71b4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:12.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.148+0000 7f71b9a00640 1 --2- 172.21.15.89:0/2437665833 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71b4103e30 0x7f71b4104210 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009900 tx=0x7f71ac02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:37:12.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.148+0000 7f71b9a00640 1 -- 172.21.15.89:0/2437665833 shutdown_connections 2024-09-10T08:37:12.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.148+0000 7f71b9a00640 1 --2- 172.21.15.89:0/2437665833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b4108f60 0x7f71b4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:12.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.148+0000 7f71b9a00640 1 --2- 172.21.15.89:0/2437665833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71b41047e0 0x7f71b4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:12.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.148+0000 7f71b9a00640 1 --2- 172.21.15.89:0/2437665833 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71b4103e30 0x7f71b4104210 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:12.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.148+0000 7f71b9a00640 1 -- 172.21.15.89:0/2437665833 >> 172.21.15.89:0/2437665833 conn(0x7f71b40ff910 msgr2=0x7f71b4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:12.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.148+0000 7f71b9a00640 1 -- 172.21.15.89:0/2437665833 shutdown_connections 2024-09-10T08:37:12.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.148+0000 7f71b9a00640 1 -- 172.21.15.89:0/2437665833 wait complete. 2024-09-10T08:37:12.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.149+0000 7f71b9a00640 1 Processor -- start 2024-09-10T08:37:12.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.149+0000 7f71b9a00640 1 -- start start 2024-09-10T08:37:12.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.150+0000 7f71b9a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b4103e30 0x7f71b419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:12.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.150+0000 7f71b9a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71b41047e0 0x7f71b419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:12.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.150+0000 7f71b9a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71b4108f60 0x7f71b419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:12.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.150+0000 7f71b9a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f71b4113fc0 con 0x7f71b4103e30 2024-09-10T08:37:12.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.150+0000 7f71b9a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f71b4113e40 con 0x7f71b41047e0 2024-09-10T08:37:12.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.150+0000 7f71b9a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f71b4114140 con 0x7f71b4108f60 2024-09-10T08:37:12.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.150+0000 7f71b3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b4103e30 0x7f71b419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:12.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.150+0000 7f71b3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71b41047e0 0x7f71b419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:12.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.150+0000 7f71b3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b4103e30 0x7f71b419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60934/0 (socket says 172.21.15.89:60934) 2024-09-10T08:37:12.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.150+0000 7f71b3e00640 1 -- 172.21.15.89:0/3466376049 learned_addr learned my addr 172.21.15.89:0/3466376049 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:12.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.150+0000 7f71b9000640 1 --2- 172.21.15.89:0/3466376049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71b4108f60 0x7f71b419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:12.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.150+0000 7f71b3400640 1 -- 172.21.15.89:0/3466376049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71b4108f60 msgr2=0x7f71b419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:12.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.150+0000 7f71b3400640 1 --2- 172.21.15.89:0/3466376049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71b4108f60 0x7f71b419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:12.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.150+0000 7f71b3400640 1 -- 172.21.15.89:0/3466376049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b4103e30 msgr2=0x7f71b419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:12.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.150+0000 7f71b3400640 1 --2- 172.21.15.89:0/3466376049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b4103e30 0x7f71b419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:12.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.150+0000 7f71b3400640 1 -- 172.21.15.89:0/3466376049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b419f490 con 0x7f71b41047e0 2024-09-10T08:37:12.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.150+0000 7f71b3e00640 1 --2- 172.21.15.89:0/3466376049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b4103e30 0x7f71b419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:37:12.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.151+0000 7f71b3400640 1 --2- 172.21.15.89:0/3466376049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71b41047e0 0x7f71b419aa20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f719c009fd0 tx=0x7f719c00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:12.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.151+0000 7f71b0c00640 1 -- 172.21.15.89:0/3466376049 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c004b20 con 0x7f71b41047e0 2024-09-10T08:37:12.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.151+0000 7f71b0c00640 1 -- 172.21.15.89:0/3466376049 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f719c004cc0 con 0x7f71b41047e0 2024-09-10T08:37:12.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.151+0000 7f71b0c00640 1 -- 172.21.15.89:0/3466376049 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c0107b0 con 0x7f71b41047e0 2024-09-10T08:37:12.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.151+0000 7f71b9a00640 1 -- 172.21.15.89:0/3466376049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b419f780 con 0x7f71b41047e0 2024-09-10T08:37:12.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.151+0000 7f71b9a00640 1 -- 172.21.15.89:0/3466376049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b41a7060 con 0x7f71b41047e0 2024-09-10T08:37:12.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.153+0000 7f71b9a00640 1 -- 172.21.15.89:0/3466376049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71b4105310 con 0x7f71b41047e0 2024-09-10T08:37:12.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.155+0000 7f71b0c00640 1 -- 172.21.15.89:0/3466376049 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f719c005000 con 0x7f71b41047e0 2024-09-10T08:37:12.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.156+0000 7f71b0c00640 1 --2- 172.21.15.89:0/3466376049 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f71980a2c00 0x7f71980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:12.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.156+0000 7f71b0c00640 1 -- 172.21.15.89:0/3466376049 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f719c0cf960 con 0x7f71b41047e0 2024-09-10T08:37:12.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.157+0000 7f71b3e00640 1 --2- 172.21.15.89:0/3466376049 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f71980a2c00 0x7f71980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:12.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.157+0000 7f71b3e00640 1 --2- 172.21.15.89:0/3466376049 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f71980a2c00 0x7f71980a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0096c0 tx=0x7f71ac005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:12.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.158+0000 7f71b0c00640 1 -- 172.21.15.89:0/3466376049 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f719c09c620 con 0x7f71b41047e0 2024-09-10T08:37:12.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:12 smithi089 ceph-mon[30757]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:12.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.340+0000 7f71b9a00640 1 -- 172.21.15.89:0/3466376049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71b4104210 con 0x7f71b41047e0 2024-09-10T08:37:12.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.342+0000 7f71b0c00640 1 -- 172.21.15.89:0/3466376049 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24236 (secure 0 0 0) 0x7f71b4104210 con 0x7f71b41047e0 2024-09-10T08:37:12.343 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:37:12.343 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":92,"fsid":"91f747bc-6f4e-11ef-bcea-c7b262605968","created":"2024-09-10T08:28:49.697035+0000","modified":"2024-09-10T08:37:08.225049+0000","last_up_change":"2024-09-10T08:36:00.535433+0000","last_in_change":"2024-09-10T08:35:37.131131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:32:02.060714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:36:52.462301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:36:55.119708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:37:03.729002+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d3b55192-bd95-4497-89ea-dc017a25c0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6803","nonce":1760523284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6805","nonce":1760523284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6809","nonce":1760523284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6807","nonce":1760523284}]},"public_addr":"172.21.15.89:6803/1760523284","cluster_addr":"172.21.15.89:6805/1760523284","heartbeat_back_addr":"172.21.15.89:6809/1760523284","heartbeat_front_addr":"172.21.15.89:6807/1760523284","state":["exists","up"]},{"osd":1,"uuid":"60aae117-cb52-4034-a700-11693171cd52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6801","nonce":2369291742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6803","nonce":2369291742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6807","nonce":2369291742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6805","nonce":2369291742}]},"public_addr":"172.21.15.111:6801/2369291742","cluster_addr":"172.21.15.111:6803/2369291742","heartbeat_back_addr":"172.21.15.111:6807/2369291742","heartbeat_front_addr":"172.21.15.111:6805/2369291742","state":["exists","up"]},{"osd":2,"uuid":"a305a019-0870-437c-a298-4f923ec54e77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6801","nonce":1092261395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6803","nonce":1092261395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6807","nonce":1092261395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6805","nonce":1092261395}]},"public_addr":"172.21.15.149:6801/1092261395","cluster_addr":"172.21.15.149:6803/1092261395","heartbeat_back_addr":"172.21.15.149:6807/1092261395","heartbeat_front_addr":"172.21.15.149:6805/1092261395","state":["exists","up"]},{"osd":3,"uuid":"d02f54c1-f883-40f6-9341-8d9fc4b712ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6811","nonce":1405773950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6813","nonce":1405773950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6817","nonce":1405773950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6815","nonce":1405773950}]},"public_addr":"172.21.15.89:6811/1405773950","cluster_addr":"172.21.15.89:6813/1405773950","heartbeat_back_addr":"172.21.15.89:6817/1405773950","heartbeat_front_addr":"172.21.15.89:6815/1405773950","state":["exists","up"]},{"osd":4,"uuid":"23cff46b-88dd-4344-be06-b35db6014f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6809","nonce":316717323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6811","nonce":316717323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6815","nonce":316717323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6813","nonce":316717323}]},"public_addr":"172.21.15.111:6809/316717323","cluster_addr":"172.21.15.111:6811/316717323","heartbeat_back_addr":"172.21.15.111:6815/316717323","heartbeat_front_addr":"172.21.15.111:6813/316717323","state":["exists","up"]},{"osd":5,"uuid":"8e516e8d-e7c6-4ebc-80a5-29932161df4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6809","nonce":1256148461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6811","nonce":1256148461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6815","nonce":1256148461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6813","nonce":1256148461}]},"public_addr":"172.21.15.149:6809/1256148461","cluster_addr":"172.21.15.149:6811/1256148461","heartbeat_back_addr":"172.21.15.149:6815/1256148461","heartbeat_front_addr":"172.21.15.149:6813/1256148461","state":["exists","up"]},{"osd":6,"uuid":"48f92472-f657-4c5a-adeb-20d4fea1ce8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6819","nonce":4190488024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6821","nonce":4190488024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6825","nonce":4190488024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6823","nonce":4190488024}]},"public_addr":"172.21.15.89:6819/4190488024","cluster_addr":"172.21.15.89:6821/4190488024","heartbeat_back_addr":"172.21.15.89:6825/4190488024","heartbeat_front_addr":"172.21.15.89:6823/4190488024","state":["exists","up"]},{"osd":7,"uuid":"14a2a6f0-c4cd-406b-b637-7527fd32babe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6817","nonce":652666376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6819","nonce":652666376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6823","nonce":652666376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6821","nonce":652666376}]},"public_addr":"172.21.15.111:6817/652666376","cluster_addr":"172.21.15.111:6819/652666376","heartbeat_back_addr":"172.21.15.111:6823/652666376","heartbeat_front_addr":"172.21.15.111:6821/652666376","state":["exists","up"]},{"osd":8,"uuid":"a54502e6-95f7-431f-8db1-54a6c7983b5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6817","nonce":2955172869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6819","nonce":2955172869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6823","nonce":2955172869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6821","nonce":2955172869}]},"public_addr":"172.21.15.149:6817/2955172869","cluster_addr":"172.21.15.149:6819/2955172869","heartbeat_back_addr":"172.21.15.149:6823/2955172869","heartbeat_front_addr":"172.21.15.149:6821/2955172869","state":["exists","up"]},{"osd":9,"uuid":"5fe17344-5b1c-4601-a633-639cc1691478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6827","nonce":2427467040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6829","nonce":2427467040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6833","nonce":2427467040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6831","nonce":2427467040}]},"public_addr":"172.21.15.89:6827/2427467040","cluster_addr":"172.21.15.89:6829/2427467040","heartbeat_back_addr":"172.21.15.89:6833/2427467040","heartbeat_front_addr":"172.21.15.89:6831/2427467040","state":["exists","up"]},{"osd":10,"uuid":"d8f29890-3064-4b67-b649-467a0ac86db7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6825","nonce":2328678744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6827","nonce":2328678744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6831","nonce":2328678744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6829","nonce":2328678744}]},"public_addr":"172.21.15.111:6825/2328678744","cluster_addr":"172.21.15.111:6827/2328678744","heartbeat_back_addr":"172.21.15.111:6831/2328678744","heartbeat_front_addr":"172.21.15.111:6829/2328678744","state":["exists","up"]},{"osd":11,"uuid":"92182c73-f139-47ca-a0ef-5046ae9b2310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6825","nonce":3130508827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6827","nonce":3130508827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6831","nonce":3130508827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6829","nonce":3130508827}]},"public_addr":"172.21.15.149:6825/3130508827","cluster_addr":"172.21.15.149:6827/3130508827","heartbeat_back_addr":"172.21.15.149:6831/3130508827","heartbeat_front_addr":"172.21.15.149:6829/3130508827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:09.031810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:33.695124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:58.065510+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:27.592838+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:53.275289+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:18.305677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:47.781535+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:12.889512+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:37.744193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:07.718408+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:33.391027+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:59.013250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/850669510":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/3541027428":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6801/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6800/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/2396468787":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/1609720980":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/995019144":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6801/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6800/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/2179977257":"2024-09-11T08:29:14.554869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:37:12.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.344+0000 7f71b9a00640 1 -- 172.21.15.89:0/3466376049 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f71980a2c00 msgr2=0x7f71980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:12.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.344+0000 7f71b9a00640 1 --2- 172.21.15.89:0/3466376049 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f71980a2c00 0x7f71980a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0096c0 tx=0x7f71ac005a20 comp rx=0 tx=0).stop 2024-09-10T08:37:12.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.344+0000 7f71b9a00640 1 -- 172.21.15.89:0/3466376049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71b41047e0 msgr2=0x7f71b419aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:12.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.344+0000 7f71b9a00640 1 --2- 172.21.15.89:0/3466376049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71b41047e0 0x7f71b419aa20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f719c009fd0 tx=0x7f719c00ef90 comp rx=0 tx=0).stop 2024-09-10T08:37:12.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.345+0000 7f71b9a00640 1 -- 172.21.15.89:0/3466376049 shutdown_connections 2024-09-10T08:37:12.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.345+0000 7f71b9a00640 1 --2- 172.21.15.89:0/3466376049 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f71980a2c00 0x7f71980a50c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:12.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.345+0000 7f71b9a00640 1 --2- 172.21.15.89:0/3466376049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71b4108f60 0x7f71b419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:12.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.345+0000 7f71b9a00640 1 --2- 172.21.15.89:0/3466376049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71b41047e0 0x7f71b419aa20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:12.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.345+0000 7f71b9a00640 1 --2- 172.21.15.89:0/3466376049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71b4103e30 0x7f71b419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:12.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.345+0000 7f71b9a00640 1 -- 172.21.15.89:0/3466376049 >> 172.21.15.89:0/3466376049 conn(0x7f71b40ff910 msgr2=0x7f71b4100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:12.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.345+0000 7f71b9a00640 1 -- 172.21.15.89:0/3466376049 shutdown_connections 2024-09-10T08:37:12.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:12.345+0000 7f71b9a00640 1 -- 172.21.15.89:0/3466376049 wait complete. 2024-09-10T08:37:12.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:12 smithi111 ceph-mon[35220]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:12.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:12 smithi149 ceph-mon[36167]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:13.222 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:37:13.549 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:13 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3466376049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:37:13.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:13 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3466376049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:37:13.594 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:13.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:13 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3466376049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:37:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.113+0000 7fd8c0000640 1 -- 172.21.15.89:0/2671910919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8b8101de0 msgr2=0x7fd8b81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:14.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.113+0000 7fd8c0000640 1 --2- 172.21.15.89:0/2671910919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8b8101de0 0x7fd8b81021c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac009900 tx=0x7fd8ac02a410 comp rx=0 tx=0).stop 2024-09-10T08:37:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.113+0000 7fd8c0000640 1 -- 172.21.15.89:0/2671910919 shutdown_connections 2024-09-10T08:37:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.113+0000 7fd8c0000640 1 --2- 172.21.15.89:0/2671910919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8b8106f10 0x7fd8b810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.113+0000 7fd8c0000640 1 --2- 172.21.15.89:0/2671910919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8b8102790 0x7fd8b81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.113+0000 7fd8c0000640 1 --2- 172.21.15.89:0/2671910919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8b8101de0 0x7fd8b81021c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.113+0000 7fd8c0000640 1 -- 172.21.15.89:0/2671910919 >> 172.21.15.89:0/2671910919 conn(0x7fd8b80fd8c0 msgr2=0x7fd8b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.113+0000 7fd8c0000640 1 -- 172.21.15.89:0/2671910919 shutdown_connections 2024-09-10T08:37:14.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.113+0000 7fd8c0000640 1 -- 172.21.15.89:0/2671910919 wait complete. 2024-09-10T08:37:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.114+0000 7fd8c0000640 1 Processor -- start 2024-09-10T08:37:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.115+0000 7fd8c0000640 1 -- start start 2024-09-10T08:37:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.115+0000 7fd8c0000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8b8101de0 0x7fd8b819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.115+0000 7fd8c0000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8b8102790 0x7fd8b819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.115+0000 7fd8c0000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8b8106f10 0x7fd8b819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.115+0000 7fd8c0000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b8111f70 con 0x7fd8b8102790 2024-09-10T08:37:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.115+0000 7fd8c0000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b8111df0 con 0x7fd8b8106f10 2024-09-10T08:37:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.115+0000 7fd8c0000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b81120f0 con 0x7fd8b8101de0 2024-09-10T08:37:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.115+0000 7fd8bcc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8b8102790 0x7fd8b819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.115+0000 7fd8bd600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8b8101de0 0x7fd8b819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:14.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.115+0000 7fd8bd600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8b8101de0 0x7fd8b819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:55914/0 (socket says 172.21.15.89:55914) 2024-09-10T08:37:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.115+0000 7fd8bd600640 1 -- 172.21.15.89:0/1155743651 learned_addr learned my addr 172.21.15.89:0/1155743651 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.115+0000 7fd8be000640 1 --2- 172.21.15.89:0/1155743651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8b8106f10 0x7fd8b819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.116+0000 7fd8bd600640 1 -- 172.21.15.89:0/1155743651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8b8106f10 msgr2=0x7fd8b819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.116+0000 7fd8bd600640 1 --2- 172.21.15.89:0/1155743651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8b8106f10 0x7fd8b819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.116+0000 7fd8bd600640 1 -- 172.21.15.89:0/1155743651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8b8102790 msgr2=0x7fd8b819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.116+0000 7fd8bd600640 1 --2- 172.21.15.89:0/1155743651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8b8102790 0x7fd8b819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.116+0000 7fd8bd600640 1 -- 172.21.15.89:0/1155743651 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b819f5e0 con 0x7fd8b8101de0 2024-09-10T08:37:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.116+0000 7fd8bcc00640 1 --2- 172.21.15.89:0/1155743651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8b8102790 0x7fd8b819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:37:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.116+0000 7fd8bd600640 1 --2- 172.21.15.89:0/1155743651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8b8101de0 0x7fd8b819a630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac006010 tx=0x7fd8ac0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.116+0000 7fd8b6000640 1 -- 172.21.15.89:0/1155743651 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac03f030 con 0x7fd8b8101de0 2024-09-10T08:37:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.116+0000 7fd8b6000640 1 -- 172.21.15.89:0/1155743651 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8ac003ce0 con 0x7fd8b8101de0 2024-09-10T08:37:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.116+0000 7fd8c0000640 1 -- 172.21.15.89:0/1155743651 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b819f870 con 0x7fd8b8101de0 2024-09-10T08:37:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.116+0000 7fd8b6000640 1 -- 172.21.15.89:0/1155743651 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac035070 con 0x7fd8b8101de0 2024-09-10T08:37:14.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.116+0000 7fd8c0000640 1 -- 172.21.15.89:0/1155743651 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b81a71b0 con 0x7fd8b8101de0 2024-09-10T08:37:14.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.117+0000 7fd8c0000640 1 -- 172.21.15.89:0/1155743651 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b81032c0 con 0x7fd8b8101de0 2024-09-10T08:37:14.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.119+0000 7fd8b6000640 1 -- 172.21.15.89:0/1155743651 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd8ac042030 con 0x7fd8b8101de0 2024-09-10T08:37:14.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.121+0000 7fd8b6000640 1 --2- 172.21.15.89:0/1155743651 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd8a00a2b80 0x7fd8a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:14.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.121+0000 7fd8b6000640 1 -- 172.21.15.89:0/1155743651 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fd8ac0ef9e0 con 0x7fd8b8101de0 2024-09-10T08:37:14.121 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.121+0000 7fd8bcc00640 1 --2- 172.21.15.89:0/1155743651 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd8a00a2b80 0x7fd8a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:14.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.121+0000 7fd8bcc00640 1 --2- 172.21.15.89:0/1155743651 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd8a00a2b80 0x7fd8a00a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80fe1a0 tx=0x7fd8a400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:14.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.123+0000 7fd8b6000640 1 -- 172.21.15.89:0/1155743651 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd8ac0bc110 con 0x7fd8b8101de0 2024-09-10T08:37:14.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.303+0000 7fd8c0000640 1 -- 172.21.15.89:0/1155743651 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8b81a7380 con 0x7fd8b8101de0 2024-09-10T08:37:14.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.305+0000 7fd8b6000640 1 -- 172.21.15.89:0/1155743651 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24236 (secure 0 0 0) 0x7fd8ac0a6e20 con 0x7fd8b8101de0 2024-09-10T08:37:14.306 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:37:14.306 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":92,"fsid":"91f747bc-6f4e-11ef-bcea-c7b262605968","created":"2024-09-10T08:28:49.697035+0000","modified":"2024-09-10T08:37:08.225049+0000","last_up_change":"2024-09-10T08:36:00.535433+0000","last_in_change":"2024-09-10T08:35:37.131131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:32:02.060714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:36:52.462301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:36:55.119708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:37:03.729002+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d3b55192-bd95-4497-89ea-dc017a25c0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6803","nonce":1760523284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6805","nonce":1760523284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6809","nonce":1760523284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6807","nonce":1760523284}]},"public_addr":"172.21.15.89:6803/1760523284","cluster_addr":"172.21.15.89:6805/1760523284","heartbeat_back_addr":"172.21.15.89:6809/1760523284","heartbeat_front_addr":"172.21.15.89:6807/1760523284","state":["exists","up"]},{"osd":1,"uuid":"60aae117-cb52-4034-a700-11693171cd52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6801","nonce":2369291742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6803","nonce":2369291742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6807","nonce":2369291742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6805","nonce":2369291742}]},"public_addr":"172.21.15.111:6801/2369291742","cluster_addr":"172.21.15.111:6803/2369291742","heartbeat_back_addr":"172.21.15.111:6807/2369291742","heartbeat_front_addr":"172.21.15.111:6805/2369291742","state":["exists","up"]},{"osd":2,"uuid":"a305a019-0870-437c-a298-4f923ec54e77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6801","nonce":1092261395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6803","nonce":1092261395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6807","nonce":1092261395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6805","nonce":1092261395}]},"public_addr":"172.21.15.149:6801/1092261395","cluster_addr":"172.21.15.149:6803/1092261395","heartbeat_back_addr":"172.21.15.149:6807/1092261395","heartbeat_front_addr":"172.21.15.149:6805/1092261395","state":["exists","up"]},{"osd":3,"uuid":"d02f54c1-f883-40f6-9341-8d9fc4b712ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6811","nonce":1405773950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6813","nonce":1405773950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6817","nonce":1405773950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6815","nonce":1405773950}]},"public_addr":"172.21.15.89:6811/1405773950","cluster_addr":"172.21.15.89:6813/1405773950","heartbeat_back_addr":"172.21.15.89:6817/1405773950","heartbeat_front_addr":"172.21.15.89:6815/1405773950","state":["exists","up"]},{"osd":4,"uuid":"23cff46b-88dd-4344-be06-b35db6014f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6809","nonce":316717323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6811","nonce":316717323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6815","nonce":316717323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6813","nonce":316717323}]},"public_addr":"172.21.15.111:6809/316717323","cluster_addr":"172.21.15.111:6811/316717323","heartbeat_back_addr":"172.21.15.111:6815/316717323","heartbeat_front_addr":"172.21.15.111:6813/316717323","state":["exists","up"]},{"osd":5,"uuid":"8e516e8d-e7c6-4ebc-80a5-29932161df4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6809","nonce":1256148461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6811","nonce":1256148461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6815","nonce":1256148461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6813","nonce":1256148461}]},"public_addr":"172.21.15.149:6809/1256148461","cluster_addr":"172.21.15.149:6811/1256148461","heartbeat_back_addr":"172.21.15.149:6815/1256148461","heartbeat_front_addr":"172.21.15.149:6813/1256148461","state":["exists","up"]},{"osd":6,"uuid":"48f92472-f657-4c5a-adeb-20d4fea1ce8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6819","nonce":4190488024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6821","nonce":4190488024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6825","nonce":4190488024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6823","nonce":4190488024}]},"public_addr":"172.21.15.89:6819/4190488024","cluster_addr":"172.21.15.89:6821/4190488024","heartbeat_back_addr":"172.21.15.89:6825/4190488024","heartbeat_front_addr":"172.21.15.89:6823/4190488024","state":["exists","up"]},{"osd":7,"uuid":"14a2a6f0-c4cd-406b-b637-7527fd32babe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6817","nonce":652666376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6819","nonce":652666376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6823","nonce":652666376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6821","nonce":652666376}]},"public_addr":"172.21.15.111:6817/652666376","cluster_addr":"172.21.15.111:6819/652666376","heartbeat_back_addr":"172.21.15.111:6823/652666376","heartbeat_front_addr":"172.21.15.111:6821/652666376","state":["exists","up"]},{"osd":8,"uuid":"a54502e6-95f7-431f-8db1-54a6c7983b5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6817","nonce":2955172869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6819","nonce":2955172869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6823","nonce":2955172869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6821","nonce":2955172869}]},"public_addr":"172.21.15.149:6817/2955172869","cluster_addr":"172.21.15.149:6819/2955172869","heartbeat_back_addr":"172.21.15.149:6823/2955172869","heartbeat_front_addr":"172.21.15.149:6821/2955172869","state":["exists","up"]},{"osd":9,"uuid":"5fe17344-5b1c-4601-a633-639cc1691478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6827","nonce":2427467040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6829","nonce":2427467040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6833","nonce":2427467040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6831","nonce":2427467040}]},"public_addr":"172.21.15.89:6827/2427467040","cluster_addr":"172.21.15.89:6829/2427467040","heartbeat_back_addr":"172.21.15.89:6833/2427467040","heartbeat_front_addr":"172.21.15.89:6831/2427467040","state":["exists","up"]},{"osd":10,"uuid":"d8f29890-3064-4b67-b649-467a0ac86db7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6825","nonce":2328678744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6827","nonce":2328678744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6831","nonce":2328678744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6829","nonce":2328678744}]},"public_addr":"172.21.15.111:6825/2328678744","cluster_addr":"172.21.15.111:6827/2328678744","heartbeat_back_addr":"172.21.15.111:6831/2328678744","heartbeat_front_addr":"172.21.15.111:6829/2328678744","state":["exists","up"]},{"osd":11,"uuid":"92182c73-f139-47ca-a0ef-5046ae9b2310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6825","nonce":3130508827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6827","nonce":3130508827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6831","nonce":3130508827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6829","nonce":3130508827}]},"public_addr":"172.21.15.149:6825/3130508827","cluster_addr":"172.21.15.149:6827/3130508827","heartbeat_back_addr":"172.21.15.149:6831/3130508827","heartbeat_front_addr":"172.21.15.149:6829/3130508827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:09.031810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:33.695124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:58.065510+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:27.592838+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:53.275289+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:18.305677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:47.781535+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:12.889512+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:37.744193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:07.718408+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:33.391027+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:59.013250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/850669510":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/3541027428":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6801/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6800/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/2396468787":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/1609720980":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/995019144":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6801/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6800/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/2179977257":"2024-09-11T08:29:14.554869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:37:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.307+0000 7fd8c0000640 1 -- 172.21.15.89:0/1155743651 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd8a00a2b80 msgr2=0x7fd8a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.307+0000 7fd8c0000640 1 --2- 172.21.15.89:0/1155743651 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd8a00a2b80 0x7fd8a00a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80fe1a0 tx=0x7fd8a400b390 comp rx=0 tx=0).stop 2024-09-10T08:37:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.307+0000 7fd8c0000640 1 -- 172.21.15.89:0/1155743651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8b8101de0 msgr2=0x7fd8b819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.307+0000 7fd8c0000640 1 --2- 172.21.15.89:0/1155743651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8b8101de0 0x7fd8b819a630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac006010 tx=0x7fd8ac0042f0 comp rx=0 tx=0).stop 2024-09-10T08:37:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.307+0000 7fd8c0000640 1 -- 172.21.15.89:0/1155743651 shutdown_connections 2024-09-10T08:37:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.307+0000 7fd8c0000640 1 --2- 172.21.15.89:0/1155743651 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd8a00a2b80 0x7fd8a00a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.307+0000 7fd8c0000640 1 --2- 172.21.15.89:0/1155743651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8b8106f10 0x7fd8b819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.307+0000 7fd8c0000640 1 --2- 172.21.15.89:0/1155743651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8b8102790 0x7fd8b819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:14.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.307+0000 7fd8c0000640 1 --2- 172.21.15.89:0/1155743651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd8b8101de0 0x7fd8b819a630 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.307+0000 7fd8c0000640 1 -- 172.21.15.89:0/1155743651 >> 172.21.15.89:0/1155743651 conn(0x7fd8b80fd8c0 msgr2=0x7fd8b80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.307+0000 7fd8c0000640 1 -- 172.21.15.89:0/1155743651 shutdown_connections 2024-09-10T08:37:14.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:14.307+0000 7fd8c0000640 1 -- 172.21.15.89:0/1155743651 wait complete. 2024-09-10T08:37:14.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:14 smithi111 ceph-mon[35220]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:14.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:14 smithi089 ceph-mon[30757]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:14.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:14 smithi149 ceph-mon[36167]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:15.229 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:37:15.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:15 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1155743651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:37:15.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:15 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1155743651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:37:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:15.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:15 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1155743651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:37:16.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.200+0000 7fb46d600640 1 -- 172.21.15.89:0/1471990131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb468074f40 msgr2=0x7fb46810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:16.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.200+0000 7fb46d600640 1 --2- 172.21.15.89:0/1471990131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb468074f40 0x7fb46810de80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb450009900 tx=0x7fb45002a480 comp rx=0 tx=0).stop 2024-09-10T08:37:16.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.200+0000 7fb46d600640 1 -- 172.21.15.89:0/1471990131 shutdown_connections 2024-09-10T08:37:16.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.200+0000 7fb46d600640 1 --2- 172.21.15.89:0/1471990131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb46810e3c0 0x7fb468110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:16.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.200+0000 7fb46d600640 1 --2- 172.21.15.89:0/1471990131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb468074f40 0x7fb46810de80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:16.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.200+0000 7fb46d600640 1 --2- 172.21.15.89:0/1471990131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb468074620 0x7fb468074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:16.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.200+0000 7fb46d600640 1 -- 172.21.15.89:0/1471990131 >> 172.21.15.89:0/1471990131 conn(0x7fb4680fd8c0 msgr2=0x7fb4680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:16.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.200+0000 7fb46d600640 1 -- 172.21.15.89:0/1471990131 shutdown_connections 2024-09-10T08:37:16.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.200+0000 7fb46d600640 1 -- 172.21.15.89:0/1471990131 wait complete. 2024-09-10T08:37:16.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.201+0000 7fb46d600640 1 Processor -- start 2024-09-10T08:37:16.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.201+0000 7fb46d600640 1 -- start start 2024-09-10T08:37:16.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.202+0000 7fb46d600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb468074620 0x7fb46819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:16.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.202+0000 7fb46d600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb468074f40 0x7fb46819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:16.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.202+0000 7fb46d600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb46810e3c0 0x7fb46819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:16.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.202+0000 7fb46d600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb468114160 con 0x7fb468074620 2024-09-10T08:37:16.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.202+0000 7fb46d600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb468113fe0 con 0x7fb46810e3c0 2024-09-10T08:37:16.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.202+0000 7fb46d600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb4681142e0 con 0x7fb468074f40 2024-09-10T08:37:16.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.202+0000 7fb466a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb468074620 0x7fb46819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:16.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.202+0000 7fb467400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb46810e3c0 0x7fb46819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:16.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.202+0000 7fb467400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb46810e3c0 0x7fb46819eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:56540/0 (socket says 172.21.15.89:56540) 2024-09-10T08:37:16.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.202+0000 7fb467400640 1 -- 172.21.15.89:0/3534996353 learned_addr learned my addr 172.21.15.89:0/3534996353 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:16.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.202+0000 7fb466a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb468074620 0x7fb46819a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39338/0 (socket says 172.21.15.89:39338) 2024-09-10T08:37:16.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.202+0000 7fb45e000640 1 --2- 172.21.15.89:0/3534996353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb468074f40 0x7fb46819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:16.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.202+0000 7fb467400640 1 -- 172.21.15.89:0/3534996353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb468074f40 msgr2=0x7fb46819ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:16.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.202+0000 7fb467400640 1 --2- 172.21.15.89:0/3534996353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb468074f40 0x7fb46819ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:16.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.202+0000 7fb467400640 1 -- 172.21.15.89:0/3534996353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb468074620 msgr2=0x7fb46819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:16.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.202+0000 7fb467400640 1 --2- 172.21.15.89:0/3534996353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb468074620 0x7fb46819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:16.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.202+0000 7fb467400640 1 -- 172.21.15.89:0/3534996353 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46819f6d0 con 0x7fb46810e3c0 2024-09-10T08:37:16.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.202+0000 7fb466a00640 1 --2- 172.21.15.89:0/3534996353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb468074620 0x7fb46819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:37:16.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.203+0000 7fb467400640 1 --2- 172.21.15.89:0/3534996353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb46810e3c0 0x7fb46819eff0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb46000e970 tx=0x7fb46000ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:16.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.203+0000 7fb45fe00640 1 -- 172.21.15.89:0/3534996353 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46000ccc0 con 0x7fb46810e3c0 2024-09-10T08:37:16.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.203+0000 7fb46d600640 1 -- 172.21.15.89:0/3534996353 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb46819f9c0 con 0x7fb46810e3c0 2024-09-10T08:37:16.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.203+0000 7fb45fe00640 1 -- 172.21.15.89:0/3534996353 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb46000ce60 con 0x7fb46810e3c0 2024-09-10T08:37:16.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.203+0000 7fb45fe00640 1 -- 172.21.15.89:0/3534996353 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460010490 con 0x7fb46810e3c0 2024-09-10T08:37:16.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.203+0000 7fb46d600640 1 -- 172.21.15.89:0/3534996353 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4681a72a0 con 0x7fb46810e3c0 2024-09-10T08:37:16.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.204+0000 7fb46d600640 1 -- 172.21.15.89:0/3534996353 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb468074a00 con 0x7fb46810e3c0 2024-09-10T08:37:16.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.206+0000 7fb45fe00640 1 -- 172.21.15.89:0/3534996353 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb460005000 con 0x7fb46810e3c0 2024-09-10T08:37:16.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.207+0000 7fb45fe00640 1 --2- 172.21.15.89:0/3534996353 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb44c0a2b80 0x7fb44c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:16.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.208+0000 7fb45fe00640 1 -- 172.21.15.89:0/3534996353 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fb4600cf880 con 0x7fb46810e3c0 2024-09-10T08:37:16.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.208+0000 7fb466a00640 1 --2- 172.21.15.89:0/3534996353 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb44c0a2b80 0x7fb44c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:16.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.208+0000 7fb466a00640 1 --2- 172.21.15.89:0/3534996353 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb44c0a2b80 0x7fb44c0a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb458009fd0 tx=0x7fb458009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:16.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.209+0000 7fb45fe00640 1 -- 172.21.15.89:0/3534996353 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb4600541b0 con 0x7fb46810e3c0 2024-09-10T08:37:16.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.421+0000 7fb46d600640 1 -- 172.21.15.89:0/3534996353 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb468109540 con 0x7fb46810e3c0 2024-09-10T08:37:16.422 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:16 smithi089 ceph-mon[30757]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:16.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.422+0000 7fb45fe00640 1 -- 172.21.15.89:0/3534996353 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11845 (secure 0 0 0) 0x7fb46009c540 con 0x7fb46810e3c0 2024-09-10T08:37:16.423 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:37:16.423 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":17,"btime":"2024-09-10T08:37:08:233738+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:37:08.225259+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24554},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24554,"qdb_cluster":[24554]},"id":1}]} 2024-09-10T08:37:16.423 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 17 2024-09-10T08:37:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.424+0000 7fb46d600640 1 -- 172.21.15.89:0/3534996353 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb44c0a2b80 msgr2=0x7fb44c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.424+0000 7fb46d600640 1 --2- 172.21.15.89:0/3534996353 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb44c0a2b80 0x7fb44c0a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb458009fd0 tx=0x7fb458009680 comp rx=0 tx=0).stop 2024-09-10T08:37:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.424+0000 7fb46d600640 1 -- 172.21.15.89:0/3534996353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb46810e3c0 msgr2=0x7fb46819eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.424+0000 7fb46d600640 1 --2- 172.21.15.89:0/3534996353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb46810e3c0 0x7fb46819eff0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb46000e970 tx=0x7fb46000ed70 comp rx=0 tx=0).stop 2024-09-10T08:37:16.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.424+0000 7fb46d600640 1 -- 172.21.15.89:0/3534996353 shutdown_connections 2024-09-10T08:37:16.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.424+0000 7fb46d600640 1 --2- 172.21.15.89:0/3534996353 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb44c0a2b80 0x7fb44c0a5040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:16.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.424+0000 7fb46d600640 1 --2- 172.21.15.89:0/3534996353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb46810e3c0 0x7fb46819eff0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:16.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.424+0000 7fb46d600640 1 --2- 172.21.15.89:0/3534996353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb468074f40 0x7fb46819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:16.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.424+0000 7fb46d600640 1 --2- 172.21.15.89:0/3534996353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb468074620 0x7fb46819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:16.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.424+0000 7fb46d600640 1 -- 172.21.15.89:0/3534996353 >> 172.21.15.89:0/3534996353 conn(0x7fb4680fd8c0 msgr2=0x7fb4680fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:16.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.424+0000 7fb46d600640 1 -- 172.21.15.89:0/3534996353 shutdown_connections 2024-09-10T08:37:16.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:16.425+0000 7fb46d600640 1 -- 172.21.15.89:0/3534996353 wait complete. 2024-09-10T08:37:16.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:16 smithi111 ceph-mon[35220]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:16.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:16 smithi149 ceph-mon[36167]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:17.288 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:37:17.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:17 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3534996353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:37:17.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:17 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3534996353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:37:17.655 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:17.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:17 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3534996353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:37:18.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.189+0000 7f846d400640 1 -- 172.21.15.89:0/2626340334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8468074590 msgr2=0x7f8468074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:18.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.189+0000 7f846d400640 1 --2- 172.21.15.89:0/2626340334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8468074590 0x7f8468074970 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8458009900 tx=0x7f845802a480 comp rx=0 tx=0).stop 2024-09-10T08:37:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.189+0000 7f846d400640 1 -- 172.21.15.89:0/2626340334 shutdown_connections 2024-09-10T08:37:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.189+0000 7f846d400640 1 --2- 172.21.15.89:0/2626340334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8468112b90 0x7f8468114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.189+0000 7f846d400640 1 --2- 172.21.15.89:0/2626340334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8468074eb0 0x7f8468112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.189+0000 7f846d400640 1 --2- 172.21.15.89:0/2626340334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8468074590 0x7f8468074970 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.189+0000 7f846d400640 1 -- 172.21.15.89:0/2626340334 >> 172.21.15.89:0/2626340334 conn(0x7f84680fd8c0 msgr2=0x7f84680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.189+0000 7f846d400640 1 -- 172.21.15.89:0/2626340334 shutdown_connections 2024-09-10T08:37:18.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.189+0000 7f846d400640 1 -- 172.21.15.89:0/2626340334 wait complete. 2024-09-10T08:37:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.190+0000 7f846d400640 1 Processor -- start 2024-09-10T08:37:18.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.190+0000 7f846d400640 1 -- start start 2024-09-10T08:37:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.191+0000 7f846d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8468074590 0x7f84681a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.191+0000 7f846d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8468074eb0 0x7f84681a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.191+0000 7f846d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8468112b90 0x7f846819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.191+0000 7f846d400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f84681185c0 con 0x7f8468074590 2024-09-10T08:37:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.191+0000 7f846d400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8468118440 con 0x7f8468112b90 2024-09-10T08:37:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.191+0000 7f846d400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8468118740 con 0x7f8468074eb0 2024-09-10T08:37:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.191+0000 7f845de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8468074eb0 0x7f84681a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.191+0000 7f8466800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8468074590 0x7f84681a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.191+0000 7f845de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8468074eb0 0x7f84681a4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:55936/0 (socket says 172.21.15.89:55936) 2024-09-10T08:37:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.191+0000 7f845de00640 1 -- 172.21.15.89:0/2001028163 learned_addr learned my addr 172.21.15.89:0/2001028163 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:18.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.191+0000 7f8467200640 1 --2- 172.21.15.89:0/2001028163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8468112b90 0x7f846819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.191+0000 7f845de00640 1 -- 172.21.15.89:0/2001028163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8468112b90 msgr2=0x7f846819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.191+0000 7f845de00640 1 --2- 172.21.15.89:0/2001028163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8468112b90 0x7f846819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.191+0000 7f845de00640 1 -- 172.21.15.89:0/2001028163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8468074590 msgr2=0x7f84681a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.191+0000 7f845de00640 1 --2- 172.21.15.89:0/2001028163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8468074590 0x7f84681a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.191+0000 7f845de00640 1 -- 172.21.15.89:0/2001028163 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846819f050 con 0x7f8468074eb0 2024-09-10T08:37:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.191+0000 7f8466800640 1 --2- 172.21.15.89:0/2001028163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8468074590 0x7f84681a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:37:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.192+0000 7f845de00640 1 --2- 172.21.15.89:0/2001028163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8468074eb0 0x7f84681a4ca0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f845000d770 tx=0x7f845000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.192+0000 7f845fe00640 1 -- 172.21.15.89:0/2001028163 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450017070 con 0x7f8468074eb0 2024-09-10T08:37:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.192+0000 7f846d400640 1 -- 172.21.15.89:0/2001028163 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846819f340 con 0x7f8468074eb0 2024-09-10T08:37:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.192+0000 7f846d400640 1 -- 172.21.15.89:0/2001028163 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84681ab630 con 0x7f8468074eb0 2024-09-10T08:37:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.192+0000 7f845fe00640 1 -- 172.21.15.89:0/2001028163 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8450005500 con 0x7f8468074eb0 2024-09-10T08:37:18.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.192+0000 7f845fe00640 1 -- 172.21.15.89:0/2001028163 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450020060 con 0x7f8468074eb0 2024-09-10T08:37:18.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.193+0000 7f846d400640 1 -- 172.21.15.89:0/2001028163 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8468074970 con 0x7f8468074eb0 2024-09-10T08:37:18.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.196+0000 7f845fe00640 1 -- 172.21.15.89:0/2001028163 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8450009cc0 con 0x7f8468074eb0 2024-09-10T08:37:18.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.197+0000 7f845fe00640 1 --2- 172.21.15.89:0/2001028163 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f844c0a2c00 0x7f844c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:18.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.198+0000 7f845fe00640 1 -- 172.21.15.89:0/2001028163 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f84500cefe0 con 0x7f8468074eb0 2024-09-10T08:37:18.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.198+0000 7f8466800640 1 --2- 172.21.15.89:0/2001028163 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f844c0a2c00 0x7f844c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:18.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.198+0000 7f8466800640 1 --2- 172.21.15.89:0/2001028163 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f844c0a2c00 0x7f844c0a50c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8458006010 tx=0x7f8458031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:18.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.199+0000 7f845fe00640 1 -- 172.21.15.89:0/2001028163 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f845009b6d0 con 0x7f8468074eb0 2024-09-10T08:37:18.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.375+0000 7f846d400640 1 -- 172.21.15.89:0/2001028163 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f846819fe50 con 0x7f8468074eb0 2024-09-10T08:37:18.377 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:18 smithi089 ceph-mon[30757]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:18.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.377+0000 7f845fe00640 1 -- 172.21.15.89:0/2001028163 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24236 (secure 0 0 0) 0x7f845009f9d0 con 0x7f8468074eb0 2024-09-10T08:37:18.378 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:37:18.379 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":92,"fsid":"91f747bc-6f4e-11ef-bcea-c7b262605968","created":"2024-09-10T08:28:49.697035+0000","modified":"2024-09-10T08:37:08.225049+0000","last_up_change":"2024-09-10T08:36:00.535433+0000","last_in_change":"2024-09-10T08:35:37.131131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:32:02.060714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:36:52.462301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:36:55.119708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:37:03.729002+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d3b55192-bd95-4497-89ea-dc017a25c0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6803","nonce":1760523284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6805","nonce":1760523284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6809","nonce":1760523284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6807","nonce":1760523284}]},"public_addr":"172.21.15.89:6803/1760523284","cluster_addr":"172.21.15.89:6805/1760523284","heartbeat_back_addr":"172.21.15.89:6809/1760523284","heartbeat_front_addr":"172.21.15.89:6807/1760523284","state":["exists","up"]},{"osd":1,"uuid":"60aae117-cb52-4034-a700-11693171cd52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6801","nonce":2369291742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6803","nonce":2369291742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6807","nonce":2369291742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6805","nonce":2369291742}]},"public_addr":"172.21.15.111:6801/2369291742","cluster_addr":"172.21.15.111:6803/2369291742","heartbeat_back_addr":"172.21.15.111:6807/2369291742","heartbeat_front_addr":"172.21.15.111:6805/2369291742","state":["exists","up"]},{"osd":2,"uuid":"a305a019-0870-437c-a298-4f923ec54e77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6801","nonce":1092261395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6803","nonce":1092261395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6807","nonce":1092261395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6805","nonce":1092261395}]},"public_addr":"172.21.15.149:6801/1092261395","cluster_addr":"172.21.15.149:6803/1092261395","heartbeat_back_addr":"172.21.15.149:6807/1092261395","heartbeat_front_addr":"172.21.15.149:6805/1092261395","state":["exists","up"]},{"osd":3,"uuid":"d02f54c1-f883-40f6-9341-8d9fc4b712ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6811","nonce":1405773950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6813","nonce":1405773950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6817","nonce":1405773950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6815","nonce":1405773950}]},"public_addr":"172.21.15.89:6811/1405773950","cluster_addr":"172.21.15.89:6813/1405773950","heartbeat_back_addr":"172.21.15.89:6817/1405773950","heartbeat_front_addr":"172.21.15.89:6815/1405773950","state":["exists","up"]},{"osd":4,"uuid":"23cff46b-88dd-4344-be06-b35db6014f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6809","nonce":316717323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6811","nonce":316717323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6815","nonce":316717323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6813","nonce":316717323}]},"public_addr":"172.21.15.111:6809/316717323","cluster_addr":"172.21.15.111:6811/316717323","heartbeat_back_addr":"172.21.15.111:6815/316717323","heartbeat_front_addr":"172.21.15.111:6813/316717323","state":["exists","up"]},{"osd":5,"uuid":"8e516e8d-e7c6-4ebc-80a5-29932161df4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6809","nonce":1256148461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6811","nonce":1256148461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6815","nonce":1256148461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6813","nonce":1256148461}]},"public_addr":"172.21.15.149:6809/1256148461","cluster_addr":"172.21.15.149:6811/1256148461","heartbeat_back_addr":"172.21.15.149:6815/1256148461","heartbeat_front_addr":"172.21.15.149:6813/1256148461","state":["exists","up"]},{"osd":6,"uuid":"48f92472-f657-4c5a-adeb-20d4fea1ce8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6819","nonce":4190488024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6821","nonce":4190488024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6825","nonce":4190488024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6823","nonce":4190488024}]},"public_addr":"172.21.15.89:6819/4190488024","cluster_addr":"172.21.15.89:6821/4190488024","heartbeat_back_addr":"172.21.15.89:6825/4190488024","heartbeat_front_addr":"172.21.15.89:6823/4190488024","state":["exists","up"]},{"osd":7,"uuid":"14a2a6f0-c4cd-406b-b637-7527fd32babe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6817","nonce":652666376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6819","nonce":652666376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6823","nonce":652666376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6821","nonce":652666376}]},"public_addr":"172.21.15.111:6817/652666376","cluster_addr":"172.21.15.111:6819/652666376","heartbeat_back_addr":"172.21.15.111:6823/652666376","heartbeat_front_addr":"172.21.15.111:6821/652666376","state":["exists","up"]},{"osd":8,"uuid":"a54502e6-95f7-431f-8db1-54a6c7983b5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6817","nonce":2955172869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6819","nonce":2955172869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6823","nonce":2955172869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6821","nonce":2955172869}]},"public_addr":"172.21.15.149:6817/2955172869","cluster_addr":"172.21.15.149:6819/2955172869","heartbeat_back_addr":"172.21.15.149:6823/2955172869","heartbeat_front_addr":"172.21.15.149:6821/2955172869","state":["exists","up"]},{"osd":9,"uuid":"5fe17344-5b1c-4601-a633-639cc1691478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6827","nonce":2427467040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6829","nonce":2427467040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6833","nonce":2427467040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6831","nonce":2427467040}]},"public_addr":"172.21.15.89:6827/2427467040","cluster_addr":"172.21.15.89:6829/2427467040","heartbeat_back_addr":"172.21.15.89:6833/2427467040","heartbeat_front_addr":"172.21.15.89:6831/2427467040","state":["exists","up"]},{"osd":10,"uuid":"d8f29890-3064-4b67-b649-467a0ac86db7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6825","nonce":2328678744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6827","nonce":2328678744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6831","nonce":2328678744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6829","nonce":2328678744}]},"public_addr":"172.21.15.111:6825/2328678744","cluster_addr":"172.21.15.111:6827/2328678744","heartbeat_back_addr":"172.21.15.111:6831/2328678744","heartbeat_front_addr":"172.21.15.111:6829/2328678744","state":["exists","up"]},{"osd":11,"uuid":"92182c73-f139-47ca-a0ef-5046ae9b2310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6825","nonce":3130508827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6827","nonce":3130508827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6831","nonce":3130508827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6829","nonce":3130508827}]},"public_addr":"172.21.15.149:6825/3130508827","cluster_addr":"172.21.15.149:6827/3130508827","heartbeat_back_addr":"172.21.15.149:6831/3130508827","heartbeat_front_addr":"172.21.15.149:6829/3130508827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:09.031810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:33.695124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:58.065510+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:27.592838+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:53.275289+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:18.305677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:47.781535+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:12.889512+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:37.744193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:07.718408+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:33.391027+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:59.013250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/850669510":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/3541027428":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6801/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6800/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/2396468787":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/1609720980":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/995019144":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6801/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6800/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/2179977257":"2024-09-11T08:29:14.554869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:37:18.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.379+0000 7f846d400640 1 -- 172.21.15.89:0/2001028163 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f844c0a2c00 msgr2=0x7f844c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:18.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.379+0000 7f846d400640 1 --2- 172.21.15.89:0/2001028163 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f844c0a2c00 0x7f844c0a50c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8458006010 tx=0x7f8458031510 comp rx=0 tx=0).stop 2024-09-10T08:37:18.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.379+0000 7f846d400640 1 -- 172.21.15.89:0/2001028163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8468074eb0 msgr2=0x7f84681a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:18.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.379+0000 7f846d400640 1 --2- 172.21.15.89:0/2001028163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8468074eb0 0x7f84681a4ca0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f845000d770 tx=0x7f845000dc30 comp rx=0 tx=0).stop 2024-09-10T08:37:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.380+0000 7f846d400640 1 -- 172.21.15.89:0/2001028163 shutdown_connections 2024-09-10T08:37:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.380+0000 7f846d400640 1 --2- 172.21.15.89:0/2001028163 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f844c0a2c00 0x7f844c0a50c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.380+0000 7f846d400640 1 --2- 172.21.15.89:0/2001028163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8468112b90 0x7f846819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.380+0000 7f846d400640 1 --2- 172.21.15.89:0/2001028163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8468074eb0 0x7f84681a4ca0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.380+0000 7f846d400640 1 --2- 172.21.15.89:0/2001028163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8468074590 0x7f84681a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.380+0000 7f846d400640 1 -- 172.21.15.89:0/2001028163 >> 172.21.15.89:0/2001028163 conn(0x7f84680fd8c0 msgr2=0x7f84680ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.380+0000 7f846d400640 1 -- 172.21.15.89:0/2001028163 shutdown_connections 2024-09-10T08:37:18.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:18.380+0000 7f846d400640 1 -- 172.21.15.89:0/2001028163 wait complete. 2024-09-10T08:37:18.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:18 smithi111 ceph-mon[35220]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:18.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:18 smithi149 ceph-mon[36167]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:19.120 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T08:36:57.162188+0000', 'modified': '2024-09-10T08:37:08.225259+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24554}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24554': {'gid': 24554, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.111:6839/3640096926', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 3640096926}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 3640096926}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24554, 'qdb_cluster': [24554]} 2024-09-10T08:37:19.120 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T08:37:19.120 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T08:37:19.486 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:19.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:19 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2001028163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:37:19.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:19 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2001028163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:37:19.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:19 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2001028163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:37:20.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.039+0000 7fbb45a00640 1 -- 172.21.15.89:0/568797179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb40108f60 msgr2=0x7fbb40110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:20.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.039+0000 7fbb45a00640 1 --2- 172.21.15.89:0/568797179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb40108f60 0x7fbb40110ae0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009ca0 tx=0x7fbb3802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.039+0000 7fbb45a00640 1 -- 172.21.15.89:0/568797179 shutdown_connections 2024-09-10T08:37:20.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.039+0000 7fbb45a00640 1 --2- 172.21.15.89:0/568797179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb40108f60 0x7fbb40110ae0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.039+0000 7fbb45a00640 1 --2- 172.21.15.89:0/568797179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb401047e0 0x7fbb40108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.039+0000 7fbb45a00640 1 --2- 172.21.15.89:0/568797179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb40103e30 0x7fbb40104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.039+0000 7fbb45a00640 1 -- 172.21.15.89:0/568797179 >> 172.21.15.89:0/568797179 conn(0x7fbb400ff8d0 msgr2=0x7fbb40101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:20.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.039+0000 7fbb45a00640 1 -- 172.21.15.89:0/568797179 shutdown_connections 2024-09-10T08:37:20.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.039+0000 7fbb45a00640 1 -- 172.21.15.89:0/568797179 wait complete. 2024-09-10T08:37:20.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.040+0000 7fbb45a00640 1 Processor -- start 2024-09-10T08:37:20.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.040+0000 7fbb45a00640 1 -- start start 2024-09-10T08:37:20.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.041+0000 7fbb45a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb40103e30 0x7fbb4019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.041+0000 7fbb45a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb401047e0 0x7fbb4019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.041+0000 7fbb45a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb40108f60 0x7fbb4019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.041+0000 7fbb45a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbb40114010 con 0x7fbb401047e0 2024-09-10T08:37:20.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.041+0000 7fbb45a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbb40113e90 con 0x7fbb40103e30 2024-09-10T08:37:20.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.041+0000 7fbb45a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbb40114190 con 0x7fbb40108f60 2024-09-10T08:37:20.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.041+0000 7fbb3f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb401047e0 0x7fbb4019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:20.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.041+0000 7fbb3f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb401047e0 0x7fbb4019a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39370/0 (socket says 172.21.15.89:39370) 2024-09-10T08:37:20.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.041+0000 7fbb3f400640 1 -- 172.21.15.89:0/1984048067 learned_addr learned my addr 172.21.15.89:0/1984048067 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:20.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.041+0000 7fbb3fe00640 1 --2- 172.21.15.89:0/1984048067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb40103e30 0x7fbb4019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:20.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.041+0000 7fbb45000640 1 --2- 172.21.15.89:0/1984048067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb40108f60 0x7fbb4019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:20.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.041+0000 7fbb3f400640 1 -- 172.21.15.89:0/1984048067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb40108f60 msgr2=0x7fbb4019ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:20.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.041+0000 7fbb3f400640 1 --2- 172.21.15.89:0/1984048067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb40108f60 0x7fbb4019ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.041+0000 7fbb3f400640 1 -- 172.21.15.89:0/1984048067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb40103e30 msgr2=0x7fbb4019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:20.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.041+0000 7fbb3f400640 1 --2- 172.21.15.89:0/1984048067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb40103e30 0x7fbb4019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.041+0000 7fbb3f400640 1 -- 172.21.15.89:0/1984048067 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4019f3c0 con 0x7fbb401047e0 2024-09-10T08:37:20.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.041+0000 7fbb3fe00640 1 --2- 172.21.15.89:0/1984048067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb40103e30 0x7fbb4019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:37:20.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.042+0000 7fbb3f400640 1 --2- 172.21.15.89:0/1984048067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb401047e0 0x7fbb4019a950 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00e9f0 tx=0x7fbb2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:20.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.042+0000 7fbb3cc00640 1 -- 172.21.15.89:0/1984048067 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c00cd80 con 0x7fbb401047e0 2024-09-10T08:37:20.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.042+0000 7fbb3cc00640 1 -- 172.21.15.89:0/1984048067 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb2c010910 con 0x7fbb401047e0 2024-09-10T08:37:20.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.042+0000 7fbb3cc00640 1 -- 172.21.15.89:0/1984048067 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c018990 con 0x7fbb401047e0 2024-09-10T08:37:20.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.042+0000 7fbb45a00640 1 -- 172.21.15.89:0/1984048067 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4019f6b0 con 0x7fbb401047e0 2024-09-10T08:37:20.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.042+0000 7fbb45a00640 1 -- 172.21.15.89:0/1984048067 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb401a6f90 con 0x7fbb401047e0 2024-09-10T08:37:20.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.046+0000 7fbb49a75740 1 Processor -- start 2024-09-10T08:37:20.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.046+0000 7fbb49a75740 1 -- start start 2024-09-10T08:37:20.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.047+0000 7fbb49a75740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a3a4e0 0x55a430a3a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.047+0000 7fbb49a75740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a3ae00 0x55a430a45180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.047+0000 7fbb49a75740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a430a456c0 0x55a430a47ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.047+0000 7fbb49a75740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a4309a00b0 con 0x55a430a3ae00 2024-09-10T08:37:20.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.047+0000 7fbb49a75740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a430947ab0 con 0x55a430a3a4e0 2024-09-10T08:37:20.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.047+0000 7fbb49a75740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a43099c070 con 0x55a430a456c0 2024-09-10T08:37:20.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.047+0000 7fbb3cc00640 1 -- 172.21.15.89:0/1984048067 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbb2c010ac0 con 0x7fbb401047e0 2024-09-10T08:37:20.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.047+0000 7fbb45000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a3a4e0 0x55a430a3a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:20.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.047+0000 7fbb45000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a3a4e0 0x55a430a3a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:56580/0 (socket says 172.21.15.89:56580) 2024-09-10T08:37:20.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.047+0000 7fbb45000640 1 -- 172.21.15.89:0/2624909842 learned_addr learned my addr 172.21.15.89:0/2624909842 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:20.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.047+0000 7fbb3fe00640 1 --2- 172.21.15.89:0/2624909842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a3ae00 0x55a430a45180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:20.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.047+0000 7fbb45000640 1 -- 172.21.15.89:0/2624909842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a430a456c0 msgr2=0x55a430a47ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:37:20.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.047+0000 7fbb45000640 1 --2- 172.21.15.89:0/2624909842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a430a456c0 0x55a430a47ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.047+0000 7fbb45000640 1 -- 172.21.15.89:0/2624909842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a3ae00 msgr2=0x55a430a45180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:37:20.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.047+0000 7fbb45000640 1 --2- 172.21.15.89:0/2624909842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a3ae00 0x55a430a45180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.047+0000 7fbb45000640 1 -- 172.21.15.89:0/2624909842 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4309544a0 con 0x55a430a3a4e0 2024-09-10T08:37:20.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.047+0000 7fbb3fe00640 1 --2- 172.21.15.89:0/2624909842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a3ae00 0x55a430a45180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:37:20.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.048+0000 7fbb45000640 1 --2- 172.21.15.89:0/2624909842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a3a4e0 0x55a430a3a8c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009870 tx=0x7fbb380098a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5331766c5c3f13 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:20.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.048+0000 7fbb35600640 1 -- 172.21.15.89:0/2624909842 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb38031db0 con 0x55a430a3a4e0 2024-09-10T08:37:20.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.048+0000 7fbb35600640 1 -- 172.21.15.89:0/2624909842 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb38036040 con 0x55a430a3a4e0 2024-09-10T08:37:20.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.048+0000 7fbb35600640 1 -- 172.21.15.89:0/2624909842 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb380043c0 con 0x55a430a3a4e0 2024-09-10T08:37:20.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.049+0000 7fbb3cc00640 1 --2- 172.21.15.89:0/1984048067 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbb240a2c00 0x7fbb240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.049+0000 7fbb3cc00640 1 -- 172.21.15.89:0/1984048067 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fbb2c0cf560 con 0x7fbb401047e0 2024-09-10T08:37:20.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.049+0000 7fbb3fe00640 1 --2- 172.21.15.89:0/1984048067 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbb240a2c00 0x7fbb240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:20.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.049+0000 7fbb49a75740 1 -- 172.21.15.89:0/2624909842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a3a4e0 msgr2=0x55a430a3a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:20.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.049+0000 7fbb49a75740 1 --2- 172.21.15.89:0/2624909842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a3a4e0 0x55a430a3a8c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009870 tx=0x7fbb380098a0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.049+0000 7fbb49a75740 1 -- 172.21.15.89:0/2624909842 shutdown_connections 2024-09-10T08:37:20.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.049+0000 7fbb49a75740 1 --2- 172.21.15.89:0/2624909842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a430a456c0 0x55a430a47ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.049+0000 7fbb49a75740 1 --2- 172.21.15.89:0/2624909842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a3ae00 0x55a430a45180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.049+0000 7fbb49a75740 1 --2- 172.21.15.89:0/2624909842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a3a4e0 0x55a430a3a8c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.049+0000 7fbb49a75740 1 -- 172.21.15.89:0/2624909842 >> 172.21.15.89:0/2624909842 conn(0x55a430a36a70 msgr2=0x55a430a36e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:20.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.049+0000 7fbb49a75740 1 -- 172.21.15.89:0/2624909842 shutdown_connections 2024-09-10T08:37:20.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.049+0000 7fbb49a75740 1 -- 172.21.15.89:0/2624909842 wait complete. 2024-09-10T08:37:20.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.050+0000 7fbb3fe00640 1 --2- 172.21.15.89:0/1984048067 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbb240a2c00 0x7fbb240a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55a430a378d0 tx=0x7fbb30005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:20.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.050+0000 7fbb49a75740 1 -- start start 2024-09-10T08:37:20.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.050+0000 7fbb49a75740 1 Processor -- start 2024-09-10T08:37:20.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.050+0000 7fbb49a75740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a430a3a4e0 0x55a430a425b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.050+0000 7fbb49a75740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a3ae00 0x55a430a42af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.050+0000 7fbb49a75740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a456c0 0x55a430a43030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.050+0000 7fbb49a75740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a430a48330 con 0x55a430a3ae00 2024-09-10T08:37:20.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.050+0000 7fbb49a75740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a430928920 con 0x55a430a456c0 2024-09-10T08:37:20.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.050+0000 7fbb49a75740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a430a484b0 con 0x55a430a3a4e0 2024-09-10T08:37:20.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.051+0000 7fbb49a75740 10 client.0 _ll_register_callbacks cb 0x55a43094c8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:37:20.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.051+0000 7fbb3fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a430a3a4e0 0x55a430a425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:20.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.051+0000 7fbb3f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a3ae00 0x55a430a42af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:20.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.051+0000 7fbb3fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a430a3a4e0 0x55a430a425b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:55960/0 (socket says 172.21.15.89:55960) 2024-09-10T08:37:20.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.051+0000 7fbb3fe00640 1 -- 172.21.15.89:0/829171614 learned_addr learned my addr 172.21.15.89:0/829171614 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:20.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.051+0000 7fbb45000640 1 --2- 172.21.15.89:0/829171614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a456c0 0x55a430a43030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:20.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.051+0000 7fbb3f400640 1 -- 172.21.15.89:0/829171614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a430a3a4e0 msgr2=0x55a430a425b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:37:20.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.051+0000 7fbb3f400640 1 --2- 172.21.15.89:0/829171614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a430a3a4e0 0x55a430a425b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.051+0000 7fbb3f400640 1 -- 172.21.15.89:0/829171614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a456c0 msgr2=0x55a430a43030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:37:20.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.051+0000 7fbb3f400640 1 --2- 172.21.15.89:0/829171614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a456c0 0x55a430a43030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.051+0000 7fbb3f400640 1 -- 172.21.15.89:0/829171614 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4307112b0 con 0x55a430a3ae00 2024-09-10T08:37:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.051+0000 7fbb45000640 1 --2- 172.21.15.89:0/829171614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a456c0 0x55a430a43030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:37:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.051+0000 7fbb3f400640 1 --2- 172.21.15.89:0/829171614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a3ae00 0x55a430a42af0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0ccd90 tx=0x7fbb2c09c220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ba70447d5b4a5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.051+0000 7fbb2a000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:37:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.052+0000 7fbb2a000640 1 -- 172.21.15.89:0/829171614 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c018d30 con 0x55a430a3ae00 2024-09-10T08:37:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.052+0000 7fbb49a75740 1 -- 172.21.15.89:0/829171614 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb38036040 con 0x55a430a3ae00 2024-09-10T08:37:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.052+0000 7fbb2a000640 1 -- 172.21.15.89:0/829171614 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fbb2c09af70 con 0x55a430a3ae00 2024-09-10T08:37:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.052+0000 7fbb2a000640 1 -- 172.21.15.89:0/829171614 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fbb2c09bb20 con 0x55a430a3ae00 2024-09-10T08:37:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.052+0000 7fbb49a75740 10 client.14598 fetch_fsmap learned FSMap version 17 2024-09-10T08:37:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.052+0000 7fbb49a75740 1 -- 172.21.15.89:0/829171614 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55a430a54030 con 0x55a430a3ae00 2024-09-10T08:37:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.052+0000 7fbb2a000640 1 -- 172.21.15.89:0/829171614 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb38036040 con 0x55a430a3ae00 2024-09-10T08:37:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.052+0000 7fbb2a000640 1 -- 172.21.15.89:0/829171614 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fbb2c097900 con 0x55a430a3ae00 2024-09-10T08:37:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.053+0000 7fbb49a75740 10 client.14598 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T08:37:20.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.053+0000 7fbb49a75740 10 client.14598 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:37:20.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.053+0000 7fbb49a75740 20 client.14598 populate_metadata read hostname 'smithi089' 2024-09-10T08:37:20.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.053+0000 7fbb49a75740 1 --2- 172.21.15.89:0/829171614 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a430a56540 0x55a430a579b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.053+0000 7fbb49a75740 4 client.14598 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:37:20.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.053+0000 7fbb49a75740 1 -- 172.21.15.89:0/829171614 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4309ad620 con 0x55a430a56540 2024-09-10T08:37:20.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.053+0000 7fbb3fe00640 1 --2- 172.21.15.89:0/829171614 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a430a56540 0x55a430a579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:20.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.054+0000 7fbb3fe00640 1 --2- 172.21.15.89:0/829171614 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a430a56540 0x55a430a579b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d566be36c30c8c49 server_cookie=6603ee540019912b in_seq=0 out_seq=0 2024-09-10T08:37:20.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.054+0000 7fbb2a000640 10 client.14598 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:37:20.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.054+0000 7fbb2a000640 1 -- 172.21.15.89:0/829171614 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a4309ad620 con 0x55a430a56540 2024-09-10T08:37:20.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.054+0000 7fbb2a000640 10 client.14598 handle_command_reply: tid=0 2024-09-10T08:37:20.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.054+0000 7fbb49a75740 1 client.14598 shutdown 2024-09-10T08:37:20.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.055+0000 7fbb49a75740 1 -- 172.21.15.89:0/829171614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a3ae00 msgr2=0x55a430a42af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:20.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.055+0000 7fbb49a75740 1 --2- 172.21.15.89:0/829171614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a3ae00 0x55a430a42af0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0ccd90 tx=0x7fbb2c09c220 comp rx=0 tx=0).stop 2024-09-10T08:37:20.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.055+0000 7fbb49a75740 1 -- 172.21.15.89:0/829171614 shutdown_connections 2024-09-10T08:37:20.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.055+0000 7fbb49a75740 1 --2- 172.21.15.89:0/829171614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a456c0 0x55a430a43030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.055+0000 7fbb49a75740 1 --2- 172.21.15.89:0/829171614 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a430a56540 0x55a430a579b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.055+0000 7fbb49a75740 1 --2- 172.21.15.89:0/829171614 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a3ae00 0x55a430a42af0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.055+0000 7fbb49a75740 1 --2- 172.21.15.89:0/829171614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a430a3a4e0 0x55a430a425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.055+0000 7fbb2a000640 0 client.14598 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:37:20.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.055+0000 7fbb49a75740 1 -- 172.21.15.89:0/829171614 >> 172.21.15.89:0/829171614 conn(0x55a430a36a70 msgr2=0x55a430a3c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:20.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.055+0000 7fbb49a75740 1 -- 172.21.15.89:0/829171614 shutdown_connections 2024-09-10T08:37:20.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.055+0000 7fbb49a75740 1 -- 172.21.15.89:0/829171614 wait complete. 2024-09-10T08:37:20.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.055+0000 7fbb49a75740 20 client.14598 trim_cache size 0 max 16384 2024-09-10T08:37:20.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.073+0000 7fbb49a75740 1 Processor -- start 2024-09-10T08:37:20.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.073+0000 7fbb49a75740 1 -- start start 2024-09-10T08:37:20.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.074+0000 7fbb49a75740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a430a3a4e0 0x55a430a40e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.074+0000 7fbb49a75740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a3ae00 0x55a430a3f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.074+0000 7fbb49a75740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a456c0 0x55a430a3fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.074+0000 7fbb49a75740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a430a55cf0 con 0x55a430a3ae00 2024-09-10T08:37:20.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.074+0000 7fbb49a75740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a430a563c0 con 0x55a430a456c0 2024-09-10T08:37:20.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.074+0000 7fbb49a75740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a430a56a90 con 0x55a430a3a4e0 2024-09-10T08:37:20.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.074+0000 7fbb3f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a3ae00 0x55a430a3f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:20.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.074+0000 7fbb3f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a3ae00 0x55a430a3f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39408/0 (socket says 172.21.15.89:39408) 2024-09-10T08:37:20.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.074+0000 7fbb3f400640 1 -- 172.21.15.89:0/1612755548 learned_addr learned my addr 172.21.15.89:0/1612755548 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:20.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.074+0000 7fbb45000640 1 --2- 172.21.15.89:0/1612755548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a456c0 0x55a430a3fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:20.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.074+0000 7fbb3fe00640 1 --2- 172.21.15.89:0/1612755548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a430a3a4e0 0x55a430a40e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:20.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.074+0000 7fbb3f400640 1 -- 172.21.15.89:0/1612755548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a430a3a4e0 msgr2=0x55a430a40e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:37:20.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.074+0000 7fbb3f400640 1 --2- 172.21.15.89:0/1612755548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a430a3a4e0 0x55a430a40e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.074+0000 7fbb3f400640 1 -- 172.21.15.89:0/1612755548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a456c0 msgr2=0x55a430a3fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:37:20.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.074+0000 7fbb3f400640 1 --2- 172.21.15.89:0/1612755548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a456c0 0x55a430a3fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.074+0000 7fbb3f400640 1 -- 172.21.15.89:0/1612755548 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a430a44f20 con 0x55a430a3ae00 2024-09-10T08:37:20.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.074+0000 7fbb45000640 1 --2- 172.21.15.89:0/1612755548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a456c0 0x55a430a3fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:37:20.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.075+0000 7fbb3f400640 1 --2- 172.21.15.89:0/1612755548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a3ae00 0x55a430a3f810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c09b820 tx=0x7fbb2c009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188c0735c02b1271 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:20.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.075+0000 7fbb34c00640 1 -- 172.21.15.89:0/1612755548 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c09fed0 con 0x55a430a3ae00 2024-09-10T08:37:20.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.075+0000 7fbb34c00640 1 -- 172.21.15.89:0/1612755548 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb2c09ce50 con 0x55a430a3ae00 2024-09-10T08:37:20.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.075+0000 7fbb34c00640 1 -- 172.21.15.89:0/1612755548 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c09af70 con 0x55a430a3ae00 2024-09-10T08:37:20.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.076+0000 7fbb49a75740 1 -- 172.21.15.89:0/1612755548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a3ae00 msgr2=0x55a430a3f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:20.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.076+0000 7fbb49a75740 1 --2- 172.21.15.89:0/1612755548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a3ae00 0x55a430a3f810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c09b820 tx=0x7fbb2c009770 comp rx=0 tx=0).stop 2024-09-10T08:37:20.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.076+0000 7fbb49a75740 1 -- 172.21.15.89:0/1612755548 shutdown_connections 2024-09-10T08:37:20.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.076+0000 7fbb49a75740 1 --2- 172.21.15.89:0/1612755548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a456c0 0x55a430a3fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.076+0000 7fbb49a75740 1 --2- 172.21.15.89:0/1612755548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a3ae00 0x55a430a3f810 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.076+0000 7fbb49a75740 1 --2- 172.21.15.89:0/1612755548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a430a3a4e0 0x55a430a40e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.076+0000 7fbb49a75740 1 -- 172.21.15.89:0/1612755548 >> 172.21.15.89:0/1612755548 conn(0x55a430a36a70 msgr2=0x55a430a36e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:20.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.076+0000 7fbb49a75740 1 -- 172.21.15.89:0/1612755548 shutdown_connections 2024-09-10T08:37:20.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.076+0000 7fbb49a75740 1 -- 172.21.15.89:0/1612755548 wait complete. 2024-09-10T08:37:20.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.076+0000 7fbb49a75740 1 -- start start 2024-09-10T08:37:20.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.076+0000 7fbb49a75740 1 Processor -- start 2024-09-10T08:37:20.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.077+0000 7fbb49a75740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a3a4e0 0x55a430a59c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.077+0000 7fbb49a75740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a430a3ae00 0x55a430a5a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.077+0000 7fbb49a75740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a456c0 0x55a430a5a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.077+0000 7fbb49a75740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a430a57830 con 0x55a430a456c0 2024-09-10T08:37:20.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.077+0000 7fbb49a75740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a430a57160 con 0x55a430a3a4e0 2024-09-10T08:37:20.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.077+0000 7fbb49a75740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a430a57f00 con 0x55a430a3ae00 2024-09-10T08:37:20.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.077+0000 7fbb49a75740 10 client.0 _ll_register_callbacks cb 0x7fbb2c09b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:37:20.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.077+0000 7fbb45000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a456c0 0x55a430a5a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:20.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.077+0000 7fbb3fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a3a4e0 0x55a430a59c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:20.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.077+0000 7fbb45000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a456c0 0x55a430a5a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39416/0 (socket says 172.21.15.89:39416) 2024-09-10T08:37:20.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.077+0000 7fbb45000640 1 -- 172.21.15.89:0/2055478434 learned_addr learned my addr 172.21.15.89:0/2055478434 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:20.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.077+0000 7fbb3f400640 1 --2- 172.21.15.89:0/2055478434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a430a3ae00 0x55a430a5a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:20.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.077+0000 7fbb45000640 1 -- 172.21.15.89:0/2055478434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a430a3ae00 msgr2=0x55a430a5a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:37:20.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.077+0000 7fbb45000640 1 --2- 172.21.15.89:0/2055478434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a430a3ae00 0x55a430a5a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.077+0000 7fbb45000640 1 -- 172.21.15.89:0/2055478434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a3a4e0 msgr2=0x55a430a59c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:37:20.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.077+0000 7fbb45000640 1 --2- 172.21.15.89:0/2055478434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a3a4e0 0x55a430a59c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.078+0000 7fbb45000640 1 -- 172.21.15.89:0/2055478434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a430a5e6d0 con 0x55a430a456c0 2024-09-10T08:37:20.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.078+0000 7fbb3fe00640 1 --2- 172.21.15.89:0/2055478434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a3a4e0 0x55a430a59c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:37:20.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.078+0000 7fbb45000640 1 --2- 172.21.15.89:0/2055478434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a456c0 0x55a430a5a6e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009910 tx=0x7fbb380314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c724df7d716d4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:20.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.078+0000 7fbb2a000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:37:20.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.078+0000 7fbb2a000640 1 -- 172.21.15.89:0/2055478434 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb38053060 con 0x55a430a456c0 2024-09-10T08:37:20.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.078+0000 7fbb2a000640 1 -- 172.21.15.89:0/2055478434 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fbb380318d0 con 0x55a430a456c0 2024-09-10T08:37:20.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.078+0000 7fbb49a75740 1 -- 172.21.15.89:0/2055478434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb2c09ce50 con 0x55a430a456c0 2024-09-10T08:37:20.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.078+0000 7fbb2a000640 1 -- 172.21.15.89:0/2055478434 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fbb3805a050 con 0x55a430a456c0 2024-09-10T08:37:20.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.078+0000 7fbb49a75740 10 client.14604 fetch_fsmap learned FSMap version 17 2024-09-10T08:37:20.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.078+0000 7fbb49a75740 1 -- 172.21.15.89:0/2055478434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55a430ac11c0 con 0x55a430a456c0 2024-09-10T08:37:20.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.079+0000 7fbb2a000640 1 -- 172.21.15.89:0/2055478434 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb2c09ce50 con 0x55a430a456c0 2024-09-10T08:37:20.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.079+0000 7fbb2a000640 1 -- 172.21.15.89:0/2055478434 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fbb380046b0 con 0x55a430a456c0 2024-09-10T08:37:20.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.079+0000 7fbb49a75740 10 client.14604 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T08:37:20.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.079+0000 7fbb49a75740 10 client.14604 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:37:20.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.079+0000 7fbb49a75740 20 client.14604 populate_metadata read hostname 'smithi089' 2024-09-10T08:37:20.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.079+0000 7fbb49a75740 1 --2- 172.21.15.89:0/2055478434 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a430ac5970 0x55a430ac4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.079+0000 7fbb49a75740 4 client.14604 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:37:20.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.079+0000 7fbb49a75740 1 -- 172.21.15.89:0/2055478434 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "status"}) -- 0x55a4309a7b30 con 0x55a430ac5970 2024-09-10T08:37:20.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.079+0000 7fbb3fe00640 1 --2- 172.21.15.89:0/2055478434 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a430ac5970 0x55a430ac4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:20.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.080+0000 7fbb3fe00640 1 --2- 172.21.15.89:0/2055478434 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a430ac5970 0x55a430ac4680 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cf394810368563b server_cookie=ba61fb6f83ff891a in_seq=0 out_seq=0 2024-09-10T08:37:20.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.080+0000 7fbb2a000640 10 client.14604 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:37:20.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.080+0000 7fbb2a000640 1 -- 172.21.15.89:0/2055478434 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a4309a7b30 con 0x55a430ac5970 2024-09-10T08:37:20.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.080+0000 7fbb2a000640 10 client.14604 handle_command_reply: tid=0 2024-09-10T08:37:20.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.080+0000 7fbb49a75740 1 client.14604 shutdown 2024-09-10T08:37:20.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.080+0000 7fbb49a75740 1 -- 172.21.15.89:0/2055478434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a456c0 msgr2=0x55a430a5a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:20.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.080+0000 7fbb49a75740 1 --2- 172.21.15.89:0/2055478434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a456c0 0x55a430a5a6e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009910 tx=0x7fbb380314e0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.080+0000 7fbb49a75740 1 -- 172.21.15.89:0/2055478434 shutdown_connections 2024-09-10T08:37:20.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.080+0000 7fbb49a75740 1 --2- 172.21.15.89:0/2055478434 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a430ac5970 0x55a430ac4680 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.080+0000 7fbb49a75740 1 --2- 172.21.15.89:0/2055478434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a430a456c0 0x55a430a5a6e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.080+0000 7fbb49a75740 1 --2- 172.21.15.89:0/2055478434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a430a3ae00 0x55a430a5a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.080+0000 7fbb49a75740 1 --2- 172.21.15.89:0/2055478434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a430a3a4e0 0x55a430a59c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.080+0000 7fbb49a75740 1 -- 172.21.15.89:0/2055478434 >> 172.21.15.89:0/2055478434 conn(0x55a430a36a70 msgr2=0x55a430a40a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:20.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.080+0000 7fbb2a000640 0 client.14604 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:37:20.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.080+0000 7fbb49a75740 1 -- 172.21.15.89:0/2055478434 shutdown_connections 2024-09-10T08:37:20.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.080+0000 7fbb49a75740 1 -- 172.21.15.89:0/2055478434 wait complete. 2024-09-10T08:37:20.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.081+0000 7fbb49a75740 20 client.14604 trim_cache size 0 max 16384 2024-09-10T08:37:20.087 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:37:20.087 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "91f747bc-6f4e-11ef-bcea-c7b262605968", 2024-09-10T08:37:20.087 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2024-09-10T08:37:20.088 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24554, 2024-09-10T08:37:20.088 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2024-09-10T08:37:20.088 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2024-09-10T08:37:20.088 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2024-09-10T08:37:20.088 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 21.933398669999999, 2024-09-10T08:37:20.088 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 17, 2024-09-10T08:37:20.088 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 92, 2024-09-10T08:37:20.088 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 92, 2024-09-10T08:37:20.088 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 34.946870701000002 2024-09-10T08:37:20.088 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:37:20.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.081+0000 7fbb45a00640 1 -- 172.21.15.89:0/1984048067 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbb240a2c00 msgr2=0x7fbb240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:20.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.081+0000 7fbb45a00640 1 --2- 172.21.15.89:0/1984048067 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbb240a2c00 0x7fbb240a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55a430a378d0 tx=0x7fbb30005f90 comp rx=0 tx=0).stop 2024-09-10T08:37:20.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.081+0000 7fbb45a00640 1 -- 172.21.15.89:0/1984048067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb401047e0 msgr2=0x7fbb4019a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:20.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.081+0000 7fbb45a00640 1 --2- 172.21.15.89:0/1984048067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb401047e0 0x7fbb4019a950 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00e9f0 tx=0x7fbb2c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.081+0000 7fbb45a00640 1 -- 172.21.15.89:0/1984048067 shutdown_connections 2024-09-10T08:37:20.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.081+0000 7fbb45a00640 1 --2- 172.21.15.89:0/1984048067 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbb240a2c00 0x7fbb240a50c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.081+0000 7fbb45a00640 1 --2- 172.21.15.89:0/1984048067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb40108f60 0x7fbb4019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.081+0000 7fbb45a00640 1 --2- 172.21.15.89:0/1984048067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb401047e0 0x7fbb4019a950 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.081+0000 7fbb45a00640 1 --2- 172.21.15.89:0/1984048067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb40103e30 0x7fbb4019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.081+0000 7fbb45a00640 1 -- 172.21.15.89:0/1984048067 >> 172.21.15.89:0/1984048067 conn(0x7fbb400ff8d0 msgr2=0x7fbb40101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:20.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.082+0000 7fbb45a00640 1 -- 172.21.15.89:0/1984048067 shutdown_connections 2024-09-10T08:37:20.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:20.082+0000 7fbb45a00640 1 -- 172.21.15.89:0/1984048067 wait complete. 2024-09-10T08:37:20.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:20 smithi111 ceph-mon[35220]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:20.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:20 smithi089 ceph-mon[30757]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:20.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:20 smithi149 ceph-mon[36167]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:20.964 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T08:37:20.964 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:37:20.965 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T08:37:21.333 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:21.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.881+0000 7f3300200640 1 -- 172.21.15.89:0/3312130638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f8101de0 msgr2=0x7f32f81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:21.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.881+0000 7f3300200640 1 --2- 172.21.15.89:0/3312130638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f8101de0 0x7f32f81021c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009ca0 tx=0x7f32ec02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:37:21.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.882+0000 7f3300200640 1 -- 172.21.15.89:0/3312130638 shutdown_connections 2024-09-10T08:37:21.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.882+0000 7f3300200640 1 --2- 172.21.15.89:0/3312130638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f8106f10 0x7f32f810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:21.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.882+0000 7f3300200640 1 --2- 172.21.15.89:0/3312130638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32f8102790 0x7f32f81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:21.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.882+0000 7f3300200640 1 --2- 172.21.15.89:0/3312130638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f8101de0 0x7f32f81021c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:21.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.882+0000 7f3300200640 1 -- 172.21.15.89:0/3312130638 >> 172.21.15.89:0/3312130638 conn(0x7f32f80fd8c0 msgr2=0x7f32f80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:21.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.882+0000 7f3300200640 1 -- 172.21.15.89:0/3312130638 shutdown_connections 2024-09-10T08:37:21.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.882+0000 7f3300200640 1 -- 172.21.15.89:0/3312130638 wait complete. 2024-09-10T08:37:21.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.882+0000 7f3300200640 1 Processor -- start 2024-09-10T08:37:21.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.883+0000 7f3300200640 1 -- start start 2024-09-10T08:37:21.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.883+0000 7f3300200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f8101de0 0x7f32f819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:21.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.883+0000 7f3300200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32f8102790 0x7f32f819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:21.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.883+0000 7f3300200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f8106f10 0x7f32f819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:21.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.883+0000 7f3300200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f32f8111f80 con 0x7f32f8101de0 2024-09-10T08:37:21.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.883+0000 7f3300200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f32f8111e00 con 0x7f32f8102790 2024-09-10T08:37:21.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.883+0000 7f3300200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32f8112100 con 0x7f32f8106f10 2024-09-10T08:37:21.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.883+0000 7f32fd800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f8101de0 0x7f32f819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:21.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.883+0000 7f32fd800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f8101de0 0x7f32f819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39422/0 (socket says 172.21.15.89:39422) 2024-09-10T08:37:21.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.883+0000 7f32fd800640 1 -- 172.21.15.89:0/3936742083 learned_addr learned my addr 172.21.15.89:0/3936742083 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:21.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.883+0000 7f32fce00640 1 --2- 172.21.15.89:0/3936742083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32f8102790 0x7f32f819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:21.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.883+0000 7f32fe200640 1 --2- 172.21.15.89:0/3936742083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f8106f10 0x7f32f819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:21.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.884+0000 7f32fce00640 1 -- 172.21.15.89:0/3936742083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f8106f10 msgr2=0x7f32f819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:21.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.884+0000 7f32fce00640 1 --2- 172.21.15.89:0/3936742083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f8106f10 0x7f32f819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:21.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.884+0000 7f32fce00640 1 -- 172.21.15.89:0/3936742083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f8101de0 msgr2=0x7f32f819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:21.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.884+0000 7f32fce00640 1 --2- 172.21.15.89:0/3936742083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f8101de0 0x7f32f819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:21.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.884+0000 7f32fce00640 1 -- 172.21.15.89:0/3936742083 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f819f640 con 0x7f32f8102790 2024-09-10T08:37:21.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.884+0000 7f32fce00640 1 --2- 172.21.15.89:0/3936742083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32f8102790 0x7f32f819abd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f32e400e9f0 tx=0x7f32e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:21.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.884+0000 7f32f6000640 1 -- 172.21.15.89:0/3936742083 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e400ccc0 con 0x7f32f8102790 2024-09-10T08:37:21.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.884+0000 7f3300200640 1 -- 172.21.15.89:0/3936742083 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f819f930 con 0x7f32f8102790 2024-09-10T08:37:21.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.884+0000 7f32f6000640 1 -- 172.21.15.89:0/3936742083 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32e4010910 con 0x7f32f8102790 2024-09-10T08:37:21.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.884+0000 7f32f6000640 1 -- 172.21.15.89:0/3936742083 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e4018960 con 0x7f32f8102790 2024-09-10T08:37:21.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.885+0000 7f3300200640 1 -- 172.21.15.89:0/3936742083 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f81a7210 con 0x7f32f8102790 2024-09-10T08:37:21.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.886+0000 7f3300200640 1 -- 172.21.15.89:0/3936742083 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f81021c0 con 0x7f32f8102790 2024-09-10T08:37:21.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.888+0000 7f32f6000640 1 -- 172.21.15.89:0/3936742083 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f32e4010ac0 con 0x7f32f8102790 2024-09-10T08:37:21.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.889+0000 7f32f6000640 1 --2- 172.21.15.89:0/3936742083 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f32e00a2b80 0x7f32e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:21.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.889+0000 7f32f6000640 1 -- 172.21.15.89:0/3936742083 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f32e40cf9f0 con 0x7f32f8102790 2024-09-10T08:37:21.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.889+0000 7f32fd800640 1 --2- 172.21.15.89:0/3936742083 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f32e00a2b80 0x7f32e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:21.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.890+0000 7f32fd800640 1 --2- 172.21.15.89:0/3936742083 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f32e00a2b80 0x7f32e00a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009a60 tx=0x7f32ec005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:21.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:21.891+0000 7f32f6000640 1 -- 172.21.15.89:0/3936742083 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32e409c060 con 0x7f32f8102790 2024-09-10T08:37:22.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:22.100+0000 7f3300200640 1 -- 172.21.15.89:0/3936742083 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f32f81a7540 con 0x7f32f8102790 2024-09-10T08:37:22.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:22.101+0000 7f32f6000640 1 -- 172.21.15.89:0/3936742083 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f32e40a0360 con 0x7f32f8102790 2024-09-10T08:37:22.101 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T08:37:22.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:22.103+0000 7f3300200640 1 -- 172.21.15.89:0/3936742083 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f32e00a2b80 msgr2=0x7f32e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:22.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:22.103+0000 7f3300200640 1 --2- 172.21.15.89:0/3936742083 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f32e00a2b80 0x7f32e00a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009a60 tx=0x7f32ec005d80 comp rx=0 tx=0).stop 2024-09-10T08:37:22.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:22.103+0000 7f3300200640 1 -- 172.21.15.89:0/3936742083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32f8102790 msgr2=0x7f32f819abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:22.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:22.103+0000 7f3300200640 1 --2- 172.21.15.89:0/3936742083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32f8102790 0x7f32f819abd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f32e400e9f0 tx=0x7f32e400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:37:22.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:22.103+0000 7f3300200640 1 -- 172.21.15.89:0/3936742083 shutdown_connections 2024-09-10T08:37:22.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:22.103+0000 7f3300200640 1 --2- 172.21.15.89:0/3936742083 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f32e00a2b80 0x7f32e00a5040 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:22.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:22.103+0000 7f3300200640 1 --2- 172.21.15.89:0/3936742083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f8106f10 0x7f32f819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:22.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:22.103+0000 7f3300200640 1 --2- 172.21.15.89:0/3936742083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32f8102790 0x7f32f819abd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:22.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:22.103+0000 7f3300200640 1 --2- 172.21.15.89:0/3936742083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f8101de0 0x7f32f819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:22.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:22.103+0000 7f3300200640 1 -- 172.21.15.89:0/3936742083 >> 172.21.15.89:0/3936742083 conn(0x7f32f80fd8c0 msgr2=0x7f32f80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:22.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:22.103+0000 7f3300200640 1 -- 172.21.15.89:0/3936742083 shutdown_connections 2024-09-10T08:37:22.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:22.103+0000 7f3300200640 1 -- 172.21.15.89:0/3936742083 wait complete. 2024-09-10T08:37:22.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:22 smithi089 ceph-mon[30757]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:22.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:22 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3936742083' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T08:37:22.899 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T08:37:22.899 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T08:37:22.899 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2024-09-10T08:37:22.899 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-10T08:37:22.899 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:37:22.899 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T08:37:22.899 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T08:37:22.899 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T08:37:22.900 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T08:37:22.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:37:22.900 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2024-09-10T08:37:22.935 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T08:37:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T08:37:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T08:37:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2024-09-10T08:37:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2024-09-10T08:37:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2024-09-10T08:37:22.936 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T08:37:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2024-09-10T08:37:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2024-09-10T08:37:22.936 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T08:37:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2024-09-10T08:37:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2024-09-10T08:37:22.936 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T08:37:22.936 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2024-09-10T08:37:22.937 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T08:37:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2024-09-10T08:37:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T08:37:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42225sec preferred_lft 42225sec 2024-09-10T08:37:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link noprefixroute 2024-09-10T08:37:22.937 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2024-09-10T08:37:22.938 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T08:37:22.938 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:37:22.938 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2024-09-10T08:37:22.938 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add name ceph-brx type bridge 2024-09-10T08:37:22.938 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip addr flush dev ceph-brx 2024-09-10T08:37:22.938 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set ceph-brx up 2024-09-10T08:37:22.938 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T08:37:22.938 DEBUG:teuthology.orchestra.run.smithi089:> ') 2024-09-10T08:37:22.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:22 smithi149 ceph-mon[36167]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:22.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:22 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3936742083' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T08:37:23.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:22 smithi111 ceph-mon[35220]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:23.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:22 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3936742083' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T08:37:23.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:37:23.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:37:23.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:37:23.348 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T08:37:23.406 INFO:teuthology.orchestra.run.smithi089.stdout:1 2024-09-10T08:37:23.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:37:23.409 DEBUG:teuthology.orchestra.run.smithi089:> route 2024-09-10T08:37:23.464 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2024-09-10T08:37:23.464 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T08:37:23.464 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T08:37:23.464 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T08:37:23.464 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T08:37:23.464 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:37:23.465 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2024-09-10T08:37:23.465 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T08:37:23.465 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T08:37:23.465 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T08:37:23.465 DEBUG:teuthology.orchestra.run.smithi089:> ') 2024-09-10T08:37:23.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:37:24.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:37:24.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:37:24.102 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2024-09-10T08:37:24.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:37:24.146 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list-id 2024-09-10T08:37:24.181 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:37:24.182 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2024-09-10T08:37:24.182 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:37:24.182 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-10T08:37:24.182 DEBUG:teuthology.orchestra.run.smithi089:> ') 2024-09-10T08:37:24.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:37:24.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:37:24.362 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-10T08:37:24.362 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:37:24.362 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2024-09-10T08:37:24.362 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-10T08:37:24.362 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T08:37:24.362 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-10T08:37:24.363 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-10T08:37:24.363 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-10T08:37:24.363 DEBUG:teuthology.orchestra.run.smithi089:> ') 2024-09-10T08:37:24.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:37:24.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:37:24.606 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:37:24.606 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2024-09-10T08:37:24.606 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2024-09-10T08:37:24.606 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T08:37:24.606 DEBUG:teuthology.orchestra.run.smithi089:> ') 2024-09-10T08:37:24.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:37:24.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:37:24.773 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-10T08:37:24.774 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T08:37:24.774 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:37:24.811 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T08:37:24.811 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T08:37:24.812 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:37:24.886 DEBUG:teuthology.orchestra.run.smithi089:> sudo modprobe fuse 2024-09-10T08:37:24.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:24 smithi149 ceph-mon[36167]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:24.975 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:24 smithi089 ceph-mon[30757]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:24.985 DEBUG:teuthology.orchestra.run.smithi089:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T08:37:25.021 INFO:teuthology.orchestra.run.smithi089.stdout:/proc 2024-09-10T08:37:25.021 INFO:teuthology.orchestra.run.smithi089.stdout:/sys 2024-09-10T08:37:25.021 INFO:teuthology.orchestra.run.smithi089.stdout:/dev 2024-09-10T08:37:25.021 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/security 2024-09-10T08:37:25.021 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/shm 2024-09-10T08:37:25.021 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/pts 2024-09-10T08:37:25.021 INFO:teuthology.orchestra.run.smithi089.stdout:/run 2024-09-10T08:37:25.021 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/cgroup 2024-09-10T08:37:25.022 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/pstore 2024-09-10T08:37:25.022 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/bpf 2024-09-10T08:37:25.022 INFO:teuthology.orchestra.run.smithi089.stdout:/ 2024-09-10T08:37:25.022 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/selinux 2024-09-10T08:37:25.022 INFO:teuthology.orchestra.run.smithi089.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:37:25.022 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/hugepages 2024-09-10T08:37:25.022 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/mqueue 2024-09-10T08:37:25.022 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/debug 2024-09-10T08:37:25.022 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/tracing 2024-09-10T08:37:25.022 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/fuse/connections 2024-09-10T08:37:25.022 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/config 2024-09-10T08:37:25.022 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T08:37:25.022 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T08:37:25.022 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T08:37:25.023 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T08:37:25.023 INFO:teuthology.orchestra.run.smithi089.stdout:/run/user/1000 2024-09-10T08:37:25.023 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/ceph 2024-09-10T08:37:25.023 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay 2024-09-10T08:37:25.023 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/464d23ed4592244eabfddda86578290c50b18381cd0cb45cf1e4dd757c52f12c/merged 2024-09-10T08:37:25.023 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/bbaa2b19f14ede464ff8db828d4003f1e815414f6abfaa6f717bad4e8ad8270a/merged 2024-09-10T08:37:25.023 INFO:teuthology.orchestra.run.smithi089.stdout:/run/user/0 2024-09-10T08:37:25.023 INFO:teuthology.orchestra.run.smithi089.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:37:25.023 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/72d346c3b4a424b3e242ff25039cd7e291d75976e2f165dc92403af3f7579412/merged 2024-09-10T08:37:25.023 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/7ffeaf1e09c3ab61386ca16c7a5f623cc756860f136b7258fe877a099004e556/merged 2024-09-10T08:37:25.023 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/4efefdaf9407474ec9b1c6e4887ef2d5b2e0fc6e97288b72f9298c8ea4bc36ae/merged 2024-09-10T08:37:25.023 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/b2882445911fb66efb3ac24447b70137482a8705ee28adf3b3863dac3dadefbd/merged 2024-09-10T08:37:25.023 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/12ce0649b002510fc4c6d7dfdde716190dc4ef70092e8e69d142bc16e44af6af/merged 2024-09-10T08:37:25.024 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/5d17df5dbd72929c3b159ea0511a9d955baa44d0a646747cf5cded5118596a37/merged 2024-09-10T08:37:25.024 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/4eb5ed7db5a1053be0912a28ceb22facaeccfbad55567975c6ef122fc33d3413/merged 2024-09-10T08:37:25.024 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/4d89994cd12257862b7d66411bf9ef1698565d377ff446ba8766503e11c332c1/merged 2024-09-10T08:37:25.024 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns 2024-09-10T08:37:25.024 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:37:25.024 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:37:25.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:37:25.025 DEBUG:teuthology.orchestra.run.smithi089:> ls /sys/fs/fuse/connections 2024-09-10T08:37:25.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:24 smithi111 ceph-mon[35220]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:25.100 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T08:37:25.100 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-10T08:37:25.145 DEBUG:teuthology.orchestra.run.smithi089:> sudo modprobe fuse 2024-09-10T08:37:25.202 DEBUG:teuthology.orchestra.run.smithi089:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T08:37:25.239 INFO:teuthology.orchestra.run.smithi089.stdout:/proc 2024-09-10T08:37:25.239 INFO:teuthology.orchestra.run.smithi089.stdout:/sys 2024-09-10T08:37:25.239 INFO:teuthology.orchestra.run.smithi089.stdout:/dev 2024-09-10T08:37:25.239 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/security 2024-09-10T08:37:25.239 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/shm 2024-09-10T08:37:25.239 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/pts 2024-09-10T08:37:25.239 INFO:teuthology.orchestra.run.smithi089.stdout:/run 2024-09-10T08:37:25.239 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/cgroup 2024-09-10T08:37:25.239 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/pstore 2024-09-10T08:37:25.240 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/bpf 2024-09-10T08:37:25.240 INFO:teuthology.orchestra.run.smithi089.stdout:/ 2024-09-10T08:37:25.240 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/selinux 2024-09-10T08:37:25.240 INFO:teuthology.orchestra.run.smithi089.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:37:25.240 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/hugepages 2024-09-10T08:37:25.240 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/mqueue 2024-09-10T08:37:25.240 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/debug 2024-09-10T08:37:25.240 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/tracing 2024-09-10T08:37:25.240 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/fuse/connections 2024-09-10T08:37:25.240 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/config 2024-09-10T08:37:25.240 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T08:37:25.240 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T08:37:25.240 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T08:37:25.241 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T08:37:25.241 INFO:teuthology.orchestra.run.smithi089.stdout:/run/user/1000 2024-09-10T08:37:25.241 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/ceph 2024-09-10T08:37:25.241 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay 2024-09-10T08:37:25.241 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/464d23ed4592244eabfddda86578290c50b18381cd0cb45cf1e4dd757c52f12c/merged 2024-09-10T08:37:25.241 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/bbaa2b19f14ede464ff8db828d4003f1e815414f6abfaa6f717bad4e8ad8270a/merged 2024-09-10T08:37:25.241 INFO:teuthology.orchestra.run.smithi089.stdout:/run/user/0 2024-09-10T08:37:25.241 INFO:teuthology.orchestra.run.smithi089.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:37:25.241 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/72d346c3b4a424b3e242ff25039cd7e291d75976e2f165dc92403af3f7579412/merged 2024-09-10T08:37:25.241 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/7ffeaf1e09c3ab61386ca16c7a5f623cc756860f136b7258fe877a099004e556/merged 2024-09-10T08:37:25.241 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/4efefdaf9407474ec9b1c6e4887ef2d5b2e0fc6e97288b72f9298c8ea4bc36ae/merged 2024-09-10T08:37:25.241 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/b2882445911fb66efb3ac24447b70137482a8705ee28adf3b3863dac3dadefbd/merged 2024-09-10T08:37:25.241 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/12ce0649b002510fc4c6d7dfdde716190dc4ef70092e8e69d142bc16e44af6af/merged 2024-09-10T08:37:25.242 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/5d17df5dbd72929c3b159ea0511a9d955baa44d0a646747cf5cded5118596a37/merged 2024-09-10T08:37:25.242 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/4eb5ed7db5a1053be0912a28ceb22facaeccfbad55567975c6ef122fc33d3413/merged 2024-09-10T08:37:25.242 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/4d89994cd12257862b7d66411bf9ef1698565d377ff446ba8766503e11c332c1/merged 2024-09-10T08:37:25.242 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns 2024-09-10T08:37:25.242 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:37:25.242 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:37:25.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:37:25.243 DEBUG:teuthology.orchestra.run.smithi089:> ls /sys/fs/fuse/connections 2024-09-10T08:37:25.324 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi089.stderr:2024-09-10T08:37:25.323+0000 7f91456ef480 -1 init, newargv = 0x559d4b57e430 newargc=15 2024-09-10T08:37:25.324 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi089.stderr:ceph-fuse[74772]: starting ceph client 2024-09-10T08:37:25.333 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi089.stderr:ceph-fuse[74772]: starting fuse 2024-09-10T08:37:26.421 DEBUG:teuthology.orchestra.run.smithi089:> sudo modprobe fuse 2024-09-10T08:37:26.485 DEBUG:teuthology.orchestra.run.smithi089:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T08:37:26.523 INFO:teuthology.orchestra.run.smithi089.stdout:/proc 2024-09-10T08:37:26.523 INFO:teuthology.orchestra.run.smithi089.stdout:/sys 2024-09-10T08:37:26.523 INFO:teuthology.orchestra.run.smithi089.stdout:/dev 2024-09-10T08:37:26.523 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/security 2024-09-10T08:37:26.523 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/shm 2024-09-10T08:37:26.523 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/pts 2024-09-10T08:37:26.524 INFO:teuthology.orchestra.run.smithi089.stdout:/run 2024-09-10T08:37:26.524 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/cgroup 2024-09-10T08:37:26.524 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/pstore 2024-09-10T08:37:26.524 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/bpf 2024-09-10T08:37:26.524 INFO:teuthology.orchestra.run.smithi089.stdout:/ 2024-09-10T08:37:26.524 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/selinux 2024-09-10T08:37:26.524 INFO:teuthology.orchestra.run.smithi089.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:37:26.524 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/hugepages 2024-09-10T08:37:26.524 INFO:teuthology.orchestra.run.smithi089.stdout:/dev/mqueue 2024-09-10T08:37:26.524 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/debug 2024-09-10T08:37:26.524 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/tracing 2024-09-10T08:37:26.524 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/fs/fuse/connections 2024-09-10T08:37:26.524 INFO:teuthology.orchestra.run.smithi089.stdout:/sys/kernel/config 2024-09-10T08:37:26.525 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T08:37:26.525 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T08:37:26.525 INFO:teuthology.orchestra.run.smithi089.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T08:37:26.525 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T08:37:26.525 INFO:teuthology.orchestra.run.smithi089.stdout:/run/user/1000 2024-09-10T08:37:26.525 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/ceph 2024-09-10T08:37:26.525 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay 2024-09-10T08:37:26.525 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/464d23ed4592244eabfddda86578290c50b18381cd0cb45cf1e4dd757c52f12c/merged 2024-09-10T08:37:26.525 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/bbaa2b19f14ede464ff8db828d4003f1e815414f6abfaa6f717bad4e8ad8270a/merged 2024-09-10T08:37:26.525 INFO:teuthology.orchestra.run.smithi089.stdout:/run/user/0 2024-09-10T08:37:26.525 INFO:teuthology.orchestra.run.smithi089.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:37:26.525 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/72d346c3b4a424b3e242ff25039cd7e291d75976e2f165dc92403af3f7579412/merged 2024-09-10T08:37:26.525 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/7ffeaf1e09c3ab61386ca16c7a5f623cc756860f136b7258fe877a099004e556/merged 2024-09-10T08:37:26.525 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/4efefdaf9407474ec9b1c6e4887ef2d5b2e0fc6e97288b72f9298c8ea4bc36ae/merged 2024-09-10T08:37:26.526 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/b2882445911fb66efb3ac24447b70137482a8705ee28adf3b3863dac3dadefbd/merged 2024-09-10T08:37:26.526 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/12ce0649b002510fc4c6d7dfdde716190dc4ef70092e8e69d142bc16e44af6af/merged 2024-09-10T08:37:26.526 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/5d17df5dbd72929c3b159ea0511a9d955baa44d0a646747cf5cded5118596a37/merged 2024-09-10T08:37:26.526 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/4eb5ed7db5a1053be0912a28ceb22facaeccfbad55567975c6ef122fc33d3413/merged 2024-09-10T08:37:26.526 INFO:teuthology.orchestra.run.smithi089.stdout:/var/lib/containers/storage/overlay/4d89994cd12257862b7d66411bf9ef1698565d377ff446ba8766503e11c332c1/merged 2024-09-10T08:37:26.526 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns 2024-09-10T08:37:26.526 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:37:26.526 INFO:teuthology.orchestra.run.smithi089.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:37:26.526 INFO:teuthology.orchestra.run.smithi089.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-10T08:37:26.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:37:26.527 DEBUG:teuthology.orchestra.run.smithi089:> ls /sys/fs/fuse/connections 2024-09-10T08:37:26.603 INFO:teuthology.orchestra.run.smithi089.stdout:120 2024-09-10T08:37:26.604 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-10T08:37:26.604 DEBUG:teuthology.orchestra.run.smithi089:> sudo stdin-killer -- python3 -c ' 2024-09-10T08:37:26.604 DEBUG:teuthology.orchestra.run.smithi089:> import glob 2024-09-10T08:37:26.604 DEBUG:teuthology.orchestra.run.smithi089:> import re 2024-09-10T08:37:26.604 DEBUG:teuthology.orchestra.run.smithi089:> import os 2024-09-10T08:37:26.604 DEBUG:teuthology.orchestra.run.smithi089:> import subprocess 2024-09-10T08:37:26.604 DEBUG:teuthology.orchestra.run.smithi089:> 2024-09-10T08:37:26.604 DEBUG:teuthology.orchestra.run.smithi089:> def _find_admin_socket(client_name): 2024-09-10T08:37:26.604 DEBUG:teuthology.orchestra.run.smithi089:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-10T08:37:26.604 DEBUG:teuthology.orchestra.run.smithi089:> files = glob.glob(asok_path) 2024-09-10T08:37:26.605 DEBUG:teuthology.orchestra.run.smithi089:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-10T08:37:26.605 DEBUG:teuthology.orchestra.run.smithi089:> 2024-09-10T08:37:26.605 DEBUG:teuthology.orchestra.run.smithi089:> # Given a non-glob path, it better be there 2024-09-10T08:37:26.605 DEBUG:teuthology.orchestra.run.smithi089:> if "*" not in asok_path: 2024-09-10T08:37:26.605 DEBUG:teuthology.orchestra.run.smithi089:> assert(len(files) == 1) 2024-09-10T08:37:26.605 DEBUG:teuthology.orchestra.run.smithi089:> return files[0] 2024-09-10T08:37:26.605 DEBUG:teuthology.orchestra.run.smithi089:> 2024-09-10T08:37:26.605 DEBUG:teuthology.orchestra.run.smithi089:> for f in files: 2024-09-10T08:37:26.605 DEBUG:teuthology.orchestra.run.smithi089:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T08:37:26.605 DEBUG:teuthology.orchestra.run.smithi089:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T08:37:26.605 DEBUG:teuthology.orchestra.run.smithi089:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T08:37:26.605 DEBUG:teuthology.orchestra.run.smithi089:> contents = proc_f.read() 2024-09-10T08:37:26.605 DEBUG:teuthology.orchestra.run.smithi089:> if mountpoint in contents: 2024-09-10T08:37:26.605 DEBUG:teuthology.orchestra.run.smithi089:> return f 2024-09-10T08:37:26.605 DEBUG:teuthology.orchestra.run.smithi089:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T08:37:26.606 DEBUG:teuthology.orchestra.run.smithi089:> 2024-09-10T08:37:26.606 DEBUG:teuthology.orchestra.run.smithi089:> print(_find_admin_socket("client.admin")) 2024-09-10T08:37:26.606 DEBUG:teuthology.orchestra.run.smithi089:> ' 2024-09-10T08:37:26.768 INFO:teuthology.orchestra.run.smithi089.stdout:/var/run/ceph/ceph-client.admin.74772.asok 2024-09-10T08:37:26.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:37:26.780 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:26 smithi089 ceph-mon[30757]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:26.783 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74772.asok 2024-09-10T08:37:26.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:37:26.783 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74772.asok status 2024-09-10T08:37:26.964 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:37:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: "metadata": { 2024-09-10T08:37:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T08:37:26.964 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T08:37:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "entity_id": "admin", 2024-09-10T08:37:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "hostname": "smithi089", 2024-09-10T08:37:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-10T08:37:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "pid": "74772", 2024-09-10T08:37:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "root": "/" 2024-09-10T08:37:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:37:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "dentry_count": 0, 2024-09-10T08:37:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "dentry_pinned_count": 0, 2024-09-10T08:37:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 14616, 2024-09-10T08:37:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "inst": { 2024-09-10T08:37:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "name": { 2024-09-10T08:37:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "client", 2024-09-10T08:37:26.965 INFO:teuthology.orchestra.run.smithi089.stdout: "num": 14616 2024-09-10T08:37:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:37:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": { 2024-09-10T08:37:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-09-10T08:37:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "192.168.0.1:0", 2024-09-10T08:37:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 600178120 2024-09-10T08:37:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:37:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:37:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": { 2024-09-10T08:37:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-09-10T08:37:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "192.168.0.1:0", 2024-09-10T08:37:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 600178120 2024-09-10T08:37:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:37:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "inst_str": "client.14616 192.168.0.1:0/600178120", 2024-09-10T08:37:26.966 INFO:teuthology.orchestra.run.smithi089.stdout: "addr_str": "192.168.0.1:0/600178120", 2024-09-10T08:37:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: "inode_count": 1, 2024-09-10T08:37:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_epoch": 17, 2024-09-10T08:37:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_epoch": 92, 2024-09-10T08:37:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_epoch_barrier": 0, 2024-09-10T08:37:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: "blocklisted": false, 2024-09-10T08:37:26.967 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs" 2024-09-10T08:37:26.967 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:37:26.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:26 smithi149 ceph-mon[36167]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:26.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:37:26.977 DEBUG:teuthology.orchestra.run.smithi089:> stat --file-system '--printf=%T 2024-09-10T08:37:26.977 DEBUG:teuthology.orchestra.run.smithi089:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:37:27.013 INFO:teuthology.orchestra.run.smithi089.stdout:fuseblk 2024-09-10T08:37:27.013 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:37:27.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:37:27.013 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:37:27.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:26 smithi111 ceph-mon[35220]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:27.110 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-10T08:37:27.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T08:37:27.212 INFO:teuthology.orchestra.run.smithi089.stdout:# file: . 2024-09-10T08:37:27.212 INFO:teuthology.orchestra.run.smithi089.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-10T08:37:27.212 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:37:27.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:37:27.220 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:37:27.220 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T08:37:27.261 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T08:37:27.662 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:28.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.266+0000 7fb9be600640 1 -- 172.21.15.89:0/1343951159 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b8102790 msgr2=0x7fb9b81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:28.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.266+0000 7fb9be600640 1 --2- 172.21.15.89:0/1343951159 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b8102790 0x7fb9b81067e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009900 tx=0x7fb9a002a410 comp rx=0 tx=0).stop 2024-09-10T08:37:28.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.266+0000 7fb9be600640 1 -- 172.21.15.89:0/1343951159 shutdown_connections 2024-09-10T08:37:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.266+0000 7fb9be600640 1 --2- 172.21.15.89:0/1343951159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9b8106f10 0x7fb9b810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.266+0000 7fb9be600640 1 --2- 172.21.15.89:0/1343951159 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b8102790 0x7fb9b81067e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.266+0000 7fb9be600640 1 --2- 172.21.15.89:0/1343951159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9b8101de0 0x7fb9b81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.266+0000 7fb9be600640 1 -- 172.21.15.89:0/1343951159 >> 172.21.15.89:0/1343951159 conn(0x7fb9b80fd8c0 msgr2=0x7fb9b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.266+0000 7fb9be600640 1 -- 172.21.15.89:0/1343951159 shutdown_connections 2024-09-10T08:37:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.266+0000 7fb9be600640 1 -- 172.21.15.89:0/1343951159 wait complete. 2024-09-10T08:37:28.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.267+0000 7fb9be600640 1 Processor -- start 2024-09-10T08:37:28.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.267+0000 7fb9be600640 1 -- start start 2024-09-10T08:37:28.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.267+0000 7fb9be600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b8101de0 0x7fb9b80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:28.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.267+0000 7fb9be600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9b8102790 0x7fb9b8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.267+0000 7fb9be600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9b8106f10 0x7fb9b8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.267+0000 7fb9be600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b8111f70 con 0x7fb9b8106f10 2024-09-10T08:37:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.267+0000 7fb9be600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b8111df0 con 0x7fb9b8101de0 2024-09-10T08:37:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.267+0000 7fb9be600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b81120f0 con 0x7fb9b8102790 2024-09-10T08:37:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.268+0000 7fb9b7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b8101de0 0x7fb9b80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.268+0000 7fb9b7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9b8106f10 0x7fb9b8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.268+0000 7fb9b7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b8101de0 0x7fb9b80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:58596/0 (socket says 172.21.15.89:58596) 2024-09-10T08:37:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.268+0000 7fb9b7400640 1 -- 172.21.15.89:0/3624338535 learned_addr learned my addr 172.21.15.89:0/3624338535 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.268+0000 7fb9aea00640 1 --2- 172.21.15.89:0/3624338535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9b8102790 0x7fb9b8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.268+0000 7fb9b7400640 1 -- 172.21.15.89:0/3624338535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9b8102790 msgr2=0x7fb9b8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.268+0000 7fb9b7400640 1 --2- 172.21.15.89:0/3624338535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9b8102790 0x7fb9b8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:28.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.268+0000 7fb9b7400640 1 -- 172.21.15.89:0/3624338535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9b8106f10 msgr2=0x7fb9b8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.268+0000 7fb9b7400640 1 --2- 172.21.15.89:0/3624338535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9b8106f10 0x7fb9b8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.268+0000 7fb9b7400640 1 -- 172.21.15.89:0/3624338535 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b81a68f0 con 0x7fb9b8101de0 2024-09-10T08:37:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.268+0000 7fb9aea00640 1 --2- 172.21.15.89:0/3624338535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9b8102790 0x7fb9b8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:37:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.268+0000 7fb9b7400640 1 --2- 172.21.15.89:0/3624338535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b8101de0 0x7fb9b80744f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800ea60 tx=0x7fb9a800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.269+0000 7fb9b4c00640 1 -- 172.21.15.89:0/3624338535 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a800cde0 con 0x7fb9b8101de0 2024-09-10T08:37:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.269+0000 7fb9b4c00640 1 -- 172.21.15.89:0/3624338535 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a80035f0 con 0x7fb9b8101de0 2024-09-10T08:37:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.269+0000 7fb9b4c00640 1 -- 172.21.15.89:0/3624338535 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8010670 con 0x7fb9b8101de0 2024-09-10T08:37:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.269+0000 7fb9be600640 1 -- 172.21.15.89:0/3624338535 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b81a6be0 con 0x7fb9b8101de0 2024-09-10T08:37:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.269+0000 7fb9be600640 1 -- 172.21.15.89:0/3624338535 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b81a70d0 con 0x7fb9b8101de0 2024-09-10T08:37:28.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.270+0000 7fb9be600640 1 -- 172.21.15.89:0/3624338535 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b81021c0 con 0x7fb9b8101de0 2024-09-10T08:37:28.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.272+0000 7fb9b4c00640 1 -- 172.21.15.89:0/3624338535 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb9a80037a0 con 0x7fb9b8101de0 2024-09-10T08:37:28.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.273+0000 7fb9b4c00640 1 --2- 172.21.15.89:0/3624338535 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb99c0a2b80 0x7fb99c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:28.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.273+0000 7fb9b4c00640 1 -- 172.21.15.89:0/3624338535 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fb9a80cf640 con 0x7fb9b8101de0 2024-09-10T08:37:28.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.273+0000 7fb9aea00640 1 --2- 172.21.15.89:0/3624338535 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb99c0a2b80 0x7fb99c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:28.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.274+0000 7fb9aea00640 1 --2- 172.21.15.89:0/3624338535 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb99c0a2b80 0x7fb99c0a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8075f80 tx=0x7fb9a0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:28.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.278+0000 7fb9b4c00640 1 -- 172.21.15.89:0/3624338535 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9a809c300 con 0x7fb9b8101de0 2024-09-10T08:37:28.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.464+0000 7fb9be600640 1 -- 172.21.15.89:0/3624338535 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb9b802d7c0 con 0x7fb9b8101de0 2024-09-10T08:37:28.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.464+0000 7fb9b4c00640 1 -- 172.21.15.89:0/3624338535 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) ==== 81+0+590 (secure 0 0 0) 0x7fb9a809bc00 con 0x7fb9b8101de0 2024-09-10T08:37:28.465 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/850669510 2024-09-11T08:29:41.472002+0000 2024-09-10T08:37:28.465 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/3541027428 2024-09-11T08:29:41.472002+0000 2024-09-10T08:37:28.465 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6801/3794982914 2024-09-11T08:29:41.472002+0000 2024-09-10T08:37:28.465 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6800/3794982914 2024-09-11T08:29:41.472002+0000 2024-09-10T08:37:28.465 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2396468787 2024-09-11T08:29:41.472002+0000 2024-09-10T08:37:28.465 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1609720980 2024-09-11T08:29:14.554869+0000 2024-09-10T08:37:28.466 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/995019144 2024-09-11T08:29:14.554869+0000 2024-09-10T08:37:28.466 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6801/4223112778 2024-09-11T08:29:14.554869+0000 2024-09-10T08:37:28.466 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6800/4223112778 2024-09-11T08:29:14.554869+0000 2024-09-10T08:37:28.466 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2179977257 2024-09-11T08:29:14.554869+0000 2024-09-10T08:37:28.466 INFO:teuthology.orchestra.run.smithi089.stderr:listed 10 entries 2024-09-10T08:37:28.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.467+0000 7fb9be600640 1 -- 172.21.15.89:0/3624338535 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb99c0a2b80 msgr2=0x7fb99c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:28.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.467+0000 7fb9be600640 1 --2- 172.21.15.89:0/3624338535 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb99c0a2b80 0x7fb99c0a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8075f80 tx=0x7fb9a0005a20 comp rx=0 tx=0).stop 2024-09-10T08:37:28.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.467+0000 7fb9be600640 1 -- 172.21.15.89:0/3624338535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b8101de0 msgr2=0x7fb9b80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:28.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.467+0000 7fb9be600640 1 --2- 172.21.15.89:0/3624338535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b8101de0 0x7fb9b80744f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800ea60 tx=0x7fb9a800ef20 comp rx=0 tx=0).stop 2024-09-10T08:37:28.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.467+0000 7fb9be600640 1 -- 172.21.15.89:0/3624338535 shutdown_connections 2024-09-10T08:37:28.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.467+0000 7fb9be600640 1 --2- 172.21.15.89:0/3624338535 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb99c0a2b80 0x7fb99c0a5040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:28.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.467+0000 7fb9be600640 1 --2- 172.21.15.89:0/3624338535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9b8106f10 0x7fb9b8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:28.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.467+0000 7fb9be600640 1 --2- 172.21.15.89:0/3624338535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9b8102790 0x7fb9b8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:28.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.467+0000 7fb9be600640 1 --2- 172.21.15.89:0/3624338535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9b8101de0 0x7fb9b80744f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:28.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.467+0000 7fb9be600640 1 -- 172.21.15.89:0/3624338535 >> 172.21.15.89:0/3624338535 conn(0x7fb9b80fd8c0 msgr2=0x7fb9b80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:28.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.467+0000 7fb9be600640 1 -- 172.21.15.89:0/3624338535 shutdown_connections 2024-09-10T08:37:28.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:28.468+0000 7fb9be600640 1 -- 172.21.15.89:0/3624338535 wait complete. 2024-09-10T08:37:28.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:28 smithi089 ceph-mon[30757]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:28.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:28 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3624338535' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:37:28.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:28 smithi149 ceph-mon[36167]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:28.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:28 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3624338535' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:37:29.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:28 smithi111 ceph-mon[35220]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:29.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:28 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3624338535' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:37:29.388 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:37:29.388 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T08:37:29.428 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T08:37:29.838 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:30.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.397+0000 7fd547200640 1 -- 172.21.15.89:0/1529707895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd540112a50 msgr2=0x7fd540114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:30.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.397+0000 7fd547200640 1 --2- 172.21.15.89:0/1529707895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd540112a50 0x7fd540114e40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd538009ca0 tx=0x7fd53802a8b0 comp rx=0 tx=0).stop 2024-09-10T08:37:30.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.398+0000 7fd547200640 1 -- 172.21.15.89:0/1529707895 shutdown_connections 2024-09-10T08:37:30.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.398+0000 7fd547200640 1 --2- 172.21.15.89:0/1529707895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd540112a50 0x7fd540114e40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:30.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.398+0000 7fd547200640 1 --2- 172.21.15.89:0/1529707895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd540077040 0x7fd540112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:30.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.398+0000 7fd547200640 1 --2- 172.21.15.89:0/1529707895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd540076720 0x7fd540076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:30.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.398+0000 7fd547200640 1 -- 172.21.15.89:0/1529707895 >> 172.21.15.89:0/1529707895 conn(0x7fd5400ffa50 msgr2=0x7fd540101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:30.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.398+0000 7fd547200640 1 -- 172.21.15.89:0/1529707895 shutdown_connections 2024-09-10T08:37:30.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.398+0000 7fd547200640 1 -- 172.21.15.89:0/1529707895 wait complete. 2024-09-10T08:37:30.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.399+0000 7fd547200640 1 Processor -- start 2024-09-10T08:37:30.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.399+0000 7fd547200640 1 -- start start 2024-09-10T08:37:30.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.399+0000 7fd547200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd540076720 0x7fd5401a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:30.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.399+0000 7fd547200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd540077040 0x7fd5401a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:30.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.399+0000 7fd547200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd540112a50 0x7fd54019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:30.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.399+0000 7fd547200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd540118600 con 0x7fd540076720 2024-09-10T08:37:30.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.399+0000 7fd547200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd540118480 con 0x7fd540077040 2024-09-10T08:37:30.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.399+0000 7fd547200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd540118780 con 0x7fd540112a50 2024-09-10T08:37:30.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.399+0000 7fd53fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd540076720 0x7fd5401a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:30.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.399+0000 7fd53fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd540076720 0x7fd5401a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55608/0 (socket says 172.21.15.89:55608) 2024-09-10T08:37:30.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.399+0000 7fd53fe00640 1 -- 172.21.15.89:0/2860725513 learned_addr learned my addr 172.21.15.89:0/2860725513 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:30.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.399+0000 7fd545200640 1 --2- 172.21.15.89:0/2860725513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd540112a50 0x7fd54019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:30.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.399+0000 7fd53f400640 1 --2- 172.21.15.89:0/2860725513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd540077040 0x7fd5401a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:30.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.400+0000 7fd53fe00640 1 -- 172.21.15.89:0/2860725513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd540112a50 msgr2=0x7fd54019e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:30.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.400+0000 7fd53fe00640 1 --2- 172.21.15.89:0/2860725513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd540112a50 0x7fd54019e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:30.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.400+0000 7fd53fe00640 1 -- 172.21.15.89:0/2860725513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd540077040 msgr2=0x7fd5401a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:30.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.400+0000 7fd53fe00640 1 --2- 172.21.15.89:0/2860725513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd540077040 0x7fd5401a4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:30.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.400+0000 7fd53fe00640 1 -- 172.21.15.89:0/2860725513 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54019f080 con 0x7fd540076720 2024-09-10T08:37:30.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.400+0000 7fd53fe00640 1 --2- 172.21.15.89:0/2860725513 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd540076720 0x7fd5401a4790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd53000e9f0 tx=0x7fd53000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:30.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.400+0000 7fd53cc00640 1 -- 172.21.15.89:0/2860725513 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53000cd80 con 0x7fd540076720 2024-09-10T08:37:30.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.400+0000 7fd547200640 1 -- 172.21.15.89:0/2860725513 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54019f370 con 0x7fd540076720 2024-09-10T08:37:30.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.400+0000 7fd53cc00640 1 -- 172.21.15.89:0/2860725513 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd530010910 con 0x7fd540076720 2024-09-10T08:37:30.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.401+0000 7fd53cc00640 1 -- 172.21.15.89:0/2860725513 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5300188f0 con 0x7fd540076720 2024-09-10T08:37:30.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.401+0000 7fd547200640 1 -- 172.21.15.89:0/2860725513 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5401ab660 con 0x7fd540076720 2024-09-10T08:37:30.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.402+0000 7fd547200640 1 -- 172.21.15.89:0/2860725513 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd540076b00 con 0x7fd540076720 2024-09-10T08:37:30.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.404+0000 7fd53cc00640 1 -- 172.21.15.89:0/2860725513 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd530010ac0 con 0x7fd540076720 2024-09-10T08:37:30.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.406+0000 7fd53cc00640 1 --2- 172.21.15.89:0/2860725513 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd5240a2c00 0x7fd5240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:30.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.406+0000 7fd53cc00640 1 -- 172.21.15.89:0/2860725513 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fd5300cf620 con 0x7fd540076720 2024-09-10T08:37:30.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.406+0000 7fd53f400640 1 --2- 172.21.15.89:0/2860725513 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd5240a2c00 0x7fd5240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:30.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.407+0000 7fd53f400640 1 --2- 172.21.15.89:0/2860725513 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd5240a2c00 0x7fd5240a50c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd540079390 tx=0x7fd52c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:30.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.410+0000 7fd53cc00640 1 -- 172.21.15.89:0/2860725513 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd53009bc90 con 0x7fd540076720 2024-09-10T08:37:30.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.602+0000 7fd547200640 1 -- 172.21.15.89:0/2860725513 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd54019fe80 con 0x7fd540076720 2024-09-10T08:37:30.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.602+0000 7fd53cc00640 1 -- 172.21.15.89:0/2860725513 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) ==== 81+0+590 (secure 0 0 0) 0x7fd53009ff90 con 0x7fd540076720 2024-09-10T08:37:30.602 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/850669510 2024-09-11T08:29:41.472002+0000 2024-09-10T08:37:30.602 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/3541027428 2024-09-11T08:29:41.472002+0000 2024-09-10T08:37:30.603 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6801/3794982914 2024-09-11T08:29:41.472002+0000 2024-09-10T08:37:30.603 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6800/3794982914 2024-09-11T08:29:41.472002+0000 2024-09-10T08:37:30.603 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2396468787 2024-09-11T08:29:41.472002+0000 2024-09-10T08:37:30.603 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1609720980 2024-09-11T08:29:14.554869+0000 2024-09-10T08:37:30.603 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/995019144 2024-09-11T08:29:14.554869+0000 2024-09-10T08:37:30.603 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6801/4223112778 2024-09-11T08:29:14.554869+0000 2024-09-10T08:37:30.603 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6800/4223112778 2024-09-11T08:29:14.554869+0000 2024-09-10T08:37:30.603 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2179977257 2024-09-11T08:29:14.554869+0000 2024-09-10T08:37:30.603 INFO:teuthology.orchestra.run.smithi089.stderr:listed 10 entries 2024-09-10T08:37:30.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.604+0000 7fd547200640 1 -- 172.21.15.89:0/2860725513 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd5240a2c00 msgr2=0x7fd5240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:30.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.604+0000 7fd547200640 1 --2- 172.21.15.89:0/2860725513 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd5240a2c00 0x7fd5240a50c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd540079390 tx=0x7fd52c00b390 comp rx=0 tx=0).stop 2024-09-10T08:37:30.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.605+0000 7fd547200640 1 -- 172.21.15.89:0/2860725513 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd540076720 msgr2=0x7fd5401a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:30.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.605+0000 7fd547200640 1 --2- 172.21.15.89:0/2860725513 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd540076720 0x7fd5401a4790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd53000e9f0 tx=0x7fd53000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:37:30.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.605+0000 7fd547200640 1 -- 172.21.15.89:0/2860725513 shutdown_connections 2024-09-10T08:37:30.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.605+0000 7fd547200640 1 --2- 172.21.15.89:0/2860725513 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd5240a2c00 0x7fd5240a50c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:30.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.605+0000 7fd547200640 1 --2- 172.21.15.89:0/2860725513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd540112a50 0x7fd54019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:30.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.605+0000 7fd547200640 1 --2- 172.21.15.89:0/2860725513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd540077040 0x7fd5401a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:30.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.605+0000 7fd547200640 1 --2- 172.21.15.89:0/2860725513 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd540076720 0x7fd5401a4790 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:30.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.605+0000 7fd547200640 1 -- 172.21.15.89:0/2860725513 >> 172.21.15.89:0/2860725513 conn(0x7fd5400ffa50 msgr2=0x7fd540101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:30.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.605+0000 7fd547200640 1 -- 172.21.15.89:0/2860725513 shutdown_connections 2024-09-10T08:37:30.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:30.605+0000 7fd547200640 1 -- 172.21.15.89:0/2860725513 wait complete. 2024-09-10T08:37:30.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:30 smithi149 ceph-mon[36167]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:30.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:30 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2860725513' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:37:30.994 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:30 smithi089 ceph-mon[30757]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:30.994 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:30 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2860725513' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:37:31.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:30 smithi111 ceph-mon[35220]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:31.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:30 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2860725513' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:37:31.470 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi089.front.sepia.ceph.com... 2024-09-10T08:37:31.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:37:31.470 DEBUG:teuthology.orchestra.run.smithi089:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:37:31.525 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi089.stderr:ceph-fuse[74772]: fuse finished with error 0 and tester_r 0 2024-09-10T08:37:31.529 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T08:37:32.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:32 smithi149 ceph-mon[36167]: pgmap v317: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T08:37:33.040 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:32 smithi089 ceph-mon[30757]: pgmap v317: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T08:37:33.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:32 smithi111 ceph-mon[35220]: pgmap v317: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T08:37:34.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:34 smithi149 ceph-mon[36167]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T08:37:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:34 smithi111 ceph-mon[35220]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T08:37:35.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:34 smithi089 ceph-mon[30757]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T08:37:36.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:36 smithi149 ceph-mon[36167]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:37:37.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:36 smithi111 ceph-mon[35220]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:37:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:36 smithi089 ceph-mon[30757]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:37:37.631 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2024-09-10T08:37:37.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:37:37.632 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-10T08:37:37.692 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:37:38.065 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:38.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.652+0000 7f6c8b800640 1 -- 172.21.15.89:0/145062879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c8410e430 msgr2=0x7f6c84110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:38.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.652+0000 7f6c8b800640 1 --2- 172.21.15.89:0/145062879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c8410e430 0x7f6c84110820 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009ca0 tx=0x7f6c7c02a950 comp rx=0 tx=0).stop 2024-09-10T08:37:38.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.652+0000 7f6c8b800640 1 -- 172.21.15.89:0/145062879 shutdown_connections 2024-09-10T08:37:38.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.652+0000 7f6c8b800640 1 --2- 172.21.15.89:0/145062879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c8410e430 0x7f6c84110820 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:38.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.652+0000 7f6c8b800640 1 --2- 172.21.15.89:0/145062879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c84074ee0 0x7f6c8410def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:38.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.652+0000 7f6c8b800640 1 --2- 172.21.15.89:0/145062879 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c840745c0 0x7f6c840749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:38.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.652+0000 7f6c8b800640 1 -- 172.21.15.89:0/145062879 >> 172.21.15.89:0/145062879 conn(0x7f6c840fd8b0 msgr2=0x7f6c840ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:38.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.652+0000 7f6c8b800640 1 -- 172.21.15.89:0/145062879 shutdown_connections 2024-09-10T08:37:38.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.652+0000 7f6c8b800640 1 -- 172.21.15.89:0/145062879 wait complete. 2024-09-10T08:37:38.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.653+0000 7f6c8b800640 1 Processor -- start 2024-09-10T08:37:38.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.653+0000 7f6c8b800640 1 -- start start 2024-09-10T08:37:38.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.654+0000 7f6c8b800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c840745c0 0x7f6c8419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:38.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.654+0000 7f6c8b800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c84074ee0 0x7f6c8419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:38.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.654+0000 7f6c8b800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c8410e430 0x7f6c8419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:38.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.654+0000 7f6c8b800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6c841141d0 con 0x7f6c8410e430 2024-09-10T08:37:38.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.654+0000 7f6c8b800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6c84114050 con 0x7f6c840745c0 2024-09-10T08:37:38.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.654+0000 7f6c8b800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6c84114350 con 0x7f6c84074ee0 2024-09-10T08:37:38.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.654+0000 7f6c83e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c84074ee0 0x7f6c8419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:38.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.654+0000 7f6c88e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c840745c0 0x7f6c8419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:38.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.654+0000 7f6c89800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c8410e430 0x7f6c8419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:38.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.654+0000 7f6c88e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c840745c0 0x7f6c8419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:39034/0 (socket says 172.21.15.89:39034) 2024-09-10T08:37:38.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.654+0000 7f6c83e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c84074ee0 0x7f6c8419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:55916/0 (socket says 172.21.15.89:55916) 2024-09-10T08:37:38.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.654+0000 7f6c88e00640 1 -- 172.21.15.89:0/1293783589 learned_addr learned my addr 172.21.15.89:0/1293783589 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:38.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.655+0000 7f6c88e00640 1 -- 172.21.15.89:0/1293783589 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c84074ee0 msgr2=0x7f6c8419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:38.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.655+0000 7f6c88e00640 1 --2- 172.21.15.89:0/1293783589 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c84074ee0 0x7f6c8419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:38.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.655+0000 7f6c88e00640 1 -- 172.21.15.89:0/1293783589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c8410e430 msgr2=0x7f6c8419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:38.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.655+0000 7f6c88e00640 1 --2- 172.21.15.89:0/1293783589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c8410e430 0x7f6c8419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:38.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.655+0000 7f6c88e00640 1 -- 172.21.15.89:0/1293783589 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8419f5f0 con 0x7f6c840745c0 2024-09-10T08:37:38.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.655+0000 7f6c83e00640 1 --2- 172.21.15.89:0/1293783589 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c84074ee0 0x7f6c8419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:37:38.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.655+0000 7f6c88e00640 1 --2- 172.21.15.89:0/1293783589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c840745c0 0x7f6c8419a640 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800ea60 tx=0x7f6c7800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:38.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.655+0000 7f6c81600640 1 -- 172.21.15.89:0/1293783589 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7800cde0 con 0x7f6c840745c0 2024-09-10T08:37:38.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.655+0000 7f6c8b800640 1 -- 172.21.15.89:0/1293783589 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8419f8e0 con 0x7f6c840745c0 2024-09-10T08:37:38.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.655+0000 7f6c81600640 1 -- 172.21.15.89:0/1293783589 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c780035f0 con 0x7f6c840745c0 2024-09-10T08:37:38.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.655+0000 7f6c81600640 1 -- 172.21.15.89:0/1293783589 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78010670 con 0x7f6c840745c0 2024-09-10T08:37:38.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.655+0000 7f6c8b800640 1 -- 172.21.15.89:0/1293783589 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c841a71c0 con 0x7f6c840745c0 2024-09-10T08:37:38.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.656+0000 7f6c8b800640 1 -- 172.21.15.89:0/1293783589 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c840749a0 con 0x7f6c840745c0 2024-09-10T08:37:38.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.658+0000 7f6c81600640 1 -- 172.21.15.89:0/1293783589 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6c780037a0 con 0x7f6c840745c0 2024-09-10T08:37:38.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.660+0000 7f6c81600640 1 --2- 172.21.15.89:0/1293783589 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6c6c0a2b80 0x7f6c6c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:38.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.660+0000 7f6c81600640 1 -- 172.21.15.89:0/1293783589 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f6c780cf570 con 0x7f6c840745c0 2024-09-10T08:37:38.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.660+0000 7f6c83e00640 1 --2- 172.21.15.89:0/1293783589 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6c6c0a2b80 0x7f6c6c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:38.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.660+0000 7f6c83e00640 1 --2- 172.21.15.89:0/1293783589 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6c6c0a2b80 0x7f6c6c0a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6c8410f750 tx=0x7f6c70005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:38.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.662+0000 7f6c81600640 1 -- 172.21.15.89:0/1293783589 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c7809c230 con 0x7f6c840745c0 2024-09-10T08:37:38.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:38 smithi089 ceph-mon[30757]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:37:38.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.839+0000 7f6c8b800640 1 -- 172.21.15.89:0/1293783589 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c8419b9a0 con 0x7f6c840745c0 2024-09-10T08:37:38.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.841+0000 7f6c81600640 1 -- 172.21.15.89:0/1293783589 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24236 (secure 0 0 0) 0x7f6c7809bb30 con 0x7f6c840745c0 2024-09-10T08:37:38.842 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:37:38.842 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":92,"fsid":"91f747bc-6f4e-11ef-bcea-c7b262605968","created":"2024-09-10T08:28:49.697035+0000","modified":"2024-09-10T08:37:08.225049+0000","last_up_change":"2024-09-10T08:36:00.535433+0000","last_in_change":"2024-09-10T08:35:37.131131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:32:02.060714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:36:52.462301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:36:55.119708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:37:03.729002+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d3b55192-bd95-4497-89ea-dc017a25c0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6803","nonce":1760523284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6805","nonce":1760523284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6809","nonce":1760523284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6807","nonce":1760523284}]},"public_addr":"172.21.15.89:6803/1760523284","cluster_addr":"172.21.15.89:6805/1760523284","heartbeat_back_addr":"172.21.15.89:6809/1760523284","heartbeat_front_addr":"172.21.15.89:6807/1760523284","state":["exists","up"]},{"osd":1,"uuid":"60aae117-cb52-4034-a700-11693171cd52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6801","nonce":2369291742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6803","nonce":2369291742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6807","nonce":2369291742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6805","nonce":2369291742}]},"public_addr":"172.21.15.111:6801/2369291742","cluster_addr":"172.21.15.111:6803/2369291742","heartbeat_back_addr":"172.21.15.111:6807/2369291742","heartbeat_front_addr":"172.21.15.111:6805/2369291742","state":["exists","up"]},{"osd":2,"uuid":"a305a019-0870-437c-a298-4f923ec54e77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6801","nonce":1092261395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6803","nonce":1092261395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6807","nonce":1092261395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6805","nonce":1092261395}]},"public_addr":"172.21.15.149:6801/1092261395","cluster_addr":"172.21.15.149:6803/1092261395","heartbeat_back_addr":"172.21.15.149:6807/1092261395","heartbeat_front_addr":"172.21.15.149:6805/1092261395","state":["exists","up"]},{"osd":3,"uuid":"d02f54c1-f883-40f6-9341-8d9fc4b712ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6811","nonce":1405773950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6813","nonce":1405773950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6817","nonce":1405773950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6815","nonce":1405773950}]},"public_addr":"172.21.15.89:6811/1405773950","cluster_addr":"172.21.15.89:6813/1405773950","heartbeat_back_addr":"172.21.15.89:6817/1405773950","heartbeat_front_addr":"172.21.15.89:6815/1405773950","state":["exists","up"]},{"osd":4,"uuid":"23cff46b-88dd-4344-be06-b35db6014f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6809","nonce":316717323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6811","nonce":316717323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6815","nonce":316717323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6813","nonce":316717323}]},"public_addr":"172.21.15.111:6809/316717323","cluster_addr":"172.21.15.111:6811/316717323","heartbeat_back_addr":"172.21.15.111:6815/316717323","heartbeat_front_addr":"172.21.15.111:6813/316717323","state":["exists","up"]},{"osd":5,"uuid":"8e516e8d-e7c6-4ebc-80a5-29932161df4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6809","nonce":1256148461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6811","nonce":1256148461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6815","nonce":1256148461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6813","nonce":1256148461}]},"public_addr":"172.21.15.149:6809/1256148461","cluster_addr":"172.21.15.149:6811/1256148461","heartbeat_back_addr":"172.21.15.149:6815/1256148461","heartbeat_front_addr":"172.21.15.149:6813/1256148461","state":["exists","up"]},{"osd":6,"uuid":"48f92472-f657-4c5a-adeb-20d4fea1ce8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6819","nonce":4190488024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6821","nonce":4190488024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6825","nonce":4190488024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6823","nonce":4190488024}]},"public_addr":"172.21.15.89:6819/4190488024","cluster_addr":"172.21.15.89:6821/4190488024","heartbeat_back_addr":"172.21.15.89:6825/4190488024","heartbeat_front_addr":"172.21.15.89:6823/4190488024","state":["exists","up"]},{"osd":7,"uuid":"14a2a6f0-c4cd-406b-b637-7527fd32babe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6817","nonce":652666376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6819","nonce":652666376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6823","nonce":652666376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6821","nonce":652666376}]},"public_addr":"172.21.15.111:6817/652666376","cluster_addr":"172.21.15.111:6819/652666376","heartbeat_back_addr":"172.21.15.111:6823/652666376","heartbeat_front_addr":"172.21.15.111:6821/652666376","state":["exists","up"]},{"osd":8,"uuid":"a54502e6-95f7-431f-8db1-54a6c7983b5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6817","nonce":2955172869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6819","nonce":2955172869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6823","nonce":2955172869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6821","nonce":2955172869}]},"public_addr":"172.21.15.149:6817/2955172869","cluster_addr":"172.21.15.149:6819/2955172869","heartbeat_back_addr":"172.21.15.149:6823/2955172869","heartbeat_front_addr":"172.21.15.149:6821/2955172869","state":["exists","up"]},{"osd":9,"uuid":"5fe17344-5b1c-4601-a633-639cc1691478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6827","nonce":2427467040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6829","nonce":2427467040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6833","nonce":2427467040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6831","nonce":2427467040}]},"public_addr":"172.21.15.89:6827/2427467040","cluster_addr":"172.21.15.89:6829/2427467040","heartbeat_back_addr":"172.21.15.89:6833/2427467040","heartbeat_front_addr":"172.21.15.89:6831/2427467040","state":["exists","up"]},{"osd":10,"uuid":"d8f29890-3064-4b67-b649-467a0ac86db7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6825","nonce":2328678744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6827","nonce":2328678744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6831","nonce":2328678744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6829","nonce":2328678744}]},"public_addr":"172.21.15.111:6825/2328678744","cluster_addr":"172.21.15.111:6827/2328678744","heartbeat_back_addr":"172.21.15.111:6831/2328678744","heartbeat_front_addr":"172.21.15.111:6829/2328678744","state":["exists","up"]},{"osd":11,"uuid":"92182c73-f139-47ca-a0ef-5046ae9b2310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6825","nonce":3130508827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6827","nonce":3130508827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6831","nonce":3130508827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6829","nonce":3130508827}]},"public_addr":"172.21.15.149:6825/3130508827","cluster_addr":"172.21.15.149:6827/3130508827","heartbeat_back_addr":"172.21.15.149:6831/3130508827","heartbeat_front_addr":"172.21.15.149:6829/3130508827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:09.031810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:33.695124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:58.065510+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:27.592838+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:53.275289+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:18.305677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:47.781535+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:12.889512+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:37.744193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:07.718408+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:33.391027+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:59.013250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/850669510":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/3541027428":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6801/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6800/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/2396468787":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/1609720980":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/995019144":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6801/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6800/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/2179977257":"2024-09-11T08:29:14.554869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:37:38.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.844+0000 7f6c8b800640 1 -- 172.21.15.89:0/1293783589 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6c6c0a2b80 msgr2=0x7f6c6c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:38.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.844+0000 7f6c8b800640 1 --2- 172.21.15.89:0/1293783589 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6c6c0a2b80 0x7f6c6c0a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6c8410f750 tx=0x7f6c70005f90 comp rx=0 tx=0).stop 2024-09-10T08:37:38.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.844+0000 7f6c8b800640 1 -- 172.21.15.89:0/1293783589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c840745c0 msgr2=0x7f6c8419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:38.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.844+0000 7f6c8b800640 1 --2- 172.21.15.89:0/1293783589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c840745c0 0x7f6c8419a640 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800ea60 tx=0x7f6c7800ef20 comp rx=0 tx=0).stop 2024-09-10T08:37:38.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.844+0000 7f6c8b800640 1 -- 172.21.15.89:0/1293783589 shutdown_connections 2024-09-10T08:37:38.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.844+0000 7f6c8b800640 1 --2- 172.21.15.89:0/1293783589 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6c6c0a2b80 0x7f6c6c0a5040 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:38.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.844+0000 7f6c8b800640 1 --2- 172.21.15.89:0/1293783589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c8410e430 0x7f6c8419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:38.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.844+0000 7f6c8b800640 1 --2- 172.21.15.89:0/1293783589 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c84074ee0 0x7f6c8419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:38.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.844+0000 7f6c8b800640 1 --2- 172.21.15.89:0/1293783589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c840745c0 0x7f6c8419a640 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:38.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.844+0000 7f6c8b800640 1 -- 172.21.15.89:0/1293783589 >> 172.21.15.89:0/1293783589 conn(0x7f6c840fd8b0 msgr2=0x7f6c840fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:38.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.844+0000 7f6c8b800640 1 -- 172.21.15.89:0/1293783589 shutdown_connections 2024-09-10T08:37:38.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:38.844+0000 7f6c8b800640 1 -- 172.21.15.89:0/1293783589 wait complete. 2024-09-10T08:37:38.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:38 smithi149 ceph-mon[36167]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:37:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:38 smithi111 ceph-mon[35220]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:37:39.606 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:37:39.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:39 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1293783589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:37:39.972 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:40.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:39 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1293783589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:37:40.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:39 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1293783589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:37:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.541+0000 7f850ba00640 1 -- 172.21.15.89:0/324287780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8504101df0 msgr2=0x7f85041021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.541+0000 7f850ba00640 1 --2- 172.21.15.89:0/324287780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8504101df0 0x7f85041021d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009900 tx=0x7f84f402a410 comp rx=0 tx=0).stop 2024-09-10T08:37:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.542+0000 7f850ba00640 1 -- 172.21.15.89:0/324287780 shutdown_connections 2024-09-10T08:37:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.542+0000 7f850ba00640 1 --2- 172.21.15.89:0/324287780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8504106f20 0x7f850410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.542+0000 7f850ba00640 1 --2- 172.21.15.89:0/324287780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85041027a0 0x7f85041067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.542+0000 7f850ba00640 1 --2- 172.21.15.89:0/324287780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8504101df0 0x7f85041021d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.542+0000 7f850ba00640 1 -- 172.21.15.89:0/324287780 >> 172.21.15.89:0/324287780 conn(0x7f85040fd8b0 msgr2=0x7f85040ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.542+0000 7f850ba00640 1 -- 172.21.15.89:0/324287780 shutdown_connections 2024-09-10T08:37:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.542+0000 7f850ba00640 1 -- 172.21.15.89:0/324287780 wait complete. 2024-09-10T08:37:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.543+0000 7f850ba00640 1 Processor -- start 2024-09-10T08:37:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.543+0000 7f850ba00640 1 -- start start 2024-09-10T08:37:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.543+0000 7f850ba00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8504101df0 0x7f85040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.543+0000 7f850ba00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85041027a0 0x7f8504074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.543+0000 7f850ba00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8504106f20 0x7f8504074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.543+0000 7f850ba00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8504111f80 con 0x7f8504106f20 2024-09-10T08:37:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.543+0000 7f850ba00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8504111e00 con 0x7f8504101df0 2024-09-10T08:37:40.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.543+0000 7f850ba00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8504112100 con 0x7f85041027a0 2024-09-10T08:37:40.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.544+0000 7f8509a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8504106f20 0x7f8504074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:40.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.544+0000 7f84fbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85041027a0 0x7f8504074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:40.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.544+0000 7f8509000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8504101df0 0x7f85040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:40.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.544+0000 7f84fbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85041027a0 0x7f8504074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:55942/0 (socket says 172.21.15.89:55942) 2024-09-10T08:37:40.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.544+0000 7f84fbe00640 1 -- 172.21.15.89:0/1530695089 learned_addr learned my addr 172.21.15.89:0/1530695089 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:40.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.544+0000 7f8509a00640 1 --2- 172.21.15.89:0/1530695089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8504106f20 0x7f8504074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58696/0 (socket says 172.21.15.89:58696) 2024-09-10T08:37:40.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.544+0000 7f8509a00640 1 -- 172.21.15.89:0/1530695089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85041027a0 msgr2=0x7f8504074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:40.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.544+0000 7f8509a00640 1 --2- 172.21.15.89:0/1530695089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85041027a0 0x7f8504074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:40.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.544+0000 7f8509a00640 1 -- 172.21.15.89:0/1530695089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8504101df0 msgr2=0x7f85040744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:40.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.544+0000 7f8509a00640 1 --2- 172.21.15.89:0/1530695089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8504101df0 0x7f85040744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:40.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.544+0000 7f8509a00640 1 -- 172.21.15.89:0/1530695089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85041a68c0 con 0x7f8504106f20 2024-09-10T08:37:40.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.544+0000 7f84fbe00640 1 --2- 172.21.15.89:0/1530695089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85041027a0 0x7f8504074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:37:40.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.544+0000 7f8509a00640 1 --2- 172.21.15.89:0/1530695089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8504106f20 0x7f8504074f70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009fd0 tx=0x7f84fc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:40.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.545+0000 7f8502000640 1 -- 172.21.15.89:0/1530695089 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc00ce20 con 0x7f8504106f20 2024-09-10T08:37:40.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.545+0000 7f8502000640 1 -- 172.21.15.89:0/1530695089 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84fc005020 con 0x7f8504106f20 2024-09-10T08:37:40.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.545+0000 7f8502000640 1 -- 172.21.15.89:0/1530695089 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc010720 con 0x7f8504106f20 2024-09-10T08:37:40.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.545+0000 7f850ba00640 1 -- 172.21.15.89:0/1530695089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85041a6bb0 con 0x7f8504106f20 2024-09-10T08:37:40.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.545+0000 7f850ba00640 1 -- 172.21.15.89:0/1530695089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85041a70a0 con 0x7f8504106f20 2024-09-10T08:37:40.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.546+0000 7f850ba00640 1 -- 172.21.15.89:0/1530695089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85041021d0 con 0x7f8504106f20 2024-09-10T08:37:40.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.548+0000 7f8502000640 1 -- 172.21.15.89:0/1530695089 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f84fc005540 con 0x7f8504106f20 2024-09-10T08:37:40.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.549+0000 7f8502000640 1 --2- 172.21.15.89:0/1530695089 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f84e80a2c00 0x7f84e80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:40.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.549+0000 7f8502000640 1 -- 172.21.15.89:0/1530695089 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f84fc0cf840 con 0x7f8504106f20 2024-09-10T08:37:40.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.549+0000 7f8509000640 1 --2- 172.21.15.89:0/1530695089 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f84e80a2c00 0x7f84e80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:40.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.550+0000 7f8509000640 1 --2- 172.21.15.89:0/1530695089 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f84e80a2c00 0x7f84e80a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009230 tx=0x7f84f40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:40.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.553+0000 7f8502000640 1 -- 172.21.15.89:0/1530695089 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f84fc09c560 con 0x7f8504106f20 2024-09-10T08:37:40.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.731+0000 7f850ba00640 1 -- 172.21.15.89:0/1530695089 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8504109b70 con 0x7f8504106f20 2024-09-10T08:37:40.731 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:40 smithi089 ceph-mon[30757]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:37:40.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.733+0000 7f8502000640 1 -- 172.21.15.89:0/1530695089 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24236 (secure 0 0 0) 0x7f84fc09be60 con 0x7f8504106f20 2024-09-10T08:37:40.733 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:37:40.733 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":92,"fsid":"91f747bc-6f4e-11ef-bcea-c7b262605968","created":"2024-09-10T08:28:49.697035+0000","modified":"2024-09-10T08:37:08.225049+0000","last_up_change":"2024-09-10T08:36:00.535433+0000","last_in_change":"2024-09-10T08:35:37.131131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:32:02.060714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:36:52.462301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:36:55.119708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:37:03.729002+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d3b55192-bd95-4497-89ea-dc017a25c0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6803","nonce":1760523284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6805","nonce":1760523284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6809","nonce":1760523284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6807","nonce":1760523284}]},"public_addr":"172.21.15.89:6803/1760523284","cluster_addr":"172.21.15.89:6805/1760523284","heartbeat_back_addr":"172.21.15.89:6809/1760523284","heartbeat_front_addr":"172.21.15.89:6807/1760523284","state":["exists","up"]},{"osd":1,"uuid":"60aae117-cb52-4034-a700-11693171cd52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6801","nonce":2369291742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6803","nonce":2369291742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6807","nonce":2369291742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6805","nonce":2369291742}]},"public_addr":"172.21.15.111:6801/2369291742","cluster_addr":"172.21.15.111:6803/2369291742","heartbeat_back_addr":"172.21.15.111:6807/2369291742","heartbeat_front_addr":"172.21.15.111:6805/2369291742","state":["exists","up"]},{"osd":2,"uuid":"a305a019-0870-437c-a298-4f923ec54e77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6801","nonce":1092261395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6803","nonce":1092261395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6807","nonce":1092261395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6805","nonce":1092261395}]},"public_addr":"172.21.15.149:6801/1092261395","cluster_addr":"172.21.15.149:6803/1092261395","heartbeat_back_addr":"172.21.15.149:6807/1092261395","heartbeat_front_addr":"172.21.15.149:6805/1092261395","state":["exists","up"]},{"osd":3,"uuid":"d02f54c1-f883-40f6-9341-8d9fc4b712ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6811","nonce":1405773950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6813","nonce":1405773950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6817","nonce":1405773950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6815","nonce":1405773950}]},"public_addr":"172.21.15.89:6811/1405773950","cluster_addr":"172.21.15.89:6813/1405773950","heartbeat_back_addr":"172.21.15.89:6817/1405773950","heartbeat_front_addr":"172.21.15.89:6815/1405773950","state":["exists","up"]},{"osd":4,"uuid":"23cff46b-88dd-4344-be06-b35db6014f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6809","nonce":316717323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6811","nonce":316717323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6815","nonce":316717323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6813","nonce":316717323}]},"public_addr":"172.21.15.111:6809/316717323","cluster_addr":"172.21.15.111:6811/316717323","heartbeat_back_addr":"172.21.15.111:6815/316717323","heartbeat_front_addr":"172.21.15.111:6813/316717323","state":["exists","up"]},{"osd":5,"uuid":"8e516e8d-e7c6-4ebc-80a5-29932161df4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6809","nonce":1256148461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6811","nonce":1256148461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6815","nonce":1256148461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6813","nonce":1256148461}]},"public_addr":"172.21.15.149:6809/1256148461","cluster_addr":"172.21.15.149:6811/1256148461","heartbeat_back_addr":"172.21.15.149:6815/1256148461","heartbeat_front_addr":"172.21.15.149:6813/1256148461","state":["exists","up"]},{"osd":6,"uuid":"48f92472-f657-4c5a-adeb-20d4fea1ce8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6819","nonce":4190488024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6821","nonce":4190488024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6825","nonce":4190488024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6823","nonce":4190488024}]},"public_addr":"172.21.15.89:6819/4190488024","cluster_addr":"172.21.15.89:6821/4190488024","heartbeat_back_addr":"172.21.15.89:6825/4190488024","heartbeat_front_addr":"172.21.15.89:6823/4190488024","state":["exists","up"]},{"osd":7,"uuid":"14a2a6f0-c4cd-406b-b637-7527fd32babe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6817","nonce":652666376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6819","nonce":652666376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6823","nonce":652666376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6821","nonce":652666376}]},"public_addr":"172.21.15.111:6817/652666376","cluster_addr":"172.21.15.111:6819/652666376","heartbeat_back_addr":"172.21.15.111:6823/652666376","heartbeat_front_addr":"172.21.15.111:6821/652666376","state":["exists","up"]},{"osd":8,"uuid":"a54502e6-95f7-431f-8db1-54a6c7983b5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6817","nonce":2955172869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6819","nonce":2955172869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6823","nonce":2955172869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6821","nonce":2955172869}]},"public_addr":"172.21.15.149:6817/2955172869","cluster_addr":"172.21.15.149:6819/2955172869","heartbeat_back_addr":"172.21.15.149:6823/2955172869","heartbeat_front_addr":"172.21.15.149:6821/2955172869","state":["exists","up"]},{"osd":9,"uuid":"5fe17344-5b1c-4601-a633-639cc1691478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6827","nonce":2427467040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6829","nonce":2427467040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6833","nonce":2427467040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6831","nonce":2427467040}]},"public_addr":"172.21.15.89:6827/2427467040","cluster_addr":"172.21.15.89:6829/2427467040","heartbeat_back_addr":"172.21.15.89:6833/2427467040","heartbeat_front_addr":"172.21.15.89:6831/2427467040","state":["exists","up"]},{"osd":10,"uuid":"d8f29890-3064-4b67-b649-467a0ac86db7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6825","nonce":2328678744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6827","nonce":2328678744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6831","nonce":2328678744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6829","nonce":2328678744}]},"public_addr":"172.21.15.111:6825/2328678744","cluster_addr":"172.21.15.111:6827/2328678744","heartbeat_back_addr":"172.21.15.111:6831/2328678744","heartbeat_front_addr":"172.21.15.111:6829/2328678744","state":["exists","up"]},{"osd":11,"uuid":"92182c73-f139-47ca-a0ef-5046ae9b2310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6825","nonce":3130508827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6827","nonce":3130508827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6831","nonce":3130508827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6829","nonce":3130508827}]},"public_addr":"172.21.15.149:6825/3130508827","cluster_addr":"172.21.15.149:6827/3130508827","heartbeat_back_addr":"172.21.15.149:6831/3130508827","heartbeat_front_addr":"172.21.15.149:6829/3130508827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:09.031810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:33.695124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:58.065510+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:27.592838+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:53.275289+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:18.305677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:47.781535+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:12.889512+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:37.744193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:07.718408+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:33.391027+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:59.013250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/850669510":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/3541027428":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6801/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6800/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/2396468787":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/1609720980":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/995019144":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6801/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6800/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/2179977257":"2024-09-11T08:29:14.554869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:37:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.735+0000 7f850ba00640 1 -- 172.21.15.89:0/1530695089 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f84e80a2c00 msgr2=0x7f84e80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.735+0000 7f850ba00640 1 --2- 172.21.15.89:0/1530695089 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f84e80a2c00 0x7f84e80a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009230 tx=0x7f84f40058d0 comp rx=0 tx=0).stop 2024-09-10T08:37:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.735+0000 7f850ba00640 1 -- 172.21.15.89:0/1530695089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8504106f20 msgr2=0x7f8504074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.735+0000 7f850ba00640 1 --2- 172.21.15.89:0/1530695089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8504106f20 0x7f8504074f70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009fd0 tx=0x7f84fc00ef10 comp rx=0 tx=0).stop 2024-09-10T08:37:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.735+0000 7f850ba00640 1 -- 172.21.15.89:0/1530695089 shutdown_connections 2024-09-10T08:37:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.735+0000 7f850ba00640 1 --2- 172.21.15.89:0/1530695089 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f84e80a2c00 0x7f84e80a50c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.735+0000 7f850ba00640 1 --2- 172.21.15.89:0/1530695089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8504106f20 0x7f8504074f70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.735+0000 7f850ba00640 1 --2- 172.21.15.89:0/1530695089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85041027a0 0x7f8504074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.735+0000 7f850ba00640 1 --2- 172.21.15.89:0/1530695089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8504101df0 0x7f85040744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:40.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.735+0000 7f850ba00640 1 -- 172.21.15.89:0/1530695089 >> 172.21.15.89:0/1530695089 conn(0x7f85040fd8b0 msgr2=0x7f85040febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:40.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.735+0000 7f850ba00640 1 -- 172.21.15.89:0/1530695089 shutdown_connections 2024-09-10T08:37:40.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:40.735+0000 7f850ba00640 1 -- 172.21.15.89:0/1530695089 wait complete. 2024-09-10T08:37:40.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:40 smithi149 ceph-mon[36167]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:37:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:40 smithi111 ceph-mon[35220]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:37:41.452 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-10T08:37:41.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:41 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1530695089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:37:41.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-09-10T08:37:41.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-09-10T08:37:41.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-10T08:37:41.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-09-10T08:37:41.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-10T08:37:41.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-09-10T08:37:41.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-09-10T08:37:41.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-10T08:37:41.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2024-09-10T08:37:41.804 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-10T08:37:41.829 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:41.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:41 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1530695089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:37:41.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:41 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-09-10T08:37:41.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:41 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-09-10T08:37:41.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:41 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-10T08:37:41.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:41 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-09-10T08:37:41.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:41 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-10T08:37:41.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:41 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-09-10T08:37:41.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:41 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-09-10T08:37:41.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:41 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-10T08:37:41.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:41 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2024-09-10T08:37:41.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:41 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-10T08:37:42.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:41 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1530695089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:37:42.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:41 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-09-10T08:37:42.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:41 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-09-10T08:37:42.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:41 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-10T08:37:42.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:41 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-09-10T08:37:42.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:41 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-09-10T08:37:42.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:41 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-09-10T08:37:42.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:41 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-09-10T08:37:42.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:41 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-10T08:37:42.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:41 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2024-09-10T08:37:42.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:41 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-10T08:37:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.474+0000 7f53ce200640 1 -- 172.21.15.89:0/1118073872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53c8108f60 msgr2=0x7f53c8110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.474+0000 7f53ce200640 1 --2- 172.21.15.89:0/1118073872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53c8108f60 0x7f53c8110ae0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009ca0 tx=0x7f53c002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:37:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.474+0000 7f53ce200640 1 -- 172.21.15.89:0/1118073872 shutdown_connections 2024-09-10T08:37:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.474+0000 7f53ce200640 1 --2- 172.21.15.89:0/1118073872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53c8108f60 0x7f53c8110ae0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.474+0000 7f53ce200640 1 --2- 172.21.15.89:0/1118073872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53c81047e0 0x7f53c8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.474+0000 7f53ce200640 1 --2- 172.21.15.89:0/1118073872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53c8103e30 0x7f53c8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.474+0000 7f53ce200640 1 -- 172.21.15.89:0/1118073872 >> 172.21.15.89:0/1118073872 conn(0x7f53c80ff910 msgr2=0x7f53c8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.474+0000 7f53ce200640 1 -- 172.21.15.89:0/1118073872 shutdown_connections 2024-09-10T08:37:42.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.474+0000 7f53ce200640 1 -- 172.21.15.89:0/1118073872 wait complete. 2024-09-10T08:37:42.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.475+0000 7f53ce200640 1 Processor -- start 2024-09-10T08:37:42.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.476+0000 7f53ce200640 1 -- start start 2024-09-10T08:37:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.476+0000 7f53ce200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53c8103e30 0x7f53c819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.476+0000 7f53ce200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53c81047e0 0x7f53c819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.476+0000 7f53ce200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53c8108f60 0x7f53c819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.476+0000 7f53ce200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f53c8114010 con 0x7f53c8108f60 2024-09-10T08:37:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.476+0000 7f53ce200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f53c8113e90 con 0x7f53c8103e30 2024-09-10T08:37:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.476+0000 7f53ce200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f53c8114190 con 0x7f53c81047e0 2024-09-10T08:37:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.476+0000 7f53c7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53c81047e0 0x7f53c819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.476+0000 7f53cce00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53c8103e30 0x7f53c819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:42.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.476+0000 7f53cd800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53c8108f60 0x7f53c819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.476+0000 7f53c7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53c81047e0 0x7f53c819a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:55960/0 (socket says 172.21.15.89:55960) 2024-09-10T08:37:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.476+0000 7f53c7e00640 1 -- 172.21.15.89:0/3326580729 learned_addr learned my addr 172.21.15.89:0/3326580729 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.477+0000 7f53c7e00640 1 -- 172.21.15.89:0/3326580729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53c8103e30 msgr2=0x7f53c819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:42.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.477+0000 7f53c7e00640 1 --2- 172.21.15.89:0/3326580729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53c8103e30 0x7f53c819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.477+0000 7f53c7e00640 1 -- 172.21.15.89:0/3326580729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53c8108f60 msgr2=0x7f53c819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.477+0000 7f53c7e00640 1 --2- 172.21.15.89:0/3326580729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53c8108f60 0x7f53c819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.477+0000 7f53c7e00640 1 -- 172.21.15.89:0/3326580729 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c819f3e0 con 0x7f53c81047e0 2024-09-10T08:37:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.477+0000 7f53cd800640 1 --2- 172.21.15.89:0/3326580729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53c8108f60 0x7f53c819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:37:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.477+0000 7f53c7e00640 1 --2- 172.21.15.89:0/3326580729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53c81047e0 0x7f53c819a970 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f53b400e9f0 tx=0x7f53b400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.477+0000 7f53c5600640 1 -- 172.21.15.89:0/3326580729 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b400cd80 con 0x7f53c81047e0 2024-09-10T08:37:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.477+0000 7f53c5600640 1 -- 172.21.15.89:0/3326580729 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53b4005500 con 0x7f53c81047e0 2024-09-10T08:37:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.478+0000 7f53ce200640 1 -- 172.21.15.89:0/3326580729 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c819f6d0 con 0x7f53c81047e0 2024-09-10T08:37:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.478+0000 7f53c5600640 1 -- 172.21.15.89:0/3326580729 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4010690 con 0x7f53c81047e0 2024-09-10T08:37:42.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.478+0000 7f53ce200640 1 -- 172.21.15.89:0/3326580729 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c81a6fb0 con 0x7f53c81047e0 2024-09-10T08:37:42.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.479+0000 7f53ce200640 1 -- 172.21.15.89:0/3326580729 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c8105310 con 0x7f53c81047e0 2024-09-10T08:37:42.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.481+0000 7f53c5600640 1 -- 172.21.15.89:0/3326580729 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f53b40046c0 con 0x7f53c81047e0 2024-09-10T08:37:42.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.482+0000 7f53c5600640 1 --2- 172.21.15.89:0/3326580729 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f53b00a2b80 0x7f53b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.482+0000 7f53c5600640 1 -- 172.21.15.89:0/3326580729 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9092+0+0 (secure 0 0 0) 0x7f53b40cf7a0 con 0x7f53c81047e0 2024-09-10T08:37:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.482+0000 7f53cce00640 1 --2- 172.21.15.89:0/3326580729 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f53b00a2b80 0x7f53b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:42.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.483+0000 7f53cce00640 1 --2- 172.21.15.89:0/3326580729 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f53b00a2b80 0x7f53b00a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009bd0 tx=0x7f53bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:42.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.484+0000 7f53c5600640 1 -- 172.21.15.89:0/3326580729 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f53b409c340 con 0x7f53c81047e0 2024-09-10T08:37:42.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:42.703+0000 7f53ce200640 1 -- 172.21.15.89:0/3326580729 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f53c8104210 con 0x7f53c81047e0 2024-09-10T08:37:42.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:42 smithi149 ceph-mon[36167]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:37:42.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-09-10T08:37:42.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-10T08:37:42.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-10T08:37:42.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-10T08:37:42.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-10T08:37:42.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-09-10T08:37:42.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-09-10T08:37:42.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-10T08:37:42.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2024-09-10T08:37:42.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-10T08:37:42.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:42 smithi149 ceph-mon[36167]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T08:37:43.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:42 smithi111 ceph-mon[35220]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:37:43.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-09-10T08:37:43.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-10T08:37:43.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-10T08:37:43.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-10T08:37:43.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-10T08:37:43.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-09-10T08:37:43.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-09-10T08:37:43.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-10T08:37:43.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2024-09-10T08:37:43.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-10T08:37:43.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:42 smithi111 ceph-mon[35220]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T08:37:43.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:42 smithi089 ceph-mon[30757]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:37:43.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-09-10T08:37:43.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-10T08:37:43.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-10T08:37:43.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-10T08:37:43.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-10T08:37:43.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-09-10T08:37:43.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-09-10T08:37:43.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-10T08:37:43.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2024-09-10T08:37:43.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-10T08:37:43.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:42 smithi089 ceph-mon[30757]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T08:37:43.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:43.681+0000 7f53c5600640 1 -- 172.21.15.89:0/3326580729 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f53b409bc40 con 0x7f53c81047e0 2024-09-10T08:37:43.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:43.683+0000 7f53ce200640 1 -- 172.21.15.89:0/3326580729 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f53b00a2b80 msgr2=0x7f53b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:43.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:43.683+0000 7f53ce200640 1 --2- 172.21.15.89:0/3326580729 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f53b00a2b80 0x7f53b00a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009bd0 tx=0x7f53bc005f90 comp rx=0 tx=0).stop 2024-09-10T08:37:43.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:43.683+0000 7f53ce200640 1 -- 172.21.15.89:0/3326580729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53c81047e0 msgr2=0x7f53c819a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:43.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:43.683+0000 7f53ce200640 1 --2- 172.21.15.89:0/3326580729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53c81047e0 0x7f53c819a970 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f53b400e9f0 tx=0x7f53b400edf0 comp rx=0 tx=0).stop 2024-09-10T08:37:43.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:43.684+0000 7f53ce200640 1 -- 172.21.15.89:0/3326580729 shutdown_connections 2024-09-10T08:37:43.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:43.684+0000 7f53ce200640 1 --2- 172.21.15.89:0/3326580729 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f53b00a2b80 0x7f53b00a5040 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:43.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:43.684+0000 7f53ce200640 1 --2- 172.21.15.89:0/3326580729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53c8108f60 0x7f53c819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:43.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:43.684+0000 7f53ce200640 1 --2- 172.21.15.89:0/3326580729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53c81047e0 0x7f53c819a970 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:43.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:43.684+0000 7f53ce200640 1 --2- 172.21.15.89:0/3326580729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53c8103e30 0x7f53c819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:43.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:43.684+0000 7f53ce200640 1 -- 172.21.15.89:0/3326580729 >> 172.21.15.89:0/3326580729 conn(0x7f53c80ff910 msgr2=0x7f53c8100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:43.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:43.684+0000 7f53ce200640 1 -- 172.21.15.89:0/3326580729 shutdown_connections 2024-09-10T08:37:43.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:43.684+0000 7f53ce200640 1 -- 172.21.15.89:0/3326580729 wait complete. 2024-09-10T08:37:43.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:43 smithi149 ceph-mon[36167]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T08:37:43.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:43 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3326580729' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T08:37:43.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:43 smithi149 ceph-mon[36167]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T08:37:44.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:43 smithi111 ceph-mon[35220]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T08:37:44.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:43 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3326580729' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T08:37:44.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:43 smithi111 ceph-mon[35220]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T08:37:44.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:43 smithi089 ceph-mon[30757]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T08:37:44.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:43 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3326580729' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T08:37:44.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:43 smithi089 ceph-mon[30757]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T08:37:44.496 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-10T08:37:44.496 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-10T08:37:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:44 smithi089 ceph-mon[30757]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:44 smithi089 ceph-mon[30757]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T08:37:44.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:44 smithi089 ceph-mon[30757]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:37:44.865 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:44.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:44 smithi149 ceph-mon[36167]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:44.972 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:44 smithi149 ceph-mon[36167]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T08:37:44.972 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:44 smithi149 ceph-mon[36167]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:37:45.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:44 smithi111 ceph-mon[35220]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:45.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:44 smithi111 ceph-mon[35220]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T08:37:45.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:44 smithi111 ceph-mon[35220]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:37:45.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.462+0000 7fc51aa00640 1 -- 172.21.15.89:0/930187954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5140745c0 msgr2=0x7fc5140749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:45.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.462+0000 7fc51aa00640 1 --2- 172.21.15.89:0/930187954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5140745c0 0x7fc5140749a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc504009900 tx=0x7fc50402a480 comp rx=0 tx=0).stop 2024-09-10T08:37:45.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.462+0000 7fc51aa00640 1 -- 172.21.15.89:0/930187954 shutdown_connections 2024-09-10T08:37:45.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.462+0000 7fc51aa00640 1 --2- 172.21.15.89:0/930187954 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc51410e430 0x7fc514110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:45.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.462+0000 7fc51aa00640 1 --2- 172.21.15.89:0/930187954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc514074ee0 0x7fc51410def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:45.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.462+0000 7fc51aa00640 1 --2- 172.21.15.89:0/930187954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5140745c0 0x7fc5140749a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:45.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.462+0000 7fc51aa00640 1 -- 172.21.15.89:0/930187954 >> 172.21.15.89:0/930187954 conn(0x7fc5140fd8b0 msgr2=0x7fc5140ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:45.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.462+0000 7fc51aa00640 1 -- 172.21.15.89:0/930187954 shutdown_connections 2024-09-10T08:37:45.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.462+0000 7fc51aa00640 1 -- 172.21.15.89:0/930187954 wait complete. 2024-09-10T08:37:45.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.463+0000 7fc51aa00640 1 Processor -- start 2024-09-10T08:37:45.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.463+0000 7fc51aa00640 1 -- start start 2024-09-10T08:37:45.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.464+0000 7fc51aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5140745c0 0x7fc51419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:45.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.464+0000 7fc51aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc514074ee0 0x7fc51419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:45.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.464+0000 7fc51aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc51410e430 0x7fc51419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:45.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.464+0000 7fc51aa00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141141f0 con 0x7fc514074ee0 2024-09-10T08:37:45.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.464+0000 7fc51aa00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc514114070 con 0x7fc5140745c0 2024-09-10T08:37:45.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.464+0000 7fc51aa00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc514114370 con 0x7fc51410e430 2024-09-10T08:37:45.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.464+0000 7fc50be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc514074ee0 0x7fc51419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:45.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.464+0000 7fc513e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5140745c0 0x7fc51419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:45.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.464+0000 7fc50be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc514074ee0 0x7fc51419abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37938/0 (socket says 172.21.15.89:37938) 2024-09-10T08:37:45.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.464+0000 7fc50be00640 1 -- 172.21.15.89:0/2374331456 learned_addr learned my addr 172.21.15.89:0/2374331456 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:45.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.464+0000 7fc518a00640 1 --2- 172.21.15.89:0/2374331456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc51410e430 0x7fc51419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:45.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.464+0000 7fc50be00640 1 -- 172.21.15.89:0/2374331456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc51410e430 msgr2=0x7fc51419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:45.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.464+0000 7fc50be00640 1 --2- 172.21.15.89:0/2374331456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc51410e430 0x7fc51419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:45.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.464+0000 7fc50be00640 1 -- 172.21.15.89:0/2374331456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5140745c0 msgr2=0x7fc51419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:45.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.464+0000 7fc50be00640 1 --2- 172.21.15.89:0/2374331456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5140745c0 0x7fc51419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:45.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.465+0000 7fc50be00640 1 -- 172.21.15.89:0/2374331456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51419f620 con 0x7fc514074ee0 2024-09-10T08:37:45.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.465+0000 7fc50be00640 1 --2- 172.21.15.89:0/2374331456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc514074ee0 0x7fc51419abb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00e9f0 tx=0x7fc4fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:45.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.465+0000 7fc511600640 1 -- 172.21.15.89:0/2374331456 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc00cd80 con 0x7fc514074ee0 2024-09-10T08:37:45.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.465+0000 7fc511600640 1 -- 172.21.15.89:0/2374331456 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4fc005500 con 0x7fc514074ee0 2024-09-10T08:37:45.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.465+0000 7fc511600640 1 -- 172.21.15.89:0/2374331456 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc010690 con 0x7fc514074ee0 2024-09-10T08:37:45.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.465+0000 7fc51aa00640 1 -- 172.21.15.89:0/2374331456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51419f910 con 0x7fc514074ee0 2024-09-10T08:37:45.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.465+0000 7fc51aa00640 1 -- 172.21.15.89:0/2374331456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5141a71f0 con 0x7fc514074ee0 2024-09-10T08:37:45.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.466+0000 7fc51aa00640 1 -- 172.21.15.89:0/2374331456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5140749a0 con 0x7fc514074ee0 2024-09-10T08:37:45.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.468+0000 7fc511600640 1 -- 172.21.15.89:0/2374331456 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc4fc0046c0 con 0x7fc514074ee0 2024-09-10T08:37:45.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.469+0000 7fc511600640 1 --2- 172.21.15.89:0/2374331456 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc4f80a2c00 0x7fc4f80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:45.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.469+0000 7fc511600640 1 -- 172.21.15.89:0/2374331456 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fc4fc0cf830 con 0x7fc514074ee0 2024-09-10T08:37:45.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.470+0000 7fc513e00640 1 --2- 172.21.15.89:0/2374331456 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc4f80a2c00 0x7fc4f80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:45.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.470+0000 7fc513e00640 1 --2- 172.21.15.89:0/2374331456 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc4f80a2c00 0x7fc4f80a50c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc504000c00 tx=0x7fc504005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:45.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.473+0000 7fc511600640 1 -- 172.21.15.89:0/2374331456 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc4fc09c3d0 con 0x7fc514074ee0 2024-09-10T08:37:45.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:45.685+0000 7fc51aa00640 1 -- 172.21.15.89:0/2374331456 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fc5141a7520 con 0x7fc514074ee0 2024-09-10T08:37:46.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:46.686+0000 7fc511600640 1 -- 172.21.15.89:0/2374331456 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7fc4fc09bcd0 con 0x7fc514074ee0 2024-09-10T08:37:46.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:46.689+0000 7fc51aa00640 1 -- 172.21.15.89:0/2374331456 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc4f80a2c00 msgr2=0x7fc4f80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:46.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:46.689+0000 7fc51aa00640 1 --2- 172.21.15.89:0/2374331456 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc4f80a2c00 0x7fc4f80a50c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc504000c00 tx=0x7fc504005b50 comp rx=0 tx=0).stop 2024-09-10T08:37:46.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:46.689+0000 7fc51aa00640 1 -- 172.21.15.89:0/2374331456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc514074ee0 msgr2=0x7fc51419abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:46.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:46.689+0000 7fc51aa00640 1 --2- 172.21.15.89:0/2374331456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc514074ee0 0x7fc51419abb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00e9f0 tx=0x7fc4fc00edf0 comp rx=0 tx=0).stop 2024-09-10T08:37:46.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:46.689+0000 7fc51aa00640 1 -- 172.21.15.89:0/2374331456 shutdown_connections 2024-09-10T08:37:46.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:46.689+0000 7fc51aa00640 1 --2- 172.21.15.89:0/2374331456 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc4f80a2c00 0x7fc4f80a50c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:46.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:46.689+0000 7fc51aa00640 1 --2- 172.21.15.89:0/2374331456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc51410e430 0x7fc51419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:46.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:46.689+0000 7fc51aa00640 1 --2- 172.21.15.89:0/2374331456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc514074ee0 0x7fc51419abb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:46.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:46.689+0000 7fc51aa00640 1 --2- 172.21.15.89:0/2374331456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5140745c0 0x7fc51419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:46.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:46.689+0000 7fc51aa00640 1 -- 172.21.15.89:0/2374331456 >> 172.21.15.89:0/2374331456 conn(0x7fc5140fd8b0 msgr2=0x7fc5140fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:46.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:46.689+0000 7fc51aa00640 1 -- 172.21.15.89:0/2374331456 shutdown_connections 2024-09-10T08:37:46.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:46.689+0000 7fc51aa00640 1 -- 172.21.15.89:0/2374331456 wait complete. 2024-09-10T08:37:46.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:46 smithi149 ceph-mon[36167]: pgmap v326: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:46.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:46 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2374331456' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-10T08:37:47.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:46 smithi111 ceph-mon[35220]: pgmap v326: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:47.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:46 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2374331456' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-10T08:37:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:46 smithi089 ceph-mon[30757]: pgmap v326: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:37:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:46 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2374331456' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-10T08:37:47.480 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-10T08:37:47.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:47 smithi089 ceph-mon[30757]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T08:37:47.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:47 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2374331456' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-10T08:37:47.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:47 smithi089 ceph-mon[30757]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T08:37:47.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:47 smithi089 ceph-mon[30757]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:37:47.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:47 smithi089 ceph-mon[30757]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-10T08:37:47.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:47 smithi089 ceph-mon[30757]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-10T08:37:47.854 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:47.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:47 smithi149 ceph-mon[36167]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T08:37:47.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:47 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2374331456' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-10T08:37:47.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:47 smithi149 ceph-mon[36167]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T08:37:47.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:47 smithi149 ceph-mon[36167]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:37:47.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:47 smithi149 ceph-mon[36167]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-10T08:37:47.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:47 smithi149 ceph-mon[36167]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-10T08:37:48.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:47 smithi111 ceph-mon[35220]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T08:37:48.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:47 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2374331456' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-10T08:37:48.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:47 smithi111 ceph-mon[35220]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T08:37:48.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:47 smithi111 ceph-mon[35220]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:37:48.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:47 smithi111 ceph-mon[35220]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-10T08:37:48.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:47 smithi111 ceph-mon[35220]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-10T08:37:48.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.437+0000 7fe06e400640 1 -- 172.21.15.89:0/29720695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe068101de0 msgr2=0x7fe0681021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:48.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.437+0000 7fe06e400640 1 --2- 172.21.15.89:0/29720695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe068101de0 0x7fe0681021c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe058009900 tx=0x7fe05802a410 comp rx=0 tx=0).stop 2024-09-10T08:37:48.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.437+0000 7fe06e400640 1 -- 172.21.15.89:0/29720695 shutdown_connections 2024-09-10T08:37:48.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.437+0000 7fe06e400640 1 --2- 172.21.15.89:0/29720695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe068106f10 0x7fe06810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:48.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.437+0000 7fe06e400640 1 --2- 172.21.15.89:0/29720695 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe068102790 0x7fe0681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:48.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.437+0000 7fe06e400640 1 --2- 172.21.15.89:0/29720695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe068101de0 0x7fe0681021c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:48.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.437+0000 7fe06e400640 1 -- 172.21.15.89:0/29720695 >> 172.21.15.89:0/29720695 conn(0x7fe0680fd8c0 msgr2=0x7fe0680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:48.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.437+0000 7fe06e400640 1 -- 172.21.15.89:0/29720695 shutdown_connections 2024-09-10T08:37:48.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.437+0000 7fe06e400640 1 -- 172.21.15.89:0/29720695 wait complete. 2024-09-10T08:37:48.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.438+0000 7fe06e400640 1 Processor -- start 2024-09-10T08:37:48.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.438+0000 7fe06e400640 1 -- start start 2024-09-10T08:37:48.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.439+0000 7fe06e400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe068101de0 0x7fe06819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:48.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.439+0000 7fe06e400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe068102790 0x7fe06819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:48.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.439+0000 7fe06e400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe068106f10 0x7fe06819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:48.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.439+0000 7fe06e400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe068111f80 con 0x7fe068102790 2024-09-10T08:37:48.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.439+0000 7fe06e400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe068111e00 con 0x7fe068106f10 2024-09-10T08:37:48.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.439+0000 7fe06e400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe068112100 con 0x7fe068101de0 2024-09-10T08:37:48.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.439+0000 7fe066a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe068102790 0x7fe06819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:48.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.439+0000 7fe066a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe068102790 0x7fe06819ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37954/0 (socket says 172.21.15.89:37954) 2024-09-10T08:37:48.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.439+0000 7fe066a00640 1 -- 172.21.15.89:0/1214895906 learned_addr learned my addr 172.21.15.89:0/1214895906 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:48.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.439+0000 7fe067e00640 1 --2- 172.21.15.89:0/1214895906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe068106f10 0x7fe06819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:48.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.439+0000 7fe067400640 1 --2- 172.21.15.89:0/1214895906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe068101de0 0x7fe06819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:48.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.440+0000 7fe066a00640 1 -- 172.21.15.89:0/1214895906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe068101de0 msgr2=0x7fe06819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:48.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.440+0000 7fe066a00640 1 --2- 172.21.15.89:0/1214895906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe068101de0 0x7fe06819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:48.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.440+0000 7fe066a00640 1 -- 172.21.15.89:0/1214895906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe068106f10 msgr2=0x7fe06819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:48.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.440+0000 7fe066a00640 1 --2- 172.21.15.89:0/1214895906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe068106f10 0x7fe06819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:48.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.440+0000 7fe066a00640 1 -- 172.21.15.89:0/1214895906 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06819f630 con 0x7fe068102790 2024-09-10T08:37:48.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.440+0000 7fe067e00640 1 --2- 172.21.15.89:0/1214895906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe068106f10 0x7fe06819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:37:48.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.440+0000 7fe066a00640 1 --2- 172.21.15.89:0/1214895906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe068102790 0x7fe06819ac60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe05400e9f0 tx=0x7fe05400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:48.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.440+0000 7fe05fe00640 1 -- 172.21.15.89:0/1214895906 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05400cd80 con 0x7fe068102790 2024-09-10T08:37:48.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.440+0000 7fe05fe00640 1 -- 172.21.15.89:0/1214895906 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe054010910 con 0x7fe068102790 2024-09-10T08:37:48.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.440+0000 7fe05fe00640 1 -- 172.21.15.89:0/1214895906 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe054018990 con 0x7fe068102790 2024-09-10T08:37:48.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.440+0000 7fe06e400640 1 -- 172.21.15.89:0/1214895906 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06819f920 con 0x7fe068102790 2024-09-10T08:37:48.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.440+0000 7fe06e400640 1 -- 172.21.15.89:0/1214895906 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0681a7200 con 0x7fe068102790 2024-09-10T08:37:48.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.442+0000 7fe06e400640 1 -- 172.21.15.89:0/1214895906 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0681021c0 con 0x7fe068102790 2024-09-10T08:37:48.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.443+0000 7fe05fe00640 1 -- 172.21.15.89:0/1214895906 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe054010ac0 con 0x7fe068102790 2024-09-10T08:37:48.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.445+0000 7fe05fe00640 1 --2- 172.21.15.89:0/1214895906 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe04c0a2b80 0x7fe04c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:48.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.445+0000 7fe05fe00640 1 -- 172.21.15.89:0/1214895906 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fe0540cf710 con 0x7fe068102790 2024-09-10T08:37:48.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.445+0000 7fe067400640 1 --2- 172.21.15.89:0/1214895906 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe04c0a2b80 0x7fe04c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:48.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.446+0000 7fe067400640 1 --2- 172.21.15.89:0/1214895906 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe04c0a2b80 0x7fe04c0a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe058000c00 tx=0x7fe058005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:48.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.449+0000 7fe05fe00640 1 -- 172.21.15.89:0/1214895906 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe05409bc60 con 0x7fe068102790 2024-09-10T08:37:48.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.658+0000 7fe06e400640 1 -- 172.21.15.89:0/1214895906 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fe0681a7530 con 0x7fe068102790 2024-09-10T08:37:48.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.706+0000 7fe05fe00640 1 -- 172.21.15.89:0/1214895906 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7fe05409ff60 con 0x7fe068102790 2024-09-10T08:37:48.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.709+0000 7fe06e400640 1 -- 172.21.15.89:0/1214895906 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe04c0a2b80 msgr2=0x7fe04c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:48.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.709+0000 7fe06e400640 1 --2- 172.21.15.89:0/1214895906 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe04c0a2b80 0x7fe04c0a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe058000c00 tx=0x7fe058005b50 comp rx=0 tx=0).stop 2024-09-10T08:37:48.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.709+0000 7fe06e400640 1 -- 172.21.15.89:0/1214895906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe068102790 msgr2=0x7fe06819ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:48.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.709+0000 7fe06e400640 1 --2- 172.21.15.89:0/1214895906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe068102790 0x7fe06819ac60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe05400e9f0 tx=0x7fe05400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:37:48.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.709+0000 7fe06e400640 1 -- 172.21.15.89:0/1214895906 shutdown_connections 2024-09-10T08:37:48.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.709+0000 7fe06e400640 1 --2- 172.21.15.89:0/1214895906 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe04c0a2b80 0x7fe04c0a5040 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:48.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.709+0000 7fe06e400640 1 --2- 172.21.15.89:0/1214895906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe068106f10 0x7fe06819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:48.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.709+0000 7fe06e400640 1 --2- 172.21.15.89:0/1214895906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe068102790 0x7fe06819ac60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:48.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.709+0000 7fe06e400640 1 --2- 172.21.15.89:0/1214895906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe068101de0 0x7fe06819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:48.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.709+0000 7fe06e400640 1 -- 172.21.15.89:0/1214895906 >> 172.21.15.89:0/1214895906 conn(0x7fe0680fd8c0 msgr2=0x7fe0680fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:48.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.710+0000 7fe06e400640 1 -- 172.21.15.89:0/1214895906 shutdown_connections 2024-09-10T08:37:48.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:48.710+0000 7fe06e400640 1 -- 172.21.15.89:0/1214895906 wait complete. 2024-09-10T08:37:48.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:48 smithi149 ceph-mon[36167]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T08:37:48.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:48 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] up:active 2024-09-10T08:37:48.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:48 smithi149 ceph-mon[36167]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T08:37:48.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:48 smithi149 ceph-mon[36167]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T08:37:48.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:48 smithi149 ceph-mon[36167]: Cluster is now healthy 2024-09-10T08:37:48.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:48 smithi149 ceph-mon[36167]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-10T08:37:48.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:48 smithi149 ceph-mon[36167]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-10T08:37:48.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:48 smithi149 ceph-mon[36167]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-10T08:37:48.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:48 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1214895906' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T08:37:49.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:48 smithi111 ceph-mon[35220]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T08:37:49.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:48 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] up:active 2024-09-10T08:37:49.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:48 smithi111 ceph-mon[35220]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T08:37:49.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:48 smithi111 ceph-mon[35220]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T08:37:49.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:48 smithi111 ceph-mon[35220]: Cluster is now healthy 2024-09-10T08:37:49.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:48 smithi111 ceph-mon[35220]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-10T08:37:49.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:48 smithi111 ceph-mon[35220]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-10T08:37:49.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:48 smithi111 ceph-mon[35220]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-10T08:37:49.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:48 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1214895906' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T08:37:49.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:48 smithi089 ceph-mon[30757]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T08:37:49.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:48 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] up:active 2024-09-10T08:37:49.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:48 smithi089 ceph-mon[30757]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T08:37:49.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:48 smithi089 ceph-mon[30757]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T08:37:49.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:48 smithi089 ceph-mon[30757]: Cluster is now healthy 2024-09-10T08:37:49.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:48 smithi089 ceph-mon[30757]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-10T08:37:49.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:48 smithi089 ceph-mon[30757]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-10T08:37:49.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:48 smithi089 ceph-mon[30757]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-10T08:37:49.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:48 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1214895906' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T08:37:49.412 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-10T08:37:49.783 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:49.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:49 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1214895906' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T08:37:49.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:49 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] up:active 2024-09-10T08:37:49.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:49 smithi149 ceph-mon[36167]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-10T08:37:49.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:49 smithi149 ceph-mon[36167]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:37:49.991 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:49 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1214895906' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T08:37:49.991 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:49 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] up:active 2024-09-10T08:37:49.991 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:49 smithi089 ceph-mon[30757]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-10T08:37:49.991 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:49 smithi089 ceph-mon[30757]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:37:50.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:49 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1214895906' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T08:37:50.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:49 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] up:active 2024-09-10T08:37:50.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:49 smithi111 ceph-mon[35220]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-10T08:37:50.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:49 smithi111 ceph-mon[35220]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:37:50.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.381+0000 7f49a4200640 1 -- 172.21.15.89:0/2357556911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f499c102d50 msgr2=0x7f499c103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:50.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.381+0000 7f49a4200640 1 --2- 172.21.15.89:0/2357556911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f499c102d50 0x7f499c103130 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4990009900 tx=0x7f499002a490 comp rx=0 tx=0).stop 2024-09-10T08:37:50.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.381+0000 7f49a4200640 1 -- 172.21.15.89:0/2357556911 shutdown_connections 2024-09-10T08:37:50.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.381+0000 7f49a4200640 1 --2- 172.21.15.89:0/2357556911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f499c10c180 0x7f499c10e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:50.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.381+0000 7f49a4200640 1 --2- 172.21.15.89:0/2357556911 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f499c103670 0x7f499c10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:50.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.381+0000 7f49a4200640 1 --2- 172.21.15.89:0/2357556911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f499c102d50 0x7f499c103130 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:50.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.381+0000 7f49a4200640 1 -- 172.21.15.89:0/2357556911 >> 172.21.15.89:0/2357556911 conn(0x7f499c0fc7d0 msgr2=0x7f499c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:50.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.381+0000 7f49a4200640 1 -- 172.21.15.89:0/2357556911 shutdown_connections 2024-09-10T08:37:50.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.381+0000 7f49a4200640 1 -- 172.21.15.89:0/2357556911 wait complete. 2024-09-10T08:37:50.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.382+0000 7f49a4200640 1 Processor -- start 2024-09-10T08:37:50.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.382+0000 7f49a4200640 1 -- start start 2024-09-10T08:37:50.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.382+0000 7f49a4200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f499c102d50 0x7f499c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:50.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.382+0000 7f49a4200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f499c103670 0x7f499c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:50.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.382+0000 7f49a4200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f499c10c180 0x7f499c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:50.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.382+0000 7f49a4200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f499c111fd0 con 0x7f499c10c180 2024-09-10T08:37:50.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.382+0000 7f49a4200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f499c111e50 con 0x7f499c102d50 2024-09-10T08:37:50.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.382+0000 7f49a4200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f499c112150 con 0x7f499c103670 2024-09-10T08:37:50.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.382+0000 7f49a2200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f499c10c180 0x7f499c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:50.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.383+0000 7f49a2200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f499c10c180 0x7f499c19cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37972/0 (socket says 172.21.15.89:37972) 2024-09-10T08:37:50.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.383+0000 7f49a2200640 1 -- 172.21.15.89:0/1612347409 learned_addr learned my addr 172.21.15.89:0/1612347409 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:50.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.383+0000 7f49a0e00640 1 --2- 172.21.15.89:0/1612347409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f499c103670 0x7f499c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:50.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.383+0000 7f49a1800640 1 --2- 172.21.15.89:0/1612347409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f499c102d50 0x7f499c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:50.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.383+0000 7f49a2200640 1 -- 172.21.15.89:0/1612347409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f499c103670 msgr2=0x7f499c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:50.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.383+0000 7f49a2200640 1 --2- 172.21.15.89:0/1612347409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f499c103670 0x7f499c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:50.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.383+0000 7f49a2200640 1 -- 172.21.15.89:0/1612347409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f499c102d50 msgr2=0x7f499c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:50.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.383+0000 7f49a2200640 1 --2- 172.21.15.89:0/1612347409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f499c102d50 0x7f499c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:50.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.383+0000 7f49a2200640 1 -- 172.21.15.89:0/1612347409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499c19d3f0 con 0x7f499c10c180 2024-09-10T08:37:50.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.383+0000 7f49a2200640 1 --2- 172.21.15.89:0/1612347409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f499c10c180 0x7f499c19cd10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f499400eaa0 tx=0x7f499400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:50.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.384+0000 7f499a000640 1 -- 172.21.15.89:0/1612347409 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499400cdb0 con 0x7f499c10c180 2024-09-10T08:37:50.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.384+0000 7f499a000640 1 -- 172.21.15.89:0/1612347409 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4994005020 con 0x7f499c10c180 2024-09-10T08:37:50.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.384+0000 7f499a000640 1 -- 172.21.15.89:0/1612347409 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49940116b0 con 0x7f499c10c180 2024-09-10T08:37:50.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.384+0000 7f49a4200640 1 -- 172.21.15.89:0/1612347409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f499c19d6e0 con 0x7f499c10c180 2024-09-10T08:37:50.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.384+0000 7f49a4200640 1 -- 172.21.15.89:0/1612347409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f499c1a4fc0 con 0x7f499c10c180 2024-09-10T08:37:50.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.385+0000 7f49a4200640 1 -- 172.21.15.89:0/1612347409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f499c103130 con 0x7f499c10c180 2024-09-10T08:37:50.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.387+0000 7f499a000640 1 -- 172.21.15.89:0/1612347409 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4994005560 con 0x7f499c10c180 2024-09-10T08:37:50.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.388+0000 7f499a000640 1 --2- 172.21.15.89:0/1612347409 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f49840a2c00 0x7f49840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:50.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.388+0000 7f499a000640 1 -- 172.21.15.89:0/1612347409 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f49940cf4e0 con 0x7f499c10c180 2024-09-10T08:37:50.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.388+0000 7f49a1800640 1 --2- 172.21.15.89:0/1612347409 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f49840a2c00 0x7f49840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:50.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.389+0000 7f49a1800640 1 --2- 172.21.15.89:0/1612347409 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f49840a2c00 0x7f49840a50c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4990000c00 tx=0x7f4990005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:50.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.392+0000 7f499a000640 1 -- 172.21.15.89:0/1612347409 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4994016030 con 0x7f499c10c180 2024-09-10T08:37:50.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.605+0000 7f49a4200640 1 -- 172.21.15.89:0/1612347409 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f499c02d7c0 con 0x7f499c10c180 2024-09-10T08:37:50.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.718+0000 7f499a000640 1 -- 172.21.15.89:0/1612347409 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f499409ba30 con 0x7f499c10c180 2024-09-10T08:37:50.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.721+0000 7f49a4200640 1 -- 172.21.15.89:0/1612347409 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f49840a2c00 msgr2=0x7f49840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:50.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.721+0000 7f49a4200640 1 --2- 172.21.15.89:0/1612347409 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f49840a2c00 0x7f49840a50c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4990000c00 tx=0x7f4990005b50 comp rx=0 tx=0).stop 2024-09-10T08:37:50.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.721+0000 7f49a4200640 1 -- 172.21.15.89:0/1612347409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f499c10c180 msgr2=0x7f499c19cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:50.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.721+0000 7f49a4200640 1 --2- 172.21.15.89:0/1612347409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f499c10c180 0x7f499c19cd10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f499400eaa0 tx=0x7f499400ef60 comp rx=0 tx=0).stop 2024-09-10T08:37:50.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.721+0000 7f49a4200640 1 -- 172.21.15.89:0/1612347409 shutdown_connections 2024-09-10T08:37:50.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.721+0000 7f49a4200640 1 --2- 172.21.15.89:0/1612347409 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f49840a2c00 0x7f49840a50c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:50.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.721+0000 7f49a4200640 1 --2- 172.21.15.89:0/1612347409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f499c10c180 0x7f499c19cd10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:50.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.721+0000 7f49a4200640 1 --2- 172.21.15.89:0/1612347409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f499c103670 0x7f499c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:50.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.721+0000 7f49a4200640 1 --2- 172.21.15.89:0/1612347409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f499c102d50 0x7f499c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:50.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.721+0000 7f49a4200640 1 -- 172.21.15.89:0/1612347409 >> 172.21.15.89:0/1612347409 conn(0x7f499c0fc7d0 msgr2=0x7f499c0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:50.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.721+0000 7f49a4200640 1 -- 172.21.15.89:0/1612347409 shutdown_connections 2024-09-10T08:37:50.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:50.721+0000 7f49a4200640 1 -- 172.21.15.89:0/1612347409 wait complete. 2024-09-10T08:37:51.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:50 smithi111 ceph-mon[35220]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T08:37:51.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:50 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1612347409' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T08:37:51.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:50 smithi089 ceph-mon[30757]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T08:37:51.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:50 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1612347409' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T08:37:51.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:50 smithi149 ceph-mon[36167]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T08:37:51.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:50 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1612347409' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T08:37:51.455 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-10T08:37:51.455 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-10T08:37:51.824 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:51.993 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:51 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1612347409' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T08:37:51.993 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:51 smithi089 ceph-mon[30757]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:37:52.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:51 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1612347409' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T08:37:52.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:51 smithi111 ceph-mon[35220]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:37:52.122 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:51 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1612347409' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T08:37:52.122 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:51 smithi149 ceph-mon[36167]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:37:52.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.385+0000 7f051b200640 1 -- 172.21.15.89:0/3817238931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c10e260 msgr2=0x7f051c110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:52.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.385+0000 7f051b200640 1 --2- 172.21.15.89:0/3817238931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c10e260 0x7f051c110650 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0514009ca0 tx=0x7f051402a950 comp rx=0 tx=0).stop 2024-09-10T08:37:52.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.385+0000 7f051b200640 1 -- 172.21.15.89:0/3817238931 shutdown_connections 2024-09-10T08:37:52.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.385+0000 7f051b200640 1 --2- 172.21.15.89:0/3817238931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c10e260 0x7f051c110650 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:52.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.385+0000 7f051b200640 1 --2- 172.21.15.89:0/3817238931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f051c074ee0 0x7f051c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:52.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.385+0000 7f051b200640 1 --2- 172.21.15.89:0/3817238931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f051c0745c0 0x7f051c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:52.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.385+0000 7f051b200640 1 -- 172.21.15.89:0/3817238931 >> 172.21.15.89:0/3817238931 conn(0x7f051c0fd740 msgr2=0x7f051c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:52.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.385+0000 7f051b200640 1 -- 172.21.15.89:0/3817238931 shutdown_connections 2024-09-10T08:37:52.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.385+0000 7f051b200640 1 -- 172.21.15.89:0/3817238931 wait complete. 2024-09-10T08:37:52.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.386+0000 7f051b200640 1 Processor -- start 2024-09-10T08:37:52.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.386+0000 7f051b200640 1 -- start start 2024-09-10T08:37:52.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.387+0000 7f051b200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f051c0745c0 0x7f051c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:52.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.387+0000 7f051b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f051c074ee0 0x7f051c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:52.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.387+0000 7f051b200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c10e260 0x7f051c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:52.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.387+0000 7f051b200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f051c113fc0 con 0x7f051c10e260 2024-09-10T08:37:52.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.387+0000 7f051b200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f051c113e40 con 0x7f051c0745c0 2024-09-10T08:37:52.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.387+0000 7f051b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f051c114140 con 0x7f051c074ee0 2024-09-10T08:37:52.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.387+0000 7f0519e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f051c0745c0 0x7f051c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:52.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.387+0000 7f051a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c10e260 0x7f051c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:52.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.387+0000 7f0519e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f051c0745c0 0x7f051c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:38812/0 (socket says 172.21.15.89:38812) 2024-09-10T08:37:52.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.387+0000 7f0519e00640 1 -- 172.21.15.89:0/186537404 learned_addr learned my addr 172.21.15.89:0/186537404 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:52.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.387+0000 7f0513e00640 1 --2- 172.21.15.89:0/186537404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f051c074ee0 0x7f051c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:52.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.387+0000 7f0519e00640 1 -- 172.21.15.89:0/186537404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f051c074ee0 msgr2=0x7f051c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:52.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.387+0000 7f0519e00640 1 --2- 172.21.15.89:0/186537404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f051c074ee0 0x7f051c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:52.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.387+0000 7f0519e00640 1 -- 172.21.15.89:0/186537404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c10e260 msgr2=0x7f051c19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:52.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.387+0000 7f0519e00640 1 --2- 172.21.15.89:0/186537404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c10e260 0x7f051c19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:52.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.387+0000 7f0519e00640 1 -- 172.21.15.89:0/186537404 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051c19f420 con 0x7f051c0745c0 2024-09-10T08:37:52.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.388+0000 7f0519e00640 1 --2- 172.21.15.89:0/186537404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f051c0745c0 0x7f051c19a470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f050c00e9f0 tx=0x7f050c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:52.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.388+0000 7f0512a00640 1 -- 172.21.15.89:0/186537404 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c00cd80 con 0x7f051c0745c0 2024-09-10T08:37:52.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.388+0000 7f0512a00640 1 -- 172.21.15.89:0/186537404 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f050c010910 con 0x7f051c0745c0 2024-09-10T08:37:52.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.388+0000 7f051b200640 1 -- 172.21.15.89:0/186537404 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f051c19f710 con 0x7f051c0745c0 2024-09-10T08:37:52.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.388+0000 7f0512a00640 1 -- 172.21.15.89:0/186537404 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c0188f0 con 0x7f051c0745c0 2024-09-10T08:37:52.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.388+0000 7f051b200640 1 -- 172.21.15.89:0/186537404 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f051c1a6ff0 con 0x7f051c0745c0 2024-09-10T08:37:52.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.389+0000 7f051b200640 1 -- 172.21.15.89:0/186537404 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f051c0749a0 con 0x7f051c0745c0 2024-09-10T08:37:52.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.391+0000 7f0512a00640 1 -- 172.21.15.89:0/186537404 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f050c010ac0 con 0x7f051c0745c0 2024-09-10T08:37:52.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.392+0000 7f0512a00640 1 --2- 172.21.15.89:0/186537404 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f05000a2b80 0x7f05000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:52.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.393+0000 7f0512a00640 1 -- 172.21.15.89:0/186537404 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f050c0cf7b0 con 0x7f051c0745c0 2024-09-10T08:37:52.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.393+0000 7f0513e00640 1 --2- 172.21.15.89:0/186537404 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f05000a2b80 0x7f05000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:52.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.393+0000 7f0513e00640 1 --2- 172.21.15.89:0/186537404 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f05000a2b80 0x7f05000a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f051c10f510 tx=0x7f0504005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:52.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.397+0000 7f0512a00640 1 -- 172.21.15.89:0/186537404 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f050c09bd00 con 0x7f051c0745c0 2024-09-10T08:37:52.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.596+0000 7f051b200640 1 -- 172.21.15.89:0/186537404 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f051c062630 con 0x7f05000a2b80 2024-09-10T08:37:52.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.622+0000 7f0512a00640 1 -- 172.21.15.89:0/186537404 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f051c062630 con 0x7f05000a2b80 2024-09-10T08:37:52.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.625+0000 7f051b200640 1 -- 172.21.15.89:0/186537404 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f05000a2b80 msgr2=0x7f05000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:52.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.625+0000 7f051b200640 1 --2- 172.21.15.89:0/186537404 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f05000a2b80 0x7f05000a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f051c10f510 tx=0x7f0504005f90 comp rx=0 tx=0).stop 2024-09-10T08:37:52.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.625+0000 7f051b200640 1 -- 172.21.15.89:0/186537404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f051c0745c0 msgr2=0x7f051c19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:52.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.625+0000 7f051b200640 1 --2- 172.21.15.89:0/186537404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f051c0745c0 0x7f051c19a470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f050c00e9f0 tx=0x7f050c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:37:52.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.625+0000 7f051b200640 1 -- 172.21.15.89:0/186537404 shutdown_connections 2024-09-10T08:37:52.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.625+0000 7f051b200640 1 --2- 172.21.15.89:0/186537404 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f05000a2b80 0x7f05000a5040 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:52.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.625+0000 7f051b200640 1 --2- 172.21.15.89:0/186537404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f051c10e260 0x7f051c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:52.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.625+0000 7f051b200640 1 --2- 172.21.15.89:0/186537404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f051c074ee0 0x7f051c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:52.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.625+0000 7f051b200640 1 --2- 172.21.15.89:0/186537404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f051c0745c0 0x7f051c19a470 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:52.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.625+0000 7f051b200640 1 -- 172.21.15.89:0/186537404 >> 172.21.15.89:0/186537404 conn(0x7f051c0fd740 msgr2=0x7f051c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:52.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.625+0000 7f051b200640 1 -- 172.21.15.89:0/186537404 shutdown_connections 2024-09-10T08:37:52.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:52.625+0000 7f051b200640 1 -- 172.21.15.89:0/186537404 wait complete. 2024-09-10T08:37:53.028 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:52 smithi089 ceph-mon[30757]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-09-10T08:37:53.028 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:52 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:37:53.029 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:37:52 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:37:52.602+0000 7f5303200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:37:53.029 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:37:52 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:37:52.602+0000 7f5303200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:37:53.029 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:37:52 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:37:52.603+0000 7f5303200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:37:53.029 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:37:52 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:37:52.603+0000 7f5303200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:37:53.029 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:37:52 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:37:52.603+0000 7f5303200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:37:53.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:52 smithi111 ceph-mon[35220]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-09-10T08:37:53.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:52 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:37:53.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:52 smithi149 ceph-mon[36167]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-09-10T08:37:53.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:52 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:37:53.401 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-10T08:37:54.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:53 smithi111 ceph-mon[35220]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:54.043 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:53 smithi089 ceph-mon[30757]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:54.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:53 smithi149 ceph-mon[36167]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:54.449 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:54.923 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:54 smithi089 ceph-mon[30757]: pgmap v330: 129 pgs: 129 active+clean; 584 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-10T08:37:54.923 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:54 smithi089 ceph-mon[30757]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-10T08:37:54.923 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:54.923 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:54.923 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:54.923 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:54 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:55.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.039+0000 7fe10e200640 1 -- 172.21.15.89:0/1588939525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe108109140 msgr2=0x7fe108110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:55.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.039+0000 7fe10e200640 1 --2- 172.21.15.89:0/1588939525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe108109140 0x7fe108110cc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe100009ca0 tx=0x7fe10002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:37:55.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.039+0000 7fe10e200640 1 -- 172.21.15.89:0/1588939525 shutdown_connections 2024-09-10T08:37:55.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.039+0000 7fe10e200640 1 --2- 172.21.15.89:0/1588939525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe108109140 0x7fe108110cc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:55.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.039+0000 7fe10e200640 1 --2- 172.21.15.89:0/1588939525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1081049c0 0x7fe108108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:55.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.039+0000 7fe10e200640 1 --2- 172.21.15.89:0/1588939525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe108104010 0x7fe1081043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:55.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.039+0000 7fe10e200640 1 -- 172.21.15.89:0/1588939525 >> 172.21.15.89:0/1588939525 conn(0x7fe1080ffa50 msgr2=0x7fe108101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:55.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.039+0000 7fe10e200640 1 -- 172.21.15.89:0/1588939525 shutdown_connections 2024-09-10T08:37:55.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.039+0000 7fe10e200640 1 -- 172.21.15.89:0/1588939525 wait complete. 2024-09-10T08:37:55.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.040+0000 7fe10e200640 1 Processor -- start 2024-09-10T08:37:55.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.040+0000 7fe10e200640 1 -- start start 2024-09-10T08:37:55.050 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:54 smithi111 ceph-mon[35220]: pgmap v330: 129 pgs: 129 active+clean; 584 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-10T08:37:55.050 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:54 smithi111 ceph-mon[35220]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-10T08:37:55.050 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:55.050 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:55.050 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:55.050 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:54 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:55.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.041+0000 7fe10e200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe108104010 0x7fe10819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:55.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.041+0000 7fe10e200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1081049c0 0x7fe10819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:55.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.041+0000 7fe10e200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe108109140 0x7fe10819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:55.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.041+0000 7fe10e200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe108114150 con 0x7fe108109140 2024-09-10T08:37:55.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.041+0000 7fe10e200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe108113fd0 con 0x7fe1081049c0 2024-09-10T08:37:55.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.041+0000 7fe10e200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081142d0 con 0x7fe108104010 2024-09-10T08:37:55.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.041+0000 7fe0fea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1081049c0 0x7fe10819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:55.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.041+0000 7fe107e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe108109140 0x7fe10819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.041+0000 7fe0fea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1081049c0 0x7fe10819ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:44794/0 (socket says 172.21.15.89:44794) 2024-09-10T08:37:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.041+0000 7fe0fea00640 1 -- 172.21.15.89:0/911369545 learned_addr learned my addr 172.21.15.89:0/911369545 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.041+0000 7fe107400640 1 --2- 172.21.15.89:0/911369545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe108104010 0x7fe10819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.042+0000 7fe0fea00640 1 -- 172.21.15.89:0/911369545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe108104010 msgr2=0x7fe10819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.042+0000 7fe0fea00640 1 --2- 172.21.15.89:0/911369545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe108104010 0x7fe10819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.042+0000 7fe0fea00640 1 -- 172.21.15.89:0/911369545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe108109140 msgr2=0x7fe10819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.042+0000 7fe0fea00640 1 --2- 172.21.15.89:0/911369545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe108109140 0x7fe10819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.042+0000 7fe0fea00640 1 -- 172.21.15.89:0/911369545 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10819f5a0 con 0x7fe1081049c0 2024-09-10T08:37:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.042+0000 7fe107400640 1 --2- 172.21.15.89:0/911369545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe108104010 0x7fe10819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:37:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.042+0000 7fe0fea00640 1 --2- 172.21.15.89:0/911369545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1081049c0 0x7fe10819ab30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000e9f0 tx=0x7fe0f000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.042+0000 7fe104c00640 1 -- 172.21.15.89:0/911369545 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f000cd80 con 0x7fe1081049c0 2024-09-10T08:37:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.042+0000 7fe10e200640 1 -- 172.21.15.89:0/911369545 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe10819f890 con 0x7fe1081049c0 2024-09-10T08:37:55.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.042+0000 7fe104c00640 1 -- 172.21.15.89:0/911369545 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0f0010910 con 0x7fe1081049c0 2024-09-10T08:37:55.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.042+0000 7fe104c00640 1 -- 172.21.15.89:0/911369545 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f00188f0 con 0x7fe1081049c0 2024-09-10T08:37:55.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.042+0000 7fe10e200640 1 -- 172.21.15.89:0/911369545 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1081a7170 con 0x7fe1081049c0 2024-09-10T08:37:55.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.043+0000 7fe10e200640 1 -- 172.21.15.89:0/911369545 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1081043f0 con 0x7fe1081049c0 2024-09-10T08:37:55.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.045+0000 7fe104c00640 1 -- 172.21.15.89:0/911369545 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe0f0010ac0 con 0x7fe1081049c0 2024-09-10T08:37:55.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.046+0000 7fe104c00640 1 --2- 172.21.15.89:0/911369545 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe0ec0a2c10 0x7fe0ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:55.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.047+0000 7fe104c00640 1 -- 172.21.15.89:0/911369545 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fe0f00cfe30 con 0x7fe1081049c0 2024-09-10T08:37:55.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.047+0000 7fe107400640 1 --2- 172.21.15.89:0/911369545 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe0ec0a2c10 0x7fe0ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:55.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.047+0000 7fe107400640 1 --2- 172.21.15.89:0/911369545 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe0ec0a2c10 0x7fe0ec0a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009c20 tx=0x7fe0f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:55.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.049+0000 7fe104c00640 1 -- 172.21.15.89:0/911369545 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe0f009c870 con 0x7fe1081049c0 2024-09-10T08:37:55.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:54 smithi149 ceph-mon[36167]: pgmap v330: 129 pgs: 129 active+clean; 584 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-10T08:37:55.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:54 smithi149 ceph-mon[36167]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-10T08:37:55.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:55.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:55.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:55.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:54 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:55.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.262+0000 7fe10e200640 1 -- 172.21.15.89:0/911369545 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe108062630 con 0x7fe0ec0a2c10 2024-09-10T08:37:55.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.288+0000 7fe104c00640 1 -- 172.21.15.89:0/911369545 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe108062630 con 0x7fe0ec0a2c10 2024-09-10T08:37:55.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.290+0000 7fe10e200640 1 -- 172.21.15.89:0/911369545 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe0ec0a2c10 msgr2=0x7fe0ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:55.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.290+0000 7fe10e200640 1 --2- 172.21.15.89:0/911369545 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe0ec0a2c10 0x7fe0ec0a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009c20 tx=0x7fe0f8005f90 comp rx=0 tx=0).stop 2024-09-10T08:37:55.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.290+0000 7fe10e200640 1 -- 172.21.15.89:0/911369545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1081049c0 msgr2=0x7fe10819ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:55.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.290+0000 7fe10e200640 1 --2- 172.21.15.89:0/911369545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1081049c0 0x7fe10819ab30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000e9f0 tx=0x7fe0f000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:37:55.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.290+0000 7fe10e200640 1 -- 172.21.15.89:0/911369545 shutdown_connections 2024-09-10T08:37:55.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.290+0000 7fe10e200640 1 --2- 172.21.15.89:0/911369545 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe0ec0a2c10 0x7fe0ec0a50d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:55.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.290+0000 7fe10e200640 1 --2- 172.21.15.89:0/911369545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe108109140 0x7fe10819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:55.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.290+0000 7fe10e200640 1 --2- 172.21.15.89:0/911369545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1081049c0 0x7fe10819ab30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:55.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.290+0000 7fe10e200640 1 --2- 172.21.15.89:0/911369545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe108104010 0x7fe10819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:55.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.290+0000 7fe10e200640 1 -- 172.21.15.89:0/911369545 >> 172.21.15.89:0/911369545 conn(0x7fe1080ffa50 msgr2=0x7fe108100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:55.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.290+0000 7fe10e200640 1 -- 172.21.15.89:0/911369545 shutdown_connections 2024-09-10T08:37:55.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:55.290+0000 7fe10e200640 1 -- 172.21.15.89:0/911369545 wait complete. 2024-09-10T08:37:56.050 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-10T08:37:56.444 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:56.920 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:56 smithi089 ceph-mon[30757]: from='client.24734 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:56.920 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:56 smithi089 ceph-mon[30757]: pgmap v331: 129 pgs: 129 active+clean; 590 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-10T08:37:56.920 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:56 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:56.920 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:56 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:57.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.037+0000 7f69eec00640 1 -- 172.21.15.89:0/2259324293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69e8102bf0 msgr2=0x7f69e8102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:57.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.037+0000 7f69eec00640 1 --2- 172.21.15.89:0/2259324293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69e8102bf0 0x7f69e8102fd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009900 tx=0x7f69dc02a480 comp rx=0 tx=0).stop 2024-09-10T08:37:57.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.038+0000 7f69eec00640 1 -- 172.21.15.89:0/2259324293 shutdown_connections 2024-09-10T08:37:57.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.038+0000 7f69eec00640 1 --2- 172.21.15.89:0/2259324293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69e810bfd0 0x7f69e810e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:57.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.038+0000 7f69eec00640 1 --2- 172.21.15.89:0/2259324293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69e8103510 0x7f69e810ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:57.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.038+0000 7f69eec00640 1 --2- 172.21.15.89:0/2259324293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69e8102bf0 0x7f69e8102fd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:57.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.038+0000 7f69eec00640 1 -- 172.21.15.89:0/2259324293 >> 172.21.15.89:0/2259324293 conn(0x7f69e80fc610 msgr2=0x7f69e80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:57.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.038+0000 7f69eec00640 1 -- 172.21.15.89:0/2259324293 shutdown_connections 2024-09-10T08:37:57.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.038+0000 7f69eec00640 1 -- 172.21.15.89:0/2259324293 wait complete. 2024-09-10T08:37:57.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.039+0000 7f69eec00640 1 Processor -- start 2024-09-10T08:37:57.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.039+0000 7f69eec00640 1 -- start start 2024-09-10T08:37:57.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.039+0000 7f69eec00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69e8102bf0 0x7f69e81982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:57.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.039+0000 7f69eec00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69e8103510 0x7f69e81987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:57.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.039+0000 7f69eec00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69e810bfd0 0x7f69e819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:57.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.039+0000 7f69eec00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f69e8111db0 con 0x7f69e8102bf0 2024-09-10T08:37:57.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.039+0000 7f69eec00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f69e8111c30 con 0x7f69e810bfd0 2024-09-10T08:37:57.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.039+0000 7f69eec00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f69e8111f30 con 0x7f69e8103510 2024-09-10T08:37:57.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.040+0000 7f69ed800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69e8102bf0 0x7f69e81982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:57.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.040+0000 7f69ed800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69e8102bf0 0x7f69e81982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49768/0 (socket says 172.21.15.89:49768) 2024-09-10T08:37:57.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.040+0000 7f69ed800640 1 -- 172.21.15.89:0/2241268803 learned_addr learned my addr 172.21.15.89:0/2241268803 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:57.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.040+0000 7f69ece00640 1 --2- 172.21.15.89:0/2241268803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69e8103510 0x7f69e81987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:57.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.040+0000 7f69ee200640 1 --2- 172.21.15.89:0/2241268803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69e810bfd0 0x7f69e819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:57.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.040+0000 7f69ed800640 1 -- 172.21.15.89:0/2241268803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69e8103510 msgr2=0x7f69e81987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:57.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.040+0000 7f69ed800640 1 --2- 172.21.15.89:0/2241268803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69e8103510 0x7f69e81987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:57.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.040+0000 7f69ed800640 1 -- 172.21.15.89:0/2241268803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69e810bfd0 msgr2=0x7f69e819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:57.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.040+0000 7f69ed800640 1 --2- 172.21.15.89:0/2241268803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69e810bfd0 0x7f69e819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:57.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.040+0000 7f69ed800640 1 -- 172.21.15.89:0/2241268803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e819d260 con 0x7f69e8102bf0 2024-09-10T08:37:57.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.041+0000 7f69ed800640 1 --2- 172.21.15.89:0/2241268803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69e8102bf0 0x7f69e81982b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f69dc006010 tx=0x7f69dc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:57.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.041+0000 7f69e6000640 1 -- 172.21.15.89:0/2241268803 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc03f030 con 0x7f69e8102bf0 2024-09-10T08:37:57.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.041+0000 7f69e6000640 1 -- 172.21.15.89:0/2241268803 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69dc003ce0 con 0x7f69e8102bf0 2024-09-10T08:37:57.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.041+0000 7f69e6000640 1 -- 172.21.15.89:0/2241268803 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc035070 con 0x7f69e8102bf0 2024-09-10T08:37:57.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.041+0000 7f69eec00640 1 -- 172.21.15.89:0/2241268803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e819d4f0 con 0x7f69e8102bf0 2024-09-10T08:37:57.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.041+0000 7f69eec00640 1 -- 172.21.15.89:0/2241268803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e819d980 con 0x7f69e8102bf0 2024-09-10T08:37:57.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:56 smithi111 ceph-mon[35220]: from='client.24734 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:57.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:56 smithi111 ceph-mon[35220]: pgmap v331: 129 pgs: 129 active+clean; 590 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-10T08:37:57.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:56 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:57.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:56 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:57.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.042+0000 7f69eec00640 1 -- 172.21.15.89:0/2241268803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e8102fd0 con 0x7f69e8102bf0 2024-09-10T08:37:57.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.045+0000 7f69e6000640 1 -- 172.21.15.89:0/2241268803 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f69dc042030 con 0x7f69e8102bf0 2024-09-10T08:37:57.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.046+0000 7f69e6000640 1 --2- 172.21.15.89:0/2241268803 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f69d00a2c10 0x7f69d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:57.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.046+0000 7f69e6000640 1 -- 172.21.15.89:0/2241268803 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f69dc0efc80 con 0x7f69e8102bf0 2024-09-10T08:37:57.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.046+0000 7f69ece00640 1 --2- 172.21.15.89:0/2241268803 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f69d00a2c10 0x7f69d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:57.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.047+0000 7f69ece00640 1 --2- 172.21.15.89:0/2241268803 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f69d00a2c10 0x7f69d00a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f69e810d480 tx=0x7f69d400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:57.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.048+0000 7f69e6000640 1 -- 172.21.15.89:0/2241268803 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f69dc0bc130 con 0x7f69e8102bf0 2024-09-10T08:37:57.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:56 smithi149 ceph-mon[36167]: from='client.24734 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:57.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:56 smithi149 ceph-mon[36167]: pgmap v331: 129 pgs: 129 active+clean; 590 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-10T08:37:57.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:56 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:57.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:56 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:57.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.300+0000 7f69eec00640 1 -- 172.21.15.89:0/2241268803 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f69e8062630 con 0x7f69d00a2c10 2024-09-10T08:37:57.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.306+0000 7f69e6000640 1 -- 172.21.15.89:0/2241268803 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f69e8062630 con 0x7f69d00a2c10 2024-09-10T08:37:57.306 INFO:teuthology.orchestra.run.smithi089.stdout:/volumes/qa/sv_0/91bf54a2-753b-4868-a57c-3240db175ed9 2024-09-10T08:37:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.308+0000 7f69eec00640 1 -- 172.21.15.89:0/2241268803 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f69d00a2c10 msgr2=0x7f69d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.308+0000 7f69eec00640 1 --2- 172.21.15.89:0/2241268803 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f69d00a2c10 0x7f69d00a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f69e810d480 tx=0x7f69d400b390 comp rx=0 tx=0).stop 2024-09-10T08:37:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.308+0000 7f69eec00640 1 -- 172.21.15.89:0/2241268803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69e8102bf0 msgr2=0x7f69e81982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.308+0000 7f69eec00640 1 --2- 172.21.15.89:0/2241268803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69e8102bf0 0x7f69e81982b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f69dc006010 tx=0x7f69dc0042f0 comp rx=0 tx=0).stop 2024-09-10T08:37:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.308+0000 7f69eec00640 1 -- 172.21.15.89:0/2241268803 shutdown_connections 2024-09-10T08:37:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.308+0000 7f69eec00640 1 --2- 172.21.15.89:0/2241268803 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f69d00a2c10 0x7f69d00a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.308+0000 7f69eec00640 1 --2- 172.21.15.89:0/2241268803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69e810bfd0 0x7f69e819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.308+0000 7f69eec00640 1 --2- 172.21.15.89:0/2241268803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69e8103510 0x7f69e81987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.308+0000 7f69eec00640 1 --2- 172.21.15.89:0/2241268803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69e8102bf0 0x7f69e81982b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.308+0000 7f69eec00640 1 -- 172.21.15.89:0/2241268803 >> 172.21.15.89:0/2241268803 conn(0x7f69e80fc610 msgr2=0x7f69e80fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.308+0000 7f69eec00640 1 -- 172.21.15.89:0/2241268803 shutdown_connections 2024-09-10T08:37:57.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:57.308+0000 7f69eec00640 1 -- 172.21.15.89:0/2241268803 wait complete. 2024-09-10T08:37:58.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:57 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:37:58.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:57 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:37:58.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:57 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:58.044 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:57 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:37:58.044 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:57 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:37:58.044 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:57 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:58.079 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-10T08:37:58.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:57 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:37:58.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:57 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:37:58.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:57 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:37:58.445 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:58.923 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:58 smithi089 ceph-mon[30757]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:58.923 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:37:58 smithi089 ceph-mon[30757]: pgmap v332: 129 pgs: 129 active+clean; 591 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-10T08:37:59.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.036+0000 7f6f62400640 1 -- 172.21.15.89:0/2372466276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f5c104970 msgr2=0x7f6f5c1089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:59.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.036+0000 7f6f62400640 1 --2- 172.21.15.89:0/2372466276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f5c104970 0x7f6f5c1089c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009900 tx=0x7f6f4402a410 comp rx=0 tx=0).stop 2024-09-10T08:37:59.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.036+0000 7f6f62400640 1 -- 172.21.15.89:0/2372466276 shutdown_connections 2024-09-10T08:37:59.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.036+0000 7f6f62400640 1 --2- 172.21.15.89:0/2372466276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f5c1090f0 0x7f6f5c110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:59.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.036+0000 7f6f62400640 1 --2- 172.21.15.89:0/2372466276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f5c104970 0x7f6f5c1089c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:59.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.036+0000 7f6f62400640 1 --2- 172.21.15.89:0/2372466276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f5c103fc0 0x7f6f5c1043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:59.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.036+0000 7f6f62400640 1 -- 172.21.15.89:0/2372466276 >> 172.21.15.89:0/2372466276 conn(0x7f6f5c0ffa20 msgr2=0x7f6f5c101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:59.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.036+0000 7f6f62400640 1 -- 172.21.15.89:0/2372466276 shutdown_connections 2024-09-10T08:37:59.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.036+0000 7f6f62400640 1 -- 172.21.15.89:0/2372466276 wait complete. 2024-09-10T08:37:59.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.037+0000 7f6f62400640 1 Processor -- start 2024-09-10T08:37:59.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.037+0000 7f6f62400640 1 -- start start 2024-09-10T08:37:59.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.037+0000 7f6f62400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f5c103fc0 0x7f6f5c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:59.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.037+0000 7f6f62400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f5c104970 0x7f6f5c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:59.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.038+0000 7f6f62400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f5c1090f0 0x7f6f5c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:59.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.038+0000 7f6f62400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c114110 con 0x7f6f5c1090f0 2024-09-10T08:37:59.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.038+0000 7f6f62400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c113f90 con 0x7f6f5c104970 2024-09-10T08:37:59.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.038+0000 7f6f62400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5c114290 con 0x7f6f5c103fc0 2024-09-10T08:37:59.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.038+0000 7f6f5be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f5c1090f0 0x7f6f5c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:59.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.038+0000 7f6f5be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f5c1090f0 0x7f6f5c19eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49778/0 (socket says 172.21.15.89:49778) 2024-09-10T08:37:59.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.038+0000 7f6f5be00640 1 -- 172.21.15.89:0/2926204140 learned_addr learned my addr 172.21.15.89:0/2926204140 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:37:59.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.038+0000 7f6f52a00640 1 --2- 172.21.15.89:0/2926204140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f5c104970 0x7f6f5c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:59.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.038+0000 7f6f5b400640 1 --2- 172.21.15.89:0/2926204140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f5c103fc0 0x7f6f5c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:59.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.038+0000 7f6f5be00640 1 -- 172.21.15.89:0/2926204140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f5c103fc0 msgr2=0x7f6f5c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:59.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.038+0000 7f6f5be00640 1 --2- 172.21.15.89:0/2926204140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f5c103fc0 0x7f6f5c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:59.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.038+0000 7f6f5be00640 1 -- 172.21.15.89:0/2926204140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f5c104970 msgr2=0x7f6f5c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:59.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.038+0000 7f6f5be00640 1 --2- 172.21.15.89:0/2926204140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f5c104970 0x7f6f5c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:59.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.038+0000 7f6f5be00640 1 -- 172.21.15.89:0/2926204140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f5c19f5b0 con 0x7f6f5c1090f0 2024-09-10T08:37:59.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.038+0000 7f6f52a00640 1 --2- 172.21.15.89:0/2926204140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f5c104970 0x7f6f5c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:37:59.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.039+0000 7f6f5be00640 1 --2- 172.21.15.89:0/2926204140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f5c1090f0 0x7f6f5c19eed0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400eb10 tx=0x7f6f5400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:59.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.039+0000 7f6f58c00640 1 -- 172.21.15.89:0/2926204140 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f54004b20 con 0x7f6f5c1090f0 2024-09-10T08:37:59.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.039+0000 7f6f62400640 1 -- 172.21.15.89:0/2926204140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f5c19f8a0 con 0x7f6f5c1090f0 2024-09-10T08:37:59.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.039+0000 7f6f58c00640 1 -- 172.21.15.89:0/2926204140 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f54004cc0 con 0x7f6f5c1090f0 2024-09-10T08:37:59.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.039+0000 7f6f58c00640 1 -- 172.21.15.89:0/2926204140 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f54010760 con 0x7f6f5c1090f0 2024-09-10T08:37:59.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.039+0000 7f6f62400640 1 -- 172.21.15.89:0/2926204140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f5c1a7180 con 0x7f6f5c1090f0 2024-09-10T08:37:59.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.040+0000 7f6f62400640 1 -- 172.21.15.89:0/2926204140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f5c1054a0 con 0x7f6f5c1090f0 2024-09-10T08:37:59.049 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:58 smithi111 ceph-mon[35220]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:59.050 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:37:58 smithi111 ceph-mon[35220]: pgmap v332: 129 pgs: 129 active+clean; 591 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-10T08:37:59.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.042+0000 7f6f58c00640 1 -- 172.21.15.89:0/2926204140 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6f54005000 con 0x7f6f5c1090f0 2024-09-10T08:37:59.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.043+0000 7f6f58c00640 1 --2- 172.21.15.89:0/2926204140 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6f400a2c10 0x7f6f400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:59.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.044+0000 7f6f58c00640 1 -- 172.21.15.89:0/2926204140 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f6f540d01d0 con 0x7f6f5c1090f0 2024-09-10T08:37:59.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.044+0000 7f6f5b400640 1 --2- 172.21.15.89:0/2926204140 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6f400a2c10 0x7f6f400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:59.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.044+0000 7f6f5b400640 1 --2- 172.21.15.89:0/2926204140 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6f400a2c10 0x7f6f400a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009bd0 tx=0x7f6f4c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:59.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.046+0000 7f6f58c00640 1 -- 172.21.15.89:0/2926204140 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f5409cc10 con 0x7f6f5c1090f0 2024-09-10T08:37:59.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:58 smithi149 ceph-mon[36167]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:59.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:37:58 smithi149 ceph-mon[36167]: pgmap v332: 129 pgs: 129 active+clean; 591 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-10T08:37:59.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.245+0000 7f6f62400640 1 -- 172.21.15.89:0/2926204140 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6f5c10c3a0 con 0x7f6f400a2c10 2024-09-10T08:37:59.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.275+0000 7f6f58c00640 1 -- 172.21.15.89:0/2926204140 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6f4c01b3f0 con 0x7f6f400a2c10 2024-09-10T08:37:59.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.277+0000 7f6f62400640 1 -- 172.21.15.89:0/2926204140 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6f400a2c10 msgr2=0x7f6f400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:59.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.277+0000 7f6f62400640 1 --2- 172.21.15.89:0/2926204140 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6f400a2c10 0x7f6f400a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009bd0 tx=0x7f6f4c005f90 comp rx=0 tx=0).stop 2024-09-10T08:37:59.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.277+0000 7f6f62400640 1 -- 172.21.15.89:0/2926204140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f5c1090f0 msgr2=0x7f6f5c19eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:59.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.277+0000 7f6f62400640 1 --2- 172.21.15.89:0/2926204140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f5c1090f0 0x7f6f5c19eed0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400eb10 tx=0x7f6f5400c2d0 comp rx=0 tx=0).stop 2024-09-10T08:37:59.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.277+0000 7f6f62400640 1 -- 172.21.15.89:0/2926204140 shutdown_connections 2024-09-10T08:37:59.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.277+0000 7f6f62400640 1 --2- 172.21.15.89:0/2926204140 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6f400a2c10 0x7f6f400a50d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:59.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.277+0000 7f6f62400640 1 --2- 172.21.15.89:0/2926204140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f5c1090f0 0x7f6f5c19eed0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:59.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.277+0000 7f6f62400640 1 --2- 172.21.15.89:0/2926204140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f5c104970 0x7f6f5c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:59.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.277+0000 7f6f62400640 1 --2- 172.21.15.89:0/2926204140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f5c103fc0 0x7f6f5c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:59.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.278+0000 7f6f62400640 1 -- 172.21.15.89:0/2926204140 >> 172.21.15.89:0/2926204140 conn(0x7f6f5c0ffa20 msgr2=0x7f6f5c100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:59.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.278+0000 7f6f62400640 1 -- 172.21.15.89:0/2926204140 shutdown_connections 2024-09-10T08:37:59.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:37:59.278+0000 7f6f62400640 1 -- 172.21.15.89:0/2926204140 wait complete. 2024-09-10T08:37:59.992 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-10T08:38:00.362 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:00.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.938+0000 7fe429e00640 1 -- 172.21.15.89:0/859840131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe424074eb0 msgr2=0x7fe424112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:00.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.938+0000 7fe429e00640 1 --2- 172.21.15.89:0/859840131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe424074eb0 0x7fe424112650 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009900 tx=0x7fe40c02a480 comp rx=0 tx=0).stop 2024-09-10T08:38:00.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.939+0000 7fe429e00640 1 -- 172.21.15.89:0/859840131 shutdown_connections 2024-09-10T08:38:00.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.939+0000 7fe429e00640 1 --2- 172.21.15.89:0/859840131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe424112b90 0x7fe424114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:00.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.939+0000 7fe429e00640 1 --2- 172.21.15.89:0/859840131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe424074eb0 0x7fe424112650 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:00.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.939+0000 7fe429e00640 1 --2- 172.21.15.89:0/859840131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe424074590 0x7fe424074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:00.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.939+0000 7fe429e00640 1 -- 172.21.15.89:0/859840131 >> 172.21.15.89:0/859840131 conn(0x7fe4240fd8c0 msgr2=0x7fe4240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:00.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.939+0000 7fe429e00640 1 -- 172.21.15.89:0/859840131 shutdown_connections 2024-09-10T08:38:00.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.939+0000 7fe429e00640 1 -- 172.21.15.89:0/859840131 wait complete. 2024-09-10T08:38:00.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.940+0000 7fe429e00640 1 Processor -- start 2024-09-10T08:38:00.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.940+0000 7fe429e00640 1 -- start start 2024-09-10T08:38:00.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.940+0000 7fe429e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe424074590 0x7fe4241a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:00.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.940+0000 7fe429e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe424074eb0 0x7fe4241a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:00.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.940+0000 7fe429e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe424112b90 0x7fe42419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.940+0000 7fe429e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe424118590 con 0x7fe424074590 2024-09-10T08:38:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.940+0000 7fe429e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe424118410 con 0x7fe424112b90 2024-09-10T08:38:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.940+0000 7fe429e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe424118710 con 0x7fe424074eb0 2024-09-10T08:38:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.940+0000 7fe423400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe424112b90 0x7fe42419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.941+0000 7fe422a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe424074590 0x7fe4241a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.941+0000 7fe423400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe424112b90 0x7fe42419e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:44840/0 (socket says 172.21.15.89:44840) 2024-09-10T08:38:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.941+0000 7fe423400640 1 -- 172.21.15.89:0/3036719161 learned_addr learned my addr 172.21.15.89:0/3036719161 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.941+0000 7fe41a000640 1 --2- 172.21.15.89:0/3036719161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe424074eb0 0x7fe4241a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.941+0000 7fe423400640 1 -- 172.21.15.89:0/3036719161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe424074eb0 msgr2=0x7fe4241a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.941+0000 7fe423400640 1 --2- 172.21.15.89:0/3036719161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe424074eb0 0x7fe4241a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.941+0000 7fe423400640 1 -- 172.21.15.89:0/3036719161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe424074590 msgr2=0x7fe4241a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.941+0000 7fe423400640 1 --2- 172.21.15.89:0/3036719161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe424074590 0x7fe4241a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:00.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.941+0000 7fe423400640 1 -- 172.21.15.89:0/3036719161 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42419f020 con 0x7fe424112b90 2024-09-10T08:38:00.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.941+0000 7fe422a00640 1 --2- 172.21.15.89:0/3036719161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe424074590 0x7fe4241a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:38:00.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.941+0000 7fe423400640 1 --2- 172.21.15.89:0/3036719161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe424112b90 0x7fe42419e8f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00eb40 tx=0x7fe41c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:00.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.941+0000 7fe428a00640 1 -- 172.21.15.89:0/3036719161 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c004b20 con 0x7fe424112b90 2024-09-10T08:38:00.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.941+0000 7fe428a00640 1 -- 172.21.15.89:0/3036719161 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe41c004cc0 con 0x7fe424112b90 2024-09-10T08:38:00.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.941+0000 7fe429e00640 1 -- 172.21.15.89:0/3036719161 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe42419f310 con 0x7fe424112b90 2024-09-10T08:38:00.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.942+0000 7fe428a00640 1 -- 172.21.15.89:0/3036719161 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c010850 con 0x7fe424112b90 2024-09-10T08:38:00.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.942+0000 7fe429e00640 1 -- 172.21.15.89:0/3036719161 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4241ab600 con 0x7fe424112b90 2024-09-10T08:38:00.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.943+0000 7fe429e00640 1 -- 172.21.15.89:0/3036719161 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe424074970 con 0x7fe424112b90 2024-09-10T08:38:00.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.945+0000 7fe428a00640 1 -- 172.21.15.89:0/3036719161 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe41c005020 con 0x7fe424112b90 2024-09-10T08:38:00.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.946+0000 7fe428a00640 1 --2- 172.21.15.89:0/3036719161 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4080a2c10 0x7fe4080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:00.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.946+0000 7fe428a00640 1 -- 172.21.15.89:0/3036719161 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fe41c0cfd20 con 0x7fe424112b90 2024-09-10T08:38:00.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.946+0000 7fe422a00640 1 --2- 172.21.15.89:0/3036719161 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4080a2c10 0x7fe4080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:00.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.947+0000 7fe422a00640 1 --2- 172.21.15.89:0/3036719161 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4080a2c10 0x7fe4080a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe42419fa30 tx=0x7fe41400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:00.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:00.950+0000 7fe428a00640 1 -- 172.21.15.89:0/3036719161 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe41c09c760 con 0x7fe424112b90 2024-09-10T08:38:01.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:00 smithi111 ceph-mon[35220]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:38:01.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:00 smithi111 ceph-mon[35220]: pgmap v333: 129 pgs: 129 active+clean; 591 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-10T08:38:01.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:00 smithi089 ceph-mon[30757]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:38:01.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:00 smithi089 ceph-mon[30757]: pgmap v333: 129 pgs: 129 active+clean; 591 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-10T08:38:01.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:01.150+0000 7fe429e00640 1 -- 172.21.15.89:0/3036719161 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fe424062630 con 0x7fe4080a2c10 2024-09-10T08:38:01.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:01.154+0000 7fe428a00640 1 -- 172.21.15.89:0/3036719161 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fe424062630 con 0x7fe4080a2c10 2024-09-10T08:38:01.155 INFO:teuthology.orchestra.run.smithi089.stdout:/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e 2024-09-10T08:38:01.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:01.156+0000 7fe429e00640 1 -- 172.21.15.89:0/3036719161 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4080a2c10 msgr2=0x7fe4080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:01.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:01.156+0000 7fe429e00640 1 --2- 172.21.15.89:0/3036719161 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4080a2c10 0x7fe4080a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe42419fa30 tx=0x7fe41400b410 comp rx=0 tx=0).stop 2024-09-10T08:38:01.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:01.156+0000 7fe429e00640 1 -- 172.21.15.89:0/3036719161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe424112b90 msgr2=0x7fe42419e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:01.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:01.156+0000 7fe429e00640 1 --2- 172.21.15.89:0/3036719161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe424112b90 0x7fe42419e8f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00eb40 tx=0x7fe41c00c2d0 comp rx=0 tx=0).stop 2024-09-10T08:38:01.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:01.157+0000 7fe429e00640 1 -- 172.21.15.89:0/3036719161 shutdown_connections 2024-09-10T08:38:01.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:01.157+0000 7fe429e00640 1 --2- 172.21.15.89:0/3036719161 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4080a2c10 0x7fe4080a50d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:01.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:01.157+0000 7fe429e00640 1 --2- 172.21.15.89:0/3036719161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe424112b90 0x7fe42419e8f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:01.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:01.157+0000 7fe429e00640 1 --2- 172.21.15.89:0/3036719161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe424074eb0 0x7fe4241a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:01.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:01.157+0000 7fe429e00640 1 --2- 172.21.15.89:0/3036719161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe424074590 0x7fe4241a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:01.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:01.157+0000 7fe429e00640 1 -- 172.21.15.89:0/3036719161 >> 172.21.15.89:0/3036719161 conn(0x7fe4240fd8c0 msgr2=0x7fe4240ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:01.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:01.157+0000 7fe429e00640 1 -- 172.21.15.89:0/3036719161 shutdown_connections 2024-09-10T08:38:01.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:01.157+0000 7fe429e00640 1 -- 172.21.15.89:0/3036719161 wait complete. 2024-09-10T08:38:01.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:00 smithi149 ceph-mon[36167]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:38:01.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:00 smithi149 ceph-mon[36167]: pgmap v333: 129 pgs: 129 active+clean; 591 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-10T08:38:01.942 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:38:02.311 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:02.902 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:02 smithi089 ceph-mon[30757]: from='client.24752 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:38:02.902 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:02 smithi089 ceph-mon[30757]: pgmap v334: 129 pgs: 129 active+clean; 623 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-09-10T08:38:02.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.906+0000 7efcd3200640 1 -- 172.21.15.89:0/1662334777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcd4102370 msgr2=0x7efcd410e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:02.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.906+0000 7efcd3200640 1 --2- 172.21.15.89:0/1662334777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcd4102370 0x7efcd410e8a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efccc009930 tx=0x7efccc02a420 comp rx=0 tx=0).stop 2024-09-10T08:38:02.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.906+0000 7efcd3200640 1 -- 172.21.15.89:0/1662334777 shutdown_connections 2024-09-10T08:38:02.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.906+0000 7efcd3200640 1 --2- 172.21.15.89:0/1662334777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcd4102370 0x7efcd410e8a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:02.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.906+0000 7efcd3200640 1 --2- 172.21.15.89:0/1662334777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efcd41019d0 0x7efcd4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:02.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.906+0000 7efcd3200640 1 --2- 172.21.15.89:0/1662334777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcd41079d0 0x7efcd4107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:02.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.906+0000 7efcd3200640 1 -- 172.21.15.89:0/1662334777 >> 172.21.15.89:0/1662334777 conn(0x7efcd40fd710 msgr2=0x7efcd40ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:02.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.907+0000 7efcd3200640 1 -- 172.21.15.89:0/1662334777 shutdown_connections 2024-09-10T08:38:02.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.907+0000 7efcd3200640 1 -- 172.21.15.89:0/1662334777 wait complete. 2024-09-10T08:38:02.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.908+0000 7efcd3200640 1 Processor -- start 2024-09-10T08:38:02.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.908+0000 7efcd3200640 1 -- start start 2024-09-10T08:38:02.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.908+0000 7efcd3200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcd41019d0 0x7efcd419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:02.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.908+0000 7efcd3200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcd4102370 0x7efcd419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:02.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.908+0000 7efcd3200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efcd41079d0 0x7efcd419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:02.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.908+0000 7efcd3200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efcd4111e20 con 0x7efcd41019d0 2024-09-10T08:38:02.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.908+0000 7efcd3200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efcd4111ca0 con 0x7efcd4102370 2024-09-10T08:38:02.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.908+0000 7efcd3200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efcd4111fa0 con 0x7efcd41079d0 2024-09-10T08:38:02.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.908+0000 7efcd1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcd41019d0 0x7efcd419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:02.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.908+0000 7efcd2800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efcd41079d0 0x7efcd419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:02.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.908+0000 7efcd1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcd41019d0 0x7efcd419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36132/0 (socket says 172.21.15.89:36132) 2024-09-10T08:38:02.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.908+0000 7efcd1e00640 1 -- 172.21.15.89:0/509945292 learned_addr learned my addr 172.21.15.89:0/509945292 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:02.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.909+0000 7efcc9400640 1 --2- 172.21.15.89:0/509945292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcd4102370 0x7efcd419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:02.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.909+0000 7efcd1e00640 1 -- 172.21.15.89:0/509945292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efcd41079d0 msgr2=0x7efcd419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:02.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.909+0000 7efcd1e00640 1 --2- 172.21.15.89:0/509945292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efcd41079d0 0x7efcd419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:02.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.909+0000 7efcd1e00640 1 -- 172.21.15.89:0/509945292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcd4102370 msgr2=0x7efcd419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:02.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.909+0000 7efcd1e00640 1 --2- 172.21.15.89:0/509945292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcd4102370 0x7efcd419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:02.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.909+0000 7efcd1e00640 1 -- 172.21.15.89:0/509945292 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd419f410 con 0x7efcd41019d0 2024-09-10T08:38:02.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.909+0000 7efcd1e00640 1 --2- 172.21.15.89:0/509945292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcd41019d0 0x7efcd419a460 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efcc400db80 tx=0x7efcc400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:02.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.909+0000 7efccb400640 1 -- 172.21.15.89:0/509945292 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc4017070 con 0x7efcd41019d0 2024-09-10T08:38:02.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.909+0000 7efccb400640 1 -- 172.21.15.89:0/509945292 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcc4003b70 con 0x7efcd41019d0 2024-09-10T08:38:02.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.909+0000 7efccb400640 1 -- 172.21.15.89:0/509945292 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc40106c0 con 0x7efcd41019d0 2024-09-10T08:38:02.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.909+0000 7efcd3200640 1 -- 172.21.15.89:0/509945292 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd419f700 con 0x7efcd41019d0 2024-09-10T08:38:02.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.910+0000 7efcd3200640 1 -- 172.21.15.89:0/509945292 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd41a6fe0 con 0x7efcd41019d0 2024-09-10T08:38:02.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.911+0000 7efcd3200640 1 -- 172.21.15.89:0/509945292 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd4101e30 con 0x7efcd41019d0 2024-09-10T08:38:02.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.912+0000 7efccb400640 1 -- 172.21.15.89:0/509945292 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efcc4004020 con 0x7efcd41019d0 2024-09-10T08:38:02.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.914+0000 7efccb400640 1 --2- 172.21.15.89:0/509945292 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efcb80a2c10 0x7efcb80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:02.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.914+0000 7efccb400640 1 -- 172.21.15.89:0/509945292 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7efcc40cf290 con 0x7efcd41019d0 2024-09-10T08:38:02.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.914+0000 7efcc9400640 1 --2- 172.21.15.89:0/509945292 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efcb80a2c10 0x7efcb80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:02.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.915+0000 7efcc9400640 1 --2- 172.21.15.89:0/509945292 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efcb80a2c10 0x7efcb80a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efcd410eb40 tx=0x7efcbc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:02.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:02.918+0000 7efccb400640 1 -- 172.21.15.89:0/509945292 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efcc4020070 con 0x7efcd41019d0 2024-09-10T08:38:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:02 smithi111 ceph-mon[35220]: from='client.24752 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:38:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:02 smithi111 ceph-mon[35220]: pgmap v334: 129 pgs: 129 active+clean; 623 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-09-10T08:38:03.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:03.126+0000 7efcd3200640 1 -- 172.21.15.89:0/509945292 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcd41a7270 con 0x7efcd41019d0 2024-09-10T08:38:03.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:03.127+0000 7efccb400640 1 -- 172.21.15.89:0/509945292 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11918 (secure 0 0 0) 0x7efcc409bcd0 con 0x7efcd41019d0 2024-09-10T08:38:03.128 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:38:03.128 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":25,"btime":"2024-09-10T08:37:50:718359+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:37:50.607307+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24496,24518,24554]},"id":1}]} 2024-09-10T08:38:03.129 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 25 2024-09-10T08:38:03.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:03.129+0000 7efcd3200640 1 -- 172.21.15.89:0/509945292 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efcb80a2c10 msgr2=0x7efcb80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:03.129+0000 7efcd3200640 1 --2- 172.21.15.89:0/509945292 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efcb80a2c10 0x7efcb80a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efcd410eb40 tx=0x7efcbc00b390 comp rx=0 tx=0).stop 2024-09-10T08:38:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:03.129+0000 7efcd3200640 1 -- 172.21.15.89:0/509945292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcd41019d0 msgr2=0x7efcd419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:03.129+0000 7efcd3200640 1 --2- 172.21.15.89:0/509945292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcd41019d0 0x7efcd419a460 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efcc400db80 tx=0x7efcc400b630 comp rx=0 tx=0).stop 2024-09-10T08:38:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:03.130+0000 7efcd3200640 1 -- 172.21.15.89:0/509945292 shutdown_connections 2024-09-10T08:38:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:03.130+0000 7efcd3200640 1 --2- 172.21.15.89:0/509945292 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efcb80a2c10 0x7efcb80a50d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:03.130+0000 7efcd3200640 1 --2- 172.21.15.89:0/509945292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efcd41079d0 0x7efcd419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:03.130+0000 7efcd3200640 1 --2- 172.21.15.89:0/509945292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcd4102370 0x7efcd419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:03.130+0000 7efcd3200640 1 --2- 172.21.15.89:0/509945292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcd41019d0 0x7efcd419a460 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:03.130+0000 7efcd3200640 1 -- 172.21.15.89:0/509945292 >> 172.21.15.89:0/509945292 conn(0x7efcd40fd710 msgr2=0x7efcd410d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:03.130+0000 7efcd3200640 1 -- 172.21.15.89:0/509945292 shutdown_connections 2024-09-10T08:38:03.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:03.130+0000 7efcd3200640 1 -- 172.21.15.89:0/509945292 wait complete. 2024-09-10T08:38:03.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:02 smithi149 ceph-mon[36167]: from='client.24752 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:38:03.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:02 smithi149 ceph-mon[36167]: pgmap v334: 129 pgs: 129 active+clean; 623 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 15 op/s 2024-09-10T08:38:04.035 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:03 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/509945292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:38:04.038 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:38:04.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:03 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/509945292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:38:04.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:03 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/509945292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:38:04.417 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:04.853 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:04 smithi089 ceph-mon[30757]: pgmap v335: 129 pgs: 129 active+clean; 623 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-09-10T08:38:04.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.963+0000 7f3092600640 1 -- 172.21.15.89:0/3596077127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f308c106f20 msgr2=0x7f308c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:04.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.963+0000 7f3092600640 1 --2- 172.21.15.89:0/3596077127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f308c106f20 0x7f308c10eaa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3084009ca0 tx=0x7f308402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:38:04.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.963+0000 7f3092600640 1 -- 172.21.15.89:0/3596077127 shutdown_connections 2024-09-10T08:38:04.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.963+0000 7f3092600640 1 --2- 172.21.15.89:0/3596077127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f308c106f20 0x7f308c10eaa0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:04.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.963+0000 7f3092600640 1 --2- 172.21.15.89:0/3596077127 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f308c1027a0 0x7f308c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:04.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.963+0000 7f3092600640 1 --2- 172.21.15.89:0/3596077127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f308c101df0 0x7f308c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:04.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.963+0000 7f3092600640 1 -- 172.21.15.89:0/3596077127 >> 172.21.15.89:0/3596077127 conn(0x7f308c0fd8b0 msgr2=0x7f308c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:04.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.963+0000 7f3092600640 1 -- 172.21.15.89:0/3596077127 shutdown_connections 2024-09-10T08:38:04.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.963+0000 7f3092600640 1 -- 172.21.15.89:0/3596077127 wait complete. 2024-09-10T08:38:04.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.964+0000 7f3092600640 1 Processor -- start 2024-09-10T08:38:04.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.965+0000 7f3092600640 1 -- start start 2024-09-10T08:38:04.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.965+0000 7f3092600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f308c101df0 0x7f308c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:04.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.965+0000 7f3092600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f308c1027a0 0x7f308c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:04.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.965+0000 7f3092600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f308c106f20 0x7f308c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:04.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.965+0000 7f3092600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f308c111fd0 con 0x7f308c101df0 2024-09-10T08:38:04.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.965+0000 7f3092600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f308c111e50 con 0x7f308c106f20 2024-09-10T08:38:04.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.965+0000 7f3092600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f308c112150 con 0x7f308c1027a0 2024-09-10T08:38:04.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.965+0000 7f308b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f308c101df0 0x7f308c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:04.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.965+0000 7f308b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f308c101df0 0x7f308c19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36154/0 (socket says 172.21.15.89:36154) 2024-09-10T08:38:04.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.965+0000 7f308b400640 1 -- 172.21.15.89:0/2448521870 learned_addr learned my addr 172.21.15.89:0/2448521870 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:04.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.965+0000 7f308aa00640 1 --2- 172.21.15.89:0/2448521870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f308c1027a0 0x7f308c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:04.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.965+0000 7f308be00640 1 --2- 172.21.15.89:0/2448521870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f308c106f20 0x7f308c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:04.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.966+0000 7f308aa00640 1 -- 172.21.15.89:0/2448521870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f308c106f20 msgr2=0x7f308c19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:04.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.966+0000 7f308aa00640 1 --2- 172.21.15.89:0/2448521870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f308c106f20 0x7f308c19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:04.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.966+0000 7f308aa00640 1 -- 172.21.15.89:0/2448521870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f308c101df0 msgr2=0x7f308c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:04.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.966+0000 7f308aa00640 1 --2- 172.21.15.89:0/2448521870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f308c101df0 0x7f308c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:04.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.966+0000 7f308aa00640 1 -- 172.21.15.89:0/2448521870 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308c19f570 con 0x7f308c1027a0 2024-09-10T08:38:04.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.966+0000 7f308b400640 1 --2- 172.21.15.89:0/2448521870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f308c101df0 0x7f308c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:38:04.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.966+0000 7f308aa00640 1 --2- 172.21.15.89:0/2448521870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f308c1027a0 0x7f308c19ab50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f307800e9f0 tx=0x7f307800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:04.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.966+0000 7f3083e00640 1 -- 172.21.15.89:0/2448521870 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307800cd80 con 0x7f308c1027a0 2024-09-10T08:38:04.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.966+0000 7f3092600640 1 -- 172.21.15.89:0/2448521870 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f308c19f860 con 0x7f308c1027a0 2024-09-10T08:38:04.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.967+0000 7f3083e00640 1 -- 172.21.15.89:0/2448521870 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30780035f0 con 0x7f308c1027a0 2024-09-10T08:38:04.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.967+0000 7f3083e00640 1 -- 172.21.15.89:0/2448521870 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078010670 con 0x7f308c1027a0 2024-09-10T08:38:04.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.967+0000 7f3092600640 1 -- 172.21.15.89:0/2448521870 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f308c1a7140 con 0x7f308c1027a0 2024-09-10T08:38:04.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.968+0000 7f3092600640 1 -- 172.21.15.89:0/2448521870 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f308c1021d0 con 0x7f308c1027a0 2024-09-10T08:38:04.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.970+0000 7f3083e00640 1 -- 172.21.15.89:0/2448521870 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f30780037a0 con 0x7f308c1027a0 2024-09-10T08:38:04.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.971+0000 7f3083e00640 1 --2- 172.21.15.89:0/2448521870 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30700a2c90 0x7f30700a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:04.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.971+0000 7f3083e00640 1 -- 172.21.15.89:0/2448521870 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f30780cf700 con 0x7f308c1027a0 2024-09-10T08:38:04.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.971+0000 7f308b400640 1 --2- 172.21.15.89:0/2448521870 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30700a2c90 0x7f30700a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:04.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.972+0000 7f308b400640 1 --2- 172.21.15.89:0/2448521870 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30700a2c90 0x7f30700a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f307c011500 tx=0x7f307c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:04.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:04.973+0000 7f3083e00640 1 -- 172.21.15.89:0/2448521870 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f307809c140 con 0x7f308c1027a0 2024-09-10T08:38:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:04 smithi111 ceph-mon[35220]: pgmap v335: 129 pgs: 129 active+clean; 623 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-09-10T08:38:05.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:05.151+0000 7f3092600640 1 -- 172.21.15.89:0/2448521870 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f308c19b920 con 0x7f308c1027a0 2024-09-10T08:38:05.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:05.153+0000 7f3083e00640 1 -- 172.21.15.89:0/2448521870 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24707 (secure 0 0 0) 0x7f308c19b920 con 0x7f308c1027a0 2024-09-10T08:38:05.154 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:38:05.155 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":94,"fsid":"91f747bc-6f4e-11ef-bcea-c7b262605968","created":"2024-09-10T08:28:49.697035+0000","modified":"2024-09-10T08:37:42.660676+0000","last_up_change":"2024-09-10T08:36:00.535433+0000","last_in_change":"2024-09-10T08:35:37.131131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:32:02.060714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:36:52.462301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:36:55.119708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:37:03.729002+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d3b55192-bd95-4497-89ea-dc017a25c0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6803","nonce":1760523284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6805","nonce":1760523284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6809","nonce":1760523284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6807","nonce":1760523284}]},"public_addr":"172.21.15.89:6803/1760523284","cluster_addr":"172.21.15.89:6805/1760523284","heartbeat_back_addr":"172.21.15.89:6809/1760523284","heartbeat_front_addr":"172.21.15.89:6807/1760523284","state":["exists","up"]},{"osd":1,"uuid":"60aae117-cb52-4034-a700-11693171cd52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6801","nonce":2369291742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6803","nonce":2369291742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6807","nonce":2369291742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6805","nonce":2369291742}]},"public_addr":"172.21.15.111:6801/2369291742","cluster_addr":"172.21.15.111:6803/2369291742","heartbeat_back_addr":"172.21.15.111:6807/2369291742","heartbeat_front_addr":"172.21.15.111:6805/2369291742","state":["exists","up"]},{"osd":2,"uuid":"a305a019-0870-437c-a298-4f923ec54e77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6801","nonce":1092261395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6803","nonce":1092261395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6807","nonce":1092261395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6805","nonce":1092261395}]},"public_addr":"172.21.15.149:6801/1092261395","cluster_addr":"172.21.15.149:6803/1092261395","heartbeat_back_addr":"172.21.15.149:6807/1092261395","heartbeat_front_addr":"172.21.15.149:6805/1092261395","state":["exists","up"]},{"osd":3,"uuid":"d02f54c1-f883-40f6-9341-8d9fc4b712ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6811","nonce":1405773950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6813","nonce":1405773950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6817","nonce":1405773950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6815","nonce":1405773950}]},"public_addr":"172.21.15.89:6811/1405773950","cluster_addr":"172.21.15.89:6813/1405773950","heartbeat_back_addr":"172.21.15.89:6817/1405773950","heartbeat_front_addr":"172.21.15.89:6815/1405773950","state":["exists","up"]},{"osd":4,"uuid":"23cff46b-88dd-4344-be06-b35db6014f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6809","nonce":316717323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6811","nonce":316717323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6815","nonce":316717323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6813","nonce":316717323}]},"public_addr":"172.21.15.111:6809/316717323","cluster_addr":"172.21.15.111:6811/316717323","heartbeat_back_addr":"172.21.15.111:6815/316717323","heartbeat_front_addr":"172.21.15.111:6813/316717323","state":["exists","up"]},{"osd":5,"uuid":"8e516e8d-e7c6-4ebc-80a5-29932161df4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6809","nonce":1256148461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6811","nonce":1256148461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6815","nonce":1256148461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6813","nonce":1256148461}]},"public_addr":"172.21.15.149:6809/1256148461","cluster_addr":"172.21.15.149:6811/1256148461","heartbeat_back_addr":"172.21.15.149:6815/1256148461","heartbeat_front_addr":"172.21.15.149:6813/1256148461","state":["exists","up"]},{"osd":6,"uuid":"48f92472-f657-4c5a-adeb-20d4fea1ce8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6819","nonce":4190488024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6821","nonce":4190488024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6825","nonce":4190488024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6823","nonce":4190488024}]},"public_addr":"172.21.15.89:6819/4190488024","cluster_addr":"172.21.15.89:6821/4190488024","heartbeat_back_addr":"172.21.15.89:6825/4190488024","heartbeat_front_addr":"172.21.15.89:6823/4190488024","state":["exists","up"]},{"osd":7,"uuid":"14a2a6f0-c4cd-406b-b637-7527fd32babe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6817","nonce":652666376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6819","nonce":652666376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6823","nonce":652666376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6821","nonce":652666376}]},"public_addr":"172.21.15.111:6817/652666376","cluster_addr":"172.21.15.111:6819/652666376","heartbeat_back_addr":"172.21.15.111:6823/652666376","heartbeat_front_addr":"172.21.15.111:6821/652666376","state":["exists","up"]},{"osd":8,"uuid":"a54502e6-95f7-431f-8db1-54a6c7983b5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6817","nonce":2955172869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6819","nonce":2955172869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6823","nonce":2955172869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6821","nonce":2955172869}]},"public_addr":"172.21.15.149:6817/2955172869","cluster_addr":"172.21.15.149:6819/2955172869","heartbeat_back_addr":"172.21.15.149:6823/2955172869","heartbeat_front_addr":"172.21.15.149:6821/2955172869","state":["exists","up"]},{"osd":9,"uuid":"5fe17344-5b1c-4601-a633-639cc1691478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6827","nonce":2427467040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6829","nonce":2427467040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6833","nonce":2427467040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6831","nonce":2427467040}]},"public_addr":"172.21.15.89:6827/2427467040","cluster_addr":"172.21.15.89:6829/2427467040","heartbeat_back_addr":"172.21.15.89:6833/2427467040","heartbeat_front_addr":"172.21.15.89:6831/2427467040","state":["exists","up"]},{"osd":10,"uuid":"d8f29890-3064-4b67-b649-467a0ac86db7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6825","nonce":2328678744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6827","nonce":2328678744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6831","nonce":2328678744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6829","nonce":2328678744}]},"public_addr":"172.21.15.111:6825/2328678744","cluster_addr":"172.21.15.111:6827/2328678744","heartbeat_back_addr":"172.21.15.111:6831/2328678744","heartbeat_front_addr":"172.21.15.111:6829/2328678744","state":["exists","up"]},{"osd":11,"uuid":"92182c73-f139-47ca-a0ef-5046ae9b2310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6825","nonce":3130508827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6827","nonce":3130508827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6831","nonce":3130508827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6829","nonce":3130508827}]},"public_addr":"172.21.15.149:6825/3130508827","cluster_addr":"172.21.15.149:6827/3130508827","heartbeat_back_addr":"172.21.15.149:6831/3130508827","heartbeat_front_addr":"172.21.15.149:6829/3130508827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:09.031810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:33.695124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:58.065510+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:27.592838+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:53.275289+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:18.305677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:47.781535+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:12.889512+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:37.744193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:07.718408+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:33.391027+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:59.013250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/850669510":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/3541027428":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6801/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6800/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/2396468787":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/1609720980":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/995019144":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6801/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6800/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/2179977257":"2024-09-11T08:29:14.554869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:38:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:05.156+0000 7f3092600640 1 -- 172.21.15.89:0/2448521870 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30700a2c90 msgr2=0x7f30700a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:05.156+0000 7f3092600640 1 --2- 172.21.15.89:0/2448521870 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30700a2c90 0x7f30700a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f307c011500 tx=0x7f307c005f90 comp rx=0 tx=0).stop 2024-09-10T08:38:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:05.156+0000 7f3092600640 1 -- 172.21.15.89:0/2448521870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f308c1027a0 msgr2=0x7f308c19ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:05.156+0000 7f3092600640 1 --2- 172.21.15.89:0/2448521870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f308c1027a0 0x7f308c19ab50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f307800e9f0 tx=0x7f307800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:38:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:05.156+0000 7f3092600640 1 -- 172.21.15.89:0/2448521870 shutdown_connections 2024-09-10T08:38:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:05.156+0000 7f3092600640 1 --2- 172.21.15.89:0/2448521870 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30700a2c90 0x7f30700a5150 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:05.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:05.156+0000 7f3092600640 1 --2- 172.21.15.89:0/2448521870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f308c106f20 0x7f308c19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:05.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:05.156+0000 7f3092600640 1 --2- 172.21.15.89:0/2448521870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f308c1027a0 0x7f308c19ab50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:05.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:05.156+0000 7f3092600640 1 --2- 172.21.15.89:0/2448521870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f308c101df0 0x7f308c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:05.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:05.156+0000 7f3092600640 1 -- 172.21.15.89:0/2448521870 >> 172.21.15.89:0/2448521870 conn(0x7f308c0fd8b0 msgr2=0x7f308c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:05.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:05.156+0000 7f3092600640 1 -- 172.21.15.89:0/2448521870 shutdown_connections 2024-09-10T08:38:05.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:05.157+0000 7f3092600640 1 -- 172.21.15.89:0/2448521870 wait complete. 2024-09-10T08:38:05.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:04 smithi149 ceph-mon[36167]: pgmap v335: 129 pgs: 129 active+clean; 623 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-09-10T08:38:05.908 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:05 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2448521870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:38:05.945 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T08:38:05.945 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:38:06.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:05 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2448521870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:38:06.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:05 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2448521870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:38:06.314 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:06.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.831+0000 7ff57c200640 1 -- 172.21.15.89:0/3582485977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff574109140 msgr2=0x7ff574110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:06.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.831+0000 7ff57c200640 1 --2- 172.21.15.89:0/3582485977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff574109140 0x7ff574110cc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009900 tx=0x7ff56c02a410 comp rx=0 tx=0).stop 2024-09-10T08:38:06.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.831+0000 7ff57c200640 1 -- 172.21.15.89:0/3582485977 shutdown_connections 2024-09-10T08:38:06.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.831+0000 7ff57c200640 1 --2- 172.21.15.89:0/3582485977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff574109140 0x7ff574110cc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:06.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.831+0000 7ff57c200640 1 --2- 172.21.15.89:0/3582485977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5741049c0 0x7ff574108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:06.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.831+0000 7ff57c200640 1 --2- 172.21.15.89:0/3582485977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff574104010 0x7ff5741043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:06.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.831+0000 7ff57c200640 1 -- 172.21.15.89:0/3582485977 >> 172.21.15.89:0/3582485977 conn(0x7ff5740ffa50 msgr2=0x7ff574101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:06.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.831+0000 7ff57c200640 1 -- 172.21.15.89:0/3582485977 shutdown_connections 2024-09-10T08:38:06.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.831+0000 7ff57c200640 1 -- 172.21.15.89:0/3582485977 wait complete. 2024-09-10T08:38:06.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.832+0000 7ff57c200640 1 Processor -- start 2024-09-10T08:38:06.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.832+0000 7ff57c200640 1 -- start start 2024-09-10T08:38:06.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.832+0000 7ff57c200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff574104010 0x7ff57419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:06.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.832+0000 7ff57c200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5741049c0 0x7ff57419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:06.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.832+0000 7ff57c200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff574109140 0x7ff57419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:06.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.832+0000 7ff57c200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff5741141a0 con 0x7ff574109140 2024-09-10T08:38:06.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.832+0000 7ff57c200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff574114020 con 0x7ff5741049c0 2024-09-10T08:38:06.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.832+0000 7ff57c200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff574114320 con 0x7ff574104010 2024-09-10T08:38:06.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.833+0000 7ff57a200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff574109140 0x7ff57419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:06.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.833+0000 7ff57a200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff574109140 0x7ff57419ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36176/0 (socket says 172.21.15.89:36176) 2024-09-10T08:38:06.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.833+0000 7ff57a200640 1 -- 172.21.15.89:0/372988684 learned_addr learned my addr 172.21.15.89:0/372988684 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:06.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.833+0000 7ff579800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff574104010 0x7ff57419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:06.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.833+0000 7ff578e00640 1 --2- 172.21.15.89:0/372988684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5741049c0 0x7ff57419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:06.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.833+0000 7ff57a200640 1 -- 172.21.15.89:0/372988684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff574104010 msgr2=0x7ff57419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:06.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.833+0000 7ff57a200640 1 --2- 172.21.15.89:0/372988684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff574104010 0x7ff57419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:06.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.833+0000 7ff57a200640 1 -- 172.21.15.89:0/372988684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5741049c0 msgr2=0x7ff57419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:06.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.833+0000 7ff57a200640 1 --2- 172.21.15.89:0/372988684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5741049c0 0x7ff57419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:06.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.833+0000 7ff57a200640 1 -- 172.21.15.89:0/372988684 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57419f650 con 0x7ff574109140 2024-09-10T08:38:06.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.833+0000 7ff578e00640 1 --2- 172.21.15.89:0/372988684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5741049c0 0x7ff57419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:38:06.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.833+0000 7ff57a200640 1 --2- 172.21.15.89:0/372988684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff574109140 0x7ff57419ef70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff56c004770 tx=0x7ff56c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:06.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.834+0000 7ff572000640 1 -- 172.21.15.89:0/372988684 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c03f070 con 0x7ff574109140 2024-09-10T08:38:06.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.834+0000 7ff572000640 1 -- 172.21.15.89:0/372988684 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff56c031c90 con 0x7ff574109140 2024-09-10T08:38:06.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.834+0000 7ff57c200640 1 -- 172.21.15.89:0/372988684 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff57419f8e0 con 0x7ff574109140 2024-09-10T08:38:06.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.834+0000 7ff572000640 1 -- 172.21.15.89:0/372988684 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c035030 con 0x7ff574109140 2024-09-10T08:38:06.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.834+0000 7ff57c200640 1 -- 172.21.15.89:0/372988684 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5741a7220 con 0x7ff574109140 2024-09-10T08:38:06.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.835+0000 7ff57c200640 1 -- 172.21.15.89:0/372988684 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5741043f0 con 0x7ff574109140 2024-09-10T08:38:06.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.837+0000 7ff572000640 1 -- 172.21.15.89:0/372988684 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff56c042070 con 0x7ff574109140 2024-09-10T08:38:06.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.838+0000 7ff572000640 1 --2- 172.21.15.89:0/372988684 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff55c0a2c90 0x7ff55c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:06.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.838+0000 7ff572000640 1 -- 172.21.15.89:0/372988684 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7ff56c0efb60 con 0x7ff574109140 2024-09-10T08:38:06.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.838+0000 7ff579800640 1 --2- 172.21.15.89:0/372988684 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff55c0a2c90 0x7ff55c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:06.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.839+0000 7ff579800640 1 --2- 172.21.15.89:0/372988684 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff55c0a2c90 0x7ff55c0a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff568005fd0 tx=0x7ff568009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:06.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:06.842+0000 7ff572000640 1 -- 172.21.15.89:0/372988684 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff56c0bbf50 con 0x7ff574109140 2024-09-10T08:38:07.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:07.017+0000 7ff57c200640 1 -- 172.21.15.89:0/372988684 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff574062630 con 0x7ff55c0a2c90 2024-09-10T08:38:07.018 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:06 smithi089 ceph-mon[30757]: pgmap v336: 129 pgs: 129 active+clean; 629 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-10T08:38:07.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:07.028+0000 7ff572000640 1 -- 172.21.15.89:0/372988684 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340666 (secure 0 0 0) 0x7ff574062630 con 0x7ff55c0a2c90 2024-09-10T08:38:07.034 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:38:07.035 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-09-10T08:38:07.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:07.032+0000 7ff57c200640 1 -- 172.21.15.89:0/372988684 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff55c0a2c90 msgr2=0x7ff55c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:07.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:07.032+0000 7ff57c200640 1 --2- 172.21.15.89:0/372988684 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff55c0a2c90 0x7ff55c0a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff568005fd0 tx=0x7ff568009760 comp rx=0 tx=0).stop 2024-09-10T08:38:07.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:07.032+0000 7ff57c200640 1 -- 172.21.15.89:0/372988684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff574109140 msgr2=0x7ff57419ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:07.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:07.032+0000 7ff57c200640 1 --2- 172.21.15.89:0/372988684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff574109140 0x7ff57419ef70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff56c004770 tx=0x7ff56c003b90 comp rx=0 tx=0).stop 2024-09-10T08:38:07.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:07.033+0000 7ff57c200640 1 -- 172.21.15.89:0/372988684 shutdown_connections 2024-09-10T08:38:07.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:07.033+0000 7ff57c200640 1 --2- 172.21.15.89:0/372988684 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff55c0a2c90 0x7ff55c0a5150 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:07.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:07.033+0000 7ff57c200640 1 --2- 172.21.15.89:0/372988684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff574109140 0x7ff57419ef70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:07.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:07.033+0000 7ff57c200640 1 --2- 172.21.15.89:0/372988684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5741049c0 0x7ff57419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:07.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:07.033+0000 7ff57c200640 1 --2- 172.21.15.89:0/372988684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff574104010 0x7ff57419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:07.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:07.033+0000 7ff57c200640 1 -- 172.21.15.89:0/372988684 >> 172.21.15.89:0/372988684 conn(0x7ff5740ffa50 msgr2=0x7ff574100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:07.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:07.033+0000 7ff57c200640 1 -- 172.21.15.89:0/372988684 shutdown_connections 2024-09-10T08:38:07.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:07.033+0000 7ff57c200640 1 -- 172.21.15.89:0/372988684 wait complete. 2024-09-10T08:38:07.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:06 smithi111 ceph-mon[35220]: pgmap v336: 129 pgs: 129 active+clean; 629 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-10T08:38:07.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:06 smithi149 ceph-mon[36167]: pgmap v336: 129 pgs: 129 active+clean; 629 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-10T08:38:07.812 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-10T08:38:05.584550+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":643703,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":236,"num_read_kb":223,"num_write":333,"num_write_kb":3038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270,"ondisk_log_size":270,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357620,"kb_used_data":12532,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584204,"statfs":{"total":1151940427776,"available":1151574224896,"internally_reserved":0,"allocated":12832768,"data_stored":8241726,"data_compressed":395913,"data_compressed_allocated":540672,"data_compressed_original":2580480,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45881,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":87,"num_write":23,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007872"},"pg_stats":[{"pgid":"2.1f","version":"94'9","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:47.715421+0000","last_change":"2024-09-10T08:36:53.720261+0000","last_active":"2024-09-10T08:37:47.715421+0000","last_peered":"2024-09-10T08:37:47.715421+0000","last_clean":"2024-09-10T08:37:47.715421+0000","last_became_active":"2024-09-10T08:36:53.720037+0000","last_became_peered":"2024-09-10T08:36:53.720037+0000","last_unstale":"2024-09-10T08:37:47.715421+0000","last_undegraded":"2024-09-10T08:37:47.715421+0000","last_fullsized":"2024-09-10T08:37:47.715421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:56.548573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.221833+0000","last_change":"2024-09-10T08:36:56.145816+0000","last_active":"2024-09-10T08:37:43.221833+0000","last_peered":"2024-09-10T08:37:43.221833+0000","last_clean":"2024-09-10T08:37:43.221833+0000","last_became_active":"2024-09-10T08:36:56.145462+0000","last_became_peered":"2024-09-10T08:36:56.145462+0000","last_unstale":"2024-09-10T08:37:43.221833+0000","last_undegraded":"2024-09-10T08:37:43.221833+0000","last_fullsized":"2024-09-10T08:37:43.221833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:53.890190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.288722+0000","last_change":"2024-09-10T08:37:05.252761+0000","last_active":"2024-09-10T08:37:43.288722+0000","last_peered":"2024-09-10T08:37:43.288722+0000","last_clean":"2024-09-10T08:37:43.288722+0000","last_became_active":"2024-09-10T08:37:05.252179+0000","last_became_peered":"2024-09-10T08:37:05.252179+0000","last_unstale":"2024-09-10T08:37:43.288722+0000","last_undegraded":"2024-09-10T08:37:43.288722+0000","last_fullsized":"2024-09-10T08:37:43.288722+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:52.854828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.222230+0000","last_change":"2024-09-10T08:36:53.735650+0000","last_active":"2024-09-10T08:37:43.222230+0000","last_peered":"2024-09-10T08:37:43.222230+0000","last_clean":"2024-09-10T08:37:43.222230+0000","last_became_active":"2024-09-10T08:36:53.735128+0000","last_became_peered":"2024-09-10T08:36:53.735128+0000","last_unstale":"2024-09-10T08:37:43.222230+0000","last_undegraded":"2024-09-10T08:37:43.222230+0000","last_fullsized":"2024-09-10T08:37:43.222230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:27.309591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.666844+0000","last_change":"2024-09-10T08:36:56.153364+0000","last_active":"2024-09-10T08:37:42.666844+0000","last_peered":"2024-09-10T08:37:42.666844+0000","last_clean":"2024-09-10T08:37:42.666844+0000","last_became_active":"2024-09-10T08:36:56.152979+0000","last_became_peered":"2024-09-10T08:36:56.152979+0000","last_unstale":"2024-09-10T08:37:42.666844+0000","last_undegraded":"2024-09-10T08:37:42.666844+0000","last_fullsized":"2024-09-10T08:37:42.666844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:12.172620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.665745+0000","last_change":"2024-09-10T08:37:05.254551+0000","last_active":"2024-09-10T08:37:42.665745+0000","last_peered":"2024-09-10T08:37:42.665745+0000","last_clean":"2024-09-10T08:37:42.665745+0000","last_became_active":"2024-09-10T08:37:05.254055+0000","last_became_peered":"2024-09-10T08:37:05.254055+0000","last_unstale":"2024-09-10T08:37:42.665745+0000","last_undegraded":"2024-09-10T08:37:42.665745+0000","last_fullsized":"2024-09-10T08:37:42.665745+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:08.472502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:47.712944+0000","last_change":"2024-09-10T08:36:53.722774+0000","last_active":"2024-09-10T08:37:47.712944+0000","last_peered":"2024-09-10T08:37:47.712944+0000","last_clean":"2024-09-10T08:37:47.712944+0000","last_became_active":"2024-09-10T08:36:53.722459+0000","last_became_peered":"2024-09-10T08:36:53.722459+0000","last_unstale":"2024-09-10T08:37:47.712944+0000","last_undegraded":"2024-09-10T08:37:47.712944+0000","last_fullsized":"2024-09-10T08:37:47.712944+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:26.612354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.289106+0000","last_change":"2024-09-10T08:36:56.143757+0000","last_active":"2024-09-10T08:37:43.289106+0000","last_peered":"2024-09-10T08:37:43.289106+0000","last_clean":"2024-09-10T08:37:43.289106+0000","last_became_active":"2024-09-10T08:36:56.143430+0000","last_became_peered":"2024-09-10T08:36:56.143430+0000","last_unstale":"2024-09-10T08:37:43.289106+0000","last_undegraded":"2024-09-10T08:37:43.289106+0000","last_fullsized":"2024-09-10T08:37:43.289106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:49.737298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.666659+0000","last_change":"2024-09-10T08:37:05.254503+0000","last_active":"2024-09-10T08:37:42.666659+0000","last_peered":"2024-09-10T08:37:42.666659+0000","last_clean":"2024-09-10T08:37:42.666659+0000","last_became_active":"2024-09-10T08:37:05.254070+0000","last_became_peered":"2024-09-10T08:37:05.254070+0000","last_unstale":"2024-09-10T08:37:42.666659+0000","last_undegraded":"2024-09-10T08:37:42.666659+0000","last_fullsized":"2024-09-10T08:37:42.666659+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:16.741698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668715+0000","last_change":"2024-09-10T08:36:53.716742+0000","last_active":"2024-09-10T08:37:42.668715+0000","last_peered":"2024-09-10T08:37:42.668715+0000","last_clean":"2024-09-10T08:37:42.668715+0000","last_became_active":"2024-09-10T08:36:53.716439+0000","last_became_peered":"2024-09-10T08:36:53.716439+0000","last_unstale":"2024-09-10T08:37:42.668715+0000","last_undegraded":"2024-09-10T08:37:42.668715+0000","last_fullsized":"2024-09-10T08:37:42.668715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:46.882805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.844590+0000","last_change":"2024-09-10T08:36:56.146841+0000","last_active":"2024-09-10T08:37:42.844590+0000","last_peered":"2024-09-10T08:37:42.844590+0000","last_clean":"2024-09-10T08:37:42.844590+0000","last_became_active":"2024-09-10T08:36:56.146531+0000","last_became_peered":"2024-09-10T08:36:56.146531+0000","last_unstale":"2024-09-10T08:37:42.844590+0000","last_undegraded":"2024-09-10T08:37:42.844590+0000","last_fullsized":"2024-09-10T08:37:42.844590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:37.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.223181+0000","last_change":"2024-09-10T08:37:43.223181+0000","last_active":"2024-09-10T08:37:43.223181+0000","last_peered":"2024-09-10T08:37:43.223181+0000","last_clean":"2024-09-10T08:37:43.223181+0000","last_became_active":"2024-09-10T08:37:43.222794+0000","last_became_peered":"2024-09-10T08:37:43.222794+0000","last_unstale":"2024-09-10T08:37:43.223181+0000","last_undegraded":"2024-09-10T08:37:43.223181+0000","last_fullsized":"2024-09-10T08:37:43.223181+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:22.204517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668683+0000","last_change":"2024-09-10T08:36:56.142161+0000","last_active":"2024-09-10T08:37:42.668683+0000","last_peered":"2024-09-10T08:37:42.668683+0000","last_clean":"2024-09-10T08:37:42.668683+0000","last_became_active":"2024-09-10T08:36:56.141838+0000","last_became_peered":"2024-09-10T08:36:56.141838+0000","last_unstale":"2024-09-10T08:37:42.668683+0000","last_undegraded":"2024-09-10T08:37:42.668683+0000","last_fullsized":"2024-09-10T08:37:42.668683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:09.909747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:46.701515+0000","last_change":"2024-09-10T08:36:53.722238+0000","last_active":"2024-09-10T08:37:46.701515+0000","last_peered":"2024-09-10T08:37:46.701515+0000","last_clean":"2024-09-10T08:37:46.701515+0000","last_became_active":"2024-09-10T08:36:53.721859+0000","last_became_peered":"2024-09-10T08:36:53.721859+0000","last_unstale":"2024-09-10T08:37:46.701515+0000","last_undegraded":"2024-09-10T08:37:46.701515+0000","last_fullsized":"2024-09-10T08:37:46.701515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:50.615090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.666365+0000","last_change":"2024-09-10T08:37:05.414041+0000","last_active":"2024-09-10T08:37:42.666365+0000","last_peered":"2024-09-10T08:37:42.666365+0000","last_clean":"2024-09-10T08:37:42.666365+0000","last_became_active":"2024-09-10T08:37:05.413286+0000","last_became_peered":"2024-09-10T08:37:05.413286+0000","last_unstale":"2024-09-10T08:37:42.666365+0000","last_undegraded":"2024-09-10T08:37:42.666365+0000","last_fullsized":"2024-09-10T08:37:42.666365+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:44.916102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.221703+0000","last_change":"2024-09-10T08:36:56.158278+0000","last_active":"2024-09-10T08:37:43.221703+0000","last_peered":"2024-09-10T08:37:43.221703+0000","last_clean":"2024-09-10T08:37:43.221703+0000","last_became_active":"2024-09-10T08:36:56.157946+0000","last_became_peered":"2024-09-10T08:36:56.157946+0000","last_unstale":"2024-09-10T08:37:43.221703+0000","last_undegraded":"2024-09-10T08:37:43.221703+0000","last_fullsized":"2024-09-10T08:37:43.221703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:42.512513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'1","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.288930+0000","last_change":"2024-09-10T08:36:53.718679+0000","last_active":"2024-09-10T08:37:43.288930+0000","last_peered":"2024-09-10T08:37:43.288930+0000","last_clean":"2024-09-10T08:37:43.288930+0000","last_became_active":"2024-09-10T08:36:53.718377+0000","last_became_peered":"2024-09-10T08:36:53.718377+0000","last_unstale":"2024-09-10T08:37:43.288930+0000","last_undegraded":"2024-09-10T08:37:43.288930+0000","last_fullsized":"2024-09-10T08:37:43.288930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:58.114825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.672349+0000","last_change":"2024-09-10T08:37:42.671399+0000","last_active":"2024-09-10T08:37:42.672349+0000","last_peered":"2024-09-10T08:37:42.672349+0000","last_clean":"2024-09-10T08:37:42.672349+0000","last_became_active":"2024-09-10T08:37:42.670865+0000","last_became_peered":"2024-09-10T08:37:42.670865+0000","last_unstale":"2024-09-10T08:37:42.672349+0000","last_undegraded":"2024-09-10T08:37:42.672349+0000","last_fullsized":"2024-09-10T08:37:42.672349+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:16.231538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664111+0000","last_change":"2024-09-10T08:36:56.144856+0000","last_active":"2024-09-10T08:37:42.664111+0000","last_peered":"2024-09-10T08:37:42.664111+0000","last_clean":"2024-09-10T08:37:42.664111+0000","last_became_active":"2024-09-10T08:36:56.144531+0000","last_became_peered":"2024-09-10T08:36:56.144531+0000","last_unstale":"2024-09-10T08:37:42.664111+0000","last_undegraded":"2024-09-10T08:37:42.664111+0000","last_fullsized":"2024-09-10T08:37:42.664111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:24.586006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:46.702988+0000","last_change":"2024-09-10T08:36:53.721093+0000","last_active":"2024-09-10T08:37:46.702988+0000","last_peered":"2024-09-10T08:37:46.702988+0000","last_clean":"2024-09-10T08:37:46.702988+0000","last_became_active":"2024-09-10T08:36:53.720538+0000","last_became_peered":"2024-09-10T08:36:53.720538+0000","last_unstale":"2024-09-10T08:37:46.702988+0000","last_undegraded":"2024-09-10T08:37:46.702988+0000","last_fullsized":"2024-09-10T08:37:46.702988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:58.492463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.669785+0000","last_change":"2024-09-10T08:37:05.254011+0000","last_active":"2024-09-10T08:37:42.669785+0000","last_peered":"2024-09-10T08:37:42.669785+0000","last_clean":"2024-09-10T08:37:42.669785+0000","last_became_active":"2024-09-10T08:37:05.253066+0000","last_became_peered":"2024-09-10T08:37:05.253066+0000","last_unstale":"2024-09-10T08:37:42.669785+0000","last_undegraded":"2024-09-10T08:37:42.669785+0000","last_fullsized":"2024-09-10T08:37:42.669785+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:16.221302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668011+0000","last_change":"2024-09-10T08:36:56.141814+0000","last_active":"2024-09-10T08:37:42.668011+0000","last_peered":"2024-09-10T08:37:42.668011+0000","last_clean":"2024-09-10T08:37:42.668011+0000","last_became_active":"2024-09-10T08:36:56.141480+0000","last_became_peered":"2024-09-10T08:36:56.141480+0000","last_unstale":"2024-09-10T08:37:42.668011+0000","last_undegraded":"2024-09-10T08:37:42.668011+0000","last_fullsized":"2024-09-10T08:37:42.668011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:40.594354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:46.702128+0000","last_change":"2024-09-10T08:36:53.734338+0000","last_active":"2024-09-10T08:37:46.702128+0000","last_peered":"2024-09-10T08:37:46.702128+0000","last_clean":"2024-09-10T08:37:46.702128+0000","last_became_active":"2024-09-10T08:36:53.734031+0000","last_became_peered":"2024-09-10T08:36:53.734031+0000","last_unstale":"2024-09-10T08:37:46.702128+0000","last_undegraded":"2024-09-10T08:37:46.702128+0000","last_fullsized":"2024-09-10T08:37:46.702128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:48.922003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.666147+0000","last_change":"2024-09-10T08:37:05.414181+0000","last_active":"2024-09-10T08:37:42.666147+0000","last_peered":"2024-09-10T08:37:42.666147+0000","last_clean":"2024-09-10T08:37:42.666147+0000","last_became_active":"2024-09-10T08:37:05.413691+0000","last_became_peered":"2024-09-10T08:37:05.413691+0000","last_unstale":"2024-09-10T08:37:42.666147+0000","last_undegraded":"2024-09-10T08:37:42.666147+0000","last_fullsized":"2024-09-10T08:37:42.666147+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:30.744949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.667482+0000","last_change":"2024-09-10T08:36:56.153956+0000","last_active":"2024-09-10T08:37:42.667482+0000","last_peered":"2024-09-10T08:37:42.667482+0000","last_clean":"2024-09-10T08:37:42.667482+0000","last_became_active":"2024-09-10T08:36:56.153609+0000","last_became_peered":"2024-09-10T08:36:56.153609+0000","last_unstale":"2024-09-10T08:37:42.667482+0000","last_undegraded":"2024-09-10T08:37:42.667482+0000","last_fullsized":"2024-09-10T08:37:42.667482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:18.185704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:46.701784+0000","last_change":"2024-09-10T08:36:53.733217+0000","last_active":"2024-09-10T08:37:46.701784+0000","last_peered":"2024-09-10T08:37:46.701784+0000","last_clean":"2024-09-10T08:37:46.701784+0000","last_became_active":"2024-09-10T08:36:53.732851+0000","last_became_peered":"2024-09-10T08:36:53.732851+0000","last_unstale":"2024-09-10T08:37:46.701784+0000","last_undegraded":"2024-09-10T08:37:46.701784+0000","last_fullsized":"2024-09-10T08:37:46.701784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:04:01.132196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.288734+0000","last_change":"2024-09-10T08:37:05.241868+0000","last_active":"2024-09-10T08:37:43.288734+0000","last_peered":"2024-09-10T08:37:43.288734+0000","last_clean":"2024-09-10T08:37:43.288734+0000","last_became_active":"2024-09-10T08:37:05.241412+0000","last_became_peered":"2024-09-10T08:37:05.241412+0000","last_unstale":"2024-09-10T08:37:43.288734+0000","last_undegraded":"2024-09-10T08:37:43.288734+0000","last_fullsized":"2024-09-10T08:37:43.288734+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:52.543353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.222307+0000","last_change":"2024-09-10T08:36:56.150722+0000","last_active":"2024-09-10T08:37:43.222307+0000","last_peered":"2024-09-10T08:37:43.222307+0000","last_clean":"2024-09-10T08:37:43.222307+0000","last_became_active":"2024-09-10T08:36:56.150423+0000","last_became_peered":"2024-09-10T08:36:56.150423+0000","last_unstale":"2024-09-10T08:37:43.222307+0000","last_undegraded":"2024-09-10T08:37:43.222307+0000","last_fullsized":"2024-09-10T08:37:43.222307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:50.272841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'6","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:56.160802+0000","last_change":"2024-09-10T08:36:53.722477+0000","last_active":"2024-09-10T08:37:56.160802+0000","last_peered":"2024-09-10T08:37:56.160802+0000","last_clean":"2024-09-10T08:37:56.160802+0000","last_became_active":"2024-09-10T08:36:53.722118+0000","last_became_peered":"2024-09-10T08:36:53.722118+0000","last_unstale":"2024-09-10T08:37:56.160802+0000","last_undegraded":"2024-09-10T08:37:56.160802+0000","last_fullsized":"2024-09-10T08:37:56.160802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:06.631646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.221528+0000","last_change":"2024-09-10T08:37:05.415656+0000","last_active":"2024-09-10T08:37:43.221528+0000","last_peered":"2024-09-10T08:37:43.221528+0000","last_clean":"2024-09-10T08:37:43.221528+0000","last_became_active":"2024-09-10T08:37:05.413537+0000","last_became_peered":"2024-09-10T08:37:05.413537+0000","last_unstale":"2024-09-10T08:37:43.221528+0000","last_undegraded":"2024-09-10T08:37:43.221528+0000","last_fullsized":"2024-09-10T08:37:43.221528+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:04.284410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.665194+0000","last_change":"2024-09-10T08:36:56.144157+0000","last_active":"2024-09-10T08:37:42.665194+0000","last_peered":"2024-09-10T08:37:42.665194+0000","last_clean":"2024-09-10T08:37:42.665194+0000","last_became_active":"2024-09-10T08:36:56.140332+0000","last_became_peered":"2024-09-10T08:36:56.140332+0000","last_unstale":"2024-09-10T08:37:42.665194+0000","last_undegraded":"2024-09-10T08:37:42.665194+0000","last_fullsized":"2024-09-10T08:37:42.665194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:09:26.097987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:46.703188+0000","last_change":"2024-09-10T08:36:53.735465+0000","last_active":"2024-09-10T08:37:46.703188+0000","last_peered":"2024-09-10T08:37:46.703188+0000","last_clean":"2024-09-10T08:37:46.703188+0000","last_became_active":"2024-09-10T08:36:53.735160+0000","last_became_peered":"2024-09-10T08:36:53.735160+0000","last_unstale":"2024-09-10T08:37:46.703188+0000","last_undegraded":"2024-09-10T08:37:46.703188+0000","last_fullsized":"2024-09-10T08:37:46.703188+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:26:52.332085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664143+0000","last_change":"2024-09-10T08:37:05.247671+0000","last_active":"2024-09-10T08:37:42.664143+0000","last_peered":"2024-09-10T08:37:42.664143+0000","last_clean":"2024-09-10T08:37:42.664143+0000","last_became_active":"2024-09-10T08:37:05.247217+0000","last_became_peered":"2024-09-10T08:37:05.247217+0000","last_unstale":"2024-09-10T08:37:42.664143+0000","last_undegraded":"2024-09-10T08:37:42.664143+0000","last_fullsized":"2024-09-10T08:37:42.664143+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:20.881304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.665718+0000","last_change":"2024-09-10T08:36:56.145420+0000","last_active":"2024-09-10T08:37:42.665718+0000","last_peered":"2024-09-10T08:37:42.665718+0000","last_clean":"2024-09-10T08:37:42.665718+0000","last_became_active":"2024-09-10T08:36:56.145074+0000","last_became_peered":"2024-09-10T08:36:56.145074+0000","last_unstale":"2024-09-10T08:37:42.665718+0000","last_undegraded":"2024-09-10T08:37:42.665718+0000","last_fullsized":"2024-09-10T08:37:42.665718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:13.849977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.670084+0000","last_change":"2024-09-10T08:36:53.738237+0000","last_active":"2024-09-10T08:37:42.670084+0000","last_peered":"2024-09-10T08:37:42.670084+0000","last_clean":"2024-09-10T08:37:42.670084+0000","last_became_active":"2024-09-10T08:36:53.737858+0000","last_became_peered":"2024-09-10T08:36:53.737858+0000","last_unstale":"2024-09-10T08:37:42.670084+0000","last_undegraded":"2024-09-10T08:37:42.670084+0000","last_fullsized":"2024-09-10T08:37:42.670084+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:35.345661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668150+0000","last_change":"2024-09-10T08:37:05.413512+0000","last_active":"2024-09-10T08:37:42.668150+0000","last_peered":"2024-09-10T08:37:42.668150+0000","last_clean":"2024-09-10T08:37:42.668150+0000","last_became_active":"2024-09-10T08:37:05.413010+0000","last_became_peered":"2024-09-10T08:37:05.413010+0000","last_unstale":"2024-09-10T08:37:42.668150+0000","last_undegraded":"2024-09-10T08:37:42.668150+0000","last_fullsized":"2024-09-10T08:37:42.668150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:05.349404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.665995+0000","last_change":"2024-09-10T08:36:56.157741+0000","last_active":"2024-09-10T08:37:42.665995+0000","last_peered":"2024-09-10T08:37:42.665995+0000","last_clean":"2024-09-10T08:37:42.665995+0000","last_became_active":"2024-09-10T08:36:56.157295+0000","last_became_peered":"2024-09-10T08:36:56.157295+0000","last_unstale":"2024-09-10T08:37:42.665995+0000","last_undegraded":"2024-09-10T08:37:42.665995+0000","last_fullsized":"2024-09-10T08:37:42.665995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:31.845864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668956+0000","last_change":"2024-09-10T08:36:53.722373+0000","last_active":"2024-09-10T08:37:42.668956+0000","last_peered":"2024-09-10T08:37:42.668956+0000","last_clean":"2024-09-10T08:37:42.668956+0000","last_became_active":"2024-09-10T08:36:53.721975+0000","last_became_peered":"2024-09-10T08:36:53.721975+0000","last_unstale":"2024-09-10T08:37:42.668956+0000","last_undegraded":"2024-09-10T08:37:42.668956+0000","last_fullsized":"2024-09-10T08:37:42.668956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:44.593018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.665010+0000","last_change":"2024-09-10T08:37:05.255362+0000","last_active":"2024-09-10T08:37:42.665010+0000","last_peered":"2024-09-10T08:37:42.665010+0000","last_clean":"2024-09-10T08:37:42.665010+0000","last_became_active":"2024-09-10T08:37:05.254967+0000","last_became_peered":"2024-09-10T08:37:05.254967+0000","last_unstale":"2024-09-10T08:37:42.665010+0000","last_undegraded":"2024-09-10T08:37:42.665010+0000","last_fullsized":"2024-09-10T08:37:42.665010+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:31:41.850961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.844332+0000","last_change":"2024-09-10T08:36:56.147699+0000","last_active":"2024-09-10T08:37:42.844332+0000","last_peered":"2024-09-10T08:37:42.844332+0000","last_clean":"2024-09-10T08:37:42.844332+0000","last_became_active":"2024-09-10T08:36:56.147349+0000","last_became_peered":"2024-09-10T08:36:56.147349+0000","last_unstale":"2024-09-10T08:37:42.844332+0000","last_undegraded":"2024-09-10T08:37:42.844332+0000","last_fullsized":"2024-09-10T08:37:42.844332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:25.248588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'4","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:58.736477+0000","last_change":"2024-09-10T08:36:53.736843+0000","last_active":"2024-09-10T08:37:58.736477+0000","last_peered":"2024-09-10T08:37:58.736477+0000","last_clean":"2024-09-10T08:37:58.736477+0000","last_became_active":"2024-09-10T08:36:53.736340+0000","last_became_peered":"2024-09-10T08:36:53.736340+0000","last_unstale":"2024-09-10T08:37:58.736477+0000","last_undegraded":"2024-09-10T08:37:58.736477+0000","last_fullsized":"2024-09-10T08:37:58.736477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:24.679679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664817+0000","last_change":"2024-09-10T08:37:05.414321+0000","last_active":"2024-09-10T08:37:42.664817+0000","last_peered":"2024-09-10T08:37:42.664817+0000","last_clean":"2024-09-10T08:37:42.664817+0000","last_became_active":"2024-09-10T08:37:05.413868+0000","last_became_peered":"2024-09-10T08:37:05.413868+0000","last_unstale":"2024-09-10T08:37:42.664817+0000","last_undegraded":"2024-09-10T08:37:42.664817+0000","last_fullsized":"2024-09-10T08:37:42.664817+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:53.988077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664070+0000","last_change":"2024-09-10T08:36:56.141910+0000","last_active":"2024-09-10T08:37:42.664070+0000","last_peered":"2024-09-10T08:37:42.664070+0000","last_clean":"2024-09-10T08:37:42.664070+0000","last_became_active":"2024-09-10T08:36:56.141579+0000","last_became_peered":"2024-09-10T08:36:56.141579+0000","last_unstale":"2024-09-10T08:37:42.664070+0000","last_undegraded":"2024-09-10T08:37:42.664070+0000","last_fullsized":"2024-09-10T08:37:42.664070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:18.464831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:47.712490+0000","last_change":"2024-09-10T08:36:53.720102+0000","last_active":"2024-09-10T08:37:47.712490+0000","last_peered":"2024-09-10T08:37:47.712490+0000","last_clean":"2024-09-10T08:37:47.712490+0000","last_became_active":"2024-09-10T08:36:53.719764+0000","last_became_peered":"2024-09-10T08:36:53.719764+0000","last_unstale":"2024-09-10T08:37:47.712490+0000","last_undegraded":"2024-09-10T08:37:47.712490+0000","last_fullsized":"2024-09-10T08:37:47.712490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:03.609338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.844144+0000","last_change":"2024-09-10T08:37:05.413818+0000","last_active":"2024-09-10T08:37:42.844144+0000","last_peered":"2024-09-10T08:37:42.844144+0000","last_clean":"2024-09-10T08:37:42.844144+0000","last_became_active":"2024-09-10T08:37:05.413267+0000","last_became_peered":"2024-09-10T08:37:05.413267+0000","last_unstale":"2024-09-10T08:37:42.844144+0000","last_undegraded":"2024-09-10T08:37:42.844144+0000","last_fullsized":"2024-09-10T08:37:42.844144+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:54.309686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.222009+0000","last_change":"2024-09-10T08:36:56.149286+0000","last_active":"2024-09-10T08:37:43.222009+0000","last_peered":"2024-09-10T08:37:43.222009+0000","last_clean":"2024-09-10T08:37:43.222009+0000","last_became_active":"2024-09-10T08:36:56.146275+0000","last_became_peered":"2024-09-10T08:36:56.146275+0000","last_unstale":"2024-09-10T08:37:43.222009+0000","last_undegraded":"2024-09-10T08:37:43.222009+0000","last_fullsized":"2024-09-10T08:37:43.222009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:19.468224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:48.719938+0000","last_change":"2024-09-10T08:36:53.719756+0000","last_active":"2024-09-10T08:37:48.719938+0000","last_peered":"2024-09-10T08:37:48.719938+0000","last_clean":"2024-09-10T08:37:48.719938+0000","last_became_active":"2024-09-10T08:36:53.719302+0000","last_became_peered":"2024-09-10T08:36:53.719302+0000","last_unstale":"2024-09-10T08:37:48.719938+0000","last_undegraded":"2024-09-10T08:37:48.719938+0000","last_fullsized":"2024-09-10T08:37:48.719938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:47.658081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":352,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668541+0000","last_change":"2024-09-10T08:35:39.150715+0000","last_active":"2024-09-10T08:37:42.668541+0000","last_peered":"2024-09-10T08:37:42.668541+0000","last_clean":"2024-09-10T08:37:42.668541+0000","last_became_active":"2024-09-10T08:35:39.150206+0000","last_became_peered":"2024-09-10T08:35:39.150206+0000","last_unstale":"2024-09-10T08:37:42.668541+0000","last_undegraded":"2024-09-10T08:37:42.668541+0000","last_fullsized":"2024-09-10T08:37:42.668541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:32:02.110870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:32:02.110870+0000","last_clean_scrub_stamp":"2024-09-10T08:32:02.110870+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:20.762043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.847198+0000","last_change":"2024-09-10T08:37:42.846547+0000","last_active":"2024-09-10T08:37:42.847198+0000","last_peered":"2024-09-10T08:37:42.847198+0000","last_clean":"2024-09-10T08:37:42.847198+0000","last_became_active":"2024-09-10T08:37:42.845884+0000","last_became_peered":"2024-09-10T08:37:42.845884+0000","last_unstale":"2024-09-10T08:37:42.847198+0000","last_undegraded":"2024-09-10T08:37:42.847198+0000","last_fullsized":"2024-09-10T08:37:42.847198+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:21.905133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.669368+0000","last_change":"2024-09-10T08:36:56.144187+0000","last_active":"2024-09-10T08:37:42.669368+0000","last_peered":"2024-09-10T08:37:42.669368+0000","last_clean":"2024-09-10T08:37:42.669368+0000","last_became_active":"2024-09-10T08:36:56.143193+0000","last_became_peered":"2024-09-10T08:36:56.143193+0000","last_unstale":"2024-09-10T08:37:42.669368+0000","last_undegraded":"2024-09-10T08:37:42.669368+0000","last_fullsized":"2024-09-10T08:37:42.669368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:14:26.981884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:47.713979+0000","last_change":"2024-09-10T08:36:53.737076+0000","last_active":"2024-09-10T08:37:47.713979+0000","last_peered":"2024-09-10T08:37:47.713979+0000","last_clean":"2024-09-10T08:37:47.713979+0000","last_became_active":"2024-09-10T08:36:53.736756+0000","last_became_peered":"2024-09-10T08:36:53.736756+0000","last_unstale":"2024-09-10T08:37:47.713979+0000","last_undegraded":"2024-09-10T08:37:47.713979+0000","last_fullsized":"2024-09-10T08:37:47.713979+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:04:06.526018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.288890+0000","last_change":"2024-09-10T08:37:05.242095+0000","last_active":"2024-09-10T08:37:43.288890+0000","last_peered":"2024-09-10T08:37:43.288890+0000","last_clean":"2024-09-10T08:37:43.288890+0000","last_became_active":"2024-09-10T08:37:05.241645+0000","last_became_peered":"2024-09-10T08:37:05.241645+0000","last_unstale":"2024-09-10T08:37:43.288890+0000","last_undegraded":"2024-09-10T08:37:43.288890+0000","last_fullsized":"2024-09-10T08:37:43.288890+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:30:49.637861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.669809+0000","last_change":"2024-09-10T08:36:56.149396+0000","last_active":"2024-09-10T08:37:42.669809+0000","last_peered":"2024-09-10T08:37:42.669809+0000","last_clean":"2024-09-10T08:37:42.669809+0000","last_became_active":"2024-09-10T08:36:56.149079+0000","last_became_peered":"2024-09-10T08:36:56.149079+0000","last_unstale":"2024-09-10T08:37:42.669809+0000","last_undegraded":"2024-09-10T08:37:42.669809+0000","last_fullsized":"2024-09-10T08:37:42.669809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:44.665448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.844798+0000","last_change":"2024-09-10T08:36:53.717950+0000","last_active":"2024-09-10T08:37:42.844798+0000","last_peered":"2024-09-10T08:37:42.844798+0000","last_clean":"2024-09-10T08:37:42.844798+0000","last_became_active":"2024-09-10T08:36:53.717668+0000","last_became_peered":"2024-09-10T08:36:53.717668+0000","last_unstale":"2024-09-10T08:37:42.844798+0000","last_undegraded":"2024-09-10T08:37:42.844798+0000","last_fullsized":"2024-09-10T08:37:42.844798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:58.692631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.666895+0000","last_change":"2024-09-10T08:37:05.260178+0000","last_active":"2024-09-10T08:37:42.666895+0000","last_peered":"2024-09-10T08:37:42.666895+0000","last_clean":"2024-09-10T08:37:42.666895+0000","last_became_active":"2024-09-10T08:37:05.259753+0000","last_became_peered":"2024-09-10T08:37:05.259753+0000","last_unstale":"2024-09-10T08:37:42.666895+0000","last_undegraded":"2024-09-10T08:37:42.666895+0000","last_fullsized":"2024-09-10T08:37:42.666895+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:19.364782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.669127+0000","last_change":"2024-09-10T08:36:56.147104+0000","last_active":"2024-09-10T08:37:42.669127+0000","last_peered":"2024-09-10T08:37:42.669127+0000","last_clean":"2024-09-10T08:37:42.669127+0000","last_became_active":"2024-09-10T08:36:56.146778+0000","last_became_peered":"2024-09-10T08:36:56.146778+0000","last_unstale":"2024-09-10T08:37:42.669127+0000","last_undegraded":"2024-09-10T08:37:42.669127+0000","last_fullsized":"2024-09-10T08:37:42.669127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:56.092157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664936+0000","last_change":"2024-09-10T08:36:53.719145+0000","last_active":"2024-09-10T08:37:42.664936+0000","last_peered":"2024-09-10T08:37:42.664936+0000","last_clean":"2024-09-10T08:37:42.664936+0000","last_became_active":"2024-09-10T08:36:53.718771+0000","last_became_peered":"2024-09-10T08:36:53.718771+0000","last_unstale":"2024-09-10T08:37:42.664936+0000","last_undegraded":"2024-09-10T08:37:42.664936+0000","last_fullsized":"2024-09-10T08:37:42.664936+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:21.109940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.843849+0000","last_change":"2024-09-10T08:37:05.242854+0000","last_active":"2024-09-10T08:37:42.843849+0000","last_peered":"2024-09-10T08:37:42.843849+0000","last_clean":"2024-09-10T08:37:42.843849+0000","last_became_active":"2024-09-10T08:37:05.242442+0000","last_became_peered":"2024-09-10T08:37:05.242442+0000","last_unstale":"2024-09-10T08:37:42.843849+0000","last_undegraded":"2024-09-10T08:37:42.843849+0000","last_fullsized":"2024-09-10T08:37:42.843849+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:36.072131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.845642+0000","last_change":"2024-09-10T08:37:42.845642+0000","last_active":"2024-09-10T08:37:42.845642+0000","last_peered":"2024-09-10T08:37:42.845642+0000","last_clean":"2024-09-10T08:37:42.845642+0000","last_became_active":"2024-09-10T08:37:42.845217+0000","last_became_peered":"2024-09-10T08:37:42.845217+0000","last_unstale":"2024-09-10T08:37:42.845642+0000","last_undegraded":"2024-09-10T08:37:42.845642+0000","last_fullsized":"2024-09-10T08:37:42.845642+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:28.020337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:48.719014+0000","last_change":"2024-09-10T08:36:53.720463+0000","last_active":"2024-09-10T08:37:48.719014+0000","last_peered":"2024-09-10T08:37:48.719014+0000","last_clean":"2024-09-10T08:37:48.719014+0000","last_became_active":"2024-09-10T08:36:53.720080+0000","last_became_peered":"2024-09-10T08:36:53.720080+0000","last_unstale":"2024-09-10T08:37:48.719014+0000","last_undegraded":"2024-09-10T08:37:48.719014+0000","last_fullsized":"2024-09-10T08:37:48.719014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:51.450294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.843769+0000","last_change":"2024-09-10T08:37:05.251261+0000","last_active":"2024-09-10T08:37:42.843769+0000","last_peered":"2024-09-10T08:37:42.843769+0000","last_clean":"2024-09-10T08:37:42.843769+0000","last_became_active":"2024-09-10T08:37:05.250938+0000","last_became_peered":"2024-09-10T08:37:05.250938+0000","last_unstale":"2024-09-10T08:37:42.843769+0000","last_undegraded":"2024-09-10T08:37:42.843769+0000","last_fullsized":"2024-09-10T08:37:42.843769+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:07:15.129899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.843971+0000","last_change":"2024-09-10T08:36:56.148191+0000","last_active":"2024-09-10T08:37:42.843971+0000","last_peered":"2024-09-10T08:37:42.843971+0000","last_clean":"2024-09-10T08:37:42.843971+0000","last_became_active":"2024-09-10T08:36:56.147362+0000","last_became_peered":"2024-09-10T08:36:56.147362+0000","last_unstale":"2024-09-10T08:37:42.843971+0000","last_undegraded":"2024-09-10T08:37:42.843971+0000","last_fullsized":"2024-09-10T08:37:42.843971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:25:30.051754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:47.714375+0000","last_change":"2024-09-10T08:36:53.734484+0000","last_active":"2024-09-10T08:37:47.714375+0000","last_peered":"2024-09-10T08:37:47.714375+0000","last_clean":"2024-09-10T08:37:47.714375+0000","last_became_active":"2024-09-10T08:36:53.734172+0000","last_became_peered":"2024-09-10T08:36:53.734172+0000","last_unstale":"2024-09-10T08:37:47.714375+0000","last_undegraded":"2024-09-10T08:37:47.714375+0000","last_fullsized":"2024-09-10T08:37:47.714375+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:10.269400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664883+0000","last_change":"2024-09-10T08:37:05.259635+0000","last_active":"2024-09-10T08:37:42.664883+0000","last_peered":"2024-09-10T08:37:42.664883+0000","last_clean":"2024-09-10T08:37:42.664883+0000","last_became_active":"2024-09-10T08:37:05.259107+0000","last_became_peered":"2024-09-10T08:37:05.259107+0000","last_unstale":"2024-09-10T08:37:42.664883+0000","last_undegraded":"2024-09-10T08:37:42.664883+0000","last_fullsized":"2024-09-10T08:37:42.664883+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:36.841138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668126+0000","last_change":"2024-09-10T08:36:56.141122+0000","last_active":"2024-09-10T08:37:42.668126+0000","last_peered":"2024-09-10T08:37:42.668126+0000","last_clean":"2024-09-10T08:37:42.668126+0000","last_became_active":"2024-09-10T08:36:56.140826+0000","last_became_peered":"2024-09-10T08:36:56.140826+0000","last_unstale":"2024-09-10T08:37:42.668126+0000","last_undegraded":"2024-09-10T08:37:42.668126+0000","last_fullsized":"2024-09-10T08:37:42.668126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:38.044090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'4","reported_seq":55,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:58.736921+0000","last_change":"2024-09-10T08:36:53.720623+0000","last_active":"2024-09-10T08:37:58.736921+0000","last_peered":"2024-09-10T08:37:58.736921+0000","last_clean":"2024-09-10T08:37:58.736921+0000","last_became_active":"2024-09-10T08:36:53.720270+0000","last_became_peered":"2024-09-10T08:36:53.720270+0000","last_unstale":"2024-09-10T08:37:58.736921+0000","last_undegraded":"2024-09-10T08:37:58.736921+0000","last_fullsized":"2024-09-10T08:37:58.736921+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:19.514068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.669881+0000","last_change":"2024-09-10T08:37:05.254324+0000","last_active":"2024-09-10T08:37:42.669881+0000","last_peered":"2024-09-10T08:37:42.669881+0000","last_clean":"2024-09-10T08:37:42.669881+0000","last_became_active":"2024-09-10T08:37:05.253611+0000","last_became_peered":"2024-09-10T08:37:05.253611+0000","last_unstale":"2024-09-10T08:37:42.669881+0000","last_undegraded":"2024-09-10T08:37:42.669881+0000","last_fullsized":"2024-09-10T08:37:42.669881+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:07.182649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668627+0000","last_change":"2024-09-10T08:36:56.139102+0000","last_active":"2024-09-10T08:37:42.668627+0000","last_peered":"2024-09-10T08:37:42.668627+0000","last_clean":"2024-09-10T08:37:42.668627+0000","last_became_active":"2024-09-10T08:36:56.137986+0000","last_became_peered":"2024-09-10T08:36:56.137986+0000","last_unstale":"2024-09-10T08:37:42.668627+0000","last_undegraded":"2024-09-10T08:37:42.668627+0000","last_fullsized":"2024-09-10T08:37:42.668627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:05:03.005727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:46.702276+0000","last_change":"2024-09-10T08:36:53.722961+0000","last_active":"2024-09-10T08:37:46.702276+0000","last_peered":"2024-09-10T08:37:46.702276+0000","last_clean":"2024-09-10T08:37:46.702276+0000","last_became_active":"2024-09-10T08:36:53.722628+0000","last_became_peered":"2024-09-10T08:36:53.722628+0000","last_unstale":"2024-09-10T08:37:46.702276+0000","last_undegraded":"2024-09-10T08:37:46.702276+0000","last_fullsized":"2024-09-10T08:37:46.702276+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:59.048632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664305+0000","last_change":"2024-09-10T08:37:05.256546+0000","last_active":"2024-09-10T08:37:42.664305+0000","last_peered":"2024-09-10T08:37:42.664305+0000","last_clean":"2024-09-10T08:37:42.664305+0000","last_became_active":"2024-09-10T08:37:05.256239+0000","last_became_peered":"2024-09-10T08:37:05.256239+0000","last_unstale":"2024-09-10T08:37:42.664305+0000","last_undegraded":"2024-09-10T08:37:42.664305+0000","last_fullsized":"2024-09-10T08:37:42.664305+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:57.403934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664318+0000","last_change":"2024-09-10T08:36:56.149285+0000","last_active":"2024-09-10T08:37:42.664318+0000","last_peered":"2024-09-10T08:37:42.664318+0000","last_clean":"2024-09-10T08:37:42.664318+0000","last_became_active":"2024-09-10T08:36:56.146873+0000","last_became_peered":"2024-09-10T08:36:56.146873+0000","last_unstale":"2024-09-10T08:37:42.664318+0000","last_undegraded":"2024-09-10T08:37:42.664318+0000","last_fullsized":"2024-09-10T08:37:42.664318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:28.976729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:47.714300+0000","last_change":"2024-09-10T08:36:53.721174+0000","last_active":"2024-09-10T08:37:47.714300+0000","last_peered":"2024-09-10T08:37:47.714300+0000","last_clean":"2024-09-10T08:37:47.714300+0000","last_became_active":"2024-09-10T08:36:53.720849+0000","last_became_peered":"2024-09-10T08:36:53.720849+0000","last_unstale":"2024-09-10T08:37:47.714300+0000","last_undegraded":"2024-09-10T08:37:47.714300+0000","last_fullsized":"2024-09-10T08:37:47.714300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:39.786848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.844261+0000","last_change":"2024-09-10T08:37:05.252256+0000","last_active":"2024-09-10T08:37:42.844261+0000","last_peered":"2024-09-10T08:37:42.844261+0000","last_clean":"2024-09-10T08:37:42.844261+0000","last_became_active":"2024-09-10T08:37:05.251865+0000","last_became_peered":"2024-09-10T08:37:05.251865+0000","last_unstale":"2024-09-10T08:37:42.844261+0000","last_undegraded":"2024-09-10T08:37:42.844261+0000","last_fullsized":"2024-09-10T08:37:42.844261+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:08.581880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668571+0000","last_change":"2024-09-10T08:36:56.142060+0000","last_active":"2024-09-10T08:37:42.668571+0000","last_peered":"2024-09-10T08:37:42.668571+0000","last_clean":"2024-09-10T08:37:42.668571+0000","last_became_active":"2024-09-10T08:36:56.141451+0000","last_became_peered":"2024-09-10T08:36:56.141451+0000","last_unstale":"2024-09-10T08:37:42.668571+0000","last_undegraded":"2024-09-10T08:37:42.668571+0000","last_fullsized":"2024-09-10T08:37:42.668571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:43.566588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'26","reported_seq":100,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:38:03.743469+0000","last_change":"2024-09-10T08:36:53.719464+0000","last_active":"2024-09-10T08:38:03.743469+0000","last_peered":"2024-09-10T08:38:03.743469+0000","last_clean":"2024-09-10T08:38:03.743469+0000","last_became_active":"2024-09-10T08:36:53.719129+0000","last_became_peered":"2024-09-10T08:36:53.719129+0000","last_unstale":"2024-09-10T08:38:03.743469+0000","last_undegraded":"2024-09-10T08:38:03.743469+0000","last_fullsized":"2024-09-10T08:38:03.743469+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:05.652018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49098,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":67,"num_write":27,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.669325+0000","last_change":"2024-09-10T08:37:05.255576+0000","last_active":"2024-09-10T08:37:42.669325+0000","last_peered":"2024-09-10T08:37:42.669325+0000","last_clean":"2024-09-10T08:37:42.669325+0000","last_became_active":"2024-09-10T08:37:05.249842+0000","last_became_peered":"2024-09-10T08:37:05.249842+0000","last_unstale":"2024-09-10T08:37:42.669325+0000","last_undegraded":"2024-09-10T08:37:42.669325+0000","last_fullsized":"2024-09-10T08:37:42.669325+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:47:49.543481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.289176+0000","last_change":"2024-09-10T08:36:56.151829+0000","last_active":"2024-09-10T08:37:43.289176+0000","last_peered":"2024-09-10T08:37:43.289176+0000","last_clean":"2024-09-10T08:37:43.289176+0000","last_became_active":"2024-09-10T08:36:56.151314+0000","last_became_peered":"2024-09-10T08:36:56.151314+0000","last_unstale":"2024-09-10T08:37:43.289176+0000","last_undegraded":"2024-09-10T08:37:43.289176+0000","last_fullsized":"2024-09-10T08:37:43.289176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:44.219671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:46.702055+0000","last_change":"2024-09-10T08:36:53.738030+0000","last_active":"2024-09-10T08:37:46.702055+0000","last_peered":"2024-09-10T08:37:46.702055+0000","last_clean":"2024-09-10T08:37:46.702055+0000","last_became_active":"2024-09-10T08:36:53.737658+0000","last_became_peered":"2024-09-10T08:36:53.737658+0000","last_unstale":"2024-09-10T08:37:46.702055+0000","last_undegraded":"2024-09-10T08:37:46.702055+0000","last_fullsized":"2024-09-10T08:37:46.702055+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:29.627554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.666129+0000","last_change":"2024-09-10T08:37:05.244473+0000","last_active":"2024-09-10T08:37:42.666129+0000","last_peered":"2024-09-10T08:37:42.666129+0000","last_clean":"2024-09-10T08:37:42.666129+0000","last_became_active":"2024-09-10T08:37:05.244035+0000","last_became_peered":"2024-09-10T08:37:05.244035+0000","last_unstale":"2024-09-10T08:37:42.666129+0000","last_undegraded":"2024-09-10T08:37:42.666129+0000","last_fullsized":"2024-09-10T08:37:42.666129+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:04.652660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.667575+0000","last_change":"2024-09-10T08:36:56.155217+0000","last_active":"2024-09-10T08:37:42.667575+0000","last_peered":"2024-09-10T08:37:42.667575+0000","last_clean":"2024-09-10T08:37:42.667575+0000","last_became_active":"2024-09-10T08:36:56.154903+0000","last_became_peered":"2024-09-10T08:36:56.154903+0000","last_unstale":"2024-09-10T08:37:42.667575+0000","last_undegraded":"2024-09-10T08:37:42.667575+0000","last_fullsized":"2024-09-10T08:37:42.667575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:47.482974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:46.701861+0000","last_change":"2024-09-10T08:36:53.721087+0000","last_active":"2024-09-10T08:37:46.701861+0000","last_peered":"2024-09-10T08:37:46.701861+0000","last_clean":"2024-09-10T08:37:46.701861+0000","last_became_active":"2024-09-10T08:36:53.720772+0000","last_became_peered":"2024-09-10T08:36:53.720772+0000","last_unstale":"2024-09-10T08:37:46.701861+0000","last_undegraded":"2024-09-10T08:37:46.701861+0000","last_fullsized":"2024-09-10T08:37:46.701861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:39.560848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668820+0000","last_change":"2024-09-10T08:37:05.254641+0000","last_active":"2024-09-10T08:37:42.668820+0000","last_peered":"2024-09-10T08:37:42.668820+0000","last_clean":"2024-09-10T08:37:42.668820+0000","last_became_active":"2024-09-10T08:37:05.249982+0000","last_became_peered":"2024-09-10T08:37:05.249982+0000","last_unstale":"2024-09-10T08:37:42.668820+0000","last_undegraded":"2024-09-10T08:37:42.668820+0000","last_fullsized":"2024-09-10T08:37:42.668820+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:21:13.851061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.289336+0000","last_change":"2024-09-10T08:36:56.142657+0000","last_active":"2024-09-10T08:37:43.289336+0000","last_peered":"2024-09-10T08:37:43.289336+0000","last_clean":"2024-09-10T08:37:43.289336+0000","last_became_active":"2024-09-10T08:36:56.142294+0000","last_became_peered":"2024-09-10T08:36:56.142294+0000","last_unstale":"2024-09-10T08:37:43.289336+0000","last_undegraded":"2024-09-10T08:37:43.289336+0000","last_fullsized":"2024-09-10T08:37:43.289336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:52.861389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"94'3","reported_seq":55,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:38:02.741968+0000","last_change":"2024-09-10T08:36:53.722466+0000","last_active":"2024-09-10T08:38:02.741968+0000","last_peered":"2024-09-10T08:38:02.741968+0000","last_clean":"2024-09-10T08:38:02.741968+0000","last_became_active":"2024-09-10T08:36:53.722145+0000","last_became_peered":"2024-09-10T08:36:53.722145+0000","last_unstale":"2024-09-10T08:38:02.741968+0000","last_undegraded":"2024-09-10T08:38:02.741968+0000","last_fullsized":"2024-09-10T08:38:02.741968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:27.041765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.221668+0000","last_change":"2024-09-10T08:37:05.251056+0000","last_active":"2024-09-10T08:37:43.221668+0000","last_peered":"2024-09-10T08:37:43.221668+0000","last_clean":"2024-09-10T08:37:43.221668+0000","last_became_active":"2024-09-10T08:37:05.250653+0000","last_became_peered":"2024-09-10T08:37:05.250653+0000","last_unstale":"2024-09-10T08:37:43.221668+0000","last_undegraded":"2024-09-10T08:37:43.221668+0000","last_fullsized":"2024-09-10T08:37:43.221668+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:21.174747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.667368+0000","last_change":"2024-09-10T08:37:05.260436+0000","last_active":"2024-09-10T08:37:42.667368+0000","last_peered":"2024-09-10T08:37:42.667368+0000","last_clean":"2024-09-10T08:37:42.667368+0000","last_became_active":"2024-09-10T08:37:05.260049+0000","last_became_peered":"2024-09-10T08:37:05.260049+0000","last_unstale":"2024-09-10T08:37:42.667368+0000","last_undegraded":"2024-09-10T08:37:42.667368+0000","last_fullsized":"2024-09-10T08:37:42.667368+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:43.394028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.669027+0000","last_change":"2024-09-10T08:36:56.146732+0000","last_active":"2024-09-10T08:37:42.669027+0000","last_peered":"2024-09-10T08:37:42.669027+0000","last_clean":"2024-09-10T08:37:42.669027+0000","last_became_active":"2024-09-10T08:36:56.145599+0000","last_became_peered":"2024-09-10T08:36:56.145599+0000","last_unstale":"2024-09-10T08:37:42.669027+0000","last_undegraded":"2024-09-10T08:37:42.669027+0000","last_fullsized":"2024-09-10T08:37:42.669027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:50.620956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.844518+0000","last_change":"2024-09-10T08:36:53.721683+0000","last_active":"2024-09-10T08:37:42.844518+0000","last_peered":"2024-09-10T08:37:42.844518+0000","last_clean":"2024-09-10T08:37:42.844518+0000","last_became_active":"2024-09-10T08:36:53.721241+0000","last_became_peered":"2024-09-10T08:36:53.721241+0000","last_unstale":"2024-09-10T08:37:42.844518+0000","last_undegraded":"2024-09-10T08:37:42.844518+0000","last_fullsized":"2024-09-10T08:37:42.844518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:03.669337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.669784+0000","last_change":"2024-09-10T08:37:42.669784+0000","last_active":"2024-09-10T08:37:42.669784+0000","last_peered":"2024-09-10T08:37:42.669784+0000","last_clean":"2024-09-10T08:37:42.669784+0000","last_became_active":"2024-09-10T08:37:42.669322+0000","last_became_peered":"2024-09-10T08:37:42.669322+0000","last_unstale":"2024-09-10T08:37:42.669784+0000","last_undegraded":"2024-09-10T08:37:42.669784+0000","last_fullsized":"2024-09-10T08:37:42.669784+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:06:57.027928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.221647+0000","last_change":"2024-09-10T08:36:56.143824+0000","last_active":"2024-09-10T08:37:43.221647+0000","last_peered":"2024-09-10T08:37:43.221647+0000","last_clean":"2024-09-10T08:37:43.221647+0000","last_became_active":"2024-09-10T08:36:56.143486+0000","last_became_peered":"2024-09-10T08:36:56.143486+0000","last_unstale":"2024-09-10T08:37:43.221647+0000","last_undegraded":"2024-09-10T08:37:43.221647+0000","last_fullsized":"2024-09-10T08:37:43.221647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:42.384863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:47.713929+0000","last_change":"2024-09-10T08:36:53.717680+0000","last_active":"2024-09-10T08:37:47.713929+0000","last_peered":"2024-09-10T08:37:47.713929+0000","last_clean":"2024-09-10T08:37:47.713929+0000","last_became_active":"2024-09-10T08:36:53.717249+0000","last_became_peered":"2024-09-10T08:36:53.717249+0000","last_unstale":"2024-09-10T08:37:47.713929+0000","last_undegraded":"2024-09-10T08:37:47.713929+0000","last_fullsized":"2024-09-10T08:37:47.713929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:12.191541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:47.714060+0000","last_change":"2024-09-10T08:36:53.723947+0000","last_active":"2024-09-10T08:37:47.714060+0000","last_peered":"2024-09-10T08:37:47.714060+0000","last_clean":"2024-09-10T08:37:47.714060+0000","last_became_active":"2024-09-10T08:36:53.723620+0000","last_became_peered":"2024-09-10T08:36:53.723620+0000","last_unstale":"2024-09-10T08:37:47.714060+0000","last_undegraded":"2024-09-10T08:37:47.714060+0000","last_fullsized":"2024-09-10T08:37:47.714060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:00.135349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.844083+0000","last_change":"2024-09-10T08:36:56.150448+0000","last_active":"2024-09-10T08:37:42.844083+0000","last_peered":"2024-09-10T08:37:42.844083+0000","last_clean":"2024-09-10T08:37:42.844083+0000","last_became_active":"2024-09-10T08:36:56.150148+0000","last_became_peered":"2024-09-10T08:36:56.150148+0000","last_unstale":"2024-09-10T08:37:42.844083+0000","last_undegraded":"2024-09-10T08:37:42.844083+0000","last_fullsized":"2024-09-10T08:37:42.844083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:02.927753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.288941+0000","last_change":"2024-09-10T08:37:05.242878+0000","last_active":"2024-09-10T08:37:43.288941+0000","last_peered":"2024-09-10T08:37:43.288941+0000","last_clean":"2024-09-10T08:37:43.288941+0000","last_became_active":"2024-09-10T08:37:05.242440+0000","last_became_peered":"2024-09-10T08:37:05.242440+0000","last_unstale":"2024-09-10T08:37:43.288941+0000","last_undegraded":"2024-09-10T08:37:43.288941+0000","last_fullsized":"2024-09-10T08:37:43.288941+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:30.957433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'5","reported_seq":56,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:59.737800+0000","last_change":"2024-09-10T08:36:53.719788+0000","last_active":"2024-09-10T08:37:59.737800+0000","last_peered":"2024-09-10T08:37:59.737800+0000","last_clean":"2024-09-10T08:37:59.737800+0000","last_became_active":"2024-09-10T08:36:53.719462+0000","last_became_peered":"2024-09-10T08:36:53.719462+0000","last_unstale":"2024-09-10T08:37:59.737800+0000","last_undegraded":"2024-09-10T08:37:59.737800+0000","last_fullsized":"2024-09-10T08:37:59.737800+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:12.250447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.673008+0000","last_change":"2024-09-10T08:37:42.673008+0000","last_active":"2024-09-10T08:37:42.673008+0000","last_peered":"2024-09-10T08:37:42.673008+0000","last_clean":"2024-09-10T08:37:42.673008+0000","last_became_active":"2024-09-10T08:37:42.672618+0000","last_became_peered":"2024-09-10T08:37:42.672618+0000","last_unstale":"2024-09-10T08:37:42.673008+0000","last_undegraded":"2024-09-10T08:37:42.673008+0000","last_fullsized":"2024-09-10T08:37:42.673008+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:00.156565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.289122+0000","last_change":"2024-09-10T08:37:05.244277+0000","last_active":"2024-09-10T08:37:43.289122+0000","last_peered":"2024-09-10T08:37:43.289122+0000","last_clean":"2024-09-10T08:37:43.289122+0000","last_became_active":"2024-09-10T08:37:05.243880+0000","last_became_peered":"2024-09-10T08:37:05.243880+0000","last_unstale":"2024-09-10T08:37:43.289122+0000","last_undegraded":"2024-09-10T08:37:43.289122+0000","last_fullsized":"2024-09-10T08:37:43.289122+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:55.275857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.663983+0000","last_change":"2024-09-10T08:36:56.141404+0000","last_active":"2024-09-10T08:37:42.663983+0000","last_peered":"2024-09-10T08:37:42.663983+0000","last_clean":"2024-09-10T08:37:42.663983+0000","last_became_active":"2024-09-10T08:36:56.141076+0000","last_became_peered":"2024-09-10T08:36:56.141076+0000","last_unstale":"2024-09-10T08:37:42.663983+0000","last_undegraded":"2024-09-10T08:37:42.663983+0000","last_fullsized":"2024-09-10T08:37:42.663983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:03:52.915654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.289101+0000","last_change":"2024-09-10T08:36:56.148285+0000","last_active":"2024-09-10T08:37:43.289101+0000","last_peered":"2024-09-10T08:37:43.289101+0000","last_clean":"2024-09-10T08:37:43.289101+0000","last_became_active":"2024-09-10T08:36:56.147970+0000","last_became_peered":"2024-09-10T08:36:56.147970+0000","last_unstale":"2024-09-10T08:37:43.289101+0000","last_undegraded":"2024-09-10T08:37:43.289101+0000","last_fullsized":"2024-09-10T08:37:43.289101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:09.506718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.288789+0000","last_change":"2024-09-10T08:36:56.157977+0000","last_active":"2024-09-10T08:37:43.288789+0000","last_peered":"2024-09-10T08:37:43.288789+0000","last_clean":"2024-09-10T08:37:43.288789+0000","last_became_active":"2024-09-10T08:36:56.157623+0000","last_became_peered":"2024-09-10T08:36:56.157623+0000","last_unstale":"2024-09-10T08:37:43.288789+0000","last_undegraded":"2024-09-10T08:37:43.288789+0000","last_fullsized":"2024-09-10T08:37:43.288789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:44.617979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.670459+0000","last_change":"2024-09-10T08:37:42.670459+0000","last_active":"2024-09-10T08:37:42.670459+0000","last_peered":"2024-09-10T08:37:42.670459+0000","last_clean":"2024-09-10T08:37:42.670459+0000","last_became_active":"2024-09-10T08:37:42.670097+0000","last_became_peered":"2024-09-10T08:37:42.670097+0000","last_unstale":"2024-09-10T08:37:42.670459+0000","last_undegraded":"2024-09-10T08:37:42.670459+0000","last_fullsized":"2024-09-10T08:37:42.670459+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:14.956492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.665105+0000","last_change":"2024-09-10T08:36:56.154850+0000","last_active":"2024-09-10T08:37:42.665105+0000","last_peered":"2024-09-10T08:37:42.665105+0000","last_clean":"2024-09-10T08:37:42.665105+0000","last_became_active":"2024-09-10T08:36:56.154545+0000","last_became_peered":"2024-09-10T08:36:56.154545+0000","last_unstale":"2024-09-10T08:37:42.665105+0000","last_undegraded":"2024-09-10T08:37:42.665105+0000","last_fullsized":"2024-09-10T08:37:42.665105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:24.639667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.669962+0000","last_change":"2024-09-10T08:37:42.669962+0000","last_active":"2024-09-10T08:37:42.669962+0000","last_peered":"2024-09-10T08:37:42.669962+0000","last_clean":"2024-09-10T08:37:42.669962+0000","last_became_active":"2024-09-10T08:37:42.669556+0000","last_became_peered":"2024-09-10T08:37:42.669556+0000","last_unstale":"2024-09-10T08:37:42.669962+0000","last_undegraded":"2024-09-10T08:37:42.669962+0000","last_fullsized":"2024-09-10T08:37:42.669962+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:23.333946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.289409+0000","last_change":"2024-09-10T08:36:56.145910+0000","last_active":"2024-09-10T08:37:43.289409+0000","last_peered":"2024-09-10T08:37:43.289409+0000","last_clean":"2024-09-10T08:37:43.289409+0000","last_became_active":"2024-09-10T08:36:56.145537+0000","last_became_peered":"2024-09-10T08:36:56.145537+0000","last_unstale":"2024-09-10T08:37:43.289409+0000","last_undegraded":"2024-09-10T08:37:43.289409+0000","last_fullsized":"2024-09-10T08:37:43.289409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:46.788241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.222267+0000","last_change":"2024-09-10T08:36:56.147340+0000","last_active":"2024-09-10T08:37:43.222267+0000","last_peered":"2024-09-10T08:37:43.222267+0000","last_clean":"2024-09-10T08:37:43.222267+0000","last_became_active":"2024-09-10T08:36:56.145762+0000","last_became_peered":"2024-09-10T08:36:56.145762+0000","last_unstale":"2024-09-10T08:37:43.222267+0000","last_undegraded":"2024-09-10T08:37:43.222267+0000","last_fullsized":"2024-09-10T08:37:43.222267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:12.339285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.844221+0000","last_change":"2024-09-10T08:36:56.145213+0000","last_active":"2024-09-10T08:37:42.844221+0000","last_peered":"2024-09-10T08:37:42.844221+0000","last_clean":"2024-09-10T08:37:42.844221+0000","last_became_active":"2024-09-10T08:36:56.144915+0000","last_became_peered":"2024-09-10T08:36:56.144915+0000","last_unstale":"2024-09-10T08:37:42.844221+0000","last_undegraded":"2024-09-10T08:37:42.844221+0000","last_fullsized":"2024-09-10T08:37:42.844221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:06.180321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.221804+0000","last_change":"2024-09-10T08:36:56.149738+0000","last_active":"2024-09-10T08:37:43.221804+0000","last_peered":"2024-09-10T08:37:43.221804+0000","last_clean":"2024-09-10T08:37:43.221804+0000","last_became_active":"2024-09-10T08:36:56.147750+0000","last_became_peered":"2024-09-10T08:36:56.147750+0000","last_unstale":"2024-09-10T08:37:43.221804+0000","last_undegraded":"2024-09-10T08:37:43.221804+0000","last_fullsized":"2024-09-10T08:37:43.221804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:47.950054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.289027+0000","last_change":"2024-09-10T08:36:56.147682+0000","last_active":"2024-09-10T08:37:43.289027+0000","last_peered":"2024-09-10T08:37:43.289027+0000","last_clean":"2024-09-10T08:37:43.289027+0000","last_became_active":"2024-09-10T08:36:56.147345+0000","last_became_peered":"2024-09-10T08:36:56.147345+0000","last_unstale":"2024-09-10T08:37:43.289027+0000","last_undegraded":"2024-09-10T08:37:43.289027+0000","last_fullsized":"2024-09-10T08:37:43.289027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:54.939482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664511+0000","last_change":"2024-09-10T08:36:56.158292+0000","last_active":"2024-09-10T08:37:42.664511+0000","last_peered":"2024-09-10T08:37:42.664511+0000","last_clean":"2024-09-10T08:37:42.664511+0000","last_became_active":"2024-09-10T08:36:56.157970+0000","last_became_peered":"2024-09-10T08:36:56.157970+0000","last_unstale":"2024-09-10T08:37:42.664511+0000","last_undegraded":"2024-09-10T08:37:42.664511+0000","last_fullsized":"2024-09-10T08:37:42.664511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:33.582126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664202+0000","last_change":"2024-09-10T08:36:56.142520+0000","last_active":"2024-09-10T08:37:42.664202+0000","last_peered":"2024-09-10T08:37:42.664202+0000","last_clean":"2024-09-10T08:37:42.664202+0000","last_became_active":"2024-09-10T08:36:56.142214+0000","last_became_peered":"2024-09-10T08:36:56.142214+0000","last_unstale":"2024-09-10T08:37:42.664202+0000","last_undegraded":"2024-09-10T08:37:42.664202+0000","last_fullsized":"2024-09-10T08:37:42.664202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:45:29.201839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.289281+0000","last_change":"2024-09-10T08:36:56.149547+0000","last_active":"2024-09-10T08:37:43.289281+0000","last_peered":"2024-09-10T08:37:43.289281+0000","last_clean":"2024-09-10T08:37:43.289281+0000","last_became_active":"2024-09-10T08:36:56.149141+0000","last_became_peered":"2024-09-10T08:36:56.149141+0000","last_unstale":"2024-09-10T08:37:43.289281+0000","last_undegraded":"2024-09-10T08:37:43.289281+0000","last_fullsized":"2024-09-10T08:37:43.289281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:49.368677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.670068+0000","last_change":"2024-09-10T08:36:56.143087+0000","last_active":"2024-09-10T08:37:42.670068+0000","last_peered":"2024-09-10T08:37:42.670068+0000","last_clean":"2024-09-10T08:37:42.670068+0000","last_became_active":"2024-09-10T08:36:56.142782+0000","last_became_peered":"2024-09-10T08:36:56.142782+0000","last_unstale":"2024-09-10T08:37:42.670068+0000","last_undegraded":"2024-09-10T08:37:42.670068+0000","last_fullsized":"2024-09-10T08:37:42.670068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:17.513103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664866+0000","last_change":"2024-09-10T08:36:56.157549+0000","last_active":"2024-09-10T08:37:42.664866+0000","last_peered":"2024-09-10T08:37:42.664866+0000","last_clean":"2024-09-10T08:37:42.664866+0000","last_became_active":"2024-09-10T08:36:56.157221+0000","last_became_peered":"2024-09-10T08:36:56.157221+0000","last_unstale":"2024-09-10T08:37:42.664866+0000","last_undegraded":"2024-09-10T08:37:42.664866+0000","last_fullsized":"2024-09-10T08:37:42.664866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:11.372085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.288782+0000","last_change":"2024-09-10T08:36:56.145654+0000","last_active":"2024-09-10T08:37:43.288782+0000","last_peered":"2024-09-10T08:37:43.288782+0000","last_clean":"2024-09-10T08:37:43.288782+0000","last_became_active":"2024-09-10T08:36:56.144126+0000","last_became_peered":"2024-09-10T08:36:56.144126+0000","last_unstale":"2024-09-10T08:37:43.288782+0000","last_undegraded":"2024-09-10T08:37:43.288782+0000","last_fullsized":"2024-09-10T08:37:43.288782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:55.498873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668470+0000","last_change":"2024-09-10T08:36:56.146423+0000","last_active":"2024-09-10T08:37:42.668470+0000","last_peered":"2024-09-10T08:37:42.668470+0000","last_clean":"2024-09-10T08:37:42.668470+0000","last_became_active":"2024-09-10T08:36:56.146126+0000","last_became_peered":"2024-09-10T08:36:56.146126+0000","last_unstale":"2024-09-10T08:37:42.668470+0000","last_undegraded":"2024-09-10T08:37:42.668470+0000","last_fullsized":"2024-09-10T08:37:42.668470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:44.665196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.666854+0000","last_change":"2024-09-10T08:36:56.151973+0000","last_active":"2024-09-10T08:37:42.666854+0000","last_peered":"2024-09-10T08:37:42.666854+0000","last_clean":"2024-09-10T08:37:42.666854+0000","last_became_active":"2024-09-10T08:36:56.151609+0000","last_became_peered":"2024-09-10T08:36:56.151609+0000","last_unstale":"2024-09-10T08:37:42.666854+0000","last_undegraded":"2024-09-10T08:37:42.666854+0000","last_fullsized":"2024-09-10T08:37:42.666854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:02.584047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664405+0000","last_change":"2024-09-10T08:36:56.141780+0000","last_active":"2024-09-10T08:37:42.664405+0000","last_peered":"2024-09-10T08:37:42.664405+0000","last_clean":"2024-09-10T08:37:42.664405+0000","last_became_active":"2024-09-10T08:36:56.141228+0000","last_became_peered":"2024-09-10T08:36:56.141228+0000","last_unstale":"2024-09-10T08:37:42.664405+0000","last_undegraded":"2024-09-10T08:37:42.664405+0000","last_fullsized":"2024-09-10T08:37:42.664405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:52:04.374228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.674303+0000","last_change":"2024-09-10T08:37:42.671346+0000","last_active":"2024-09-10T08:37:42.674303+0000","last_peered":"2024-09-10T08:37:42.674303+0000","last_clean":"2024-09-10T08:37:42.674303+0000","last_became_active":"2024-09-10T08:37:42.670895+0000","last_became_peered":"2024-09-10T08:37:42.670895+0000","last_unstale":"2024-09-10T08:37:42.674303+0000","last_undegraded":"2024-09-10T08:37:42.674303+0000","last_fullsized":"2024-09-10T08:37:42.674303+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:16:31.930628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.669181+0000","last_change":"2024-09-10T08:36:56.149014+0000","last_active":"2024-09-10T08:37:42.669181+0000","last_peered":"2024-09-10T08:37:42.669181+0000","last_clean":"2024-09-10T08:37:42.669181+0000","last_became_active":"2024-09-10T08:36:56.148551+0000","last_became_peered":"2024-09-10T08:36:56.148551+0000","last_unstale":"2024-09-10T08:37:42.669181+0000","last_undegraded":"2024-09-10T08:37:42.669181+0000","last_fullsized":"2024-09-10T08:37:42.669181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:35:52.250943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.666087+0000","last_change":"2024-09-10T08:36:56.141247+0000","last_active":"2024-09-10T08:37:42.666087+0000","last_peered":"2024-09-10T08:37:42.666087+0000","last_clean":"2024-09-10T08:37:42.666087+0000","last_became_active":"2024-09-10T08:36:56.140956+0000","last_became_peered":"2024-09-10T08:36:56.140956+0000","last_unstale":"2024-09-10T08:37:42.666087+0000","last_undegraded":"2024-09-10T08:37:42.666087+0000","last_fullsized":"2024-09-10T08:37:42.666087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:45.008424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.289423+0000","last_change":"2024-09-10T08:36:56.149116+0000","last_active":"2024-09-10T08:37:43.289423+0000","last_peered":"2024-09-10T08:37:43.289423+0000","last_clean":"2024-09-10T08:37:43.289423+0000","last_became_active":"2024-09-10T08:36:56.148783+0000","last_became_peered":"2024-09-10T08:36:56.148783+0000","last_unstale":"2024-09-10T08:37:43.289423+0000","last_undegraded":"2024-09-10T08:37:43.289423+0000","last_fullsized":"2024-09-10T08:37:43.289423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:37.119214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.846747+0000","last_change":"2024-09-10T08:37:42.846322+0000","last_active":"2024-09-10T08:37:42.846747+0000","last_peered":"2024-09-10T08:37:42.846747+0000","last_clean":"2024-09-10T08:37:42.846747+0000","last_became_active":"2024-09-10T08:37:42.846022+0000","last_became_peered":"2024-09-10T08:37:42.846022+0000","last_unstale":"2024-09-10T08:37:42.846747+0000","last_undegraded":"2024-09-10T08:37:42.846747+0000","last_fullsized":"2024-09-10T08:37:42.846747+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:38.142362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668299+0000","last_change":"2024-09-10T08:36:56.148537+0000","last_active":"2024-09-10T08:37:42.668299+0000","last_peered":"2024-09-10T08:37:42.668299+0000","last_clean":"2024-09-10T08:37:42.668299+0000","last_became_active":"2024-09-10T08:36:56.148185+0000","last_became_peered":"2024-09-10T08:36:56.148185+0000","last_unstale":"2024-09-10T08:37:42.668299+0000","last_undegraded":"2024-09-10T08:37:42.668299+0000","last_fullsized":"2024-09-10T08:37:42.668299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:00.426513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.666311+0000","last_change":"2024-09-10T08:36:56.138513+0000","last_active":"2024-09-10T08:37:42.666311+0000","last_peered":"2024-09-10T08:37:42.666311+0000","last_clean":"2024-09-10T08:37:42.666311+0000","last_became_active":"2024-09-10T08:36:56.138207+0000","last_became_peered":"2024-09-10T08:36:56.138207+0000","last_unstale":"2024-09-10T08:37:42.666311+0000","last_undegraded":"2024-09-10T08:37:42.666311+0000","last_fullsized":"2024-09-10T08:37:42.666311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:34.581430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668091+0000","last_change":"2024-09-10T08:36:56.146612+0000","last_active":"2024-09-10T08:37:42.668091+0000","last_peered":"2024-09-10T08:37:42.668091+0000","last_clean":"2024-09-10T08:37:42.668091+0000","last_became_active":"2024-09-10T08:36:56.146251+0000","last_became_peered":"2024-09-10T08:36:56.146251+0000","last_unstale":"2024-09-10T08:37:42.668091+0000","last_undegraded":"2024-09-10T08:37:42.668091+0000","last_fullsized":"2024-09-10T08:37:42.668091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:47.336389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.667090+0000","last_change":"2024-09-10T08:36:56.148260+0000","last_active":"2024-09-10T08:37:42.667090+0000","last_peered":"2024-09-10T08:37:42.667090+0000","last_clean":"2024-09-10T08:37:42.667090+0000","last_became_active":"2024-09-10T08:36:56.147949+0000","last_became_peered":"2024-09-10T08:36:56.147949+0000","last_unstale":"2024-09-10T08:37:42.667090+0000","last_undegraded":"2024-09-10T08:37:42.667090+0000","last_fullsized":"2024-09-10T08:37:42.667090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:52:19.762297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.667150+0000","last_change":"2024-09-10T08:36:56.156045+0000","last_active":"2024-09-10T08:37:42.667150+0000","last_peered":"2024-09-10T08:37:42.667150+0000","last_clean":"2024-09-10T08:37:42.667150+0000","last_became_active":"2024-09-10T08:36:56.154727+0000","last_became_peered":"2024-09-10T08:36:56.154727+0000","last_unstale":"2024-09-10T08:37:42.667150+0000","last_undegraded":"2024-09-10T08:37:42.667150+0000","last_fullsized":"2024-09-10T08:37:42.667150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:35:39.992683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.667375+0000","last_change":"2024-09-10T08:36:56.154394+0000","last_active":"2024-09-10T08:37:42.667375+0000","last_peered":"2024-09-10T08:37:42.667375+0000","last_clean":"2024-09-10T08:37:42.667375+0000","last_became_active":"2024-09-10T08:36:56.153855+0000","last_became_peered":"2024-09-10T08:36:56.153855+0000","last_unstale":"2024-09-10T08:37:42.667375+0000","last_undegraded":"2024-09-10T08:37:42.667375+0000","last_fullsized":"2024-09-10T08:37:42.667375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:14.987147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664653+0000","last_change":"2024-09-10T08:36:56.157517+0000","last_active":"2024-09-10T08:37:42.664653+0000","last_peered":"2024-09-10T08:37:42.664653+0000","last_clean":"2024-09-10T08:37:42.664653+0000","last_became_active":"2024-09-10T08:36:56.157133+0000","last_became_peered":"2024-09-10T08:36:56.157133+0000","last_unstale":"2024-09-10T08:37:42.664653+0000","last_undegraded":"2024-09-10T08:37:42.664653+0000","last_fullsized":"2024-09-10T08:37:42.664653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:17:29.573719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53101,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":110,"num_read_kb":114,"num_write":150,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":159303,"data_compressed":5667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":131,"ondisk_log_size":131,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282455,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":528370,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678048,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1167360,"data_stored":1173321,"data_compressed":61354,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841573,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":524510,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103211,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":528526,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332144,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":1124437,"data_compressed":59465,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495669,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":523782,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757307,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":1119632,"data_compressed":59465,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":528825,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149765,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":523976,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":568387,"data_compressed":25428,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1077248,"data_stored":572790,"data_compressed":25428,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":525170,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":49098,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":50448,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":51069,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:38:07.820 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:38:08.186 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:08.888 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:08 smithi089 ceph-mon[30757]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:38:08.888 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:08 smithi089 ceph-mon[30757]: pgmap v337: 129 pgs: 129 active+clean; 629 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-09-10T08:38:08.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.887+0000 7f4927600640 1 -- 172.21.15.89:0/234557769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4920101df0 msgr2=0x7f49201021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:08.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.887+0000 7f4927600640 1 --2- 172.21.15.89:0/234557769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4920101df0 0x7f49201021d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4910009900 tx=0x7f491002a400 comp rx=0 tx=0).stop 2024-09-10T08:38:08.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.887+0000 7f4927600640 1 -- 172.21.15.89:0/234557769 shutdown_connections 2024-09-10T08:38:08.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.887+0000 7f4927600640 1 --2- 172.21.15.89:0/234557769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4920106f20 0x7f492010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:08.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.887+0000 7f4927600640 1 --2- 172.21.15.89:0/234557769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49201027a0 0x7f49201067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:08.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.887+0000 7f4927600640 1 --2- 172.21.15.89:0/234557769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4920101df0 0x7f49201021d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:08.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.887+0000 7f4927600640 1 -- 172.21.15.89:0/234557769 >> 172.21.15.89:0/234557769 conn(0x7f49200fd8b0 msgr2=0x7f49200ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:08.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.887+0000 7f4927600640 1 -- 172.21.15.89:0/234557769 shutdown_connections 2024-09-10T08:38:08.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.888+0000 7f4927600640 1 -- 172.21.15.89:0/234557769 wait complete. 2024-09-10T08:38:08.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.888+0000 7f4927600640 1 Processor -- start 2024-09-10T08:38:08.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.889+0000 7f4927600640 1 -- start start 2024-09-10T08:38:08.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.889+0000 7f4927600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4920101df0 0x7f4920198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:08.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.889+0000 7f4927600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49201027a0 0x7f4920198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:08.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.889+0000 7f4927600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4920106f20 0x7f492019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:08.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.889+0000 7f4927600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4920111fd0 con 0x7f49201027a0 2024-09-10T08:38:08.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.889+0000 7f4927600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4920111e50 con 0x7f4920101df0 2024-09-10T08:38:08.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.889+0000 7f4927600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4920112150 con 0x7f4920106f20 2024-09-10T08:38:08.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.889+0000 7f4924c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4920101df0 0x7f4920198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:08.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.889+0000 7f4917e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49201027a0 0x7f4920198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:08.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.889+0000 7f4925600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4920106f20 0x7f492019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:08.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.889+0000 7f4924c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4920101df0 0x7f4920198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35600/0 (socket says 172.21.15.89:35600) 2024-09-10T08:38:08.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.889+0000 7f4924c00640 1 -- 172.21.15.89:0/2329936543 learned_addr learned my addr 172.21.15.89:0/2329936543 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:08.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.890+0000 7f4917e00640 1 -- 172.21.15.89:0/2329936543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4920106f20 msgr2=0x7f492019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:08.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.890+0000 7f4917e00640 1 --2- 172.21.15.89:0/2329936543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4920106f20 0x7f492019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:08.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.890+0000 7f4917e00640 1 -- 172.21.15.89:0/2329936543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4920101df0 msgr2=0x7f4920198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:08.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.890+0000 7f4917e00640 1 --2- 172.21.15.89:0/2329936543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4920101df0 0x7f4920198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:08.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.890+0000 7f4917e00640 1 -- 172.21.15.89:0/2329936543 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492019d3c0 con 0x7f49201027a0 2024-09-10T08:38:08.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.890+0000 7f4917e00640 1 --2- 172.21.15.89:0/2329936543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49201027a0 0x7f4920198950 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f490800d720 tx=0x7f490800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:08.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.890+0000 7f491e000640 1 -- 172.21.15.89:0/2329936543 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908017070 con 0x7f49201027a0 2024-09-10T08:38:08.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.891+0000 7f4927600640 1 -- 172.21.15.89:0/2329936543 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f492019d6b0 con 0x7f49201027a0 2024-09-10T08:38:08.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.891+0000 7f491e000640 1 -- 172.21.15.89:0/2329936543 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490800de30 con 0x7f49201027a0 2024-09-10T08:38:08.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.891+0000 7f491e000640 1 -- 172.21.15.89:0/2329936543 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49080047a0 con 0x7f49201027a0 2024-09-10T08:38:08.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.891+0000 7f4927600640 1 -- 172.21.15.89:0/2329936543 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49201a4f90 con 0x7f49201027a0 2024-09-10T08:38:08.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.892+0000 7f4927600640 1 -- 172.21.15.89:0/2329936543 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49201021d0 con 0x7f49201027a0 2024-09-10T08:38:08.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.893+0000 7f491e000640 1 -- 172.21.15.89:0/2329936543 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f49080099d0 con 0x7f49201027a0 2024-09-10T08:38:08.895 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.895+0000 7f491e000640 1 --2- 172.21.15.89:0/2329936543 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f49040a2c90 0x7f49040a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:08.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.895+0000 7f491e000640 1 -- 172.21.15.89:0/2329936543 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f49080cf0d0 con 0x7f49201027a0 2024-09-10T08:38:08.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.895+0000 7f4924c00640 1 --2- 172.21.15.89:0/2329936543 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f49040a2c90 0x7f49040a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:08.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.896+0000 7f4924c00640 1 --2- 172.21.15.89:0/2329936543 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f49040a2c90 0x7f49040a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f49100096c0 tx=0x7f49100046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:08.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:08.899+0000 7f491e000640 1 -- 172.21.15.89:0/2329936543 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4908020020 con 0x7f49201027a0 2024-09-10T08:38:09.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:08 smithi111 ceph-mon[35220]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:38:09.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:08 smithi111 ceph-mon[35220]: pgmap v337: 129 pgs: 129 active+clean; 629 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-09-10T08:38:09.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:09.070+0000 7f4927600640 1 -- 172.21.15.89:0/2329936543 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f492010a160 con 0x7f49040a2c90 2024-09-10T08:38:09.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:09.078+0000 7f491e000640 1 -- 172.21.15.89:0/2329936543 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340666 (secure 0 0 0) 0x7f492010a160 con 0x7f49040a2c90 2024-09-10T08:38:09.082 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:38:09.083 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-09-10T08:38:09.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:09.083+0000 7f4927600640 1 -- 172.21.15.89:0/2329936543 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f49040a2c90 msgr2=0x7f49040a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:09.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:09.083+0000 7f4927600640 1 --2- 172.21.15.89:0/2329936543 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f49040a2c90 0x7f49040a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f49100096c0 tx=0x7f49100046c0 comp rx=0 tx=0).stop 2024-09-10T08:38:09.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:09.083+0000 7f4927600640 1 -- 172.21.15.89:0/2329936543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49201027a0 msgr2=0x7f4920198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:09.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:09.083+0000 7f4927600640 1 --2- 172.21.15.89:0/2329936543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49201027a0 0x7f4920198950 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f490800d720 tx=0x7f490800dbe0 comp rx=0 tx=0).stop 2024-09-10T08:38:09.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:09.083+0000 7f4927600640 1 -- 172.21.15.89:0/2329936543 shutdown_connections 2024-09-10T08:38:09.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:09.083+0000 7f4927600640 1 --2- 172.21.15.89:0/2329936543 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f49040a2c90 0x7f49040a5150 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:09.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:09.083+0000 7f4927600640 1 --2- 172.21.15.89:0/2329936543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4920106f20 0x7f492019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:09.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:09.083+0000 7f4927600640 1 --2- 172.21.15.89:0/2329936543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49201027a0 0x7f4920198950 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:09.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:09.083+0000 7f4927600640 1 --2- 172.21.15.89:0/2329936543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4920101df0 0x7f4920198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:09.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:09.083+0000 7f4927600640 1 -- 172.21.15.89:0/2329936543 >> 172.21.15.89:0/2329936543 conn(0x7f49200fd8b0 msgr2=0x7f49200feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:09.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:09.083+0000 7f4927600640 1 -- 172.21.15.89:0/2329936543 shutdown_connections 2024-09-10T08:38:09.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:09.083+0000 7f4927600640 1 -- 172.21.15.89:0/2329936543 wait complete. 2024-09-10T08:38:09.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:08 smithi149 ceph-mon[36167]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:38:09.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:08 smithi149 ceph-mon[36167]: pgmap v337: 129 pgs: 129 active+clean; 629 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-09-10T08:38:09.814 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-09-10T08:38:07.584974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":643703,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":251,"num_read_kb":228,"num_write":339,"num_write_kb":3038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357636,"kb_used_data":12548,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584188,"statfs":{"total":1151940427776,"available":1151574208512,"internally_reserved":0,"allocated":12849152,"data_stored":8257627,"data_compressed":395913,"data_compressed_allocated":540672,"data_compressed_original":2580480,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":39279,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":75,"num_write":25,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007822"},"pg_stats":[{"pgid":"2.1f","version":"94'9","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:47.715421+0000","last_change":"2024-09-10T08:36:53.720261+0000","last_active":"2024-09-10T08:37:47.715421+0000","last_peered":"2024-09-10T08:37:47.715421+0000","last_clean":"2024-09-10T08:37:47.715421+0000","last_became_active":"2024-09-10T08:36:53.720037+0000","last_became_peered":"2024-09-10T08:36:53.720037+0000","last_unstale":"2024-09-10T08:37:47.715421+0000","last_undegraded":"2024-09-10T08:37:47.715421+0000","last_fullsized":"2024-09-10T08:37:47.715421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:56.548573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.221833+0000","last_change":"2024-09-10T08:36:56.145816+0000","last_active":"2024-09-10T08:37:43.221833+0000","last_peered":"2024-09-10T08:37:43.221833+0000","last_clean":"2024-09-10T08:37:43.221833+0000","last_became_active":"2024-09-10T08:36:56.145462+0000","last_became_peered":"2024-09-10T08:36:56.145462+0000","last_unstale":"2024-09-10T08:37:43.221833+0000","last_undegraded":"2024-09-10T08:37:43.221833+0000","last_fullsized":"2024-09-10T08:37:43.221833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:53.890190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.288722+0000","last_change":"2024-09-10T08:37:05.252761+0000","last_active":"2024-09-10T08:37:43.288722+0000","last_peered":"2024-09-10T08:37:43.288722+0000","last_clean":"2024-09-10T08:37:43.288722+0000","last_became_active":"2024-09-10T08:37:05.252179+0000","last_became_peered":"2024-09-10T08:37:05.252179+0000","last_unstale":"2024-09-10T08:37:43.288722+0000","last_undegraded":"2024-09-10T08:37:43.288722+0000","last_fullsized":"2024-09-10T08:37:43.288722+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:52.854828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.222230+0000","last_change":"2024-09-10T08:36:53.735650+0000","last_active":"2024-09-10T08:37:43.222230+0000","last_peered":"2024-09-10T08:37:43.222230+0000","last_clean":"2024-09-10T08:37:43.222230+0000","last_became_active":"2024-09-10T08:36:53.735128+0000","last_became_peered":"2024-09-10T08:36:53.735128+0000","last_unstale":"2024-09-10T08:37:43.222230+0000","last_undegraded":"2024-09-10T08:37:43.222230+0000","last_fullsized":"2024-09-10T08:37:43.222230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:27.309591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.666844+0000","last_change":"2024-09-10T08:36:56.153364+0000","last_active":"2024-09-10T08:37:42.666844+0000","last_peered":"2024-09-10T08:37:42.666844+0000","last_clean":"2024-09-10T08:37:42.666844+0000","last_became_active":"2024-09-10T08:36:56.152979+0000","last_became_peered":"2024-09-10T08:36:56.152979+0000","last_unstale":"2024-09-10T08:37:42.666844+0000","last_undegraded":"2024-09-10T08:37:42.666844+0000","last_fullsized":"2024-09-10T08:37:42.666844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:12.172620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.665745+0000","last_change":"2024-09-10T08:37:05.254551+0000","last_active":"2024-09-10T08:37:42.665745+0000","last_peered":"2024-09-10T08:37:42.665745+0000","last_clean":"2024-09-10T08:37:42.665745+0000","last_became_active":"2024-09-10T08:37:05.254055+0000","last_became_peered":"2024-09-10T08:37:05.254055+0000","last_unstale":"2024-09-10T08:37:42.665745+0000","last_undegraded":"2024-09-10T08:37:42.665745+0000","last_fullsized":"2024-09-10T08:37:42.665745+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:08.472502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:47.712944+0000","last_change":"2024-09-10T08:36:53.722774+0000","last_active":"2024-09-10T08:37:47.712944+0000","last_peered":"2024-09-10T08:37:47.712944+0000","last_clean":"2024-09-10T08:37:47.712944+0000","last_became_active":"2024-09-10T08:36:53.722459+0000","last_became_peered":"2024-09-10T08:36:53.722459+0000","last_unstale":"2024-09-10T08:37:47.712944+0000","last_undegraded":"2024-09-10T08:37:47.712944+0000","last_fullsized":"2024-09-10T08:37:47.712944+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:26.612354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.289106+0000","last_change":"2024-09-10T08:36:56.143757+0000","last_active":"2024-09-10T08:37:43.289106+0000","last_peered":"2024-09-10T08:37:43.289106+0000","last_clean":"2024-09-10T08:37:43.289106+0000","last_became_active":"2024-09-10T08:36:56.143430+0000","last_became_peered":"2024-09-10T08:36:56.143430+0000","last_unstale":"2024-09-10T08:37:43.289106+0000","last_undegraded":"2024-09-10T08:37:43.289106+0000","last_fullsized":"2024-09-10T08:37:43.289106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:49.737298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.666659+0000","last_change":"2024-09-10T08:37:05.254503+0000","last_active":"2024-09-10T08:37:42.666659+0000","last_peered":"2024-09-10T08:37:42.666659+0000","last_clean":"2024-09-10T08:37:42.666659+0000","last_became_active":"2024-09-10T08:37:05.254070+0000","last_became_peered":"2024-09-10T08:37:05.254070+0000","last_unstale":"2024-09-10T08:37:42.666659+0000","last_undegraded":"2024-09-10T08:37:42.666659+0000","last_fullsized":"2024-09-10T08:37:42.666659+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:16.741698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668715+0000","last_change":"2024-09-10T08:36:53.716742+0000","last_active":"2024-09-10T08:37:42.668715+0000","last_peered":"2024-09-10T08:37:42.668715+0000","last_clean":"2024-09-10T08:37:42.668715+0000","last_became_active":"2024-09-10T08:36:53.716439+0000","last_became_peered":"2024-09-10T08:36:53.716439+0000","last_unstale":"2024-09-10T08:37:42.668715+0000","last_undegraded":"2024-09-10T08:37:42.668715+0000","last_fullsized":"2024-09-10T08:37:42.668715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:46.882805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.844590+0000","last_change":"2024-09-10T08:36:56.146841+0000","last_active":"2024-09-10T08:37:42.844590+0000","last_peered":"2024-09-10T08:37:42.844590+0000","last_clean":"2024-09-10T08:37:42.844590+0000","last_became_active":"2024-09-10T08:36:56.146531+0000","last_became_peered":"2024-09-10T08:36:56.146531+0000","last_unstale":"2024-09-10T08:37:42.844590+0000","last_undegraded":"2024-09-10T08:37:42.844590+0000","last_fullsized":"2024-09-10T08:37:42.844590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:37.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.223181+0000","last_change":"2024-09-10T08:37:43.223181+0000","last_active":"2024-09-10T08:37:43.223181+0000","last_peered":"2024-09-10T08:37:43.223181+0000","last_clean":"2024-09-10T08:37:43.223181+0000","last_became_active":"2024-09-10T08:37:43.222794+0000","last_became_peered":"2024-09-10T08:37:43.222794+0000","last_unstale":"2024-09-10T08:37:43.223181+0000","last_undegraded":"2024-09-10T08:37:43.223181+0000","last_fullsized":"2024-09-10T08:37:43.223181+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:22.204517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668683+0000","last_change":"2024-09-10T08:36:56.142161+0000","last_active":"2024-09-10T08:37:42.668683+0000","last_peered":"2024-09-10T08:37:42.668683+0000","last_clean":"2024-09-10T08:37:42.668683+0000","last_became_active":"2024-09-10T08:36:56.141838+0000","last_became_peered":"2024-09-10T08:36:56.141838+0000","last_unstale":"2024-09-10T08:37:42.668683+0000","last_undegraded":"2024-09-10T08:37:42.668683+0000","last_fullsized":"2024-09-10T08:37:42.668683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:09.909747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:46.701515+0000","last_change":"2024-09-10T08:36:53.722238+0000","last_active":"2024-09-10T08:37:46.701515+0000","last_peered":"2024-09-10T08:37:46.701515+0000","last_clean":"2024-09-10T08:37:46.701515+0000","last_became_active":"2024-09-10T08:36:53.721859+0000","last_became_peered":"2024-09-10T08:36:53.721859+0000","last_unstale":"2024-09-10T08:37:46.701515+0000","last_undegraded":"2024-09-10T08:37:46.701515+0000","last_fullsized":"2024-09-10T08:37:46.701515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:50.615090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.666365+0000","last_change":"2024-09-10T08:37:05.414041+0000","last_active":"2024-09-10T08:37:42.666365+0000","last_peered":"2024-09-10T08:37:42.666365+0000","last_clean":"2024-09-10T08:37:42.666365+0000","last_became_active":"2024-09-10T08:37:05.413286+0000","last_became_peered":"2024-09-10T08:37:05.413286+0000","last_unstale":"2024-09-10T08:37:42.666365+0000","last_undegraded":"2024-09-10T08:37:42.666365+0000","last_fullsized":"2024-09-10T08:37:42.666365+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:44.916102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.221703+0000","last_change":"2024-09-10T08:36:56.158278+0000","last_active":"2024-09-10T08:37:43.221703+0000","last_peered":"2024-09-10T08:37:43.221703+0000","last_clean":"2024-09-10T08:37:43.221703+0000","last_became_active":"2024-09-10T08:36:56.157946+0000","last_became_peered":"2024-09-10T08:36:56.157946+0000","last_unstale":"2024-09-10T08:37:43.221703+0000","last_undegraded":"2024-09-10T08:37:43.221703+0000","last_fullsized":"2024-09-10T08:37:43.221703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:42.512513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'1","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.288930+0000","last_change":"2024-09-10T08:36:53.718679+0000","last_active":"2024-09-10T08:37:43.288930+0000","last_peered":"2024-09-10T08:37:43.288930+0000","last_clean":"2024-09-10T08:37:43.288930+0000","last_became_active":"2024-09-10T08:36:53.718377+0000","last_became_peered":"2024-09-10T08:36:53.718377+0000","last_unstale":"2024-09-10T08:37:43.288930+0000","last_undegraded":"2024-09-10T08:37:43.288930+0000","last_fullsized":"2024-09-10T08:37:43.288930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:58.114825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.672349+0000","last_change":"2024-09-10T08:37:42.671399+0000","last_active":"2024-09-10T08:37:42.672349+0000","last_peered":"2024-09-10T08:37:42.672349+0000","last_clean":"2024-09-10T08:37:42.672349+0000","last_became_active":"2024-09-10T08:37:42.670865+0000","last_became_peered":"2024-09-10T08:37:42.670865+0000","last_unstale":"2024-09-10T08:37:42.672349+0000","last_undegraded":"2024-09-10T08:37:42.672349+0000","last_fullsized":"2024-09-10T08:37:42.672349+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:16.231538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664111+0000","last_change":"2024-09-10T08:36:56.144856+0000","last_active":"2024-09-10T08:37:42.664111+0000","last_peered":"2024-09-10T08:37:42.664111+0000","last_clean":"2024-09-10T08:37:42.664111+0000","last_became_active":"2024-09-10T08:36:56.144531+0000","last_became_peered":"2024-09-10T08:36:56.144531+0000","last_unstale":"2024-09-10T08:37:42.664111+0000","last_undegraded":"2024-09-10T08:37:42.664111+0000","last_fullsized":"2024-09-10T08:37:42.664111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:24.586006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:46.702988+0000","last_change":"2024-09-10T08:36:53.721093+0000","last_active":"2024-09-10T08:37:46.702988+0000","last_peered":"2024-09-10T08:37:46.702988+0000","last_clean":"2024-09-10T08:37:46.702988+0000","last_became_active":"2024-09-10T08:36:53.720538+0000","last_became_peered":"2024-09-10T08:36:53.720538+0000","last_unstale":"2024-09-10T08:37:46.702988+0000","last_undegraded":"2024-09-10T08:37:46.702988+0000","last_fullsized":"2024-09-10T08:37:46.702988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:58.492463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.669785+0000","last_change":"2024-09-10T08:37:05.254011+0000","last_active":"2024-09-10T08:37:42.669785+0000","last_peered":"2024-09-10T08:37:42.669785+0000","last_clean":"2024-09-10T08:37:42.669785+0000","last_became_active":"2024-09-10T08:37:05.253066+0000","last_became_peered":"2024-09-10T08:37:05.253066+0000","last_unstale":"2024-09-10T08:37:42.669785+0000","last_undegraded":"2024-09-10T08:37:42.669785+0000","last_fullsized":"2024-09-10T08:37:42.669785+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:16.221302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668011+0000","last_change":"2024-09-10T08:36:56.141814+0000","last_active":"2024-09-10T08:37:42.668011+0000","last_peered":"2024-09-10T08:37:42.668011+0000","last_clean":"2024-09-10T08:37:42.668011+0000","last_became_active":"2024-09-10T08:36:56.141480+0000","last_became_peered":"2024-09-10T08:36:56.141480+0000","last_unstale":"2024-09-10T08:37:42.668011+0000","last_undegraded":"2024-09-10T08:37:42.668011+0000","last_fullsized":"2024-09-10T08:37:42.668011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:40.594354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:46.702128+0000","last_change":"2024-09-10T08:36:53.734338+0000","last_active":"2024-09-10T08:37:46.702128+0000","last_peered":"2024-09-10T08:37:46.702128+0000","last_clean":"2024-09-10T08:37:46.702128+0000","last_became_active":"2024-09-10T08:36:53.734031+0000","last_became_peered":"2024-09-10T08:36:53.734031+0000","last_unstale":"2024-09-10T08:37:46.702128+0000","last_undegraded":"2024-09-10T08:37:46.702128+0000","last_fullsized":"2024-09-10T08:37:46.702128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:48.922003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.666147+0000","last_change":"2024-09-10T08:37:05.414181+0000","last_active":"2024-09-10T08:37:42.666147+0000","last_peered":"2024-09-10T08:37:42.666147+0000","last_clean":"2024-09-10T08:37:42.666147+0000","last_became_active":"2024-09-10T08:37:05.413691+0000","last_became_peered":"2024-09-10T08:37:05.413691+0000","last_unstale":"2024-09-10T08:37:42.666147+0000","last_undegraded":"2024-09-10T08:37:42.666147+0000","last_fullsized":"2024-09-10T08:37:42.666147+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:30.744949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.667482+0000","last_change":"2024-09-10T08:36:56.153956+0000","last_active":"2024-09-10T08:37:42.667482+0000","last_peered":"2024-09-10T08:37:42.667482+0000","last_clean":"2024-09-10T08:37:42.667482+0000","last_became_active":"2024-09-10T08:36:56.153609+0000","last_became_peered":"2024-09-10T08:36:56.153609+0000","last_unstale":"2024-09-10T08:37:42.667482+0000","last_undegraded":"2024-09-10T08:37:42.667482+0000","last_fullsized":"2024-09-10T08:37:42.667482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:18.185704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:46.701784+0000","last_change":"2024-09-10T08:36:53.733217+0000","last_active":"2024-09-10T08:37:46.701784+0000","last_peered":"2024-09-10T08:37:46.701784+0000","last_clean":"2024-09-10T08:37:46.701784+0000","last_became_active":"2024-09-10T08:36:53.732851+0000","last_became_peered":"2024-09-10T08:36:53.732851+0000","last_unstale":"2024-09-10T08:37:46.701784+0000","last_undegraded":"2024-09-10T08:37:46.701784+0000","last_fullsized":"2024-09-10T08:37:46.701784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:04:01.132196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.288734+0000","last_change":"2024-09-10T08:37:05.241868+0000","last_active":"2024-09-10T08:37:43.288734+0000","last_peered":"2024-09-10T08:37:43.288734+0000","last_clean":"2024-09-10T08:37:43.288734+0000","last_became_active":"2024-09-10T08:37:05.241412+0000","last_became_peered":"2024-09-10T08:37:05.241412+0000","last_unstale":"2024-09-10T08:37:43.288734+0000","last_undegraded":"2024-09-10T08:37:43.288734+0000","last_fullsized":"2024-09-10T08:37:43.288734+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:52.543353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.222307+0000","last_change":"2024-09-10T08:36:56.150722+0000","last_active":"2024-09-10T08:37:43.222307+0000","last_peered":"2024-09-10T08:37:43.222307+0000","last_clean":"2024-09-10T08:37:43.222307+0000","last_became_active":"2024-09-10T08:36:56.150423+0000","last_became_peered":"2024-09-10T08:36:56.150423+0000","last_unstale":"2024-09-10T08:37:43.222307+0000","last_undegraded":"2024-09-10T08:37:43.222307+0000","last_fullsized":"2024-09-10T08:37:43.222307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:50.272841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'9","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:38:03.161649+0000","last_change":"2024-09-10T08:36:53.722477+0000","last_active":"2024-09-10T08:38:03.161649+0000","last_peered":"2024-09-10T08:38:03.161649+0000","last_clean":"2024-09-10T08:38:03.161649+0000","last_became_active":"2024-09-10T08:36:53.722118+0000","last_became_peered":"2024-09-10T08:36:53.722118+0000","last_unstale":"2024-09-10T08:38:03.161649+0000","last_undegraded":"2024-09-10T08:38:03.161649+0000","last_fullsized":"2024-09-10T08:38:03.161649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:06.631646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.221528+0000","last_change":"2024-09-10T08:37:05.415656+0000","last_active":"2024-09-10T08:37:43.221528+0000","last_peered":"2024-09-10T08:37:43.221528+0000","last_clean":"2024-09-10T08:37:43.221528+0000","last_became_active":"2024-09-10T08:37:05.413537+0000","last_became_peered":"2024-09-10T08:37:05.413537+0000","last_unstale":"2024-09-10T08:37:43.221528+0000","last_undegraded":"2024-09-10T08:37:43.221528+0000","last_fullsized":"2024-09-10T08:37:43.221528+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:04.284410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.665194+0000","last_change":"2024-09-10T08:36:56.144157+0000","last_active":"2024-09-10T08:37:42.665194+0000","last_peered":"2024-09-10T08:37:42.665194+0000","last_clean":"2024-09-10T08:37:42.665194+0000","last_became_active":"2024-09-10T08:36:56.140332+0000","last_became_peered":"2024-09-10T08:36:56.140332+0000","last_unstale":"2024-09-10T08:37:42.665194+0000","last_undegraded":"2024-09-10T08:37:42.665194+0000","last_fullsized":"2024-09-10T08:37:42.665194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:09:26.097987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:46.703188+0000","last_change":"2024-09-10T08:36:53.735465+0000","last_active":"2024-09-10T08:37:46.703188+0000","last_peered":"2024-09-10T08:37:46.703188+0000","last_clean":"2024-09-10T08:37:46.703188+0000","last_became_active":"2024-09-10T08:36:53.735160+0000","last_became_peered":"2024-09-10T08:36:53.735160+0000","last_unstale":"2024-09-10T08:37:46.703188+0000","last_undegraded":"2024-09-10T08:37:46.703188+0000","last_fullsized":"2024-09-10T08:37:46.703188+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:26:52.332085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664143+0000","last_change":"2024-09-10T08:37:05.247671+0000","last_active":"2024-09-10T08:37:42.664143+0000","last_peered":"2024-09-10T08:37:42.664143+0000","last_clean":"2024-09-10T08:37:42.664143+0000","last_became_active":"2024-09-10T08:37:05.247217+0000","last_became_peered":"2024-09-10T08:37:05.247217+0000","last_unstale":"2024-09-10T08:37:42.664143+0000","last_undegraded":"2024-09-10T08:37:42.664143+0000","last_fullsized":"2024-09-10T08:37:42.664143+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:20.881304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.665718+0000","last_change":"2024-09-10T08:36:56.145420+0000","last_active":"2024-09-10T08:37:42.665718+0000","last_peered":"2024-09-10T08:37:42.665718+0000","last_clean":"2024-09-10T08:37:42.665718+0000","last_became_active":"2024-09-10T08:36:56.145074+0000","last_became_peered":"2024-09-10T08:36:56.145074+0000","last_unstale":"2024-09-10T08:37:42.665718+0000","last_undegraded":"2024-09-10T08:37:42.665718+0000","last_fullsized":"2024-09-10T08:37:42.665718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:13.849977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.670084+0000","last_change":"2024-09-10T08:36:53.738237+0000","last_active":"2024-09-10T08:37:42.670084+0000","last_peered":"2024-09-10T08:37:42.670084+0000","last_clean":"2024-09-10T08:37:42.670084+0000","last_became_active":"2024-09-10T08:36:53.737858+0000","last_became_peered":"2024-09-10T08:36:53.737858+0000","last_unstale":"2024-09-10T08:37:42.670084+0000","last_undegraded":"2024-09-10T08:37:42.670084+0000","last_fullsized":"2024-09-10T08:37:42.670084+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:35.345661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668150+0000","last_change":"2024-09-10T08:37:05.413512+0000","last_active":"2024-09-10T08:37:42.668150+0000","last_peered":"2024-09-10T08:37:42.668150+0000","last_clean":"2024-09-10T08:37:42.668150+0000","last_became_active":"2024-09-10T08:37:05.413010+0000","last_became_peered":"2024-09-10T08:37:05.413010+0000","last_unstale":"2024-09-10T08:37:42.668150+0000","last_undegraded":"2024-09-10T08:37:42.668150+0000","last_fullsized":"2024-09-10T08:37:42.668150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:05.349404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.665995+0000","last_change":"2024-09-10T08:36:56.157741+0000","last_active":"2024-09-10T08:37:42.665995+0000","last_peered":"2024-09-10T08:37:42.665995+0000","last_clean":"2024-09-10T08:37:42.665995+0000","last_became_active":"2024-09-10T08:36:56.157295+0000","last_became_peered":"2024-09-10T08:36:56.157295+0000","last_unstale":"2024-09-10T08:37:42.665995+0000","last_undegraded":"2024-09-10T08:37:42.665995+0000","last_fullsized":"2024-09-10T08:37:42.665995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:31.845864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668956+0000","last_change":"2024-09-10T08:36:53.722373+0000","last_active":"2024-09-10T08:37:42.668956+0000","last_peered":"2024-09-10T08:37:42.668956+0000","last_clean":"2024-09-10T08:37:42.668956+0000","last_became_active":"2024-09-10T08:36:53.721975+0000","last_became_peered":"2024-09-10T08:36:53.721975+0000","last_unstale":"2024-09-10T08:37:42.668956+0000","last_undegraded":"2024-09-10T08:37:42.668956+0000","last_fullsized":"2024-09-10T08:37:42.668956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:44.593018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.665010+0000","last_change":"2024-09-10T08:37:05.255362+0000","last_active":"2024-09-10T08:37:42.665010+0000","last_peered":"2024-09-10T08:37:42.665010+0000","last_clean":"2024-09-10T08:37:42.665010+0000","last_became_active":"2024-09-10T08:37:05.254967+0000","last_became_peered":"2024-09-10T08:37:05.254967+0000","last_unstale":"2024-09-10T08:37:42.665010+0000","last_undegraded":"2024-09-10T08:37:42.665010+0000","last_fullsized":"2024-09-10T08:37:42.665010+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:31:41.850961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.844332+0000","last_change":"2024-09-10T08:36:56.147699+0000","last_active":"2024-09-10T08:37:42.844332+0000","last_peered":"2024-09-10T08:37:42.844332+0000","last_clean":"2024-09-10T08:37:42.844332+0000","last_became_active":"2024-09-10T08:36:56.147349+0000","last_became_peered":"2024-09-10T08:36:56.147349+0000","last_unstale":"2024-09-10T08:37:42.844332+0000","last_undegraded":"2024-09-10T08:37:42.844332+0000","last_fullsized":"2024-09-10T08:37:42.844332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:25.248588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'4","reported_seq":57,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:38:03.743306+0000","last_change":"2024-09-10T08:36:53.736843+0000","last_active":"2024-09-10T08:38:03.743306+0000","last_peered":"2024-09-10T08:38:03.743306+0000","last_clean":"2024-09-10T08:38:03.743306+0000","last_became_active":"2024-09-10T08:36:53.736340+0000","last_became_peered":"2024-09-10T08:36:53.736340+0000","last_unstale":"2024-09-10T08:38:03.743306+0000","last_undegraded":"2024-09-10T08:38:03.743306+0000","last_fullsized":"2024-09-10T08:38:03.743306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:24.679679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664817+0000","last_change":"2024-09-10T08:37:05.414321+0000","last_active":"2024-09-10T08:37:42.664817+0000","last_peered":"2024-09-10T08:37:42.664817+0000","last_clean":"2024-09-10T08:37:42.664817+0000","last_became_active":"2024-09-10T08:37:05.413868+0000","last_became_peered":"2024-09-10T08:37:05.413868+0000","last_unstale":"2024-09-10T08:37:42.664817+0000","last_undegraded":"2024-09-10T08:37:42.664817+0000","last_fullsized":"2024-09-10T08:37:42.664817+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:53.988077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664070+0000","last_change":"2024-09-10T08:36:56.141910+0000","last_active":"2024-09-10T08:37:42.664070+0000","last_peered":"2024-09-10T08:37:42.664070+0000","last_clean":"2024-09-10T08:37:42.664070+0000","last_became_active":"2024-09-10T08:36:56.141579+0000","last_became_peered":"2024-09-10T08:36:56.141579+0000","last_unstale":"2024-09-10T08:37:42.664070+0000","last_undegraded":"2024-09-10T08:37:42.664070+0000","last_fullsized":"2024-09-10T08:37:42.664070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:18.464831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:47.712490+0000","last_change":"2024-09-10T08:36:53.720102+0000","last_active":"2024-09-10T08:37:47.712490+0000","last_peered":"2024-09-10T08:37:47.712490+0000","last_clean":"2024-09-10T08:37:47.712490+0000","last_became_active":"2024-09-10T08:36:53.719764+0000","last_became_peered":"2024-09-10T08:36:53.719764+0000","last_unstale":"2024-09-10T08:37:47.712490+0000","last_undegraded":"2024-09-10T08:37:47.712490+0000","last_fullsized":"2024-09-10T08:37:47.712490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:03.609338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.844144+0000","last_change":"2024-09-10T08:37:05.413818+0000","last_active":"2024-09-10T08:37:42.844144+0000","last_peered":"2024-09-10T08:37:42.844144+0000","last_clean":"2024-09-10T08:37:42.844144+0000","last_became_active":"2024-09-10T08:37:05.413267+0000","last_became_peered":"2024-09-10T08:37:05.413267+0000","last_unstale":"2024-09-10T08:37:42.844144+0000","last_undegraded":"2024-09-10T08:37:42.844144+0000","last_fullsized":"2024-09-10T08:37:42.844144+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:54.309686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.222009+0000","last_change":"2024-09-10T08:36:56.149286+0000","last_active":"2024-09-10T08:37:43.222009+0000","last_peered":"2024-09-10T08:37:43.222009+0000","last_clean":"2024-09-10T08:37:43.222009+0000","last_became_active":"2024-09-10T08:36:56.146275+0000","last_became_peered":"2024-09-10T08:36:56.146275+0000","last_unstale":"2024-09-10T08:37:43.222009+0000","last_undegraded":"2024-09-10T08:37:43.222009+0000","last_fullsized":"2024-09-10T08:37:43.222009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:19.468224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:48.719938+0000","last_change":"2024-09-10T08:36:53.719756+0000","last_active":"2024-09-10T08:37:48.719938+0000","last_peered":"2024-09-10T08:37:48.719938+0000","last_clean":"2024-09-10T08:37:48.719938+0000","last_became_active":"2024-09-10T08:36:53.719302+0000","last_became_peered":"2024-09-10T08:36:53.719302+0000","last_unstale":"2024-09-10T08:37:48.719938+0000","last_undegraded":"2024-09-10T08:37:48.719938+0000","last_fullsized":"2024-09-10T08:37:48.719938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:47.658081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":352,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668541+0000","last_change":"2024-09-10T08:35:39.150715+0000","last_active":"2024-09-10T08:37:42.668541+0000","last_peered":"2024-09-10T08:37:42.668541+0000","last_clean":"2024-09-10T08:37:42.668541+0000","last_became_active":"2024-09-10T08:35:39.150206+0000","last_became_peered":"2024-09-10T08:35:39.150206+0000","last_unstale":"2024-09-10T08:37:42.668541+0000","last_undegraded":"2024-09-10T08:37:42.668541+0000","last_fullsized":"2024-09-10T08:37:42.668541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:32:02.110870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:32:02.110870+0000","last_clean_scrub_stamp":"2024-09-10T08:32:02.110870+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:20.762043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.847198+0000","last_change":"2024-09-10T08:37:42.846547+0000","last_active":"2024-09-10T08:37:42.847198+0000","last_peered":"2024-09-10T08:37:42.847198+0000","last_clean":"2024-09-10T08:37:42.847198+0000","last_became_active":"2024-09-10T08:37:42.845884+0000","last_became_peered":"2024-09-10T08:37:42.845884+0000","last_unstale":"2024-09-10T08:37:42.847198+0000","last_undegraded":"2024-09-10T08:37:42.847198+0000","last_fullsized":"2024-09-10T08:37:42.847198+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:21.905133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.669368+0000","last_change":"2024-09-10T08:36:56.144187+0000","last_active":"2024-09-10T08:37:42.669368+0000","last_peered":"2024-09-10T08:37:42.669368+0000","last_clean":"2024-09-10T08:37:42.669368+0000","last_became_active":"2024-09-10T08:36:56.143193+0000","last_became_peered":"2024-09-10T08:36:56.143193+0000","last_unstale":"2024-09-10T08:37:42.669368+0000","last_undegraded":"2024-09-10T08:37:42.669368+0000","last_fullsized":"2024-09-10T08:37:42.669368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:14:26.981884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:47.713979+0000","last_change":"2024-09-10T08:36:53.737076+0000","last_active":"2024-09-10T08:37:47.713979+0000","last_peered":"2024-09-10T08:37:47.713979+0000","last_clean":"2024-09-10T08:37:47.713979+0000","last_became_active":"2024-09-10T08:36:53.736756+0000","last_became_peered":"2024-09-10T08:36:53.736756+0000","last_unstale":"2024-09-10T08:37:47.713979+0000","last_undegraded":"2024-09-10T08:37:47.713979+0000","last_fullsized":"2024-09-10T08:37:47.713979+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:04:06.526018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.288890+0000","last_change":"2024-09-10T08:37:05.242095+0000","last_active":"2024-09-10T08:37:43.288890+0000","last_peered":"2024-09-10T08:37:43.288890+0000","last_clean":"2024-09-10T08:37:43.288890+0000","last_became_active":"2024-09-10T08:37:05.241645+0000","last_became_peered":"2024-09-10T08:37:05.241645+0000","last_unstale":"2024-09-10T08:37:43.288890+0000","last_undegraded":"2024-09-10T08:37:43.288890+0000","last_fullsized":"2024-09-10T08:37:43.288890+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:30:49.637861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.669809+0000","last_change":"2024-09-10T08:36:56.149396+0000","last_active":"2024-09-10T08:37:42.669809+0000","last_peered":"2024-09-10T08:37:42.669809+0000","last_clean":"2024-09-10T08:37:42.669809+0000","last_became_active":"2024-09-10T08:36:56.149079+0000","last_became_peered":"2024-09-10T08:36:56.149079+0000","last_unstale":"2024-09-10T08:37:42.669809+0000","last_undegraded":"2024-09-10T08:37:42.669809+0000","last_fullsized":"2024-09-10T08:37:42.669809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:44.665448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.844798+0000","last_change":"2024-09-10T08:36:53.717950+0000","last_active":"2024-09-10T08:37:42.844798+0000","last_peered":"2024-09-10T08:37:42.844798+0000","last_clean":"2024-09-10T08:37:42.844798+0000","last_became_active":"2024-09-10T08:36:53.717668+0000","last_became_peered":"2024-09-10T08:36:53.717668+0000","last_unstale":"2024-09-10T08:37:42.844798+0000","last_undegraded":"2024-09-10T08:37:42.844798+0000","last_fullsized":"2024-09-10T08:37:42.844798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:58.692631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.666895+0000","last_change":"2024-09-10T08:37:05.260178+0000","last_active":"2024-09-10T08:37:42.666895+0000","last_peered":"2024-09-10T08:37:42.666895+0000","last_clean":"2024-09-10T08:37:42.666895+0000","last_became_active":"2024-09-10T08:37:05.259753+0000","last_became_peered":"2024-09-10T08:37:05.259753+0000","last_unstale":"2024-09-10T08:37:42.666895+0000","last_undegraded":"2024-09-10T08:37:42.666895+0000","last_fullsized":"2024-09-10T08:37:42.666895+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:19.364782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.669127+0000","last_change":"2024-09-10T08:36:56.147104+0000","last_active":"2024-09-10T08:37:42.669127+0000","last_peered":"2024-09-10T08:37:42.669127+0000","last_clean":"2024-09-10T08:37:42.669127+0000","last_became_active":"2024-09-10T08:36:56.146778+0000","last_became_peered":"2024-09-10T08:36:56.146778+0000","last_unstale":"2024-09-10T08:37:42.669127+0000","last_undegraded":"2024-09-10T08:37:42.669127+0000","last_fullsized":"2024-09-10T08:37:42.669127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:56.092157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664936+0000","last_change":"2024-09-10T08:36:53.719145+0000","last_active":"2024-09-10T08:37:42.664936+0000","last_peered":"2024-09-10T08:37:42.664936+0000","last_clean":"2024-09-10T08:37:42.664936+0000","last_became_active":"2024-09-10T08:36:53.718771+0000","last_became_peered":"2024-09-10T08:36:53.718771+0000","last_unstale":"2024-09-10T08:37:42.664936+0000","last_undegraded":"2024-09-10T08:37:42.664936+0000","last_fullsized":"2024-09-10T08:37:42.664936+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:21.109940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.843849+0000","last_change":"2024-09-10T08:37:05.242854+0000","last_active":"2024-09-10T08:37:42.843849+0000","last_peered":"2024-09-10T08:37:42.843849+0000","last_clean":"2024-09-10T08:37:42.843849+0000","last_became_active":"2024-09-10T08:37:05.242442+0000","last_became_peered":"2024-09-10T08:37:05.242442+0000","last_unstale":"2024-09-10T08:37:42.843849+0000","last_undegraded":"2024-09-10T08:37:42.843849+0000","last_fullsized":"2024-09-10T08:37:42.843849+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:36.072131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.845642+0000","last_change":"2024-09-10T08:37:42.845642+0000","last_active":"2024-09-10T08:37:42.845642+0000","last_peered":"2024-09-10T08:37:42.845642+0000","last_clean":"2024-09-10T08:37:42.845642+0000","last_became_active":"2024-09-10T08:37:42.845217+0000","last_became_peered":"2024-09-10T08:37:42.845217+0000","last_unstale":"2024-09-10T08:37:42.845642+0000","last_undegraded":"2024-09-10T08:37:42.845642+0000","last_fullsized":"2024-09-10T08:37:42.845642+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:28.020337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:48.719014+0000","last_change":"2024-09-10T08:36:53.720463+0000","last_active":"2024-09-10T08:37:48.719014+0000","last_peered":"2024-09-10T08:37:48.719014+0000","last_clean":"2024-09-10T08:37:48.719014+0000","last_became_active":"2024-09-10T08:36:53.720080+0000","last_became_peered":"2024-09-10T08:36:53.720080+0000","last_unstale":"2024-09-10T08:37:48.719014+0000","last_undegraded":"2024-09-10T08:37:48.719014+0000","last_fullsized":"2024-09-10T08:37:48.719014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:51.450294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.843769+0000","last_change":"2024-09-10T08:37:05.251261+0000","last_active":"2024-09-10T08:37:42.843769+0000","last_peered":"2024-09-10T08:37:42.843769+0000","last_clean":"2024-09-10T08:37:42.843769+0000","last_became_active":"2024-09-10T08:37:05.250938+0000","last_became_peered":"2024-09-10T08:37:05.250938+0000","last_unstale":"2024-09-10T08:37:42.843769+0000","last_undegraded":"2024-09-10T08:37:42.843769+0000","last_fullsized":"2024-09-10T08:37:42.843769+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:07:15.129899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.843971+0000","last_change":"2024-09-10T08:36:56.148191+0000","last_active":"2024-09-10T08:37:42.843971+0000","last_peered":"2024-09-10T08:37:42.843971+0000","last_clean":"2024-09-10T08:37:42.843971+0000","last_became_active":"2024-09-10T08:36:56.147362+0000","last_became_peered":"2024-09-10T08:36:56.147362+0000","last_unstale":"2024-09-10T08:37:42.843971+0000","last_undegraded":"2024-09-10T08:37:42.843971+0000","last_fullsized":"2024-09-10T08:37:42.843971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:25:30.051754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:47.714375+0000","last_change":"2024-09-10T08:36:53.734484+0000","last_active":"2024-09-10T08:37:47.714375+0000","last_peered":"2024-09-10T08:37:47.714375+0000","last_clean":"2024-09-10T08:37:47.714375+0000","last_became_active":"2024-09-10T08:36:53.734172+0000","last_became_peered":"2024-09-10T08:36:53.734172+0000","last_unstale":"2024-09-10T08:37:47.714375+0000","last_undegraded":"2024-09-10T08:37:47.714375+0000","last_fullsized":"2024-09-10T08:37:47.714375+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:10.269400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664883+0000","last_change":"2024-09-10T08:37:05.259635+0000","last_active":"2024-09-10T08:37:42.664883+0000","last_peered":"2024-09-10T08:37:42.664883+0000","last_clean":"2024-09-10T08:37:42.664883+0000","last_became_active":"2024-09-10T08:37:05.259107+0000","last_became_peered":"2024-09-10T08:37:05.259107+0000","last_unstale":"2024-09-10T08:37:42.664883+0000","last_undegraded":"2024-09-10T08:37:42.664883+0000","last_fullsized":"2024-09-10T08:37:42.664883+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:36.841138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668126+0000","last_change":"2024-09-10T08:36:56.141122+0000","last_active":"2024-09-10T08:37:42.668126+0000","last_peered":"2024-09-10T08:37:42.668126+0000","last_clean":"2024-09-10T08:37:42.668126+0000","last_became_active":"2024-09-10T08:36:56.140826+0000","last_became_peered":"2024-09-10T08:36:56.140826+0000","last_unstale":"2024-09-10T08:37:42.668126+0000","last_undegraded":"2024-09-10T08:37:42.668126+0000","last_fullsized":"2024-09-10T08:37:42.668126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:38.044090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'4","reported_seq":60,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:38:03.743835+0000","last_change":"2024-09-10T08:36:53.720623+0000","last_active":"2024-09-10T08:38:03.743835+0000","last_peered":"2024-09-10T08:38:03.743835+0000","last_clean":"2024-09-10T08:38:03.743835+0000","last_became_active":"2024-09-10T08:36:53.720270+0000","last_became_peered":"2024-09-10T08:36:53.720270+0000","last_unstale":"2024-09-10T08:38:03.743835+0000","last_undegraded":"2024-09-10T08:38:03.743835+0000","last_fullsized":"2024-09-10T08:38:03.743835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:19.514068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.669881+0000","last_change":"2024-09-10T08:37:05.254324+0000","last_active":"2024-09-10T08:37:42.669881+0000","last_peered":"2024-09-10T08:37:42.669881+0000","last_clean":"2024-09-10T08:37:42.669881+0000","last_became_active":"2024-09-10T08:37:05.253611+0000","last_became_peered":"2024-09-10T08:37:05.253611+0000","last_unstale":"2024-09-10T08:37:42.669881+0000","last_undegraded":"2024-09-10T08:37:42.669881+0000","last_fullsized":"2024-09-10T08:37:42.669881+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:07.182649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668627+0000","last_change":"2024-09-10T08:36:56.139102+0000","last_active":"2024-09-10T08:37:42.668627+0000","last_peered":"2024-09-10T08:37:42.668627+0000","last_clean":"2024-09-10T08:37:42.668627+0000","last_became_active":"2024-09-10T08:36:56.137986+0000","last_became_peered":"2024-09-10T08:36:56.137986+0000","last_unstale":"2024-09-10T08:37:42.668627+0000","last_undegraded":"2024-09-10T08:37:42.668627+0000","last_fullsized":"2024-09-10T08:37:42.668627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:05:03.005727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:46.702276+0000","last_change":"2024-09-10T08:36:53.722961+0000","last_active":"2024-09-10T08:37:46.702276+0000","last_peered":"2024-09-10T08:37:46.702276+0000","last_clean":"2024-09-10T08:37:46.702276+0000","last_became_active":"2024-09-10T08:36:53.722628+0000","last_became_peered":"2024-09-10T08:36:53.722628+0000","last_unstale":"2024-09-10T08:37:46.702276+0000","last_undegraded":"2024-09-10T08:37:46.702276+0000","last_fullsized":"2024-09-10T08:37:46.702276+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:59.048632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664305+0000","last_change":"2024-09-10T08:37:05.256546+0000","last_active":"2024-09-10T08:37:42.664305+0000","last_peered":"2024-09-10T08:37:42.664305+0000","last_clean":"2024-09-10T08:37:42.664305+0000","last_became_active":"2024-09-10T08:37:05.256239+0000","last_became_peered":"2024-09-10T08:37:05.256239+0000","last_unstale":"2024-09-10T08:37:42.664305+0000","last_undegraded":"2024-09-10T08:37:42.664305+0000","last_fullsized":"2024-09-10T08:37:42.664305+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:57.403934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664318+0000","last_change":"2024-09-10T08:36:56.149285+0000","last_active":"2024-09-10T08:37:42.664318+0000","last_peered":"2024-09-10T08:37:42.664318+0000","last_clean":"2024-09-10T08:37:42.664318+0000","last_became_active":"2024-09-10T08:36:56.146873+0000","last_became_peered":"2024-09-10T08:36:56.146873+0000","last_unstale":"2024-09-10T08:37:42.664318+0000","last_undegraded":"2024-09-10T08:37:42.664318+0000","last_fullsized":"2024-09-10T08:37:42.664318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:28.976729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:47.714300+0000","last_change":"2024-09-10T08:36:53.721174+0000","last_active":"2024-09-10T08:37:47.714300+0000","last_peered":"2024-09-10T08:37:47.714300+0000","last_clean":"2024-09-10T08:37:47.714300+0000","last_became_active":"2024-09-10T08:36:53.720849+0000","last_became_peered":"2024-09-10T08:36:53.720849+0000","last_unstale":"2024-09-10T08:37:47.714300+0000","last_undegraded":"2024-09-10T08:37:47.714300+0000","last_fullsized":"2024-09-10T08:37:47.714300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:39.786848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.844261+0000","last_change":"2024-09-10T08:37:05.252256+0000","last_active":"2024-09-10T08:37:42.844261+0000","last_peered":"2024-09-10T08:37:42.844261+0000","last_clean":"2024-09-10T08:37:42.844261+0000","last_became_active":"2024-09-10T08:37:05.251865+0000","last_became_peered":"2024-09-10T08:37:05.251865+0000","last_unstale":"2024-09-10T08:37:42.844261+0000","last_undegraded":"2024-09-10T08:37:42.844261+0000","last_fullsized":"2024-09-10T08:37:42.844261+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:08.581880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668571+0000","last_change":"2024-09-10T08:36:56.142060+0000","last_active":"2024-09-10T08:37:42.668571+0000","last_peered":"2024-09-10T08:37:42.668571+0000","last_clean":"2024-09-10T08:37:42.668571+0000","last_became_active":"2024-09-10T08:36:56.141451+0000","last_became_peered":"2024-09-10T08:36:56.141451+0000","last_unstale":"2024-09-10T08:37:42.668571+0000","last_undegraded":"2024-09-10T08:37:42.668571+0000","last_fullsized":"2024-09-10T08:37:42.668571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:43.566588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'26","reported_seq":100,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:38:03.743469+0000","last_change":"2024-09-10T08:36:53.719464+0000","last_active":"2024-09-10T08:38:03.743469+0000","last_peered":"2024-09-10T08:38:03.743469+0000","last_clean":"2024-09-10T08:38:03.743469+0000","last_became_active":"2024-09-10T08:36:53.719129+0000","last_became_peered":"2024-09-10T08:36:53.719129+0000","last_unstale":"2024-09-10T08:38:03.743469+0000","last_undegraded":"2024-09-10T08:38:03.743469+0000","last_fullsized":"2024-09-10T08:38:03.743469+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:05.652018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49098,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":67,"num_write":27,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.669325+0000","last_change":"2024-09-10T08:37:05.255576+0000","last_active":"2024-09-10T08:37:42.669325+0000","last_peered":"2024-09-10T08:37:42.669325+0000","last_clean":"2024-09-10T08:37:42.669325+0000","last_became_active":"2024-09-10T08:37:05.249842+0000","last_became_peered":"2024-09-10T08:37:05.249842+0000","last_unstale":"2024-09-10T08:37:42.669325+0000","last_undegraded":"2024-09-10T08:37:42.669325+0000","last_fullsized":"2024-09-10T08:37:42.669325+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:47:49.543481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.289176+0000","last_change":"2024-09-10T08:36:56.151829+0000","last_active":"2024-09-10T08:37:43.289176+0000","last_peered":"2024-09-10T08:37:43.289176+0000","last_clean":"2024-09-10T08:37:43.289176+0000","last_became_active":"2024-09-10T08:36:56.151314+0000","last_became_peered":"2024-09-10T08:36:56.151314+0000","last_unstale":"2024-09-10T08:37:43.289176+0000","last_undegraded":"2024-09-10T08:37:43.289176+0000","last_fullsized":"2024-09-10T08:37:43.289176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:44.219671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:46.702055+0000","last_change":"2024-09-10T08:36:53.738030+0000","last_active":"2024-09-10T08:37:46.702055+0000","last_peered":"2024-09-10T08:37:46.702055+0000","last_clean":"2024-09-10T08:37:46.702055+0000","last_became_active":"2024-09-10T08:36:53.737658+0000","last_became_peered":"2024-09-10T08:36:53.737658+0000","last_unstale":"2024-09-10T08:37:46.702055+0000","last_undegraded":"2024-09-10T08:37:46.702055+0000","last_fullsized":"2024-09-10T08:37:46.702055+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:29.627554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.666129+0000","last_change":"2024-09-10T08:37:05.244473+0000","last_active":"2024-09-10T08:37:42.666129+0000","last_peered":"2024-09-10T08:37:42.666129+0000","last_clean":"2024-09-10T08:37:42.666129+0000","last_became_active":"2024-09-10T08:37:05.244035+0000","last_became_peered":"2024-09-10T08:37:05.244035+0000","last_unstale":"2024-09-10T08:37:42.666129+0000","last_undegraded":"2024-09-10T08:37:42.666129+0000","last_fullsized":"2024-09-10T08:37:42.666129+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:04.652660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.667575+0000","last_change":"2024-09-10T08:36:56.155217+0000","last_active":"2024-09-10T08:37:42.667575+0000","last_peered":"2024-09-10T08:37:42.667575+0000","last_clean":"2024-09-10T08:37:42.667575+0000","last_became_active":"2024-09-10T08:36:56.154903+0000","last_became_peered":"2024-09-10T08:36:56.154903+0000","last_unstale":"2024-09-10T08:37:42.667575+0000","last_undegraded":"2024-09-10T08:37:42.667575+0000","last_fullsized":"2024-09-10T08:37:42.667575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:47.482974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:46.701861+0000","last_change":"2024-09-10T08:36:53.721087+0000","last_active":"2024-09-10T08:37:46.701861+0000","last_peered":"2024-09-10T08:37:46.701861+0000","last_clean":"2024-09-10T08:37:46.701861+0000","last_became_active":"2024-09-10T08:36:53.720772+0000","last_became_peered":"2024-09-10T08:36:53.720772+0000","last_unstale":"2024-09-10T08:37:46.701861+0000","last_undegraded":"2024-09-10T08:37:46.701861+0000","last_fullsized":"2024-09-10T08:37:46.701861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:39.560848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668820+0000","last_change":"2024-09-10T08:37:05.254641+0000","last_active":"2024-09-10T08:37:42.668820+0000","last_peered":"2024-09-10T08:37:42.668820+0000","last_clean":"2024-09-10T08:37:42.668820+0000","last_became_active":"2024-09-10T08:37:05.249982+0000","last_became_peered":"2024-09-10T08:37:05.249982+0000","last_unstale":"2024-09-10T08:37:42.668820+0000","last_undegraded":"2024-09-10T08:37:42.668820+0000","last_fullsized":"2024-09-10T08:37:42.668820+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:21:13.851061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.289336+0000","last_change":"2024-09-10T08:36:56.142657+0000","last_active":"2024-09-10T08:37:43.289336+0000","last_peered":"2024-09-10T08:37:43.289336+0000","last_clean":"2024-09-10T08:37:43.289336+0000","last_became_active":"2024-09-10T08:36:56.142294+0000","last_became_peered":"2024-09-10T08:36:56.142294+0000","last_unstale":"2024-09-10T08:37:43.289336+0000","last_undegraded":"2024-09-10T08:37:43.289336+0000","last_fullsized":"2024-09-10T08:37:43.289336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:52.861389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"94'3","reported_seq":55,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:38:02.741968+0000","last_change":"2024-09-10T08:36:53.722466+0000","last_active":"2024-09-10T08:38:02.741968+0000","last_peered":"2024-09-10T08:38:02.741968+0000","last_clean":"2024-09-10T08:38:02.741968+0000","last_became_active":"2024-09-10T08:36:53.722145+0000","last_became_peered":"2024-09-10T08:36:53.722145+0000","last_unstale":"2024-09-10T08:38:02.741968+0000","last_undegraded":"2024-09-10T08:38:02.741968+0000","last_fullsized":"2024-09-10T08:38:02.741968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:27.041765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.221668+0000","last_change":"2024-09-10T08:37:05.251056+0000","last_active":"2024-09-10T08:37:43.221668+0000","last_peered":"2024-09-10T08:37:43.221668+0000","last_clean":"2024-09-10T08:37:43.221668+0000","last_became_active":"2024-09-10T08:37:05.250653+0000","last_became_peered":"2024-09-10T08:37:05.250653+0000","last_unstale":"2024-09-10T08:37:43.221668+0000","last_undegraded":"2024-09-10T08:37:43.221668+0000","last_fullsized":"2024-09-10T08:37:43.221668+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:21.174747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.667368+0000","last_change":"2024-09-10T08:37:05.260436+0000","last_active":"2024-09-10T08:37:42.667368+0000","last_peered":"2024-09-10T08:37:42.667368+0000","last_clean":"2024-09-10T08:37:42.667368+0000","last_became_active":"2024-09-10T08:37:05.260049+0000","last_became_peered":"2024-09-10T08:37:05.260049+0000","last_unstale":"2024-09-10T08:37:42.667368+0000","last_undegraded":"2024-09-10T08:37:42.667368+0000","last_fullsized":"2024-09-10T08:37:42.667368+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:43.394028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.669027+0000","last_change":"2024-09-10T08:36:56.146732+0000","last_active":"2024-09-10T08:37:42.669027+0000","last_peered":"2024-09-10T08:37:42.669027+0000","last_clean":"2024-09-10T08:37:42.669027+0000","last_became_active":"2024-09-10T08:36:56.145599+0000","last_became_peered":"2024-09-10T08:36:56.145599+0000","last_unstale":"2024-09-10T08:37:42.669027+0000","last_undegraded":"2024-09-10T08:37:42.669027+0000","last_fullsized":"2024-09-10T08:37:42.669027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:50.620956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.844518+0000","last_change":"2024-09-10T08:36:53.721683+0000","last_active":"2024-09-10T08:37:42.844518+0000","last_peered":"2024-09-10T08:37:42.844518+0000","last_clean":"2024-09-10T08:37:42.844518+0000","last_became_active":"2024-09-10T08:36:53.721241+0000","last_became_peered":"2024-09-10T08:36:53.721241+0000","last_unstale":"2024-09-10T08:37:42.844518+0000","last_undegraded":"2024-09-10T08:37:42.844518+0000","last_fullsized":"2024-09-10T08:37:42.844518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:03.669337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.669784+0000","last_change":"2024-09-10T08:37:42.669784+0000","last_active":"2024-09-10T08:37:42.669784+0000","last_peered":"2024-09-10T08:37:42.669784+0000","last_clean":"2024-09-10T08:37:42.669784+0000","last_became_active":"2024-09-10T08:37:42.669322+0000","last_became_peered":"2024-09-10T08:37:42.669322+0000","last_unstale":"2024-09-10T08:37:42.669784+0000","last_undegraded":"2024-09-10T08:37:42.669784+0000","last_fullsized":"2024-09-10T08:37:42.669784+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:06:57.027928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.221647+0000","last_change":"2024-09-10T08:36:56.143824+0000","last_active":"2024-09-10T08:37:43.221647+0000","last_peered":"2024-09-10T08:37:43.221647+0000","last_clean":"2024-09-10T08:37:43.221647+0000","last_became_active":"2024-09-10T08:36:56.143486+0000","last_became_peered":"2024-09-10T08:36:56.143486+0000","last_unstale":"2024-09-10T08:37:43.221647+0000","last_undegraded":"2024-09-10T08:37:43.221647+0000","last_fullsized":"2024-09-10T08:37:43.221647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:42.384863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:47.713929+0000","last_change":"2024-09-10T08:36:53.717680+0000","last_active":"2024-09-10T08:37:47.713929+0000","last_peered":"2024-09-10T08:37:47.713929+0000","last_clean":"2024-09-10T08:37:47.713929+0000","last_became_active":"2024-09-10T08:36:53.717249+0000","last_became_peered":"2024-09-10T08:36:53.717249+0000","last_unstale":"2024-09-10T08:37:47.713929+0000","last_undegraded":"2024-09-10T08:37:47.713929+0000","last_fullsized":"2024-09-10T08:37:47.713929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:12.191541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:47.714060+0000","last_change":"2024-09-10T08:36:53.723947+0000","last_active":"2024-09-10T08:37:47.714060+0000","last_peered":"2024-09-10T08:37:47.714060+0000","last_clean":"2024-09-10T08:37:47.714060+0000","last_became_active":"2024-09-10T08:36:53.723620+0000","last_became_peered":"2024-09-10T08:36:53.723620+0000","last_unstale":"2024-09-10T08:37:47.714060+0000","last_undegraded":"2024-09-10T08:37:47.714060+0000","last_fullsized":"2024-09-10T08:37:47.714060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:00.135349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.844083+0000","last_change":"2024-09-10T08:36:56.150448+0000","last_active":"2024-09-10T08:37:42.844083+0000","last_peered":"2024-09-10T08:37:42.844083+0000","last_clean":"2024-09-10T08:37:42.844083+0000","last_became_active":"2024-09-10T08:36:56.150148+0000","last_became_peered":"2024-09-10T08:36:56.150148+0000","last_unstale":"2024-09-10T08:37:42.844083+0000","last_undegraded":"2024-09-10T08:37:42.844083+0000","last_fullsized":"2024-09-10T08:37:42.844083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:02.927753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.288941+0000","last_change":"2024-09-10T08:37:05.242878+0000","last_active":"2024-09-10T08:37:43.288941+0000","last_peered":"2024-09-10T08:37:43.288941+0000","last_clean":"2024-09-10T08:37:43.288941+0000","last_became_active":"2024-09-10T08:37:05.242440+0000","last_became_peered":"2024-09-10T08:37:05.242440+0000","last_unstale":"2024-09-10T08:37:43.288941+0000","last_undegraded":"2024-09-10T08:37:43.288941+0000","last_fullsized":"2024-09-10T08:37:43.288941+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:30.957433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'5","reported_seq":61,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:38:04.744984+0000","last_change":"2024-09-10T08:36:53.719788+0000","last_active":"2024-09-10T08:38:04.744984+0000","last_peered":"2024-09-10T08:38:04.744984+0000","last_clean":"2024-09-10T08:38:04.744984+0000","last_became_active":"2024-09-10T08:36:53.719462+0000","last_became_peered":"2024-09-10T08:36:53.719462+0000","last_unstale":"2024-09-10T08:38:04.744984+0000","last_undegraded":"2024-09-10T08:38:04.744984+0000","last_fullsized":"2024-09-10T08:38:04.744984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:12.250447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.673008+0000","last_change":"2024-09-10T08:37:42.673008+0000","last_active":"2024-09-10T08:37:42.673008+0000","last_peered":"2024-09-10T08:37:42.673008+0000","last_clean":"2024-09-10T08:37:42.673008+0000","last_became_active":"2024-09-10T08:37:42.672618+0000","last_became_peered":"2024-09-10T08:37:42.672618+0000","last_unstale":"2024-09-10T08:37:42.673008+0000","last_undegraded":"2024-09-10T08:37:42.673008+0000","last_fullsized":"2024-09-10T08:37:42.673008+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:00.156565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.289122+0000","last_change":"2024-09-10T08:37:05.244277+0000","last_active":"2024-09-10T08:37:43.289122+0000","last_peered":"2024-09-10T08:37:43.289122+0000","last_clean":"2024-09-10T08:37:43.289122+0000","last_became_active":"2024-09-10T08:37:05.243880+0000","last_became_peered":"2024-09-10T08:37:05.243880+0000","last_unstale":"2024-09-10T08:37:43.289122+0000","last_undegraded":"2024-09-10T08:37:43.289122+0000","last_fullsized":"2024-09-10T08:37:43.289122+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:55.275857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.663983+0000","last_change":"2024-09-10T08:36:56.141404+0000","last_active":"2024-09-10T08:37:42.663983+0000","last_peered":"2024-09-10T08:37:42.663983+0000","last_clean":"2024-09-10T08:37:42.663983+0000","last_became_active":"2024-09-10T08:36:56.141076+0000","last_became_peered":"2024-09-10T08:36:56.141076+0000","last_unstale":"2024-09-10T08:37:42.663983+0000","last_undegraded":"2024-09-10T08:37:42.663983+0000","last_fullsized":"2024-09-10T08:37:42.663983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:03:52.915654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.289101+0000","last_change":"2024-09-10T08:36:56.148285+0000","last_active":"2024-09-10T08:37:43.289101+0000","last_peered":"2024-09-10T08:37:43.289101+0000","last_clean":"2024-09-10T08:37:43.289101+0000","last_became_active":"2024-09-10T08:36:56.147970+0000","last_became_peered":"2024-09-10T08:36:56.147970+0000","last_unstale":"2024-09-10T08:37:43.289101+0000","last_undegraded":"2024-09-10T08:37:43.289101+0000","last_fullsized":"2024-09-10T08:37:43.289101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:09.506718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.288789+0000","last_change":"2024-09-10T08:36:56.157977+0000","last_active":"2024-09-10T08:37:43.288789+0000","last_peered":"2024-09-10T08:37:43.288789+0000","last_clean":"2024-09-10T08:37:43.288789+0000","last_became_active":"2024-09-10T08:36:56.157623+0000","last_became_peered":"2024-09-10T08:36:56.157623+0000","last_unstale":"2024-09-10T08:37:43.288789+0000","last_undegraded":"2024-09-10T08:37:43.288789+0000","last_fullsized":"2024-09-10T08:37:43.288789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:44.617979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.670459+0000","last_change":"2024-09-10T08:37:42.670459+0000","last_active":"2024-09-10T08:37:42.670459+0000","last_peered":"2024-09-10T08:37:42.670459+0000","last_clean":"2024-09-10T08:37:42.670459+0000","last_became_active":"2024-09-10T08:37:42.670097+0000","last_became_peered":"2024-09-10T08:37:42.670097+0000","last_unstale":"2024-09-10T08:37:42.670459+0000","last_undegraded":"2024-09-10T08:37:42.670459+0000","last_fullsized":"2024-09-10T08:37:42.670459+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:14.956492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.665105+0000","last_change":"2024-09-10T08:36:56.154850+0000","last_active":"2024-09-10T08:37:42.665105+0000","last_peered":"2024-09-10T08:37:42.665105+0000","last_clean":"2024-09-10T08:37:42.665105+0000","last_became_active":"2024-09-10T08:36:56.154545+0000","last_became_peered":"2024-09-10T08:36:56.154545+0000","last_unstale":"2024-09-10T08:37:42.665105+0000","last_undegraded":"2024-09-10T08:37:42.665105+0000","last_fullsized":"2024-09-10T08:37:42.665105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:24.639667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.669962+0000","last_change":"2024-09-10T08:37:42.669962+0000","last_active":"2024-09-10T08:37:42.669962+0000","last_peered":"2024-09-10T08:37:42.669962+0000","last_clean":"2024-09-10T08:37:42.669962+0000","last_became_active":"2024-09-10T08:37:42.669556+0000","last_became_peered":"2024-09-10T08:37:42.669556+0000","last_unstale":"2024-09-10T08:37:42.669962+0000","last_undegraded":"2024-09-10T08:37:42.669962+0000","last_fullsized":"2024-09-10T08:37:42.669962+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:23.333946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.289409+0000","last_change":"2024-09-10T08:36:56.145910+0000","last_active":"2024-09-10T08:37:43.289409+0000","last_peered":"2024-09-10T08:37:43.289409+0000","last_clean":"2024-09-10T08:37:43.289409+0000","last_became_active":"2024-09-10T08:36:56.145537+0000","last_became_peered":"2024-09-10T08:36:56.145537+0000","last_unstale":"2024-09-10T08:37:43.289409+0000","last_undegraded":"2024-09-10T08:37:43.289409+0000","last_fullsized":"2024-09-10T08:37:43.289409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:46.788241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.222267+0000","last_change":"2024-09-10T08:36:56.147340+0000","last_active":"2024-09-10T08:37:43.222267+0000","last_peered":"2024-09-10T08:37:43.222267+0000","last_clean":"2024-09-10T08:37:43.222267+0000","last_became_active":"2024-09-10T08:36:56.145762+0000","last_became_peered":"2024-09-10T08:36:56.145762+0000","last_unstale":"2024-09-10T08:37:43.222267+0000","last_undegraded":"2024-09-10T08:37:43.222267+0000","last_fullsized":"2024-09-10T08:37:43.222267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:12.339285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.844221+0000","last_change":"2024-09-10T08:36:56.145213+0000","last_active":"2024-09-10T08:37:42.844221+0000","last_peered":"2024-09-10T08:37:42.844221+0000","last_clean":"2024-09-10T08:37:42.844221+0000","last_became_active":"2024-09-10T08:36:56.144915+0000","last_became_peered":"2024-09-10T08:36:56.144915+0000","last_unstale":"2024-09-10T08:37:42.844221+0000","last_undegraded":"2024-09-10T08:37:42.844221+0000","last_fullsized":"2024-09-10T08:37:42.844221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:06.180321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.221804+0000","last_change":"2024-09-10T08:36:56.149738+0000","last_active":"2024-09-10T08:37:43.221804+0000","last_peered":"2024-09-10T08:37:43.221804+0000","last_clean":"2024-09-10T08:37:43.221804+0000","last_became_active":"2024-09-10T08:36:56.147750+0000","last_became_peered":"2024-09-10T08:36:56.147750+0000","last_unstale":"2024-09-10T08:37:43.221804+0000","last_undegraded":"2024-09-10T08:37:43.221804+0000","last_fullsized":"2024-09-10T08:37:43.221804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:47.950054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.289027+0000","last_change":"2024-09-10T08:36:56.147682+0000","last_active":"2024-09-10T08:37:43.289027+0000","last_peered":"2024-09-10T08:37:43.289027+0000","last_clean":"2024-09-10T08:37:43.289027+0000","last_became_active":"2024-09-10T08:36:56.147345+0000","last_became_peered":"2024-09-10T08:36:56.147345+0000","last_unstale":"2024-09-10T08:37:43.289027+0000","last_undegraded":"2024-09-10T08:37:43.289027+0000","last_fullsized":"2024-09-10T08:37:43.289027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:54.939482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664511+0000","last_change":"2024-09-10T08:36:56.158292+0000","last_active":"2024-09-10T08:37:42.664511+0000","last_peered":"2024-09-10T08:37:42.664511+0000","last_clean":"2024-09-10T08:37:42.664511+0000","last_became_active":"2024-09-10T08:36:56.157970+0000","last_became_peered":"2024-09-10T08:36:56.157970+0000","last_unstale":"2024-09-10T08:37:42.664511+0000","last_undegraded":"2024-09-10T08:37:42.664511+0000","last_fullsized":"2024-09-10T08:37:42.664511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:33.582126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664202+0000","last_change":"2024-09-10T08:36:56.142520+0000","last_active":"2024-09-10T08:37:42.664202+0000","last_peered":"2024-09-10T08:37:42.664202+0000","last_clean":"2024-09-10T08:37:42.664202+0000","last_became_active":"2024-09-10T08:36:56.142214+0000","last_became_peered":"2024-09-10T08:36:56.142214+0000","last_unstale":"2024-09-10T08:37:42.664202+0000","last_undegraded":"2024-09-10T08:37:42.664202+0000","last_fullsized":"2024-09-10T08:37:42.664202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:45:29.201839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.289281+0000","last_change":"2024-09-10T08:36:56.149547+0000","last_active":"2024-09-10T08:37:43.289281+0000","last_peered":"2024-09-10T08:37:43.289281+0000","last_clean":"2024-09-10T08:37:43.289281+0000","last_became_active":"2024-09-10T08:36:56.149141+0000","last_became_peered":"2024-09-10T08:36:56.149141+0000","last_unstale":"2024-09-10T08:37:43.289281+0000","last_undegraded":"2024-09-10T08:37:43.289281+0000","last_fullsized":"2024-09-10T08:37:43.289281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:49.368677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.670068+0000","last_change":"2024-09-10T08:36:56.143087+0000","last_active":"2024-09-10T08:37:42.670068+0000","last_peered":"2024-09-10T08:37:42.670068+0000","last_clean":"2024-09-10T08:37:42.670068+0000","last_became_active":"2024-09-10T08:36:56.142782+0000","last_became_peered":"2024-09-10T08:36:56.142782+0000","last_unstale":"2024-09-10T08:37:42.670068+0000","last_undegraded":"2024-09-10T08:37:42.670068+0000","last_fullsized":"2024-09-10T08:37:42.670068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:17.513103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664866+0000","last_change":"2024-09-10T08:36:56.157549+0000","last_active":"2024-09-10T08:37:42.664866+0000","last_peered":"2024-09-10T08:37:42.664866+0000","last_clean":"2024-09-10T08:37:42.664866+0000","last_became_active":"2024-09-10T08:36:56.157221+0000","last_became_peered":"2024-09-10T08:36:56.157221+0000","last_unstale":"2024-09-10T08:37:42.664866+0000","last_undegraded":"2024-09-10T08:37:42.664866+0000","last_fullsized":"2024-09-10T08:37:42.664866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:11.372085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.288782+0000","last_change":"2024-09-10T08:36:56.145654+0000","last_active":"2024-09-10T08:37:43.288782+0000","last_peered":"2024-09-10T08:37:43.288782+0000","last_clean":"2024-09-10T08:37:43.288782+0000","last_became_active":"2024-09-10T08:36:56.144126+0000","last_became_peered":"2024-09-10T08:36:56.144126+0000","last_unstale":"2024-09-10T08:37:43.288782+0000","last_undegraded":"2024-09-10T08:37:43.288782+0000","last_fullsized":"2024-09-10T08:37:43.288782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:55.498873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668470+0000","last_change":"2024-09-10T08:36:56.146423+0000","last_active":"2024-09-10T08:37:42.668470+0000","last_peered":"2024-09-10T08:37:42.668470+0000","last_clean":"2024-09-10T08:37:42.668470+0000","last_became_active":"2024-09-10T08:36:56.146126+0000","last_became_peered":"2024-09-10T08:36:56.146126+0000","last_unstale":"2024-09-10T08:37:42.668470+0000","last_undegraded":"2024-09-10T08:37:42.668470+0000","last_fullsized":"2024-09-10T08:37:42.668470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:44.665196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.666854+0000","last_change":"2024-09-10T08:36:56.151973+0000","last_active":"2024-09-10T08:37:42.666854+0000","last_peered":"2024-09-10T08:37:42.666854+0000","last_clean":"2024-09-10T08:37:42.666854+0000","last_became_active":"2024-09-10T08:36:56.151609+0000","last_became_peered":"2024-09-10T08:36:56.151609+0000","last_unstale":"2024-09-10T08:37:42.666854+0000","last_undegraded":"2024-09-10T08:37:42.666854+0000","last_fullsized":"2024-09-10T08:37:42.666854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:02.584047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664405+0000","last_change":"2024-09-10T08:36:56.141780+0000","last_active":"2024-09-10T08:37:42.664405+0000","last_peered":"2024-09-10T08:37:42.664405+0000","last_clean":"2024-09-10T08:37:42.664405+0000","last_became_active":"2024-09-10T08:36:56.141228+0000","last_became_peered":"2024-09-10T08:36:56.141228+0000","last_unstale":"2024-09-10T08:37:42.664405+0000","last_undegraded":"2024-09-10T08:37:42.664405+0000","last_fullsized":"2024-09-10T08:37:42.664405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:52:04.374228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.674303+0000","last_change":"2024-09-10T08:37:42.671346+0000","last_active":"2024-09-10T08:37:42.674303+0000","last_peered":"2024-09-10T08:37:42.674303+0000","last_clean":"2024-09-10T08:37:42.674303+0000","last_became_active":"2024-09-10T08:37:42.670895+0000","last_became_peered":"2024-09-10T08:37:42.670895+0000","last_unstale":"2024-09-10T08:37:42.674303+0000","last_undegraded":"2024-09-10T08:37:42.674303+0000","last_fullsized":"2024-09-10T08:37:42.674303+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:16:31.930628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.669181+0000","last_change":"2024-09-10T08:36:56.149014+0000","last_active":"2024-09-10T08:37:42.669181+0000","last_peered":"2024-09-10T08:37:42.669181+0000","last_clean":"2024-09-10T08:37:42.669181+0000","last_became_active":"2024-09-10T08:36:56.148551+0000","last_became_peered":"2024-09-10T08:36:56.148551+0000","last_unstale":"2024-09-10T08:37:42.669181+0000","last_undegraded":"2024-09-10T08:37:42.669181+0000","last_fullsized":"2024-09-10T08:37:42.669181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:35:52.250943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.666087+0000","last_change":"2024-09-10T08:36:56.141247+0000","last_active":"2024-09-10T08:37:42.666087+0000","last_peered":"2024-09-10T08:37:42.666087+0000","last_clean":"2024-09-10T08:37:42.666087+0000","last_became_active":"2024-09-10T08:36:56.140956+0000","last_became_peered":"2024-09-10T08:36:56.140956+0000","last_unstale":"2024-09-10T08:37:42.666087+0000","last_undegraded":"2024-09-10T08:37:42.666087+0000","last_fullsized":"2024-09-10T08:37:42.666087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:45.008424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:43.289423+0000","last_change":"2024-09-10T08:36:56.149116+0000","last_active":"2024-09-10T08:37:43.289423+0000","last_peered":"2024-09-10T08:37:43.289423+0000","last_clean":"2024-09-10T08:37:43.289423+0000","last_became_active":"2024-09-10T08:36:56.148783+0000","last_became_peered":"2024-09-10T08:36:56.148783+0000","last_unstale":"2024-09-10T08:37:43.289423+0000","last_undegraded":"2024-09-10T08:37:43.289423+0000","last_fullsized":"2024-09-10T08:37:43.289423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:37.119214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.846747+0000","last_change":"2024-09-10T08:37:42.846322+0000","last_active":"2024-09-10T08:37:42.846747+0000","last_peered":"2024-09-10T08:37:42.846747+0000","last_clean":"2024-09-10T08:37:42.846747+0000","last_became_active":"2024-09-10T08:37:42.846022+0000","last_became_peered":"2024-09-10T08:37:42.846022+0000","last_unstale":"2024-09-10T08:37:42.846747+0000","last_undegraded":"2024-09-10T08:37:42.846747+0000","last_fullsized":"2024-09-10T08:37:42.846747+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:38.142362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668299+0000","last_change":"2024-09-10T08:36:56.148537+0000","last_active":"2024-09-10T08:37:42.668299+0000","last_peered":"2024-09-10T08:37:42.668299+0000","last_clean":"2024-09-10T08:37:42.668299+0000","last_became_active":"2024-09-10T08:36:56.148185+0000","last_became_peered":"2024-09-10T08:36:56.148185+0000","last_unstale":"2024-09-10T08:37:42.668299+0000","last_undegraded":"2024-09-10T08:37:42.668299+0000","last_fullsized":"2024-09-10T08:37:42.668299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:00.426513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.666311+0000","last_change":"2024-09-10T08:36:56.138513+0000","last_active":"2024-09-10T08:37:42.666311+0000","last_peered":"2024-09-10T08:37:42.666311+0000","last_clean":"2024-09-10T08:37:42.666311+0000","last_became_active":"2024-09-10T08:36:56.138207+0000","last_became_peered":"2024-09-10T08:36:56.138207+0000","last_unstale":"2024-09-10T08:37:42.666311+0000","last_undegraded":"2024-09-10T08:37:42.666311+0000","last_fullsized":"2024-09-10T08:37:42.666311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:34.581430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.668091+0000","last_change":"2024-09-10T08:36:56.146612+0000","last_active":"2024-09-10T08:37:42.668091+0000","last_peered":"2024-09-10T08:37:42.668091+0000","last_clean":"2024-09-10T08:37:42.668091+0000","last_became_active":"2024-09-10T08:36:56.146251+0000","last_became_peered":"2024-09-10T08:36:56.146251+0000","last_unstale":"2024-09-10T08:37:42.668091+0000","last_undegraded":"2024-09-10T08:37:42.668091+0000","last_fullsized":"2024-09-10T08:37:42.668091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:47.336389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.667090+0000","last_change":"2024-09-10T08:36:56.148260+0000","last_active":"2024-09-10T08:37:42.667090+0000","last_peered":"2024-09-10T08:37:42.667090+0000","last_clean":"2024-09-10T08:37:42.667090+0000","last_became_active":"2024-09-10T08:36:56.147949+0000","last_became_peered":"2024-09-10T08:36:56.147949+0000","last_unstale":"2024-09-10T08:37:42.667090+0000","last_undegraded":"2024-09-10T08:37:42.667090+0000","last_fullsized":"2024-09-10T08:37:42.667090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:52:19.762297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.667150+0000","last_change":"2024-09-10T08:36:56.156045+0000","last_active":"2024-09-10T08:37:42.667150+0000","last_peered":"2024-09-10T08:37:42.667150+0000","last_clean":"2024-09-10T08:37:42.667150+0000","last_became_active":"2024-09-10T08:36:56.154727+0000","last_became_peered":"2024-09-10T08:36:56.154727+0000","last_unstale":"2024-09-10T08:37:42.667150+0000","last_undegraded":"2024-09-10T08:37:42.667150+0000","last_fullsized":"2024-09-10T08:37:42.667150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:35:39.992683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.667375+0000","last_change":"2024-09-10T08:36:56.154394+0000","last_active":"2024-09-10T08:37:42.667375+0000","last_peered":"2024-09-10T08:37:42.667375+0000","last_clean":"2024-09-10T08:37:42.667375+0000","last_became_active":"2024-09-10T08:36:56.153855+0000","last_became_peered":"2024-09-10T08:36:56.153855+0000","last_unstale":"2024-09-10T08:37:42.667375+0000","last_undegraded":"2024-09-10T08:37:42.667375+0000","last_fullsized":"2024-09-10T08:37:42.667375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:14.987147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T08:37:42.664653+0000","last_change":"2024-09-10T08:36:56.157517+0000","last_active":"2024-09-10T08:37:42.664653+0000","last_peered":"2024-09-10T08:37:42.664653+0000","last_clean":"2024-09-10T08:37:42.664653+0000","last_became_active":"2024-09-10T08:36:56.157133+0000","last_became_peered":"2024-09-10T08:36:56.157133+0000","last_unstale":"2024-09-10T08:37:42.664653+0000","last_undegraded":"2024-09-10T08:37:42.664653+0000","last_fullsized":"2024-09-10T08:37:42.664653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:17:29.573719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53101,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":125,"num_read_kb":119,"num_write":156,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":179599,"data_compressed":5667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282455,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":528370,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678048,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1167360,"data_stored":1173321,"data_compressed":61354,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841574,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":524510,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103212,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":528526,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332145,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":1124437,"data_compressed":59465,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":523782,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":1119632,"data_compressed":59465,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":528825,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149765,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":523976,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":574140,"data_compressed":25428,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1089536,"data_stored":582938,"data_compressed":25428,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":525170,"data_compressed":23539,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":59246,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60596,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":51069,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:38:09.820 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T08:38:09.820 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T08:38:09.821 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T08:38:10.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:09 smithi111 ceph-mon[35220]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:38:10.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:09 smithi111 ceph-mon[35220]: pgmap v338: 129 pgs: 129 active+clean; 629 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-09-10T08:38:10.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:09 smithi089 ceph-mon[30757]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:38:10.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:09 smithi089 ceph-mon[30757]: pgmap v338: 129 pgs: 129 active+clean; 629 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-09-10T08:38:10.185 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:10.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:09 smithi149 ceph-mon[36167]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:38:10.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:09 smithi149 ceph-mon[36167]: pgmap v338: 129 pgs: 129 active+clean; 629 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-09-10T08:38:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.754+0000 7f17e7e00640 1 -- 172.21.15.89:0/2342440896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e01037c0 msgr2=0x7f17e010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.754+0000 7f17e7e00640 1 --2- 172.21.15.89:0/2342440896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e01037c0 0x7f17e010a050 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f17d8009be0 tx=0x7f17d802a6c0 comp rx=0 tx=0).stop 2024-09-10T08:38:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.755+0000 7f17e7e00640 1 -- 172.21.15.89:0/2342440896 shutdown_connections 2024-09-10T08:38:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.755+0000 7f17e7e00640 1 --2- 172.21.15.89:0/2342440896 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e01037c0 0x7f17e010a050 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.755+0000 7f17e7e00640 1 --2- 172.21.15.89:0/2342440896 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17e0102e00 0x7f17e0103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.755+0000 7f17e7e00640 1 --2- 172.21.15.89:0/2342440896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e0101c00 0x7f17e0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.755+0000 7f17e7e00640 1 -- 172.21.15.89:0/2342440896 >> 172.21.15.89:0/2342440896 conn(0x7f17e00fd390 msgr2=0x7f17e00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.755+0000 7f17e7e00640 1 -- 172.21.15.89:0/2342440896 shutdown_connections 2024-09-10T08:38:10.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.755+0000 7f17e7e00640 1 -- 172.21.15.89:0/2342440896 wait complete. 2024-09-10T08:38:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.756+0000 7f17e7e00640 1 Processor -- start 2024-09-10T08:38:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.756+0000 7f17e7e00640 1 -- start start 2024-09-10T08:38:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.756+0000 7f17e7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e0101c00 0x7f17e019a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.756+0000 7f17e7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17e0102e00 0x7f17e019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.756+0000 7f17e7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e01037c0 0x7f17e01a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.756+0000 7f17e7e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f17e010d860 con 0x7f17e01037c0 2024-09-10T08:38:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.756+0000 7f17e7e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f17e010d6e0 con 0x7f17e0102e00 2024-09-10T08:38:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.756+0000 7f17e7e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f17e010d9e0 con 0x7f17e0101c00 2024-09-10T08:38:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.756+0000 7f17e5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e01037c0 0x7f17e01a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.757+0000 7f17e4a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17e0102e00 0x7f17e019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.757+0000 7f17e5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e01037c0 0x7f17e01a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36208/0 (socket says 172.21.15.89:36208) 2024-09-10T08:38:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.757+0000 7f17e5e00640 1 -- 172.21.15.89:0/2982530735 learned_addr learned my addr 172.21.15.89:0/2982530735 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:10.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.757+0000 7f17e4a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17e0102e00 0x7f17e019a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35612/0 (socket says 172.21.15.89:35612) 2024-09-10T08:38:10.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.757+0000 7f17e5400640 1 --2- 172.21.15.89:0/2982530735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e0101c00 0x7f17e019a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:10.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.757+0000 7f17e5e00640 1 -- 172.21.15.89:0/2982530735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e0101c00 msgr2=0x7f17e019a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:10.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.757+0000 7f17e5e00640 1 --2- 172.21.15.89:0/2982530735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e0101c00 0x7f17e019a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:10.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.757+0000 7f17e5e00640 1 -- 172.21.15.89:0/2982530735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17e0102e00 msgr2=0x7f17e019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:10.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.757+0000 7f17e5e00640 1 --2- 172.21.15.89:0/2982530735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17e0102e00 0x7f17e019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:10.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.757+0000 7f17e5e00640 1 -- 172.21.15.89:0/2982530735 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e01a1ca0 con 0x7f17e01037c0 2024-09-10T08:38:10.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.757+0000 7f17e4a00640 1 --2- 172.21.15.89:0/2982530735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17e0102e00 0x7f17e019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:38:10.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.757+0000 7f17e5e00640 1 --2- 172.21.15.89:0/2982530735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e01037c0 0x7f17e01a1630 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f17d8009b30 tx=0x7f17d80305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:10.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.758+0000 7f17de000640 1 -- 172.21.15.89:0/2982530735 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8041020 con 0x7f17e01037c0 2024-09-10T08:38:10.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.758+0000 7f17de000640 1 -- 172.21.15.89:0/2982530735 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17d8030740 con 0x7f17e01037c0 2024-09-10T08:38:10.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.758+0000 7f17e7e00640 1 -- 172.21.15.89:0/2982530735 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e01a1f30 con 0x7f17e01037c0 2024-09-10T08:38:10.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.758+0000 7f17de000640 1 -- 172.21.15.89:0/2982530735 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8036040 con 0x7f17e01037c0 2024-09-10T08:38:10.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.758+0000 7f17e7e00640 1 -- 172.21.15.89:0/2982530735 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e01a23e0 con 0x7f17e01037c0 2024-09-10T08:38:10.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.759+0000 7f17e7e00640 1 -- 172.21.15.89:0/2982530735 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17e01a2610 con 0x7f17e01037c0 2024-09-10T08:38:10.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.761+0000 7f17de000640 1 -- 172.21.15.89:0/2982530735 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f17d8044020 con 0x7f17e01037c0 2024-09-10T08:38:10.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.763+0000 7f17de000640 1 --2- 172.21.15.89:0/2982530735 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f17c80a2cc0 0x7f17c80a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:10.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.763+0000 7f17de000640 1 -- 172.21.15.89:0/2982530735 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f17d80f0180 con 0x7f17e01037c0 2024-09-10T08:38:10.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.763+0000 7f17e5400640 1 --2- 172.21.15.89:0/2982530735 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f17c80a2cc0 0x7f17c80a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:10.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.763+0000 7f17e5400640 1 --2- 172.21.15.89:0/2982530735 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f17c80a2cc0 0x7f17c80a5180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f17e0102c60 tx=0x7f17d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:10.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:10.767+0000 7f17de000640 1 -- 172.21.15.89:0/2982530735 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f17d804b020 con 0x7f17e01037c0 2024-09-10T08:38:11.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:11.004+0000 7f17e7e00640 1 -- 172.21.15.89:0/2982530735 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f17e01a2920 con 0x7f17e01037c0 2024-09-10T08:38:11.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:11.006+0000 7f17de000640 1 -- 172.21.15.89:0/2982530735 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f17d80bcbc0 con 0x7f17e01037c0 2024-09-10T08:38:11.008 INFO:teuthology.orchestra.run.smithi089.stdout:[client.0] 2024-09-10T08:38:11.008 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQBzBeBmgNVbABAA/XxmB1A6NCLpNUQhDjDiuA== 2024-09-10T08:38:11.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:11.010+0000 7f17e7e00640 1 -- 172.21.15.89:0/2982530735 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f17c80a2cc0 msgr2=0x7f17c80a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:11.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:11.010+0000 7f17e7e00640 1 --2- 172.21.15.89:0/2982530735 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f17c80a2cc0 0x7f17c80a5180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f17e0102c60 tx=0x7f17d4005f90 comp rx=0 tx=0).stop 2024-09-10T08:38:11.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:11.010+0000 7f17e7e00640 1 -- 172.21.15.89:0/2982530735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e01037c0 msgr2=0x7f17e01a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:11.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:11.010+0000 7f17e7e00640 1 --2- 172.21.15.89:0/2982530735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e01037c0 0x7f17e01a1630 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f17d8009b30 tx=0x7f17d80305d0 comp rx=0 tx=0).stop 2024-09-10T08:38:11.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:11.010+0000 7f17e7e00640 1 -- 172.21.15.89:0/2982530735 shutdown_connections 2024-09-10T08:38:11.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:11.010+0000 7f17e7e00640 1 --2- 172.21.15.89:0/2982530735 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f17c80a2cc0 0x7f17c80a5180 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:11.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:11.010+0000 7f17e7e00640 1 --2- 172.21.15.89:0/2982530735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17e01037c0 0x7f17e01a1630 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:11.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:11.010+0000 7f17e7e00640 1 --2- 172.21.15.89:0/2982530735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17e0102e00 0x7f17e019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:11.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:11.010+0000 7f17e7e00640 1 --2- 172.21.15.89:0/2982530735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e0101c00 0x7f17e019a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:11.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:11.010+0000 7f17e7e00640 1 -- 172.21.15.89:0/2982530735 >> 172.21.15.89:0/2982530735 conn(0x7f17e00fd390 msgr2=0x7f17e0074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:11.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:11.010+0000 7f17e7e00640 1 -- 172.21.15.89:0/2982530735 shutdown_connections 2024-09-10T08:38:11.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:11.010+0000 7f17e7e00640 1 -- 172.21.15.89:0/2982530735 wait complete. 2024-09-10T08:38:11.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:11 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2982530735' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T08:38:11.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:11 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2982530735' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T08:38:11.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:11 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2982530735' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T08:38:11.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:11 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2982530735' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T08:38:11.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:11 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2982530735' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T08:38:11.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:11 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2982530735' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T08:38:11.825 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:38:11.825 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T08:38:11.825 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T08:38:11.906 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T08:38:11.907 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T08:38:11.907 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T08:38:12.313 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:12.451 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:12 smithi089 ceph-mon[30757]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-09-10T08:38:12.473 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:12 smithi149 ceph-mon[36167]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-09-10T08:38:12.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:12 smithi111 ceph-mon[35220]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-09-10T08:38:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.865+0000 7f76bae00640 1 -- 172.21.15.89:0/3877800818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b4108f60 msgr2=0x7f76b4110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.865+0000 7f76bae00640 1 --2- 172.21.15.89:0/3877800818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b4108f60 0x7f76b4110ae0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009900 tx=0x7f76ac02a410 comp rx=0 tx=0).stop 2024-09-10T08:38:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.866+0000 7f76bae00640 1 -- 172.21.15.89:0/3877800818 shutdown_connections 2024-09-10T08:38:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.866+0000 7f76bae00640 1 --2- 172.21.15.89:0/3877800818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b4108f60 0x7f76b4110ae0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.866+0000 7f76bae00640 1 --2- 172.21.15.89:0/3877800818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76b41047e0 0x7f76b4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.866+0000 7f76bae00640 1 --2- 172.21.15.89:0/3877800818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76b4103e30 0x7f76b4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.866+0000 7f76bae00640 1 -- 172.21.15.89:0/3877800818 >> 172.21.15.89:0/3877800818 conn(0x7f76b40ff8d0 msgr2=0x7f76b4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.866+0000 7f76bae00640 1 -- 172.21.15.89:0/3877800818 shutdown_connections 2024-09-10T08:38:12.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.866+0000 7f76bae00640 1 -- 172.21.15.89:0/3877800818 wait complete. 2024-09-10T08:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.867+0000 7f76bae00640 1 Processor -- start 2024-09-10T08:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.867+0000 7f76bae00640 1 -- start start 2024-09-10T08:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.867+0000 7f76bae00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b4103e30 0x7f76b419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.867+0000 7f76bae00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76b41047e0 0x7f76b419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.867+0000 7f76bae00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76b4108f60 0x7f76b419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.867+0000 7f76bae00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f76b4114010 con 0x7f76b4103e30 2024-09-10T08:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.867+0000 7f76bae00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f76b4113e90 con 0x7f76b4108f60 2024-09-10T08:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.867+0000 7f76bae00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f76b4114190 con 0x7f76b41047e0 2024-09-10T08:38:12.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.867+0000 7f76b9a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b4103e30 0x7f76b419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.868+0000 7f76b9a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b4103e30 0x7f76b419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54048/0 (socket says 172.21.15.89:54048) 2024-09-10T08:38:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.868+0000 7f76b9a00640 1 -- 172.21.15.89:0/386698239 learned_addr learned my addr 172.21.15.89:0/386698239 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.868+0000 7f76b9000640 1 --2- 172.21.15.89:0/386698239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76b41047e0 0x7f76b419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.868+0000 7f76ba400640 1 --2- 172.21.15.89:0/386698239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76b4108f60 0x7f76b419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.868+0000 7f76b9a00640 1 -- 172.21.15.89:0/386698239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76b41047e0 msgr2=0x7f76b419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.868+0000 7f76b9a00640 1 --2- 172.21.15.89:0/386698239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76b41047e0 0x7f76b419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.868+0000 7f76b9a00640 1 -- 172.21.15.89:0/386698239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76b4108f60 msgr2=0x7f76b419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.868+0000 7f76b9a00640 1 --2- 172.21.15.89:0/386698239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76b4108f60 0x7f76b419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.868+0000 7f76b9a00640 1 -- 172.21.15.89:0/386698239 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b419f3c0 con 0x7f76b4103e30 2024-09-10T08:38:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.868+0000 7f76b9000640 1 --2- 172.21.15.89:0/386698239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76b41047e0 0x7f76b419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:38:12.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.868+0000 7f76b9a00640 1 --2- 172.21.15.89:0/386698239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b4103e30 0x7f76b419a410 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f76a800eb90 tx=0x7f76a800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:12.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.869+0000 7f76b2000640 1 -- 172.21.15.89:0/386698239 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8004d20 con 0x7f76b4103e30 2024-09-10T08:38:12.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.869+0000 7f76b2000640 1 -- 172.21.15.89:0/386698239 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76a800ee90 con 0x7f76b4103e30 2024-09-10T08:38:12.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.869+0000 7f76b2000640 1 -- 172.21.15.89:0/386698239 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8010830 con 0x7f76b4103e30 2024-09-10T08:38:12.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.869+0000 7f76bae00640 1 -- 172.21.15.89:0/386698239 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b419f6b0 con 0x7f76b4103e30 2024-09-10T08:38:12.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.869+0000 7f76bae00640 1 -- 172.21.15.89:0/386698239 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b41a6f90 con 0x7f76b4103e30 2024-09-10T08:38:12.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.870+0000 7f76bae00640 1 -- 172.21.15.89:0/386698239 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b4104210 con 0x7f76b4103e30 2024-09-10T08:38:12.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.872+0000 7f76b2000640 1 -- 172.21.15.89:0/386698239 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f76a8005020 con 0x7f76b4103e30 2024-09-10T08:38:12.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.873+0000 7f76b2000640 1 --2- 172.21.15.89:0/386698239 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f769c0a2c90 0x7f769c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:12.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.873+0000 7f76b2000640 1 -- 172.21.15.89:0/386698239 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f76a80cfb30 con 0x7f76b4103e30 2024-09-10T08:38:12.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.873+0000 7f76b9000640 1 --2- 172.21.15.89:0/386698239 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f769c0a2c90 0x7f769c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:12.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.874+0000 7f76b9000640 1 --2- 172.21.15.89:0/386698239 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f769c0a2c90 0x7f769c0a5150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f76b41000d0 tx=0x7f76a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:12.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:12.877+0000 7f76b2000640 1 -- 172.21.15.89:0/386698239 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76a809c570 con 0x7f76b4103e30 2024-09-10T08:38:13.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:13.095+0000 7f76bae00640 1 -- 172.21.15.89:0/386698239 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f76b419b770 con 0x7f76b4103e30 2024-09-10T08:38:13.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:13.101+0000 7f76b2000640 1 -- 172.21.15.89:0/386698239 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263201 (secure 0 0 0) 0x7f76a809be70 con 0x7f76b4103e30 2024-09-10T08:38:13.105 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:38:13.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:13.105+0000 7f76bae00640 1 -- 172.21.15.89:0/386698239 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f769c0a2c90 msgr2=0x7f769c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:13.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:13.105+0000 7f76bae00640 1 --2- 172.21.15.89:0/386698239 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f769c0a2c90 0x7f769c0a5150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f76b41000d0 tx=0x7f76a0005f90 comp rx=0 tx=0).stop 2024-09-10T08:38:13.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:13.105+0000 7f76bae00640 1 -- 172.21.15.89:0/386698239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b4103e30 msgr2=0x7f76b419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:13.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:13.105+0000 7f76bae00640 1 --2- 172.21.15.89:0/386698239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b4103e30 0x7f76b419a410 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f76a800eb90 tx=0x7f76a800c6a0 comp rx=0 tx=0).stop 2024-09-10T08:38:13.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:13.105+0000 7f76bae00640 1 -- 172.21.15.89:0/386698239 shutdown_connections 2024-09-10T08:38:13.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:13.105+0000 7f76bae00640 1 --2- 172.21.15.89:0/386698239 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f769c0a2c90 0x7f769c0a5150 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:13.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:13.105+0000 7f76bae00640 1 --2- 172.21.15.89:0/386698239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76b4108f60 0x7f76b419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:13.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:13.105+0000 7f76bae00640 1 --2- 172.21.15.89:0/386698239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76b41047e0 0x7f76b419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:13.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:13.105+0000 7f76bae00640 1 --2- 172.21.15.89:0/386698239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f76b4103e30 0x7f76b419a410 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:13.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:13.105+0000 7f76bae00640 1 -- 172.21.15.89:0/386698239 >> 172.21.15.89:0/386698239 conn(0x7f76b40ff8d0 msgr2=0x7f76b4100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:13.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:13.106+0000 7f76bae00640 1 -- 172.21.15.89:0/386698239 shutdown_connections 2024-09-10T08:38:13.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:13.106+0000 7f76bae00640 1 -- 172.21.15.89:0/386698239 wait complete. 2024-09-10T08:38:13.415 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:13 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/386698239' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T08:38:13.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:13 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/386698239' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T08:38:13.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:13 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/386698239' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T08:38:13.805 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":1633345099},{"type":"v1","addr":"172.21.15.89:6801","nonce":1633345099}]},"active_addr":"172.21.15.89:6801/1633345099","active_change":"2024-09-10T08:29:41.472108+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.89:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":3689147449}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":3600869128}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":2807915475}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":3367934879}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.89:0","nonce":1827464790}]}]} 2024-09-10T08:38:13.811 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T08:38:13.811 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T08:38:13.811 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:38:14.182 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:14.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:14 smithi149 ceph-mon[36167]: pgmap v340: 129 pgs: 129 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-10T08:38:14.479 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:14 smithi089 ceph-mon[30757]: pgmap v340: 129 pgs: 129 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-10T08:38:14.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:14 smithi111 ceph-mon[35220]: pgmap v340: 129 pgs: 129 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-10T08:38:14.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.737+0000 7f0f2b800640 1 -- 172.21.15.89:0/97340856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f24106f00 msgr2=0x7f0f2410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:14.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.737+0000 7f0f2b800640 1 --2- 172.21.15.89:0/97340856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f24106f00 0x7f0f2410ea80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009ca0 tx=0x7f0f1c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:38:14.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.737+0000 7f0f2b800640 1 -- 172.21.15.89:0/97340856 shutdown_connections 2024-09-10T08:38:14.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.737+0000 7f0f2b800640 1 --2- 172.21.15.89:0/97340856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f24106f00 0x7f0f2410ea80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:14.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.737+0000 7f0f2b800640 1 --2- 172.21.15.89:0/97340856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f24102780 0x7f0f241067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:14.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.737+0000 7f0f2b800640 1 --2- 172.21.15.89:0/97340856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f24101dd0 0x7f0f241021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:14.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.737+0000 7f0f2b800640 1 -- 172.21.15.89:0/97340856 >> 172.21.15.89:0/97340856 conn(0x7f0f240fd890 msgr2=0x7f0f240ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:14.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.738+0000 7f0f2b800640 1 -- 172.21.15.89:0/97340856 shutdown_connections 2024-09-10T08:38:14.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.738+0000 7f0f2b800640 1 -- 172.21.15.89:0/97340856 wait complete. 2024-09-10T08:38:14.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.739+0000 7f0f2b800640 1 Processor -- start 2024-09-10T08:38:14.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.739+0000 7f0f2b800640 1 -- start start 2024-09-10T08:38:14.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.739+0000 7f0f2b800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f24101dd0 0x7f0f2419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:14.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.739+0000 7f0f2b800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f24102780 0x7f0f2419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:14.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.739+0000 7f0f2b800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f24106f00 0x7f0f2419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:14.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.739+0000 7f0f2b800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0f24111fb0 con 0x7f0f24106f00 2024-09-10T08:38:14.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.739+0000 7f0f2b800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0f24111e30 con 0x7f0f24101dd0 2024-09-10T08:38:14.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.739+0000 7f0f2b800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0f24112130 con 0x7f0f24102780 2024-09-10T08:38:14.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.740+0000 7f0f29800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f24106f00 0x7f0f2419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:14.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.740+0000 7f0f28e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f24101dd0 0x7f0f2419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:14.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.740+0000 7f0f28e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f24101dd0 0x7f0f2419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:56814/0 (socket says 172.21.15.89:56814) 2024-09-10T08:38:14.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.740+0000 7f0f23e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f24102780 0x7f0f2419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:14.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.740+0000 7f0f28e00640 1 -- 172.21.15.89:0/177368669 learned_addr learned my addr 172.21.15.89:0/177368669 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:14.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.740+0000 7f0f23e00640 1 -- 172.21.15.89:0/177368669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f24101dd0 msgr2=0x7f0f2419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:14.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.740+0000 7f0f23e00640 1 --2- 172.21.15.89:0/177368669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f24101dd0 0x7f0f2419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:14.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.740+0000 7f0f23e00640 1 -- 172.21.15.89:0/177368669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f24106f00 msgr2=0x7f0f2419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:14.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.740+0000 7f0f23e00640 1 --2- 172.21.15.89:0/177368669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f24106f00 0x7f0f2419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:14.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.740+0000 7f0f23e00640 1 -- 172.21.15.89:0/177368669 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2419f5e0 con 0x7f0f24102780 2024-09-10T08:38:14.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.741+0000 7f0f23e00640 1 --2- 172.21.15.89:0/177368669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f24102780 0x7f0f2419ab70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000e9f0 tx=0x7f0f1000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:14.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.741+0000 7f0f21600640 1 -- 172.21.15.89:0/177368669 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1000ccc0 con 0x7f0f24102780 2024-09-10T08:38:14.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.741+0000 7f0f21600640 1 -- 172.21.15.89:0/177368669 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f10005020 con 0x7f0f24102780 2024-09-10T08:38:14.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.741+0000 7f0f2b800640 1 -- 172.21.15.89:0/177368669 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2419f8d0 con 0x7f0f24102780 2024-09-10T08:38:14.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.741+0000 7f0f21600640 1 -- 172.21.15.89:0/177368669 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f100049e0 con 0x7f0f24102780 2024-09-10T08:38:14.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.741+0000 7f0f2b800640 1 -- 172.21.15.89:0/177368669 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f241a71b0 con 0x7f0f24102780 2024-09-10T08:38:14.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.742+0000 7f0f2b800640 1 -- 172.21.15.89:0/177368669 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f241021b0 con 0x7f0f24102780 2024-09-10T08:38:14.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.744+0000 7f0f21600640 1 -- 172.21.15.89:0/177368669 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0f10005560 con 0x7f0f24102780 2024-09-10T08:38:14.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.745+0000 7f0f21600640 1 --2- 172.21.15.89:0/177368669 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0f0c0a2c10 0x7f0f0c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:14.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.746+0000 7f0f21600640 1 -- 172.21.15.89:0/177368669 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f0f100cf550 con 0x7f0f24102780 2024-09-10T08:38:14.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.746+0000 7f0f28e00640 1 --2- 172.21.15.89:0/177368669 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0f0c0a2c10 0x7f0f0c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:14.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.746+0000 7f0f28e00640 1 --2- 172.21.15.89:0/177368669 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0f0c0a2c10 0x7f0f0c0a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009c90 tx=0x7f0f18005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:14.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.747+0000 7f0f21600640 1 -- 172.21.15.89:0/177368669 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0f100051d0 con 0x7f0f24102780 2024-09-10T08:38:14.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.925+0000 7f0f2b800640 1 -- 172.21.15.89:0/177368669 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f2419b990 con 0x7f0f24102780 2024-09-10T08:38:14.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.927+0000 7f0f21600640 1 -- 172.21.15.89:0/177368669 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24707 (secure 0 0 0) 0x7f0f1009c020 con 0x7f0f24102780 2024-09-10T08:38:14.927 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:38:14.928 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":94,"fsid":"91f747bc-6f4e-11ef-bcea-c7b262605968","created":"2024-09-10T08:28:49.697035+0000","modified":"2024-09-10T08:37:42.660676+0000","last_up_change":"2024-09-10T08:36:00.535433+0000","last_in_change":"2024-09-10T08:35:37.131131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:32:02.060714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:36:52.462301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:36:55.119708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:37:03.729002+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d3b55192-bd95-4497-89ea-dc017a25c0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6803","nonce":1760523284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6805","nonce":1760523284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6809","nonce":1760523284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6807","nonce":1760523284}]},"public_addr":"172.21.15.89:6803/1760523284","cluster_addr":"172.21.15.89:6805/1760523284","heartbeat_back_addr":"172.21.15.89:6809/1760523284","heartbeat_front_addr":"172.21.15.89:6807/1760523284","state":["exists","up"]},{"osd":1,"uuid":"60aae117-cb52-4034-a700-11693171cd52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6801","nonce":2369291742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6803","nonce":2369291742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6807","nonce":2369291742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6805","nonce":2369291742}]},"public_addr":"172.21.15.111:6801/2369291742","cluster_addr":"172.21.15.111:6803/2369291742","heartbeat_back_addr":"172.21.15.111:6807/2369291742","heartbeat_front_addr":"172.21.15.111:6805/2369291742","state":["exists","up"]},{"osd":2,"uuid":"a305a019-0870-437c-a298-4f923ec54e77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6801","nonce":1092261395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6803","nonce":1092261395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6807","nonce":1092261395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6805","nonce":1092261395}]},"public_addr":"172.21.15.149:6801/1092261395","cluster_addr":"172.21.15.149:6803/1092261395","heartbeat_back_addr":"172.21.15.149:6807/1092261395","heartbeat_front_addr":"172.21.15.149:6805/1092261395","state":["exists","up"]},{"osd":3,"uuid":"d02f54c1-f883-40f6-9341-8d9fc4b712ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6811","nonce":1405773950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6813","nonce":1405773950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6817","nonce":1405773950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6815","nonce":1405773950}]},"public_addr":"172.21.15.89:6811/1405773950","cluster_addr":"172.21.15.89:6813/1405773950","heartbeat_back_addr":"172.21.15.89:6817/1405773950","heartbeat_front_addr":"172.21.15.89:6815/1405773950","state":["exists","up"]},{"osd":4,"uuid":"23cff46b-88dd-4344-be06-b35db6014f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6809","nonce":316717323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6811","nonce":316717323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6815","nonce":316717323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6813","nonce":316717323}]},"public_addr":"172.21.15.111:6809/316717323","cluster_addr":"172.21.15.111:6811/316717323","heartbeat_back_addr":"172.21.15.111:6815/316717323","heartbeat_front_addr":"172.21.15.111:6813/316717323","state":["exists","up"]},{"osd":5,"uuid":"8e516e8d-e7c6-4ebc-80a5-29932161df4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6809","nonce":1256148461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6811","nonce":1256148461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6815","nonce":1256148461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6813","nonce":1256148461}]},"public_addr":"172.21.15.149:6809/1256148461","cluster_addr":"172.21.15.149:6811/1256148461","heartbeat_back_addr":"172.21.15.149:6815/1256148461","heartbeat_front_addr":"172.21.15.149:6813/1256148461","state":["exists","up"]},{"osd":6,"uuid":"48f92472-f657-4c5a-adeb-20d4fea1ce8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6819","nonce":4190488024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6821","nonce":4190488024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6825","nonce":4190488024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6823","nonce":4190488024}]},"public_addr":"172.21.15.89:6819/4190488024","cluster_addr":"172.21.15.89:6821/4190488024","heartbeat_back_addr":"172.21.15.89:6825/4190488024","heartbeat_front_addr":"172.21.15.89:6823/4190488024","state":["exists","up"]},{"osd":7,"uuid":"14a2a6f0-c4cd-406b-b637-7527fd32babe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6817","nonce":652666376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6819","nonce":652666376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6823","nonce":652666376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6821","nonce":652666376}]},"public_addr":"172.21.15.111:6817/652666376","cluster_addr":"172.21.15.111:6819/652666376","heartbeat_back_addr":"172.21.15.111:6823/652666376","heartbeat_front_addr":"172.21.15.111:6821/652666376","state":["exists","up"]},{"osd":8,"uuid":"a54502e6-95f7-431f-8db1-54a6c7983b5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6817","nonce":2955172869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6819","nonce":2955172869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6823","nonce":2955172869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6821","nonce":2955172869}]},"public_addr":"172.21.15.149:6817/2955172869","cluster_addr":"172.21.15.149:6819/2955172869","heartbeat_back_addr":"172.21.15.149:6823/2955172869","heartbeat_front_addr":"172.21.15.149:6821/2955172869","state":["exists","up"]},{"osd":9,"uuid":"5fe17344-5b1c-4601-a633-639cc1691478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6827","nonce":2427467040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6829","nonce":2427467040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6833","nonce":2427467040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6831","nonce":2427467040}]},"public_addr":"172.21.15.89:6827/2427467040","cluster_addr":"172.21.15.89:6829/2427467040","heartbeat_back_addr":"172.21.15.89:6833/2427467040","heartbeat_front_addr":"172.21.15.89:6831/2427467040","state":["exists","up"]},{"osd":10,"uuid":"d8f29890-3064-4b67-b649-467a0ac86db7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6825","nonce":2328678744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6827","nonce":2328678744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6831","nonce":2328678744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6829","nonce":2328678744}]},"public_addr":"172.21.15.111:6825/2328678744","cluster_addr":"172.21.15.111:6827/2328678744","heartbeat_back_addr":"172.21.15.111:6831/2328678744","heartbeat_front_addr":"172.21.15.111:6829/2328678744","state":["exists","up"]},{"osd":11,"uuid":"92182c73-f139-47ca-a0ef-5046ae9b2310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6825","nonce":3130508827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6827","nonce":3130508827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6831","nonce":3130508827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6829","nonce":3130508827}]},"public_addr":"172.21.15.149:6825/3130508827","cluster_addr":"172.21.15.149:6827/3130508827","heartbeat_back_addr":"172.21.15.149:6831/3130508827","heartbeat_front_addr":"172.21.15.149:6829/3130508827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:09.031810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:33.695124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:58.065510+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:27.592838+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:53.275289+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:18.305677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:47.781535+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:12.889512+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:37.744193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:07.718408+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:33.391027+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:59.013250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/850669510":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/3541027428":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6801/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6800/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/2396468787":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/1609720980":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/995019144":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6801/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6800/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/2179977257":"2024-09-11T08:29:14.554869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:38:14.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.930+0000 7f0f2b800640 1 -- 172.21.15.89:0/177368669 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0f0c0a2c10 msgr2=0x7f0f0c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:14.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.930+0000 7f0f2b800640 1 --2- 172.21.15.89:0/177368669 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0f0c0a2c10 0x7f0f0c0a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009c90 tx=0x7f0f18005f90 comp rx=0 tx=0).stop 2024-09-10T08:38:14.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.930+0000 7f0f2b800640 1 -- 172.21.15.89:0/177368669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f24102780 msgr2=0x7f0f2419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:14.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.930+0000 7f0f2b800640 1 --2- 172.21.15.89:0/177368669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f24102780 0x7f0f2419ab70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000e9f0 tx=0x7f0f1000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:38:14.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.930+0000 7f0f2b800640 1 -- 172.21.15.89:0/177368669 shutdown_connections 2024-09-10T08:38:14.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.930+0000 7f0f2b800640 1 --2- 172.21.15.89:0/177368669 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0f0c0a2c10 0x7f0f0c0a50d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:14.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.930+0000 7f0f2b800640 1 --2- 172.21.15.89:0/177368669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f24106f00 0x7f0f2419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:14.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.930+0000 7f0f2b800640 1 --2- 172.21.15.89:0/177368669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f24102780 0x7f0f2419ab70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:14.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.930+0000 7f0f2b800640 1 --2- 172.21.15.89:0/177368669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f24101dd0 0x7f0f2419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:14.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.930+0000 7f0f2b800640 1 -- 172.21.15.89:0/177368669 >> 172.21.15.89:0/177368669 conn(0x7f0f240fd890 msgr2=0x7f0f240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:14.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.930+0000 7f0f2b800640 1 -- 172.21.15.89:0/177368669 shutdown_connections 2024-09-10T08:38:14.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:14.930+0000 7f0f2b800640 1 -- 172.21.15.89:0/177368669 wait complete. 2024-09-10T08:38:15.342 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:15 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/177368669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:38:15.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:15 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/177368669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:38:15.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:15 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/177368669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:38:15.698 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T08:38:15.698 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:38:16.064 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:16.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:16 smithi149 ceph-mon[36167]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-10T08:38:16.499 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:16 smithi089 ceph-mon[30757]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-10T08:38:16.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:16 smithi111 ceph-mon[35220]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-10T08:38:16.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.609+0000 7f412fc00640 1 -- 172.21.15.89:0/1023581394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4128074620 msgr2=0x7f4128074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:16.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.609+0000 7f412fc00640 1 --2- 172.21.15.89:0/1023581394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4128074620 0x7f4128074a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4118009900 tx=0x7f411802a480 comp rx=0 tx=0).stop 2024-09-10T08:38:16.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.609+0000 7f412fc00640 1 -- 172.21.15.89:0/1023581394 shutdown_connections 2024-09-10T08:38:16.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.609+0000 7f412fc00640 1 --2- 172.21.15.89:0/1023581394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f412810e3c0 0x7f4128110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:16.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.609+0000 7f412fc00640 1 --2- 172.21.15.89:0/1023581394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4128074f40 0x7f412810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:16.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.609+0000 7f412fc00640 1 --2- 172.21.15.89:0/1023581394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4128074620 0x7f4128074a00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:16.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.609+0000 7f412fc00640 1 -- 172.21.15.89:0/1023581394 >> 172.21.15.89:0/1023581394 conn(0x7f41280fd8c0 msgr2=0x7f41280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:16.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.609+0000 7f412fc00640 1 -- 172.21.15.89:0/1023581394 shutdown_connections 2024-09-10T08:38:16.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.609+0000 7f412fc00640 1 -- 172.21.15.89:0/1023581394 wait complete. 2024-09-10T08:38:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.610+0000 7f412fc00640 1 Processor -- start 2024-09-10T08:38:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.610+0000 7f412fc00640 1 -- start start 2024-09-10T08:38:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.610+0000 7f412fc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4128074620 0x7f412819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.610+0000 7f412fc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4128074f40 0x7f412819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.611+0000 7f412fc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f412810e3c0 0x7f412819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.611+0000 7f412fc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4128114150 con 0x7f412810e3c0 2024-09-10T08:38:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.611+0000 7f412fc00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4128113fd0 con 0x7f4128074620 2024-09-10T08:38:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.611+0000 7f412fc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f41281142d0 con 0x7f4128074f40 2024-09-10T08:38:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.611+0000 7f412d200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4128074620 0x7f412819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.611+0000 7f412d200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4128074620 0x7f412819a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:56832/0 (socket says 172.21.15.89:56832) 2024-09-10T08:38:16.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.611+0000 7f412d200640 1 -- 172.21.15.89:0/3687173591 learned_addr learned my addr 172.21.15.89:0/3687173591 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.611+0000 7f411fe00640 1 --2- 172.21.15.89:0/3687173591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4128074f40 0x7f412819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.611+0000 7f412dc00640 1 --2- 172.21.15.89:0/3687173591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f412810e3c0 0x7f412819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.611+0000 7f412d200640 1 -- 172.21.15.89:0/3687173591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4128074f40 msgr2=0x7f412819ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.611+0000 7f412d200640 1 --2- 172.21.15.89:0/3687173591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4128074f40 0x7f412819ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.611+0000 7f412d200640 1 -- 172.21.15.89:0/3687173591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f412810e3c0 msgr2=0x7f412819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.611+0000 7f412d200640 1 --2- 172.21.15.89:0/3687173591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f412810e3c0 0x7f412819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.611+0000 7f412d200640 1 -- 172.21.15.89:0/3687173591 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412819f6c0 con 0x7f4128074620 2024-09-10T08:38:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.611+0000 7f412dc00640 1 --2- 172.21.15.89:0/3687173591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f412810e3c0 0x7f412819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:38:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.612+0000 7f412d200640 1 --2- 172.21.15.89:0/3687173591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4128074620 0x7f412819a710 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4118006010 tx=0x7f41180042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.612+0000 7f4126000640 1 -- 172.21.15.89:0/3687173591 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411803f030 con 0x7f4128074620 2024-09-10T08:38:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.612+0000 7f412fc00640 1 -- 172.21.15.89:0/3687173591 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412819f950 con 0x7f4128074620 2024-09-10T08:38:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.612+0000 7f4126000640 1 -- 172.21.15.89:0/3687173591 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4118003ce0 con 0x7f4128074620 2024-09-10T08:38:16.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.612+0000 7f4126000640 1 -- 172.21.15.89:0/3687173591 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118035070 con 0x7f4128074620 2024-09-10T08:38:16.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.612+0000 7f412fc00640 1 -- 172.21.15.89:0/3687173591 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41281a7290 con 0x7f4128074620 2024-09-10T08:38:16.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.613+0000 7f412fc00640 1 -- 172.21.15.89:0/3687173591 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4128074a00 con 0x7f4128074620 2024-09-10T08:38:16.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.615+0000 7f4126000640 1 -- 172.21.15.89:0/3687173591 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4118042030 con 0x7f4128074620 2024-09-10T08:38:16.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.616+0000 7f4126000640 1 --2- 172.21.15.89:0/3687173591 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f410c0a2c90 0x7f410c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:16.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.616+0000 7f4126000640 1 -- 172.21.15.89:0/3687173591 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f41180efac0 con 0x7f4128074620 2024-09-10T08:38:16.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.617+0000 7f411fe00640 1 --2- 172.21.15.89:0/3687173591 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f410c0a2c90 0x7f410c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:16.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.617+0000 7f411fe00640 1 --2- 172.21.15.89:0/3687173591 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f410c0a2c90 0x7f410c0a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f412810f8b0 tx=0x7f411000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:16.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.618+0000 7f4126000640 1 -- 172.21.15.89:0/3687173591 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f41180bc5c0 con 0x7f4128074620 2024-09-10T08:38:16.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.798+0000 7f412fc00640 1 -- 172.21.15.89:0/3687173591 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f412819ba70 con 0x7f4128074620 2024-09-10T08:38:16.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.799+0000 7f4126000640 1 -- 172.21.15.89:0/3687173591 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24707 (secure 0 0 0) 0x7f41180bbec0 con 0x7f4128074620 2024-09-10T08:38:16.800 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:38:16.800 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":94,"fsid":"91f747bc-6f4e-11ef-bcea-c7b262605968","created":"2024-09-10T08:28:49.697035+0000","modified":"2024-09-10T08:37:42.660676+0000","last_up_change":"2024-09-10T08:36:00.535433+0000","last_in_change":"2024-09-10T08:35:37.131131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:32:02.060714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:36:52.462301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:36:55.119708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:37:03.729002+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d3b55192-bd95-4497-89ea-dc017a25c0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6803","nonce":1760523284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6805","nonce":1760523284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6809","nonce":1760523284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6807","nonce":1760523284}]},"public_addr":"172.21.15.89:6803/1760523284","cluster_addr":"172.21.15.89:6805/1760523284","heartbeat_back_addr":"172.21.15.89:6809/1760523284","heartbeat_front_addr":"172.21.15.89:6807/1760523284","state":["exists","up"]},{"osd":1,"uuid":"60aae117-cb52-4034-a700-11693171cd52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6801","nonce":2369291742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6803","nonce":2369291742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6807","nonce":2369291742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6805","nonce":2369291742}]},"public_addr":"172.21.15.111:6801/2369291742","cluster_addr":"172.21.15.111:6803/2369291742","heartbeat_back_addr":"172.21.15.111:6807/2369291742","heartbeat_front_addr":"172.21.15.111:6805/2369291742","state":["exists","up"]},{"osd":2,"uuid":"a305a019-0870-437c-a298-4f923ec54e77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6801","nonce":1092261395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6803","nonce":1092261395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6807","nonce":1092261395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6805","nonce":1092261395}]},"public_addr":"172.21.15.149:6801/1092261395","cluster_addr":"172.21.15.149:6803/1092261395","heartbeat_back_addr":"172.21.15.149:6807/1092261395","heartbeat_front_addr":"172.21.15.149:6805/1092261395","state":["exists","up"]},{"osd":3,"uuid":"d02f54c1-f883-40f6-9341-8d9fc4b712ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6811","nonce":1405773950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6813","nonce":1405773950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6817","nonce":1405773950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6815","nonce":1405773950}]},"public_addr":"172.21.15.89:6811/1405773950","cluster_addr":"172.21.15.89:6813/1405773950","heartbeat_back_addr":"172.21.15.89:6817/1405773950","heartbeat_front_addr":"172.21.15.89:6815/1405773950","state":["exists","up"]},{"osd":4,"uuid":"23cff46b-88dd-4344-be06-b35db6014f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6809","nonce":316717323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6811","nonce":316717323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6815","nonce":316717323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6813","nonce":316717323}]},"public_addr":"172.21.15.111:6809/316717323","cluster_addr":"172.21.15.111:6811/316717323","heartbeat_back_addr":"172.21.15.111:6815/316717323","heartbeat_front_addr":"172.21.15.111:6813/316717323","state":["exists","up"]},{"osd":5,"uuid":"8e516e8d-e7c6-4ebc-80a5-29932161df4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6809","nonce":1256148461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6811","nonce":1256148461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6815","nonce":1256148461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6813","nonce":1256148461}]},"public_addr":"172.21.15.149:6809/1256148461","cluster_addr":"172.21.15.149:6811/1256148461","heartbeat_back_addr":"172.21.15.149:6815/1256148461","heartbeat_front_addr":"172.21.15.149:6813/1256148461","state":["exists","up"]},{"osd":6,"uuid":"48f92472-f657-4c5a-adeb-20d4fea1ce8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6819","nonce":4190488024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6821","nonce":4190488024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6825","nonce":4190488024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6823","nonce":4190488024}]},"public_addr":"172.21.15.89:6819/4190488024","cluster_addr":"172.21.15.89:6821/4190488024","heartbeat_back_addr":"172.21.15.89:6825/4190488024","heartbeat_front_addr":"172.21.15.89:6823/4190488024","state":["exists","up"]},{"osd":7,"uuid":"14a2a6f0-c4cd-406b-b637-7527fd32babe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6817","nonce":652666376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6819","nonce":652666376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6823","nonce":652666376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6821","nonce":652666376}]},"public_addr":"172.21.15.111:6817/652666376","cluster_addr":"172.21.15.111:6819/652666376","heartbeat_back_addr":"172.21.15.111:6823/652666376","heartbeat_front_addr":"172.21.15.111:6821/652666376","state":["exists","up"]},{"osd":8,"uuid":"a54502e6-95f7-431f-8db1-54a6c7983b5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6817","nonce":2955172869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6819","nonce":2955172869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6823","nonce":2955172869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6821","nonce":2955172869}]},"public_addr":"172.21.15.149:6817/2955172869","cluster_addr":"172.21.15.149:6819/2955172869","heartbeat_back_addr":"172.21.15.149:6823/2955172869","heartbeat_front_addr":"172.21.15.149:6821/2955172869","state":["exists","up"]},{"osd":9,"uuid":"5fe17344-5b1c-4601-a633-639cc1691478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6827","nonce":2427467040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6829","nonce":2427467040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6833","nonce":2427467040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6831","nonce":2427467040}]},"public_addr":"172.21.15.89:6827/2427467040","cluster_addr":"172.21.15.89:6829/2427467040","heartbeat_back_addr":"172.21.15.89:6833/2427467040","heartbeat_front_addr":"172.21.15.89:6831/2427467040","state":["exists","up"]},{"osd":10,"uuid":"d8f29890-3064-4b67-b649-467a0ac86db7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6825","nonce":2328678744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6827","nonce":2328678744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6831","nonce":2328678744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6829","nonce":2328678744}]},"public_addr":"172.21.15.111:6825/2328678744","cluster_addr":"172.21.15.111:6827/2328678744","heartbeat_back_addr":"172.21.15.111:6831/2328678744","heartbeat_front_addr":"172.21.15.111:6829/2328678744","state":["exists","up"]},{"osd":11,"uuid":"92182c73-f139-47ca-a0ef-5046ae9b2310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6825","nonce":3130508827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6827","nonce":3130508827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6831","nonce":3130508827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6829","nonce":3130508827}]},"public_addr":"172.21.15.149:6825/3130508827","cluster_addr":"172.21.15.149:6827/3130508827","heartbeat_back_addr":"172.21.15.149:6831/3130508827","heartbeat_front_addr":"172.21.15.149:6829/3130508827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:09.031810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:33.695124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:58.065510+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:27.592838+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:53.275289+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:18.305677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:47.781535+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:12.889512+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:37.744193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:07.718408+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:33.391027+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:59.013250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/850669510":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/3541027428":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6801/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6800/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/2396468787":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/1609720980":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/995019144":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6801/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6800/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/2179977257":"2024-09-11T08:29:14.554869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:38:16.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.802+0000 7f412fc00640 1 -- 172.21.15.89:0/3687173591 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f410c0a2c90 msgr2=0x7f410c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:16.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.802+0000 7f412fc00640 1 --2- 172.21.15.89:0/3687173591 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f410c0a2c90 0x7f410c0a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f412810f8b0 tx=0x7f411000b410 comp rx=0 tx=0).stop 2024-09-10T08:38:16.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.802+0000 7f412fc00640 1 -- 172.21.15.89:0/3687173591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4128074620 msgr2=0x7f412819a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:16.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.802+0000 7f412fc00640 1 --2- 172.21.15.89:0/3687173591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4128074620 0x7f412819a710 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4118006010 tx=0x7f41180042f0 comp rx=0 tx=0).stop 2024-09-10T08:38:16.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.802+0000 7f412fc00640 1 -- 172.21.15.89:0/3687173591 shutdown_connections 2024-09-10T08:38:16.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.802+0000 7f412fc00640 1 --2- 172.21.15.89:0/3687173591 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f410c0a2c90 0x7f410c0a5150 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:16.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.802+0000 7f412fc00640 1 --2- 172.21.15.89:0/3687173591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f412810e3c0 0x7f412819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:16.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.802+0000 7f412fc00640 1 --2- 172.21.15.89:0/3687173591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4128074f40 0x7f412819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:16.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.802+0000 7f412fc00640 1 --2- 172.21.15.89:0/3687173591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4128074620 0x7f412819a710 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:16.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.802+0000 7f412fc00640 1 -- 172.21.15.89:0/3687173591 >> 172.21.15.89:0/3687173591 conn(0x7f41280fd8c0 msgr2=0x7f41280fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:16.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.803+0000 7f412fc00640 1 -- 172.21.15.89:0/3687173591 shutdown_connections 2024-09-10T08:38:16.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:16.803+0000 7f412fc00640 1 -- 172.21.15.89:0/3687173591 wait complete. 2024-09-10T08:38:17.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:17 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3687173591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:38:17.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:17 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3687173591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:38:17.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:17 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3687173591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:38:17.654 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T08:38:17.655 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T08:38:17.655 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T08:38:17.655 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T08:38:17.656 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T08:38:17.656 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T08:38:17.657 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T08:38:17.657 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T08:38:17.658 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-10T08:38:17.658 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-10T08:38:17.658 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-10T08:38:17.659 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-10T08:38:18.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:18 smithi149 ceph-mon[36167]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-10T08:38:18.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:18 smithi111 ceph-mon[35220]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-10T08:38:18.552 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:18 smithi089 ceph-mon[30757]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-10T08:38:18.615 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:18.649 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:18.649 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:18.719 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:18.727 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:19.257 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:19.261 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:19.268 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:19.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.711+0000 7f425c400640 1 -- 172.21.15.89:0/4007266983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4254104010 msgr2=0x7f42541043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:19.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.711+0000 7f425c400640 1 --2- 172.21.15.89:0/4007266983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4254104010 0x7f42541043f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4248009ca0 tx=0x7f424802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:38:19.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.712+0000 7f425c400640 1 -- 172.21.15.89:0/4007266983 shutdown_connections 2024-09-10T08:38:19.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.712+0000 7f425c400640 1 --2- 172.21.15.89:0/4007266983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4254109140 0x7f4254110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:19.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.712+0000 7f425c400640 1 --2- 172.21.15.89:0/4007266983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42541049c0 0x7f4254108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:19.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.712+0000 7f425c400640 1 --2- 172.21.15.89:0/4007266983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4254104010 0x7f42541043f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:19.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.712+0000 7f425c400640 1 -- 172.21.15.89:0/4007266983 >> 172.21.15.89:0/4007266983 conn(0x7f42540ffa50 msgr2=0x7f4254101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:19.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.712+0000 7f425c400640 1 -- 172.21.15.89:0/4007266983 shutdown_connections 2024-09-10T08:38:19.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.712+0000 7f425c400640 1 -- 172.21.15.89:0/4007266983 wait complete. 2024-09-10T08:38:19.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.713+0000 7f425c400640 1 Processor -- start 2024-09-10T08:38:19.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.713+0000 7f425c400640 1 -- start start 2024-09-10T08:38:19.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.714+0000 7f425c400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4254104010 0x7f425419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:19.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.714+0000 7f425c400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42541049c0 0x7f425419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:19.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.714+0000 7f425c400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4254109140 0x7f425419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:19.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.714+0000 7f425c400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f42541141c0 con 0x7f42541049c0 2024-09-10T08:38:19.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.714+0000 7f425c400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4254114040 con 0x7f4254104010 2024-09-10T08:38:19.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.714+0000 7f425c400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4254114340 con 0x7f4254109140 2024-09-10T08:38:19.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.714+0000 7f4259a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4254104010 0x7f425419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:19.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.714+0000 7f4259a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4254104010 0x7f425419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:56854/0 (socket says 172.21.15.89:56854) 2024-09-10T08:38:19.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.714+0000 7f4259a00640 1 -- 172.21.15.89:0/1120978925 learned_addr learned my addr 172.21.15.89:0/1120978925 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:19.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.714+0000 7f4259000640 1 --2- 172.21.15.89:0/1120978925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42541049c0 0x7f425419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:19.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.714+0000 7f425a400640 1 --2- 172.21.15.89:0/1120978925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4254109140 0x7f425419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:19.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.714+0000 7f4259a00640 1 -- 172.21.15.89:0/1120978925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4254109140 msgr2=0x7f425419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:19.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.714+0000 7f4259a00640 1 --2- 172.21.15.89:0/1120978925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4254109140 0x7f425419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:19.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.714+0000 7f4259a00640 1 -- 172.21.15.89:0/1120978925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42541049c0 msgr2=0x7f425419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:19.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.714+0000 7f4259a00640 1 --2- 172.21.15.89:0/1120978925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42541049c0 0x7f425419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:19.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.714+0000 7f4259a00640 1 -- 172.21.15.89:0/1120978925 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425419f5d0 con 0x7f4254104010 2024-09-10T08:38:19.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.715+0000 7f4259a00640 1 --2- 172.21.15.89:0/1120978925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4254104010 0x7f425419a620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4248005e30 tx=0x7f4248003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:19.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.715+0000 7f4252000640 1 -- 172.21.15.89:0/1120978925 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42480316d0 con 0x7f4254104010 2024-09-10T08:38:19.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.715+0000 7f425c400640 1 -- 172.21.15.89:0/1120978925 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f425419f860 con 0x7f4254104010 2024-09-10T08:38:19.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.715+0000 7f4252000640 1 -- 172.21.15.89:0/1120978925 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4248031870 con 0x7f4254104010 2024-09-10T08:38:19.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.715+0000 7f4252000640 1 -- 172.21.15.89:0/1120978925 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424802fb70 con 0x7f4254104010 2024-09-10T08:38:19.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.715+0000 7f425c400640 1 -- 172.21.15.89:0/1120978925 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42541a71a0 con 0x7f4254104010 2024-09-10T08:38:19.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.716+0000 7f425c400640 1 -- 172.21.15.89:0/1120978925 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4254062940 con 0x7f4254104010 2024-09-10T08:38:19.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.718+0000 7f4252000640 1 -- 172.21.15.89:0/1120978925 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f424802fd10 con 0x7f4254104010 2024-09-10T08:38:19.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.719+0000 7f4252000640 1 --2- 172.21.15.89:0/1120978925 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f423c0a2c10 0x7f423c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:19.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.719+0000 7f4252000640 1 -- 172.21.15.89:0/1120978925 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f42480f0b10 con 0x7f4254104010 2024-09-10T08:38:19.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.719+0000 7f4259000640 1 --2- 172.21.15.89:0/1120978925 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f423c0a2c10 0x7f423c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:19.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.720+0000 7f4252000640 1 --2- 172.21.15.89:0/1120978925 >> [v2:172.21.15.149:6824/3130508827,v1:172.21.15.149:6825/3130508827] conn(0x7f423c0adcb0 0x7f423c0b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:19.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.720+0000 7f4252000640 1 -- 172.21.15.89:0/1120978925 --> [v2:172.21.15.149:6824/3130508827,v1:172.21.15.149:6825/3130508827] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4248050ea0 con 0x7f423c0adcb0 2024-09-10T08:38:19.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.720+0000 7f4252000640 1 -- 172.21.15.89:0/1120978925 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f42480f0ec0 con 0x7f4254104010 2024-09-10T08:38:19.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.720+0000 7f4259000640 1 --2- 172.21.15.89:0/1120978925 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f423c0a2c10 0x7f423c0a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f42540ffe70 tx=0x7f424000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:19.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.720+0000 7f425a400640 1 --2- 172.21.15.89:0/1120978925 >> [v2:172.21.15.149:6824/3130508827,v1:172.21.15.149:6825/3130508827] conn(0x7f423c0adcb0 0x7f423c0b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:19.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.721+0000 7f425a400640 1 --2- 172.21.15.89:0/1120978925 >> [v2:172.21.15.149:6824/3130508827,v1:172.21.15.149:6825/3130508827] conn(0x7f423c0adcb0 0x7f423c0b0150 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:19.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.722+0000 7f4252000640 1 -- 172.21.15.89:0/1120978925 <== osd.11 v2:172.21.15.149:6824/3130508827 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4248050ea0 con 0x7f423c0adcb0 2024-09-10T08:38:19.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.736+0000 7f425c400640 1 -- 172.21.15.89:0/1120978925 --> [v2:172.21.15.149:6824/3130508827,v1:172.21.15.149:6825/3130508827] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4254062d40 con 0x7f423c0adcb0 2024-09-10T08:38:19.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.736+0000 7f4252000640 1 -- 172.21.15.89:0/1120978925 <== osd.11 v2:172.21.15.149:6824/3130508827 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4254062d40 con 0x7f423c0adcb0 2024-09-10T08:38:19.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.737+0000 7f425c400640 1 -- 172.21.15.89:0/1120978925 >> [v2:172.21.15.149:6824/3130508827,v1:172.21.15.149:6825/3130508827] conn(0x7f423c0adcb0 msgr2=0x7f423c0b0150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:19.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.737+0000 7f425c400640 1 --2- 172.21.15.89:0/1120978925 >> [v2:172.21.15.149:6824/3130508827,v1:172.21.15.149:6825/3130508827] conn(0x7f423c0adcb0 0x7f423c0b0150 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:19.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.737+0000 7f425c400640 1 -- 172.21.15.89:0/1120978925 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f423c0a2c10 msgr2=0x7f423c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:19.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.737+0000 7f425c400640 1 --2- 172.21.15.89:0/1120978925 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f423c0a2c10 0x7f423c0a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f42540ffe70 tx=0x7f424000b410 comp rx=0 tx=0).stop 2024-09-10T08:38:19.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.737+0000 7f425c400640 1 -- 172.21.15.89:0/1120978925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4254104010 msgr2=0x7f425419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:19.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.737+0000 7f425c400640 1 --2- 172.21.15.89:0/1120978925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4254104010 0x7f425419a620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4248005e30 tx=0x7f4248003f90 comp rx=0 tx=0).stop 2024-09-10T08:38:19.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.737+0000 7f425a400640 1 -- 172.21.15.89:0/1120978925 reap_dead start 2024-09-10T08:38:19.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.737+0000 7f425c400640 1 -- 172.21.15.89:0/1120978925 shutdown_connections 2024-09-10T08:38:19.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.737+0000 7f425c400640 1 -- 172.21.15.89:0/1120978925 >> 172.21.15.89:0/1120978925 conn(0x7f42540ffa50 msgr2=0x7f4254100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:19.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.737+0000 7f425c400640 1 -- 172.21.15.89:0/1120978925 shutdown_connections 2024-09-10T08:38:19.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:19.737+0000 7f425c400640 1 -- 172.21.15.89:0/1120978925 wait complete. 2024-09-10T08:38:20.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.170+0000 7f3d87a00640 1 -- 172.21.15.89:0/2463863639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d80102790 msgr2=0x7f3d801067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:20.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.170+0000 7f3d87a00640 1 --2- 172.21.15.89:0/2463863639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d80102790 0x7f3d801067e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009900 tx=0x7f3d6802a410 comp rx=0 tx=0).stop 2024-09-10T08:38:20.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.170+0000 7f3d87a00640 1 -- 172.21.15.89:0/2463863639 shutdown_connections 2024-09-10T08:38:20.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.170+0000 7f3d87a00640 1 --2- 172.21.15.89:0/2463863639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d80106f10 0x7f3d8010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:20.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.170+0000 7f3d87a00640 1 --2- 172.21.15.89:0/2463863639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d80102790 0x7f3d801067e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.170+0000 7f3d87a00640 1 --2- 172.21.15.89:0/2463863639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d80101de0 0x7f3d801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.170+0000 7f3d87a00640 1 -- 172.21.15.89:0/2463863639 >> 172.21.15.89:0/2463863639 conn(0x7f3d800fd8c0 msgr2=0x7f3d800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.170+0000 7f3d87a00640 1 -- 172.21.15.89:0/2463863639 shutdown_connections 2024-09-10T08:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.170+0000 7f3d87a00640 1 -- 172.21.15.89:0/2463863639 wait complete. 2024-09-10T08:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.172+0000 7f3d87a00640 1 Processor -- start 2024-09-10T08:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.172+0000 7f3d87a00640 1 -- start start 2024-09-10T08:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.172+0000 7f3d87a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d80101de0 0x7f3d80198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.172+0000 7f3d87a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d80102790 0x7f3d801989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.172+0000 7f3d87a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d80106f10 0x7f3d8019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.172+0000 7f3d87a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3d80111fc0 con 0x7f3d80101de0 2024-09-10T08:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.172+0000 7f3d87a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3d80111e40 con 0x7f3d80102790 2024-09-10T08:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.172+0000 7f3d87a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3d80112140 con 0x7f3d80106f10 2024-09-10T08:38:20.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.173+0000 7f3d85000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d80101de0 0x7f3d80198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.173+0000 7f3d85000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d80101de0 0x7f3d80198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54120/0 (socket says 172.21.15.89:54120) 2024-09-10T08:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.173+0000 7f3d85000640 1 -- 172.21.15.89:0/553170544 learned_addr learned my addr 172.21.15.89:0/553170544 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.173+0000 7f3d85a00640 1 --2- 172.21.15.89:0/553170544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d80106f10 0x7f3d8019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.173+0000 7f3d77e00640 1 --2- 172.21.15.89:0/553170544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d80102790 0x7f3d801989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.173+0000 7f3d85000640 1 -- 172.21.15.89:0/553170544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d80106f10 msgr2=0x7f3d8019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.173+0000 7f3d85000640 1 --2- 172.21.15.89:0/553170544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d80106f10 0x7f3d8019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.173+0000 7f3d85000640 1 -- 172.21.15.89:0/553170544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d80102790 msgr2=0x7f3d801989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.173+0000 7f3d85000640 1 --2- 172.21.15.89:0/553170544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d80102790 0x7f3d801989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.173+0000 7f3d85000640 1 -- 172.21.15.89:0/553170544 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8019d430 con 0x7f3d80101de0 2024-09-10T08:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.173+0000 7f3d85000640 1 --2- 172.21.15.89:0/553170544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d80101de0 0x7f3d80198480 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000e9f0 tx=0x7f3d7000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.174+0000 7f3d7e000640 1 -- 172.21.15.89:0/553170544 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7000cd80 con 0x7f3d80101de0 2024-09-10T08:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.174+0000 7f3d7e000640 1 -- 172.21.15.89:0/553170544 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d70010910 con 0x7f3d80101de0 2024-09-10T08:38:20.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.174+0000 7f3d87a00640 1 -- 172.21.15.89:0/553170544 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d8019d720 con 0x7f3d80101de0 2024-09-10T08:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.174+0000 7f3d7e000640 1 -- 172.21.15.89:0/553170544 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70018990 con 0x7f3d80101de0 2024-09-10T08:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.174+0000 7f3d87a00640 1 -- 172.21.15.89:0/553170544 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d801a5000 con 0x7f3d80101de0 2024-09-10T08:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.175+0000 7f3d87a00640 1 -- 172.21.15.89:0/553170544 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3d80062980 con 0x7f3d80101de0 2024-09-10T08:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.177+0000 7f3d7e000640 1 -- 172.21.15.89:0/553170544 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3d70010ac0 con 0x7f3d80101de0 2024-09-10T08:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.178+0000 7f3d7e000640 1 --2- 172.21.15.89:0/553170544 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3d640a2c10 0x7f3d640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.178+0000 7f3d7e000640 1 -- 172.21.15.89:0/553170544 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f3d700cffc0 con 0x7f3d80101de0 2024-09-10T08:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.178+0000 7f3d77e00640 1 --2- 172.21.15.89:0/553170544 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3d640a2c10 0x7f3d640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.178+0000 7f3d7e000640 1 --2- 172.21.15.89:0/553170544 >> [v2:172.21.15.149:6816/2955172869,v1:172.21.15.149:6817/2955172869] conn(0x7f3d640adde0 0x7f3d640b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.178+0000 7f3d7e000640 1 -- 172.21.15.89:0/553170544 --> [v2:172.21.15.149:6816/2955172869,v1:172.21.15.149:6817/2955172869] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3d70022e50 con 0x7f3d640adde0 2024-09-10T08:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.178+0000 7f3d7e000640 1 -- 172.21.15.89:0/553170544 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f3d700d0370 con 0x7f3d80101de0 2024-09-10T08:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.179+0000 7f3d85a00640 1 --2- 172.21.15.89:0/553170544 >> [v2:172.21.15.149:6816/2955172869,v1:172.21.15.149:6817/2955172869] conn(0x7f3d640adde0 0x7f3d640b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:20.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.179+0000 7f3d77e00640 1 --2- 172.21.15.89:0/553170544 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3d640a2c10 0x7f3d640a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3d68000c00 tx=0x7f3d68005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:20.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.179+0000 7f3d85a00640 1 --2- 172.21.15.89:0/553170544 >> [v2:172.21.15.149:6816/2955172869,v1:172.21.15.149:6817/2955172869] conn(0x7f3d640adde0 0x7f3d640b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:20.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.180+0000 7f3d7e000640 1 -- 172.21.15.89:0/553170544 <== osd.8 v2:172.21.15.149:6816/2955172869 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3d70022e50 con 0x7f3d640adde0 2024-09-10T08:38:20.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.196+0000 7f3d87a00640 1 -- 172.21.15.89:0/553170544 --> [v2:172.21.15.149:6816/2955172869,v1:172.21.15.149:6817/2955172869] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3d80062d80 con 0x7f3d640adde0 2024-09-10T08:38:20.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.197+0000 7f3d7e000640 1 -- 172.21.15.89:0/553170544 <== osd.8 v2:172.21.15.149:6816/2955172869 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3d80062d80 con 0x7f3d640adde0 2024-09-10T08:38:20.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.197+0000 7f3d87a00640 1 -- 172.21.15.89:0/553170544 >> [v2:172.21.15.149:6816/2955172869,v1:172.21.15.149:6817/2955172869] conn(0x7f3d640adde0 msgr2=0x7f3d640b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:20.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.197+0000 7f3d87a00640 1 --2- 172.21.15.89:0/553170544 >> [v2:172.21.15.149:6816/2955172869,v1:172.21.15.149:6817/2955172869] conn(0x7f3d640adde0 0x7f3d640b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.198+0000 7f3d87a00640 1 -- 172.21.15.89:0/553170544 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3d640a2c10 msgr2=0x7f3d640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.198+0000 7f3d87a00640 1 --2- 172.21.15.89:0/553170544 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3d640a2c10 0x7f3d640a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3d68000c00 tx=0x7f3d68005a20 comp rx=0 tx=0).stop 2024-09-10T08:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.198+0000 7f3d87a00640 1 -- 172.21.15.89:0/553170544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d80101de0 msgr2=0x7f3d80198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.198+0000 7f3d87a00640 1 --2- 172.21.15.89:0/553170544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d80101de0 0x7f3d80198480 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000e9f0 tx=0x7f3d7000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.198+0000 7f3d85a00640 1 -- 172.21.15.89:0/553170544 reap_dead start 2024-09-10T08:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.198+0000 7f3d87a00640 1 -- 172.21.15.89:0/553170544 shutdown_connections 2024-09-10T08:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.198+0000 7f3d87a00640 1 -- 172.21.15.89:0/553170544 >> 172.21.15.89:0/553170544 conn(0x7f3d800fd8c0 msgr2=0x7f3d800febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.198+0000 7f3d87a00640 1 -- 172.21.15.89:0/553170544 shutdown_connections 2024-09-10T08:38:20.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:20.198+0000 7f3d87a00640 1 -- 172.21.15.89:0/553170544 wait complete. 2024-09-10T08:38:20.199 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:20.305 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:20.306 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:20.410 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:20.693 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:20 smithi089 ceph-mon[30757]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-10T08:38:20.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:20 smithi149 ceph-mon[36167]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-10T08:38:21.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:20 smithi111 ceph-mon[35220]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-10T08:38:21.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.366+0000 7f9cdf000640 1 -- 172.21.15.89:0/306648830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cd8109120 msgr2=0x7f9cd8110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:21.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.366+0000 7f9cdf000640 1 --2- 172.21.15.89:0/306648830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cd8109120 0x7f9cd8110ca0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009ca0 tx=0x7f9cd002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:38:21.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.367+0000 7f9cdf000640 1 -- 172.21.15.89:0/306648830 shutdown_connections 2024-09-10T08:38:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.367+0000 7f9cdf000640 1 --2- 172.21.15.89:0/306648830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cd8109120 0x7f9cd8110ca0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.367+0000 7f9cdf000640 1 --2- 172.21.15.89:0/306648830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cd81049a0 0x7f9cd81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.367+0000 7f9cdf000640 1 --2- 172.21.15.89:0/306648830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cd8103ff0 0x7f9cd81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.367+0000 7f9cdf000640 1 -- 172.21.15.89:0/306648830 >> 172.21.15.89:0/306648830 conn(0x7f9cd80ffa90 msgr2=0x7f9cd8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.367+0000 7f9cdf000640 1 -- 172.21.15.89:0/306648830 shutdown_connections 2024-09-10T08:38:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.367+0000 7f9cdf000640 1 -- 172.21.15.89:0/306648830 wait complete. 2024-09-10T08:38:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.368+0000 7f9cdf000640 1 Processor -- start 2024-09-10T08:38:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.368+0000 7f9cdf000640 1 -- start start 2024-09-10T08:38:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.368+0000 7f9cdf000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cd8103ff0 0x7f9cd819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.368+0000 7f9cdf000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cd81049a0 0x7f9cd819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.368+0000 7f9cdf000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cd8109120 0x7f9cd819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.368+0000 7f9cdf000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd81141a0 con 0x7f9cd81049a0 2024-09-10T08:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.368+0000 7f9cdf000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd8114020 con 0x7f9cd8109120 2024-09-10T08:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.368+0000 7f9cdf000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd8114320 con 0x7f9cd8103ff0 2024-09-10T08:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.369+0000 7f9cd7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cd81049a0 0x7f9cd819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.369+0000 7f9cd7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cd8103ff0 0x7f9cd819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.369+0000 7f9cdd000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cd8109120 0x7f9cd819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.369+0000 7f9cd7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cd81049a0 0x7f9cd819ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54132/0 (socket says 172.21.15.89:54132) 2024-09-10T08:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.369+0000 7f9cd7400640 1 -- 172.21.15.89:0/461394091 learned_addr learned my addr 172.21.15.89:0/461394091 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.369+0000 7f9cd7400640 1 -- 172.21.15.89:0/461394091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cd8103ff0 msgr2=0x7f9cd819a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.369+0000 7f9cd7400640 1 --2- 172.21.15.89:0/461394091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cd8103ff0 0x7f9cd819a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.369+0000 7f9cd7400640 1 -- 172.21.15.89:0/461394091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cd8109120 msgr2=0x7f9cd819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.369+0000 7f9cd7400640 1 --2- 172.21.15.89:0/461394091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cd8109120 0x7f9cd819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.369+0000 7f9cd7400640 1 -- 172.21.15.89:0/461394091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd819f590 con 0x7f9cd81049a0 2024-09-10T08:38:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.369+0000 7f9cdd000640 1 --2- 172.21.15.89:0/461394091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cd8109120 0x7f9cd819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:38:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.369+0000 7f9cd7400640 1 --2- 172.21.15.89:0/461394091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cd81049a0 0x7f9cd819ab20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400e9f0 tx=0x7f9cc400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.370+0000 7f9cd4c00640 1 -- 172.21.15.89:0/461394091 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc400cd80 con 0x7f9cd81049a0 2024-09-10T08:38:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.370+0000 7f9cd4c00640 1 -- 172.21.15.89:0/461394091 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cc4010910 con 0x7f9cd81049a0 2024-09-10T08:38:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.370+0000 7f9cdf000640 1 -- 172.21.15.89:0/461394091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd819f880 con 0x7f9cd81049a0 2024-09-10T08:38:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.370+0000 7f9cd4c00640 1 -- 172.21.15.89:0/461394091 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4018990 con 0x7f9cd81049a0 2024-09-10T08:38:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.370+0000 7f9cdf000640 1 -- 172.21.15.89:0/461394091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd81a7160 con 0x7f9cd81049a0 2024-09-10T08:38:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.371+0000 7f9cdf000640 1 -- 172.21.15.89:0/461394091 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9cd8101020 con 0x7f9cd81049a0 2024-09-10T08:38:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.373+0000 7f9cd4c00640 1 -- 172.21.15.89:0/461394091 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9cc4010ac0 con 0x7f9cd81049a0 2024-09-10T08:38:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.375+0000 7f9cd4c00640 1 --2- 172.21.15.89:0/461394091 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9cbc0a2c10 0x7f9cbc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.375+0000 7f9cd4c00640 1 -- 172.21.15.89:0/461394091 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f9cc40cf820 con 0x7f9cd81049a0 2024-09-10T08:38:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.375+0000 7f9cd7e00640 1 --2- 172.21.15.89:0/461394091 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9cbc0a2c10 0x7f9cbc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.375+0000 7f9cd4c00640 1 --2- 172.21.15.89:0/461394091 >> [v2:172.21.15.149:6808/1256148461,v1:172.21.15.149:6809/1256148461] conn(0x7f9cbc0adde0 0x7f9cbc0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.375+0000 7f9cd4c00640 1 -- 172.21.15.89:0/461394091 --> [v2:172.21.15.149:6808/1256148461,v1:172.21.15.149:6809/1256148461] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9cbc0b0950 con 0x7f9cbc0adde0 2024-09-10T08:38:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.375+0000 7f9cd4c00640 1 -- 172.21.15.89:0/461394091 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f9cc40d6050 con 0x7f9cd81049a0 2024-09-10T08:38:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.376+0000 7f9cdd000640 1 --2- 172.21.15.89:0/461394091 >> [v2:172.21.15.149:6808/1256148461,v1:172.21.15.149:6809/1256148461] conn(0x7f9cbc0adde0 0x7f9cbc0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.376+0000 7f9cd7e00640 1 --2- 172.21.15.89:0/461394091 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9cbc0a2c10 0x7f9cbc0a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009bd0 tx=0x7f9cc8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.376+0000 7f9cdd000640 1 --2- 172.21.15.89:0/461394091 >> [v2:172.21.15.149:6808/1256148461,v1:172.21.15.149:6809/1256148461] conn(0x7f9cbc0adde0 0x7f9cbc0b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.377+0000 7f9cd4c00640 1 -- 172.21.15.89:0/461394091 <== osd.5 v2:172.21.15.149:6808/1256148461 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9cbc0b0950 con 0x7f9cbc0adde0 2024-09-10T08:38:21.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.392+0000 7f9cdf000640 1 -- 172.21.15.89:0/461394091 --> [v2:172.21.15.149:6808/1256148461,v1:172.21.15.149:6809/1256148461] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9cd81054d0 con 0x7f9cbc0adde0 2024-09-10T08:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.392+0000 7f9cd4c00640 1 -- 172.21.15.89:0/461394091 <== osd.5 v2:172.21.15.149:6808/1256148461 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9cd81054d0 con 0x7f9cbc0adde0 2024-09-10T08:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.393+0000 7f9cdf000640 1 -- 172.21.15.89:0/461394091 >> [v2:172.21.15.149:6808/1256148461,v1:172.21.15.149:6809/1256148461] conn(0x7f9cbc0adde0 msgr2=0x7f9cbc0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.393+0000 7f9cdf000640 1 --2- 172.21.15.89:0/461394091 >> [v2:172.21.15.149:6808/1256148461,v1:172.21.15.149:6809/1256148461] conn(0x7f9cbc0adde0 0x7f9cbc0b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.393+0000 7f9cdf000640 1 -- 172.21.15.89:0/461394091 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9cbc0a2c10 msgr2=0x7f9cbc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.393+0000 7f9cdf000640 1 --2- 172.21.15.89:0/461394091 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9cbc0a2c10 0x7f9cbc0a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009bd0 tx=0x7f9cc8005f90 comp rx=0 tx=0).stop 2024-09-10T08:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.393+0000 7f9cdf000640 1 -- 172.21.15.89:0/461394091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cd81049a0 msgr2=0x7f9cd819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.393+0000 7f9cdf000640 1 --2- 172.21.15.89:0/461394091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9cd81049a0 0x7f9cd819ab20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400e9f0 tx=0x7f9cc400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.393+0000 7f9cdd000640 1 -- 172.21.15.89:0/461394091 reap_dead start 2024-09-10T08:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.393+0000 7f9cdf000640 1 -- 172.21.15.89:0/461394091 shutdown_connections 2024-09-10T08:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.393+0000 7f9cdf000640 1 -- 172.21.15.89:0/461394091 >> 172.21.15.89:0/461394091 conn(0x7f9cd80ffa90 msgr2=0x7f9cd81005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.393+0000 7f9cdf000640 1 -- 172.21.15.89:0/461394091 shutdown_connections 2024-09-10T08:38:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.393+0000 7f9cdf000640 1 -- 172.21.15.89:0/461394091 wait complete. 2024-09-10T08:38:21.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.963+0000 7f634f800640 1 -- 172.21.15.89:0/915309074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6348103ff0 msgr2=0x7f63481043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:21.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.963+0000 7f634f800640 1 --2- 172.21.15.89:0/915309074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6348103ff0 0x7f63481043d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f633c009860 tx=0x7f633c02a340 comp rx=0 tx=0).stop 2024-09-10T08:38:21.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.963+0000 7f634f800640 1 -- 172.21.15.89:0/915309074 shutdown_connections 2024-09-10T08:38:21.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.963+0000 7f634f800640 1 --2- 172.21.15.89:0/915309074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6348109120 0x7f6348110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:21.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.963+0000 7f634f800640 1 --2- 172.21.15.89:0/915309074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63481049a0 0x7f63481089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:21.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.963+0000 7f634f800640 1 --2- 172.21.15.89:0/915309074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6348103ff0 0x7f63481043d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:21.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.963+0000 7f634f800640 1 -- 172.21.15.89:0/915309074 >> 172.21.15.89:0/915309074 conn(0x7f63480ffa90 msgr2=0x7f6348101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:21.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.963+0000 7f634f800640 1 -- 172.21.15.89:0/915309074 shutdown_connections 2024-09-10T08:38:21.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.963+0000 7f634f800640 1 -- 172.21.15.89:0/915309074 wait complete. 2024-09-10T08:38:21.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.964+0000 7f634f800640 1 Processor -- start 2024-09-10T08:38:21.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.964+0000 7f634f800640 1 -- start start 2024-09-10T08:38:21.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.964+0000 7f634f800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6348103ff0 0x7f634819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:21.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.964+0000 7f634f800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63481049a0 0x7f634819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:21.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.964+0000 7f634f800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6348109120 0x7f634819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:21.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.964+0000 7f634f800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6348114150 con 0x7f6348103ff0 2024-09-10T08:38:21.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.964+0000 7f634f800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6348113fd0 con 0x7f63481049a0 2024-09-10T08:38:21.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.964+0000 7f634f800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f63481142d0 con 0x7f6348109120 2024-09-10T08:38:21.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.965+0000 7f634ce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6348103ff0 0x7f634819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:21.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.965+0000 7f634ce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6348103ff0 0x7f634819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54160/0 (socket says 172.21.15.89:54160) 2024-09-10T08:38:21.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.965+0000 7f634ce00640 1 -- 172.21.15.89:0/3740405858 learned_addr learned my addr 172.21.15.89:0/3740405858 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:21.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.965+0000 7f6347e00640 1 --2- 172.21.15.89:0/3740405858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63481049a0 0x7f634819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:21.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.965+0000 7f634d800640 1 --2- 172.21.15.89:0/3740405858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6348109120 0x7f634819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:21.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.965+0000 7f634ce00640 1 -- 172.21.15.89:0/3740405858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6348109120 msgr2=0x7f634819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:21.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.965+0000 7f634ce00640 1 --2- 172.21.15.89:0/3740405858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6348109120 0x7f634819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:21.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.965+0000 7f634ce00640 1 -- 172.21.15.89:0/3740405858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63481049a0 msgr2=0x7f634819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:21.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.965+0000 7f634ce00640 1 --2- 172.21.15.89:0/3740405858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63481049a0 0x7f634819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:21.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.965+0000 7f634ce00640 1 -- 172.21.15.89:0/3740405858 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634819f5c0 con 0x7f6348103ff0 2024-09-10T08:38:21.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.965+0000 7f634ce00640 1 --2- 172.21.15.89:0/3740405858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6348103ff0 0x7f634819a610 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f633c035040 tx=0x7f633c033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:21.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.966+0000 7f6345600640 1 -- 172.21.15.89:0/3740405858 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c037040 con 0x7f6348103ff0 2024-09-10T08:38:21.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.966+0000 7f6345600640 1 -- 172.21.15.89:0/3740405858 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f633c040070 con 0x7f6348103ff0 2024-09-10T08:38:21.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.966+0000 7f6345600640 1 -- 172.21.15.89:0/3740405858 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c0039b0 con 0x7f6348103ff0 2024-09-10T08:38:21.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.966+0000 7f634f800640 1 -- 172.21.15.89:0/3740405858 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634819f850 con 0x7f6348103ff0 2024-09-10T08:38:21.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.966+0000 7f634f800640 1 -- 172.21.15.89:0/3740405858 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63481a7190 con 0x7f6348103ff0 2024-09-10T08:38:21.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.967+0000 7f634f800640 1 -- 172.21.15.89:0/3740405858 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6348062940 con 0x7f6348103ff0 2024-09-10T08:38:21.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.969+0000 7f6345600640 1 -- 172.21.15.89:0/3740405858 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f633c0305e0 con 0x7f6348103ff0 2024-09-10T08:38:21.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.970+0000 7f6345600640 1 --2- 172.21.15.89:0/3740405858 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f63300a2c10 0x7f63300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:21.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.970+0000 7f6345600640 1 -- 172.21.15.89:0/3740405858 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f633c0eff20 con 0x7f6348103ff0 2024-09-10T08:38:21.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.970+0000 7f6347e00640 1 --2- 172.21.15.89:0/3740405858 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f63300a2c10 0x7f63300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:21.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.970+0000 7f6345600640 1 --2- 172.21.15.89:0/3740405858 >> [v2:172.21.15.89:6810/1405773950,v1:172.21.15.89:6811/1405773950] conn(0x7f63300adcb0 0x7f63300b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:21.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.970+0000 7f6345600640 1 -- 172.21.15.89:0/3740405858 --> [v2:172.21.15.89:6810/1405773950,v1:172.21.15.89:6811/1405773950] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f633c001d30 con 0x7f63300adcb0 2024-09-10T08:38:21.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.971+0000 7f6345600640 1 -- 172.21.15.89:0/3740405858 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f633c0f02d0 con 0x7f6348103ff0 2024-09-10T08:38:21.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.971+0000 7f634d800640 1 --2- 172.21.15.89:0/3740405858 >> [v2:172.21.15.89:6810/1405773950,v1:172.21.15.89:6811/1405773950] conn(0x7f63300adcb0 0x7f63300b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:21.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.971+0000 7f6347e00640 1 --2- 172.21.15.89:0/3740405858 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f63300a2c10 0x7f63300a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f63480fff90 tx=0x7f6334009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:21.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.971+0000 7f634d800640 1 --2- 172.21.15.89:0/3740405858 >> [v2:172.21.15.89:6810/1405773950,v1:172.21.15.89:6811/1405773950] conn(0x7f63300adcb0 0x7f63300b0150 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:21.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.972+0000 7f6345600640 1 -- 172.21.15.89:0/3740405858 <== osd.3 v2:172.21.15.89:6810/1405773950 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f633c001d30 con 0x7f63300adcb0 2024-09-10T08:38:21.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.986+0000 7f634f800640 1 -- 172.21.15.89:0/3740405858 --> [v2:172.21.15.89:6810/1405773950,v1:172.21.15.89:6811/1405773950] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6348062dc0 con 0x7f63300adcb0 2024-09-10T08:38:21.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.987+0000 7f6345600640 1 -- 172.21.15.89:0/3740405858 <== osd.3 v2:172.21.15.89:6810/1405773950 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6348062dc0 con 0x7f63300adcb0 2024-09-10T08:38:21.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.987+0000 7f634f800640 1 -- 172.21.15.89:0/3740405858 >> [v2:172.21.15.89:6810/1405773950,v1:172.21.15.89:6811/1405773950] conn(0x7f63300adcb0 msgr2=0x7f63300b0150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:21.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.987+0000 7f634f800640 1 --2- 172.21.15.89:0/3740405858 >> [v2:172.21.15.89:6810/1405773950,v1:172.21.15.89:6811/1405773950] conn(0x7f63300adcb0 0x7f63300b0150 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:21.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.987+0000 7f634f800640 1 -- 172.21.15.89:0/3740405858 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f63300a2c10 msgr2=0x7f63300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:21.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.987+0000 7f634f800640 1 --2- 172.21.15.89:0/3740405858 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f63300a2c10 0x7f63300a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f63480fff90 tx=0x7f6334009260 comp rx=0 tx=0).stop 2024-09-10T08:38:21.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.987+0000 7f634f800640 1 -- 172.21.15.89:0/3740405858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6348103ff0 msgr2=0x7f634819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:21.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.987+0000 7f634f800640 1 --2- 172.21.15.89:0/3740405858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6348103ff0 0x7f634819a610 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f633c035040 tx=0x7f633c033b60 comp rx=0 tx=0).stop 2024-09-10T08:38:21.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.987+0000 7f634d800640 1 -- 172.21.15.89:0/3740405858 reap_dead start 2024-09-10T08:38:21.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.987+0000 7f634f800640 1 -- 172.21.15.89:0/3740405858 shutdown_connections 2024-09-10T08:38:21.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.987+0000 7f634f800640 1 -- 172.21.15.89:0/3740405858 >> 172.21.15.89:0/3740405858 conn(0x7f63480ffa90 msgr2=0x7f6348100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:21.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.987+0000 7f634f800640 1 -- 172.21.15.89:0/3740405858 shutdown_connections 2024-09-10T08:38:21.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:21.988+0000 7f634f800640 1 -- 172.21.15.89:0/3740405858 wait complete. 2024-09-10T08:38:22.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:22 smithi089 ceph-mon[30757]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-10T08:38:22.909 INFO:teuthology.orchestra.run.smithi089.stdout:249108103215 2024-09-10T08:38:22.910 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-10T08:38:22.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:22 smithi149 ceph-mon[36167]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-10T08:38:23.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:22 smithi111 ceph-mon[35220]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-10T08:38:23.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.594+0000 7fba2da00640 1 -- 172.21.15.89:0/2408370368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba281079c0 msgr2=0x7fba28107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:23.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.594+0000 7fba2da00640 1 --2- 172.21.15.89:0/2408370368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba281079c0 0x7fba28107da0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fba18009930 tx=0x7fba1802a420 comp rx=0 tx=0).stop 2024-09-10T08:38:23.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.594+0000 7fba2da00640 1 -- 172.21.15.89:0/2408370368 shutdown_connections 2024-09-10T08:38:23.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.594+0000 7fba2da00640 1 --2- 172.21.15.89:0/2408370368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba28102360 0x7fba2810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:23.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.594+0000 7fba2da00640 1 --2- 172.21.15.89:0/2408370368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba281019c0 0x7fba28101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:23.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.594+0000 7fba2da00640 1 --2- 172.21.15.89:0/2408370368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba281079c0 0x7fba28107da0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:23.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.594+0000 7fba2da00640 1 -- 172.21.15.89:0/2408370368 >> 172.21.15.89:0/2408370368 conn(0x7fba280fd740 msgr2=0x7fba280ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:23.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.594+0000 7fba2da00640 1 -- 172.21.15.89:0/2408370368 shutdown_connections 2024-09-10T08:38:23.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.594+0000 7fba2da00640 1 -- 172.21.15.89:0/2408370368 wait complete. 2024-09-10T08:38:23.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.595+0000 7fba2da00640 1 Processor -- start 2024-09-10T08:38:23.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.595+0000 7fba2da00640 1 -- start start 2024-09-10T08:38:23.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.596+0000 7fba2da00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba281019c0 0x7fba28198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:23.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.596+0000 7fba2da00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba28102360 0x7fba28198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:23.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.596+0000 7fba2da00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba281079c0 0x7fba2819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:23.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.596+0000 7fba2da00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fba28111dc0 con 0x7fba281019c0 2024-09-10T08:38:23.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.596+0000 7fba2da00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fba28111c40 con 0x7fba281079c0 2024-09-10T08:38:23.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.596+0000 7fba2da00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fba28111f40 con 0x7fba28102360 2024-09-10T08:38:23.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.596+0000 7fba27e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba281019c0 0x7fba28198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:23.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.596+0000 7fba27e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba281019c0 0x7fba28198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57742/0 (socket says 172.21.15.89:57742) 2024-09-10T08:38:23.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.596+0000 7fba27e00640 1 -- 172.21.15.89:0/642390177 learned_addr learned my addr 172.21.15.89:0/642390177 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:23.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.596+0000 7fba2d000640 1 --2- 172.21.15.89:0/642390177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba281079c0 0x7fba2819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:23.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.596+0000 7fba27400640 1 --2- 172.21.15.89:0/642390177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba28102360 0x7fba28198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:23.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.596+0000 7fba2d000640 1 -- 172.21.15.89:0/642390177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba28102360 msgr2=0x7fba28198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:23.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.596+0000 7fba2d000640 1 --2- 172.21.15.89:0/642390177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba28102360 0x7fba28198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:23.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.596+0000 7fba2d000640 1 -- 172.21.15.89:0/642390177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba281019c0 msgr2=0x7fba28198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:23.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.596+0000 7fba2d000640 1 --2- 172.21.15.89:0/642390177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba281019c0 0x7fba28198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:23.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.596+0000 7fba2d000640 1 -- 172.21.15.89:0/642390177 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2819d2b0 con 0x7fba281079c0 2024-09-10T08:38:23.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.596+0000 7fba27400640 1 --2- 172.21.15.89:0/642390177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba28102360 0x7fba28198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:38:23.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.596+0000 7fba27e00640 1 --2- 172.21.15.89:0/642390177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba281019c0 0x7fba28198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:38:23.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.597+0000 7fba2d000640 1 --2- 172.21.15.89:0/642390177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba281079c0 0x7fba2819cbd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba20009fd0 tx=0x7fba2000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:23.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.597+0000 7fba24c00640 1 -- 172.21.15.89:0/642390177 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2000ce20 con 0x7fba281079c0 2024-09-10T08:38:23.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.597+0000 7fba2da00640 1 -- 172.21.15.89:0/642390177 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2819d5a0 con 0x7fba281079c0 2024-09-10T08:38:23.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.597+0000 7fba24c00640 1 -- 172.21.15.89:0/642390177 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba20005020 con 0x7fba281079c0 2024-09-10T08:38:23.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.597+0000 7fba24c00640 1 -- 172.21.15.89:0/642390177 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20010720 con 0x7fba281079c0 2024-09-10T08:38:23.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.597+0000 7fba2da00640 1 -- 172.21.15.89:0/642390177 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba281a4e80 con 0x7fba281079c0 2024-09-10T08:38:23.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.598+0000 7fba2da00640 1 -- 172.21.15.89:0/642390177 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fba28062980 con 0x7fba281079c0 2024-09-10T08:38:23.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.600+0000 7fba24c00640 1 -- 172.21.15.89:0/642390177 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fba20005540 con 0x7fba281079c0 2024-09-10T08:38:23.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.601+0000 7fba24c00640 1 --2- 172.21.15.89:0/642390177 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fba0c0a2c10 0x7fba0c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:23.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.602+0000 7fba24c00640 1 -- 172.21.15.89:0/642390177 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fba200cfdf0 con 0x7fba281079c0 2024-09-10T08:38:23.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.602+0000 7fba27e00640 1 --2- 172.21.15.89:0/642390177 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fba0c0a2c10 0x7fba0c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:23.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.602+0000 7fba24c00640 1 --2- 172.21.15.89:0/642390177 >> [v2:172.21.15.111:6800/2369291742,v1:172.21.15.111:6801/2369291742] conn(0x7fba0c0adde0 0x7fba0c0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:23.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.602+0000 7fba24c00640 1 -- 172.21.15.89:0/642390177 --> [v2:172.21.15.111:6800/2369291742,v1:172.21.15.111:6801/2369291742] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fba0c0b0950 con 0x7fba0c0adde0 2024-09-10T08:38:23.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.602+0000 7fba24c00640 1 -- 172.21.15.89:0/642390177 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fba200d01a0 con 0x7fba281079c0 2024-09-10T08:38:23.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.602+0000 7fba27e00640 1 --2- 172.21.15.89:0/642390177 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fba0c0a2c10 0x7fba0c0a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fba18000c00 tx=0x7fba18009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:23.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.602+0000 7fba27400640 1 --2- 172.21.15.89:0/642390177 >> [v2:172.21.15.111:6800/2369291742,v1:172.21.15.111:6801/2369291742] conn(0x7fba0c0adde0 0x7fba0c0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:23.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.603+0000 7fba27400640 1 --2- 172.21.15.89:0/642390177 >> [v2:172.21.15.111:6800/2369291742,v1:172.21.15.111:6801/2369291742] conn(0x7fba0c0adde0 0x7fba0c0b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:23.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.604+0000 7fba24c00640 1 -- 172.21.15.89:0/642390177 <== osd.1 v2:172.21.15.111:6800/2369291742 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fba0c0b0950 con 0x7fba0c0adde0 2024-09-10T08:38:23.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.618+0000 7fba2da00640 1 -- 172.21.15.89:0/642390177 --> [v2:172.21.15.111:6800/2369291742,v1:172.21.15.111:6801/2369291742] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fba28062e00 con 0x7fba0c0adde0 2024-09-10T08:38:23.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.619+0000 7fba24c00640 1 -- 172.21.15.89:0/642390177 <== osd.1 v2:172.21.15.111:6800/2369291742 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fba28062e00 con 0x7fba0c0adde0 2024-09-10T08:38:23.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.619+0000 7fba2da00640 1 -- 172.21.15.89:0/642390177 >> [v2:172.21.15.111:6800/2369291742,v1:172.21.15.111:6801/2369291742] conn(0x7fba0c0adde0 msgr2=0x7fba0c0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:23.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.619+0000 7fba2da00640 1 --2- 172.21.15.89:0/642390177 >> [v2:172.21.15.111:6800/2369291742,v1:172.21.15.111:6801/2369291742] conn(0x7fba0c0adde0 0x7fba0c0b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:23.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.619+0000 7fba2da00640 1 -- 172.21.15.89:0/642390177 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fba0c0a2c10 msgr2=0x7fba0c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:23.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.619+0000 7fba2da00640 1 --2- 172.21.15.89:0/642390177 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fba0c0a2c10 0x7fba0c0a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fba18000c00 tx=0x7fba18009560 comp rx=0 tx=0).stop 2024-09-10T08:38:23.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.619+0000 7fba2da00640 1 -- 172.21.15.89:0/642390177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba281079c0 msgr2=0x7fba2819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:23.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.619+0000 7fba2da00640 1 --2- 172.21.15.89:0/642390177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba281079c0 0x7fba2819cbd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba20009fd0 tx=0x7fba2000ef10 comp rx=0 tx=0).stop 2024-09-10T08:38:23.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.619+0000 7fba2d000640 1 -- 172.21.15.89:0/642390177 reap_dead start 2024-09-10T08:38:23.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.619+0000 7fba2da00640 1 -- 172.21.15.89:0/642390177 shutdown_connections 2024-09-10T08:38:23.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.619+0000 7fba2da00640 1 -- 172.21.15.89:0/642390177 >> 172.21.15.89:0/642390177 conn(0x7fba280fd740 msgr2=0x7fba280fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:23.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.619+0000 7fba2da00640 1 -- 172.21.15.89:0/642390177 shutdown_connections 2024-09-10T08:38:23.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.619+0000 7fba2da00640 1 -- 172.21.15.89:0/642390177 wait complete. 2024-09-10T08:38:24.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.184+0000 7f91a7600640 1 -- 172.21.15.89:0/1211262223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91a0104010 msgr2=0x7f91a01043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:24.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.184+0000 7f91a7600640 1 --2- 172.21.15.89:0/1211262223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91a0104010 0x7f91a01043f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9194009860 tx=0x7f919402a340 comp rx=0 tx=0).stop 2024-09-10T08:38:24.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.184+0000 7f91a7600640 1 -- 172.21.15.89:0/1211262223 shutdown_connections 2024-09-10T08:38:24.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.184+0000 7f91a7600640 1 --2- 172.21.15.89:0/1211262223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91a0109140 0x7f91a0110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.184+0000 7f91a7600640 1 --2- 172.21.15.89:0/1211262223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91a01049c0 0x7f91a0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.184+0000 7f91a7600640 1 --2- 172.21.15.89:0/1211262223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91a0104010 0x7f91a01043f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.184+0000 7f91a7600640 1 -- 172.21.15.89:0/1211262223 >> 172.21.15.89:0/1211262223 conn(0x7f91a00ffa50 msgr2=0x7f91a0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.184+0000 7f91a7600640 1 -- 172.21.15.89:0/1211262223 shutdown_connections 2024-09-10T08:38:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.184+0000 7f91a7600640 1 -- 172.21.15.89:0/1211262223 wait complete. 2024-09-10T08:38:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.185+0000 7f91a7600640 1 Processor -- start 2024-09-10T08:38:24.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.185+0000 7f91a7600640 1 -- start start 2024-09-10T08:38:24.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.186+0000 7f91a7600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91a0104010 0x7f91a019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:24.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.186+0000 7f91a7600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91a01049c0 0x7f91a019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:24.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.186+0000 7f91a7600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91a0109140 0x7f91a019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:24.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.186+0000 7f91a7600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f91a0114170 con 0x7f91a01049c0 2024-09-10T08:38:24.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.186+0000 7f91a7600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f91a0113ff0 con 0x7f91a0104010 2024-09-10T08:38:24.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.186+0000 7f91a7600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f91a01142f0 con 0x7f91a0109140 2024-09-10T08:38:24.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.186+0000 7f919fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91a01049c0 0x7f91a019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:24.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.186+0000 7f919fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91a01049c0 0x7f91a019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54182/0 (socket says 172.21.15.89:54182) 2024-09-10T08:38:24.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.186+0000 7f919fe00640 1 -- 172.21.15.89:0/3686228624 learned_addr learned my addr 172.21.15.89:0/3686228624 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:24.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.186+0000 7f91a4c00640 1 --2- 172.21.15.89:0/3686228624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91a0104010 0x7f91a019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:24.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.186+0000 7f91a5600640 1 --2- 172.21.15.89:0/3686228624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91a0109140 0x7f91a019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:24.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.186+0000 7f919fe00640 1 -- 172.21.15.89:0/3686228624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91a0109140 msgr2=0x7f91a019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:24.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.186+0000 7f919fe00640 1 --2- 172.21.15.89:0/3686228624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91a0109140 0x7f91a019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:24.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.186+0000 7f919fe00640 1 -- 172.21.15.89:0/3686228624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91a0104010 msgr2=0x7f91a019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:24.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.186+0000 7f919fe00640 1 --2- 172.21.15.89:0/3686228624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91a0104010 0x7f91a019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:24.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.186+0000 7f919fe00640 1 -- 172.21.15.89:0/3686228624 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a019f5e0 con 0x7f91a01049c0 2024-09-10T08:38:24.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.187+0000 7f919fe00640 1 --2- 172.21.15.89:0/3686228624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91a01049c0 0x7f91a019ab70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f918c00e9c0 tx=0x7f918c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:24.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.187+0000 7f919d600640 1 -- 172.21.15.89:0/3686228624 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c00cde0 con 0x7f91a01049c0 2024-09-10T08:38:24.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.187+0000 7f91a7600640 1 -- 172.21.15.89:0/3686228624 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a019f8d0 con 0x7f91a01049c0 2024-09-10T08:38:24.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.187+0000 7f919d600640 1 -- 172.21.15.89:0/3686228624 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f918c0035f0 con 0x7f91a01049c0 2024-09-10T08:38:24.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.187+0000 7f91a7600640 1 -- 172.21.15.89:0/3686228624 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a01a71b0 con 0x7f91a01049c0 2024-09-10T08:38:24.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.187+0000 7f919d600640 1 -- 172.21.15.89:0/3686228624 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c018630 con 0x7f91a01049c0 2024-09-10T08:38:24.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.188+0000 7f91a7600640 1 -- 172.21.15.89:0/3686228624 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f91a0062940 con 0x7f91a01049c0 2024-09-10T08:38:24.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.190+0000 7f919d600640 1 -- 172.21.15.89:0/3686228624 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f918c0037a0 con 0x7f91a01049c0 2024-09-10T08:38:24.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.191+0000 7f919d600640 1 --2- 172.21.15.89:0/3686228624 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f91880a2c10 0x7f91880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:24.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.191+0000 7f919d600640 1 -- 172.21.15.89:0/3686228624 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f918c0cfcf0 con 0x7f91a01049c0 2024-09-10T08:38:24.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.191+0000 7f91a4c00640 1 --2- 172.21.15.89:0/3686228624 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f91880a2c10 0x7f91880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:24.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.191+0000 7f919d600640 1 --2- 172.21.15.89:0/3686228624 >> [v2:172.21.15.111:6824/2328678744,v1:172.21.15.111:6825/2328678744] conn(0x7f91880adde0 0x7f91880b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:24.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.191+0000 7f919d600640 1 -- 172.21.15.89:0/3686228624 --> [v2:172.21.15.111:6824/2328678744,v1:172.21.15.111:6825/2328678744] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f91880b0950 con 0x7f91880adde0 2024-09-10T08:38:24.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.191+0000 7f919d600640 1 -- 172.21.15.89:0/3686228624 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f918c0d00a0 con 0x7f91a01049c0 2024-09-10T08:38:24.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.192+0000 7f91a5600640 1 --2- 172.21.15.89:0/3686228624 >> [v2:172.21.15.111:6824/2328678744,v1:172.21.15.111:6825/2328678744] conn(0x7f91880adde0 0x7f91880b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:24.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.192+0000 7f91a4c00640 1 --2- 172.21.15.89:0/3686228624 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f91880a2c10 0x7f91880a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f919402a930 tx=0x7f91940056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:24.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.192+0000 7f91a5600640 1 --2- 172.21.15.89:0/3686228624 >> [v2:172.21.15.111:6824/2328678744,v1:172.21.15.111:6825/2328678744] conn(0x7f91880adde0 0x7f91880b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:24.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.193+0000 7f919d600640 1 -- 172.21.15.89:0/3686228624 <== osd.10 v2:172.21.15.111:6824/2328678744 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f91880b0950 con 0x7f91880adde0 2024-09-10T08:38:24.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.208+0000 7f91a7600640 1 -- 172.21.15.89:0/3686228624 --> [v2:172.21.15.111:6824/2328678744,v1:172.21.15.111:6825/2328678744] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f91a0062d40 con 0x7f91880adde0 2024-09-10T08:38:24.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.208+0000 7f919d600640 1 -- 172.21.15.89:0/3686228624 <== osd.10 v2:172.21.15.111:6824/2328678744 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f91a0062d40 con 0x7f91880adde0 2024-09-10T08:38:24.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.209+0000 7f91a7600640 1 -- 172.21.15.89:0/3686228624 >> [v2:172.21.15.111:6824/2328678744,v1:172.21.15.111:6825/2328678744] conn(0x7f91880adde0 msgr2=0x7f91880b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:24.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.209+0000 7f91a7600640 1 --2- 172.21.15.89:0/3686228624 >> [v2:172.21.15.111:6824/2328678744,v1:172.21.15.111:6825/2328678744] conn(0x7f91880adde0 0x7f91880b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:24.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.209+0000 7f91a7600640 1 -- 172.21.15.89:0/3686228624 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f91880a2c10 msgr2=0x7f91880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:24.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.209+0000 7f91a7600640 1 --2- 172.21.15.89:0/3686228624 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f91880a2c10 0x7f91880a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f919402a930 tx=0x7f91940056d0 comp rx=0 tx=0).stop 2024-09-10T08:38:24.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.209+0000 7f91a7600640 1 -- 172.21.15.89:0/3686228624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91a01049c0 msgr2=0x7f91a019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:24.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.209+0000 7f91a7600640 1 --2- 172.21.15.89:0/3686228624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91a01049c0 0x7f91a019ab70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f918c00e9c0 tx=0x7f918c00ee80 comp rx=0 tx=0).stop 2024-09-10T08:38:24.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.209+0000 7f91a5600640 1 -- 172.21.15.89:0/3686228624 reap_dead start 2024-09-10T08:38:24.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.209+0000 7f91a7600640 1 -- 172.21.15.89:0/3686228624 shutdown_connections 2024-09-10T08:38:24.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.209+0000 7f91a7600640 1 -- 172.21.15.89:0/3686228624 >> 172.21.15.89:0/3686228624 conn(0x7f91a00ffa50 msgr2=0x7f91a0100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:24.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.209+0000 7f91a7600640 1 -- 172.21.15.89:0/3686228624 shutdown_connections 2024-09-10T08:38:24.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:22.209+0000 7f91a7600640 1 -- 172.21.15.89:0/3686228624 wait complete. 2024-09-10T08:38:24.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.847+0000 7fa548200640 1 -- 172.21.15.89:0/650321616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa540102790 msgr2=0x7fa5401067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:24.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.847+0000 7fa548200640 1 --2- 172.21.15.89:0/650321616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa540102790 0x7fa5401067e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009900 tx=0x7fa52c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:38:24.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.847+0000 7fa548200640 1 -- 172.21.15.89:0/650321616 shutdown_connections 2024-09-10T08:38:24.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.847+0000 7fa548200640 1 --2- 172.21.15.89:0/650321616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa540106f10 0x7fa54010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:24.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.847+0000 7fa548200640 1 --2- 172.21.15.89:0/650321616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa540102790 0x7fa5401067e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:24.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.847+0000 7fa548200640 1 --2- 172.21.15.89:0/650321616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa540101de0 0x7fa5401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:24.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.847+0000 7fa548200640 1 -- 172.21.15.89:0/650321616 >> 172.21.15.89:0/650321616 conn(0x7fa5400fd8c0 msgr2=0x7fa5400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:24.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.847+0000 7fa548200640 1 -- 172.21.15.89:0/650321616 shutdown_connections 2024-09-10T08:38:24.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.847+0000 7fa548200640 1 -- 172.21.15.89:0/650321616 wait complete. 2024-09-10T08:38:24.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.848+0000 7fa548200640 1 Processor -- start 2024-09-10T08:38:24.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.848+0000 7fa548200640 1 -- start start 2024-09-10T08:38:24.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.849+0000 7fa548200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa540101de0 0x7fa54019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:24.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.849+0000 7fa548200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa540102790 0x7fa54019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:24.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.849+0000 7fa548200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa540106f10 0x7fa54019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:24.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.849+0000 7fa548200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa540111fc0 con 0x7fa540102790 2024-09-10T08:38:24.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.849+0000 7fa548200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa540111e40 con 0x7fa540101de0 2024-09-10T08:38:24.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.849+0000 7fa548200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa540112140 con 0x7fa540106f10 2024-09-10T08:38:24.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.849+0000 7fa544e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa540102790 0x7fa54019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:24.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.849+0000 7fa545800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa540101de0 0x7fa54019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:24.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.849+0000 7fa545800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa540101de0 0x7fa54019a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:36474/0 (socket says 172.21.15.89:36474) 2024-09-10T08:38:24.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.849+0000 7fa544e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa540102790 0x7fa54019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57780/0 (socket says 172.21.15.89:57780) 2024-09-10T08:38:24.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.849+0000 7fa545800640 1 -- 172.21.15.89:0/1357255749 learned_addr learned my addr 172.21.15.89:0/1357255749 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:24.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.849+0000 7fa546200640 1 --2- 172.21.15.89:0/1357255749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa540106f10 0x7fa54019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:24.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.849+0000 7fa544e00640 1 -- 172.21.15.89:0/1357255749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa540106f10 msgr2=0x7fa54019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:24.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.849+0000 7fa544e00640 1 --2- 172.21.15.89:0/1357255749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa540106f10 0x7fa54019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:24.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.849+0000 7fa544e00640 1 -- 172.21.15.89:0/1357255749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa540101de0 msgr2=0x7fa54019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:24.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.849+0000 7fa544e00640 1 --2- 172.21.15.89:0/1357255749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa540101de0 0x7fa54019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:24.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.849+0000 7fa544e00640 1 -- 172.21.15.89:0/1357255749 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa54019f610 con 0x7fa540102790 2024-09-10T08:38:24.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.850+0000 7fa544e00640 1 --2- 172.21.15.89:0/1357255749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa540102790 0x7fa54019aba0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa52c02a910 tx=0x7fa52c003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:24.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.850+0000 7fa53e000640 1 -- 172.21.15.89:0/1357255749 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c03f070 con 0x7fa540102790 2024-09-10T08:38:24.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.850+0000 7fa53e000640 1 -- 172.21.15.89:0/1357255749 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa52c031c90 con 0x7fa540102790 2024-09-10T08:38:24.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.850+0000 7fa53e000640 1 -- 172.21.15.89:0/1357255749 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c035030 con 0x7fa540102790 2024-09-10T08:38:24.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.850+0000 7fa548200640 1 -- 172.21.15.89:0/1357255749 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa54019f8a0 con 0x7fa540102790 2024-09-10T08:38:24.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.850+0000 7fa548200640 1 -- 172.21.15.89:0/1357255749 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa54019fd30 con 0x7fa540102790 2024-09-10T08:38:24.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.851+0000 7fa548200640 1 -- 172.21.15.89:0/1357255749 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa540075780 con 0x7fa540102790 2024-09-10T08:38:24.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.853+0000 7fa53e000640 1 -- 172.21.15.89:0/1357255749 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa52c042070 con 0x7fa540102790 2024-09-10T08:38:24.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.854+0000 7fa53e000640 1 --2- 172.21.15.89:0/1357255749 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5280a2c90 0x7fa5280a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:24.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.854+0000 7fa53e000640 1 -- 172.21.15.89:0/1357255749 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fa52c0f03a0 con 0x7fa540102790 2024-09-10T08:38:24.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.855+0000 7fa545800640 1 --2- 172.21.15.89:0/1357255749 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5280a2c90 0x7fa5280a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:24.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.855+0000 7fa53e000640 1 --2- 172.21.15.89:0/1357255749 >> [v2:172.21.15.149:6800/1092261395,v1:172.21.15.149:6801/1092261395] conn(0x7fa5280ade50 0x7fa5280b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:24.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.855+0000 7fa53e000640 1 -- 172.21.15.89:0/1357255749 --> [v2:172.21.15.149:6800/1092261395,v1:172.21.15.149:6801/1092261395] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa52c034e50 con 0x7fa5280ade50 2024-09-10T08:38:24.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.855+0000 7fa53e000640 1 -- 172.21.15.89:0/1357255749 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fa52c0f0750 con 0x7fa540102790 2024-09-10T08:38:24.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.855+0000 7fa546200640 1 --2- 172.21.15.89:0/1357255749 >> [v2:172.21.15.149:6800/1092261395,v1:172.21.15.149:6801/1092261395] conn(0x7fa5280ade50 0x7fa5280b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:24.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.855+0000 7fa545800640 1 --2- 172.21.15.89:0/1357255749 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5280a2c90 0x7fa5280a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa534009c90 tx=0x7fa534005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:24.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.856+0000 7fa546200640 1 --2- 172.21.15.89:0/1357255749 >> [v2:172.21.15.149:6800/1092261395,v1:172.21.15.149:6801/1092261395] conn(0x7fa5280ade50 0x7fa5280b02f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:24.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.857+0000 7fa53e000640 1 -- 172.21.15.89:0/1357255749 <== osd.2 v2:172.21.15.149:6800/1092261395 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa52c034e50 con 0x7fa5280ade50 2024-09-10T08:38:24.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.871+0000 7fa548200640 1 -- 172.21.15.89:0/1357255749 --> [v2:172.21.15.149:6800/1092261395,v1:172.21.15.149:6801/1092261395] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa5401032c0 con 0x7fa5280ade50 2024-09-10T08:38:24.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.871+0000 7fa53e000640 1 -- 172.21.15.89:0/1357255749 <== osd.2 v2:172.21.15.149:6800/1092261395 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa5401032c0 con 0x7fa5280ade50 2024-09-10T08:38:24.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.872+0000 7fa548200640 1 -- 172.21.15.89:0/1357255749 >> [v2:172.21.15.149:6800/1092261395,v1:172.21.15.149:6801/1092261395] conn(0x7fa5280ade50 msgr2=0x7fa5280b02f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:24.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.872+0000 7fa548200640 1 --2- 172.21.15.89:0/1357255749 >> [v2:172.21.15.149:6800/1092261395,v1:172.21.15.149:6801/1092261395] conn(0x7fa5280ade50 0x7fa5280b02f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:24.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.872+0000 7fa548200640 1 -- 172.21.15.89:0/1357255749 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5280a2c90 msgr2=0x7fa5280a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:24.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.872+0000 7fa548200640 1 --2- 172.21.15.89:0/1357255749 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5280a2c90 0x7fa5280a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa534009c90 tx=0x7fa534005f90 comp rx=0 tx=0).stop 2024-09-10T08:38:24.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.872+0000 7fa548200640 1 -- 172.21.15.89:0/1357255749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa540102790 msgr2=0x7fa54019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:24.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.872+0000 7fa548200640 1 --2- 172.21.15.89:0/1357255749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa540102790 0x7fa54019aba0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa52c02a910 tx=0x7fa52c003b20 comp rx=0 tx=0).stop 2024-09-10T08:38:24.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.872+0000 7fa546200640 1 -- 172.21.15.89:0/1357255749 reap_dead start 2024-09-10T08:38:24.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.872+0000 7fa548200640 1 -- 172.21.15.89:0/1357255749 shutdown_connections 2024-09-10T08:38:24.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.872+0000 7fa548200640 1 -- 172.21.15.89:0/1357255749 >> 172.21.15.89:0/1357255749 conn(0x7fa5400fd8c0 msgr2=0x7fa5400feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:24.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.872+0000 7fa548200640 1 -- 172.21.15.89:0/1357255749 shutdown_connections 2024-09-10T08:38:24.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.872+0000 7fa548200640 1 -- 172.21.15.89:0/1357255749 wait complete. 2024-09-10T08:38:24.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:24 smithi149 ceph-mon[36167]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-10T08:38:25.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:24 smithi111 ceph-mon[35220]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-10T08:38:25.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:24 smithi089 ceph-mon[30757]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-10T08:38:25.940 INFO:teuthology.orchestra.run.smithi089.stdout:304942678052 2024-09-10T08:38:25.941 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-10T08:38:26.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.123+0000 7ff337000640 1 -- 172.21.15.89:0/158981329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff330102790 msgr2=0x7ff3301067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:26.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.123+0000 7ff337000640 1 --2- 172.21.15.89:0/158981329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff330102790 0x7ff3301067e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009900 tx=0x7ff31c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:38:26.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.123+0000 7ff337000640 1 -- 172.21.15.89:0/158981329 shutdown_connections 2024-09-10T08:38:26.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.123+0000 7ff337000640 1 --2- 172.21.15.89:0/158981329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff330106f10 0x7ff33010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:26.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.123+0000 7ff337000640 1 --2- 172.21.15.89:0/158981329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff330102790 0x7ff3301067e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:26.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.123+0000 7ff337000640 1 --2- 172.21.15.89:0/158981329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff330101de0 0x7ff3301021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:26.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.123+0000 7ff337000640 1 -- 172.21.15.89:0/158981329 >> 172.21.15.89:0/158981329 conn(0x7ff3300fd8c0 msgr2=0x7ff3300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:26.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.123+0000 7ff337000640 1 -- 172.21.15.89:0/158981329 shutdown_connections 2024-09-10T08:38:26.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.123+0000 7ff337000640 1 -- 172.21.15.89:0/158981329 wait complete. 2024-09-10T08:38:26.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.124+0000 7ff337000640 1 Processor -- start 2024-09-10T08:38:26.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.124+0000 7ff337000640 1 -- start start 2024-09-10T08:38:26.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.125+0000 7ff337000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff330101de0 0x7ff33019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:26.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.125+0000 7ff337000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff330102790 0x7ff33019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:26.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.125+0000 7ff337000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff330106f10 0x7ff33019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:26.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.125+0000 7ff337000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff330111fc0 con 0x7ff330106f10 2024-09-10T08:38:26.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.125+0000 7ff337000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff330111e40 con 0x7ff330102790 2024-09-10T08:38:26.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.125+0000 7ff337000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff330112140 con 0x7ff330101de0 2024-09-10T08:38:26.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.125+0000 7ff32f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff330102790 0x7ff33019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:26.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.125+0000 7ff32f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff330102790 0x7ff33019ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:36498/0 (socket says 172.21.15.89:36498) 2024-09-10T08:38:26.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.125+0000 7ff32f400640 1 -- 172.21.15.89:0/3267593260 learned_addr learned my addr 172.21.15.89:0/3267593260 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:26.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.125+0000 7ff32fe00640 1 --2- 172.21.15.89:0/3267593260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff330101de0 0x7ff33019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:26.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.125+0000 7ff335000640 1 --2- 172.21.15.89:0/3267593260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff330106f10 0x7ff33019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:26.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.126+0000 7ff32fe00640 1 -- 172.21.15.89:0/3267593260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff330102790 msgr2=0x7ff33019ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:26.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.126+0000 7ff32fe00640 1 --2- 172.21.15.89:0/3267593260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff330102790 0x7ff33019ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:26.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.126+0000 7ff32fe00640 1 -- 172.21.15.89:0/3267593260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff330106f10 msgr2=0x7ff33019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:26.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.126+0000 7ff32fe00640 1 --2- 172.21.15.89:0/3267593260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff330106f10 0x7ff33019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:26.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.126+0000 7ff32fe00640 1 -- 172.21.15.89:0/3267593260 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33019f6b0 con 0x7ff330101de0 2024-09-10T08:38:26.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.126+0000 7ff335000640 1 --2- 172.21.15.89:0/3267593260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff330106f10 0x7ff33019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:38:26.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.126+0000 7ff32fe00640 1 --2- 172.21.15.89:0/3267593260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff330101de0 0x7ff33019a700 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff32000e9f0 tx=0x7ff32000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:26.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.126+0000 7ff32cc00640 1 -- 172.21.15.89:0/3267593260 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32000ccc0 con 0x7ff330101de0 2024-09-10T08:38:26.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.126+0000 7ff337000640 1 -- 172.21.15.89:0/3267593260 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33019f9a0 con 0x7ff330101de0 2024-09-10T08:38:26.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.127+0000 7ff337000640 1 -- 172.21.15.89:0/3267593260 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3301a7280 con 0x7ff330101de0 2024-09-10T08:38:26.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.127+0000 7ff32cc00640 1 -- 172.21.15.89:0/3267593260 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff320010910 con 0x7ff330101de0 2024-09-10T08:38:26.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.127+0000 7ff32cc00640 1 -- 172.21.15.89:0/3267593260 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff320018880 con 0x7ff330101de0 2024-09-10T08:38:26.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.127+0000 7ff337000640 1 -- 172.21.15.89:0/3267593260 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff310001030 con 0x7ff330101de0 2024-09-10T08:38:26.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.130+0000 7ff32cc00640 1 -- 172.21.15.89:0/3267593260 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff320010430 con 0x7ff330101de0 2024-09-10T08:38:26.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.131+0000 7ff32cc00640 1 --2- 172.21.15.89:0/3267593260 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3140a2c90 0x7ff3140a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:26.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.131+0000 7ff32cc00640 1 -- 172.21.15.89:0/3267593260 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7ff3200cf940 con 0x7ff330101de0 2024-09-10T08:38:26.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.131+0000 7ff32f400640 1 --2- 172.21.15.89:0/3267593260 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3140a2c90 0x7ff3140a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:26.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.131+0000 7ff32cc00640 1 --2- 172.21.15.89:0/3267593260 >> [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024] conn(0x7ff3140ade60 0x7ff3140b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:26.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.131+0000 7ff32cc00640 1 -- 172.21.15.89:0/3267593260 --> [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff3140b09d0 con 0x7ff3140ade60 2024-09-10T08:38:26.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.131+0000 7ff32cc00640 1 -- 172.21.15.89:0/3267593260 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7ff3200cfcf0 con 0x7ff330101de0 2024-09-10T08:38:26.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.132+0000 7ff335000640 1 --2- 172.21.15.89:0/3267593260 >> [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024] conn(0x7ff3140ade60 0x7ff3140b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:26.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.132+0000 7ff32f400640 1 --2- 172.21.15.89:0/3267593260 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3140a2c90 0x7ff3140a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff31c0096c0 tx=0x7ff31c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:26.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.132+0000 7ff335000640 1 --2- 172.21.15.89:0/3267593260 >> [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024] conn(0x7ff3140ade60 0x7ff3140b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:26.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.133+0000 7ff32cc00640 1 -- 172.21.15.89:0/3267593260 <== osd.6 v2:172.21.15.89:6818/4190488024 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff3140b09d0 con 0x7ff3140ade60 2024-09-10T08:38:26.147 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.147+0000 7ff337000640 1 -- 172.21.15.89:0/3267593260 --> [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff310002dd0 con 0x7ff3140ade60 2024-09-10T08:38:26.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.148+0000 7ff32cc00640 1 -- 172.21.15.89:0/3267593260 <== osd.6 v2:172.21.15.89:6818/4190488024 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff310002dd0 con 0x7ff3140ade60 2024-09-10T08:38:26.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.148+0000 7ff337000640 1 -- 172.21.15.89:0/3267593260 >> [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024] conn(0x7ff3140ade60 msgr2=0x7ff3140b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:26.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.148+0000 7ff337000640 1 --2- 172.21.15.89:0/3267593260 >> [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024] conn(0x7ff3140ade60 0x7ff3140b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:26.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.149+0000 7ff337000640 1 -- 172.21.15.89:0/3267593260 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3140a2c90 msgr2=0x7ff3140a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:26.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.149+0000 7ff337000640 1 --2- 172.21.15.89:0/3267593260 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3140a2c90 0x7ff3140a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff31c0096c0 tx=0x7ff31c0046c0 comp rx=0 tx=0).stop 2024-09-10T08:38:26.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.149+0000 7ff337000640 1 -- 172.21.15.89:0/3267593260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff330101de0 msgr2=0x7ff33019a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:26.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.149+0000 7ff337000640 1 --2- 172.21.15.89:0/3267593260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff330101de0 0x7ff33019a700 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff32000e9f0 tx=0x7ff32000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:38:26.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.149+0000 7ff335000640 1 -- 172.21.15.89:0/3267593260 reap_dead start 2024-09-10T08:38:26.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.149+0000 7ff337000640 1 -- 172.21.15.89:0/3267593260 shutdown_connections 2024-09-10T08:38:26.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.149+0000 7ff337000640 1 -- 172.21.15.89:0/3267593260 >> 172.21.15.89:0/3267593260 conn(0x7ff3300fd8c0 msgr2=0x7ff3300feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:26.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.149+0000 7ff337000640 1 -- 172.21.15.89:0/3267593260 shutdown_connections 2024-09-10T08:38:26.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.149+0000 7ff337000640 1 -- 172.21.15.89:0/3267593260 wait complete. 2024-09-10T08:38:26.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.446+0000 7f364a000640 1 -- 172.21.15.89:0/267454988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3644104010 msgr2=0x7f36441043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:26.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.446+0000 7f364a000640 1 --2- 172.21.15.89:0/267454988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3644104010 0x7f36441043f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3634009900 tx=0x7f363402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:38:26.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.446+0000 7f364a000640 1 -- 172.21.15.89:0/267454988 shutdown_connections 2024-09-10T08:38:26.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.446+0000 7f364a000640 1 --2- 172.21.15.89:0/267454988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3644109140 0x7f3644110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:26.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.446+0000 7f364a000640 1 --2- 172.21.15.89:0/267454988 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36441049c0 0x7f3644108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:26.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.446+0000 7f364a000640 1 --2- 172.21.15.89:0/267454988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3644104010 0x7f36441043f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:26.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.447+0000 7f364a000640 1 -- 172.21.15.89:0/267454988 >> 172.21.15.89:0/267454988 conn(0x7f36440ffa50 msgr2=0x7f3644101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:26.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.447+0000 7f364a000640 1 -- 172.21.15.89:0/267454988 shutdown_connections 2024-09-10T08:38:26.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.447+0000 7f364a000640 1 -- 172.21.15.89:0/267454988 wait complete. 2024-09-10T08:38:26.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.448+0000 7f364a000640 1 Processor -- start 2024-09-10T08:38:26.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.448+0000 7f364a000640 1 -- start start 2024-09-10T08:38:26.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.448+0000 7f364a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3644104010 0x7f364419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:26.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.448+0000 7f364a000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36441049c0 0x7f364419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:26.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.448+0000 7f364a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3644109140 0x7f364419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:26.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.448+0000 7f364a000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3644114160 con 0x7f36441049c0 2024-09-10T08:38:26.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.448+0000 7f364a000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3644113fe0 con 0x7f3644109140 2024-09-10T08:38:26.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.448+0000 7f364a000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f36441142e0 con 0x7f3644104010 2024-09-10T08:38:26.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.448+0000 7f363be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36441049c0 0x7f364419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:26.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.448+0000 7f3643400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3644104010 0x7f364419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:26.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.448+0000 7f3643400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3644104010 0x7f364419a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:35194/0 (socket says 172.21.15.89:35194) 2024-09-10T08:38:26.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.448+0000 7f3643400640 1 -- 172.21.15.89:0/1866588715 learned_addr learned my addr 172.21.15.89:0/1866588715 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:26.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.449+0000 7f3643e00640 1 --2- 172.21.15.89:0/1866588715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3644109140 0x7f364419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:26.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.449+0000 7f363be00640 1 -- 172.21.15.89:0/1866588715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3644104010 msgr2=0x7f364419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:26.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.449+0000 7f363be00640 1 --2- 172.21.15.89:0/1866588715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3644104010 0x7f364419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:26.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.449+0000 7f363be00640 1 -- 172.21.15.89:0/1866588715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3644109140 msgr2=0x7f364419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:26.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.449+0000 7f363be00640 1 --2- 172.21.15.89:0/1866588715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3644109140 0x7f364419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:26.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.449+0000 7f363be00640 1 -- 172.21.15.89:0/1866588715 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364419f680 con 0x7f36441049c0 2024-09-10T08:38:26.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.449+0000 7f3643400640 1 --2- 172.21.15.89:0/1866588715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3644104010 0x7f364419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:38:26.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.449+0000 7f363be00640 1 --2- 172.21.15.89:0/1866588715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36441049c0 0x7f364419ac10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f362c00e9f0 tx=0x7f362c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:26.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.449+0000 7f3640c00640 1 -- 172.21.15.89:0/1866588715 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c00ccc0 con 0x7f36441049c0 2024-09-10T08:38:26.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.449+0000 7f364a000640 1 -- 172.21.15.89:0/1866588715 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364419f970 con 0x7f36441049c0 2024-09-10T08:38:26.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.450+0000 7f3640c00640 1 -- 172.21.15.89:0/1866588715 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f362c005020 con 0x7f36441049c0 2024-09-10T08:38:26.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.450+0000 7f3640c00640 1 -- 172.21.15.89:0/1866588715 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c0049e0 con 0x7f36441049c0 2024-09-10T08:38:26.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.450+0000 7f364a000640 1 -- 172.21.15.89:0/1866588715 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36441a7250 con 0x7f36441049c0 2024-09-10T08:38:26.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.450+0000 7f364a000640 1 -- 172.21.15.89:0/1866588715 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3644062900 con 0x7f36441049c0 2024-09-10T08:38:26.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.452+0000 7f3640c00640 1 -- 172.21.15.89:0/1866588715 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f362c005560 con 0x7f36441049c0 2024-09-10T08:38:26.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.454+0000 7f3640c00640 1 --2- 172.21.15.89:0/1866588715 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f36280a2c10 0x7f36280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:26.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.454+0000 7f3640c00640 1 -- 172.21.15.89:0/1866588715 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f362c0cf740 con 0x7f36441049c0 2024-09-10T08:38:26.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.454+0000 7f3643400640 1 --2- 172.21.15.89:0/1866588715 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f36280a2c10 0x7f36280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:26.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.454+0000 7f3640c00640 1 --2- 172.21.15.89:0/1866588715 >> [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040] conn(0x7f36280adde0 0x7f36280b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:26.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.454+0000 7f3640c00640 1 -- 172.21.15.89:0/1866588715 --> [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f36280b0950 con 0x7f36280adde0 2024-09-10T08:38:26.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.454+0000 7f3640c00640 1 -- 172.21.15.89:0/1866588715 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f362c0cfb90 con 0x7f36441049c0 2024-09-10T08:38:26.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.454+0000 7f3643e00640 1 --2- 172.21.15.89:0/1866588715 >> [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040] conn(0x7f36280adde0 0x7f36280b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:26.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.455+0000 7f3643400640 1 --2- 172.21.15.89:0/1866588715 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f36280a2c10 0x7f36280a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f36340096c0 tx=0x7f3634005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:26.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.455+0000 7f3643e00640 1 --2- 172.21.15.89:0/1866588715 >> [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040] conn(0x7f36280adde0 0x7f36280b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:26.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.456+0000 7f3640c00640 1 -- 172.21.15.89:0/1866588715 <== osd.9 v2:172.21.15.89:6826/2427467040 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f36280b0950 con 0x7f36280adde0 2024-09-10T08:38:26.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.472+0000 7f364a000640 1 -- 172.21.15.89:0/1866588715 --> [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3644062d80 con 0x7f36280adde0 2024-09-10T08:38:26.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.472+0000 7f3640c00640 1 -- 172.21.15.89:0/1866588715 <== osd.9 v2:172.21.15.89:6826/2427467040 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3644062d80 con 0x7f36280adde0 2024-09-10T08:38:26.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.473+0000 7f364a000640 1 -- 172.21.15.89:0/1866588715 >> [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040] conn(0x7f36280adde0 msgr2=0x7f36280b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:26.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.473+0000 7f364a000640 1 --2- 172.21.15.89:0/1866588715 >> [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040] conn(0x7f36280adde0 0x7f36280b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:26.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.473+0000 7f364a000640 1 -- 172.21.15.89:0/1866588715 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f36280a2c10 msgr2=0x7f36280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:26.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.473+0000 7f364a000640 1 --2- 172.21.15.89:0/1866588715 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f36280a2c10 0x7f36280a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f36340096c0 tx=0x7f3634005a20 comp rx=0 tx=0).stop 2024-09-10T08:38:26.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.473+0000 7f364a000640 1 -- 172.21.15.89:0/1866588715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36441049c0 msgr2=0x7f364419ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:26.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.473+0000 7f364a000640 1 --2- 172.21.15.89:0/1866588715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f36441049c0 0x7f364419ac10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f362c00e9f0 tx=0x7f362c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:38:26.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.473+0000 7f3643e00640 1 -- 172.21.15.89:0/1866588715 reap_dead start 2024-09-10T08:38:26.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.473+0000 7f364a000640 1 -- 172.21.15.89:0/1866588715 shutdown_connections 2024-09-10T08:38:26.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.473+0000 7f364a000640 1 -- 172.21.15.89:0/1866588715 >> 172.21.15.89:0/1866588715 conn(0x7f36440ffa50 msgr2=0x7f3644100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:26.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.473+0000 7f364a000640 1 -- 172.21.15.89:0/1866588715 shutdown_connections 2024-09-10T08:38:26.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:26.473+0000 7f364a000640 1 -- 172.21.15.89:0/1866588715 wait complete. 2024-09-10T08:38:26.759 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:26 smithi089 ceph-mon[30757]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-10T08:38:26.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:26 smithi149 ceph-mon[36167]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-10T08:38:27.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:26 smithi111 ceph-mon[35220]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-10T08:38:27.208 INFO:teuthology.orchestra.run.smithi089.stdout:163208757312 2024-09-10T08:38:27.208 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T08:38:27.885 INFO:teuthology.orchestra.run.smithi089.stdout:339302416415 2024-09-10T08:38:27.885 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-10T08:38:28.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.177+0000 7f2d77e00640 1 -- 172.21.15.89:0/31036142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d78102360 msgr2=0x7f2d7810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:28.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.177+0000 7f2d77e00640 1 --2- 172.21.15.89:0/31036142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d78102360 0x7f2d7810e890 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009cd0 tx=0x7f2d7002a8e0 comp rx=0 tx=0).stop 2024-09-10T08:38:28.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.177+0000 7f2d77e00640 1 -- 172.21.15.89:0/31036142 shutdown_connections 2024-09-10T08:38:28.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.177+0000 7f2d77e00640 1 --2- 172.21.15.89:0/31036142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d78102360 0x7f2d7810e890 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:28.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.177+0000 7f2d77e00640 1 --2- 172.21.15.89:0/31036142 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d781019c0 0x7f2d78101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:28.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.177+0000 7f2d77e00640 1 --2- 172.21.15.89:0/31036142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d781079c0 0x7f2d78107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:28.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.177+0000 7f2d77e00640 1 -- 172.21.15.89:0/31036142 >> 172.21.15.89:0/31036142 conn(0x7f2d780fd740 msgr2=0x7f2d780ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:28.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.177+0000 7f2d77e00640 1 -- 172.21.15.89:0/31036142 shutdown_connections 2024-09-10T08:38:28.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.177+0000 7f2d77e00640 1 -- 172.21.15.89:0/31036142 wait complete. 2024-09-10T08:38:28.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.178+0000 7f2d77e00640 1 Processor -- start 2024-09-10T08:38:28.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.178+0000 7f2d77e00640 1 -- start start 2024-09-10T08:38:28.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.179+0000 7f2d77e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d781019c0 0x7f2d7819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:28.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.179+0000 7f2d77e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d78102360 0x7f2d7819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:28.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.179+0000 7f2d77e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d781079c0 0x7f2d7819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:28.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.179+0000 7f2d77e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2d78111e10 con 0x7f2d781019c0 2024-09-10T08:38:28.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.179+0000 7f2d77e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2d78111c90 con 0x7f2d78102360 2024-09-10T08:38:28.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.179+0000 7f2d77e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2d78111f90 con 0x7f2d781079c0 2024-09-10T08:38:28.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.179+0000 7f2d6fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d78102360 0x7f2d7819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:28.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.179+0000 7f2d76a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d781019c0 0x7f2d7819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:28.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.179+0000 7f2d6fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d78102360 0x7f2d7819a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:36542/0 (socket says 172.21.15.89:36542) 2024-09-10T08:38:28.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.179+0000 7f2d6fe00640 1 -- 172.21.15.89:0/3311352946 learned_addr learned my addr 172.21.15.89:0/3311352946 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:28.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.179+0000 7f2d77400640 1 --2- 172.21.15.89:0/3311352946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d781079c0 0x7f2d7819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:28.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.179+0000 7f2d6fe00640 1 -- 172.21.15.89:0/3311352946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d781079c0 msgr2=0x7f2d7819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:28.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.179+0000 7f2d6fe00640 1 --2- 172.21.15.89:0/3311352946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d781079c0 0x7f2d7819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:28.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.180+0000 7f2d6fe00640 1 -- 172.21.15.89:0/3311352946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d781019c0 msgr2=0x7f2d7819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:28.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.180+0000 7f2d6fe00640 1 --2- 172.21.15.89:0/3311352946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d781019c0 0x7f2d7819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:28.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.180+0000 7f2d6fe00640 1 -- 172.21.15.89:0/3311352946 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7819f440 con 0x7f2d78102360 2024-09-10T08:38:28.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.180+0000 7f2d6fe00640 1 --2- 172.21.15.89:0/3311352946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d78102360 0x7f2d7819a9d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000ea60 tx=0x7f2d6000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:28.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.180+0000 7f2d6f400640 1 -- 172.21.15.89:0/3311352946 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000cde0 con 0x7f2d78102360 2024-09-10T08:38:28.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.180+0000 7f2d77e00640 1 -- 172.21.15.89:0/3311352946 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7819f730 con 0x7f2d78102360 2024-09-10T08:38:28.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.180+0000 7f2d6f400640 1 -- 172.21.15.89:0/3311352946 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d600035f0 con 0x7f2d78102360 2024-09-10T08:38:28.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.180+0000 7f2d6f400640 1 -- 172.21.15.89:0/3311352946 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60010670 con 0x7f2d78102360 2024-09-10T08:38:28.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.180+0000 7f2d77e00640 1 -- 172.21.15.89:0/3311352946 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d781a7010 con 0x7f2d78102360 2024-09-10T08:38:28.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.181+0000 7f2d77e00640 1 -- 172.21.15.89:0/3311352946 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d780fe7b0 con 0x7f2d78102360 2024-09-10T08:38:28.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.184+0000 7f2d6f400640 1 -- 172.21.15.89:0/3311352946 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2d600037a0 con 0x7f2d78102360 2024-09-10T08:38:28.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.185+0000 7f2d6f400640 1 --2- 172.21.15.89:0/3311352946 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2d5c0a2c10 0x7f2d5c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:28.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.185+0000 7f2d6f400640 1 -- 172.21.15.89:0/3311352946 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f2d600cfb20 con 0x7f2d78102360 2024-09-10T08:38:28.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.186+0000 7f2d76a00640 1 --2- 172.21.15.89:0/3311352946 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2d5c0a2c10 0x7f2d5c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:28.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.186+0000 7f2d6f400640 1 --2- 172.21.15.89:0/3311352946 >> [v2:172.21.15.111:6816/652666376,v1:172.21.15.111:6817/652666376] conn(0x7f2d5c0adde0 0x7f2d5c0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:28.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.186+0000 7f2d6f400640 1 -- 172.21.15.89:0/3311352946 --> [v2:172.21.15.111:6816/652666376,v1:172.21.15.111:6817/652666376] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d6001aec0 con 0x7f2d5c0adde0 2024-09-10T08:38:28.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.186+0000 7f2d6f400640 1 -- 172.21.15.89:0/3311352946 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f2d600cfed0 con 0x7f2d78102360 2024-09-10T08:38:28.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.186+0000 7f2d77400640 1 --2- 172.21.15.89:0/3311352946 >> [v2:172.21.15.111:6816/652666376,v1:172.21.15.111:6817/652666376] conn(0x7f2d5c0adde0 0x7f2d5c0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:28.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.186+0000 7f2d76a00640 1 --2- 172.21.15.89:0/3311352946 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2d5c0a2c10 0x7f2d5c0a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009b50 tx=0x7f2d68005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:28.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.187+0000 7f2d77400640 1 --2- 172.21.15.89:0/3311352946 >> [v2:172.21.15.111:6816/652666376,v1:172.21.15.111:6817/652666376] conn(0x7f2d5c0adde0 0x7f2d5c0b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:28.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.188+0000 7f2d6f400640 1 -- 172.21.15.89:0/3311352946 <== osd.7 v2:172.21.15.111:6816/652666376 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d6001aec0 con 0x7f2d5c0adde0 2024-09-10T08:38:28.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.202+0000 7f2d77e00640 1 -- 172.21.15.89:0/3311352946 --> [v2:172.21.15.111:6816/652666376,v1:172.21.15.111:6817/652666376] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2d78103100 con 0x7f2d5c0adde0 2024-09-10T08:38:28.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.202+0000 7f2d6f400640 1 -- 172.21.15.89:0/3311352946 <== osd.7 v2:172.21.15.111:6816/652666376 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2d78103100 con 0x7f2d5c0adde0 2024-09-10T08:38:28.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.203+0000 7f2d77e00640 1 -- 172.21.15.89:0/3311352946 >> [v2:172.21.15.111:6816/652666376,v1:172.21.15.111:6817/652666376] conn(0x7f2d5c0adde0 msgr2=0x7f2d5c0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:28.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.203+0000 7f2d77e00640 1 --2- 172.21.15.89:0/3311352946 >> [v2:172.21.15.111:6816/652666376,v1:172.21.15.111:6817/652666376] conn(0x7f2d5c0adde0 0x7f2d5c0b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:28.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.203+0000 7f2d77e00640 1 -- 172.21.15.89:0/3311352946 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2d5c0a2c10 msgr2=0x7f2d5c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:28.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.203+0000 7f2d77e00640 1 --2- 172.21.15.89:0/3311352946 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2d5c0a2c10 0x7f2d5c0a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009b50 tx=0x7f2d68005f90 comp rx=0 tx=0).stop 2024-09-10T08:38:28.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.203+0000 7f2d77e00640 1 -- 172.21.15.89:0/3311352946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d78102360 msgr2=0x7f2d7819a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:28.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.203+0000 7f2d77e00640 1 --2- 172.21.15.89:0/3311352946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d78102360 0x7f2d7819a9d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000ea60 tx=0x7f2d6000ef20 comp rx=0 tx=0).stop 2024-09-10T08:38:28.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.203+0000 7f2d77400640 1 -- 172.21.15.89:0/3311352946 reap_dead start 2024-09-10T08:38:28.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.203+0000 7f2d77e00640 1 -- 172.21.15.89:0/3311352946 shutdown_connections 2024-09-10T08:38:28.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.203+0000 7f2d77e00640 1 -- 172.21.15.89:0/3311352946 >> 172.21.15.89:0/3311352946 conn(0x7f2d780fd740 msgr2=0x7f2d780fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:28.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.203+0000 7f2d77e00640 1 -- 172.21.15.89:0/3311352946 shutdown_connections 2024-09-10T08:38:28.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:28.203+0000 7f2d77e00640 1 -- 172.21.15.89:0/3311352946 wait complete. 2024-09-10T08:38:28.315 INFO:teuthology.orchestra.run.smithi089.stdout:111669149769 2024-09-10T08:38:28.316 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T08:38:28.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.724+0000 7f6035200640 1 -- 172.21.15.89:0/1585395144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6030102bf0 msgr2=0x7f6030102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.724+0000 7f6035200640 1 --2- 172.21.15.89:0/1585395144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6030102bf0 0x7f6030102fd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6020009900 tx=0x7f602002a480 comp rx=0 tx=0).stop 2024-09-10T08:38:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.725+0000 7f6035200640 1 -- 172.21.15.89:0/1585395144 shutdown_connections 2024-09-10T08:38:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.725+0000 7f6035200640 1 --2- 172.21.15.89:0/1585395144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f603010bfd0 0x7f603010e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.725+0000 7f6035200640 1 --2- 172.21.15.89:0/1585395144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6030103510 0x7f603010ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.725+0000 7f6035200640 1 --2- 172.21.15.89:0/1585395144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6030102bf0 0x7f6030102fd0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.725+0000 7f6035200640 1 -- 172.21.15.89:0/1585395144 >> 172.21.15.89:0/1585395144 conn(0x7f60300fc610 msgr2=0x7f60300fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.725+0000 7f6035200640 1 -- 172.21.15.89:0/1585395144 shutdown_connections 2024-09-10T08:38:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.725+0000 7f6035200640 1 -- 172.21.15.89:0/1585395144 wait complete. 2024-09-10T08:38:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.726+0000 7f6035200640 1 Processor -- start 2024-09-10T08:38:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.726+0000 7f6035200640 1 -- start start 2024-09-10T08:38:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.726+0000 7f6035200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6030102bf0 0x7f603019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.726+0000 7f6035200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6030103510 0x7f603019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:28.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.726+0000 7f6035200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f603010bfd0 0x7f603019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.726+0000 7f6035200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6030111e00 con 0x7f6030103510 2024-09-10T08:38:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.726+0000 7f6035200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6030111c80 con 0x7f6030102bf0 2024-09-10T08:38:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.726+0000 7f6035200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6030111f80 con 0x7f603010bfd0 2024-09-10T08:38:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.726+0000 7f602ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6030103510 0x7f603019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.727+0000 7f602ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6030103510 0x7f603019a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57766/0 (socket says 172.21.15.89:57766) 2024-09-10T08:38:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.727+0000 7f602ea00640 1 -- 172.21.15.89:0/108974815 learned_addr learned my addr 172.21.15.89:0/108974815 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.727+0000 7f602fe00640 1 --2- 172.21.15.89:0/108974815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f603010bfd0 0x7f603019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.727+0000 7f602f400640 1 --2- 172.21.15.89:0/108974815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6030102bf0 0x7f603019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.727+0000 7f602ea00640 1 -- 172.21.15.89:0/108974815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f603010bfd0 msgr2=0x7f603019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.727+0000 7f602ea00640 1 --2- 172.21.15.89:0/108974815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f603010bfd0 0x7f603019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.727+0000 7f602ea00640 1 -- 172.21.15.89:0/108974815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6030102bf0 msgr2=0x7f603019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:28.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.727+0000 7f602ea00640 1 --2- 172.21.15.89:0/108974815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6030102bf0 0x7f603019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.727+0000 7f602ea00640 1 -- 172.21.15.89:0/108974815 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603019f420 con 0x7f6030103510 2024-09-10T08:38:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.727+0000 7f602fe00640 1 --2- 172.21.15.89:0/108974815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f603010bfd0 0x7f603019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:38:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.727+0000 7f602ea00640 1 --2- 172.21.15.89:0/108974815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6030103510 0x7f603019a9b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f601c00e9f0 tx=0x7f601c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.728+0000 7f6027e00640 1 -- 172.21.15.89:0/108974815 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c00cd80 con 0x7f6030103510 2024-09-10T08:38:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.728+0000 7f6027e00640 1 -- 172.21.15.89:0/108974815 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f601c010910 con 0x7f6030103510 2024-09-10T08:38:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.728+0000 7f6035200640 1 -- 172.21.15.89:0/108974815 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603019f710 con 0x7f6030103510 2024-09-10T08:38:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.728+0000 7f6027e00640 1 -- 172.21.15.89:0/108974815 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c018990 con 0x7f6030103510 2024-09-10T08:38:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.728+0000 7f6035200640 1 -- 172.21.15.89:0/108974815 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60301a6ff0 con 0x7f6030103510 2024-09-10T08:38:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.729+0000 7f6035200640 1 -- 172.21.15.89:0/108974815 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6030102300 con 0x7f6030103510 2024-09-10T08:38:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.733+0000 7f6027e00640 1 -- 172.21.15.89:0/108974815 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f601c010ac0 con 0x7f6030103510 2024-09-10T08:38:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.735+0000 7f6027e00640 1 --2- 172.21.15.89:0/108974815 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f60140a2c90 0x7f60140a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.735+0000 7f6027e00640 1 -- 172.21.15.89:0/108974815 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f601c0cfed0 con 0x7f6030103510 2024-09-10T08:38:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.735+0000 7f602f400640 1 --2- 172.21.15.89:0/108974815 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f60140a2c90 0x7f60140a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:28.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.735+0000 7f6027e00640 1 --2- 172.21.15.89:0/108974815 >> [v2:172.21.15.111:6808/316717323,v1:172.21.15.111:6809/316717323] conn(0x7f60140ade60 0x7f60140b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.735+0000 7f6027e00640 1 -- 172.21.15.89:0/108974815 --> [v2:172.21.15.111:6808/316717323,v1:172.21.15.111:6809/316717323] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f601c010760 con 0x7f60140ade60 2024-09-10T08:38:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.735+0000 7f6027e00640 1 -- 172.21.15.89:0/108974815 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f601c0d0280 con 0x7f6030103510 2024-09-10T08:38:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.736+0000 7f602fe00640 1 --2- 172.21.15.89:0/108974815 >> [v2:172.21.15.111:6808/316717323,v1:172.21.15.111:6809/316717323] conn(0x7f60140ade60 0x7f60140b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.736+0000 7f602f400640 1 --2- 172.21.15.89:0/108974815 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f60140a2c90 0x7f60140a5150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6020000c00 tx=0x7f6020005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.736+0000 7f602fe00640 1 --2- 172.21.15.89:0/108974815 >> [v2:172.21.15.111:6808/316717323,v1:172.21.15.111:6809/316717323] conn(0x7f60140ade60 0x7f60140b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:28.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.737+0000 7f6027e00640 1 -- 172.21.15.89:0/108974815 <== osd.4 v2:172.21.15.111:6808/316717323 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f601c010760 con 0x7f60140ade60 2024-09-10T08:38:28.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.753+0000 7f6035200640 1 -- 172.21.15.89:0/108974815 --> [v2:172.21.15.111:6808/316717323,v1:172.21.15.111:6809/316717323] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f603006a7f0 con 0x7f60140ade60 2024-09-10T08:38:28.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.753+0000 7f6027e00640 1 -- 172.21.15.89:0/108974815 <== osd.4 v2:172.21.15.111:6808/316717323 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f603006a7f0 con 0x7f60140ade60 2024-09-10T08:38:28.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.754+0000 7f6035200640 1 -- 172.21.15.89:0/108974815 >> [v2:172.21.15.111:6808/316717323,v1:172.21.15.111:6809/316717323] conn(0x7f60140ade60 msgr2=0x7f60140b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:28.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.754+0000 7f6035200640 1 --2- 172.21.15.89:0/108974815 >> [v2:172.21.15.111:6808/316717323,v1:172.21.15.111:6809/316717323] conn(0x7f60140ade60 0x7f60140b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:28.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.754+0000 7f6035200640 1 -- 172.21.15.89:0/108974815 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f60140a2c90 msgr2=0x7f60140a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:28.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.754+0000 7f6035200640 1 --2- 172.21.15.89:0/108974815 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f60140a2c90 0x7f60140a5150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6020000c00 tx=0x7f6020005b50 comp rx=0 tx=0).stop 2024-09-10T08:38:28.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.754+0000 7f6035200640 1 -- 172.21.15.89:0/108974815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6030103510 msgr2=0x7f603019a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:28.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.754+0000 7f6035200640 1 --2- 172.21.15.89:0/108974815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6030103510 0x7f603019a9b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f601c00e9f0 tx=0x7f601c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:38:28.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.754+0000 7f602fe00640 1 -- 172.21.15.89:0/108974815 reap_dead start 2024-09-10T08:38:28.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.754+0000 7f6035200640 1 -- 172.21.15.89:0/108974815 shutdown_connections 2024-09-10T08:38:28.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.754+0000 7f6035200640 1 -- 172.21.15.89:0/108974815 >> 172.21.15.89:0/108974815 conn(0x7f60300fc610 msgr2=0x7f603010d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:28.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.754+0000 7f6035200640 1 -- 172.21.15.89:0/108974815 shutdown_connections 2024-09-10T08:38:28.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:23.754+0000 7f6035200640 1 -- 172.21.15.89:0/108974815 wait complete. 2024-09-10T08:38:28.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:28 smithi149 ceph-mon[36167]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-09-10T08:38:29.020 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:28 smithi089 ceph-mon[30757]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-09-10T08:38:29.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:28 smithi111 ceph-mon[35220]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-09-10T08:38:29.839 INFO:teuthology.orchestra.run.smithi089.stdout:219043332150 2024-09-10T08:38:29.840 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T08:38:30.264 INFO:teuthology.orchestra.run.smithi089.stdout:55834574933 2024-09-10T08:38:30.264 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T08:38:30.742 INFO:teuthology.orchestra.run.smithi089.stdout:141733920836 2024-09-10T08:38:30.742 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T08:38:30.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:30 smithi149 ceph-mon[36167]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-10T08:38:31.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:30 smithi111 ceph-mon[35220]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-10T08:38:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:30 smithi089 ceph-mon[30757]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-10T08:38:31.201 INFO:teuthology.orchestra.run.smithi089.stdout:283467841579 2024-09-10T08:38:31.202 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-10T08:38:31.569 INFO:teuthology.orchestra.run.smithi089.stdout:77309411408 2024-09-10T08:38:32.018 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T08:38:32.529 INFO:teuthology.orchestra.run.smithi089.stdout:197568495675 2024-09-10T08:38:32.529 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T08:38:32.784 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:32 smithi089 ceph-mon[30757]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T08:38:32.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.850+0000 7f1b51600640 1 -- 172.21.15.89:0/3019157638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b4c1091f0 msgr2=0x7f1b4c1095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:32.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.850+0000 7f1b51600640 1 --2- 172.21.15.89:0/3019157638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b4c1091f0 0x7f1b4c1095d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009ca0 tx=0x7f1b3c02a940 comp rx=0 tx=0).stop 2024-09-10T08:38:32.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.850+0000 7f1b51600640 1 -- 172.21.15.89:0/3019157638 shutdown_connections 2024-09-10T08:38:32.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.850+0000 7f1b51600640 1 --2- 172.21.15.89:0/3019157638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b4c114ab0 0x7f1b4c116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:32.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.850+0000 7f1b51600640 1 --2- 172.21.15.89:0/3019157638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b4c109b10 0x7f1b4c114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:32.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.850+0000 7f1b51600640 1 --2- 172.21.15.89:0/3019157638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b4c1091f0 0x7f1b4c1095d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:32.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.850+0000 7f1b51600640 1 -- 172.21.15.89:0/3019157638 >> 172.21.15.89:0/3019157638 conn(0x7f1b4c06e140 msgr2=0x7f1b4c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:32.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.851+0000 7f1b51600640 1 -- 172.21.15.89:0/3019157638 shutdown_connections 2024-09-10T08:38:32.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.852+0000 7f1b51600640 1 -- 172.21.15.89:0/3019157638 wait complete. 2024-09-10T08:38:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.853+0000 7f1b51600640 1 Processor -- start 2024-09-10T08:38:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.853+0000 7f1b51600640 1 -- start start 2024-09-10T08:38:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.854+0000 7f1b51600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b4c1091f0 0x7f1b4c1a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.854+0000 7f1b51600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b4c109b10 0x7f1b4c1a6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.854+0000 7f1b51600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b4c114ab0 0x7f1b4c1a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:32.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.854+0000 7f1b51600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c11a7f0 con 0x7f1b4c109b10 2024-09-10T08:38:32.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.854+0000 7f1b51600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c11a670 con 0x7f1b4c1091f0 2024-09-10T08:38:32.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.854+0000 7f1b51600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c11a970 con 0x7f1b4c114ab0 2024-09-10T08:38:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.855+0000 7f1b4b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b4c114ab0 0x7f1b4c1a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.855+0000 7f1b4b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b4c114ab0 0x7f1b4c1a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:48962/0 (socket says 172.21.15.89:48962) 2024-09-10T08:38:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.855+0000 7f1b4b400640 1 -- 172.21.15.89:0/563783762 learned_addr learned my addr 172.21.15.89:0/563783762 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:32.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.855+0000 7f1b4aa00640 1 --2- 172.21.15.89:0/563783762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b4c1091f0 0x7f1b4c1a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:32.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.855+0000 7f1b42000640 1 --2- 172.21.15.89:0/563783762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b4c109b10 0x7f1b4c1a6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:32.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.855+0000 7f1b4b400640 1 -- 172.21.15.89:0/563783762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b4c1091f0 msgr2=0x7f1b4c1a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:32.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.856+0000 7f1b4b400640 1 --2- 172.21.15.89:0/563783762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1b4c1091f0 0x7f1b4c1a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:32.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.856+0000 7f1b4b400640 1 -- 172.21.15.89:0/563783762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b4c109b10 msgr2=0x7f1b4c1a6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:32.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.856+0000 7f1b4b400640 1 --2- 172.21.15.89:0/563783762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b4c109b10 0x7f1b4c1a6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:32.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.856+0000 7f1b4b400640 1 -- 172.21.15.89:0/563783762 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4c1a11d0 con 0x7f1b4c114ab0 2024-09-10T08:38:32.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.856+0000 7f1b42000640 1 --2- 172.21.15.89:0/563783762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b4c109b10 0x7f1b4c1a6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:38:32.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.857+0000 7f1b4b400640 1 --2- 172.21.15.89:0/563783762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b4c114ab0 0x7f1b4c1a0aa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400c970 tx=0x7f1b4400ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:32.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.857+0000 7f1b43e00640 1 -- 172.21.15.89:0/563783762 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b44007bb0 con 0x7f1b4c114ab0 2024-09-10T08:38:32.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.857+0000 7f1b43e00640 1 -- 172.21.15.89:0/563783762 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b44007d50 con 0x7f1b4c114ab0 2024-09-10T08:38:32.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.857+0000 7f1b43e00640 1 -- 172.21.15.89:0/563783762 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b44018770 con 0x7f1b4c114ab0 2024-09-10T08:38:32.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.857+0000 7f1b51600640 1 -- 172.21.15.89:0/563783762 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4c1a14c0 con 0x7f1b4c114ab0 2024-09-10T08:38:32.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.858+0000 7f1b51600640 1 -- 172.21.15.89:0/563783762 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4c10a300 con 0x7f1b4c114ab0 2024-09-10T08:38:32.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.859+0000 7f1b51600640 1 -- 172.21.15.89:0/563783762 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1b4c0712c0 con 0x7f1b4c114ab0 2024-09-10T08:38:32.866 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:32.866 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:32.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.864+0000 7f1b43e00640 1 -- 172.21.15.89:0/563783762 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1b4400f450 con 0x7f1b4c114ab0 2024-09-10T08:38:32.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.867+0000 7f1b43e00640 1 --2- 172.21.15.89:0/563783762 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1b300a2d10 0x7f1b300a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:32.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.867+0000 7f1b43e00640 1 -- 172.21.15.89:0/563783762 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f1b440cf810 con 0x7f1b4c114ab0 2024-09-10T08:38:32.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.867+0000 7f1b4aa00640 1 --2- 172.21.15.89:0/563783762 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1b300a2d10 0x7f1b300a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.867+0000 7f1b43e00640 1 --2- 172.21.15.89:0/563783762 >> [v2:172.21.15.89:6802/1760523284,v1:172.21.15.89:6803/1760523284] conn(0x7f1b300adee0 0x7f1b300b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.867+0000 7f1b43e00640 1 -- 172.21.15.89:0/563783762 --> [v2:172.21.15.89:6802/1760523284,v1:172.21.15.89:6803/1760523284] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1b4400a0a0 con 0x7f1b300adee0 2024-09-10T08:38:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.867+0000 7f1b43e00640 1 -- 172.21.15.89:0/563783762 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f1b440cfc00 con 0x7f1b4c114ab0 2024-09-10T08:38:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.868+0000 7f1b42000640 1 --2- 172.21.15.89:0/563783762 >> [v2:172.21.15.89:6802/1760523284,v1:172.21.15.89:6803/1760523284] conn(0x7f1b300adee0 0x7f1b300b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.869+0000 7f1b42000640 1 --2- 172.21.15.89:0/563783762 >> [v2:172.21.15.89:6802/1760523284,v1:172.21.15.89:6803/1760523284] conn(0x7f1b300adee0 0x7f1b300b0380 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.870+0000 7f1b43e00640 1 -- 172.21.15.89:0/563783762 <== osd.0 v2:172.21.15.89:6802/1760523284 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b4400a0a0 con 0x7f1b300adee0 2024-09-10T08:38:32.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.872+0000 7f1b4aa00640 1 --2- 172.21.15.89:0/563783762 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1b300a2d10 0x7f1b300a51d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c1a1be0 tx=0x7f1b3c005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:32.877 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:32.898 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:32.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.919+0000 7f1b51600640 1 -- 172.21.15.89:0/563783762 --> [v2:172.21.15.89:6802/1760523284,v1:172.21.15.89:6803/1760523284] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1b4c073a40 con 0x7f1b300adee0 2024-09-10T08:38:32.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.920+0000 7f1b43e00640 1 -- 172.21.15.89:0/563783762 <== osd.0 v2:172.21.15.89:6802/1760523284 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1b4c073a40 con 0x7f1b300adee0 2024-09-10T08:38:32.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.920+0000 7f1b51600640 1 -- 172.21.15.89:0/563783762 >> [v2:172.21.15.89:6802/1760523284,v1:172.21.15.89:6803/1760523284] conn(0x7f1b300adee0 msgr2=0x7f1b300b0380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:32.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.920+0000 7f1b51600640 1 --2- 172.21.15.89:0/563783762 >> [v2:172.21.15.89:6802/1760523284,v1:172.21.15.89:6803/1760523284] conn(0x7f1b300adee0 0x7f1b300b0380 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:32.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.921+0000 7f1b51600640 1 -- 172.21.15.89:0/563783762 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1b300a2d10 msgr2=0x7f1b300a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:32.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.921+0000 7f1b51600640 1 --2- 172.21.15.89:0/563783762 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1b300a2d10 0x7f1b300a51d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c1a1be0 tx=0x7f1b3c005d80 comp rx=0 tx=0).stop 2024-09-10T08:38:32.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.921+0000 7f1b51600640 1 -- 172.21.15.89:0/563783762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b4c114ab0 msgr2=0x7f1b4c1a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:32.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.921+0000 7f1b51600640 1 --2- 172.21.15.89:0/563783762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b4c114ab0 0x7f1b4c1a0aa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400c970 tx=0x7f1b4400ce30 comp rx=0 tx=0).stop 2024-09-10T08:38:32.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.921+0000 7f1b4b400640 1 -- 172.21.15.89:0/563783762 reap_dead start 2024-09-10T08:38:32.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.921+0000 7f1b51600640 1 -- 172.21.15.89:0/563783762 shutdown_connections 2024-09-10T08:38:32.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.921+0000 7f1b51600640 1 -- 172.21.15.89:0/563783762 >> 172.21.15.89:0/563783762 conn(0x7f1b4c06e140 msgr2=0x7f1b4c070860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:32.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.921+0000 7f1b51600640 1 -- 172.21.15.89:0/563783762 shutdown_connections 2024-09-10T08:38:32.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:32.921+0000 7f1b51600640 1 -- 172.21.15.89:0/563783762 wait complete. 2024-09-10T08:38:32.947 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:32.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:32 smithi149 ceph-mon[36167]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T08:38:33.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:32 smithi111 ceph-mon[35220]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T08:38:34.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.066+0000 7f9f7d200640 1 -- 172.21.15.89:0/1483562253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f78074f40 msgr2=0x7f9f7810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:34.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.066+0000 7f9f7d200640 1 --2- 172.21.15.89:0/1483562253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f78074f40 0x7f9f7810de80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9f64009900 tx=0x7f9f6402a480 comp rx=0 tx=0).stop 2024-09-10T08:38:34.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.066+0000 7f9f7d200640 1 -- 172.21.15.89:0/1483562253 shutdown_connections 2024-09-10T08:38:34.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.066+0000 7f9f7d200640 1 --2- 172.21.15.89:0/1483562253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f7810e3c0 0x7f9f78110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.066+0000 7f9f7d200640 1 --2- 172.21.15.89:0/1483562253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f78074f40 0x7f9f7810de80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.066+0000 7f9f7d200640 1 --2- 172.21.15.89:0/1483562253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f78074620 0x7f9f78074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.066+0000 7f9f7d200640 1 -- 172.21.15.89:0/1483562253 >> 172.21.15.89:0/1483562253 conn(0x7f9f780fd8c0 msgr2=0x7f9f780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.066+0000 7f9f7d200640 1 -- 172.21.15.89:0/1483562253 shutdown_connections 2024-09-10T08:38:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.066+0000 7f9f7d200640 1 -- 172.21.15.89:0/1483562253 wait complete. 2024-09-10T08:38:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.068+0000 7f9f7d200640 1 Processor -- start 2024-09-10T08:38:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.068+0000 7f9f7d200640 1 -- start start 2024-09-10T08:38:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.068+0000 7f9f7d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f78074620 0x7f9f7819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.068+0000 7f9f7d200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f78074f40 0x7f9f7819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.068+0000 7f9f7d200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f7810e3c0 0x7f9f7819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.068+0000 7f9f7d200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9f781141b0 con 0x7f9f78074620 2024-09-10T08:38:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.068+0000 7f9f7d200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9f78114030 con 0x7f9f7810e3c0 2024-09-10T08:38:34.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.068+0000 7f9f7d200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9f78114330 con 0x7f9f78074f40 2024-09-10T08:38:34.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.069+0000 7f9f75e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f78074620 0x7f9f7819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:34.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.069+0000 7f9f75400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f78074f40 0x7f9f7819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:34.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.069+0000 7f9f75400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f78074f40 0x7f9f7819abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:48990/0 (socket says 172.21.15.89:48990) 2024-09-10T08:38:34.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.069+0000 7f9f76800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f7810e3c0 0x7f9f7819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:34.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.069+0000 7f9f75400640 1 -- 172.21.15.89:0/2998805855 learned_addr learned my addr 172.21.15.89:0/2998805855 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:34.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.069+0000 7f9f76800640 1 -- 172.21.15.89:0/2998805855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f78074f40 msgr2=0x7f9f7819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:34.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.069+0000 7f9f76800640 1 --2- 172.21.15.89:0/2998805855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f78074f40 0x7f9f7819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:34.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.069+0000 7f9f76800640 1 -- 172.21.15.89:0/2998805855 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f78074620 msgr2=0x7f9f7819a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:34.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.069+0000 7f9f76800640 1 --2- 172.21.15.89:0/2998805855 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f78074620 0x7f9f7819a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:34.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.069+0000 7f9f76800640 1 -- 172.21.15.89:0/2998805855 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7819f660 con 0x7f9f7810e3c0 2024-09-10T08:38:34.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.069+0000 7f9f76800640 1 --2- 172.21.15.89:0/2998805855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f7810e3c0 0x7f9f7819ef80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000ec50 tx=0x7f9f7000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:34.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.070+0000 7f9f6ea00640 1 -- 172.21.15.89:0/2998805855 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70003b70 con 0x7f9f7810e3c0 2024-09-10T08:38:34.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.070+0000 7f9f6ea00640 1 -- 172.21.15.89:0/2998805855 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f70003d10 con 0x7f9f7810e3c0 2024-09-10T08:38:34.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.070+0000 7f9f6ea00640 1 -- 172.21.15.89:0/2998805855 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70010890 con 0x7f9f7810e3c0 2024-09-10T08:38:34.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.070+0000 7f9f7d200640 1 -- 172.21.15.89:0/2998805855 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f7819f950 con 0x7f9f7810e3c0 2024-09-10T08:38:34.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.070+0000 7f9f7d200640 1 -- 172.21.15.89:0/2998805855 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f781a7230 con 0x7f9f7810e3c0 2024-09-10T08:38:34.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.071+0000 7f9f7d200640 1 -- 172.21.15.89:0/2998805855 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f78074a00 con 0x7f9f7810e3c0 2024-09-10T08:38:34.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.073+0000 7f9f6ea00640 1 -- 172.21.15.89:0/2998805855 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9f70004b40 con 0x7f9f7810e3c0 2024-09-10T08:38:34.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.074+0000 7f9f6ea00640 1 --2- 172.21.15.89:0/2998805855 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9f5c0a2c10 0x7f9f5c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:34.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.074+0000 7f9f6ea00640 1 -- 172.21.15.89:0/2998805855 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f9f700cfb80 con 0x7f9f7810e3c0 2024-09-10T08:38:34.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.075+0000 7f9f75e00640 1 --2- 172.21.15.89:0/2998805855 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9f5c0a2c10 0x7f9f5c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:34.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.075+0000 7f9f75e00640 1 --2- 172.21.15.89:0/2998805855 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9f5c0a2c10 0x7f9f5c0a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009bd0 tx=0x7f9f68005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:34.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.076+0000 7f9f6ea00640 1 -- 172.21.15.89:0/2998805855 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9f7009c650 con 0x7f9f7810e3c0 2024-09-10T08:38:34.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.274+0000 7f9f7d200640 1 -- 172.21.15.89:0/2998805855 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f9f78108fb0 con 0x7f9f7810e3c0 2024-09-10T08:38:34.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.275+0000 7f9f6ea00640 1 -- 172.21.15.89:0/2998805855 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9f7009bf50 con 0x7f9f7810e3c0 2024-09-10T08:38:34.276 INFO:teuthology.orchestra.run.smithi089.stdout:141733920838 2024-09-10T08:38:34.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.277+0000 7f9f7d200640 1 -- 172.21.15.89:0/2998805855 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9f5c0a2c10 msgr2=0x7f9f5c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:34.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.277+0000 7f9f7d200640 1 --2- 172.21.15.89:0/2998805855 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9f5c0a2c10 0x7f9f5c0a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009bd0 tx=0x7f9f68005f90 comp rx=0 tx=0).stop 2024-09-10T08:38:34.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.277+0000 7f9f7d200640 1 -- 172.21.15.89:0/2998805855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f7810e3c0 msgr2=0x7f9f7819ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:34.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.277+0000 7f9f7d200640 1 --2- 172.21.15.89:0/2998805855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f7810e3c0 0x7f9f7819ef80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000ec50 tx=0x7f9f7000c6a0 comp rx=0 tx=0).stop 2024-09-10T08:38:34.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.277+0000 7f9f7d200640 1 -- 172.21.15.89:0/2998805855 shutdown_connections 2024-09-10T08:38:34.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.277+0000 7f9f7d200640 1 --2- 172.21.15.89:0/2998805855 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9f5c0a2c10 0x7f9f5c0a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:34.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.277+0000 7f9f7d200640 1 --2- 172.21.15.89:0/2998805855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f7810e3c0 0x7f9f7819ef80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:34.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.277+0000 7f9f7d200640 1 --2- 172.21.15.89:0/2998805855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f78074f40 0x7f9f7819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:34.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.277+0000 7f9f7d200640 1 --2- 172.21.15.89:0/2998805855 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9f78074620 0x7f9f7819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:34.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.277+0000 7f9f7d200640 1 -- 172.21.15.89:0/2998805855 >> 172.21.15.89:0/2998805855 conn(0x7f9f780fd8c0 msgr2=0x7f9f780fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:34.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.278+0000 7f9f7d200640 1 -- 172.21.15.89:0/2998805855 shutdown_connections 2024-09-10T08:38:34.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:34.278+0000 7f9f7d200640 1 -- 172.21.15.89:0/2998805855 wait complete. 2024-09-10T08:38:34.755 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:34 smithi089 ceph-mon[30757]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:38:34.755 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:34 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2998805855' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T08:38:34.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:34 smithi149 ceph-mon[36167]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:38:34.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:34 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2998805855' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T08:38:35.012 INFO:teuthology.orchestra.run.smithi089.stdout:34359738459 2024-09-10T08:38:35.013 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T08:38:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:34 smithi111 ceph-mon[35220]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:38:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:34 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2998805855' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T08:38:35.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.776+0000 7fdf57c00640 1 -- 172.21.15.89:0/2634864645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf5010d530 msgr2=0x7fdf501150b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:35.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.776+0000 7fdf57c00640 1 --2- 172.21.15.89:0/2634864645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf5010d530 0x7fdf501150b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009ca0 tx=0x7fdf4802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:38:35.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.777+0000 7fdf57c00640 1 -- 172.21.15.89:0/2634864645 shutdown_connections 2024-09-10T08:38:35.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.777+0000 7fdf57c00640 1 --2- 172.21.15.89:0/2634864645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf5010d530 0x7fdf501150b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:35.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.777+0000 7fdf57c00640 1 --2- 172.21.15.89:0/2634864645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf50108e00 0x7fdf5010ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:35.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.777+0000 7fdf57c00640 1 --2- 172.21.15.89:0/2634864645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf50108450 0x7fdf50108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:35.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.777+0000 7fdf57c00640 1 -- 172.21.15.89:0/2634864645 >> 172.21.15.89:0/2634864645 conn(0x7fdf50077ba0 msgr2=0x7fdf50079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:35.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.777+0000 7fdf57c00640 1 -- 172.21.15.89:0/2634864645 shutdown_connections 2024-09-10T08:38:35.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.777+0000 7fdf57c00640 1 -- 172.21.15.89:0/2634864645 wait complete. 2024-09-10T08:38:35.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.778+0000 7fdf57c00640 1 Processor -- start 2024-09-10T08:38:35.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.779+0000 7fdf57c00640 1 -- start start 2024-09-10T08:38:35.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.779+0000 7fdf57c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf50108450 0x7fdf501a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:35.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.779+0000 7fdf57c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf50108e00 0x7fdf501a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:35.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.779+0000 7fdf57c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf5010d530 0x7fdf5019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:35.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.779+0000 7fdf57c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdf501185b0 con 0x7fdf50108e00 2024-09-10T08:38:35.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.779+0000 7fdf57c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdf50118430 con 0x7fdf5010d530 2024-09-10T08:38:35.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.779+0000 7fdf57c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdf50118730 con 0x7fdf50108450 2024-09-10T08:38:35.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.779+0000 7fdf55200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf50108450 0x7fdf501a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:35.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.779+0000 7fdf4fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf50108e00 0x7fdf501a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:35.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.779+0000 7fdf55200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf50108450 0x7fdf501a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:49010/0 (socket says 172.21.15.89:49010) 2024-09-10T08:38:35.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.779+0000 7fdf55200640 1 -- 172.21.15.89:0/3728653511 learned_addr learned my addr 172.21.15.89:0/3728653511 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:35.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.780+0000 7fdf55c00640 1 --2- 172.21.15.89:0/3728653511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf5010d530 0x7fdf5019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:35.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.780+0000 7fdf55200640 1 -- 172.21.15.89:0/3728653511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf5010d530 msgr2=0x7fdf5019e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.780+0000 7fdf55200640 1 --2- 172.21.15.89:0/3728653511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf5010d530 0x7fdf5019e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.780+0000 7fdf55200640 1 -- 172.21.15.89:0/3728653511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf50108e00 msgr2=0x7fdf501a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.780+0000 7fdf55200640 1 --2- 172.21.15.89:0/3728653511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf50108e00 0x7fdf501a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.780+0000 7fdf55200640 1 -- 172.21.15.89:0/3728653511 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5019efc0 con 0x7fdf50108450 2024-09-10T08:38:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.780+0000 7fdf55c00640 1 --2- 172.21.15.89:0/3728653511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf5010d530 0x7fdf5019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:38:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.780+0000 7fdf4fe00640 1 --2- 172.21.15.89:0/3728653511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf50108e00 0x7fdf501a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:38:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.781+0000 7fdf55200640 1 --2- 172.21.15.89:0/3728653511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf50108450 0x7fdf501a46a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400d950 tx=0x7fdf4400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:35.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.781+0000 7fdf4d600640 1 -- 172.21.15.89:0/3728653511 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf44017030 con 0x7fdf50108450 2024-09-10T08:38:35.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.781+0000 7fdf4d600640 1 -- 172.21.15.89:0/3728653511 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf44004020 con 0x7fdf50108450 2024-09-10T08:38:35.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.781+0000 7fdf57c00640 1 -- 172.21.15.89:0/3728653511 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5019f2b0 con 0x7fdf50108450 2024-09-10T08:38:35.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.781+0000 7fdf4d600640 1 -- 172.21.15.89:0/3728653511 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf440047a0 con 0x7fdf50108450 2024-09-10T08:38:35.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.781+0000 7fdf57c00640 1 -- 172.21.15.89:0/3728653511 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf501ab500 con 0x7fdf50108450 2024-09-10T08:38:35.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.783+0000 7fdf57c00640 1 -- 172.21.15.89:0/3728653511 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf501098e0 con 0x7fdf50108450 2024-09-10T08:38:35.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.785+0000 7fdf4d600640 1 -- 172.21.15.89:0/3728653511 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdf440041d0 con 0x7fdf50108450 2024-09-10T08:38:35.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.787+0000 7fdf4d600640 1 --2- 172.21.15.89:0/3728653511 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdf380a2c90 0x7fdf380a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:35.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.787+0000 7fdf4d600640 1 -- 172.21.15.89:0/3728653511 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fdf440ceef0 con 0x7fdf50108450 2024-09-10T08:38:35.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.787+0000 7fdf4fe00640 1 --2- 172.21.15.89:0/3728653511 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdf380a2c90 0x7fdf380a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:35.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.788+0000 7fdf4fe00640 1 --2- 172.21.15.89:0/3728653511 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdf380a2c90 0x7fdf380a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdf50107f50 tx=0x7fdf3c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:35.791 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.791+0000 7fdf4d600640 1 -- 172.21.15.89:0/3728653511 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf4409b930 con 0x7fdf50108450 2024-09-10T08:38:35.926 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:35.949 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920836 got 141733920838 for osd.4 2024-09-10T08:38:35.950 DEBUG:teuthology.parallel:result is None 2024-09-10T08:38:35.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.978+0000 7fdf57c00640 1 -- 172.21.15.89:0/3728653511 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fdf50108830 con 0x7fdf50108450 2024-09-10T08:38:35.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.979+0000 7fdf4d600640 1 -- 172.21.15.89:0/3728653511 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdf4409b230 con 0x7fdf50108450 2024-09-10T08:38:35.979 INFO:teuthology.orchestra.run.smithi089.stdout:111669149771 2024-09-10T08:38:35.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.981+0000 7fdf57c00640 1 -- 172.21.15.89:0/3728653511 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdf380a2c90 msgr2=0x7fdf380a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:35.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.981+0000 7fdf57c00640 1 --2- 172.21.15.89:0/3728653511 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdf380a2c90 0x7fdf380a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdf50107f50 tx=0x7fdf3c005f90 comp rx=0 tx=0).stop 2024-09-10T08:38:35.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.981+0000 7fdf57c00640 1 -- 172.21.15.89:0/3728653511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf50108450 msgr2=0x7fdf501a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:35.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.981+0000 7fdf57c00640 1 --2- 172.21.15.89:0/3728653511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf50108450 0x7fdf501a46a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400d950 tx=0x7fdf4400de10 comp rx=0 tx=0).stop 2024-09-10T08:38:35.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.981+0000 7fdf57c00640 1 -- 172.21.15.89:0/3728653511 shutdown_connections 2024-09-10T08:38:35.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.981+0000 7fdf57c00640 1 --2- 172.21.15.89:0/3728653511 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdf380a2c90 0x7fdf380a5150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:35.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.981+0000 7fdf57c00640 1 --2- 172.21.15.89:0/3728653511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf5010d530 0x7fdf5019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:35.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.981+0000 7fdf57c00640 1 --2- 172.21.15.89:0/3728653511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf50108e00 0x7fdf501a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:35.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.981+0000 7fdf57c00640 1 --2- 172.21.15.89:0/3728653511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf50108450 0x7fdf501a46a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:35.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.981+0000 7fdf57c00640 1 -- 172.21.15.89:0/3728653511 >> 172.21.15.89:0/3728653511 conn(0x7fdf50077ba0 msgr2=0x7fdf50079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:35.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.981+0000 7fdf57c00640 1 -- 172.21.15.89:0/3728653511 shutdown_connections 2024-09-10T08:38:35.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:35.981+0000 7fdf57c00640 1 -- 172.21.15.89:0/3728653511 wait complete. 2024-09-10T08:38:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.566+0000 7fb00e600640 1 -- 172.21.15.89:0/1775669234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb008102790 msgr2=0x7fb0081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.566+0000 7fb00e600640 1 --2- 172.21.15.89:0/1775669234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb008102790 0x7fb0081067e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faff0009900 tx=0x7faff002a410 comp rx=0 tx=0).stop 2024-09-10T08:38:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.566+0000 7fb00e600640 1 -- 172.21.15.89:0/1775669234 shutdown_connections 2024-09-10T08:38:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.566+0000 7fb00e600640 1 --2- 172.21.15.89:0/1775669234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb008106f10 0x7fb00810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:36.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.566+0000 7fb00e600640 1 --2- 172.21.15.89:0/1775669234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb008102790 0x7fb0081067e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:36.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.566+0000 7fb00e600640 1 --2- 172.21.15.89:0/1775669234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb008101de0 0x7fb0081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:36.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.566+0000 7fb00e600640 1 -- 172.21.15.89:0/1775669234 >> 172.21.15.89:0/1775669234 conn(0x7fb0080fd8c0 msgr2=0x7fb0080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:36.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.566+0000 7fb00e600640 1 -- 172.21.15.89:0/1775669234 shutdown_connections 2024-09-10T08:38:36.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.566+0000 7fb00e600640 1 -- 172.21.15.89:0/1775669234 wait complete. 2024-09-10T08:38:36.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.567+0000 7fb00e600640 1 Processor -- start 2024-09-10T08:38:36.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.567+0000 7fb00e600640 1 -- start start 2024-09-10T08:38:36.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.568+0000 7fb00e600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb008101de0 0x7fb008198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:36.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.568+0000 7fb00e600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb008102790 0x7fb0081989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:36.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.568+0000 7fb00e600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb008106f10 0x7fb00819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:36.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.568+0000 7fb00e600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb008111fc0 con 0x7fb008106f10 2024-09-10T08:38:36.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.568+0000 7fb00e600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb008111e40 con 0x7fb008101de0 2024-09-10T08:38:36.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.568+0000 7fb00e600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb008112140 con 0x7fb008102790 2024-09-10T08:38:36.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.568+0000 7fb007400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb008101de0 0x7fb008198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:36.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.568+0000 7fb007400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb008101de0 0x7fb008198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:48466/0 (socket says 172.21.15.89:48466) 2024-09-10T08:38:36.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.568+0000 7faffea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb008102790 0x7fb0081989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:36.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.568+0000 7fb007400640 1 -- 172.21.15.89:0/4011349036 learned_addr learned my addr 172.21.15.89:0/4011349036 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:36.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.568+0000 7fb007e00640 1 --2- 172.21.15.89:0/4011349036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb008106f10 0x7fb00819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:36.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.569+0000 7fb007400640 1 -- 172.21.15.89:0/4011349036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb008102790 msgr2=0x7fb0081989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:36.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.569+0000 7fb007400640 1 --2- 172.21.15.89:0/4011349036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb008102790 0x7fb0081989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:36.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.569+0000 7fb007400640 1 -- 172.21.15.89:0/4011349036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb008106f10 msgr2=0x7fb00819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:36.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.569+0000 7fb007400640 1 --2- 172.21.15.89:0/4011349036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb008106f10 0x7fb00819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:36.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.569+0000 7fb007400640 1 -- 172.21.15.89:0/4011349036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00819d430 con 0x7fb008101de0 2024-09-10T08:38:36.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.569+0000 7fb007e00640 1 --2- 172.21.15.89:0/4011349036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb008106f10 0x7fb00819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:38:36.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.569+0000 7fb007400640 1 --2- 172.21.15.89:0/4011349036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb008101de0 0x7fb008198480 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faff800e9f0 tx=0x7faff800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:36.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.569+0000 7fb004c00640 1 -- 172.21.15.89:0/4011349036 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff800cd80 con 0x7fb008101de0 2024-09-10T08:38:36.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.569+0000 7fb00e600640 1 -- 172.21.15.89:0/4011349036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00819d720 con 0x7fb008101de0 2024-09-10T08:38:36.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.569+0000 7fb004c00640 1 -- 172.21.15.89:0/4011349036 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faff8010910 con 0x7fb008101de0 2024-09-10T08:38:36.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.569+0000 7fb004c00640 1 -- 172.21.15.89:0/4011349036 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff80188f0 con 0x7fb008101de0 2024-09-10T08:38:36.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.569+0000 7fb00e600640 1 -- 172.21.15.89:0/4011349036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0081a5000 con 0x7fb008101de0 2024-09-10T08:38:36.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.570+0000 7fb00e600640 1 -- 172.21.15.89:0/4011349036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0081021c0 con 0x7fb008101de0 2024-09-10T08:38:36.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.572+0000 7fb004c00640 1 -- 172.21.15.89:0/4011349036 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7faff8010ac0 con 0x7fb008101de0 2024-09-10T08:38:36.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.574+0000 7fb004c00640 1 --2- 172.21.15.89:0/4011349036 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fafec0a2c90 0x7fafec0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:36.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.575+0000 7fb004c00640 1 -- 172.21.15.89:0/4011349036 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7faff80cfd30 con 0x7fb008101de0 2024-09-10T08:38:36.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.575+0000 7faffea00640 1 --2- 172.21.15.89:0/4011349036 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fafec0a2c90 0x7fafec0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:36.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.575+0000 7faffea00640 1 --2- 172.21.15.89:0/4011349036 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fafec0a2c90 0x7fafec0a5150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faff0000c00 tx=0x7faff0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:36.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.576+0000 7fb004c00640 1 -- 172.21.15.89:0/4011349036 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faff809c770 con 0x7fb008101de0 2024-09-10T08:38:36.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.754+0000 7fb00e600640 1 -- 172.21.15.89:0/4011349036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fb00810a180 con 0x7fb008101de0 2024-09-10T08:38:36.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.755+0000 7fb004c00640 1 -- 172.21.15.89:0/4011349036 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7faff809c070 con 0x7fb008101de0 2024-09-10T08:38:36.796 INFO:teuthology.orchestra.run.smithi089.stdout:249108103218 2024-09-10T08:38:36.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.757+0000 7fb00e600640 1 -- 172.21.15.89:0/4011349036 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fafec0a2c90 msgr2=0x7fafec0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:36.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.757+0000 7fb00e600640 1 --2- 172.21.15.89:0/4011349036 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fafec0a2c90 0x7fafec0a5150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faff0000c00 tx=0x7faff0005b50 comp rx=0 tx=0).stop 2024-09-10T08:38:36.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.757+0000 7fb00e600640 1 -- 172.21.15.89:0/4011349036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb008101de0 msgr2=0x7fb008198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:36.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.757+0000 7fb00e600640 1 --2- 172.21.15.89:0/4011349036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb008101de0 0x7fb008198480 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faff800e9f0 tx=0x7faff800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:38:36.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.757+0000 7fb00e600640 1 -- 172.21.15.89:0/4011349036 shutdown_connections 2024-09-10T08:38:36.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.757+0000 7fb00e600640 1 --2- 172.21.15.89:0/4011349036 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fafec0a2c90 0x7fafec0a5150 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:36.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.757+0000 7fb00e600640 1 --2- 172.21.15.89:0/4011349036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb008106f10 0x7fb00819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:36.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.757+0000 7fb00e600640 1 --2- 172.21.15.89:0/4011349036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb008102790 0x7fb0081989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:36.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.758+0000 7fb00e600640 1 --2- 172.21.15.89:0/4011349036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb008101de0 0x7fb008198480 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:36.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.758+0000 7fb00e600640 1 -- 172.21.15.89:0/4011349036 >> 172.21.15.89:0/4011349036 conn(0x7fb0080fd8c0 msgr2=0x7fb0080febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:36.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.758+0000 7fb00e600640 1 -- 172.21.15.89:0/4011349036 shutdown_connections 2024-09-10T08:38:36.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:36.758+0000 7fb00e600640 1 -- 172.21.15.89:0/4011349036 wait complete. 2024-09-10T08:38:36.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:36 smithi149 ceph-mon[36167]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-10T08:38:36.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:36 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3728653511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T08:38:37.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:36 smithi111 ceph-mon[35220]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-10T08:38:37.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:36 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3728653511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T08:38:37.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:36 smithi089 ceph-mon[30757]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-10T08:38:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:36 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3728653511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T08:38:37.356 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:37.357 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:37.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.652+0000 7f32fcc00640 1 -- 172.21.15.89:0/2813017536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f81049a0 msgr2=0x7f32f81089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:37.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.652+0000 7f32fcc00640 1 --2- 172.21.15.89:0/2813017536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f81049a0 0x7f32f81089f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f32e4009900 tx=0x7f32e402a410 comp rx=0 tx=0).stop 2024-09-10T08:38:37.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.652+0000 7f32fcc00640 1 -- 172.21.15.89:0/2813017536 shutdown_connections 2024-09-10T08:38:37.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.652+0000 7f32fcc00640 1 --2- 172.21.15.89:0/2813017536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f8109120 0x7f32f8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:37.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.652+0000 7f32fcc00640 1 --2- 172.21.15.89:0/2813017536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f81049a0 0x7f32f81089f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:37.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.652+0000 7f32fcc00640 1 --2- 172.21.15.89:0/2813017536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32f8103ff0 0x7f32f81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:37.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.652+0000 7f32fcc00640 1 -- 172.21.15.89:0/2813017536 >> 172.21.15.89:0/2813017536 conn(0x7f32f80ffa90 msgr2=0x7f32f8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:37.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.652+0000 7f32fcc00640 1 -- 172.21.15.89:0/2813017536 shutdown_connections 2024-09-10T08:38:37.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.652+0000 7f32fcc00640 1 -- 172.21.15.89:0/2813017536 wait complete. 2024-09-10T08:38:37.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.653+0000 7f32fcc00640 1 Processor -- start 2024-09-10T08:38:37.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.653+0000 7f32fcc00640 1 -- start start 2024-09-10T08:38:37.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.654+0000 7f32fcc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f8103ff0 0x7f32f819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:37.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.654+0000 7f32fcc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32f81049a0 0x7f32f819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:37.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.654+0000 7f32fcc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f8109120 0x7f32f819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:37.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.654+0000 7f32fcc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f32f8114140 con 0x7f32f8109120 2024-09-10T08:38:37.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.654+0000 7f32fcc00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f32f8113fc0 con 0x7f32f81049a0 2024-09-10T08:38:37.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.654+0000 7f32fcc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32f81142c0 con 0x7f32f8103ff0 2024-09-10T08:38:37.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.654+0000 7f32f5400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32f81049a0 0x7f32f819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:37.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.654+0000 7f32f5e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f8103ff0 0x7f32f819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:37.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.654+0000 7f32f5400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32f81049a0 0x7f32f819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:48476/0 (socket says 172.21.15.89:48476) 2024-09-10T08:38:37.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.654+0000 7f32f5400640 1 -- 172.21.15.89:0/157851354 learned_addr learned my addr 172.21.15.89:0/157851354 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:37.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.654+0000 7f32f6800640 1 --2- 172.21.15.89:0/157851354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f8109120 0x7f32f819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:37.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.655+0000 7f32f5e00640 1 -- 172.21.15.89:0/157851354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32f81049a0 msgr2=0x7f32f819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:37.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.655+0000 7f32f5e00640 1 --2- 172.21.15.89:0/157851354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32f81049a0 0x7f32f819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:37.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.655+0000 7f32f5e00640 1 -- 172.21.15.89:0/157851354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f8109120 msgr2=0x7f32f819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:37.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.655+0000 7f32f5e00640 1 --2- 172.21.15.89:0/157851354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f8109120 0x7f32f819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:37.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.655+0000 7f32f5e00640 1 -- 172.21.15.89:0/157851354 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f819f5d0 con 0x7f32f8103ff0 2024-09-10T08:38:37.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.655+0000 7f32f6800640 1 --2- 172.21.15.89:0/157851354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f8109120 0x7f32f819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:38:37.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.656+0000 7f32f5e00640 1 --2- 172.21.15.89:0/157851354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f8103ff0 0x7f32f819a620 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f32e800e9f0 tx=0x7f32e800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:37.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.657+0000 7f32eea00640 1 -- 172.21.15.89:0/157851354 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e800ccc0 con 0x7f32f8103ff0 2024-09-10T08:38:37.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.657+0000 7f32eea00640 1 -- 172.21.15.89:0/157851354 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32e8010910 con 0x7f32f8103ff0 2024-09-10T08:38:37.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.657+0000 7f32eea00640 1 -- 172.21.15.89:0/157851354 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8018960 con 0x7f32f8103ff0 2024-09-10T08:38:37.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.657+0000 7f32fcc00640 1 -- 172.21.15.89:0/157851354 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f819f8c0 con 0x7f32f8103ff0 2024-09-10T08:38:37.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.657+0000 7f32fcc00640 1 -- 172.21.15.89:0/157851354 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f81a71a0 con 0x7f32f8103ff0 2024-09-10T08:38:37.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.658+0000 7f32fcc00640 1 -- 172.21.15.89:0/157851354 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f81043d0 con 0x7f32f8103ff0 2024-09-10T08:38:37.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.660+0000 7f32eea00640 1 -- 172.21.15.89:0/157851354 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f32e8010ac0 con 0x7f32f8103ff0 2024-09-10T08:38:37.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.661+0000 7f32eea00640 1 --2- 172.21.15.89:0/157851354 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f32dc0a2c90 0x7f32dc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:37.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.662+0000 7f32eea00640 1 -- 172.21.15.89:0/157851354 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f32e80cf880 con 0x7f32f8103ff0 2024-09-10T08:38:37.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.662+0000 7f32f5400640 1 --2- 172.21.15.89:0/157851354 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f32dc0a2c90 0x7f32dc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:37.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.662+0000 7f32f5400640 1 --2- 172.21.15.89:0/157851354 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f32dc0a2c90 0x7f32dc0a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f32e4031550 tx=0x7f32e4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:37.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.665+0000 7f32eea00640 1 -- 172.21.15.89:0/157851354 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32e809bc70 con 0x7f32f8103ff0 2024-09-10T08:38:37.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:37 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/4011349036' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T08:38:37.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.856+0000 7f32fcc00640 1 -- 172.21.15.89:0/157851354 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f32f810c3c0 con 0x7f32f8103ff0 2024-09-10T08:38:37.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.857+0000 7f32eea00640 1 -- 172.21.15.89:0/157851354 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f32e809ff70 con 0x7f32f8103ff0 2024-09-10T08:38:37.857 INFO:teuthology.orchestra.run.smithi089.stdout:304942678055 2024-09-10T08:38:37.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.859+0000 7f32fcc00640 1 -- 172.21.15.89:0/157851354 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f32dc0a2c90 msgr2=0x7f32dc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:37.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.859+0000 7f32fcc00640 1 --2- 172.21.15.89:0/157851354 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f32dc0a2c90 0x7f32dc0a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f32e4031550 tx=0x7f32e4005a20 comp rx=0 tx=0).stop 2024-09-10T08:38:37.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.859+0000 7f32fcc00640 1 -- 172.21.15.89:0/157851354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f8103ff0 msgr2=0x7f32f819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:37.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.859+0000 7f32fcc00640 1 --2- 172.21.15.89:0/157851354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f8103ff0 0x7f32f819a620 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f32e800e9f0 tx=0x7f32e800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:38:37.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.859+0000 7f32fcc00640 1 -- 172.21.15.89:0/157851354 shutdown_connections 2024-09-10T08:38:37.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.859+0000 7f32fcc00640 1 --2- 172.21.15.89:0/157851354 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f32dc0a2c90 0x7f32dc0a5150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:37.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.859+0000 7f32fcc00640 1 --2- 172.21.15.89:0/157851354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f8109120 0x7f32f819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:37.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.859+0000 7f32fcc00640 1 --2- 172.21.15.89:0/157851354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32f81049a0 0x7f32f819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:37.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.859+0000 7f32fcc00640 1 --2- 172.21.15.89:0/157851354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f8103ff0 0x7f32f819a620 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:37.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.859+0000 7f32fcc00640 1 -- 172.21.15.89:0/157851354 >> 172.21.15.89:0/157851354 conn(0x7f32f80ffa90 msgr2=0x7f32f8100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:37.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.859+0000 7f32fcc00640 1 -- 172.21.15.89:0/157851354 shutdown_connections 2024-09-10T08:38:37.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.859+0000 7f32fcc00640 1 -- 172.21.15.89:0/157851354 wait complete. 2024-09-10T08:38:37.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.876+0000 7f2598e00640 1 -- 172.21.15.89:0/3163489905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2594074eb0 msgr2=0x7f25941124a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:37.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.876+0000 7f2598e00640 1 --2- 172.21.15.89:0/3163489905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2594074eb0 0x7f25941124a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2580009900 tx=0x7f258002a480 comp rx=0 tx=0).stop 2024-09-10T08:38:37.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.876+0000 7f2598e00640 1 -- 172.21.15.89:0/3163489905 shutdown_connections 2024-09-10T08:38:37.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.876+0000 7f2598e00640 1 --2- 172.21.15.89:0/3163489905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25941129e0 0x7f2594114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:37.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.876+0000 7f2598e00640 1 --2- 172.21.15.89:0/3163489905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2594074eb0 0x7f25941124a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:37.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.876+0000 7f2598e00640 1 --2- 172.21.15.89:0/3163489905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2594074590 0x7f2594074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:37.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.876+0000 7f2598e00640 1 -- 172.21.15.89:0/3163489905 >> 172.21.15.89:0/3163489905 conn(0x7f25940fd710 msgr2=0x7f25940ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:37.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.876+0000 7f2598e00640 1 -- 172.21.15.89:0/3163489905 shutdown_connections 2024-09-10T08:38:37.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.876+0000 7f2598e00640 1 -- 172.21.15.89:0/3163489905 wait complete. 2024-09-10T08:38:37.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.878+0000 7f2598e00640 1 Processor -- start 2024-09-10T08:38:37.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.878+0000 7f2598e00640 1 -- start start 2024-09-10T08:38:37.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.878+0000 7f2598e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2594074590 0x7f25941a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:37.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.878+0000 7f2598e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2594074eb0 0x7f25941a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:37.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.878+0000 7f2598e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25941129e0 0x7f259419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:37.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.878+0000 7f2598e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f25941183b0 con 0x7f2594074590 2024-09-10T08:38:37.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.878+0000 7f2598e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2594118230 con 0x7f25941129e0 2024-09-10T08:38:37.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.878+0000 7f2598e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2594118530 con 0x7f2594074eb0 2024-09-10T08:38:37.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.879+0000 7f2593400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2594074590 0x7f25941a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:37.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.879+0000 7f2593400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2594074590 0x7f25941a4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43884/0 (socket says 172.21.15.89:43884) 2024-09-10T08:38:37.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.879+0000 7f2593400640 1 -- 172.21.15.89:0/12880081 learned_addr learned my addr 172.21.15.89:0/12880081 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:37.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.879+0000 7f2592a00640 1 --2- 172.21.15.89:0/12880081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2594074eb0 0x7f25941a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:37.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.879+0000 7f2593e00640 1 --2- 172.21.15.89:0/12880081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25941129e0 0x7f259419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:37.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.879+0000 7f2593e00640 1 -- 172.21.15.89:0/12880081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2594074eb0 msgr2=0x7f25941a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:37.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.879+0000 7f2593e00640 1 --2- 172.21.15.89:0/12880081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2594074eb0 0x7f25941a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:37.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.879+0000 7f2593e00640 1 -- 172.21.15.89:0/12880081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2594074590 msgr2=0x7f25941a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:37.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.879+0000 7f2593e00640 1 --2- 172.21.15.89:0/12880081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2594074590 0x7f25941a4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:37.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.879+0000 7f2593e00640 1 -- 172.21.15.89:0/12880081 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259419ef90 con 0x7f25941129e0 2024-09-10T08:38:37.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.879+0000 7f2593400640 1 --2- 172.21.15.89:0/12880081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2594074590 0x7f25941a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:38:37.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.880+0000 7f2593e00640 1 --2- 172.21.15.89:0/12880081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25941129e0 0x7f259419e720 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f258c005fd0 tx=0x7f258c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:37.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.880+0000 7f258be00640 1 -- 172.21.15.89:0/12880081 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c010630 con 0x7f25941129e0 2024-09-10T08:38:37.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.880+0000 7f258be00640 1 -- 172.21.15.89:0/12880081 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f258c010cb0 con 0x7f25941129e0 2024-09-10T08:38:37.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.880+0000 7f2598e00640 1 -- 172.21.15.89:0/12880081 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f259419f280 con 0x7f25941129e0 2024-09-10T08:38:37.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.880+0000 7f258be00640 1 -- 172.21.15.89:0/12880081 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c017c80 con 0x7f25941129e0 2024-09-10T08:38:37.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.880+0000 7f2598e00640 1 -- 172.21.15.89:0/12880081 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25941ab5d0 con 0x7f25941129e0 2024-09-10T08:38:37.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.881+0000 7f2598e00640 1 -- 172.21.15.89:0/12880081 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2594075cd0 con 0x7f25941129e0 2024-09-10T08:38:37.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.884+0000 7f258be00640 1 -- 172.21.15.89:0/12880081 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f258c0107d0 con 0x7f25941129e0 2024-09-10T08:38:37.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.886+0000 7f258be00640 1 --2- 172.21.15.89:0/12880081 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f25780a2c90 0x7f25780a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:37.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.886+0000 7f258be00640 1 -- 172.21.15.89:0/12880081 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f258c0cfc70 con 0x7f25941129e0 2024-09-10T08:38:37.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.886+0000 7f2593400640 1 --2- 172.21.15.89:0/12880081 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f25780a2c90 0x7f25780a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:37.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.887+0000 7f2593400640 1 --2- 172.21.15.89:0/12880081 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f25780a2c90 0x7f25780a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2584011500 tx=0x7f2584005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:37.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:37.890+0000 7f258be00640 1 -- 172.21.15.89:0/12880081 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f258c09c6b0 con 0x7f25941129e0 2024-09-10T08:38:37.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:37 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/4011349036' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T08:38:38.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:37 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/4011349036' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T08:38:38.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:38.070+0000 7f2598e00640 1 -- 172.21.15.89:0/12880081 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f259419fc50 con 0x7f25941129e0 2024-09-10T08:38:38.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:38.070+0000 7f258be00640 1 -- 172.21.15.89:0/12880081 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f258c09bfb0 con 0x7f25941129e0 2024-09-10T08:38:38.071 INFO:teuthology.orchestra.run.smithi089.stdout:219043332151 2024-09-10T08:38:38.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:38.072+0000 7f2598e00640 1 -- 172.21.15.89:0/12880081 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f25780a2c90 msgr2=0x7f25780a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:38.072+0000 7f2598e00640 1 --2- 172.21.15.89:0/12880081 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f25780a2c90 0x7f25780a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2584011500 tx=0x7f2584005f90 comp rx=0 tx=0).stop 2024-09-10T08:38:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:38.072+0000 7f2598e00640 1 -- 172.21.15.89:0/12880081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25941129e0 msgr2=0x7f259419e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:38.072+0000 7f2598e00640 1 --2- 172.21.15.89:0/12880081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25941129e0 0x7f259419e720 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f258c005fd0 tx=0x7f258c00c6a0 comp rx=0 tx=0).stop 2024-09-10T08:38:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:38.073+0000 7f2598e00640 1 -- 172.21.15.89:0/12880081 shutdown_connections 2024-09-10T08:38:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:38.073+0000 7f2598e00640 1 --2- 172.21.15.89:0/12880081 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f25780a2c90 0x7f25780a5150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:38.073+0000 7f2598e00640 1 --2- 172.21.15.89:0/12880081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25941129e0 0x7f259419e720 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:38.073+0000 7f2598e00640 1 --2- 172.21.15.89:0/12880081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2594074eb0 0x7f25941a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:38.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:38.073+0000 7f2598e00640 1 --2- 172.21.15.89:0/12880081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2594074590 0x7f25941a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:38.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:38.073+0000 7f2598e00640 1 -- 172.21.15.89:0/12880081 >> 172.21.15.89:0/12880081 conn(0x7f25940fd710 msgr2=0x7f25940ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:38.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:38.073+0000 7f2598e00640 1 -- 172.21.15.89:0/12880081 shutdown_connections 2024-09-10T08:38:38.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:38.073+0000 7f2598e00640 1 -- 172.21.15.89:0/12880081 wait complete. 2024-09-10T08:38:38.917 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:38 smithi089 ceph-mon[30757]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T08:38:38.917 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:38 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/157851354' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T08:38:38.917 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:38 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/12880081' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T08:38:38.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:38 smithi149 ceph-mon[36167]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T08:38:38.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:38 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/157851354' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T08:38:38.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:38 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/12880081' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T08:38:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:38 smithi111 ceph-mon[35220]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T08:38:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:38 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/157851354' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T08:38:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:38 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/12880081' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T08:38:39.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.510+0000 7fe4f5e00640 1 -- 172.21.15.89:0/2885730071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4f0109140 msgr2=0x7fe4f0110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:39.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.510+0000 7fe4f5e00640 1 --2- 172.21.15.89:0/2885730071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4f0109140 0x7fe4f0110cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009ca0 tx=0x7fe4e802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:38:39.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.510+0000 7fe4f5e00640 1 -- 172.21.15.89:0/2885730071 shutdown_connections 2024-09-10T08:38:39.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.510+0000 7fe4f5e00640 1 --2- 172.21.15.89:0/2885730071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4f0109140 0x7fe4f0110cc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:39.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.510+0000 7fe4f5e00640 1 --2- 172.21.15.89:0/2885730071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4f01049c0 0x7fe4f0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:39.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.510+0000 7fe4f5e00640 1 --2- 172.21.15.89:0/2885730071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4f0104010 0x7fe4f01043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:39.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.510+0000 7fe4f5e00640 1 -- 172.21.15.89:0/2885730071 >> 172.21.15.89:0/2885730071 conn(0x7fe4f00ffa50 msgr2=0x7fe4f0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.510+0000 7fe4f5e00640 1 -- 172.21.15.89:0/2885730071 shutdown_connections 2024-09-10T08:38:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.510+0000 7fe4f5e00640 1 -- 172.21.15.89:0/2885730071 wait complete. 2024-09-10T08:38:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.511+0000 7fe4f5e00640 1 Processor -- start 2024-09-10T08:38:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.511+0000 7fe4f5e00640 1 -- start start 2024-09-10T08:38:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.511+0000 7fe4f5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4f0104010 0x7fe4f019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.511+0000 7fe4f5e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4f01049c0 0x7fe4f019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.511+0000 7fe4f5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4f0109140 0x7fe4f019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.511+0000 7fe4f5e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f0114150 con 0x7fe4f0109140 2024-09-10T08:38:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.511+0000 7fe4f5e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f0113fd0 con 0x7fe4f0104010 2024-09-10T08:38:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.511+0000 7fe4f5e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f01142d0 con 0x7fe4f01049c0 2024-09-10T08:38:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.512+0000 7fe4ef400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4f0109140 0x7fe4f019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:39.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.512+0000 7fe4ef400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4f0109140 0x7fe4f019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43902/0 (socket says 172.21.15.89:43902) 2024-09-10T08:38:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.512+0000 7fe4ef400640 1 -- 172.21.15.89:0/3718950506 learned_addr learned my addr 172.21.15.89:0/3718950506 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.512+0000 7fe4ee000640 1 --2- 172.21.15.89:0/3718950506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4f01049c0 0x7fe4f019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.512+0000 7fe4eea00640 1 --2- 172.21.15.89:0/3718950506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4f0104010 0x7fe4f019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.512+0000 7fe4ef400640 1 -- 172.21.15.89:0/3718950506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4f01049c0 msgr2=0x7fe4f019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.512+0000 7fe4ef400640 1 --2- 172.21.15.89:0/3718950506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4f01049c0 0x7fe4f019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.512+0000 7fe4ef400640 1 -- 172.21.15.89:0/3718950506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4f0104010 msgr2=0x7fe4f019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.512+0000 7fe4ef400640 1 --2- 172.21.15.89:0/3718950506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4f0104010 0x7fe4f019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.512+0000 7fe4ef400640 1 -- 172.21.15.89:0/3718950506 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f019f5e0 con 0x7fe4f0109140 2024-09-10T08:38:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.512+0000 7fe4ef400640 1 --2- 172.21.15.89:0/3718950506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4f0109140 0x7fe4f019ef00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8031a20 tx=0x7fe4e8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.513+0000 7fe4f4a00640 1 -- 172.21.15.89:0/3718950506 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8003b70 con 0x7fe4f0109140 2024-09-10T08:38:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.513+0000 7fe4f4a00640 1 -- 172.21.15.89:0/3718950506 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e8003d10 con 0x7fe4f0109140 2024-09-10T08:38:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.513+0000 7fe4f5e00640 1 -- 172.21.15.89:0/3718950506 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f019f870 con 0x7fe4f0109140 2024-09-10T08:38:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.513+0000 7fe4f4a00640 1 -- 172.21.15.89:0/3718950506 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e802fc40 con 0x7fe4f0109140 2024-09-10T08:38:39.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.513+0000 7fe4f5e00640 1 -- 172.21.15.89:0/3718950506 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f01a71b0 con 0x7fe4f0109140 2024-09-10T08:38:39.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.514+0000 7fe4f5e00640 1 -- 172.21.15.89:0/3718950506 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f01043f0 con 0x7fe4f0109140 2024-09-10T08:38:39.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.516+0000 7fe4f4a00640 1 -- 172.21.15.89:0/3718950506 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe4e8004050 con 0x7fe4f0109140 2024-09-10T08:38:39.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.517+0000 7fe4f4a00640 1 --2- 172.21.15.89:0/3718950506 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4d40a2c90 0x7fe4d40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:39.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.517+0000 7fe4f4a00640 1 -- 172.21.15.89:0/3718950506 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fe4e80f04c0 con 0x7fe4f0109140 2024-09-10T08:38:39.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.517+0000 7fe4eea00640 1 --2- 172.21.15.89:0/3718950506 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4d40a2c90 0x7fe4d40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:39.840 INFO:teuthology.orchestra.run.smithi089.stdout:283467841581 2024-09-10T08:38:39.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.518+0000 7fe4eea00640 1 --2- 172.21.15.89:0/3718950506 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4d40a2c90 0x7fe4d40a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0009bd0 tx=0x7fe4e0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:39.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.521+0000 7fe4f4a00640 1 -- 172.21.15.89:0/3718950506 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4e80bd1c0 con 0x7fe4f0109140 2024-09-10T08:38:39.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.701+0000 7fe4f5e00640 1 -- 172.21.15.89:0/3718950506 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fe4f010bdf0 con 0x7fe4f0109140 2024-09-10T08:38:39.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.701+0000 7fe4f4a00640 1 -- 172.21.15.89:0/3718950506 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe4e80bcac0 con 0x7fe4f0109140 2024-09-10T08:38:39.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.703+0000 7fe4f5e00640 1 -- 172.21.15.89:0/3718950506 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4d40a2c90 msgr2=0x7fe4d40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:39.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.703+0000 7fe4f5e00640 1 --2- 172.21.15.89:0/3718950506 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4d40a2c90 0x7fe4d40a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0009bd0 tx=0x7fe4e0005f90 comp rx=0 tx=0).stop 2024-09-10T08:38:39.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.703+0000 7fe4f5e00640 1 -- 172.21.15.89:0/3718950506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4f0109140 msgr2=0x7fe4f019ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:39.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.703+0000 7fe4f5e00640 1 --2- 172.21.15.89:0/3718950506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4f0109140 0x7fe4f019ef00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8031a20 tx=0x7fe4e8031a50 comp rx=0 tx=0).stop 2024-09-10T08:38:39.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.704+0000 7fe4f5e00640 1 -- 172.21.15.89:0/3718950506 shutdown_connections 2024-09-10T08:38:39.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.704+0000 7fe4f5e00640 1 --2- 172.21.15.89:0/3718950506 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4d40a2c90 0x7fe4d40a5150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:39.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.704+0000 7fe4f5e00640 1 --2- 172.21.15.89:0/3718950506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4f0109140 0x7fe4f019ef00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:39.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.704+0000 7fe4f5e00640 1 --2- 172.21.15.89:0/3718950506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4f01049c0 0x7fe4f019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:39.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.704+0000 7fe4f5e00640 1 --2- 172.21.15.89:0/3718950506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4f0104010 0x7fe4f019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:39.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.704+0000 7fe4f5e00640 1 -- 172.21.15.89:0/3718950506 >> 172.21.15.89:0/3718950506 conn(0x7fe4f00ffa50 msgr2=0x7fe4f0100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:39.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.704+0000 7fe4f5e00640 1 -- 172.21.15.89:0/3718950506 shutdown_connections 2024-09-10T08:38:39.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:39.704+0000 7fe4f5e00640 1 -- 172.21.15.89:0/3718950506 wait complete. 2024-09-10T08:38:40.512 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103215 got 249108103218 for osd.8 2024-09-10T08:38:40.512 DEBUG:teuthology.parallel:result is None 2024-09-10T08:38:40.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:40 smithi089 ceph-mon[30757]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-10T08:38:40.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:40 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3718950506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T08:38:40.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.810+0000 7fe904c00640 1 -- 172.21.15.89:0/326675933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe900102790 msgr2=0x7fe9001067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.810+0000 7fe904c00640 1 --2- 172.21.15.89:0/326675933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe900102790 0x7fe9001067e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009860 tx=0x7fe8e802a340 comp rx=0 tx=0).stop 2024-09-10T08:38:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.811+0000 7fe904c00640 1 -- 172.21.15.89:0/326675933 shutdown_connections 2024-09-10T08:38:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.811+0000 7fe904c00640 1 --2- 172.21.15.89:0/326675933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe900106f10 0x7fe90010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.811+0000 7fe904c00640 1 --2- 172.21.15.89:0/326675933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe900102790 0x7fe9001067e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.811+0000 7fe904c00640 1 --2- 172.21.15.89:0/326675933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe900101de0 0x7fe9001021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.811+0000 7fe904c00640 1 -- 172.21.15.89:0/326675933 >> 172.21.15.89:0/326675933 conn(0x7fe9000fd8c0 msgr2=0x7fe9000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.811+0000 7fe904c00640 1 -- 172.21.15.89:0/326675933 shutdown_connections 2024-09-10T08:38:40.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.811+0000 7fe904c00640 1 -- 172.21.15.89:0/326675933 wait complete. 2024-09-10T08:38:40.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.812+0000 7fe904c00640 1 Processor -- start 2024-09-10T08:38:40.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.812+0000 7fe904c00640 1 -- start start 2024-09-10T08:38:40.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.812+0000 7fe904c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe900101de0 0x7fe90019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:40.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.812+0000 7fe904c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe900102790 0x7fe90019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:40.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.812+0000 7fe904c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe900106f10 0x7fe90019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:40.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.812+0000 7fe904c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe900111f70 con 0x7fe900101de0 2024-09-10T08:38:40.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.812+0000 7fe904c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe900111df0 con 0x7fe900106f10 2024-09-10T08:38:40.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.812+0000 7fe904c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe9001120f0 con 0x7fe900102790 2024-09-10T08:38:40.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.812+0000 7fe8fde00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe900101de0 0x7fe90019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:40.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.812+0000 7fe8fde00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe900101de0 0x7fe90019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43908/0 (socket says 172.21.15.89:43908) 2024-09-10T08:38:40.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.813+0000 7fe8fe800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe900106f10 0x7fe90019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:40.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.813+0000 7fe8fde00640 1 -- 172.21.15.89:0/1830134462 learned_addr learned my addr 172.21.15.89:0/1830134462 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:40.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.813+0000 7fe8f7e00640 1 --2- 172.21.15.89:0/1830134462 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe900102790 0x7fe90019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:40.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.813+0000 7fe8fe800640 1 -- 172.21.15.89:0/1830134462 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe900102790 msgr2=0x7fe90019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:40.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.813+0000 7fe8fe800640 1 --2- 172.21.15.89:0/1830134462 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe900102790 0x7fe90019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:40.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.813+0000 7fe8fe800640 1 -- 172.21.15.89:0/1830134462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe900101de0 msgr2=0x7fe90019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:40.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.813+0000 7fe8fe800640 1 --2- 172.21.15.89:0/1830134462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe900101de0 0x7fe90019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:40.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.813+0000 7fe8fe800640 1 -- 172.21.15.89:0/1830134462 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90019f5f0 con 0x7fe900106f10 2024-09-10T08:38:40.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.813+0000 7fe8fde00640 1 --2- 172.21.15.89:0/1830134462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe900101de0 0x7fe90019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:38:40.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.813+0000 7fe8fe800640 1 --2- 172.21.15.89:0/1830134462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe900106f10 0x7fe90019ef10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009fd0 tx=0x7fe8f800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:40.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.814+0000 7fe8f6a00640 1 -- 172.21.15.89:0/1830134462 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f800ce20 con 0x7fe900106f10 2024-09-10T08:38:40.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.814+0000 7fe904c00640 1 -- 172.21.15.89:0/1830134462 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90019f8e0 con 0x7fe900106f10 2024-09-10T08:38:40.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.814+0000 7fe8f6a00640 1 -- 172.21.15.89:0/1830134462 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8f8005020 con 0x7fe900106f10 2024-09-10T08:38:40.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.814+0000 7fe8f6a00640 1 -- 172.21.15.89:0/1830134462 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8010720 con 0x7fe900106f10 2024-09-10T08:38:40.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.814+0000 7fe904c00640 1 -- 172.21.15.89:0/1830134462 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9001a71c0 con 0x7fe900106f10 2024-09-10T08:38:40.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.815+0000 7fe904c00640 1 -- 172.21.15.89:0/1830134462 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9001021c0 con 0x7fe900106f10 2024-09-10T08:38:40.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.817+0000 7fe8f6a00640 1 -- 172.21.15.89:0/1830134462 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe8f8005540 con 0x7fe900106f10 2024-09-10T08:38:40.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.818+0000 7fe8f6a00640 1 --2- 172.21.15.89:0/1830134462 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe8e40a2c10 0x7fe8e40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.818+0000 7fe8f6a00640 1 -- 172.21.15.89:0/1830134462 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fe8f80cfbf0 con 0x7fe900106f10 2024-09-10T08:38:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.818+0000 7fe8fde00640 1 --2- 172.21.15.89:0/1830134462 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe8e40a2c10 0x7fe8e40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:40.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.819+0000 7fe8fde00640 1 --2- 172.21.15.89:0/1830134462 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe8e40a2c10 0x7fe8e40a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00096c0 tx=0x7fe8f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:40.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:40.820+0000 7fe8f6a00640 1 -- 172.21.15.89:0/1830134462 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8f809c6c0 con 0x7fe900106f10 2024-09-10T08:38:40.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:40 smithi149 ceph-mon[36167]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-10T08:38:40.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:40 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3718950506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T08:38:41.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.002+0000 7fe904c00640 1 -- 172.21.15.89:0/1830134462 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe9001a7450 con 0x7fe900106f10 2024-09-10T08:38:41.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.002+0000 7fe8f6a00640 1 -- 172.21.15.89:0/1830134462 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe8f809bfc0 con 0x7fe900106f10 2024-09-10T08:38:41.003 INFO:teuthology.orchestra.run.smithi089.stdout:55834574936 2024-09-10T08:38:41.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.005+0000 7fe904c00640 1 -- 172.21.15.89:0/1830134462 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe8e40a2c10 msgr2=0x7fe8e40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.005+0000 7fe904c00640 1 --2- 172.21.15.89:0/1830134462 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe8e40a2c10 0x7fe8e40a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00096c0 tx=0x7fe8f0009260 comp rx=0 tx=0).stop 2024-09-10T08:38:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.005+0000 7fe904c00640 1 -- 172.21.15.89:0/1830134462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe900106f10 msgr2=0x7fe90019ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.005+0000 7fe904c00640 1 --2- 172.21.15.89:0/1830134462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe900106f10 0x7fe90019ef10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009fd0 tx=0x7fe8f800ef10 comp rx=0 tx=0).stop 2024-09-10T08:38:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.005+0000 7fe904c00640 1 -- 172.21.15.89:0/1830134462 shutdown_connections 2024-09-10T08:38:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.005+0000 7fe904c00640 1 --2- 172.21.15.89:0/1830134462 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe8e40a2c10 0x7fe8e40a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.005+0000 7fe904c00640 1 --2- 172.21.15.89:0/1830134462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe900106f10 0x7fe90019ef10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.005+0000 7fe904c00640 1 --2- 172.21.15.89:0/1830134462 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe900102790 0x7fe90019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.005+0000 7fe904c00640 1 --2- 172.21.15.89:0/1830134462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe900101de0 0x7fe90019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.005+0000 7fe904c00640 1 -- 172.21.15.89:0/1830134462 >> 172.21.15.89:0/1830134462 conn(0x7fe9000fd8c0 msgr2=0x7fe9000feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:41.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.005+0000 7fe904c00640 1 -- 172.21.15.89:0/1830134462 shutdown_connections 2024-09-10T08:38:41.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.005+0000 7fe904c00640 1 -- 172.21.15.89:0/1830134462 wait complete. 2024-09-10T08:38:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:40 smithi111 ceph-mon[35220]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-10T08:38:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:40 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3718950506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T08:38:41.199 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149769 got 111669149771 for osd.3 2024-09-10T08:38:41.200 DEBUG:teuthology.parallel:result is None 2024-09-10T08:38:41.639 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678052 got 304942678055 for osd.10 2024-09-10T08:38:41.639 DEBUG:teuthology.parallel:result is None 2024-09-10T08:38:41.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.793+0000 7ffad7400640 1 -- 172.21.15.89:0/1268163457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffad0101de0 msgr2=0x7ffad01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:41.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.793+0000 7ffad7400640 1 --2- 172.21.15.89:0/1268163457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffad0101de0 0x7ffad01021c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009900 tx=0x7ffac002a410 comp rx=0 tx=0).stop 2024-09-10T08:38:41.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.794+0000 7ffad7400640 1 -- 172.21.15.89:0/1268163457 shutdown_connections 2024-09-10T08:38:41.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.794+0000 7ffad7400640 1 --2- 172.21.15.89:0/1268163457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad0106f10 0x7ffad010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:41.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.794+0000 7ffad7400640 1 --2- 172.21.15.89:0/1268163457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffad0102790 0x7ffad01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:41.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.794+0000 7ffad7400640 1 --2- 172.21.15.89:0/1268163457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffad0101de0 0x7ffad01021c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:41.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.794+0000 7ffad7400640 1 -- 172.21.15.89:0/1268163457 >> 172.21.15.89:0/1268163457 conn(0x7ffad00fd8c0 msgr2=0x7ffad00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:41.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.794+0000 7ffad7400640 1 -- 172.21.15.89:0/1268163457 shutdown_connections 2024-09-10T08:38:41.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.794+0000 7ffad7400640 1 -- 172.21.15.89:0/1268163457 wait complete. 2024-09-10T08:38:41.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.795+0000 7ffad7400640 1 Processor -- start 2024-09-10T08:38:41.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.795+0000 7ffad7400640 1 -- start start 2024-09-10T08:38:41.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.795+0000 7ffad7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad0101de0 0x7ffad019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:41.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.795+0000 7ffad7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffad0102790 0x7ffad019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:41.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.795+0000 7ffad7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffad0106f10 0x7ffad019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:41.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.795+0000 7ffad7400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ffad0111fc0 con 0x7ffad0102790 2024-09-10T08:38:41.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.795+0000 7ffad7400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffad0111e40 con 0x7ffad0101de0 2024-09-10T08:38:41.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.795+0000 7ffad7400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffad0112140 con 0x7ffad0106f10 2024-09-10T08:38:41.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.796+0000 7ffac7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffad0102790 0x7ffad019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:41.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.796+0000 7ffad5400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffad0106f10 0x7ffad019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:41.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.796+0000 7ffad4a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad0101de0 0x7ffad019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:41.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.796+0000 7ffac7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffad0102790 0x7ffad019abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43918/0 (socket says 172.21.15.89:43918) 2024-09-10T08:38:41.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.796+0000 7ffad5400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffad0106f10 0x7ffad019ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:49102/0 (socket says 172.21.15.89:49102) 2024-09-10T08:38:41.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.796+0000 7ffac7e00640 1 -- 172.21.15.89:0/2123475294 learned_addr learned my addr 172.21.15.89:0/2123475294 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:41.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.796+0000 7ffad4a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad0101de0 0x7ffad019a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:48542/0 (socket says 172.21.15.89:48542) 2024-09-10T08:38:41.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.796+0000 7ffad5400640 1 -- 172.21.15.89:0/2123475294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad0101de0 msgr2=0x7ffad019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:41.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.796+0000 7ffad5400640 1 --2- 172.21.15.89:0/2123475294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad0101de0 0x7ffad019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:41.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.796+0000 7ffad5400640 1 -- 172.21.15.89:0/2123475294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffad0102790 msgr2=0x7ffad019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:41.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.796+0000 7ffad5400640 1 --2- 172.21.15.89:0/2123475294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffad0102790 0x7ffad019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:41.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.796+0000 7ffad5400640 1 -- 172.21.15.89:0/2123475294 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad019f630 con 0x7ffad0106f10 2024-09-10T08:38:41.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.796+0000 7ffad4a00640 1 --2- 172.21.15.89:0/2123475294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad0101de0 0x7ffad019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:38:41.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.796+0000 7ffad5400640 1 --2- 172.21.15.89:0/2123475294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffad0106f10 0x7ffad019ef50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffac800da10 tx=0x7ffac800ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:41.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.797+0000 7fface000640 1 -- 172.21.15.89:0/2123475294 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac800bd70 con 0x7ffad0106f10 2024-09-10T08:38:41.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.797+0000 7ffad7400640 1 -- 172.21.15.89:0/2123475294 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad019f920 con 0x7ffad0106f10 2024-09-10T08:38:41.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.797+0000 7fface000640 1 -- 172.21.15.89:0/2123475294 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffac8003e50 con 0x7ffad0106f10 2024-09-10T08:38:41.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.797+0000 7fface000640 1 -- 172.21.15.89:0/2123475294 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac80106a0 con 0x7ffad0106f10 2024-09-10T08:38:41.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.797+0000 7ffad7400640 1 -- 172.21.15.89:0/2123475294 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad01a7200 con 0x7ffad0106f10 2024-09-10T08:38:41.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.798+0000 7ffad7400640 1 -- 172.21.15.89:0/2123475294 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad01021c0 con 0x7ffad0106f10 2024-09-10T08:38:41.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.800+0000 7fface000640 1 -- 172.21.15.89:0/2123475294 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ffac8004b40 con 0x7ffad0106f10 2024-09-10T08:38:41.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.801+0000 7fface000640 1 --2- 172.21.15.89:0/2123475294 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffab40a2c90 0x7ffab40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:41.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.801+0000 7fface000640 1 -- 172.21.15.89:0/2123475294 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7ffac80cf9f0 con 0x7ffad0106f10 2024-09-10T08:38:41.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.801+0000 7ffad4a00640 1 --2- 172.21.15.89:0/2123475294 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffab40a2c90 0x7ffab40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:41.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.802+0000 7ffad4a00640 1 --2- 172.21.15.89:0/2123475294 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffab40a2c90 0x7ffab40a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffac0000c00 tx=0x7ffac0031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:41.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.803+0000 7fface000640 1 -- 172.21.15.89:0/2123475294 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffac809c4b0 con 0x7ffad0106f10 2024-09-10T08:38:41.939 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:41 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1830134462' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T08:38:41.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:41 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1830134462' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T08:38:41.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.989+0000 7ffad7400640 1 -- 172.21.15.89:0/2123475294 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7ffad019b9e0 con 0x7ffad0106f10 2024-09-10T08:38:41.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.990+0000 7fface000640 1 -- 172.21.15.89:0/2123475294 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ffad019b9e0 con 0x7ffad0106f10 2024-09-10T08:38:41.990 INFO:teuthology.orchestra.run.smithi089.stdout:339302416420 2024-09-10T08:38:41.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.992+0000 7ffad7400640 1 -- 172.21.15.89:0/2123475294 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffab40a2c90 msgr2=0x7ffab40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:41.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.992+0000 7ffad7400640 1 --2- 172.21.15.89:0/2123475294 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffab40a2c90 0x7ffab40a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffac0000c00 tx=0x7ffac0031510 comp rx=0 tx=0).stop 2024-09-10T08:38:41.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.992+0000 7ffad7400640 1 -- 172.21.15.89:0/2123475294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffad0106f10 msgr2=0x7ffad019ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:41.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.992+0000 7ffad7400640 1 --2- 172.21.15.89:0/2123475294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffad0106f10 0x7ffad019ef50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffac800da10 tx=0x7ffac800ded0 comp rx=0 tx=0).stop 2024-09-10T08:38:41.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.992+0000 7ffad7400640 1 -- 172.21.15.89:0/2123475294 shutdown_connections 2024-09-10T08:38:41.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.992+0000 7ffad7400640 1 --2- 172.21.15.89:0/2123475294 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffab40a2c90 0x7ffab40a5150 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:41.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.992+0000 7ffad7400640 1 --2- 172.21.15.89:0/2123475294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffad0106f10 0x7ffad019ef50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:41.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.992+0000 7ffad7400640 1 --2- 172.21.15.89:0/2123475294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffad0102790 0x7ffad019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:41.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.992+0000 7ffad7400640 1 --2- 172.21.15.89:0/2123475294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffad0101de0 0x7ffad019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:41.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.992+0000 7ffad7400640 1 -- 172.21.15.89:0/2123475294 >> 172.21.15.89:0/2123475294 conn(0x7ffad00fd8c0 msgr2=0x7ffad00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:41.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.992+0000 7ffad7400640 1 -- 172.21.15.89:0/2123475294 shutdown_connections 2024-09-10T08:38:41.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:41.992+0000 7ffad7400640 1 -- 172.21.15.89:0/2123475294 wait complete. 2024-09-10T08:38:42.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:41 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1830134462' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T08:38:42.742 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841579 got 283467841581 for osd.9 2024-09-10T08:38:42.742 DEBUG:teuthology.parallel:result is None 2024-09-10T08:38:42.897 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:42 smithi089 ceph-mon[30757]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T08:38:42.897 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-10T08:38:42.897 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-10T08:38:42.897 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-09-10T08:38:42.897 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-09-10T08:38:42.897 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:42 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2123475294' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T08:38:42.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:42 smithi149 ceph-mon[36167]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T08:38:42.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-10T08:38:42.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-10T08:38:42.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-09-10T08:38:42.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-09-10T08:38:42.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:42 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2123475294' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T08:38:43.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:42 smithi111 ceph-mon[35220]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T08:38:43.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-10T08:38:43.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-10T08:38:43.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-09-10T08:38:43.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-09-10T08:38:43.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:42 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2123475294' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T08:38:43.075 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:43.109 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332150 got 219043332151 for osd.7 2024-09-10T08:38:43.109 DEBUG:teuthology.parallel:result is None 2024-09-10T08:38:43.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.492+0000 7f2ea8e00640 1 -- 172.21.15.89:0/683065579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ea41049c0 msgr2=0x7f2ea4108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.492+0000 7f2ea8e00640 1 --2- 172.21.15.89:0/683065579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ea41049c0 0x7f2ea4108a10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009900 tx=0x7f2e8c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:38:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.492+0000 7f2ea8e00640 1 -- 172.21.15.89:0/683065579 shutdown_connections 2024-09-10T08:38:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.492+0000 7f2ea8e00640 1 --2- 172.21.15.89:0/683065579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ea4109140 0x7f2ea4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.492+0000 7f2ea8e00640 1 --2- 172.21.15.89:0/683065579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ea41049c0 0x7f2ea4108a10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.492+0000 7f2ea8e00640 1 --2- 172.21.15.89:0/683065579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ea4104010 0x7f2ea41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.492+0000 7f2ea8e00640 1 -- 172.21.15.89:0/683065579 >> 172.21.15.89:0/683065579 conn(0x7f2ea40ffa50 msgr2=0x7f2ea4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.492+0000 7f2ea8e00640 1 -- 172.21.15.89:0/683065579 shutdown_connections 2024-09-10T08:38:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.492+0000 7f2ea8e00640 1 -- 172.21.15.89:0/683065579 wait complete. 2024-09-10T08:38:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.493+0000 7f2ea8e00640 1 Processor -- start 2024-09-10T08:38:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.493+0000 7f2ea8e00640 1 -- start start 2024-09-10T08:38:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.494+0000 7f2ea8e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ea4104010 0x7f2ea419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:43.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.494+0000 7f2ea8e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ea41049c0 0x7f2ea419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.494+0000 7f2ea8e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ea4109140 0x7f2ea419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.494+0000 7f2ea8e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea4114160 con 0x7f2ea4104010 2024-09-10T08:38:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.494+0000 7f2ea8e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea4113fe0 con 0x7f2ea41049c0 2024-09-10T08:38:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.494+0000 7f2ea8e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41142e0 con 0x7f2ea4109140 2024-09-10T08:38:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.494+0000 7f2ea1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ea4104010 0x7f2ea419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.494+0000 7f2ea1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ea4104010 0x7f2ea419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43940/0 (socket says 172.21.15.89:43940) 2024-09-10T08:38:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.494+0000 7f2ea1e00640 1 -- 172.21.15.89:0/2839311434 learned_addr learned my addr 172.21.15.89:0/2839311434 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.494+0000 7f2e99400640 1 --2- 172.21.15.89:0/2839311434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ea41049c0 0x7f2ea419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.494+0000 7f2ea2800640 1 --2- 172.21.15.89:0/2839311434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ea4109140 0x7f2ea419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.495+0000 7f2ea1e00640 1 -- 172.21.15.89:0/2839311434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ea4109140 msgr2=0x7f2ea419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.495+0000 7f2ea1e00640 1 --2- 172.21.15.89:0/2839311434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ea4109140 0x7f2ea419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:43.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.495+0000 7f2ea1e00640 1 -- 172.21.15.89:0/2839311434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ea41049c0 msgr2=0x7f2ea419ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.495+0000 7f2ea1e00640 1 --2- 172.21.15.89:0/2839311434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ea41049c0 0x7f2ea419ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.495+0000 7f2ea1e00640 1 -- 172.21.15.89:0/2839311434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea419f6e0 con 0x7f2ea4104010 2024-09-10T08:38:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.495+0000 7f2ea1e00640 1 --2- 172.21.15.89:0/2839311434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ea4104010 0x7f2ea419a730 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400e9f0 tx=0x7f2e9400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.495+0000 7f2e9b400640 1 -- 172.21.15.89:0/2839311434 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9400cd80 con 0x7f2ea4104010 2024-09-10T08:38:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.495+0000 7f2e9b400640 1 -- 172.21.15.89:0/2839311434 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e94005500 con 0x7f2ea4104010 2024-09-10T08:38:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.495+0000 7f2e9b400640 1 -- 172.21.15.89:0/2839311434 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94010690 con 0x7f2ea4104010 2024-09-10T08:38:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.495+0000 7f2ea8e00640 1 -- 172.21.15.89:0/2839311434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea419f9d0 con 0x7f2ea4104010 2024-09-10T08:38:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.495+0000 7f2ea8e00640 1 -- 172.21.15.89:0/2839311434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea41a72b0 con 0x7f2ea4104010 2024-09-10T08:38:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.496+0000 7f2ea8e00640 1 -- 172.21.15.89:0/2839311434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ea41043f0 con 0x7f2ea4104010 2024-09-10T08:38:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.498+0000 7f2e9b400640 1 -- 172.21.15.89:0/2839311434 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2e940046c0 con 0x7f2ea4104010 2024-09-10T08:38:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.499+0000 7f2e9b400640 1 --2- 172.21.15.89:0/2839311434 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2e880a2c90 0x7f2e880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:43.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.500+0000 7f2e9b400640 1 -- 172.21.15.89:0/2839311434 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f2e940cf910 con 0x7f2ea4104010 2024-09-10T08:38:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.500+0000 7f2e99400640 1 --2- 172.21.15.89:0/2839311434 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2e880a2c90 0x7f2e880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.501+0000 7f2e99400640 1 --2- 172.21.15.89:0/2839311434 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2e880a2c90 0x7f2e880a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009230 tx=0x7f2e8c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.504+0000 7f2e9b400640 1 -- 172.21.15.89:0/2839311434 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2e9409c350 con 0x7f2ea4104010 2024-09-10T08:38:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.691+0000 7f2ea8e00640 1 -- 172.21.15.89:0/2839311434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f2ea419ba90 con 0x7f2ea4104010 2024-09-10T08:38:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.691+0000 7f2e9b400640 1 -- 172.21.15.89:0/2839311434 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2e9409bc50 con 0x7f2ea4104010 2024-09-10T08:38:43.242 INFO:teuthology.orchestra.run.smithi089.stdout:77309411412 2024-09-10T08:38:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.694+0000 7f2ea8e00640 1 -- 172.21.15.89:0/2839311434 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2e880a2c90 msgr2=0x7f2e880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:43.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.694+0000 7f2ea8e00640 1 --2- 172.21.15.89:0/2839311434 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2e880a2c90 0x7f2e880a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009230 tx=0x7f2e8c0058d0 comp rx=0 tx=0).stop 2024-09-10T08:38:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.694+0000 7f2ea8e00640 1 -- 172.21.15.89:0/2839311434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ea4104010 msgr2=0x7f2ea419a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.694+0000 7f2ea8e00640 1 --2- 172.21.15.89:0/2839311434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ea4104010 0x7f2ea419a730 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400e9f0 tx=0x7f2e9400edf0 comp rx=0 tx=0).stop 2024-09-10T08:38:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.694+0000 7f2ea8e00640 1 -- 172.21.15.89:0/2839311434 shutdown_connections 2024-09-10T08:38:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.694+0000 7f2ea8e00640 1 --2- 172.21.15.89:0/2839311434 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2e880a2c90 0x7f2e880a5150 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.694+0000 7f2ea8e00640 1 --2- 172.21.15.89:0/2839311434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ea4109140 0x7f2ea419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.694+0000 7f2ea8e00640 1 --2- 172.21.15.89:0/2839311434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ea41049c0 0x7f2ea419ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.694+0000 7f2ea8e00640 1 --2- 172.21.15.89:0/2839311434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ea4104010 0x7f2ea419a730 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.694+0000 7f2ea8e00640 1 -- 172.21.15.89:0/2839311434 >> 172.21.15.89:0/2839311434 conn(0x7f2ea40ffa50 msgr2=0x7f2ea4100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.694+0000 7f2ea8e00640 1 -- 172.21.15.89:0/2839311434 shutdown_connections 2024-09-10T08:38:43.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:42.694+0000 7f2ea8e00640 1 -- 172.21.15.89:0/2839311434 wait complete. 2024-09-10T08:38:43.950 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:43 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-10T08:38:43.950 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:43 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-10T08:38:43.951 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:43 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-09-10T08:38:43.951 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:43 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-09-10T08:38:43.951 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:43 smithi089 ceph-mon[30757]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T08:38:43.951 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:43 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2839311434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T08:38:43.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:43 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-10T08:38:43.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:43 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-10T08:38:43.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:43 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-09-10T08:38:43.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:43 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-09-10T08:38:43.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:43 smithi149 ceph-mon[36167]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T08:38:43.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:43 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2839311434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T08:38:44.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:43 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-10T08:38:44.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:43 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-10T08:38:44.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:43 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-09-10T08:38:44.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:43 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-09-10T08:38:44.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:43 smithi111 ceph-mon[35220]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T08:38:44.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:43 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2839311434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T08:38:44.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.431+0000 7f7f56a00640 1 -- 172.21.15.89:0/330695411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f501047e0 msgr2=0x7f7f50108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:44.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.431+0000 7f7f56a00640 1 --2- 172.21.15.89:0/330695411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f501047e0 0x7f7f50108830 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009900 tx=0x7f7f3c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:38:44.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.431+0000 7f7f56a00640 1 -- 172.21.15.89:0/330695411 shutdown_connections 2024-09-10T08:38:44.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.431+0000 7f7f56a00640 1 --2- 172.21.15.89:0/330695411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f50108f60 0x7f7f50110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:44.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.431+0000 7f7f56a00640 1 --2- 172.21.15.89:0/330695411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f501047e0 0x7f7f50108830 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:44.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.431+0000 7f7f56a00640 1 --2- 172.21.15.89:0/330695411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f50103e30 0x7f7f50104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:44.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.431+0000 7f7f56a00640 1 -- 172.21.15.89:0/330695411 >> 172.21.15.89:0/330695411 conn(0x7f7f500ff8d0 msgr2=0x7f7f50101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:44.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.431+0000 7f7f56a00640 1 -- 172.21.15.89:0/330695411 shutdown_connections 2024-09-10T08:38:44.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.431+0000 7f7f56a00640 1 -- 172.21.15.89:0/330695411 wait complete. 2024-09-10T08:38:44.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.432+0000 7f7f56a00640 1 Processor -- start 2024-09-10T08:38:44.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.432+0000 7f7f56a00640 1 -- start start 2024-09-10T08:38:44.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.432+0000 7f7f56a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f50103e30 0x7f7f5019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:44.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.432+0000 7f7f56a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f501047e0 0x7f7f5019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:44.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.432+0000 7f7f56a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f50108f60 0x7f7f5019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:44.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.432+0000 7f7f56a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7f50114010 con 0x7f7f50108f60 2024-09-10T08:38:44.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.432+0000 7f7f56a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7f50113e90 con 0x7f7f501047e0 2024-09-10T08:38:44.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.432+0000 7f7f56a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7f50114190 con 0x7f7f50103e30 2024-09-10T08:38:44.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.433+0000 7f7f56000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f50108f60 0x7f7f5019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:44.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.433+0000 7f7f55600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f50103e30 0x7f7f5019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:44.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.433+0000 7f7f55600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f50103e30 0x7f7f5019a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44928/0 (socket says 172.21.15.89:44928) 2024-09-10T08:38:44.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.433+0000 7f7f55600640 1 -- 172.21.15.89:0/2240774646 learned_addr learned my addr 172.21.15.89:0/2240774646 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:44.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.433+0000 7f7f54c00640 1 --2- 172.21.15.89:0/2240774646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f501047e0 0x7f7f5019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:44.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.433+0000 7f7f56000640 1 -- 172.21.15.89:0/2240774646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f50103e30 msgr2=0x7f7f5019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:44.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.433+0000 7f7f56000640 1 --2- 172.21.15.89:0/2240774646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f50103e30 0x7f7f5019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:44.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.433+0000 7f7f56000640 1 -- 172.21.15.89:0/2240774646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f501047e0 msgr2=0x7f7f5019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:44.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.433+0000 7f7f56000640 1 --2- 172.21.15.89:0/2240774646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f501047e0 0x7f7f5019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:44.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.433+0000 7f7f56000640 1 -- 172.21.15.89:0/2240774646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5019f440 con 0x7f7f50108f60 2024-09-10T08:38:44.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.433+0000 7f7f55600640 1 --2- 172.21.15.89:0/2240774646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f50103e30 0x7f7f5019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:38:44.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.433+0000 7f7f56000640 1 --2- 172.21.15.89:0/2240774646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f50108f60 0x7f7f5019ed60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800e940 tx=0x7f7f4800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:44.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.434+0000 7f7f4e000640 1 -- 172.21.15.89:0/2240774646 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48004d20 con 0x7f7f50108f60 2024-09-10T08:38:44.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.434+0000 7f7f4e000640 1 -- 172.21.15.89:0/2240774646 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f48005020 con 0x7f7f50108f60 2024-09-10T08:38:44.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.434+0000 7f7f56a00640 1 -- 172.21.15.89:0/2240774646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f5019f730 con 0x7f7f50108f60 2024-09-10T08:38:44.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.434+0000 7f7f4e000640 1 -- 172.21.15.89:0/2240774646 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f480116b0 con 0x7f7f50108f60 2024-09-10T08:38:44.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.434+0000 7f7f56a00640 1 -- 172.21.15.89:0/2240774646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f501a7010 con 0x7f7f50108f60 2024-09-10T08:38:44.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.435+0000 7f7f56a00640 1 -- 172.21.15.89:0/2240774646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f50104210 con 0x7f7f50108f60 2024-09-10T08:38:44.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.437+0000 7f7f4e000640 1 -- 172.21.15.89:0/2240774646 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7f480054d0 con 0x7f7f50108f60 2024-09-10T08:38:44.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.438+0000 7f7f4e000640 1 --2- 172.21.15.89:0/2240774646 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7f380a2c10 0x7f7f380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:44.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.438+0000 7f7f4e000640 1 -- 172.21.15.89:0/2240774646 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f7f480cf940 con 0x7f7f50108f60 2024-09-10T08:38:44.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.438+0000 7f7f55600640 1 --2- 172.21.15.89:0/2240774646 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7f380a2c10 0x7f7f380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:44.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.439+0000 7f7f55600640 1 --2- 172.21.15.89:0/2240774646 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7f380a2c10 0x7f7f380a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7f44011550 tx=0x7f7f44005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:44.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.442+0000 7f7f4e000640 1 -- 172.21.15.89:0/2240774646 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7f48016030 con 0x7f7f50108f60 2024-09-10T08:38:44.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.626+0000 7f7f56a00640 1 -- 172.21.15.89:0/2240774646 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f7f5010bbb0 con 0x7f7f50108f60 2024-09-10T08:38:44.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.627+0000 7f7f4e000640 1 -- 172.21.15.89:0/2240774646 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7f4809bcc0 con 0x7f7f50108f60 2024-09-10T08:38:44.628 INFO:teuthology.orchestra.run.smithi089.stdout:197568495678 2024-09-10T08:38:44.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.629+0000 7f7f56a00640 1 -- 172.21.15.89:0/2240774646 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7f380a2c10 msgr2=0x7f7f380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:44.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.629+0000 7f7f56a00640 1 --2- 172.21.15.89:0/2240774646 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7f380a2c10 0x7f7f380a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7f44011550 tx=0x7f7f44005f90 comp rx=0 tx=0).stop 2024-09-10T08:38:44.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.629+0000 7f7f56a00640 1 -- 172.21.15.89:0/2240774646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f50108f60 msgr2=0x7f7f5019ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:44.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.629+0000 7f7f56a00640 1 --2- 172.21.15.89:0/2240774646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f50108f60 0x7f7f5019ed60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800e940 tx=0x7f7f4800ee00 comp rx=0 tx=0).stop 2024-09-10T08:38:44.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.629+0000 7f7f56a00640 1 -- 172.21.15.89:0/2240774646 shutdown_connections 2024-09-10T08:38:44.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.629+0000 7f7f56a00640 1 --2- 172.21.15.89:0/2240774646 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7f380a2c10 0x7f7f380a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:44.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.629+0000 7f7f56a00640 1 --2- 172.21.15.89:0/2240774646 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f50108f60 0x7f7f5019ed60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:44.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.629+0000 7f7f56a00640 1 --2- 172.21.15.89:0/2240774646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f501047e0 0x7f7f5019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:44.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.629+0000 7f7f56a00640 1 --2- 172.21.15.89:0/2240774646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f50103e30 0x7f7f5019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:44.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.629+0000 7f7f56a00640 1 -- 172.21.15.89:0/2240774646 >> 172.21.15.89:0/2240774646 conn(0x7f7f500ff8d0 msgr2=0x7f7f50100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:44.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.629+0000 7f7f56a00640 1 -- 172.21.15.89:0/2240774646 shutdown_connections 2024-09-10T08:38:44.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:44.629+0000 7f7f56a00640 1 -- 172.21.15.89:0/2240774646 wait complete. 2024-09-10T08:38:44.806 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:44 smithi089 ceph-mon[30757]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-10T08:38:44.806 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:44 smithi089 ceph-mon[30757]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T08:38:44.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:44 smithi149 ceph-mon[36167]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-10T08:38:44.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:44 smithi149 ceph-mon[36167]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T08:38:45.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:44 smithi111 ceph-mon[35220]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-10T08:38:45.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:44 smithi111 ceph-mon[35220]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T08:38:45.234 INFO:tasks.cephadm.ceph_manager.ceph:need seq 339302416415 got 339302416420 for osd.11 2024-09-10T08:38:45.235 DEBUG:teuthology.parallel:result is None 2024-09-10T08:38:45.376 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574933 got 55834574936 for osd.1 2024-09-10T08:38:45.376 DEBUG:teuthology.parallel:result is None 2024-09-10T08:38:45.779 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:45 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2240774646' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T08:38:45.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.581+0000 7f2416600640 1 -- 172.21.15.89:0/2878122084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24101079c0 msgr2=0x7f2410107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:45.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.581+0000 7f2416600640 1 --2- 172.21.15.89:0/2878122084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24101079c0 0x7f2410107da0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2404009890 tx=0x7f240402a360 comp rx=0 tx=0).stop 2024-09-10T08:38:45.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.581+0000 7f2416600640 1 -- 172.21.15.89:0/2878122084 shutdown_connections 2024-09-10T08:38:45.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.581+0000 7f2416600640 1 --2- 172.21.15.89:0/2878122084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2410102360 0x7f241010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:45.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.581+0000 7f2416600640 1 --2- 172.21.15.89:0/2878122084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24101019c0 0x7f2410101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:45.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.581+0000 7f2416600640 1 --2- 172.21.15.89:0/2878122084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24101079c0 0x7f2410107da0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:45.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.581+0000 7f2416600640 1 -- 172.21.15.89:0/2878122084 >> 172.21.15.89:0/2878122084 conn(0x7f24100fd740 msgr2=0x7f24100ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:45.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.581+0000 7f2416600640 1 -- 172.21.15.89:0/2878122084 shutdown_connections 2024-09-10T08:38:45.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.581+0000 7f2416600640 1 -- 172.21.15.89:0/2878122084 wait complete. 2024-09-10T08:38:45.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.582+0000 7f2416600640 1 Processor -- start 2024-09-10T08:38:45.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.582+0000 7f2416600640 1 -- start start 2024-09-10T08:38:45.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.583+0000 7f2416600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24101019c0 0x7f2410198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:45.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.583+0000 7f2416600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2410102360 0x7f2410198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:45.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.583+0000 7f2416600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24101079c0 0x7f241019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:45.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.583+0000 7f2416600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2410111dc0 con 0x7f24101019c0 2024-09-10T08:38:45.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.583+0000 7f2416600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2410111c40 con 0x7f2410102360 2024-09-10T08:38:45.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.583+0000 7f2416600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2410111f40 con 0x7f24101079c0 2024-09-10T08:38:45.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.583+0000 7f2415200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24101019c0 0x7f2410198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:45.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.583+0000 7f2415200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24101019c0 0x7f2410198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34810/0 (socket says 172.21.15.89:34810) 2024-09-10T08:38:45.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.583+0000 7f2415200640 1 -- 172.21.15.89:0/185220140 learned_addr learned my addr 172.21.15.89:0/185220140 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:45.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.583+0000 7f240fe00640 1 --2- 172.21.15.89:0/185220140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2410102360 0x7f2410198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:45.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.583+0000 7f2415c00640 1 --2- 172.21.15.89:0/185220140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24101079c0 0x7f241019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:45.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.584+0000 7f2415200640 1 -- 172.21.15.89:0/185220140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24101079c0 msgr2=0x7f241019cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:45.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.584+0000 7f2415200640 1 --2- 172.21.15.89:0/185220140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24101079c0 0x7f241019cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:45.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.584+0000 7f2415200640 1 -- 172.21.15.89:0/185220140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2410102360 msgr2=0x7f2410198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:45.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.584+0000 7f2415200640 1 --2- 172.21.15.89:0/185220140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2410102360 0x7f2410198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:45.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.584+0000 7f2415200640 1 -- 172.21.15.89:0/185220140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241019d2b0 con 0x7f24101019c0 2024-09-10T08:38:45.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.584+0000 7f240fe00640 1 --2- 172.21.15.89:0/185220140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2410102360 0x7f2410198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:38:45.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.584+0000 7f2415200640 1 --2- 172.21.15.89:0/185220140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24101019c0 0x7f2410198300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2404000c00 tx=0x7f2404033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:45.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.584+0000 7f240d600640 1 -- 172.21.15.89:0/185220140 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240403f020 con 0x7f24101019c0 2024-09-10T08:38:45.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.584+0000 7f2416600640 1 -- 172.21.15.89:0/185220140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241019d540 con 0x7f24101019c0 2024-09-10T08:38:45.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.584+0000 7f240d600640 1 -- 172.21.15.89:0/185220140 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2404032770 con 0x7f24101019c0 2024-09-10T08:38:45.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.584+0000 7f240d600640 1 -- 172.21.15.89:0/185220140 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404037040 con 0x7f24101019c0 2024-09-10T08:38:45.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.584+0000 7f2416600640 1 -- 172.21.15.89:0/185220140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24101a4e80 con 0x7f24101019c0 2024-09-10T08:38:45.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.585+0000 7f2416600640 1 -- 172.21.15.89:0/185220140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2410101e20 con 0x7f24101019c0 2024-09-10T08:38:45.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.590+0000 7f240d600640 1 -- 172.21.15.89:0/185220140 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2404042020 con 0x7f24101019c0 2024-09-10T08:38:45.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.591+0000 7f240d600640 1 --2- 172.21.15.89:0/185220140 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23f80a2c90 0x7f23f80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:45.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.591+0000 7f240d600640 1 -- 172.21.15.89:0/185220140 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f24040ef520 con 0x7f24101019c0 2024-09-10T08:38:45.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.591+0000 7f240fe00640 1 --2- 172.21.15.89:0/185220140 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23f80a2c90 0x7f23f80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:45.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.592+0000 7f240fe00640 1 --2- 172.21.15.89:0/185220140 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23f80a2c90 0x7f23f80a5150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f24100ffd50 tx=0x7f23fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:45.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.593+0000 7f240d600640 1 -- 172.21.15.89:0/185220140 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2404049030 con 0x7f24101019c0 2024-09-10T08:38:45.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.778+0000 7f2416600640 1 -- 172.21.15.89:0/185220140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f2410109f30 con 0x7f24101019c0 2024-09-10T08:38:45.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.778+0000 7f240d600640 1 -- 172.21.15.89:0/185220140 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f24040bc080 con 0x7f24101019c0 2024-09-10T08:38:45.825 INFO:teuthology.orchestra.run.smithi089.stdout:163208757316 2024-09-10T08:38:45.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.781+0000 7f2416600640 1 -- 172.21.15.89:0/185220140 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23f80a2c90 msgr2=0x7f23f80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:45.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.781+0000 7f2416600640 1 --2- 172.21.15.89:0/185220140 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23f80a2c90 0x7f23f80a5150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f24100ffd50 tx=0x7f23fc005f90 comp rx=0 tx=0).stop 2024-09-10T08:38:45.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.781+0000 7f2416600640 1 -- 172.21.15.89:0/185220140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24101019c0 msgr2=0x7f2410198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:45.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.781+0000 7f2416600640 1 --2- 172.21.15.89:0/185220140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24101019c0 0x7f2410198300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2404000c00 tx=0x7f2404033d10 comp rx=0 tx=0).stop 2024-09-10T08:38:45.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.781+0000 7f2416600640 1 -- 172.21.15.89:0/185220140 shutdown_connections 2024-09-10T08:38:45.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.781+0000 7f2416600640 1 --2- 172.21.15.89:0/185220140 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23f80a2c90 0x7f23f80a5150 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:45.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.781+0000 7f2416600640 1 --2- 172.21.15.89:0/185220140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24101079c0 0x7f241019cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:45.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.781+0000 7f2416600640 1 --2- 172.21.15.89:0/185220140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2410102360 0x7f2410198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:45.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.781+0000 7f2416600640 1 --2- 172.21.15.89:0/185220140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24101019c0 0x7f2410198300 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:45.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.781+0000 7f2416600640 1 -- 172.21.15.89:0/185220140 >> 172.21.15.89:0/185220140 conn(0x7f24100fd740 msgr2=0x7f24100fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:45.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.781+0000 7f2416600640 1 -- 172.21.15.89:0/185220140 shutdown_connections 2024-09-10T08:38:45.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:45.781+0000 7f2416600640 1 -- 172.21.15.89:0/185220140 wait complete. 2024-09-10T08:38:45.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:45 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2240774646' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T08:38:46.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:45 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2240774646' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T08:38:46.700 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757312 got 163208757316 for osd.5 2024-09-10T08:38:46.701 DEBUG:teuthology.parallel:result is None 2024-09-10T08:38:46.792 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:46 smithi089 ceph-mon[30757]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-10T08:38:46.793 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:46 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/185220140' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T08:38:46.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:46 smithi149 ceph-mon[36167]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-10T08:38:46.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:46 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/185220140' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T08:38:47.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:46 smithi111 ceph-mon[35220]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-10T08:38:47.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:46 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/185220140' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T08:38:47.143 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495675 got 197568495678 for osd.6 2024-09-10T08:38:47.144 DEBUG:teuthology.parallel:result is None 2024-09-10T08:38:47.480 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:47.509 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411408 got 77309411412 for osd.2 2024-09-10T08:38:47.509 DEBUG:teuthology.parallel:result is None 2024-09-10T08:38:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.056+0000 7fd486400640 1 -- 172.21.15.89:0/3181707995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd480074590 msgr2=0x7fd480074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.056+0000 7fd486400640 1 --2- 172.21.15.89:0/3181707995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd480074590 0x7fd480074970 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd470009900 tx=0x7fd47002a480 comp rx=0 tx=0).stop 2024-09-10T08:38:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.056+0000 7fd486400640 1 -- 172.21.15.89:0/3181707995 shutdown_connections 2024-09-10T08:38:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.056+0000 7fd486400640 1 --2- 172.21.15.89:0/3181707995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd480112b90 0x7fd480114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.056+0000 7fd486400640 1 --2- 172.21.15.89:0/3181707995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd480074eb0 0x7fd480112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.056+0000 7fd486400640 1 --2- 172.21.15.89:0/3181707995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd480074590 0x7fd480074970 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.056+0000 7fd486400640 1 -- 172.21.15.89:0/3181707995 >> 172.21.15.89:0/3181707995 conn(0x7fd4800fd8c0 msgr2=0x7fd4800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.056+0000 7fd486400640 1 -- 172.21.15.89:0/3181707995 shutdown_connections 2024-09-10T08:38:48.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.056+0000 7fd486400640 1 -- 172.21.15.89:0/3181707995 wait complete. 2024-09-10T08:38:48.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.057+0000 7fd486400640 1 Processor -- start 2024-09-10T08:38:48.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.057+0000 7fd486400640 1 -- start start 2024-09-10T08:38:48.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.057+0000 7fd486400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd480074590 0x7fd4801a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:48.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.057+0000 7fd486400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd480074eb0 0x7fd4801a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:48.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.057+0000 7fd486400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd480112b90 0x7fd48019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:48.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.057+0000 7fd486400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd480118570 con 0x7fd480074590 2024-09-10T08:38:48.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.057+0000 7fd486400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801183f0 con 0x7fd480112b90 2024-09-10T08:38:48.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.057+0000 7fd486400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801186f0 con 0x7fd480074eb0 2024-09-10T08:38:48.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.058+0000 7fd47f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd480074590 0x7fd4801a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:48.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.058+0000 7fd47ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd480074eb0 0x7fd4801a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:48.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.058+0000 7fd47f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd480074590 0x7fd4801a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34826/0 (socket says 172.21.15.89:34826) 2024-09-10T08:38:48.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.058+0000 7fd47f400640 1 -- 172.21.15.89:0/817825391 learned_addr learned my addr 172.21.15.89:0/817825391 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:48.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.058+0000 7fd47fe00640 1 --2- 172.21.15.89:0/817825391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd480112b90 0x7fd48019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:48.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.058+0000 7fd47f400640 1 -- 172.21.15.89:0/817825391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd480074eb0 msgr2=0x7fd4801a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:48.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.058+0000 7fd47f400640 1 --2- 172.21.15.89:0/817825391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd480074eb0 0x7fd4801a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:48.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.058+0000 7fd47f400640 1 -- 172.21.15.89:0/817825391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd480112b90 msgr2=0x7fd48019e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:48.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.058+0000 7fd47f400640 1 --2- 172.21.15.89:0/817825391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd480112b90 0x7fd48019e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:48.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.058+0000 7fd47f400640 1 -- 172.21.15.89:0/817825391 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48019f010 con 0x7fd480074590 2024-09-10T08:38:48.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.058+0000 7fd47fe00640 1 --2- 172.21.15.89:0/817825391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd480112b90 0x7fd48019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:38:48.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.059+0000 7fd47f400640 1 --2- 172.21.15.89:0/817825391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd480074590 0x7fd4801a4720 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd470006010 tx=0x7fd470004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:48.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.059+0000 7fd477e00640 1 -- 172.21.15.89:0/817825391 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47003f030 con 0x7fd480074590 2024-09-10T08:38:48.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.059+0000 7fd477e00640 1 -- 172.21.15.89:0/817825391 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd470004440 con 0x7fd480074590 2024-09-10T08:38:48.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.059+0000 7fd486400640 1 -- 172.21.15.89:0/817825391 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48019f2a0 con 0x7fd480074590 2024-09-10T08:38:48.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.059+0000 7fd477e00640 1 -- 172.21.15.89:0/817825391 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470035070 con 0x7fd480074590 2024-09-10T08:38:48.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.059+0000 7fd486400640 1 -- 172.21.15.89:0/817825391 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4801ab5f0 con 0x7fd480074590 2024-09-10T08:38:48.061 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.060+0000 7fd486400640 1 -- 172.21.15.89:0/817825391 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd480074970 con 0x7fd480074590 2024-09-10T08:38:48.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.062+0000 7fd477e00640 1 -- 172.21.15.89:0/817825391 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd470042030 con 0x7fd480074590 2024-09-10T08:38:48.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.063+0000 7fd477e00640 1 --2- 172.21.15.89:0/817825391 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd4640a2c10 0x7fd4640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:48.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.063+0000 7fd477e00640 1 -- 172.21.15.89:0/817825391 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fd4700efc60 con 0x7fd480074590 2024-09-10T08:38:48.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.064+0000 7fd47ea00640 1 --2- 172.21.15.89:0/817825391 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd4640a2c10 0x7fd4640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:48.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.064+0000 7fd47ea00640 1 --2- 172.21.15.89:0/817825391 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd4640a2c10 0x7fd4640a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd4800fff50 tx=0x7fd46c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:48.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.065+0000 7fd477e00640 1 -- 172.21.15.89:0/817825391 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd4700bc0a0 con 0x7fd480074590 2024-09-10T08:38:48.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.245+0000 7fd486400640 1 -- 172.21.15.89:0/817825391 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd4801ab7c0 con 0x7fd480074590 2024-09-10T08:38:48.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.245+0000 7fd477e00640 1 -- 172.21.15.89:0/817825391 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd4700c03a0 con 0x7fd480074590 2024-09-10T08:38:48.246 INFO:teuthology.orchestra.run.smithi089.stdout:34359738462 2024-09-10T08:38:48.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.247+0000 7fd486400640 1 -- 172.21.15.89:0/817825391 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd4640a2c10 msgr2=0x7fd4640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:48.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.247+0000 7fd486400640 1 --2- 172.21.15.89:0/817825391 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd4640a2c10 0x7fd4640a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd4800fff50 tx=0x7fd46c005f90 comp rx=0 tx=0).stop 2024-09-10T08:38:48.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.247+0000 7fd486400640 1 -- 172.21.15.89:0/817825391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd480074590 msgr2=0x7fd4801a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:48.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.247+0000 7fd486400640 1 --2- 172.21.15.89:0/817825391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd480074590 0x7fd4801a4720 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd470006010 tx=0x7fd470004040 comp rx=0 tx=0).stop 2024-09-10T08:38:48.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.247+0000 7fd486400640 1 -- 172.21.15.89:0/817825391 shutdown_connections 2024-09-10T08:38:48.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.247+0000 7fd486400640 1 --2- 172.21.15.89:0/817825391 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd4640a2c10 0x7fd4640a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:48.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.247+0000 7fd486400640 1 --2- 172.21.15.89:0/817825391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd480112b90 0x7fd48019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:48.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.247+0000 7fd486400640 1 --2- 172.21.15.89:0/817825391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd480074eb0 0x7fd4801a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:48.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.247+0000 7fd486400640 1 --2- 172.21.15.89:0/817825391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd480074590 0x7fd4801a4720 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:48.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.247+0000 7fd486400640 1 -- 172.21.15.89:0/817825391 >> 172.21.15.89:0/817825391 conn(0x7fd4800fd8c0 msgr2=0x7fd4800ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:48.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.248+0000 7fd486400640 1 -- 172.21.15.89:0/817825391 shutdown_connections 2024-09-10T08:38:48.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:48.248+0000 7fd486400640 1 -- 172.21.15.89:0/817825391 wait complete. 2024-09-10T08:38:48.969 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:48 smithi089 ceph-mon[30757]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 145 B/s, 0 objects/s recovering 2024-09-10T08:38:48.969 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:48 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/817825391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T08:38:48.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:48 smithi149 ceph-mon[36167]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 145 B/s, 0 objects/s recovering 2024-09-10T08:38:48.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:48 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/817825391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T08:38:49.006 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738459 got 34359738462 for osd.0 2024-09-10T08:38:49.007 DEBUG:teuthology.parallel:result is None 2024-09-10T08:38:49.007 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T08:38:49.007 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:38:49.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:48 smithi111 ceph-mon[35220]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 145 B/s, 0 objects/s recovering 2024-09-10T08:38:49.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:48 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/817825391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T08:38:49.377 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:49.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.949+0000 7f5139000640 1 -- 172.21.15.89:0/1384373602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5134074620 msgr2=0x7f5134074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:49.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.949+0000 7f5139000640 1 --2- 172.21.15.89:0/1384373602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5134074620 0x7f5134074a00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5124009900 tx=0x7f512402a480 comp rx=0 tx=0).stop 2024-09-10T08:38:49.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.949+0000 7f5139000640 1 -- 172.21.15.89:0/1384373602 shutdown_connections 2024-09-10T08:38:49.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.949+0000 7f5139000640 1 --2- 172.21.15.89:0/1384373602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f513410e3c0 0x7f5134110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:49.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.949+0000 7f5139000640 1 --2- 172.21.15.89:0/1384373602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5134074f40 0x7f513410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:49.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.949+0000 7f5139000640 1 --2- 172.21.15.89:0/1384373602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5134074620 0x7f5134074a00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:49.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.949+0000 7f5139000640 1 -- 172.21.15.89:0/1384373602 >> 172.21.15.89:0/1384373602 conn(0x7f51340fd8c0 msgr2=0x7f51340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:49.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.949+0000 7f5139000640 1 -- 172.21.15.89:0/1384373602 shutdown_connections 2024-09-10T08:38:49.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.949+0000 7f5139000640 1 -- 172.21.15.89:0/1384373602 wait complete. 2024-09-10T08:38:49.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.950+0000 7f5139000640 1 Processor -- start 2024-09-10T08:38:49.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.950+0000 7f5139000640 1 -- start start 2024-09-10T08:38:49.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.951+0000 7f5139000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5134074620 0x7f513419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:49.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.951+0000 7f5139000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5134074f40 0x7f513419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:49.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.951+0000 7f5139000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f513410e3c0 0x7f513419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:49.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.951+0000 7f5139000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5134114170 con 0x7f513410e3c0 2024-09-10T08:38:49.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.951+0000 7f5139000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5134113ff0 con 0x7f5134074620 2024-09-10T08:38:49.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.951+0000 7f5139000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f51341142f0 con 0x7f5134074f40 2024-09-10T08:38:49.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.951+0000 7f5131e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5134074620 0x7f513419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:49.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.951+0000 7f5131e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5134074620 0x7f513419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:54522/0 (socket says 172.21.15.89:54522) 2024-09-10T08:38:49.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.951+0000 7f5131e00640 1 -- 172.21.15.89:0/3912507704 learned_addr learned my addr 172.21.15.89:0/3912507704 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:49.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.951+0000 7f5129400640 1 --2- 172.21.15.89:0/3912507704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5134074f40 0x7f513419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:49.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.951+0000 7f5132800640 1 --2- 172.21.15.89:0/3912507704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f513410e3c0 0x7f513419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:49.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.952+0000 7f5131e00640 1 -- 172.21.15.89:0/3912507704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5134074f40 msgr2=0x7f513419ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:49.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.952+0000 7f5131e00640 1 --2- 172.21.15.89:0/3912507704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5134074f40 0x7f513419ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:49.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.952+0000 7f5131e00640 1 -- 172.21.15.89:0/3912507704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f513410e3c0 msgr2=0x7f513419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:49.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.952+0000 7f5131e00640 1 --2- 172.21.15.89:0/3912507704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f513410e3c0 0x7f513419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:49.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.952+0000 7f5131e00640 1 -- 172.21.15.89:0/3912507704 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f513419f6c0 con 0x7f5134074620 2024-09-10T08:38:49.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.952+0000 7f5132800640 1 --2- 172.21.15.89:0/3912507704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f513410e3c0 0x7f513419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:38:49.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.952+0000 7f5131e00640 1 --2- 172.21.15.89:0/3912507704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5134074620 0x7f513419a710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5124006010 tx=0x7f51240042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:49.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.952+0000 7f512b400640 1 -- 172.21.15.89:0/3912507704 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512403f030 con 0x7f5134074620 2024-09-10T08:38:49.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.952+0000 7f5139000640 1 -- 172.21.15.89:0/3912507704 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f513419f950 con 0x7f5134074620 2024-09-10T08:38:49.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.952+0000 7f512b400640 1 -- 172.21.15.89:0/3912507704 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5124003ce0 con 0x7f5134074620 2024-09-10T08:38:49.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.952+0000 7f512b400640 1 -- 172.21.15.89:0/3912507704 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124035070 con 0x7f5134074620 2024-09-10T08:38:49.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.952+0000 7f5139000640 1 -- 172.21.15.89:0/3912507704 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51341a7290 con 0x7f5134074620 2024-09-10T08:38:49.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.953+0000 7f5139000640 1 -- 172.21.15.89:0/3912507704 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5134075cd0 con 0x7f5134074620 2024-09-10T08:38:49.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.955+0000 7f512b400640 1 -- 172.21.15.89:0/3912507704 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5124042030 con 0x7f5134074620 2024-09-10T08:38:49.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.956+0000 7f512b400640 1 --2- 172.21.15.89:0/3912507704 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f51180a2c10 0x7f51180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:49.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.957+0000 7f512b400640 1 -- 172.21.15.89:0/3912507704 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f51240efe20 con 0x7f5134074620 2024-09-10T08:38:49.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.957+0000 7f5129400640 1 --2- 172.21.15.89:0/3912507704 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f51180a2c10 0x7f51180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:49.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.957+0000 7f5129400640 1 --2- 172.21.15.89:0/3912507704 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f51180a2c10 0x7f51180a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f513410f840 tx=0x7f511c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:49.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:49.959+0000 7f512b400640 1 -- 172.21.15.89:0/3912507704 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f51240bc8b0 con 0x7f5134074620 2024-09-10T08:38:50.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:50.130+0000 7f5139000640 1 -- 172.21.15.89:0/3912507704 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5134062630 con 0x7f51180a2c10 2024-09-10T08:38:50.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:50.141+0000 7f512b400640 1 -- 172.21.15.89:0/3912507704 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340712 (secure 0 0 0) 0x7f5134062630 con 0x7f51180a2c10 2024-09-10T08:38:50.146 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:38:50.148 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-09-10T08:38:50.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:50.146+0000 7f5139000640 1 -- 172.21.15.89:0/3912507704 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f51180a2c10 msgr2=0x7f51180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:50.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:50.146+0000 7f5139000640 1 --2- 172.21.15.89:0/3912507704 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f51180a2c10 0x7f51180a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f513410f840 tx=0x7f511c00b390 comp rx=0 tx=0).stop 2024-09-10T08:38:50.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:50.146+0000 7f5139000640 1 -- 172.21.15.89:0/3912507704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5134074620 msgr2=0x7f513419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:50.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:50.146+0000 7f5139000640 1 --2- 172.21.15.89:0/3912507704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5134074620 0x7f513419a710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5124006010 tx=0x7f51240042f0 comp rx=0 tx=0).stop 2024-09-10T08:38:50.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:50.146+0000 7f5139000640 1 -- 172.21.15.89:0/3912507704 shutdown_connections 2024-09-10T08:38:50.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:50.146+0000 7f5139000640 1 --2- 172.21.15.89:0/3912507704 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f51180a2c10 0x7f51180a50d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:50.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:50.146+0000 7f5139000640 1 --2- 172.21.15.89:0/3912507704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f513410e3c0 0x7f513419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:50.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:50.146+0000 7f5139000640 1 --2- 172.21.15.89:0/3912507704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5134074f40 0x7f513419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:50.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:50.146+0000 7f5139000640 1 --2- 172.21.15.89:0/3912507704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5134074620 0x7f513419a710 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:50.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:50.146+0000 7f5139000640 1 -- 172.21.15.89:0/3912507704 >> 172.21.15.89:0/3912507704 conn(0x7f51340fd8c0 msgr2=0x7f51340fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:50.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:50.146+0000 7f5139000640 1 -- 172.21.15.89:0/3912507704 shutdown_connections 2024-09-10T08:38:50.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:50.146+0000 7f5139000640 1 -- 172.21.15.89:0/3912507704 wait complete. 2024-09-10T08:38:50.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:50 smithi089 ceph-mon[30757]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s; 145 B/s, 0 objects/s recovering 2024-09-10T08:38:50.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:50 smithi149 ceph-mon[36167]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s; 145 B/s, 0 objects/s recovering 2024-09-10T08:38:51.006 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-10T08:38:49.597726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658116,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":491,"num_read_kb":362,"num_write":347,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181900,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358028,"kb_used_data":12872,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583796,"statfs":{"total":1151940427776,"available":1151573807104,"internally_reserved":0,"allocated":13180928,"data_stored":8518190,"data_compressed":467733,"data_compressed_allocated":638976,"data_compressed_original":2777088,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":14,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002558"},"pg_stats":[{"pgid":"2.1f","version":"94'9","reported_seq":66,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.663079+0000","last_change":"2024-09-10T08:38:43.663079+0000","last_active":"2024-09-10T08:38:43.663079+0000","last_peered":"2024-09-10T08:38:43.663079+0000","last_clean":"2024-09-10T08:38:43.663079+0000","last_became_active":"2024-09-10T08:38:43.656506+0000","last_became_peered":"2024-09-10T08:38:43.656506+0000","last_unstale":"2024-09-10T08:38:43.663079+0000","last_undegraded":"2024-09-10T08:38:43.663079+0000","last_fullsized":"2024-09-10T08:38:43.663079+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:36.881564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631528+0000","last_change":"2024-09-10T08:36:56.145816+0000","last_active":"2024-09-10T08:38:43.631528+0000","last_peered":"2024-09-10T08:38:43.631528+0000","last_clean":"2024-09-10T08:38:43.631528+0000","last_became_active":"2024-09-10T08:36:56.145462+0000","last_became_peered":"2024-09-10T08:36:56.145462+0000","last_unstale":"2024-09-10T08:38:43.631528+0000","last_undegraded":"2024-09-10T08:38:43.631528+0000","last_fullsized":"2024-09-10T08:38:43.631528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:53.890190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629816+0000","last_change":"2024-09-10T08:37:05.252761+0000","last_active":"2024-09-10T08:38:43.629816+0000","last_peered":"2024-09-10T08:38:43.629816+0000","last_clean":"2024-09-10T08:38:43.629816+0000","last_became_active":"2024-09-10T08:37:05.252179+0000","last_became_peered":"2024-09-10T08:37:05.252179+0000","last_unstale":"2024-09-10T08:38:43.629816+0000","last_undegraded":"2024-09-10T08:38:43.629816+0000","last_fullsized":"2024-09-10T08:38:43.629816+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:52.854828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631803+0000","last_change":"2024-09-10T08:36:53.735650+0000","last_active":"2024-09-10T08:38:43.631803+0000","last_peered":"2024-09-10T08:38:43.631803+0000","last_clean":"2024-09-10T08:38:43.631803+0000","last_became_active":"2024-09-10T08:36:53.735128+0000","last_became_peered":"2024-09-10T08:36:53.735128+0000","last_unstale":"2024-09-10T08:38:43.631803+0000","last_undegraded":"2024-09-10T08:38:43.631803+0000","last_fullsized":"2024-09-10T08:38:43.631803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:27.309591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633500+0000","last_change":"2024-09-10T08:36:56.153364+0000","last_active":"2024-09-10T08:38:43.633500+0000","last_peered":"2024-09-10T08:38:43.633500+0000","last_clean":"2024-09-10T08:38:43.633500+0000","last_became_active":"2024-09-10T08:36:56.152979+0000","last_became_peered":"2024-09-10T08:36:56.152979+0000","last_unstale":"2024-09-10T08:38:43.633500+0000","last_undegraded":"2024-09-10T08:38:43.633500+0000","last_fullsized":"2024-09-10T08:38:43.633500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:12.172620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654710+0000","last_change":"2024-09-10T08:37:05.254551+0000","last_active":"2024-09-10T08:38:43.654710+0000","last_peered":"2024-09-10T08:38:43.654710+0000","last_clean":"2024-09-10T08:38:43.654710+0000","last_became_active":"2024-09-10T08:37:05.254055+0000","last_became_peered":"2024-09-10T08:37:05.254055+0000","last_unstale":"2024-09-10T08:38:43.654710+0000","last_undegraded":"2024-09-10T08:38:43.654710+0000","last_fullsized":"2024-09-10T08:38:43.654710+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:08.472502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626455+0000","last_change":"2024-09-10T08:36:53.722774+0000","last_active":"2024-09-10T08:38:43.626455+0000","last_peered":"2024-09-10T08:38:43.626455+0000","last_clean":"2024-09-10T08:38:43.626455+0000","last_became_active":"2024-09-10T08:36:53.722459+0000","last_became_peered":"2024-09-10T08:36:53.722459+0000","last_unstale":"2024-09-10T08:38:43.626455+0000","last_undegraded":"2024-09-10T08:38:43.626455+0000","last_fullsized":"2024-09-10T08:38:43.626455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:26.612354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630770+0000","last_change":"2024-09-10T08:36:56.143757+0000","last_active":"2024-09-10T08:38:43.630770+0000","last_peered":"2024-09-10T08:38:43.630770+0000","last_clean":"2024-09-10T08:38:43.630770+0000","last_became_active":"2024-09-10T08:36:56.143430+0000","last_became_peered":"2024-09-10T08:36:56.143430+0000","last_unstale":"2024-09-10T08:38:43.630770+0000","last_undegraded":"2024-09-10T08:38:43.630770+0000","last_fullsized":"2024-09-10T08:38:43.630770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:49.737298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655071+0000","last_change":"2024-09-10T08:37:05.254503+0000","last_active":"2024-09-10T08:38:43.655071+0000","last_peered":"2024-09-10T08:38:43.655071+0000","last_clean":"2024-09-10T08:38:43.655071+0000","last_became_active":"2024-09-10T08:37:05.254070+0000","last_became_peered":"2024-09-10T08:37:05.254070+0000","last_unstale":"2024-09-10T08:38:43.655071+0000","last_undegraded":"2024-09-10T08:38:43.655071+0000","last_fullsized":"2024-09-10T08:38:43.655071+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:16.741698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626406+0000","last_change":"2024-09-10T08:36:53.716742+0000","last_active":"2024-09-10T08:38:43.626406+0000","last_peered":"2024-09-10T08:38:43.626406+0000","last_clean":"2024-09-10T08:38:43.626406+0000","last_became_active":"2024-09-10T08:36:53.716439+0000","last_became_peered":"2024-09-10T08:36:53.716439+0000","last_unstale":"2024-09-10T08:38:43.626406+0000","last_undegraded":"2024-09-10T08:38:43.626406+0000","last_fullsized":"2024-09-10T08:38:43.626406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:46.882805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655230+0000","last_change":"2024-09-10T08:36:56.146841+0000","last_active":"2024-09-10T08:38:43.655230+0000","last_peered":"2024-09-10T08:38:43.655230+0000","last_clean":"2024-09-10T08:38:43.655230+0000","last_became_active":"2024-09-10T08:36:56.146531+0000","last_became_peered":"2024-09-10T08:36:56.146531+0000","last_unstale":"2024-09-10T08:38:43.655230+0000","last_undegraded":"2024-09-10T08:38:43.655230+0000","last_fullsized":"2024-09-10T08:38:43.655230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:37.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654823+0000","last_change":"2024-09-10T08:37:43.223181+0000","last_active":"2024-09-10T08:38:43.654823+0000","last_peered":"2024-09-10T08:38:43.654823+0000","last_clean":"2024-09-10T08:38:43.654823+0000","last_became_active":"2024-09-10T08:37:43.222794+0000","last_became_peered":"2024-09-10T08:37:43.222794+0000","last_unstale":"2024-09-10T08:38:43.654823+0000","last_undegraded":"2024-09-10T08:38:43.654823+0000","last_fullsized":"2024-09-10T08:38:43.654823+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:22.204517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631680+0000","last_change":"2024-09-10T08:36:56.142161+0000","last_active":"2024-09-10T08:38:43.631680+0000","last_peered":"2024-09-10T08:38:43.631680+0000","last_clean":"2024-09-10T08:38:43.631680+0000","last_became_active":"2024-09-10T08:36:56.141838+0000","last_became_peered":"2024-09-10T08:36:56.141838+0000","last_unstale":"2024-09-10T08:38:43.631680+0000","last_undegraded":"2024-09-10T08:38:43.631680+0000","last_fullsized":"2024-09-10T08:38:43.631680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:09.909747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626066+0000","last_change":"2024-09-10T08:36:53.722238+0000","last_active":"2024-09-10T08:38:43.626066+0000","last_peered":"2024-09-10T08:38:43.626066+0000","last_clean":"2024-09-10T08:38:43.626066+0000","last_became_active":"2024-09-10T08:36:53.721859+0000","last_became_peered":"2024-09-10T08:36:53.721859+0000","last_unstale":"2024-09-10T08:38:43.626066+0000","last_undegraded":"2024-09-10T08:38:43.626066+0000","last_fullsized":"2024-09-10T08:38:43.626066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:50.615090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655128+0000","last_change":"2024-09-10T08:37:05.414041+0000","last_active":"2024-09-10T08:38:43.655128+0000","last_peered":"2024-09-10T08:38:43.655128+0000","last_clean":"2024-09-10T08:38:43.655128+0000","last_became_active":"2024-09-10T08:37:05.413286+0000","last_became_peered":"2024-09-10T08:37:05.413286+0000","last_unstale":"2024-09-10T08:38:43.655128+0000","last_undegraded":"2024-09-10T08:38:43.655128+0000","last_fullsized":"2024-09-10T08:38:43.655128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:44.916102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631587+0000","last_change":"2024-09-10T08:36:56.158278+0000","last_active":"2024-09-10T08:38:43.631587+0000","last_peered":"2024-09-10T08:38:43.631587+0000","last_clean":"2024-09-10T08:38:43.631587+0000","last_became_active":"2024-09-10T08:36:56.157946+0000","last_became_peered":"2024-09-10T08:36:56.157946+0000","last_unstale":"2024-09-10T08:38:43.631587+0000","last_undegraded":"2024-09-10T08:38:43.631587+0000","last_fullsized":"2024-09-10T08:38:43.631587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:42.512513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.635498+0000","last_change":"2024-09-10T08:38:43.635077+0000","last_active":"2024-09-10T08:38:43.635498+0000","last_peered":"2024-09-10T08:38:43.635498+0000","last_clean":"2024-09-10T08:38:43.635498+0000","last_became_active":"2024-09-10T08:38:43.631957+0000","last_became_peered":"2024-09-10T08:38:43.631957+0000","last_unstale":"2024-09-10T08:38:43.635498+0000","last_undegraded":"2024-09-10T08:38:43.635498+0000","last_fullsized":"2024-09-10T08:38:43.635498+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:50.837247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655061+0000","last_change":"2024-09-10T08:37:42.671399+0000","last_active":"2024-09-10T08:38:43.655061+0000","last_peered":"2024-09-10T08:38:43.655061+0000","last_clean":"2024-09-10T08:38:43.655061+0000","last_became_active":"2024-09-10T08:37:42.670865+0000","last_became_peered":"2024-09-10T08:37:42.670865+0000","last_unstale":"2024-09-10T08:38:43.655061+0000","last_undegraded":"2024-09-10T08:38:43.655061+0000","last_fullsized":"2024-09-10T08:38:43.655061+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:16.231538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627914+0000","last_change":"2024-09-10T08:36:56.144856+0000","last_active":"2024-09-10T08:38:43.627914+0000","last_peered":"2024-09-10T08:38:43.627914+0000","last_clean":"2024-09-10T08:38:43.627914+0000","last_became_active":"2024-09-10T08:36:56.144531+0000","last_became_peered":"2024-09-10T08:36:56.144531+0000","last_unstale":"2024-09-10T08:38:43.627914+0000","last_undegraded":"2024-09-10T08:38:43.627914+0000","last_fullsized":"2024-09-10T08:38:43.627914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:24.586006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631493+0000","last_change":"2024-09-10T08:36:53.721093+0000","last_active":"2024-09-10T08:38:43.631493+0000","last_peered":"2024-09-10T08:38:43.631493+0000","last_clean":"2024-09-10T08:38:43.631493+0000","last_became_active":"2024-09-10T08:36:53.720538+0000","last_became_peered":"2024-09-10T08:36:53.720538+0000","last_unstale":"2024-09-10T08:38:43.631493+0000","last_undegraded":"2024-09-10T08:38:43.631493+0000","last_fullsized":"2024-09-10T08:38:43.631493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:58.492463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629856+0000","last_change":"2024-09-10T08:37:05.254011+0000","last_active":"2024-09-10T08:38:43.629856+0000","last_peered":"2024-09-10T08:38:43.629856+0000","last_clean":"2024-09-10T08:38:43.629856+0000","last_became_active":"2024-09-10T08:37:05.253066+0000","last_became_peered":"2024-09-10T08:37:05.253066+0000","last_unstale":"2024-09-10T08:38:43.629856+0000","last_undegraded":"2024-09-10T08:38:43.629856+0000","last_fullsized":"2024-09-10T08:38:43.629856+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:16.221302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.625842+0000","last_change":"2024-09-10T08:36:56.141814+0000","last_active":"2024-09-10T08:38:43.625842+0000","last_peered":"2024-09-10T08:38:43.625842+0000","last_clean":"2024-09-10T08:38:43.625842+0000","last_became_active":"2024-09-10T08:36:56.141480+0000","last_became_peered":"2024-09-10T08:36:56.141480+0000","last_unstale":"2024-09-10T08:38:43.625842+0000","last_undegraded":"2024-09-10T08:38:43.625842+0000","last_fullsized":"2024-09-10T08:38:43.625842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:40.594354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.635917+0000","last_change":"2024-09-10T08:38:43.635917+0000","last_active":"2024-09-10T08:38:43.635917+0000","last_peered":"2024-09-10T08:38:43.635917+0000","last_clean":"2024-09-10T08:38:43.635917+0000","last_became_active":"2024-09-10T08:38:43.633406+0000","last_became_peered":"2024-09-10T08:38:43.633406+0000","last_unstale":"2024-09-10T08:38:43.635917+0000","last_undegraded":"2024-09-10T08:38:43.635917+0000","last_fullsized":"2024-09-10T08:38:43.635917+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:44.619072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655247+0000","last_change":"2024-09-10T08:37:05.414181+0000","last_active":"2024-09-10T08:38:43.655247+0000","last_peered":"2024-09-10T08:38:43.655247+0000","last_clean":"2024-09-10T08:38:43.655247+0000","last_became_active":"2024-09-10T08:37:05.413691+0000","last_became_peered":"2024-09-10T08:37:05.413691+0000","last_unstale":"2024-09-10T08:38:43.655247+0000","last_undegraded":"2024-09-10T08:38:43.655247+0000","last_fullsized":"2024-09-10T08:38:43.655247+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:30.744949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633275+0000","last_change":"2024-09-10T08:36:56.153956+0000","last_active":"2024-09-10T08:38:43.633275+0000","last_peered":"2024-09-10T08:38:43.633275+0000","last_clean":"2024-09-10T08:38:43.633275+0000","last_became_active":"2024-09-10T08:36:56.153609+0000","last_became_peered":"2024-09-10T08:36:56.153609+0000","last_unstale":"2024-09-10T08:38:43.633275+0000","last_undegraded":"2024-09-10T08:38:43.633275+0000","last_fullsized":"2024-09-10T08:38:43.633275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:18.185704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655491+0000","last_change":"2024-09-10T08:36:53.733217+0000","last_active":"2024-09-10T08:38:43.655491+0000","last_peered":"2024-09-10T08:38:43.655491+0000","last_clean":"2024-09-10T08:38:43.655491+0000","last_became_active":"2024-09-10T08:36:53.732851+0000","last_became_peered":"2024-09-10T08:36:53.732851+0000","last_unstale":"2024-09-10T08:38:43.655491+0000","last_undegraded":"2024-09-10T08:38:43.655491+0000","last_fullsized":"2024-09-10T08:38:43.655491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:04:01.132196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629833+0000","last_change":"2024-09-10T08:37:05.241868+0000","last_active":"2024-09-10T08:38:43.629833+0000","last_peered":"2024-09-10T08:38:43.629833+0000","last_clean":"2024-09-10T08:38:43.629833+0000","last_became_active":"2024-09-10T08:37:05.241412+0000","last_became_peered":"2024-09-10T08:37:05.241412+0000","last_unstale":"2024-09-10T08:38:43.629833+0000","last_undegraded":"2024-09-10T08:38:43.629833+0000","last_fullsized":"2024-09-10T08:38:43.629833+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:52.543353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631931+0000","last_change":"2024-09-10T08:36:56.150722+0000","last_active":"2024-09-10T08:38:43.631931+0000","last_peered":"2024-09-10T08:38:43.631931+0000","last_clean":"2024-09-10T08:38:43.631931+0000","last_became_active":"2024-09-10T08:36:56.150423+0000","last_became_peered":"2024-09-10T08:36:56.150423+0000","last_unstale":"2024-09-10T08:38:43.631931+0000","last_undegraded":"2024-09-10T08:38:43.631931+0000","last_fullsized":"2024-09-10T08:38:43.631931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:50.272841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'10","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629985+0000","last_change":"2024-09-10T08:36:53.722477+0000","last_active":"2024-09-10T08:38:43.629985+0000","last_peered":"2024-09-10T08:38:43.629985+0000","last_clean":"2024-09-10T08:38:43.629985+0000","last_became_active":"2024-09-10T08:36:53.722118+0000","last_became_peered":"2024-09-10T08:36:53.722118+0000","last_unstale":"2024-09-10T08:38:43.629985+0000","last_undegraded":"2024-09-10T08:38:43.629985+0000","last_fullsized":"2024-09-10T08:38:43.629985+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:06.631646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630935+0000","last_change":"2024-09-10T08:37:05.415656+0000","last_active":"2024-09-10T08:38:43.630935+0000","last_peered":"2024-09-10T08:38:43.630935+0000","last_clean":"2024-09-10T08:38:43.630935+0000","last_became_active":"2024-09-10T08:37:05.413537+0000","last_became_peered":"2024-09-10T08:37:05.413537+0000","last_unstale":"2024-09-10T08:38:43.630935+0000","last_undegraded":"2024-09-10T08:38:43.630935+0000","last_fullsized":"2024-09-10T08:38:43.630935+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:04.284410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628492+0000","last_change":"2024-09-10T08:36:56.144157+0000","last_active":"2024-09-10T08:38:43.628492+0000","last_peered":"2024-09-10T08:38:43.628492+0000","last_clean":"2024-09-10T08:38:43.628492+0000","last_became_active":"2024-09-10T08:36:56.140332+0000","last_became_peered":"2024-09-10T08:36:56.140332+0000","last_unstale":"2024-09-10T08:38:43.628492+0000","last_undegraded":"2024-09-10T08:38:43.628492+0000","last_fullsized":"2024-09-10T08:38:43.628492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:09:26.097987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626457+0000","last_change":"2024-09-10T08:36:53.735465+0000","last_active":"2024-09-10T08:38:43.626457+0000","last_peered":"2024-09-10T08:38:43.626457+0000","last_clean":"2024-09-10T08:38:43.626457+0000","last_became_active":"2024-09-10T08:36:53.735160+0000","last_became_peered":"2024-09-10T08:36:53.735160+0000","last_unstale":"2024-09-10T08:38:43.626457+0000","last_undegraded":"2024-09-10T08:38:43.626457+0000","last_fullsized":"2024-09-10T08:38:43.626457+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:26:52.332085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627895+0000","last_change":"2024-09-10T08:37:05.247671+0000","last_active":"2024-09-10T08:38:43.627895+0000","last_peered":"2024-09-10T08:38:43.627895+0000","last_clean":"2024-09-10T08:38:43.627895+0000","last_became_active":"2024-09-10T08:37:05.247217+0000","last_became_peered":"2024-09-10T08:37:05.247217+0000","last_unstale":"2024-09-10T08:38:43.627895+0000","last_undegraded":"2024-09-10T08:38:43.627895+0000","last_fullsized":"2024-09-10T08:38:43.627895+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:20.881304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627370+0000","last_change":"2024-09-10T08:36:56.145420+0000","last_active":"2024-09-10T08:38:43.627370+0000","last_peered":"2024-09-10T08:38:43.627370+0000","last_clean":"2024-09-10T08:38:43.627370+0000","last_became_active":"2024-09-10T08:36:56.145074+0000","last_became_peered":"2024-09-10T08:36:56.145074+0000","last_unstale":"2024-09-10T08:38:43.627370+0000","last_undegraded":"2024-09-10T08:38:43.627370+0000","last_fullsized":"2024-09-10T08:38:43.627370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:13.849977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629766+0000","last_change":"2024-09-10T08:36:53.738237+0000","last_active":"2024-09-10T08:38:43.629766+0000","last_peered":"2024-09-10T08:38:43.629766+0000","last_clean":"2024-09-10T08:38:43.629766+0000","last_became_active":"2024-09-10T08:36:53.737858+0000","last_became_peered":"2024-09-10T08:36:53.737858+0000","last_unstale":"2024-09-10T08:38:43.629766+0000","last_undegraded":"2024-09-10T08:38:43.629766+0000","last_fullsized":"2024-09-10T08:38:43.629766+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:35.345661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.625917+0000","last_change":"2024-09-10T08:37:05.413512+0000","last_active":"2024-09-10T08:38:43.625917+0000","last_peered":"2024-09-10T08:38:43.625917+0000","last_clean":"2024-09-10T08:38:43.625917+0000","last_became_active":"2024-09-10T08:37:05.413010+0000","last_became_peered":"2024-09-10T08:37:05.413010+0000","last_unstale":"2024-09-10T08:38:43.625917+0000","last_undegraded":"2024-09-10T08:38:43.625917+0000","last_fullsized":"2024-09-10T08:38:43.625917+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:05.349404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627152+0000","last_change":"2024-09-10T08:36:56.157741+0000","last_active":"2024-09-10T08:38:43.627152+0000","last_peered":"2024-09-10T08:38:43.627152+0000","last_clean":"2024-09-10T08:38:43.627152+0000","last_became_active":"2024-09-10T08:36:56.157295+0000","last_became_peered":"2024-09-10T08:36:56.157295+0000","last_unstale":"2024-09-10T08:38:43.627152+0000","last_undegraded":"2024-09-10T08:38:43.627152+0000","last_fullsized":"2024-09-10T08:38:43.627152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:31.845864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631783+0000","last_change":"2024-09-10T08:36:53.722373+0000","last_active":"2024-09-10T08:38:43.631783+0000","last_peered":"2024-09-10T08:38:43.631783+0000","last_clean":"2024-09-10T08:38:43.631783+0000","last_became_active":"2024-09-10T08:36:53.721975+0000","last_became_peered":"2024-09-10T08:36:53.721975+0000","last_unstale":"2024-09-10T08:38:43.631783+0000","last_undegraded":"2024-09-10T08:38:43.631783+0000","last_fullsized":"2024-09-10T08:38:43.631783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:44.593018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628307+0000","last_change":"2024-09-10T08:37:05.255362+0000","last_active":"2024-09-10T08:38:43.628307+0000","last_peered":"2024-09-10T08:38:43.628307+0000","last_clean":"2024-09-10T08:38:43.628307+0000","last_became_active":"2024-09-10T08:37:05.254967+0000","last_became_peered":"2024-09-10T08:37:05.254967+0000","last_unstale":"2024-09-10T08:38:43.628307+0000","last_undegraded":"2024-09-10T08:38:43.628307+0000","last_fullsized":"2024-09-10T08:38:43.628307+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:31:41.850961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654865+0000","last_change":"2024-09-10T08:36:56.147699+0000","last_active":"2024-09-10T08:38:43.654865+0000","last_peered":"2024-09-10T08:38:43.654865+0000","last_clean":"2024-09-10T08:38:43.654865+0000","last_became_active":"2024-09-10T08:36:56.147349+0000","last_became_peered":"2024-09-10T08:36:56.147349+0000","last_unstale":"2024-09-10T08:38:43.654865+0000","last_undegraded":"2024-09-10T08:38:43.654865+0000","last_fullsized":"2024-09-10T08:38:43.654865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:25.248588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'4","reported_seq":101,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.799243+0000","last_change":"2024-09-10T08:36:53.736843+0000","last_active":"2024-09-10T08:38:43.799243+0000","last_peered":"2024-09-10T08:38:43.799243+0000","last_clean":"2024-09-10T08:38:43.799243+0000","last_became_active":"2024-09-10T08:36:53.736340+0000","last_became_peered":"2024-09-10T08:36:53.736340+0000","last_unstale":"2024-09-10T08:38:43.799243+0000","last_undegraded":"2024-09-10T08:38:43.799243+0000","last_fullsized":"2024-09-10T08:38:43.799243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:24.679679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":57,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628492+0000","last_change":"2024-09-10T08:37:05.414321+0000","last_active":"2024-09-10T08:38:43.628492+0000","last_peered":"2024-09-10T08:38:43.628492+0000","last_clean":"2024-09-10T08:38:43.628492+0000","last_became_active":"2024-09-10T08:37:05.413868+0000","last_became_peered":"2024-09-10T08:37:05.413868+0000","last_unstale":"2024-09-10T08:38:43.628492+0000","last_undegraded":"2024-09-10T08:38:43.628492+0000","last_fullsized":"2024-09-10T08:38:43.628492+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:53.988077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627830+0000","last_change":"2024-09-10T08:36:56.141910+0000","last_active":"2024-09-10T08:38:43.627830+0000","last_peered":"2024-09-10T08:38:43.627830+0000","last_clean":"2024-09-10T08:38:43.627830+0000","last_became_active":"2024-09-10T08:36:56.141579+0000","last_became_peered":"2024-09-10T08:36:56.141579+0000","last_unstale":"2024-09-10T08:38:43.627830+0000","last_undegraded":"2024-09-10T08:38:43.627830+0000","last_fullsized":"2024-09-10T08:38:43.627830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:18.464831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631695+0000","last_change":"2024-09-10T08:36:53.720102+0000","last_active":"2024-09-10T08:38:43.631695+0000","last_peered":"2024-09-10T08:38:43.631695+0000","last_clean":"2024-09-10T08:38:43.631695+0000","last_became_active":"2024-09-10T08:36:53.719764+0000","last_became_peered":"2024-09-10T08:36:53.719764+0000","last_unstale":"2024-09-10T08:38:43.631695+0000","last_undegraded":"2024-09-10T08:38:43.631695+0000","last_fullsized":"2024-09-10T08:38:43.631695+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:03.609338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655157+0000","last_change":"2024-09-10T08:37:05.413818+0000","last_active":"2024-09-10T08:38:43.655157+0000","last_peered":"2024-09-10T08:38:43.655157+0000","last_clean":"2024-09-10T08:38:43.655157+0000","last_became_active":"2024-09-10T08:37:05.413267+0000","last_became_peered":"2024-09-10T08:37:05.413267+0000","last_unstale":"2024-09-10T08:38:43.655157+0000","last_undegraded":"2024-09-10T08:38:43.655157+0000","last_fullsized":"2024-09-10T08:38:43.655157+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:54.309686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631369+0000","last_change":"2024-09-10T08:36:56.149286+0000","last_active":"2024-09-10T08:38:43.631369+0000","last_peered":"2024-09-10T08:38:43.631369+0000","last_clean":"2024-09-10T08:38:43.631369+0000","last_became_active":"2024-09-10T08:36:56.146275+0000","last_became_peered":"2024-09-10T08:36:56.146275+0000","last_unstale":"2024-09-10T08:38:43.631369+0000","last_undegraded":"2024-09-10T08:38:43.631369+0000","last_fullsized":"2024-09-10T08:38:43.631369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:19.468224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655198+0000","last_change":"2024-09-10T08:36:53.719756+0000","last_active":"2024-09-10T08:38:43.655198+0000","last_peered":"2024-09-10T08:38:43.655198+0000","last_clean":"2024-09-10T08:38:43.655198+0000","last_became_active":"2024-09-10T08:36:53.719302+0000","last_became_peered":"2024-09-10T08:36:53.719302+0000","last_unstale":"2024-09-10T08:38:43.655198+0000","last_undegraded":"2024-09-10T08:38:43.655198+0000","last_fullsized":"2024-09-10T08:38:43.655198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:47.658081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631788+0000","last_change":"2024-09-10T08:35:39.150715+0000","last_active":"2024-09-10T08:38:43.631788+0000","last_peered":"2024-09-10T08:38:43.631788+0000","last_clean":"2024-09-10T08:38:43.631788+0000","last_became_active":"2024-09-10T08:35:39.150206+0000","last_became_peered":"2024-09-10T08:35:39.150206+0000","last_unstale":"2024-09-10T08:38:43.631788+0000","last_undegraded":"2024-09-10T08:38:43.631788+0000","last_fullsized":"2024-09-10T08:38:43.631788+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:32:02.110870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:32:02.110870+0000","last_clean_scrub_stamp":"2024-09-10T08:32:02.110870+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:20.762043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631951+0000","last_change":"2024-09-10T08:37:42.846547+0000","last_active":"2024-09-10T08:38:43.631951+0000","last_peered":"2024-09-10T08:38:43.631951+0000","last_clean":"2024-09-10T08:38:43.631951+0000","last_became_active":"2024-09-10T08:37:42.845884+0000","last_became_peered":"2024-09-10T08:37:42.845884+0000","last_unstale":"2024-09-10T08:38:43.631951+0000","last_undegraded":"2024-09-10T08:38:43.631951+0000","last_fullsized":"2024-09-10T08:38:43.631951+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:21.905133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.632062+0000","last_change":"2024-09-10T08:36:56.144187+0000","last_active":"2024-09-10T08:38:43.632062+0000","last_peered":"2024-09-10T08:38:43.632062+0000","last_clean":"2024-09-10T08:38:43.632062+0000","last_became_active":"2024-09-10T08:36:56.143193+0000","last_became_peered":"2024-09-10T08:36:56.143193+0000","last_unstale":"2024-09-10T08:38:43.632062+0000","last_undegraded":"2024-09-10T08:38:43.632062+0000","last_fullsized":"2024-09-10T08:38:43.632062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:14:26.981884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.639660+0000","last_change":"2024-09-10T08:38:43.639286+0000","last_active":"2024-09-10T08:38:43.639660+0000","last_peered":"2024-09-10T08:38:43.639660+0000","last_clean":"2024-09-10T08:38:43.639660+0000","last_became_active":"2024-09-10T08:38:43.633552+0000","last_became_peered":"2024-09-10T08:38:43.633552+0000","last_unstale":"2024-09-10T08:38:43.639660+0000","last_undegraded":"2024-09-10T08:38:43.639660+0000","last_fullsized":"2024-09-10T08:38:43.639660+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:23.985190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630150+0000","last_change":"2024-09-10T08:37:05.242095+0000","last_active":"2024-09-10T08:38:43.630150+0000","last_peered":"2024-09-10T08:38:43.630150+0000","last_clean":"2024-09-10T08:38:43.630150+0000","last_became_active":"2024-09-10T08:37:05.241645+0000","last_became_peered":"2024-09-10T08:37:05.241645+0000","last_unstale":"2024-09-10T08:38:43.630150+0000","last_undegraded":"2024-09-10T08:38:43.630150+0000","last_fullsized":"2024-09-10T08:38:43.630150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:30:49.637861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631813+0000","last_change":"2024-09-10T08:36:56.149396+0000","last_active":"2024-09-10T08:38:43.631813+0000","last_peered":"2024-09-10T08:38:43.631813+0000","last_clean":"2024-09-10T08:38:43.631813+0000","last_became_active":"2024-09-10T08:36:56.149079+0000","last_became_peered":"2024-09-10T08:36:56.149079+0000","last_unstale":"2024-09-10T08:38:43.631813+0000","last_undegraded":"2024-09-10T08:38:43.631813+0000","last_fullsized":"2024-09-10T08:38:43.631813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:44.665448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655423+0000","last_change":"2024-09-10T08:36:53.717950+0000","last_active":"2024-09-10T08:38:43.655423+0000","last_peered":"2024-09-10T08:38:43.655423+0000","last_clean":"2024-09-10T08:38:43.655423+0000","last_became_active":"2024-09-10T08:36:53.717668+0000","last_became_peered":"2024-09-10T08:36:53.717668+0000","last_unstale":"2024-09-10T08:38:43.655423+0000","last_undegraded":"2024-09-10T08:38:43.655423+0000","last_fullsized":"2024-09-10T08:38:43.655423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:58.692631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633095+0000","last_change":"2024-09-10T08:37:05.260178+0000","last_active":"2024-09-10T08:38:43.633095+0000","last_peered":"2024-09-10T08:38:43.633095+0000","last_clean":"2024-09-10T08:38:43.633095+0000","last_became_active":"2024-09-10T08:37:05.259753+0000","last_became_peered":"2024-09-10T08:37:05.259753+0000","last_unstale":"2024-09-10T08:38:43.633095+0000","last_undegraded":"2024-09-10T08:38:43.633095+0000","last_fullsized":"2024-09-10T08:38:43.633095+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:19.364782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.632330+0000","last_change":"2024-09-10T08:36:56.147104+0000","last_active":"2024-09-10T08:38:43.632330+0000","last_peered":"2024-09-10T08:38:43.632330+0000","last_clean":"2024-09-10T08:38:43.632330+0000","last_became_active":"2024-09-10T08:36:56.146778+0000","last_became_peered":"2024-09-10T08:36:56.146778+0000","last_unstale":"2024-09-10T08:38:43.632330+0000","last_undegraded":"2024-09-10T08:38:43.632330+0000","last_fullsized":"2024-09-10T08:38:43.632330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:56.092157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628351+0000","last_change":"2024-09-10T08:36:53.719145+0000","last_active":"2024-09-10T08:38:43.628351+0000","last_peered":"2024-09-10T08:38:43.628351+0000","last_clean":"2024-09-10T08:38:43.628351+0000","last_became_active":"2024-09-10T08:36:53.718771+0000","last_became_peered":"2024-09-10T08:36:53.718771+0000","last_unstale":"2024-09-10T08:38:43.628351+0000","last_undegraded":"2024-09-10T08:38:43.628351+0000","last_fullsized":"2024-09-10T08:38:43.628351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:21.109940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654677+0000","last_change":"2024-09-10T08:37:05.242854+0000","last_active":"2024-09-10T08:38:43.654677+0000","last_peered":"2024-09-10T08:38:43.654677+0000","last_clean":"2024-09-10T08:38:43.654677+0000","last_became_active":"2024-09-10T08:37:05.242442+0000","last_became_peered":"2024-09-10T08:37:05.242442+0000","last_unstale":"2024-09-10T08:38:43.654677+0000","last_undegraded":"2024-09-10T08:38:43.654677+0000","last_fullsized":"2024-09-10T08:38:43.654677+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:36.072131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628176+0000","last_change":"2024-09-10T08:37:42.845642+0000","last_active":"2024-09-10T08:38:43.628176+0000","last_peered":"2024-09-10T08:38:43.628176+0000","last_clean":"2024-09-10T08:38:43.628176+0000","last_became_active":"2024-09-10T08:37:42.845217+0000","last_became_peered":"2024-09-10T08:37:42.845217+0000","last_unstale":"2024-09-10T08:38:43.628176+0000","last_undegraded":"2024-09-10T08:38:43.628176+0000","last_fullsized":"2024-09-10T08:38:43.628176+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:28.020337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626705+0000","last_change":"2024-09-10T08:36:53.720463+0000","last_active":"2024-09-10T08:38:43.626705+0000","last_peered":"2024-09-10T08:38:43.626705+0000","last_clean":"2024-09-10T08:38:43.626705+0000","last_became_active":"2024-09-10T08:36:53.720080+0000","last_became_peered":"2024-09-10T08:36:53.720080+0000","last_unstale":"2024-09-10T08:38:43.626705+0000","last_undegraded":"2024-09-10T08:38:43.626705+0000","last_fullsized":"2024-09-10T08:38:43.626705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:51.450294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654831+0000","last_change":"2024-09-10T08:37:05.251261+0000","last_active":"2024-09-10T08:38:43.654831+0000","last_peered":"2024-09-10T08:38:43.654831+0000","last_clean":"2024-09-10T08:38:43.654831+0000","last_became_active":"2024-09-10T08:37:05.250938+0000","last_became_peered":"2024-09-10T08:37:05.250938+0000","last_unstale":"2024-09-10T08:38:43.654831+0000","last_undegraded":"2024-09-10T08:38:43.654831+0000","last_fullsized":"2024-09-10T08:38:43.654831+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:07:15.129899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654959+0000","last_change":"2024-09-10T08:36:56.148191+0000","last_active":"2024-09-10T08:38:43.654959+0000","last_peered":"2024-09-10T08:38:43.654959+0000","last_clean":"2024-09-10T08:38:43.654959+0000","last_became_active":"2024-09-10T08:36:56.147362+0000","last_became_peered":"2024-09-10T08:36:56.147362+0000","last_unstale":"2024-09-10T08:38:43.654959+0000","last_undegraded":"2024-09-10T08:38:43.654959+0000","last_fullsized":"2024-09-10T08:38:43.654959+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:25:30.051754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":56,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.625844+0000","last_change":"2024-09-10T08:36:53.734484+0000","last_active":"2024-09-10T08:38:43.625844+0000","last_peered":"2024-09-10T08:38:43.625844+0000","last_clean":"2024-09-10T08:38:43.625844+0000","last_became_active":"2024-09-10T08:36:53.734172+0000","last_became_peered":"2024-09-10T08:36:53.734172+0000","last_unstale":"2024-09-10T08:38:43.625844+0000","last_undegraded":"2024-09-10T08:38:43.625844+0000","last_fullsized":"2024-09-10T08:38:43.625844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:10.269400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628251+0000","last_change":"2024-09-10T08:37:05.259635+0000","last_active":"2024-09-10T08:38:43.628251+0000","last_peered":"2024-09-10T08:38:43.628251+0000","last_clean":"2024-09-10T08:38:43.628251+0000","last_became_active":"2024-09-10T08:37:05.259107+0000","last_became_peered":"2024-09-10T08:37:05.259107+0000","last_unstale":"2024-09-10T08:38:43.628251+0000","last_undegraded":"2024-09-10T08:38:43.628251+0000","last_fullsized":"2024-09-10T08:38:43.628251+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:36.841138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631420+0000","last_change":"2024-09-10T08:36:56.141122+0000","last_active":"2024-09-10T08:38:43.631420+0000","last_peered":"2024-09-10T08:38:43.631420+0000","last_clean":"2024-09-10T08:38:43.631420+0000","last_became_active":"2024-09-10T08:36:56.140826+0000","last_became_peered":"2024-09-10T08:36:56.140826+0000","last_unstale":"2024-09-10T08:38:43.631420+0000","last_undegraded":"2024-09-10T08:38:43.631420+0000","last_fullsized":"2024-09-10T08:38:43.631420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:38.044090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'4","reported_seq":104,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.800068+0000","last_change":"2024-09-10T08:36:53.720623+0000","last_active":"2024-09-10T08:38:43.800068+0000","last_peered":"2024-09-10T08:38:43.800068+0000","last_clean":"2024-09-10T08:38:43.800068+0000","last_became_active":"2024-09-10T08:36:53.720270+0000","last_became_peered":"2024-09-10T08:36:53.720270+0000","last_unstale":"2024-09-10T08:38:43.800068+0000","last_undegraded":"2024-09-10T08:38:43.800068+0000","last_fullsized":"2024-09-10T08:38:43.800068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:19.514068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629891+0000","last_change":"2024-09-10T08:37:05.254324+0000","last_active":"2024-09-10T08:38:43.629891+0000","last_peered":"2024-09-10T08:38:43.629891+0000","last_clean":"2024-09-10T08:38:43.629891+0000","last_became_active":"2024-09-10T08:37:05.253611+0000","last_became_peered":"2024-09-10T08:37:05.253611+0000","last_unstale":"2024-09-10T08:38:43.629891+0000","last_undegraded":"2024-09-10T08:38:43.629891+0000","last_fullsized":"2024-09-10T08:38:43.629891+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:07.182649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.632053+0000","last_change":"2024-09-10T08:36:56.139102+0000","last_active":"2024-09-10T08:38:43.632053+0000","last_peered":"2024-09-10T08:38:43.632053+0000","last_clean":"2024-09-10T08:38:43.632053+0000","last_became_active":"2024-09-10T08:36:56.137986+0000","last_became_peered":"2024-09-10T08:36:56.137986+0000","last_unstale":"2024-09-10T08:38:43.632053+0000","last_undegraded":"2024-09-10T08:38:43.632053+0000","last_fullsized":"2024-09-10T08:38:43.632053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:05:03.005727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654954+0000","last_change":"2024-09-10T08:36:53.722961+0000","last_active":"2024-09-10T08:38:43.654954+0000","last_peered":"2024-09-10T08:38:43.654954+0000","last_clean":"2024-09-10T08:38:43.654954+0000","last_became_active":"2024-09-10T08:36:53.722628+0000","last_became_peered":"2024-09-10T08:36:53.722628+0000","last_unstale":"2024-09-10T08:38:43.654954+0000","last_undegraded":"2024-09-10T08:38:43.654954+0000","last_fullsized":"2024-09-10T08:38:43.654954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:59.048632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627916+0000","last_change":"2024-09-10T08:37:05.256546+0000","last_active":"2024-09-10T08:38:43.627916+0000","last_peered":"2024-09-10T08:38:43.627916+0000","last_clean":"2024-09-10T08:38:43.627916+0000","last_became_active":"2024-09-10T08:37:05.256239+0000","last_became_peered":"2024-09-10T08:37:05.256239+0000","last_unstale":"2024-09-10T08:38:43.627916+0000","last_undegraded":"2024-09-10T08:38:43.627916+0000","last_fullsized":"2024-09-10T08:38:43.627916+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:57.403934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627921+0000","last_change":"2024-09-10T08:36:56.149285+0000","last_active":"2024-09-10T08:38:43.627921+0000","last_peered":"2024-09-10T08:38:43.627921+0000","last_clean":"2024-09-10T08:38:43.627921+0000","last_became_active":"2024-09-10T08:36:56.146873+0000","last_became_peered":"2024-09-10T08:36:56.146873+0000","last_unstale":"2024-09-10T08:38:43.627921+0000","last_undegraded":"2024-09-10T08:38:43.627921+0000","last_fullsized":"2024-09-10T08:38:43.627921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:28.976729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630229+0000","last_change":"2024-09-10T08:36:53.721174+0000","last_active":"2024-09-10T08:38:43.630229+0000","last_peered":"2024-09-10T08:38:43.630229+0000","last_clean":"2024-09-10T08:38:43.630229+0000","last_became_active":"2024-09-10T08:36:53.720849+0000","last_became_peered":"2024-09-10T08:36:53.720849+0000","last_unstale":"2024-09-10T08:38:43.630229+0000","last_undegraded":"2024-09-10T08:38:43.630229+0000","last_fullsized":"2024-09-10T08:38:43.630229+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:39.786848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655227+0000","last_change":"2024-09-10T08:37:05.252256+0000","last_active":"2024-09-10T08:38:43.655227+0000","last_peered":"2024-09-10T08:38:43.655227+0000","last_clean":"2024-09-10T08:38:43.655227+0000","last_became_active":"2024-09-10T08:37:05.251865+0000","last_became_peered":"2024-09-10T08:37:05.251865+0000","last_unstale":"2024-09-10T08:38:43.655227+0000","last_undegraded":"2024-09-10T08:38:43.655227+0000","last_fullsized":"2024-09-10T08:38:43.655227+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:08.581880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626344+0000","last_change":"2024-09-10T08:36:56.142060+0000","last_active":"2024-09-10T08:38:43.626344+0000","last_peered":"2024-09-10T08:38:43.626344+0000","last_clean":"2024-09-10T08:38:43.626344+0000","last_became_active":"2024-09-10T08:36:56.141451+0000","last_became_peered":"2024-09-10T08:36:56.141451+0000","last_unstale":"2024-09-10T08:38:43.626344+0000","last_undegraded":"2024-09-10T08:38:43.626344+0000","last_fullsized":"2024-09-10T08:38:43.626344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:43.566588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'32","reported_seq":192,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.803133+0000","last_change":"2024-09-10T08:36:53.719464+0000","last_active":"2024-09-10T08:38:43.803133+0000","last_peered":"2024-09-10T08:38:43.803133+0000","last_clean":"2024-09-10T08:38:43.803133+0000","last_became_active":"2024-09-10T08:36:53.719129+0000","last_became_peered":"2024-09-10T08:36:53.719129+0000","last_unstale":"2024-09-10T08:38:43.803133+0000","last_undegraded":"2024-09-10T08:38:43.803133+0000","last_fullsized":"2024-09-10T08:38:43.803133+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:05.652018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63511,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":122,"num_write":33,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629358+0000","last_change":"2024-09-10T08:37:05.255576+0000","last_active":"2024-09-10T08:38:43.629358+0000","last_peered":"2024-09-10T08:38:43.629358+0000","last_clean":"2024-09-10T08:38:43.629358+0000","last_became_active":"2024-09-10T08:37:05.249842+0000","last_became_peered":"2024-09-10T08:37:05.249842+0000","last_unstale":"2024-09-10T08:38:43.629358+0000","last_undegraded":"2024-09-10T08:38:43.629358+0000","last_fullsized":"2024-09-10T08:38:43.629358+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:47:49.543481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630860+0000","last_change":"2024-09-10T08:36:56.151829+0000","last_active":"2024-09-10T08:38:43.630860+0000","last_peered":"2024-09-10T08:38:43.630860+0000","last_clean":"2024-09-10T08:38:43.630860+0000","last_became_active":"2024-09-10T08:36:56.151314+0000","last_became_peered":"2024-09-10T08:36:56.151314+0000","last_unstale":"2024-09-10T08:38:43.630860+0000","last_undegraded":"2024-09-10T08:38:43.630860+0000","last_fullsized":"2024-09-10T08:38:43.630860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:44.219671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655360+0000","last_change":"2024-09-10T08:36:53.738030+0000","last_active":"2024-09-10T08:38:43.655360+0000","last_peered":"2024-09-10T08:38:43.655360+0000","last_clean":"2024-09-10T08:38:43.655360+0000","last_became_active":"2024-09-10T08:36:53.737658+0000","last_became_peered":"2024-09-10T08:36:53.737658+0000","last_unstale":"2024-09-10T08:38:43.655360+0000","last_undegraded":"2024-09-10T08:38:43.655360+0000","last_fullsized":"2024-09-10T08:38:43.655360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:29.627554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654959+0000","last_change":"2024-09-10T08:37:05.244473+0000","last_active":"2024-09-10T08:38:43.654959+0000","last_peered":"2024-09-10T08:38:43.654959+0000","last_clean":"2024-09-10T08:38:43.654959+0000","last_became_active":"2024-09-10T08:37:05.244035+0000","last_became_peered":"2024-09-10T08:37:05.244035+0000","last_unstale":"2024-09-10T08:38:43.654959+0000","last_undegraded":"2024-09-10T08:38:43.654959+0000","last_fullsized":"2024-09-10T08:38:43.654959+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:04.652660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633507+0000","last_change":"2024-09-10T08:36:56.155217+0000","last_active":"2024-09-10T08:38:43.633507+0000","last_peered":"2024-09-10T08:38:43.633507+0000","last_clean":"2024-09-10T08:38:43.633507+0000","last_became_active":"2024-09-10T08:36:56.154903+0000","last_became_peered":"2024-09-10T08:36:56.154903+0000","last_unstale":"2024-09-10T08:38:43.633507+0000","last_undegraded":"2024-09-10T08:38:43.633507+0000","last_fullsized":"2024-09-10T08:38:43.633507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:47.482974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.632193+0000","last_change":"2024-09-10T08:36:53.721087+0000","last_active":"2024-09-10T08:38:43.632193+0000","last_peered":"2024-09-10T08:38:43.632193+0000","last_clean":"2024-09-10T08:38:43.632193+0000","last_became_active":"2024-09-10T08:36:53.720772+0000","last_became_peered":"2024-09-10T08:36:53.720772+0000","last_unstale":"2024-09-10T08:38:43.632193+0000","last_undegraded":"2024-09-10T08:38:43.632193+0000","last_fullsized":"2024-09-10T08:38:43.632193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:39.560848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629112+0000","last_change":"2024-09-10T08:37:05.254641+0000","last_active":"2024-09-10T08:38:43.629112+0000","last_peered":"2024-09-10T08:38:43.629112+0000","last_clean":"2024-09-10T08:38:43.629112+0000","last_became_active":"2024-09-10T08:37:05.249982+0000","last_became_peered":"2024-09-10T08:37:05.249982+0000","last_unstale":"2024-09-10T08:38:43.629112+0000","last_undegraded":"2024-09-10T08:38:43.629112+0000","last_fullsized":"2024-09-10T08:38:43.629112+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:21:13.851061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630749+0000","last_change":"2024-09-10T08:36:56.142657+0000","last_active":"2024-09-10T08:38:43.630749+0000","last_peered":"2024-09-10T08:38:43.630749+0000","last_clean":"2024-09-10T08:38:43.630749+0000","last_became_active":"2024-09-10T08:36:56.142294+0000","last_became_peered":"2024-09-10T08:36:56.142294+0000","last_unstale":"2024-09-10T08:38:43.630749+0000","last_undegraded":"2024-09-10T08:38:43.630749+0000","last_fullsized":"2024-09-10T08:38:43.630749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:52.861389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"94'3","reported_seq":99,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627551+0000","last_change":"2024-09-10T08:36:53.722466+0000","last_active":"2024-09-10T08:38:43.627551+0000","last_peered":"2024-09-10T08:38:43.627551+0000","last_clean":"2024-09-10T08:38:43.627551+0000","last_became_active":"2024-09-10T08:36:53.722145+0000","last_became_peered":"2024-09-10T08:36:53.722145+0000","last_unstale":"2024-09-10T08:38:43.627551+0000","last_undegraded":"2024-09-10T08:38:43.627551+0000","last_fullsized":"2024-09-10T08:38:43.627551+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:27.041765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631303+0000","last_change":"2024-09-10T08:37:05.251056+0000","last_active":"2024-09-10T08:38:43.631303+0000","last_peered":"2024-09-10T08:38:43.631303+0000","last_clean":"2024-09-10T08:38:43.631303+0000","last_became_active":"2024-09-10T08:37:05.250653+0000","last_became_peered":"2024-09-10T08:37:05.250653+0000","last_unstale":"2024-09-10T08:38:43.631303+0000","last_undegraded":"2024-09-10T08:38:43.631303+0000","last_fullsized":"2024-09-10T08:38:43.631303+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:21.174747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633073+0000","last_change":"2024-09-10T08:37:05.260436+0000","last_active":"2024-09-10T08:38:43.633073+0000","last_peered":"2024-09-10T08:38:43.633073+0000","last_clean":"2024-09-10T08:38:43.633073+0000","last_became_active":"2024-09-10T08:37:05.260049+0000","last_became_peered":"2024-09-10T08:37:05.260049+0000","last_unstale":"2024-09-10T08:38:43.633073+0000","last_undegraded":"2024-09-10T08:38:43.633073+0000","last_fullsized":"2024-09-10T08:38:43.633073+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:43.394028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629156+0000","last_change":"2024-09-10T08:36:56.146732+0000","last_active":"2024-09-10T08:38:43.629156+0000","last_peered":"2024-09-10T08:38:43.629156+0000","last_clean":"2024-09-10T08:38:43.629156+0000","last_became_active":"2024-09-10T08:36:56.145599+0000","last_became_peered":"2024-09-10T08:36:56.145599+0000","last_unstale":"2024-09-10T08:38:43.629156+0000","last_undegraded":"2024-09-10T08:38:43.629156+0000","last_fullsized":"2024-09-10T08:38:43.629156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:50.620956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655039+0000","last_change":"2024-09-10T08:36:53.721683+0000","last_active":"2024-09-10T08:38:43.655039+0000","last_peered":"2024-09-10T08:38:43.655039+0000","last_clean":"2024-09-10T08:38:43.655039+0000","last_became_active":"2024-09-10T08:36:53.721241+0000","last_became_peered":"2024-09-10T08:36:53.721241+0000","last_unstale":"2024-09-10T08:38:43.655039+0000","last_undegraded":"2024-09-10T08:38:43.655039+0000","last_fullsized":"2024-09-10T08:38:43.655039+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:03.669337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628220+0000","last_change":"2024-09-10T08:37:42.669784+0000","last_active":"2024-09-10T08:38:43.628220+0000","last_peered":"2024-09-10T08:38:43.628220+0000","last_clean":"2024-09-10T08:38:43.628220+0000","last_became_active":"2024-09-10T08:37:42.669322+0000","last_became_peered":"2024-09-10T08:37:42.669322+0000","last_unstale":"2024-09-10T08:38:43.628220+0000","last_undegraded":"2024-09-10T08:38:43.628220+0000","last_fullsized":"2024-09-10T08:38:43.628220+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:06:57.027928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631075+0000","last_change":"2024-09-10T08:36:56.143824+0000","last_active":"2024-09-10T08:38:43.631075+0000","last_peered":"2024-09-10T08:38:43.631075+0000","last_clean":"2024-09-10T08:38:43.631075+0000","last_became_active":"2024-09-10T08:36:56.143486+0000","last_became_peered":"2024-09-10T08:36:56.143486+0000","last_unstale":"2024-09-10T08:38:43.631075+0000","last_undegraded":"2024-09-10T08:38:43.631075+0000","last_fullsized":"2024-09-10T08:38:43.631075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:42.384863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627882+0000","last_change":"2024-09-10T08:36:53.717680+0000","last_active":"2024-09-10T08:38:43.627882+0000","last_peered":"2024-09-10T08:38:43.627882+0000","last_clean":"2024-09-10T08:38:43.627882+0000","last_became_active":"2024-09-10T08:36:53.717249+0000","last_became_peered":"2024-09-10T08:36:53.717249+0000","last_unstale":"2024-09-10T08:38:43.627882+0000","last_undegraded":"2024-09-10T08:38:43.627882+0000","last_fullsized":"2024-09-10T08:38:43.627882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:12.191541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627126+0000","last_change":"2024-09-10T08:36:53.723947+0000","last_active":"2024-09-10T08:38:43.627126+0000","last_peered":"2024-09-10T08:38:43.627126+0000","last_clean":"2024-09-10T08:38:43.627126+0000","last_became_active":"2024-09-10T08:36:53.723620+0000","last_became_peered":"2024-09-10T08:36:53.723620+0000","last_unstale":"2024-09-10T08:38:43.627126+0000","last_undegraded":"2024-09-10T08:38:43.627126+0000","last_fullsized":"2024-09-10T08:38:43.627126+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:00.135349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655037+0000","last_change":"2024-09-10T08:36:56.150448+0000","last_active":"2024-09-10T08:38:43.655037+0000","last_peered":"2024-09-10T08:38:43.655037+0000","last_clean":"2024-09-10T08:38:43.655037+0000","last_became_active":"2024-09-10T08:36:56.150148+0000","last_became_peered":"2024-09-10T08:36:56.150148+0000","last_unstale":"2024-09-10T08:38:43.655037+0000","last_undegraded":"2024-09-10T08:38:43.655037+0000","last_fullsized":"2024-09-10T08:38:43.655037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:02.927753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630130+0000","last_change":"2024-09-10T08:37:05.242878+0000","last_active":"2024-09-10T08:38:43.630130+0000","last_peered":"2024-09-10T08:38:43.630130+0000","last_clean":"2024-09-10T08:38:43.630130+0000","last_became_active":"2024-09-10T08:37:05.242440+0000","last_became_peered":"2024-09-10T08:37:05.242440+0000","last_unstale":"2024-09-10T08:38:43.630130+0000","last_undegraded":"2024-09-10T08:38:43.630130+0000","last_fullsized":"2024-09-10T08:38:43.630130+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:30.957433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'5","reported_seq":104,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.803485+0000","last_change":"2024-09-10T08:36:53.719788+0000","last_active":"2024-09-10T08:38:43.803485+0000","last_peered":"2024-09-10T08:38:43.803485+0000","last_clean":"2024-09-10T08:38:43.803485+0000","last_became_active":"2024-09-10T08:36:53.719462+0000","last_became_peered":"2024-09-10T08:36:53.719462+0000","last_unstale":"2024-09-10T08:38:43.803485+0000","last_undegraded":"2024-09-10T08:38:43.803485+0000","last_fullsized":"2024-09-10T08:38:43.803485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:12.250447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626009+0000","last_change":"2024-09-10T08:37:42.673008+0000","last_active":"2024-09-10T08:38:43.626009+0000","last_peered":"2024-09-10T08:38:43.626009+0000","last_clean":"2024-09-10T08:38:43.626009+0000","last_became_active":"2024-09-10T08:37:42.672618+0000","last_became_peered":"2024-09-10T08:37:42.672618+0000","last_unstale":"2024-09-10T08:38:43.626009+0000","last_undegraded":"2024-09-10T08:38:43.626009+0000","last_fullsized":"2024-09-10T08:38:43.626009+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:00.156565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630734+0000","last_change":"2024-09-10T08:37:05.244277+0000","last_active":"2024-09-10T08:38:43.630734+0000","last_peered":"2024-09-10T08:38:43.630734+0000","last_clean":"2024-09-10T08:38:43.630734+0000","last_became_active":"2024-09-10T08:37:05.243880+0000","last_became_peered":"2024-09-10T08:37:05.243880+0000","last_unstale":"2024-09-10T08:38:43.630734+0000","last_undegraded":"2024-09-10T08:38:43.630734+0000","last_fullsized":"2024-09-10T08:38:43.630734+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:55.275857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627792+0000","last_change":"2024-09-10T08:36:56.141404+0000","last_active":"2024-09-10T08:38:43.627792+0000","last_peered":"2024-09-10T08:38:43.627792+0000","last_clean":"2024-09-10T08:38:43.627792+0000","last_became_active":"2024-09-10T08:36:56.141076+0000","last_became_peered":"2024-09-10T08:36:56.141076+0000","last_unstale":"2024-09-10T08:38:43.627792+0000","last_undegraded":"2024-09-10T08:38:43.627792+0000","last_fullsized":"2024-09-10T08:38:43.627792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:03:52.915654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630119+0000","last_change":"2024-09-10T08:36:56.148285+0000","last_active":"2024-09-10T08:38:43.630119+0000","last_peered":"2024-09-10T08:38:43.630119+0000","last_clean":"2024-09-10T08:38:43.630119+0000","last_became_active":"2024-09-10T08:36:56.147970+0000","last_became_peered":"2024-09-10T08:36:56.147970+0000","last_unstale":"2024-09-10T08:38:43.630119+0000","last_undegraded":"2024-09-10T08:38:43.630119+0000","last_fullsized":"2024-09-10T08:38:43.630119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:09.506718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630184+0000","last_change":"2024-09-10T08:36:56.157977+0000","last_active":"2024-09-10T08:38:43.630184+0000","last_peered":"2024-09-10T08:38:43.630184+0000","last_clean":"2024-09-10T08:38:43.630184+0000","last_became_active":"2024-09-10T08:36:56.157623+0000","last_became_peered":"2024-09-10T08:36:56.157623+0000","last_unstale":"2024-09-10T08:38:43.630184+0000","last_undegraded":"2024-09-10T08:38:43.630184+0000","last_fullsized":"2024-09-10T08:38:43.630184+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:44.617979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633131+0000","last_change":"2024-09-10T08:37:42.670459+0000","last_active":"2024-09-10T08:38:43.633131+0000","last_peered":"2024-09-10T08:38:43.633131+0000","last_clean":"2024-09-10T08:38:43.633131+0000","last_became_active":"2024-09-10T08:37:42.670097+0000","last_became_peered":"2024-09-10T08:37:42.670097+0000","last_unstale":"2024-09-10T08:38:43.633131+0000","last_undegraded":"2024-09-10T08:38:43.633131+0000","last_fullsized":"2024-09-10T08:38:43.633131+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:14.956492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628490+0000","last_change":"2024-09-10T08:36:56.154850+0000","last_active":"2024-09-10T08:38:43.628490+0000","last_peered":"2024-09-10T08:38:43.628490+0000","last_clean":"2024-09-10T08:38:43.628490+0000","last_became_active":"2024-09-10T08:36:56.154545+0000","last_became_peered":"2024-09-10T08:36:56.154545+0000","last_unstale":"2024-09-10T08:38:43.628490+0000","last_undegraded":"2024-09-10T08:38:43.628490+0000","last_fullsized":"2024-09-10T08:38:43.628490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:24.639667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628189+0000","last_change":"2024-09-10T08:37:42.669962+0000","last_active":"2024-09-10T08:38:43.628189+0000","last_peered":"2024-09-10T08:38:43.628189+0000","last_clean":"2024-09-10T08:38:43.628189+0000","last_became_active":"2024-09-10T08:37:42.669556+0000","last_became_peered":"2024-09-10T08:37:42.669556+0000","last_unstale":"2024-09-10T08:38:43.628189+0000","last_undegraded":"2024-09-10T08:38:43.628189+0000","last_fullsized":"2024-09-10T08:38:43.628189+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:23.333946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631230+0000","last_change":"2024-09-10T08:36:56.145910+0000","last_active":"2024-09-10T08:38:43.631230+0000","last_peered":"2024-09-10T08:38:43.631230+0000","last_clean":"2024-09-10T08:38:43.631230+0000","last_became_active":"2024-09-10T08:36:56.145537+0000","last_became_peered":"2024-09-10T08:36:56.145537+0000","last_unstale":"2024-09-10T08:38:43.631230+0000","last_undegraded":"2024-09-10T08:38:43.631230+0000","last_fullsized":"2024-09-10T08:38:43.631230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:46.788241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631941+0000","last_change":"2024-09-10T08:36:56.147340+0000","last_active":"2024-09-10T08:38:43.631941+0000","last_peered":"2024-09-10T08:38:43.631941+0000","last_clean":"2024-09-10T08:38:43.631941+0000","last_became_active":"2024-09-10T08:36:56.145762+0000","last_became_peered":"2024-09-10T08:36:56.145762+0000","last_unstale":"2024-09-10T08:38:43.631941+0000","last_undegraded":"2024-09-10T08:38:43.631941+0000","last_fullsized":"2024-09-10T08:38:43.631941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:12.339285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654832+0000","last_change":"2024-09-10T08:36:56.145213+0000","last_active":"2024-09-10T08:38:43.654832+0000","last_peered":"2024-09-10T08:38:43.654832+0000","last_clean":"2024-09-10T08:38:43.654832+0000","last_became_active":"2024-09-10T08:36:56.144915+0000","last_became_peered":"2024-09-10T08:36:56.144915+0000","last_unstale":"2024-09-10T08:38:43.654832+0000","last_undegraded":"2024-09-10T08:38:43.654832+0000","last_fullsized":"2024-09-10T08:38:43.654832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:06.180321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631378+0000","last_change":"2024-09-10T08:36:56.149738+0000","last_active":"2024-09-10T08:38:43.631378+0000","last_peered":"2024-09-10T08:38:43.631378+0000","last_clean":"2024-09-10T08:38:43.631378+0000","last_became_active":"2024-09-10T08:36:56.147750+0000","last_became_peered":"2024-09-10T08:36:56.147750+0000","last_unstale":"2024-09-10T08:38:43.631378+0000","last_undegraded":"2024-09-10T08:38:43.631378+0000","last_fullsized":"2024-09-10T08:38:43.631378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:47.950054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630316+0000","last_change":"2024-09-10T08:36:56.147682+0000","last_active":"2024-09-10T08:38:43.630316+0000","last_peered":"2024-09-10T08:38:43.630316+0000","last_clean":"2024-09-10T08:38:43.630316+0000","last_became_active":"2024-09-10T08:36:56.147345+0000","last_became_peered":"2024-09-10T08:36:56.147345+0000","last_unstale":"2024-09-10T08:38:43.630316+0000","last_undegraded":"2024-09-10T08:38:43.630316+0000","last_fullsized":"2024-09-10T08:38:43.630316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:54.939482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628008+0000","last_change":"2024-09-10T08:36:56.158292+0000","last_active":"2024-09-10T08:38:43.628008+0000","last_peered":"2024-09-10T08:38:43.628008+0000","last_clean":"2024-09-10T08:38:43.628008+0000","last_became_active":"2024-09-10T08:36:56.157970+0000","last_became_peered":"2024-09-10T08:36:56.157970+0000","last_unstale":"2024-09-10T08:38:43.628008+0000","last_undegraded":"2024-09-10T08:38:43.628008+0000","last_fullsized":"2024-09-10T08:38:43.628008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:33.582126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628048+0000","last_change":"2024-09-10T08:36:56.142520+0000","last_active":"2024-09-10T08:38:43.628048+0000","last_peered":"2024-09-10T08:38:43.628048+0000","last_clean":"2024-09-10T08:38:43.628048+0000","last_became_active":"2024-09-10T08:36:56.142214+0000","last_became_peered":"2024-09-10T08:36:56.142214+0000","last_unstale":"2024-09-10T08:38:43.628048+0000","last_undegraded":"2024-09-10T08:38:43.628048+0000","last_fullsized":"2024-09-10T08:38:43.628048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:45:29.201839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630914+0000","last_change":"2024-09-10T08:36:56.149547+0000","last_active":"2024-09-10T08:38:43.630914+0000","last_peered":"2024-09-10T08:38:43.630914+0000","last_clean":"2024-09-10T08:38:43.630914+0000","last_became_active":"2024-09-10T08:36:56.149141+0000","last_became_peered":"2024-09-10T08:36:56.149141+0000","last_unstale":"2024-09-10T08:38:43.630914+0000","last_undegraded":"2024-09-10T08:38:43.630914+0000","last_fullsized":"2024-09-10T08:38:43.630914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:49.368677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629885+0000","last_change":"2024-09-10T08:36:56.143087+0000","last_active":"2024-09-10T08:38:43.629885+0000","last_peered":"2024-09-10T08:38:43.629885+0000","last_clean":"2024-09-10T08:38:43.629885+0000","last_became_active":"2024-09-10T08:36:56.142782+0000","last_became_peered":"2024-09-10T08:36:56.142782+0000","last_unstale":"2024-09-10T08:38:43.629885+0000","last_undegraded":"2024-09-10T08:38:43.629885+0000","last_fullsized":"2024-09-10T08:38:43.629885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:17.513103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628447+0000","last_change":"2024-09-10T08:36:56.157549+0000","last_active":"2024-09-10T08:38:43.628447+0000","last_peered":"2024-09-10T08:38:43.628447+0000","last_clean":"2024-09-10T08:38:43.628447+0000","last_became_active":"2024-09-10T08:36:56.157221+0000","last_became_peered":"2024-09-10T08:36:56.157221+0000","last_unstale":"2024-09-10T08:38:43.628447+0000","last_undegraded":"2024-09-10T08:38:43.628447+0000","last_fullsized":"2024-09-10T08:38:43.628447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:11.372085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629960+0000","last_change":"2024-09-10T08:36:56.145654+0000","last_active":"2024-09-10T08:38:43.629960+0000","last_peered":"2024-09-10T08:38:43.629960+0000","last_clean":"2024-09-10T08:38:43.629960+0000","last_became_active":"2024-09-10T08:36:56.144126+0000","last_became_peered":"2024-09-10T08:36:56.144126+0000","last_unstale":"2024-09-10T08:38:43.629960+0000","last_undegraded":"2024-09-10T08:38:43.629960+0000","last_fullsized":"2024-09-10T08:38:43.629960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:55.498873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.632681+0000","last_change":"2024-09-10T08:36:56.146423+0000","last_active":"2024-09-10T08:38:43.632681+0000","last_peered":"2024-09-10T08:38:43.632681+0000","last_clean":"2024-09-10T08:38:43.632681+0000","last_became_active":"2024-09-10T08:36:56.146126+0000","last_became_peered":"2024-09-10T08:36:56.146126+0000","last_unstale":"2024-09-10T08:38:43.632681+0000","last_undegraded":"2024-09-10T08:38:43.632681+0000","last_fullsized":"2024-09-10T08:38:43.632681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:44.665196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.632922+0000","last_change":"2024-09-10T08:36:56.151973+0000","last_active":"2024-09-10T08:38:43.632922+0000","last_peered":"2024-09-10T08:38:43.632922+0000","last_clean":"2024-09-10T08:38:43.632922+0000","last_became_active":"2024-09-10T08:36:56.151609+0000","last_became_peered":"2024-09-10T08:36:56.151609+0000","last_unstale":"2024-09-10T08:38:43.632922+0000","last_undegraded":"2024-09-10T08:38:43.632922+0000","last_fullsized":"2024-09-10T08:38:43.632922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:02.584047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627915+0000","last_change":"2024-09-10T08:36:56.141780+0000","last_active":"2024-09-10T08:38:43.627915+0000","last_peered":"2024-09-10T08:38:43.627915+0000","last_clean":"2024-09-10T08:38:43.627915+0000","last_became_active":"2024-09-10T08:36:56.141228+0000","last_became_peered":"2024-09-10T08:36:56.141228+0000","last_unstale":"2024-09-10T08:38:43.627915+0000","last_undegraded":"2024-09-10T08:38:43.627915+0000","last_fullsized":"2024-09-10T08:38:43.627915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:52:04.374228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629654+0000","last_change":"2024-09-10T08:37:42.671346+0000","last_active":"2024-09-10T08:38:43.629654+0000","last_peered":"2024-09-10T08:38:43.629654+0000","last_clean":"2024-09-10T08:38:43.629654+0000","last_became_active":"2024-09-10T08:37:42.670895+0000","last_became_peered":"2024-09-10T08:37:42.670895+0000","last_unstale":"2024-09-10T08:38:43.629654+0000","last_undegraded":"2024-09-10T08:38:43.629654+0000","last_fullsized":"2024-09-10T08:38:43.629654+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:16:31.930628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.632053+0000","last_change":"2024-09-10T08:36:56.149014+0000","last_active":"2024-09-10T08:38:43.632053+0000","last_peered":"2024-09-10T08:38:43.632053+0000","last_clean":"2024-09-10T08:38:43.632053+0000","last_became_active":"2024-09-10T08:36:56.148551+0000","last_became_peered":"2024-09-10T08:36:56.148551+0000","last_unstale":"2024-09-10T08:38:43.632053+0000","last_undegraded":"2024-09-10T08:38:43.632053+0000","last_fullsized":"2024-09-10T08:38:43.632053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:35:52.250943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627572+0000","last_change":"2024-09-10T08:36:56.141247+0000","last_active":"2024-09-10T08:38:43.627572+0000","last_peered":"2024-09-10T08:38:43.627572+0000","last_clean":"2024-09-10T08:38:43.627572+0000","last_became_active":"2024-09-10T08:36:56.140956+0000","last_became_peered":"2024-09-10T08:36:56.140956+0000","last_unstale":"2024-09-10T08:38:43.627572+0000","last_undegraded":"2024-09-10T08:38:43.627572+0000","last_fullsized":"2024-09-10T08:38:43.627572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:45.008424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630691+0000","last_change":"2024-09-10T08:36:56.149116+0000","last_active":"2024-09-10T08:38:43.630691+0000","last_peered":"2024-09-10T08:38:43.630691+0000","last_clean":"2024-09-10T08:38:43.630691+0000","last_became_active":"2024-09-10T08:36:56.148783+0000","last_became_peered":"2024-09-10T08:36:56.148783+0000","last_unstale":"2024-09-10T08:38:43.630691+0000","last_undegraded":"2024-09-10T08:38:43.630691+0000","last_fullsized":"2024-09-10T08:38:43.630691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:37.119214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629016+0000","last_change":"2024-09-10T08:37:42.846322+0000","last_active":"2024-09-10T08:38:43.629016+0000","last_peered":"2024-09-10T08:38:43.629016+0000","last_clean":"2024-09-10T08:38:43.629016+0000","last_became_active":"2024-09-10T08:37:42.846022+0000","last_became_peered":"2024-09-10T08:37:42.846022+0000","last_unstale":"2024-09-10T08:38:43.629016+0000","last_undegraded":"2024-09-10T08:38:43.629016+0000","last_fullsized":"2024-09-10T08:38:43.629016+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:38.142362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631593+0000","last_change":"2024-09-10T08:36:56.148537+0000","last_active":"2024-09-10T08:38:43.631593+0000","last_peered":"2024-09-10T08:38:43.631593+0000","last_clean":"2024-09-10T08:38:43.631593+0000","last_became_active":"2024-09-10T08:36:56.148185+0000","last_became_peered":"2024-09-10T08:36:56.148185+0000","last_unstale":"2024-09-10T08:38:43.631593+0000","last_undegraded":"2024-09-10T08:38:43.631593+0000","last_fullsized":"2024-09-10T08:38:43.631593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:00.426513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627054+0000","last_change":"2024-09-10T08:36:56.138513+0000","last_active":"2024-09-10T08:38:43.627054+0000","last_peered":"2024-09-10T08:38:43.627054+0000","last_clean":"2024-09-10T08:38:43.627054+0000","last_became_active":"2024-09-10T08:36:56.138207+0000","last_became_peered":"2024-09-10T08:36:56.138207+0000","last_unstale":"2024-09-10T08:38:43.627054+0000","last_undegraded":"2024-09-10T08:38:43.627054+0000","last_fullsized":"2024-09-10T08:38:43.627054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:34.581430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.625900+0000","last_change":"2024-09-10T08:36:56.146612+0000","last_active":"2024-09-10T08:38:43.625900+0000","last_peered":"2024-09-10T08:38:43.625900+0000","last_clean":"2024-09-10T08:38:43.625900+0000","last_became_active":"2024-09-10T08:36:56.146251+0000","last_became_peered":"2024-09-10T08:36:56.146251+0000","last_unstale":"2024-09-10T08:38:43.625900+0000","last_undegraded":"2024-09-10T08:38:43.625900+0000","last_fullsized":"2024-09-10T08:38:43.625900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:47.336389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655270+0000","last_change":"2024-09-10T08:36:56.148260+0000","last_active":"2024-09-10T08:38:43.655270+0000","last_peered":"2024-09-10T08:38:43.655270+0000","last_clean":"2024-09-10T08:38:43.655270+0000","last_became_active":"2024-09-10T08:36:56.147949+0000","last_became_peered":"2024-09-10T08:36:56.147949+0000","last_unstale":"2024-09-10T08:38:43.655270+0000","last_undegraded":"2024-09-10T08:38:43.655270+0000","last_fullsized":"2024-09-10T08:38:43.655270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:52:19.762297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633344+0000","last_change":"2024-09-10T08:36:56.156045+0000","last_active":"2024-09-10T08:38:43.633344+0000","last_peered":"2024-09-10T08:38:43.633344+0000","last_clean":"2024-09-10T08:38:43.633344+0000","last_became_active":"2024-09-10T08:36:56.154727+0000","last_became_peered":"2024-09-10T08:36:56.154727+0000","last_unstale":"2024-09-10T08:38:43.633344+0000","last_undegraded":"2024-09-10T08:38:43.633344+0000","last_fullsized":"2024-09-10T08:38:43.633344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:35:39.992683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633092+0000","last_change":"2024-09-10T08:36:56.154394+0000","last_active":"2024-09-10T08:38:43.633092+0000","last_peered":"2024-09-10T08:38:43.633092+0000","last_clean":"2024-09-10T08:38:43.633092+0000","last_became_active":"2024-09-10T08:36:56.153855+0000","last_became_peered":"2024-09-10T08:36:56.153855+0000","last_unstale":"2024-09-10T08:38:43.633092+0000","last_undegraded":"2024-09-10T08:38:43.633092+0000","last_fullsized":"2024-09-10T08:38:43.633092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:14.987147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628479+0000","last_change":"2024-09-10T08:36:56.157517+0000","last_active":"2024-09-10T08:38:43.628479+0000","last_peered":"2024-09-10T08:38:43.628479+0000","last_clean":"2024-09-10T08:38:43.628479+0000","last_became_active":"2024-09-10T08:36:56.157133+0000","last_became_peered":"2024-09-10T08:36:56.157133+0000","last_unstale":"2024-09-10T08:38:43.628479+0000","last_undegraded":"2024-09-10T08:38:43.628479+0000","last_fullsized":"2024-09-10T08:38:43.628479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:17:29.573719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67514,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":365,"num_read_kb":253,"num_write":164,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":202542,"data_compressed":5667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282455,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416421,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1056768,"data_stored":547326,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678057,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28212,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716940,"statfs":{"total":95995035648,"available":95966146560,"internally_reserved":0,"allocated":1208320,"data_stored":1206690,"data_compressed":67339,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841583,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1069056,"data_stored":543466,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103221,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1069056,"data_stored":546900,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332154,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1122304,"data_stored":1143393,"data_compressed":65450,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495679,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1052672,"data_stored":542738,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757317,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1130496,"data_stored":1138588,"data_compressed":65450,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1069056,"data_stored":548363,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1064960,"data_stored":542932,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1138688,"data_stored":606927,"data_compressed":31413,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1122304,"data_stored":606741,"data_compressed":31413,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1077248,"data_stored":544126,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64093,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64279,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65482,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:38:51.013 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:38:51.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:50 smithi111 ceph-mon[35220]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s; 145 B/s, 0 objects/s recovering 2024-09-10T08:38:51.377 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:51.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:51 smithi089 ceph-mon[30757]: from='client.24944 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:38:51.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.916+0000 7f99cbe00640 1 -- 172.21.15.89:0/1750860240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99cc074510 msgr2=0x7f99cc0748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:51.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.916+0000 7f99cbe00640 1 --2- 172.21.15.89:0/1750860240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99cc074510 0x7f99cc0748f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009900 tx=0x7f99bc02a480 comp rx=0 tx=0).stop 2024-09-10T08:38:51.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.917+0000 7f99cbe00640 1 -- 172.21.15.89:0/1750860240 shutdown_connections 2024-09-10T08:38:51.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.917+0000 7f99cbe00640 1 --2- 172.21.15.89:0/1750860240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99cc10e280 0x7f99cc110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:51.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.917+0000 7f99cbe00640 1 --2- 172.21.15.89:0/1750860240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99cc074e30 0x7f99cc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:51.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.917+0000 7f99cbe00640 1 --2- 172.21.15.89:0/1750860240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99cc074510 0x7f99cc0748f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:51.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.917+0000 7f99cbe00640 1 -- 172.21.15.89:0/1750860240 >> 172.21.15.89:0/1750860240 conn(0x7f99cc0fd6e0 msgr2=0x7f99cc0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:51.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.917+0000 7f99cbe00640 1 -- 172.21.15.89:0/1750860240 shutdown_connections 2024-09-10T08:38:51.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.917+0000 7f99cbe00640 1 -- 172.21.15.89:0/1750860240 wait complete. 2024-09-10T08:38:51.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.918+0000 7f99cbe00640 1 Processor -- start 2024-09-10T08:38:51.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.918+0000 7f99cbe00640 1 -- start start 2024-09-10T08:38:51.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.919+0000 7f99cbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99cc074510 0x7f99cc19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:51.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.919+0000 7f99cbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99cc074e30 0x7f99cc19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:51.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.919+0000 7f99cbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99cc10e280 0x7f99cc19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:51.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.919+0000 7f99cbe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc113fb0 con 0x7f99cc10e280 2024-09-10T08:38:51.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.919+0000 7f99cbe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc113e30 con 0x7f99cc074e30 2024-09-10T08:38:51.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.919+0000 7f99cbe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc114130 con 0x7f99cc074510 2024-09-10T08:38:51.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.919+0000 7f99caa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99cc074510 0x7f99cc19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:51.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.919+0000 7f99caa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99cc074510 0x7f99cc19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:45010/0 (socket says 172.21.15.89:45010) 2024-09-10T08:38:51.920 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.919+0000 7f99caa00640 1 -- 172.21.15.89:0/17325655 learned_addr learned my addr 172.21.15.89:0/17325655 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:51.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.919+0000 7f99c2000640 1 --2- 172.21.15.89:0/17325655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99cc074e30 0x7f99cc19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:51.921 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.919+0000 7f99cb400640 1 --2- 172.21.15.89:0/17325655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99cc10e280 0x7f99cc19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.919+0000 7f99caa00640 1 -- 172.21.15.89:0/17325655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99cc074e30 msgr2=0x7f99cc19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.919+0000 7f99caa00640 1 --2- 172.21.15.89:0/17325655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99cc074e30 0x7f99cc19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.919+0000 7f99caa00640 1 -- 172.21.15.89:0/17325655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99cc10e280 msgr2=0x7f99cc19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.919+0000 7f99caa00640 1 --2- 172.21.15.89:0/17325655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99cc10e280 0x7f99cc19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.919+0000 7f99caa00640 1 -- 172.21.15.89:0/17325655 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99cc19f460 con 0x7f99cc074510 2024-09-10T08:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.920+0000 7f99caa00640 1 --2- 172.21.15.89:0/17325655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99cc074510 0x7f99cc19a4b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f99bc006010 tx=0x7f99bc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.920+0000 7f99c3e00640 1 -- 172.21.15.89:0/17325655 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc03f030 con 0x7f99cc074510 2024-09-10T08:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.920+0000 7f99c3e00640 1 -- 172.21.15.89:0/17325655 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99bc003ce0 con 0x7f99cc074510 2024-09-10T08:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.920+0000 7f99cbe00640 1 -- 172.21.15.89:0/17325655 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99cc19f6f0 con 0x7f99cc074510 2024-09-10T08:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.920+0000 7f99c3e00640 1 -- 172.21.15.89:0/17325655 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc035070 con 0x7f99cc074510 2024-09-10T08:38:51.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.920+0000 7f99cbe00640 1 -- 172.21.15.89:0/17325655 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99cc1a7030 con 0x7f99cc074510 2024-09-10T08:38:51.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.922+0000 7f99cbe00640 1 -- 172.21.15.89:0/17325655 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99cc0748f0 con 0x7f99cc074510 2024-09-10T08:38:51.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.923+0000 7f99c3e00640 1 -- 172.21.15.89:0/17325655 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f99bc042030 con 0x7f99cc074510 2024-09-10T08:38:51.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.924+0000 7f99c3e00640 1 --2- 172.21.15.89:0/17325655 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f99b00a2c90 0x7f99b00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:51.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.924+0000 7f99c3e00640 1 -- 172.21.15.89:0/17325655 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f99bc0efab0 con 0x7f99cc074510 2024-09-10T08:38:51.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.925+0000 7f99c2000640 1 --2- 172.21.15.89:0/17325655 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f99b00a2c90 0x7f99b00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:51.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.925+0000 7f99c2000640 1 --2- 172.21.15.89:0/17325655 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f99b00a2c90 0x7f99b00a5150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f99cc10f5a0 tx=0x7f99b4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:51.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:51.929+0000 7f99c3e00640 1 -- 172.21.15.89:0/17325655 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99bc0bbef0 con 0x7f99cc074510 2024-09-10T08:38:51.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:51 smithi149 ceph-mon[36167]: from='client.24944 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:38:52.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:51 smithi111 ceph-mon[35220]: from='client.24944 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:38:52.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:52.098+0000 7f99cbe00640 1 -- 172.21.15.89:0/17325655 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99cc062630 con 0x7f99b00a2c90 2024-09-10T08:38:52.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:52.106+0000 7f99c3e00640 1 -- 172.21.15.89:0/17325655 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340689 (secure 0 0 0) 0x7f99cc062630 con 0x7f99b00a2c90 2024-09-10T08:38:52.108 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:38:52.113 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-09-10T08:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:52.111+0000 7f99cbe00640 1 -- 172.21.15.89:0/17325655 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f99b00a2c90 msgr2=0x7f99b00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:52.111+0000 7f99cbe00640 1 --2- 172.21.15.89:0/17325655 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f99b00a2c90 0x7f99b00a5150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f99cc10f5a0 tx=0x7f99b4005f90 comp rx=0 tx=0).stop 2024-09-10T08:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:52.111+0000 7f99cbe00640 1 -- 172.21.15.89:0/17325655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99cc074510 msgr2=0x7f99cc19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:52.111+0000 7f99cbe00640 1 --2- 172.21.15.89:0/17325655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99cc074510 0x7f99cc19a4b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f99bc006010 tx=0x7f99bc0042f0 comp rx=0 tx=0).stop 2024-09-10T08:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:52.111+0000 7f99cbe00640 1 -- 172.21.15.89:0/17325655 shutdown_connections 2024-09-10T08:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:52.111+0000 7f99cbe00640 1 --2- 172.21.15.89:0/17325655 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f99b00a2c90 0x7f99b00a5150 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:52.111+0000 7f99cbe00640 1 --2- 172.21.15.89:0/17325655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99cc10e280 0x7f99cc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:52.111+0000 7f99cbe00640 1 --2- 172.21.15.89:0/17325655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99cc074e30 0x7f99cc19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:52.111+0000 7f99cbe00640 1 --2- 172.21.15.89:0/17325655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99cc074510 0x7f99cc19a4b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:52.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:52.111+0000 7f99cbe00640 1 -- 172.21.15.89:0/17325655 >> 172.21.15.89:0/17325655 conn(0x7f99cc0fd6e0 msgr2=0x7f99cc0fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:52.111+0000 7f99cbe00640 1 -- 172.21.15.89:0/17325655 shutdown_connections 2024-09-10T08:38:52.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:52.111+0000 7f99cbe00640 1 -- 172.21.15.89:0/17325655 wait complete. 2024-09-10T08:38:52.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:52 smithi089 ceph-mon[30757]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 129 B/s, 0 objects/s recovering 2024-09-10T08:38:52.870 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-09-10T08:38:51.598164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658116,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":521,"num_read_kb":377,"num_write":347,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181900,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358028,"kb_used_data":12872,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583796,"statfs":{"total":1151940427776,"available":1151573807104,"internally_reserved":0,"allocated":13180928,"data_stored":8518190,"data_compressed":467733,"data_compressed_allocated":638976,"data_compressed_original":2777088,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":29,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.979373"},"pg_stats":[{"pgid":"2.1f","version":"94'9","reported_seq":66,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.663079+0000","last_change":"2024-09-10T08:38:43.663079+0000","last_active":"2024-09-10T08:38:43.663079+0000","last_peered":"2024-09-10T08:38:43.663079+0000","last_clean":"2024-09-10T08:38:43.663079+0000","last_became_active":"2024-09-10T08:38:43.656506+0000","last_became_peered":"2024-09-10T08:38:43.656506+0000","last_unstale":"2024-09-10T08:38:43.663079+0000","last_undegraded":"2024-09-10T08:38:43.663079+0000","last_fullsized":"2024-09-10T08:38:43.663079+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:36.881564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631528+0000","last_change":"2024-09-10T08:36:56.145816+0000","last_active":"2024-09-10T08:38:43.631528+0000","last_peered":"2024-09-10T08:38:43.631528+0000","last_clean":"2024-09-10T08:38:43.631528+0000","last_became_active":"2024-09-10T08:36:56.145462+0000","last_became_peered":"2024-09-10T08:36:56.145462+0000","last_unstale":"2024-09-10T08:38:43.631528+0000","last_undegraded":"2024-09-10T08:38:43.631528+0000","last_fullsized":"2024-09-10T08:38:43.631528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:53.890190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629816+0000","last_change":"2024-09-10T08:37:05.252761+0000","last_active":"2024-09-10T08:38:43.629816+0000","last_peered":"2024-09-10T08:38:43.629816+0000","last_clean":"2024-09-10T08:38:43.629816+0000","last_became_active":"2024-09-10T08:37:05.252179+0000","last_became_peered":"2024-09-10T08:37:05.252179+0000","last_unstale":"2024-09-10T08:38:43.629816+0000","last_undegraded":"2024-09-10T08:38:43.629816+0000","last_fullsized":"2024-09-10T08:38:43.629816+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:52.854828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631803+0000","last_change":"2024-09-10T08:36:53.735650+0000","last_active":"2024-09-10T08:38:43.631803+0000","last_peered":"2024-09-10T08:38:43.631803+0000","last_clean":"2024-09-10T08:38:43.631803+0000","last_became_active":"2024-09-10T08:36:53.735128+0000","last_became_peered":"2024-09-10T08:36:53.735128+0000","last_unstale":"2024-09-10T08:38:43.631803+0000","last_undegraded":"2024-09-10T08:38:43.631803+0000","last_fullsized":"2024-09-10T08:38:43.631803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:27.309591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633500+0000","last_change":"2024-09-10T08:36:56.153364+0000","last_active":"2024-09-10T08:38:43.633500+0000","last_peered":"2024-09-10T08:38:43.633500+0000","last_clean":"2024-09-10T08:38:43.633500+0000","last_became_active":"2024-09-10T08:36:56.152979+0000","last_became_peered":"2024-09-10T08:36:56.152979+0000","last_unstale":"2024-09-10T08:38:43.633500+0000","last_undegraded":"2024-09-10T08:38:43.633500+0000","last_fullsized":"2024-09-10T08:38:43.633500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:12.172620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654710+0000","last_change":"2024-09-10T08:37:05.254551+0000","last_active":"2024-09-10T08:38:43.654710+0000","last_peered":"2024-09-10T08:38:43.654710+0000","last_clean":"2024-09-10T08:38:43.654710+0000","last_became_active":"2024-09-10T08:37:05.254055+0000","last_became_peered":"2024-09-10T08:37:05.254055+0000","last_unstale":"2024-09-10T08:38:43.654710+0000","last_undegraded":"2024-09-10T08:38:43.654710+0000","last_fullsized":"2024-09-10T08:38:43.654710+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:08.472502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626455+0000","last_change":"2024-09-10T08:36:53.722774+0000","last_active":"2024-09-10T08:38:43.626455+0000","last_peered":"2024-09-10T08:38:43.626455+0000","last_clean":"2024-09-10T08:38:43.626455+0000","last_became_active":"2024-09-10T08:36:53.722459+0000","last_became_peered":"2024-09-10T08:36:53.722459+0000","last_unstale":"2024-09-10T08:38:43.626455+0000","last_undegraded":"2024-09-10T08:38:43.626455+0000","last_fullsized":"2024-09-10T08:38:43.626455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:26.612354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630770+0000","last_change":"2024-09-10T08:36:56.143757+0000","last_active":"2024-09-10T08:38:43.630770+0000","last_peered":"2024-09-10T08:38:43.630770+0000","last_clean":"2024-09-10T08:38:43.630770+0000","last_became_active":"2024-09-10T08:36:56.143430+0000","last_became_peered":"2024-09-10T08:36:56.143430+0000","last_unstale":"2024-09-10T08:38:43.630770+0000","last_undegraded":"2024-09-10T08:38:43.630770+0000","last_fullsized":"2024-09-10T08:38:43.630770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:49.737298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655071+0000","last_change":"2024-09-10T08:37:05.254503+0000","last_active":"2024-09-10T08:38:43.655071+0000","last_peered":"2024-09-10T08:38:43.655071+0000","last_clean":"2024-09-10T08:38:43.655071+0000","last_became_active":"2024-09-10T08:37:05.254070+0000","last_became_peered":"2024-09-10T08:37:05.254070+0000","last_unstale":"2024-09-10T08:38:43.655071+0000","last_undegraded":"2024-09-10T08:38:43.655071+0000","last_fullsized":"2024-09-10T08:38:43.655071+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:16.741698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626406+0000","last_change":"2024-09-10T08:36:53.716742+0000","last_active":"2024-09-10T08:38:43.626406+0000","last_peered":"2024-09-10T08:38:43.626406+0000","last_clean":"2024-09-10T08:38:43.626406+0000","last_became_active":"2024-09-10T08:36:53.716439+0000","last_became_peered":"2024-09-10T08:36:53.716439+0000","last_unstale":"2024-09-10T08:38:43.626406+0000","last_undegraded":"2024-09-10T08:38:43.626406+0000","last_fullsized":"2024-09-10T08:38:43.626406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:46.882805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655230+0000","last_change":"2024-09-10T08:36:56.146841+0000","last_active":"2024-09-10T08:38:43.655230+0000","last_peered":"2024-09-10T08:38:43.655230+0000","last_clean":"2024-09-10T08:38:43.655230+0000","last_became_active":"2024-09-10T08:36:56.146531+0000","last_became_peered":"2024-09-10T08:36:56.146531+0000","last_unstale":"2024-09-10T08:38:43.655230+0000","last_undegraded":"2024-09-10T08:38:43.655230+0000","last_fullsized":"2024-09-10T08:38:43.655230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:37.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654823+0000","last_change":"2024-09-10T08:37:43.223181+0000","last_active":"2024-09-10T08:38:43.654823+0000","last_peered":"2024-09-10T08:38:43.654823+0000","last_clean":"2024-09-10T08:38:43.654823+0000","last_became_active":"2024-09-10T08:37:43.222794+0000","last_became_peered":"2024-09-10T08:37:43.222794+0000","last_unstale":"2024-09-10T08:38:43.654823+0000","last_undegraded":"2024-09-10T08:38:43.654823+0000","last_fullsized":"2024-09-10T08:38:43.654823+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:22.204517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631680+0000","last_change":"2024-09-10T08:36:56.142161+0000","last_active":"2024-09-10T08:38:43.631680+0000","last_peered":"2024-09-10T08:38:43.631680+0000","last_clean":"2024-09-10T08:38:43.631680+0000","last_became_active":"2024-09-10T08:36:56.141838+0000","last_became_peered":"2024-09-10T08:36:56.141838+0000","last_unstale":"2024-09-10T08:38:43.631680+0000","last_undegraded":"2024-09-10T08:38:43.631680+0000","last_fullsized":"2024-09-10T08:38:43.631680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:09.909747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626066+0000","last_change":"2024-09-10T08:36:53.722238+0000","last_active":"2024-09-10T08:38:43.626066+0000","last_peered":"2024-09-10T08:38:43.626066+0000","last_clean":"2024-09-10T08:38:43.626066+0000","last_became_active":"2024-09-10T08:36:53.721859+0000","last_became_peered":"2024-09-10T08:36:53.721859+0000","last_unstale":"2024-09-10T08:38:43.626066+0000","last_undegraded":"2024-09-10T08:38:43.626066+0000","last_fullsized":"2024-09-10T08:38:43.626066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:50.615090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655128+0000","last_change":"2024-09-10T08:37:05.414041+0000","last_active":"2024-09-10T08:38:43.655128+0000","last_peered":"2024-09-10T08:38:43.655128+0000","last_clean":"2024-09-10T08:38:43.655128+0000","last_became_active":"2024-09-10T08:37:05.413286+0000","last_became_peered":"2024-09-10T08:37:05.413286+0000","last_unstale":"2024-09-10T08:38:43.655128+0000","last_undegraded":"2024-09-10T08:38:43.655128+0000","last_fullsized":"2024-09-10T08:38:43.655128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:44.916102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631587+0000","last_change":"2024-09-10T08:36:56.158278+0000","last_active":"2024-09-10T08:38:43.631587+0000","last_peered":"2024-09-10T08:38:43.631587+0000","last_clean":"2024-09-10T08:38:43.631587+0000","last_became_active":"2024-09-10T08:36:56.157946+0000","last_became_peered":"2024-09-10T08:36:56.157946+0000","last_unstale":"2024-09-10T08:38:43.631587+0000","last_undegraded":"2024-09-10T08:38:43.631587+0000","last_fullsized":"2024-09-10T08:38:43.631587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:42.512513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.635498+0000","last_change":"2024-09-10T08:38:43.635077+0000","last_active":"2024-09-10T08:38:43.635498+0000","last_peered":"2024-09-10T08:38:43.635498+0000","last_clean":"2024-09-10T08:38:43.635498+0000","last_became_active":"2024-09-10T08:38:43.631957+0000","last_became_peered":"2024-09-10T08:38:43.631957+0000","last_unstale":"2024-09-10T08:38:43.635498+0000","last_undegraded":"2024-09-10T08:38:43.635498+0000","last_fullsized":"2024-09-10T08:38:43.635498+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:50.837247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655061+0000","last_change":"2024-09-10T08:37:42.671399+0000","last_active":"2024-09-10T08:38:43.655061+0000","last_peered":"2024-09-10T08:38:43.655061+0000","last_clean":"2024-09-10T08:38:43.655061+0000","last_became_active":"2024-09-10T08:37:42.670865+0000","last_became_peered":"2024-09-10T08:37:42.670865+0000","last_unstale":"2024-09-10T08:38:43.655061+0000","last_undegraded":"2024-09-10T08:38:43.655061+0000","last_fullsized":"2024-09-10T08:38:43.655061+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:16.231538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627914+0000","last_change":"2024-09-10T08:36:56.144856+0000","last_active":"2024-09-10T08:38:43.627914+0000","last_peered":"2024-09-10T08:38:43.627914+0000","last_clean":"2024-09-10T08:38:43.627914+0000","last_became_active":"2024-09-10T08:36:56.144531+0000","last_became_peered":"2024-09-10T08:36:56.144531+0000","last_unstale":"2024-09-10T08:38:43.627914+0000","last_undegraded":"2024-09-10T08:38:43.627914+0000","last_fullsized":"2024-09-10T08:38:43.627914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:24.586006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631493+0000","last_change":"2024-09-10T08:36:53.721093+0000","last_active":"2024-09-10T08:38:43.631493+0000","last_peered":"2024-09-10T08:38:43.631493+0000","last_clean":"2024-09-10T08:38:43.631493+0000","last_became_active":"2024-09-10T08:36:53.720538+0000","last_became_peered":"2024-09-10T08:36:53.720538+0000","last_unstale":"2024-09-10T08:38:43.631493+0000","last_undegraded":"2024-09-10T08:38:43.631493+0000","last_fullsized":"2024-09-10T08:38:43.631493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:58.492463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629856+0000","last_change":"2024-09-10T08:37:05.254011+0000","last_active":"2024-09-10T08:38:43.629856+0000","last_peered":"2024-09-10T08:38:43.629856+0000","last_clean":"2024-09-10T08:38:43.629856+0000","last_became_active":"2024-09-10T08:37:05.253066+0000","last_became_peered":"2024-09-10T08:37:05.253066+0000","last_unstale":"2024-09-10T08:38:43.629856+0000","last_undegraded":"2024-09-10T08:38:43.629856+0000","last_fullsized":"2024-09-10T08:38:43.629856+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:16.221302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.625842+0000","last_change":"2024-09-10T08:36:56.141814+0000","last_active":"2024-09-10T08:38:43.625842+0000","last_peered":"2024-09-10T08:38:43.625842+0000","last_clean":"2024-09-10T08:38:43.625842+0000","last_became_active":"2024-09-10T08:36:56.141480+0000","last_became_peered":"2024-09-10T08:36:56.141480+0000","last_unstale":"2024-09-10T08:38:43.625842+0000","last_undegraded":"2024-09-10T08:38:43.625842+0000","last_fullsized":"2024-09-10T08:38:43.625842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:40.594354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.635917+0000","last_change":"2024-09-10T08:38:43.635917+0000","last_active":"2024-09-10T08:38:43.635917+0000","last_peered":"2024-09-10T08:38:43.635917+0000","last_clean":"2024-09-10T08:38:43.635917+0000","last_became_active":"2024-09-10T08:38:43.633406+0000","last_became_peered":"2024-09-10T08:38:43.633406+0000","last_unstale":"2024-09-10T08:38:43.635917+0000","last_undegraded":"2024-09-10T08:38:43.635917+0000","last_fullsized":"2024-09-10T08:38:43.635917+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:44.619072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655247+0000","last_change":"2024-09-10T08:37:05.414181+0000","last_active":"2024-09-10T08:38:43.655247+0000","last_peered":"2024-09-10T08:38:43.655247+0000","last_clean":"2024-09-10T08:38:43.655247+0000","last_became_active":"2024-09-10T08:37:05.413691+0000","last_became_peered":"2024-09-10T08:37:05.413691+0000","last_unstale":"2024-09-10T08:38:43.655247+0000","last_undegraded":"2024-09-10T08:38:43.655247+0000","last_fullsized":"2024-09-10T08:38:43.655247+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:30.744949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633275+0000","last_change":"2024-09-10T08:36:56.153956+0000","last_active":"2024-09-10T08:38:43.633275+0000","last_peered":"2024-09-10T08:38:43.633275+0000","last_clean":"2024-09-10T08:38:43.633275+0000","last_became_active":"2024-09-10T08:36:56.153609+0000","last_became_peered":"2024-09-10T08:36:56.153609+0000","last_unstale":"2024-09-10T08:38:43.633275+0000","last_undegraded":"2024-09-10T08:38:43.633275+0000","last_fullsized":"2024-09-10T08:38:43.633275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:18.185704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655491+0000","last_change":"2024-09-10T08:36:53.733217+0000","last_active":"2024-09-10T08:38:43.655491+0000","last_peered":"2024-09-10T08:38:43.655491+0000","last_clean":"2024-09-10T08:38:43.655491+0000","last_became_active":"2024-09-10T08:36:53.732851+0000","last_became_peered":"2024-09-10T08:36:53.732851+0000","last_unstale":"2024-09-10T08:38:43.655491+0000","last_undegraded":"2024-09-10T08:38:43.655491+0000","last_fullsized":"2024-09-10T08:38:43.655491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:04:01.132196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629833+0000","last_change":"2024-09-10T08:37:05.241868+0000","last_active":"2024-09-10T08:38:43.629833+0000","last_peered":"2024-09-10T08:38:43.629833+0000","last_clean":"2024-09-10T08:38:43.629833+0000","last_became_active":"2024-09-10T08:37:05.241412+0000","last_became_peered":"2024-09-10T08:37:05.241412+0000","last_unstale":"2024-09-10T08:38:43.629833+0000","last_undegraded":"2024-09-10T08:38:43.629833+0000","last_fullsized":"2024-09-10T08:38:43.629833+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:52.543353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631931+0000","last_change":"2024-09-10T08:36:56.150722+0000","last_active":"2024-09-10T08:38:43.631931+0000","last_peered":"2024-09-10T08:38:43.631931+0000","last_clean":"2024-09-10T08:38:43.631931+0000","last_became_active":"2024-09-10T08:36:56.150423+0000","last_became_peered":"2024-09-10T08:36:56.150423+0000","last_unstale":"2024-09-10T08:38:43.631931+0000","last_undegraded":"2024-09-10T08:38:43.631931+0000","last_fullsized":"2024-09-10T08:38:43.631931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:50.272841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'10","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629985+0000","last_change":"2024-09-10T08:36:53.722477+0000","last_active":"2024-09-10T08:38:43.629985+0000","last_peered":"2024-09-10T08:38:43.629985+0000","last_clean":"2024-09-10T08:38:43.629985+0000","last_became_active":"2024-09-10T08:36:53.722118+0000","last_became_peered":"2024-09-10T08:36:53.722118+0000","last_unstale":"2024-09-10T08:38:43.629985+0000","last_undegraded":"2024-09-10T08:38:43.629985+0000","last_fullsized":"2024-09-10T08:38:43.629985+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:06.631646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630935+0000","last_change":"2024-09-10T08:37:05.415656+0000","last_active":"2024-09-10T08:38:43.630935+0000","last_peered":"2024-09-10T08:38:43.630935+0000","last_clean":"2024-09-10T08:38:43.630935+0000","last_became_active":"2024-09-10T08:37:05.413537+0000","last_became_peered":"2024-09-10T08:37:05.413537+0000","last_unstale":"2024-09-10T08:38:43.630935+0000","last_undegraded":"2024-09-10T08:38:43.630935+0000","last_fullsized":"2024-09-10T08:38:43.630935+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:04.284410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628492+0000","last_change":"2024-09-10T08:36:56.144157+0000","last_active":"2024-09-10T08:38:43.628492+0000","last_peered":"2024-09-10T08:38:43.628492+0000","last_clean":"2024-09-10T08:38:43.628492+0000","last_became_active":"2024-09-10T08:36:56.140332+0000","last_became_peered":"2024-09-10T08:36:56.140332+0000","last_unstale":"2024-09-10T08:38:43.628492+0000","last_undegraded":"2024-09-10T08:38:43.628492+0000","last_fullsized":"2024-09-10T08:38:43.628492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:09:26.097987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626457+0000","last_change":"2024-09-10T08:36:53.735465+0000","last_active":"2024-09-10T08:38:43.626457+0000","last_peered":"2024-09-10T08:38:43.626457+0000","last_clean":"2024-09-10T08:38:43.626457+0000","last_became_active":"2024-09-10T08:36:53.735160+0000","last_became_peered":"2024-09-10T08:36:53.735160+0000","last_unstale":"2024-09-10T08:38:43.626457+0000","last_undegraded":"2024-09-10T08:38:43.626457+0000","last_fullsized":"2024-09-10T08:38:43.626457+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:26:52.332085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627895+0000","last_change":"2024-09-10T08:37:05.247671+0000","last_active":"2024-09-10T08:38:43.627895+0000","last_peered":"2024-09-10T08:38:43.627895+0000","last_clean":"2024-09-10T08:38:43.627895+0000","last_became_active":"2024-09-10T08:37:05.247217+0000","last_became_peered":"2024-09-10T08:37:05.247217+0000","last_unstale":"2024-09-10T08:38:43.627895+0000","last_undegraded":"2024-09-10T08:38:43.627895+0000","last_fullsized":"2024-09-10T08:38:43.627895+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:20.881304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627370+0000","last_change":"2024-09-10T08:36:56.145420+0000","last_active":"2024-09-10T08:38:43.627370+0000","last_peered":"2024-09-10T08:38:43.627370+0000","last_clean":"2024-09-10T08:38:43.627370+0000","last_became_active":"2024-09-10T08:36:56.145074+0000","last_became_peered":"2024-09-10T08:36:56.145074+0000","last_unstale":"2024-09-10T08:38:43.627370+0000","last_undegraded":"2024-09-10T08:38:43.627370+0000","last_fullsized":"2024-09-10T08:38:43.627370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:13.849977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629766+0000","last_change":"2024-09-10T08:36:53.738237+0000","last_active":"2024-09-10T08:38:43.629766+0000","last_peered":"2024-09-10T08:38:43.629766+0000","last_clean":"2024-09-10T08:38:43.629766+0000","last_became_active":"2024-09-10T08:36:53.737858+0000","last_became_peered":"2024-09-10T08:36:53.737858+0000","last_unstale":"2024-09-10T08:38:43.629766+0000","last_undegraded":"2024-09-10T08:38:43.629766+0000","last_fullsized":"2024-09-10T08:38:43.629766+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:35.345661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.625917+0000","last_change":"2024-09-10T08:37:05.413512+0000","last_active":"2024-09-10T08:38:43.625917+0000","last_peered":"2024-09-10T08:38:43.625917+0000","last_clean":"2024-09-10T08:38:43.625917+0000","last_became_active":"2024-09-10T08:37:05.413010+0000","last_became_peered":"2024-09-10T08:37:05.413010+0000","last_unstale":"2024-09-10T08:38:43.625917+0000","last_undegraded":"2024-09-10T08:38:43.625917+0000","last_fullsized":"2024-09-10T08:38:43.625917+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:05.349404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627152+0000","last_change":"2024-09-10T08:36:56.157741+0000","last_active":"2024-09-10T08:38:43.627152+0000","last_peered":"2024-09-10T08:38:43.627152+0000","last_clean":"2024-09-10T08:38:43.627152+0000","last_became_active":"2024-09-10T08:36:56.157295+0000","last_became_peered":"2024-09-10T08:36:56.157295+0000","last_unstale":"2024-09-10T08:38:43.627152+0000","last_undegraded":"2024-09-10T08:38:43.627152+0000","last_fullsized":"2024-09-10T08:38:43.627152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:31.845864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631783+0000","last_change":"2024-09-10T08:36:53.722373+0000","last_active":"2024-09-10T08:38:43.631783+0000","last_peered":"2024-09-10T08:38:43.631783+0000","last_clean":"2024-09-10T08:38:43.631783+0000","last_became_active":"2024-09-10T08:36:53.721975+0000","last_became_peered":"2024-09-10T08:36:53.721975+0000","last_unstale":"2024-09-10T08:38:43.631783+0000","last_undegraded":"2024-09-10T08:38:43.631783+0000","last_fullsized":"2024-09-10T08:38:43.631783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:44.593018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628307+0000","last_change":"2024-09-10T08:37:05.255362+0000","last_active":"2024-09-10T08:38:43.628307+0000","last_peered":"2024-09-10T08:38:43.628307+0000","last_clean":"2024-09-10T08:38:43.628307+0000","last_became_active":"2024-09-10T08:37:05.254967+0000","last_became_peered":"2024-09-10T08:37:05.254967+0000","last_unstale":"2024-09-10T08:38:43.628307+0000","last_undegraded":"2024-09-10T08:38:43.628307+0000","last_fullsized":"2024-09-10T08:38:43.628307+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:31:41.850961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654865+0000","last_change":"2024-09-10T08:36:56.147699+0000","last_active":"2024-09-10T08:38:43.654865+0000","last_peered":"2024-09-10T08:38:43.654865+0000","last_clean":"2024-09-10T08:38:43.654865+0000","last_became_active":"2024-09-10T08:36:56.147349+0000","last_became_peered":"2024-09-10T08:36:56.147349+0000","last_unstale":"2024-09-10T08:38:43.654865+0000","last_undegraded":"2024-09-10T08:38:43.654865+0000","last_fullsized":"2024-09-10T08:38:43.654865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:25.248588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'4","reported_seq":106,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:48.808493+0000","last_change":"2024-09-10T08:36:53.736843+0000","last_active":"2024-09-10T08:38:48.808493+0000","last_peered":"2024-09-10T08:38:48.808493+0000","last_clean":"2024-09-10T08:38:48.808493+0000","last_became_active":"2024-09-10T08:36:53.736340+0000","last_became_peered":"2024-09-10T08:36:53.736340+0000","last_unstale":"2024-09-10T08:38:48.808493+0000","last_undegraded":"2024-09-10T08:38:48.808493+0000","last_fullsized":"2024-09-10T08:38:48.808493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:24.679679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":62,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628492+0000","last_change":"2024-09-10T08:37:05.414321+0000","last_active":"2024-09-10T08:38:43.628492+0000","last_peered":"2024-09-10T08:38:43.628492+0000","last_clean":"2024-09-10T08:38:43.628492+0000","last_became_active":"2024-09-10T08:37:05.413868+0000","last_became_peered":"2024-09-10T08:37:05.413868+0000","last_unstale":"2024-09-10T08:38:43.628492+0000","last_undegraded":"2024-09-10T08:38:43.628492+0000","last_fullsized":"2024-09-10T08:38:43.628492+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:53.988077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627830+0000","last_change":"2024-09-10T08:36:56.141910+0000","last_active":"2024-09-10T08:38:43.627830+0000","last_peered":"2024-09-10T08:38:43.627830+0000","last_clean":"2024-09-10T08:38:43.627830+0000","last_became_active":"2024-09-10T08:36:56.141579+0000","last_became_peered":"2024-09-10T08:36:56.141579+0000","last_unstale":"2024-09-10T08:38:43.627830+0000","last_undegraded":"2024-09-10T08:38:43.627830+0000","last_fullsized":"2024-09-10T08:38:43.627830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:18.464831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631695+0000","last_change":"2024-09-10T08:36:53.720102+0000","last_active":"2024-09-10T08:38:43.631695+0000","last_peered":"2024-09-10T08:38:43.631695+0000","last_clean":"2024-09-10T08:38:43.631695+0000","last_became_active":"2024-09-10T08:36:53.719764+0000","last_became_peered":"2024-09-10T08:36:53.719764+0000","last_unstale":"2024-09-10T08:38:43.631695+0000","last_undegraded":"2024-09-10T08:38:43.631695+0000","last_fullsized":"2024-09-10T08:38:43.631695+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:03.609338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655157+0000","last_change":"2024-09-10T08:37:05.413818+0000","last_active":"2024-09-10T08:38:43.655157+0000","last_peered":"2024-09-10T08:38:43.655157+0000","last_clean":"2024-09-10T08:38:43.655157+0000","last_became_active":"2024-09-10T08:37:05.413267+0000","last_became_peered":"2024-09-10T08:37:05.413267+0000","last_unstale":"2024-09-10T08:38:43.655157+0000","last_undegraded":"2024-09-10T08:38:43.655157+0000","last_fullsized":"2024-09-10T08:38:43.655157+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:54.309686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631369+0000","last_change":"2024-09-10T08:36:56.149286+0000","last_active":"2024-09-10T08:38:43.631369+0000","last_peered":"2024-09-10T08:38:43.631369+0000","last_clean":"2024-09-10T08:38:43.631369+0000","last_became_active":"2024-09-10T08:36:56.146275+0000","last_became_peered":"2024-09-10T08:36:56.146275+0000","last_unstale":"2024-09-10T08:38:43.631369+0000","last_undegraded":"2024-09-10T08:38:43.631369+0000","last_fullsized":"2024-09-10T08:38:43.631369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:19.468224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655198+0000","last_change":"2024-09-10T08:36:53.719756+0000","last_active":"2024-09-10T08:38:43.655198+0000","last_peered":"2024-09-10T08:38:43.655198+0000","last_clean":"2024-09-10T08:38:43.655198+0000","last_became_active":"2024-09-10T08:36:53.719302+0000","last_became_peered":"2024-09-10T08:36:53.719302+0000","last_unstale":"2024-09-10T08:38:43.655198+0000","last_undegraded":"2024-09-10T08:38:43.655198+0000","last_fullsized":"2024-09-10T08:38:43.655198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:47.658081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631788+0000","last_change":"2024-09-10T08:35:39.150715+0000","last_active":"2024-09-10T08:38:43.631788+0000","last_peered":"2024-09-10T08:38:43.631788+0000","last_clean":"2024-09-10T08:38:43.631788+0000","last_became_active":"2024-09-10T08:35:39.150206+0000","last_became_peered":"2024-09-10T08:35:39.150206+0000","last_unstale":"2024-09-10T08:38:43.631788+0000","last_undegraded":"2024-09-10T08:38:43.631788+0000","last_fullsized":"2024-09-10T08:38:43.631788+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:32:02.110870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:32:02.110870+0000","last_clean_scrub_stamp":"2024-09-10T08:32:02.110870+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:20.762043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631951+0000","last_change":"2024-09-10T08:37:42.846547+0000","last_active":"2024-09-10T08:38:43.631951+0000","last_peered":"2024-09-10T08:38:43.631951+0000","last_clean":"2024-09-10T08:38:43.631951+0000","last_became_active":"2024-09-10T08:37:42.845884+0000","last_became_peered":"2024-09-10T08:37:42.845884+0000","last_unstale":"2024-09-10T08:38:43.631951+0000","last_undegraded":"2024-09-10T08:38:43.631951+0000","last_fullsized":"2024-09-10T08:38:43.631951+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:21.905133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.632062+0000","last_change":"2024-09-10T08:36:56.144187+0000","last_active":"2024-09-10T08:38:43.632062+0000","last_peered":"2024-09-10T08:38:43.632062+0000","last_clean":"2024-09-10T08:38:43.632062+0000","last_became_active":"2024-09-10T08:36:56.143193+0000","last_became_peered":"2024-09-10T08:36:56.143193+0000","last_unstale":"2024-09-10T08:38:43.632062+0000","last_undegraded":"2024-09-10T08:38:43.632062+0000","last_fullsized":"2024-09-10T08:38:43.632062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:14:26.981884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.639660+0000","last_change":"2024-09-10T08:38:43.639286+0000","last_active":"2024-09-10T08:38:43.639660+0000","last_peered":"2024-09-10T08:38:43.639660+0000","last_clean":"2024-09-10T08:38:43.639660+0000","last_became_active":"2024-09-10T08:38:43.633552+0000","last_became_peered":"2024-09-10T08:38:43.633552+0000","last_unstale":"2024-09-10T08:38:43.639660+0000","last_undegraded":"2024-09-10T08:38:43.639660+0000","last_fullsized":"2024-09-10T08:38:43.639660+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:23.985190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630150+0000","last_change":"2024-09-10T08:37:05.242095+0000","last_active":"2024-09-10T08:38:43.630150+0000","last_peered":"2024-09-10T08:38:43.630150+0000","last_clean":"2024-09-10T08:38:43.630150+0000","last_became_active":"2024-09-10T08:37:05.241645+0000","last_became_peered":"2024-09-10T08:37:05.241645+0000","last_unstale":"2024-09-10T08:38:43.630150+0000","last_undegraded":"2024-09-10T08:38:43.630150+0000","last_fullsized":"2024-09-10T08:38:43.630150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:30:49.637861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631813+0000","last_change":"2024-09-10T08:36:56.149396+0000","last_active":"2024-09-10T08:38:43.631813+0000","last_peered":"2024-09-10T08:38:43.631813+0000","last_clean":"2024-09-10T08:38:43.631813+0000","last_became_active":"2024-09-10T08:36:56.149079+0000","last_became_peered":"2024-09-10T08:36:56.149079+0000","last_unstale":"2024-09-10T08:38:43.631813+0000","last_undegraded":"2024-09-10T08:38:43.631813+0000","last_fullsized":"2024-09-10T08:38:43.631813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:44.665448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655423+0000","last_change":"2024-09-10T08:36:53.717950+0000","last_active":"2024-09-10T08:38:43.655423+0000","last_peered":"2024-09-10T08:38:43.655423+0000","last_clean":"2024-09-10T08:38:43.655423+0000","last_became_active":"2024-09-10T08:36:53.717668+0000","last_became_peered":"2024-09-10T08:36:53.717668+0000","last_unstale":"2024-09-10T08:38:43.655423+0000","last_undegraded":"2024-09-10T08:38:43.655423+0000","last_fullsized":"2024-09-10T08:38:43.655423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:58.692631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633095+0000","last_change":"2024-09-10T08:37:05.260178+0000","last_active":"2024-09-10T08:38:43.633095+0000","last_peered":"2024-09-10T08:38:43.633095+0000","last_clean":"2024-09-10T08:38:43.633095+0000","last_became_active":"2024-09-10T08:37:05.259753+0000","last_became_peered":"2024-09-10T08:37:05.259753+0000","last_unstale":"2024-09-10T08:38:43.633095+0000","last_undegraded":"2024-09-10T08:38:43.633095+0000","last_fullsized":"2024-09-10T08:38:43.633095+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:19.364782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.632330+0000","last_change":"2024-09-10T08:36:56.147104+0000","last_active":"2024-09-10T08:38:43.632330+0000","last_peered":"2024-09-10T08:38:43.632330+0000","last_clean":"2024-09-10T08:38:43.632330+0000","last_became_active":"2024-09-10T08:36:56.146778+0000","last_became_peered":"2024-09-10T08:36:56.146778+0000","last_unstale":"2024-09-10T08:38:43.632330+0000","last_undegraded":"2024-09-10T08:38:43.632330+0000","last_fullsized":"2024-09-10T08:38:43.632330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:56.092157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628351+0000","last_change":"2024-09-10T08:36:53.719145+0000","last_active":"2024-09-10T08:38:43.628351+0000","last_peered":"2024-09-10T08:38:43.628351+0000","last_clean":"2024-09-10T08:38:43.628351+0000","last_became_active":"2024-09-10T08:36:53.718771+0000","last_became_peered":"2024-09-10T08:36:53.718771+0000","last_unstale":"2024-09-10T08:38:43.628351+0000","last_undegraded":"2024-09-10T08:38:43.628351+0000","last_fullsized":"2024-09-10T08:38:43.628351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:21.109940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654677+0000","last_change":"2024-09-10T08:37:05.242854+0000","last_active":"2024-09-10T08:38:43.654677+0000","last_peered":"2024-09-10T08:38:43.654677+0000","last_clean":"2024-09-10T08:38:43.654677+0000","last_became_active":"2024-09-10T08:37:05.242442+0000","last_became_peered":"2024-09-10T08:37:05.242442+0000","last_unstale":"2024-09-10T08:38:43.654677+0000","last_undegraded":"2024-09-10T08:38:43.654677+0000","last_fullsized":"2024-09-10T08:38:43.654677+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:36.072131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628176+0000","last_change":"2024-09-10T08:37:42.845642+0000","last_active":"2024-09-10T08:38:43.628176+0000","last_peered":"2024-09-10T08:38:43.628176+0000","last_clean":"2024-09-10T08:38:43.628176+0000","last_became_active":"2024-09-10T08:37:42.845217+0000","last_became_peered":"2024-09-10T08:37:42.845217+0000","last_unstale":"2024-09-10T08:38:43.628176+0000","last_undegraded":"2024-09-10T08:38:43.628176+0000","last_fullsized":"2024-09-10T08:38:43.628176+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:28.020337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626705+0000","last_change":"2024-09-10T08:36:53.720463+0000","last_active":"2024-09-10T08:38:43.626705+0000","last_peered":"2024-09-10T08:38:43.626705+0000","last_clean":"2024-09-10T08:38:43.626705+0000","last_became_active":"2024-09-10T08:36:53.720080+0000","last_became_peered":"2024-09-10T08:36:53.720080+0000","last_unstale":"2024-09-10T08:38:43.626705+0000","last_undegraded":"2024-09-10T08:38:43.626705+0000","last_fullsized":"2024-09-10T08:38:43.626705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:51.450294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654831+0000","last_change":"2024-09-10T08:37:05.251261+0000","last_active":"2024-09-10T08:38:43.654831+0000","last_peered":"2024-09-10T08:38:43.654831+0000","last_clean":"2024-09-10T08:38:43.654831+0000","last_became_active":"2024-09-10T08:37:05.250938+0000","last_became_peered":"2024-09-10T08:37:05.250938+0000","last_unstale":"2024-09-10T08:38:43.654831+0000","last_undegraded":"2024-09-10T08:38:43.654831+0000","last_fullsized":"2024-09-10T08:38:43.654831+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:07:15.129899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654959+0000","last_change":"2024-09-10T08:36:56.148191+0000","last_active":"2024-09-10T08:38:43.654959+0000","last_peered":"2024-09-10T08:38:43.654959+0000","last_clean":"2024-09-10T08:38:43.654959+0000","last_became_active":"2024-09-10T08:36:56.147362+0000","last_became_peered":"2024-09-10T08:36:56.147362+0000","last_unstale":"2024-09-10T08:38:43.654959+0000","last_undegraded":"2024-09-10T08:38:43.654959+0000","last_fullsized":"2024-09-10T08:38:43.654959+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:25:30.051754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":56,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.625844+0000","last_change":"2024-09-10T08:36:53.734484+0000","last_active":"2024-09-10T08:38:43.625844+0000","last_peered":"2024-09-10T08:38:43.625844+0000","last_clean":"2024-09-10T08:38:43.625844+0000","last_became_active":"2024-09-10T08:36:53.734172+0000","last_became_peered":"2024-09-10T08:36:53.734172+0000","last_unstale":"2024-09-10T08:38:43.625844+0000","last_undegraded":"2024-09-10T08:38:43.625844+0000","last_fullsized":"2024-09-10T08:38:43.625844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:10.269400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628251+0000","last_change":"2024-09-10T08:37:05.259635+0000","last_active":"2024-09-10T08:38:43.628251+0000","last_peered":"2024-09-10T08:38:43.628251+0000","last_clean":"2024-09-10T08:38:43.628251+0000","last_became_active":"2024-09-10T08:37:05.259107+0000","last_became_peered":"2024-09-10T08:37:05.259107+0000","last_unstale":"2024-09-10T08:38:43.628251+0000","last_undegraded":"2024-09-10T08:38:43.628251+0000","last_fullsized":"2024-09-10T08:38:43.628251+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:36.841138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631420+0000","last_change":"2024-09-10T08:36:56.141122+0000","last_active":"2024-09-10T08:38:43.631420+0000","last_peered":"2024-09-10T08:38:43.631420+0000","last_clean":"2024-09-10T08:38:43.631420+0000","last_became_active":"2024-09-10T08:36:56.140826+0000","last_became_peered":"2024-09-10T08:36:56.140826+0000","last_unstale":"2024-09-10T08:38:43.631420+0000","last_undegraded":"2024-09-10T08:38:43.631420+0000","last_fullsized":"2024-09-10T08:38:43.631420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:38.044090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'4","reported_seq":109,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:48.809301+0000","last_change":"2024-09-10T08:36:53.720623+0000","last_active":"2024-09-10T08:38:48.809301+0000","last_peered":"2024-09-10T08:38:48.809301+0000","last_clean":"2024-09-10T08:38:48.809301+0000","last_became_active":"2024-09-10T08:36:53.720270+0000","last_became_peered":"2024-09-10T08:36:53.720270+0000","last_unstale":"2024-09-10T08:38:48.809301+0000","last_undegraded":"2024-09-10T08:38:48.809301+0000","last_fullsized":"2024-09-10T08:38:48.809301+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:19.514068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629891+0000","last_change":"2024-09-10T08:37:05.254324+0000","last_active":"2024-09-10T08:38:43.629891+0000","last_peered":"2024-09-10T08:38:43.629891+0000","last_clean":"2024-09-10T08:38:43.629891+0000","last_became_active":"2024-09-10T08:37:05.253611+0000","last_became_peered":"2024-09-10T08:37:05.253611+0000","last_unstale":"2024-09-10T08:38:43.629891+0000","last_undegraded":"2024-09-10T08:38:43.629891+0000","last_fullsized":"2024-09-10T08:38:43.629891+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:07.182649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.632053+0000","last_change":"2024-09-10T08:36:56.139102+0000","last_active":"2024-09-10T08:38:43.632053+0000","last_peered":"2024-09-10T08:38:43.632053+0000","last_clean":"2024-09-10T08:38:43.632053+0000","last_became_active":"2024-09-10T08:36:56.137986+0000","last_became_peered":"2024-09-10T08:36:56.137986+0000","last_unstale":"2024-09-10T08:38:43.632053+0000","last_undegraded":"2024-09-10T08:38:43.632053+0000","last_fullsized":"2024-09-10T08:38:43.632053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:05:03.005727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654954+0000","last_change":"2024-09-10T08:36:53.722961+0000","last_active":"2024-09-10T08:38:43.654954+0000","last_peered":"2024-09-10T08:38:43.654954+0000","last_clean":"2024-09-10T08:38:43.654954+0000","last_became_active":"2024-09-10T08:36:53.722628+0000","last_became_peered":"2024-09-10T08:36:53.722628+0000","last_unstale":"2024-09-10T08:38:43.654954+0000","last_undegraded":"2024-09-10T08:38:43.654954+0000","last_fullsized":"2024-09-10T08:38:43.654954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:59.048632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627916+0000","last_change":"2024-09-10T08:37:05.256546+0000","last_active":"2024-09-10T08:38:43.627916+0000","last_peered":"2024-09-10T08:38:43.627916+0000","last_clean":"2024-09-10T08:38:43.627916+0000","last_became_active":"2024-09-10T08:37:05.256239+0000","last_became_peered":"2024-09-10T08:37:05.256239+0000","last_unstale":"2024-09-10T08:38:43.627916+0000","last_undegraded":"2024-09-10T08:38:43.627916+0000","last_fullsized":"2024-09-10T08:38:43.627916+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:57.403934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627921+0000","last_change":"2024-09-10T08:36:56.149285+0000","last_active":"2024-09-10T08:38:43.627921+0000","last_peered":"2024-09-10T08:38:43.627921+0000","last_clean":"2024-09-10T08:38:43.627921+0000","last_became_active":"2024-09-10T08:36:56.146873+0000","last_became_peered":"2024-09-10T08:36:56.146873+0000","last_unstale":"2024-09-10T08:38:43.627921+0000","last_undegraded":"2024-09-10T08:38:43.627921+0000","last_fullsized":"2024-09-10T08:38:43.627921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:28.976729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630229+0000","last_change":"2024-09-10T08:36:53.721174+0000","last_active":"2024-09-10T08:38:43.630229+0000","last_peered":"2024-09-10T08:38:43.630229+0000","last_clean":"2024-09-10T08:38:43.630229+0000","last_became_active":"2024-09-10T08:36:53.720849+0000","last_became_peered":"2024-09-10T08:36:53.720849+0000","last_unstale":"2024-09-10T08:38:43.630229+0000","last_undegraded":"2024-09-10T08:38:43.630229+0000","last_fullsized":"2024-09-10T08:38:43.630229+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:39.786848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655227+0000","last_change":"2024-09-10T08:37:05.252256+0000","last_active":"2024-09-10T08:38:43.655227+0000","last_peered":"2024-09-10T08:38:43.655227+0000","last_clean":"2024-09-10T08:38:43.655227+0000","last_became_active":"2024-09-10T08:37:05.251865+0000","last_became_peered":"2024-09-10T08:37:05.251865+0000","last_unstale":"2024-09-10T08:38:43.655227+0000","last_undegraded":"2024-09-10T08:38:43.655227+0000","last_fullsized":"2024-09-10T08:38:43.655227+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:08.581880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626344+0000","last_change":"2024-09-10T08:36:56.142060+0000","last_active":"2024-09-10T08:38:43.626344+0000","last_peered":"2024-09-10T08:38:43.626344+0000","last_clean":"2024-09-10T08:38:43.626344+0000","last_became_active":"2024-09-10T08:36:56.141451+0000","last_became_peered":"2024-09-10T08:36:56.141451+0000","last_unstale":"2024-09-10T08:38:43.626344+0000","last_undegraded":"2024-09-10T08:38:43.626344+0000","last_fullsized":"2024-09-10T08:38:43.626344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:43.566588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'32","reported_seq":202,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:48.809142+0000","last_change":"2024-09-10T08:36:53.719464+0000","last_active":"2024-09-10T08:38:48.809142+0000","last_peered":"2024-09-10T08:38:48.809142+0000","last_clean":"2024-09-10T08:38:48.809142+0000","last_became_active":"2024-09-10T08:36:53.719129+0000","last_became_peered":"2024-09-10T08:36:53.719129+0000","last_unstale":"2024-09-10T08:38:48.809142+0000","last_undegraded":"2024-09-10T08:38:48.809142+0000","last_fullsized":"2024-09-10T08:38:48.809142+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:05.652018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63511,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":130,"num_read_kb":127,"num_write":33,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629358+0000","last_change":"2024-09-10T08:37:05.255576+0000","last_active":"2024-09-10T08:38:43.629358+0000","last_peered":"2024-09-10T08:38:43.629358+0000","last_clean":"2024-09-10T08:38:43.629358+0000","last_became_active":"2024-09-10T08:37:05.249842+0000","last_became_peered":"2024-09-10T08:37:05.249842+0000","last_unstale":"2024-09-10T08:38:43.629358+0000","last_undegraded":"2024-09-10T08:38:43.629358+0000","last_fullsized":"2024-09-10T08:38:43.629358+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:47:49.543481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630860+0000","last_change":"2024-09-10T08:36:56.151829+0000","last_active":"2024-09-10T08:38:43.630860+0000","last_peered":"2024-09-10T08:38:43.630860+0000","last_clean":"2024-09-10T08:38:43.630860+0000","last_became_active":"2024-09-10T08:36:56.151314+0000","last_became_peered":"2024-09-10T08:36:56.151314+0000","last_unstale":"2024-09-10T08:38:43.630860+0000","last_undegraded":"2024-09-10T08:38:43.630860+0000","last_fullsized":"2024-09-10T08:38:43.630860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:44.219671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655360+0000","last_change":"2024-09-10T08:36:53.738030+0000","last_active":"2024-09-10T08:38:43.655360+0000","last_peered":"2024-09-10T08:38:43.655360+0000","last_clean":"2024-09-10T08:38:43.655360+0000","last_became_active":"2024-09-10T08:36:53.737658+0000","last_became_peered":"2024-09-10T08:36:53.737658+0000","last_unstale":"2024-09-10T08:38:43.655360+0000","last_undegraded":"2024-09-10T08:38:43.655360+0000","last_fullsized":"2024-09-10T08:38:43.655360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:29.627554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654959+0000","last_change":"2024-09-10T08:37:05.244473+0000","last_active":"2024-09-10T08:38:43.654959+0000","last_peered":"2024-09-10T08:38:43.654959+0000","last_clean":"2024-09-10T08:38:43.654959+0000","last_became_active":"2024-09-10T08:37:05.244035+0000","last_became_peered":"2024-09-10T08:37:05.244035+0000","last_unstale":"2024-09-10T08:38:43.654959+0000","last_undegraded":"2024-09-10T08:38:43.654959+0000","last_fullsized":"2024-09-10T08:38:43.654959+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:04.652660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633507+0000","last_change":"2024-09-10T08:36:56.155217+0000","last_active":"2024-09-10T08:38:43.633507+0000","last_peered":"2024-09-10T08:38:43.633507+0000","last_clean":"2024-09-10T08:38:43.633507+0000","last_became_active":"2024-09-10T08:36:56.154903+0000","last_became_peered":"2024-09-10T08:36:56.154903+0000","last_unstale":"2024-09-10T08:38:43.633507+0000","last_undegraded":"2024-09-10T08:38:43.633507+0000","last_fullsized":"2024-09-10T08:38:43.633507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:47.482974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.632193+0000","last_change":"2024-09-10T08:36:53.721087+0000","last_active":"2024-09-10T08:38:43.632193+0000","last_peered":"2024-09-10T08:38:43.632193+0000","last_clean":"2024-09-10T08:38:43.632193+0000","last_became_active":"2024-09-10T08:36:53.720772+0000","last_became_peered":"2024-09-10T08:36:53.720772+0000","last_unstale":"2024-09-10T08:38:43.632193+0000","last_undegraded":"2024-09-10T08:38:43.632193+0000","last_fullsized":"2024-09-10T08:38:43.632193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:39.560848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629112+0000","last_change":"2024-09-10T08:37:05.254641+0000","last_active":"2024-09-10T08:38:43.629112+0000","last_peered":"2024-09-10T08:38:43.629112+0000","last_clean":"2024-09-10T08:38:43.629112+0000","last_became_active":"2024-09-10T08:37:05.249982+0000","last_became_peered":"2024-09-10T08:37:05.249982+0000","last_unstale":"2024-09-10T08:38:43.629112+0000","last_undegraded":"2024-09-10T08:38:43.629112+0000","last_fullsized":"2024-09-10T08:38:43.629112+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:21:13.851061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630749+0000","last_change":"2024-09-10T08:36:56.142657+0000","last_active":"2024-09-10T08:38:43.630749+0000","last_peered":"2024-09-10T08:38:43.630749+0000","last_clean":"2024-09-10T08:38:43.630749+0000","last_became_active":"2024-09-10T08:36:56.142294+0000","last_became_peered":"2024-09-10T08:36:56.142294+0000","last_unstale":"2024-09-10T08:38:43.630749+0000","last_undegraded":"2024-09-10T08:38:43.630749+0000","last_fullsized":"2024-09-10T08:38:43.630749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:52.861389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"94'3","reported_seq":104,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:47.807974+0000","last_change":"2024-09-10T08:36:53.722466+0000","last_active":"2024-09-10T08:38:47.807974+0000","last_peered":"2024-09-10T08:38:47.807974+0000","last_clean":"2024-09-10T08:38:47.807974+0000","last_became_active":"2024-09-10T08:36:53.722145+0000","last_became_peered":"2024-09-10T08:36:53.722145+0000","last_unstale":"2024-09-10T08:38:47.807974+0000","last_undegraded":"2024-09-10T08:38:47.807974+0000","last_fullsized":"2024-09-10T08:38:47.807974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:27.041765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":61,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631303+0000","last_change":"2024-09-10T08:37:05.251056+0000","last_active":"2024-09-10T08:38:43.631303+0000","last_peered":"2024-09-10T08:38:43.631303+0000","last_clean":"2024-09-10T08:38:43.631303+0000","last_became_active":"2024-09-10T08:37:05.250653+0000","last_became_peered":"2024-09-10T08:37:05.250653+0000","last_unstale":"2024-09-10T08:38:43.631303+0000","last_undegraded":"2024-09-10T08:38:43.631303+0000","last_fullsized":"2024-09-10T08:38:43.631303+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:21.174747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633073+0000","last_change":"2024-09-10T08:37:05.260436+0000","last_active":"2024-09-10T08:38:43.633073+0000","last_peered":"2024-09-10T08:38:43.633073+0000","last_clean":"2024-09-10T08:38:43.633073+0000","last_became_active":"2024-09-10T08:37:05.260049+0000","last_became_peered":"2024-09-10T08:37:05.260049+0000","last_unstale":"2024-09-10T08:38:43.633073+0000","last_undegraded":"2024-09-10T08:38:43.633073+0000","last_fullsized":"2024-09-10T08:38:43.633073+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:43.394028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629156+0000","last_change":"2024-09-10T08:36:56.146732+0000","last_active":"2024-09-10T08:38:43.629156+0000","last_peered":"2024-09-10T08:38:43.629156+0000","last_clean":"2024-09-10T08:38:43.629156+0000","last_became_active":"2024-09-10T08:36:56.145599+0000","last_became_peered":"2024-09-10T08:36:56.145599+0000","last_unstale":"2024-09-10T08:38:43.629156+0000","last_undegraded":"2024-09-10T08:38:43.629156+0000","last_fullsized":"2024-09-10T08:38:43.629156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:50.620956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655039+0000","last_change":"2024-09-10T08:36:53.721683+0000","last_active":"2024-09-10T08:38:43.655039+0000","last_peered":"2024-09-10T08:38:43.655039+0000","last_clean":"2024-09-10T08:38:43.655039+0000","last_became_active":"2024-09-10T08:36:53.721241+0000","last_became_peered":"2024-09-10T08:36:53.721241+0000","last_unstale":"2024-09-10T08:38:43.655039+0000","last_undegraded":"2024-09-10T08:38:43.655039+0000","last_fullsized":"2024-09-10T08:38:43.655039+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:03.669337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628220+0000","last_change":"2024-09-10T08:37:42.669784+0000","last_active":"2024-09-10T08:38:43.628220+0000","last_peered":"2024-09-10T08:38:43.628220+0000","last_clean":"2024-09-10T08:38:43.628220+0000","last_became_active":"2024-09-10T08:37:42.669322+0000","last_became_peered":"2024-09-10T08:37:42.669322+0000","last_unstale":"2024-09-10T08:38:43.628220+0000","last_undegraded":"2024-09-10T08:38:43.628220+0000","last_fullsized":"2024-09-10T08:38:43.628220+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:06:57.027928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631075+0000","last_change":"2024-09-10T08:36:56.143824+0000","last_active":"2024-09-10T08:38:43.631075+0000","last_peered":"2024-09-10T08:38:43.631075+0000","last_clean":"2024-09-10T08:38:43.631075+0000","last_became_active":"2024-09-10T08:36:56.143486+0000","last_became_peered":"2024-09-10T08:36:56.143486+0000","last_unstale":"2024-09-10T08:38:43.631075+0000","last_undegraded":"2024-09-10T08:38:43.631075+0000","last_fullsized":"2024-09-10T08:38:43.631075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:42.384863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627882+0000","last_change":"2024-09-10T08:36:53.717680+0000","last_active":"2024-09-10T08:38:43.627882+0000","last_peered":"2024-09-10T08:38:43.627882+0000","last_clean":"2024-09-10T08:38:43.627882+0000","last_became_active":"2024-09-10T08:36:53.717249+0000","last_became_peered":"2024-09-10T08:36:53.717249+0000","last_unstale":"2024-09-10T08:38:43.627882+0000","last_undegraded":"2024-09-10T08:38:43.627882+0000","last_fullsized":"2024-09-10T08:38:43.627882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:12.191541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627126+0000","last_change":"2024-09-10T08:36:53.723947+0000","last_active":"2024-09-10T08:38:43.627126+0000","last_peered":"2024-09-10T08:38:43.627126+0000","last_clean":"2024-09-10T08:38:43.627126+0000","last_became_active":"2024-09-10T08:36:53.723620+0000","last_became_peered":"2024-09-10T08:36:53.723620+0000","last_unstale":"2024-09-10T08:38:43.627126+0000","last_undegraded":"2024-09-10T08:38:43.627126+0000","last_fullsized":"2024-09-10T08:38:43.627126+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:00.135349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655037+0000","last_change":"2024-09-10T08:36:56.150448+0000","last_active":"2024-09-10T08:38:43.655037+0000","last_peered":"2024-09-10T08:38:43.655037+0000","last_clean":"2024-09-10T08:38:43.655037+0000","last_became_active":"2024-09-10T08:36:56.150148+0000","last_became_peered":"2024-09-10T08:36:56.150148+0000","last_unstale":"2024-09-10T08:38:43.655037+0000","last_undegraded":"2024-09-10T08:38:43.655037+0000","last_fullsized":"2024-09-10T08:38:43.655037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:02.927753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630130+0000","last_change":"2024-09-10T08:37:05.242878+0000","last_active":"2024-09-10T08:38:43.630130+0000","last_peered":"2024-09-10T08:38:43.630130+0000","last_clean":"2024-09-10T08:38:43.630130+0000","last_became_active":"2024-09-10T08:37:05.242440+0000","last_became_peered":"2024-09-10T08:37:05.242440+0000","last_unstale":"2024-09-10T08:38:43.630130+0000","last_undegraded":"2024-09-10T08:38:43.630130+0000","last_fullsized":"2024-09-10T08:38:43.630130+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:30.957433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'5","reported_seq":109,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:48.809610+0000","last_change":"2024-09-10T08:36:53.719788+0000","last_active":"2024-09-10T08:38:48.809610+0000","last_peered":"2024-09-10T08:38:48.809610+0000","last_clean":"2024-09-10T08:38:48.809610+0000","last_became_active":"2024-09-10T08:36:53.719462+0000","last_became_peered":"2024-09-10T08:36:53.719462+0000","last_unstale":"2024-09-10T08:38:48.809610+0000","last_undegraded":"2024-09-10T08:38:48.809610+0000","last_fullsized":"2024-09-10T08:38:48.809610+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:12.250447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626009+0000","last_change":"2024-09-10T08:37:42.673008+0000","last_active":"2024-09-10T08:38:43.626009+0000","last_peered":"2024-09-10T08:38:43.626009+0000","last_clean":"2024-09-10T08:38:43.626009+0000","last_became_active":"2024-09-10T08:37:42.672618+0000","last_became_peered":"2024-09-10T08:37:42.672618+0000","last_unstale":"2024-09-10T08:38:43.626009+0000","last_undegraded":"2024-09-10T08:38:43.626009+0000","last_fullsized":"2024-09-10T08:38:43.626009+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:00.156565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630734+0000","last_change":"2024-09-10T08:37:05.244277+0000","last_active":"2024-09-10T08:38:43.630734+0000","last_peered":"2024-09-10T08:38:43.630734+0000","last_clean":"2024-09-10T08:38:43.630734+0000","last_became_active":"2024-09-10T08:37:05.243880+0000","last_became_peered":"2024-09-10T08:37:05.243880+0000","last_unstale":"2024-09-10T08:38:43.630734+0000","last_undegraded":"2024-09-10T08:38:43.630734+0000","last_fullsized":"2024-09-10T08:38:43.630734+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:55.275857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627792+0000","last_change":"2024-09-10T08:36:56.141404+0000","last_active":"2024-09-10T08:38:43.627792+0000","last_peered":"2024-09-10T08:38:43.627792+0000","last_clean":"2024-09-10T08:38:43.627792+0000","last_became_active":"2024-09-10T08:36:56.141076+0000","last_became_peered":"2024-09-10T08:36:56.141076+0000","last_unstale":"2024-09-10T08:38:43.627792+0000","last_undegraded":"2024-09-10T08:38:43.627792+0000","last_fullsized":"2024-09-10T08:38:43.627792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:03:52.915654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630119+0000","last_change":"2024-09-10T08:36:56.148285+0000","last_active":"2024-09-10T08:38:43.630119+0000","last_peered":"2024-09-10T08:38:43.630119+0000","last_clean":"2024-09-10T08:38:43.630119+0000","last_became_active":"2024-09-10T08:36:56.147970+0000","last_became_peered":"2024-09-10T08:36:56.147970+0000","last_unstale":"2024-09-10T08:38:43.630119+0000","last_undegraded":"2024-09-10T08:38:43.630119+0000","last_fullsized":"2024-09-10T08:38:43.630119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:09.506718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630184+0000","last_change":"2024-09-10T08:36:56.157977+0000","last_active":"2024-09-10T08:38:43.630184+0000","last_peered":"2024-09-10T08:38:43.630184+0000","last_clean":"2024-09-10T08:38:43.630184+0000","last_became_active":"2024-09-10T08:36:56.157623+0000","last_became_peered":"2024-09-10T08:36:56.157623+0000","last_unstale":"2024-09-10T08:38:43.630184+0000","last_undegraded":"2024-09-10T08:38:43.630184+0000","last_fullsized":"2024-09-10T08:38:43.630184+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:44.617979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633131+0000","last_change":"2024-09-10T08:37:42.670459+0000","last_active":"2024-09-10T08:38:43.633131+0000","last_peered":"2024-09-10T08:38:43.633131+0000","last_clean":"2024-09-10T08:38:43.633131+0000","last_became_active":"2024-09-10T08:37:42.670097+0000","last_became_peered":"2024-09-10T08:37:42.670097+0000","last_unstale":"2024-09-10T08:38:43.633131+0000","last_undegraded":"2024-09-10T08:38:43.633131+0000","last_fullsized":"2024-09-10T08:38:43.633131+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:14.956492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628490+0000","last_change":"2024-09-10T08:36:56.154850+0000","last_active":"2024-09-10T08:38:43.628490+0000","last_peered":"2024-09-10T08:38:43.628490+0000","last_clean":"2024-09-10T08:38:43.628490+0000","last_became_active":"2024-09-10T08:36:56.154545+0000","last_became_peered":"2024-09-10T08:36:56.154545+0000","last_unstale":"2024-09-10T08:38:43.628490+0000","last_undegraded":"2024-09-10T08:38:43.628490+0000","last_fullsized":"2024-09-10T08:38:43.628490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:24.639667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628189+0000","last_change":"2024-09-10T08:37:42.669962+0000","last_active":"2024-09-10T08:38:43.628189+0000","last_peered":"2024-09-10T08:38:43.628189+0000","last_clean":"2024-09-10T08:38:43.628189+0000","last_became_active":"2024-09-10T08:37:42.669556+0000","last_became_peered":"2024-09-10T08:37:42.669556+0000","last_unstale":"2024-09-10T08:38:43.628189+0000","last_undegraded":"2024-09-10T08:38:43.628189+0000","last_fullsized":"2024-09-10T08:38:43.628189+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:23.333946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631230+0000","last_change":"2024-09-10T08:36:56.145910+0000","last_active":"2024-09-10T08:38:43.631230+0000","last_peered":"2024-09-10T08:38:43.631230+0000","last_clean":"2024-09-10T08:38:43.631230+0000","last_became_active":"2024-09-10T08:36:56.145537+0000","last_became_peered":"2024-09-10T08:36:56.145537+0000","last_unstale":"2024-09-10T08:38:43.631230+0000","last_undegraded":"2024-09-10T08:38:43.631230+0000","last_fullsized":"2024-09-10T08:38:43.631230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:46.788241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631941+0000","last_change":"2024-09-10T08:36:56.147340+0000","last_active":"2024-09-10T08:38:43.631941+0000","last_peered":"2024-09-10T08:38:43.631941+0000","last_clean":"2024-09-10T08:38:43.631941+0000","last_became_active":"2024-09-10T08:36:56.145762+0000","last_became_peered":"2024-09-10T08:36:56.145762+0000","last_unstale":"2024-09-10T08:38:43.631941+0000","last_undegraded":"2024-09-10T08:38:43.631941+0000","last_fullsized":"2024-09-10T08:38:43.631941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:12.339285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654832+0000","last_change":"2024-09-10T08:36:56.145213+0000","last_active":"2024-09-10T08:38:43.654832+0000","last_peered":"2024-09-10T08:38:43.654832+0000","last_clean":"2024-09-10T08:38:43.654832+0000","last_became_active":"2024-09-10T08:36:56.144915+0000","last_became_peered":"2024-09-10T08:36:56.144915+0000","last_unstale":"2024-09-10T08:38:43.654832+0000","last_undegraded":"2024-09-10T08:38:43.654832+0000","last_fullsized":"2024-09-10T08:38:43.654832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:06.180321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631378+0000","last_change":"2024-09-10T08:36:56.149738+0000","last_active":"2024-09-10T08:38:43.631378+0000","last_peered":"2024-09-10T08:38:43.631378+0000","last_clean":"2024-09-10T08:38:43.631378+0000","last_became_active":"2024-09-10T08:36:56.147750+0000","last_became_peered":"2024-09-10T08:36:56.147750+0000","last_unstale":"2024-09-10T08:38:43.631378+0000","last_undegraded":"2024-09-10T08:38:43.631378+0000","last_fullsized":"2024-09-10T08:38:43.631378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:47.950054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630316+0000","last_change":"2024-09-10T08:36:56.147682+0000","last_active":"2024-09-10T08:38:43.630316+0000","last_peered":"2024-09-10T08:38:43.630316+0000","last_clean":"2024-09-10T08:38:43.630316+0000","last_became_active":"2024-09-10T08:36:56.147345+0000","last_became_peered":"2024-09-10T08:36:56.147345+0000","last_unstale":"2024-09-10T08:38:43.630316+0000","last_undegraded":"2024-09-10T08:38:43.630316+0000","last_fullsized":"2024-09-10T08:38:43.630316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:54.939482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628008+0000","last_change":"2024-09-10T08:36:56.158292+0000","last_active":"2024-09-10T08:38:43.628008+0000","last_peered":"2024-09-10T08:38:43.628008+0000","last_clean":"2024-09-10T08:38:43.628008+0000","last_became_active":"2024-09-10T08:36:56.157970+0000","last_became_peered":"2024-09-10T08:36:56.157970+0000","last_unstale":"2024-09-10T08:38:43.628008+0000","last_undegraded":"2024-09-10T08:38:43.628008+0000","last_fullsized":"2024-09-10T08:38:43.628008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:33.582126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628048+0000","last_change":"2024-09-10T08:36:56.142520+0000","last_active":"2024-09-10T08:38:43.628048+0000","last_peered":"2024-09-10T08:38:43.628048+0000","last_clean":"2024-09-10T08:38:43.628048+0000","last_became_active":"2024-09-10T08:36:56.142214+0000","last_became_peered":"2024-09-10T08:36:56.142214+0000","last_unstale":"2024-09-10T08:38:43.628048+0000","last_undegraded":"2024-09-10T08:38:43.628048+0000","last_fullsized":"2024-09-10T08:38:43.628048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:45:29.201839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630914+0000","last_change":"2024-09-10T08:36:56.149547+0000","last_active":"2024-09-10T08:38:43.630914+0000","last_peered":"2024-09-10T08:38:43.630914+0000","last_clean":"2024-09-10T08:38:43.630914+0000","last_became_active":"2024-09-10T08:36:56.149141+0000","last_became_peered":"2024-09-10T08:36:56.149141+0000","last_unstale":"2024-09-10T08:38:43.630914+0000","last_undegraded":"2024-09-10T08:38:43.630914+0000","last_fullsized":"2024-09-10T08:38:43.630914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:49.368677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629885+0000","last_change":"2024-09-10T08:36:56.143087+0000","last_active":"2024-09-10T08:38:43.629885+0000","last_peered":"2024-09-10T08:38:43.629885+0000","last_clean":"2024-09-10T08:38:43.629885+0000","last_became_active":"2024-09-10T08:36:56.142782+0000","last_became_peered":"2024-09-10T08:36:56.142782+0000","last_unstale":"2024-09-10T08:38:43.629885+0000","last_undegraded":"2024-09-10T08:38:43.629885+0000","last_fullsized":"2024-09-10T08:38:43.629885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:17.513103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628447+0000","last_change":"2024-09-10T08:36:56.157549+0000","last_active":"2024-09-10T08:38:43.628447+0000","last_peered":"2024-09-10T08:38:43.628447+0000","last_clean":"2024-09-10T08:38:43.628447+0000","last_became_active":"2024-09-10T08:36:56.157221+0000","last_became_peered":"2024-09-10T08:36:56.157221+0000","last_unstale":"2024-09-10T08:38:43.628447+0000","last_undegraded":"2024-09-10T08:38:43.628447+0000","last_fullsized":"2024-09-10T08:38:43.628447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:11.372085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629960+0000","last_change":"2024-09-10T08:36:56.145654+0000","last_active":"2024-09-10T08:38:43.629960+0000","last_peered":"2024-09-10T08:38:43.629960+0000","last_clean":"2024-09-10T08:38:43.629960+0000","last_became_active":"2024-09-10T08:36:56.144126+0000","last_became_peered":"2024-09-10T08:36:56.144126+0000","last_unstale":"2024-09-10T08:38:43.629960+0000","last_undegraded":"2024-09-10T08:38:43.629960+0000","last_fullsized":"2024-09-10T08:38:43.629960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:55.498873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.632681+0000","last_change":"2024-09-10T08:36:56.146423+0000","last_active":"2024-09-10T08:38:43.632681+0000","last_peered":"2024-09-10T08:38:43.632681+0000","last_clean":"2024-09-10T08:38:43.632681+0000","last_became_active":"2024-09-10T08:36:56.146126+0000","last_became_peered":"2024-09-10T08:36:56.146126+0000","last_unstale":"2024-09-10T08:38:43.632681+0000","last_undegraded":"2024-09-10T08:38:43.632681+0000","last_fullsized":"2024-09-10T08:38:43.632681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:44.665196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.632922+0000","last_change":"2024-09-10T08:36:56.151973+0000","last_active":"2024-09-10T08:38:43.632922+0000","last_peered":"2024-09-10T08:38:43.632922+0000","last_clean":"2024-09-10T08:38:43.632922+0000","last_became_active":"2024-09-10T08:36:56.151609+0000","last_became_peered":"2024-09-10T08:36:56.151609+0000","last_unstale":"2024-09-10T08:38:43.632922+0000","last_undegraded":"2024-09-10T08:38:43.632922+0000","last_fullsized":"2024-09-10T08:38:43.632922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:02.584047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627915+0000","last_change":"2024-09-10T08:36:56.141780+0000","last_active":"2024-09-10T08:38:43.627915+0000","last_peered":"2024-09-10T08:38:43.627915+0000","last_clean":"2024-09-10T08:38:43.627915+0000","last_became_active":"2024-09-10T08:36:56.141228+0000","last_became_peered":"2024-09-10T08:36:56.141228+0000","last_unstale":"2024-09-10T08:38:43.627915+0000","last_undegraded":"2024-09-10T08:38:43.627915+0000","last_fullsized":"2024-09-10T08:38:43.627915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:52:04.374228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629654+0000","last_change":"2024-09-10T08:37:42.671346+0000","last_active":"2024-09-10T08:38:43.629654+0000","last_peered":"2024-09-10T08:38:43.629654+0000","last_clean":"2024-09-10T08:38:43.629654+0000","last_became_active":"2024-09-10T08:37:42.670895+0000","last_became_peered":"2024-09-10T08:37:42.670895+0000","last_unstale":"2024-09-10T08:38:43.629654+0000","last_undegraded":"2024-09-10T08:38:43.629654+0000","last_fullsized":"2024-09-10T08:38:43.629654+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:16:31.930628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.632053+0000","last_change":"2024-09-10T08:36:56.149014+0000","last_active":"2024-09-10T08:38:43.632053+0000","last_peered":"2024-09-10T08:38:43.632053+0000","last_clean":"2024-09-10T08:38:43.632053+0000","last_became_active":"2024-09-10T08:36:56.148551+0000","last_became_peered":"2024-09-10T08:36:56.148551+0000","last_unstale":"2024-09-10T08:38:43.632053+0000","last_undegraded":"2024-09-10T08:38:43.632053+0000","last_fullsized":"2024-09-10T08:38:43.632053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:35:52.250943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627572+0000","last_change":"2024-09-10T08:36:56.141247+0000","last_active":"2024-09-10T08:38:43.627572+0000","last_peered":"2024-09-10T08:38:43.627572+0000","last_clean":"2024-09-10T08:38:43.627572+0000","last_became_active":"2024-09-10T08:36:56.140956+0000","last_became_peered":"2024-09-10T08:36:56.140956+0000","last_unstale":"2024-09-10T08:38:43.627572+0000","last_undegraded":"2024-09-10T08:38:43.627572+0000","last_fullsized":"2024-09-10T08:38:43.627572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:45.008424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630691+0000","last_change":"2024-09-10T08:36:56.149116+0000","last_active":"2024-09-10T08:38:43.630691+0000","last_peered":"2024-09-10T08:38:43.630691+0000","last_clean":"2024-09-10T08:38:43.630691+0000","last_became_active":"2024-09-10T08:36:56.148783+0000","last_became_peered":"2024-09-10T08:36:56.148783+0000","last_unstale":"2024-09-10T08:38:43.630691+0000","last_undegraded":"2024-09-10T08:38:43.630691+0000","last_fullsized":"2024-09-10T08:38:43.630691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:37.119214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629016+0000","last_change":"2024-09-10T08:37:42.846322+0000","last_active":"2024-09-10T08:38:43.629016+0000","last_peered":"2024-09-10T08:38:43.629016+0000","last_clean":"2024-09-10T08:38:43.629016+0000","last_became_active":"2024-09-10T08:37:42.846022+0000","last_became_peered":"2024-09-10T08:37:42.846022+0000","last_unstale":"2024-09-10T08:38:43.629016+0000","last_undegraded":"2024-09-10T08:38:43.629016+0000","last_fullsized":"2024-09-10T08:38:43.629016+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:38.142362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631593+0000","last_change":"2024-09-10T08:36:56.148537+0000","last_active":"2024-09-10T08:38:43.631593+0000","last_peered":"2024-09-10T08:38:43.631593+0000","last_clean":"2024-09-10T08:38:43.631593+0000","last_became_active":"2024-09-10T08:36:56.148185+0000","last_became_peered":"2024-09-10T08:36:56.148185+0000","last_unstale":"2024-09-10T08:38:43.631593+0000","last_undegraded":"2024-09-10T08:38:43.631593+0000","last_fullsized":"2024-09-10T08:38:43.631593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:00.426513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627054+0000","last_change":"2024-09-10T08:36:56.138513+0000","last_active":"2024-09-10T08:38:43.627054+0000","last_peered":"2024-09-10T08:38:43.627054+0000","last_clean":"2024-09-10T08:38:43.627054+0000","last_became_active":"2024-09-10T08:36:56.138207+0000","last_became_peered":"2024-09-10T08:36:56.138207+0000","last_unstale":"2024-09-10T08:38:43.627054+0000","last_undegraded":"2024-09-10T08:38:43.627054+0000","last_fullsized":"2024-09-10T08:38:43.627054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:34.581430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.625900+0000","last_change":"2024-09-10T08:36:56.146612+0000","last_active":"2024-09-10T08:38:43.625900+0000","last_peered":"2024-09-10T08:38:43.625900+0000","last_clean":"2024-09-10T08:38:43.625900+0000","last_became_active":"2024-09-10T08:36:56.146251+0000","last_became_peered":"2024-09-10T08:36:56.146251+0000","last_unstale":"2024-09-10T08:38:43.625900+0000","last_undegraded":"2024-09-10T08:38:43.625900+0000","last_fullsized":"2024-09-10T08:38:43.625900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:47.336389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655270+0000","last_change":"2024-09-10T08:36:56.148260+0000","last_active":"2024-09-10T08:38:43.655270+0000","last_peered":"2024-09-10T08:38:43.655270+0000","last_clean":"2024-09-10T08:38:43.655270+0000","last_became_active":"2024-09-10T08:36:56.147949+0000","last_became_peered":"2024-09-10T08:36:56.147949+0000","last_unstale":"2024-09-10T08:38:43.655270+0000","last_undegraded":"2024-09-10T08:38:43.655270+0000","last_fullsized":"2024-09-10T08:38:43.655270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:52:19.762297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633344+0000","last_change":"2024-09-10T08:36:56.156045+0000","last_active":"2024-09-10T08:38:43.633344+0000","last_peered":"2024-09-10T08:38:43.633344+0000","last_clean":"2024-09-10T08:38:43.633344+0000","last_became_active":"2024-09-10T08:36:56.154727+0000","last_became_peered":"2024-09-10T08:36:56.154727+0000","last_unstale":"2024-09-10T08:38:43.633344+0000","last_undegraded":"2024-09-10T08:38:43.633344+0000","last_fullsized":"2024-09-10T08:38:43.633344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:35:39.992683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633092+0000","last_change":"2024-09-10T08:36:56.154394+0000","last_active":"2024-09-10T08:38:43.633092+0000","last_peered":"2024-09-10T08:38:43.633092+0000","last_clean":"2024-09-10T08:38:43.633092+0000","last_became_active":"2024-09-10T08:36:56.153855+0000","last_became_peered":"2024-09-10T08:36:56.153855+0000","last_unstale":"2024-09-10T08:38:43.633092+0000","last_undegraded":"2024-09-10T08:38:43.633092+0000","last_fullsized":"2024-09-10T08:38:43.633092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:14.987147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628479+0000","last_change":"2024-09-10T08:36:56.157517+0000","last_active":"2024-09-10T08:38:43.628479+0000","last_peered":"2024-09-10T08:38:43.628479+0000","last_clean":"2024-09-10T08:38:43.628479+0000","last_became_active":"2024-09-10T08:36:56.157133+0000","last_became_peered":"2024-09-10T08:36:56.157133+0000","last_unstale":"2024-09-10T08:38:43.628479+0000","last_undegraded":"2024-09-10T08:38:43.628479+0000","last_fullsized":"2024-09-10T08:38:43.628479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:17:29.573719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67514,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":395,"num_read_kb":268,"num_write":164,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":202542,"data_compressed":5667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282455,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416422,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1056768,"data_stored":547326,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678058,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28212,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716940,"statfs":{"total":95995035648,"available":95966146560,"internally_reserved":0,"allocated":1208320,"data_stored":1206690,"data_compressed":67339,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841584,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1069056,"data_stored":543466,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103222,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1069056,"data_stored":546900,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332155,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1122304,"data_stored":1143393,"data_compressed":65450,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495680,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1052672,"data_stored":542738,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757318,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1130496,"data_stored":1138588,"data_compressed":65450,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1069056,"data_stored":548363,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1064960,"data_stored":542932,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1138688,"data_stored":606927,"data_compressed":31413,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1122304,"data_stored":606741,"data_compressed":31413,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1077248,"data_stored":544126,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64093,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64279,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65482,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:38:52.875 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T08:38:52.876 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T08:38:52.876 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T08:38:52.876 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T08:38:52.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:52 smithi149 ceph-mon[36167]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 129 B/s, 0 objects/s recovering 2024-09-10T08:38:53.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:52 smithi111 ceph-mon[35220]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 129 B/s, 0 objects/s recovering 2024-09-10T08:38:53.242 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:53.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.811+0000 7fb4c3200640 1 -- 172.21.15.89:0/1819046470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4c41079d0 msgr2=0x7fb4c4107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:53.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.811+0000 7fb4c3200640 1 --2- 172.21.15.89:0/1819046470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4c41079d0 0x7fb4c4107db0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4009930 tx=0x7fb4b402a420 comp rx=0 tx=0).stop 2024-09-10T08:38:53.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.811+0000 7fb4c3200640 1 -- 172.21.15.89:0/1819046470 shutdown_connections 2024-09-10T08:38:53.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.811+0000 7fb4c3200640 1 --2- 172.21.15.89:0/1819046470 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4c4102370 0x7fb4c410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:53.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.811+0000 7fb4c3200640 1 --2- 172.21.15.89:0/1819046470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4c41019d0 0x7fb4c4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:53.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.811+0000 7fb4c3200640 1 --2- 172.21.15.89:0/1819046470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4c41079d0 0x7fb4c4107db0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:53.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.811+0000 7fb4c3200640 1 -- 172.21.15.89:0/1819046470 >> 172.21.15.89:0/1819046470 conn(0x7fb4c40fd710 msgr2=0x7fb4c40ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:53.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.811+0000 7fb4c3200640 1 -- 172.21.15.89:0/1819046470 shutdown_connections 2024-09-10T08:38:53.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.811+0000 7fb4c3200640 1 -- 172.21.15.89:0/1819046470 wait complete. 2024-09-10T08:38:53.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.812+0000 7fb4c3200640 1 Processor -- start 2024-09-10T08:38:53.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.812+0000 7fb4c3200640 1 -- start start 2024-09-10T08:38:53.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.813+0000 7fb4c3200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4c41019d0 0x7fb4c41982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:53.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.813+0000 7fb4c3200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4c4102370 0x7fb4c41987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:53.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.813+0000 7fb4c3200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4c41079d0 0x7fb4c419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:53.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.813+0000 7fb4c3200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c4111dd0 con 0x7fb4c41079d0 2024-09-10T08:38:53.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.813+0000 7fb4c3200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c4111c50 con 0x7fb4c41019d0 2024-09-10T08:38:53.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.813+0000 7fb4c3200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c4111f50 con 0x7fb4c4102370 2024-09-10T08:38:53.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.813+0000 7fb4c1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4c41019d0 0x7fb4c41982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:53.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.813+0000 7fb4c1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4c41019d0 0x7fb4c41982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:42872/0 (socket says 172.21.15.89:42872) 2024-09-10T08:38:53.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.813+0000 7fb4c1e00640 1 -- 172.21.15.89:0/2736025668 learned_addr learned my addr 172.21.15.89:0/2736025668 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:53.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.813+0000 7fb4c2800640 1 --2- 172.21.15.89:0/2736025668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4c41079d0 0x7fb4c419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:53.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.813+0000 7fb4bbe00640 1 --2- 172.21.15.89:0/2736025668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4c4102370 0x7fb4c41987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:53.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.813+0000 7fb4c1e00640 1 -- 172.21.15.89:0/2736025668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4c4102370 msgr2=0x7fb4c41987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:53.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.813+0000 7fb4c1e00640 1 --2- 172.21.15.89:0/2736025668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4c4102370 0x7fb4c41987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:53.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.813+0000 7fb4c1e00640 1 -- 172.21.15.89:0/2736025668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4c41079d0 msgr2=0x7fb4c419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:53.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.813+0000 7fb4c1e00640 1 --2- 172.21.15.89:0/2736025668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4c41079d0 0x7fb4c419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:53.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.814+0000 7fb4c1e00640 1 -- 172.21.15.89:0/2736025668 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c419d260 con 0x7fb4c41019d0 2024-09-10T08:38:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.814+0000 7fb4c1e00640 1 --2- 172.21.15.89:0/2736025668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4c41019d0 0x7fb4c41982b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4009230 tx=0x7fb4b4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.814+0000 7fb4baa00640 1 -- 172.21.15.89:0/2736025668 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b403f030 con 0x7fb4c41019d0 2024-09-10T08:38:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.814+0000 7fb4c3200640 1 -- 172.21.15.89:0/2736025668 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c419d4f0 con 0x7fb4c41019d0 2024-09-10T08:38:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.814+0000 7fb4baa00640 1 -- 172.21.15.89:0/2736025668 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4b4004440 con 0x7fb4c41019d0 2024-09-10T08:38:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.814+0000 7fb4baa00640 1 -- 172.21.15.89:0/2736025668 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4035070 con 0x7fb4c41019d0 2024-09-10T08:38:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.815+0000 7fb4c3200640 1 -- 172.21.15.89:0/2736025668 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c41a4e30 con 0x7fb4c41019d0 2024-09-10T08:38:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.815+0000 7fb4c3200640 1 -- 172.21.15.89:0/2736025668 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4c4101e30 con 0x7fb4c41019d0 2024-09-10T08:38:53.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.818+0000 7fb4baa00640 1 -- 172.21.15.89:0/2736025668 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb4b4042030 con 0x7fb4c41019d0 2024-09-10T08:38:53.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.819+0000 7fb4baa00640 1 --2- 172.21.15.89:0/2736025668 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb4a80a2c10 0x7fb4a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:53.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.819+0000 7fb4baa00640 1 -- 172.21.15.89:0/2736025668 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb4b40efcf0 con 0x7fb4c41019d0 2024-09-10T08:38:53.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.819+0000 7fb4bbe00640 1 --2- 172.21.15.89:0/2736025668 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb4a80a2c10 0x7fb4a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:53.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.820+0000 7fb4bbe00640 1 --2- 172.21.15.89:0/2736025668 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb4a80a2c10 0x7fb4a80a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4107090 tx=0x7fb4ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:53.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:53.821+0000 7fb4baa00640 1 -- 172.21.15.89:0/2736025668 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb4b40bc070 con 0x7fb4c41019d0 2024-09-10T08:38:53.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:53 smithi149 ceph-mon[36167]: from='client.24922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:38:54.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:54.031+0000 7fb4c3200640 1 -- 172.21.15.89:0/2736025668 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb4c4109f40 con 0x7fb4c41019d0 2024-09-10T08:38:54.031 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:53 smithi089 ceph-mon[30757]: from='client.24922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:38:54.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:54.031+0000 7fb4baa00640 1 -- 172.21.15.89:0/2736025668 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb4b40c0370 con 0x7fb4c41019d0 2024-09-10T08:38:54.031 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:38:54.031 INFO:teuthology.orchestra.run.smithi089.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T08:38:54.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:54.033+0000 7fb4c3200640 1 -- 172.21.15.89:0/2736025668 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb4a80a2c10 msgr2=0x7fb4a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:54.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:54.033+0000 7fb4c3200640 1 --2- 172.21.15.89:0/2736025668 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb4a80a2c10 0x7fb4a80a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4107090 tx=0x7fb4ac005f90 comp rx=0 tx=0).stop 2024-09-10T08:38:54.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:54.034+0000 7fb4c3200640 1 -- 172.21.15.89:0/2736025668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4c41019d0 msgr2=0x7fb4c41982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:54.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:54.034+0000 7fb4c3200640 1 --2- 172.21.15.89:0/2736025668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4c41019d0 0x7fb4c41982b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4009230 tx=0x7fb4b4004040 comp rx=0 tx=0).stop 2024-09-10T08:38:54.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:54.034+0000 7fb4c3200640 1 -- 172.21.15.89:0/2736025668 shutdown_connections 2024-09-10T08:38:54.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:54.034+0000 7fb4c3200640 1 --2- 172.21.15.89:0/2736025668 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb4a80a2c10 0x7fb4a80a50d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:54.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:54.034+0000 7fb4c3200640 1 --2- 172.21.15.89:0/2736025668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4c41079d0 0x7fb4c419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:54.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:54.034+0000 7fb4c3200640 1 --2- 172.21.15.89:0/2736025668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4c4102370 0x7fb4c41987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:54.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:54.034+0000 7fb4c3200640 1 --2- 172.21.15.89:0/2736025668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4c41019d0 0x7fb4c41982b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:54.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:54.034+0000 7fb4c3200640 1 -- 172.21.15.89:0/2736025668 >> 172.21.15.89:0/2736025668 conn(0x7fb4c40fd710 msgr2=0x7fb4c40ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:54.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:54.034+0000 7fb4c3200640 1 -- 172.21.15.89:0/2736025668 shutdown_connections 2024-09-10T08:38:54.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:54.034+0000 7fb4c3200640 1 -- 172.21.15.89:0/2736025668 wait complete. 2024-09-10T08:38:54.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:53 smithi111 ceph-mon[35220]: from='client.24922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:38:54.747 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:54 smithi089 ceph-mon[30757]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 116 B/s, 0 objects/s recovering 2024-09-10T08:38:54.748 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:54 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2736025668' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T08:38:54.785 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T08:38:54.786 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:38:54.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:54 smithi149 ceph-mon[36167]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 116 B/s, 0 objects/s recovering 2024-09-10T08:38:54.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:54 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2736025668' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T08:38:55.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:54 smithi111 ceph-mon[35220]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 116 B/s, 0 objects/s recovering 2024-09-10T08:38:55.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:54 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2736025668' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T08:38:55.150 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:55.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.720+0000 7f816c800640 1 -- 172.21.15.89:0/1506163255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81641049a0 msgr2=0x7f81641089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:55.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.720+0000 7f816c800640 1 --2- 172.21.15.89:0/1506163255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81641049a0 0x7f81641089f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8154009900 tx=0x7f815402a410 comp rx=0 tx=0).stop 2024-09-10T08:38:55.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.721+0000 7f816c800640 1 -- 172.21.15.89:0/1506163255 shutdown_connections 2024-09-10T08:38:55.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.721+0000 7f816c800640 1 --2- 172.21.15.89:0/1506163255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8164109120 0x7f8164110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:55.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.721+0000 7f816c800640 1 --2- 172.21.15.89:0/1506163255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81641049a0 0x7f81641089f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:55.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.721+0000 7f816c800640 1 --2- 172.21.15.89:0/1506163255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8164103ff0 0x7f81641043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:55.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.721+0000 7f816c800640 1 -- 172.21.15.89:0/1506163255 >> 172.21.15.89:0/1506163255 conn(0x7f81640ffa90 msgr2=0x7f8164101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:55.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.721+0000 7f816c800640 1 -- 172.21.15.89:0/1506163255 shutdown_connections 2024-09-10T08:38:55.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.721+0000 7f816c800640 1 -- 172.21.15.89:0/1506163255 wait complete. 2024-09-10T08:38:55.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.722+0000 7f816c800640 1 Processor -- start 2024-09-10T08:38:55.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.722+0000 7f816c800640 1 -- start start 2024-09-10T08:38:55.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.722+0000 7f816c800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8164103ff0 0x7f816419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:55.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.722+0000 7f816c800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81641049a0 0x7f816419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:55.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.722+0000 7f816c800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8164109120 0x7f816419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:55.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.722+0000 7f816c800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8164114120 con 0x7f81641049a0 2024-09-10T08:38:55.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.722+0000 7f816c800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8164113fa0 con 0x7f8164109120 2024-09-10T08:38:55.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.722+0000 7f816c800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f81641142a0 con 0x7f8164103ff0 2024-09-10T08:38:55.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.722+0000 7f8169e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8164103ff0 0x7f816419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:55.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.722+0000 7f8169400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81641049a0 0x7f816419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:55.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.722+0000 7f8169e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8164103ff0 0x7f816419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:49620/0 (socket says 172.21.15.89:49620) 2024-09-10T08:38:55.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.722+0000 7f8169e00640 1 -- 172.21.15.89:0/4282341128 learned_addr learned my addr 172.21.15.89:0/4282341128 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:55.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.723+0000 7f816a800640 1 --2- 172.21.15.89:0/4282341128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8164109120 0x7f816419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:55.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.723+0000 7f8169e00640 1 -- 172.21.15.89:0/4282341128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8164109120 msgr2=0x7f816419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:55.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.723+0000 7f8169e00640 1 --2- 172.21.15.89:0/4282341128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8164109120 0x7f816419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:55.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.723+0000 7f8169e00640 1 -- 172.21.15.89:0/4282341128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81641049a0 msgr2=0x7f816419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:55.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.723+0000 7f8169e00640 1 --2- 172.21.15.89:0/4282341128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81641049a0 0x7f816419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:55.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.723+0000 7f8169e00640 1 -- 172.21.15.89:0/4282341128 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816419f5d0 con 0x7f8164103ff0 2024-09-10T08:38:55.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.723+0000 7f8169e00640 1 --2- 172.21.15.89:0/4282341128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8164103ff0 0x7f816419a620 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f815800ea60 tx=0x7f815800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:55.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.723+0000 7f815ea00640 1 -- 172.21.15.89:0/4282341128 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815800cde0 con 0x7f8164103ff0 2024-09-10T08:38:55.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.723+0000 7f815ea00640 1 -- 172.21.15.89:0/4282341128 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81580035f0 con 0x7f8164103ff0 2024-09-10T08:38:55.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.723+0000 7f816c800640 1 -- 172.21.15.89:0/4282341128 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816419f8c0 con 0x7f8164103ff0 2024-09-10T08:38:55.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.723+0000 7f815ea00640 1 -- 172.21.15.89:0/4282341128 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158010670 con 0x7f8164103ff0 2024-09-10T08:38:55.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.724+0000 7f816c800640 1 -- 172.21.15.89:0/4282341128 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81641a71a0 con 0x7f8164103ff0 2024-09-10T08:38:55.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.725+0000 7f816c800640 1 -- 172.21.15.89:0/4282341128 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81641043d0 con 0x7f8164103ff0 2024-09-10T08:38:55.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.726+0000 7f815ea00640 1 -- 172.21.15.89:0/4282341128 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f81580037a0 con 0x7f8164103ff0 2024-09-10T08:38:55.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.727+0000 7f815ea00640 1 --2- 172.21.15.89:0/4282341128 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f814c0a2c10 0x7f814c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:55.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.728+0000 7f815ea00640 1 -- 172.21.15.89:0/4282341128 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f81580cf870 con 0x7f8164103ff0 2024-09-10T08:38:55.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.728+0000 7f8169400640 1 --2- 172.21.15.89:0/4282341128 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f814c0a2c10 0x7f814c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:55.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.728+0000 7f8169400640 1 --2- 172.21.15.89:0/4282341128 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f814c0a2c10 0x7f814c0a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8154009230 tx=0x7f81540058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:55.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.732+0000 7f815ea00640 1 -- 172.21.15.89:0/4282341128 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f815809c240 con 0x7f8164103ff0 2024-09-10T08:38:55.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.941+0000 7f816c800640 1 -- 172.21.15.89:0/4282341128 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f816410c3c0 con 0x7f8164103ff0 2024-09-10T08:38:55.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.942+0000 7f815ea00640 1 -- 172.21.15.89:0/4282341128 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11918 (secure 0 0 0) 0x7f815809bb40 con 0x7f8164103ff0 2024-09-10T08:38:55.942 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:38:55.942 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":25,"btime":"2024-09-10T08:37:50:718359+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:37:50.607307+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24496,24518,24554]},"id":1}]} 2024-09-10T08:38:55.942 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 25 2024-09-10T08:38:55.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.945+0000 7f816c800640 1 -- 172.21.15.89:0/4282341128 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f814c0a2c10 msgr2=0x7f814c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:55.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.945+0000 7f816c800640 1 --2- 172.21.15.89:0/4282341128 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f814c0a2c10 0x7f814c0a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8154009230 tx=0x7f81540058d0 comp rx=0 tx=0).stop 2024-09-10T08:38:55.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.945+0000 7f816c800640 1 -- 172.21.15.89:0/4282341128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8164103ff0 msgr2=0x7f816419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:55.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.945+0000 7f816c800640 1 --2- 172.21.15.89:0/4282341128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8164103ff0 0x7f816419a620 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f815800ea60 tx=0x7f815800ef20 comp rx=0 tx=0).stop 2024-09-10T08:38:55.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.945+0000 7f816c800640 1 -- 172.21.15.89:0/4282341128 shutdown_connections 2024-09-10T08:38:55.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.945+0000 7f816c800640 1 --2- 172.21.15.89:0/4282341128 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f814c0a2c10 0x7f814c0a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:55.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.945+0000 7f816c800640 1 --2- 172.21.15.89:0/4282341128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8164109120 0x7f816419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:55.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.945+0000 7f816c800640 1 --2- 172.21.15.89:0/4282341128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81641049a0 0x7f816419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:55.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.945+0000 7f816c800640 1 --2- 172.21.15.89:0/4282341128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8164103ff0 0x7f816419a620 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:55.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.945+0000 7f816c800640 1 -- 172.21.15.89:0/4282341128 >> 172.21.15.89:0/4282341128 conn(0x7f81640ffa90 msgr2=0x7f8164100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:55.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.945+0000 7f816c800640 1 -- 172.21.15.89:0/4282341128 shutdown_connections 2024-09-10T08:38:55.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:55.945+0000 7f816c800640 1 -- 172.21.15.89:0/4282341128 wait complete. 2024-09-10T08:38:56.719 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:38:56.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:56 smithi149 ceph-mon[36167]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 97 B/s, 0 objects/s recovering 2024-09-10T08:38:56.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:56 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/4282341128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:38:57.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:56 smithi111 ceph-mon[35220]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 97 B/s, 0 objects/s recovering 2024-09-10T08:38:57.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:56 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/4282341128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:38:57.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:56 smithi089 ceph-mon[30757]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 97 B/s, 0 objects/s recovering 2024-09-10T08:38:57.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:56 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/4282341128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:38:57.085 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.697+0000 7fc8eaa00640 1 -- 172.21.15.89:0/1245048240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8e4109b10 msgr2=0x7fc8e4114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.697+0000 7fc8eaa00640 1 --2- 172.21.15.89:0/1245048240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8e4109b10 0x7fc8e4114570 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009900 tx=0x7fc8d002a480 comp rx=0 tx=0).stop 2024-09-10T08:38:57.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.697+0000 7fc8eaa00640 1 -- 172.21.15.89:0/1245048240 shutdown_connections 2024-09-10T08:38:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.697+0000 7fc8eaa00640 1 --2- 172.21.15.89:0/1245048240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e4114ab0 0x7fc8e4116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.697+0000 7fc8eaa00640 1 --2- 172.21.15.89:0/1245048240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8e4109b10 0x7fc8e4114570 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.697+0000 7fc8eaa00640 1 --2- 172.21.15.89:0/1245048240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8e41091f0 0x7fc8e41095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.697+0000 7fc8eaa00640 1 -- 172.21.15.89:0/1245048240 >> 172.21.15.89:0/1245048240 conn(0x7fc8e406e140 msgr2=0x7fc8e406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:57.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.697+0000 7fc8eaa00640 1 -- 172.21.15.89:0/1245048240 shutdown_connections 2024-09-10T08:38:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.697+0000 7fc8eaa00640 1 -- 172.21.15.89:0/1245048240 wait complete. 2024-09-10T08:38:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.698+0000 7fc8eaa00640 1 Processor -- start 2024-09-10T08:38:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.699+0000 7fc8eaa00640 1 -- start start 2024-09-10T08:38:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.699+0000 7fc8eaa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8e41091f0 0x7fc8e4113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.699+0000 7fc8eaa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8e4109b10 0x7fc8e4113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.699+0000 7fc8eaa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e4114ab0 0x7fc8e41abce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.699+0000 7fc8eaa00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e411a800 con 0x7fc8e4109b10 2024-09-10T08:38:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.699+0000 7fc8eaa00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e411a680 con 0x7fc8e41091f0 2024-09-10T08:38:57.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.699+0000 7fc8eaa00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e411a980 con 0x7fc8e4114ab0 2024-09-10T08:38:57.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.699+0000 7fc8e3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8e41091f0 0x7fc8e4113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:57.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.699+0000 7fc8e8a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e4114ab0 0x7fc8e41abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:57.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.699+0000 7fc8e3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8e4109b10 0x7fc8e4113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:57.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.699+0000 7fc8e3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8e41091f0 0x7fc8e4113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:42920/0 (socket says 172.21.15.89:42920) 2024-09-10T08:38:57.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.699+0000 7fc8e3e00640 1 -- 172.21.15.89:0/47992859 learned_addr learned my addr 172.21.15.89:0/47992859 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:38:57.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.700+0000 7fc8e3e00640 1 -- 172.21.15.89:0/47992859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e4114ab0 msgr2=0x7fc8e41abce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:57.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.700+0000 7fc8e3e00640 1 --2- 172.21.15.89:0/47992859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e4114ab0 0x7fc8e41abce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:57.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.700+0000 7fc8e3e00640 1 -- 172.21.15.89:0/47992859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8e4109b10 msgr2=0x7fc8e4113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:57.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.700+0000 7fc8e3e00640 1 --2- 172.21.15.89:0/47992859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8e4109b10 0x7fc8e4113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:57.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.700+0000 7fc8e3e00640 1 -- 172.21.15.89:0/47992859 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e41ac410 con 0x7fc8e41091f0 2024-09-10T08:38:57.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.700+0000 7fc8e3e00640 1 --2- 172.21.15.89:0/47992859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8e41091f0 0x7fc8e4113970 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400e9f0 tx=0x7fc8d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:57.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.700+0000 7fc8e0c00640 1 -- 172.21.15.89:0/47992859 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d400cd80 con 0x7fc8e41091f0 2024-09-10T08:38:57.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.700+0000 7fc8e0c00640 1 -- 172.21.15.89:0/47992859 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8d4010910 con 0x7fc8e41091f0 2024-09-10T08:38:57.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.700+0000 7fc8e0c00640 1 -- 172.21.15.89:0/47992859 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4018990 con 0x7fc8e41091f0 2024-09-10T08:38:57.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.700+0000 7fc8eaa00640 1 -- 172.21.15.89:0/47992859 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e41ac700 con 0x7fc8e41091f0 2024-09-10T08:38:57.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.701+0000 7fc8eaa00640 1 -- 172.21.15.89:0/47992859 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e41b3f50 con 0x7fc8e41091f0 2024-09-10T08:38:57.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.702+0000 7fc8eaa00640 1 -- 172.21.15.89:0/47992859 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e41095d0 con 0x7fc8e41091f0 2024-09-10T08:38:57.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.704+0000 7fc8e0c00640 1 -- 172.21.15.89:0/47992859 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc8d4010ac0 con 0x7fc8e41091f0 2024-09-10T08:38:57.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.705+0000 7fc8e0c00640 1 --2- 172.21.15.89:0/47992859 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc8c80a2c90 0x7fc8c80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:57.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.705+0000 7fc8e0c00640 1 -- 172.21.15.89:0/47992859 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fc8d40cfcd0 con 0x7fc8e41091f0 2024-09-10T08:38:57.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.706+0000 7fc8e3400640 1 --2- 172.21.15.89:0/47992859 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc8c80a2c90 0x7fc8c80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:57.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.706+0000 7fc8e3400640 1 --2- 172.21.15.89:0/47992859 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc8c80a2c90 0x7fc8c80a5150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0000c00 tx=0x7fc8d0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:57.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.707+0000 7fc8e0c00640 1 -- 172.21.15.89:0/47992859 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc8d409c050 con 0x7fc8e41091f0 2024-09-10T08:38:57.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.940+0000 7fc8eaa00640 1 -- 172.21.15.89:0/47992859 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8e41a8700 con 0x7fc8e41091f0 2024-09-10T08:38:57.940 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:57 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:57.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.941+0000 7fc8e0c00640 1 -- 172.21.15.89:0/47992859 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11918 (secure 0 0 0) 0x7fc8d40a0350 con 0x7fc8e41091f0 2024-09-10T08:38:57.941 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:38:57.942 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":25,"btime":"2024-09-10T08:37:50:718359+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:37:50.607307+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24496,24518,24554]},"id":1}]} 2024-09-10T08:38:57.942 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 25 2024-09-10T08:38:57.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.944+0000 7fc8eaa00640 1 -- 172.21.15.89:0/47992859 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc8c80a2c90 msgr2=0x7fc8c80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:57.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.944+0000 7fc8eaa00640 1 --2- 172.21.15.89:0/47992859 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc8c80a2c90 0x7fc8c80a5150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0000c00 tx=0x7fc8d0005b50 comp rx=0 tx=0).stop 2024-09-10T08:38:57.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.944+0000 7fc8eaa00640 1 -- 172.21.15.89:0/47992859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8e41091f0 msgr2=0x7fc8e4113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:57.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.944+0000 7fc8eaa00640 1 --2- 172.21.15.89:0/47992859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8e41091f0 0x7fc8e4113970 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400e9f0 tx=0x7fc8d400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:38:57.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.944+0000 7fc8eaa00640 1 -- 172.21.15.89:0/47992859 shutdown_connections 2024-09-10T08:38:57.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.944+0000 7fc8eaa00640 1 --2- 172.21.15.89:0/47992859 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc8c80a2c90 0x7fc8c80a5150 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:57.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.944+0000 7fc8eaa00640 1 --2- 172.21.15.89:0/47992859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e4114ab0 0x7fc8e41abce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:57.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.944+0000 7fc8eaa00640 1 --2- 172.21.15.89:0/47992859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8e4109b10 0x7fc8e4113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:57.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.944+0000 7fc8eaa00640 1 --2- 172.21.15.89:0/47992859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8e41091f0 0x7fc8e4113970 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:57.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.944+0000 7fc8eaa00640 1 -- 172.21.15.89:0/47992859 >> 172.21.15.89:0/47992859 conn(0x7fc8e406e140 msgr2=0x7fc8e4071d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:57.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.944+0000 7fc8eaa00640 1 -- 172.21.15.89:0/47992859 shutdown_connections 2024-09-10T08:38:57.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:38:57.944+0000 7fc8eaa00640 1 -- 172.21.15.89:0/47992859 wait complete. 2024-09-10T08:38:57.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:57 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:57.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:57 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:58.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:58 smithi149 ceph-mon[36167]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s; 96 B/s, 0 objects/s recovering 2024-09-10T08:38:58.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:58 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/47992859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:38:59.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:58 smithi111 ceph-mon[35220]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s; 96 B/s, 0 objects/s recovering 2024-09-10T08:38:59.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:58 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/47992859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:38:59.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:58 smithi089 ceph-mon[30757]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s; 96 B/s, 0 objects/s recovering 2024-09-10T08:38:59.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:58 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/47992859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:38:59.405 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:38:59.804 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:00.131 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:00.131 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:00.131 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:00.131 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:59 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:00.131 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:38:59 smithi089 ceph-mon[30757]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:00.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:00.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:00.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:00.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:59 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:00.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:38:59 smithi149 ceph-mon[36167]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:00.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:00.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:00.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:00.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:59 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:00.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:38:59 smithi111 ceph-mon[35220]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:00.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.392+0000 7f5e47200640 1 -- 172.21.15.89:0/4229917756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e40103e30 msgr2=0x7f5e40104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:00.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.392+0000 7f5e47200640 1 --2- 172.21.15.89:0/4229917756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e40103e30 0x7f5e40104210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009900 tx=0x7f5e3402a410 comp rx=0 tx=0).stop 2024-09-10T08:39:00.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.392+0000 7f5e47200640 1 -- 172.21.15.89:0/4229917756 shutdown_connections 2024-09-10T08:39:00.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.392+0000 7f5e47200640 1 --2- 172.21.15.89:0/4229917756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e40108f60 0x7f5e40110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:00.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.392+0000 7f5e47200640 1 --2- 172.21.15.89:0/4229917756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e401047e0 0x7f5e40108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:00.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.392+0000 7f5e47200640 1 --2- 172.21.15.89:0/4229917756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e40103e30 0x7f5e40104210 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:00.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.392+0000 7f5e47200640 1 -- 172.21.15.89:0/4229917756 >> 172.21.15.89:0/4229917756 conn(0x7f5e400ff910 msgr2=0x7f5e40101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:00.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.392+0000 7f5e47200640 1 -- 172.21.15.89:0/4229917756 shutdown_connections 2024-09-10T08:39:00.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.392+0000 7f5e47200640 1 -- 172.21.15.89:0/4229917756 wait complete. 2024-09-10T08:39:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.393+0000 7f5e47200640 1 Processor -- start 2024-09-10T08:39:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.393+0000 7f5e47200640 1 -- start start 2024-09-10T08:39:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.394+0000 7f5e47200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e40103e30 0x7f5e4019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.394+0000 7f5e47200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e401047e0 0x7f5e4019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.394+0000 7f5e47200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e40108f60 0x7f5e4019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.394+0000 7f5e47200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5e40113fd0 con 0x7f5e401047e0 2024-09-10T08:39:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.394+0000 7f5e47200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5e40113e50 con 0x7f5e40103e30 2024-09-10T08:39:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.394+0000 7f5e47200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5e40114150 con 0x7f5e40108f60 2024-09-10T08:39:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.394+0000 7f5e45400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e401047e0 0x7f5e4019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:00.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.394+0000 7f5e45400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e401047e0 0x7f5e4019a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33322/0 (socket says 172.21.15.89:33322) 2024-09-10T08:39:00.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.394+0000 7f5e45400640 1 -- 172.21.15.89:0/3990992391 learned_addr learned my addr 172.21.15.89:0/3990992391 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:00.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.394+0000 7f5e45e00640 1 --2- 172.21.15.89:0/3990992391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e40103e30 0x7f5e4019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:00.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.394+0000 7f5e46800640 1 --2- 172.21.15.89:0/3990992391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e40108f60 0x7f5e4019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:00.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.395+0000 7f5e45400640 1 -- 172.21.15.89:0/3990992391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e40108f60 msgr2=0x7f5e4019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:00.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.395+0000 7f5e45400640 1 --2- 172.21.15.89:0/3990992391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e40108f60 0x7f5e4019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:00.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.395+0000 7f5e45400640 1 -- 172.21.15.89:0/3990992391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e40103e30 msgr2=0x7f5e4019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:00.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.395+0000 7f5e45400640 1 --2- 172.21.15.89:0/3990992391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e40103e30 0x7f5e4019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:00.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.395+0000 7f5e45400640 1 -- 172.21.15.89:0/3990992391 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4019f420 con 0x7f5e401047e0 2024-09-10T08:39:00.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.395+0000 7f5e45400640 1 --2- 172.21.15.89:0/3990992391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e401047e0 0x7f5e4019a9b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000d950 tx=0x7f5e3000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:00.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.395+0000 7f5e3aa00640 1 -- 172.21.15.89:0/3990992391 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30017030 con 0x7f5e401047e0 2024-09-10T08:39:00.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.395+0000 7f5e3aa00640 1 -- 172.21.15.89:0/3990992391 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e30004020 con 0x7f5e401047e0 2024-09-10T08:39:00.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.395+0000 7f5e3aa00640 1 -- 172.21.15.89:0/3990992391 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e300047a0 con 0x7f5e401047e0 2024-09-10T08:39:00.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.395+0000 7f5e47200640 1 -- 172.21.15.89:0/3990992391 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e4019f710 con 0x7f5e401047e0 2024-09-10T08:39:00.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.395+0000 7f5e47200640 1 -- 172.21.15.89:0/3990992391 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e401a6ff0 con 0x7f5e401047e0 2024-09-10T08:39:00.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.397+0000 7f5e47200640 1 -- 172.21.15.89:0/3990992391 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e40105310 con 0x7f5e401047e0 2024-09-10T08:39:00.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.399+0000 7f5e3aa00640 1 -- 172.21.15.89:0/3990992391 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5e300041d0 con 0x7f5e401047e0 2024-09-10T08:39:00.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.400+0000 7f5e3aa00640 1 --2- 172.21.15.89:0/3990992391 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5e280a2c10 0x7f5e280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:00.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.400+0000 7f5e3aa00640 1 -- 172.21.15.89:0/3990992391 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f5e300ceff0 con 0x7f5e401047e0 2024-09-10T08:39:00.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.401+0000 7f5e45e00640 1 --2- 172.21.15.89:0/3990992391 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5e280a2c10 0x7f5e280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:00.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.401+0000 7f5e45e00640 1 --2- 172.21.15.89:0/3990992391 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5e280a2c10 0x7f5e280a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5e34000c00 tx=0x7f5e34005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:00.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.405+0000 7f5e3aa00640 1 -- 172.21.15.89:0/3990992391 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e3009b9c0 con 0x7f5e401047e0 2024-09-10T08:39:00.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.611+0000 7f5e47200640 1 -- 172.21.15.89:0/3990992391 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e4010bbb0 con 0x7f5e401047e0 2024-09-10T08:39:00.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.614+0000 7f5e3aa00640 1 -- 172.21.15.89:0/3990992391 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24894 (secure 0 0 0) 0x7f5e3009b2c0 con 0x7f5e401047e0 2024-09-10T08:39:00.615 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:39:00.615 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":96,"fsid":"91f747bc-6f4e-11ef-bcea-c7b262605968","created":"2024-09-10T08:28:49.697035+0000","modified":"2024-09-10T08:38:43.618497+0000","last_up_change":"2024-09-10T08:36:00.535433+0000","last_in_change":"2024-09-10T08:35:37.131131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:32:02.060714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:36:52.462301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:36:55.119708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:37:03.729002+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d3b55192-bd95-4497-89ea-dc017a25c0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6803","nonce":1760523284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6805","nonce":1760523284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6809","nonce":1760523284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6807","nonce":1760523284}]},"public_addr":"172.21.15.89:6803/1760523284","cluster_addr":"172.21.15.89:6805/1760523284","heartbeat_back_addr":"172.21.15.89:6809/1760523284","heartbeat_front_addr":"172.21.15.89:6807/1760523284","state":["exists","up"]},{"osd":1,"uuid":"60aae117-cb52-4034-a700-11693171cd52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6801","nonce":2369291742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6803","nonce":2369291742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6807","nonce":2369291742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6805","nonce":2369291742}]},"public_addr":"172.21.15.111:6801/2369291742","cluster_addr":"172.21.15.111:6803/2369291742","heartbeat_back_addr":"172.21.15.111:6807/2369291742","heartbeat_front_addr":"172.21.15.111:6805/2369291742","state":["exists","up"]},{"osd":2,"uuid":"a305a019-0870-437c-a298-4f923ec54e77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6801","nonce":1092261395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6803","nonce":1092261395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6807","nonce":1092261395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6805","nonce":1092261395}]},"public_addr":"172.21.15.149:6801/1092261395","cluster_addr":"172.21.15.149:6803/1092261395","heartbeat_back_addr":"172.21.15.149:6807/1092261395","heartbeat_front_addr":"172.21.15.149:6805/1092261395","state":["exists","up"]},{"osd":3,"uuid":"d02f54c1-f883-40f6-9341-8d9fc4b712ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6811","nonce":1405773950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6813","nonce":1405773950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6817","nonce":1405773950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6815","nonce":1405773950}]},"public_addr":"172.21.15.89:6811/1405773950","cluster_addr":"172.21.15.89:6813/1405773950","heartbeat_back_addr":"172.21.15.89:6817/1405773950","heartbeat_front_addr":"172.21.15.89:6815/1405773950","state":["exists","up"]},{"osd":4,"uuid":"23cff46b-88dd-4344-be06-b35db6014f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6809","nonce":316717323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6811","nonce":316717323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6815","nonce":316717323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6813","nonce":316717323}]},"public_addr":"172.21.15.111:6809/316717323","cluster_addr":"172.21.15.111:6811/316717323","heartbeat_back_addr":"172.21.15.111:6815/316717323","heartbeat_front_addr":"172.21.15.111:6813/316717323","state":["exists","up"]},{"osd":5,"uuid":"8e516e8d-e7c6-4ebc-80a5-29932161df4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6809","nonce":1256148461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6811","nonce":1256148461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6815","nonce":1256148461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6813","nonce":1256148461}]},"public_addr":"172.21.15.149:6809/1256148461","cluster_addr":"172.21.15.149:6811/1256148461","heartbeat_back_addr":"172.21.15.149:6815/1256148461","heartbeat_front_addr":"172.21.15.149:6813/1256148461","state":["exists","up"]},{"osd":6,"uuid":"48f92472-f657-4c5a-adeb-20d4fea1ce8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6819","nonce":4190488024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6821","nonce":4190488024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6825","nonce":4190488024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6823","nonce":4190488024}]},"public_addr":"172.21.15.89:6819/4190488024","cluster_addr":"172.21.15.89:6821/4190488024","heartbeat_back_addr":"172.21.15.89:6825/4190488024","heartbeat_front_addr":"172.21.15.89:6823/4190488024","state":["exists","up"]},{"osd":7,"uuid":"14a2a6f0-c4cd-406b-b637-7527fd32babe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6817","nonce":652666376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6819","nonce":652666376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6823","nonce":652666376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6821","nonce":652666376}]},"public_addr":"172.21.15.111:6817/652666376","cluster_addr":"172.21.15.111:6819/652666376","heartbeat_back_addr":"172.21.15.111:6823/652666376","heartbeat_front_addr":"172.21.15.111:6821/652666376","state":["exists","up"]},{"osd":8,"uuid":"a54502e6-95f7-431f-8db1-54a6c7983b5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6817","nonce":2955172869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6819","nonce":2955172869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6823","nonce":2955172869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6821","nonce":2955172869}]},"public_addr":"172.21.15.149:6817/2955172869","cluster_addr":"172.21.15.149:6819/2955172869","heartbeat_back_addr":"172.21.15.149:6823/2955172869","heartbeat_front_addr":"172.21.15.149:6821/2955172869","state":["exists","up"]},{"osd":9,"uuid":"5fe17344-5b1c-4601-a633-639cc1691478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6827","nonce":2427467040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6829","nonce":2427467040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6833","nonce":2427467040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6831","nonce":2427467040}]},"public_addr":"172.21.15.89:6827/2427467040","cluster_addr":"172.21.15.89:6829/2427467040","heartbeat_back_addr":"172.21.15.89:6833/2427467040","heartbeat_front_addr":"172.21.15.89:6831/2427467040","state":["exists","up"]},{"osd":10,"uuid":"d8f29890-3064-4b67-b649-467a0ac86db7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6825","nonce":2328678744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6827","nonce":2328678744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6831","nonce":2328678744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6829","nonce":2328678744}]},"public_addr":"172.21.15.111:6825/2328678744","cluster_addr":"172.21.15.111:6827/2328678744","heartbeat_back_addr":"172.21.15.111:6831/2328678744","heartbeat_front_addr":"172.21.15.111:6829/2328678744","state":["exists","up"]},{"osd":11,"uuid":"92182c73-f139-47ca-a0ef-5046ae9b2310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6825","nonce":3130508827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6827","nonce":3130508827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6831","nonce":3130508827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6829","nonce":3130508827}]},"public_addr":"172.21.15.149:6825/3130508827","cluster_addr":"172.21.15.149:6827/3130508827","heartbeat_back_addr":"172.21.15.149:6831/3130508827","heartbeat_front_addr":"172.21.15.149:6829/3130508827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:09.031810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:33.695124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:58.065510+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:27.592838+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:53.275289+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:18.305677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:47.781535+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:12.889512+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:37.744193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:07.718408+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:33.391027+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:59.013250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/850669510":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/3541027428":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6801/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6800/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/2396468787":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/1609720980":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/995019144":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6801/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6800/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/2179977257":"2024-09-11T08:29:14.554869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:39:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.617+0000 7f5e47200640 1 -- 172.21.15.89:0/3990992391 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5e280a2c10 msgr2=0x7f5e280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.617+0000 7f5e47200640 1 --2- 172.21.15.89:0/3990992391 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5e280a2c10 0x7f5e280a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5e34000c00 tx=0x7f5e34005b50 comp rx=0 tx=0).stop 2024-09-10T08:39:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.617+0000 7f5e47200640 1 -- 172.21.15.89:0/3990992391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e401047e0 msgr2=0x7f5e4019a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.617+0000 7f5e47200640 1 --2- 172.21.15.89:0/3990992391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e401047e0 0x7f5e4019a9b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000d950 tx=0x7f5e3000de10 comp rx=0 tx=0).stop 2024-09-10T08:39:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.617+0000 7f5e47200640 1 -- 172.21.15.89:0/3990992391 shutdown_connections 2024-09-10T08:39:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.617+0000 7f5e47200640 1 --2- 172.21.15.89:0/3990992391 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5e280a2c10 0x7f5e280a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.617+0000 7f5e47200640 1 --2- 172.21.15.89:0/3990992391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e40108f60 0x7f5e4019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.617+0000 7f5e47200640 1 --2- 172.21.15.89:0/3990992391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e401047e0 0x7f5e4019a9b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.617+0000 7f5e47200640 1 --2- 172.21.15.89:0/3990992391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e40103e30 0x7f5e4019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.617+0000 7f5e47200640 1 -- 172.21.15.89:0/3990992391 >> 172.21.15.89:0/3990992391 conn(0x7f5e400ff910 msgr2=0x7f5e40100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:00.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.617+0000 7f5e47200640 1 -- 172.21.15.89:0/3990992391 shutdown_connections 2024-09-10T08:39:00.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:00.617+0000 7f5e47200640 1 -- 172.21.15.89:0/3990992391 wait complete. 2024-09-10T08:39:01.441 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:39:01.821 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:01.961 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:01 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:01.961 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:01 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:01.961 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:01 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3990992391' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:39:01.961 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:01 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:01.961 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:01 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:01.961 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:01 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:01.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:01 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:01.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:01 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:01.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:01 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3990992391' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:39:01.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:01 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:01.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:01 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:01.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:01 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:02.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:01 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:02.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:01 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:02.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:01 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3990992391' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:39:02.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:01 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:02.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:01 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:02.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:01 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:39:02.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.398+0000 7f0530200640 1 -- 172.21.15.89:0/774029612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0528106f10 msgr2=0x7f052810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:02.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.398+0000 7f0530200640 1 --2- 172.21.15.89:0/774029612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0528106f10 0x7f052810ea90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0520009ca0 tx=0x7f052002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:39:02.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.398+0000 7f0530200640 1 -- 172.21.15.89:0/774029612 shutdown_connections 2024-09-10T08:39:02.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.398+0000 7f0530200640 1 --2- 172.21.15.89:0/774029612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0528106f10 0x7f052810ea90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:02.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.398+0000 7f0530200640 1 --2- 172.21.15.89:0/774029612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0528102790 0x7f05281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:02.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.398+0000 7f0530200640 1 --2- 172.21.15.89:0/774029612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0528101de0 0x7f05281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:02.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.398+0000 7f0530200640 1 -- 172.21.15.89:0/774029612 >> 172.21.15.89:0/774029612 conn(0x7f05280fd8c0 msgr2=0x7f05280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:02.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.398+0000 7f0530200640 1 -- 172.21.15.89:0/774029612 shutdown_connections 2024-09-10T08:39:02.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.398+0000 7f0530200640 1 -- 172.21.15.89:0/774029612 wait complete. 2024-09-10T08:39:02.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.400+0000 7f0530200640 1 Processor -- start 2024-09-10T08:39:02.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.400+0000 7f0530200640 1 -- start start 2024-09-10T08:39:02.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.400+0000 7f0530200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0528101de0 0x7f052819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:02.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.400+0000 7f0530200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0528102790 0x7f052819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:02.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.400+0000 7f0530200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0528106f10 0x7f052819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:02.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.400+0000 7f0530200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0528111f70 con 0x7f0528106f10 2024-09-10T08:39:02.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.400+0000 7f0530200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0528111df0 con 0x7f0528102790 2024-09-10T08:39:02.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.400+0000 7f0530200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f05281120f0 con 0x7f0528101de0 2024-09-10T08:39:02.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.401+0000 7f052d800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0528101de0 0x7f052819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:02.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.401+0000 7f052d800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0528101de0 0x7f052819a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:49666/0 (socket says 172.21.15.89:49666) 2024-09-10T08:39:02.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.401+0000 7f052d800640 1 -- 172.21.15.89:0/1209696144 learned_addr learned my addr 172.21.15.89:0/1209696144 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:02.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.401+0000 7f052ce00640 1 --2- 172.21.15.89:0/1209696144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0528102790 0x7f052819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:02.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.401+0000 7f052e200640 1 --2- 172.21.15.89:0/1209696144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0528106f10 0x7f052819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:02.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.401+0000 7f052d800640 1 -- 172.21.15.89:0/1209696144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0528102790 msgr2=0x7f052819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:02.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.401+0000 7f052d800640 1 --2- 172.21.15.89:0/1209696144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0528102790 0x7f052819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:02.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.401+0000 7f052d800640 1 -- 172.21.15.89:0/1209696144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0528106f10 msgr2=0x7f052819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:02.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.401+0000 7f052d800640 1 --2- 172.21.15.89:0/1209696144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0528106f10 0x7f052819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:02.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.401+0000 7f052d800640 1 -- 172.21.15.89:0/1209696144 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052819f610 con 0x7f0528101de0 2024-09-10T08:39:02.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.401+0000 7f052e200640 1 --2- 172.21.15.89:0/1209696144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0528106f10 0x7f052819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:02.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.402+0000 7f052d800640 1 --2- 172.21.15.89:0/1209696144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0528101de0 0x7f052819a660 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f051c00e9f0 tx=0x7f051c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:02.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.402+0000 7f0526000640 1 -- 172.21.15.89:0/1209696144 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c00cd80 con 0x7f0528101de0 2024-09-10T08:39:02.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.402+0000 7f0530200640 1 -- 172.21.15.89:0/1209696144 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f052819f900 con 0x7f0528101de0 2024-09-10T08:39:02.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.402+0000 7f0526000640 1 -- 172.21.15.89:0/1209696144 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f051c005500 con 0x7f0528101de0 2024-09-10T08:39:02.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.402+0000 7f0526000640 1 -- 172.21.15.89:0/1209696144 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c010690 con 0x7f0528101de0 2024-09-10T08:39:02.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.402+0000 7f0530200640 1 -- 172.21.15.89:0/1209696144 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05281a71e0 con 0x7f0528101de0 2024-09-10T08:39:02.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.403+0000 7f0530200640 1 -- 172.21.15.89:0/1209696144 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05281021c0 con 0x7f0528101de0 2024-09-10T08:39:02.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.407+0000 7f0526000640 1 -- 172.21.15.89:0/1209696144 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f051c0046c0 con 0x7f0528101de0 2024-09-10T08:39:02.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.408+0000 7f0526000640 1 --2- 172.21.15.89:0/1209696144 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f05100a2c10 0x7f05100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:02.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.408+0000 7f0526000640 1 -- 172.21.15.89:0/1209696144 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f051c0cf9e0 con 0x7f0528101de0 2024-09-10T08:39:02.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.408+0000 7f052ce00640 1 --2- 172.21.15.89:0/1209696144 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f05100a2c10 0x7f05100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:02.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.409+0000 7f052ce00640 1 --2- 172.21.15.89:0/1209696144 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f05100a2c10 0x7f05100a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f052810eed0 tx=0x7f051400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:02.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.412+0000 7f0526000640 1 -- 172.21.15.89:0/1209696144 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f051c09c3b0 con 0x7f0528101de0 2024-09-10T08:39:02.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.631+0000 7f0530200640 1 -- 172.21.15.89:0/1209696144 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05281a7410 con 0x7f0528101de0 2024-09-10T08:39:02.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.632+0000 7f0526000640 1 -- 172.21.15.89:0/1209696144 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11918 (secure 0 0 0) 0x7f051c09bcb0 con 0x7f0528101de0 2024-09-10T08:39:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:39:02.633 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":25,"btime":"2024-09-10T08:37:50:718359+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:37:50.607307+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24496,24518,24554]},"id":1}]} 2024-09-10T08:39:02.633 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 25 2024-09-10T08:39:02.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.634+0000 7f0530200640 1 -- 172.21.15.89:0/1209696144 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f05100a2c10 msgr2=0x7f05100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:02.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.634+0000 7f0530200640 1 --2- 172.21.15.89:0/1209696144 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f05100a2c10 0x7f05100a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f052810eed0 tx=0x7f051400b410 comp rx=0 tx=0).stop 2024-09-10T08:39:02.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.634+0000 7f0530200640 1 -- 172.21.15.89:0/1209696144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0528101de0 msgr2=0x7f052819a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:02.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.634+0000 7f0530200640 1 --2- 172.21.15.89:0/1209696144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0528101de0 0x7f052819a660 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f051c00e9f0 tx=0x7f051c00edf0 comp rx=0 tx=0).stop 2024-09-10T08:39:02.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.634+0000 7f0530200640 1 -- 172.21.15.89:0/1209696144 shutdown_connections 2024-09-10T08:39:02.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.634+0000 7f0530200640 1 --2- 172.21.15.89:0/1209696144 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f05100a2c10 0x7f05100a50d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:02.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.634+0000 7f0530200640 1 --2- 172.21.15.89:0/1209696144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0528106f10 0x7f052819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:02.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.634+0000 7f0530200640 1 --2- 172.21.15.89:0/1209696144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0528102790 0x7f052819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:02.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.634+0000 7f0530200640 1 --2- 172.21.15.89:0/1209696144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0528101de0 0x7f052819a660 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:02.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.634+0000 7f0530200640 1 -- 172.21.15.89:0/1209696144 >> 172.21.15.89:0/1209696144 conn(0x7f05280fd8c0 msgr2=0x7f05280fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:02.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.634+0000 7f0530200640 1 -- 172.21.15.89:0/1209696144 shutdown_connections 2024-09-10T08:39:02.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:02.634+0000 7f0530200640 1 -- 172.21.15.89:0/1209696144 wait complete. 2024-09-10T08:39:02.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:02 smithi149 ceph-mon[36167]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T08:39:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:02 smithi111 ceph-mon[35220]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T08:39:03.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:02 smithi089 ceph-mon[30757]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T08:39:03.435 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T08:36:57.162188+0000', 'modified': '2024-09-10T08:37:50.607307+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24554, 'mds_1': 24518, 'mds_2': 24496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24472': {'gid': 24472, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6835/881263752', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6834', 'nonce': 881263752}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 881263752}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6837/2367754626', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 2367754626}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 2367754626}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6835/285258866', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 285258866}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 285258866}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.149:6839/2020874598', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 2020874598}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 2020874598}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.111:6837/779590543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6836', 'nonce': 779590543}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 779590543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24554': {'gid': 24554, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.111:6839/3640096926', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 3640096926}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 3640096926}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24554, 'qdb_cluster': [24496, 24518, 24554]} 2024-09-10T08:39:03.435 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-10T08:39:03.435 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T08:39:03.678 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:03 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1209696144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:39:03.813 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:03.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:03 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1209696144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:39:04.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:03 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1209696144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:39:04.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.405+0000 7f23abe00640 1 -- 172.21.15.89:0/880306028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23ac108f60 msgr2=0x7f23ac110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:04.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.405+0000 7f23abe00640 1 --2- 172.21.15.89:0/880306028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23ac108f60 0x7f23ac110ae0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f23a4009be0 tx=0x7f23a402a6c0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.405+0000 7f23abe00640 1 -- 172.21.15.89:0/880306028 shutdown_connections 2024-09-10T08:39:04.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.405+0000 7f23abe00640 1 --2- 172.21.15.89:0/880306028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23ac108f60 0x7f23ac110ae0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.405+0000 7f23abe00640 1 --2- 172.21.15.89:0/880306028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23ac1047e0 0x7f23ac108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.405+0000 7f23abe00640 1 --2- 172.21.15.89:0/880306028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23ac103e30 0x7f23ac104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.405+0000 7f23abe00640 1 -- 172.21.15.89:0/880306028 >> 172.21.15.89:0/880306028 conn(0x7f23ac0ff8d0 msgr2=0x7f23ac101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:04.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.405+0000 7f23abe00640 1 -- 172.21.15.89:0/880306028 shutdown_connections 2024-09-10T08:39:04.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.405+0000 7f23abe00640 1 -- 172.21.15.89:0/880306028 wait complete. 2024-09-10T08:39:04.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.406+0000 7f23abe00640 1 Processor -- start 2024-09-10T08:39:04.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.406+0000 7f23abe00640 1 -- start start 2024-09-10T08:39:04.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.407+0000 7f23abe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23ac103e30 0x7f23ac19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:04.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.407+0000 7f23abe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23ac1047e0 0x7f23ac19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:04.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.407+0000 7f23abe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23ac108f60 0x7f23ac19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:04.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.407+0000 7f23abe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac113fc0 con 0x7f23ac108f60 2024-09-10T08:39:04.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.407+0000 7f23abe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac113e40 con 0x7f23ac1047e0 2024-09-10T08:39:04.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.407+0000 7f23abe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac114140 con 0x7f23ac103e30 2024-09-10T08:39:04.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.407+0000 7f23aaa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23ac103e30 0x7f23ac19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:04.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.407+0000 7f23ab400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23ac108f60 0x7f23ac19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:04.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.407+0000 7f23aaa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23ac103e30 0x7f23ac19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:42490/0 (socket says 172.21.15.89:42490) 2024-09-10T08:39:04.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.407+0000 7f23aaa00640 1 -- 172.21.15.89:0/2494441039 learned_addr learned my addr 172.21.15.89:0/2494441039 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:04.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.407+0000 7f23aa000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23ac1047e0 0x7f23ac19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:04.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.407+0000 7f23aaa00640 1 -- 172.21.15.89:0/2494441039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23ac1047e0 msgr2=0x7f23ac19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:04.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.407+0000 7f23aaa00640 1 --2- 172.21.15.89:0/2494441039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23ac1047e0 0x7f23ac19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.407+0000 7f23aaa00640 1 -- 172.21.15.89:0/2494441039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23ac108f60 msgr2=0x7f23ac19ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:04.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.407+0000 7f23aaa00640 1 --2- 172.21.15.89:0/2494441039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23ac108f60 0x7f23ac19ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.407+0000 7f23aaa00640 1 -- 172.21.15.89:0/2494441039 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ac19f3b0 con 0x7f23ac103e30 2024-09-10T08:39:04.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.408+0000 7f23aaa00640 1 --2- 172.21.15.89:0/2494441039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23ac103e30 0x7f23ac19a400 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f239c00e9c0 tx=0x7f239c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:04.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.408+0000 7f23b0a00640 1 -- 172.21.15.89:0/2494441039 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c00cde0 con 0x7f23ac103e30 2024-09-10T08:39:04.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.408+0000 7f23abe00640 1 -- 172.21.15.89:0/2494441039 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ac19f6a0 con 0x7f23ac103e30 2024-09-10T08:39:04.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.408+0000 7f23b0a00640 1 -- 172.21.15.89:0/2494441039 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f239c0035f0 con 0x7f23ac103e30 2024-09-10T08:39:04.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.408+0000 7f23b0a00640 1 -- 172.21.15.89:0/2494441039 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c010650 con 0x7f23ac103e30 2024-09-10T08:39:04.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.408+0000 7f23abe00640 1 -- 172.21.15.89:0/2494441039 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ac1a6f80 con 0x7f23ac103e30 2024-09-10T08:39:04.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.411+0000 7f23b4a6d740 1 Processor -- start 2024-09-10T08:39:04.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.411+0000 7f23b0a00640 1 -- 172.21.15.89:0/2494441039 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f239c0037a0 con 0x7f23ac103e30 2024-09-10T08:39:04.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.411+0000 7f23b4a6d740 1 -- start start 2024-09-10T08:39:04.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.412+0000 7f23b4a6d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6274e0 0x56327a6278c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:04.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.412+0000 7f23b4a6d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 0x56327a632180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:04.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.412+0000 7f23b4a6d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6326c0 0x56327a634ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:04.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.412+0000 7f23b4a6d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56327a58d0b0 con 0x56327a627e00 2024-09-10T08:39:04.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.412+0000 7f23b4a6d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56327a534ab0 con 0x56327a6326c0 2024-09-10T08:39:04.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.412+0000 7f23b4a6d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56327a589070 con 0x56327a6274e0 2024-09-10T08:39:04.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.412+0000 7f23aa000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6326c0 0x56327a634ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:04.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.412+0000 7f23aa000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6326c0 0x56327a634ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:45034/0 (socket says 172.21.15.89:45034) 2024-09-10T08:39:04.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.412+0000 7f23aa000640 1 -- 172.21.15.89:0/1839870684 learned_addr learned my addr 172.21.15.89:0/1839870684 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:04.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.412+0000 7f23ab400640 1 --2- 172.21.15.89:0/1839870684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6274e0 0x56327a6278c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:04.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.412+0000 7f23aaa00640 1 --2- 172.21.15.89:0/1839870684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 0x56327a632180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:04.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.413+0000 7f23aa000640 1 -- 172.21.15.89:0/1839870684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6274e0 msgr2=0x56327a6278c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:04.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.413+0000 7f23aa000640 1 --2- 172.21.15.89:0/1839870684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6274e0 0x56327a6278c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.413+0000 7f23aa000640 1 -- 172.21.15.89:0/1839870684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 msgr2=0x56327a632180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:04.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.413+0000 7f23aa000640 1 --2- 172.21.15.89:0/1839870684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 0x56327a632180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.413 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.413+0000 7f23aa000640 1 -- 172.21.15.89:0/1839870684 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56327a5414a0 con 0x56327a6326c0 2024-09-10T08:39:04.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.413+0000 7f23aa000640 1 --2- 172.21.15.89:0/1839870684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6326c0 0x56327a634ab0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2398009c20 tx=0x7f2398011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdfdf06a7d716212 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:04.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.413+0000 7f23b0a00640 1 --2- 172.21.15.89:0/2494441039 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23900a2c10 0x7f23900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:04.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.413+0000 7f2397e00640 1 -- 172.21.15.89:0/1839870684 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398004230 con 0x56327a6326c0 2024-09-10T08:39:04.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.413+0000 7f2397e00640 1 -- 172.21.15.89:0/1839870684 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23980043d0 con 0x56327a6326c0 2024-09-10T08:39:04.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.413+0000 7f2397e00640 1 -- 172.21.15.89:0/1839870684 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398004d60 con 0x56327a6326c0 2024-09-10T08:39:04.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.413+0000 7f23b0a00640 1 -- 172.21.15.89:0/2494441039 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f239c0cf8c0 con 0x7f23ac103e30 2024-09-10T08:39:04.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.413+0000 7f23ab400640 1 --2- 172.21.15.89:0/2494441039 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23900a2c10 0x7f23900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:04.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.414+0000 7f23b4a6d740 1 -- 172.21.15.89:0/1839870684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6326c0 msgr2=0x56327a634ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:04.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.414+0000 7f23b4a6d740 1 --2- 172.21.15.89:0/1839870684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6326c0 0x56327a634ab0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2398009c20 tx=0x7f2398011810 comp rx=0 tx=0).stop 2024-09-10T08:39:04.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.414+0000 7f23b4a6d740 1 -- 172.21.15.89:0/1839870684 shutdown_connections 2024-09-10T08:39:04.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.414+0000 7f23b4a6d740 1 --2- 172.21.15.89:0/1839870684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6326c0 0x56327a634ab0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.414+0000 7f23b4a6d740 1 --2- 172.21.15.89:0/1839870684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 0x56327a632180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.414+0000 7f23b4a6d740 1 --2- 172.21.15.89:0/1839870684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6274e0 0x56327a6278c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.414+0000 7f23b4a6d740 1 -- 172.21.15.89:0/1839870684 >> 172.21.15.89:0/1839870684 conn(0x56327a623a70 msgr2=0x56327a623e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:04.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.414+0000 7f23ab400640 1 --2- 172.21.15.89:0/2494441039 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23900a2c10 0x7f23900a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f23a4004af0 tx=0x7f23a4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:04.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.414+0000 7f23b4a6d740 1 -- 172.21.15.89:0/1839870684 shutdown_connections 2024-09-10T08:39:04.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.414+0000 7f23b4a6d740 1 -- 172.21.15.89:0/1839870684 wait complete. 2024-09-10T08:39:04.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.415+0000 7f23b4a6d740 1 -- start start 2024-09-10T08:39:04.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.415+0000 7f23b4a6d740 1 Processor -- start 2024-09-10T08:39:04.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.415+0000 7f23b4a6d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6274e0 0x56327a62f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:04.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.415+0000 7f23b4a6d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 0x56327a62fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:04.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.415+0000 7f23b4a6d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6326c0 0x56327a62ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:04.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.415+0000 7f23b4a6d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56327a635330 con 0x56327a627e00 2024-09-10T08:39:04.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.415+0000 7f23b4a6d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56327a515920 con 0x56327a6274e0 2024-09-10T08:39:04.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.415+0000 7f23b4a6d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56327a6354b0 con 0x56327a6326c0 2024-09-10T08:39:04.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.416+0000 7f23b4a6d740 10 client.0 _ll_register_callbacks cb 0x56327a5398a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:39:04.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.416+0000 7f23aaa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6274e0 0x56327a62f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:04.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.416+0000 7f23aaa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6274e0 0x56327a62f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:45044/0 (socket says 172.21.15.89:45044) 2024-09-10T08:39:04.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.416+0000 7f23aaa00640 1 -- 172.21.15.89:0/1389285201 learned_addr learned my addr 172.21.15.89:0/1389285201 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:04.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.416+0000 7f23aa000640 1 --2- 172.21.15.89:0/1389285201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 0x56327a62fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:04.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.416+0000 7f23ab400640 1 --2- 172.21.15.89:0/1389285201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6326c0 0x56327a62ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:04.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.416+0000 7f23aaa00640 1 -- 172.21.15.89:0/1389285201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6326c0 msgr2=0x56327a62ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:04.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.416+0000 7f23aaa00640 1 --2- 172.21.15.89:0/1389285201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6326c0 0x56327a62ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.416+0000 7f23aaa00640 1 -- 172.21.15.89:0/1389285201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 msgr2=0x56327a62fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:04.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.416+0000 7f23aaa00640 1 --2- 172.21.15.89:0/1389285201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 0x56327a62fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.416+0000 7f23aaa00640 1 -- 172.21.15.89:0/1389285201 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56327a2fe2b0 con 0x56327a6274e0 2024-09-10T08:39:04.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.416+0000 7f23ab400640 1 --2- 172.21.15.89:0/1389285201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6326c0 0x56327a62ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:39:04.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.416+0000 7f23aaa00640 1 --2- 172.21.15.89:0/1389285201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6274e0 0x56327a62f550 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f239c09b8f0 tx=0x7f239c09b5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eb59b970f1e43eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:04.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.416+0000 7f2394c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:39:04.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.417+0000 7f2394c00640 1 -- 172.21.15.89:0/1389285201 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c09ffa0 con 0x56327a6274e0 2024-09-10T08:39:04.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.417+0000 7f2394c00640 1 -- 172.21.15.89:0/1389285201 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f239c09b040 con 0x56327a6274e0 2024-09-10T08:39:04.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.417+0000 7f23b4a6d740 1 -- 172.21.15.89:0/1389285201 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23980043d0 con 0x56327a6274e0 2024-09-10T08:39:04.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.417+0000 7f2394c00640 1 -- 172.21.15.89:0/1389285201 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f239c0a4860 con 0x56327a6274e0 2024-09-10T08:39:04.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.417+0000 7f23b4a6d740 10 client.24992 fetch_fsmap learned FSMap version 25 2024-09-10T08:39:04.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.417+0000 7f23b4a6d740 1 -- 172.21.15.89:0/1389285201 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56327a640f70 con 0x56327a6274e0 2024-09-10T08:39:04.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.417+0000 7f2394c00640 1 -- 172.21.15.89:0/1389285201 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23980043d0 con 0x56327a6274e0 2024-09-10T08:39:04.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.417+0000 7f2394c00640 1 -- 172.21.15.89:0/1389285201 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f239c0a1130 con 0x56327a6274e0 2024-09-10T08:39:04.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.417+0000 7f23b4a6d740 10 client.24992 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:39:04.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.417+0000 7f23b4a6d740 10 client.24992 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T08:39:04.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.418+0000 7f23b4a6d740 20 client.24992 populate_metadata read hostname 'smithi089' 2024-09-10T08:39:04.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.418+0000 7f23b4a6d740 1 --2- 172.21.15.89:0/1389285201 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x56327a643550 0x56327a6449c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:04.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.418+0000 7f23b4a6d740 4 client.24992 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:39:04.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.418+0000 7f23b4a6d740 1 -- 172.21.15.89:0/1389285201 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56327a59a620 con 0x56327a643550 2024-09-10T08:39:04.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.418+0000 7f23aa000640 1 --2- 172.21.15.89:0/1389285201 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x56327a643550 0x56327a6449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:04.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.419+0000 7f23aa000640 1 --2- 172.21.15.89:0/1389285201 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x56327a643550 0x56327a6449c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eeda5eaac994a592 server_cookie=6bbeb7a93d1e489d in_seq=0 out_seq=0 2024-09-10T08:39:04.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.419+0000 7f2394c00640 10 client.24992 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:39:04.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.420+0000 7f2394c00640 1 -- 172.21.15.89:0/1389285201 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56327a59a620 con 0x56327a643550 2024-09-10T08:39:04.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.420+0000 7f2394c00640 10 client.24992 handle_command_reply: tid=0 2024-09-10T08:39:04.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.420+0000 7f23b4a6d740 1 client.24992 shutdown 2024-09-10T08:39:04.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.420+0000 7f23b4a6d740 1 -- 172.21.15.89:0/1389285201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6274e0 msgr2=0x56327a62f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:04.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.420+0000 7f23b4a6d740 1 --2- 172.21.15.89:0/1389285201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6274e0 0x56327a62f550 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f239c09b8f0 tx=0x7f239c09b5d0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.420+0000 7f23b4a6d740 1 -- 172.21.15.89:0/1389285201 shutdown_connections 2024-09-10T08:39:04.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.420+0000 7f23b4a6d740 1 --2- 172.21.15.89:0/1389285201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6326c0 0x56327a62ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.420+0000 7f23b4a6d740 1 --2- 172.21.15.89:0/1389285201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 0x56327a62fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.420+0000 7f23b4a6d740 1 --2- 172.21.15.89:0/1389285201 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x56327a643550 0x56327a6449c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.420+0000 7f23b4a6d740 1 --2- 172.21.15.89:0/1389285201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6274e0 0x56327a62f550 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.420+0000 7f2394c00640 0 client.24992 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:39:04.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.420+0000 7f23b4a6d740 1 -- 172.21.15.89:0/1389285201 >> 172.21.15.89:0/1389285201 conn(0x56327a623a70 msgr2=0x56327a6297a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:04.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.420+0000 7f23b4a6d740 1 -- 172.21.15.89:0/1389285201 shutdown_connections 2024-09-10T08:39:04.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.420+0000 7f23b4a6d740 1 -- 172.21.15.89:0/1389285201 wait complete. 2024-09-10T08:39:04.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.420+0000 7f23b4a6d740 20 client.24992 trim_cache size 0 max 16384 2024-09-10T08:39:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.439+0000 7f23b4a6d740 1 Processor -- start 2024-09-10T08:39:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.439+0000 7f23b4a6d740 1 -- start start 2024-09-10T08:39:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.439+0000 7f23b4a6d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6274e0 0x56327a642e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.439+0000 7f23b4a6d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 0x56327a62ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.439+0000 7f23b4a6d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6326c0 0x56327a62c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.439+0000 7f23b4a6d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56327a642d00 con 0x56327a627e00 2024-09-10T08:39:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.439+0000 7f23b4a6d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56327a6433d0 con 0x56327a6274e0 2024-09-10T08:39:04.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.439+0000 7f23b4a6d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56327a643aa0 con 0x56327a6326c0 2024-09-10T08:39:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.439+0000 7f23aa000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 0x56327a62ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.439+0000 7f23aa000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 0x56327a62ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52112/0 (socket says 172.21.15.89:52112) 2024-09-10T08:39:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.439+0000 7f23aa000640 1 -- 172.21.15.89:0/3507182770 learned_addr learned my addr 172.21.15.89:0/3507182770 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.440+0000 7f23aaa00640 1 --2- 172.21.15.89:0/3507182770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6274e0 0x56327a642e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.440+0000 7f23ab400640 1 --2- 172.21.15.89:0/3507182770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6326c0 0x56327a62c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.440+0000 7f23aa000640 1 -- 172.21.15.89:0/3507182770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6326c0 msgr2=0x56327a62c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.440+0000 7f23aa000640 1 --2- 172.21.15.89:0/3507182770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6326c0 0x56327a62c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.440+0000 7f23aa000640 1 -- 172.21.15.89:0/3507182770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6274e0 msgr2=0x56327a642e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:04.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.440+0000 7f23aa000640 1 --2- 172.21.15.89:0/3507182770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6274e0 0x56327a642e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.440+0000 7f23aa000640 1 -- 172.21.15.89:0/3507182770 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56327a6238a0 con 0x56327a627e00 2024-09-10T08:39:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.440+0000 7f23aaa00640 1 --2- 172.21.15.89:0/3507182770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6274e0 0x56327a642e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:39:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.440+0000 7f23aa000640 1 --2- 172.21.15.89:0/3507182770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 0x56327a62ea90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2398009c20 tx=0x7f2398004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89abd6143a3a768a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.440+0000 7f2397400640 1 -- 172.21.15.89:0/3507182770 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239801e040 con 0x56327a627e00 2024-09-10T08:39:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.440+0000 7f2397400640 1 -- 172.21.15.89:0/3507182770 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f239802a070 con 0x56327a627e00 2024-09-10T08:39:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.440+0000 7f2397400640 1 -- 172.21.15.89:0/3507182770 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23980306d0 con 0x56327a627e00 2024-09-10T08:39:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.441+0000 7f23b4a6d740 1 -- 172.21.15.89:0/3507182770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 msgr2=0x56327a62ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.441+0000 7f23b4a6d740 1 --2- 172.21.15.89:0/3507182770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 0x56327a62ea90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2398009c20 tx=0x7f2398004580 comp rx=0 tx=0).stop 2024-09-10T08:39:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.441+0000 7f23b4a6d740 1 -- 172.21.15.89:0/3507182770 shutdown_connections 2024-09-10T08:39:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.441+0000 7f23b4a6d740 1 --2- 172.21.15.89:0/3507182770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6326c0 0x56327a62c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.441+0000 7f23b4a6d740 1 --2- 172.21.15.89:0/3507182770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 0x56327a62ea90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.441+0000 7f23b4a6d740 1 --2- 172.21.15.89:0/3507182770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6274e0 0x56327a642e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.441+0000 7f23b4a6d740 1 -- 172.21.15.89:0/3507182770 >> 172.21.15.89:0/3507182770 conn(0x56327a623a70 msgr2=0x56327a62de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:04.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.441+0000 7f23b4a6d740 1 -- 172.21.15.89:0/3507182770 shutdown_connections 2024-09-10T08:39:04.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.441+0000 7f23b4a6d740 1 -- 172.21.15.89:0/3507182770 wait complete. 2024-09-10T08:39:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.442+0000 7f23b4a6d740 1 -- start start 2024-09-10T08:39:04.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.442+0000 7f23b4a6d740 1 Processor -- start 2024-09-10T08:39:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.442+0000 7f23b4a6d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6274e0 0x56327a646a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.442+0000 7f23b4a6d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 0x56327a646fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.442+0000 7f23b4a6d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6326c0 0x56327a6474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.442+0000 7f23b4a6d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56327a644840 con 0x56327a627e00 2024-09-10T08:39:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.442+0000 7f23b4a6d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56327a644170 con 0x56327a6274e0 2024-09-10T08:39:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.442+0000 7f23b4a6d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56327a644f10 con 0x56327a6326c0 2024-09-10T08:39:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.442+0000 7f23b4a6d740 10 client.0 _ll_register_callbacks cb 0x7f239c09b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:39:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.443+0000 7f23aa000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 0x56327a646fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.443+0000 7f23aaa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6274e0 0x56327a646a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.443+0000 7f23aaa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6274e0 0x56327a646a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:45066/0 (socket says 172.21.15.89:45066) 2024-09-10T08:39:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.443+0000 7f23aa000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 0x56327a646fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52126/0 (socket says 172.21.15.89:52126) 2024-09-10T08:39:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.443+0000 7f23aaa00640 1 -- 172.21.15.89:0/505097694 learned_addr learned my addr 172.21.15.89:0/505097694 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:04.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.443+0000 7f23ab400640 1 --2- 172.21.15.89:0/505097694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6326c0 0x56327a6474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.443+0000 7f23aaa00640 1 -- 172.21.15.89:0/505097694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6326c0 msgr2=0x56327a6474f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.443+0000 7f23aaa00640 1 --2- 172.21.15.89:0/505097694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6326c0 0x56327a6474f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.443+0000 7f23aaa00640 1 -- 172.21.15.89:0/505097694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 msgr2=0x56327a646fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.443+0000 7f23aaa00640 1 --2- 172.21.15.89:0/505097694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 0x56327a646fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.443+0000 7f23aaa00640 1 -- 172.21.15.89:0/505097694 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56327a64b2e0 con 0x56327a6274e0 2024-09-10T08:39:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.443+0000 7f23aa000640 1 --2- 172.21.15.89:0/505097694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 0x56327a646fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:39:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.443+0000 7f23aaa00640 1 --2- 172.21.15.89:0/505097694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6274e0 0x56327a646a70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f239c0ccf20 tx=0x7f239c0092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45af81dd7431309b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.443+0000 7f2394c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:39:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.443+0000 7f2394c00640 1 -- 172.21.15.89:0/505097694 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c086580 con 0x56327a6274e0 2024-09-10T08:39:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.444+0000 7f23b4a6d740 1 -- 172.21.15.89:0/505097694 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f239802a070 con 0x56327a6274e0 2024-09-10T08:39:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.444+0000 7f2394c00640 1 -- 172.21.15.89:0/505097694 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f239c09b040 con 0x56327a6274e0 2024-09-10T08:39:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.444+0000 7f2394c00640 1 -- 172.21.15.89:0/505097694 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f239c09ffa0 con 0x56327a6274e0 2024-09-10T08:39:04.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.444+0000 7f23b4a6d740 10 client.24998 fetch_fsmap learned FSMap version 25 2024-09-10T08:39:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.444+0000 7f23b4a6d740 1 -- 172.21.15.89:0/505097694 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56327a6adfe0 con 0x56327a6274e0 2024-09-10T08:39:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.444+0000 7f2394c00640 1 -- 172.21.15.89:0/505097694 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f239802a070 con 0x56327a6274e0 2024-09-10T08:39:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.444+0000 7f2394c00640 1 -- 172.21.15.89:0/505097694 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f239c097b40 con 0x56327a6274e0 2024-09-10T08:39:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.444+0000 7f23b4a6d740 10 client.24998 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:39:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.444+0000 7f23b4a6d740 10 client.24998 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T08:39:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.444+0000 7f23b4a6d740 20 client.24998 populate_metadata read hostname 'smithi089' 2024-09-10T08:39:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.444+0000 7f23b4a6d740 1 --2- 172.21.15.89:0/505097694 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x56327a6b2a10 0x56327a6b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.444+0000 7f23b4a6d740 4 client.24998 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:39:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.444+0000 7f23b4a6d740 1 -- 172.21.15.89:0/505097694 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "status"}) -- 0x56327a594b30 con 0x56327a6b2a10 2024-09-10T08:39:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.445+0000 7f23aa000640 1 --2- 172.21.15.89:0/505097694 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x56327a6b2a10 0x56327a6b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.445+0000 7f23aa000640 1 --2- 172.21.15.89:0/505097694 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x56327a6b2a10 0x56327a6b1720 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2953321104abd99a server_cookie=3b5f88a38c89e9f6 in_seq=0 out_seq=0 2024-09-10T08:39:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.445+0000 7f2394c00640 10 client.24998 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:39:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.445+0000 7f2394c00640 1 -- 172.21.15.89:0/505097694 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56327a594b30 con 0x56327a6b2a10 2024-09-10T08:39:04.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.445+0000 7f2394c00640 10 client.24998 handle_command_reply: tid=0 2024-09-10T08:39:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.445+0000 7f23b4a6d740 1 client.24998 shutdown 2024-09-10T08:39:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.446+0000 7f23b4a6d740 1 -- 172.21.15.89:0/505097694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6274e0 msgr2=0x56327a646a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.446+0000 7f23b4a6d740 1 --2- 172.21.15.89:0/505097694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6274e0 0x56327a646a70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f239c0ccf20 tx=0x7f239c0092d0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.446+0000 7f23b4a6d740 1 -- 172.21.15.89:0/505097694 shutdown_connections 2024-09-10T08:39:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.446+0000 7f23b4a6d740 1 --2- 172.21.15.89:0/505097694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56327a6326c0 0x56327a6474f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.446+0000 7f23b4a6d740 1 --2- 172.21.15.89:0/505097694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56327a627e00 0x56327a646fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.446+0000 7f23b4a6d740 1 --2- 172.21.15.89:0/505097694 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x56327a6b2a10 0x56327a6b1720 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.446+0000 7f23b4a6d740 1 --2- 172.21.15.89:0/505097694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56327a6274e0 0x56327a646a70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.446+0000 7f2394c00640 0 client.24998 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:39:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.446+0000 7f23b4a6d740 1 -- 172.21.15.89:0/505097694 >> 172.21.15.89:0/505097694 conn(0x56327a623a70 msgr2=0x56327a63df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.446+0000 7f23b4a6d740 1 -- 172.21.15.89:0/505097694 shutdown_connections 2024-09-10T08:39:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.446+0000 7f23b4a6d740 1 -- 172.21.15.89:0/505097694 wait complete. 2024-09-10T08:39:04.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.446+0000 7f23b4a6d740 20 client.24998 trim_cache size 0 max 16384 2024-09-10T08:39:04.450 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:39:04.450 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "91f747bc-6f4e-11ef-bcea-c7b262605968", 2024-09-10T08:39:04.450 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 2, 2024-09-10T08:39:04.450 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24496, 2024-09-10T08:39:04.450 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2024-09-10T08:39:04.450 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2024-09-10T08:39:04.450 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2024-09-10T08:39:04.451 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 76.740551280999995, 2024-09-10T08:39:04.451 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 25, 2024-09-10T08:39:04.451 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 96, 2024-09-10T08:39:04.451 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T08:39:04.451 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 148.978632334 2024-09-10T08:39:04.451 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:39:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.447+0000 7f23abe00640 1 -- 172.21.15.89:0/2494441039 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23900a2c10 msgr2=0x7f23900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:04.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.447+0000 7f23abe00640 1 --2- 172.21.15.89:0/2494441039 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23900a2c10 0x7f23900a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f23a4004af0 tx=0x7f23a4005fd0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.447+0000 7f23abe00640 1 -- 172.21.15.89:0/2494441039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23ac103e30 msgr2=0x7f23ac19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.447+0000 7f23abe00640 1 --2- 172.21.15.89:0/2494441039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23ac103e30 0x7f23ac19a400 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f239c00e9c0 tx=0x7f239c00ee80 comp rx=0 tx=0).stop 2024-09-10T08:39:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.447+0000 7f23abe00640 1 -- 172.21.15.89:0/2494441039 shutdown_connections 2024-09-10T08:39:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.447+0000 7f23abe00640 1 --2- 172.21.15.89:0/2494441039 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23900a2c10 0x7f23900a50d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.447+0000 7f23abe00640 1 --2- 172.21.15.89:0/2494441039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23ac108f60 0x7f23ac19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.447+0000 7f23abe00640 1 --2- 172.21.15.89:0/2494441039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23ac1047e0 0x7f23ac19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.447+0000 7f23abe00640 1 --2- 172.21.15.89:0/2494441039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23ac103e30 0x7f23ac19a400 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.447+0000 7f23abe00640 1 -- 172.21.15.89:0/2494441039 >> 172.21.15.89:0/2494441039 conn(0x7f23ac0ff8d0 msgr2=0x7f23ac100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.447+0000 7f23abe00640 1 -- 172.21.15.89:0/2494441039 shutdown_connections 2024-09-10T08:39:04.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:04.447+0000 7f23abe00640 1 -- 172.21.15.89:0/2494441039 wait complete. 2024-09-10T08:39:04.901 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:04 smithi089 ceph-mon[30757]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:04.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:04 smithi149 ceph-mon[36167]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:04 smithi111 ceph-mon[35220]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:05.290 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T08:39:05.660 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:06.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.230+0000 7efc97e00640 1 -- 172.21.15.89:0/3250846217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc981019c0 msgr2=0x7efc98101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:06.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.230+0000 7efc97e00640 1 --2- 172.21.15.89:0/3250846217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc981019c0 0x7efc98101e20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efc80009930 tx=0x7efc8002a420 comp rx=0 tx=0).stop 2024-09-10T08:39:06.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.230+0000 7efc97e00640 1 -- 172.21.15.89:0/3250846217 shutdown_connections 2024-09-10T08:39:06.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.230+0000 7efc97e00640 1 --2- 172.21.15.89:0/3250846217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc98102360 0x7efc9810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.230+0000 7efc97e00640 1 --2- 172.21.15.89:0/3250846217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc981019c0 0x7efc98101e20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.230+0000 7efc97e00640 1 --2- 172.21.15.89:0/3250846217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc981079c0 0x7efc98107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.230+0000 7efc97e00640 1 -- 172.21.15.89:0/3250846217 >> 172.21.15.89:0/3250846217 conn(0x7efc980fd740 msgr2=0x7efc980ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:06.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.230+0000 7efc97e00640 1 -- 172.21.15.89:0/3250846217 shutdown_connections 2024-09-10T08:39:06.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.230+0000 7efc97e00640 1 -- 172.21.15.89:0/3250846217 wait complete. 2024-09-10T08:39:06.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.231+0000 7efc97e00640 1 Processor -- start 2024-09-10T08:39:06.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.231+0000 7efc97e00640 1 -- start start 2024-09-10T08:39:06.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.231+0000 7efc97e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc981019c0 0x7efc980744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:06.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.231+0000 7efc97e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc98102360 0x7efc98074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:06.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.231+0000 7efc97e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc981079c0 0x7efc98074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:06.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.232+0000 7efc97e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efc98111dd0 con 0x7efc98102360 2024-09-10T08:39:06.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.232+0000 7efc97e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efc98111c50 con 0x7efc981079c0 2024-09-10T08:39:06.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.232+0000 7efc97e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efc98111f50 con 0x7efc981019c0 2024-09-10T08:39:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.232+0000 7efc8e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc98102360 0x7efc98074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.232+0000 7efc96a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc981019c0 0x7efc980744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.232+0000 7efc97400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc981079c0 0x7efc98074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.232+0000 7efc96a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc981019c0 0x7efc980744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:42542/0 (socket says 172.21.15.89:42542) 2024-09-10T08:39:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.232+0000 7efc8e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc98102360 0x7efc98074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52146/0 (socket says 172.21.15.89:52146) 2024-09-10T08:39:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.232+0000 7efc96a00640 1 -- 172.21.15.89:0/2486672949 learned_addr learned my addr 172.21.15.89:0/2486672949 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.232+0000 7efc97400640 1 -- 172.21.15.89:0/2486672949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc981019c0 msgr2=0x7efc980744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.232+0000 7efc97400640 1 --2- 172.21.15.89:0/2486672949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc981019c0 0x7efc980744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.232+0000 7efc97400640 1 -- 172.21.15.89:0/2486672949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc98102360 msgr2=0x7efc98074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.232+0000 7efc97400640 1 --2- 172.21.15.89:0/2486672949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc98102360 0x7efc98074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.232+0000 7efc97400640 1 -- 172.21.15.89:0/2486672949 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc981a6770 con 0x7efc981079c0 2024-09-10T08:39:06.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.233+0000 7efc97400640 1 --2- 172.21.15.89:0/2486672949 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc981079c0 0x7efc98074f70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efc9000e970 tx=0x7efc9000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:06.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.233+0000 7efc8fe00640 1 -- 172.21.15.89:0/2486672949 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9000ccc0 con 0x7efc981079c0 2024-09-10T08:39:06.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.233+0000 7efc8fe00640 1 -- 172.21.15.89:0/2486672949 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc90005020 con 0x7efc981079c0 2024-09-10T08:39:06.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.233+0000 7efc97e00640 1 -- 172.21.15.89:0/2486672949 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc981a6a60 con 0x7efc981079c0 2024-09-10T08:39:06.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.233+0000 7efc8fe00640 1 -- 172.21.15.89:0/2486672949 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90004a00 con 0x7efc981079c0 2024-09-10T08:39:06.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.233+0000 7efc97e00640 1 -- 172.21.15.89:0/2486672949 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc981a6f20 con 0x7efc981079c0 2024-09-10T08:39:06.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.236+0000 7efca0263740 1 Processor -- start 2024-09-10T08:39:06.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.236+0000 7efc8fe00640 1 -- 172.21.15.89:0/2486672949 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efc900051d0 con 0x7efc981079c0 2024-09-10T08:39:06.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.236+0000 7efca0263740 1 -- start start 2024-09-10T08:39:06.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.237+0000 7efca0263740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34b44e0 0x55c4d34b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:06.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.237+0000 7efca0263740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b4e00 0x55c4d34bf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:06.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.237+0000 7efca0263740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34bf6c0 0x55c4d34c1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:06.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.237+0000 7efca0263740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c4d341a0b0 con 0x55c4d34b44e0 2024-09-10T08:39:06.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.237+0000 7efca0263740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c4d33c1ab0 con 0x55c4d34bf6c0 2024-09-10T08:39:06.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.237+0000 7efca0263740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c4d3416070 con 0x55c4d34b4e00 2024-09-10T08:39:06.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.237+0000 7efc8e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34b44e0 0x55c4d34b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:06.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.237+0000 7efc8e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34b44e0 0x55c4d34b48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52160/0 (socket says 172.21.15.89:52160) 2024-09-10T08:39:06.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.237+0000 7efc8e000640 1 -- 172.21.15.89:0/2175504261 learned_addr learned my addr 172.21.15.89:0/2175504261 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:06.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.237+0000 7efc96a00640 1 --2- 172.21.15.89:0/2175504261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b4e00 0x55c4d34bf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:06.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.237+0000 7efc8e000640 1 --2- 172.21.15.89:0/2175504261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34bf6c0 0x55c4d34c1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:06.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.237+0000 7efc8e000640 1 -- 172.21.15.89:0/2175504261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b4e00 msgr2=0x55c4d34bf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:06.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.237+0000 7efc8e000640 1 --2- 172.21.15.89:0/2175504261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b4e00 0x55c4d34bf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.237+0000 7efc8e000640 1 -- 172.21.15.89:0/2175504261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34bf6c0 msgr2=0x55c4d34c1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:06.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.237+0000 7efc8e000640 1 --2- 172.21.15.89:0/2175504261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34bf6c0 0x55c4d34c1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.237+0000 7efc8e000640 1 -- 172.21.15.89:0/2175504261 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4d33ce4a0 con 0x55c4d34b44e0 2024-09-10T08:39:06.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.238+0000 7efc8e000640 1 --2- 172.21.15.89:0/2175504261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34b44e0 0x55c4d34b48c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efc8002aa10 tx=0x7efc80005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd547e9d975e3a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:06.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.238+0000 7efc87400640 1 -- 172.21.15.89:0/2175504261 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc80044020 con 0x55c4d34b44e0 2024-09-10T08:39:06.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.238+0000 7efc8fe00640 1 --2- 172.21.15.89:0/2486672949 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efc7c0a2c10 0x7efc7c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:06.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.238+0000 7efc87400640 1 -- 172.21.15.89:0/2175504261 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc800044b0 con 0x55c4d34b44e0 2024-09-10T08:39:06.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.238+0000 7efc87400640 1 -- 172.21.15.89:0/2175504261 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc80035040 con 0x55c4d34b44e0 2024-09-10T08:39:06.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.238+0000 7efc8fe00640 1 -- 172.21.15.89:0/2486672949 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7efc900cf570 con 0x7efc981079c0 2024-09-10T08:39:06.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.238+0000 7efc96a00640 1 --2- 172.21.15.89:0/2486672949 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efc7c0a2c10 0x7efc7c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:06.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.239+0000 7efca0263740 1 -- 172.21.15.89:0/2175504261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34b44e0 msgr2=0x55c4d34b48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:06.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.239+0000 7efca0263740 1 --2- 172.21.15.89:0/2175504261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34b44e0 0x55c4d34b48c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efc8002aa10 tx=0x7efc80005b50 comp rx=0 tx=0).stop 2024-09-10T08:39:06.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.239+0000 7efca0263740 1 -- 172.21.15.89:0/2175504261 shutdown_connections 2024-09-10T08:39:06.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.239+0000 7efca0263740 1 --2- 172.21.15.89:0/2175504261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34bf6c0 0x55c4d34c1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.239+0000 7efca0263740 1 --2- 172.21.15.89:0/2175504261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b4e00 0x55c4d34bf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.239+0000 7efca0263740 1 --2- 172.21.15.89:0/2175504261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34b44e0 0x55c4d34b48c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.239+0000 7efca0263740 1 -- 172.21.15.89:0/2175504261 >> 172.21.15.89:0/2175504261 conn(0x55c4d34b0a70 msgr2=0x55c4d34b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:06.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.239+0000 7efc96a00640 1 --2- 172.21.15.89:0/2486672949 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efc7c0a2c10 0x7efc7c0a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55c4d34b18d0 tx=0x7efc88009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:06.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.239+0000 7efca0263740 1 -- 172.21.15.89:0/2175504261 shutdown_connections 2024-09-10T08:39:06.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.239+0000 7efca0263740 1 -- 172.21.15.89:0/2175504261 wait complete. 2024-09-10T08:39:06.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.240+0000 7efca0263740 1 -- start start 2024-09-10T08:39:06.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.240+0000 7efca0263740 1 Processor -- start 2024-09-10T08:39:06.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.240+0000 7efca0263740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34b44e0 0x55c4d34bc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:06.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.240+0000 7efca0263740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b4e00 0x55c4d34bca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:06.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.240+0000 7efca0263740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34bf6c0 0x55c4d34bcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:06.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.240+0000 7efca0263740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c4d34c2330 con 0x55c4d34bf6c0 2024-09-10T08:39:06.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.240+0000 7efca0263740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c4d33a2920 con 0x55c4d34b44e0 2024-09-10T08:39:06.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.240+0000 7efca0263740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c4d34c24b0 con 0x55c4d34b4e00 2024-09-10T08:39:06.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.240+0000 7efca0263740 10 client.0 _ll_register_callbacks cb 0x55c4d33c68a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:39:06.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.241+0000 7efc97400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34bf6c0 0x55c4d34bcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:06.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.241+0000 7efc8e000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b4e00 0x55c4d34bca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:06.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.241+0000 7efc8e000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b4e00 0x55c4d34bca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:42560/0 (socket says 172.21.15.89:42560) 2024-09-10T08:39:06.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.241+0000 7efc8e000640 1 -- 172.21.15.89:0/1150923955 learned_addr learned my addr 172.21.15.89:0/1150923955 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.241+0000 7efc96a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34b44e0 0x55c4d34bc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.241+0000 7efc8e000640 1 -- 172.21.15.89:0/1150923955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34b44e0 msgr2=0x55c4d34bc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.241+0000 7efc8e000640 1 --2- 172.21.15.89:0/1150923955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34b44e0 0x55c4d34bc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.241+0000 7efc8e000640 1 -- 172.21.15.89:0/1150923955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34bf6c0 msgr2=0x55c4d34bcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.241+0000 7efc8e000640 1 --2- 172.21.15.89:0/1150923955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34bf6c0 0x55c4d34bcfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.241+0000 7efc8e000640 1 -- 172.21.15.89:0/1150923955 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4d318b2b0 con 0x55c4d34b4e00 2024-09-10T08:39:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.241+0000 7efc8e000640 1 --2- 172.21.15.89:0/1150923955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b4e00 0x55c4d34bca80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efc80009900 tx=0x7efc80009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae47a2f9891d8894 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.241+0000 7efc7be00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:39:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.242+0000 7efc7be00640 1 -- 172.21.15.89:0/1150923955 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc80044030 con 0x55c4d34b4e00 2024-09-10T08:39:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.242+0000 7efca0263740 1 -- 172.21.15.89:0/1150923955 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc800044b0 con 0x55c4d34b4e00 2024-09-10T08:39:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.242+0000 7efc7be00640 1 -- 172.21.15.89:0/1150923955 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7efc80005280 con 0x55c4d34b4e00 2024-09-10T08:39:06.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.242+0000 7efc7be00640 1 -- 172.21.15.89:0/1150923955 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efc80057050 con 0x55c4d34b4e00 2024-09-10T08:39:06.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.242+0000 7efca0263740 10 client.24982 fetch_fsmap learned FSMap version 25 2024-09-10T08:39:06.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.242+0000 7efca0263740 1 -- 172.21.15.89:0/1150923955 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c4d34c2330 con 0x55c4d34b4e00 2024-09-10T08:39:06.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.243+0000 7efc7be00640 1 -- 172.21.15.89:0/1150923955 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc800044b0 con 0x55c4d34b4e00 2024-09-10T08:39:06.247 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.243+0000 7efc7be00640 1 -- 172.21.15.89:0/1150923955 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7efc80049450 con 0x55c4d34b4e00 2024-09-10T08:39:06.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.243+0000 7efca0263740 10 client.24982 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:39:06.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.243+0000 7efca0263740 10 client.24982 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T08:39:06.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.243+0000 7efca0263740 20 client.24982 populate_metadata read hostname 'smithi089' 2024-09-10T08:39:06.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.243+0000 7efca0263740 1 --2- 172.21.15.89:0/1150923955 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55c4d34d0110 0x55c4d34d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:06.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.243+0000 7efca0263740 4 client.24982 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:39:06.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.243+0000 7efca0263740 1 -- 172.21.15.89:0/1150923955 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4d3427620 con 0x55c4d34d0110 2024-09-10T08:39:06.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.244+0000 7efc96a00640 1 --2- 172.21.15.89:0/1150923955 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55c4d34d0110 0x55c4d34d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:06.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.244+0000 7efc96a00640 1 --2- 172.21.15.89:0/1150923955 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55c4d34d0110 0x55c4d34d1580 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45fd5a516074dccd server_cookie=ede9a17c532f29aa in_seq=0 out_seq=0 2024-09-10T08:39:06.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.244+0000 7efc7be00640 10 client.24982 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:39:06.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.245+0000 7efc7be00640 1 -- 172.21.15.89:0/1150923955 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4d3427620 con 0x55c4d34d0110 2024-09-10T08:39:06.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.245+0000 7efc7be00640 10 client.24982 handle_command_reply: tid=0 2024-09-10T08:39:06.248 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.245+0000 7efca0263740 1 client.24982 shutdown 2024-09-10T08:39:06.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.246+0000 7efca0263740 1 -- 172.21.15.89:0/1150923955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b4e00 msgr2=0x55c4d34bca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:06.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.246+0000 7efca0263740 1 --2- 172.21.15.89:0/1150923955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b4e00 0x55c4d34bca80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efc80009900 tx=0x7efc80009520 comp rx=0 tx=0).stop 2024-09-10T08:39:06.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.246+0000 7efca0263740 1 -- 172.21.15.89:0/1150923955 shutdown_connections 2024-09-10T08:39:06.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.246+0000 7efca0263740 1 --2- 172.21.15.89:0/1150923955 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55c4d34d0110 0x55c4d34d1580 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.246+0000 7efca0263740 1 --2- 172.21.15.89:0/1150923955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34bf6c0 0x55c4d34bcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.246+0000 7efca0263740 1 --2- 172.21.15.89:0/1150923955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b4e00 0x55c4d34bca80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.246+0000 7efc7be00640 0 client.24982 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:39:06.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.246+0000 7efca0263740 1 --2- 172.21.15.89:0/1150923955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34b44e0 0x55c4d34bc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.246+0000 7efca0263740 1 -- 172.21.15.89:0/1150923955 >> 172.21.15.89:0/1150923955 conn(0x55c4d34b0a70 msgr2=0x55c4d34b67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:06.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.246+0000 7efca0263740 1 -- 172.21.15.89:0/1150923955 shutdown_connections 2024-09-10T08:39:06.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.246+0000 7efca0263740 1 -- 172.21.15.89:0/1150923955 wait complete. 2024-09-10T08:39:06.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.246+0000 7efca0263740 20 client.24982 trim_cache size 0 max 16384 2024-09-10T08:39:06.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.264+0000 7efca0263740 1 Processor -- start 2024-09-10T08:39:06.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.264+0000 7efca0263740 1 -- start start 2024-09-10T08:39:06.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.264+0000 7efca0263740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b44e0 0x55c4d34b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:06.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.264+0000 7efca0263740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34b4e00 0x55c4d34b9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:06.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.264+0000 7efca0263740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34bf6c0 0x55c4d34b9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:06.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.264+0000 7efca0263740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c4d34cf8c0 con 0x55c4d34bf6c0 2024-09-10T08:39:06.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.264+0000 7efca0263740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c4d34cff90 con 0x55c4d34b4e00 2024-09-10T08:39:06.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.264+0000 7efca0263740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c4d34d0660 con 0x55c4d34b44e0 2024-09-10T08:39:06.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.265+0000 7efc97400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34bf6c0 0x55c4d34b9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:06.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.265+0000 7efc8e000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34b4e00 0x55c4d34b9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:06.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.265+0000 7efc97400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34bf6c0 0x55c4d34b9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52192/0 (socket says 172.21.15.89:52192) 2024-09-10T08:39:06.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.265+0000 7efc97400640 1 -- 172.21.15.89:0/3281566292 learned_addr learned my addr 172.21.15.89:0/3281566292 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:06.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.265+0000 7efc96a00640 1 --2- 172.21.15.89:0/3281566292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b44e0 0x55c4d34b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:06.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.265+0000 7efc97400640 1 -- 172.21.15.89:0/3281566292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b44e0 msgr2=0x55c4d34b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:06.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.265+0000 7efc97400640 1 --2- 172.21.15.89:0/3281566292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b44e0 0x55c4d34b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.265+0000 7efc97400640 1 -- 172.21.15.89:0/3281566292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34b4e00 msgr2=0x55c4d34b9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:06.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.265+0000 7efc97400640 1 --2- 172.21.15.89:0/3281566292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34b4e00 0x55c4d34b9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.265+0000 7efc97400640 1 -- 172.21.15.89:0/3281566292 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4d34bb530 con 0x55c4d34bf6c0 2024-09-10T08:39:06.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.265+0000 7efc8e000640 1 --2- 172.21.15.89:0/3281566292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34b4e00 0x55c4d34b9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:06.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.265+0000 7efc96a00640 1 --2- 172.21.15.89:0/3281566292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b44e0 0x55c4d34b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:06.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.265+0000 7efc97400640 1 --2- 172.21.15.89:0/3281566292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34bf6c0 0x55c4d34b9d40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efc9000c6a0 tx=0x7efc90097ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26fcf2f335713e88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:06.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.266+0000 7efc86a00640 1 -- 172.21.15.89:0/3281566292 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc900b3020 con 0x55c4d34bf6c0 2024-09-10T08:39:06.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.266+0000 7efc86a00640 1 -- 172.21.15.89:0/3281566292 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc900a44b0 con 0x55c4d34bf6c0 2024-09-10T08:39:06.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.266+0000 7efc86a00640 1 -- 172.21.15.89:0/3281566292 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc900cf070 con 0x55c4d34bf6c0 2024-09-10T08:39:06.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.266+0000 7efca0263740 1 -- 172.21.15.89:0/3281566292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34bf6c0 msgr2=0x55c4d34b9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:06.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.266+0000 7efca0263740 1 --2- 172.21.15.89:0/3281566292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34bf6c0 0x55c4d34b9d40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efc9000c6a0 tx=0x7efc90097ae0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.266+0000 7efca0263740 1 -- 172.21.15.89:0/3281566292 shutdown_connections 2024-09-10T08:39:06.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.266+0000 7efca0263740 1 --2- 172.21.15.89:0/3281566292 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34bf6c0 0x55c4d34b9d40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.266+0000 7efca0263740 1 --2- 172.21.15.89:0/3281566292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34b4e00 0x55c4d34b9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.266+0000 7efca0263740 1 --2- 172.21.15.89:0/3281566292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b44e0 0x55c4d34b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.266+0000 7efca0263740 1 -- 172.21.15.89:0/3281566292 >> 172.21.15.89:0/3281566292 conn(0x55c4d34b0a70 msgr2=0x55c4d34cf120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:06.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.266+0000 7efca0263740 1 -- 172.21.15.89:0/3281566292 shutdown_connections 2024-09-10T08:39:06.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.266+0000 7efca0263740 1 -- 172.21.15.89:0/3281566292 wait complete. 2024-09-10T08:39:06.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.267+0000 7efca0263740 1 -- start start 2024-09-10T08:39:06.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.267+0000 7efca0263740 1 Processor -- start 2024-09-10T08:39:06.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.267+0000 7efca0263740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34b44e0 0x55c4d34b96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:06.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.267+0000 7efca0263740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b4e00 0x55c4d34b9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:06.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.267+0000 7efca0263740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34bf6c0 0x55c4d34ba120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:06.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.267+0000 7efca0263740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c4d34d1400 con 0x55c4d34b44e0 2024-09-10T08:39:06.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.267+0000 7efca0263740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c4d34d0d30 con 0x55c4d34bf6c0 2024-09-10T08:39:06.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.267+0000 7efca0263740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c4d34d1ad0 con 0x55c4d34b4e00 2024-09-10T08:39:06.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.267+0000 7efca0263740 10 client.0 _ll_register_callbacks cb 0x7efc80004700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:39:06.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.268+0000 7efc96a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34b44e0 0x55c4d34b96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:06.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.268+0000 7efc96a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34b44e0 0x55c4d34b96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52194/0 (socket says 172.21.15.89:52194) 2024-09-10T08:39:06.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.268+0000 7efc96a00640 1 -- 172.21.15.89:0/817179760 learned_addr learned my addr 172.21.15.89:0/817179760 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:06.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.268+0000 7efc8e000640 1 --2- 172.21.15.89:0/817179760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b4e00 0x55c4d34b9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:06.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.268+0000 7efc97400640 1 --2- 172.21.15.89:0/817179760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34bf6c0 0x55c4d34ba120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:06.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.268+0000 7efc96a00640 1 -- 172.21.15.89:0/817179760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b4e00 msgr2=0x55c4d34b9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:06.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.268+0000 7efc96a00640 1 --2- 172.21.15.89:0/817179760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b4e00 0x55c4d34b9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.268+0000 7efc96a00640 1 -- 172.21.15.89:0/817179760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34bf6c0 msgr2=0x55c4d34ba120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:06.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.268+0000 7efc96a00640 1 --2- 172.21.15.89:0/817179760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34bf6c0 0x55c4d34ba120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.268+0000 7efc96a00640 1 -- 172.21.15.89:0/817179760 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4d34d7e60 con 0x55c4d34b44e0 2024-09-10T08:39:06.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.268+0000 7efc96a00640 1 --2- 172.21.15.89:0/817179760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34b44e0 0x55c4d34b96a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55c4d34d1de0 tx=0x7efc88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff36ee6161e0a49b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:06.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.268+0000 7efc7be00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:39:06.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.269+0000 7efc7be00640 1 -- 172.21.15.89:0/817179760 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88010e90 con 0x55c4d34b44e0 2024-09-10T08:39:06.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.269+0000 7efc7be00640 1 -- 172.21.15.89:0/817179760 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7efc88003970 con 0x55c4d34b44e0 2024-09-10T08:39:06.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.269+0000 7efca0263740 1 -- 172.21.15.89:0/817179760 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc900a44b0 con 0x55c4d34b44e0 2024-09-10T08:39:06.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.269+0000 7efc7be00640 1 -- 172.21.15.89:0/817179760 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efc88003b70 con 0x55c4d34b44e0 2024-09-10T08:39:06.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.269+0000 7efca0263740 10 client.14967 fetch_fsmap learned FSMap version 25 2024-09-10T08:39:06.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.269+0000 7efca0263740 1 -- 172.21.15.89:0/817179760 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c4d35433a0 con 0x55c4d34b44e0 2024-09-10T08:39:06.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.269+0000 7efc7be00640 1 -- 172.21.15.89:0/817179760 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc900a44b0 con 0x55c4d34b44e0 2024-09-10T08:39:06.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.269+0000 7efc7be00640 1 -- 172.21.15.89:0/817179760 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7efc88004410 con 0x55c4d34b44e0 2024-09-10T08:39:06.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.270+0000 7efca0263740 10 client.14967 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:39:06.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.270+0000 7efca0263740 10 client.14967 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T08:39:06.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.270+0000 7efca0263740 20 client.14967 populate_metadata read hostname 'smithi089' 2024-09-10T08:39:06.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.270+0000 7efca0263740 1 --2- 172.21.15.89:0/817179760 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55c4d3548010 0x55c4d3546d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:06.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.270+0000 7efca0263740 4 client.14967 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:39:06.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.270+0000 7efca0263740 1 -- 172.21.15.89:0/817179760 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "status"}) -- 0x55c4d3421b30 con 0x55c4d3548010 2024-09-10T08:39:06.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.270+0000 7efc8e000640 1 --2- 172.21.15.89:0/817179760 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55c4d3548010 0x55c4d3546d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.271+0000 7efc7be00640 10 client.14967 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:39:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.271+0000 7efc8e000640 1 --2- 172.21.15.89:0/817179760 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55c4d3548010 0x55c4d3546d20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd6e7af6eadf539c server_cookie=cb3fdd986ede788e in_seq=0 out_seq=0 2024-09-10T08:39:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.271+0000 7efc7be00640 1 -- 172.21.15.89:0/817179760 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c4d3421b30 con 0x55c4d3548010 2024-09-10T08:39:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.271+0000 7efc7be00640 10 client.14967 handle_command_reply: tid=0 2024-09-10T08:39:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.271+0000 7efca0263740 1 client.14967 shutdown 2024-09-10T08:39:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.271+0000 7efca0263740 1 -- 172.21.15.89:0/817179760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34b44e0 msgr2=0x55c4d34b96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.271+0000 7efca0263740 1 --2- 172.21.15.89:0/817179760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34b44e0 0x55c4d34b96a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55c4d34d1de0 tx=0x7efc88004a40 comp rx=0 tx=0).stop 2024-09-10T08:39:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.271+0000 7efca0263740 1 -- 172.21.15.89:0/817179760 shutdown_connections 2024-09-10T08:39:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.271+0000 7efca0263740 1 --2- 172.21.15.89:0/817179760 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55c4d3548010 0x55c4d3546d20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.271+0000 7efca0263740 1 --2- 172.21.15.89:0/817179760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c4d34bf6c0 0x55c4d34ba120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.271+0000 7efca0263740 1 --2- 172.21.15.89:0/817179760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c4d34b4e00 0x55c4d34b9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.271+0000 7efca0263740 1 --2- 172.21.15.89:0/817179760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c4d34b44e0 0x55c4d34b96a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.271+0000 7efc7be00640 0 client.14967 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:39:06.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.271+0000 7efca0263740 1 -- 172.21.15.89:0/817179760 >> 172.21.15.89:0/817179760 conn(0x55c4d34b0a70 msgr2=0x55c4d34ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:06.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.271+0000 7efca0263740 1 -- 172.21.15.89:0/817179760 shutdown_connections 2024-09-10T08:39:06.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.272+0000 7efca0263740 1 -- 172.21.15.89:0/817179760 wait complete. 2024-09-10T08:39:06.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.272+0000 7efca0263740 20 client.14967 trim_cache size 0 max 16384 2024-09-10T08:39:06.275 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:39:06.275 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "91f747bc-6f4e-11ef-bcea-c7b262605968", 2024-09-10T08:39:06.275 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 1, 2024-09-10T08:39:06.275 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24518, 2024-09-10T08:39:06.275 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2024-09-10T08:39:06.276 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2024-09-10T08:39:06.276 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2024-09-10T08:39:06.276 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 79.578470330000002, 2024-09-10T08:39:06.276 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 25, 2024-09-10T08:39:06.276 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 96, 2024-09-10T08:39:06.276 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T08:39:06.276 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 153.43533209700001 2024-09-10T08:39:06.276 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:39:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.272+0000 7efc97e00640 1 -- 172.21.15.89:0/2486672949 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efc7c0a2c10 msgr2=0x7efc7c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.272+0000 7efc97e00640 1 --2- 172.21.15.89:0/2486672949 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efc7c0a2c10 0x7efc7c0a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55c4d34b18d0 tx=0x7efc88009800 comp rx=0 tx=0).stop 2024-09-10T08:39:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.272+0000 7efc97e00640 1 -- 172.21.15.89:0/2486672949 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc981079c0 msgr2=0x7efc98074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.272+0000 7efc97e00640 1 --2- 172.21.15.89:0/2486672949 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc981079c0 0x7efc98074f70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efc9000e970 tx=0x7efc9000ee30 comp rx=0 tx=0).stop 2024-09-10T08:39:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.273+0000 7efc97e00640 1 -- 172.21.15.89:0/2486672949 shutdown_connections 2024-09-10T08:39:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.273+0000 7efc97e00640 1 --2- 172.21.15.89:0/2486672949 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efc7c0a2c10 0x7efc7c0a50d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.273+0000 7efc97e00640 1 --2- 172.21.15.89:0/2486672949 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc981079c0 0x7efc98074f70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.273+0000 7efc97e00640 1 --2- 172.21.15.89:0/2486672949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc98102360 0x7efc98074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.273+0000 7efc97e00640 1 --2- 172.21.15.89:0/2486672949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc981019c0 0x7efc980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:06.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.273+0000 7efc97e00640 1 -- 172.21.15.89:0/2486672949 >> 172.21.15.89:0/2486672949 conn(0x7efc980fd740 msgr2=0x7efc980fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:06.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.273+0000 7efc97e00640 1 -- 172.21.15.89:0/2486672949 shutdown_connections 2024-09-10T08:39:06.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:06.273+0000 7efc97e00640 1 -- 172.21.15.89:0/2486672949 wait complete. 2024-09-10T08:39:06.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:06 smithi149 ceph-mon[36167]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:39:07.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:06 smithi111 ceph-mon[35220]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:39:07.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:06 smithi089 ceph-mon[30757]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:39:07.128 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T08:39:07.493 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:08.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.073+0000 7f8aaa600640 1 -- 172.21.15.89:0/1367473437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8aa4112b90 msgr2=0x7f8aa4114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:08.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.073+0000 7f8aaa600640 1 --2- 172.21.15.89:0/1367473437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8aa4112b90 0x7f8aa4114f80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c009ca0 tx=0x7f8a9c02a950 comp rx=0 tx=0).stop 2024-09-10T08:39:08.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.073+0000 7f8aaa600640 1 -- 172.21.15.89:0/1367473437 shutdown_connections 2024-09-10T08:39:08.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.073+0000 7f8aaa600640 1 --2- 172.21.15.89:0/1367473437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8aa4112b90 0x7f8aa4114f80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.073+0000 7f8aaa600640 1 --2- 172.21.15.89:0/1367473437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8aa4074eb0 0x7f8aa4112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.073+0000 7f8aaa600640 1 --2- 172.21.15.89:0/1367473437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8aa4074590 0x7f8aa4074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.073+0000 7f8aaa600640 1 -- 172.21.15.89:0/1367473437 >> 172.21.15.89:0/1367473437 conn(0x7f8aa40fd8c0 msgr2=0x7f8aa40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:08.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.073+0000 7f8aaa600640 1 -- 172.21.15.89:0/1367473437 shutdown_connections 2024-09-10T08:39:08.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.073+0000 7f8aaa600640 1 -- 172.21.15.89:0/1367473437 wait complete. 2024-09-10T08:39:08.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.074+0000 7f8aaa600640 1 Processor -- start 2024-09-10T08:39:08.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.075+0000 7f8aaa600640 1 -- start start 2024-09-10T08:39:08.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.075+0000 7f8aaa600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8aa4074590 0x7f8aa41a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:08.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.075+0000 7f8aaa600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8aa4074eb0 0x7f8aa41a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:08.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.075+0000 7f8aaa600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8aa4112b90 0x7f8aa419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:08.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.075+0000 7f8aaa600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa4118550 con 0x7f8aa4074eb0 2024-09-10T08:39:08.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.075+0000 7f8aaa600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41183d0 con 0x7f8aa4074590 2024-09-10T08:39:08.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.075+0000 7f8aaa600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41186d0 con 0x7f8aa4112b90 2024-09-10T08:39:08.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.075+0000 7f8aa2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8aa4074eb0 0x7f8aa41a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:08.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.075+0000 7f8aa3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8aa4074590 0x7f8aa41a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:08.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.075+0000 7f8aa3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8aa4074590 0x7f8aa41a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:45140/0 (socket says 172.21.15.89:45140) 2024-09-10T08:39:08.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.075+0000 7f8aa3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8aa4112b90 0x7f8aa419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:08.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.075+0000 7f8aa3400640 1 -- 172.21.15.89:0/3105956769 learned_addr learned my addr 172.21.15.89:0/3105956769 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:08.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.076+0000 7f8aa3400640 1 -- 172.21.15.89:0/3105956769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8aa4112b90 msgr2=0x7f8aa419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:08.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.076+0000 7f8aa3400640 1 --2- 172.21.15.89:0/3105956769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8aa4112b90 0x7f8aa419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.076+0000 7f8aa3400640 1 -- 172.21.15.89:0/3105956769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8aa4074eb0 msgr2=0x7f8aa41a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:08.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.076+0000 7f8aa3400640 1 --2- 172.21.15.89:0/3105956769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8aa4074eb0 0x7f8aa41a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.076+0000 7f8aa3400640 1 -- 172.21.15.89:0/3105956769 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa419f010 con 0x7f8aa4074590 2024-09-10T08:39:08.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.076+0000 7f8aa3400640 1 --2- 172.21.15.89:0/3105956769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8aa4074590 0x7f8aa41a4720 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400d950 tx=0x7f8a9400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:08.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.076+0000 7f8a9be00640 1 -- 172.21.15.89:0/3105956769 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94017070 con 0x7f8aa4074590 2024-09-10T08:39:08.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.076+0000 7f8aaa600640 1 -- 172.21.15.89:0/3105956769 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa419f300 con 0x7f8aa4074590 2024-09-10T08:39:08.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.076+0000 7f8a9be00640 1 -- 172.21.15.89:0/3105956769 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a94009c30 con 0x7f8aa4074590 2024-09-10T08:39:08.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.076+0000 7f8a9be00640 1 -- 172.21.15.89:0/3105956769 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9400f6b0 con 0x7f8aa4074590 2024-09-10T08:39:08.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.077+0000 7f8aaa600640 1 -- 172.21.15.89:0/3105956769 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa41ab5f0 con 0x7f8aa4074590 2024-09-10T08:39:08.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.079+0000 7f8aad07a740 1 Processor -- start 2024-09-10T08:39:08.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.080+0000 7f8a9be00640 1 -- 172.21.15.89:0/3105956769 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8a940109d0 con 0x7f8aa4074590 2024-09-10T08:39:08.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.080+0000 7f8aad07a740 1 -- start start 2024-09-10T08:39:08.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.080+0000 7f8aad07a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e70c330 0x56363e70c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:08.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.080+0000 7f8aad07a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e70cc50 0x56363e716fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:08.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.080+0000 7f8aad07a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e717510 0x56363e719900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:08.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.080+0000 7f8aad07a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56363e6720b0 con 0x56363e70c330 2024-09-10T08:39:08.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.080+0000 7f8aad07a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56363e619ab0 con 0x56363e70cc50 2024-09-10T08:39:08.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.080+0000 7f8aad07a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56363e66e070 con 0x56363e717510 2024-09-10T08:39:08.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.080+0000 7f8aa3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e70c330 0x56363e70c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:08.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.080+0000 7f8aa2a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e717510 0x56363e719900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:08.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.080+0000 7f8aa3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e70cc50 0x56363e716fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:08.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.080+0000 7f8aa3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e70c330 0x56363e70c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52232/0 (socket says 172.21.15.89:52232) 2024-09-10T08:39:08.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.080+0000 7f8aa3e00640 1 -- 172.21.15.89:0/2725102967 learned_addr learned my addr 172.21.15.89:0/2725102967 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:08.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.081+0000 7f8aa2a00640 1 -- 172.21.15.89:0/2725102967 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e70cc50 msgr2=0x56363e716fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:08.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.081+0000 7f8aa2a00640 1 --2- 172.21.15.89:0/2725102967 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e70cc50 0x56363e716fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.081+0000 7f8aa2a00640 1 -- 172.21.15.89:0/2725102967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e70c330 msgr2=0x56363e70c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:08.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.081+0000 7f8aa2a00640 1 --2- 172.21.15.89:0/2725102967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e70c330 0x56363e70c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.081+0000 7f8aa2a00640 1 -- 172.21.15.89:0/2725102967 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56363e6264a0 con 0x56363e717510 2024-09-10T08:39:08.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.081+0000 7f8aa2a00640 1 --2- 172.21.15.89:0/2725102967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e717510 0x56363e719900 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8aa41a00d0 tx=0x7f8a90011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ece80e1f3a63ec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:08.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.081+0000 7f8a9be00640 1 --2- 172.21.15.89:0/3105956769 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8a880a2c10 0x7f8a880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:08.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.081+0000 7f8a98c00640 1 -- 172.21.15.89:0/2725102967 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90004050 con 0x56363e717510 2024-09-10T08:39:08.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.081+0000 7f8a9be00640 1 -- 172.21.15.89:0/3105956769 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f8a940cf1c0 con 0x7f8aa4074590 2024-09-10T08:39:08.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.081+0000 7f8a98c00640 1 -- 172.21.15.89:0/2725102967 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a90005500 con 0x56363e717510 2024-09-10T08:39:08.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.081+0000 7f8a98c00640 1 -- 172.21.15.89:0/2725102967 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90004d90 con 0x56363e717510 2024-09-10T08:39:08.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.082+0000 7f8aa3e00640 1 --2- 172.21.15.89:0/3105956769 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8a880a2c10 0x7f8a880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:08.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.082+0000 7f8aad07a740 1 -- 172.21.15.89:0/2725102967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e717510 msgr2=0x56363e719900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:08.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.082+0000 7f8aad07a740 1 --2- 172.21.15.89:0/2725102967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e717510 0x56363e719900 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8aa41a00d0 tx=0x7f8a90011810 comp rx=0 tx=0).stop 2024-09-10T08:39:08.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.082+0000 7f8aa3e00640 1 --2- 172.21.15.89:0/3105956769 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8a880a2c10 0x7f8a880a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c009ca0 tx=0x7f8a9c005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:08.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.082+0000 7f8aad07a740 1 -- 172.21.15.89:0/2725102967 shutdown_connections 2024-09-10T08:39:08.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.082+0000 7f8aad07a740 1 --2- 172.21.15.89:0/2725102967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e717510 0x56363e719900 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.082+0000 7f8aad07a740 1 --2- 172.21.15.89:0/2725102967 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e70cc50 0x56363e716fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.082+0000 7f8aad07a740 1 --2- 172.21.15.89:0/2725102967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e70c330 0x56363e70c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.082+0000 7f8aad07a740 1 -- 172.21.15.89:0/2725102967 >> 172.21.15.89:0/2725102967 conn(0x56363e7088c0 msgr2=0x56363e708ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:08.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.083+0000 7f8aad07a740 1 -- 172.21.15.89:0/2725102967 shutdown_connections 2024-09-10T08:39:08.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.083+0000 7f8aad07a740 1 -- 172.21.15.89:0/2725102967 wait complete. 2024-09-10T08:39:08.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.083+0000 7f8aad07a740 1 -- start start 2024-09-10T08:39:08.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.083+0000 7f8aad07a740 1 Processor -- start 2024-09-10T08:39:08.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.084+0000 7f8aad07a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e70c330 0x56363e714580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:08.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.084+0000 7f8aad07a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e70cc50 0x56363e714ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:08.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.084+0000 7f8aad07a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e717510 0x56363e715000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:08.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.084+0000 7f8aad07a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56363e71a060 con 0x56363e717510 2024-09-10T08:39:08.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.084+0000 7f8aad07a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56363e5fa920 con 0x56363e70cc50 2024-09-10T08:39:08.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.084+0000 7f8aad07a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56363e71a1e0 con 0x56363e70c330 2024-09-10T08:39:08.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.084+0000 7f8aad07a740 10 client.0 _ll_register_callbacks cb 0x56363e61e8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:39:08.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.084+0000 7f8aa3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e70c330 0x56363e714580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:08.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.084+0000 7f8aa2a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e70cc50 0x56363e714ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:08.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.084+0000 7f8aa3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e70c330 0x56363e714580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:42614/0 (socket says 172.21.15.89:42614) 2024-09-10T08:39:08.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.084+0000 7f8aa3400640 1 -- 172.21.15.89:0/2422060993 learned_addr learned my addr 172.21.15.89:0/2422060993 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:08.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.084+0000 7f8aa3e00640 1 --2- 172.21.15.89:0/2422060993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e717510 0x56363e715000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:08.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.084+0000 7f8aa3400640 1 -- 172.21.15.89:0/2422060993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e70cc50 msgr2=0x56363e714ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:08.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.084+0000 7f8aa3400640 1 --2- 172.21.15.89:0/2422060993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e70cc50 0x56363e714ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.084+0000 7f8aa3400640 1 -- 172.21.15.89:0/2422060993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e717510 msgr2=0x56363e715000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:08.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.084+0000 7f8aa3400640 1 --2- 172.21.15.89:0/2422060993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e717510 0x56363e715000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.084+0000 7f8aa3400640 1 -- 172.21.15.89:0/2422060993 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56363e3e32b0 con 0x56363e70c330 2024-09-10T08:39:08.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.084+0000 7f8aa3e00640 1 --2- 172.21.15.89:0/2422060993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e717510 0x56363e715000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:39:08.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.085+0000 7f8aa3400640 1 --2- 172.21.15.89:0/2422060993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e70c330 0x56363e714580 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8a94097f10 tx=0x7f8a940d27a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c5055d0cb2426b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:08.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.085+0000 7f8a8d600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:39:08.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.085+0000 7f8a8d600640 1 -- 172.21.15.89:0/2422060993 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94020070 con 0x56363e70c330 2024-09-10T08:39:08.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.085+0000 7f8a8d600640 1 -- 172.21.15.89:0/2422060993 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f8a940cc990 con 0x56363e70c330 2024-09-10T08:39:08.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.085+0000 7f8aad07a740 1 -- 172.21.15.89:0/2422060993 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a90005500 con 0x56363e70c330 2024-09-10T08:39:08.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.085+0000 7f8a8d600640 1 -- 172.21.15.89:0/2422060993 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8a9409aea0 con 0x56363e70c330 2024-09-10T08:39:08.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.085+0000 7f8aad07a740 10 client.25000 fetch_fsmap learned FSMap version 25 2024-09-10T08:39:08.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.085+0000 7f8aad07a740 1 -- 172.21.15.89:0/2422060993 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56363e725e10 con 0x56363e70c330 2024-09-10T08:39:08.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.086+0000 7f8a8d600640 1 -- 172.21.15.89:0/2422060993 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a90005500 con 0x56363e70c330 2024-09-10T08:39:08.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.086+0000 7f8a8d600640 1 -- 172.21.15.89:0/2422060993 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8a9409a990 con 0x56363e70c330 2024-09-10T08:39:08.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.086+0000 7f8aad07a740 10 client.25000 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:39:08.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.086+0000 7f8aad07a740 10 client.25000 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:39:08.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.086+0000 7f8aad07a740 20 client.25000 populate_metadata read hostname 'smithi089' 2024-09-10T08:39:08.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.086+0000 7f8aad07a740 1 --2- 172.21.15.89:0/2422060993 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56363e728410 0x56363e729880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:08.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.086+0000 7f8aad07a740 4 client.25000 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:39:08.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.086+0000 7f8aad07a740 1 -- 172.21.15.89:0/2422060993 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56363e67f620 con 0x56363e728410 2024-09-10T08:39:08.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.087+0000 7f8aa2a00640 1 --2- 172.21.15.89:0/2422060993 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56363e728410 0x56363e729880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:08.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.087+0000 7f8aa2a00640 1 --2- 172.21.15.89:0/2422060993 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56363e728410 0x56363e729880 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31b03eb18d07aa13 server_cookie=4d38fbc47b4faa1c in_seq=0 out_seq=0 2024-09-10T08:39:08.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.087+0000 7f8a8d600640 10 client.25000 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:39:08.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.088+0000 7f8a8d600640 1 -- 172.21.15.89:0/2422060993 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56363e67f620 con 0x56363e728410 2024-09-10T08:39:08.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.088+0000 7f8a8d600640 10 client.25000 handle_command_reply: tid=0 2024-09-10T08:39:08.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.088+0000 7f8aad07a740 1 client.25000 shutdown 2024-09-10T08:39:08.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.088+0000 7f8aad07a740 1 -- 172.21.15.89:0/2422060993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e70c330 msgr2=0x56363e714580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:08.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.088+0000 7f8aad07a740 1 --2- 172.21.15.89:0/2422060993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e70c330 0x56363e714580 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8a94097f10 tx=0x7f8a940d27a0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.088+0000 7f8aad07a740 1 -- 172.21.15.89:0/2422060993 shutdown_connections 2024-09-10T08:39:08.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.088+0000 7f8aad07a740 1 --2- 172.21.15.89:0/2422060993 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56363e728410 0x56363e729880 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.088+0000 7f8aad07a740 1 --2- 172.21.15.89:0/2422060993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e717510 0x56363e715000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.088+0000 7f8a8d600640 0 client.25000 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:39:08.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.088+0000 7f8aad07a740 1 --2- 172.21.15.89:0/2422060993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e70cc50 0x56363e714ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.088+0000 7f8aad07a740 1 --2- 172.21.15.89:0/2422060993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e70c330 0x56363e714580 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.088+0000 7f8aad07a740 1 -- 172.21.15.89:0/2422060993 >> 172.21.15.89:0/2422060993 conn(0x56363e7088c0 msgr2=0x56363e70e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:08.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.089+0000 7f8aad07a740 1 -- 172.21.15.89:0/2422060993 shutdown_connections 2024-09-10T08:39:08.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.089+0000 7f8aad07a740 1 -- 172.21.15.89:0/2422060993 wait complete. 2024-09-10T08:39:08.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.089+0000 7f8aad07a740 20 client.25000 trim_cache size 0 max 16384 2024-09-10T08:39:08.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.107+0000 7f8aad07a740 1 Processor -- start 2024-09-10T08:39:08.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.107+0000 7f8aad07a740 1 -- start start 2024-09-10T08:39:08.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.107+0000 7f8aad07a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e70c330 0x56363e70ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:08.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.107+0000 7f8aad07a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e70cc50 0x56363e712eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:08.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.107+0000 7f8aad07a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e717510 0x56363e7133f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:08.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.107+0000 7f8aad07a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56363e727bc0 con 0x56363e717510 2024-09-10T08:39:08.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.107+0000 7f8aad07a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56363e728290 con 0x56363e70c330 2024-09-10T08:39:08.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.107+0000 7f8aad07a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56363e728960 con 0x56363e70cc50 2024-09-10T08:39:08.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.107+0000 7f8aa3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e717510 0x56363e7133f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:08.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.108+0000 7f8aa3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e717510 0x56363e7133f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52248/0 (socket says 172.21.15.89:52248) 2024-09-10T08:39:08.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.108+0000 7f8aa3e00640 1 -- 172.21.15.89:0/3559899298 learned_addr learned my addr 172.21.15.89:0/3559899298 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:08.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.108+0000 7f8aa2a00640 1 --2- 172.21.15.89:0/3559899298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e70cc50 0x56363e712eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:08.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.108+0000 7f8aa3400640 1 --2- 172.21.15.89:0/3559899298 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e70c330 0x56363e70ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:08.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.108+0000 7f8aa2a00640 1 -- 172.21.15.89:0/3559899298 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e70c330 msgr2=0x56363e70ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:08.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.108+0000 7f8aa2a00640 1 --2- 172.21.15.89:0/3559899298 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e70c330 0x56363e70ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.108+0000 7f8aa2a00640 1 -- 172.21.15.89:0/3559899298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e717510 msgr2=0x56363e7133f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:08.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.108+0000 7f8aa2a00640 1 --2- 172.21.15.89:0/3559899298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e717510 0x56363e7133f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.108+0000 7f8aa2a00640 1 -- 172.21.15.89:0/3559899298 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56363e712b30 con 0x56363e70cc50 2024-09-10T08:39:08.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.108+0000 7f8aa2a00640 1 --2- 172.21.15.89:0/3559899298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e70cc50 0x56363e712eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8a90004650 tx=0x7f8a90009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad7a4ee73bbdc2e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:08.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.109+0000 7f8a8fe00640 1 -- 172.21.15.89:0/3559899298 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9002c3e0 con 0x56363e70cc50 2024-09-10T08:39:08.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.109+0000 7f8a8fe00640 1 -- 172.21.15.89:0/3559899298 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9002ca60 con 0x56363e70cc50 2024-09-10T08:39:08.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.109+0000 7f8a8fe00640 1 -- 172.21.15.89:0/3559899298 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90019a50 con 0x56363e70cc50 2024-09-10T08:39:08.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.109+0000 7f8aad07a740 1 -- 172.21.15.89:0/3559899298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e70cc50 msgr2=0x56363e712eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:08.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.109+0000 7f8aad07a740 1 --2- 172.21.15.89:0/3559899298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e70cc50 0x56363e712eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8a90004650 tx=0x7f8a90009680 comp rx=0 tx=0).stop 2024-09-10T08:39:08.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.109+0000 7f8aad07a740 1 -- 172.21.15.89:0/3559899298 shutdown_connections 2024-09-10T08:39:08.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.109+0000 7f8aad07a740 1 --2- 172.21.15.89:0/3559899298 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e717510 0x56363e7133f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.109+0000 7f8aad07a740 1 --2- 172.21.15.89:0/3559899298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e70cc50 0x56363e712eb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.109+0000 7f8aad07a740 1 --2- 172.21.15.89:0/3559899298 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e70c330 0x56363e70ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.109+0000 7f8aad07a740 1 -- 172.21.15.89:0/3559899298 >> 172.21.15.89:0/3559899298 conn(0x56363e7088c0 msgr2=0x56363e708ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:08.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.109+0000 7f8aad07a740 1 -- 172.21.15.89:0/3559899298 shutdown_connections 2024-09-10T08:39:08.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.109+0000 7f8aad07a740 1 -- 172.21.15.89:0/3559899298 wait complete. 2024-09-10T08:39:08.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.110+0000 7f8aad07a740 1 -- start start 2024-09-10T08:39:08.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.110+0000 7f8aad07a740 1 Processor -- start 2024-09-10T08:39:08.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.111+0000 7f8aad07a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e70c330 0x56363e7115a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:08.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.111+0000 7f8aad07a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e70cc50 0x56363e711ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:08.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.111+0000 7f8aad07a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e717510 0x56363e712020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:08.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.111+0000 7f8aad07a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56363e729700 con 0x56363e70c330 2024-09-10T08:39:08.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.111+0000 7f8aad07a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56363e729030 con 0x56363e717510 2024-09-10T08:39:08.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.111+0000 7f8aad07a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56363e729dd0 con 0x56363e70cc50 2024-09-10T08:39:08.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.111+0000 7f8aad07a740 10 client.0 _ll_register_callbacks cb 0x7f8a94097120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:39:08.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.111+0000 7f8aa3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e70c330 0x56363e7115a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:08.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.111+0000 7f8aa2a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e70cc50 0x56363e711ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:08.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.111+0000 7f8aa3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e717510 0x56363e712020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:08.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.111+0000 7f8aa3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e717510 0x56363e712020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:45174/0 (socket says 172.21.15.89:45174) 2024-09-10T08:39:08.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.111+0000 7f8aa3e00640 1 -- 172.21.15.89:0/2104316531 learned_addr learned my addr 172.21.15.89:0/2104316531 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:08.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.111+0000 7f8aa3e00640 1 -- 172.21.15.89:0/2104316531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e70cc50 msgr2=0x56363e711ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:08.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.111+0000 7f8aa3e00640 1 --2- 172.21.15.89:0/2104316531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e70cc50 0x56363e711ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.111+0000 7f8aa3e00640 1 -- 172.21.15.89:0/2104316531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e70c330 msgr2=0x56363e7115a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:08.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.111+0000 7f8aa3e00640 1 --2- 172.21.15.89:0/2104316531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e70c330 0x56363e7115a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.111+0000 7f8aa3e00640 1 -- 172.21.15.89:0/2104316531 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56363e72fff0 con 0x56363e717510 2024-09-10T08:39:08.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.111+0000 7f8aa2a00640 1 --2- 172.21.15.89:0/2104316531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e70cc50 0x56363e711ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:08.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.112+0000 7f8aa3e00640 1 --2- 172.21.15.89:0/2104316531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e717510 0x56363e712020 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c006010 tx=0x7f8a9c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9728a2037cd5a4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:08.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.112+0000 7f8a8d600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:39:08.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.112+0000 7f8a8d600640 1 -- 172.21.15.89:0/2104316531 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c034040 con 0x56363e717510 2024-09-10T08:39:08.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.112+0000 7f8aad07a740 1 -- 172.21.15.89:0/2104316531 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a9002ca60 con 0x56363e717510 2024-09-10T08:39:08.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.112+0000 7f8a8d600640 1 -- 172.21.15.89:0/2104316531 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f8a9c004250 con 0x56363e717510 2024-09-10T08:39:08.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.112+0000 7f8a8d600640 1 -- 172.21.15.89:0/2104316531 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8a9c05c050 con 0x56363e717510 2024-09-10T08:39:08.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.112+0000 7f8aad07a740 10 client.25034 fetch_fsmap learned FSMap version 25 2024-09-10T08:39:08.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.113+0000 7f8aad07a740 1 -- 172.21.15.89:0/2104316531 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56363e792d30 con 0x56363e717510 2024-09-10T08:39:08.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.113+0000 7f8a8d600640 1 -- 172.21.15.89:0/2104316531 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9002ca60 con 0x56363e717510 2024-09-10T08:39:08.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.113+0000 7f8a8d600640 1 -- 172.21.15.89:0/2104316531 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8a9c03a430 con 0x56363e717510 2024-09-10T08:39:08.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.113+0000 7f8aad07a740 10 client.25034 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:39:08.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.113+0000 7f8aad07a740 10 client.25034 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:39:08.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.113+0000 7f8aad07a740 20 client.25034 populate_metadata read hostname 'smithi089' 2024-09-10T08:39:08.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.113+0000 7f8aad07a740 1 --2- 172.21.15.89:0/2104316531 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56363e797a10 0x56363e796720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:08.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.113+0000 7f8aad07a740 4 client.25034 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:39:08.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.113+0000 7f8aad07a740 1 -- 172.21.15.89:0/2104316531 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "status"}) -- 0x56363e679b30 con 0x56363e797a10 2024-09-10T08:39:08.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.114+0000 7f8aa3400640 1 --2- 172.21.15.89:0/2104316531 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56363e797a10 0x56363e796720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:08.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.114+0000 7f8aa3400640 1 --2- 172.21.15.89:0/2104316531 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56363e797a10 0x56363e796720 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e93149f7a11cc960 server_cookie=23b5d5d65efcf9c9 in_seq=0 out_seq=0 2024-09-10T08:39:08.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.114+0000 7f8a8d600640 10 client.25034 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:39:08.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.114+0000 7f8a8d600640 1 -- 172.21.15.89:0/2104316531 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56363e679b30 con 0x56363e797a10 2024-09-10T08:39:08.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.114+0000 7f8a8d600640 10 client.25034 handle_command_reply: tid=0 2024-09-10T08:39:08.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.114+0000 7f8aad07a740 1 client.25034 shutdown 2024-09-10T08:39:08.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.115+0000 7f8aad07a740 1 -- 172.21.15.89:0/2104316531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e717510 msgr2=0x56363e712020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:08.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.115+0000 7f8aad07a740 1 --2- 172.21.15.89:0/2104316531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e717510 0x56363e712020 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c006010 tx=0x7f8a9c0095c0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.115+0000 7f8aad07a740 1 -- 172.21.15.89:0/2104316531 shutdown_connections 2024-09-10T08:39:08.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.115+0000 7f8aad07a740 1 --2- 172.21.15.89:0/2104316531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56363e717510 0x56363e712020 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.115+0000 7f8aad07a740 1 --2- 172.21.15.89:0/2104316531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56363e70cc50 0x56363e711ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.115+0000 7f8aad07a740 1 --2- 172.21.15.89:0/2104316531 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56363e797a10 0x56363e796720 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.115+0000 7f8aad07a740 1 --2- 172.21.15.89:0/2104316531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56363e70c330 0x56363e7115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.115+0000 7f8aad07a740 1 -- 172.21.15.89:0/2104316531 >> 172.21.15.89:0/2104316531 conn(0x56363e7088c0 msgr2=0x56363e712e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:08.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.115+0000 7f8a8d600640 0 client.25034 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:39:08.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.115+0000 7f8aad07a740 1 -- 172.21.15.89:0/2104316531 shutdown_connections 2024-09-10T08:39:08.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.115+0000 7f8aad07a740 1 -- 172.21.15.89:0/2104316531 wait complete. 2024-09-10T08:39:08.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.115+0000 7f8aad07a740 20 client.25034 trim_cache size 0 max 16384 2024-09-10T08:39:08.118 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:39:08.118 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "91f747bc-6f4e-11ef-bcea-c7b262605968", 2024-09-10T08:39:08.118 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2024-09-10T08:39:08.118 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24554, 2024-09-10T08:39:08.118 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2024-09-10T08:39:08.118 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2024-09-10T08:39:08.118 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2024-09-10T08:39:08.118 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 129.96678163600001, 2024-09-10T08:39:08.119 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 25, 2024-09-10T08:39:08.119 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 96, 2024-09-10T08:39:08.119 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T08:39:08.119 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 142.98024622200001 2024-09-10T08:39:08.119 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:39:08.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.116+0000 7f8aaa600640 1 -- 172.21.15.89:0/3105956769 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8a880a2c10 msgr2=0x7f8a880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:08.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.116+0000 7f8aaa600640 1 --2- 172.21.15.89:0/3105956769 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8a880a2c10 0x7f8a880a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c009ca0 tx=0x7f8a9c005c50 comp rx=0 tx=0).stop 2024-09-10T08:39:08.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.116+0000 7f8aaa600640 1 -- 172.21.15.89:0/3105956769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8aa4074590 msgr2=0x7f8aa41a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:08.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.116+0000 7f8aaa600640 1 --2- 172.21.15.89:0/3105956769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8aa4074590 0x7f8aa41a4720 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400d950 tx=0x7f8a9400de10 comp rx=0 tx=0).stop 2024-09-10T08:39:08.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.116+0000 7f8aaa600640 1 -- 172.21.15.89:0/3105956769 shutdown_connections 2024-09-10T08:39:08.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.116+0000 7f8aaa600640 1 --2- 172.21.15.89:0/3105956769 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8a880a2c10 0x7f8a880a50d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.116+0000 7f8aaa600640 1 --2- 172.21.15.89:0/3105956769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8aa4112b90 0x7f8aa419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.116+0000 7f8aaa600640 1 --2- 172.21.15.89:0/3105956769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8aa4074eb0 0x7f8aa41a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.116+0000 7f8aaa600640 1 --2- 172.21.15.89:0/3105956769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8aa4074590 0x7f8aa41a4720 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:08.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.116+0000 7f8aaa600640 1 -- 172.21.15.89:0/3105956769 >> 172.21.15.89:0/3105956769 conn(0x7f8aa40fd8c0 msgr2=0x7f8aa40ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:08.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.116+0000 7f8aaa600640 1 -- 172.21.15.89:0/3105956769 shutdown_connections 2024-09-10T08:39:08.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:08.116+0000 7f8aaa600640 1 -- 172.21.15.89:0/3105956769 wait complete. 2024-09-10T08:39:08.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:08 smithi089 ceph-mon[30757]: pgmap v369: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:39:08.883 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T08:39:08.883 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T08:39:08.894 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi089.front.sepia.ceph.com 2024-09-10T08:39:08.894 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-10T08:39:08.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:08 smithi149 ceph-mon[36167]: pgmap v369: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:39:09.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:08 smithi111 ceph-mon[35220]: pgmap v369: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:39:09.259 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:09.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.872+0000 7ff534a00640 1 -- 172.21.15.89:0/1759411249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5301098e0 msgr2=0x7ff53010bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:09.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.872+0000 7ff534a00640 1 --2- 172.21.15.89:0/1759411249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5301098e0 0x7ff53010bd80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff528009ca0 tx=0x7ff52802a950 comp rx=0 tx=0).stop 2024-09-10T08:39:09.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.872+0000 7ff534a00640 1 -- 172.21.15.89:0/1759411249 shutdown_connections 2024-09-10T08:39:09.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.872+0000 7ff534a00640 1 --2- 172.21.15.89:0/1759411249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5301098e0 0x7ff53010bd80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:09.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.872+0000 7ff534a00640 1 --2- 172.21.15.89:0/1759411249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff530074bf0 0x7ff530075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:09.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.872+0000 7ff534a00640 1 --2- 172.21.15.89:0/1759411249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5300761c0 0x7ff530074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:09.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.872+0000 7ff534a00640 1 -- 172.21.15.89:0/1759411249 >> 172.21.15.89:0/1759411249 conn(0x7ff5300fd1f0 msgr2=0x7ff5300ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:09.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.872+0000 7ff534a00640 1 -- 172.21.15.89:0/1759411249 shutdown_connections 2024-09-10T08:39:09.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.872+0000 7ff534a00640 1 -- 172.21.15.89:0/1759411249 wait complete. 2024-09-10T08:39:09.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.873+0000 7ff534a00640 1 Processor -- start 2024-09-10T08:39:09.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.873+0000 7ff534a00640 1 -- start start 2024-09-10T08:39:09.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.873+0000 7ff534a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff530074bf0 0x7ff530199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:09.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.873+0000 7ff534a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5300761c0 0x7ff53019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:09.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.873+0000 7ff534a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5301098e0 0x7ff5301a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:09.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.873+0000 7ff534a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff53010f8a0 con 0x7ff530074bf0 2024-09-10T08:39:09.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.873+0000 7ff534a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff53010f720 con 0x7ff5300761c0 2024-09-10T08:39:09.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.873+0000 7ff534a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff53010fa20 con 0x7ff5301098e0 2024-09-10T08:39:09.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.873+0000 7ff52e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff530074bf0 0x7ff530199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:09.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.873+0000 7ff52e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff530074bf0 0x7ff530199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52272/0 (socket says 172.21.15.89:52272) 2024-09-10T08:39:09.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.873+0000 7ff52e800640 1 -- 172.21.15.89:0/301890888 learned_addr learned my addr 172.21.15.89:0/301890888 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:09.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.874+0000 7ff52de00640 1 --2- 172.21.15.89:0/301890888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5300761c0 0x7ff53019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:09.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.874+0000 7ff52f200640 1 --2- 172.21.15.89:0/301890888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5301098e0 0x7ff5301a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:09.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.874+0000 7ff52e800640 1 -- 172.21.15.89:0/301890888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5301098e0 msgr2=0x7ff5301a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:09.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.874+0000 7ff52e800640 1 --2- 172.21.15.89:0/301890888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5301098e0 0x7ff5301a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:09.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.874+0000 7ff52e800640 1 -- 172.21.15.89:0/301890888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5300761c0 msgr2=0x7ff53019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:09.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.874+0000 7ff52e800640 1 --2- 172.21.15.89:0/301890888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5300761c0 0x7ff53019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:09.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.874+0000 7ff52e800640 1 -- 172.21.15.89:0/301890888 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5301a1b00 con 0x7ff530074bf0 2024-09-10T08:39:09.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.874+0000 7ff52de00640 1 --2- 172.21.15.89:0/301890888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5300761c0 0x7ff53019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:39:09.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.874+0000 7ff52f200640 1 --2- 172.21.15.89:0/301890888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5301098e0 0x7ff5301a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:39:09.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.874+0000 7ff52e800640 1 --2- 172.21.15.89:0/301890888 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff530074bf0 0x7ff530199ed0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff52000e9f0 tx=0x7ff52000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:09.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.874+0000 7ff527400640 1 -- 172.21.15.89:0/301890888 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52000cd80 con 0x7ff530074bf0 2024-09-10T08:39:09.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.874+0000 7ff527400640 1 -- 172.21.15.89:0/301890888 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff520010910 con 0x7ff530074bf0 2024-09-10T08:39:09.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.874+0000 7ff527400640 1 -- 172.21.15.89:0/301890888 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520018990 con 0x7ff530074bf0 2024-09-10T08:39:09.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.874+0000 7ff534a00640 1 -- 172.21.15.89:0/301890888 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5301a1df0 con 0x7ff530074bf0 2024-09-10T08:39:09.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.875+0000 7ff534a00640 1 -- 172.21.15.89:0/301890888 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5301a21f0 con 0x7ff530074bf0 2024-09-10T08:39:09.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.876+0000 7ff534a00640 1 -- 172.21.15.89:0/301890888 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5301086a0 con 0x7ff530074bf0 2024-09-10T08:39:09.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.878+0000 7ff527400640 1 -- 172.21.15.89:0/301890888 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff520010ac0 con 0x7ff530074bf0 2024-09-10T08:39:09.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.879+0000 7ff527400640 1 --2- 172.21.15.89:0/301890888 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff5140a2cc0 0x7ff5140a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:09.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.879+0000 7ff527400640 1 -- 172.21.15.89:0/301890888 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff5200cf970 con 0x7ff530074bf0 2024-09-10T08:39:09.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.879+0000 7ff52de00640 1 --2- 172.21.15.89:0/301890888 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff5140a2cc0 0x7ff5140a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:09.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.880+0000 7ff52de00640 1 --2- 172.21.15.89:0/301890888 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff5140a2cc0 0x7ff5140a5180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff53019b3f0 tx=0x7ff51c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:09.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:09.883+0000 7ff527400640 1 -- 172.21.15.89:0/301890888 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff52009c340 con 0x7ff530074bf0 2024-09-10T08:39:10.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:10.074+0000 7ff534a00640 1 -- 172.21.15.89:0/301890888 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7ff530062630 con 0x7ff5140a2cc0 2024-09-10T08:39:10.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:10.075+0000 7ff527400640 1 -- 172.21.15.89:0/301890888 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7ff530062630 con 0x7ff5140a2cc0 2024-09-10T08:39:10.075 INFO:teuthology.orchestra.run.smithi089.stdout:Backend: cephadm 2024-09-10T08:39:10.075 INFO:teuthology.orchestra.run.smithi089.stdout:Available: Yes 2024-09-10T08:39:10.075 INFO:teuthology.orchestra.run.smithi089.stdout:Paused: No 2024-09-10T08:39:10.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:10.077+0000 7ff534a00640 1 -- 172.21.15.89:0/301890888 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff5140a2cc0 msgr2=0x7ff5140a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:10.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:10.077+0000 7ff534a00640 1 --2- 172.21.15.89:0/301890888 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff5140a2cc0 0x7ff5140a5180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff53019b3f0 tx=0x7ff51c005f90 comp rx=0 tx=0).stop 2024-09-10T08:39:10.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:10.077+0000 7ff534a00640 1 -- 172.21.15.89:0/301890888 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff530074bf0 msgr2=0x7ff530199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:10.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:10.077+0000 7ff534a00640 1 --2- 172.21.15.89:0/301890888 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff530074bf0 0x7ff530199ed0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff52000e9f0 tx=0x7ff52000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:39:10.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:10.077+0000 7ff534a00640 1 -- 172.21.15.89:0/301890888 shutdown_connections 2024-09-10T08:39:10.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:10.077+0000 7ff534a00640 1 --2- 172.21.15.89:0/301890888 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff5140a2cc0 0x7ff5140a5180 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:10.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:10.077+0000 7ff534a00640 1 --2- 172.21.15.89:0/301890888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5301098e0 0x7ff5301a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:10.077 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:10.077+0000 7ff534a00640 1 --2- 172.21.15.89:0/301890888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5300761c0 0x7ff53019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:10.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:10.077+0000 7ff534a00640 1 --2- 172.21.15.89:0/301890888 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff530074bf0 0x7ff530199ed0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:10.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:10.077+0000 7ff534a00640 1 -- 172.21.15.89:0/301890888 >> 172.21.15.89:0/301890888 conn(0x7ff5300fd1f0 msgr2=0x7ff530105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:10.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:10.077+0000 7ff534a00640 1 -- 172.21.15.89:0/301890888 shutdown_connections 2024-09-10T08:39:10.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:10.077+0000 7ff534a00640 1 -- 172.21.15.89:0/301890888 wait complete. 2024-09-10T08:39:10.742 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:10 smithi089 ceph-mon[30757]: pgmap v370: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:10.780 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-10T08:39:10.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:10 smithi149 ceph-mon[36167]: pgmap v370: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:11.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:10 smithi111 ceph-mon[35220]: pgmap v370: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:11.149 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:11.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.766+0000 7f146be00640 1 -- 172.21.15.89:0/265624882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f146c103c50 msgr2=0x7f146c104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:11.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.766+0000 7f146be00640 1 --2- 172.21.15.89:0/265624882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f146c103c50 0x7f146c104050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f145c009900 tx=0x7f145c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:39:11.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.766+0000 7f146be00640 1 -- 172.21.15.89:0/265624882 shutdown_connections 2024-09-10T08:39:11.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.766+0000 7f146be00640 1 --2- 172.21.15.89:0/265624882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f146c105810 0x7f146c10c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:11.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.766+0000 7f146be00640 1 --2- 172.21.15.89:0/265624882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f146c104e50 0x7f146c1052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:11.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.766+0000 7f146be00640 1 --2- 172.21.15.89:0/265624882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f146c103c50 0x7f146c104050 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:11.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.766+0000 7f146be00640 1 -- 172.21.15.89:0/265624882 >> 172.21.15.89:0/265624882 conn(0x7f146c0ff3e0 msgr2=0x7f146c101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:11.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.766+0000 7f146be00640 1 -- 172.21.15.89:0/265624882 shutdown_connections 2024-09-10T08:39:11.781 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.766+0000 7f146be00640 1 -- 172.21.15.89:0/265624882 wait complete. 2024-09-10T08:39:11.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.767+0000 7f146be00640 1 Processor -- start 2024-09-10T08:39:11.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.767+0000 7f146be00640 1 -- start start 2024-09-10T08:39:11.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.768+0000 7f146be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f146c103c50 0x7f146c199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:11.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.768+0000 7f146be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f146c104e50 0x7f146c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:11.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.768+0000 7f146be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f146c105810 0x7f146c1a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:11.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.768+0000 7f146be00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f146c10f8b0 con 0x7f146c104e50 2024-09-10T08:39:11.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.768+0000 7f146be00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f146c10f730 con 0x7f146c105810 2024-09-10T08:39:11.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.768+0000 7f146be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f146c10fa30 con 0x7f146c103c50 2024-09-10T08:39:11.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.768+0000 7f146a000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f146c104e50 0x7f146c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:11.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.768+0000 7f146b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f146c105810 0x7f146c1a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:11.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.768+0000 7f146aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f146c103c50 0x7f146c199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:11.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.768+0000 7f146b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f146c105810 0x7f146c1a14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:45212/0 (socket says 172.21.15.89:45212) 2024-09-10T08:39:11.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.768+0000 7f146b400640 1 -- 172.21.15.89:0/530196944 learned_addr learned my addr 172.21.15.89:0/530196944 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:11.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.768+0000 7f146aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f146c103c50 0x7f146c199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:42668/0 (socket says 172.21.15.89:42668) 2024-09-10T08:39:11.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.768+0000 7f146b400640 1 -- 172.21.15.89:0/530196944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f146c103c50 msgr2=0x7f146c199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:11.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.768+0000 7f146b400640 1 --2- 172.21.15.89:0/530196944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f146c103c50 0x7f146c199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:11.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.768+0000 7f146b400640 1 -- 172.21.15.89:0/530196944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f146c104e50 msgr2=0x7f146c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:11.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.768+0000 7f146b400640 1 --2- 172.21.15.89:0/530196944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f146c104e50 0x7f146c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:11.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.768+0000 7f146b400640 1 -- 172.21.15.89:0/530196944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146c1a1b60 con 0x7f146c105810 2024-09-10T08:39:11.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.769+0000 7f146b400640 1 --2- 172.21.15.89:0/530196944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f146c105810 0x7f146c1a14f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f146400eb10 tx=0x7f146400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:11.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.769+0000 7f1470a00640 1 -- 172.21.15.89:0/530196944 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464004b20 con 0x7f146c105810 2024-09-10T08:39:11.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.769+0000 7f1470a00640 1 -- 172.21.15.89:0/530196944 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1464004cc0 con 0x7f146c105810 2024-09-10T08:39:11.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.769+0000 7f146be00640 1 -- 172.21.15.89:0/530196944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146c1a1e50 con 0x7f146c105810 2024-09-10T08:39:11.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.769+0000 7f1470a00640 1 -- 172.21.15.89:0/530196944 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464010760 con 0x7f146c105810 2024-09-10T08:39:11.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.769+0000 7f146be00640 1 -- 172.21.15.89:0/530196944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f146c1a22b0 con 0x7f146c105810 2024-09-10T08:39:11.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.770+0000 7f146be00640 1 -- 172.21.15.89:0/530196944 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f146c10b050 con 0x7f146c105810 2024-09-10T08:39:11.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.772+0000 7f1470a00640 1 -- 172.21.15.89:0/530196944 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1464005020 con 0x7f146c105810 2024-09-10T08:39:11.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.773+0000 7f1470a00640 1 --2- 172.21.15.89:0/530196944 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14500a2c40 0x7f14500a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:11.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.773+0000 7f1470a00640 1 -- 172.21.15.89:0/530196944 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f14640cfb80 con 0x7f146c105810 2024-09-10T08:39:11.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.773+0000 7f146aa00640 1 --2- 172.21.15.89:0/530196944 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14500a2c40 0x7f14500a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:11.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.774+0000 7f146aa00640 1 --2- 172.21.15.89:0/530196944 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14500a2c40 0x7f14500a5100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f145c005ab0 tx=0x7f145c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:11.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.777+0000 7f1470a00640 1 -- 172.21.15.89:0/530196944 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f146409c550 con 0x7f146c105810 2024-09-10T08:39:11.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.962+0000 7f146be00640 1 -- 172.21.15.89:0/530196944 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f146c062630 con 0x7f14500a2c40 2024-09-10T08:39:11.963 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:11 smithi089 ceph-mon[30757]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:11.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:11 smithi149 ceph-mon[36167]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:11.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.974+0000 7f1470a00640 1 -- 172.21.15.89:0/530196944 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7f146c062630 con 0x7f14500a2c40 2024-09-10T08:39:11.975 INFO:teuthology.orchestra.run.smithi089.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T08:39:11.975 INFO:teuthology.orchestra.run.smithi089.stdout:mds.a smithi089 running (2m) 11s ago 2m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 d384f926c0a9 2024-09-10T08:39:11.975 INFO:teuthology.orchestra.run.smithi089.stdout:mds.b smithi111 running (2m) 13s ago 2m 26.0M - 19.1.1-305-g1cf819b7 41d671128357 d725d119605e 2024-09-10T08:39:11.975 INFO:teuthology.orchestra.run.smithi089.stdout:mds.c smithi149 running (2m) 13s ago 2m 17.2M - 19.1.1-305-g1cf819b7 41d671128357 a437630f0b55 2024-09-10T08:39:11.975 INFO:teuthology.orchestra.run.smithi089.stdout:mds.d smithi089 running (2m) 11s ago 2m 13.2M - 19.1.1-305-g1cf819b7 41d671128357 50ead59fd148 2024-09-10T08:39:11.975 INFO:teuthology.orchestra.run.smithi089.stdout:mds.e smithi111 running (2m) 13s ago 2m 18.5M - 19.1.1-305-g1cf819b7 41d671128357 c196a0b6f6fc 2024-09-10T08:39:11.975 INFO:teuthology.orchestra.run.smithi089.stdout:mds.f smithi149 running (2m) 13s ago 2m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 67b01eafeae7 2024-09-10T08:39:11.976 INFO:teuthology.orchestra.run.smithi089.stdout:mds.g smithi089 running (2m) 11s ago 2m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e499f39fca99 2024-09-10T08:39:11.976 INFO:teuthology.orchestra.run.smithi089.stdout:mds.h smithi111 running (2m) 13s ago 2m 18.6M - 19.1.1-305-g1cf819b7 41d671128357 34d380ce62f1 2024-09-10T08:39:11.976 INFO:teuthology.orchestra.run.smithi089.stdout:mds.i smithi149 running (2m) 13s ago 2m 20.0M - 19.1.1-305-g1cf819b7 41d671128357 f9e956b83d57 2024-09-10T08:39:11.976 INFO:teuthology.orchestra.run.smithi089.stdout:mds.j smithi089 running (2m) 11s ago 2m 13.2M - 19.1.1-305-g1cf819b7 41d671128357 2d77cfd89db9 2024-09-10T08:39:11.976 INFO:teuthology.orchestra.run.smithi089.stdout:mds.k smithi111 running (2m) 13s ago 2m 13.8M - 19.1.1-305-g1cf819b7 41d671128357 33aea65ed9d8 2024-09-10T08:39:11.976 INFO:teuthology.orchestra.run.smithi089.stdout:mds.l smithi149 running (3m) 13s ago 2m 16.3M - 19.1.1-305-g1cf819b7 41d671128357 2d0dd4a58e26 2024-09-10T08:39:11.976 INFO:teuthology.orchestra.run.smithi089.stdout:mgr.x smithi089 *:9283,8765 running (10m) 11s ago 10m 509M - 19.1.1-305-g1cf819b7 41d671128357 68b20d409ffe 2024-09-10T08:39:11.976 INFO:teuthology.orchestra.run.smithi089.stdout:mgr.y smithi111 *:8443,8765 running (8m) 13s ago 8m 442M - 19.1.1-305-g1cf819b7 41d671128357 1e5793fe7519 2024-09-10T08:39:11.976 INFO:teuthology.orchestra.run.smithi089.stdout:mgr.z smithi149 *:8443,8765 running (8m) 13s ago 8m 442M - 19.1.1-305-g1cf819b7 41d671128357 ee474161c8b2 2024-09-10T08:39:11.976 INFO:teuthology.orchestra.run.smithi089.stdout:mon.a smithi089 running (10m) 11s ago 10m 50.0M 2048M 19.1.1-305-g1cf819b7 41d671128357 975872fc80f0 2024-09-10T08:39:11.976 INFO:teuthology.orchestra.run.smithi089.stdout:mon.b smithi111 running (8m) 13s ago 8m 41.5M 2048M 19.1.1-305-g1cf819b7 41d671128357 041932f325a2 2024-09-10T08:39:11.976 INFO:teuthology.orchestra.run.smithi089.stdout:mon.c smithi149 running (8m) 13s ago 8m 43.0M 2048M 19.1.1-305-g1cf819b7 41d671128357 4f4d000116de 2024-09-10T08:39:11.976 INFO:teuthology.orchestra.run.smithi089.stdout:osd.0 smithi089 running (8m) 11s ago 8m 55.3M 4272M 19.1.1-305-g1cf819b7 41d671128357 f3b906760536 2024-09-10T08:39:11.977 INFO:teuthology.orchestra.run.smithi089.stdout:osd.1 smithi111 running (7m) 13s ago 7m 59.4M 4272M 19.1.1-305-g1cf819b7 41d671128357 e145fb0578cb 2024-09-10T08:39:11.977 INFO:teuthology.orchestra.run.smithi089.stdout:osd.2 smithi149 running (7m) 13s ago 7m 46.5M 1453M 19.1.1-305-g1cf819b7 41d671128357 2f97966fd9b0 2024-09-10T08:39:11.977 INFO:teuthology.orchestra.run.smithi089.stdout:osd.3 smithi089 running (6m) 11s ago 6m 54.8M 4272M 19.1.1-305-g1cf819b7 41d671128357 d2172eaee4d2 2024-09-10T08:39:11.977 INFO:teuthology.orchestra.run.smithi089.stdout:osd.4 smithi111 running (6m) 13s ago 6m 54.3M 4272M 19.1.1-305-g1cf819b7 41d671128357 75b1494d5a72 2024-09-10T08:39:11.977 INFO:teuthology.orchestra.run.smithi089.stdout:osd.5 smithi149 running (6m) 13s ago 6m 47.2M 1453M 19.1.1-305-g1cf819b7 41d671128357 0fdf25d0d4be 2024-09-10T08:39:11.977 INFO:teuthology.orchestra.run.smithi089.stdout:osd.6 smithi089 running (5m) 11s ago 5m 41.4M 4272M 19.1.1-305-g1cf819b7 41d671128357 31ff944bd932 2024-09-10T08:39:11.977 INFO:teuthology.orchestra.run.smithi089.stdout:osd.7 smithi111 running (5m) 13s ago 5m 46.2M 4272M 19.1.1-305-g1cf819b7 41d671128357 922f7f563448 2024-09-10T08:39:11.977 INFO:teuthology.orchestra.run.smithi089.stdout:osd.8 smithi149 running (4m) 13s ago 4m 54.9M 1453M 19.1.1-305-g1cf819b7 41d671128357 000059b9f303 2024-09-10T08:39:11.977 INFO:teuthology.orchestra.run.smithi089.stdout:osd.9 smithi089 running (4m) 11s ago 4m 44.0M 4272M 19.1.1-305-g1cf819b7 41d671128357 197f27152e48 2024-09-10T08:39:11.977 INFO:teuthology.orchestra.run.smithi089.stdout:osd.10 smithi111 running (3m) 13s ago 3m 43.7M 4272M 19.1.1-305-g1cf819b7 41d671128357 8dd49e38f9b2 2024-09-10T08:39:11.977 INFO:teuthology.orchestra.run.smithi089.stdout:osd.11 smithi149 running (3m) 13s ago 3m 44.7M 1453M 19.1.1-305-g1cf819b7 41d671128357 0b95f8eee16b 2024-09-10T08:39:11.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.977+0000 7f146be00640 1 -- 172.21.15.89:0/530196944 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14500a2c40 msgr2=0x7f14500a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:11.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.977+0000 7f146be00640 1 --2- 172.21.15.89:0/530196944 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14500a2c40 0x7f14500a5100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f145c005ab0 tx=0x7f145c000bc0 comp rx=0 tx=0).stop 2024-09-10T08:39:11.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.977+0000 7f146be00640 1 -- 172.21.15.89:0/530196944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f146c105810 msgr2=0x7f146c1a14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:11.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.977+0000 7f146be00640 1 --2- 172.21.15.89:0/530196944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f146c105810 0x7f146c1a14f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f146400eb10 tx=0x7f146400c2d0 comp rx=0 tx=0).stop 2024-09-10T08:39:11.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.977+0000 7f146be00640 1 -- 172.21.15.89:0/530196944 shutdown_connections 2024-09-10T08:39:11.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.977+0000 7f146be00640 1 --2- 172.21.15.89:0/530196944 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14500a2c40 0x7f14500a5100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:11.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.977+0000 7f146be00640 1 --2- 172.21.15.89:0/530196944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f146c105810 0x7f146c1a14f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:11.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.977+0000 7f146be00640 1 --2- 172.21.15.89:0/530196944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f146c104e50 0x7f146c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:11.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.977+0000 7f146be00640 1 --2- 172.21.15.89:0/530196944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f146c103c50 0x7f146c199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:11.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.977+0000 7f146be00640 1 -- 172.21.15.89:0/530196944 >> 172.21.15.89:0/530196944 conn(0x7f146c0ff3e0 msgr2=0x7f146c108220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:11.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.977+0000 7f146be00640 1 -- 172.21.15.89:0/530196944 shutdown_connections 2024-09-10T08:39:11.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:11.977+0000 7f146be00640 1 -- 172.21.15.89:0/530196944 wait complete. 2024-09-10T08:39:12.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:11 smithi111 ceph-mon[35220]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:12.732 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-10T08:39:12.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:12 smithi149 ceph-mon[36167]: pgmap v371: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T08:39:13.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:12 smithi111 ceph-mon[35220]: pgmap v371: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T08:39:13.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:12 smithi089 ceph-mon[30757]: pgmap v371: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T08:39:13.098 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.684+0000 7fe428600640 1 -- 172.21.15.89:0/3748037966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4201037c0 msgr2=0x7fe42010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.684+0000 7fe428600640 1 --2- 172.21.15.89:0/3748037966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4201037c0 0x7fe42010a050 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe418009ca0 tx=0x7fe41802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:39:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.685+0000 7fe428600640 1 -- 172.21.15.89:0/3748037966 shutdown_connections 2024-09-10T08:39:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.685+0000 7fe428600640 1 --2- 172.21.15.89:0/3748037966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4201037c0 0x7fe42010a050 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.685+0000 7fe428600640 1 --2- 172.21.15.89:0/3748037966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe420102e00 0x7fe420103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.685+0000 7fe428600640 1 --2- 172.21.15.89:0/3748037966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe420101c00 0x7fe420102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.685+0000 7fe428600640 1 -- 172.21.15.89:0/3748037966 >> 172.21.15.89:0/3748037966 conn(0x7fe4200fd390 msgr2=0x7fe4200ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.685+0000 7fe428600640 1 -- 172.21.15.89:0/3748037966 shutdown_connections 2024-09-10T08:39:13.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.685+0000 7fe428600640 1 -- 172.21.15.89:0/3748037966 wait complete. 2024-09-10T08:39:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.686+0000 7fe428600640 1 Processor -- start 2024-09-10T08:39:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.686+0000 7fe428600640 1 -- start start 2024-09-10T08:39:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.686+0000 7fe428600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe420101c00 0x7fe42019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:13.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.686+0000 7fe428600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe420102e00 0x7fe42019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.686+0000 7fe428600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4201037c0 0x7fe4201a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.686+0000 7fe428600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe42010d880 con 0x7fe420102e00 2024-09-10T08:39:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.686+0000 7fe428600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe42010d700 con 0x7fe4201037c0 2024-09-10T08:39:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.686+0000 7fe428600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe42010da00 con 0x7fe420101c00 2024-09-10T08:39:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.686+0000 7fe425200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe420102e00 0x7fe42019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.686+0000 7fe425c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe420101c00 0x7fe42019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.686+0000 7fe425200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe420102e00 0x7fe42019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47234/0 (socket says 172.21.15.89:47234) 2024-09-10T08:39:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.686+0000 7fe425200640 1 -- 172.21.15.89:0/1356846810 learned_addr learned my addr 172.21.15.89:0/1356846810 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.687+0000 7fe426600640 1 --2- 172.21.15.89:0/1356846810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4201037c0 0x7fe4201a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.687+0000 7fe425c00640 1 -- 172.21.15.89:0/1356846810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4201037c0 msgr2=0x7fe4201a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.687+0000 7fe425c00640 1 --2- 172.21.15.89:0/1356846810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4201037c0 0x7fe4201a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.687+0000 7fe425c00640 1 -- 172.21.15.89:0/1356846810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe420102e00 msgr2=0x7fe42019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:13.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.687+0000 7fe425c00640 1 --2- 172.21.15.89:0/1356846810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe420102e00 0x7fe42019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.687+0000 7fe425c00640 1 -- 172.21.15.89:0/1356846810 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4201a1cf0 con 0x7fe420101c00 2024-09-10T08:39:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.687+0000 7fe425c00640 1 --2- 172.21.15.89:0/1356846810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe420101c00 0x7fe42019a0c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe41400e9c0 tx=0x7fe41400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.687+0000 7fe41e000640 1 -- 172.21.15.89:0/1356846810 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41400cd80 con 0x7fe420101c00 2024-09-10T08:39:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.687+0000 7fe41e000640 1 -- 172.21.15.89:0/1356846810 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe414010910 con 0x7fe420101c00 2024-09-10T08:39:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.687+0000 7fe428600640 1 -- 172.21.15.89:0/1356846810 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4201a1fe0 con 0x7fe420101c00 2024-09-10T08:39:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.688+0000 7fe41e000640 1 -- 172.21.15.89:0/1356846810 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414018980 con 0x7fe420101c00 2024-09-10T08:39:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.688+0000 7fe428600640 1 -- 172.21.15.89:0/1356846810 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4201a2430 con 0x7fe420101c00 2024-09-10T08:39:13.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.689+0000 7fe428600640 1 -- 172.21.15.89:0/1356846810 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4201a2690 con 0x7fe420101c00 2024-09-10T08:39:13.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.690+0000 7fe41e000640 1 -- 172.21.15.89:0/1356846810 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe41401f020 con 0x7fe420101c00 2024-09-10T08:39:13.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.692+0000 7fe41e000640 1 --2- 172.21.15.89:0/1356846810 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4080a2bb0 0x7fe4080a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:13.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.692+0000 7fe41e000640 1 -- 172.21.15.89:0/1356846810 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fe4140cf420 con 0x7fe420101c00 2024-09-10T08:39:13.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.692+0000 7fe425200640 1 --2- 172.21.15.89:0/1356846810 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4080a2bb0 0x7fe4080a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:13.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.693+0000 7fe425200640 1 --2- 172.21.15.89:0/1356846810 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4080a2bb0 0x7fe4080a5070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe42019b5e0 tx=0x7fe40c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:13.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.696+0000 7fe41e000640 1 -- 172.21.15.89:0/1356846810 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe41409bf50 con 0x7fe420101c00 2024-09-10T08:39:13.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.892+0000 7fe428600640 1 -- 172.21.15.89:0/1356846810 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fe420075b40 con 0x7fe4080a2bb0 2024-09-10T08:39:13.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.898+0000 7fe41e000640 1 -- 172.21.15.89:0/1356846810 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fe420075b40 con 0x7fe4080a2bb0 2024-09-10T08:39:13.899 INFO:teuthology.orchestra.run.smithi089.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-10T08:39:13.899 INFO:teuthology.orchestra.run.smithi089.stdout:mds.all 12/12 15s ago 3m smithi089=a;smithi089=d;smithi089=g;smithi089=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2024-09-10T08:39:13.899 INFO:teuthology.orchestra.run.smithi089.stdout:mgr 3/3 15s ago 8m smithi089=x;smithi111=y;smithi149=z;count:3 2024-09-10T08:39:13.899 INFO:teuthology.orchestra.run.smithi089.stdout:mon 3/3 15s ago 9m smithi089:172.21.15.89=a;smithi111:172.21.15.111=b;smithi149:172.21.15.149=c;count:3 2024-09-10T08:39:13.899 INFO:teuthology.orchestra.run.smithi089.stdout:osd 12 15s ago - 2024-09-10T08:39:13.899 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:13 smithi089 ceph-mon[30757]: from='client.25046 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:13.900 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:13 smithi089 ceph-mon[30757]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T08:39:13.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.900+0000 7fe428600640 1 -- 172.21.15.89:0/1356846810 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4080a2bb0 msgr2=0x7fe4080a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:13.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.900+0000 7fe428600640 1 --2- 172.21.15.89:0/1356846810 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4080a2bb0 0x7fe4080a5070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe42019b5e0 tx=0x7fe40c005f90 comp rx=0 tx=0).stop 2024-09-10T08:39:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.900+0000 7fe428600640 1 -- 172.21.15.89:0/1356846810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe420101c00 msgr2=0x7fe42019a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.900+0000 7fe428600640 1 --2- 172.21.15.89:0/1356846810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe420101c00 0x7fe42019a0c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe41400e9c0 tx=0x7fe41400ee80 comp rx=0 tx=0).stop 2024-09-10T08:39:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.900+0000 7fe428600640 1 -- 172.21.15.89:0/1356846810 shutdown_connections 2024-09-10T08:39:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.900+0000 7fe428600640 1 --2- 172.21.15.89:0/1356846810 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4080a2bb0 0x7fe4080a5070 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.900+0000 7fe428600640 1 --2- 172.21.15.89:0/1356846810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4201037c0 0x7fe4201a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.900+0000 7fe428600640 1 --2- 172.21.15.89:0/1356846810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe420102e00 0x7fe42019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.900+0000 7fe428600640 1 --2- 172.21.15.89:0/1356846810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe420101c00 0x7fe42019a0c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.900+0000 7fe428600640 1 -- 172.21.15.89:0/1356846810 >> 172.21.15.89:0/1356846810 conn(0x7fe4200fd390 msgr2=0x7fe420074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.901+0000 7fe428600640 1 -- 172.21.15.89:0/1356846810 shutdown_connections 2024-09-10T08:39:13.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:13.901+0000 7fe428600640 1 -- 172.21.15.89:0/1356846810 wait complete. 2024-09-10T08:39:13.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:13 smithi149 ceph-mon[36167]: from='client.25046 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:13.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:13 smithi149 ceph-mon[36167]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T08:39:14.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:13 smithi111 ceph-mon[35220]: from='client.25046 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:14.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:13 smithi111 ceph-mon[35220]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T08:39:14.631 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-10T08:39:14.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:14 smithi149 ceph-mon[36167]: pgmap v372: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:14.999 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:15.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:14 smithi111 ceph-mon[35220]: pgmap v372: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:15.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:14 smithi089 ceph-mon[30757]: pgmap v372: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.594+0000 7f964b200640 1 -- 172.21.15.89:0/2492898145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f964c105810 msgr2=0x7f964c10c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.594+0000 7f964b200640 1 --2- 172.21.15.89:0/2492898145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f964c105810 0x7f964c10c0a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9644009ca0 tx=0x7f964402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:39:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.594+0000 7f964b200640 1 -- 172.21.15.89:0/2492898145 shutdown_connections 2024-09-10T08:39:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.594+0000 7f964b200640 1 --2- 172.21.15.89:0/2492898145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f964c105810 0x7f964c10c0a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.594+0000 7f964b200640 1 --2- 172.21.15.89:0/2492898145 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f964c104e50 0x7f964c1052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.594+0000 7f964b200640 1 --2- 172.21.15.89:0/2492898145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f964c103c50 0x7f964c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.594+0000 7f964b200640 1 -- 172.21.15.89:0/2492898145 >> 172.21.15.89:0/2492898145 conn(0x7f964c0ff3e0 msgr2=0x7f964c101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:15.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.594+0000 7f964b200640 1 -- 172.21.15.89:0/2492898145 shutdown_connections 2024-09-10T08:39:15.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.594+0000 7f964b200640 1 -- 172.21.15.89:0/2492898145 wait complete. 2024-09-10T08:39:15.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.595+0000 7f964b200640 1 Processor -- start 2024-09-10T08:39:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.595+0000 7f964b200640 1 -- start start 2024-09-10T08:39:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.596+0000 7f964b200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f964c103c50 0x7f964c199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.596+0000 7f964b200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f964c104e50 0x7f964c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.596+0000 7f964b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f964c105810 0x7f964c1a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.596+0000 7f964b200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f964c10f8d0 con 0x7f964c103c50 2024-09-10T08:39:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.596+0000 7f964b200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f964c10f750 con 0x7f964c104e50 2024-09-10T08:39:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.596+0000 7f964b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f964c10fa50 con 0x7f964c105810 2024-09-10T08:39:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.596+0000 7f9649400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f964c104e50 0x7f964c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.596+0000 7f9649400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f964c104e50 0x7f964c19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:42330/0 (socket says 172.21.15.89:42330) 2024-09-10T08:39:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.596+0000 7f9649400640 1 -- 172.21.15.89:0/3164641755 learned_addr learned my addr 172.21.15.89:0/3164641755 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:15.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.596+0000 7f9649e00640 1 --2- 172.21.15.89:0/3164641755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f964c103c50 0x7f964c199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:15.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.596+0000 7f964a800640 1 --2- 172.21.15.89:0/3164641755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f964c105810 0x7f964c1a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.596+0000 7f9649400640 1 -- 172.21.15.89:0/3164641755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f964c105810 msgr2=0x7f964c1a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.596+0000 7f9649400640 1 --2- 172.21.15.89:0/3164641755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f964c105810 0x7f964c1a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.596+0000 7f9649400640 1 -- 172.21.15.89:0/3164641755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f964c103c50 msgr2=0x7f964c199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.596+0000 7f9649400640 1 --2- 172.21.15.89:0/3164641755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f964c103c50 0x7f964c199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.596+0000 7f9649400640 1 -- 172.21.15.89:0/3164641755 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f964c1a1ad0 con 0x7f964c104e50 2024-09-10T08:39:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.597+0000 7f9649400640 1 --2- 172.21.15.89:0/3164641755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f964c104e50 0x7f964c19a3e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f963800e9f0 tx=0x7f963800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.597+0000 7f9642a00640 1 -- 172.21.15.89:0/3164641755 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963800cd80 con 0x7f964c104e50 2024-09-10T08:39:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.597+0000 7f964b200640 1 -- 172.21.15.89:0/3164641755 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f964c1a1dc0 con 0x7f964c104e50 2024-09-10T08:39:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.597+0000 7f9642a00640 1 -- 172.21.15.89:0/3164641755 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9638010910 con 0x7f964c104e50 2024-09-10T08:39:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.597+0000 7f9642a00640 1 -- 172.21.15.89:0/3164641755 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96380188f0 con 0x7f964c104e50 2024-09-10T08:39:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.597+0000 7f964b200640 1 -- 172.21.15.89:0/3164641755 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f964c1a2280 con 0x7f964c104e50 2024-09-10T08:39:15.599 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.598+0000 7f964b200640 1 -- 172.21.15.89:0/3164641755 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f964c10af80 con 0x7f964c104e50 2024-09-10T08:39:15.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.600+0000 7f9642a00640 1 -- 172.21.15.89:0/3164641755 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9638010ac0 con 0x7f964c104e50 2024-09-10T08:39:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.601+0000 7f9642a00640 1 --2- 172.21.15.89:0/3164641755 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f96300a2bb0 0x7f96300a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.601+0000 7f9642a00640 1 -- 172.21.15.89:0/3164641755 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f96380cfce0 con 0x7f964c104e50 2024-09-10T08:39:15.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.602+0000 7f9649e00640 1 --2- 172.21.15.89:0/3164641755 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f96300a2bb0 0x7f96300a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:15.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.602+0000 7f9649e00640 1 --2- 172.21.15.89:0/3164641755 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f96300a2bb0 0x7f96300a5070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f964c104cb0 tx=0x7f963c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:15.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.603+0000 7f9642a00640 1 -- 172.21.15.89:0/3164641755 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f963809c1c0 con 0x7f964c104e50 2024-09-10T08:39:15.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.787+0000 7f964b200640 1 -- 172.21.15.89:0/3164641755 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f964c062630 con 0x7f96300a2bb0 2024-09-10T08:39:15.788 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:15 smithi089 ceph-mon[30757]: from='client.25024 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:15.789 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.789+0000 7f9642a00640 1 -- 172.21.15.89:0/3164641755 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f964c062630 con 0x7f96300a2bb0 2024-09-10T08:39:15.789 INFO:teuthology.orchestra.run.smithi089.stdout:HOST ADDR LABELS STATUS 2024-09-10T08:39:15.789 INFO:teuthology.orchestra.run.smithi089.stdout:smithi089 172.21.15.89 2024-09-10T08:39:15.789 INFO:teuthology.orchestra.run.smithi089.stdout:smithi111 172.21.15.111 2024-09-10T08:39:15.789 INFO:teuthology.orchestra.run.smithi089.stdout:smithi149 172.21.15.149 2024-09-10T08:39:15.790 INFO:teuthology.orchestra.run.smithi089.stdout:3 hosts in cluster 2024-09-10T08:39:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.791+0000 7f964b200640 1 -- 172.21.15.89:0/3164641755 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f96300a2bb0 msgr2=0x7f96300a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.791+0000 7f964b200640 1 --2- 172.21.15.89:0/3164641755 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f96300a2bb0 0x7f96300a5070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f964c104cb0 tx=0x7f963c005f90 comp rx=0 tx=0).stop 2024-09-10T08:39:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.791+0000 7f964b200640 1 -- 172.21.15.89:0/3164641755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f964c104e50 msgr2=0x7f964c19a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.791+0000 7f964b200640 1 --2- 172.21.15.89:0/3164641755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f964c104e50 0x7f964c19a3e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f963800e9f0 tx=0x7f963800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:39:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.791+0000 7f964b200640 1 -- 172.21.15.89:0/3164641755 shutdown_connections 2024-09-10T08:39:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.791+0000 7f964b200640 1 --2- 172.21.15.89:0/3164641755 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f96300a2bb0 0x7f96300a5070 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.791+0000 7f964b200640 1 --2- 172.21.15.89:0/3164641755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f964c105810 0x7f964c1a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.791+0000 7f964b200640 1 --2- 172.21.15.89:0/3164641755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f964c104e50 0x7f964c19a3e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.791+0000 7f964b200640 1 --2- 172.21.15.89:0/3164641755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f964c103c50 0x7f964c199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:15.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.791+0000 7f964b200640 1 -- 172.21.15.89:0/3164641755 >> 172.21.15.89:0/3164641755 conn(0x7f964c0ff3e0 msgr2=0x7f964c108150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:15.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.791+0000 7f964b200640 1 -- 172.21.15.89:0/3164641755 shutdown_connections 2024-09-10T08:39:15.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:15.791+0000 7f964b200640 1 -- 172.21.15.89:0/3164641755 wait complete. 2024-09-10T08:39:15.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:15 smithi149 ceph-mon[36167]: from='client.25024 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:16.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:15 smithi111 ceph-mon[35220]: from='client.25024 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:16.556 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-10T08:39:16.802 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:16 smithi089 ceph-mon[30757]: pgmap v373: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T08:39:16.925 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:16.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:16 smithi149 ceph-mon[36167]: pgmap v373: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T08:39:17.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:16 smithi111 ceph-mon[35220]: pgmap v373: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T08:39:17.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.444+0000 7fbbfae00640 1 -- 172.21.15.89:0/3235923459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf4103600 msgr2=0x7fbbf4109e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:17.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.444+0000 7fbbfae00640 1 --2- 172.21.15.89:0/3235923459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf4103600 0x7fbbf4109e90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009900 tx=0x7fbbec02a410 comp rx=0 tx=0).stop 2024-09-10T08:39:17.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.444+0000 7fbbfae00640 1 -- 172.21.15.89:0/3235923459 shutdown_connections 2024-09-10T08:39:17.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.444+0000 7fbbfae00640 1 --2- 172.21.15.89:0/3235923459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf4103600 0x7fbbf4109e90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:17.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.444+0000 7fbbfae00640 1 --2- 172.21.15.89:0/3235923459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbf4102c40 0x7fbbf41030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:17.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.444+0000 7fbbfae00640 1 --2- 172.21.15.89:0/3235923459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbf4101a40 0x7fbbf4101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:17.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.444+0000 7fbbfae00640 1 -- 172.21.15.89:0/3235923459 >> 172.21.15.89:0/3235923459 conn(0x7fbbf40fd1f0 msgr2=0x7fbbf40ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:17.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.444+0000 7fbbfae00640 1 -- 172.21.15.89:0/3235923459 shutdown_connections 2024-09-10T08:39:17.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.444+0000 7fbbfae00640 1 -- 172.21.15.89:0/3235923459 wait complete. 2024-09-10T08:39:17.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.445+0000 7fbbfae00640 1 Processor -- start 2024-09-10T08:39:17.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.445+0000 7fbbfae00640 1 -- start start 2024-09-10T08:39:17.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.445+0000 7fbbfae00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbf4101a40 0x7fbbf4197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:17.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.445+0000 7fbbfae00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbf4102c40 0x7fbbf41981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:17.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.445+0000 7fbbfae00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf4103600 0x7fbbf419f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:17.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.445+0000 7fbbfae00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf410d6f0 con 0x7fbbf4103600 2024-09-10T08:39:17.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.445+0000 7fbbfae00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf410d570 con 0x7fbbf4102c40 2024-09-10T08:39:17.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.445+0000 7fbbfae00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf410d870 con 0x7fbbf4101a40 2024-09-10T08:39:17.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.446+0000 7fbbfa400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf4103600 0x7fbbf419f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:17.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.446+0000 7fbbf9a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbf4101a40 0x7fbbf4197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:17.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.446+0000 7fbbfa400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf4103600 0x7fbbf419f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47278/0 (socket says 172.21.15.89:47278) 2024-09-10T08:39:17.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.446+0000 7fbbfa400640 1 -- 172.21.15.89:0/3846827871 learned_addr learned my addr 172.21.15.89:0/3846827871 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:17.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.446+0000 7fbbf9000640 1 --2- 172.21.15.89:0/3846827871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbf4102c40 0x7fbbf41981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:17.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.446+0000 7fbbfa400640 1 -- 172.21.15.89:0/3846827871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbf4101a40 msgr2=0x7fbbf4197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:17.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.446+0000 7fbbfa400640 1 --2- 172.21.15.89:0/3846827871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbf4101a40 0x7fbbf4197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:17.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.446+0000 7fbbfa400640 1 -- 172.21.15.89:0/3846827871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbf4102c40 msgr2=0x7fbbf41981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:17.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.446+0000 7fbbfa400640 1 --2- 172.21.15.89:0/3846827871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbf4102c40 0x7fbbf41981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:17.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.446+0000 7fbbfa400640 1 -- 172.21.15.89:0/3846827871 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf419f8b0 con 0x7fbbf4103600 2024-09-10T08:39:17.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.446+0000 7fbbf9a00640 1 --2- 172.21.15.89:0/3846827871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbf4101a40 0x7fbbf4197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:17.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.446+0000 7fbbf9000640 1 --2- 172.21.15.89:0/3846827871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbf4102c40 0x7fbbf41981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:17.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.447+0000 7fbbfa400640 1 --2- 172.21.15.89:0/3846827871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf4103600 0x7fbbf419f240 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbbec02a930 tx=0x7fbbec032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:17.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.447+0000 7fbbf2000640 1 -- 172.21.15.89:0/3846827871 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec03f030 con 0x7fbbf4103600 2024-09-10T08:39:17.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.447+0000 7fbbf2000640 1 -- 172.21.15.89:0/3846827871 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbec032e50 con 0x7fbbf4103600 2024-09-10T08:39:17.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.447+0000 7fbbfae00640 1 -- 172.21.15.89:0/3846827871 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf419fb40 con 0x7fbbf4103600 2024-09-10T08:39:17.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.447+0000 7fbbf2000640 1 -- 172.21.15.89:0/3846827871 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec035070 con 0x7fbbf4103600 2024-09-10T08:39:17.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.447+0000 7fbbfae00640 1 -- 172.21.15.89:0/3846827871 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf41a0070 con 0x7fbbf4103600 2024-09-10T08:39:17.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.448+0000 7fbbfae00640 1 -- 172.21.15.89:0/3846827871 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbf4108d70 con 0x7fbbf4103600 2024-09-10T08:39:17.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.450+0000 7fbbf2000640 1 -- 172.21.15.89:0/3846827871 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbbec042030 con 0x7fbbf4103600 2024-09-10T08:39:17.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.451+0000 7fbbf2000640 1 --2- 172.21.15.89:0/3846827871 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbbdc0a2c00 0x7fbbdc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:17.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.451+0000 7fbbf2000640 1 -- 172.21.15.89:0/3846827871 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fbbec0ef750 con 0x7fbbf4103600 2024-09-10T08:39:17.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.451+0000 7fbbf9a00640 1 --2- 172.21.15.89:0/3846827871 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbbdc0a2c00 0x7fbbdc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:17.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.452+0000 7fbbf9a00640 1 --2- 172.21.15.89:0/3846827871 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbbdc0a2c00 0x7fbbdc0a50c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4198b70 tx=0x7fbbe8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:17.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.455+0000 7fbbf2000640 1 -- 172.21.15.89:0/3846827871 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbbec0bbc30 con 0x7fbbf4103600 2024-09-10T08:39:17.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.639+0000 7fbbfae00640 1 -- 172.21.15.89:0/3846827871 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fbbf4062630 con 0x7fbbdc0a2c00 2024-09-10T08:39:17.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.641+0000 7fbbf2000640 1 -- 172.21.15.89:0/3846827871 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fbbf4062630 con 0x7fbbdc0a2c00 2024-09-10T08:39:17.642 INFO:teuthology.orchestra.run.smithi089.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-10T08:39:17.642 INFO:teuthology.orchestra.run.smithi089.stdout:smithi089 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040126400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-09-10T08:39:17.642 INFO:teuthology.orchestra.run.smithi089.stdout:smithi111 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400VZ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-10T08:39:17.642 INFO:teuthology.orchestra.run.smithi089.stdout:smithi149 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-10T08:39:17.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.643+0000 7fbbfae00640 1 -- 172.21.15.89:0/3846827871 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbbdc0a2c00 msgr2=0x7fbbdc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:17.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.643+0000 7fbbfae00640 1 --2- 172.21.15.89:0/3846827871 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbbdc0a2c00 0x7fbbdc0a50c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4198b70 tx=0x7fbbe8009500 comp rx=0 tx=0).stop 2024-09-10T08:39:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.643+0000 7fbbfae00640 1 -- 172.21.15.89:0/3846827871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf4103600 msgr2=0x7fbbf419f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.643+0000 7fbbfae00640 1 --2- 172.21.15.89:0/3846827871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf4103600 0x7fbbf419f240 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbbec02a930 tx=0x7fbbec032c60 comp rx=0 tx=0).stop 2024-09-10T08:39:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.643+0000 7fbbfae00640 1 -- 172.21.15.89:0/3846827871 shutdown_connections 2024-09-10T08:39:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.644+0000 7fbbfae00640 1 --2- 172.21.15.89:0/3846827871 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbbdc0a2c00 0x7fbbdc0a50c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.644+0000 7fbbfae00640 1 --2- 172.21.15.89:0/3846827871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbf4103600 0x7fbbf419f240 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.644+0000 7fbbfae00640 1 --2- 172.21.15.89:0/3846827871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbf4102c40 0x7fbbf41981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.644+0000 7fbbfae00640 1 --2- 172.21.15.89:0/3846827871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbf4101a40 0x7fbbf4197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.644+0000 7fbbfae00640 1 -- 172.21.15.89:0/3846827871 >> 172.21.15.89:0/3846827871 conn(0x7fbbf40fd1f0 msgr2=0x7fbbf4105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.644+0000 7fbbfae00640 1 -- 172.21.15.89:0/3846827871 shutdown_connections 2024-09-10T08:39:17.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:17.644+0000 7fbbfae00640 1 -- 172.21.15.89:0/3846827871 wait complete. 2024-09-10T08:39:17.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:17 smithi149 ceph-mon[36167]: from='client.25058 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:17.991 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:17 smithi089 ceph-mon[30757]: from='client.25058 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:18.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:17 smithi111 ceph-mon[35220]: from='client.25058 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:18.406 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T08:39:18.417 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi089.front.sepia.ceph.com 2024-09-10T08:39:18.418 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-10T08:39:18.786 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:18.924 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:18 smithi089 ceph-mon[30757]: pgmap v374: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:39:18.925 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:18 smithi089 ceph-mon[30757]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:18.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:18 smithi149 ceph-mon[36167]: pgmap v374: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:39:18.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:18 smithi149 ceph-mon[36167]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:19.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:18 smithi111 ceph-mon[35220]: pgmap v374: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:39:19.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:18 smithi111 ceph-mon[35220]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:19.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.342+0000 7f68fb000640 1 -- 172.21.15.89:0/4071219001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68f4103c50 msgr2=0x7f68f4104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:19.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.342+0000 7f68fb000640 1 --2- 172.21.15.89:0/4071219001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68f4103c50 0x7f68f4104050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f68e8009900 tx=0x7f68e802a410 comp rx=0 tx=0).stop 2024-09-10T08:39:19.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.342+0000 7f68fb000640 1 -- 172.21.15.89:0/4071219001 shutdown_connections 2024-09-10T08:39:19.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.342+0000 7f68fb000640 1 --2- 172.21.15.89:0/4071219001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68f4105810 0x7f68f410c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:19.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.342+0000 7f68fb000640 1 --2- 172.21.15.89:0/4071219001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f4104e50 0x7f68f41052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:19.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.342+0000 7f68fb000640 1 --2- 172.21.15.89:0/4071219001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68f4103c50 0x7f68f4104050 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:19.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.342+0000 7f68fb000640 1 -- 172.21.15.89:0/4071219001 >> 172.21.15.89:0/4071219001 conn(0x7f68f40ff3e0 msgr2=0x7f68f4101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:19.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.342+0000 7f68fb000640 1 -- 172.21.15.89:0/4071219001 shutdown_connections 2024-09-10T08:39:19.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.342+0000 7f68fb000640 1 -- 172.21.15.89:0/4071219001 wait complete. 2024-09-10T08:39:19.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.343+0000 7f68fb000640 1 Processor -- start 2024-09-10T08:39:19.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.343+0000 7f68fb000640 1 -- start start 2024-09-10T08:39:19.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.343+0000 7f68fb000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68f4103c50 0x7f68f4199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:19.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.343+0000 7f68fb000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68f4104e50 0x7f68f419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:19.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.343+0000 7f68fb000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f4105810 0x7f68f41a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:19.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.343+0000 7f68fb000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f68f410f930 con 0x7f68f4103c50 2024-09-10T08:39:19.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.343+0000 7f68fb000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f68f410f7b0 con 0x7f68f4105810 2024-09-10T08:39:19.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.343+0000 7f68fb000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f68f410fab0 con 0x7f68f4104e50 2024-09-10T08:39:19.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.344+0000 7f68f9c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68f4103c50 0x7f68f4199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:19.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.344+0000 7f68f9c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68f4103c50 0x7f68f4199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47300/0 (socket says 172.21.15.89:47300) 2024-09-10T08:39:19.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.344+0000 7f68f9c00640 1 -- 172.21.15.89:0/4197740005 learned_addr learned my addr 172.21.15.89:0/4197740005 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:19.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.344+0000 7f68fa600640 1 --2- 172.21.15.89:0/4197740005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f4105810 0x7f68f41a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:19.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.344+0000 7f68f9200640 1 --2- 172.21.15.89:0/4197740005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68f4104e50 0x7f68f419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:19.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.344+0000 7f68f9c00640 1 -- 172.21.15.89:0/4197740005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68f4104e50 msgr2=0x7f68f419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:19.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.344+0000 7f68f9c00640 1 --2- 172.21.15.89:0/4197740005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68f4104e50 0x7f68f419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:19.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.344+0000 7f68f9c00640 1 -- 172.21.15.89:0/4197740005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f4105810 msgr2=0x7f68f41a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:19.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.344+0000 7f68f9c00640 1 --2- 172.21.15.89:0/4197740005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f4105810 0x7f68f41a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:19.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.344+0000 7f68f9c00640 1 -- 172.21.15.89:0/4197740005 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f41a1b30 con 0x7f68f4103c50 2024-09-10T08:39:19.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.344+0000 7f68f9c00640 1 --2- 172.21.15.89:0/4197740005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68f4103c50 0x7f68f4199f00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f68e8031440 tx=0x7f68e8032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:19.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.345+0000 7f68f2000640 1 -- 172.21.15.89:0/4197740005 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e803f030 con 0x7f68f4103c50 2024-09-10T08:39:19.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.345+0000 7f68f2000640 1 -- 172.21.15.89:0/4197740005 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68e8003ce0 con 0x7f68f4103c50 2024-09-10T08:39:19.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.345+0000 7f68fb000640 1 -- 172.21.15.89:0/4197740005 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68f41a1dc0 con 0x7f68f4103c50 2024-09-10T08:39:19.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.345+0000 7f68f2000640 1 -- 172.21.15.89:0/4197740005 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e8035070 con 0x7f68f4103c50 2024-09-10T08:39:19.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.345+0000 7f68fb000640 1 -- 172.21.15.89:0/4197740005 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68f41a21e0 con 0x7f68f4103c50 2024-09-10T08:39:19.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.346+0000 7f68fb000640 1 -- 172.21.15.89:0/4197740005 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68f410b050 con 0x7f68f4103c50 2024-09-10T08:39:19.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.348+0000 7f68f2000640 1 -- 172.21.15.89:0/4197740005 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f68e8042030 con 0x7f68f4103c50 2024-09-10T08:39:19.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.349+0000 7f68f2000640 1 --2- 172.21.15.89:0/4197740005 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f68dc0a2bb0 0x7f68dc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:19.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.349+0000 7f68f2000640 1 -- 172.21.15.89:0/4197740005 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f68e80ef930 con 0x7f68f4103c50 2024-09-10T08:39:19.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.349+0000 7f68f9200640 1 --2- 172.21.15.89:0/4197740005 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f68dc0a2bb0 0x7f68dc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:19.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.350+0000 7f68f9200640 1 --2- 172.21.15.89:0/4197740005 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f68dc0a2bb0 0x7f68dc0a5070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f68f419b420 tx=0x7f68e000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:19.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.354+0000 7f68f2000640 1 -- 172.21.15.89:0/4197740005 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68e80bbe10 con 0x7f68f4103c50 2024-09-10T08:39:19.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.566+0000 7f68fb000640 1 -- 172.21.15.89:0/4197740005 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f68f4062e90 con 0x7f68f4103c50 2024-09-10T08:39:19.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.566+0000 7f68f2000640 1 -- 172.21.15.89:0/4197740005 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3092 (secure 0 0 0) 0x7f68e80c0110 con 0x7f68f4103c50 2024-09-10T08:39:19.568 INFO:teuthology.orchestra.run.smithi089.stdout:e25 2024-09-10T08:39:19.568 INFO:teuthology.orchestra.run.smithi089.stdout:btime 2024-09-10T08:37:50:718359+0000 2024-09-10T08:39:19.568 INFO:teuthology.orchestra.run.smithi089.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T08:39:19.568 INFO:teuthology.orchestra.run.smithi089.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T08:39:19.568 INFO:teuthology.orchestra.run.smithi089.stdout:legacy client fscid: 1 2024-09-10T08:39:19.568 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:39:19.568 INFO:teuthology.orchestra.run.smithi089.stdout:Filesystem 'cephfs' (1) 2024-09-10T08:39:19.568 INFO:teuthology.orchestra.run.smithi089.stdout:fs_name cephfs 2024-09-10T08:39:19.568 INFO:teuthology.orchestra.run.smithi089.stdout:epoch 25 2024-09-10T08:39:19.568 INFO:teuthology.orchestra.run.smithi089.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T08:39:19.568 INFO:teuthology.orchestra.run.smithi089.stdout:created 2024-09-10T08:36:57.162188+0000 2024-09-10T08:39:19.568 INFO:teuthology.orchestra.run.smithi089.stdout:modified 2024-09-10T08:37:50.607307+0000 2024-09-10T08:39:19.569 INFO:teuthology.orchestra.run.smithi089.stdout:tableserver 0 2024-09-10T08:39:19.569 INFO:teuthology.orchestra.run.smithi089.stdout:root 0 2024-09-10T08:39:19.569 INFO:teuthology.orchestra.run.smithi089.stdout:session_timeout 300 2024-09-10T08:39:19.569 INFO:teuthology.orchestra.run.smithi089.stdout:session_autoclose 300 2024-09-10T08:39:19.569 INFO:teuthology.orchestra.run.smithi089.stdout:max_file_size 1099511627776 2024-09-10T08:39:19.569 INFO:teuthology.orchestra.run.smithi089.stdout:max_xattr_size 65536 2024-09-10T08:39:19.569 INFO:teuthology.orchestra.run.smithi089.stdout:required_client_features {} 2024-09-10T08:39:19.569 INFO:teuthology.orchestra.run.smithi089.stdout:last_failure 0 2024-09-10T08:39:19.569 INFO:teuthology.orchestra.run.smithi089.stdout:last_failure_osd_epoch 0 2024-09-10T08:39:19.569 INFO:teuthology.orchestra.run.smithi089.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T08:39:19.569 INFO:teuthology.orchestra.run.smithi089.stdout:max_mds 3 2024-09-10T08:39:19.569 INFO:teuthology.orchestra.run.smithi089.stdout:in 0,1,2 2024-09-10T08:39:19.570 INFO:teuthology.orchestra.run.smithi089.stdout:up {0=24554,1=24518,2=24496} 2024-09-10T08:39:19.570 INFO:teuthology.orchestra.run.smithi089.stdout:failed 2024-09-10T08:39:19.570 INFO:teuthology.orchestra.run.smithi089.stdout:damaged 2024-09-10T08:39:19.570 INFO:teuthology.orchestra.run.smithi089.stdout:stopped 2024-09-10T08:39:19.570 INFO:teuthology.orchestra.run.smithi089.stdout:data_pools [3,4] 2024-09-10T08:39:19.570 INFO:teuthology.orchestra.run.smithi089.stdout:metadata_pool 2 2024-09-10T08:39:19.570 INFO:teuthology.orchestra.run.smithi089.stdout:inline_data disabled 2024-09-10T08:39:19.570 INFO:teuthology.orchestra.run.smithi089.stdout:balancer 2024-09-10T08:39:19.570 INFO:teuthology.orchestra.run.smithi089.stdout:bal_rank_mask -1 2024-09-10T08:39:19.570 INFO:teuthology.orchestra.run.smithi089.stdout:standby_count_wanted 0 2024-09-10T08:39:19.570 INFO:teuthology.orchestra.run.smithi089.stdout:qdb_cluster leader: 24554 members: 24496,24518,24554 2024-09-10T08:39:19.570 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.b{0:24554} state up:active seq 5 addr [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:39:19.570 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.e{0:24494} state up:standby-replay seq 1 addr [v2:172.21.15.111:6834/285258866,v1:172.21.15.111:6835/285258866] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:39:19.570 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.h{1:24518} state up:active seq 20 addr [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:39:19.571 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.c{1:24484} state up:standby-replay seq 1 addr [v2:172.21.15.149:6836/2367754626,v1:172.21.15.149:6837/2367754626] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:39:19.571 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.i{2:24496} state up:active seq 20 addr [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:39:19.571 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.f{2:24472} state up:standby-replay seq 1 addr [v2:172.21.15.149:6834/881263752,v1:172.21.15.149:6835/881263752] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:39:19.571 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:39:19.571 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:39:19.571 INFO:teuthology.orchestra.run.smithi089.stdout:Standby daemons: 2024-09-10T08:39:19.571 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:39:19.571 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.j{-1:14433} state up:standby seq 1 addr [v2:172.21.15.89:6834/2129325722,v1:172.21.15.89:6835/2129325722] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:39:19.571 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.d{-1:14445} state up:standby seq 1 addr [v2:172.21.15.89:6836/3715416661,v1:172.21.15.89:6837/3715416661] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:39:19.571 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.g{-1:14490} state up:standby seq 1 addr [v2:172.21.15.89:6838/2573030321,v1:172.21.15.89:6839/2573030321] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:39:19.571 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.a{-1:14496} state up:standby seq 1 addr [v2:172.21.15.89:6840/3524299195,v1:172.21.15.89:6841/3524299195] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:39:19.571 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.l{-1:24430} state up:standby seq 1 addr [v2:172.21.15.149:6832/210698608,v1:172.21.15.149:6833/210698608] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:39:19.571 INFO:teuthology.orchestra.run.smithi089.stdout:[mds.k{-1:24464} state up:standby seq 1 addr [v2:172.21.15.111:6832/2902311847,v1:172.21.15.111:6833/2902311847] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:39:19.572 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 25 2024-09-10T08:39:19.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.569+0000 7f68fb000640 1 -- 172.21.15.89:0/4197740005 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f68dc0a2bb0 msgr2=0x7f68dc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:19.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.569+0000 7f68fb000640 1 --2- 172.21.15.89:0/4197740005 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f68dc0a2bb0 0x7f68dc0a5070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f68f419b420 tx=0x7f68e000b410 comp rx=0 tx=0).stop 2024-09-10T08:39:19.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.569+0000 7f68fb000640 1 -- 172.21.15.89:0/4197740005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68f4103c50 msgr2=0x7f68f4199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:19.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.569+0000 7f68fb000640 1 --2- 172.21.15.89:0/4197740005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68f4103c50 0x7f68f4199f00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f68e8031440 tx=0x7f68e8032ce0 comp rx=0 tx=0).stop 2024-09-10T08:39:19.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.569+0000 7f68fb000640 1 -- 172.21.15.89:0/4197740005 shutdown_connections 2024-09-10T08:39:19.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.569+0000 7f68fb000640 1 --2- 172.21.15.89:0/4197740005 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f68dc0a2bb0 0x7f68dc0a5070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:19.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.569+0000 7f68fb000640 1 --2- 172.21.15.89:0/4197740005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f4105810 0x7f68f41a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:19.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.569+0000 7f68fb000640 1 --2- 172.21.15.89:0/4197740005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68f4104e50 0x7f68f419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:19.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.569+0000 7f68fb000640 1 --2- 172.21.15.89:0/4197740005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68f4103c50 0x7f68f4199f00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:19.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.569+0000 7f68fb000640 1 -- 172.21.15.89:0/4197740005 >> 172.21.15.89:0/4197740005 conn(0x7f68f40ff3e0 msgr2=0x7f68f4108220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:19.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.569+0000 7f68fb000640 1 -- 172.21.15.89:0/4197740005 shutdown_connections 2024-09-10T08:39:19.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:19.569+0000 7f68fb000640 1 -- 172.21.15.89:0/4197740005 wait complete. 2024-09-10T08:39:19.954 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:19 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/4197740005' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T08:39:19.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:19 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/4197740005' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T08:39:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:19 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/4197740005' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T08:39:20.376 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-10T08:39:20.744 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:20.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:20 smithi149 ceph-mon[36167]: pgmap v375: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-10T08:39:21.023 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:20 smithi089 ceph-mon[30757]: pgmap v375: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-10T08:39:21.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:20 smithi111 ceph-mon[35220]: pgmap v375: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-10T08:39:21.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.260+0000 7fb5dd600640 1 -- 172.21.15.89:0/2811504137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5d8101c00 msgr2=0x7fb5d8102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:21.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.260+0000 7fb5dd600640 1 --2- 172.21.15.89:0/2811504137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5d8101c00 0x7fb5d8102000 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8009900 tx=0x7fb5c802a400 comp rx=0 tx=0).stop 2024-09-10T08:39:21.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.260+0000 7fb5dd600640 1 -- 172.21.15.89:0/2811504137 shutdown_connections 2024-09-10T08:39:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.260+0000 7fb5dd600640 1 --2- 172.21.15.89:0/2811504137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d81037c0 0x7fb5d810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.260+0000 7fb5dd600640 1 --2- 172.21.15.89:0/2811504137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5d8102e00 0x7fb5d8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.260+0000 7fb5dd600640 1 --2- 172.21.15.89:0/2811504137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5d8101c00 0x7fb5d8102000 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.260+0000 7fb5dd600640 1 -- 172.21.15.89:0/2811504137 >> 172.21.15.89:0/2811504137 conn(0x7fb5d80fd390 msgr2=0x7fb5d80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.260+0000 7fb5dd600640 1 -- 172.21.15.89:0/2811504137 shutdown_connections 2024-09-10T08:39:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.260+0000 7fb5dd600640 1 -- 172.21.15.89:0/2811504137 wait complete. 2024-09-10T08:39:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.261+0000 7fb5dd600640 1 Processor -- start 2024-09-10T08:39:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.262+0000 7fb5dd600640 1 -- start start 2024-09-10T08:39:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.262+0000 7fb5dd600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5d8101c00 0x7fb5d8075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.262+0000 7fb5dd600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5d8102e00 0x7fb5d80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.262+0000 7fb5dd600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d81037c0 0x7fb5d8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:21.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.262+0000 7fb5dd600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d810d8e0 con 0x7fb5d8102e00 2024-09-10T08:39:21.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.262+0000 7fb5dd600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d810d760 con 0x7fb5d8101c00 2024-09-10T08:39:21.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.262+0000 7fb5dd600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d810da60 con 0x7fb5d81037c0 2024-09-10T08:39:21.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.262+0000 7fb5d6000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5d8102e00 0x7fb5d80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:21.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.262+0000 7fb5d6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5d8101c00 0x7fb5d8075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:21.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.262+0000 7fb5d6000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5d8102e00 0x7fb5d80745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47318/0 (socket says 172.21.15.89:47318) 2024-09-10T08:39:21.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.262+0000 7fb5d6000640 1 -- 172.21.15.89:0/1675546284 learned_addr learned my addr 172.21.15.89:0/1675546284 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:21.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.262+0000 7fb5d6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5d8101c00 0x7fb5d8075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:42394/0 (socket says 172.21.15.89:42394) 2024-09-10T08:39:21.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.262+0000 7fb5d7400640 1 --2- 172.21.15.89:0/1675546284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d81037c0 0x7fb5d8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:21.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.263+0000 7fb5d6000640 1 -- 172.21.15.89:0/1675546284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d81037c0 msgr2=0x7fb5d8074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:21.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.263+0000 7fb5d6000640 1 --2- 172.21.15.89:0/1675546284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d81037c0 0x7fb5d8074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:21.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.263+0000 7fb5d6000640 1 -- 172.21.15.89:0/1675546284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5d8101c00 msgr2=0x7fb5d8075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:21.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.263+0000 7fb5d6000640 1 --2- 172.21.15.89:0/1675546284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5d8101c00 0x7fb5d8075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:21.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.263+0000 7fb5d6000640 1 -- 172.21.15.89:0/1675546284 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d80753e0 con 0x7fb5d8102e00 2024-09-10T08:39:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.263+0000 7fb5d6a00640 1 --2- 172.21.15.89:0/1675546284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5d8101c00 0x7fb5d8075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.263+0000 7fb5d6000640 1 --2- 172.21.15.89:0/1675546284 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5d8102e00 0x7fb5d80745e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb5c400e9f0 tx=0x7fb5c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.263+0000 7fb5cf400640 1 -- 172.21.15.89:0/1675546284 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c400cd80 con 0x7fb5d8102e00 2024-09-10T08:39:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.263+0000 7fb5cf400640 1 -- 172.21.15.89:0/1675546284 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5c4010910 con 0x7fb5d8102e00 2024-09-10T08:39:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.263+0000 7fb5dd600640 1 -- 172.21.15.89:0/1675546284 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d81a20b0 con 0x7fb5d8102e00 2024-09-10T08:39:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.263+0000 7fb5cf400640 1 -- 172.21.15.89:0/1675546284 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4018990 con 0x7fb5d8102e00 2024-09-10T08:39:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.264+0000 7fb5dd600640 1 -- 172.21.15.89:0/1675546284 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d81a25c0 con 0x7fb5d8102e00 2024-09-10T08:39:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.265+0000 7fb5dd600640 1 -- 172.21.15.89:0/1675546284 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5d8108fa0 con 0x7fb5d8102e00 2024-09-10T08:39:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.266+0000 7fb5cf400640 1 -- 172.21.15.89:0/1675546284 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb5c4010ac0 con 0x7fb5d8102e00 2024-09-10T08:39:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.268+0000 7fb5cf400640 1 --2- 172.21.15.89:0/1675546284 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb5bc0a2b80 0x7fb5bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.268+0000 7fb5cf400640 1 -- 172.21.15.89:0/1675546284 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb5c40cf7f0 con 0x7fb5d8102e00 2024-09-10T08:39:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.268+0000 7fb5d6a00640 1 --2- 172.21.15.89:0/1675546284 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb5bc0a2b80 0x7fb5bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.269+0000 7fb5d6a00640 1 --2- 172.21.15.89:0/1675546284 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb5bc0a2b80 0x7fb5bc0a5040 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb5c802a9f0 tx=0x7fb5c80315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:21.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.272+0000 7fb5cf400640 1 -- 172.21.15.89:0/1675546284 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5c409bd30 con 0x7fb5d8102e00 2024-09-10T08:39:21.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.448+0000 7fb5dd600640 1 -- 172.21.15.89:0/1675546284 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fb5d8062df0 con 0x7fb5d8102e00 2024-09-10T08:39:21.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.449+0000 7fb5cf400640 1 -- 172.21.15.89:0/1675546284 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v96) ==== 56+0+5709 (secure 0 0 0) 0x7fb5c40a0030 con 0x7fb5d8102e00 2024-09-10T08:39:21.450 INFO:teuthology.orchestra.run.smithi089.stdout:epoch 96 2024-09-10T08:39:21.450 INFO:teuthology.orchestra.run.smithi089.stdout:fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:39:21.451 INFO:teuthology.orchestra.run.smithi089.stdout:created 2024-09-10T08:28:49.697035+0000 2024-09-10T08:39:21.451 INFO:teuthology.orchestra.run.smithi089.stdout:modified 2024-09-10T08:38:43.618497+0000 2024-09-10T08:39:21.451 INFO:teuthology.orchestra.run.smithi089.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-10T08:39:21.451 INFO:teuthology.orchestra.run.smithi089.stdout:crush_version 27 2024-09-10T08:39:21.451 INFO:teuthology.orchestra.run.smithi089.stdout:full_ratio 0.95 2024-09-10T08:39:21.451 INFO:teuthology.orchestra.run.smithi089.stdout:backfillfull_ratio 0.9 2024-09-10T08:39:21.452 INFO:teuthology.orchestra.run.smithi089.stdout:nearfull_ratio 0.85 2024-09-10T08:39:21.452 INFO:teuthology.orchestra.run.smithi089.stdout:require_min_compat_client luminous 2024-09-10T08:39:21.452 INFO:teuthology.orchestra.run.smithi089.stdout:min_compat_client luminous 2024-09-10T08:39:21.452 INFO:teuthology.orchestra.run.smithi089.stdout:require_osd_release squid 2024-09-10T08:39:21.452 INFO:teuthology.orchestra.run.smithi089.stdout:stretch_mode_enabled false 2024-09-10T08:39:21.452 INFO:teuthology.orchestra.run.smithi089.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 23 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-10T08:39:21.453 INFO:teuthology.orchestra.run.smithi089.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-10T08:39:21.453 INFO:teuthology.orchestra.run.smithi089.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-10T08:39:21.453 INFO:teuthology.orchestra.run.smithi089.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 92 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-10T08:39:21.453 INFO:teuthology.orchestra.run.smithi089.stdout:max_osd 12 2024-09-10T08:39:21.453 INFO:teuthology.orchestra.run.smithi089.stdout:osd.0 up in weight 1 up_from 8 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6802/1760523284,v1:172.21.15.89:6803/1760523284] [v2:172.21.15.89:6804/1760523284,v1:172.21.15.89:6805/1760523284] exists,up d3b55192-bd95-4497-89ea-dc017a25c0cc 2024-09-10T08:39:21.453 INFO:teuthology.orchestra.run.smithi089.stdout:osd.1 up in weight 1 up_from 13 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6800/2369291742,v1:172.21.15.111:6801/2369291742] [v2:172.21.15.111:6802/2369291742,v1:172.21.15.111:6803/2369291742] exists,up 60aae117-cb52-4034-a700-11693171cd52 2024-09-10T08:39:21.454 INFO:teuthology.orchestra.run.smithi089.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6800/1092261395,v1:172.21.15.149:6801/1092261395] [v2:172.21.15.149:6802/1092261395,v1:172.21.15.149:6803/1092261395] exists,up a305a019-0870-437c-a298-4f923ec54e77 2024-09-10T08:39:21.454 INFO:teuthology.orchestra.run.smithi089.stdout:osd.3 up in weight 1 up_from 26 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6810/1405773950,v1:172.21.15.89:6811/1405773950] [v2:172.21.15.89:6812/1405773950,v1:172.21.15.89:6813/1405773950] exists,up d02f54c1-f883-40f6-9341-8d9fc4b712ac 2024-09-10T08:39:21.454 INFO:teuthology.orchestra.run.smithi089.stdout:osd.4 up in weight 1 up_from 33 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6808/316717323,v1:172.21.15.111:6809/316717323] [v2:172.21.15.111:6810/316717323,v1:172.21.15.111:6811/316717323] exists,up 23cff46b-88dd-4344-be06-b35db6014f7f 2024-09-10T08:39:21.454 INFO:teuthology.orchestra.run.smithi089.stdout:osd.5 up in weight 1 up_from 38 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6808/1256148461,v1:172.21.15.149:6809/1256148461] [v2:172.21.15.149:6810/1256148461,v1:172.21.15.149:6811/1256148461] exists,up 8e516e8d-e7c6-4ebc-80a5-29932161df4c 2024-09-10T08:39:21.454 INFO:teuthology.orchestra.run.smithi089.stdout:osd.6 up in weight 1 up_from 46 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6818/4190488024,v1:172.21.15.89:6819/4190488024] [v2:172.21.15.89:6820/4190488024,v1:172.21.15.89:6821/4190488024] exists,up 48f92472-f657-4c5a-adeb-20d4fea1ce8a 2024-09-10T08:39:21.454 INFO:teuthology.orchestra.run.smithi089.stdout:osd.7 up in weight 1 up_from 51 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6816/652666376,v1:172.21.15.111:6817/652666376] [v2:172.21.15.111:6818/652666376,v1:172.21.15.111:6819/652666376] exists,up 14a2a6f0-c4cd-406b-b637-7527fd32babe 2024-09-10T08:39:21.455 INFO:teuthology.orchestra.run.smithi089.stdout:osd.8 up in weight 1 up_from 58 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6816/2955172869,v1:172.21.15.149:6817/2955172869] [v2:172.21.15.149:6818/2955172869,v1:172.21.15.149:6819/2955172869] exists,up a54502e6-95f7-431f-8db1-54a6c7983b5d 2024-09-10T08:39:21.455 INFO:teuthology.orchestra.run.smithi089.stdout:osd.9 up in weight 1 up_from 66 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6826/2427467040,v1:172.21.15.89:6827/2427467040] [v2:172.21.15.89:6828/2427467040,v1:172.21.15.89:6829/2427467040] exists,up 5fe17344-5b1c-4601-a633-639cc1691478 2024-09-10T08:39:21.455 INFO:teuthology.orchestra.run.smithi089.stdout:osd.10 up in weight 1 up_from 71 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6824/2328678744,v1:172.21.15.111:6825/2328678744] [v2:172.21.15.111:6826/2328678744,v1:172.21.15.111:6827/2328678744] exists,up d8f29890-3064-4b67-b649-467a0ac86db7 2024-09-10T08:39:21.455 INFO:teuthology.orchestra.run.smithi089.stdout:osd.11 up in weight 1 up_from 79 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6824/3130508827,v1:172.21.15.149:6825/3130508827] [v2:172.21.15.149:6826/3130508827,v1:172.21.15.149:6827/3130508827] exists,up 92182c73-f139-47ca-a0ef-5046ae9b2310 2024-09-10T08:39:21.455 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 2.8 [2,11] 2024-09-10T08:39:21.455 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 2.a [2,11] 2024-09-10T08:39:21.456 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 2.c [2,1] 2024-09-10T08:39:21.456 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 2.1f [8,4] 2024-09-10T08:39:21.456 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 3.e [4,7] 2024-09-10T08:39:21.456 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 3.1a [5,6] 2024-09-10T08:39:21.456 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 3.23 [4,8] 2024-09-10T08:39:21.456 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 3.25 [5,9] 2024-09-10T08:39:21.457 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 3.34 [4,9] 2024-09-10T08:39:21.457 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 3.38 [3,9] 2024-09-10T08:39:21.457 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 4.5 [9,10] 2024-09-10T08:39:21.457 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 4.c [9,10] 2024-09-10T08:39:21.457 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 4.1a [5,8] 2024-09-10T08:39:21.458 INFO:teuthology.orchestra.run.smithi089.stdout:pg_upmap_items 4.1f [9,8] 2024-09-10T08:39:21.458 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/850669510 expires 2024-09-11T08:29:41.472002+0000 2024-09-10T08:39:21.458 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/3541027428 expires 2024-09-11T08:29:41.472002+0000 2024-09-10T08:39:21.458 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:6801/3794982914 expires 2024-09-11T08:29:41.472002+0000 2024-09-10T08:39:21.458 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:6800/3794982914 expires 2024-09-11T08:29:41.472002+0000 2024-09-10T08:39:21.459 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/2396468787 expires 2024-09-11T08:29:41.472002+0000 2024-09-10T08:39:21.459 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/1609720980 expires 2024-09-11T08:29:14.554869+0000 2024-09-10T08:39:21.459 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/995019144 expires 2024-09-11T08:29:14.554869+0000 2024-09-10T08:39:21.459 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:6801/4223112778 expires 2024-09-11T08:29:14.554869+0000 2024-09-10T08:39:21.459 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:6800/4223112778 expires 2024-09-11T08:29:14.554869+0000 2024-09-10T08:39:21.459 INFO:teuthology.orchestra.run.smithi089.stdout:blocklist 172.21.15.89:0/2179977257 expires 2024-09-11T08:29:14.554869+0000 2024-09-10T08:39:21.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.451+0000 7fb5dd600640 1 -- 172.21.15.89:0/1675546284 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb5bc0a2b80 msgr2=0x7fb5bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:21.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.451+0000 7fb5dd600640 1 --2- 172.21.15.89:0/1675546284 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb5bc0a2b80 0x7fb5bc0a5040 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb5c802a9f0 tx=0x7fb5c80315b0 comp rx=0 tx=0).stop 2024-09-10T08:39:21.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.451+0000 7fb5dd600640 1 -- 172.21.15.89:0/1675546284 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5d8102e00 msgr2=0x7fb5d80745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:21.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.451+0000 7fb5dd600640 1 --2- 172.21.15.89:0/1675546284 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5d8102e00 0x7fb5d80745e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb5c400e9f0 tx=0x7fb5c400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:39:21.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.452+0000 7fb5dd600640 1 -- 172.21.15.89:0/1675546284 shutdown_connections 2024-09-10T08:39:21.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.452+0000 7fb5dd600640 1 --2- 172.21.15.89:0/1675546284 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb5bc0a2b80 0x7fb5bc0a5040 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:21.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.452+0000 7fb5dd600640 1 --2- 172.21.15.89:0/1675546284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d81037c0 0x7fb5d8074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:21.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.452+0000 7fb5dd600640 1 --2- 172.21.15.89:0/1675546284 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5d8102e00 0x7fb5d80745e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:21.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.452+0000 7fb5dd600640 1 --2- 172.21.15.89:0/1675546284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5d8101c00 0x7fb5d8075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:21.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.452+0000 7fb5dd600640 1 -- 172.21.15.89:0/1675546284 >> 172.21.15.89:0/1675546284 conn(0x7fb5d80fd390 msgr2=0x7fb5d8106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:21.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.452+0000 7fb5dd600640 1 -- 172.21.15.89:0/1675546284 shutdown_connections 2024-09-10T08:39:21.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:21.452+0000 7fb5dd600640 1 -- 172.21.15.89:0/1675546284 wait complete. 2024-09-10T08:39:21.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:21 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1675546284' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T08:39:22.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:21 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1675546284' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T08:39:22.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:21 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1675546284' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T08:39:22.242 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-10T08:39:22.254 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:39:22.621 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:22.940 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:22 smithi089 ceph-mon[30757]: pgmap v376: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-10T08:39:22.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:22 smithi149 ceph-mon[36167]: pgmap v376: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-10T08:39:23.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:22 smithi111 ceph-mon[35220]: pgmap v376: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-10T08:39:23.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.167+0000 7f62cca00640 1 -- 172.21.15.89:0/3775029898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62c81047e0 msgr2=0x7f62c8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:23.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.167+0000 7f62cca00640 1 --2- 172.21.15.89:0/3775029898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62c81047e0 0x7f62c8108830 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009900 tx=0x7f62b402a410 comp rx=0 tx=0).stop 2024-09-10T08:39:23.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.168+0000 7f62cca00640 1 -- 172.21.15.89:0/3775029898 shutdown_connections 2024-09-10T08:39:23.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.168+0000 7f62cca00640 1 --2- 172.21.15.89:0/3775029898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62c8108f60 0x7f62c8110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:23.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.168+0000 7f62cca00640 1 --2- 172.21.15.89:0/3775029898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62c81047e0 0x7f62c8108830 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:23.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.168+0000 7f62cca00640 1 --2- 172.21.15.89:0/3775029898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62c8103e30 0x7f62c8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:23.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.168+0000 7f62cca00640 1 -- 172.21.15.89:0/3775029898 >> 172.21.15.89:0/3775029898 conn(0x7f62c80ff8d0 msgr2=0x7f62c8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:23.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.168+0000 7f62cca00640 1 -- 172.21.15.89:0/3775029898 shutdown_connections 2024-09-10T08:39:23.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.168+0000 7f62cca00640 1 -- 172.21.15.89:0/3775029898 wait complete. 2024-09-10T08:39:23.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.168+0000 7f62cca00640 1 Processor -- start 2024-09-10T08:39:23.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.169+0000 7f62cca00640 1 -- start start 2024-09-10T08:39:23.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.169+0000 7f62cca00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62c8103e30 0x7f62c8076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:23.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.169+0000 7f62cca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62c81047e0 0x7f62c8076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:23.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.169+0000 7f62cca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62c8108f60 0x7f62c8077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:23.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.169+0000 7f62cca00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f62c8114010 con 0x7f62c81047e0 2024-09-10T08:39:23.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.169+0000 7f62cca00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f62c8113e90 con 0x7f62c8103e30 2024-09-10T08:39:23.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.169+0000 7f62cca00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f62c8114190 con 0x7f62c8108f60 2024-09-10T08:39:23.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.169+0000 7f62c5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62c81047e0 0x7f62c8076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:23.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.169+0000 7f62c5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62c81047e0 0x7f62c8076c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50876/0 (socket says 172.21.15.89:50876) 2024-09-10T08:39:23.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.169+0000 7f62c5e00640 1 -- 172.21.15.89:0/2378032028 learned_addr learned my addr 172.21.15.89:0/2378032028 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:23.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.169+0000 7f62c7200640 1 --2- 172.21.15.89:0/2378032028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62c8108f60 0x7f62c8077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:23.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.169+0000 7f62c6800640 1 --2- 172.21.15.89:0/2378032028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62c8103e30 0x7f62c8076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:23.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.170+0000 7f62c5e00640 1 -- 172.21.15.89:0/2378032028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62c8108f60 msgr2=0x7f62c8077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:23.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.170+0000 7f62c5e00640 1 --2- 172.21.15.89:0/2378032028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62c8108f60 0x7f62c8077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:23.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.170+0000 7f62c5e00640 1 -- 172.21.15.89:0/2378032028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62c8103e30 msgr2=0x7f62c8076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:23.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.170+0000 7f62c5e00640 1 --2- 172.21.15.89:0/2378032028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62c8103e30 0x7f62c8076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:23.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.170+0000 7f62c5e00640 1 -- 172.21.15.89:0/2378032028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c81aab10 con 0x7f62c81047e0 2024-09-10T08:39:23.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.170+0000 7f62c5e00640 1 --2- 172.21.15.89:0/2378032028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62c81047e0 0x7f62c8076c40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f62b402aa00 tx=0x7f62b4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:23.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.170+0000 7f62bf400640 1 -- 172.21.15.89:0/2378032028 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b403f070 con 0x7f62c81047e0 2024-09-10T08:39:23.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.170+0000 7f62bf400640 1 -- 172.21.15.89:0/2378032028 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62b4031c90 con 0x7f62c81047e0 2024-09-10T08:39:23.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.170+0000 7f62cca00640 1 -- 172.21.15.89:0/2378032028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c81aada0 con 0x7f62c81047e0 2024-09-10T08:39:23.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.170+0000 7f62bf400640 1 -- 172.21.15.89:0/2378032028 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b4035030 con 0x7f62c81047e0 2024-09-10T08:39:23.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.170+0000 7f62cca00640 1 -- 172.21.15.89:0/2378032028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c81ab270 con 0x7f62c81047e0 2024-09-10T08:39:23.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.172+0000 7f62cca00640 1 -- 172.21.15.89:0/2378032028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62c8105310 con 0x7f62c81047e0 2024-09-10T08:39:23.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.173+0000 7f62bf400640 1 -- 172.21.15.89:0/2378032028 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f62b4042070 con 0x7f62c81047e0 2024-09-10T08:39:23.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.174+0000 7f62bf400640 1 --2- 172.21.15.89:0/2378032028 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f62ac0a2c00 0x7f62ac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:23.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.175+0000 7f62bf400640 1 -- 172.21.15.89:0/2378032028 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f62b40ef940 con 0x7f62c81047e0 2024-09-10T08:39:23.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.175+0000 7f62c6800640 1 --2- 172.21.15.89:0/2378032028 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f62ac0a2c00 0x7f62ac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:23.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.175+0000 7f62c6800640 1 --2- 172.21.15.89:0/2378032028 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f62ac0a2c00 0x7f62ac0a50c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009bd0 tx=0x7f62b8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:23.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.178+0000 7f62bf400640 1 -- 172.21.15.89:0/2378032028 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62b40bbee0 con 0x7f62c81047e0 2024-09-10T08:39:23.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.390+0000 7f62cca00640 1 -- 172.21.15.89:0/2378032028 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62c8104210 con 0x7f62c81047e0 2024-09-10T08:39:23.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.391+0000 7f62bf400640 1 -- 172.21.15.89:0/2378032028 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11918 (secure 0 0 0) 0x7f62b40c01e0 con 0x7f62c81047e0 2024-09-10T08:39:23.392 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:39:23.392 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":25,"btime":"2024-09-10T08:37:50:718359+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:37:50.607307+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24496,24518,24554]},"id":1}]} 2024-09-10T08:39:23.392 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 25 2024-09-10T08:39:23.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.394+0000 7f62cca00640 1 -- 172.21.15.89:0/2378032028 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f62ac0a2c00 msgr2=0x7f62ac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:23.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.394+0000 7f62cca00640 1 --2- 172.21.15.89:0/2378032028 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f62ac0a2c00 0x7f62ac0a50c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009bd0 tx=0x7f62b8005f90 comp rx=0 tx=0).stop 2024-09-10T08:39:23.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.394+0000 7f62cca00640 1 -- 172.21.15.89:0/2378032028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62c81047e0 msgr2=0x7f62c8076c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:23.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.394+0000 7f62cca00640 1 --2- 172.21.15.89:0/2378032028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62c81047e0 0x7f62c8076c40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f62b402aa00 tx=0x7f62b4003c10 comp rx=0 tx=0).stop 2024-09-10T08:39:23.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.394+0000 7f62cca00640 1 -- 172.21.15.89:0/2378032028 shutdown_connections 2024-09-10T08:39:23.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.394+0000 7f62cca00640 1 --2- 172.21.15.89:0/2378032028 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f62ac0a2c00 0x7f62ac0a50c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:23.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.394+0000 7f62cca00640 1 --2- 172.21.15.89:0/2378032028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62c8108f60 0x7f62c8077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:23.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.394+0000 7f62cca00640 1 --2- 172.21.15.89:0/2378032028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62c81047e0 0x7f62c8076c40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:23.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.394+0000 7f62cca00640 1 --2- 172.21.15.89:0/2378032028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62c8103e30 0x7f62c8076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:23.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.394+0000 7f62cca00640 1 -- 172.21.15.89:0/2378032028 >> 172.21.15.89:0/2378032028 conn(0x7f62c80ff8d0 msgr2=0x7f62c8100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:23.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.394+0000 7f62cca00640 1 -- 172.21.15.89:0/2378032028 shutdown_connections 2024-09-10T08:39:23.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:23.394+0000 7f62cca00640 1 -- 172.21.15.89:0/2378032028 wait complete. 2024-09-10T08:39:23.835 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:23 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2378032028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:39:23.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:23 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2378032028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:39:24.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:23 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2378032028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:39:24.246 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:39:24.611 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:24.938 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:24 smithi089 ceph-mon[30757]: pgmap v377: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-10T08:39:24.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:24 smithi149 ceph-mon[36167]: pgmap v377: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-10T08:39:25.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:24 smithi111 ceph-mon[35220]: pgmap v377: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-10T08:39:25.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.194+0000 7f4562a00640 1 -- 172.21.15.89:0/3355764954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f455c102790 msgr2=0x7f455c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:25.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.194+0000 7f4562a00640 1 --2- 172.21.15.89:0/3355764954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f455c102790 0x7f455c1067e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4544009900 tx=0x7f454402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:39:25.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.195+0000 7f4562a00640 1 -- 172.21.15.89:0/3355764954 shutdown_connections 2024-09-10T08:39:25.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.195+0000 7f4562a00640 1 --2- 172.21.15.89:0/3355764954 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f455c106f10 0x7f455c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:25.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.195+0000 7f4562a00640 1 --2- 172.21.15.89:0/3355764954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f455c102790 0x7f455c1067e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:25.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.195+0000 7f4562a00640 1 --2- 172.21.15.89:0/3355764954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f455c101de0 0x7f455c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:25.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.195+0000 7f4562a00640 1 -- 172.21.15.89:0/3355764954 >> 172.21.15.89:0/3355764954 conn(0x7f455c0fd8c0 msgr2=0x7f455c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:25.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.195+0000 7f4562a00640 1 -- 172.21.15.89:0/3355764954 shutdown_connections 2024-09-10T08:39:25.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.195+0000 7f4562a00640 1 -- 172.21.15.89:0/3355764954 wait complete. 2024-09-10T08:39:25.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.196+0000 7f4562a00640 1 Processor -- start 2024-09-10T08:39:25.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.196+0000 7f4562a00640 1 -- start start 2024-09-10T08:39:25.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.197+0000 7f4562a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f455c101de0 0x7f455c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:25.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.197+0000 7f4562a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f455c102790 0x7f455c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:25.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.197+0000 7f4562a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f455c106f10 0x7f455c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:25.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.197+0000 7f4562a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f455c111f70 con 0x7f455c101de0 2024-09-10T08:39:25.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.197+0000 7f4562a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f455c111df0 con 0x7f455c106f10 2024-09-10T08:39:25.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.197+0000 7f4562a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1120f0 con 0x7f455c102790 2024-09-10T08:39:25.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.197+0000 7f455be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f455c101de0 0x7f455c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:25.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.197+0000 7f455be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f455c101de0 0x7f455c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50896/0 (socket says 172.21.15.89:50896) 2024-09-10T08:39:25.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.197+0000 7f455be00640 1 -- 172.21.15.89:0/1550306330 learned_addr learned my addr 172.21.15.89:0/1550306330 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:25.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.197+0000 7f4560a00640 1 --2- 172.21.15.89:0/1550306330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f455c106f10 0x7f455c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:25.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.197+0000 7f4553400640 1 --2- 172.21.15.89:0/1550306330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f455c102790 0x7f455c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:25.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.197+0000 7f455be00640 1 -- 172.21.15.89:0/1550306330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f455c102790 msgr2=0x7f455c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:25.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.197+0000 7f455be00640 1 --2- 172.21.15.89:0/1550306330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f455c102790 0x7f455c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:25.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.197+0000 7f455be00640 1 -- 172.21.15.89:0/1550306330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f455c106f10 msgr2=0x7f455c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:25.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.197+0000 7f455be00640 1 --2- 172.21.15.89:0/1550306330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f455c106f10 0x7f455c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:25.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.197+0000 7f455be00640 1 -- 172.21.15.89:0/1550306330 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c1a68e0 con 0x7f455c101de0 2024-09-10T08:39:25.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.197+0000 7f4553400640 1 --2- 172.21.15.89:0/1550306330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f455c102790 0x7f455c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:39:25.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.198+0000 7f455be00640 1 --2- 172.21.15.89:0/1550306330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f455c101de0 0x7f455c0744f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f454c00d950 tx=0x7f454c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:25.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.198+0000 7f4559600640 1 -- 172.21.15.89:0/1550306330 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c017030 con 0x7f455c101de0 2024-09-10T08:39:25.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.198+0000 7f4559600640 1 -- 172.21.15.89:0/1550306330 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454c004020 con 0x7f455c101de0 2024-09-10T08:39:25.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.198+0000 7f4562a00640 1 -- 172.21.15.89:0/1550306330 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455c1a6bd0 con 0x7f455c101de0 2024-09-10T08:39:25.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.198+0000 7f4559600640 1 -- 172.21.15.89:0/1550306330 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c0047a0 con 0x7f455c101de0 2024-09-10T08:39:25.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.198+0000 7f4562a00640 1 -- 172.21.15.89:0/1550306330 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455c1a7110 con 0x7f455c101de0 2024-09-10T08:39:25.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.199+0000 7f4562a00640 1 -- 172.21.15.89:0/1550306330 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f455c1021c0 con 0x7f455c101de0 2024-09-10T08:39:25.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.201+0000 7f4559600640 1 -- 172.21.15.89:0/1550306330 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f454c0041d0 con 0x7f455c101de0 2024-09-10T08:39:25.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.202+0000 7f4559600640 1 --2- 172.21.15.89:0/1550306330 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f45400a2b80 0x7f45400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:25.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.202+0000 7f4559600640 1 -- 172.21.15.89:0/1550306330 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f454c0cee30 con 0x7f455c101de0 2024-09-10T08:39:25.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.202+0000 7f4553400640 1 --2- 172.21.15.89:0/1550306330 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f45400a2b80 0x7f45400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:25.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.203+0000 7f4553400640 1 --2- 172.21.15.89:0/1550306330 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f45400a2b80 0x7f45400a5040 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f455c075f80 tx=0x7f4544005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:25.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.206+0000 7f4559600640 1 -- 172.21.15.89:0/1550306330 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f454c09b960 con 0x7f455c101de0 2024-09-10T08:39:25.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.415+0000 7f4562a00640 1 -- 172.21.15.89:0/1550306330 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f455c10a180 con 0x7f455c101de0 2024-09-10T08:39:25.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.416+0000 7f4559600640 1 -- 172.21.15.89:0/1550306330 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11918 (secure 0 0 0) 0x7f454c09b260 con 0x7f455c101de0 2024-09-10T08:39:25.417 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:39:25.417 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":25,"btime":"2024-09-10T08:37:50:718359+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:37:50.607307+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24496,24518,24554]},"id":1}]} 2024-09-10T08:39:25.418 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 25 2024-09-10T08:39:25.420 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.420+0000 7f4562a00640 1 -- 172.21.15.89:0/1550306330 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f45400a2b80 msgr2=0x7f45400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:25.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.420+0000 7f4562a00640 1 --2- 172.21.15.89:0/1550306330 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f45400a2b80 0x7f45400a5040 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f455c075f80 tx=0x7f4544005a20 comp rx=0 tx=0).stop 2024-09-10T08:39:25.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.420+0000 7f4562a00640 1 -- 172.21.15.89:0/1550306330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f455c101de0 msgr2=0x7f455c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:25.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.420+0000 7f4562a00640 1 --2- 172.21.15.89:0/1550306330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f455c101de0 0x7f455c0744f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f454c00d950 tx=0x7f454c00de10 comp rx=0 tx=0).stop 2024-09-10T08:39:25.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.420+0000 7f4562a00640 1 -- 172.21.15.89:0/1550306330 shutdown_connections 2024-09-10T08:39:25.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.420+0000 7f4562a00640 1 --2- 172.21.15.89:0/1550306330 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f45400a2b80 0x7f45400a5040 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:25.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.420+0000 7f4562a00640 1 --2- 172.21.15.89:0/1550306330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f455c106f10 0x7f455c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:25.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.420+0000 7f4562a00640 1 --2- 172.21.15.89:0/1550306330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f455c102790 0x7f455c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:25.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.420+0000 7f4562a00640 1 --2- 172.21.15.89:0/1550306330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f455c101de0 0x7f455c0744f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:25.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.420+0000 7f4562a00640 1 -- 172.21.15.89:0/1550306330 >> 172.21.15.89:0/1550306330 conn(0x7f455c0fd8c0 msgr2=0x7f455c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:25.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.420+0000 7f4562a00640 1 -- 172.21.15.89:0/1550306330 shutdown_connections 2024-09-10T08:39:25.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:25.420+0000 7f4562a00640 1 -- 172.21.15.89:0/1550306330 wait complete. 2024-09-10T08:39:25.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:25 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1550306330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:39:26.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:25 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1550306330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:39:26.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:25 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1550306330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:39:26.169 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:39:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:26.944 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:26 smithi111 ceph-mon[35220]: pgmap v378: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:39:26.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:26 smithi149 ceph-mon[36167]: pgmap v378: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:39:26.998 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:26 smithi089 ceph-mon[30757]: pgmap v378: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:39:27.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.104+0000 7f24dc400640 1 -- 172.21.15.89:0/248153868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24d4102790 msgr2=0x7f24d41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:27.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.104+0000 7f24dc400640 1 --2- 172.21.15.89:0/248153868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24d4102790 0x7f24d41067e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009900 tx=0x7f24c002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:39:27.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.105+0000 7f24dc400640 1 -- 172.21.15.89:0/248153868 shutdown_connections 2024-09-10T08:39:27.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.105+0000 7f24dc400640 1 --2- 172.21.15.89:0/248153868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24d4106f10 0x7f24d410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:27.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.105+0000 7f24dc400640 1 --2- 172.21.15.89:0/248153868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24d4102790 0x7f24d41067e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:27.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.105+0000 7f24dc400640 1 --2- 172.21.15.89:0/248153868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24d4101de0 0x7f24d41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:27.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.105+0000 7f24dc400640 1 -- 172.21.15.89:0/248153868 >> 172.21.15.89:0/248153868 conn(0x7f24d40fd8c0 msgr2=0x7f24d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:27.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.105+0000 7f24dc400640 1 -- 172.21.15.89:0/248153868 shutdown_connections 2024-09-10T08:39:27.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.105+0000 7f24dc400640 1 -- 172.21.15.89:0/248153868 wait complete. 2024-09-10T08:39:27.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.106+0000 7f24dc400640 1 Processor -- start 2024-09-10T08:39:27.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.106+0000 7f24dc400640 1 -- start start 2024-09-10T08:39:27.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.106+0000 7f24dc400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24d4101de0 0x7f24d419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:27.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.106+0000 7f24dc400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24d4102790 0x7f24d419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:27.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.106+0000 7f24dc400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24d4106f10 0x7f24d419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:27.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.106+0000 7f24dc400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f24d4111f80 con 0x7f24d4102790 2024-09-10T08:39:27.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.106+0000 7f24dc400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f24d4111e00 con 0x7f24d4101de0 2024-09-10T08:39:27.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.106+0000 7f24dc400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f24d4112100 con 0x7f24d4106f10 2024-09-10T08:39:27.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.106+0000 7f24d9000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24d4102790 0x7f24d419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:27.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.106+0000 7f24d9000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24d4102790 0x7f24d419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50916/0 (socket says 172.21.15.89:50916) 2024-09-10T08:39:27.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.106+0000 7f24d9000640 1 -- 172.21.15.89:0/155531938 learned_addr learned my addr 172.21.15.89:0/155531938 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:27.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.107+0000 7f24d9a00640 1 --2- 172.21.15.89:0/155531938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24d4101de0 0x7f24d419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:27.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.107+0000 7f24da400640 1 --2- 172.21.15.89:0/155531938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24d4106f10 0x7f24d419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:27.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.107+0000 7f24d9000640 1 -- 172.21.15.89:0/155531938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24d4106f10 msgr2=0x7f24d419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:27.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.107+0000 7f24d9000640 1 --2- 172.21.15.89:0/155531938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24d4106f10 0x7f24d419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:27.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.107+0000 7f24d9000640 1 -- 172.21.15.89:0/155531938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24d4101de0 msgr2=0x7f24d419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:27.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.107+0000 7f24d9000640 1 --2- 172.21.15.89:0/155531938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24d4101de0 0x7f24d419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:27.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.107+0000 7f24d9000640 1 -- 172.21.15.89:0/155531938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d419f5e0 con 0x7f24d4102790 2024-09-10T08:39:27.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.107+0000 7f24da400640 1 --2- 172.21.15.89:0/155531938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24d4106f10 0x7f24d419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:27.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.107+0000 7f24d9000640 1 --2- 172.21.15.89:0/155531938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24d4102790 0x7f24d419ab70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f24c0004770 tx=0x7f24c0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:27.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.108+0000 7f24d2000640 1 -- 172.21.15.89:0/155531938 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c003f070 con 0x7f24d4102790 2024-09-10T08:39:27.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.108+0000 7f24d2000640 1 -- 172.21.15.89:0/155531938 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24c0031ca0 con 0x7f24d4102790 2024-09-10T08:39:27.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.108+0000 7f24dc400640 1 -- 172.21.15.89:0/155531938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d419f870 con 0x7f24d4102790 2024-09-10T08:39:27.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.108+0000 7f24d2000640 1 -- 172.21.15.89:0/155531938 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0035030 con 0x7f24d4102790 2024-09-10T08:39:27.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.108+0000 7f24dc400640 1 -- 172.21.15.89:0/155531938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d41a71b0 con 0x7f24d4102790 2024-09-10T08:39:27.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.109+0000 7f24dc400640 1 -- 172.21.15.89:0/155531938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24d41021c0 con 0x7f24d4102790 2024-09-10T08:39:27.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.111+0000 7f24d2000640 1 -- 172.21.15.89:0/155531938 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f24c0042070 con 0x7f24d4102790 2024-09-10T08:39:27.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.112+0000 7f24d2000640 1 --2- 172.21.15.89:0/155531938 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f24bc0a2c00 0x7f24bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:27.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.112+0000 7f24d2000640 1 -- 172.21.15.89:0/155531938 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f24c00ef9b0 con 0x7f24d4102790 2024-09-10T08:39:27.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.112+0000 7f24d9a00640 1 --2- 172.21.15.89:0/155531938 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f24bc0a2c00 0x7f24bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:27.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.113+0000 7f24d9a00640 1 --2- 172.21.15.89:0/155531938 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f24bc0a2c00 0x7f24bc0a50c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f24c80099a0 tx=0x7f24c8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:27.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.116+0000 7f24d2000640 1 -- 172.21.15.89:0/155531938 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f24c00bbf50 con 0x7f24d4102790 2024-09-10T08:39:27.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.292+0000 7f24dc400640 1 -- 172.21.15.89:0/155531938 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24d41a73b0 con 0x7f24d4102790 2024-09-10T08:39:27.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.295+0000 7f24d2000640 1 -- 172.21.15.89:0/155531938 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24894 (secure 0 0 0) 0x7f24c00c0250 con 0x7f24d4102790 2024-09-10T08:39:27.296 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:39:27.296 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":96,"fsid":"91f747bc-6f4e-11ef-bcea-c7b262605968","created":"2024-09-10T08:28:49.697035+0000","modified":"2024-09-10T08:38:43.618497+0000","last_up_change":"2024-09-10T08:36:00.535433+0000","last_in_change":"2024-09-10T08:35:37.131131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:32:02.060714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:36:52.462301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:36:55.119708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:37:03.729002+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d3b55192-bd95-4497-89ea-dc017a25c0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6803","nonce":1760523284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6805","nonce":1760523284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6809","nonce":1760523284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6807","nonce":1760523284}]},"public_addr":"172.21.15.89:6803/1760523284","cluster_addr":"172.21.15.89:6805/1760523284","heartbeat_back_addr":"172.21.15.89:6809/1760523284","heartbeat_front_addr":"172.21.15.89:6807/1760523284","state":["exists","up"]},{"osd":1,"uuid":"60aae117-cb52-4034-a700-11693171cd52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6801","nonce":2369291742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6803","nonce":2369291742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6807","nonce":2369291742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6805","nonce":2369291742}]},"public_addr":"172.21.15.111:6801/2369291742","cluster_addr":"172.21.15.111:6803/2369291742","heartbeat_back_addr":"172.21.15.111:6807/2369291742","heartbeat_front_addr":"172.21.15.111:6805/2369291742","state":["exists","up"]},{"osd":2,"uuid":"a305a019-0870-437c-a298-4f923ec54e77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6801","nonce":1092261395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6803","nonce":1092261395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6807","nonce":1092261395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6805","nonce":1092261395}]},"public_addr":"172.21.15.149:6801/1092261395","cluster_addr":"172.21.15.149:6803/1092261395","heartbeat_back_addr":"172.21.15.149:6807/1092261395","heartbeat_front_addr":"172.21.15.149:6805/1092261395","state":["exists","up"]},{"osd":3,"uuid":"d02f54c1-f883-40f6-9341-8d9fc4b712ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6811","nonce":1405773950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6813","nonce":1405773950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6817","nonce":1405773950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6815","nonce":1405773950}]},"public_addr":"172.21.15.89:6811/1405773950","cluster_addr":"172.21.15.89:6813/1405773950","heartbeat_back_addr":"172.21.15.89:6817/1405773950","heartbeat_front_addr":"172.21.15.89:6815/1405773950","state":["exists","up"]},{"osd":4,"uuid":"23cff46b-88dd-4344-be06-b35db6014f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6809","nonce":316717323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6811","nonce":316717323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6815","nonce":316717323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6813","nonce":316717323}]},"public_addr":"172.21.15.111:6809/316717323","cluster_addr":"172.21.15.111:6811/316717323","heartbeat_back_addr":"172.21.15.111:6815/316717323","heartbeat_front_addr":"172.21.15.111:6813/316717323","state":["exists","up"]},{"osd":5,"uuid":"8e516e8d-e7c6-4ebc-80a5-29932161df4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6809","nonce":1256148461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6811","nonce":1256148461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6815","nonce":1256148461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6813","nonce":1256148461}]},"public_addr":"172.21.15.149:6809/1256148461","cluster_addr":"172.21.15.149:6811/1256148461","heartbeat_back_addr":"172.21.15.149:6815/1256148461","heartbeat_front_addr":"172.21.15.149:6813/1256148461","state":["exists","up"]},{"osd":6,"uuid":"48f92472-f657-4c5a-adeb-20d4fea1ce8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6819","nonce":4190488024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6821","nonce":4190488024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6825","nonce":4190488024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6823","nonce":4190488024}]},"public_addr":"172.21.15.89:6819/4190488024","cluster_addr":"172.21.15.89:6821/4190488024","heartbeat_back_addr":"172.21.15.89:6825/4190488024","heartbeat_front_addr":"172.21.15.89:6823/4190488024","state":["exists","up"]},{"osd":7,"uuid":"14a2a6f0-c4cd-406b-b637-7527fd32babe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6817","nonce":652666376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6819","nonce":652666376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6823","nonce":652666376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6821","nonce":652666376}]},"public_addr":"172.21.15.111:6817/652666376","cluster_addr":"172.21.15.111:6819/652666376","heartbeat_back_addr":"172.21.15.111:6823/652666376","heartbeat_front_addr":"172.21.15.111:6821/652666376","state":["exists","up"]},{"osd":8,"uuid":"a54502e6-95f7-431f-8db1-54a6c7983b5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6817","nonce":2955172869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6819","nonce":2955172869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6823","nonce":2955172869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6821","nonce":2955172869}]},"public_addr":"172.21.15.149:6817/2955172869","cluster_addr":"172.21.15.149:6819/2955172869","heartbeat_back_addr":"172.21.15.149:6823/2955172869","heartbeat_front_addr":"172.21.15.149:6821/2955172869","state":["exists","up"]},{"osd":9,"uuid":"5fe17344-5b1c-4601-a633-639cc1691478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6827","nonce":2427467040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6829","nonce":2427467040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6833","nonce":2427467040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6831","nonce":2427467040}]},"public_addr":"172.21.15.89:6827/2427467040","cluster_addr":"172.21.15.89:6829/2427467040","heartbeat_back_addr":"172.21.15.89:6833/2427467040","heartbeat_front_addr":"172.21.15.89:6831/2427467040","state":["exists","up"]},{"osd":10,"uuid":"d8f29890-3064-4b67-b649-467a0ac86db7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6825","nonce":2328678744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6827","nonce":2328678744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6831","nonce":2328678744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6829","nonce":2328678744}]},"public_addr":"172.21.15.111:6825/2328678744","cluster_addr":"172.21.15.111:6827/2328678744","heartbeat_back_addr":"172.21.15.111:6831/2328678744","heartbeat_front_addr":"172.21.15.111:6829/2328678744","state":["exists","up"]},{"osd":11,"uuid":"92182c73-f139-47ca-a0ef-5046ae9b2310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6825","nonce":3130508827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6827","nonce":3130508827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6831","nonce":3130508827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6829","nonce":3130508827}]},"public_addr":"172.21.15.149:6825/3130508827","cluster_addr":"172.21.15.149:6827/3130508827","heartbeat_back_addr":"172.21.15.149:6831/3130508827","heartbeat_front_addr":"172.21.15.149:6829/3130508827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:09.031810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:33.695124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:58.065510+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:27.592838+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:53.275289+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:18.305677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:47.781535+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:12.889512+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:37.744193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:07.718408+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:33.391027+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:59.013250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/850669510":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/3541027428":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6801/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6800/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/2396468787":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/1609720980":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/995019144":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6801/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6800/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/2179977257":"2024-09-11T08:29:14.554869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:39:27.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.297+0000 7f24dc400640 1 -- 172.21.15.89:0/155531938 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f24bc0a2c00 msgr2=0x7f24bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:27.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.298+0000 7f24dc400640 1 --2- 172.21.15.89:0/155531938 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f24bc0a2c00 0x7f24bc0a50c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f24c80099a0 tx=0x7f24c8009500 comp rx=0 tx=0).stop 2024-09-10T08:39:27.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.298+0000 7f24dc400640 1 -- 172.21.15.89:0/155531938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24d4102790 msgr2=0x7f24d419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:27.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.298+0000 7f24dc400640 1 --2- 172.21.15.89:0/155531938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24d4102790 0x7f24d419ab70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f24c0004770 tx=0x7f24c0003c10 comp rx=0 tx=0).stop 2024-09-10T08:39:27.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.298+0000 7f24dc400640 1 -- 172.21.15.89:0/155531938 shutdown_connections 2024-09-10T08:39:27.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.298+0000 7f24dc400640 1 --2- 172.21.15.89:0/155531938 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f24bc0a2c00 0x7f24bc0a50c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:27.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.298+0000 7f24dc400640 1 --2- 172.21.15.89:0/155531938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24d4106f10 0x7f24d419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:27.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.298+0000 7f24dc400640 1 --2- 172.21.15.89:0/155531938 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24d4102790 0x7f24d419ab70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:27.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.298+0000 7f24dc400640 1 --2- 172.21.15.89:0/155531938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24d4101de0 0x7f24d419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:27.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.298+0000 7f24dc400640 1 -- 172.21.15.89:0/155531938 >> 172.21.15.89:0/155531938 conn(0x7f24d40fd8c0 msgr2=0x7f24d40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:27.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.298+0000 7f24dc400640 1 -- 172.21.15.89:0/155531938 shutdown_connections 2024-09-10T08:39:27.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:27.298+0000 7f24dc400640 1 -- 172.21.15.89:0/155531938 wait complete. 2024-09-10T08:39:27.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:27 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/155531938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:39:28.039 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:27 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/155531938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:39:28.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:27 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/155531938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:39:28.078 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T08:36:57.162188+0000', 'modified': '2024-09-10T08:37:50.607307+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24554, 'mds_1': 24518, 'mds_2': 24496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24472': {'gid': 24472, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6835/881263752', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6834', 'nonce': 881263752}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 881263752}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6837/2367754626', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 2367754626}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 2367754626}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6835/285258866', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 285258866}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 285258866}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.149:6839/2020874598', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 2020874598}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 2020874598}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.111:6837/779590543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6836', 'nonce': 779590543}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 779590543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24554': {'gid': 24554, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.111:6839/3640096926', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 3640096926}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 3640096926}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24554, 'qdb_cluster': [24496, 24518, 24554]} 2024-09-10T08:39:28.078 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-10T08:39:28.078 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T08:39:28.447 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:28.918 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:28 smithi089 ceph-mon[30757]: pgmap v379: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-10T08:39:28.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:28 smithi149 ceph-mon[36167]: pgmap v379: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-10T08:39:29.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.027+0000 7ff757200640 1 -- 172.21.15.89:0/2736137498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff75810bfc0 msgr2=0x7ff75810e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:29.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.027+0000 7ff757200640 1 --2- 172.21.15.89:0/2736137498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff75810bfc0 0x7ff75810e460 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff750009900 tx=0x7ff75002a480 comp rx=0 tx=0).stop 2024-09-10T08:39:29.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.027+0000 7ff757200640 1 -- 172.21.15.89:0/2736137498 shutdown_connections 2024-09-10T08:39:29.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.027+0000 7ff757200640 1 --2- 172.21.15.89:0/2736137498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff75810bfc0 0x7ff75810e460 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.027+0000 7ff757200640 1 --2- 172.21.15.89:0/2736137498 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7581034b0 0x7ff75810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.027+0000 7ff757200640 1 --2- 172.21.15.89:0/2736137498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff758102b90 0x7ff758102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.027+0000 7ff757200640 1 -- 172.21.15.89:0/2736137498 >> 172.21.15.89:0/2736137498 conn(0x7ff7580fc610 msgr2=0x7ff7580fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:29.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.027+0000 7ff757200640 1 -- 172.21.15.89:0/2736137498 shutdown_connections 2024-09-10T08:39:29.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.027+0000 7ff757200640 1 -- 172.21.15.89:0/2736137498 wait complete. 2024-09-10T08:39:29.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.028+0000 7ff757200640 1 Processor -- start 2024-09-10T08:39:29.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.028+0000 7ff757200640 1 -- start start 2024-09-10T08:39:29.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.028+0000 7ff757200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff758102b90 0x7ff75819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:29.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.028+0000 7ff757200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7581034b0 0x7ff75819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:29.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.028+0000 7ff757200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff75810bfc0 0x7ff75819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:29.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.029+0000 7ff757200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff758111e10 con 0x7ff75810bfc0 2024-09-10T08:39:29.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.029+0000 7ff757200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff758111c90 con 0x7ff7581034b0 2024-09-10T08:39:29.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.029+0000 7ff757200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff758111f90 con 0x7ff758102b90 2024-09-10T08:39:29.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.029+0000 7ff755e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff758102b90 0x7ff75819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:29.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.029+0000 7ff755e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff758102b90 0x7ff75819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:56662/0 (socket says 172.21.15.89:56662) 2024-09-10T08:39:29.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.029+0000 7ff755e00640 1 -- 172.21.15.89:0/885025801 learned_addr learned my addr 172.21.15.89:0/885025801 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:29.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.029+0000 7ff74fe00640 1 --2- 172.21.15.89:0/885025801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7581034b0 0x7ff75819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:29.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.029+0000 7ff756800640 1 --2- 172.21.15.89:0/885025801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff75810bfc0 0x7ff75819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:29.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.029+0000 7ff74fe00640 1 -- 172.21.15.89:0/885025801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff758102b90 msgr2=0x7ff75819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:29.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.029+0000 7ff74fe00640 1 --2- 172.21.15.89:0/885025801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff758102b90 0x7ff75819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.029+0000 7ff74fe00640 1 -- 172.21.15.89:0/885025801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff75810bfc0 msgr2=0x7ff75819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:29.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.029+0000 7ff74fe00640 1 --2- 172.21.15.89:0/885025801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff75810bfc0 0x7ff75819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.029+0000 7ff74fe00640 1 -- 172.21.15.89:0/885025801 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75819f430 con 0x7ff7581034b0 2024-09-10T08:39:29.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.030+0000 7ff74fe00640 1 --2- 172.21.15.89:0/885025801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7581034b0 0x7ff75819a9c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff74000e7a0 tx=0x7ff74000ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:29.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.030+0000 7ff74ea00640 1 -- 172.21.15.89:0/885025801 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74000ccc0 con 0x7ff7581034b0 2024-09-10T08:39:29.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.030+0000 7ff757200640 1 -- 172.21.15.89:0/885025801 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75819f720 con 0x7ff7581034b0 2024-09-10T08:39:29.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.030+0000 7ff74ea00640 1 -- 172.21.15.89:0/885025801 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff740009530 con 0x7ff7581034b0 2024-09-10T08:39:29.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.030+0000 7ff74ea00640 1 -- 172.21.15.89:0/885025801 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740018880 con 0x7ff7581034b0 2024-09-10T08:39:29.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.030+0000 7ff757200640 1 -- 172.21.15.89:0/885025801 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7581a7000 con 0x7ff7581034b0 2024-09-10T08:39:29.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.033+0000 7ff75f5c3740 1 Processor -- start 2024-09-10T08:39:29.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.033+0000 7ff74ea00640 1 -- 172.21.15.89:0/885025801 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff740018a20 con 0x7ff7581034b0 2024-09-10T08:39:29.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.033+0000 7ff75f5c3740 1 -- start start 2024-09-10T08:39:29.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.034+0000 7ff75f5c3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e3e4e0 0x562d87e3e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:29.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.034+0000 7ff75f5c3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e3ee00 0x562d87e49180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:29.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.034+0000 7ff75f5c3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e496c0 0x562d87e4bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:29.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.034+0000 7ff75f5c3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562d87da40b0 con 0x562d87e3e4e0 2024-09-10T08:39:29.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.034+0000 7ff75f5c3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562d87d4bab0 con 0x562d87e496c0 2024-09-10T08:39:29.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.034+0000 7ff75f5c3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562d87da0070 con 0x562d87e3ee00 2024-09-10T08:39:29.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.034+0000 7ff755e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e3ee00 0x562d87e49180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:29.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.034+0000 7ff756800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e3e4e0 0x562d87e3e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:29.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.034+0000 7ff74fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e496c0 0x562d87e4bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:29.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.034+0000 7ff756800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e3e4e0 0x562d87e3e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50928/0 (socket says 172.21.15.89:50928) 2024-09-10T08:39:29.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.034+0000 7ff756800640 1 -- 172.21.15.89:0/2301355660 learned_addr learned my addr 172.21.15.89:0/2301355660 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:29.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.035+0000 7ff755e00640 1 -- 172.21.15.89:0/2301355660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e496c0 msgr2=0x562d87e4bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:29.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.035+0000 7ff755e00640 1 --2- 172.21.15.89:0/2301355660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e496c0 0x562d87e4bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.035+0000 7ff755e00640 1 -- 172.21.15.89:0/2301355660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e3e4e0 msgr2=0x562d87e3e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:29.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.035+0000 7ff755e00640 1 --2- 172.21.15.89:0/2301355660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e3e4e0 0x562d87e3e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.035+0000 7ff755e00640 1 -- 172.21.15.89:0/2301355660 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d87d584a0 con 0x562d87e3ee00 2024-09-10T08:39:29.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.035+0000 7ff755e00640 1 --2- 172.21.15.89:0/2301355660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e3ee00 0x562d87e49180 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff7480114c0 tx=0x7ff748011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48741344576aae92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:29.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.035+0000 7ff74ea00640 1 --2- 172.21.15.89:0/885025801 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff73c0a2b80 0x7ff73c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:29.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.035+0000 7ff747000640 1 -- 172.21.15.89:0/2301355660 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748004b20 con 0x562d87e3ee00 2024-09-10T08:39:29.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.035+0000 7ff747000640 1 -- 172.21.15.89:0/2301355660 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff748004cc0 con 0x562d87e3ee00 2024-09-10T08:39:29.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.035+0000 7ff747000640 1 -- 172.21.15.89:0/2301355660 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7480054e0 con 0x562d87e3ee00 2024-09-10T08:39:29.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.035+0000 7ff74ea00640 1 -- 172.21.15.89:0/885025801 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff7400cf580 con 0x7ff7581034b0 2024-09-10T08:39:29.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.036+0000 7ff756800640 1 --2- 172.21.15.89:0/885025801 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff73c0a2b80 0x7ff73c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:29.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.036+0000 7ff75f5c3740 1 -- 172.21.15.89:0/2301355660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e3ee00 msgr2=0x562d87e49180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:29.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.036+0000 7ff75f5c3740 1 --2- 172.21.15.89:0/2301355660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e3ee00 0x562d87e49180 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff7480114c0 tx=0x7ff748011980 comp rx=0 tx=0).stop 2024-09-10T08:39:29.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.036+0000 7ff756800640 1 --2- 172.21.15.89:0/885025801 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff73c0a2b80 0x7ff73c0a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff750006010 tx=0x7ff7500046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:29.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.036+0000 7ff75f5c3740 1 -- 172.21.15.89:0/2301355660 shutdown_connections 2024-09-10T08:39:29.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.036+0000 7ff75f5c3740 1 --2- 172.21.15.89:0/2301355660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e496c0 0x562d87e4bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.036+0000 7ff75f5c3740 1 --2- 172.21.15.89:0/2301355660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e3ee00 0x562d87e49180 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.036+0000 7ff75f5c3740 1 --2- 172.21.15.89:0/2301355660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e3e4e0 0x562d87e3e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.036+0000 7ff75f5c3740 1 -- 172.21.15.89:0/2301355660 >> 172.21.15.89:0/2301355660 conn(0x562d87e3aa70 msgr2=0x562d87e3ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:29.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.036+0000 7ff75f5c3740 1 -- 172.21.15.89:0/2301355660 shutdown_connections 2024-09-10T08:39:29.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.036+0000 7ff75f5c3740 1 -- 172.21.15.89:0/2301355660 wait complete. 2024-09-10T08:39:29.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.037+0000 7ff75f5c3740 1 -- start start 2024-09-10T08:39:29.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.037+0000 7ff75f5c3740 1 Processor -- start 2024-09-10T08:39:29.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.038+0000 7ff75f5c3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e3e4e0 0x562d87e464b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:29.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.038+0000 7ff75f5c3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e3ee00 0x562d87e469f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:29.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.038+0000 7ff75f5c3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e496c0 0x562d87e46f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:29.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.038+0000 7ff75f5c3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562d87e4c330 con 0x562d87e3ee00 2024-09-10T08:39:29.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.038+0000 7ff75f5c3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562d87d2c920 con 0x562d87e3e4e0 2024-09-10T08:39:29.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.038+0000 7ff75f5c3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562d87e4c4b0 con 0x562d87e496c0 2024-09-10T08:39:29.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.038+0000 7ff75f5c3740 10 client.0 _ll_register_callbacks cb 0x562d87d508a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:39:29.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.038+0000 7ff74fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e3ee00 0x562d87e469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:29.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.038+0000 7ff755e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e3e4e0 0x562d87e464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:29.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.038+0000 7ff755e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e3e4e0 0x562d87e464b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:49586/0 (socket says 172.21.15.89:49586) 2024-09-10T08:39:29.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.038+0000 7ff755e00640 1 -- 172.21.15.89:0/1916413285 learned_addr learned my addr 172.21.15.89:0/1916413285 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:29.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.038+0000 7ff756800640 1 --2- 172.21.15.89:0/1916413285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e496c0 0x562d87e46f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:29.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.038+0000 7ff74fe00640 1 -- 172.21.15.89:0/1916413285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e496c0 msgr2=0x562d87e46f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:29.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.038+0000 7ff74fe00640 1 --2- 172.21.15.89:0/1916413285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e496c0 0x562d87e46f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.038+0000 7ff74fe00640 1 -- 172.21.15.89:0/1916413285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e3e4e0 msgr2=0x562d87e464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:29.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.038+0000 7ff74fe00640 1 --2- 172.21.15.89:0/1916413285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e3e4e0 0x562d87e464b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.038+0000 7ff74fe00640 1 -- 172.21.15.89:0/1916413285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d87b152b0 con 0x562d87e3ee00 2024-09-10T08:39:29.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.039+0000 7ff74fe00640 1 --2- 172.21.15.89:0/1916413285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e3ee00 0x562d87e469f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x562d87e3bdf0 tx=0x7ff740097dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104a8abc5a8ae25a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:29.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.039+0000 7ff73b400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:39:29.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.039+0000 7ff73b400640 1 -- 172.21.15.89:0/1916413285 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7400cf070 con 0x562d87e3ee00 2024-09-10T08:39:29.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.039+0000 7ff73b400640 1 -- 172.21.15.89:0/1916413285 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff7400cf210 con 0x562d87e3ee00 2024-09-10T08:39:29.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.039+0000 7ff75f5c3740 1 -- 172.21.15.89:0/1916413285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff748004cc0 con 0x562d87e3ee00 2024-09-10T08:39:29.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.039+0000 7ff73b400640 1 -- 172.21.15.89:0/1916413285 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff74000ce60 con 0x562d87e3ee00 2024-09-10T08:39:29.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.039+0000 7ff75f5c3740 10 client.15054 fetch_fsmap learned FSMap version 25 2024-09-10T08:39:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.039+0000 7ff75f5c3740 1 -- 172.21.15.89:0/1916413285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d87e57f10 con 0x562d87e3ee00 2024-09-10T08:39:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.039+0000 7ff73b400640 1 -- 172.21.15.89:0/1916413285 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff748004cc0 con 0x562d87e3ee00 2024-09-10T08:39:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.040+0000 7ff73b400640 1 -- 172.21.15.89:0/1916413285 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff74009cce0 con 0x562d87e3ee00 2024-09-10T08:39:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.040+0000 7ff75f5c3740 10 client.15054 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:39:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.040+0000 7ff75f5c3740 10 client.15054 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T08:39:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.040+0000 7ff75f5c3740 20 client.15054 populate_metadata read hostname 'smithi089' 2024-09-10T08:39:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.040+0000 7ff75f5c3740 1 --2- 172.21.15.89:0/1916413285 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x562d87e5a4c0 0x562d87e5b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.040+0000 7ff75f5c3740 4 client.15054 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:39:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.040+0000 7ff75f5c3740 1 -- 172.21.15.89:0/1916413285 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d87db1620 con 0x562d87e5a4c0 2024-09-10T08:39:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.040+0000 7ff755e00640 1 --2- 172.21.15.89:0/1916413285 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x562d87e5a4c0 0x562d87e5b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.041+0000 7ff755e00640 1 --2- 172.21.15.89:0/1916413285 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x562d87e5a4c0 0x562d87e5b930 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2aa637b8629033e1 server_cookie=7669aa8e4cd3fa66 in_seq=0 out_seq=0 2024-09-10T08:39:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.041+0000 7ff73b400640 10 client.15054 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:39:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.042+0000 7ff73b400640 1 -- 172.21.15.89:0/1916413285 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d87db1620 con 0x562d87e5a4c0 2024-09-10T08:39:29.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.042+0000 7ff73b400640 10 client.15054 handle_command_reply: tid=0 2024-09-10T08:39:29.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.042+0000 7ff75f5c3740 1 client.15054 shutdown 2024-09-10T08:39:29.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.042+0000 7ff75f5c3740 1 -- 172.21.15.89:0/1916413285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e3ee00 msgr2=0x562d87e469f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:29.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.042+0000 7ff75f5c3740 1 --2- 172.21.15.89:0/1916413285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e3ee00 0x562d87e469f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x562d87e3bdf0 tx=0x7ff740097dc0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.042+0000 7ff75f5c3740 1 -- 172.21.15.89:0/1916413285 shutdown_connections 2024-09-10T08:39:29.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.042+0000 7ff75f5c3740 1 --2- 172.21.15.89:0/1916413285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e496c0 0x562d87e46f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.042+0000 7ff75f5c3740 1 --2- 172.21.15.89:0/1916413285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e3ee00 0x562d87e469f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.042+0000 7ff75f5c3740 1 --2- 172.21.15.89:0/1916413285 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x562d87e5a4c0 0x562d87e5b930 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.042+0000 7ff75f5c3740 1 --2- 172.21.15.89:0/1916413285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e3e4e0 0x562d87e464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.042+0000 7ff75f5c3740 1 -- 172.21.15.89:0/1916413285 >> 172.21.15.89:0/1916413285 conn(0x562d87e3aa70 msgr2=0x562d87e39bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:29.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.042+0000 7ff73b400640 0 client.15054 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:39:29.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.042+0000 7ff75f5c3740 1 -- 172.21.15.89:0/1916413285 shutdown_connections 2024-09-10T08:39:29.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.042+0000 7ff75f5c3740 1 -- 172.21.15.89:0/1916413285 wait complete. 2024-09-10T08:39:29.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.043+0000 7ff75f5c3740 20 client.15054 trim_cache size 0 max 16384 2024-09-10T08:39:29.046 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:28 smithi111 ceph-mon[35220]: pgmap v379: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-10T08:39:29.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.061+0000 7ff75f5c3740 1 Processor -- start 2024-09-10T08:39:29.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.061+0000 7ff75f5c3740 1 -- start start 2024-09-10T08:39:29.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.061+0000 7ff75f5c3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e3e4e0 0x562d87e43750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.061+0000 7ff75f5c3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e3ee00 0x562d87e43c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.061+0000 7ff75f5c3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e496c0 0x562d87e44da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.061+0000 7ff75f5c3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562d87e59c70 con 0x562d87e496c0 2024-09-10T08:39:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.061+0000 7ff75f5c3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562d87e5a340 con 0x562d87e3e4e0 2024-09-10T08:39:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.061+0000 7ff75f5c3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562d87e5aa10 con 0x562d87e3ee00 2024-09-10T08:39:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.061+0000 7ff756800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e496c0 0x562d87e44da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.061+0000 7ff756800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e496c0 0x562d87e44da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50952/0 (socket says 172.21.15.89:50952) 2024-09-10T08:39:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.061+0000 7ff756800640 1 -- 172.21.15.89:0/295569150 learned_addr learned my addr 172.21.15.89:0/295569150 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.061+0000 7ff755e00640 1 --2- 172.21.15.89:0/295569150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e3e4e0 0x562d87e43750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.062+0000 7ff74fe00640 1 --2- 172.21.15.89:0/295569150 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e3ee00 0x562d87e43c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.062+0000 7ff756800640 1 -- 172.21.15.89:0/295569150 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e3ee00 msgr2=0x562d87e43c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.062+0000 7ff756800640 1 --2- 172.21.15.89:0/295569150 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e3ee00 0x562d87e43c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.062+0000 7ff756800640 1 -- 172.21.15.89:0/295569150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e3e4e0 msgr2=0x562d87e43750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:29.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.062+0000 7ff756800640 1 --2- 172.21.15.89:0/295569150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e3e4e0 0x562d87e43750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.062+0000 7ff756800640 1 -- 172.21.15.89:0/295569150 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d87e3a8a0 con 0x562d87e496c0 2024-09-10T08:39:29.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.062+0000 7ff756800640 1 --2- 172.21.15.89:0/295569150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e496c0 0x562d87e44da0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff750009900 tx=0x7ff75003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41f87a3f3898b7b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:29.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.062+0000 7ff746600640 1 -- 172.21.15.89:0/295569150 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750035040 con 0x562d87e496c0 2024-09-10T08:39:29.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.062+0000 7ff746600640 1 -- 172.21.15.89:0/295569150 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff75004f070 con 0x562d87e496c0 2024-09-10T08:39:29.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.062+0000 7ff746600640 1 -- 172.21.15.89:0/295569150 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750054900 con 0x562d87e496c0 2024-09-10T08:39:29.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.063+0000 7ff75f5c3740 1 -- 172.21.15.89:0/295569150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e496c0 msgr2=0x562d87e44da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:29.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.063+0000 7ff75f5c3740 1 --2- 172.21.15.89:0/295569150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e496c0 0x562d87e44da0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff750009900 tx=0x7ff75003f000 comp rx=0 tx=0).stop 2024-09-10T08:39:29.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.063+0000 7ff75f5c3740 1 -- 172.21.15.89:0/295569150 shutdown_connections 2024-09-10T08:39:29.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.063+0000 7ff75f5c3740 1 --2- 172.21.15.89:0/295569150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e496c0 0x562d87e44da0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.063+0000 7ff75f5c3740 1 --2- 172.21.15.89:0/295569150 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e3ee00 0x562d87e43c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.063+0000 7ff75f5c3740 1 --2- 172.21.15.89:0/295569150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e3e4e0 0x562d87e43750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.063+0000 7ff75f5c3740 1 -- 172.21.15.89:0/295569150 >> 172.21.15.89:0/295569150 conn(0x562d87e3aa70 msgr2=0x562d87e41860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:29.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.063+0000 7ff75f5c3740 1 -- 172.21.15.89:0/295569150 shutdown_connections 2024-09-10T08:39:29.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.063+0000 7ff75f5c3740 1 -- 172.21.15.89:0/295569150 wait complete. 2024-09-10T08:39:29.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.064+0000 7ff75f5c3740 1 -- start start 2024-09-10T08:39:29.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.064+0000 7ff75f5c3740 1 Processor -- start 2024-09-10T08:39:29.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.064+0000 7ff75f5c3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e3e4e0 0x562d87e644c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:29.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.064+0000 7ff75f5c3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e3ee00 0x562d87e393b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:29.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.064+0000 7ff75f5c3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e496c0 0x562d87e39b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:29.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.064+0000 7ff75f5c3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562d87e5b7b0 con 0x562d87e3ee00 2024-09-10T08:39:29.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.064+0000 7ff75f5c3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562d87e5b0e0 con 0x562d87e496c0 2024-09-10T08:39:29.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.064+0000 7ff75f5c3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562d87e5be80 con 0x562d87e3e4e0 2024-09-10T08:39:29.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.064+0000 7ff75f5c3740 10 client.0 _ll_register_callbacks cb 0x7ff7400d2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:39:29.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.065+0000 7ff755e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e3e4e0 0x562d87e644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:29.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.065+0000 7ff74fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e3ee00 0x562d87e393b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:29.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.065+0000 7ff755e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e3e4e0 0x562d87e644c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:56702/0 (socket says 172.21.15.89:56702) 2024-09-10T08:39:29.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.065+0000 7ff755e00640 1 -- 172.21.15.89:0/511503969 learned_addr learned my addr 172.21.15.89:0/511503969 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.065+0000 7ff756800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e496c0 0x562d87e39b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.065+0000 7ff756800640 1 -- 172.21.15.89:0/511503969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e3e4e0 msgr2=0x562d87e644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.065+0000 7ff756800640 1 --2- 172.21.15.89:0/511503969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e3e4e0 0x562d87e644c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.065+0000 7ff756800640 1 -- 172.21.15.89:0/511503969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e3ee00 msgr2=0x562d87e393b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.065+0000 7ff756800640 1 --2- 172.21.15.89:0/511503969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e3ee00 0x562d87e393b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.065+0000 7ff756800640 1 -- 172.21.15.89:0/511503969 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d87e61fb0 con 0x562d87e496c0 2024-09-10T08:39:29.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.065+0000 7ff74fe00640 1 --2- 172.21.15.89:0/511503969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e3ee00 0x562d87e393b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.065+0000 7ff756800640 1 --2- 172.21.15.89:0/511503969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e496c0 0x562d87e39b40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff75002aa70 tx=0x7ff750032f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a9027bd92032ad4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.065+0000 7ff73b400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:39:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.066+0000 7ff73b400640 1 -- 172.21.15.89:0/511503969 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750061060 con 0x562d87e496c0 2024-09-10T08:39:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.066+0000 7ff73b400640 1 -- 172.21.15.89:0/511503969 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff750003a00 con 0x562d87e496c0 2024-09-10T08:39:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.066+0000 7ff75f5c3740 1 -- 172.21.15.89:0/511503969 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff75004f070 con 0x562d87e496c0 2024-09-10T08:39:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.066+0000 7ff73b400640 1 -- 172.21.15.89:0/511503969 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff750068050 con 0x562d87e496c0 2024-09-10T08:39:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.066+0000 7ff75f5c3740 10 client.25112 fetch_fsmap learned FSMap version 25 2024-09-10T08:39:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.066+0000 7ff75f5c3740 1 -- 172.21.15.89:0/511503969 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d87ec4d50 con 0x562d87e496c0 2024-09-10T08:39:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.066+0000 7ff73b400640 1 -- 172.21.15.89:0/511503969 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff75004f070 con 0x562d87e496c0 2024-09-10T08:39:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.066+0000 7ff73b400640 1 -- 172.21.15.89:0/511503969 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff75003d450 con 0x562d87e496c0 2024-09-10T08:39:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.067+0000 7ff75f5c3740 10 client.25112 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:39:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.067+0000 7ff75f5c3740 10 client.25112 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T08:39:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.067+0000 7ff75f5c3740 20 client.25112 populate_metadata read hostname 'smithi089' 2024-09-10T08:39:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.067+0000 7ff75f5c3740 1 --2- 172.21.15.89:0/511503969 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x562d87ec9350 0x562d87ec8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.067+0000 7ff75f5c3740 4 client.25112 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:39:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.067+0000 7ff75f5c3740 1 -- 172.21.15.89:0/511503969 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "status"}) -- 0x562d87dabb30 con 0x562d87ec9350 2024-09-10T08:39:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.067+0000 7ff755e00640 1 --2- 172.21.15.89:0/511503969 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x562d87ec9350 0x562d87ec8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.068+0000 7ff755e00640 1 --2- 172.21.15.89:0/511503969 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x562d87ec9350 0x562d87ec8060 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c0c8d8c400fe4ae5 server_cookie=854b2bca7f259c4e in_seq=0 out_seq=0 2024-09-10T08:39:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.068+0000 7ff73b400640 10 client.25112 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:39:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.068+0000 7ff73b400640 1 -- 172.21.15.89:0/511503969 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562d87dabb30 con 0x562d87ec9350 2024-09-10T08:39:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.068+0000 7ff73b400640 10 client.25112 handle_command_reply: tid=0 2024-09-10T08:39:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.068+0000 7ff75f5c3740 1 client.25112 shutdown 2024-09-10T08:39:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.068+0000 7ff75f5c3740 1 -- 172.21.15.89:0/511503969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e496c0 msgr2=0x562d87e39b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.068+0000 7ff75f5c3740 1 --2- 172.21.15.89:0/511503969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e496c0 0x562d87e39b40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff75002aa70 tx=0x7ff750032f80 comp rx=0 tx=0).stop 2024-09-10T08:39:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.068+0000 7ff75f5c3740 1 -- 172.21.15.89:0/511503969 shutdown_connections 2024-09-10T08:39:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.068+0000 7ff75f5c3740 1 --2- 172.21.15.89:0/511503969 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x562d87ec9350 0x562d87ec8060 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.068+0000 7ff75f5c3740 1 --2- 172.21.15.89:0/511503969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562d87e496c0 0x562d87e39b40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.068+0000 7ff75f5c3740 1 --2- 172.21.15.89:0/511503969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562d87e3ee00 0x562d87e393b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.068+0000 7ff73b400640 0 client.25112 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:39:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.068+0000 7ff75f5c3740 1 --2- 172.21.15.89:0/511503969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562d87e3e4e0 0x562d87e644c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.068+0000 7ff75f5c3740 1 -- 172.21.15.89:0/511503969 >> 172.21.15.89:0/511503969 conn(0x562d87e3aa70 msgr2=0x562d87e44da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.068+0000 7ff75f5c3740 1 -- 172.21.15.89:0/511503969 shutdown_connections 2024-09-10T08:39:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.068+0000 7ff75f5c3740 1 -- 172.21.15.89:0/511503969 wait complete. 2024-09-10T08:39:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.068+0000 7ff75f5c3740 20 client.25112 trim_cache size 0 max 16384 2024-09-10T08:39:29.073 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:39:29.073 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "91f747bc-6f4e-11ef-bcea-c7b262605968", 2024-09-10T08:39:29.073 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 2, 2024-09-10T08:39:29.073 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24496, 2024-09-10T08:39:29.073 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2024-09-10T08:39:29.073 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2024-09-10T08:39:29.074 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2024-09-10T08:39:29.074 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 101.36405368200001, 2024-09-10T08:39:29.074 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 25, 2024-09-10T08:39:29.074 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 96, 2024-09-10T08:39:29.074 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T08:39:29.074 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 173.602134232 2024-09-10T08:39:29.074 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:39:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.069+0000 7ff757200640 1 -- 172.21.15.89:0/885025801 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff73c0a2b80 msgr2=0x7ff73c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.069+0000 7ff757200640 1 --2- 172.21.15.89:0/885025801 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff73c0a2b80 0x7ff73c0a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff750006010 tx=0x7ff7500046c0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.069+0000 7ff757200640 1 -- 172.21.15.89:0/885025801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7581034b0 msgr2=0x7ff75819a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.069+0000 7ff757200640 1 --2- 172.21.15.89:0/885025801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7581034b0 0x7ff75819a9c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff74000e7a0 tx=0x7ff74000ec60 comp rx=0 tx=0).stop 2024-09-10T08:39:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.069+0000 7ff757200640 1 -- 172.21.15.89:0/885025801 shutdown_connections 2024-09-10T08:39:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.069+0000 7ff757200640 1 --2- 172.21.15.89:0/885025801 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff73c0a2b80 0x7ff73c0a5040 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.069+0000 7ff757200640 1 --2- 172.21.15.89:0/885025801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff75810bfc0 0x7ff75819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.069+0000 7ff757200640 1 --2- 172.21.15.89:0/885025801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7581034b0 0x7ff75819a9c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.069+0000 7ff757200640 1 --2- 172.21.15.89:0/885025801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff758102b90 0x7ff75819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.069+0000 7ff757200640 1 -- 172.21.15.89:0/885025801 >> 172.21.15.89:0/885025801 conn(0x7ff7580fc610 msgr2=0x7ff75810d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:29.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.069+0000 7ff757200640 1 -- 172.21.15.89:0/885025801 shutdown_connections 2024-09-10T08:39:29.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:29.069+0000 7ff757200640 1 -- 172.21.15.89:0/885025801 wait complete. 2024-09-10T08:39:29.798 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T08:39:30.163 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:30.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.724+0000 7f1853200640 1 -- 172.21.15.89:0/2229015342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f184c103e30 msgr2=0x7f184c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:30.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.724+0000 7f1853200640 1 --2- 172.21.15.89:0/2229015342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f184c103e30 0x7f184c104210 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1840009860 tx=0x7f184002a340 comp rx=0 tx=0).stop 2024-09-10T08:39:30.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.724+0000 7f1853200640 1 -- 172.21.15.89:0/2229015342 shutdown_connections 2024-09-10T08:39:30.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.724+0000 7f1853200640 1 --2- 172.21.15.89:0/2229015342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c108f60 0x7f184c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.724+0000 7f1853200640 1 --2- 172.21.15.89:0/2229015342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f184c1047e0 0x7f184c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.724+0000 7f1853200640 1 --2- 172.21.15.89:0/2229015342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f184c103e30 0x7f184c104210 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.724+0000 7f1853200640 1 -- 172.21.15.89:0/2229015342 >> 172.21.15.89:0/2229015342 conn(0x7f184c0ff8d0 msgr2=0x7f184c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:30.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.724+0000 7f1853200640 1 -- 172.21.15.89:0/2229015342 shutdown_connections 2024-09-10T08:39:30.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.724+0000 7f1853200640 1 -- 172.21.15.89:0/2229015342 wait complete. 2024-09-10T08:39:30.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.725+0000 7f1853200640 1 Processor -- start 2024-09-10T08:39:30.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.726+0000 7f1853200640 1 -- start start 2024-09-10T08:39:30.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.726+0000 7f1853200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f184c103e30 0x7f184c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:30.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.726+0000 7f1853200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f184c1047e0 0x7f184c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:30.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.726+0000 7f1853200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c108f60 0x7f184c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:30.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.726+0000 7f1853200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f184c113fc0 con 0x7f184c108f60 2024-09-10T08:39:30.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.726+0000 7f1853200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f184c113e40 con 0x7f184c1047e0 2024-09-10T08:39:30.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.726+0000 7f1853200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f184c114140 con 0x7f184c103e30 2024-09-10T08:39:30.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.726+0000 7f1851e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f184c103e30 0x7f184c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:30.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.726+0000 7f1851400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f184c1047e0 0x7f184c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:30.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.726+0000 7f1851400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f184c1047e0 0x7f184c19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:49624/0 (socket says 172.21.15.89:49624) 2024-09-10T08:39:30.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.726+0000 7f1851e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f184c103e30 0x7f184c19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:56714/0 (socket says 172.21.15.89:56714) 2024-09-10T08:39:30.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.726+0000 7f1851400640 1 -- 172.21.15.89:0/2964268154 learned_addr learned my addr 172.21.15.89:0/2964268154 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:30.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.726+0000 7f1852800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c108f60 0x7f184c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:30.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.727+0000 7f1851400640 1 -- 172.21.15.89:0/2964268154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f184c103e30 msgr2=0x7f184c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:30.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.727+0000 7f1851400640 1 --2- 172.21.15.89:0/2964268154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f184c103e30 0x7f184c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.727+0000 7f1851400640 1 -- 172.21.15.89:0/2964268154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c108f60 msgr2=0x7f184c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:30.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.727+0000 7f1851400640 1 --2- 172.21.15.89:0/2964268154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c108f60 0x7f184c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.727+0000 7f1851400640 1 -- 172.21.15.89:0/2964268154 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184c19f450 con 0x7f184c1047e0 2024-09-10T08:39:30.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.727+0000 7f1851e00640 1 --2- 172.21.15.89:0/2964268154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f184c103e30 0x7f184c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:30.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.727+0000 7f1851400640 1 --2- 172.21.15.89:0/2964268154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f184c1047e0 0x7f184c19a9e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f183c00e9f0 tx=0x7f183c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:30.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.727+0000 7f1846a00640 1 -- 172.21.15.89:0/2964268154 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c00cd80 con 0x7f184c1047e0 2024-09-10T08:39:30.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.727+0000 7f1846a00640 1 -- 172.21.15.89:0/2964268154 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f183c005500 con 0x7f184c1047e0 2024-09-10T08:39:30.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.727+0000 7f1853200640 1 -- 172.21.15.89:0/2964268154 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184c19f740 con 0x7f184c1047e0 2024-09-10T08:39:30.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.727+0000 7f1846a00640 1 -- 172.21.15.89:0/2964268154 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c010690 con 0x7f184c1047e0 2024-09-10T08:39:30.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.727+0000 7f1853200640 1 -- 172.21.15.89:0/2964268154 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f184c1a7020 con 0x7f184c1047e0 2024-09-10T08:39:30.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.730+0000 7f1846a00640 1 -- 172.21.15.89:0/2964268154 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f183c0046c0 con 0x7f184c1047e0 2024-09-10T08:39:30.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.730+0000 7f1857265740 1 Processor -- start 2024-09-10T08:39:30.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.731+0000 7f1857265740 1 -- start start 2024-09-10T08:39:30.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.731+0000 7f1857265740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e2194904e0 0x55e2194908c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:30.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.731+0000 7f1857265740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e219490e00 0x55e21949b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:30.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.731+0000 7f1857265740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e21949b6c0 0x55e21949dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:30.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.731+0000 7f1857265740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e2193f60b0 con 0x55e2194904e0 2024-09-10T08:39:30.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.731+0000 7f1857265740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e21939dab0 con 0x55e219490e00 2024-09-10T08:39:30.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.731+0000 7f1857265740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e2193f2070 con 0x55e21949b6c0 2024-09-10T08:39:30.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.731+0000 7f1852800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e2194904e0 0x55e2194908c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:30.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.731+0000 7f1851e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e219490e00 0x55e21949b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:30.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.731+0000 7f1852800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e2194904e0 0x55e2194908c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50992/0 (socket says 172.21.15.89:50992) 2024-09-10T08:39:30.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.731+0000 7f1852800640 1 -- 172.21.15.89:0/321524348 learned_addr learned my addr 172.21.15.89:0/321524348 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:30.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.731+0000 7f1851400640 1 --2- 172.21.15.89:0/321524348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e21949b6c0 0x55e21949dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:30.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.732+0000 7f1852800640 1 -- 172.21.15.89:0/321524348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e21949b6c0 msgr2=0x55e21949dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:30.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.732+0000 7f1852800640 1 --2- 172.21.15.89:0/321524348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e21949b6c0 0x55e21949dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.732+0000 7f1852800640 1 -- 172.21.15.89:0/321524348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e219490e00 msgr2=0x55e21949b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.732+0000 7f1852800640 1 --2- 172.21.15.89:0/321524348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e219490e00 0x55e21949b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.732+0000 7f1852800640 1 -- 172.21.15.89:0/321524348 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2193aa4a0 con 0x55e2194904e0 2024-09-10T08:39:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.732+0000 7f1851e00640 1 --2- 172.21.15.89:0/321524348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e219490e00 0x55e21949b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.732+0000 7f1846a00640 1 --2- 172.21.15.89:0/2964268154 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f18340a2b80 0x7f18340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.732+0000 7f1846a00640 1 -- 172.21.15.89:0/2964268154 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f183c0cf850 con 0x7f184c1047e0 2024-09-10T08:39:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.732+0000 7f1851e00640 1 --2- 172.21.15.89:0/2964268154 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f18340a2b80 0x7f18340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.733+0000 7f1852800640 1 --2- 172.21.15.89:0/321524348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e2194904e0 0x55e2194908c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f18480099f0 tx=0x7f18480095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadf8afec8c22c1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.733+0000 7f183b000640 1 -- 172.21.15.89:0/321524348 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184801d070 con 0x55e2194904e0 2024-09-10T08:39:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.733+0000 7f183b000640 1 -- 172.21.15.89:0/321524348 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1848003e40 con 0x55e2194904e0 2024-09-10T08:39:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.733+0000 7f183b000640 1 -- 172.21.15.89:0/321524348 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184800db40 con 0x55e2194904e0 2024-09-10T08:39:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.733+0000 7f1857265740 1 -- 172.21.15.89:0/321524348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e2194904e0 msgr2=0x55e2194908c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.733+0000 7f1857265740 1 --2- 172.21.15.89:0/321524348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e2194904e0 0x55e2194908c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f18480099f0 tx=0x7f18480095c0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.733+0000 7f1851e00640 1 --2- 172.21.15.89:0/2964268154 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f18340a2b80 0x7f18340a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f184002a930 tx=0x7f18400058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.734+0000 7f1857265740 1 -- 172.21.15.89:0/321524348 shutdown_connections 2024-09-10T08:39:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.734+0000 7f1857265740 1 --2- 172.21.15.89:0/321524348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e21949b6c0 0x55e21949dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.734+0000 7f1857265740 1 --2- 172.21.15.89:0/321524348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e219490e00 0x55e21949b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.734+0000 7f1857265740 1 --2- 172.21.15.89:0/321524348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e2194904e0 0x55e2194908c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.734+0000 7f1857265740 1 -- 172.21.15.89:0/321524348 >> 172.21.15.89:0/321524348 conn(0x55e21948ca70 msgr2=0x55e21948ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.734+0000 7f1857265740 1 -- 172.21.15.89:0/321524348 shutdown_connections 2024-09-10T08:39:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.734+0000 7f1857265740 1 -- 172.21.15.89:0/321524348 wait complete. 2024-09-10T08:39:30.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.734+0000 7f1857265740 1 -- start start 2024-09-10T08:39:30.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.734+0000 7f1857265740 1 Processor -- start 2024-09-10T08:39:30.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.735+0000 7f1857265740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2194904e0 0x55e219498590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:30.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.735+0000 7f1857265740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e219490e00 0x55e219498ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:30.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.735+0000 7f1857265740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e21949b6c0 0x55e219499010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:30.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.735+0000 7f1857265740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e21949e380 con 0x55e21949b6c0 2024-09-10T08:39:30.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.735+0000 7f1857265740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e21937e920 con 0x55e219490e00 2024-09-10T08:39:30.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.735+0000 7f1857265740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e21949e500 con 0x55e2194904e0 2024-09-10T08:39:30.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.735+0000 7f1857265740 10 client.0 _ll_register_callbacks cb 0x55e2193a28a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:39:30.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.735+0000 7f1851e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2194904e0 0x55e219498590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:30.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.735+0000 7f1852800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e21949b6c0 0x55e219499010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:30.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.735+0000 7f1851400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e219490e00 0x55e219498ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:30.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.735+0000 7f1851e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2194904e0 0x55e219498590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:56742/0 (socket says 172.21.15.89:56742) 2024-09-10T08:39:30.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.735+0000 7f1851400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e219490e00 0x55e219498ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:49652/0 (socket says 172.21.15.89:49652) 2024-09-10T08:39:30.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.735+0000 7f1851e00640 1 -- 172.21.15.89:0/65403880 learned_addr learned my addr 172.21.15.89:0/65403880 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:30.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.736+0000 7f1851e00640 1 -- 172.21.15.89:0/65403880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e219490e00 msgr2=0x55e219498ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:30.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.736+0000 7f1851e00640 1 --2- 172.21.15.89:0/65403880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e219490e00 0x55e219498ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.736+0000 7f1851e00640 1 -- 172.21.15.89:0/65403880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e21949b6c0 msgr2=0x55e219499010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:30.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.736+0000 7f1851e00640 1 --2- 172.21.15.89:0/65403880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e21949b6c0 0x55e219499010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.736+0000 7f1851e00640 1 -- 172.21.15.89:0/65403880 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2191672b0 con 0x55e2194904e0 2024-09-10T08:39:30.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.736+0000 7f1852800640 1 --2- 172.21.15.89:0/65403880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e21949b6c0 0x55e219499010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:30.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.736+0000 7f1851e00640 1 --2- 172.21.15.89:0/65403880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2194904e0 0x55e219498590 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1840031000 tx=0x7f184002a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c4b4086cfc7735a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:30.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.736+0000 7f182f400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:39:30.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.736+0000 7f182f400640 1 -- 172.21.15.89:0/65403880 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840037040 con 0x55e2194904e0 2024-09-10T08:39:30.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.736+0000 7f182f400640 1 -- 172.21.15.89:0/65403880 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f1840003b80 con 0x55e2194904e0 2024-09-10T08:39:30.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.736+0000 7f1857265740 1 -- 172.21.15.89:0/65403880 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e219403620 con 0x55e2194904e0 2024-09-10T08:39:30.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.737+0000 7f182f400640 1 -- 172.21.15.89:0/65403880 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1840056050 con 0x55e2194904e0 2024-09-10T08:39:30.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.737+0000 7f1857265740 10 client.25096 fetch_fsmap learned FSMap version 25 2024-09-10T08:39:30.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.737+0000 7f1857265740 1 -- 172.21.15.89:0/65403880 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e2194aa010 con 0x55e2194904e0 2024-09-10T08:39:30.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.737+0000 7f182f400640 1 -- 172.21.15.89:0/65403880 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e219403620 con 0x55e2194904e0 2024-09-10T08:39:30.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.737+0000 7f182f400640 1 -- 172.21.15.89:0/65403880 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f184003d450 con 0x55e2194904e0 2024-09-10T08:39:30.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.737+0000 7f1857265740 10 client.25096 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:39:30.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.737+0000 7f1857265740 10 client.25096 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T08:39:30.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.738+0000 7f1857265740 20 client.25096 populate_metadata read hostname 'smithi089' 2024-09-10T08:39:30.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.738+0000 7f1857265740 1 --2- 172.21.15.89:0/65403880 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55e2194ac650 0x55e2194adac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:30.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.738+0000 7f1857265740 4 client.25096 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:39:30.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.738+0000 7f1857265740 1 -- 172.21.15.89:0/65403880 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2193fdb30 con 0x55e2194ac650 2024-09-10T08:39:30.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.738+0000 7f1851400640 1 --2- 172.21.15.89:0/65403880 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55e2194ac650 0x55e2194adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:30.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.738+0000 7f1851400640 1 --2- 172.21.15.89:0/65403880 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55e2194ac650 0x55e2194adac0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b4e7ccb0e8c5157 server_cookie=732ba7d8a1869d0 in_seq=0 out_seq=0 2024-09-10T08:39:30.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.738+0000 7f182f400640 10 client.25096 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:39:30.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.739+0000 7f182f400640 1 -- 172.21.15.89:0/65403880 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e2193fdb30 con 0x55e2194ac650 2024-09-10T08:39:30.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.739+0000 7f182f400640 10 client.25096 handle_command_reply: tid=0 2024-09-10T08:39:30.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.739+0000 7f1857265740 1 client.25096 shutdown 2024-09-10T08:39:30.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.739+0000 7f1857265740 1 -- 172.21.15.89:0/65403880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2194904e0 msgr2=0x55e219498590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:30.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.739+0000 7f1857265740 1 --2- 172.21.15.89:0/65403880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2194904e0 0x55e219498590 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1840031000 tx=0x7f184002a340 comp rx=0 tx=0).stop 2024-09-10T08:39:30.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.740+0000 7f1857265740 1 -- 172.21.15.89:0/65403880 shutdown_connections 2024-09-10T08:39:30.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.740+0000 7f1857265740 1 --2- 172.21.15.89:0/65403880 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55e2194ac650 0x55e2194adac0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.740+0000 7f1857265740 1 --2- 172.21.15.89:0/65403880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e21949b6c0 0x55e219499010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.740+0000 7f1857265740 1 --2- 172.21.15.89:0/65403880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e219490e00 0x55e219498ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.740+0000 7f182f400640 0 client.25096 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:39:30.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.740+0000 7f1857265740 1 --2- 172.21.15.89:0/65403880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2194904e0 0x55e219498590 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.740+0000 7f1857265740 1 -- 172.21.15.89:0/65403880 >> 172.21.15.89:0/65403880 conn(0x55e21948ca70 msgr2=0x55e21948bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:30.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.740+0000 7f1857265740 1 -- 172.21.15.89:0/65403880 shutdown_connections 2024-09-10T08:39:30.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.740+0000 7f1857265740 1 -- 172.21.15.89:0/65403880 wait complete. 2024-09-10T08:39:30.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.740+0000 7f1857265740 20 client.25096 trim_cache size 0 max 16384 2024-09-10T08:39:30.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.759+0000 7f1857265740 1 Processor -- start 2024-09-10T08:39:30.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.759+0000 7f1857265740 1 -- start start 2024-09-10T08:39:30.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.759+0000 7f1857265740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2194904e0 0x55e219492f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:30.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.759+0000 7f1857265740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e219490e00 0x55e21949a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:30.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.759+0000 7f1857265740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e21949b6c0 0x55e219498ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:30.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.759+0000 7f1857265740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e2194abe00 con 0x55e21949b6c0 2024-09-10T08:39:30.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.759+0000 7f1857265740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e2194ac4d0 con 0x55e219490e00 2024-09-10T08:39:30.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.759+0000 7f1857265740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e2194acba0 con 0x55e2194904e0 2024-09-10T08:39:30.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.759+0000 7f1851e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2194904e0 0x55e219492f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:30.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.759+0000 7f1851e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2194904e0 0x55e219492f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:56748/0 (socket says 172.21.15.89:56748) 2024-09-10T08:39:30.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.759+0000 7f1851e00640 1 -- 172.21.15.89:0/663560962 learned_addr learned my addr 172.21.15.89:0/663560962 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:30.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.759+0000 7f1852800640 1 --2- 172.21.15.89:0/663560962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e21949b6c0 0x55e219498ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:30.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.759+0000 7f1851400640 1 --2- 172.21.15.89:0/663560962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e219490e00 0x55e21949a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:30.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.760+0000 7f1851e00640 1 -- 172.21.15.89:0/663560962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e219490e00 msgr2=0x55e21949a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:30.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.760+0000 7f1851e00640 1 --2- 172.21.15.89:0/663560962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e219490e00 0x55e21949a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.760+0000 7f1851e00640 1 -- 172.21.15.89:0/663560962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e21949b6c0 msgr2=0x55e219498ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:30.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.760+0000 7f1851e00640 1 --2- 172.21.15.89:0/663560962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e21949b6c0 0x55e219498ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.760+0000 7f1851e00640 1 -- 172.21.15.89:0/663560962 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e219496ae0 con 0x55e2194904e0 2024-09-10T08:39:30.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.760+0000 7f1851e00640 1 --2- 172.21.15.89:0/663560962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2194904e0 0x55e219492f90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1840005b60 tx=0x7f1840005af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94aec525e969d30a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:30.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.760+0000 7f183a600640 1 -- 172.21.15.89:0/663560962 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840037040 con 0x55e2194904e0 2024-09-10T08:39:30.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.760+0000 7f183a600640 1 -- 172.21.15.89:0/663560962 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f184004d070 con 0x55e2194904e0 2024-09-10T08:39:30.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.760+0000 7f183a600640 1 -- 172.21.15.89:0/663560962 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18400486b0 con 0x55e2194904e0 2024-09-10T08:39:30.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.761+0000 7f1857265740 1 -- 172.21.15.89:0/663560962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2194904e0 msgr2=0x55e219492f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:30.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.761+0000 7f1857265740 1 --2- 172.21.15.89:0/663560962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2194904e0 0x55e219492f90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1840005b60 tx=0x7f1840005af0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.761+0000 7f1857265740 1 -- 172.21.15.89:0/663560962 shutdown_connections 2024-09-10T08:39:30.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.761+0000 7f1857265740 1 --2- 172.21.15.89:0/663560962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e21949b6c0 0x55e219498ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.761+0000 7f1857265740 1 --2- 172.21.15.89:0/663560962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e219490e00 0x55e21949a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.761+0000 7f1857265740 1 --2- 172.21.15.89:0/663560962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2194904e0 0x55e219492f90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.761+0000 7f1857265740 1 -- 172.21.15.89:0/663560962 >> 172.21.15.89:0/663560962 conn(0x55e21948ca70 msgr2=0x55e219499df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:30.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.761+0000 7f1857265740 1 -- 172.21.15.89:0/663560962 shutdown_connections 2024-09-10T08:39:30.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.761+0000 7f1857265740 1 -- 172.21.15.89:0/663560962 wait complete. 2024-09-10T08:39:30.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.761+0000 7f1857265740 1 -- start start 2024-09-10T08:39:30.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.762+0000 7f1857265740 1 Processor -- start 2024-09-10T08:39:30.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.762+0000 7f1857265740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2194904e0 0x55e2194b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:30.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.762+0000 7f1857265740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e219490e00 0x55e2194af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:30.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.762+0000 7f1857265740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e21949b6c0 0x55e2194afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:30.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.762+0000 7f1857265740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e2194ad940 con 0x55e21949b6c0 2024-09-10T08:39:30.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.762+0000 7f1857265740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e2194ad270 con 0x55e2194904e0 2024-09-10T08:39:30.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.762+0000 7f1857265740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e2194ae010 con 0x55e219490e00 2024-09-10T08:39:30.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.762+0000 7f1857265740 10 client.0 _ll_register_callbacks cb 0x7f1840009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:39:30.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.762+0000 7f1851400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e219490e00 0x55e2194af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:30.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.762+0000 7f1851400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e219490e00 0x55e2194af590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:56756/0 (socket says 172.21.15.89:56756) 2024-09-10T08:39:30.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.762+0000 7f1851400640 1 -- 172.21.15.89:0/1953311810 learned_addr learned my addr 172.21.15.89:0/1953311810 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:30.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.762+0000 7f1852800640 1 --2- 172.21.15.89:0/1953311810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e21949b6c0 0x55e2194afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:30.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.762+0000 7f1851e00640 1 --2- 172.21.15.89:0/1953311810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2194904e0 0x55e2194b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:30.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.763+0000 7f1851400640 1 -- 172.21.15.89:0/1953311810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2194904e0 msgr2=0x55e2194b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:30.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.763+0000 7f1851400640 1 --2- 172.21.15.89:0/1953311810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2194904e0 0x55e2194b6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.763+0000 7f1851400640 1 -- 172.21.15.89:0/1953311810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e21949b6c0 msgr2=0x55e2194afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:30.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.763+0000 7f1851400640 1 --2- 172.21.15.89:0/1953311810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e21949b6c0 0x55e2194afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.763+0000 7f1851400640 1 -- 172.21.15.89:0/1953311810 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2194b0040 con 0x55e219490e00 2024-09-10T08:39:30.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.763+0000 7f1851400640 1 --2- 172.21.15.89:0/1953311810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e219490e00 0x55e2194af590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f183c09bc10 tx=0x7f183c0d5530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4828caa2bb7d0b16 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:30.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.763+0000 7f182f400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:39:30.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.763+0000 7f182f400640 1 -- 172.21.15.89:0/1953311810 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c09b5e0 con 0x55e219490e00 2024-09-10T08:39:30.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.763+0000 7f1857265740 1 -- 172.21.15.89:0/1953311810 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f184004d070 con 0x55e219490e00 2024-09-10T08:39:30.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.763+0000 7f182f400640 1 -- 172.21.15.89:0/1953311810 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f183c0982f0 con 0x55e219490e00 2024-09-10T08:39:30.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.764+0000 7f182f400640 1 -- 172.21.15.89:0/1953311810 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f183c09c380 con 0x55e219490e00 2024-09-10T08:39:30.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.764+0000 7f1857265740 10 client.25102 fetch_fsmap learned FSMap version 25 2024-09-10T08:39:30.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.764+0000 7f1857265740 1 -- 172.21.15.89:0/1953311810 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e21951f570 con 0x55e219490e00 2024-09-10T08:39:30.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.764+0000 7f182f400640 1 -- 172.21.15.89:0/1953311810 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f184004d070 con 0x55e219490e00 2024-09-10T08:39:30.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.764+0000 7f182f400640 1 -- 172.21.15.89:0/1953311810 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f183c0a4460 con 0x55e219490e00 2024-09-10T08:39:30.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.764+0000 7f1857265740 10 client.25102 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:39:30.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.765+0000 7f1857265740 10 client.25102 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T08:39:30.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.765+0000 7f1857265740 20 client.25102 populate_metadata read hostname 'smithi089' 2024-09-10T08:39:30.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.765+0000 7f1857265740 1 --2- 172.21.15.89:0/1953311810 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55e219523ee0 0x55e219522bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:30.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.765+0000 7f1857265740 4 client.25102 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:39:30.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.765+0000 7f1857265740 1 -- 172.21.15.89:0/1953311810 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "status"}) -- 0x55e219379f80 con 0x55e219523ee0 2024-09-10T08:39:30.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.765+0000 7f1851e00640 1 --2- 172.21.15.89:0/1953311810 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55e219523ee0 0x55e219522bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:30.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.765+0000 7f1851e00640 1 --2- 172.21.15.89:0/1953311810 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55e219523ee0 0x55e219522bf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f690e06568c884 server_cookie=4cf1ceb5faae383 in_seq=0 out_seq=0 2024-09-10T08:39:30.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.765+0000 7f182f400640 10 client.25102 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:39:30.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.766+0000 7f182f400640 1 -- 172.21.15.89:0/1953311810 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55e219379f80 con 0x55e219523ee0 2024-09-10T08:39:30.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.766+0000 7f182f400640 10 client.25102 handle_command_reply: tid=0 2024-09-10T08:39:30.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.766+0000 7f1857265740 1 client.25102 shutdown 2024-09-10T08:39:30.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.766+0000 7f1857265740 1 -- 172.21.15.89:0/1953311810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e219490e00 msgr2=0x55e2194af590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:30.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.766+0000 7f1857265740 1 --2- 172.21.15.89:0/1953311810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e219490e00 0x55e2194af590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f183c09bc10 tx=0x7f183c0d5530 comp rx=0 tx=0).stop 2024-09-10T08:39:30.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.766+0000 7f1857265740 1 -- 172.21.15.89:0/1953311810 shutdown_connections 2024-09-10T08:39:30.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.766+0000 7f1857265740 1 --2- 172.21.15.89:0/1953311810 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55e219523ee0 0x55e219522bf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.766+0000 7f1857265740 1 --2- 172.21.15.89:0/1953311810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e21949b6c0 0x55e2194afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.766+0000 7f1857265740 1 --2- 172.21.15.89:0/1953311810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e219490e00 0x55e2194af590 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.766+0000 7f1857265740 1 --2- 172.21.15.89:0/1953311810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2194904e0 0x55e2194b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.766+0000 7f182f400640 0 client.25102 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:39:30.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.766+0000 7f1857265740 1 -- 172.21.15.89:0/1953311810 >> 172.21.15.89:0/1953311810 conn(0x55e21948ca70 msgr2=0x55e2194a6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:30.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.766+0000 7f1857265740 1 -- 172.21.15.89:0/1953311810 shutdown_connections 2024-09-10T08:39:30.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.766+0000 7f1857265740 1 -- 172.21.15.89:0/1953311810 wait complete. 2024-09-10T08:39:30.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.766+0000 7f1857265740 20 client.25102 trim_cache size 0 max 16384 2024-09-10T08:39:30.770 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:39:30.771 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "91f747bc-6f4e-11ef-bcea-c7b262605968", 2024-09-10T08:39:30.771 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 1, 2024-09-10T08:39:30.771 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24518, 2024-09-10T08:39:30.771 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2024-09-10T08:39:30.771 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2024-09-10T08:39:30.771 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2024-09-10T08:39:30.771 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 104.074081466, 2024-09-10T08:39:30.771 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 25, 2024-09-10T08:39:30.771 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 96, 2024-09-10T08:39:30.771 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T08:39:30.771 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 177.930929316 2024-09-10T08:39:30.771 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:39:30.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.767+0000 7f1853200640 1 -- 172.21.15.89:0/2964268154 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f18340a2b80 msgr2=0x7f18340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:30.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.767+0000 7f1853200640 1 --2- 172.21.15.89:0/2964268154 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f18340a2b80 0x7f18340a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f184002a930 tx=0x7f18400058d0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.767+0000 7f1853200640 1 -- 172.21.15.89:0/2964268154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f184c1047e0 msgr2=0x7f184c19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:30.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.767+0000 7f1853200640 1 --2- 172.21.15.89:0/2964268154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f184c1047e0 0x7f184c19a9e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f183c00e9f0 tx=0x7f183c00edf0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.767+0000 7f1853200640 1 -- 172.21.15.89:0/2964268154 shutdown_connections 2024-09-10T08:39:30.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.767+0000 7f1853200640 1 --2- 172.21.15.89:0/2964268154 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f18340a2b80 0x7f18340a5040 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.767+0000 7f1853200640 1 --2- 172.21.15.89:0/2964268154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c108f60 0x7f184c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.767+0000 7f1853200640 1 --2- 172.21.15.89:0/2964268154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f184c1047e0 0x7f184c19a9e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.767+0000 7f1853200640 1 --2- 172.21.15.89:0/2964268154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f184c103e30 0x7f184c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:30.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.767+0000 7f1853200640 1 -- 172.21.15.89:0/2964268154 >> 172.21.15.89:0/2964268154 conn(0x7f184c0ff8d0 msgr2=0x7f184c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:30.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.767+0000 7f1853200640 1 -- 172.21.15.89:0/2964268154 shutdown_connections 2024-09-10T08:39:30.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:30.767+0000 7f1853200640 1 -- 172.21.15.89:0/2964268154 wait complete. 2024-09-10T08:39:30.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:30 smithi149 ceph-mon[36167]: pgmap v380: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:31.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:30 smithi111 ceph-mon[35220]: pgmap v380: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:31.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:30 smithi089 ceph-mon[30757]: pgmap v380: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:31.541 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T08:39:31.909 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:32.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.478+0000 7f67fca00640 1 -- 172.21.15.89:0/607829733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67f8102370 msgr2=0x7f67f810e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:32.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.478+0000 7f67fca00640 1 --2- 172.21.15.89:0/607829733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67f8102370 0x7f67f810e8a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f67f0009ca0 tx=0x7f67f002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.478+0000 7f67fca00640 1 -- 172.21.15.89:0/607829733 shutdown_connections 2024-09-10T08:39:32.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.478+0000 7f67fca00640 1 --2- 172.21.15.89:0/607829733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67f8102370 0x7f67f810e8a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.478+0000 7f67fca00640 1 --2- 172.21.15.89:0/607829733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67f81019d0 0x7f67f8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.478+0000 7f67fca00640 1 --2- 172.21.15.89:0/607829733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f81079d0 0x7f67f8107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.478+0000 7f67fca00640 1 -- 172.21.15.89:0/607829733 >> 172.21.15.89:0/607829733 conn(0x7f67f80fd710 msgr2=0x7f67f80ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:32.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.479+0000 7f67fca00640 1 -- 172.21.15.89:0/607829733 shutdown_connections 2024-09-10T08:39:32.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.479+0000 7f67fca00640 1 -- 172.21.15.89:0/607829733 wait complete. 2024-09-10T08:39:32.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.479+0000 7f67fca00640 1 Processor -- start 2024-09-10T08:39:32.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.480+0000 7f67fca00640 1 -- start start 2024-09-10T08:39:32.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.480+0000 7f67fca00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67f81019d0 0x7f67f80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:32.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.480+0000 7f67fca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f8102370 0x7f67f8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:32.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.480+0000 7f67fca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67f81079d0 0x7f67f8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:32.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.480+0000 7f67fca00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f67f8111e20 con 0x7f67f8102370 2024-09-10T08:39:32.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.480+0000 7f67fca00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f67f8111ca0 con 0x7f67f81019d0 2024-09-10T08:39:32.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.480+0000 7f67fca00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f67f8111fa0 con 0x7f67f81079d0 2024-09-10T08:39:32.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.480+0000 7f67f5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f8102370 0x7f67f8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:32.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.480+0000 7f67f6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67f81019d0 0x7f67f80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:32.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.480+0000 7f67f6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67f81019d0 0x7f67f80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:49682/0 (socket says 172.21.15.89:49682) 2024-09-10T08:39:32.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.480+0000 7f67f6800640 1 -- 172.21.15.89:0/4244222382 learned_addr learned my addr 172.21.15.89:0/4244222382 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:32.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.480+0000 7f67f7200640 1 --2- 172.21.15.89:0/4244222382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67f81079d0 0x7f67f8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:32.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.481+0000 7f67f6800640 1 -- 172.21.15.89:0/4244222382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67f81079d0 msgr2=0x7f67f8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:32.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.481+0000 7f67f6800640 1 --2- 172.21.15.89:0/4244222382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67f81079d0 0x7f67f8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.481+0000 7f67f6800640 1 -- 172.21.15.89:0/4244222382 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f8102370 msgr2=0x7f67f8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:32.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.481+0000 7f67f6800640 1 --2- 172.21.15.89:0/4244222382 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f8102370 0x7f67f8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.481+0000 7f67f6800640 1 -- 172.21.15.89:0/4244222382 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f81a6750 con 0x7f67f81019d0 2024-09-10T08:39:32.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.481+0000 7f67f6800640 1 --2- 172.21.15.89:0/4244222382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67f81019d0 0x7f67f80744f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f67e800e9f0 tx=0x7f67e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:32.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.481+0000 7f67ef400640 1 -- 172.21.15.89:0/4244222382 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e800cd80 con 0x7f67f81019d0 2024-09-10T08:39:32.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.481+0000 7f67fca00640 1 -- 172.21.15.89:0/4244222382 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f81a6a40 con 0x7f67f81019d0 2024-09-10T08:39:32.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.481+0000 7f67ef400640 1 -- 172.21.15.89:0/4244222382 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e8010910 con 0x7f67f81019d0 2024-09-10T08:39:32.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.482+0000 7f67ef400640 1 -- 172.21.15.89:0/4244222382 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e80188f0 con 0x7f67f81019d0 2024-09-10T08:39:32.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.482+0000 7f67fca00640 1 -- 172.21.15.89:0/4244222382 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f81a6f80 con 0x7f67f81019d0 2024-09-10T08:39:32.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.485+0000 7f67fffe9740 1 Processor -- start 2024-09-10T08:39:32.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.485+0000 7f67fffe9740 1 -- start start 2024-09-10T08:39:32.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.485+0000 7f67ef400640 1 -- 172.21.15.89:0/4244222382 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f67e8010ac0 con 0x7f67f81019d0 2024-09-10T08:39:32.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.485+0000 7f67fffe9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f84e0 0x55e3fb7f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:32.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.485+0000 7f67fffe9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb7f8e00 0x55e3fb803180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:32.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.485+0000 7f67fffe9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb8036c0 0x55e3fb805ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:32.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.485+0000 7f67fffe9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e3fb75e0b0 con 0x55e3fb8036c0 2024-09-10T08:39:32.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.485+0000 7f67fffe9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e3fb705ab0 con 0x55e3fb7f8e00 2024-09-10T08:39:32.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.485+0000 7f67fffe9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e3fb75a070 con 0x55e3fb7f84e0 2024-09-10T08:39:32.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.486+0000 7f67f6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb7f8e00 0x55e3fb803180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:32.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.486+0000 7f67f5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb8036c0 0x55e3fb805ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:32.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.486+0000 7f67f7200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f84e0 0x55e3fb7f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:32.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.486+0000 7f67f5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb8036c0 0x55e3fb805ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51048/0 (socket says 172.21.15.89:51048) 2024-09-10T08:39:32.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.486+0000 7f67f6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb7f8e00 0x55e3fb803180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:49684/0 (socket says 172.21.15.89:49684) 2024-09-10T08:39:32.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.486+0000 7f67f6800640 1 -- 172.21.15.89:0/2085355068 learned_addr learned my addr 172.21.15.89:0/2085355068 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:32.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.486+0000 7f67f7200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f84e0 0x55e3fb7f88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:56786/0 (socket says 172.21.15.89:56786) 2024-09-10T08:39:32.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.486+0000 7f67f7200640 1 -- 172.21.15.89:0/2085355068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb7f8e00 msgr2=0x55e3fb803180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:32.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.486+0000 7f67f7200640 1 --2- 172.21.15.89:0/2085355068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb7f8e00 0x55e3fb803180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.486+0000 7f67f7200640 1 -- 172.21.15.89:0/2085355068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb8036c0 msgr2=0x55e3fb805ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:32.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.486+0000 7f67f7200640 1 --2- 172.21.15.89:0/2085355068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb8036c0 0x55e3fb805ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.486+0000 7f67f7200640 1 -- 172.21.15.89:0/2085355068 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3fb7124a0 con 0x55e3fb7f84e0 2024-09-10T08:39:32.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.486+0000 7f67f7200640 1 --2- 172.21.15.89:0/2085355068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f84e0 0x55e3fb7f88c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f67f0005e30 tx=0x7f67f00093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=193c4811a44e66be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:32.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.487+0000 7f67e3400640 1 -- 172.21.15.89:0/2085355068 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f0031da0 con 0x55e3fb7f84e0 2024-09-10T08:39:32.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.487+0000 7f67e3400640 1 -- 172.21.15.89:0/2085355068 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f0036030 con 0x55e3fb7f84e0 2024-09-10T08:39:32.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.487+0000 7f67e3400640 1 -- 172.21.15.89:0/2085355068 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f00042a0 con 0x55e3fb7f84e0 2024-09-10T08:39:32.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.487+0000 7f67ef400640 1 --2- 172.21.15.89:0/4244222382 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f67dc0a2b80 0x7f67dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:32.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.487+0000 7f67f6800640 1 --2- 172.21.15.89:0/4244222382 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f67dc0a2b80 0x7f67dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:32.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.487+0000 7f67ef400640 1 -- 172.21.15.89:0/4244222382 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f67e80cf7b0 con 0x7f67f81019d0 2024-09-10T08:39:32.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.487+0000 7f67fffe9740 1 -- 172.21.15.89:0/2085355068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f84e0 msgr2=0x55e3fb7f88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:32.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.487+0000 7f67fffe9740 1 --2- 172.21.15.89:0/2085355068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f84e0 0x55e3fb7f88c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f67f0005e30 tx=0x7f67f00093e0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.489+0000 7f67fffe9740 1 -- 172.21.15.89:0/2085355068 shutdown_connections 2024-09-10T08:39:32.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.489+0000 7f67fffe9740 1 --2- 172.21.15.89:0/2085355068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb8036c0 0x55e3fb805ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.489+0000 7f67fffe9740 1 --2- 172.21.15.89:0/2085355068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb7f8e00 0x55e3fb803180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.489+0000 7f67fffe9740 1 --2- 172.21.15.89:0/2085355068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f84e0 0x55e3fb7f88c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.489+0000 7f67fffe9740 1 -- 172.21.15.89:0/2085355068 >> 172.21.15.89:0/2085355068 conn(0x55e3fb7f4a70 msgr2=0x55e3fb7f4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:32.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.489+0000 7f67fffe9740 1 -- 172.21.15.89:0/2085355068 shutdown_connections 2024-09-10T08:39:32.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.489+0000 7f67fffe9740 1 -- 172.21.15.89:0/2085355068 wait complete. 2024-09-10T08:39:32.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.489+0000 7f67f6800640 1 --2- 172.21.15.89:0/4244222382 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f67dc0a2b80 0x7f67dc0a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f67e80a4320 tx=0x7f67e80d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:32.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.489+0000 7f67fffe9740 1 -- start start 2024-09-10T08:39:32.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.489+0000 7f67fffe9740 1 Processor -- start 2024-09-10T08:39:32.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.489+0000 7f67fffe9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb7f84e0 0x55e3fb800580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:32.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.489+0000 7f67fffe9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f8e00 0x55e3fb800ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:32.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.489+0000 7f67fffe9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb8036c0 0x55e3fb801000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:32.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.489+0000 7f67fffe9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e3fb8061b0 con 0x55e3fb7f84e0 2024-09-10T08:39:32.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.489+0000 7f67fffe9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e3fb6e6920 con 0x55e3fb8036c0 2024-09-10T08:39:32.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.489+0000 7f67fffe9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e3fb806330 con 0x55e3fb7f8e00 2024-09-10T08:39:32.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.489+0000 7f67fffe9740 10 client.0 _ll_register_callbacks cb 0x55e3fb70a8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:39:32.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.490+0000 7f67f7200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb7f84e0 0x55e3fb800580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:32.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.490+0000 7f67f7200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb7f84e0 0x55e3fb800580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51058/0 (socket says 172.21.15.89:51058) 2024-09-10T08:39:32.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.490+0000 7f67f7200640 1 -- 172.21.15.89:0/1627681309 learned_addr learned my addr 172.21.15.89:0/1627681309 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:32.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.490+0000 7f67f5e00640 1 --2- 172.21.15.89:0/1627681309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f8e00 0x55e3fb800ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:32.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.490+0000 7f67f7200640 1 --2- 172.21.15.89:0/1627681309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb8036c0 0x55e3fb801000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:32.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.490+0000 7f67f7200640 1 -- 172.21.15.89:0/1627681309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f8e00 msgr2=0x55e3fb800ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:32.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.490+0000 7f67f7200640 1 --2- 172.21.15.89:0/1627681309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f8e00 0x55e3fb800ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.490+0000 7f67f7200640 1 -- 172.21.15.89:0/1627681309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb8036c0 msgr2=0x55e3fb801000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:32.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.490+0000 7f67f7200640 1 --2- 172.21.15.89:0/1627681309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb8036c0 0x55e3fb801000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.490+0000 7f67f7200640 1 -- 172.21.15.89:0/1627681309 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3fb4cf2b0 con 0x55e3fb7f84e0 2024-09-10T08:39:32.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.490+0000 7f67f7200640 1 --2- 172.21.15.89:0/1627681309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb7f84e0 0x55e3fb800580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f67f0005cf0 tx=0x7f67f002f4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86d99270510985a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:32.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.490+0000 7f67d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:39:32.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.491+0000 7f67d7e00640 1 -- 172.21.15.89:0/1627681309 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f0058060 con 0x55e3fb7f84e0 2024-09-10T08:39:32.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.491+0000 7f67d7e00640 1 -- 172.21.15.89:0/1627681309 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f67f0003d80 con 0x55e3fb7f84e0 2024-09-10T08:39:32.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.491+0000 7f67fffe9740 1 -- 172.21.15.89:0/1627681309 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67f0036030 con 0x55e3fb7f84e0 2024-09-10T08:39:32.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.491+0000 7f67d7e00640 1 -- 172.21.15.89:0/1627681309 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f67f005f050 con 0x55e3fb7f84e0 2024-09-10T08:39:32.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.491+0000 7f67fffe9740 10 client.15090 fetch_fsmap learned FSMap version 25 2024-09-10T08:39:32.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.491+0000 7f67fffe9740 1 -- 172.21.15.89:0/1627681309 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e3fb8061b0 con 0x55e3fb7f84e0 2024-09-10T08:39:32.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.491+0000 7f67d7e00640 1 -- 172.21.15.89:0/1627681309 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f0036030 con 0x55e3fb7f84e0 2024-09-10T08:39:32.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.492+0000 7f67d7e00640 1 -- 172.21.15.89:0/1627681309 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f67f0043070 con 0x55e3fb7f84e0 2024-09-10T08:39:32.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.492+0000 7f67fffe9740 10 client.15090 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:39:32.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.492+0000 7f67fffe9740 10 client.15090 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:39:32.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.492+0000 7f67fffe9740 20 client.15090 populate_metadata read hostname 'smithi089' 2024-09-10T08:39:32.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.492+0000 7f67fffe9740 1 --2- 172.21.15.89:0/1627681309 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e3fb814190 0x55e3fb815600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:32.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.492+0000 7f67fffe9740 4 client.15090 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:39:32.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.492+0000 7f67fffe9740 1 -- 172.21.15.89:0/1627681309 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3fb76b620 con 0x55e3fb814190 2024-09-10T08:39:32.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.492+0000 7f67f5e00640 1 --2- 172.21.15.89:0/1627681309 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e3fb814190 0x55e3fb815600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:32.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.493+0000 7f67f5e00640 1 --2- 172.21.15.89:0/1627681309 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e3fb814190 0x55e3fb815600 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79907b9013002c90 server_cookie=87b55d09090ca1f in_seq=0 out_seq=0 2024-09-10T08:39:32.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.493+0000 7f67d7e00640 10 client.15090 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:39:32.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.494+0000 7f67d7e00640 1 -- 172.21.15.89:0/1627681309 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3fb76b620 con 0x55e3fb814190 2024-09-10T08:39:32.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.494+0000 7f67d7e00640 10 client.15090 handle_command_reply: tid=0 2024-09-10T08:39:32.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.494+0000 7f67fffe9740 1 client.15090 shutdown 2024-09-10T08:39:32.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.494+0000 7f67fffe9740 1 -- 172.21.15.89:0/1627681309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb7f84e0 msgr2=0x55e3fb800580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:32.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.494+0000 7f67fffe9740 1 --2- 172.21.15.89:0/1627681309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb7f84e0 0x55e3fb800580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f67f0005cf0 tx=0x7f67f002f4d0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.494+0000 7f67fffe9740 1 -- 172.21.15.89:0/1627681309 shutdown_connections 2024-09-10T08:39:32.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.494+0000 7f67fffe9740 1 --2- 172.21.15.89:0/1627681309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb8036c0 0x55e3fb801000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.494+0000 7f67fffe9740 1 --2- 172.21.15.89:0/1627681309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f8e00 0x55e3fb800ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.494+0000 7f67fffe9740 1 --2- 172.21.15.89:0/1627681309 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e3fb814190 0x55e3fb815600 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.494+0000 7f67fffe9740 1 --2- 172.21.15.89:0/1627681309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb7f84e0 0x55e3fb800580 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.494+0000 7f67fffe9740 1 -- 172.21.15.89:0/1627681309 >> 172.21.15.89:0/1627681309 conn(0x55e3fb7f4a70 msgr2=0x55e3fb7fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:32.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.494+0000 7f67d7e00640 0 client.15090 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:39:32.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.494+0000 7f67fffe9740 1 -- 172.21.15.89:0/1627681309 shutdown_connections 2024-09-10T08:39:32.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.494+0000 7f67fffe9740 1 -- 172.21.15.89:0/1627681309 wait complete. 2024-09-10T08:39:32.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.494+0000 7f67fffe9740 20 client.15090 trim_cache size 0 max 16384 2024-09-10T08:39:32.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.513+0000 7f67fffe9740 1 Processor -- start 2024-09-10T08:39:32.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.513+0000 7f67fffe9740 1 -- start start 2024-09-10T08:39:32.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.513+0000 7f67fffe9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb7f84e0 0x55e3fb814f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:32.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.513+0000 7f67fffe9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f8e00 0x55e3fb7fee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:32.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.513+0000 7f67fffe9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb8036c0 0x55e3fb7ff390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:32.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.513+0000 7f67fffe9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e3fb813940 con 0x55e3fb7f84e0 2024-09-10T08:39:32.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.513+0000 7f67fffe9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e3fb814010 con 0x55e3fb8036c0 2024-09-10T08:39:32.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.513+0000 7f67fffe9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e3fb8146e0 con 0x55e3fb7f8e00 2024-09-10T08:39:32.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.513+0000 7f67f7200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb7f84e0 0x55e3fb814f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:32.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.513+0000 7f67f7200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb7f84e0 0x55e3fb814f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51070/0 (socket says 172.21.15.89:51070) 2024-09-10T08:39:32.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.513+0000 7f67f7200640 1 -- 172.21.15.89:0/2924282641 learned_addr learned my addr 172.21.15.89:0/2924282641 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:32.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.513+0000 7f67f5e00640 1 --2- 172.21.15.89:0/2924282641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f8e00 0x55e3fb7fee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:32.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.514+0000 7f67f7200640 1 --2- 172.21.15.89:0/2924282641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb8036c0 0x55e3fb7ff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:32.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.514+0000 7f67f5e00640 1 -- 172.21.15.89:0/2924282641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb8036c0 msgr2=0x55e3fb7ff390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:32.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.514+0000 7f67f5e00640 1 --2- 172.21.15.89:0/2924282641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb8036c0 0x55e3fb7ff390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.514+0000 7f67f5e00640 1 -- 172.21.15.89:0/2924282641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb7f84e0 msgr2=0x55e3fb814f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:32.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.514+0000 7f67f5e00640 1 --2- 172.21.15.89:0/2924282641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb7f84e0 0x55e3fb814f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.514+0000 7f67f5e00640 1 -- 172.21.15.89:0/2924282641 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3fb7f8310 con 0x55e3fb7f8e00 2024-09-10T08:39:32.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.514+0000 7f67f7200640 1 --2- 172.21.15.89:0/2924282641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb7f84e0 0x55e3fb814f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:32.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.514+0000 7f67f5e00640 1 --2- 172.21.15.89:0/2924282641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f8e00 0x55e3fb7fee50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55e3fb801bc0 tx=0x7f67e4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48286907896e8348 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:32.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.514+0000 7f67e2a00640 1 -- 172.21.15.89:0/2924282641 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4003d20 con 0x55e3fb7f8e00 2024-09-10T08:39:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.515+0000 7f67e2a00640 1 -- 172.21.15.89:0/2924282641 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e4010910 con 0x55e3fb7f8e00 2024-09-10T08:39:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.515+0000 7f67e2a00640 1 -- 172.21.15.89:0/2924282641 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e401f890 con 0x55e3fb7f8e00 2024-09-10T08:39:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.515+0000 7f67fffe9740 1 -- 172.21.15.89:0/2924282641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f8e00 msgr2=0x55e3fb7fee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.515+0000 7f67fffe9740 1 --2- 172.21.15.89:0/2924282641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f8e00 0x55e3fb7fee50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55e3fb801bc0 tx=0x7f67e4005f90 comp rx=0 tx=0).stop 2024-09-10T08:39:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.515+0000 7f67fffe9740 1 -- 172.21.15.89:0/2924282641 shutdown_connections 2024-09-10T08:39:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.515+0000 7f67fffe9740 1 --2- 172.21.15.89:0/2924282641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb8036c0 0x55e3fb7ff390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.515+0000 7f67fffe9740 1 --2- 172.21.15.89:0/2924282641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f8e00 0x55e3fb7fee50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.515+0000 7f67fffe9740 1 --2- 172.21.15.89:0/2924282641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb7f84e0 0x55e3fb814f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.515+0000 7f67fffe9740 1 -- 172.21.15.89:0/2924282641 >> 172.21.15.89:0/2924282641 conn(0x55e3fb7f4a70 msgr2=0x55e3fb7fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.516+0000 7f67fffe9740 1 -- 172.21.15.89:0/2924282641 shutdown_connections 2024-09-10T08:39:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.516+0000 7f67fffe9740 1 -- 172.21.15.89:0/2924282641 wait complete. 2024-09-10T08:39:32.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.516+0000 7f67fffe9740 1 -- start start 2024-09-10T08:39:32.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.516+0000 7f67fffe9740 1 Processor -- start 2024-09-10T08:39:32.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.517+0000 7f67fffe9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f84e0 0x55e3fb7fcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:32.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.517+0000 7f67fffe9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb7f8e00 0x55e3fb7fd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:32.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.517+0000 7f67fffe9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb8036c0 0x55e3fb81c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:32.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.517+0000 7f67fffe9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e3fb815480 con 0x55e3fb8036c0 2024-09-10T08:39:32.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.517+0000 7f67fffe9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e3fb814db0 con 0x55e3fb7f8e00 2024-09-10T08:39:32.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.517+0000 7f67fffe9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e3fb815b50 con 0x55e3fb7f84e0 2024-09-10T08:39:32.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.517+0000 7f67fffe9740 10 client.0 _ll_register_callbacks cb 0x7f67f0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:39:32.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.517+0000 7f67f5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb7f8e00 0x55e3fb7fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:32.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.517+0000 7f67f7200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb8036c0 0x55e3fb81c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:32.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.517+0000 7f67f5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb7f8e00 0x55e3fb7fd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:49710/0 (socket says 172.21.15.89:49710) 2024-09-10T08:39:32.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.517+0000 7f67f7200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb8036c0 0x55e3fb81c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51078/0 (socket says 172.21.15.89:51078) 2024-09-10T08:39:32.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.517+0000 7f67f5e00640 1 -- 172.21.15.89:0/1783461851 learned_addr learned my addr 172.21.15.89:0/1783461851 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:32.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.517+0000 7f67f7200640 1 --2- 172.21.15.89:0/1783461851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f84e0 0x55e3fb7fcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:32.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.517+0000 7f67f7200640 1 -- 172.21.15.89:0/1783461851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f84e0 msgr2=0x55e3fb7fcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:32.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.518+0000 7f67f7200640 1 --2- 172.21.15.89:0/1783461851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f84e0 0x55e3fb7fcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.518+0000 7f67f7200640 1 -- 172.21.15.89:0/1783461851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb7f8e00 msgr2=0x55e3fb7fd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:32.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.518+0000 7f67f7200640 1 --2- 172.21.15.89:0/1783461851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb7f8e00 0x55e3fb7fd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.518+0000 7f67f7200640 1 -- 172.21.15.89:0/1783461851 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3fb81cf40 con 0x55e3fb8036c0 2024-09-10T08:39:32.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.518+0000 7f67f7200640 1 --2- 172.21.15.89:0/1783461851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb8036c0 0x55e3fb81c850 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f67f002fcf0 tx=0x7f67f0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4847be659b2c4a94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:32.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.518+0000 7f67d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:39:32.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.518+0000 7f67d7e00640 1 -- 172.21.15.89:0/1783461851 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f0058030 con 0x55e3fb8036c0 2024-09-10T08:39:32.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.518+0000 7f67d7e00640 1 -- 172.21.15.89:0/1783461851 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f67f00305c0 con 0x55e3fb8036c0 2024-09-10T08:39:32.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.518+0000 7f67fffe9740 1 -- 172.21.15.89:0/1783461851 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67e4010910 con 0x55e3fb8036c0 2024-09-10T08:39:32.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.518+0000 7f67d7e00640 1 -- 172.21.15.89:0/1783461851 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f67f0009880 con 0x55e3fb8036c0 2024-09-10T08:39:32.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.518+0000 7f67fffe9740 10 client.15096 fetch_fsmap learned FSMap version 25 2024-09-10T08:39:32.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.518+0000 7f67fffe9740 1 -- 172.21.15.89:0/1783461851 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e3fb81e220 con 0x55e3fb8036c0 2024-09-10T08:39:32.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.519+0000 7f67d7e00640 1 -- 172.21.15.89:0/1783461851 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e4010910 con 0x55e3fb8036c0 2024-09-10T08:39:32.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.519+0000 7f67d7e00640 1 -- 172.21.15.89:0/1783461851 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f67f0043050 con 0x55e3fb8036c0 2024-09-10T08:39:32.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.519+0000 7f67fffe9740 10 client.15096 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T08:39:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.519+0000 7f67fffe9740 10 client.15096 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:39:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.519+0000 7f67fffe9740 20 client.15096 populate_metadata read hostname 'smithi089' 2024-09-10T08:39:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.519+0000 7f67fffe9740 1 --2- 172.21.15.89:0/1783461851 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e3fb882e90 0x55e3fb881ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.519+0000 7f67fffe9740 4 client.15096 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:39:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.519+0000 7f67fffe9740 1 -- 172.21.15.89:0/1783461851 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "status"}) -- 0x55e3fb765b30 con 0x55e3fb882e90 2024-09-10T08:39:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.520+0000 7f67f5e00640 1 --2- 172.21.15.89:0/1783461851 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e3fb882e90 0x55e3fb881ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.520+0000 7f67f5e00640 1 --2- 172.21.15.89:0/1783461851 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e3fb882e90 0x55e3fb881ba0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5909da9dc7b96d9a server_cookie=25257ca4eb9f2019 in_seq=0 out_seq=0 2024-09-10T08:39:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.520+0000 7f67d7e00640 10 client.15096 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:39:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.520+0000 7f67d7e00640 1 -- 172.21.15.89:0/1783461851 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55e3fb765b30 con 0x55e3fb882e90 2024-09-10T08:39:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.520+0000 7f67d7e00640 10 client.15096 handle_command_reply: tid=0 2024-09-10T08:39:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.520+0000 7f67fffe9740 1 client.15096 shutdown 2024-09-10T08:39:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.520+0000 7f67fffe9740 1 -- 172.21.15.89:0/1783461851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb8036c0 msgr2=0x55e3fb81c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:32.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.520+0000 7f67fffe9740 1 --2- 172.21.15.89:0/1783461851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb8036c0 0x55e3fb81c850 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f67f002fcf0 tx=0x7f67f0000f40 comp rx=0 tx=0).stop 2024-09-10T08:39:32.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.521+0000 7f67fffe9740 1 -- 172.21.15.89:0/1783461851 shutdown_connections 2024-09-10T08:39:32.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.521+0000 7f67fffe9740 1 --2- 172.21.15.89:0/1783461851 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e3fb882e90 0x55e3fb881ba0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.521+0000 7f67fffe9740 1 --2- 172.21.15.89:0/1783461851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e3fb8036c0 0x55e3fb81c850 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.521+0000 7f67d7e00640 0 client.15096 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:39:32.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.521+0000 7f67fffe9740 1 --2- 172.21.15.89:0/1783461851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3fb7f8e00 0x55e3fb7fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.521+0000 7f67fffe9740 1 --2- 172.21.15.89:0/1783461851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3fb7f84e0 0x55e3fb7fcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.521+0000 7f67fffe9740 1 -- 172.21.15.89:0/1783461851 >> 172.21.15.89:0/1783461851 conn(0x55e3fb7f4a70 msgr2=0x55e3fb80e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:32.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.521+0000 7f67fffe9740 1 -- 172.21.15.89:0/1783461851 shutdown_connections 2024-09-10T08:39:32.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.521+0000 7f67fffe9740 1 -- 172.21.15.89:0/1783461851 wait complete. 2024-09-10T08:39:32.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.521+0000 7f67fffe9740 20 client.15096 trim_cache size 0 max 16384 2024-09-10T08:39:32.525 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:39:32.525 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "91f747bc-6f4e-11ef-bcea-c7b262605968", 2024-09-10T08:39:32.525 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2024-09-10T08:39:32.525 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24554, 2024-09-10T08:39:32.525 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2024-09-10T08:39:32.525 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2024-09-10T08:39:32.525 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2024-09-10T08:39:32.525 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 154.373788808, 2024-09-10T08:39:32.526 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 25, 2024-09-10T08:39:32.526 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 96, 2024-09-10T08:39:32.526 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T08:39:32.526 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 167.387251338 2024-09-10T08:39:32.526 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:39:32.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.522+0000 7f67fca00640 1 -- 172.21.15.89:0/4244222382 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f67dc0a2b80 msgr2=0x7f67dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:32.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.522+0000 7f67fca00640 1 --2- 172.21.15.89:0/4244222382 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f67dc0a2b80 0x7f67dc0a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f67e80a4320 tx=0x7f67e80d5530 comp rx=0 tx=0).stop 2024-09-10T08:39:32.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.522+0000 7f67fca00640 1 -- 172.21.15.89:0/4244222382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67f81019d0 msgr2=0x7f67f80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:32.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.522+0000 7f67fca00640 1 --2- 172.21.15.89:0/4244222382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67f81019d0 0x7f67f80744f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f67e800e9f0 tx=0x7f67e800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.522+0000 7f67fca00640 1 -- 172.21.15.89:0/4244222382 shutdown_connections 2024-09-10T08:39:32.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.522+0000 7f67fca00640 1 --2- 172.21.15.89:0/4244222382 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f67dc0a2b80 0x7f67dc0a5040 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.522+0000 7f67fca00640 1 --2- 172.21.15.89:0/4244222382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67f81079d0 0x7f67f8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.522+0000 7f67fca00640 1 --2- 172.21.15.89:0/4244222382 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f8102370 0x7f67f8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.522+0000 7f67fca00640 1 --2- 172.21.15.89:0/4244222382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67f81019d0 0x7f67f80744f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:32.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.522+0000 7f67fca00640 1 -- 172.21.15.89:0/4244222382 >> 172.21.15.89:0/4244222382 conn(0x7f67f80fd710 msgr2=0x7f67f80fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:32.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.522+0000 7f67fca00640 1 -- 172.21.15.89:0/4244222382 shutdown_connections 2024-09-10T08:39:32.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:32.522+0000 7f67fca00640 1 -- 172.21.15.89:0/4244222382 wait complete. 2024-09-10T08:39:32.877 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:32 smithi089 ceph-mon[30757]: pgmap v381: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T08:39:32.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:32 smithi149 ceph-mon[36167]: pgmap v381: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T08:39:33.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:32 smithi111 ceph-mon[35220]: pgmap v381: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T08:39:33.391 INFO:teuthology.run_tasks:Running task sequential... 2024-09-10T08:39:33.402 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-10T08:39:33.402 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-10T08:39:33.402 INFO:teuthology.task.print:Enabling mgr modules 2024-09-10T08:39:33.402 INFO:teuthology.run_tasks:Running task kclient... 2024-09-10T08:39:33.460 INFO:tasks.kclient:Mounting kernel clients... 2024-09-10T08:39:33.475 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-10T08:39:33.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:39:33.476 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2024-09-10T08:39:33.512 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-10T08:39:33.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:39:33.513 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:39:33.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:39:33.613 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link delete ceph-brx 2024-09-10T08:39:33.919 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-10T08:39:33.919 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T08:39:33.919 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T08:39:33.920 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T08:39:34.287 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:34.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.862+0000 7f0b4ae00640 1 -- 172.21.15.89:0/2941859754 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b44109140 msgr2=0x7f0b44110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:34.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.862+0000 7f0b4ae00640 1 --2- 172.21.15.89:0/2941859754 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b44109140 0x7f0b44110cc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c009ca0 tx=0x7f0b3c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:39:34.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.862+0000 7f0b4ae00640 1 -- 172.21.15.89:0/2941859754 shutdown_connections 2024-09-10T08:39:34.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.862+0000 7f0b4ae00640 1 --2- 172.21.15.89:0/2941859754 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b44109140 0x7f0b44110cc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:34.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.862+0000 7f0b4ae00640 1 --2- 172.21.15.89:0/2941859754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b441049c0 0x7f0b44108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:34.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.862+0000 7f0b4ae00640 1 --2- 172.21.15.89:0/2941859754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b44104010 0x7f0b441043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:34.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.862+0000 7f0b4ae00640 1 -- 172.21.15.89:0/2941859754 >> 172.21.15.89:0/2941859754 conn(0x7f0b440ffa50 msgr2=0x7f0b44101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:34.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.862+0000 7f0b4ae00640 1 -- 172.21.15.89:0/2941859754 shutdown_connections 2024-09-10T08:39:34.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.862+0000 7f0b4ae00640 1 -- 172.21.15.89:0/2941859754 wait complete. 2024-09-10T08:39:34.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.863+0000 7f0b4ae00640 1 Processor -- start 2024-09-10T08:39:34.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.863+0000 7f0b4ae00640 1 -- start start 2024-09-10T08:39:34.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.864+0000 7f0b4ae00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b44104010 0x7f0b4419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:34.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.864+0000 7f0b4ae00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b441049c0 0x7f0b4419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:34.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.864+0000 7f0b4ae00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b44109140 0x7f0b4419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:34.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.864+0000 7f0b4ae00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0b44114170 con 0x7f0b44104010 2024-09-10T08:39:34.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.864+0000 7f0b4ae00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0b44113ff0 con 0x7f0b441049c0 2024-09-10T08:39:34.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.864+0000 7f0b4ae00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0b441142f0 con 0x7f0b44109140 2024-09-10T08:39:34.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.864+0000 7f0b43e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b44104010 0x7f0b4419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:34.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.864+0000 7f0b43400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b441049c0 0x7f0b4419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:34.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.864+0000 7f0b43400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b441049c0 0x7f0b4419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:52778/0 (socket says 172.21.15.89:52778) 2024-09-10T08:39:34.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.864+0000 7f0b43400640 1 -- 172.21.15.89:0/2114270531 learned_addr learned my addr 172.21.15.89:0/2114270531 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:34.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.864+0000 7f0b48e00640 1 --2- 172.21.15.89:0/2114270531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b44109140 0x7f0b4419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:34.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.864+0000 7f0b43400640 1 -- 172.21.15.89:0/2114270531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b44109140 msgr2=0x7f0b4419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:34.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.865+0000 7f0b43400640 1 --2- 172.21.15.89:0/2114270531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b44109140 0x7f0b4419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:34.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.865+0000 7f0b43400640 1 -- 172.21.15.89:0/2114270531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b44104010 msgr2=0x7f0b4419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:34.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.865+0000 7f0b43400640 1 --2- 172.21.15.89:0/2114270531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b44104010 0x7f0b4419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:34.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.865+0000 7f0b43400640 1 -- 172.21.15.89:0/2114270531 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4419f5e0 con 0x7f0b441049c0 2024-09-10T08:39:34.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.865+0000 7f0b48e00640 1 --2- 172.21.15.89:0/2114270531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b44109140 0x7f0b4419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:39:34.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.865+0000 7f0b43400640 1 --2- 172.21.15.89:0/2114270531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b441049c0 0x7f0b4419ab70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000d770 tx=0x7f0b3000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:34.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.865+0000 7f0b40c00640 1 -- 172.21.15.89:0/2114270531 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30017070 con 0x7f0b441049c0 2024-09-10T08:39:34.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.865+0000 7f0b4ae00640 1 -- 172.21.15.89:0/2114270531 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b4419f8d0 con 0x7f0b441049c0 2024-09-10T08:39:34.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.865+0000 7f0b40c00640 1 -- 172.21.15.89:0/2114270531 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b30005500 con 0x7f0b441049c0 2024-09-10T08:39:34.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.865+0000 7f0b40c00640 1 -- 172.21.15.89:0/2114270531 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30004e00 con 0x7f0b441049c0 2024-09-10T08:39:34.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.865+0000 7f0b4ae00640 1 -- 172.21.15.89:0/2114270531 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b441a71b0 con 0x7f0b441049c0 2024-09-10T08:39:34.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.866+0000 7f0b4ae00640 1 -- 172.21.15.89:0/2114270531 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b441043f0 con 0x7f0b441049c0 2024-09-10T08:39:34.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.870+0000 7f0b40c00640 1 -- 172.21.15.89:0/2114270531 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0b30004fa0 con 0x7f0b441049c0 2024-09-10T08:39:34.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.870+0000 7f0b40c00640 1 --2- 172.21.15.89:0/2114270531 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0b280a2b80 0x7f0b280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:34.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.870+0000 7f0b40c00640 1 -- 172.21.15.89:0/2114270531 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f0b300cee90 con 0x7f0b441049c0 2024-09-10T08:39:34.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.870+0000 7f0b43e00640 1 --2- 172.21.15.89:0/2114270531 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0b280a2b80 0x7f0b280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:34.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.871+0000 7f0b43e00640 1 --2- 172.21.15.89:0/2114270531 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0b280a2b80 0x7f0b280a5040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009bd0 tx=0x7f0b34005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:34.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:34.876+0000 7f0b40c00640 1 -- 172.21.15.89:0/2114270531 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b3009b9c0 con 0x7f0b441049c0 2024-09-10T08:39:34.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:34 smithi149 ceph-mon[36167]: pgmap v382: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:34 smithi111 ceph-mon[35220]: pgmap v382: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:35.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:34 smithi089 ceph-mon[30757]: pgmap v382: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:35.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:35.085+0000 7f0b4ae00640 1 -- 172.21.15.89:0/2114270531 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0b4410bdf0 con 0x7f0b441049c0 2024-09-10T08:39:35.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:35.087+0000 7f0b40c00640 1 -- 172.21.15.89:0/2114270531 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+88 (secure 0 0 0) 0x7f0b3009b2c0 con 0x7f0b441049c0 2024-09-10T08:39:35.086 INFO:teuthology.orchestra.run.smithi089.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T08:39:35.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:35.089+0000 7f0b4ae00640 1 -- 172.21.15.89:0/2114270531 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0b280a2b80 msgr2=0x7f0b280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:35.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:35.089+0000 7f0b4ae00640 1 --2- 172.21.15.89:0/2114270531 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0b280a2b80 0x7f0b280a5040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009bd0 tx=0x7f0b34005f90 comp rx=0 tx=0).stop 2024-09-10T08:39:35.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:35.089+0000 7f0b4ae00640 1 -- 172.21.15.89:0/2114270531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b441049c0 msgr2=0x7f0b4419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:35.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:35.089+0000 7f0b4ae00640 1 --2- 172.21.15.89:0/2114270531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b441049c0 0x7f0b4419ab70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000d770 tx=0x7f0b3000dc30 comp rx=0 tx=0).stop 2024-09-10T08:39:35.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:35.089+0000 7f0b4ae00640 1 -- 172.21.15.89:0/2114270531 shutdown_connections 2024-09-10T08:39:35.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:35.089+0000 7f0b4ae00640 1 --2- 172.21.15.89:0/2114270531 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0b280a2b80 0x7f0b280a5040 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:35.089+0000 7f0b4ae00640 1 --2- 172.21.15.89:0/2114270531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b44109140 0x7f0b4419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:35.089+0000 7f0b4ae00640 1 --2- 172.21.15.89:0/2114270531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b441049c0 0x7f0b4419ab70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:35.089+0000 7f0b4ae00640 1 --2- 172.21.15.89:0/2114270531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b44104010 0x7f0b4419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:35.089+0000 7f0b4ae00640 1 -- 172.21.15.89:0/2114270531 >> 172.21.15.89:0/2114270531 conn(0x7f0b440ffa50 msgr2=0x7f0b44100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:35.089+0000 7f0b4ae00640 1 -- 172.21.15.89:0/2114270531 shutdown_connections 2024-09-10T08:39:35.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:35.089+0000 7f0b4ae00640 1 -- 172.21.15.89:0/2114270531 wait complete. 2024-09-10T08:39:35.792 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T08:39:35.792 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T08:39:35.792 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi089.front.sepia.ceph.com 2024-09-10T08:39:35.792 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T08:39:35.792 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T08:39:35.792 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T08:39:35.792 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T08:39:35.792 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T08:39:35.792 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T08:39:35.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:39:35.793 DEBUG:teuthology.orchestra.run.smithi089:> ip addr 2024-09-10T08:39:35.828 INFO:teuthology.orchestra.run.smithi089.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T08:39:35.828 INFO:teuthology.orchestra.run.smithi089.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T08:39:35.828 INFO:teuthology.orchestra.run.smithi089.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T08:39:35.828 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2024-09-10T08:39:35.828 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 ::1/128 scope host 2024-09-10T08:39:35.828 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2024-09-10T08:39:35.828 INFO:teuthology.orchestra.run.smithi089.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T08:39:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c6 brd ff:ff:ff:ff:ff:ff 2024-09-10T08:39:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f0 2024-09-10T08:39:35.829 INFO:teuthology.orchestra.run.smithi089.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T08:39:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:d8:a4:c7 brd ff:ff:ff:ff:ff:ff 2024-09-10T08:39:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: altname enp7s0f1 2024-09-10T08:39:35.829 INFO:teuthology.orchestra.run.smithi089.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T08:39:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:00 brd ff:ff:ff:ff:ff:ff 2024-09-10T08:39:35.829 INFO:teuthology.orchestra.run.smithi089.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T08:39:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: link/ether 0c:c4:7a:8f:d5:01 brd ff:ff:ff:ff:ff:ff 2024-09-10T08:39:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: inet 172.21.15.89/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T08:39:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft 42092sec preferred_lft 42092sec 2024-09-10T08:39:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: inet6 fe80::ec4:7aff:fe8f:d501/64 scope link noprefixroute 2024-09-10T08:39:35.829 INFO:teuthology.orchestra.run.smithi089.stdout: valid_lft forever preferred_lft forever 2024-09-10T08:39:35.830 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T08:39:35.830 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:39:35.831 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2024-09-10T08:39:35.831 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add name ceph-brx type bridge 2024-09-10T08:39:35.831 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip addr flush dev ceph-brx 2024-09-10T08:39:35.831 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set ceph-brx up 2024-09-10T08:39:35.831 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T08:39:35.831 DEBUG:teuthology.orchestra.run.smithi089:> ') 2024-09-10T08:39:35.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:39:35.966 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:35 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2114270531' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T08:39:35.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:35 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2114270531' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T08:39:36.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:35 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2114270531' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T08:39:36.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:39:36.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:39:36.055 DEBUG:teuthology.orchestra.run.smithi089:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T08:39:36.109 INFO:teuthology.orchestra.run.smithi089.stdout:1 2024-09-10T08:39:36.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:39:36.113 DEBUG:teuthology.orchestra.run.smithi089:> route 2024-09-10T08:39:36.150 INFO:teuthology.orchestra.run.smithi089.stdout:Kernel IP routing table 2024-09-10T08:39:36.150 INFO:teuthology.orchestra.run.smithi089.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T08:39:36.150 INFO:teuthology.orchestra.run.smithi089.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T08:39:36.150 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T08:39:36.150 INFO:teuthology.orchestra.run.smithi089.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T08:39:36.151 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:39:36.151 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2024-09-10T08:39:36.151 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T08:39:36.151 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T08:39:36.151 DEBUG:teuthology.orchestra.run.smithi089:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T08:39:36.151 DEBUG:teuthology.orchestra.run.smithi089:> ') 2024-09-10T08:39:36.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:39:36.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:39:36.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:39:36.349 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2024-09-10T08:39:36.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:39:36.384 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list-id 2024-09-10T08:39:36.460 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:39:36.460 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2024-09-10T08:39:36.460 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T08:39:36.460 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T08:39:36.460 DEBUG:teuthology.orchestra.run.smithi089:> ') 2024-09-10T08:39:36.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:39:36.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:39:36.638 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T08:39:36.638 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:39:36.638 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2024-09-10T08:39:36.639 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T08:39:36.639 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T08:39:36.639 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T08:39:36.639 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T08:39:36.639 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T08:39:36.639 DEBUG:teuthology.orchestra.run.smithi089:> ') 2024-09-10T08:39:36.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:39:36.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:39:36.855 DEBUG:teuthology.orchestra.run.smithi089:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:39:36.855 DEBUG:teuthology.orchestra.run.smithi089:> set -e 2024-09-10T08:39:36.856 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set brx.0 up 2024-09-10T08:39:36.856 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T08:39:36.856 DEBUG:teuthology.orchestra.run.smithi089:> ') 2024-09-10T08:39:36.934 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:39:36.950 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:36 smithi089 ceph-mon[30757]: pgmap v383: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:39:36.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:36 smithi149 ceph-mon[36167]: pgmap v383: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:39:37.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:39:37.023 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T08:39:37.023 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T08:39:37.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:36 smithi111 ceph-mon[35220]: pgmap v383: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-10T08:39:37.058 INFO:teuthology.orchestra.run.smithi089.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T08:39:37.058 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T08:39:37.059 DEBUG:teuthology.orchestra.run.smithi089:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T08:39:37.133 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-10T08:39:37.133 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-10T08:39:37.133 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e 2024-09-10T08:39:37.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:39:37.133 DEBUG:teuthology.orchestra.run.smithi089:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-09-10T08:39:37.262 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-09-10T08:39:37.262 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-09-10T08:39:37.912 INFO:teuthology.orchestra.run.smithi089.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-09-10T08:39:37.912 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-09-10T08:39:37.912 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: resolved to: "172.21.15.89:6789,172.21.15.111:6789,172.21.15.149:6789" 2024-09-10T08:39:37.913 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: trying mount with new device syntax: 0@91f747bc-6f4e-11ef-bcea-c7b262605968.cephfs=/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e 2024-09-10T08:39:37.913 INFO:teuthology.orchestra.run.smithi089.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,name=0,key=0,mon_addr=172.21.15.89:6789/172.21.15.111:6789/172.21.15.149:6789" will pass to kernel 2024-09-10T08:39:37.914 INFO:teuthology.orchestra.run.smithi089.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T08:39:37.914 INFO:teuthology.orchestra.run.smithi089.stdout: You just mounted a file system that supports labels which does not 2024-09-10T08:39:37.914 INFO:teuthology.orchestra.run.smithi089.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T08:39:37.914 INFO:teuthology.orchestra.run.smithi089.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T08:39:37.914 INFO:teuthology.orchestra.run.smithi089.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T08:39:37.917 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T08:39:37.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:39:37.917 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T08:39:37.980 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T08:39:38.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T08:39:38.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:39:38.071 INFO:teuthology.orchestra.run.smithi089.stdout:client25157 2024-09-10T08:39:38.072 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:39:38.072 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T08:39:38.109 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T08:39:38.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T08:39:38.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:39:38.239 INFO:teuthology.orchestra.run.smithi089.stdout:client25157 2024-09-10T08:39:38.239 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T08:39:38.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T08:39:38.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:39:38.329 INFO:teuthology.orchestra.run.smithi089.stdout:client25157 2024-09-10T08:39:38.329 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968.client25157/status') 2024-09-10T08:39:38.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:39:38.428 INFO:teuthology.orchestra.run.smithi089.stdout:instance: client.25157 (3)172.21.15.89:0/1421961840 2024-09-10T08:39:38.428 INFO:teuthology.orchestra.run.smithi089.stdout:blocklisted: false 2024-09-10T08:39:38.428 INFO:teuthology.orchestra.run.smithi089.stderr:0+1 records in 2024-09-10T08:39:38.429 INFO:teuthology.orchestra.run.smithi089.stderr:0+1 records out 2024-09-10T08:39:38.429 INFO:teuthology.orchestra.run.smithi089.stderr:71 bytes copied, 8.2963e-05 s, 856 kB/s 2024-09-10T08:39:38.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:39:38.439 INFO:teuthology.run_tasks:Running task exec... 2024-09-10T08:39:38.450 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T08:39:38.450 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi089.front.sepia.ceph.com 2024-09-10T08:39:38.450 DEBUG:teuthology.orchestra.run.smithi089:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-10T08:39:38.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.603+0000 7f021a000640 1 -- 172.21.15.89:0/806964423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02141036f0 msgr2=0x7f0214109f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:38.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.603+0000 7f021a000640 1 --2- 172.21.15.89:0/806964423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02141036f0 0x7f0214109f80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f020c009ca0 tx=0x7f020c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:39:38.603 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.603+0000 7f021a000640 1 -- 172.21.15.89:0/806964423 shutdown_connections 2024-09-10T08:39:38.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.603+0000 7f021a000640 1 --2- 172.21.15.89:0/806964423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02141036f0 0x7f0214109f80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:38.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.603+0000 7f021a000640 1 --2- 172.21.15.89:0/806964423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0214102d30 0x7f02141031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:38.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.603+0000 7f021a000640 1 --2- 172.21.15.89:0/806964423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0214101b40 0x7f0214101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:38.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.603+0000 7f021a000640 1 -- 172.21.15.89:0/806964423 >> 172.21.15.89:0/806964423 conn(0x7f02140fd2e0 msgr2=0x7f02140ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:38.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.603+0000 7f021a000640 1 -- 172.21.15.89:0/806964423 shutdown_connections 2024-09-10T08:39:38.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.603+0000 7f021a000640 1 -- 172.21.15.89:0/806964423 wait complete. 2024-09-10T08:39:38.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.604+0000 7f021a000640 1 Processor -- start 2024-09-10T08:39:38.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.604+0000 7f021a000640 1 -- start start 2024-09-10T08:39:38.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.605+0000 7f021a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0214101b40 0x7f0214199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:38.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.605+0000 7f021a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0214102d30 0x7f021419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:38.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.605+0000 7f021a000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02141036f0 0x7f02141a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:38.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.605+0000 7f021a000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f021410d7c0 con 0x7f02141036f0 2024-09-10T08:39:38.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.605+0000 7f021a000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f021410d640 con 0x7f0214102d30 2024-09-10T08:39:38.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.605+0000 7f021a000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f021410d940 con 0x7f0214101b40 2024-09-10T08:39:38.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.605+0000 7f0212a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0214102d30 0x7f021419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:38.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.605+0000 7f0212a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0214102d30 0x7f021419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:52804/0 (socket says 172.21.15.89:52804) 2024-09-10T08:39:38.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.605+0000 7f0212a00640 1 -- 172.21.15.89:0/707882422 learned_addr learned my addr 172.21.15.89:0/707882422 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:38.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.605+0000 7f0213400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0214101b40 0x7f0214199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:38.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.605+0000 7f0213e00640 1 --2- 172.21.15.89:0/707882422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02141036f0 0x7f02141a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:38.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.605+0000 7f0212a00640 1 -- 172.21.15.89:0/707882422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0214101b40 msgr2=0x7f0214199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:38.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.605+0000 7f0212a00640 1 --2- 172.21.15.89:0/707882422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0214101b40 0x7f0214199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:38.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.605+0000 7f0212a00640 1 -- 172.21.15.89:0/707882422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02141036f0 msgr2=0x7f02141a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:38.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.605+0000 7f0212a00640 1 --2- 172.21.15.89:0/707882422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02141036f0 0x7f02141a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:38.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.605+0000 7f0212a00640 1 -- 172.21.15.89:0/707882422 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02141a1bc0 con 0x7f0214102d30 2024-09-10T08:39:38.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.606+0000 7f0212a00640 1 --2- 172.21.15.89:0/707882422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0214102d30 0x7f021419a4e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f020000e9f0 tx=0x7f020000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:38.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.606+0000 7f020be00640 1 -- 172.21.15.89:0/707882422 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020000cd80 con 0x7f0214102d30 2024-09-10T08:39:38.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.606+0000 7f020be00640 1 -- 172.21.15.89:0/707882422 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0200005500 con 0x7f0214102d30 2024-09-10T08:39:38.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.606+0000 7f020be00640 1 -- 172.21.15.89:0/707882422 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0200010690 con 0x7f0214102d30 2024-09-10T08:39:38.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.606+0000 7f021a000640 1 -- 172.21.15.89:0/707882422 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02141a1eb0 con 0x7f0214102d30 2024-09-10T08:39:38.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.606+0000 7f021a000640 1 -- 172.21.15.89:0/707882422 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02141a2320 con 0x7f0214102d30 2024-09-10T08:39:38.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.607+0000 7f021a000640 1 -- 172.21.15.89:0/707882422 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02141a25b0 con 0x7f0214102d30 2024-09-10T08:39:38.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.609+0000 7f020be00640 1 -- 172.21.15.89:0/707882422 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f02000046c0 con 0x7f0214102d30 2024-09-10T08:39:38.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.610+0000 7f020be00640 1 --2- 172.21.15.89:0/707882422 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f01f80a2c30 0x7f01f80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:38.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.610+0000 7f020be00640 1 -- 172.21.15.89:0/707882422 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f02000cfa20 con 0x7f0214102d30 2024-09-10T08:39:38.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.611+0000 7f0213400640 1 --2- 172.21.15.89:0/707882422 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f01f80a2c30 0x7f01f80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:38.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.611+0000 7f0213400640 1 --2- 172.21.15.89:0/707882422 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f01f80a2c30 0x7f01f80a50f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0214102b90 tx=0x7f0204005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:38.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.612+0000 7f020be00640 1 -- 172.21.15.89:0/707882422 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f02000541b0 con 0x7f0214102d30 2024-09-10T08:39:38.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:38.832+0000 7f021a000640 1 -- 172.21.15.89:0/707882422 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f02141a2780 con 0x7f0214102d30 2024-09-10T08:39:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:38 smithi111 ceph-mon[35220]: pgmap v384: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:39:39.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:38 smithi089 ceph-mon[30757]: pgmap v384: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:39:39.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:38 smithi149 ceph-mon[36167]: pgmap v384: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-10T08:39:39.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.730+0000 7f020be00640 1 -- 172.21.15.89:0/707882422 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f020009c550 con 0x7f0214102d30 2024-09-10T08:39:39.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.733+0000 7f021a000640 1 -- 172.21.15.89:0/707882422 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f01f80a2c30 msgr2=0x7f01f80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:39.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.733+0000 7f021a000640 1 --2- 172.21.15.89:0/707882422 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f01f80a2c30 0x7f01f80a50f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0214102b90 tx=0x7f0204005f90 comp rx=0 tx=0).stop 2024-09-10T08:39:39.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.733+0000 7f021a000640 1 -- 172.21.15.89:0/707882422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0214102d30 msgr2=0x7f021419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:39.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.733+0000 7f021a000640 1 --2- 172.21.15.89:0/707882422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0214102d30 0x7f021419a4e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f020000e9f0 tx=0x7f020000edf0 comp rx=0 tx=0).stop 2024-09-10T08:39:39.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.733+0000 7f021a000640 1 -- 172.21.15.89:0/707882422 shutdown_connections 2024-09-10T08:39:39.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.733+0000 7f021a000640 1 --2- 172.21.15.89:0/707882422 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f01f80a2c30 0x7f01f80a50f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:39.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.733+0000 7f021a000640 1 --2- 172.21.15.89:0/707882422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02141036f0 0x7f02141a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:39.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.733+0000 7f021a000640 1 --2- 172.21.15.89:0/707882422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0214102d30 0x7f021419a4e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:39.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.733+0000 7f021a000640 1 --2- 172.21.15.89:0/707882422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0214101b40 0x7f0214199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:39.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.733+0000 7f021a000640 1 -- 172.21.15.89:0/707882422 >> 172.21.15.89:0/707882422 conn(0x7f02140fd2e0 msgr2=0x7f0214074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:39.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.733+0000 7f021a000640 1 -- 172.21.15.89:0/707882422 shutdown_connections 2024-09-10T08:39:39.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.733+0000 7f021a000640 1 -- 172.21.15.89:0/707882422 wait complete. 2024-09-10T08:39:39.751 DEBUG:teuthology.orchestra.run.smithi089:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-10T08:39:39.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.911+0000 7f8d1f200640 1 -- 172.21.15.89:0/2979094314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d20101580 msgr2=0x7f8d20101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:39.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.911+0000 7f8d1f200640 1 --2- 172.21.15.89:0/2979094314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d20101580 0x7f8d20101a00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8d08009900 tx=0x7f8d0802a410 comp rx=0 tx=0).stop 2024-09-10T08:39:39.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.911+0000 7f8d1f200640 1 -- 172.21.15.89:0/2979094314 shutdown_connections 2024-09-10T08:39:39.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.911+0000 7f8d1f200640 1 --2- 172.21.15.89:0/2979094314 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d20101f40 0x7f8d20109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:39.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.911+0000 7f8d1f200640 1 --2- 172.21.15.89:0/2979094314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d20101580 0x7f8d20101a00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:39.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.911+0000 7f8d1f200640 1 --2- 172.21.15.89:0/2979094314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d20103050 0x7f8d20103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:39.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.911+0000 7f8d1f200640 1 -- 172.21.15.89:0/2979094314 >> 172.21.15.89:0/2979094314 conn(0x7f8d200fd0d0 msgr2=0x7f8d200ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:39.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.911+0000 7f8d1f200640 1 -- 172.21.15.89:0/2979094314 shutdown_connections 2024-09-10T08:39:39.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.911+0000 7f8d1f200640 1 -- 172.21.15.89:0/2979094314 wait complete. 2024-09-10T08:39:39.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.912+0000 7f8d1f200640 1 Processor -- start 2024-09-10T08:39:39.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.912+0000 7f8d1f200640 1 -- start start 2024-09-10T08:39:39.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.912+0000 7f8d1f200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d20101580 0x7f8d20199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:39.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.912+0000 7f8d1f200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d20101f40 0x7f8d2019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:39.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.912+0000 7f8d1f200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d20103050 0x7f8d201a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:39.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.912+0000 7f8d1f200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2010d720 con 0x7f8d20103050 2024-09-10T08:39:39.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.912+0000 7f8d1f200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2010d5a0 con 0x7f8d20101580 2024-09-10T08:39:39.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.912+0000 7f8d1f200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2010d8a0 con 0x7f8d20101f40 2024-09-10T08:39:39.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.912+0000 7f8d1e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d20103050 0x7f8d201a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:39.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.913+0000 7f8d1e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d20103050 0x7f8d201a1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54718/0 (socket says 172.21.15.89:54718) 2024-09-10T08:39:39.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.913+0000 7f8d1e800640 1 -- 172.21.15.89:0/3152829939 learned_addr learned my addr 172.21.15.89:0/3152829939 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:39.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.913+0000 7f8d1de00640 1 --2- 172.21.15.89:0/3152829939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d20101580 0x7f8d20199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:39.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.913+0000 7f8d17e00640 1 --2- 172.21.15.89:0/3152829939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d20101f40 0x7f8d2019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:39.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.913+0000 7f8d1e800640 1 -- 172.21.15.89:0/3152829939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d20101f40 msgr2=0x7f8d2019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:39.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.913+0000 7f8d1e800640 1 --2- 172.21.15.89:0/3152829939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d20101f40 0x7f8d2019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:39.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.913+0000 7f8d1e800640 1 -- 172.21.15.89:0/3152829939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d20101580 msgr2=0x7f8d20199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:39.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.913+0000 7f8d1e800640 1 --2- 172.21.15.89:0/3152829939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d20101580 0x7f8d20199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:39.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.913+0000 7f8d1e800640 1 -- 172.21.15.89:0/3152829939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d201a1bf0 con 0x7f8d20103050 2024-09-10T08:39:39.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.913+0000 7f8d17e00640 1 --2- 172.21.15.89:0/3152829939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d20101f40 0x7f8d2019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:39:39.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.913+0000 7f8d1e800640 1 --2- 172.21.15.89:0/3152829939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d20103050 0x7f8d201a1580 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009fd0 tx=0x7f8d1800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:39.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.913+0000 7f8d16a00640 1 -- 172.21.15.89:0/3152829939 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d18017070 con 0x7f8d20103050 2024-09-10T08:39:39.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.913+0000 7f8d16a00640 1 -- 172.21.15.89:0/3152829939 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d18003e50 con 0x7f8d20103050 2024-09-10T08:39:39.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.913+0000 7f8d1f200640 1 -- 172.21.15.89:0/3152829939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d201a1ee0 con 0x7f8d20103050 2024-09-10T08:39:39.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.913+0000 7f8d16a00640 1 -- 172.21.15.89:0/3152829939 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d180106c0 con 0x7f8d20103050 2024-09-10T08:39:39.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.914+0000 7f8d1f200640 1 -- 172.21.15.89:0/3152829939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d201a2350 con 0x7f8d20103050 2024-09-10T08:39:39.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.915+0000 7f8d1f200640 1 -- 172.21.15.89:0/3152829939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d20101a00 con 0x7f8d20103050 2024-09-10T08:39:39.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.917+0000 7f8d16a00640 1 -- 172.21.15.89:0/3152829939 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8d18003970 con 0x7f8d20103050 2024-09-10T08:39:39.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.919+0000 7f8d16a00640 1 --2- 172.21.15.89:0/3152829939 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8d040a2b80 0x7f8d040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:39.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.919+0000 7f8d16a00640 1 -- 172.21.15.89:0/3152829939 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f8d180cf250 con 0x7f8d20103050 2024-09-10T08:39:39.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.919+0000 7f8d1de00640 1 --2- 172.21.15.89:0/3152829939 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8d040a2b80 0x7f8d040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:39.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.919+0000 7f8d1de00640 1 --2- 172.21.15.89:0/3152829939 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8d040a2b80 0x7f8d040a5040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8d10005fd0 tx=0x7f8d1000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:39.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:39.923+0000 7f8d16a00640 1 -- 172.21.15.89:0/3152829939 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8d1809bd80 con 0x7f8d20103050 2024-09-10T08:39:40.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:39 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/707882422' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-10T08:39:40.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:39 smithi111 ceph-mon[35220]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-10T08:39:40.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:39 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/707882422' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-10T08:39:40.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:39 smithi089 ceph-mon[30757]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-10T08:39:40.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.126+0000 7f8d1f200640 1 -- 172.21.15.89:0/3152829939 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f8d20109af0 con 0x7f8d040a2b80 2024-09-10T08:39:40.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.150+0000 7f8d16a00640 1 -- 172.21.15.89:0/3152829939 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f8d20109af0 con 0x7f8d040a2b80 2024-09-10T08:39:40.150 INFO:teuthology.orchestra.run.smithi089.stdout:{} 2024-09-10T08:39:40.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.152+0000 7f8d1f200640 1 -- 172.21.15.89:0/3152829939 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8d040a2b80 msgr2=0x7f8d040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:40.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.152+0000 7f8d1f200640 1 --2- 172.21.15.89:0/3152829939 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8d040a2b80 0x7f8d040a5040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8d10005fd0 tx=0x7f8d1000b410 comp rx=0 tx=0).stop 2024-09-10T08:39:40.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.152+0000 7f8d1f200640 1 -- 172.21.15.89:0/3152829939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d20103050 msgr2=0x7f8d201a1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:40.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.152+0000 7f8d1f200640 1 --2- 172.21.15.89:0/3152829939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d20103050 0x7f8d201a1580 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009fd0 tx=0x7f8d1800b260 comp rx=0 tx=0).stop 2024-09-10T08:39:40.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.152+0000 7f8d1f200640 1 -- 172.21.15.89:0/3152829939 shutdown_connections 2024-09-10T08:39:40.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.152+0000 7f8d1f200640 1 --2- 172.21.15.89:0/3152829939 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8d040a2b80 0x7f8d040a5040 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.152+0000 7f8d1f200640 1 --2- 172.21.15.89:0/3152829939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d20103050 0x7f8d201a1580 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.152+0000 7f8d1f200640 1 --2- 172.21.15.89:0/3152829939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d20101f40 0x7f8d2019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.152+0000 7f8d1f200640 1 --2- 172.21.15.89:0/3152829939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d20101580 0x7f8d20199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.152+0000 7f8d1f200640 1 -- 172.21.15.89:0/3152829939 >> 172.21.15.89:0/3152829939 conn(0x7f8d200fd0d0 msgr2=0x7f8d20108330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:40.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.152+0000 7f8d1f200640 1 -- 172.21.15.89:0/3152829939 shutdown_connections 2024-09-10T08:39:40.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.152+0000 7f8d1f200640 1 -- 172.21.15.89:0/3152829939 wait complete. 2024-09-10T08:39:40.168 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-10T08:39:40.179 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-10T08:39:40.189 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T08:39:40.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:39 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/707882422' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-10T08:39:40.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:39 smithi149 ceph-mon[36167]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-10T08:39:40.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.344+0000 7ff7ad200640 1 -- 172.21.15.89:0/3065587552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a8104f20 msgr2=0x7ff7a81053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:40.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.344+0000 7ff7ad200640 1 --2- 172.21.15.89:0/3065587552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a8104f20 0x7ff7a81053a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff794009900 tx=0x7ff79402a410 comp rx=0 tx=0).stop 2024-09-10T08:39:40.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.344+0000 7ff7ad200640 1 -- 172.21.15.89:0/3065587552 shutdown_connections 2024-09-10T08:39:40.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.344+0000 7ff7ad200640 1 --2- 172.21.15.89:0/3065587552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7a81058e0 0x7ff7a810c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.344+0000 7ff7ad200640 1 --2- 172.21.15.89:0/3065587552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a8104f20 0x7ff7a81053a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.344+0000 7ff7ad200640 1 --2- 172.21.15.89:0/3065587552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7a8103d30 0x7ff7a8104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.344+0000 7ff7ad200640 1 -- 172.21.15.89:0/3065587552 >> 172.21.15.89:0/3065587552 conn(0x7ff7a80ff4b0 msgr2=0x7ff7a81018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:40.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.345+0000 7ff7ad200640 1 -- 172.21.15.89:0/3065587552 shutdown_connections 2024-09-10T08:39:40.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.345+0000 7ff7ad200640 1 -- 172.21.15.89:0/3065587552 wait complete. 2024-09-10T08:39:40.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.345+0000 7ff7ad200640 1 Processor -- start 2024-09-10T08:39:40.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.346+0000 7ff7ad200640 1 -- start start 2024-09-10T08:39:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.346+0000 7ff7ad200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7a8103d30 0x7ff7a819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:40.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.346+0000 7ff7ad200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a8104f20 0x7ff7a819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.346+0000 7ff7ad200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7a81058e0 0x7ff7a81a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.346+0000 7ff7ad200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a810fa00 con 0x7ff7a8104f20 2024-09-10T08:39:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.346+0000 7ff7ad200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a810f880 con 0x7ff7a8103d30 2024-09-10T08:39:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.346+0000 7ff7ad200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a810fb80 con 0x7ff7a81058e0 2024-09-10T08:39:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.346+0000 7ff7a5400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a8104f20 0x7ff7a819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.346+0000 7ff7a5400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a8104f20 0x7ff7a819a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54756/0 (socket says 172.21.15.89:54756) 2024-09-10T08:39:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.346+0000 7ff7a5400640 1 -- 172.21.15.89:0/3509814276 learned_addr learned my addr 172.21.15.89:0/3509814276 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.346+0000 7ff7a6800640 1 --2- 172.21.15.89:0/3509814276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7a81058e0 0x7ff7a81a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.346+0000 7ff7a5e00640 1 --2- 172.21.15.89:0/3509814276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7a8103d30 0x7ff7a819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.346+0000 7ff7a5400640 1 -- 172.21.15.89:0/3509814276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7a81058e0 msgr2=0x7ff7a81a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.346+0000 7ff7a5400640 1 --2- 172.21.15.89:0/3509814276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7a81058e0 0x7ff7a81a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.346+0000 7ff7a5400640 1 -- 172.21.15.89:0/3509814276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7a8103d30 msgr2=0x7ff7a819a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:40.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.346+0000 7ff7a5400640 1 --2- 172.21.15.89:0/3509814276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7a8103d30 0x7ff7a819a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.347+0000 7ff7a5400640 1 -- 172.21.15.89:0/3509814276 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a81a1c40 con 0x7ff7a8104f20 2024-09-10T08:39:40.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.347+0000 7ff7a5400640 1 --2- 172.21.15.89:0/3509814276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a8104f20 0x7ff7a819a560 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff794031440 tx=0x7ff794032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:40.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.347+0000 7ff79ea00640 1 -- 172.21.15.89:0/3509814276 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79403f030 con 0x7ff7a8104f20 2024-09-10T08:39:40.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.347+0000 7ff79ea00640 1 -- 172.21.15.89:0/3509814276 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff794003ce0 con 0x7ff7a8104f20 2024-09-10T08:39:40.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.347+0000 7ff79ea00640 1 -- 172.21.15.89:0/3509814276 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794035070 con 0x7ff7a8104f20 2024-09-10T08:39:40.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.347+0000 7ff7ad200640 1 -- 172.21.15.89:0/3509814276 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a81a1ed0 con 0x7ff7a8104f20 2024-09-10T08:39:40.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.347+0000 7ff7ad200640 1 -- 172.21.15.89:0/3509814276 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a81a2330 con 0x7ff7a8104f20 2024-09-10T08:39:40.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.348+0000 7ff7ad200640 1 -- 172.21.15.89:0/3509814276 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7a810b0d0 con 0x7ff7a8104f20 2024-09-10T08:39:40.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.350+0000 7ff79ea00640 1 -- 172.21.15.89:0/3509814276 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff794042030 con 0x7ff7a8104f20 2024-09-10T08:39:40.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.351+0000 7ff79ea00640 1 --2- 172.21.15.89:0/3509814276 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff78c0a2b90 0x7ff78c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:40.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.352+0000 7ff79ea00640 1 -- 172.21.15.89:0/3509814276 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff7940ef990 con 0x7ff7a8104f20 2024-09-10T08:39:40.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.352+0000 7ff7a5e00640 1 --2- 172.21.15.89:0/3509814276 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff78c0a2b90 0x7ff78c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:40.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.352+0000 7ff7a5e00640 1 --2- 172.21.15.89:0/3509814276 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff78c0a2b90 0x7ff78c0a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff798005fd0 tx=0x7ff79800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:40.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.355+0000 7ff79ea00640 1 -- 172.21.15.89:0/3509814276 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff7940bbe70 con 0x7ff7a8104f20 2024-09-10T08:39:40.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.534+0000 7ff7ad200640 1 -- 172.21.15.89:0/3509814276 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7a8062e90 con 0x7ff7a8104f20 2024-09-10T08:39:40.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.536+0000 7ff79ea00640 1 -- 172.21.15.89:0/3509814276 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24894 (secure 0 0 0) 0x7ff7940c0170 con 0x7ff7a8104f20 2024-09-10T08:39:40.536 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:39:40.536 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":96,"fsid":"91f747bc-6f4e-11ef-bcea-c7b262605968","created":"2024-09-10T08:28:49.697035+0000","modified":"2024-09-10T08:38:43.618497+0000","last_up_change":"2024-09-10T08:36:00.535433+0000","last_in_change":"2024-09-10T08:35:37.131131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:32:02.060714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:36:52.462301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:36:55.119708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:37:03.729002+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d3b55192-bd95-4497-89ea-dc017a25c0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6803","nonce":1760523284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6805","nonce":1760523284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6809","nonce":1760523284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6807","nonce":1760523284}]},"public_addr":"172.21.15.89:6803/1760523284","cluster_addr":"172.21.15.89:6805/1760523284","heartbeat_back_addr":"172.21.15.89:6809/1760523284","heartbeat_front_addr":"172.21.15.89:6807/1760523284","state":["exists","up"]},{"osd":1,"uuid":"60aae117-cb52-4034-a700-11693171cd52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6801","nonce":2369291742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6803","nonce":2369291742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6807","nonce":2369291742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6805","nonce":2369291742}]},"public_addr":"172.21.15.111:6801/2369291742","cluster_addr":"172.21.15.111:6803/2369291742","heartbeat_back_addr":"172.21.15.111:6807/2369291742","heartbeat_front_addr":"172.21.15.111:6805/2369291742","state":["exists","up"]},{"osd":2,"uuid":"a305a019-0870-437c-a298-4f923ec54e77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6801","nonce":1092261395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6803","nonce":1092261395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6807","nonce":1092261395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6805","nonce":1092261395}]},"public_addr":"172.21.15.149:6801/1092261395","cluster_addr":"172.21.15.149:6803/1092261395","heartbeat_back_addr":"172.21.15.149:6807/1092261395","heartbeat_front_addr":"172.21.15.149:6805/1092261395","state":["exists","up"]},{"osd":3,"uuid":"d02f54c1-f883-40f6-9341-8d9fc4b712ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6811","nonce":1405773950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6813","nonce":1405773950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6817","nonce":1405773950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6815","nonce":1405773950}]},"public_addr":"172.21.15.89:6811/1405773950","cluster_addr":"172.21.15.89:6813/1405773950","heartbeat_back_addr":"172.21.15.89:6817/1405773950","heartbeat_front_addr":"172.21.15.89:6815/1405773950","state":["exists","up"]},{"osd":4,"uuid":"23cff46b-88dd-4344-be06-b35db6014f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6809","nonce":316717323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6811","nonce":316717323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6815","nonce":316717323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6813","nonce":316717323}]},"public_addr":"172.21.15.111:6809/316717323","cluster_addr":"172.21.15.111:6811/316717323","heartbeat_back_addr":"172.21.15.111:6815/316717323","heartbeat_front_addr":"172.21.15.111:6813/316717323","state":["exists","up"]},{"osd":5,"uuid":"8e516e8d-e7c6-4ebc-80a5-29932161df4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6809","nonce":1256148461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6811","nonce":1256148461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6815","nonce":1256148461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6813","nonce":1256148461}]},"public_addr":"172.21.15.149:6809/1256148461","cluster_addr":"172.21.15.149:6811/1256148461","heartbeat_back_addr":"172.21.15.149:6815/1256148461","heartbeat_front_addr":"172.21.15.149:6813/1256148461","state":["exists","up"]},{"osd":6,"uuid":"48f92472-f657-4c5a-adeb-20d4fea1ce8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6819","nonce":4190488024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6821","nonce":4190488024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6825","nonce":4190488024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6823","nonce":4190488024}]},"public_addr":"172.21.15.89:6819/4190488024","cluster_addr":"172.21.15.89:6821/4190488024","heartbeat_back_addr":"172.21.15.89:6825/4190488024","heartbeat_front_addr":"172.21.15.89:6823/4190488024","state":["exists","up"]},{"osd":7,"uuid":"14a2a6f0-c4cd-406b-b637-7527fd32babe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6817","nonce":652666376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6819","nonce":652666376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6823","nonce":652666376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6821","nonce":652666376}]},"public_addr":"172.21.15.111:6817/652666376","cluster_addr":"172.21.15.111:6819/652666376","heartbeat_back_addr":"172.21.15.111:6823/652666376","heartbeat_front_addr":"172.21.15.111:6821/652666376","state":["exists","up"]},{"osd":8,"uuid":"a54502e6-95f7-431f-8db1-54a6c7983b5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6817","nonce":2955172869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6819","nonce":2955172869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6823","nonce":2955172869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6821","nonce":2955172869}]},"public_addr":"172.21.15.149:6817/2955172869","cluster_addr":"172.21.15.149:6819/2955172869","heartbeat_back_addr":"172.21.15.149:6823/2955172869","heartbeat_front_addr":"172.21.15.149:6821/2955172869","state":["exists","up"]},{"osd":9,"uuid":"5fe17344-5b1c-4601-a633-639cc1691478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6827","nonce":2427467040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6829","nonce":2427467040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6833","nonce":2427467040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6831","nonce":2427467040}]},"public_addr":"172.21.15.89:6827/2427467040","cluster_addr":"172.21.15.89:6829/2427467040","heartbeat_back_addr":"172.21.15.89:6833/2427467040","heartbeat_front_addr":"172.21.15.89:6831/2427467040","state":["exists","up"]},{"osd":10,"uuid":"d8f29890-3064-4b67-b649-467a0ac86db7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6825","nonce":2328678744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6827","nonce":2328678744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6831","nonce":2328678744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6829","nonce":2328678744}]},"public_addr":"172.21.15.111:6825/2328678744","cluster_addr":"172.21.15.111:6827/2328678744","heartbeat_back_addr":"172.21.15.111:6831/2328678744","heartbeat_front_addr":"172.21.15.111:6829/2328678744","state":["exists","up"]},{"osd":11,"uuid":"92182c73-f139-47ca-a0ef-5046ae9b2310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6825","nonce":3130508827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6827","nonce":3130508827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6831","nonce":3130508827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6829","nonce":3130508827}]},"public_addr":"172.21.15.149:6825/3130508827","cluster_addr":"172.21.15.149:6827/3130508827","heartbeat_back_addr":"172.21.15.149:6831/3130508827","heartbeat_front_addr":"172.21.15.149:6829/3130508827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:09.031810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:33.695124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:58.065510+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:27.592838+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:53.275289+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:18.305677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:47.781535+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:12.889512+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:37.744193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:07.718408+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:33.391027+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:59.013250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/850669510":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/3541027428":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6801/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6800/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/2396468787":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/1609720980":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/995019144":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6801/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6800/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/2179977257":"2024-09-11T08:29:14.554869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:39:40.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.538+0000 7ff7ad200640 1 -- 172.21.15.89:0/3509814276 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff78c0a2b90 msgr2=0x7ff78c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.538+0000 7ff7ad200640 1 --2- 172.21.15.89:0/3509814276 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff78c0a2b90 0x7ff78c0a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff798005fd0 tx=0x7ff79800b410 comp rx=0 tx=0).stop 2024-09-10T08:39:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.538+0000 7ff7ad200640 1 -- 172.21.15.89:0/3509814276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a8104f20 msgr2=0x7ff7a819a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.538+0000 7ff7ad200640 1 --2- 172.21.15.89:0/3509814276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a8104f20 0x7ff7a819a560 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff794031440 tx=0x7ff794032ce0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.538+0000 7ff7ad200640 1 -- 172.21.15.89:0/3509814276 shutdown_connections 2024-09-10T08:39:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.538+0000 7ff7ad200640 1 --2- 172.21.15.89:0/3509814276 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff78c0a2b90 0x7ff78c0a5050 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.538+0000 7ff7ad200640 1 --2- 172.21.15.89:0/3509814276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7a81058e0 0x7ff7a81a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.538+0000 7ff7ad200640 1 --2- 172.21.15.89:0/3509814276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a8104f20 0x7ff7a819a560 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.538+0000 7ff7ad200640 1 --2- 172.21.15.89:0/3509814276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7a8103d30 0x7ff7a819a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.538+0000 7ff7ad200640 1 -- 172.21.15.89:0/3509814276 >> 172.21.15.89:0/3509814276 conn(0x7ff7a80ff4b0 msgr2=0x7ff7a81082a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.539+0000 7ff7ad200640 1 -- 172.21.15.89:0/3509814276 shutdown_connections 2024-09-10T08:39:40.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.539+0000 7ff7ad200640 1 -- 172.21.15.89:0/3509814276 wait complete. 2024-09-10T08:39:40.552 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:39:40 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:39:40.132+0000 7f5303200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:39:40.552 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:39:40 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:39:40.132+0000 7f5303200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:39:40.552 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:39:40 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:39:40.132+0000 7f5303200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:39:40.553 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:39:40 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:39:40.132+0000 7f5303200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:39:40.553 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:39:40 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:39:40.132+0000 7f5303200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:39:40.555 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T08:32:02.060714+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T08:36:52.462301+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T08:36:55.119708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T08:37:03.729002+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-10T08:39:40.556 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T08:39:40.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.715+0000 7fede1400640 1 -- 172.21.15.89:0/3257506826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feddc103c10 msgr2=0x7feddc104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:40.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.715+0000 7fede1400640 1 --2- 172.21.15.89:0/3257506826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feddc103c10 0x7feddc104010 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009900 tx=0x7fedcc02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.716+0000 7fede1400640 1 -- 172.21.15.89:0/3257506826 shutdown_connections 2024-09-10T08:39:40.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.716+0000 7fede1400640 1 --2- 172.21.15.89:0/3257506826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feddc1057c0 0x7feddc10c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.716+0000 7fede1400640 1 --2- 172.21.15.89:0/3257506826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feddc104e00 0x7feddc105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.716+0000 7fede1400640 1 --2- 172.21.15.89:0/3257506826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feddc103c10 0x7feddc104010 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.716+0000 7fede1400640 1 -- 172.21.15.89:0/3257506826 >> 172.21.15.89:0/3257506826 conn(0x7feddc0ff390 msgr2=0x7feddc1017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:40.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.716+0000 7fede1400640 1 -- 172.21.15.89:0/3257506826 shutdown_connections 2024-09-10T08:39:40.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.716+0000 7fede1400640 1 -- 172.21.15.89:0/3257506826 wait complete. 2024-09-10T08:39:40.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.717+0000 7fede1400640 1 Processor -- start 2024-09-10T08:39:40.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.717+0000 7fede1400640 1 -- start start 2024-09-10T08:39:40.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.717+0000 7fede1400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feddc103c10 0x7feddc199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:40.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.717+0000 7fede1400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feddc104e00 0x7feddc19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:40.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.717+0000 7fede1400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feddc1057c0 0x7feddc1a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.717+0000 7fede1400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7feddc10f890 con 0x7feddc1057c0 2024-09-10T08:39:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.717+0000 7fede1400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7feddc10f710 con 0x7feddc104e00 2024-09-10T08:39:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.717+0000 7fede1400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7feddc10fa10 con 0x7feddc103c10 2024-09-10T08:39:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.717+0000 7fede0a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feddc1057c0 0x7feddc1a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.717+0000 7feddbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feddc103c10 0x7feddc199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.717+0000 7fede0a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feddc1057c0 0x7feddc1a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54762/0 (socket says 172.21.15.89:54762) 2024-09-10T08:39:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.718+0000 7fede0a00640 1 -- 172.21.15.89:0/3795478637 learned_addr learned my addr 172.21.15.89:0/3795478637 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.717+0000 7feddb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feddc104e00 0x7feddc19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.718+0000 7feddbe00640 1 -- 172.21.15.89:0/3795478637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feddc104e00 msgr2=0x7feddc19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.718+0000 7feddbe00640 1 --2- 172.21.15.89:0/3795478637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feddc104e00 0x7feddc19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.718+0000 7feddbe00640 1 -- 172.21.15.89:0/3795478637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feddc1057c0 msgr2=0x7feddc1a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:40.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.718+0000 7feddbe00640 1 --2- 172.21.15.89:0/3795478637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feddc1057c0 0x7feddc1a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.718+0000 7feddbe00640 1 -- 172.21.15.89:0/3795478637 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feddc1a1b40 con 0x7feddc103c10 2024-09-10T08:39:40.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.718+0000 7fede0a00640 1 --2- 172.21.15.89:0/3795478637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feddc1057c0 0x7feddc1a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:40.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.718+0000 7feddbe00640 1 --2- 172.21.15.89:0/3795478637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feddc103c10 0x7feddc199f20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009230 tx=0x7fedcc031b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:40.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.718+0000 7fedd8c00640 1 -- 172.21.15.89:0/3795478637 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc03f070 con 0x7feddc103c10 2024-09-10T08:39:40.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.718+0000 7fedd8c00640 1 -- 172.21.15.89:0/3795478637 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedcc003cd0 con 0x7feddc103c10 2024-09-10T08:39:40.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.718+0000 7fede1400640 1 -- 172.21.15.89:0/3795478637 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feddc1a1dd0 con 0x7feddc103c10 2024-09-10T08:39:40.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.719+0000 7fedd8c00640 1 -- 172.21.15.89:0/3795478637 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc035070 con 0x7feddc103c10 2024-09-10T08:39:40.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.719+0000 7fede1400640 1 -- 172.21.15.89:0/3795478637 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feddc1a21d0 con 0x7feddc103c10 2024-09-10T08:39:40.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.720+0000 7fede1400640 1 -- 172.21.15.89:0/3795478637 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feddc10b050 con 0x7feddc103c10 2024-09-10T08:39:40.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.722+0000 7fedd8c00640 1 -- 172.21.15.89:0/3795478637 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fedcc03b4f0 con 0x7feddc103c10 2024-09-10T08:39:40.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.723+0000 7fedd8c00640 1 --2- 172.21.15.89:0/3795478637 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fedc00a2b90 0x7fedc00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:40.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.723+0000 7fedd8c00640 1 -- 172.21.15.89:0/3795478637 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fedcc0efc50 con 0x7feddc103c10 2024-09-10T08:39:40.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.723+0000 7feddb400640 1 --2- 172.21.15.89:0/3795478637 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fedc00a2b90 0x7fedc00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:40.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.724+0000 7feddb400640 1 --2- 172.21.15.89:0/3795478637 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fedc00a2b90 0x7fedc00a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7feddc19b430 tx=0x7fedc800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:40.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.725+0000 7fedd8c00640 1 -- 172.21.15.89:0/3795478637 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fedcc0bc130 con 0x7feddc103c10 2024-09-10T08:39:40.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:40 smithi089 ceph-mon[30757]: pgmap v385: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:40.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:40 smithi089 ceph-mon[30757]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-10T08:39:40.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:40 smithi089 ceph-mon[30757]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:39:40.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:40 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3509814276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:39:40.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.905+0000 7fede1400640 1 -- 172.21.15.89:0/3795478637 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feddc062df0 con 0x7feddc103c10 2024-09-10T08:39:40.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.905+0000 7fedd8c00640 1 -- 172.21.15.89:0/3795478637 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7fedcc0c0430 con 0x7feddc103c10 2024-09-10T08:39:40.905 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2024-09-10T08:39:40.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.908+0000 7fede1400640 1 -- 172.21.15.89:0/3795478637 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fedc00a2b90 msgr2=0x7fedc00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:40.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.908+0000 7fede1400640 1 --2- 172.21.15.89:0/3795478637 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fedc00a2b90 0x7fedc00a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7feddc19b430 tx=0x7fedc800b390 comp rx=0 tx=0).stop 2024-09-10T08:39:40.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.908+0000 7fede1400640 1 -- 172.21.15.89:0/3795478637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feddc103c10 msgr2=0x7feddc199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:40.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.908+0000 7fede1400640 1 --2- 172.21.15.89:0/3795478637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feddc103c10 0x7feddc199f20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009230 tx=0x7fedcc031b60 comp rx=0 tx=0).stop 2024-09-10T08:39:40.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.908+0000 7fede1400640 1 -- 172.21.15.89:0/3795478637 shutdown_connections 2024-09-10T08:39:40.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.908+0000 7fede1400640 1 --2- 172.21.15.89:0/3795478637 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fedc00a2b90 0x7fedc00a5050 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.908+0000 7fede1400640 1 --2- 172.21.15.89:0/3795478637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feddc1057c0 0x7feddc1a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.908+0000 7fede1400640 1 --2- 172.21.15.89:0/3795478637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feddc104e00 0x7feddc19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.908+0000 7fede1400640 1 --2- 172.21.15.89:0/3795478637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feddc103c10 0x7feddc199f20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:40.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.908+0000 7fede1400640 1 -- 172.21.15.89:0/3795478637 >> 172.21.15.89:0/3795478637 conn(0x7feddc0ff390 msgr2=0x7feddc108220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:40.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.908+0000 7fede1400640 1 -- 172.21.15.89:0/3795478637 shutdown_connections 2024-09-10T08:39:40.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:40.908+0000 7fede1400640 1 -- 172.21.15.89:0/3795478637 wait complete. 2024-09-10T08:39:40.925 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T08:39:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:40 smithi111 ceph-mon[35220]: pgmap v385: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:40 smithi111 ceph-mon[35220]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-10T08:39:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:40 smithi111 ceph-mon[35220]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:39:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:40 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3509814276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:39:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.086+0000 7ff30f200640 1 -- 172.21.15.89:0/1036025637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff308102d00 msgr2=0x7ff308103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.086+0000 7ff30f200640 1 --2- 172.21.15.89:0/1036025637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff308102d00 0x7ff308103180 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009900 tx=0x7ff2f402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.086+0000 7ff30f200640 1 -- 172.21.15.89:0/1036025637 shutdown_connections 2024-09-10T08:39:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.086+0000 7ff30f200640 1 --2- 172.21.15.89:0/1036025637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3081036c0 0x7ff308109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.086+0000 7ff30f200640 1 --2- 172.21.15.89:0/1036025637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff308102d00 0x7ff308103180 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.086+0000 7ff30f200640 1 --2- 172.21.15.89:0/1036025637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff308101b10 0x7ff308101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.086+0000 7ff30f200640 1 -- 172.21.15.89:0/1036025637 >> 172.21.15.89:0/1036025637 conn(0x7ff3080fd2b0 msgr2=0x7ff3080ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:41.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.086+0000 7ff30f200640 1 -- 172.21.15.89:0/1036025637 shutdown_connections 2024-09-10T08:39:41.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.086+0000 7ff30f200640 1 -- 172.21.15.89:0/1036025637 wait complete. 2024-09-10T08:39:41.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.087+0000 7ff30f200640 1 Processor -- start 2024-09-10T08:39:41.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.087+0000 7ff30f200640 1 -- start start 2024-09-10T08:39:41.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.087+0000 7ff30f200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff308101b10 0x7ff308075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:41.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.088+0000 7ff30f200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff308102d00 0x7ff3080745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:41.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.088+0000 7ff30f200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3081036c0 0x7ff308074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:41.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.088+0000 7ff30f200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff30810d770 con 0x7ff3081036c0 2024-09-10T08:39:41.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.088+0000 7ff30f200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff30810d5f0 con 0x7ff308101b10 2024-09-10T08:39:41.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.088+0000 7ff30f200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff30810d8f0 con 0x7ff308102d00 2024-09-10T08:39:41.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.088+0000 7ff307e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff308101b10 0x7ff308075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.088+0000 7ff30d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3081036c0 0x7ff308074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.088+0000 7ff307e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff308101b10 0x7ff308075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:52888/0 (socket says 172.21.15.89:52888) 2024-09-10T08:39:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.088+0000 7ff307e00640 1 -- 172.21.15.89:0/3584633463 learned_addr learned my addr 172.21.15.89:0/3584633463 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.088+0000 7ff307400640 1 --2- 172.21.15.89:0/3584633463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff308102d00 0x7ff3080745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.088+0000 7ff307e00640 1 -- 172.21.15.89:0/3584633463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff308102d00 msgr2=0x7ff3080745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:41.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.088+0000 7ff307e00640 1 --2- 172.21.15.89:0/3584633463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff308102d00 0x7ff3080745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.088+0000 7ff307e00640 1 -- 172.21.15.89:0/3584633463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3081036c0 msgr2=0x7ff308074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:41.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.088+0000 7ff307e00640 1 --2- 172.21.15.89:0/3584633463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3081036c0 0x7ff308074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.088+0000 7ff307e00640 1 -- 172.21.15.89:0/3584633463 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3080753b0 con 0x7ff308101b10 2024-09-10T08:39:41.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.088+0000 7ff307400640 1 --2- 172.21.15.89:0/3584633463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff308102d00 0x7ff3080745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:39:41.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.088+0000 7ff30d200640 1 --2- 172.21.15.89:0/3584633463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3081036c0 0x7ff308074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:41.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.089+0000 7ff307e00640 1 --2- 172.21.15.89:0/3584633463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff308101b10 0x7ff308075fa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800e9c0 tx=0x7ff2f800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:41.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.089+0000 7ff304c00640 1 -- 172.21.15.89:0/3584633463 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f800cd80 con 0x7ff308101b10 2024-09-10T08:39:41.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.089+0000 7ff304c00640 1 -- 172.21.15.89:0/3584633463 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f80035f0 con 0x7ff308101b10 2024-09-10T08:39:41.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.089+0000 7ff30f200640 1 -- 172.21.15.89:0/3584633463 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3081a1f70 con 0x7ff308101b10 2024-09-10T08:39:41.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.089+0000 7ff304c00640 1 -- 172.21.15.89:0/3584633463 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8010690 con 0x7ff308101b10 2024-09-10T08:39:41.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.089+0000 7ff30f200640 1 -- 172.21.15.89:0/3584633463 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3081a2370 con 0x7ff308101b10 2024-09-10T08:39:41.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.090+0000 7ff30f200640 1 -- 172.21.15.89:0/3584633463 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff308108f40 con 0x7ff308101b10 2024-09-10T08:39:41.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.092+0000 7ff304c00640 1 -- 172.21.15.89:0/3584633463 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff2f80037a0 con 0x7ff308101b10 2024-09-10T08:39:41.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.093+0000 7ff304c00640 1 --2- 172.21.15.89:0/3584633463 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff2ec0a2b90 0x7ff2ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:41.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.093+0000 7ff304c00640 1 -- 172.21.15.89:0/3584633463 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff2f80cf920 con 0x7ff308101b10 2024-09-10T08:39:41.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.093+0000 7ff307400640 1 --2- 172.21.15.89:0/3584633463 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff2ec0a2b90 0x7ff2ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:41.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.094+0000 7ff307400640 1 --2- 172.21.15.89:0/3584633463 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff2ec0a2b90 0x7ff2ec0a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4000c00 tx=0x7ff2f40095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:41.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.095+0000 7ff304c00640 1 -- 172.21.15.89:0/3584633463 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2f809c450 con 0x7ff308101b10 2024-09-10T08:39:41.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:40 smithi149 ceph-mon[36167]: pgmap v385: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T08:39:41.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:40 smithi149 ceph-mon[36167]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-10T08:39:41.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:40 smithi149 ceph-mon[36167]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:39:41.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:40 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3509814276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:39:41.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.266+0000 7ff30f200640 1 -- 172.21.15.89:0/3584633463 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff30802da70 con 0x7ff308101b10 2024-09-10T08:39:41.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.267+0000 7ff304c00640 1 -- 172.21.15.89:0/3584633463 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7ff2f809bd50 con 0x7ff308101b10 2024-09-10T08:39:41.266 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2024-09-10T08:39:41.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.269+0000 7ff30f200640 1 -- 172.21.15.89:0/3584633463 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff2ec0a2b90 msgr2=0x7ff2ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:41.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.269+0000 7ff30f200640 1 --2- 172.21.15.89:0/3584633463 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff2ec0a2b90 0x7ff2ec0a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4000c00 tx=0x7ff2f40095f0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.269+0000 7ff30f200640 1 -- 172.21.15.89:0/3584633463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff308101b10 msgr2=0x7ff308075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:41.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.269+0000 7ff30f200640 1 --2- 172.21.15.89:0/3584633463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff308101b10 0x7ff308075fa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800e9c0 tx=0x7ff2f800ee80 comp rx=0 tx=0).stop 2024-09-10T08:39:41.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.269+0000 7ff30f200640 1 -- 172.21.15.89:0/3584633463 shutdown_connections 2024-09-10T08:39:41.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.269+0000 7ff30f200640 1 --2- 172.21.15.89:0/3584633463 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff2ec0a2b90 0x7ff2ec0a5050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.269+0000 7ff30f200640 1 --2- 172.21.15.89:0/3584633463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3081036c0 0x7ff308074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.269+0000 7ff30f200640 1 --2- 172.21.15.89:0/3584633463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff308102d00 0x7ff3080745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.269+0000 7ff30f200640 1 --2- 172.21.15.89:0/3584633463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff308101b10 0x7ff308075fa0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.269+0000 7ff30f200640 1 -- 172.21.15.89:0/3584633463 >> 172.21.15.89:0/3584633463 conn(0x7ff3080fd2b0 msgr2=0x7ff308106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:41.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.269+0000 7ff30f200640 1 -- 172.21.15.89:0/3584633463 shutdown_connections 2024-09-10T08:39:41.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.269+0000 7ff30f200640 1 -- 172.21.15.89:0/3584633463 wait complete. 2024-09-10T08:39:41.284 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T08:39:41.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.440+0000 7f4827000640 1 -- 172.21.15.89:0/1819668788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48201024b0 msgr2=0x7f4820108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:41.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.440+0000 7f4827000640 1 --2- 172.21.15.89:0/1819668788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48201024b0 0x7f4820108d40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4818009ca0 tx=0x7f481802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.441+0000 7f4827000640 1 -- 172.21.15.89:0/1819668788 shutdown_connections 2024-09-10T08:39:41.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.441+0000 7f4827000640 1 --2- 172.21.15.89:0/1819668788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f48201024b0 0x7f4820108d40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.441+0000 7f4827000640 1 --2- 172.21.15.89:0/1819668788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4820101af0 0x7f4820101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.441+0000 7f4827000640 1 --2- 172.21.15.89:0/1819668788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4820100900 0x7f4820100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.441+0000 7f4827000640 1 -- 172.21.15.89:0/1819668788 >> 172.21.15.89:0/1819668788 conn(0x7f48200fc0a0 msgr2=0x7f48200fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:41.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.441+0000 7f4827000640 1 -- 172.21.15.89:0/1819668788 shutdown_connections 2024-09-10T08:39:41.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.441+0000 7f4827000640 1 -- 172.21.15.89:0/1819668788 wait complete. 2024-09-10T08:39:41.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.442+0000 7f4827000640 1 Processor -- start 2024-09-10T08:39:41.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.442+0000 7f4827000640 1 -- start start 2024-09-10T08:39:41.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.442+0000 7f4827000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4820100900 0x7f4820199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:41.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.442+0000 7f4827000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4820101af0 0x7f482019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:41.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.442+0000 7f4827000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48201024b0 0x7f48201a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:41.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.442+0000 7f4827000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f482010d730 con 0x7f4820101af0 2024-09-10T08:39:41.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.442+0000 7f4827000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f482010d5b0 con 0x7f4820100900 2024-09-10T08:39:41.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.442+0000 7f4827000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f482010d8b0 con 0x7f48201024b0 2024-09-10T08:39:41.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.442+0000 7f4825200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4820101af0 0x7f482019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:41.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.442+0000 7f4826600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48201024b0 0x7f48201a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:41.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.442+0000 7f4826600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48201024b0 0x7f48201a1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:38646/0 (socket says 172.21.15.89:38646) 2024-09-10T08:39:41.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.442+0000 7f4826600640 1 -- 172.21.15.89:0/2128815148 learned_addr learned my addr 172.21.15.89:0/2128815148 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:41.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.442+0000 7f4825200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4820101af0 0x7f482019a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54786/0 (socket says 172.21.15.89:54786) 2024-09-10T08:39:41.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.442+0000 7f4825c00640 1 --2- 172.21.15.89:0/2128815148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4820100900 0x7f4820199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:41.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.443+0000 7f4825200640 1 -- 172.21.15.89:0/2128815148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48201024b0 msgr2=0x7f48201a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:41.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.443+0000 7f4825200640 1 --2- 172.21.15.89:0/2128815148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48201024b0 0x7f48201a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.443+0000 7f4825200640 1 -- 172.21.15.89:0/2128815148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4820100900 msgr2=0x7f4820199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:41.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.443+0000 7f4825200640 1 --2- 172.21.15.89:0/2128815148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4820100900 0x7f4820199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.443+0000 7f4825200640 1 -- 172.21.15.89:0/2128815148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48201a1ab0 con 0x7f4820101af0 2024-09-10T08:39:41.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.443+0000 7f4825200640 1 --2- 172.21.15.89:0/2128815148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4820101af0 0x7f482019a3d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f480c00d6d0 tx=0x7f480c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:41.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.443+0000 7f481e000640 1 -- 172.21.15.89:0/2128815148 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c017070 con 0x7f4820101af0 2024-09-10T08:39:41.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.443+0000 7f481e000640 1 -- 172.21.15.89:0/2128815148 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f480c009820 con 0x7f4820101af0 2024-09-10T08:39:41.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.443+0000 7f4827000640 1 -- 172.21.15.89:0/2128815148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48201a1da0 con 0x7f4820101af0 2024-09-10T08:39:41.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.443+0000 7f481e000640 1 -- 172.21.15.89:0/2128815148 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c004d00 con 0x7f4820101af0 2024-09-10T08:39:41.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.443+0000 7f4827000640 1 -- 172.21.15.89:0/2128815148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48201a2260 con 0x7f4820101af0 2024-09-10T08:39:41.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.444+0000 7f4827000640 1 -- 172.21.15.89:0/2128815148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48201a2490 con 0x7f4820101af0 2024-09-10T08:39:41.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.446+0000 7f481e000640 1 -- 172.21.15.89:0/2128815148 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f480c0099d0 con 0x7f4820101af0 2024-09-10T08:39:41.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.448+0000 7f481e000640 1 --2- 172.21.15.89:0/2128815148 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f48080a2b90 0x7f48080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:41.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.448+0000 7f481e000640 1 -- 172.21.15.89:0/2128815148 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f480c0cef40 con 0x7f4820101af0 2024-09-10T08:39:41.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.448+0000 7f4825c00640 1 --2- 172.21.15.89:0/2128815148 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f48080a2b90 0x7f48080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:41.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.449+0000 7f4825c00640 1 --2- 172.21.15.89:0/2128815148 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f48080a2b90 0x7f48080a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f48140095f0 tx=0x7f4814005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:41.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.452+0000 7f481e000640 1 -- 172.21.15.89:0/2128815148 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f480c09b420 con 0x7f4820101af0 2024-09-10T08:39:41.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.628+0000 7f4827000640 1 -- 172.21.15.89:0/2128815148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f48201a2660 con 0x7f4820101af0 2024-09-10T08:39:41.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.628+0000 7f481e000640 1 -- 172.21.15.89:0/2128815148 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7f480c09f720 con 0x7f4820101af0 2024-09-10T08:39:41.628 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2024-09-10T08:39:41.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.630+0000 7f4827000640 1 -- 172.21.15.89:0/2128815148 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f48080a2b90 msgr2=0x7f48080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:41.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.630+0000 7f4827000640 1 --2- 172.21.15.89:0/2128815148 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f48080a2b90 0x7f48080a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f48140095f0 tx=0x7f4814005f90 comp rx=0 tx=0).stop 2024-09-10T08:39:41.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.631+0000 7f4827000640 1 -- 172.21.15.89:0/2128815148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4820101af0 msgr2=0x7f482019a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:41.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.631+0000 7f4827000640 1 --2- 172.21.15.89:0/2128815148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4820101af0 0x7f482019a3d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f480c00d6d0 tx=0x7f480c00db90 comp rx=0 tx=0).stop 2024-09-10T08:39:41.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.631+0000 7f4827000640 1 -- 172.21.15.89:0/2128815148 shutdown_connections 2024-09-10T08:39:41.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.631+0000 7f4827000640 1 --2- 172.21.15.89:0/2128815148 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f48080a2b90 0x7f48080a5050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.631+0000 7f4827000640 1 --2- 172.21.15.89:0/2128815148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48201024b0 0x7f48201a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.631+0000 7f4827000640 1 --2- 172.21.15.89:0/2128815148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4820101af0 0x7f482019a3d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.631+0000 7f4827000640 1 --2- 172.21.15.89:0/2128815148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4820100900 0x7f4820199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.631+0000 7f4827000640 1 -- 172.21.15.89:0/2128815148 >> 172.21.15.89:0/2128815148 conn(0x7f48200fc0a0 msgr2=0x7f4820109700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:41.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.631+0000 7f4827000640 1 -- 172.21.15.89:0/2128815148 shutdown_connections 2024-09-10T08:39:41.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.631+0000 7f4827000640 1 -- 172.21.15.89:0/2128815148 wait complete. 2024-09-10T08:39:41.646 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-10T08:39:41.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.812+0000 7f7f2bc00640 1 -- 172.21.15.89:0/1637389985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f24074bf0 msgr2=0x7f7f24075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:41.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.812+0000 7f7f2bc00640 1 --2- 172.21.15.89:0/1637389985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f24074bf0 0x7f7f24075050 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009900 tx=0x7f7f1002a480 comp rx=0 tx=0).stop 2024-09-10T08:39:41.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.812+0000 7f7f2bc00640 1 -- 172.21.15.89:0/1637389985 shutdown_connections 2024-09-10T08:39:41.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.812+0000 7f7f2bc00640 1 --2- 172.21.15.89:0/1637389985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f24109990 0x7f7f2410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.812+0000 7f7f2bc00640 1 --2- 172.21.15.89:0/1637389985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f24074bf0 0x7f7f24075050 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.812+0000 7f7f2bc00640 1 --2- 172.21.15.89:0/1637389985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f240761d0 0x7f7f24074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.812+0000 7f7f2bc00640 1 -- 172.21.15.89:0/1637389985 >> 172.21.15.89:0/1637389985 conn(0x7f7f240fd2a0 msgr2=0x7f7f240ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:41.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.813+0000 7f7f2bc00640 1 -- 172.21.15.89:0/1637389985 shutdown_connections 2024-09-10T08:39:41.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.813+0000 7f7f2bc00640 1 -- 172.21.15.89:0/1637389985 wait complete. 2024-09-10T08:39:41.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.814+0000 7f7f2bc00640 1 Processor -- start 2024-09-10T08:39:41.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.814+0000 7f7f2bc00640 1 -- start start 2024-09-10T08:39:41.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.814+0000 7f7f2bc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f24074bf0 0x7f7f2419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:41.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.814+0000 7f7f2bc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f240761d0 0x7f7f2419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:41.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.814+0000 7f7f2bc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f24109990 0x7f7f241a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:41.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.814+0000 7f7f2bc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2410f9e0 con 0x7f7f240761d0 2024-09-10T08:39:41.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.814+0000 7f7f2bc00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2410f860 con 0x7f7f24074bf0 2024-09-10T08:39:41.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.814+0000 7f7f2bc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2410fb60 con 0x7f7f24109990 2024-09-10T08:39:41.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.814+0000 7f7f23e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f240761d0 0x7f7f2419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:41.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.814+0000 7f7f23e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f240761d0 0x7f7f2419a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54796/0 (socket says 172.21.15.89:54796) 2024-09-10T08:39:41.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.814+0000 7f7f23e00640 1 -- 172.21.15.89:0/900311701 learned_addr learned my addr 172.21.15.89:0/900311701 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:41.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.814+0000 7f7f29200640 1 --2- 172.21.15.89:0/900311701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f24074bf0 0x7f7f2419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:41.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.814+0000 7f7f29c00640 1 --2- 172.21.15.89:0/900311701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f24109990 0x7f7f241a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:41.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.815+0000 7f7f29200640 1 -- 172.21.15.89:0/900311701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f24109990 msgr2=0x7f7f241a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:41.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.815+0000 7f7f29200640 1 --2- 172.21.15.89:0/900311701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f24109990 0x7f7f241a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.815+0000 7f7f29200640 1 -- 172.21.15.89:0/900311701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f240761d0 msgr2=0x7f7f2419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:41.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.815+0000 7f7f29200640 1 --2- 172.21.15.89:0/900311701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f240761d0 0x7f7f2419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:41.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.815+0000 7f7f29200640 1 -- 172.21.15.89:0/900311701 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f241a1cb0 con 0x7f7f24074bf0 2024-09-10T08:39:41.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.815+0000 7f7f23e00640 1 --2- 172.21.15.89:0/900311701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f240761d0 0x7f7f2419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:41.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.815+0000 7f7f29200640 1 --2- 172.21.15.89:0/900311701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f24074bf0 0x7f7f2419a090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800e9f0 tx=0x7f7f1800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:41.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.815+0000 7f7f21600640 1 -- 172.21.15.89:0/900311701 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1800ccc0 con 0x7f7f24074bf0 2024-09-10T08:39:41.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.815+0000 7f7f21600640 1 -- 172.21.15.89:0/900311701 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f18005020 con 0x7f7f24074bf0 2024-09-10T08:39:41.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.815+0000 7f7f2bc00640 1 -- 172.21.15.89:0/900311701 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f241a1fa0 con 0x7f7f24074bf0 2024-09-10T08:39:41.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.815+0000 7f7f21600640 1 -- 172.21.15.89:0/900311701 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f180049e0 con 0x7f7f24074bf0 2024-09-10T08:39:41.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.815+0000 7f7f2bc00640 1 -- 172.21.15.89:0/900311701 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f241a23b0 con 0x7f7f24074bf0 2024-09-10T08:39:41.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.816+0000 7f7f2bc00640 1 -- 172.21.15.89:0/900311701 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f24108750 con 0x7f7f24074bf0 2024-09-10T08:39:41.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.818+0000 7f7f21600640 1 -- 172.21.15.89:0/900311701 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f7f18005560 con 0x7f7f24074bf0 2024-09-10T08:39:41.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.819+0000 7f7f21600640 1 --2- 172.21.15.89:0/900311701 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7f0c0a2c20 0x7f7f0c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:41.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.820+0000 7f7f21600640 1 -- 172.21.15.89:0/900311701 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f7f180cf530 con 0x7f7f24074bf0 2024-09-10T08:39:41.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.820+0000 7f7f23e00640 1 --2- 172.21.15.89:0/900311701 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7f0c0a2c20 0x7f7f0c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:41.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.820+0000 7f7f23e00640 1 --2- 172.21.15.89:0/900311701 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7f0c0a2c20 0x7f7f0c0a50e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7f2419b5a0 tx=0x7f7f100315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:41.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.824+0000 7f7f21600640 1 -- 172.21.15.89:0/900311701 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7f180051d0 con 0x7f7f24074bf0 2024-09-10T08:39:41.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.997+0000 7f7f2bc00640 1 -- 172.21.15.89:0/900311701 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7f2402da30 con 0x7f7f24074bf0 2024-09-10T08:39:41.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:41.997+0000 7f7f21600640 1 -- 172.21.15.89:0/900311701 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) ==== 103+0+11 (secure 0 0 0) 0x7f7f1809bf00 con 0x7f7f24074bf0 2024-09-10T08:39:41.997 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2024-09-10T08:39:42.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.000+0000 7f7f2bc00640 1 -- 172.21.15.89:0/900311701 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7f0c0a2c20 msgr2=0x7f7f0c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:42.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.000+0000 7f7f2bc00640 1 --2- 172.21.15.89:0/900311701 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7f0c0a2c20 0x7f7f0c0a50e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7f2419b5a0 tx=0x7f7f100315b0 comp rx=0 tx=0).stop 2024-09-10T08:39:42.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.000+0000 7f7f2bc00640 1 -- 172.21.15.89:0/900311701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f24074bf0 msgr2=0x7f7f2419a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:42.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.000+0000 7f7f2bc00640 1 --2- 172.21.15.89:0/900311701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f24074bf0 0x7f7f2419a090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800e9f0 tx=0x7f7f1800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:39:42.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.000+0000 7f7f2bc00640 1 -- 172.21.15.89:0/900311701 shutdown_connections 2024-09-10T08:39:42.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.000+0000 7f7f2bc00640 1 --2- 172.21.15.89:0/900311701 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7f0c0a2c20 0x7f7f0c0a50e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:42.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.000+0000 7f7f2bc00640 1 --2- 172.21.15.89:0/900311701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f24109990 0x7f7f241a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:42.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.000+0000 7f7f2bc00640 1 --2- 172.21.15.89:0/900311701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f240761d0 0x7f7f2419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:42.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.000+0000 7f7f2bc00640 1 --2- 172.21.15.89:0/900311701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f24074bf0 0x7f7f2419a090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:42.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.000+0000 7f7f2bc00640 1 -- 172.21.15.89:0/900311701 >> 172.21.15.89:0/900311701 conn(0x7f7f240fd2a0 msgr2=0x7f7f24105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:42.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.000+0000 7f7f2bc00640 1 -- 172.21.15.89:0/900311701 shutdown_connections 2024-09-10T08:39:42.000 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.000+0000 7f7f2bc00640 1 -- 172.21.15.89:0/900311701 wait complete. 2024-09-10T08:39:42.013 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:41 smithi089 ceph-mon[30757]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:42.013 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:41 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3795478637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:39:42.013 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:41 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3584633463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T08:39:42.013 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:41 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2128815148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T08:39:42.015 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-10T08:39:42.015 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:39:42.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:41 smithi111 ceph-mon[35220]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:42.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:41 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3795478637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:39:42.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:41 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3584633463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T08:39:42.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:41 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2128815148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T08:39:42.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:41 smithi149 ceph-mon[36167]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:42.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:41 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3795478637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:39:42.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:41 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3584633463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T08:39:42.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:41 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2128815148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T08:39:42.382 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:42.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:42 smithi089 ceph-mon[30757]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-10T08:39:42.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:42 smithi089 ceph-mon[30757]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-09-10T08:39:42.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:42 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/900311701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T08:39:42.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.924+0000 7f2057400640 1 -- 172.21.15.89:0/3395875313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20501027a0 msgr2=0x7f20501067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:42.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.924+0000 7f2057400640 1 --2- 172.21.15.89:0/3395875313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20501027a0 0x7f20501067f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f203c009900 tx=0x7f203c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:39:42.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.925+0000 7f2057400640 1 -- 172.21.15.89:0/3395875313 shutdown_connections 2024-09-10T08:39:42.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.925+0000 7f2057400640 1 --2- 172.21.15.89:0/3395875313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2050106f20 0x7f205010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:42.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.925+0000 7f2057400640 1 --2- 172.21.15.89:0/3395875313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20501027a0 0x7f20501067f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:42.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.925+0000 7f2057400640 1 --2- 172.21.15.89:0/3395875313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2050101df0 0x7f20501021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:42.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.925+0000 7f2057400640 1 -- 172.21.15.89:0/3395875313 >> 172.21.15.89:0/3395875313 conn(0x7f20500fd8b0 msgr2=0x7f20500ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:42.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.925+0000 7f2057400640 1 -- 172.21.15.89:0/3395875313 shutdown_connections 2024-09-10T08:39:42.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.925+0000 7f2057400640 1 -- 172.21.15.89:0/3395875313 wait complete. 2024-09-10T08:39:42.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.926+0000 7f2057400640 1 Processor -- start 2024-09-10T08:39:42.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.926+0000 7f2057400640 1 -- start start 2024-09-10T08:39:42.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.926+0000 7f2057400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2050101df0 0x7f205019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:42.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.926+0000 7f2057400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20501027a0 0x7f205019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:42.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.926+0000 7f2057400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2050106f20 0x7f205019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:42.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.926+0000 7f2057400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2050111fd0 con 0x7f2050106f20 2024-09-10T08:39:42.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.926+0000 7f2057400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2050111e50 con 0x7f20501027a0 2024-09-10T08:39:42.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.926+0000 7f2057400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2050112150 con 0x7f2050101df0 2024-09-10T08:39:42.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.927+0000 7f2055400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2050106f20 0x7f205019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:42.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.927+0000 7f2055400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2050106f20 0x7f205019eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59614/0 (socket says 172.21.15.89:59614) 2024-09-10T08:39:42.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.927+0000 7f2054a00640 1 --2- 172.21.15.89:0/1084251085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2050101df0 0x7f205019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:42.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.927+0000 7f2055400640 1 -- 172.21.15.89:0/1084251085 learned_addr learned my addr 172.21.15.89:0/1084251085 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:42.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.927+0000 7f204fe00640 1 --2- 172.21.15.89:0/1084251085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20501027a0 0x7f205019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:42.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.927+0000 7f2055400640 1 -- 172.21.15.89:0/1084251085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2050101df0 msgr2=0x7f205019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:42.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.927+0000 7f2055400640 1 --2- 172.21.15.89:0/1084251085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2050101df0 0x7f205019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:42.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.927+0000 7f2055400640 1 -- 172.21.15.89:0/1084251085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20501027a0 msgr2=0x7f205019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:42.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.927+0000 7f2055400640 1 --2- 172.21.15.89:0/1084251085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20501027a0 0x7f205019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:42.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.927+0000 7f2055400640 1 -- 172.21.15.89:0/1084251085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205019f5b0 con 0x7f2050106f20 2024-09-10T08:39:42.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.927+0000 7f2055400640 1 --2- 172.21.15.89:0/1084251085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2050106f20 0x7f205019eed0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f204800eaa0 tx=0x7f204800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:42.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.928+0000 7f204d600640 1 -- 172.21.15.89:0/1084251085 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204800cdb0 con 0x7f2050106f20 2024-09-10T08:39:42.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.928+0000 7f204d600640 1 -- 172.21.15.89:0/1084251085 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2048005020 con 0x7f2050106f20 2024-09-10T08:39:42.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.928+0000 7f2057400640 1 -- 172.21.15.89:0/1084251085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205019f8a0 con 0x7f2050106f20 2024-09-10T08:39:42.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.928+0000 7f204d600640 1 -- 172.21.15.89:0/1084251085 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20480116b0 con 0x7f2050106f20 2024-09-10T08:39:42.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.928+0000 7f2057400640 1 -- 172.21.15.89:0/1084251085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20501a7180 con 0x7f2050106f20 2024-09-10T08:39:42.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.929+0000 7f2057400640 1 -- 172.21.15.89:0/1084251085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20501021d0 con 0x7f2050106f20 2024-09-10T08:39:42.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.931+0000 7f204d600640 1 -- 172.21.15.89:0/1084251085 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f2048005560 con 0x7f2050106f20 2024-09-10T08:39:42.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.933+0000 7f204d600640 1 --2- 172.21.15.89:0/1084251085 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f20380a2c90 0x7f20380a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:42.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.933+0000 7f204d600640 1 -- 172.21.15.89:0/1084251085 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f20480cf690 con 0x7f2050106f20 2024-09-10T08:39:42.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.933+0000 7f2054a00640 1 --2- 172.21.15.89:0/1084251085 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f20380a2c90 0x7f20380a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:42.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.934+0000 7f2054a00640 1 --2- 172.21.15.89:0/1084251085 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f20380a2c90 0x7f20380a5150 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2044005d00 tx=0x7f204400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:42.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:42.938+0000 7f204d600640 1 -- 172.21.15.89:0/1084251085 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2048016030 con 0x7f2050106f20 2024-09-10T08:39:43.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:42 smithi111 ceph-mon[35220]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-10T08:39:43.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:42 smithi111 ceph-mon[35220]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-09-10T08:39:43.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:42 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/900311701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T08:39:43.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:43.149+0000 7f2057400640 1 -- 172.21.15.89:0/1084251085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20501a7480 con 0x7f2050106f20 2024-09-10T08:39:43.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:43.149+0000 7f204d600640 1 -- 172.21.15.89:0/1084251085 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11918 (secure 0 0 0) 0x7f204809ba10 con 0x7f2050106f20 2024-09-10T08:39:43.149 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:39:43.150 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-10T08:39:39:721431+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:39:38.834417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24518,24496]},"id":1}]} 2024-09-10T08:39:43.150 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-10T08:39:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:43.152+0000 7f2057400640 1 -- 172.21.15.89:0/1084251085 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f20380a2c90 msgr2=0x7f20380a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:43.152+0000 7f2057400640 1 --2- 172.21.15.89:0/1084251085 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f20380a2c90 0x7f20380a5150 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2044005d00 tx=0x7f204400b410 comp rx=0 tx=0).stop 2024-09-10T08:39:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:43.152+0000 7f2057400640 1 -- 172.21.15.89:0/1084251085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2050106f20 msgr2=0x7f205019eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:43.152+0000 7f2057400640 1 --2- 172.21.15.89:0/1084251085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2050106f20 0x7f205019eed0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f204800eaa0 tx=0x7f204800ef60 comp rx=0 tx=0).stop 2024-09-10T08:39:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:43.152+0000 7f2057400640 1 -- 172.21.15.89:0/1084251085 shutdown_connections 2024-09-10T08:39:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:43.152+0000 7f2057400640 1 --2- 172.21.15.89:0/1084251085 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f20380a2c90 0x7f20380a5150 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:43.152+0000 7f2057400640 1 --2- 172.21.15.89:0/1084251085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2050106f20 0x7f205019eed0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:43.152+0000 7f2057400640 1 --2- 172.21.15.89:0/1084251085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20501027a0 0x7f205019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:43.152+0000 7f2057400640 1 --2- 172.21.15.89:0/1084251085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2050101df0 0x7f205019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:43.152+0000 7f2057400640 1 -- 172.21.15.89:0/1084251085 >> 172.21.15.89:0/1084251085 conn(0x7f20500fd8b0 msgr2=0x7f20500ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:43.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:43.152+0000 7f2057400640 1 -- 172.21.15.89:0/1084251085 shutdown_connections 2024-09-10T08:39:43.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:43.152+0000 7f2057400640 1 -- 172.21.15.89:0/1084251085 wait complete. 2024-09-10T08:39:43.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:42 smithi149 ceph-mon[36167]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-10T08:39:43.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:42 smithi149 ceph-mon[36167]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-09-10T08:39:43.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:42 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/900311701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T08:39:43.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:43 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1084251085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:39:44.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:43 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1084251085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:39:44.056 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-10T08:39:44.056 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-10T08:39:44.057 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T08:39:44.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.218+0000 7f1851800640 1 -- 172.21.15.89:0/3222767677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c104f10 msgr2=0x7f184c105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:44.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.218+0000 7f1851800640 1 --2- 172.21.15.89:0/3222767677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c104f10 0x7f184c105390 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1838009900 tx=0x7f183802a410 comp rx=0 tx=0).stop 2024-09-10T08:39:44.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.218+0000 7f1851800640 1 -- 172.21.15.89:0/3222767677 shutdown_connections 2024-09-10T08:39:44.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.218+0000 7f1851800640 1 --2- 172.21.15.89:0/3222767677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f184c1058d0 0x7f184c10c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.218+0000 7f1851800640 1 --2- 172.21.15.89:0/3222767677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c104f10 0x7f184c105390 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.218+0000 7f1851800640 1 --2- 172.21.15.89:0/3222767677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f184c103d20 0x7f184c104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.218+0000 7f1851800640 1 -- 172.21.15.89:0/3222767677 >> 172.21.15.89:0/3222767677 conn(0x7f184c0ff480 msgr2=0x7f184c1018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:44.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.219+0000 7f1851800640 1 -- 172.21.15.89:0/3222767677 shutdown_connections 2024-09-10T08:39:44.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.219+0000 7f1851800640 1 -- 172.21.15.89:0/3222767677 wait complete. 2024-09-10T08:39:44.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.219+0000 7f1851800640 1 Processor -- start 2024-09-10T08:39:44.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.219+0000 7f1851800640 1 -- start start 2024-09-10T08:39:44.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.220+0000 7f1851800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f184c103d20 0x7f184c19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:44.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.220+0000 7f1851800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f184c104f10 0x7f184c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:44.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.220+0000 7f1851800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c1058d0 0x7f184c1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:44.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.220+0000 7f1851800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f184c10f9f0 con 0x7f184c1058d0 2024-09-10T08:39:44.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.220+0000 7f1851800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f184c10f870 con 0x7f184c104f10 2024-09-10T08:39:44.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.220+0000 7f1851800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f184c10fb70 con 0x7f184c103d20 2024-09-10T08:39:44.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.220+0000 7f184b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c1058d0 0x7f184c1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:44.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.220+0000 7f184b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c1058d0 0x7f184c1a15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59628/0 (socket says 172.21.15.89:59628) 2024-09-10T08:39:44.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.220+0000 7f184b400640 1 -- 172.21.15.89:0/3543533819 learned_addr learned my addr 172.21.15.89:0/3543533819 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:44.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.220+0000 7f184a000640 1 --2- 172.21.15.89:0/3543533819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f184c104f10 0x7f184c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:44.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.220+0000 7f184aa00640 1 --2- 172.21.15.89:0/3543533819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f184c103d20 0x7f184c19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:44.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.221+0000 7f184b400640 1 -- 172.21.15.89:0/3543533819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f184c103d20 msgr2=0x7f184c19a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:44.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.221+0000 7f184b400640 1 --2- 172.21.15.89:0/3543533819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f184c103d20 0x7f184c19a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.221+0000 7f184b400640 1 -- 172.21.15.89:0/3543533819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f184c104f10 msgr2=0x7f184c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:44.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.221+0000 7f184b400640 1 --2- 172.21.15.89:0/3543533819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f184c104f10 0x7f184c19a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.221+0000 7f184b400640 1 -- 172.21.15.89:0/3543533819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184c1a1c30 con 0x7f184c1058d0 2024-09-10T08:39:44.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.221+0000 7f184b400640 1 --2- 172.21.15.89:0/3543533819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c1058d0 0x7f184c1a15c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f184400eb10 tx=0x7f184400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:44.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.221+0000 7f1843400640 1 -- 172.21.15.89:0/3543533819 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844004b20 con 0x7f184c1058d0 2024-09-10T08:39:44.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.221+0000 7f1843400640 1 -- 172.21.15.89:0/3543533819 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1844004cc0 con 0x7f184c1058d0 2024-09-10T08:39:44.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.221+0000 7f1851800640 1 -- 172.21.15.89:0/3543533819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184c1a1f20 con 0x7f184c1058d0 2024-09-10T08:39:44.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.221+0000 7f1843400640 1 -- 172.21.15.89:0/3543533819 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844010760 con 0x7f184c1058d0 2024-09-10T08:39:44.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.221+0000 7f1851800640 1 -- 172.21.15.89:0/3543533819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f184c1a2430 con 0x7f184c1058d0 2024-09-10T08:39:44.224 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:43 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1084251085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:39:44.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.222+0000 7f1851800640 1 -- 172.21.15.89:0/3543533819 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f184c10b060 con 0x7f184c1058d0 2024-09-10T08:39:44.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.224+0000 7f1843400640 1 -- 172.21.15.89:0/3543533819 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f1844005020 con 0x7f184c1058d0 2024-09-10T08:39:44.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.225+0000 7f1843400640 1 --2- 172.21.15.89:0/3543533819 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f18300a2ca0 0x7f18300a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:44.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.226+0000 7f1843400640 1 -- 172.21.15.89:0/3543533819 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f18440cfcd0 con 0x7f184c1058d0 2024-09-10T08:39:44.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.226+0000 7f184aa00640 1 --2- 172.21.15.89:0/3543533819 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f18300a2ca0 0x7f18300a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:44.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.226+0000 7f184aa00640 1 --2- 172.21.15.89:0/3543533819 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f18300a2ca0 0x7f18300a5160 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f183c0095f0 tx=0x7f183c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:44.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.229+0000 7f1843400640 1 -- 172.21.15.89:0/3543533819 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f184409c6a0 con 0x7f184c1058d0 2024-09-10T08:39:44.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.401+0000 7f1851800640 1 -- 172.21.15.89:0/3543533819 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f184c062630 con 0x7f18300a2ca0 2024-09-10T08:39:44.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.411+0000 7f1843400640 1 -- 172.21.15.89:0/3543533819 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340807 (secure 0 0 0) 0x7f184c062630 con 0x7f18300a2ca0 2024-09-10T08:39:44.417 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:39:44.419 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-09-10T08:39:44.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.416+0000 7f1851800640 1 -- 172.21.15.89:0/3543533819 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f18300a2ca0 msgr2=0x7f18300a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:44.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.416+0000 7f1851800640 1 --2- 172.21.15.89:0/3543533819 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f18300a2ca0 0x7f18300a5160 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f183c0095f0 tx=0x7f183c005f90 comp rx=0 tx=0).stop 2024-09-10T08:39:44.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.416+0000 7f1851800640 1 -- 172.21.15.89:0/3543533819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c1058d0 msgr2=0x7f184c1a15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:44.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.416+0000 7f1851800640 1 --2- 172.21.15.89:0/3543533819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c1058d0 0x7f184c1a15c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f184400eb10 tx=0x7f184400c2d0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.416+0000 7f1851800640 1 -- 172.21.15.89:0/3543533819 shutdown_connections 2024-09-10T08:39:44.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.416+0000 7f1851800640 1 --2- 172.21.15.89:0/3543533819 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f18300a2ca0 0x7f18300a5160 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.416+0000 7f1851800640 1 --2- 172.21.15.89:0/3543533819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c1058d0 0x7f184c1a15c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.416+0000 7f1851800640 1 --2- 172.21.15.89:0/3543533819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f184c104f10 0x7f184c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.416+0000 7f1851800640 1 --2- 172.21.15.89:0/3543533819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f184c103d20 0x7f184c19a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.416+0000 7f1851800640 1 -- 172.21.15.89:0/3543533819 >> 172.21.15.89:0/3543533819 conn(0x7f184c0ff480 msgr2=0x7f184c108230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:44.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.416+0000 7f1851800640 1 -- 172.21.15.89:0/3543533819 shutdown_connections 2024-09-10T08:39:44.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.416+0000 7f1851800640 1 -- 172.21.15.89:0/3543533819 wait complete. 2024-09-10T08:39:44.434 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-09-10T08:39:43.612696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659914,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":825,"num_read_kb":531,"num_write":369,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181900,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":797,"ondisk_log_size":797,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357964,"kb_used_data":12876,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583860,"statfs":{"total":1151940427776,"available":1151573872640,"internally_reserved":0,"allocated":13185024,"data_stored":8527112,"data_compressed":467733,"data_compressed_allocated":638976,"data_compressed_original":2777088,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1408,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":32,"num_write":15,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003513"},"pg_stats":[{"pgid":"2.1f","version":"94'9","reported_seq":67,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.663591+0000","last_change":"2024-09-10T08:38:43.663079+0000","last_active":"2024-09-10T08:38:43.663591+0000","last_peered":"2024-09-10T08:38:43.663591+0000","last_clean":"2024-09-10T08:38:43.663591+0000","last_became_active":"2024-09-10T08:38:43.656506+0000","last_became_peered":"2024-09-10T08:38:43.656506+0000","last_unstale":"2024-09-10T08:38:43.663591+0000","last_undegraded":"2024-09-10T08:38:43.663591+0000","last_fullsized":"2024-09-10T08:38:43.663591+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:36.881564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.579344+0000","last_change":"2024-09-10T08:36:56.145816+0000","last_active":"2024-09-10T08:39:11.579344+0000","last_peered":"2024-09-10T08:39:11.579344+0000","last_clean":"2024-09-10T08:39:11.579344+0000","last_became_active":"2024-09-10T08:36:56.145462+0000","last_became_peered":"2024-09-10T08:36:56.145462+0000","last_unstale":"2024-09-10T08:39:11.579344+0000","last_undegraded":"2024-09-10T08:39:11.579344+0000","last_fullsized":"2024-09-10T08:39:11.579344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:53.890190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629879+0000","last_change":"2024-09-10T08:37:05.252761+0000","last_active":"2024-09-10T08:38:43.629879+0000","last_peered":"2024-09-10T08:38:43.629879+0000","last_clean":"2024-09-10T08:38:43.629879+0000","last_became_active":"2024-09-10T08:37:05.252179+0000","last_became_peered":"2024-09-10T08:37:05.252179+0000","last_unstale":"2024-09-10T08:38:43.629879+0000","last_undegraded":"2024-09-10T08:38:43.629879+0000","last_fullsized":"2024-09-10T08:38:43.629879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:52.854828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631850+0000","last_change":"2024-09-10T08:36:53.735650+0000","last_active":"2024-09-10T08:38:43.631850+0000","last_peered":"2024-09-10T08:38:43.631850+0000","last_clean":"2024-09-10T08:38:43.631850+0000","last_became_active":"2024-09-10T08:36:53.735128+0000","last_became_peered":"2024-09-10T08:36:53.735128+0000","last_unstale":"2024-09-10T08:38:43.631850+0000","last_undegraded":"2024-09-10T08:38:43.631850+0000","last_fullsized":"2024-09-10T08:38:43.631850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:27.309591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592923+0000","last_change":"2024-09-10T08:36:56.153364+0000","last_active":"2024-09-10T08:39:11.592923+0000","last_peered":"2024-09-10T08:39:11.592923+0000","last_clean":"2024-09-10T08:39:11.592923+0000","last_became_active":"2024-09-10T08:36:56.152979+0000","last_became_peered":"2024-09-10T08:36:56.152979+0000","last_unstale":"2024-09-10T08:39:11.592923+0000","last_undegraded":"2024-09-10T08:39:11.592923+0000","last_fullsized":"2024-09-10T08:39:11.592923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:12.172620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654774+0000","last_change":"2024-09-10T08:37:05.254551+0000","last_active":"2024-09-10T08:38:43.654774+0000","last_peered":"2024-09-10T08:38:43.654774+0000","last_clean":"2024-09-10T08:38:43.654774+0000","last_became_active":"2024-09-10T08:37:05.254055+0000","last_became_peered":"2024-09-10T08:37:05.254055+0000","last_unstale":"2024-09-10T08:38:43.654774+0000","last_undegraded":"2024-09-10T08:38:43.654774+0000","last_fullsized":"2024-09-10T08:38:43.654774+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:08.472502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626522+0000","last_change":"2024-09-10T08:36:53.722774+0000","last_active":"2024-09-10T08:38:43.626522+0000","last_peered":"2024-09-10T08:38:43.626522+0000","last_clean":"2024-09-10T08:38:43.626522+0000","last_became_active":"2024-09-10T08:36:53.722459+0000","last_became_peered":"2024-09-10T08:36:53.722459+0000","last_unstale":"2024-09-10T08:38:43.626522+0000","last_undegraded":"2024-09-10T08:38:43.626522+0000","last_fullsized":"2024-09-10T08:38:43.626522+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:26.612354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592240+0000","last_change":"2024-09-10T08:36:56.143757+0000","last_active":"2024-09-10T08:39:11.592240+0000","last_peered":"2024-09-10T08:39:11.592240+0000","last_clean":"2024-09-10T08:39:11.592240+0000","last_became_active":"2024-09-10T08:36:56.143430+0000","last_became_peered":"2024-09-10T08:36:56.143430+0000","last_unstale":"2024-09-10T08:39:11.592240+0000","last_undegraded":"2024-09-10T08:39:11.592240+0000","last_fullsized":"2024-09-10T08:39:11.592240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:49.737298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655120+0000","last_change":"2024-09-10T08:37:05.254503+0000","last_active":"2024-09-10T08:38:43.655120+0000","last_peered":"2024-09-10T08:38:43.655120+0000","last_clean":"2024-09-10T08:38:43.655120+0000","last_became_active":"2024-09-10T08:37:05.254070+0000","last_became_peered":"2024-09-10T08:37:05.254070+0000","last_unstale":"2024-09-10T08:38:43.655120+0000","last_undegraded":"2024-09-10T08:38:43.655120+0000","last_fullsized":"2024-09-10T08:38:43.655120+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:16.741698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626438+0000","last_change":"2024-09-10T08:36:53.716742+0000","last_active":"2024-09-10T08:38:43.626438+0000","last_peered":"2024-09-10T08:38:43.626438+0000","last_clean":"2024-09-10T08:38:43.626438+0000","last_became_active":"2024-09-10T08:36:53.716439+0000","last_became_peered":"2024-09-10T08:36:53.716439+0000","last_unstale":"2024-09-10T08:38:43.626438+0000","last_undegraded":"2024-09-10T08:38:43.626438+0000","last_fullsized":"2024-09-10T08:38:43.626438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:46.882805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.581828+0000","last_change":"2024-09-10T08:36:56.146841+0000","last_active":"2024-09-10T08:39:11.581828+0000","last_peered":"2024-09-10T08:39:11.581828+0000","last_clean":"2024-09-10T08:39:11.581828+0000","last_became_active":"2024-09-10T08:36:56.146531+0000","last_became_peered":"2024-09-10T08:36:56.146531+0000","last_unstale":"2024-09-10T08:39:11.581828+0000","last_undegraded":"2024-09-10T08:39:11.581828+0000","last_fullsized":"2024-09-10T08:39:11.581828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:37.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654877+0000","last_change":"2024-09-10T08:37:43.223181+0000","last_active":"2024-09-10T08:38:43.654877+0000","last_peered":"2024-09-10T08:38:43.654877+0000","last_clean":"2024-09-10T08:38:43.654877+0000","last_became_active":"2024-09-10T08:37:43.222794+0000","last_became_peered":"2024-09-10T08:37:43.222794+0000","last_unstale":"2024-09-10T08:38:43.654877+0000","last_undegraded":"2024-09-10T08:38:43.654877+0000","last_fullsized":"2024-09-10T08:38:43.654877+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:22.204517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593593+0000","last_change":"2024-09-10T08:36:56.142161+0000","last_active":"2024-09-10T08:39:11.593593+0000","last_peered":"2024-09-10T08:39:11.593593+0000","last_clean":"2024-09-10T08:39:11.593593+0000","last_became_active":"2024-09-10T08:36:56.141838+0000","last_became_peered":"2024-09-10T08:36:56.141838+0000","last_unstale":"2024-09-10T08:39:11.593593+0000","last_undegraded":"2024-09-10T08:39:11.593593+0000","last_fullsized":"2024-09-10T08:39:11.593593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:09.909747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626125+0000","last_change":"2024-09-10T08:36:53.722238+0000","last_active":"2024-09-10T08:38:43.626125+0000","last_peered":"2024-09-10T08:38:43.626125+0000","last_clean":"2024-09-10T08:38:43.626125+0000","last_became_active":"2024-09-10T08:36:53.721859+0000","last_became_peered":"2024-09-10T08:36:53.721859+0000","last_unstale":"2024-09-10T08:38:43.626125+0000","last_undegraded":"2024-09-10T08:38:43.626125+0000","last_fullsized":"2024-09-10T08:38:43.626125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:50.615090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655183+0000","last_change":"2024-09-10T08:37:05.414041+0000","last_active":"2024-09-10T08:38:43.655183+0000","last_peered":"2024-09-10T08:38:43.655183+0000","last_clean":"2024-09-10T08:38:43.655183+0000","last_became_active":"2024-09-10T08:37:05.413286+0000","last_became_peered":"2024-09-10T08:37:05.413286+0000","last_unstale":"2024-09-10T08:38:43.655183+0000","last_undegraded":"2024-09-10T08:38:43.655183+0000","last_fullsized":"2024-09-10T08:38:43.655183+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:44.916102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.572936+0000","last_change":"2024-09-10T08:36:56.158278+0000","last_active":"2024-09-10T08:39:11.572936+0000","last_peered":"2024-09-10T08:39:11.572936+0000","last_clean":"2024-09-10T08:39:11.572936+0000","last_became_active":"2024-09-10T08:36:56.157946+0000","last_became_peered":"2024-09-10T08:36:56.157946+0000","last_unstale":"2024-09-10T08:39:11.572936+0000","last_undegraded":"2024-09-10T08:39:11.572936+0000","last_fullsized":"2024-09-10T08:39:11.572936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:42.512513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"96'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:38.713339+0000","last_change":"2024-09-10T08:38:43.635077+0000","last_active":"2024-09-10T08:39:38.713339+0000","last_peered":"2024-09-10T08:39:38.713339+0000","last_clean":"2024-09-10T08:39:38.713339+0000","last_became_active":"2024-09-10T08:38:43.631957+0000","last_became_peered":"2024-09-10T08:38:43.631957+0000","last_unstale":"2024-09-10T08:39:38.713339+0000","last_undegraded":"2024-09-10T08:39:38.713339+0000","last_fullsized":"2024-09-10T08:39:38.713339+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:50.837247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655126+0000","last_change":"2024-09-10T08:37:42.671399+0000","last_active":"2024-09-10T08:38:43.655126+0000","last_peered":"2024-09-10T08:38:43.655126+0000","last_clean":"2024-09-10T08:38:43.655126+0000","last_became_active":"2024-09-10T08:37:42.670865+0000","last_became_peered":"2024-09-10T08:37:42.670865+0000","last_unstale":"2024-09-10T08:38:43.655126+0000","last_undegraded":"2024-09-10T08:38:43.655126+0000","last_fullsized":"2024-09-10T08:38:43.655126+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:16.231538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.577960+0000","last_change":"2024-09-10T08:36:56.144856+0000","last_active":"2024-09-10T08:39:11.577960+0000","last_peered":"2024-09-10T08:39:11.577960+0000","last_clean":"2024-09-10T08:39:11.577960+0000","last_became_active":"2024-09-10T08:36:56.144531+0000","last_became_peered":"2024-09-10T08:36:56.144531+0000","last_unstale":"2024-09-10T08:39:11.577960+0000","last_undegraded":"2024-09-10T08:39:11.577960+0000","last_fullsized":"2024-09-10T08:39:11.577960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:24.586006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631541+0000","last_change":"2024-09-10T08:36:53.721093+0000","last_active":"2024-09-10T08:38:43.631541+0000","last_peered":"2024-09-10T08:38:43.631541+0000","last_clean":"2024-09-10T08:38:43.631541+0000","last_became_active":"2024-09-10T08:36:53.720538+0000","last_became_peered":"2024-09-10T08:36:53.720538+0000","last_unstale":"2024-09-10T08:38:43.631541+0000","last_undegraded":"2024-09-10T08:38:43.631541+0000","last_fullsized":"2024-09-10T08:38:43.631541+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:58.492463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629902+0000","last_change":"2024-09-10T08:37:05.254011+0000","last_active":"2024-09-10T08:38:43.629902+0000","last_peered":"2024-09-10T08:38:43.629902+0000","last_clean":"2024-09-10T08:38:43.629902+0000","last_became_active":"2024-09-10T08:37:05.253066+0000","last_became_peered":"2024-09-10T08:37:05.253066+0000","last_unstale":"2024-09-10T08:38:43.629902+0000","last_undegraded":"2024-09-10T08:38:43.629902+0000","last_fullsized":"2024-09-10T08:38:43.629902+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:16.221302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.553299+0000","last_change":"2024-09-10T08:36:56.141814+0000","last_active":"2024-09-10T08:39:11.553299+0000","last_peered":"2024-09-10T08:39:11.553299+0000","last_clean":"2024-09-10T08:39:11.553299+0000","last_became_active":"2024-09-10T08:36:56.141480+0000","last_became_peered":"2024-09-10T08:36:56.141480+0000","last_unstale":"2024-09-10T08:39:11.553299+0000","last_undegraded":"2024-09-10T08:39:11.553299+0000","last_fullsized":"2024-09-10T08:39:11.553299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:40.594354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.636347+0000","last_change":"2024-09-10T08:38:43.635917+0000","last_active":"2024-09-10T08:38:43.636347+0000","last_peered":"2024-09-10T08:38:43.636347+0000","last_clean":"2024-09-10T08:38:43.636347+0000","last_became_active":"2024-09-10T08:38:43.633406+0000","last_became_peered":"2024-09-10T08:38:43.633406+0000","last_unstale":"2024-09-10T08:38:43.636347+0000","last_undegraded":"2024-09-10T08:38:43.636347+0000","last_fullsized":"2024-09-10T08:38:43.636347+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:44.619072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655297+0000","last_change":"2024-09-10T08:37:05.414181+0000","last_active":"2024-09-10T08:38:43.655297+0000","last_peered":"2024-09-10T08:38:43.655297+0000","last_clean":"2024-09-10T08:38:43.655297+0000","last_became_active":"2024-09-10T08:37:05.413691+0000","last_became_peered":"2024-09-10T08:37:05.413691+0000","last_unstale":"2024-09-10T08:38:43.655297+0000","last_undegraded":"2024-09-10T08:38:43.655297+0000","last_fullsized":"2024-09-10T08:38:43.655297+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:30.744949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584567+0000","last_change":"2024-09-10T08:36:56.153956+0000","last_active":"2024-09-10T08:39:11.584567+0000","last_peered":"2024-09-10T08:39:11.584567+0000","last_clean":"2024-09-10T08:39:11.584567+0000","last_became_active":"2024-09-10T08:36:56.153609+0000","last_became_peered":"2024-09-10T08:36:56.153609+0000","last_unstale":"2024-09-10T08:39:11.584567+0000","last_undegraded":"2024-09-10T08:39:11.584567+0000","last_fullsized":"2024-09-10T08:39:11.584567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:18.185704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655520+0000","last_change":"2024-09-10T08:36:53.733217+0000","last_active":"2024-09-10T08:38:43.655520+0000","last_peered":"2024-09-10T08:38:43.655520+0000","last_clean":"2024-09-10T08:38:43.655520+0000","last_became_active":"2024-09-10T08:36:53.732851+0000","last_became_peered":"2024-09-10T08:36:53.732851+0000","last_unstale":"2024-09-10T08:38:43.655520+0000","last_undegraded":"2024-09-10T08:38:43.655520+0000","last_fullsized":"2024-09-10T08:38:43.655520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:04:01.132196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629889+0000","last_change":"2024-09-10T08:37:05.241868+0000","last_active":"2024-09-10T08:38:43.629889+0000","last_peered":"2024-09-10T08:38:43.629889+0000","last_clean":"2024-09-10T08:38:43.629889+0000","last_became_active":"2024-09-10T08:37:05.241412+0000","last_became_peered":"2024-09-10T08:37:05.241412+0000","last_unstale":"2024-09-10T08:38:43.629889+0000","last_undegraded":"2024-09-10T08:38:43.629889+0000","last_fullsized":"2024-09-10T08:38:43.629889+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:52.543353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.575701+0000","last_change":"2024-09-10T08:36:56.150722+0000","last_active":"2024-09-10T08:39:11.575701+0000","last_peered":"2024-09-10T08:39:11.575701+0000","last_clean":"2024-09-10T08:39:11.575701+0000","last_became_active":"2024-09-10T08:36:56.150423+0000","last_became_peered":"2024-09-10T08:36:56.150423+0000","last_unstale":"2024-09-10T08:39:11.575701+0000","last_undegraded":"2024-09-10T08:39:11.575701+0000","last_fullsized":"2024-09-10T08:39:11.575701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:50.272841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'13","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:38.175435+0000","last_change":"2024-09-10T08:36:53.722477+0000","last_active":"2024-09-10T08:39:38.175435+0000","last_peered":"2024-09-10T08:39:38.175435+0000","last_clean":"2024-09-10T08:39:38.175435+0000","last_became_active":"2024-09-10T08:36:53.722118+0000","last_became_peered":"2024-09-10T08:36:53.722118+0000","last_unstale":"2024-09-10T08:39:38.175435+0000","last_undegraded":"2024-09-10T08:39:38.175435+0000","last_fullsized":"2024-09-10T08:39:38.175435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:06.631646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630992+0000","last_change":"2024-09-10T08:37:05.415656+0000","last_active":"2024-09-10T08:38:43.630992+0000","last_peered":"2024-09-10T08:38:43.630992+0000","last_clean":"2024-09-10T08:38:43.630992+0000","last_became_active":"2024-09-10T08:37:05.413537+0000","last_became_peered":"2024-09-10T08:37:05.413537+0000","last_unstale":"2024-09-10T08:38:43.630992+0000","last_undegraded":"2024-09-10T08:38:43.630992+0000","last_fullsized":"2024-09-10T08:38:43.630992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:04.284410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585069+0000","last_change":"2024-09-10T08:36:56.144157+0000","last_active":"2024-09-10T08:39:11.585069+0000","last_peered":"2024-09-10T08:39:11.585069+0000","last_clean":"2024-09-10T08:39:11.585069+0000","last_became_active":"2024-09-10T08:36:56.140332+0000","last_became_peered":"2024-09-10T08:36:56.140332+0000","last_unstale":"2024-09-10T08:39:11.585069+0000","last_undegraded":"2024-09-10T08:39:11.585069+0000","last_fullsized":"2024-09-10T08:39:11.585069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:09:26.097987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626517+0000","last_change":"2024-09-10T08:36:53.735465+0000","last_active":"2024-09-10T08:38:43.626517+0000","last_peered":"2024-09-10T08:38:43.626517+0000","last_clean":"2024-09-10T08:38:43.626517+0000","last_became_active":"2024-09-10T08:36:53.735160+0000","last_became_peered":"2024-09-10T08:36:53.735160+0000","last_unstale":"2024-09-10T08:38:43.626517+0000","last_undegraded":"2024-09-10T08:38:43.626517+0000","last_fullsized":"2024-09-10T08:38:43.626517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:26:52.332085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627959+0000","last_change":"2024-09-10T08:37:05.247671+0000","last_active":"2024-09-10T08:38:43.627959+0000","last_peered":"2024-09-10T08:38:43.627959+0000","last_clean":"2024-09-10T08:38:43.627959+0000","last_became_active":"2024-09-10T08:37:05.247217+0000","last_became_peered":"2024-09-10T08:37:05.247217+0000","last_unstale":"2024-09-10T08:38:43.627959+0000","last_undegraded":"2024-09-10T08:38:43.627959+0000","last_fullsized":"2024-09-10T08:38:43.627959+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:20.881304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.579998+0000","last_change":"2024-09-10T08:36:56.145420+0000","last_active":"2024-09-10T08:39:11.579998+0000","last_peered":"2024-09-10T08:39:11.579998+0000","last_clean":"2024-09-10T08:39:11.579998+0000","last_became_active":"2024-09-10T08:36:56.145074+0000","last_became_peered":"2024-09-10T08:36:56.145074+0000","last_unstale":"2024-09-10T08:39:11.579998+0000","last_undegraded":"2024-09-10T08:39:11.579998+0000","last_fullsized":"2024-09-10T08:39:11.579998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:13.849977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629812+0000","last_change":"2024-09-10T08:36:53.738237+0000","last_active":"2024-09-10T08:38:43.629812+0000","last_peered":"2024-09-10T08:38:43.629812+0000","last_clean":"2024-09-10T08:38:43.629812+0000","last_became_active":"2024-09-10T08:36:53.737858+0000","last_became_peered":"2024-09-10T08:36:53.737858+0000","last_unstale":"2024-09-10T08:38:43.629812+0000","last_undegraded":"2024-09-10T08:38:43.629812+0000","last_fullsized":"2024-09-10T08:38:43.629812+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:35.345661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.625982+0000","last_change":"2024-09-10T08:37:05.413512+0000","last_active":"2024-09-10T08:38:43.625982+0000","last_peered":"2024-09-10T08:38:43.625982+0000","last_clean":"2024-09-10T08:38:43.625982+0000","last_became_active":"2024-09-10T08:37:05.413010+0000","last_became_peered":"2024-09-10T08:37:05.413010+0000","last_unstale":"2024-09-10T08:38:43.625982+0000","last_undegraded":"2024-09-10T08:38:43.625982+0000","last_fullsized":"2024-09-10T08:38:43.625982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:05.349404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583802+0000","last_change":"2024-09-10T08:36:56.157741+0000","last_active":"2024-09-10T08:39:11.583802+0000","last_peered":"2024-09-10T08:39:11.583802+0000","last_clean":"2024-09-10T08:39:11.583802+0000","last_became_active":"2024-09-10T08:36:56.157295+0000","last_became_peered":"2024-09-10T08:36:56.157295+0000","last_unstale":"2024-09-10T08:39:11.583802+0000","last_undegraded":"2024-09-10T08:39:11.583802+0000","last_fullsized":"2024-09-10T08:39:11.583802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:31.845864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631834+0000","last_change":"2024-09-10T08:36:53.722373+0000","last_active":"2024-09-10T08:38:43.631834+0000","last_peered":"2024-09-10T08:38:43.631834+0000","last_clean":"2024-09-10T08:38:43.631834+0000","last_became_active":"2024-09-10T08:36:53.721975+0000","last_became_peered":"2024-09-10T08:36:53.721975+0000","last_unstale":"2024-09-10T08:38:43.631834+0000","last_undegraded":"2024-09-10T08:38:43.631834+0000","last_fullsized":"2024-09-10T08:38:43.631834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:44.593018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628351+0000","last_change":"2024-09-10T08:37:05.255362+0000","last_active":"2024-09-10T08:38:43.628351+0000","last_peered":"2024-09-10T08:38:43.628351+0000","last_clean":"2024-09-10T08:38:43.628351+0000","last_became_active":"2024-09-10T08:37:05.254967+0000","last_became_peered":"2024-09-10T08:37:05.254967+0000","last_unstale":"2024-09-10T08:38:43.628351+0000","last_undegraded":"2024-09-10T08:38:43.628351+0000","last_fullsized":"2024-09-10T08:38:43.628351+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:31:41.850961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.578715+0000","last_change":"2024-09-10T08:36:56.147699+0000","last_active":"2024-09-10T08:39:11.578715+0000","last_peered":"2024-09-10T08:39:11.578715+0000","last_clean":"2024-09-10T08:39:11.578715+0000","last_became_active":"2024-09-10T08:36:56.147349+0000","last_became_peered":"2024-09-10T08:36:56.147349+0000","last_unstale":"2024-09-10T08:39:11.578715+0000","last_undegraded":"2024-09-10T08:39:11.578715+0000","last_fullsized":"2024-09-10T08:39:11.578715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:25.248588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"96'6","reported_seq":159,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:39.886723+0000","last_change":"2024-09-10T08:36:53.736843+0000","last_active":"2024-09-10T08:39:39.886723+0000","last_peered":"2024-09-10T08:39:39.886723+0000","last_clean":"2024-09-10T08:39:39.886723+0000","last_became_active":"2024-09-10T08:36:53.736340+0000","last_became_peered":"2024-09-10T08:36:53.736340+0000","last_unstale":"2024-09-10T08:39:39.886723+0000","last_undegraded":"2024-09-10T08:39:39.886723+0000","last_fullsized":"2024-09-10T08:39:39.886723+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:24.679679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628575+0000","last_change":"2024-09-10T08:37:05.414321+0000","last_active":"2024-09-10T08:38:43.628575+0000","last_peered":"2024-09-10T08:38:43.628575+0000","last_clean":"2024-09-10T08:38:43.628575+0000","last_became_active":"2024-09-10T08:37:05.413868+0000","last_became_peered":"2024-09-10T08:37:05.413868+0000","last_unstale":"2024-09-10T08:38:43.628575+0000","last_undegraded":"2024-09-10T08:38:43.628575+0000","last_fullsized":"2024-09-10T08:38:43.628575+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:53.988077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583174+0000","last_change":"2024-09-10T08:36:56.141910+0000","last_active":"2024-09-10T08:39:11.583174+0000","last_peered":"2024-09-10T08:39:11.583174+0000","last_clean":"2024-09-10T08:39:11.583174+0000","last_became_active":"2024-09-10T08:36:56.141579+0000","last_became_peered":"2024-09-10T08:36:56.141579+0000","last_unstale":"2024-09-10T08:39:11.583174+0000","last_undegraded":"2024-09-10T08:39:11.583174+0000","last_fullsized":"2024-09-10T08:39:11.583174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:18.464831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631756+0000","last_change":"2024-09-10T08:36:53.720102+0000","last_active":"2024-09-10T08:38:43.631756+0000","last_peered":"2024-09-10T08:38:43.631756+0000","last_clean":"2024-09-10T08:38:43.631756+0000","last_became_active":"2024-09-10T08:36:53.719764+0000","last_became_peered":"2024-09-10T08:36:53.719764+0000","last_unstale":"2024-09-10T08:38:43.631756+0000","last_undegraded":"2024-09-10T08:38:43.631756+0000","last_fullsized":"2024-09-10T08:38:43.631756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:03.609338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655205+0000","last_change":"2024-09-10T08:37:05.413818+0000","last_active":"2024-09-10T08:38:43.655205+0000","last_peered":"2024-09-10T08:38:43.655205+0000","last_clean":"2024-09-10T08:38:43.655205+0000","last_became_active":"2024-09-10T08:37:05.413267+0000","last_became_peered":"2024-09-10T08:37:05.413267+0000","last_unstale":"2024-09-10T08:38:43.655205+0000","last_undegraded":"2024-09-10T08:38:43.655205+0000","last_fullsized":"2024-09-10T08:38:43.655205+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:54.309686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.590486+0000","last_change":"2024-09-10T08:36:56.149286+0000","last_active":"2024-09-10T08:39:11.590486+0000","last_peered":"2024-09-10T08:39:11.590486+0000","last_clean":"2024-09-10T08:39:11.590486+0000","last_became_active":"2024-09-10T08:36:56.146275+0000","last_became_peered":"2024-09-10T08:36:56.146275+0000","last_unstale":"2024-09-10T08:39:11.590486+0000","last_undegraded":"2024-09-10T08:39:11.590486+0000","last_fullsized":"2024-09-10T08:39:11.590486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:19.468224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655269+0000","last_change":"2024-09-10T08:36:53.719756+0000","last_active":"2024-09-10T08:38:43.655269+0000","last_peered":"2024-09-10T08:38:43.655269+0000","last_clean":"2024-09-10T08:38:43.655269+0000","last_became_active":"2024-09-10T08:36:53.719302+0000","last_became_peered":"2024-09-10T08:36:53.719302+0000","last_unstale":"2024-09-10T08:38:43.655269+0000","last_undegraded":"2024-09-10T08:38:43.655269+0000","last_fullsized":"2024-09-10T08:38:43.655269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:47.658081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631788+0000","last_change":"2024-09-10T08:35:39.150715+0000","last_active":"2024-09-10T08:38:43.631788+0000","last_peered":"2024-09-10T08:38:43.631788+0000","last_clean":"2024-09-10T08:38:43.631788+0000","last_became_active":"2024-09-10T08:35:39.150206+0000","last_became_peered":"2024-09-10T08:35:39.150206+0000","last_unstale":"2024-09-10T08:38:43.631788+0000","last_undegraded":"2024-09-10T08:38:43.631788+0000","last_fullsized":"2024-09-10T08:38:43.631788+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:32:02.110870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:32:02.110870+0000","last_clean_scrub_stamp":"2024-09-10T08:32:02.110870+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:20.762043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631998+0000","last_change":"2024-09-10T08:37:42.846547+0000","last_active":"2024-09-10T08:38:43.631998+0000","last_peered":"2024-09-10T08:38:43.631998+0000","last_clean":"2024-09-10T08:38:43.631998+0000","last_became_active":"2024-09-10T08:37:42.845884+0000","last_became_peered":"2024-09-10T08:37:42.845884+0000","last_unstale":"2024-09-10T08:38:43.631998+0000","last_undegraded":"2024-09-10T08:38:43.631998+0000","last_fullsized":"2024-09-10T08:38:43.631998+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:21.905133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593781+0000","last_change":"2024-09-10T08:36:56.144187+0000","last_active":"2024-09-10T08:39:11.593781+0000","last_peered":"2024-09-10T08:39:11.593781+0000","last_clean":"2024-09-10T08:39:11.593781+0000","last_became_active":"2024-09-10T08:36:56.143193+0000","last_became_peered":"2024-09-10T08:36:56.143193+0000","last_unstale":"2024-09-10T08:39:11.593781+0000","last_undegraded":"2024-09-10T08:39:11.593781+0000","last_fullsized":"2024-09-10T08:39:11.593781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:14:26.981884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.639660+0000","last_change":"2024-09-10T08:38:43.639286+0000","last_active":"2024-09-10T08:38:43.639660+0000","last_peered":"2024-09-10T08:38:43.639660+0000","last_clean":"2024-09-10T08:38:43.639660+0000","last_became_active":"2024-09-10T08:38:43.633552+0000","last_became_peered":"2024-09-10T08:38:43.633552+0000","last_unstale":"2024-09-10T08:38:43.639660+0000","last_undegraded":"2024-09-10T08:38:43.639660+0000","last_fullsized":"2024-09-10T08:38:43.639660+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:23.985190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630209+0000","last_change":"2024-09-10T08:37:05.242095+0000","last_active":"2024-09-10T08:38:43.630209+0000","last_peered":"2024-09-10T08:38:43.630209+0000","last_clean":"2024-09-10T08:38:43.630209+0000","last_became_active":"2024-09-10T08:37:05.241645+0000","last_became_peered":"2024-09-10T08:37:05.241645+0000","last_unstale":"2024-09-10T08:38:43.630209+0000","last_undegraded":"2024-09-10T08:38:43.630209+0000","last_fullsized":"2024-09-10T08:38:43.630209+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:30:49.637861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588835+0000","last_change":"2024-09-10T08:36:56.149396+0000","last_active":"2024-09-10T08:39:11.588835+0000","last_peered":"2024-09-10T08:39:11.588835+0000","last_clean":"2024-09-10T08:39:11.588835+0000","last_became_active":"2024-09-10T08:36:56.149079+0000","last_became_peered":"2024-09-10T08:36:56.149079+0000","last_unstale":"2024-09-10T08:39:11.588835+0000","last_undegraded":"2024-09-10T08:39:11.588835+0000","last_fullsized":"2024-09-10T08:39:11.588835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:44.665448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655467+0000","last_change":"2024-09-10T08:36:53.717950+0000","last_active":"2024-09-10T08:38:43.655467+0000","last_peered":"2024-09-10T08:38:43.655467+0000","last_clean":"2024-09-10T08:38:43.655467+0000","last_became_active":"2024-09-10T08:36:53.717668+0000","last_became_peered":"2024-09-10T08:36:53.717668+0000","last_unstale":"2024-09-10T08:38:43.655467+0000","last_undegraded":"2024-09-10T08:38:43.655467+0000","last_fullsized":"2024-09-10T08:38:43.655467+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:58.692631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633140+0000","last_change":"2024-09-10T08:37:05.260178+0000","last_active":"2024-09-10T08:38:43.633140+0000","last_peered":"2024-09-10T08:38:43.633140+0000","last_clean":"2024-09-10T08:38:43.633140+0000","last_became_active":"2024-09-10T08:37:05.259753+0000","last_became_peered":"2024-09-10T08:37:05.259753+0000","last_unstale":"2024-09-10T08:38:43.633140+0000","last_undegraded":"2024-09-10T08:38:43.633140+0000","last_fullsized":"2024-09-10T08:38:43.633140+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:19.364782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588360+0000","last_change":"2024-09-10T08:36:56.147104+0000","last_active":"2024-09-10T08:39:11.588360+0000","last_peered":"2024-09-10T08:39:11.588360+0000","last_clean":"2024-09-10T08:39:11.588360+0000","last_became_active":"2024-09-10T08:36:56.146778+0000","last_became_peered":"2024-09-10T08:36:56.146778+0000","last_unstale":"2024-09-10T08:39:11.588360+0000","last_undegraded":"2024-09-10T08:39:11.588360+0000","last_fullsized":"2024-09-10T08:39:11.588360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:56.092157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628399+0000","last_change":"2024-09-10T08:36:53.719145+0000","last_active":"2024-09-10T08:38:43.628399+0000","last_peered":"2024-09-10T08:38:43.628399+0000","last_clean":"2024-09-10T08:38:43.628399+0000","last_became_active":"2024-09-10T08:36:53.718771+0000","last_became_peered":"2024-09-10T08:36:53.718771+0000","last_unstale":"2024-09-10T08:38:43.628399+0000","last_undegraded":"2024-09-10T08:38:43.628399+0000","last_fullsized":"2024-09-10T08:38:43.628399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:21.109940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654733+0000","last_change":"2024-09-10T08:37:05.242854+0000","last_active":"2024-09-10T08:38:43.654733+0000","last_peered":"2024-09-10T08:38:43.654733+0000","last_clean":"2024-09-10T08:38:43.654733+0000","last_became_active":"2024-09-10T08:37:05.242442+0000","last_became_peered":"2024-09-10T08:37:05.242442+0000","last_unstale":"2024-09-10T08:38:43.654733+0000","last_undegraded":"2024-09-10T08:38:43.654733+0000","last_fullsized":"2024-09-10T08:38:43.654733+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:36.072131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.570294+0000","last_change":"2024-09-10T08:37:42.845642+0000","last_active":"2024-09-10T08:39:11.570294+0000","last_peered":"2024-09-10T08:39:11.570294+0000","last_clean":"2024-09-10T08:39:11.570294+0000","last_became_active":"2024-09-10T08:37:42.845217+0000","last_became_peered":"2024-09-10T08:37:42.845217+0000","last_unstale":"2024-09-10T08:39:11.570294+0000","last_undegraded":"2024-09-10T08:39:11.570294+0000","last_fullsized":"2024-09-10T08:39:11.570294+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:28.020337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626734+0000","last_change":"2024-09-10T08:36:53.720463+0000","last_active":"2024-09-10T08:38:43.626734+0000","last_peered":"2024-09-10T08:38:43.626734+0000","last_clean":"2024-09-10T08:38:43.626734+0000","last_became_active":"2024-09-10T08:36:53.720080+0000","last_became_peered":"2024-09-10T08:36:53.720080+0000","last_unstale":"2024-09-10T08:38:43.626734+0000","last_undegraded":"2024-09-10T08:38:43.626734+0000","last_fullsized":"2024-09-10T08:38:43.626734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:51.450294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654955+0000","last_change":"2024-09-10T08:37:05.251261+0000","last_active":"2024-09-10T08:38:43.654955+0000","last_peered":"2024-09-10T08:38:43.654955+0000","last_clean":"2024-09-10T08:38:43.654955+0000","last_became_active":"2024-09-10T08:37:05.250938+0000","last_became_peered":"2024-09-10T08:37:05.250938+0000","last_unstale":"2024-09-10T08:38:43.654955+0000","last_undegraded":"2024-09-10T08:38:43.654955+0000","last_fullsized":"2024-09-10T08:38:43.654955+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:07:15.129899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.581822+0000","last_change":"2024-09-10T08:36:56.148191+0000","last_active":"2024-09-10T08:39:11.581822+0000","last_peered":"2024-09-10T08:39:11.581822+0000","last_clean":"2024-09-10T08:39:11.581822+0000","last_became_active":"2024-09-10T08:36:56.147362+0000","last_became_peered":"2024-09-10T08:36:56.147362+0000","last_unstale":"2024-09-10T08:39:11.581822+0000","last_undegraded":"2024-09-10T08:39:11.581822+0000","last_fullsized":"2024-09-10T08:39:11.581822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:25:30.051754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626042+0000","last_change":"2024-09-10T08:36:53.734484+0000","last_active":"2024-09-10T08:38:43.626042+0000","last_peered":"2024-09-10T08:38:43.626042+0000","last_clean":"2024-09-10T08:38:43.626042+0000","last_became_active":"2024-09-10T08:36:53.734172+0000","last_became_peered":"2024-09-10T08:36:53.734172+0000","last_unstale":"2024-09-10T08:38:43.626042+0000","last_undegraded":"2024-09-10T08:38:43.626042+0000","last_fullsized":"2024-09-10T08:38:43.626042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:10.269400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628299+0000","last_change":"2024-09-10T08:37:05.259635+0000","last_active":"2024-09-10T08:38:43.628299+0000","last_peered":"2024-09-10T08:38:43.628299+0000","last_clean":"2024-09-10T08:38:43.628299+0000","last_became_active":"2024-09-10T08:37:05.259107+0000","last_became_peered":"2024-09-10T08:37:05.259107+0000","last_unstale":"2024-09-10T08:38:43.628299+0000","last_undegraded":"2024-09-10T08:38:43.628299+0000","last_fullsized":"2024-09-10T08:38:43.628299+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:36.841138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588992+0000","last_change":"2024-09-10T08:36:56.141122+0000","last_active":"2024-09-10T08:39:11.588992+0000","last_peered":"2024-09-10T08:39:11.588992+0000","last_clean":"2024-09-10T08:39:11.588992+0000","last_became_active":"2024-09-10T08:36:56.140826+0000","last_became_peered":"2024-09-10T08:36:56.140826+0000","last_unstale":"2024-09-10T08:39:11.588992+0000","last_undegraded":"2024-09-10T08:39:11.588992+0000","last_fullsized":"2024-09-10T08:39:11.588992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:38.044090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"96'6","reported_seq":163,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:39.887564+0000","last_change":"2024-09-10T08:36:53.720623+0000","last_active":"2024-09-10T08:39:39.887564+0000","last_peered":"2024-09-10T08:39:39.887564+0000","last_clean":"2024-09-10T08:39:39.887564+0000","last_became_active":"2024-09-10T08:36:53.720270+0000","last_became_peered":"2024-09-10T08:36:53.720270+0000","last_unstale":"2024-09-10T08:39:39.887564+0000","last_undegraded":"2024-09-10T08:39:39.887564+0000","last_fullsized":"2024-09-10T08:39:39.887564+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:19.514068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629937+0000","last_change":"2024-09-10T08:37:05.254324+0000","last_active":"2024-09-10T08:38:43.629937+0000","last_peered":"2024-09-10T08:38:43.629937+0000","last_clean":"2024-09-10T08:38:43.629937+0000","last_became_active":"2024-09-10T08:37:05.253611+0000","last_became_peered":"2024-09-10T08:37:05.253611+0000","last_unstale":"2024-09-10T08:38:43.629937+0000","last_undegraded":"2024-09-10T08:38:43.629937+0000","last_fullsized":"2024-09-10T08:38:43.629937+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:07.182649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.591257+0000","last_change":"2024-09-10T08:36:56.139102+0000","last_active":"2024-09-10T08:39:11.591257+0000","last_peered":"2024-09-10T08:39:11.591257+0000","last_clean":"2024-09-10T08:39:11.591257+0000","last_became_active":"2024-09-10T08:36:56.137986+0000","last_became_peered":"2024-09-10T08:36:56.137986+0000","last_unstale":"2024-09-10T08:39:11.591257+0000","last_undegraded":"2024-09-10T08:39:11.591257+0000","last_fullsized":"2024-09-10T08:39:11.591257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:05:03.005727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655021+0000","last_change":"2024-09-10T08:36:53.722961+0000","last_active":"2024-09-10T08:38:43.655021+0000","last_peered":"2024-09-10T08:38:43.655021+0000","last_clean":"2024-09-10T08:38:43.655021+0000","last_became_active":"2024-09-10T08:36:53.722628+0000","last_became_peered":"2024-09-10T08:36:53.722628+0000","last_unstale":"2024-09-10T08:38:43.655021+0000","last_undegraded":"2024-09-10T08:38:43.655021+0000","last_fullsized":"2024-09-10T08:38:43.655021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:59.048632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628104+0000","last_change":"2024-09-10T08:37:05.256546+0000","last_active":"2024-09-10T08:38:43.628104+0000","last_peered":"2024-09-10T08:38:43.628104+0000","last_clean":"2024-09-10T08:38:43.628104+0000","last_became_active":"2024-09-10T08:37:05.256239+0000","last_became_peered":"2024-09-10T08:37:05.256239+0000","last_unstale":"2024-09-10T08:38:43.628104+0000","last_undegraded":"2024-09-10T08:38:43.628104+0000","last_fullsized":"2024-09-10T08:38:43.628104+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:57.403934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.582848+0000","last_change":"2024-09-10T08:36:56.149285+0000","last_active":"2024-09-10T08:39:11.582848+0000","last_peered":"2024-09-10T08:39:11.582848+0000","last_clean":"2024-09-10T08:39:11.582848+0000","last_became_active":"2024-09-10T08:36:56.146873+0000","last_became_peered":"2024-09-10T08:36:56.146873+0000","last_unstale":"2024-09-10T08:39:11.582848+0000","last_undegraded":"2024-09-10T08:39:11.582848+0000","last_fullsized":"2024-09-10T08:39:11.582848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:28.976729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630368+0000","last_change":"2024-09-10T08:36:53.721174+0000","last_active":"2024-09-10T08:38:43.630368+0000","last_peered":"2024-09-10T08:38:43.630368+0000","last_clean":"2024-09-10T08:38:43.630368+0000","last_became_active":"2024-09-10T08:36:53.720849+0000","last_became_peered":"2024-09-10T08:36:53.720849+0000","last_unstale":"2024-09-10T08:38:43.630368+0000","last_undegraded":"2024-09-10T08:38:43.630368+0000","last_fullsized":"2024-09-10T08:38:43.630368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:39.786848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655286+0000","last_change":"2024-09-10T08:37:05.252256+0000","last_active":"2024-09-10T08:38:43.655286+0000","last_peered":"2024-09-10T08:38:43.655286+0000","last_clean":"2024-09-10T08:38:43.655286+0000","last_became_active":"2024-09-10T08:37:05.251865+0000","last_became_peered":"2024-09-10T08:37:05.251865+0000","last_unstale":"2024-09-10T08:38:43.655286+0000","last_undegraded":"2024-09-10T08:38:43.655286+0000","last_fullsized":"2024-09-10T08:38:43.655286+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:08.581880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567773+0000","last_change":"2024-09-10T08:36:56.142060+0000","last_active":"2024-09-10T08:39:11.567773+0000","last_peered":"2024-09-10T08:39:11.567773+0000","last_clean":"2024-09-10T08:39:11.567773+0000","last_became_active":"2024-09-10T08:36:56.141451+0000","last_became_peered":"2024-09-10T08:36:56.141451+0000","last_unstale":"2024-09-10T08:39:11.567773+0000","last_undegraded":"2024-09-10T08:39:11.567773+0000","last_fullsized":"2024-09-10T08:39:11.567773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:43.566588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'37","reported_seq":310,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:38.877082+0000","last_change":"2024-09-10T08:36:53.719464+0000","last_active":"2024-09-10T08:39:38.877082+0000","last_peered":"2024-09-10T08:39:38.877082+0000","last_clean":"2024-09-10T08:39:38.877082+0000","last_became_active":"2024-09-10T08:36:53.719129+0000","last_became_peered":"2024-09-10T08:36:53.719129+0000","last_unstale":"2024-09-10T08:39:38.877082+0000","last_undegraded":"2024-09-10T08:39:38.877082+0000","last_fullsized":"2024-09-10T08:39:38.877082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:05.652018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64071,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":232,"num_read_kb":179,"num_write":38,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629405+0000","last_change":"2024-09-10T08:37:05.255576+0000","last_active":"2024-09-10T08:38:43.629405+0000","last_peered":"2024-09-10T08:38:43.629405+0000","last_clean":"2024-09-10T08:38:43.629405+0000","last_became_active":"2024-09-10T08:37:05.249842+0000","last_became_peered":"2024-09-10T08:37:05.249842+0000","last_unstale":"2024-09-10T08:38:43.629405+0000","last_undegraded":"2024-09-10T08:38:43.629405+0000","last_fullsized":"2024-09-10T08:38:43.629405+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:47:49.543481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593397+0000","last_change":"2024-09-10T08:36:56.151829+0000","last_active":"2024-09-10T08:39:11.593397+0000","last_peered":"2024-09-10T08:39:11.593397+0000","last_clean":"2024-09-10T08:39:11.593397+0000","last_became_active":"2024-09-10T08:36:56.151314+0000","last_became_peered":"2024-09-10T08:36:56.151314+0000","last_unstale":"2024-09-10T08:39:11.593397+0000","last_undegraded":"2024-09-10T08:39:11.593397+0000","last_fullsized":"2024-09-10T08:39:11.593397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:44.219671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655398+0000","last_change":"2024-09-10T08:36:53.738030+0000","last_active":"2024-09-10T08:38:43.655398+0000","last_peered":"2024-09-10T08:38:43.655398+0000","last_clean":"2024-09-10T08:38:43.655398+0000","last_became_active":"2024-09-10T08:36:53.737658+0000","last_became_peered":"2024-09-10T08:36:53.737658+0000","last_unstale":"2024-09-10T08:38:43.655398+0000","last_undegraded":"2024-09-10T08:38:43.655398+0000","last_fullsized":"2024-09-10T08:38:43.655398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:29.627554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654996+0000","last_change":"2024-09-10T08:37:05.244473+0000","last_active":"2024-09-10T08:38:43.654996+0000","last_peered":"2024-09-10T08:38:43.654996+0000","last_clean":"2024-09-10T08:38:43.654996+0000","last_became_active":"2024-09-10T08:37:05.244035+0000","last_became_peered":"2024-09-10T08:37:05.244035+0000","last_unstale":"2024-09-10T08:38:43.654996+0000","last_undegraded":"2024-09-10T08:38:43.654996+0000","last_fullsized":"2024-09-10T08:38:43.654996+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:04.652660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584024+0000","last_change":"2024-09-10T08:36:56.155217+0000","last_active":"2024-09-10T08:39:11.584024+0000","last_peered":"2024-09-10T08:39:11.584024+0000","last_clean":"2024-09-10T08:39:11.584024+0000","last_became_active":"2024-09-10T08:36:56.154903+0000","last_became_peered":"2024-09-10T08:36:56.154903+0000","last_unstale":"2024-09-10T08:39:11.584024+0000","last_undegraded":"2024-09-10T08:39:11.584024+0000","last_fullsized":"2024-09-10T08:39:11.584024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:47.482974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.632239+0000","last_change":"2024-09-10T08:36:53.721087+0000","last_active":"2024-09-10T08:38:43.632239+0000","last_peered":"2024-09-10T08:38:43.632239+0000","last_clean":"2024-09-10T08:38:43.632239+0000","last_became_active":"2024-09-10T08:36:53.720772+0000","last_became_peered":"2024-09-10T08:36:53.720772+0000","last_unstale":"2024-09-10T08:38:43.632239+0000","last_undegraded":"2024-09-10T08:38:43.632239+0000","last_fullsized":"2024-09-10T08:38:43.632239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:39.560848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629150+0000","last_change":"2024-09-10T08:37:05.254641+0000","last_active":"2024-09-10T08:38:43.629150+0000","last_peered":"2024-09-10T08:38:43.629150+0000","last_clean":"2024-09-10T08:38:43.629150+0000","last_became_active":"2024-09-10T08:37:05.249982+0000","last_became_peered":"2024-09-10T08:37:05.249982+0000","last_unstale":"2024-09-10T08:38:43.629150+0000","last_undegraded":"2024-09-10T08:38:43.629150+0000","last_fullsized":"2024-09-10T08:38:43.629150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:21:13.851061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592510+0000","last_change":"2024-09-10T08:36:56.142657+0000","last_active":"2024-09-10T08:39:11.592510+0000","last_peered":"2024-09-10T08:39:11.592510+0000","last_clean":"2024-09-10T08:39:11.592510+0000","last_became_active":"2024-09-10T08:36:56.142294+0000","last_became_peered":"2024-09-10T08:36:56.142294+0000","last_unstale":"2024-09-10T08:39:11.592510+0000","last_undegraded":"2024-09-10T08:39:11.592510+0000","last_fullsized":"2024-09-10T08:39:11.592510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:52.861389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'4","reported_seq":156,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:37.909693+0000","last_change":"2024-09-10T08:36:53.722466+0000","last_active":"2024-09-10T08:39:37.909693+0000","last_peered":"2024-09-10T08:39:37.909693+0000","last_clean":"2024-09-10T08:39:37.909693+0000","last_became_active":"2024-09-10T08:36:53.722145+0000","last_became_peered":"2024-09-10T08:36:53.722145+0000","last_unstale":"2024-09-10T08:39:37.909693+0000","last_undegraded":"2024-09-10T08:39:37.909693+0000","last_fullsized":"2024-09-10T08:39:37.909693+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:27.041765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631352+0000","last_change":"2024-09-10T08:37:05.251056+0000","last_active":"2024-09-10T08:38:43.631352+0000","last_peered":"2024-09-10T08:38:43.631352+0000","last_clean":"2024-09-10T08:38:43.631352+0000","last_became_active":"2024-09-10T08:37:05.250653+0000","last_became_peered":"2024-09-10T08:37:05.250653+0000","last_unstale":"2024-09-10T08:38:43.631352+0000","last_undegraded":"2024-09-10T08:38:43.631352+0000","last_fullsized":"2024-09-10T08:38:43.631352+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:21.174747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633128+0000","last_change":"2024-09-10T08:37:05.260436+0000","last_active":"2024-09-10T08:38:43.633128+0000","last_peered":"2024-09-10T08:38:43.633128+0000","last_clean":"2024-09-10T08:38:43.633128+0000","last_became_active":"2024-09-10T08:37:05.260049+0000","last_became_peered":"2024-09-10T08:37:05.260049+0000","last_unstale":"2024-09-10T08:38:43.633128+0000","last_undegraded":"2024-09-10T08:38:43.633128+0000","last_fullsized":"2024-09-10T08:38:43.633128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:43.394028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585930+0000","last_change":"2024-09-10T08:36:56.146732+0000","last_active":"2024-09-10T08:39:11.585930+0000","last_peered":"2024-09-10T08:39:11.585930+0000","last_clean":"2024-09-10T08:39:11.585930+0000","last_became_active":"2024-09-10T08:36:56.145599+0000","last_became_peered":"2024-09-10T08:36:56.145599+0000","last_unstale":"2024-09-10T08:39:11.585930+0000","last_undegraded":"2024-09-10T08:39:11.585930+0000","last_fullsized":"2024-09-10T08:39:11.585930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:50.620956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655135+0000","last_change":"2024-09-10T08:36:53.721683+0000","last_active":"2024-09-10T08:38:43.655135+0000","last_peered":"2024-09-10T08:38:43.655135+0000","last_clean":"2024-09-10T08:38:43.655135+0000","last_became_active":"2024-09-10T08:36:53.721241+0000","last_became_peered":"2024-09-10T08:36:53.721241+0000","last_unstale":"2024-09-10T08:38:43.655135+0000","last_undegraded":"2024-09-10T08:38:43.655135+0000","last_fullsized":"2024-09-10T08:38:43.655135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:03.669337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628275+0000","last_change":"2024-09-10T08:37:42.669784+0000","last_active":"2024-09-10T08:38:43.628275+0000","last_peered":"2024-09-10T08:38:43.628275+0000","last_clean":"2024-09-10T08:38:43.628275+0000","last_became_active":"2024-09-10T08:37:42.669322+0000","last_became_peered":"2024-09-10T08:37:42.669322+0000","last_unstale":"2024-09-10T08:38:43.628275+0000","last_undegraded":"2024-09-10T08:38:43.628275+0000","last_fullsized":"2024-09-10T08:38:43.628275+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:06:57.027928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.574617+0000","last_change":"2024-09-10T08:36:56.143824+0000","last_active":"2024-09-10T08:39:11.574617+0000","last_peered":"2024-09-10T08:39:11.574617+0000","last_clean":"2024-09-10T08:39:11.574617+0000","last_became_active":"2024-09-10T08:36:56.143486+0000","last_became_peered":"2024-09-10T08:36:56.143486+0000","last_unstale":"2024-09-10T08:39:11.574617+0000","last_undegraded":"2024-09-10T08:39:11.574617+0000","last_fullsized":"2024-09-10T08:39:11.574617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:42.384863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627976+0000","last_change":"2024-09-10T08:36:53.717680+0000","last_active":"2024-09-10T08:38:43.627976+0000","last_peered":"2024-09-10T08:38:43.627976+0000","last_clean":"2024-09-10T08:38:43.627976+0000","last_became_active":"2024-09-10T08:36:53.717249+0000","last_became_peered":"2024-09-10T08:36:53.717249+0000","last_unstale":"2024-09-10T08:38:43.627976+0000","last_undegraded":"2024-09-10T08:38:43.627976+0000","last_fullsized":"2024-09-10T08:38:43.627976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:12.191541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627200+0000","last_change":"2024-09-10T08:36:53.723947+0000","last_active":"2024-09-10T08:38:43.627200+0000","last_peered":"2024-09-10T08:38:43.627200+0000","last_clean":"2024-09-10T08:38:43.627200+0000","last_became_active":"2024-09-10T08:36:53.723620+0000","last_became_peered":"2024-09-10T08:36:53.723620+0000","last_unstale":"2024-09-10T08:38:43.627200+0000","last_undegraded":"2024-09-10T08:38:43.627200+0000","last_fullsized":"2024-09-10T08:38:43.627200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:00.135349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583145+0000","last_change":"2024-09-10T08:36:56.150448+0000","last_active":"2024-09-10T08:39:11.583145+0000","last_peered":"2024-09-10T08:39:11.583145+0000","last_clean":"2024-09-10T08:39:11.583145+0000","last_became_active":"2024-09-10T08:36:56.150148+0000","last_became_peered":"2024-09-10T08:36:56.150148+0000","last_unstale":"2024-09-10T08:39:11.583145+0000","last_undegraded":"2024-09-10T08:39:11.583145+0000","last_fullsized":"2024-09-10T08:39:11.583145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:02.927753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630191+0000","last_change":"2024-09-10T08:37:05.242878+0000","last_active":"2024-09-10T08:38:43.630191+0000","last_peered":"2024-09-10T08:38:43.630191+0000","last_clean":"2024-09-10T08:38:43.630191+0000","last_became_active":"2024-09-10T08:37:05.242440+0000","last_became_peered":"2024-09-10T08:37:05.242440+0000","last_unstale":"2024-09-10T08:38:43.630191+0000","last_undegraded":"2024-09-10T08:38:43.630191+0000","last_fullsized":"2024-09-10T08:38:43.630191+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:30.957433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'7","reported_seq":163,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:39.878942+0000","last_change":"2024-09-10T08:36:53.719788+0000","last_active":"2024-09-10T08:39:39.878942+0000","last_peered":"2024-09-10T08:39:39.878942+0000","last_clean":"2024-09-10T08:39:39.878942+0000","last_became_active":"2024-09-10T08:36:53.719462+0000","last_became_peered":"2024-09-10T08:36:53.719462+0000","last_unstale":"2024-09-10T08:39:39.878942+0000","last_undegraded":"2024-09-10T08:39:39.878942+0000","last_fullsized":"2024-09-10T08:39:39.878942+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:12.250447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1326,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583170+0000","last_change":"2024-09-10T08:37:42.673008+0000","last_active":"2024-09-10T08:39:11.583170+0000","last_peered":"2024-09-10T08:39:11.583170+0000","last_clean":"2024-09-10T08:39:11.583170+0000","last_became_active":"2024-09-10T08:37:42.672618+0000","last_became_peered":"2024-09-10T08:37:42.672618+0000","last_unstale":"2024-09-10T08:39:11.583170+0000","last_undegraded":"2024-09-10T08:39:11.583170+0000","last_fullsized":"2024-09-10T08:39:11.583170+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:00.156565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630783+0000","last_change":"2024-09-10T08:37:05.244277+0000","last_active":"2024-09-10T08:38:43.630783+0000","last_peered":"2024-09-10T08:38:43.630783+0000","last_clean":"2024-09-10T08:38:43.630783+0000","last_became_active":"2024-09-10T08:37:05.243880+0000","last_became_peered":"2024-09-10T08:37:05.243880+0000","last_unstale":"2024-09-10T08:38:43.630783+0000","last_undegraded":"2024-09-10T08:38:43.630783+0000","last_fullsized":"2024-09-10T08:38:43.630783+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:55.275857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.573131+0000","last_change":"2024-09-10T08:36:56.141404+0000","last_active":"2024-09-10T08:39:11.573131+0000","last_peered":"2024-09-10T08:39:11.573131+0000","last_clean":"2024-09-10T08:39:11.573131+0000","last_became_active":"2024-09-10T08:36:56.141076+0000","last_became_peered":"2024-09-10T08:36:56.141076+0000","last_unstale":"2024-09-10T08:39:11.573131+0000","last_undegraded":"2024-09-10T08:39:11.573131+0000","last_fullsized":"2024-09-10T08:39:11.573131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:03:52.915654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.569383+0000","last_change":"2024-09-10T08:36:56.148285+0000","last_active":"2024-09-10T08:39:11.569383+0000","last_peered":"2024-09-10T08:39:11.569383+0000","last_clean":"2024-09-10T08:39:11.569383+0000","last_became_active":"2024-09-10T08:36:56.147970+0000","last_became_peered":"2024-09-10T08:36:56.147970+0000","last_unstale":"2024-09-10T08:39:11.569383+0000","last_undegraded":"2024-09-10T08:39:11.569383+0000","last_fullsized":"2024-09-10T08:39:11.569383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:09.506718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.571047+0000","last_change":"2024-09-10T08:36:56.157977+0000","last_active":"2024-09-10T08:39:11.571047+0000","last_peered":"2024-09-10T08:39:11.571047+0000","last_clean":"2024-09-10T08:39:11.571047+0000","last_became_active":"2024-09-10T08:36:56.157623+0000","last_became_peered":"2024-09-10T08:36:56.157623+0000","last_unstale":"2024-09-10T08:39:11.571047+0000","last_undegraded":"2024-09-10T08:39:11.571047+0000","last_fullsized":"2024-09-10T08:39:11.571047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:44.617979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584899+0000","last_change":"2024-09-10T08:37:42.670459+0000","last_active":"2024-09-10T08:39:11.584899+0000","last_peered":"2024-09-10T08:39:11.584899+0000","last_clean":"2024-09-10T08:39:11.584899+0000","last_became_active":"2024-09-10T08:37:42.670097+0000","last_became_peered":"2024-09-10T08:37:42.670097+0000","last_unstale":"2024-09-10T08:39:11.584899+0000","last_undegraded":"2024-09-10T08:39:11.584899+0000","last_fullsized":"2024-09-10T08:39:11.584899+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:14.956492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584527+0000","last_change":"2024-09-10T08:36:56.154850+0000","last_active":"2024-09-10T08:39:11.584527+0000","last_peered":"2024-09-10T08:39:11.584527+0000","last_clean":"2024-09-10T08:39:11.584527+0000","last_became_active":"2024-09-10T08:36:56.154545+0000","last_became_peered":"2024-09-10T08:36:56.154545+0000","last_unstale":"2024-09-10T08:39:11.584527+0000","last_undegraded":"2024-09-10T08:39:11.584527+0000","last_fullsized":"2024-09-10T08:39:11.584527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:24.639667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.569973+0000","last_change":"2024-09-10T08:37:42.669962+0000","last_active":"2024-09-10T08:39:11.569973+0000","last_peered":"2024-09-10T08:39:11.569973+0000","last_clean":"2024-09-10T08:39:11.569973+0000","last_became_active":"2024-09-10T08:37:42.669556+0000","last_became_peered":"2024-09-10T08:37:42.669556+0000","last_unstale":"2024-09-10T08:39:11.569973+0000","last_undegraded":"2024-09-10T08:39:11.569973+0000","last_fullsized":"2024-09-10T08:39:11.569973+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:23.333946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.578061+0000","last_change":"2024-09-10T08:36:56.145910+0000","last_active":"2024-09-10T08:39:11.578061+0000","last_peered":"2024-09-10T08:39:11.578061+0000","last_clean":"2024-09-10T08:39:11.578061+0000","last_became_active":"2024-09-10T08:36:56.145537+0000","last_became_peered":"2024-09-10T08:36:56.145537+0000","last_unstale":"2024-09-10T08:39:11.578061+0000","last_undegraded":"2024-09-10T08:39:11.578061+0000","last_fullsized":"2024-09-10T08:39:11.578061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:46.788241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567880+0000","last_change":"2024-09-10T08:36:56.147340+0000","last_active":"2024-09-10T08:39:11.567880+0000","last_peered":"2024-09-10T08:39:11.567880+0000","last_clean":"2024-09-10T08:39:11.567880+0000","last_became_active":"2024-09-10T08:36:56.145762+0000","last_became_peered":"2024-09-10T08:36:56.145762+0000","last_unstale":"2024-09-10T08:39:11.567880+0000","last_undegraded":"2024-09-10T08:39:11.567880+0000","last_fullsized":"2024-09-10T08:39:11.567880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:12.339285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.581431+0000","last_change":"2024-09-10T08:36:56.145213+0000","last_active":"2024-09-10T08:39:11.581431+0000","last_peered":"2024-09-10T08:39:11.581431+0000","last_clean":"2024-09-10T08:39:11.581431+0000","last_became_active":"2024-09-10T08:36:56.144915+0000","last_became_peered":"2024-09-10T08:36:56.144915+0000","last_unstale":"2024-09-10T08:39:11.581431+0000","last_undegraded":"2024-09-10T08:39:11.581431+0000","last_fullsized":"2024-09-10T08:39:11.581431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:06.180321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.574743+0000","last_change":"2024-09-10T08:36:56.149738+0000","last_active":"2024-09-10T08:39:11.574743+0000","last_peered":"2024-09-10T08:39:11.574743+0000","last_clean":"2024-09-10T08:39:11.574743+0000","last_became_active":"2024-09-10T08:36:56.147750+0000","last_became_peered":"2024-09-10T08:36:56.147750+0000","last_unstale":"2024-09-10T08:39:11.574743+0000","last_undegraded":"2024-09-10T08:39:11.574743+0000","last_fullsized":"2024-09-10T08:39:11.574743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:47.950054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.591937+0000","last_change":"2024-09-10T08:36:56.147682+0000","last_active":"2024-09-10T08:39:11.591937+0000","last_peered":"2024-09-10T08:39:11.591937+0000","last_clean":"2024-09-10T08:39:11.591937+0000","last_became_active":"2024-09-10T08:36:56.147345+0000","last_became_peered":"2024-09-10T08:36:56.147345+0000","last_unstale":"2024-09-10T08:39:11.591937+0000","last_undegraded":"2024-09-10T08:39:11.591937+0000","last_fullsized":"2024-09-10T08:39:11.591937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:54.939482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583378+0000","last_change":"2024-09-10T08:36:56.158292+0000","last_active":"2024-09-10T08:39:11.583378+0000","last_peered":"2024-09-10T08:39:11.583378+0000","last_clean":"2024-09-10T08:39:11.583378+0000","last_became_active":"2024-09-10T08:36:56.157970+0000","last_became_peered":"2024-09-10T08:36:56.157970+0000","last_unstale":"2024-09-10T08:39:11.583378+0000","last_undegraded":"2024-09-10T08:39:11.583378+0000","last_fullsized":"2024-09-10T08:39:11.583378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:33.582126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567916+0000","last_change":"2024-09-10T08:36:56.142520+0000","last_active":"2024-09-10T08:39:11.567916+0000","last_peered":"2024-09-10T08:39:11.567916+0000","last_clean":"2024-09-10T08:39:11.567916+0000","last_became_active":"2024-09-10T08:36:56.142214+0000","last_became_peered":"2024-09-10T08:36:56.142214+0000","last_unstale":"2024-09-10T08:39:11.567916+0000","last_undegraded":"2024-09-10T08:39:11.567916+0000","last_fullsized":"2024-09-10T08:39:11.567916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:45:29.201839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584167+0000","last_change":"2024-09-10T08:36:56.149547+0000","last_active":"2024-09-10T08:39:11.584167+0000","last_peered":"2024-09-10T08:39:11.584167+0000","last_clean":"2024-09-10T08:39:11.584167+0000","last_became_active":"2024-09-10T08:36:56.149141+0000","last_became_peered":"2024-09-10T08:36:56.149141+0000","last_unstale":"2024-09-10T08:39:11.584167+0000","last_undegraded":"2024-09-10T08:39:11.584167+0000","last_fullsized":"2024-09-10T08:39:11.584167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:49.368677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585864+0000","last_change":"2024-09-10T08:36:56.143087+0000","last_active":"2024-09-10T08:39:11.585864+0000","last_peered":"2024-09-10T08:39:11.585864+0000","last_clean":"2024-09-10T08:39:11.585864+0000","last_became_active":"2024-09-10T08:36:56.142782+0000","last_became_peered":"2024-09-10T08:36:56.142782+0000","last_unstale":"2024-09-10T08:39:11.585864+0000","last_undegraded":"2024-09-10T08:39:11.585864+0000","last_fullsized":"2024-09-10T08:39:11.585864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:17.513103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592876+0000","last_change":"2024-09-10T08:36:56.157549+0000","last_active":"2024-09-10T08:39:11.592876+0000","last_peered":"2024-09-10T08:39:11.592876+0000","last_clean":"2024-09-10T08:39:11.592876+0000","last_became_active":"2024-09-10T08:36:56.157221+0000","last_became_peered":"2024-09-10T08:36:56.157221+0000","last_unstale":"2024-09-10T08:39:11.592876+0000","last_undegraded":"2024-09-10T08:39:11.592876+0000","last_fullsized":"2024-09-10T08:39:11.592876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:11.372085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593199+0000","last_change":"2024-09-10T08:36:56.145654+0000","last_active":"2024-09-10T08:39:11.593199+0000","last_peered":"2024-09-10T08:39:11.593199+0000","last_clean":"2024-09-10T08:39:11.593199+0000","last_became_active":"2024-09-10T08:36:56.144126+0000","last_became_peered":"2024-09-10T08:36:56.144126+0000","last_unstale":"2024-09-10T08:39:11.593199+0000","last_undegraded":"2024-09-10T08:39:11.593199+0000","last_fullsized":"2024-09-10T08:39:11.593199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:55.498873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.582479+0000","last_change":"2024-09-10T08:36:56.146423+0000","last_active":"2024-09-10T08:39:11.582479+0000","last_peered":"2024-09-10T08:39:11.582479+0000","last_clean":"2024-09-10T08:39:11.582479+0000","last_became_active":"2024-09-10T08:36:56.146126+0000","last_became_peered":"2024-09-10T08:36:56.146126+0000","last_unstale":"2024-09-10T08:39:11.582479+0000","last_undegraded":"2024-09-10T08:39:11.582479+0000","last_fullsized":"2024-09-10T08:39:11.582479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:44.665196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583332+0000","last_change":"2024-09-10T08:36:56.151973+0000","last_active":"2024-09-10T08:39:11.583332+0000","last_peered":"2024-09-10T08:39:11.583332+0000","last_clean":"2024-09-10T08:39:11.583332+0000","last_became_active":"2024-09-10T08:36:56.151609+0000","last_became_peered":"2024-09-10T08:36:56.151609+0000","last_unstale":"2024-09-10T08:39:11.583332+0000","last_undegraded":"2024-09-10T08:39:11.583332+0000","last_fullsized":"2024-09-10T08:39:11.583332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:02.584047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584509+0000","last_change":"2024-09-10T08:36:56.141780+0000","last_active":"2024-09-10T08:39:11.584509+0000","last_peered":"2024-09-10T08:39:11.584509+0000","last_clean":"2024-09-10T08:39:11.584509+0000","last_became_active":"2024-09-10T08:36:56.141228+0000","last_became_peered":"2024-09-10T08:36:56.141228+0000","last_unstale":"2024-09-10T08:39:11.584509+0000","last_undegraded":"2024-09-10T08:39:11.584509+0000","last_fullsized":"2024-09-10T08:39:11.584509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:52:04.374228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585477+0000","last_change":"2024-09-10T08:37:42.671346+0000","last_active":"2024-09-10T08:39:11.585477+0000","last_peered":"2024-09-10T08:39:11.585477+0000","last_clean":"2024-09-10T08:39:11.585477+0000","last_became_active":"2024-09-10T08:37:42.670895+0000","last_became_peered":"2024-09-10T08:37:42.670895+0000","last_unstale":"2024-09-10T08:39:11.585477+0000","last_undegraded":"2024-09-10T08:39:11.585477+0000","last_fullsized":"2024-09-10T08:39:11.585477+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:16:31.930628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593796+0000","last_change":"2024-09-10T08:36:56.149014+0000","last_active":"2024-09-10T08:39:11.593796+0000","last_peered":"2024-09-10T08:39:11.593796+0000","last_clean":"2024-09-10T08:39:11.593796+0000","last_became_active":"2024-09-10T08:36:56.148551+0000","last_became_peered":"2024-09-10T08:36:56.148551+0000","last_unstale":"2024-09-10T08:39:11.593796+0000","last_undegraded":"2024-09-10T08:39:11.593796+0000","last_fullsized":"2024-09-10T08:39:11.593796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:35:52.250943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.586709+0000","last_change":"2024-09-10T08:36:56.141247+0000","last_active":"2024-09-10T08:39:11.586709+0000","last_peered":"2024-09-10T08:39:11.586709+0000","last_clean":"2024-09-10T08:39:11.586709+0000","last_became_active":"2024-09-10T08:36:56.140956+0000","last_became_peered":"2024-09-10T08:36:56.140956+0000","last_unstale":"2024-09-10T08:39:11.586709+0000","last_undegraded":"2024-09-10T08:39:11.586709+0000","last_fullsized":"2024-09-10T08:39:11.586709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:45.008424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592783+0000","last_change":"2024-09-10T08:36:56.149116+0000","last_active":"2024-09-10T08:39:11.592783+0000","last_peered":"2024-09-10T08:39:11.592783+0000","last_clean":"2024-09-10T08:39:11.592783+0000","last_became_active":"2024-09-10T08:36:56.148783+0000","last_became_peered":"2024-09-10T08:36:56.148783+0000","last_unstale":"2024-09-10T08:39:11.592783+0000","last_undegraded":"2024-09-10T08:39:11.592783+0000","last_fullsized":"2024-09-10T08:39:11.592783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:37.119214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585754+0000","last_change":"2024-09-10T08:37:42.846322+0000","last_active":"2024-09-10T08:39:11.585754+0000","last_peered":"2024-09-10T08:39:11.585754+0000","last_clean":"2024-09-10T08:39:11.585754+0000","last_became_active":"2024-09-10T08:37:42.846022+0000","last_became_peered":"2024-09-10T08:37:42.846022+0000","last_unstale":"2024-09-10T08:39:11.585754+0000","last_undegraded":"2024-09-10T08:39:11.585754+0000","last_fullsized":"2024-09-10T08:39:11.585754+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:38.142362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588751+0000","last_change":"2024-09-10T08:36:56.148537+0000","last_active":"2024-09-10T08:39:11.588751+0000","last_peered":"2024-09-10T08:39:11.588751+0000","last_clean":"2024-09-10T08:39:11.588751+0000","last_became_active":"2024-09-10T08:36:56.148185+0000","last_became_peered":"2024-09-10T08:36:56.148185+0000","last_unstale":"2024-09-10T08:39:11.588751+0000","last_undegraded":"2024-09-10T08:39:11.588751+0000","last_fullsized":"2024-09-10T08:39:11.588751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:00.426513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.586233+0000","last_change":"2024-09-10T08:36:56.138513+0000","last_active":"2024-09-10T08:39:11.586233+0000","last_peered":"2024-09-10T08:39:11.586233+0000","last_clean":"2024-09-10T08:39:11.586233+0000","last_became_active":"2024-09-10T08:36:56.138207+0000","last_became_peered":"2024-09-10T08:36:56.138207+0000","last_unstale":"2024-09-10T08:39:11.586233+0000","last_undegraded":"2024-09-10T08:39:11.586233+0000","last_fullsized":"2024-09-10T08:39:11.586233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:34.581430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.582686+0000","last_change":"2024-09-10T08:36:56.146612+0000","last_active":"2024-09-10T08:39:11.582686+0000","last_peered":"2024-09-10T08:39:11.582686+0000","last_clean":"2024-09-10T08:39:11.582686+0000","last_became_active":"2024-09-10T08:36:56.146251+0000","last_became_peered":"2024-09-10T08:36:56.146251+0000","last_unstale":"2024-09-10T08:39:11.582686+0000","last_undegraded":"2024-09-10T08:39:11.582686+0000","last_fullsized":"2024-09-10T08:39:11.582686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:47.336389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567939+0000","last_change":"2024-09-10T08:36:56.148260+0000","last_active":"2024-09-10T08:39:11.567939+0000","last_peered":"2024-09-10T08:39:11.567939+0000","last_clean":"2024-09-10T08:39:11.567939+0000","last_became_active":"2024-09-10T08:36:56.147949+0000","last_became_peered":"2024-09-10T08:36:56.147949+0000","last_unstale":"2024-09-10T08:39:11.567939+0000","last_undegraded":"2024-09-10T08:39:11.567939+0000","last_fullsized":"2024-09-10T08:39:11.567939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:52:19.762297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592986+0000","last_change":"2024-09-10T08:36:56.156045+0000","last_active":"2024-09-10T08:39:11.592986+0000","last_peered":"2024-09-10T08:39:11.592986+0000","last_clean":"2024-09-10T08:39:11.592986+0000","last_became_active":"2024-09-10T08:36:56.154727+0000","last_became_peered":"2024-09-10T08:36:56.154727+0000","last_unstale":"2024-09-10T08:39:11.592986+0000","last_undegraded":"2024-09-10T08:39:11.592986+0000","last_fullsized":"2024-09-10T08:39:11.592986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:35:39.992683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.576067+0000","last_change":"2024-09-10T08:36:56.154394+0000","last_active":"2024-09-10T08:39:11.576067+0000","last_peered":"2024-09-10T08:39:11.576067+0000","last_clean":"2024-09-10T08:39:11.576067+0000","last_became_active":"2024-09-10T08:36:56.153855+0000","last_became_peered":"2024-09-10T08:36:56.153855+0000","last_unstale":"2024-09-10T08:39:11.576067+0000","last_undegraded":"2024-09-10T08:39:11.576067+0000","last_fullsized":"2024-09-10T08:39:11.576067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:14.987147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592634+0000","last_change":"2024-09-10T08:36:56.157517+0000","last_active":"2024-09-10T08:39:11.592634+0000","last_peered":"2024-09-10T08:39:11.592634+0000","last_clean":"2024-09-10T08:39:11.592634+0000","last_became_active":"2024-09-10T08:36:56.157133+0000","last_became_peered":"2024-09-10T08:36:56.157133+0000","last_unstale":"2024-09-10T08:39:11.592634+0000","last_undegraded":"2024-09-10T08:39:11.592634+0000","last_fullsized":"2024-09-10T08:39:11.592634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:17:29.573719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69312,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":699,"num_read_kb":422,"num_write":186,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":216957,"data_compressed":5667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282455,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416432,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":547326,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678068,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28204,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716948,"statfs":{"total":95995035648,"available":95966154752,"internally_reserved":0,"allocated":1208320,"data_stored":1207391,"data_compressed":67339,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841594,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":543466,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103232,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":546900,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332165,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":1144631,"data_compressed":65450,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495690,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":542738,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757328,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":1138899,"data_compressed":65450,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920852,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":548897,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149785,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":542932,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":607798,"data_compressed":31413,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1126400,"data_stored":612008,"data_compressed":31413,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1077248,"data_stored":544126,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69360,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":70250,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":66183,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:39:44.520 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T08:39:44.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.675+0000 7f587be00640 1 -- 172.21.15.89:0/1026346093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587c101970 msgr2=0x7f587c101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:44.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.675+0000 7f587be00640 1 --2- 172.21.15.89:0/1026346093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587c101970 0x7f587c101d70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f586c009900 tx=0x7f586c02a410 comp rx=0 tx=0).stop 2024-09-10T08:39:44.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.675+0000 7f587be00640 1 -- 172.21.15.89:0/1026346093 shutdown_connections 2024-09-10T08:39:44.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.675+0000 7f587be00640 1 --2- 172.21.15.89:0/1026346093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f587c103520 0x7f587c109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.675+0000 7f587be00640 1 --2- 172.21.15.89:0/1026346093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f587c102b60 0x7f587c102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.675+0000 7f587be00640 1 --2- 172.21.15.89:0/1026346093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587c101970 0x7f587c101d70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.675+0000 7f587be00640 1 -- 172.21.15.89:0/1026346093 >> 172.21.15.89:0/1026346093 conn(0x7f587c0fd0d0 msgr2=0x7f587c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:44.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.675+0000 7f587be00640 1 -- 172.21.15.89:0/1026346093 shutdown_connections 2024-09-10T08:39:44.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.675+0000 7f587be00640 1 -- 172.21.15.89:0/1026346093 wait complete. 2024-09-10T08:39:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.676+0000 7f587be00640 1 Processor -- start 2024-09-10T08:39:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.676+0000 7f587be00640 1 -- start start 2024-09-10T08:39:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.676+0000 7f587be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f587c101970 0x7f587c197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.677+0000 7f587be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f587c102b60 0x7f587c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.677+0000 7f587be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587c103520 0x7f587c19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.677+0000 7f587be00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f587c10d620 con 0x7f587c103520 2024-09-10T08:39:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.677+0000 7f587be00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f587c10d4a0 con 0x7f587c102b60 2024-09-10T08:39:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.677+0000 7f587be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f587c10d7a0 con 0x7f587c101970 2024-09-10T08:39:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.677+0000 7f587b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587c103520 0x7f587c19f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.677+0000 7f587b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587c103520 0x7f587c19f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59654/0 (socket says 172.21.15.89:59654) 2024-09-10T08:39:44.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.677+0000 7f587b400640 1 -- 172.21.15.89:0/992157527 learned_addr learned my addr 172.21.15.89:0/992157527 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.677+0000 7f587a000640 1 --2- 172.21.15.89:0/992157527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f587c102b60 0x7f587c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.677+0000 7f587aa00640 1 --2- 172.21.15.89:0/992157527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f587c101970 0x7f587c197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.678+0000 7f587b400640 1 -- 172.21.15.89:0/992157527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f587c101970 msgr2=0x7f587c197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.678+0000 7f587b400640 1 --2- 172.21.15.89:0/992157527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f587c101970 0x7f587c197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.678+0000 7f587b400640 1 -- 172.21.15.89:0/992157527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f587c102b60 msgr2=0x7f587c198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.678+0000 7f587b400640 1 --2- 172.21.15.89:0/992157527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f587c102b60 0x7f587c198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.678+0000 7f587b400640 1 -- 172.21.15.89:0/992157527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c19f900 con 0x7f587c103520 2024-09-10T08:39:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.678+0000 7f587a000640 1 --2- 172.21.15.89:0/992157527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f587c102b60 0x7f587c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.678+0000 7f587aa00640 1 --2- 172.21.15.89:0/992157527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f587c101970 0x7f587c197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.678+0000 7f587b400640 1 --2- 172.21.15.89:0/992157527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587c103520 0x7f587c19f260 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f587400eb10 tx=0x7f587400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.678+0000 7f5873400640 1 -- 172.21.15.89:0/992157527 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874004b20 con 0x7f587c103520 2024-09-10T08:39:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.678+0000 7f5873400640 1 -- 172.21.15.89:0/992157527 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5874004cc0 con 0x7f587c103520 2024-09-10T08:39:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.678+0000 7f587be00640 1 -- 172.21.15.89:0/992157527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587c19fbf0 con 0x7f587c103520 2024-09-10T08:39:44.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.678+0000 7f5873400640 1 -- 172.21.15.89:0/992157527 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874010760 con 0x7f587c103520 2024-09-10T08:39:44.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.678+0000 7f587be00640 1 -- 172.21.15.89:0/992157527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587c1a0110 con 0x7f587c103520 2024-09-10T08:39:44.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.679+0000 7f587be00640 1 -- 172.21.15.89:0/992157527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587c108db0 con 0x7f587c103520 2024-09-10T08:39:44.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.681+0000 7f5873400640 1 -- 172.21.15.89:0/992157527 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f5874005020 con 0x7f587c103520 2024-09-10T08:39:44.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.682+0000 7f5873400640 1 --2- 172.21.15.89:0/992157527 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f58600a2ca0 0x7f58600a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:44.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.682+0000 7f5873400640 1 -- 172.21.15.89:0/992157527 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f58740cfb60 con 0x7f587c103520 2024-09-10T08:39:44.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.683+0000 7f587aa00640 1 --2- 172.21.15.89:0/992157527 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f58600a2ca0 0x7f58600a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:44.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.683+0000 7f587aa00640 1 --2- 172.21.15.89:0/992157527 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f58600a2ca0 0x7f58600a5160 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f586c02aa00 tx=0x7f586c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:44.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.686+0000 7f5873400640 1 -- 172.21.15.89:0/992157527 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f587409c590 con 0x7f587c103520 2024-09-10T08:39:44.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.859+0000 7f587be00640 1 -- 172.21.15.89:0/992157527 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f587c062630 con 0x7f58600a2ca0 2024-09-10T08:39:44.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.867+0000 7f5873400640 1 -- 172.21.15.89:0/992157527 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340807 (secure 0 0 0) 0x7f587c062630 con 0x7f58600a2ca0 2024-09-10T08:39:44.871 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:39:44.873 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-09-10T08:39:44.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.872+0000 7f587be00640 1 -- 172.21.15.89:0/992157527 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f58600a2ca0 msgr2=0x7f58600a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:44.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.872+0000 7f587be00640 1 --2- 172.21.15.89:0/992157527 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f58600a2ca0 0x7f58600a5160 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f586c02aa00 tx=0x7f586c0315b0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.872+0000 7f587be00640 1 -- 172.21.15.89:0/992157527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587c103520 msgr2=0x7f587c19f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:44.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.872+0000 7f587be00640 1 --2- 172.21.15.89:0/992157527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587c103520 0x7f587c19f260 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f587400eb10 tx=0x7f587400c2d0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.872+0000 7f587be00640 1 -- 172.21.15.89:0/992157527 shutdown_connections 2024-09-10T08:39:44.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.872+0000 7f587be00640 1 --2- 172.21.15.89:0/992157527 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f58600a2ca0 0x7f58600a5160 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.872+0000 7f587be00640 1 --2- 172.21.15.89:0/992157527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f587c103520 0x7f587c19f260 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.872+0000 7f587be00640 1 --2- 172.21.15.89:0/992157527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f587c102b60 0x7f587c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.872+0000 7f587be00640 1 --2- 172.21.15.89:0/992157527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f587c101970 0x7f587c197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:44.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.872+0000 7f587be00640 1 -- 172.21.15.89:0/992157527 >> 172.21.15.89:0/992157527 conn(0x7f587c0fd0d0 msgr2=0x7f587c105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:44.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.872+0000 7f587be00640 1 -- 172.21.15.89:0/992157527 shutdown_connections 2024-09-10T08:39:44.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:44.872+0000 7f587be00640 1 -- 172.21.15.89:0/992157527 wait complete. 2024-09-10T08:39:44.888 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-09-10T08:39:43.612696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659914,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":825,"num_read_kb":531,"num_write":369,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181900,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":797,"ondisk_log_size":797,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357964,"kb_used_data":12876,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583860,"statfs":{"total":1151940427776,"available":1151573872640,"internally_reserved":0,"allocated":13185024,"data_stored":8527112,"data_compressed":467733,"data_compressed_allocated":638976,"data_compressed_original":2777088,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1408,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":32,"num_write":15,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003513"},"pg_stats":[{"pgid":"2.1f","version":"94'9","reported_seq":67,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.663591+0000","last_change":"2024-09-10T08:38:43.663079+0000","last_active":"2024-09-10T08:38:43.663591+0000","last_peered":"2024-09-10T08:38:43.663591+0000","last_clean":"2024-09-10T08:38:43.663591+0000","last_became_active":"2024-09-10T08:38:43.656506+0000","last_became_peered":"2024-09-10T08:38:43.656506+0000","last_unstale":"2024-09-10T08:38:43.663591+0000","last_undegraded":"2024-09-10T08:38:43.663591+0000","last_fullsized":"2024-09-10T08:38:43.663591+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:36.881564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.579344+0000","last_change":"2024-09-10T08:36:56.145816+0000","last_active":"2024-09-10T08:39:11.579344+0000","last_peered":"2024-09-10T08:39:11.579344+0000","last_clean":"2024-09-10T08:39:11.579344+0000","last_became_active":"2024-09-10T08:36:56.145462+0000","last_became_peered":"2024-09-10T08:36:56.145462+0000","last_unstale":"2024-09-10T08:39:11.579344+0000","last_undegraded":"2024-09-10T08:39:11.579344+0000","last_fullsized":"2024-09-10T08:39:11.579344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:53.890190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629879+0000","last_change":"2024-09-10T08:37:05.252761+0000","last_active":"2024-09-10T08:38:43.629879+0000","last_peered":"2024-09-10T08:38:43.629879+0000","last_clean":"2024-09-10T08:38:43.629879+0000","last_became_active":"2024-09-10T08:37:05.252179+0000","last_became_peered":"2024-09-10T08:37:05.252179+0000","last_unstale":"2024-09-10T08:38:43.629879+0000","last_undegraded":"2024-09-10T08:38:43.629879+0000","last_fullsized":"2024-09-10T08:38:43.629879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:52.854828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631850+0000","last_change":"2024-09-10T08:36:53.735650+0000","last_active":"2024-09-10T08:38:43.631850+0000","last_peered":"2024-09-10T08:38:43.631850+0000","last_clean":"2024-09-10T08:38:43.631850+0000","last_became_active":"2024-09-10T08:36:53.735128+0000","last_became_peered":"2024-09-10T08:36:53.735128+0000","last_unstale":"2024-09-10T08:38:43.631850+0000","last_undegraded":"2024-09-10T08:38:43.631850+0000","last_fullsized":"2024-09-10T08:38:43.631850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:27.309591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592923+0000","last_change":"2024-09-10T08:36:56.153364+0000","last_active":"2024-09-10T08:39:11.592923+0000","last_peered":"2024-09-10T08:39:11.592923+0000","last_clean":"2024-09-10T08:39:11.592923+0000","last_became_active":"2024-09-10T08:36:56.152979+0000","last_became_peered":"2024-09-10T08:36:56.152979+0000","last_unstale":"2024-09-10T08:39:11.592923+0000","last_undegraded":"2024-09-10T08:39:11.592923+0000","last_fullsized":"2024-09-10T08:39:11.592923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:12.172620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654774+0000","last_change":"2024-09-10T08:37:05.254551+0000","last_active":"2024-09-10T08:38:43.654774+0000","last_peered":"2024-09-10T08:38:43.654774+0000","last_clean":"2024-09-10T08:38:43.654774+0000","last_became_active":"2024-09-10T08:37:05.254055+0000","last_became_peered":"2024-09-10T08:37:05.254055+0000","last_unstale":"2024-09-10T08:38:43.654774+0000","last_undegraded":"2024-09-10T08:38:43.654774+0000","last_fullsized":"2024-09-10T08:38:43.654774+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:08.472502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626522+0000","last_change":"2024-09-10T08:36:53.722774+0000","last_active":"2024-09-10T08:38:43.626522+0000","last_peered":"2024-09-10T08:38:43.626522+0000","last_clean":"2024-09-10T08:38:43.626522+0000","last_became_active":"2024-09-10T08:36:53.722459+0000","last_became_peered":"2024-09-10T08:36:53.722459+0000","last_unstale":"2024-09-10T08:38:43.626522+0000","last_undegraded":"2024-09-10T08:38:43.626522+0000","last_fullsized":"2024-09-10T08:38:43.626522+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:26.612354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592240+0000","last_change":"2024-09-10T08:36:56.143757+0000","last_active":"2024-09-10T08:39:11.592240+0000","last_peered":"2024-09-10T08:39:11.592240+0000","last_clean":"2024-09-10T08:39:11.592240+0000","last_became_active":"2024-09-10T08:36:56.143430+0000","last_became_peered":"2024-09-10T08:36:56.143430+0000","last_unstale":"2024-09-10T08:39:11.592240+0000","last_undegraded":"2024-09-10T08:39:11.592240+0000","last_fullsized":"2024-09-10T08:39:11.592240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:49.737298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655120+0000","last_change":"2024-09-10T08:37:05.254503+0000","last_active":"2024-09-10T08:38:43.655120+0000","last_peered":"2024-09-10T08:38:43.655120+0000","last_clean":"2024-09-10T08:38:43.655120+0000","last_became_active":"2024-09-10T08:37:05.254070+0000","last_became_peered":"2024-09-10T08:37:05.254070+0000","last_unstale":"2024-09-10T08:38:43.655120+0000","last_undegraded":"2024-09-10T08:38:43.655120+0000","last_fullsized":"2024-09-10T08:38:43.655120+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:16.741698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626438+0000","last_change":"2024-09-10T08:36:53.716742+0000","last_active":"2024-09-10T08:38:43.626438+0000","last_peered":"2024-09-10T08:38:43.626438+0000","last_clean":"2024-09-10T08:38:43.626438+0000","last_became_active":"2024-09-10T08:36:53.716439+0000","last_became_peered":"2024-09-10T08:36:53.716439+0000","last_unstale":"2024-09-10T08:38:43.626438+0000","last_undegraded":"2024-09-10T08:38:43.626438+0000","last_fullsized":"2024-09-10T08:38:43.626438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:46.882805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.581828+0000","last_change":"2024-09-10T08:36:56.146841+0000","last_active":"2024-09-10T08:39:11.581828+0000","last_peered":"2024-09-10T08:39:11.581828+0000","last_clean":"2024-09-10T08:39:11.581828+0000","last_became_active":"2024-09-10T08:36:56.146531+0000","last_became_peered":"2024-09-10T08:36:56.146531+0000","last_unstale":"2024-09-10T08:39:11.581828+0000","last_undegraded":"2024-09-10T08:39:11.581828+0000","last_fullsized":"2024-09-10T08:39:11.581828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:37.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654877+0000","last_change":"2024-09-10T08:37:43.223181+0000","last_active":"2024-09-10T08:38:43.654877+0000","last_peered":"2024-09-10T08:38:43.654877+0000","last_clean":"2024-09-10T08:38:43.654877+0000","last_became_active":"2024-09-10T08:37:43.222794+0000","last_became_peered":"2024-09-10T08:37:43.222794+0000","last_unstale":"2024-09-10T08:38:43.654877+0000","last_undegraded":"2024-09-10T08:38:43.654877+0000","last_fullsized":"2024-09-10T08:38:43.654877+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:22.204517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593593+0000","last_change":"2024-09-10T08:36:56.142161+0000","last_active":"2024-09-10T08:39:11.593593+0000","last_peered":"2024-09-10T08:39:11.593593+0000","last_clean":"2024-09-10T08:39:11.593593+0000","last_became_active":"2024-09-10T08:36:56.141838+0000","last_became_peered":"2024-09-10T08:36:56.141838+0000","last_unstale":"2024-09-10T08:39:11.593593+0000","last_undegraded":"2024-09-10T08:39:11.593593+0000","last_fullsized":"2024-09-10T08:39:11.593593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:09.909747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626125+0000","last_change":"2024-09-10T08:36:53.722238+0000","last_active":"2024-09-10T08:38:43.626125+0000","last_peered":"2024-09-10T08:38:43.626125+0000","last_clean":"2024-09-10T08:38:43.626125+0000","last_became_active":"2024-09-10T08:36:53.721859+0000","last_became_peered":"2024-09-10T08:36:53.721859+0000","last_unstale":"2024-09-10T08:38:43.626125+0000","last_undegraded":"2024-09-10T08:38:43.626125+0000","last_fullsized":"2024-09-10T08:38:43.626125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:50.615090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655183+0000","last_change":"2024-09-10T08:37:05.414041+0000","last_active":"2024-09-10T08:38:43.655183+0000","last_peered":"2024-09-10T08:38:43.655183+0000","last_clean":"2024-09-10T08:38:43.655183+0000","last_became_active":"2024-09-10T08:37:05.413286+0000","last_became_peered":"2024-09-10T08:37:05.413286+0000","last_unstale":"2024-09-10T08:38:43.655183+0000","last_undegraded":"2024-09-10T08:38:43.655183+0000","last_fullsized":"2024-09-10T08:38:43.655183+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:44.916102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.572936+0000","last_change":"2024-09-10T08:36:56.158278+0000","last_active":"2024-09-10T08:39:11.572936+0000","last_peered":"2024-09-10T08:39:11.572936+0000","last_clean":"2024-09-10T08:39:11.572936+0000","last_became_active":"2024-09-10T08:36:56.157946+0000","last_became_peered":"2024-09-10T08:36:56.157946+0000","last_unstale":"2024-09-10T08:39:11.572936+0000","last_undegraded":"2024-09-10T08:39:11.572936+0000","last_fullsized":"2024-09-10T08:39:11.572936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:42.512513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"96'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:38.713339+0000","last_change":"2024-09-10T08:38:43.635077+0000","last_active":"2024-09-10T08:39:38.713339+0000","last_peered":"2024-09-10T08:39:38.713339+0000","last_clean":"2024-09-10T08:39:38.713339+0000","last_became_active":"2024-09-10T08:38:43.631957+0000","last_became_peered":"2024-09-10T08:38:43.631957+0000","last_unstale":"2024-09-10T08:39:38.713339+0000","last_undegraded":"2024-09-10T08:39:38.713339+0000","last_fullsized":"2024-09-10T08:39:38.713339+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:50.837247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655126+0000","last_change":"2024-09-10T08:37:42.671399+0000","last_active":"2024-09-10T08:38:43.655126+0000","last_peered":"2024-09-10T08:38:43.655126+0000","last_clean":"2024-09-10T08:38:43.655126+0000","last_became_active":"2024-09-10T08:37:42.670865+0000","last_became_peered":"2024-09-10T08:37:42.670865+0000","last_unstale":"2024-09-10T08:38:43.655126+0000","last_undegraded":"2024-09-10T08:38:43.655126+0000","last_fullsized":"2024-09-10T08:38:43.655126+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:16.231538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.577960+0000","last_change":"2024-09-10T08:36:56.144856+0000","last_active":"2024-09-10T08:39:11.577960+0000","last_peered":"2024-09-10T08:39:11.577960+0000","last_clean":"2024-09-10T08:39:11.577960+0000","last_became_active":"2024-09-10T08:36:56.144531+0000","last_became_peered":"2024-09-10T08:36:56.144531+0000","last_unstale":"2024-09-10T08:39:11.577960+0000","last_undegraded":"2024-09-10T08:39:11.577960+0000","last_fullsized":"2024-09-10T08:39:11.577960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:24.586006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631541+0000","last_change":"2024-09-10T08:36:53.721093+0000","last_active":"2024-09-10T08:38:43.631541+0000","last_peered":"2024-09-10T08:38:43.631541+0000","last_clean":"2024-09-10T08:38:43.631541+0000","last_became_active":"2024-09-10T08:36:53.720538+0000","last_became_peered":"2024-09-10T08:36:53.720538+0000","last_unstale":"2024-09-10T08:38:43.631541+0000","last_undegraded":"2024-09-10T08:38:43.631541+0000","last_fullsized":"2024-09-10T08:38:43.631541+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:58.492463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629902+0000","last_change":"2024-09-10T08:37:05.254011+0000","last_active":"2024-09-10T08:38:43.629902+0000","last_peered":"2024-09-10T08:38:43.629902+0000","last_clean":"2024-09-10T08:38:43.629902+0000","last_became_active":"2024-09-10T08:37:05.253066+0000","last_became_peered":"2024-09-10T08:37:05.253066+0000","last_unstale":"2024-09-10T08:38:43.629902+0000","last_undegraded":"2024-09-10T08:38:43.629902+0000","last_fullsized":"2024-09-10T08:38:43.629902+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:16.221302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.553299+0000","last_change":"2024-09-10T08:36:56.141814+0000","last_active":"2024-09-10T08:39:11.553299+0000","last_peered":"2024-09-10T08:39:11.553299+0000","last_clean":"2024-09-10T08:39:11.553299+0000","last_became_active":"2024-09-10T08:36:56.141480+0000","last_became_peered":"2024-09-10T08:36:56.141480+0000","last_unstale":"2024-09-10T08:39:11.553299+0000","last_undegraded":"2024-09-10T08:39:11.553299+0000","last_fullsized":"2024-09-10T08:39:11.553299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:40.594354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.636347+0000","last_change":"2024-09-10T08:38:43.635917+0000","last_active":"2024-09-10T08:38:43.636347+0000","last_peered":"2024-09-10T08:38:43.636347+0000","last_clean":"2024-09-10T08:38:43.636347+0000","last_became_active":"2024-09-10T08:38:43.633406+0000","last_became_peered":"2024-09-10T08:38:43.633406+0000","last_unstale":"2024-09-10T08:38:43.636347+0000","last_undegraded":"2024-09-10T08:38:43.636347+0000","last_fullsized":"2024-09-10T08:38:43.636347+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:44.619072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655297+0000","last_change":"2024-09-10T08:37:05.414181+0000","last_active":"2024-09-10T08:38:43.655297+0000","last_peered":"2024-09-10T08:38:43.655297+0000","last_clean":"2024-09-10T08:38:43.655297+0000","last_became_active":"2024-09-10T08:37:05.413691+0000","last_became_peered":"2024-09-10T08:37:05.413691+0000","last_unstale":"2024-09-10T08:38:43.655297+0000","last_undegraded":"2024-09-10T08:38:43.655297+0000","last_fullsized":"2024-09-10T08:38:43.655297+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:30.744949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584567+0000","last_change":"2024-09-10T08:36:56.153956+0000","last_active":"2024-09-10T08:39:11.584567+0000","last_peered":"2024-09-10T08:39:11.584567+0000","last_clean":"2024-09-10T08:39:11.584567+0000","last_became_active":"2024-09-10T08:36:56.153609+0000","last_became_peered":"2024-09-10T08:36:56.153609+0000","last_unstale":"2024-09-10T08:39:11.584567+0000","last_undegraded":"2024-09-10T08:39:11.584567+0000","last_fullsized":"2024-09-10T08:39:11.584567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:18.185704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655520+0000","last_change":"2024-09-10T08:36:53.733217+0000","last_active":"2024-09-10T08:38:43.655520+0000","last_peered":"2024-09-10T08:38:43.655520+0000","last_clean":"2024-09-10T08:38:43.655520+0000","last_became_active":"2024-09-10T08:36:53.732851+0000","last_became_peered":"2024-09-10T08:36:53.732851+0000","last_unstale":"2024-09-10T08:38:43.655520+0000","last_undegraded":"2024-09-10T08:38:43.655520+0000","last_fullsized":"2024-09-10T08:38:43.655520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:04:01.132196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629889+0000","last_change":"2024-09-10T08:37:05.241868+0000","last_active":"2024-09-10T08:38:43.629889+0000","last_peered":"2024-09-10T08:38:43.629889+0000","last_clean":"2024-09-10T08:38:43.629889+0000","last_became_active":"2024-09-10T08:37:05.241412+0000","last_became_peered":"2024-09-10T08:37:05.241412+0000","last_unstale":"2024-09-10T08:38:43.629889+0000","last_undegraded":"2024-09-10T08:38:43.629889+0000","last_fullsized":"2024-09-10T08:38:43.629889+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:52.543353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.575701+0000","last_change":"2024-09-10T08:36:56.150722+0000","last_active":"2024-09-10T08:39:11.575701+0000","last_peered":"2024-09-10T08:39:11.575701+0000","last_clean":"2024-09-10T08:39:11.575701+0000","last_became_active":"2024-09-10T08:36:56.150423+0000","last_became_peered":"2024-09-10T08:36:56.150423+0000","last_unstale":"2024-09-10T08:39:11.575701+0000","last_undegraded":"2024-09-10T08:39:11.575701+0000","last_fullsized":"2024-09-10T08:39:11.575701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:50.272841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'13","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:38.175435+0000","last_change":"2024-09-10T08:36:53.722477+0000","last_active":"2024-09-10T08:39:38.175435+0000","last_peered":"2024-09-10T08:39:38.175435+0000","last_clean":"2024-09-10T08:39:38.175435+0000","last_became_active":"2024-09-10T08:36:53.722118+0000","last_became_peered":"2024-09-10T08:36:53.722118+0000","last_unstale":"2024-09-10T08:39:38.175435+0000","last_undegraded":"2024-09-10T08:39:38.175435+0000","last_fullsized":"2024-09-10T08:39:38.175435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:06.631646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630992+0000","last_change":"2024-09-10T08:37:05.415656+0000","last_active":"2024-09-10T08:38:43.630992+0000","last_peered":"2024-09-10T08:38:43.630992+0000","last_clean":"2024-09-10T08:38:43.630992+0000","last_became_active":"2024-09-10T08:37:05.413537+0000","last_became_peered":"2024-09-10T08:37:05.413537+0000","last_unstale":"2024-09-10T08:38:43.630992+0000","last_undegraded":"2024-09-10T08:38:43.630992+0000","last_fullsized":"2024-09-10T08:38:43.630992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:04.284410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585069+0000","last_change":"2024-09-10T08:36:56.144157+0000","last_active":"2024-09-10T08:39:11.585069+0000","last_peered":"2024-09-10T08:39:11.585069+0000","last_clean":"2024-09-10T08:39:11.585069+0000","last_became_active":"2024-09-10T08:36:56.140332+0000","last_became_peered":"2024-09-10T08:36:56.140332+0000","last_unstale":"2024-09-10T08:39:11.585069+0000","last_undegraded":"2024-09-10T08:39:11.585069+0000","last_fullsized":"2024-09-10T08:39:11.585069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:09:26.097987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626517+0000","last_change":"2024-09-10T08:36:53.735465+0000","last_active":"2024-09-10T08:38:43.626517+0000","last_peered":"2024-09-10T08:38:43.626517+0000","last_clean":"2024-09-10T08:38:43.626517+0000","last_became_active":"2024-09-10T08:36:53.735160+0000","last_became_peered":"2024-09-10T08:36:53.735160+0000","last_unstale":"2024-09-10T08:38:43.626517+0000","last_undegraded":"2024-09-10T08:38:43.626517+0000","last_fullsized":"2024-09-10T08:38:43.626517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:26:52.332085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627959+0000","last_change":"2024-09-10T08:37:05.247671+0000","last_active":"2024-09-10T08:38:43.627959+0000","last_peered":"2024-09-10T08:38:43.627959+0000","last_clean":"2024-09-10T08:38:43.627959+0000","last_became_active":"2024-09-10T08:37:05.247217+0000","last_became_peered":"2024-09-10T08:37:05.247217+0000","last_unstale":"2024-09-10T08:38:43.627959+0000","last_undegraded":"2024-09-10T08:38:43.627959+0000","last_fullsized":"2024-09-10T08:38:43.627959+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:20.881304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.579998+0000","last_change":"2024-09-10T08:36:56.145420+0000","last_active":"2024-09-10T08:39:11.579998+0000","last_peered":"2024-09-10T08:39:11.579998+0000","last_clean":"2024-09-10T08:39:11.579998+0000","last_became_active":"2024-09-10T08:36:56.145074+0000","last_became_peered":"2024-09-10T08:36:56.145074+0000","last_unstale":"2024-09-10T08:39:11.579998+0000","last_undegraded":"2024-09-10T08:39:11.579998+0000","last_fullsized":"2024-09-10T08:39:11.579998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:13.849977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629812+0000","last_change":"2024-09-10T08:36:53.738237+0000","last_active":"2024-09-10T08:38:43.629812+0000","last_peered":"2024-09-10T08:38:43.629812+0000","last_clean":"2024-09-10T08:38:43.629812+0000","last_became_active":"2024-09-10T08:36:53.737858+0000","last_became_peered":"2024-09-10T08:36:53.737858+0000","last_unstale":"2024-09-10T08:38:43.629812+0000","last_undegraded":"2024-09-10T08:38:43.629812+0000","last_fullsized":"2024-09-10T08:38:43.629812+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:35.345661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.625982+0000","last_change":"2024-09-10T08:37:05.413512+0000","last_active":"2024-09-10T08:38:43.625982+0000","last_peered":"2024-09-10T08:38:43.625982+0000","last_clean":"2024-09-10T08:38:43.625982+0000","last_became_active":"2024-09-10T08:37:05.413010+0000","last_became_peered":"2024-09-10T08:37:05.413010+0000","last_unstale":"2024-09-10T08:38:43.625982+0000","last_undegraded":"2024-09-10T08:38:43.625982+0000","last_fullsized":"2024-09-10T08:38:43.625982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:05.349404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583802+0000","last_change":"2024-09-10T08:36:56.157741+0000","last_active":"2024-09-10T08:39:11.583802+0000","last_peered":"2024-09-10T08:39:11.583802+0000","last_clean":"2024-09-10T08:39:11.583802+0000","last_became_active":"2024-09-10T08:36:56.157295+0000","last_became_peered":"2024-09-10T08:36:56.157295+0000","last_unstale":"2024-09-10T08:39:11.583802+0000","last_undegraded":"2024-09-10T08:39:11.583802+0000","last_fullsized":"2024-09-10T08:39:11.583802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:31.845864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631834+0000","last_change":"2024-09-10T08:36:53.722373+0000","last_active":"2024-09-10T08:38:43.631834+0000","last_peered":"2024-09-10T08:38:43.631834+0000","last_clean":"2024-09-10T08:38:43.631834+0000","last_became_active":"2024-09-10T08:36:53.721975+0000","last_became_peered":"2024-09-10T08:36:53.721975+0000","last_unstale":"2024-09-10T08:38:43.631834+0000","last_undegraded":"2024-09-10T08:38:43.631834+0000","last_fullsized":"2024-09-10T08:38:43.631834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:44.593018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628351+0000","last_change":"2024-09-10T08:37:05.255362+0000","last_active":"2024-09-10T08:38:43.628351+0000","last_peered":"2024-09-10T08:38:43.628351+0000","last_clean":"2024-09-10T08:38:43.628351+0000","last_became_active":"2024-09-10T08:37:05.254967+0000","last_became_peered":"2024-09-10T08:37:05.254967+0000","last_unstale":"2024-09-10T08:38:43.628351+0000","last_undegraded":"2024-09-10T08:38:43.628351+0000","last_fullsized":"2024-09-10T08:38:43.628351+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:31:41.850961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.578715+0000","last_change":"2024-09-10T08:36:56.147699+0000","last_active":"2024-09-10T08:39:11.578715+0000","last_peered":"2024-09-10T08:39:11.578715+0000","last_clean":"2024-09-10T08:39:11.578715+0000","last_became_active":"2024-09-10T08:36:56.147349+0000","last_became_peered":"2024-09-10T08:36:56.147349+0000","last_unstale":"2024-09-10T08:39:11.578715+0000","last_undegraded":"2024-09-10T08:39:11.578715+0000","last_fullsized":"2024-09-10T08:39:11.578715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:25.248588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"96'6","reported_seq":159,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:39.886723+0000","last_change":"2024-09-10T08:36:53.736843+0000","last_active":"2024-09-10T08:39:39.886723+0000","last_peered":"2024-09-10T08:39:39.886723+0000","last_clean":"2024-09-10T08:39:39.886723+0000","last_became_active":"2024-09-10T08:36:53.736340+0000","last_became_peered":"2024-09-10T08:36:53.736340+0000","last_unstale":"2024-09-10T08:39:39.886723+0000","last_undegraded":"2024-09-10T08:39:39.886723+0000","last_fullsized":"2024-09-10T08:39:39.886723+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:24.679679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628575+0000","last_change":"2024-09-10T08:37:05.414321+0000","last_active":"2024-09-10T08:38:43.628575+0000","last_peered":"2024-09-10T08:38:43.628575+0000","last_clean":"2024-09-10T08:38:43.628575+0000","last_became_active":"2024-09-10T08:37:05.413868+0000","last_became_peered":"2024-09-10T08:37:05.413868+0000","last_unstale":"2024-09-10T08:38:43.628575+0000","last_undegraded":"2024-09-10T08:38:43.628575+0000","last_fullsized":"2024-09-10T08:38:43.628575+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:53.988077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583174+0000","last_change":"2024-09-10T08:36:56.141910+0000","last_active":"2024-09-10T08:39:11.583174+0000","last_peered":"2024-09-10T08:39:11.583174+0000","last_clean":"2024-09-10T08:39:11.583174+0000","last_became_active":"2024-09-10T08:36:56.141579+0000","last_became_peered":"2024-09-10T08:36:56.141579+0000","last_unstale":"2024-09-10T08:39:11.583174+0000","last_undegraded":"2024-09-10T08:39:11.583174+0000","last_fullsized":"2024-09-10T08:39:11.583174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:18.464831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631756+0000","last_change":"2024-09-10T08:36:53.720102+0000","last_active":"2024-09-10T08:38:43.631756+0000","last_peered":"2024-09-10T08:38:43.631756+0000","last_clean":"2024-09-10T08:38:43.631756+0000","last_became_active":"2024-09-10T08:36:53.719764+0000","last_became_peered":"2024-09-10T08:36:53.719764+0000","last_unstale":"2024-09-10T08:38:43.631756+0000","last_undegraded":"2024-09-10T08:38:43.631756+0000","last_fullsized":"2024-09-10T08:38:43.631756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:03.609338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655205+0000","last_change":"2024-09-10T08:37:05.413818+0000","last_active":"2024-09-10T08:38:43.655205+0000","last_peered":"2024-09-10T08:38:43.655205+0000","last_clean":"2024-09-10T08:38:43.655205+0000","last_became_active":"2024-09-10T08:37:05.413267+0000","last_became_peered":"2024-09-10T08:37:05.413267+0000","last_unstale":"2024-09-10T08:38:43.655205+0000","last_undegraded":"2024-09-10T08:38:43.655205+0000","last_fullsized":"2024-09-10T08:38:43.655205+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:54.309686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.590486+0000","last_change":"2024-09-10T08:36:56.149286+0000","last_active":"2024-09-10T08:39:11.590486+0000","last_peered":"2024-09-10T08:39:11.590486+0000","last_clean":"2024-09-10T08:39:11.590486+0000","last_became_active":"2024-09-10T08:36:56.146275+0000","last_became_peered":"2024-09-10T08:36:56.146275+0000","last_unstale":"2024-09-10T08:39:11.590486+0000","last_undegraded":"2024-09-10T08:39:11.590486+0000","last_fullsized":"2024-09-10T08:39:11.590486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:19.468224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655269+0000","last_change":"2024-09-10T08:36:53.719756+0000","last_active":"2024-09-10T08:38:43.655269+0000","last_peered":"2024-09-10T08:38:43.655269+0000","last_clean":"2024-09-10T08:38:43.655269+0000","last_became_active":"2024-09-10T08:36:53.719302+0000","last_became_peered":"2024-09-10T08:36:53.719302+0000","last_unstale":"2024-09-10T08:38:43.655269+0000","last_undegraded":"2024-09-10T08:38:43.655269+0000","last_fullsized":"2024-09-10T08:38:43.655269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:47.658081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631788+0000","last_change":"2024-09-10T08:35:39.150715+0000","last_active":"2024-09-10T08:38:43.631788+0000","last_peered":"2024-09-10T08:38:43.631788+0000","last_clean":"2024-09-10T08:38:43.631788+0000","last_became_active":"2024-09-10T08:35:39.150206+0000","last_became_peered":"2024-09-10T08:35:39.150206+0000","last_unstale":"2024-09-10T08:38:43.631788+0000","last_undegraded":"2024-09-10T08:38:43.631788+0000","last_fullsized":"2024-09-10T08:38:43.631788+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:32:02.110870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:32:02.110870+0000","last_clean_scrub_stamp":"2024-09-10T08:32:02.110870+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:20.762043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631998+0000","last_change":"2024-09-10T08:37:42.846547+0000","last_active":"2024-09-10T08:38:43.631998+0000","last_peered":"2024-09-10T08:38:43.631998+0000","last_clean":"2024-09-10T08:38:43.631998+0000","last_became_active":"2024-09-10T08:37:42.845884+0000","last_became_peered":"2024-09-10T08:37:42.845884+0000","last_unstale":"2024-09-10T08:38:43.631998+0000","last_undegraded":"2024-09-10T08:38:43.631998+0000","last_fullsized":"2024-09-10T08:38:43.631998+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:21.905133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593781+0000","last_change":"2024-09-10T08:36:56.144187+0000","last_active":"2024-09-10T08:39:11.593781+0000","last_peered":"2024-09-10T08:39:11.593781+0000","last_clean":"2024-09-10T08:39:11.593781+0000","last_became_active":"2024-09-10T08:36:56.143193+0000","last_became_peered":"2024-09-10T08:36:56.143193+0000","last_unstale":"2024-09-10T08:39:11.593781+0000","last_undegraded":"2024-09-10T08:39:11.593781+0000","last_fullsized":"2024-09-10T08:39:11.593781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:14:26.981884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.639660+0000","last_change":"2024-09-10T08:38:43.639286+0000","last_active":"2024-09-10T08:38:43.639660+0000","last_peered":"2024-09-10T08:38:43.639660+0000","last_clean":"2024-09-10T08:38:43.639660+0000","last_became_active":"2024-09-10T08:38:43.633552+0000","last_became_peered":"2024-09-10T08:38:43.633552+0000","last_unstale":"2024-09-10T08:38:43.639660+0000","last_undegraded":"2024-09-10T08:38:43.639660+0000","last_fullsized":"2024-09-10T08:38:43.639660+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:23.985190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630209+0000","last_change":"2024-09-10T08:37:05.242095+0000","last_active":"2024-09-10T08:38:43.630209+0000","last_peered":"2024-09-10T08:38:43.630209+0000","last_clean":"2024-09-10T08:38:43.630209+0000","last_became_active":"2024-09-10T08:37:05.241645+0000","last_became_peered":"2024-09-10T08:37:05.241645+0000","last_unstale":"2024-09-10T08:38:43.630209+0000","last_undegraded":"2024-09-10T08:38:43.630209+0000","last_fullsized":"2024-09-10T08:38:43.630209+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:30:49.637861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588835+0000","last_change":"2024-09-10T08:36:56.149396+0000","last_active":"2024-09-10T08:39:11.588835+0000","last_peered":"2024-09-10T08:39:11.588835+0000","last_clean":"2024-09-10T08:39:11.588835+0000","last_became_active":"2024-09-10T08:36:56.149079+0000","last_became_peered":"2024-09-10T08:36:56.149079+0000","last_unstale":"2024-09-10T08:39:11.588835+0000","last_undegraded":"2024-09-10T08:39:11.588835+0000","last_fullsized":"2024-09-10T08:39:11.588835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:44.665448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655467+0000","last_change":"2024-09-10T08:36:53.717950+0000","last_active":"2024-09-10T08:38:43.655467+0000","last_peered":"2024-09-10T08:38:43.655467+0000","last_clean":"2024-09-10T08:38:43.655467+0000","last_became_active":"2024-09-10T08:36:53.717668+0000","last_became_peered":"2024-09-10T08:36:53.717668+0000","last_unstale":"2024-09-10T08:38:43.655467+0000","last_undegraded":"2024-09-10T08:38:43.655467+0000","last_fullsized":"2024-09-10T08:38:43.655467+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:58.692631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633140+0000","last_change":"2024-09-10T08:37:05.260178+0000","last_active":"2024-09-10T08:38:43.633140+0000","last_peered":"2024-09-10T08:38:43.633140+0000","last_clean":"2024-09-10T08:38:43.633140+0000","last_became_active":"2024-09-10T08:37:05.259753+0000","last_became_peered":"2024-09-10T08:37:05.259753+0000","last_unstale":"2024-09-10T08:38:43.633140+0000","last_undegraded":"2024-09-10T08:38:43.633140+0000","last_fullsized":"2024-09-10T08:38:43.633140+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:19.364782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588360+0000","last_change":"2024-09-10T08:36:56.147104+0000","last_active":"2024-09-10T08:39:11.588360+0000","last_peered":"2024-09-10T08:39:11.588360+0000","last_clean":"2024-09-10T08:39:11.588360+0000","last_became_active":"2024-09-10T08:36:56.146778+0000","last_became_peered":"2024-09-10T08:36:56.146778+0000","last_unstale":"2024-09-10T08:39:11.588360+0000","last_undegraded":"2024-09-10T08:39:11.588360+0000","last_fullsized":"2024-09-10T08:39:11.588360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:56.092157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628399+0000","last_change":"2024-09-10T08:36:53.719145+0000","last_active":"2024-09-10T08:38:43.628399+0000","last_peered":"2024-09-10T08:38:43.628399+0000","last_clean":"2024-09-10T08:38:43.628399+0000","last_became_active":"2024-09-10T08:36:53.718771+0000","last_became_peered":"2024-09-10T08:36:53.718771+0000","last_unstale":"2024-09-10T08:38:43.628399+0000","last_undegraded":"2024-09-10T08:38:43.628399+0000","last_fullsized":"2024-09-10T08:38:43.628399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:21.109940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654733+0000","last_change":"2024-09-10T08:37:05.242854+0000","last_active":"2024-09-10T08:38:43.654733+0000","last_peered":"2024-09-10T08:38:43.654733+0000","last_clean":"2024-09-10T08:38:43.654733+0000","last_became_active":"2024-09-10T08:37:05.242442+0000","last_became_peered":"2024-09-10T08:37:05.242442+0000","last_unstale":"2024-09-10T08:38:43.654733+0000","last_undegraded":"2024-09-10T08:38:43.654733+0000","last_fullsized":"2024-09-10T08:38:43.654733+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:36.072131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.570294+0000","last_change":"2024-09-10T08:37:42.845642+0000","last_active":"2024-09-10T08:39:11.570294+0000","last_peered":"2024-09-10T08:39:11.570294+0000","last_clean":"2024-09-10T08:39:11.570294+0000","last_became_active":"2024-09-10T08:37:42.845217+0000","last_became_peered":"2024-09-10T08:37:42.845217+0000","last_unstale":"2024-09-10T08:39:11.570294+0000","last_undegraded":"2024-09-10T08:39:11.570294+0000","last_fullsized":"2024-09-10T08:39:11.570294+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:28.020337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626734+0000","last_change":"2024-09-10T08:36:53.720463+0000","last_active":"2024-09-10T08:38:43.626734+0000","last_peered":"2024-09-10T08:38:43.626734+0000","last_clean":"2024-09-10T08:38:43.626734+0000","last_became_active":"2024-09-10T08:36:53.720080+0000","last_became_peered":"2024-09-10T08:36:53.720080+0000","last_unstale":"2024-09-10T08:38:43.626734+0000","last_undegraded":"2024-09-10T08:38:43.626734+0000","last_fullsized":"2024-09-10T08:38:43.626734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:51.450294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654955+0000","last_change":"2024-09-10T08:37:05.251261+0000","last_active":"2024-09-10T08:38:43.654955+0000","last_peered":"2024-09-10T08:38:43.654955+0000","last_clean":"2024-09-10T08:38:43.654955+0000","last_became_active":"2024-09-10T08:37:05.250938+0000","last_became_peered":"2024-09-10T08:37:05.250938+0000","last_unstale":"2024-09-10T08:38:43.654955+0000","last_undegraded":"2024-09-10T08:38:43.654955+0000","last_fullsized":"2024-09-10T08:38:43.654955+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:07:15.129899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.581822+0000","last_change":"2024-09-10T08:36:56.148191+0000","last_active":"2024-09-10T08:39:11.581822+0000","last_peered":"2024-09-10T08:39:11.581822+0000","last_clean":"2024-09-10T08:39:11.581822+0000","last_became_active":"2024-09-10T08:36:56.147362+0000","last_became_peered":"2024-09-10T08:36:56.147362+0000","last_unstale":"2024-09-10T08:39:11.581822+0000","last_undegraded":"2024-09-10T08:39:11.581822+0000","last_fullsized":"2024-09-10T08:39:11.581822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:25:30.051754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626042+0000","last_change":"2024-09-10T08:36:53.734484+0000","last_active":"2024-09-10T08:38:43.626042+0000","last_peered":"2024-09-10T08:38:43.626042+0000","last_clean":"2024-09-10T08:38:43.626042+0000","last_became_active":"2024-09-10T08:36:53.734172+0000","last_became_peered":"2024-09-10T08:36:53.734172+0000","last_unstale":"2024-09-10T08:38:43.626042+0000","last_undegraded":"2024-09-10T08:38:43.626042+0000","last_fullsized":"2024-09-10T08:38:43.626042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:10.269400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628299+0000","last_change":"2024-09-10T08:37:05.259635+0000","last_active":"2024-09-10T08:38:43.628299+0000","last_peered":"2024-09-10T08:38:43.628299+0000","last_clean":"2024-09-10T08:38:43.628299+0000","last_became_active":"2024-09-10T08:37:05.259107+0000","last_became_peered":"2024-09-10T08:37:05.259107+0000","last_unstale":"2024-09-10T08:38:43.628299+0000","last_undegraded":"2024-09-10T08:38:43.628299+0000","last_fullsized":"2024-09-10T08:38:43.628299+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:36.841138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588992+0000","last_change":"2024-09-10T08:36:56.141122+0000","last_active":"2024-09-10T08:39:11.588992+0000","last_peered":"2024-09-10T08:39:11.588992+0000","last_clean":"2024-09-10T08:39:11.588992+0000","last_became_active":"2024-09-10T08:36:56.140826+0000","last_became_peered":"2024-09-10T08:36:56.140826+0000","last_unstale":"2024-09-10T08:39:11.588992+0000","last_undegraded":"2024-09-10T08:39:11.588992+0000","last_fullsized":"2024-09-10T08:39:11.588992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:38.044090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"96'6","reported_seq":163,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:39.887564+0000","last_change":"2024-09-10T08:36:53.720623+0000","last_active":"2024-09-10T08:39:39.887564+0000","last_peered":"2024-09-10T08:39:39.887564+0000","last_clean":"2024-09-10T08:39:39.887564+0000","last_became_active":"2024-09-10T08:36:53.720270+0000","last_became_peered":"2024-09-10T08:36:53.720270+0000","last_unstale":"2024-09-10T08:39:39.887564+0000","last_undegraded":"2024-09-10T08:39:39.887564+0000","last_fullsized":"2024-09-10T08:39:39.887564+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:19.514068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629937+0000","last_change":"2024-09-10T08:37:05.254324+0000","last_active":"2024-09-10T08:38:43.629937+0000","last_peered":"2024-09-10T08:38:43.629937+0000","last_clean":"2024-09-10T08:38:43.629937+0000","last_became_active":"2024-09-10T08:37:05.253611+0000","last_became_peered":"2024-09-10T08:37:05.253611+0000","last_unstale":"2024-09-10T08:38:43.629937+0000","last_undegraded":"2024-09-10T08:38:43.629937+0000","last_fullsized":"2024-09-10T08:38:43.629937+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:07.182649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.591257+0000","last_change":"2024-09-10T08:36:56.139102+0000","last_active":"2024-09-10T08:39:11.591257+0000","last_peered":"2024-09-10T08:39:11.591257+0000","last_clean":"2024-09-10T08:39:11.591257+0000","last_became_active":"2024-09-10T08:36:56.137986+0000","last_became_peered":"2024-09-10T08:36:56.137986+0000","last_unstale":"2024-09-10T08:39:11.591257+0000","last_undegraded":"2024-09-10T08:39:11.591257+0000","last_fullsized":"2024-09-10T08:39:11.591257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:05:03.005727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655021+0000","last_change":"2024-09-10T08:36:53.722961+0000","last_active":"2024-09-10T08:38:43.655021+0000","last_peered":"2024-09-10T08:38:43.655021+0000","last_clean":"2024-09-10T08:38:43.655021+0000","last_became_active":"2024-09-10T08:36:53.722628+0000","last_became_peered":"2024-09-10T08:36:53.722628+0000","last_unstale":"2024-09-10T08:38:43.655021+0000","last_undegraded":"2024-09-10T08:38:43.655021+0000","last_fullsized":"2024-09-10T08:38:43.655021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:59.048632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628104+0000","last_change":"2024-09-10T08:37:05.256546+0000","last_active":"2024-09-10T08:38:43.628104+0000","last_peered":"2024-09-10T08:38:43.628104+0000","last_clean":"2024-09-10T08:38:43.628104+0000","last_became_active":"2024-09-10T08:37:05.256239+0000","last_became_peered":"2024-09-10T08:37:05.256239+0000","last_unstale":"2024-09-10T08:38:43.628104+0000","last_undegraded":"2024-09-10T08:38:43.628104+0000","last_fullsized":"2024-09-10T08:38:43.628104+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:57.403934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.582848+0000","last_change":"2024-09-10T08:36:56.149285+0000","last_active":"2024-09-10T08:39:11.582848+0000","last_peered":"2024-09-10T08:39:11.582848+0000","last_clean":"2024-09-10T08:39:11.582848+0000","last_became_active":"2024-09-10T08:36:56.146873+0000","last_became_peered":"2024-09-10T08:36:56.146873+0000","last_unstale":"2024-09-10T08:39:11.582848+0000","last_undegraded":"2024-09-10T08:39:11.582848+0000","last_fullsized":"2024-09-10T08:39:11.582848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:28.976729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630368+0000","last_change":"2024-09-10T08:36:53.721174+0000","last_active":"2024-09-10T08:38:43.630368+0000","last_peered":"2024-09-10T08:38:43.630368+0000","last_clean":"2024-09-10T08:38:43.630368+0000","last_became_active":"2024-09-10T08:36:53.720849+0000","last_became_peered":"2024-09-10T08:36:53.720849+0000","last_unstale":"2024-09-10T08:38:43.630368+0000","last_undegraded":"2024-09-10T08:38:43.630368+0000","last_fullsized":"2024-09-10T08:38:43.630368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:39.786848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655286+0000","last_change":"2024-09-10T08:37:05.252256+0000","last_active":"2024-09-10T08:38:43.655286+0000","last_peered":"2024-09-10T08:38:43.655286+0000","last_clean":"2024-09-10T08:38:43.655286+0000","last_became_active":"2024-09-10T08:37:05.251865+0000","last_became_peered":"2024-09-10T08:37:05.251865+0000","last_unstale":"2024-09-10T08:38:43.655286+0000","last_undegraded":"2024-09-10T08:38:43.655286+0000","last_fullsized":"2024-09-10T08:38:43.655286+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:08.581880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567773+0000","last_change":"2024-09-10T08:36:56.142060+0000","last_active":"2024-09-10T08:39:11.567773+0000","last_peered":"2024-09-10T08:39:11.567773+0000","last_clean":"2024-09-10T08:39:11.567773+0000","last_became_active":"2024-09-10T08:36:56.141451+0000","last_became_peered":"2024-09-10T08:36:56.141451+0000","last_unstale":"2024-09-10T08:39:11.567773+0000","last_undegraded":"2024-09-10T08:39:11.567773+0000","last_fullsized":"2024-09-10T08:39:11.567773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:43.566588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'37","reported_seq":310,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:38.877082+0000","last_change":"2024-09-10T08:36:53.719464+0000","last_active":"2024-09-10T08:39:38.877082+0000","last_peered":"2024-09-10T08:39:38.877082+0000","last_clean":"2024-09-10T08:39:38.877082+0000","last_became_active":"2024-09-10T08:36:53.719129+0000","last_became_peered":"2024-09-10T08:36:53.719129+0000","last_unstale":"2024-09-10T08:39:38.877082+0000","last_undegraded":"2024-09-10T08:39:38.877082+0000","last_fullsized":"2024-09-10T08:39:38.877082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:05.652018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64071,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":232,"num_read_kb":179,"num_write":38,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629405+0000","last_change":"2024-09-10T08:37:05.255576+0000","last_active":"2024-09-10T08:38:43.629405+0000","last_peered":"2024-09-10T08:38:43.629405+0000","last_clean":"2024-09-10T08:38:43.629405+0000","last_became_active":"2024-09-10T08:37:05.249842+0000","last_became_peered":"2024-09-10T08:37:05.249842+0000","last_unstale":"2024-09-10T08:38:43.629405+0000","last_undegraded":"2024-09-10T08:38:43.629405+0000","last_fullsized":"2024-09-10T08:38:43.629405+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:47:49.543481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593397+0000","last_change":"2024-09-10T08:36:56.151829+0000","last_active":"2024-09-10T08:39:11.593397+0000","last_peered":"2024-09-10T08:39:11.593397+0000","last_clean":"2024-09-10T08:39:11.593397+0000","last_became_active":"2024-09-10T08:36:56.151314+0000","last_became_peered":"2024-09-10T08:36:56.151314+0000","last_unstale":"2024-09-10T08:39:11.593397+0000","last_undegraded":"2024-09-10T08:39:11.593397+0000","last_fullsized":"2024-09-10T08:39:11.593397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:44.219671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655398+0000","last_change":"2024-09-10T08:36:53.738030+0000","last_active":"2024-09-10T08:38:43.655398+0000","last_peered":"2024-09-10T08:38:43.655398+0000","last_clean":"2024-09-10T08:38:43.655398+0000","last_became_active":"2024-09-10T08:36:53.737658+0000","last_became_peered":"2024-09-10T08:36:53.737658+0000","last_unstale":"2024-09-10T08:38:43.655398+0000","last_undegraded":"2024-09-10T08:38:43.655398+0000","last_fullsized":"2024-09-10T08:38:43.655398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:29.627554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654996+0000","last_change":"2024-09-10T08:37:05.244473+0000","last_active":"2024-09-10T08:38:43.654996+0000","last_peered":"2024-09-10T08:38:43.654996+0000","last_clean":"2024-09-10T08:38:43.654996+0000","last_became_active":"2024-09-10T08:37:05.244035+0000","last_became_peered":"2024-09-10T08:37:05.244035+0000","last_unstale":"2024-09-10T08:38:43.654996+0000","last_undegraded":"2024-09-10T08:38:43.654996+0000","last_fullsized":"2024-09-10T08:38:43.654996+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:04.652660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584024+0000","last_change":"2024-09-10T08:36:56.155217+0000","last_active":"2024-09-10T08:39:11.584024+0000","last_peered":"2024-09-10T08:39:11.584024+0000","last_clean":"2024-09-10T08:39:11.584024+0000","last_became_active":"2024-09-10T08:36:56.154903+0000","last_became_peered":"2024-09-10T08:36:56.154903+0000","last_unstale":"2024-09-10T08:39:11.584024+0000","last_undegraded":"2024-09-10T08:39:11.584024+0000","last_fullsized":"2024-09-10T08:39:11.584024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:47.482974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.632239+0000","last_change":"2024-09-10T08:36:53.721087+0000","last_active":"2024-09-10T08:38:43.632239+0000","last_peered":"2024-09-10T08:38:43.632239+0000","last_clean":"2024-09-10T08:38:43.632239+0000","last_became_active":"2024-09-10T08:36:53.720772+0000","last_became_peered":"2024-09-10T08:36:53.720772+0000","last_unstale":"2024-09-10T08:38:43.632239+0000","last_undegraded":"2024-09-10T08:38:43.632239+0000","last_fullsized":"2024-09-10T08:38:43.632239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:39.560848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629150+0000","last_change":"2024-09-10T08:37:05.254641+0000","last_active":"2024-09-10T08:38:43.629150+0000","last_peered":"2024-09-10T08:38:43.629150+0000","last_clean":"2024-09-10T08:38:43.629150+0000","last_became_active":"2024-09-10T08:37:05.249982+0000","last_became_peered":"2024-09-10T08:37:05.249982+0000","last_unstale":"2024-09-10T08:38:43.629150+0000","last_undegraded":"2024-09-10T08:38:43.629150+0000","last_fullsized":"2024-09-10T08:38:43.629150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:21:13.851061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592510+0000","last_change":"2024-09-10T08:36:56.142657+0000","last_active":"2024-09-10T08:39:11.592510+0000","last_peered":"2024-09-10T08:39:11.592510+0000","last_clean":"2024-09-10T08:39:11.592510+0000","last_became_active":"2024-09-10T08:36:56.142294+0000","last_became_peered":"2024-09-10T08:36:56.142294+0000","last_unstale":"2024-09-10T08:39:11.592510+0000","last_undegraded":"2024-09-10T08:39:11.592510+0000","last_fullsized":"2024-09-10T08:39:11.592510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:52.861389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'4","reported_seq":156,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:37.909693+0000","last_change":"2024-09-10T08:36:53.722466+0000","last_active":"2024-09-10T08:39:37.909693+0000","last_peered":"2024-09-10T08:39:37.909693+0000","last_clean":"2024-09-10T08:39:37.909693+0000","last_became_active":"2024-09-10T08:36:53.722145+0000","last_became_peered":"2024-09-10T08:36:53.722145+0000","last_unstale":"2024-09-10T08:39:37.909693+0000","last_undegraded":"2024-09-10T08:39:37.909693+0000","last_fullsized":"2024-09-10T08:39:37.909693+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:27.041765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631352+0000","last_change":"2024-09-10T08:37:05.251056+0000","last_active":"2024-09-10T08:38:43.631352+0000","last_peered":"2024-09-10T08:38:43.631352+0000","last_clean":"2024-09-10T08:38:43.631352+0000","last_became_active":"2024-09-10T08:37:05.250653+0000","last_became_peered":"2024-09-10T08:37:05.250653+0000","last_unstale":"2024-09-10T08:38:43.631352+0000","last_undegraded":"2024-09-10T08:38:43.631352+0000","last_fullsized":"2024-09-10T08:38:43.631352+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:21.174747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633128+0000","last_change":"2024-09-10T08:37:05.260436+0000","last_active":"2024-09-10T08:38:43.633128+0000","last_peered":"2024-09-10T08:38:43.633128+0000","last_clean":"2024-09-10T08:38:43.633128+0000","last_became_active":"2024-09-10T08:37:05.260049+0000","last_became_peered":"2024-09-10T08:37:05.260049+0000","last_unstale":"2024-09-10T08:38:43.633128+0000","last_undegraded":"2024-09-10T08:38:43.633128+0000","last_fullsized":"2024-09-10T08:38:43.633128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:43.394028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585930+0000","last_change":"2024-09-10T08:36:56.146732+0000","last_active":"2024-09-10T08:39:11.585930+0000","last_peered":"2024-09-10T08:39:11.585930+0000","last_clean":"2024-09-10T08:39:11.585930+0000","last_became_active":"2024-09-10T08:36:56.145599+0000","last_became_peered":"2024-09-10T08:36:56.145599+0000","last_unstale":"2024-09-10T08:39:11.585930+0000","last_undegraded":"2024-09-10T08:39:11.585930+0000","last_fullsized":"2024-09-10T08:39:11.585930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:50.620956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655135+0000","last_change":"2024-09-10T08:36:53.721683+0000","last_active":"2024-09-10T08:38:43.655135+0000","last_peered":"2024-09-10T08:38:43.655135+0000","last_clean":"2024-09-10T08:38:43.655135+0000","last_became_active":"2024-09-10T08:36:53.721241+0000","last_became_peered":"2024-09-10T08:36:53.721241+0000","last_unstale":"2024-09-10T08:38:43.655135+0000","last_undegraded":"2024-09-10T08:38:43.655135+0000","last_fullsized":"2024-09-10T08:38:43.655135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:03.669337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628275+0000","last_change":"2024-09-10T08:37:42.669784+0000","last_active":"2024-09-10T08:38:43.628275+0000","last_peered":"2024-09-10T08:38:43.628275+0000","last_clean":"2024-09-10T08:38:43.628275+0000","last_became_active":"2024-09-10T08:37:42.669322+0000","last_became_peered":"2024-09-10T08:37:42.669322+0000","last_unstale":"2024-09-10T08:38:43.628275+0000","last_undegraded":"2024-09-10T08:38:43.628275+0000","last_fullsized":"2024-09-10T08:38:43.628275+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:06:57.027928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.574617+0000","last_change":"2024-09-10T08:36:56.143824+0000","last_active":"2024-09-10T08:39:11.574617+0000","last_peered":"2024-09-10T08:39:11.574617+0000","last_clean":"2024-09-10T08:39:11.574617+0000","last_became_active":"2024-09-10T08:36:56.143486+0000","last_became_peered":"2024-09-10T08:36:56.143486+0000","last_unstale":"2024-09-10T08:39:11.574617+0000","last_undegraded":"2024-09-10T08:39:11.574617+0000","last_fullsized":"2024-09-10T08:39:11.574617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:42.384863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627976+0000","last_change":"2024-09-10T08:36:53.717680+0000","last_active":"2024-09-10T08:38:43.627976+0000","last_peered":"2024-09-10T08:38:43.627976+0000","last_clean":"2024-09-10T08:38:43.627976+0000","last_became_active":"2024-09-10T08:36:53.717249+0000","last_became_peered":"2024-09-10T08:36:53.717249+0000","last_unstale":"2024-09-10T08:38:43.627976+0000","last_undegraded":"2024-09-10T08:38:43.627976+0000","last_fullsized":"2024-09-10T08:38:43.627976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:12.191541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627200+0000","last_change":"2024-09-10T08:36:53.723947+0000","last_active":"2024-09-10T08:38:43.627200+0000","last_peered":"2024-09-10T08:38:43.627200+0000","last_clean":"2024-09-10T08:38:43.627200+0000","last_became_active":"2024-09-10T08:36:53.723620+0000","last_became_peered":"2024-09-10T08:36:53.723620+0000","last_unstale":"2024-09-10T08:38:43.627200+0000","last_undegraded":"2024-09-10T08:38:43.627200+0000","last_fullsized":"2024-09-10T08:38:43.627200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:00.135349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583145+0000","last_change":"2024-09-10T08:36:56.150448+0000","last_active":"2024-09-10T08:39:11.583145+0000","last_peered":"2024-09-10T08:39:11.583145+0000","last_clean":"2024-09-10T08:39:11.583145+0000","last_became_active":"2024-09-10T08:36:56.150148+0000","last_became_peered":"2024-09-10T08:36:56.150148+0000","last_unstale":"2024-09-10T08:39:11.583145+0000","last_undegraded":"2024-09-10T08:39:11.583145+0000","last_fullsized":"2024-09-10T08:39:11.583145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:02.927753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630191+0000","last_change":"2024-09-10T08:37:05.242878+0000","last_active":"2024-09-10T08:38:43.630191+0000","last_peered":"2024-09-10T08:38:43.630191+0000","last_clean":"2024-09-10T08:38:43.630191+0000","last_became_active":"2024-09-10T08:37:05.242440+0000","last_became_peered":"2024-09-10T08:37:05.242440+0000","last_unstale":"2024-09-10T08:38:43.630191+0000","last_undegraded":"2024-09-10T08:38:43.630191+0000","last_fullsized":"2024-09-10T08:38:43.630191+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:30.957433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'7","reported_seq":163,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:39.878942+0000","last_change":"2024-09-10T08:36:53.719788+0000","last_active":"2024-09-10T08:39:39.878942+0000","last_peered":"2024-09-10T08:39:39.878942+0000","last_clean":"2024-09-10T08:39:39.878942+0000","last_became_active":"2024-09-10T08:36:53.719462+0000","last_became_peered":"2024-09-10T08:36:53.719462+0000","last_unstale":"2024-09-10T08:39:39.878942+0000","last_undegraded":"2024-09-10T08:39:39.878942+0000","last_fullsized":"2024-09-10T08:39:39.878942+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:12.250447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1326,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583170+0000","last_change":"2024-09-10T08:37:42.673008+0000","last_active":"2024-09-10T08:39:11.583170+0000","last_peered":"2024-09-10T08:39:11.583170+0000","last_clean":"2024-09-10T08:39:11.583170+0000","last_became_active":"2024-09-10T08:37:42.672618+0000","last_became_peered":"2024-09-10T08:37:42.672618+0000","last_unstale":"2024-09-10T08:39:11.583170+0000","last_undegraded":"2024-09-10T08:39:11.583170+0000","last_fullsized":"2024-09-10T08:39:11.583170+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:00.156565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630783+0000","last_change":"2024-09-10T08:37:05.244277+0000","last_active":"2024-09-10T08:38:43.630783+0000","last_peered":"2024-09-10T08:38:43.630783+0000","last_clean":"2024-09-10T08:38:43.630783+0000","last_became_active":"2024-09-10T08:37:05.243880+0000","last_became_peered":"2024-09-10T08:37:05.243880+0000","last_unstale":"2024-09-10T08:38:43.630783+0000","last_undegraded":"2024-09-10T08:38:43.630783+0000","last_fullsized":"2024-09-10T08:38:43.630783+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:55.275857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.573131+0000","last_change":"2024-09-10T08:36:56.141404+0000","last_active":"2024-09-10T08:39:11.573131+0000","last_peered":"2024-09-10T08:39:11.573131+0000","last_clean":"2024-09-10T08:39:11.573131+0000","last_became_active":"2024-09-10T08:36:56.141076+0000","last_became_peered":"2024-09-10T08:36:56.141076+0000","last_unstale":"2024-09-10T08:39:11.573131+0000","last_undegraded":"2024-09-10T08:39:11.573131+0000","last_fullsized":"2024-09-10T08:39:11.573131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:03:52.915654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.569383+0000","last_change":"2024-09-10T08:36:56.148285+0000","last_active":"2024-09-10T08:39:11.569383+0000","last_peered":"2024-09-10T08:39:11.569383+0000","last_clean":"2024-09-10T08:39:11.569383+0000","last_became_active":"2024-09-10T08:36:56.147970+0000","last_became_peered":"2024-09-10T08:36:56.147970+0000","last_unstale":"2024-09-10T08:39:11.569383+0000","last_undegraded":"2024-09-10T08:39:11.569383+0000","last_fullsized":"2024-09-10T08:39:11.569383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:09.506718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.571047+0000","last_change":"2024-09-10T08:36:56.157977+0000","last_active":"2024-09-10T08:39:11.571047+0000","last_peered":"2024-09-10T08:39:11.571047+0000","last_clean":"2024-09-10T08:39:11.571047+0000","last_became_active":"2024-09-10T08:36:56.157623+0000","last_became_peered":"2024-09-10T08:36:56.157623+0000","last_unstale":"2024-09-10T08:39:11.571047+0000","last_undegraded":"2024-09-10T08:39:11.571047+0000","last_fullsized":"2024-09-10T08:39:11.571047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:44.617979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584899+0000","last_change":"2024-09-10T08:37:42.670459+0000","last_active":"2024-09-10T08:39:11.584899+0000","last_peered":"2024-09-10T08:39:11.584899+0000","last_clean":"2024-09-10T08:39:11.584899+0000","last_became_active":"2024-09-10T08:37:42.670097+0000","last_became_peered":"2024-09-10T08:37:42.670097+0000","last_unstale":"2024-09-10T08:39:11.584899+0000","last_undegraded":"2024-09-10T08:39:11.584899+0000","last_fullsized":"2024-09-10T08:39:11.584899+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:14.956492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584527+0000","last_change":"2024-09-10T08:36:56.154850+0000","last_active":"2024-09-10T08:39:11.584527+0000","last_peered":"2024-09-10T08:39:11.584527+0000","last_clean":"2024-09-10T08:39:11.584527+0000","last_became_active":"2024-09-10T08:36:56.154545+0000","last_became_peered":"2024-09-10T08:36:56.154545+0000","last_unstale":"2024-09-10T08:39:11.584527+0000","last_undegraded":"2024-09-10T08:39:11.584527+0000","last_fullsized":"2024-09-10T08:39:11.584527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:24.639667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.569973+0000","last_change":"2024-09-10T08:37:42.669962+0000","last_active":"2024-09-10T08:39:11.569973+0000","last_peered":"2024-09-10T08:39:11.569973+0000","last_clean":"2024-09-10T08:39:11.569973+0000","last_became_active":"2024-09-10T08:37:42.669556+0000","last_became_peered":"2024-09-10T08:37:42.669556+0000","last_unstale":"2024-09-10T08:39:11.569973+0000","last_undegraded":"2024-09-10T08:39:11.569973+0000","last_fullsized":"2024-09-10T08:39:11.569973+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:23.333946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.578061+0000","last_change":"2024-09-10T08:36:56.145910+0000","last_active":"2024-09-10T08:39:11.578061+0000","last_peered":"2024-09-10T08:39:11.578061+0000","last_clean":"2024-09-10T08:39:11.578061+0000","last_became_active":"2024-09-10T08:36:56.145537+0000","last_became_peered":"2024-09-10T08:36:56.145537+0000","last_unstale":"2024-09-10T08:39:11.578061+0000","last_undegraded":"2024-09-10T08:39:11.578061+0000","last_fullsized":"2024-09-10T08:39:11.578061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:46.788241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567880+0000","last_change":"2024-09-10T08:36:56.147340+0000","last_active":"2024-09-10T08:39:11.567880+0000","last_peered":"2024-09-10T08:39:11.567880+0000","last_clean":"2024-09-10T08:39:11.567880+0000","last_became_active":"2024-09-10T08:36:56.145762+0000","last_became_peered":"2024-09-10T08:36:56.145762+0000","last_unstale":"2024-09-10T08:39:11.567880+0000","last_undegraded":"2024-09-10T08:39:11.567880+0000","last_fullsized":"2024-09-10T08:39:11.567880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:12.339285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.581431+0000","last_change":"2024-09-10T08:36:56.145213+0000","last_active":"2024-09-10T08:39:11.581431+0000","last_peered":"2024-09-10T08:39:11.581431+0000","last_clean":"2024-09-10T08:39:11.581431+0000","last_became_active":"2024-09-10T08:36:56.144915+0000","last_became_peered":"2024-09-10T08:36:56.144915+0000","last_unstale":"2024-09-10T08:39:11.581431+0000","last_undegraded":"2024-09-10T08:39:11.581431+0000","last_fullsized":"2024-09-10T08:39:11.581431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:06.180321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.574743+0000","last_change":"2024-09-10T08:36:56.149738+0000","last_active":"2024-09-10T08:39:11.574743+0000","last_peered":"2024-09-10T08:39:11.574743+0000","last_clean":"2024-09-10T08:39:11.574743+0000","last_became_active":"2024-09-10T08:36:56.147750+0000","last_became_peered":"2024-09-10T08:36:56.147750+0000","last_unstale":"2024-09-10T08:39:11.574743+0000","last_undegraded":"2024-09-10T08:39:11.574743+0000","last_fullsized":"2024-09-10T08:39:11.574743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:47.950054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.591937+0000","last_change":"2024-09-10T08:36:56.147682+0000","last_active":"2024-09-10T08:39:11.591937+0000","last_peered":"2024-09-10T08:39:11.591937+0000","last_clean":"2024-09-10T08:39:11.591937+0000","last_became_active":"2024-09-10T08:36:56.147345+0000","last_became_peered":"2024-09-10T08:36:56.147345+0000","last_unstale":"2024-09-10T08:39:11.591937+0000","last_undegraded":"2024-09-10T08:39:11.591937+0000","last_fullsized":"2024-09-10T08:39:11.591937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:54.939482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583378+0000","last_change":"2024-09-10T08:36:56.158292+0000","last_active":"2024-09-10T08:39:11.583378+0000","last_peered":"2024-09-10T08:39:11.583378+0000","last_clean":"2024-09-10T08:39:11.583378+0000","last_became_active":"2024-09-10T08:36:56.157970+0000","last_became_peered":"2024-09-10T08:36:56.157970+0000","last_unstale":"2024-09-10T08:39:11.583378+0000","last_undegraded":"2024-09-10T08:39:11.583378+0000","last_fullsized":"2024-09-10T08:39:11.583378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:33.582126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567916+0000","last_change":"2024-09-10T08:36:56.142520+0000","last_active":"2024-09-10T08:39:11.567916+0000","last_peered":"2024-09-10T08:39:11.567916+0000","last_clean":"2024-09-10T08:39:11.567916+0000","last_became_active":"2024-09-10T08:36:56.142214+0000","last_became_peered":"2024-09-10T08:36:56.142214+0000","last_unstale":"2024-09-10T08:39:11.567916+0000","last_undegraded":"2024-09-10T08:39:11.567916+0000","last_fullsized":"2024-09-10T08:39:11.567916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:45:29.201839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584167+0000","last_change":"2024-09-10T08:36:56.149547+0000","last_active":"2024-09-10T08:39:11.584167+0000","last_peered":"2024-09-10T08:39:11.584167+0000","last_clean":"2024-09-10T08:39:11.584167+0000","last_became_active":"2024-09-10T08:36:56.149141+0000","last_became_peered":"2024-09-10T08:36:56.149141+0000","last_unstale":"2024-09-10T08:39:11.584167+0000","last_undegraded":"2024-09-10T08:39:11.584167+0000","last_fullsized":"2024-09-10T08:39:11.584167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:49.368677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585864+0000","last_change":"2024-09-10T08:36:56.143087+0000","last_active":"2024-09-10T08:39:11.585864+0000","last_peered":"2024-09-10T08:39:11.585864+0000","last_clean":"2024-09-10T08:39:11.585864+0000","last_became_active":"2024-09-10T08:36:56.142782+0000","last_became_peered":"2024-09-10T08:36:56.142782+0000","last_unstale":"2024-09-10T08:39:11.585864+0000","last_undegraded":"2024-09-10T08:39:11.585864+0000","last_fullsized":"2024-09-10T08:39:11.585864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:17.513103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592876+0000","last_change":"2024-09-10T08:36:56.157549+0000","last_active":"2024-09-10T08:39:11.592876+0000","last_peered":"2024-09-10T08:39:11.592876+0000","last_clean":"2024-09-10T08:39:11.592876+0000","last_became_active":"2024-09-10T08:36:56.157221+0000","last_became_peered":"2024-09-10T08:36:56.157221+0000","last_unstale":"2024-09-10T08:39:11.592876+0000","last_undegraded":"2024-09-10T08:39:11.592876+0000","last_fullsized":"2024-09-10T08:39:11.592876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:11.372085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593199+0000","last_change":"2024-09-10T08:36:56.145654+0000","last_active":"2024-09-10T08:39:11.593199+0000","last_peered":"2024-09-10T08:39:11.593199+0000","last_clean":"2024-09-10T08:39:11.593199+0000","last_became_active":"2024-09-10T08:36:56.144126+0000","last_became_peered":"2024-09-10T08:36:56.144126+0000","last_unstale":"2024-09-10T08:39:11.593199+0000","last_undegraded":"2024-09-10T08:39:11.593199+0000","last_fullsized":"2024-09-10T08:39:11.593199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:55.498873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.582479+0000","last_change":"2024-09-10T08:36:56.146423+0000","last_active":"2024-09-10T08:39:11.582479+0000","last_peered":"2024-09-10T08:39:11.582479+0000","last_clean":"2024-09-10T08:39:11.582479+0000","last_became_active":"2024-09-10T08:36:56.146126+0000","last_became_peered":"2024-09-10T08:36:56.146126+0000","last_unstale":"2024-09-10T08:39:11.582479+0000","last_undegraded":"2024-09-10T08:39:11.582479+0000","last_fullsized":"2024-09-10T08:39:11.582479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:44.665196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583332+0000","last_change":"2024-09-10T08:36:56.151973+0000","last_active":"2024-09-10T08:39:11.583332+0000","last_peered":"2024-09-10T08:39:11.583332+0000","last_clean":"2024-09-10T08:39:11.583332+0000","last_became_active":"2024-09-10T08:36:56.151609+0000","last_became_peered":"2024-09-10T08:36:56.151609+0000","last_unstale":"2024-09-10T08:39:11.583332+0000","last_undegraded":"2024-09-10T08:39:11.583332+0000","last_fullsized":"2024-09-10T08:39:11.583332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:02.584047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584509+0000","last_change":"2024-09-10T08:36:56.141780+0000","last_active":"2024-09-10T08:39:11.584509+0000","last_peered":"2024-09-10T08:39:11.584509+0000","last_clean":"2024-09-10T08:39:11.584509+0000","last_became_active":"2024-09-10T08:36:56.141228+0000","last_became_peered":"2024-09-10T08:36:56.141228+0000","last_unstale":"2024-09-10T08:39:11.584509+0000","last_undegraded":"2024-09-10T08:39:11.584509+0000","last_fullsized":"2024-09-10T08:39:11.584509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:52:04.374228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585477+0000","last_change":"2024-09-10T08:37:42.671346+0000","last_active":"2024-09-10T08:39:11.585477+0000","last_peered":"2024-09-10T08:39:11.585477+0000","last_clean":"2024-09-10T08:39:11.585477+0000","last_became_active":"2024-09-10T08:37:42.670895+0000","last_became_peered":"2024-09-10T08:37:42.670895+0000","last_unstale":"2024-09-10T08:39:11.585477+0000","last_undegraded":"2024-09-10T08:39:11.585477+0000","last_fullsized":"2024-09-10T08:39:11.585477+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:16:31.930628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593796+0000","last_change":"2024-09-10T08:36:56.149014+0000","last_active":"2024-09-10T08:39:11.593796+0000","last_peered":"2024-09-10T08:39:11.593796+0000","last_clean":"2024-09-10T08:39:11.593796+0000","last_became_active":"2024-09-10T08:36:56.148551+0000","last_became_peered":"2024-09-10T08:36:56.148551+0000","last_unstale":"2024-09-10T08:39:11.593796+0000","last_undegraded":"2024-09-10T08:39:11.593796+0000","last_fullsized":"2024-09-10T08:39:11.593796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:35:52.250943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.586709+0000","last_change":"2024-09-10T08:36:56.141247+0000","last_active":"2024-09-10T08:39:11.586709+0000","last_peered":"2024-09-10T08:39:11.586709+0000","last_clean":"2024-09-10T08:39:11.586709+0000","last_became_active":"2024-09-10T08:36:56.140956+0000","last_became_peered":"2024-09-10T08:36:56.140956+0000","last_unstale":"2024-09-10T08:39:11.586709+0000","last_undegraded":"2024-09-10T08:39:11.586709+0000","last_fullsized":"2024-09-10T08:39:11.586709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:45.008424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592783+0000","last_change":"2024-09-10T08:36:56.149116+0000","last_active":"2024-09-10T08:39:11.592783+0000","last_peered":"2024-09-10T08:39:11.592783+0000","last_clean":"2024-09-10T08:39:11.592783+0000","last_became_active":"2024-09-10T08:36:56.148783+0000","last_became_peered":"2024-09-10T08:36:56.148783+0000","last_unstale":"2024-09-10T08:39:11.592783+0000","last_undegraded":"2024-09-10T08:39:11.592783+0000","last_fullsized":"2024-09-10T08:39:11.592783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:37.119214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585754+0000","last_change":"2024-09-10T08:37:42.846322+0000","last_active":"2024-09-10T08:39:11.585754+0000","last_peered":"2024-09-10T08:39:11.585754+0000","last_clean":"2024-09-10T08:39:11.585754+0000","last_became_active":"2024-09-10T08:37:42.846022+0000","last_became_peered":"2024-09-10T08:37:42.846022+0000","last_unstale":"2024-09-10T08:39:11.585754+0000","last_undegraded":"2024-09-10T08:39:11.585754+0000","last_fullsized":"2024-09-10T08:39:11.585754+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:38.142362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588751+0000","last_change":"2024-09-10T08:36:56.148537+0000","last_active":"2024-09-10T08:39:11.588751+0000","last_peered":"2024-09-10T08:39:11.588751+0000","last_clean":"2024-09-10T08:39:11.588751+0000","last_became_active":"2024-09-10T08:36:56.148185+0000","last_became_peered":"2024-09-10T08:36:56.148185+0000","last_unstale":"2024-09-10T08:39:11.588751+0000","last_undegraded":"2024-09-10T08:39:11.588751+0000","last_fullsized":"2024-09-10T08:39:11.588751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:00.426513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.586233+0000","last_change":"2024-09-10T08:36:56.138513+0000","last_active":"2024-09-10T08:39:11.586233+0000","last_peered":"2024-09-10T08:39:11.586233+0000","last_clean":"2024-09-10T08:39:11.586233+0000","last_became_active":"2024-09-10T08:36:56.138207+0000","last_became_peered":"2024-09-10T08:36:56.138207+0000","last_unstale":"2024-09-10T08:39:11.586233+0000","last_undegraded":"2024-09-10T08:39:11.586233+0000","last_fullsized":"2024-09-10T08:39:11.586233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:34.581430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.582686+0000","last_change":"2024-09-10T08:36:56.146612+0000","last_active":"2024-09-10T08:39:11.582686+0000","last_peered":"2024-09-10T08:39:11.582686+0000","last_clean":"2024-09-10T08:39:11.582686+0000","last_became_active":"2024-09-10T08:36:56.146251+0000","last_became_peered":"2024-09-10T08:36:56.146251+0000","last_unstale":"2024-09-10T08:39:11.582686+0000","last_undegraded":"2024-09-10T08:39:11.582686+0000","last_fullsized":"2024-09-10T08:39:11.582686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:47.336389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567939+0000","last_change":"2024-09-10T08:36:56.148260+0000","last_active":"2024-09-10T08:39:11.567939+0000","last_peered":"2024-09-10T08:39:11.567939+0000","last_clean":"2024-09-10T08:39:11.567939+0000","last_became_active":"2024-09-10T08:36:56.147949+0000","last_became_peered":"2024-09-10T08:36:56.147949+0000","last_unstale":"2024-09-10T08:39:11.567939+0000","last_undegraded":"2024-09-10T08:39:11.567939+0000","last_fullsized":"2024-09-10T08:39:11.567939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:52:19.762297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592986+0000","last_change":"2024-09-10T08:36:56.156045+0000","last_active":"2024-09-10T08:39:11.592986+0000","last_peered":"2024-09-10T08:39:11.592986+0000","last_clean":"2024-09-10T08:39:11.592986+0000","last_became_active":"2024-09-10T08:36:56.154727+0000","last_became_peered":"2024-09-10T08:36:56.154727+0000","last_unstale":"2024-09-10T08:39:11.592986+0000","last_undegraded":"2024-09-10T08:39:11.592986+0000","last_fullsized":"2024-09-10T08:39:11.592986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:35:39.992683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.576067+0000","last_change":"2024-09-10T08:36:56.154394+0000","last_active":"2024-09-10T08:39:11.576067+0000","last_peered":"2024-09-10T08:39:11.576067+0000","last_clean":"2024-09-10T08:39:11.576067+0000","last_became_active":"2024-09-10T08:36:56.153855+0000","last_became_peered":"2024-09-10T08:36:56.153855+0000","last_unstale":"2024-09-10T08:39:11.576067+0000","last_undegraded":"2024-09-10T08:39:11.576067+0000","last_fullsized":"2024-09-10T08:39:11.576067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:14.987147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592634+0000","last_change":"2024-09-10T08:36:56.157517+0000","last_active":"2024-09-10T08:39:11.592634+0000","last_peered":"2024-09-10T08:39:11.592634+0000","last_clean":"2024-09-10T08:39:11.592634+0000","last_became_active":"2024-09-10T08:36:56.157133+0000","last_became_peered":"2024-09-10T08:36:56.157133+0000","last_unstale":"2024-09-10T08:39:11.592634+0000","last_undegraded":"2024-09-10T08:39:11.592634+0000","last_fullsized":"2024-09-10T08:39:11.592634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:17:29.573719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69312,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":699,"num_read_kb":422,"num_write":186,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":216957,"data_compressed":5667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282455,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416432,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":547326,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678068,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28204,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716948,"statfs":{"total":95995035648,"available":95966154752,"internally_reserved":0,"allocated":1208320,"data_stored":1207391,"data_compressed":67339,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841594,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":543466,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103232,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":546900,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332165,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":1144631,"data_compressed":65450,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495690,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":542738,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757328,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":1138899,"data_compressed":65450,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920852,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":548897,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149785,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":542932,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":607798,"data_compressed":31413,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1126400,"data_stored":612008,"data_compressed":31413,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1077248,"data_stored":544126,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69360,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":70250,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":66183,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:39:44.894 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-10T08:39:44.894 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T08:39:45.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:44 smithi111 ceph-mon[35220]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-10T08:39:45.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:44 smithi089 ceph-mon[30757]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-10T08:39:45.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.084+0000 7fac57600640 1 -- 172.21.15.89:0/3599675814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac50103d30 msgr2=0x7fac50104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:45.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.084+0000 7fac57600640 1 --2- 172.21.15.89:0/3599675814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac50103d30 0x7fac50104130 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fac44009860 tx=0x7fac4402a340 comp rx=0 tx=0).stop 2024-09-10T08:39:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.084+0000 7fac57600640 1 -- 172.21.15.89:0/3599675814 shutdown_connections 2024-09-10T08:39:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.084+0000 7fac57600640 1 --2- 172.21.15.89:0/3599675814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac501058e0 0x7fac5010c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.084+0000 7fac57600640 1 --2- 172.21.15.89:0/3599675814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac50104f20 0x7fac501053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.084+0000 7fac57600640 1 --2- 172.21.15.89:0/3599675814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac50103d30 0x7fac50104130 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.084+0000 7fac57600640 1 -- 172.21.15.89:0/3599675814 >> 172.21.15.89:0/3599675814 conn(0x7fac500ff4b0 msgr2=0x7fac501018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.085+0000 7fac57600640 1 -- 172.21.15.89:0/3599675814 shutdown_connections 2024-09-10T08:39:45.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.085+0000 7fac57600640 1 -- 172.21.15.89:0/3599675814 wait complete. 2024-09-10T08:39:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.085+0000 7fac57600640 1 Processor -- start 2024-09-10T08:39:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.086+0000 7fac57600640 1 -- start start 2024-09-10T08:39:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.086+0000 7fac57600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac50103d30 0x7fac5019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.086+0000 7fac57600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac50104f20 0x7fac5019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.086+0000 7fac57600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac501058e0 0x7fac501a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.086+0000 7fac57600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fac5010f9b0 con 0x7fac501058e0 2024-09-10T08:39:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.086+0000 7fac57600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fac5010f830 con 0x7fac50103d30 2024-09-10T08:39:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.086+0000 7fac57600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fac5010fb30 con 0x7fac50104f20 2024-09-10T08:39:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.086+0000 7fac55600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac501058e0 0x7fac501a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.086+0000 7fac55600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac501058e0 0x7fac501a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59660/0 (socket says 172.21.15.89:59660) 2024-09-10T08:39:45.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.086+0000 7fac55600640 1 -- 172.21.15.89:0/4022478558 learned_addr learned my addr 172.21.15.89:0/4022478558 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.086+0000 7fac54c00640 1 --2- 172.21.15.89:0/4022478558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac50103d30 0x7fac5019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.086+0000 7fac4fe00640 1 --2- 172.21.15.89:0/4022478558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac50104f20 0x7fac5019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.087+0000 7fac55600640 1 -- 172.21.15.89:0/4022478558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac50104f20 msgr2=0x7fac5019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.087+0000 7fac55600640 1 --2- 172.21.15.89:0/4022478558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac50104f20 0x7fac5019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.087+0000 7fac55600640 1 -- 172.21.15.89:0/4022478558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac50103d30 msgr2=0x7fac5019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.087+0000 7fac55600640 1 --2- 172.21.15.89:0/4022478558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac50103d30 0x7fac5019a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.087+0000 7fac55600640 1 -- 172.21.15.89:0/4022478558 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac501a1cc0 con 0x7fac501058e0 2024-09-10T08:39:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.087+0000 7fac4fe00640 1 --2- 172.21.15.89:0/4022478558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac50104f20 0x7fac5019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:39:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.087+0000 7fac54c00640 1 --2- 172.21.15.89:0/4022478558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac50103d30 0x7fac5019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.087+0000 7fac55600640 1 --2- 172.21.15.89:0/4022478558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac501058e0 0x7fac501a1650 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fac4800efc0 tx=0x7fac4800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.087+0000 7fac4d600640 1 -- 172.21.15.89:0/4022478558 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48003970 con 0x7fac501058e0 2024-09-10T08:39:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.087+0000 7fac4d600640 1 -- 172.21.15.89:0/4022478558 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac48004020 con 0x7fac501058e0 2024-09-10T08:39:45.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.087+0000 7fac57600640 1 -- 172.21.15.89:0/4022478558 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac501a1fb0 con 0x7fac501058e0 2024-09-10T08:39:45.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.087+0000 7fac4d600640 1 -- 172.21.15.89:0/4022478558 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4800b9f0 con 0x7fac501058e0 2024-09-10T08:39:45.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.087+0000 7fac57600640 1 -- 172.21.15.89:0/4022478558 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac501a2420 con 0x7fac501058e0 2024-09-10T08:39:45.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.088+0000 7fac57600640 1 -- 172.21.15.89:0/4022478558 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac5010b190 con 0x7fac501058e0 2024-09-10T08:39:45.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.090+0000 7fac4d600640 1 -- 172.21.15.89:0/4022478558 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fac4801d020 con 0x7fac501058e0 2024-09-10T08:39:45.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.091+0000 7fac4d600640 1 --2- 172.21.15.89:0/4022478558 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fac380a2c20 0x7fac380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:45.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.091+0000 7fac4d600640 1 -- 172.21.15.89:0/4022478558 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fac480cf520 con 0x7fac501058e0 2024-09-10T08:39:45.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.092+0000 7fac54c00640 1 --2- 172.21.15.89:0/4022478558 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fac380a2c20 0x7fac380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:45.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.092+0000 7fac54c00640 1 --2- 172.21.15.89:0/4022478558 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fac380a2c20 0x7fac380a50e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fac44009830 tx=0x7fac44030570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:45.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.095+0000 7fac4d600640 1 -- 172.21.15.89:0/4022478558 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac4809bef0 con 0x7fac501058e0 2024-09-10T08:39:45.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:44 smithi149 ceph-mon[36167]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-10T08:39:45.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.270+0000 7fac57600640 1 -- 172.21.15.89:0/4022478558 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fac50062630 con 0x7fac380a2c20 2024-09-10T08:39:45.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.278+0000 7fac4d600640 1 -- 172.21.15.89:0/4022478558 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340807 (secure 0 0 0) 0x7fac50062630 con 0x7fac380a2c20 2024-09-10T08:39:45.280 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:39:45.284 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-09-10T08:39:45.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.284+0000 7fac57600640 1 -- 172.21.15.89:0/4022478558 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fac380a2c20 msgr2=0x7fac380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:45.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.284+0000 7fac57600640 1 --2- 172.21.15.89:0/4022478558 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fac380a2c20 0x7fac380a50e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fac44009830 tx=0x7fac44030570 comp rx=0 tx=0).stop 2024-09-10T08:39:45.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.284+0000 7fac57600640 1 -- 172.21.15.89:0/4022478558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac501058e0 msgr2=0x7fac501a1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:45.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.284+0000 7fac57600640 1 --2- 172.21.15.89:0/4022478558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac501058e0 0x7fac501a1650 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fac4800efc0 tx=0x7fac4800c800 comp rx=0 tx=0).stop 2024-09-10T08:39:45.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.284+0000 7fac57600640 1 -- 172.21.15.89:0/4022478558 shutdown_connections 2024-09-10T08:39:45.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.284+0000 7fac57600640 1 --2- 172.21.15.89:0/4022478558 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fac380a2c20 0x7fac380a50e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:45.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.284+0000 7fac57600640 1 --2- 172.21.15.89:0/4022478558 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac501058e0 0x7fac501a1650 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:45.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.284+0000 7fac57600640 1 --2- 172.21.15.89:0/4022478558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac50104f20 0x7fac5019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:45.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.284+0000 7fac57600640 1 --2- 172.21.15.89:0/4022478558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac50103d30 0x7fac5019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:45.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.284+0000 7fac57600640 1 -- 172.21.15.89:0/4022478558 >> 172.21.15.89:0/4022478558 conn(0x7fac500ff4b0 msgr2=0x7fac50108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:45.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.284+0000 7fac57600640 1 -- 172.21.15.89:0/4022478558 shutdown_connections 2024-09-10T08:39:45.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:45.284+0000 7fac57600640 1 -- 172.21.15.89:0/4022478558 wait complete. 2024-09-10T08:39:45.301 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-09-10T08:39:43.612696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659914,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":825,"num_read_kb":531,"num_write":369,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181900,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":797,"ondisk_log_size":797,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357964,"kb_used_data":12876,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583860,"statfs":{"total":1151940427776,"available":1151573872640,"internally_reserved":0,"allocated":13185024,"data_stored":8527112,"data_compressed":467733,"data_compressed_allocated":638976,"data_compressed_original":2777088,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1408,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":32,"num_write":15,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003513"},"pg_stats":[{"pgid":"2.1f","version":"94'9","reported_seq":67,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.663591+0000","last_change":"2024-09-10T08:38:43.663079+0000","last_active":"2024-09-10T08:38:43.663591+0000","last_peered":"2024-09-10T08:38:43.663591+0000","last_clean":"2024-09-10T08:38:43.663591+0000","last_became_active":"2024-09-10T08:38:43.656506+0000","last_became_peered":"2024-09-10T08:38:43.656506+0000","last_unstale":"2024-09-10T08:38:43.663591+0000","last_undegraded":"2024-09-10T08:38:43.663591+0000","last_fullsized":"2024-09-10T08:38:43.663591+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:36.881564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.579344+0000","last_change":"2024-09-10T08:36:56.145816+0000","last_active":"2024-09-10T08:39:11.579344+0000","last_peered":"2024-09-10T08:39:11.579344+0000","last_clean":"2024-09-10T08:39:11.579344+0000","last_became_active":"2024-09-10T08:36:56.145462+0000","last_became_peered":"2024-09-10T08:36:56.145462+0000","last_unstale":"2024-09-10T08:39:11.579344+0000","last_undegraded":"2024-09-10T08:39:11.579344+0000","last_fullsized":"2024-09-10T08:39:11.579344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:53.890190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629879+0000","last_change":"2024-09-10T08:37:05.252761+0000","last_active":"2024-09-10T08:38:43.629879+0000","last_peered":"2024-09-10T08:38:43.629879+0000","last_clean":"2024-09-10T08:38:43.629879+0000","last_became_active":"2024-09-10T08:37:05.252179+0000","last_became_peered":"2024-09-10T08:37:05.252179+0000","last_unstale":"2024-09-10T08:38:43.629879+0000","last_undegraded":"2024-09-10T08:38:43.629879+0000","last_fullsized":"2024-09-10T08:38:43.629879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:52.854828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631850+0000","last_change":"2024-09-10T08:36:53.735650+0000","last_active":"2024-09-10T08:38:43.631850+0000","last_peered":"2024-09-10T08:38:43.631850+0000","last_clean":"2024-09-10T08:38:43.631850+0000","last_became_active":"2024-09-10T08:36:53.735128+0000","last_became_peered":"2024-09-10T08:36:53.735128+0000","last_unstale":"2024-09-10T08:38:43.631850+0000","last_undegraded":"2024-09-10T08:38:43.631850+0000","last_fullsized":"2024-09-10T08:38:43.631850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:27.309591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592923+0000","last_change":"2024-09-10T08:36:56.153364+0000","last_active":"2024-09-10T08:39:11.592923+0000","last_peered":"2024-09-10T08:39:11.592923+0000","last_clean":"2024-09-10T08:39:11.592923+0000","last_became_active":"2024-09-10T08:36:56.152979+0000","last_became_peered":"2024-09-10T08:36:56.152979+0000","last_unstale":"2024-09-10T08:39:11.592923+0000","last_undegraded":"2024-09-10T08:39:11.592923+0000","last_fullsized":"2024-09-10T08:39:11.592923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:12.172620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654774+0000","last_change":"2024-09-10T08:37:05.254551+0000","last_active":"2024-09-10T08:38:43.654774+0000","last_peered":"2024-09-10T08:38:43.654774+0000","last_clean":"2024-09-10T08:38:43.654774+0000","last_became_active":"2024-09-10T08:37:05.254055+0000","last_became_peered":"2024-09-10T08:37:05.254055+0000","last_unstale":"2024-09-10T08:38:43.654774+0000","last_undegraded":"2024-09-10T08:38:43.654774+0000","last_fullsized":"2024-09-10T08:38:43.654774+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:08.472502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626522+0000","last_change":"2024-09-10T08:36:53.722774+0000","last_active":"2024-09-10T08:38:43.626522+0000","last_peered":"2024-09-10T08:38:43.626522+0000","last_clean":"2024-09-10T08:38:43.626522+0000","last_became_active":"2024-09-10T08:36:53.722459+0000","last_became_peered":"2024-09-10T08:36:53.722459+0000","last_unstale":"2024-09-10T08:38:43.626522+0000","last_undegraded":"2024-09-10T08:38:43.626522+0000","last_fullsized":"2024-09-10T08:38:43.626522+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:26.612354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592240+0000","last_change":"2024-09-10T08:36:56.143757+0000","last_active":"2024-09-10T08:39:11.592240+0000","last_peered":"2024-09-10T08:39:11.592240+0000","last_clean":"2024-09-10T08:39:11.592240+0000","last_became_active":"2024-09-10T08:36:56.143430+0000","last_became_peered":"2024-09-10T08:36:56.143430+0000","last_unstale":"2024-09-10T08:39:11.592240+0000","last_undegraded":"2024-09-10T08:39:11.592240+0000","last_fullsized":"2024-09-10T08:39:11.592240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:49.737298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655120+0000","last_change":"2024-09-10T08:37:05.254503+0000","last_active":"2024-09-10T08:38:43.655120+0000","last_peered":"2024-09-10T08:38:43.655120+0000","last_clean":"2024-09-10T08:38:43.655120+0000","last_became_active":"2024-09-10T08:37:05.254070+0000","last_became_peered":"2024-09-10T08:37:05.254070+0000","last_unstale":"2024-09-10T08:38:43.655120+0000","last_undegraded":"2024-09-10T08:38:43.655120+0000","last_fullsized":"2024-09-10T08:38:43.655120+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:16.741698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626438+0000","last_change":"2024-09-10T08:36:53.716742+0000","last_active":"2024-09-10T08:38:43.626438+0000","last_peered":"2024-09-10T08:38:43.626438+0000","last_clean":"2024-09-10T08:38:43.626438+0000","last_became_active":"2024-09-10T08:36:53.716439+0000","last_became_peered":"2024-09-10T08:36:53.716439+0000","last_unstale":"2024-09-10T08:38:43.626438+0000","last_undegraded":"2024-09-10T08:38:43.626438+0000","last_fullsized":"2024-09-10T08:38:43.626438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:46.882805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.581828+0000","last_change":"2024-09-10T08:36:56.146841+0000","last_active":"2024-09-10T08:39:11.581828+0000","last_peered":"2024-09-10T08:39:11.581828+0000","last_clean":"2024-09-10T08:39:11.581828+0000","last_became_active":"2024-09-10T08:36:56.146531+0000","last_became_peered":"2024-09-10T08:36:56.146531+0000","last_unstale":"2024-09-10T08:39:11.581828+0000","last_undegraded":"2024-09-10T08:39:11.581828+0000","last_fullsized":"2024-09-10T08:39:11.581828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:37.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654877+0000","last_change":"2024-09-10T08:37:43.223181+0000","last_active":"2024-09-10T08:38:43.654877+0000","last_peered":"2024-09-10T08:38:43.654877+0000","last_clean":"2024-09-10T08:38:43.654877+0000","last_became_active":"2024-09-10T08:37:43.222794+0000","last_became_peered":"2024-09-10T08:37:43.222794+0000","last_unstale":"2024-09-10T08:38:43.654877+0000","last_undegraded":"2024-09-10T08:38:43.654877+0000","last_fullsized":"2024-09-10T08:38:43.654877+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:22.204517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593593+0000","last_change":"2024-09-10T08:36:56.142161+0000","last_active":"2024-09-10T08:39:11.593593+0000","last_peered":"2024-09-10T08:39:11.593593+0000","last_clean":"2024-09-10T08:39:11.593593+0000","last_became_active":"2024-09-10T08:36:56.141838+0000","last_became_peered":"2024-09-10T08:36:56.141838+0000","last_unstale":"2024-09-10T08:39:11.593593+0000","last_undegraded":"2024-09-10T08:39:11.593593+0000","last_fullsized":"2024-09-10T08:39:11.593593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:09.909747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626125+0000","last_change":"2024-09-10T08:36:53.722238+0000","last_active":"2024-09-10T08:38:43.626125+0000","last_peered":"2024-09-10T08:38:43.626125+0000","last_clean":"2024-09-10T08:38:43.626125+0000","last_became_active":"2024-09-10T08:36:53.721859+0000","last_became_peered":"2024-09-10T08:36:53.721859+0000","last_unstale":"2024-09-10T08:38:43.626125+0000","last_undegraded":"2024-09-10T08:38:43.626125+0000","last_fullsized":"2024-09-10T08:38:43.626125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:50.615090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655183+0000","last_change":"2024-09-10T08:37:05.414041+0000","last_active":"2024-09-10T08:38:43.655183+0000","last_peered":"2024-09-10T08:38:43.655183+0000","last_clean":"2024-09-10T08:38:43.655183+0000","last_became_active":"2024-09-10T08:37:05.413286+0000","last_became_peered":"2024-09-10T08:37:05.413286+0000","last_unstale":"2024-09-10T08:38:43.655183+0000","last_undegraded":"2024-09-10T08:38:43.655183+0000","last_fullsized":"2024-09-10T08:38:43.655183+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:44.916102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.572936+0000","last_change":"2024-09-10T08:36:56.158278+0000","last_active":"2024-09-10T08:39:11.572936+0000","last_peered":"2024-09-10T08:39:11.572936+0000","last_clean":"2024-09-10T08:39:11.572936+0000","last_became_active":"2024-09-10T08:36:56.157946+0000","last_became_peered":"2024-09-10T08:36:56.157946+0000","last_unstale":"2024-09-10T08:39:11.572936+0000","last_undegraded":"2024-09-10T08:39:11.572936+0000","last_fullsized":"2024-09-10T08:39:11.572936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:42.512513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"96'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:38.713339+0000","last_change":"2024-09-10T08:38:43.635077+0000","last_active":"2024-09-10T08:39:38.713339+0000","last_peered":"2024-09-10T08:39:38.713339+0000","last_clean":"2024-09-10T08:39:38.713339+0000","last_became_active":"2024-09-10T08:38:43.631957+0000","last_became_peered":"2024-09-10T08:38:43.631957+0000","last_unstale":"2024-09-10T08:39:38.713339+0000","last_undegraded":"2024-09-10T08:39:38.713339+0000","last_fullsized":"2024-09-10T08:39:38.713339+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:50.837247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655126+0000","last_change":"2024-09-10T08:37:42.671399+0000","last_active":"2024-09-10T08:38:43.655126+0000","last_peered":"2024-09-10T08:38:43.655126+0000","last_clean":"2024-09-10T08:38:43.655126+0000","last_became_active":"2024-09-10T08:37:42.670865+0000","last_became_peered":"2024-09-10T08:37:42.670865+0000","last_unstale":"2024-09-10T08:38:43.655126+0000","last_undegraded":"2024-09-10T08:38:43.655126+0000","last_fullsized":"2024-09-10T08:38:43.655126+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:16.231538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.577960+0000","last_change":"2024-09-10T08:36:56.144856+0000","last_active":"2024-09-10T08:39:11.577960+0000","last_peered":"2024-09-10T08:39:11.577960+0000","last_clean":"2024-09-10T08:39:11.577960+0000","last_became_active":"2024-09-10T08:36:56.144531+0000","last_became_peered":"2024-09-10T08:36:56.144531+0000","last_unstale":"2024-09-10T08:39:11.577960+0000","last_undegraded":"2024-09-10T08:39:11.577960+0000","last_fullsized":"2024-09-10T08:39:11.577960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:24.586006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631541+0000","last_change":"2024-09-10T08:36:53.721093+0000","last_active":"2024-09-10T08:38:43.631541+0000","last_peered":"2024-09-10T08:38:43.631541+0000","last_clean":"2024-09-10T08:38:43.631541+0000","last_became_active":"2024-09-10T08:36:53.720538+0000","last_became_peered":"2024-09-10T08:36:53.720538+0000","last_unstale":"2024-09-10T08:38:43.631541+0000","last_undegraded":"2024-09-10T08:38:43.631541+0000","last_fullsized":"2024-09-10T08:38:43.631541+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:58.492463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629902+0000","last_change":"2024-09-10T08:37:05.254011+0000","last_active":"2024-09-10T08:38:43.629902+0000","last_peered":"2024-09-10T08:38:43.629902+0000","last_clean":"2024-09-10T08:38:43.629902+0000","last_became_active":"2024-09-10T08:37:05.253066+0000","last_became_peered":"2024-09-10T08:37:05.253066+0000","last_unstale":"2024-09-10T08:38:43.629902+0000","last_undegraded":"2024-09-10T08:38:43.629902+0000","last_fullsized":"2024-09-10T08:38:43.629902+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:16.221302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.553299+0000","last_change":"2024-09-10T08:36:56.141814+0000","last_active":"2024-09-10T08:39:11.553299+0000","last_peered":"2024-09-10T08:39:11.553299+0000","last_clean":"2024-09-10T08:39:11.553299+0000","last_became_active":"2024-09-10T08:36:56.141480+0000","last_became_peered":"2024-09-10T08:36:56.141480+0000","last_unstale":"2024-09-10T08:39:11.553299+0000","last_undegraded":"2024-09-10T08:39:11.553299+0000","last_fullsized":"2024-09-10T08:39:11.553299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:40.594354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.636347+0000","last_change":"2024-09-10T08:38:43.635917+0000","last_active":"2024-09-10T08:38:43.636347+0000","last_peered":"2024-09-10T08:38:43.636347+0000","last_clean":"2024-09-10T08:38:43.636347+0000","last_became_active":"2024-09-10T08:38:43.633406+0000","last_became_peered":"2024-09-10T08:38:43.633406+0000","last_unstale":"2024-09-10T08:38:43.636347+0000","last_undegraded":"2024-09-10T08:38:43.636347+0000","last_fullsized":"2024-09-10T08:38:43.636347+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:44.619072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655297+0000","last_change":"2024-09-10T08:37:05.414181+0000","last_active":"2024-09-10T08:38:43.655297+0000","last_peered":"2024-09-10T08:38:43.655297+0000","last_clean":"2024-09-10T08:38:43.655297+0000","last_became_active":"2024-09-10T08:37:05.413691+0000","last_became_peered":"2024-09-10T08:37:05.413691+0000","last_unstale":"2024-09-10T08:38:43.655297+0000","last_undegraded":"2024-09-10T08:38:43.655297+0000","last_fullsized":"2024-09-10T08:38:43.655297+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:30.744949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584567+0000","last_change":"2024-09-10T08:36:56.153956+0000","last_active":"2024-09-10T08:39:11.584567+0000","last_peered":"2024-09-10T08:39:11.584567+0000","last_clean":"2024-09-10T08:39:11.584567+0000","last_became_active":"2024-09-10T08:36:56.153609+0000","last_became_peered":"2024-09-10T08:36:56.153609+0000","last_unstale":"2024-09-10T08:39:11.584567+0000","last_undegraded":"2024-09-10T08:39:11.584567+0000","last_fullsized":"2024-09-10T08:39:11.584567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:18.185704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655520+0000","last_change":"2024-09-10T08:36:53.733217+0000","last_active":"2024-09-10T08:38:43.655520+0000","last_peered":"2024-09-10T08:38:43.655520+0000","last_clean":"2024-09-10T08:38:43.655520+0000","last_became_active":"2024-09-10T08:36:53.732851+0000","last_became_peered":"2024-09-10T08:36:53.732851+0000","last_unstale":"2024-09-10T08:38:43.655520+0000","last_undegraded":"2024-09-10T08:38:43.655520+0000","last_fullsized":"2024-09-10T08:38:43.655520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:04:01.132196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629889+0000","last_change":"2024-09-10T08:37:05.241868+0000","last_active":"2024-09-10T08:38:43.629889+0000","last_peered":"2024-09-10T08:38:43.629889+0000","last_clean":"2024-09-10T08:38:43.629889+0000","last_became_active":"2024-09-10T08:37:05.241412+0000","last_became_peered":"2024-09-10T08:37:05.241412+0000","last_unstale":"2024-09-10T08:38:43.629889+0000","last_undegraded":"2024-09-10T08:38:43.629889+0000","last_fullsized":"2024-09-10T08:38:43.629889+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:52.543353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.575701+0000","last_change":"2024-09-10T08:36:56.150722+0000","last_active":"2024-09-10T08:39:11.575701+0000","last_peered":"2024-09-10T08:39:11.575701+0000","last_clean":"2024-09-10T08:39:11.575701+0000","last_became_active":"2024-09-10T08:36:56.150423+0000","last_became_peered":"2024-09-10T08:36:56.150423+0000","last_unstale":"2024-09-10T08:39:11.575701+0000","last_undegraded":"2024-09-10T08:39:11.575701+0000","last_fullsized":"2024-09-10T08:39:11.575701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:50.272841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'13","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:38.175435+0000","last_change":"2024-09-10T08:36:53.722477+0000","last_active":"2024-09-10T08:39:38.175435+0000","last_peered":"2024-09-10T08:39:38.175435+0000","last_clean":"2024-09-10T08:39:38.175435+0000","last_became_active":"2024-09-10T08:36:53.722118+0000","last_became_peered":"2024-09-10T08:36:53.722118+0000","last_unstale":"2024-09-10T08:39:38.175435+0000","last_undegraded":"2024-09-10T08:39:38.175435+0000","last_fullsized":"2024-09-10T08:39:38.175435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:06.631646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630992+0000","last_change":"2024-09-10T08:37:05.415656+0000","last_active":"2024-09-10T08:38:43.630992+0000","last_peered":"2024-09-10T08:38:43.630992+0000","last_clean":"2024-09-10T08:38:43.630992+0000","last_became_active":"2024-09-10T08:37:05.413537+0000","last_became_peered":"2024-09-10T08:37:05.413537+0000","last_unstale":"2024-09-10T08:38:43.630992+0000","last_undegraded":"2024-09-10T08:38:43.630992+0000","last_fullsized":"2024-09-10T08:38:43.630992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:04.284410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585069+0000","last_change":"2024-09-10T08:36:56.144157+0000","last_active":"2024-09-10T08:39:11.585069+0000","last_peered":"2024-09-10T08:39:11.585069+0000","last_clean":"2024-09-10T08:39:11.585069+0000","last_became_active":"2024-09-10T08:36:56.140332+0000","last_became_peered":"2024-09-10T08:36:56.140332+0000","last_unstale":"2024-09-10T08:39:11.585069+0000","last_undegraded":"2024-09-10T08:39:11.585069+0000","last_fullsized":"2024-09-10T08:39:11.585069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:09:26.097987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626517+0000","last_change":"2024-09-10T08:36:53.735465+0000","last_active":"2024-09-10T08:38:43.626517+0000","last_peered":"2024-09-10T08:38:43.626517+0000","last_clean":"2024-09-10T08:38:43.626517+0000","last_became_active":"2024-09-10T08:36:53.735160+0000","last_became_peered":"2024-09-10T08:36:53.735160+0000","last_unstale":"2024-09-10T08:38:43.626517+0000","last_undegraded":"2024-09-10T08:38:43.626517+0000","last_fullsized":"2024-09-10T08:38:43.626517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:26:52.332085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627959+0000","last_change":"2024-09-10T08:37:05.247671+0000","last_active":"2024-09-10T08:38:43.627959+0000","last_peered":"2024-09-10T08:38:43.627959+0000","last_clean":"2024-09-10T08:38:43.627959+0000","last_became_active":"2024-09-10T08:37:05.247217+0000","last_became_peered":"2024-09-10T08:37:05.247217+0000","last_unstale":"2024-09-10T08:38:43.627959+0000","last_undegraded":"2024-09-10T08:38:43.627959+0000","last_fullsized":"2024-09-10T08:38:43.627959+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:20.881304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.579998+0000","last_change":"2024-09-10T08:36:56.145420+0000","last_active":"2024-09-10T08:39:11.579998+0000","last_peered":"2024-09-10T08:39:11.579998+0000","last_clean":"2024-09-10T08:39:11.579998+0000","last_became_active":"2024-09-10T08:36:56.145074+0000","last_became_peered":"2024-09-10T08:36:56.145074+0000","last_unstale":"2024-09-10T08:39:11.579998+0000","last_undegraded":"2024-09-10T08:39:11.579998+0000","last_fullsized":"2024-09-10T08:39:11.579998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:13.849977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629812+0000","last_change":"2024-09-10T08:36:53.738237+0000","last_active":"2024-09-10T08:38:43.629812+0000","last_peered":"2024-09-10T08:38:43.629812+0000","last_clean":"2024-09-10T08:38:43.629812+0000","last_became_active":"2024-09-10T08:36:53.737858+0000","last_became_peered":"2024-09-10T08:36:53.737858+0000","last_unstale":"2024-09-10T08:38:43.629812+0000","last_undegraded":"2024-09-10T08:38:43.629812+0000","last_fullsized":"2024-09-10T08:38:43.629812+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:35.345661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.625982+0000","last_change":"2024-09-10T08:37:05.413512+0000","last_active":"2024-09-10T08:38:43.625982+0000","last_peered":"2024-09-10T08:38:43.625982+0000","last_clean":"2024-09-10T08:38:43.625982+0000","last_became_active":"2024-09-10T08:37:05.413010+0000","last_became_peered":"2024-09-10T08:37:05.413010+0000","last_unstale":"2024-09-10T08:38:43.625982+0000","last_undegraded":"2024-09-10T08:38:43.625982+0000","last_fullsized":"2024-09-10T08:38:43.625982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:05.349404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583802+0000","last_change":"2024-09-10T08:36:56.157741+0000","last_active":"2024-09-10T08:39:11.583802+0000","last_peered":"2024-09-10T08:39:11.583802+0000","last_clean":"2024-09-10T08:39:11.583802+0000","last_became_active":"2024-09-10T08:36:56.157295+0000","last_became_peered":"2024-09-10T08:36:56.157295+0000","last_unstale":"2024-09-10T08:39:11.583802+0000","last_undegraded":"2024-09-10T08:39:11.583802+0000","last_fullsized":"2024-09-10T08:39:11.583802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:31.845864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631834+0000","last_change":"2024-09-10T08:36:53.722373+0000","last_active":"2024-09-10T08:38:43.631834+0000","last_peered":"2024-09-10T08:38:43.631834+0000","last_clean":"2024-09-10T08:38:43.631834+0000","last_became_active":"2024-09-10T08:36:53.721975+0000","last_became_peered":"2024-09-10T08:36:53.721975+0000","last_unstale":"2024-09-10T08:38:43.631834+0000","last_undegraded":"2024-09-10T08:38:43.631834+0000","last_fullsized":"2024-09-10T08:38:43.631834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:44.593018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628351+0000","last_change":"2024-09-10T08:37:05.255362+0000","last_active":"2024-09-10T08:38:43.628351+0000","last_peered":"2024-09-10T08:38:43.628351+0000","last_clean":"2024-09-10T08:38:43.628351+0000","last_became_active":"2024-09-10T08:37:05.254967+0000","last_became_peered":"2024-09-10T08:37:05.254967+0000","last_unstale":"2024-09-10T08:38:43.628351+0000","last_undegraded":"2024-09-10T08:38:43.628351+0000","last_fullsized":"2024-09-10T08:38:43.628351+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:31:41.850961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.578715+0000","last_change":"2024-09-10T08:36:56.147699+0000","last_active":"2024-09-10T08:39:11.578715+0000","last_peered":"2024-09-10T08:39:11.578715+0000","last_clean":"2024-09-10T08:39:11.578715+0000","last_became_active":"2024-09-10T08:36:56.147349+0000","last_became_peered":"2024-09-10T08:36:56.147349+0000","last_unstale":"2024-09-10T08:39:11.578715+0000","last_undegraded":"2024-09-10T08:39:11.578715+0000","last_fullsized":"2024-09-10T08:39:11.578715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:25.248588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"96'6","reported_seq":159,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:39.886723+0000","last_change":"2024-09-10T08:36:53.736843+0000","last_active":"2024-09-10T08:39:39.886723+0000","last_peered":"2024-09-10T08:39:39.886723+0000","last_clean":"2024-09-10T08:39:39.886723+0000","last_became_active":"2024-09-10T08:36:53.736340+0000","last_became_peered":"2024-09-10T08:36:53.736340+0000","last_unstale":"2024-09-10T08:39:39.886723+0000","last_undegraded":"2024-09-10T08:39:39.886723+0000","last_fullsized":"2024-09-10T08:39:39.886723+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:24.679679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628575+0000","last_change":"2024-09-10T08:37:05.414321+0000","last_active":"2024-09-10T08:38:43.628575+0000","last_peered":"2024-09-10T08:38:43.628575+0000","last_clean":"2024-09-10T08:38:43.628575+0000","last_became_active":"2024-09-10T08:37:05.413868+0000","last_became_peered":"2024-09-10T08:37:05.413868+0000","last_unstale":"2024-09-10T08:38:43.628575+0000","last_undegraded":"2024-09-10T08:38:43.628575+0000","last_fullsized":"2024-09-10T08:38:43.628575+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:53.988077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583174+0000","last_change":"2024-09-10T08:36:56.141910+0000","last_active":"2024-09-10T08:39:11.583174+0000","last_peered":"2024-09-10T08:39:11.583174+0000","last_clean":"2024-09-10T08:39:11.583174+0000","last_became_active":"2024-09-10T08:36:56.141579+0000","last_became_peered":"2024-09-10T08:36:56.141579+0000","last_unstale":"2024-09-10T08:39:11.583174+0000","last_undegraded":"2024-09-10T08:39:11.583174+0000","last_fullsized":"2024-09-10T08:39:11.583174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:18.464831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631756+0000","last_change":"2024-09-10T08:36:53.720102+0000","last_active":"2024-09-10T08:38:43.631756+0000","last_peered":"2024-09-10T08:38:43.631756+0000","last_clean":"2024-09-10T08:38:43.631756+0000","last_became_active":"2024-09-10T08:36:53.719764+0000","last_became_peered":"2024-09-10T08:36:53.719764+0000","last_unstale":"2024-09-10T08:38:43.631756+0000","last_undegraded":"2024-09-10T08:38:43.631756+0000","last_fullsized":"2024-09-10T08:38:43.631756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:03.609338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655205+0000","last_change":"2024-09-10T08:37:05.413818+0000","last_active":"2024-09-10T08:38:43.655205+0000","last_peered":"2024-09-10T08:38:43.655205+0000","last_clean":"2024-09-10T08:38:43.655205+0000","last_became_active":"2024-09-10T08:37:05.413267+0000","last_became_peered":"2024-09-10T08:37:05.413267+0000","last_unstale":"2024-09-10T08:38:43.655205+0000","last_undegraded":"2024-09-10T08:38:43.655205+0000","last_fullsized":"2024-09-10T08:38:43.655205+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:54.309686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.590486+0000","last_change":"2024-09-10T08:36:56.149286+0000","last_active":"2024-09-10T08:39:11.590486+0000","last_peered":"2024-09-10T08:39:11.590486+0000","last_clean":"2024-09-10T08:39:11.590486+0000","last_became_active":"2024-09-10T08:36:56.146275+0000","last_became_peered":"2024-09-10T08:36:56.146275+0000","last_unstale":"2024-09-10T08:39:11.590486+0000","last_undegraded":"2024-09-10T08:39:11.590486+0000","last_fullsized":"2024-09-10T08:39:11.590486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:19.468224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655269+0000","last_change":"2024-09-10T08:36:53.719756+0000","last_active":"2024-09-10T08:38:43.655269+0000","last_peered":"2024-09-10T08:38:43.655269+0000","last_clean":"2024-09-10T08:38:43.655269+0000","last_became_active":"2024-09-10T08:36:53.719302+0000","last_became_peered":"2024-09-10T08:36:53.719302+0000","last_unstale":"2024-09-10T08:38:43.655269+0000","last_undegraded":"2024-09-10T08:38:43.655269+0000","last_fullsized":"2024-09-10T08:38:43.655269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:47.658081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631788+0000","last_change":"2024-09-10T08:35:39.150715+0000","last_active":"2024-09-10T08:38:43.631788+0000","last_peered":"2024-09-10T08:38:43.631788+0000","last_clean":"2024-09-10T08:38:43.631788+0000","last_became_active":"2024-09-10T08:35:39.150206+0000","last_became_peered":"2024-09-10T08:35:39.150206+0000","last_unstale":"2024-09-10T08:38:43.631788+0000","last_undegraded":"2024-09-10T08:38:43.631788+0000","last_fullsized":"2024-09-10T08:38:43.631788+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:32:02.110870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:32:02.110870+0000","last_clean_scrub_stamp":"2024-09-10T08:32:02.110870+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:20.762043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631998+0000","last_change":"2024-09-10T08:37:42.846547+0000","last_active":"2024-09-10T08:38:43.631998+0000","last_peered":"2024-09-10T08:38:43.631998+0000","last_clean":"2024-09-10T08:38:43.631998+0000","last_became_active":"2024-09-10T08:37:42.845884+0000","last_became_peered":"2024-09-10T08:37:42.845884+0000","last_unstale":"2024-09-10T08:38:43.631998+0000","last_undegraded":"2024-09-10T08:38:43.631998+0000","last_fullsized":"2024-09-10T08:38:43.631998+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:21.905133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593781+0000","last_change":"2024-09-10T08:36:56.144187+0000","last_active":"2024-09-10T08:39:11.593781+0000","last_peered":"2024-09-10T08:39:11.593781+0000","last_clean":"2024-09-10T08:39:11.593781+0000","last_became_active":"2024-09-10T08:36:56.143193+0000","last_became_peered":"2024-09-10T08:36:56.143193+0000","last_unstale":"2024-09-10T08:39:11.593781+0000","last_undegraded":"2024-09-10T08:39:11.593781+0000","last_fullsized":"2024-09-10T08:39:11.593781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:14:26.981884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.639660+0000","last_change":"2024-09-10T08:38:43.639286+0000","last_active":"2024-09-10T08:38:43.639660+0000","last_peered":"2024-09-10T08:38:43.639660+0000","last_clean":"2024-09-10T08:38:43.639660+0000","last_became_active":"2024-09-10T08:38:43.633552+0000","last_became_peered":"2024-09-10T08:38:43.633552+0000","last_unstale":"2024-09-10T08:38:43.639660+0000","last_undegraded":"2024-09-10T08:38:43.639660+0000","last_fullsized":"2024-09-10T08:38:43.639660+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:23.985190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630209+0000","last_change":"2024-09-10T08:37:05.242095+0000","last_active":"2024-09-10T08:38:43.630209+0000","last_peered":"2024-09-10T08:38:43.630209+0000","last_clean":"2024-09-10T08:38:43.630209+0000","last_became_active":"2024-09-10T08:37:05.241645+0000","last_became_peered":"2024-09-10T08:37:05.241645+0000","last_unstale":"2024-09-10T08:38:43.630209+0000","last_undegraded":"2024-09-10T08:38:43.630209+0000","last_fullsized":"2024-09-10T08:38:43.630209+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:30:49.637861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588835+0000","last_change":"2024-09-10T08:36:56.149396+0000","last_active":"2024-09-10T08:39:11.588835+0000","last_peered":"2024-09-10T08:39:11.588835+0000","last_clean":"2024-09-10T08:39:11.588835+0000","last_became_active":"2024-09-10T08:36:56.149079+0000","last_became_peered":"2024-09-10T08:36:56.149079+0000","last_unstale":"2024-09-10T08:39:11.588835+0000","last_undegraded":"2024-09-10T08:39:11.588835+0000","last_fullsized":"2024-09-10T08:39:11.588835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:44.665448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655467+0000","last_change":"2024-09-10T08:36:53.717950+0000","last_active":"2024-09-10T08:38:43.655467+0000","last_peered":"2024-09-10T08:38:43.655467+0000","last_clean":"2024-09-10T08:38:43.655467+0000","last_became_active":"2024-09-10T08:36:53.717668+0000","last_became_peered":"2024-09-10T08:36:53.717668+0000","last_unstale":"2024-09-10T08:38:43.655467+0000","last_undegraded":"2024-09-10T08:38:43.655467+0000","last_fullsized":"2024-09-10T08:38:43.655467+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:58.692631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633140+0000","last_change":"2024-09-10T08:37:05.260178+0000","last_active":"2024-09-10T08:38:43.633140+0000","last_peered":"2024-09-10T08:38:43.633140+0000","last_clean":"2024-09-10T08:38:43.633140+0000","last_became_active":"2024-09-10T08:37:05.259753+0000","last_became_peered":"2024-09-10T08:37:05.259753+0000","last_unstale":"2024-09-10T08:38:43.633140+0000","last_undegraded":"2024-09-10T08:38:43.633140+0000","last_fullsized":"2024-09-10T08:38:43.633140+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:19.364782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588360+0000","last_change":"2024-09-10T08:36:56.147104+0000","last_active":"2024-09-10T08:39:11.588360+0000","last_peered":"2024-09-10T08:39:11.588360+0000","last_clean":"2024-09-10T08:39:11.588360+0000","last_became_active":"2024-09-10T08:36:56.146778+0000","last_became_peered":"2024-09-10T08:36:56.146778+0000","last_unstale":"2024-09-10T08:39:11.588360+0000","last_undegraded":"2024-09-10T08:39:11.588360+0000","last_fullsized":"2024-09-10T08:39:11.588360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:56.092157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628399+0000","last_change":"2024-09-10T08:36:53.719145+0000","last_active":"2024-09-10T08:38:43.628399+0000","last_peered":"2024-09-10T08:38:43.628399+0000","last_clean":"2024-09-10T08:38:43.628399+0000","last_became_active":"2024-09-10T08:36:53.718771+0000","last_became_peered":"2024-09-10T08:36:53.718771+0000","last_unstale":"2024-09-10T08:38:43.628399+0000","last_undegraded":"2024-09-10T08:38:43.628399+0000","last_fullsized":"2024-09-10T08:38:43.628399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:21.109940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654733+0000","last_change":"2024-09-10T08:37:05.242854+0000","last_active":"2024-09-10T08:38:43.654733+0000","last_peered":"2024-09-10T08:38:43.654733+0000","last_clean":"2024-09-10T08:38:43.654733+0000","last_became_active":"2024-09-10T08:37:05.242442+0000","last_became_peered":"2024-09-10T08:37:05.242442+0000","last_unstale":"2024-09-10T08:38:43.654733+0000","last_undegraded":"2024-09-10T08:38:43.654733+0000","last_fullsized":"2024-09-10T08:38:43.654733+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:36.072131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.570294+0000","last_change":"2024-09-10T08:37:42.845642+0000","last_active":"2024-09-10T08:39:11.570294+0000","last_peered":"2024-09-10T08:39:11.570294+0000","last_clean":"2024-09-10T08:39:11.570294+0000","last_became_active":"2024-09-10T08:37:42.845217+0000","last_became_peered":"2024-09-10T08:37:42.845217+0000","last_unstale":"2024-09-10T08:39:11.570294+0000","last_undegraded":"2024-09-10T08:39:11.570294+0000","last_fullsized":"2024-09-10T08:39:11.570294+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:28.020337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626734+0000","last_change":"2024-09-10T08:36:53.720463+0000","last_active":"2024-09-10T08:38:43.626734+0000","last_peered":"2024-09-10T08:38:43.626734+0000","last_clean":"2024-09-10T08:38:43.626734+0000","last_became_active":"2024-09-10T08:36:53.720080+0000","last_became_peered":"2024-09-10T08:36:53.720080+0000","last_unstale":"2024-09-10T08:38:43.626734+0000","last_undegraded":"2024-09-10T08:38:43.626734+0000","last_fullsized":"2024-09-10T08:38:43.626734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:51.450294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654955+0000","last_change":"2024-09-10T08:37:05.251261+0000","last_active":"2024-09-10T08:38:43.654955+0000","last_peered":"2024-09-10T08:38:43.654955+0000","last_clean":"2024-09-10T08:38:43.654955+0000","last_became_active":"2024-09-10T08:37:05.250938+0000","last_became_peered":"2024-09-10T08:37:05.250938+0000","last_unstale":"2024-09-10T08:38:43.654955+0000","last_undegraded":"2024-09-10T08:38:43.654955+0000","last_fullsized":"2024-09-10T08:38:43.654955+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:07:15.129899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.581822+0000","last_change":"2024-09-10T08:36:56.148191+0000","last_active":"2024-09-10T08:39:11.581822+0000","last_peered":"2024-09-10T08:39:11.581822+0000","last_clean":"2024-09-10T08:39:11.581822+0000","last_became_active":"2024-09-10T08:36:56.147362+0000","last_became_peered":"2024-09-10T08:36:56.147362+0000","last_unstale":"2024-09-10T08:39:11.581822+0000","last_undegraded":"2024-09-10T08:39:11.581822+0000","last_fullsized":"2024-09-10T08:39:11.581822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:25:30.051754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626042+0000","last_change":"2024-09-10T08:36:53.734484+0000","last_active":"2024-09-10T08:38:43.626042+0000","last_peered":"2024-09-10T08:38:43.626042+0000","last_clean":"2024-09-10T08:38:43.626042+0000","last_became_active":"2024-09-10T08:36:53.734172+0000","last_became_peered":"2024-09-10T08:36:53.734172+0000","last_unstale":"2024-09-10T08:38:43.626042+0000","last_undegraded":"2024-09-10T08:38:43.626042+0000","last_fullsized":"2024-09-10T08:38:43.626042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:10.269400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628299+0000","last_change":"2024-09-10T08:37:05.259635+0000","last_active":"2024-09-10T08:38:43.628299+0000","last_peered":"2024-09-10T08:38:43.628299+0000","last_clean":"2024-09-10T08:38:43.628299+0000","last_became_active":"2024-09-10T08:37:05.259107+0000","last_became_peered":"2024-09-10T08:37:05.259107+0000","last_unstale":"2024-09-10T08:38:43.628299+0000","last_undegraded":"2024-09-10T08:38:43.628299+0000","last_fullsized":"2024-09-10T08:38:43.628299+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:36.841138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588992+0000","last_change":"2024-09-10T08:36:56.141122+0000","last_active":"2024-09-10T08:39:11.588992+0000","last_peered":"2024-09-10T08:39:11.588992+0000","last_clean":"2024-09-10T08:39:11.588992+0000","last_became_active":"2024-09-10T08:36:56.140826+0000","last_became_peered":"2024-09-10T08:36:56.140826+0000","last_unstale":"2024-09-10T08:39:11.588992+0000","last_undegraded":"2024-09-10T08:39:11.588992+0000","last_fullsized":"2024-09-10T08:39:11.588992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:38.044090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"96'6","reported_seq":163,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:39.887564+0000","last_change":"2024-09-10T08:36:53.720623+0000","last_active":"2024-09-10T08:39:39.887564+0000","last_peered":"2024-09-10T08:39:39.887564+0000","last_clean":"2024-09-10T08:39:39.887564+0000","last_became_active":"2024-09-10T08:36:53.720270+0000","last_became_peered":"2024-09-10T08:36:53.720270+0000","last_unstale":"2024-09-10T08:39:39.887564+0000","last_undegraded":"2024-09-10T08:39:39.887564+0000","last_fullsized":"2024-09-10T08:39:39.887564+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:19.514068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629937+0000","last_change":"2024-09-10T08:37:05.254324+0000","last_active":"2024-09-10T08:38:43.629937+0000","last_peered":"2024-09-10T08:38:43.629937+0000","last_clean":"2024-09-10T08:38:43.629937+0000","last_became_active":"2024-09-10T08:37:05.253611+0000","last_became_peered":"2024-09-10T08:37:05.253611+0000","last_unstale":"2024-09-10T08:38:43.629937+0000","last_undegraded":"2024-09-10T08:38:43.629937+0000","last_fullsized":"2024-09-10T08:38:43.629937+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:07.182649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.591257+0000","last_change":"2024-09-10T08:36:56.139102+0000","last_active":"2024-09-10T08:39:11.591257+0000","last_peered":"2024-09-10T08:39:11.591257+0000","last_clean":"2024-09-10T08:39:11.591257+0000","last_became_active":"2024-09-10T08:36:56.137986+0000","last_became_peered":"2024-09-10T08:36:56.137986+0000","last_unstale":"2024-09-10T08:39:11.591257+0000","last_undegraded":"2024-09-10T08:39:11.591257+0000","last_fullsized":"2024-09-10T08:39:11.591257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:05:03.005727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655021+0000","last_change":"2024-09-10T08:36:53.722961+0000","last_active":"2024-09-10T08:38:43.655021+0000","last_peered":"2024-09-10T08:38:43.655021+0000","last_clean":"2024-09-10T08:38:43.655021+0000","last_became_active":"2024-09-10T08:36:53.722628+0000","last_became_peered":"2024-09-10T08:36:53.722628+0000","last_unstale":"2024-09-10T08:38:43.655021+0000","last_undegraded":"2024-09-10T08:38:43.655021+0000","last_fullsized":"2024-09-10T08:38:43.655021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:59.048632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628104+0000","last_change":"2024-09-10T08:37:05.256546+0000","last_active":"2024-09-10T08:38:43.628104+0000","last_peered":"2024-09-10T08:38:43.628104+0000","last_clean":"2024-09-10T08:38:43.628104+0000","last_became_active":"2024-09-10T08:37:05.256239+0000","last_became_peered":"2024-09-10T08:37:05.256239+0000","last_unstale":"2024-09-10T08:38:43.628104+0000","last_undegraded":"2024-09-10T08:38:43.628104+0000","last_fullsized":"2024-09-10T08:38:43.628104+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:57.403934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.582848+0000","last_change":"2024-09-10T08:36:56.149285+0000","last_active":"2024-09-10T08:39:11.582848+0000","last_peered":"2024-09-10T08:39:11.582848+0000","last_clean":"2024-09-10T08:39:11.582848+0000","last_became_active":"2024-09-10T08:36:56.146873+0000","last_became_peered":"2024-09-10T08:36:56.146873+0000","last_unstale":"2024-09-10T08:39:11.582848+0000","last_undegraded":"2024-09-10T08:39:11.582848+0000","last_fullsized":"2024-09-10T08:39:11.582848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:28.976729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630368+0000","last_change":"2024-09-10T08:36:53.721174+0000","last_active":"2024-09-10T08:38:43.630368+0000","last_peered":"2024-09-10T08:38:43.630368+0000","last_clean":"2024-09-10T08:38:43.630368+0000","last_became_active":"2024-09-10T08:36:53.720849+0000","last_became_peered":"2024-09-10T08:36:53.720849+0000","last_unstale":"2024-09-10T08:38:43.630368+0000","last_undegraded":"2024-09-10T08:38:43.630368+0000","last_fullsized":"2024-09-10T08:38:43.630368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:39.786848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655286+0000","last_change":"2024-09-10T08:37:05.252256+0000","last_active":"2024-09-10T08:38:43.655286+0000","last_peered":"2024-09-10T08:38:43.655286+0000","last_clean":"2024-09-10T08:38:43.655286+0000","last_became_active":"2024-09-10T08:37:05.251865+0000","last_became_peered":"2024-09-10T08:37:05.251865+0000","last_unstale":"2024-09-10T08:38:43.655286+0000","last_undegraded":"2024-09-10T08:38:43.655286+0000","last_fullsized":"2024-09-10T08:38:43.655286+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:08.581880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567773+0000","last_change":"2024-09-10T08:36:56.142060+0000","last_active":"2024-09-10T08:39:11.567773+0000","last_peered":"2024-09-10T08:39:11.567773+0000","last_clean":"2024-09-10T08:39:11.567773+0000","last_became_active":"2024-09-10T08:36:56.141451+0000","last_became_peered":"2024-09-10T08:36:56.141451+0000","last_unstale":"2024-09-10T08:39:11.567773+0000","last_undegraded":"2024-09-10T08:39:11.567773+0000","last_fullsized":"2024-09-10T08:39:11.567773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:43.566588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'37","reported_seq":310,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:38.877082+0000","last_change":"2024-09-10T08:36:53.719464+0000","last_active":"2024-09-10T08:39:38.877082+0000","last_peered":"2024-09-10T08:39:38.877082+0000","last_clean":"2024-09-10T08:39:38.877082+0000","last_became_active":"2024-09-10T08:36:53.719129+0000","last_became_peered":"2024-09-10T08:36:53.719129+0000","last_unstale":"2024-09-10T08:39:38.877082+0000","last_undegraded":"2024-09-10T08:39:38.877082+0000","last_fullsized":"2024-09-10T08:39:38.877082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:05.652018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64071,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":232,"num_read_kb":179,"num_write":38,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629405+0000","last_change":"2024-09-10T08:37:05.255576+0000","last_active":"2024-09-10T08:38:43.629405+0000","last_peered":"2024-09-10T08:38:43.629405+0000","last_clean":"2024-09-10T08:38:43.629405+0000","last_became_active":"2024-09-10T08:37:05.249842+0000","last_became_peered":"2024-09-10T08:37:05.249842+0000","last_unstale":"2024-09-10T08:38:43.629405+0000","last_undegraded":"2024-09-10T08:38:43.629405+0000","last_fullsized":"2024-09-10T08:38:43.629405+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:47:49.543481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593397+0000","last_change":"2024-09-10T08:36:56.151829+0000","last_active":"2024-09-10T08:39:11.593397+0000","last_peered":"2024-09-10T08:39:11.593397+0000","last_clean":"2024-09-10T08:39:11.593397+0000","last_became_active":"2024-09-10T08:36:56.151314+0000","last_became_peered":"2024-09-10T08:36:56.151314+0000","last_unstale":"2024-09-10T08:39:11.593397+0000","last_undegraded":"2024-09-10T08:39:11.593397+0000","last_fullsized":"2024-09-10T08:39:11.593397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:44.219671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655398+0000","last_change":"2024-09-10T08:36:53.738030+0000","last_active":"2024-09-10T08:38:43.655398+0000","last_peered":"2024-09-10T08:38:43.655398+0000","last_clean":"2024-09-10T08:38:43.655398+0000","last_became_active":"2024-09-10T08:36:53.737658+0000","last_became_peered":"2024-09-10T08:36:53.737658+0000","last_unstale":"2024-09-10T08:38:43.655398+0000","last_undegraded":"2024-09-10T08:38:43.655398+0000","last_fullsized":"2024-09-10T08:38:43.655398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:29.627554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654996+0000","last_change":"2024-09-10T08:37:05.244473+0000","last_active":"2024-09-10T08:38:43.654996+0000","last_peered":"2024-09-10T08:38:43.654996+0000","last_clean":"2024-09-10T08:38:43.654996+0000","last_became_active":"2024-09-10T08:37:05.244035+0000","last_became_peered":"2024-09-10T08:37:05.244035+0000","last_unstale":"2024-09-10T08:38:43.654996+0000","last_undegraded":"2024-09-10T08:38:43.654996+0000","last_fullsized":"2024-09-10T08:38:43.654996+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:04.652660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584024+0000","last_change":"2024-09-10T08:36:56.155217+0000","last_active":"2024-09-10T08:39:11.584024+0000","last_peered":"2024-09-10T08:39:11.584024+0000","last_clean":"2024-09-10T08:39:11.584024+0000","last_became_active":"2024-09-10T08:36:56.154903+0000","last_became_peered":"2024-09-10T08:36:56.154903+0000","last_unstale":"2024-09-10T08:39:11.584024+0000","last_undegraded":"2024-09-10T08:39:11.584024+0000","last_fullsized":"2024-09-10T08:39:11.584024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:47.482974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.632239+0000","last_change":"2024-09-10T08:36:53.721087+0000","last_active":"2024-09-10T08:38:43.632239+0000","last_peered":"2024-09-10T08:38:43.632239+0000","last_clean":"2024-09-10T08:38:43.632239+0000","last_became_active":"2024-09-10T08:36:53.720772+0000","last_became_peered":"2024-09-10T08:36:53.720772+0000","last_unstale":"2024-09-10T08:38:43.632239+0000","last_undegraded":"2024-09-10T08:38:43.632239+0000","last_fullsized":"2024-09-10T08:38:43.632239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:39.560848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629150+0000","last_change":"2024-09-10T08:37:05.254641+0000","last_active":"2024-09-10T08:38:43.629150+0000","last_peered":"2024-09-10T08:38:43.629150+0000","last_clean":"2024-09-10T08:38:43.629150+0000","last_became_active":"2024-09-10T08:37:05.249982+0000","last_became_peered":"2024-09-10T08:37:05.249982+0000","last_unstale":"2024-09-10T08:38:43.629150+0000","last_undegraded":"2024-09-10T08:38:43.629150+0000","last_fullsized":"2024-09-10T08:38:43.629150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:21:13.851061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592510+0000","last_change":"2024-09-10T08:36:56.142657+0000","last_active":"2024-09-10T08:39:11.592510+0000","last_peered":"2024-09-10T08:39:11.592510+0000","last_clean":"2024-09-10T08:39:11.592510+0000","last_became_active":"2024-09-10T08:36:56.142294+0000","last_became_peered":"2024-09-10T08:36:56.142294+0000","last_unstale":"2024-09-10T08:39:11.592510+0000","last_undegraded":"2024-09-10T08:39:11.592510+0000","last_fullsized":"2024-09-10T08:39:11.592510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:52.861389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'4","reported_seq":156,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:37.909693+0000","last_change":"2024-09-10T08:36:53.722466+0000","last_active":"2024-09-10T08:39:37.909693+0000","last_peered":"2024-09-10T08:39:37.909693+0000","last_clean":"2024-09-10T08:39:37.909693+0000","last_became_active":"2024-09-10T08:36:53.722145+0000","last_became_peered":"2024-09-10T08:36:53.722145+0000","last_unstale":"2024-09-10T08:39:37.909693+0000","last_undegraded":"2024-09-10T08:39:37.909693+0000","last_fullsized":"2024-09-10T08:39:37.909693+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:27.041765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631352+0000","last_change":"2024-09-10T08:37:05.251056+0000","last_active":"2024-09-10T08:38:43.631352+0000","last_peered":"2024-09-10T08:38:43.631352+0000","last_clean":"2024-09-10T08:38:43.631352+0000","last_became_active":"2024-09-10T08:37:05.250653+0000","last_became_peered":"2024-09-10T08:37:05.250653+0000","last_unstale":"2024-09-10T08:38:43.631352+0000","last_undegraded":"2024-09-10T08:38:43.631352+0000","last_fullsized":"2024-09-10T08:38:43.631352+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:21.174747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633128+0000","last_change":"2024-09-10T08:37:05.260436+0000","last_active":"2024-09-10T08:38:43.633128+0000","last_peered":"2024-09-10T08:38:43.633128+0000","last_clean":"2024-09-10T08:38:43.633128+0000","last_became_active":"2024-09-10T08:37:05.260049+0000","last_became_peered":"2024-09-10T08:37:05.260049+0000","last_unstale":"2024-09-10T08:38:43.633128+0000","last_undegraded":"2024-09-10T08:38:43.633128+0000","last_fullsized":"2024-09-10T08:38:43.633128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:43.394028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585930+0000","last_change":"2024-09-10T08:36:56.146732+0000","last_active":"2024-09-10T08:39:11.585930+0000","last_peered":"2024-09-10T08:39:11.585930+0000","last_clean":"2024-09-10T08:39:11.585930+0000","last_became_active":"2024-09-10T08:36:56.145599+0000","last_became_peered":"2024-09-10T08:36:56.145599+0000","last_unstale":"2024-09-10T08:39:11.585930+0000","last_undegraded":"2024-09-10T08:39:11.585930+0000","last_fullsized":"2024-09-10T08:39:11.585930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:50.620956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655135+0000","last_change":"2024-09-10T08:36:53.721683+0000","last_active":"2024-09-10T08:38:43.655135+0000","last_peered":"2024-09-10T08:38:43.655135+0000","last_clean":"2024-09-10T08:38:43.655135+0000","last_became_active":"2024-09-10T08:36:53.721241+0000","last_became_peered":"2024-09-10T08:36:53.721241+0000","last_unstale":"2024-09-10T08:38:43.655135+0000","last_undegraded":"2024-09-10T08:38:43.655135+0000","last_fullsized":"2024-09-10T08:38:43.655135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:03.669337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628275+0000","last_change":"2024-09-10T08:37:42.669784+0000","last_active":"2024-09-10T08:38:43.628275+0000","last_peered":"2024-09-10T08:38:43.628275+0000","last_clean":"2024-09-10T08:38:43.628275+0000","last_became_active":"2024-09-10T08:37:42.669322+0000","last_became_peered":"2024-09-10T08:37:42.669322+0000","last_unstale":"2024-09-10T08:38:43.628275+0000","last_undegraded":"2024-09-10T08:38:43.628275+0000","last_fullsized":"2024-09-10T08:38:43.628275+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:06:57.027928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.574617+0000","last_change":"2024-09-10T08:36:56.143824+0000","last_active":"2024-09-10T08:39:11.574617+0000","last_peered":"2024-09-10T08:39:11.574617+0000","last_clean":"2024-09-10T08:39:11.574617+0000","last_became_active":"2024-09-10T08:36:56.143486+0000","last_became_peered":"2024-09-10T08:36:56.143486+0000","last_unstale":"2024-09-10T08:39:11.574617+0000","last_undegraded":"2024-09-10T08:39:11.574617+0000","last_fullsized":"2024-09-10T08:39:11.574617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:42.384863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627976+0000","last_change":"2024-09-10T08:36:53.717680+0000","last_active":"2024-09-10T08:38:43.627976+0000","last_peered":"2024-09-10T08:38:43.627976+0000","last_clean":"2024-09-10T08:38:43.627976+0000","last_became_active":"2024-09-10T08:36:53.717249+0000","last_became_peered":"2024-09-10T08:36:53.717249+0000","last_unstale":"2024-09-10T08:38:43.627976+0000","last_undegraded":"2024-09-10T08:38:43.627976+0000","last_fullsized":"2024-09-10T08:38:43.627976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:12.191541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627200+0000","last_change":"2024-09-10T08:36:53.723947+0000","last_active":"2024-09-10T08:38:43.627200+0000","last_peered":"2024-09-10T08:38:43.627200+0000","last_clean":"2024-09-10T08:38:43.627200+0000","last_became_active":"2024-09-10T08:36:53.723620+0000","last_became_peered":"2024-09-10T08:36:53.723620+0000","last_unstale":"2024-09-10T08:38:43.627200+0000","last_undegraded":"2024-09-10T08:38:43.627200+0000","last_fullsized":"2024-09-10T08:38:43.627200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:00.135349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583145+0000","last_change":"2024-09-10T08:36:56.150448+0000","last_active":"2024-09-10T08:39:11.583145+0000","last_peered":"2024-09-10T08:39:11.583145+0000","last_clean":"2024-09-10T08:39:11.583145+0000","last_became_active":"2024-09-10T08:36:56.150148+0000","last_became_peered":"2024-09-10T08:36:56.150148+0000","last_unstale":"2024-09-10T08:39:11.583145+0000","last_undegraded":"2024-09-10T08:39:11.583145+0000","last_fullsized":"2024-09-10T08:39:11.583145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:02.927753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630191+0000","last_change":"2024-09-10T08:37:05.242878+0000","last_active":"2024-09-10T08:38:43.630191+0000","last_peered":"2024-09-10T08:38:43.630191+0000","last_clean":"2024-09-10T08:38:43.630191+0000","last_became_active":"2024-09-10T08:37:05.242440+0000","last_became_peered":"2024-09-10T08:37:05.242440+0000","last_unstale":"2024-09-10T08:38:43.630191+0000","last_undegraded":"2024-09-10T08:38:43.630191+0000","last_fullsized":"2024-09-10T08:38:43.630191+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:30.957433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'7","reported_seq":163,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:39.878942+0000","last_change":"2024-09-10T08:36:53.719788+0000","last_active":"2024-09-10T08:39:39.878942+0000","last_peered":"2024-09-10T08:39:39.878942+0000","last_clean":"2024-09-10T08:39:39.878942+0000","last_became_active":"2024-09-10T08:36:53.719462+0000","last_became_peered":"2024-09-10T08:36:53.719462+0000","last_unstale":"2024-09-10T08:39:39.878942+0000","last_undegraded":"2024-09-10T08:39:39.878942+0000","last_fullsized":"2024-09-10T08:39:39.878942+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:12.250447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1326,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583170+0000","last_change":"2024-09-10T08:37:42.673008+0000","last_active":"2024-09-10T08:39:11.583170+0000","last_peered":"2024-09-10T08:39:11.583170+0000","last_clean":"2024-09-10T08:39:11.583170+0000","last_became_active":"2024-09-10T08:37:42.672618+0000","last_became_peered":"2024-09-10T08:37:42.672618+0000","last_unstale":"2024-09-10T08:39:11.583170+0000","last_undegraded":"2024-09-10T08:39:11.583170+0000","last_fullsized":"2024-09-10T08:39:11.583170+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:00.156565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630783+0000","last_change":"2024-09-10T08:37:05.244277+0000","last_active":"2024-09-10T08:38:43.630783+0000","last_peered":"2024-09-10T08:38:43.630783+0000","last_clean":"2024-09-10T08:38:43.630783+0000","last_became_active":"2024-09-10T08:37:05.243880+0000","last_became_peered":"2024-09-10T08:37:05.243880+0000","last_unstale":"2024-09-10T08:38:43.630783+0000","last_undegraded":"2024-09-10T08:38:43.630783+0000","last_fullsized":"2024-09-10T08:38:43.630783+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:55.275857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.573131+0000","last_change":"2024-09-10T08:36:56.141404+0000","last_active":"2024-09-10T08:39:11.573131+0000","last_peered":"2024-09-10T08:39:11.573131+0000","last_clean":"2024-09-10T08:39:11.573131+0000","last_became_active":"2024-09-10T08:36:56.141076+0000","last_became_peered":"2024-09-10T08:36:56.141076+0000","last_unstale":"2024-09-10T08:39:11.573131+0000","last_undegraded":"2024-09-10T08:39:11.573131+0000","last_fullsized":"2024-09-10T08:39:11.573131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:03:52.915654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.569383+0000","last_change":"2024-09-10T08:36:56.148285+0000","last_active":"2024-09-10T08:39:11.569383+0000","last_peered":"2024-09-10T08:39:11.569383+0000","last_clean":"2024-09-10T08:39:11.569383+0000","last_became_active":"2024-09-10T08:36:56.147970+0000","last_became_peered":"2024-09-10T08:36:56.147970+0000","last_unstale":"2024-09-10T08:39:11.569383+0000","last_undegraded":"2024-09-10T08:39:11.569383+0000","last_fullsized":"2024-09-10T08:39:11.569383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:09.506718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.571047+0000","last_change":"2024-09-10T08:36:56.157977+0000","last_active":"2024-09-10T08:39:11.571047+0000","last_peered":"2024-09-10T08:39:11.571047+0000","last_clean":"2024-09-10T08:39:11.571047+0000","last_became_active":"2024-09-10T08:36:56.157623+0000","last_became_peered":"2024-09-10T08:36:56.157623+0000","last_unstale":"2024-09-10T08:39:11.571047+0000","last_undegraded":"2024-09-10T08:39:11.571047+0000","last_fullsized":"2024-09-10T08:39:11.571047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:44.617979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584899+0000","last_change":"2024-09-10T08:37:42.670459+0000","last_active":"2024-09-10T08:39:11.584899+0000","last_peered":"2024-09-10T08:39:11.584899+0000","last_clean":"2024-09-10T08:39:11.584899+0000","last_became_active":"2024-09-10T08:37:42.670097+0000","last_became_peered":"2024-09-10T08:37:42.670097+0000","last_unstale":"2024-09-10T08:39:11.584899+0000","last_undegraded":"2024-09-10T08:39:11.584899+0000","last_fullsized":"2024-09-10T08:39:11.584899+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:14.956492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584527+0000","last_change":"2024-09-10T08:36:56.154850+0000","last_active":"2024-09-10T08:39:11.584527+0000","last_peered":"2024-09-10T08:39:11.584527+0000","last_clean":"2024-09-10T08:39:11.584527+0000","last_became_active":"2024-09-10T08:36:56.154545+0000","last_became_peered":"2024-09-10T08:36:56.154545+0000","last_unstale":"2024-09-10T08:39:11.584527+0000","last_undegraded":"2024-09-10T08:39:11.584527+0000","last_fullsized":"2024-09-10T08:39:11.584527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:24.639667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.569973+0000","last_change":"2024-09-10T08:37:42.669962+0000","last_active":"2024-09-10T08:39:11.569973+0000","last_peered":"2024-09-10T08:39:11.569973+0000","last_clean":"2024-09-10T08:39:11.569973+0000","last_became_active":"2024-09-10T08:37:42.669556+0000","last_became_peered":"2024-09-10T08:37:42.669556+0000","last_unstale":"2024-09-10T08:39:11.569973+0000","last_undegraded":"2024-09-10T08:39:11.569973+0000","last_fullsized":"2024-09-10T08:39:11.569973+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:23.333946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.578061+0000","last_change":"2024-09-10T08:36:56.145910+0000","last_active":"2024-09-10T08:39:11.578061+0000","last_peered":"2024-09-10T08:39:11.578061+0000","last_clean":"2024-09-10T08:39:11.578061+0000","last_became_active":"2024-09-10T08:36:56.145537+0000","last_became_peered":"2024-09-10T08:36:56.145537+0000","last_unstale":"2024-09-10T08:39:11.578061+0000","last_undegraded":"2024-09-10T08:39:11.578061+0000","last_fullsized":"2024-09-10T08:39:11.578061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:46.788241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567880+0000","last_change":"2024-09-10T08:36:56.147340+0000","last_active":"2024-09-10T08:39:11.567880+0000","last_peered":"2024-09-10T08:39:11.567880+0000","last_clean":"2024-09-10T08:39:11.567880+0000","last_became_active":"2024-09-10T08:36:56.145762+0000","last_became_peered":"2024-09-10T08:36:56.145762+0000","last_unstale":"2024-09-10T08:39:11.567880+0000","last_undegraded":"2024-09-10T08:39:11.567880+0000","last_fullsized":"2024-09-10T08:39:11.567880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:12.339285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.581431+0000","last_change":"2024-09-10T08:36:56.145213+0000","last_active":"2024-09-10T08:39:11.581431+0000","last_peered":"2024-09-10T08:39:11.581431+0000","last_clean":"2024-09-10T08:39:11.581431+0000","last_became_active":"2024-09-10T08:36:56.144915+0000","last_became_peered":"2024-09-10T08:36:56.144915+0000","last_unstale":"2024-09-10T08:39:11.581431+0000","last_undegraded":"2024-09-10T08:39:11.581431+0000","last_fullsized":"2024-09-10T08:39:11.581431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:06.180321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.574743+0000","last_change":"2024-09-10T08:36:56.149738+0000","last_active":"2024-09-10T08:39:11.574743+0000","last_peered":"2024-09-10T08:39:11.574743+0000","last_clean":"2024-09-10T08:39:11.574743+0000","last_became_active":"2024-09-10T08:36:56.147750+0000","last_became_peered":"2024-09-10T08:36:56.147750+0000","last_unstale":"2024-09-10T08:39:11.574743+0000","last_undegraded":"2024-09-10T08:39:11.574743+0000","last_fullsized":"2024-09-10T08:39:11.574743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:47.950054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.591937+0000","last_change":"2024-09-10T08:36:56.147682+0000","last_active":"2024-09-10T08:39:11.591937+0000","last_peered":"2024-09-10T08:39:11.591937+0000","last_clean":"2024-09-10T08:39:11.591937+0000","last_became_active":"2024-09-10T08:36:56.147345+0000","last_became_peered":"2024-09-10T08:36:56.147345+0000","last_unstale":"2024-09-10T08:39:11.591937+0000","last_undegraded":"2024-09-10T08:39:11.591937+0000","last_fullsized":"2024-09-10T08:39:11.591937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:54.939482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583378+0000","last_change":"2024-09-10T08:36:56.158292+0000","last_active":"2024-09-10T08:39:11.583378+0000","last_peered":"2024-09-10T08:39:11.583378+0000","last_clean":"2024-09-10T08:39:11.583378+0000","last_became_active":"2024-09-10T08:36:56.157970+0000","last_became_peered":"2024-09-10T08:36:56.157970+0000","last_unstale":"2024-09-10T08:39:11.583378+0000","last_undegraded":"2024-09-10T08:39:11.583378+0000","last_fullsized":"2024-09-10T08:39:11.583378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:33.582126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567916+0000","last_change":"2024-09-10T08:36:56.142520+0000","last_active":"2024-09-10T08:39:11.567916+0000","last_peered":"2024-09-10T08:39:11.567916+0000","last_clean":"2024-09-10T08:39:11.567916+0000","last_became_active":"2024-09-10T08:36:56.142214+0000","last_became_peered":"2024-09-10T08:36:56.142214+0000","last_unstale":"2024-09-10T08:39:11.567916+0000","last_undegraded":"2024-09-10T08:39:11.567916+0000","last_fullsized":"2024-09-10T08:39:11.567916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:45:29.201839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584167+0000","last_change":"2024-09-10T08:36:56.149547+0000","last_active":"2024-09-10T08:39:11.584167+0000","last_peered":"2024-09-10T08:39:11.584167+0000","last_clean":"2024-09-10T08:39:11.584167+0000","last_became_active":"2024-09-10T08:36:56.149141+0000","last_became_peered":"2024-09-10T08:36:56.149141+0000","last_unstale":"2024-09-10T08:39:11.584167+0000","last_undegraded":"2024-09-10T08:39:11.584167+0000","last_fullsized":"2024-09-10T08:39:11.584167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:49.368677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585864+0000","last_change":"2024-09-10T08:36:56.143087+0000","last_active":"2024-09-10T08:39:11.585864+0000","last_peered":"2024-09-10T08:39:11.585864+0000","last_clean":"2024-09-10T08:39:11.585864+0000","last_became_active":"2024-09-10T08:36:56.142782+0000","last_became_peered":"2024-09-10T08:36:56.142782+0000","last_unstale":"2024-09-10T08:39:11.585864+0000","last_undegraded":"2024-09-10T08:39:11.585864+0000","last_fullsized":"2024-09-10T08:39:11.585864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:17.513103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592876+0000","last_change":"2024-09-10T08:36:56.157549+0000","last_active":"2024-09-10T08:39:11.592876+0000","last_peered":"2024-09-10T08:39:11.592876+0000","last_clean":"2024-09-10T08:39:11.592876+0000","last_became_active":"2024-09-10T08:36:56.157221+0000","last_became_peered":"2024-09-10T08:36:56.157221+0000","last_unstale":"2024-09-10T08:39:11.592876+0000","last_undegraded":"2024-09-10T08:39:11.592876+0000","last_fullsized":"2024-09-10T08:39:11.592876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:11.372085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593199+0000","last_change":"2024-09-10T08:36:56.145654+0000","last_active":"2024-09-10T08:39:11.593199+0000","last_peered":"2024-09-10T08:39:11.593199+0000","last_clean":"2024-09-10T08:39:11.593199+0000","last_became_active":"2024-09-10T08:36:56.144126+0000","last_became_peered":"2024-09-10T08:36:56.144126+0000","last_unstale":"2024-09-10T08:39:11.593199+0000","last_undegraded":"2024-09-10T08:39:11.593199+0000","last_fullsized":"2024-09-10T08:39:11.593199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:55.498873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.582479+0000","last_change":"2024-09-10T08:36:56.146423+0000","last_active":"2024-09-10T08:39:11.582479+0000","last_peered":"2024-09-10T08:39:11.582479+0000","last_clean":"2024-09-10T08:39:11.582479+0000","last_became_active":"2024-09-10T08:36:56.146126+0000","last_became_peered":"2024-09-10T08:36:56.146126+0000","last_unstale":"2024-09-10T08:39:11.582479+0000","last_undegraded":"2024-09-10T08:39:11.582479+0000","last_fullsized":"2024-09-10T08:39:11.582479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:44.665196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583332+0000","last_change":"2024-09-10T08:36:56.151973+0000","last_active":"2024-09-10T08:39:11.583332+0000","last_peered":"2024-09-10T08:39:11.583332+0000","last_clean":"2024-09-10T08:39:11.583332+0000","last_became_active":"2024-09-10T08:36:56.151609+0000","last_became_peered":"2024-09-10T08:36:56.151609+0000","last_unstale":"2024-09-10T08:39:11.583332+0000","last_undegraded":"2024-09-10T08:39:11.583332+0000","last_fullsized":"2024-09-10T08:39:11.583332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:02.584047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584509+0000","last_change":"2024-09-10T08:36:56.141780+0000","last_active":"2024-09-10T08:39:11.584509+0000","last_peered":"2024-09-10T08:39:11.584509+0000","last_clean":"2024-09-10T08:39:11.584509+0000","last_became_active":"2024-09-10T08:36:56.141228+0000","last_became_peered":"2024-09-10T08:36:56.141228+0000","last_unstale":"2024-09-10T08:39:11.584509+0000","last_undegraded":"2024-09-10T08:39:11.584509+0000","last_fullsized":"2024-09-10T08:39:11.584509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:52:04.374228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585477+0000","last_change":"2024-09-10T08:37:42.671346+0000","last_active":"2024-09-10T08:39:11.585477+0000","last_peered":"2024-09-10T08:39:11.585477+0000","last_clean":"2024-09-10T08:39:11.585477+0000","last_became_active":"2024-09-10T08:37:42.670895+0000","last_became_peered":"2024-09-10T08:37:42.670895+0000","last_unstale":"2024-09-10T08:39:11.585477+0000","last_undegraded":"2024-09-10T08:39:11.585477+0000","last_fullsized":"2024-09-10T08:39:11.585477+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:16:31.930628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593796+0000","last_change":"2024-09-10T08:36:56.149014+0000","last_active":"2024-09-10T08:39:11.593796+0000","last_peered":"2024-09-10T08:39:11.593796+0000","last_clean":"2024-09-10T08:39:11.593796+0000","last_became_active":"2024-09-10T08:36:56.148551+0000","last_became_peered":"2024-09-10T08:36:56.148551+0000","last_unstale":"2024-09-10T08:39:11.593796+0000","last_undegraded":"2024-09-10T08:39:11.593796+0000","last_fullsized":"2024-09-10T08:39:11.593796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:35:52.250943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.586709+0000","last_change":"2024-09-10T08:36:56.141247+0000","last_active":"2024-09-10T08:39:11.586709+0000","last_peered":"2024-09-10T08:39:11.586709+0000","last_clean":"2024-09-10T08:39:11.586709+0000","last_became_active":"2024-09-10T08:36:56.140956+0000","last_became_peered":"2024-09-10T08:36:56.140956+0000","last_unstale":"2024-09-10T08:39:11.586709+0000","last_undegraded":"2024-09-10T08:39:11.586709+0000","last_fullsized":"2024-09-10T08:39:11.586709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:45.008424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592783+0000","last_change":"2024-09-10T08:36:56.149116+0000","last_active":"2024-09-10T08:39:11.592783+0000","last_peered":"2024-09-10T08:39:11.592783+0000","last_clean":"2024-09-10T08:39:11.592783+0000","last_became_active":"2024-09-10T08:36:56.148783+0000","last_became_peered":"2024-09-10T08:36:56.148783+0000","last_unstale":"2024-09-10T08:39:11.592783+0000","last_undegraded":"2024-09-10T08:39:11.592783+0000","last_fullsized":"2024-09-10T08:39:11.592783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:37.119214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585754+0000","last_change":"2024-09-10T08:37:42.846322+0000","last_active":"2024-09-10T08:39:11.585754+0000","last_peered":"2024-09-10T08:39:11.585754+0000","last_clean":"2024-09-10T08:39:11.585754+0000","last_became_active":"2024-09-10T08:37:42.846022+0000","last_became_peered":"2024-09-10T08:37:42.846022+0000","last_unstale":"2024-09-10T08:39:11.585754+0000","last_undegraded":"2024-09-10T08:39:11.585754+0000","last_fullsized":"2024-09-10T08:39:11.585754+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:38.142362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588751+0000","last_change":"2024-09-10T08:36:56.148537+0000","last_active":"2024-09-10T08:39:11.588751+0000","last_peered":"2024-09-10T08:39:11.588751+0000","last_clean":"2024-09-10T08:39:11.588751+0000","last_became_active":"2024-09-10T08:36:56.148185+0000","last_became_peered":"2024-09-10T08:36:56.148185+0000","last_unstale":"2024-09-10T08:39:11.588751+0000","last_undegraded":"2024-09-10T08:39:11.588751+0000","last_fullsized":"2024-09-10T08:39:11.588751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:00.426513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.586233+0000","last_change":"2024-09-10T08:36:56.138513+0000","last_active":"2024-09-10T08:39:11.586233+0000","last_peered":"2024-09-10T08:39:11.586233+0000","last_clean":"2024-09-10T08:39:11.586233+0000","last_became_active":"2024-09-10T08:36:56.138207+0000","last_became_peered":"2024-09-10T08:36:56.138207+0000","last_unstale":"2024-09-10T08:39:11.586233+0000","last_undegraded":"2024-09-10T08:39:11.586233+0000","last_fullsized":"2024-09-10T08:39:11.586233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:34.581430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.582686+0000","last_change":"2024-09-10T08:36:56.146612+0000","last_active":"2024-09-10T08:39:11.582686+0000","last_peered":"2024-09-10T08:39:11.582686+0000","last_clean":"2024-09-10T08:39:11.582686+0000","last_became_active":"2024-09-10T08:36:56.146251+0000","last_became_peered":"2024-09-10T08:36:56.146251+0000","last_unstale":"2024-09-10T08:39:11.582686+0000","last_undegraded":"2024-09-10T08:39:11.582686+0000","last_fullsized":"2024-09-10T08:39:11.582686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:47.336389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567939+0000","last_change":"2024-09-10T08:36:56.148260+0000","last_active":"2024-09-10T08:39:11.567939+0000","last_peered":"2024-09-10T08:39:11.567939+0000","last_clean":"2024-09-10T08:39:11.567939+0000","last_became_active":"2024-09-10T08:36:56.147949+0000","last_became_peered":"2024-09-10T08:36:56.147949+0000","last_unstale":"2024-09-10T08:39:11.567939+0000","last_undegraded":"2024-09-10T08:39:11.567939+0000","last_fullsized":"2024-09-10T08:39:11.567939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:52:19.762297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592986+0000","last_change":"2024-09-10T08:36:56.156045+0000","last_active":"2024-09-10T08:39:11.592986+0000","last_peered":"2024-09-10T08:39:11.592986+0000","last_clean":"2024-09-10T08:39:11.592986+0000","last_became_active":"2024-09-10T08:36:56.154727+0000","last_became_peered":"2024-09-10T08:36:56.154727+0000","last_unstale":"2024-09-10T08:39:11.592986+0000","last_undegraded":"2024-09-10T08:39:11.592986+0000","last_fullsized":"2024-09-10T08:39:11.592986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:35:39.992683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.576067+0000","last_change":"2024-09-10T08:36:56.154394+0000","last_active":"2024-09-10T08:39:11.576067+0000","last_peered":"2024-09-10T08:39:11.576067+0000","last_clean":"2024-09-10T08:39:11.576067+0000","last_became_active":"2024-09-10T08:36:56.153855+0000","last_became_peered":"2024-09-10T08:36:56.153855+0000","last_unstale":"2024-09-10T08:39:11.576067+0000","last_undegraded":"2024-09-10T08:39:11.576067+0000","last_fullsized":"2024-09-10T08:39:11.576067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:14.987147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592634+0000","last_change":"2024-09-10T08:36:56.157517+0000","last_active":"2024-09-10T08:39:11.592634+0000","last_peered":"2024-09-10T08:39:11.592634+0000","last_clean":"2024-09-10T08:39:11.592634+0000","last_became_active":"2024-09-10T08:36:56.157133+0000","last_became_peered":"2024-09-10T08:36:56.157133+0000","last_unstale":"2024-09-10T08:39:11.592634+0000","last_undegraded":"2024-09-10T08:39:11.592634+0000","last_fullsized":"2024-09-10T08:39:11.592634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:17:29.573719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69312,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":699,"num_read_kb":422,"num_write":186,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":216957,"data_compressed":5667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282455,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416432,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":547326,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678068,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28204,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716948,"statfs":{"total":95995035648,"available":95966154752,"internally_reserved":0,"allocated":1208320,"data_stored":1207391,"data_compressed":67339,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841594,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":543466,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103232,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":546900,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332165,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":1144631,"data_compressed":65450,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495690,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":542738,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757328,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":1138899,"data_compressed":65450,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920852,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":548897,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149785,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":542932,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":607798,"data_compressed":31413,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1126400,"data_stored":612008,"data_compressed":31413,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1077248,"data_stored":544126,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69360,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":70250,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":66183,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:39:45.306 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:39:45.707 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:46.023 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:45 smithi089 ceph-mon[30757]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:39:46.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:45 smithi111 ceph-mon[35220]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:39:46.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:45 smithi149 ceph-mon[36167]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:39:46.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.287+0000 7f3252800640 1 -- 172.21.15.89:0/841324440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f324c0745c0 msgr2=0x7f324c0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:46.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.287+0000 7f3252800640 1 --2- 172.21.15.89:0/841324440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f324c0745c0 0x7f324c0749a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3240009900 tx=0x7f324002a480 comp rx=0 tx=0).stop 2024-09-10T08:39:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.287+0000 7f3252800640 1 -- 172.21.15.89:0/841324440 shutdown_connections 2024-09-10T08:39:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.287+0000 7f3252800640 1 --2- 172.21.15.89:0/841324440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f324c10e260 0x7f324c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.287+0000 7f3252800640 1 --2- 172.21.15.89:0/841324440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f324c074ee0 0x7f324c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.287+0000 7f3252800640 1 --2- 172.21.15.89:0/841324440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f324c0745c0 0x7f324c0749a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.287+0000 7f3252800640 1 -- 172.21.15.89:0/841324440 >> 172.21.15.89:0/841324440 conn(0x7f324c0fd740 msgr2=0x7f324c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.287+0000 7f3252800640 1 -- 172.21.15.89:0/841324440 shutdown_connections 2024-09-10T08:39:46.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.287+0000 7f3252800640 1 -- 172.21.15.89:0/841324440 wait complete. 2024-09-10T08:39:46.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.289+0000 7f3252800640 1 Processor -- start 2024-09-10T08:39:46.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.289+0000 7f3252800640 1 -- start start 2024-09-10T08:39:46.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.289+0000 7f3252800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f324c0745c0 0x7f324c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:46.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.289+0000 7f3252800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f324c074ee0 0x7f324c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:46.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.289+0000 7f3252800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f324c10e260 0x7f324c19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:46.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.289+0000 7f3252800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f324c114000 con 0x7f324c0745c0 2024-09-10T08:39:46.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.289+0000 7f3252800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f324c113e80 con 0x7f324c10e260 2024-09-10T08:39:46.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.289+0000 7f3252800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f324c114180 con 0x7f324c074ee0 2024-09-10T08:39:46.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.290+0000 7f3251400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f324c0745c0 0x7f324c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:46.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.290+0000 7f3251400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f324c0745c0 0x7f324c19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59676/0 (socket says 172.21.15.89:59676) 2024-09-10T08:39:46.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.290+0000 7f3251400640 1 -- 172.21.15.89:0/2148992649 learned_addr learned my addr 172.21.15.89:0/2148992649 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:46.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.290+0000 7f3250a00640 1 --2- 172.21.15.89:0/2148992649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f324c074ee0 0x7f324c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:46.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.290+0000 7f3251e00640 1 --2- 172.21.15.89:0/2148992649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f324c10e260 0x7f324c19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:46.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.290+0000 7f3251400640 1 -- 172.21.15.89:0/2148992649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f324c074ee0 msgr2=0x7f324c19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:46.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.290+0000 7f3251400640 1 --2- 172.21.15.89:0/2148992649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f324c074ee0 0x7f324c19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:46.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.290+0000 7f3251400640 1 -- 172.21.15.89:0/2148992649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f324c10e260 msgr2=0x7f324c19ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:46.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.290+0000 7f3251400640 1 --2- 172.21.15.89:0/2148992649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f324c10e260 0x7f324c19ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:46.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.290+0000 7f3251400640 1 -- 172.21.15.89:0/2148992649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324c19f550 con 0x7f324c0745c0 2024-09-10T08:39:46.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.290+0000 7f3251400640 1 --2- 172.21.15.89:0/2148992649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f324c0745c0 0x7f324c19a5a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3240004770 tx=0x7f3240003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:46.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.291+0000 7f324a000640 1 -- 172.21.15.89:0/2148992649 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324003f070 con 0x7f324c0745c0 2024-09-10T08:39:46.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.291+0000 7f324a000640 1 -- 172.21.15.89:0/2148992649 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3240031ca0 con 0x7f324c0745c0 2024-09-10T08:39:46.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.291+0000 7f324a000640 1 -- 172.21.15.89:0/2148992649 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3240035030 con 0x7f324c0745c0 2024-09-10T08:39:46.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.291+0000 7f3252800640 1 -- 172.21.15.89:0/2148992649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f324c19f7e0 con 0x7f324c0745c0 2024-09-10T08:39:46.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.291+0000 7f3252800640 1 -- 172.21.15.89:0/2148992649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f324c1a7120 con 0x7f324c0745c0 2024-09-10T08:39:46.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.292+0000 7f3252800640 1 -- 172.21.15.89:0/2148992649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f324c0749a0 con 0x7f324c0745c0 2024-09-10T08:39:46.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.294+0000 7f324a000640 1 -- 172.21.15.89:0/2148992649 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f3240042070 con 0x7f324c0745c0 2024-09-10T08:39:46.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.296+0000 7f324a000640 1 --2- 172.21.15.89:0/2148992649 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f32340a2c90 0x7f32340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:46.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.296+0000 7f324a000640 1 -- 172.21.15.89:0/2148992649 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f32400ef8a0 con 0x7f324c0745c0 2024-09-10T08:39:46.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.296+0000 7f3250a00640 1 --2- 172.21.15.89:0/2148992649 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f32340a2c90 0x7f32340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:46.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.297+0000 7f3250a00640 1 --2- 172.21.15.89:0/2148992649 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f32340a2c90 0x7f32340a5150 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f324c10f650 tx=0x7f323800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:46.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.300+0000 7f324a000640 1 -- 172.21.15.89:0/2148992649 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32400bbc80 con 0x7f324c0745c0 2024-09-10T08:39:46.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.512+0000 7f3252800640 1 -- 172.21.15.89:0/2148992649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f324c1093e0 con 0x7f324c0745c0 2024-09-10T08:39:46.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.512+0000 7f324a000640 1 -- 172.21.15.89:0/2148992649 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11918 (secure 0 0 0) 0x7f32400bff80 con 0x7f324c0745c0 2024-09-10T08:39:46.513 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:39:46.513 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-10T08:39:39:721431+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:39:38.834417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24518,24496]},"id":1}]} 2024-09-10T08:39:46.513 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-10T08:39:46.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.515+0000 7f3252800640 1 -- 172.21.15.89:0/2148992649 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f32340a2c90 msgr2=0x7f32340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:46.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.515+0000 7f3252800640 1 --2- 172.21.15.89:0/2148992649 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f32340a2c90 0x7f32340a5150 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f324c10f650 tx=0x7f323800b390 comp rx=0 tx=0).stop 2024-09-10T08:39:46.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.515+0000 7f3252800640 1 -- 172.21.15.89:0/2148992649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f324c0745c0 msgr2=0x7f324c19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:46.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.515+0000 7f3252800640 1 --2- 172.21.15.89:0/2148992649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f324c0745c0 0x7f324c19a5a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3240004770 tx=0x7f3240003c10 comp rx=0 tx=0).stop 2024-09-10T08:39:46.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.515+0000 7f3252800640 1 -- 172.21.15.89:0/2148992649 shutdown_connections 2024-09-10T08:39:46.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.515+0000 7f3252800640 1 --2- 172.21.15.89:0/2148992649 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f32340a2c90 0x7f32340a5150 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:46.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.515+0000 7f3252800640 1 --2- 172.21.15.89:0/2148992649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f324c10e260 0x7f324c19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:46.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.515+0000 7f3252800640 1 --2- 172.21.15.89:0/2148992649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f324c074ee0 0x7f324c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:46.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.515+0000 7f3252800640 1 --2- 172.21.15.89:0/2148992649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f324c0745c0 0x7f324c19a5a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:46.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.515+0000 7f3252800640 1 -- 172.21.15.89:0/2148992649 >> 172.21.15.89:0/2148992649 conn(0x7f324c0fd740 msgr2=0x7f324c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:46.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.515+0000 7f3252800640 1 -- 172.21.15.89:0/2148992649 shutdown_connections 2024-09-10T08:39:46.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:46.515+0000 7f3252800640 1 -- 172.21.15.89:0/2148992649 wait complete. 2024-09-10T08:39:46.911 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:46 smithi089 ceph-mon[30757]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:39:46.911 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:46 smithi089 ceph-mon[30757]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:39:46.911 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:46 smithi089 ceph-mon[30757]: pgmap v388: 129 pgs: 129 active+clean; 649 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-10T08:39:46.912 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:46 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2148992649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:39:47.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:46 smithi111 ceph-mon[35220]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:39:47.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:46 smithi111 ceph-mon[35220]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:39:47.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:46 smithi111 ceph-mon[35220]: pgmap v388: 129 pgs: 129 active+clean; 649 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-10T08:39:47.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:46 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2148992649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:39:47.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:46 smithi149 ceph-mon[36167]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:39:47.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:46 smithi149 ceph-mon[36167]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:39:47.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:46 smithi149 ceph-mon[36167]: pgmap v388: 129 pgs: 129 active+clean; 649 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-10T08:39:47.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:46 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2148992649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:39:47.442 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:39:47.812 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:48.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.390+0000 7f4073e00640 1 -- 172.21.15.89:0/3072921848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40741079c0 msgr2=0x7f4074107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:48.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.390+0000 7f4073e00640 1 --2- 172.21.15.89:0/3072921848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40741079c0 0x7f4074107da0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4064009930 tx=0x7f406402a420 comp rx=0 tx=0).stop 2024-09-10T08:39:48.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.390+0000 7f4073e00640 1 -- 172.21.15.89:0/3072921848 shutdown_connections 2024-09-10T08:39:48.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.390+0000 7f4073e00640 1 --2- 172.21.15.89:0/3072921848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4074102360 0x7f407410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:48.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.390+0000 7f4073e00640 1 --2- 172.21.15.89:0/3072921848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40741019c0 0x7f4074101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:48.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.390+0000 7f4073e00640 1 --2- 172.21.15.89:0/3072921848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40741079c0 0x7f4074107da0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:48.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.390+0000 7f4073e00640 1 -- 172.21.15.89:0/3072921848 >> 172.21.15.89:0/3072921848 conn(0x7f40740fd740 msgr2=0x7f40740ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:48.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.390+0000 7f4073e00640 1 -- 172.21.15.89:0/3072921848 shutdown_connections 2024-09-10T08:39:48.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.390+0000 7f4073e00640 1 -- 172.21.15.89:0/3072921848 wait complete. 2024-09-10T08:39:48.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.391+0000 7f4073e00640 1 Processor -- start 2024-09-10T08:39:48.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.391+0000 7f4073e00640 1 -- start start 2024-09-10T08:39:48.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.391+0000 7f4073e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40741019c0 0x7f407419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:48.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.392+0000 7f4073e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4074102360 0x7f407419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:48.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.392+0000 7f4073e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40741079c0 0x7f407419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:48.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.392+0000 7f4073e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4074111e10 con 0x7f40741019c0 2024-09-10T08:39:48.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.392+0000 7f4073e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4074111c90 con 0x7f4074102360 2024-09-10T08:39:48.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.392+0000 7f4073e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4074111f90 con 0x7f40741079c0 2024-09-10T08:39:48.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.392+0000 7f4072a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40741019c0 0x7f407419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:48.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.392+0000 7f4072a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40741019c0 0x7f407419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59700/0 (socket says 172.21.15.89:59700) 2024-09-10T08:39:48.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.392+0000 7f4072a00640 1 -- 172.21.15.89:0/3182879371 learned_addr learned my addr 172.21.15.89:0/3182879371 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:48.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.392+0000 7f406be00640 1 --2- 172.21.15.89:0/3182879371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4074102360 0x7f407419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:48.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.392+0000 7f4073400640 1 --2- 172.21.15.89:0/3182879371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40741079c0 0x7f407419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:48.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.392+0000 7f4072a00640 1 -- 172.21.15.89:0/3182879371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40741079c0 msgr2=0x7f407419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:48.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.392+0000 7f4072a00640 1 --2- 172.21.15.89:0/3182879371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40741079c0 0x7f407419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:48.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.392+0000 7f4072a00640 1 -- 172.21.15.89:0/3182879371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4074102360 msgr2=0x7f407419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:48.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.392+0000 7f4072a00640 1 --2- 172.21.15.89:0/3182879371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4074102360 0x7f407419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:48.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.392+0000 7f4072a00640 1 -- 172.21.15.89:0/3182879371 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407419f4a0 con 0x7f40741019c0 2024-09-10T08:39:48.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.392+0000 7f406be00640 1 --2- 172.21.15.89:0/3182879371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4074102360 0x7f407419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:48.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.392+0000 7f4073400640 1 --2- 172.21.15.89:0/3182879371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40741079c0 0x7f407419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:39:48.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.393+0000 7f4072a00640 1 --2- 172.21.15.89:0/3182879371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40741019c0 0x7f407419a4f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f406402a940 tx=0x7f4064003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:48.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.393+0000 7f4078a00640 1 -- 172.21.15.89:0/3182879371 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406403f070 con 0x7f40741019c0 2024-09-10T08:39:48.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.393+0000 7f4078a00640 1 -- 172.21.15.89:0/3182879371 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4064031c90 con 0x7f40741019c0 2024-09-10T08:39:48.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.393+0000 7f4078a00640 1 -- 172.21.15.89:0/3182879371 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064035030 con 0x7f40741019c0 2024-09-10T08:39:48.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.393+0000 7f4073e00640 1 -- 172.21.15.89:0/3182879371 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f407419f730 con 0x7f40741019c0 2024-09-10T08:39:48.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.393+0000 7f4073e00640 1 -- 172.21.15.89:0/3182879371 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f407419fbc0 con 0x7f40741019c0 2024-09-10T08:39:48.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.394+0000 7f4073e00640 1 -- 172.21.15.89:0/3182879371 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4074101e20 con 0x7f40741019c0 2024-09-10T08:39:48.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.396+0000 7f4078a00640 1 -- 172.21.15.89:0/3182879371 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f4064042070 con 0x7f40741019c0 2024-09-10T08:39:48.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.397+0000 7f4078a00640 1 --2- 172.21.15.89:0/3182879371 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f40580a2c90 0x7f40580a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:48.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.397+0000 7f4078a00640 1 -- 172.21.15.89:0/3182879371 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f40640efcc0 con 0x7f40741019c0 2024-09-10T08:39:48.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.398+0000 7f406be00640 1 --2- 172.21.15.89:0/3182879371 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f40580a2c90 0x7f40580a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:48.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.398+0000 7f406be00640 1 --2- 172.21.15.89:0/3182879371 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f40580a2c90 0x7f40580a5150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f407410e540 tx=0x7f405c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:48.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.402+0000 7f4078a00640 1 -- 172.21.15.89:0/3182879371 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40640f6080 con 0x7f40741019c0 2024-09-10T08:39:48.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.578+0000 7f4073e00640 1 -- 172.21.15.89:0/3182879371 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40741a7190 con 0x7f40741019c0 2024-09-10T08:39:48.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.580+0000 7f4078a00640 1 -- 172.21.15.89:0/3182879371 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24894 (secure 0 0 0) 0x7f40640bc690 con 0x7f40741019c0 2024-09-10T08:39:48.580 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:39:48.580 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":96,"fsid":"91f747bc-6f4e-11ef-bcea-c7b262605968","created":"2024-09-10T08:28:49.697035+0000","modified":"2024-09-10T08:38:43.618497+0000","last_up_change":"2024-09-10T08:36:00.535433+0000","last_in_change":"2024-09-10T08:35:37.131131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:32:02.060714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:36:52.462301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:36:55.119708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:37:03.729002+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d3b55192-bd95-4497-89ea-dc017a25c0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6803","nonce":1760523284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6805","nonce":1760523284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6809","nonce":1760523284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6807","nonce":1760523284}]},"public_addr":"172.21.15.89:6803/1760523284","cluster_addr":"172.21.15.89:6805/1760523284","heartbeat_back_addr":"172.21.15.89:6809/1760523284","heartbeat_front_addr":"172.21.15.89:6807/1760523284","state":["exists","up"]},{"osd":1,"uuid":"60aae117-cb52-4034-a700-11693171cd52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6801","nonce":2369291742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6803","nonce":2369291742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6807","nonce":2369291742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6805","nonce":2369291742}]},"public_addr":"172.21.15.111:6801/2369291742","cluster_addr":"172.21.15.111:6803/2369291742","heartbeat_back_addr":"172.21.15.111:6807/2369291742","heartbeat_front_addr":"172.21.15.111:6805/2369291742","state":["exists","up"]},{"osd":2,"uuid":"a305a019-0870-437c-a298-4f923ec54e77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6801","nonce":1092261395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6803","nonce":1092261395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6807","nonce":1092261395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6805","nonce":1092261395}]},"public_addr":"172.21.15.149:6801/1092261395","cluster_addr":"172.21.15.149:6803/1092261395","heartbeat_back_addr":"172.21.15.149:6807/1092261395","heartbeat_front_addr":"172.21.15.149:6805/1092261395","state":["exists","up"]},{"osd":3,"uuid":"d02f54c1-f883-40f6-9341-8d9fc4b712ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6811","nonce":1405773950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6813","nonce":1405773950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6817","nonce":1405773950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6815","nonce":1405773950}]},"public_addr":"172.21.15.89:6811/1405773950","cluster_addr":"172.21.15.89:6813/1405773950","heartbeat_back_addr":"172.21.15.89:6817/1405773950","heartbeat_front_addr":"172.21.15.89:6815/1405773950","state":["exists","up"]},{"osd":4,"uuid":"23cff46b-88dd-4344-be06-b35db6014f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6809","nonce":316717323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6811","nonce":316717323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6815","nonce":316717323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6813","nonce":316717323}]},"public_addr":"172.21.15.111:6809/316717323","cluster_addr":"172.21.15.111:6811/316717323","heartbeat_back_addr":"172.21.15.111:6815/316717323","heartbeat_front_addr":"172.21.15.111:6813/316717323","state":["exists","up"]},{"osd":5,"uuid":"8e516e8d-e7c6-4ebc-80a5-29932161df4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6809","nonce":1256148461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6811","nonce":1256148461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6815","nonce":1256148461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6813","nonce":1256148461}]},"public_addr":"172.21.15.149:6809/1256148461","cluster_addr":"172.21.15.149:6811/1256148461","heartbeat_back_addr":"172.21.15.149:6815/1256148461","heartbeat_front_addr":"172.21.15.149:6813/1256148461","state":["exists","up"]},{"osd":6,"uuid":"48f92472-f657-4c5a-adeb-20d4fea1ce8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6819","nonce":4190488024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6821","nonce":4190488024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6825","nonce":4190488024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6823","nonce":4190488024}]},"public_addr":"172.21.15.89:6819/4190488024","cluster_addr":"172.21.15.89:6821/4190488024","heartbeat_back_addr":"172.21.15.89:6825/4190488024","heartbeat_front_addr":"172.21.15.89:6823/4190488024","state":["exists","up"]},{"osd":7,"uuid":"14a2a6f0-c4cd-406b-b637-7527fd32babe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6817","nonce":652666376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6819","nonce":652666376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6823","nonce":652666376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6821","nonce":652666376}]},"public_addr":"172.21.15.111:6817/652666376","cluster_addr":"172.21.15.111:6819/652666376","heartbeat_back_addr":"172.21.15.111:6823/652666376","heartbeat_front_addr":"172.21.15.111:6821/652666376","state":["exists","up"]},{"osd":8,"uuid":"a54502e6-95f7-431f-8db1-54a6c7983b5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6817","nonce":2955172869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6819","nonce":2955172869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6823","nonce":2955172869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6821","nonce":2955172869}]},"public_addr":"172.21.15.149:6817/2955172869","cluster_addr":"172.21.15.149:6819/2955172869","heartbeat_back_addr":"172.21.15.149:6823/2955172869","heartbeat_front_addr":"172.21.15.149:6821/2955172869","state":["exists","up"]},{"osd":9,"uuid":"5fe17344-5b1c-4601-a633-639cc1691478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6827","nonce":2427467040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6829","nonce":2427467040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6833","nonce":2427467040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6831","nonce":2427467040}]},"public_addr":"172.21.15.89:6827/2427467040","cluster_addr":"172.21.15.89:6829/2427467040","heartbeat_back_addr":"172.21.15.89:6833/2427467040","heartbeat_front_addr":"172.21.15.89:6831/2427467040","state":["exists","up"]},{"osd":10,"uuid":"d8f29890-3064-4b67-b649-467a0ac86db7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6825","nonce":2328678744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6827","nonce":2328678744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6831","nonce":2328678744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6829","nonce":2328678744}]},"public_addr":"172.21.15.111:6825/2328678744","cluster_addr":"172.21.15.111:6827/2328678744","heartbeat_back_addr":"172.21.15.111:6831/2328678744","heartbeat_front_addr":"172.21.15.111:6829/2328678744","state":["exists","up"]},{"osd":11,"uuid":"92182c73-f139-47ca-a0ef-5046ae9b2310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6825","nonce":3130508827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6827","nonce":3130508827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6831","nonce":3130508827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6829","nonce":3130508827}]},"public_addr":"172.21.15.149:6825/3130508827","cluster_addr":"172.21.15.149:6827/3130508827","heartbeat_back_addr":"172.21.15.149:6831/3130508827","heartbeat_front_addr":"172.21.15.149:6829/3130508827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:09.031810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:33.695124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:58.065510+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:27.592838+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:53.275289+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:18.305677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:47.781535+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:12.889512+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:37.744193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:07.718408+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:33.391027+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:59.013250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/850669510":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/3541027428":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6801/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6800/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/2396468787":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/1609720980":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/995019144":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6801/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6800/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/2179977257":"2024-09-11T08:29:14.554869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:39:48.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.582+0000 7f4073e00640 1 -- 172.21.15.89:0/3182879371 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f40580a2c90 msgr2=0x7f40580a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:48.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.582+0000 7f4073e00640 1 --2- 172.21.15.89:0/3182879371 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f40580a2c90 0x7f40580a5150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f407410e540 tx=0x7f405c005f90 comp rx=0 tx=0).stop 2024-09-10T08:39:48.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.582+0000 7f4073e00640 1 -- 172.21.15.89:0/3182879371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40741019c0 msgr2=0x7f407419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:48.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.582+0000 7f4073e00640 1 --2- 172.21.15.89:0/3182879371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40741019c0 0x7f407419a4f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f406402a940 tx=0x7f4064003b90 comp rx=0 tx=0).stop 2024-09-10T08:39:48.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.582+0000 7f4073e00640 1 -- 172.21.15.89:0/3182879371 shutdown_connections 2024-09-10T08:39:48.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.582+0000 7f4073e00640 1 --2- 172.21.15.89:0/3182879371 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f40580a2c90 0x7f40580a5150 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:48.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.582+0000 7f4073e00640 1 --2- 172.21.15.89:0/3182879371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40741079c0 0x7f407419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:48.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.582+0000 7f4073e00640 1 --2- 172.21.15.89:0/3182879371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4074102360 0x7f407419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.582+0000 7f4073e00640 1 --2- 172.21.15.89:0/3182879371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40741019c0 0x7f407419a4f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.582+0000 7f4073e00640 1 -- 172.21.15.89:0/3182879371 >> 172.21.15.89:0/3182879371 conn(0x7f40740fd740 msgr2=0x7f40740fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.582+0000 7f4073e00640 1 -- 172.21.15.89:0/3182879371 shutdown_connections 2024-09-10T08:39:48.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:48.583+0000 7f4073e00640 1 -- 172.21.15.89:0/3182879371 wait complete. 2024-09-10T08:39:48.958 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:48 smithi089 ceph-mon[30757]: pgmap v389: 129 pgs: 129 active+clean; 649 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-10T08:39:48.958 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:48 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3182879371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:39:49.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:48 smithi111 ceph-mon[35220]: pgmap v389: 129 pgs: 129 active+clean; 649 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-10T08:39:49.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:48 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3182879371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:39:49.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:48 smithi149 ceph-mon[36167]: pgmap v389: 129 pgs: 129 active+clean; 649 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-10T08:39:49.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:48 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3182879371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:39:49.444 DEBUG:tasks.fwd_scrub:Yielding 2024-09-10T08:39:49.444 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-10T08:39:49.456 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi089.front.sepia.ceph.com 2024-09-10T08:39:49.456 DEBUG:teuthology.orchestra.run.smithi089:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-10T08:39:49.456 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-10T08:39:49.456 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:39:49.457 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:39:49.461 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-10T08:39:49.474 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T08:39:49.474 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:39:49.842 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:49.860 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:50.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.439+0000 7f2336200640 1 -- 172.21.15.89:0/2647497264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2330103e30 msgr2=0x7f2330104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:50.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.439+0000 7f2336200640 1 --2- 172.21.15.89:0/2647497264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2330103e30 0x7f2330104210 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2320009900 tx=0x7f232002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.439+0000 7f2336200640 1 -- 172.21.15.89:0/2647497264 shutdown_connections 2024-09-10T08:39:50.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.439+0000 7f2336200640 1 --2- 172.21.15.89:0/2647497264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2330108f60 0x7f2330110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.439+0000 7f2336200640 1 --2- 172.21.15.89:0/2647497264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23301047e0 0x7f2330108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.439+0000 7f2336200640 1 --2- 172.21.15.89:0/2647497264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2330103e30 0x7f2330104210 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.439+0000 7f2336200640 1 -- 172.21.15.89:0/2647497264 >> 172.21.15.89:0/2647497264 conn(0x7f23300ff8d0 msgr2=0x7f2330101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:50.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.439+0000 7f2336200640 1 -- 172.21.15.89:0/2647497264 shutdown_connections 2024-09-10T08:39:50.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.439+0000 7f2336200640 1 -- 172.21.15.89:0/2647497264 wait complete. 2024-09-10T08:39:50.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.440+0000 7f2336200640 1 Processor -- start 2024-09-10T08:39:50.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.440+0000 7f2336200640 1 -- start start 2024-09-10T08:39:50.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.441+0000 7f2336200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2330103e30 0x7f233019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:50.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.441+0000 7f2336200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23301047e0 0x7f233019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:50.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.441+0000 7f2336200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2330108f60 0x7f233019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:50.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.441+0000 7f2336200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2330113fd0 con 0x7f2330108f60 2024-09-10T08:39:50.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.441+0000 7f2336200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2330113e50 con 0x7f23301047e0 2024-09-10T08:39:50.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.441+0000 7f2336200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2330114150 con 0x7f2330103e30 2024-09-10T08:39:50.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.441+0000 7f2335800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2330108f60 0x7f233019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:50.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.441+0000 7f2327e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23301047e0 0x7f233019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:50.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.441+0000 7f2335800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2330108f60 0x7f233019ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59724/0 (socket says 172.21.15.89:59724) 2024-09-10T08:39:50.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.441+0000 7f2327e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23301047e0 0x7f233019a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:53380/0 (socket says 172.21.15.89:53380) 2024-09-10T08:39:50.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.441+0000 7f2335800640 1 -- 172.21.15.89:0/1806268067 learned_addr learned my addr 172.21.15.89:0/1806268067 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:50.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.441+0000 7f2334e00640 1 --2- 172.21.15.89:0/1806268067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2330103e30 0x7f233019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:50.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.441+0000 7f2327e00640 1 -- 172.21.15.89:0/1806268067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2330103e30 msgr2=0x7f233019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:50.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.441+0000 7f2327e00640 1 --2- 172.21.15.89:0/1806268067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2330103e30 0x7f233019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.442+0000 7f2327e00640 1 -- 172.21.15.89:0/1806268067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2330108f60 msgr2=0x7f233019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:50.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.442+0000 7f2327e00640 1 --2- 172.21.15.89:0/1806268067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2330108f60 0x7f233019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.442+0000 7f2327e00640 1 -- 172.21.15.89:0/1806268067 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f233019f420 con 0x7f23301047e0 2024-09-10T08:39:50.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.442+0000 7f2327e00640 1 --2- 172.21.15.89:0/1806268067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23301047e0 0x7f233019a9b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f231800d6d0 tx=0x7f231800db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:50.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.443+0000 7f232e000640 1 -- 172.21.15.89:0/1806268067 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318017070 con 0x7f23301047e0 2024-09-10T08:39:50.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.443+0000 7f2336200640 1 -- 172.21.15.89:0/1806268067 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f233019f710 con 0x7f23301047e0 2024-09-10T08:39:50.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.443+0000 7f232e000640 1 -- 172.21.15.89:0/1806268067 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2318009820 con 0x7f23301047e0 2024-09-10T08:39:50.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.443+0000 7f232e000640 1 -- 172.21.15.89:0/1806268067 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318004cd0 con 0x7f23301047e0 2024-09-10T08:39:50.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.443+0000 7f2336200640 1 -- 172.21.15.89:0/1806268067 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23301a6ff0 con 0x7f23301047e0 2024-09-10T08:39:50.445 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.446+0000 7f233a1fe740 1 Processor -- start 2024-09-10T08:39:50.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.446+0000 7f232e000640 1 -- 172.21.15.89:0/1806268067 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f23180099d0 con 0x7f23301047e0 2024-09-10T08:39:50.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.446+0000 7f233a1fe740 1 -- start start 2024-09-10T08:39:50.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.446+0000 7f233a1fe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026ff4cb0 0x559026ff5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:50.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.446+0000 7f233a1fe740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026ff55d0 0x559026fff950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:50.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.446+0000 7f233a1fe740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026fffe90 0x559027002280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:50.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.446+0000 7f233a1fe740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559026f5b290 con 0x559026fffe90 2024-09-10T08:39:50.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.446+0000 7f233a1fe740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559026f02b40 con 0x559026ff55d0 2024-09-10T08:39:50.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.446+0000 7f233a1fe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559026f57250 con 0x559026ff4cb0 2024-09-10T08:39:50.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.447+0000 7f2334e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026ff55d0 0x559026fff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:50.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.447+0000 7f2327e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026fffe90 0x559027002280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:50.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.447+0000 7f2335800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026ff4cb0 0x559026ff5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:50.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.447+0000 7f2334e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026ff55d0 0x559026fff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:53386/0 (socket says 172.21.15.89:53386) 2024-09-10T08:39:50.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.447+0000 7f2334e00640 1 -- 172.21.15.89:0/1898641221 learned_addr learned my addr 172.21.15.89:0/1898641221 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:50.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.447+0000 7f2334e00640 1 -- 172.21.15.89:0/1898641221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026ff4cb0 msgr2=0x559026ff5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:50.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.447+0000 7f2334e00640 1 --2- 172.21.15.89:0/1898641221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026ff4cb0 0x559026ff5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.447+0000 7f2334e00640 1 -- 172.21.15.89:0/1898641221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026fffe90 msgr2=0x559027002280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:50.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.447+0000 7f2334e00640 1 --2- 172.21.15.89:0/1898641221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026fffe90 0x559027002280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.447+0000 7f2334e00640 1 -- 172.21.15.89:0/1898641221 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559026ccc6a0 con 0x559026ff55d0 2024-09-10T08:39:50.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.448+0000 7f232e000640 1 --2- 172.21.15.89:0/1806268067 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23140a2c10 0x7f23140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:50.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.448+0000 7f2334e00640 1 --2- 172.21.15.89:0/1898641221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026ff55d0 0x559026fff950 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2320009900 tx=0x7f2320004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72907c1b8d47b035 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:50.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.448+0000 7f232e000640 1 -- 172.21.15.89:0/1806268067 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f23180cf0f0 con 0x7f23301047e0 2024-09-10T08:39:50.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.448+0000 7f2335800640 1 --2- 172.21.15.89:0/1806268067 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23140a2c10 0x7f23140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:50.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.448+0000 7f2326000640 1 -- 172.21.15.89:0/1898641221 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320035040 con 0x559026ff55d0 2024-09-10T08:39:50.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.449+0000 7f2326000640 1 -- 172.21.15.89:0/1898641221 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2320004430 con 0x559026ff55d0 2024-09-10T08:39:50.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.449+0000 7f2326000640 1 -- 172.21.15.89:0/1898641221 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23200035f0 con 0x559026ff55d0 2024-09-10T08:39:50.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.449+0000 7f233a1fe740 1 -- 172.21.15.89:0/1898641221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026ff55d0 msgr2=0x559026fff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:50.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.449+0000 7f2335800640 1 --2- 172.21.15.89:0/1806268067 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23140a2c10 0x7f23140a50d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f23280049d0 tx=0x7f2328009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:50.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.449+0000 7f233a1fe740 1 --2- 172.21.15.89:0/1898641221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026ff55d0 0x559026fff950 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2320009900 tx=0x7f2320004040 comp rx=0 tx=0).stop 2024-09-10T08:39:50.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.449+0000 7f233a1fe740 1 -- 172.21.15.89:0/1898641221 shutdown_connections 2024-09-10T08:39:50.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.450+0000 7f233a1fe740 1 --2- 172.21.15.89:0/1898641221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026fffe90 0x559027002280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.450+0000 7f233a1fe740 1 --2- 172.21.15.89:0/1898641221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026ff55d0 0x559026fff950 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.450+0000 7f233a1fe740 1 --2- 172.21.15.89:0/1898641221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026ff4cb0 0x559026ff5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.450+0000 7f233a1fe740 1 -- 172.21.15.89:0/1898641221 >> 172.21.15.89:0/1898641221 conn(0x559026ff1240 msgr2=0x559026ff1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:50.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.450+0000 7f233a1fe740 1 -- 172.21.15.89:0/1898641221 shutdown_connections 2024-09-10T08:39:50.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.450+0000 7f233a1fe740 1 -- 172.21.15.89:0/1898641221 wait complete. 2024-09-10T08:39:50.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.450+0000 7f233a1fe740 1 -- start start 2024-09-10T08:39:50.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.450+0000 7f233a1fe740 1 Processor -- start 2024-09-10T08:39:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.451+0000 7f233a1fe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026ff4cb0 0x559026fff710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.451+0000 7f233a1fe740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026ff55d0 0x559026ffc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.451+0000 7f233a1fe740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026fffe90 0x559026ffcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.451+0000 7f233a1fe740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559027002b00 con 0x559026ff55d0 2024-09-10T08:39:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.451+0000 7f233a1fe740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559026f15fa0 con 0x559026fffe90 2024-09-10T08:39:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.451+0000 7f233a1fe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559027002c80 con 0x559026ff4cb0 2024-09-10T08:39:50.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.451+0000 7f233a1fe740 10 client.0 _ll_register_callbacks cb 0x559026c69ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:39:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.451+0000 7f2327e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026ff55d0 0x559026ffc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.451+0000 7f2327e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026ff55d0 0x559026ffc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59736/0 (socket says 172.21.15.89:59736) 2024-09-10T08:39:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.451+0000 7f2327e00640 1 -- 172.21.15.89:0/2132117020 learned_addr learned my addr 172.21.15.89:0/2132117020 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.451+0000 7f2335800640 1 --2- 172.21.15.89:0/2132117020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026fffe90 0x559026ffcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.451+0000 7f2334e00640 1 --2- 172.21.15.89:0/2132117020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026ff4cb0 0x559026fff710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.452+0000 7f2327e00640 1 -- 172.21.15.89:0/2132117020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026ff4cb0 msgr2=0x559026fff710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.452+0000 7f2327e00640 1 --2- 172.21.15.89:0/2132117020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026ff4cb0 0x559026fff710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.452+0000 7f2327e00640 1 -- 172.21.15.89:0/2132117020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026fffe90 msgr2=0x559026ffcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.452+0000 7f2327e00640 1 --2- 172.21.15.89:0/2132117020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026fffe90 0x559026ffcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.452+0000 7f2327e00640 1 -- 172.21.15.89:0/2132117020 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559026ffd6e0 con 0x559026ff55d0 2024-09-10T08:39:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.452+0000 7f2335800640 1 --2- 172.21.15.89:0/2132117020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026fffe90 0x559026ffcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.452+0000 7f2327e00640 1 --2- 172.21.15.89:0/2132117020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026ff55d0 0x559026ffc9c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f231809b400 tx=0x7f23180977d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c324ca5e3c585bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:50.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.452+0000 7f231ea00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:39:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.452+0000 7f231ea00640 1 -- 172.21.15.89:0/2132117020 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318020050 con 0x559026ff55d0 2024-09-10T08:39:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.452+0000 7f231ea00640 1 -- 172.21.15.89:0/2132117020 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f2318097ae0 con 0x559026ff55d0 2024-09-10T08:39:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.452+0000 7f233a1fe740 1 -- 172.21.15.89:0/2132117020 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2320004430 con 0x559026ff55d0 2024-09-10T08:39:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.453+0000 7f231ea00640 1 -- 172.21.15.89:0/2132117020 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f231809b470 con 0x559026ff55d0 2024-09-10T08:39:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.453+0000 7f233a1fe740 10 client.15198 fetch_fsmap learned FSMap version 26 2024-09-10T08:39:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.453+0000 7f233a1fe740 1 -- 172.21.15.89:0/2132117020 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55902700e990 con 0x559026ff55d0 2024-09-10T08:39:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.453+0000 7f231ea00640 1 -- 172.21.15.89:0/2132117020 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2320004430 con 0x559026ff55d0 2024-09-10T08:39:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.453+0000 7f231ea00640 1 -- 172.21.15.89:0/2132117020 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f23180d1110 con 0x559026ff55d0 2024-09-10T08:39:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.453+0000 7f233a1fe740 10 client.15198 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:39:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.453+0000 7f233a1fe740 10 client.15198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:39:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.453+0000 7f233a1fe740 20 client.15198 populate_metadata read hostname 'smithi089' 2024-09-10T08:39:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.453+0000 7f233a1fe740 1 --2- 172.21.15.89:0/2132117020 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55902700fe50 0x559027012280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.453+0000 7f233a1fe740 4 client.15198 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:39:50.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.453+0000 7f233a1fe740 1 -- 172.21.15.89:0/2132117020 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559026f68800 con 0x55902700fe50 2024-09-10T08:39:50.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.454+0000 7f2334e00640 1 --2- 172.21.15.89:0/2132117020 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55902700fe50 0x559027012280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:50.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.454+0000 7f2334e00640 1 --2- 172.21.15.89:0/2132117020 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55902700fe50 0x559027012280 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d45c412718053497 server_cookie=ce25dfdb78836be5 in_seq=0 out_seq=0 2024-09-10T08:39:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.454+0000 7f231ea00640 10 client.15198 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:39:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.455+0000 7f231ea00640 1 -- 172.21.15.89:0/2132117020 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559026f68800 con 0x55902700fe50 2024-09-10T08:39:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.455+0000 7f231ea00640 10 client.15198 handle_command_reply: tid=0 2024-09-10T08:39:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.455+0000 7f233a1fe740 1 client.15198 shutdown 2024-09-10T08:39:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.455+0000 7f233a1fe740 1 -- 172.21.15.89:0/2132117020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026ff55d0 msgr2=0x559026ffc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.455+0000 7f233a1fe740 1 --2- 172.21.15.89:0/2132117020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026ff55d0 0x559026ffc9c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f231809b400 tx=0x7f23180977d0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.455+0000 7f233a1fe740 1 -- 172.21.15.89:0/2132117020 shutdown_connections 2024-09-10T08:39:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.455+0000 7f233a1fe740 1 --2- 172.21.15.89:0/2132117020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026fffe90 0x559026ffcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.455+0000 7f233a1fe740 1 --2- 172.21.15.89:0/2132117020 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55902700fe50 0x559027012280 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.455+0000 7f233a1fe740 1 --2- 172.21.15.89:0/2132117020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026ff55d0 0x559026ffc9c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.455+0000 7f233a1fe740 1 --2- 172.21.15.89:0/2132117020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026ff4cb0 0x559026fff710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.455+0000 7f233a1fe740 1 -- 172.21.15.89:0/2132117020 >> 172.21.15.89:0/2132117020 conn(0x559026ff1240 msgr2=0x559026ff6f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.456+0000 7f231ea00640 0 client.15198 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:39:50.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.456+0000 7f233a1fe740 1 -- 172.21.15.89:0/2132117020 shutdown_connections 2024-09-10T08:39:50.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.456+0000 7f233a1fe740 1 -- 172.21.15.89:0/2132117020 wait complete. 2024-09-10T08:39:50.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.456+0000 7f233a1fe740 20 client.15198 trim_cache size 0 max 16384 2024-09-10T08:39:50.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.474+0000 7f233a1fe740 1 Processor -- start 2024-09-10T08:39:50.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.474+0000 7f233a1fe740 1 -- start start 2024-09-10T08:39:50.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.474+0000 7f233a1fe740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026ff4cb0 0x559027017b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:50.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.474+0000 7f233a1fe740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026ff55d0 0x559027018090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:50.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.474+0000 7f233a1fe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026fffe90 0x559026ff9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:50.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.474+0000 7f233a1fe740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f23301b3760 con 0x559026ff55d0 2024-09-10T08:39:50.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.474+0000 7f233a1fe740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f23301b1030 con 0x559026ff4cb0 2024-09-10T08:39:50.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.474+0000 7f233a1fe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f23301aee80 con 0x559026fffe90 2024-09-10T08:39:50.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.474+0000 7f2327e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026ff55d0 0x559027018090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:50.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.474+0000 7f2334e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026ff4cb0 0x559027017b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:50.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.474+0000 7f2335800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026fffe90 0x559026ff9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:50.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.474+0000 7f2334e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026ff4cb0 0x559027017b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:53402/0 (socket says 172.21.15.89:53402) 2024-09-10T08:39:50.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.474+0000 7f2334e00640 1 -- 172.21.15.89:0/3984647510 learned_addr learned my addr 172.21.15.89:0/3984647510 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:50.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.474+0000 7f2335800640 1 --2- 172.21.15.89:0/3984647510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026fffe90 0x559026ff9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:34738/0 (socket says 172.21.15.89:34738) 2024-09-10T08:39:50.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.475+0000 7f2334e00640 1 -- 172.21.15.89:0/3984647510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026fffe90 msgr2=0x559026ff9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:50.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.475+0000 7f2334e00640 1 --2- 172.21.15.89:0/3984647510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026fffe90 0x559026ff9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.475+0000 7f2334e00640 1 -- 172.21.15.89:0/3984647510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026ff55d0 msgr2=0x559027018090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:50.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.475+0000 7f2334e00640 1 --2- 172.21.15.89:0/3984647510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026ff55d0 0x559027018090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.475+0000 7f2334e00640 1 -- 172.21.15.89:0/3984647510 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55902700bd90 con 0x559026ff4cb0 2024-09-10T08:39:50.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.475+0000 7f2334e00640 1 --2- 172.21.15.89:0/3984647510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026ff4cb0 0x559027017b50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2320004bd0 tx=0x7f2320004b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd19efe86f4621de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:50.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.475+0000 7f2325600640 1 -- 172.21.15.89:0/3984647510 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320035040 con 0x559026ff4cb0 2024-09-10T08:39:50.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.475+0000 7f2325600640 1 -- 172.21.15.89:0/3984647510 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23200043f0 con 0x559026ff4cb0 2024-09-10T08:39:50.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.475+0000 7f2325600640 1 -- 172.21.15.89:0/3984647510 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320003850 con 0x559026ff4cb0 2024-09-10T08:39:50.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.476+0000 7f233a1fe740 1 -- 172.21.15.89:0/3984647510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026ff4cb0 msgr2=0x559027017b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:50.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.476+0000 7f233a1fe740 1 --2- 172.21.15.89:0/3984647510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026ff4cb0 0x559027017b50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2320004bd0 tx=0x7f2320004b60 comp rx=0 tx=0).stop 2024-09-10T08:39:50.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.476+0000 7f233a1fe740 1 -- 172.21.15.89:0/3984647510 shutdown_connections 2024-09-10T08:39:50.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.476+0000 7f233a1fe740 1 --2- 172.21.15.89:0/3984647510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026fffe90 0x559026ff9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.476+0000 7f233a1fe740 1 --2- 172.21.15.89:0/3984647510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026ff55d0 0x559027018090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.476+0000 7f233a1fe740 1 --2- 172.21.15.89:0/3984647510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026ff4cb0 0x559027017b50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.476+0000 7f233a1fe740 1 -- 172.21.15.89:0/3984647510 >> 172.21.15.89:0/3984647510 conn(0x559026ff1240 msgr2=0x559026ffb290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:50.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.476+0000 7f233a1fe740 1 -- 172.21.15.89:0/3984647510 shutdown_connections 2024-09-10T08:39:50.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.476+0000 7f233a1fe740 1 -- 172.21.15.89:0/3984647510 wait complete. 2024-09-10T08:39:50.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.476+0000 7f233a1fe740 1 -- start start 2024-09-10T08:39:50.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.477+0000 7f233a1fe740 1 Processor -- start 2024-09-10T08:39:50.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.477+0000 7f233a1fe740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026ff4cb0 0x559027003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:50.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.477+0000 7f233a1fe740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026ff55d0 0x5590270042d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:50.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.477+0000 7f233a1fe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026fffe90 0x559027004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:50.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.477+0000 7f233a1fe740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f23301af4f0 con 0x559026ff4cb0 2024-09-10T08:39:50.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.477+0000 7f233a1fe740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f23301b0960 con 0x559026ff55d0 2024-09-10T08:39:50.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.477+0000 7f233a1fe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f23301afbc0 con 0x559026fffe90 2024-09-10T08:39:50.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.477+0000 7f233a1fe740 10 client.0 _ll_register_callbacks cb 0x7f23180057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:39:50.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.477+0000 7f2334e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026ff4cb0 0x559027003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:50.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.477+0000 7f2334e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026ff4cb0 0x559027003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59750/0 (socket says 172.21.15.89:59750) 2024-09-10T08:39:50.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.478+0000 7f2334e00640 1 -- 172.21.15.89:0/2921374308 learned_addr learned my addr 172.21.15.89:0/2921374308 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:50.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.478+0000 7f2327e00640 1 --2- 172.21.15.89:0/2921374308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026ff55d0 0x5590270042d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:50.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.478+0000 7f2335800640 1 --2- 172.21.15.89:0/2921374308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026fffe90 0x559027004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.478+0000 7f2334e00640 1 -- 172.21.15.89:0/2921374308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026fffe90 msgr2=0x559027004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.478+0000 7f2334e00640 1 --2- 172.21.15.89:0/2921374308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026fffe90 0x559027004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.478+0000 7f2334e00640 1 -- 172.21.15.89:0/2921374308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026ff55d0 msgr2=0x5590270042d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.478+0000 7f2334e00640 1 --2- 172.21.15.89:0/2921374308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026ff55d0 0x5590270042d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.478+0000 7f2334e00640 1 -- 172.21.15.89:0/2921374308 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559027079c20 con 0x559026ff4cb0 2024-09-10T08:39:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.478+0000 7f2327e00640 1 --2- 172.21.15.89:0/2921374308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026ff55d0 0x5590270042d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.478+0000 7f2334e00640 1 --2- 172.21.15.89:0/2921374308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026ff4cb0 0x559027003d90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2320031550 tx=0x7f2320000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4f48c0c1293d43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.478+0000 7f231ea00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:39:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.479+0000 7f231ea00640 1 -- 172.21.15.89:0/2921374308 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23200315b0 con 0x559026ff4cb0 2024-09-10T08:39:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.479+0000 7f231ea00640 1 -- 172.21.15.89:0/2921374308 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f2320031750 con 0x559026ff4cb0 2024-09-10T08:39:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.479+0000 7f233a1fe740 1 -- 172.21.15.89:0/2921374308 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23200043f0 con 0x559026ff4cb0 2024-09-10T08:39:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.479+0000 7f231ea00640 1 -- 172.21.15.89:0/2921374308 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2320031950 con 0x559026ff4cb0 2024-09-10T08:39:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.479+0000 7f233a1fe740 10 client.15204 fetch_fsmap learned FSMap version 26 2024-09-10T08:39:50.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.479+0000 7f233a1fe740 1 -- 172.21.15.89:0/2921374308 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55902707af20 con 0x559026ff4cb0 2024-09-10T08:39:50.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.479+0000 7f231ea00640 1 -- 172.21.15.89:0/2921374308 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23200043f0 con 0x559026ff4cb0 2024-09-10T08:39:50.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.479+0000 7f231ea00640 1 -- 172.21.15.89:0/2921374308 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2320050620 con 0x559026ff4cb0 2024-09-10T08:39:50.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.479+0000 7f233a1fe740 10 client.15204 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:39:50.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.479+0000 7f233a1fe740 10 client.15204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:39:50.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.479+0000 7f233a1fe740 20 client.15204 populate_metadata read hostname 'smithi089' 2024-09-10T08:39:50.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.480+0000 7f233a1fe740 1 --2- 172.21.15.89:0/2921374308 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55902707b1a0 0x55902707d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:50.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.480+0000 7f233a1fe740 4 client.15204 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:39:50.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.480+0000 7f233a1fe740 1 -- 172.21.15.89:0/2921374308 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559026f62d10 con 0x55902707b1a0 2024-09-10T08:39:50.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.480+0000 7f2327e00640 1 --2- 172.21.15.89:0/2921374308 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55902707b1a0 0x55902707d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:50.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.480+0000 7f2327e00640 1 --2- 172.21.15.89:0/2921374308 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55902707b1a0 0x55902707d590 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd3c3b8185e04dd3 server_cookie=c5c553b624e3aff6 in_seq=0 out_seq=0 2024-09-10T08:39:50.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.480+0000 7f231ea00640 10 client.15204 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:39:50.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.482+0000 7f231ea00640 1 -- 172.21.15.89:0/2921374308 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559026f62d10 con 0x55902707b1a0 2024-09-10T08:39:50.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.482+0000 7f231ea00640 10 client.15204 handle_command_reply: tid=0 2024-09-10T08:39:50.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.482+0000 7f233a1fe740 1 client.15204 shutdown 2024-09-10T08:39:50.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.482+0000 7f233a1fe740 1 -- 172.21.15.89:0/2921374308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026ff4cb0 msgr2=0x559027003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:50.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.482+0000 7f233a1fe740 1 --2- 172.21.15.89:0/2921374308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026ff4cb0 0x559027003d90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2320031550 tx=0x7f2320000bc0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.482+0000 7f233a1fe740 1 -- 172.21.15.89:0/2921374308 shutdown_connections 2024-09-10T08:39:50.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.482+0000 7f233a1fe740 1 --2- 172.21.15.89:0/2921374308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559026fffe90 0x559027004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.482+0000 7f233a1fe740 1 --2- 172.21.15.89:0/2921374308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559026ff55d0 0x5590270042d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.482+0000 7f233a1fe740 1 --2- 172.21.15.89:0/2921374308 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55902707b1a0 0x55902707d590 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.482+0000 7f233a1fe740 1 --2- 172.21.15.89:0/2921374308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559026ff4cb0 0x559027003d90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.482+0000 7f233a1fe740 1 -- 172.21.15.89:0/2921374308 >> 172.21.15.89:0/2921374308 conn(0x559026ff1240 msgr2=0x559026ff77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:50.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.482+0000 7f231ea00640 0 client.15204 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:39:50.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.482+0000 7f233a1fe740 1 -- 172.21.15.89:0/2921374308 shutdown_connections 2024-09-10T08:39:50.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.482+0000 7f233a1fe740 1 -- 172.21.15.89:0/2921374308 wait complete. 2024-09-10T08:39:50.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.482+0000 7f233a1fe740 20 client.15204 trim_cache size 0 max 16384 2024-09-10T08:39:50.485 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:39:50.485 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2024-09-10T08:39:50.485 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "d4cc1db6-0abb-4e16-a0c7-f28265fb2d0a", 2024-09-10T08:39:50.485 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2024-09-10T08:39:50.485 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:39:50.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.483+0000 7f2336200640 1 -- 172.21.15.89:0/1806268067 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23140a2c10 msgr2=0x7f23140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:50.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.483+0000 7f2336200640 1 --2- 172.21.15.89:0/1806268067 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23140a2c10 0x7f23140a50d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f23280049d0 tx=0x7f2328009500 comp rx=0 tx=0).stop 2024-09-10T08:39:50.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.483+0000 7f2336200640 1 -- 172.21.15.89:0/1806268067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23301047e0 msgr2=0x7f233019a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:50.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.483+0000 7f2336200640 1 --2- 172.21.15.89:0/1806268067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23301047e0 0x7f233019a9b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f231800d6d0 tx=0x7f231800db90 comp rx=0 tx=0).stop 2024-09-10T08:39:50.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.483+0000 7f2336200640 1 -- 172.21.15.89:0/1806268067 shutdown_connections 2024-09-10T08:39:50.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.483+0000 7f2336200640 1 --2- 172.21.15.89:0/1806268067 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23140a2c10 0x7f23140a50d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.483+0000 7f2336200640 1 --2- 172.21.15.89:0/1806268067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2330108f60 0x7f233019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.483+0000 7f2336200640 1 --2- 172.21.15.89:0/1806268067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23301047e0 0x7f233019a9b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.483+0000 7f2336200640 1 --2- 172.21.15.89:0/1806268067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2330103e30 0x7f233019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:50.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.483+0000 7f2336200640 1 -- 172.21.15.89:0/1806268067 >> 172.21.15.89:0/1806268067 conn(0x7f23300ff8d0 msgr2=0x7f2330100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:50.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.483+0000 7f2336200640 1 -- 172.21.15.89:0/1806268067 shutdown_connections 2024-09-10T08:39:50.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:50.483+0000 7f2336200640 1 -- 172.21.15.89:0/1806268067 wait complete. 2024-09-10T08:39:51.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:50 smithi111 ceph-mon[35220]: pgmap v390: 129 pgs: 129 active+clean; 649 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-10T08:39:51.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:50 smithi089 ceph-mon[30757]: pgmap v390: 129 pgs: 129 active+clean; 649 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-10T08:39:51.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:50 smithi149 ceph-mon[36167]: pgmap v390: 129 pgs: 129 active+clean; 649 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-10T08:39:51.572 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:39:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.747+0000 7faf5c400640 1 -- 172.21.15.89:0/1487508973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf54109120 msgr2=0x7faf54110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.747+0000 7faf5c400640 1 --2- 172.21.15.89:0/1487508973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf54109120 0x7faf54110ca0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faf4c009ca0 tx=0x7faf4c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:39:51.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.748+0000 7faf5c400640 1 -- 172.21.15.89:0/1487508973 shutdown_connections 2024-09-10T08:39:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.748+0000 7faf5c400640 1 --2- 172.21.15.89:0/1487508973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf54109120 0x7faf54110ca0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.748+0000 7faf5c400640 1 --2- 172.21.15.89:0/1487508973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf541049a0 0x7faf541089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.748+0000 7faf5c400640 1 --2- 172.21.15.89:0/1487508973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf54103ff0 0x7faf541043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.748+0000 7faf5c400640 1 -- 172.21.15.89:0/1487508973 >> 172.21.15.89:0/1487508973 conn(0x7faf540ffa90 msgr2=0x7faf54101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.748+0000 7faf5c400640 1 -- 172.21.15.89:0/1487508973 shutdown_connections 2024-09-10T08:39:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.748+0000 7faf5c400640 1 -- 172.21.15.89:0/1487508973 wait complete. 2024-09-10T08:39:51.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.749+0000 7faf5c400640 1 Processor -- start 2024-09-10T08:39:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.749+0000 7faf5c400640 1 -- start start 2024-09-10T08:39:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.749+0000 7faf5c400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf54103ff0 0x7faf5419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.749+0000 7faf5c400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf541049a0 0x7faf5419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.749+0000 7faf5c400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf54109120 0x7faf5419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.749+0000 7faf5c400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7faf54114180 con 0x7faf541049a0 2024-09-10T08:39:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.749+0000 7faf5c400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faf54114000 con 0x7faf54103ff0 2024-09-10T08:39:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.749+0000 7faf5c400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faf54114300 con 0x7faf54109120 2024-09-10T08:39:51.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.750+0000 7faf59000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf541049a0 0x7faf5419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.750+0000 7faf59000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf541049a0 0x7faf5419aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59780/0 (socket says 172.21.15.89:59780) 2024-09-10T08:39:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.750+0000 7faf59000640 1 -- 172.21.15.89:0/645227232 learned_addr learned my addr 172.21.15.89:0/645227232 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.750+0000 7faf59a00640 1 --2- 172.21.15.89:0/645227232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf54103ff0 0x7faf5419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.750+0000 7faf5a400640 1 --2- 172.21.15.89:0/645227232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf54109120 0x7faf5419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.750+0000 7faf59a00640 1 -- 172.21.15.89:0/645227232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf54109120 msgr2=0x7faf5419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.750+0000 7faf59a00640 1 --2- 172.21.15.89:0/645227232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf54109120 0x7faf5419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.750+0000 7faf59a00640 1 -- 172.21.15.89:0/645227232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf541049a0 msgr2=0x7faf5419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.750+0000 7faf59a00640 1 --2- 172.21.15.89:0/645227232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf541049a0 0x7faf5419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.750+0000 7faf59a00640 1 -- 172.21.15.89:0/645227232 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5419f550 con 0x7faf54103ff0 2024-09-10T08:39:51.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.750+0000 7faf59000640 1 --2- 172.21.15.89:0/645227232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf541049a0 0x7faf5419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.751+0000 7faf59a00640 1 --2- 172.21.15.89:0/645227232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf54103ff0 0x7faf5419a5a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faf4800e9f0 tx=0x7faf4800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.751+0000 7faf52000640 1 -- 172.21.15.89:0/645227232 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4800ccc0 con 0x7faf54103ff0 2024-09-10T08:39:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.751+0000 7faf5c400640 1 -- 172.21.15.89:0/645227232 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf5419f840 con 0x7faf54103ff0 2024-09-10T08:39:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.751+0000 7faf52000640 1 -- 172.21.15.89:0/645227232 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf48010910 con 0x7faf54103ff0 2024-09-10T08:39:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.751+0000 7faf52000640 1 -- 172.21.15.89:0/645227232 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48018960 con 0x7faf54103ff0 2024-09-10T08:39:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.751+0000 7faf5c400640 1 -- 172.21.15.89:0/645227232 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf541a7120 con 0x7faf54103ff0 2024-09-10T08:39:51.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.752+0000 7faf5c400640 1 -- 172.21.15.89:0/645227232 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf541043d0 con 0x7faf54103ff0 2024-09-10T08:39:51.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.754+0000 7faf52000640 1 -- 172.21.15.89:0/645227232 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7faf48010ac0 con 0x7faf54103ff0 2024-09-10T08:39:51.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.755+0000 7faf52000640 1 --2- 172.21.15.89:0/645227232 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7faf3c0a2c10 0x7faf3c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:51.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.756+0000 7faf52000640 1 -- 172.21.15.89:0/645227232 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7faf480cfe50 con 0x7faf54103ff0 2024-09-10T08:39:51.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.756+0000 7faf59000640 1 --2- 172.21.15.89:0/645227232 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7faf3c0a2c10 0x7faf3c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:51.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.756+0000 7faf59000640 1 --2- 172.21.15.89:0/645227232 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7faf3c0a2c10 0x7faf3c0a50d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faf5410aaf0 tx=0x7faf40005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:51.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.758+0000 7faf52000640 1 -- 172.21.15.89:0/645227232 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf4809c1d0 con 0x7faf54103ff0 2024-09-10T08:39:51.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:51 smithi089 ceph-mon[30757]: scrub queued for path: / 2024-09-10T08:39:51.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:51 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:39:51.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:51 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:39:51.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:51 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:39:51.971 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:51.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.973+0000 7faf5c400640 1 -- 172.21.15.89:0/645227232 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faf54100cc0 con 0x7faf3c0a2c10 2024-09-10T08:39:51.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.982+0000 7faf52000640 1 -- 172.21.15.89:0/645227232 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340807 (secure 0 0 0) 0x7faf54100cc0 con 0x7faf3c0a2c10 2024-09-10T08:39:51.987 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:39:51.989 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-09-10T08:39:51.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.986+0000 7faf5c400640 1 -- 172.21.15.89:0/645227232 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7faf3c0a2c10 msgr2=0x7faf3c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:51.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.986+0000 7faf5c400640 1 --2- 172.21.15.89:0/645227232 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7faf3c0a2c10 0x7faf3c0a50d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faf5410aaf0 tx=0x7faf40005f90 comp rx=0 tx=0).stop 2024-09-10T08:39:51.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.987+0000 7faf5c400640 1 -- 172.21.15.89:0/645227232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf54103ff0 msgr2=0x7faf5419a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:51.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.987+0000 7faf5c400640 1 --2- 172.21.15.89:0/645227232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf54103ff0 0x7faf5419a5a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faf4800e9f0 tx=0x7faf4800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:39:51.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.987+0000 7faf5c400640 1 -- 172.21.15.89:0/645227232 shutdown_connections 2024-09-10T08:39:51.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.987+0000 7faf5c400640 1 --2- 172.21.15.89:0/645227232 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7faf3c0a2c10 0x7faf3c0a50d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:51.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.987+0000 7faf5c400640 1 --2- 172.21.15.89:0/645227232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf54109120 0x7faf5419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:51.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.987+0000 7faf5c400640 1 --2- 172.21.15.89:0/645227232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf541049a0 0x7faf5419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:51.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.987+0000 7faf5c400640 1 --2- 172.21.15.89:0/645227232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf54103ff0 0x7faf5419a5a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:51.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.987+0000 7faf5c400640 1 -- 172.21.15.89:0/645227232 >> 172.21.15.89:0/645227232 conn(0x7faf540ffa90 msgr2=0x7faf541005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:51.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.987+0000 7faf5c400640 1 -- 172.21.15.89:0/645227232 shutdown_connections 2024-09-10T08:39:51.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:51.987+0000 7faf5c400640 1 -- 172.21.15.89:0/645227232 wait complete. 2024-09-10T08:39:52.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:51 smithi111 ceph-mon[35220]: scrub queued for path: / 2024-09-10T08:39:52.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:51 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:39:52.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:51 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:39:52.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:51 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:39:52.122 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:51 smithi149 ceph-mon[36167]: scrub queued for path: / 2024-09-10T08:39:52.122 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:51 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:39:52.122 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:51 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:39:52.122 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:51 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:39:52.844 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-10T08:39:51.614774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667141,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":890,"num_read_kb":572,"num_write":371,"num_write_kb":3080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181900,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":799,"ondisk_log_size":799,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357984,"kb_used_data":12896,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583840,"statfs":{"total":1151940427776,"available":1151573852160,"internally_reserved":0,"allocated":13205504,"data_stored":8545265,"data_compressed":467733,"data_compressed_allocated":638976,"data_compressed_original":2777088,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8635,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":97,"num_read_kb":58,"num_write":17,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003096"},"pg_stats":[{"pgid":"2.1f","version":"94'9","reported_seq":68,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:00.658898+0000","last_change":"2024-09-10T08:38:43.663079+0000","last_active":"2024-09-10T08:39:00.658898+0000","last_peered":"2024-09-10T08:39:00.658898+0000","last_clean":"2024-09-10T08:39:00.658898+0000","last_became_active":"2024-09-10T08:38:43.656506+0000","last_became_peered":"2024-09-10T08:38:43.656506+0000","last_unstale":"2024-09-10T08:39:00.658898+0000","last_undegraded":"2024-09-10T08:39:00.658898+0000","last_fullsized":"2024-09-10T08:39:00.658898+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:36.881564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.579344+0000","last_change":"2024-09-10T08:36:56.145816+0000","last_active":"2024-09-10T08:39:11.579344+0000","last_peered":"2024-09-10T08:39:11.579344+0000","last_clean":"2024-09-10T08:39:11.579344+0000","last_became_active":"2024-09-10T08:36:56.145462+0000","last_became_peered":"2024-09-10T08:36:56.145462+0000","last_unstale":"2024-09-10T08:39:11.579344+0000","last_undegraded":"2024-09-10T08:39:11.579344+0000","last_fullsized":"2024-09-10T08:39:11.579344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:53.890190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629879+0000","last_change":"2024-09-10T08:37:05.252761+0000","last_active":"2024-09-10T08:38:43.629879+0000","last_peered":"2024-09-10T08:38:43.629879+0000","last_clean":"2024-09-10T08:38:43.629879+0000","last_became_active":"2024-09-10T08:37:05.252179+0000","last_became_peered":"2024-09-10T08:37:05.252179+0000","last_unstale":"2024-09-10T08:38:43.629879+0000","last_undegraded":"2024-09-10T08:38:43.629879+0000","last_fullsized":"2024-09-10T08:38:43.629879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:52.854828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631850+0000","last_change":"2024-09-10T08:36:53.735650+0000","last_active":"2024-09-10T08:38:43.631850+0000","last_peered":"2024-09-10T08:38:43.631850+0000","last_clean":"2024-09-10T08:38:43.631850+0000","last_became_active":"2024-09-10T08:36:53.735128+0000","last_became_peered":"2024-09-10T08:36:53.735128+0000","last_unstale":"2024-09-10T08:38:43.631850+0000","last_undegraded":"2024-09-10T08:38:43.631850+0000","last_fullsized":"2024-09-10T08:38:43.631850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:27.309591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592923+0000","last_change":"2024-09-10T08:36:56.153364+0000","last_active":"2024-09-10T08:39:11.592923+0000","last_peered":"2024-09-10T08:39:11.592923+0000","last_clean":"2024-09-10T08:39:11.592923+0000","last_became_active":"2024-09-10T08:36:56.152979+0000","last_became_peered":"2024-09-10T08:36:56.152979+0000","last_unstale":"2024-09-10T08:39:11.592923+0000","last_undegraded":"2024-09-10T08:39:11.592923+0000","last_fullsized":"2024-09-10T08:39:11.592923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:12.172620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654774+0000","last_change":"2024-09-10T08:37:05.254551+0000","last_active":"2024-09-10T08:38:43.654774+0000","last_peered":"2024-09-10T08:38:43.654774+0000","last_clean":"2024-09-10T08:38:43.654774+0000","last_became_active":"2024-09-10T08:37:05.254055+0000","last_became_peered":"2024-09-10T08:37:05.254055+0000","last_unstale":"2024-09-10T08:38:43.654774+0000","last_undegraded":"2024-09-10T08:38:43.654774+0000","last_fullsized":"2024-09-10T08:38:43.654774+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:08.472502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626522+0000","last_change":"2024-09-10T08:36:53.722774+0000","last_active":"2024-09-10T08:38:43.626522+0000","last_peered":"2024-09-10T08:38:43.626522+0000","last_clean":"2024-09-10T08:38:43.626522+0000","last_became_active":"2024-09-10T08:36:53.722459+0000","last_became_peered":"2024-09-10T08:36:53.722459+0000","last_unstale":"2024-09-10T08:38:43.626522+0000","last_undegraded":"2024-09-10T08:38:43.626522+0000","last_fullsized":"2024-09-10T08:38:43.626522+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:26.612354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592240+0000","last_change":"2024-09-10T08:36:56.143757+0000","last_active":"2024-09-10T08:39:11.592240+0000","last_peered":"2024-09-10T08:39:11.592240+0000","last_clean":"2024-09-10T08:39:11.592240+0000","last_became_active":"2024-09-10T08:36:56.143430+0000","last_became_peered":"2024-09-10T08:36:56.143430+0000","last_unstale":"2024-09-10T08:39:11.592240+0000","last_undegraded":"2024-09-10T08:39:11.592240+0000","last_fullsized":"2024-09-10T08:39:11.592240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:49.737298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655120+0000","last_change":"2024-09-10T08:37:05.254503+0000","last_active":"2024-09-10T08:38:43.655120+0000","last_peered":"2024-09-10T08:38:43.655120+0000","last_clean":"2024-09-10T08:38:43.655120+0000","last_became_active":"2024-09-10T08:37:05.254070+0000","last_became_peered":"2024-09-10T08:37:05.254070+0000","last_unstale":"2024-09-10T08:38:43.655120+0000","last_undegraded":"2024-09-10T08:38:43.655120+0000","last_fullsized":"2024-09-10T08:38:43.655120+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:16.741698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626438+0000","last_change":"2024-09-10T08:36:53.716742+0000","last_active":"2024-09-10T08:38:43.626438+0000","last_peered":"2024-09-10T08:38:43.626438+0000","last_clean":"2024-09-10T08:38:43.626438+0000","last_became_active":"2024-09-10T08:36:53.716439+0000","last_became_peered":"2024-09-10T08:36:53.716439+0000","last_unstale":"2024-09-10T08:38:43.626438+0000","last_undegraded":"2024-09-10T08:38:43.626438+0000","last_fullsized":"2024-09-10T08:38:43.626438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:46.882805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.581828+0000","last_change":"2024-09-10T08:36:56.146841+0000","last_active":"2024-09-10T08:39:11.581828+0000","last_peered":"2024-09-10T08:39:11.581828+0000","last_clean":"2024-09-10T08:39:11.581828+0000","last_became_active":"2024-09-10T08:36:56.146531+0000","last_became_peered":"2024-09-10T08:36:56.146531+0000","last_unstale":"2024-09-10T08:39:11.581828+0000","last_undegraded":"2024-09-10T08:39:11.581828+0000","last_fullsized":"2024-09-10T08:39:11.581828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:37.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654877+0000","last_change":"2024-09-10T08:37:43.223181+0000","last_active":"2024-09-10T08:38:43.654877+0000","last_peered":"2024-09-10T08:38:43.654877+0000","last_clean":"2024-09-10T08:38:43.654877+0000","last_became_active":"2024-09-10T08:37:43.222794+0000","last_became_peered":"2024-09-10T08:37:43.222794+0000","last_unstale":"2024-09-10T08:38:43.654877+0000","last_undegraded":"2024-09-10T08:38:43.654877+0000","last_fullsized":"2024-09-10T08:38:43.654877+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:22.204517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593593+0000","last_change":"2024-09-10T08:36:56.142161+0000","last_active":"2024-09-10T08:39:11.593593+0000","last_peered":"2024-09-10T08:39:11.593593+0000","last_clean":"2024-09-10T08:39:11.593593+0000","last_became_active":"2024-09-10T08:36:56.141838+0000","last_became_peered":"2024-09-10T08:36:56.141838+0000","last_unstale":"2024-09-10T08:39:11.593593+0000","last_undegraded":"2024-09-10T08:39:11.593593+0000","last_fullsized":"2024-09-10T08:39:11.593593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:09.909747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626125+0000","last_change":"2024-09-10T08:36:53.722238+0000","last_active":"2024-09-10T08:38:43.626125+0000","last_peered":"2024-09-10T08:38:43.626125+0000","last_clean":"2024-09-10T08:38:43.626125+0000","last_became_active":"2024-09-10T08:36:53.721859+0000","last_became_peered":"2024-09-10T08:36:53.721859+0000","last_unstale":"2024-09-10T08:38:43.626125+0000","last_undegraded":"2024-09-10T08:38:43.626125+0000","last_fullsized":"2024-09-10T08:38:43.626125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:50.615090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655183+0000","last_change":"2024-09-10T08:37:05.414041+0000","last_active":"2024-09-10T08:38:43.655183+0000","last_peered":"2024-09-10T08:38:43.655183+0000","last_clean":"2024-09-10T08:38:43.655183+0000","last_became_active":"2024-09-10T08:37:05.413286+0000","last_became_peered":"2024-09-10T08:37:05.413286+0000","last_unstale":"2024-09-10T08:38:43.655183+0000","last_undegraded":"2024-09-10T08:38:43.655183+0000","last_fullsized":"2024-09-10T08:38:43.655183+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:44.916102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.572936+0000","last_change":"2024-09-10T08:36:56.158278+0000","last_active":"2024-09-10T08:39:11.572936+0000","last_peered":"2024-09-10T08:39:11.572936+0000","last_clean":"2024-09-10T08:39:11.572936+0000","last_became_active":"2024-09-10T08:36:56.157946+0000","last_became_peered":"2024-09-10T08:36:56.157946+0000","last_unstale":"2024-09-10T08:39:11.572936+0000","last_undegraded":"2024-09-10T08:39:11.572936+0000","last_fullsized":"2024-09-10T08:39:11.572936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:42.512513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"96'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:38.713339+0000","last_change":"2024-09-10T08:38:43.635077+0000","last_active":"2024-09-10T08:39:38.713339+0000","last_peered":"2024-09-10T08:39:38.713339+0000","last_clean":"2024-09-10T08:39:38.713339+0000","last_became_active":"2024-09-10T08:38:43.631957+0000","last_became_peered":"2024-09-10T08:38:43.631957+0000","last_unstale":"2024-09-10T08:39:38.713339+0000","last_undegraded":"2024-09-10T08:39:38.713339+0000","last_fullsized":"2024-09-10T08:39:38.713339+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:50.837247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655126+0000","last_change":"2024-09-10T08:37:42.671399+0000","last_active":"2024-09-10T08:38:43.655126+0000","last_peered":"2024-09-10T08:38:43.655126+0000","last_clean":"2024-09-10T08:38:43.655126+0000","last_became_active":"2024-09-10T08:37:42.670865+0000","last_became_peered":"2024-09-10T08:37:42.670865+0000","last_unstale":"2024-09-10T08:38:43.655126+0000","last_undegraded":"2024-09-10T08:38:43.655126+0000","last_fullsized":"2024-09-10T08:38:43.655126+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:16.231538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.577960+0000","last_change":"2024-09-10T08:36:56.144856+0000","last_active":"2024-09-10T08:39:11.577960+0000","last_peered":"2024-09-10T08:39:11.577960+0000","last_clean":"2024-09-10T08:39:11.577960+0000","last_became_active":"2024-09-10T08:36:56.144531+0000","last_became_peered":"2024-09-10T08:36:56.144531+0000","last_unstale":"2024-09-10T08:39:11.577960+0000","last_undegraded":"2024-09-10T08:39:11.577960+0000","last_fullsized":"2024-09-10T08:39:11.577960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:24.586006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631541+0000","last_change":"2024-09-10T08:36:53.721093+0000","last_active":"2024-09-10T08:38:43.631541+0000","last_peered":"2024-09-10T08:38:43.631541+0000","last_clean":"2024-09-10T08:38:43.631541+0000","last_became_active":"2024-09-10T08:36:53.720538+0000","last_became_peered":"2024-09-10T08:36:53.720538+0000","last_unstale":"2024-09-10T08:38:43.631541+0000","last_undegraded":"2024-09-10T08:38:43.631541+0000","last_fullsized":"2024-09-10T08:38:43.631541+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:58.492463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629902+0000","last_change":"2024-09-10T08:37:05.254011+0000","last_active":"2024-09-10T08:38:43.629902+0000","last_peered":"2024-09-10T08:38:43.629902+0000","last_clean":"2024-09-10T08:38:43.629902+0000","last_became_active":"2024-09-10T08:37:05.253066+0000","last_became_peered":"2024-09-10T08:37:05.253066+0000","last_unstale":"2024-09-10T08:38:43.629902+0000","last_undegraded":"2024-09-10T08:38:43.629902+0000","last_fullsized":"2024-09-10T08:38:43.629902+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:16.221302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.553299+0000","last_change":"2024-09-10T08:36:56.141814+0000","last_active":"2024-09-10T08:39:11.553299+0000","last_peered":"2024-09-10T08:39:11.553299+0000","last_clean":"2024-09-10T08:39:11.553299+0000","last_became_active":"2024-09-10T08:36:56.141480+0000","last_became_peered":"2024-09-10T08:36:56.141480+0000","last_unstale":"2024-09-10T08:39:11.553299+0000","last_undegraded":"2024-09-10T08:39:11.553299+0000","last_fullsized":"2024-09-10T08:39:11.553299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:40.594354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.636347+0000","last_change":"2024-09-10T08:38:43.635917+0000","last_active":"2024-09-10T08:38:43.636347+0000","last_peered":"2024-09-10T08:38:43.636347+0000","last_clean":"2024-09-10T08:38:43.636347+0000","last_became_active":"2024-09-10T08:38:43.633406+0000","last_became_peered":"2024-09-10T08:38:43.633406+0000","last_unstale":"2024-09-10T08:38:43.636347+0000","last_undegraded":"2024-09-10T08:38:43.636347+0000","last_fullsized":"2024-09-10T08:38:43.636347+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:44.619072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655297+0000","last_change":"2024-09-10T08:37:05.414181+0000","last_active":"2024-09-10T08:38:43.655297+0000","last_peered":"2024-09-10T08:38:43.655297+0000","last_clean":"2024-09-10T08:38:43.655297+0000","last_became_active":"2024-09-10T08:37:05.413691+0000","last_became_peered":"2024-09-10T08:37:05.413691+0000","last_unstale":"2024-09-10T08:38:43.655297+0000","last_undegraded":"2024-09-10T08:38:43.655297+0000","last_fullsized":"2024-09-10T08:38:43.655297+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:30.744949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584567+0000","last_change":"2024-09-10T08:36:56.153956+0000","last_active":"2024-09-10T08:39:11.584567+0000","last_peered":"2024-09-10T08:39:11.584567+0000","last_clean":"2024-09-10T08:39:11.584567+0000","last_became_active":"2024-09-10T08:36:56.153609+0000","last_became_peered":"2024-09-10T08:36:56.153609+0000","last_unstale":"2024-09-10T08:39:11.584567+0000","last_undegraded":"2024-09-10T08:39:11.584567+0000","last_fullsized":"2024-09-10T08:39:11.584567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:18.185704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655520+0000","last_change":"2024-09-10T08:36:53.733217+0000","last_active":"2024-09-10T08:38:43.655520+0000","last_peered":"2024-09-10T08:38:43.655520+0000","last_clean":"2024-09-10T08:38:43.655520+0000","last_became_active":"2024-09-10T08:36:53.732851+0000","last_became_peered":"2024-09-10T08:36:53.732851+0000","last_unstale":"2024-09-10T08:38:43.655520+0000","last_undegraded":"2024-09-10T08:38:43.655520+0000","last_fullsized":"2024-09-10T08:38:43.655520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:04:01.132196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629889+0000","last_change":"2024-09-10T08:37:05.241868+0000","last_active":"2024-09-10T08:38:43.629889+0000","last_peered":"2024-09-10T08:38:43.629889+0000","last_clean":"2024-09-10T08:38:43.629889+0000","last_became_active":"2024-09-10T08:37:05.241412+0000","last_became_peered":"2024-09-10T08:37:05.241412+0000","last_unstale":"2024-09-10T08:38:43.629889+0000","last_undegraded":"2024-09-10T08:38:43.629889+0000","last_fullsized":"2024-09-10T08:38:43.629889+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:52.543353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.575701+0000","last_change":"2024-09-10T08:36:56.150722+0000","last_active":"2024-09-10T08:39:11.575701+0000","last_peered":"2024-09-10T08:39:11.575701+0000","last_clean":"2024-09-10T08:39:11.575701+0000","last_became_active":"2024-09-10T08:36:56.150423+0000","last_became_peered":"2024-09-10T08:36:56.150423+0000","last_unstale":"2024-09-10T08:39:11.575701+0000","last_undegraded":"2024-09-10T08:39:11.575701+0000","last_fullsized":"2024-09-10T08:39:11.575701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:50.272841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'13","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:38.175435+0000","last_change":"2024-09-10T08:36:53.722477+0000","last_active":"2024-09-10T08:39:38.175435+0000","last_peered":"2024-09-10T08:39:38.175435+0000","last_clean":"2024-09-10T08:39:38.175435+0000","last_became_active":"2024-09-10T08:36:53.722118+0000","last_became_peered":"2024-09-10T08:36:53.722118+0000","last_unstale":"2024-09-10T08:39:38.175435+0000","last_undegraded":"2024-09-10T08:39:38.175435+0000","last_fullsized":"2024-09-10T08:39:38.175435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:06.631646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630992+0000","last_change":"2024-09-10T08:37:05.415656+0000","last_active":"2024-09-10T08:38:43.630992+0000","last_peered":"2024-09-10T08:38:43.630992+0000","last_clean":"2024-09-10T08:38:43.630992+0000","last_became_active":"2024-09-10T08:37:05.413537+0000","last_became_peered":"2024-09-10T08:37:05.413537+0000","last_unstale":"2024-09-10T08:38:43.630992+0000","last_undegraded":"2024-09-10T08:38:43.630992+0000","last_fullsized":"2024-09-10T08:38:43.630992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:04.284410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585069+0000","last_change":"2024-09-10T08:36:56.144157+0000","last_active":"2024-09-10T08:39:11.585069+0000","last_peered":"2024-09-10T08:39:11.585069+0000","last_clean":"2024-09-10T08:39:11.585069+0000","last_became_active":"2024-09-10T08:36:56.140332+0000","last_became_peered":"2024-09-10T08:36:56.140332+0000","last_unstale":"2024-09-10T08:39:11.585069+0000","last_undegraded":"2024-09-10T08:39:11.585069+0000","last_fullsized":"2024-09-10T08:39:11.585069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:09:26.097987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626517+0000","last_change":"2024-09-10T08:36:53.735465+0000","last_active":"2024-09-10T08:38:43.626517+0000","last_peered":"2024-09-10T08:38:43.626517+0000","last_clean":"2024-09-10T08:38:43.626517+0000","last_became_active":"2024-09-10T08:36:53.735160+0000","last_became_peered":"2024-09-10T08:36:53.735160+0000","last_unstale":"2024-09-10T08:38:43.626517+0000","last_undegraded":"2024-09-10T08:38:43.626517+0000","last_fullsized":"2024-09-10T08:38:43.626517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:26:52.332085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627959+0000","last_change":"2024-09-10T08:37:05.247671+0000","last_active":"2024-09-10T08:38:43.627959+0000","last_peered":"2024-09-10T08:38:43.627959+0000","last_clean":"2024-09-10T08:38:43.627959+0000","last_became_active":"2024-09-10T08:37:05.247217+0000","last_became_peered":"2024-09-10T08:37:05.247217+0000","last_unstale":"2024-09-10T08:38:43.627959+0000","last_undegraded":"2024-09-10T08:38:43.627959+0000","last_fullsized":"2024-09-10T08:38:43.627959+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:20.881304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.579998+0000","last_change":"2024-09-10T08:36:56.145420+0000","last_active":"2024-09-10T08:39:11.579998+0000","last_peered":"2024-09-10T08:39:11.579998+0000","last_clean":"2024-09-10T08:39:11.579998+0000","last_became_active":"2024-09-10T08:36:56.145074+0000","last_became_peered":"2024-09-10T08:36:56.145074+0000","last_unstale":"2024-09-10T08:39:11.579998+0000","last_undegraded":"2024-09-10T08:39:11.579998+0000","last_fullsized":"2024-09-10T08:39:11.579998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:13.849977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629812+0000","last_change":"2024-09-10T08:36:53.738237+0000","last_active":"2024-09-10T08:38:43.629812+0000","last_peered":"2024-09-10T08:38:43.629812+0000","last_clean":"2024-09-10T08:38:43.629812+0000","last_became_active":"2024-09-10T08:36:53.737858+0000","last_became_peered":"2024-09-10T08:36:53.737858+0000","last_unstale":"2024-09-10T08:38:43.629812+0000","last_undegraded":"2024-09-10T08:38:43.629812+0000","last_fullsized":"2024-09-10T08:38:43.629812+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:35.345661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.625982+0000","last_change":"2024-09-10T08:37:05.413512+0000","last_active":"2024-09-10T08:38:43.625982+0000","last_peered":"2024-09-10T08:38:43.625982+0000","last_clean":"2024-09-10T08:38:43.625982+0000","last_became_active":"2024-09-10T08:37:05.413010+0000","last_became_peered":"2024-09-10T08:37:05.413010+0000","last_unstale":"2024-09-10T08:38:43.625982+0000","last_undegraded":"2024-09-10T08:38:43.625982+0000","last_fullsized":"2024-09-10T08:38:43.625982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:05.349404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583802+0000","last_change":"2024-09-10T08:36:56.157741+0000","last_active":"2024-09-10T08:39:11.583802+0000","last_peered":"2024-09-10T08:39:11.583802+0000","last_clean":"2024-09-10T08:39:11.583802+0000","last_became_active":"2024-09-10T08:36:56.157295+0000","last_became_peered":"2024-09-10T08:36:56.157295+0000","last_unstale":"2024-09-10T08:39:11.583802+0000","last_undegraded":"2024-09-10T08:39:11.583802+0000","last_fullsized":"2024-09-10T08:39:11.583802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:31.845864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631834+0000","last_change":"2024-09-10T08:36:53.722373+0000","last_active":"2024-09-10T08:38:43.631834+0000","last_peered":"2024-09-10T08:38:43.631834+0000","last_clean":"2024-09-10T08:38:43.631834+0000","last_became_active":"2024-09-10T08:36:53.721975+0000","last_became_peered":"2024-09-10T08:36:53.721975+0000","last_unstale":"2024-09-10T08:38:43.631834+0000","last_undegraded":"2024-09-10T08:38:43.631834+0000","last_fullsized":"2024-09-10T08:38:43.631834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:44.593018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628351+0000","last_change":"2024-09-10T08:37:05.255362+0000","last_active":"2024-09-10T08:38:43.628351+0000","last_peered":"2024-09-10T08:38:43.628351+0000","last_clean":"2024-09-10T08:38:43.628351+0000","last_became_active":"2024-09-10T08:37:05.254967+0000","last_became_peered":"2024-09-10T08:37:05.254967+0000","last_unstale":"2024-09-10T08:38:43.628351+0000","last_undegraded":"2024-09-10T08:38:43.628351+0000","last_fullsized":"2024-09-10T08:38:43.628351+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:31:41.850961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.578715+0000","last_change":"2024-09-10T08:36:56.147699+0000","last_active":"2024-09-10T08:39:11.578715+0000","last_peered":"2024-09-10T08:39:11.578715+0000","last_clean":"2024-09-10T08:39:11.578715+0000","last_became_active":"2024-09-10T08:36:56.147349+0000","last_became_peered":"2024-09-10T08:36:56.147349+0000","last_unstale":"2024-09-10T08:39:11.578715+0000","last_undegraded":"2024-09-10T08:39:11.578715+0000","last_fullsized":"2024-09-10T08:39:11.578715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:25.248588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"96'6","reported_seq":169,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:48.903998+0000","last_change":"2024-09-10T08:36:53.736843+0000","last_active":"2024-09-10T08:39:48.903998+0000","last_peered":"2024-09-10T08:39:48.903998+0000","last_clean":"2024-09-10T08:39:48.903998+0000","last_became_active":"2024-09-10T08:36:53.736340+0000","last_became_peered":"2024-09-10T08:36:53.736340+0000","last_unstale":"2024-09-10T08:39:48.903998+0000","last_undegraded":"2024-09-10T08:39:48.903998+0000","last_fullsized":"2024-09-10T08:39:48.903998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:24.679679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628575+0000","last_change":"2024-09-10T08:37:05.414321+0000","last_active":"2024-09-10T08:38:43.628575+0000","last_peered":"2024-09-10T08:38:43.628575+0000","last_clean":"2024-09-10T08:38:43.628575+0000","last_became_active":"2024-09-10T08:37:05.413868+0000","last_became_peered":"2024-09-10T08:37:05.413868+0000","last_unstale":"2024-09-10T08:38:43.628575+0000","last_undegraded":"2024-09-10T08:38:43.628575+0000","last_fullsized":"2024-09-10T08:38:43.628575+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:53.988077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583174+0000","last_change":"2024-09-10T08:36:56.141910+0000","last_active":"2024-09-10T08:39:11.583174+0000","last_peered":"2024-09-10T08:39:11.583174+0000","last_clean":"2024-09-10T08:39:11.583174+0000","last_became_active":"2024-09-10T08:36:56.141579+0000","last_became_peered":"2024-09-10T08:36:56.141579+0000","last_unstale":"2024-09-10T08:39:11.583174+0000","last_undegraded":"2024-09-10T08:39:11.583174+0000","last_fullsized":"2024-09-10T08:39:11.583174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:18.464831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631756+0000","last_change":"2024-09-10T08:36:53.720102+0000","last_active":"2024-09-10T08:38:43.631756+0000","last_peered":"2024-09-10T08:38:43.631756+0000","last_clean":"2024-09-10T08:38:43.631756+0000","last_became_active":"2024-09-10T08:36:53.719764+0000","last_became_peered":"2024-09-10T08:36:53.719764+0000","last_unstale":"2024-09-10T08:38:43.631756+0000","last_undegraded":"2024-09-10T08:38:43.631756+0000","last_fullsized":"2024-09-10T08:38:43.631756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:03.609338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655205+0000","last_change":"2024-09-10T08:37:05.413818+0000","last_active":"2024-09-10T08:38:43.655205+0000","last_peered":"2024-09-10T08:38:43.655205+0000","last_clean":"2024-09-10T08:38:43.655205+0000","last_became_active":"2024-09-10T08:37:05.413267+0000","last_became_peered":"2024-09-10T08:37:05.413267+0000","last_unstale":"2024-09-10T08:38:43.655205+0000","last_undegraded":"2024-09-10T08:38:43.655205+0000","last_fullsized":"2024-09-10T08:38:43.655205+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:54.309686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.590486+0000","last_change":"2024-09-10T08:36:56.149286+0000","last_active":"2024-09-10T08:39:11.590486+0000","last_peered":"2024-09-10T08:39:11.590486+0000","last_clean":"2024-09-10T08:39:11.590486+0000","last_became_active":"2024-09-10T08:36:56.146275+0000","last_became_peered":"2024-09-10T08:36:56.146275+0000","last_unstale":"2024-09-10T08:39:11.590486+0000","last_undegraded":"2024-09-10T08:39:11.590486+0000","last_fullsized":"2024-09-10T08:39:11.590486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:19.468224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655269+0000","last_change":"2024-09-10T08:36:53.719756+0000","last_active":"2024-09-10T08:38:43.655269+0000","last_peered":"2024-09-10T08:38:43.655269+0000","last_clean":"2024-09-10T08:38:43.655269+0000","last_became_active":"2024-09-10T08:36:53.719302+0000","last_became_peered":"2024-09-10T08:36:53.719302+0000","last_unstale":"2024-09-10T08:38:43.655269+0000","last_undegraded":"2024-09-10T08:38:43.655269+0000","last_fullsized":"2024-09-10T08:38:43.655269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:47.658081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631788+0000","last_change":"2024-09-10T08:35:39.150715+0000","last_active":"2024-09-10T08:38:43.631788+0000","last_peered":"2024-09-10T08:38:43.631788+0000","last_clean":"2024-09-10T08:38:43.631788+0000","last_became_active":"2024-09-10T08:35:39.150206+0000","last_became_peered":"2024-09-10T08:35:39.150206+0000","last_unstale":"2024-09-10T08:38:43.631788+0000","last_undegraded":"2024-09-10T08:38:43.631788+0000","last_fullsized":"2024-09-10T08:38:43.631788+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:32:02.110870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:32:02.110870+0000","last_clean_scrub_stamp":"2024-09-10T08:32:02.110870+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:20.762043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631998+0000","last_change":"2024-09-10T08:37:42.846547+0000","last_active":"2024-09-10T08:38:43.631998+0000","last_peered":"2024-09-10T08:38:43.631998+0000","last_clean":"2024-09-10T08:38:43.631998+0000","last_became_active":"2024-09-10T08:37:42.845884+0000","last_became_peered":"2024-09-10T08:37:42.845884+0000","last_unstale":"2024-09-10T08:38:43.631998+0000","last_undegraded":"2024-09-10T08:38:43.631998+0000","last_fullsized":"2024-09-10T08:38:43.631998+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:21.905133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593781+0000","last_change":"2024-09-10T08:36:56.144187+0000","last_active":"2024-09-10T08:39:11.593781+0000","last_peered":"2024-09-10T08:39:11.593781+0000","last_clean":"2024-09-10T08:39:11.593781+0000","last_became_active":"2024-09-10T08:36:56.143193+0000","last_became_peered":"2024-09-10T08:36:56.143193+0000","last_unstale":"2024-09-10T08:39:11.593781+0000","last_undegraded":"2024-09-10T08:39:11.593781+0000","last_fullsized":"2024-09-10T08:39:11.593781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:14:26.981884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.639660+0000","last_change":"2024-09-10T08:38:43.639286+0000","last_active":"2024-09-10T08:38:43.639660+0000","last_peered":"2024-09-10T08:38:43.639660+0000","last_clean":"2024-09-10T08:38:43.639660+0000","last_became_active":"2024-09-10T08:38:43.633552+0000","last_became_peered":"2024-09-10T08:38:43.633552+0000","last_unstale":"2024-09-10T08:38:43.639660+0000","last_undegraded":"2024-09-10T08:38:43.639660+0000","last_fullsized":"2024-09-10T08:38:43.639660+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:23.985190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630209+0000","last_change":"2024-09-10T08:37:05.242095+0000","last_active":"2024-09-10T08:38:43.630209+0000","last_peered":"2024-09-10T08:38:43.630209+0000","last_clean":"2024-09-10T08:38:43.630209+0000","last_became_active":"2024-09-10T08:37:05.241645+0000","last_became_peered":"2024-09-10T08:37:05.241645+0000","last_unstale":"2024-09-10T08:38:43.630209+0000","last_undegraded":"2024-09-10T08:38:43.630209+0000","last_fullsized":"2024-09-10T08:38:43.630209+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:30:49.637861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588835+0000","last_change":"2024-09-10T08:36:56.149396+0000","last_active":"2024-09-10T08:39:11.588835+0000","last_peered":"2024-09-10T08:39:11.588835+0000","last_clean":"2024-09-10T08:39:11.588835+0000","last_became_active":"2024-09-10T08:36:56.149079+0000","last_became_peered":"2024-09-10T08:36:56.149079+0000","last_unstale":"2024-09-10T08:39:11.588835+0000","last_undegraded":"2024-09-10T08:39:11.588835+0000","last_fullsized":"2024-09-10T08:39:11.588835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:44.665448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655467+0000","last_change":"2024-09-10T08:36:53.717950+0000","last_active":"2024-09-10T08:38:43.655467+0000","last_peered":"2024-09-10T08:38:43.655467+0000","last_clean":"2024-09-10T08:38:43.655467+0000","last_became_active":"2024-09-10T08:36:53.717668+0000","last_became_peered":"2024-09-10T08:36:53.717668+0000","last_unstale":"2024-09-10T08:38:43.655467+0000","last_undegraded":"2024-09-10T08:38:43.655467+0000","last_fullsized":"2024-09-10T08:38:43.655467+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:58.692631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633140+0000","last_change":"2024-09-10T08:37:05.260178+0000","last_active":"2024-09-10T08:38:43.633140+0000","last_peered":"2024-09-10T08:38:43.633140+0000","last_clean":"2024-09-10T08:38:43.633140+0000","last_became_active":"2024-09-10T08:37:05.259753+0000","last_became_peered":"2024-09-10T08:37:05.259753+0000","last_unstale":"2024-09-10T08:38:43.633140+0000","last_undegraded":"2024-09-10T08:38:43.633140+0000","last_fullsized":"2024-09-10T08:38:43.633140+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:19.364782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588360+0000","last_change":"2024-09-10T08:36:56.147104+0000","last_active":"2024-09-10T08:39:11.588360+0000","last_peered":"2024-09-10T08:39:11.588360+0000","last_clean":"2024-09-10T08:39:11.588360+0000","last_became_active":"2024-09-10T08:36:56.146778+0000","last_became_peered":"2024-09-10T08:36:56.146778+0000","last_unstale":"2024-09-10T08:39:11.588360+0000","last_undegraded":"2024-09-10T08:39:11.588360+0000","last_fullsized":"2024-09-10T08:39:11.588360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:56.092157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628399+0000","last_change":"2024-09-10T08:36:53.719145+0000","last_active":"2024-09-10T08:38:43.628399+0000","last_peered":"2024-09-10T08:38:43.628399+0000","last_clean":"2024-09-10T08:38:43.628399+0000","last_became_active":"2024-09-10T08:36:53.718771+0000","last_became_peered":"2024-09-10T08:36:53.718771+0000","last_unstale":"2024-09-10T08:38:43.628399+0000","last_undegraded":"2024-09-10T08:38:43.628399+0000","last_fullsized":"2024-09-10T08:38:43.628399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:21.109940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654733+0000","last_change":"2024-09-10T08:37:05.242854+0000","last_active":"2024-09-10T08:38:43.654733+0000","last_peered":"2024-09-10T08:38:43.654733+0000","last_clean":"2024-09-10T08:38:43.654733+0000","last_became_active":"2024-09-10T08:37:05.242442+0000","last_became_peered":"2024-09-10T08:37:05.242442+0000","last_unstale":"2024-09-10T08:38:43.654733+0000","last_undegraded":"2024-09-10T08:38:43.654733+0000","last_fullsized":"2024-09-10T08:38:43.654733+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:36.072131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.570294+0000","last_change":"2024-09-10T08:37:42.845642+0000","last_active":"2024-09-10T08:39:11.570294+0000","last_peered":"2024-09-10T08:39:11.570294+0000","last_clean":"2024-09-10T08:39:11.570294+0000","last_became_active":"2024-09-10T08:37:42.845217+0000","last_became_peered":"2024-09-10T08:37:42.845217+0000","last_unstale":"2024-09-10T08:39:11.570294+0000","last_undegraded":"2024-09-10T08:39:11.570294+0000","last_fullsized":"2024-09-10T08:39:11.570294+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:28.020337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626734+0000","last_change":"2024-09-10T08:36:53.720463+0000","last_active":"2024-09-10T08:38:43.626734+0000","last_peered":"2024-09-10T08:38:43.626734+0000","last_clean":"2024-09-10T08:38:43.626734+0000","last_became_active":"2024-09-10T08:36:53.720080+0000","last_became_peered":"2024-09-10T08:36:53.720080+0000","last_unstale":"2024-09-10T08:38:43.626734+0000","last_undegraded":"2024-09-10T08:38:43.626734+0000","last_fullsized":"2024-09-10T08:38:43.626734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:51.450294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654955+0000","last_change":"2024-09-10T08:37:05.251261+0000","last_active":"2024-09-10T08:38:43.654955+0000","last_peered":"2024-09-10T08:38:43.654955+0000","last_clean":"2024-09-10T08:38:43.654955+0000","last_became_active":"2024-09-10T08:37:05.250938+0000","last_became_peered":"2024-09-10T08:37:05.250938+0000","last_unstale":"2024-09-10T08:38:43.654955+0000","last_undegraded":"2024-09-10T08:38:43.654955+0000","last_fullsized":"2024-09-10T08:38:43.654955+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:07:15.129899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.581822+0000","last_change":"2024-09-10T08:36:56.148191+0000","last_active":"2024-09-10T08:39:11.581822+0000","last_peered":"2024-09-10T08:39:11.581822+0000","last_clean":"2024-09-10T08:39:11.581822+0000","last_became_active":"2024-09-10T08:36:56.147362+0000","last_became_peered":"2024-09-10T08:36:56.147362+0000","last_unstale":"2024-09-10T08:39:11.581822+0000","last_undegraded":"2024-09-10T08:39:11.581822+0000","last_fullsized":"2024-09-10T08:39:11.581822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:25:30.051754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626042+0000","last_change":"2024-09-10T08:36:53.734484+0000","last_active":"2024-09-10T08:38:43.626042+0000","last_peered":"2024-09-10T08:38:43.626042+0000","last_clean":"2024-09-10T08:38:43.626042+0000","last_became_active":"2024-09-10T08:36:53.734172+0000","last_became_peered":"2024-09-10T08:36:53.734172+0000","last_unstale":"2024-09-10T08:38:43.626042+0000","last_undegraded":"2024-09-10T08:38:43.626042+0000","last_fullsized":"2024-09-10T08:38:43.626042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:10.269400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628299+0000","last_change":"2024-09-10T08:37:05.259635+0000","last_active":"2024-09-10T08:38:43.628299+0000","last_peered":"2024-09-10T08:38:43.628299+0000","last_clean":"2024-09-10T08:38:43.628299+0000","last_became_active":"2024-09-10T08:37:05.259107+0000","last_became_peered":"2024-09-10T08:37:05.259107+0000","last_unstale":"2024-09-10T08:38:43.628299+0000","last_undegraded":"2024-09-10T08:38:43.628299+0000","last_fullsized":"2024-09-10T08:38:43.628299+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:36.841138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588992+0000","last_change":"2024-09-10T08:36:56.141122+0000","last_active":"2024-09-10T08:39:11.588992+0000","last_peered":"2024-09-10T08:39:11.588992+0000","last_clean":"2024-09-10T08:39:11.588992+0000","last_became_active":"2024-09-10T08:36:56.140826+0000","last_became_peered":"2024-09-10T08:36:56.140826+0000","last_unstale":"2024-09-10T08:39:11.588992+0000","last_undegraded":"2024-09-10T08:39:11.588992+0000","last_fullsized":"2024-09-10T08:39:11.588992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:38.044090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"96'6","reported_seq":174,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:48.904683+0000","last_change":"2024-09-10T08:36:53.720623+0000","last_active":"2024-09-10T08:39:48.904683+0000","last_peered":"2024-09-10T08:39:48.904683+0000","last_clean":"2024-09-10T08:39:48.904683+0000","last_became_active":"2024-09-10T08:36:53.720270+0000","last_became_peered":"2024-09-10T08:36:53.720270+0000","last_unstale":"2024-09-10T08:39:48.904683+0000","last_undegraded":"2024-09-10T08:39:48.904683+0000","last_fullsized":"2024-09-10T08:39:48.904683+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:19.514068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629937+0000","last_change":"2024-09-10T08:37:05.254324+0000","last_active":"2024-09-10T08:38:43.629937+0000","last_peered":"2024-09-10T08:38:43.629937+0000","last_clean":"2024-09-10T08:38:43.629937+0000","last_became_active":"2024-09-10T08:37:05.253611+0000","last_became_peered":"2024-09-10T08:37:05.253611+0000","last_unstale":"2024-09-10T08:38:43.629937+0000","last_undegraded":"2024-09-10T08:38:43.629937+0000","last_fullsized":"2024-09-10T08:38:43.629937+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:07.182649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.591257+0000","last_change":"2024-09-10T08:36:56.139102+0000","last_active":"2024-09-10T08:39:11.591257+0000","last_peered":"2024-09-10T08:39:11.591257+0000","last_clean":"2024-09-10T08:39:11.591257+0000","last_became_active":"2024-09-10T08:36:56.137986+0000","last_became_peered":"2024-09-10T08:36:56.137986+0000","last_unstale":"2024-09-10T08:39:11.591257+0000","last_undegraded":"2024-09-10T08:39:11.591257+0000","last_fullsized":"2024-09-10T08:39:11.591257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:05:03.005727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655021+0000","last_change":"2024-09-10T08:36:53.722961+0000","last_active":"2024-09-10T08:38:43.655021+0000","last_peered":"2024-09-10T08:38:43.655021+0000","last_clean":"2024-09-10T08:38:43.655021+0000","last_became_active":"2024-09-10T08:36:53.722628+0000","last_became_peered":"2024-09-10T08:36:53.722628+0000","last_unstale":"2024-09-10T08:38:43.655021+0000","last_undegraded":"2024-09-10T08:38:43.655021+0000","last_fullsized":"2024-09-10T08:38:43.655021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:59.048632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628104+0000","last_change":"2024-09-10T08:37:05.256546+0000","last_active":"2024-09-10T08:38:43.628104+0000","last_peered":"2024-09-10T08:38:43.628104+0000","last_clean":"2024-09-10T08:38:43.628104+0000","last_became_active":"2024-09-10T08:37:05.256239+0000","last_became_peered":"2024-09-10T08:37:05.256239+0000","last_unstale":"2024-09-10T08:38:43.628104+0000","last_undegraded":"2024-09-10T08:38:43.628104+0000","last_fullsized":"2024-09-10T08:38:43.628104+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:57.403934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.582848+0000","last_change":"2024-09-10T08:36:56.149285+0000","last_active":"2024-09-10T08:39:11.582848+0000","last_peered":"2024-09-10T08:39:11.582848+0000","last_clean":"2024-09-10T08:39:11.582848+0000","last_became_active":"2024-09-10T08:36:56.146873+0000","last_became_peered":"2024-09-10T08:36:56.146873+0000","last_unstale":"2024-09-10T08:39:11.582848+0000","last_undegraded":"2024-09-10T08:39:11.582848+0000","last_fullsized":"2024-09-10T08:39:11.582848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:28.976729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630368+0000","last_change":"2024-09-10T08:36:53.721174+0000","last_active":"2024-09-10T08:38:43.630368+0000","last_peered":"2024-09-10T08:38:43.630368+0000","last_clean":"2024-09-10T08:38:43.630368+0000","last_became_active":"2024-09-10T08:36:53.720849+0000","last_became_peered":"2024-09-10T08:36:53.720849+0000","last_unstale":"2024-09-10T08:38:43.630368+0000","last_undegraded":"2024-09-10T08:38:43.630368+0000","last_fullsized":"2024-09-10T08:38:43.630368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:39.786848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655286+0000","last_change":"2024-09-10T08:37:05.252256+0000","last_active":"2024-09-10T08:38:43.655286+0000","last_peered":"2024-09-10T08:38:43.655286+0000","last_clean":"2024-09-10T08:38:43.655286+0000","last_became_active":"2024-09-10T08:37:05.251865+0000","last_became_peered":"2024-09-10T08:37:05.251865+0000","last_unstale":"2024-09-10T08:38:43.655286+0000","last_undegraded":"2024-09-10T08:38:43.655286+0000","last_fullsized":"2024-09-10T08:38:43.655286+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:08.581880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567773+0000","last_change":"2024-09-10T08:36:56.142060+0000","last_active":"2024-09-10T08:39:11.567773+0000","last_peered":"2024-09-10T08:39:11.567773+0000","last_clean":"2024-09-10T08:39:11.567773+0000","last_became_active":"2024-09-10T08:36:56.141451+0000","last_became_peered":"2024-09-10T08:36:56.141451+0000","last_unstale":"2024-09-10T08:39:11.567773+0000","last_undegraded":"2024-09-10T08:39:11.567773+0000","last_fullsized":"2024-09-10T08:39:11.567773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:43.566588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'39","reported_seq":334,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:48.893004+0000","last_change":"2024-09-10T08:36:53.719464+0000","last_active":"2024-09-10T08:39:48.893004+0000","last_peered":"2024-09-10T08:39:48.893004+0000","last_clean":"2024-09-10T08:39:48.893004+0000","last_became_active":"2024-09-10T08:36:53.719129+0000","last_became_peered":"2024-09-10T08:36:53.719129+0000","last_unstale":"2024-09-10T08:39:48.893004+0000","last_undegraded":"2024-09-10T08:39:48.893004+0000","last_fullsized":"2024-09-10T08:39:48.893004+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:05.652018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71298,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":254,"num_read_kb":197,"num_write":40,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629405+0000","last_change":"2024-09-10T08:37:05.255576+0000","last_active":"2024-09-10T08:38:43.629405+0000","last_peered":"2024-09-10T08:38:43.629405+0000","last_clean":"2024-09-10T08:38:43.629405+0000","last_became_active":"2024-09-10T08:37:05.249842+0000","last_became_peered":"2024-09-10T08:37:05.249842+0000","last_unstale":"2024-09-10T08:38:43.629405+0000","last_undegraded":"2024-09-10T08:38:43.629405+0000","last_fullsized":"2024-09-10T08:38:43.629405+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:47:49.543481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593397+0000","last_change":"2024-09-10T08:36:56.151829+0000","last_active":"2024-09-10T08:39:11.593397+0000","last_peered":"2024-09-10T08:39:11.593397+0000","last_clean":"2024-09-10T08:39:11.593397+0000","last_became_active":"2024-09-10T08:36:56.151314+0000","last_became_peered":"2024-09-10T08:36:56.151314+0000","last_unstale":"2024-09-10T08:39:11.593397+0000","last_undegraded":"2024-09-10T08:39:11.593397+0000","last_fullsized":"2024-09-10T08:39:11.593397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:44.219671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655398+0000","last_change":"2024-09-10T08:36:53.738030+0000","last_active":"2024-09-10T08:38:43.655398+0000","last_peered":"2024-09-10T08:38:43.655398+0000","last_clean":"2024-09-10T08:38:43.655398+0000","last_became_active":"2024-09-10T08:36:53.737658+0000","last_became_peered":"2024-09-10T08:36:53.737658+0000","last_unstale":"2024-09-10T08:38:43.655398+0000","last_undegraded":"2024-09-10T08:38:43.655398+0000","last_fullsized":"2024-09-10T08:38:43.655398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:29.627554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654996+0000","last_change":"2024-09-10T08:37:05.244473+0000","last_active":"2024-09-10T08:38:43.654996+0000","last_peered":"2024-09-10T08:38:43.654996+0000","last_clean":"2024-09-10T08:38:43.654996+0000","last_became_active":"2024-09-10T08:37:05.244035+0000","last_became_peered":"2024-09-10T08:37:05.244035+0000","last_unstale":"2024-09-10T08:38:43.654996+0000","last_undegraded":"2024-09-10T08:38:43.654996+0000","last_fullsized":"2024-09-10T08:38:43.654996+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:04.652660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584024+0000","last_change":"2024-09-10T08:36:56.155217+0000","last_active":"2024-09-10T08:39:11.584024+0000","last_peered":"2024-09-10T08:39:11.584024+0000","last_clean":"2024-09-10T08:39:11.584024+0000","last_became_active":"2024-09-10T08:36:56.154903+0000","last_became_peered":"2024-09-10T08:36:56.154903+0000","last_unstale":"2024-09-10T08:39:11.584024+0000","last_undegraded":"2024-09-10T08:39:11.584024+0000","last_fullsized":"2024-09-10T08:39:11.584024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:47.482974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.632239+0000","last_change":"2024-09-10T08:36:53.721087+0000","last_active":"2024-09-10T08:38:43.632239+0000","last_peered":"2024-09-10T08:38:43.632239+0000","last_clean":"2024-09-10T08:38:43.632239+0000","last_became_active":"2024-09-10T08:36:53.720772+0000","last_became_peered":"2024-09-10T08:36:53.720772+0000","last_unstale":"2024-09-10T08:38:43.632239+0000","last_undegraded":"2024-09-10T08:38:43.632239+0000","last_fullsized":"2024-09-10T08:38:43.632239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:39.560848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629150+0000","last_change":"2024-09-10T08:37:05.254641+0000","last_active":"2024-09-10T08:38:43.629150+0000","last_peered":"2024-09-10T08:38:43.629150+0000","last_clean":"2024-09-10T08:38:43.629150+0000","last_became_active":"2024-09-10T08:37:05.249982+0000","last_became_peered":"2024-09-10T08:37:05.249982+0000","last_unstale":"2024-09-10T08:38:43.629150+0000","last_undegraded":"2024-09-10T08:38:43.629150+0000","last_fullsized":"2024-09-10T08:38:43.629150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:21:13.851061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592510+0000","last_change":"2024-09-10T08:36:56.142657+0000","last_active":"2024-09-10T08:39:11.592510+0000","last_peered":"2024-09-10T08:39:11.592510+0000","last_clean":"2024-09-10T08:39:11.592510+0000","last_became_active":"2024-09-10T08:36:56.142294+0000","last_became_peered":"2024-09-10T08:36:56.142294+0000","last_unstale":"2024-09-10T08:39:11.592510+0000","last_undegraded":"2024-09-10T08:39:11.592510+0000","last_fullsized":"2024-09-10T08:39:11.592510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:52.861389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'4","reported_seq":166,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:47.891688+0000","last_change":"2024-09-10T08:36:53.722466+0000","last_active":"2024-09-10T08:39:47.891688+0000","last_peered":"2024-09-10T08:39:47.891688+0000","last_clean":"2024-09-10T08:39:47.891688+0000","last_became_active":"2024-09-10T08:36:53.722145+0000","last_became_peered":"2024-09-10T08:36:53.722145+0000","last_unstale":"2024-09-10T08:39:47.891688+0000","last_undegraded":"2024-09-10T08:39:47.891688+0000","last_fullsized":"2024-09-10T08:39:47.891688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:27.041765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631352+0000","last_change":"2024-09-10T08:37:05.251056+0000","last_active":"2024-09-10T08:38:43.631352+0000","last_peered":"2024-09-10T08:38:43.631352+0000","last_clean":"2024-09-10T08:38:43.631352+0000","last_became_active":"2024-09-10T08:37:05.250653+0000","last_became_peered":"2024-09-10T08:37:05.250653+0000","last_unstale":"2024-09-10T08:38:43.631352+0000","last_undegraded":"2024-09-10T08:38:43.631352+0000","last_fullsized":"2024-09-10T08:38:43.631352+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:21.174747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633128+0000","last_change":"2024-09-10T08:37:05.260436+0000","last_active":"2024-09-10T08:38:43.633128+0000","last_peered":"2024-09-10T08:38:43.633128+0000","last_clean":"2024-09-10T08:38:43.633128+0000","last_became_active":"2024-09-10T08:37:05.260049+0000","last_became_peered":"2024-09-10T08:37:05.260049+0000","last_unstale":"2024-09-10T08:38:43.633128+0000","last_undegraded":"2024-09-10T08:38:43.633128+0000","last_fullsized":"2024-09-10T08:38:43.633128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:43.394028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585930+0000","last_change":"2024-09-10T08:36:56.146732+0000","last_active":"2024-09-10T08:39:11.585930+0000","last_peered":"2024-09-10T08:39:11.585930+0000","last_clean":"2024-09-10T08:39:11.585930+0000","last_became_active":"2024-09-10T08:36:56.145599+0000","last_became_peered":"2024-09-10T08:36:56.145599+0000","last_unstale":"2024-09-10T08:39:11.585930+0000","last_undegraded":"2024-09-10T08:39:11.585930+0000","last_fullsized":"2024-09-10T08:39:11.585930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:50.620956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655135+0000","last_change":"2024-09-10T08:36:53.721683+0000","last_active":"2024-09-10T08:38:43.655135+0000","last_peered":"2024-09-10T08:38:43.655135+0000","last_clean":"2024-09-10T08:38:43.655135+0000","last_became_active":"2024-09-10T08:36:53.721241+0000","last_became_peered":"2024-09-10T08:36:53.721241+0000","last_unstale":"2024-09-10T08:38:43.655135+0000","last_undegraded":"2024-09-10T08:38:43.655135+0000","last_fullsized":"2024-09-10T08:38:43.655135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:03.669337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628275+0000","last_change":"2024-09-10T08:37:42.669784+0000","last_active":"2024-09-10T08:38:43.628275+0000","last_peered":"2024-09-10T08:38:43.628275+0000","last_clean":"2024-09-10T08:38:43.628275+0000","last_became_active":"2024-09-10T08:37:42.669322+0000","last_became_peered":"2024-09-10T08:37:42.669322+0000","last_unstale":"2024-09-10T08:38:43.628275+0000","last_undegraded":"2024-09-10T08:38:43.628275+0000","last_fullsized":"2024-09-10T08:38:43.628275+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:06:57.027928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.574617+0000","last_change":"2024-09-10T08:36:56.143824+0000","last_active":"2024-09-10T08:39:11.574617+0000","last_peered":"2024-09-10T08:39:11.574617+0000","last_clean":"2024-09-10T08:39:11.574617+0000","last_became_active":"2024-09-10T08:36:56.143486+0000","last_became_peered":"2024-09-10T08:36:56.143486+0000","last_unstale":"2024-09-10T08:39:11.574617+0000","last_undegraded":"2024-09-10T08:39:11.574617+0000","last_fullsized":"2024-09-10T08:39:11.574617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:42.384863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627976+0000","last_change":"2024-09-10T08:36:53.717680+0000","last_active":"2024-09-10T08:38:43.627976+0000","last_peered":"2024-09-10T08:38:43.627976+0000","last_clean":"2024-09-10T08:38:43.627976+0000","last_became_active":"2024-09-10T08:36:53.717249+0000","last_became_peered":"2024-09-10T08:36:53.717249+0000","last_unstale":"2024-09-10T08:38:43.627976+0000","last_undegraded":"2024-09-10T08:38:43.627976+0000","last_fullsized":"2024-09-10T08:38:43.627976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:12.191541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627200+0000","last_change":"2024-09-10T08:36:53.723947+0000","last_active":"2024-09-10T08:38:43.627200+0000","last_peered":"2024-09-10T08:38:43.627200+0000","last_clean":"2024-09-10T08:38:43.627200+0000","last_became_active":"2024-09-10T08:36:53.723620+0000","last_became_peered":"2024-09-10T08:36:53.723620+0000","last_unstale":"2024-09-10T08:38:43.627200+0000","last_undegraded":"2024-09-10T08:38:43.627200+0000","last_fullsized":"2024-09-10T08:38:43.627200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:00.135349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583145+0000","last_change":"2024-09-10T08:36:56.150448+0000","last_active":"2024-09-10T08:39:11.583145+0000","last_peered":"2024-09-10T08:39:11.583145+0000","last_clean":"2024-09-10T08:39:11.583145+0000","last_became_active":"2024-09-10T08:36:56.150148+0000","last_became_peered":"2024-09-10T08:36:56.150148+0000","last_unstale":"2024-09-10T08:39:11.583145+0000","last_undegraded":"2024-09-10T08:39:11.583145+0000","last_fullsized":"2024-09-10T08:39:11.583145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:02.927753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630191+0000","last_change":"2024-09-10T08:37:05.242878+0000","last_active":"2024-09-10T08:38:43.630191+0000","last_peered":"2024-09-10T08:38:43.630191+0000","last_clean":"2024-09-10T08:38:43.630191+0000","last_became_active":"2024-09-10T08:37:05.242440+0000","last_became_peered":"2024-09-10T08:37:05.242440+0000","last_unstale":"2024-09-10T08:38:43.630191+0000","last_undegraded":"2024-09-10T08:38:43.630191+0000","last_fullsized":"2024-09-10T08:38:43.630191+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:30.957433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'7","reported_seq":174,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:48.893331+0000","last_change":"2024-09-10T08:36:53.719788+0000","last_active":"2024-09-10T08:39:48.893331+0000","last_peered":"2024-09-10T08:39:48.893331+0000","last_clean":"2024-09-10T08:39:48.893331+0000","last_became_active":"2024-09-10T08:36:53.719462+0000","last_became_peered":"2024-09-10T08:36:53.719462+0000","last_unstale":"2024-09-10T08:39:48.893331+0000","last_undegraded":"2024-09-10T08:39:48.893331+0000","last_fullsized":"2024-09-10T08:39:48.893331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:12.250447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1326,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583170+0000","last_change":"2024-09-10T08:37:42.673008+0000","last_active":"2024-09-10T08:39:11.583170+0000","last_peered":"2024-09-10T08:39:11.583170+0000","last_clean":"2024-09-10T08:39:11.583170+0000","last_became_active":"2024-09-10T08:37:42.672618+0000","last_became_peered":"2024-09-10T08:37:42.672618+0000","last_unstale":"2024-09-10T08:39:11.583170+0000","last_undegraded":"2024-09-10T08:39:11.583170+0000","last_fullsized":"2024-09-10T08:39:11.583170+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:00.156565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630783+0000","last_change":"2024-09-10T08:37:05.244277+0000","last_active":"2024-09-10T08:38:43.630783+0000","last_peered":"2024-09-10T08:38:43.630783+0000","last_clean":"2024-09-10T08:38:43.630783+0000","last_became_active":"2024-09-10T08:37:05.243880+0000","last_became_peered":"2024-09-10T08:37:05.243880+0000","last_unstale":"2024-09-10T08:38:43.630783+0000","last_undegraded":"2024-09-10T08:38:43.630783+0000","last_fullsized":"2024-09-10T08:38:43.630783+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:55.275857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.573131+0000","last_change":"2024-09-10T08:36:56.141404+0000","last_active":"2024-09-10T08:39:11.573131+0000","last_peered":"2024-09-10T08:39:11.573131+0000","last_clean":"2024-09-10T08:39:11.573131+0000","last_became_active":"2024-09-10T08:36:56.141076+0000","last_became_peered":"2024-09-10T08:36:56.141076+0000","last_unstale":"2024-09-10T08:39:11.573131+0000","last_undegraded":"2024-09-10T08:39:11.573131+0000","last_fullsized":"2024-09-10T08:39:11.573131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:03:52.915654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.569383+0000","last_change":"2024-09-10T08:36:56.148285+0000","last_active":"2024-09-10T08:39:11.569383+0000","last_peered":"2024-09-10T08:39:11.569383+0000","last_clean":"2024-09-10T08:39:11.569383+0000","last_became_active":"2024-09-10T08:36:56.147970+0000","last_became_peered":"2024-09-10T08:36:56.147970+0000","last_unstale":"2024-09-10T08:39:11.569383+0000","last_undegraded":"2024-09-10T08:39:11.569383+0000","last_fullsized":"2024-09-10T08:39:11.569383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:09.506718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.571047+0000","last_change":"2024-09-10T08:36:56.157977+0000","last_active":"2024-09-10T08:39:11.571047+0000","last_peered":"2024-09-10T08:39:11.571047+0000","last_clean":"2024-09-10T08:39:11.571047+0000","last_became_active":"2024-09-10T08:36:56.157623+0000","last_became_peered":"2024-09-10T08:36:56.157623+0000","last_unstale":"2024-09-10T08:39:11.571047+0000","last_undegraded":"2024-09-10T08:39:11.571047+0000","last_fullsized":"2024-09-10T08:39:11.571047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:44.617979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584899+0000","last_change":"2024-09-10T08:37:42.670459+0000","last_active":"2024-09-10T08:39:11.584899+0000","last_peered":"2024-09-10T08:39:11.584899+0000","last_clean":"2024-09-10T08:39:11.584899+0000","last_became_active":"2024-09-10T08:37:42.670097+0000","last_became_peered":"2024-09-10T08:37:42.670097+0000","last_unstale":"2024-09-10T08:39:11.584899+0000","last_undegraded":"2024-09-10T08:39:11.584899+0000","last_fullsized":"2024-09-10T08:39:11.584899+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:14.956492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584527+0000","last_change":"2024-09-10T08:36:56.154850+0000","last_active":"2024-09-10T08:39:11.584527+0000","last_peered":"2024-09-10T08:39:11.584527+0000","last_clean":"2024-09-10T08:39:11.584527+0000","last_became_active":"2024-09-10T08:36:56.154545+0000","last_became_peered":"2024-09-10T08:36:56.154545+0000","last_unstale":"2024-09-10T08:39:11.584527+0000","last_undegraded":"2024-09-10T08:39:11.584527+0000","last_fullsized":"2024-09-10T08:39:11.584527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:24.639667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.569973+0000","last_change":"2024-09-10T08:37:42.669962+0000","last_active":"2024-09-10T08:39:11.569973+0000","last_peered":"2024-09-10T08:39:11.569973+0000","last_clean":"2024-09-10T08:39:11.569973+0000","last_became_active":"2024-09-10T08:37:42.669556+0000","last_became_peered":"2024-09-10T08:37:42.669556+0000","last_unstale":"2024-09-10T08:39:11.569973+0000","last_undegraded":"2024-09-10T08:39:11.569973+0000","last_fullsized":"2024-09-10T08:39:11.569973+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:23.333946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.578061+0000","last_change":"2024-09-10T08:36:56.145910+0000","last_active":"2024-09-10T08:39:11.578061+0000","last_peered":"2024-09-10T08:39:11.578061+0000","last_clean":"2024-09-10T08:39:11.578061+0000","last_became_active":"2024-09-10T08:36:56.145537+0000","last_became_peered":"2024-09-10T08:36:56.145537+0000","last_unstale":"2024-09-10T08:39:11.578061+0000","last_undegraded":"2024-09-10T08:39:11.578061+0000","last_fullsized":"2024-09-10T08:39:11.578061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:46.788241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567880+0000","last_change":"2024-09-10T08:36:56.147340+0000","last_active":"2024-09-10T08:39:11.567880+0000","last_peered":"2024-09-10T08:39:11.567880+0000","last_clean":"2024-09-10T08:39:11.567880+0000","last_became_active":"2024-09-10T08:36:56.145762+0000","last_became_peered":"2024-09-10T08:36:56.145762+0000","last_unstale":"2024-09-10T08:39:11.567880+0000","last_undegraded":"2024-09-10T08:39:11.567880+0000","last_fullsized":"2024-09-10T08:39:11.567880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:12.339285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.581431+0000","last_change":"2024-09-10T08:36:56.145213+0000","last_active":"2024-09-10T08:39:11.581431+0000","last_peered":"2024-09-10T08:39:11.581431+0000","last_clean":"2024-09-10T08:39:11.581431+0000","last_became_active":"2024-09-10T08:36:56.144915+0000","last_became_peered":"2024-09-10T08:36:56.144915+0000","last_unstale":"2024-09-10T08:39:11.581431+0000","last_undegraded":"2024-09-10T08:39:11.581431+0000","last_fullsized":"2024-09-10T08:39:11.581431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:06.180321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.574743+0000","last_change":"2024-09-10T08:36:56.149738+0000","last_active":"2024-09-10T08:39:11.574743+0000","last_peered":"2024-09-10T08:39:11.574743+0000","last_clean":"2024-09-10T08:39:11.574743+0000","last_became_active":"2024-09-10T08:36:56.147750+0000","last_became_peered":"2024-09-10T08:36:56.147750+0000","last_unstale":"2024-09-10T08:39:11.574743+0000","last_undegraded":"2024-09-10T08:39:11.574743+0000","last_fullsized":"2024-09-10T08:39:11.574743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:47.950054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.591937+0000","last_change":"2024-09-10T08:36:56.147682+0000","last_active":"2024-09-10T08:39:11.591937+0000","last_peered":"2024-09-10T08:39:11.591937+0000","last_clean":"2024-09-10T08:39:11.591937+0000","last_became_active":"2024-09-10T08:36:56.147345+0000","last_became_peered":"2024-09-10T08:36:56.147345+0000","last_unstale":"2024-09-10T08:39:11.591937+0000","last_undegraded":"2024-09-10T08:39:11.591937+0000","last_fullsized":"2024-09-10T08:39:11.591937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:54.939482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583378+0000","last_change":"2024-09-10T08:36:56.158292+0000","last_active":"2024-09-10T08:39:11.583378+0000","last_peered":"2024-09-10T08:39:11.583378+0000","last_clean":"2024-09-10T08:39:11.583378+0000","last_became_active":"2024-09-10T08:36:56.157970+0000","last_became_peered":"2024-09-10T08:36:56.157970+0000","last_unstale":"2024-09-10T08:39:11.583378+0000","last_undegraded":"2024-09-10T08:39:11.583378+0000","last_fullsized":"2024-09-10T08:39:11.583378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:33.582126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567916+0000","last_change":"2024-09-10T08:36:56.142520+0000","last_active":"2024-09-10T08:39:11.567916+0000","last_peered":"2024-09-10T08:39:11.567916+0000","last_clean":"2024-09-10T08:39:11.567916+0000","last_became_active":"2024-09-10T08:36:56.142214+0000","last_became_peered":"2024-09-10T08:36:56.142214+0000","last_unstale":"2024-09-10T08:39:11.567916+0000","last_undegraded":"2024-09-10T08:39:11.567916+0000","last_fullsized":"2024-09-10T08:39:11.567916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:45:29.201839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584167+0000","last_change":"2024-09-10T08:36:56.149547+0000","last_active":"2024-09-10T08:39:11.584167+0000","last_peered":"2024-09-10T08:39:11.584167+0000","last_clean":"2024-09-10T08:39:11.584167+0000","last_became_active":"2024-09-10T08:36:56.149141+0000","last_became_peered":"2024-09-10T08:36:56.149141+0000","last_unstale":"2024-09-10T08:39:11.584167+0000","last_undegraded":"2024-09-10T08:39:11.584167+0000","last_fullsized":"2024-09-10T08:39:11.584167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:49.368677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585864+0000","last_change":"2024-09-10T08:36:56.143087+0000","last_active":"2024-09-10T08:39:11.585864+0000","last_peered":"2024-09-10T08:39:11.585864+0000","last_clean":"2024-09-10T08:39:11.585864+0000","last_became_active":"2024-09-10T08:36:56.142782+0000","last_became_peered":"2024-09-10T08:36:56.142782+0000","last_unstale":"2024-09-10T08:39:11.585864+0000","last_undegraded":"2024-09-10T08:39:11.585864+0000","last_fullsized":"2024-09-10T08:39:11.585864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:17.513103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592876+0000","last_change":"2024-09-10T08:36:56.157549+0000","last_active":"2024-09-10T08:39:11.592876+0000","last_peered":"2024-09-10T08:39:11.592876+0000","last_clean":"2024-09-10T08:39:11.592876+0000","last_became_active":"2024-09-10T08:36:56.157221+0000","last_became_peered":"2024-09-10T08:36:56.157221+0000","last_unstale":"2024-09-10T08:39:11.592876+0000","last_undegraded":"2024-09-10T08:39:11.592876+0000","last_fullsized":"2024-09-10T08:39:11.592876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:11.372085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593199+0000","last_change":"2024-09-10T08:36:56.145654+0000","last_active":"2024-09-10T08:39:11.593199+0000","last_peered":"2024-09-10T08:39:11.593199+0000","last_clean":"2024-09-10T08:39:11.593199+0000","last_became_active":"2024-09-10T08:36:56.144126+0000","last_became_peered":"2024-09-10T08:36:56.144126+0000","last_unstale":"2024-09-10T08:39:11.593199+0000","last_undegraded":"2024-09-10T08:39:11.593199+0000","last_fullsized":"2024-09-10T08:39:11.593199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:55.498873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.582479+0000","last_change":"2024-09-10T08:36:56.146423+0000","last_active":"2024-09-10T08:39:11.582479+0000","last_peered":"2024-09-10T08:39:11.582479+0000","last_clean":"2024-09-10T08:39:11.582479+0000","last_became_active":"2024-09-10T08:36:56.146126+0000","last_became_peered":"2024-09-10T08:36:56.146126+0000","last_unstale":"2024-09-10T08:39:11.582479+0000","last_undegraded":"2024-09-10T08:39:11.582479+0000","last_fullsized":"2024-09-10T08:39:11.582479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:44.665196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583332+0000","last_change":"2024-09-10T08:36:56.151973+0000","last_active":"2024-09-10T08:39:11.583332+0000","last_peered":"2024-09-10T08:39:11.583332+0000","last_clean":"2024-09-10T08:39:11.583332+0000","last_became_active":"2024-09-10T08:36:56.151609+0000","last_became_peered":"2024-09-10T08:36:56.151609+0000","last_unstale":"2024-09-10T08:39:11.583332+0000","last_undegraded":"2024-09-10T08:39:11.583332+0000","last_fullsized":"2024-09-10T08:39:11.583332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:02.584047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584509+0000","last_change":"2024-09-10T08:36:56.141780+0000","last_active":"2024-09-10T08:39:11.584509+0000","last_peered":"2024-09-10T08:39:11.584509+0000","last_clean":"2024-09-10T08:39:11.584509+0000","last_became_active":"2024-09-10T08:36:56.141228+0000","last_became_peered":"2024-09-10T08:36:56.141228+0000","last_unstale":"2024-09-10T08:39:11.584509+0000","last_undegraded":"2024-09-10T08:39:11.584509+0000","last_fullsized":"2024-09-10T08:39:11.584509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:52:04.374228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585477+0000","last_change":"2024-09-10T08:37:42.671346+0000","last_active":"2024-09-10T08:39:11.585477+0000","last_peered":"2024-09-10T08:39:11.585477+0000","last_clean":"2024-09-10T08:39:11.585477+0000","last_became_active":"2024-09-10T08:37:42.670895+0000","last_became_peered":"2024-09-10T08:37:42.670895+0000","last_unstale":"2024-09-10T08:39:11.585477+0000","last_undegraded":"2024-09-10T08:39:11.585477+0000","last_fullsized":"2024-09-10T08:39:11.585477+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:16:31.930628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593796+0000","last_change":"2024-09-10T08:36:56.149014+0000","last_active":"2024-09-10T08:39:11.593796+0000","last_peered":"2024-09-10T08:39:11.593796+0000","last_clean":"2024-09-10T08:39:11.593796+0000","last_became_active":"2024-09-10T08:36:56.148551+0000","last_became_peered":"2024-09-10T08:36:56.148551+0000","last_unstale":"2024-09-10T08:39:11.593796+0000","last_undegraded":"2024-09-10T08:39:11.593796+0000","last_fullsized":"2024-09-10T08:39:11.593796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:35:52.250943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.586709+0000","last_change":"2024-09-10T08:36:56.141247+0000","last_active":"2024-09-10T08:39:11.586709+0000","last_peered":"2024-09-10T08:39:11.586709+0000","last_clean":"2024-09-10T08:39:11.586709+0000","last_became_active":"2024-09-10T08:36:56.140956+0000","last_became_peered":"2024-09-10T08:36:56.140956+0000","last_unstale":"2024-09-10T08:39:11.586709+0000","last_undegraded":"2024-09-10T08:39:11.586709+0000","last_fullsized":"2024-09-10T08:39:11.586709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:45.008424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592783+0000","last_change":"2024-09-10T08:36:56.149116+0000","last_active":"2024-09-10T08:39:11.592783+0000","last_peered":"2024-09-10T08:39:11.592783+0000","last_clean":"2024-09-10T08:39:11.592783+0000","last_became_active":"2024-09-10T08:36:56.148783+0000","last_became_peered":"2024-09-10T08:36:56.148783+0000","last_unstale":"2024-09-10T08:39:11.592783+0000","last_undegraded":"2024-09-10T08:39:11.592783+0000","last_fullsized":"2024-09-10T08:39:11.592783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:37.119214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585754+0000","last_change":"2024-09-10T08:37:42.846322+0000","last_active":"2024-09-10T08:39:11.585754+0000","last_peered":"2024-09-10T08:39:11.585754+0000","last_clean":"2024-09-10T08:39:11.585754+0000","last_became_active":"2024-09-10T08:37:42.846022+0000","last_became_peered":"2024-09-10T08:37:42.846022+0000","last_unstale":"2024-09-10T08:39:11.585754+0000","last_undegraded":"2024-09-10T08:39:11.585754+0000","last_fullsized":"2024-09-10T08:39:11.585754+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:38.142362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588751+0000","last_change":"2024-09-10T08:36:56.148537+0000","last_active":"2024-09-10T08:39:11.588751+0000","last_peered":"2024-09-10T08:39:11.588751+0000","last_clean":"2024-09-10T08:39:11.588751+0000","last_became_active":"2024-09-10T08:36:56.148185+0000","last_became_peered":"2024-09-10T08:36:56.148185+0000","last_unstale":"2024-09-10T08:39:11.588751+0000","last_undegraded":"2024-09-10T08:39:11.588751+0000","last_fullsized":"2024-09-10T08:39:11.588751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:00.426513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.586233+0000","last_change":"2024-09-10T08:36:56.138513+0000","last_active":"2024-09-10T08:39:11.586233+0000","last_peered":"2024-09-10T08:39:11.586233+0000","last_clean":"2024-09-10T08:39:11.586233+0000","last_became_active":"2024-09-10T08:36:56.138207+0000","last_became_peered":"2024-09-10T08:36:56.138207+0000","last_unstale":"2024-09-10T08:39:11.586233+0000","last_undegraded":"2024-09-10T08:39:11.586233+0000","last_fullsized":"2024-09-10T08:39:11.586233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:34.581430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.582686+0000","last_change":"2024-09-10T08:36:56.146612+0000","last_active":"2024-09-10T08:39:11.582686+0000","last_peered":"2024-09-10T08:39:11.582686+0000","last_clean":"2024-09-10T08:39:11.582686+0000","last_became_active":"2024-09-10T08:36:56.146251+0000","last_became_peered":"2024-09-10T08:36:56.146251+0000","last_unstale":"2024-09-10T08:39:11.582686+0000","last_undegraded":"2024-09-10T08:39:11.582686+0000","last_fullsized":"2024-09-10T08:39:11.582686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:47.336389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567939+0000","last_change":"2024-09-10T08:36:56.148260+0000","last_active":"2024-09-10T08:39:11.567939+0000","last_peered":"2024-09-10T08:39:11.567939+0000","last_clean":"2024-09-10T08:39:11.567939+0000","last_became_active":"2024-09-10T08:36:56.147949+0000","last_became_peered":"2024-09-10T08:36:56.147949+0000","last_unstale":"2024-09-10T08:39:11.567939+0000","last_undegraded":"2024-09-10T08:39:11.567939+0000","last_fullsized":"2024-09-10T08:39:11.567939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:52:19.762297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592986+0000","last_change":"2024-09-10T08:36:56.156045+0000","last_active":"2024-09-10T08:39:11.592986+0000","last_peered":"2024-09-10T08:39:11.592986+0000","last_clean":"2024-09-10T08:39:11.592986+0000","last_became_active":"2024-09-10T08:36:56.154727+0000","last_became_peered":"2024-09-10T08:36:56.154727+0000","last_unstale":"2024-09-10T08:39:11.592986+0000","last_undegraded":"2024-09-10T08:39:11.592986+0000","last_fullsized":"2024-09-10T08:39:11.592986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:35:39.992683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.576067+0000","last_change":"2024-09-10T08:36:56.154394+0000","last_active":"2024-09-10T08:39:11.576067+0000","last_peered":"2024-09-10T08:39:11.576067+0000","last_clean":"2024-09-10T08:39:11.576067+0000","last_became_active":"2024-09-10T08:36:56.153855+0000","last_became_peered":"2024-09-10T08:36:56.153855+0000","last_unstale":"2024-09-10T08:39:11.576067+0000","last_undegraded":"2024-09-10T08:39:11.576067+0000","last_fullsized":"2024-09-10T08:39:11.576067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:14.987147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592634+0000","last_change":"2024-09-10T08:36:56.157517+0000","last_active":"2024-09-10T08:39:11.592634+0000","last_peered":"2024-09-10T08:39:11.592634+0000","last_clean":"2024-09-10T08:39:11.592634+0000","last_became_active":"2024-09-10T08:36:56.157133+0000","last_became_peered":"2024-09-10T08:36:56.157133+0000","last_unstale":"2024-09-10T08:39:11.592634+0000","last_undegraded":"2024-09-10T08:39:11.592634+0000","last_fullsized":"2024-09-10T08:39:11.592634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:17:29.573719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76539,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":764,"num_read_kb":463,"num_write":188,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":230755,"data_compressed":5667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282455,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416434,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":547326,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678070,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28212,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716940,"statfs":{"total":95995035648,"available":95966146560,"internally_reserved":0,"allocated":1216512,"data_stored":1215011,"data_compressed":67339,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841596,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":543466,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103234,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":546900,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332167,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":1144631,"data_compressed":65450,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495692,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":542738,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757330,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":1139292,"data_compressed":65450,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920854,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":548897,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149787,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":542932,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":615418,"data_compressed":31413,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712864,"statfs":{"total":95995035648,"available":95961972736,"internally_reserved":0,"allocated":1130496,"data_stored":614528,"data_compressed":31413,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1077248,"data_stored":544126,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71880,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":73908,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73803,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:39:52.850 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:39:53.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:52 smithi111 ceph-mon[35220]: pgmap v391: 129 pgs: 129 active+clean; 652 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-10T08:39:53.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:52 smithi089 ceph-mon[30757]: pgmap v391: 129 pgs: 129 active+clean; 652 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-10T08:39:53.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:52 smithi149 ceph-mon[36167]: pgmap v391: 129 pgs: 129 active+clean; 652 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-10T08:39:53.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.390+0000 7f26cfe00640 1 -- 172.21.15.89:0/3100851134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26d01079c0 msgr2=0x7f26d0107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:53.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.390+0000 7f26cfe00640 1 --2- 172.21.15.89:0/3100851134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26d01079c0 0x7f26d0107da0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009930 tx=0x7f26c002a420 comp rx=0 tx=0).stop 2024-09-10T08:39:53.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.390+0000 7f26cfe00640 1 -- 172.21.15.89:0/3100851134 shutdown_connections 2024-09-10T08:39:53.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.390+0000 7f26cfe00640 1 --2- 172.21.15.89:0/3100851134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26d0102360 0x7f26d010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.390+0000 7f26cfe00640 1 --2- 172.21.15.89:0/3100851134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26d01019c0 0x7f26d0101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.390+0000 7f26cfe00640 1 --2- 172.21.15.89:0/3100851134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26d01079c0 0x7f26d0107da0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.390+0000 7f26cfe00640 1 -- 172.21.15.89:0/3100851134 >> 172.21.15.89:0/3100851134 conn(0x7f26d00fd740 msgr2=0x7f26d00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:53.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.390+0000 7f26cfe00640 1 -- 172.21.15.89:0/3100851134 shutdown_connections 2024-09-10T08:39:53.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.390+0000 7f26cfe00640 1 -- 172.21.15.89:0/3100851134 wait complete. 2024-09-10T08:39:53.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.391+0000 7f26cfe00640 1 Processor -- start 2024-09-10T08:39:53.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.392+0000 7f26cfe00640 1 -- start start 2024-09-10T08:39:53.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.392+0000 7f26cfe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26d01019c0 0x7f26d00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:53.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.392+0000 7f26cfe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26d0102360 0x7f26d0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:53.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.392+0000 7f26cfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26d01079c0 0x7f26d0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:53.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.392+0000 7f26cfe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f26d0111e10 con 0x7f26d0102360 2024-09-10T08:39:53.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.392+0000 7f26cfe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f26d0111c90 con 0x7f26d01079c0 2024-09-10T08:39:53.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.392+0000 7f26cfe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f26d0111f90 con 0x7f26d01019c0 2024-09-10T08:39:53.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.392+0000 7f26c7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26d0102360 0x7f26d0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:53.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.392+0000 7f26cea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26d01019c0 0x7f26d00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:53.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.392+0000 7f26c7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26d0102360 0x7f26d0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49512/0 (socket says 172.21.15.89:49512) 2024-09-10T08:39:53.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.392+0000 7f26c7e00640 1 -- 172.21.15.89:0/204481252 learned_addr learned my addr 172.21.15.89:0/204481252 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:53.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.393+0000 7f26cf400640 1 --2- 172.21.15.89:0/204481252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26d01079c0 0x7f26d0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:53.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.393+0000 7f26c7e00640 1 -- 172.21.15.89:0/204481252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26d01019c0 msgr2=0x7f26d00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:53.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.393+0000 7f26c7e00640 1 --2- 172.21.15.89:0/204481252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26d01019c0 0x7f26d00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.393+0000 7f26c7e00640 1 -- 172.21.15.89:0/204481252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26d01079c0 msgr2=0x7f26d0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:53.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.393+0000 7f26c7e00640 1 --2- 172.21.15.89:0/204481252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26d01079c0 0x7f26d0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.393+0000 7f26c7e00640 1 -- 172.21.15.89:0/204481252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d01aacc0 con 0x7f26d0102360 2024-09-10T08:39:53.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.393+0000 7f26cea00640 1 --2- 172.21.15.89:0/204481252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26d01019c0 0x7f26d00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:53.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.393+0000 7f26c7e00640 1 --2- 172.21.15.89:0/204481252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26d0102360 0x7f26d0074a30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f26b800e9f0 tx=0x7f26b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:53.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.393+0000 7f26c7400640 1 -- 172.21.15.89:0/204481252 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b800cd80 con 0x7f26d0102360 2024-09-10T08:39:53.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.394+0000 7f26c7400640 1 -- 172.21.15.89:0/204481252 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b8005500 con 0x7f26d0102360 2024-09-10T08:39:53.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.394+0000 7f26cfe00640 1 -- 172.21.15.89:0/204481252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d01aaef0 con 0x7f26d0102360 2024-09-10T08:39:53.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.394+0000 7f26cfe00640 1 -- 172.21.15.89:0/204481252 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d01ab2f0 con 0x7f26d0102360 2024-09-10T08:39:53.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.394+0000 7f26c7400640 1 -- 172.21.15.89:0/204481252 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8010690 con 0x7f26d0102360 2024-09-10T08:39:53.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.397+0000 7f26d7fd0740 1 Processor -- start 2024-09-10T08:39:53.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.397+0000 7f26c7400640 1 -- 172.21.15.89:0/204481252 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f26b80046c0 con 0x7f26d0102360 2024-09-10T08:39:53.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.397+0000 7f26d7fd0740 1 -- start start 2024-09-10T08:39:53.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.398+0000 7f26d7fd0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810595d50 0x55e810596130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:53.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.398+0000 7f26d7fd0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810596670 0x55e8105a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:53.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.398+0000 7f26d7fd0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e8105a0f30 0x55e8105a3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:53.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.398+0000 7f26d7fd0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e8104fc3e0 con 0x55e810596670 2024-09-10T08:39:53.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.398+0000 7f26d7fd0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e8104a3d80 con 0x55e8105a0f30 2024-09-10T08:39:53.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.398+0000 7f26d7fd0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e8104f83a0 con 0x55e810595d50 2024-09-10T08:39:53.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.398+0000 7f26cea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810596670 0x55e8105a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:53.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.398+0000 7f26cf400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810595d50 0x55e810596130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:53.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.398+0000 7f26cf400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810595d50 0x55e810596130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:42748/0 (socket says 172.21.15.89:42748) 2024-09-10T08:39:53.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.398+0000 7f26cf400640 1 -- 172.21.15.89:0/3882637121 learned_addr learned my addr 172.21.15.89:0/3882637121 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:53.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.398+0000 7f26c7e00640 1 --2- 172.21.15.89:0/3882637121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e8105a0f30 0x55e8105a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:53.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.399+0000 7f26cf400640 1 -- 172.21.15.89:0/3882637121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e8105a0f30 msgr2=0x55e8105a3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:53.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.399+0000 7f26cf400640 1 --2- 172.21.15.89:0/3882637121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e8105a0f30 0x55e8105a3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.399+0000 7f26cf400640 1 -- 172.21.15.89:0/3882637121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810596670 msgr2=0x55e8105a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:53.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.399+0000 7f26cf400640 1 --2- 172.21.15.89:0/3882637121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810596670 0x55e8105a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.399+0000 7f26cf400640 1 -- 172.21.15.89:0/3882637121 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8104c2ae0 con 0x55e810595d50 2024-09-10T08:39:53.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.399+0000 7f26c7400640 1 --2- 172.21.15.89:0/204481252 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f26b40a2c10 0x7f26b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:53.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.399+0000 7f26c7400640 1 -- 172.21.15.89:0/204481252 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f26b80cfa00 con 0x7f26d0102360 2024-09-10T08:39:53.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.399+0000 7f26cf400640 1 --2- 172.21.15.89:0/3882637121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810595d50 0x55e810596130 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009f50 tx=0x7f26c80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9064d129545796a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:53.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.399+0000 7f26cea00640 1 --2- 172.21.15.89:0/204481252 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f26b40a2c10 0x7f26b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:53.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.399+0000 7f26bf400640 1 -- 172.21.15.89:0/3882637121 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8004d20 con 0x55e810595d50 2024-09-10T08:39:53.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.400+0000 7f26bf400640 1 -- 172.21.15.89:0/3882637121 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c8004050 con 0x55e810595d50 2024-09-10T08:39:53.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.400+0000 7f26bf400640 1 -- 172.21.15.89:0/3882637121 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8005280 con 0x55e810595d50 2024-09-10T08:39:53.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.400+0000 7f26d7fd0740 1 -- 172.21.15.89:0/3882637121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810595d50 msgr2=0x55e810596130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:53.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.400+0000 7f26cea00640 1 --2- 172.21.15.89:0/204481252 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f26b40a2c10 0x7f26b40a50d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f26c0006010 tx=0x7f26c0031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:53.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.400+0000 7f26d7fd0740 1 --2- 172.21.15.89:0/3882637121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810595d50 0x55e810596130 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009f50 tx=0x7f26c80118d0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.400+0000 7f26d7fd0740 1 -- 172.21.15.89:0/3882637121 shutdown_connections 2024-09-10T08:39:53.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.400+0000 7f26d7fd0740 1 --2- 172.21.15.89:0/3882637121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e8105a0f30 0x55e8105a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.400+0000 7f26d7fd0740 1 --2- 172.21.15.89:0/3882637121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810596670 0x55e8105a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.400+0000 7f26d7fd0740 1 --2- 172.21.15.89:0/3882637121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810595d50 0x55e810596130 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.400+0000 7f26d7fd0740 1 -- 172.21.15.89:0/3882637121 >> 172.21.15.89:0/3882637121 conn(0x55e8105922e0 msgr2=0x55e8105926c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:53.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.401+0000 7f26d7fd0740 1 -- 172.21.15.89:0/3882637121 shutdown_connections 2024-09-10T08:39:53.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.401+0000 7f26d7fd0740 1 -- 172.21.15.89:0/3882637121 wait complete. 2024-09-10T08:39:53.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.401+0000 7f26d7fd0740 1 -- start start 2024-09-10T08:39:53.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.402+0000 7f26d7fd0740 1 Processor -- start 2024-09-10T08:39:53.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.402+0000 7f26d7fd0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810595d50 0x55e81059de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:53.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.402+0000 7f26d7fd0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810596670 0x55e81059e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:53.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.402+0000 7f26d7fd0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e8105a0f30 0x55e81059e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:53.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.402+0000 7f26d7fd0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e8105a3ba0 con 0x55e810595d50 2024-09-10T08:39:53.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.402+0000 7f26d7fd0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e8104b70f0 con 0x55e8105a0f30 2024-09-10T08:39:53.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.402+0000 7f26d7fd0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e8105a3d20 con 0x55e810596670 2024-09-10T08:39:53.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.402+0000 7f26d7fd0740 10 client.0 _ll_register_callbacks cb 0x55e8104a89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:39:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.403+0000 7f26cea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810595d50 0x55e81059de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.403+0000 7f26c7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810596670 0x55e81059e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.403+0000 7f26c7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810596670 0x55e81059e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:42750/0 (socket says 172.21.15.89:42750) 2024-09-10T08:39:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.403+0000 7f26c7e00640 1 -- 172.21.15.89:0/1053048575 learned_addr learned my addr 172.21.15.89:0/1053048575 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.403+0000 7f26cf400640 1 --2- 172.21.15.89:0/1053048575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e8105a0f30 0x55e81059e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.403+0000 7f26c7e00640 1 -- 172.21.15.89:0/1053048575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e8105a0f30 msgr2=0x55e81059e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.403+0000 7f26c7e00640 1 --2- 172.21.15.89:0/1053048575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e8105a0f30 0x55e81059e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.403+0000 7f26c7e00640 1 -- 172.21.15.89:0/1053048575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810595d50 msgr2=0x55e81059de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.403+0000 7f26c7e00640 1 --2- 172.21.15.89:0/1053048575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810595d50 0x55e81059de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.403+0000 7f26c7e00640 1 -- 172.21.15.89:0/1053048575 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e81026d3c0 con 0x55e810596670 2024-09-10T08:39:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.403+0000 7f26cea00640 1 --2- 172.21.15.89:0/1053048575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810595d50 0x55e81059de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:39:53.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.403+0000 7f26c7e00640 1 --2- 172.21.15.89:0/1053048575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810596670 0x55e81059e360 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f26b809b9d0 tx=0x7f26b8005730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5125cc3bcc9e395 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.403+0000 7f26b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:39:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.404+0000 7f26b3e00640 1 -- 172.21.15.89:0/1053048575 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b809b630 con 0x55e810596670 2024-09-10T08:39:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.404+0000 7f26b3e00640 1 -- 172.21.15.89:0/1053048575 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f26b80a44b0 con 0x55e810596670 2024-09-10T08:39:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.404+0000 7f26d7fd0740 1 -- 172.21.15.89:0/1053048575 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c8004050 con 0x55e810596670 2024-09-10T08:39:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.404+0000 7f26b3e00640 1 -- 172.21.15.89:0/1053048575 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f26b809c3d0 con 0x55e810596670 2024-09-10T08:39:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.404+0000 7f26d7fd0740 10 client.25246 fetch_fsmap learned FSMap version 26 2024-09-10T08:39:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.404+0000 7f26d7fd0740 1 -- 172.21.15.89:0/1053048575 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e8105af8b0 con 0x55e810596670 2024-09-10T08:39:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.404+0000 7f26b3e00640 1 -- 172.21.15.89:0/1053048575 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c8004050 con 0x55e810596670 2024-09-10T08:39:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.404+0000 7f26b3e00640 1 -- 172.21.15.89:0/1053048575 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f26b809b1b0 con 0x55e810596670 2024-09-10T08:39:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.404+0000 7f26d7fd0740 10 client.25246 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:39:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.404+0000 7f26d7fd0740 10 client.25246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:39:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.405+0000 7f26d7fd0740 20 client.25246 populate_metadata read hostname 'smithi089' 2024-09-10T08:39:53.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.405+0000 7f26d7fd0740 1 --2- 172.21.15.89:0/1053048575 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e8105b0d70 0x55e8105b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.405+0000 7f26d7fd0740 4 client.25246 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:39:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.405+0000 7f26d7fd0740 1 -- 172.21.15.89:0/1053048575 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e810509950 con 0x55e8105b0d70 2024-09-10T08:39:53.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.405+0000 7f26cea00640 1 --2- 172.21.15.89:0/1053048575 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e8105b0d70 0x55e8105b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:53.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.406+0000 7f26cea00640 1 --2- 172.21.15.89:0/1053048575 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e8105b0d70 0x55e8105b31a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e972c173b42dc77b server_cookie=c8a2677819e7aefa in_seq=0 out_seq=0 2024-09-10T08:39:53.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.406+0000 7f26b3e00640 10 client.25246 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:39:53.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.406+0000 7f26b3e00640 1 -- 172.21.15.89:0/1053048575 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e810509950 con 0x55e8105b0d70 2024-09-10T08:39:53.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.406+0000 7f26b3e00640 10 client.25246 handle_command_reply: tid=0 2024-09-10T08:39:53.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.406+0000 7f26d7fd0740 1 client.25246 shutdown 2024-09-10T08:39:53.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.407+0000 7f26d7fd0740 1 -- 172.21.15.89:0/1053048575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810596670 msgr2=0x55e81059e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:53.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.407+0000 7f26d7fd0740 1 --2- 172.21.15.89:0/1053048575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810596670 0x55e81059e360 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f26b809b9d0 tx=0x7f26b8005730 comp rx=0 tx=0).stop 2024-09-10T08:39:53.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.407+0000 7f26d7fd0740 1 -- 172.21.15.89:0/1053048575 shutdown_connections 2024-09-10T08:39:53.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.407+0000 7f26d7fd0740 1 --2- 172.21.15.89:0/1053048575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e8105a0f30 0x55e81059e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.407+0000 7f26d7fd0740 1 --2- 172.21.15.89:0/1053048575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810596670 0x55e81059e360 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.407+0000 7f26d7fd0740 1 --2- 172.21.15.89:0/1053048575 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e8105b0d70 0x55e8105b31a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.407+0000 7f26d7fd0740 1 --2- 172.21.15.89:0/1053048575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810595d50 0x55e81059de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.407+0000 7f26d7fd0740 1 -- 172.21.15.89:0/1053048575 >> 172.21.15.89:0/1053048575 conn(0x55e8105922e0 msgr2=0x55e810598010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:53.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.407+0000 7f26b3e00640 0 client.25246 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:39:53.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.407+0000 7f26d7fd0740 1 -- 172.21.15.89:0/1053048575 shutdown_connections 2024-09-10T08:39:53.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.407+0000 7f26d7fd0740 1 -- 172.21.15.89:0/1053048575 wait complete. 2024-09-10T08:39:53.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.407+0000 7f26d7fd0740 20 client.25246 trim_cache size 0 max 16384 2024-09-10T08:39:53.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.426+0000 7f26d7fd0740 1 Processor -- start 2024-09-10T08:39:53.425 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.426+0000 7f26d7fd0740 1 -- start start 2024-09-10T08:39:53.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.426+0000 7f26d7fd0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810595d50 0x55e8105987b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:53.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.426+0000 7f26d7fd0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810596670 0x55e8105b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:53.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.426+0000 7f26d7fd0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e8105a0f30 0x55e8105b8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:53.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.426+0000 7f26d7fd0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f26d01b7b80 con 0x55e810595d50 2024-09-10T08:39:53.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.426+0000 7f26d7fd0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f26d01b5450 con 0x55e8105a0f30 2024-09-10T08:39:53.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.426+0000 7f26d7fd0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f26d01b32a0 con 0x55e810596670 2024-09-10T08:39:53.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.426+0000 7f26cea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810595d50 0x55e8105987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:53.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.427+0000 7f26c7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810596670 0x55e8105b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:53.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.427+0000 7f26cea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810595d50 0x55e8105987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49538/0 (socket says 172.21.15.89:49538) 2024-09-10T08:39:53.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.427+0000 7f26cea00640 1 -- 172.21.15.89:0/2767429901 learned_addr learned my addr 172.21.15.89:0/2767429901 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:53.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.427+0000 7f26cf400640 1 --2- 172.21.15.89:0/2767429901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e8105a0f30 0x55e8105b8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:53.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.427+0000 7f26c7e00640 1 -- 172.21.15.89:0/2767429901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e8105a0f30 msgr2=0x55e8105b8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:53.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.427+0000 7f26c7e00640 1 --2- 172.21.15.89:0/2767429901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e8105a0f30 0x55e8105b8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.427+0000 7f26c7e00640 1 -- 172.21.15.89:0/2767429901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810595d50 msgr2=0x55e8105987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:53.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.427+0000 7f26c7e00640 1 --2- 172.21.15.89:0/2767429901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810595d50 0x55e8105987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.427+0000 7f26c7e00640 1 -- 172.21.15.89:0/2767429901 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e810596be0 con 0x55e810596670 2024-09-10T08:39:53.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.427+0000 7f26cea00640 1 --2- 172.21.15.89:0/2767429901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810595d50 0x55e8105987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:53.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.427+0000 7f26c7e00640 1 --2- 172.21.15.89:0/2767429901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810596670 0x55e8105b8a70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f26b800c6a0 tx=0x7f26b809b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4db082e44801d831 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:53.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.428+0000 7f26bea00640 1 -- 172.21.15.89:0/2767429901 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b809b630 con 0x55e810596670 2024-09-10T08:39:53.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.428+0000 7f26bea00640 1 -- 172.21.15.89:0/2767429901 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b809ac30 con 0x55e810596670 2024-09-10T08:39:53.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.428+0000 7f26bea00640 1 -- 172.21.15.89:0/2767429901 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b809b120 con 0x55e810596670 2024-09-10T08:39:53.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.429+0000 7f26d7fd0740 1 -- 172.21.15.89:0/2767429901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810596670 msgr2=0x55e8105b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:53.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.429+0000 7f26d7fd0740 1 --2- 172.21.15.89:0/2767429901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810596670 0x55e8105b8a70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f26b800c6a0 tx=0x7f26b809b8d0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.429+0000 7f26d7fd0740 1 -- 172.21.15.89:0/2767429901 shutdown_connections 2024-09-10T08:39:53.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.429+0000 7f26d7fd0740 1 --2- 172.21.15.89:0/2767429901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e8105a0f30 0x55e8105b8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.429+0000 7f26d7fd0740 1 --2- 172.21.15.89:0/2767429901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810596670 0x55e8105b8a70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.429+0000 7f26d7fd0740 1 --2- 172.21.15.89:0/2767429901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810595d50 0x55e8105987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.429+0000 7f26d7fd0740 1 -- 172.21.15.89:0/2767429901 >> 172.21.15.89:0/2767429901 conn(0x55e8105922e0 msgr2=0x55e81059c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:53.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.429+0000 7f26d7fd0740 1 -- 172.21.15.89:0/2767429901 shutdown_connections 2024-09-10T08:39:53.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.429+0000 7f26d7fd0740 1 -- 172.21.15.89:0/2767429901 wait complete. 2024-09-10T08:39:53.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.429+0000 7f26d7fd0740 1 -- start start 2024-09-10T08:39:53.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.429+0000 7f26d7fd0740 1 Processor -- start 2024-09-10T08:39:53.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.430+0000 7f26d7fd0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810595d50 0x55e8105a5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:53.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.430+0000 7f26d7fd0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e810596670 0x55e8105a5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:53.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.430+0000 7f26d7fd0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e8105a0f30 0x55e81061aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:53.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.430+0000 7f26d7fd0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f26d01b3910 con 0x55e810595d50 2024-09-10T08:39:53.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.430+0000 7f26d7fd0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f26d01b4d80 con 0x55e810596670 2024-09-10T08:39:53.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.430+0000 7f26d7fd0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f26d01b3fe0 con 0x55e8105a0f30 2024-09-10T08:39:53.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.430+0000 7f26d7fd0740 10 client.0 _ll_register_callbacks cb 0x7f26b800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:39:53.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.430+0000 7f26cea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810595d50 0x55e8105a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:53.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.430+0000 7f26cf400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e8105a0f30 0x55e81061aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:53.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.431+0000 7f26cea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810595d50 0x55e8105a5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49552/0 (socket says 172.21.15.89:49552) 2024-09-10T08:39:53.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.431+0000 7f26cea00640 1 -- 172.21.15.89:0/1520497140 learned_addr learned my addr 172.21.15.89:0/1520497140 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:53.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.431+0000 7f26cea00640 1 -- 172.21.15.89:0/1520497140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e8105a0f30 msgr2=0x55e81061aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:53.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.431+0000 7f26cea00640 1 --2- 172.21.15.89:0/1520497140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e8105a0f30 0x55e81061aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.431+0000 7f26cea00640 1 -- 172.21.15.89:0/1520497140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e810596670 msgr2=0x55e8105a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:39:53.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.431+0000 7f26cea00640 1 --2- 172.21.15.89:0/1520497140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e810596670 0x55e8105a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.431+0000 7f26cea00640 1 -- 172.21.15.89:0/1520497140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e810593390 con 0x55e810595d50 2024-09-10T08:39:53.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.431+0000 7f26cea00640 1 --2- 172.21.15.89:0/1520497140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810595d50 0x55e8105a5430 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f26c002a940 tx=0x7f26c0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61efdbec8ce14491 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:53.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.431+0000 7f26b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:39:53.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.432+0000 7f26b3e00640 1 -- 172.21.15.89:0/1520497140 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0057060 con 0x55e810595d50 2024-09-10T08:39:53.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.432+0000 7f26b3e00640 1 -- 172.21.15.89:0/1520497140 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f26c00315b0 con 0x55e810595d50 2024-09-10T08:39:53.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.432+0000 7f26d7fd0740 1 -- 172.21.15.89:0/1520497140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26b809ac30 con 0x55e810595d50 2024-09-10T08:39:53.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.432+0000 7f26b3e00640 1 -- 172.21.15.89:0/1520497140 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f26c005e050 con 0x55e810595d50 2024-09-10T08:39:53.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.432+0000 7f26d7fd0740 10 client.15228 fetch_fsmap learned FSMap version 26 2024-09-10T08:39:53.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.432+0000 7f26d7fd0740 1 -- 172.21.15.89:0/1520497140 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e81061c130 con 0x55e810595d50 2024-09-10T08:39:53.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.432+0000 7f26b3e00640 1 -- 172.21.15.89:0/1520497140 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b809ac30 con 0x55e810595d50 2024-09-10T08:39:53.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.432+0000 7f26b3e00640 1 -- 172.21.15.89:0/1520497140 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f26c0044040 con 0x55e810595d50 2024-09-10T08:39:53.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.433+0000 7f26d7fd0740 10 client.15228 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:39:53.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.433+0000 7f26d7fd0740 10 client.15228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:39:53.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.433+0000 7f26d7fd0740 20 client.15228 populate_metadata read hostname 'smithi089' 2024-09-10T08:39:53.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.433+0000 7f26d7fd0740 1 --2- 172.21.15.89:0/1520497140 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e81061c960 0x55e81061eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:53.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.433+0000 7f26d7fd0740 4 client.15228 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:39:53.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.433+0000 7f26d7fd0740 1 -- 172.21.15.89:0/1520497140 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e810503e60 con 0x55e81061c960 2024-09-10T08:39:53.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.433+0000 7f26c7e00640 1 --2- 172.21.15.89:0/1520497140 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e81061c960 0x55e81061eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:53.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.434+0000 7f26c7e00640 1 --2- 172.21.15.89:0/1520497140 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e81061c960 0x55e81061eda0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=709ed546550e75a5 server_cookie=612669af3d30c57b in_seq=0 out_seq=0 2024-09-10T08:39:53.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.434+0000 7f26b3e00640 10 client.15228 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:39:53.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.434+0000 7f26b3e00640 1 -- 172.21.15.89:0/1520497140 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e810503e60 con 0x55e81061c960 2024-09-10T08:39:53.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.434+0000 7f26b3e00640 10 client.15228 handle_command_reply: tid=0 2024-09-10T08:39:53.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.434+0000 7f26d7fd0740 1 client.15228 shutdown 2024-09-10T08:39:53.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.434+0000 7f26d7fd0740 1 -- 172.21.15.89:0/1520497140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810595d50 msgr2=0x55e8105a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:53.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.434+0000 7f26d7fd0740 1 --2- 172.21.15.89:0/1520497140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810595d50 0x55e8105a5430 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f26c002a940 tx=0x7f26c0009260 comp rx=0 tx=0).stop 2024-09-10T08:39:53.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.434+0000 7f26d7fd0740 1 -- 172.21.15.89:0/1520497140 shutdown_connections 2024-09-10T08:39:53.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.434+0000 7f26d7fd0740 1 --2- 172.21.15.89:0/1520497140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e8105a0f30 0x55e81061aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.434+0000 7f26d7fd0740 1 --2- 172.21.15.89:0/1520497140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e810596670 0x55e8105a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.434+0000 7f26d7fd0740 1 --2- 172.21.15.89:0/1520497140 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e81061c960 0x55e81061eda0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.434+0000 7f26d7fd0740 1 --2- 172.21.15.89:0/1520497140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e810595d50 0x55e8105a5430 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.434+0000 7f26b3e00640 0 client.15228 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:39:53.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.434+0000 7f26d7fd0740 1 -- 172.21.15.89:0/1520497140 >> 172.21.15.89:0/1520497140 conn(0x55e8105922e0 msgr2=0x55e81059dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:53.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.434+0000 7f26d7fd0740 1 -- 172.21.15.89:0/1520497140 shutdown_connections 2024-09-10T08:39:53.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.434+0000 7f26d7fd0740 1 -- 172.21.15.89:0/1520497140 wait complete. 2024-09-10T08:39:53.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.434+0000 7f26d7fd0740 20 client.15228 trim_cache size 0 max 16384 2024-09-10T08:39:53.438 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:39:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:39:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2024-09-10T08:39:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: "d4cc1db6-0abb-4e16-a0c7-f28265fb2d0a": { 2024-09-10T08:39:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2024-09-10T08:39:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "d4cc1db6-0abb-4e16-a0c7-f28265fb2d0a", 2024-09-10T08:39:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2024-09-10T08:39:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:39:53.438 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:39:53.438 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:39:53.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.435+0000 7f26cfe00640 1 -- 172.21.15.89:0/204481252 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f26b40a2c10 msgr2=0x7f26b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:53.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.435+0000 7f26cfe00640 1 --2- 172.21.15.89:0/204481252 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f26b40a2c10 0x7f26b40a50d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f26c0006010 tx=0x7f26c0031510 comp rx=0 tx=0).stop 2024-09-10T08:39:53.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.435+0000 7f26cfe00640 1 -- 172.21.15.89:0/204481252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26d0102360 msgr2=0x7f26d0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:53.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.435+0000 7f26cfe00640 1 --2- 172.21.15.89:0/204481252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26d0102360 0x7f26d0074a30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f26b800e9f0 tx=0x7f26b800edf0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.435+0000 7f26cfe00640 1 -- 172.21.15.89:0/204481252 shutdown_connections 2024-09-10T08:39:53.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.435+0000 7f26cfe00640 1 --2- 172.21.15.89:0/204481252 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f26b40a2c10 0x7f26b40a50d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.435+0000 7f26cfe00640 1 --2- 172.21.15.89:0/204481252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26d01079c0 0x7f26d0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.435+0000 7f26cfe00640 1 --2- 172.21.15.89:0/204481252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26d0102360 0x7f26d0074a30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.435+0000 7f26cfe00640 1 --2- 172.21.15.89:0/204481252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26d01019c0 0x7f26d00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:53.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.435+0000 7f26cfe00640 1 -- 172.21.15.89:0/204481252 >> 172.21.15.89:0/204481252 conn(0x7f26d00fd740 msgr2=0x7f26d00fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:53.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.436+0000 7f26cfe00640 1 -- 172.21.15.89:0/204481252 shutdown_connections 2024-09-10T08:39:53.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:53.436+0000 7f26cfe00640 1 -- 172.21.15.89:0/204481252 wait complete. 2024-09-10T08:39:54.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:53 smithi111 ceph-mon[35220]: from='client.25265 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:39:54.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:53 smithi089 ceph-mon[30757]: from='client.25265 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:39:54.178 INFO:tasks.cephfs.filesystem:scrub status for tag:d4cc1db6-0abb-4e16-a0c7-f28265fb2d0a - {'path': '/', 'tag': 'd4cc1db6-0abb-4e16-a0c7-f28265fb2d0a', 'options': 'recursive,force'} 2024-09-10T08:39:54.191 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:54.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:53 smithi149 ceph-mon[36167]: from='client.25265 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:39:54.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.754+0000 7f3f56e00640 1 -- 172.21.15.89:0/1984504314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f50112870 msgr2=0x7f3f50114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:54.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.754+0000 7f3f56e00640 1 --2- 172.21.15.89:0/1984504314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f50112870 0x7f3f50114c60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009ca0 tx=0x7f3f4802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:39:54.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.754+0000 7f3f56e00640 1 -- 172.21.15.89:0/1984504314 shutdown_connections 2024-09-10T08:39:54.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.754+0000 7f3f56e00640 1 --2- 172.21.15.89:0/1984504314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f50112870 0x7f3f50114c60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:54.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.754+0000 7f3f56e00640 1 --2- 172.21.15.89:0/1984504314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f50077040 0x7f3f50112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:54.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.754+0000 7f3f56e00640 1 --2- 172.21.15.89:0/1984504314 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f50076720 0x7f3f50076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:54.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.754+0000 7f3f56e00640 1 -- 172.21.15.89:0/1984504314 >> 172.21.15.89:0/1984504314 conn(0x7f3f500ff8d0 msgr2=0x7f3f50101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:54.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.754+0000 7f3f56e00640 1 -- 172.21.15.89:0/1984504314 shutdown_connections 2024-09-10T08:39:54.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.754+0000 7f3f56e00640 1 -- 172.21.15.89:0/1984504314 wait complete. 2024-09-10T08:39:54.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.755+0000 7f3f56e00640 1 Processor -- start 2024-09-10T08:39:54.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.756+0000 7f3f56e00640 1 -- start start 2024-09-10T08:39:54.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.756+0000 7f3f56e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f50076720 0x7f3f501a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:54.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.756+0000 7f3f56e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f50077040 0x7f3f501a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:54.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.756+0000 7f3f56e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f50112870 0x7f3f5019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:54.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.756+0000 7f3f56e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3f50118420 con 0x7f3f50077040 2024-09-10T08:39:54.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.756+0000 7f3f56e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3f501182a0 con 0x7f3f50112870 2024-09-10T08:39:54.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.756+0000 7f3f56e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3f501185a0 con 0x7f3f50076720 2024-09-10T08:39:54.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.756+0000 7f3f55000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f50077040 0x7f3f501a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:54.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.756+0000 7f3f55000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f50077040 0x7f3f501a4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49568/0 (socket says 172.21.15.89:49568) 2024-09-10T08:39:54.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.756+0000 7f3f55000640 1 -- 172.21.15.89:0/618832061 learned_addr learned my addr 172.21.15.89:0/618832061 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:54.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.756+0000 7f3f55a00640 1 --2- 172.21.15.89:0/618832061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f50076720 0x7f3f501a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:54.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.756+0000 7f3f56400640 1 --2- 172.21.15.89:0/618832061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f50112870 0x7f3f5019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:54.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.757+0000 7f3f55000640 1 -- 172.21.15.89:0/618832061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f50076720 msgr2=0x7f3f501a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:54.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.757+0000 7f3f55000640 1 --2- 172.21.15.89:0/618832061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f50076720 0x7f3f501a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:54.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.757+0000 7f3f55000640 1 -- 172.21.15.89:0/618832061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f50112870 msgr2=0x7f3f5019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:54.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.757+0000 7f3f55000640 1 --2- 172.21.15.89:0/618832061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f50112870 0x7f3f5019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:54.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.757+0000 7f3f55000640 1 -- 172.21.15.89:0/618832061 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5019ef40 con 0x7f3f50077040 2024-09-10T08:39:54.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.757+0000 7f3f56400640 1 --2- 172.21.15.89:0/618832061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f50112870 0x7f3f5019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:54.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.757+0000 7f3f55000640 1 --2- 172.21.15.89:0/618832061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f50077040 0x7f3f501a4b40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00e9f0 tx=0x7f3f3c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:54.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.757+0000 7f3f4e000640 1 -- 172.21.15.89:0/618832061 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c00cd80 con 0x7f3f50077040 2024-09-10T08:39:54.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.757+0000 7f3f56e00640 1 -- 172.21.15.89:0/618832061 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5019f230 con 0x7f3f50077040 2024-09-10T08:39:54.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.757+0000 7f3f4e000640 1 -- 172.21.15.89:0/618832061 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f3c005500 con 0x7f3f50077040 2024-09-10T08:39:54.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.757+0000 7f3f4e000640 1 -- 172.21.15.89:0/618832061 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c010690 con 0x7f3f50077040 2024-09-10T08:39:54.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.758+0000 7f3f56e00640 1 -- 172.21.15.89:0/618832061 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f501ab580 con 0x7f3f50077040 2024-09-10T08:39:54.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.759+0000 7f3f56e00640 1 -- 172.21.15.89:0/618832061 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f50077e60 con 0x7f3f50077040 2024-09-10T08:39:54.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.761+0000 7f3f4e000640 1 -- 172.21.15.89:0/618832061 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f3f3c0046c0 con 0x7f3f50077040 2024-09-10T08:39:54.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.762+0000 7f3f4e000640 1 --2- 172.21.15.89:0/618832061 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3f380a2c10 0x7f3f380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:54.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.762+0000 7f3f4e000640 1 -- 172.21.15.89:0/618832061 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f3f3c0cf9a0 con 0x7f3f50077040 2024-09-10T08:39:54.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.762+0000 7f3f55a00640 1 --2- 172.21.15.89:0/618832061 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3f380a2c10 0x7f3f380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:54.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.763+0000 7f3f55a00640 1 --2- 172.21.15.89:0/618832061 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3f380a2c10 0x7f3f380a50d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009bd0 tx=0x7f3f44005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:54.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.766+0000 7f3f4e000640 1 -- 172.21.15.89:0/618832061 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f3c09c370 con 0x7f3f50077040 2024-09-10T08:39:54.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.938+0000 7f3f56e00640 1 -- 172.21.15.89:0/618832061 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3f50062630 con 0x7f3f380a2c10 2024-09-10T08:39:54.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.946+0000 7f3f4e000640 1 -- 172.21.15.89:0/618832061 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340805 (secure 0 0 0) 0x7f3f50062630 con 0x7f3f380a2c10 2024-09-10T08:39:54.951 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:39:54.952 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-09-10T08:39:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.951+0000 7f3f56e00640 1 -- 172.21.15.89:0/618832061 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3f380a2c10 msgr2=0x7f3f380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.951+0000 7f3f56e00640 1 --2- 172.21.15.89:0/618832061 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3f380a2c10 0x7f3f380a50d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009bd0 tx=0x7f3f44005f90 comp rx=0 tx=0).stop 2024-09-10T08:39:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.951+0000 7f3f56e00640 1 -- 172.21.15.89:0/618832061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f50077040 msgr2=0x7f3f501a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.951+0000 7f3f56e00640 1 --2- 172.21.15.89:0/618832061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f50077040 0x7f3f501a4b40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00e9f0 tx=0x7f3f3c00edf0 comp rx=0 tx=0).stop 2024-09-10T08:39:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.951+0000 7f3f56e00640 1 -- 172.21.15.89:0/618832061 shutdown_connections 2024-09-10T08:39:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.951+0000 7f3f56e00640 1 --2- 172.21.15.89:0/618832061 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3f380a2c10 0x7f3f380a50d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.951+0000 7f3f56e00640 1 --2- 172.21.15.89:0/618832061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f50112870 0x7f3f5019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.951+0000 7f3f56e00640 1 --2- 172.21.15.89:0/618832061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f50077040 0x7f3f501a4b40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.951+0000 7f3f56e00640 1 --2- 172.21.15.89:0/618832061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f50076720 0x7f3f501a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.951+0000 7f3f56e00640 1 -- 172.21.15.89:0/618832061 >> 172.21.15.89:0/618832061 conn(0x7f3f500ff8d0 msgr2=0x7f3f50101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.951+0000 7f3f56e00640 1 -- 172.21.15.89:0/618832061 shutdown_connections 2024-09-10T08:39:54.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:54.951+0000 7f3f56e00640 1 -- 172.21.15.89:0/618832061 wait complete. 2024-09-10T08:39:55.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:54 smithi111 ceph-mon[35220]: pgmap v392: 129 pgs: 129 active+clean; 652 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-10T08:39:55.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:54 smithi089 ceph-mon[30757]: pgmap v392: 129 pgs: 129 active+clean; 652 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-10T08:39:55.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:54 smithi149 ceph-mon[36167]: pgmap v392: 129 pgs: 129 active+clean; 652 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-10T08:39:55.892 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-09-10T08:39:53.615475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667141,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":890,"num_read_kb":572,"num_write":371,"num_write_kb":3080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181900,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":799,"ondisk_log_size":799,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357984,"kb_used_data":12896,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583840,"statfs":{"total":1151940427776,"available":1151573852160,"internally_reserved":0,"allocated":13205504,"data_stored":8546403,"data_compressed":467733,"data_compressed_allocated":638976,"data_compressed_original":2777088,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7227,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":41,"num_write":2,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003429"},"pg_stats":[{"pgid":"2.1f","version":"94'9","reported_seq":68,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:00.658898+0000","last_change":"2024-09-10T08:38:43.663079+0000","last_active":"2024-09-10T08:39:00.658898+0000","last_peered":"2024-09-10T08:39:00.658898+0000","last_clean":"2024-09-10T08:39:00.658898+0000","last_became_active":"2024-09-10T08:38:43.656506+0000","last_became_peered":"2024-09-10T08:38:43.656506+0000","last_unstale":"2024-09-10T08:39:00.658898+0000","last_undegraded":"2024-09-10T08:39:00.658898+0000","last_fullsized":"2024-09-10T08:39:00.658898+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:36.881564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.579344+0000","last_change":"2024-09-10T08:36:56.145816+0000","last_active":"2024-09-10T08:39:11.579344+0000","last_peered":"2024-09-10T08:39:11.579344+0000","last_clean":"2024-09-10T08:39:11.579344+0000","last_became_active":"2024-09-10T08:36:56.145462+0000","last_became_peered":"2024-09-10T08:36:56.145462+0000","last_unstale":"2024-09-10T08:39:11.579344+0000","last_undegraded":"2024-09-10T08:39:11.579344+0000","last_fullsized":"2024-09-10T08:39:11.579344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:53.890190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629879+0000","last_change":"2024-09-10T08:37:05.252761+0000","last_active":"2024-09-10T08:38:43.629879+0000","last_peered":"2024-09-10T08:38:43.629879+0000","last_clean":"2024-09-10T08:38:43.629879+0000","last_became_active":"2024-09-10T08:37:05.252179+0000","last_became_peered":"2024-09-10T08:37:05.252179+0000","last_unstale":"2024-09-10T08:38:43.629879+0000","last_undegraded":"2024-09-10T08:38:43.629879+0000","last_fullsized":"2024-09-10T08:38:43.629879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:52.854828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631850+0000","last_change":"2024-09-10T08:36:53.735650+0000","last_active":"2024-09-10T08:38:43.631850+0000","last_peered":"2024-09-10T08:38:43.631850+0000","last_clean":"2024-09-10T08:38:43.631850+0000","last_became_active":"2024-09-10T08:36:53.735128+0000","last_became_peered":"2024-09-10T08:36:53.735128+0000","last_unstale":"2024-09-10T08:38:43.631850+0000","last_undegraded":"2024-09-10T08:38:43.631850+0000","last_fullsized":"2024-09-10T08:38:43.631850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:27.309591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592923+0000","last_change":"2024-09-10T08:36:56.153364+0000","last_active":"2024-09-10T08:39:11.592923+0000","last_peered":"2024-09-10T08:39:11.592923+0000","last_clean":"2024-09-10T08:39:11.592923+0000","last_became_active":"2024-09-10T08:36:56.152979+0000","last_became_peered":"2024-09-10T08:36:56.152979+0000","last_unstale":"2024-09-10T08:39:11.592923+0000","last_undegraded":"2024-09-10T08:39:11.592923+0000","last_fullsized":"2024-09-10T08:39:11.592923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:12.172620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654774+0000","last_change":"2024-09-10T08:37:05.254551+0000","last_active":"2024-09-10T08:38:43.654774+0000","last_peered":"2024-09-10T08:38:43.654774+0000","last_clean":"2024-09-10T08:38:43.654774+0000","last_became_active":"2024-09-10T08:37:05.254055+0000","last_became_peered":"2024-09-10T08:37:05.254055+0000","last_unstale":"2024-09-10T08:38:43.654774+0000","last_undegraded":"2024-09-10T08:38:43.654774+0000","last_fullsized":"2024-09-10T08:38:43.654774+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:08.472502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626522+0000","last_change":"2024-09-10T08:36:53.722774+0000","last_active":"2024-09-10T08:38:43.626522+0000","last_peered":"2024-09-10T08:38:43.626522+0000","last_clean":"2024-09-10T08:38:43.626522+0000","last_became_active":"2024-09-10T08:36:53.722459+0000","last_became_peered":"2024-09-10T08:36:53.722459+0000","last_unstale":"2024-09-10T08:38:43.626522+0000","last_undegraded":"2024-09-10T08:38:43.626522+0000","last_fullsized":"2024-09-10T08:38:43.626522+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:26.612354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592240+0000","last_change":"2024-09-10T08:36:56.143757+0000","last_active":"2024-09-10T08:39:11.592240+0000","last_peered":"2024-09-10T08:39:11.592240+0000","last_clean":"2024-09-10T08:39:11.592240+0000","last_became_active":"2024-09-10T08:36:56.143430+0000","last_became_peered":"2024-09-10T08:36:56.143430+0000","last_unstale":"2024-09-10T08:39:11.592240+0000","last_undegraded":"2024-09-10T08:39:11.592240+0000","last_fullsized":"2024-09-10T08:39:11.592240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:49.737298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655120+0000","last_change":"2024-09-10T08:37:05.254503+0000","last_active":"2024-09-10T08:38:43.655120+0000","last_peered":"2024-09-10T08:38:43.655120+0000","last_clean":"2024-09-10T08:38:43.655120+0000","last_became_active":"2024-09-10T08:37:05.254070+0000","last_became_peered":"2024-09-10T08:37:05.254070+0000","last_unstale":"2024-09-10T08:38:43.655120+0000","last_undegraded":"2024-09-10T08:38:43.655120+0000","last_fullsized":"2024-09-10T08:38:43.655120+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:16.741698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626438+0000","last_change":"2024-09-10T08:36:53.716742+0000","last_active":"2024-09-10T08:38:43.626438+0000","last_peered":"2024-09-10T08:38:43.626438+0000","last_clean":"2024-09-10T08:38:43.626438+0000","last_became_active":"2024-09-10T08:36:53.716439+0000","last_became_peered":"2024-09-10T08:36:53.716439+0000","last_unstale":"2024-09-10T08:38:43.626438+0000","last_undegraded":"2024-09-10T08:38:43.626438+0000","last_fullsized":"2024-09-10T08:38:43.626438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:46.882805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.581828+0000","last_change":"2024-09-10T08:36:56.146841+0000","last_active":"2024-09-10T08:39:11.581828+0000","last_peered":"2024-09-10T08:39:11.581828+0000","last_clean":"2024-09-10T08:39:11.581828+0000","last_became_active":"2024-09-10T08:36:56.146531+0000","last_became_peered":"2024-09-10T08:36:56.146531+0000","last_unstale":"2024-09-10T08:39:11.581828+0000","last_undegraded":"2024-09-10T08:39:11.581828+0000","last_fullsized":"2024-09-10T08:39:11.581828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:37.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654877+0000","last_change":"2024-09-10T08:37:43.223181+0000","last_active":"2024-09-10T08:38:43.654877+0000","last_peered":"2024-09-10T08:38:43.654877+0000","last_clean":"2024-09-10T08:38:43.654877+0000","last_became_active":"2024-09-10T08:37:43.222794+0000","last_became_peered":"2024-09-10T08:37:43.222794+0000","last_unstale":"2024-09-10T08:38:43.654877+0000","last_undegraded":"2024-09-10T08:38:43.654877+0000","last_fullsized":"2024-09-10T08:38:43.654877+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:22.204517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593593+0000","last_change":"2024-09-10T08:36:56.142161+0000","last_active":"2024-09-10T08:39:11.593593+0000","last_peered":"2024-09-10T08:39:11.593593+0000","last_clean":"2024-09-10T08:39:11.593593+0000","last_became_active":"2024-09-10T08:36:56.141838+0000","last_became_peered":"2024-09-10T08:36:56.141838+0000","last_unstale":"2024-09-10T08:39:11.593593+0000","last_undegraded":"2024-09-10T08:39:11.593593+0000","last_fullsized":"2024-09-10T08:39:11.593593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:09.909747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626125+0000","last_change":"2024-09-10T08:36:53.722238+0000","last_active":"2024-09-10T08:38:43.626125+0000","last_peered":"2024-09-10T08:38:43.626125+0000","last_clean":"2024-09-10T08:38:43.626125+0000","last_became_active":"2024-09-10T08:36:53.721859+0000","last_became_peered":"2024-09-10T08:36:53.721859+0000","last_unstale":"2024-09-10T08:38:43.626125+0000","last_undegraded":"2024-09-10T08:38:43.626125+0000","last_fullsized":"2024-09-10T08:38:43.626125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:50.615090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655183+0000","last_change":"2024-09-10T08:37:05.414041+0000","last_active":"2024-09-10T08:38:43.655183+0000","last_peered":"2024-09-10T08:38:43.655183+0000","last_clean":"2024-09-10T08:38:43.655183+0000","last_became_active":"2024-09-10T08:37:05.413286+0000","last_became_peered":"2024-09-10T08:37:05.413286+0000","last_unstale":"2024-09-10T08:38:43.655183+0000","last_undegraded":"2024-09-10T08:38:43.655183+0000","last_fullsized":"2024-09-10T08:38:43.655183+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:44.916102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.572936+0000","last_change":"2024-09-10T08:36:56.158278+0000","last_active":"2024-09-10T08:39:11.572936+0000","last_peered":"2024-09-10T08:39:11.572936+0000","last_clean":"2024-09-10T08:39:11.572936+0000","last_became_active":"2024-09-10T08:36:56.157946+0000","last_became_peered":"2024-09-10T08:36:56.157946+0000","last_unstale":"2024-09-10T08:39:11.572936+0000","last_undegraded":"2024-09-10T08:39:11.572936+0000","last_fullsized":"2024-09-10T08:39:11.572936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:42.512513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"96'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:38.713339+0000","last_change":"2024-09-10T08:38:43.635077+0000","last_active":"2024-09-10T08:39:38.713339+0000","last_peered":"2024-09-10T08:39:38.713339+0000","last_clean":"2024-09-10T08:39:38.713339+0000","last_became_active":"2024-09-10T08:38:43.631957+0000","last_became_peered":"2024-09-10T08:38:43.631957+0000","last_unstale":"2024-09-10T08:39:38.713339+0000","last_undegraded":"2024-09-10T08:39:38.713339+0000","last_fullsized":"2024-09-10T08:39:38.713339+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:50.837247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655126+0000","last_change":"2024-09-10T08:37:42.671399+0000","last_active":"2024-09-10T08:38:43.655126+0000","last_peered":"2024-09-10T08:38:43.655126+0000","last_clean":"2024-09-10T08:38:43.655126+0000","last_became_active":"2024-09-10T08:37:42.670865+0000","last_became_peered":"2024-09-10T08:37:42.670865+0000","last_unstale":"2024-09-10T08:38:43.655126+0000","last_undegraded":"2024-09-10T08:38:43.655126+0000","last_fullsized":"2024-09-10T08:38:43.655126+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:16.231538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.577960+0000","last_change":"2024-09-10T08:36:56.144856+0000","last_active":"2024-09-10T08:39:11.577960+0000","last_peered":"2024-09-10T08:39:11.577960+0000","last_clean":"2024-09-10T08:39:11.577960+0000","last_became_active":"2024-09-10T08:36:56.144531+0000","last_became_peered":"2024-09-10T08:36:56.144531+0000","last_unstale":"2024-09-10T08:39:11.577960+0000","last_undegraded":"2024-09-10T08:39:11.577960+0000","last_fullsized":"2024-09-10T08:39:11.577960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:24.586006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631541+0000","last_change":"2024-09-10T08:36:53.721093+0000","last_active":"2024-09-10T08:38:43.631541+0000","last_peered":"2024-09-10T08:38:43.631541+0000","last_clean":"2024-09-10T08:38:43.631541+0000","last_became_active":"2024-09-10T08:36:53.720538+0000","last_became_peered":"2024-09-10T08:36:53.720538+0000","last_unstale":"2024-09-10T08:38:43.631541+0000","last_undegraded":"2024-09-10T08:38:43.631541+0000","last_fullsized":"2024-09-10T08:38:43.631541+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:58.492463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629902+0000","last_change":"2024-09-10T08:37:05.254011+0000","last_active":"2024-09-10T08:38:43.629902+0000","last_peered":"2024-09-10T08:38:43.629902+0000","last_clean":"2024-09-10T08:38:43.629902+0000","last_became_active":"2024-09-10T08:37:05.253066+0000","last_became_peered":"2024-09-10T08:37:05.253066+0000","last_unstale":"2024-09-10T08:38:43.629902+0000","last_undegraded":"2024-09-10T08:38:43.629902+0000","last_fullsized":"2024-09-10T08:38:43.629902+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:16.221302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.553299+0000","last_change":"2024-09-10T08:36:56.141814+0000","last_active":"2024-09-10T08:39:11.553299+0000","last_peered":"2024-09-10T08:39:11.553299+0000","last_clean":"2024-09-10T08:39:11.553299+0000","last_became_active":"2024-09-10T08:36:56.141480+0000","last_became_peered":"2024-09-10T08:36:56.141480+0000","last_unstale":"2024-09-10T08:39:11.553299+0000","last_undegraded":"2024-09-10T08:39:11.553299+0000","last_fullsized":"2024-09-10T08:39:11.553299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:40.594354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.636347+0000","last_change":"2024-09-10T08:38:43.635917+0000","last_active":"2024-09-10T08:38:43.636347+0000","last_peered":"2024-09-10T08:38:43.636347+0000","last_clean":"2024-09-10T08:38:43.636347+0000","last_became_active":"2024-09-10T08:38:43.633406+0000","last_became_peered":"2024-09-10T08:38:43.633406+0000","last_unstale":"2024-09-10T08:38:43.636347+0000","last_undegraded":"2024-09-10T08:38:43.636347+0000","last_fullsized":"2024-09-10T08:38:43.636347+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:44.619072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655297+0000","last_change":"2024-09-10T08:37:05.414181+0000","last_active":"2024-09-10T08:38:43.655297+0000","last_peered":"2024-09-10T08:38:43.655297+0000","last_clean":"2024-09-10T08:38:43.655297+0000","last_became_active":"2024-09-10T08:37:05.413691+0000","last_became_peered":"2024-09-10T08:37:05.413691+0000","last_unstale":"2024-09-10T08:38:43.655297+0000","last_undegraded":"2024-09-10T08:38:43.655297+0000","last_fullsized":"2024-09-10T08:38:43.655297+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:30.744949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584567+0000","last_change":"2024-09-10T08:36:56.153956+0000","last_active":"2024-09-10T08:39:11.584567+0000","last_peered":"2024-09-10T08:39:11.584567+0000","last_clean":"2024-09-10T08:39:11.584567+0000","last_became_active":"2024-09-10T08:36:56.153609+0000","last_became_peered":"2024-09-10T08:36:56.153609+0000","last_unstale":"2024-09-10T08:39:11.584567+0000","last_undegraded":"2024-09-10T08:39:11.584567+0000","last_fullsized":"2024-09-10T08:39:11.584567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:18.185704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655520+0000","last_change":"2024-09-10T08:36:53.733217+0000","last_active":"2024-09-10T08:38:43.655520+0000","last_peered":"2024-09-10T08:38:43.655520+0000","last_clean":"2024-09-10T08:38:43.655520+0000","last_became_active":"2024-09-10T08:36:53.732851+0000","last_became_peered":"2024-09-10T08:36:53.732851+0000","last_unstale":"2024-09-10T08:38:43.655520+0000","last_undegraded":"2024-09-10T08:38:43.655520+0000","last_fullsized":"2024-09-10T08:38:43.655520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:04:01.132196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629889+0000","last_change":"2024-09-10T08:37:05.241868+0000","last_active":"2024-09-10T08:38:43.629889+0000","last_peered":"2024-09-10T08:38:43.629889+0000","last_clean":"2024-09-10T08:38:43.629889+0000","last_became_active":"2024-09-10T08:37:05.241412+0000","last_became_peered":"2024-09-10T08:37:05.241412+0000","last_unstale":"2024-09-10T08:38:43.629889+0000","last_undegraded":"2024-09-10T08:38:43.629889+0000","last_fullsized":"2024-09-10T08:38:43.629889+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:52.543353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.575701+0000","last_change":"2024-09-10T08:36:56.150722+0000","last_active":"2024-09-10T08:39:11.575701+0000","last_peered":"2024-09-10T08:39:11.575701+0000","last_clean":"2024-09-10T08:39:11.575701+0000","last_became_active":"2024-09-10T08:36:56.150423+0000","last_became_peered":"2024-09-10T08:36:56.150423+0000","last_unstale":"2024-09-10T08:39:11.575701+0000","last_undegraded":"2024-09-10T08:39:11.575701+0000","last_fullsized":"2024-09-10T08:39:11.575701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:50.272841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'13","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:38.175435+0000","last_change":"2024-09-10T08:36:53.722477+0000","last_active":"2024-09-10T08:39:38.175435+0000","last_peered":"2024-09-10T08:39:38.175435+0000","last_clean":"2024-09-10T08:39:38.175435+0000","last_became_active":"2024-09-10T08:36:53.722118+0000","last_became_peered":"2024-09-10T08:36:53.722118+0000","last_unstale":"2024-09-10T08:39:38.175435+0000","last_undegraded":"2024-09-10T08:39:38.175435+0000","last_fullsized":"2024-09-10T08:39:38.175435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:06.631646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630992+0000","last_change":"2024-09-10T08:37:05.415656+0000","last_active":"2024-09-10T08:38:43.630992+0000","last_peered":"2024-09-10T08:38:43.630992+0000","last_clean":"2024-09-10T08:38:43.630992+0000","last_became_active":"2024-09-10T08:37:05.413537+0000","last_became_peered":"2024-09-10T08:37:05.413537+0000","last_unstale":"2024-09-10T08:38:43.630992+0000","last_undegraded":"2024-09-10T08:38:43.630992+0000","last_fullsized":"2024-09-10T08:38:43.630992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:04.284410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585069+0000","last_change":"2024-09-10T08:36:56.144157+0000","last_active":"2024-09-10T08:39:11.585069+0000","last_peered":"2024-09-10T08:39:11.585069+0000","last_clean":"2024-09-10T08:39:11.585069+0000","last_became_active":"2024-09-10T08:36:56.140332+0000","last_became_peered":"2024-09-10T08:36:56.140332+0000","last_unstale":"2024-09-10T08:39:11.585069+0000","last_undegraded":"2024-09-10T08:39:11.585069+0000","last_fullsized":"2024-09-10T08:39:11.585069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:09:26.097987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626517+0000","last_change":"2024-09-10T08:36:53.735465+0000","last_active":"2024-09-10T08:38:43.626517+0000","last_peered":"2024-09-10T08:38:43.626517+0000","last_clean":"2024-09-10T08:38:43.626517+0000","last_became_active":"2024-09-10T08:36:53.735160+0000","last_became_peered":"2024-09-10T08:36:53.735160+0000","last_unstale":"2024-09-10T08:38:43.626517+0000","last_undegraded":"2024-09-10T08:38:43.626517+0000","last_fullsized":"2024-09-10T08:38:43.626517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:26:52.332085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627959+0000","last_change":"2024-09-10T08:37:05.247671+0000","last_active":"2024-09-10T08:38:43.627959+0000","last_peered":"2024-09-10T08:38:43.627959+0000","last_clean":"2024-09-10T08:38:43.627959+0000","last_became_active":"2024-09-10T08:37:05.247217+0000","last_became_peered":"2024-09-10T08:37:05.247217+0000","last_unstale":"2024-09-10T08:38:43.627959+0000","last_undegraded":"2024-09-10T08:38:43.627959+0000","last_fullsized":"2024-09-10T08:38:43.627959+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:20.881304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.579998+0000","last_change":"2024-09-10T08:36:56.145420+0000","last_active":"2024-09-10T08:39:11.579998+0000","last_peered":"2024-09-10T08:39:11.579998+0000","last_clean":"2024-09-10T08:39:11.579998+0000","last_became_active":"2024-09-10T08:36:56.145074+0000","last_became_peered":"2024-09-10T08:36:56.145074+0000","last_unstale":"2024-09-10T08:39:11.579998+0000","last_undegraded":"2024-09-10T08:39:11.579998+0000","last_fullsized":"2024-09-10T08:39:11.579998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:13.849977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629812+0000","last_change":"2024-09-10T08:36:53.738237+0000","last_active":"2024-09-10T08:38:43.629812+0000","last_peered":"2024-09-10T08:38:43.629812+0000","last_clean":"2024-09-10T08:38:43.629812+0000","last_became_active":"2024-09-10T08:36:53.737858+0000","last_became_peered":"2024-09-10T08:36:53.737858+0000","last_unstale":"2024-09-10T08:38:43.629812+0000","last_undegraded":"2024-09-10T08:38:43.629812+0000","last_fullsized":"2024-09-10T08:38:43.629812+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:35.345661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.625982+0000","last_change":"2024-09-10T08:37:05.413512+0000","last_active":"2024-09-10T08:38:43.625982+0000","last_peered":"2024-09-10T08:38:43.625982+0000","last_clean":"2024-09-10T08:38:43.625982+0000","last_became_active":"2024-09-10T08:37:05.413010+0000","last_became_peered":"2024-09-10T08:37:05.413010+0000","last_unstale":"2024-09-10T08:38:43.625982+0000","last_undegraded":"2024-09-10T08:38:43.625982+0000","last_fullsized":"2024-09-10T08:38:43.625982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:05.349404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583802+0000","last_change":"2024-09-10T08:36:56.157741+0000","last_active":"2024-09-10T08:39:11.583802+0000","last_peered":"2024-09-10T08:39:11.583802+0000","last_clean":"2024-09-10T08:39:11.583802+0000","last_became_active":"2024-09-10T08:36:56.157295+0000","last_became_peered":"2024-09-10T08:36:56.157295+0000","last_unstale":"2024-09-10T08:39:11.583802+0000","last_undegraded":"2024-09-10T08:39:11.583802+0000","last_fullsized":"2024-09-10T08:39:11.583802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:31.845864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631834+0000","last_change":"2024-09-10T08:36:53.722373+0000","last_active":"2024-09-10T08:38:43.631834+0000","last_peered":"2024-09-10T08:38:43.631834+0000","last_clean":"2024-09-10T08:38:43.631834+0000","last_became_active":"2024-09-10T08:36:53.721975+0000","last_became_peered":"2024-09-10T08:36:53.721975+0000","last_unstale":"2024-09-10T08:38:43.631834+0000","last_undegraded":"2024-09-10T08:38:43.631834+0000","last_fullsized":"2024-09-10T08:38:43.631834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:44.593018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628351+0000","last_change":"2024-09-10T08:37:05.255362+0000","last_active":"2024-09-10T08:38:43.628351+0000","last_peered":"2024-09-10T08:38:43.628351+0000","last_clean":"2024-09-10T08:38:43.628351+0000","last_became_active":"2024-09-10T08:37:05.254967+0000","last_became_peered":"2024-09-10T08:37:05.254967+0000","last_unstale":"2024-09-10T08:38:43.628351+0000","last_undegraded":"2024-09-10T08:38:43.628351+0000","last_fullsized":"2024-09-10T08:38:43.628351+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:31:41.850961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.578715+0000","last_change":"2024-09-10T08:36:56.147699+0000","last_active":"2024-09-10T08:39:11.578715+0000","last_peered":"2024-09-10T08:39:11.578715+0000","last_clean":"2024-09-10T08:39:11.578715+0000","last_became_active":"2024-09-10T08:36:56.147349+0000","last_became_peered":"2024-09-10T08:36:56.147349+0000","last_unstale":"2024-09-10T08:39:11.578715+0000","last_undegraded":"2024-09-10T08:39:11.578715+0000","last_fullsized":"2024-09-10T08:39:11.578715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:25.248588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"96'6","reported_seq":169,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:48.903998+0000","last_change":"2024-09-10T08:36:53.736843+0000","last_active":"2024-09-10T08:39:48.903998+0000","last_peered":"2024-09-10T08:39:48.903998+0000","last_clean":"2024-09-10T08:39:48.903998+0000","last_became_active":"2024-09-10T08:36:53.736340+0000","last_became_peered":"2024-09-10T08:36:53.736340+0000","last_unstale":"2024-09-10T08:39:48.903998+0000","last_undegraded":"2024-09-10T08:39:48.903998+0000","last_fullsized":"2024-09-10T08:39:48.903998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:24.679679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628575+0000","last_change":"2024-09-10T08:37:05.414321+0000","last_active":"2024-09-10T08:38:43.628575+0000","last_peered":"2024-09-10T08:38:43.628575+0000","last_clean":"2024-09-10T08:38:43.628575+0000","last_became_active":"2024-09-10T08:37:05.413868+0000","last_became_peered":"2024-09-10T08:37:05.413868+0000","last_unstale":"2024-09-10T08:38:43.628575+0000","last_undegraded":"2024-09-10T08:38:43.628575+0000","last_fullsized":"2024-09-10T08:38:43.628575+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:53.988077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583174+0000","last_change":"2024-09-10T08:36:56.141910+0000","last_active":"2024-09-10T08:39:11.583174+0000","last_peered":"2024-09-10T08:39:11.583174+0000","last_clean":"2024-09-10T08:39:11.583174+0000","last_became_active":"2024-09-10T08:36:56.141579+0000","last_became_peered":"2024-09-10T08:36:56.141579+0000","last_unstale":"2024-09-10T08:39:11.583174+0000","last_undegraded":"2024-09-10T08:39:11.583174+0000","last_fullsized":"2024-09-10T08:39:11.583174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:18.464831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631756+0000","last_change":"2024-09-10T08:36:53.720102+0000","last_active":"2024-09-10T08:38:43.631756+0000","last_peered":"2024-09-10T08:38:43.631756+0000","last_clean":"2024-09-10T08:38:43.631756+0000","last_became_active":"2024-09-10T08:36:53.719764+0000","last_became_peered":"2024-09-10T08:36:53.719764+0000","last_unstale":"2024-09-10T08:38:43.631756+0000","last_undegraded":"2024-09-10T08:38:43.631756+0000","last_fullsized":"2024-09-10T08:38:43.631756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:03.609338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655205+0000","last_change":"2024-09-10T08:37:05.413818+0000","last_active":"2024-09-10T08:38:43.655205+0000","last_peered":"2024-09-10T08:38:43.655205+0000","last_clean":"2024-09-10T08:38:43.655205+0000","last_became_active":"2024-09-10T08:37:05.413267+0000","last_became_peered":"2024-09-10T08:37:05.413267+0000","last_unstale":"2024-09-10T08:38:43.655205+0000","last_undegraded":"2024-09-10T08:38:43.655205+0000","last_fullsized":"2024-09-10T08:38:43.655205+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:54.309686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.590486+0000","last_change":"2024-09-10T08:36:56.149286+0000","last_active":"2024-09-10T08:39:11.590486+0000","last_peered":"2024-09-10T08:39:11.590486+0000","last_clean":"2024-09-10T08:39:11.590486+0000","last_became_active":"2024-09-10T08:36:56.146275+0000","last_became_peered":"2024-09-10T08:36:56.146275+0000","last_unstale":"2024-09-10T08:39:11.590486+0000","last_undegraded":"2024-09-10T08:39:11.590486+0000","last_fullsized":"2024-09-10T08:39:11.590486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:19.468224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655269+0000","last_change":"2024-09-10T08:36:53.719756+0000","last_active":"2024-09-10T08:38:43.655269+0000","last_peered":"2024-09-10T08:38:43.655269+0000","last_clean":"2024-09-10T08:38:43.655269+0000","last_became_active":"2024-09-10T08:36:53.719302+0000","last_became_peered":"2024-09-10T08:36:53.719302+0000","last_unstale":"2024-09-10T08:38:43.655269+0000","last_undegraded":"2024-09-10T08:38:43.655269+0000","last_fullsized":"2024-09-10T08:38:43.655269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:47.658081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631788+0000","last_change":"2024-09-10T08:35:39.150715+0000","last_active":"2024-09-10T08:38:43.631788+0000","last_peered":"2024-09-10T08:38:43.631788+0000","last_clean":"2024-09-10T08:38:43.631788+0000","last_became_active":"2024-09-10T08:35:39.150206+0000","last_became_peered":"2024-09-10T08:35:39.150206+0000","last_unstale":"2024-09-10T08:38:43.631788+0000","last_undegraded":"2024-09-10T08:38:43.631788+0000","last_fullsized":"2024-09-10T08:38:43.631788+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:32:02.110870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:32:02.110870+0000","last_clean_scrub_stamp":"2024-09-10T08:32:02.110870+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:20.762043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631998+0000","last_change":"2024-09-10T08:37:42.846547+0000","last_active":"2024-09-10T08:38:43.631998+0000","last_peered":"2024-09-10T08:38:43.631998+0000","last_clean":"2024-09-10T08:38:43.631998+0000","last_became_active":"2024-09-10T08:37:42.845884+0000","last_became_peered":"2024-09-10T08:37:42.845884+0000","last_unstale":"2024-09-10T08:38:43.631998+0000","last_undegraded":"2024-09-10T08:38:43.631998+0000","last_fullsized":"2024-09-10T08:38:43.631998+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:21.905133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593781+0000","last_change":"2024-09-10T08:36:56.144187+0000","last_active":"2024-09-10T08:39:11.593781+0000","last_peered":"2024-09-10T08:39:11.593781+0000","last_clean":"2024-09-10T08:39:11.593781+0000","last_became_active":"2024-09-10T08:36:56.143193+0000","last_became_peered":"2024-09-10T08:36:56.143193+0000","last_unstale":"2024-09-10T08:39:11.593781+0000","last_undegraded":"2024-09-10T08:39:11.593781+0000","last_fullsized":"2024-09-10T08:39:11.593781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:14:26.981884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.639660+0000","last_change":"2024-09-10T08:38:43.639286+0000","last_active":"2024-09-10T08:38:43.639660+0000","last_peered":"2024-09-10T08:38:43.639660+0000","last_clean":"2024-09-10T08:38:43.639660+0000","last_became_active":"2024-09-10T08:38:43.633552+0000","last_became_peered":"2024-09-10T08:38:43.633552+0000","last_unstale":"2024-09-10T08:38:43.639660+0000","last_undegraded":"2024-09-10T08:38:43.639660+0000","last_fullsized":"2024-09-10T08:38:43.639660+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:23.985190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630209+0000","last_change":"2024-09-10T08:37:05.242095+0000","last_active":"2024-09-10T08:38:43.630209+0000","last_peered":"2024-09-10T08:38:43.630209+0000","last_clean":"2024-09-10T08:38:43.630209+0000","last_became_active":"2024-09-10T08:37:05.241645+0000","last_became_peered":"2024-09-10T08:37:05.241645+0000","last_unstale":"2024-09-10T08:38:43.630209+0000","last_undegraded":"2024-09-10T08:38:43.630209+0000","last_fullsized":"2024-09-10T08:38:43.630209+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:30:49.637861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588835+0000","last_change":"2024-09-10T08:36:56.149396+0000","last_active":"2024-09-10T08:39:11.588835+0000","last_peered":"2024-09-10T08:39:11.588835+0000","last_clean":"2024-09-10T08:39:11.588835+0000","last_became_active":"2024-09-10T08:36:56.149079+0000","last_became_peered":"2024-09-10T08:36:56.149079+0000","last_unstale":"2024-09-10T08:39:11.588835+0000","last_undegraded":"2024-09-10T08:39:11.588835+0000","last_fullsized":"2024-09-10T08:39:11.588835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:44.665448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655467+0000","last_change":"2024-09-10T08:36:53.717950+0000","last_active":"2024-09-10T08:38:43.655467+0000","last_peered":"2024-09-10T08:38:43.655467+0000","last_clean":"2024-09-10T08:38:43.655467+0000","last_became_active":"2024-09-10T08:36:53.717668+0000","last_became_peered":"2024-09-10T08:36:53.717668+0000","last_unstale":"2024-09-10T08:38:43.655467+0000","last_undegraded":"2024-09-10T08:38:43.655467+0000","last_fullsized":"2024-09-10T08:38:43.655467+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:58.692631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633140+0000","last_change":"2024-09-10T08:37:05.260178+0000","last_active":"2024-09-10T08:38:43.633140+0000","last_peered":"2024-09-10T08:38:43.633140+0000","last_clean":"2024-09-10T08:38:43.633140+0000","last_became_active":"2024-09-10T08:37:05.259753+0000","last_became_peered":"2024-09-10T08:37:05.259753+0000","last_unstale":"2024-09-10T08:38:43.633140+0000","last_undegraded":"2024-09-10T08:38:43.633140+0000","last_fullsized":"2024-09-10T08:38:43.633140+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:19.364782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588360+0000","last_change":"2024-09-10T08:36:56.147104+0000","last_active":"2024-09-10T08:39:11.588360+0000","last_peered":"2024-09-10T08:39:11.588360+0000","last_clean":"2024-09-10T08:39:11.588360+0000","last_became_active":"2024-09-10T08:36:56.146778+0000","last_became_peered":"2024-09-10T08:36:56.146778+0000","last_unstale":"2024-09-10T08:39:11.588360+0000","last_undegraded":"2024-09-10T08:39:11.588360+0000","last_fullsized":"2024-09-10T08:39:11.588360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:56.092157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628399+0000","last_change":"2024-09-10T08:36:53.719145+0000","last_active":"2024-09-10T08:38:43.628399+0000","last_peered":"2024-09-10T08:38:43.628399+0000","last_clean":"2024-09-10T08:38:43.628399+0000","last_became_active":"2024-09-10T08:36:53.718771+0000","last_became_peered":"2024-09-10T08:36:53.718771+0000","last_unstale":"2024-09-10T08:38:43.628399+0000","last_undegraded":"2024-09-10T08:38:43.628399+0000","last_fullsized":"2024-09-10T08:38:43.628399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:21.109940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654733+0000","last_change":"2024-09-10T08:37:05.242854+0000","last_active":"2024-09-10T08:38:43.654733+0000","last_peered":"2024-09-10T08:38:43.654733+0000","last_clean":"2024-09-10T08:38:43.654733+0000","last_became_active":"2024-09-10T08:37:05.242442+0000","last_became_peered":"2024-09-10T08:37:05.242442+0000","last_unstale":"2024-09-10T08:38:43.654733+0000","last_undegraded":"2024-09-10T08:38:43.654733+0000","last_fullsized":"2024-09-10T08:38:43.654733+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:36.072131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.570294+0000","last_change":"2024-09-10T08:37:42.845642+0000","last_active":"2024-09-10T08:39:11.570294+0000","last_peered":"2024-09-10T08:39:11.570294+0000","last_clean":"2024-09-10T08:39:11.570294+0000","last_became_active":"2024-09-10T08:37:42.845217+0000","last_became_peered":"2024-09-10T08:37:42.845217+0000","last_unstale":"2024-09-10T08:39:11.570294+0000","last_undegraded":"2024-09-10T08:39:11.570294+0000","last_fullsized":"2024-09-10T08:39:11.570294+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:28.020337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626734+0000","last_change":"2024-09-10T08:36:53.720463+0000","last_active":"2024-09-10T08:38:43.626734+0000","last_peered":"2024-09-10T08:38:43.626734+0000","last_clean":"2024-09-10T08:38:43.626734+0000","last_became_active":"2024-09-10T08:36:53.720080+0000","last_became_peered":"2024-09-10T08:36:53.720080+0000","last_unstale":"2024-09-10T08:38:43.626734+0000","last_undegraded":"2024-09-10T08:38:43.626734+0000","last_fullsized":"2024-09-10T08:38:43.626734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:51.450294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654955+0000","last_change":"2024-09-10T08:37:05.251261+0000","last_active":"2024-09-10T08:38:43.654955+0000","last_peered":"2024-09-10T08:38:43.654955+0000","last_clean":"2024-09-10T08:38:43.654955+0000","last_became_active":"2024-09-10T08:37:05.250938+0000","last_became_peered":"2024-09-10T08:37:05.250938+0000","last_unstale":"2024-09-10T08:38:43.654955+0000","last_undegraded":"2024-09-10T08:38:43.654955+0000","last_fullsized":"2024-09-10T08:38:43.654955+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:07:15.129899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.581822+0000","last_change":"2024-09-10T08:36:56.148191+0000","last_active":"2024-09-10T08:39:11.581822+0000","last_peered":"2024-09-10T08:39:11.581822+0000","last_clean":"2024-09-10T08:39:11.581822+0000","last_became_active":"2024-09-10T08:36:56.147362+0000","last_became_peered":"2024-09-10T08:36:56.147362+0000","last_unstale":"2024-09-10T08:39:11.581822+0000","last_undegraded":"2024-09-10T08:39:11.581822+0000","last_fullsized":"2024-09-10T08:39:11.581822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:25:30.051754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626042+0000","last_change":"2024-09-10T08:36:53.734484+0000","last_active":"2024-09-10T08:38:43.626042+0000","last_peered":"2024-09-10T08:38:43.626042+0000","last_clean":"2024-09-10T08:38:43.626042+0000","last_became_active":"2024-09-10T08:36:53.734172+0000","last_became_peered":"2024-09-10T08:36:53.734172+0000","last_unstale":"2024-09-10T08:38:43.626042+0000","last_undegraded":"2024-09-10T08:38:43.626042+0000","last_fullsized":"2024-09-10T08:38:43.626042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:10.269400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628299+0000","last_change":"2024-09-10T08:37:05.259635+0000","last_active":"2024-09-10T08:38:43.628299+0000","last_peered":"2024-09-10T08:38:43.628299+0000","last_clean":"2024-09-10T08:38:43.628299+0000","last_became_active":"2024-09-10T08:37:05.259107+0000","last_became_peered":"2024-09-10T08:37:05.259107+0000","last_unstale":"2024-09-10T08:38:43.628299+0000","last_undegraded":"2024-09-10T08:38:43.628299+0000","last_fullsized":"2024-09-10T08:38:43.628299+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:36.841138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588992+0000","last_change":"2024-09-10T08:36:56.141122+0000","last_active":"2024-09-10T08:39:11.588992+0000","last_peered":"2024-09-10T08:39:11.588992+0000","last_clean":"2024-09-10T08:39:11.588992+0000","last_became_active":"2024-09-10T08:36:56.140826+0000","last_became_peered":"2024-09-10T08:36:56.140826+0000","last_unstale":"2024-09-10T08:39:11.588992+0000","last_undegraded":"2024-09-10T08:39:11.588992+0000","last_fullsized":"2024-09-10T08:39:11.588992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:38.044090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"96'6","reported_seq":174,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:48.904683+0000","last_change":"2024-09-10T08:36:53.720623+0000","last_active":"2024-09-10T08:39:48.904683+0000","last_peered":"2024-09-10T08:39:48.904683+0000","last_clean":"2024-09-10T08:39:48.904683+0000","last_became_active":"2024-09-10T08:36:53.720270+0000","last_became_peered":"2024-09-10T08:36:53.720270+0000","last_unstale":"2024-09-10T08:39:48.904683+0000","last_undegraded":"2024-09-10T08:39:48.904683+0000","last_fullsized":"2024-09-10T08:39:48.904683+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:19.514068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629937+0000","last_change":"2024-09-10T08:37:05.254324+0000","last_active":"2024-09-10T08:38:43.629937+0000","last_peered":"2024-09-10T08:38:43.629937+0000","last_clean":"2024-09-10T08:38:43.629937+0000","last_became_active":"2024-09-10T08:37:05.253611+0000","last_became_peered":"2024-09-10T08:37:05.253611+0000","last_unstale":"2024-09-10T08:38:43.629937+0000","last_undegraded":"2024-09-10T08:38:43.629937+0000","last_fullsized":"2024-09-10T08:38:43.629937+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:07.182649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.591257+0000","last_change":"2024-09-10T08:36:56.139102+0000","last_active":"2024-09-10T08:39:11.591257+0000","last_peered":"2024-09-10T08:39:11.591257+0000","last_clean":"2024-09-10T08:39:11.591257+0000","last_became_active":"2024-09-10T08:36:56.137986+0000","last_became_peered":"2024-09-10T08:36:56.137986+0000","last_unstale":"2024-09-10T08:39:11.591257+0000","last_undegraded":"2024-09-10T08:39:11.591257+0000","last_fullsized":"2024-09-10T08:39:11.591257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:05:03.005727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655021+0000","last_change":"2024-09-10T08:36:53.722961+0000","last_active":"2024-09-10T08:38:43.655021+0000","last_peered":"2024-09-10T08:38:43.655021+0000","last_clean":"2024-09-10T08:38:43.655021+0000","last_became_active":"2024-09-10T08:36:53.722628+0000","last_became_peered":"2024-09-10T08:36:53.722628+0000","last_unstale":"2024-09-10T08:38:43.655021+0000","last_undegraded":"2024-09-10T08:38:43.655021+0000","last_fullsized":"2024-09-10T08:38:43.655021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:59.048632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628104+0000","last_change":"2024-09-10T08:37:05.256546+0000","last_active":"2024-09-10T08:38:43.628104+0000","last_peered":"2024-09-10T08:38:43.628104+0000","last_clean":"2024-09-10T08:38:43.628104+0000","last_became_active":"2024-09-10T08:37:05.256239+0000","last_became_peered":"2024-09-10T08:37:05.256239+0000","last_unstale":"2024-09-10T08:38:43.628104+0000","last_undegraded":"2024-09-10T08:38:43.628104+0000","last_fullsized":"2024-09-10T08:38:43.628104+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:57.403934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.582848+0000","last_change":"2024-09-10T08:36:56.149285+0000","last_active":"2024-09-10T08:39:11.582848+0000","last_peered":"2024-09-10T08:39:11.582848+0000","last_clean":"2024-09-10T08:39:11.582848+0000","last_became_active":"2024-09-10T08:36:56.146873+0000","last_became_peered":"2024-09-10T08:36:56.146873+0000","last_unstale":"2024-09-10T08:39:11.582848+0000","last_undegraded":"2024-09-10T08:39:11.582848+0000","last_fullsized":"2024-09-10T08:39:11.582848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:28.976729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630368+0000","last_change":"2024-09-10T08:36:53.721174+0000","last_active":"2024-09-10T08:38:43.630368+0000","last_peered":"2024-09-10T08:38:43.630368+0000","last_clean":"2024-09-10T08:38:43.630368+0000","last_became_active":"2024-09-10T08:36:53.720849+0000","last_became_peered":"2024-09-10T08:36:53.720849+0000","last_unstale":"2024-09-10T08:38:43.630368+0000","last_undegraded":"2024-09-10T08:38:43.630368+0000","last_fullsized":"2024-09-10T08:38:43.630368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:39.786848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655286+0000","last_change":"2024-09-10T08:37:05.252256+0000","last_active":"2024-09-10T08:38:43.655286+0000","last_peered":"2024-09-10T08:38:43.655286+0000","last_clean":"2024-09-10T08:38:43.655286+0000","last_became_active":"2024-09-10T08:37:05.251865+0000","last_became_peered":"2024-09-10T08:37:05.251865+0000","last_unstale":"2024-09-10T08:38:43.655286+0000","last_undegraded":"2024-09-10T08:38:43.655286+0000","last_fullsized":"2024-09-10T08:38:43.655286+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:08.581880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567773+0000","last_change":"2024-09-10T08:36:56.142060+0000","last_active":"2024-09-10T08:39:11.567773+0000","last_peered":"2024-09-10T08:39:11.567773+0000","last_clean":"2024-09-10T08:39:11.567773+0000","last_became_active":"2024-09-10T08:36:56.141451+0000","last_became_peered":"2024-09-10T08:36:56.141451+0000","last_unstale":"2024-09-10T08:39:11.567773+0000","last_undegraded":"2024-09-10T08:39:11.567773+0000","last_fullsized":"2024-09-10T08:39:11.567773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:43.566588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'39","reported_seq":334,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:48.893004+0000","last_change":"2024-09-10T08:36:53.719464+0000","last_active":"2024-09-10T08:39:48.893004+0000","last_peered":"2024-09-10T08:39:48.893004+0000","last_clean":"2024-09-10T08:39:48.893004+0000","last_became_active":"2024-09-10T08:36:53.719129+0000","last_became_peered":"2024-09-10T08:36:53.719129+0000","last_unstale":"2024-09-10T08:39:48.893004+0000","last_undegraded":"2024-09-10T08:39:48.893004+0000","last_fullsized":"2024-09-10T08:39:48.893004+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:05.652018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71298,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":254,"num_read_kb":197,"num_write":40,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629405+0000","last_change":"2024-09-10T08:37:05.255576+0000","last_active":"2024-09-10T08:38:43.629405+0000","last_peered":"2024-09-10T08:38:43.629405+0000","last_clean":"2024-09-10T08:38:43.629405+0000","last_became_active":"2024-09-10T08:37:05.249842+0000","last_became_peered":"2024-09-10T08:37:05.249842+0000","last_unstale":"2024-09-10T08:38:43.629405+0000","last_undegraded":"2024-09-10T08:38:43.629405+0000","last_fullsized":"2024-09-10T08:38:43.629405+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:47:49.543481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593397+0000","last_change":"2024-09-10T08:36:56.151829+0000","last_active":"2024-09-10T08:39:11.593397+0000","last_peered":"2024-09-10T08:39:11.593397+0000","last_clean":"2024-09-10T08:39:11.593397+0000","last_became_active":"2024-09-10T08:36:56.151314+0000","last_became_peered":"2024-09-10T08:36:56.151314+0000","last_unstale":"2024-09-10T08:39:11.593397+0000","last_undegraded":"2024-09-10T08:39:11.593397+0000","last_fullsized":"2024-09-10T08:39:11.593397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:44.219671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655398+0000","last_change":"2024-09-10T08:36:53.738030+0000","last_active":"2024-09-10T08:38:43.655398+0000","last_peered":"2024-09-10T08:38:43.655398+0000","last_clean":"2024-09-10T08:38:43.655398+0000","last_became_active":"2024-09-10T08:36:53.737658+0000","last_became_peered":"2024-09-10T08:36:53.737658+0000","last_unstale":"2024-09-10T08:38:43.655398+0000","last_undegraded":"2024-09-10T08:38:43.655398+0000","last_fullsized":"2024-09-10T08:38:43.655398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:29.627554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654996+0000","last_change":"2024-09-10T08:37:05.244473+0000","last_active":"2024-09-10T08:38:43.654996+0000","last_peered":"2024-09-10T08:38:43.654996+0000","last_clean":"2024-09-10T08:38:43.654996+0000","last_became_active":"2024-09-10T08:37:05.244035+0000","last_became_peered":"2024-09-10T08:37:05.244035+0000","last_unstale":"2024-09-10T08:38:43.654996+0000","last_undegraded":"2024-09-10T08:38:43.654996+0000","last_fullsized":"2024-09-10T08:38:43.654996+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:04.652660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584024+0000","last_change":"2024-09-10T08:36:56.155217+0000","last_active":"2024-09-10T08:39:11.584024+0000","last_peered":"2024-09-10T08:39:11.584024+0000","last_clean":"2024-09-10T08:39:11.584024+0000","last_became_active":"2024-09-10T08:36:56.154903+0000","last_became_peered":"2024-09-10T08:36:56.154903+0000","last_unstale":"2024-09-10T08:39:11.584024+0000","last_undegraded":"2024-09-10T08:39:11.584024+0000","last_fullsized":"2024-09-10T08:39:11.584024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:47.482974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.632239+0000","last_change":"2024-09-10T08:36:53.721087+0000","last_active":"2024-09-10T08:38:43.632239+0000","last_peered":"2024-09-10T08:38:43.632239+0000","last_clean":"2024-09-10T08:38:43.632239+0000","last_became_active":"2024-09-10T08:36:53.720772+0000","last_became_peered":"2024-09-10T08:36:53.720772+0000","last_unstale":"2024-09-10T08:38:43.632239+0000","last_undegraded":"2024-09-10T08:38:43.632239+0000","last_fullsized":"2024-09-10T08:38:43.632239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:39.560848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629150+0000","last_change":"2024-09-10T08:37:05.254641+0000","last_active":"2024-09-10T08:38:43.629150+0000","last_peered":"2024-09-10T08:38:43.629150+0000","last_clean":"2024-09-10T08:38:43.629150+0000","last_became_active":"2024-09-10T08:37:05.249982+0000","last_became_peered":"2024-09-10T08:37:05.249982+0000","last_unstale":"2024-09-10T08:38:43.629150+0000","last_undegraded":"2024-09-10T08:38:43.629150+0000","last_fullsized":"2024-09-10T08:38:43.629150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:21:13.851061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592510+0000","last_change":"2024-09-10T08:36:56.142657+0000","last_active":"2024-09-10T08:39:11.592510+0000","last_peered":"2024-09-10T08:39:11.592510+0000","last_clean":"2024-09-10T08:39:11.592510+0000","last_became_active":"2024-09-10T08:36:56.142294+0000","last_became_peered":"2024-09-10T08:36:56.142294+0000","last_unstale":"2024-09-10T08:39:11.592510+0000","last_undegraded":"2024-09-10T08:39:11.592510+0000","last_fullsized":"2024-09-10T08:39:11.592510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:52.861389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'4","reported_seq":166,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:47.891688+0000","last_change":"2024-09-10T08:36:53.722466+0000","last_active":"2024-09-10T08:39:47.891688+0000","last_peered":"2024-09-10T08:39:47.891688+0000","last_clean":"2024-09-10T08:39:47.891688+0000","last_became_active":"2024-09-10T08:36:53.722145+0000","last_became_peered":"2024-09-10T08:36:53.722145+0000","last_unstale":"2024-09-10T08:39:47.891688+0000","last_undegraded":"2024-09-10T08:39:47.891688+0000","last_fullsized":"2024-09-10T08:39:47.891688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:27.041765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631352+0000","last_change":"2024-09-10T08:37:05.251056+0000","last_active":"2024-09-10T08:38:43.631352+0000","last_peered":"2024-09-10T08:38:43.631352+0000","last_clean":"2024-09-10T08:38:43.631352+0000","last_became_active":"2024-09-10T08:37:05.250653+0000","last_became_peered":"2024-09-10T08:37:05.250653+0000","last_unstale":"2024-09-10T08:38:43.631352+0000","last_undegraded":"2024-09-10T08:38:43.631352+0000","last_fullsized":"2024-09-10T08:38:43.631352+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:21.174747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633128+0000","last_change":"2024-09-10T08:37:05.260436+0000","last_active":"2024-09-10T08:38:43.633128+0000","last_peered":"2024-09-10T08:38:43.633128+0000","last_clean":"2024-09-10T08:38:43.633128+0000","last_became_active":"2024-09-10T08:37:05.260049+0000","last_became_peered":"2024-09-10T08:37:05.260049+0000","last_unstale":"2024-09-10T08:38:43.633128+0000","last_undegraded":"2024-09-10T08:38:43.633128+0000","last_fullsized":"2024-09-10T08:38:43.633128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:43.394028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585930+0000","last_change":"2024-09-10T08:36:56.146732+0000","last_active":"2024-09-10T08:39:11.585930+0000","last_peered":"2024-09-10T08:39:11.585930+0000","last_clean":"2024-09-10T08:39:11.585930+0000","last_became_active":"2024-09-10T08:36:56.145599+0000","last_became_peered":"2024-09-10T08:36:56.145599+0000","last_unstale":"2024-09-10T08:39:11.585930+0000","last_undegraded":"2024-09-10T08:39:11.585930+0000","last_fullsized":"2024-09-10T08:39:11.585930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:50.620956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655135+0000","last_change":"2024-09-10T08:36:53.721683+0000","last_active":"2024-09-10T08:38:43.655135+0000","last_peered":"2024-09-10T08:38:43.655135+0000","last_clean":"2024-09-10T08:38:43.655135+0000","last_became_active":"2024-09-10T08:36:53.721241+0000","last_became_peered":"2024-09-10T08:36:53.721241+0000","last_unstale":"2024-09-10T08:38:43.655135+0000","last_undegraded":"2024-09-10T08:38:43.655135+0000","last_fullsized":"2024-09-10T08:38:43.655135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:03.669337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628275+0000","last_change":"2024-09-10T08:37:42.669784+0000","last_active":"2024-09-10T08:38:43.628275+0000","last_peered":"2024-09-10T08:38:43.628275+0000","last_clean":"2024-09-10T08:38:43.628275+0000","last_became_active":"2024-09-10T08:37:42.669322+0000","last_became_peered":"2024-09-10T08:37:42.669322+0000","last_unstale":"2024-09-10T08:38:43.628275+0000","last_undegraded":"2024-09-10T08:38:43.628275+0000","last_fullsized":"2024-09-10T08:38:43.628275+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:06:57.027928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.574617+0000","last_change":"2024-09-10T08:36:56.143824+0000","last_active":"2024-09-10T08:39:11.574617+0000","last_peered":"2024-09-10T08:39:11.574617+0000","last_clean":"2024-09-10T08:39:11.574617+0000","last_became_active":"2024-09-10T08:36:56.143486+0000","last_became_peered":"2024-09-10T08:36:56.143486+0000","last_unstale":"2024-09-10T08:39:11.574617+0000","last_undegraded":"2024-09-10T08:39:11.574617+0000","last_fullsized":"2024-09-10T08:39:11.574617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:42.384863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627976+0000","last_change":"2024-09-10T08:36:53.717680+0000","last_active":"2024-09-10T08:38:43.627976+0000","last_peered":"2024-09-10T08:38:43.627976+0000","last_clean":"2024-09-10T08:38:43.627976+0000","last_became_active":"2024-09-10T08:36:53.717249+0000","last_became_peered":"2024-09-10T08:36:53.717249+0000","last_unstale":"2024-09-10T08:38:43.627976+0000","last_undegraded":"2024-09-10T08:38:43.627976+0000","last_fullsized":"2024-09-10T08:38:43.627976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:12.191541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627200+0000","last_change":"2024-09-10T08:36:53.723947+0000","last_active":"2024-09-10T08:38:43.627200+0000","last_peered":"2024-09-10T08:38:43.627200+0000","last_clean":"2024-09-10T08:38:43.627200+0000","last_became_active":"2024-09-10T08:36:53.723620+0000","last_became_peered":"2024-09-10T08:36:53.723620+0000","last_unstale":"2024-09-10T08:38:43.627200+0000","last_undegraded":"2024-09-10T08:38:43.627200+0000","last_fullsized":"2024-09-10T08:38:43.627200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:00.135349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583145+0000","last_change":"2024-09-10T08:36:56.150448+0000","last_active":"2024-09-10T08:39:11.583145+0000","last_peered":"2024-09-10T08:39:11.583145+0000","last_clean":"2024-09-10T08:39:11.583145+0000","last_became_active":"2024-09-10T08:36:56.150148+0000","last_became_peered":"2024-09-10T08:36:56.150148+0000","last_unstale":"2024-09-10T08:39:11.583145+0000","last_undegraded":"2024-09-10T08:39:11.583145+0000","last_fullsized":"2024-09-10T08:39:11.583145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:02.927753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630191+0000","last_change":"2024-09-10T08:37:05.242878+0000","last_active":"2024-09-10T08:38:43.630191+0000","last_peered":"2024-09-10T08:38:43.630191+0000","last_clean":"2024-09-10T08:38:43.630191+0000","last_became_active":"2024-09-10T08:37:05.242440+0000","last_became_peered":"2024-09-10T08:37:05.242440+0000","last_unstale":"2024-09-10T08:38:43.630191+0000","last_undegraded":"2024-09-10T08:38:43.630191+0000","last_fullsized":"2024-09-10T08:38:43.630191+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:30.957433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'7","reported_seq":174,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:48.893331+0000","last_change":"2024-09-10T08:36:53.719788+0000","last_active":"2024-09-10T08:39:48.893331+0000","last_peered":"2024-09-10T08:39:48.893331+0000","last_clean":"2024-09-10T08:39:48.893331+0000","last_became_active":"2024-09-10T08:36:53.719462+0000","last_became_peered":"2024-09-10T08:36:53.719462+0000","last_unstale":"2024-09-10T08:39:48.893331+0000","last_undegraded":"2024-09-10T08:39:48.893331+0000","last_fullsized":"2024-09-10T08:39:48.893331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:12.250447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1326,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583170+0000","last_change":"2024-09-10T08:37:42.673008+0000","last_active":"2024-09-10T08:39:11.583170+0000","last_peered":"2024-09-10T08:39:11.583170+0000","last_clean":"2024-09-10T08:39:11.583170+0000","last_became_active":"2024-09-10T08:37:42.672618+0000","last_became_peered":"2024-09-10T08:37:42.672618+0000","last_unstale":"2024-09-10T08:39:11.583170+0000","last_undegraded":"2024-09-10T08:39:11.583170+0000","last_fullsized":"2024-09-10T08:39:11.583170+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:00.156565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630783+0000","last_change":"2024-09-10T08:37:05.244277+0000","last_active":"2024-09-10T08:38:43.630783+0000","last_peered":"2024-09-10T08:38:43.630783+0000","last_clean":"2024-09-10T08:38:43.630783+0000","last_became_active":"2024-09-10T08:37:05.243880+0000","last_became_peered":"2024-09-10T08:37:05.243880+0000","last_unstale":"2024-09-10T08:38:43.630783+0000","last_undegraded":"2024-09-10T08:38:43.630783+0000","last_fullsized":"2024-09-10T08:38:43.630783+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:55.275857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.573131+0000","last_change":"2024-09-10T08:36:56.141404+0000","last_active":"2024-09-10T08:39:11.573131+0000","last_peered":"2024-09-10T08:39:11.573131+0000","last_clean":"2024-09-10T08:39:11.573131+0000","last_became_active":"2024-09-10T08:36:56.141076+0000","last_became_peered":"2024-09-10T08:36:56.141076+0000","last_unstale":"2024-09-10T08:39:11.573131+0000","last_undegraded":"2024-09-10T08:39:11.573131+0000","last_fullsized":"2024-09-10T08:39:11.573131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:03:52.915654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.569383+0000","last_change":"2024-09-10T08:36:56.148285+0000","last_active":"2024-09-10T08:39:11.569383+0000","last_peered":"2024-09-10T08:39:11.569383+0000","last_clean":"2024-09-10T08:39:11.569383+0000","last_became_active":"2024-09-10T08:36:56.147970+0000","last_became_peered":"2024-09-10T08:36:56.147970+0000","last_unstale":"2024-09-10T08:39:11.569383+0000","last_undegraded":"2024-09-10T08:39:11.569383+0000","last_fullsized":"2024-09-10T08:39:11.569383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:09.506718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.571047+0000","last_change":"2024-09-10T08:36:56.157977+0000","last_active":"2024-09-10T08:39:11.571047+0000","last_peered":"2024-09-10T08:39:11.571047+0000","last_clean":"2024-09-10T08:39:11.571047+0000","last_became_active":"2024-09-10T08:36:56.157623+0000","last_became_peered":"2024-09-10T08:36:56.157623+0000","last_unstale":"2024-09-10T08:39:11.571047+0000","last_undegraded":"2024-09-10T08:39:11.571047+0000","last_fullsized":"2024-09-10T08:39:11.571047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:44.617979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584899+0000","last_change":"2024-09-10T08:37:42.670459+0000","last_active":"2024-09-10T08:39:11.584899+0000","last_peered":"2024-09-10T08:39:11.584899+0000","last_clean":"2024-09-10T08:39:11.584899+0000","last_became_active":"2024-09-10T08:37:42.670097+0000","last_became_peered":"2024-09-10T08:37:42.670097+0000","last_unstale":"2024-09-10T08:39:11.584899+0000","last_undegraded":"2024-09-10T08:39:11.584899+0000","last_fullsized":"2024-09-10T08:39:11.584899+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:14.956492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584527+0000","last_change":"2024-09-10T08:36:56.154850+0000","last_active":"2024-09-10T08:39:11.584527+0000","last_peered":"2024-09-10T08:39:11.584527+0000","last_clean":"2024-09-10T08:39:11.584527+0000","last_became_active":"2024-09-10T08:36:56.154545+0000","last_became_peered":"2024-09-10T08:36:56.154545+0000","last_unstale":"2024-09-10T08:39:11.584527+0000","last_undegraded":"2024-09-10T08:39:11.584527+0000","last_fullsized":"2024-09-10T08:39:11.584527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:24.639667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.569973+0000","last_change":"2024-09-10T08:37:42.669962+0000","last_active":"2024-09-10T08:39:11.569973+0000","last_peered":"2024-09-10T08:39:11.569973+0000","last_clean":"2024-09-10T08:39:11.569973+0000","last_became_active":"2024-09-10T08:37:42.669556+0000","last_became_peered":"2024-09-10T08:37:42.669556+0000","last_unstale":"2024-09-10T08:39:11.569973+0000","last_undegraded":"2024-09-10T08:39:11.569973+0000","last_fullsized":"2024-09-10T08:39:11.569973+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:23.333946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.578061+0000","last_change":"2024-09-10T08:36:56.145910+0000","last_active":"2024-09-10T08:39:11.578061+0000","last_peered":"2024-09-10T08:39:11.578061+0000","last_clean":"2024-09-10T08:39:11.578061+0000","last_became_active":"2024-09-10T08:36:56.145537+0000","last_became_peered":"2024-09-10T08:36:56.145537+0000","last_unstale":"2024-09-10T08:39:11.578061+0000","last_undegraded":"2024-09-10T08:39:11.578061+0000","last_fullsized":"2024-09-10T08:39:11.578061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:46.788241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567880+0000","last_change":"2024-09-10T08:36:56.147340+0000","last_active":"2024-09-10T08:39:11.567880+0000","last_peered":"2024-09-10T08:39:11.567880+0000","last_clean":"2024-09-10T08:39:11.567880+0000","last_became_active":"2024-09-10T08:36:56.145762+0000","last_became_peered":"2024-09-10T08:36:56.145762+0000","last_unstale":"2024-09-10T08:39:11.567880+0000","last_undegraded":"2024-09-10T08:39:11.567880+0000","last_fullsized":"2024-09-10T08:39:11.567880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:12.339285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.581431+0000","last_change":"2024-09-10T08:36:56.145213+0000","last_active":"2024-09-10T08:39:11.581431+0000","last_peered":"2024-09-10T08:39:11.581431+0000","last_clean":"2024-09-10T08:39:11.581431+0000","last_became_active":"2024-09-10T08:36:56.144915+0000","last_became_peered":"2024-09-10T08:36:56.144915+0000","last_unstale":"2024-09-10T08:39:11.581431+0000","last_undegraded":"2024-09-10T08:39:11.581431+0000","last_fullsized":"2024-09-10T08:39:11.581431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:06.180321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.574743+0000","last_change":"2024-09-10T08:36:56.149738+0000","last_active":"2024-09-10T08:39:11.574743+0000","last_peered":"2024-09-10T08:39:11.574743+0000","last_clean":"2024-09-10T08:39:11.574743+0000","last_became_active":"2024-09-10T08:36:56.147750+0000","last_became_peered":"2024-09-10T08:36:56.147750+0000","last_unstale":"2024-09-10T08:39:11.574743+0000","last_undegraded":"2024-09-10T08:39:11.574743+0000","last_fullsized":"2024-09-10T08:39:11.574743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:47.950054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.591937+0000","last_change":"2024-09-10T08:36:56.147682+0000","last_active":"2024-09-10T08:39:11.591937+0000","last_peered":"2024-09-10T08:39:11.591937+0000","last_clean":"2024-09-10T08:39:11.591937+0000","last_became_active":"2024-09-10T08:36:56.147345+0000","last_became_peered":"2024-09-10T08:36:56.147345+0000","last_unstale":"2024-09-10T08:39:11.591937+0000","last_undegraded":"2024-09-10T08:39:11.591937+0000","last_fullsized":"2024-09-10T08:39:11.591937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:54.939482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583378+0000","last_change":"2024-09-10T08:36:56.158292+0000","last_active":"2024-09-10T08:39:11.583378+0000","last_peered":"2024-09-10T08:39:11.583378+0000","last_clean":"2024-09-10T08:39:11.583378+0000","last_became_active":"2024-09-10T08:36:56.157970+0000","last_became_peered":"2024-09-10T08:36:56.157970+0000","last_unstale":"2024-09-10T08:39:11.583378+0000","last_undegraded":"2024-09-10T08:39:11.583378+0000","last_fullsized":"2024-09-10T08:39:11.583378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:33.582126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567916+0000","last_change":"2024-09-10T08:36:56.142520+0000","last_active":"2024-09-10T08:39:11.567916+0000","last_peered":"2024-09-10T08:39:11.567916+0000","last_clean":"2024-09-10T08:39:11.567916+0000","last_became_active":"2024-09-10T08:36:56.142214+0000","last_became_peered":"2024-09-10T08:36:56.142214+0000","last_unstale":"2024-09-10T08:39:11.567916+0000","last_undegraded":"2024-09-10T08:39:11.567916+0000","last_fullsized":"2024-09-10T08:39:11.567916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:45:29.201839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584167+0000","last_change":"2024-09-10T08:36:56.149547+0000","last_active":"2024-09-10T08:39:11.584167+0000","last_peered":"2024-09-10T08:39:11.584167+0000","last_clean":"2024-09-10T08:39:11.584167+0000","last_became_active":"2024-09-10T08:36:56.149141+0000","last_became_peered":"2024-09-10T08:36:56.149141+0000","last_unstale":"2024-09-10T08:39:11.584167+0000","last_undegraded":"2024-09-10T08:39:11.584167+0000","last_fullsized":"2024-09-10T08:39:11.584167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:49.368677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585864+0000","last_change":"2024-09-10T08:36:56.143087+0000","last_active":"2024-09-10T08:39:11.585864+0000","last_peered":"2024-09-10T08:39:11.585864+0000","last_clean":"2024-09-10T08:39:11.585864+0000","last_became_active":"2024-09-10T08:36:56.142782+0000","last_became_peered":"2024-09-10T08:36:56.142782+0000","last_unstale":"2024-09-10T08:39:11.585864+0000","last_undegraded":"2024-09-10T08:39:11.585864+0000","last_fullsized":"2024-09-10T08:39:11.585864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:17.513103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592876+0000","last_change":"2024-09-10T08:36:56.157549+0000","last_active":"2024-09-10T08:39:11.592876+0000","last_peered":"2024-09-10T08:39:11.592876+0000","last_clean":"2024-09-10T08:39:11.592876+0000","last_became_active":"2024-09-10T08:36:56.157221+0000","last_became_peered":"2024-09-10T08:36:56.157221+0000","last_unstale":"2024-09-10T08:39:11.592876+0000","last_undegraded":"2024-09-10T08:39:11.592876+0000","last_fullsized":"2024-09-10T08:39:11.592876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:11.372085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593199+0000","last_change":"2024-09-10T08:36:56.145654+0000","last_active":"2024-09-10T08:39:11.593199+0000","last_peered":"2024-09-10T08:39:11.593199+0000","last_clean":"2024-09-10T08:39:11.593199+0000","last_became_active":"2024-09-10T08:36:56.144126+0000","last_became_peered":"2024-09-10T08:36:56.144126+0000","last_unstale":"2024-09-10T08:39:11.593199+0000","last_undegraded":"2024-09-10T08:39:11.593199+0000","last_fullsized":"2024-09-10T08:39:11.593199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:55.498873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.582479+0000","last_change":"2024-09-10T08:36:56.146423+0000","last_active":"2024-09-10T08:39:11.582479+0000","last_peered":"2024-09-10T08:39:11.582479+0000","last_clean":"2024-09-10T08:39:11.582479+0000","last_became_active":"2024-09-10T08:36:56.146126+0000","last_became_peered":"2024-09-10T08:36:56.146126+0000","last_unstale":"2024-09-10T08:39:11.582479+0000","last_undegraded":"2024-09-10T08:39:11.582479+0000","last_fullsized":"2024-09-10T08:39:11.582479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:44.665196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583332+0000","last_change":"2024-09-10T08:36:56.151973+0000","last_active":"2024-09-10T08:39:11.583332+0000","last_peered":"2024-09-10T08:39:11.583332+0000","last_clean":"2024-09-10T08:39:11.583332+0000","last_became_active":"2024-09-10T08:36:56.151609+0000","last_became_peered":"2024-09-10T08:36:56.151609+0000","last_unstale":"2024-09-10T08:39:11.583332+0000","last_undegraded":"2024-09-10T08:39:11.583332+0000","last_fullsized":"2024-09-10T08:39:11.583332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:02.584047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584509+0000","last_change":"2024-09-10T08:36:56.141780+0000","last_active":"2024-09-10T08:39:11.584509+0000","last_peered":"2024-09-10T08:39:11.584509+0000","last_clean":"2024-09-10T08:39:11.584509+0000","last_became_active":"2024-09-10T08:36:56.141228+0000","last_became_peered":"2024-09-10T08:36:56.141228+0000","last_unstale":"2024-09-10T08:39:11.584509+0000","last_undegraded":"2024-09-10T08:39:11.584509+0000","last_fullsized":"2024-09-10T08:39:11.584509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:52:04.374228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585477+0000","last_change":"2024-09-10T08:37:42.671346+0000","last_active":"2024-09-10T08:39:11.585477+0000","last_peered":"2024-09-10T08:39:11.585477+0000","last_clean":"2024-09-10T08:39:11.585477+0000","last_became_active":"2024-09-10T08:37:42.670895+0000","last_became_peered":"2024-09-10T08:37:42.670895+0000","last_unstale":"2024-09-10T08:39:11.585477+0000","last_undegraded":"2024-09-10T08:39:11.585477+0000","last_fullsized":"2024-09-10T08:39:11.585477+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:16:31.930628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593796+0000","last_change":"2024-09-10T08:36:56.149014+0000","last_active":"2024-09-10T08:39:11.593796+0000","last_peered":"2024-09-10T08:39:11.593796+0000","last_clean":"2024-09-10T08:39:11.593796+0000","last_became_active":"2024-09-10T08:36:56.148551+0000","last_became_peered":"2024-09-10T08:36:56.148551+0000","last_unstale":"2024-09-10T08:39:11.593796+0000","last_undegraded":"2024-09-10T08:39:11.593796+0000","last_fullsized":"2024-09-10T08:39:11.593796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:35:52.250943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.586709+0000","last_change":"2024-09-10T08:36:56.141247+0000","last_active":"2024-09-10T08:39:11.586709+0000","last_peered":"2024-09-10T08:39:11.586709+0000","last_clean":"2024-09-10T08:39:11.586709+0000","last_became_active":"2024-09-10T08:36:56.140956+0000","last_became_peered":"2024-09-10T08:36:56.140956+0000","last_unstale":"2024-09-10T08:39:11.586709+0000","last_undegraded":"2024-09-10T08:39:11.586709+0000","last_fullsized":"2024-09-10T08:39:11.586709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:45.008424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592783+0000","last_change":"2024-09-10T08:36:56.149116+0000","last_active":"2024-09-10T08:39:11.592783+0000","last_peered":"2024-09-10T08:39:11.592783+0000","last_clean":"2024-09-10T08:39:11.592783+0000","last_became_active":"2024-09-10T08:36:56.148783+0000","last_became_peered":"2024-09-10T08:36:56.148783+0000","last_unstale":"2024-09-10T08:39:11.592783+0000","last_undegraded":"2024-09-10T08:39:11.592783+0000","last_fullsized":"2024-09-10T08:39:11.592783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:37.119214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585754+0000","last_change":"2024-09-10T08:37:42.846322+0000","last_active":"2024-09-10T08:39:11.585754+0000","last_peered":"2024-09-10T08:39:11.585754+0000","last_clean":"2024-09-10T08:39:11.585754+0000","last_became_active":"2024-09-10T08:37:42.846022+0000","last_became_peered":"2024-09-10T08:37:42.846022+0000","last_unstale":"2024-09-10T08:39:11.585754+0000","last_undegraded":"2024-09-10T08:39:11.585754+0000","last_fullsized":"2024-09-10T08:39:11.585754+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:38.142362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588751+0000","last_change":"2024-09-10T08:36:56.148537+0000","last_active":"2024-09-10T08:39:11.588751+0000","last_peered":"2024-09-10T08:39:11.588751+0000","last_clean":"2024-09-10T08:39:11.588751+0000","last_became_active":"2024-09-10T08:36:56.148185+0000","last_became_peered":"2024-09-10T08:36:56.148185+0000","last_unstale":"2024-09-10T08:39:11.588751+0000","last_undegraded":"2024-09-10T08:39:11.588751+0000","last_fullsized":"2024-09-10T08:39:11.588751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:00.426513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.586233+0000","last_change":"2024-09-10T08:36:56.138513+0000","last_active":"2024-09-10T08:39:11.586233+0000","last_peered":"2024-09-10T08:39:11.586233+0000","last_clean":"2024-09-10T08:39:11.586233+0000","last_became_active":"2024-09-10T08:36:56.138207+0000","last_became_peered":"2024-09-10T08:36:56.138207+0000","last_unstale":"2024-09-10T08:39:11.586233+0000","last_undegraded":"2024-09-10T08:39:11.586233+0000","last_fullsized":"2024-09-10T08:39:11.586233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:34.581430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.582686+0000","last_change":"2024-09-10T08:36:56.146612+0000","last_active":"2024-09-10T08:39:11.582686+0000","last_peered":"2024-09-10T08:39:11.582686+0000","last_clean":"2024-09-10T08:39:11.582686+0000","last_became_active":"2024-09-10T08:36:56.146251+0000","last_became_peered":"2024-09-10T08:36:56.146251+0000","last_unstale":"2024-09-10T08:39:11.582686+0000","last_undegraded":"2024-09-10T08:39:11.582686+0000","last_fullsized":"2024-09-10T08:39:11.582686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:47.336389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567939+0000","last_change":"2024-09-10T08:36:56.148260+0000","last_active":"2024-09-10T08:39:11.567939+0000","last_peered":"2024-09-10T08:39:11.567939+0000","last_clean":"2024-09-10T08:39:11.567939+0000","last_became_active":"2024-09-10T08:36:56.147949+0000","last_became_peered":"2024-09-10T08:36:56.147949+0000","last_unstale":"2024-09-10T08:39:11.567939+0000","last_undegraded":"2024-09-10T08:39:11.567939+0000","last_fullsized":"2024-09-10T08:39:11.567939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:52:19.762297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592986+0000","last_change":"2024-09-10T08:36:56.156045+0000","last_active":"2024-09-10T08:39:11.592986+0000","last_peered":"2024-09-10T08:39:11.592986+0000","last_clean":"2024-09-10T08:39:11.592986+0000","last_became_active":"2024-09-10T08:36:56.154727+0000","last_became_peered":"2024-09-10T08:36:56.154727+0000","last_unstale":"2024-09-10T08:39:11.592986+0000","last_undegraded":"2024-09-10T08:39:11.592986+0000","last_fullsized":"2024-09-10T08:39:11.592986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:35:39.992683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.576067+0000","last_change":"2024-09-10T08:36:56.154394+0000","last_active":"2024-09-10T08:39:11.576067+0000","last_peered":"2024-09-10T08:39:11.576067+0000","last_clean":"2024-09-10T08:39:11.576067+0000","last_became_active":"2024-09-10T08:36:56.153855+0000","last_became_peered":"2024-09-10T08:36:56.153855+0000","last_unstale":"2024-09-10T08:39:11.576067+0000","last_undegraded":"2024-09-10T08:39:11.576067+0000","last_fullsized":"2024-09-10T08:39:11.576067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:14.987147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592634+0000","last_change":"2024-09-10T08:36:56.157517+0000","last_active":"2024-09-10T08:39:11.592634+0000","last_peered":"2024-09-10T08:39:11.592634+0000","last_clean":"2024-09-10T08:39:11.592634+0000","last_became_active":"2024-09-10T08:36:56.157133+0000","last_became_peered":"2024-09-10T08:36:56.157133+0000","last_unstale":"2024-09-10T08:39:11.592634+0000","last_undegraded":"2024-09-10T08:39:11.592634+0000","last_fullsized":"2024-09-10T08:39:11.592634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:17:29.573719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76539,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":764,"num_read_kb":463,"num_write":188,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":231893,"data_compressed":5667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282455,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416434,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":547326,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678070,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28212,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716940,"statfs":{"total":95995035648,"available":95966146560,"internally_reserved":0,"allocated":1216512,"data_stored":1215011,"data_compressed":67339,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841596,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":543466,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103234,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":546900,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332167,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":1144631,"data_compressed":65450,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495692,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":542738,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757330,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":1139292,"data_compressed":65450,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920854,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":548897,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149787,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":542932,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":615418,"data_compressed":31413,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712864,"statfs":{"total":95995035648,"available":95961972736,"internally_reserved":0,"allocated":1130496,"data_stored":615666,"data_compressed":31413,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1077248,"data_stored":544126,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":73018,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":73908,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73803,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:39:55.900 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T08:39:55.900 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:39:56.267 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:56.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.868+0000 7fbf2f400640 1 -- 172.21.15.89:0/1090383246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf28104010 msgr2=0x7fbf281043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:56.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.868+0000 7fbf2f400640 1 --2- 172.21.15.89:0/1090383246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf28104010 0x7fbf281043f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009900 tx=0x7fbf1c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:39:56.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.868+0000 7fbf2f400640 1 -- 172.21.15.89:0/1090383246 shutdown_connections 2024-09-10T08:39:56.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.868+0000 7fbf2f400640 1 --2- 172.21.15.89:0/1090383246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf28109140 0x7fbf28110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:56.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.868+0000 7fbf2f400640 1 --2- 172.21.15.89:0/1090383246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf281049c0 0x7fbf28108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:56.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.868+0000 7fbf2f400640 1 --2- 172.21.15.89:0/1090383246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf28104010 0x7fbf281043f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:56.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.869+0000 7fbf2f400640 1 -- 172.21.15.89:0/1090383246 >> 172.21.15.89:0/1090383246 conn(0x7fbf280ffa50 msgr2=0x7fbf28101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:56.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.869+0000 7fbf2f400640 1 -- 172.21.15.89:0/1090383246 shutdown_connections 2024-09-10T08:39:56.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.869+0000 7fbf2f400640 1 -- 172.21.15.89:0/1090383246 wait complete. 2024-09-10T08:39:56.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.869+0000 7fbf2f400640 1 Processor -- start 2024-09-10T08:39:56.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.870+0000 7fbf2f400640 1 -- start start 2024-09-10T08:39:56.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.870+0000 7fbf2f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf28104010 0x7fbf2819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:56.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.870+0000 7fbf2f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf281049c0 0x7fbf2819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:56.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.870+0000 7fbf2f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf28109140 0x7fbf2819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:56.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.870+0000 7fbf2f400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbf28114140 con 0x7fbf281049c0 2024-09-10T08:39:56.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.870+0000 7fbf2f400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbf28113fc0 con 0x7fbf28109140 2024-09-10T08:39:56.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.870+0000 7fbf2f400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbf281142c0 con 0x7fbf28104010 2024-09-10T08:39:56.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.870+0000 7fbf27e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf281049c0 0x7fbf2819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:56.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.870+0000 7fbf27e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf281049c0 0x7fbf2819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49584/0 (socket says 172.21.15.89:49584) 2024-09-10T08:39:56.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.870+0000 7fbf27e00640 1 -- 172.21.15.89:0/3211197352 learned_addr learned my addr 172.21.15.89:0/3211197352 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:56.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.870+0000 7fbf2ca00640 1 --2- 172.21.15.89:0/3211197352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf28104010 0x7fbf2819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:56.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.870+0000 7fbf2d400640 1 --2- 172.21.15.89:0/3211197352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf28109140 0x7fbf2819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:56.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.871+0000 7fbf27e00640 1 -- 172.21.15.89:0/3211197352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf28104010 msgr2=0x7fbf2819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:56.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.871+0000 7fbf27e00640 1 --2- 172.21.15.89:0/3211197352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf28104010 0x7fbf2819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:56.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.871+0000 7fbf27e00640 1 -- 172.21.15.89:0/3211197352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf28109140 msgr2=0x7fbf2819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:56.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.871+0000 7fbf27e00640 1 --2- 172.21.15.89:0/3211197352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf28109140 0x7fbf2819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:56.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.871+0000 7fbf27e00640 1 -- 172.21.15.89:0/3211197352 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2819f5d0 con 0x7fbf281049c0 2024-09-10T08:39:56.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.871+0000 7fbf27e00640 1 --2- 172.21.15.89:0/3211197352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf281049c0 0x7fbf2819ab60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400d950 tx=0x7fbf1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:56.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.871+0000 7fbf25600640 1 -- 172.21.15.89:0/3211197352 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14017030 con 0x7fbf281049c0 2024-09-10T08:39:56.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.871+0000 7fbf25600640 1 -- 172.21.15.89:0/3211197352 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf14004020 con 0x7fbf281049c0 2024-09-10T08:39:56.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.871+0000 7fbf2f400640 1 -- 172.21.15.89:0/3211197352 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2819f8c0 con 0x7fbf281049c0 2024-09-10T08:39:56.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.871+0000 7fbf2f400640 1 -- 172.21.15.89:0/3211197352 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf281a71a0 con 0x7fbf281049c0 2024-09-10T08:39:56.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.872+0000 7fbf25600640 1 -- 172.21.15.89:0/3211197352 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf140047a0 con 0x7fbf281049c0 2024-09-10T08:39:56.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.873+0000 7fbf2f400640 1 -- 172.21.15.89:0/3211197352 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf08005290 con 0x7fbf281049c0 2024-09-10T08:39:56.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.875+0000 7fbf25600640 1 -- 172.21.15.89:0/3211197352 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fbf140041d0 con 0x7fbf281049c0 2024-09-10T08:39:56.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.876+0000 7fbf25600640 1 --2- 172.21.15.89:0/3211197352 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbf100a2c10 0x7fbf100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:56.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.877+0000 7fbf25600640 1 -- 172.21.15.89:0/3211197352 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fbf140cf440 con 0x7fbf281049c0 2024-09-10T08:39:56.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.877+0000 7fbf2ca00640 1 --2- 172.21.15.89:0/3211197352 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbf100a2c10 0x7fbf100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:56.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.877+0000 7fbf2ca00640 1 --2- 172.21.15.89:0/3211197352 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbf100a2c10 0x7fbf100a50d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009230 tx=0x7fbf1c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:56.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:56.878+0000 7fbf25600640 1 -- 172.21.15.89:0/3211197352 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbf1409be70 con 0x7fbf281049c0 2024-09-10T08:39:57.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:56 smithi111 ceph-mon[35220]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:39:57.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:56 smithi111 ceph-mon[35220]: pgmap v393: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-10T08:39:57.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:57.051+0000 7fbf2f400640 1 -- 172.21.15.89:0/3211197352 --> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf08000bc0 con 0x7fbf100a2c10 2024-09-10T08:39:57.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:56 smithi089 ceph-mon[30757]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:39:57.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:56 smithi089 ceph-mon[30757]: pgmap v393: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-10T08:39:57.060 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:57.060+0000 7fbf25600640 1 -- 172.21.15.89:0/3211197352 <== mgr.14150 v2:172.21.15.89:6800/1633345099 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340806 (secure 0 0 0) 0x7fbf08000bc0 con 0x7fbf100a2c10 2024-09-10T08:39:57.065 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:39:57.067 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-09-10T08:39:57.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:57.064+0000 7fbf2f400640 1 -- 172.21.15.89:0/3211197352 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbf100a2c10 msgr2=0x7fbf100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:57.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:57.064+0000 7fbf2f400640 1 --2- 172.21.15.89:0/3211197352 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbf100a2c10 0x7fbf100a50d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009230 tx=0x7fbf1c0058d0 comp rx=0 tx=0).stop 2024-09-10T08:39:57.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:57.064+0000 7fbf2f400640 1 -- 172.21.15.89:0/3211197352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf281049c0 msgr2=0x7fbf2819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:57.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:57.064+0000 7fbf2f400640 1 --2- 172.21.15.89:0/3211197352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf281049c0 0x7fbf2819ab60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400d950 tx=0x7fbf1400de10 comp rx=0 tx=0).stop 2024-09-10T08:39:57.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:57.064+0000 7fbf2f400640 1 -- 172.21.15.89:0/3211197352 shutdown_connections 2024-09-10T08:39:57.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:57.064+0000 7fbf2f400640 1 --2- 172.21.15.89:0/3211197352 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbf100a2c10 0x7fbf100a50d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:57.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:57.064+0000 7fbf2f400640 1 --2- 172.21.15.89:0/3211197352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf28109140 0x7fbf2819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:57.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:57.064+0000 7fbf2f400640 1 --2- 172.21.15.89:0/3211197352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf281049c0 0x7fbf2819ab60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:57.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:57.064+0000 7fbf2f400640 1 --2- 172.21.15.89:0/3211197352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf28104010 0x7fbf2819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:57.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:57.064+0000 7fbf2f400640 1 -- 172.21.15.89:0/3211197352 >> 172.21.15.89:0/3211197352 conn(0x7fbf280ffa50 msgr2=0x7fbf28100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:57.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:57.065+0000 7fbf2f400640 1 -- 172.21.15.89:0/3211197352 shutdown_connections 2024-09-10T08:39:57.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:57.065+0000 7fbf2f400640 1 -- 172.21.15.89:0/3211197352 wait complete. 2024-09-10T08:39:57.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:56 smithi149 ceph-mon[36167]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:39:57.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:56 smithi149 ceph-mon[36167]: pgmap v393: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-10T08:39:57.818 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-10T08:39:55.615915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668279,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":912,"num_read_kb":585,"num_write":372,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181900,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":800,"ondisk_log_size":800,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357984,"kb_used_data":12896,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583840,"statfs":{"total":1151940427776,"available":1151573852160,"internally_reserved":0,"allocated":13205504,"data_stored":8547541,"data_compressed":467733,"data_compressed_allocated":638976,"data_compressed_original":2777088,"omap_allocated":19142,"internal_metadata":353154362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8365,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":54,"num_write":3,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003219"},"pg_stats":[{"pgid":"2.1f","version":"94'9","reported_seq":68,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:00.658898+0000","last_change":"2024-09-10T08:38:43.663079+0000","last_active":"2024-09-10T08:39:00.658898+0000","last_peered":"2024-09-10T08:39:00.658898+0000","last_clean":"2024-09-10T08:39:00.658898+0000","last_became_active":"2024-09-10T08:38:43.656506+0000","last_became_peered":"2024-09-10T08:38:43.656506+0000","last_unstale":"2024-09-10T08:39:00.658898+0000","last_undegraded":"2024-09-10T08:39:00.658898+0000","last_fullsized":"2024-09-10T08:39:00.658898+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:36.881564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.579344+0000","last_change":"2024-09-10T08:36:56.145816+0000","last_active":"2024-09-10T08:39:11.579344+0000","last_peered":"2024-09-10T08:39:11.579344+0000","last_clean":"2024-09-10T08:39:11.579344+0000","last_became_active":"2024-09-10T08:36:56.145462+0000","last_became_peered":"2024-09-10T08:36:56.145462+0000","last_unstale":"2024-09-10T08:39:11.579344+0000","last_undegraded":"2024-09-10T08:39:11.579344+0000","last_fullsized":"2024-09-10T08:39:11.579344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:53.890190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629879+0000","last_change":"2024-09-10T08:37:05.252761+0000","last_active":"2024-09-10T08:38:43.629879+0000","last_peered":"2024-09-10T08:38:43.629879+0000","last_clean":"2024-09-10T08:38:43.629879+0000","last_became_active":"2024-09-10T08:37:05.252179+0000","last_became_peered":"2024-09-10T08:37:05.252179+0000","last_unstale":"2024-09-10T08:38:43.629879+0000","last_undegraded":"2024-09-10T08:38:43.629879+0000","last_fullsized":"2024-09-10T08:38:43.629879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:52.854828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631850+0000","last_change":"2024-09-10T08:36:53.735650+0000","last_active":"2024-09-10T08:38:43.631850+0000","last_peered":"2024-09-10T08:38:43.631850+0000","last_clean":"2024-09-10T08:38:43.631850+0000","last_became_active":"2024-09-10T08:36:53.735128+0000","last_became_peered":"2024-09-10T08:36:53.735128+0000","last_unstale":"2024-09-10T08:38:43.631850+0000","last_undegraded":"2024-09-10T08:38:43.631850+0000","last_fullsized":"2024-09-10T08:38:43.631850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:27.309591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592923+0000","last_change":"2024-09-10T08:36:56.153364+0000","last_active":"2024-09-10T08:39:11.592923+0000","last_peered":"2024-09-10T08:39:11.592923+0000","last_clean":"2024-09-10T08:39:11.592923+0000","last_became_active":"2024-09-10T08:36:56.152979+0000","last_became_peered":"2024-09-10T08:36:56.152979+0000","last_unstale":"2024-09-10T08:39:11.592923+0000","last_undegraded":"2024-09-10T08:39:11.592923+0000","last_fullsized":"2024-09-10T08:39:11.592923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:12.172620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654774+0000","last_change":"2024-09-10T08:37:05.254551+0000","last_active":"2024-09-10T08:38:43.654774+0000","last_peered":"2024-09-10T08:38:43.654774+0000","last_clean":"2024-09-10T08:38:43.654774+0000","last_became_active":"2024-09-10T08:37:05.254055+0000","last_became_peered":"2024-09-10T08:37:05.254055+0000","last_unstale":"2024-09-10T08:38:43.654774+0000","last_undegraded":"2024-09-10T08:38:43.654774+0000","last_fullsized":"2024-09-10T08:38:43.654774+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:08.472502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626522+0000","last_change":"2024-09-10T08:36:53.722774+0000","last_active":"2024-09-10T08:38:43.626522+0000","last_peered":"2024-09-10T08:38:43.626522+0000","last_clean":"2024-09-10T08:38:43.626522+0000","last_became_active":"2024-09-10T08:36:53.722459+0000","last_became_peered":"2024-09-10T08:36:53.722459+0000","last_unstale":"2024-09-10T08:38:43.626522+0000","last_undegraded":"2024-09-10T08:38:43.626522+0000","last_fullsized":"2024-09-10T08:38:43.626522+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:26.612354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592240+0000","last_change":"2024-09-10T08:36:56.143757+0000","last_active":"2024-09-10T08:39:11.592240+0000","last_peered":"2024-09-10T08:39:11.592240+0000","last_clean":"2024-09-10T08:39:11.592240+0000","last_became_active":"2024-09-10T08:36:56.143430+0000","last_became_peered":"2024-09-10T08:36:56.143430+0000","last_unstale":"2024-09-10T08:39:11.592240+0000","last_undegraded":"2024-09-10T08:39:11.592240+0000","last_fullsized":"2024-09-10T08:39:11.592240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:49.737298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655120+0000","last_change":"2024-09-10T08:37:05.254503+0000","last_active":"2024-09-10T08:38:43.655120+0000","last_peered":"2024-09-10T08:38:43.655120+0000","last_clean":"2024-09-10T08:38:43.655120+0000","last_became_active":"2024-09-10T08:37:05.254070+0000","last_became_peered":"2024-09-10T08:37:05.254070+0000","last_unstale":"2024-09-10T08:38:43.655120+0000","last_undegraded":"2024-09-10T08:38:43.655120+0000","last_fullsized":"2024-09-10T08:38:43.655120+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:16.741698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626438+0000","last_change":"2024-09-10T08:36:53.716742+0000","last_active":"2024-09-10T08:38:43.626438+0000","last_peered":"2024-09-10T08:38:43.626438+0000","last_clean":"2024-09-10T08:38:43.626438+0000","last_became_active":"2024-09-10T08:36:53.716439+0000","last_became_peered":"2024-09-10T08:36:53.716439+0000","last_unstale":"2024-09-10T08:38:43.626438+0000","last_undegraded":"2024-09-10T08:38:43.626438+0000","last_fullsized":"2024-09-10T08:38:43.626438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:46.882805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.581828+0000","last_change":"2024-09-10T08:36:56.146841+0000","last_active":"2024-09-10T08:39:11.581828+0000","last_peered":"2024-09-10T08:39:11.581828+0000","last_clean":"2024-09-10T08:39:11.581828+0000","last_became_active":"2024-09-10T08:36:56.146531+0000","last_became_peered":"2024-09-10T08:36:56.146531+0000","last_unstale":"2024-09-10T08:39:11.581828+0000","last_undegraded":"2024-09-10T08:39:11.581828+0000","last_fullsized":"2024-09-10T08:39:11.581828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:37.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654877+0000","last_change":"2024-09-10T08:37:43.223181+0000","last_active":"2024-09-10T08:38:43.654877+0000","last_peered":"2024-09-10T08:38:43.654877+0000","last_clean":"2024-09-10T08:38:43.654877+0000","last_became_active":"2024-09-10T08:37:43.222794+0000","last_became_peered":"2024-09-10T08:37:43.222794+0000","last_unstale":"2024-09-10T08:38:43.654877+0000","last_undegraded":"2024-09-10T08:38:43.654877+0000","last_fullsized":"2024-09-10T08:38:43.654877+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:22.204517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593593+0000","last_change":"2024-09-10T08:36:56.142161+0000","last_active":"2024-09-10T08:39:11.593593+0000","last_peered":"2024-09-10T08:39:11.593593+0000","last_clean":"2024-09-10T08:39:11.593593+0000","last_became_active":"2024-09-10T08:36:56.141838+0000","last_became_peered":"2024-09-10T08:36:56.141838+0000","last_unstale":"2024-09-10T08:39:11.593593+0000","last_undegraded":"2024-09-10T08:39:11.593593+0000","last_fullsized":"2024-09-10T08:39:11.593593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:09.909747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626125+0000","last_change":"2024-09-10T08:36:53.722238+0000","last_active":"2024-09-10T08:38:43.626125+0000","last_peered":"2024-09-10T08:38:43.626125+0000","last_clean":"2024-09-10T08:38:43.626125+0000","last_became_active":"2024-09-10T08:36:53.721859+0000","last_became_peered":"2024-09-10T08:36:53.721859+0000","last_unstale":"2024-09-10T08:38:43.626125+0000","last_undegraded":"2024-09-10T08:38:43.626125+0000","last_fullsized":"2024-09-10T08:38:43.626125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:50.615090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655183+0000","last_change":"2024-09-10T08:37:05.414041+0000","last_active":"2024-09-10T08:38:43.655183+0000","last_peered":"2024-09-10T08:38:43.655183+0000","last_clean":"2024-09-10T08:38:43.655183+0000","last_became_active":"2024-09-10T08:37:05.413286+0000","last_became_peered":"2024-09-10T08:37:05.413286+0000","last_unstale":"2024-09-10T08:38:43.655183+0000","last_undegraded":"2024-09-10T08:38:43.655183+0000","last_fullsized":"2024-09-10T08:38:43.655183+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:44.916102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.572936+0000","last_change":"2024-09-10T08:36:56.158278+0000","last_active":"2024-09-10T08:39:11.572936+0000","last_peered":"2024-09-10T08:39:11.572936+0000","last_clean":"2024-09-10T08:39:11.572936+0000","last_became_active":"2024-09-10T08:36:56.157946+0000","last_became_peered":"2024-09-10T08:36:56.157946+0000","last_unstale":"2024-09-10T08:39:11.572936+0000","last_undegraded":"2024-09-10T08:39:11.572936+0000","last_fullsized":"2024-09-10T08:39:11.572936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:42.512513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"96'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:38.713339+0000","last_change":"2024-09-10T08:38:43.635077+0000","last_active":"2024-09-10T08:39:38.713339+0000","last_peered":"2024-09-10T08:39:38.713339+0000","last_clean":"2024-09-10T08:39:38.713339+0000","last_became_active":"2024-09-10T08:38:43.631957+0000","last_became_peered":"2024-09-10T08:38:43.631957+0000","last_unstale":"2024-09-10T08:39:38.713339+0000","last_undegraded":"2024-09-10T08:39:38.713339+0000","last_fullsized":"2024-09-10T08:39:38.713339+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:50.837247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655126+0000","last_change":"2024-09-10T08:37:42.671399+0000","last_active":"2024-09-10T08:38:43.655126+0000","last_peered":"2024-09-10T08:38:43.655126+0000","last_clean":"2024-09-10T08:38:43.655126+0000","last_became_active":"2024-09-10T08:37:42.670865+0000","last_became_peered":"2024-09-10T08:37:42.670865+0000","last_unstale":"2024-09-10T08:38:43.655126+0000","last_undegraded":"2024-09-10T08:38:43.655126+0000","last_fullsized":"2024-09-10T08:38:43.655126+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:16.231538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.577960+0000","last_change":"2024-09-10T08:36:56.144856+0000","last_active":"2024-09-10T08:39:11.577960+0000","last_peered":"2024-09-10T08:39:11.577960+0000","last_clean":"2024-09-10T08:39:11.577960+0000","last_became_active":"2024-09-10T08:36:56.144531+0000","last_became_peered":"2024-09-10T08:36:56.144531+0000","last_unstale":"2024-09-10T08:39:11.577960+0000","last_undegraded":"2024-09-10T08:39:11.577960+0000","last_fullsized":"2024-09-10T08:39:11.577960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:24.586006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631541+0000","last_change":"2024-09-10T08:36:53.721093+0000","last_active":"2024-09-10T08:38:43.631541+0000","last_peered":"2024-09-10T08:38:43.631541+0000","last_clean":"2024-09-10T08:38:43.631541+0000","last_became_active":"2024-09-10T08:36:53.720538+0000","last_became_peered":"2024-09-10T08:36:53.720538+0000","last_unstale":"2024-09-10T08:38:43.631541+0000","last_undegraded":"2024-09-10T08:38:43.631541+0000","last_fullsized":"2024-09-10T08:38:43.631541+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:58.492463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629902+0000","last_change":"2024-09-10T08:37:05.254011+0000","last_active":"2024-09-10T08:38:43.629902+0000","last_peered":"2024-09-10T08:38:43.629902+0000","last_clean":"2024-09-10T08:38:43.629902+0000","last_became_active":"2024-09-10T08:37:05.253066+0000","last_became_peered":"2024-09-10T08:37:05.253066+0000","last_unstale":"2024-09-10T08:38:43.629902+0000","last_undegraded":"2024-09-10T08:38:43.629902+0000","last_fullsized":"2024-09-10T08:38:43.629902+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:16.221302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.553299+0000","last_change":"2024-09-10T08:36:56.141814+0000","last_active":"2024-09-10T08:39:11.553299+0000","last_peered":"2024-09-10T08:39:11.553299+0000","last_clean":"2024-09-10T08:39:11.553299+0000","last_became_active":"2024-09-10T08:36:56.141480+0000","last_became_peered":"2024-09-10T08:36:56.141480+0000","last_unstale":"2024-09-10T08:39:11.553299+0000","last_undegraded":"2024-09-10T08:39:11.553299+0000","last_fullsized":"2024-09-10T08:39:11.553299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:40.594354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.636347+0000","last_change":"2024-09-10T08:38:43.635917+0000","last_active":"2024-09-10T08:38:43.636347+0000","last_peered":"2024-09-10T08:38:43.636347+0000","last_clean":"2024-09-10T08:38:43.636347+0000","last_became_active":"2024-09-10T08:38:43.633406+0000","last_became_peered":"2024-09-10T08:38:43.633406+0000","last_unstale":"2024-09-10T08:38:43.636347+0000","last_undegraded":"2024-09-10T08:38:43.636347+0000","last_fullsized":"2024-09-10T08:38:43.636347+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:44.619072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655297+0000","last_change":"2024-09-10T08:37:05.414181+0000","last_active":"2024-09-10T08:38:43.655297+0000","last_peered":"2024-09-10T08:38:43.655297+0000","last_clean":"2024-09-10T08:38:43.655297+0000","last_became_active":"2024-09-10T08:37:05.413691+0000","last_became_peered":"2024-09-10T08:37:05.413691+0000","last_unstale":"2024-09-10T08:38:43.655297+0000","last_undegraded":"2024-09-10T08:38:43.655297+0000","last_fullsized":"2024-09-10T08:38:43.655297+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:30.744949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584567+0000","last_change":"2024-09-10T08:36:56.153956+0000","last_active":"2024-09-10T08:39:11.584567+0000","last_peered":"2024-09-10T08:39:11.584567+0000","last_clean":"2024-09-10T08:39:11.584567+0000","last_became_active":"2024-09-10T08:36:56.153609+0000","last_became_peered":"2024-09-10T08:36:56.153609+0000","last_unstale":"2024-09-10T08:39:11.584567+0000","last_undegraded":"2024-09-10T08:39:11.584567+0000","last_fullsized":"2024-09-10T08:39:11.584567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:18.185704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655520+0000","last_change":"2024-09-10T08:36:53.733217+0000","last_active":"2024-09-10T08:38:43.655520+0000","last_peered":"2024-09-10T08:38:43.655520+0000","last_clean":"2024-09-10T08:38:43.655520+0000","last_became_active":"2024-09-10T08:36:53.732851+0000","last_became_peered":"2024-09-10T08:36:53.732851+0000","last_unstale":"2024-09-10T08:38:43.655520+0000","last_undegraded":"2024-09-10T08:38:43.655520+0000","last_fullsized":"2024-09-10T08:38:43.655520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:04:01.132196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629889+0000","last_change":"2024-09-10T08:37:05.241868+0000","last_active":"2024-09-10T08:38:43.629889+0000","last_peered":"2024-09-10T08:38:43.629889+0000","last_clean":"2024-09-10T08:38:43.629889+0000","last_became_active":"2024-09-10T08:37:05.241412+0000","last_became_peered":"2024-09-10T08:37:05.241412+0000","last_unstale":"2024-09-10T08:38:43.629889+0000","last_undegraded":"2024-09-10T08:38:43.629889+0000","last_fullsized":"2024-09-10T08:38:43.629889+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:52.543353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.575701+0000","last_change":"2024-09-10T08:36:56.150722+0000","last_active":"2024-09-10T08:39:11.575701+0000","last_peered":"2024-09-10T08:39:11.575701+0000","last_clean":"2024-09-10T08:39:11.575701+0000","last_became_active":"2024-09-10T08:36:56.150423+0000","last_became_peered":"2024-09-10T08:36:56.150423+0000","last_unstale":"2024-09-10T08:39:11.575701+0000","last_undegraded":"2024-09-10T08:39:11.575701+0000","last_fullsized":"2024-09-10T08:39:11.575701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:50.272841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'13","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:38.175435+0000","last_change":"2024-09-10T08:36:53.722477+0000","last_active":"2024-09-10T08:39:38.175435+0000","last_peered":"2024-09-10T08:39:38.175435+0000","last_clean":"2024-09-10T08:39:38.175435+0000","last_became_active":"2024-09-10T08:36:53.722118+0000","last_became_peered":"2024-09-10T08:36:53.722118+0000","last_unstale":"2024-09-10T08:39:38.175435+0000","last_undegraded":"2024-09-10T08:39:38.175435+0000","last_fullsized":"2024-09-10T08:39:38.175435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:06.631646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630992+0000","last_change":"2024-09-10T08:37:05.415656+0000","last_active":"2024-09-10T08:38:43.630992+0000","last_peered":"2024-09-10T08:38:43.630992+0000","last_clean":"2024-09-10T08:38:43.630992+0000","last_became_active":"2024-09-10T08:37:05.413537+0000","last_became_peered":"2024-09-10T08:37:05.413537+0000","last_unstale":"2024-09-10T08:38:43.630992+0000","last_undegraded":"2024-09-10T08:38:43.630992+0000","last_fullsized":"2024-09-10T08:38:43.630992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:04.284410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585069+0000","last_change":"2024-09-10T08:36:56.144157+0000","last_active":"2024-09-10T08:39:11.585069+0000","last_peered":"2024-09-10T08:39:11.585069+0000","last_clean":"2024-09-10T08:39:11.585069+0000","last_became_active":"2024-09-10T08:36:56.140332+0000","last_became_peered":"2024-09-10T08:36:56.140332+0000","last_unstale":"2024-09-10T08:39:11.585069+0000","last_undegraded":"2024-09-10T08:39:11.585069+0000","last_fullsized":"2024-09-10T08:39:11.585069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:09:26.097987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626517+0000","last_change":"2024-09-10T08:36:53.735465+0000","last_active":"2024-09-10T08:38:43.626517+0000","last_peered":"2024-09-10T08:38:43.626517+0000","last_clean":"2024-09-10T08:38:43.626517+0000","last_became_active":"2024-09-10T08:36:53.735160+0000","last_became_peered":"2024-09-10T08:36:53.735160+0000","last_unstale":"2024-09-10T08:38:43.626517+0000","last_undegraded":"2024-09-10T08:38:43.626517+0000","last_fullsized":"2024-09-10T08:38:43.626517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:26:52.332085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627959+0000","last_change":"2024-09-10T08:37:05.247671+0000","last_active":"2024-09-10T08:38:43.627959+0000","last_peered":"2024-09-10T08:38:43.627959+0000","last_clean":"2024-09-10T08:38:43.627959+0000","last_became_active":"2024-09-10T08:37:05.247217+0000","last_became_peered":"2024-09-10T08:37:05.247217+0000","last_unstale":"2024-09-10T08:38:43.627959+0000","last_undegraded":"2024-09-10T08:38:43.627959+0000","last_fullsized":"2024-09-10T08:38:43.627959+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:20.881304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.579998+0000","last_change":"2024-09-10T08:36:56.145420+0000","last_active":"2024-09-10T08:39:11.579998+0000","last_peered":"2024-09-10T08:39:11.579998+0000","last_clean":"2024-09-10T08:39:11.579998+0000","last_became_active":"2024-09-10T08:36:56.145074+0000","last_became_peered":"2024-09-10T08:36:56.145074+0000","last_unstale":"2024-09-10T08:39:11.579998+0000","last_undegraded":"2024-09-10T08:39:11.579998+0000","last_fullsized":"2024-09-10T08:39:11.579998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:13.849977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629812+0000","last_change":"2024-09-10T08:36:53.738237+0000","last_active":"2024-09-10T08:38:43.629812+0000","last_peered":"2024-09-10T08:38:43.629812+0000","last_clean":"2024-09-10T08:38:43.629812+0000","last_became_active":"2024-09-10T08:36:53.737858+0000","last_became_peered":"2024-09-10T08:36:53.737858+0000","last_unstale":"2024-09-10T08:38:43.629812+0000","last_undegraded":"2024-09-10T08:38:43.629812+0000","last_fullsized":"2024-09-10T08:38:43.629812+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:35.345661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.625982+0000","last_change":"2024-09-10T08:37:05.413512+0000","last_active":"2024-09-10T08:38:43.625982+0000","last_peered":"2024-09-10T08:38:43.625982+0000","last_clean":"2024-09-10T08:38:43.625982+0000","last_became_active":"2024-09-10T08:37:05.413010+0000","last_became_peered":"2024-09-10T08:37:05.413010+0000","last_unstale":"2024-09-10T08:38:43.625982+0000","last_undegraded":"2024-09-10T08:38:43.625982+0000","last_fullsized":"2024-09-10T08:38:43.625982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:05.349404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583802+0000","last_change":"2024-09-10T08:36:56.157741+0000","last_active":"2024-09-10T08:39:11.583802+0000","last_peered":"2024-09-10T08:39:11.583802+0000","last_clean":"2024-09-10T08:39:11.583802+0000","last_became_active":"2024-09-10T08:36:56.157295+0000","last_became_peered":"2024-09-10T08:36:56.157295+0000","last_unstale":"2024-09-10T08:39:11.583802+0000","last_undegraded":"2024-09-10T08:39:11.583802+0000","last_fullsized":"2024-09-10T08:39:11.583802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:31.845864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631834+0000","last_change":"2024-09-10T08:36:53.722373+0000","last_active":"2024-09-10T08:38:43.631834+0000","last_peered":"2024-09-10T08:38:43.631834+0000","last_clean":"2024-09-10T08:38:43.631834+0000","last_became_active":"2024-09-10T08:36:53.721975+0000","last_became_peered":"2024-09-10T08:36:53.721975+0000","last_unstale":"2024-09-10T08:38:43.631834+0000","last_undegraded":"2024-09-10T08:38:43.631834+0000","last_fullsized":"2024-09-10T08:38:43.631834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:44.593018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628351+0000","last_change":"2024-09-10T08:37:05.255362+0000","last_active":"2024-09-10T08:38:43.628351+0000","last_peered":"2024-09-10T08:38:43.628351+0000","last_clean":"2024-09-10T08:38:43.628351+0000","last_became_active":"2024-09-10T08:37:05.254967+0000","last_became_peered":"2024-09-10T08:37:05.254967+0000","last_unstale":"2024-09-10T08:38:43.628351+0000","last_undegraded":"2024-09-10T08:38:43.628351+0000","last_fullsized":"2024-09-10T08:38:43.628351+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:31:41.850961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.578715+0000","last_change":"2024-09-10T08:36:56.147699+0000","last_active":"2024-09-10T08:39:11.578715+0000","last_peered":"2024-09-10T08:39:11.578715+0000","last_clean":"2024-09-10T08:39:11.578715+0000","last_became_active":"2024-09-10T08:36:56.147349+0000","last_became_peered":"2024-09-10T08:36:56.147349+0000","last_unstale":"2024-09-10T08:39:11.578715+0000","last_undegraded":"2024-09-10T08:39:11.578715+0000","last_fullsized":"2024-09-10T08:39:11.578715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:25.248588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"96'6","reported_seq":169,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:48.903998+0000","last_change":"2024-09-10T08:36:53.736843+0000","last_active":"2024-09-10T08:39:48.903998+0000","last_peered":"2024-09-10T08:39:48.903998+0000","last_clean":"2024-09-10T08:39:48.903998+0000","last_became_active":"2024-09-10T08:36:53.736340+0000","last_became_peered":"2024-09-10T08:36:53.736340+0000","last_unstale":"2024-09-10T08:39:48.903998+0000","last_undegraded":"2024-09-10T08:39:48.903998+0000","last_fullsized":"2024-09-10T08:39:48.903998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:24.679679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628575+0000","last_change":"2024-09-10T08:37:05.414321+0000","last_active":"2024-09-10T08:38:43.628575+0000","last_peered":"2024-09-10T08:38:43.628575+0000","last_clean":"2024-09-10T08:38:43.628575+0000","last_became_active":"2024-09-10T08:37:05.413868+0000","last_became_peered":"2024-09-10T08:37:05.413868+0000","last_unstale":"2024-09-10T08:38:43.628575+0000","last_undegraded":"2024-09-10T08:38:43.628575+0000","last_fullsized":"2024-09-10T08:38:43.628575+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:53.988077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583174+0000","last_change":"2024-09-10T08:36:56.141910+0000","last_active":"2024-09-10T08:39:11.583174+0000","last_peered":"2024-09-10T08:39:11.583174+0000","last_clean":"2024-09-10T08:39:11.583174+0000","last_became_active":"2024-09-10T08:36:56.141579+0000","last_became_peered":"2024-09-10T08:36:56.141579+0000","last_unstale":"2024-09-10T08:39:11.583174+0000","last_undegraded":"2024-09-10T08:39:11.583174+0000","last_fullsized":"2024-09-10T08:39:11.583174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:18.464831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631756+0000","last_change":"2024-09-10T08:36:53.720102+0000","last_active":"2024-09-10T08:38:43.631756+0000","last_peered":"2024-09-10T08:38:43.631756+0000","last_clean":"2024-09-10T08:38:43.631756+0000","last_became_active":"2024-09-10T08:36:53.719764+0000","last_became_peered":"2024-09-10T08:36:53.719764+0000","last_unstale":"2024-09-10T08:38:43.631756+0000","last_undegraded":"2024-09-10T08:38:43.631756+0000","last_fullsized":"2024-09-10T08:38:43.631756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:41:03.609338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655205+0000","last_change":"2024-09-10T08:37:05.413818+0000","last_active":"2024-09-10T08:38:43.655205+0000","last_peered":"2024-09-10T08:38:43.655205+0000","last_clean":"2024-09-10T08:38:43.655205+0000","last_became_active":"2024-09-10T08:37:05.413267+0000","last_became_peered":"2024-09-10T08:37:05.413267+0000","last_unstale":"2024-09-10T08:38:43.655205+0000","last_undegraded":"2024-09-10T08:38:43.655205+0000","last_fullsized":"2024-09-10T08:38:43.655205+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:54.309686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.590486+0000","last_change":"2024-09-10T08:36:56.149286+0000","last_active":"2024-09-10T08:39:11.590486+0000","last_peered":"2024-09-10T08:39:11.590486+0000","last_clean":"2024-09-10T08:39:11.590486+0000","last_became_active":"2024-09-10T08:36:56.146275+0000","last_became_peered":"2024-09-10T08:36:56.146275+0000","last_unstale":"2024-09-10T08:39:11.590486+0000","last_undegraded":"2024-09-10T08:39:11.590486+0000","last_fullsized":"2024-09-10T08:39:11.590486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:19.468224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655269+0000","last_change":"2024-09-10T08:36:53.719756+0000","last_active":"2024-09-10T08:38:43.655269+0000","last_peered":"2024-09-10T08:38:43.655269+0000","last_clean":"2024-09-10T08:38:43.655269+0000","last_became_active":"2024-09-10T08:36:53.719302+0000","last_became_peered":"2024-09-10T08:36:53.719302+0000","last_unstale":"2024-09-10T08:38:43.655269+0000","last_undegraded":"2024-09-10T08:38:43.655269+0000","last_fullsized":"2024-09-10T08:38:43.655269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:47.658081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631788+0000","last_change":"2024-09-10T08:35:39.150715+0000","last_active":"2024-09-10T08:38:43.631788+0000","last_peered":"2024-09-10T08:38:43.631788+0000","last_clean":"2024-09-10T08:38:43.631788+0000","last_became_active":"2024-09-10T08:35:39.150206+0000","last_became_peered":"2024-09-10T08:35:39.150206+0000","last_unstale":"2024-09-10T08:38:43.631788+0000","last_undegraded":"2024-09-10T08:38:43.631788+0000","last_fullsized":"2024-09-10T08:38:43.631788+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:32:02.110870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:32:02.110870+0000","last_clean_scrub_stamp":"2024-09-10T08:32:02.110870+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:20.762043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":34,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631998+0000","last_change":"2024-09-10T08:37:42.846547+0000","last_active":"2024-09-10T08:38:43.631998+0000","last_peered":"2024-09-10T08:38:43.631998+0000","last_clean":"2024-09-10T08:38:43.631998+0000","last_became_active":"2024-09-10T08:37:42.845884+0000","last_became_peered":"2024-09-10T08:37:42.845884+0000","last_unstale":"2024-09-10T08:38:43.631998+0000","last_undegraded":"2024-09-10T08:38:43.631998+0000","last_fullsized":"2024-09-10T08:38:43.631998+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:21.905133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593781+0000","last_change":"2024-09-10T08:36:56.144187+0000","last_active":"2024-09-10T08:39:11.593781+0000","last_peered":"2024-09-10T08:39:11.593781+0000","last_clean":"2024-09-10T08:39:11.593781+0000","last_became_active":"2024-09-10T08:36:56.143193+0000","last_became_peered":"2024-09-10T08:36:56.143193+0000","last_unstale":"2024-09-10T08:39:11.593781+0000","last_undegraded":"2024-09-10T08:39:11.593781+0000","last_fullsized":"2024-09-10T08:39:11.593781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:14:26.981884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.639660+0000","last_change":"2024-09-10T08:38:43.639286+0000","last_active":"2024-09-10T08:38:43.639660+0000","last_peered":"2024-09-10T08:38:43.639660+0000","last_clean":"2024-09-10T08:38:43.639660+0000","last_became_active":"2024-09-10T08:38:43.633552+0000","last_became_peered":"2024-09-10T08:38:43.633552+0000","last_unstale":"2024-09-10T08:38:43.639660+0000","last_undegraded":"2024-09-10T08:38:43.639660+0000","last_fullsized":"2024-09-10T08:38:43.639660+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:23.985190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630209+0000","last_change":"2024-09-10T08:37:05.242095+0000","last_active":"2024-09-10T08:38:43.630209+0000","last_peered":"2024-09-10T08:38:43.630209+0000","last_clean":"2024-09-10T08:38:43.630209+0000","last_became_active":"2024-09-10T08:37:05.241645+0000","last_became_peered":"2024-09-10T08:37:05.241645+0000","last_unstale":"2024-09-10T08:38:43.630209+0000","last_undegraded":"2024-09-10T08:38:43.630209+0000","last_fullsized":"2024-09-10T08:38:43.630209+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:30:49.637861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588835+0000","last_change":"2024-09-10T08:36:56.149396+0000","last_active":"2024-09-10T08:39:11.588835+0000","last_peered":"2024-09-10T08:39:11.588835+0000","last_clean":"2024-09-10T08:39:11.588835+0000","last_became_active":"2024-09-10T08:36:56.149079+0000","last_became_peered":"2024-09-10T08:36:56.149079+0000","last_unstale":"2024-09-10T08:39:11.588835+0000","last_undegraded":"2024-09-10T08:39:11.588835+0000","last_fullsized":"2024-09-10T08:39:11.588835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:44.665448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655467+0000","last_change":"2024-09-10T08:36:53.717950+0000","last_active":"2024-09-10T08:38:43.655467+0000","last_peered":"2024-09-10T08:38:43.655467+0000","last_clean":"2024-09-10T08:38:43.655467+0000","last_became_active":"2024-09-10T08:36:53.717668+0000","last_became_peered":"2024-09-10T08:36:53.717668+0000","last_unstale":"2024-09-10T08:38:43.655467+0000","last_undegraded":"2024-09-10T08:38:43.655467+0000","last_fullsized":"2024-09-10T08:38:43.655467+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:58.692631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633140+0000","last_change":"2024-09-10T08:37:05.260178+0000","last_active":"2024-09-10T08:38:43.633140+0000","last_peered":"2024-09-10T08:38:43.633140+0000","last_clean":"2024-09-10T08:38:43.633140+0000","last_became_active":"2024-09-10T08:37:05.259753+0000","last_became_peered":"2024-09-10T08:37:05.259753+0000","last_unstale":"2024-09-10T08:38:43.633140+0000","last_undegraded":"2024-09-10T08:38:43.633140+0000","last_fullsized":"2024-09-10T08:38:43.633140+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:19.364782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588360+0000","last_change":"2024-09-10T08:36:56.147104+0000","last_active":"2024-09-10T08:39:11.588360+0000","last_peered":"2024-09-10T08:39:11.588360+0000","last_clean":"2024-09-10T08:39:11.588360+0000","last_became_active":"2024-09-10T08:36:56.146778+0000","last_became_peered":"2024-09-10T08:36:56.146778+0000","last_unstale":"2024-09-10T08:39:11.588360+0000","last_undegraded":"2024-09-10T08:39:11.588360+0000","last_fullsized":"2024-09-10T08:39:11.588360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:56.092157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628399+0000","last_change":"2024-09-10T08:36:53.719145+0000","last_active":"2024-09-10T08:38:43.628399+0000","last_peered":"2024-09-10T08:38:43.628399+0000","last_clean":"2024-09-10T08:38:43.628399+0000","last_became_active":"2024-09-10T08:36:53.718771+0000","last_became_peered":"2024-09-10T08:36:53.718771+0000","last_unstale":"2024-09-10T08:38:43.628399+0000","last_undegraded":"2024-09-10T08:38:43.628399+0000","last_fullsized":"2024-09-10T08:38:43.628399+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:21.109940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654733+0000","last_change":"2024-09-10T08:37:05.242854+0000","last_active":"2024-09-10T08:38:43.654733+0000","last_peered":"2024-09-10T08:38:43.654733+0000","last_clean":"2024-09-10T08:38:43.654733+0000","last_became_active":"2024-09-10T08:37:05.242442+0000","last_became_peered":"2024-09-10T08:37:05.242442+0000","last_unstale":"2024-09-10T08:38:43.654733+0000","last_undegraded":"2024-09-10T08:38:43.654733+0000","last_fullsized":"2024-09-10T08:38:43.654733+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:36.072131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.570294+0000","last_change":"2024-09-10T08:37:42.845642+0000","last_active":"2024-09-10T08:39:11.570294+0000","last_peered":"2024-09-10T08:39:11.570294+0000","last_clean":"2024-09-10T08:39:11.570294+0000","last_became_active":"2024-09-10T08:37:42.845217+0000","last_became_peered":"2024-09-10T08:37:42.845217+0000","last_unstale":"2024-09-10T08:39:11.570294+0000","last_undegraded":"2024-09-10T08:39:11.570294+0000","last_fullsized":"2024-09-10T08:39:11.570294+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:28.020337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626734+0000","last_change":"2024-09-10T08:36:53.720463+0000","last_active":"2024-09-10T08:38:43.626734+0000","last_peered":"2024-09-10T08:38:43.626734+0000","last_clean":"2024-09-10T08:38:43.626734+0000","last_became_active":"2024-09-10T08:36:53.720080+0000","last_became_peered":"2024-09-10T08:36:53.720080+0000","last_unstale":"2024-09-10T08:38:43.626734+0000","last_undegraded":"2024-09-10T08:38:43.626734+0000","last_fullsized":"2024-09-10T08:38:43.626734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:51.450294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654955+0000","last_change":"2024-09-10T08:37:05.251261+0000","last_active":"2024-09-10T08:38:43.654955+0000","last_peered":"2024-09-10T08:38:43.654955+0000","last_clean":"2024-09-10T08:38:43.654955+0000","last_became_active":"2024-09-10T08:37:05.250938+0000","last_became_peered":"2024-09-10T08:37:05.250938+0000","last_unstale":"2024-09-10T08:38:43.654955+0000","last_undegraded":"2024-09-10T08:38:43.654955+0000","last_fullsized":"2024-09-10T08:38:43.654955+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:07:15.129899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.581822+0000","last_change":"2024-09-10T08:36:56.148191+0000","last_active":"2024-09-10T08:39:11.581822+0000","last_peered":"2024-09-10T08:39:11.581822+0000","last_clean":"2024-09-10T08:39:11.581822+0000","last_became_active":"2024-09-10T08:36:56.147362+0000","last_became_peered":"2024-09-10T08:36:56.147362+0000","last_unstale":"2024-09-10T08:39:11.581822+0000","last_undegraded":"2024-09-10T08:39:11.581822+0000","last_fullsized":"2024-09-10T08:39:11.581822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:25:30.051754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.626042+0000","last_change":"2024-09-10T08:36:53.734484+0000","last_active":"2024-09-10T08:38:43.626042+0000","last_peered":"2024-09-10T08:38:43.626042+0000","last_clean":"2024-09-10T08:38:43.626042+0000","last_became_active":"2024-09-10T08:36:53.734172+0000","last_became_peered":"2024-09-10T08:36:53.734172+0000","last_unstale":"2024-09-10T08:38:43.626042+0000","last_undegraded":"2024-09-10T08:38:43.626042+0000","last_fullsized":"2024-09-10T08:38:43.626042+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:10.269400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628299+0000","last_change":"2024-09-10T08:37:05.259635+0000","last_active":"2024-09-10T08:38:43.628299+0000","last_peered":"2024-09-10T08:38:43.628299+0000","last_clean":"2024-09-10T08:38:43.628299+0000","last_became_active":"2024-09-10T08:37:05.259107+0000","last_became_peered":"2024-09-10T08:37:05.259107+0000","last_unstale":"2024-09-10T08:38:43.628299+0000","last_undegraded":"2024-09-10T08:38:43.628299+0000","last_fullsized":"2024-09-10T08:38:43.628299+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:36.841138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588992+0000","last_change":"2024-09-10T08:36:56.141122+0000","last_active":"2024-09-10T08:39:11.588992+0000","last_peered":"2024-09-10T08:39:11.588992+0000","last_clean":"2024-09-10T08:39:11.588992+0000","last_became_active":"2024-09-10T08:36:56.140826+0000","last_became_peered":"2024-09-10T08:36:56.140826+0000","last_unstale":"2024-09-10T08:39:11.588992+0000","last_undegraded":"2024-09-10T08:39:11.588992+0000","last_fullsized":"2024-09-10T08:39:11.588992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:38.044090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"96'6","reported_seq":179,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:53.913332+0000","last_change":"2024-09-10T08:36:53.720623+0000","last_active":"2024-09-10T08:39:53.913332+0000","last_peered":"2024-09-10T08:39:53.913332+0000","last_clean":"2024-09-10T08:39:53.913332+0000","last_became_active":"2024-09-10T08:36:53.720270+0000","last_became_peered":"2024-09-10T08:36:53.720270+0000","last_unstale":"2024-09-10T08:39:53.913332+0000","last_undegraded":"2024-09-10T08:39:53.913332+0000","last_fullsized":"2024-09-10T08:39:53.913332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:19.514068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629937+0000","last_change":"2024-09-10T08:37:05.254324+0000","last_active":"2024-09-10T08:38:43.629937+0000","last_peered":"2024-09-10T08:38:43.629937+0000","last_clean":"2024-09-10T08:38:43.629937+0000","last_became_active":"2024-09-10T08:37:05.253611+0000","last_became_peered":"2024-09-10T08:37:05.253611+0000","last_unstale":"2024-09-10T08:38:43.629937+0000","last_undegraded":"2024-09-10T08:38:43.629937+0000","last_fullsized":"2024-09-10T08:38:43.629937+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:07.182649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.591257+0000","last_change":"2024-09-10T08:36:56.139102+0000","last_active":"2024-09-10T08:39:11.591257+0000","last_peered":"2024-09-10T08:39:11.591257+0000","last_clean":"2024-09-10T08:39:11.591257+0000","last_became_active":"2024-09-10T08:36:56.137986+0000","last_became_peered":"2024-09-10T08:36:56.137986+0000","last_unstale":"2024-09-10T08:39:11.591257+0000","last_undegraded":"2024-09-10T08:39:11.591257+0000","last_fullsized":"2024-09-10T08:39:11.591257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:05:03.005727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655021+0000","last_change":"2024-09-10T08:36:53.722961+0000","last_active":"2024-09-10T08:38:43.655021+0000","last_peered":"2024-09-10T08:38:43.655021+0000","last_clean":"2024-09-10T08:38:43.655021+0000","last_became_active":"2024-09-10T08:36:53.722628+0000","last_became_peered":"2024-09-10T08:36:53.722628+0000","last_unstale":"2024-09-10T08:38:43.655021+0000","last_undegraded":"2024-09-10T08:38:43.655021+0000","last_fullsized":"2024-09-10T08:38:43.655021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:59.048632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628104+0000","last_change":"2024-09-10T08:37:05.256546+0000","last_active":"2024-09-10T08:38:43.628104+0000","last_peered":"2024-09-10T08:38:43.628104+0000","last_clean":"2024-09-10T08:38:43.628104+0000","last_became_active":"2024-09-10T08:37:05.256239+0000","last_became_peered":"2024-09-10T08:37:05.256239+0000","last_unstale":"2024-09-10T08:38:43.628104+0000","last_undegraded":"2024-09-10T08:38:43.628104+0000","last_fullsized":"2024-09-10T08:38:43.628104+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:57.403934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.582848+0000","last_change":"2024-09-10T08:36:56.149285+0000","last_active":"2024-09-10T08:39:11.582848+0000","last_peered":"2024-09-10T08:39:11.582848+0000","last_clean":"2024-09-10T08:39:11.582848+0000","last_became_active":"2024-09-10T08:36:56.146873+0000","last_became_peered":"2024-09-10T08:36:56.146873+0000","last_unstale":"2024-09-10T08:39:11.582848+0000","last_undegraded":"2024-09-10T08:39:11.582848+0000","last_fullsized":"2024-09-10T08:39:11.582848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:28.976729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630368+0000","last_change":"2024-09-10T08:36:53.721174+0000","last_active":"2024-09-10T08:38:43.630368+0000","last_peered":"2024-09-10T08:38:43.630368+0000","last_clean":"2024-09-10T08:38:43.630368+0000","last_became_active":"2024-09-10T08:36:53.720849+0000","last_became_peered":"2024-09-10T08:36:53.720849+0000","last_unstale":"2024-09-10T08:38:43.630368+0000","last_undegraded":"2024-09-10T08:38:43.630368+0000","last_fullsized":"2024-09-10T08:38:43.630368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:39.786848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655286+0000","last_change":"2024-09-10T08:37:05.252256+0000","last_active":"2024-09-10T08:38:43.655286+0000","last_peered":"2024-09-10T08:38:43.655286+0000","last_clean":"2024-09-10T08:38:43.655286+0000","last_became_active":"2024-09-10T08:37:05.251865+0000","last_became_peered":"2024-09-10T08:37:05.251865+0000","last_unstale":"2024-09-10T08:38:43.655286+0000","last_undegraded":"2024-09-10T08:38:43.655286+0000","last_fullsized":"2024-09-10T08:38:43.655286+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:08.581880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567773+0000","last_change":"2024-09-10T08:36:56.142060+0000","last_active":"2024-09-10T08:39:11.567773+0000","last_peered":"2024-09-10T08:39:11.567773+0000","last_clean":"2024-09-10T08:39:11.567773+0000","last_became_active":"2024-09-10T08:36:56.141451+0000","last_became_peered":"2024-09-10T08:36:56.141451+0000","last_unstale":"2024-09-10T08:39:11.567773+0000","last_undegraded":"2024-09-10T08:39:11.567773+0000","last_fullsized":"2024-09-10T08:39:11.567773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:43.566588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'40","reported_seq":347,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:53.900348+0000","last_change":"2024-09-10T08:36:53.719464+0000","last_active":"2024-09-10T08:39:53.900348+0000","last_peered":"2024-09-10T08:39:53.900348+0000","last_clean":"2024-09-10T08:39:53.900348+0000","last_became_active":"2024-09-10T08:36:53.719129+0000","last_became_peered":"2024-09-10T08:36:53.719129+0000","last_unstale":"2024-09-10T08:39:53.900348+0000","last_undegraded":"2024-09-10T08:39:53.900348+0000","last_fullsized":"2024-09-10T08:39:53.900348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:05.652018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":72436,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":266,"num_read_kb":205,"num_write":41,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629405+0000","last_change":"2024-09-10T08:37:05.255576+0000","last_active":"2024-09-10T08:38:43.629405+0000","last_peered":"2024-09-10T08:38:43.629405+0000","last_clean":"2024-09-10T08:38:43.629405+0000","last_became_active":"2024-09-10T08:37:05.249842+0000","last_became_peered":"2024-09-10T08:37:05.249842+0000","last_unstale":"2024-09-10T08:38:43.629405+0000","last_undegraded":"2024-09-10T08:38:43.629405+0000","last_fullsized":"2024-09-10T08:38:43.629405+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:47:49.543481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593397+0000","last_change":"2024-09-10T08:36:56.151829+0000","last_active":"2024-09-10T08:39:11.593397+0000","last_peered":"2024-09-10T08:39:11.593397+0000","last_clean":"2024-09-10T08:39:11.593397+0000","last_became_active":"2024-09-10T08:36:56.151314+0000","last_became_peered":"2024-09-10T08:36:56.151314+0000","last_unstale":"2024-09-10T08:39:11.593397+0000","last_undegraded":"2024-09-10T08:39:11.593397+0000","last_fullsized":"2024-09-10T08:39:11.593397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:57:44.219671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655398+0000","last_change":"2024-09-10T08:36:53.738030+0000","last_active":"2024-09-10T08:38:43.655398+0000","last_peered":"2024-09-10T08:38:43.655398+0000","last_clean":"2024-09-10T08:38:43.655398+0000","last_became_active":"2024-09-10T08:36:53.737658+0000","last_became_peered":"2024-09-10T08:36:53.737658+0000","last_unstale":"2024-09-10T08:38:43.655398+0000","last_undegraded":"2024-09-10T08:38:43.655398+0000","last_fullsized":"2024-09-10T08:38:43.655398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:29.627554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.654996+0000","last_change":"2024-09-10T08:37:05.244473+0000","last_active":"2024-09-10T08:38:43.654996+0000","last_peered":"2024-09-10T08:38:43.654996+0000","last_clean":"2024-09-10T08:38:43.654996+0000","last_became_active":"2024-09-10T08:37:05.244035+0000","last_became_peered":"2024-09-10T08:37:05.244035+0000","last_unstale":"2024-09-10T08:38:43.654996+0000","last_undegraded":"2024-09-10T08:38:43.654996+0000","last_fullsized":"2024-09-10T08:38:43.654996+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:04.652660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584024+0000","last_change":"2024-09-10T08:36:56.155217+0000","last_active":"2024-09-10T08:39:11.584024+0000","last_peered":"2024-09-10T08:39:11.584024+0000","last_clean":"2024-09-10T08:39:11.584024+0000","last_became_active":"2024-09-10T08:36:56.154903+0000","last_became_peered":"2024-09-10T08:36:56.154903+0000","last_unstale":"2024-09-10T08:39:11.584024+0000","last_undegraded":"2024-09-10T08:39:11.584024+0000","last_fullsized":"2024-09-10T08:39:11.584024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:47.482974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.632239+0000","last_change":"2024-09-10T08:36:53.721087+0000","last_active":"2024-09-10T08:38:43.632239+0000","last_peered":"2024-09-10T08:38:43.632239+0000","last_clean":"2024-09-10T08:38:43.632239+0000","last_became_active":"2024-09-10T08:36:53.720772+0000","last_became_peered":"2024-09-10T08:36:53.720772+0000","last_unstale":"2024-09-10T08:38:43.632239+0000","last_undegraded":"2024-09-10T08:38:43.632239+0000","last_fullsized":"2024-09-10T08:38:43.632239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:39.560848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.629150+0000","last_change":"2024-09-10T08:37:05.254641+0000","last_active":"2024-09-10T08:38:43.629150+0000","last_peered":"2024-09-10T08:38:43.629150+0000","last_clean":"2024-09-10T08:38:43.629150+0000","last_became_active":"2024-09-10T08:37:05.249982+0000","last_became_peered":"2024-09-10T08:37:05.249982+0000","last_unstale":"2024-09-10T08:38:43.629150+0000","last_undegraded":"2024-09-10T08:38:43.629150+0000","last_fullsized":"2024-09-10T08:38:43.629150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:21:13.851061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592510+0000","last_change":"2024-09-10T08:36:56.142657+0000","last_active":"2024-09-10T08:39:11.592510+0000","last_peered":"2024-09-10T08:39:11.592510+0000","last_clean":"2024-09-10T08:39:11.592510+0000","last_became_active":"2024-09-10T08:36:56.142294+0000","last_became_peered":"2024-09-10T08:36:56.142294+0000","last_unstale":"2024-09-10T08:39:11.592510+0000","last_undegraded":"2024-09-10T08:39:11.592510+0000","last_fullsized":"2024-09-10T08:39:11.592510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:52.861389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'4","reported_seq":171,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:52.898938+0000","last_change":"2024-09-10T08:36:53.722466+0000","last_active":"2024-09-10T08:39:52.898938+0000","last_peered":"2024-09-10T08:39:52.898938+0000","last_clean":"2024-09-10T08:39:52.898938+0000","last_became_active":"2024-09-10T08:36:53.722145+0000","last_became_peered":"2024-09-10T08:36:53.722145+0000","last_unstale":"2024-09-10T08:39:52.898938+0000","last_undegraded":"2024-09-10T08:39:52.898938+0000","last_fullsized":"2024-09-10T08:39:52.898938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:27.041765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.631352+0000","last_change":"2024-09-10T08:37:05.251056+0000","last_active":"2024-09-10T08:38:43.631352+0000","last_peered":"2024-09-10T08:38:43.631352+0000","last_clean":"2024-09-10T08:38:43.631352+0000","last_became_active":"2024-09-10T08:37:05.250653+0000","last_became_peered":"2024-09-10T08:37:05.250653+0000","last_unstale":"2024-09-10T08:38:43.631352+0000","last_undegraded":"2024-09-10T08:38:43.631352+0000","last_fullsized":"2024-09-10T08:38:43.631352+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:21.174747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.633128+0000","last_change":"2024-09-10T08:37:05.260436+0000","last_active":"2024-09-10T08:38:43.633128+0000","last_peered":"2024-09-10T08:38:43.633128+0000","last_clean":"2024-09-10T08:38:43.633128+0000","last_became_active":"2024-09-10T08:37:05.260049+0000","last_became_peered":"2024-09-10T08:37:05.260049+0000","last_unstale":"2024-09-10T08:38:43.633128+0000","last_undegraded":"2024-09-10T08:38:43.633128+0000","last_fullsized":"2024-09-10T08:38:43.633128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:43.394028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585930+0000","last_change":"2024-09-10T08:36:56.146732+0000","last_active":"2024-09-10T08:39:11.585930+0000","last_peered":"2024-09-10T08:39:11.585930+0000","last_clean":"2024-09-10T08:39:11.585930+0000","last_became_active":"2024-09-10T08:36:56.145599+0000","last_became_peered":"2024-09-10T08:36:56.145599+0000","last_unstale":"2024-09-10T08:39:11.585930+0000","last_undegraded":"2024-09-10T08:39:11.585930+0000","last_fullsized":"2024-09-10T08:39:11.585930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:50.620956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.655135+0000","last_change":"2024-09-10T08:36:53.721683+0000","last_active":"2024-09-10T08:38:43.655135+0000","last_peered":"2024-09-10T08:38:43.655135+0000","last_clean":"2024-09-10T08:38:43.655135+0000","last_became_active":"2024-09-10T08:36:53.721241+0000","last_became_peered":"2024-09-10T08:36:53.721241+0000","last_unstale":"2024-09-10T08:38:43.655135+0000","last_undegraded":"2024-09-10T08:38:43.655135+0000","last_fullsized":"2024-09-10T08:38:43.655135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:03.669337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":33,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.628275+0000","last_change":"2024-09-10T08:37:42.669784+0000","last_active":"2024-09-10T08:38:43.628275+0000","last_peered":"2024-09-10T08:38:43.628275+0000","last_clean":"2024-09-10T08:38:43.628275+0000","last_became_active":"2024-09-10T08:37:42.669322+0000","last_became_peered":"2024-09-10T08:37:42.669322+0000","last_unstale":"2024-09-10T08:38:43.628275+0000","last_undegraded":"2024-09-10T08:38:43.628275+0000","last_fullsized":"2024-09-10T08:38:43.628275+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:06:57.027928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.574617+0000","last_change":"2024-09-10T08:36:56.143824+0000","last_active":"2024-09-10T08:39:11.574617+0000","last_peered":"2024-09-10T08:39:11.574617+0000","last_clean":"2024-09-10T08:39:11.574617+0000","last_became_active":"2024-09-10T08:36:56.143486+0000","last_became_peered":"2024-09-10T08:36:56.143486+0000","last_unstale":"2024-09-10T08:39:11.574617+0000","last_undegraded":"2024-09-10T08:39:11.574617+0000","last_fullsized":"2024-09-10T08:39:11.574617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:42.384863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627976+0000","last_change":"2024-09-10T08:36:53.717680+0000","last_active":"2024-09-10T08:38:43.627976+0000","last_peered":"2024-09-10T08:38:43.627976+0000","last_clean":"2024-09-10T08:38:43.627976+0000","last_became_active":"2024-09-10T08:36:53.717249+0000","last_became_peered":"2024-09-10T08:36:53.717249+0000","last_unstale":"2024-09-10T08:38:43.627976+0000","last_undegraded":"2024-09-10T08:38:43.627976+0000","last_fullsized":"2024-09-10T08:38:43.627976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:12.191541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.627200+0000","last_change":"2024-09-10T08:36:53.723947+0000","last_active":"2024-09-10T08:38:43.627200+0000","last_peered":"2024-09-10T08:38:43.627200+0000","last_clean":"2024-09-10T08:38:43.627200+0000","last_became_active":"2024-09-10T08:36:53.723620+0000","last_became_peered":"2024-09-10T08:36:53.723620+0000","last_unstale":"2024-09-10T08:38:43.627200+0000","last_undegraded":"2024-09-10T08:38:43.627200+0000","last_fullsized":"2024-09-10T08:38:43.627200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:00.135349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583145+0000","last_change":"2024-09-10T08:36:56.150448+0000","last_active":"2024-09-10T08:39:11.583145+0000","last_peered":"2024-09-10T08:39:11.583145+0000","last_clean":"2024-09-10T08:39:11.583145+0000","last_became_active":"2024-09-10T08:36:56.150148+0000","last_became_peered":"2024-09-10T08:36:56.150148+0000","last_unstale":"2024-09-10T08:39:11.583145+0000","last_undegraded":"2024-09-10T08:39:11.583145+0000","last_fullsized":"2024-09-10T08:39:11.583145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:02.927753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630191+0000","last_change":"2024-09-10T08:37:05.242878+0000","last_active":"2024-09-10T08:38:43.630191+0000","last_peered":"2024-09-10T08:38:43.630191+0000","last_clean":"2024-09-10T08:38:43.630191+0000","last_became_active":"2024-09-10T08:37:05.242440+0000","last_became_peered":"2024-09-10T08:37:05.242440+0000","last_unstale":"2024-09-10T08:38:43.630191+0000","last_undegraded":"2024-09-10T08:38:43.630191+0000","last_fullsized":"2024-09-10T08:38:43.630191+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:30.957433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'7","reported_seq":174,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:48.893331+0000","last_change":"2024-09-10T08:36:53.719788+0000","last_active":"2024-09-10T08:39:48.893331+0000","last_peered":"2024-09-10T08:39:48.893331+0000","last_clean":"2024-09-10T08:39:48.893331+0000","last_became_active":"2024-09-10T08:36:53.719462+0000","last_became_peered":"2024-09-10T08:36:53.719462+0000","last_unstale":"2024-09-10T08:39:48.893331+0000","last_undegraded":"2024-09-10T08:39:48.893331+0000","last_fullsized":"2024-09-10T08:39:48.893331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:52.699703+0000","last_clean_scrub_stamp":"2024-09-10T08:36:52.699703+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:12.250447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1326,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583170+0000","last_change":"2024-09-10T08:37:42.673008+0000","last_active":"2024-09-10T08:39:11.583170+0000","last_peered":"2024-09-10T08:39:11.583170+0000","last_clean":"2024-09-10T08:39:11.583170+0000","last_became_active":"2024-09-10T08:37:42.672618+0000","last_became_peered":"2024-09-10T08:37:42.672618+0000","last_unstale":"2024-09-10T08:39:11.583170+0000","last_undegraded":"2024-09-10T08:39:11.583170+0000","last_fullsized":"2024-09-10T08:39:11.583170+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:00.156565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:38:43.630783+0000","last_change":"2024-09-10T08:37:05.244277+0000","last_active":"2024-09-10T08:38:43.630783+0000","last_peered":"2024-09-10T08:38:43.630783+0000","last_clean":"2024-09-10T08:38:43.630783+0000","last_became_active":"2024-09-10T08:37:05.243880+0000","last_became_peered":"2024-09-10T08:37:05.243880+0000","last_unstale":"2024-09-10T08:38:43.630783+0000","last_undegraded":"2024-09-10T08:38:43.630783+0000","last_fullsized":"2024-09-10T08:38:43.630783+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:37:04.218943+0000","last_clean_scrub_stamp":"2024-09-10T08:37:04.218943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:55.275857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.573131+0000","last_change":"2024-09-10T08:36:56.141404+0000","last_active":"2024-09-10T08:39:11.573131+0000","last_peered":"2024-09-10T08:39:11.573131+0000","last_clean":"2024-09-10T08:39:11.573131+0000","last_became_active":"2024-09-10T08:36:56.141076+0000","last_became_peered":"2024-09-10T08:36:56.141076+0000","last_unstale":"2024-09-10T08:39:11.573131+0000","last_undegraded":"2024-09-10T08:39:11.573131+0000","last_fullsized":"2024-09-10T08:39:11.573131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:03:52.915654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.569383+0000","last_change":"2024-09-10T08:36:56.148285+0000","last_active":"2024-09-10T08:39:11.569383+0000","last_peered":"2024-09-10T08:39:11.569383+0000","last_clean":"2024-09-10T08:39:11.569383+0000","last_became_active":"2024-09-10T08:36:56.147970+0000","last_became_peered":"2024-09-10T08:36:56.147970+0000","last_unstale":"2024-09-10T08:39:11.569383+0000","last_undegraded":"2024-09-10T08:39:11.569383+0000","last_fullsized":"2024-09-10T08:39:11.569383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:09.506718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.571047+0000","last_change":"2024-09-10T08:36:56.157977+0000","last_active":"2024-09-10T08:39:11.571047+0000","last_peered":"2024-09-10T08:39:11.571047+0000","last_clean":"2024-09-10T08:39:11.571047+0000","last_became_active":"2024-09-10T08:36:56.157623+0000","last_became_peered":"2024-09-10T08:36:56.157623+0000","last_unstale":"2024-09-10T08:39:11.571047+0000","last_undegraded":"2024-09-10T08:39:11.571047+0000","last_fullsized":"2024-09-10T08:39:11.571047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:44.617979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584899+0000","last_change":"2024-09-10T08:37:42.670459+0000","last_active":"2024-09-10T08:39:11.584899+0000","last_peered":"2024-09-10T08:39:11.584899+0000","last_clean":"2024-09-10T08:39:11.584899+0000","last_became_active":"2024-09-10T08:37:42.670097+0000","last_became_peered":"2024-09-10T08:37:42.670097+0000","last_unstale":"2024-09-10T08:39:11.584899+0000","last_undegraded":"2024-09-10T08:39:11.584899+0000","last_fullsized":"2024-09-10T08:39:11.584899+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:14.956492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584527+0000","last_change":"2024-09-10T08:36:56.154850+0000","last_active":"2024-09-10T08:39:11.584527+0000","last_peered":"2024-09-10T08:39:11.584527+0000","last_clean":"2024-09-10T08:39:11.584527+0000","last_became_active":"2024-09-10T08:36:56.154545+0000","last_became_peered":"2024-09-10T08:36:56.154545+0000","last_unstale":"2024-09-10T08:39:11.584527+0000","last_undegraded":"2024-09-10T08:39:11.584527+0000","last_fullsized":"2024-09-10T08:39:11.584527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:24.639667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.569973+0000","last_change":"2024-09-10T08:37:42.669962+0000","last_active":"2024-09-10T08:39:11.569973+0000","last_peered":"2024-09-10T08:39:11.569973+0000","last_clean":"2024-09-10T08:39:11.569973+0000","last_became_active":"2024-09-10T08:37:42.669556+0000","last_became_peered":"2024-09-10T08:37:42.669556+0000","last_unstale":"2024-09-10T08:39:11.569973+0000","last_undegraded":"2024-09-10T08:39:11.569973+0000","last_fullsized":"2024-09-10T08:39:11.569973+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:23.333946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.578061+0000","last_change":"2024-09-10T08:36:56.145910+0000","last_active":"2024-09-10T08:39:11.578061+0000","last_peered":"2024-09-10T08:39:11.578061+0000","last_clean":"2024-09-10T08:39:11.578061+0000","last_became_active":"2024-09-10T08:36:56.145537+0000","last_became_peered":"2024-09-10T08:36:56.145537+0000","last_unstale":"2024-09-10T08:39:11.578061+0000","last_undegraded":"2024-09-10T08:39:11.578061+0000","last_fullsized":"2024-09-10T08:39:11.578061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:46.788241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":39,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567880+0000","last_change":"2024-09-10T08:36:56.147340+0000","last_active":"2024-09-10T08:39:11.567880+0000","last_peered":"2024-09-10T08:39:11.567880+0000","last_clean":"2024-09-10T08:39:11.567880+0000","last_became_active":"2024-09-10T08:36:56.145762+0000","last_became_peered":"2024-09-10T08:36:56.145762+0000","last_unstale":"2024-09-10T08:39:11.567880+0000","last_undegraded":"2024-09-10T08:39:11.567880+0000","last_fullsized":"2024-09-10T08:39:11.567880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:12.339285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.581431+0000","last_change":"2024-09-10T08:36:56.145213+0000","last_active":"2024-09-10T08:39:11.581431+0000","last_peered":"2024-09-10T08:39:11.581431+0000","last_clean":"2024-09-10T08:39:11.581431+0000","last_became_active":"2024-09-10T08:36:56.144915+0000","last_became_peered":"2024-09-10T08:36:56.144915+0000","last_unstale":"2024-09-10T08:39:11.581431+0000","last_undegraded":"2024-09-10T08:39:11.581431+0000","last_fullsized":"2024-09-10T08:39:11.581431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:06.180321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.574743+0000","last_change":"2024-09-10T08:36:56.149738+0000","last_active":"2024-09-10T08:39:11.574743+0000","last_peered":"2024-09-10T08:39:11.574743+0000","last_clean":"2024-09-10T08:39:11.574743+0000","last_became_active":"2024-09-10T08:36:56.147750+0000","last_became_peered":"2024-09-10T08:36:56.147750+0000","last_unstale":"2024-09-10T08:39:11.574743+0000","last_undegraded":"2024-09-10T08:39:11.574743+0000","last_fullsized":"2024-09-10T08:39:11.574743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:54:47.950054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.591937+0000","last_change":"2024-09-10T08:36:56.147682+0000","last_active":"2024-09-10T08:39:11.591937+0000","last_peered":"2024-09-10T08:39:11.591937+0000","last_clean":"2024-09-10T08:39:11.591937+0000","last_became_active":"2024-09-10T08:36:56.147345+0000","last_became_peered":"2024-09-10T08:36:56.147345+0000","last_unstale":"2024-09-10T08:39:11.591937+0000","last_undegraded":"2024-09-10T08:39:11.591937+0000","last_fullsized":"2024-09-10T08:39:11.591937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:54.939482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583378+0000","last_change":"2024-09-10T08:36:56.158292+0000","last_active":"2024-09-10T08:39:11.583378+0000","last_peered":"2024-09-10T08:39:11.583378+0000","last_clean":"2024-09-10T08:39:11.583378+0000","last_became_active":"2024-09-10T08:36:56.157970+0000","last_became_peered":"2024-09-10T08:36:56.157970+0000","last_unstale":"2024-09-10T08:39:11.583378+0000","last_undegraded":"2024-09-10T08:39:11.583378+0000","last_fullsized":"2024-09-10T08:39:11.583378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:33.582126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567916+0000","last_change":"2024-09-10T08:36:56.142520+0000","last_active":"2024-09-10T08:39:11.567916+0000","last_peered":"2024-09-10T08:39:11.567916+0000","last_clean":"2024-09-10T08:39:11.567916+0000","last_became_active":"2024-09-10T08:36:56.142214+0000","last_became_peered":"2024-09-10T08:36:56.142214+0000","last_unstale":"2024-09-10T08:39:11.567916+0000","last_undegraded":"2024-09-10T08:39:11.567916+0000","last_fullsized":"2024-09-10T08:39:11.567916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:45:29.201839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584167+0000","last_change":"2024-09-10T08:36:56.149547+0000","last_active":"2024-09-10T08:39:11.584167+0000","last_peered":"2024-09-10T08:39:11.584167+0000","last_clean":"2024-09-10T08:39:11.584167+0000","last_became_active":"2024-09-10T08:36:56.149141+0000","last_became_peered":"2024-09-10T08:36:56.149141+0000","last_unstale":"2024-09-10T08:39:11.584167+0000","last_undegraded":"2024-09-10T08:39:11.584167+0000","last_fullsized":"2024-09-10T08:39:11.584167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:49.368677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585864+0000","last_change":"2024-09-10T08:36:56.143087+0000","last_active":"2024-09-10T08:39:11.585864+0000","last_peered":"2024-09-10T08:39:11.585864+0000","last_clean":"2024-09-10T08:39:11.585864+0000","last_became_active":"2024-09-10T08:36:56.142782+0000","last_became_peered":"2024-09-10T08:36:56.142782+0000","last_unstale":"2024-09-10T08:39:11.585864+0000","last_undegraded":"2024-09-10T08:39:11.585864+0000","last_fullsized":"2024-09-10T08:39:11.585864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:17.513103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592876+0000","last_change":"2024-09-10T08:36:56.157549+0000","last_active":"2024-09-10T08:39:11.592876+0000","last_peered":"2024-09-10T08:39:11.592876+0000","last_clean":"2024-09-10T08:39:11.592876+0000","last_became_active":"2024-09-10T08:36:56.157221+0000","last_became_peered":"2024-09-10T08:36:56.157221+0000","last_unstale":"2024-09-10T08:39:11.592876+0000","last_undegraded":"2024-09-10T08:39:11.592876+0000","last_fullsized":"2024-09-10T08:39:11.592876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:11.372085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593199+0000","last_change":"2024-09-10T08:36:56.145654+0000","last_active":"2024-09-10T08:39:11.593199+0000","last_peered":"2024-09-10T08:39:11.593199+0000","last_clean":"2024-09-10T08:39:11.593199+0000","last_became_active":"2024-09-10T08:36:56.144126+0000","last_became_peered":"2024-09-10T08:36:56.144126+0000","last_unstale":"2024-09-10T08:39:11.593199+0000","last_undegraded":"2024-09-10T08:39:11.593199+0000","last_fullsized":"2024-09-10T08:39:11.593199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:55.498873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.582479+0000","last_change":"2024-09-10T08:36:56.146423+0000","last_active":"2024-09-10T08:39:11.582479+0000","last_peered":"2024-09-10T08:39:11.582479+0000","last_clean":"2024-09-10T08:39:11.582479+0000","last_became_active":"2024-09-10T08:36:56.146126+0000","last_became_peered":"2024-09-10T08:36:56.146126+0000","last_unstale":"2024-09-10T08:39:11.582479+0000","last_undegraded":"2024-09-10T08:39:11.582479+0000","last_fullsized":"2024-09-10T08:39:11.582479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:44.665196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.583332+0000","last_change":"2024-09-10T08:36:56.151973+0000","last_active":"2024-09-10T08:39:11.583332+0000","last_peered":"2024-09-10T08:39:11.583332+0000","last_clean":"2024-09-10T08:39:11.583332+0000","last_became_active":"2024-09-10T08:36:56.151609+0000","last_became_peered":"2024-09-10T08:36:56.151609+0000","last_unstale":"2024-09-10T08:39:11.583332+0000","last_undegraded":"2024-09-10T08:39:11.583332+0000","last_fullsized":"2024-09-10T08:39:11.583332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:02.584047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.584509+0000","last_change":"2024-09-10T08:36:56.141780+0000","last_active":"2024-09-10T08:39:11.584509+0000","last_peered":"2024-09-10T08:39:11.584509+0000","last_clean":"2024-09-10T08:39:11.584509+0000","last_became_active":"2024-09-10T08:36:56.141228+0000","last_became_peered":"2024-09-10T08:36:56.141228+0000","last_unstale":"2024-09-10T08:39:11.584509+0000","last_undegraded":"2024-09-10T08:39:11.584509+0000","last_fullsized":"2024-09-10T08:39:11.584509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:52:04.374228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585477+0000","last_change":"2024-09-10T08:37:42.671346+0000","last_active":"2024-09-10T08:39:11.585477+0000","last_peered":"2024-09-10T08:39:11.585477+0000","last_clean":"2024-09-10T08:39:11.585477+0000","last_became_active":"2024-09-10T08:37:42.670895+0000","last_became_peered":"2024-09-10T08:37:42.670895+0000","last_unstale":"2024-09-10T08:39:11.585477+0000","last_undegraded":"2024-09-10T08:39:11.585477+0000","last_fullsized":"2024-09-10T08:39:11.585477+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:16:31.930628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.593796+0000","last_change":"2024-09-10T08:36:56.149014+0000","last_active":"2024-09-10T08:39:11.593796+0000","last_peered":"2024-09-10T08:39:11.593796+0000","last_clean":"2024-09-10T08:39:11.593796+0000","last_became_active":"2024-09-10T08:36:56.148551+0000","last_became_peered":"2024-09-10T08:36:56.148551+0000","last_unstale":"2024-09-10T08:39:11.593796+0000","last_undegraded":"2024-09-10T08:39:11.593796+0000","last_fullsized":"2024-09-10T08:39:11.593796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:35:52.250943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.586709+0000","last_change":"2024-09-10T08:36:56.141247+0000","last_active":"2024-09-10T08:39:11.586709+0000","last_peered":"2024-09-10T08:39:11.586709+0000","last_clean":"2024-09-10T08:39:11.586709+0000","last_became_active":"2024-09-10T08:36:56.140956+0000","last_became_peered":"2024-09-10T08:36:56.140956+0000","last_unstale":"2024-09-10T08:39:11.586709+0000","last_undegraded":"2024-09-10T08:39:11.586709+0000","last_fullsized":"2024-09-10T08:39:11.586709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:45.008424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592783+0000","last_change":"2024-09-10T08:36:56.149116+0000","last_active":"2024-09-10T08:39:11.592783+0000","last_peered":"2024-09-10T08:39:11.592783+0000","last_clean":"2024-09-10T08:39:11.592783+0000","last_became_active":"2024-09-10T08:36:56.148783+0000","last_became_peered":"2024-09-10T08:36:56.148783+0000","last_unstale":"2024-09-10T08:39:11.592783+0000","last_undegraded":"2024-09-10T08:39:11.592783+0000","last_fullsized":"2024-09-10T08:39:11.592783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:37.119214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.585754+0000","last_change":"2024-09-10T08:37:42.846322+0000","last_active":"2024-09-10T08:39:11.585754+0000","last_peered":"2024-09-10T08:39:11.585754+0000","last_clean":"2024-09-10T08:39:11.585754+0000","last_became_active":"2024-09-10T08:37:42.846022+0000","last_became_peered":"2024-09-10T08:37:42.846022+0000","last_unstale":"2024-09-10T08:39:11.585754+0000","last_undegraded":"2024-09-10T08:39:11.585754+0000","last_fullsized":"2024-09-10T08:39:11.585754+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:38.142362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.588751+0000","last_change":"2024-09-10T08:36:56.148537+0000","last_active":"2024-09-10T08:39:11.588751+0000","last_peered":"2024-09-10T08:39:11.588751+0000","last_clean":"2024-09-10T08:39:11.588751+0000","last_became_active":"2024-09-10T08:36:56.148185+0000","last_became_peered":"2024-09-10T08:36:56.148185+0000","last_unstale":"2024-09-10T08:39:11.588751+0000","last_undegraded":"2024-09-10T08:39:11.588751+0000","last_fullsized":"2024-09-10T08:39:11.588751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:00.426513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.586233+0000","last_change":"2024-09-10T08:36:56.138513+0000","last_active":"2024-09-10T08:39:11.586233+0000","last_peered":"2024-09-10T08:39:11.586233+0000","last_clean":"2024-09-10T08:39:11.586233+0000","last_became_active":"2024-09-10T08:36:56.138207+0000","last_became_peered":"2024-09-10T08:36:56.138207+0000","last_unstale":"2024-09-10T08:39:11.586233+0000","last_undegraded":"2024-09-10T08:39:11.586233+0000","last_fullsized":"2024-09-10T08:39:11.586233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:34.581430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.582686+0000","last_change":"2024-09-10T08:36:56.146612+0000","last_active":"2024-09-10T08:39:11.582686+0000","last_peered":"2024-09-10T08:39:11.582686+0000","last_clean":"2024-09-10T08:39:11.582686+0000","last_became_active":"2024-09-10T08:36:56.146251+0000","last_became_peered":"2024-09-10T08:36:56.146251+0000","last_unstale":"2024-09-10T08:39:11.582686+0000","last_undegraded":"2024-09-10T08:39:11.582686+0000","last_fullsized":"2024-09-10T08:39:11.582686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:47.336389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.567939+0000","last_change":"2024-09-10T08:36:56.148260+0000","last_active":"2024-09-10T08:39:11.567939+0000","last_peered":"2024-09-10T08:39:11.567939+0000","last_clean":"2024-09-10T08:39:11.567939+0000","last_became_active":"2024-09-10T08:36:56.147949+0000","last_became_peered":"2024-09-10T08:36:56.147949+0000","last_unstale":"2024-09-10T08:39:11.567939+0000","last_undegraded":"2024-09-10T08:39:11.567939+0000","last_fullsized":"2024-09-10T08:39:11.567939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:52:19.762297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592986+0000","last_change":"2024-09-10T08:36:56.156045+0000","last_active":"2024-09-10T08:39:11.592986+0000","last_peered":"2024-09-10T08:39:11.592986+0000","last_clean":"2024-09-10T08:39:11.592986+0000","last_became_active":"2024-09-10T08:36:56.154727+0000","last_became_peered":"2024-09-10T08:36:56.154727+0000","last_unstale":"2024-09-10T08:39:11.592986+0000","last_undegraded":"2024-09-10T08:39:11.592986+0000","last_fullsized":"2024-09-10T08:39:11.592986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:35:39.992683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.576067+0000","last_change":"2024-09-10T08:36:56.154394+0000","last_active":"2024-09-10T08:39:11.576067+0000","last_peered":"2024-09-10T08:39:11.576067+0000","last_clean":"2024-09-10T08:39:11.576067+0000","last_became_active":"2024-09-10T08:36:56.153855+0000","last_became_peered":"2024-09-10T08:36:56.153855+0000","last_unstale":"2024-09-10T08:39:11.576067+0000","last_undegraded":"2024-09-10T08:39:11.576067+0000","last_fullsized":"2024-09-10T08:39:11.576067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:14.987147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T08:39:11.592634+0000","last_change":"2024-09-10T08:36:56.157517+0000","last_active":"2024-09-10T08:39:11.592634+0000","last_peered":"2024-09-10T08:39:11.592634+0000","last_clean":"2024-09-10T08:39:11.592634+0000","last_became_active":"2024-09-10T08:36:56.157133+0000","last_became_peered":"2024-09-10T08:36:56.157133+0000","last_unstale":"2024-09-10T08:39:11.592634+0000","last_undegraded":"2024-09-10T08:39:11.592634+0000","last_fullsized":"2024-09-10T08:39:11.592634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:36:55.119827+0000","last_clean_scrub_stamp":"2024-09-10T08:36:55.119827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:17:29.573719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77677,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":786,"num_read_kb":476,"num_write":189,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":233031,"data_compressed":5667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282455,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416434,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":547326,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678071,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28212,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716940,"statfs":{"total":95995035648,"available":95966146560,"internally_reserved":0,"allocated":1216512,"data_stored":1216149,"data_compressed":67339,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841596,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":543466,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103234,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":546900,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332167,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":1144631,"data_compressed":65450,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495692,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":542738,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757330,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":1139292,"data_compressed":65450,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920855,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":548897,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149788,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":542932,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":615418,"data_compressed":31413,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712864,"statfs":{"total":95995035648,"available":95961972736,"internally_reserved":0,"allocated":1130496,"data_stored":615666,"data_compressed":31413,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1077248,"data_stored":544126,"data_compressed":29524,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34275,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":73018,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":73908,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":74941,"data_compressed":1889,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:39:57.824 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:39:58.190 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:58.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.765+0000 7fe38dc00640 1 -- 172.21.15.89:0/345129193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3881049a0 msgr2=0x7fe3881089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:58.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.765+0000 7fe38dc00640 1 --2- 172.21.15.89:0/345129193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3881049a0 0x7fe3881089f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe374009900 tx=0x7fe37402a410 comp rx=0 tx=0).stop 2024-09-10T08:39:58.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.765+0000 7fe38dc00640 1 -- 172.21.15.89:0/345129193 shutdown_connections 2024-09-10T08:39:58.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.765+0000 7fe38dc00640 1 --2- 172.21.15.89:0/345129193 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe388109120 0x7fe388110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:58.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.765+0000 7fe38dc00640 1 --2- 172.21.15.89:0/345129193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3881049a0 0x7fe3881089f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:58.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.765+0000 7fe38dc00640 1 --2- 172.21.15.89:0/345129193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe388103ff0 0x7fe3881043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:58.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.765+0000 7fe38dc00640 1 -- 172.21.15.89:0/345129193 >> 172.21.15.89:0/345129193 conn(0x7fe3880ffa90 msgr2=0x7fe388101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:58.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.765+0000 7fe38dc00640 1 -- 172.21.15.89:0/345129193 shutdown_connections 2024-09-10T08:39:58.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.765+0000 7fe38dc00640 1 -- 172.21.15.89:0/345129193 wait complete. 2024-09-10T08:39:58.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.766+0000 7fe38dc00640 1 Processor -- start 2024-09-10T08:39:58.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.766+0000 7fe38dc00640 1 -- start start 2024-09-10T08:39:58.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.766+0000 7fe38dc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe388103ff0 0x7fe38819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:58.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.766+0000 7fe38dc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3881049a0 0x7fe38819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:58.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.766+0000 7fe38dc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe388109120 0x7fe38819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:58.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.766+0000 7fe38dc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe388114190 con 0x7fe388109120 2024-09-10T08:39:58.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.766+0000 7fe38dc00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe388114010 con 0x7fe388103ff0 2024-09-10T08:39:58.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.766+0000 7fe38dc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe388114310 con 0x7fe3881049a0 2024-09-10T08:39:58.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.767+0000 7fe387400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe388109120 0x7fe38819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:58.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.767+0000 7fe387400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe388109120 0x7fe38819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49600/0 (socket says 172.21.15.89:49600) 2024-09-10T08:39:58.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.767+0000 7fe386a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe388103ff0 0x7fe38819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:58.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.767+0000 7fe387400640 1 -- 172.21.15.89:0/810093113 learned_addr learned my addr 172.21.15.89:0/810093113 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:39:58.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.767+0000 7fe386000640 1 --2- 172.21.15.89:0/810093113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3881049a0 0x7fe38819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:58.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.767+0000 7fe387400640 1 -- 172.21.15.89:0/810093113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3881049a0 msgr2=0x7fe38819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:58.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.767+0000 7fe387400640 1 --2- 172.21.15.89:0/810093113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3881049a0 0x7fe38819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:58.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.767+0000 7fe387400640 1 -- 172.21.15.89:0/810093113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe388103ff0 msgr2=0x7fe38819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:58.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.767+0000 7fe387400640 1 --2- 172.21.15.89:0/810093113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe388103ff0 0x7fe38819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:58.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.767+0000 7fe387400640 1 -- 172.21.15.89:0/810093113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38819f5a0 con 0x7fe388109120 2024-09-10T08:39:58.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.767+0000 7fe386a00640 1 --2- 172.21.15.89:0/810093113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe388103ff0 0x7fe38819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:39:58.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.767+0000 7fe386000640 1 --2- 172.21.15.89:0/810093113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3881049a0 0x7fe38819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:39:58.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.768+0000 7fe387400640 1 --2- 172.21.15.89:0/810093113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe388109120 0x7fe38819eec0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe38000eb10 tx=0x7fe38000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:58.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.768+0000 7fe37f400640 1 -- 172.21.15.89:0/810093113 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe380004b20 con 0x7fe388109120 2024-09-10T08:39:58.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.768+0000 7fe38dc00640 1 -- 172.21.15.89:0/810093113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe38819f890 con 0x7fe388109120 2024-09-10T08:39:58.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.768+0000 7fe37f400640 1 -- 172.21.15.89:0/810093113 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe380004cc0 con 0x7fe388109120 2024-09-10T08:39:58.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.768+0000 7fe37f400640 1 -- 172.21.15.89:0/810093113 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe380010700 con 0x7fe388109120 2024-09-10T08:39:58.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.768+0000 7fe38dc00640 1 -- 172.21.15.89:0/810093113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3881a7170 con 0x7fe388109120 2024-09-10T08:39:58.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.769+0000 7fe38dc00640 1 -- 172.21.15.89:0/810093113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3881043d0 con 0x7fe388109120 2024-09-10T08:39:58.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.771+0000 7fe37f400640 1 -- 172.21.15.89:0/810093113 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fe380005020 con 0x7fe388109120 2024-09-10T08:39:58.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.772+0000 7fe37f400640 1 --2- 172.21.15.89:0/810093113 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe36c0a2c10 0x7fe36c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:58.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.772+0000 7fe37f400640 1 -- 172.21.15.89:0/810093113 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fe3800cfca0 con 0x7fe388109120 2024-09-10T08:39:58.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.772+0000 7fe386a00640 1 --2- 172.21.15.89:0/810093113 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe36c0a2c10 0x7fe36c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:58.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.773+0000 7fe386a00640 1 --2- 172.21.15.89:0/810093113 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe36c0a2c10 0x7fe36c0a50d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe378009fc0 tx=0x7fe378005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:58.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.776+0000 7fe37f400640 1 -- 172.21.15.89:0/810093113 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe38009c670 con 0x7fe388109120 2024-09-10T08:39:58.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.989+0000 7fe38dc00640 1 -- 172.21.15.89:0/810093113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3881a7400 con 0x7fe388109120 2024-09-10T08:39:58.991 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:58 smithi089 ceph-mon[30757]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:39:58.991 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:58 smithi089 ceph-mon[30757]: pgmap v394: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:39:58.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.990+0000 7fe37f400640 1 -- 172.21.15.89:0/810093113 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11918 (secure 0 0 0) 0x7fe38009bf70 con 0x7fe388109120 2024-09-10T08:39:58.992 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:39:58.992 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-10T08:39:39:721431+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:39:38.834417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24518,24496]},"id":1}]} 2024-09-10T08:39:58.992 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-10T08:39:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.993+0000 7fe38dc00640 1 -- 172.21.15.89:0/810093113 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe36c0a2c10 msgr2=0x7fe36c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.993+0000 7fe38dc00640 1 --2- 172.21.15.89:0/810093113 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe36c0a2c10 0x7fe36c0a50d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe378009fc0 tx=0x7fe378005f90 comp rx=0 tx=0).stop 2024-09-10T08:39:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.993+0000 7fe38dc00640 1 -- 172.21.15.89:0/810093113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe388109120 msgr2=0x7fe38819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.993+0000 7fe38dc00640 1 --2- 172.21.15.89:0/810093113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe388109120 0x7fe38819eec0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe38000eb10 tx=0x7fe38000c2d0 comp rx=0 tx=0).stop 2024-09-10T08:39:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.993+0000 7fe38dc00640 1 -- 172.21.15.89:0/810093113 shutdown_connections 2024-09-10T08:39:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.993+0000 7fe38dc00640 1 --2- 172.21.15.89:0/810093113 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe36c0a2c10 0x7fe36c0a50d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.993+0000 7fe38dc00640 1 --2- 172.21.15.89:0/810093113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe388109120 0x7fe38819eec0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:58.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.993+0000 7fe38dc00640 1 --2- 172.21.15.89:0/810093113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3881049a0 0x7fe38819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.993+0000 7fe38dc00640 1 --2- 172.21.15.89:0/810093113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe388103ff0 0x7fe38819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.993+0000 7fe38dc00640 1 -- 172.21.15.89:0/810093113 >> 172.21.15.89:0/810093113 conn(0x7fe3880ffa90 msgr2=0x7fe388101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.993+0000 7fe38dc00640 1 -- 172.21.15.89:0/810093113 shutdown_connections 2024-09-10T08:39:58.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:39:58.993+0000 7fe38dc00640 1 -- 172.21.15.89:0/810093113 wait complete. 2024-09-10T08:39:59.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:58 smithi111 ceph-mon[35220]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:39:59.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:58 smithi111 ceph-mon[35220]: pgmap v394: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:39:59.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:58 smithi149 ceph-mon[36167]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:39:59.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:58 smithi149 ceph-mon[36167]: pgmap v394: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-10T08:39:59.843 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:40:00.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:39:59 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/810093113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:40:00.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:39:59 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/810093113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:40:00.213 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:40:00.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:39:59 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/810093113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:40:00.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.768+0000 7f5209200640 1 -- 172.21.15.89:0/2235756320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5204106f10 msgr2=0x7f520410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:00.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.768+0000 7f5209200640 1 --2- 172.21.15.89:0/2235756320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5204106f10 0x7f520410ea90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009ca0 tx=0x7f51fc02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:40:00.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.768+0000 7f5209200640 1 -- 172.21.15.89:0/2235756320 shutdown_connections 2024-09-10T08:40:00.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.768+0000 7f5209200640 1 --2- 172.21.15.89:0/2235756320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5204106f10 0x7f520410ea90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:00.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.768+0000 7f5209200640 1 --2- 172.21.15.89:0/2235756320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5204102790 0x7f52041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:00.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.768+0000 7f5209200640 1 --2- 172.21.15.89:0/2235756320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5204101de0 0x7f52041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:00.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.768+0000 7f5209200640 1 -- 172.21.15.89:0/2235756320 >> 172.21.15.89:0/2235756320 conn(0x7f52040fd8c0 msgr2=0x7f52040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:00.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.768+0000 7f5209200640 1 -- 172.21.15.89:0/2235756320 shutdown_connections 2024-09-10T08:40:00.770 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.768+0000 7f5209200640 1 -- 172.21.15.89:0/2235756320 wait complete. 2024-09-10T08:40:00.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.769+0000 7f5209200640 1 Processor -- start 2024-09-10T08:40:00.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.769+0000 7f5209200640 1 -- start start 2024-09-10T08:40:00.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.770+0000 7f5209200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5204101de0 0x7f52040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:00.771 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.770+0000 7f5209200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5204102790 0x7f5204074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:00.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.770+0000 7f5209200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5204106f10 0x7f5204074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:00.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.770+0000 7f5209200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5204111f70 con 0x7f5204101de0 2024-09-10T08:40:00.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.770+0000 7f5209200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5204111df0 con 0x7f5204106f10 2024-09-10T08:40:00.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.770+0000 7f5209200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f52041120f0 con 0x7f5204102790 2024-09-10T08:40:00.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.770+0000 7f5201e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5204101de0 0x7f52040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:00.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.770+0000 7f5201e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5204101de0 0x7f52040744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49618/0 (socket says 172.21.15.89:49618) 2024-09-10T08:40:00.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.770+0000 7f5201e00640 1 -- 172.21.15.89:0/2076892831 learned_addr learned my addr 172.21.15.89:0/2076892831 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:00.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.770+0000 7f5202800640 1 --2- 172.21.15.89:0/2076892831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5204106f10 0x7f5204074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:00.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.770+0000 7f5201400640 1 --2- 172.21.15.89:0/2076892831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5204102790 0x7f5204074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:00.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.770+0000 7f5201e00640 1 -- 172.21.15.89:0/2076892831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5204102790 msgr2=0x7f5204074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:00.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.770+0000 7f5201e00640 1 --2- 172.21.15.89:0/2076892831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5204102790 0x7f5204074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:00.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.770+0000 7f5201e00640 1 -- 172.21.15.89:0/2076892831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5204106f10 msgr2=0x7f5204074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:00.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.770+0000 7f5201e00640 1 --2- 172.21.15.89:0/2076892831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5204106f10 0x7f5204074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:00.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.770+0000 7f5201e00640 1 -- 172.21.15.89:0/2076892831 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52041a6840 con 0x7f5204101de0 2024-09-10T08:40:00.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.771+0000 7f5201e00640 1 --2- 172.21.15.89:0/2076892831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5204101de0 0x7f52040744f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f51f400e9f0 tx=0x7f51f400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:00.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.771+0000 7f51faa00640 1 -- 172.21.15.89:0/2076892831 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f400ccc0 con 0x7f5204101de0 2024-09-10T08:40:00.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.771+0000 7f51faa00640 1 -- 172.21.15.89:0/2076892831 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51f400ce60 con 0x7f5204101de0 2024-09-10T08:40:00.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.771+0000 7f51faa00640 1 -- 172.21.15.89:0/2076892831 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4010690 con 0x7f5204101de0 2024-09-10T08:40:00.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.771+0000 7f5209200640 1 -- 172.21.15.89:0/2076892831 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52041a6b30 con 0x7f5204101de0 2024-09-10T08:40:00.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.771+0000 7f5209200640 1 -- 172.21.15.89:0/2076892831 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52041a7020 con 0x7f5204101de0 2024-09-10T08:40:00.774 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.772+0000 7f5209200640 1 -- 172.21.15.89:0/2076892831 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52041021c0 con 0x7f5204101de0 2024-09-10T08:40:00.775 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.774+0000 7f51faa00640 1 -- 172.21.15.89:0/2076892831 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f51f40046c0 con 0x7f5204101de0 2024-09-10T08:40:00.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.775+0000 7f51faa00640 1 --2- 172.21.15.89:0/2076892831 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f51e80a2c10 0x7f51e80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:00.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.776+0000 7f51faa00640 1 -- 172.21.15.89:0/2076892831 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f51f40cf9c0 con 0x7f5204101de0 2024-09-10T08:40:00.776 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.776+0000 7f5201400640 1 --2- 172.21.15.89:0/2076892831 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f51e80a2c10 0x7f51e80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:00.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.776+0000 7f5201400640 1 --2- 172.21.15.89:0/2076892831 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f51e80a2c10 0x7f51e80a50d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5204108730 tx=0x7f51f000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:00.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.779+0000 7f51faa00640 1 -- 172.21.15.89:0/2076892831 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f51f409c3f0 con 0x7f5204101de0 2024-09-10T08:40:00.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.989+0000 7f5209200640 1 -- 172.21.15.89:0/2076892831 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52041a7320 con 0x7f5204101de0 2024-09-10T08:40:00.990 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:00 smithi089 ceph-mon[30757]: pgmap v395: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-10T08:40:00.990 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:00 smithi089 ceph-mon[30757]: overall HEALTH_OK 2024-09-10T08:40:00.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.989+0000 7f51faa00640 1 -- 172.21.15.89:0/2076892831 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11918 (secure 0 0 0) 0x7f51f409bcf0 con 0x7f5204101de0 2024-09-10T08:40:00.991 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:40:00.991 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-10T08:39:39:721431+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:39:38.834417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24518,24496]},"id":1}]} 2024-09-10T08:40:00.991 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-10T08:40:00.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.993+0000 7f5209200640 1 -- 172.21.15.89:0/2076892831 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f51e80a2c10 msgr2=0x7f51e80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:00.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.993+0000 7f5209200640 1 --2- 172.21.15.89:0/2076892831 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f51e80a2c10 0x7f51e80a50d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5204108730 tx=0x7f51f000a2d0 comp rx=0 tx=0).stop 2024-09-10T08:40:00.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.993+0000 7f5209200640 1 -- 172.21.15.89:0/2076892831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5204101de0 msgr2=0x7f52040744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:00.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.993+0000 7f5209200640 1 --2- 172.21.15.89:0/2076892831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5204101de0 0x7f52040744f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f51f400e9f0 tx=0x7f51f400edf0 comp rx=0 tx=0).stop 2024-09-10T08:40:00.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.993+0000 7f5209200640 1 -- 172.21.15.89:0/2076892831 shutdown_connections 2024-09-10T08:40:00.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.993+0000 7f5209200640 1 --2- 172.21.15.89:0/2076892831 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f51e80a2c10 0x7f51e80a50d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:00.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.993+0000 7f5209200640 1 --2- 172.21.15.89:0/2076892831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5204106f10 0x7f5204074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:00.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.993+0000 7f5209200640 1 --2- 172.21.15.89:0/2076892831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5204102790 0x7f5204074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:00.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.993+0000 7f5209200640 1 --2- 172.21.15.89:0/2076892831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5204101de0 0x7f52040744f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:00.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.993+0000 7f5209200640 1 -- 172.21.15.89:0/2076892831 >> 172.21.15.89:0/2076892831 conn(0x7f52040fd8c0 msgr2=0x7f52040fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:00.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.994+0000 7f5209200640 1 -- 172.21.15.89:0/2076892831 shutdown_connections 2024-09-10T08:40:00.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:00.994+0000 7f5209200640 1 -- 172.21.15.89:0/2076892831 wait complete. 2024-09-10T08:40:01.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:00 smithi111 ceph-mon[35220]: pgmap v395: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-10T08:40:01.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:00 smithi111 ceph-mon[35220]: overall HEALTH_OK 2024-09-10T08:40:01.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:00 smithi149 ceph-mon[36167]: pgmap v395: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-10T08:40:01.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:00 smithi149 ceph-mon[36167]: overall HEALTH_OK 2024-09-10T08:40:01.928 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:40:02.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:01 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2076892831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:40:02.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:01 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:40:02.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:01 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2076892831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:40:02.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:01 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:40:02.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:01 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2076892831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:40:02.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:01 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:40:02.355 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:40:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.654+0000 7f90bbc00640 1 -- 172.21.15.89:0/4198345628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4105910 msgr2=0x7f90b410c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.654+0000 7f90bbc00640 1 --2- 172.21.15.89:0/4198345628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4105910 0x7f90b410c1a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009ca0 tx=0x7f90ac02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.654+0000 7f90bbc00640 1 -- 172.21.15.89:0/4198345628 shutdown_connections 2024-09-10T08:40:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.654+0000 7f90bbc00640 1 --2- 172.21.15.89:0/4198345628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4105910 0x7f90b410c1a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.654+0000 7f90bbc00640 1 --2- 172.21.15.89:0/4198345628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90b4104f50 0x7f90b41053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.654+0000 7f90bbc00640 1 --2- 172.21.15.89:0/4198345628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90b4103d60 0x7f90b4104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.654+0000 7f90bbc00640 1 -- 172.21.15.89:0/4198345628 >> 172.21.15.89:0/4198345628 conn(0x7f90b40ff4e0 msgr2=0x7f90b4101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.654+0000 7f90bbc00640 1 -- 172.21.15.89:0/4198345628 shutdown_connections 2024-09-10T08:40:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.654+0000 7f90bbc00640 1 -- 172.21.15.89:0/4198345628 wait complete. 2024-09-10T08:40:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.655+0000 7f90bbc00640 1 Processor -- start 2024-09-10T08:40:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.656+0000 7f90bbc00640 1 -- start start 2024-09-10T08:40:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.656+0000 7f90bbc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4103d60 0x7f90b4199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.656+0000 7f90bbc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90b4104f50 0x7f90b419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.656+0000 7f90bbc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90b4105910 0x7f90b41a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.656+0000 7f90bbc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f90b410fa30 con 0x7f90b4103d60 2024-09-10T08:40:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.656+0000 7f90bbc00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f90b410f8b0 con 0x7f90b4104f50 2024-09-10T08:40:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.656+0000 7f90bbc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f90b410fbb0 con 0x7f90b4105910 2024-09-10T08:40:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.656+0000 7f90b9200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4103d60 0x7f90b4199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.657+0000 7f90b9200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4103d60 0x7f90b4199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49642/0 (socket says 172.21.15.89:49642) 2024-09-10T08:40:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.657+0000 7f90b9200640 1 -- 172.21.15.89:0/2904631854 learned_addr learned my addr 172.21.15.89:0/2904631854 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.657+0000 7f90b9c00640 1 --2- 172.21.15.89:0/2904631854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90b4105910 0x7f90b41a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.657+0000 7f90b3e00640 1 --2- 172.21.15.89:0/2904631854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90b4104f50 0x7f90b419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.657+0000 7f90b3e00640 1 -- 172.21.15.89:0/2904631854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90b4105910 msgr2=0x7f90b41a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.657+0000 7f90b3e00640 1 --2- 172.21.15.89:0/2904631854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90b4105910 0x7f90b41a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.657+0000 7f90b3e00640 1 -- 172.21.15.89:0/2904631854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4103d60 msgr2=0x7f90b4199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.657+0000 7f90b3e00640 1 --2- 172.21.15.89:0/2904631854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4103d60 0x7f90b4199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.657+0000 7f90b3e00640 1 -- 172.21.15.89:0/2904631854 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b41a1bb0 con 0x7f90b4104f50 2024-09-10T08:40:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.657+0000 7f90b3e00640 1 --2- 172.21.15.89:0/2904631854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90b4104f50 0x7f90b419a4d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f90a000e9f0 tx=0x7f90a000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.658+0000 7f90b1600640 1 -- 172.21.15.89:0/2904631854 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a000ccc0 con 0x7f90b4104f50 2024-09-10T08:40:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.658+0000 7f90b1600640 1 -- 172.21.15.89:0/2904631854 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a0010910 con 0x7f90b4104f50 2024-09-10T08:40:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.658+0000 7f90bbc00640 1 -- 172.21.15.89:0/2904631854 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b41a1ea0 con 0x7f90b4104f50 2024-09-10T08:40:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.658+0000 7f90b1600640 1 -- 172.21.15.89:0/2904631854 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0018960 con 0x7f90b4104f50 2024-09-10T08:40:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.658+0000 7f90bbc00640 1 -- 172.21.15.89:0/2904631854 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b41a23b0 con 0x7f90b4104f50 2024-09-10T08:40:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.661+0000 7f90bea6a740 1 Processor -- start 2024-09-10T08:40:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.661+0000 7f90bea6a740 1 -- start start 2024-09-10T08:40:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.661+0000 7f90bea6a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e090 0x55c63270e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.661+0000 7f90b1600640 1 -- 172.21.15.89:0/2904631854 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f90a0010ac0 con 0x7f90b4104f50 2024-09-10T08:40:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.661+0000 7f90bea6a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c63270e9d0 0x55c632713490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.661+0000 7f90bea6a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c632713a60 0x55c632715f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.661+0000 7f90bea6a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c632673100 con 0x55c63270e9d0 2024-09-10T08:40:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.661+0000 7f90bea6a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c63261b1b0 con 0x55c632713a60 2024-09-10T08:40:02.662 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.661+0000 7f90bea6a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c63266f0c0 con 0x55c63270e090 2024-09-10T08:40:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.662+0000 7f90b9200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c63270e9d0 0x55c632713490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.662+0000 7f90b3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c632713a60 0x55c632715f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.662+0000 7f90b9c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e090 0x55c63270e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.662+0000 7f90b9200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c63270e9d0 0x55c632713490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49652/0 (socket says 172.21.15.89:49652) 2024-09-10T08:40:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.662+0000 7f90b9c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e090 0x55c63270e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:42886/0 (socket says 172.21.15.89:42886) 2024-09-10T08:40:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.662+0000 7f90b9200640 1 -- 172.21.15.89:0/22841737 learned_addr learned my addr 172.21.15.89:0/22841737 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.662+0000 7f90b3e00640 1 -- 172.21.15.89:0/22841737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e090 msgr2=0x55c63270e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.662+0000 7f90b3e00640 1 --2- 172.21.15.89:0/22841737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e090 0x55c63270e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.662+0000 7f90b3e00640 1 -- 172.21.15.89:0/22841737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c63270e9d0 msgr2=0x55c632713490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.662+0000 7f90b3e00640 1 --2- 172.21.15.89:0/22841737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c63270e9d0 0x55c632713490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.662+0000 7f90b3e00640 1 -- 172.21.15.89:0/22841737 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c632386210 con 0x55c632713a60 2024-09-10T08:40:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.663+0000 7f90b3e00640 1 --2- 172.21.15.89:0/22841737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c632713a60 0x55c632715f20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f90a00cce30 tx=0x7f90a00d72a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e052cb4c35fa6034 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.663+0000 7f90a6000640 1 -- 172.21.15.89:0/22841737 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a00de070 con 0x55c632713a60 2024-09-10T08:40:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.663+0000 7f90a6000640 1 -- 172.21.15.89:0/22841737 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a00da900 con 0x55c632713a60 2024-09-10T08:40:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.663+0000 7f90a6000640 1 -- 172.21.15.89:0/22841737 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a00d15a0 con 0x55c632713a60 2024-09-10T08:40:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.663+0000 7f90b1600640 1 --2- 172.21.15.89:0/2904631854 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f909c0a2d50 0x7f909c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.663+0000 7f90b1600640 1 -- 172.21.15.89:0/2904631854 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f90a00cf8e0 con 0x7f90b4104f50 2024-09-10T08:40:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.663+0000 7f90b9c00640 1 --2- 172.21.15.89:0/2904631854 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f909c0a2d50 0x7f909c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.664+0000 7f90bea6a740 1 -- 172.21.15.89:0/22841737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c632713a60 msgr2=0x55c632715f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.664+0000 7f90bea6a740 1 --2- 172.21.15.89:0/22841737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c632713a60 0x55c632715f20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f90a00cce30 tx=0x7f90a00d72a0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.664+0000 7f90bea6a740 1 -- 172.21.15.89:0/22841737 shutdown_connections 2024-09-10T08:40:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.664+0000 7f90bea6a740 1 --2- 172.21.15.89:0/22841737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c632713a60 0x55c632715f20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.664+0000 7f90bea6a740 1 --2- 172.21.15.89:0/22841737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c63270e9d0 0x55c632713490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.664+0000 7f90bea6a740 1 --2- 172.21.15.89:0/22841737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e090 0x55c63270e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.664+0000 7f90bea6a740 1 -- 172.21.15.89:0/22841737 >> 172.21.15.89:0/22841737 conn(0x55c632709700 msgr2=0x55c632709b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.664+0000 7f90bea6a740 1 -- 172.21.15.89:0/22841737 shutdown_connections 2024-09-10T08:40:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.664+0000 7f90bea6a740 1 -- 172.21.15.89:0/22841737 wait complete. 2024-09-10T08:40:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.664+0000 7f90b9c00640 1 --2- 172.21.15.89:0/2904631854 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f909c0a2d50 0x7f909c0a5210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f90ac005e30 tx=0x7f90ac031bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.665+0000 7f90bea6a740 1 -- start start 2024-09-10T08:40:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.665+0000 7f90bea6a740 1 Processor -- start 2024-09-10T08:40:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.665+0000 7f90bea6a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c63270e090 0x55c632718240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.665+0000 7f90bea6a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e9d0 0x55c632718780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.665+0000 7f90bea6a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c632713a60 0x55c63271f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.665+0000 7f90bea6a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c632716a30 con 0x55c632713a60 2024-09-10T08:40:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.665+0000 7f90bea6a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c6325fc090 con 0x55c63270e090 2024-09-10T08:40:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.665+0000 7f90bea6a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c632716bb0 con 0x55c63270e9d0 2024-09-10T08:40:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.666+0000 7f90bea6a740 10 client.0 _ll_register_callbacks cb 0x55c632650f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.666+0000 7f90b9c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c632713a60 0x55c63271f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.666+0000 7f90b3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e9d0 0x55c632718780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.666+0000 7f90b9200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c63270e090 0x55c632718240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.666+0000 7f90b9c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c632713a60 0x55c63271f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49658/0 (socket says 172.21.15.89:49658) 2024-09-10T08:40:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.666+0000 7f90b3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e9d0 0x55c632718780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:42900/0 (socket says 172.21.15.89:42900) 2024-09-10T08:40:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.666+0000 7f90b9c00640 1 -- 172.21.15.89:0/1981229562 learned_addr learned my addr 172.21.15.89:0/1981229562 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.666+0000 7f90b9200640 1 -- 172.21.15.89:0/1981229562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e9d0 msgr2=0x55c632718780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.666+0000 7f90b9200640 1 --2- 172.21.15.89:0/1981229562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e9d0 0x55c632718780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.666+0000 7f90b9200640 1 -- 172.21.15.89:0/1981229562 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c632713a60 msgr2=0x55c63271f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.666+0000 7f90b9200640 1 --2- 172.21.15.89:0/1981229562 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c632713a60 0x55c63271f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.666+0000 7f90b9200640 1 -- 172.21.15.89:0/1981229562 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c632452620 con 0x55c63270e090 2024-09-10T08:40:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.667+0000 7f90b9200640 1 --2- 172.21.15.89:0/1981229562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c63270e090 0x55c632718240 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55c63270a580 tx=0x7f90a8012bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8703650d43b332ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.667+0000 7f909aa00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:40:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.667+0000 7f909aa00640 1 -- 172.21.15.89:0/1981229562 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a800f8f0 con 0x55c63270e090 2024-09-10T08:40:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.667+0000 7f909aa00640 1 -- 172.21.15.89:0/1981229562 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f90a800fa90 con 0x55c63270e090 2024-09-10T08:40:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.667+0000 7f90bea6a740 1 -- 172.21.15.89:0/1981229562 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a00da900 con 0x55c63270e090 2024-09-10T08:40:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.667+0000 7f909aa00640 1 -- 172.21.15.89:0/1981229562 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f90a800fc90 con 0x55c63270e090 2024-09-10T08:40:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.667+0000 7f90bea6a740 10 client.25313 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.667+0000 7f90bea6a740 1 -- 172.21.15.89:0/1981229562 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c632722680 con 0x55c63270e090 2024-09-10T08:40:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.668+0000 7f909aa00640 1 -- 172.21.15.89:0/1981229562 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a00da900 con 0x55c63270e090 2024-09-10T08:40:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.668+0000 7f909aa00640 1 -- 172.21.15.89:0/1981229562 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f90a8004b30 con 0x55c63270e090 2024-09-10T08:40:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.668+0000 7f90bea6a740 10 client.25313 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.668+0000 7f90bea6a740 10 client.25313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:40:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.668+0000 7f90bea6a740 20 client.25313 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.668+0000 7f90bea6a740 1 --2- 172.21.15.89:0/1981229562 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c632723da0 0x55c632726240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.668+0000 7f90bea6a740 4 client.25313 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:40:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.668+0000 7f90bea6a740 1 -- 172.21.15.89:0/1981229562 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c632680830 con 0x55c632723da0 2024-09-10T08:40:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.669+0000 7f90b3e00640 1 --2- 172.21.15.89:0/1981229562 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c632723da0 0x55c632726240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.669+0000 7f909aa00640 10 client.25313 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.669+0000 7f90b3e00640 1 --2- 172.21.15.89:0/1981229562 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c632723da0 0x55c632726240 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0d95e6e3a2a47fa server_cookie=cfdd90cff729b1d5 in_seq=0 out_seq=0 2024-09-10T08:40:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.670+0000 7f909aa00640 1 -- 172.21.15.89:0/1981229562 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c632680830 con 0x55c632723da0 2024-09-10T08:40:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.670+0000 7f909aa00640 10 client.25313 handle_command_reply: tid=0 2024-09-10T08:40:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.670+0000 7f90bea6a740 1 client.25313 shutdown 2024-09-10T08:40:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.670+0000 7f90bea6a740 1 -- 172.21.15.89:0/1981229562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c63270e090 msgr2=0x55c632718240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.670+0000 7f90bea6a740 1 --2- 172.21.15.89:0/1981229562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c63270e090 0x55c632718240 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55c63270a580 tx=0x7f90a8012bd0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.670+0000 7f90bea6a740 1 -- 172.21.15.89:0/1981229562 shutdown_connections 2024-09-10T08:40:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.670+0000 7f90bea6a740 1 --2- 172.21.15.89:0/1981229562 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c632723da0 0x55c632726240 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.670+0000 7f90bea6a740 1 --2- 172.21.15.89:0/1981229562 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c632713a60 0x55c63271f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.670+0000 7f90bea6a740 1 --2- 172.21.15.89:0/1981229562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e9d0 0x55c632718780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.670+0000 7f909aa00640 0 client.25313 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.670+0000 7f90bea6a740 1 --2- 172.21.15.89:0/1981229562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c63270e090 0x55c632718240 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.670+0000 7f90bea6a740 1 -- 172.21.15.89:0/1981229562 >> 172.21.15.89:0/1981229562 conn(0x55c632709700 msgr2=0x55c63270af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.671+0000 7f90bea6a740 1 -- 172.21.15.89:0/1981229562 shutdown_connections 2024-09-10T08:40:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.671+0000 7f90bea6a740 1 -- 172.21.15.89:0/1981229562 wait complete. 2024-09-10T08:40:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.671+0000 7f90bea6a740 20 client.25313 trim_cache size 0 max 16384 2024-09-10T08:40:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.686+0000 7f90bea6a740 1 Processor -- start 2024-09-10T08:40:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.686+0000 7f90bea6a740 1 -- start start 2024-09-10T08:40:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.686+0000 7f90bea6a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c63270e090 0x55c632718660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.686+0000 7f90bea6a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e9d0 0x55c63271f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.686+0000 7f90bea6a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c632713a60 0x55c63272ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.686+0000 7f90bea6a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f90b41af320 con 0x55c632713a60 2024-09-10T08:40:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.686+0000 7f90bea6a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f90b41acbf0 con 0x55c63270e090 2024-09-10T08:40:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.686+0000 7f90bea6a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f90b41aaa40 con 0x55c63270e9d0 2024-09-10T08:40:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.687+0000 7f90b9c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c632713a60 0x55c63272ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.687+0000 7f90b9c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c632713a60 0x55c63272ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49670/0 (socket says 172.21.15.89:49670) 2024-09-10T08:40:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.687+0000 7f90b9c00640 1 -- 172.21.15.89:0/2680268565 learned_addr learned my addr 172.21.15.89:0/2680268565 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.687+0000 7f90b3e00640 1 --2- 172.21.15.89:0/2680268565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e9d0 0x55c63271f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.687+0000 7f90b9200640 1 --2- 172.21.15.89:0/2680268565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c63270e090 0x55c632718660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.687+0000 7f90b3e00640 1 -- 172.21.15.89:0/2680268565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c63270e090 msgr2=0x55c632718660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.687+0000 7f90b3e00640 1 --2- 172.21.15.89:0/2680268565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c63270e090 0x55c632718660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.687+0000 7f90b3e00640 1 -- 172.21.15.89:0/2680268565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c632713a60 msgr2=0x55c63272ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.687+0000 7f90b3e00640 1 --2- 172.21.15.89:0/2680268565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c632713a60 0x55c63272ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.687+0000 7f90b3e00640 1 -- 172.21.15.89:0/2680268565 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c63270eeb0 con 0x55c63270e9d0 2024-09-10T08:40:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.687+0000 7f90b9c00640 1 --2- 172.21.15.89:0/2680268565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c632713a60 0x55c63272ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.687+0000 7f90b3e00640 1 --2- 172.21.15.89:0/2680268565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e9d0 0x55c63271f730 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f90a009bbf0 tx=0x7f90a0009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=162306d652a015dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.688+0000 7f90a5600640 1 -- 172.21.15.89:0/2680268565 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a00de040 con 0x55c63270e9d0 2024-09-10T08:40:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.688+0000 7f90a5600640 1 -- 172.21.15.89:0/2680268565 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a009bc60 con 0x55c63270e9d0 2024-09-10T08:40:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.688+0000 7f90a5600640 1 -- 172.21.15.89:0/2680268565 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a009a030 con 0x55c63270e9d0 2024-09-10T08:40:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.689+0000 7f90bea6a740 1 -- 172.21.15.89:0/2680268565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e9d0 msgr2=0x55c63271f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.689+0000 7f90bea6a740 1 --2- 172.21.15.89:0/2680268565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e9d0 0x55c63271f730 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f90a009bbf0 tx=0x7f90a0009700 comp rx=0 tx=0).stop 2024-09-10T08:40:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.689+0000 7f90bea6a740 1 -- 172.21.15.89:0/2680268565 shutdown_connections 2024-09-10T08:40:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.689+0000 7f90bea6a740 1 --2- 172.21.15.89:0/2680268565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c632713a60 0x55c63272ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.689+0000 7f90bea6a740 1 --2- 172.21.15.89:0/2680268565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e9d0 0x55c63271f730 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.689+0000 7f90bea6a740 1 --2- 172.21.15.89:0/2680268565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c63270e090 0x55c632718660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.689+0000 7f90bea6a740 1 -- 172.21.15.89:0/2680268565 >> 172.21.15.89:0/2680268565 conn(0x55c632709700 msgr2=0x55c632708880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.689+0000 7f90bea6a740 1 -- 172.21.15.89:0/2680268565 shutdown_connections 2024-09-10T08:40:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.689+0000 7f90bea6a740 1 -- 172.21.15.89:0/2680268565 wait complete. 2024-09-10T08:40:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.689+0000 7f90bea6a740 1 -- start start 2024-09-10T08:40:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.689+0000 7f90bea6a740 1 Processor -- start 2024-09-10T08:40:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.690+0000 7f90bea6a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e090 0x55c63271d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.690+0000 7f90bea6a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c63270e9d0 0x55c63271dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.690+0000 7f90bea6a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c632713a60 0x55c63271ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.690+0000 7f90bea6a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f90b41ab0b0 con 0x55c63270e9d0 2024-09-10T08:40:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.690+0000 7f90bea6a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f90b41ac520 con 0x55c632713a60 2024-09-10T08:40:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.690+0000 7f90bea6a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f90b41ab780 con 0x55c63270e090 2024-09-10T08:40:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.690+0000 7f90bea6a740 10 client.0 _ll_register_callbacks cb 0x7f90a80098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.690+0000 7f90b9c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c632713a60 0x55c63271ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.690+0000 7f90b9c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c632713a60 0x55c63271ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35906/0 (socket says 172.21.15.89:35906) 2024-09-10T08:40:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.690+0000 7f90b3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c63270e9d0 0x55c63271dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.690+0000 7f90b9200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e090 0x55c63271d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.690+0000 7f90b9c00640 1 -- 172.21.15.89:0/626989045 learned_addr learned my addr 172.21.15.89:0/626989045 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.690+0000 7f90b9c00640 1 -- 172.21.15.89:0/626989045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e090 msgr2=0x55c63271d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.691+0000 7f90b9c00640 1 --2- 172.21.15.89:0/626989045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e090 0x55c63271d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.691+0000 7f90b9c00640 1 -- 172.21.15.89:0/626989045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c63270e9d0 msgr2=0x55c63271dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.691+0000 7f90b9c00640 1 --2- 172.21.15.89:0/626989045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c63270e9d0 0x55c63271dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.691+0000 7f90b9c00640 1 -- 172.21.15.89:0/626989045 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c63271f210 con 0x55c632713a60 2024-09-10T08:40:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.691+0000 7f90b3e00640 1 --2- 172.21.15.89:0/626989045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c63270e9d0 0x55c63271dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.691+0000 7f90b9200640 1 --2- 172.21.15.89:0/626989045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e090 0x55c63271d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.691+0000 7f90b9c00640 1 --2- 172.21.15.89:0/626989045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c632713a60 0x55c63271ec00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009ca0 tx=0x7f90ac02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f22c7010be40eff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.691+0000 7f909aa00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:40:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.691+0000 7f909aa00640 1 -- 172.21.15.89:0/626989045 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac003e20 con 0x55c632713a60 2024-09-10T08:40:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.691+0000 7f909aa00640 1 -- 172.21.15.89:0/626989045 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f90ac004250 con 0x55c632713a60 2024-09-10T08:40:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.692+0000 7f90bea6a740 1 -- 172.21.15.89:0/626989045 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a009bc60 con 0x55c632713a60 2024-09-10T08:40:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.692+0000 7f909aa00640 1 -- 172.21.15.89:0/626989045 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f90ac004450 con 0x55c632713a60 2024-09-10T08:40:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.692+0000 7f90bea6a740 10 client.25319 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.692+0000 7f90bea6a740 1 -- 172.21.15.89:0/626989045 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c63278f440 con 0x55c632713a60 2024-09-10T08:40:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.693+0000 7f909aa00640 1 -- 172.21.15.89:0/626989045 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a009bc60 con 0x55c632713a60 2024-09-10T08:40:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.693+0000 7f909aa00640 1 -- 172.21.15.89:0/626989045 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f90ac053030 con 0x55c632713a60 2024-09-10T08:40:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.693+0000 7f90bea6a740 10 client.25319 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.693+0000 7f90bea6a740 10 client.25319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:40:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.693+0000 7f90bea6a740 20 client.25319 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.693+0000 7f90bea6a740 1 --2- 172.21.15.89:0/626989045 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c63278fe70 0x55c632792290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.693+0000 7f90bea6a740 4 client.25319 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:40:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.693+0000 7f90bea6a740 1 -- 172.21.15.89:0/626989045 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c63267ab80 con 0x55c63278fe70 2024-09-10T08:40:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.693+0000 7f90b9200640 1 --2- 172.21.15.89:0/626989045 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c63278fe70 0x55c632792290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.694+0000 7f90b9200640 1 --2- 172.21.15.89:0/626989045 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c63278fe70 0x55c632792290 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f25e9bb4792e114 server_cookie=957254a0345cfcbc in_seq=0 out_seq=0 2024-09-10T08:40:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.694+0000 7f909aa00640 10 client.25319 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.706+0000 7f909aa00640 1 -- 172.21.15.89:0/626989045 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c63267ab80 con 0x55c63278fe70 2024-09-10T08:40:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.706+0000 7f909aa00640 10 client.25319 handle_command_reply: tid=0 2024-09-10T08:40:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.706+0000 7f90bea6a740 1 client.25319 shutdown 2024-09-10T08:40:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.706+0000 7f90bea6a740 1 -- 172.21.15.89:0/626989045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c632713a60 msgr2=0x55c63271ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.706+0000 7f90bea6a740 1 --2- 172.21.15.89:0/626989045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c632713a60 0x55c63271ec00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009ca0 tx=0x7f90ac02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.707+0000 7f90bea6a740 1 -- 172.21.15.89:0/626989045 shutdown_connections 2024-09-10T08:40:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.707+0000 7f90bea6a740 1 --2- 172.21.15.89:0/626989045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c632713a60 0x55c63271ec00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.707+0000 7f90bea6a740 1 --2- 172.21.15.89:0/626989045 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c63278fe70 0x55c632792290 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.707+0000 7f90bea6a740 1 --2- 172.21.15.89:0/626989045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c63270e9d0 0x55c63271dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.707+0000 7f90bea6a740 1 --2- 172.21.15.89:0/626989045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c63270e090 0x55c63271d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.707+0000 7f90bea6a740 1 -- 172.21.15.89:0/626989045 >> 172.21.15.89:0/626989045 conn(0x55c632709700 msgr2=0x55c63270a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.707+0000 7f909aa00640 0 client.25319 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.707+0000 7f90bea6a740 1 -- 172.21.15.89:0/626989045 shutdown_connections 2024-09-10T08:40:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.707+0000 7f90bea6a740 1 -- 172.21.15.89:0/626989045 wait complete. 2024-09-10T08:40:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.707+0000 7f90bea6a740 20 client.25319 trim_cache size 0 max 16384 2024-09-10T08:40:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:40:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:40:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:40:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:40:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.708+0000 7f90bbc00640 1 -- 172.21.15.89:0/2904631854 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f909c0a2d50 msgr2=0x7f909c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.708+0000 7f90bbc00640 1 --2- 172.21.15.89:0/2904631854 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f909c0a2d50 0x7f909c0a5210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f90ac005e30 tx=0x7f90ac031bb0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.708+0000 7f90bbc00640 1 -- 172.21.15.89:0/2904631854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90b4104f50 msgr2=0x7f90b419a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.708+0000 7f90bbc00640 1 --2- 172.21.15.89:0/2904631854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90b4104f50 0x7f90b419a4d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f90a000e9f0 tx=0x7f90a000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.708+0000 7f90bbc00640 1 -- 172.21.15.89:0/2904631854 shutdown_connections 2024-09-10T08:40:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.708+0000 7f90bbc00640 1 --2- 172.21.15.89:0/2904631854 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f909c0a2d50 0x7f909c0a5210 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.708+0000 7f90bbc00640 1 --2- 172.21.15.89:0/2904631854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90b4105910 0x7f90b41a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.708+0000 7f90bbc00640 1 --2- 172.21.15.89:0/2904631854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90b4104f50 0x7f90b419a4d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.708+0000 7f90bbc00640 1 --2- 172.21.15.89:0/2904631854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4103d60 0x7f90b4199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.708+0000 7f90bbc00640 1 -- 172.21.15.89:0/2904631854 >> 172.21.15.89:0/2904631854 conn(0x7f90b40ff4e0 msgr2=0x7f90b41081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.708+0000 7f90bbc00640 1 -- 172.21.15.89:0/2904631854 shutdown_connections 2024-09-10T08:40:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:02.708+0000 7f90bbc00640 1 -- 172.21.15.89:0/2904631854 wait complete. 2024-09-10T08:40:02.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.959+0000 7fb05f200640 1 -- 172.21.15.89:0/2071778030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb058102b90 msgr2=0x7fb058102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:02.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.959+0000 7fb05f200640 1 --2- 172.21.15.89:0/2071778030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb058102b90 0x7fb058102f70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009900 tx=0x7fb04c02a480 comp rx=0 tx=0).stop 2024-09-10T08:40:02.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.959+0000 7fb05f200640 1 -- 172.21.15.89:0/2071778030 shutdown_connections 2024-09-10T08:40:02.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.959+0000 7fb05f200640 1 --2- 172.21.15.89:0/2071778030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb05810bfc0 0x7fb05810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.959+0000 7fb05f200640 1 --2- 172.21.15.89:0/2071778030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0581034b0 0x7fb05810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.959+0000 7fb05f200640 1 --2- 172.21.15.89:0/2071778030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb058102b90 0x7fb058102f70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.959+0000 7fb05f200640 1 -- 172.21.15.89:0/2071778030 >> 172.21.15.89:0/2071778030 conn(0x7fb0580fc610 msgr2=0x7fb0580fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:02.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.959+0000 7fb05f200640 1 -- 172.21.15.89:0/2071778030 shutdown_connections 2024-09-10T08:40:02.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.959+0000 7fb05f200640 1 -- 172.21.15.89:0/2071778030 wait complete. 2024-09-10T08:40:02.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.960+0000 7fb05f200640 1 Processor -- start 2024-09-10T08:40:02.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.960+0000 7fb05f200640 1 -- start start 2024-09-10T08:40:02.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.961+0000 7fb05f200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb058102b90 0x7fb0581982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.961+0000 7fb05f200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0581034b0 0x7fb058198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.961+0000 7fb05f200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb05810bfc0 0x7fb05819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.961+0000 7fb05f200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb058111dc0 con 0x7fb0581034b0 2024-09-10T08:40:02.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.961+0000 7fb05f200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb058111c40 con 0x7fb05810bfc0 2024-09-10T08:40:02.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.961+0000 7fb05f200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb058111f40 con 0x7fb058102b90 2024-09-10T08:40:02.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.961+0000 7fb05d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0581034b0 0x7fb058198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.961+0000 7fb05de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb058102b90 0x7fb0581982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.961+0000 7fb05e800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb05810bfc0 0x7fb05819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.961+0000 7fb05d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0581034b0 0x7fb058198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33498/0 (socket says 172.21.15.89:33498) 2024-09-10T08:40:02.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.961+0000 7fb05d400640 1 -- 172.21.15.89:0/636227913 learned_addr learned my addr 172.21.15.89:0/636227913 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:02.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.962+0000 7fb05de00640 1 -- 172.21.15.89:0/636227913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb05810bfc0 msgr2=0x7fb05819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:02.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.962+0000 7fb05de00640 1 --2- 172.21.15.89:0/636227913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb05810bfc0 0x7fb05819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.962+0000 7fb05de00640 1 -- 172.21.15.89:0/636227913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0581034b0 msgr2=0x7fb058198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:02.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.962+0000 7fb05de00640 1 --2- 172.21.15.89:0/636227913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0581034b0 0x7fb058198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:02.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.962+0000 7fb05de00640 1 -- 172.21.15.89:0/636227913 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05819d270 con 0x7fb058102b90 2024-09-10T08:40:02.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.962+0000 7fb05de00640 1 --2- 172.21.15.89:0/636227913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb058102b90 0x7fb0581982c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb04c005940 tx=0x7fb04c0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:02.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.962+0000 7fb052a00640 1 -- 172.21.15.89:0/636227913 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c03f030 con 0x7fb058102b90 2024-09-10T08:40:02.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.962+0000 7fb05f200640 1 -- 172.21.15.89:0/636227913 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05819d500 con 0x7fb058102b90 2024-09-10T08:40:02.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.962+0000 7fb052a00640 1 -- 172.21.15.89:0/636227913 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb04c0044c0 con 0x7fb058102b90 2024-09-10T08:40:02.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.962+0000 7fb052a00640 1 -- 172.21.15.89:0/636227913 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c035070 con 0x7fb058102b90 2024-09-10T08:40:02.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.963+0000 7fb05f200640 1 -- 172.21.15.89:0/636227913 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0581a4e40 con 0x7fb058102b90 2024-09-10T08:40:02.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.964+0000 7fb05f200640 1 -- 172.21.15.89:0/636227913 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb05806a850 con 0x7fb058102b90 2024-09-10T08:40:02.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.965+0000 7fb052a00640 1 -- 172.21.15.89:0/636227913 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fb04c042030 con 0x7fb058102b90 2024-09-10T08:40:02.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.967+0000 7fb052a00640 1 --2- 172.21.15.89:0/636227913 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb0400a2c10 0x7fb0400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:02.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.967+0000 7fb052a00640 1 -- 172.21.15.89:0/636227913 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb04c0ef8a0 con 0x7fb058102b90 2024-09-10T08:40:02.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.967+0000 7fb05d400640 1 --2- 172.21.15.89:0/636227913 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb0400a2c10 0x7fb0400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:02.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.968+0000 7fb05d400640 1 --2- 172.21.15.89:0/636227913 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb0400a2c10 0x7fb0400a50d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb05810d390 tx=0x7fb048005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:02.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:02.970+0000 7fb052a00640 1 -- 172.21.15.89:0/636227913 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb04c0bbc20 con 0x7fb058102b90 2024-09-10T08:40:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:02 smithi111 ceph-mon[35220]: pgmap v396: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-10T08:40:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:02 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:40:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:02 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:40:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:02 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:40:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:02 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:40:03.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:02 smithi089 ceph-mon[30757]: pgmap v396: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-10T08:40:03.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:02 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:40:03.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:02 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:40:03.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:02 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:40:03.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:02 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:40:03.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:03.155+0000 7fb05f200640 1 -- 172.21.15.89:0/636227913 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb058107060 con 0x7fb058102b90 2024-09-10T08:40:03.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:03.157+0000 7fb052a00640 1 -- 172.21.15.89:0/636227913 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24894 (secure 0 0 0) 0x7fb04c0bff20 con 0x7fb058102b90 2024-09-10T08:40:03.158 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:40:03.158 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":96,"fsid":"91f747bc-6f4e-11ef-bcea-c7b262605968","created":"2024-09-10T08:28:49.697035+0000","modified":"2024-09-10T08:38:43.618497+0000","last_up_change":"2024-09-10T08:36:00.535433+0000","last_in_change":"2024-09-10T08:35:37.131131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:32:02.060714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:36:52.462301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:36:55.119708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:37:03.729002+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d3b55192-bd95-4497-89ea-dc017a25c0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6803","nonce":1760523284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6805","nonce":1760523284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6809","nonce":1760523284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6807","nonce":1760523284}]},"public_addr":"172.21.15.89:6803/1760523284","cluster_addr":"172.21.15.89:6805/1760523284","heartbeat_back_addr":"172.21.15.89:6809/1760523284","heartbeat_front_addr":"172.21.15.89:6807/1760523284","state":["exists","up"]},{"osd":1,"uuid":"60aae117-cb52-4034-a700-11693171cd52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6801","nonce":2369291742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6803","nonce":2369291742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6807","nonce":2369291742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6805","nonce":2369291742}]},"public_addr":"172.21.15.111:6801/2369291742","cluster_addr":"172.21.15.111:6803/2369291742","heartbeat_back_addr":"172.21.15.111:6807/2369291742","heartbeat_front_addr":"172.21.15.111:6805/2369291742","state":["exists","up"]},{"osd":2,"uuid":"a305a019-0870-437c-a298-4f923ec54e77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6801","nonce":1092261395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6803","nonce":1092261395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6807","nonce":1092261395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6805","nonce":1092261395}]},"public_addr":"172.21.15.149:6801/1092261395","cluster_addr":"172.21.15.149:6803/1092261395","heartbeat_back_addr":"172.21.15.149:6807/1092261395","heartbeat_front_addr":"172.21.15.149:6805/1092261395","state":["exists","up"]},{"osd":3,"uuid":"d02f54c1-f883-40f6-9341-8d9fc4b712ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6811","nonce":1405773950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6813","nonce":1405773950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6817","nonce":1405773950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6815","nonce":1405773950}]},"public_addr":"172.21.15.89:6811/1405773950","cluster_addr":"172.21.15.89:6813/1405773950","heartbeat_back_addr":"172.21.15.89:6817/1405773950","heartbeat_front_addr":"172.21.15.89:6815/1405773950","state":["exists","up"]},{"osd":4,"uuid":"23cff46b-88dd-4344-be06-b35db6014f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6809","nonce":316717323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6811","nonce":316717323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6815","nonce":316717323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6813","nonce":316717323}]},"public_addr":"172.21.15.111:6809/316717323","cluster_addr":"172.21.15.111:6811/316717323","heartbeat_back_addr":"172.21.15.111:6815/316717323","heartbeat_front_addr":"172.21.15.111:6813/316717323","state":["exists","up"]},{"osd":5,"uuid":"8e516e8d-e7c6-4ebc-80a5-29932161df4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6809","nonce":1256148461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6811","nonce":1256148461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6815","nonce":1256148461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6813","nonce":1256148461}]},"public_addr":"172.21.15.149:6809/1256148461","cluster_addr":"172.21.15.149:6811/1256148461","heartbeat_back_addr":"172.21.15.149:6815/1256148461","heartbeat_front_addr":"172.21.15.149:6813/1256148461","state":["exists","up"]},{"osd":6,"uuid":"48f92472-f657-4c5a-adeb-20d4fea1ce8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6819","nonce":4190488024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6821","nonce":4190488024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6825","nonce":4190488024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6823","nonce":4190488024}]},"public_addr":"172.21.15.89:6819/4190488024","cluster_addr":"172.21.15.89:6821/4190488024","heartbeat_back_addr":"172.21.15.89:6825/4190488024","heartbeat_front_addr":"172.21.15.89:6823/4190488024","state":["exists","up"]},{"osd":7,"uuid":"14a2a6f0-c4cd-406b-b637-7527fd32babe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6817","nonce":652666376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6819","nonce":652666376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6823","nonce":652666376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6821","nonce":652666376}]},"public_addr":"172.21.15.111:6817/652666376","cluster_addr":"172.21.15.111:6819/652666376","heartbeat_back_addr":"172.21.15.111:6823/652666376","heartbeat_front_addr":"172.21.15.111:6821/652666376","state":["exists","up"]},{"osd":8,"uuid":"a54502e6-95f7-431f-8db1-54a6c7983b5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6817","nonce":2955172869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6819","nonce":2955172869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6823","nonce":2955172869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6821","nonce":2955172869}]},"public_addr":"172.21.15.149:6817/2955172869","cluster_addr":"172.21.15.149:6819/2955172869","heartbeat_back_addr":"172.21.15.149:6823/2955172869","heartbeat_front_addr":"172.21.15.149:6821/2955172869","state":["exists","up"]},{"osd":9,"uuid":"5fe17344-5b1c-4601-a633-639cc1691478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6827","nonce":2427467040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6829","nonce":2427467040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6833","nonce":2427467040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6831","nonce":2427467040}]},"public_addr":"172.21.15.89:6827/2427467040","cluster_addr":"172.21.15.89:6829/2427467040","heartbeat_back_addr":"172.21.15.89:6833/2427467040","heartbeat_front_addr":"172.21.15.89:6831/2427467040","state":["exists","up"]},{"osd":10,"uuid":"d8f29890-3064-4b67-b649-467a0ac86db7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6825","nonce":2328678744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6827","nonce":2328678744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6831","nonce":2328678744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6829","nonce":2328678744}]},"public_addr":"172.21.15.111:6825/2328678744","cluster_addr":"172.21.15.111:6827/2328678744","heartbeat_back_addr":"172.21.15.111:6831/2328678744","heartbeat_front_addr":"172.21.15.111:6829/2328678744","state":["exists","up"]},{"osd":11,"uuid":"92182c73-f139-47ca-a0ef-5046ae9b2310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6825","nonce":3130508827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6827","nonce":3130508827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6831","nonce":3130508827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6829","nonce":3130508827}]},"public_addr":"172.21.15.149:6825/3130508827","cluster_addr":"172.21.15.149:6827/3130508827","heartbeat_back_addr":"172.21.15.149:6831/3130508827","heartbeat_front_addr":"172.21.15.149:6829/3130508827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:09.031810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:33.695124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:58.065510+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:27.592838+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:53.275289+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:18.305677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:47.781535+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:12.889512+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:37.744193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:07.718408+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:33.391027+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:59.013250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/850669510":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/3541027428":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6801/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6800/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/2396468787":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/1609720980":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/995019144":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6801/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6800/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/2179977257":"2024-09-11T08:29:14.554869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:40:03.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:03.159+0000 7fb05f200640 1 -- 172.21.15.89:0/636227913 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb0400a2c10 msgr2=0x7fb0400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:03.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:03.159+0000 7fb05f200640 1 --2- 172.21.15.89:0/636227913 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb0400a2c10 0x7fb0400a50d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb05810d390 tx=0x7fb048005f90 comp rx=0 tx=0).stop 2024-09-10T08:40:03.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:03.159+0000 7fb05f200640 1 -- 172.21.15.89:0/636227913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb058102b90 msgr2=0x7fb0581982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:03.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:03.159+0000 7fb05f200640 1 --2- 172.21.15.89:0/636227913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb058102b90 0x7fb0581982c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb04c005940 tx=0x7fb04c0040c0 comp rx=0 tx=0).stop 2024-09-10T08:40:03.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:03.160+0000 7fb05f200640 1 -- 172.21.15.89:0/636227913 shutdown_connections 2024-09-10T08:40:03.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:03.160+0000 7fb05f200640 1 --2- 172.21.15.89:0/636227913 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb0400a2c10 0x7fb0400a50d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:03.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:03.160+0000 7fb05f200640 1 --2- 172.21.15.89:0/636227913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb05810bfc0 0x7fb05819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:03.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:03.160+0000 7fb05f200640 1 --2- 172.21.15.89:0/636227913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0581034b0 0x7fb058198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:03.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:03.160+0000 7fb05f200640 1 --2- 172.21.15.89:0/636227913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb058102b90 0x7fb0581982c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:03.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:03.160+0000 7fb05f200640 1 -- 172.21.15.89:0/636227913 >> 172.21.15.89:0/636227913 conn(0x7fb0580fc610 msgr2=0x7fb0580fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:03.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:03.160+0000 7fb05f200640 1 -- 172.21.15.89:0/636227913 shutdown_connections 2024-09-10T08:40:03.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:03.160+0000 7fb05f200640 1 -- 172.21.15.89:0/636227913 wait complete. 2024-09-10T08:40:03.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:02 smithi149 ceph-mon[36167]: pgmap v396: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-10T08:40:03.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:02 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:40:03.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:02 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:40:03.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:02 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:40:03.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:02 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:40:03.974 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-10T08:40:04.024 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:03 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:04.025 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:03 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:40:04.025 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:03 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:40:04.025 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:03 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/636227913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:40:04.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:03 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:04.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:03 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:40:04.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:03 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:40:04.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:03 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/636227913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:40:04.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:03 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:04.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:03 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:40:04.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:03 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:40:04.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:03 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/636227913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:40:04.352 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:40:05.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:04 smithi089 ceph-mon[30757]: pgmap v397: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-10T08:40:05.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:04 smithi149 ceph-mon[36167]: pgmap v397: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-10T08:40:05.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:04 smithi111 ceph-mon[35220]: pgmap v397: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-10T08:40:05.808 INFO:teuthology.orchestra.run.smithi089.stdout:{"run_dir":"/var/run/ceph"} 2024-09-10T08:40:05.809 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-10T08:40:05.809 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 9893 2024-09-10T08:40:05.809 DEBUG:tasks.quiescer:Yielding 2024-09-10T08:40:05.809 INFO:teuthology.run_tasks:Running task workunit... 2024-09-10T08:40:05.823 INFO:tasks.workunit:Pulling workunits from ref 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:40:05.824 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-10T08:40:05.824 INFO:tasks.workunit:timeout=3h 2024-09-10T08:40:05.824 INFO:tasks.workunit:cleanup=True 2024-09-10T08:40:05.824 DEBUG:teuthology.orchestra.run.smithi089:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-10T08:40:05.825 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:40:05.862 INFO:teuthology.orchestra.run.smithi089.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-10T08:40:05.862 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-10T08:40:05.862 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-10T08:40:05.863 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:40:05.863 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-10T08:40:05.863 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-10 08:37:59.252094823 +0000 2024-09-10T08:40:05.863 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-10 08:37:59.252094823 +0000 2024-09-10T08:40:05.863 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-10 08:39:37.971370719 +0000 2024-09-10T08:40:05.863 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-10 08:37:59.252094823 +0000 2024-09-10T08:40:05.863 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-10T08:40:05.863 DEBUG:teuthology.orchestra.run.smithi089:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-10T08:40:05.951 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:05 smithi089 ceph-mon[30757]: scrub summary: idle 2024-09-10T08:40:05.952 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:05 smithi089 ceph-mon[30757]: pgmap v398: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-10T08:40:05.969 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:40:06.008 INFO:tasks.workunit.client.0.smithi089.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-10T08:40:06.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:05 smithi149 ceph-mon[36167]: scrub summary: idle 2024-09-10T08:40:06.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:05 smithi149 ceph-mon[36167]: pgmap v398: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-10T08:40:06.235 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:40:06.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:05 smithi111 ceph-mon[35220]: scrub summary: idle 2024-09-10T08:40:06.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:05 smithi111 ceph-mon[35220]: pgmap v398: 129 pgs: 129 active+clean; 653 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-10T08:40:06.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.800+0000 7fef07200640 1 -- 172.21.15.89:0/2648527447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef08103e30 msgr2=0x7fef08104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:06.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.800+0000 7fef07200640 1 --2- 172.21.15.89:0/2648527447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef08103e30 0x7fef08104210 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7feef8009900 tx=0x7feef802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:40:06.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.800+0000 7fef07200640 1 -- 172.21.15.89:0/2648527447 shutdown_connections 2024-09-10T08:40:06.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.800+0000 7fef07200640 1 --2- 172.21.15.89:0/2648527447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef08108f60 0x7fef08110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:06.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.800+0000 7fef07200640 1 --2- 172.21.15.89:0/2648527447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef081047e0 0x7fef08108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:06.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.800+0000 7fef07200640 1 --2- 172.21.15.89:0/2648527447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef08103e30 0x7fef08104210 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:06.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.800+0000 7fef07200640 1 -- 172.21.15.89:0/2648527447 >> 172.21.15.89:0/2648527447 conn(0x7fef080ff8d0 msgr2=0x7fef08101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:06.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.800+0000 7fef07200640 1 -- 172.21.15.89:0/2648527447 shutdown_connections 2024-09-10T08:40:06.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.800+0000 7fef07200640 1 -- 172.21.15.89:0/2648527447 wait complete. 2024-09-10T08:40:06.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.801+0000 7fef07200640 1 Processor -- start 2024-09-10T08:40:06.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.801+0000 7fef07200640 1 -- start start 2024-09-10T08:40:06.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.802+0000 7fef07200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef08103e30 0x7fef0819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:06.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.802+0000 7fef07200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef081047e0 0x7fef0819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:06.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.802+0000 7fef07200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef08108f60 0x7fef0819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:06.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.802+0000 7fef07200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fef08113fd0 con 0x7fef081047e0 2024-09-10T08:40:06.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.802+0000 7fef07200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef08113e50 con 0x7fef08103e30 2024-09-10T08:40:06.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.802+0000 7fef07200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fef08114150 con 0x7fef08108f60 2024-09-10T08:40:06.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.802+0000 7fef05400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef081047e0 0x7fef0819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:06.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.802+0000 7fef05e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef08103e30 0x7fef0819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:06.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.802+0000 7fef05e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef08103e30 0x7fef0819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:60802/0 (socket says 172.21.15.89:60802) 2024-09-10T08:40:06.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.802+0000 7fef05e00640 1 -- 172.21.15.89:0/1858775790 learned_addr learned my addr 172.21.15.89:0/1858775790 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:06.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.802+0000 7fef06800640 1 --2- 172.21.15.89:0/1858775790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef08108f60 0x7fef0819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:06.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.803+0000 7fef05400640 1 -- 172.21.15.89:0/1858775790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef08108f60 msgr2=0x7fef0819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:06.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.803+0000 7fef05400640 1 --2- 172.21.15.89:0/1858775790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef08108f60 0x7fef0819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:06.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.803+0000 7fef05400640 1 -- 172.21.15.89:0/1858775790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef08103e30 msgr2=0x7fef0819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:06.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.803+0000 7fef05400640 1 --2- 172.21.15.89:0/1858775790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef08103e30 0x7fef0819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:06.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.803+0000 7fef05400640 1 -- 172.21.15.89:0/1858775790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0819f420 con 0x7fef081047e0 2024-09-10T08:40:06.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.803+0000 7fef05e00640 1 --2- 172.21.15.89:0/1858775790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef08103e30 0x7fef0819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:06.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.803+0000 7fef05400640 1 --2- 172.21.15.89:0/1858775790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef081047e0 0x7fef0819a9b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7feef400d6d0 tx=0x7feef400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:06.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.803+0000 7feefea00640 1 -- 172.21.15.89:0/1858775790 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4017070 con 0x7fef081047e0 2024-09-10T08:40:06.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.803+0000 7feefea00640 1 -- 172.21.15.89:0/1858775790 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feef4009820 con 0x7fef081047e0 2024-09-10T08:40:06.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.803+0000 7fef07200640 1 -- 172.21.15.89:0/1858775790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0819f710 con 0x7fef081047e0 2024-09-10T08:40:06.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.803+0000 7feefea00640 1 -- 172.21.15.89:0/1858775790 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4004d00 con 0x7fef081047e0 2024-09-10T08:40:06.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.803+0000 7fef07200640 1 -- 172.21.15.89:0/1858775790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef081a6ff0 con 0x7fef081047e0 2024-09-10T08:40:06.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.805+0000 7fef07200640 1 -- 172.21.15.89:0/1858775790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef08104210 con 0x7fef081047e0 2024-09-10T08:40:06.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.806+0000 7feefea00640 1 -- 172.21.15.89:0/1858775790 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7feef4005430 con 0x7fef081047e0 2024-09-10T08:40:06.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.808+0000 7feefea00640 1 --2- 172.21.15.89:0/1858775790 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7feeec0a2c90 0x7feeec0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:06.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.808+0000 7feefea00640 1 -- 172.21.15.89:0/1858775790 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7feef40ced30 con 0x7fef081047e0 2024-09-10T08:40:06.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.808+0000 7fef05e00640 1 --2- 172.21.15.89:0/1858775790 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7feeec0a2c90 0x7feeec0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:06.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.809+0000 7fef05e00640 1 --2- 172.21.15.89:0/1858775790 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7feeec0a2c90 0x7feeec0a5150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7feef80096c0 tx=0x7feef80046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:06.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:06.812+0000 7feefea00640 1 -- 172.21.15.89:0/1858775790 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feef409b0b0 con 0x7fef081047e0 2024-09-10T08:40:07.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:07.018+0000 7fef07200640 1 -- 172.21.15.89:0/1858775790 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef0802d7c0 con 0x7fef081047e0 2024-09-10T08:40:07.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:07.019+0000 7feefea00640 1 -- 172.21.15.89:0/1858775790 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11918 (secure 0 0 0) 0x7feef409f3b0 con 0x7fef081047e0 2024-09-10T08:40:07.020 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:40:07.020 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-10T08:39:39:721431+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:39:38.834417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24518,24496]},"id":1}]} 2024-09-10T08:40:07.021 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-10T08:40:07.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:07.021+0000 7fef07200640 1 -- 172.21.15.89:0/1858775790 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7feeec0a2c90 msgr2=0x7feeec0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:07.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:07.021+0000 7fef07200640 1 --2- 172.21.15.89:0/1858775790 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7feeec0a2c90 0x7feeec0a5150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7feef80096c0 tx=0x7feef80046c0 comp rx=0 tx=0).stop 2024-09-10T08:40:07.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:07.022+0000 7fef07200640 1 -- 172.21.15.89:0/1858775790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef081047e0 msgr2=0x7fef0819a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:07.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:07.022+0000 7fef07200640 1 --2- 172.21.15.89:0/1858775790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef081047e0 0x7fef0819a9b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7feef400d6d0 tx=0x7feef400db90 comp rx=0 tx=0).stop 2024-09-10T08:40:07.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:07.022+0000 7fef07200640 1 -- 172.21.15.89:0/1858775790 shutdown_connections 2024-09-10T08:40:07.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:07.022+0000 7fef07200640 1 --2- 172.21.15.89:0/1858775790 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7feeec0a2c90 0x7feeec0a5150 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:07.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:07.022+0000 7fef07200640 1 --2- 172.21.15.89:0/1858775790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef08108f60 0x7fef0819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:07.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:07.022+0000 7fef07200640 1 --2- 172.21.15.89:0/1858775790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef081047e0 0x7fef0819a9b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:07.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:07.022+0000 7fef07200640 1 --2- 172.21.15.89:0/1858775790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef08103e30 0x7fef0819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:07.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:07.022+0000 7fef07200640 1 -- 172.21.15.89:0/1858775790 >> 172.21.15.89:0/1858775790 conn(0x7fef080ff8d0 msgr2=0x7fef08100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:07.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:07.022+0000 7fef07200640 1 -- 172.21.15.89:0/1858775790 shutdown_connections 2024-09-10T08:40:07.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:07.022+0000 7fef07200640 1 -- 172.21.15.89:0/1858775790 wait complete. 2024-09-10T08:40:07.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:07 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1858775790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:40:07.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:07 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1858775790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:40:07.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:07 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1858775790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:40:07.896 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T08:36:57.162188+0000', 'modified': '2024-09-10T08:39:38.834417+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24554, 'mds_1': 24518, 'mds_2': 24496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24472': {'gid': 24472, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6835/881263752', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6834', 'nonce': 881263752}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 881263752}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6837/2367754626', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 2367754626}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 2367754626}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6835/285258866', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 285258866}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 285258866}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.149:6839/2020874598', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 2020874598}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 2020874598}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.111:6837/779590543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6836', 'nonce': 779590543}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 779590543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24554': {'gid': 24554, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.111:6839/3640096926', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 3640096926}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 3640096926}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24554, 'qdb_cluster': [24554, 24518, 24496]} 2024-09-10T08:40:07.896 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-10T08:40:07.897 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T08:40:08.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:08 smithi089 ceph-mon[30757]: pgmap v399: 129 pgs: 129 active+clean; 654 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-09-10T08:40:08.310 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:40:08.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:08 smithi149 ceph-mon[36167]: pgmap v399: 129 pgs: 129 active+clean; 654 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-09-10T08:40:08.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:08 smithi111 ceph-mon[35220]: pgmap v399: 129 pgs: 129 active+clean; 654 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-09-10T08:40:08.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.865+0000 7f600ca00640 1 -- 172.21.15.89:0/4161981272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6008101df0 msgr2=0x7f60081021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:08.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.865+0000 7f600ca00640 1 --2- 172.21.15.89:0/4161981272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6008101df0 0x7f60081021d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009900 tx=0x7f5ff802a410 comp rx=0 tx=0).stop 2024-09-10T08:40:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.865+0000 7f600ca00640 1 -- 172.21.15.89:0/4161981272 shutdown_connections 2024-09-10T08:40:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.865+0000 7f600ca00640 1 --2- 172.21.15.89:0/4161981272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6008106f20 0x7f600810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.865+0000 7f600ca00640 1 --2- 172.21.15.89:0/4161981272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60081027a0 0x7f60081067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.865+0000 7f600ca00640 1 --2- 172.21.15.89:0/4161981272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6008101df0 0x7f60081021d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.865+0000 7f600ca00640 1 -- 172.21.15.89:0/4161981272 >> 172.21.15.89:0/4161981272 conn(0x7f60080fd8b0 msgr2=0x7f60080ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.865+0000 7f600ca00640 1 -- 172.21.15.89:0/4161981272 shutdown_connections 2024-09-10T08:40:08.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.865+0000 7f600ca00640 1 -- 172.21.15.89:0/4161981272 wait complete. 2024-09-10T08:40:08.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.866+0000 7f600ca00640 1 Processor -- start 2024-09-10T08:40:08.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.866+0000 7f600ca00640 1 -- start start 2024-09-10T08:40:08.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.867+0000 7f600ca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6008101df0 0x7f600819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:08.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.867+0000 7f600ca00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60081027a0 0x7f600819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:08.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.867+0000 7f600ca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6008106f20 0x7f600819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.867+0000 7f600ca00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6008111f80 con 0x7f6008101df0 2024-09-10T08:40:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.867+0000 7f600ca00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6008111e00 con 0x7f60081027a0 2024-09-10T08:40:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.867+0000 7f600ca00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6008112100 con 0x7f6008106f20 2024-09-10T08:40:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.867+0000 7f6005e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6008101df0 0x7f600819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.867+0000 7f6005400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60081027a0 0x7f600819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.867+0000 7f6005e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6008101df0 0x7f600819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33538/0 (socket says 172.21.15.89:33538) 2024-09-10T08:40:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.867+0000 7f6005e00640 1 -- 172.21.15.89:0/2049583407 learned_addr learned my addr 172.21.15.89:0/2049583407 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.867+0000 7f6005400640 1 --2- 172.21.15.89:0/2049583407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60081027a0 0x7f600819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:60822/0 (socket says 172.21.15.89:60822) 2024-09-10T08:40:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.867+0000 7f6006800640 1 --2- 172.21.15.89:0/2049583407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6008106f20 0x7f600819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.868+0000 7f6005400640 1 -- 172.21.15.89:0/2049583407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6008106f20 msgr2=0x7f600819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.868+0000 7f6005400640 1 --2- 172.21.15.89:0/2049583407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6008106f20 0x7f600819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.868+0000 7f6005400640 1 -- 172.21.15.89:0/2049583407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6008101df0 msgr2=0x7f600819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:08.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.868+0000 7f6005400640 1 --2- 172.21.15.89:0/2049583407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6008101df0 0x7f600819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.868+0000 7f6005400640 1 -- 172.21.15.89:0/2049583407 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600819f5d0 con 0x7f60081027a0 2024-09-10T08:40:08.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.868+0000 7f6005e00640 1 --2- 172.21.15.89:0/2049583407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6008101df0 0x7f600819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:08.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.868+0000 7f6005400640 1 --2- 172.21.15.89:0/2049583407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60081027a0 0x7f600819ab60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400e9f0 tx=0x7f5ff400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:08.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.868+0000 7f5ffea00640 1 -- 172.21.15.89:0/2049583407 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff400ccc0 con 0x7f60081027a0 2024-09-10T08:40:08.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.868+0000 7f600ca00640 1 -- 172.21.15.89:0/2049583407 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f600819f8c0 con 0x7f60081027a0 2024-09-10T08:40:08.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.868+0000 7f5ffea00640 1 -- 172.21.15.89:0/2049583407 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ff4005020 con 0x7f60081027a0 2024-09-10T08:40:08.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.868+0000 7f5ffea00640 1 -- 172.21.15.89:0/2049583407 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff40049e0 con 0x7f60081027a0 2024-09-10T08:40:08.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.868+0000 7f600ca00640 1 -- 172.21.15.89:0/2049583407 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60081a71a0 con 0x7f60081027a0 2024-09-10T08:40:08.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.872+0000 7f600f4cd740 1 Processor -- start 2024-09-10T08:40:08.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.873+0000 7f600f4cd740 1 -- start start 2024-09-10T08:40:08.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.873+0000 7f600f4cd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0330 0x55ad3a9c0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:08.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.873+0000 7f600f4cd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:08.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.873+0000 7f5ffea00640 1 -- 172.21.15.89:0/2049583407 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f5ff4005560 con 0x7f60081027a0 2024-09-10T08:40:08.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.873+0000 7f600f4cd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9cb510 0x55ad3a9cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:08.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.873+0000 7f600f4cd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ad3a9260b0 con 0x55ad3a9c0330 2024-09-10T08:40:08.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.873+0000 7f600f4cd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ad3a8cdab0 con 0x55ad3a9c0c50 2024-09-10T08:40:08.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.873+0000 7f600f4cd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ad3a922070 con 0x55ad3a9cb510 2024-09-10T08:40:08.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.873+0000 7f6006800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0330 0x55ad3a9c0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:08.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.873+0000 7f6006800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0330 0x55ad3a9c0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33554/0 (socket says 172.21.15.89:33554) 2024-09-10T08:40:08.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.873+0000 7f6006800640 1 -- 172.21.15.89:0/797936058 learned_addr learned my addr 172.21.15.89:0/797936058 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:08.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.873+0000 7f6005400640 1 --2- 172.21.15.89:0/797936058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9cb510 0x55ad3a9cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:08.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.873+0000 7f6005e00640 1 --2- 172.21.15.89:0/797936058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:08.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.874+0000 7f6005e00640 1 -- 172.21.15.89:0/797936058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9cb510 msgr2=0x55ad3a9cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:08.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.874+0000 7f6005e00640 1 --2- 172.21.15.89:0/797936058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9cb510 0x55ad3a9cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.874+0000 7f6005e00640 1 -- 172.21.15.89:0/797936058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0330 msgr2=0x55ad3a9c0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:08.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.874+0000 7f6005e00640 1 --2- 172.21.15.89:0/797936058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0330 0x55ad3a9c0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.874+0000 7f6005e00640 1 -- 172.21.15.89:0/797936058 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad3a8da4a0 con 0x55ad3a9c0c50 2024-09-10T08:40:08.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.874+0000 7f6006800640 1 --2- 172.21.15.89:0/797936058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0330 0x55ad3a9c0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:08.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.874+0000 7f6005e00640 1 --2- 172.21.15.89:0/797936058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9cafd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009230 tx=0x7f5ff8005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b27920fd92e80609 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:08.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.875+0000 7f5ff3000640 1 -- 172.21.15.89:0/797936058 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8040070 con 0x55ad3a9c0c50 2024-09-10T08:40:08.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.875+0000 7f5ff3000640 1 -- 172.21.15.89:0/797936058 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ff80319f0 con 0x55ad3a9c0c50 2024-09-10T08:40:08.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.875+0000 7f5ff3000640 1 -- 172.21.15.89:0/797936058 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8035040 con 0x55ad3a9c0c50 2024-09-10T08:40:08.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.875+0000 7f5ffea00640 1 --2- 172.21.15.89:0/2049583407 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5fec0a2c90 0x7f5fec0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:08.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.875+0000 7f5ffea00640 1 -- 172.21.15.89:0/2049583407 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f5ff40cf470 con 0x7f60081027a0 2024-09-10T08:40:08.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.875+0000 7f6006800640 1 --2- 172.21.15.89:0/2049583407 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5fec0a2c90 0x7f5fec0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:08.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.875+0000 7f600f4cd740 1 -- 172.21.15.89:0/797936058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0c50 msgr2=0x55ad3a9cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:08.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.875+0000 7f600f4cd740 1 --2- 172.21.15.89:0/797936058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9cafd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009230 tx=0x7f5ff8005e80 comp rx=0 tx=0).stop 2024-09-10T08:40:08.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.876+0000 7f600f4cd740 1 -- 172.21.15.89:0/797936058 shutdown_connections 2024-09-10T08:40:08.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.876+0000 7f600f4cd740 1 --2- 172.21.15.89:0/797936058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9cb510 0x55ad3a9cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.876+0000 7f600f4cd740 1 --2- 172.21.15.89:0/797936058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9cafd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.876+0000 7f600f4cd740 1 --2- 172.21.15.89:0/797936058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0330 0x55ad3a9c0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.876+0000 7f600f4cd740 1 -- 172.21.15.89:0/797936058 >> 172.21.15.89:0/797936058 conn(0x55ad3a9bc8c0 msgr2=0x55ad3a9bcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:08.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.876+0000 7f600f4cd740 1 -- 172.21.15.89:0/797936058 shutdown_connections 2024-09-10T08:40:08.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.876+0000 7f6006800640 1 --2- 172.21.15.89:0/2049583407 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5fec0a2c90 0x7f5fec0a5150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6000005f40 tx=0x7f60000096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:08.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.876+0000 7f600f4cd740 1 -- 172.21.15.89:0/797936058 wait complete. 2024-09-10T08:40:08.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.876+0000 7f600f4cd740 1 -- start start 2024-09-10T08:40:08.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.876+0000 7f600f4cd740 1 Processor -- start 2024-09-10T08:40:08.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.877+0000 7f600f4cd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0330 0x55ad3a9c8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:08.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.877+0000 7f600f4cd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:08.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.877+0000 7f600f4cd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9cb510 0x55ad3a9c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:08.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.877+0000 7f600f4cd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ad3a9ce060 con 0x55ad3a9c0c50 2024-09-10T08:40:08.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.877+0000 7f600f4cd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ad3a8ae920 con 0x55ad3a9c0330 2024-09-10T08:40:08.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.877+0000 7f600f4cd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ad3a9ce1e0 con 0x55ad3a9cb510 2024-09-10T08:40:08.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.877+0000 7f600f4cd740 10 client.0 _ll_register_callbacks cb 0x55ad3a8d28a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:08.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.877+0000 7f6005400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:08.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.877+0000 7f6005400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33558/0 (socket says 172.21.15.89:33558) 2024-09-10T08:40:08.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.877+0000 7f6005400640 1 -- 172.21.15.89:0/3570176090 learned_addr learned my addr 172.21.15.89:0/3570176090 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:08.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.877+0000 7f6005e00640 1 --2- 172.21.15.89:0/3570176090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0330 0x55ad3a9c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:08.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.877+0000 7f6006800640 1 --2- 172.21.15.89:0/3570176090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9cb510 0x55ad3a9c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:08.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.877+0000 7f6005e00640 1 -- 172.21.15.89:0/3570176090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9cb510 msgr2=0x55ad3a9c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:08.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.878+0000 7f6005e00640 1 --2- 172.21.15.89:0/3570176090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9cb510 0x55ad3a9c90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.878+0000 7f6005e00640 1 -- 172.21.15.89:0/3570176090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0c50 msgr2=0x55ad3a9c8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:08.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.878+0000 7f6005e00640 1 --2- 172.21.15.89:0/3570176090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9c8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.878+0000 7f6005e00640 1 -- 172.21.15.89:0/3570176090 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad3a6972b0 con 0x55ad3a9c0330 2024-09-10T08:40:08.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.878+0000 7f6005400640 1 --2- 172.21.15.89:0/3570176090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:08.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.878+0000 7f6005e00640 1 --2- 172.21.15.89:0/3570176090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0330 0x55ad3a9c8650 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8005ab0 tx=0x7f5ff8009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=113a4154ca45b4e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:08.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.878+0000 7f5fe7400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:40:08.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.878+0000 7f5fe7400640 1 -- 172.21.15.89:0/3570176090 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff804b070 con 0x55ad3a9c0330 2024-09-10T08:40:08.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.878+0000 7f600f4cd740 1 -- 172.21.15.89:0/3570176090 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ff80319f0 con 0x55ad3a9c0330 2024-09-10T08:40:08.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.878+0000 7f5fe7400640 1 -- 172.21.15.89:0/3570176090 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f5ff8004450 con 0x55ad3a9c0330 2024-09-10T08:40:08.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.879+0000 7f5fe7400640 1 -- 172.21.15.89:0/3570176090 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ff8057050 con 0x55ad3a9c0330 2024-09-10T08:40:08.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.879+0000 7f600f4cd740 10 client.25343 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:08.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.879+0000 7f600f4cd740 1 -- 172.21.15.89:0/3570176090 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ad3a9d9e70 con 0x55ad3a9c0330 2024-09-10T08:40:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.879+0000 7f5fe7400640 1 -- 172.21.15.89:0/3570176090 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ff80319f0 con 0x55ad3a9c0330 2024-09-10T08:40:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.879+0000 7f5fe7400640 1 -- 172.21.15.89:0/3570176090 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5ff803c740 con 0x55ad3a9c0330 2024-09-10T08:40:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.879+0000 7f600f4cd740 10 client.25343 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.879+0000 7f600f4cd740 10 client.25343 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T08:40:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.879+0000 7f600f4cd740 20 client.25343 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.879+0000 7f600f4cd740 1 --2- 172.21.15.89:0/3570176090 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55ad3a9dbfa0 0x55ad3a9dd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.879+0000 7f600f4cd740 4 client.25343 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:40:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.879+0000 7f600f4cd740 1 -- 172.21.15.89:0/3570176090 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad3a933620 con 0x55ad3a9dbfa0 2024-09-10T08:40:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.880+0000 7f6005400640 1 --2- 172.21.15.89:0/3570176090 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55ad3a9dbfa0 0x55ad3a9dd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.880+0000 7f6005400640 1 --2- 172.21.15.89:0/3570176090 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55ad3a9dbfa0 0x55ad3a9dd410 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=35106bbdbde218a7 server_cookie=4e3db2ac30450092 in_seq=0 out_seq=0 2024-09-10T08:40:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.880+0000 7f5fe7400640 10 client.25343 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:40:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.881+0000 7f5fe7400640 1 -- 172.21.15.89:0/3570176090 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad3a933620 con 0x55ad3a9dbfa0 2024-09-10T08:40:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.881+0000 7f5fe7400640 10 client.25343 handle_command_reply: tid=0 2024-09-10T08:40:08.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.881+0000 7f600f4cd740 1 client.25343 shutdown 2024-09-10T08:40:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.881+0000 7f600f4cd740 1 -- 172.21.15.89:0/3570176090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0330 msgr2=0x55ad3a9c8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.881+0000 7f600f4cd740 1 --2- 172.21.15.89:0/3570176090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0330 0x55ad3a9c8650 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8005ab0 tx=0x7f5ff8009520 comp rx=0 tx=0).stop 2024-09-10T08:40:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.882+0000 7f600f4cd740 1 -- 172.21.15.89:0/3570176090 shutdown_connections 2024-09-10T08:40:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.882+0000 7f600f4cd740 1 --2- 172.21.15.89:0/3570176090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9cb510 0x55ad3a9c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.882+0000 7f600f4cd740 1 --2- 172.21.15.89:0/3570176090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.882+0000 7f600f4cd740 1 --2- 172.21.15.89:0/3570176090 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55ad3a9dbfa0 0x55ad3a9dd410 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.882+0000 7f600f4cd740 1 --2- 172.21.15.89:0/3570176090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0330 0x55ad3a9c8650 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.882+0000 7f600f4cd740 1 -- 172.21.15.89:0/3570176090 >> 172.21.15.89:0/3570176090 conn(0x55ad3a9bc8c0 msgr2=0x55ad3a9c25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.882+0000 7f5fe7400640 0 client.25343 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:40:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.882+0000 7f600f4cd740 1 -- 172.21.15.89:0/3570176090 shutdown_connections 2024-09-10T08:40:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.882+0000 7f600f4cd740 1 -- 172.21.15.89:0/3570176090 wait complete. 2024-09-10T08:40:08.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.882+0000 7f600f4cd740 20 client.25343 trim_cache size 0 max 16384 2024-09-10T08:40:08.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.900+0000 7f600f4cd740 1 Processor -- start 2024-09-10T08:40:08.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.900+0000 7f600f4cd740 1 -- start start 2024-09-10T08:40:08.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.900+0000 7f600f4cd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0330 0x55ad3a9dcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:08.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.900+0000 7f600f4cd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:08.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.900+0000 7f600f4cd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9cb510 0x55ad3a9c5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:08.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.900+0000 7f600f4cd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ad3a9db750 con 0x55ad3a9cb510 2024-09-10T08:40:08.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.900+0000 7f600f4cd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ad3a9dbe20 con 0x55ad3a9c0330 2024-09-10T08:40:08.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.900+0000 7f600f4cd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ad3a9dc4f0 con 0x55ad3a9c0c50 2024-09-10T08:40:08.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.901+0000 7f6006800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9cb510 0x55ad3a9c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:08.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.901+0000 7f6006800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9cb510 0x55ad3a9c5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33574/0 (socket says 172.21.15.89:33574) 2024-09-10T08:40:08.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.901+0000 7f6006800640 1 -- 172.21.15.89:0/1476972237 learned_addr learned my addr 172.21.15.89:0/1476972237 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:08.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.901+0000 7f6005400640 1 --2- 172.21.15.89:0/1476972237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:08.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.901+0000 7f6005e00640 1 --2- 172.21.15.89:0/1476972237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0330 0x55ad3a9dcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:08.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.901+0000 7f6006800640 1 -- 172.21.15.89:0/1476972237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9c0c50 msgr2=0x55ad3a9c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:08.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.901+0000 7f6006800640 1 --2- 172.21.15.89:0/1476972237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.901+0000 7f6006800640 1 -- 172.21.15.89:0/1476972237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0330 msgr2=0x55ad3a9dcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:08.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.901+0000 7f6006800640 1 --2- 172.21.15.89:0/1476972237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0330 0x55ad3a9dcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.901+0000 7f6006800640 1 -- 172.21.15.89:0/1476972237 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad3a9cafc0 con 0x55ad3a9cb510 2024-09-10T08:40:08.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.901+0000 7f6005400640 1 --2- 172.21.15.89:0/1476972237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:08.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.901+0000 7f6005e00640 1 --2- 172.21.15.89:0/1476972237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0330 0x55ad3a9dcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:08.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.902+0000 7f6006800640 1 --2- 172.21.15.89:0/1476972237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9cb510 0x55ad3a9c5930 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6000011fb0 tx=0x7f6000019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6298bba4a854965 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:08.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.902+0000 7f5ff2600640 1 -- 172.21.15.89:0/1476972237 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600001e9a0 con 0x55ad3a9cb510 2024-09-10T08:40:08.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.902+0000 7f5ff2600640 1 -- 172.21.15.89:0/1476972237 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f600001eb40 con 0x55ad3a9cb510 2024-09-10T08:40:08.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.902+0000 7f5ff2600640 1 -- 172.21.15.89:0/1476972237 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600001c650 con 0x55ad3a9cb510 2024-09-10T08:40:08.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.902+0000 7f600f4cd740 1 -- 172.21.15.89:0/1476972237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9cb510 msgr2=0x55ad3a9c5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:08.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.902+0000 7f600f4cd740 1 --2- 172.21.15.89:0/1476972237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9cb510 0x55ad3a9c5930 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6000011fb0 tx=0x7f6000019000 comp rx=0 tx=0).stop 2024-09-10T08:40:08.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.903+0000 7f600f4cd740 1 -- 172.21.15.89:0/1476972237 shutdown_connections 2024-09-10T08:40:08.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.903+0000 7f600f4cd740 1 --2- 172.21.15.89:0/1476972237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9cb510 0x55ad3a9c5930 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.903+0000 7f600f4cd740 1 --2- 172.21.15.89:0/1476972237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.903+0000 7f600f4cd740 1 --2- 172.21.15.89:0/1476972237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0330 0x55ad3a9dcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.903+0000 7f600f4cd740 1 -- 172.21.15.89:0/1476972237 >> 172.21.15.89:0/1476972237 conn(0x55ad3a9bc8c0 msgr2=0x55ad3a9c2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:08.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.903+0000 7f600f4cd740 1 -- 172.21.15.89:0/1476972237 shutdown_connections 2024-09-10T08:40:08.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.903+0000 7f600f4cd740 1 -- 172.21.15.89:0/1476972237 wait complete. 2024-09-10T08:40:08.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.903+0000 7f600f4cd740 1 -- start start 2024-09-10T08:40:08.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.903+0000 7f600f4cd740 1 Processor -- start 2024-09-10T08:40:08.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.904+0000 7f600f4cd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0330 0x55ad3a9c5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:08.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.904+0000 7f600f4cd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:08.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.904+0000 7f600f4cd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9cb510 0x55ad3a9c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:08.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.904+0000 7f600f4cd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ad3a9dd290 con 0x55ad3a9c0330 2024-09-10T08:40:08.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.904+0000 7f600f4cd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ad3a9dcbc0 con 0x55ad3a9c0c50 2024-09-10T08:40:08.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.904+0000 7f600f4cd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ad3a9dd960 con 0x55ad3a9cb510 2024-09-10T08:40:08.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.904+0000 7f600f4cd740 10 client.0 _ll_register_callbacks cb 0x7f5ff8005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:08.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.904+0000 7f6005e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0330 0x55ad3a9c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:08.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.904+0000 7f6005e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0330 0x55ad3a9c5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33580/0 (socket says 172.21.15.89:33580) 2024-09-10T08:40:08.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.904+0000 7f6005e00640 1 -- 172.21.15.89:0/453239478 learned_addr learned my addr 172.21.15.89:0/453239478 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:08.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.904+0000 7f6005400640 1 --2- 172.21.15.89:0/453239478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:08.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.904+0000 7f6006800640 1 --2- 172.21.15.89:0/453239478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9cb510 0x55ad3a9c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:08.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.904+0000 7f6005e00640 1 -- 172.21.15.89:0/453239478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9cb510 msgr2=0x55ad3a9c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:08.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.904+0000 7f6005e00640 1 --2- 172.21.15.89:0/453239478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9cb510 0x55ad3a9c60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.904+0000 7f6005e00640 1 -- 172.21.15.89:0/453239478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0c50 msgr2=0x55ad3a9c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:08.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.904+0000 7f6005e00640 1 --2- 172.21.15.89:0/453239478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9c5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.904+0000 7f6005e00640 1 -- 172.21.15.89:0/453239478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad3a9e3ba0 con 0x55ad3a9c0330 2024-09-10T08:40:08.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.904+0000 7f6005400640 1 --2- 172.21.15.89:0/453239478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:08.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.905+0000 7f6005e00640 1 --2- 172.21.15.89:0/453239478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0330 0x55ad3a9c5640 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009800 tx=0x7f5ff8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1a2ab48d6dc67c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:08.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.905+0000 7f5fe7400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:40:08.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.905+0000 7f5fe7400640 1 -- 172.21.15.89:0/453239478 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff804b040 con 0x55ad3a9c0330 2024-09-10T08:40:08.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.905+0000 7f5fe7400640 1 -- 172.21.15.89:0/453239478 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f5ff803c440 con 0x55ad3a9c0330 2024-09-10T08:40:08.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.905+0000 7f600f4cd740 1 -- 172.21.15.89:0/453239478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f600001eb40 con 0x55ad3a9c0330 2024-09-10T08:40:08.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.905+0000 7f5fe7400640 1 -- 172.21.15.89:0/453239478 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ff8035070 con 0x55ad3a9c0330 2024-09-10T08:40:08.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.905+0000 7f600f4cd740 10 client.15300 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:08.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.905+0000 7f600f4cd740 1 -- 172.21.15.89:0/453239478 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ad3aa468e0 con 0x55ad3a9c0330 2024-09-10T08:40:08.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.905+0000 7f5fe7400640 1 -- 172.21.15.89:0/453239478 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f600001eb40 con 0x55ad3a9c0330 2024-09-10T08:40:08.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.906+0000 7f5fe7400640 1 -- 172.21.15.89:0/453239478 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5ff8044450 con 0x55ad3a9c0330 2024-09-10T08:40:08.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.906+0000 7f600f4cd740 10 client.15300 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:08.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.906+0000 7f600f4cd740 10 client.15300 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T08:40:08.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.906+0000 7f600f4cd740 20 client.15300 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:08.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.906+0000 7f600f4cd740 1 --2- 172.21.15.89:0/453239478 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55ad3aa4b240 0x55ad3aa49f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:08.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.906+0000 7f600f4cd740 4 client.15300 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:40:08.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.906+0000 7f600f4cd740 1 -- 172.21.15.89:0/453239478 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "status"}) -- 0x55ad3a92db30 con 0x55ad3aa4b240 2024-09-10T08:40:08.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.906+0000 7f6005400640 1 --2- 172.21.15.89:0/453239478 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55ad3aa4b240 0x55ad3aa49f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:08.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.907+0000 7f6005400640 1 --2- 172.21.15.89:0/453239478 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55ad3aa4b240 0x55ad3aa49f50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6df4c15907b227c2 server_cookie=fbd9e9564f688afa in_seq=0 out_seq=0 2024-09-10T08:40:08.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.907+0000 7f5fe7400640 10 client.15300 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:40:08.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.907+0000 7f5fe7400640 1 -- 172.21.15.89:0/453239478 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ad3a92db30 con 0x55ad3aa4b240 2024-09-10T08:40:08.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.907+0000 7f5fe7400640 10 client.15300 handle_command_reply: tid=0 2024-09-10T08:40:08.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.907+0000 7f600f4cd740 1 client.15300 shutdown 2024-09-10T08:40:08.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.907+0000 7f600f4cd740 1 -- 172.21.15.89:0/453239478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0330 msgr2=0x55ad3a9c5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:08.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.907+0000 7f600f4cd740 1 --2- 172.21.15.89:0/453239478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0330 0x55ad3a9c5640 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009800 tx=0x7f5ff8005e80 comp rx=0 tx=0).stop 2024-09-10T08:40:08.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.907+0000 7f600f4cd740 1 -- 172.21.15.89:0/453239478 shutdown_connections 2024-09-10T08:40:08.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.907+0000 7f600f4cd740 1 --2- 172.21.15.89:0/453239478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad3a9cb510 0x55ad3a9c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.907+0000 7f600f4cd740 1 --2- 172.21.15.89:0/453239478 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55ad3aa4b240 0x55ad3aa49f50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.907+0000 7f600f4cd740 1 --2- 172.21.15.89:0/453239478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ad3a9c0c50 0x55ad3a9c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.907+0000 7f600f4cd740 1 --2- 172.21.15.89:0/453239478 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ad3a9c0330 0x55ad3a9c5640 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.907+0000 7f600f4cd740 1 -- 172.21.15.89:0/453239478 >> 172.21.15.89:0/453239478 conn(0x55ad3a9bc8c0 msgr2=0x55ad3a9d61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:08.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.907+0000 7f5fe7400640 0 client.15300 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:40:08.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.907+0000 7f600f4cd740 1 -- 172.21.15.89:0/453239478 shutdown_connections 2024-09-10T08:40:08.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.907+0000 7f600f4cd740 1 -- 172.21.15.89:0/453239478 wait complete. 2024-09-10T08:40:08.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.907+0000 7f600f4cd740 20 client.15300 trim_cache size 0 max 16384 2024-09-10T08:40:08.914 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:40:08.914 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "91f747bc-6f4e-11ef-bcea-c7b262605968", 2024-09-10T08:40:08.914 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 2, 2024-09-10T08:40:08.914 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24496, 2024-09-10T08:40:08.914 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2024-09-10T08:40:08.914 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2024-09-10T08:40:08.914 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2024-09-10T08:40:08.914 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 141.20301365099999, 2024-09-10T08:40:08.915 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 26, 2024-09-10T08:40:08.915 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 96, 2024-09-10T08:40:08.915 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 96, 2024-09-10T08:40:08.915 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 213.44109406300001 2024-09-10T08:40:08.915 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:40:08.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.908+0000 7f600ca00640 1 -- 172.21.15.89:0/2049583407 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5fec0a2c90 msgr2=0x7f5fec0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:08.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.908+0000 7f600ca00640 1 --2- 172.21.15.89:0/2049583407 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5fec0a2c90 0x7f5fec0a5150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6000005f40 tx=0x7f60000096a0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.908+0000 7f600ca00640 1 -- 172.21.15.89:0/2049583407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60081027a0 msgr2=0x7f600819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:08.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.908+0000 7f600ca00640 1 --2- 172.21.15.89:0/2049583407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60081027a0 0x7f600819ab60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400e9f0 tx=0x7f5ff400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.908+0000 7f600ca00640 1 -- 172.21.15.89:0/2049583407 shutdown_connections 2024-09-10T08:40:08.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.908+0000 7f600ca00640 1 --2- 172.21.15.89:0/2049583407 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5fec0a2c90 0x7f5fec0a5150 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.908+0000 7f600ca00640 1 --2- 172.21.15.89:0/2049583407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6008106f20 0x7f600819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.908+0000 7f600ca00640 1 --2- 172.21.15.89:0/2049583407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60081027a0 0x7f600819ab60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.908+0000 7f600ca00640 1 --2- 172.21.15.89:0/2049583407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6008101df0 0x7f600819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:08.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.908+0000 7f600ca00640 1 -- 172.21.15.89:0/2049583407 >> 172.21.15.89:0/2049583407 conn(0x7f60080fd8b0 msgr2=0x7f60080febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:08.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.908+0000 7f600ca00640 1 -- 172.21.15.89:0/2049583407 shutdown_connections 2024-09-10T08:40:08.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:08.908+0000 7f600ca00640 1 -- 172.21.15.89:0/2049583407 wait complete. 2024-09-10T08:40:09.780 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T08:40:10.149 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:40:10.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.700+0000 7f9ac2e00640 1 -- 172.21.15.89:0/3560630464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9abc112a30 msgr2=0x7f9abc114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:10.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.700+0000 7f9ac2e00640 1 --2- 172.21.15.89:0/3560630464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9abc112a30 0x7f9abc114e20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4009be0 tx=0x7f9ab402a6c0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.701+0000 7f9ac2e00640 1 -- 172.21.15.89:0/3560630464 shutdown_connections 2024-09-10T08:40:10.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.701+0000 7f9ac2e00640 1 --2- 172.21.15.89:0/3560630464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9abc112a30 0x7f9abc114e20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.701+0000 7f9ac2e00640 1 --2- 172.21.15.89:0/3560630464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9abc077040 0x7f9abc1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.701+0000 7f9ac2e00640 1 --2- 172.21.15.89:0/3560630464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9abc076720 0x7f9abc076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.701+0000 7f9ac2e00640 1 -- 172.21.15.89:0/3560630464 >> 172.21.15.89:0/3560630464 conn(0x7f9abc0ffa90 msgr2=0x7f9abc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:10.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.701+0000 7f9ac2e00640 1 -- 172.21.15.89:0/3560630464 shutdown_connections 2024-09-10T08:40:10.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.701+0000 7f9ac2e00640 1 -- 172.21.15.89:0/3560630464 wait complete. 2024-09-10T08:40:10.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.702+0000 7f9ac2e00640 1 Processor -- start 2024-09-10T08:40:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.702+0000 7f9ac2e00640 1 -- start start 2024-09-10T08:40:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.702+0000 7f9ac2e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9abc076720 0x7f9abc1a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.702+0000 7f9ac2e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9abc077040 0x7f9abc1a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.702+0000 7f9ac2e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9abc112a30 0x7f9abc19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.702+0000 7f9ac2e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc118570 con 0x7f9abc076720 2024-09-10T08:40:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.702+0000 7f9ac2e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1183f0 con 0x7f9abc077040 2024-09-10T08:40:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.702+0000 7f9ac2e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1186f0 con 0x7f9abc112a30 2024-09-10T08:40:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.702+0000 7f9abbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9abc076720 0x7f9abc1a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.702+0000 7f9abbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9abc076720 0x7f9abc1a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33610/0 (socket says 172.21.15.89:33610) 2024-09-10T08:40:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.702+0000 7f9abbe00640 1 -- 172.21.15.89:0/1982831969 learned_addr learned my addr 172.21.15.89:0/1982831969 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.702+0000 7f9ac0e00640 1 --2- 172.21.15.89:0/1982831969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9abc112a30 0x7f9abc19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.703+0000 7f9abb400640 1 --2- 172.21.15.89:0/1982831969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9abc077040 0x7f9abc1a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:10.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.703+0000 7f9abbe00640 1 -- 172.21.15.89:0/1982831969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9abc112a30 msgr2=0x7f9abc19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.703+0000 7f9abbe00640 1 --2- 172.21.15.89:0/1982831969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9abc112a30 0x7f9abc19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.703+0000 7f9abbe00640 1 -- 172.21.15.89:0/1982831969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9abc077040 msgr2=0x7f9abc1a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.703+0000 7f9abbe00640 1 --2- 172.21.15.89:0/1982831969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9abc077040 0x7f9abc1a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.703+0000 7f9abbe00640 1 -- 172.21.15.89:0/1982831969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9abc19ef90 con 0x7f9abc076720 2024-09-10T08:40:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.703+0000 7f9abbe00640 1 --2- 172.21.15.89:0/1982831969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9abc076720 0x7f9abc1a46a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00d950 tx=0x7f9aac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.703+0000 7f9ab8c00640 1 -- 172.21.15.89:0/1982831969 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac017030 con 0x7f9abc076720 2024-09-10T08:40:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.703+0000 7f9ac2e00640 1 -- 172.21.15.89:0/1982831969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9abc19f280 con 0x7f9abc076720 2024-09-10T08:40:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.703+0000 7f9ab8c00640 1 -- 172.21.15.89:0/1982831969 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aac004020 con 0x7f9abc076720 2024-09-10T08:40:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.703+0000 7f9ab8c00640 1 -- 172.21.15.89:0/1982831969 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac0047a0 con 0x7f9abc076720 2024-09-10T08:40:10.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.704+0000 7f9ac2e00640 1 -- 172.21.15.89:0/1982831969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9abc1ab570 con 0x7f9abc076720 2024-09-10T08:40:10.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.706+0000 7f9ab8c00640 1 -- 172.21.15.89:0/1982831969 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f9aac0041d0 con 0x7f9abc076720 2024-09-10T08:40:10.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.707+0000 7f9ac584d740 1 Processor -- start 2024-09-10T08:40:10.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.707+0000 7f9ac584d740 1 -- start start 2024-09-10T08:40:10.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.707+0000 7f9ac584d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996b3330 0x5643996b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:10.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.707+0000 7f9ac584d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996b3c50 0x5643996bdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:10.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.707+0000 7f9ac584d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996be510 0x5643996c0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:10.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.707+0000 7f9ac584d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5643996190b0 con 0x5643996b3c50 2024-09-10T08:40:10.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.707+0000 7f9ac584d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5643995c0ab0 con 0x5643996b3330 2024-09-10T08:40:10.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.707+0000 7f9ac584d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564399615070 con 0x5643996be510 2024-09-10T08:40:10.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.708+0000 7f9abbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996b3c50 0x5643996bdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:10.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.708+0000 7f9abb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996be510 0x5643996c0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:10.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.708+0000 7f9ac0e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996b3330 0x5643996b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:10.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.708+0000 7f9abb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996be510 0x5643996c0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44838/0 (socket says 172.21.15.89:44838) 2024-09-10T08:40:10.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.708+0000 7f9abb400640 1 -- 172.21.15.89:0/1980572229 learned_addr learned my addr 172.21.15.89:0/1980572229 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:10.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.708+0000 7f9abbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996b3c50 0x5643996bdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33624/0 (socket says 172.21.15.89:33624) 2024-09-10T08:40:10.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.708+0000 7f9ac0e00640 1 -- 172.21.15.89:0/1980572229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996be510 msgr2=0x5643996c0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:10.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.708+0000 7f9ac0e00640 1 --2- 172.21.15.89:0/1980572229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996be510 0x5643996c0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.708+0000 7f9ac0e00640 1 -- 172.21.15.89:0/1980572229 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996b3c50 msgr2=0x5643996bdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:10.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.708+0000 7f9ac0e00640 1 --2- 172.21.15.89:0/1980572229 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996b3c50 0x5643996bdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.708+0000 7f9ac0e00640 1 -- 172.21.15.89:0/1980572229 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643995cd4a0 con 0x5643996b3330 2024-09-10T08:40:10.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.708+0000 7f9ab8c00640 1 --2- 172.21.15.89:0/1982831969 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9aa00a2c10 0x7f9aa00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:10.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.709+0000 7f9ac0e00640 1 --2- 172.21.15.89:0/1980572229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996b3330 0x5643996b3710 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ab402acb0 tx=0x7f9ab4005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adf8a40005ce27a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:10.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.709+0000 7f9ab8c00640 1 -- 172.21.15.89:0/1982831969 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f9aac0cef30 con 0x7f9abc076720 2024-09-10T08:40:10.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.709+0000 7f9abbe00640 1 --2- 172.21.15.89:0/1982831969 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9aa00a2c10 0x7f9aa00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:10.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.709+0000 7f9ab1600640 1 -- 172.21.15.89:0/1980572229 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4041030 con 0x5643996b3330 2024-09-10T08:40:10.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.709+0000 7f9ab1600640 1 -- 172.21.15.89:0/1980572229 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab4030bf0 con 0x5643996b3330 2024-09-10T08:40:10.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.709+0000 7f9ab1600640 1 -- 172.21.15.89:0/1980572229 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4036040 con 0x5643996b3330 2024-09-10T08:40:10.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.710+0000 7f9ac584d740 1 -- 172.21.15.89:0/1980572229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996b3330 msgr2=0x5643996b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:10.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.710+0000 7f9ac584d740 1 --2- 172.21.15.89:0/1980572229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996b3330 0x5643996b3710 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ab402acb0 tx=0x7f9ab4005ec0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.710+0000 7f9abbe00640 1 --2- 172.21.15.89:0/1982831969 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9aa00a2c10 0x7f9aa00a50d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9aac09af00 tx=0x7f9aac0d29e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.710+0000 7f9ac584d740 1 -- 172.21.15.89:0/1980572229 shutdown_connections 2024-09-10T08:40:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.710+0000 7f9ac584d740 1 --2- 172.21.15.89:0/1980572229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996be510 0x5643996c0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.710+0000 7f9ac584d740 1 --2- 172.21.15.89:0/1980572229 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996b3c50 0x5643996bdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.710+0000 7f9ac584d740 1 --2- 172.21.15.89:0/1980572229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996b3330 0x5643996b3710 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.710+0000 7f9ac584d740 1 -- 172.21.15.89:0/1980572229 >> 172.21.15.89:0/1980572229 conn(0x5643996af8c0 msgr2=0x5643996afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.710+0000 7f9ac584d740 1 -- 172.21.15.89:0/1980572229 shutdown_connections 2024-09-10T08:40:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.710+0000 7f9ac584d740 1 -- 172.21.15.89:0/1980572229 wait complete. 2024-09-10T08:40:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.711+0000 7f9ac584d740 1 -- start start 2024-09-10T08:40:10.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.711+0000 7f9ac584d740 1 Processor -- start 2024-09-10T08:40:10.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.711+0000 7f9ac584d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996b3330 0x5643996bb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:10.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.711+0000 7f9ac584d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996b3c50 0x5643996bbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:10.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.711+0000 7f9ac584d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996be510 0x5643996bc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:10.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.711+0000 7f9ac584d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5643996c1060 con 0x5643996be510 2024-09-10T08:40:10.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.711+0000 7f9ac584d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5643995a1920 con 0x5643996b3330 2024-09-10T08:40:10.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.711+0000 7f9ac584d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5643996c11e0 con 0x5643996b3c50 2024-09-10T08:40:10.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.711+0000 7f9ac584d740 10 client.0 _ll_register_callbacks cb 0x5643995c58a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.712+0000 7f9ac0e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996be510 0x5643996bc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.712+0000 7f9ac0e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996be510 0x5643996bc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33628/0 (socket says 172.21.15.89:33628) 2024-09-10T08:40:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.712+0000 7f9ac0e00640 1 -- 172.21.15.89:0/3630209866 learned_addr learned my addr 172.21.15.89:0/3630209866 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.712+0000 7f9abb400640 1 --2- 172.21.15.89:0/3630209866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996b3c50 0x5643996bbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.712+0000 7f9ac0e00640 1 --2- 172.21.15.89:0/3630209866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996b3330 0x5643996bb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.712+0000 7f9ac0e00640 1 -- 172.21.15.89:0/3630209866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996b3c50 msgr2=0x5643996bbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.712+0000 7f9ac0e00640 1 --2- 172.21.15.89:0/3630209866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996b3c50 0x5643996bbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.712+0000 7f9ac0e00640 1 -- 172.21.15.89:0/3630209866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996b3330 msgr2=0x5643996bb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.712+0000 7f9ac0e00640 1 --2- 172.21.15.89:0/3630209866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996b3330 0x5643996bb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.712+0000 7f9ac0e00640 1 -- 172.21.15.89:0/3630209866 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56439938a2b0 con 0x5643996be510 2024-09-10T08:40:10.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.712+0000 7f9abb400640 1 --2- 172.21.15.89:0/3630209866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996b3c50 0x5643996bbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:10.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.712+0000 7f9ac0e00640 1 --2- 172.21.15.89:0/3630209866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996be510 0x5643996bc050 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40099a0 tx=0x7f9ab4003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac293d7dde65fe35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:10.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.712+0000 7f9aa6000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:40:10.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.713+0000 7f9aa6000640 1 -- 172.21.15.89:0/3630209866 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4036050 con 0x5643996be510 2024-09-10T08:40:10.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.713+0000 7f9aa6000640 1 -- 172.21.15.89:0/3630209866 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f9ab40303a0 con 0x5643996be510 2024-09-10T08:40:10.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.713+0000 7f9ac584d740 1 -- 172.21.15.89:0/3630209866 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ab4030bf0 con 0x5643996be510 2024-09-10T08:40:10.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.713+0000 7f9aa6000640 1 -- 172.21.15.89:0/3630209866 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9ab405a050 con 0x5643996be510 2024-09-10T08:40:10.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.713+0000 7f9ac584d740 10 client.15312 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:10.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.713+0000 7f9ac584d740 1 -- 172.21.15.89:0/3630209866 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5643996cce60 con 0x5643996be510 2024-09-10T08:40:10.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.713+0000 7f9aa6000640 1 -- 172.21.15.89:0/3630209866 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab4030bf0 con 0x5643996be510 2024-09-10T08:40:10.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.714+0000 7f9aa6000640 1 -- 172.21.15.89:0/3630209866 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9ab4004600 con 0x5643996be510 2024-09-10T08:40:10.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.714+0000 7f9ac584d740 10 client.15312 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:10.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.714+0000 7f9ac584d740 10 client.15312 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T08:40:10.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.714+0000 7f9ac584d740 20 client.15312 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:10.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.714+0000 7f9ac584d740 1 --2- 172.21.15.89:0/3630209866 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x5643996cf010 0x5643996d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:10.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.714+0000 7f9ac584d740 4 client.15312 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:40:10.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.714+0000 7f9ac584d740 1 -- 172.21.15.89:0/3630209866 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564399626620 con 0x5643996cf010 2024-09-10T08:40:10.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.714+0000 7f9abb400640 1 --2- 172.21.15.89:0/3630209866 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x5643996cf010 0x5643996d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:10.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.715+0000 7f9abb400640 1 --2- 172.21.15.89:0/3630209866 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x5643996cf010 0x5643996d0480 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a264462dc0e8456a server_cookie=7af879e63235f683 in_seq=0 out_seq=0 2024-09-10T08:40:10.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.715+0000 7f9aa6000640 10 client.15312 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:40:10.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.715+0000 7f9aa6000640 1 -- 172.21.15.89:0/3630209866 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564399626620 con 0x5643996cf010 2024-09-10T08:40:10.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.715+0000 7f9aa6000640 10 client.15312 handle_command_reply: tid=0 2024-09-10T08:40:10.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.715+0000 7f9ac584d740 1 client.15312 shutdown 2024-09-10T08:40:10.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.716+0000 7f9ac584d740 1 -- 172.21.15.89:0/3630209866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996be510 msgr2=0x5643996bc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:10.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.716+0000 7f9ac584d740 1 --2- 172.21.15.89:0/3630209866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996be510 0x5643996bc050 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40099a0 tx=0x7f9ab4003d60 comp rx=0 tx=0).stop 2024-09-10T08:40:10.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.716+0000 7f9ac584d740 1 -- 172.21.15.89:0/3630209866 shutdown_connections 2024-09-10T08:40:10.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.716+0000 7f9ac584d740 1 --2- 172.21.15.89:0/3630209866 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x5643996cf010 0x5643996d0480 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.716+0000 7f9ac584d740 1 --2- 172.21.15.89:0/3630209866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996be510 0x5643996bc050 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.716+0000 7f9ac584d740 1 --2- 172.21.15.89:0/3630209866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996b3c50 0x5643996bbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.716+0000 7f9ac584d740 1 --2- 172.21.15.89:0/3630209866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996b3330 0x5643996bb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.716+0000 7f9ac584d740 1 -- 172.21.15.89:0/3630209866 >> 172.21.15.89:0/3630209866 conn(0x5643996af8c0 msgr2=0x5643996b55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:10.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.716+0000 7f9aa6000640 0 client.15312 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:40:10.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.716+0000 7f9ac584d740 1 -- 172.21.15.89:0/3630209866 shutdown_connections 2024-09-10T08:40:10.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.716+0000 7f9ac584d740 1 -- 172.21.15.89:0/3630209866 wait complete. 2024-09-10T08:40:10.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.716+0000 7f9ac584d740 20 client.15312 trim_cache size 0 max 16384 2024-09-10T08:40:10.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.734+0000 7f9ac584d740 1 Processor -- start 2024-09-10T08:40:10.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.735+0000 7f9ac584d740 1 -- start start 2024-09-10T08:40:10.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.735+0000 7f9ac584d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996b3330 0x5643996af210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:10.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.735+0000 7f9ac584d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996b3c50 0x5643996bbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:10.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.735+0000 7f9ac584d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996be510 0x5643996b8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:10.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.735+0000 7f9ac584d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5643996ce7c0 con 0x5643996b3c50 2024-09-10T08:40:10.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.735+0000 7f9ac584d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5643996cee90 con 0x5643996b3330 2024-09-10T08:40:10.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.735+0000 7f9ac584d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5643996cf560 con 0x5643996be510 2024-09-10T08:40:10.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.735+0000 7f9abb400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996b3c50 0x5643996bbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:10.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.735+0000 7f9abb400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996b3c50 0x5643996bbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33630/0 (socket says 172.21.15.89:33630) 2024-09-10T08:40:10.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.735+0000 7f9abb400640 1 -- 172.21.15.89:0/3032455461 learned_addr learned my addr 172.21.15.89:0/3032455461 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:10.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.735+0000 7f9ac0e00640 1 --2- 172.21.15.89:0/3032455461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996b3330 0x5643996af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:10.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.735+0000 7f9ac0e00640 1 --2- 172.21.15.89:0/3032455461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996be510 0x5643996b8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:10.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.736+0000 7f9abb400640 1 -- 172.21.15.89:0/3032455461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996be510 msgr2=0x5643996b8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:10.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.736+0000 7f9abb400640 1 --2- 172.21.15.89:0/3032455461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996be510 0x5643996b8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.736+0000 7f9abb400640 1 -- 172.21.15.89:0/3032455461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996b3330 msgr2=0x5643996af210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:10.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.736+0000 7f9abb400640 1 --2- 172.21.15.89:0/3032455461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996b3330 0x5643996af210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.736+0000 7f9abb400640 1 -- 172.21.15.89:0/3032455461 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643996bdf40 con 0x5643996b3c50 2024-09-10T08:40:10.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.736+0000 7f9ac0e00640 1 --2- 172.21.15.89:0/3032455461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996be510 0x5643996b8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:40:10.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.736+0000 7f9abb400640 1 --2- 172.21.15.89:0/3032455461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996b3c50 0x5643996bbea0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8000c00 tx=0x7f9aa8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d49f4792f96140 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:10.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.736+0000 7f9ab0c00640 1 -- 172.21.15.89:0/3032455461 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8004220 con 0x5643996b3c50 2024-09-10T08:40:10.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.736+0000 7f9ab0c00640 1 -- 172.21.15.89:0/3032455461 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aa80043c0 con 0x5643996b3c50 2024-09-10T08:40:10.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.736+0000 7f9ab0c00640 1 -- 172.21.15.89:0/3032455461 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8010a20 con 0x5643996b3c50 2024-09-10T08:40:10.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.737+0000 7f9ac584d740 1 -- 172.21.15.89:0/3032455461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996b3c50 msgr2=0x5643996bbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:10.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.737+0000 7f9ac584d740 1 --2- 172.21.15.89:0/3032455461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996b3c50 0x5643996bbea0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8000c00 tx=0x7f9aa8009260 comp rx=0 tx=0).stop 2024-09-10T08:40:10.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.737+0000 7f9ac584d740 1 -- 172.21.15.89:0/3032455461 shutdown_connections 2024-09-10T08:40:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.737+0000 7f9ac584d740 1 --2- 172.21.15.89:0/3032455461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996be510 0x5643996b8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.737+0000 7f9ac584d740 1 --2- 172.21.15.89:0/3032455461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996b3c50 0x5643996bbea0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.737+0000 7f9ac584d740 1 --2- 172.21.15.89:0/3032455461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996b3330 0x5643996af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.737+0000 7f9ac584d740 1 -- 172.21.15.89:0/3032455461 >> 172.21.15.89:0/3032455461 conn(0x5643996af8c0 msgr2=0x5643996b9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.737+0000 7f9ac584d740 1 -- 172.21.15.89:0/3032455461 shutdown_connections 2024-09-10T08:40:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.737+0000 7f9ac584d740 1 -- 172.21.15.89:0/3032455461 wait complete. 2024-09-10T08:40:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.737+0000 7f9ac584d740 1 -- start start 2024-09-10T08:40:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.738+0000 7f9ac584d740 1 Processor -- start 2024-09-10T08:40:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.738+0000 7f9ac584d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996b3330 0x5643996d8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.738+0000 7f9ac584d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996b3c50 0x5643996d8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.738+0000 7f9ac584d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996be510 0x5643996b8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:10.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.738+0000 7f9ac584d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5643996d0300 con 0x5643996b3330 2024-09-10T08:40:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.738+0000 7f9ac584d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5643996cfc30 con 0x5643996be510 2024-09-10T08:40:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.738+0000 7f9ac584d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5643996d09d0 con 0x5643996b3c50 2024-09-10T08:40:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.738+0000 7f9ac584d740 10 client.0 _ll_register_callbacks cb 0x7f9ab4003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.738+0000 7f9ac0e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996b3330 0x5643996d8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.738+0000 7f9abb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996b3c50 0x5643996d8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.738+0000 7f9ac0e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996b3330 0x5643996d8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33642/0 (socket says 172.21.15.89:33642) 2024-09-10T08:40:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.738+0000 7f9abb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996b3c50 0x5643996d8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44860/0 (socket says 172.21.15.89:44860) 2024-09-10T08:40:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.738+0000 7f9ac0e00640 1 -- 172.21.15.89:0/1253548815 learned_addr learned my addr 172.21.15.89:0/1253548815 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.739+0000 7f9ac0e00640 1 --2- 172.21.15.89:0/1253548815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996be510 0x5643996b8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.739+0000 7f9ac0e00640 1 -- 172.21.15.89:0/1253548815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996b3c50 msgr2=0x5643996d8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.739+0000 7f9ac0e00640 1 --2- 172.21.15.89:0/1253548815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996b3c50 0x5643996d8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.739+0000 7f9ac0e00640 1 -- 172.21.15.89:0/1253548815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996be510 msgr2=0x5643996b8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.739+0000 7f9ac0e00640 1 --2- 172.21.15.89:0/1253548815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996be510 0x5643996b8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.739+0000 7f9ac0e00640 1 -- 172.21.15.89:0/1253548815 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643996b88b0 con 0x5643996b3330 2024-09-10T08:40:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.739+0000 7f9ac0e00640 1 --2- 172.21.15.89:0/1253548815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996b3330 0x5643996d8a60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4003fd0 tx=0x7f9ab4003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46ec22f3565eb09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.739+0000 7f9aa6000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:40:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.739+0000 7f9aa6000640 1 -- 172.21.15.89:0/1253548815 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4036040 con 0x5643996b3330 2024-09-10T08:40:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.739+0000 7f9aa6000640 1 -- 172.21.15.89:0/1253548815 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f9ab4003dd0 con 0x5643996b3330 2024-09-10T08:40:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.739+0000 7f9ac584d740 1 -- 172.21.15.89:0/1253548815 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aa80043c0 con 0x5643996b3330 2024-09-10T08:40:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.740+0000 7f9aa6000640 1 -- 172.21.15.89:0/1253548815 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9ab4041040 con 0x5643996b3330 2024-09-10T08:40:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.740+0000 7f9ac584d740 10 client.15318 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.740+0000 7f9ac584d740 1 -- 172.21.15.89:0/1253548815 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5643996d7160 con 0x5643996b3330 2024-09-10T08:40:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.740+0000 7f9aa6000640 1 -- 172.21.15.89:0/1253548815 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aa80043c0 con 0x5643996b3330 2024-09-10T08:40:10.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.740+0000 7f9aa6000640 1 -- 172.21.15.89:0/1253548815 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9ab40303a0 con 0x5643996b3330 2024-09-10T08:40:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.740+0000 7f9ac584d740 10 client.15318 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.740+0000 7f9ac584d740 10 client.15318 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T08:40:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.740+0000 7f9ac584d740 20 client.15318 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.740+0000 7f9ac584d740 1 --2- 172.21.15.89:0/1253548815 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x56439973ddb0 0x56439973cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.740+0000 7f9ac584d740 4 client.15318 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:40:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.740+0000 7f9ac584d740 1 -- 172.21.15.89:0/1253548815 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "status"}) -- 0x564399620b30 con 0x56439973ddb0 2024-09-10T08:40:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.741+0000 7f9abb400640 1 --2- 172.21.15.89:0/1253548815 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x56439973ddb0 0x56439973cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.741+0000 7f9abb400640 1 --2- 172.21.15.89:0/1253548815 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x56439973ddb0 0x56439973cac0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=987542bb930da4a0 server_cookie=5101b33a4936181a in_seq=0 out_seq=0 2024-09-10T08:40:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.741+0000 7f9aa6000640 10 client.15318 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:40:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.741+0000 7f9aa6000640 1 -- 172.21.15.89:0/1253548815 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x564399620b30 con 0x56439973ddb0 2024-09-10T08:40:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.741+0000 7f9aa6000640 10 client.15318 handle_command_reply: tid=0 2024-09-10T08:40:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.741+0000 7f9ac584d740 1 client.15318 shutdown 2024-09-10T08:40:10.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.742+0000 7f9ac584d740 1 -- 172.21.15.89:0/1253548815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996b3330 msgr2=0x5643996d8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:10.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.742+0000 7f9ac584d740 1 --2- 172.21.15.89:0/1253548815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996b3330 0x5643996d8a60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4003fd0 tx=0x7f9ab4003cd0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.742+0000 7f9ac584d740 1 -- 172.21.15.89:0/1253548815 shutdown_connections 2024-09-10T08:40:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.742+0000 7f9ac584d740 1 --2- 172.21.15.89:0/1253548815 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x56439973ddb0 0x56439973cac0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.742+0000 7f9ac584d740 1 --2- 172.21.15.89:0/1253548815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643996be510 0x5643996b8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.742+0000 7f9ac584d740 1 --2- 172.21.15.89:0/1253548815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643996b3c50 0x5643996d8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.742+0000 7f9ac584d740 1 --2- 172.21.15.89:0/1253548815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643996b3330 0x5643996d8a60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.742+0000 7f9ac584d740 1 -- 172.21.15.89:0/1253548815 >> 172.21.15.89:0/1253548815 conn(0x5643996af8c0 msgr2=0x5643996c9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.742+0000 7f9aa6000640 0 client.15318 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:40:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.742+0000 7f9ac584d740 1 -- 172.21.15.89:0/1253548815 shutdown_connections 2024-09-10T08:40:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.742+0000 7f9ac584d740 1 -- 172.21.15.89:0/1253548815 wait complete. 2024-09-10T08:40:10.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.742+0000 7f9ac584d740 20 client.15318 trim_cache size 0 max 16384 2024-09-10T08:40:10.749 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:40:10.749 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "91f747bc-6f4e-11ef-bcea-c7b262605968", 2024-09-10T08:40:10.749 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 1, 2024-09-10T08:40:10.749 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24518, 2024-09-10T08:40:10.749 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2024-09-10T08:40:10.749 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2024-09-10T08:40:10.749 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2024-09-10T08:40:10.749 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 144.04994979, 2024-09-10T08:40:10.749 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 26, 2024-09-10T08:40:10.750 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 96, 2024-09-10T08:40:10.752 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 96, 2024-09-10T08:40:10.752 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 217.906797947 2024-09-10T08:40:10.752 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:40:10.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.743+0000 7f9ac2e00640 1 -- 172.21.15.89:0/1982831969 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9aa00a2c10 msgr2=0x7f9aa00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:10.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.743+0000 7f9ac2e00640 1 --2- 172.21.15.89:0/1982831969 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9aa00a2c10 0x7f9aa00a50d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9aac09af00 tx=0x7f9aac0d29e0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.743+0000 7f9ac2e00640 1 -- 172.21.15.89:0/1982831969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9abc076720 msgr2=0x7f9abc1a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:10.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.743+0000 7f9ac2e00640 1 --2- 172.21.15.89:0/1982831969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9abc076720 0x7f9abc1a46a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00d950 tx=0x7f9aac00de10 comp rx=0 tx=0).stop 2024-09-10T08:40:10.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.743+0000 7f9ac2e00640 1 -- 172.21.15.89:0/1982831969 shutdown_connections 2024-09-10T08:40:10.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.743+0000 7f9ac2e00640 1 --2- 172.21.15.89:0/1982831969 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9aa00a2c10 0x7f9aa00a50d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.743+0000 7f9ac2e00640 1 --2- 172.21.15.89:0/1982831969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9abc112a30 0x7f9abc19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.743+0000 7f9ac2e00640 1 --2- 172.21.15.89:0/1982831969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9abc077040 0x7f9abc1a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.743+0000 7f9ac2e00640 1 --2- 172.21.15.89:0/1982831969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9abc076720 0x7f9abc1a46a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:10.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.743+0000 7f9ac2e00640 1 -- 172.21.15.89:0/1982831969 >> 172.21.15.89:0/1982831969 conn(0x7f9abc0ffa90 msgr2=0x7f9abc101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:10.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.743+0000 7f9ac2e00640 1 -- 172.21.15.89:0/1982831969 shutdown_connections 2024-09-10T08:40:10.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:10.743+0000 7f9ac2e00640 1 -- 172.21.15.89:0/1982831969 wait complete. 2024-09-10T08:40:10.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:10 smithi149 ceph-mon[36167]: pgmap v400: 129 pgs: 129 active+clean; 654 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-09-10T08:40:11.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:10 smithi111 ceph-mon[35220]: pgmap v400: 129 pgs: 129 active+clean; 654 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-09-10T08:40:11.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:10 smithi089 ceph-mon[30757]: pgmap v400: 129 pgs: 129 active+clean; 654 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-09-10T08:40:11.541 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T08:40:11.915 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:40:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.491+0000 7fb2ebe00640 1 -- 172.21.15.89:0/564188662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2ec102360 msgr2=0x7fb2ec10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.491+0000 7fb2ebe00640 1 --2- 172.21.15.89:0/564188662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2ec102360 0x7fb2ec10e890 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009cd0 tx=0x7fb2e402a8e0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.492+0000 7fb2ebe00640 1 -- 172.21.15.89:0/564188662 shutdown_connections 2024-09-10T08:40:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.492+0000 7fb2ebe00640 1 --2- 172.21.15.89:0/564188662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2ec102360 0x7fb2ec10e890 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.492+0000 7fb2ebe00640 1 --2- 172.21.15.89:0/564188662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2ec1019c0 0x7fb2ec101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.492+0000 7fb2ebe00640 1 --2- 172.21.15.89:0/564188662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2ec1079c0 0x7fb2ec107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.492+0000 7fb2ebe00640 1 -- 172.21.15.89:0/564188662 >> 172.21.15.89:0/564188662 conn(0x7fb2ec0fd6e0 msgr2=0x7fb2ec0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.492+0000 7fb2ebe00640 1 -- 172.21.15.89:0/564188662 shutdown_connections 2024-09-10T08:40:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.492+0000 7fb2ebe00640 1 -- 172.21.15.89:0/564188662 wait complete. 2024-09-10T08:40:12.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.493+0000 7fb2ebe00640 1 Processor -- start 2024-09-10T08:40:12.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.493+0000 7fb2ebe00640 1 -- start start 2024-09-10T08:40:12.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.493+0000 7fb2ebe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2ec1019c0 0x7fb2ec19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:12.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.493+0000 7fb2ebe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2ec102360 0x7fb2ec19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:12.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.493+0000 7fb2ebe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2ec1079c0 0x7fb2ec19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:12.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.493+0000 7fb2ebe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec111e10 con 0x7fb2ec1079c0 2024-09-10T08:40:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.493+0000 7fb2ebe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec111c90 con 0x7fb2ec102360 2024-09-10T08:40:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.493+0000 7fb2ebe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec111f90 con 0x7fb2ec1019c0 2024-09-10T08:40:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.493+0000 7fb2eaa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2ec1019c0 0x7fb2ec19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.493+0000 7fb2eaa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2ec1019c0 0x7fb2ec19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44866/0 (socket says 172.21.15.89:44866) 2024-09-10T08:40:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.493+0000 7fb2eaa00640 1 -- 172.21.15.89:0/4238465674 learned_addr learned my addr 172.21.15.89:0/4238465674 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.493+0000 7fb2eb400640 1 --2- 172.21.15.89:0/4238465674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2ec1079c0 0x7fb2ec19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.494+0000 7fb2ea000640 1 --2- 172.21.15.89:0/4238465674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2ec102360 0x7fb2ec19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.494+0000 7fb2eaa00640 1 -- 172.21.15.89:0/4238465674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2ec102360 msgr2=0x7fb2ec19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.494+0000 7fb2eaa00640 1 --2- 172.21.15.89:0/4238465674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2ec102360 0x7fb2ec19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.494+0000 7fb2eaa00640 1 -- 172.21.15.89:0/4238465674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2ec1079c0 msgr2=0x7fb2ec19ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.494+0000 7fb2eaa00640 1 --2- 172.21.15.89:0/4238465674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2ec1079c0 0x7fb2ec19ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.494+0000 7fb2eaa00640 1 -- 172.21.15.89:0/4238465674 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2ec19f390 con 0x7fb2ec1019c0 2024-09-10T08:40:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.494+0000 7fb2eaa00640 1 --2- 172.21.15.89:0/4238465674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2ec1019c0 0x7fb2ec19a3e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00e9f0 tx=0x7fb2dc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.495+0000 7fb2e3400640 1 -- 172.21.15.89:0/4238465674 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc00cd80 con 0x7fb2ec1019c0 2024-09-10T08:40:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.495+0000 7fb2e3400640 1 -- 172.21.15.89:0/4238465674 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2dc010910 con 0x7fb2ec1019c0 2024-09-10T08:40:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.495+0000 7fb2ebe00640 1 -- 172.21.15.89:0/4238465674 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2ec19f680 con 0x7fb2ec1019c0 2024-09-10T08:40:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.495+0000 7fb2e3400640 1 -- 172.21.15.89:0/4238465674 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc018990 con 0x7fb2ec1019c0 2024-09-10T08:40:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.495+0000 7fb2ebe00640 1 -- 172.21.15.89:0/4238465674 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2ec1a6f60 con 0x7fb2ec1019c0 2024-09-10T08:40:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.498+0000 7fb2f45bb740 1 Processor -- start 2024-09-10T08:40:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.498+0000 7fb2f45bb740 1 -- start start 2024-09-10T08:40:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.498+0000 7fb2e3400640 1 -- 172.21.15.89:0/4238465674 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fb2dc010ac0 con 0x7fb2ec1019c0 2024-09-10T08:40:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.498+0000 7fb2f45bb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370db4e0 0x55a9370db8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:12.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.498+0000 7fb2f45bb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370dbe00 0x55a9370e6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.498+0000 7fb2f45bb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370e66c0 0x55a9370e8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.498+0000 7fb2f45bb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a9370410b0 con 0x55a9370dbe00 2024-09-10T08:40:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.498+0000 7fb2f45bb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a936fe8ab0 con 0x55a9370db4e0 2024-09-10T08:40:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.498+0000 7fb2f45bb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a93703d070 con 0x55a9370e66c0 2024-09-10T08:40:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.498+0000 7fb2eaa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370dbe00 0x55a9370e6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.499+0000 7fb2eb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370db4e0 0x55a9370db8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.499+0000 7fb2ea000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370e66c0 0x55a9370e8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.499+0000 7fb2eb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370db4e0 0x55a9370db8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:60924/0 (socket says 172.21.15.89:60924) 2024-09-10T08:40:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.499+0000 7fb2ea000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370e66c0 0x55a9370e8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44882/0 (socket says 172.21.15.89:44882) 2024-09-10T08:40:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.499+0000 7fb2eb400640 1 -- 172.21.15.89:0/1493079955 learned_addr learned my addr 172.21.15.89:0/1493079955 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:12.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.499+0000 7fb2eb400640 1 -- 172.21.15.89:0/1493079955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370e66c0 msgr2=0x55a9370e8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.499+0000 7fb2eb400640 1 --2- 172.21.15.89:0/1493079955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370e66c0 0x55a9370e8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.499+0000 7fb2eb400640 1 -- 172.21.15.89:0/1493079955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370dbe00 msgr2=0x55a9370e6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.499+0000 7fb2eb400640 1 --2- 172.21.15.89:0/1493079955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370dbe00 0x55a9370e6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.499+0000 7fb2eb400640 1 -- 172.21.15.89:0/1493079955 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a936ff54a0 con 0x55a9370db4e0 2024-09-10T08:40:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.499+0000 7fb2eaa00640 1 --2- 172.21.15.89:0/1493079955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370dbe00 0x55a9370e6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.499+0000 7fb2eb400640 1 --2- 172.21.15.89:0/1493079955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370db4e0 0x55a9370db8c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009cd0 tx=0x7fb2e4009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c483c4e43be0944 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.500+0000 7fb2e3400640 1 --2- 172.21.15.89:0/4238465674 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb2d00a2c10 0x7fb2d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.500+0000 7fb2d7e00640 1 -- 172.21.15.89:0/1493079955 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4049070 con 0x55a9370db4e0 2024-09-10T08:40:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.500+0000 7fb2e3400640 1 -- 172.21.15.89:0/4238465674 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb2dc0cf890 con 0x7fb2ec1019c0 2024-09-10T08:40:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.500+0000 7fb2eaa00640 1 --2- 172.21.15.89:0/4238465674 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb2d00a2c10 0x7fb2d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:12.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.500+0000 7fb2d7e00640 1 -- 172.21.15.89:0/1493079955 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2e4036040 con 0x55a9370db4e0 2024-09-10T08:40:12.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.500+0000 7fb2d7e00640 1 -- 172.21.15.89:0/1493079955 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4004040 con 0x55a9370db4e0 2024-09-10T08:40:12.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.501+0000 7fb2f45bb740 1 -- 172.21.15.89:0/1493079955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370db4e0 msgr2=0x55a9370db8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:12.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.501+0000 7fb2eaa00640 1 --2- 172.21.15.89:0/4238465674 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb2d00a2c10 0x7fb2d00a50d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc09b910 tx=0x7fb2dc0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:12.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.501+0000 7fb2f45bb740 1 --2- 172.21.15.89:0/1493079955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370db4e0 0x55a9370db8c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009cd0 tx=0x7fb2e4009a20 comp rx=0 tx=0).stop 2024-09-10T08:40:12.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.501+0000 7fb2f45bb740 1 -- 172.21.15.89:0/1493079955 shutdown_connections 2024-09-10T08:40:12.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.501+0000 7fb2f45bb740 1 --2- 172.21.15.89:0/1493079955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370e66c0 0x55a9370e8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.501+0000 7fb2f45bb740 1 --2- 172.21.15.89:0/1493079955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370dbe00 0x55a9370e6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.501+0000 7fb2f45bb740 1 --2- 172.21.15.89:0/1493079955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370db4e0 0x55a9370db8c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.501+0000 7fb2f45bb740 1 -- 172.21.15.89:0/1493079955 >> 172.21.15.89:0/1493079955 conn(0x55a9370d7a70 msgr2=0x55a9370d7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:12.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.501+0000 7fb2f45bb740 1 -- 172.21.15.89:0/1493079955 shutdown_connections 2024-09-10T08:40:12.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.501+0000 7fb2f45bb740 1 -- 172.21.15.89:0/1493079955 wait complete. 2024-09-10T08:40:12.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.501+0000 7fb2f45bb740 1 -- start start 2024-09-10T08:40:12.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.502+0000 7fb2f45bb740 1 Processor -- start 2024-09-10T08:40:12.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.502+0000 7fb2f45bb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370db4e0 0x55a9370e3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:12.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.502+0000 7fb2f45bb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370dbe00 0x55a9370e3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:12.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.502+0000 7fb2f45bb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370e66c0 0x55a9370e40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:12.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.502+0000 7fb2f45bb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a9370e9330 con 0x55a9370e66c0 2024-09-10T08:40:12.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.502+0000 7fb2f45bb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a936fc9920 con 0x55a9370dbe00 2024-09-10T08:40:12.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.502+0000 7fb2f45bb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a9370e94b0 con 0x55a9370db4e0 2024-09-10T08:40:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.502+0000 7fb2f45bb740 10 client.0 _ll_register_callbacks cb 0x55a936fed8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.502+0000 7fb2eb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370db4e0 0x55a9370e3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.502+0000 7fb2eb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370db4e0 0x55a9370e3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44896/0 (socket says 172.21.15.89:44896) 2024-09-10T08:40:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.503+0000 7fb2eb400640 1 -- 172.21.15.89:0/1942758656 learned_addr learned my addr 172.21.15.89:0/1942758656 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.503+0000 7fb2eb400640 1 --2- 172.21.15.89:0/1942758656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370e66c0 0x55a9370e40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.503+0000 7fb2ea000640 1 --2- 172.21.15.89:0/1942758656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370dbe00 0x55a9370e3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.503+0000 7fb2ea000640 1 -- 172.21.15.89:0/1942758656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370db4e0 msgr2=0x55a9370e3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.503+0000 7fb2ea000640 1 --2- 172.21.15.89:0/1942758656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370db4e0 0x55a9370e3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.503+0000 7fb2ea000640 1 -- 172.21.15.89:0/1942758656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370e66c0 msgr2=0x55a9370e40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.503+0000 7fb2ea000640 1 --2- 172.21.15.89:0/1942758656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370e66c0 0x55a9370e40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.503+0000 7fb2ea000640 1 -- 172.21.15.89:0/1942758656 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a936db22b0 con 0x55a9370dbe00 2024-09-10T08:40:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.503+0000 7fb2ea000640 1 --2- 172.21.15.89:0/1942758656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370dbe00 0x55a9370e3bb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec10ec50 tx=0x7fb2d8012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dea9e408cccffa7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.503+0000 7fb2d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:40:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.504+0000 7fb2d4c00640 1 -- 172.21.15.89:0/1942758656 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d800f920 con 0x55a9370dbe00 2024-09-10T08:40:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.504+0000 7fb2f45bb740 1 -- 172.21.15.89:0/1942758656 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2e4036040 con 0x55a9370dbe00 2024-09-10T08:40:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.504+0000 7fb2d4c00640 1 -- 172.21.15.89:0/1942758656 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb2d800fac0 con 0x55a9370dbe00 2024-09-10T08:40:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.504+0000 7fb2d4c00640 1 -- 172.21.15.89:0/1942758656 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb2d800fcc0 con 0x55a9370dbe00 2024-09-10T08:40:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.504+0000 7fb2f45bb740 10 client.25379 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.504+0000 7fb2f45bb740 1 -- 172.21.15.89:0/1942758656 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a9370f5150 con 0x55a9370dbe00 2024-09-10T08:40:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.504+0000 7fb2d4c00640 1 -- 172.21.15.89:0/1942758656 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2e4036040 con 0x55a9370dbe00 2024-09-10T08:40:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.504+0000 7fb2d4c00640 1 -- 172.21.15.89:0/1942758656 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb2d801c420 con 0x55a9370dbe00 2024-09-10T08:40:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.505+0000 7fb2f45bb740 10 client.25379 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.505+0000 7fb2f45bb740 10 client.25379 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:40:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.505+0000 7fb2f45bb740 20 client.25379 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.505+0000 7fb2f45bb740 1 --2- 172.21.15.89:0/1942758656 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a9370f77b0 0x55a9370f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.505+0000 7fb2f45bb740 4 client.25379 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:40:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.505+0000 7fb2f45bb740 1 -- 172.21.15.89:0/1942758656 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a93704e620 con 0x55a9370f77b0 2024-09-10T08:40:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.505+0000 7fb2eb400640 1 --2- 172.21.15.89:0/1942758656 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a9370f77b0 0x55a9370f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.506+0000 7fb2eb400640 1 --2- 172.21.15.89:0/1942758656 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a9370f77b0 0x55a9370f8c20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f3d8e09677b4320 server_cookie=f0f021bc86b0ff44 in_seq=0 out_seq=0 2024-09-10T08:40:12.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.506+0000 7fb2d4c00640 10 client.25379 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.506+0000 7fb2d4c00640 1 -- 172.21.15.89:0/1942758656 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a93704e620 con 0x55a9370f77b0 2024-09-10T08:40:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.506+0000 7fb2d4c00640 10 client.25379 handle_command_reply: tid=0 2024-09-10T08:40:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.506+0000 7fb2f45bb740 1 client.25379 shutdown 2024-09-10T08:40:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.507+0000 7fb2f45bb740 1 -- 172.21.15.89:0/1942758656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370dbe00 msgr2=0x55a9370e3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.507+0000 7fb2f45bb740 1 --2- 172.21.15.89:0/1942758656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370dbe00 0x55a9370e3bb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec10ec50 tx=0x7fb2d8012b60 comp rx=0 tx=0).stop 2024-09-10T08:40:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.507+0000 7fb2f45bb740 1 -- 172.21.15.89:0/1942758656 shutdown_connections 2024-09-10T08:40:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.507+0000 7fb2f45bb740 1 --2- 172.21.15.89:0/1942758656 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a9370f77b0 0x55a9370f8c20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.507+0000 7fb2f45bb740 1 --2- 172.21.15.89:0/1942758656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370e66c0 0x55a9370e40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.507+0000 7fb2f45bb740 1 --2- 172.21.15.89:0/1942758656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370dbe00 0x55a9370e3bb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.507+0000 7fb2d4c00640 0 client.25379 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.507+0000 7fb2f45bb740 1 --2- 172.21.15.89:0/1942758656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370db4e0 0x55a9370e3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.507+0000 7fb2f45bb740 1 -- 172.21.15.89:0/1942758656 >> 172.21.15.89:0/1942758656 conn(0x55a9370d7a70 msgr2=0x55a9370d6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.507+0000 7fb2f45bb740 1 -- 172.21.15.89:0/1942758656 shutdown_connections 2024-09-10T08:40:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.507+0000 7fb2f45bb740 1 -- 172.21.15.89:0/1942758656 wait complete. 2024-09-10T08:40:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.507+0000 7fb2f45bb740 20 client.25379 trim_cache size 0 max 16384 2024-09-10T08:40:12.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.525+0000 7fb2f45bb740 1 Processor -- start 2024-09-10T08:40:12.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.525+0000 7fb2f45bb740 1 -- start start 2024-09-10T08:40:12.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.526+0000 7fb2f45bb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370db4e0 0x55a9370f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:12.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.526+0000 7fb2f45bb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370dbe00 0x55a9370e1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:12.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.526+0000 7fb2f45bb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370e66c0 0x55a9370e2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:12.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.526+0000 7fb2f45bb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a9370f6f60 con 0x55a9370dbe00 2024-09-10T08:40:12.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.526+0000 7fb2f45bb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a9370f7630 con 0x55a9370e66c0 2024-09-10T08:40:12.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.526+0000 7fb2f45bb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a9370f7d00 con 0x55a9370db4e0 2024-09-10T08:40:12.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.526+0000 7fb2ea000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370dbe00 0x55a9370e1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:12.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.526+0000 7fb2ea000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370dbe00 0x55a9370e1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33686/0 (socket says 172.21.15.89:33686) 2024-09-10T08:40:12.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.526+0000 7fb2ea000640 1 -- 172.21.15.89:0/1889763907 learned_addr learned my addr 172.21.15.89:0/1889763907 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:12.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.526+0000 7fb2eb400640 1 --2- 172.21.15.89:0/1889763907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370db4e0 0x55a9370f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:12.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.526+0000 7fb2eb400640 1 --2- 172.21.15.89:0/1889763907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370e66c0 0x55a9370e2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:12.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.526+0000 7fb2ea000640 1 -- 172.21.15.89:0/1889763907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370db4e0 msgr2=0x55a9370f8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:12.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.526+0000 7fb2ea000640 1 --2- 172.21.15.89:0/1889763907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370db4e0 0x55a9370f8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.526+0000 7fb2ea000640 1 -- 172.21.15.89:0/1889763907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370e66c0 msgr2=0x55a9370e2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:12.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.526+0000 7fb2ea000640 1 --2- 172.21.15.89:0/1889763907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370e66c0 0x55a9370e2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.526+0000 7fb2ea000640 1 -- 172.21.15.89:0/1889763907 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9370f33c0 con 0x55a9370dbe00 2024-09-10T08:40:12.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.526+0000 7fb2eb400640 1 --2- 172.21.15.89:0/1889763907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370e66c0 0x55a9370e2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:40:12.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.527+0000 7fb2ea000640 1 --2- 172.21.15.89:0/1889763907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370dbe00 0x55a9370e1f40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80098d0 tx=0x7fb2d80057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f143566b32b3858a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:12.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.527+0000 7fb2d7400640 1 -- 172.21.15.89:0/1889763907 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d800f920 con 0x55a9370dbe00 2024-09-10T08:40:12.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.527+0000 7fb2d7400640 1 -- 172.21.15.89:0/1889763907 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2d800fac0 con 0x55a9370dbe00 2024-09-10T08:40:12.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.527+0000 7fb2d7400640 1 -- 172.21.15.89:0/1889763907 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d80035f0 con 0x55a9370dbe00 2024-09-10T08:40:12.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.527+0000 7fb2f45bb740 1 -- 172.21.15.89:0/1889763907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370dbe00 msgr2=0x55a9370e1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:12.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.527+0000 7fb2f45bb740 1 --2- 172.21.15.89:0/1889763907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370dbe00 0x55a9370e1f40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80098d0 tx=0x7fb2d80057d0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.528+0000 7fb2f45bb740 1 -- 172.21.15.89:0/1889763907 shutdown_connections 2024-09-10T08:40:12.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.528+0000 7fb2f45bb740 1 --2- 172.21.15.89:0/1889763907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370e66c0 0x55a9370e2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.528+0000 7fb2f45bb740 1 --2- 172.21.15.89:0/1889763907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370dbe00 0x55a9370e1f40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.528+0000 7fb2f45bb740 1 --2- 172.21.15.89:0/1889763907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370db4e0 0x55a9370f8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.528+0000 7fb2f45bb740 1 -- 172.21.15.89:0/1889763907 >> 172.21.15.89:0/1889763907 conn(0x55a9370d7a70 msgr2=0x55a9370de940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:12.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.528+0000 7fb2f45bb740 1 -- 172.21.15.89:0/1889763907 shutdown_connections 2024-09-10T08:40:12.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.528+0000 7fb2f45bb740 1 -- 172.21.15.89:0/1889763907 wait complete. 2024-09-10T08:40:12.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.528+0000 7fb2f45bb740 1 -- start start 2024-09-10T08:40:12.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.528+0000 7fb2f45bb740 1 Processor -- start 2024-09-10T08:40:12.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.529+0000 7fb2f45bb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370db4e0 0x55a9370fb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:12.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.529+0000 7fb2f45bb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370dbe00 0x55a9370fb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:12.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.529+0000 7fb2f45bb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370e66c0 0x55a9370fbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:12.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.529+0000 7fb2f45bb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a9370f8aa0 con 0x55a9370e66c0 2024-09-10T08:40:12.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.529+0000 7fb2f45bb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a9370f83d0 con 0x55a9370dbe00 2024-09-10T08:40:12.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.529+0000 7fb2f45bb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a9370f9170 con 0x55a9370db4e0 2024-09-10T08:40:12.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.529+0000 7fb2f45bb740 10 client.0 _ll_register_callbacks cb 0x7fb2d8009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:12.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.529+0000 7fb2ea000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370dbe00 0x55a9370fb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:12.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.529+0000 7fb2ea000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370dbe00 0x55a9370fb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:60940/0 (socket says 172.21.15.89:60940) 2024-09-10T08:40:12.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.529+0000 7fb2ea000640 1 -- 172.21.15.89:0/569569768 learned_addr learned my addr 172.21.15.89:0/569569768 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:12.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.529+0000 7fb2eb400640 1 --2- 172.21.15.89:0/569569768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370e66c0 0x55a9370fbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:12.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.529+0000 7fb2eb400640 1 --2- 172.21.15.89:0/569569768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370db4e0 0x55a9370fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:12.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.529+0000 7fb2ea000640 1 -- 172.21.15.89:0/569569768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370db4e0 msgr2=0x55a9370fb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:12.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.529+0000 7fb2ea000640 1 --2- 172.21.15.89:0/569569768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370db4e0 0x55a9370fb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.529+0000 7fb2ea000640 1 -- 172.21.15.89:0/569569768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370e66c0 msgr2=0x55a9370fbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:12.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.529+0000 7fb2ea000640 1 --2- 172.21.15.89:0/569569768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370e66c0 0x55a9370fbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.529+0000 7fb2ea000640 1 -- 172.21.15.89:0/569569768 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9370ff630 con 0x55a9370dbe00 2024-09-10T08:40:12.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.529+0000 7fb2eb400640 1 --2- 172.21.15.89:0/569569768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370e66c0 0x55a9370fbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:40:12.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.530+0000 7fb2ea000640 1 --2- 172.21.15.89:0/569569768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370dbe00 0x55a9370fb6c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb2d801c910 tx=0x7fb2d80052d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dadfb519be70265 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:12.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.530+0000 7fb2d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:40:12.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.530+0000 7fb2d4c00640 1 -- 172.21.15.89:0/569569768 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d801ca80 con 0x55a9370dbe00 2024-09-10T08:40:12.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.530+0000 7fb2d4c00640 1 -- 172.21.15.89:0/569569768 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb2d801cc20 con 0x55a9370dbe00 2024-09-10T08:40:12.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.530+0000 7fb2f45bb740 1 -- 172.21.15.89:0/569569768 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2d800fac0 con 0x55a9370dbe00 2024-09-10T08:40:12.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.531+0000 7fb2d4c00640 1 -- 172.21.15.89:0/569569768 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb2d803a050 con 0x55a9370dbe00 2024-09-10T08:40:12.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.531+0000 7fb2f45bb740 10 client.25385 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:12.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.531+0000 7fb2f45bb740 1 -- 172.21.15.89:0/569569768 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a937162290 con 0x55a9370dbe00 2024-09-10T08:40:12.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.531+0000 7fb2d4c00640 1 -- 172.21.15.89:0/569569768 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2d800fac0 con 0x55a9370dbe00 2024-09-10T08:40:12.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.531+0000 7fb2d4c00640 1 -- 172.21.15.89:0/569569768 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb2d802a5e0 con 0x55a9370dbe00 2024-09-10T08:40:12.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.531+0000 7fb2f45bb740 10 client.25385 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:12.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.531+0000 7fb2f45bb740 10 client.25385 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:40:12.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.531+0000 7fb2f45bb740 20 client.25385 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:12.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.531+0000 7fb2f45bb740 1 --2- 172.21.15.89:0/569569768 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a937166be0 0x55a9371658f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:12.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.531+0000 7fb2f45bb740 4 client.25385 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:40:12.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.531+0000 7fb2f45bb740 1 -- 172.21.15.89:0/569569768 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "status"}) -- 0x55a937048b30 con 0x55a937166be0 2024-09-10T08:40:12.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.532+0000 7fb2eb400640 1 --2- 172.21.15.89:0/569569768 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a937166be0 0x55a9371658f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:12.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.532+0000 7fb2eb400640 1 --2- 172.21.15.89:0/569569768 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a937166be0 0x55a9371658f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1eeb109139a91e6 server_cookie=e9f6cb68eb71497a in_seq=0 out_seq=0 2024-09-10T08:40:12.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.532+0000 7fb2d4c00640 10 client.25385 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:12.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.532+0000 7fb2d4c00640 1 -- 172.21.15.89:0/569569768 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a937048b30 con 0x55a937166be0 2024-09-10T08:40:12.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.532+0000 7fb2d4c00640 10 client.25385 handle_command_reply: tid=0 2024-09-10T08:40:12.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.532+0000 7fb2f45bb740 1 client.25385 shutdown 2024-09-10T08:40:12.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.532+0000 7fb2f45bb740 1 -- 172.21.15.89:0/569569768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370dbe00 msgr2=0x55a9370fb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:12.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.532+0000 7fb2f45bb740 1 --2- 172.21.15.89:0/569569768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370dbe00 0x55a9370fb6c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb2d801c910 tx=0x7fb2d80052d0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.532+0000 7fb2f45bb740 1 -- 172.21.15.89:0/569569768 shutdown_connections 2024-09-10T08:40:12.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.532+0000 7fb2f45bb740 1 --2- 172.21.15.89:0/569569768 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a937166be0 0x55a9371658f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.532+0000 7fb2f45bb740 1 --2- 172.21.15.89:0/569569768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a9370e66c0 0x55a9370fbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.532+0000 7fb2f45bb740 1 --2- 172.21.15.89:0/569569768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9370dbe00 0x55a9370fb6c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.532+0000 7fb2f45bb740 1 --2- 172.21.15.89:0/569569768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a9370db4e0 0x55a9370fb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.532+0000 7fb2d4c00640 0 client.25385 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:12.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.532+0000 7fb2f45bb740 1 -- 172.21.15.89:0/569569768 >> 172.21.15.89:0/569569768 conn(0x55a9370d7a70 msgr2=0x55a9370e1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:12.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.533+0000 7fb2f45bb740 1 -- 172.21.15.89:0/569569768 shutdown_connections 2024-09-10T08:40:12.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.533+0000 7fb2f45bb740 1 -- 172.21.15.89:0/569569768 wait complete. 2024-09-10T08:40:12.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.533+0000 7fb2f45bb740 20 client.25385 trim_cache size 0 max 16384 2024-09-10T08:40:12.538 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:40:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: "cluster_fsid": "91f747bc-6f4e-11ef-bcea-c7b262605968", 2024-09-10T08:40:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: "whoami": 0, 2024-09-10T08:40:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 24554, 2024-09-10T08:40:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: "want_state": "up:active", 2024-09-10T08:40:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: "state": "up:active", 2024-09-10T08:40:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: "fs_name": "cephfs", 2024-09-10T08:40:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: "rank_uptime": 194.385699351, 2024-09-10T08:40:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: "mdsmap_epoch": 26, 2024-09-10T08:40:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch": 96, 2024-09-10T08:40:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_epoch_barrier": 96, 2024-09-10T08:40:12.539 INFO:teuthology.orchestra.run.smithi089.stdout: "uptime": 207.39916180899999 2024-09-10T08:40:12.539 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:40:12.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.533+0000 7fb2ebe00640 1 -- 172.21.15.89:0/4238465674 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb2d00a2c10 msgr2=0x7fb2d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:12.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.533+0000 7fb2ebe00640 1 --2- 172.21.15.89:0/4238465674 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb2d00a2c10 0x7fb2d00a50d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc09b910 tx=0x7fb2dc0d5530 comp rx=0 tx=0).stop 2024-09-10T08:40:12.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.533+0000 7fb2ebe00640 1 -- 172.21.15.89:0/4238465674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2ec1019c0 msgr2=0x7fb2ec19a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:12.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.533+0000 7fb2ebe00640 1 --2- 172.21.15.89:0/4238465674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2ec1019c0 0x7fb2ec19a3e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00e9f0 tx=0x7fb2dc00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.533+0000 7fb2ebe00640 1 -- 172.21.15.89:0/4238465674 shutdown_connections 2024-09-10T08:40:12.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.533+0000 7fb2ebe00640 1 --2- 172.21.15.89:0/4238465674 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb2d00a2c10 0x7fb2d00a50d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.533+0000 7fb2ebe00640 1 --2- 172.21.15.89:0/4238465674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2ec1079c0 0x7fb2ec19ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.534+0000 7fb2ebe00640 1 --2- 172.21.15.89:0/4238465674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2ec102360 0x7fb2ec19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.534+0000 7fb2ebe00640 1 --2- 172.21.15.89:0/4238465674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2ec1019c0 0x7fb2ec19a3e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:12.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.534+0000 7fb2ebe00640 1 -- 172.21.15.89:0/4238465674 >> 172.21.15.89:0/4238465674 conn(0x7fb2ec0fd6e0 msgr2=0x7fb2ec10d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:12.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.534+0000 7fb2ebe00640 1 -- 172.21.15.89:0/4238465674 shutdown_connections 2024-09-10T08:40:12.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:12.534+0000 7fb2ebe00640 1 -- 172.21.15.89:0/4238465674 wait complete. 2024-09-10T08:40:12.933 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:12 smithi089 ceph-mon[30757]: pgmap v401: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-09-10T08:40:12.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:12 smithi149 ceph-mon[36167]: pgmap v401: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-09-10T08:40:13.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:12 smithi111 ceph-mon[35220]: pgmap v401: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-09-10T08:40:13.342 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-10T08:40:14.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:14 smithi149 ceph-mon[36167]: pgmap v402: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-10T08:40:15.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:14 smithi111 ceph-mon[35220]: pgmap v402: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-10T08:40:15.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:14 smithi089 ceph-mon[30757]: pgmap v402: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-10T08:40:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.832+0000 7f78a3000640 1 -- 172.21.15.89:0/1202429759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f789c103d60 msgr2=0x7f789c104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.832+0000 7f78a3000640 1 --2- 172.21.15.89:0/1202429759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f789c103d60 0x7f789c104160 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f788c009900 tx=0x7f788c02a410 comp rx=0 tx=0).stop 2024-09-10T08:40:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.833+0000 7f78a3000640 1 -- 172.21.15.89:0/1202429759 shutdown_connections 2024-09-10T08:40:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.833+0000 7f78a3000640 1 --2- 172.21.15.89:0/1202429759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f789c105910 0x7f789c10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.833+0000 7f78a3000640 1 --2- 172.21.15.89:0/1202429759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f789c104f50 0x7f789c1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.833+0000 7f78a3000640 1 --2- 172.21.15.89:0/1202429759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f789c103d60 0x7f789c104160 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.833+0000 7f78a3000640 1 -- 172.21.15.89:0/1202429759 >> 172.21.15.89:0/1202429759 conn(0x7f789c0ff4e0 msgr2=0x7f789c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.833+0000 7f78a3000640 1 -- 172.21.15.89:0/1202429759 shutdown_connections 2024-09-10T08:40:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.833+0000 7f78a3000640 1 -- 172.21.15.89:0/1202429759 wait complete. 2024-09-10T08:40:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.834+0000 7f78a3000640 1 Processor -- start 2024-09-10T08:40:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.834+0000 7f78a3000640 1 -- start start 2024-09-10T08:40:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.834+0000 7f78a3000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f789c103d60 0x7f789c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.834+0000 7f78a3000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f789c104f50 0x7f789c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.834+0000 7f78a3000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f789c105910 0x7f789c1a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.834+0000 7f78a3000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f789c10fa30 con 0x7f789c103d60 2024-09-10T08:40:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.834+0000 7f78a3000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f789c10f8b0 con 0x7f789c104f50 2024-09-10T08:40:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.834+0000 7f78a3000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f789c10fbb0 con 0x7f789c105910 2024-09-10T08:40:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.834+0000 7f789be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f789c103d60 0x7f789c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.834+0000 7f789be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f789c103d60 0x7f789c199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33658/0 (socket says 172.21.15.89:33658) 2024-09-10T08:40:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.834+0000 7f789be00640 1 -- 172.21.15.89:0/331641519 learned_addr learned my addr 172.21.15.89:0/331641519 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.835+0000 7f78a1000640 1 --2- 172.21.15.89:0/331641519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f789c105910 0x7f789c1a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.835+0000 7f7893e00640 1 --2- 172.21.15.89:0/331641519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f789c104f50 0x7f789c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.835+0000 7f789be00640 1 -- 172.21.15.89:0/331641519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f789c105910 msgr2=0x7f789c1a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.835+0000 7f789be00640 1 --2- 172.21.15.89:0/331641519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f789c105910 0x7f789c1a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.835+0000 7f789be00640 1 -- 172.21.15.89:0/331641519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f789c104f50 msgr2=0x7f789c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.835+0000 7f789be00640 1 --2- 172.21.15.89:0/331641519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f789c104f50 0x7f789c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.835+0000 7f789be00640 1 -- 172.21.15.89:0/331641519 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f789c1a1be0 con 0x7f789c103d60 2024-09-10T08:40:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.835+0000 7f7893e00640 1 --2- 172.21.15.89:0/331641519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f789c104f50 0x7f789c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:40:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.835+0000 7f789be00640 1 --2- 172.21.15.89:0/331641519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f789c103d60 0x7f789c199fc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f788c031440 tx=0x7f788c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.835+0000 7f7899600640 1 -- 172.21.15.89:0/331641519 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c03f030 con 0x7f789c103d60 2024-09-10T08:40:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.836+0000 7f7899600640 1 -- 172.21.15.89:0/331641519 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f788c003ce0 con 0x7f789c103d60 2024-09-10T08:40:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.836+0000 7f78a3000640 1 -- 172.21.15.89:0/331641519 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f789c1a1e70 con 0x7f789c103d60 2024-09-10T08:40:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.836+0000 7f7899600640 1 -- 172.21.15.89:0/331641519 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c035070 con 0x7f789c103d60 2024-09-10T08:40:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.836+0000 7f78a3000640 1 -- 172.21.15.89:0/331641519 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f789c1a22d0 con 0x7f789c103d60 2024-09-10T08:40:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.839+0000 7f7899600640 1 -- 172.21.15.89:0/331641519 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f788c042030 con 0x7f789c103d60 2024-09-10T08:40:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.839+0000 7f78a605a740 1 Processor -- start 2024-09-10T08:40:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.839+0000 7f78a605a740 1 -- start start 2024-09-10T08:40:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.839+0000 7f78a605a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d824c090 0x55a0d824c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.839+0000 7f78a605a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d824c9d0 0x55a0d8251490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.840+0000 7f78a605a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d8251a60 0x55a0d8253f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.840+0000 7f78a605a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a0d81b1100 con 0x55a0d824c090 2024-09-10T08:40:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.840+0000 7f78a605a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a0d81591b0 con 0x55a0d8251a60 2024-09-10T08:40:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.840+0000 7f78a605a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a0d81ad0c0 con 0x55a0d824c9d0 2024-09-10T08:40:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.840+0000 7f78a1000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d824c090 0x55a0d824c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.840+0000 7f789be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d824c9d0 0x55a0d8251490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.840+0000 7f7893e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d8251a60 0x55a0d8253f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.840+0000 7f78a1000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d824c090 0x55a0d824c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33666/0 (socket says 172.21.15.89:33666) 2024-09-10T08:40:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.840+0000 7f78a1000640 1 -- 172.21.15.89:0/256989990 learned_addr learned my addr 172.21.15.89:0/256989990 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.840+0000 7f7893e00640 1 -- 172.21.15.89:0/256989990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d824c9d0 msgr2=0x55a0d8251490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.840+0000 7f7893e00640 1 --2- 172.21.15.89:0/256989990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d824c9d0 0x55a0d8251490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.840+0000 7f7893e00640 1 -- 172.21.15.89:0/256989990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d824c090 msgr2=0x55a0d824c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.840+0000 7f7893e00640 1 --2- 172.21.15.89:0/256989990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d824c090 0x55a0d824c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.840+0000 7f7893e00640 1 -- 172.21.15.89:0/256989990 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d7ec4210 con 0x55a0d8251a60 2024-09-10T08:40:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.840+0000 7f789be00640 1 --2- 172.21.15.89:0/256989990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d824c9d0 0x55a0d8251490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.841+0000 7f7893e00640 1 --2- 172.21.15.89:0/256989990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d8251a60 0x55a0d8253f20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7884005fd0 tx=0x7f788400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5678b958c5bf7e06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.841+0000 7f7899600640 1 --2- 172.21.15.89:0/331641519 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f78800a2d50 0x7f78800a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.841+0000 7f7891600640 1 -- 172.21.15.89:0/256989990 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7884005020 con 0x55a0d8251a60 2024-09-10T08:40:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.841+0000 7f7891600640 1 -- 172.21.15.89:0/256989990 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78840047a0 con 0x55a0d8251a60 2024-09-10T08:40:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.841+0000 7f7891600640 1 -- 172.21.15.89:0/256989990 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7884011710 con 0x55a0d8251a60 2024-09-10T08:40:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.841+0000 7f7899600640 1 -- 172.21.15.89:0/331641519 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f788c0efb90 con 0x7f789c103d60 2024-09-10T08:40:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.841+0000 7f78a1000640 1 --2- 172.21.15.89:0/331641519 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f78800a2d50 0x7f78800a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.842+0000 7f78a605a740 1 -- 172.21.15.89:0/256989990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d8251a60 msgr2=0x55a0d8253f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.842+0000 7f78a1000640 1 --2- 172.21.15.89:0/331641519 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f78800a2d50 0x7f78800a5210 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7894011630 tx=0x7f7894009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.842+0000 7f78a605a740 1 --2- 172.21.15.89:0/256989990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d8251a60 0x55a0d8253f20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7884005fd0 tx=0x7f788400f730 comp rx=0 tx=0).stop 2024-09-10T08:40:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.842+0000 7f78a605a740 1 -- 172.21.15.89:0/256989990 shutdown_connections 2024-09-10T08:40:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.842+0000 7f78a605a740 1 --2- 172.21.15.89:0/256989990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d8251a60 0x55a0d8253f20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.842+0000 7f78a605a740 1 --2- 172.21.15.89:0/256989990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d824c9d0 0x55a0d8251490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.842+0000 7f78a605a740 1 --2- 172.21.15.89:0/256989990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d824c090 0x55a0d824c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.842+0000 7f78a605a740 1 -- 172.21.15.89:0/256989990 >> 172.21.15.89:0/256989990 conn(0x55a0d8247700 msgr2=0x55a0d8247b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.842+0000 7f78a605a740 1 -- 172.21.15.89:0/256989990 shutdown_connections 2024-09-10T08:40:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.842+0000 7f78a605a740 1 -- 172.21.15.89:0/256989990 wait complete. 2024-09-10T08:40:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.842+0000 7f78a605a740 1 -- start start 2024-09-10T08:40:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.843+0000 7f78a605a740 1 Processor -- start 2024-09-10T08:40:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.843+0000 7f78a605a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d824c090 0x55a0d82561e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.843+0000 7f78a605a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d824c9d0 0x55a0d8256720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.843+0000 7f78a605a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d8251a60 0x55a0d825d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.843+0000 7f78a605a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a0d8254a30 con 0x55a0d824c9d0 2024-09-10T08:40:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.843+0000 7f78a605a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a0d813a090 con 0x55a0d824c090 2024-09-10T08:40:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.843+0000 7f78a605a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a0d8254bb0 con 0x55a0d8251a60 2024-09-10T08:40:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.843+0000 7f78a605a740 10 client.0 _ll_register_callbacks cb 0x55a0d818ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.843+0000 7f789be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d824c090 0x55a0d82561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.843+0000 7f7893e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d824c9d0 0x55a0d8256720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.843+0000 7f789be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d824c090 0x55a0d82561e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50730/0 (socket says 172.21.15.89:50730) 2024-09-10T08:40:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.843+0000 7f789be00640 1 -- 172.21.15.89:0/3623252928 learned_addr learned my addr 172.21.15.89:0/3623252928 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.843+0000 7f78a1000640 1 --2- 172.21.15.89:0/3623252928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d8251a60 0x55a0d825d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.844+0000 7f789be00640 1 -- 172.21.15.89:0/3623252928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d8251a60 msgr2=0x55a0d825d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.844+0000 7f789be00640 1 --2- 172.21.15.89:0/3623252928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d8251a60 0x55a0d825d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.844+0000 7f789be00640 1 -- 172.21.15.89:0/3623252928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d824c9d0 msgr2=0x55a0d8256720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.844+0000 7f789be00640 1 --2- 172.21.15.89:0/3623252928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d824c9d0 0x55a0d8256720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.844+0000 7f789be00640 1 -- 172.21.15.89:0/3623252928 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d7f90620 con 0x55a0d824c090 2024-09-10T08:40:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.844+0000 7f789be00640 1 --2- 172.21.15.89:0/3623252928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d824c090 0x55a0d82561e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f788c0b8830 tx=0x7f788c02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d6e47bbb0987e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.844+0000 7f788a000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:40:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.844+0000 7f788a000640 1 -- 172.21.15.89:0/3623252928 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c049070 con 0x55a0d824c090 2024-09-10T08:40:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.844+0000 7f788a000640 1 -- 172.21.15.89:0/3623252928 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f788c0ed360 con 0x55a0d824c090 2024-09-10T08:40:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.844+0000 7f78a605a740 1 -- 172.21.15.89:0/3623252928 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78840047a0 con 0x55a0d824c090 2024-09-10T08:40:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.844+0000 7f788a000640 1 -- 172.21.15.89:0/3623252928 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f788c0f4040 con 0x55a0d824c090 2024-09-10T08:40:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.844+0000 7f78a605a740 10 client.25397 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.844+0000 7f78a605a740 1 -- 172.21.15.89:0/3623252928 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a0d8260550 con 0x55a0d824c090 2024-09-10T08:40:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.845+0000 7f788a000640 1 -- 172.21.15.89:0/3623252928 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78840047a0 con 0x55a0d824c090 2024-09-10T08:40:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.845+0000 7f788a000640 1 -- 172.21.15.89:0/3623252928 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f788c03b460 con 0x55a0d824c090 2024-09-10T08:40:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.845+0000 7f78a605a740 10 client.25397 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.845+0000 7f78a605a740 10 client.25397 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:40:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.845+0000 7f78a605a740 20 client.25397 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.845+0000 7f78a605a740 1 --2- 172.21.15.89:0/3623252928 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a0d8261cd0 0x55a0d82640c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.845+0000 7f78a605a740 4 client.25397 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:40:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.845+0000 7f78a605a740 1 -- 172.21.15.89:0/3623252928 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0d81be830 con 0x55a0d8261cd0 2024-09-10T08:40:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.845+0000 7f7893e00640 1 --2- 172.21.15.89:0/3623252928 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a0d8261cd0 0x55a0d82640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.846+0000 7f7893e00640 1 --2- 172.21.15.89:0/3623252928 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a0d8261cd0 0x55a0d82640c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b395717f62534fc server_cookie=2fa70204bc4f96fa in_seq=0 out_seq=0 2024-09-10T08:40:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.846+0000 7f788a000640 10 client.25397 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.846+0000 7f788a000640 1 -- 172.21.15.89:0/3623252928 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a0d81be830 con 0x55a0d8261cd0 2024-09-10T08:40:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.846+0000 7f788a000640 10 client.25397 handle_command_reply: tid=0 2024-09-10T08:40:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.847+0000 7f78a605a740 1 client.25397 shutdown 2024-09-10T08:40:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.847+0000 7f78a605a740 1 -- 172.21.15.89:0/3623252928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d824c090 msgr2=0x55a0d82561e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.847+0000 7f78a605a740 1 --2- 172.21.15.89:0/3623252928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d824c090 0x55a0d82561e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f788c0b8830 tx=0x7f788c02a410 comp rx=0 tx=0).stop 2024-09-10T08:40:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.847+0000 7f78a605a740 1 -- 172.21.15.89:0/3623252928 shutdown_connections 2024-09-10T08:40:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.847+0000 7f78a605a740 1 --2- 172.21.15.89:0/3623252928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d8251a60 0x55a0d825d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.847+0000 7f78a605a740 1 --2- 172.21.15.89:0/3623252928 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a0d8261cd0 0x55a0d82640c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.847+0000 7f78a605a740 1 --2- 172.21.15.89:0/3623252928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d824c9d0 0x55a0d8256720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.847+0000 7f788a000640 0 client.25397 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.847+0000 7f78a605a740 1 --2- 172.21.15.89:0/3623252928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d824c090 0x55a0d82561e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.847+0000 7f78a605a740 1 -- 172.21.15.89:0/3623252928 >> 172.21.15.89:0/3623252928 conn(0x55a0d8247700 msgr2=0x55a0d8248f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.847+0000 7f78a605a740 1 -- 172.21.15.89:0/3623252928 shutdown_connections 2024-09-10T08:40:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.847+0000 7f78a605a740 1 -- 172.21.15.89:0/3623252928 wait complete. 2024-09-10T08:40:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.847+0000 7f78a605a740 20 client.25397 trim_cache size 0 max 16384 2024-09-10T08:40:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.862+0000 7f78a605a740 1 Processor -- start 2024-09-10T08:40:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.862+0000 7f78a605a740 1 -- start start 2024-09-10T08:40:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.862+0000 7f78a605a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d824c090 0x55a0d8255f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.862+0000 7f78a605a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d824c9d0 0x55a0d825d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.862+0000 7f78a605a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d8251a60 0x55a0d82564d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.862+0000 7f78a605a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f789c1af180 con 0x55a0d824c9d0 2024-09-10T08:40:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.862+0000 7f78a605a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f789c1aca50 con 0x55a0d8251a60 2024-09-10T08:40:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.862+0000 7f78a605a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f789c1aa8a0 con 0x55a0d824c090 2024-09-10T08:40:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.862+0000 7f7893e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d824c9d0 0x55a0d825d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.863+0000 7f7893e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d824c9d0 0x55a0d825d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33678/0 (socket says 172.21.15.89:33678) 2024-09-10T08:40:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.863+0000 7f7893e00640 1 -- 172.21.15.89:0/22967231 learned_addr learned my addr 172.21.15.89:0/22967231 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.863+0000 7f789be00640 1 --2- 172.21.15.89:0/22967231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d824c090 0x55a0d8255f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.863+0000 7f78a1000640 1 --2- 172.21.15.89:0/22967231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d8251a60 0x55a0d82564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.863+0000 7f7893e00640 1 -- 172.21.15.89:0/22967231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d824c090 msgr2=0x55a0d8255f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.863+0000 7f7893e00640 1 --2- 172.21.15.89:0/22967231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d824c090 0x55a0d8255f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.863+0000 7f7893e00640 1 -- 172.21.15.89:0/22967231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d8251a60 msgr2=0x55a0d82564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.863+0000 7f7893e00640 1 --2- 172.21.15.89:0/22967231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d8251a60 0x55a0d82564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.863+0000 7f7893e00640 1 -- 172.21.15.89:0/22967231 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d8251800 con 0x55a0d824c9d0 2024-09-10T08:40:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.863+0000 7f78a1000640 1 --2- 172.21.15.89:0/22967231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d8251a60 0x55a0d82564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:40:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.863+0000 7f7893e00640 1 --2- 172.21.15.89:0/22967231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d824c9d0 0x55a0d825d720 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55a0d8248580 tx=0x7f788400b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e85cb8489d3ad79 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.863+0000 7f7890c00640 1 -- 172.21.15.89:0/22967231 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7884005340 con 0x55a0d824c9d0 2024-09-10T08:40:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.863+0000 7f7890c00640 1 -- 172.21.15.89:0/22967231 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78840054e0 con 0x55a0d824c9d0 2024-09-10T08:40:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.863+0000 7f7890c00640 1 -- 172.21.15.89:0/22967231 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78840116a0 con 0x55a0d824c9d0 2024-09-10T08:40:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.864+0000 7f78a605a740 1 -- 172.21.15.89:0/22967231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d824c9d0 msgr2=0x55a0d825d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.864+0000 7f78a605a740 1 --2- 172.21.15.89:0/22967231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d824c9d0 0x55a0d825d720 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55a0d8248580 tx=0x7f788400b5c0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.864+0000 7f78a605a740 1 -- 172.21.15.89:0/22967231 shutdown_connections 2024-09-10T08:40:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.864+0000 7f78a605a740 1 --2- 172.21.15.89:0/22967231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d8251a60 0x55a0d82564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.864+0000 7f78a605a740 1 --2- 172.21.15.89:0/22967231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d824c9d0 0x55a0d825d720 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.864+0000 7f78a605a740 1 --2- 172.21.15.89:0/22967231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d824c090 0x55a0d8255f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.864+0000 7f78a605a740 1 -- 172.21.15.89:0/22967231 >> 172.21.15.89:0/22967231 conn(0x55a0d8247700 msgr2=0x55a0d824db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.864+0000 7f78a605a740 1 -- 172.21.15.89:0/22967231 shutdown_connections 2024-09-10T08:40:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.864+0000 7f78a605a740 1 -- 172.21.15.89:0/22967231 wait complete. 2024-09-10T08:40:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.864+0000 7f78a605a740 1 -- start start 2024-09-10T08:40:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.865+0000 7f78a605a740 1 Processor -- start 2024-09-10T08:40:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.865+0000 7f78a605a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d824c090 0x55a0d825a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.865+0000 7f78a605a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d824c9d0 0x55a0d825a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.865+0000 7f78a605a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d8251a60 0x55a0d82ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.865+0000 7f78a605a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f789c1aaf10 con 0x55a0d8251a60 2024-09-10T08:40:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.865+0000 7f78a605a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f789c1ac380 con 0x55a0d824c9d0 2024-09-10T08:40:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.865+0000 7f78a605a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f789c1ab5e0 con 0x55a0d824c090 2024-09-10T08:40:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.865+0000 7f78a605a740 10 client.0 _ll_register_callbacks cb 0x7f788c0f3330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.865+0000 7f789be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d824c090 0x55a0d825a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.865+0000 7f78a1000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d8251a60 0x55a0d82ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.865+0000 7f789be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d824c090 0x55a0d825a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:47838/0 (socket says 172.21.15.89:47838) 2024-09-10T08:40:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.865+0000 7f789be00640 1 -- 172.21.15.89:0/2628686281 learned_addr learned my addr 172.21.15.89:0/2628686281 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.865+0000 7f7893e00640 1 --2- 172.21.15.89:0/2628686281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d824c9d0 0x55a0d825a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.866+0000 7f7893e00640 1 -- 172.21.15.89:0/2628686281 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d824c090 msgr2=0x55a0d825a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.866+0000 7f7893e00640 1 --2- 172.21.15.89:0/2628686281 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d824c090 0x55a0d825a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.866+0000 7f7893e00640 1 -- 172.21.15.89:0/2628686281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d8251a60 msgr2=0x55a0d82ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.866+0000 7f7893e00640 1 --2- 172.21.15.89:0/2628686281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d8251a60 0x55a0d82ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.866+0000 7f7893e00640 1 -- 172.21.15.89:0/2628686281 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d825af60 con 0x55a0d824c9d0 2024-09-10T08:40:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.866+0000 7f78a1000640 1 --2- 172.21.15.89:0/2628686281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d8251a60 0x55a0d82ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.866+0000 7f789be00640 1 --2- 172.21.15.89:0/2628686281 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d824c090 0x55a0d825a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.866+0000 7f7893e00640 1 --2- 172.21.15.89:0/2628686281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d824c9d0 0x55a0d825a920 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55a0d824d690 tx=0x7f7884005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bab96fe35dde8efc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.866+0000 7f788a000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:40:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.866+0000 7f788a000640 1 -- 172.21.15.89:0/2628686281 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78840037f0 con 0x55a0d824c9d0 2024-09-10T08:40:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.866+0000 7f78a605a740 1 -- 172.21.15.89:0/2628686281 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78840054e0 con 0x55a0d824c9d0 2024-09-10T08:40:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.866+0000 7f788a000640 1 -- 172.21.15.89:0/2628686281 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f7884003990 con 0x55a0d824c9d0 2024-09-10T08:40:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.867+0000 7f788a000640 1 -- 172.21.15.89:0/2628686281 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7884003b90 con 0x55a0d824c9d0 2024-09-10T08:40:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.867+0000 7f78a605a740 10 client.25403 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.867+0000 7f78a605a740 1 -- 172.21.15.89:0/2628686281 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a0d82cbf90 con 0x55a0d824c9d0 2024-09-10T08:40:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.867+0000 7f788a000640 1 -- 172.21.15.89:0/2628686281 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78840054e0 con 0x55a0d824c9d0 2024-09-10T08:40:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.867+0000 7f788a000640 1 -- 172.21.15.89:0/2628686281 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f788401b410 con 0x55a0d824c9d0 2024-09-10T08:40:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.867+0000 7f78a605a740 10 client.25403 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.867+0000 7f78a605a740 10 client.25403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:40:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.867+0000 7f78a605a740 20 client.25403 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.867+0000 7f78a605a740 1 --2- 172.21.15.89:0/2628686281 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a0d82cc690 0x55a0d82cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.867+0000 7f78a605a740 4 client.25403 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:40:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.867+0000 7f78a605a740 1 -- 172.21.15.89:0/2628686281 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0d81b8b80 con 0x55a0d82cc690 2024-09-10T08:40:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.867+0000 7f789be00640 1 --2- 172.21.15.89:0/2628686281 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a0d82cc690 0x55a0d82cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.868+0000 7f789be00640 1 --2- 172.21.15.89:0/2628686281 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a0d82cc690 0x55a0d82cea80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9037d6cd43119f61 server_cookie=3fdc8b4a510827d7 in_seq=0 out_seq=0 2024-09-10T08:40:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.868+0000 7f788a000640 10 client.25403 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.875+0000 7f788a000640 1 -- 172.21.15.89:0/2628686281 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0d81b8b80 con 0x55a0d82cc690 2024-09-10T08:40:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.875+0000 7f788a000640 10 client.25403 handle_command_reply: tid=0 2024-09-10T08:40:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.875+0000 7f78a605a740 1 client.25403 shutdown 2024-09-10T08:40:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.876+0000 7f78a605a740 1 -- 172.21.15.89:0/2628686281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d824c9d0 msgr2=0x55a0d825a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.876+0000 7f78a605a740 1 --2- 172.21.15.89:0/2628686281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d824c9d0 0x55a0d825a920 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55a0d824d690 tx=0x7f7884005720 comp rx=0 tx=0).stop 2024-09-10T08:40:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.876+0000 7f78a605a740 1 -- 172.21.15.89:0/2628686281 shutdown_connections 2024-09-10T08:40:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.876+0000 7f78a605a740 1 --2- 172.21.15.89:0/2628686281 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a0d82cc690 0x55a0d82cea80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.876+0000 7f78a605a740 1 --2- 172.21.15.89:0/2628686281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a0d8251a60 0x55a0d82ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.876+0000 7f788a000640 0 client.25403 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.876+0000 7f78a605a740 1 --2- 172.21.15.89:0/2628686281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0d824c9d0 0x55a0d825a920 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.876+0000 7f78a605a740 1 --2- 172.21.15.89:0/2628686281 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d824c090 0x55a0d825a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.876+0000 7f78a605a740 1 -- 172.21.15.89:0/2628686281 >> 172.21.15.89:0/2628686281 conn(0x55a0d8247700 msgr2=0x55a0d8255ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.876+0000 7f78a605a740 1 -- 172.21.15.89:0/2628686281 shutdown_connections 2024-09-10T08:40:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.876+0000 7f78a605a740 1 -- 172.21.15.89:0/2628686281 wait complete. 2024-09-10T08:40:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.876+0000 7f78a605a740 20 client.25403 trim_cache size 0 max 16384 2024-09-10T08:40:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:40:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:40:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:40:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:40:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.877+0000 7f78a3000640 1 -- 172.21.15.89:0/331641519 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f78800a2d50 msgr2=0x7f78800a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.877+0000 7f78a3000640 1 --2- 172.21.15.89:0/331641519 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f78800a2d50 0x7f78800a5210 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7894011630 tx=0x7f7894009500 comp rx=0 tx=0).stop 2024-09-10T08:40:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.877+0000 7f78a3000640 1 -- 172.21.15.89:0/331641519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f789c103d60 msgr2=0x7f789c199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.877+0000 7f78a3000640 1 --2- 172.21.15.89:0/331641519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f789c103d60 0x7f789c199fc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f788c031440 tx=0x7f788c032ce0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.877+0000 7f78a3000640 1 -- 172.21.15.89:0/331641519 shutdown_connections 2024-09-10T08:40:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.877+0000 7f78a3000640 1 --2- 172.21.15.89:0/331641519 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f78800a2d50 0x7f78800a5210 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.877+0000 7f78a3000640 1 --2- 172.21.15.89:0/331641519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f789c105910 0x7f789c1a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.877+0000 7f78a3000640 1 --2- 172.21.15.89:0/331641519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f789c104f50 0x7f789c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.877+0000 7f78a3000640 1 --2- 172.21.15.89:0/331641519 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f789c103d60 0x7f789c199fc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.877+0000 7f78a3000640 1 -- 172.21.15.89:0/331641519 >> 172.21.15.89:0/331641519 conn(0x7f789c0ff4e0 msgr2=0x7f789c108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.877+0000 7f78a3000640 1 -- 172.21.15.89:0/331641519 shutdown_connections 2024-09-10T08:40:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:15.877+0000 7f78a3000640 1 -- 172.21.15.89:0/331641519 wait complete. 2024-09-10T08:40:16.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:16 smithi149 ceph-mon[36167]: pgmap v403: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-09-10T08:40:17.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:16 smithi111 ceph-mon[35220]: pgmap v403: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-09-10T08:40:17.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:16 smithi089 ceph-mon[30757]: pgmap v403: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-09-10T08:40:18.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:18 smithi149 ceph-mon[36167]: pgmap v404: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-10T08:40:19.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:18 smithi111 ceph-mon[35220]: pgmap v404: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-10T08:40:19.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:18 smithi089 ceph-mon[30757]: pgmap v404: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-10T08:40:20.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:20 smithi149 ceph-mon[36167]: pgmap v405: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-10T08:40:21.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:20 smithi111 ceph-mon[35220]: pgmap v405: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-10T08:40:21.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:20 smithi089 ceph-mon[30757]: pgmap v405: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-10T08:40:22.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:22 smithi149 ceph-mon[36167]: pgmap v406: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-10T08:40:23.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:22 smithi111 ceph-mon[35220]: pgmap v406: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-10T08:40:23.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:22 smithi089 ceph-mon[30757]: pgmap v406: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-09-10T08:40:24.180 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:40:24.570 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:40:24.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:24 smithi149 ceph-mon[36167]: pgmap v407: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-10T08:40:25.015 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:24 smithi089 ceph-mon[30757]: pgmap v407: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-10T08:40:25.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:24 smithi111 ceph-mon[35220]: pgmap v407: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-10T08:40:25.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.129+0000 7f02f3200640 1 -- 172.21.15.89:0/1185136030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02f410e280 msgr2=0x7f02f4110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:25.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.129+0000 7f02f3200640 1 --2- 172.21.15.89:0/1185136030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02f410e280 0x7f02f4110670 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f02ec009ca0 tx=0x7f02ec02a950 comp rx=0 tx=0).stop 2024-09-10T08:40:25.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.129+0000 7f02f3200640 1 -- 172.21.15.89:0/1185136030 shutdown_connections 2024-09-10T08:40:25.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.129+0000 7f02f3200640 1 --2- 172.21.15.89:0/1185136030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02f410e280 0x7f02f4110670 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.129+0000 7f02f3200640 1 --2- 172.21.15.89:0/1185136030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02f4074e30 0x7f02f410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.129+0000 7f02f3200640 1 --2- 172.21.15.89:0/1185136030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02f4074510 0x7f02f40748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.129+0000 7f02f3200640 1 -- 172.21.15.89:0/1185136030 >> 172.21.15.89:0/1185136030 conn(0x7f02f40fd740 msgr2=0x7f02f40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:25.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.130+0000 7f02f3200640 1 -- 172.21.15.89:0/1185136030 shutdown_connections 2024-09-10T08:40:25.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.130+0000 7f02f3200640 1 -- 172.21.15.89:0/1185136030 wait complete. 2024-09-10T08:40:25.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.131+0000 7f02f3200640 1 Processor -- start 2024-09-10T08:40:25.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.131+0000 7f02f3200640 1 -- start start 2024-09-10T08:40:25.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.131+0000 7f02f3200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02f4074510 0x7f02f419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:25.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.131+0000 7f02f3200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02f4074e30 0x7f02f419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:25.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.131+0000 7f02f3200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02f410e280 0x7f02f419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:25.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.131+0000 7f02f3200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4114000 con 0x7f02f4074510 2024-09-10T08:40:25.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.131+0000 7f02f3200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4113e80 con 0x7f02f4074e30 2024-09-10T08:40:25.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.131+0000 7f02f3200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4114180 con 0x7f02f410e280 2024-09-10T08:40:25.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.131+0000 7f02f1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02f4074510 0x7f02f419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:25.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.131+0000 7f02f1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02f4074510 0x7f02f419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59208/0 (socket says 172.21.15.89:59208) 2024-09-10T08:40:25.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.131+0000 7f02f1e00640 1 -- 172.21.15.89:0/2608372410 learned_addr learned my addr 172.21.15.89:0/2608372410 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:25.148 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.131+0000 7f02f2800640 1 --2- 172.21.15.89:0/2608372410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02f410e280 0x7f02f419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:25.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.132+0000 7f02f1400640 1 --2- 172.21.15.89:0/2608372410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02f4074e30 0x7f02f419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:25.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.132+0000 7f02f1e00640 1 -- 172.21.15.89:0/2608372410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02f410e280 msgr2=0x7f02f419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:25.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.132+0000 7f02f1e00640 1 --2- 172.21.15.89:0/2608372410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02f410e280 0x7f02f419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.132+0000 7f02f1e00640 1 -- 172.21.15.89:0/2608372410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02f4074e30 msgr2=0x7f02f419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:25.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.132+0000 7f02f1e00640 1 --2- 172.21.15.89:0/2608372410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02f4074e30 0x7f02f419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.132+0000 7f02f1e00640 1 -- 172.21.15.89:0/2608372410 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f419f3f0 con 0x7f02f4074510 2024-09-10T08:40:25.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.132+0000 7f02f1e00640 1 --2- 172.21.15.89:0/2608372410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02f4074510 0x7f02f419a440 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f02e400e9f0 tx=0x7f02e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:25.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.132+0000 7f02eaa00640 1 -- 172.21.15.89:0/2608372410 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e400cd80 con 0x7f02f4074510 2024-09-10T08:40:25.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.132+0000 7f02eaa00640 1 -- 172.21.15.89:0/2608372410 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e4010910 con 0x7f02f4074510 2024-09-10T08:40:25.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.132+0000 7f02eaa00640 1 -- 172.21.15.89:0/2608372410 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e4018990 con 0x7f02f4074510 2024-09-10T08:40:25.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.133+0000 7f02f3200640 1 -- 172.21.15.89:0/2608372410 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f419f6e0 con 0x7f02f4074510 2024-09-10T08:40:25.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.133+0000 7f02f3200640 1 -- 172.21.15.89:0/2608372410 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f41a6fc0 con 0x7f02f4074510 2024-09-10T08:40:25.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.137+0000 7f02fb85f740 1 Processor -- start 2024-09-10T08:40:25.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.137+0000 7f02fb85f740 1 -- start start 2024-09-10T08:40:25.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.137+0000 7f02eaa00640 1 -- 172.21.15.89:0/2608372410 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f02e4010ac0 con 0x7f02f4074510 2024-09-10T08:40:25.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.138+0000 7f02fb85f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff3d50 0x55bb5cff4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:25.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.138+0000 7f02fb85f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cff4670 0x55bb5cffe9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:25.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.138+0000 7f02fb85f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cffef30 0x55bb5d001320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:25.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.138+0000 7f02fb85f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bb5cf5a3e0 con 0x55bb5cffef30 2024-09-10T08:40:25.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.138+0000 7f02fb85f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55bb5cf01d80 con 0x55bb5cff4670 2024-09-10T08:40:25.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.138+0000 7f02fb85f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bb5cf563a0 con 0x55bb5cff3d50 2024-09-10T08:40:25.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.138+0000 7f02f1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cff4670 0x55bb5cffe9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:25.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.138+0000 7f02f1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cff4670 0x55bb5cffe9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35738/0 (socket says 172.21.15.89:35738) 2024-09-10T08:40:25.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.138+0000 7f02f1e00640 1 -- 172.21.15.89:0/837712190 learned_addr learned my addr 172.21.15.89:0/837712190 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:25.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.138+0000 7f02f1400640 1 --2- 172.21.15.89:0/837712190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cffef30 0x55bb5d001320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:25.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.138+0000 7f02f2800640 1 --2- 172.21.15.89:0/837712190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff3d50 0x55bb5cff4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:25.150 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.138+0000 7f02f1e00640 1 -- 172.21.15.89:0/837712190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff3d50 msgr2=0x55bb5cff4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:25.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.138+0000 7f02f1e00640 1 --2- 172.21.15.89:0/837712190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff3d50 0x55bb5cff4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.138+0000 7f02f1e00640 1 -- 172.21.15.89:0/837712190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cffef30 msgr2=0x55bb5d001320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:25.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.138+0000 7f02f1e00640 1 --2- 172.21.15.89:0/837712190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cffef30 0x55bb5d001320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.138+0000 7f02f1e00640 1 -- 172.21.15.89:0/837712190 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb5cf20ae0 con 0x55bb5cff4670 2024-09-10T08:40:25.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.139+0000 7f02f2800640 1 --2- 172.21.15.89:0/837712190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff3d50 0x55bb5cff4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:40:25.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.139+0000 7f02eaa00640 1 --2- 172.21.15.89:0/2608372410 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f02d80a2c10 0x7f02d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:25.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.139+0000 7f02f1e00640 1 --2- 172.21.15.89:0/837712190 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cff4670 0x55bb5cffe9f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f02e40ccdd0 tx=0x7f02e40d7670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=366a5abca72dacc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:25.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.139+0000 7f02eaa00640 1 -- 172.21.15.89:0/2608372410 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f02e40cf8b0 con 0x7f02f4074510 2024-09-10T08:40:25.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.139+0000 7f02df000640 1 -- 172.21.15.89:0/837712190 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e4097f30 con 0x55bb5cff4670 2024-09-10T08:40:25.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.139+0000 7f02df000640 1 -- 172.21.15.89:0/837712190 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e409bc90 con 0x55bb5cff4670 2024-09-10T08:40:25.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.139+0000 7f02df000640 1 -- 172.21.15.89:0/837712190 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e409b5f0 con 0x55bb5cff4670 2024-09-10T08:40:25.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.139+0000 7f02f2800640 1 --2- 172.21.15.89:0/2608372410 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f02d80a2c10 0x7f02d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:25.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.140+0000 7f02f2800640 1 --2- 172.21.15.89:0/2608372410 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f02d80a2c10 0x7f02d80a50d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f02ec004af0 tx=0x7f02ec005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:25.151 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.140+0000 7f02fb85f740 1 -- 172.21.15.89:0/837712190 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cff4670 msgr2=0x55bb5cffe9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:25.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.140+0000 7f02fb85f740 1 --2- 172.21.15.89:0/837712190 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cff4670 0x55bb5cffe9f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f02e40ccdd0 tx=0x7f02e40d7670 comp rx=0 tx=0).stop 2024-09-10T08:40:25.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.140+0000 7f02fb85f740 1 -- 172.21.15.89:0/837712190 shutdown_connections 2024-09-10T08:40:25.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.140+0000 7f02fb85f740 1 --2- 172.21.15.89:0/837712190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cffef30 0x55bb5d001320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.140+0000 7f02fb85f740 1 --2- 172.21.15.89:0/837712190 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cff4670 0x55bb5cffe9f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.140+0000 7f02fb85f740 1 --2- 172.21.15.89:0/837712190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff3d50 0x55bb5cff4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.140+0000 7f02fb85f740 1 -- 172.21.15.89:0/837712190 >> 172.21.15.89:0/837712190 conn(0x55bb5cff02e0 msgr2=0x55bb5cff06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:25.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.140+0000 7f02fb85f740 1 -- 172.21.15.89:0/837712190 shutdown_connections 2024-09-10T08:40:25.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.140+0000 7f02fb85f740 1 -- 172.21.15.89:0/837712190 wait complete. 2024-09-10T08:40:25.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.141+0000 7f02fb85f740 1 -- start start 2024-09-10T08:40:25.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.141+0000 7f02fb85f740 1 Processor -- start 2024-09-10T08:40:25.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.141+0000 7f02fb85f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff3d50 0x55bb5cffbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:25.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.141+0000 7f02fb85f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cff4670 0x55bb5cffc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:25.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.141+0000 7f02fb85f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cffef30 0x55bb5cffc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:25.152 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.141+0000 7f02fb85f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bb5d001ba0 con 0x55bb5cff4670 2024-09-10T08:40:25.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.141+0000 7f02fb85f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55bb5cf150f0 con 0x55bb5cffef30 2024-09-10T08:40:25.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.141+0000 7f02fb85f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bb5d001d20 con 0x55bb5cff3d50 2024-09-10T08:40:25.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.142+0000 7f02fb85f740 10 client.0 _ll_register_callbacks cb 0x55bb5cf069e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:25.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.142+0000 7f02f1400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cff4670 0x55bb5cffc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:25.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.142+0000 7f02f1e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff3d50 0x55bb5cffbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:25.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.142+0000 7f02f1400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cff4670 0x55bb5cffc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59226/0 (socket says 172.21.15.89:59226) 2024-09-10T08:40:25.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.142+0000 7f02f1400640 1 -- 172.21.15.89:0/3598331490 learned_addr learned my addr 172.21.15.89:0/3598331490 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:25.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.142+0000 7f02f2800640 1 --2- 172.21.15.89:0/3598331490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cffef30 0x55bb5cffc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:25.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.142+0000 7f02f1e00640 1 -- 172.21.15.89:0/3598331490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cffef30 msgr2=0x55bb5cffc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:25.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.142+0000 7f02f1e00640 1 --2- 172.21.15.89:0/3598331490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cffef30 0x55bb5cffc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.142+0000 7f02f1e00640 1 -- 172.21.15.89:0/3598331490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cff4670 msgr2=0x55bb5cffc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:25.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.142+0000 7f02f1e00640 1 --2- 172.21.15.89:0/3598331490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cff4670 0x55bb5cffc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.142+0000 7f02f1e00640 1 -- 172.21.15.89:0/3598331490 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb5cccb3c0 con 0x55bb5cff3d50 2024-09-10T08:40:25.153 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.142+0000 7f02f1400640 1 --2- 172.21.15.89:0/3598331490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cff4670 0x55bb5cffc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:25.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.143+0000 7f02f1e00640 1 --2- 172.21.15.89:0/3598331490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff3d50 0x55bb5cffbe10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55bb5cff1140 tx=0x7f02e400cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88bade43a5c321ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:25.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.143+0000 7f02d3400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:40:25.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.143+0000 7f02d3400640 1 -- 172.21.15.89:0/3598331490 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e409b7d0 con 0x55bb5cff3d50 2024-09-10T08:40:25.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.143+0000 7f02d3400640 1 -- 172.21.15.89:0/3598331490 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f02e409bef0 con 0x55bb5cff3d50 2024-09-10T08:40:25.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.143+0000 7f02fb85f740 1 -- 172.21.15.89:0/3598331490 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02e409bc90 con 0x55bb5cff3d50 2024-09-10T08:40:25.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.143+0000 7f02d3400640 1 -- 172.21.15.89:0/3598331490 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f02e409ba60 con 0x55bb5cff3d50 2024-09-10T08:40:25.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.143+0000 7f02fb85f740 10 client.25387 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:25.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.143+0000 7f02fb85f740 1 -- 172.21.15.89:0/3598331490 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bb5d001ba0 con 0x55bb5cff3d50 2024-09-10T08:40:25.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.144+0000 7f02d3400640 1 -- 172.21.15.89:0/3598331490 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e409bc90 con 0x55bb5cff3d50 2024-09-10T08:40:25.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.144+0000 7f02d3400640 1 -- 172.21.15.89:0/3598331490 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f02e40a43c0 con 0x55bb5cff3d50 2024-09-10T08:40:25.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.144+0000 7f02fb85f740 10 client.25387 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:25.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.144+0000 7f02fb85f740 10 client.25387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:40:25.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.144+0000 7f02fb85f740 20 client.25387 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:25.154 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.144+0000 7f02fb85f740 1 --2- 172.21.15.89:0/3598331490 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bb5d00e9d0 0x55bb5d010dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:25.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.144+0000 7f02fb85f740 4 client.25387 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:40:25.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.144+0000 7f02fb85f740 1 -- 172.21.15.89:0/3598331490 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb5cf67950 con 0x55bb5d00e9d0 2024-09-10T08:40:25.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.145+0000 7f02f1400640 1 --2- 172.21.15.89:0/3598331490 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bb5d00e9d0 0x55bb5d010dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:25.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.145+0000 7f02f1400640 1 --2- 172.21.15.89:0/3598331490 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bb5d00e9d0 0x55bb5d010dc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=258be57ef0e0fd86 server_cookie=22c25902ffbaead2 in_seq=0 out_seq=0 2024-09-10T08:40:25.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.145+0000 7f02d3400640 10 client.25387 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:25.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.146+0000 7f02d3400640 1 -- 172.21.15.89:0/3598331490 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb5cf67950 con 0x55bb5d00e9d0 2024-09-10T08:40:25.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.146+0000 7f02d3400640 10 client.25387 handle_command_reply: tid=0 2024-09-10T08:40:25.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.146+0000 7f02fb85f740 1 client.25387 shutdown 2024-09-10T08:40:25.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.146+0000 7f02fb85f740 1 -- 172.21.15.89:0/3598331490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff3d50 msgr2=0x55bb5cffbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:25.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.146+0000 7f02fb85f740 1 --2- 172.21.15.89:0/3598331490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff3d50 0x55bb5cffbe10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55bb5cff1140 tx=0x7f02e400cf90 comp rx=0 tx=0).stop 2024-09-10T08:40:25.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.146+0000 7f02fb85f740 1 -- 172.21.15.89:0/3598331490 shutdown_connections 2024-09-10T08:40:25.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.146+0000 7f02fb85f740 1 --2- 172.21.15.89:0/3598331490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cffef30 0x55bb5cffc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.146+0000 7f02fb85f740 1 --2- 172.21.15.89:0/3598331490 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bb5d00e9d0 0x55bb5d010dc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.146+0000 7f02fb85f740 1 --2- 172.21.15.89:0/3598331490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cff4670 0x55bb5cffc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.146+0000 7f02fb85f740 1 --2- 172.21.15.89:0/3598331490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff3d50 0x55bb5cffbe10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.146+0000 7f02fb85f740 1 -- 172.21.15.89:0/3598331490 >> 172.21.15.89:0/3598331490 conn(0x55bb5cff02e0 msgr2=0x55bb5cff5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:25.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.146+0000 7f02d3400640 0 client.25387 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:25.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.146+0000 7f02fb85f740 1 -- 172.21.15.89:0/3598331490 shutdown_connections 2024-09-10T08:40:25.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.146+0000 7f02fb85f740 1 -- 172.21.15.89:0/3598331490 wait complete. 2024-09-10T08:40:25.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.147+0000 7f02fb85f740 20 client.25387 trim_cache size 0 max 16384 2024-09-10T08:40:25.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.168+0000 7f02fb85f740 1 Processor -- start 2024-09-10T08:40:25.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.168+0000 7f02fb85f740 1 -- start start 2024-09-10T08:40:25.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.169+0000 7f02fb85f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff3d50 0x55bb5d016500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:25.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.169+0000 7f02fb85f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cff4670 0x55bb5d016a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:25.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.169+0000 7f02fb85f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cffef30 0x55bb5cff9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:25.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.169+0000 7f02fb85f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41b3730 con 0x55bb5cffef30 2024-09-10T08:40:25.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.169+0000 7f02fb85f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41b1000 con 0x55bb5cff4670 2024-09-10T08:40:25.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.169+0000 7f02fb85f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41aee50 con 0x55bb5cff3d50 2024-09-10T08:40:25.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.169+0000 7f02f1e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff3d50 0x55bb5d016500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:25.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.169+0000 7f02f1400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cff4670 0x55bb5d016a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:25.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.169+0000 7f02f2800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cffef30 0x55bb5cff9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:25.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.169+0000 7f02f1e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff3d50 0x55bb5d016500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44444/0 (socket says 172.21.15.89:44444) 2024-09-10T08:40:25.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.169+0000 7f02f1400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cff4670 0x55bb5d016a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35756/0 (socket says 172.21.15.89:35756) 2024-09-10T08:40:25.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.169+0000 7f02f1e00640 1 -- 172.21.15.89:0/665571251 learned_addr learned my addr 172.21.15.89:0/665571251 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:25.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.170+0000 7f02f1e00640 1 -- 172.21.15.89:0/665571251 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cff4670 msgr2=0x55bb5d016a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:25.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.170+0000 7f02f1e00640 1 --2- 172.21.15.89:0/665571251 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cff4670 0x55bb5d016a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.170+0000 7f02f1e00640 1 -- 172.21.15.89:0/665571251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cffef30 msgr2=0x55bb5cff9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:25.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.170+0000 7f02f1e00640 1 --2- 172.21.15.89:0/665571251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cffef30 0x55bb5cff9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.170+0000 7f02f1e00640 1 -- 172.21.15.89:0/665571251 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb5cfefb70 con 0x55bb5cff3d50 2024-09-10T08:40:25.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.170+0000 7f02f1e00640 1 --2- 172.21.15.89:0/665571251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff3d50 0x55bb5d016500 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f02e40cfc20 tx=0x7f02e409a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78e3fc7f9c21137 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:25.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.170+0000 7f02de600640 1 -- 172.21.15.89:0/665571251 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e409b7d0 con 0x55bb5cff3d50 2024-09-10T08:40:25.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.170+0000 7f02de600640 1 -- 172.21.15.89:0/665571251 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e409bc90 con 0x55bb5cff3d50 2024-09-10T08:40:25.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.170+0000 7f02de600640 1 -- 172.21.15.89:0/665571251 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e4097f30 con 0x55bb5cff3d50 2024-09-10T08:40:25.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.171+0000 7f02fb85f740 1 -- 172.21.15.89:0/665571251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff3d50 msgr2=0x55bb5d016500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:25.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.171+0000 7f02fb85f740 1 --2- 172.21.15.89:0/665571251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff3d50 0x55bb5d016500 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f02e40cfc20 tx=0x7f02e409a1a0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.171+0000 7f02fb85f740 1 -- 172.21.15.89:0/665571251 shutdown_connections 2024-09-10T08:40:25.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.171+0000 7f02fb85f740 1 --2- 172.21.15.89:0/665571251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cffef30 0x55bb5cff9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.171+0000 7f02fb85f740 1 --2- 172.21.15.89:0/665571251 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cff4670 0x55bb5d016a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.171+0000 7f02fb85f740 1 --2- 172.21.15.89:0/665571251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff3d50 0x55bb5d016500 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.171+0000 7f02fb85f740 1 -- 172.21.15.89:0/665571251 >> 172.21.15.89:0/665571251 conn(0x55bb5cff02e0 msgr2=0x55bb5cff5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:25.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.171+0000 7f02fb85f740 1 -- 172.21.15.89:0/665571251 shutdown_connections 2024-09-10T08:40:25.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.171+0000 7f02fb85f740 1 -- 172.21.15.89:0/665571251 wait complete. 2024-09-10T08:40:25.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.172+0000 7f02fb85f740 1 -- start start 2024-09-10T08:40:25.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.172+0000 7f02fb85f740 1 Processor -- start 2024-09-10T08:40:25.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.172+0000 7f02fb85f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cff3d50 0x55bb5d0024a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:25.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.172+0000 7f02fb85f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff4670 0x55bb5d0029e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:25.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.172+0000 7f02fb85f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cffef30 0x55bb5d0774c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:25.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.172+0000 7f02fb85f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41af4c0 con 0x55bb5cffef30 2024-09-10T08:40:25.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.172+0000 7f02fb85f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41b0930 con 0x55bb5cff3d50 2024-09-10T08:40:25.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.172+0000 7f02fb85f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41afb90 con 0x55bb5cff4670 2024-09-10T08:40:25.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.172+0000 7f02fb85f740 10 client.0 _ll_register_callbacks cb 0x7f02e40cfb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:25.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.173+0000 7f02f1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cff3d50 0x55bb5d0024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:25.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.173+0000 7f02f1400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff4670 0x55bb5d0029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:25.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.173+0000 7f02f1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cff3d50 0x55bb5d0024a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35770/0 (socket says 172.21.15.89:35770) 2024-09-10T08:40:25.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.173+0000 7f02f1e00640 1 -- 172.21.15.89:0/2146523959 learned_addr learned my addr 172.21.15.89:0/2146523959 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:25.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.173+0000 7f02f2800640 1 --2- 172.21.15.89:0/2146523959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cffef30 0x55bb5d0774c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:25.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.173+0000 7f02f1e00640 1 -- 172.21.15.89:0/2146523959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff4670 msgr2=0x55bb5d0029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:25.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.173+0000 7f02f1e00640 1 --2- 172.21.15.89:0/2146523959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff4670 0x55bb5d0029e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.173+0000 7f02f1e00640 1 -- 172.21.15.89:0/2146523959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cffef30 msgr2=0x55bb5d0774c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:25.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.173+0000 7f02f1e00640 1 --2- 172.21.15.89:0/2146523959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cffef30 0x55bb5d0774c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.173+0000 7f02f1e00640 1 -- 172.21.15.89:0/2146523959 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb5d077ab0 con 0x55bb5cff3d50 2024-09-10T08:40:25.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.173+0000 7f02f1e00640 1 --2- 172.21.15.89:0/2146523959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cff3d50 0x55bb5d0024a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f02e4009380 tx=0x7f02e4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37cd172b9424b19c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:25.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.173+0000 7f02d3400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:40:25.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.174+0000 7f02d3400640 1 -- 172.21.15.89:0/2146523959 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e409b5f0 con 0x55bb5cff3d50 2024-09-10T08:40:25.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.174+0000 7f02d3400640 1 -- 172.21.15.89:0/2146523959 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f02e409b2e0 con 0x55bb5cff3d50 2024-09-10T08:40:25.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.174+0000 7f02fb85f740 1 -- 172.21.15.89:0/2146523959 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02e409bc90 con 0x55bb5cff3d50 2024-09-10T08:40:25.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.174+0000 7f02d3400640 1 -- 172.21.15.89:0/2146523959 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f02e40d7ea0 con 0x55bb5cff3d50 2024-09-10T08:40:25.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.174+0000 7f02fb85f740 10 client.25421 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:25.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.174+0000 7f02fb85f740 1 -- 172.21.15.89:0/2146523959 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bb5d077ab0 con 0x55bb5cff3d50 2024-09-10T08:40:25.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.174+0000 7f02d3400640 1 -- 172.21.15.89:0/2146523959 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e409bc90 con 0x55bb5cff3d50 2024-09-10T08:40:25.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.174+0000 7f02d3400640 1 -- 172.21.15.89:0/2146523959 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f02e40ba350 con 0x55bb5cff3d50 2024-09-10T08:40:25.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.174+0000 7f02fb85f740 10 client.25421 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:25.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.174+0000 7f02fb85f740 10 client.25421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:40:25.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.174+0000 7f02fb85f740 20 client.25421 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:25.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.174+0000 7f02fb85f740 1 --2- 172.21.15.89:0/2146523959 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bb5d078ff0 0x55bb5d07b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:25.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.174+0000 7f02fb85f740 4 client.25421 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:40:25.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.174+0000 7f02fb85f740 1 -- 172.21.15.89:0/2146523959 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bb5cf61e60 con 0x55bb5d078ff0 2024-09-10T08:40:25.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.175+0000 7f02f1400640 1 --2- 172.21.15.89:0/2146523959 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bb5d078ff0 0x55bb5d07b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:25.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.175+0000 7f02f1400640 1 --2- 172.21.15.89:0/2146523959 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bb5d078ff0 0x55bb5d07b3e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74c0e4217480d630 server_cookie=c6abc7ec54f8ea7e in_seq=0 out_seq=0 2024-09-10T08:40:25.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.175+0000 7f02d3400640 10 client.25421 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:25.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.176+0000 7f02d3400640 1 -- 172.21.15.89:0/2146523959 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bb5cf61e60 con 0x55bb5d078ff0 2024-09-10T08:40:25.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.176+0000 7f02d3400640 10 client.25421 handle_command_reply: tid=0 2024-09-10T08:40:25.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.176+0000 7f02fb85f740 1 client.25421 shutdown 2024-09-10T08:40:25.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.176+0000 7f02fb85f740 1 -- 172.21.15.89:0/2146523959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cff3d50 msgr2=0x55bb5d0024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:25.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.176+0000 7f02fb85f740 1 --2- 172.21.15.89:0/2146523959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cff3d50 0x55bb5d0024a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f02e4009380 tx=0x7f02e4000bc0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.176+0000 7f02fb85f740 1 -- 172.21.15.89:0/2146523959 shutdown_connections 2024-09-10T08:40:25.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.176+0000 7f02fb85f740 1 --2- 172.21.15.89:0/2146523959 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bb5d078ff0 0x55bb5d07b3e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.176+0000 7f02fb85f740 1 --2- 172.21.15.89:0/2146523959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bb5cffef30 0x55bb5d0774c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.176+0000 7f02fb85f740 1 --2- 172.21.15.89:0/2146523959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb5cff4670 0x55bb5d0029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.176+0000 7f02fb85f740 1 --2- 172.21.15.89:0/2146523959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bb5cff3d50 0x55bb5d0024a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.176+0000 7f02d3400640 0 client.25421 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:25.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.176+0000 7f02fb85f740 1 -- 172.21.15.89:0/2146523959 >> 172.21.15.89:0/2146523959 conn(0x55bb5cff02e0 msgr2=0x55bb5cffa6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:25.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.176+0000 7f02fb85f740 1 -- 172.21.15.89:0/2146523959 shutdown_connections 2024-09-10T08:40:25.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.176+0000 7f02fb85f740 1 -- 172.21.15.89:0/2146523959 wait complete. 2024-09-10T08:40:25.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.176+0000 7f02fb85f740 20 client.25421 trim_cache size 0 max 16384 2024-09-10T08:40:25.180 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:40:25.180 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2024-09-10T08:40:25.180 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2024-09-10T08:40:25.181 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:40:25.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.177+0000 7f02f3200640 1 -- 172.21.15.89:0/2608372410 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f02d80a2c10 msgr2=0x7f02d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:25.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.177+0000 7f02f3200640 1 --2- 172.21.15.89:0/2608372410 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f02d80a2c10 0x7f02d80a50d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f02ec004af0 tx=0x7f02ec005d80 comp rx=0 tx=0).stop 2024-09-10T08:40:25.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.177+0000 7f02f3200640 1 -- 172.21.15.89:0/2608372410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02f4074510 msgr2=0x7f02f419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:25.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.177+0000 7f02f3200640 1 --2- 172.21.15.89:0/2608372410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02f4074510 0x7f02f419a440 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f02e400e9f0 tx=0x7f02e400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.177+0000 7f02f3200640 1 -- 172.21.15.89:0/2608372410 shutdown_connections 2024-09-10T08:40:25.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.177+0000 7f02f3200640 1 --2- 172.21.15.89:0/2608372410 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f02d80a2c10 0x7f02d80a50d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.177+0000 7f02f3200640 1 --2- 172.21.15.89:0/2608372410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02f410e280 0x7f02f419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.177+0000 7f02f3200640 1 --2- 172.21.15.89:0/2608372410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02f4074e30 0x7f02f419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.177+0000 7f02f3200640 1 --2- 172.21.15.89:0/2608372410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02f4074510 0x7f02f419a440 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:25.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.177+0000 7f02f3200640 1 -- 172.21.15.89:0/2608372410 >> 172.21.15.89:0/2608372410 conn(0x7f02f40fd740 msgr2=0x7f02f40fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:25.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.177+0000 7f02f3200640 1 -- 172.21.15.89:0/2608372410 shutdown_connections 2024-09-10T08:40:25.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:25.177+0000 7f02f3200640 1 -- 172.21.15.89:0/2608372410 wait complete. 2024-09-10T08:40:25.998 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:40:25.998 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:40:26.395 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:40:26.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:26 smithi089 ceph-mon[30757]: pgmap v408: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 937 B/s wr, 8 op/s 2024-09-10T08:40:26.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.965+0000 7f5639200640 1 -- 172.21.15.89:0/2576779931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56341019c0 msgr2=0x7f5634101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:26.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.965+0000 7f5639200640 1 --2- 172.21.15.89:0/2576779931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56341019c0 0x7f5634101e20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5620009860 tx=0x7f562002a340 comp rx=0 tx=0).stop 2024-09-10T08:40:26.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.966+0000 7f5639200640 1 -- 172.21.15.89:0/2576779931 shutdown_connections 2024-09-10T08:40:26.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.966+0000 7f5639200640 1 --2- 172.21.15.89:0/2576779931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5634102360 0x7f563410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:26.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.966+0000 7f5639200640 1 --2- 172.21.15.89:0/2576779931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56341019c0 0x7f5634101e20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:26.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.966+0000 7f5639200640 1 --2- 172.21.15.89:0/2576779931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56341079c0 0x7f5634107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:26.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.966+0000 7f5639200640 1 -- 172.21.15.89:0/2576779931 >> 172.21.15.89:0/2576779931 conn(0x7f56340fd740 msgr2=0x7f56340ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:26.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.966+0000 7f5639200640 1 -- 172.21.15.89:0/2576779931 shutdown_connections 2024-09-10T08:40:26.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.966+0000 7f5639200640 1 -- 172.21.15.89:0/2576779931 wait complete. 2024-09-10T08:40:26.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.967+0000 7f5639200640 1 Processor -- start 2024-09-10T08:40:26.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.967+0000 7f5639200640 1 -- start start 2024-09-10T08:40:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.968+0000 7f5639200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56341019c0 0x7f56340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.968+0000 7f5639200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5634102360 0x7f5634074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.968+0000 7f5639200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56341079c0 0x7f5634074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.968+0000 7f5639200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5634111dc0 con 0x7f56341019c0 2024-09-10T08:40:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.968+0000 7f5639200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5634111c40 con 0x7f56341079c0 2024-09-10T08:40:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.968+0000 7f5639200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5634111f40 con 0x7f5634102360 2024-09-10T08:40:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.968+0000 7f5633400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56341019c0 0x7f56340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.968+0000 7f5633e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56341079c0 0x7f5634074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.968+0000 7f5633e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56341079c0 0x7f5634074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35780/0 (socket says 172.21.15.89:35780) 2024-09-10T08:40:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.968+0000 7f5633e00640 1 -- 172.21.15.89:0/881490458 learned_addr learned my addr 172.21.15.89:0/881490458 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:26.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.968+0000 7f5632a00640 1 --2- 172.21.15.89:0/881490458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5634102360 0x7f5634074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:26.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.968+0000 7f5633e00640 1 -- 172.21.15.89:0/881490458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5634102360 msgr2=0x7f5634074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:26.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.968+0000 7f5633e00640 1 --2- 172.21.15.89:0/881490458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5634102360 0x7f5634074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.968+0000 7f5633e00640 1 -- 172.21.15.89:0/881490458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56341019c0 msgr2=0x7f56340744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.968+0000 7f5633e00640 1 --2- 172.21.15.89:0/881490458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56341019c0 0x7f56340744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.968+0000 7f5633e00640 1 -- 172.21.15.89:0/881490458 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56341a67a0 con 0x7f56341079c0 2024-09-10T08:40:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.968+0000 7f5632a00640 1 --2- 172.21.15.89:0/881490458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5634102360 0x7f5634074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:40:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.969+0000 7f5633e00640 1 --2- 172.21.15.89:0/881490458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56341079c0 0x7f5634074f70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f562c00eb10 tx=0x7f562c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.969+0000 7f562be00640 1 -- 172.21.15.89:0/881490458 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c00ce20 con 0x7f56341079c0 2024-09-10T08:40:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.969+0000 7f562be00640 1 -- 172.21.15.89:0/881490458 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f562c005020 con 0x7f56341079c0 2024-09-10T08:40:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.969+0000 7f562be00640 1 -- 172.21.15.89:0/881490458 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c010690 con 0x7f56341079c0 2024-09-10T08:40:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.969+0000 7f5639200640 1 -- 172.21.15.89:0/881490458 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56341a6a90 con 0x7f56341079c0 2024-09-10T08:40:26.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.969+0000 7f5639200640 1 -- 172.21.15.89:0/881490458 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56341a6f50 con 0x7f56341079c0 2024-09-10T08:40:26.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:26 smithi149 ceph-mon[36167]: pgmap v408: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 937 B/s wr, 8 op/s 2024-09-10T08:40:26.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.970+0000 7f5639200640 1 -- 172.21.15.89:0/881490458 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5634103100 con 0x7f56341079c0 2024-09-10T08:40:26.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.972+0000 7f562be00640 1 -- 172.21.15.89:0/881490458 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f562c005540 con 0x7f56341079c0 2024-09-10T08:40:26.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.974+0000 7f562be00640 1 --2- 172.21.15.89:0/881490458 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f56180a2c10 0x7f56180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:26.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.974+0000 7f562be00640 1 -- 172.21.15.89:0/881490458 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f562c0cfc40 con 0x7f56341079c0 2024-09-10T08:40:26.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.974+0000 7f5633400640 1 --2- 172.21.15.89:0/881490458 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f56180a2c10 0x7f56180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:26.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.975+0000 7f5633400640 1 --2- 172.21.15.89:0/881490458 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f56180a2c10 0x7f56180a50d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5624005fd0 tx=0x7f5624009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:26.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:26.976+0000 7f562be00640 1 -- 172.21.15.89:0/881490458 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f562c09c610 con 0x7f56341079c0 2024-09-10T08:40:27.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:26 smithi111 ceph-mon[35220]: pgmap v408: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 937 B/s wr, 8 op/s 2024-09-10T08:40:27.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:27.204+0000 7f5639200640 1 -- 172.21.15.89:0/881490458 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5634101e20 con 0x7f56341079c0 2024-09-10T08:40:27.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:27.205+0000 7f562be00640 1 -- 172.21.15.89:0/881490458 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11918 (secure 0 0 0) 0x7f5634101e20 con 0x7f56341079c0 2024-09-10T08:40:27.205 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:40:27.205 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-10T08:40:27.206 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-10T08:39:39:721431+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:39:38.834417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24518,24496]},"id":1}]} 2024-09-10T08:40:27.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:27.208+0000 7f5639200640 1 -- 172.21.15.89:0/881490458 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f56180a2c10 msgr2=0x7f56180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:27.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:27.208+0000 7f5639200640 1 --2- 172.21.15.89:0/881490458 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f56180a2c10 0x7f56180a50d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5624005fd0 tx=0x7f5624009260 comp rx=0 tx=0).stop 2024-09-10T08:40:27.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:27.208+0000 7f5639200640 1 -- 172.21.15.89:0/881490458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56341079c0 msgr2=0x7f5634074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:27.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:27.208+0000 7f5639200640 1 --2- 172.21.15.89:0/881490458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56341079c0 0x7f5634074f70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f562c00eb10 tx=0x7f562c00ef10 comp rx=0 tx=0).stop 2024-09-10T08:40:27.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:27.208+0000 7f5639200640 1 -- 172.21.15.89:0/881490458 shutdown_connections 2024-09-10T08:40:27.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:27.208+0000 7f5639200640 1 --2- 172.21.15.89:0/881490458 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f56180a2c10 0x7f56180a50d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:27.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:27.208+0000 7f5639200640 1 --2- 172.21.15.89:0/881490458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56341079c0 0x7f5634074f70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:27.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:27.208+0000 7f5639200640 1 --2- 172.21.15.89:0/881490458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5634102360 0x7f5634074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:27.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:27.208+0000 7f5639200640 1 --2- 172.21.15.89:0/881490458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56341019c0 0x7f56340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:27.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:27.208+0000 7f5639200640 1 -- 172.21.15.89:0/881490458 >> 172.21.15.89:0/881490458 conn(0x7f56340fd740 msgr2=0x7f56340fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:27.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:27.209+0000 7f5639200640 1 -- 172.21.15.89:0/881490458 shutdown_connections 2024-09-10T08:40:27.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:27.209+0000 7f5639200640 1 -- 172.21.15.89:0/881490458 wait complete. 2024-09-10T08:40:27.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:27 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/881490458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:40:27.974 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:27 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/881490458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:40:28.019 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:40:28.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:27 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/881490458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:40:28.343 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T08:40:28.344 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T08:40:28.344 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:40:28.407 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:40:28.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:28 smithi149 ceph-mon[36167]: pgmap v409: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 937 B/s wr, 8 op/s 2024-09-10T08:40:28.971 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:28 smithi089 ceph-mon[30757]: pgmap v409: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 937 B/s wr, 8 op/s 2024-09-10T08:40:29.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.002+0000 7f6027000640 1 -- 172.21.15.89:0/2844021015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60201036e0 msgr2=0x7f6020107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:29.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.002+0000 7f6027000640 1 --2- 172.21.15.89:0/2844021015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60201036e0 0x7f6020107b90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f600c009930 tx=0x7f600c02a420 comp rx=0 tx=0).stop 2024-09-10T08:40:29.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.002+0000 7f6027000640 1 -- 172.21.15.89:0/2844021015 shutdown_connections 2024-09-10T08:40:29.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.002+0000 7f6027000640 1 --2- 172.21.15.89:0/2844021015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60201082a0 0x7f602010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.002+0000 7f6027000640 1 --2- 172.21.15.89:0/2844021015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60201036e0 0x7f6020107b90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.002+0000 7f6027000640 1 --2- 172.21.15.89:0/2844021015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6020102d30 0x7f6020103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.002+0000 7f6027000640 1 -- 172.21.15.89:0/2844021015 >> 172.21.15.89:0/2844021015 conn(0x7f60200fc7d0 msgr2=0x7f60200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:29.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.002+0000 7f6027000640 1 -- 172.21.15.89:0/2844021015 shutdown_connections 2024-09-10T08:40:29.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.002+0000 7f6027000640 1 -- 172.21.15.89:0/2844021015 wait complete. 2024-09-10T08:40:29.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.003+0000 7f6027000640 1 Processor -- start 2024-09-10T08:40:29.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.003+0000 7f6027000640 1 -- start start 2024-09-10T08:40:29.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.004+0000 7f6027000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6020102d30 0x7f60201a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.004+0000 7f6027000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60201036e0 0x7f60201a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.004+0000 7f6027000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60201082a0 0x7f602019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.004+0000 7f6027000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6020112040 con 0x7f6020102d30 2024-09-10T08:40:29.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.004+0000 7f6027000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6020111ec0 con 0x7f60201082a0 2024-09-10T08:40:29.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.004+0000 7f6027000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60201121c0 con 0x7f60201036e0 2024-09-10T08:40:29.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.004+0000 7f6025000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60201082a0 0x7f602019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.004+0000 7f6025000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60201082a0 0x7f602019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35806/0 (socket says 172.21.15.89:35806) 2024-09-10T08:40:29.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.004+0000 7f6025000640 1 -- 172.21.15.89:0/1841058753 learned_addr learned my addr 172.21.15.89:0/1841058753 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:29.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.004+0000 7f601fe00640 1 --2- 172.21.15.89:0/1841058753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6020102d30 0x7f60201a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.005+0000 7f601f400640 1 --2- 172.21.15.89:0/1841058753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60201036e0 0x7f60201a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.005+0000 7f6025000640 1 -- 172.21.15.89:0/1841058753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60201036e0 msgr2=0x7f60201a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:29.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.005+0000 7f6025000640 1 --2- 172.21.15.89:0/1841058753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60201036e0 0x7f60201a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.005+0000 7f6025000640 1 -- 172.21.15.89:0/1841058753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6020102d30 msgr2=0x7f60201a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:29.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.005+0000 7f6025000640 1 --2- 172.21.15.89:0/1841058753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6020102d30 0x7f60201a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.005+0000 7f6025000640 1 -- 172.21.15.89:0/1841058753 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602019ad10 con 0x7f60201082a0 2024-09-10T08:40:29.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.005+0000 7f601fe00640 1 --2- 172.21.15.89:0/1841058753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6020102d30 0x7f60201a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:40:29.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.005+0000 7f601f400640 1 --2- 172.21.15.89:0/1841058753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60201036e0 0x7f60201a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:40:29.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.006+0000 7f6025000640 1 --2- 172.21.15.89:0/1841058753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60201082a0 0x7f602019a5e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f601800eb10 tx=0x7f601800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:29.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.006+0000 7f601cc00640 1 -- 172.21.15.89:0/1841058753 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018004b20 con 0x7f60201082a0 2024-09-10T08:40:29.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.006+0000 7f601cc00640 1 -- 172.21.15.89:0/1841058753 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6018004cc0 con 0x7f60201082a0 2024-09-10T08:40:29.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.006+0000 7f601cc00640 1 -- 172.21.15.89:0/1841058753 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018010760 con 0x7f60201082a0 2024-09-10T08:40:29.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.006+0000 7f6027000640 1 -- 172.21.15.89:0/1841058753 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602019b000 con 0x7f60201082a0 2024-09-10T08:40:29.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.006+0000 7f6027000640 1 -- 172.21.15.89:0/1841058753 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60201a72f0 con 0x7f60201082a0 2024-09-10T08:40:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.011+0000 7f1ad6c00640 1 -- 172.21.15.89:0/1413740660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad0074c50 msgr2=0x7f1ad00750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.011+0000 7f1ad6c00640 1 --2- 172.21.15.89:0/1413740660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad0074c50 0x7f1ad00750b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1abc009900 tx=0x7f1abc02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.011+0000 7f601cc00640 1 -- 172.21.15.89:0/1841058753 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f6018005020 con 0x7f60201082a0 2024-09-10T08:40:29.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.011+0000 7f60299ef740 1 Processor -- start 2024-09-10T08:40:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.012+0000 7f1ad6c00640 1 -- 172.21.15.89:0/1413740660 shutdown_connections 2024-09-10T08:40:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.012+0000 7f1ad6c00640 1 --2- 172.21.15.89:0/1413740660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad01099d0 0x7f1ad010be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.012+0000 7f1ad6c00640 1 --2- 172.21.15.89:0/1413740660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad0074c50 0x7f1ad00750b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.012+0000 7f1ad6c00640 1 --2- 172.21.15.89:0/1413740660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ad0076230 0x7f1ad0074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.012+0000 7f1ad6c00640 1 -- 172.21.15.89:0/1413740660 >> 172.21.15.89:0/1413740660 conn(0x7f1ad00fd2e0 msgr2=0x7f1ad00ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.012+0000 7f1ad6c00640 1 -- 172.21.15.89:0/1413740660 shutdown_connections 2024-09-10T08:40:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.012+0000 7f1ad6c00640 1 -- 172.21.15.89:0/1413740660 wait complete. 2024-09-10T08:40:29.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.012+0000 7f60299ef740 1 -- start start 2024-09-10T08:40:29.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.012+0000 7f60299ef740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86a8c70 0x55daa86a9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.012+0000 7f60299ef740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86a9590 0x55daa86b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.012+0000 7f60299ef740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86b3e50 0x55daa86b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.012+0000 7f60299ef740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55daa860f3e0 con 0x55daa86a9590 2024-09-10T08:40:29.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.012+0000 7f60299ef740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55daa85b6d80 con 0x55daa86b3e50 2024-09-10T08:40:29.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.012+0000 7f60299ef740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55daa860b3a0 con 0x55daa86a8c70 2024-09-10T08:40:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.014+0000 7f1ad6c00640 1 Processor -- start 2024-09-10T08:40:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.014+0000 7f1ad6c00640 1 -- start start 2024-09-10T08:40:29.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.013+0000 7f601fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86a9590 0x55daa86b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.013+0000 7f601f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86b3e50 0x55daa86b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.013+0000 7f601fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86a9590 0x55daa86b3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59306/0 (socket says 172.21.15.89:59306) 2024-09-10T08:40:29.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.013+0000 7f601f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86b3e50 0x55daa86b6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35822/0 (socket says 172.21.15.89:35822) 2024-09-10T08:40:29.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.013+0000 7f601fe00640 1 -- 172.21.15.89:0/3637214192 learned_addr learned my addr 172.21.15.89:0/3637214192 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:29.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.013+0000 7f601cc00640 1 --2- 172.21.15.89:0/1841058753 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f60040a2c90 0x7f60040a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.013+0000 7f601f400640 1 --2- 172.21.15.89:0/3637214192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86a8c70 0x55daa86a9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.013+0000 7f601cc00640 1 -- 172.21.15.89:0/1841058753 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f60180cfbc0 con 0x7f60201082a0 2024-09-10T08:40:29.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.014+0000 7f601fe00640 1 -- 172.21.15.89:0/3637214192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86a8c70 msgr2=0x55daa86a9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:29.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.014+0000 7f601fe00640 1 --2- 172.21.15.89:0/3637214192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86a8c70 0x55daa86a9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.014+0000 7f601fe00640 1 -- 172.21.15.89:0/3637214192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86b3e50 msgr2=0x55daa86b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:29.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.014+0000 7f601fe00640 1 --2- 172.21.15.89:0/3637214192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86b3e50 0x55daa86b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.014+0000 7f601fe00640 1 -- 172.21.15.89:0/3637214192 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daa85d5ae0 con 0x55daa86a9590 2024-09-10T08:40:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.015+0000 7f1ad6c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad0074c50 0x7f1ad019e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.015+0000 7f1ad6c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad0076230 0x7f1ad019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.015+0000 7f1ad6c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ad01099d0 0x7f1ad01a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.015+0000 7f1acf400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad0076230 0x7f1ad019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.015+0000 7f1acf400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad0076230 0x7f1ad019e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44528/0 (socket says 172.21.15.89:44528) 2024-09-10T08:40:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.015+0000 7f1acfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad0074c50 0x7f1ad019e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.015+0000 7f1acf400640 1 -- 172.21.15.89:0/2408386407 learned_addr learned my addr 172.21.15.89:0/2408386407 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.015+0000 7f1ad6c00640 1 -- 172.21.15.89:0/2408386407 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad010fa00 con 0x7f1ad01099d0 2024-09-10T08:40:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.015+0000 7f1ad4c00640 1 --2- 172.21.15.89:0/2408386407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ad01099d0 0x7f1ad01a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.014+0000 7f6025000640 1 --2- 172.21.15.89:0/1841058753 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f60040a2c90 0x7f60040a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.014+0000 7f601fe00640 1 --2- 172.21.15.89:0/3637214192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86a9590 0x55daa86b3910 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6010009d50 tx=0x7f60100117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18a84fd141a9b06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:29.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.015+0000 7f6015600640 1 -- 172.21.15.89:0/3637214192 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010012650 con 0x55daa86a9590 2024-09-10T08:40:29.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.015+0000 7f6015600640 1 -- 172.21.15.89:0/3637214192 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6010009e40 con 0x55daa86a9590 2024-09-10T08:40:29.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.015+0000 7f6015600640 1 -- 172.21.15.89:0/3637214192 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601000fc90 con 0x55daa86a9590 2024-09-10T08:40:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.016+0000 7f1ad6c00640 1 -- 172.21.15.89:0/2408386407 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad010f880 con 0x7f1ad0074c50 2024-09-10T08:40:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.016+0000 7f1ad6c00640 1 -- 172.21.15.89:0/2408386407 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad010fb80 con 0x7f1ad0076230 2024-09-10T08:40:29.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.016+0000 7f60299ef740 1 -- 172.21.15.89:0/3637214192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86a9590 msgr2=0x55daa86b3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:29.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.016+0000 7f6025000640 1 --2- 172.21.15.89:0/1841058753 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f60040a2c90 0x7f60040a5150 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f60180988c0 tx=0x7f6018005b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:29.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.016+0000 7f60299ef740 1 --2- 172.21.15.89:0/3637214192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86a9590 0x55daa86b3910 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6010009d50 tx=0x7f60100117b0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.016+0000 7f1acf400640 1 -- 172.21.15.89:0/2408386407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad0074c50 msgr2=0x7f1ad019e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.016+0000 7f1acf400640 1 --2- 172.21.15.89:0/2408386407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad0074c50 0x7f1ad019e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.016+0000 7f1acf400640 1 -- 172.21.15.89:0/2408386407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ad01099d0 msgr2=0x7f1ad01a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.016+0000 7f1acf400640 1 --2- 172.21.15.89:0/2408386407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ad01099d0 0x7f1ad01a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.016+0000 7f1acf400640 1 -- 172.21.15.89:0/2408386407 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad01a5fa0 con 0x7f1ad0076230 2024-09-10T08:40:29.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.016+0000 7f60299ef740 1 -- 172.21.15.89:0/3637214192 shutdown_connections 2024-09-10T08:40:29.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.016+0000 7f60299ef740 1 --2- 172.21.15.89:0/3637214192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86b3e50 0x55daa86b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.016+0000 7f60299ef740 1 --2- 172.21.15.89:0/3637214192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86a9590 0x55daa86b3910 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.016+0000 7f60299ef740 1 --2- 172.21.15.89:0/3637214192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86a8c70 0x55daa86a9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.016+0000 7f60299ef740 1 -- 172.21.15.89:0/3637214192 >> 172.21.15.89:0/3637214192 conn(0x55daa86a5200 msgr2=0x55daa86a55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:29.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.017+0000 7f60299ef740 1 -- 172.21.15.89:0/3637214192 shutdown_connections 2024-09-10T08:40:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.017+0000 7f1acf400640 1 --2- 172.21.15.89:0/2408386407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad0076230 0x7f1ad019e8c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1abc031440 tx=0x7f1abc032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:29.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.017+0000 7f60299ef740 1 -- 172.21.15.89:0/3637214192 wait complete. 2024-09-10T08:40:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.018+0000 7f1accc00640 1 -- 172.21.15.89:0/2408386407 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc03f030 con 0x7f1ad0076230 2024-09-10T08:40:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.018+0000 7f1accc00640 1 -- 172.21.15.89:0/2408386407 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1abc003ce0 con 0x7f1ad0076230 2024-09-10T08:40:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.018+0000 7f1ad6c00640 1 -- 172.21.15.89:0/2408386407 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad01a6230 con 0x7f1ad0076230 2024-09-10T08:40:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.018+0000 7f1accc00640 1 -- 172.21.15.89:0/2408386407 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc035070 con 0x7f1ad0076230 2024-09-10T08:40:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.018+0000 7f1ad6c00640 1 -- 172.21.15.89:0/2408386407 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad01a6770 con 0x7f1ad0076230 2024-09-10T08:40:29.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.018+0000 7f60299ef740 1 -- start start 2024-09-10T08:40:29.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.018+0000 7f60299ef740 1 Processor -- start 2024-09-10T08:40:29.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.019+0000 7f60299ef740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86a8c70 0x55daa86b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.019+0000 7f60299ef740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86a9590 0x55daa86b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.019+0000 7f60299ef740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86b3e50 0x55daa86b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.019+0000 7f60299ef740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55daa86b6a20 con 0x55daa86a9590 2024-09-10T08:40:29.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.019+0000 7f60299ef740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55daa85ca0f0 con 0x55daa86a8c70 2024-09-10T08:40:29.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.019+0000 7f60299ef740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55daa86b6ba0 con 0x55daa86b3e50 2024-09-10T08:40:29.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.019+0000 7f60299ef740 10 client.0 _ll_register_callbacks cb 0x55daa85b1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:29.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.020+0000 7f601f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86b3e50 0x55daa86b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.020+0000 7f601f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86b3e50 0x55daa86b17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44534/0 (socket says 172.21.15.89:44534) 2024-09-10T08:40:29.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.020+0000 7f601f400640 1 -- 172.21.15.89:0/2371740252 learned_addr learned my addr 172.21.15.89:0/2371740252 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:29.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.020+0000 7f601f400640 1 --2- 172.21.15.89:0/2371740252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86a8c70 0x55daa86b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.020+0000 7f601f400640 1 -- 172.21.15.89:0/2371740252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86a8c70 msgr2=0x55daa86b0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:29.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.020+0000 7f601f400640 1 --2- 172.21.15.89:0/2371740252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86a8c70 0x55daa86b0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.020+0000 7f601f400640 1 -- 172.21.15.89:0/2371740252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86a9590 msgr2=0x55daa86b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:29.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.020+0000 7f601fe00640 1 --2- 172.21.15.89:0/2371740252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86a9590 0x55daa86b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.021+0000 7f601f400640 1 --2- 172.21.15.89:0/2371740252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86a9590 0x55daa86b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.021+0000 7f601f400640 1 -- 172.21.15.89:0/2371740252 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daa83803c0 con 0x55daa86b3e50 2024-09-10T08:40:29.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.021+0000 7f601fe00640 1 --2- 172.21.15.89:0/2371740252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86a9590 0x55daa86b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:40:29.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.021+0000 7f601f400640 1 --2- 172.21.15.89:0/2371740252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86b3e50 0x55daa86b17e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f60200697d0 tx=0x7f600c040040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13bed9915e80ae77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:29.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.021+0000 7f600a000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:40:29.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.021+0000 7f600a000640 1 -- 172.21.15.89:0/2371740252 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c050060 con 0x55daa86b3e50 2024-09-10T08:40:29.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.021+0000 7f600a000640 1 -- 172.21.15.89:0/2371740252 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f600c03da30 con 0x55daa86b3e50 2024-09-10T08:40:29.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.022+0000 7f60299ef740 1 -- 172.21.15.89:0/2371740252 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55daa861c950 con 0x55daa86b3e50 2024-09-10T08:40:29.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.022+0000 7f600a000640 1 -- 172.21.15.89:0/2371740252 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f600c035040 con 0x55daa86b3e50 2024-09-10T08:40:29.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.022+0000 7f60299ef740 10 client.25414 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:29.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.022+0000 7f60299ef740 1 -- 172.21.15.89:0/2371740252 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55daa86c2a90 con 0x55daa86b3e50 2024-09-10T08:40:29.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.023+0000 7f600a000640 1 -- 172.21.15.89:0/2371740252 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55daa861c950 con 0x55daa86b3e50 2024-09-10T08:40:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.023+0000 7f1ad9a33740 1 Processor -- start 2024-09-10T08:40:29.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.023+0000 7f600a000640 1 -- 172.21.15.89:0/2371740252 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f600c03a040 con 0x55daa86b3e50 2024-09-10T08:40:29.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.023+0000 7f60299ef740 10 client.25414 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:29.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.023+0000 7f60299ef740 10 client.25414 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:40:29.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.023+0000 7f60299ef740 20 client.25414 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:29.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.023+0000 7f60299ef740 1 --2- 172.21.15.89:0/2371740252 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55daa86c3f50 0x55daa86c6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.023+0000 7f60299ef740 4 client.25414 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:40:29.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.023+0000 7f60299ef740 1 -- 172.21.15.89:0/2371740252 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55daa8616e60 con 0x55daa86c3f50 2024-09-10T08:40:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.023+0000 7f1ad9a33740 1 -- start start 2024-09-10T08:40:29.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.024+0000 7f601fe00640 1 --2- 172.21.15.89:0/2371740252 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55daa86c3f50 0x55daa86c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.024+0000 7f1accc00640 1 -- 172.21.15.89:0/2408386407 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f1abc042030 con 0x7f1ad0076230 2024-09-10T08:40:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.024+0000 7f1ad9a33740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397ea090 0x55bf397ea490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.024+0000 7f1ad9a33740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397ea9d0 0x55bf397ef490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.024+0000 7f1ad9a33740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397efa60 0x55bf397f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.024+0000 7f1ad9a33740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bf3974f100 con 0x55bf397ea090 2024-09-10T08:40:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.024+0000 7f1ad9a33740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55bf396f71b0 con 0x55bf397ea9d0 2024-09-10T08:40:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.024+0000 7f1ad9a33740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bf3974b0c0 con 0x55bf397efa60 2024-09-10T08:40:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.024+0000 7f1ad4c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397ea090 0x55bf397ea490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.024+0000 7f1acf400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397efa60 0x55bf397f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.024+0000 7f1acf400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397efa60 0x55bf397f1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44540/0 (socket says 172.21.15.89:44540) 2024-09-10T08:40:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.024+0000 7f1ad4c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397ea090 0x55bf397ea490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59338/0 (socket says 172.21.15.89:59338) 2024-09-10T08:40:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.024+0000 7f1acf400640 1 -- 172.21.15.89:0/1048385029 learned_addr learned my addr 172.21.15.89:0/1048385029 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.024+0000 7f1acfe00640 1 --2- 172.21.15.89:0/1048385029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397ea9d0 0x55bf397ef490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.025+0000 7f1acfe00640 1 -- 172.21.15.89:0/1048385029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397efa60 msgr2=0x55bf397f1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.025+0000 7f1acfe00640 1 --2- 172.21.15.89:0/1048385029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397efa60 0x55bf397f1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.025+0000 7f1acfe00640 1 -- 172.21.15.89:0/1048385029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397ea090 msgr2=0x55bf397ea490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.025+0000 7f1acfe00640 1 --2- 172.21.15.89:0/1048385029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397ea090 0x55bf397ea490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.025+0000 7f1acfe00640 1 -- 172.21.15.89:0/1048385029 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf39462210 con 0x55bf397ea9d0 2024-09-10T08:40:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.025+0000 7f1acf400640 1 --2- 172.21.15.89:0/1048385029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397efa60 0x55bf397f1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.025+0000 7f1ad4c00640 1 --2- 172.21.15.89:0/1048385029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397ea090 0x55bf397ea490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:29.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.025+0000 7f601fe00640 1 --2- 172.21.15.89:0/2371740252 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55daa86c3f50 0x55daa86c6340 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d8e1bc7634bd032 server_cookie=f391739e5639534b in_seq=0 out_seq=0 2024-09-10T08:40:29.047 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:28 smithi111 ceph-mon[35220]: pgmap v409: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 937 B/s wr, 8 op/s 2024-09-10T08:40:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.025+0000 7f1accc00640 1 --2- 172.21.15.89:0/2408386407 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1ab40a2da0 0x7f1ab40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.026+0000 7f1accc00640 1 -- 172.21.15.89:0/2408386407 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f1abc0efb70 con 0x7f1ad0076230 2024-09-10T08:40:29.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.025+0000 7f600a000640 10 client.25414 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:40:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.026+0000 7f1ad4c00640 1 --2- 172.21.15.89:0/2408386407 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1ab40a2da0 0x7f1ab40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.026+0000 7f600a000640 1 -- 172.21.15.89:0/2371740252 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55daa8616e60 con 0x55daa86c3f50 2024-09-10T08:40:29.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.026+0000 7f600a000640 10 client.25414 handle_command_reply: tid=0 2024-09-10T08:40:29.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.026+0000 7f60299ef740 1 client.25414 shutdown 2024-09-10T08:40:29.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.026+0000 7f60299ef740 1 -- 172.21.15.89:0/2371740252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86b3e50 msgr2=0x55daa86b17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:29.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.026+0000 7f60299ef740 1 --2- 172.21.15.89:0/2371740252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86b3e50 0x55daa86b17e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f60200697d0 tx=0x7f600c040040 comp rx=0 tx=0).stop 2024-09-10T08:40:29.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.026+0000 7f60299ef740 1 -- 172.21.15.89:0/2371740252 shutdown_connections 2024-09-10T08:40:29.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.026+0000 7f60299ef740 1 --2- 172.21.15.89:0/2371740252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86b3e50 0x55daa86b17e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.026+0000 7f60299ef740 1 --2- 172.21.15.89:0/2371740252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86a9590 0x55daa86b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.026+0000 7f60299ef740 1 --2- 172.21.15.89:0/2371740252 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55daa86c3f50 0x55daa86c6340 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.026+0000 7f60299ef740 1 --2- 172.21.15.89:0/2371740252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86a8c70 0x55daa86b0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.026+0000 7f60299ef740 1 -- 172.21.15.89:0/2371740252 >> 172.21.15.89:0/2371740252 conn(0x55daa86a5200 msgr2=0x55daa86a4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:29.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.026+0000 7f600a000640 0 client.25414 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:40:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.027+0000 7f1acfe00640 1 --2- 172.21.15.89:0/1048385029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397ea9d0 0x55bf397ef490 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000bbf0 tx=0x7f1ac00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=932d83b3d58f908b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:29.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.027+0000 7f60299ef740 1 -- 172.21.15.89:0/2371740252 shutdown_connections 2024-09-10T08:40:29.052 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.027+0000 7f1ad4c00640 1 --2- 172.21.15.89:0/2408386407 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1ab40a2da0 0x7f1ab40a5260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8010620 tx=0x7f1ac8007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:29.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.027+0000 7f60299ef740 1 -- 172.21.15.89:0/2371740252 wait complete. 2024-09-10T08:40:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.027+0000 7f1ac5200640 1 -- 172.21.15.89:0/1048385029 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0004030 con 0x55bf397ea9d0 2024-09-10T08:40:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.027+0000 7f1ac5200640 1 -- 172.21.15.89:0/1048385029 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ac00041d0 con 0x55bf397ea9d0 2024-09-10T08:40:29.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.027+0000 7f60299ef740 20 client.25414 trim_cache size 0 max 16384 2024-09-10T08:40:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.027+0000 7f1ac5200640 1 -- 172.21.15.89:0/1048385029 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00116c0 con 0x55bf397ea9d0 2024-09-10T08:40:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.028+0000 7f1ad9a33740 1 -- 172.21.15.89:0/1048385029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397ea9d0 msgr2=0x55bf397ef490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.028+0000 7f1ad9a33740 1 --2- 172.21.15.89:0/1048385029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397ea9d0 0x55bf397ef490 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000bbf0 tx=0x7f1ac00107a0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.028+0000 7f1ad9a33740 1 -- 172.21.15.89:0/1048385029 shutdown_connections 2024-09-10T08:40:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.028+0000 7f1ad9a33740 1 --2- 172.21.15.89:0/1048385029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397efa60 0x55bf397f1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.028+0000 7f1ad9a33740 1 --2- 172.21.15.89:0/1048385029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397ea9d0 0x55bf397ef490 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.028+0000 7f1ad9a33740 1 --2- 172.21.15.89:0/1048385029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397ea090 0x55bf397ea490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.028+0000 7f1ad9a33740 1 -- 172.21.15.89:0/1048385029 >> 172.21.15.89:0/1048385029 conn(0x55bf397e5700 msgr2=0x55bf397e5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.028+0000 7f1ad9a33740 1 -- 172.21.15.89:0/1048385029 shutdown_connections 2024-09-10T08:40:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.028+0000 7f1ad9a33740 1 -- 172.21.15.89:0/1048385029 wait complete. 2024-09-10T08:40:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.029+0000 7f1ad9a33740 1 -- start start 2024-09-10T08:40:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.029+0000 7f1ad9a33740 1 Processor -- start 2024-09-10T08:40:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.030+0000 7f1ad9a33740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397ea090 0x55bf397f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.030+0000 7f1ad9a33740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397ea9d0 0x55bf397f4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.030+0000 7f1ad9a33740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397efa60 0x55bf397fb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.030+0000 7f1ad9a33740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bf397f2a30 con 0x55bf397efa60 2024-09-10T08:40:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.030+0000 7f1ad9a33740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55bf396d8090 con 0x55bf397ea090 2024-09-10T08:40:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.030+0000 7f1ad9a33740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bf397f2bb0 con 0x55bf397ea9d0 2024-09-10T08:40:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.030+0000 7f1ad9a33740 10 client.0 _ll_register_callbacks cb 0x55bf3972cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.030+0000 7f1acfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397ea090 0x55bf397f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.030+0000 7f1acfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397ea090 0x55bf397f42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35858/0 (socket says 172.21.15.89:35858) 2024-09-10T08:40:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.030+0000 7f1acfe00640 1 -- 172.21.15.89:0/1764257371 learned_addr learned my addr 172.21.15.89:0/1764257371 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.030+0000 7f1acf400640 1 --2- 172.21.15.89:0/1764257371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397ea9d0 0x55bf397f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.030+0000 7f1ad4c00640 1 --2- 172.21.15.89:0/1764257371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397efa60 0x55bf397fb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.030+0000 7f1acfe00640 1 -- 172.21.15.89:0/1764257371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397ea9d0 msgr2=0x55bf397f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.030+0000 7f1acfe00640 1 --2- 172.21.15.89:0/1764257371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397ea9d0 0x55bf397f4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.030+0000 7f1acfe00640 1 -- 172.21.15.89:0/1764257371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397efa60 msgr2=0x55bf397fb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.031+0000 7f1acfe00640 1 --2- 172.21.15.89:0/1764257371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397efa60 0x55bf397fb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.031+0000 7f1acfe00640 1 -- 172.21.15.89:0/1764257371 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf3952e620 con 0x55bf397ea090 2024-09-10T08:40:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.031+0000 7f1ad4c00640 1 --2- 172.21.15.89:0/1764257371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397efa60 0x55bf397fb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:40:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.032+0000 7f1acfe00640 1 --2- 172.21.15.89:0/1764257371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397ea090 0x55bf397f42e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0000c00 tx=0x7f1ac000bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64000dd8ef1e097 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.032+0000 7f1ab9600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:40:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.032+0000 7f1ab9600640 1 -- 172.21.15.89:0/1764257371 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0005040 con 0x55bf397ea090 2024-09-10T08:40:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.032+0000 7f1ab9600640 1 -- 172.21.15.89:0/1764257371 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f1ac00051e0 con 0x55bf397ea090 2024-09-10T08:40:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.032+0000 7f1ad9a33740 1 -- 172.21.15.89:0/1764257371 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac00041d0 con 0x55bf397ea090 2024-09-10T08:40:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.032+0000 7f1ab9600640 1 -- 172.21.15.89:0/1764257371 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1ac00053e0 con 0x55bf397ea090 2024-09-10T08:40:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.032+0000 7f1ad9a33740 10 client.25451 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.033+0000 7f1ad9a33740 1 -- 172.21.15.89:0/1764257371 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bf397fe650 con 0x55bf397ea090 2024-09-10T08:40:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.033+0000 7f1ab9600640 1 -- 172.21.15.89:0/1764257371 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ac00041d0 con 0x55bf397ea090 2024-09-10T08:40:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.033+0000 7f1ab9600640 1 -- 172.21.15.89:0/1764257371 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1ac0004ac0 con 0x55bf397ea090 2024-09-10T08:40:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.033+0000 7f1ad9a33740 10 client.25451 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.033+0000 7f1ad9a33740 10 client.25451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:40:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.033+0000 7f1ad9a33740 20 client.25451 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.033+0000 7f1ad9a33740 1 --2- 172.21.15.89:0/1764257371 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bf397ffc00 0x55bf39802070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.033+0000 7f1ad9a33740 4 client.25451 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:40:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.033+0000 7f1ad9a33740 1 -- 172.21.15.89:0/1764257371 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf3975c830 con 0x55bf397ffc00 2024-09-10T08:40:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.033+0000 7f1acf400640 1 --2- 172.21.15.89:0/1764257371 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bf397ffc00 0x55bf39802070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.034+0000 7f1ab9600640 10 client.25451 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.034+0000 7f1acf400640 1 --2- 172.21.15.89:0/1764257371 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bf397ffc00 0x55bf39802070 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77973de9f668668e server_cookie=3557531d1ac69765 in_seq=0 out_seq=0 2024-09-10T08:40:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.035+0000 7f1ab9600640 1 -- 172.21.15.89:0/1764257371 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf3975c830 con 0x55bf397ffc00 2024-09-10T08:40:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.035+0000 7f1ab9600640 10 client.25451 handle_command_reply: tid=0 2024-09-10T08:40:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.035+0000 7f1ad9a33740 1 client.25451 shutdown 2024-09-10T08:40:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.035+0000 7f1ad9a33740 1 -- 172.21.15.89:0/1764257371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397ea090 msgr2=0x55bf397f42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.035+0000 7f1ad9a33740 1 --2- 172.21.15.89:0/1764257371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397ea090 0x55bf397f42e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0000c00 tx=0x7f1ac000bfd0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.035+0000 7f1ad9a33740 1 -- 172.21.15.89:0/1764257371 shutdown_connections 2024-09-10T08:40:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.035+0000 7f1ad9a33740 1 --2- 172.21.15.89:0/1764257371 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bf397ffc00 0x55bf39802070 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.035+0000 7f1ad9a33740 1 --2- 172.21.15.89:0/1764257371 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397efa60 0x55bf397fb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.035+0000 7f1ad9a33740 1 --2- 172.21.15.89:0/1764257371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397ea9d0 0x55bf397f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.035+0000 7f1ad9a33740 1 --2- 172.21.15.89:0/1764257371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397ea090 0x55bf397f42e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.035+0000 7f1ab9600640 0 client.25451 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.035+0000 7f1ad9a33740 1 -- 172.21.15.89:0/1764257371 >> 172.21.15.89:0/1764257371 conn(0x55bf397e5700 msgr2=0x55bf397e6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.035+0000 7f1ad9a33740 1 -- 172.21.15.89:0/1764257371 shutdown_connections 2024-09-10T08:40:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.035+0000 7f1ad9a33740 1 -- 172.21.15.89:0/1764257371 wait complete. 2024-09-10T08:40:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.036+0000 7f1ad9a33740 20 client.25451 trim_cache size 0 max 16384 2024-09-10T08:40:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.047+0000 7f60299ef740 1 Processor -- start 2024-09-10T08:40:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.048+0000 7f60299ef740 1 -- start start 2024-09-10T08:40:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.048+0000 7f60299ef740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86a8c70 0x55daa86b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.048+0000 7f60299ef740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86a9590 0x55daa86b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.048+0000 7f60299ef740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86b3e50 0x55daa86cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.048+0000 7f60299ef740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f60201b3af0 con 0x55daa86b3e50 2024-09-10T08:40:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.048+0000 7f60299ef740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f60201b13c0 con 0x55daa86a8c70 2024-09-10T08:40:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.048+0000 7f60299ef740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60201af210 con 0x55daa86a9590 2024-09-10T08:40:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.048+0000 7f601f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86b3e50 0x55daa86cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.048+0000 7f601f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86b3e50 0x55daa86cbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59350/0 (socket says 172.21.15.89:59350) 2024-09-10T08:40:29.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.048+0000 7f601f400640 1 -- 172.21.15.89:0/1496613123 learned_addr learned my addr 172.21.15.89:0/1496613123 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.049+0000 7f601f400640 1 --2- 172.21.15.89:0/1496613123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86a8c70 0x55daa86b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.049+0000 7f601fe00640 1 --2- 172.21.15.89:0/1496613123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86a9590 0x55daa86b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.049+0000 7f601f400640 1 -- 172.21.15.89:0/1496613123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86a9590 msgr2=0x55daa86b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.049+0000 7f601f400640 1 --2- 172.21.15.89:0/1496613123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86a9590 0x55daa86b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.049+0000 7f601f400640 1 -- 172.21.15.89:0/1496613123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86a8c70 msgr2=0x55daa86b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.049+0000 7f601f400640 1 --2- 172.21.15.89:0/1496613123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86a8c70 0x55daa86b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.049+0000 7f601f400640 1 -- 172.21.15.89:0/1496613123 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daa86a8aa0 con 0x55daa86b3e50 2024-09-10T08:40:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.049+0000 7f601f400640 1 --2- 172.21.15.89:0/1496613123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86b3e50 0x55daa86cbc10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f600c04bfb0 tx=0x7f600c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e47cff2c02f4c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.050+0000 7f6014c00640 1 -- 172.21.15.89:0/1496613123 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c050040 con 0x55daa86b3e50 2024-09-10T08:40:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.050+0000 7f6014c00640 1 -- 172.21.15.89:0/1496613123 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f600c03d8f0 con 0x55daa86b3e50 2024-09-10T08:40:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.050+0000 7f6014c00640 1 -- 172.21.15.89:0/1496613123 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c0399d0 con 0x55daa86b3e50 2024-09-10T08:40:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.050+0000 7f60299ef740 1 -- 172.21.15.89:0/1496613123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86b3e50 msgr2=0x55daa86cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:29.071 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.050+0000 7f60299ef740 1 --2- 172.21.15.89:0/1496613123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86b3e50 0x55daa86cbc10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f600c04bfb0 tx=0x7f600c000bc0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.051+0000 7f60299ef740 1 -- 172.21.15.89:0/1496613123 shutdown_connections 2024-09-10T08:40:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.051+0000 7f60299ef740 1 --2- 172.21.15.89:0/1496613123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86b3e50 0x55daa86cbc10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.051+0000 7f60299ef740 1 --2- 172.21.15.89:0/1496613123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86a9590 0x55daa86b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.051+0000 7f60299ef740 1 --2- 172.21.15.89:0/1496613123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86a8c70 0x55daa86b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.051+0000 7f60299ef740 1 -- 172.21.15.89:0/1496613123 >> 172.21.15.89:0/1496613123 conn(0x55daa86a5200 msgr2=0x55daa86aaee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.051+0000 7f60299ef740 1 -- 172.21.15.89:0/1496613123 shutdown_connections 2024-09-10T08:40:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.051+0000 7f60299ef740 1 -- 172.21.15.89:0/1496613123 wait complete. 2024-09-10T08:40:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.051+0000 7f60299ef740 1 -- start start 2024-09-10T08:40:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.052+0000 7f60299ef740 1 Processor -- start 2024-09-10T08:40:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.052+0000 7f60299ef740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86a8c70 0x55daa86b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.052+0000 7f60299ef740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86a9590 0x55daa86b89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.052+0000 7f60299ef740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86b3e50 0x55daa8727c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.052+0000 7f60299ef740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f60201af880 con 0x55daa86b3e50 2024-09-10T08:40:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.052+0000 7f60299ef740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f60201b0cf0 con 0x55daa86a9590 2024-09-10T08:40:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.052+0000 7f60299ef740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60201aff50 con 0x55daa86a8c70 2024-09-10T08:40:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.052+0000 7f60299ef740 10 client.0 _ll_register_callbacks cb 0x7f600c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.053+0000 7f601fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86a9590 0x55daa86b89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.053+0000 7f601f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86a8c70 0x55daa86b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.053+0000 7f601f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86a8c70 0x55daa86b84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44570/0 (socket says 172.21.15.89:44570) 2024-09-10T08:40:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.053+0000 7f601fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86a9590 0x55daa86b89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35874/0 (socket says 172.21.15.89:35874) 2024-09-10T08:40:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.053+0000 7f601f400640 1 -- 172.21.15.89:0/3637990526 learned_addr learned my addr 172.21.15.89:0/3637990526 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.053+0000 7f601f400640 1 --2- 172.21.15.89:0/3637990526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86b3e50 0x55daa8727c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.054+0000 7f601fe00640 1 -- 172.21.15.89:0/3637990526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86a8c70 msgr2=0x55daa86b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.054+0000 7f601fe00640 1 --2- 172.21.15.89:0/3637990526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86a8c70 0x55daa86b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.073 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.054+0000 7f601fe00640 1 -- 172.21.15.89:0/3637990526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86b3e50 msgr2=0x55daa8727c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:29.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.054+0000 7f601fe00640 1 --2- 172.21.15.89:0/3637990526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86b3e50 0x55daa8727c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.054+0000 7f601fe00640 1 -- 172.21.15.89:0/3637990526 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daa86a8460 con 0x55daa86a9590 2024-09-10T08:40:29.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.054+0000 7f601f400640 1 --2- 172.21.15.89:0/3637990526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86b3e50 0x55daa8727c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:29.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.055+0000 7f601fe00640 1 --2- 172.21.15.89:0/3637990526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86a9590 0x55daa86b89f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55daa86b05f0 tx=0x7f60100096f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c8e9c3e3b2d8483 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:29.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.055+0000 7f600a000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:40:29.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.055+0000 7f600a000640 1 -- 172.21.15.89:0/3637990526 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601000f980 con 0x55daa86a9590 2024-09-10T08:40:29.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.055+0000 7f600a000640 1 -- 172.21.15.89:0/3637990526 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f601000fb20 con 0x55daa86a9590 2024-09-10T08:40:29.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.055+0000 7f60299ef740 1 -- 172.21.15.89:0/3637990526 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f600c03d8f0 con 0x55daa86a9590 2024-09-10T08:40:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.056+0000 7f1ad9a33740 1 Processor -- start 2024-09-10T08:40:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.055+0000 7f600a000640 1 -- 172.21.15.89:0/3637990526 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f601000fd40 con 0x55daa86a9590 2024-09-10T08:40:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.055+0000 7f60299ef740 10 client.25457 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.055+0000 7f60299ef740 1 -- 172.21.15.89:0/3637990526 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55daa86a0770 con 0x55daa86a9590 2024-09-10T08:40:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.056+0000 7f1ad9a33740 1 -- start start 2024-09-10T08:40:29.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.056+0000 7f600a000640 1 -- 172.21.15.89:0/3637990526 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f600c03d8f0 con 0x55daa86a9590 2024-09-10T08:40:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.056+0000 7f1ad9a33740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397ea090 0x55bf397e4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.056+0000 7f1ad9a33740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397ea9d0 0x55bf397fb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.056+0000 7f600a000640 1 -- 172.21.15.89:0/3637990526 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6010003aa0 con 0x55daa86a9590 2024-09-10T08:40:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.056+0000 7f1ad9a33740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397efa60 0x55bf397e6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.056+0000 7f1ad9a33740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad01aaec0 con 0x55bf397ea090 2024-09-10T08:40:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.056+0000 7f1ad9a33740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad01a8790 con 0x55bf397efa60 2024-09-10T08:40:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.056+0000 7f1ad9a33740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad0073f70 con 0x55bf397ea9d0 2024-09-10T08:40:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.056+0000 7f1acfe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397ea090 0x55bf397e4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.056+0000 7f1ad4c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397efa60 0x55bf397e6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.057+0000 7f1ad4c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397efa60 0x55bf397e6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35882/0 (socket says 172.21.15.89:35882) 2024-09-10T08:40:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.057+0000 7f1ad4c00640 1 -- 172.21.15.89:0/2013821544 learned_addr learned my addr 172.21.15.89:0/2013821544 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.057+0000 7f1acf400640 1 --2- 172.21.15.89:0/2013821544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397ea9d0 0x55bf397fb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.056+0000 7f60299ef740 10 client.25457 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:29.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.056+0000 7f60299ef740 10 client.25457 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:40:29.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.056+0000 7f60299ef740 20 client.25457 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:29.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.056+0000 7f60299ef740 1 --2- 172.21.15.89:0/3637990526 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55daa872f8b0 0x55daa8731cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.056+0000 7f60299ef740 4 client.25457 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:40:29.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.056+0000 7f60299ef740 1 -- 172.21.15.89:0/3637990526 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55daa8593010 con 0x55daa872f8b0 2024-09-10T08:40:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.057+0000 7f1ad4c00640 1 -- 172.21.15.89:0/2013821544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397ea9d0 msgr2=0x55bf397fb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.057+0000 7f1ad4c00640 1 --2- 172.21.15.89:0/2013821544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397ea9d0 0x55bf397fb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.057+0000 7f1ad4c00640 1 -- 172.21.15.89:0/2013821544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397ea090 msgr2=0x55bf397e4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.057+0000 7f1ad4c00640 1 --2- 172.21.15.89:0/2013821544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397ea090 0x55bf397e4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.057+0000 7f1ad4c00640 1 -- 172.21.15.89:0/2013821544 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf397eef70 con 0x55bf397efa60 2024-09-10T08:40:29.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.057+0000 7f601f400640 1 --2- 172.21.15.89:0/3637990526 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55daa872f8b0 0x55daa8731cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.058+0000 7f1ad4c00640 1 --2- 172.21.15.89:0/2013821544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397efa60 0x55bf397e6af0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800f8c0 tx=0x7f1ac8015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77d6e54d9b76b412 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:29.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.058+0000 7f600a000640 10 client.25457 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:40:29.080 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.058+0000 7f601f400640 1 --2- 172.21.15.89:0/3637990526 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55daa872f8b0 0x55daa8731cd0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b617811ca9dff15b server_cookie=1243cf9b7ad6e9d6 in_seq=0 out_seq=0 2024-09-10T08:40:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.058+0000 7f1abbe00640 1 -- 172.21.15.89:0/2013821544 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac800fe40 con 0x55bf397efa60 2024-09-10T08:40:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.058+0000 7f1abbe00640 1 -- 172.21.15.89:0/2013821544 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ac8003e50 con 0x55bf397efa60 2024-09-10T08:40:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.058+0000 7f1abbe00640 1 -- 172.21.15.89:0/2013821544 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8004050 con 0x55bf397efa60 2024-09-10T08:40:29.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.058+0000 7f600a000640 1 -- 172.21.15.89:0/3637990526 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55daa8593010 con 0x55daa872f8b0 2024-09-10T08:40:29.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.058+0000 7f600a000640 10 client.25457 handle_command_reply: tid=0 2024-09-10T08:40:29.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.058+0000 7f60299ef740 1 client.25457 shutdown 2024-09-10T08:40:29.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.058+0000 7f60299ef740 1 -- 172.21.15.89:0/3637990526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86a9590 msgr2=0x55daa86b89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:29.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.058+0000 7f60299ef740 1 --2- 172.21.15.89:0/3637990526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86a9590 0x55daa86b89f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55daa86b05f0 tx=0x7f60100096f0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.059+0000 7f60299ef740 1 -- 172.21.15.89:0/3637990526 shutdown_connections 2024-09-10T08:40:29.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.059+0000 7f60299ef740 1 --2- 172.21.15.89:0/3637990526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55daa86b3e50 0x55daa8727c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.059+0000 7f60299ef740 1 --2- 172.21.15.89:0/3637990526 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55daa872f8b0 0x55daa8731cd0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.059+0000 7f60299ef740 1 --2- 172.21.15.89:0/3637990526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55daa86a9590 0x55daa86b89f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.059+0000 7f60299ef740 1 --2- 172.21.15.89:0/3637990526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daa86a8c70 0x55daa86b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.059+0000 7f60299ef740 1 -- 172.21.15.89:0/3637990526 >> 172.21.15.89:0/3637990526 conn(0x55daa86a5200 msgr2=0x55daa86a3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:29.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.059+0000 7f600a000640 0 client.25457 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:40:29.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.059+0000 7f60299ef740 1 -- 172.21.15.89:0/3637990526 shutdown_connections 2024-09-10T08:40:29.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.059+0000 7f60299ef740 1 -- 172.21.15.89:0/3637990526 wait complete. 2024-09-10T08:40:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.059+0000 7f1ad9a33740 1 -- 172.21.15.89:0/2013821544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397efa60 msgr2=0x55bf397e6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.059+0000 7f1ad9a33740 1 --2- 172.21.15.89:0/2013821544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397efa60 0x55bf397e6af0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800f8c0 tx=0x7f1ac8015000 comp rx=0 tx=0).stop 2024-09-10T08:40:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.059+0000 7f1ad9a33740 1 -- 172.21.15.89:0/2013821544 shutdown_connections 2024-09-10T08:40:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.059+0000 7f1ad9a33740 1 --2- 172.21.15.89:0/2013821544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397efa60 0x55bf397e6af0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.059+0000 7f1ad9a33740 1 --2- 172.21.15.89:0/2013821544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397ea9d0 0x55bf397fb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.059+0000 7f1ad9a33740 1 --2- 172.21.15.89:0/2013821544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397ea090 0x55bf397e4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.059+0000 7f1ad9a33740 1 -- 172.21.15.89:0/2013821544 >> 172.21.15.89:0/2013821544 conn(0x55bf397e5700 msgr2=0x55bf397e4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:29.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.059+0000 7f60299ef740 20 client.25457 trim_cache size 0 max 16384 2024-09-10T08:40:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.059+0000 7f1ad9a33740 1 -- 172.21.15.89:0/2013821544 shutdown_connections 2024-09-10T08:40:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.059+0000 7f1ad9a33740 1 -- 172.21.15.89:0/2013821544 wait complete. 2024-09-10T08:40:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.060+0000 7f1ad9a33740 1 -- start start 2024-09-10T08:40:29.084 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:40:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.060+0000 7f1ad9a33740 1 Processor -- start 2024-09-10T08:40:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.060+0000 7f1ad9a33740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397ea090 0x55bf397f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.060+0000 7f6027000640 1 -- 172.21.15.89:0/1841058753 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f60040a2c90 msgr2=0x7f60040a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:29.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.060+0000 7f6027000640 1 --2- 172.21.15.89:0/1841058753 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f60040a2c90 0x7f60040a5150 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f60180988c0 tx=0x7f6018005b40 comp rx=0 tx=0).stop 2024-09-10T08:40:29.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.060+0000 7f6027000640 1 -- 172.21.15.89:0/1841058753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60201082a0 msgr2=0x7f602019a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:29.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.060+0000 7f6027000640 1 --2- 172.21.15.89:0/1841058753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60201082a0 0x7f602019a5e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f601800eb10 tx=0x7f601800c2d0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f1ad9a33740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397ea9d0 0x55bf397f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f1ad9a33740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397efa60 0x55bf397fa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f1ad9a33740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad01a6c50 con 0x55bf397ea9d0 2024-09-10T08:40:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f1ad9a33740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad01a80c0 con 0x55bf397ea090 2024-09-10T08:40:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f1ad9a33740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad01a7320 con 0x55bf397efa60 2024-09-10T08:40:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f1ad9a33740 10 client.0 _ll_register_callbacks cb 0x7f1ac8007940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f1acfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397ea090 0x55bf397f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f1acfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397ea090 0x55bf397f8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35896/0 (socket says 172.21.15.89:35896) 2024-09-10T08:40:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f1acfe00640 1 -- 172.21.15.89:0/2188167241 learned_addr learned my addr 172.21.15.89:0/2188167241 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:29.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f6027000640 1 -- 172.21.15.89:0/1841058753 shutdown_connections 2024-09-10T08:40:29.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f6027000640 1 --2- 172.21.15.89:0/1841058753 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f60040a2c90 0x7f60040a5150 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f6027000640 1 --2- 172.21.15.89:0/1841058753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60201082a0 0x7f602019a5e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f6027000640 1 --2- 172.21.15.89:0/1841058753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60201036e0 0x7f60201a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f6027000640 1 --2- 172.21.15.89:0/1841058753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6020102d30 0x7f60201a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f6027000640 1 -- 172.21.15.89:0/1841058753 >> 172.21.15.89:0/1841058753 conn(0x7f60200fc7d0 msgr2=0x7f6020101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f1acf400640 1 --2- 172.21.15.89:0/2188167241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397ea9d0 0x55bf397f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f1acfe00640 1 -- 172.21.15.89:0/2188167241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397efa60 msgr2=0x55bf397fa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f1acfe00640 1 --2- 172.21.15.89:0/2188167241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397efa60 0x55bf397fa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f1acfe00640 1 -- 172.21.15.89:0/2188167241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397ea9d0 msgr2=0x55bf397f94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f1acfe00640 1 --2- 172.21.15.89:0/2188167241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397ea9d0 0x55bf397f94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f1acfe00640 1 -- 172.21.15.89:0/2188167241 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf397faac0 con 0x55bf397ea090 2024-09-10T08:40:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f1acf400640 1 --2- 172.21.15.89:0/2188167241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397ea9d0 0x55bf397f94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:40:29.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f6027000640 1 -- 172.21.15.89:0/1841058753 shutdown_connections 2024-09-10T08:40:29.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:29.061+0000 7f6027000640 1 -- 172.21.15.89:0/1841058753 wait complete. 2024-09-10T08:40:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.062+0000 7f1acfe00640 1 --2- 172.21.15.89:0/2188167241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397ea090 0x55bf397f8f60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000bb10 tx=0x7f1ac0005580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e3ffd23dd0af5dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.062+0000 7f1ab9600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:40:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.062+0000 7f1ab9600640 1 -- 172.21.15.89:0/2188167241 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0011720 con 0x55bf397ea090 2024-09-10T08:40:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.062+0000 7f1ab9600640 1 -- 172.21.15.89:0/2188167241 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f1ac00119b0 con 0x55bf397ea090 2024-09-10T08:40:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.062+0000 7f1ad9a33740 1 -- 172.21.15.89:0/2188167241 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac8003e50 con 0x55bf397ea090 2024-09-10T08:40:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.062+0000 7f1ab9600640 1 -- 172.21.15.89:0/2188167241 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1ac0011bb0 con 0x55bf397ea090 2024-09-10T08:40:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.062+0000 7f1ad9a33740 10 client.25463 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.062+0000 7f1ad9a33740 1 -- 172.21.15.89:0/2188167241 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bf3986a950 con 0x55bf397ea090 2024-09-10T08:40:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.063+0000 7f1ab9600640 1 -- 172.21.15.89:0/2188167241 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ac8003e50 con 0x55bf397ea090 2024-09-10T08:40:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.063+0000 7f1ab9600640 1 -- 172.21.15.89:0/2188167241 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1ac00345e0 con 0x55bf397ea090 2024-09-10T08:40:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.063+0000 7f1ad9a33740 10 client.25463 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.063+0000 7f1ad9a33740 10 client.25463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:40:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.063+0000 7f1ad9a33740 20 client.25463 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.063+0000 7f1ad9a33740 1 --2- 172.21.15.89:0/2188167241 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bf3986b1f0 0x55bf3986d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.063+0000 7f1ad9a33740 4 client.25463 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:40:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.063+0000 7f1ad9a33740 1 -- 172.21.15.89:0/2188167241 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf39756b80 con 0x55bf3986b1f0 2024-09-10T08:40:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.063+0000 7f1acf400640 1 --2- 172.21.15.89:0/2188167241 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bf3986b1f0 0x55bf3986d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.064+0000 7f1ab9600640 10 client.25463 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.064+0000 7f1acf400640 1 --2- 172.21.15.89:0/2188167241 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bf3986b1f0 0x55bf3986d610 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61fadcaa317073e3 server_cookie=b59da59fab5587bc in_seq=0 out_seq=0 2024-09-10T08:40:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.067+0000 7f1ab9600640 1 -- 172.21.15.89:0/2188167241 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf39756b80 con 0x55bf3986b1f0 2024-09-10T08:40:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.067+0000 7f1ab9600640 10 client.25463 handle_command_reply: tid=0 2024-09-10T08:40:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.067+0000 7f1ad9a33740 1 client.25463 shutdown 2024-09-10T08:40:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.067+0000 7f1ad9a33740 1 -- 172.21.15.89:0/2188167241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397ea090 msgr2=0x55bf397f8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.067+0000 7f1ad9a33740 1 --2- 172.21.15.89:0/2188167241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397ea090 0x55bf397f8f60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000bb10 tx=0x7f1ac0005580 comp rx=0 tx=0).stop 2024-09-10T08:40:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.067+0000 7f1ad9a33740 1 -- 172.21.15.89:0/2188167241 shutdown_connections 2024-09-10T08:40:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.067+0000 7f1ad9a33740 1 --2- 172.21.15.89:0/2188167241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf397efa60 0x55bf397fa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.067+0000 7f1ad9a33740 1 --2- 172.21.15.89:0/2188167241 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bf3986b1f0 0x55bf3986d610 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.067+0000 7f1ad9a33740 1 --2- 172.21.15.89:0/2188167241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bf397ea9d0 0x55bf397f94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.067+0000 7f1ad9a33740 1 --2- 172.21.15.89:0/2188167241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf397ea090 0x55bf397f8f60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.067+0000 7f1ab9600640 0 client.25463 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.067+0000 7f1ad9a33740 1 -- 172.21.15.89:0/2188167241 >> 172.21.15.89:0/2188167241 conn(0x55bf397e5700 msgr2=0x55bf397e4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.067+0000 7f1ad9a33740 1 -- 172.21.15.89:0/2188167241 shutdown_connections 2024-09-10T08:40:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.067+0000 7f1ad9a33740 1 -- 172.21.15.89:0/2188167241 wait complete. 2024-09-10T08:40:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.068+0000 7f1ad9a33740 20 client.25463 trim_cache size 0 max 16384 2024-09-10T08:40:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:40:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:40:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:40:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:40:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.068+0000 7f1ad6c00640 1 -- 172.21.15.89:0/2408386407 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1ab40a2da0 msgr2=0x7f1ab40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.068+0000 7f1ad6c00640 1 --2- 172.21.15.89:0/2408386407 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1ab40a2da0 0x7f1ab40a5260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8010620 tx=0x7f1ac8007520 comp rx=0 tx=0).stop 2024-09-10T08:40:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.068+0000 7f1ad6c00640 1 -- 172.21.15.89:0/2408386407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad0076230 msgr2=0x7f1ad019e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.068+0000 7f1ad6c00640 1 --2- 172.21.15.89:0/2408386407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad0076230 0x7f1ad019e8c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1abc031440 tx=0x7f1abc032da0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.068+0000 7f1ad6c00640 1 -- 172.21.15.89:0/2408386407 shutdown_connections 2024-09-10T08:40:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.068+0000 7f1ad6c00640 1 --2- 172.21.15.89:0/2408386407 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1ab40a2da0 0x7f1ab40a5260 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.068+0000 7f1ad6c00640 1 --2- 172.21.15.89:0/2408386407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ad01099d0 0x7f1ad01a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.068+0000 7f1ad6c00640 1 --2- 172.21.15.89:0/2408386407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ad0076230 0x7f1ad019e8c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.068+0000 7f1ad6c00640 1 --2- 172.21.15.89:0/2408386407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ad0074c50 0x7f1ad019e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.068+0000 7f1ad6c00640 1 -- 172.21.15.89:0/2408386407 >> 172.21.15.89:0/2408386407 conn(0x7f1ad00fd2e0 msgr2=0x7f1ad0101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.069+0000 7f1ad6c00640 1 -- 172.21.15.89:0/2408386407 shutdown_connections 2024-09-10T08:40:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:29.069+0000 7f1ad6c00640 1 -- 172.21.15.89:0/2408386407 wait complete. 2024-09-10T08:40:29.097 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:40:30.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.033+0000 7f62a8c00640 1 -- 172.21.15.89:0/1545142262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a410e430 msgr2=0x7f62a4110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:30.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.033+0000 7f62a8c00640 1 --2- 172.21.15.89:0/1545142262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a410e430 0x7f62a4110820 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f629c009ca0 tx=0x7f629c02a950 comp rx=0 tx=0).stop 2024-09-10T08:40:30.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.034+0000 7f62a8c00640 1 -- 172.21.15.89:0/1545142262 shutdown_connections 2024-09-10T08:40:30.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.034+0000 7f62a8c00640 1 --2- 172.21.15.89:0/1545142262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a410e430 0x7f62a4110820 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:30.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.034+0000 7f62a8c00640 1 --2- 172.21.15.89:0/1545142262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62a4074ee0 0x7f62a410def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:30.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.034+0000 7f62a8c00640 1 --2- 172.21.15.89:0/1545142262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a40745c0 0x7f62a40749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:30.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.034+0000 7f62a8c00640 1 -- 172.21.15.89:0/1545142262 >> 172.21.15.89:0/1545142262 conn(0x7f62a40fd8b0 msgr2=0x7f62a40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:30.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.034+0000 7f62a8c00640 1 -- 172.21.15.89:0/1545142262 shutdown_connections 2024-09-10T08:40:30.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.034+0000 7f62a8c00640 1 -- 172.21.15.89:0/1545142262 wait complete. 2024-09-10T08:40:30.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.035+0000 7f62a8c00640 1 Processor -- start 2024-09-10T08:40:30.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.035+0000 7f62a8c00640 1 -- start start 2024-09-10T08:40:30.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.035+0000 7f62a8c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62a40745c0 0x7f62a419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:30.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.035+0000 7f62a8c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a4074ee0 0x7f62a419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:30.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.035+0000 7f62a8c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a410e430 0x7f62a419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:30.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.035+0000 7f62a8c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f62a41141d0 con 0x7f62a4074ee0 2024-09-10T08:40:30.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.035+0000 7f62a8c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f62a4114050 con 0x7f62a410e430 2024-09-10T08:40:30.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.035+0000 7f62a8c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f62a4114350 con 0x7f62a40745c0 2024-09-10T08:40:30.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.036+0000 7f62a1400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a4074ee0 0x7f62a419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:30.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.036+0000 7f62a1400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a4074ee0 0x7f62a419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59396/0 (socket says 172.21.15.89:59396) 2024-09-10T08:40:30.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.036+0000 7f62a2800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a410e430 0x7f62a419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:30.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.036+0000 7f62a1400640 1 -- 172.21.15.89:0/2162210670 learned_addr learned my addr 172.21.15.89:0/2162210670 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:30.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.036+0000 7f62a1e00640 1 --2- 172.21.15.89:0/2162210670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62a40745c0 0x7f62a419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:30.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.036+0000 7f62a2800640 1 -- 172.21.15.89:0/2162210670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62a40745c0 msgr2=0x7f62a419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:30.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.036+0000 7f62a2800640 1 --2- 172.21.15.89:0/2162210670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62a40745c0 0x7f62a419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:30.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.036+0000 7f62a2800640 1 -- 172.21.15.89:0/2162210670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a4074ee0 msgr2=0x7f62a419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:30.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.036+0000 7f62a2800640 1 --2- 172.21.15.89:0/2162210670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a4074ee0 0x7f62a419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:30.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.036+0000 7f62a2800640 1 -- 172.21.15.89:0/2162210670 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a419f5f0 con 0x7f62a410e430 2024-09-10T08:40:30.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.036+0000 7f62a1400640 1 --2- 172.21.15.89:0/2162210670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a4074ee0 0x7f62a419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:30.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.037+0000 7f62a2800640 1 --2- 172.21.15.89:0/2162210670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a410e430 0x7f62a419ef10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f629c02af40 tx=0x7f629c003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:30.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.037+0000 7f629aa00640 1 -- 172.21.15.89:0/2162210670 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c034040 con 0x7f62a410e430 2024-09-10T08:40:30.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.037+0000 7f62a8c00640 1 -- 172.21.15.89:0/2162210670 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a419f880 con 0x7f62a410e430 2024-09-10T08:40:30.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.037+0000 7f62a8c00640 1 -- 172.21.15.89:0/2162210670 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a41a71c0 con 0x7f62a410e430 2024-09-10T08:40:30.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.037+0000 7f629aa00640 1 -- 172.21.15.89:0/2162210670 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f629c02f940 con 0x7f62a410e430 2024-09-10T08:40:30.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.037+0000 7f629aa00640 1 -- 172.21.15.89:0/2162210670 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c0045a0 con 0x7f62a410e430 2024-09-10T08:40:30.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.038+0000 7f62a8c00640 1 -- 172.21.15.89:0/2162210670 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a40749a0 con 0x7f62a410e430 2024-09-10T08:40:30.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.040+0000 7f629aa00640 1 -- 172.21.15.89:0/2162210670 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f629c037020 con 0x7f62a410e430 2024-09-10T08:40:30.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.042+0000 7f629aa00640 1 --2- 172.21.15.89:0/2162210670 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f62880a2c90 0x7f62880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:30.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.043+0000 7f629aa00640 1 -- 172.21.15.89:0/2162210670 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f629c0f0130 con 0x7f62a410e430 2024-09-10T08:40:30.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.043+0000 7f62a1e00640 1 --2- 172.21.15.89:0/2162210670 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f62880a2c90 0x7f62880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:30.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.044+0000 7f62a1e00640 1 --2- 172.21.15.89:0/2162210670 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f62880a2c90 0x7f62880a5150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f62940095f0 tx=0x7f6294005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:30.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.046+0000 7f629aa00640 1 -- 172.21.15.89:0/2162210670 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f629c0bcc10 con 0x7f62a410e430 2024-09-10T08:40:30.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.286+0000 7f62a8c00640 1 -- 172.21.15.89:0/2162210670 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62a41095b0 con 0x7f62a410e430 2024-09-10T08:40:30.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.287+0000 7f629aa00640 1 -- 172.21.15.89:0/2162210670 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11918 (secure 0 0 0) 0x7f629c0bc510 con 0x7f62a410e430 2024-09-10T08:40:30.287 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:40:30.288 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-10T08:39:39:721431+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:39:38.834417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24518,24496]},"id":1}]} 2024-09-10T08:40:30.288 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-10T08:40:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.290+0000 7f62a8c00640 1 -- 172.21.15.89:0/2162210670 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f62880a2c90 msgr2=0x7f62880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.290+0000 7f62a8c00640 1 --2- 172.21.15.89:0/2162210670 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f62880a2c90 0x7f62880a5150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f62940095f0 tx=0x7f6294005f90 comp rx=0 tx=0).stop 2024-09-10T08:40:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.290+0000 7f62a8c00640 1 -- 172.21.15.89:0/2162210670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a410e430 msgr2=0x7f62a419ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.290+0000 7f62a8c00640 1 --2- 172.21.15.89:0/2162210670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a410e430 0x7f62a419ef10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f629c02af40 tx=0x7f629c003b80 comp rx=0 tx=0).stop 2024-09-10T08:40:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.290+0000 7f62a8c00640 1 -- 172.21.15.89:0/2162210670 shutdown_connections 2024-09-10T08:40:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.290+0000 7f62a8c00640 1 --2- 172.21.15.89:0/2162210670 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f62880a2c90 0x7f62880a5150 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:30.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.290+0000 7f62a8c00640 1 --2- 172.21.15.89:0/2162210670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a410e430 0x7f62a419ef10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:30.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.290+0000 7f62a8c00640 1 --2- 172.21.15.89:0/2162210670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62a4074ee0 0x7f62a419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:30.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.290+0000 7f62a8c00640 1 --2- 172.21.15.89:0/2162210670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62a40745c0 0x7f62a419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:30.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.290+0000 7f62a8c00640 1 -- 172.21.15.89:0/2162210670 >> 172.21.15.89:0/2162210670 conn(0x7f62a40fd8b0 msgr2=0x7f62a40fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:30.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.290+0000 7f62a8c00640 1 -- 172.21.15.89:0/2162210670 shutdown_connections 2024-09-10T08:40:30.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:30.290+0000 7f62a8c00640 1 -- 172.21.15.89:0/2162210670 wait complete. 2024-09-10T08:40:30.427 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:40:30.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:30 smithi149 ceph-mon[36167]: pgmap v410: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 937 B/s wr, 7 op/s 2024-09-10T08:40:30.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:30 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2162210670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:40:31.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:30 smithi111 ceph-mon[35220]: pgmap v410: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 937 B/s wr, 7 op/s 2024-09-10T08:40:31.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:30 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2162210670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:40:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:30 smithi089 ceph-mon[30757]: pgmap v410: 129 pgs: 129 active+clean; 663 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 937 B/s wr, 7 op/s 2024-09-10T08:40:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:30 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2162210670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:40:31.116 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24554 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T08:40:31.117 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24554 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T08:40:31.230 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:40:31.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.808+0000 7fa5af600640 1 -- 172.21.15.89:0/2717323046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5a81049a0 msgr2=0x7fa5a81089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:31.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.808+0000 7fa5af600640 1 --2- 172.21.15.89:0/2717323046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5a81049a0 0x7fa5a81089f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa594009900 tx=0x7fa59402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.808+0000 7fa5af600640 1 -- 172.21.15.89:0/2717323046 shutdown_connections 2024-09-10T08:40:31.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.808+0000 7fa5af600640 1 --2- 172.21.15.89:0/2717323046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5a8109120 0x7fa5a8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.808+0000 7fa5af600640 1 --2- 172.21.15.89:0/2717323046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5a81049a0 0x7fa5a81089f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.808+0000 7fa5af600640 1 --2- 172.21.15.89:0/2717323046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5a8103ff0 0x7fa5a81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.808+0000 7fa5af600640 1 -- 172.21.15.89:0/2717323046 >> 172.21.15.89:0/2717323046 conn(0x7fa5a80ffa90 msgr2=0x7fa5a8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:31.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.808+0000 7fa5af600640 1 -- 172.21.15.89:0/2717323046 shutdown_connections 2024-09-10T08:40:31.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.808+0000 7fa5af600640 1 -- 172.21.15.89:0/2717323046 wait complete. 2024-09-10T08:40:31.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.809+0000 7fa5af600640 1 Processor -- start 2024-09-10T08:40:31.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.809+0000 7fa5af600640 1 -- start start 2024-09-10T08:40:31.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.810+0000 7fa5af600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5a8103ff0 0x7fa5a8076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:31.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.810+0000 7fa5af600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5a81049a0 0x7fa5a8076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:31.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.810+0000 7fa5af600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5a8109120 0x7fa5a8077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.810+0000 7fa5af600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a8114170 con 0x7fa5a8109120 2024-09-10T08:40:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.810+0000 7fa5af600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a8113ff0 con 0x7fa5a8103ff0 2024-09-10T08:40:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.810+0000 7fa5af600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a81142f0 con 0x7fa5a81049a0 2024-09-10T08:40:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.810+0000 7fa5acc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5a8103ff0 0x7fa5a8076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.810+0000 7fa5acc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5a8103ff0 0x7fa5a8076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35930/0 (socket says 172.21.15.89:35930) 2024-09-10T08:40:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.810+0000 7fa5acc00640 1 -- 172.21.15.89:0/3416934119 learned_addr learned my addr 172.21.15.89:0/3416934119 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.810+0000 7fa5a7e00640 1 --2- 172.21.15.89:0/3416934119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5a81049a0 0x7fa5a8076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:31.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.810+0000 7fa5ad600640 1 --2- 172.21.15.89:0/3416934119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5a8109120 0x7fa5a8077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.811+0000 7fa5a7e00640 1 -- 172.21.15.89:0/3416934119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5a8103ff0 msgr2=0x7fa5a8076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.811+0000 7fa5a7e00640 1 --2- 172.21.15.89:0/3416934119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5a8103ff0 0x7fa5a8076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.811+0000 7fa5a7e00640 1 -- 172.21.15.89:0/3416934119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5a8109120 msgr2=0x7fa5a8077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.811+0000 7fa5a7e00640 1 --2- 172.21.15.89:0/3416934119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5a8109120 0x7fa5a8077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.811+0000 7fa5a7e00640 1 -- 172.21.15.89:0/3416934119 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a81aacf0 con 0x7fa5a81049a0 2024-09-10T08:40:31.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.811+0000 7fa5a7e00640 1 --2- 172.21.15.89:0/3416934119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5a81049a0 0x7fa5a8076c40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa594009230 tx=0x7fa594003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.812+0000 7fa5a5600640 1 -- 172.21.15.89:0/3416934119 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59403f070 con 0x7fa5a81049a0 2024-09-10T08:40:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.812+0000 7fa5a5600640 1 -- 172.21.15.89:0/3416934119 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa594031c90 con 0x7fa5a81049a0 2024-09-10T08:40:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.812+0000 7fa5a5600640 1 -- 172.21.15.89:0/3416934119 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594035030 con 0x7fa5a81049a0 2024-09-10T08:40:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.812+0000 7fa5af600640 1 -- 172.21.15.89:0/3416934119 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a81aaf80 con 0x7fa5a81049a0 2024-09-10T08:40:31.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.812+0000 7fa5af600640 1 -- 172.21.15.89:0/3416934119 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a81ab3e0 con 0x7fa5a81049a0 2024-09-10T08:40:31.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.816+0000 7fa5b2007740 1 Processor -- start 2024-09-10T08:40:31.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.816+0000 7fa5b2007740 1 -- start start 2024-09-10T08:40:31.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.816+0000 7fa5b2007740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ec5c70 0x555cc0ec6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:31.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.816+0000 7fa5b2007740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 0x555cc0ed0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.817+0000 7fa5b2007740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ed0e50 0x555cc0ed3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.817+0000 7fa5b2007740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555cc0e2c3e0 con 0x555cc0ec5c70 2024-09-10T08:40:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.817+0000 7fa5b2007740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555cc0dd3d80 con 0x555cc0ed0e50 2024-09-10T08:40:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.817+0000 7fa5b2007740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555cc0e283a0 con 0x555cc0ec6590 2024-09-10T08:40:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.817+0000 7fa5acc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 0x555cc0ed0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.817+0000 7fa5acc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 0x555cc0ed0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44646/0 (socket says 172.21.15.89:44646) 2024-09-10T08:40:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.817+0000 7fa5acc00640 1 -- 172.21.15.89:0/217121913 learned_addr learned my addr 172.21.15.89:0/217121913 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.817+0000 7fa5a5600640 1 -- 172.21.15.89:0/3416934119 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fa594042070 con 0x7fa5a81049a0 2024-09-10T08:40:31.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.817+0000 7fa5acc00640 1 -- 172.21.15.89:0/217121913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ed0e50 msgr2=0x555cc0ed3240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:40:31.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.817+0000 7fa5ad600640 1 --2- 172.21.15.89:0/217121913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ec5c70 0x555cc0ec6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:31.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.817+0000 7fa5acc00640 1 --2- 172.21.15.89:0/217121913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ed0e50 0x555cc0ed3240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.817+0000 7fa5acc00640 1 -- 172.21.15.89:0/217121913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ec5c70 msgr2=0x555cc0ec6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:31.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.817+0000 7fa5acc00640 1 --2- 172.21.15.89:0/217121913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ec5c70 0x555cc0ec6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.817+0000 7fa5acc00640 1 -- 172.21.15.89:0/217121913 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc0df2ae0 con 0x555cc0ec6590 2024-09-10T08:40:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.817+0000 7fa5ad600640 1 --2- 172.21.15.89:0/217121913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ec5c70 0x555cc0ec6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:40:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.818+0000 7fa5acc00640 1 --2- 172.21.15.89:0/217121913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 0x555cc0ed0910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009dc0 tx=0x7fa59c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f006316a8d1cb1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.818+0000 7fa599c00640 1 -- 172.21.15.89:0/217121913 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c005220 con 0x555cc0ec6590 2024-09-10T08:40:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.819+0000 7fa599c00640 1 -- 172.21.15.89:0/217121913 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa59c004c80 con 0x555cc0ec6590 2024-09-10T08:40:31.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.819+0000 7fa599c00640 1 -- 172.21.15.89:0/217121913 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c019910 con 0x555cc0ec6590 2024-09-10T08:40:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.819+0000 7fa5b2007740 1 -- 172.21.15.89:0/217121913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 msgr2=0x555cc0ed0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.819+0000 7fa5b2007740 1 --2- 172.21.15.89:0/217121913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 0x555cc0ed0910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009dc0 tx=0x7fa59c011810 comp rx=0 tx=0).stop 2024-09-10T08:40:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.820+0000 7fa5a5600640 1 --2- 172.21.15.89:0/3416934119 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5900a2c90 0x7fa5900a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.820+0000 7fa5b2007740 1 -- 172.21.15.89:0/217121913 shutdown_connections 2024-09-10T08:40:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.820+0000 7fa5b2007740 1 --2- 172.21.15.89:0/217121913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ed0e50 0x555cc0ed3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.820+0000 7fa5b2007740 1 --2- 172.21.15.89:0/217121913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 0x555cc0ed0910 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.820+0000 7fa5b2007740 1 --2- 172.21.15.89:0/217121913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ec5c70 0x555cc0ec6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.820+0000 7fa5b2007740 1 -- 172.21.15.89:0/217121913 >> 172.21.15.89:0/217121913 conn(0x555cc0ec2200 msgr2=0x555cc0ec25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.820+0000 7fa5a5600640 1 -- 172.21.15.89:0/3416934119 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa5940efa10 con 0x7fa5a81049a0 2024-09-10T08:40:31.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.820+0000 7fa5ad600640 1 --2- 172.21.15.89:0/3416934119 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5900a2c90 0x7fa5900a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:31.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.821+0000 7fa5b2007740 1 -- 172.21.15.89:0/217121913 shutdown_connections 2024-09-10T08:40:31.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.821+0000 7fa5b2007740 1 -- 172.21.15.89:0/217121913 wait complete. 2024-09-10T08:40:31.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.821+0000 7fa5ad600640 1 --2- 172.21.15.89:0/3416934119 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5900a2c90 0x7fa5900a5150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0005fd0 tx=0x7fa5a0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:31.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.821+0000 7fa5b2007740 1 -- start start 2024-09-10T08:40:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.821+0000 7fa5b2007740 1 Processor -- start 2024-09-10T08:40:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.822+0000 7fa5b2007740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ec5c70 0x555cc0ecdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.822+0000 7fa5b2007740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 0x555cc0ece230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.822+0000 7fa5b2007740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ed0e50 0x555cc0ece770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.822+0000 7fa5b2007740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555cc0ed3a20 con 0x555cc0ed0e50 2024-09-10T08:40:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.822+0000 7fa5b2007740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555cc0de70f0 con 0x555cc0ec5c70 2024-09-10T08:40:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.822+0000 7fa5b2007740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555cc0ed3ba0 con 0x555cc0ec6590 2024-09-10T08:40:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.822+0000 7fa5b2007740 10 client.0 _ll_register_callbacks cb 0x555cc0dcebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.822+0000 7fa5acc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ec5c70 0x555cc0ecdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.822+0000 7fa5acc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ec5c70 0x555cc0ecdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35936/0 (socket says 172.21.15.89:35936) 2024-09-10T08:40:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.822+0000 7fa5acc00640 1 -- 172.21.15.89:0/965277720 learned_addr learned my addr 172.21.15.89:0/965277720 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.822+0000 7fa5ad600640 1 --2- 172.21.15.89:0/965277720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ed0e50 0x555cc0ece770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.823+0000 7fa5acc00640 1 -- 172.21.15.89:0/965277720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 msgr2=0x555cc0ece230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:31.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.823+0000 7fa5a7e00640 1 --2- 172.21.15.89:0/965277720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 0x555cc0ece230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:31.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.823+0000 7fa5acc00640 1 --2- 172.21.15.89:0/965277720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 0x555cc0ece230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.823+0000 7fa5acc00640 1 -- 172.21.15.89:0/965277720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ed0e50 msgr2=0x555cc0ece770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:31.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.823+0000 7fa5acc00640 1 --2- 172.21.15.89:0/965277720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ed0e50 0x555cc0ece770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.823+0000 7fa5acc00640 1 -- 172.21.15.89:0/965277720 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc0b9d3c0 con 0x555cc0ec5c70 2024-09-10T08:40:31.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.823+0000 7fa5ad600640 1 --2- 172.21.15.89:0/965277720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ed0e50 0x555cc0ece770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:40:31.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.823+0000 7fa5acc00640 1 --2- 172.21.15.89:0/965277720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ec5c70 0x555cc0ecdcf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa59c000c00 tx=0x7fa59c004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38062c095a525403 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:31.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.823+0000 7fa58e000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:40:31.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.823+0000 7fa58e000640 1 -- 172.21.15.89:0/965277720 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c027020 con 0x555cc0ec5c70 2024-09-10T08:40:31.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.824+0000 7fa5b2007740 1 -- 172.21.15.89:0/965277720 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa59c004c80 con 0x555cc0ec5c70 2024-09-10T08:40:31.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.824+0000 7fa58e000640 1 -- 172.21.15.89:0/965277720 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa59c004460 con 0x555cc0ec5c70 2024-09-10T08:40:31.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.824+0000 7fa58e000640 1 -- 172.21.15.89:0/965277720 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa59c032050 con 0x555cc0ec5c70 2024-09-10T08:40:31.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.824+0000 7fa5b2007740 10 client.25478 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:31.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.824+0000 7fa5b2007740 1 -- 172.21.15.89:0/965277720 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555cc0edf9b0 con 0x555cc0ec5c70 2024-09-10T08:40:31.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.824+0000 7fa58e000640 1 -- 172.21.15.89:0/965277720 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa59c004c80 con 0x555cc0ec5c70 2024-09-10T08:40:31.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.825+0000 7fa58e000640 1 -- 172.21.15.89:0/965277720 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa59c0035f0 con 0x555cc0ec5c70 2024-09-10T08:40:31.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.825+0000 7fa5b2007740 10 client.25478 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:31.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.825+0000 7fa5b2007740 10 client.25478 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:40:31.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.825+0000 7fa5b2007740 20 client.25478 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:31.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.825+0000 7fa5b2007740 1 --2- 172.21.15.89:0/965277720 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x555cc0ee0dc0 0x555cc0ee31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:31.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.825+0000 7fa5b2007740 4 client.25478 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:40:31.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.825+0000 7fa5b2007740 1 -- 172.21.15.89:0/965277720 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cc0e39950 con 0x555cc0ee0dc0 2024-09-10T08:40:31.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.825+0000 7fa5a7e00640 1 --2- 172.21.15.89:0/965277720 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x555cc0ee0dc0 0x555cc0ee31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:31.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.826+0000 7fa5a7e00640 1 --2- 172.21.15.89:0/965277720 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x555cc0ee0dc0 0x555cc0ee31b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a68195a2f10a621a server_cookie=1c52988773edcc5f in_seq=0 out_seq=0 2024-09-10T08:40:31.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.826+0000 7fa58e000640 10 client.25478 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:40:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.827+0000 7fa58e000640 1 -- 172.21.15.89:0/965277720 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555cc0e39950 con 0x555cc0ee0dc0 2024-09-10T08:40:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.827+0000 7fa58e000640 10 client.25478 handle_command_reply: tid=0 2024-09-10T08:40:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.827+0000 7fa5b2007740 1 client.25478 shutdown 2024-09-10T08:40:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.827+0000 7fa5b2007740 1 -- 172.21.15.89:0/965277720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ec5c70 msgr2=0x555cc0ecdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.827+0000 7fa5b2007740 1 --2- 172.21.15.89:0/965277720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ec5c70 0x555cc0ecdcf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa59c000c00 tx=0x7fa59c004970 comp rx=0 tx=0).stop 2024-09-10T08:40:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.827+0000 7fa5b2007740 1 -- 172.21.15.89:0/965277720 shutdown_connections 2024-09-10T08:40:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.827+0000 7fa5b2007740 1 --2- 172.21.15.89:0/965277720 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x555cc0ee0dc0 0x555cc0ee31b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.827+0000 7fa5b2007740 1 --2- 172.21.15.89:0/965277720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ed0e50 0x555cc0ece770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.827+0000 7fa5b2007740 1 --2- 172.21.15.89:0/965277720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 0x555cc0ece230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.827+0000 7fa5b2007740 1 --2- 172.21.15.89:0/965277720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ec5c70 0x555cc0ecdcf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.827+0000 7fa58e000640 0 client.25478 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:40:31.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.827+0000 7fa5b2007740 1 -- 172.21.15.89:0/965277720 >> 172.21.15.89:0/965277720 conn(0x555cc0ec2200 msgr2=0x555cc0ec7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:31.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.827+0000 7fa5b2007740 1 -- 172.21.15.89:0/965277720 shutdown_connections 2024-09-10T08:40:31.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.827+0000 7fa5b2007740 1 -- 172.21.15.89:0/965277720 wait complete. 2024-09-10T08:40:31.830 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.828+0000 7fa5b2007740 20 client.25478 trim_cache size 0 max 16384 2024-09-10T08:40:31.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.846+0000 7fa5b2007740 1 Processor -- start 2024-09-10T08:40:31.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.846+0000 7fa5b2007740 1 -- start start 2024-09-10T08:40:31.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.846+0000 7fa5b2007740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ec5c70 0x555cc0ecd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:31.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.846+0000 7fa5b2007740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 0x555cc0ecfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:31.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.846+0000 7fa5b2007740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ed0e50 0x555cc0ee89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:31.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.846+0000 7fa5b2007740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a81b7c70 con 0x555cc0ec5c70 2024-09-10T08:40:31.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.846+0000 7fa5b2007740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a81b5540 con 0x555cc0ed0e50 2024-09-10T08:40:31.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.846+0000 7fa5b2007740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a81b3390 con 0x555cc0ec6590 2024-09-10T08:40:31.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.847+0000 7fa5acc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ec5c70 0x555cc0ecd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:31.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.847+0000 7fa5ad600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ed0e50 0x555cc0ee89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:31.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.847+0000 7fa5ad600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ed0e50 0x555cc0ee89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35946/0 (socket says 172.21.15.89:35946) 2024-09-10T08:40:31.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.847+0000 7fa5ad600640 1 -- 172.21.15.89:0/3200924768 learned_addr learned my addr 172.21.15.89:0/3200924768 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:31.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.847+0000 7fa5a7e00640 1 --2- 172.21.15.89:0/3200924768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 0x555cc0ecfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:31.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.847+0000 7fa5ad600640 1 -- 172.21.15.89:0/3200924768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 msgr2=0x555cc0ecfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:31.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.847+0000 7fa5ad600640 1 --2- 172.21.15.89:0/3200924768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 0x555cc0ecfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.847+0000 7fa5ad600640 1 -- 172.21.15.89:0/3200924768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ec5c70 msgr2=0x555cc0ecd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:31.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.847+0000 7fa5ad600640 1 --2- 172.21.15.89:0/3200924768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ec5c70 0x555cc0ecd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.847+0000 7fa5ad600640 1 -- 172.21.15.89:0/3200924768 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc0ee3770 con 0x555cc0ed0e50 2024-09-10T08:40:31.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.848+0000 7fa5a7e00640 1 --2- 172.21.15.89:0/3200924768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 0x555cc0ecfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:40:31.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.848+0000 7fa5ad600640 1 --2- 172.21.15.89:0/3200924768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ed0e50 0x555cc0ee89a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0011830 tx=0x7fa5a0016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=986a104254fd229b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:31.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.848+0000 7fa599200640 1 -- 172.21.15.89:0/3200924768 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a001f990 con 0x555cc0ed0e50 2024-09-10T08:40:31.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.848+0000 7fa599200640 1 -- 172.21.15.89:0/3200924768 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a001fb30 con 0x555cc0ed0e50 2024-09-10T08:40:31.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.848+0000 7fa599200640 1 -- 172.21.15.89:0/3200924768 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0010630 con 0x555cc0ed0e50 2024-09-10T08:40:31.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.849+0000 7fa5b2007740 1 -- 172.21.15.89:0/3200924768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ed0e50 msgr2=0x555cc0ee89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:31.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.849+0000 7fa5b2007740 1 --2- 172.21.15.89:0/3200924768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ed0e50 0x555cc0ee89a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0011830 tx=0x7fa5a0016000 comp rx=0 tx=0).stop 2024-09-10T08:40:31.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.849+0000 7fa5b2007740 1 -- 172.21.15.89:0/3200924768 shutdown_connections 2024-09-10T08:40:31.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.849+0000 7fa5b2007740 1 --2- 172.21.15.89:0/3200924768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ed0e50 0x555cc0ee89a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.849+0000 7fa5b2007740 1 --2- 172.21.15.89:0/3200924768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 0x555cc0ecfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.849+0000 7fa5b2007740 1 --2- 172.21.15.89:0/3200924768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ec5c70 0x555cc0ecd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.849+0000 7fa5b2007740 1 -- 172.21.15.89:0/3200924768 >> 172.21.15.89:0/3200924768 conn(0x555cc0ec2200 msgr2=0x555cc0ecf550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:31.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.850+0000 7fa5b2007740 1 -- 172.21.15.89:0/3200924768 shutdown_connections 2024-09-10T08:40:31.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.850+0000 7fa5b2007740 1 -- 172.21.15.89:0/3200924768 wait complete. 2024-09-10T08:40:31.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.850+0000 7fa5b2007740 1 -- start start 2024-09-10T08:40:31.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.850+0000 7fa5b2007740 1 Processor -- start 2024-09-10T08:40:31.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.851+0000 7fa5b2007740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ec5c70 0x555cc0ed9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:31.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.851+0000 7fa5b2007740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 0x555cc0ed47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:31.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.851+0000 7fa5b2007740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ed0e50 0x555cc0ed4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:31.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.851+0000 7fa5b2007740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a81b3a00 con 0x555cc0ed0e50 2024-09-10T08:40:31.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.851+0000 7fa5b2007740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a81b4e70 con 0x555cc0ec5c70 2024-09-10T08:40:31.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.851+0000 7fa5b2007740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a81b40d0 con 0x555cc0ec6590 2024-09-10T08:40:31.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.851+0000 7fa5b2007740 10 client.0 _ll_register_callbacks cb 0x7fa59c009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:31.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.851+0000 7fa5acc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ec5c70 0x555cc0ed9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:31.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.851+0000 7fa5acc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ec5c70 0x555cc0ed9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35952/0 (socket says 172.21.15.89:35952) 2024-09-10T08:40:31.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.851+0000 7fa5acc00640 1 -- 172.21.15.89:0/2042491179 learned_addr learned my addr 172.21.15.89:0/2042491179 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:31.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.851+0000 7fa5ad600640 1 --2- 172.21.15.89:0/2042491179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ed0e50 0x555cc0ed4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:31.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.851+0000 7fa5acc00640 1 -- 172.21.15.89:0/2042491179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 msgr2=0x555cc0ed47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:31.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.851+0000 7fa5acc00640 1 --2- 172.21.15.89:0/2042491179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 0x555cc0ed47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.851+0000 7fa5acc00640 1 -- 172.21.15.89:0/2042491179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ed0e50 msgr2=0x555cc0ed4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:31.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.851+0000 7fa5acc00640 1 --2- 172.21.15.89:0/2042491179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ed0e50 0x555cc0ed4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.851+0000 7fa5acc00640 1 -- 172.21.15.89:0/2042491179 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc0ebd770 con 0x555cc0ec5c70 2024-09-10T08:40:31.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.851+0000 7fa5ad600640 1 --2- 172.21.15.89:0/2042491179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ed0e50 0x555cc0ed4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:40:31.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.852+0000 7fa5acc00640 1 --2- 172.21.15.89:0/2042491179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ec5c70 0x555cc0ed9490 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009ce0 tx=0x7fa59c0095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86b98a39e35fb6db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:31.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.852+0000 7fa58e000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:40:31.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.852+0000 7fa58e000640 1 -- 172.21.15.89:0/2042491179 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c027050 con 0x555cc0ec5c70 2024-09-10T08:40:31.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.852+0000 7fa58e000640 1 -- 172.21.15.89:0/2042491179 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa59c003c80 con 0x555cc0ec5c70 2024-09-10T08:40:31.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.852+0000 7fa5b2007740 1 -- 172.21.15.89:0/2042491179 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5a001fb30 con 0x555cc0ec5c70 2024-09-10T08:40:31.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.852+0000 7fa58e000640 1 -- 172.21.15.89:0/2042491179 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa59c011d10 con 0x555cc0ec5c70 2024-09-10T08:40:31.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.852+0000 7fa5b2007740 10 client.25484 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:31.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.853+0000 7fa5b2007740 1 -- 172.21.15.89:0/2042491179 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555cc0ec2030 con 0x555cc0ec5c70 2024-09-10T08:40:31.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.853+0000 7fa58e000640 1 -- 172.21.15.89:0/2042491179 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a001fb30 con 0x555cc0ec5c70 2024-09-10T08:40:31.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.853+0000 7fa58e000640 1 -- 172.21.15.89:0/2042491179 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa59c019ab0 con 0x555cc0ec5c70 2024-09-10T08:40:31.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.853+0000 7fa5b2007740 10 client.25484 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:31.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.853+0000 7fa5b2007740 10 client.25484 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:40:31.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.853+0000 7fa5b2007740 20 client.25484 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:31.856 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.853+0000 7fa5b2007740 1 --2- 172.21.15.89:0/2042491179 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x555cc0f4bbc0 0x555cc0f4dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:31.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.853+0000 7fa5b2007740 4 client.25484 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:40:31.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.853+0000 7fa5b2007740 1 -- 172.21.15.89:0/2042491179 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555cc0e33e60 con 0x555cc0f4bbc0 2024-09-10T08:40:31.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.853+0000 7fa5a7e00640 1 --2- 172.21.15.89:0/2042491179 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x555cc0f4bbc0 0x555cc0f4dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:31.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.854+0000 7fa5a7e00640 1 --2- 172.21.15.89:0/2042491179 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x555cc0f4bbc0 0x555cc0f4dff0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4974e39e8c453943 server_cookie=9f1d41fe773cbdd2 in_seq=0 out_seq=0 2024-09-10T08:40:31.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.854+0000 7fa58e000640 10 client.25484 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:40:31.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.854+0000 7fa58e000640 1 -- 172.21.15.89:0/2042491179 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555cc0e33e60 con 0x555cc0f4bbc0 2024-09-10T08:40:31.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.854+0000 7fa58e000640 10 client.25484 handle_command_reply: tid=0 2024-09-10T08:40:31.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.854+0000 7fa5b2007740 1 client.25484 shutdown 2024-09-10T08:40:31.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.854+0000 7fa5b2007740 1 -- 172.21.15.89:0/2042491179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ec5c70 msgr2=0x555cc0ed9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:31.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.854+0000 7fa5b2007740 1 --2- 172.21.15.89:0/2042491179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ec5c70 0x555cc0ed9490 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009ce0 tx=0x7fa59c0095a0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.854+0000 7fa5b2007740 1 -- 172.21.15.89:0/2042491179 shutdown_connections 2024-09-10T08:40:31.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.854+0000 7fa5b2007740 1 --2- 172.21.15.89:0/2042491179 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x555cc0f4bbc0 0x555cc0f4dff0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.855+0000 7fa5b2007740 1 --2- 172.21.15.89:0/2042491179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cc0ed0e50 0x555cc0ed4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.855+0000 7fa58e000640 0 client.25484 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:40:31.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.855+0000 7fa5b2007740 1 --2- 172.21.15.89:0/2042491179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc0ec6590 0x555cc0ed47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.855+0000 7fa5b2007740 1 --2- 172.21.15.89:0/2042491179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cc0ec5c70 0x555cc0ed9490 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.855+0000 7fa5b2007740 1 -- 172.21.15.89:0/2042491179 >> 172.21.15.89:0/2042491179 conn(0x555cc0ec2200 msgr2=0x555cc0ecfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:31.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.855+0000 7fa5b2007740 1 -- 172.21.15.89:0/2042491179 shutdown_connections 2024-09-10T08:40:31.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.855+0000 7fa5b2007740 1 -- 172.21.15.89:0/2042491179 wait complete. 2024-09-10T08:40:31.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.855+0000 7fa5b2007740 20 client.25484 trim_cache size 0 max 16384 2024-09-10T08:40:31.859 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:40:31.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.856+0000 7fa5af600640 1 -- 172.21.15.89:0/3416934119 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5900a2c90 msgr2=0x7fa5900a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:31.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.856+0000 7fa5af600640 1 --2- 172.21.15.89:0/3416934119 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5900a2c90 0x7fa5900a5150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0005fd0 tx=0x7fa5a0009500 comp rx=0 tx=0).stop 2024-09-10T08:40:31.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.856+0000 7fa5af600640 1 -- 172.21.15.89:0/3416934119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5a81049a0 msgr2=0x7fa5a8076c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:31.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.856+0000 7fa5af600640 1 --2- 172.21.15.89:0/3416934119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5a81049a0 0x7fa5a8076c40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa594009230 tx=0x7fa594003b20 comp rx=0 tx=0).stop 2024-09-10T08:40:31.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.856+0000 7fa5af600640 1 -- 172.21.15.89:0/3416934119 shutdown_connections 2024-09-10T08:40:31.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.856+0000 7fa5af600640 1 --2- 172.21.15.89:0/3416934119 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5900a2c90 0x7fa5900a5150 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.856+0000 7fa5af600640 1 --2- 172.21.15.89:0/3416934119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5a8109120 0x7fa5a8077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.856+0000 7fa5af600640 1 --2- 172.21.15.89:0/3416934119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5a81049a0 0x7fa5a8076c40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.856+0000 7fa5af600640 1 --2- 172.21.15.89:0/3416934119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5a8103ff0 0x7fa5a8076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:31.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.856+0000 7fa5af600640 1 -- 172.21.15.89:0/3416934119 >> 172.21.15.89:0/3416934119 conn(0x7fa5a80ffa90 msgr2=0x7fa5a8100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:31.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.856+0000 7fa5af600640 1 -- 172.21.15.89:0/3416934119 shutdown_connections 2024-09-10T08:40:31.860 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:31.856+0000 7fa5af600640 1 -- 172.21.15.89:0/3416934119 wait complete. 2024-09-10T08:40:32.130 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:40:32.702 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:40:32.934 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:32 smithi089 ceph-mon[30757]: pgmap v411: 129 pgs: 129 active+clean; 664 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T08:40:32.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:32 smithi149 ceph-mon[36167]: pgmap v411: 129 pgs: 129 active+clean; 664 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T08:40:33.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:32 smithi111 ceph-mon[35220]: pgmap v411: 129 pgs: 129 active+clean; 664 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T08:40:33.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.223+0000 7f98cf600640 1 -- 172.21.15.89:0/3155464660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98c810e3c0 msgr2=0x7f98c8110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:33.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.223+0000 7f98cf600640 1 --2- 172.21.15.89:0/3155464660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98c810e3c0 0x7f98c8110810 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009ca0 tx=0x7f98c002a950 comp rx=0 tx=0).stop 2024-09-10T08:40:33.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.223+0000 7f98cf600640 1 -- 172.21.15.89:0/3155464660 shutdown_connections 2024-09-10T08:40:33.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.223+0000 7f98cf600640 1 --2- 172.21.15.89:0/3155464660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98c810e3c0 0x7f98c8110810 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.223+0000 7f98cf600640 1 --2- 172.21.15.89:0/3155464660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98c8074f40 0x7f98c810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.223+0000 7f98cf600640 1 --2- 172.21.15.89:0/3155464660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c8074620 0x7f98c8074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.223+0000 7f98cf600640 1 -- 172.21.15.89:0/3155464660 >> 172.21.15.89:0/3155464660 conn(0x7f98c80fd8c0 msgr2=0x7f98c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:33.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.223+0000 7f98cf600640 1 -- 172.21.15.89:0/3155464660 shutdown_connections 2024-09-10T08:40:33.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.223+0000 7f98cf600640 1 -- 172.21.15.89:0/3155464660 wait complete. 2024-09-10T08:40:33.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.224+0000 7f98cf600640 1 Processor -- start 2024-09-10T08:40:33.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.224+0000 7f98cf600640 1 -- start start 2024-09-10T08:40:33.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.225+0000 7f98cf600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c8074620 0x7f98c819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:33.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.225+0000 7f98cf600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98c8074f40 0x7f98c819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:33.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.225+0000 7f98cf600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98c810e3c0 0x7f98c819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:33.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.225+0000 7f98cf600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f98c8114170 con 0x7f98c8074f40 2024-09-10T08:40:33.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.225+0000 7f98cf600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f98c8113ff0 con 0x7f98c8074620 2024-09-10T08:40:33.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.225+0000 7f98cf600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81142f0 con 0x7f98c810e3c0 2024-09-10T08:40:33.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.225+0000 7f98c7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98c8074f40 0x7f98c819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:33.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.225+0000 7f98ccc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c8074620 0x7f98c819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:33.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.225+0000 7f98ccc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c8074620 0x7f98c819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:47916/0 (socket says 172.21.15.89:47916) 2024-09-10T08:40:33.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.225+0000 7f98ccc00640 1 -- 172.21.15.89:0/3303447807 learned_addr learned my addr 172.21.15.89:0/3303447807 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:33.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.225+0000 7f98cd600640 1 --2- 172.21.15.89:0/3303447807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98c810e3c0 0x7f98c819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:33.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.226+0000 7f98ccc00640 1 -- 172.21.15.89:0/3303447807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98c810e3c0 msgr2=0x7f98c819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:33.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.226+0000 7f98ccc00640 1 --2- 172.21.15.89:0/3303447807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98c810e3c0 0x7f98c819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.226+0000 7f98ccc00640 1 -- 172.21.15.89:0/3303447807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98c8074f40 msgr2=0x7f98c819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:33.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.226+0000 7f98ccc00640 1 --2- 172.21.15.89:0/3303447807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98c8074f40 0x7f98c819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.226+0000 7f98ccc00640 1 -- 172.21.15.89:0/3303447807 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c819f620 con 0x7f98c8074620 2024-09-10T08:40:33.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.226+0000 7f98ccc00640 1 --2- 172.21.15.89:0/3303447807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c8074620 0x7f98c819a670 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00e9f0 tx=0x7f98bc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:33.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.227+0000 7f98c5600640 1 -- 172.21.15.89:0/3303447807 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc00cd80 con 0x7f98c8074620 2024-09-10T08:40:33.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.227+0000 7f98c5600640 1 -- 172.21.15.89:0/3303447807 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98bc010910 con 0x7f98c8074620 2024-09-10T08:40:33.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.227+0000 7f98c5600640 1 -- 172.21.15.89:0/3303447807 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc018990 con 0x7f98c8074620 2024-09-10T08:40:33.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.227+0000 7f98cf600640 1 -- 172.21.15.89:0/3303447807 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c819f910 con 0x7f98c8074620 2024-09-10T08:40:33.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.228+0000 7f98cf600640 1 -- 172.21.15.89:0/3303447807 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c81a71f0 con 0x7f98c8074620 2024-09-10T08:40:33.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.231+0000 7f98d21fa740 1 Processor -- start 2024-09-10T08:40:33.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.231+0000 7f98d21fa740 1 -- start start 2024-09-10T08:40:33.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.231+0000 7f98d21fa740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca182d0 0x55b0dca186b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:33.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.231+0000 7f98d21fa740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca18bf0 0x55b0dca2b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:33.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.231+0000 7f98d21fa740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca2bda0 0x55b0dca2e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:33.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.231+0000 7f98d21fa740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b0dc97e530 con 0x55b0dca2bda0 2024-09-10T08:40:33.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.231+0000 7f98d21fa740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b0dc925860 con 0x55b0dca182d0 2024-09-10T08:40:33.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.231+0000 7f98d21fa740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b0dc97a4f0 con 0x55b0dca18bf0 2024-09-10T08:40:33.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.232+0000 7f98c7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca2bda0 0x55b0dca2e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:33.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.232+0000 7f98cd600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca182d0 0x55b0dca186b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:33.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.232+0000 7f98cd600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca182d0 0x55b0dca186b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:47932/0 (socket says 172.21.15.89:47932) 2024-09-10T08:40:33.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.232+0000 7f98cd600640 1 -- 172.21.15.89:0/2629955821 learned_addr learned my addr 172.21.15.89:0/2629955821 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:33.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.232+0000 7f98c5600640 1 -- 172.21.15.89:0/3303447807 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f98bc010ac0 con 0x7f98c8074620 2024-09-10T08:40:33.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.232+0000 7f98cd600640 1 -- 172.21.15.89:0/2629955821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca18bf0 msgr2=0x55b0dca2b860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:40:33.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.232+0000 7f98cd600640 1 --2- 172.21.15.89:0/2629955821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca18bf0 0x55b0dca2b860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.232+0000 7f98cd600640 1 -- 172.21.15.89:0/2629955821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca2bda0 msgr2=0x55b0dca2e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:33.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.232+0000 7f98cd600640 1 --2- 172.21.15.89:0/2629955821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca2bda0 0x55b0dca2e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.232+0000 7f98cd600640 1 -- 172.21.15.89:0/2629955821 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0dc6efb90 con 0x55b0dca182d0 2024-09-10T08:40:33.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.233+0000 7f98cd600640 1 --2- 172.21.15.89:0/2629955821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca182d0 0x55b0dca186b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f98c0035000 tx=0x7f98c00093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77f26ff03b5e304d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:33.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.233+0000 7f98b9c00640 1 -- 172.21.15.89:0/2629955821 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0034040 con 0x55b0dca182d0 2024-09-10T08:40:33.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.233+0000 7f98b9c00640 1 -- 172.21.15.89:0/2629955821 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98c002f940 con 0x55b0dca182d0 2024-09-10T08:40:33.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.233+0000 7f98b9c00640 1 -- 172.21.15.89:0/2629955821 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0004080 con 0x55b0dca182d0 2024-09-10T08:40:33.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.234+0000 7f98d21fa740 1 -- 172.21.15.89:0/2629955821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca182d0 msgr2=0x55b0dca186b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:33.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.234+0000 7f98d21fa740 1 --2- 172.21.15.89:0/2629955821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca182d0 0x55b0dca186b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f98c0035000 tx=0x7f98c00093e0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.234+0000 7f98d21fa740 1 -- 172.21.15.89:0/2629955821 shutdown_connections 2024-09-10T08:40:33.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.234+0000 7f98d21fa740 1 --2- 172.21.15.89:0/2629955821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca2bda0 0x55b0dca2e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.234+0000 7f98d21fa740 1 --2- 172.21.15.89:0/2629955821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca18bf0 0x55b0dca2b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.234+0000 7f98d21fa740 1 --2- 172.21.15.89:0/2629955821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca182d0 0x55b0dca186b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.234+0000 7f98d21fa740 1 -- 172.21.15.89:0/2629955821 >> 172.21.15.89:0/2629955821 conn(0x55b0dca14940 msgr2=0x55b0dca14d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:33.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.235+0000 7f98c5600640 1 --2- 172.21.15.89:0/3303447807 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f98b00a2c20 0x7f98b00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:33.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.234+0000 7f98d21fa740 1 -- 172.21.15.89:0/2629955821 shutdown_connections 2024-09-10T08:40:33.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.235+0000 7f98c5600640 1 -- 172.21.15.89:0/3303447807 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f98bc0cfa30 con 0x7f98c8074620 2024-09-10T08:40:33.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.235+0000 7f98d21fa740 1 -- 172.21.15.89:0/2629955821 wait complete. 2024-09-10T08:40:33.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.235+0000 7f98c7e00640 1 --2- 172.21.15.89:0/3303447807 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f98b00a2c20 0x7f98b00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:33.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.236+0000 7f98d21fa740 1 -- start start 2024-09-10T08:40:33.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.236+0000 7f98d21fa740 1 Processor -- start 2024-09-10T08:40:33.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.237+0000 7f98d21fa740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca182d0 0x55b0dca28de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:33.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.237+0000 7f98d21fa740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca18bf0 0x55b0dca29320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:33.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.237+0000 7f98d21fa740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca2bda0 0x55b0dca29860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:33.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.237+0000 7f98d21fa740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b0dca2e8b0 con 0x55b0dca2bda0 2024-09-10T08:40:33.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.237+0000 7f98d21fa740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b0dc938e60 con 0x55b0dca182d0 2024-09-10T08:40:33.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.237+0000 7f98d21fa740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b0dca2ea30 con 0x55b0dca18bf0 2024-09-10T08:40:33.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.237+0000 7f98d21fa740 10 client.0 _ll_register_callbacks cb 0x55b0dc93dcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:33.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.237+0000 7f98c7e00640 1 --2- 172.21.15.89:0/3303447807 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f98b00a2c20 0x7f98b00a50e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f98c810f750 tx=0x7f98b4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:33.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.237+0000 7f98ccc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca182d0 0x55b0dca28de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:33.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.237+0000 7f98ccc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca182d0 0x55b0dca28de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:47938/0 (socket says 172.21.15.89:47938) 2024-09-10T08:40:33.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.237+0000 7f98ccc00640 1 -- 172.21.15.89:0/3027487677 learned_addr learned my addr 172.21.15.89:0/3027487677 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:33.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.238+0000 7f98cd600640 1 --2- 172.21.15.89:0/3027487677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca2bda0 0x55b0dca29860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:33.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.238+0000 7f98c7e00640 1 --2- 172.21.15.89:0/3027487677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca18bf0 0x55b0dca29320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:33.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.238+0000 7f98c7e00640 1 -- 172.21.15.89:0/3027487677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca182d0 msgr2=0x55b0dca28de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:33.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.238+0000 7f98c7e00640 1 --2- 172.21.15.89:0/3027487677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca182d0 0x55b0dca28de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.238+0000 7f98c7e00640 1 -- 172.21.15.89:0/3027487677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca2bda0 msgr2=0x55b0dca29860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:33.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.238+0000 7f98c7e00640 1 --2- 172.21.15.89:0/3027487677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca2bda0 0x55b0dca29860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.238+0000 7f98c7e00640 1 -- 172.21.15.89:0/3027487677 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0dc944990 con 0x55b0dca18bf0 2024-09-10T08:40:33.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.239+0000 7f98cd600640 1 --2- 172.21.15.89:0/3027487677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca2bda0 0x55b0dca29860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:33.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.239+0000 7f98c7e00640 1 --2- 172.21.15.89:0/3027487677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca18bf0 0x55b0dca29320 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f98b40128e0 tx=0x7f98b4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8641962e5ed1488d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:33.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.239+0000 7f98ae000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:40:33.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.239+0000 7f98ae000640 1 -- 172.21.15.89:0/3027487677 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4003e40 con 0x55b0dca18bf0 2024-09-10T08:40:33.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.239+0000 7f98ae000640 1 -- 172.21.15.89:0/3027487677 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f98b4005020 con 0x55b0dca18bf0 2024-09-10T08:40:33.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.239+0000 7f98d21fa740 1 -- 172.21.15.89:0/3027487677 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98c002f940 con 0x55b0dca18bf0 2024-09-10T08:40:33.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.240+0000 7f98ae000640 1 -- 172.21.15.89:0/3027487677 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f98b4005220 con 0x55b0dca18bf0 2024-09-10T08:40:33.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.240+0000 7f98d21fa740 10 client.25456 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:33.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.240+0000 7f98d21fa740 1 -- 172.21.15.89:0/3027487677 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b0dca3a6a0 con 0x55b0dca18bf0 2024-09-10T08:40:33.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.240+0000 7f98ae000640 1 -- 172.21.15.89:0/3027487677 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98c002f940 con 0x55b0dca18bf0 2024-09-10T08:40:33.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.241+0000 7f98ae000640 1 -- 172.21.15.89:0/3027487677 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f98b4027070 con 0x55b0dca18bf0 2024-09-10T08:40:33.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.241+0000 7f98d21fa740 10 client.25456 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:33.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.241+0000 7f98d21fa740 10 client.25456 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:40:33.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.241+0000 7f98d21fa740 20 client.25456 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:33.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.241+0000 7f98d21fa740 1 --2- 172.21.15.89:0/3027487677 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b0dca3bc00 0x55b0dca3e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:33.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.241+0000 7f98d21fa740 4 client.25456 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:40:33.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.241+0000 7f98d21fa740 1 -- 172.21.15.89:0/3027487677 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0dc921310 con 0x55b0dca3bc00 2024-09-10T08:40:33.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.241+0000 7f98ccc00640 1 --2- 172.21.15.89:0/3027487677 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b0dca3bc00 0x55b0dca3e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:33.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.242+0000 7f98ccc00640 1 --2- 172.21.15.89:0/3027487677 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b0dca3bc00 0x55b0dca3e020 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc17e7cd1a89f808 server_cookie=109ca4007533fc56 in_seq=0 out_seq=0 2024-09-10T08:40:33.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.242+0000 7f98ae000640 10 client.25456 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:33.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.243+0000 7f98ae000640 1 -- 172.21.15.89:0/3027487677 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b0dc921310 con 0x55b0dca3bc00 2024-09-10T08:40:33.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.243+0000 7f98ae000640 10 client.25456 handle_command_reply: tid=0 2024-09-10T08:40:33.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.243+0000 7f98d21fa740 1 client.25456 shutdown 2024-09-10T08:40:33.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.243+0000 7f98d21fa740 1 -- 172.21.15.89:0/3027487677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca18bf0 msgr2=0x55b0dca29320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:33.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.243+0000 7f98d21fa740 1 --2- 172.21.15.89:0/3027487677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca18bf0 0x55b0dca29320 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f98b40128e0 tx=0x7f98b4004730 comp rx=0 tx=0).stop 2024-09-10T08:40:33.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.243+0000 7f98d21fa740 1 -- 172.21.15.89:0/3027487677 shutdown_connections 2024-09-10T08:40:33.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.243+0000 7f98d21fa740 1 --2- 172.21.15.89:0/3027487677 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b0dca3bc00 0x55b0dca3e020 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.243+0000 7f98ae000640 0 client.25456 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:33.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.243+0000 7f98d21fa740 1 --2- 172.21.15.89:0/3027487677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca2bda0 0x55b0dca29860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.243+0000 7f98d21fa740 1 --2- 172.21.15.89:0/3027487677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca18bf0 0x55b0dca29320 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.243+0000 7f98d21fa740 1 --2- 172.21.15.89:0/3027487677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca182d0 0x55b0dca28de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.243+0000 7f98d21fa740 1 -- 172.21.15.89:0/3027487677 >> 172.21.15.89:0/3027487677 conn(0x55b0dca14940 msgr2=0x55b0dca13c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:33.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.244+0000 7f98d21fa740 1 -- 172.21.15.89:0/3027487677 shutdown_connections 2024-09-10T08:40:33.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.244+0000 7f98d21fa740 1 -- 172.21.15.89:0/3027487677 wait complete. 2024-09-10T08:40:33.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.244+0000 7f98d21fa740 20 client.25456 trim_cache size 0 max 16384 2024-09-10T08:40:33.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.267+0000 7f98d21fa740 1 Processor -- start 2024-09-10T08:40:33.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.267+0000 7f98d21fa740 1 -- start start 2024-09-10T08:40:33.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.267+0000 7f98d21fa740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca182d0 0x55b0dca296b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:33.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.268+0000 7f98d21fa740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca18bf0 0x55b0dca438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:33.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.268+0000 7f98d21fa740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca2bda0 0x55b0dca43e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:33.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.268+0000 7f98d21fa740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81af590 con 0x55b0dca18bf0 2024-09-10T08:40:33.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.268+0000 7f98d21fa740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81ace60 con 0x55b0dca182d0 2024-09-10T08:40:33.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.268+0000 7f98d21fa740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81aacb0 con 0x55b0dca2bda0 2024-09-10T08:40:33.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.268+0000 7f98ccc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca182d0 0x55b0dca296b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:33.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.268+0000 7f98ccc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca182d0 0x55b0dca296b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:47946/0 (socket says 172.21.15.89:47946) 2024-09-10T08:40:33.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.268+0000 7f98ccc00640 1 -- 172.21.15.89:0/2674272183 learned_addr learned my addr 172.21.15.89:0/2674272183 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:33.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.268+0000 7f98c7e00640 1 --2- 172.21.15.89:0/2674272183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca18bf0 0x55b0dca438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:33.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.269+0000 7f98cd600640 1 --2- 172.21.15.89:0/2674272183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca2bda0 0x55b0dca43e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:33.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.269+0000 7f98c7e00640 1 -- 172.21.15.89:0/2674272183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca2bda0 msgr2=0x55b0dca43e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:33.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.269+0000 7f98c7e00640 1 --2- 172.21.15.89:0/2674272183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca2bda0 0x55b0dca43e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.269+0000 7f98c7e00640 1 -- 172.21.15.89:0/2674272183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca182d0 msgr2=0x55b0dca296b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:33.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.269+0000 7f98c7e00640 1 --2- 172.21.15.89:0/2674272183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca182d0 0x55b0dca296b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.269+0000 7f98c7e00640 1 -- 172.21.15.89:0/2674272183 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0dca1ad30 con 0x55b0dca18bf0 2024-09-10T08:40:33.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.269+0000 7f98cd600640 1 --2- 172.21.15.89:0/2674272183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca2bda0 0x55b0dca43e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:40:33.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.270+0000 7f98c7e00640 1 --2- 172.21.15.89:0/2674272183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca18bf0 0x55b0dca438f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f98b401f710 tx=0x7f98b4009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e92adbe4554263 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:33.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.270+0000 7f98b9200640 1 -- 172.21.15.89:0/2674272183 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4003e40 con 0x55b0dca18bf0 2024-09-10T08:40:33.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.270+0000 7f98b9200640 1 -- 172.21.15.89:0/2674272183 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98b401f900 con 0x55b0dca18bf0 2024-09-10T08:40:33.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.270+0000 7f98b9200640 1 -- 172.21.15.89:0/2674272183 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4005020 con 0x55b0dca18bf0 2024-09-10T08:40:33.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.271+0000 7f98d21fa740 1 -- 172.21.15.89:0/2674272183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca18bf0 msgr2=0x55b0dca438f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:33.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.271+0000 7f98d21fa740 1 --2- 172.21.15.89:0/2674272183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca18bf0 0x55b0dca438f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f98b401f710 tx=0x7f98b4009a00 comp rx=0 tx=0).stop 2024-09-10T08:40:33.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.271+0000 7f98d21fa740 1 -- 172.21.15.89:0/2674272183 shutdown_connections 2024-09-10T08:40:33.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.271+0000 7f98d21fa740 1 --2- 172.21.15.89:0/2674272183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca2bda0 0x55b0dca43e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.271+0000 7f98d21fa740 1 --2- 172.21.15.89:0/2674272183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca18bf0 0x55b0dca438f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.271+0000 7f98d21fa740 1 --2- 172.21.15.89:0/2674272183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca182d0 0x55b0dca296b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.271+0000 7f98d21fa740 1 -- 172.21.15.89:0/2674272183 >> 172.21.15.89:0/2674272183 conn(0x55b0dca14940 msgr2=0x55b0dca27da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:33.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.271+0000 7f98d21fa740 1 -- 172.21.15.89:0/2674272183 shutdown_connections 2024-09-10T08:40:33.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.272+0000 7f98d21fa740 1 -- 172.21.15.89:0/2674272183 wait complete. 2024-09-10T08:40:33.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.272+0000 7f98d21fa740 1 -- start start 2024-09-10T08:40:33.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.273+0000 7f98d21fa740 1 Processor -- start 2024-09-10T08:40:33.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.273+0000 7f98d21fa740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca182d0 0x55b0dca33cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:33.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.273+0000 7f98d21fa740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca18bf0 0x55b0dca34200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:33.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.273+0000 7f98d21fa740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca2bda0 0x55b0dca9fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:33.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.273+0000 7f98d21fa740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81ab320 con 0x55b0dca182d0 2024-09-10T08:40:33.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.273+0000 7f98d21fa740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81ac790 con 0x55b0dca18bf0 2024-09-10T08:40:33.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.273+0000 7f98d21fa740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81ab9f0 con 0x55b0dca2bda0 2024-09-10T08:40:33.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.273+0000 7f98d21fa740 10 client.0 _ll_register_callbacks cb 0x7f98b4009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:33.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.274+0000 7f98cd600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca2bda0 0x55b0dca9fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:33.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.274+0000 7f98c7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca18bf0 0x55b0dca34200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:33.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.274+0000 7f98c7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca18bf0 0x55b0dca34200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:47958/0 (socket says 172.21.15.89:47958) 2024-09-10T08:40:33.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.274+0000 7f98c7e00640 1 -- 172.21.15.89:0/4116461250 learned_addr learned my addr 172.21.15.89:0/4116461250 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:33.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.274+0000 7f98ccc00640 1 --2- 172.21.15.89:0/4116461250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca182d0 0x55b0dca33cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:33.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.274+0000 7f98c7e00640 1 -- 172.21.15.89:0/4116461250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca2bda0 msgr2=0x55b0dca9fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:33.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.274+0000 7f98c7e00640 1 --2- 172.21.15.89:0/4116461250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca2bda0 0x55b0dca9fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.274+0000 7f98c7e00640 1 -- 172.21.15.89:0/4116461250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca182d0 msgr2=0x55b0dca33cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:33.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.274+0000 7f98c7e00640 1 --2- 172.21.15.89:0/4116461250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca182d0 0x55b0dca33cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.274+0000 7f98c7e00640 1 -- 172.21.15.89:0/4116461250 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0dcaa0000 con 0x55b0dca18bf0 2024-09-10T08:40:33.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.275+0000 7f98c7e00640 1 --2- 172.21.15.89:0/4116461250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca18bf0 0x55b0dca34200 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f98b401f8d0 tx=0x7f98b4003e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba2561e66fea0c42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:33.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.275+0000 7f98ae000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:40:33.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.275+0000 7f98ae000640 1 -- 172.21.15.89:0/4116461250 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4045060 con 0x55b0dca18bf0 2024-09-10T08:40:33.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.275+0000 7f98ae000640 1 -- 172.21.15.89:0/4116461250 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f98b4004920 con 0x55b0dca18bf0 2024-09-10T08:40:33.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.275+0000 7f98d21fa740 1 -- 172.21.15.89:0/4116461250 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98b401f900 con 0x55b0dca18bf0 2024-09-10T08:40:33.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.276+0000 7f98ae000640 1 -- 172.21.15.89:0/4116461250 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f98b404c050 con 0x55b0dca18bf0 2024-09-10T08:40:33.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.276+0000 7f98d21fa740 10 client.25502 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:33.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.276+0000 7f98d21fa740 1 -- 172.21.15.89:0/4116461250 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b0dca1f670 con 0x55b0dca18bf0 2024-09-10T08:40:33.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.276+0000 7f98ae000640 1 -- 172.21.15.89:0/4116461250 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98b401f900 con 0x55b0dca18bf0 2024-09-10T08:40:33.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.276+0000 7f98ae000640 1 -- 172.21.15.89:0/4116461250 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f98b403f070 con 0x55b0dca18bf0 2024-09-10T08:40:33.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.276+0000 7f98d21fa740 10 client.25502 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:33.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.276+0000 7f98d21fa740 10 client.25502 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:40:33.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.276+0000 7f98d21fa740 20 client.25502 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:33.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.276+0000 7f98d21fa740 1 --2- 172.21.15.89:0/4116461250 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b0dca1fe30 0x55b0dca22250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:33.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.276+0000 7f98d21fa740 4 client.25502 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T08:40:33.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.276+0000 7f98d21fa740 1 -- 172.21.15.89:0/4116461250 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55b0dc98bc60 con 0x55b0dca1fe30 2024-09-10T08:40:33.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.277+0000 7f98ccc00640 1 --2- 172.21.15.89:0/4116461250 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b0dca1fe30 0x55b0dca22250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:33.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.277+0000 7f98ccc00640 1 --2- 172.21.15.89:0/4116461250 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b0dca1fe30 0x55b0dca22250 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c429bcd48fb4aca1 server_cookie=3c3c02391e7cd235 in_seq=0 out_seq=0 2024-09-10T08:40:33.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.277+0000 7f98ae000640 10 client.25502 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:33.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.289+0000 7f98ae000640 1 -- 172.21.15.89:0/4116461250 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55b0dc98bc60 con 0x55b0dca1fe30 2024-09-10T08:40:33.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.289+0000 7f98ae000640 10 client.25502 handle_command_reply: tid=0 2024-09-10T08:40:33.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.289+0000 7f98d21fa740 1 client.25502 shutdown 2024-09-10T08:40:33.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.290+0000 7f98d21fa740 1 -- 172.21.15.89:0/4116461250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca18bf0 msgr2=0x55b0dca34200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:33.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.290+0000 7f98d21fa740 1 --2- 172.21.15.89:0/4116461250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca18bf0 0x55b0dca34200 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f98b401f8d0 tx=0x7f98b4003e40 comp rx=0 tx=0).stop 2024-09-10T08:40:33.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.290+0000 7f98d21fa740 1 -- 172.21.15.89:0/4116461250 shutdown_connections 2024-09-10T08:40:33.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.290+0000 7f98d21fa740 1 --2- 172.21.15.89:0/4116461250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0dca2bda0 0x55b0dca9fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.290+0000 7f98d21fa740 1 --2- 172.21.15.89:0/4116461250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0dca18bf0 0x55b0dca34200 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.290+0000 7f98d21fa740 1 --2- 172.21.15.89:0/4116461250 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b0dca1fe30 0x55b0dca22250 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.290+0000 7f98d21fa740 1 --2- 172.21.15.89:0/4116461250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0dca182d0 0x55b0dca33cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.290+0000 7f98d21fa740 1 -- 172.21.15.89:0/4116461250 >> 172.21.15.89:0/4116461250 conn(0x55b0dca14940 msgr2=0x55b0dca19d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.290+0000 7f98ae000640 0 client.25502 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.290+0000 7f98d21fa740 1 -- 172.21.15.89:0/4116461250 shutdown_connections 2024-09-10T08:40:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.290+0000 7f98d21fa740 1 -- 172.21.15.89:0/4116461250 wait complete. 2024-09-10T08:40:33.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.291+0000 7f98d21fa740 20 client.25502 trim_cache size 0 max 16384 2024-09-10T08:40:33.292 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:40:33.292 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 26, 2024-09-10T08:40:33.292 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24554, 2024-09-10T08:40:33.292 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 2, 2024-09-10T08:40:33.292 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": { 2024-09-10T08:40:33.292 INFO:teuthology.orchestra.run.smithi089.stdout: "9590b514": { 2024-09-10T08:40:33.292 INFO:teuthology.orchestra.run.smithi089.stdout: "version": 2, 2024-09-10T08:40:33.292 INFO:teuthology.orchestra.run.smithi089.stdout: "age_ref": 0.0, 2024-09-10T08:40:33.293 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:40:33.293 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCED", 2024-09-10T08:40:33.293 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:40:33.293 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:40:33.293 INFO:teuthology.orchestra.run.smithi089.stdout: "timeout": 90.0, 2024-09-10T08:40:33.293 INFO:teuthology.orchestra.run.smithi089.stdout: "expiration": 150.0, 2024-09-10T08:40:33.293 INFO:teuthology.orchestra.run.smithi089.stdout: "members": { 2024-09-10T08:40:33.293 INFO:teuthology.orchestra.run.smithi089.stdout: "file:/": { 2024-09-10T08:40:33.293 INFO:teuthology.orchestra.run.smithi089.stdout: "excluded": false, 2024-09-10T08:40:33.293 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:40:33.293 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCED", 2024-09-10T08:40:33.293 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:40:33.293 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:40:33.294 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:40:33.294 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:40:33.294 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:40:33.294 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:40:33.294 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:40:33.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.292+0000 7f98cf600640 1 -- 172.21.15.89:0/3303447807 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f98b00a2c20 msgr2=0x7f98b00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:33.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.292+0000 7f98cf600640 1 --2- 172.21.15.89:0/3303447807 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f98b00a2c20 0x7f98b00a50e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f98c810f750 tx=0x7f98b4005f90 comp rx=0 tx=0).stop 2024-09-10T08:40:33.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.292+0000 7f98cf600640 1 -- 172.21.15.89:0/3303447807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c8074620 msgr2=0x7f98c819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:33.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.292+0000 7f98cf600640 1 --2- 172.21.15.89:0/3303447807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c8074620 0x7f98c819a670 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00e9f0 tx=0x7f98bc00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.292+0000 7f98cf600640 1 -- 172.21.15.89:0/3303447807 shutdown_connections 2024-09-10T08:40:33.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.292+0000 7f98cf600640 1 --2- 172.21.15.89:0/3303447807 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f98b00a2c20 0x7f98b00a50e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.292+0000 7f98cf600640 1 --2- 172.21.15.89:0/3303447807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98c810e3c0 0x7f98c819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.292+0000 7f98cf600640 1 --2- 172.21.15.89:0/3303447807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98c8074f40 0x7f98c819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.292+0000 7f98cf600640 1 --2- 172.21.15.89:0/3303447807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c8074620 0x7f98c819a670 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:33.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.292+0000 7f98cf600640 1 -- 172.21.15.89:0/3303447807 >> 172.21.15.89:0/3303447807 conn(0x7f98c80fd8c0 msgr2=0x7f98c80fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:33.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.292+0000 7f98cf600640 1 -- 172.21.15.89:0/3303447807 shutdown_connections 2024-09-10T08:40:33.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:33.292+0000 7f98cf600640 1 -- 172.21.15.89:0/3303447807 wait complete. 2024-09-10T08:40:34.116 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:40:34.144 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9590b514', quiesce took 5.8 seconds. Will release after: 24.2 2024-09-10T08:40:34.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.698+0000 7fadafc00640 1 -- 172.21.15.89:0/3653256180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fada8104010 msgr2=0x7fada81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:34.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.698+0000 7fadafc00640 1 --2- 172.21.15.89:0/3653256180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fada8104010 0x7fada81043f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fad9c009900 tx=0x7fad9c02a410 comp rx=0 tx=0).stop 2024-09-10T08:40:34.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.698+0000 7fadafc00640 1 -- 172.21.15.89:0/3653256180 shutdown_connections 2024-09-10T08:40:34.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.698+0000 7fadafc00640 1 --2- 172.21.15.89:0/3653256180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fada8109140 0x7fada8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.698+0000 7fadafc00640 1 --2- 172.21.15.89:0/3653256180 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fada81049c0 0x7fada8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.698+0000 7fadafc00640 1 --2- 172.21.15.89:0/3653256180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fada8104010 0x7fada81043f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.698+0000 7fadafc00640 1 -- 172.21.15.89:0/3653256180 >> 172.21.15.89:0/3653256180 conn(0x7fada80ffa50 msgr2=0x7fada8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:34.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.698+0000 7fadafc00640 1 -- 172.21.15.89:0/3653256180 shutdown_connections 2024-09-10T08:40:34.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.698+0000 7fadafc00640 1 -- 172.21.15.89:0/3653256180 wait complete. 2024-09-10T08:40:34.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.699+0000 7fadafc00640 1 Processor -- start 2024-09-10T08:40:34.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.699+0000 7fadafc00640 1 -- start start 2024-09-10T08:40:34.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.700+0000 7fadafc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fada8104010 0x7fada819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:34.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.700+0000 7fadafc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fada81049c0 0x7fada819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:34.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.700+0000 7fadafc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fada8109140 0x7fada819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:34.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.700+0000 7fadafc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fada8114140 con 0x7fada8104010 2024-09-10T08:40:34.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.700+0000 7fadafc00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fada8113fc0 con 0x7fada81049c0 2024-09-10T08:40:34.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.700+0000 7fadafc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fada81142c0 con 0x7fada8109140 2024-09-10T08:40:34.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.700+0000 7fadad200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fada8104010 0x7fada819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:34.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.700+0000 7fadad200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fada8104010 0x7fada819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47104/0 (socket says 172.21.15.89:47104) 2024-09-10T08:40:34.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.700+0000 7fadad200640 1 -- 172.21.15.89:0/1122778818 learned_addr learned my addr 172.21.15.89:0/1122778818 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:34.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.700+0000 7fada7e00640 1 --2- 172.21.15.89:0/1122778818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fada81049c0 0x7fada819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:34.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.700+0000 7fadadc00640 1 --2- 172.21.15.89:0/1122778818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fada8109140 0x7fada819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:34.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.700+0000 7fadad200640 1 -- 172.21.15.89:0/1122778818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fada8109140 msgr2=0x7fada819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:34.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.700+0000 7fadad200640 1 --2- 172.21.15.89:0/1122778818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fada8109140 0x7fada819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.700+0000 7fadad200640 1 -- 172.21.15.89:0/1122778818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fada81049c0 msgr2=0x7fada819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:34.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.700+0000 7fadad200640 1 --2- 172.21.15.89:0/1122778818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fada81049c0 0x7fada819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.700+0000 7fadad200640 1 -- 172.21.15.89:0/1122778818 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada819f5b0 con 0x7fada8104010 2024-09-10T08:40:34.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.700+0000 7fadadc00640 1 --2- 172.21.15.89:0/1122778818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fada8109140 0x7fada819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:40:34.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.701+0000 7fadad200640 1 --2- 172.21.15.89:0/1122778818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fada8104010 0x7fada819a600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fad9c006010 tx=0x7fad9c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:34.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.701+0000 7fada5600640 1 -- 172.21.15.89:0/1122778818 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c03f030 con 0x7fada8104010 2024-09-10T08:40:34.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.701+0000 7fada5600640 1 -- 172.21.15.89:0/1122778818 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad9c004440 con 0x7fada8104010 2024-09-10T08:40:34.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.701+0000 7fadafc00640 1 -- 172.21.15.89:0/1122778818 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada819f840 con 0x7fada8104010 2024-09-10T08:40:34.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.701+0000 7fada5600640 1 -- 172.21.15.89:0/1122778818 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c035070 con 0x7fada8104010 2024-09-10T08:40:34.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.701+0000 7fadafc00640 1 -- 172.21.15.89:0/1122778818 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada81a7180 con 0x7fada8104010 2024-09-10T08:40:34.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.705+0000 7fada5600640 1 -- 172.21.15.89:0/1122778818 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fad9c042030 con 0x7fada8104010 2024-09-10T08:40:34.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.705+0000 7fadb25fb740 1 Processor -- start 2024-09-10T08:40:34.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.705+0000 7fadb25fb740 1 -- start start 2024-09-10T08:40:34.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.705+0000 7fadb25fb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856777c70 0x557856778050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:34.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.705+0000 7fadb25fb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 0x557856782910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:34.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.705+0000 7fadb25fb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 0x557856785240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:34.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.705+0000 7fadb25fb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5578566de3e0 con 0x557856782e50 2024-09-10T08:40:34.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.705+0000 7fadb25fb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557856685d80 con 0x557856777c70 2024-09-10T08:40:34.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.705+0000 7fadb25fb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5578566da3a0 con 0x557856778590 2024-09-10T08:40:34.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.706+0000 7fadad200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 0x557856782910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:34.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.706+0000 7fadadc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856777c70 0x557856778050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:34.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.706+0000 7fadad200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 0x557856782910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:34676/0 (socket says 172.21.15.89:34676) 2024-09-10T08:40:34.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.706+0000 7fadad200640 1 -- 172.21.15.89:0/1475282124 learned_addr learned my addr 172.21.15.89:0/1475282124 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:34.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.706+0000 7fadadc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856777c70 0x557856778050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:47992/0 (socket says 172.21.15.89:47992) 2024-09-10T08:40:34.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.706+0000 7fada7e00640 1 --2- 172.21.15.89:0/1475282124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 0x557856785240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:34.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.706+0000 7fadad200640 1 -- 172.21.15.89:0/1475282124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856777c70 msgr2=0x557856778050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:34.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.706+0000 7fadad200640 1 --2- 172.21.15.89:0/1475282124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856777c70 0x557856778050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.706+0000 7fadad200640 1 -- 172.21.15.89:0/1475282124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 msgr2=0x557856785240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:34.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.706+0000 7fadad200640 1 --2- 172.21.15.89:0/1475282124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 0x557856785240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.706+0000 7fadad200640 1 -- 172.21.15.89:0/1475282124 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578566a4ae0 con 0x557856778590 2024-09-10T08:40:34.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.706+0000 7fada5600640 1 --2- 172.21.15.89:0/1122778818 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fad900a2c10 0x7fad900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:34.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.706+0000 7fadadc00640 1 --2- 172.21.15.89:0/1475282124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856777c70 0x557856778050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:40:34.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.707+0000 7fada5600640 1 -- 172.21.15.89:0/1122778818 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fad9c0ef9b0 con 0x7fada8104010 2024-09-10T08:40:34.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.707+0000 7fadadc00640 1 --2- 172.21.15.89:0/1122778818 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fad900a2c10 0x7fad900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:34.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.707+0000 7fadad200640 1 --2- 172.21.15.89:0/1475282124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 0x557856782910 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0eced0 tx=0x7fad9c0f5cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=924efa0db4cb9205 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:34.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.707+0000 7fad9a000640 1 -- 172.21.15.89:0/1475282124 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c049020 con 0x557856778590 2024-09-10T08:40:34.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.707+0000 7fad9a000640 1 -- 172.21.15.89:0/1475282124 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad9c0bc3e0 con 0x557856778590 2024-09-10T08:40:34.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.707+0000 7fad9a000640 1 -- 172.21.15.89:0/1475282124 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c0b7f80 con 0x557856778590 2024-09-10T08:40:34.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.708+0000 7fadadc00640 1 --2- 172.21.15.89:0/1122778818 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fad900a2c10 0x7fad900a50d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fada0005fd0 tx=0x7fada0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:34.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.708+0000 7fadb25fb740 1 -- 172.21.15.89:0/1475282124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 msgr2=0x557856782910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:34.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.708+0000 7fadb25fb740 1 --2- 172.21.15.89:0/1475282124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 0x557856782910 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0eced0 tx=0x7fad9c0f5cd0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.708+0000 7fadb25fb740 1 -- 172.21.15.89:0/1475282124 shutdown_connections 2024-09-10T08:40:34.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.708+0000 7fadb25fb740 1 --2- 172.21.15.89:0/1475282124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 0x557856785240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.708+0000 7fadb25fb740 1 --2- 172.21.15.89:0/1475282124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 0x557856782910 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.708+0000 7fadb25fb740 1 --2- 172.21.15.89:0/1475282124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856777c70 0x557856778050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.708+0000 7fadb25fb740 1 -- 172.21.15.89:0/1475282124 >> 172.21.15.89:0/1475282124 conn(0x557856774200 msgr2=0x5578567745e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:34.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.708+0000 7fadb25fb740 1 -- 172.21.15.89:0/1475282124 shutdown_connections 2024-09-10T08:40:34.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.708+0000 7fadb25fb740 1 -- 172.21.15.89:0/1475282124 wait complete. 2024-09-10T08:40:34.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.709+0000 7fadb25fb740 1 -- start start 2024-09-10T08:40:34.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.709+0000 7fadb25fb740 1 Processor -- start 2024-09-10T08:40:34.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.710+0000 7fadb25fb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856777c70 0x55785677fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:34.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.710+0000 7fadb25fb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 0x5578567801e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:34.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.710+0000 7fadb25fb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 0x557856780720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:34.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.710+0000 7fadb25fb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5578567857e0 con 0x557856782e50 2024-09-10T08:40:34.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.710+0000 7fadb25fb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5578566990f0 con 0x557856777c70 2024-09-10T08:40:34.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.710+0000 7fadb25fb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557856785960 con 0x557856778590 2024-09-10T08:40:34.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.710+0000 7fadb25fb740 10 client.0 _ll_register_callbacks cb 0x557856680bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:34.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.710+0000 7fada7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 0x5578567801e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:34.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.710+0000 7fadad200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856777c70 0x55785677fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:34.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.710+0000 7fada7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 0x5578567801e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:34690/0 (socket says 172.21.15.89:34690) 2024-09-10T08:40:34.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.710+0000 7fadad200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856777c70 0x55785677fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:48008/0 (socket says 172.21.15.89:48008) 2024-09-10T08:40:34.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.710+0000 7fada7e00640 1 -- 172.21.15.89:0/2886981901 learned_addr learned my addr 172.21.15.89:0/2886981901 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:34.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.710+0000 7fadadc00640 1 --2- 172.21.15.89:0/2886981901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 0x557856780720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:34.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.711+0000 7fada7e00640 1 -- 172.21.15.89:0/2886981901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856777c70 msgr2=0x55785677fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:34.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.711+0000 7fada7e00640 1 --2- 172.21.15.89:0/2886981901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856777c70 0x55785677fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.711+0000 7fada7e00640 1 -- 172.21.15.89:0/2886981901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 msgr2=0x557856780720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:34.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.711+0000 7fada7e00640 1 --2- 172.21.15.89:0/2886981901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 0x557856780720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.711+0000 7fada7e00640 1 -- 172.21.15.89:0/2886981901 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55785644f3c0 con 0x557856778590 2024-09-10T08:40:34.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.711+0000 7fadadc00640 1 --2- 172.21.15.89:0/2886981901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 0x557856780720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:40:34.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.711+0000 7fada7e00640 1 --2- 172.21.15.89:0/2886981901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 0x5578567801e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fad94012700 tx=0x7fad94012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80310dc1283a0bd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:34.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.711+0000 7fad8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:40:34.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.711+0000 7fad8ea00640 1 -- 172.21.15.89:0/2886981901 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9400f8c0 con 0x557856778590 2024-09-10T08:40:34.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.711+0000 7fadb25fb740 1 -- 172.21.15.89:0/2886981901 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad9c0bc3e0 con 0x557856778590 2024-09-10T08:40:34.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.711+0000 7fad8ea00640 1 -- 172.21.15.89:0/2886981901 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fad9400fa60 con 0x557856778590 2024-09-10T08:40:34.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.712+0000 7fad8ea00640 1 -- 172.21.15.89:0/2886981901 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fad94025050 con 0x557856778590 2024-09-10T08:40:34.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.712+0000 7fadb25fb740 10 client.25474 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:34.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.712+0000 7fadb25fb740 1 -- 172.21.15.89:0/2886981901 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557856791750 con 0x557856778590 2024-09-10T08:40:34.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.712+0000 7fad8ea00640 1 -- 172.21.15.89:0/2886981901 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad9c0bc3e0 con 0x557856778590 2024-09-10T08:40:34.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.712+0000 7fad8ea00640 1 -- 172.21.15.89:0/2886981901 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fad94010430 con 0x557856778590 2024-09-10T08:40:34.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.712+0000 7fadb25fb740 10 client.25474 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:34.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.712+0000 7fadb25fb740 10 client.25474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:40:34.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.712+0000 7fadb25fb740 20 client.25474 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:34.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.712+0000 7fadb25fb740 1 --2- 172.21.15.89:0/2886981901 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x557856792c60 0x5578567950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:34.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.712+0000 7fadb25fb740 4 client.25474 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:40:34.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.712+0000 7fadb25fb740 1 -- 172.21.15.89:0/2886981901 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578566eb950 con 0x557856792c60 2024-09-10T08:40:34.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.713+0000 7fadad200640 1 --2- 172.21.15.89:0/2886981901 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x557856792c60 0x5578567950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:34.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.713+0000 7fadad200640 1 --2- 172.21.15.89:0/2886981901 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x557856792c60 0x5578567950a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e714f3bb84cb25de server_cookie=95b2225fed1774d8 in_seq=0 out_seq=0 2024-09-10T08:40:34.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.713+0000 7fad8ea00640 10 client.25474 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:34.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.714+0000 7fad8ea00640 1 -- 172.21.15.89:0/2886981901 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578566eb950 con 0x557856792c60 2024-09-10T08:40:34.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.714+0000 7fad8ea00640 10 client.25474 handle_command_reply: tid=0 2024-09-10T08:40:34.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.714+0000 7fadb25fb740 1 client.25474 shutdown 2024-09-10T08:40:34.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.715+0000 7fadb25fb740 1 -- 172.21.15.89:0/2886981901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 msgr2=0x5578567801e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:34.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.715+0000 7fadb25fb740 1 --2- 172.21.15.89:0/2886981901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 0x5578567801e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fad94012700 tx=0x7fad94012bc0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.715+0000 7fadb25fb740 1 -- 172.21.15.89:0/2886981901 shutdown_connections 2024-09-10T08:40:34.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.715+0000 7fadb25fb740 1 --2- 172.21.15.89:0/2886981901 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x557856792c60 0x5578567950a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.715+0000 7fadb25fb740 1 --2- 172.21.15.89:0/2886981901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 0x557856780720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.715+0000 7fadb25fb740 1 --2- 172.21.15.89:0/2886981901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 0x5578567801e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.715+0000 7fadb25fb740 1 --2- 172.21.15.89:0/2886981901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856777c70 0x55785677fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.715+0000 7fadb25fb740 1 -- 172.21.15.89:0/2886981901 >> 172.21.15.89:0/2886981901 conn(0x557856774200 msgr2=0x557856779ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:34.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.715+0000 7fad8ea00640 0 client.25474 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:34.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.715+0000 7fadb25fb740 1 -- 172.21.15.89:0/2886981901 shutdown_connections 2024-09-10T08:40:34.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.715+0000 7fadb25fb740 1 -- 172.21.15.89:0/2886981901 wait complete. 2024-09-10T08:40:34.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.715+0000 7fadb25fb740 20 client.25474 trim_cache size 0 max 16384 2024-09-10T08:40:34.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.729+0000 7fadb25fb740 1 Processor -- start 2024-09-10T08:40:34.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.730+0000 7fadb25fb740 1 -- start start 2024-09-10T08:40:34.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.730+0000 7fadb25fb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856777c70 0x55785677a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:34.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.730+0000 7fadb25fb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856778590 0x55785679a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:34.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.730+0000 7fadb25fb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 0x55785679aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:34.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.730+0000 7fadb25fb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fada81b38c0 con 0x557856782e50 2024-09-10T08:40:34.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.730+0000 7fadb25fb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fada81b1190 con 0x557856778590 2024-09-10T08:40:34.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.730+0000 7fadb25fb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fada81aefe0 con 0x557856777c70 2024-09-10T08:40:34.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.730+0000 7fada7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856778590 0x55785679a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:34.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.730+0000 7fada7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856778590 0x55785679a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:48016/0 (socket says 172.21.15.89:48016) 2024-09-10T08:40:34.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.730+0000 7fada7e00640 1 -- 172.21.15.89:0/4171922046 learned_addr learned my addr 172.21.15.89:0/4171922046 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:34.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.730+0000 7fadadc00640 1 --2- 172.21.15.89:0/4171922046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 0x55785679aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:34.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.730+0000 7fadad200640 1 --2- 172.21.15.89:0/4171922046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856777c70 0x55785677a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:34.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.731+0000 7fada7e00640 1 -- 172.21.15.89:0/4171922046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856777c70 msgr2=0x55785677a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:34.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.731+0000 7fada7e00640 1 --2- 172.21.15.89:0/4171922046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856777c70 0x55785677a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.731+0000 7fada7e00640 1 -- 172.21.15.89:0/4171922046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 msgr2=0x55785679aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:34.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.731+0000 7fada7e00640 1 --2- 172.21.15.89:0/4171922046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 0x55785679aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.731+0000 7fada7e00640 1 -- 172.21.15.89:0/4171922046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557856774030 con 0x557856778590 2024-09-10T08:40:34.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.731+0000 7fada7e00640 1 --2- 172.21.15.89:0/4171922046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856778590 0x55785679a970 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fad94009e00 tx=0x7fad94009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18b8aeeea6e73919 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:34.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.731+0000 7fad99600640 1 -- 172.21.15.89:0/4171922046 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94010430 con 0x557856778590 2024-09-10T08:40:34.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.731+0000 7fad99600640 1 -- 172.21.15.89:0/4171922046 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad940105d0 con 0x557856778590 2024-09-10T08:40:34.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.731+0000 7fad99600640 1 -- 172.21.15.89:0/4171922046 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad940035f0 con 0x557856778590 2024-09-10T08:40:34.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.732+0000 7fadb25fb740 1 -- 172.21.15.89:0/4171922046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856778590 msgr2=0x55785679a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:34.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.732+0000 7fadb25fb740 1 --2- 172.21.15.89:0/4171922046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856778590 0x55785679a970 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fad94009e00 tx=0x7fad94009640 comp rx=0 tx=0).stop 2024-09-10T08:40:34.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.732+0000 7fadb25fb740 1 -- 172.21.15.89:0/4171922046 shutdown_connections 2024-09-10T08:40:34.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.732+0000 7fadb25fb740 1 --2- 172.21.15.89:0/4171922046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 0x55785679aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.732+0000 7fadb25fb740 1 --2- 172.21.15.89:0/4171922046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856778590 0x55785679a970 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.732+0000 7fadb25fb740 1 --2- 172.21.15.89:0/4171922046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856777c70 0x55785677a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.732+0000 7fadb25fb740 1 -- 172.21.15.89:0/4171922046 >> 172.21.15.89:0/4171922046 conn(0x557856774200 msgr2=0x55785677ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:34.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.732+0000 7fadb25fb740 1 -- 172.21.15.89:0/4171922046 shutdown_connections 2024-09-10T08:40:34.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.732+0000 7fadb25fb740 1 -- 172.21.15.89:0/4171922046 wait complete. 2024-09-10T08:40:34.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.733+0000 7fadb25fb740 1 -- start start 2024-09-10T08:40:34.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.733+0000 7fadb25fb740 1 Processor -- start 2024-09-10T08:40:34.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.733+0000 7fadb25fb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856777c70 0x55785678afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:34.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.733+0000 7fadb25fb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 0x55785677c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:34.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.733+0000 7fadb25fb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 0x55785677cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:34.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.733+0000 7fadb25fb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fada81af650 con 0x557856782e50 2024-09-10T08:40:34.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.733+0000 7fadb25fb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fada81b0ac0 con 0x557856777c70 2024-09-10T08:40:34.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.733+0000 7fadb25fb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fada81afd20 con 0x557856778590 2024-09-10T08:40:34.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.733+0000 7fadb25fb740 10 client.0 _ll_register_callbacks cb 0x7fad94009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:34.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.734+0000 7fada7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 0x55785677c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:34.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.734+0000 7fadad200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856777c70 0x55785678afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:34.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.734+0000 7fada7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 0x55785677c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:34702/0 (socket says 172.21.15.89:34702) 2024-09-10T08:40:34.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.734+0000 7fada7e00640 1 -- 172.21.15.89:0/1445532169 learned_addr learned my addr 172.21.15.89:0/1445532169 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:34.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.734+0000 7fadadc00640 1 --2- 172.21.15.89:0/1445532169 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 0x55785677cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:34.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.734+0000 7fada7e00640 1 -- 172.21.15.89:0/1445532169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856777c70 msgr2=0x55785678afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:34.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.734+0000 7fada7e00640 1 --2- 172.21.15.89:0/1445532169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856777c70 0x55785678afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.734+0000 7fada7e00640 1 -- 172.21.15.89:0/1445532169 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 msgr2=0x55785677cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:34.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.734+0000 7fada7e00640 1 --2- 172.21.15.89:0/1445532169 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 0x55785677cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.734+0000 7fada7e00640 1 -- 172.21.15.89:0/1445532169 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55785676f770 con 0x557856778590 2024-09-10T08:40:34.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.734+0000 7fadad200640 1 --2- 172.21.15.89:0/1445532169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856777c70 0x55785678afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:34.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.735+0000 7fada7e00640 1 --2- 172.21.15.89:0/1445532169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 0x55785677c890 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fad94010940 tx=0x7fad94011c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4769cefec98360ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:34.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.735+0000 7fad8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:40:34.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.735+0000 7fad8ea00640 1 -- 172.21.15.89:0/1445532169 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94010430 con 0x557856778590 2024-09-10T08:40:34.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.735+0000 7fad8ea00640 1 -- 172.21.15.89:0/1445532169 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fad94022040 con 0x557856778590 2024-09-10T08:40:34.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.735+0000 7fadb25fb740 1 -- 172.21.15.89:0/1445532169 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad940105d0 con 0x557856778590 2024-09-10T08:40:34.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.735+0000 7fad8ea00640 1 -- 172.21.15.89:0/1445532169 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fad94039050 con 0x557856778590 2024-09-10T08:40:34.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.735+0000 7fadb25fb740 10 client.25480 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:34.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.735+0000 7fadb25fb740 1 -- 172.21.15.89:0/1445532169 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5578567fd610 con 0x557856778590 2024-09-10T08:40:34.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.735+0000 7fad8ea00640 1 -- 172.21.15.89:0/1445532169 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad940105d0 con 0x557856778590 2024-09-10T08:40:34.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.736+0000 7fad8ea00640 1 -- 172.21.15.89:0/1445532169 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fad940035f0 con 0x557856778590 2024-09-10T08:40:34.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.736+0000 7fadb25fb740 10 client.25480 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:34.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.736+0000 7fadb25fb740 10 client.25480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:40:34.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.736+0000 7fadb25fb740 20 client.25480 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:34.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.736+0000 7fadb25fb740 1 --2- 172.21.15.89:0/1445532169 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5578567fdcd0 0x5578568000f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:34.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.736+0000 7fadb25fb740 4 client.25480 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:40:34.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.736+0000 7fadb25fb740 1 -- 172.21.15.89:0/1445532169 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5578566e5e60 con 0x5578567fdcd0 2024-09-10T08:40:34.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.736+0000 7fadad200640 1 --2- 172.21.15.89:0/1445532169 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5578567fdcd0 0x5578568000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:34.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.737+0000 7fadad200640 1 --2- 172.21.15.89:0/1445532169 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5578567fdcd0 0x5578568000f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=216022fcd7b1f805 server_cookie=6956e8f1ec54e040 in_seq=0 out_seq=0 2024-09-10T08:40:34.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.737+0000 7fad8ea00640 10 client.25480 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:34.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.737+0000 7fad8ea00640 1 -- 172.21.15.89:0/1445532169 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5578566e5e60 con 0x5578567fdcd0 2024-09-10T08:40:34.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.737+0000 7fad8ea00640 10 client.25480 handle_command_reply: tid=0 2024-09-10T08:40:34.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.737+0000 7fadb25fb740 1 client.25480 shutdown 2024-09-10T08:40:34.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.737+0000 7fadb25fb740 1 -- 172.21.15.89:0/1445532169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 msgr2=0x55785677c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:34.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.737+0000 7fadb25fb740 1 --2- 172.21.15.89:0/1445532169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 0x55785677c890 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fad94010940 tx=0x7fad94011c30 comp rx=0 tx=0).stop 2024-09-10T08:40:34.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.737+0000 7fadb25fb740 1 -- 172.21.15.89:0/1445532169 shutdown_connections 2024-09-10T08:40:34.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.737+0000 7fadb25fb740 1 --2- 172.21.15.89:0/1445532169 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5578567fdcd0 0x5578568000f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.737+0000 7fadb25fb740 1 --2- 172.21.15.89:0/1445532169 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557856782e50 0x55785677cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.737+0000 7fadb25fb740 1 --2- 172.21.15.89:0/1445532169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557856778590 0x55785677c890 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.737+0000 7fadb25fb740 1 --2- 172.21.15.89:0/1445532169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557856777c70 0x55785678afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.737+0000 7fadb25fb740 1 -- 172.21.15.89:0/1445532169 >> 172.21.15.89:0/1445532169 conn(0x557856774200 msgr2=0x557856772b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:34.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.737+0000 7fad8ea00640 0 client.25480 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:34.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.737+0000 7fadb25fb740 1 -- 172.21.15.89:0/1445532169 shutdown_connections 2024-09-10T08:40:34.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.738+0000 7fadb25fb740 1 -- 172.21.15.89:0/1445532169 wait complete. 2024-09-10T08:40:34.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.738+0000 7fadb25fb740 20 client.25480 trim_cache size 0 max 16384 2024-09-10T08:40:34.742 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:40:34.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.738+0000 7fadafc00640 1 -- 172.21.15.89:0/1122778818 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fad900a2c10 msgr2=0x7fad900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:34.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.738+0000 7fadafc00640 1 --2- 172.21.15.89:0/1122778818 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fad900a2c10 0x7fad900a50d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fada0005fd0 tx=0x7fada0009500 comp rx=0 tx=0).stop 2024-09-10T08:40:34.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.738+0000 7fadafc00640 1 -- 172.21.15.89:0/1122778818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fada8104010 msgr2=0x7fada819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:34.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.738+0000 7fadafc00640 1 --2- 172.21.15.89:0/1122778818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fada8104010 0x7fada819a600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fad9c006010 tx=0x7fad9c004040 comp rx=0 tx=0).stop 2024-09-10T08:40:34.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.738+0000 7fadafc00640 1 -- 172.21.15.89:0/1122778818 shutdown_connections 2024-09-10T08:40:34.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.738+0000 7fadafc00640 1 --2- 172.21.15.89:0/1122778818 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fad900a2c10 0x7fad900a50d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.738+0000 7fadafc00640 1 --2- 172.21.15.89:0/1122778818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fada8109140 0x7fada819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.738+0000 7fadafc00640 1 --2- 172.21.15.89:0/1122778818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fada81049c0 0x7fada819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.738+0000 7fadafc00640 1 --2- 172.21.15.89:0/1122778818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fada8104010 0x7fada819a600 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:34.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.738+0000 7fadafc00640 1 -- 172.21.15.89:0/1122778818 >> 172.21.15.89:0/1122778818 conn(0x7fada80ffa50 msgr2=0x7fada8100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:34.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.739+0000 7fadafc00640 1 -- 172.21.15.89:0/1122778818 shutdown_connections 2024-09-10T08:40:34.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:34.739+0000 7fadafc00640 1 -- 172.21.15.89:0/1122778818 wait complete. 2024-09-10T08:40:34.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:34 smithi149 ceph-mon[36167]: pgmap v412: 129 pgs: 129 active+clean; 664 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T08:40:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:34 smithi111 ceph-mon[35220]: pgmap v412: 129 pgs: 129 active+clean; 664 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T08:40:35.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:34 smithi089 ceph-mon[30757]: pgmap v412: 129 pgs: 129 active+clean; 664 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T08:40:36.510 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:40:36.510 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:40:36.970 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:40:36.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:36 smithi149 ceph-mon[36167]: pgmap v413: 129 pgs: 129 active+clean; 669 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:40:37.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:36 smithi111 ceph-mon[35220]: pgmap v413: 129 pgs: 129 active+clean; 669 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:40:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:36 smithi089 ceph-mon[30757]: pgmap v413: 129 pgs: 129 active+clean; 669 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:40:37.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.632+0000 7ff3cda00640 1 -- 172.21.15.89:0/3338923495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3c81049a0 msgr2=0x7ff3c81089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:37.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.632+0000 7ff3cda00640 1 --2- 172.21.15.89:0/3338923495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3c81049a0 0x7ff3c81089f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009900 tx=0x7ff3b402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.632+0000 7ff3cda00640 1 -- 172.21.15.89:0/3338923495 shutdown_connections 2024-09-10T08:40:37.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.632+0000 7ff3cda00640 1 --2- 172.21.15.89:0/3338923495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3c8109120 0x7ff3c8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.632+0000 7ff3cda00640 1 --2- 172.21.15.89:0/3338923495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3c81049a0 0x7ff3c81089f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.632+0000 7ff3cda00640 1 --2- 172.21.15.89:0/3338923495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3c8103ff0 0x7ff3c81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.632+0000 7ff3cda00640 1 -- 172.21.15.89:0/3338923495 >> 172.21.15.89:0/3338923495 conn(0x7ff3c80ffa90 msgr2=0x7ff3c8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:37.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.632+0000 7ff3cda00640 1 -- 172.21.15.89:0/3338923495 shutdown_connections 2024-09-10T08:40:37.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.632+0000 7ff3cda00640 1 -- 172.21.15.89:0/3338923495 wait complete. 2024-09-10T08:40:37.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.634+0000 7ff3cda00640 1 Processor -- start 2024-09-10T08:40:37.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.634+0000 7ff3cda00640 1 -- start start 2024-09-10T08:40:37.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.634+0000 7ff3cda00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3c8103ff0 0x7ff3c8110b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:37.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.634+0000 7ff3cda00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3c81049a0 0x7ff3c810dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:37.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.634+0000 7ff3cda00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3c8109120 0x7ff3c810e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:37.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.634+0000 7ff3cda00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c8114190 con 0x7ff3c8103ff0 2024-09-10T08:40:37.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.634+0000 7ff3cda00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c8114010 con 0x7ff3c8109120 2024-09-10T08:40:37.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.634+0000 7ff3cda00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c8114310 con 0x7ff3c81049a0 2024-09-10T08:40:37.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.635+0000 7ff3c7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3c8109120 0x7ff3c810e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:37.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.635+0000 7ff3c6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3c8103ff0 0x7ff3c8110b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:37.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.635+0000 7ff3c7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3c8109120 0x7ff3c810e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:48030/0 (socket says 172.21.15.89:48030) 2024-09-10T08:40:37.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.635+0000 7ff3c7400640 1 -- 172.21.15.89:0/3505475324 learned_addr learned my addr 172.21.15.89:0/3505475324 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:37.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.635+0000 7ff3c7400640 1 -- 172.21.15.89:0/3505475324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3c81049a0 msgr2=0x7ff3c810dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:37.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.635+0000 7ff3c7400640 1 --2- 172.21.15.89:0/3505475324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3c81049a0 0x7ff3c810dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.635+0000 7ff3c7400640 1 -- 172.21.15.89:0/3505475324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3c8103ff0 msgr2=0x7ff3c8110b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:37.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.635+0000 7ff3c7400640 1 --2- 172.21.15.89:0/3505475324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3c8103ff0 0x7ff3c8110b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.635+0000 7ff3c7400640 1 -- 172.21.15.89:0/3505475324 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c81aaed0 con 0x7ff3c8109120 2024-09-10T08:40:37.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.635+0000 7ff3c7400640 1 --2- 172.21.15.89:0/3505475324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3c8109120 0x7ff3c810e120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff3c000eb10 tx=0x7ff3c000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:37.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.636+0000 7ff3bf400640 1 -- 172.21.15.89:0/3505475324 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0004b20 con 0x7ff3c8109120 2024-09-10T08:40:37.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.636+0000 7ff3bf400640 1 -- 172.21.15.89:0/3505475324 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c0004cc0 con 0x7ff3c8109120 2024-09-10T08:40:37.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.636+0000 7ff3bf400640 1 -- 172.21.15.89:0/3505475324 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0010760 con 0x7ff3c8109120 2024-09-10T08:40:37.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.636+0000 7ff3cda00640 1 -- 172.21.15.89:0/3505475324 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c81ab0a0 con 0x7ff3c8109120 2024-09-10T08:40:37.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.636+0000 7ff3cda00640 1 -- 172.21.15.89:0/3505475324 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c81ab4f0 con 0x7ff3c8109120 2024-09-10T08:40:37.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.640+0000 7ff3d03e3740 1 Processor -- start 2024-09-10T08:40:37.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.641+0000 7ff3bf400640 1 -- 172.21.15.89:0/3505475324 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7ff3c0005020 con 0x7ff3c8109120 2024-09-10T08:40:37.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.641+0000 7ff3d03e3740 1 -- start start 2024-09-10T08:40:37.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.642+0000 7ff3d03e3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b612ac0 0x560c2b612ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:37.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.642+0000 7ff3d03e3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 0x560c2b61d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:37.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.642+0000 7ff3d03e3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b61dca0 0x560c2b620090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:37.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.642+0000 7ff3d03e3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560c2b579290 con 0x560c2b6133e0 2024-09-10T08:40:37.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.642+0000 7ff3d03e3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560c2b520b40 con 0x560c2b61dca0 2024-09-10T08:40:37.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.642+0000 7ff3d03e3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560c2b575250 con 0x560c2b612ac0 2024-09-10T08:40:37.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.642+0000 7ff3c6000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b612ac0 0x560c2b612ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:37.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.642+0000 7ff3c6000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b612ac0 0x560c2b612ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:34722/0 (socket says 172.21.15.89:34722) 2024-09-10T08:40:37.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.642+0000 7ff3c6000640 1 -- 172.21.15.89:0/2214333823 learned_addr learned my addr 172.21.15.89:0/2214333823 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:37.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.642+0000 7ff3c6000640 1 --2- 172.21.15.89:0/2214333823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b61dca0 0x560c2b620090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:37.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.642+0000 7ff3c6a00640 1 --2- 172.21.15.89:0/2214333823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 0x560c2b61d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:37.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.643+0000 7ff3c6000640 1 -- 172.21.15.89:0/2214333823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b61dca0 msgr2=0x560c2b620090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:37.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.643+0000 7ff3c6000640 1 --2- 172.21.15.89:0/2214333823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b61dca0 0x560c2b620090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.643+0000 7ff3c6000640 1 -- 172.21.15.89:0/2214333823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 msgr2=0x560c2b61d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:37.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.643+0000 7ff3c6000640 1 --2- 172.21.15.89:0/2214333823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 0x560c2b61d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.643+0000 7ff3c6000640 1 -- 172.21.15.89:0/2214333823 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c2b2ea6a0 con 0x560c2b612ac0 2024-09-10T08:40:37.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.643+0000 7ff3c6a00640 1 --2- 172.21.15.89:0/2214333823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 0x560c2b61d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:40:37.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.644+0000 7ff3c6000640 1 --2- 172.21.15.89:0/2214333823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b612ac0 0x560c2b612ea0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4031550 tx=0x7ff3b4005b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32cc13671cec082d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:37.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.644+0000 7ff3bf400640 1 --2- 172.21.15.89:0/3505475324 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3ac0a2c90 0x7ff3ac0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:37.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.644+0000 7ff3bf400640 1 -- 172.21.15.89:0/3505475324 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff3c00cfb40 con 0x7ff3c8109120 2024-09-10T08:40:37.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.644+0000 7ff3b3e00640 1 -- 172.21.15.89:0/2214333823 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4035070 con 0x560c2b612ac0 2024-09-10T08:40:37.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.644+0000 7ff3b3e00640 1 -- 172.21.15.89:0/2214333823 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b4031e40 con 0x560c2b612ac0 2024-09-10T08:40:37.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.644+0000 7ff3b3e00640 1 -- 172.21.15.89:0/2214333823 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4003ab0 con 0x560c2b612ac0 2024-09-10T08:40:37.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.644+0000 7ff3c7400640 1 --2- 172.21.15.89:0/3505475324 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3ac0a2c90 0x7ff3ac0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:37.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.645+0000 7ff3d03e3740 1 -- 172.21.15.89:0/2214333823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b612ac0 msgr2=0x560c2b612ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:37.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.645+0000 7ff3d03e3740 1 --2- 172.21.15.89:0/2214333823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b612ac0 0x560c2b612ea0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4031550 tx=0x7ff3b4005b70 comp rx=0 tx=0).stop 2024-09-10T08:40:37.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.645+0000 7ff3c7400640 1 --2- 172.21.15.89:0/3505475324 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3ac0a2c90 0x7ff3ac0a5150 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0023030 tx=0x7ff3c0005b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:37.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.646+0000 7ff3d03e3740 1 -- 172.21.15.89:0/2214333823 shutdown_connections 2024-09-10T08:40:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.646+0000 7ff3d03e3740 1 --2- 172.21.15.89:0/2214333823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b61dca0 0x560c2b620090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.646+0000 7ff3d03e3740 1 --2- 172.21.15.89:0/2214333823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 0x560c2b61d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.646+0000 7ff3d03e3740 1 --2- 172.21.15.89:0/2214333823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b612ac0 0x560c2b612ea0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.646+0000 7ff3d03e3740 1 -- 172.21.15.89:0/2214333823 >> 172.21.15.89:0/2214333823 conn(0x560c2b60f050 msgr2=0x560c2b60f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.646+0000 7ff3d03e3740 1 -- 172.21.15.89:0/2214333823 shutdown_connections 2024-09-10T08:40:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.646+0000 7ff3d03e3740 1 -- 172.21.15.89:0/2214333823 wait complete. 2024-09-10T08:40:37.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.647+0000 7ff3d03e3740 1 -- start start 2024-09-10T08:40:37.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.647+0000 7ff3d03e3740 1 Processor -- start 2024-09-10T08:40:37.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.648+0000 7ff3d03e3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 0x560c2b61d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:37.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.648+0000 7ff3d03e3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 0x560c2b61a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:37.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.648+0000 7ff3d03e3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b61dca0 0x560c2b61af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:37.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.648+0000 7ff3d03e3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560c2b620910 con 0x560c2b6133e0 2024-09-10T08:40:37.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.648+0000 7ff3d03e3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560c2b533fa0 con 0x560c2b612ac0 2024-09-10T08:40:37.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.648+0000 7ff3d03e3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560c2b620a90 con 0x560c2b61dca0 2024-09-10T08:40:37.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.648+0000 7ff3d03e3740 10 client.0 _ll_register_callbacks cb 0x560c2b287ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:37.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.649+0000 7ff3c6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 0x560c2b61d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:37.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.649+0000 7ff3c6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 0x560c2b61d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:48050/0 (socket says 172.21.15.89:48050) 2024-09-10T08:40:37.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.649+0000 7ff3c6000640 1 -- 172.21.15.89:0/470774492 learned_addr learned my addr 172.21.15.89:0/470774492 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.649+0000 7ff3c6000640 1 --2- 172.21.15.89:0/470774492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b61dca0 0x560c2b61af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.649+0000 7ff3c6a00640 1 --2- 172.21.15.89:0/470774492 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 0x560c2b61a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.649+0000 7ff3c6000640 1 -- 172.21.15.89:0/470774492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b61dca0 msgr2=0x560c2b61af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.649+0000 7ff3c6000640 1 --2- 172.21.15.89:0/470774492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b61dca0 0x560c2b61af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.649+0000 7ff3c6000640 1 -- 172.21.15.89:0/470774492 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 msgr2=0x560c2b61a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.649+0000 7ff3c6000640 1 --2- 172.21.15.89:0/470774492 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 0x560c2b61a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.649+0000 7ff3c6000640 1 -- 172.21.15.89:0/470774492 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c2b61b770 con 0x560c2b612ac0 2024-09-10T08:40:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.649+0000 7ff3c6a00640 1 --2- 172.21.15.89:0/470774492 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 0x560c2b61a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:40:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.649+0000 7ff3c6000640 1 --2- 172.21.15.89:0/470774492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 0x560c2b61d6a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009640 tx=0x7ff3b4003690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=132cb7056c96141 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.649+0000 7ff3b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:40:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.650+0000 7ff3b0c00640 1 -- 172.21.15.89:0/470774492 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4051020 con 0x560c2b612ac0 2024-09-10T08:40:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.650+0000 7ff3b0c00640 1 -- 172.21.15.89:0/470774492 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff3b404b880 con 0x560c2b612ac0 2024-09-10T08:40:37.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.650+0000 7ff3d03e3740 1 -- 172.21.15.89:0/470774492 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560c2b586800 con 0x560c2b612ac0 2024-09-10T08:40:37.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.651+0000 7ff3b0c00640 1 -- 172.21.15.89:0/470774492 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff3b4035030 con 0x560c2b612ac0 2024-09-10T08:40:37.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.651+0000 7ff3d03e3740 10 client.25532 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:37.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.651+0000 7ff3d03e3740 1 -- 172.21.15.89:0/470774492 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560c2b62c8d0 con 0x560c2b612ac0 2024-09-10T08:40:37.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.651+0000 7ff3b0c00640 1 -- 172.21.15.89:0/470774492 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560c2b586800 con 0x560c2b612ac0 2024-09-10T08:40:37.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.651+0000 7ff3b0c00640 1 -- 172.21.15.89:0/470774492 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff3b403a040 con 0x560c2b612ac0 2024-09-10T08:40:37.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.652+0000 7ff3d03e3740 10 client.25532 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:37.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.652+0000 7ff3d03e3740 10 client.25532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:40:37.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.652+0000 7ff3d03e3740 20 client.25532 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:37.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.652+0000 7ff3d03e3740 1 --2- 172.21.15.89:0/470774492 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560c2b62dc20 0x560c2b630010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:37.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.652+0000 7ff3d03e3740 4 client.25532 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:40:37.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.652+0000 7ff3d03e3740 1 -- 172.21.15.89:0/470774492 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c2b580d10 con 0x560c2b62dc20 2024-09-10T08:40:37.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.652+0000 7ff3c6a00640 1 --2- 172.21.15.89:0/470774492 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560c2b62dc20 0x560c2b630010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:37.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.653+0000 7ff3c6a00640 1 --2- 172.21.15.89:0/470774492 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560c2b62dc20 0x560c2b630010 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b9f380f75b680f7 server_cookie=f58f9471747acb54 in_seq=0 out_seq=0 2024-09-10T08:40:37.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.653+0000 7ff3b0c00640 10 client.25532 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:37.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.654+0000 7ff3b0c00640 1 -- 172.21.15.89:0/470774492 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c2b580d10 con 0x560c2b62dc20 2024-09-10T08:40:37.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.654+0000 7ff3b0c00640 10 client.25532 handle_command_reply: tid=0 2024-09-10T08:40:37.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.654+0000 7ff3d03e3740 1 client.25532 shutdown 2024-09-10T08:40:37.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.654+0000 7ff3d03e3740 1 -- 172.21.15.89:0/470774492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 msgr2=0x560c2b61d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:37.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.654+0000 7ff3d03e3740 1 --2- 172.21.15.89:0/470774492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 0x560c2b61d6a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009640 tx=0x7ff3b4003690 comp rx=0 tx=0).stop 2024-09-10T08:40:37.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.654+0000 7ff3d03e3740 1 -- 172.21.15.89:0/470774492 shutdown_connections 2024-09-10T08:40:37.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.654+0000 7ff3d03e3740 1 --2- 172.21.15.89:0/470774492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b61dca0 0x560c2b61af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.654+0000 7ff3d03e3740 1 --2- 172.21.15.89:0/470774492 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560c2b62dc20 0x560c2b630010 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.654+0000 7ff3d03e3740 1 --2- 172.21.15.89:0/470774492 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 0x560c2b61a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.654+0000 7ff3d03e3740 1 --2- 172.21.15.89:0/470774492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 0x560c2b61d6a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.654+0000 7ff3d03e3740 1 -- 172.21.15.89:0/470774492 >> 172.21.15.89:0/470774492 conn(0x560c2b60f050 msgr2=0x560c2b60e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:37.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.654+0000 7ff3b0c00640 0 client.25532 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:37.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.655+0000 7ff3d03e3740 1 -- 172.21.15.89:0/470774492 shutdown_connections 2024-09-10T08:40:37.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.655+0000 7ff3d03e3740 1 -- 172.21.15.89:0/470774492 wait complete. 2024-09-10T08:40:37.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.655+0000 7ff3d03e3740 20 client.25532 trim_cache size 0 max 16384 2024-09-10T08:40:37.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.673+0000 7ff3d03e3740 1 Processor -- start 2024-09-10T08:40:37.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.673+0000 7ff3d03e3740 1 -- start start 2024-09-10T08:40:37.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.673+0000 7ff3d03e3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 0x560c2b60e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:37.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.673+0000 7ff3d03e3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 0x560c2b6293f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:37.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.673+0000 7ff3d03e3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b61dca0 0x560c2b617bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:37.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.673+0000 7ff3d03e3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c81b7c00 con 0x560c2b6133e0 2024-09-10T08:40:37.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.673+0000 7ff3d03e3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c81b54d0 con 0x560c2b612ac0 2024-09-10T08:40:37.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.673+0000 7ff3d03e3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c81b3320 con 0x560c2b61dca0 2024-09-10T08:40:37.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.674+0000 7ff3c6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 0x560c2b60e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:37.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.674+0000 7ff3c6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 0x560c2b6293f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:37.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.674+0000 7ff3c6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 0x560c2b60e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:48056/0 (socket says 172.21.15.89:48056) 2024-09-10T08:40:37.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.674+0000 7ff3c6000640 1 -- 172.21.15.89:0/30473057 learned_addr learned my addr 172.21.15.89:0/30473057 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:37.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.674+0000 7ff3c6000640 1 --2- 172.21.15.89:0/30473057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b61dca0 0x560c2b617bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:37.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.674+0000 7ff3c6000640 1 -- 172.21.15.89:0/30473057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b61dca0 msgr2=0x560c2b617bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:37.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.674+0000 7ff3c6000640 1 --2- 172.21.15.89:0/30473057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b61dca0 0x560c2b617bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.674+0000 7ff3c6000640 1 -- 172.21.15.89:0/30473057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 msgr2=0x560c2b6293f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:37.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.674+0000 7ff3c6000640 1 --2- 172.21.15.89:0/30473057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 0x560c2b6293f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.674+0000 7ff3c6000640 1 -- 172.21.15.89:0/30473057 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c2b61b2b0 con 0x560c2b612ac0 2024-09-10T08:40:37.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.675+0000 7ff3c6000640 1 --2- 172.21.15.89:0/30473057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 0x560c2b60e9b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4003660 tx=0x7ff3b4005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff93c6127f4be9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:37.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.675+0000 7ff3b3400640 1 -- 172.21.15.89:0/30473057 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4041030 con 0x560c2b612ac0 2024-09-10T08:40:37.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.675+0000 7ff3b3400640 1 -- 172.21.15.89:0/30473057 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b4005850 con 0x560c2b612ac0 2024-09-10T08:40:37.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.675+0000 7ff3b3400640 1 -- 172.21.15.89:0/30473057 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4051040 con 0x560c2b612ac0 2024-09-10T08:40:37.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.675+0000 7ff3d03e3740 1 -- 172.21.15.89:0/30473057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 msgr2=0x560c2b60e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:37.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.675+0000 7ff3d03e3740 1 --2- 172.21.15.89:0/30473057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 0x560c2b60e9b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4003660 tx=0x7ff3b4005760 comp rx=0 tx=0).stop 2024-09-10T08:40:37.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.676+0000 7ff3d03e3740 1 -- 172.21.15.89:0/30473057 shutdown_connections 2024-09-10T08:40:37.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.676+0000 7ff3d03e3740 1 --2- 172.21.15.89:0/30473057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b61dca0 0x560c2b617bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.676+0000 7ff3d03e3740 1 --2- 172.21.15.89:0/30473057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 0x560c2b6293f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.676+0000 7ff3d03e3740 1 --2- 172.21.15.89:0/30473057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 0x560c2b60e9b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.676+0000 7ff3d03e3740 1 -- 172.21.15.89:0/30473057 >> 172.21.15.89:0/30473057 conn(0x560c2b60f050 msgr2=0x560c2b628ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:37.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.676+0000 7ff3d03e3740 1 -- 172.21.15.89:0/30473057 shutdown_connections 2024-09-10T08:40:37.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.676+0000 7ff3d03e3740 1 -- 172.21.15.89:0/30473057 wait complete. 2024-09-10T08:40:37.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.676+0000 7ff3d03e3740 1 -- start start 2024-09-10T08:40:37.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.676+0000 7ff3d03e3740 1 Processor -- start 2024-09-10T08:40:37.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.677+0000 7ff3d03e3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 0x560c2b631530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:37.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.677+0000 7ff3d03e3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 0x560c2b631a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:37.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.677+0000 7ff3d03e3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b61dca0 0x560c2b697560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:37.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.677+0000 7ff3d03e3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c81b3990 con 0x560c2b6133e0 2024-09-10T08:40:37.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.677+0000 7ff3d03e3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c81b4e00 con 0x560c2b612ac0 2024-09-10T08:40:37.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.677+0000 7ff3d03e3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c81b4060 con 0x560c2b61dca0 2024-09-10T08:40:37.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.677+0000 7ff3d03e3740 10 client.0 _ll_register_callbacks cb 0x7ff3b8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:37.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.677+0000 7ff3c6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 0x560c2b631530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:37.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.677+0000 7ff3c6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 0x560c2b631a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:37.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.677+0000 7ff3c6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 0x560c2b631530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:48072/0 (socket says 172.21.15.89:48072) 2024-09-10T08:40:37.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.677+0000 7ff3c6000640 1 -- 172.21.15.89:0/1162952535 learned_addr learned my addr 172.21.15.89:0/1162952535 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:37.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.677+0000 7ff3c6000640 1 --2- 172.21.15.89:0/1162952535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b61dca0 0x560c2b697560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:37.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.678+0000 7ff3c6000640 1 -- 172.21.15.89:0/1162952535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b61dca0 msgr2=0x560c2b697560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:37.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.678+0000 7ff3c6000640 1 --2- 172.21.15.89:0/1162952535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b61dca0 0x560c2b697560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.678+0000 7ff3c6000640 1 -- 172.21.15.89:0/1162952535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 msgr2=0x560c2b631a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:37.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.678+0000 7ff3c6000640 1 --2- 172.21.15.89:0/1162952535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 0x560c2b631a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.678+0000 7ff3c6000640 1 -- 172.21.15.89:0/1162952535 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c2b697b20 con 0x560c2b612ac0 2024-09-10T08:40:37.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.678+0000 7ff3c6a00640 1 --2- 172.21.15.89:0/1162952535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 0x560c2b631a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:37.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.678+0000 7ff3c6000640 1 --2- 172.21.15.89:0/1162952535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 0x560c2b631530 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff3b402a910 tx=0x7ff3b404bc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbed6f5cd2a29c6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:37.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.678+0000 7ff3b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:40:37.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.679+0000 7ff3b0c00640 1 -- 172.21.15.89:0/1162952535 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4051070 con 0x560c2b612ac0 2024-09-10T08:40:37.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.679+0000 7ff3b0c00640 1 -- 172.21.15.89:0/1162952535 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff3b4049670 con 0x560c2b612ac0 2024-09-10T08:40:37.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.679+0000 7ff3d03e3740 1 -- 172.21.15.89:0/1162952535 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3b4005850 con 0x560c2b612ac0 2024-09-10T08:40:37.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.679+0000 7ff3b0c00640 1 -- 172.21.15.89:0/1162952535 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff3b4066040 con 0x560c2b612ac0 2024-09-10T08:40:37.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.679+0000 7ff3d03e3740 10 client.25538 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:37.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.679+0000 7ff3d03e3740 1 -- 172.21.15.89:0/1162952535 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560c2b698e20 con 0x560c2b612ac0 2024-09-10T08:40:37.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.679+0000 7ff3b0c00640 1 -- 172.21.15.89:0/1162952535 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b4005850 con 0x560c2b612ac0 2024-09-10T08:40:37.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.680+0000 7ff3b0c00640 1 -- 172.21.15.89:0/1162952535 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff3b4061680 con 0x560c2b612ac0 2024-09-10T08:40:37.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.680+0000 7ff3d03e3740 10 client.25538 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:37.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.680+0000 7ff3d03e3740 10 client.25538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:40:37.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.680+0000 7ff3d03e3740 20 client.25538 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:37.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.680+0000 7ff3d03e3740 1 --2- 172.21.15.89:0/1162952535 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560c2b6991b0 0x560c2b69b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:37.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.680+0000 7ff3d03e3740 4 client.25538 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:40:37.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.680+0000 7ff3d03e3740 1 -- 172.21.15.89:0/1162952535 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560c2b4fcfb0 con 0x560c2b6991b0 2024-09-10T08:40:37.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.680+0000 7ff3c6a00640 1 --2- 172.21.15.89:0/1162952535 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560c2b6991b0 0x560c2b69b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:37.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.681+0000 7ff3c6a00640 1 --2- 172.21.15.89:0/1162952535 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560c2b6991b0 0x560c2b69b5a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61c7b8d9f75467e9 server_cookie=9911db14624739fb in_seq=0 out_seq=0 2024-09-10T08:40:37.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.681+0000 7ff3b0c00640 10 client.25538 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:37.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.682+0000 7ff3b0c00640 1 -- 172.21.15.89:0/1162952535 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560c2b4fcfb0 con 0x560c2b6991b0 2024-09-10T08:40:37.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.682+0000 7ff3b0c00640 10 client.25538 handle_command_reply: tid=0 2024-09-10T08:40:37.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.682+0000 7ff3d03e3740 1 client.25538 shutdown 2024-09-10T08:40:37.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.682+0000 7ff3d03e3740 1 -- 172.21.15.89:0/1162952535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 msgr2=0x560c2b631530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:37.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.682+0000 7ff3d03e3740 1 --2- 172.21.15.89:0/1162952535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 0x560c2b631530 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff3b402a910 tx=0x7ff3b404bc70 comp rx=0 tx=0).stop 2024-09-10T08:40:37.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.683+0000 7ff3d03e3740 1 -- 172.21.15.89:0/1162952535 shutdown_connections 2024-09-10T08:40:37.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.683+0000 7ff3d03e3740 1 --2- 172.21.15.89:0/1162952535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560c2b61dca0 0x560c2b697560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.683+0000 7ff3d03e3740 1 --2- 172.21.15.89:0/1162952535 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560c2b6991b0 0x560c2b69b5a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.683+0000 7ff3d03e3740 1 --2- 172.21.15.89:0/1162952535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c2b6133e0 0x560c2b631a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.683+0000 7ff3d03e3740 1 --2- 172.21.15.89:0/1162952535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c2b612ac0 0x560c2b631530 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.683+0000 7ff3d03e3740 1 -- 172.21.15.89:0/1162952535 >> 172.21.15.89:0/1162952535 conn(0x560c2b60f050 msgr2=0x560c2b629500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:37.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.683+0000 7ff3b0c00640 0 client.25538 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:37.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.683+0000 7ff3d03e3740 1 -- 172.21.15.89:0/1162952535 shutdown_connections 2024-09-10T08:40:37.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.684+0000 7ff3d03e3740 1 -- 172.21.15.89:0/1162952535 wait complete. 2024-09-10T08:40:37.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.684+0000 7ff3d03e3740 20 client.25538 trim_cache size 0 max 16384 2024-09-10T08:40:37.686 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:40:37.686 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2024-09-10T08:40:37.687 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "7bc45783-bf82-448b-a2eb-70f773ce98f9", 2024-09-10T08:40:37.687 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2024-09-10T08:40:37.687 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:40:37.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.685+0000 7ff3cda00640 1 -- 172.21.15.89:0/3505475324 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3ac0a2c90 msgr2=0x7ff3ac0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:37.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.685+0000 7ff3cda00640 1 --2- 172.21.15.89:0/3505475324 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3ac0a2c90 0x7ff3ac0a5150 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0023030 tx=0x7ff3c0005b40 comp rx=0 tx=0).stop 2024-09-10T08:40:37.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.685+0000 7ff3cda00640 1 -- 172.21.15.89:0/3505475324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3c8109120 msgr2=0x7ff3c810e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:37.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.685+0000 7ff3cda00640 1 --2- 172.21.15.89:0/3505475324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3c8109120 0x7ff3c810e120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff3c000eb10 tx=0x7ff3c000c2d0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.685+0000 7ff3cda00640 1 -- 172.21.15.89:0/3505475324 shutdown_connections 2024-09-10T08:40:37.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.685+0000 7ff3cda00640 1 --2- 172.21.15.89:0/3505475324 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3ac0a2c90 0x7ff3ac0a5150 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.685+0000 7ff3cda00640 1 --2- 172.21.15.89:0/3505475324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3c8109120 0x7ff3c810e120 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.685+0000 7ff3cda00640 1 --2- 172.21.15.89:0/3505475324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3c81049a0 0x7ff3c810dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.685+0000 7ff3cda00640 1 --2- 172.21.15.89:0/3505475324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3c8103ff0 0x7ff3c8110b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:37.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.685+0000 7ff3cda00640 1 -- 172.21.15.89:0/3505475324 >> 172.21.15.89:0/3505475324 conn(0x7ff3c80ffa90 msgr2=0x7ff3c8100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:37.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.686+0000 7ff3cda00640 1 -- 172.21.15.89:0/3505475324 shutdown_connections 2024-09-10T08:40:37.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:37.686+0000 7ff3cda00640 1 -- 172.21.15.89:0/3505475324 wait complete. 2024-09-10T08:40:38.502 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:40:39.010 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:40:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:38 smithi111 ceph-mon[35220]: pgmap v414: 129 pgs: 129 active+clean; 669 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T08:40:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:38 smithi111 ceph-mon[35220]: scrub queued for path: / 2024-09-10T08:40:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:38 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:40:39.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:38 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:40:39.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:38 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:40:39.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:38 smithi089 ceph-mon[30757]: pgmap v414: 129 pgs: 129 active+clean; 669 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T08:40:39.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:38 smithi089 ceph-mon[30757]: scrub queued for path: / 2024-09-10T08:40:39.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:38 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:40:39.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:38 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:40:39.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:38 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:40:39.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:38 smithi149 ceph-mon[36167]: pgmap v414: 129 pgs: 129 active+clean; 669 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T08:40:39.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:38 smithi149 ceph-mon[36167]: scrub queued for path: / 2024-09-10T08:40:39.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:38 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:40:39.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:38 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:40:39.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:38 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:40:39.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.686+0000 7f0326000640 1 -- 172.21.15.89:0/2028694081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0320103e30 msgr2=0x7f0320104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:39.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.686+0000 7f0326000640 1 --2- 172.21.15.89:0/2028694081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0320103e30 0x7f0320104210 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0314009900 tx=0x7f031402a410 comp rx=0 tx=0).stop 2024-09-10T08:40:39.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.686+0000 7f0326000640 1 -- 172.21.15.89:0/2028694081 shutdown_connections 2024-09-10T08:40:39.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.686+0000 7f0326000640 1 --2- 172.21.15.89:0/2028694081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0320108f60 0x7f0320110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.686+0000 7f0326000640 1 --2- 172.21.15.89:0/2028694081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03201047e0 0x7f0320108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.686+0000 7f0326000640 1 --2- 172.21.15.89:0/2028694081 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0320103e30 0x7f0320104210 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.686+0000 7f0326000640 1 -- 172.21.15.89:0/2028694081 >> 172.21.15.89:0/2028694081 conn(0x7f03200ff910 msgr2=0x7f0320101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:39.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.686+0000 7f0326000640 1 -- 172.21.15.89:0/2028694081 shutdown_connections 2024-09-10T08:40:39.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.686+0000 7f0326000640 1 -- 172.21.15.89:0/2028694081 wait complete. 2024-09-10T08:40:39.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.688+0000 7f0326000640 1 Processor -- start 2024-09-10T08:40:39.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.688+0000 7f0326000640 1 -- start start 2024-09-10T08:40:39.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.688+0000 7f0326000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0320103e30 0x7f0320110a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:39.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.688+0000 7f0326000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03201047e0 0x7f032010dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:39.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.688+0000 7f0326000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0320108f60 0x7f032010e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:39.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.688+0000 7f0326000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0320113f80 con 0x7f0320108f60 2024-09-10T08:40:39.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.688+0000 7f0326000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0320113e00 con 0x7f0320103e30 2024-09-10T08:40:39.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.688+0000 7f0326000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0320114100 con 0x7f03201047e0 2024-09-10T08:40:39.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.688+0000 7f031fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03201047e0 0x7f032010dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:39.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.688+0000 7f031fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03201047e0 0x7f032010dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:34770/0 (socket says 172.21.15.89:34770) 2024-09-10T08:40:39.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.689+0000 7f031fe00640 1 -- 172.21.15.89:0/3001115479 learned_addr learned my addr 172.21.15.89:0/3001115479 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:39.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.689+0000 7f0324c00640 1 --2- 172.21.15.89:0/3001115479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0320103e30 0x7f0320110a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:39.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.689+0000 7f0325600640 1 --2- 172.21.15.89:0/3001115479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0320108f60 0x7f032010e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.689+0000 7f031fe00640 1 -- 172.21.15.89:0/3001115479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0320103e30 msgr2=0x7f0320110a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.689+0000 7f031fe00640 1 --2- 172.21.15.89:0/3001115479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0320103e30 0x7f0320110a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.689+0000 7f031fe00640 1 -- 172.21.15.89:0/3001115479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0320108f60 msgr2=0x7f032010e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.689+0000 7f031fe00640 1 --2- 172.21.15.89:0/3001115479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0320108f60 0x7f032010e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.689+0000 7f031fe00640 1 -- 172.21.15.89:0/3001115479 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03201a0480 con 0x7f03201047e0 2024-09-10T08:40:39.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.689+0000 7f031fe00640 1 --2- 172.21.15.89:0/3001115479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03201047e0 0x7f032010dab0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f030c00e9f0 tx=0x7f030c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:39.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.690+0000 7f031d600640 1 -- 172.21.15.89:0/3001115479 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c00cd80 con 0x7f03201047e0 2024-09-10T08:40:39.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.690+0000 7f031d600640 1 -- 172.21.15.89:0/3001115479 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f030c010910 con 0x7f03201047e0 2024-09-10T08:40:39.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.690+0000 7f031d600640 1 -- 172.21.15.89:0/3001115479 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c018990 con 0x7f03201047e0 2024-09-10T08:40:39.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.690+0000 7f0326000640 1 -- 172.21.15.89:0/3001115479 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03201a0770 con 0x7f03201047e0 2024-09-10T08:40:39.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.690+0000 7f0326000640 1 -- 172.21.15.89:0/3001115479 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03201a0c60 con 0x7f03201047e0 2024-09-10T08:40:39.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.694+0000 7f031d600640 1 -- 172.21.15.89:0/3001115479 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f030c010ac0 con 0x7f03201047e0 2024-09-10T08:40:39.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.696+0000 7f031d600640 1 --2- 172.21.15.89:0/3001115479 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f03080a2c10 0x7f03080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:39.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.696+0000 7f031d600640 1 -- 172.21.15.89:0/3001115479 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f030c0cfac0 con 0x7f03201047e0 2024-09-10T08:40:39.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.697+0000 7f0324c00640 1 --2- 172.21.15.89:0/3001115479 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f03080a2c10 0x7f03080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:39.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.697+0000 7f032a060740 1 Processor -- start 2024-09-10T08:40:39.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.697+0000 7f032a060740 1 -- start start 2024-09-10T08:40:39.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.697+0000 7f032a060740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040591d50 0x561040592130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:39.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.697+0000 7f032a060740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561040592670 0x56104059c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:39.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.697+0000 7f032a060740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56104059cf30 0x56104059f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:39.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.697+0000 7f032a060740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5610404f83e0 con 0x56104059cf30 2024-09-10T08:40:39.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.697+0000 7f032a060740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56104049fd80 con 0x561040592670 2024-09-10T08:40:39.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.697+0000 7f032a060740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5610404f43a0 con 0x561040591d50 2024-09-10T08:40:39.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.698+0000 7f0324c00640 1 --2- 172.21.15.89:0/3001115479 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f03080a2c10 0x7f03080a50d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0314009230 tx=0x7f03140058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:39.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.698+0000 7f0325600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56104059cf30 0x56104059f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:39.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.698+0000 7f0325600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56104059cf30 0x56104059f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47250/0 (socket says 172.21.15.89:47250) 2024-09-10T08:40:39.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.698+0000 7f0325600640 1 -- 172.21.15.89:0/562696096 learned_addr learned my addr 172.21.15.89:0/562696096 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:39.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.698+0000 7f0324c00640 1 --2- 172.21.15.89:0/562696096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040591d50 0x561040592130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:39.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.698+0000 7f031fe00640 1 --2- 172.21.15.89:0/562696096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561040592670 0x56104059c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:39.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.699+0000 7f0325600640 1 -- 172.21.15.89:0/562696096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040591d50 msgr2=0x561040592130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:39.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.699+0000 7f0325600640 1 --2- 172.21.15.89:0/562696096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040591d50 0x561040592130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.699+0000 7f0325600640 1 -- 172.21.15.89:0/562696096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561040592670 msgr2=0x56104059c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:39.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.699+0000 7f0325600640 1 --2- 172.21.15.89:0/562696096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561040592670 0x56104059c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.699+0000 7f0325600640 1 -- 172.21.15.89:0/562696096 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610404beae0 con 0x56104059cf30 2024-09-10T08:40:39.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.699+0000 7f0325600640 1 --2- 172.21.15.89:0/562696096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56104059cf30 0x56104059f320 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0318009f50 tx=0x7f03180118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e1e4894b407f79 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:39.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.699+0000 7f0311c00640 1 -- 172.21.15.89:0/562696096 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318004b20 con 0x56104059cf30 2024-09-10T08:40:39.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.700+0000 7f0311c00640 1 -- 172.21.15.89:0/562696096 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0318004cc0 con 0x56104059cf30 2024-09-10T08:40:39.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.700+0000 7f0311c00640 1 -- 172.21.15.89:0/562696096 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318005020 con 0x56104059cf30 2024-09-10T08:40:39.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.700+0000 7f032a060740 1 -- 172.21.15.89:0/562696096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56104059cf30 msgr2=0x56104059f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:39.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.700+0000 7f032a060740 1 --2- 172.21.15.89:0/562696096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56104059cf30 0x56104059f320 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0318009f50 tx=0x7f03180118d0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.701+0000 7f032a060740 1 -- 172.21.15.89:0/562696096 shutdown_connections 2024-09-10T08:40:39.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.701+0000 7f032a060740 1 --2- 172.21.15.89:0/562696096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56104059cf30 0x56104059f320 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.701+0000 7f032a060740 1 --2- 172.21.15.89:0/562696096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561040592670 0x56104059c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.701+0000 7f032a060740 1 --2- 172.21.15.89:0/562696096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040591d50 0x561040592130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.701+0000 7f032a060740 1 -- 172.21.15.89:0/562696096 >> 172.21.15.89:0/562696096 conn(0x56104058e2e0 msgr2=0x56104058e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:39.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.701+0000 7f032a060740 1 -- 172.21.15.89:0/562696096 shutdown_connections 2024-09-10T08:40:39.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.701+0000 7f032a060740 1 -- 172.21.15.89:0/562696096 wait complete. 2024-09-10T08:40:39.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.702+0000 7f032a060740 1 -- start start 2024-09-10T08:40:39.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.702+0000 7f032a060740 1 Processor -- start 2024-09-10T08:40:39.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.703+0000 7f032a060740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561040591d50 0x561040599e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:39.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.703+0000 7f032a060740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040592670 0x56104059a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:39.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.703+0000 7f032a060740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56104059cf30 0x56104059a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:39.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.703+0000 7f032a060740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56104059fba0 con 0x56104059cf30 2024-09-10T08:40:39.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.703+0000 7f032a060740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5610404b30f0 con 0x561040591d50 2024-09-10T08:40:39.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.703+0000 7f032a060740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56104059fd20 con 0x561040592670 2024-09-10T08:40:39.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.703+0000 7f032a060740 10 client.0 _ll_register_callbacks cb 0x5610404a49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:39.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.703+0000 7f0324c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561040591d50 0x561040599e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:39.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.703+0000 7f0324c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561040591d50 0x561040599e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:48118/0 (socket says 172.21.15.89:48118) 2024-09-10T08:40:39.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.703+0000 7f0324c00640 1 -- 172.21.15.89:0/2443709138 learned_addr learned my addr 172.21.15.89:0/2443709138 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:39.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.703+0000 7f031fe00640 1 --2- 172.21.15.89:0/2443709138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040592670 0x56104059a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:39.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.704+0000 7f0325600640 1 --2- 172.21.15.89:0/2443709138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56104059cf30 0x56104059a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:39.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.704+0000 7f031fe00640 1 -- 172.21.15.89:0/2443709138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561040591d50 msgr2=0x561040599e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:39.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.704+0000 7f031fe00640 1 --2- 172.21.15.89:0/2443709138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561040591d50 0x561040599e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.704+0000 7f031fe00640 1 -- 172.21.15.89:0/2443709138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56104059cf30 msgr2=0x56104059a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:39.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.704+0000 7f031fe00640 1 --2- 172.21.15.89:0/2443709138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56104059cf30 0x56104059a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.704+0000 7f031fe00640 1 -- 172.21.15.89:0/2443709138 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610402693c0 con 0x561040592670 2024-09-10T08:40:39.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.704+0000 7f0325600640 1 --2- 172.21.15.89:0/2443709138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56104059cf30 0x56104059a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:40:39.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.705+0000 7f031fe00640 1 --2- 172.21.15.89:0/2443709138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040592670 0x56104059a3a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f030c0cd040 tx=0x7f030c09be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1fb0a8ee8fc04af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:39.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.705+0000 7f0306000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:40:39.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.705+0000 7f0306000640 1 -- 172.21.15.89:0/2443709138 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c0a4770 con 0x561040592670 2024-09-10T08:40:39.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.705+0000 7f0306000640 1 -- 172.21.15.89:0/2443709138 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f030c018b30 con 0x561040592670 2024-09-10T08:40:39.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.706+0000 7f032a060740 1 -- 172.21.15.89:0/2443709138 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0318004cc0 con 0x561040592670 2024-09-10T08:40:39.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.706+0000 7f0306000640 1 -- 172.21.15.89:0/2443709138 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f030c018d30 con 0x561040592670 2024-09-10T08:40:39.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.706+0000 7f032a060740 10 client.25507 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:39.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.706+0000 7f032a060740 1 -- 172.21.15.89:0/2443709138 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5610405ab900 con 0x561040592670 2024-09-10T08:40:39.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.706+0000 7f0306000640 1 -- 172.21.15.89:0/2443709138 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0318004cc0 con 0x561040592670 2024-09-10T08:40:39.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.706+0000 7f0306000640 1 -- 172.21.15.89:0/2443709138 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f030c097b50 con 0x561040592670 2024-09-10T08:40:39.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.707+0000 7f032a060740 10 client.25507 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:39.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.707+0000 7f032a060740 10 client.25507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:40:39.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.707+0000 7f032a060740 20 client.25507 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:39.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.707+0000 7f032a060740 1 --2- 172.21.15.89:0/2443709138 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5610405acca0 0x5610405af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:39.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.707+0000 7f032a060740 4 client.25507 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:40:39.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.707+0000 7f032a060740 1 -- 172.21.15.89:0/2443709138 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561040505950 con 0x5610405acca0 2024-09-10T08:40:39.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.707+0000 7f0324c00640 1 --2- 172.21.15.89:0/2443709138 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5610405acca0 0x5610405af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:39.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.708+0000 7f0324c00640 1 --2- 172.21.15.89:0/2443709138 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5610405acca0 0x5610405af0c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e645912aa968741f server_cookie=9127e1e0de1fce7a in_seq=0 out_seq=0 2024-09-10T08:40:39.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.708+0000 7f0306000640 10 client.25507 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:39.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.709+0000 7f0306000640 1 -- 172.21.15.89:0/2443709138 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561040505950 con 0x5610405acca0 2024-09-10T08:40:39.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.709+0000 7f0306000640 10 client.25507 handle_command_reply: tid=0 2024-09-10T08:40:39.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.709+0000 7f032a060740 1 client.25507 shutdown 2024-09-10T08:40:39.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.710+0000 7f032a060740 1 -- 172.21.15.89:0/2443709138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040592670 msgr2=0x56104059a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:39.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.710+0000 7f032a060740 1 --2- 172.21.15.89:0/2443709138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040592670 0x56104059a3a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f030c0cd040 tx=0x7f030c09be50 comp rx=0 tx=0).stop 2024-09-10T08:40:39.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.710+0000 7f032a060740 1 -- 172.21.15.89:0/2443709138 shutdown_connections 2024-09-10T08:40:39.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.710+0000 7f032a060740 1 --2- 172.21.15.89:0/2443709138 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5610405acca0 0x5610405af0c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.710+0000 7f032a060740 1 --2- 172.21.15.89:0/2443709138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56104059cf30 0x56104059a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.710+0000 7f032a060740 1 --2- 172.21.15.89:0/2443709138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040592670 0x56104059a3a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.710+0000 7f0306000640 0 client.25507 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:39.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.710+0000 7f032a060740 1 --2- 172.21.15.89:0/2443709138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561040591d50 0x561040599e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.710+0000 7f032a060740 1 -- 172.21.15.89:0/2443709138 >> 172.21.15.89:0/2443709138 conn(0x56104058e2e0 msgr2=0x561040593fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:39.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.710+0000 7f032a060740 1 -- 172.21.15.89:0/2443709138 shutdown_connections 2024-09-10T08:40:39.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.710+0000 7f032a060740 1 -- 172.21.15.89:0/2443709138 wait complete. 2024-09-10T08:40:39.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.710+0000 7f032a060740 20 client.25507 trim_cache size 0 max 16384 2024-09-10T08:40:39.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.743+0000 7f032a060740 1 Processor -- start 2024-09-10T08:40:39.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.743+0000 7f032a060740 1 -- start start 2024-09-10T08:40:39.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.744+0000 7f032a060740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561040591d50 0x56104059a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:39.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.744+0000 7f032a060740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040592670 0x561040593eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:39.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.744+0000 7f032a060740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56104059cf30 0x5610405b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:39.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.744+0000 7f032a060740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f03201c03e0 con 0x561040591d50 2024-09-10T08:40:39.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.744+0000 7f032a060740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f03201bdcb0 con 0x56104059cf30 2024-09-10T08:40:39.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.744+0000 7f032a060740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f03201bbb00 con 0x561040592670 2024-09-10T08:40:39.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.744+0000 7f0324c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561040591d50 0x56104059a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:39.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.744+0000 7f0325600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56104059cf30 0x5610405b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:39.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.744+0000 7f0324c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561040591d50 0x56104059a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47266/0 (socket says 172.21.15.89:47266) 2024-09-10T08:40:39.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.744+0000 7f0324c00640 1 -- 172.21.15.89:0/1072936177 learned_addr learned my addr 172.21.15.89:0/1072936177 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:39.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.744+0000 7f031fe00640 1 --2- 172.21.15.89:0/1072936177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040592670 0x561040593eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:39.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.745+0000 7f0325600640 1 -- 172.21.15.89:0/1072936177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040592670 msgr2=0x561040593eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:39.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.745+0000 7f0325600640 1 --2- 172.21.15.89:0/1072936177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040592670 0x561040593eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.745+0000 7f0325600640 1 -- 172.21.15.89:0/1072936177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561040591d50 msgr2=0x56104059a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:39.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.745+0000 7f0325600640 1 --2- 172.21.15.89:0/1072936177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561040591d50 0x56104059a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.745+0000 7f0325600640 1 -- 172.21.15.89:0/1072936177 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56104058f470 con 0x56104059cf30 2024-09-10T08:40:39.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.745+0000 7f0325600640 1 --2- 172.21.15.89:0/1072936177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56104059cf30 0x5610405b4990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f031801b040 tx=0x7f0318003a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bb5c82ce7fc3844 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:39.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.745+0000 7f0311200640 1 -- 172.21.15.89:0/1072936177 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318003c20 con 0x56104059cf30 2024-09-10T08:40:39.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.745+0000 7f0311200640 1 -- 172.21.15.89:0/1072936177 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0318027070 con 0x56104059cf30 2024-09-10T08:40:39.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.745+0000 7f0311200640 1 -- 172.21.15.89:0/1072936177 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03180226b0 con 0x56104059cf30 2024-09-10T08:40:39.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.746+0000 7f032a060740 1 -- 172.21.15.89:0/1072936177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56104059cf30 msgr2=0x5610405b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:39.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.746+0000 7f032a060740 1 --2- 172.21.15.89:0/1072936177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56104059cf30 0x5610405b4990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f031801b040 tx=0x7f0318003a30 comp rx=0 tx=0).stop 2024-09-10T08:40:39.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.747+0000 7f032a060740 1 -- 172.21.15.89:0/1072936177 shutdown_connections 2024-09-10T08:40:39.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.747+0000 7f032a060740 1 --2- 172.21.15.89:0/1072936177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56104059cf30 0x5610405b4990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.747+0000 7f032a060740 1 --2- 172.21.15.89:0/1072936177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040592670 0x561040593eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.747+0000 7f032a060740 1 --2- 172.21.15.89:0/1072936177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561040591d50 0x56104059a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.747+0000 7f032a060740 1 -- 172.21.15.89:0/1072936177 >> 172.21.15.89:0/1072936177 conn(0x56104058e2e0 msgr2=0x5610405937a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:39.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.747+0000 7f032a060740 1 -- 172.21.15.89:0/1072936177 shutdown_connections 2024-09-10T08:40:39.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.747+0000 7f032a060740 1 -- 172.21.15.89:0/1072936177 wait complete. 2024-09-10T08:40:39.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.747+0000 7f032a060740 1 -- start start 2024-09-10T08:40:39.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.748+0000 7f032a060740 1 Processor -- start 2024-09-10T08:40:39.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.748+0000 7f032a060740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040591d50 0x5610405a5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:39.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.748+0000 7f032a060740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561040592670 0x5610405a5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:39.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.748+0000 7f032a060740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56104059cf30 0x5610405a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:39.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.748+0000 7f032a060740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f03201bc170 con 0x561040592670 2024-09-10T08:40:39.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.748+0000 7f032a060740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f03201bd5e0 con 0x56104059cf30 2024-09-10T08:40:39.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.748+0000 7f032a060740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f03201bc840 con 0x561040591d50 2024-09-10T08:40:39.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.748+0000 7f032a060740 10 client.0 _ll_register_callbacks cb 0x7f030c09c490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:39.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.748+0000 7f031fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561040592670 0x5610405a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:39.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.749+0000 7f031fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561040592670 0x5610405a5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47276/0 (socket says 172.21.15.89:47276) 2024-09-10T08:40:39.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.749+0000 7f031fe00640 1 -- 172.21.15.89:0/1919685053 learned_addr learned my addr 172.21.15.89:0/1919685053 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:39.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.749+0000 7f0324c00640 1 --2- 172.21.15.89:0/1919685053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040591d50 0x5610405a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:39.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.749+0000 7f0325600640 1 --2- 172.21.15.89:0/1919685053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56104059cf30 0x5610405a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:39.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.749+0000 7f031fe00640 1 -- 172.21.15.89:0/1919685053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040591d50 msgr2=0x5610405a5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:39.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.749+0000 7f031fe00640 1 --2- 172.21.15.89:0/1919685053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040591d50 0x5610405a5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.749+0000 7f031fe00640 1 -- 172.21.15.89:0/1919685053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56104059cf30 msgr2=0x5610405a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:39.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.749+0000 7f031fe00640 1 --2- 172.21.15.89:0/1919685053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56104059cf30 0x5610405a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.749+0000 7f031fe00640 1 -- 172.21.15.89:0/1919685053 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610405a25e0 con 0x561040592670 2024-09-10T08:40:39.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.749+0000 7f0325600640 1 --2- 172.21.15.89:0/1919685053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56104059cf30 0x5610405a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:39.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.749+0000 7f0324c00640 1 --2- 172.21.15.89:0/1919685053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040591d50 0x5610405a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:40:39.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.750+0000 7f031fe00640 1 --2- 172.21.15.89:0/1919685053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561040592670 0x5610405a5690 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f030c098710 tx=0x7f030c0a47a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfaef61d30c2efc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:39.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.750+0000 7f0306000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:40:39.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.750+0000 7f0306000640 1 -- 172.21.15.89:0/1919685053 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c09bb60 con 0x561040592670 2024-09-10T08:40:39.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.750+0000 7f0306000640 1 -- 172.21.15.89:0/1919685053 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f030c018b30 con 0x561040592670 2024-09-10T08:40:39.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.750+0000 7f032a060740 1 -- 172.21.15.89:0/1919685053 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0318027070 con 0x561040592670 2024-09-10T08:40:39.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.750+0000 7f0306000640 1 -- 172.21.15.89:0/1919685053 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f030c09add0 con 0x561040592670 2024-09-10T08:40:39.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.750+0000 7f032a060740 10 client.15495 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:39.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.750+0000 7f032a060740 1 -- 172.21.15.89:0/1919685053 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561040618300 con 0x561040592670 2024-09-10T08:40:39.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.751+0000 7f0306000640 1 -- 172.21.15.89:0/1919685053 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0318027070 con 0x561040592670 2024-09-10T08:40:39.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.751+0000 7f0306000640 1 -- 172.21.15.89:0/1919685053 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f030c0a12d0 con 0x561040592670 2024-09-10T08:40:39.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.751+0000 7f032a060740 10 client.15495 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:39.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.751+0000 7f032a060740 10 client.15495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:40:39.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.751+0000 7f032a060740 20 client.15495 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:39.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.751+0000 7f032a060740 1 --2- 172.21.15.89:0/1919685053 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x561040618a70 0x56104061ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:39.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.751+0000 7f032a060740 4 client.15495 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:40:39.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.751+0000 7f032a060740 1 -- 172.21.15.89:0/1919685053 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5610404ffe60 con 0x561040618a70 2024-09-10T08:40:39.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.752+0000 7f0324c00640 1 --2- 172.21.15.89:0/1919685053 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x561040618a70 0x56104061ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:39.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.752+0000 7f0324c00640 1 --2- 172.21.15.89:0/1919685053 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x561040618a70 0x56104061ae90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd87da144de15d9f server_cookie=ae99b52020207b89 in_seq=0 out_seq=0 2024-09-10T08:40:39.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.752+0000 7f0306000640 10 client.15495 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:39.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.753+0000 7f0306000640 1 -- 172.21.15.89:0/1919685053 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5610404ffe60 con 0x561040618a70 2024-09-10T08:40:39.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.753+0000 7f0306000640 10 client.15495 handle_command_reply: tid=0 2024-09-10T08:40:39.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.753+0000 7f032a060740 1 client.15495 shutdown 2024-09-10T08:40:39.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.753+0000 7f032a060740 1 -- 172.21.15.89:0/1919685053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561040592670 msgr2=0x5610405a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:39.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.753+0000 7f032a060740 1 --2- 172.21.15.89:0/1919685053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561040592670 0x5610405a5690 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f030c098710 tx=0x7f030c0a47a0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.753+0000 7f032a060740 1 -- 172.21.15.89:0/1919685053 shutdown_connections 2024-09-10T08:40:39.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.753+0000 7f032a060740 1 --2- 172.21.15.89:0/1919685053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56104059cf30 0x5610405a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.753+0000 7f032a060740 1 --2- 172.21.15.89:0/1919685053 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x561040618a70 0x56104061ae90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.753+0000 7f032a060740 1 --2- 172.21.15.89:0/1919685053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561040592670 0x5610405a5690 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.753+0000 7f032a060740 1 --2- 172.21.15.89:0/1919685053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561040591d50 0x5610405a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.753+0000 7f032a060740 1 -- 172.21.15.89:0/1919685053 >> 172.21.15.89:0/1919685053 conn(0x56104058e2e0 msgr2=0x56104058d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:39.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.753+0000 7f0306000640 0 client.15495 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:39.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.754+0000 7f032a060740 1 -- 172.21.15.89:0/1919685053 shutdown_connections 2024-09-10T08:40:39.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.754+0000 7f032a060740 1 -- 172.21.15.89:0/1919685053 wait complete. 2024-09-10T08:40:39.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.754+0000 7f032a060740 20 client.15495 trim_cache size 0 max 16384 2024-09-10T08:40:39.756 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:40:39.756 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:40:39.756 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2024-09-10T08:40:39.757 INFO:teuthology.orchestra.run.smithi089.stdout: "7bc45783-bf82-448b-a2eb-70f773ce98f9": { 2024-09-10T08:40:39.757 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2024-09-10T08:40:39.757 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "7bc45783-bf82-448b-a2eb-70f773ce98f9", 2024-09-10T08:40:39.757 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2024-09-10T08:40:39.757 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:40:39.757 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:40:39.757 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:40:39.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.755+0000 7f0326000640 1 -- 172.21.15.89:0/3001115479 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f03080a2c10 msgr2=0x7f03080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:39.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.755+0000 7f0326000640 1 --2- 172.21.15.89:0/3001115479 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f03080a2c10 0x7f03080a50d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0314009230 tx=0x7f03140058d0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.755+0000 7f0326000640 1 -- 172.21.15.89:0/3001115479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03201047e0 msgr2=0x7f032010dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:39.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.755+0000 7f0326000640 1 --2- 172.21.15.89:0/3001115479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03201047e0 0x7f032010dab0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f030c00e9f0 tx=0x7f030c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.755+0000 7f0326000640 1 -- 172.21.15.89:0/3001115479 shutdown_connections 2024-09-10T08:40:39.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.755+0000 7f0326000640 1 --2- 172.21.15.89:0/3001115479 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f03080a2c10 0x7f03080a50d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.755+0000 7f0326000640 1 --2- 172.21.15.89:0/3001115479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0320108f60 0x7f032010e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.755+0000 7f0326000640 1 --2- 172.21.15.89:0/3001115479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03201047e0 0x7f032010dab0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.755+0000 7f0326000640 1 --2- 172.21.15.89:0/3001115479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0320103e30 0x7f0320110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:39.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.755+0000 7f0326000640 1 -- 172.21.15.89:0/3001115479 >> 172.21.15.89:0/3001115479 conn(0x7f03200ff910 msgr2=0x7f0320100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:39.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.755+0000 7f0326000640 1 -- 172.21.15.89:0/3001115479 shutdown_connections 2024-09-10T08:40:39.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:39.755+0000 7f0326000640 1 -- 172.21.15.89:0/3001115479 wait complete. 2024-09-10T08:40:40.708 INFO:tasks.cephfs.filesystem:scrub status for tag:7bc45783-bf82-448b-a2eb-70f773ce98f9 - {'path': '/', 'tag': '7bc45783-bf82-448b-a2eb-70f773ce98f9', 'options': 'recursive,force'} 2024-09-10T08:40:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:40 smithi111 ceph-mon[35220]: pgmap v415: 129 pgs: 129 active+clean; 669 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-10T08:40:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:40 smithi111 ceph-mon[35220]: scrub summary: idle 2024-09-10T08:40:41.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:40 smithi089 ceph-mon[30757]: pgmap v415: 129 pgs: 129 active+clean; 669 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-10T08:40:41.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:40 smithi089 ceph-mon[30757]: scrub summary: idle 2024-09-10T08:40:41.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:40 smithi149 ceph-mon[36167]: pgmap v415: 129 pgs: 129 active+clean; 669 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-10T08:40:41.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:40 smithi149 ceph-mon[36167]: scrub summary: idle 2024-09-10T08:40:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.202+0000 7fa4aa400640 1 -- 172.21.15.89:0/4017497719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a4076bb0 msgr2=0x7fa4a4076fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.202+0000 7fa4aa400640 1 --2- 172.21.15.89:0/4017497719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a4076bb0 0x7fa4a4076fb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa498009900 tx=0x7fa49802a410 comp rx=0 tx=0).stop 2024-09-10T08:40:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.202+0000 7fa4aa400640 1 -- 172.21.15.89:0/4017497719 shutdown_connections 2024-09-10T08:40:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.202+0000 7fa4aa400640 1 --2- 172.21.15.89:0/4017497719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4a4078790 0x7fa4a4079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.202+0000 7fa4aa400640 1 --2- 172.21.15.89:0/4017497719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4a4077df0 0x7fa4a4078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.202+0000 7fa4aa400640 1 --2- 172.21.15.89:0/4017497719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a4076bb0 0x7fa4a4076fb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.202+0000 7fa4aa400640 1 -- 172.21.15.89:0/4017497719 >> 172.21.15.89:0/4017497719 conn(0x7fa4a40ff2f0 msgr2=0x7fa4a4101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.202+0000 7fa4aa400640 1 -- 172.21.15.89:0/4017497719 shutdown_connections 2024-09-10T08:40:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.202+0000 7fa4aa400640 1 -- 172.21.15.89:0/4017497719 wait complete. 2024-09-10T08:40:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.204+0000 7fa4aa400640 1 Processor -- start 2024-09-10T08:40:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.204+0000 7fa4aa400640 1 -- start start 2024-09-10T08:40:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.204+0000 7fa4aa400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4a4076bb0 0x7fa4a419e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.204+0000 7fa4aa400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a4077df0 0x7fa4a419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.204+0000 7fa4aa400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4a4078790 0x7fa4a41a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.204+0000 7fa4aa400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a4113c50 con 0x7fa4a4077df0 2024-09-10T08:40:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.204+0000 7fa4aa400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a4113ad0 con 0x7fa4a4078790 2024-09-10T08:40:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.204+0000 7fa4aa400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a4113dd0 con 0x7fa4a4076bb0 2024-09-10T08:40:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.204+0000 7fa4a9000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4a4076bb0 0x7fa4a419e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.204+0000 7fa4a3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a4077df0 0x7fa4a419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.205+0000 7fa4a9000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4a4076bb0 0x7fa4a419e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:34814/0 (socket says 172.21.15.89:34814) 2024-09-10T08:40:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.205+0000 7fa4a9000640 1 -- 172.21.15.89:0/2252122432 learned_addr learned my addr 172.21.15.89:0/2252122432 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.205+0000 7fa4a3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a4077df0 0x7fa4a419e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47290/0 (socket says 172.21.15.89:47290) 2024-09-10T08:40:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.205+0000 7fa4a9a00640 1 --2- 172.21.15.89:0/2252122432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4a4078790 0x7fa4a41a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.205+0000 7fa4a9000640 1 -- 172.21.15.89:0/2252122432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4a4078790 msgr2=0x7fa4a41a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.205+0000 7fa4a9000640 1 --2- 172.21.15.89:0/2252122432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4a4078790 0x7fa4a41a5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.205+0000 7fa4a9000640 1 -- 172.21.15.89:0/2252122432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a4077df0 msgr2=0x7fa4a419e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.205+0000 7fa4a9000640 1 --2- 172.21.15.89:0/2252122432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a4077df0 0x7fa4a419e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.205+0000 7fa4a9000640 1 -- 172.21.15.89:0/2252122432 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a41a5ea0 con 0x7fa4a4076bb0 2024-09-10T08:40:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.205+0000 7fa4a9000640 1 --2- 172.21.15.89:0/2252122432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4a4076bb0 0x7fa4a419e280 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa498031440 tx=0x7fa498032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.206+0000 7fa4a1600640 1 -- 172.21.15.89:0/2252122432 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49803f030 con 0x7fa4a4076bb0 2024-09-10T08:40:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.206+0000 7fa4aa400640 1 -- 172.21.15.89:0/2252122432 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a41a6130 con 0x7fa4a4076bb0 2024-09-10T08:40:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.206+0000 7fa4a1600640 1 -- 172.21.15.89:0/2252122432 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa498003ce0 con 0x7fa4a4076bb0 2024-09-10T08:40:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.206+0000 7fa4a1600640 1 -- 172.21.15.89:0/2252122432 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498035070 con 0x7fa4a4076bb0 2024-09-10T08:40:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.206+0000 7fa4aa400640 1 -- 172.21.15.89:0/2252122432 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a41a6620 con 0x7fa4a4076bb0 2024-09-10T08:40:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.209+0000 7fa4a1600640 1 -- 172.21.15.89:0/2252122432 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa498042030 con 0x7fa4a4076bb0 2024-09-10T08:40:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.210+0000 7fa4ae8cb740 1 Processor -- start 2024-09-10T08:40:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.210+0000 7fa4ae8cb740 1 -- start start 2024-09-10T08:40:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.210+0000 7fa4ae8cb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922c260 0x55b3a922c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.210+0000 7fa4ae8cb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922cba0 0x55b3a9231660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.210+0000 7fa4ae8cb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3a9231c30 0x55b3a92340f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.210+0000 7fa4ae8cb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b3a9191100 con 0x55b3a922c260 2024-09-10T08:40:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.210+0000 7fa4ae8cb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b3a91391b0 con 0x55b3a922cba0 2024-09-10T08:40:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.210+0000 7fa4ae8cb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b3a918d0c0 con 0x55b3a9231c30 2024-09-10T08:40:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.211+0000 7fa4a9a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922c260 0x55b3a922c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.211+0000 7fa4a9000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922cba0 0x55b3a9231660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.211+0000 7fa4a9a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922c260 0x55b3a922c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47294/0 (socket says 172.21.15.89:47294) 2024-09-10T08:40:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.211+0000 7fa4a9a00640 1 -- 172.21.15.89:0/3070459440 learned_addr learned my addr 172.21.15.89:0/3070459440 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.211+0000 7fa4a9000640 1 -- 172.21.15.89:0/3070459440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3a9231c30 msgr2=0x55b3a92340f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:40:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.211+0000 7fa4a9000640 1 --2- 172.21.15.89:0/3070459440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3a9231c30 0x55b3a92340f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.211+0000 7fa4a9000640 1 -- 172.21.15.89:0/3070459440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922c260 msgr2=0x55b3a922c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.211+0000 7fa4a9000640 1 --2- 172.21.15.89:0/3070459440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922c260 0x55b3a922c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.211+0000 7fa4a9000640 1 -- 172.21.15.89:0/3070459440 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3a8ea4210 con 0x55b3a922cba0 2024-09-10T08:40:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.212+0000 7fa4a1600640 1 --2- 172.21.15.89:0/2252122432 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa48c0a2b80 0x7fa48c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.212+0000 7fa4a1600640 1 -- 172.21.15.89:0/2252122432 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa4980efc20 con 0x7fa4a4076bb0 2024-09-10T08:40:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.212+0000 7fa4a9000640 1 --2- 172.21.15.89:0/3070459440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922cba0 0x55b3a9231660 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa4980ed320 tx=0x7fa4980f3e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e2fc3d82b07cd4b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.212+0000 7fa4a9a00640 1 --2- 172.21.15.89:0/2252122432 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa48c0a2b80 0x7fa48c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.212+0000 7fa496000640 1 -- 172.21.15.89:0/3070459440 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498049020 con 0x55b3a922cba0 2024-09-10T08:40:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.212+0000 7fa496000640 1 -- 172.21.15.89:0/3070459440 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4980bc160 con 0x55b3a922cba0 2024-09-10T08:40:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.212+0000 7fa496000640 1 -- 172.21.15.89:0/3070459440 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4980b8400 con 0x55b3a922cba0 2024-09-10T08:40:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.213+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/3070459440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922cba0 msgr2=0x55b3a9231660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.213+0000 7fa4a9a00640 1 --2- 172.21.15.89:0/2252122432 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa48c0a2b80 0x7fa48c0a5040 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0116b0 tx=0x7fa49c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.213+0000 7fa4ae8cb740 1 --2- 172.21.15.89:0/3070459440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922cba0 0x55b3a9231660 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa4980ed320 tx=0x7fa4980f3e80 comp rx=0 tx=0).stop 2024-09-10T08:40:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.213+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/3070459440 shutdown_connections 2024-09-10T08:40:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.213+0000 7fa4ae8cb740 1 --2- 172.21.15.89:0/3070459440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3a9231c30 0x55b3a92340f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.213+0000 7fa4ae8cb740 1 --2- 172.21.15.89:0/3070459440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922cba0 0x55b3a9231660 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.213+0000 7fa4ae8cb740 1 --2- 172.21.15.89:0/3070459440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922c260 0x55b3a922c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.213+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/3070459440 >> 172.21.15.89:0/3070459440 conn(0x55b3a92278d0 msgr2=0x55b3a9227cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.213+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/3070459440 shutdown_connections 2024-09-10T08:40:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.214+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/3070459440 wait complete. 2024-09-10T08:40:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.214+0000 7fa4ae8cb740 1 -- start start 2024-09-10T08:40:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.214+0000 7fa4ae8cb740 1 Processor -- start 2024-09-10T08:40:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.215+0000 7fa4ae8cb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922c260 0x55b3a92361f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.215+0000 7fa4ae8cb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922cba0 0x55b3a9236730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.215+0000 7fa4ae8cb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3a9231c30 0x55b3a923d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.215+0000 7fa4ae8cb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b3a92349c0 con 0x55b3a922cba0 2024-09-10T08:40:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.215+0000 7fa4ae8cb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b3a911a090 con 0x55b3a922c260 2024-09-10T08:40:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.215+0000 7fa4ae8cb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b3a9234b40 con 0x55b3a9231c30 2024-09-10T08:40:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.215+0000 7fa4ae8cb740 10 client.0 _ll_register_callbacks cb 0x55b3a916ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.215+0000 7fa4a3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922cba0 0x55b3a9236730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.215+0000 7fa4a3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922cba0 0x55b3a9236730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47296/0 (socket says 172.21.15.89:47296) 2024-09-10T08:40:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.215+0000 7fa4a3e00640 1 -- 172.21.15.89:0/817584452 learned_addr learned my addr 172.21.15.89:0/817584452 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.215+0000 7fa4a9000640 1 --2- 172.21.15.89:0/817584452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922c260 0x55b3a92361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.215+0000 7fa4a9a00640 1 --2- 172.21.15.89:0/817584452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3a9231c30 0x55b3a923d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.216+0000 7fa4a3e00640 1 -- 172.21.15.89:0/817584452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3a9231c30 msgr2=0x55b3a923d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.216+0000 7fa4a3e00640 1 --2- 172.21.15.89:0/817584452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3a9231c30 0x55b3a923d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.216+0000 7fa4a3e00640 1 -- 172.21.15.89:0/817584452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922c260 msgr2=0x55b3a92361f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.216+0000 7fa4a3e00640 1 --2- 172.21.15.89:0/817584452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922c260 0x55b3a92361f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.216+0000 7fa4a3e00640 1 -- 172.21.15.89:0/817584452 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3a8f70620 con 0x55b3a922cba0 2024-09-10T08:40:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.216+0000 7fa4a9000640 1 --2- 172.21.15.89:0/817584452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922c260 0x55b3a92361f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.216+0000 7fa4a3e00640 1 --2- 172.21.15.89:0/817584452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922cba0 0x55b3a9236730 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa4900116f0 tx=0x7fa490011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79bb643414735824 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.216+0000 7fa48aa00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:40:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.216+0000 7fa48aa00640 1 -- 172.21.15.89:0/817584452 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49000f8c0 con 0x55b3a922cba0 2024-09-10T08:40:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.216+0000 7fa48aa00640 1 -- 172.21.15.89:0/817584452 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa49000fa60 con 0x55b3a922cba0 2024-09-10T08:40:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.216+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/817584452 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4980bc160 con 0x55b3a922cba0 2024-09-10T08:40:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.217+0000 7fa48aa00640 1 -- 172.21.15.89:0/817584452 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa49000fc60 con 0x55b3a922cba0 2024-09-10T08:40:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.217+0000 7fa4ae8cb740 10 client.15507 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.217+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/817584452 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b3a9240630 con 0x55b3a922cba0 2024-09-10T08:40:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.217+0000 7fa48aa00640 1 -- 172.21.15.89:0/817584452 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4980bc160 con 0x55b3a922cba0 2024-09-10T08:40:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.217+0000 7fa48aa00640 1 -- 172.21.15.89:0/817584452 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa490018070 con 0x55b3a922cba0 2024-09-10T08:40:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.217+0000 7fa4ae8cb740 10 client.15507 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.217+0000 7fa4ae8cb740 10 client.15507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:40:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.217+0000 7fa4ae8cb740 20 client.15507 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.217+0000 7fa4ae8cb740 1 --2- 172.21.15.89:0/817584452 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b3a9241d00 0x55b3a9244150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.217+0000 7fa4ae8cb740 4 client.15507 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:40:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.217+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/817584452 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3a919e830 con 0x55b3a9241d00 2024-09-10T08:40:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.218+0000 7fa4a9000640 1 --2- 172.21.15.89:0/817584452 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b3a9241d00 0x55b3a9244150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.218+0000 7fa4a9000640 1 --2- 172.21.15.89:0/817584452 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b3a9241d00 0x55b3a9244150 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d791bf0be847ff server_cookie=31efa79a920b38ae in_seq=0 out_seq=0 2024-09-10T08:40:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.218+0000 7fa48aa00640 10 client.15507 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.219+0000 7fa48aa00640 1 -- 172.21.15.89:0/817584452 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3a919e830 con 0x55b3a9241d00 2024-09-10T08:40:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.219+0000 7fa48aa00640 10 client.15507 handle_command_reply: tid=0 2024-09-10T08:40:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.219+0000 7fa4ae8cb740 1 client.15507 shutdown 2024-09-10T08:40:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.219+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/817584452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922cba0 msgr2=0x55b3a9236730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.219+0000 7fa4ae8cb740 1 --2- 172.21.15.89:0/817584452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922cba0 0x55b3a9236730 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa4900116f0 tx=0x7fa490011bb0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.219+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/817584452 shutdown_connections 2024-09-10T08:40:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.219+0000 7fa4ae8cb740 1 --2- 172.21.15.89:0/817584452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3a9231c30 0x55b3a923d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.219+0000 7fa4ae8cb740 1 --2- 172.21.15.89:0/817584452 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b3a9241d00 0x55b3a9244150 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.219+0000 7fa4ae8cb740 1 --2- 172.21.15.89:0/817584452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922cba0 0x55b3a9236730 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.219+0000 7fa4ae8cb740 1 --2- 172.21.15.89:0/817584452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922c260 0x55b3a92361f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.219+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/817584452 >> 172.21.15.89:0/817584452 conn(0x55b3a92278d0 msgr2=0x55b3a92290e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.220+0000 7fa48aa00640 0 client.15507 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.220+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/817584452 shutdown_connections 2024-09-10T08:40:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.220+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/817584452 wait complete. 2024-09-10T08:40:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.220+0000 7fa4ae8cb740 20 client.15507 trim_cache size 0 max 16384 2024-09-10T08:40:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.248+0000 7fa4ae8cb740 1 Processor -- start 2024-09-10T08:40:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.248+0000 7fa4ae8cb740 1 -- start start 2024-09-10T08:40:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.248+0000 7fa4ae8cb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922c260 0x55b3a9227cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.248+0000 7fa4ae8cb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922cba0 0x55b3a923d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.248+0000 7fa4ae8cb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3a9231c30 0x55b3a92281f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.248+0000 7fa4ae8cb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41b35a0 con 0x55b3a922cba0 2024-09-10T08:40:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.248+0000 7fa4ae8cb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41b0e70 con 0x55b3a922c260 2024-09-10T08:40:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.248+0000 7fa4ae8cb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41aecc0 con 0x55b3a9231c30 2024-09-10T08:40:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.248+0000 7fa4a3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922cba0 0x55b3a923d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.249+0000 7fa4a3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922cba0 0x55b3a923d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47302/0 (socket says 172.21.15.89:47302) 2024-09-10T08:40:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.249+0000 7fa4a3e00640 1 -- 172.21.15.89:0/2673360644 learned_addr learned my addr 172.21.15.89:0/2673360644 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.249+0000 7fa4a9000640 1 --2- 172.21.15.89:0/2673360644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922c260 0x55b3a9227cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.249+0000 7fa4a9a00640 1 --2- 172.21.15.89:0/2673360644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3a9231c30 0x55b3a92281f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.249+0000 7fa4a9000640 1 -- 172.21.15.89:0/2673360644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3a9231c30 msgr2=0x55b3a92281f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.249+0000 7fa4a9000640 1 --2- 172.21.15.89:0/2673360644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3a9231c30 0x55b3a92281f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.249+0000 7fa4a9000640 1 -- 172.21.15.89:0/2673360644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922cba0 msgr2=0x55b3a923d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.249+0000 7fa4a9000640 1 --2- 172.21.15.89:0/2673360644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922cba0 0x55b3a923d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.249+0000 7fa4a9000640 1 -- 172.21.15.89:0/2673360644 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3a92276f0 con 0x55b3a922c260 2024-09-10T08:40:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.249+0000 7fa4a3e00640 1 --2- 172.21.15.89:0/2673360644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922cba0 0x55b3a923d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.249+0000 7fa4a9000640 1 --2- 172.21.15.89:0/2673360644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922c260 0x55b3a9227cb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x55b3a922d5d0 tx=0x7fa4980f3700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19afc4c9de829c64 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.249+0000 7fa495600640 1 -- 172.21.15.89:0/2673360644 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498049040 con 0x55b3a922c260 2024-09-10T08:40:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.249+0000 7fa495600640 1 -- 172.21.15.89:0/2673360644 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4980f3210 con 0x55b3a922c260 2024-09-10T08:40:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.250+0000 7fa495600640 1 -- 172.21.15.89:0/2673360644 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4980bbac0 con 0x55b3a922c260 2024-09-10T08:40:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.250+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/2673360644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922c260 msgr2=0x55b3a9227cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.250+0000 7fa4ae8cb740 1 --2- 172.21.15.89:0/2673360644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922c260 0x55b3a9227cb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x55b3a922d5d0 tx=0x7fa4980f3700 comp rx=0 tx=0).stop 2024-09-10T08:40:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.250+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/2673360644 shutdown_connections 2024-09-10T08:40:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.250+0000 7fa4ae8cb740 1 --2- 172.21.15.89:0/2673360644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3a9231c30 0x55b3a92281f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.250+0000 7fa4ae8cb740 1 --2- 172.21.15.89:0/2673360644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922cba0 0x55b3a923d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.250+0000 7fa4ae8cb740 1 --2- 172.21.15.89:0/2673360644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922c260 0x55b3a9227cb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.250+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/2673360644 >> 172.21.15.89:0/2673360644 conn(0x55b3a92278d0 msgr2=0x55b3a92361f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.250+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/2673360644 shutdown_connections 2024-09-10T08:40:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.250+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/2673360644 wait complete. 2024-09-10T08:40:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.251+0000 7fa4ae8cb740 1 -- start start 2024-09-10T08:40:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.251+0000 7fa4ae8cb740 1 Processor -- start 2024-09-10T08:40:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.252+0000 7fa4ae8cb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922c260 0x55b3a923d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.252+0000 7fa4ae8cb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922cba0 0x55b3a9239f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.252+0000 7fa4ae8cb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3a9231c30 0x55b3a923a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.252+0000 7fa4ae8cb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41af330 con 0x55b3a922c260 2024-09-10T08:40:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.252+0000 7fa4ae8cb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41b07a0 con 0x55b3a922cba0 2024-09-10T08:40:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.252+0000 7fa4ae8cb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41afa00 con 0x55b3a9231c30 2024-09-10T08:40:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.252+0000 7fa4ae8cb740 10 client.0 _ll_register_callbacks cb 0x7fa4900097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.252+0000 7fa4a9000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922c260 0x55b3a923d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.252+0000 7fa4a9000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922c260 0x55b3a923d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47318/0 (socket says 172.21.15.89:47318) 2024-09-10T08:40:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.252+0000 7fa4a9000640 1 -- 172.21.15.89:0/2817489727 learned_addr learned my addr 172.21.15.89:0/2817489727 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.252+0000 7fa4a9a00640 1 --2- 172.21.15.89:0/2817489727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3a9231c30 0x55b3a923a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.252+0000 7fa4a3e00640 1 --2- 172.21.15.89:0/2817489727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922cba0 0x55b3a9239f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.253+0000 7fa4a9a00640 1 -- 172.21.15.89:0/2817489727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922cba0 msgr2=0x55b3a9239f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.253+0000 7fa4a9a00640 1 --2- 172.21.15.89:0/2817489727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922cba0 0x55b3a9239f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.253+0000 7fa4a9a00640 1 -- 172.21.15.89:0/2817489727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922c260 msgr2=0x55b3a923d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.253+0000 7fa4a9a00640 1 --2- 172.21.15.89:0/2817489727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922c260 0x55b3a923d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.253+0000 7fa4a9a00640 1 -- 172.21.15.89:0/2817489727 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3a923aad0 con 0x55b3a9231c30 2024-09-10T08:40:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.253+0000 7fa4a3e00640 1 --2- 172.21.15.89:0/2817489727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922cba0 0x55b3a9239f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:40:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.253+0000 7fa4a9000640 1 --2- 172.21.15.89:0/2817489727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922c260 0x55b3a923d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:40:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.253+0000 7fa4a9a00640 1 --2- 172.21.15.89:0/2817489727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3a9231c30 0x55b3a923a460 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55b3a921bf90 tx=0x7fa49c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15991d40983acdf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.253+0000 7fa48aa00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:40:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.253+0000 7fa48aa00640 1 -- 172.21.15.89:0/2817489727 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c02e060 con 0x55b3a9231c30 2024-09-10T08:40:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.253+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/2817489727 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4980f3210 con 0x55b3a9231c30 2024-09-10T08:40:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.253+0000 7fa48aa00640 1 -- 172.21.15.89:0/2817489727 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa49c010700 con 0x55b3a9231c30 2024-09-10T08:40:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.254+0000 7fa48aa00640 1 -- 172.21.15.89:0/2817489727 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa49c035050 con 0x55b3a9231c30 2024-09-10T08:40:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.254+0000 7fa4ae8cb740 10 client.25528 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.254+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/2817489727 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b3a92abc30 con 0x55b3a9231c30 2024-09-10T08:40:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.254+0000 7fa48aa00640 1 -- 172.21.15.89:0/2817489727 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4980f3210 con 0x55b3a9231c30 2024-09-10T08:40:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.254+0000 7fa48aa00640 1 -- 172.21.15.89:0/2817489727 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa49c01eac0 con 0x55b3a9231c30 2024-09-10T08:40:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.254+0000 7fa4ae8cb740 10 client.25528 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.254+0000 7fa4ae8cb740 10 client.25528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:40:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.254+0000 7fa4ae8cb740 20 client.25528 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.254+0000 7fa4ae8cb740 1 --2- 172.21.15.89:0/2817489727 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b3a92ac5c0 0x55b3a92ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.254+0000 7fa4ae8cb740 4 client.25528 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:40:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.254+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/2817489727 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b3a9198b80 con 0x55b3a92ac5c0 2024-09-10T08:40:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.255+0000 7fa4a9000640 1 --2- 172.21.15.89:0/2817489727 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b3a92ac5c0 0x55b3a92ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.255+0000 7fa4a9000640 1 --2- 172.21.15.89:0/2817489727 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b3a92ac5c0 0x55b3a92ae9e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce600d32ce884dec server_cookie=dea5728d85753beb in_seq=0 out_seq=0 2024-09-10T08:40:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.255+0000 7fa48aa00640 10 client.25528 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.263+0000 7fa48aa00640 1 -- 172.21.15.89:0/2817489727 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b3a9198b80 con 0x55b3a92ac5c0 2024-09-10T08:40:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.263+0000 7fa48aa00640 10 client.25528 handle_command_reply: tid=0 2024-09-10T08:40:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.263+0000 7fa4ae8cb740 1 client.25528 shutdown 2024-09-10T08:40:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.263+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/2817489727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3a9231c30 msgr2=0x55b3a923a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.263+0000 7fa4ae8cb740 1 --2- 172.21.15.89:0/2817489727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3a9231c30 0x55b3a923a460 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55b3a921bf90 tx=0x7fa49c004a40 comp rx=0 tx=0).stop 2024-09-10T08:40:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.263+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/2817489727 shutdown_connections 2024-09-10T08:40:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.263+0000 7fa4ae8cb740 1 --2- 172.21.15.89:0/2817489727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b3a9231c30 0x55b3a923a460 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.263+0000 7fa4ae8cb740 1 --2- 172.21.15.89:0/2817489727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3a922cba0 0x55b3a9239f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.263+0000 7fa4ae8cb740 1 --2- 172.21.15.89:0/2817489727 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b3a92ac5c0 0x55b3a92ae9e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.263+0000 7fa4ae8cb740 1 --2- 172.21.15.89:0/2817489727 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b3a922c260 0x55b3a923d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.263+0000 7fa48aa00640 0 client.25528 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.263+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/2817489727 >> 172.21.15.89:0/2817489727 conn(0x55b3a92278d0 msgr2=0x55b3a92261f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.263+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/2817489727 shutdown_connections 2024-09-10T08:40:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.263+0000 7fa4ae8cb740 1 -- 172.21.15.89:0/2817489727 wait complete. 2024-09-10T08:40:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.264+0000 7fa4ae8cb740 20 client.25528 trim_cache size 0 max 16384 2024-09-10T08:40:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:40:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:40:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:40:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:40:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.264+0000 7fa4aa400640 1 -- 172.21.15.89:0/2252122432 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa48c0a2b80 msgr2=0x7fa48c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.264+0000 7fa4aa400640 1 --2- 172.21.15.89:0/2252122432 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa48c0a2b80 0x7fa48c0a5040 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0116b0 tx=0x7fa49c009500 comp rx=0 tx=0).stop 2024-09-10T08:40:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.264+0000 7fa4aa400640 1 -- 172.21.15.89:0/2252122432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4a4076bb0 msgr2=0x7fa4a419e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.264+0000 7fa4aa400640 1 --2- 172.21.15.89:0/2252122432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4a4076bb0 0x7fa4a419e280 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa498031440 tx=0x7fa498032ce0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.264+0000 7fa4aa400640 1 -- 172.21.15.89:0/2252122432 shutdown_connections 2024-09-10T08:40:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.264+0000 7fa4aa400640 1 --2- 172.21.15.89:0/2252122432 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa48c0a2b80 0x7fa48c0a5040 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.264+0000 7fa4aa400640 1 --2- 172.21.15.89:0/2252122432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4a4078790 0x7fa4a41a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.264+0000 7fa4aa400640 1 --2- 172.21.15.89:0/2252122432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a4077df0 0x7fa4a419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.264+0000 7fa4aa400640 1 --2- 172.21.15.89:0/2252122432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4a4076bb0 0x7fa4a419e280 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa498031440 tx=0x7fa498032ce0 comp rx=0 tx=0).stop 2024-09-10T08:40:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.264+0000 7fa4aa400640 1 -- 172.21.15.89:0/2252122432 >> 172.21.15.89:0/2252122432 conn(0x7fa4a40ff2f0 msgr2=0x7fa4a4100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.265+0000 7fa4aa400640 1 -- 172.21.15.89:0/2252122432 shutdown_connections 2024-09-10T08:40:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:42.265+0000 7fa4aa400640 1 -- 172.21.15.89:0/2252122432 wait complete. 2024-09-10T08:40:43.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:42 smithi111 ceph-mon[35220]: pgmap v416: 129 pgs: 129 active+clean; 669 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-10T08:40:43.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:42 smithi111 ceph-mon[35220]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-09-10T08:40:43.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:42 smithi089 ceph-mon[30757]: pgmap v416: 129 pgs: 129 active+clean; 669 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-10T08:40:43.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:42 smithi089 ceph-mon[30757]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-09-10T08:40:43.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:42 smithi149 ceph-mon[36167]: pgmap v416: 129 pgs: 129 active+clean; 669 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-10T08:40:43.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:42 smithi149 ceph-mon[36167]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-09-10T08:40:45.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:44 smithi111 ceph-mon[35220]: pgmap v417: 129 pgs: 129 active+clean; 669 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-10T08:40:45.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:44 smithi089 ceph-mon[30757]: pgmap v417: 129 pgs: 129 active+clean; 669 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-10T08:40:45.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:44 smithi149 ceph-mon[36167]: pgmap v417: 129 pgs: 129 active+clean; 669 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-10T08:40:47.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:46 smithi111 ceph-mon[35220]: pgmap v418: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-10T08:40:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:46 smithi089 ceph-mon[30757]: pgmap v418: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-10T08:40:47.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:46 smithi149 ceph-mon[36167]: pgmap v418: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-10T08:40:49.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:48 smithi111 ceph-mon[35220]: pgmap v419: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-09-10T08:40:49.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:48 smithi089 ceph-mon[30757]: pgmap v419: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-09-10T08:40:49.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:48 smithi149 ceph-mon[36167]: pgmap v419: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-09-10T08:40:51.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:50 smithi111 ceph-mon[35220]: pgmap v420: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-09-10T08:40:51.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:50 smithi089 ceph-mon[30757]: pgmap v420: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-09-10T08:40:51.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:50 smithi149 ceph-mon[36167]: pgmap v420: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-09-10T08:40:53.022 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:52 smithi149 ceph-mon[36167]: pgmap v421: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-10T08:40:53.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:52 smithi111 ceph-mon[35220]: pgmap v421: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-10T08:40:53.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:52 smithi089 ceph-mon[30757]: pgmap v421: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-10T08:40:55.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:54 smithi111 ceph-mon[35220]: pgmap v422: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-10T08:40:55.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:54 smithi089 ceph-mon[30757]: pgmap v422: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-10T08:40:55.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:54 smithi149 ceph-mon[36167]: pgmap v422: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-10T08:40:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.390+0000 7f1cc3e00640 1 -- 172.21.15.89:0/4143609016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cc4104d40 msgr2=0x7f1cc41051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.390+0000 7f1cc3e00640 1 --2- 172.21.15.89:0/4143609016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cc4104d40 0x7f1cc41051c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009900 tx=0x7f1cac02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.391+0000 7f1cc3e00640 1 -- 172.21.15.89:0/4143609016 shutdown_connections 2024-09-10T08:40:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.391+0000 7f1cc3e00640 1 --2- 172.21.15.89:0/4143609016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cc4105700 0x7f1cc410bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.391+0000 7f1cc3e00640 1 --2- 172.21.15.89:0/4143609016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cc4104d40 0x7f1cc41051c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.391+0000 7f1cc3e00640 1 --2- 172.21.15.89:0/4143609016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cc4103b50 0x7f1cc4103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.391+0000 7f1cc3e00640 1 -- 172.21.15.89:0/4143609016 >> 172.21.15.89:0/4143609016 conn(0x7f1cc40ff2b0 msgr2=0x7f1cc41016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.391+0000 7f1cc3e00640 1 -- 172.21.15.89:0/4143609016 shutdown_connections 2024-09-10T08:40:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.392+0000 7f1cc3e00640 1 -- 172.21.15.89:0/4143609016 wait complete. 2024-09-10T08:40:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.393+0000 7f1cc3e00640 1 Processor -- start 2024-09-10T08:40:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.393+0000 7f1cc3e00640 1 -- start start 2024-09-10T08:40:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.393+0000 7f1cc3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cc4103b50 0x7f1cc4199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.393+0000 7f1cc3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cc4104d40 0x7f1cc419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.393+0000 7f1cc3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cc4105700 0x7f1cc41a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.393+0000 7f1cc3e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc410f820 con 0x7f1cc4105700 2024-09-10T08:40:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.393+0000 7f1cc3e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc410f6a0 con 0x7f1cc4104d40 2024-09-10T08:40:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.393+0000 7f1cc3e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc410f9a0 con 0x7f1cc4103b50 2024-09-10T08:40:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.394+0000 7f1cc2a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cc4103b50 0x7f1cc4199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.394+0000 7f1cc2000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cc4104d40 0x7f1cc419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.394+0000 7f1cc3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cc4105700 0x7f1cc41a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.394+0000 7f1cc2a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cc4103b50 0x7f1cc4199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:55766/0 (socket says 172.21.15.89:55766) 2024-09-10T08:40:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.394+0000 7f1cc2a00640 1 -- 172.21.15.89:0/2830616718 learned_addr learned my addr 172.21.15.89:0/2830616718 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.394+0000 7f1cc2a00640 1 -- 172.21.15.89:0/2830616718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cc4104d40 msgr2=0x7f1cc419a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.394+0000 7f1cc2a00640 1 --2- 172.21.15.89:0/2830616718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cc4104d40 0x7f1cc419a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.394+0000 7f1cc2a00640 1 -- 172.21.15.89:0/2830616718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cc4105700 msgr2=0x7f1cc41a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.394+0000 7f1cc2a00640 1 --2- 172.21.15.89:0/2830616718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cc4105700 0x7f1cc41a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.394+0000 7f1cc2a00640 1 -- 172.21.15.89:0/2830616718 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc41a1ac0 con 0x7f1cc4103b50 2024-09-10T08:40:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.395+0000 7f1cc2a00640 1 --2- 172.21.15.89:0/2830616718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cc4103b50 0x7f1cc4199ea0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400e9c0 tx=0x7f1cb400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.395+0000 7f1cbb400640 1 -- 172.21.15.89:0/2830616718 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb400cde0 con 0x7f1cc4103b50 2024-09-10T08:40:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.395+0000 7f1cc3e00640 1 -- 172.21.15.89:0/2830616718 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc41a1db0 con 0x7f1cc4103b50 2024-09-10T08:40:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.395+0000 7f1cbb400640 1 -- 172.21.15.89:0/2830616718 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb4010910 con 0x7f1cc4103b50 2024-09-10T08:40:55.398 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.395+0000 7f1cbb400640 1 -- 172.21.15.89:0/2830616718 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4018a10 con 0x7f1cc4103b50 2024-09-10T08:40:55.398 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.395+0000 7f1cc3e00640 1 -- 172.21.15.89:0/2830616718 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc41a2220 con 0x7f1cc4103b50 2024-09-10T08:40:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.399+0000 7f1ccc834740 1 Processor -- start 2024-09-10T08:40:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.399+0000 7f1ccc834740 1 -- start start 2024-09-10T08:40:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.400+0000 7f1ccc834740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054260 0x5637c3054660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.400+0000 7f1cbb400640 1 -- 172.21.15.89:0/2830616718 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1cb401f020 con 0x7f1cc4103b50 2024-09-10T08:40:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.400+0000 7f1ccc834740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3054ba0 0x5637c3059660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.400+0000 7f1ccc834740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3059c30 0x5637c305c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.400+0000 7f1ccc834740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5637c2fb9100 con 0x5637c3054ba0 2024-09-10T08:40:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.400+0000 7f1ccc834740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5637c2f611b0 con 0x5637c3059c30 2024-09-10T08:40:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.400+0000 7f1ccc834740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5637c2fb50c0 con 0x5637c3054260 2024-09-10T08:40:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.400+0000 7f1cc2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3054ba0 0x5637c3059660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.400+0000 7f1cc2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3054ba0 0x5637c3059660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37604/0 (socket says 172.21.15.89:37604) 2024-09-10T08:40:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.400+0000 7f1cc2a00640 1 -- 172.21.15.89:0/21735712 learned_addr learned my addr 172.21.15.89:0/21735712 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.400+0000 7f1cc2000640 1 --2- 172.21.15.89:0/21735712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3059c30 0x5637c305c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.401+0000 7f1cc3400640 1 --2- 172.21.15.89:0/21735712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054260 0x5637c3054660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.401+0000 7f1cc2000640 1 -- 172.21.15.89:0/21735712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054260 msgr2=0x5637c3054660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.401+0000 7f1cc2000640 1 --2- 172.21.15.89:0/21735712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054260 0x5637c3054660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.401+0000 7f1cc2000640 1 -- 172.21.15.89:0/21735712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3054ba0 msgr2=0x5637c3059660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.401+0000 7f1cc2000640 1 --2- 172.21.15.89:0/21735712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3054ba0 0x5637c3059660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.401+0000 7f1cc2000640 1 -- 172.21.15.89:0/21735712 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637c2ccc210 con 0x5637c3059c30 2024-09-10T08:40:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.401+0000 7f1cc2000640 1 --2- 172.21.15.89:0/21735712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3059c30 0x5637c305c0f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1cac02a910 tx=0x7f1cac005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30d29466ded05027 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.402+0000 7f1cb3e00640 1 -- 172.21.15.89:0/21735712 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac035040 con 0x5637c3059c30 2024-09-10T08:40:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.402+0000 7f1cb3e00640 1 -- 172.21.15.89:0/21735712 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cac005520 con 0x5637c3059c30 2024-09-10T08:40:55.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.402+0000 7f1cb3e00640 1 -- 172.21.15.89:0/21735712 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac0316b0 con 0x5637c3059c30 2024-09-10T08:40:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.402+0000 7f1cbb400640 1 --2- 172.21.15.89:0/2830616718 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1ca80a2b80 0x7f1ca80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:55.405 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.403+0000 7f1cbb400640 1 -- 172.21.15.89:0/2830616718 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f1cb40cf620 con 0x7f1cc4103b50 2024-09-10T08:40:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.403+0000 7f1ccc834740 1 -- 172.21.15.89:0/21735712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3059c30 msgr2=0x5637c305c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.403+0000 7f1ccc834740 1 --2- 172.21.15.89:0/21735712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3059c30 0x5637c305c0f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1cac02a910 tx=0x7f1cac005c00 comp rx=0 tx=0).stop 2024-09-10T08:40:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.403+0000 7f1cc3400640 1 --2- 172.21.15.89:0/2830616718 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1ca80a2b80 0x7f1ca80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.403+0000 7f1ccc834740 1 -- 172.21.15.89:0/21735712 shutdown_connections 2024-09-10T08:40:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.403+0000 7f1ccc834740 1 --2- 172.21.15.89:0/21735712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3059c30 0x5637c305c0f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.403+0000 7f1ccc834740 1 --2- 172.21.15.89:0/21735712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3054ba0 0x5637c3059660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.403+0000 7f1ccc834740 1 --2- 172.21.15.89:0/21735712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054260 0x5637c3054660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.403+0000 7f1ccc834740 1 -- 172.21.15.89:0/21735712 >> 172.21.15.89:0/21735712 conn(0x5637c304f8d0 msgr2=0x5637c304fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.403+0000 7f1ccc834740 1 -- 172.21.15.89:0/21735712 shutdown_connections 2024-09-10T08:40:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.404+0000 7f1ccc834740 1 -- 172.21.15.89:0/21735712 wait complete. 2024-09-10T08:40:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.404+0000 7f1cc3400640 1 --2- 172.21.15.89:0/2830616718 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1ca80a2b80 0x7f1ca80a5040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc005fd0 tx=0x7f1cbc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:55.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.404+0000 7f1ccc834740 1 -- start start 2024-09-10T08:40:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.405+0000 7f1ccc834740 1 Processor -- start 2024-09-10T08:40:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.405+0000 7f1ccc834740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3054260 0x5637c305e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.405+0000 7f1ccc834740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054ba0 0x5637c305e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.405+0000 7f1ccc834740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3059c30 0x5637c3065990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.405+0000 7f1ccc834740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5637c305cba0 con 0x5637c3054260 2024-09-10T08:40:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.405+0000 7f1ccc834740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5637c2f42090 con 0x5637c3059c30 2024-09-10T08:40:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.405+0000 7f1ccc834740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5637c305cd20 con 0x5637c3054ba0 2024-09-10T08:40:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.405+0000 7f1ccc834740 10 client.0 _ll_register_callbacks cb 0x5637c2f96f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.405+0000 7f1cc2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3054260 0x5637c305e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.406+0000 7f1cc2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3054260 0x5637c305e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37616/0 (socket says 172.21.15.89:37616) 2024-09-10T08:40:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.406+0000 7f1cc2a00640 1 -- 172.21.15.89:0/2486115638 learned_addr learned my addr 172.21.15.89:0/2486115638 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.406+0000 7f1cc2000640 1 --2- 172.21.15.89:0/2486115638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054ba0 0x5637c305e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.406+0000 7f1cc3400640 1 --2- 172.21.15.89:0/2486115638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3059c30 0x5637c3065990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.406+0000 7f1cc2a00640 1 -- 172.21.15.89:0/2486115638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054ba0 msgr2=0x5637c305e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.406+0000 7f1cc2a00640 1 --2- 172.21.15.89:0/2486115638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054ba0 0x5637c305e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.406+0000 7f1cc2a00640 1 -- 172.21.15.89:0/2486115638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3059c30 msgr2=0x5637c3065990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.406+0000 7f1cc2a00640 1 --2- 172.21.15.89:0/2486115638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3059c30 0x5637c3065990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.406+0000 7f1cc2a00640 1 -- 172.21.15.89:0/2486115638 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637c2d98620 con 0x5637c3054260 2024-09-10T08:40:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.407+0000 7f1cc2a00640 1 --2- 172.21.15.89:0/2486115638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3054260 0x5637c305e410 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40d52f0 tx=0x7f1cb409b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58fedc2ea4591479 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.407+0000 7f1cb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:40:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.407+0000 7f1cb0c00640 1 -- 172.21.15.89:0/2486115638 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb409c1b0 con 0x5637c3054260 2024-09-10T08:40:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.407+0000 7f1cb0c00640 1 -- 172.21.15.89:0/2486115638 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f1cb40cf300 con 0x5637c3054260 2024-09-10T08:40:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.407+0000 7f1ccc834740 1 -- 172.21.15.89:0/2486115638 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cac005520 con 0x5637c3054260 2024-09-10T08:40:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.407+0000 7f1cb0c00640 1 -- 172.21.15.89:0/2486115638 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1cb409bab0 con 0x5637c3054260 2024-09-10T08:40:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.407+0000 7f1ccc834740 10 client.15525 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.407+0000 7f1ccc834740 1 -- 172.21.15.89:0/2486115638 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5637c30710b0 con 0x5637c3054260 2024-09-10T08:40:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.408+0000 7f1cb0c00640 1 -- 172.21.15.89:0/2486115638 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cac005520 con 0x5637c3054260 2024-09-10T08:40:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.409+0000 7f1cb0c00640 1 -- 172.21.15.89:0/2486115638 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1cb4010ac0 con 0x5637c3054260 2024-09-10T08:40:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.409+0000 7f1ccc834740 10 client.15525 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.409+0000 7f1ccc834740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:40:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.409+0000 7f1ccc834740 20 client.15525 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.409+0000 7f1ccc834740 1 --2- 172.21.15.89:0/2486115638 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5637c30728e0 0x5637c3074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.409+0000 7f1ccc834740 4 client.15525 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:40:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.409+0000 7f1ccc834740 1 -- 172.21.15.89:0/2486115638 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637c2fc6830 con 0x5637c30728e0 2024-09-10T08:40:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.409+0000 7f1cc2000640 1 --2- 172.21.15.89:0/2486115638 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5637c30728e0 0x5637c3074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.410+0000 7f1cc2000640 1 --2- 172.21.15.89:0/2486115638 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5637c30728e0 0x5637c3074cd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a53bf70172d5baf server_cookie=d28975b5b2fb4f69 in_seq=0 out_seq=0 2024-09-10T08:40:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.410+0000 7f1cb0c00640 10 client.15525 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.410+0000 7f1cb0c00640 1 -- 172.21.15.89:0/2486115638 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5637c2fc6830 con 0x5637c30728e0 2024-09-10T08:40:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.410+0000 7f1cb0c00640 10 client.15525 handle_command_reply: tid=0 2024-09-10T08:40:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.411+0000 7f1ccc834740 1 client.15525 shutdown 2024-09-10T08:40:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.411+0000 7f1ccc834740 1 -- 172.21.15.89:0/2486115638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3054260 msgr2=0x5637c305e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.411+0000 7f1ccc834740 1 --2- 172.21.15.89:0/2486115638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3054260 0x5637c305e410 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40d52f0 tx=0x7f1cb409b6b0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.411+0000 7f1ccc834740 1 -- 172.21.15.89:0/2486115638 shutdown_connections 2024-09-10T08:40:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.411+0000 7f1ccc834740 1 --2- 172.21.15.89:0/2486115638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3059c30 0x5637c3065990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.411+0000 7f1ccc834740 1 --2- 172.21.15.89:0/2486115638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054ba0 0x5637c305e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.411+0000 7f1ccc834740 1 --2- 172.21.15.89:0/2486115638 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5637c30728e0 0x5637c3074cd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.411+0000 7f1ccc834740 1 --2- 172.21.15.89:0/2486115638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3054260 0x5637c305e410 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.411+0000 7f1ccc834740 1 -- 172.21.15.89:0/2486115638 >> 172.21.15.89:0/2486115638 conn(0x5637c304f8d0 msgr2=0x5637c30510e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.411+0000 7f1cb0c00640 0 client.15525 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.411+0000 7f1ccc834740 1 -- 172.21.15.89:0/2486115638 shutdown_connections 2024-09-10T08:40:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.411+0000 7f1ccc834740 1 -- 172.21.15.89:0/2486115638 wait complete. 2024-09-10T08:40:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.411+0000 7f1ccc834740 20 client.15525 trim_cache size 0 max 16384 2024-09-10T08:40:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.426+0000 7f1ccc834740 1 Processor -- start 2024-09-10T08:40:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.426+0000 7f1ccc834740 1 -- start start 2024-09-10T08:40:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.427+0000 7f1ccc834740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3054260 0x5637c304ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.427+0000 7f1ccc834740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054ba0 0x5637c3065950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.427+0000 7f1ccc834740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3059c30 0x5637c3079490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.427+0000 7f1ccc834740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41aad20 con 0x5637c3054260 2024-09-10T08:40:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.427+0000 7f1ccc834740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41a85f0 con 0x5637c3059c30 2024-09-10T08:40:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.427+0000 7f1ccc834740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41a6460 con 0x5637c3054ba0 2024-09-10T08:40:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.427+0000 7f1cc2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3054260 0x5637c304ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.427+0000 7f1cc2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3054260 0x5637c304ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37630/0 (socket says 172.21.15.89:37630) 2024-09-10T08:40:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.427+0000 7f1cc2a00640 1 -- 172.21.15.89:0/2954644018 learned_addr learned my addr 172.21.15.89:0/2954644018 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.427+0000 7f1cc2000640 1 --2- 172.21.15.89:0/2954644018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054ba0 0x5637c3065950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.427+0000 7f1cc3400640 1 --2- 172.21.15.89:0/2954644018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3059c30 0x5637c3079490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.428+0000 7f1cc2a00640 1 -- 172.21.15.89:0/2954644018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054ba0 msgr2=0x5637c3065950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.428+0000 7f1cc2a00640 1 --2- 172.21.15.89:0/2954644018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054ba0 0x5637c3065950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.428+0000 7f1cc2a00640 1 -- 172.21.15.89:0/2954644018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3059c30 msgr2=0x5637c3079490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.428+0000 7f1cc2a00640 1 --2- 172.21.15.89:0/2954644018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3059c30 0x5637c3079490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.428+0000 7f1cc2a00640 1 -- 172.21.15.89:0/2954644018 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637c3055080 con 0x5637c3054260 2024-09-10T08:40:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.428+0000 7f1cc2000640 1 --2- 172.21.15.89:0/2954644018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054ba0 0x5637c3065950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:40:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.428+0000 7f1cc2a00640 1 --2- 172.21.15.89:0/2954644018 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3054260 0x5637c304ea10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40a42e0 tx=0x7f1cb4099fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdbac7060c12f1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.428+0000 7f1cb3400640 1 -- 172.21.15.89:0/2954644018 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb409fe60 con 0x5637c3054260 2024-09-10T08:40:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.428+0000 7f1cb3400640 1 -- 172.21.15.89:0/2954644018 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb40a4720 con 0x5637c3054260 2024-09-10T08:40:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.428+0000 7f1cb3400640 1 -- 172.21.15.89:0/2954644018 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb409af00 con 0x5637c3054260 2024-09-10T08:40:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.429+0000 7f1ccc834740 1 -- 172.21.15.89:0/2954644018 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3054260 msgr2=0x5637c304ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.429+0000 7f1ccc834740 1 --2- 172.21.15.89:0/2954644018 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3054260 0x5637c304ea10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40a42e0 tx=0x7f1cb4099fc0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.430+0000 7f1ccc834740 1 -- 172.21.15.89:0/2954644018 shutdown_connections 2024-09-10T08:40:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.430+0000 7f1ccc834740 1 --2- 172.21.15.89:0/2954644018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3059c30 0x5637c3079490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.430+0000 7f1ccc834740 1 --2- 172.21.15.89:0/2954644018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054ba0 0x5637c3065950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.430+0000 7f1ccc834740 1 --2- 172.21.15.89:0/2954644018 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3054260 0x5637c304ea10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.430+0000 7f1ccc834740 1 -- 172.21.15.89:0/2954644018 >> 172.21.15.89:0/2954644018 conn(0x5637c304f8d0 msgr2=0x5637c305e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.430+0000 7f1ccc834740 1 -- 172.21.15.89:0/2954644018 shutdown_connections 2024-09-10T08:40:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.430+0000 7f1ccc834740 1 -- 172.21.15.89:0/2954644018 wait complete. 2024-09-10T08:40:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.430+0000 7f1ccc834740 1 -- start start 2024-09-10T08:40:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.431+0000 7f1ccc834740 1 Processor -- start 2024-09-10T08:40:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.431+0000 7f1ccc834740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3054260 0x5637c3063300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.431+0000 7f1ccc834740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054ba0 0x5637c3063840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.431+0000 7f1ccc834740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3059c30 0x5637c3064850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.431+0000 7f1ccc834740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41a6ab0 con 0x5637c3059c30 2024-09-10T08:40:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.431+0000 7f1ccc834740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41a7f20 con 0x5637c3054260 2024-09-10T08:40:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.431+0000 7f1ccc834740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41a7180 con 0x5637c3054ba0 2024-09-10T08:40:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.431+0000 7f1ccc834740 10 client.0 _ll_register_callbacks cb 0x7f1cb4097d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:40:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.431+0000 7f1cc3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3059c30 0x5637c3064850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.431+0000 7f1cc3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3059c30 0x5637c3064850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37644/0 (socket says 172.21.15.89:37644) 2024-09-10T08:40:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.431+0000 7f1cc3400640 1 -- 172.21.15.89:0/3426210601 learned_addr learned my addr 172.21.15.89:0/3426210601 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.432+0000 7f1cc2a00640 1 --2- 172.21.15.89:0/3426210601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3054260 0x5637c3063300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.432+0000 7f1cc2000640 1 --2- 172.21.15.89:0/3426210601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054ba0 0x5637c3063840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.432+0000 7f1cc2a00640 1 -- 172.21.15.89:0/3426210601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054ba0 msgr2=0x5637c3063840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.432+0000 7f1cc2a00640 1 --2- 172.21.15.89:0/3426210601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054ba0 0x5637c3063840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.432+0000 7f1cc2a00640 1 -- 172.21.15.89:0/3426210601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3059c30 msgr2=0x5637c3064850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:40:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.432+0000 7f1cc2a00640 1 --2- 172.21.15.89:0/3426210601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3059c30 0x5637c3064850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.432+0000 7f1cc2a00640 1 -- 172.21.15.89:0/3426210601 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637c3064e60 con 0x5637c3054260 2024-09-10T08:40:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.432+0000 7f1cc2a00640 1 --2- 172.21.15.89:0/3426210601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3054260 0x5637c3063300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400c6a0 tx=0x7f1cb4097e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c31bf508347b44ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.432+0000 7f1cb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:40:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.432+0000 7f1cb0c00640 1 -- 172.21.15.89:0/3426210601 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb409c1b0 con 0x5637c3054260 2024-09-10T08:40:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.433+0000 7f1cb0c00640 1 -- 172.21.15.89:0/3426210601 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f1cb409af00 con 0x5637c3054260 2024-09-10T08:40:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.433+0000 7f1ccc834740 1 -- 172.21.15.89:0/3426210601 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb40a4720 con 0x5637c3054260 2024-09-10T08:40:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.433+0000 7f1cb0c00640 1 -- 172.21.15.89:0/3426210601 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1cb409fe60 con 0x5637c3054260 2024-09-10T08:40:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.433+0000 7f1ccc834740 10 client.25592 fetch_fsmap learned FSMap version 26 2024-09-10T08:40:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.433+0000 7f1ccc834740 1 -- 172.21.15.89:0/3426210601 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5637c30dd700 con 0x5637c3054260 2024-09-10T08:40:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.433+0000 7f1cb0c00640 1 -- 172.21.15.89:0/3426210601 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb40a4720 con 0x5637c3054260 2024-09-10T08:40:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.433+0000 7f1cb0c00640 1 -- 172.21.15.89:0/3426210601 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1cb409a970 con 0x5637c3054260 2024-09-10T08:40:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.433+0000 7f1ccc834740 10 client.25592 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:40:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.433+0000 7f1ccc834740 10 client.25592 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:40:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.433+0000 7f1ccc834740 20 client.25592 populate_metadata read hostname 'smithi089' 2024-09-10T08:40:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.433+0000 7f1ccc834740 1 --2- 172.21.15.89:0/3426210601 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5637c30ddff0 0x5637c30e0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.433+0000 7f1ccc834740 4 client.25592 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:40:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.433+0000 7f1ccc834740 1 -- 172.21.15.89:0/3426210601 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5637c2fc0b80 con 0x5637c30ddff0 2024-09-10T08:40:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.434+0000 7f1cc2000640 1 --2- 172.21.15.89:0/3426210601 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5637c30ddff0 0x5637c30e0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.434+0000 7f1cc2000640 1 --2- 172.21.15.89:0/3426210601 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5637c30ddff0 0x5637c30e0410 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc6c15f9467fd516 server_cookie=904ce3710ea0bcd in_seq=0 out_seq=0 2024-09-10T08:40:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.434+0000 7f1cb0c00640 10 client.25592 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.438+0000 7f1cb0c00640 1 -- 172.21.15.89:0/3426210601 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5637c2fc0b80 con 0x5637c30ddff0 2024-09-10T08:40:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.438+0000 7f1cb0c00640 10 client.25592 handle_command_reply: tid=0 2024-09-10T08:40:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.438+0000 7f1ccc834740 1 client.25592 shutdown 2024-09-10T08:40:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.438+0000 7f1ccc834740 1 -- 172.21.15.89:0/3426210601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3054260 msgr2=0x5637c3063300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.438+0000 7f1ccc834740 1 --2- 172.21.15.89:0/3426210601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3054260 0x5637c3063300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400c6a0 tx=0x7f1cb4097e00 comp rx=0 tx=0).stop 2024-09-10T08:40:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.439+0000 7f1ccc834740 1 -- 172.21.15.89:0/3426210601 shutdown_connections 2024-09-10T08:40:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.439+0000 7f1ccc834740 1 --2- 172.21.15.89:0/3426210601 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5637c30ddff0 0x5637c30e0410 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.439+0000 7f1ccc834740 1 --2- 172.21.15.89:0/3426210601 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5637c3059c30 0x5637c3064850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.439+0000 7f1ccc834740 1 --2- 172.21.15.89:0/3426210601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637c3054ba0 0x5637c3063840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.439+0000 7f1ccc834740 1 --2- 172.21.15.89:0/3426210601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5637c3054260 0x5637c3063300 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.439+0000 7f1ccc834740 1 -- 172.21.15.89:0/3426210601 >> 172.21.15.89:0/3426210601 conn(0x5637c304f8d0 msgr2=0x5637c3050fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.439+0000 7f1cb0c00640 0 client.25592 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:40:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.439+0000 7f1ccc834740 1 -- 172.21.15.89:0/3426210601 shutdown_connections 2024-09-10T08:40:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.439+0000 7f1ccc834740 1 -- 172.21.15.89:0/3426210601 wait complete. 2024-09-10T08:40:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.439+0000 7f1ccc834740 20 client.25592 trim_cache size 0 max 16384 2024-09-10T08:40:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:40:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:40:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:40:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:40:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.440+0000 7f1cc3e00640 1 -- 172.21.15.89:0/2830616718 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1ca80a2b80 msgr2=0x7f1ca80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.440+0000 7f1cc3e00640 1 --2- 172.21.15.89:0/2830616718 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1ca80a2b80 0x7f1ca80a5040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc005fd0 tx=0x7f1cbc009500 comp rx=0 tx=0).stop 2024-09-10T08:40:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.440+0000 7f1cc3e00640 1 -- 172.21.15.89:0/2830616718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cc4103b50 msgr2=0x7f1cc4199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.440+0000 7f1cc3e00640 1 --2- 172.21.15.89:0/2830616718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cc4103b50 0x7f1cc4199ea0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400e9c0 tx=0x7f1cb400ee80 comp rx=0 tx=0).stop 2024-09-10T08:40:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.440+0000 7f1cc3e00640 1 -- 172.21.15.89:0/2830616718 shutdown_connections 2024-09-10T08:40:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.440+0000 7f1cc3e00640 1 --2- 172.21.15.89:0/2830616718 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1ca80a2b80 0x7f1ca80a5040 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.440+0000 7f1cc3e00640 1 --2- 172.21.15.89:0/2830616718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1cc4105700 0x7f1cc41a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.440+0000 7f1cc3e00640 1 --2- 172.21.15.89:0/2830616718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cc4104d40 0x7f1cc419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.440+0000 7f1cc3e00640 1 --2- 172.21.15.89:0/2830616718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cc4103b50 0x7f1cc4199ea0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.440+0000 7f1cc3e00640 1 -- 172.21.15.89:0/2830616718 >> 172.21.15.89:0/2830616718 conn(0x7f1cc40ff2b0 msgr2=0x7f1cc41080e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.440+0000 7f1cc3e00640 1 -- 172.21.15.89:0/2830616718 shutdown_connections 2024-09-10T08:40:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:40:55.440+0000 7f1cc3e00640 1 -- 172.21.15.89:0/2830616718 wait complete. 2024-09-10T08:40:57.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:56 smithi111 ceph-mon[35220]: pgmap v423: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-10T08:40:57.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:56 smithi089 ceph-mon[30757]: pgmap v423: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-10T08:40:57.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:56 smithi149 ceph-mon[36167]: pgmap v423: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-10T08:40:58.345 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:40:58.889 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:40:59.034 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:40:58 smithi089 ceph-mon[30757]: pgmap v424: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-10T08:40:59.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:40:58 smithi111 ceph-mon[35220]: pgmap v424: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-10T08:40:59.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:40:58 smithi149 ceph-mon[36167]: pgmap v424: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-10T08:40:59.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.587+0000 7f85c7e00640 1 -- 172.21.15.89:0/1424260195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85c8074510 msgr2=0x7f85c80748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:59.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.587+0000 7f85c7e00640 1 --2- 172.21.15.89:0/1424260195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85c8074510 0x7f85c80748f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009900 tx=0x7f85b802a480 comp rx=0 tx=0).stop 2024-09-10T08:40:59.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.587+0000 7f85c7e00640 1 -- 172.21.15.89:0/1424260195 shutdown_connections 2024-09-10T08:40:59.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.587+0000 7f85c7e00640 1 --2- 172.21.15.89:0/1424260195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85c810e280 0x7f85c8110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:59.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.587+0000 7f85c7e00640 1 --2- 172.21.15.89:0/1424260195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85c8074e30 0x7f85c810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:59.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.587+0000 7f85c7e00640 1 --2- 172.21.15.89:0/1424260195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85c8074510 0x7f85c80748f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:59.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.587+0000 7f85c7e00640 1 -- 172.21.15.89:0/1424260195 >> 172.21.15.89:0/1424260195 conn(0x7f85c80fd740 msgr2=0x7f85c80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:59.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.587+0000 7f85c7e00640 1 -- 172.21.15.89:0/1424260195 shutdown_connections 2024-09-10T08:40:59.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.587+0000 7f85c7e00640 1 -- 172.21.15.89:0/1424260195 wait complete. 2024-09-10T08:40:59.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.589+0000 7f85c7e00640 1 Processor -- start 2024-09-10T08:40:59.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.589+0000 7f85c7e00640 1 -- start start 2024-09-10T08:40:59.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.589+0000 7f85c7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85c8074510 0x7f85c819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:59.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.589+0000 7f85c7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85c8074e30 0x7f85c819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:59.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.589+0000 7f85c7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85c810e280 0x7f85c819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:59.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.589+0000 7f85c7e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f85c8113fb0 con 0x7f85c8074e30 2024-09-10T08:40:59.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.589+0000 7f85c7e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f85c8113e30 con 0x7f85c8074510 2024-09-10T08:40:59.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.589+0000 7f85c7e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f85c8114130 con 0x7f85c810e280 2024-09-10T08:40:59.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.590+0000 7f85c6000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85c8074e30 0x7f85c819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:59.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.590+0000 7f85c7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85c810e280 0x7f85c819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:59.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.590+0000 7f85c6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85c8074510 0x7f85c819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:59.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.590+0000 7f85c7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85c810e280 0x7f85c819edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:55820/0 (socket says 172.21.15.89:55820) 2024-09-10T08:40:59.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.590+0000 7f85c7400640 1 -- 172.21.15.89:0/426181069 learned_addr learned my addr 172.21.15.89:0/426181069 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:40:59.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.590+0000 7f85c7400640 1 -- 172.21.15.89:0/426181069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85c8074510 msgr2=0x7f85c819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:59.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.590+0000 7f85c7400640 1 --2- 172.21.15.89:0/426181069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85c8074510 0x7f85c819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:59.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.590+0000 7f85c7400640 1 -- 172.21.15.89:0/426181069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85c8074e30 msgr2=0x7f85c819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:59.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.590+0000 7f85c7400640 1 --2- 172.21.15.89:0/426181069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85c8074e30 0x7f85c819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:59.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.590+0000 7f85c7400640 1 -- 172.21.15.89:0/426181069 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c819f4a0 con 0x7f85c810e280 2024-09-10T08:40:59.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.591+0000 7f85c7400640 1 --2- 172.21.15.89:0/426181069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85c810e280 0x7f85c819edc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f85c000eb40 tx=0x7f85c000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:59.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.591+0000 7f85bf400640 1 -- 172.21.15.89:0/426181069 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0004b20 con 0x7f85c810e280 2024-09-10T08:40:59.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.591+0000 7f85bf400640 1 -- 172.21.15.89:0/426181069 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c0004cc0 con 0x7f85c810e280 2024-09-10T08:40:59.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.591+0000 7f85c7e00640 1 -- 172.21.15.89:0/426181069 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c819f790 con 0x7f85c810e280 2024-09-10T08:40:59.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.591+0000 7f85bf400640 1 -- 172.21.15.89:0/426181069 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0010850 con 0x7f85c810e280 2024-09-10T08:40:59.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.591+0000 7f85c7e00640 1 -- 172.21.15.89:0/426181069 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c81a7070 con 0x7f85c810e280 2024-09-10T08:40:59.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.593+0000 7f85c7e00640 1 -- 172.21.15.89:0/426181069 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85c80748f0 con 0x7f85c810e280 2024-09-10T08:40:59.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.595+0000 7f85bf400640 1 -- 172.21.15.89:0/426181069 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f85c0005020 con 0x7f85c810e280 2024-09-10T08:40:59.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.597+0000 7f85bf400640 1 --2- 172.21.15.89:0/426181069 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f85ac0a2c00 0x7f85ac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:59.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.597+0000 7f85bf400640 1 -- 172.21.15.89:0/426181069 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f85c00cfc00 con 0x7f85c810e280 2024-09-10T08:40:59.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.598+0000 7f85c6a00640 1 --2- 172.21.15.89:0/426181069 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f85ac0a2c00 0x7f85ac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:59.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.598+0000 7f85c6a00640 1 --2- 172.21.15.89:0/426181069 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f85ac0a2c00 0x7f85ac0a50c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f85b8005ab0 tx=0x7f85b8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:59.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.600+0000 7f85bf400640 1 -- 172.21.15.89:0/426181069 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85c009c730 con 0x7f85c810e280 2024-09-10T08:40:59.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.938+0000 7f85c7e00640 1 -- 172.21.15.89:0/426181069 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85c802d7c0 con 0x7f85c810e280 2024-09-10T08:40:59.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.939+0000 7f85bf400640 1 -- 172.21.15.89:0/426181069 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11918 (secure 0 0 0) 0x7f85c009c030 con 0x7f85c810e280 2024-09-10T08:40:59.941 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:40:59.941 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-10T08:39:39:721431+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:39:38.834417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24518,24496]},"id":1}]} 2024-09-10T08:40:59.941 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-10T08:40:59.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.943+0000 7f85c7e00640 1 -- 172.21.15.89:0/426181069 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f85ac0a2c00 msgr2=0x7f85ac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:59.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.943+0000 7f85c7e00640 1 --2- 172.21.15.89:0/426181069 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f85ac0a2c00 0x7f85ac0a50c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f85b8005ab0 tx=0x7f85b8005a20 comp rx=0 tx=0).stop 2024-09-10T08:40:59.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.943+0000 7f85c7e00640 1 -- 172.21.15.89:0/426181069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85c810e280 msgr2=0x7f85c819edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:59.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.943+0000 7f85c7e00640 1 --2- 172.21.15.89:0/426181069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85c810e280 0x7f85c819edc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f85c000eb40 tx=0x7f85c000c2d0 comp rx=0 tx=0).stop 2024-09-10T08:40:59.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.943+0000 7f85c7e00640 1 -- 172.21.15.89:0/426181069 shutdown_connections 2024-09-10T08:40:59.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.943+0000 7f85c7e00640 1 --2- 172.21.15.89:0/426181069 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f85ac0a2c00 0x7f85ac0a50c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:59.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.943+0000 7f85c7e00640 1 --2- 172.21.15.89:0/426181069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85c810e280 0x7f85c819edc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:59.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.943+0000 7f85c7e00640 1 --2- 172.21.15.89:0/426181069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85c8074e30 0x7f85c819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:59.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.943+0000 7f85c7e00640 1 --2- 172.21.15.89:0/426181069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85c8074510 0x7f85c819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:59.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.943+0000 7f85c7e00640 1 -- 172.21.15.89:0/426181069 >> 172.21.15.89:0/426181069 conn(0x7f85c80fd740 msgr2=0x7f85c80fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:59.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.944+0000 7f85c7e00640 1 -- 172.21.15.89:0/426181069 shutdown_connections 2024-09-10T08:40:59.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:40:59.944+0000 7f85c7e00640 1 -- 172.21.15.89:0/426181069 wait complete. 2024-09-10T08:41:00.818 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24554 quiesce db --set-id 9590b514 --release --await' 2024-09-10T08:41:00.819 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24554 quiesce db --set-id 9590b514 --release --await 2024-09-10T08:41:01.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:00 smithi111 ceph-mon[35220]: pgmap v425: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-10T08:41:01.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:00 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/426181069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:41:01.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:00 smithi089 ceph-mon[30757]: pgmap v425: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-10T08:41:01.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:00 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/426181069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:41:01.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:00 smithi149 ceph-mon[36167]: pgmap v425: 129 pgs: 129 active+clean; 671 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-10T08:41:01.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:00 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/426181069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:41:01.233 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:41:01.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.941+0000 7fef31a00640 1 -- 172.21.15.89:0/667965030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef2c1047e0 msgr2=0x7fef2c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:01.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.941+0000 7fef31a00640 1 --2- 172.21.15.89:0/667965030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef2c1047e0 0x7fef2c108830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fef18009900 tx=0x7fef1802a410 comp rx=0 tx=0).stop 2024-09-10T08:41:01.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.942+0000 7fef31a00640 1 -- 172.21.15.89:0/667965030 shutdown_connections 2024-09-10T08:41:01.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.942+0000 7fef31a00640 1 --2- 172.21.15.89:0/667965030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef2c108f60 0x7fef2c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:01.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.942+0000 7fef31a00640 1 --2- 172.21.15.89:0/667965030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef2c1047e0 0x7fef2c108830 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:01.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.942+0000 7fef31a00640 1 --2- 172.21.15.89:0/667965030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef2c103e30 0x7fef2c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:01.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.942+0000 7fef31a00640 1 -- 172.21.15.89:0/667965030 >> 172.21.15.89:0/667965030 conn(0x7fef2c0ff910 msgr2=0x7fef2c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:01.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.942+0000 7fef31a00640 1 -- 172.21.15.89:0/667965030 shutdown_connections 2024-09-10T08:41:01.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.942+0000 7fef31a00640 1 -- 172.21.15.89:0/667965030 wait complete. 2024-09-10T08:41:01.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.943+0000 7fef31a00640 1 Processor -- start 2024-09-10T08:41:01.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.944+0000 7fef31a00640 1 -- start start 2024-09-10T08:41:01.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.944+0000 7fef31a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef2c103e30 0x7fef2c1109c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:01.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.944+0000 7fef31a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef2c1047e0 0x7fef2c10da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:01.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.944+0000 7fef31a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef2c108f60 0x7fef2c10dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:01.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.944+0000 7fef31a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c114010 con 0x7fef2c108f60 2024-09-10T08:41:01.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.944+0000 7fef31a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c113e90 con 0x7fef2c1047e0 2024-09-10T08:41:01.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.944+0000 7fef31a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c114190 con 0x7fef2c103e30 2024-09-10T08:41:01.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.944+0000 7fef2be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef2c103e30 0x7fef2c1109c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:01.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.944+0000 7fef2be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef2c103e30 0x7fef2c1109c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:55832/0 (socket says 172.21.15.89:55832) 2024-09-10T08:41:01.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.944+0000 7fef2be00640 1 -- 172.21.15.89:0/4048642009 learned_addr learned my addr 172.21.15.89:0/4048642009 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:01.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.945+0000 7fef31000640 1 --2- 172.21.15.89:0/4048642009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef2c108f60 0x7fef2c10dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:01.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.945+0000 7fef2be00640 1 -- 172.21.15.89:0/4048642009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef2c1047e0 msgr2=0x7fef2c10da70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T08:41:01.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.945+0000 7fef2be00640 1 --2- 172.21.15.89:0/4048642009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef2c1047e0 0x7fef2c10da70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:01.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.945+0000 7fef2be00640 1 -- 172.21.15.89:0/4048642009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef2c108f60 msgr2=0x7fef2c10dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:01.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.945+0000 7fef2be00640 1 --2- 172.21.15.89:0/4048642009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef2c108f60 0x7fef2c10dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:01.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.945+0000 7fef2be00640 1 -- 172.21.15.89:0/4048642009 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef2c10e8b0 con 0x7fef2c103e30 2024-09-10T08:41:01.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.945+0000 7fef2be00640 1 --2- 172.21.15.89:0/4048642009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef2c103e30 0x7fef2c1109c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00ea60 tx=0x7fef1c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:01.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.946+0000 7fef28c00640 1 -- 172.21.15.89:0/4048642009 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c00cde0 con 0x7fef2c103e30 2024-09-10T08:41:01.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.946+0000 7fef28c00640 1 -- 172.21.15.89:0/4048642009 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef1c0035f0 con 0x7fef2c103e30 2024-09-10T08:41:01.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.946+0000 7fef28c00640 1 -- 172.21.15.89:0/4048642009 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c010690 con 0x7fef2c103e30 2024-09-10T08:41:01.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.946+0000 7fef31a00640 1 -- 172.21.15.89:0/4048642009 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef2c1aae10 con 0x7fef2c103e30 2024-09-10T08:41:01.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.946+0000 7fef31a00640 1 -- 172.21.15.89:0/4048642009 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef2c1ab2d0 con 0x7fef2c103e30 2024-09-10T08:41:01.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.950+0000 7fef359ce740 1 Processor -- start 2024-09-10T08:41:01.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.951+0000 7fef359ce740 1 -- start start 2024-09-10T08:41:01.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.951+0000 7fef359ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db59520 0x55f94db59900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:01.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.951+0000 7fef359ce740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f94db59e40 0x55f94db641c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:01.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.951+0000 7fef359ce740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db64700 0x55f94db66af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:01.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.951+0000 7fef359ce740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f94dabf650 con 0x55f94db59e40 2024-09-10T08:41:01.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.951+0000 7fef359ce740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f94da66f10 con 0x55f94db64700 2024-09-10T08:41:01.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.951+0000 7fef359ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f94dabb610 con 0x55f94db59520 2024-09-10T08:41:01.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.952+0000 7fef2b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db64700 0x55f94db66af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:01.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.952+0000 7fef2b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db64700 0x55f94db66af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:45902/0 (socket says 172.21.15.89:45902) 2024-09-10T08:41:01.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.952+0000 7fef2b400640 1 -- 172.21.15.89:0/2085959149 learned_addr learned my addr 172.21.15.89:0/2085959149 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:01.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.952+0000 7fef31000640 1 --2- 172.21.15.89:0/2085959149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db59520 0x55f94db59900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:01.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.952+0000 7fef28c00640 1 -- 172.21.15.89:0/4048642009 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fef1c0037a0 con 0x7fef2c103e30 2024-09-10T08:41:01.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.952+0000 7fef2b400640 1 -- 172.21.15.89:0/2085959149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db59520 msgr2=0x55f94db59900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:01.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.952+0000 7fef2b400640 1 --2- 172.21.15.89:0/2085959149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db59520 0x55f94db59900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:01.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.952+0000 7fef2b400640 1 -- 172.21.15.89:0/2085959149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f94db59e40 msgr2=0x55f94db641c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:01.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.952+0000 7fef2b400640 1 --2- 172.21.15.89:0/2085959149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f94db59e40 0x55f94db641c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:01.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.952+0000 7fef2b400640 1 -- 172.21.15.89:0/2085959149 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f94d7ce580 con 0x55f94db64700 2024-09-10T08:41:01.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.953+0000 7fef2b400640 1 --2- 172.21.15.89:0/2085959149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db64700 0x55f94db66af0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fef1802a930 tx=0x7fef18009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eea4315b00bb9c3a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:01.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.953+0000 7fef31000640 1 --2- 172.21.15.89:0/2085959149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db59520 0x55f94db59900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:41:01.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.954+0000 7fef21600640 1 -- 172.21.15.89:0/2085959149 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1803e030 con 0x55f94db64700 2024-09-10T08:41:01.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.954+0000 7fef21600640 1 -- 172.21.15.89:0/2085959149 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef180035f0 con 0x55f94db64700 2024-09-10T08:41:01.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.954+0000 7fef21600640 1 -- 172.21.15.89:0/2085959149 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18035040 con 0x55f94db64700 2024-09-10T08:41:01.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.954+0000 7fef28c00640 1 --2- 172.21.15.89:0/4048642009 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fef100a2c80 0x7fef100a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:01.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.954+0000 7fef28c00640 1 -- 172.21.15.89:0/4048642009 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fef1c0cf730 con 0x7fef2c103e30 2024-09-10T08:41:01.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.955+0000 7fef31000640 1 --2- 172.21.15.89:0/4048642009 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fef100a2c80 0x7fef100a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:01.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.955+0000 7fef359ce740 1 -- 172.21.15.89:0/2085959149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db64700 msgr2=0x55f94db66af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:01.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.955+0000 7fef359ce740 1 --2- 172.21.15.89:0/2085959149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db64700 0x55f94db66af0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fef1802a930 tx=0x7fef18009590 comp rx=0 tx=0).stop 2024-09-10T08:41:01.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.956+0000 7fef31000640 1 --2- 172.21.15.89:0/4048642009 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fef100a2c80 0x7fef100a5140 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fef24005fd0 tx=0x7fef24009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:01.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.956+0000 7fef359ce740 1 -- 172.21.15.89:0/2085959149 shutdown_connections 2024-09-10T08:41:01.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.956+0000 7fef359ce740 1 --2- 172.21.15.89:0/2085959149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db64700 0x55f94db66af0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:01.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.956+0000 7fef359ce740 1 --2- 172.21.15.89:0/2085959149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f94db59e40 0x55f94db641c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:01.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.956+0000 7fef359ce740 1 --2- 172.21.15.89:0/2085959149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db59520 0x55f94db59900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:01.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.956+0000 7fef359ce740 1 -- 172.21.15.89:0/2085959149 >> 172.21.15.89:0/2085959149 conn(0x55f94db55ab0 msgr2=0x55f94db55e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:01.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.956+0000 7fef359ce740 1 -- 172.21.15.89:0/2085959149 shutdown_connections 2024-09-10T08:41:01.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.957+0000 7fef359ce740 1 -- 172.21.15.89:0/2085959149 wait complete. 2024-09-10T08:41:01.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.957+0000 7fef359ce740 1 -- start start 2024-09-10T08:41:01.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.958+0000 7fef359ce740 1 Processor -- start 2024-09-10T08:41:01.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.958+0000 7fef359ce740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db59520 0x55f94db61770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:01.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.958+0000 7fef359ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db59e40 0x55f94db61cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:01.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.958+0000 7fef359ce740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f94db64700 0x55f94db621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:01.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.958+0000 7fef359ce740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f94db672d0 con 0x55f94db64700 2024-09-10T08:41:01.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.958+0000 7fef359ce740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f94da7a350 con 0x55f94db59520 2024-09-10T08:41:01.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.958+0000 7fef359ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f94db67450 con 0x55f94db59e40 2024-09-10T08:41:01.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.958+0000 7fef359ce740 10 client.0 _ll_register_callbacks cb 0x55f94da9d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:01.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.959+0000 7fef2be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db59520 0x55f94db61770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:01.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.959+0000 7fef2be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db59520 0x55f94db61770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:45918/0 (socket says 172.21.15.89:45918) 2024-09-10T08:41:01.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.959+0000 7fef2b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db59e40 0x55f94db61cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:01.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.959+0000 7fef2be00640 1 -- 172.21.15.89:0/2860857083 learned_addr learned my addr 172.21.15.89:0/2860857083 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:01.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.959+0000 7fef2b400640 1 -- 172.21.15.89:0/2860857083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db59520 msgr2=0x55f94db61770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:01.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.959+0000 7fef2b400640 1 --2- 172.21.15.89:0/2860857083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db59520 0x55f94db61770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:01.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.959+0000 7fef2b400640 1 -- 172.21.15.89:0/2860857083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f94db64700 msgr2=0x55f94db621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:01.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.959+0000 7fef2b400640 1 --2- 172.21.15.89:0/2860857083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f94db64700 0x55f94db621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:01.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.959+0000 7fef2b400640 1 -- 172.21.15.89:0/2860857083 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f94d898960 con 0x55f94db59e40 2024-09-10T08:41:01.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.961+0000 7fef2b400640 1 --2- 172.21.15.89:0/2860857083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db59e40 0x55f94db61cb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fef18031550 tx=0x7fef1802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acd672c332b9b3d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:01.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.961+0000 7fef16000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:41:01.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.961+0000 7fef16000640 1 -- 172.21.15.89:0/2860857083 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1804a070 con 0x55f94db59e40 2024-09-10T08:41:01.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.961+0000 7fef16000640 1 -- 172.21.15.89:0/2860857083 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fef18003bb0 con 0x55f94db59e40 2024-09-10T08:41:01.961 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.962+0000 7fef359ce740 1 -- 172.21.15.89:0/2860857083 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef180035f0 con 0x55f94db59e40 2024-09-10T08:41:01.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.962+0000 7fef16000640 1 -- 172.21.15.89:0/2860857083 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fef18056050 con 0x55f94db59e40 2024-09-10T08:41:01.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.962+0000 7fef359ce740 10 client.25561 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:01.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.962+0000 7fef359ce740 1 -- 172.21.15.89:0/2860857083 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f94db672d0 con 0x55f94db59e40 2024-09-10T08:41:01.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.963+0000 7fef16000640 1 -- 172.21.15.89:0/2860857083 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef180035f0 con 0x55f94db59e40 2024-09-10T08:41:01.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.963+0000 7fef16000640 1 -- 172.21.15.89:0/2860857083 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fef18043650 con 0x55f94db59e40 2024-09-10T08:41:01.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.964+0000 7fef359ce740 10 client.25561 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:01.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.964+0000 7fef359ce740 10 client.25561 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:41:01.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.964+0000 7fef359ce740 20 client.25561 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:01.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.964+0000 7fef359ce740 1 --2- 172.21.15.89:0/2860857083 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f94db74140 0x55f94db76530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:01.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.964+0000 7fef359ce740 4 client.25561 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:41:01.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.964+0000 7fef359ce740 1 -- 172.21.15.89:0/2860857083 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f94daccbc0 con 0x55f94db74140 2024-09-10T08:41:01.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.964+0000 7fef2be00640 1 --2- 172.21.15.89:0/2860857083 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f94db74140 0x55f94db76530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:01.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.966+0000 7fef2be00640 1 --2- 172.21.15.89:0/2860857083 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f94db74140 0x55f94db76530 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9a7ea8477926d7 server_cookie=e01d22bab135316e in_seq=0 out_seq=0 2024-09-10T08:41:01.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.966+0000 7fef16000640 10 client.25561 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:01.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.966+0000 7fef16000640 1 -- 172.21.15.89:0/2860857083 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f94daccbc0 con 0x55f94db74140 2024-09-10T08:41:01.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.967+0000 7fef16000640 10 client.25561 handle_command_reply: tid=0 2024-09-10T08:41:01.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.967+0000 7fef359ce740 1 client.25561 shutdown 2024-09-10T08:41:01.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.967+0000 7fef359ce740 1 -- 172.21.15.89:0/2860857083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db59e40 msgr2=0x55f94db61cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:01.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.967+0000 7fef359ce740 1 --2- 172.21.15.89:0/2860857083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db59e40 0x55f94db61cb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fef18031550 tx=0x7fef1802a410 comp rx=0 tx=0).stop 2024-09-10T08:41:01.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.967+0000 7fef359ce740 1 -- 172.21.15.89:0/2860857083 shutdown_connections 2024-09-10T08:41:01.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.967+0000 7fef359ce740 1 --2- 172.21.15.89:0/2860857083 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f94db74140 0x55f94db76530 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:01.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.967+0000 7fef359ce740 1 --2- 172.21.15.89:0/2860857083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f94db64700 0x55f94db621f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:01.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.967+0000 7fef359ce740 1 --2- 172.21.15.89:0/2860857083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db59e40 0x55f94db61cb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:01.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.967+0000 7fef359ce740 1 --2- 172.21.15.89:0/2860857083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db59520 0x55f94db61770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:01.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.967+0000 7fef359ce740 1 -- 172.21.15.89:0/2860857083 >> 172.21.15.89:0/2860857083 conn(0x55f94db55ab0 msgr2=0x55f94db54cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:01.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.967+0000 7fef16000640 0 client.25561 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:01.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.967+0000 7fef359ce740 1 -- 172.21.15.89:0/2860857083 shutdown_connections 2024-09-10T08:41:01.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.968+0000 7fef359ce740 1 -- 172.21.15.89:0/2860857083 wait complete. 2024-09-10T08:41:01.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:01.968+0000 7fef359ce740 20 client.25561 trim_cache size 0 max 16384 2024-09-10T08:41:02.002 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.002+0000 7fef359ce740 1 Processor -- start 2024-09-10T08:41:02.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.003+0000 7fef359ce740 1 -- start start 2024-09-10T08:41:02.003 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.003+0000 7fef359ce740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f94db55ab0 0x55f94db60060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:02.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.003+0000 7fef359ce740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db59520 0x55f94db54be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:02.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.003+0000 7fef359ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db59e40 0x55f94db7bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:02.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.003+0000 7fef359ce740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c1b7b50 con 0x55f94db55ab0 2024-09-10T08:41:02.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.003+0000 7fef359ce740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c1b5420 con 0x55f94db59520 2024-09-10T08:41:02.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.003+0000 7fef359ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c1b3270 con 0x55f94db59e40 2024-09-10T08:41:02.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.003+0000 7fef2b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db59520 0x55f94db54be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:02.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.004+0000 7fef2b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db59520 0x55f94db54be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:45922/0 (socket says 172.21.15.89:45922) 2024-09-10T08:41:02.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.004+0000 7fef2b400640 1 -- 172.21.15.89:0/1427906250 learned_addr learned my addr 172.21.15.89:0/1427906250 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:02.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.004+0000 7fef2be00640 1 --2- 172.21.15.89:0/1427906250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f94db55ab0 0x55f94db60060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:02.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.004+0000 7fef31000640 1 --2- 172.21.15.89:0/1427906250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db59e40 0x55f94db7bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:02.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.004+0000 7fef2b400640 1 -- 172.21.15.89:0/1427906250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db59e40 msgr2=0x55f94db7bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:02.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.004+0000 7fef2b400640 1 --2- 172.21.15.89:0/1427906250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db59e40 0x55f94db7bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:02.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.004+0000 7fef2b400640 1 -- 172.21.15.89:0/1427906250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f94db55ab0 msgr2=0x55f94db60060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:02.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.004+0000 7fef2b400640 1 --2- 172.21.15.89:0/1427906250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f94db55ab0 0x55f94db60060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:02.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.004+0000 7fef2b400640 1 -- 172.21.15.89:0/1427906250 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f94db60740 con 0x55f94db59520 2024-09-10T08:41:02.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.004+0000 7fef2b400640 1 --2- 172.21.15.89:0/1427906250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db59520 0x55f94db54be0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fef18009900 tx=0x7fef18005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fc94c0af21a60e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:02.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.005+0000 7fef20c00640 1 -- 172.21.15.89:0/1427906250 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1804a070 con 0x55f94db59520 2024-09-10T08:41:02.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.005+0000 7fef20c00640 1 -- 172.21.15.89:0/1427906250 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef180039b0 con 0x55f94db59520 2024-09-10T08:41:02.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.005+0000 7fef20c00640 1 -- 172.21.15.89:0/1427906250 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1803e070 con 0x55f94db59520 2024-09-10T08:41:02.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.006+0000 7fef359ce740 1 -- 172.21.15.89:0/1427906250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db59520 msgr2=0x55f94db54be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:02.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.006+0000 7fef359ce740 1 --2- 172.21.15.89:0/1427906250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db59520 0x55f94db54be0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fef18009900 tx=0x7fef18005bd0 comp rx=0 tx=0).stop 2024-09-10T08:41:02.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.007+0000 7fef359ce740 1 -- 172.21.15.89:0/1427906250 shutdown_connections 2024-09-10T08:41:02.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.007+0000 7fef359ce740 1 --2- 172.21.15.89:0/1427906250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db59e40 0x55f94db7bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:02.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.007+0000 7fef359ce740 1 --2- 172.21.15.89:0/1427906250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db59520 0x55f94db54be0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:02.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.007+0000 7fef359ce740 1 --2- 172.21.15.89:0/1427906250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f94db55ab0 0x55f94db60060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:02.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.007+0000 7fef359ce740 1 -- 172.21.15.89:0/1427906250 >> 172.21.15.89:0/1427906250 conn(0x55f94db62350 msgr2=0x55f94db544d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:02.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.008+0000 7fef359ce740 1 -- 172.21.15.89:0/1427906250 shutdown_connections 2024-09-10T08:41:02.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.008+0000 7fef359ce740 1 -- 172.21.15.89:0/1427906250 wait complete. 2024-09-10T08:41:02.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.008+0000 7fef359ce740 1 -- start start 2024-09-10T08:41:02.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.009+0000 7fef359ce740 1 Processor -- start 2024-09-10T08:41:02.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.009+0000 7fef359ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db55ab0 0x55f94db6ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:02.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.010+0000 7fef359ce740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db59520 0x55f94db69670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:02.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.010+0000 7fef359ce740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f94db59e40 0x55f94db69bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:02.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.010+0000 7fef2be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db55ab0 0x55f94db6ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:02.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.010+0000 7fef2be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db55ab0 0x55f94db6ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:55872/0 (socket says 172.21.15.89:55872) 2024-09-10T08:41:02.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.010+0000 7fef359ce740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c1b38e0 con 0x55f94db59e40 2024-09-10T08:41:02.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.010+0000 7fef359ce740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c1b4d50 con 0x55f94db59520 2024-09-10T08:41:02.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.010+0000 7fef359ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c1b3fb0 con 0x55f94db55ab0 2024-09-10T08:41:02.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.010+0000 7fef2be00640 1 -- 172.21.15.89:0/4079073962 learned_addr learned my addr 172.21.15.89:0/4079073962 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:02.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.011+0000 7fef2b400640 1 --2- 172.21.15.89:0/4079073962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db59520 0x55f94db69670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:02.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.011+0000 7fef31000640 1 --2- 172.21.15.89:0/4079073962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f94db59e40 0x55f94db69bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:02.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.011+0000 7fef359ce740 10 client.0 _ll_register_callbacks cb 0x7fef1c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:02.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.011+0000 7fef2be00640 1 -- 172.21.15.89:0/4079073962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db59520 msgr2=0x55f94db69670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:02.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.011+0000 7fef2be00640 1 --2- 172.21.15.89:0/4079073962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db59520 0x55f94db69670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:02.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.011+0000 7fef2be00640 1 -- 172.21.15.89:0/4079073962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f94db59e40 msgr2=0x55f94db69bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:02.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.011+0000 7fef2be00640 1 --2- 172.21.15.89:0/4079073962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f94db59e40 0x55f94db69bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:02.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.011+0000 7fef2be00640 1 -- 172.21.15.89:0/4079073962 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f94db6a420 con 0x55f94db55ab0 2024-09-10T08:41:02.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.012+0000 7fef2be00640 1 --2- 172.21.15.89:0/4079073962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db55ab0 0x55f94db6ce90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fef1c09b860 tx=0x7fef1c00efb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bda161da65bbe85d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:02.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.012+0000 7fef16000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:41:02.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.012+0000 7fef16000640 1 -- 172.21.15.89:0/4079073962 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c0b8070 con 0x55f94db55ab0 2024-09-10T08:41:02.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.012+0000 7fef16000640 1 -- 172.21.15.89:0/4079073962 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fef1c0ccf10 con 0x55f94db55ab0 2024-09-10T08:41:02.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.012+0000 7fef359ce740 1 -- 172.21.15.89:0/4079073962 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef180039b0 con 0x55f94db55ab0 2024-09-10T08:41:02.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.013+0000 7fef16000640 1 -- 172.21.15.89:0/4079073962 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fef1c081050 con 0x55f94db55ab0 2024-09-10T08:41:02.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.013+0000 7fef359ce740 10 client.25567 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:02.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.013+0000 7fef359ce740 1 -- 172.21.15.89:0/4079073962 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f94dbdfd20 con 0x55f94db55ab0 2024-09-10T08:41:02.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.013+0000 7fef16000640 1 -- 172.21.15.89:0/4079073962 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef180039b0 con 0x55f94db55ab0 2024-09-10T08:41:02.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.013+0000 7fef16000640 1 -- 172.21.15.89:0/4079073962 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fef1c0981d0 con 0x55f94db55ab0 2024-09-10T08:41:02.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.013+0000 7fef359ce740 10 client.25567 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:02.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.013+0000 7fef359ce740 10 client.25567 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:41:02.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.014+0000 7fef359ce740 20 client.25567 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:02.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.014+0000 7fef359ce740 1 --2- 172.21.15.89:0/4079073962 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f94db64700 0x55f94dbe2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:02.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.014+0000 7fef359ce740 4 client.25567 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9590b514", "release": true, "await": true}] 2024-09-10T08:41:02.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.014+0000 7fef359ce740 1 -- 172.21.15.89:0/4079073962 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9590b514", "release": true, "await": true}) -- 0x55f94dac70d0 con 0x55f94db64700 2024-09-10T08:41:02.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.014+0000 7fef2b400640 1 --2- 172.21.15.89:0/4079073962 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f94db64700 0x55f94dbe2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:02.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.015+0000 7fef2b400640 1 --2- 172.21.15.89:0/4079073962 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f94db64700 0x55f94dbe2460 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d8007218aa8d5be server_cookie=80c96c674cb45a6b in_seq=0 out_seq=0 2024-09-10T08:41:02.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.015+0000 7fef16000640 10 client.25567 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:02.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.016+0000 7fef16000640 1 -- 172.21.15.89:0/4079073962 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55f94dac70d0 con 0x55f94db64700 2024-09-10T08:41:02.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.016+0000 7fef16000640 10 client.25567 handle_command_reply: tid=0 2024-09-10T08:41:02.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.016+0000 7fef359ce740 1 client.25567 shutdown 2024-09-10T08:41:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.017+0000 7fef359ce740 1 -- 172.21.15.89:0/4079073962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db55ab0 msgr2=0x55f94db6ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.017+0000 7fef359ce740 1 --2- 172.21.15.89:0/4079073962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db55ab0 0x55f94db6ce90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fef1c09b860 tx=0x7fef1c00efb0 comp rx=0 tx=0).stop 2024-09-10T08:41:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.017+0000 7fef359ce740 1 -- 172.21.15.89:0/4079073962 shutdown_connections 2024-09-10T08:41:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.017+0000 7fef359ce740 1 --2- 172.21.15.89:0/4079073962 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f94db64700 0x55f94dbe2460 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.017+0000 7fef359ce740 1 --2- 172.21.15.89:0/4079073962 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f94db59e40 0x55f94db69bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:02.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.017+0000 7fef359ce740 1 --2- 172.21.15.89:0/4079073962 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f94db59520 0x55f94db69670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:02.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.017+0000 7fef359ce740 1 --2- 172.21.15.89:0/4079073962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94db55ab0 0x55f94db6ce90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:02.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.017+0000 7fef359ce740 1 -- 172.21.15.89:0/4079073962 >> 172.21.15.89:0/4079073962 conn(0x55f94db62350 msgr2=0x55f94db544d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:02.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.017+0000 7fef16000640 0 client.25567 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:02.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.018+0000 7fef359ce740 1 -- 172.21.15.89:0/4079073962 shutdown_connections 2024-09-10T08:41:02.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.019+0000 7fef359ce740 1 -- 172.21.15.89:0/4079073962 wait complete. 2024-09-10T08:41:02.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.019+0000 7fef359ce740 20 client.25567 trim_cache size 0 max 16384 2024-09-10T08:41:02.027 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:41:02.028 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 26, 2024-09-10T08:41:02.028 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24554, 2024-09-10T08:41:02.028 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 4, 2024-09-10T08:41:02.028 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": { 2024-09-10T08:41:02.028 INFO:teuthology.orchestra.run.smithi089.stdout: "9590b514": { 2024-09-10T08:41:02.029 INFO:teuthology.orchestra.run.smithi089.stdout: "version": 4, 2024-09-10T08:41:02.029 INFO:teuthology.orchestra.run.smithi089.stdout: "age_ref": 0.0, 2024-09-10T08:41:02.029 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:41:02.029 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RELEASED", 2024-09-10T08:41:02.029 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:41:02.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:41:02.030 INFO:teuthology.orchestra.run.smithi089.stdout: "timeout": 90.0, 2024-09-10T08:41:02.030 INFO:teuthology.orchestra.run.smithi089.stdout: "expiration": 150.0, 2024-09-10T08:41:02.030 INFO:teuthology.orchestra.run.smithi089.stdout: "members": { 2024-09-10T08:41:02.030 INFO:teuthology.orchestra.run.smithi089.stdout: "file:/": { 2024-09-10T08:41:02.030 INFO:teuthology.orchestra.run.smithi089.stdout: "excluded": false, 2024-09-10T08:41:02.030 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:41:02.031 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RELEASED", 2024-09-10T08:41:02.031 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:41:02.031 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:41:02.031 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:41:02.031 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:41:02.031 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:41:02.032 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:41:02.032 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:41:02.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.020+0000 7fef31a00640 1 -- 172.21.15.89:0/4048642009 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fef100a2c80 msgr2=0x7fef100a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:02.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.020+0000 7fef31a00640 1 --2- 172.21.15.89:0/4048642009 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fef100a2c80 0x7fef100a5140 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fef24005fd0 tx=0x7fef24009500 comp rx=0 tx=0).stop 2024-09-10T08:41:02.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.021+0000 7fef31a00640 1 -- 172.21.15.89:0/4048642009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef2c103e30 msgr2=0x7fef2c1109c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:02.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.021+0000 7fef31a00640 1 --2- 172.21.15.89:0/4048642009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef2c103e30 0x7fef2c1109c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00ea60 tx=0x7fef1c00ef20 comp rx=0 tx=0).stop 2024-09-10T08:41:02.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.022+0000 7fef31a00640 1 -- 172.21.15.89:0/4048642009 shutdown_connections 2024-09-10T08:41:02.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.022+0000 7fef31a00640 1 --2- 172.21.15.89:0/4048642009 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fef100a2c80 0x7fef100a5140 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:02.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.022+0000 7fef31a00640 1 --2- 172.21.15.89:0/4048642009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef2c108f60 0x7fef2c10dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:02.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.022+0000 7fef31a00640 1 --2- 172.21.15.89:0/4048642009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef2c1047e0 0x7fef2c10da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:02.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.022+0000 7fef31a00640 1 --2- 172.21.15.89:0/4048642009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef2c103e30 0x7fef2c1109c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:02.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.022+0000 7fef31a00640 1 -- 172.21.15.89:0/4048642009 >> 172.21.15.89:0/4048642009 conn(0x7fef2c0ff910 msgr2=0x7fef2c100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:02.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.023+0000 7fef31a00640 1 -- 172.21.15.89:0/4048642009 shutdown_connections 2024-09-10T08:41:02.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:02.023+0000 7fef31a00640 1 -- 172.21.15.89:0/4048642009 wait complete. 2024-09-10T08:41:02.834 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9590b514', total seconds elapsed: 34.5 2024-09-10T08:41:02.834 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T08:41:02.835 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:41:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:02 smithi111 ceph-mon[35220]: pgmap v426: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T08:41:03.055 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:02 smithi089 ceph-mon[30757]: pgmap v426: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T08:41:03.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:02 smithi149 ceph-mon[36167]: pgmap v426: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T08:41:03.381 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:41:03.808 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:03 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:41:03.808 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:03 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:41:03.809 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:03 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:41:04.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:03 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:41:04.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:03 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:41:04.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:03 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:41:04.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.153+0000 7fa575800640 1 -- 172.21.15.89:0/3258036290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa57010e280 msgr2=0x7fa570110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:04.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.153+0000 7fa575800640 1 --2- 172.21.15.89:0/3258036290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa57010e280 0x7fa570110670 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa568009ca0 tx=0x7fa56802a940 comp rx=0 tx=0).stop 2024-09-10T08:41:04.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.153+0000 7fa575800640 1 -- 172.21.15.89:0/3258036290 shutdown_connections 2024-09-10T08:41:04.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.153+0000 7fa575800640 1 --2- 172.21.15.89:0/3258036290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa57010e280 0x7fa570110670 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:04.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.153+0000 7fa575800640 1 --2- 172.21.15.89:0/3258036290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa570074e30 0x7fa57010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:04.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.153+0000 7fa575800640 1 --2- 172.21.15.89:0/3258036290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa570074510 0x7fa5700748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:04.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.153+0000 7fa575800640 1 -- 172.21.15.89:0/3258036290 >> 172.21.15.89:0/3258036290 conn(0x7fa5700fd740 msgr2=0x7fa5700ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:04.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.153+0000 7fa575800640 1 -- 172.21.15.89:0/3258036290 shutdown_connections 2024-09-10T08:41:04.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.153+0000 7fa575800640 1 -- 172.21.15.89:0/3258036290 wait complete. 2024-09-10T08:41:04.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.154+0000 7fa575800640 1 Processor -- start 2024-09-10T08:41:04.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.156+0000 7fa575800640 1 -- start start 2024-09-10T08:41:04.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.156+0000 7fa575800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa570074510 0x7fa57019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:04.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.156+0000 7fa575800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa570074e30 0x7fa57019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:04.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.156+0000 7fa575800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa57010e280 0x7fa57019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:04.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.156+0000 7fa575800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa570113fc0 con 0x7fa570074e30 2024-09-10T08:41:04.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.156+0000 7fa575800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa570113e40 con 0x7fa57010e280 2024-09-10T08:41:04.157 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.156+0000 7fa575800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa570114140 con 0x7fa570074510 2024-09-10T08:41:04.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.157+0000 7fa56fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa570074510 0x7fa57019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:04.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.158+0000 7fa56fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa570074510 0x7fa57019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:57244/0 (socket says 172.21.15.89:57244) 2024-09-10T08:41:04.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.158+0000 7fa56f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa570074e30 0x7fa57019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:04.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.158+0000 7fa56f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa570074e30 0x7fa57019aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47720/0 (socket says 172.21.15.89:47720) 2024-09-10T08:41:04.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.158+0000 7fa56f400640 1 -- 172.21.15.89:0/3321310903 learned_addr learned my addr 172.21.15.89:0/3321310903 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:04.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.159+0000 7fa574e00640 1 --2- 172.21.15.89:0/3321310903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa57010e280 0x7fa57019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:04.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.159+0000 7fa56f400640 1 -- 172.21.15.89:0/3321310903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa570074510 msgr2=0x7fa57019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:04.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.159+0000 7fa56f400640 1 --2- 172.21.15.89:0/3321310903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa570074510 0x7fa57019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:04.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.159+0000 7fa56f400640 1 -- 172.21.15.89:0/3321310903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa57010e280 msgr2=0x7fa57019edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:04.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.159+0000 7fa56f400640 1 --2- 172.21.15.89:0/3321310903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa57010e280 0x7fa57019edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:04.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.159+0000 7fa56f400640 1 -- 172.21.15.89:0/3321310903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57019f4a0 con 0x7fa570074e30 2024-09-10T08:41:04.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.160+0000 7fa56fe00640 1 --2- 172.21.15.89:0/3321310903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa570074510 0x7fa57019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:41:04.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.160+0000 7fa56f400640 1 --2- 172.21.15.89:0/3321310903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa570074e30 0x7fa57019aa30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00ed60 tx=0x7fa55c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:04.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.161+0000 7fa56cc00640 1 -- 172.21.15.89:0/3321310903 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c009e10 con 0x7fa570074e30 2024-09-10T08:41:04.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.161+0000 7fa56cc00640 1 -- 172.21.15.89:0/3321310903 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa55c010910 con 0x7fa570074e30 2024-09-10T08:41:04.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.161+0000 7fa56cc00640 1 -- 172.21.15.89:0/3321310903 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c018a90 con 0x7fa570074e30 2024-09-10T08:41:04.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.161+0000 7fa575800640 1 -- 172.21.15.89:0/3321310903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa57019f790 con 0x7fa570074e30 2024-09-10T08:41:04.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.162+0000 7fa575800640 1 -- 172.21.15.89:0/3321310903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5701018d0 con 0x7fa570074e30 2024-09-10T08:41:04.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.164+0000 7fa575800640 1 -- 172.21.15.89:0/3321310903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa550005290 con 0x7fa570074e30 2024-09-10T08:41:04.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.167+0000 7fa56cc00640 1 -- 172.21.15.89:0/3321310903 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa55c018c30 con 0x7fa570074e30 2024-09-10T08:41:04.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.169+0000 7fa56cc00640 1 --2- 172.21.15.89:0/3321310903 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5540a2c00 0x7fa5540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:04.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.169+0000 7fa56cc00640 1 -- 172.21.15.89:0/3321310903 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa55c0cf770 con 0x7fa570074e30 2024-09-10T08:41:04.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.170+0000 7fa56fe00640 1 --2- 172.21.15.89:0/3321310903 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5540a2c00 0x7fa5540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:04.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.171+0000 7fa56fe00640 1 --2- 172.21.15.89:0/3321310903 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5540a2c00 0x7fa5540a50c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa5600058c0 tx=0x7fa560005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:04.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.173+0000 7fa56cc00640 1 -- 172.21.15.89:0/3321310903 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa55c09c2a0 con 0x7fa570074e30 2024-09-10T08:41:04.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:03 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:41:04.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:03 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:41:04.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:03 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:41:04.606 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.606+0000 7fa575800640 1 -- 172.21.15.89:0/3321310903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa550005580 con 0x7fa570074e30 2024-09-10T08:41:04.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.607+0000 7fa56cc00640 1 -- 172.21.15.89:0/3321310903 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11918 (secure 0 0 0) 0x7fa55c09bba0 con 0x7fa570074e30 2024-09-10T08:41:04.607 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:41:04.607 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-10T08:39:39:721431+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:39:38.834417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24518,24496]},"id":1}]} 2024-09-10T08:41:04.607 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-10T08:41:04.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.610+0000 7fa575800640 1 -- 172.21.15.89:0/3321310903 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5540a2c00 msgr2=0x7fa5540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:04.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.610+0000 7fa575800640 1 --2- 172.21.15.89:0/3321310903 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5540a2c00 0x7fa5540a50c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa5600058c0 tx=0x7fa560005850 comp rx=0 tx=0).stop 2024-09-10T08:41:04.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.610+0000 7fa575800640 1 -- 172.21.15.89:0/3321310903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa570074e30 msgr2=0x7fa57019aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:04.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.610+0000 7fa575800640 1 --2- 172.21.15.89:0/3321310903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa570074e30 0x7fa57019aa30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00ed60 tx=0x7fa55c00c6a0 comp rx=0 tx=0).stop 2024-09-10T08:41:04.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.610+0000 7fa575800640 1 -- 172.21.15.89:0/3321310903 shutdown_connections 2024-09-10T08:41:04.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.610+0000 7fa575800640 1 --2- 172.21.15.89:0/3321310903 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5540a2c00 0x7fa5540a50c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:04.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.610+0000 7fa575800640 1 --2- 172.21.15.89:0/3321310903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa57010e280 0x7fa57019edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:04.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.610+0000 7fa575800640 1 --2- 172.21.15.89:0/3321310903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa570074e30 0x7fa57019aa30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:04.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.610+0000 7fa575800640 1 --2- 172.21.15.89:0/3321310903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa570074510 0x7fa57019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:04.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.610+0000 7fa575800640 1 -- 172.21.15.89:0/3321310903 >> 172.21.15.89:0/3321310903 conn(0x7fa5700fd740 msgr2=0x7fa5700fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:04.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.611+0000 7fa575800640 1 -- 172.21.15.89:0/3321310903 shutdown_connections 2024-09-10T08:41:04.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:04.611+0000 7fa575800640 1 -- 172.21.15.89:0/3321310903 wait complete. 2024-09-10T08:41:04.995 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:04 smithi089 ceph-mon[30757]: pgmap v427: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T08:41:04.995 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:04 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:04.995 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:04 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:41:04.995 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:04 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:41:04.995 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:04 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3321310903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:41:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:04 smithi111 ceph-mon[35220]: pgmap v427: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T08:41:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:04 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:04 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:41:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:04 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:41:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:04 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3321310903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:41:05.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:04 smithi149 ceph-mon[36167]: pgmap v427: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T08:41:05.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:04 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:05.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:04 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:41:05.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:04 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:41:05.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:04 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3321310903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:41:05.506 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24554 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T08:41:05.506 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24554 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T08:41:05.956 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:41:06.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.627+0000 7fcaf0c00640 1 -- 172.21.15.89:0/3491656943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaec103ff0 msgr2=0x7fcaec1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:06.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.627+0000 7fcaf0c00640 1 --2- 172.21.15.89:0/3491656943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaec103ff0 0x7fcaec1043d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcadc009900 tx=0x7fcadc02a410 comp rx=0 tx=0).stop 2024-09-10T08:41:06.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.627+0000 7fcaf0c00640 1 -- 172.21.15.89:0/3491656943 shutdown_connections 2024-09-10T08:41:06.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.627+0000 7fcaf0c00640 1 --2- 172.21.15.89:0/3491656943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcaec109120 0x7fcaec110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.627+0000 7fcaf0c00640 1 --2- 172.21.15.89:0/3491656943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaec1049a0 0x7fcaec1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.627+0000 7fcaf0c00640 1 --2- 172.21.15.89:0/3491656943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaec103ff0 0x7fcaec1043d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.627+0000 7fcaf0c00640 1 -- 172.21.15.89:0/3491656943 >> 172.21.15.89:0/3491656943 conn(0x7fcaec0ffa90 msgr2=0x7fcaec101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:06.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.627+0000 7fcaf0c00640 1 -- 172.21.15.89:0/3491656943 shutdown_connections 2024-09-10T08:41:06.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.628+0000 7fcaf0c00640 1 -- 172.21.15.89:0/3491656943 wait complete. 2024-09-10T08:41:06.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.629+0000 7fcaf0c00640 1 Processor -- start 2024-09-10T08:41:06.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.629+0000 7fcaf0c00640 1 -- start start 2024-09-10T08:41:06.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.629+0000 7fcaf0c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcaec103ff0 0x7fcaec19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:06.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.629+0000 7fcaf0c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaec1049a0 0x7fcaec19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:06.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.629+0000 7fcaf0c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaec109120 0x7fcaec19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:06.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.629+0000 7fcaf0c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec114150 con 0x7fcaec109120 2024-09-10T08:41:06.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.629+0000 7fcaf0c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec113fd0 con 0x7fcaec1049a0 2024-09-10T08:41:06.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.629+0000 7fcaf0c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec1142d0 con 0x7fcaec103ff0 2024-09-10T08:41:06.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.630+0000 7fcaea800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaec109120 0x7fcaec19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:06.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.630+0000 7fcaea800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaec109120 0x7fcaec19eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47750/0 (socket says 172.21.15.89:47750) 2024-09-10T08:41:06.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.630+0000 7fcaea800640 1 -- 172.21.15.89:0/2455054425 learned_addr learned my addr 172.21.15.89:0/2455054425 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:06.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.630+0000 7fcae9400640 1 --2- 172.21.15.89:0/2455054425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaec1049a0 0x7fcaec19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:06.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.630+0000 7fcae9e00640 1 --2- 172.21.15.89:0/2455054425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcaec103ff0 0x7fcaec19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:06.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.630+0000 7fcaea800640 1 -- 172.21.15.89:0/2455054425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcaec103ff0 msgr2=0x7fcaec19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:06.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.630+0000 7fcaea800640 1 --2- 172.21.15.89:0/2455054425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcaec103ff0 0x7fcaec19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.630+0000 7fcaea800640 1 -- 172.21.15.89:0/2455054425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaec1049a0 msgr2=0x7fcaec19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:06.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.630+0000 7fcaea800640 1 --2- 172.21.15.89:0/2455054425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaec1049a0 0x7fcaec19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.630+0000 7fcaea800640 1 -- 172.21.15.89:0/2455054425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec19f5c0 con 0x7fcaec109120 2024-09-10T08:41:06.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.630+0000 7fcae9e00640 1 --2- 172.21.15.89:0/2455054425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcaec103ff0 0x7fcaec19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:41:06.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.631+0000 7fcaea800640 1 --2- 172.21.15.89:0/2455054425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaec109120 0x7fcaec19eec0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcae400eb10 tx=0x7fcae400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:06.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.631+0000 7fcae2a00640 1 -- 172.21.15.89:0/2455054425 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae4004b20 con 0x7fcaec109120 2024-09-10T08:41:06.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.631+0000 7fcae2a00640 1 -- 172.21.15.89:0/2455054425 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcae4004cc0 con 0x7fcaec109120 2024-09-10T08:41:06.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.631+0000 7fcae2a00640 1 -- 172.21.15.89:0/2455054425 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae4010760 con 0x7fcaec109120 2024-09-10T08:41:06.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.632+0000 7fcaf0c00640 1 -- 172.21.15.89:0/2455054425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaec19f8b0 con 0x7fcaec109120 2024-09-10T08:41:06.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.632+0000 7fcaf0c00640 1 -- 172.21.15.89:0/2455054425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaec06bc10 con 0x7fcaec109120 2024-09-10T08:41:06.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.635+0000 7fcae2a00640 1 -- 172.21.15.89:0/2455054425 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fcae4005000 con 0x7fcaec109120 2024-09-10T08:41:06.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.636+0000 7fcaf3614740 1 Processor -- start 2024-09-10T08:41:06.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.636+0000 7fcaf3614740 1 -- start start 2024-09-10T08:41:06.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.636+0000 7fcae2a00640 1 --2- 172.21.15.89:0/2455054425 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcad00a2c00 0x7fcad00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:06.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.637+0000 7fcae2a00640 1 -- 172.21.15.89:0/2455054425 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fcae40cfb50 con 0x7fcaec109120 2024-09-10T08:41:06.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.637+0000 7fcaf3614740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f902b92d0 0x557f902b96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:06.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.637+0000 7fcaf3614740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902b9bf0 0x557f902cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:06.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.637+0000 7fcaf3614740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902ccda0 0x557f902cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:06.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.637+0000 7fcaf3614740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557f9021f530 con 0x557f902b9bf0 2024-09-10T08:41:06.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.637+0000 7fcaf3614740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557f901c6860 con 0x557f902ccda0 2024-09-10T08:41:06.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.637+0000 7fcaf3614740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557f9021b4f0 con 0x557f902b92d0 2024-09-10T08:41:06.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.637+0000 7fcae9e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f902b92d0 0x557f902b96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:06.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.637+0000 7fcaea800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902ccda0 0x557f902cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:06.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.637+0000 7fcae9400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902b9bf0 0x557f902cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:06.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.637+0000 7fcaea800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902ccda0 0x557f902cf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:58294/0 (socket says 172.21.15.89:58294) 2024-09-10T08:41:06.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.637+0000 7fcaea800640 1 -- 172.21.15.89:0/1485074871 learned_addr learned my addr 172.21.15.89:0/1485074871 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:06.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.637+0000 7fcae9400640 1 --2- 172.21.15.89:0/2455054425 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcad00a2c00 0x7fcad00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:06.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.638+0000 7fcaea800640 1 -- 172.21.15.89:0/1485074871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f902b92d0 msgr2=0x557f902b96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:06.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.638+0000 7fcaea800640 1 --2- 172.21.15.89:0/1485074871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f902b92d0 0x557f902b96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.638+0000 7fcaea800640 1 -- 172.21.15.89:0/1485074871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902b9bf0 msgr2=0x557f902cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:06.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.638+0000 7fcaea800640 1 --2- 172.21.15.89:0/1485074871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902b9bf0 0x557f902cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.638+0000 7fcaea800640 1 -- 172.21.15.89:0/1485074871 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f8ff90b90 con 0x557f902ccda0 2024-09-10T08:41:06.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.638+0000 7fcae9400640 1 --2- 172.21.15.89:0/1485074871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902b9bf0 0x557f902cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:06.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.639+0000 7fcaea800640 1 --2- 172.21.15.89:0/1485074871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902ccda0 0x557f902cf190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcae4098950 tx=0x7fcae409bbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d525874dd13399d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:06.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.639+0000 7fcae9400640 1 --2- 172.21.15.89:0/2455054425 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcad00a2c00 0x7fcad00a50c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0ffeb0 tx=0x7fcad8016040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:06.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.640+0000 7fcad7000640 1 -- 172.21.15.89:0/1485074871 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae40bc070 con 0x557f902ccda0 2024-09-10T08:41:06.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.640+0000 7fcad7000640 1 -- 172.21.15.89:0/1485074871 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcae40a0330 con 0x557f902ccda0 2024-09-10T08:41:06.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.640+0000 7fcad7000640 1 -- 172.21.15.89:0/1485074871 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae40bae20 con 0x557f902ccda0 2024-09-10T08:41:06.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.641+0000 7fcaf3614740 1 -- 172.21.15.89:0/1485074871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902ccda0 msgr2=0x557f902cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:06.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.641+0000 7fcaf3614740 1 --2- 172.21.15.89:0/1485074871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902ccda0 0x557f902cf190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcae4098950 tx=0x7fcae409bbf0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.641+0000 7fcaf3614740 1 -- 172.21.15.89:0/1485074871 shutdown_connections 2024-09-10T08:41:06.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.641+0000 7fcaf3614740 1 --2- 172.21.15.89:0/1485074871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902ccda0 0x557f902cf190 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.641+0000 7fcaf3614740 1 --2- 172.21.15.89:0/1485074871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902b9bf0 0x557f902cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.641+0000 7fcaf3614740 1 --2- 172.21.15.89:0/1485074871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f902b92d0 0x557f902b96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.641+0000 7fcaf3614740 1 -- 172.21.15.89:0/1485074871 >> 172.21.15.89:0/1485074871 conn(0x557f902b5940 msgr2=0x557f902b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:06.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.642+0000 7fcaf3614740 1 -- 172.21.15.89:0/1485074871 shutdown_connections 2024-09-10T08:41:06.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.642+0000 7fcaf3614740 1 -- 172.21.15.89:0/1485074871 wait complete. 2024-09-10T08:41:06.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.642+0000 7fcaf3614740 1 -- start start 2024-09-10T08:41:06.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.643+0000 7fcaf3614740 1 Processor -- start 2024-09-10T08:41:06.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.643+0000 7fcaf3614740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f902b92d0 0x557f902c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:06.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.643+0000 7fcaf3614740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902b9bf0 0x557f902ca270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:06.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.643+0000 7fcaf3614740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902ccda0 0x557f902ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:06.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.643+0000 7fcaf3614740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557f902cf910 con 0x557f902ccda0 2024-09-10T08:41:06.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.643+0000 7fcaf3614740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557f901d9e60 con 0x557f902b9bf0 2024-09-10T08:41:06.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.643+0000 7fcaf3614740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557f902cfa90 con 0x557f902b92d0 2024-09-10T08:41:06.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.643+0000 7fcaf3614740 10 client.0 _ll_register_callbacks cb 0x557f901decc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:06.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.644+0000 7fcaea800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902ccda0 0x557f902ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:06.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.644+0000 7fcaea800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902ccda0 0x557f902ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47768/0 (socket says 172.21.15.89:47768) 2024-09-10T08:41:06.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.644+0000 7fcaea800640 1 -- 172.21.15.89:0/1781658063 learned_addr learned my addr 172.21.15.89:0/1781658063 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:06.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.644+0000 7fcae9400640 1 --2- 172.21.15.89:0/1781658063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902b9bf0 0x557f902ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:06.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.644+0000 7fcae9e00640 1 --2- 172.21.15.89:0/1781658063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f902b92d0 0x557f902c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:06.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.645+0000 7fcae9400640 1 -- 172.21.15.89:0/1781658063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f902b92d0 msgr2=0x557f902c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:06.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.645+0000 7fcae9400640 1 --2- 172.21.15.89:0/1781658063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f902b92d0 0x557f902c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.645+0000 7fcae9400640 1 -- 172.21.15.89:0/1781658063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902ccda0 msgr2=0x557f902ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:06.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.645+0000 7fcae9400640 1 --2- 172.21.15.89:0/1781658063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902ccda0 0x557f902ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.645+0000 7fcae9400640 1 -- 172.21.15.89:0/1781658063 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f901e5990 con 0x557f902b9bf0 2024-09-10T08:41:06.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.645+0000 7fcaea800640 1 --2- 172.21.15.89:0/1781658063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902ccda0 0x557f902ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:06.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.645+0000 7fcae9400640 1 --2- 172.21.15.89:0/1781658063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902b9bf0 0x557f902ca270 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcad80117b0 tx=0x7fcad8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9ef7ca8cc08565c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:06.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.645+0000 7fcacb400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:41:06.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.646+0000 7fcacb400640 1 -- 172.21.15.89:0/1781658063 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad8012c60 con 0x557f902b9bf0 2024-09-10T08:41:06.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.646+0000 7fcacb400640 1 -- 172.21.15.89:0/1781658063 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fcad8012e00 con 0x557f902b9bf0 2024-09-10T08:41:06.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.646+0000 7fcaf3614740 1 -- 172.21.15.89:0/1781658063 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcae40a0330 con 0x557f902b9bf0 2024-09-10T08:41:06.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.646+0000 7fcacb400640 1 -- 172.21.15.89:0/1781658063 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcad8030050 con 0x557f902b9bf0 2024-09-10T08:41:06.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.646+0000 7fcaf3614740 10 client.25625 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:06.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.646+0000 7fcaf3614740 1 -- 172.21.15.89:0/1781658063 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557f902db690 con 0x557f902b9bf0 2024-09-10T08:41:06.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.647+0000 7fcacb400640 1 -- 172.21.15.89:0/1781658063 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcae40a0330 con 0x557f902b9bf0 2024-09-10T08:41:06.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.647+0000 7fcacb400640 1 -- 172.21.15.89:0/1781658063 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcad8004b80 con 0x557f902b9bf0 2024-09-10T08:41:06.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.647+0000 7fcaf3614740 10 client.25625 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:06.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.647+0000 7fcaf3614740 10 client.25625 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:41:06.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.647+0000 7fcaf3614740 20 client.25625 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:06.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.647+0000 7fcaf3614740 1 --2- 172.21.15.89:0/1781658063 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x557f902dcbf0 0x557f902df010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:06.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.647+0000 7fcaf3614740 4 client.25625 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:41:06.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.647+0000 7fcaf3614740 1 -- 172.21.15.89:0/1781658063 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f901c2310 con 0x557f902dcbf0 2024-09-10T08:41:06.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.648+0000 7fcae9e00640 1 --2- 172.21.15.89:0/1781658063 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x557f902dcbf0 0x557f902df010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:06.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.649+0000 7fcae9e00640 1 --2- 172.21.15.89:0/1781658063 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x557f902dcbf0 0x557f902df010 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18abcbc00aec8075 server_cookie=71c8b6e9199fd74f in_seq=0 out_seq=0 2024-09-10T08:41:06.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.649+0000 7fcacb400640 10 client.25625 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:06.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.650+0000 7fcacb400640 1 -- 172.21.15.89:0/1781658063 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557f901c2310 con 0x557f902dcbf0 2024-09-10T08:41:06.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.650+0000 7fcacb400640 10 client.25625 handle_command_reply: tid=0 2024-09-10T08:41:06.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.650+0000 7fcaf3614740 1 client.25625 shutdown 2024-09-10T08:41:06.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.650+0000 7fcaf3614740 1 -- 172.21.15.89:0/1781658063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902b9bf0 msgr2=0x557f902ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:06.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.650+0000 7fcaf3614740 1 --2- 172.21.15.89:0/1781658063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902b9bf0 0x557f902ca270 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcad80117b0 tx=0x7fcad8004730 comp rx=0 tx=0).stop 2024-09-10T08:41:06.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.651+0000 7fcaf3614740 1 -- 172.21.15.89:0/1781658063 shutdown_connections 2024-09-10T08:41:06.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.651+0000 7fcaf3614740 1 --2- 172.21.15.89:0/1781658063 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x557f902dcbf0 0x557f902df010 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.651+0000 7fcacb400640 0 client.25625 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:06.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.651+0000 7fcaf3614740 1 --2- 172.21.15.89:0/1781658063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902ccda0 0x557f902ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.651+0000 7fcaf3614740 1 --2- 172.21.15.89:0/1781658063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902b9bf0 0x557f902ca270 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.651+0000 7fcaf3614740 1 --2- 172.21.15.89:0/1781658063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f902b92d0 0x557f902c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.651+0000 7fcaf3614740 1 -- 172.21.15.89:0/1781658063 >> 172.21.15.89:0/1781658063 conn(0x557f902b5940 msgr2=0x557f902b4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:06.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.651+0000 7fcaf3614740 1 -- 172.21.15.89:0/1781658063 shutdown_connections 2024-09-10T08:41:06.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.651+0000 7fcaf3614740 1 -- 172.21.15.89:0/1781658063 wait complete. 2024-09-10T08:41:06.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.651+0000 7fcaf3614740 20 client.25625 trim_cache size 0 max 16384 2024-09-10T08:41:06.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.669+0000 7fcaf3614740 1 Processor -- start 2024-09-10T08:41:06.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.669+0000 7fcaf3614740 1 -- start start 2024-09-10T08:41:06.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.669+0000 7fcaf3614740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f902b92d0 0x557f902b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:06.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.669+0000 7fcaf3614740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902b9bf0 0x557f902e48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:06.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.669+0000 7fcaf3614740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902ccda0 0x557f902e4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:06.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.669+0000 7fcaf3614740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec074080 con 0x557f902b9bf0 2024-09-10T08:41:06.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.669+0000 7fcaf3614740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec071950 con 0x557f902ccda0 2024-09-10T08:41:06.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.669+0000 7fcaf3614740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec06f7a0 con 0x557f902b92d0 2024-09-10T08:41:06.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.670+0000 7fcaea800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902ccda0 0x557f902e4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:06.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.670+0000 7fcaea800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902ccda0 0x557f902e4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:58316/0 (socket says 172.21.15.89:58316) 2024-09-10T08:41:06.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.670+0000 7fcaea800640 1 -- 172.21.15.89:0/2578831245 learned_addr learned my addr 172.21.15.89:0/2578831245 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:06.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.670+0000 7fcae9400640 1 --2- 172.21.15.89:0/2578831245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902b9bf0 0x557f902e48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:06.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.671+0000 7fcae9e00640 1 --2- 172.21.15.89:0/2578831245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f902b92d0 0x557f902b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:06.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.671+0000 7fcae9400640 1 -- 172.21.15.89:0/2578831245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f902b92d0 msgr2=0x557f902b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:06.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.671+0000 7fcae9400640 1 --2- 172.21.15.89:0/2578831245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f902b92d0 0x557f902b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.671+0000 7fcae9400640 1 -- 172.21.15.89:0/2578831245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902ccda0 msgr2=0x557f902e4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:06.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.671+0000 7fcae9400640 1 --2- 172.21.15.89:0/2578831245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902ccda0 0x557f902e4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.671+0000 7fcae9400640 1 -- 172.21.15.89:0/2578831245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f902b5770 con 0x557f902b9bf0 2024-09-10T08:41:06.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.671+0000 7fcae9400640 1 --2- 172.21.15.89:0/2578831245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902b9bf0 0x557f902e48c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcad8004f20 tx=0x7fcad8009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac16c701666e08a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:06.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.672+0000 7fcad6600640 1 -- 172.21.15.89:0/2578831245 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad8011e90 con 0x557f902b9bf0 2024-09-10T08:41:06.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.672+0000 7fcad6600640 1 -- 172.21.15.89:0/2578831245 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad8028a10 con 0x557f902b9bf0 2024-09-10T08:41:06.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.672+0000 7fcad6600640 1 -- 172.21.15.89:0/2578831245 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad8004a00 con 0x557f902b9bf0 2024-09-10T08:41:06.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.673+0000 7fcaf3614740 1 -- 172.21.15.89:0/2578831245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902b9bf0 msgr2=0x557f902e48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:06.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.673+0000 7fcaf3614740 1 --2- 172.21.15.89:0/2578831245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902b9bf0 0x557f902e48c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcad8004f20 tx=0x7fcad8009900 comp rx=0 tx=0).stop 2024-09-10T08:41:06.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.673+0000 7fcaf3614740 1 -- 172.21.15.89:0/2578831245 shutdown_connections 2024-09-10T08:41:06.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.673+0000 7fcaf3614740 1 --2- 172.21.15.89:0/2578831245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902ccda0 0x557f902e4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.673+0000 7fcaf3614740 1 --2- 172.21.15.89:0/2578831245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902b9bf0 0x557f902e48c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.673+0000 7fcaf3614740 1 --2- 172.21.15.89:0/2578831245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f902b92d0 0x557f902b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.673+0000 7fcaf3614740 1 -- 172.21.15.89:0/2578831245 >> 172.21.15.89:0/2578831245 conn(0x557f902b5940 msgr2=0x557f902c8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:06.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.673+0000 7fcaf3614740 1 -- 172.21.15.89:0/2578831245 shutdown_connections 2024-09-10T08:41:06.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.674+0000 7fcaf3614740 1 -- 172.21.15.89:0/2578831245 wait complete. 2024-09-10T08:41:06.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.674+0000 7fcaf3614740 1 -- start start 2024-09-10T08:41:06.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.675+0000 7fcaf3614740 1 Processor -- start 2024-09-10T08:41:06.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.675+0000 7fcaf3614740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902b92d0 0x557f902d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:06.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.676+0000 7fcaf3614740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902b9bf0 0x557f902d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:06.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.676+0000 7fcaf3614740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f902ccda0 0x557f90346890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:06.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.676+0000 7fcaf3614740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec06fe10 con 0x557f902b92d0 2024-09-10T08:41:06.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.676+0000 7fcaf3614740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec071280 con 0x557f902b9bf0 2024-09-10T08:41:06.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.676+0000 7fcaf3614740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec0704e0 con 0x557f902ccda0 2024-09-10T08:41:06.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.676+0000 7fcaf3614740 10 client.0 _ll_register_callbacks cb 0x7fcad8009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:06.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.676+0000 7fcae9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902b92d0 0x557f902d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:06.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.676+0000 7fcae9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902b92d0 0x557f902d12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47788/0 (socket says 172.21.15.89:47788) 2024-09-10T08:41:06.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.676+0000 7fcae9e00640 1 -- 172.21.15.89:0/1021357005 learned_addr learned my addr 172.21.15.89:0/1021357005 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:06.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.676+0000 7fcae9400640 1 --2- 172.21.15.89:0/1021357005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902b9bf0 0x557f902d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:06.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.676+0000 7fcaea800640 1 --2- 172.21.15.89:0/1021357005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f902ccda0 0x557f90346890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:06.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.677+0000 7fcae9400640 1 -- 172.21.15.89:0/1021357005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f902ccda0 msgr2=0x557f90346890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:06.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.677+0000 7fcae9400640 1 --2- 172.21.15.89:0/1021357005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f902ccda0 0x557f90346890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.677+0000 7fcae9400640 1 -- 172.21.15.89:0/1021357005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902b92d0 msgr2=0x557f902d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:06.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.679+0000 7fcae9400640 1 --2- 172.21.15.89:0/1021357005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902b92d0 0x557f902d12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.679+0000 7fcae9400640 1 -- 172.21.15.89:0/1021357005 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f90346ed0 con 0x557f902b9bf0 2024-09-10T08:41:06.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.679+0000 7fcae9e00640 1 --2- 172.21.15.89:0/1021357005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902b92d0 0x557f902d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:06.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.679+0000 7fcae9400640 1 --2- 172.21.15.89:0/1021357005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902b9bf0 0x557f902d1800 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcad8004f20 tx=0x7fcad8011fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a15764d7dbe01f7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:06.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.679+0000 7fcacb400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:41:06.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.680+0000 7fcacb400640 1 -- 172.21.15.89:0/1021357005 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad8040070 con 0x557f902b9bf0 2024-09-10T08:41:06.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.680+0000 7fcacb400640 1 -- 172.21.15.89:0/1021357005 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fcad8028dc0 con 0x557f902b9bf0 2024-09-10T08:41:06.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.680+0000 7fcaf3614740 1 -- 172.21.15.89:0/1021357005 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcad8028a10 con 0x557f902b9bf0 2024-09-10T08:41:06.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.680+0000 7fcacb400640 1 -- 172.21.15.89:0/1021357005 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcad804a050 con 0x557f902b9bf0 2024-09-10T08:41:06.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.681+0000 7fcaf3614740 10 client.25631 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:06.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.681+0000 7fcaf3614740 1 -- 172.21.15.89:0/1021357005 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557f90348190 con 0x557f902b9bf0 2024-09-10T08:41:06.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.681+0000 7fcacb400640 1 -- 172.21.15.89:0/1021357005 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad8028a10 con 0x557f902b9bf0 2024-09-10T08:41:06.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.681+0000 7fcacb400640 1 -- 172.21.15.89:0/1021357005 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcad8004fa0 con 0x557f902b9bf0 2024-09-10T08:41:06.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.681+0000 7fcaf3614740 10 client.25631 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:06.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.681+0000 7fcaf3614740 10 client.25631 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:41:06.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.681+0000 7fcaf3614740 20 client.25631 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:06.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.681+0000 7fcaf3614740 1 --2- 172.21.15.89:0/1021357005 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x557f903488f0 0x557f9034ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:06.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.681+0000 7fcaf3614740 4 client.25631 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T08:41:06.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.681+0000 7fcaf3614740 1 -- 172.21.15.89:0/1021357005 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x557f9022cc60 con 0x557f903488f0 2024-09-10T08:41:06.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.682+0000 7fcae9e00640 1 --2- 172.21.15.89:0/1021357005 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x557f903488f0 0x557f9034ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:06.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.683+0000 7fcacb400640 10 client.25631 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:06.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.683+0000 7fcae9e00640 1 --2- 172.21.15.89:0/1021357005 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x557f903488f0 0x557f9034ad10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ba3022d873c99b6 server_cookie=365b976b94f682a1 in_seq=0 out_seq=0 2024-09-10T08:41:06.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.689+0000 7fcacb400640 1 -- 172.21.15.89:0/1021357005 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x557f9022cc60 con 0x557f903488f0 2024-09-10T08:41:06.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.689+0000 7fcacb400640 10 client.25631 handle_command_reply: tid=0 2024-09-10T08:41:06.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.689+0000 7fcaf3614740 1 client.25631 shutdown 2024-09-10T08:41:06.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.690+0000 7fcaf3614740 1 -- 172.21.15.89:0/1021357005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902b9bf0 msgr2=0x557f902d1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:06.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.690+0000 7fcaf3614740 1 --2- 172.21.15.89:0/1021357005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902b9bf0 0x557f902d1800 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcad8004f20 tx=0x7fcad8011fd0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.690+0000 7fcaf3614740 1 -- 172.21.15.89:0/1021357005 shutdown_connections 2024-09-10T08:41:06.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.690+0000 7fcaf3614740 1 --2- 172.21.15.89:0/1021357005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f902ccda0 0x557f90346890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.690+0000 7fcaf3614740 1 --2- 172.21.15.89:0/1021357005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f902b9bf0 0x557f902d1800 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.690+0000 7fcaf3614740 1 --2- 172.21.15.89:0/1021357005 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x557f903488f0 0x557f9034ad10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.690+0000 7fcaf3614740 1 --2- 172.21.15.89:0/1021357005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557f902b92d0 0x557f902d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.690+0000 7fcaf3614740 1 -- 172.21.15.89:0/1021357005 >> 172.21.15.89:0/1021357005 conn(0x557f902b5940 msgr2=0x557f902c9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:06.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.690+0000 7fcacb400640 0 client.25631 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:06.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.691+0000 7fcaf3614740 1 -- 172.21.15.89:0/1021357005 shutdown_connections 2024-09-10T08:41:06.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.691+0000 7fcaf3614740 1 -- 172.21.15.89:0/1021357005 wait complete. 2024-09-10T08:41:06.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.691+0000 7fcaf3614740 20 client.25631 trim_cache size 0 max 16384 2024-09-10T08:41:06.692 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:41:06.692 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 26, 2024-09-10T08:41:06.692 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24554, 2024-09-10T08:41:06.692 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 6, 2024-09-10T08:41:06.692 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": { 2024-09-10T08:41:06.692 INFO:teuthology.orchestra.run.smithi089.stdout: "ada3e43a": { 2024-09-10T08:41:06.692 INFO:teuthology.orchestra.run.smithi089.stdout: "version": 6, 2024-09-10T08:41:06.692 INFO:teuthology.orchestra.run.smithi089.stdout: "age_ref": 0.0, 2024-09-10T08:41:06.692 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:41:06.692 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCED", 2024-09-10T08:41:06.692 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:41:06.693 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:41:06.693 INFO:teuthology.orchestra.run.smithi089.stdout: "timeout": 90.0, 2024-09-10T08:41:06.693 INFO:teuthology.orchestra.run.smithi089.stdout: "expiration": 150.0, 2024-09-10T08:41:06.693 INFO:teuthology.orchestra.run.smithi089.stdout: "members": { 2024-09-10T08:41:06.693 INFO:teuthology.orchestra.run.smithi089.stdout: "file:/": { 2024-09-10T08:41:06.693 INFO:teuthology.orchestra.run.smithi089.stdout: "excluded": false, 2024-09-10T08:41:06.693 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:41:06.693 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCED", 2024-09-10T08:41:06.693 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:41:06.693 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:41:06.693 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:41:06.693 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:41:06.693 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:41:06.693 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:41:06.694 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:41:06.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.693+0000 7fcaf0c00640 1 -- 172.21.15.89:0/2455054425 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcad00a2c00 msgr2=0x7fcad00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:06.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.693+0000 7fcaf0c00640 1 --2- 172.21.15.89:0/2455054425 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcad00a2c00 0x7fcad00a50c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0ffeb0 tx=0x7fcad8016040 comp rx=0 tx=0).stop 2024-09-10T08:41:06.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.693+0000 7fcaf0c00640 1 -- 172.21.15.89:0/2455054425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaec109120 msgr2=0x7fcaec19eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:06.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.693+0000 7fcaf0c00640 1 --2- 172.21.15.89:0/2455054425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaec109120 0x7fcaec19eec0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcae400eb10 tx=0x7fcae400c2d0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.693+0000 7fcaf0c00640 1 -- 172.21.15.89:0/2455054425 shutdown_connections 2024-09-10T08:41:06.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.693+0000 7fcaf0c00640 1 --2- 172.21.15.89:0/2455054425 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcad00a2c00 0x7fcad00a50c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.693+0000 7fcaf0c00640 1 --2- 172.21.15.89:0/2455054425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcaec109120 0x7fcaec19eec0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.693+0000 7fcaf0c00640 1 --2- 172.21.15.89:0/2455054425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaec1049a0 0x7fcaec19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.693+0000 7fcaf0c00640 1 --2- 172.21.15.89:0/2455054425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcaec103ff0 0x7fcaec19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:06.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.693+0000 7fcaf0c00640 1 -- 172.21.15.89:0/2455054425 >> 172.21.15.89:0/2455054425 conn(0x7fcaec0ffa90 msgr2=0x7fcaec100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:06.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.694+0000 7fcaf0c00640 1 -- 172.21.15.89:0/2455054425 shutdown_connections 2024-09-10T08:41:06.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:06.694+0000 7fcaf0c00640 1 -- 172.21.15.89:0/2455054425 wait complete. 2024-09-10T08:41:07.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:06 smithi111 ceph-mon[35220]: pgmap v428: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T08:41:07.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:06 smithi089 ceph-mon[30757]: pgmap v428: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T08:41:07.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:06 smithi149 ceph-mon[36167]: pgmap v428: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T08:41:07.659 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ada3e43a', quiesce took 4.8 seconds. Will release after: 25.2 2024-09-10T08:41:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.623+0000 7f2266c00640 1 -- 172.21.15.89:0/2259558257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2260101f40 msgr2=0x7f2260109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.623+0000 7f2266c00640 1 --2- 172.21.15.89:0/2259558257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2260101f40 0x7f2260109db0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2258009ca0 tx=0x7f225802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.624+0000 7f2266c00640 1 -- 172.21.15.89:0/2259558257 shutdown_connections 2024-09-10T08:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.624+0000 7f2266c00640 1 --2- 172.21.15.89:0/2259558257 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2260101f40 0x7f2260109db0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.624+0000 7f2266c00640 1 --2- 172.21.15.89:0/2259558257 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2260101580 0x7f2260101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.624+0000 7f2266c00640 1 --2- 172.21.15.89:0/2259558257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2260103050 0x7f2260103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.624+0000 7f2266c00640 1 -- 172.21.15.89:0/2259558257 >> 172.21.15.89:0/2259558257 conn(0x7f22600fd0d0 msgr2=0x7f22600ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.624+0000 7f2266c00640 1 -- 172.21.15.89:0/2259558257 shutdown_connections 2024-09-10T08:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.624+0000 7f2266c00640 1 -- 172.21.15.89:0/2259558257 wait complete. 2024-09-10T08:41:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.625+0000 7f2266c00640 1 Processor -- start 2024-09-10T08:41:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.625+0000 7f2266c00640 1 -- start start 2024-09-10T08:41:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.625+0000 7f2266c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2260101580 0x7f2260197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.625+0000 7f2266c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2260101f40 0x7f2260198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.625+0000 7f2266c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2260103050 0x7f226019f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.625+0000 7f2266c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f226010d720 con 0x7f2260101580 2024-09-10T08:41:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.625+0000 7f2266c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f226010d5a0 con 0x7f2260103050 2024-09-10T08:41:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.625+0000 7f2266c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f226010d8a0 con 0x7f2260101f40 2024-09-10T08:41:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.625+0000 7f2265800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2260101580 0x7f2260197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.625+0000 7f2265800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2260101580 0x7f2260197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47810/0 (socket says 172.21.15.89:47810) 2024-09-10T08:41:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.625+0000 7f2265800640 1 -- 172.21.15.89:0/617878400 learned_addr learned my addr 172.21.15.89:0/617878400 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.626+0000 7f2264e00640 1 --2- 172.21.15.89:0/617878400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2260101f40 0x7f2260198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.626+0000 7f2266200640 1 --2- 172.21.15.89:0/617878400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2260103050 0x7f226019f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.626+0000 7f2265800640 1 -- 172.21.15.89:0/617878400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2260101f40 msgr2=0x7f2260198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.626+0000 7f2265800640 1 --2- 172.21.15.89:0/617878400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2260101f40 0x7f2260198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.626+0000 7f2265800640 1 -- 172.21.15.89:0/617878400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2260103050 msgr2=0x7f226019f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.626+0000 7f2265800640 1 --2- 172.21.15.89:0/617878400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2260103050 0x7f226019f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.626+0000 7f2265800640 1 -- 172.21.15.89:0/617878400 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22600744a0 con 0x7f2260101580 2024-09-10T08:41:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.626+0000 7f2266200640 1 --2- 172.21.15.89:0/617878400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2260103050 0x7f226019f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:41:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.626+0000 7f2265800640 1 --2- 172.21.15.89:0/617878400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2260101580 0x7f2260197d50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f225400d770 tx=0x7f225400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.627+0000 7f225e000640 1 -- 172.21.15.89:0/617878400 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225401c070 con 0x7f2260101580 2024-09-10T08:41:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.627+0000 7f225e000640 1 -- 172.21.15.89:0/617878400 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2254009820 con 0x7f2260101580 2024-09-10T08:41:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.627+0000 7f2266c00640 1 -- 172.21.15.89:0/617878400 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2260074790 con 0x7f2260101580 2024-09-10T08:41:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.627+0000 7f225e000640 1 -- 172.21.15.89:0/617878400 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22540038e0 con 0x7f2260101580 2024-09-10T08:41:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.627+0000 7f2266c00640 1 -- 172.21.15.89:0/617878400 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2260074ce0 con 0x7f2260101580 2024-09-10T08:41:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.632+0000 7f225e000640 1 -- 172.21.15.89:0/617878400 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f22540099d0 con 0x7f2260101580 2024-09-10T08:41:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.633+0000 7f225e000640 1 --2- 172.21.15.89:0/617878400 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f22480a2b80 0x7f22480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.633+0000 7f226aff3740 1 Processor -- start 2024-09-10T08:41:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.633+0000 7f225e000640 1 -- 172.21.15.89:0/617878400 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f22540d7400 con 0x7f2260101580 2024-09-10T08:41:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.634+0000 7f226aff3740 1 -- start start 2024-09-10T08:41:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.634+0000 7f2264e00640 1 --2- 172.21.15.89:0/617878400 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f22480a2b80 0x7f22480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.634+0000 7f226aff3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b2d260 0x5600f5b2d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.634+0000 7f226aff3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600f5b2dba0 0x5600f5b32660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.634+0000 7f226aff3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b32c30 0x5600f5b350f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.634+0000 7f226aff3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5600f5a92100 con 0x5600f5b32c30 2024-09-10T08:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.634+0000 7f226aff3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5600f5a3a1b0 con 0x5600f5b2d260 2024-09-10T08:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.634+0000 7f226aff3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5600f5a8e0c0 con 0x5600f5b2dba0 2024-09-10T08:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.634+0000 7f2266200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b32c30 0x5600f5b350f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.634+0000 7f2266200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b32c30 0x5600f5b350f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47822/0 (socket says 172.21.15.89:47822) 2024-09-10T08:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.634+0000 7f2266200640 1 -- 172.21.15.89:0/1532857677 learned_addr learned my addr 172.21.15.89:0/1532857677 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.635+0000 7f2265800640 1 --2- 172.21.15.89:0/1532857677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b2d260 0x5600f5b2d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.635+0000 7f2264e00640 1 --2- 172.21.15.89:0/617878400 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f22480a2b80 0x7f22480a5040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2260199260 tx=0x7f224c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.635+0000 7f2265800640 1 -- 172.21.15.89:0/1532857677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600f5b2dba0 msgr2=0x5600f5b32660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.635+0000 7f2265800640 1 --2- 172.21.15.89:0/1532857677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600f5b2dba0 0x5600f5b32660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.635+0000 7f2265800640 1 -- 172.21.15.89:0/1532857677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b32c30 msgr2=0x5600f5b350f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.635+0000 7f2265800640 1 --2- 172.21.15.89:0/1532857677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b32c30 0x5600f5b350f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.635+0000 7f2265800640 1 -- 172.21.15.89:0/1532857677 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600f57a5210 con 0x5600f5b2d260 2024-09-10T08:41:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.635+0000 7f2265800640 1 --2- 172.21.15.89:0/1532857677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b2d260 0x5600f5b2d660 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2254009610 tx=0x7f22540a4050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ef975625b2f2076 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.636+0000 7f2252a00640 1 -- 172.21.15.89:0/1532857677 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22540ac500 con 0x5600f5b2d260 2024-09-10T08:41:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.636+0000 7f2252a00640 1 -- 172.21.15.89:0/1532857677 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22540a1b90 con 0x5600f5b2d260 2024-09-10T08:41:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.636+0000 7f2252a00640 1 -- 172.21.15.89:0/1532857677 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22540ac070 con 0x5600f5b2d260 2024-09-10T08:41:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.637+0000 7f226aff3740 1 -- 172.21.15.89:0/1532857677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b2d260 msgr2=0x5600f5b2d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.637+0000 7f226aff3740 1 --2- 172.21.15.89:0/1532857677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b2d260 0x5600f5b2d660 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2254009610 tx=0x7f22540a4050 comp rx=0 tx=0).stop 2024-09-10T08:41:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.637+0000 7f226aff3740 1 -- 172.21.15.89:0/1532857677 shutdown_connections 2024-09-10T08:41:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.637+0000 7f226aff3740 1 --2- 172.21.15.89:0/1532857677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b32c30 0x5600f5b350f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.637+0000 7f226aff3740 1 --2- 172.21.15.89:0/1532857677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600f5b2dba0 0x5600f5b32660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.637+0000 7f226aff3740 1 --2- 172.21.15.89:0/1532857677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b2d260 0x5600f5b2d660 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.637+0000 7f226aff3740 1 -- 172.21.15.89:0/1532857677 >> 172.21.15.89:0/1532857677 conn(0x5600f5b288d0 msgr2=0x5600f5b28cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.637+0000 7f226aff3740 1 -- 172.21.15.89:0/1532857677 shutdown_connections 2024-09-10T08:41:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.637+0000 7f226aff3740 1 -- 172.21.15.89:0/1532857677 wait complete. 2024-09-10T08:41:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.637+0000 7f226aff3740 1 -- start start 2024-09-10T08:41:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.638+0000 7f226aff3740 1 Processor -- start 2024-09-10T08:41:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.638+0000 7f226aff3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b2d260 0x5600f5b37560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.638+0000 7f226aff3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600f5b2dba0 0x5600f5b37aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.638+0000 7f226aff3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b32c30 0x5600f5b3eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.638+0000 7f226aff3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5600f5b35c00 con 0x5600f5b2d260 2024-09-10T08:41:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.638+0000 7f226aff3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5600f5a1b090 con 0x5600f5b32c30 2024-09-10T08:41:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.638+0000 7f226aff3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5600f5b35d80 con 0x5600f5b2dba0 2024-09-10T08:41:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.638+0000 7f226aff3740 10 client.0 _ll_register_callbacks cb 0x5600f5a6ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.639+0000 7f2265800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b2d260 0x5600f5b37560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.639+0000 7f2265800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b2d260 0x5600f5b37560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47826/0 (socket says 172.21.15.89:47826) 2024-09-10T08:41:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.639+0000 7f2265800640 1 -- 172.21.15.89:0/902425230 learned_addr learned my addr 172.21.15.89:0/902425230 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.639+0000 7f2266200640 1 --2- 172.21.15.89:0/902425230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b32c30 0x5600f5b3eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.639+0000 7f2265800640 1 -- 172.21.15.89:0/902425230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600f5b2dba0 msgr2=0x5600f5b37aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.639+0000 7f2265800640 1 --2- 172.21.15.89:0/902425230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600f5b2dba0 0x5600f5b37aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.639+0000 7f2265800640 1 -- 172.21.15.89:0/902425230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b32c30 msgr2=0x5600f5b3eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.639+0000 7f2265800640 1 --2- 172.21.15.89:0/902425230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b32c30 0x5600f5b3eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.639+0000 7f2265800640 1 -- 172.21.15.89:0/902425230 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600f5871620 con 0x5600f5b2d260 2024-09-10T08:41:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.640+0000 7f2265800640 1 --2- 172.21.15.89:0/902425230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b2d260 0x5600f5b37560 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f225409fca0 tx=0x7f22540d70c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8907174abcfa3322 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.640+0000 7f2247400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:41:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.640+0000 7f2247400640 1 -- 172.21.15.89:0/902425230 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225408e0f0 con 0x5600f5b2d260 2024-09-10T08:41:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.640+0000 7f226aff3740 1 -- 172.21.15.89:0/902425230 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22540a1b90 con 0x5600f5b2d260 2024-09-10T08:41:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.640+0000 7f2247400640 1 -- 172.21.15.89:0/902425230 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f22540ac070 con 0x5600f5b2d260 2024-09-10T08:41:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.640+0000 7f2247400640 1 -- 172.21.15.89:0/902425230 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f225408e290 con 0x5600f5b2d260 2024-09-10T08:41:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.640+0000 7f226aff3740 10 client.15579 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.640+0000 7f226aff3740 1 -- 172.21.15.89:0/902425230 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5600f5b418d0 con 0x5600f5b2d260 2024-09-10T08:41:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.641+0000 7f2247400640 1 -- 172.21.15.89:0/902425230 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22540a1b90 con 0x5600f5b2d260 2024-09-10T08:41:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.641+0000 7f2247400640 1 -- 172.21.15.89:0/902425230 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f22540a2ce0 con 0x5600f5b2d260 2024-09-10T08:41:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.641+0000 7f226aff3740 10 client.15579 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.641+0000 7f226aff3740 10 client.15579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:41:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.641+0000 7f226aff3740 20 client.15579 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.641+0000 7f226aff3740 1 --2- 172.21.15.89:0/902425230 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5600f5b42c20 0x5600f5b45010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.641+0000 7f226aff3740 4 client.15579 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:41:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.641+0000 7f226aff3740 1 -- 172.21.15.89:0/902425230 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600f5a9f830 con 0x5600f5b42c20 2024-09-10T08:41:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.641+0000 7f2264e00640 1 --2- 172.21.15.89:0/902425230 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5600f5b42c20 0x5600f5b45010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.642+0000 7f2264e00640 1 --2- 172.21.15.89:0/902425230 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5600f5b42c20 0x5600f5b45010 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=543f05ad1a208805 server_cookie=750272e0937fd6a9 in_seq=0 out_seq=0 2024-09-10T08:41:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.642+0000 7f2247400640 10 client.15579 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.643+0000 7f2247400640 1 -- 172.21.15.89:0/902425230 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5600f5a9f830 con 0x5600f5b42c20 2024-09-10T08:41:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.643+0000 7f2247400640 10 client.15579 handle_command_reply: tid=0 2024-09-10T08:41:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.643+0000 7f226aff3740 1 client.15579 shutdown 2024-09-10T08:41:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.643+0000 7f226aff3740 1 -- 172.21.15.89:0/902425230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b2d260 msgr2=0x5600f5b37560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.643+0000 7f226aff3740 1 --2- 172.21.15.89:0/902425230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b2d260 0x5600f5b37560 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f225409fca0 tx=0x7f22540d70c0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.644+0000 7f226aff3740 1 -- 172.21.15.89:0/902425230 shutdown_connections 2024-09-10T08:41:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.644+0000 7f226aff3740 1 --2- 172.21.15.89:0/902425230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b32c30 0x5600f5b3eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.644+0000 7f226aff3740 1 --2- 172.21.15.89:0/902425230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600f5b2dba0 0x5600f5b37aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.644+0000 7f226aff3740 1 --2- 172.21.15.89:0/902425230 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5600f5b42c20 0x5600f5b45010 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.644+0000 7f226aff3740 1 --2- 172.21.15.89:0/902425230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b2d260 0x5600f5b37560 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.644+0000 7f2247400640 0 client.15579 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.644+0000 7f226aff3740 1 -- 172.21.15.89:0/902425230 >> 172.21.15.89:0/902425230 conn(0x5600f5b288d0 msgr2=0x5600f5b2a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.644+0000 7f226aff3740 1 -- 172.21.15.89:0/902425230 shutdown_connections 2024-09-10T08:41:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.644+0000 7f226aff3740 1 -- 172.21.15.89:0/902425230 wait complete. 2024-09-10T08:41:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.644+0000 7f226aff3740 20 client.15579 trim_cache size 0 max 16384 2024-09-10T08:41:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.672+0000 7f226aff3740 1 Processor -- start 2024-09-10T08:41:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.672+0000 7f226aff3740 1 -- start start 2024-09-10T08:41:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.673+0000 7f226aff3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b2d260 0x5600f5b37310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.673+0000 7f226aff3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b2dba0 0x5600f5b3ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.673+0000 7f226aff3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600f5b32c30 0x5600f5b37850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.673+0000 7f226aff3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f22601aae90 con 0x5600f5b2d260 2024-09-10T08:41:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.673+0000 7f226aff3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f22601a8760 con 0x5600f5b2dba0 2024-09-10T08:41:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.673+0000 7f226aff3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f22601a65b0 con 0x5600f5b32c30 2024-09-10T08:41:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.673+0000 7f2264e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b2dba0 0x5600f5b3ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.673+0000 7f2265800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b2d260 0x5600f5b37310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.673+0000 7f2264e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b2dba0 0x5600f5b3ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:58366/0 (socket says 172.21.15.89:58366) 2024-09-10T08:41:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.673+0000 7f2264e00640 1 -- 172.21.15.89:0/2490793971 learned_addr learned my addr 172.21.15.89:0/2490793971 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.673+0000 7f2266200640 1 --2- 172.21.15.89:0/2490793971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600f5b32c30 0x5600f5b37850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.674+0000 7f2264e00640 1 -- 172.21.15.89:0/2490793971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600f5b32c30 msgr2=0x5600f5b37850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.674+0000 7f2264e00640 1 --2- 172.21.15.89:0/2490793971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600f5b32c30 0x5600f5b37850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.674+0000 7f2264e00640 1 -- 172.21.15.89:0/2490793971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b2d260 msgr2=0x5600f5b37310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.674+0000 7f2264e00640 1 --2- 172.21.15.89:0/2490793971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b2d260 0x5600f5b37310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.674+0000 7f2264e00640 1 -- 172.21.15.89:0/2490793971 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600f5b286f0 con 0x5600f5b2dba0 2024-09-10T08:41:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.674+0000 7f2265800640 1 --2- 172.21.15.89:0/2490793971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b2d260 0x5600f5b37310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.674+0000 7f2264e00640 1 --2- 172.21.15.89:0/2490793971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b2dba0 0x5600f5b3ea00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f224c013010 tx=0x7f224c013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=549fc2e6b6ebe92d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.675+0000 7f2252000640 1 -- 172.21.15.89:0/2490793971 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c026330 con 0x5600f5b2dba0 2024-09-10T08:41:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.675+0000 7f2252000640 1 -- 172.21.15.89:0/2490793971 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f224c02c070 con 0x5600f5b2dba0 2024-09-10T08:41:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.675+0000 7f2252000640 1 -- 172.21.15.89:0/2490793971 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c01d970 con 0x5600f5b2dba0 2024-09-10T08:41:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.676+0000 7f226aff3740 1 -- 172.21.15.89:0/2490793971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b2dba0 msgr2=0x5600f5b3ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.676+0000 7f226aff3740 1 --2- 172.21.15.89:0/2490793971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b2dba0 0x5600f5b3ea00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f224c013010 tx=0x7f224c013040 comp rx=0 tx=0).stop 2024-09-10T08:41:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.676+0000 7f226aff3740 1 -- 172.21.15.89:0/2490793971 shutdown_connections 2024-09-10T08:41:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.676+0000 7f226aff3740 1 --2- 172.21.15.89:0/2490793971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600f5b32c30 0x5600f5b37850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.676+0000 7f226aff3740 1 --2- 172.21.15.89:0/2490793971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b2dba0 0x5600f5b3ea00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.676+0000 7f226aff3740 1 --2- 172.21.15.89:0/2490793971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b2d260 0x5600f5b37310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.676+0000 7f226aff3740 1 -- 172.21.15.89:0/2490793971 >> 172.21.15.89:0/2490793971 conn(0x5600f5b288d0 msgr2=0x5600f5b2ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.676+0000 7f226aff3740 1 -- 172.21.15.89:0/2490793971 shutdown_connections 2024-09-10T08:41:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.676+0000 7f226aff3740 1 -- 172.21.15.89:0/2490793971 wait complete. 2024-09-10T08:41:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.676+0000 7f226aff3740 1 -- start start 2024-09-10T08:41:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.677+0000 7f226aff3740 1 Processor -- start 2024-09-10T08:41:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.677+0000 7f226aff3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b2d260 0x5600f5b3eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.677+0000 7f226aff3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b2dba0 0x5600f5b3b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.677+0000 7f226aff3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600f5b32c30 0x5600f5b3bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.677+0000 7f226aff3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f22601a6c20 con 0x5600f5b2d260 2024-09-10T08:41:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.677+0000 7f226aff3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f22601a8090 con 0x5600f5b2dba0 2024-09-10T08:41:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.677+0000 7f226aff3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f22601a72f0 con 0x5600f5b32c30 2024-09-10T08:41:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.677+0000 7f226aff3740 10 client.0 _ll_register_callbacks cb 0x7f224c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.678+0000 7f2264e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b2dba0 0x5600f5b3b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.678+0000 7f2266200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600f5b32c30 0x5600f5b3bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.678+0000 7f2264e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b2dba0 0x5600f5b3b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:58374/0 (socket says 172.21.15.89:58374) 2024-09-10T08:41:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.678+0000 7f2264e00640 1 -- 172.21.15.89:0/1384743451 learned_addr learned my addr 172.21.15.89:0/1384743451 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.678+0000 7f2265800640 1 --2- 172.21.15.89:0/1384743451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b2d260 0x5600f5b3eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.678+0000 7f2264e00640 1 -- 172.21.15.89:0/1384743451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600f5b32c30 msgr2=0x5600f5b3bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.678+0000 7f2264e00640 1 --2- 172.21.15.89:0/1384743451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600f5b32c30 0x5600f5b3bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.678+0000 7f2264e00640 1 -- 172.21.15.89:0/1384743451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b2d260 msgr2=0x5600f5b3eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.678+0000 7f2264e00640 1 --2- 172.21.15.89:0/1384743451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b2d260 0x5600f5b3eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.678+0000 7f2264e00640 1 -- 172.21.15.89:0/1384743451 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600f5bab7e0 con 0x5600f5b2dba0 2024-09-10T08:41:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.678+0000 7f2265800640 1 --2- 172.21.15.89:0/1384743451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b2d260 0x5600f5b3eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:41:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.679+0000 7f2264e00640 1 --2- 172.21.15.89:0/1384743451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b2dba0 0x5600f5b3b5a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f224c009730 tx=0x7f224c005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48c82ac9f7c30da5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.679+0000 7f2247400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:41:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.679+0000 7f2247400640 1 -- 172.21.15.89:0/1384743451 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c004020 con 0x5600f5b2dba0 2024-09-10T08:41:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.679+0000 7f2247400640 1 -- 172.21.15.89:0/1384743451 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f224c0041c0 con 0x5600f5b2dba0 2024-09-10T08:41:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.680+0000 7f226aff3740 1 -- 172.21.15.89:0/1384743451 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f224c02c070 con 0x5600f5b2dba0 2024-09-10T08:41:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.680+0000 7f2247400640 1 -- 172.21.15.89:0/1384743451 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f224c0043c0 con 0x5600f5b2dba0 2024-09-10T08:41:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.680+0000 7f226aff3740 10 client.25649 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.680+0000 7f226aff3740 1 -- 172.21.15.89:0/1384743451 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5600f5bacac0 con 0x5600f5b2dba0 2024-09-10T08:41:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.680+0000 7f2247400640 1 -- 172.21.15.89:0/1384743451 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f224c02c070 con 0x5600f5b2dba0 2024-09-10T08:41:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.680+0000 7f2247400640 1 -- 172.21.15.89:0/1384743451 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f224c003b90 con 0x5600f5b2dba0 2024-09-10T08:41:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.681+0000 7f226aff3740 10 client.25649 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.681+0000 7f226aff3740 10 client.25649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:41:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.681+0000 7f226aff3740 20 client.25649 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.681+0000 7f226aff3740 1 --2- 172.21.15.89:0/1384743451 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5600f5bad450 0x5600f5baf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.681+0000 7f226aff3740 4 client.25649 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:41:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.681+0000 7f226aff3740 1 -- 172.21.15.89:0/1384743451 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5600f5a99b80 con 0x5600f5bad450 2024-09-10T08:41:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.681+0000 7f2265800640 1 --2- 172.21.15.89:0/1384743451 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5600f5bad450 0x5600f5baf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.682+0000 7f2265800640 1 --2- 172.21.15.89:0/1384743451 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5600f5bad450 0x5600f5baf870 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8937849012e459eb server_cookie=f3285bf7f0be493e in_seq=0 out_seq=0 2024-09-10T08:41:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.682+0000 7f2247400640 10 client.25649 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.687+0000 7f2247400640 1 -- 172.21.15.89:0/1384743451 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5600f5a99b80 con 0x5600f5bad450 2024-09-10T08:41:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.687+0000 7f2247400640 10 client.25649 handle_command_reply: tid=0 2024-09-10T08:41:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.687+0000 7f226aff3740 1 client.25649 shutdown 2024-09-10T08:41:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.687+0000 7f226aff3740 1 -- 172.21.15.89:0/1384743451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b2dba0 msgr2=0x5600f5b3b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.687+0000 7f226aff3740 1 --2- 172.21.15.89:0/1384743451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b2dba0 0x5600f5b3b5a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f224c009730 tx=0x7f224c005760 comp rx=0 tx=0).stop 2024-09-10T08:41:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.687+0000 7f226aff3740 1 -- 172.21.15.89:0/1384743451 shutdown_connections 2024-09-10T08:41:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.687+0000 7f226aff3740 1 --2- 172.21.15.89:0/1384743451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600f5b32c30 0x5600f5b3bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.688+0000 7f226aff3740 1 --2- 172.21.15.89:0/1384743451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5600f5b2dba0 0x5600f5b3b5a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.688+0000 7f226aff3740 1 --2- 172.21.15.89:0/1384743451 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5600f5bad450 0x5600f5baf870 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.688+0000 7f226aff3740 1 --2- 172.21.15.89:0/1384743451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5600f5b2d260 0x5600f5b3eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.688+0000 7f226aff3740 1 -- 172.21.15.89:0/1384743451 >> 172.21.15.89:0/1384743451 conn(0x5600f5b288d0 msgr2=0x5600f5b372e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.688+0000 7f2247400640 0 client.25649 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.688+0000 7f226aff3740 1 -- 172.21.15.89:0/1384743451 shutdown_connections 2024-09-10T08:41:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.688+0000 7f226aff3740 1 -- 172.21.15.89:0/1384743451 wait complete. 2024-09-10T08:41:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.688+0000 7f226aff3740 20 client.25649 trim_cache size 0 max 16384 2024-09-10T08:41:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:41:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:41:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:41:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:41:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.689+0000 7f2266c00640 1 -- 172.21.15.89:0/617878400 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f22480a2b80 msgr2=0x7f22480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.689+0000 7f2266c00640 1 --2- 172.21.15.89:0/617878400 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f22480a2b80 0x7f22480a5040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2260199260 tx=0x7f224c005f90 comp rx=0 tx=0).stop 2024-09-10T08:41:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.689+0000 7f2266c00640 1 -- 172.21.15.89:0/617878400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2260101580 msgr2=0x7f2260197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.689+0000 7f2266c00640 1 --2- 172.21.15.89:0/617878400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2260101580 0x7f2260197d50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f225400d770 tx=0x7f225400dc30 comp rx=0 tx=0).stop 2024-09-10T08:41:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.689+0000 7f2266c00640 1 -- 172.21.15.89:0/617878400 shutdown_connections 2024-09-10T08:41:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.689+0000 7f2266c00640 1 --2- 172.21.15.89:0/617878400 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f22480a2b80 0x7f22480a5040 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.689+0000 7f2266c00640 1 --2- 172.21.15.89:0/617878400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2260103050 0x7f226019f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.689+0000 7f2266c00640 1 --2- 172.21.15.89:0/617878400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2260101f40 0x7f2260198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.689+0000 7f2266c00640 1 --2- 172.21.15.89:0/617878400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2260101580 0x7f2260197d50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.689+0000 7f2266c00640 1 -- 172.21.15.89:0/617878400 >> 172.21.15.89:0/617878400 conn(0x7f22600fd0d0 msgr2=0x7f2260105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.690+0000 7f2266c00640 1 -- 172.21.15.89:0/617878400 shutdown_connections 2024-09-10T08:41:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:08.690+0000 7f2266c00640 1 -- 172.21.15.89:0/617878400 wait complete. 2024-09-10T08:41:09.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:08 smithi111 ceph-mon[35220]: pgmap v429: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T08:41:09.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:08 smithi089 ceph-mon[30757]: pgmap v429: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T08:41:09.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:08 smithi149 ceph-mon[36167]: pgmap v429: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T08:41:10.709 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:41:11.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:10 smithi111 ceph-mon[35220]: pgmap v430: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T08:41:11.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:10 smithi089 ceph-mon[30757]: pgmap v430: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T08:41:11.091 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:41:11.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:10 smithi149 ceph-mon[36167]: pgmap v430: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T08:41:13.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:12 smithi111 ceph-mon[35220]: pgmap v431: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-10T08:41:13.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:12 smithi089 ceph-mon[30757]: pgmap v431: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-10T08:41:13.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:12 smithi149 ceph-mon[36167]: pgmap v431: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-10T08:41:14.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.349+0000 7f3b35600640 1 -- 172.21.15.89:0/3066770287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b30104010 msgr2=0x7f3b301043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:14.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.349+0000 7f3b35600640 1 --2- 172.21.15.89:0/3066770287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b30104010 0x7f3b301043f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009900 tx=0x7f3b2002a410 comp rx=0 tx=0).stop 2024-09-10T08:41:14.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.350+0000 7f3b35600640 1 -- 172.21.15.89:0/3066770287 shutdown_connections 2024-09-10T08:41:14.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.350+0000 7f3b35600640 1 --2- 172.21.15.89:0/3066770287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b30109140 0x7f3b30110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.350+0000 7f3b35600640 1 --2- 172.21.15.89:0/3066770287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b301049c0 0x7f3b30108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.350+0000 7f3b35600640 1 --2- 172.21.15.89:0/3066770287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b30104010 0x7f3b301043f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.350+0000 7f3b35600640 1 -- 172.21.15.89:0/3066770287 >> 172.21.15.89:0/3066770287 conn(0x7f3b300ffa50 msgr2=0x7f3b30101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:14.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.350+0000 7f3b35600640 1 -- 172.21.15.89:0/3066770287 shutdown_connections 2024-09-10T08:41:14.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.350+0000 7f3b35600640 1 -- 172.21.15.89:0/3066770287 wait complete. 2024-09-10T08:41:14.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.351+0000 7f3b35600640 1 Processor -- start 2024-09-10T08:41:14.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.351+0000 7f3b35600640 1 -- start start 2024-09-10T08:41:14.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.351+0000 7f3b35600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b30104010 0x7f3b3019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.351+0000 7f3b35600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b301049c0 0x7f3b3019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.351+0000 7f3b35600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b30109140 0x7f3b3019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.351+0000 7f3b35600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3b30114190 con 0x7f3b301049c0 2024-09-10T08:41:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.351+0000 7f3b35600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3b30114010 con 0x7f3b30104010 2024-09-10T08:41:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.351+0000 7f3b35600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3b30114310 con 0x7f3b30109140 2024-09-10T08:41:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.351+0000 7f3b27e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b301049c0 0x7f3b3019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.351+0000 7f3b27e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b301049c0 0x7f3b3019abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38394/0 (socket says 172.21.15.89:38394) 2024-09-10T08:41:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.352+0000 7f3b27e00640 1 -- 172.21.15.89:0/538452113 learned_addr learned my addr 172.21.15.89:0/538452113 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.352+0000 7f3b2f400640 1 --2- 172.21.15.89:0/538452113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b30109140 0x7f3b3019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.352+0000 7f3b2ea00640 1 --2- 172.21.15.89:0/538452113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b30104010 0x7f3b3019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.352+0000 7f3b2ea00640 1 -- 172.21.15.89:0/538452113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b30109140 msgr2=0x7f3b3019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:14.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.352+0000 7f3b2ea00640 1 --2- 172.21.15.89:0/538452113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b30109140 0x7f3b3019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.352+0000 7f3b2ea00640 1 -- 172.21.15.89:0/538452113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b301049c0 msgr2=0x7f3b3019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.352+0000 7f3b2ea00640 1 --2- 172.21.15.89:0/538452113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b301049c0 0x7f3b3019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.352+0000 7f3b2ea00640 1 -- 172.21.15.89:0/538452113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3019f660 con 0x7f3b30104010 2024-09-10T08:41:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.352+0000 7f3b27e00640 1 --2- 172.21.15.89:0/538452113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b301049c0 0x7f3b3019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.352+0000 7f3b2ea00640 1 --2- 172.21.15.89:0/538452113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b30104010 0x7f3b3019a6b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3b2002a930 tx=0x7f3b20003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.353+0000 7f3b27400640 1 -- 172.21.15.89:0/538452113 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2003f070 con 0x7f3b30104010 2024-09-10T08:41:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.353+0000 7f3b27400640 1 -- 172.21.15.89:0/538452113 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b20031c90 con 0x7f3b30104010 2024-09-10T08:41:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.353+0000 7f3b27400640 1 -- 172.21.15.89:0/538452113 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b20035030 con 0x7f3b30104010 2024-09-10T08:41:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.353+0000 7f3b35600640 1 -- 172.21.15.89:0/538452113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3019f8f0 con 0x7f3b30104010 2024-09-10T08:41:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.353+0000 7f3b35600640 1 -- 172.21.15.89:0/538452113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b301a7230 con 0x7f3b30104010 2024-09-10T08:41:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.356+0000 7f3b27400640 1 -- 172.21.15.89:0/538452113 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3b20042070 con 0x7f3b30104010 2024-09-10T08:41:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.356+0000 7f3b381d0740 1 Processor -- start 2024-09-10T08:41:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.356+0000 7f3b381d0740 1 -- start start 2024-09-10T08:41:14.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.356+0000 7f3b381d0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d30227b90 0x563d30227f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.356+0000 7f3b381d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d302284b0 0x563d30232830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.356+0000 7f3b381d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30232d70 0x563d30235160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.356+0000 7f3b381d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563d3018e3e0 con 0x563d30232d70 2024-09-10T08:41:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.356+0000 7f3b381d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563d30135d80 con 0x563d302284b0 2024-09-10T08:41:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.356+0000 7f3b381d0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563d3018a3a0 con 0x563d30227b90 2024-09-10T08:41:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.357+0000 7f3b27e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30232d70 0x563d30235160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.357+0000 7f3b27e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30232d70 0x563d30235160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38396/0 (socket says 172.21.15.89:38396) 2024-09-10T08:41:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.357+0000 7f3b27e00640 1 -- 172.21.15.89:0/1658947265 learned_addr learned my addr 172.21.15.89:0/1658947265 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.357+0000 7f3b2f400640 1 --2- 172.21.15.89:0/1658947265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d30227b90 0x563d30227f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.357+0000 7f3b2ea00640 1 --2- 172.21.15.89:0/1658947265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d302284b0 0x563d30232830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:14.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.357+0000 7f3b27e00640 1 -- 172.21.15.89:0/1658947265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d30227b90 msgr2=0x563d30227f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.357+0000 7f3b27e00640 1 --2- 172.21.15.89:0/1658947265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d30227b90 0x563d30227f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.357+0000 7f3b27e00640 1 -- 172.21.15.89:0/1658947265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d302284b0 msgr2=0x563d30232830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.357+0000 7f3b27e00640 1 --2- 172.21.15.89:0/1658947265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d302284b0 0x563d30232830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.357+0000 7f3b27e00640 1 -- 172.21.15.89:0/1658947265 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d30154ae0 con 0x563d30232d70 2024-09-10T08:41:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.357+0000 7f3b2ea00640 1 --2- 172.21.15.89:0/1658947265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d302284b0 0x563d30232830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:41:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.357+0000 7f3b27e00640 1 --2- 172.21.15.89:0/1658947265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30232d70 0x563d30235160 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3b180115f0 tx=0x7f3b18011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3448397012493a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.358+0000 7f3b1f400640 1 -- 172.21.15.89:0/1658947265 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1801e070 con 0x563d30232d70 2024-09-10T08:41:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.358+0000 7f3b1f400640 1 -- 172.21.15.89:0/1658947265 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b1800f660 con 0x563d30232d70 2024-09-10T08:41:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.358+0000 7f3b1f400640 1 -- 172.21.15.89:0/1658947265 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18003830 con 0x563d30232d70 2024-09-10T08:41:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.358+0000 7f3b27400640 1 --2- 172.21.15.89:0/538452113 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3b140a2c00 0x7f3b140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.358+0000 7f3b27400640 1 -- 172.21.15.89:0/538452113 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f3b200efcd0 con 0x7f3b30104010 2024-09-10T08:41:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.358+0000 7f3b2f400640 1 --2- 172.21.15.89:0/538452113 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3b140a2c00 0x7f3b140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:14.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.358+0000 7f3b381d0740 1 -- 172.21.15.89:0/1658947265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30232d70 msgr2=0x563d30235160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.358+0000 7f3b381d0740 1 --2- 172.21.15.89:0/1658947265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30232d70 0x563d30235160 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3b180115f0 tx=0x7f3b18011ab0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.359+0000 7f3b381d0740 1 -- 172.21.15.89:0/1658947265 shutdown_connections 2024-09-10T08:41:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.359+0000 7f3b381d0740 1 --2- 172.21.15.89:0/1658947265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30232d70 0x563d30235160 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.359+0000 7f3b381d0740 1 --2- 172.21.15.89:0/1658947265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d302284b0 0x563d30232830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.359+0000 7f3b381d0740 1 --2- 172.21.15.89:0/1658947265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d30227b90 0x563d30227f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.359+0000 7f3b381d0740 1 -- 172.21.15.89:0/1658947265 >> 172.21.15.89:0/1658947265 conn(0x563d30224120 msgr2=0x563d30224500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.359+0000 7f3b381d0740 1 -- 172.21.15.89:0/1658947265 shutdown_connections 2024-09-10T08:41:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.359+0000 7f3b2f400640 1 --2- 172.21.15.89:0/538452113 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3b140a2c00 0x7f3b140a50c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3b28005fd0 tx=0x7f3b28009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.359+0000 7f3b381d0740 1 -- 172.21.15.89:0/1658947265 wait complete. 2024-09-10T08:41:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.359+0000 7f3b381d0740 1 -- start start 2024-09-10T08:41:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.360+0000 7f3b381d0740 1 Processor -- start 2024-09-10T08:41:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.360+0000 7f3b381d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30227b90 0x563d3022fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.360+0000 7f3b381d0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d302284b0 0x563d30230120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:14.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.360+0000 7f3b381d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d30232d70 0x563d30230660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.360+0000 7f3b381d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563d302359e0 con 0x563d30227b90 2024-09-10T08:41:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.360+0000 7f3b381d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563d301490f0 con 0x563d30232d70 2024-09-10T08:41:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.360+0000 7f3b381d0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563d30235b60 con 0x563d302284b0 2024-09-10T08:41:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.360+0000 7f3b381d0740 10 client.0 _ll_register_callbacks cb 0x563d3013a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.360+0000 7f3b2ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30227b90 0x563d3022fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.360+0000 7f3b2f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d30232d70 0x563d30230660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.360+0000 7f3b2ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30227b90 0x563d3022fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38404/0 (socket says 172.21.15.89:38404) 2024-09-10T08:41:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.360+0000 7f3b2ea00640 1 -- 172.21.15.89:0/1974157767 learned_addr learned my addr 172.21.15.89:0/1974157767 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.360+0000 7f3b27e00640 1 --2- 172.21.15.89:0/1974157767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d302284b0 0x563d30230120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.361+0000 7f3b2ea00640 1 -- 172.21.15.89:0/1974157767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d302284b0 msgr2=0x563d30230120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.361+0000 7f3b2ea00640 1 --2- 172.21.15.89:0/1974157767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d302284b0 0x563d30230120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.361+0000 7f3b2ea00640 1 -- 172.21.15.89:0/1974157767 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d30232d70 msgr2=0x563d30230660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:14.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.361+0000 7f3b2ea00640 1 --2- 172.21.15.89:0/1974157767 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d30232d70 0x563d30230660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.361+0000 7f3b2ea00640 1 -- 172.21.15.89:0/1974157767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d2feff3c0 con 0x563d30227b90 2024-09-10T08:41:14.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.361+0000 7f3b2ea00640 1 --2- 172.21.15.89:0/1974157767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30227b90 0x563d3022fbe0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3b200c4840 tx=0x7f3b20031490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c05c9cc8ff09aa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:14.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.361+0000 7f3b13e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:41:14.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.361+0000 7f3b13e00640 1 -- 172.21.15.89:0/1974157767 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b20049070 con 0x563d30227b90 2024-09-10T08:41:14.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.362+0000 7f3b13e00640 1 -- 172.21.15.89:0/1974157767 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f3b200dc0b0 con 0x563d30227b90 2024-09-10T08:41:14.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.362+0000 7f3b381d0740 1 -- 172.21.15.89:0/1974157767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b1800f660 con 0x563d30227b90 2024-09-10T08:41:14.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.362+0000 7f3b13e00640 1 -- 172.21.15.89:0/1974157767 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3b20031e40 con 0x563d30227b90 2024-09-10T08:41:14.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.362+0000 7f3b381d0740 10 client.15597 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:14.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.362+0000 7f3b381d0740 1 -- 172.21.15.89:0/1974157767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563d30241640 con 0x563d30227b90 2024-09-10T08:41:14.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.362+0000 7f3b13e00640 1 -- 172.21.15.89:0/1974157767 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b1800f660 con 0x563d30227b90 2024-09-10T08:41:14.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.362+0000 7f3b13e00640 1 -- 172.21.15.89:0/1974157767 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3b200b8820 con 0x563d30227b90 2024-09-10T08:41:14.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.362+0000 7f3b381d0740 10 client.15597 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:14.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.362+0000 7f3b381d0740 10 client.15597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:41:14.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.362+0000 7f3b381d0740 20 client.15597 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:14.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.362+0000 7f3b381d0740 1 --2- 172.21.15.89:0/1974157767 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563d30242a30 0x563d30244e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:14.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.362+0000 7f3b381d0740 4 client.15597 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:41:14.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.362+0000 7f3b381d0740 1 -- 172.21.15.89:0/1974157767 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d3019b950 con 0x563d30242a30 2024-09-10T08:41:14.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.363+0000 7f3b27e00640 1 --2- 172.21.15.89:0/1974157767 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563d30242a30 0x563d30244e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:14.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.363+0000 7f3b27e00640 1 --2- 172.21.15.89:0/1974157767 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563d30242a30 0x563d30244e80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dd9faee8c6beae5 server_cookie=c4f5fce8cf896f3a in_seq=0 out_seq=0 2024-09-10T08:41:14.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.363+0000 7f3b13e00640 10 client.15597 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:14.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.364+0000 7f3b13e00640 1 -- 172.21.15.89:0/1974157767 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d3019b950 con 0x563d30242a30 2024-09-10T08:41:14.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.364+0000 7f3b13e00640 10 client.15597 handle_command_reply: tid=0 2024-09-10T08:41:14.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.364+0000 7f3b381d0740 1 client.15597 shutdown 2024-09-10T08:41:14.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.364+0000 7f3b381d0740 1 -- 172.21.15.89:0/1974157767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30227b90 msgr2=0x563d3022fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:14.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.364+0000 7f3b381d0740 1 --2- 172.21.15.89:0/1974157767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30227b90 0x563d3022fbe0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3b200c4840 tx=0x7f3b20031490 comp rx=0 tx=0).stop 2024-09-10T08:41:14.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.364+0000 7f3b381d0740 1 -- 172.21.15.89:0/1974157767 shutdown_connections 2024-09-10T08:41:14.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.364+0000 7f3b381d0740 1 --2- 172.21.15.89:0/1974157767 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d30232d70 0x563d30230660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.364+0000 7f3b381d0740 1 --2- 172.21.15.89:0/1974157767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d302284b0 0x563d30230120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.364+0000 7f3b381d0740 1 --2- 172.21.15.89:0/1974157767 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563d30242a30 0x563d30244e80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.364+0000 7f3b381d0740 1 --2- 172.21.15.89:0/1974157767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30227b90 0x563d3022fbe0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.364+0000 7f3b381d0740 1 -- 172.21.15.89:0/1974157767 >> 172.21.15.89:0/1974157767 conn(0x563d30224120 msgr2=0x563d30229e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:14.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.365+0000 7f3b13e00640 0 client.15597 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:14.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.365+0000 7f3b381d0740 1 -- 172.21.15.89:0/1974157767 shutdown_connections 2024-09-10T08:41:14.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.365+0000 7f3b381d0740 1 -- 172.21.15.89:0/1974157767 wait complete. 2024-09-10T08:41:14.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.365+0000 7f3b381d0740 20 client.15597 trim_cache size 0 max 16384 2024-09-10T08:41:14.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.398+0000 7f3b381d0740 1 Processor -- start 2024-09-10T08:41:14.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.399+0000 7f3b381d0740 1 -- start start 2024-09-10T08:41:14.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.399+0000 7f3b381d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30227b90 0x563d3022a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:14.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.399+0000 7f3b381d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d302284b0 0x563d30231b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:14.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.399+0000 7f3b381d0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d30232d70 0x563d3024a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:14.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.399+0000 7f3b381d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3b301b3820 con 0x563d30227b90 2024-09-10T08:41:14.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.399+0000 7f3b381d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3b301b10f0 con 0x563d302284b0 2024-09-10T08:41:14.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.399+0000 7f3b381d0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3b301aef40 con 0x563d30232d70 2024-09-10T08:41:14.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.399+0000 7f3b2ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30227b90 0x563d3022a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:14.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.399+0000 7f3b2ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30227b90 0x563d3022a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38418/0 (socket says 172.21.15.89:38418) 2024-09-10T08:41:14.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.399+0000 7f3b2ea00640 1 -- 172.21.15.89:0/1449191491 learned_addr learned my addr 172.21.15.89:0/1449191491 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:14.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.399+0000 7f3b27e00640 1 --2- 172.21.15.89:0/1449191491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d302284b0 0x563d30231b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:14.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.399+0000 7f3b2f400640 1 --2- 172.21.15.89:0/1449191491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d30232d70 0x563d3024a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:14.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.400+0000 7f3b2ea00640 1 -- 172.21.15.89:0/1449191491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d30232d70 msgr2=0x563d3024a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:14.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.400+0000 7f3b2ea00640 1 --2- 172.21.15.89:0/1449191491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d30232d70 0x563d3024a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.400+0000 7f3b2ea00640 1 -- 172.21.15.89:0/1449191491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d302284b0 msgr2=0x563d30231b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:14.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.400+0000 7f3b2ea00640 1 --2- 172.21.15.89:0/1449191491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d302284b0 0x563d30231b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.400+0000 7f3b2ea00640 1 -- 172.21.15.89:0/1449191491 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d30223bc0 con 0x563d30227b90 2024-09-10T08:41:14.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.400+0000 7f3b2f400640 1 --2- 172.21.15.89:0/1449191491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d30232d70 0x563d3024a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:41:14.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.400+0000 7f3b2ea00640 1 --2- 172.21.15.89:0/1449191491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30227b90 0x563d3022a5f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3b20046040 tx=0x7f3b2002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2affb8114946a5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:14.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.400+0000 7f3b1ea00640 1 -- 172.21.15.89:0/1449191491 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b20049040 con 0x563d30227b90 2024-09-10T08:41:14.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.400+0000 7f3b1ea00640 1 -- 172.21.15.89:0/1449191491 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b200d4020 con 0x563d30227b90 2024-09-10T08:41:14.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.400+0000 7f3b1ea00640 1 -- 172.21.15.89:0/1449191491 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b200c04b0 con 0x563d30227b90 2024-09-10T08:41:14.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.401+0000 7f3b381d0740 1 -- 172.21.15.89:0/1449191491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30227b90 msgr2=0x563d3022a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:14.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.401+0000 7f3b381d0740 1 --2- 172.21.15.89:0/1449191491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30227b90 0x563d3022a5f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3b20046040 tx=0x7f3b2002a410 comp rx=0 tx=0).stop 2024-09-10T08:41:14.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.401+0000 7f3b381d0740 1 -- 172.21.15.89:0/1449191491 shutdown_connections 2024-09-10T08:41:14.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.401+0000 7f3b381d0740 1 --2- 172.21.15.89:0/1449191491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d30232d70 0x563d3024a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.401+0000 7f3b381d0740 1 --2- 172.21.15.89:0/1449191491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d302284b0 0x563d30231b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.401+0000 7f3b381d0740 1 --2- 172.21.15.89:0/1449191491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30227b90 0x563d3022a5f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.401+0000 7f3b381d0740 1 -- 172.21.15.89:0/1449191491 >> 172.21.15.89:0/1449191491 conn(0x563d30224120 msgr2=0x563d30231440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:14.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.401+0000 7f3b381d0740 1 -- 172.21.15.89:0/1449191491 shutdown_connections 2024-09-10T08:41:14.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.401+0000 7f3b381d0740 1 -- 172.21.15.89:0/1449191491 wait complete. 2024-09-10T08:41:14.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.402+0000 7f3b381d0740 1 -- start start 2024-09-10T08:41:14.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.402+0000 7f3b381d0740 1 Processor -- start 2024-09-10T08:41:14.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.402+0000 7f3b381d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d30227b90 0x563d3023b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:14.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.402+0000 7f3b381d0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d302284b0 0x563d302369a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:14.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.402+0000 7f3b381d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30232d70 0x563d30236fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:14.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.402+0000 7f3b381d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3b301af5b0 con 0x563d30232d70 2024-09-10T08:41:14.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.402+0000 7f3b381d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3b301b0a20 con 0x563d30227b90 2024-09-10T08:41:14.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.402+0000 7f3b381d0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3b301afc80 con 0x563d302284b0 2024-09-10T08:41:14.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.402+0000 7f3b381d0740 10 client.0 _ll_register_callbacks cb 0x7f3b200f3420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:14.406 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.403+0000 7f3b2ea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d30227b90 0x563d3023b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:14.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.403+0000 7f3b27e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d302284b0 0x563d302369a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:14.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.403+0000 7f3b2ea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d30227b90 0x563d3023b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:44872/0 (socket says 172.21.15.89:44872) 2024-09-10T08:41:14.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.403+0000 7f3b2ea00640 1 -- 172.21.15.89:0/884595409 learned_addr learned my addr 172.21.15.89:0/884595409 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:14.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.403+0000 7f3b2f400640 1 --2- 172.21.15.89:0/884595409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30232d70 0x563d30236fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:14.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.403+0000 7f3b2ea00640 1 -- 172.21.15.89:0/884595409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d302284b0 msgr2=0x563d302369a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:14.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.403+0000 7f3b2ea00640 1 --2- 172.21.15.89:0/884595409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d302284b0 0x563d302369a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.403+0000 7f3b2ea00640 1 -- 172.21.15.89:0/884595409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30232d70 msgr2=0x563d30236fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:14.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.403+0000 7f3b2ea00640 1 --2- 172.21.15.89:0/884595409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30232d70 0x563d30236fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.403+0000 7f3b2ea00640 1 -- 172.21.15.89:0/884595409 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d302377e0 con 0x563d30227b90 2024-09-10T08:41:14.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.404+0000 7f3b2ea00640 1 --2- 172.21.15.89:0/884595409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d30227b90 0x563d3023b670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3b200bbeb0 tx=0x7f3b20031490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4dd93782a86d5fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:14.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.404+0000 7f3b13e00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:41:14.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.404+0000 7f3b13e00640 1 -- 172.21.15.89:0/884595409 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2009f060 con 0x563d30227b90 2024-09-10T08:41:14.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.404+0000 7f3b13e00640 1 -- 172.21.15.89:0/884595409 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f3b200bc640 con 0x563d30227b90 2024-09-10T08:41:14.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.404+0000 7f3b381d0740 1 -- 172.21.15.89:0/884595409 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b200d4020 con 0x563d30227b90 2024-09-10T08:41:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.404+0000 7f3b13e00640 1 -- 172.21.15.89:0/884595409 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3b20049070 con 0x563d30227b90 2024-09-10T08:41:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.404+0000 7f3b381d0740 10 client.25667 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.404+0000 7f3b381d0740 1 -- 172.21.15.89:0/884595409 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563d302377e0 con 0x563d30227b90 2024-09-10T08:41:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.404+0000 7f3b13e00640 1 -- 172.21.15.89:0/884595409 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b200d4020 con 0x563d30227b90 2024-09-10T08:41:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.404+0000 7f3b13e00640 1 -- 172.21.15.89:0/884595409 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3b200c48e0 con 0x563d30227b90 2024-09-10T08:41:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.404+0000 7f3b381d0740 10 client.25667 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.404+0000 7f3b381d0740 10 client.25667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:41:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.404+0000 7f3b381d0740 20 client.25667 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.405+0000 7f3b381d0740 1 --2- 172.21.15.89:0/884595409 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563d302adb10 0x563d302aff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.405+0000 7f3b381d0740 4 client.25667 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:41:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.405+0000 7f3b381d0740 1 -- 172.21.15.89:0/884595409 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563d30195e60 con 0x563d302adb10 2024-09-10T08:41:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.405+0000 7f3b27e00640 1 --2- 172.21.15.89:0/884595409 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563d302adb10 0x563d302aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:14.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.405+0000 7f3b27e00640 1 --2- 172.21.15.89:0/884595409 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563d302adb10 0x563d302aff00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=974ab197f47a84ab server_cookie=2d0c96ab76be0195 in_seq=0 out_seq=0 2024-09-10T08:41:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.405+0000 7f3b13e00640 10 client.25667 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.406+0000 7f3b13e00640 1 -- 172.21.15.89:0/884595409 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563d30195e60 con 0x563d302adb10 2024-09-10T08:41:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.406+0000 7f3b13e00640 10 client.25667 handle_command_reply: tid=0 2024-09-10T08:41:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.406+0000 7f3b381d0740 1 client.25667 shutdown 2024-09-10T08:41:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.406+0000 7f3b381d0740 1 -- 172.21.15.89:0/884595409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d30227b90 msgr2=0x563d3023b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.406+0000 7f3b381d0740 1 --2- 172.21.15.89:0/884595409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d30227b90 0x563d3023b670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3b200bbeb0 tx=0x7f3b20031490 comp rx=0 tx=0).stop 2024-09-10T08:41:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.406+0000 7f3b381d0740 1 -- 172.21.15.89:0/884595409 shutdown_connections 2024-09-10T08:41:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.406+0000 7f3b381d0740 1 --2- 172.21.15.89:0/884595409 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563d302adb10 0x563d302aff00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.406+0000 7f3b381d0740 1 --2- 172.21.15.89:0/884595409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563d30232d70 0x563d30236fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.406+0000 7f3b381d0740 1 --2- 172.21.15.89:0/884595409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d302284b0 0x563d302369a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.406+0000 7f3b381d0740 1 --2- 172.21.15.89:0/884595409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d30227b90 0x563d3023b670 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.406+0000 7f3b13e00640 0 client.25667 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.406+0000 7f3b381d0740 1 -- 172.21.15.89:0/884595409 >> 172.21.15.89:0/884595409 conn(0x563d30224120 msgr2=0x563d3022e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:14.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.406+0000 7f3b381d0740 1 -- 172.21.15.89:0/884595409 shutdown_connections 2024-09-10T08:41:14.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.406+0000 7f3b381d0740 1 -- 172.21.15.89:0/884595409 wait complete. 2024-09-10T08:41:14.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.406+0000 7f3b381d0740 20 client.25667 trim_cache size 0 max 16384 2024-09-10T08:41:14.410 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:41:14.410 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2024-09-10T08:41:14.410 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2024-09-10T08:41:14.410 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:41:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.407+0000 7f3b35600640 1 -- 172.21.15.89:0/538452113 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3b140a2c00 msgr2=0x7f3b140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.407+0000 7f3b35600640 1 --2- 172.21.15.89:0/538452113 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3b140a2c00 0x7f3b140a50c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3b28005fd0 tx=0x7f3b28009500 comp rx=0 tx=0).stop 2024-09-10T08:41:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.407+0000 7f3b35600640 1 -- 172.21.15.89:0/538452113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b30104010 msgr2=0x7f3b3019a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.407+0000 7f3b35600640 1 --2- 172.21.15.89:0/538452113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b30104010 0x7f3b3019a6b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3b2002a930 tx=0x7f3b20003b20 comp rx=0 tx=0).stop 2024-09-10T08:41:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.407+0000 7f3b35600640 1 -- 172.21.15.89:0/538452113 shutdown_connections 2024-09-10T08:41:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.407+0000 7f3b35600640 1 --2- 172.21.15.89:0/538452113 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3b140a2c00 0x7f3b140a50c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.407+0000 7f3b35600640 1 --2- 172.21.15.89:0/538452113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b30109140 0x7f3b3019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.407+0000 7f3b35600640 1 --2- 172.21.15.89:0/538452113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b301049c0 0x7f3b3019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.407+0000 7f3b35600640 1 --2- 172.21.15.89:0/538452113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b30104010 0x7f3b3019a6b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:14.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.407+0000 7f3b35600640 1 -- 172.21.15.89:0/538452113 >> 172.21.15.89:0/538452113 conn(0x7f3b300ffa50 msgr2=0x7f3b30100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:14.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.407+0000 7f3b35600640 1 -- 172.21.15.89:0/538452113 shutdown_connections 2024-09-10T08:41:14.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:14.407+0000 7f3b35600640 1 -- 172.21.15.89:0/538452113 wait complete. 2024-09-10T08:41:15.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:14 smithi089 ceph-mon[30757]: pgmap v432: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-10T08:41:15.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:14 smithi149 ceph-mon[36167]: pgmap v432: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-10T08:41:15.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:14 smithi111 ceph-mon[35220]: pgmap v432: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-10T08:41:15.436 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:41:15.436 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:41:15.818 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:41:16.243 INFO:tasks.workunit.client.0.smithi089.stderr:Updating files: 59% (7551/12697) Updating files: 60% (7619/12697) Updating files: 61% (7746/12697) Updating files: 62% (7873/12697) Updating files: 63% (8000/12697) Updating files: 64% (8127/12697) Updating files: 65% (8254/12697) Updating files: 66% (8381/12697) Updating files: 67% (8507/12697) Updating files: 68% (8634/12697) Updating files: 69% (8761/12697) Updating files: 70% (8888/12697) Updating files: 71% (9015/12697) Updating files: 72% (9142/12697) Updating files: 73% (9269/12697) Updating files: 74% (9396/12697) Updating files: 75% (9523/12697) Updating files: 76% (9650/12697) Updating files: 77% (9777/12697) Updating files: 78% (9904/12697) Updating files: 79% (10031/12697) Updating files: 80% (10158/12697) Updating files: 81% (10285/12697) Updating files: 82% (10412/12697) Updating files: 83% (10539/12697) Updating files: 84% (10666/12697) Updating files: 85% (10793/12697) Updating files: 86% (10920/12697) Updating files: 87% (11047/12697) Updating files: 88% (11174/12697) Updating files: 89% (11301/12697) Updating files: 90% (11428/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 97% (12384/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-10T08:41:16.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.426+0000 7f1c2be00640 1 -- 172.21.15.89:0/192634316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c2c10e260 msgr2=0x7f1c2c110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.426+0000 7f1c2be00640 1 --2- 172.21.15.89:0/192634316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c2c10e260 0x7f1c2c110650 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009ca0 tx=0x7f1c2402a950 comp rx=0 tx=0).stop 2024-09-10T08:41:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.426+0000 7f1c2be00640 1 -- 172.21.15.89:0/192634316 shutdown_connections 2024-09-10T08:41:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.426+0000 7f1c2be00640 1 --2- 172.21.15.89:0/192634316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c2c10e260 0x7f1c2c110650 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.426+0000 7f1c2be00640 1 --2- 172.21.15.89:0/192634316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c2c074ee0 0x7f1c2c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.426+0000 7f1c2be00640 1 --2- 172.21.15.89:0/192634316 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c2c0745c0 0x7f1c2c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.426+0000 7f1c2be00640 1 -- 172.21.15.89:0/192634316 >> 172.21.15.89:0/192634316 conn(0x7f1c2c0fd740 msgr2=0x7f1c2c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.427+0000 7f1c2be00640 1 -- 172.21.15.89:0/192634316 shutdown_connections 2024-09-10T08:41:16.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.427+0000 7f1c2be00640 1 -- 172.21.15.89:0/192634316 wait complete. 2024-09-10T08:41:16.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.428+0000 7f1c2be00640 1 Processor -- start 2024-09-10T08:41:16.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.428+0000 7f1c2be00640 1 -- start start 2024-09-10T08:41:16.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.428+0000 7f1c2be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c2c0745c0 0x7f1c2c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:16.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.428+0000 7f1c2be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c2c074ee0 0x7f1c2c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:16.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.428+0000 7f1c2be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c2c10e260 0x7f1c2c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:16.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.428+0000 7f1c2be00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c114000 con 0x7f1c2c074ee0 2024-09-10T08:41:16.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.428+0000 7f1c2be00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c113e80 con 0x7f1c2c10e260 2024-09-10T08:41:16.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.428+0000 7f1c2be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c114180 con 0x7f1c2c0745c0 2024-09-10T08:41:16.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.428+0000 7f1c2aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c2c0745c0 0x7f1c2c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:16.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.428+0000 7f1c2aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c2c0745c0 0x7f1c2c19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:33002/0 (socket says 172.21.15.89:33002) 2024-09-10T08:41:16.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.428+0000 7f1c2aa00640 1 -- 172.21.15.89:0/1350709742 learned_addr learned my addr 172.21.15.89:0/1350709742 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:16.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.428+0000 7f1c2a000640 1 --2- 172.21.15.89:0/1350709742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c2c074ee0 0x7f1c2c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:16.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.429+0000 7f1c2b400640 1 --2- 172.21.15.89:0/1350709742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c2c10e260 0x7f1c2c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:16.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.429+0000 7f1c2aa00640 1 -- 172.21.15.89:0/1350709742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c2c10e260 msgr2=0x7f1c2c19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:16.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.429+0000 7f1c2aa00640 1 --2- 172.21.15.89:0/1350709742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c2c10e260 0x7f1c2c19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:16.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.429+0000 7f1c2aa00640 1 -- 172.21.15.89:0/1350709742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c2c074ee0 msgr2=0x7f1c2c19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:16.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.429+0000 7f1c2aa00640 1 --2- 172.21.15.89:0/1350709742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c2c074ee0 0x7f1c2c19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:16.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.429+0000 7f1c2aa00640 1 -- 172.21.15.89:0/1350709742 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c19f400 con 0x7f1c2c0745c0 2024-09-10T08:41:16.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.429+0000 7f1c2aa00640 1 --2- 172.21.15.89:0/1350709742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c2c0745c0 0x7f1c2c19a450 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00d950 tx=0x7f1c1c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:16.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.430+0000 7f1c30a00640 1 -- 172.21.15.89:0/1350709742 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c017030 con 0x7f1c2c0745c0 2024-09-10T08:41:16.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.430+0000 7f1c2be00640 1 -- 172.21.15.89:0/1350709742 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2c19f6f0 con 0x7f1c2c0745c0 2024-09-10T08:41:16.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.430+0000 7f1c30a00640 1 -- 172.21.15.89:0/1350709742 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1c004020 con 0x7f1c2c0745c0 2024-09-10T08:41:16.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.430+0000 7f1c30a00640 1 -- 172.21.15.89:0/1350709742 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c003d60 con 0x7f1c2c0745c0 2024-09-10T08:41:16.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.430+0000 7f1c2be00640 1 -- 172.21.15.89:0/1350709742 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c2c1a6fd0 con 0x7f1c2c0745c0 2024-09-10T08:41:16.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.431+0000 7f1c2be00640 1 -- 172.21.15.89:0/1350709742 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c2c0749a0 con 0x7f1c2c0745c0 2024-09-10T08:41:16.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.434+0000 7f1c30a00640 1 -- 172.21.15.89:0/1350709742 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1c1c01a070 con 0x7f1c2c0745c0 2024-09-10T08:41:16.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.436+0000 7f1c30a00640 1 --2- 172.21.15.89:0/1350709742 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1c100a2b80 0x7f1c100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:16.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.436+0000 7f1c30a00640 1 -- 172.21.15.89:0/1350709742 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f1c1c0ced00 con 0x7f1c2c0745c0 2024-09-10T08:41:16.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.436+0000 7f1c2a000640 1 --2- 172.21.15.89:0/1350709742 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1c100a2b80 0x7f1c100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:16.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.437+0000 7f1c2a000640 1 --2- 172.21.15.89:0/1350709742 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1c100a2b80 0x7f1c100a5040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c10f510 tx=0x7f1c1800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:16.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.440+0000 7f1c30a00640 1 -- 172.21.15.89:0/1350709742 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c1c021070 con 0x7f1c2c0745c0 2024-09-10T08:41:16.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.653+0000 7f1c2be00640 1 -- 172.21.15.89:0/1350709742 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c2c02d7c0 con 0x7f1c2c0745c0 2024-09-10T08:41:16.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.654+0000 7f1c30a00640 1 -- 172.21.15.89:0/1350709742 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11918 (secure 0 0 0) 0x7f1c1c09b830 con 0x7f1c2c0745c0 2024-09-10T08:41:16.654 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:41:16.654 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-10T08:41:16.655 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-10T08:39:39:721431+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:39:38.834417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24518,24496]},"id":1}]} 2024-09-10T08:41:16.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.656+0000 7f1c2be00640 1 -- 172.21.15.89:0/1350709742 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1c100a2b80 msgr2=0x7f1c100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.656+0000 7f1c2be00640 1 --2- 172.21.15.89:0/1350709742 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1c100a2b80 0x7f1c100a5040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c10f510 tx=0x7f1c1800b410 comp rx=0 tx=0).stop 2024-09-10T08:41:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.657+0000 7f1c2be00640 1 -- 172.21.15.89:0/1350709742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c2c0745c0 msgr2=0x7f1c2c19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.657+0000 7f1c2be00640 1 --2- 172.21.15.89:0/1350709742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c2c0745c0 0x7f1c2c19a450 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00d950 tx=0x7f1c1c00de10 comp rx=0 tx=0).stop 2024-09-10T08:41:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.657+0000 7f1c2be00640 1 -- 172.21.15.89:0/1350709742 shutdown_connections 2024-09-10T08:41:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.657+0000 7f1c2be00640 1 --2- 172.21.15.89:0/1350709742 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1c100a2b80 0x7f1c100a5040 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.657+0000 7f1c2be00640 1 --2- 172.21.15.89:0/1350709742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c2c10e260 0x7f1c2c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.657+0000 7f1c2be00640 1 --2- 172.21.15.89:0/1350709742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c2c074ee0 0x7f1c2c19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.657+0000 7f1c2be00640 1 --2- 172.21.15.89:0/1350709742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c2c0745c0 0x7f1c2c19a450 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.657+0000 7f1c2be00640 1 -- 172.21.15.89:0/1350709742 >> 172.21.15.89:0/1350709742 conn(0x7f1c2c0fd740 msgr2=0x7f1c2c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.657+0000 7f1c2be00640 1 -- 172.21.15.89:0/1350709742 shutdown_connections 2024-09-10T08:41:16.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:16.657+0000 7f1c2be00640 1 -- 172.21.15.89:0/1350709742 wait complete. 2024-09-10T08:41:16.916 INFO:tasks.workunit.client.0.smithi089.stderr:Note: switching to '1cf819b7212a6048e92fc78c4eff963344f6d56b'. 2024-09-10T08:41:16.916 INFO:tasks.workunit.client.0.smithi089.stderr: 2024-09-10T08:41:16.916 INFO:tasks.workunit.client.0.smithi089.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T08:41:16.916 INFO:tasks.workunit.client.0.smithi089.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T08:41:16.916 INFO:tasks.workunit.client.0.smithi089.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T08:41:16.916 INFO:tasks.workunit.client.0.smithi089.stderr: 2024-09-10T08:41:16.916 INFO:tasks.workunit.client.0.smithi089.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T08:41:16.916 INFO:tasks.workunit.client.0.smithi089.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T08:41:16.916 INFO:tasks.workunit.client.0.smithi089.stderr: 2024-09-10T08:41:16.917 INFO:tasks.workunit.client.0.smithi089.stderr: git switch -c 2024-09-10T08:41:16.917 INFO:tasks.workunit.client.0.smithi089.stderr: 2024-09-10T08:41:16.917 INFO:tasks.workunit.client.0.smithi089.stderr:Or undo this operation with: 2024-09-10T08:41:16.917 INFO:tasks.workunit.client.0.smithi089.stderr: 2024-09-10T08:41:16.917 INFO:tasks.workunit.client.0.smithi089.stderr: git switch - 2024-09-10T08:41:16.917 INFO:tasks.workunit.client.0.smithi089.stderr: 2024-09-10T08:41:16.917 INFO:tasks.workunit.client.0.smithi089.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T08:41:16.917 INFO:tasks.workunit.client.0.smithi089.stderr: 2024-09-10T08:41:16.917 INFO:tasks.workunit.client.0.smithi089.stderr:HEAD is now at 1cf819b7212 Merge PR #59672 into wip-rishabh-testing-20240909.165402-squid 2024-09-10T08:41:16.930 DEBUG:teuthology.orchestra.run.smithi089:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-10T08:41:17.014 INFO:tasks.workunit.client.0.smithi089.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T08:41:17.019 INFO:tasks.workunit.client.0.smithi089.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T08:41:17.019 INFO:tasks.workunit.client.0.smithi089.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T08:41:17.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:16 smithi089 ceph-mon[30757]: pgmap v433: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-10T08:41:17.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:16 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1350709742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:41:17.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:16 smithi149 ceph-mon[36167]: pgmap v433: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-10T08:41:17.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:16 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1350709742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:41:17.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:16 smithi111 ceph-mon[35220]: pgmap v433: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-10T08:41:17.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:16 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1350709742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:41:17.869 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:41:18.287 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:41:18.333 INFO:tasks.workunit.client.0.smithi089.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T08:41:18.515 INFO:tasks.workunit.client.0.smithi089.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T08:41:18.583 INFO:tasks.workunit.client.0.smithi089.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T08:41:18.586 INFO:tasks.workunit.client.0.smithi089.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T08:41:18.586 INFO:tasks.workunit.client.0.smithi089.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T08:41:18.674 INFO:tasks.workunit.client.0.smithi089.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T08:41:18.684 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:41:18.684 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-10T08:41:18.722 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-09-10T08:41:18.723 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-09-10T08:41:18.723 DEBUG:teuthology.orchestra.run.smithi089:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1cf819b7212a6048e92fc78c4eff963344f6d56b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-09-10T08:41:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.953+0000 7f5713200640 1 -- 172.21.15.89:0/3212350919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5714076720 msgr2=0x7f5714076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.953+0000 7f5713200640 1 --2- 172.21.15.89:0/3212350919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5714076720 0x7f5714076b00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5704009900 tx=0x7f570402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.953+0000 7f5713200640 1 -- 172.21.15.89:0/3212350919 shutdown_connections 2024-09-10T08:41:18.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.953+0000 7f5713200640 1 --2- 172.21.15.89:0/3212350919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5714112870 0x7f5714114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.953+0000 7f5713200640 1 --2- 172.21.15.89:0/3212350919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5714077040 0x7f5714112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.953+0000 7f5713200640 1 --2- 172.21.15.89:0/3212350919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5714076720 0x7f5714076b00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.953+0000 7f5713200640 1 -- 172.21.15.89:0/3212350919 >> 172.21.15.89:0/3212350919 conn(0x7f57140ff8d0 msgr2=0x7f5714101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.954+0000 7f5713200640 1 -- 172.21.15.89:0/3212350919 shutdown_connections 2024-09-10T08:41:18.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.954+0000 7f5713200640 1 -- 172.21.15.89:0/3212350919 wait complete. 2024-09-10T08:41:18.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.955+0000 7f5713200640 1 Processor -- start 2024-09-10T08:41:18.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.955+0000 7f5713200640 1 -- start start 2024-09-10T08:41:18.958 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:18 smithi089 ceph-mon[30757]: pgmap v434: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-10T08:41:18.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.955+0000 7f5713200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5714076720 0x7f57141a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:18.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.956+0000 7f5713200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5714077040 0x7f57141a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:18.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.956+0000 7f5713200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5714112870 0x7f571419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:18.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.956+0000 7f5713200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f57141183e0 con 0x7f5714076720 2024-09-10T08:41:18.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.956+0000 7f5713200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5714118260 con 0x7f5714112870 2024-09-10T08:41:18.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.956+0000 7f5713200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5714118560 con 0x7f5714077040 2024-09-10T08:41:18.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.956+0000 7f5711e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5714076720 0x7f57141a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:18.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.956+0000 7f5711e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5714076720 0x7f57141a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38468/0 (socket says 172.21.15.89:38468) 2024-09-10T08:41:18.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.956+0000 7f5711e00640 1 -- 172.21.15.89:0/2431133596 learned_addr learned my addr 172.21.15.89:0/2431133596 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:18.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.956+0000 7f5712800640 1 --2- 172.21.15.89:0/2431133596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5714112870 0x7f571419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:18.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.956+0000 7f5711400640 1 --2- 172.21.15.89:0/2431133596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5714077040 0x7f57141a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:18.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.956+0000 7f5711e00640 1 -- 172.21.15.89:0/2431133596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5714077040 msgr2=0x7f57141a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:18.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.956+0000 7f5711e00640 1 --2- 172.21.15.89:0/2431133596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5714077040 0x7f57141a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.956+0000 7f5711e00640 1 -- 172.21.15.89:0/2431133596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5714112870 msgr2=0x7f571419e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:18.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.956+0000 7f5711e00640 1 --2- 172.21.15.89:0/2431133596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5714112870 0x7f571419e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.956+0000 7f5711e00640 1 -- 172.21.15.89:0/2431133596 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571419ef80 con 0x7f5714076720 2024-09-10T08:41:18.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.957+0000 7f5711e00640 1 --2- 172.21.15.89:0/2431133596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5714076720 0x7f57141a4640 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5704006010 tx=0x7f5704004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:18.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.957+0000 7f570aa00640 1 -- 172.21.15.89:0/2431133596 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570403f030 con 0x7f5714076720 2024-09-10T08:41:18.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.957+0000 7f570aa00640 1 -- 172.21.15.89:0/2431133596 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5704004440 con 0x7f5714076720 2024-09-10T08:41:18.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.957+0000 7f570aa00640 1 -- 172.21.15.89:0/2431133596 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704035070 con 0x7f5714076720 2024-09-10T08:41:18.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.957+0000 7f5713200640 1 -- 172.21.15.89:0/2431133596 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f571419f210 con 0x7f5714076720 2024-09-10T08:41:18.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.957+0000 7f5713200640 1 -- 172.21.15.89:0/2431133596 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57141ab3f0 con 0x7f5714076720 2024-09-10T08:41:18.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.960+0000 7f570aa00640 1 -- 172.21.15.89:0/2431133596 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5704042030 con 0x7f5714076720 2024-09-10T08:41:18.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.961+0000 7f571b650740 1 Processor -- start 2024-09-10T08:41:18.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.961+0000 7f571b650740 1 -- start start 2024-09-10T08:41:18.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.961+0000 7f571b650740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749157e40 0x559749158220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:18.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.961+0000 7f571b650740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749158760 0x559749162ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:18.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.961+0000 7f571b650740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749163020 0x559749165410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:18.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.961+0000 7f571b650740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5597490be3e0 con 0x559749157e40 2024-09-10T08:41:18.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.961+0000 7f571b650740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559749065d80 con 0x559749158760 2024-09-10T08:41:18.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.961+0000 7f571b650740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5597490ba3a0 con 0x559749163020 2024-09-10T08:41:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.961+0000 7f5712800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749157e40 0x559749158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.961+0000 7f5712800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749157e40 0x559749158220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38482/0 (socket says 172.21.15.89:38482) 2024-09-10T08:41:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.961+0000 7f5712800640 1 -- 172.21.15.89:0/2466012522 learned_addr learned my addr 172.21.15.89:0/2466012522 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.961+0000 7f5711400640 1 --2- 172.21.15.89:0/2466012522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749163020 0x559749165410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.962+0000 7f5711e00640 1 --2- 172.21.15.89:0/2466012522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749158760 0x559749162ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.962+0000 7f5711400640 1 -- 172.21.15.89:0/2466012522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749158760 msgr2=0x559749162ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.962+0000 7f5711400640 1 --2- 172.21.15.89:0/2466012522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749158760 0x559749162ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.962+0000 7f5711400640 1 -- 172.21.15.89:0/2466012522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749157e40 msgr2=0x559749158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.962+0000 7f5711400640 1 --2- 172.21.15.89:0/2466012522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749157e40 0x559749158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.962+0000 7f5711400640 1 -- 172.21.15.89:0/2466012522 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559749084ae0 con 0x559749163020 2024-09-10T08:41:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.962+0000 7f5712800640 1 --2- 172.21.15.89:0/2466012522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749157e40 0x559749158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.962+0000 7f5711e00640 1 --2- 172.21.15.89:0/2466012522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749158760 0x559749162ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:41:18.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.962+0000 7f5711400640 1 --2- 172.21.15.89:0/2466012522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749163020 0x559749165410 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5700009bf0 tx=0x7f5700011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd023df7c85fa759 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.962+0000 7f570aa00640 1 --2- 172.21.15.89:0/2431133596 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f56f80a2b80 0x7f56f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.962+0000 7f56ff000640 1 -- 172.21.15.89:0/2466012522 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570001e070 con 0x559749163020 2024-09-10T08:41:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.963+0000 7f56ff000640 1 -- 172.21.15.89:0/2466012522 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5700009da0 con 0x559749163020 2024-09-10T08:41:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.963+0000 7f56ff000640 1 -- 172.21.15.89:0/2466012522 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700018870 con 0x559749163020 2024-09-10T08:41:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.963+0000 7f570aa00640 1 -- 172.21.15.89:0/2431133596 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f57040ef7b0 con 0x7f5714076720 2024-09-10T08:41:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.963+0000 7f5712800640 1 --2- 172.21.15.89:0/2431133596 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f56f80a2b80 0x7f56f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.963+0000 7f571b650740 1 -- 172.21.15.89:0/2466012522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749163020 msgr2=0x559749165410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.963+0000 7f571b650740 1 --2- 172.21.15.89:0/2466012522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749163020 0x559749165410 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5700009bf0 tx=0x7f5700011810 comp rx=0 tx=0).stop 2024-09-10T08:41:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.964+0000 7f5712800640 1 --2- 172.21.15.89:0/2431133596 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f56f80a2b80 0x7f56f80a5040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f570c009fd0 tx=0x7f570c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.964+0000 7f571b650740 1 -- 172.21.15.89:0/2466012522 shutdown_connections 2024-09-10T08:41:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.964+0000 7f571b650740 1 --2- 172.21.15.89:0/2466012522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749163020 0x559749165410 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.964+0000 7f571b650740 1 --2- 172.21.15.89:0/2466012522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749158760 0x559749162ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.964+0000 7f571b650740 1 --2- 172.21.15.89:0/2466012522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749157e40 0x559749158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.964+0000 7f571b650740 1 -- 172.21.15.89:0/2466012522 >> 172.21.15.89:0/2466012522 conn(0x5597491543d0 msgr2=0x5597491547b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.964+0000 7f571b650740 1 -- 172.21.15.89:0/2466012522 shutdown_connections 2024-09-10T08:41:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.964+0000 7f571b650740 1 -- 172.21.15.89:0/2466012522 wait complete. 2024-09-10T08:41:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.964+0000 7f571b650740 1 -- start start 2024-09-10T08:41:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.964+0000 7f571b650740 1 Processor -- start 2024-09-10T08:41:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.965+0000 7f571b650740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749157e40 0x55974915ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.965+0000 7f571b650740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749158760 0x559749160480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.965+0000 7f571b650740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749163020 0x5597491609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.965+0000 7f571b650740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559749165bf0 con 0x559749158760 2024-09-10T08:41:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.965+0000 7f571b650740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5597490790f0 con 0x559749157e40 2024-09-10T08:41:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.965+0000 7f571b650740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559749165d70 con 0x559749163020 2024-09-10T08:41:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.965+0000 7f571b650740 10 client.0 _ll_register_callbacks cb 0x559749060bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:18.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.965+0000 7f5712800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749163020 0x5597491609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.965+0000 7f5711400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749158760 0x559749160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.965+0000 7f5712800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749163020 0x5597491609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:33030/0 (socket says 172.21.15.89:33030) 2024-09-10T08:41:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.965+0000 7f5711e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749157e40 0x55974915ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.965+0000 7f5712800640 1 -- 172.21.15.89:0/2052952493 learned_addr learned my addr 172.21.15.89:0/2052952493 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.966+0000 7f5712800640 1 -- 172.21.15.89:0/2052952493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749157e40 msgr2=0x55974915ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.966+0000 7f5712800640 1 --2- 172.21.15.89:0/2052952493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749157e40 0x55974915ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.966+0000 7f5712800640 1 -- 172.21.15.89:0/2052952493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749158760 msgr2=0x559749160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.966+0000 7f5712800640 1 --2- 172.21.15.89:0/2052952493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749158760 0x559749160480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.966+0000 7f5712800640 1 -- 172.21.15.89:0/2052952493 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559748e2f3c0 con 0x559749163020 2024-09-10T08:41:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.966+0000 7f5712800640 1 --2- 172.21.15.89:0/2052952493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749163020 0x5597491609c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f570c00f6d0 tx=0x7f570c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5f159d378beefa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.966+0000 7f56f3400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:41:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.966+0000 7f56f3400640 1 -- 172.21.15.89:0/2052952493 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c003b80 con 0x559749163020 2024-09-10T08:41:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.966+0000 7f571b650740 1 -- 172.21.15.89:0/2052952493 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5700009da0 con 0x559749163020 2024-09-10T08:41:18.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.966+0000 7f56f3400640 1 -- 172.21.15.89:0/2052952493 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f570c003d20 con 0x559749163020 2024-09-10T08:41:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.967+0000 7f56f3400640 1 -- 172.21.15.89:0/2052952493 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f570c030050 con 0x559749163020 2024-09-10T08:41:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.967+0000 7f571b650740 10 client.25633 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.967+0000 7f571b650740 1 -- 172.21.15.89:0/2052952493 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559749171cf0 con 0x559749163020 2024-09-10T08:41:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.967+0000 7f56f3400640 1 -- 172.21.15.89:0/2052952493 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5700009da0 con 0x559749163020 2024-09-10T08:41:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.967+0000 7f56f3400640 1 -- 172.21.15.89:0/2052952493 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f570c026030 con 0x559749163020 2024-09-10T08:41:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.967+0000 7f571b650740 10 client.25633 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.967+0000 7f571b650740 10 client.25633 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:41:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.967+0000 7f571b650740 20 client.25633 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.967+0000 7f571b650740 1 --2- 172.21.15.89:0/2052952493 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5597491731b0 0x5597491755c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.967+0000 7f571b650740 4 client.25633 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:41:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.967+0000 7f571b650740 1 -- 172.21.15.89:0/2052952493 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597490cb950 con 0x5597491731b0 2024-09-10T08:41:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.968+0000 7f5711e00640 1 --2- 172.21.15.89:0/2052952493 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5597491731b0 0x5597491755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:18.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.968+0000 7f5711e00640 1 --2- 172.21.15.89:0/2052952493 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5597491731b0 0x5597491755c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b3cd84c6245c78db server_cookie=520b39dcf5dc2ba3 in_seq=0 out_seq=0 2024-09-10T08:41:18.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.968+0000 7f56f3400640 10 client.25633 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:41:18.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.969+0000 7f56f3400640 1 -- 172.21.15.89:0/2052952493 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597490cb950 con 0x5597491731b0 2024-09-10T08:41:18.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.969+0000 7f56f3400640 10 client.25633 handle_command_reply: tid=0 2024-09-10T08:41:18.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.969+0000 7f571b650740 1 client.25633 shutdown 2024-09-10T08:41:18.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.969+0000 7f571b650740 1 -- 172.21.15.89:0/2052952493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749163020 msgr2=0x5597491609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:18.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.969+0000 7f571b650740 1 --2- 172.21.15.89:0/2052952493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749163020 0x5597491609c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f570c00f6d0 tx=0x7f570c004ab0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.969+0000 7f571b650740 1 -- 172.21.15.89:0/2052952493 shutdown_connections 2024-09-10T08:41:18.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.969+0000 7f571b650740 1 --2- 172.21.15.89:0/2052952493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749163020 0x5597491609c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.969+0000 7f571b650740 1 --2- 172.21.15.89:0/2052952493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749158760 0x559749160480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.969+0000 7f571b650740 1 --2- 172.21.15.89:0/2052952493 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5597491731b0 0x5597491755c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.969+0000 7f571b650740 1 --2- 172.21.15.89:0/2052952493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749157e40 0x55974915ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.969+0000 7f56f3400640 0 client.25633 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:41:18.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.969+0000 7f571b650740 1 -- 172.21.15.89:0/2052952493 >> 172.21.15.89:0/2052952493 conn(0x5597491543d0 msgr2=0x55974915a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:18.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.969+0000 7f571b650740 1 -- 172.21.15.89:0/2052952493 shutdown_connections 2024-09-10T08:41:18.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.969+0000 7f571b650740 1 -- 172.21.15.89:0/2052952493 wait complete. 2024-09-10T08:41:18.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.970+0000 7f571b650740 20 client.25633 trim_cache size 0 max 16384 2024-09-10T08:41:18.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.983+0000 7f571b650740 1 Processor -- start 2024-09-10T08:41:18.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.983+0000 7f571b650740 1 -- start start 2024-09-10T08:41:18.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.983+0000 7f571b650740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749157e40 0x55974915a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:18.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.983+0000 7f571b650740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749158760 0x559749161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:18.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.983+0000 7f571b650740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749163020 0x55974917ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:18.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.983+0000 7f571b650740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f57141b7af0 con 0x559749158760 2024-09-10T08:41:18.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.983+0000 7f571b650740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f57141b53c0 con 0x559749163020 2024-09-10T08:41:18.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.983+0000 7f571b650740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f57141b3210 con 0x559749157e40 2024-09-10T08:41:18.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.984+0000 7f5711400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749158760 0x559749161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:18.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.984+0000 7f5711400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749158760 0x559749161eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38500/0 (socket says 172.21.15.89:38500) 2024-09-10T08:41:18.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.984+0000 7f5711400640 1 -- 172.21.15.89:0/1363919364 learned_addr learned my addr 172.21.15.89:0/1363919364 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:18.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.984+0000 7f5711e00640 1 --2- 172.21.15.89:0/1363919364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749157e40 0x55974915a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:18.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.984+0000 7f5712800640 1 --2- 172.21.15.89:0/1363919364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749163020 0x55974917ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:18.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.984+0000 7f5711400640 1 -- 172.21.15.89:0/1363919364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749157e40 msgr2=0x55974915a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:18.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.984+0000 7f5711400640 1 --2- 172.21.15.89:0/1363919364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749157e40 0x55974915a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.984+0000 7f5711400640 1 -- 172.21.15.89:0/1363919364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749163020 msgr2=0x55974917ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:18.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.984+0000 7f5711400640 1 --2- 172.21.15.89:0/1363919364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749163020 0x55974917ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.984+0000 7f5711400640 1 -- 172.21.15.89:0/1363919364 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597491628b0 con 0x559749158760 2024-09-10T08:41:18.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.984+0000 7f5711400640 1 --2- 172.21.15.89:0/1363919364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749158760 0x559749161eb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5700018420 tx=0x7f57000183b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8873d83d39261024 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:18.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.985+0000 7f56fe600640 1 -- 172.21.15.89:0/1363919364 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570001e040 con 0x559749158760 2024-09-10T08:41:18.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.985+0000 7f56fe600640 1 -- 172.21.15.89:0/1363919364 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5700009e60 con 0x559749158760 2024-09-10T08:41:18.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.985+0000 7f56fe600640 1 -- 172.21.15.89:0/1363919364 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700018580 con 0x559749158760 2024-09-10T08:41:18.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.985+0000 7f571b650740 1 -- 172.21.15.89:0/1363919364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749158760 msgr2=0x559749161eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:18.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.985+0000 7f571b650740 1 --2- 172.21.15.89:0/1363919364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749158760 0x559749161eb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5700018420 tx=0x7f57000183b0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.985+0000 7f571b650740 1 -- 172.21.15.89:0/1363919364 shutdown_connections 2024-09-10T08:41:18.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.985+0000 7f571b650740 1 --2- 172.21.15.89:0/1363919364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749163020 0x55974917ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.985+0000 7f571b650740 1 --2- 172.21.15.89:0/1363919364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749158760 0x559749161eb0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.985+0000 7f571b650740 1 --2- 172.21.15.89:0/1363919364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749157e40 0x55974915a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.985+0000 7f571b650740 1 -- 172.21.15.89:0/1363919364 >> 172.21.15.89:0/1363919364 conn(0x5597491543d0 msgr2=0x5597491617a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:18.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.986+0000 7f571b650740 1 -- 172.21.15.89:0/1363919364 shutdown_connections 2024-09-10T08:41:18.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.986+0000 7f571b650740 1 -- 172.21.15.89:0/1363919364 wait complete. 2024-09-10T08:41:18.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.986+0000 7f571b650740 1 -- start start 2024-09-10T08:41:18.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.986+0000 7f571b650740 1 Processor -- start 2024-09-10T08:41:18.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.987+0000 7f571b650740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749157e40 0x55974916b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:18.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.987+0000 7f571b650740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749158760 0x55974915cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:18.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.987+0000 7f571b650740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749163020 0x55974915d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:18.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.987+0000 7f571b650740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f57141b3880 con 0x559749158760 2024-09-10T08:41:18.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.987+0000 7f571b650740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f57141b4cf0 con 0x559749163020 2024-09-10T08:41:18.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.987+0000 7f571b650740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f57141b3f50 con 0x559749157e40 2024-09-10T08:41:18.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.987+0000 7f571b650740 10 client.0 _ll_register_callbacks cb 0x7f570c009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:18.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.987+0000 7f5711400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749158760 0x55974915cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:18.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.987+0000 7f5711400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749158760 0x55974915cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38514/0 (socket says 172.21.15.89:38514) 2024-09-10T08:41:18.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.987+0000 7f5711400640 1 -- 172.21.15.89:0/4090506011 learned_addr learned my addr 172.21.15.89:0/4090506011 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:18.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.987+0000 7f5711e00640 1 --2- 172.21.15.89:0/4090506011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749157e40 0x55974916b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:18.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.987+0000 7f5712800640 1 --2- 172.21.15.89:0/4090506011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749163020 0x55974915d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:18.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.987+0000 7f5711400640 1 -- 172.21.15.89:0/4090506011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749157e40 msgr2=0x55974916b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:18.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.987+0000 7f5711400640 1 --2- 172.21.15.89:0/4090506011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749157e40 0x55974916b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.987+0000 7f5711400640 1 -- 172.21.15.89:0/4090506011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749163020 msgr2=0x55974915d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:18.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.987+0000 7f5711400640 1 --2- 172.21.15.89:0/4090506011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749163020 0x55974915d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.988+0000 7f5711400640 1 -- 172.21.15.89:0/4090506011 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55974915dab0 con 0x559749158760 2024-09-10T08:41:18.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.988+0000 7f5711e00640 1 --2- 172.21.15.89:0/4090506011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749157e40 0x55974916b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:18.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.988+0000 7f5712800640 1 --2- 172.21.15.89:0/4090506011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749163020 0x55974915d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:18.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.988+0000 7f5711400640 1 --2- 172.21.15.89:0/4090506011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749158760 0x55974915cd00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f57000093a0 tx=0x7f5700003a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0b855354921b5c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:18.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.988+0000 7f56f3400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:41:18.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.988+0000 7f56f3400640 1 -- 172.21.15.89:0/4090506011 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570001e030 con 0x559749158760 2024-09-10T08:41:18.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.988+0000 7f56f3400640 1 -- 172.21.15.89:0/4090506011 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f5700005380 con 0x559749158760 2024-09-10T08:41:18.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.988+0000 7f571b650740 1 -- 172.21.15.89:0/4090506011 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5700009e60 con 0x559749158760 2024-09-10T08:41:18.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.988+0000 7f56f3400640 1 -- 172.21.15.89:0/4090506011 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5700005580 con 0x559749158760 2024-09-10T08:41:18.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.988+0000 7f571b650740 10 client.15627 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:18.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.988+0000 7f571b650740 1 -- 172.21.15.89:0/4090506011 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5597491ddf00 con 0x559749158760 2024-09-10T08:41:18.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.989+0000 7f56f3400640 1 -- 172.21.15.89:0/4090506011 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5700009e60 con 0x559749158760 2024-09-10T08:41:18.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.989+0000 7f56f3400640 1 -- 172.21.15.89:0/4090506011 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5700018570 con 0x559749158760 2024-09-10T08:41:18.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.989+0000 7f571b650740 10 client.15627 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:18.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.989+0000 7f571b650740 10 client.15627 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:41:18.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.989+0000 7f571b650740 20 client.15627 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:18.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.989+0000 7f571b650740 1 --2- 172.21.15.89:0/4090506011 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5597491de400 0x5597491e0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:18.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.989+0000 7f571b650740 4 client.15627 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:41:18.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.989+0000 7f571b650740 1 -- 172.21.15.89:0/4090506011 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5597490c5e60 con 0x5597491de400 2024-09-10T08:41:18.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.989+0000 7f5711e00640 1 --2- 172.21.15.89:0/4090506011 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5597491de400 0x5597491e0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:18.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.990+0000 7f5711e00640 1 --2- 172.21.15.89:0/4090506011 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5597491de400 0x5597491e0820 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e12f326b3e750d0c server_cookie=92cd0840da8007ea in_seq=0 out_seq=0 2024-09-10T08:41:18.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.990+0000 7f56f3400640 10 client.15627 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:41:18.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.990+0000 7f56f3400640 1 -- 172.21.15.89:0/4090506011 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5597490c5e60 con 0x5597491de400 2024-09-10T08:41:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.990+0000 7f56f3400640 10 client.15627 handle_command_reply: tid=0 2024-09-10T08:41:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.990+0000 7f571b650740 1 client.15627 shutdown 2024-09-10T08:41:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.990+0000 7f571b650740 1 -- 172.21.15.89:0/4090506011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749158760 msgr2=0x55974915cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.990+0000 7f571b650740 1 --2- 172.21.15.89:0/4090506011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749158760 0x55974915cd00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f57000093a0 tx=0x7f5700003a60 comp rx=0 tx=0).stop 2024-09-10T08:41:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.990+0000 7f571b650740 1 -- 172.21.15.89:0/4090506011 shutdown_connections 2024-09-10T08:41:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.990+0000 7f571b650740 1 --2- 172.21.15.89:0/4090506011 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5597491de400 0x5597491e0820 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.990+0000 7f571b650740 1 --2- 172.21.15.89:0/4090506011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559749163020 0x55974915d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.990+0000 7f571b650740 1 --2- 172.21.15.89:0/4090506011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559749158760 0x55974915cd00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.990+0000 7f571b650740 1 --2- 172.21.15.89:0/4090506011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559749157e40 0x55974916b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.990+0000 7f571b650740 1 -- 172.21.15.89:0/4090506011 >> 172.21.15.89:0/4090506011 conn(0x5597491543d0 msgr2=0x5597491617a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.990+0000 7f56f3400640 0 client.15627 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:41:18.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.991+0000 7f571b650740 1 -- 172.21.15.89:0/4090506011 shutdown_connections 2024-09-10T08:41:18.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.991+0000 7f571b650740 1 -- 172.21.15.89:0/4090506011 wait complete. 2024-09-10T08:41:18.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.991+0000 7f571b650740 20 client.15627 trim_cache size 0 max 16384 2024-09-10T08:41:18.996 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:41:18.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.991+0000 7f5713200640 1 -- 172.21.15.89:0/2431133596 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f56f80a2b80 msgr2=0x7f56f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:18.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.991+0000 7f5713200640 1 --2- 172.21.15.89:0/2431133596 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f56f80a2b80 0x7f56f80a5040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f570c009fd0 tx=0x7f570c009500 comp rx=0 tx=0).stop 2024-09-10T08:41:18.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.991+0000 7f5713200640 1 -- 172.21.15.89:0/2431133596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5714076720 msgr2=0x7f57141a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:18.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.991+0000 7f5713200640 1 --2- 172.21.15.89:0/2431133596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5714076720 0x7f57141a4640 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5704006010 tx=0x7f5704004040 comp rx=0 tx=0).stop 2024-09-10T08:41:18.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.991+0000 7f5713200640 1 -- 172.21.15.89:0/2431133596 shutdown_connections 2024-09-10T08:41:18.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.991+0000 7f5713200640 1 --2- 172.21.15.89:0/2431133596 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f56f80a2b80 0x7f56f80a5040 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.991+0000 7f5713200640 1 --2- 172.21.15.89:0/2431133596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5714112870 0x7f571419e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.991+0000 7f5713200640 1 --2- 172.21.15.89:0/2431133596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5714077040 0x7f57141a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.991+0000 7f5713200640 1 --2- 172.21.15.89:0/2431133596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5714076720 0x7f57141a4640 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:18.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.991+0000 7f5713200640 1 -- 172.21.15.89:0/2431133596 >> 172.21.15.89:0/2431133596 conn(0x7f57140ff8d0 msgr2=0x7f5714101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:18.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.992+0000 7f5713200640 1 -- 172.21.15.89:0/2431133596 shutdown_connections 2024-09-10T08:41:18.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:18.992+0000 7f5713200640 1 -- 172.21.15.89:0/2431133596 wait complete. 2024-09-10T08:41:19.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:18 smithi149 ceph-mon[36167]: pgmap v434: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-10T08:41:19.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:18 smithi111 ceph-mon[35220]: pgmap v434: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-10T08:41:19.864 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:41:20.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:19 smithi149 ceph-mon[36167]: pgmap v435: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-10T08:41:20.234 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:41:20.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:19 smithi111 ceph-mon[35220]: pgmap v435: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-10T08:41:20.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:19 smithi089 ceph-mon[30757]: pgmap v435: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-10T08:41:20.782 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.774+0000 7f89c2a00640 1 -- 172.21.15.89:0/2975816982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89bc102790 msgr2=0x7f89bc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:20.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.774+0000 7f89c2a00640 1 --2- 172.21.15.89:0/2975816982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89bc102790 0x7f89bc1067e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009900 tx=0x7f89a802a410 comp rx=0 tx=0).stop 2024-09-10T08:41:20.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.774+0000 7f89c2a00640 1 -- 172.21.15.89:0/2975816982 shutdown_connections 2024-09-10T08:41:20.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.774+0000 7f89c2a00640 1 --2- 172.21.15.89:0/2975816982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89bc106f10 0x7f89bc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.774+0000 7f89c2a00640 1 --2- 172.21.15.89:0/2975816982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89bc102790 0x7f89bc1067e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.783 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.774+0000 7f89c2a00640 1 --2- 172.21.15.89:0/2975816982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89bc101de0 0x7f89bc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.774+0000 7f89c2a00640 1 -- 172.21.15.89:0/2975816982 >> 172.21.15.89:0/2975816982 conn(0x7f89bc0fd8c0 msgr2=0x7f89bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:20.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.774+0000 7f89c2a00640 1 -- 172.21.15.89:0/2975816982 shutdown_connections 2024-09-10T08:41:20.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.774+0000 7f89c2a00640 1 -- 172.21.15.89:0/2975816982 wait complete. 2024-09-10T08:41:20.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.775+0000 7f89c2a00640 1 Processor -- start 2024-09-10T08:41:20.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.775+0000 7f89c2a00640 1 -- start start 2024-09-10T08:41:20.784 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.775+0000 7f89c2a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89bc101de0 0x7f89bc19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:20.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.775+0000 7f89c2a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89bc102790 0x7f89bc19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:20.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.776+0000 7f89c2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89bc106f10 0x7f89bc19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:20.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.776+0000 7f89c2a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc111f70 con 0x7f89bc106f10 2024-09-10T08:41:20.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.776+0000 7f89c2a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc111df0 con 0x7f89bc102790 2024-09-10T08:41:20.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.776+0000 7f89c2a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1120f0 con 0x7f89bc101de0 2024-09-10T08:41:20.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.776+0000 7f89c0a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89bc106f10 0x7f89bc19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:20.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.776+0000 7f89c0a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89bc106f10 0x7f89bc19ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38534/0 (socket says 172.21.15.89:38534) 2024-09-10T08:41:20.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.776+0000 7f89c0a00640 1 -- 172.21.15.89:0/2331346006 learned_addr learned my addr 172.21.15.89:0/2331346006 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:20.785 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.776+0000 7f89bb400640 1 --2- 172.21.15.89:0/2331346006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89bc102790 0x7f89bc19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:20.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.776+0000 7f89bbe00640 1 --2- 172.21.15.89:0/2331346006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89bc101de0 0x7f89bc19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:20.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.776+0000 7f89c0a00640 1 -- 172.21.15.89:0/2331346006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89bc101de0 msgr2=0x7f89bc19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:20.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.776+0000 7f89c0a00640 1 --2- 172.21.15.89:0/2331346006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89bc101de0 0x7f89bc19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.776+0000 7f89c0a00640 1 -- 172.21.15.89:0/2331346006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89bc102790 msgr2=0x7f89bc19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:20.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.776+0000 7f89c0a00640 1 --2- 172.21.15.89:0/2331346006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89bc102790 0x7f89bc19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.776+0000 7f89c0a00640 1 -- 172.21.15.89:0/2331346006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89bc19f600 con 0x7f89bc106f10 2024-09-10T08:41:20.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.777+0000 7f89c0a00640 1 --2- 172.21.15.89:0/2331346006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89bc106f10 0x7f89bc19ef20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f89b400eb10 tx=0x7f89b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:20.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.777+0000 7f89b8c00640 1 -- 172.21.15.89:0/2331346006 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4004b20 con 0x7f89bc106f10 2024-09-10T08:41:20.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.777+0000 7f89b8c00640 1 -- 172.21.15.89:0/2331346006 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89b4004cc0 con 0x7f89bc106f10 2024-09-10T08:41:20.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.777+0000 7f89c2a00640 1 -- 172.21.15.89:0/2331346006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89bc19f8f0 con 0x7f89bc106f10 2024-09-10T08:41:20.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.777+0000 7f89b8c00640 1 -- 172.21.15.89:0/2331346006 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4010760 con 0x7f89bc106f10 2024-09-10T08:41:20.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.777+0000 7f89c2a00640 1 -- 172.21.15.89:0/2331346006 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89bc1a71d0 con 0x7f89bc106f10 2024-09-10T08:41:20.786 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.781+0000 7f89c54d0740 1 Processor -- start 2024-09-10T08:41:20.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.781+0000 7f89b8c00640 1 -- 172.21.15.89:0/2331346006 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f89b4005020 con 0x7f89bc106f10 2024-09-10T08:41:20.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.781+0000 7f89c54d0740 1 -- start start 2024-09-10T08:41:20.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.782+0000 7f89c54d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a45c70 0x560a57a46050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:20.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.782+0000 7f89c54d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a46590 0x560a57a50910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:20.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.782+0000 7f89c54d0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a50e50 0x560a57a53240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:20.787 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.782+0000 7f89c54d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560a579ac3e0 con 0x560a57a46590 2024-09-10T08:41:20.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.782+0000 7f89c54d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560a57953d80 con 0x560a57a45c70 2024-09-10T08:41:20.788 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.782+0000 7f89c54d0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560a579a83a0 con 0x560a57a50e50 2024-09-10T08:41:20.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.782+0000 7f89bb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a45c70 0x560a57a46050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:20.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.782+0000 7f89bbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a46590 0x560a57a50910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:20.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.782+0000 7f89bb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a45c70 0x560a57a46050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:44960/0 (socket says 172.21.15.89:44960) 2024-09-10T08:41:20.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.782+0000 7f89bb400640 1 -- 172.21.15.89:0/65228982 learned_addr learned my addr 172.21.15.89:0/65228982 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:20.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.782+0000 7f89bb400640 1 --2- 172.21.15.89:0/65228982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a50e50 0x560a57a53240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:20.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.783+0000 7f89bbe00640 1 -- 172.21.15.89:0/65228982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a50e50 msgr2=0x560a57a53240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:20.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.783+0000 7f89bbe00640 1 --2- 172.21.15.89:0/65228982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a50e50 0x560a57a53240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.793 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.783+0000 7f89bbe00640 1 -- 172.21.15.89:0/65228982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a45c70 msgr2=0x560a57a46050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:20.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.783+0000 7f89bbe00640 1 --2- 172.21.15.89:0/65228982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a45c70 0x560a57a46050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.783+0000 7f89bbe00640 1 -- 172.21.15.89:0/65228982 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a57972ae0 con 0x560a57a46590 2024-09-10T08:41:20.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.783+0000 7f89b8c00640 1 --2- 172.21.15.89:0/2331346006 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f89a00a2b80 0x7f89a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:20.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.783+0000 7f89b8c00640 1 -- 172.21.15.89:0/2331346006 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f89b40cfab0 con 0x7f89bc106f10 2024-09-10T08:41:20.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.783+0000 7f89bbe00640 1 --2- 172.21.15.89:0/65228982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a46590 0x560a57a50910 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009c20 tx=0x7f89ac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efde3b560ec2198 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:20.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.783+0000 7f89bb400640 1 --2- 172.21.15.89:0/2331346006 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f89a00a2b80 0x7f89a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:20.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.783+0000 7f89b1200640 1 -- 172.21.15.89:0/65228982 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac004050 con 0x560a57a46590 2024-09-10T08:41:20.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.783+0000 7f89b1200640 1 -- 172.21.15.89:0/65228982 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89ac005500 con 0x560a57a46590 2024-09-10T08:41:20.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.783+0000 7f89b1200640 1 -- 172.21.15.89:0/65228982 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac004e00 con 0x560a57a46590 2024-09-10T08:41:20.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.784+0000 7f89c54d0740 1 -- 172.21.15.89:0/65228982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a46590 msgr2=0x560a57a50910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:20.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.784+0000 7f89bb400640 1 --2- 172.21.15.89:0/2331346006 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f89a00a2b80 0x7f89a00a5040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f89a8005ec0 tx=0x7f89a80314e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:20.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.784+0000 7f89c54d0740 1 --2- 172.21.15.89:0/65228982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a46590 0x560a57a50910 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009c20 tx=0x7f89ac011810 comp rx=0 tx=0).stop 2024-09-10T08:41:20.794 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.784+0000 7f89c54d0740 1 -- 172.21.15.89:0/65228982 shutdown_connections 2024-09-10T08:41:20.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.784+0000 7f89c54d0740 1 --2- 172.21.15.89:0/65228982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a50e50 0x560a57a53240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.784+0000 7f89c54d0740 1 --2- 172.21.15.89:0/65228982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a46590 0x560a57a50910 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.784+0000 7f89c54d0740 1 --2- 172.21.15.89:0/65228982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a45c70 0x560a57a46050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.784+0000 7f89c54d0740 1 -- 172.21.15.89:0/65228982 >> 172.21.15.89:0/65228982 conn(0x560a57a42200 msgr2=0x560a57a425e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:20.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.784+0000 7f89c54d0740 1 -- 172.21.15.89:0/65228982 shutdown_connections 2024-09-10T08:41:20.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.784+0000 7f89c54d0740 1 -- 172.21.15.89:0/65228982 wait complete. 2024-09-10T08:41:20.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.785+0000 7f89c54d0740 1 -- start start 2024-09-10T08:41:20.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.785+0000 7f89c54d0740 1 Processor -- start 2024-09-10T08:41:20.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.786+0000 7f89c54d0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a45c70 0x560a57a4dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:20.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.786+0000 7f89c54d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a46590 0x560a57a4e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:20.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.786+0000 7f89c54d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a50e50 0x560a57a4e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:20.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.786+0000 7f89c54d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560a57a53a20 con 0x560a57a50e50 2024-09-10T08:41:20.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.786+0000 7f89c54d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560a579670f0 con 0x560a57a46590 2024-09-10T08:41:20.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.786+0000 7f89c54d0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560a57a53ba0 con 0x560a57a45c70 2024-09-10T08:41:20.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.786+0000 7f89c54d0740 10 client.0 _ll_register_callbacks cb 0x560a5794ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:20.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.786+0000 7f89c0a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a50e50 0x560a57a4e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:20.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.786+0000 7f89c0a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a50e50 0x560a57a4e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38560/0 (socket says 172.21.15.89:38560) 2024-09-10T08:41:20.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.786+0000 7f89c0a00640 1 -- 172.21.15.89:0/820530988 learned_addr learned my addr 172.21.15.89:0/820530988 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:20.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.786+0000 7f89bb400640 1 --2- 172.21.15.89:0/820530988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a46590 0x560a57a4e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:20.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.786+0000 7f89bbe00640 1 --2- 172.21.15.89:0/820530988 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a45c70 0x560a57a4dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:20.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.786+0000 7f89c0a00640 1 -- 172.21.15.89:0/820530988 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a45c70 msgr2=0x560a57a4dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:20.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.786+0000 7f89c0a00640 1 --2- 172.21.15.89:0/820530988 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a45c70 0x560a57a4dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.786+0000 7f89c0a00640 1 -- 172.21.15.89:0/820530988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a46590 msgr2=0x560a57a4e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:20.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.786+0000 7f89c0a00640 1 --2- 172.21.15.89:0/820530988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a46590 0x560a57a4e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.786+0000 7f89c0a00640 1 -- 172.21.15.89:0/820530988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a5771d3c0 con 0x560a57a50e50 2024-09-10T08:41:20.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.786+0000 7f89bb400640 1 --2- 172.21.15.89:0/820530988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a46590 0x560a57a4e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:20.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.787+0000 7f89c0a00640 1 --2- 172.21.15.89:0/820530988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a50e50 0x560a57a4e7a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f89b4000f80 tx=0x7f89b409b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534ac393d5b2a0d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:20.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.787+0000 7f89a5600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:41:20.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.787+0000 7f89a5600640 1 -- 172.21.15.89:0/820530988 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4005510 con 0x560a57a50e50 2024-09-10T08:41:20.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.787+0000 7f89a5600640 1 -- 172.21.15.89:0/820530988 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f89b409b330 con 0x560a57a50e50 2024-09-10T08:41:20.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.787+0000 7f89c54d0740 1 -- 172.21.15.89:0/820530988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89ac005500 con 0x560a57a50e50 2024-09-10T08:41:20.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.787+0000 7f89a5600640 1 -- 172.21.15.89:0/820530988 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f89b40a0290 con 0x560a57a50e50 2024-09-10T08:41:20.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.787+0000 7f89c54d0740 10 client.15639 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:20.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.787+0000 7f89c54d0740 1 -- 172.21.15.89:0/820530988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560a57a5fa40 con 0x560a57a50e50 2024-09-10T08:41:20.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.787+0000 7f89a5600640 1 -- 172.21.15.89:0/820530988 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89ac005500 con 0x560a57a50e50 2024-09-10T08:41:20.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.788+0000 7f89a5600640 1 -- 172.21.15.89:0/820530988 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f89b409ada0 con 0x560a57a50e50 2024-09-10T08:41:20.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.788+0000 7f89c54d0740 10 client.15639 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:20.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.788+0000 7f89c54d0740 10 client.15639 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:41:20.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.788+0000 7f89c54d0740 20 client.15639 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:20.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.788+0000 7f89c54d0740 1 --2- 172.21.15.89:0/820530988 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x560a57a60e50 0x560a57a632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:20.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.788+0000 7f89c54d0740 4 client.15639 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:41:20.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.788+0000 7f89c54d0740 1 -- 172.21.15.89:0/820530988 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a579b9950 con 0x560a57a60e50 2024-09-10T08:41:20.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.788+0000 7f89bbe00640 1 --2- 172.21.15.89:0/820530988 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x560a57a60e50 0x560a57a632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:20.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.789+0000 7f89bbe00640 1 --2- 172.21.15.89:0/820530988 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x560a57a60e50 0x560a57a632c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=335586309a645379 server_cookie=5896cd6ec6b8908e in_seq=0 out_seq=0 2024-09-10T08:41:20.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.789+0000 7f89a5600640 10 client.15639 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:41:20.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.790+0000 7f89a5600640 1 -- 172.21.15.89:0/820530988 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a579b9950 con 0x560a57a60e50 2024-09-10T08:41:20.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.790+0000 7f89a5600640 10 client.15639 handle_command_reply: tid=0 2024-09-10T08:41:20.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.790+0000 7f89c54d0740 1 client.15639 shutdown 2024-09-10T08:41:20.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.790+0000 7f89c54d0740 1 -- 172.21.15.89:0/820530988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a50e50 msgr2=0x560a57a4e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:20.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.790+0000 7f89c54d0740 1 --2- 172.21.15.89:0/820530988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a50e50 0x560a57a4e7a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f89b4000f80 tx=0x7f89b409b840 comp rx=0 tx=0).stop 2024-09-10T08:41:20.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.790+0000 7f89c54d0740 1 -- 172.21.15.89:0/820530988 shutdown_connections 2024-09-10T08:41:20.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.790+0000 7f89c54d0740 1 --2- 172.21.15.89:0/820530988 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x560a57a60e50 0x560a57a632c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.790+0000 7f89c54d0740 1 --2- 172.21.15.89:0/820530988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a50e50 0x560a57a4e7a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.790+0000 7f89c54d0740 1 --2- 172.21.15.89:0/820530988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a46590 0x560a57a4e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.790+0000 7f89c54d0740 1 --2- 172.21.15.89:0/820530988 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a45c70 0x560a57a4dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.790+0000 7f89a5600640 0 client.15639 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:41:20.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.790+0000 7f89c54d0740 1 -- 172.21.15.89:0/820530988 >> 172.21.15.89:0/820530988 conn(0x560a57a42200 msgr2=0x560a57a47f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:20.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.791+0000 7f89c54d0740 1 -- 172.21.15.89:0/820530988 shutdown_connections 2024-09-10T08:41:20.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.791+0000 7f89c54d0740 1 -- 172.21.15.89:0/820530988 wait complete. 2024-09-10T08:41:20.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.791+0000 7f89c54d0740 20 client.15639 trim_cache size 0 max 16384 2024-09-10T08:41:20.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.804+0000 7f89c54d0740 1 Processor -- start 2024-09-10T08:41:20.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.804+0000 7f89c54d0740 1 -- start start 2024-09-10T08:41:20.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.804+0000 7f89c54d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a42200 0x560a57a4c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:20.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.804+0000 7f89c54d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a45c70 0x560a57a68bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:20.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.804+0000 7f89c54d0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a46590 0x560a57a690f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:20.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.804+0000 7f89c54d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1b39b0 con 0x560a57a42200 2024-09-10T08:41:20.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.804+0000 7f89c54d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1b1280 con 0x560a57a45c70 2024-09-10T08:41:20.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.804+0000 7f89c54d0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1af0d0 con 0x560a57a46590 2024-09-10T08:41:20.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.805+0000 7f89bbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a42200 0x560a57a4c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.805+0000 7f89c0a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a46590 0x560a57a690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.805+0000 7f89bb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a45c70 0x560a57a68bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.805+0000 7f89bbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a42200 0x560a57a4c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38566/0 (socket says 172.21.15.89:38566) 2024-09-10T08:41:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.805+0000 7f89c0a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a46590 0x560a57a690f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:33094/0 (socket says 172.21.15.89:33094) 2024-09-10T08:41:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.805+0000 7f89bbe00640 1 -- 172.21.15.89:0/3775922739 learned_addr learned my addr 172.21.15.89:0/3775922739 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.805+0000 7f89bb400640 1 -- 172.21.15.89:0/3775922739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a46590 msgr2=0x560a57a690f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:20.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.805+0000 7f89bb400640 1 --2- 172.21.15.89:0/3775922739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a46590 0x560a57a690f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.805+0000 7f89bb400640 1 -- 172.21.15.89:0/3775922739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a42200 msgr2=0x560a57a4c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:20.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.805+0000 7f89bb400640 1 --2- 172.21.15.89:0/3775922739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a42200 0x560a57a4c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.805+0000 7f89bb400640 1 -- 172.21.15.89:0/3775922739 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a57a4ccd0 con 0x560a57a45c70 2024-09-10T08:41:20.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.805+0000 7f89bbe00640 1 --2- 172.21.15.89:0/3775922739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a42200 0x560a57a4c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:20.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.806+0000 7f89bb400640 1 --2- 172.21.15.89:0/3775922739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a45c70 0x560a57a68bb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f89a8031550 tx=0x7f89a8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbb09dd421d5313c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:20.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.806+0000 7f89a7e00640 1 -- 172.21.15.89:0/3775922739 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8035040 con 0x560a57a45c70 2024-09-10T08:41:20.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.806+0000 7f89a7e00640 1 -- 172.21.15.89:0/3775922739 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89a8003ad0 con 0x560a57a45c70 2024-09-10T08:41:20.809 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.806+0000 7f89a7e00640 1 -- 172.21.15.89:0/3775922739 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a803db70 con 0x560a57a45c70 2024-09-10T08:41:20.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.806+0000 7f89c54d0740 1 -- 172.21.15.89:0/3775922739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a45c70 msgr2=0x560a57a68bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:20.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.806+0000 7f89c54d0740 1 --2- 172.21.15.89:0/3775922739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a45c70 0x560a57a68bb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f89a8031550 tx=0x7f89a8009260 comp rx=0 tx=0).stop 2024-09-10T08:41:20.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.806+0000 7f89c54d0740 1 -- 172.21.15.89:0/3775922739 shutdown_connections 2024-09-10T08:41:20.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.806+0000 7f89c54d0740 1 --2- 172.21.15.89:0/3775922739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a46590 0x560a57a690f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.806+0000 7f89c54d0740 1 --2- 172.21.15.89:0/3775922739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a45c70 0x560a57a68bb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.806+0000 7f89c54d0740 1 --2- 172.21.15.89:0/3775922739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a42200 0x560a57a4c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.806+0000 7f89c54d0740 1 -- 172.21.15.89:0/3775922739 >> 172.21.15.89:0/3775922739 conn(0x560a57a49710 msgr2=0x560a57a4fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:20.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.807+0000 7f89c54d0740 1 -- 172.21.15.89:0/3775922739 shutdown_connections 2024-09-10T08:41:20.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.807+0000 7f89c54d0740 1 -- 172.21.15.89:0/3775922739 wait complete. 2024-09-10T08:41:20.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.807+0000 7f89c54d0740 1 -- start start 2024-09-10T08:41:20.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.807+0000 7f89c54d0740 1 Processor -- start 2024-09-10T08:41:20.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.808+0000 7f89c54d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a42200 0x560a57a595c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:20.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.808+0000 7f89c54d0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a45c70 0x560a57a54990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:20.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.808+0000 7f89c54d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a46590 0x560a57a54ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:20.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.808+0000 7f89c54d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1af740 con 0x560a57a46590 2024-09-10T08:41:20.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.808+0000 7f89c54d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1b0bb0 con 0x560a57a42200 2024-09-10T08:41:20.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.808+0000 7f89c54d0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1afe10 con 0x560a57a45c70 2024-09-10T08:41:20.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.808+0000 7f89c54d0740 10 client.0 _ll_register_callbacks cb 0x7f89b409bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:20.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.808+0000 7f89bbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a42200 0x560a57a595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:20.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.808+0000 7f89c0a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a46590 0x560a57a54ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:20.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.808+0000 7f89bbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a42200 0x560a57a595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:44996/0 (socket says 172.21.15.89:44996) 2024-09-10T08:41:20.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.808+0000 7f89bbe00640 1 -- 172.21.15.89:0/3261665574 learned_addr learned my addr 172.21.15.89:0/3261665574 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:20.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.808+0000 7f89bb400640 1 --2- 172.21.15.89:0/3261665574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a45c70 0x560a57a54990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:20.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.808+0000 7f89bbe00640 1 -- 172.21.15.89:0/3261665574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a45c70 msgr2=0x560a57a54990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:20.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.808+0000 7f89bbe00640 1 --2- 172.21.15.89:0/3261665574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a45c70 0x560a57a54990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.808+0000 7f89bbe00640 1 -- 172.21.15.89:0/3261665574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a46590 msgr2=0x560a57a54ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:20.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.808+0000 7f89bbe00640 1 --2- 172.21.15.89:0/3261665574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a46590 0x560a57a54ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.808+0000 7f89bbe00640 1 -- 172.21.15.89:0/3261665574 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a57a55720 con 0x560a57a42200 2024-09-10T08:41:20.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.808+0000 7f89c0a00640 1 --2- 172.21.15.89:0/3261665574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a46590 0x560a57a54ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:20.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.809+0000 7f89bbe00640 1 --2- 172.21.15.89:0/3261665574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a42200 0x560a57a595c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f89ac01b040 tx=0x7f89ac009e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7e5dea20962aec6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:20.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.809+0000 7f89a5600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:41:20.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.809+0000 7f89a5600640 1 -- 172.21.15.89:0/3261665574 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac004450 con 0x560a57a42200 2024-09-10T08:41:20.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.809+0000 7f89a5600640 1 -- 172.21.15.89:0/3261665574 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f89ac005040 con 0x560a57a42200 2024-09-10T08:41:20.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.809+0000 7f89c54d0740 1 -- 172.21.15.89:0/3261665574 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89a8003ad0 con 0x560a57a42200 2024-09-10T08:41:20.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.809+0000 7f89a5600640 1 -- 172.21.15.89:0/3261665574 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f89ac005240 con 0x560a57a42200 2024-09-10T08:41:20.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.809+0000 7f89c54d0740 10 client.25709 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:20.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.809+0000 7f89c54d0740 1 -- 172.21.15.89:0/3261665574 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560a57acbc10 con 0x560a57a42200 2024-09-10T08:41:20.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.809+0000 7f89a5600640 1 -- 172.21.15.89:0/3261665574 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89a8003ad0 con 0x560a57a42200 2024-09-10T08:41:20.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.810+0000 7f89a5600640 1 -- 172.21.15.89:0/3261665574 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f89ac0035f0 con 0x560a57a42200 2024-09-10T08:41:20.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.810+0000 7f89c54d0740 10 client.25709 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:20.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.810+0000 7f89c54d0740 10 client.25709 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:41:20.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.810+0000 7f89c54d0740 20 client.25709 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:20.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.810+0000 7f89c54d0740 1 --2- 172.21.15.89:0/3261665574 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x560a57a50e50 0x560a57ace1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:20.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.810+0000 7f89c54d0740 4 client.25709 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:41:20.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.810+0000 7f89c54d0740 1 -- 172.21.15.89:0/3261665574 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a579b3e60 con 0x560a57a50e50 2024-09-10T08:41:20.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.810+0000 7f89bb400640 1 --2- 172.21.15.89:0/3261665574 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x560a57a50e50 0x560a57ace1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:20.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.811+0000 7f89bb400640 1 --2- 172.21.15.89:0/3261665574 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x560a57a50e50 0x560a57ace1b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49a84ff96b951cc6 server_cookie=e8934a11997e1e42 in_seq=0 out_seq=0 2024-09-10T08:41:20.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.811+0000 7f89a5600640 10 client.25709 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:41:20.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.811+0000 7f89a5600640 1 -- 172.21.15.89:0/3261665574 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a579b3e60 con 0x560a57a50e50 2024-09-10T08:41:20.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.811+0000 7f89a5600640 10 client.25709 handle_command_reply: tid=0 2024-09-10T08:41:20.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.811+0000 7f89c54d0740 1 client.25709 shutdown 2024-09-10T08:41:20.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.811+0000 7f89c54d0740 1 -- 172.21.15.89:0/3261665574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a42200 msgr2=0x560a57a595c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:20.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.811+0000 7f89c54d0740 1 --2- 172.21.15.89:0/3261665574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a42200 0x560a57a595c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f89ac01b040 tx=0x7f89ac009e20 comp rx=0 tx=0).stop 2024-09-10T08:41:20.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.811+0000 7f89c54d0740 1 -- 172.21.15.89:0/3261665574 shutdown_connections 2024-09-10T08:41:20.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.811+0000 7f89c54d0740 1 --2- 172.21.15.89:0/3261665574 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x560a57a50e50 0x560a57ace1b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.811+0000 7f89c54d0740 1 --2- 172.21.15.89:0/3261665574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560a57a46590 0x560a57a54ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.811+0000 7f89c54d0740 1 --2- 172.21.15.89:0/3261665574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560a57a45c70 0x560a57a54990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.811+0000 7f89c54d0740 1 --2- 172.21.15.89:0/3261665574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a57a42200 0x560a57a595c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.811+0000 7f89c54d0740 1 -- 172.21.15.89:0/3261665574 >> 172.21.15.89:0/3261665574 conn(0x560a57a49710 msgr2=0x560a57a4fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:20.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.811+0000 7f89a5600640 0 client.25709 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:41:20.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.811+0000 7f89c54d0740 1 -- 172.21.15.89:0/3261665574 shutdown_connections 2024-09-10T08:41:20.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.811+0000 7f89c54d0740 1 -- 172.21.15.89:0/3261665574 wait complete. 2024-09-10T08:41:20.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.811+0000 7f89c54d0740 20 client.25709 trim_cache size 0 max 16384 2024-09-10T08:41:20.822 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:41:20.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.812+0000 7f89c2a00640 1 -- 172.21.15.89:0/2331346006 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f89a00a2b80 msgr2=0x7f89a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:20.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.812+0000 7f89c2a00640 1 --2- 172.21.15.89:0/2331346006 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f89a00a2b80 0x7f89a00a5040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f89a8005ec0 tx=0x7f89a80314e0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.812+0000 7f89c2a00640 1 -- 172.21.15.89:0/2331346006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89bc106f10 msgr2=0x7f89bc19ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:20.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.812+0000 7f89c2a00640 1 --2- 172.21.15.89:0/2331346006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89bc106f10 0x7f89bc19ef20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f89b400eb10 tx=0x7f89b400c2d0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.812+0000 7f89c2a00640 1 -- 172.21.15.89:0/2331346006 shutdown_connections 2024-09-10T08:41:20.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.812+0000 7f89c2a00640 1 --2- 172.21.15.89:0/2331346006 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f89a00a2b80 0x7f89a00a5040 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.812+0000 7f89c2a00640 1 --2- 172.21.15.89:0/2331346006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89bc106f10 0x7f89bc19ef20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.812+0000 7f89c2a00640 1 --2- 172.21.15.89:0/2331346006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f89bc102790 0x7f89bc19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.812+0000 7f89c2a00640 1 --2- 172.21.15.89:0/2331346006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89bc101de0 0x7f89bc19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:20.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.812+0000 7f89c2a00640 1 -- 172.21.15.89:0/2331346006 >> 172.21.15.89:0/2331346006 conn(0x7f89bc0fd8c0 msgr2=0x7f89bc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:20.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.812+0000 7f89c2a00640 1 -- 172.21.15.89:0/2331346006 shutdown_connections 2024-09-10T08:41:20.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:20.812+0000 7f89c2a00640 1 -- 172.21.15.89:0/2331346006 wait complete. 2024-09-10T08:41:21.699 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:41:21.816 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.812+0000 7f0e0a600640 1 -- 172.21.15.89:0/3900014711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e04104f50 msgr2=0x7f0e041053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.812+0000 7f0e0a600640 1 --2- 172.21.15.89:0/3900014711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e04104f50 0x7f0e041053d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009900 tx=0x7f0df002a410 comp rx=0 tx=0).stop 2024-09-10T08:41:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.812+0000 7f0e0a600640 1 -- 172.21.15.89:0/3900014711 shutdown_connections 2024-09-10T08:41:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.812+0000 7f0e0a600640 1 --2- 172.21.15.89:0/3900014711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e04105910 0x7f0e0410c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.812+0000 7f0e0a600640 1 --2- 172.21.15.89:0/3900014711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e04104f50 0x7f0e041053d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.812+0000 7f0e0a600640 1 --2- 172.21.15.89:0/3900014711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e04103d60 0x7f0e04104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.812+0000 7f0e0a600640 1 -- 172.21.15.89:0/3900014711 >> 172.21.15.89:0/3900014711 conn(0x7f0e040ff4e0 msgr2=0x7f0e04101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.812+0000 7f0e0a600640 1 -- 172.21.15.89:0/3900014711 shutdown_connections 2024-09-10T08:41:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.812+0000 7f0e0a600640 1 -- 172.21.15.89:0/3900014711 wait complete. 2024-09-10T08:41:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.813+0000 7f0e0a600640 1 Processor -- start 2024-09-10T08:41:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.813+0000 7f0e0a600640 1 -- start start 2024-09-10T08:41:21.819 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.813+0000 7f0e0a600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e04103d60 0x7f0e04199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:21.819 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.813+0000 7f0e0a600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e04104f50 0x7f0e0419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:21.819 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.813+0000 7f0e0a600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e04105910 0x7f0e041a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:21.819 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.813+0000 7f0e0a600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0410f9e0 con 0x7f0e04104f50 2024-09-10T08:41:21.819 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.813+0000 7f0e0a600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0410f860 con 0x7f0e04105910 2024-09-10T08:41:21.819 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.813+0000 7f0e0a600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0410fb60 con 0x7f0e04103d60 2024-09-10T08:41:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.813+0000 7f0e02a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e04104f50 0x7f0e0419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.813+0000 7f0e02a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e04104f50 0x7f0e0419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38612/0 (socket says 172.21.15.89:38612) 2024-09-10T08:41:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.813+0000 7f0e02a00640 1 -- 172.21.15.89:0/4206632402 learned_addr learned my addr 172.21.15.89:0/4206632402 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.814+0000 7f0e03400640 1 --2- 172.21.15.89:0/4206632402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e04103d60 0x7f0e04199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.814+0000 7f0e03e00640 1 --2- 172.21.15.89:0/4206632402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e04105910 0x7f0e041a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.814+0000 7f0e02a00640 1 -- 172.21.15.89:0/4206632402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e04103d60 msgr2=0x7f0e04199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.814+0000 7f0e02a00640 1 --2- 172.21.15.89:0/4206632402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e04103d60 0x7f0e04199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.814+0000 7f0e02a00640 1 -- 172.21.15.89:0/4206632402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e04105910 msgr2=0x7f0e041a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.814+0000 7f0e02a00640 1 --2- 172.21.15.89:0/4206632402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e04105910 0x7f0e041a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.814+0000 7f0e02a00640 1 -- 172.21.15.89:0/4206632402 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e041a1b90 con 0x7f0e04104f50 2024-09-10T08:41:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.814+0000 7f0e02a00640 1 --2- 172.21.15.89:0/4206632402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e04104f50 0x7f0e0419a4b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009230 tx=0x7f0df0031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.814+0000 7f0dfbe00640 1 -- 172.21.15.89:0/4206632402 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df003f030 con 0x7f0e04104f50 2024-09-10T08:41:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.814+0000 7f0dfbe00640 1 -- 172.21.15.89:0/4206632402 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0df0003ad0 con 0x7f0e04104f50 2024-09-10T08:41:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.814+0000 7f0e0a600640 1 -- 172.21.15.89:0/4206632402 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e041a1e20 con 0x7f0e04104f50 2024-09-10T08:41:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.815+0000 7f0dfbe00640 1 -- 172.21.15.89:0/4206632402 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df0035070 con 0x7f0e04104f50 2024-09-10T08:41:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.815+0000 7f0e0a600640 1 -- 172.21.15.89:0/4206632402 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e041a2290 con 0x7f0e04104f50 2024-09-10T08:41:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.818+0000 7f0dfbe00640 1 -- 172.21.15.89:0/4206632402 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0df0042030 con 0x7f0e04104f50 2024-09-10T08:41:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.819+0000 7f0e0d5fe740 1 Processor -- start 2024-09-10T08:41:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.819+0000 7f0dfbe00640 1 --2- 172.21.15.89:0/4206632402 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0de80a2cb0 0x7f0de80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.819+0000 7f0e0d5fe740 1 -- start start 2024-09-10T08:41:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.819+0000 7f0dfbe00640 1 -- 172.21.15.89:0/4206632402 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f0df00ef890 con 0x7f0e04104f50 2024-09-10T08:41:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.819+0000 7f0e03e00640 1 --2- 172.21.15.89:0/4206632402 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0de80a2cb0 0x7f0de80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.819+0000 7f0e0d5fe740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d1929090 0x55b4d1929490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.819+0000 7f0e0d5fe740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d19299d0 0x55b4d192e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.819+0000 7f0e0d5fe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d192ea60 0x55b4d1930f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.819+0000 7f0e0d5fe740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b4d188e100 con 0x55b4d19299d0 2024-09-10T08:41:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.819+0000 7f0e0d5fe740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b4d18361b0 con 0x55b4d1929090 2024-09-10T08:41:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.819+0000 7f0e0d5fe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b4d188a0c0 con 0x55b4d192ea60 2024-09-10T08:41:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.820+0000 7f0e02a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d19299d0 0x55b4d192e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.820+0000 7f0e03400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d1929090 0x55b4d1929490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.820+0000 7f0e03400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d1929090 0x55b4d1929490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:45022/0 (socket says 172.21.15.89:45022) 2024-09-10T08:41:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.820+0000 7f0e03400640 1 -- 172.21.15.89:0/797559082 learned_addr learned my addr 172.21.15.89:0/797559082 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.820+0000 7f0e03e00640 1 --2- 172.21.15.89:0/797559082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d192ea60 0x55b4d1930f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.820+0000 7f0e03400640 1 -- 172.21.15.89:0/797559082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d192ea60 msgr2=0x55b4d1930f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.820+0000 7f0e03400640 1 --2- 172.21.15.89:0/797559082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d192ea60 0x55b4d1930f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.820+0000 7f0e03400640 1 -- 172.21.15.89:0/797559082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d19299d0 msgr2=0x55b4d192e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.820+0000 7f0e03400640 1 --2- 172.21.15.89:0/797559082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d19299d0 0x55b4d192e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.820+0000 7f0e03400640 1 -- 172.21.15.89:0/797559082 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4d15a1210 con 0x55b4d1929090 2024-09-10T08:41:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.820+0000 7f0e02a00640 1 --2- 172.21.15.89:0/797559082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d19299d0 0x55b4d192e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.820+0000 7f0e03e00640 1 --2- 172.21.15.89:0/797559082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d192ea60 0x55b4d1930f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:41:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.820+0000 7f0e03e00640 1 --2- 172.21.15.89:0/4206632402 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0de80a2cb0 0x7f0de80a5170 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc005fd0 tx=0x7f0dfc009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.820+0000 7f0e03400640 1 --2- 172.21.15.89:0/797559082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d1929090 0x55b4d1929490 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009ca0 tx=0x7f0df4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=828619f35a0d2aab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.821+0000 7f0df8c00640 1 -- 172.21.15.89:0/797559082 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4004220 con 0x55b4d1929090 2024-09-10T08:41:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.821+0000 7f0df8c00640 1 -- 172.21.15.89:0/797559082 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0df40043c0 con 0x55b4d1929090 2024-09-10T08:41:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.821+0000 7f0df8c00640 1 -- 172.21.15.89:0/797559082 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4019930 con 0x55b4d1929090 2024-09-10T08:41:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.821+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/797559082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d1929090 msgr2=0x55b4d1929490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.821+0000 7f0e0d5fe740 1 --2- 172.21.15.89:0/797559082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d1929090 0x55b4d1929490 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009ca0 tx=0x7f0df4011810 comp rx=0 tx=0).stop 2024-09-10T08:41:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.822+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/797559082 shutdown_connections 2024-09-10T08:41:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.822+0000 7f0e0d5fe740 1 --2- 172.21.15.89:0/797559082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d192ea60 0x55b4d1930f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.822+0000 7f0e0d5fe740 1 --2- 172.21.15.89:0/797559082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d19299d0 0x55b4d192e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.822+0000 7f0e0d5fe740 1 --2- 172.21.15.89:0/797559082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d1929090 0x55b4d1929490 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.822+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/797559082 >> 172.21.15.89:0/797559082 conn(0x55b4d1924700 msgr2=0x55b4d1924b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.822+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/797559082 shutdown_connections 2024-09-10T08:41:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.822+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/797559082 wait complete. 2024-09-10T08:41:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.822+0000 7f0e0d5fe740 1 -- start start 2024-09-10T08:41:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.822+0000 7f0e0d5fe740 1 Processor -- start 2024-09-10T08:41:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.823+0000 7f0e0d5fe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d1929090 0x55b4d1933160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.823+0000 7f0e0d5fe740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d19299d0 0x55b4d19336a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.823+0000 7f0e0d5fe740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d192ea60 0x55b4d193a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.823+0000 7f0e0d5fe740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b4d1931a30 con 0x55b4d19299d0 2024-09-10T08:41:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.823+0000 7f0e0d5fe740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b4d1817090 con 0x55b4d192ea60 2024-09-10T08:41:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.823+0000 7f0e0d5fe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b4d1931bb0 con 0x55b4d1929090 2024-09-10T08:41:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.823+0000 7f0e0d5fe740 10 client.0 _ll_register_callbacks cb 0x55b4d186bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.823+0000 7f0e02a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d19299d0 0x55b4d19336a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.823+0000 7f0e02a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d19299d0 0x55b4d19336a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38636/0 (socket says 172.21.15.89:38636) 2024-09-10T08:41:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.823+0000 7f0e02a00640 1 -- 172.21.15.89:0/3703531541 learned_addr learned my addr 172.21.15.89:0/3703531541 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.823+0000 7f0e03e00640 1 --2- 172.21.15.89:0/3703531541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d192ea60 0x55b4d193a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.823+0000 7f0e03400640 1 --2- 172.21.15.89:0/3703531541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d1929090 0x55b4d1933160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.823+0000 7f0e02a00640 1 -- 172.21.15.89:0/3703531541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d1929090 msgr2=0x55b4d1933160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.823+0000 7f0e02a00640 1 --2- 172.21.15.89:0/3703531541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d1929090 0x55b4d1933160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.823+0000 7f0e02a00640 1 -- 172.21.15.89:0/3703531541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d192ea60 msgr2=0x55b4d193a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.823+0000 7f0e02a00640 1 --2- 172.21.15.89:0/3703531541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d192ea60 0x55b4d193a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.823+0000 7f0e02a00640 1 -- 172.21.15.89:0/3703531541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4d166d620 con 0x55b4d19299d0 2024-09-10T08:41:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.823+0000 7f0e03e00640 1 --2- 172.21.15.89:0/3703531541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d192ea60 0x55b4d193a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:41:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.824+0000 7f0e02a00640 1 --2- 172.21.15.89:0/3703531541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d19299d0 0x55b4d19336a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0df0031440 tx=0x7f0df00c01b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958a4bf8fede918 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.824+0000 7f0ded600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:41:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.824+0000 7f0ded600640 1 -- 172.21.15.89:0/3703531541 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df0049040 con 0x55b4d19299d0 2024-09-10T08:41:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.824+0000 7f0ded600640 1 -- 172.21.15.89:0/3703531541 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f0df0003c80 con 0x55b4d19299d0 2024-09-10T08:41:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.824+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/3703531541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0df40043c0 con 0x55b4d19299d0 2024-09-10T08:41:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.824+0000 7f0ded600640 1 -- 172.21.15.89:0/3703531541 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0df003bc70 con 0x55b4d19299d0 2024-09-10T08:41:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.824+0000 7f0e0d5fe740 10 client.15657 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.824+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/3703531541 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b4d193d440 con 0x55b4d19299d0 2024-09-10T08:41:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.824+0000 7f0ded600640 1 -- 172.21.15.89:0/3703531541 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0df40043c0 con 0x55b4d19299d0 2024-09-10T08:41:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.825+0000 7f0ded600640 1 -- 172.21.15.89:0/3703531541 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0df00bbfd0 con 0x55b4d19299d0 2024-09-10T08:41:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.825+0000 7f0e0d5fe740 10 client.15657 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.825+0000 7f0e0d5fe740 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:41:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.825+0000 7f0e0d5fe740 20 client.15657 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.825+0000 7f0e0d5fe740 1 --2- 172.21.15.89:0/3703531541 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b4d193ebc0 0x55b4d1941010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.825+0000 7f0e0d5fe740 4 client.15657 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:41:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.825+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/3703531541 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4d189b830 con 0x55b4d193ebc0 2024-09-10T08:41:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.825+0000 7f0e03400640 1 --2- 172.21.15.89:0/3703531541 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b4d193ebc0 0x55b4d1941010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.826+0000 7f0e03400640 1 --2- 172.21.15.89:0/3703531541 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b4d193ebc0 0x55b4d1941010 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=993b849ec0141baf server_cookie=2c41340da87ca518 in_seq=0 out_seq=0 2024-09-10T08:41:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.826+0000 7f0ded600640 10 client.15657 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.826+0000 7f0ded600640 1 -- 172.21.15.89:0/3703531541 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4d189b830 con 0x55b4d193ebc0 2024-09-10T08:41:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.826+0000 7f0ded600640 10 client.15657 handle_command_reply: tid=0 2024-09-10T08:41:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.827+0000 7f0e0d5fe740 1 client.15657 shutdown 2024-09-10T08:41:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.827+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/3703531541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d19299d0 msgr2=0x55b4d19336a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.827+0000 7f0e0d5fe740 1 --2- 172.21.15.89:0/3703531541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d19299d0 0x55b4d19336a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0df0031440 tx=0x7f0df00c01b0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.827+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/3703531541 shutdown_connections 2024-09-10T08:41:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.827+0000 7f0e0d5fe740 1 --2- 172.21.15.89:0/3703531541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d192ea60 0x55b4d193a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.827+0000 7f0e0d5fe740 1 --2- 172.21.15.89:0/3703531541 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b4d193ebc0 0x55b4d1941010 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.827+0000 7f0e0d5fe740 1 --2- 172.21.15.89:0/3703531541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d19299d0 0x55b4d19336a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.827+0000 7f0e0d5fe740 1 --2- 172.21.15.89:0/3703531541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d1929090 0x55b4d1933160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.827+0000 7f0ded600640 0 client.15657 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.827+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/3703531541 >> 172.21.15.89:0/3703531541 conn(0x55b4d1924700 msgr2=0x55b4d1925f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.827+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/3703531541 shutdown_connections 2024-09-10T08:41:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.827+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/3703531541 wait complete. 2024-09-10T08:41:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.827+0000 7f0e0d5fe740 20 client.15657 trim_cache size 0 max 16384 2024-09-10T08:41:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.842+0000 7f0e0d5fe740 1 Processor -- start 2024-09-10T08:41:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.842+0000 7f0e0d5fe740 1 -- start start 2024-09-10T08:41:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.842+0000 7f0e0d5fe740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d1929090 0x55b4d19255b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.842+0000 7f0e0d5fe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d19299d0 0x55b4d193a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.842+0000 7f0e0d5fe740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d192ea60 0x55b4d1925af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.842+0000 7f0e0d5fe740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0e041af220 con 0x55b4d192ea60 2024-09-10T08:41:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.842+0000 7f0e0d5fe740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0e041acaf0 con 0x55b4d1929090 2024-09-10T08:41:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.842+0000 7f0e0d5fe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0e041aa940 con 0x55b4d19299d0 2024-09-10T08:41:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.842+0000 7f0e03e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d192ea60 0x55b4d1925af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.843+0000 7f0e02a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d19299d0 0x55b4d193a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.843+0000 7f0e03e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d192ea60 0x55b4d1925af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38642/0 (socket says 172.21.15.89:38642) 2024-09-10T08:41:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.843+0000 7f0e03e00640 1 -- 172.21.15.89:0/2211815708 learned_addr learned my addr 172.21.15.89:0/2211815708 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.843+0000 7f0e03400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d1929090 0x55b4d19255b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.843+0000 7f0e02a00640 1 -- 172.21.15.89:0/2211815708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d1929090 msgr2=0x55b4d19255b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.843+0000 7f0e02a00640 1 --2- 172.21.15.89:0/2211815708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d1929090 0x55b4d19255b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.843+0000 7f0e02a00640 1 -- 172.21.15.89:0/2211815708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d192ea60 msgr2=0x55b4d1925af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.843+0000 7f0e02a00640 1 --2- 172.21.15.89:0/2211815708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d192ea60 0x55b4d1925af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.843+0000 7f0e02a00640 1 -- 172.21.15.89:0/2211815708 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4d1929eb0 con 0x55b4d19299d0 2024-09-10T08:41:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.843+0000 7f0e03e00640 1 --2- 172.21.15.89:0/2211815708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d192ea60 0x55b4d1925af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.843+0000 7f0e02a00640 1 --2- 172.21.15.89:0/2211815708 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d19299d0 0x55b4d193a6a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0df009af30 tx=0x7f0df00057d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8181a23ec5bba9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.843+0000 7f0defe00640 1 -- 172.21.15.89:0/2211815708 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df0049040 con 0x55b4d19299d0 2024-09-10T08:41:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.843+0000 7f0defe00640 1 -- 172.21.15.89:0/2211815708 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0df00b83e0 con 0x55b4d19299d0 2024-09-10T08:41:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.843+0000 7f0defe00640 1 -- 172.21.15.89:0/2211815708 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df0003c80 con 0x55b4d19299d0 2024-09-10T08:41:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.844+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/2211815708 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d19299d0 msgr2=0x55b4d193a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.844+0000 7f0e0d5fe740 1 --2- 172.21.15.89:0/2211815708 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d19299d0 0x55b4d193a6a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0df009af30 tx=0x7f0df00057d0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.844+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/2211815708 shutdown_connections 2024-09-10T08:41:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.844+0000 7f0e0d5fe740 1 --2- 172.21.15.89:0/2211815708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d192ea60 0x55b4d1925af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.844+0000 7f0e0d5fe740 1 --2- 172.21.15.89:0/2211815708 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d19299d0 0x55b4d193a6a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.844+0000 7f0e0d5fe740 1 --2- 172.21.15.89:0/2211815708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d1929090 0x55b4d19255b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.844+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/2211815708 >> 172.21.15.89:0/2211815708 conn(0x55b4d1924700 msgr2=0x55b4d1933160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.844+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/2211815708 shutdown_connections 2024-09-10T08:41:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.844+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/2211815708 wait complete. 2024-09-10T08:41:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.845+0000 7f0e0d5fe740 1 -- start start 2024-09-10T08:41:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.845+0000 7f0e0d5fe740 1 Processor -- start 2024-09-10T08:41:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.845+0000 7f0e0d5fe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d1929090 0x55b4d1937960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.845+0000 7f0e0d5fe740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d19299d0 0x55b4d1937ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.845+0000 7f0e0d5fe740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d192ea60 0x55b4d1938eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.845+0000 7f0e0d5fe740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0e041aafb0 con 0x55b4d19299d0 2024-09-10T08:41:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.845+0000 7f0e0d5fe740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0e041ac420 con 0x55b4d192ea60 2024-09-10T08:41:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.845+0000 7f0e0d5fe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0e041ab680 con 0x55b4d1929090 2024-09-10T08:41:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.845+0000 7f0e0d5fe740 10 client.0 _ll_register_callbacks cb 0x7f0df00bbbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.846+0000 7f0e03400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d1929090 0x55b4d1937960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.846+0000 7f0e03400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d1929090 0x55b4d1937960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:33168/0 (socket says 172.21.15.89:33168) 2024-09-10T08:41:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.846+0000 7f0e02a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d19299d0 0x55b4d1937ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.846+0000 7f0e03400640 1 -- 172.21.15.89:0/2055699831 learned_addr learned my addr 172.21.15.89:0/2055699831 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.846+0000 7f0e03e00640 1 --2- 172.21.15.89:0/2055699831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d192ea60 0x55b4d1938eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.846+0000 7f0e03400640 1 -- 172.21.15.89:0/2055699831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d192ea60 msgr2=0x55b4d1938eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.846+0000 7f0e03400640 1 --2- 172.21.15.89:0/2055699831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d192ea60 0x55b4d1938eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.846+0000 7f0e03400640 1 -- 172.21.15.89:0/2055699831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d19299d0 msgr2=0x55b4d1937ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.846+0000 7f0e03400640 1 --2- 172.21.15.89:0/2055699831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d19299d0 0x55b4d1937ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.846+0000 7f0e03400640 1 -- 172.21.15.89:0/2055699831 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4d19394c0 con 0x55b4d1929090 2024-09-10T08:41:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.846+0000 7f0e03400640 1 --2- 172.21.15.89:0/2055699831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d1929090 0x55b4d1937960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0df4000c00 tx=0x7f0df4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=333d2b269cc3e47c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.846+0000 7f0ded600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:41:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.846+0000 7f0ded600640 1 -- 172.21.15.89:0/2055699831 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4005460 con 0x55b4d1929090 2024-09-10T08:41:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.846+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/2055699831 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0df00b83e0 con 0x55b4d1929090 2024-09-10T08:41:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.846+0000 7f0ded600640 1 -- 172.21.15.89:0/2055699831 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f0df40035f0 con 0x55b4d1929090 2024-09-10T08:41:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.847+0000 7f0ded600640 1 -- 172.21.15.89:0/2055699831 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0df403c050 con 0x55b4d1929090 2024-09-10T08:41:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.847+0000 7f0e0d5fe740 10 client.25672 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.847+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/2055699831 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b4d193a840 con 0x55b4d1929090 2024-09-10T08:41:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.847+0000 7f0ded600640 1 -- 172.21.15.89:0/2055699831 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0df00b83e0 con 0x55b4d1929090 2024-09-10T08:41:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.847+0000 7f0ded600640 1 -- 172.21.15.89:0/2055699831 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0df4019880 con 0x55b4d1929090 2024-09-10T08:41:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.847+0000 7f0e0d5fe740 10 client.25672 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.847+0000 7f0e0d5fe740 10 client.25672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:41:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.847+0000 7f0e0d5fe740 20 client.25672 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.847+0000 7f0e0d5fe740 1 --2- 172.21.15.89:0/2055699831 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b4d19a9d80 0x55b4d19ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.847+0000 7f0e0d5fe740 4 client.25672 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:41:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.847+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/2055699831 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b4d1895b80 con 0x55b4d19a9d80 2024-09-10T08:41:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.848+0000 7f0e02a00640 1 --2- 172.21.15.89:0/2055699831 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b4d19a9d80 0x55b4d19ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.848+0000 7f0e02a00640 1 --2- 172.21.15.89:0/2055699831 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b4d19a9d80 0x55b4d19ac1a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2e67af5278dae84 server_cookie=9a0cc64c112c9f87 in_seq=0 out_seq=0 2024-09-10T08:41:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.848+0000 7f0ded600640 10 client.25672 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.852+0000 7f0ded600640 1 -- 172.21.15.89:0/2055699831 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b4d1895b80 con 0x55b4d19a9d80 2024-09-10T08:41:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.852+0000 7f0ded600640 10 client.25672 handle_command_reply: tid=0 2024-09-10T08:41:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.852+0000 7f0e0d5fe740 1 client.25672 shutdown 2024-09-10T08:41:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.852+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/2055699831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d1929090 msgr2=0x55b4d1937960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.852+0000 7f0e0d5fe740 1 --2- 172.21.15.89:0/2055699831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d1929090 0x55b4d1937960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0df4000c00 tx=0x7f0df4009c00 comp rx=0 tx=0).stop 2024-09-10T08:41:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.852+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/2055699831 shutdown_connections 2024-09-10T08:41:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.852+0000 7f0e0d5fe740 1 --2- 172.21.15.89:0/2055699831 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4d192ea60 0x55b4d1938eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.852+0000 7f0e0d5fe740 1 --2- 172.21.15.89:0/2055699831 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b4d19a9d80 0x55b4d19ac1a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.852+0000 7f0e0d5fe740 1 --2- 172.21.15.89:0/2055699831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4d19299d0 0x55b4d1937ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.852+0000 7f0e0d5fe740 1 --2- 172.21.15.89:0/2055699831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4d1929090 0x55b4d1937960 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.852+0000 7f0ded600640 0 client.25672 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.852+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/2055699831 >> 172.21.15.89:0/2055699831 conn(0x55b4d1924700 msgr2=0x55b4d1923060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.852+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/2055699831 shutdown_connections 2024-09-10T08:41:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.852+0000 7f0e0d5fe740 1 -- 172.21.15.89:0/2055699831 wait complete. 2024-09-10T08:41:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.852+0000 7f0e0d5fe740 20 client.25672 trim_cache size 0 max 16384 2024-09-10T08:41:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:41:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:41:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:41:21.855 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:41:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.853+0000 7f0e0a600640 1 -- 172.21.15.89:0/4206632402 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0de80a2cb0 msgr2=0x7f0de80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.853+0000 7f0e0a600640 1 --2- 172.21.15.89:0/4206632402 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0de80a2cb0 0x7f0de80a5170 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc005fd0 tx=0x7f0dfc009600 comp rx=0 tx=0).stop 2024-09-10T08:41:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.853+0000 7f0e0a600640 1 -- 172.21.15.89:0/4206632402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e04104f50 msgr2=0x7f0e0419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.853+0000 7f0e0a600640 1 --2- 172.21.15.89:0/4206632402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e04104f50 0x7f0e0419a4b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009230 tx=0x7f0df0031b60 comp rx=0 tx=0).stop 2024-09-10T08:41:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.853+0000 7f0e0a600640 1 -- 172.21.15.89:0/4206632402 shutdown_connections 2024-09-10T08:41:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.853+0000 7f0e0a600640 1 --2- 172.21.15.89:0/4206632402 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0de80a2cb0 0x7f0de80a5170 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.853+0000 7f0e0a600640 1 --2- 172.21.15.89:0/4206632402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e04105910 0x7f0e041a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.853+0000 7f0e0a600640 1 --2- 172.21.15.89:0/4206632402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e04104f50 0x7f0e0419a4b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.853+0000 7f0e0a600640 1 --2- 172.21.15.89:0/4206632402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e04103d60 0x7f0e04199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.853+0000 7f0e0a600640 1 -- 172.21.15.89:0/4206632402 >> 172.21.15.89:0/4206632402 conn(0x7f0e040ff4e0 msgr2=0x7f0e04108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:21.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.853+0000 7f0e0a600640 1 -- 172.21.15.89:0/4206632402 shutdown_connections 2024-09-10T08:41:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:21.853+0000 7f0e0a600640 1 -- 172.21.15.89:0/4206632402 wait complete. 2024-09-10T08:41:22.080 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:41:22.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.652+0000 7f6c19800640 1 -- 172.21.15.89:0/2539232012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c14102790 msgr2=0x7f6c141067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:22.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.652+0000 7f6c19800640 1 --2- 172.21.15.89:0/2539232012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c14102790 0x7f6c141067e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009900 tx=0x7f6c0002a410 comp rx=0 tx=0).stop 2024-09-10T08:41:22.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.652+0000 7f6c19800640 1 -- 172.21.15.89:0/2539232012 shutdown_connections 2024-09-10T08:41:22.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.653+0000 7f6c19800640 1 --2- 172.21.15.89:0/2539232012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c14106f10 0x7f6c1410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.653+0000 7f6c19800640 1 --2- 172.21.15.89:0/2539232012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c14102790 0x7f6c141067e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.653+0000 7f6c19800640 1 --2- 172.21.15.89:0/2539232012 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c14101de0 0x7f6c141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.653+0000 7f6c19800640 1 -- 172.21.15.89:0/2539232012 >> 172.21.15.89:0/2539232012 conn(0x7f6c140fd8c0 msgr2=0x7f6c140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:22.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.653+0000 7f6c19800640 1 -- 172.21.15.89:0/2539232012 shutdown_connections 2024-09-10T08:41:22.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.653+0000 7f6c19800640 1 -- 172.21.15.89:0/2539232012 wait complete. 2024-09-10T08:41:22.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.654+0000 7f6c19800640 1 Processor -- start 2024-09-10T08:41:22.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.654+0000 7f6c19800640 1 -- start start 2024-09-10T08:41:22.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.654+0000 7f6c19800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c14101de0 0x7f6c14198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:22.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.654+0000 7f6c19800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c14102790 0x7f6c141989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:22.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.655+0000 7f6c19800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c14106f10 0x7f6c1419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:22.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.655+0000 7f6c19800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6c14111f70 con 0x7f6c14101de0 2024-09-10T08:41:22.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.655+0000 7f6c19800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6c14111df0 con 0x7f6c14106f10 2024-09-10T08:41:22.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.655+0000 7f6c19800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6c141120f0 con 0x7f6c14102790 2024-09-10T08:41:22.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.655+0000 7f6c12a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c14101de0 0x7f6c14198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.655+0000 7f6c12a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c14101de0 0x7f6c14198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38670/0 (socket says 172.21.15.89:38670) 2024-09-10T08:41:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.655+0000 7f6c12a00640 1 -- 172.21.15.89:0/571155968 learned_addr learned my addr 172.21.15.89:0/571155968 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.655+0000 7f6c12000640 1 --2- 172.21.15.89:0/571155968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c14102790 0x7f6c141989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.655+0000 7f6c13400640 1 --2- 172.21.15.89:0/571155968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c14106f10 0x7f6c1419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.655+0000 7f6c12a00640 1 -- 172.21.15.89:0/571155968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c14102790 msgr2=0x7f6c141989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.655+0000 7f6c12a00640 1 --2- 172.21.15.89:0/571155968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c14102790 0x7f6c141989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.655+0000 7f6c12a00640 1 -- 172.21.15.89:0/571155968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c14106f10 msgr2=0x7f6c1419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.655+0000 7f6c12a00640 1 --2- 172.21.15.89:0/571155968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c14106f10 0x7f6c1419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.655+0000 7f6c12a00640 1 -- 172.21.15.89:0/571155968 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1419d420 con 0x7f6c14101de0 2024-09-10T08:41:22.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.656+0000 7f6c12a00640 1 --2- 172.21.15.89:0/571155968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c14101de0 0x7f6c14198470 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400e9f0 tx=0x7f6c0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:22.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.656+0000 7f6c0b400640 1 -- 172.21.15.89:0/571155968 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0400cd80 con 0x7f6c14101de0 2024-09-10T08:41:22.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.656+0000 7f6c0b400640 1 -- 172.21.15.89:0/571155968 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c04010910 con 0x7f6c14101de0 2024-09-10T08:41:22.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.656+0000 7f6c19800640 1 -- 172.21.15.89:0/571155968 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c1419d710 con 0x7f6c14101de0 2024-09-10T08:41:22.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.656+0000 7f6c0b400640 1 -- 172.21.15.89:0/571155968 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c04018990 con 0x7f6c14101de0 2024-09-10T08:41:22.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.656+0000 7f6c19800640 1 -- 172.21.15.89:0/571155968 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c141a4ff0 con 0x7f6c14101de0 2024-09-10T08:41:22.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.659+0000 7f6c0b400640 1 -- 172.21.15.89:0/571155968 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6c04010ac0 con 0x7f6c14101de0 2024-09-10T08:41:22.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.659+0000 7f6c1c208740 1 Processor -- start 2024-09-10T08:41:22.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.660+0000 7f6c1c208740 1 -- start start 2024-09-10T08:41:22.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.660+0000 7f6c1c208740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564389286080 0x564389286460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:22.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.660+0000 7f6c1c208740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643892869a0 0x564389290d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:22.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.660+0000 7f6c1c208740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389291260 0x564389293650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:22.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.660+0000 7f6c1c208740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5643891e83e0 con 0x5643892869a0 2024-09-10T08:41:22.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.660+0000 7f6c1c208740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56438918fd80 con 0x564389286080 2024-09-10T08:41:22.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.660+0000 7f6c1c208740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5643891e43a0 con 0x564389291260 2024-09-10T08:41:22.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.660+0000 7f6c12a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643892869a0 0x564389290d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:22.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.660+0000 7f6c13400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564389286080 0x564389286460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:22.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.660+0000 7f6c13400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564389286080 0x564389286460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:45064/0 (socket says 172.21.15.89:45064) 2024-09-10T08:41:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.660+0000 7f6c13400640 1 -- 172.21.15.89:0/2082443794 learned_addr learned my addr 172.21.15.89:0/2082443794 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.661+0000 7f6c12000640 1 --2- 172.21.15.89:0/2082443794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389291260 0x564389293650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.661+0000 7f6c13400640 1 -- 172.21.15.89:0/2082443794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389291260 msgr2=0x564389293650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.661+0000 7f6c13400640 1 --2- 172.21.15.89:0/2082443794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389291260 0x564389293650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.661+0000 7f6c13400640 1 -- 172.21.15.89:0/2082443794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643892869a0 msgr2=0x564389290d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.661+0000 7f6c13400640 1 --2- 172.21.15.89:0/2082443794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643892869a0 0x564389290d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.661+0000 7f6c13400640 1 -- 172.21.15.89:0/2082443794 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643891aeae0 con 0x564389286080 2024-09-10T08:41:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.661+0000 7f6c13400640 1 --2- 172.21.15.89:0/2082443794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564389286080 0x564389286460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009f50 tx=0x7f6c0c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dd2684eea334623 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:22.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.661+0000 7f6c0b400640 1 --2- 172.21.15.89:0/571155968 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6bf80a2c00 0x7f6bf80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.662+0000 7f6bffe00640 1 -- 172.21.15.89:0/2082443794 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c004d20 con 0x564389286080 2024-09-10T08:41:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.662+0000 7f6bffe00640 1 -- 172.21.15.89:0/2082443794 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c0c004050 con 0x564389286080 2024-09-10T08:41:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.662+0000 7f6bffe00640 1 -- 172.21.15.89:0/2082443794 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c005280 con 0x564389286080 2024-09-10T08:41:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.662+0000 7f6c12a00640 1 --2- 172.21.15.89:0/571155968 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6bf80a2c00 0x7f6bf80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.662+0000 7f6c0b400640 1 -- 172.21.15.89:0/571155968 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f6c040cf6e0 con 0x7f6c14101de0 2024-09-10T08:41:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.662+0000 7f6c1c208740 1 -- 172.21.15.89:0/2082443794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564389286080 msgr2=0x564389286460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.662+0000 7f6c1c208740 1 --2- 172.21.15.89:0/2082443794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564389286080 0x564389286460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009f50 tx=0x7f6c0c0118d0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.662+0000 7f6c12a00640 1 --2- 172.21.15.89:0/571155968 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6bf80a2c00 0x7f6bf80a50c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400e9c0 tx=0x7f6c040d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:22.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.662+0000 7f6c1c208740 1 -- 172.21.15.89:0/2082443794 shutdown_connections 2024-09-10T08:41:22.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.662+0000 7f6c1c208740 1 --2- 172.21.15.89:0/2082443794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389291260 0x564389293650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.662+0000 7f6c1c208740 1 --2- 172.21.15.89:0/2082443794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643892869a0 0x564389290d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.663+0000 7f6c1c208740 1 --2- 172.21.15.89:0/2082443794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564389286080 0x564389286460 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.663+0000 7f6c1c208740 1 -- 172.21.15.89:0/2082443794 >> 172.21.15.89:0/2082443794 conn(0x564389282610 msgr2=0x5643892829f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:22.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.663+0000 7f6c1c208740 1 -- 172.21.15.89:0/2082443794 shutdown_connections 2024-09-10T08:41:22.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.663+0000 7f6c1c208740 1 -- 172.21.15.89:0/2082443794 wait complete. 2024-09-10T08:41:22.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.663+0000 7f6c1c208740 1 -- start start 2024-09-10T08:41:22.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.663+0000 7f6c1c208740 1 Processor -- start 2024-09-10T08:41:22.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.664+0000 7f6c1c208740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389286080 0x56438928e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:22.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.664+0000 7f6c1c208740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643892869a0 0x56438928e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:22.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.664+0000 7f6c1c208740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564389291260 0x56438928ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:22.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.664+0000 7f6c1c208740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564389293e30 con 0x5643892869a0 2024-09-10T08:41:22.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.664+0000 7f6c1c208740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5643891a30f0 con 0x564389291260 2024-09-10T08:41:22.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.664+0000 7f6c1c208740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564389293fb0 con 0x564389286080 2024-09-10T08:41:22.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.664+0000 7f6c1c208740 10 client.0 _ll_register_callbacks cb 0x56438918abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:22.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.664+0000 7f6c12000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643892869a0 0x56438928e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:22.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.664+0000 7f6c12000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643892869a0 0x56438928e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38690/0 (socket says 172.21.15.89:38690) 2024-09-10T08:41:22.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.664+0000 7f6c12000640 1 -- 172.21.15.89:0/1303004726 learned_addr learned my addr 172.21.15.89:0/1303004726 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:22.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.664+0000 7f6c13400640 1 --2- 172.21.15.89:0/1303004726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389286080 0x56438928e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:22.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.664+0000 7f6c13400640 1 --2- 172.21.15.89:0/1303004726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564389291260 0x56438928ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:22.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.665+0000 7f6c12000640 1 -- 172.21.15.89:0/1303004726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389286080 msgr2=0x56438928e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:22.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.665+0000 7f6c12000640 1 --2- 172.21.15.89:0/1303004726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389286080 0x56438928e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.665+0000 7f6c12000640 1 -- 172.21.15.89:0/1303004726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564389291260 msgr2=0x56438928ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:22.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.665+0000 7f6c12000640 1 --2- 172.21.15.89:0/1303004726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564389291260 0x56438928ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.665+0000 7f6c12000640 1 -- 172.21.15.89:0/1303004726 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564388f593c0 con 0x5643892869a0 2024-09-10T08:41:22.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.665+0000 7f6c13400640 1 --2- 172.21.15.89:0/1303004726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564389291260 0x56438928ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:41:22.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.665+0000 7f6c12000640 1 --2- 172.21.15.89:0/1303004726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643892869a0 0x56438928e710 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6c0002a930 tx=0x7f6c000317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74309c51f60ce521 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:22.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.665+0000 7f6bfcc00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:41:22.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.665+0000 7f6bfcc00640 1 -- 172.21.15.89:0/1303004726 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c00035040 con 0x5643892869a0 2024-09-10T08:41:22.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.665+0000 7f6c1c208740 1 -- 172.21.15.89:0/1303004726 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c0c004050 con 0x5643892869a0 2024-09-10T08:41:22.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.665+0000 7f6bfcc00640 1 -- 172.21.15.89:0/1303004726 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f6c00004040 con 0x5643892869a0 2024-09-10T08:41:22.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.666+0000 7f6bfcc00640 1 -- 172.21.15.89:0/1303004726 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6c00004240 con 0x5643892869a0 2024-09-10T08:41:22.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.666+0000 7f6c1c208740 10 client.15675 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:22.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.666+0000 7f6c1c208740 1 -- 172.21.15.89:0/1303004726 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56438929fef0 con 0x5643892869a0 2024-09-10T08:41:22.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.666+0000 7f6bfcc00640 1 -- 172.21.15.89:0/1303004726 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c0c004050 con 0x5643892869a0 2024-09-10T08:41:22.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.666+0000 7f6bfcc00640 1 -- 172.21.15.89:0/1303004726 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6c0003c4d0 con 0x5643892869a0 2024-09-10T08:41:22.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.666+0000 7f6c1c208740 10 client.15675 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:22.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.666+0000 7f6c1c208740 10 client.15675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:41:22.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.666+0000 7f6c1c208740 20 client.15675 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:22.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.666+0000 7f6c1c208740 1 --2- 172.21.15.89:0/1303004726 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5643892a1450 0x5643892a3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.666+0000 7f6c1c208740 4 client.15675 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:41:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.666+0000 7f6c1c208740 1 -- 172.21.15.89:0/1303004726 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643891f5950 con 0x5643892a1450 2024-09-10T08:41:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.667+0000 7f6c13400640 1 --2- 172.21.15.89:0/1303004726 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5643892a1450 0x5643892a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.667+0000 7f6c13400640 1 --2- 172.21.15.89:0/1303004726 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5643892a1450 0x5643892a3870 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b19ab77182a24e8 server_cookie=8189a7f462beb8f2 in_seq=0 out_seq=0 2024-09-10T08:41:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.667+0000 7f6bfcc00640 10 client.15675 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.668+0000 7f6bfcc00640 1 -- 172.21.15.89:0/1303004726 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5643891f5950 con 0x5643892a1450 2024-09-10T08:41:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.668+0000 7f6bfcc00640 10 client.15675 handle_command_reply: tid=0 2024-09-10T08:41:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.668+0000 7f6c1c208740 1 client.15675 shutdown 2024-09-10T08:41:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.668+0000 7f6c1c208740 1 -- 172.21.15.89:0/1303004726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643892869a0 msgr2=0x56438928e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.668+0000 7f6c1c208740 1 --2- 172.21.15.89:0/1303004726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643892869a0 0x56438928e710 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6c0002a930 tx=0x7f6c000317e0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.668+0000 7f6c1c208740 1 -- 172.21.15.89:0/1303004726 shutdown_connections 2024-09-10T08:41:22.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.668+0000 7f6c1c208740 1 --2- 172.21.15.89:0/1303004726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564389291260 0x56438928ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.668+0000 7f6c1c208740 1 --2- 172.21.15.89:0/1303004726 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5643892a1450 0x5643892a3870 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.668+0000 7f6c1c208740 1 --2- 172.21.15.89:0/1303004726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643892869a0 0x56438928e710 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.668+0000 7f6c1c208740 1 --2- 172.21.15.89:0/1303004726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389286080 0x56438928e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.668+0000 7f6c1c208740 1 -- 172.21.15.89:0/1303004726 >> 172.21.15.89:0/1303004726 conn(0x564389282610 msgr2=0x56438927d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:22.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.668+0000 7f6bfcc00640 0 client.15675 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:22.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.669+0000 7f6c1c208740 1 -- 172.21.15.89:0/1303004726 shutdown_connections 2024-09-10T08:41:22.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.669+0000 7f6c1c208740 1 -- 172.21.15.89:0/1303004726 wait complete. 2024-09-10T08:41:22.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.669+0000 7f6c1c208740 20 client.15675 trim_cache size 0 max 16384 2024-09-10T08:41:22.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.682+0000 7f6c1c208740 1 Processor -- start 2024-09-10T08:41:22.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.682+0000 7f6c1c208740 1 -- start start 2024-09-10T08:41:22.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.683+0000 7f6c1c208740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389286080 0x56438928eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:22.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.683+0000 7f6c1c208740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643892869a0 0x5643892a9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:22.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.683+0000 7f6c1c208740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564389291260 0x5643892a9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:22.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.683+0000 7f6c1c208740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6c141af4b0 con 0x5643892869a0 2024-09-10T08:41:22.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.683+0000 7f6c1c208740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6c141acd80 con 0x564389291260 2024-09-10T08:41:22.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.683+0000 7f6c1c208740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6c141aabd0 con 0x564389286080 2024-09-10T08:41:22.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.683+0000 7f6c12000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643892869a0 0x5643892a9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:22.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.683+0000 7f6c13400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389286080 0x56438928eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:22.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.683+0000 7f6c12000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643892869a0 0x5643892a9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38694/0 (socket says 172.21.15.89:38694) 2024-09-10T08:41:22.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.683+0000 7f6c12000640 1 -- 172.21.15.89:0/2791020827 learned_addr learned my addr 172.21.15.89:0/2791020827 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:22.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.683+0000 7f6c13400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389286080 0x56438928eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:33208/0 (socket says 172.21.15.89:33208) 2024-09-10T08:41:22.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.683+0000 7f6c13400640 1 --2- 172.21.15.89:0/2791020827 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564389291260 0x5643892a9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:22.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.683+0000 7f6c13400640 1 -- 172.21.15.89:0/2791020827 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564389291260 msgr2=0x5643892a9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:22.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.683+0000 7f6c13400640 1 --2- 172.21.15.89:0/2791020827 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564389291260 0x5643892a9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.683+0000 7f6c13400640 1 -- 172.21.15.89:0/2791020827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643892869a0 msgr2=0x5643892a9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:22.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.683+0000 7f6c13400640 1 --2- 172.21.15.89:0/2791020827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643892869a0 0x5643892a9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.683+0000 7f6c13400640 1 -- 172.21.15.89:0/2791020827 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564389285eb0 con 0x564389286080 2024-09-10T08:41:22.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.684+0000 7f6c13400640 1 --2- 172.21.15.89:0/2791020827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389286080 0x56438928eaa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c011dd0 tx=0x7f6c0c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9e47ea868e04633 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:22.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.684+0000 7f6bff400640 1 -- 172.21.15.89:0/2791020827 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c004820 con 0x564389286080 2024-09-10T08:41:22.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.684+0000 7f6bff400640 1 -- 172.21.15.89:0/2791020827 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c0c004280 con 0x564389286080 2024-09-10T08:41:22.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.684+0000 7f6bff400640 1 -- 172.21.15.89:0/2791020827 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c027970 con 0x564389286080 2024-09-10T08:41:22.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.685+0000 7f6c1c208740 1 -- 172.21.15.89:0/2791020827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389286080 msgr2=0x56438928eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:22.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.685+0000 7f6c1c208740 1 --2- 172.21.15.89:0/2791020827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389286080 0x56438928eaa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c011dd0 tx=0x7f6c0c0118d0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.685+0000 7f6c1c208740 1 -- 172.21.15.89:0/2791020827 shutdown_connections 2024-09-10T08:41:22.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.685+0000 7f6c1c208740 1 --2- 172.21.15.89:0/2791020827 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564389291260 0x5643892a9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.685+0000 7f6c1c208740 1 --2- 172.21.15.89:0/2791020827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5643892869a0 0x5643892a9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.685+0000 7f6c1c208740 1 --2- 172.21.15.89:0/2791020827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389286080 0x56438928eaa0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.685+0000 7f6c1c208740 1 -- 172.21.15.89:0/2791020827 >> 172.21.15.89:0/2791020827 conn(0x564389282610 msgr2=0x564389290250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:22.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.685+0000 7f6c1c208740 1 -- 172.21.15.89:0/2791020827 shutdown_connections 2024-09-10T08:41:22.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.685+0000 7f6c1c208740 1 -- 172.21.15.89:0/2791020827 wait complete. 2024-09-10T08:41:22.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.685+0000 7f6c1c208740 1 -- start start 2024-09-10T08:41:22.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.685+0000 7f6c1c208740 1 Processor -- start 2024-09-10T08:41:22.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.686+0000 7f6c1c208740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389286080 0x564389299730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:22.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.686+0000 7f6c1c208740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643892869a0 0x564389294a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:22.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.686+0000 7f6c1c208740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564389291260 0x564389294fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:22.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.686+0000 7f6c1c208740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6c141ab240 con 0x564389291260 2024-09-10T08:41:22.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.686+0000 7f6c1c208740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6c141ac6b0 con 0x5643892869a0 2024-09-10T08:41:22.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.686+0000 7f6c1c208740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6c141ab910 con 0x564389286080 2024-09-10T08:41:22.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.686+0000 7f6c1c208740 10 client.0 _ll_register_callbacks cb 0x7f6c0c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:22.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.686+0000 7f6c13400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564389291260 0x564389294fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:22.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.686+0000 7f6c13400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564389291260 0x564389294fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38700/0 (socket says 172.21.15.89:38700) 2024-09-10T08:41:22.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.686+0000 7f6c13400640 1 -- 172.21.15.89:0/319326404 learned_addr learned my addr 172.21.15.89:0/319326404 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:22.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.686+0000 7f6c12000640 1 --2- 172.21.15.89:0/319326404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643892869a0 0x564389294a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:22.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.686+0000 7f6c13400640 1 --2- 172.21.15.89:0/319326404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389286080 0x564389299730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:22.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.687+0000 7f6c12000640 1 -- 172.21.15.89:0/319326404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389286080 msgr2=0x564389299730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:22.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.687+0000 7f6c12000640 1 --2- 172.21.15.89:0/319326404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389286080 0x564389299730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.687+0000 7f6c12000640 1 -- 172.21.15.89:0/319326404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564389291260 msgr2=0x564389294fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:22.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.687+0000 7f6c12000640 1 --2- 172.21.15.89:0/319326404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564389291260 0x564389294fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.687+0000 7f6c12000640 1 -- 172.21.15.89:0/319326404 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643892832a0 con 0x5643892869a0 2024-09-10T08:41:22.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.687+0000 7f6c12000640 1 --2- 172.21.15.89:0/319326404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643892869a0 0x564389294a60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6c0003c970 tx=0x7f6c0002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b8f55d4f05b7e8d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:22.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.687+0000 7f6bfcc00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:41:22.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.687+0000 7f6bfcc00640 1 -- 172.21.15.89:0/319326404 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c00058020 con 0x5643892869a0 2024-09-10T08:41:22.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.687+0000 7f6c1c208740 1 -- 172.21.15.89:0/319326404 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c0c004280 con 0x5643892869a0 2024-09-10T08:41:22.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.687+0000 7f6bfcc00640 1 -- 172.21.15.89:0/319326404 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f6c0003c440 con 0x5643892869a0 2024-09-10T08:41:22.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.688+0000 7f6bfcc00640 1 -- 172.21.15.89:0/319326404 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6c00063050 con 0x5643892869a0 2024-09-10T08:41:22.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.688+0000 7f6c1c208740 10 client.25745 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:22.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.688+0000 7f6c1c208740 1 -- 172.21.15.89:0/319326404 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56438930bd30 con 0x5643892869a0 2024-09-10T08:41:22.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.688+0000 7f6bfcc00640 1 -- 172.21.15.89:0/319326404 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c0c004280 con 0x5643892869a0 2024-09-10T08:41:22.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.688+0000 7f6bfcc00640 1 -- 172.21.15.89:0/319326404 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6c000046c0 con 0x5643892869a0 2024-09-10T08:41:22.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.688+0000 7f6c1c208740 10 client.25745 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:22.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.688+0000 7f6c1c208740 10 client.25745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:41:22.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.688+0000 7f6c1c208740 20 client.25745 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:22.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.688+0000 7f6c1c208740 1 --2- 172.21.15.89:0/319326404 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56438930c4a0 0x56438930e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:22.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.688+0000 7f6c1c208740 4 client.25745 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:41:22.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.688+0000 7f6c1c208740 1 -- 172.21.15.89:0/319326404 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5643891efe60 con 0x56438930c4a0 2024-09-10T08:41:22.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.689+0000 7f6c13400640 1 --2- 172.21.15.89:0/319326404 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56438930c4a0 0x56438930e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:22.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.689+0000 7f6c13400640 1 --2- 172.21.15.89:0/319326404 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56438930c4a0 0x56438930e8c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caf4a25ad9cedffa server_cookie=4e574a7dd05b5961 in_seq=0 out_seq=0 2024-09-10T08:41:22.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.689+0000 7f6bfcc00640 10 client.25745 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:22.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.689+0000 7f6bfcc00640 1 -- 172.21.15.89:0/319326404 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5643891efe60 con 0x56438930c4a0 2024-09-10T08:41:22.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.689+0000 7f6bfcc00640 10 client.25745 handle_command_reply: tid=0 2024-09-10T08:41:22.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.689+0000 7f6c1c208740 1 client.25745 shutdown 2024-09-10T08:41:22.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.689+0000 7f6c1c208740 1 -- 172.21.15.89:0/319326404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643892869a0 msgr2=0x564389294a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:22.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.689+0000 7f6c1c208740 1 --2- 172.21.15.89:0/319326404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643892869a0 0x564389294a60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6c0003c970 tx=0x7f6c0002a410 comp rx=0 tx=0).stop 2024-09-10T08:41:22.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.690+0000 7f6c1c208740 1 -- 172.21.15.89:0/319326404 shutdown_connections 2024-09-10T08:41:22.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.690+0000 7f6c1c208740 1 --2- 172.21.15.89:0/319326404 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56438930c4a0 0x56438930e8c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.690+0000 7f6c1c208740 1 --2- 172.21.15.89:0/319326404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564389291260 0x564389294fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.690+0000 7f6c1c208740 1 --2- 172.21.15.89:0/319326404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5643892869a0 0x564389294a60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.690+0000 7f6c1c208740 1 --2- 172.21.15.89:0/319326404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564389286080 0x564389299730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.690+0000 7f6bfcc00640 0 client.25745 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:22.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.690+0000 7f6c1c208740 1 -- 172.21.15.89:0/319326404 >> 172.21.15.89:0/319326404 conn(0x564389282610 msgr2=0x56438927cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:22.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.690+0000 7f6c1c208740 1 -- 172.21.15.89:0/319326404 shutdown_connections 2024-09-10T08:41:22.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.690+0000 7f6c1c208740 1 -- 172.21.15.89:0/319326404 wait complete. 2024-09-10T08:41:22.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.690+0000 7f6c1c208740 20 client.25745 trim_cache size 0 max 16384 2024-09-10T08:41:22.703 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:41:22.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.690+0000 7f6c19800640 1 -- 172.21.15.89:0/571155968 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6bf80a2c00 msgr2=0x7f6bf80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:22.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.690+0000 7f6c19800640 1 --2- 172.21.15.89:0/571155968 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6bf80a2c00 0x7f6bf80a50c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400e9c0 tx=0x7f6c040d5530 comp rx=0 tx=0).stop 2024-09-10T08:41:22.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.690+0000 7f6c19800640 1 -- 172.21.15.89:0/571155968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c14101de0 msgr2=0x7f6c14198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:22.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.690+0000 7f6c19800640 1 --2- 172.21.15.89:0/571155968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c14101de0 0x7f6c14198470 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400e9f0 tx=0x7f6c0400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.691+0000 7f6c19800640 1 -- 172.21.15.89:0/571155968 shutdown_connections 2024-09-10T08:41:22.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.691+0000 7f6c19800640 1 --2- 172.21.15.89:0/571155968 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6bf80a2c00 0x7f6bf80a50c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.691+0000 7f6c19800640 1 --2- 172.21.15.89:0/571155968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c14106f10 0x7f6c1419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.691+0000 7f6c19800640 1 --2- 172.21.15.89:0/571155968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c14102790 0x7f6c141989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.691+0000 7f6c19800640 1 --2- 172.21.15.89:0/571155968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c14101de0 0x7f6c14198470 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:22.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.691+0000 7f6c19800640 1 -- 172.21.15.89:0/571155968 >> 172.21.15.89:0/571155968 conn(0x7f6c140fd8c0 msgr2=0x7f6c140feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:22.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.691+0000 7f6c19800640 1 -- 172.21.15.89:0/571155968 shutdown_connections 2024-09-10T08:41:22.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:22.691+0000 7f6c19800640 1 -- 172.21.15.89:0/571155968 wait complete. 2024-09-10T08:41:22.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:22 smithi089 ceph-mon[30757]: pgmap v436: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-10T08:41:22.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:22 smithi149 ceph-mon[36167]: pgmap v436: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-10T08:41:23.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:22 smithi111 ceph-mon[35220]: pgmap v436: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-10T08:41:24.510 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:41:24.511 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:41:24.890 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:41:24.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:24 smithi149 ceph-mon[36167]: pgmap v437: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T08:41:25.027 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:24 smithi089 ceph-mon[30757]: pgmap v437: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T08:41:25.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:24 smithi111 ceph-mon[35220]: pgmap v437: 129 pgs: 129 active+clean; 672 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T08:41:25.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.440+0000 7fb652600640 1 -- 172.21.15.89:0/922668333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb64c1049a0 msgr2=0x7fb64c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:25.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.440+0000 7fb652600640 1 --2- 172.21.15.89:0/922668333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb64c1049a0 0x7fb64c1089f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb638009860 tx=0x7fb63802a340 comp rx=0 tx=0).stop 2024-09-10T08:41:25.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.440+0000 7fb652600640 1 -- 172.21.15.89:0/922668333 shutdown_connections 2024-09-10T08:41:25.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.440+0000 7fb652600640 1 --2- 172.21.15.89:0/922668333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb64c109120 0x7fb64c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.440+0000 7fb652600640 1 --2- 172.21.15.89:0/922668333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb64c1049a0 0x7fb64c1089f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.440+0000 7fb652600640 1 --2- 172.21.15.89:0/922668333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb64c103ff0 0x7fb64c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.440+0000 7fb652600640 1 -- 172.21.15.89:0/922668333 >> 172.21.15.89:0/922668333 conn(0x7fb64c0ffa90 msgr2=0x7fb64c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.440+0000 7fb652600640 1 -- 172.21.15.89:0/922668333 shutdown_connections 2024-09-10T08:41:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.440+0000 7fb652600640 1 -- 172.21.15.89:0/922668333 wait complete. 2024-09-10T08:41:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.441+0000 7fb652600640 1 Processor -- start 2024-09-10T08:41:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.441+0000 7fb652600640 1 -- start start 2024-09-10T08:41:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.442+0000 7fb652600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb64c103ff0 0x7fb64c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.442+0000 7fb652600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb64c1049a0 0x7fb64c19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.442+0000 7fb652600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb64c109120 0x7fb64c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.442+0000 7fb652600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c114140 con 0x7fb64c103ff0 2024-09-10T08:41:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.442+0000 7fb652600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c113fc0 con 0x7fb64c109120 2024-09-10T08:41:25.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.442+0000 7fb652600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c1142c0 con 0x7fb64c1049a0 2024-09-10T08:41:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.442+0000 7fb64b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb64c103ff0 0x7fb64c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.442+0000 7fb64b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb64c103ff0 0x7fb64c19a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47700/0 (socket says 172.21.15.89:47700) 2024-09-10T08:41:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.442+0000 7fb64b400640 1 -- 172.21.15.89:0/3225493114 learned_addr learned my addr 172.21.15.89:0/3225493114 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.442+0000 7fb64be00640 1 --2- 172.21.15.89:0/3225493114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb64c109120 0x7fb64c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.442+0000 7fb64aa00640 1 --2- 172.21.15.89:0/3225493114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb64c1049a0 0x7fb64c19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.442+0000 7fb64b400640 1 -- 172.21.15.89:0/3225493114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb64c1049a0 msgr2=0x7fb64c19ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.442+0000 7fb64b400640 1 --2- 172.21.15.89:0/3225493114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb64c1049a0 0x7fb64c19ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.442+0000 7fb64b400640 1 -- 172.21.15.89:0/3225493114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb64c109120 msgr2=0x7fb64c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.442+0000 7fb64b400640 1 --2- 172.21.15.89:0/3225493114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb64c109120 0x7fb64c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.442+0000 7fb64b400640 1 -- 172.21.15.89:0/3225493114 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64c19f670 con 0x7fb64c103ff0 2024-09-10T08:41:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.442+0000 7fb64aa00640 1 --2- 172.21.15.89:0/3225493114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb64c1049a0 0x7fb64c19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:41:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.443+0000 7fb64b400640 1 --2- 172.21.15.89:0/3225493114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb64c103ff0 0x7fb64c19a6c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00e9f0 tx=0x7fb63c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:25.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.443+0000 7fb643e00640 1 -- 172.21.15.89:0/3225493114 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c00cd80 con 0x7fb64c103ff0 2024-09-10T08:41:25.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.443+0000 7fb643e00640 1 -- 172.21.15.89:0/3225493114 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63c005500 con 0x7fb64c103ff0 2024-09-10T08:41:25.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.443+0000 7fb643e00640 1 -- 172.21.15.89:0/3225493114 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c010690 con 0x7fb64c103ff0 2024-09-10T08:41:25.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.443+0000 7fb652600640 1 -- 172.21.15.89:0/3225493114 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64c19f960 con 0x7fb64c103ff0 2024-09-10T08:41:25.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.443+0000 7fb652600640 1 -- 172.21.15.89:0/3225493114 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb64c1a7240 con 0x7fb64c103ff0 2024-09-10T08:41:25.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.446+0000 7fb643e00640 1 -- 172.21.15.89:0/3225493114 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb63c0046c0 con 0x7fb64c103ff0 2024-09-10T08:41:25.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.446+0000 7fb65507e740 1 Processor -- start 2024-09-10T08:41:25.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.446+0000 7fb65507e740 1 -- start start 2024-09-10T08:41:25.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.446+0000 7fb65507e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563c60664ac0 0x563c60664ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:25.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.447+0000 7fb65507e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c606653e0 0x563c6066f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:25.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.447+0000 7fb65507e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c6066fca0 0x563c60672090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:25.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.447+0000 7fb65507e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563c605cb290 con 0x563c606653e0 2024-09-10T08:41:25.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.447+0000 7fb65507e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563c60572b40 con 0x563c60664ac0 2024-09-10T08:41:25.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.447+0000 7fb65507e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563c605c7250 con 0x563c6066fca0 2024-09-10T08:41:25.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.447+0000 7fb64b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c606653e0 0x563c6066f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:25.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.447+0000 7fb64b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c606653e0 0x563c6066f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47712/0 (socket says 172.21.15.89:47712) 2024-09-10T08:41:25.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.447+0000 7fb64b400640 1 -- 172.21.15.89:0/2495811197 learned_addr learned my addr 172.21.15.89:0/2495811197 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:25.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.447+0000 7fb64be00640 1 --2- 172.21.15.89:0/2495811197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563c60664ac0 0x563c60664ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:25.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.447+0000 7fb64aa00640 1 --2- 172.21.15.89:0/2495811197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c6066fca0 0x563c60672090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:25.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.447+0000 7fb64b400640 1 -- 172.21.15.89:0/2495811197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c6066fca0 msgr2=0x563c60672090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:25.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.447+0000 7fb64b400640 1 --2- 172.21.15.89:0/2495811197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c6066fca0 0x563c60672090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.447+0000 7fb64b400640 1 -- 172.21.15.89:0/2495811197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563c60664ac0 msgr2=0x563c60664ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:25.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.447+0000 7fb64b400640 1 --2- 172.21.15.89:0/2495811197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563c60664ac0 0x563c60664ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.447+0000 7fb64b400640 1 -- 172.21.15.89:0/2495811197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c6033c6a0 con 0x563c606653e0 2024-09-10T08:41:25.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.447+0000 7fb64aa00640 1 --2- 172.21.15.89:0/2495811197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c6066fca0 0x563c60672090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:41:25.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.448+0000 7fb64b400640 1 --2- 172.21.15.89:0/2495811197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c606653e0 0x563c6066f760 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb63c0d5a90 tx=0x7fb63c0d5b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a41b221d526576b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:25.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.448+0000 7fb643e00640 1 --2- 172.21.15.89:0/3225493114 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb6300a2b80 0x7fb6300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:25.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.448+0000 7fb640c00640 1 -- 172.21.15.89:0/2495811197 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c0de070 con 0x563c606653e0 2024-09-10T08:41:25.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.448+0000 7fb640c00640 1 -- 172.21.15.89:0/2495811197 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63c013190 con 0x563c606653e0 2024-09-10T08:41:25.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.448+0000 7fb640c00640 1 -- 172.21.15.89:0/2495811197 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c01c980 con 0x563c606653e0 2024-09-10T08:41:25.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.448+0000 7fb643e00640 1 -- 172.21.15.89:0/3225493114 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb63c0cf900 con 0x7fb64c103ff0 2024-09-10T08:41:25.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.448+0000 7fb64be00640 1 --2- 172.21.15.89:0/3225493114 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb6300a2b80 0x7fb6300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:25.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.449+0000 7fb65507e740 1 -- 172.21.15.89:0/2495811197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c606653e0 msgr2=0x563c6066f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:25.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.449+0000 7fb65507e740 1 --2- 172.21.15.89:0/2495811197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c606653e0 0x563c6066f760 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb63c0d5a90 tx=0x7fb63c0d5b80 comp rx=0 tx=0).stop 2024-09-10T08:41:25.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.449+0000 7fb64be00640 1 --2- 172.21.15.89:0/3225493114 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb6300a2b80 0x7fb6300a5040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb644005fd0 tx=0x7fb644010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:25.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.449+0000 7fb65507e740 1 -- 172.21.15.89:0/2495811197 shutdown_connections 2024-09-10T08:41:25.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.449+0000 7fb65507e740 1 --2- 172.21.15.89:0/2495811197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c6066fca0 0x563c60672090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.449+0000 7fb65507e740 1 --2- 172.21.15.89:0/2495811197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c606653e0 0x563c6066f760 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.449+0000 7fb65507e740 1 --2- 172.21.15.89:0/2495811197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563c60664ac0 0x563c60664ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.449+0000 7fb65507e740 1 -- 172.21.15.89:0/2495811197 >> 172.21.15.89:0/2495811197 conn(0x563c60661050 msgr2=0x563c60661430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:25.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.449+0000 7fb65507e740 1 -- 172.21.15.89:0/2495811197 shutdown_connections 2024-09-10T08:41:25.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.449+0000 7fb65507e740 1 -- 172.21.15.89:0/2495811197 wait complete. 2024-09-10T08:41:25.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.450+0000 7fb65507e740 1 -- start start 2024-09-10T08:41:25.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.450+0000 7fb65507e740 1 Processor -- start 2024-09-10T08:41:25.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.450+0000 7fb65507e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c60664ac0 0x563c6066f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:25.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.450+0000 7fb65507e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c606653e0 0x563c6066c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:25.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.450+0000 7fb65507e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563c6066fca0 0x563c6066cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:25.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.450+0000 7fb65507e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563c60672910 con 0x563c60664ac0 2024-09-10T08:41:25.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.450+0000 7fb65507e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563c60585fa0 con 0x563c6066fca0 2024-09-10T08:41:25.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.450+0000 7fb65507e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563c60672a90 con 0x563c606653e0 2024-09-10T08:41:25.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.450+0000 7fb65507e740 10 client.0 _ll_register_callbacks cb 0x563c602d9ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:25.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.451+0000 7fb64b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c60664ac0 0x563c6066f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:25.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.451+0000 7fb64b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c60664ac0 0x563c6066f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47722/0 (socket says 172.21.15.89:47722) 2024-09-10T08:41:25.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.451+0000 7fb64b400640 1 -- 172.21.15.89:0/2964258824 learned_addr learned my addr 172.21.15.89:0/2964258824 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:25.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.451+0000 7fb64aa00640 1 --2- 172.21.15.89:0/2964258824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c606653e0 0x563c6066c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:25.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.451+0000 7fb64be00640 1 --2- 172.21.15.89:0/2964258824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563c6066fca0 0x563c6066cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:25.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.451+0000 7fb64b400640 1 -- 172.21.15.89:0/2964258824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c606653e0 msgr2=0x563c6066c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:25.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.451+0000 7fb64b400640 1 --2- 172.21.15.89:0/2964258824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c606653e0 0x563c6066c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.451+0000 7fb64b400640 1 -- 172.21.15.89:0/2964258824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563c6066fca0 msgr2=0x563c6066cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:25.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.451+0000 7fb64b400640 1 --2- 172.21.15.89:0/2964258824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563c6066fca0 0x563c6066cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.451+0000 7fb64b400640 1 -- 172.21.15.89:0/2964258824 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c6066d5b0 con 0x563c60664ac0 2024-09-10T08:41:25.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.451+0000 7fb64b400640 1 --2- 172.21.15.89:0/2964258824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c60664ac0 0x563c6066f5c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb63c0d5a60 tx=0x7fb63c00acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6921e3c73b5c609b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:25.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.451+0000 7fb635600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:41:25.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.452+0000 7fb635600640 1 -- 172.21.15.89:0/2964258824 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c00a6d0 con 0x563c60664ac0 2024-09-10T08:41:25.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.452+0000 7fb635600640 1 -- 172.21.15.89:0/2964258824 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb63c0cd0e0 con 0x563c60664ac0 2024-09-10T08:41:25.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.452+0000 7fb65507e740 1 -- 172.21.15.89:0/2964258824 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563c605d8800 con 0x563c60664ac0 2024-09-10T08:41:25.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.452+0000 7fb635600640 1 -- 172.21.15.89:0/2964258824 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb63c00a8f0 con 0x563c60664ac0 2024-09-10T08:41:25.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.452+0000 7fb65507e740 10 client.15693 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:25.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.452+0000 7fb65507e740 1 -- 172.21.15.89:0/2964258824 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563c6067e7f0 con 0x563c60664ac0 2024-09-10T08:41:25.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.452+0000 7fb635600640 1 -- 172.21.15.89:0/2964258824 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563c605d8800 con 0x563c60664ac0 2024-09-10T08:41:25.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.452+0000 7fb635600640 1 -- 172.21.15.89:0/2964258824 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb63c0983a0 con 0x563c60664ac0 2024-09-10T08:41:25.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.453+0000 7fb65507e740 10 client.15693 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:25.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.453+0000 7fb65507e740 10 client.15693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:41:25.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.453+0000 7fb65507e740 20 client.15693 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:25.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.453+0000 7fb65507e740 1 --2- 172.21.15.89:0/2964258824 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563c6067fb40 0x563c60681f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:25.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.453+0000 7fb65507e740 4 client.15693 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:41:25.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.453+0000 7fb65507e740 1 -- 172.21.15.89:0/2964258824 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c605d2d10 con 0x563c6067fb40 2024-09-10T08:41:25.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.453+0000 7fb64aa00640 1 --2- 172.21.15.89:0/2964258824 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563c6067fb40 0x563c60681f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:25.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.454+0000 7fb64aa00640 1 --2- 172.21.15.89:0/2964258824 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563c6067fb40 0x563c60681f30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef001c16246b27a6 server_cookie=4abe08f459850771 in_seq=0 out_seq=0 2024-09-10T08:41:25.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.454+0000 7fb635600640 10 client.15693 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:25.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.454+0000 7fb635600640 1 -- 172.21.15.89:0/2964258824 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c605d2d10 con 0x563c6067fb40 2024-09-10T08:41:25.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.454+0000 7fb635600640 10 client.15693 handle_command_reply: tid=0 2024-09-10T08:41:25.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.454+0000 7fb65507e740 1 client.15693 shutdown 2024-09-10T08:41:25.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.455+0000 7fb65507e740 1 -- 172.21.15.89:0/2964258824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c60664ac0 msgr2=0x563c6066f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:25.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.455+0000 7fb65507e740 1 --2- 172.21.15.89:0/2964258824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c60664ac0 0x563c6066f5c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb63c0d5a60 tx=0x7fb63c00acb0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.455+0000 7fb65507e740 1 -- 172.21.15.89:0/2964258824 shutdown_connections 2024-09-10T08:41:25.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.455+0000 7fb65507e740 1 --2- 172.21.15.89:0/2964258824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563c6066fca0 0x563c6066cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.455+0000 7fb65507e740 1 --2- 172.21.15.89:0/2964258824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c606653e0 0x563c6066c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.455+0000 7fb65507e740 1 --2- 172.21.15.89:0/2964258824 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563c6067fb40 0x563c60681f30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.455+0000 7fb65507e740 1 --2- 172.21.15.89:0/2964258824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c60664ac0 0x563c6066f5c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.455+0000 7fb65507e740 1 -- 172.21.15.89:0/2964258824 >> 172.21.15.89:0/2964258824 conn(0x563c60661050 msgr2=0x563c60666d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:25.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.455+0000 7fb635600640 0 client.15693 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:25.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.455+0000 7fb65507e740 1 -- 172.21.15.89:0/2964258824 shutdown_connections 2024-09-10T08:41:25.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.455+0000 7fb65507e740 1 -- 172.21.15.89:0/2964258824 wait complete. 2024-09-10T08:41:25.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.455+0000 7fb65507e740 20 client.15693 trim_cache size 0 max 16384 2024-09-10T08:41:25.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.473+0000 7fb65507e740 1 Processor -- start 2024-09-10T08:41:25.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.473+0000 7fb65507e740 1 -- start start 2024-09-10T08:41:25.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.473+0000 7fb65507e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c606653e0 0x563c60684f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:25.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.473+0000 7fb65507e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c6066fca0 0x563c60685490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:25.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.473+0000 7fb65507e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563c606859d0 0x563c60669ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:25.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.473+0000 7fb65507e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c1b3840 con 0x563c606653e0 2024-09-10T08:41:25.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.473+0000 7fb65507e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c1b1110 con 0x563c606859d0 2024-09-10T08:41:25.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.473+0000 7fb65507e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c1aef60 con 0x563c6066fca0 2024-09-10T08:41:25.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.474+0000 7fb64b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c606653e0 0x563c60684f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:25.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.474+0000 7fb64b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c606653e0 0x563c60684f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47730/0 (socket says 172.21.15.89:47730) 2024-09-10T08:41:25.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.474+0000 7fb64b400640 1 -- 172.21.15.89:0/2720388586 learned_addr learned my addr 172.21.15.89:0/2720388586 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:25.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.474+0000 7fb64be00640 1 --2- 172.21.15.89:0/2720388586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563c606859d0 0x563c60669ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:25.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.474+0000 7fb64aa00640 1 --2- 172.21.15.89:0/2720388586 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c6066fca0 0x563c60685490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:25.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.474+0000 7fb64b400640 1 -- 172.21.15.89:0/2720388586 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c6066fca0 msgr2=0x563c60685490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:25.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.474+0000 7fb64b400640 1 --2- 172.21.15.89:0/2720388586 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c6066fca0 0x563c60685490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.474+0000 7fb64b400640 1 -- 172.21.15.89:0/2720388586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563c606859d0 msgr2=0x563c60669ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:25.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.474+0000 7fb64b400640 1 --2- 172.21.15.89:0/2720388586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563c606859d0 0x563c60669ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.474+0000 7fb64b400640 1 -- 172.21.15.89:0/2720388586 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c6066d3b0 con 0x563c606653e0 2024-09-10T08:41:25.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.474+0000 7fb64b400640 1 --2- 172.21.15.89:0/2720388586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c606653e0 0x563c60684f50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb63c0a4470 tx=0x7fb63c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06cf0ac7ea93da5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:25.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.475+0000 7fb637e00640 1 -- 172.21.15.89:0/2720388586 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c0de070 con 0x563c606653e0 2024-09-10T08:41:25.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.475+0000 7fb637e00640 1 -- 172.21.15.89:0/2720388586 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63c0d7670 con 0x563c606653e0 2024-09-10T08:41:25.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.475+0000 7fb637e00640 1 -- 172.21.15.89:0/2720388586 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c00ab10 con 0x563c606653e0 2024-09-10T08:41:25.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.475+0000 7fb65507e740 1 -- 172.21.15.89:0/2720388586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c606653e0 msgr2=0x563c60684f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:25.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.475+0000 7fb65507e740 1 --2- 172.21.15.89:0/2720388586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c606653e0 0x563c60684f50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb63c0a4470 tx=0x7fb63c00cf20 comp rx=0 tx=0).stop 2024-09-10T08:41:25.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.475+0000 7fb65507e740 1 -- 172.21.15.89:0/2720388586 shutdown_connections 2024-09-10T08:41:25.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.475+0000 7fb65507e740 1 --2- 172.21.15.89:0/2720388586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563c606859d0 0x563c60669ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.475+0000 7fb65507e740 1 --2- 172.21.15.89:0/2720388586 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c6066fca0 0x563c60685490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.475+0000 7fb65507e740 1 --2- 172.21.15.89:0/2720388586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c606653e0 0x563c60684f50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.475+0000 7fb65507e740 1 -- 172.21.15.89:0/2720388586 >> 172.21.15.89:0/2720388586 conn(0x563c60661050 msgr2=0x563c6066b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:25.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.476+0000 7fb65507e740 1 -- 172.21.15.89:0/2720388586 shutdown_connections 2024-09-10T08:41:25.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.476+0000 7fb65507e740 1 -- 172.21.15.89:0/2720388586 wait complete. 2024-09-10T08:41:25.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.476+0000 7fb65507e740 1 -- start start 2024-09-10T08:41:25.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.476+0000 7fb65507e740 1 Processor -- start 2024-09-10T08:41:25.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.477+0000 7fb65507e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563c606653e0 0x563c606853e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:25.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.477+0000 7fb65507e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c6066fca0 0x563c60682730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:25.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.477+0000 7fb65507e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c606859d0 0x563c60682d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:25.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.477+0000 7fb65507e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c1af5d0 con 0x563c6066fca0 2024-09-10T08:41:25.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.477+0000 7fb65507e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c1b0a40 con 0x563c606653e0 2024-09-10T08:41:25.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.477+0000 7fb65507e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c1afca0 con 0x563c606859d0 2024-09-10T08:41:25.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.477+0000 7fb65507e740 10 client.0 _ll_register_callbacks cb 0x7fb644009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:25.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.477+0000 7fb64aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c6066fca0 0x563c60682730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:25.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.477+0000 7fb64aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c6066fca0 0x563c60682730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47736/0 (socket says 172.21.15.89:47736) 2024-09-10T08:41:25.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.477+0000 7fb64aa00640 1 -- 172.21.15.89:0/3717258994 learned_addr learned my addr 172.21.15.89:0/3717258994 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:25.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.477+0000 7fb64b400640 1 --2- 172.21.15.89:0/3717258994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563c606653e0 0x563c606853e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:25.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.477+0000 7fb64be00640 1 --2- 172.21.15.89:0/3717258994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c606859d0 0x563c60682d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:25.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.477+0000 7fb64aa00640 1 -- 172.21.15.89:0/3717258994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c606859d0 msgr2=0x563c60682d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:25.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.477+0000 7fb64aa00640 1 --2- 172.21.15.89:0/3717258994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c606859d0 0x563c60682d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.477+0000 7fb64aa00640 1 -- 172.21.15.89:0/3717258994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563c606653e0 msgr2=0x563c606853e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:25.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.477+0000 7fb64aa00640 1 --2- 172.21.15.89:0/3717258994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563c606653e0 0x563c606853e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.477+0000 7fb64aa00640 1 -- 172.21.15.89:0/3717258994 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c60683570 con 0x563c6066fca0 2024-09-10T08:41:25.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.478+0000 7fb64aa00640 1 --2- 172.21.15.89:0/3717258994 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c6066fca0 0x563c60682730 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb63802a900 tx=0x7fb638030480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa50d039b62450d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:25.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.478+0000 7fb635600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:41:25.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.478+0000 7fb635600640 1 -- 172.21.15.89:0/3717258994 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638037040 con 0x563c6066fca0 2024-09-10T08:41:25.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.478+0000 7fb635600640 1 -- 172.21.15.89:0/3717258994 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb638004480 con 0x563c6066fca0 2024-09-10T08:41:25.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.478+0000 7fb65507e740 1 -- 172.21.15.89:0/3717258994 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb63c0d7670 con 0x563c6066fca0 2024-09-10T08:41:25.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.478+0000 7fb635600640 1 -- 172.21.15.89:0/3717258994 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb638055050 con 0x563c6066fca0 2024-09-10T08:41:25.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.478+0000 7fb65507e740 10 client.15699 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:25.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.478+0000 7fb65507e740 1 -- 172.21.15.89:0/3717258994 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563c606ea620 con 0x563c6066fca0 2024-09-10T08:41:25.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.479+0000 7fb635600640 1 -- 172.21.15.89:0/3717258994 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63c0d7670 con 0x563c6066fca0 2024-09-10T08:41:25.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.479+0000 7fb635600640 1 -- 172.21.15.89:0/3717258994 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb6380046c0 con 0x563c6066fca0 2024-09-10T08:41:25.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.479+0000 7fb65507e740 10 client.15699 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:25.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.479+0000 7fb65507e740 10 client.15699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:41:25.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.479+0000 7fb65507e740 20 client.15699 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:25.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.479+0000 7fb65507e740 1 --2- 172.21.15.89:0/3717258994 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563c606eae20 0x563c606ed240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:25.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.479+0000 7fb65507e740 4 client.15699 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:41:25.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.479+0000 7fb65507e740 1 -- 172.21.15.89:0/3717258994 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563c6054efb0 con 0x563c606eae20 2024-09-10T08:41:25.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.479+0000 7fb64b400640 1 --2- 172.21.15.89:0/3717258994 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563c606eae20 0x563c606ed240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:25.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.480+0000 7fb64b400640 1 --2- 172.21.15.89:0/3717258994 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563c606eae20 0x563c606ed240 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95df2fa907e9a720 server_cookie=99ca7e461328bb87 in_seq=0 out_seq=0 2024-09-10T08:41:25.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.480+0000 7fb635600640 10 client.15699 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:25.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.481+0000 7fb635600640 1 -- 172.21.15.89:0/3717258994 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563c6054efb0 con 0x563c606eae20 2024-09-10T08:41:25.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.481+0000 7fb635600640 10 client.15699 handle_command_reply: tid=0 2024-09-10T08:41:25.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.481+0000 7fb65507e740 1 client.15699 shutdown 2024-09-10T08:41:25.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.481+0000 7fb65507e740 1 -- 172.21.15.89:0/3717258994 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c6066fca0 msgr2=0x563c60682730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:25.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.481+0000 7fb65507e740 1 --2- 172.21.15.89:0/3717258994 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c6066fca0 0x563c60682730 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb63802a900 tx=0x7fb638030480 comp rx=0 tx=0).stop 2024-09-10T08:41:25.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.481+0000 7fb65507e740 1 -- 172.21.15.89:0/3717258994 shutdown_connections 2024-09-10T08:41:25.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.481+0000 7fb65507e740 1 --2- 172.21.15.89:0/3717258994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c606859d0 0x563c60682d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.481+0000 7fb65507e740 1 --2- 172.21.15.89:0/3717258994 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563c606eae20 0x563c606ed240 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.481+0000 7fb65507e740 1 --2- 172.21.15.89:0/3717258994 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563c6066fca0 0x563c60682730 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.481+0000 7fb65507e740 1 --2- 172.21.15.89:0/3717258994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563c606653e0 0x563c606853e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.481+0000 7fb635600640 0 client.15699 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:25.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.481+0000 7fb65507e740 1 -- 172.21.15.89:0/3717258994 >> 172.21.15.89:0/3717258994 conn(0x563c60661050 msgr2=0x563c6066b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:25.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.481+0000 7fb65507e740 1 -- 172.21.15.89:0/3717258994 shutdown_connections 2024-09-10T08:41:25.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.481+0000 7fb65507e740 1 -- 172.21.15.89:0/3717258994 wait complete. 2024-09-10T08:41:25.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.482+0000 7fb65507e740 20 client.15699 trim_cache size 0 max 16384 2024-09-10T08:41:25.485 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:41:25.485 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2024-09-10T08:41:25.486 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "0423b541-f3e7-4035-8c01-889d0e1776fc", 2024-09-10T08:41:25.486 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2024-09-10T08:41:25.486 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:41:25.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.482+0000 7fb652600640 1 -- 172.21.15.89:0/3225493114 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb6300a2b80 msgr2=0x7fb6300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:25.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.482+0000 7fb652600640 1 --2- 172.21.15.89:0/3225493114 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb6300a2b80 0x7fb6300a5040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb644005fd0 tx=0x7fb644010040 comp rx=0 tx=0).stop 2024-09-10T08:41:25.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.482+0000 7fb652600640 1 -- 172.21.15.89:0/3225493114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb64c103ff0 msgr2=0x7fb64c19a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:25.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.482+0000 7fb652600640 1 --2- 172.21.15.89:0/3225493114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb64c103ff0 0x7fb64c19a6c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00e9f0 tx=0x7fb63c00edf0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.482+0000 7fb652600640 1 -- 172.21.15.89:0/3225493114 shutdown_connections 2024-09-10T08:41:25.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.482+0000 7fb652600640 1 --2- 172.21.15.89:0/3225493114 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb6300a2b80 0x7fb6300a5040 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.482+0000 7fb652600640 1 --2- 172.21.15.89:0/3225493114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb64c109120 0x7fb64c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.482+0000 7fb652600640 1 --2- 172.21.15.89:0/3225493114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb64c1049a0 0x7fb64c19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.482+0000 7fb652600640 1 --2- 172.21.15.89:0/3225493114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb64c103ff0 0x7fb64c19a6c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:25.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.482+0000 7fb652600640 1 -- 172.21.15.89:0/3225493114 >> 172.21.15.89:0/3225493114 conn(0x7fb64c0ffa90 msgr2=0x7fb64c100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:25.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.483+0000 7fb652600640 1 -- 172.21.15.89:0/3225493114 shutdown_connections 2024-09-10T08:41:25.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:25.483+0000 7fb652600640 1 -- 172.21.15.89:0/3225493114 wait complete. 2024-09-10T08:41:26.315 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:41:26.681 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:41:26.959 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:26 smithi089 ceph-mon[30757]: scrub queued for path: / 2024-09-10T08:41:26.959 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:26 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:41:26.959 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:26 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:41:26.959 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:26 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:41:26.959 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:26 smithi089 ceph-mon[30757]: pgmap v438: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:41:26.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:26 smithi149 ceph-mon[36167]: scrub queued for path: / 2024-09-10T08:41:26.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:26 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:41:26.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:26 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:41:26.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:26 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:41:26.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:26 smithi149 ceph-mon[36167]: pgmap v438: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:41:27.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:26 smithi111 ceph-mon[35220]: scrub queued for path: / 2024-09-10T08:41:27.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:26 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:41:27.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:26 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:41:27.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:26 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:41:27.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:26 smithi111 ceph-mon[35220]: pgmap v438: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:41:27.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.224+0000 7f860be00640 1 -- 172.21.15.89:0/3641188164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f860c108f60 msgr2=0x7f860c110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:27.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.224+0000 7f860be00640 1 --2- 172.21.15.89:0/3641188164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f860c108f60 0x7f860c110ae0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8604009ca0 tx=0x7f860402a8b0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.224+0000 7f860be00640 1 -- 172.21.15.89:0/3641188164 shutdown_connections 2024-09-10T08:41:27.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.224+0000 7f860be00640 1 --2- 172.21.15.89:0/3641188164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f860c108f60 0x7f860c110ae0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.224+0000 7f860be00640 1 --2- 172.21.15.89:0/3641188164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f860c1047e0 0x7f860c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.224+0000 7f860be00640 1 --2- 172.21.15.89:0/3641188164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f860c103e30 0x7f860c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.224+0000 7f860be00640 1 -- 172.21.15.89:0/3641188164 >> 172.21.15.89:0/3641188164 conn(0x7f860c0ff910 msgr2=0x7f860c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:27.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.224+0000 7f860be00640 1 -- 172.21.15.89:0/3641188164 shutdown_connections 2024-09-10T08:41:27.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.224+0000 7f860be00640 1 -- 172.21.15.89:0/3641188164 wait complete. 2024-09-10T08:41:27.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.225+0000 7f860be00640 1 Processor -- start 2024-09-10T08:41:27.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.225+0000 7f860be00640 1 -- start start 2024-09-10T08:41:27.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.226+0000 7f860be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f860c103e30 0x7f860c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:27.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.226+0000 7f860be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f860c1047e0 0x7f860c19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:27.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.226+0000 7f860be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f860c108f60 0x7f860c19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:27.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.226+0000 7f860be00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f860c113fc0 con 0x7f860c1047e0 2024-09-10T08:41:27.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.226+0000 7f860be00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f860c113e40 con 0x7f860c103e30 2024-09-10T08:41:27.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.226+0000 7f860be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f860c114140 con 0x7f860c108f60 2024-09-10T08:41:27.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.226+0000 7f860a000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f860c1047e0 0x7f860c19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:27.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.226+0000 7f860aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f860c103e30 0x7f860c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:27.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.226+0000 7f860a000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f860c1047e0 0x7f860c19a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47752/0 (socket says 172.21.15.89:47752) 2024-09-10T08:41:27.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.226+0000 7f860a000640 1 -- 172.21.15.89:0/3885957990 learned_addr learned my addr 172.21.15.89:0/3885957990 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:27.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.226+0000 7f860aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f860c103e30 0x7f860c19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:46836/0 (socket says 172.21.15.89:46836) 2024-09-10T08:41:27.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.226+0000 7f860b400640 1 --2- 172.21.15.89:0/3885957990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f860c108f60 0x7f860c19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:27.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.227+0000 7f860aa00640 1 -- 172.21.15.89:0/3885957990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f860c108f60 msgr2=0x7f860c19ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:27.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.227+0000 7f860aa00640 1 --2- 172.21.15.89:0/3885957990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f860c108f60 0x7f860c19ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.227+0000 7f860aa00640 1 -- 172.21.15.89:0/3885957990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f860c1047e0 msgr2=0x7f860c19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:27.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.227+0000 7f860aa00640 1 --2- 172.21.15.89:0/3885957990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f860c1047e0 0x7f860c19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.227+0000 7f860aa00640 1 -- 172.21.15.89:0/3885957990 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860c19f3b0 con 0x7f860c103e30 2024-09-10T08:41:27.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.227+0000 7f860a000640 1 --2- 172.21.15.89:0/3885957990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f860c1047e0 0x7f860c19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:27.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.227+0000 7f860aa00640 1 --2- 172.21.15.89:0/3885957990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f860c103e30 0x7f860c19a400 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00e9f0 tx=0x7f85fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:27.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.227+0000 7f8603400640 1 -- 172.21.15.89:0/3885957990 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc00ccc0 con 0x7f860c103e30 2024-09-10T08:41:27.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.227+0000 7f860be00640 1 -- 172.21.15.89:0/3885957990 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860c19f6a0 con 0x7f860c103e30 2024-09-10T08:41:27.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.227+0000 7f8603400640 1 -- 172.21.15.89:0/3885957990 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85fc010910 con 0x7f860c103e30 2024-09-10T08:41:27.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.227+0000 7f8603400640 1 -- 172.21.15.89:0/3885957990 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc018960 con 0x7f860c103e30 2024-09-10T08:41:27.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.227+0000 7f860be00640 1 -- 172.21.15.89:0/3885957990 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f860c1a6f80 con 0x7f860c103e30 2024-09-10T08:41:27.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.231+0000 7f861460f740 1 Processor -- start 2024-09-10T08:41:27.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.231+0000 7f861460f740 1 -- start start 2024-09-10T08:41:27.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.232+0000 7f861460f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15be6d50 0x560f15be7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:27.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.232+0000 7f861460f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15be7670 0x560f15bf19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:27.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.232+0000 7f8603400640 1 -- 172.21.15.89:0/3885957990 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f85fc010ac0 con 0x7f860c103e30 2024-09-10T08:41:27.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.232+0000 7f861460f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15bf1f30 0x560f15bf4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:27.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.232+0000 7f861460f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560f15b4d3e0 con 0x560f15be7670 2024-09-10T08:41:27.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.232+0000 7f861460f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560f15af4d80 con 0x560f15be6d50 2024-09-10T08:41:27.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.232+0000 7f861460f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560f15b493a0 con 0x560f15bf1f30 2024-09-10T08:41:27.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.232+0000 7f860b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15be6d50 0x560f15be7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:27.234 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.232+0000 7f860b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15be6d50 0x560f15be7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:46850/0 (socket says 172.21.15.89:46850) 2024-09-10T08:41:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.232+0000 7f860b400640 1 -- 172.21.15.89:0/568915357 learned_addr learned my addr 172.21.15.89:0/568915357 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.232+0000 7f860aa00640 1 --2- 172.21.15.89:0/568915357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15be7670 0x560f15bf19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.232+0000 7f860a000640 1 --2- 172.21.15.89:0/568915357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15bf1f30 0x560f15bf4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.232+0000 7f860b400640 1 -- 172.21.15.89:0/568915357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15bf1f30 msgr2=0x560f15bf4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.232+0000 7f860b400640 1 --2- 172.21.15.89:0/568915357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15bf1f30 0x560f15bf4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.232+0000 7f860b400640 1 -- 172.21.15.89:0/568915357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15be7670 msgr2=0x560f15bf19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.232+0000 7f860b400640 1 --2- 172.21.15.89:0/568915357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15be7670 0x560f15bf19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.233+0000 7f860b400640 1 -- 172.21.15.89:0/568915357 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f15b13ae0 con 0x560f15be6d50 2024-09-10T08:41:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.233+0000 7f860a000640 1 --2- 172.21.15.89:0/568915357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15bf1f30 0x560f15bf4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:41:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.233+0000 7f860b400640 1 --2- 172.21.15.89:0/568915357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15be6d50 0x560f15be7130 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8604004ad0 tx=0x7f8604009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18b3ee2136de552d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.233+0000 7f85f7e00640 1 -- 172.21.15.89:0/568915357 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604030660 con 0x560f15be6d50 2024-09-10T08:41:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.233+0000 7f85f7e00640 1 -- 172.21.15.89:0/568915357 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8604036030 con 0x560f15be6d50 2024-09-10T08:41:27.235 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.233+0000 7f85f7e00640 1 -- 172.21.15.89:0/568915357 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604004400 con 0x560f15be6d50 2024-09-10T08:41:27.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.234+0000 7f8603400640 1 --2- 172.21.15.89:0/3885957990 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f85f00a2b80 0x7f85f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:27.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.234+0000 7f8603400640 1 -- 172.21.15.89:0/3885957990 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f85fc0cf7a0 con 0x7f860c103e30 2024-09-10T08:41:27.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.234+0000 7f860aa00640 1 --2- 172.21.15.89:0/3885957990 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f85f00a2b80 0x7f85f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.234+0000 7f861460f740 1 -- 172.21.15.89:0/568915357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15be6d50 msgr2=0x560f15be7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.234+0000 7f861460f740 1 --2- 172.21.15.89:0/568915357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15be6d50 0x560f15be7130 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8604004ad0 tx=0x7f8604009360 comp rx=0 tx=0).stop 2024-09-10T08:41:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.234+0000 7f861460f740 1 -- 172.21.15.89:0/568915357 shutdown_connections 2024-09-10T08:41:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.234+0000 7f861460f740 1 --2- 172.21.15.89:0/568915357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15bf1f30 0x560f15bf4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.234+0000 7f861460f740 1 --2- 172.21.15.89:0/568915357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15be7670 0x560f15bf19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.234+0000 7f861460f740 1 --2- 172.21.15.89:0/568915357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15be6d50 0x560f15be7130 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.234+0000 7f861460f740 1 -- 172.21.15.89:0/568915357 >> 172.21.15.89:0/568915357 conn(0x560f15be32e0 msgr2=0x560f15be36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.234+0000 7f861460f740 1 -- 172.21.15.89:0/568915357 shutdown_connections 2024-09-10T08:41:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.235+0000 7f861460f740 1 -- 172.21.15.89:0/568915357 wait complete. 2024-09-10T08:41:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.235+0000 7f860aa00640 1 --2- 172.21.15.89:0/3885957990 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f85f00a2b80 0x7f85f00a5040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0cd240 tx=0x7f85fc009700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.235+0000 7f861460f740 1 -- start start 2024-09-10T08:41:27.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.235+0000 7f861460f740 1 Processor -- start 2024-09-10T08:41:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.236+0000 7f861460f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15be6d50 0x560f15beee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.236+0000 7f861460f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15be7670 0x560f15bef350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.236+0000 7f861460f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15bf1f30 0x560f15bef890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.236+0000 7f861460f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560f15bf4ba0 con 0x560f15be6d50 2024-09-10T08:41:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.236+0000 7f861460f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560f15b080f0 con 0x560f15bf1f30 2024-09-10T08:41:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.236+0000 7f861460f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560f15bf4d20 con 0x560f15be7670 2024-09-10T08:41:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.236+0000 7f861460f740 10 client.0 _ll_register_callbacks cb 0x560f15af99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.236+0000 7f860b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15be6d50 0x560f15beee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.236+0000 7f860b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15be6d50 0x560f15beee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47780/0 (socket says 172.21.15.89:47780) 2024-09-10T08:41:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.236+0000 7f860b400640 1 -- 172.21.15.89:0/3160980615 learned_addr learned my addr 172.21.15.89:0/3160980615 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:27.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.236+0000 7f860a000640 1 --2- 172.21.15.89:0/3160980615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15be7670 0x560f15bef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.236+0000 7f860b400640 1 --2- 172.21.15.89:0/3160980615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15bf1f30 0x560f15bef890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.237+0000 7f860a000640 1 -- 172.21.15.89:0/3160980615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15bf1f30 msgr2=0x560f15bef890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.237+0000 7f860a000640 1 --2- 172.21.15.89:0/3160980615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15bf1f30 0x560f15bef890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.237+0000 7f860a000640 1 -- 172.21.15.89:0/3160980615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15be6d50 msgr2=0x560f15beee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.237+0000 7f860a000640 1 --2- 172.21.15.89:0/3160980615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15be6d50 0x560f15beee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.237+0000 7f860a000640 1 -- 172.21.15.89:0/3160980615 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f158be3c0 con 0x560f15be7670 2024-09-10T08:41:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.237+0000 7f860b400640 1 --2- 172.21.15.89:0/3160980615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15be6d50 0x560f15beee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.237+0000 7f860a000640 1 --2- 172.21.15.89:0/3160980615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15be7670 0x560f15bef350 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009fd0 tx=0x7f85f8012e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21162af86865f8ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.237+0000 7f85f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:41:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.237+0000 7f85f4c00640 1 -- 172.21.15.89:0/3160980615 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f800fa00 con 0x560f15be7670 2024-09-10T08:41:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.237+0000 7f85f4c00640 1 -- 172.21.15.89:0/3160980615 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f85f800fba0 con 0x560f15be7670 2024-09-10T08:41:27.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.237+0000 7f861460f740 1 -- 172.21.15.89:0/3160980615 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8604036030 con 0x560f15be7670 2024-09-10T08:41:27.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.238+0000 7f85f4c00640 1 -- 172.21.15.89:0/3160980615 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f85f800fda0 con 0x560f15be7670 2024-09-10T08:41:27.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.238+0000 7f861460f740 10 client.25720 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:27.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.238+0000 7f861460f740 1 -- 172.21.15.89:0/3160980615 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560f15c090c0 con 0x560f15be7670 2024-09-10T08:41:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.238+0000 7f85f4c00640 1 -- 172.21.15.89:0/3160980615 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8604036030 con 0x560f15be7670 2024-09-10T08:41:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.238+0000 7f85f4c00640 1 -- 172.21.15.89:0/3160980615 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f85f8010530 con 0x560f15be7670 2024-09-10T08:41:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.238+0000 7f861460f740 10 client.25720 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.238+0000 7f861460f740 10 client.25720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:41:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.238+0000 7f861460f740 20 client.25720 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.238+0000 7f861460f740 1 --2- 172.21.15.89:0/3160980615 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560f15c0a580 0x560f15c0c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:27.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.238+0000 7f861460f740 4 client.25720 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:41:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.238+0000 7f861460f740 1 -- 172.21.15.89:0/3160980615 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f15b5a950 con 0x560f15c0a580 2024-09-10T08:41:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.239+0000 7f860b400640 1 --2- 172.21.15.89:0/3160980615 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560f15c0a580 0x560f15c0c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.239+0000 7f860b400640 1 --2- 172.21.15.89:0/3160980615 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560f15c0a580 0x560f15c0c990 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97148f8bb16888fc server_cookie=b43d7c2d9053f3b9 in_seq=0 out_seq=0 2024-09-10T08:41:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.239+0000 7f85f4c00640 10 client.25720 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.240+0000 7f85f4c00640 1 -- 172.21.15.89:0/3160980615 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f15b5a950 con 0x560f15c0a580 2024-09-10T08:41:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.240+0000 7f85f4c00640 10 client.25720 handle_command_reply: tid=0 2024-09-10T08:41:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.240+0000 7f861460f740 1 client.25720 shutdown 2024-09-10T08:41:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.240+0000 7f861460f740 1 -- 172.21.15.89:0/3160980615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15be7670 msgr2=0x560f15bef350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.240+0000 7f861460f740 1 --2- 172.21.15.89:0/3160980615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15be7670 0x560f15bef350 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009fd0 tx=0x7f85f8012e70 comp rx=0 tx=0).stop 2024-09-10T08:41:27.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.240+0000 7f861460f740 1 -- 172.21.15.89:0/3160980615 shutdown_connections 2024-09-10T08:41:27.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.240+0000 7f861460f740 1 --2- 172.21.15.89:0/3160980615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15bf1f30 0x560f15bef890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.240+0000 7f861460f740 1 --2- 172.21.15.89:0/3160980615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15be7670 0x560f15bef350 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.240+0000 7f861460f740 1 --2- 172.21.15.89:0/3160980615 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560f15c0a580 0x560f15c0c990 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.240+0000 7f861460f740 1 --2- 172.21.15.89:0/3160980615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15be6d50 0x560f15beee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.240+0000 7f861460f740 1 -- 172.21.15.89:0/3160980615 >> 172.21.15.89:0/3160980615 conn(0x560f15be32e0 msgr2=0x560f15be9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:27.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.240+0000 7f85f4c00640 0 client.25720 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:27.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.241+0000 7f861460f740 1 -- 172.21.15.89:0/3160980615 shutdown_connections 2024-09-10T08:41:27.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.241+0000 7f861460f740 1 -- 172.21.15.89:0/3160980615 wait complete. 2024-09-10T08:41:27.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.241+0000 7f861460f740 20 client.25720 trim_cache size 0 max 16384 2024-09-10T08:41:27.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.258+0000 7f861460f740 1 Processor -- start 2024-09-10T08:41:27.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.258+0000 7f861460f740 1 -- start start 2024-09-10T08:41:27.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.258+0000 7f861460f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15be6d50 0x560f15be97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:27.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.258+0000 7f861460f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15be7670 0x560f15bee6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:27.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.258+0000 7f861460f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15bf1f30 0x560f15bf0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:27.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.258+0000 7f861460f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f860c1af270 con 0x560f15bf1f30 2024-09-10T08:41:27.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.258+0000 7f861460f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f860c1acb40 con 0x560f15be7670 2024-09-10T08:41:27.260 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.258+0000 7f861460f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f860c1aa990 con 0x560f15be6d50 2024-09-10T08:41:27.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.259+0000 7f860b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15bf1f30 0x560f15bf0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.259+0000 7f860b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15be6d50 0x560f15be97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.259+0000 7f860b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15be6d50 0x560f15be97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:52980/0 (socket says 172.21.15.89:52980) 2024-09-10T08:41:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.259+0000 7f860b400640 1 -- 172.21.15.89:0/3683470143 learned_addr learned my addr 172.21.15.89:0/3683470143 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.259+0000 7f860a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15be7670 0x560f15bee6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.259+0000 7f860b400640 1 -- 172.21.15.89:0/3683470143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15be7670 msgr2=0x560f15bee6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.259+0000 7f860b400640 1 --2- 172.21.15.89:0/3683470143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15be7670 0x560f15bee6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.259+0000 7f860b400640 1 -- 172.21.15.89:0/3683470143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15bf1f30 msgr2=0x560f15bf0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.259+0000 7f860b400640 1 --2- 172.21.15.89:0/3683470143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15bf1f30 0x560f15bf0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.259+0000 7f860b400640 1 -- 172.21.15.89:0/3683470143 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f15beddc0 con 0x560f15be6d50 2024-09-10T08:41:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.259+0000 7f860b400640 1 --2- 172.21.15.89:0/3683470143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15be6d50 0x560f15be97b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8604003cf0 tx=0x7f8604003ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b194444fb3d330d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.260+0000 7f85f7400640 1 -- 172.21.15.89:0/3683470143 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86040309f0 con 0x560f15be6d50 2024-09-10T08:41:27.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.260+0000 7f85f7400640 1 -- 172.21.15.89:0/3683470143 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8604036070 con 0x560f15be6d50 2024-09-10T08:41:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.260+0000 7f85f7400640 1 -- 172.21.15.89:0/3683470143 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86040316a0 con 0x560f15be6d50 2024-09-10T08:41:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.260+0000 7f861460f740 1 -- 172.21.15.89:0/3683470143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15be6d50 msgr2=0x560f15be97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.260+0000 7f861460f740 1 --2- 172.21.15.89:0/3683470143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15be6d50 0x560f15be97b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8604003cf0 tx=0x7f8604003ea0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.260+0000 7f861460f740 1 -- 172.21.15.89:0/3683470143 shutdown_connections 2024-09-10T08:41:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.260+0000 7f861460f740 1 --2- 172.21.15.89:0/3683470143 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15bf1f30 0x560f15bf0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.260+0000 7f861460f740 1 --2- 172.21.15.89:0/3683470143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15be7670 0x560f15bee6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.260+0000 7f861460f740 1 --2- 172.21.15.89:0/3683470143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15be6d50 0x560f15be97b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.260+0000 7f861460f740 1 -- 172.21.15.89:0/3683470143 >> 172.21.15.89:0/3683470143 conn(0x560f15be32e0 msgr2=0x560f15bf0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.260+0000 7f861460f740 1 -- 172.21.15.89:0/3683470143 shutdown_connections 2024-09-10T08:41:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.261+0000 7f861460f740 1 -- 172.21.15.89:0/3683470143 wait complete. 2024-09-10T08:41:27.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.261+0000 7f861460f740 1 -- start start 2024-09-10T08:41:27.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.261+0000 7f861460f740 1 Processor -- start 2024-09-10T08:41:27.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.261+0000 7f861460f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15be6d50 0x560f15c02e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:27.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.261+0000 7f861460f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15be7670 0x560f15beba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:27.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.261+0000 7f861460f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15bf1f30 0x560f15bebfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:27.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.262+0000 7f861460f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f860c1ab000 con 0x560f15bf1f30 2024-09-10T08:41:27.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.262+0000 7f861460f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f860c1ac470 con 0x560f15be7670 2024-09-10T08:41:27.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.262+0000 7f861460f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f860c1ab6d0 con 0x560f15be6d50 2024-09-10T08:41:27.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.262+0000 7f861460f740 10 client.0 _ll_register_callbacks cb 0x7f85f8012620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:27.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.262+0000 7f860b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15bf1f30 0x560f15bebfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:27.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.262+0000 7f860a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15be7670 0x560f15beba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:27.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.262+0000 7f860b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15bf1f30 0x560f15bebfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47806/0 (socket says 172.21.15.89:47806) 2024-09-10T08:41:27.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.262+0000 7f860b400640 1 -- 172.21.15.89:0/2523033073 learned_addr learned my addr 172.21.15.89:0/2523033073 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:27.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.262+0000 7f860b400640 1 --2- 172.21.15.89:0/2523033073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15be6d50 0x560f15c02e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:27.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.262+0000 7f860b400640 1 -- 172.21.15.89:0/2523033073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15be6d50 msgr2=0x560f15c02e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:27.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.262+0000 7f860b400640 1 --2- 172.21.15.89:0/2523033073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15be6d50 0x560f15c02e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.262+0000 7f860b400640 1 -- 172.21.15.89:0/2523033073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15be7670 msgr2=0x560f15beba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:27.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.262+0000 7f860b400640 1 --2- 172.21.15.89:0/2523033073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15be7670 0x560f15beba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.262+0000 7f860b400640 1 -- 172.21.15.89:0/2523033073 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f15be74a0 con 0x560f15bf1f30 2024-09-10T08:41:27.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.262+0000 7f860a000640 1 --2- 172.21.15.89:0/2523033073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15be7670 0x560f15beba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:27.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.263+0000 7f860b400640 1 --2- 172.21.15.89:0/2523033073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15bf1f30 0x560f15bebfc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8604009960 tx=0x7f8604057040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133cb54e2053a7ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:27.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.263+0000 7f85f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:41:27.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.263+0000 7f85f4c00640 1 -- 172.21.15.89:0/2523033073 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86040309f0 con 0x560f15bf1f30 2024-09-10T08:41:27.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.263+0000 7f861460f740 1 -- 172.21.15.89:0/2523033073 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8604036070 con 0x560f15bf1f30 2024-09-10T08:41:27.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.263+0000 7f85f4c00640 1 -- 172.21.15.89:0/2523033073 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f8604030b90 con 0x560f15bf1f30 2024-09-10T08:41:27.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.263+0000 7f85f4c00640 1 -- 172.21.15.89:0/2523033073 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8604030d90 con 0x560f15bf1f30 2024-09-10T08:41:27.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.263+0000 7f861460f740 10 client.15717 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:27.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.263+0000 7f861460f740 1 -- 172.21.15.89:0/2523033073 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560f15c74fe0 con 0x560f15bf1f30 2024-09-10T08:41:27.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.263+0000 7f85f4c00640 1 -- 172.21.15.89:0/2523033073 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8604036070 con 0x560f15bf1f30 2024-09-10T08:41:27.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.263+0000 7f85f4c00640 1 -- 172.21.15.89:0/2523033073 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8604050040 con 0x560f15bf1f30 2024-09-10T08:41:27.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.264+0000 7f861460f740 10 client.15717 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:27.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.264+0000 7f861460f740 10 client.15717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:41:27.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.264+0000 7f861460f740 20 client.15717 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:27.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.264+0000 7f861460f740 1 --2- 172.21.15.89:0/2523033073 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560f15c75270 0x560f15c77660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:27.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.264+0000 7f861460f740 4 client.15717 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:41:27.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.264+0000 7f861460f740 1 -- 172.21.15.89:0/2523033073 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560f15b54e60 con 0x560f15c75270 2024-09-10T08:41:27.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.264+0000 7f860a000640 1 --2- 172.21.15.89:0/2523033073 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560f15c75270 0x560f15c77660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:27.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.264+0000 7f860a000640 1 --2- 172.21.15.89:0/2523033073 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560f15c75270 0x560f15c77660 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=185b7575d78c75c server_cookie=b722c861bc9adb5f in_seq=0 out_seq=0 2024-09-10T08:41:27.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.264+0000 7f85f4c00640 10 client.15717 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:27.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.265+0000 7f85f4c00640 1 -- 172.21.15.89:0/2523033073 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560f15b54e60 con 0x560f15c75270 2024-09-10T08:41:27.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.265+0000 7f85f4c00640 10 client.15717 handle_command_reply: tid=0 2024-09-10T08:41:27.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.265+0000 7f861460f740 1 client.15717 shutdown 2024-09-10T08:41:27.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.265+0000 7f861460f740 1 -- 172.21.15.89:0/2523033073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15bf1f30 msgr2=0x560f15bebfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:27.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.265+0000 7f861460f740 1 --2- 172.21.15.89:0/2523033073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15bf1f30 0x560f15bebfc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8604009960 tx=0x7f8604057040 comp rx=0 tx=0).stop 2024-09-10T08:41:27.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.265+0000 7f861460f740 1 -- 172.21.15.89:0/2523033073 shutdown_connections 2024-09-10T08:41:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.265+0000 7f861460f740 1 --2- 172.21.15.89:0/2523033073 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560f15c75270 0x560f15c77660 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.265+0000 7f861460f740 1 --2- 172.21.15.89:0/2523033073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f15bf1f30 0x560f15bebfc0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.265+0000 7f861460f740 1 --2- 172.21.15.89:0/2523033073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f15be7670 0x560f15beba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.265+0000 7f85f4c00640 0 client.15717 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.265+0000 7f861460f740 1 --2- 172.21.15.89:0/2523033073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f15be6d50 0x560f15c02e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.265+0000 7f861460f740 1 -- 172.21.15.89:0/2523033073 >> 172.21.15.89:0/2523033073 conn(0x560f15be32e0 msgr2=0x560f15be2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.265+0000 7f861460f740 1 -- 172.21.15.89:0/2523033073 shutdown_connections 2024-09-10T08:41:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.265+0000 7f861460f740 1 -- 172.21.15.89:0/2523033073 wait complete. 2024-09-10T08:41:27.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.265+0000 7f861460f740 20 client.15717 trim_cache size 0 max 16384 2024-09-10T08:41:27.271 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:41:27.271 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:41:27.272 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2024-09-10T08:41:27.272 INFO:teuthology.orchestra.run.smithi089.stdout: "0423b541-f3e7-4035-8c01-889d0e1776fc": { 2024-09-10T08:41:27.272 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2024-09-10T08:41:27.272 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "0423b541-f3e7-4035-8c01-889d0e1776fc", 2024-09-10T08:41:27.272 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2024-09-10T08:41:27.272 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:41:27.272 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:41:27.272 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:41:27.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.266+0000 7f860be00640 1 -- 172.21.15.89:0/3885957990 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f85f00a2b80 msgr2=0x7f85f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:27.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.266+0000 7f860be00640 1 --2- 172.21.15.89:0/3885957990 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f85f00a2b80 0x7f85f00a5040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0cd240 tx=0x7f85fc009700 comp rx=0 tx=0).stop 2024-09-10T08:41:27.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.266+0000 7f860be00640 1 -- 172.21.15.89:0/3885957990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f860c103e30 msgr2=0x7f860c19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:27.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.266+0000 7f860be00640 1 --2- 172.21.15.89:0/3885957990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f860c103e30 0x7f860c19a400 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00e9f0 tx=0x7f85fc00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.266+0000 7f860be00640 1 -- 172.21.15.89:0/3885957990 shutdown_connections 2024-09-10T08:41:27.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.266+0000 7f860be00640 1 --2- 172.21.15.89:0/3885957990 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f85f00a2b80 0x7f85f00a5040 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.266+0000 7f860be00640 1 --2- 172.21.15.89:0/3885957990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f860c108f60 0x7f860c19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.266+0000 7f860be00640 1 --2- 172.21.15.89:0/3885957990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f860c1047e0 0x7f860c19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.266+0000 7f860be00640 1 --2- 172.21.15.89:0/3885957990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f860c103e30 0x7f860c19a400 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:27.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.266+0000 7f860be00640 1 -- 172.21.15.89:0/3885957990 >> 172.21.15.89:0/3885957990 conn(0x7f860c0ff910 msgr2=0x7f860c100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:27.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.266+0000 7f860be00640 1 -- 172.21.15.89:0/3885957990 shutdown_connections 2024-09-10T08:41:27.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:27.266+0000 7f860be00640 1 -- 172.21.15.89:0/3885957990 wait complete. 2024-09-10T08:41:28.057 INFO:tasks.cephfs.filesystem:scrub status for tag:0423b541-f3e7-4035-8c01-889d0e1776fc - {'path': '/', 'tag': '0423b541-f3e7-4035-8c01-889d0e1776fc', 'options': 'recursive,force'} 2024-09-10T08:41:28.973 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:28 smithi149 ceph-mon[36167]: pgmap v439: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:41:29.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:28 smithi111 ceph-mon[35220]: pgmap v439: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:41:29.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:28 smithi089 ceph-mon[30757]: pgmap v439: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-10T08:41:30.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:30 smithi149 ceph-mon[36167]: pgmap v440: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T08:41:30.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:30 smithi149 ceph-mon[36167]: scrub summary: idle 2024-09-10T08:41:31.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:30 smithi111 ceph-mon[35220]: pgmap v440: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T08:41:31.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:30 smithi111 ceph-mon[35220]: scrub summary: idle 2024-09-10T08:41:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:30 smithi089 ceph-mon[30757]: pgmap v440: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T08:41:31.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:30 smithi089 ceph-mon[30757]: scrub summary: idle 2024-09-10T08:41:32.861 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:41:32.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:32 smithi149 ceph-mon[36167]: pgmap v441: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-10T08:41:33.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:32 smithi111 ceph-mon[35220]: pgmap v441: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-10T08:41:33.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:32 smithi089 ceph-mon[30757]: pgmap v441: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-10T08:41:33.251 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:41:33.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.798+0000 7f23f8e00640 1 -- 172.21.15.89:0/1336722156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23f40745c0 msgr2=0x7f23f40749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:33.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.798+0000 7f23f8e00640 1 --2- 172.21.15.89:0/1336722156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23f40745c0 0x7f23f40749a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009900 tx=0x7f23e402a480 comp rx=0 tx=0).stop 2024-09-10T08:41:33.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.799+0000 7f23f8e00640 1 -- 172.21.15.89:0/1336722156 shutdown_connections 2024-09-10T08:41:33.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.799+0000 7f23f8e00640 1 --2- 172.21.15.89:0/1336722156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23f410e260 0x7f23f4110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:33.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.799+0000 7f23f8e00640 1 --2- 172.21.15.89:0/1336722156 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23f4074ee0 0x7f23f410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:33.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.799+0000 7f23f8e00640 1 --2- 172.21.15.89:0/1336722156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23f40745c0 0x7f23f40749a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:33.800 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.799+0000 7f23f8e00640 1 -- 172.21.15.89:0/1336722156 >> 172.21.15.89:0/1336722156 conn(0x7f23f40fd740 msgr2=0x7f23f40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:33.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.799+0000 7f23f8e00640 1 -- 172.21.15.89:0/1336722156 shutdown_connections 2024-09-10T08:41:33.801 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.799+0000 7f23f8e00640 1 -- 172.21.15.89:0/1336722156 wait complete. 2024-09-10T08:41:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.800+0000 7f23f8e00640 1 Processor -- start 2024-09-10T08:41:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.800+0000 7f23f8e00640 1 -- start start 2024-09-10T08:41:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.800+0000 7f23f8e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23f40745c0 0x7f23f419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.800+0000 7f23f8e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23f4074ee0 0x7f23f419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.800+0000 7f23f8e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23f410e260 0x7f23f419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.800+0000 7f23f8e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f23f4113fb0 con 0x7f23f410e260 2024-09-10T08:41:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.800+0000 7f23f8e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f23f4113e30 con 0x7f23f4074ee0 2024-09-10T08:41:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.800+0000 7f23f8e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f23f4114130 con 0x7f23f40745c0 2024-09-10T08:41:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.801+0000 7f23f3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23f410e260 0x7f23f419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:33.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.801+0000 7f23f3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23f40745c0 0x7f23f419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:33.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.801+0000 7f23f3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23f40745c0 0x7f23f419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:51262/0 (socket says 172.21.15.89:51262) 2024-09-10T08:41:33.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.801+0000 7f23f3400640 1 -- 172.21.15.89:0/3417919645 learned_addr learned my addr 172.21.15.89:0/3417919645 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:33.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.801+0000 7f23f2a00640 1 --2- 172.21.15.89:0/3417919645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23f4074ee0 0x7f23f419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:33.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.801+0000 7f23f3e00640 1 -- 172.21.15.89:0/3417919645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23f40745c0 msgr2=0x7f23f419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:33.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.801+0000 7f23f3e00640 1 --2- 172.21.15.89:0/3417919645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23f40745c0 0x7f23f419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:33.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.801+0000 7f23f3e00640 1 -- 172.21.15.89:0/3417919645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23f4074ee0 msgr2=0x7f23f419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:33.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.801+0000 7f23f3e00640 1 --2- 172.21.15.89:0/3417919645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23f4074ee0 0x7f23f419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:33.803 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.801+0000 7f23f3e00640 1 -- 172.21.15.89:0/3417919645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f419f450 con 0x7f23f410e260 2024-09-10T08:41:33.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.802+0000 7f23f3e00640 1 --2- 172.21.15.89:0/3417919645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23f410e260 0x7f23f419ed70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f23ec005fd0 tx=0x7f23ec00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:33.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.802+0000 7f23ebe00640 1 -- 172.21.15.89:0/3417919645 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec00cdf0 con 0x7f23f410e260 2024-09-10T08:41:33.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.802+0000 7f23ebe00640 1 -- 172.21.15.89:0/3417919645 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23ec005020 con 0x7f23f410e260 2024-09-10T08:41:33.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.802+0000 7f23f8e00640 1 -- 172.21.15.89:0/3417919645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f419f740 con 0x7f23f410e260 2024-09-10T08:41:33.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.802+0000 7f23ebe00640 1 -- 172.21.15.89:0/3417919645 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec0106d0 con 0x7f23f410e260 2024-09-10T08:41:33.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.802+0000 7f23f8e00640 1 -- 172.21.15.89:0/3417919645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f41a7020 con 0x7f23f410e260 2024-09-10T08:41:33.804 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.803+0000 7f23f8e00640 1 -- 172.21.15.89:0/3417919645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f40749a0 con 0x7f23f410e260 2024-09-10T08:41:33.806 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.805+0000 7f23ebe00640 1 -- 172.21.15.89:0/3417919645 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f23ec0054d0 con 0x7f23f410e260 2024-09-10T08:41:33.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.806+0000 7f23ebe00640 1 --2- 172.21.15.89:0/3417919645 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23d80a2b80 0x7f23d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:33.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.807+0000 7f23ebe00640 1 -- 172.21.15.89:0/3417919645 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f23ec0cfb20 con 0x7f23f410e260 2024-09-10T08:41:33.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.807+0000 7f23f3400640 1 --2- 172.21.15.89:0/3417919645 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23d80a2b80 0x7f23d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:33.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.808+0000 7f23f3400640 1 --2- 172.21.15.89:0/3417919645 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23d80a2b80 0x7f23d80a5040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f23e40096c0 tx=0x7f23e4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:33.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:33.811+0000 7f23ebe00640 1 -- 172.21.15.89:0/3417919645 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f23ec09c650 con 0x7f23f410e260 2024-09-10T08:41:34.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:34.029+0000 7f23f8e00640 1 -- 172.21.15.89:0/3417919645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23f402d7c0 con 0x7f23f410e260 2024-09-10T08:41:34.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:34.030+0000 7f23ebe00640 1 -- 172.21.15.89:0/3417919645 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11918 (secure 0 0 0) 0x7f23ec09bf50 con 0x7f23f410e260 2024-09-10T08:41:34.031 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:41:34.031 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":26,"btime":"2024-09-10T08:39:39:721431+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:39:38.834417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24518,24496]},"id":1}]} 2024-09-10T08:41:34.031 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 26 2024-09-10T08:41:34.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:34.032+0000 7f23f8e00640 1 -- 172.21.15.89:0/3417919645 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23d80a2b80 msgr2=0x7f23d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:34.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:34.032+0000 7f23f8e00640 1 --2- 172.21.15.89:0/3417919645 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23d80a2b80 0x7f23d80a5040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f23e40096c0 tx=0x7f23e4005a20 comp rx=0 tx=0).stop 2024-09-10T08:41:34.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:34.032+0000 7f23f8e00640 1 -- 172.21.15.89:0/3417919645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23f410e260 msgr2=0x7f23f419ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:34.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:34.032+0000 7f23f8e00640 1 --2- 172.21.15.89:0/3417919645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23f410e260 0x7f23f419ed70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f23ec005fd0 tx=0x7f23ec00ef10 comp rx=0 tx=0).stop 2024-09-10T08:41:34.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:34.032+0000 7f23f8e00640 1 -- 172.21.15.89:0/3417919645 shutdown_connections 2024-09-10T08:41:34.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:34.032+0000 7f23f8e00640 1 --2- 172.21.15.89:0/3417919645 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23d80a2b80 0x7f23d80a5040 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:34.032+0000 7f23f8e00640 1 --2- 172.21.15.89:0/3417919645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23f410e260 0x7f23f419ed70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:34.032+0000 7f23f8e00640 1 --2- 172.21.15.89:0/3417919645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23f4074ee0 0x7f23f419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:34.032+0000 7f23f8e00640 1 --2- 172.21.15.89:0/3417919645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23f40745c0 0x7f23f419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:34.032+0000 7f23f8e00640 1 -- 172.21.15.89:0/3417919645 >> 172.21.15.89:0/3417919645 conn(0x7f23f40fd740 msgr2=0x7f23f40fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:34.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:34.033+0000 7f23f8e00640 1 -- 172.21.15.89:0/3417919645 shutdown_connections 2024-09-10T08:41:34.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:34.033+0000 7f23f8e00640 1 -- 172.21.15.89:0/3417919645 wait complete. 2024-09-10T08:41:34.939 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24554 quiesce db --set-id ada3e43a --release --await' 2024-09-10T08:41:34.939 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24554 quiesce db --set-id ada3e43a --release --await 2024-09-10T08:41:34.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:34 smithi149 ceph-mon[36167]: pgmap v442: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-10T08:41:34.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:34 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3417919645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:41:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.973+0000 7fa96de00640 1 -- 172.21.15.89:0/1268537580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9681023c0 msgr2=0x7fa9681047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.973+0000 7fa96de00640 1 --2- 172.21.15.89:0/1268537580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9681023c0 0x7fa9681047b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009900 tx=0x7fa95c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.973+0000 7fa96de00640 1 -- 172.21.15.89:0/1268537580 shutdown_connections 2024-09-10T08:41:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.973+0000 7fa96de00640 1 --2- 172.21.15.89:0/1268537580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa968107620 0x7fa968109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.973+0000 7fa96de00640 1 --2- 172.21.15.89:0/1268537580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa968104cf0 0x7fa9681070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.973+0000 7fa96de00640 1 --2- 172.21.15.89:0/1268537580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9681023c0 0x7fa9681047b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.973+0000 7fa96de00640 1 -- 172.21.15.89:0/1268537580 >> 172.21.15.89:0/1268537580 conn(0x7fa9680fbfa0 msgr2=0x7fa9680fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.973+0000 7fa96de00640 1 -- 172.21.15.89:0/1268537580 shutdown_connections 2024-09-10T08:41:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.973+0000 7fa96de00640 1 -- 172.21.15.89:0/1268537580 wait complete. 2024-09-10T08:41:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.974+0000 7fa96de00640 1 Processor -- start 2024-09-10T08:41:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.974+0000 7fa96de00640 1 -- start start 2024-09-10T08:41:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.975+0000 7fa96de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9681023c0 0x7fa968199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.975+0000 7fa96de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa968104cf0 0x7fa96819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.975+0000 7fa96de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa968107620 0x7fa9681a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.975+0000 7fa96de00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa96810d620 con 0x7fa9681023c0 2024-09-10T08:41:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.975+0000 7fa96de00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa96810d4a0 con 0x7fa968104cf0 2024-09-10T08:41:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.975+0000 7fa96de00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa96810d7a0 con 0x7fa968107620 2024-09-10T08:41:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.975+0000 7fa96ca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9681023c0 0x7fa968199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.975+0000 7fa96ca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9681023c0 0x7fa968199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56526/0 (socket says 172.21.15.89:56526) 2024-09-10T08:41:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.975+0000 7fa96ca00640 1 -- 172.21.15.89:0/3244125733 learned_addr learned my addr 172.21.15.89:0/3244125733 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.975+0000 7fa967e00640 1 --2- 172.21.15.89:0/3244125733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa968104cf0 0x7fa96819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.975+0000 7fa96d400640 1 --2- 172.21.15.89:0/3244125733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa968107620 0x7fa9681a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.975+0000 7fa96ca00640 1 -- 172.21.15.89:0/3244125733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa968107620 msgr2=0x7fa9681a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.975+0000 7fa96ca00640 1 --2- 172.21.15.89:0/3244125733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa968107620 0x7fa9681a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.975+0000 7fa96ca00640 1 -- 172.21.15.89:0/3244125733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa968104cf0 msgr2=0x7fa96819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.975+0000 7fa96ca00640 1 --2- 172.21.15.89:0/3244125733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa968104cf0 0x7fa96819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.975+0000 7fa96ca00640 1 -- 172.21.15.89:0/3244125733 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9681a1b00 con 0x7fa9681023c0 2024-09-10T08:41:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.976+0000 7fa96ca00640 1 --2- 172.21.15.89:0/3244125733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9681023c0 0x7fa968199ee0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009230 tx=0x7fa95c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.976+0000 7fa965600640 1 -- 172.21.15.89:0/3244125733 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c03f030 con 0x7fa9681023c0 2024-09-10T08:41:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.976+0000 7fa965600640 1 -- 172.21.15.89:0/3244125733 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa95c003ce0 con 0x7fa9681023c0 2024-09-10T08:41:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.976+0000 7fa96de00640 1 -- 172.21.15.89:0/3244125733 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9681a1d90 con 0x7fa9681023c0 2024-09-10T08:41:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.976+0000 7fa965600640 1 -- 172.21.15.89:0/3244125733 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c035070 con 0x7fa9681023c0 2024-09-10T08:41:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.976+0000 7fa96de00640 1 -- 172.21.15.89:0/3244125733 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9681a21d0 con 0x7fa9681023c0 2024-09-10T08:41:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.979+0000 7fa965600640 1 -- 172.21.15.89:0/3244125733 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa95c042030 con 0x7fa9681023c0 2024-09-10T08:41:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.980+0000 7fa9723e2740 1 Processor -- start 2024-09-10T08:41:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.981+0000 7fa9723e2740 1 -- start start 2024-09-10T08:41:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.981+0000 7fa965600640 1 --2- 172.21.15.89:0/3244125733 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa9500a2b80 0x7fa9500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.981+0000 7fa9723e2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248e8260 0x5630248e8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.981+0000 7fa9723e2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248e8ba0 0x5630248ed660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.981+0000 7fa965600640 1 -- 172.21.15.89:0/3244125733 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa95c0ef930 con 0x7fa9681023c0 2024-09-10T08:41:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.981+0000 7fa9723e2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248edc30 0x5630248f00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.981+0000 7fa9723e2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56302484d100 con 0x5630248e8260 2024-09-10T08:41:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.981+0000 7fa9723e2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5630247f51b0 con 0x5630248edc30 2024-09-10T08:41:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.981+0000 7fa9723e2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5630248490c0 con 0x5630248e8ba0 2024-09-10T08:41:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.981+0000 7fa96d400640 1 --2- 172.21.15.89:0/3244125733 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa9500a2b80 0x7fa9500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.981+0000 7fa96ca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248e8260 0x5630248e8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.981+0000 7fa96ca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248e8260 0x5630248e8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56542/0 (socket says 172.21.15.89:56542) 2024-09-10T08:41:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.981+0000 7fa96ca00640 1 -- 172.21.15.89:0/2121552760 learned_addr learned my addr 172.21.15.89:0/2121552760 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.981+0000 7fa96d400640 1 --2- 172.21.15.89:0/2121552760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248edc30 0x5630248f00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.981+0000 7fa967e00640 1 --2- 172.21.15.89:0/2121552760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248e8ba0 0x5630248ed660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.982+0000 7fa96ca00640 1 -- 172.21.15.89:0/2121552760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248e8ba0 msgr2=0x5630248ed660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.982+0000 7fa96ca00640 1 --2- 172.21.15.89:0/2121552760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248e8ba0 0x5630248ed660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.982+0000 7fa96ca00640 1 -- 172.21.15.89:0/2121552760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248edc30 msgr2=0x5630248f00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.982+0000 7fa96ca00640 1 --2- 172.21.15.89:0/2121552760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248edc30 0x5630248f00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.982+0000 7fa96ca00640 1 -- 172.21.15.89:0/2121552760 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563024560210 con 0x5630248e8260 2024-09-10T08:41:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.982+0000 7fa967e00640 1 --2- 172.21.15.89:0/2121552760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248e8ba0 0x5630248ed660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:41:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.982+0000 7fa96ca00640 1 --2- 172.21.15.89:0/2121552760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248e8260 0x5630248e8660 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0bbd50 tx=0x7fa95c0bb9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c831ed2eead0fa6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.982+0000 7fa96d400640 1 --2- 172.21.15.89:0/3244125733 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa9500a2b80 0x7fa9500a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa960005c40 tx=0x7fa960009df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.982+0000 7fa959c00640 1 -- 172.21.15.89:0/2121552760 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c049040 con 0x5630248e8260 2024-09-10T08:41:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.982+0000 7fa959c00640 1 -- 172.21.15.89:0/2121552760 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa95c0bc4c0 con 0x5630248e8260 2024-09-10T08:41:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.982+0000 7fa959c00640 1 -- 172.21.15.89:0/2121552760 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c0b8430 con 0x5630248e8260 2024-09-10T08:41:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.983+0000 7fa9723e2740 1 -- 172.21.15.89:0/2121552760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248e8260 msgr2=0x5630248e8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.983+0000 7fa9723e2740 1 --2- 172.21.15.89:0/2121552760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248e8260 0x5630248e8660 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0bbd50 tx=0x7fa95c0bb9c0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.983+0000 7fa9723e2740 1 -- 172.21.15.89:0/2121552760 shutdown_connections 2024-09-10T08:41:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.983+0000 7fa9723e2740 1 --2- 172.21.15.89:0/2121552760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248edc30 0x5630248f00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.983+0000 7fa9723e2740 1 --2- 172.21.15.89:0/2121552760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248e8ba0 0x5630248ed660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.983+0000 7fa9723e2740 1 --2- 172.21.15.89:0/2121552760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248e8260 0x5630248e8660 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.983+0000 7fa9723e2740 1 -- 172.21.15.89:0/2121552760 >> 172.21.15.89:0/2121552760 conn(0x5630248e38d0 msgr2=0x5630248e3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.983+0000 7fa9723e2740 1 -- 172.21.15.89:0/2121552760 shutdown_connections 2024-09-10T08:41:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.983+0000 7fa9723e2740 1 -- 172.21.15.89:0/2121552760 wait complete. 2024-09-10T08:41:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.984+0000 7fa9723e2740 1 -- start start 2024-09-10T08:41:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.984+0000 7fa9723e2740 1 Processor -- start 2024-09-10T08:41:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.985+0000 7fa9723e2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248e8260 0x5630248f2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.985+0000 7fa9723e2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248e8ba0 0x5630248f2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.985+0000 7fa9723e2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248edc30 0x5630248f9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.985+0000 7fa9723e2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5630248f09c0 con 0x5630248edc30 2024-09-10T08:41:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.985+0000 7fa9723e2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5630247d6090 con 0x5630248e8ba0 2024-09-10T08:41:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.985+0000 7fa9723e2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5630248f0b40 con 0x5630248e8260 2024-09-10T08:41:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.985+0000 7fa9723e2740 10 client.0 _ll_register_callbacks cb 0x56302482af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.985+0000 7fa96d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248edc30 0x5630248f9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.985+0000 7fa96d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248edc30 0x5630248f9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56552/0 (socket says 172.21.15.89:56552) 2024-09-10T08:41:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.985+0000 7fa96d400640 1 -- 172.21.15.89:0/943780772 learned_addr learned my addr 172.21.15.89:0/943780772 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.985+0000 7fa967e00640 1 --2- 172.21.15.89:0/943780772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248e8ba0 0x5630248f2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.985+0000 7fa96ca00640 1 --2- 172.21.15.89:0/943780772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248e8260 0x5630248f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.985+0000 7fa96d400640 1 -- 172.21.15.89:0/943780772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248e8260 msgr2=0x5630248f2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.985+0000 7fa96d400640 1 --2- 172.21.15.89:0/943780772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248e8260 0x5630248f2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.985+0000 7fa96d400640 1 -- 172.21.15.89:0/943780772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248e8ba0 msgr2=0x5630248f2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.985+0000 7fa96d400640 1 --2- 172.21.15.89:0/943780772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248e8ba0 0x5630248f2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.986+0000 7fa96d400640 1 -- 172.21.15.89:0/943780772 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56302462c620 con 0x5630248edc30 2024-09-10T08:41:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.986+0000 7fa967e00640 1 --2- 172.21.15.89:0/943780772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248e8ba0 0x5630248f2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:41:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.986+0000 7fa96d400640 1 --2- 172.21.15.89:0/943780772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248edc30 0x5630248f9680 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa960012fd0 tx=0x7fa96001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a7d185cabd927c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.986+0000 7fa94e000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:41:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.986+0000 7fa94e000640 1 -- 172.21.15.89:0/943780772 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa960012110 con 0x5630248edc30 2024-09-10T08:41:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.986+0000 7fa94e000640 1 -- 172.21.15.89:0/943780772 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa960004540 con 0x5630248edc30 2024-09-10T08:41:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.986+0000 7fa9723e2740 1 -- 172.21.15.89:0/943780772 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa95c0bc4c0 con 0x5630248edc30 2024-09-10T08:41:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.986+0000 7fa94e000640 1 -- 172.21.15.89:0/943780772 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa960031050 con 0x5630248edc30 2024-09-10T08:41:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.986+0000 7fa9723e2740 10 client.15735 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.986+0000 7fa9723e2740 1 -- 172.21.15.89:0/943780772 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5630248fc4e0 con 0x5630248edc30 2024-09-10T08:41:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.987+0000 7fa94e000640 1 -- 172.21.15.89:0/943780772 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa95c0bc4c0 con 0x5630248edc30 2024-09-10T08:41:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.987+0000 7fa94e000640 1 -- 172.21.15.89:0/943780772 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa96001f050 con 0x5630248edc30 2024-09-10T08:41:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.987+0000 7fa9723e2740 10 client.15735 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.987+0000 7fa9723e2740 10 client.15735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:41:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.987+0000 7fa9723e2740 20 client.15735 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.987+0000 7fa9723e2740 1 --2- 172.21.15.89:0/943780772 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5630248fdbd0 0x563024900020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.987+0000 7fa9723e2740 4 client.15735 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:41:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.987+0000 7fa9723e2740 1 -- 172.21.15.89:0/943780772 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56302485a830 con 0x5630248fdbd0 2024-09-10T08:41:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.987+0000 7fa96ca00640 1 --2- 172.21.15.89:0/943780772 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5630248fdbd0 0x563024900020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.988+0000 7fa96ca00640 1 --2- 172.21.15.89:0/943780772 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5630248fdbd0 0x563024900020 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a157ff48ab149725 server_cookie=266b47ca00bfc919 in_seq=0 out_seq=0 2024-09-10T08:41:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.988+0000 7fa94e000640 10 client.15735 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.989+0000 7fa94e000640 1 -- 172.21.15.89:0/943780772 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56302485a830 con 0x5630248fdbd0 2024-09-10T08:41:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.989+0000 7fa94e000640 10 client.15735 handle_command_reply: tid=0 2024-09-10T08:41:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.989+0000 7fa9723e2740 1 client.15735 shutdown 2024-09-10T08:41:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.989+0000 7fa9723e2740 1 -- 172.21.15.89:0/943780772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248edc30 msgr2=0x5630248f9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.989+0000 7fa9723e2740 1 --2- 172.21.15.89:0/943780772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248edc30 0x5630248f9680 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa960012fd0 tx=0x7fa96001c000 comp rx=0 tx=0).stop 2024-09-10T08:41:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.989+0000 7fa9723e2740 1 -- 172.21.15.89:0/943780772 shutdown_connections 2024-09-10T08:41:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.989+0000 7fa9723e2740 1 --2- 172.21.15.89:0/943780772 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5630248fdbd0 0x563024900020 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.989+0000 7fa9723e2740 1 --2- 172.21.15.89:0/943780772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248edc30 0x5630248f9680 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.989+0000 7fa9723e2740 1 --2- 172.21.15.89:0/943780772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248e8ba0 0x5630248f2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.989+0000 7fa9723e2740 1 --2- 172.21.15.89:0/943780772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248e8260 0x5630248f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.989+0000 7fa94e000640 0 client.15735 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.989+0000 7fa9723e2740 1 -- 172.21.15.89:0/943780772 >> 172.21.15.89:0/943780772 conn(0x5630248e38d0 msgr2=0x5630248e50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.989+0000 7fa9723e2740 1 -- 172.21.15.89:0/943780772 shutdown_connections 2024-09-10T08:41:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.989+0000 7fa9723e2740 1 -- 172.21.15.89:0/943780772 wait complete. 2024-09-10T08:41:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:34.989+0000 7fa9723e2740 20 client.15735 trim_cache size 0 max 16384 2024-09-10T08:41:34.994 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:34 smithi089 ceph-mon[30757]: pgmap v442: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-10T08:41:34.994 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:34 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3417919645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:41:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.005+0000 7fa9723e2740 1 Processor -- start 2024-09-10T08:41:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.005+0000 7fa9723e2740 1 -- start start 2024-09-10T08:41:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.005+0000 7fa9723e2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248e8260 0x5630248f2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.005+0000 7fa9723e2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248e8ba0 0x5630248f9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.005+0000 7fa9723e2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248edc30 0x5630248f2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.005+0000 7fa9723e2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681af1b0 con 0x5630248e8ba0 2024-09-10T08:41:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.005+0000 7fa9723e2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681aca80 con 0x5630248e8260 2024-09-10T08:41:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.005+0000 7fa9723e2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681aa8d0 con 0x5630248edc30 2024-09-10T08:41:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.006+0000 7fa96ca00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248e8260 0x5630248f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.005+0000 7fa967e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248e8ba0 0x5630248f9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.006+0000 7fa96d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248edc30 0x5630248f2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.006+0000 7fa967e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248e8ba0 0x5630248f9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56568/0 (socket says 172.21.15.89:56568) 2024-09-10T08:41:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.006+0000 7fa967e00640 1 -- 172.21.15.89:0/3859729380 learned_addr learned my addr 172.21.15.89:0/3859729380 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.006+0000 7fa967e00640 1 -- 172.21.15.89:0/3859729380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248edc30 msgr2=0x5630248f2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.006+0000 7fa967e00640 1 --2- 172.21.15.89:0/3859729380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248edc30 0x5630248f2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.006+0000 7fa967e00640 1 -- 172.21.15.89:0/3859729380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248e8260 msgr2=0x5630248f2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.006+0000 7fa967e00640 1 --2- 172.21.15.89:0/3859729380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248e8260 0x5630248f2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.006+0000 7fa967e00640 1 -- 172.21.15.89:0/3859729380 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630248e4560 con 0x5630248e8ba0 2024-09-10T08:41:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.006+0000 7fa96d400640 1 --2- 172.21.15.89:0/3859729380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248edc30 0x5630248f2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.006+0000 7fa967e00640 1 --2- 172.21.15.89:0/3859729380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248e8ba0 0x5630248f9640 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa954009bd0 tx=0x7fa954010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db38463c2a38b458 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.006+0000 7fa959200640 1 -- 172.21.15.89:0/3859729380 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954005220 con 0x5630248e8ba0 2024-09-10T08:41:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.007+0000 7fa959200640 1 -- 172.21.15.89:0/3859729380 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa954009e40 con 0x5630248e8ba0 2024-09-10T08:41:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.007+0000 7fa959200640 1 -- 172.21.15.89:0/3859729380 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95401c720 con 0x5630248e8ba0 2024-09-10T08:41:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.007+0000 7fa9723e2740 1 -- 172.21.15.89:0/3859729380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248e8ba0 msgr2=0x5630248f9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.007+0000 7fa9723e2740 1 --2- 172.21.15.89:0/3859729380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248e8ba0 0x5630248f9640 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa954009bd0 tx=0x7fa954010800 comp rx=0 tx=0).stop 2024-09-10T08:41:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.007+0000 7fa9723e2740 1 -- 172.21.15.89:0/3859729380 shutdown_connections 2024-09-10T08:41:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.007+0000 7fa9723e2740 1 --2- 172.21.15.89:0/3859729380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248edc30 0x5630248f2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.007+0000 7fa9723e2740 1 --2- 172.21.15.89:0/3859729380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248e8ba0 0x5630248f9640 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.007+0000 7fa9723e2740 1 --2- 172.21.15.89:0/3859729380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248e8260 0x5630248f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.007+0000 7fa9723e2740 1 -- 172.21.15.89:0/3859729380 >> 172.21.15.89:0/3859729380 conn(0x5630248e38d0 msgr2=0x5630248e9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.007+0000 7fa9723e2740 1 -- 172.21.15.89:0/3859729380 shutdown_connections 2024-09-10T08:41:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.007+0000 7fa9723e2740 1 -- 172.21.15.89:0/3859729380 wait complete. 2024-09-10T08:41:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.008+0000 7fa9723e2740 1 -- start start 2024-09-10T08:41:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.008+0000 7fa9723e2740 1 Processor -- start 2024-09-10T08:41:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.008+0000 7fa9723e2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248e8260 0x5630248f6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.008+0000 7fa9723e2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248e8ba0 0x5630248f6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.008+0000 7fa9723e2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248edc30 0x5630249668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.008+0000 7fa9723e2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681aaf40 con 0x5630248edc30 2024-09-10T08:41:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.008+0000 7fa9723e2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681ac3b0 con 0x5630248e8260 2024-09-10T08:41:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.008+0000 7fa9723e2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681ab610 con 0x5630248e8ba0 2024-09-10T08:41:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.009+0000 7fa9723e2740 10 client.0 _ll_register_callbacks cb 0x7fa96000eb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.009+0000 7fa96d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248edc30 0x5630249668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.009+0000 7fa96ca00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248e8260 0x5630248f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.009+0000 7fa96ca00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248e8260 0x5630248f6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57652/0 (socket says 172.21.15.89:57652) 2024-09-10T08:41:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.009+0000 7fa96ca00640 1 -- 172.21.15.89:0/2602350854 learned_addr learned my addr 172.21.15.89:0/2602350854 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.009+0000 7fa96d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248edc30 0x5630249668f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56580/0 (socket says 172.21.15.89:56580) 2024-09-10T08:41:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.009+0000 7fa967e00640 1 --2- 172.21.15.89:0/2602350854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248e8ba0 0x5630248f6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.009+0000 7fa96ca00640 1 -- 172.21.15.89:0/2602350854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248e8ba0 msgr2=0x5630248f6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.009+0000 7fa96ca00640 1 --2- 172.21.15.89:0/2602350854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248e8ba0 0x5630248f6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.009+0000 7fa96ca00640 1 -- 172.21.15.89:0/2602350854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248edc30 msgr2=0x5630249668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.009+0000 7fa96ca00640 1 --2- 172.21.15.89:0/2602350854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248edc30 0x5630249668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.009+0000 7fa96ca00640 1 -- 172.21.15.89:0/2602350854 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630248f6f80 con 0x5630248e8260 2024-09-10T08:41:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.009+0000 7fa96ca00640 1 --2- 172.21.15.89:0/2602350854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248e8260 0x5630248f6400 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0314e0 tx=0x7fa95c0058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d81e04a855130061 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.009+0000 7fa94e000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:41:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.009+0000 7fa94e000640 1 -- 172.21.15.89:0/2602350854 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c049040 con 0x5630248e8260 2024-09-10T08:41:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.010+0000 7fa94e000640 1 -- 172.21.15.89:0/2602350854 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa95c03b460 con 0x5630248e8260 2024-09-10T08:41:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.010+0000 7fa9723e2740 1 -- 172.21.15.89:0/2602350854 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563024854b80 con 0x5630248e8260 2024-09-10T08:41:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.010+0000 7fa94e000640 1 -- 172.21.15.89:0/2602350854 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa95c0bc4c0 con 0x5630248e8260 2024-09-10T08:41:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.010+0000 7fa9723e2740 10 client.25805 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.010+0000 7fa9723e2740 1 -- 172.21.15.89:0/2602350854 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563024967f10 con 0x5630248e8260 2024-09-10T08:41:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.010+0000 7fa94e000640 1 -- 172.21.15.89:0/2602350854 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563024854b80 con 0x5630248e8260 2024-09-10T08:41:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.010+0000 7fa94e000640 1 -- 172.21.15.89:0/2602350854 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa95c0c45a0 con 0x5630248e8260 2024-09-10T08:41:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.010+0000 7fa9723e2740 10 client.25805 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.010+0000 7fa9723e2740 10 client.25805 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:41:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.010+0000 7fa9723e2740 20 client.25805 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.010+0000 7fa9723e2740 1 --2- 172.21.15.89:0/2602350854 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563024968730 0x56302496ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.010+0000 7fa9723e2740 4 client.25805 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:41:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.010+0000 7fa9723e2740 1 -- 172.21.15.89:0/2602350854 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630247d17c0 con 0x563024968730 2024-09-10T08:41:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.011+0000 7fa967e00640 1 --2- 172.21.15.89:0/2602350854 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563024968730 0x56302496ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.011+0000 7fa967e00640 1 --2- 172.21.15.89:0/2602350854 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563024968730 0x56302496ab50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=505ed6b0c347fb34 server_cookie=f450d625fcf725ba in_seq=0 out_seq=0 2024-09-10T08:41:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.011+0000 7fa94e000640 10 client.25805 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.015+0000 7fa94e000640 1 -- 172.21.15.89:0/2602350854 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5630247d17c0 con 0x563024968730 2024-09-10T08:41:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.015+0000 7fa94e000640 10 client.25805 handle_command_reply: tid=0 2024-09-10T08:41:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.015+0000 7fa9723e2740 1 client.25805 shutdown 2024-09-10T08:41:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.015+0000 7fa9723e2740 1 -- 172.21.15.89:0/2602350854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248e8260 msgr2=0x5630248f6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.015+0000 7fa9723e2740 1 --2- 172.21.15.89:0/2602350854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248e8260 0x5630248f6400 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0314e0 tx=0x7fa95c0058d0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.015+0000 7fa9723e2740 1 -- 172.21.15.89:0/2602350854 shutdown_connections 2024-09-10T08:41:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.015+0000 7fa9723e2740 1 --2- 172.21.15.89:0/2602350854 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563024968730 0x56302496ab50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.015+0000 7fa9723e2740 1 --2- 172.21.15.89:0/2602350854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630248edc30 0x5630249668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.015+0000 7fa9723e2740 1 --2- 172.21.15.89:0/2602350854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630248e8ba0 0x5630248f6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.015+0000 7fa9723e2740 1 --2- 172.21.15.89:0/2602350854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630248e8260 0x5630248f6400 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.015+0000 7fa9723e2740 1 -- 172.21.15.89:0/2602350854 >> 172.21.15.89:0/2602350854 conn(0x5630248e38d0 msgr2=0x5630248e9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.015+0000 7fa94e000640 0 client.25805 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.015+0000 7fa9723e2740 1 -- 172.21.15.89:0/2602350854 shutdown_connections 2024-09-10T08:41:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.015+0000 7fa9723e2740 1 -- 172.21.15.89:0/2602350854 wait complete. 2024-09-10T08:41:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.015+0000 7fa9723e2740 20 client.25805 trim_cache size 0 max 16384 2024-09-10T08:41:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:41:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:41:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:41:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:41:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.016+0000 7fa96de00640 1 -- 172.21.15.89:0/3244125733 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa9500a2b80 msgr2=0x7fa9500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.016+0000 7fa96de00640 1 --2- 172.21.15.89:0/3244125733 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa9500a2b80 0x7fa9500a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa960005c40 tx=0x7fa960009df0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.016+0000 7fa96de00640 1 -- 172.21.15.89:0/3244125733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9681023c0 msgr2=0x7fa968199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.016+0000 7fa96de00640 1 --2- 172.21.15.89:0/3244125733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9681023c0 0x7fa968199ee0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009230 tx=0x7fa95c0042f0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.016+0000 7fa96de00640 1 -- 172.21.15.89:0/3244125733 shutdown_connections 2024-09-10T08:41:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.016+0000 7fa96de00640 1 --2- 172.21.15.89:0/3244125733 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa9500a2b80 0x7fa9500a5040 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.016+0000 7fa96de00640 1 --2- 172.21.15.89:0/3244125733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa968107620 0x7fa9681a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.016+0000 7fa96de00640 1 --2- 172.21.15.89:0/3244125733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa968104cf0 0x7fa96819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.016+0000 7fa96de00640 1 --2- 172.21.15.89:0/3244125733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9681023c0 0x7fa968199ee0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.016+0000 7fa96de00640 1 -- 172.21.15.89:0/3244125733 >> 172.21.15.89:0/3244125733 conn(0x7fa9680fbfa0 msgr2=0x7fa968101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.016+0000 7fa96de00640 1 -- 172.21.15.89:0/3244125733 shutdown_connections 2024-09-10T08:41:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:35.016+0000 7fa96de00640 1 -- 172.21.15.89:0/3244125733 wait complete. 2024-09-10T08:41:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:34 smithi111 ceph-mon[35220]: pgmap v442: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-10T08:41:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:34 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3417919645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:41:35.317 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:41:35.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.864+0000 7f2a32e00640 1 -- 172.21.15.89:0/1893245474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a2c10bfc0 msgr2=0x7f2a2c10e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:35.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.864+0000 7f2a32e00640 1 --2- 172.21.15.89:0/1893245474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a2c10bfc0 0x7f2a2c10e460 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009ca0 tx=0x7f2a2402a950 comp rx=0 tx=0).stop 2024-09-10T08:41:35.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.864+0000 7f2a32e00640 1 -- 172.21.15.89:0/1893245474 shutdown_connections 2024-09-10T08:41:35.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.864+0000 7f2a32e00640 1 --2- 172.21.15.89:0/1893245474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a2c10bfc0 0x7f2a2c10e460 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.864+0000 7f2a32e00640 1 --2- 172.21.15.89:0/1893245474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a2c1034b0 0x7f2a2c10b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.864+0000 7f2a32e00640 1 --2- 172.21.15.89:0/1893245474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a2c102b90 0x7f2a2c102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.864+0000 7f2a32e00640 1 -- 172.21.15.89:0/1893245474 >> 172.21.15.89:0/1893245474 conn(0x7f2a2c0fc610 msgr2=0x7f2a2c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:35.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.865+0000 7f2a32e00640 1 -- 172.21.15.89:0/1893245474 shutdown_connections 2024-09-10T08:41:35.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.865+0000 7f2a32e00640 1 -- 172.21.15.89:0/1893245474 wait complete. 2024-09-10T08:41:35.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.866+0000 7f2a32e00640 1 Processor -- start 2024-09-10T08:41:35.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.866+0000 7f2a32e00640 1 -- start start 2024-09-10T08:41:35.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.866+0000 7f2a32e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a2c102b90 0x7f2a2c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.866+0000 7f2a32e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a2c1034b0 0x7f2a2c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.866+0000 7f2a32e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a2c10bfc0 0x7f2a2c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.866+0000 7f2a32e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c111df0 con 0x7f2a2c1034b0 2024-09-10T08:41:35.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.867+0000 7f2a32e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c111c70 con 0x7f2a2c10bfc0 2024-09-10T08:41:35.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.867+0000 7f2a32e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c111f70 con 0x7f2a2c102b90 2024-09-10T08:41:35.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.867+0000 7f2a31000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a2c1034b0 0x7f2a2c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.867+0000 7f2a31a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a2c102b90 0x7f2a2c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.867+0000 7f2a31000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a2c1034b0 0x7f2a2c19a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56588/0 (socket says 172.21.15.89:56588) 2024-09-10T08:41:35.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.867+0000 7f2a31000640 1 -- 172.21.15.89:0/1284263495 learned_addr learned my addr 172.21.15.89:0/1284263495 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:35.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.867+0000 7f2a32400640 1 --2- 172.21.15.89:0/1284263495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a2c10bfc0 0x7f2a2c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.867+0000 7f2a31000640 1 -- 172.21.15.89:0/1284263495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a2c102b90 msgr2=0x7f2a2c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:35.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.867+0000 7f2a31000640 1 --2- 172.21.15.89:0/1284263495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a2c102b90 0x7f2a2c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.867+0000 7f2a31000640 1 -- 172.21.15.89:0/1284263495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a2c10bfc0 msgr2=0x7f2a2c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:35.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.867+0000 7f2a31000640 1 --2- 172.21.15.89:0/1284263495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a2c10bfc0 0x7f2a2c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.867+0000 7f2a31000640 1 -- 172.21.15.89:0/1284263495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2c19f410 con 0x7f2a2c1034b0 2024-09-10T08:41:35.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.867+0000 7f2a32400640 1 --2- 172.21.15.89:0/1284263495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a2c10bfc0 0x7f2a2c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:35.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.867+0000 7f2a31a00640 1 --2- 172.21.15.89:0/1284263495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a2c102b90 0x7f2a2c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:35.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.868+0000 7f2a31000640 1 --2- 172.21.15.89:0/1284263495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a2c1034b0 0x7f2a2c19a9a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800d750 tx=0x7f2a1800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:35.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.868+0000 7f2a2a000640 1 -- 172.21.15.89:0/1284263495 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18017070 con 0x7f2a2c1034b0 2024-09-10T08:41:35.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.868+0000 7f2a32e00640 1 -- 172.21.15.89:0/1284263495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a2c19f700 con 0x7f2a2c1034b0 2024-09-10T08:41:35.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.868+0000 7f2a2a000640 1 -- 172.21.15.89:0/1284263495 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a18005020 con 0x7f2a2c1034b0 2024-09-10T08:41:35.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.868+0000 7f2a2a000640 1 -- 172.21.15.89:0/1284263495 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18004d00 con 0x7f2a2c1034b0 2024-09-10T08:41:35.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.868+0000 7f2a32e00640 1 -- 172.21.15.89:0/1284263495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a2c1a6fe0 con 0x7f2a2c1034b0 2024-09-10T08:41:35.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.871+0000 7f2a2a000640 1 -- 172.21.15.89:0/1284263495 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2a18009a30 con 0x7f2a2c1034b0 2024-09-10T08:41:35.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.871+0000 7f2a36eb5740 1 Processor -- start 2024-09-10T08:41:35.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.871+0000 7f2a36eb5740 1 -- start start 2024-09-10T08:41:35.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.872+0000 7f2a36eb5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8a4520 0x5630ec8a4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.872+0000 7f2a36eb5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8a4e40 0x5630ec8af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.872+0000 7f2a36eb5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8af700 0x5630ec8b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.872+0000 7f2a36eb5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5630ec80a650 con 0x5630ec8a4e40 2024-09-10T08:41:35.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.872+0000 7f2a36eb5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5630ec7b1f10 con 0x5630ec8a4520 2024-09-10T08:41:35.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.872+0000 7f2a36eb5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5630ec806610 con 0x5630ec8af700 2024-09-10T08:41:35.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.872+0000 7f2a31a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8a4e40 0x5630ec8af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.872+0000 7f2a32400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8a4520 0x5630ec8a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.872+0000 7f2a32400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8a4520 0x5630ec8a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57680/0 (socket says 172.21.15.89:57680) 2024-09-10T08:41:35.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.872+0000 7f2a32400640 1 -- 172.21.15.89:0/3541140657 learned_addr learned my addr 172.21.15.89:0/3541140657 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:35.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.872+0000 7f2a31000640 1 --2- 172.21.15.89:0/3541140657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8af700 0x5630ec8b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.873+0000 7f2a31a00640 1 -- 172.21.15.89:0/3541140657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8af700 msgr2=0x5630ec8b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:35.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.873+0000 7f2a31a00640 1 --2- 172.21.15.89:0/3541140657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8af700 0x5630ec8b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.873+0000 7f2a31a00640 1 -- 172.21.15.89:0/3541140657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8a4520 msgr2=0x5630ec8a4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:35.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.873+0000 7f2a31a00640 1 --2- 172.21.15.89:0/3541140657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8a4520 0x5630ec8a4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.873+0000 7f2a31a00640 1 -- 172.21.15.89:0/3541140657 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630ec519580 con 0x5630ec8a4e40 2024-09-10T08:41:35.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.873+0000 7f2a31a00640 1 --2- 172.21.15.89:0/3541140657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8a4e40 0x5630ec8af1c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009fd0 tx=0x7f2a200118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7783a9b86bea3bab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:35.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.873+0000 7f2a2a000640 1 --2- 172.21.15.89:0/1284263495 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2a140a2b80 0x7f2a140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.873+0000 7f2a1ea00640 1 -- 172.21.15.89:0/3541140657 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a20004030 con 0x5630ec8a4e40 2024-09-10T08:41:35.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.873+0000 7f2a1ea00640 1 -- 172.21.15.89:0/3541140657 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a20005500 con 0x5630ec8a4e40 2024-09-10T08:41:35.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.873+0000 7f2a1ea00640 1 -- 172.21.15.89:0/3541140657 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a20004da0 con 0x5630ec8a4e40 2024-09-10T08:41:35.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.873+0000 7f2a2a000640 1 -- 172.21.15.89:0/1284263495 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f2a180ced80 con 0x7f2a2c1034b0 2024-09-10T08:41:35.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.873+0000 7f2a32400640 1 --2- 172.21.15.89:0/1284263495 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2a140a2b80 0x7f2a140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.874+0000 7f2a36eb5740 1 -- 172.21.15.89:0/3541140657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8a4e40 msgr2=0x5630ec8af1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:35.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.874+0000 7f2a36eb5740 1 --2- 172.21.15.89:0/3541140657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8a4e40 0x5630ec8af1c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009fd0 tx=0x7f2a200118d0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.874+0000 7f2a36eb5740 1 -- 172.21.15.89:0/3541140657 shutdown_connections 2024-09-10T08:41:35.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.874+0000 7f2a36eb5740 1 --2- 172.21.15.89:0/3541140657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8af700 0x5630ec8b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.874+0000 7f2a36eb5740 1 --2- 172.21.15.89:0/3541140657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8a4e40 0x5630ec8af1c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.874+0000 7f2a36eb5740 1 --2- 172.21.15.89:0/3541140657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8a4520 0x5630ec8a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.874+0000 7f2a36eb5740 1 -- 172.21.15.89:0/3541140657 >> 172.21.15.89:0/3541140657 conn(0x5630ec8a0ab0 msgr2=0x5630ec8a0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:35.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.874+0000 7f2a32400640 1 --2- 172.21.15.89:0/1284263495 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2a140a2b80 0x7f2a140a5040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009a60 tx=0x7f2a24005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:35.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.874+0000 7f2a36eb5740 1 -- 172.21.15.89:0/3541140657 shutdown_connections 2024-09-10T08:41:35.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.874+0000 7f2a36eb5740 1 -- 172.21.15.89:0/3541140657 wait complete. 2024-09-10T08:41:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.875+0000 7f2a36eb5740 1 -- start start 2024-09-10T08:41:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.875+0000 7f2a36eb5740 1 Processor -- start 2024-09-10T08:41:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.875+0000 7f2a36eb5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8a4520 0x5630ec8ac540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.875+0000 7f2a36eb5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8a4e40 0x5630ec8aca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.875+0000 7f2a36eb5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8af700 0x5630ec8acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.875+0000 7f2a36eb5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5630ec8b22d0 con 0x5630ec8a4e40 2024-09-10T08:41:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.875+0000 7f2a36eb5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5630ec7c5350 con 0x5630ec8af700 2024-09-10T08:41:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.875+0000 7f2a36eb5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5630ec8b2450 con 0x5630ec8a4520 2024-09-10T08:41:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.876+0000 7f2a36eb5740 10 client.0 _ll_register_callbacks cb 0x5630ec7e8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.876+0000 7f2a31000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8a4e40 0x5630ec8aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.876+0000 7f2a31000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8a4e40 0x5630ec8aca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56610/0 (socket says 172.21.15.89:56610) 2024-09-10T08:41:35.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.876+0000 7f2a31000640 1 -- 172.21.15.89:0/1837528736 learned_addr learned my addr 172.21.15.89:0/1837528736 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:35.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.876+0000 7f2a31a00640 1 --2- 172.21.15.89:0/1837528736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8a4520 0x5630ec8ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.876+0000 7f2a32400640 1 --2- 172.21.15.89:0/1837528736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8af700 0x5630ec8acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.876+0000 7f2a31000640 1 -- 172.21.15.89:0/1837528736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8a4520 msgr2=0x5630ec8ac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:35.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.876+0000 7f2a31000640 1 --2- 172.21.15.89:0/1837528736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8a4520 0x5630ec8ac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.876+0000 7f2a31000640 1 -- 172.21.15.89:0/1837528736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8af700 msgr2=0x5630ec8acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:35.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.876+0000 7f2a31000640 1 --2- 172.21.15.89:0/1837528736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8af700 0x5630ec8acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.876+0000 7f2a31000640 1 -- 172.21.15.89:0/1837528736 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630ec5e3960 con 0x5630ec8a4e40 2024-09-10T08:41:35.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.876+0000 7f2a31a00640 1 --2- 172.21.15.89:0/1837528736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8a4520 0x5630ec8ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:35.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.877+0000 7f2a31000640 1 --2- 172.21.15.89:0/1837528736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8a4e40 0x5630ec8aca80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2a1809af60 tx=0x7f2a180975c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75ec403dfb632f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:35.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.877+0000 7f2a13400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:41:35.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.877+0000 7f2a13400640 1 -- 172.21.15.89:0/1837528736 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a180d0070 con 0x5630ec8a4e40 2024-09-10T08:41:35.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.877+0000 7f2a36eb5740 1 -- 172.21.15.89:0/1837528736 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a20005500 con 0x5630ec8a4e40 2024-09-10T08:41:35.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.877+0000 7f2a13400640 1 -- 172.21.15.89:0/1837528736 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f2a180cc560 con 0x5630ec8a4e40 2024-09-10T08:41:35.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.877+0000 7f2a13400640 1 -- 172.21.15.89:0/1837528736 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2a180b4050 con 0x5630ec8a4e40 2024-09-10T08:41:35.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.877+0000 7f2a36eb5740 10 client.15753 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:35.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.877+0000 7f2a36eb5740 1 -- 172.21.15.89:0/1837528736 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5630ec8bde40 con 0x5630ec8a4e40 2024-09-10T08:41:35.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.877+0000 7f2a13400640 1 -- 172.21.15.89:0/1837528736 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a20005500 con 0x5630ec8a4e40 2024-09-10T08:41:35.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.877+0000 7f2a13400640 1 -- 172.21.15.89:0/1837528736 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2a180d3040 con 0x5630ec8a4e40 2024-09-10T08:41:35.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.878+0000 7f2a36eb5740 10 client.15753 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:35.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.878+0000 7f2a36eb5740 10 client.15753 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:41:35.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.878+0000 7f2a36eb5740 20 client.15753 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:35.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.878+0000 7f2a36eb5740 1 --2- 172.21.15.89:0/1837528736 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5630ec8bf250 0x5630ec8c16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.878+0000 7f2a36eb5740 4 client.15753 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:41:35.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.878+0000 7f2a36eb5740 1 -- 172.21.15.89:0/1837528736 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630ec817bc0 con 0x5630ec8bf250 2024-09-10T08:41:35.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.878+0000 7f2a31a00640 1 --2- 172.21.15.89:0/1837528736 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5630ec8bf250 0x5630ec8c16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.879+0000 7f2a31a00640 1 --2- 172.21.15.89:0/1837528736 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5630ec8bf250 0x5630ec8c16d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d034b733291604e server_cookie=ca1e7af805bfd380 in_seq=0 out_seq=0 2024-09-10T08:41:35.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.879+0000 7f2a13400640 10 client.15753 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:35.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.879+0000 7f2a13400640 1 -- 172.21.15.89:0/1837528736 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630ec817bc0 con 0x5630ec8bf250 2024-09-10T08:41:35.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.879+0000 7f2a13400640 10 client.15753 handle_command_reply: tid=0 2024-09-10T08:41:35.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.880+0000 7f2a36eb5740 1 client.15753 shutdown 2024-09-10T08:41:35.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.880+0000 7f2a36eb5740 1 -- 172.21.15.89:0/1837528736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8a4e40 msgr2=0x5630ec8aca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:35.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.880+0000 7f2a36eb5740 1 --2- 172.21.15.89:0/1837528736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8a4e40 0x5630ec8aca80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2a1809af60 tx=0x7f2a180975c0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.880+0000 7f2a36eb5740 1 -- 172.21.15.89:0/1837528736 shutdown_connections 2024-09-10T08:41:35.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.880+0000 7f2a36eb5740 1 --2- 172.21.15.89:0/1837528736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8af700 0x5630ec8acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.880+0000 7f2a36eb5740 1 --2- 172.21.15.89:0/1837528736 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5630ec8bf250 0x5630ec8c16d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.880+0000 7f2a36eb5740 1 --2- 172.21.15.89:0/1837528736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8a4e40 0x5630ec8aca80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.880+0000 7f2a36eb5740 1 --2- 172.21.15.89:0/1837528736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8a4520 0x5630ec8ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.880+0000 7f2a13400640 0 client.15753 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:35.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.880+0000 7f2a36eb5740 1 -- 172.21.15.89:0/1837528736 >> 172.21.15.89:0/1837528736 conn(0x5630ec8a0ab0 msgr2=0x5630ec89fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:35.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.880+0000 7f2a36eb5740 1 -- 172.21.15.89:0/1837528736 shutdown_connections 2024-09-10T08:41:35.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.880+0000 7f2a36eb5740 1 -- 172.21.15.89:0/1837528736 wait complete. 2024-09-10T08:41:35.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.880+0000 7f2a36eb5740 20 client.15753 trim_cache size 0 max 16384 2024-09-10T08:41:35.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.897+0000 7f2a36eb5740 1 Processor -- start 2024-09-10T08:41:35.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.897+0000 7f2a36eb5740 1 -- start start 2024-09-10T08:41:35.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.898+0000 7f2a36eb5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8a4520 0x5630ec89fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.898+0000 7f2a36eb5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8a4e40 0x5630ec8a6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.898+0000 7f2a36eb5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8af700 0x5630ec8c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.898+0000 7f2a36eb5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c1b3770 con 0x5630ec8a4520 2024-09-10T08:41:35.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.898+0000 7f2a36eb5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c1b1040 con 0x5630ec8a4e40 2024-09-10T08:41:35.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.898+0000 7f2a36eb5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c1aee90 con 0x5630ec8af700 2024-09-10T08:41:35.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.898+0000 7f2a31a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8a4520 0x5630ec89fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.898+0000 7f2a31a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8a4520 0x5630ec89fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56612/0 (socket says 172.21.15.89:56612) 2024-09-10T08:41:35.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.898+0000 7f2a31a00640 1 -- 172.21.15.89:0/159515555 learned_addr learned my addr 172.21.15.89:0/159515555 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:35.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.898+0000 7f2a31000640 1 --2- 172.21.15.89:0/159515555 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8a4e40 0x5630ec8a6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.898+0000 7f2a32400640 1 --2- 172.21.15.89:0/159515555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8af700 0x5630ec8c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.898+0000 7f2a31a00640 1 -- 172.21.15.89:0/159515555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8af700 msgr2=0x5630ec8c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:35.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.898+0000 7f2a31a00640 1 --2- 172.21.15.89:0/159515555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8af700 0x5630ec8c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.898+0000 7f2a31a00640 1 -- 172.21.15.89:0/159515555 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8a4e40 msgr2=0x5630ec8a6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:35.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.898+0000 7f2a31a00640 1 --2- 172.21.15.89:0/159515555 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8a4e40 0x5630ec8a6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.898+0000 7f2a31a00640 1 -- 172.21.15.89:0/159515555 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630ec8aee10 con 0x5630ec8a4520 2024-09-10T08:41:35.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.898+0000 7f2a31000640 1 --2- 172.21.15.89:0/159515555 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8a4e40 0x5630ec8a6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:41:35.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.899+0000 7f2a31a00640 1 --2- 172.21.15.89:0/159515555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8a4520 0x5630ec89fb40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009ef0 tx=0x7f2a200118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7491c5de4c2fce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:35.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.899+0000 7f2a1e000640 1 -- 172.21.15.89:0/159515555 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a20032070 con 0x5630ec8a4520 2024-09-10T08:41:35.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.899+0000 7f2a1e000640 1 -- 172.21.15.89:0/159515555 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a200052d0 con 0x5630ec8a4520 2024-09-10T08:41:35.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.899+0000 7f2a1e000640 1 -- 172.21.15.89:0/159515555 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2000f800 con 0x5630ec8a4520 2024-09-10T08:41:35.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.899+0000 7f2a36eb5740 1 -- 172.21.15.89:0/159515555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8a4520 msgr2=0x5630ec89fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:35.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.899+0000 7f2a36eb5740 1 --2- 172.21.15.89:0/159515555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8a4520 0x5630ec89fb40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009ef0 tx=0x7f2a200118d0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.899+0000 7f2a36eb5740 1 -- 172.21.15.89:0/159515555 shutdown_connections 2024-09-10T08:41:35.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.899+0000 7f2a36eb5740 1 --2- 172.21.15.89:0/159515555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8af700 0x5630ec8c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.899+0000 7f2a36eb5740 1 --2- 172.21.15.89:0/159515555 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8a4e40 0x5630ec8a6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.899+0000 7f2a36eb5740 1 --2- 172.21.15.89:0/159515555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8a4520 0x5630ec89fb40 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.899+0000 7f2a36eb5740 1 -- 172.21.15.89:0/159515555 >> 172.21.15.89:0/159515555 conn(0x5630ec8a0ab0 msgr2=0x5630ec8ae4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:35.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.900+0000 7f2a36eb5740 1 -- 172.21.15.89:0/159515555 shutdown_connections 2024-09-10T08:41:35.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.900+0000 7f2a36eb5740 1 -- 172.21.15.89:0/159515555 wait complete. 2024-09-10T08:41:35.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.900+0000 7f2a36eb5740 1 -- start start 2024-09-10T08:41:35.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.900+0000 7f2a36eb5740 1 Processor -- start 2024-09-10T08:41:35.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.901+0000 7f2a36eb5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8a4520 0x5630ec8b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.901+0000 7f2a36eb5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8a4e40 0x5630ec8b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.901+0000 7f2a36eb5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8af700 0x5630ec928f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.901+0000 7f2a36eb5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c1af500 con 0x5630ec8af700 2024-09-10T08:41:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.901+0000 7f2a36eb5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c1b0970 con 0x5630ec8a4520 2024-09-10T08:41:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.901+0000 7f2a36eb5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c1afbd0 con 0x5630ec8a4e40 2024-09-10T08:41:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.901+0000 7f2a36eb5740 10 client.0 _ll_register_callbacks cb 0x7f2a180d24b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.901+0000 7f2a32400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8af700 0x5630ec928f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.901+0000 7f2a31a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8a4520 0x5630ec8b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.901+0000 7f2a32400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8af700 0x5630ec928f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56622/0 (socket says 172.21.15.89:56622) 2024-09-10T08:41:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.901+0000 7f2a32400640 1 -- 172.21.15.89:0/1235424209 learned_addr learned my addr 172.21.15.89:0/1235424209 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.901+0000 7f2a31000640 1 --2- 172.21.15.89:0/1235424209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8a4e40 0x5630ec8b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.901+0000 7f2a32400640 1 -- 172.21.15.89:0/1235424209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8a4e40 msgr2=0x5630ec8b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.901+0000 7f2a32400640 1 --2- 172.21.15.89:0/1235424209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8a4e40 0x5630ec8b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.901+0000 7f2a32400640 1 -- 172.21.15.89:0/1235424209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8a4520 msgr2=0x5630ec8b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.901+0000 7f2a32400640 1 --2- 172.21.15.89:0/1235424209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8a4520 0x5630ec8b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.901+0000 7f2a32400640 1 -- 172.21.15.89:0/1235424209 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630ec929460 con 0x5630ec8af700 2024-09-10T08:41:35.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.902+0000 7f2a31a00640 1 --2- 172.21.15.89:0/1235424209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8a4520 0x5630ec8b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:35.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.902+0000 7f2a32400640 1 --2- 172.21.15.89:0/1235424209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8af700 0x5630ec928f20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2a2402ae70 tx=0x7f2a240095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da2c3817b1980a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:35.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.902+0000 7f2a13400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:41:35.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.902+0000 7f2a13400640 1 -- 172.21.15.89:0/1235424209 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24034040 con 0x5630ec8af700 2024-09-10T08:41:35.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.902+0000 7f2a13400640 1 -- 172.21.15.89:0/1235424209 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f2a24053670 con 0x5630ec8af700 2024-09-10T08:41:35.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.902+0000 7f2a36eb5740 1 -- 172.21.15.89:0/1235424209 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a200052d0 con 0x5630ec8af700 2024-09-10T08:41:35.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.902+0000 7f2a13400640 1 -- 172.21.15.89:0/1235424209 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2a24061050 con 0x5630ec8af700 2024-09-10T08:41:35.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.902+0000 7f2a36eb5740 10 client.15759 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:35.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.902+0000 7f2a36eb5740 1 -- 172.21.15.89:0/1235424209 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5630ec92a6e0 con 0x5630ec8af700 2024-09-10T08:41:35.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.903+0000 7f2a13400640 1 -- 172.21.15.89:0/1235424209 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a200052d0 con 0x5630ec8af700 2024-09-10T08:41:35.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.903+0000 7f2a13400640 1 -- 172.21.15.89:0/1235424209 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2a2403aaf0 con 0x5630ec8af700 2024-09-10T08:41:35.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.903+0000 7f2a36eb5740 10 client.15759 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:35.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.903+0000 7f2a36eb5740 10 client.15759 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:41:35.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.903+0000 7f2a36eb5740 20 client.15759 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:35.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.903+0000 7f2a36eb5740 1 --2- 172.21.15.89:0/1235424209 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5630ec92ae80 0x5630ec92d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:35.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.903+0000 7f2a36eb5740 4 client.15759 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ada3e43a", "release": true, "await": true}] 2024-09-10T08:41:35.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.903+0000 7f2a36eb5740 1 -- 172.21.15.89:0/1235424209 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ada3e43a", "release": true, "await": true}) -- 0x5630ec8120d0 con 0x5630ec92ae80 2024-09-10T08:41:35.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.903+0000 7f2a31a00640 1 --2- 172.21.15.89:0/1235424209 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5630ec92ae80 0x5630ec92d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:35.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.904+0000 7f2a31a00640 1 --2- 172.21.15.89:0/1235424209 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5630ec92ae80 0x5630ec92d2a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4851197ce77da5e9 server_cookie=f20c21302c80f24a in_seq=0 out_seq=0 2024-09-10T08:41:35.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.904+0000 7f2a13400640 10 client.15759 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:35.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.907+0000 7f2a13400640 1 -- 172.21.15.89:0/1235424209 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5630ec8120d0 con 0x5630ec92ae80 2024-09-10T08:41:35.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.907+0000 7f2a13400640 10 client.15759 handle_command_reply: tid=0 2024-09-10T08:41:35.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.907+0000 7f2a36eb5740 1 client.15759 shutdown 2024-09-10T08:41:35.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.907+0000 7f2a36eb5740 1 -- 172.21.15.89:0/1235424209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8af700 msgr2=0x5630ec928f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:35.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.907+0000 7f2a36eb5740 1 --2- 172.21.15.89:0/1235424209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8af700 0x5630ec928f20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2a2402ae70 tx=0x7f2a240095c0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.907+0000 7f2a36eb5740 1 -- 172.21.15.89:0/1235424209 shutdown_connections 2024-09-10T08:41:35.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.907+0000 7f2a36eb5740 1 --2- 172.21.15.89:0/1235424209 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5630ec92ae80 0x5630ec92d2a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.907+0000 7f2a36eb5740 1 --2- 172.21.15.89:0/1235424209 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5630ec8af700 0x5630ec928f20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.907+0000 7f2a36eb5740 1 --2- 172.21.15.89:0/1235424209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630ec8a4e40 0x5630ec8b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.907+0000 7f2a36eb5740 1 --2- 172.21.15.89:0/1235424209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630ec8a4520 0x5630ec8b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.907+0000 7f2a36eb5740 1 -- 172.21.15.89:0/1235424209 >> 172.21.15.89:0/1235424209 conn(0x5630ec8a0ab0 msgr2=0x5630ec8ac500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:35.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.907+0000 7f2a13400640 0 client.15759 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:35.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.908+0000 7f2a36eb5740 1 -- 172.21.15.89:0/1235424209 shutdown_connections 2024-09-10T08:41:35.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.908+0000 7f2a36eb5740 1 -- 172.21.15.89:0/1235424209 wait complete. 2024-09-10T08:41:35.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.908+0000 7f2a36eb5740 20 client.15759 trim_cache size 0 max 16384 2024-09-10T08:41:35.910 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:41:35.910 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 26, 2024-09-10T08:41:35.910 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24554, 2024-09-10T08:41:35.910 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 8, 2024-09-10T08:41:35.910 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": { 2024-09-10T08:41:35.910 INFO:teuthology.orchestra.run.smithi089.stdout: "ada3e43a": { 2024-09-10T08:41:35.911 INFO:teuthology.orchestra.run.smithi089.stdout: "version": 8, 2024-09-10T08:41:35.911 INFO:teuthology.orchestra.run.smithi089.stdout: "age_ref": 0.0, 2024-09-10T08:41:35.911 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:41:35.911 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RELEASED", 2024-09-10T08:41:35.911 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:41:35.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:41:35.911 INFO:teuthology.orchestra.run.smithi089.stdout: "timeout": 90.0, 2024-09-10T08:41:35.911 INFO:teuthology.orchestra.run.smithi089.stdout: "expiration": 150.0, 2024-09-10T08:41:35.911 INFO:teuthology.orchestra.run.smithi089.stdout: "members": { 2024-09-10T08:41:35.911 INFO:teuthology.orchestra.run.smithi089.stdout: "file:/": { 2024-09-10T08:41:35.911 INFO:teuthology.orchestra.run.smithi089.stdout: "excluded": false, 2024-09-10T08:41:35.911 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:41:35.911 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RELEASED", 2024-09-10T08:41:35.911 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:41:35.912 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:41:35.912 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:41:35.912 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:41:35.912 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:41:35.912 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:41:35.912 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:41:35.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.908+0000 7f2a32e00640 1 -- 172.21.15.89:0/1284263495 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2a140a2b80 msgr2=0x7f2a140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:35.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.908+0000 7f2a32e00640 1 --2- 172.21.15.89:0/1284263495 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2a140a2b80 0x7f2a140a5040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009a60 tx=0x7f2a24005d80 comp rx=0 tx=0).stop 2024-09-10T08:41:35.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.908+0000 7f2a32e00640 1 -- 172.21.15.89:0/1284263495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a2c1034b0 msgr2=0x7f2a2c19a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:35.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.908+0000 7f2a32e00640 1 --2- 172.21.15.89:0/1284263495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a2c1034b0 0x7f2a2c19a9a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800d750 tx=0x7f2a1800dc10 comp rx=0 tx=0).stop 2024-09-10T08:41:35.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.909+0000 7f2a32e00640 1 -- 172.21.15.89:0/1284263495 shutdown_connections 2024-09-10T08:41:35.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.909+0000 7f2a32e00640 1 --2- 172.21.15.89:0/1284263495 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2a140a2b80 0x7f2a140a5040 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.909+0000 7f2a32e00640 1 --2- 172.21.15.89:0/1284263495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a2c10bfc0 0x7f2a2c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.909+0000 7f2a32e00640 1 --2- 172.21.15.89:0/1284263495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a2c1034b0 0x7f2a2c19a9a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.909+0000 7f2a32e00640 1 --2- 172.21.15.89:0/1284263495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a2c102b90 0x7f2a2c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:35.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.909+0000 7f2a32e00640 1 -- 172.21.15.89:0/1284263495 >> 172.21.15.89:0/1284263495 conn(0x7f2a2c0fc610 msgr2=0x7f2a2c10d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:35.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.909+0000 7f2a32e00640 1 -- 172.21.15.89:0/1284263495 shutdown_connections 2024-09-10T08:41:35.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:35.909+0000 7f2a32e00640 1 -- 172.21.15.89:0/1284263495 wait complete. 2024-09-10T08:41:35.922 INFO:tasks.workunit.client.0.smithi089.stdout:getting blogbench 2024-09-10T08:41:35.922 INFO:tasks.workunit.client.0.smithi089.stderr:+ echo 'getting blogbench' 2024-09-10T08:41:35.922 INFO:tasks.workunit.client.0.smithi089.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-10T08:41:36.006 INFO:tasks.workunit.client.0.smithi089.stderr:--2024-09-10 08:41:36-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-10T08:41:36.017 INFO:tasks.workunit.client.0.smithi089.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-10T08:41:36.038 INFO:tasks.workunit.client.0.smithi089.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-10T08:41:36.058 INFO:tasks.workunit.client.0.smithi089.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-10T08:41:36.059 INFO:tasks.workunit.client.0.smithi089.stderr:Length: 103355 (101K) [application/octet-stream] 2024-09-10T08:41:36.060 INFO:tasks.workunit.client.0.smithi089.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-09-10T08:41:36.065 INFO:tasks.workunit.client.0.smithi089.stderr: 2024-09-10T08:41:36.100 INFO:tasks.workunit.client.0.smithi089.stderr: 0K .......... .......... .......... .......... .......... 49% 1.26M 0s 2024-09-10T08:41:36.121 INFO:tasks.workunit.client.0.smithi089.stderr: 50K .......... .......... .......... .......... .......... 99% 2.37M 0s 2024-09-10T08:41:36.121 INFO:tasks.workunit.client.0.smithi089.stderr: 100K 100% 1.74T=0.06s 2024-09-10T08:41:36.121 INFO:tasks.workunit.client.0.smithi089.stderr: 2024-09-10T08:41:36.121 INFO:tasks.workunit.client.0.smithi089.stderr:2024-09-10 08:41:36 (1.66 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-09-10T08:41:36.121 INFO:tasks.workunit.client.0.smithi089.stderr: 2024-09-10T08:41:36.121 INFO:tasks.workunit.client.0.smithi089.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-09-10T08:41:36.299 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/ 2024-09-10T08:41:36.301 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/m4/ 2024-09-10T08:41:36.303 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-09-10T08:41:36.305 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/man/ 2024-09-10T08:41:36.307 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/man/blogbench.8 2024-09-10T08:41:36.308 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/man/Makefile.am 2024-09-10T08:41:36.310 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/man/Makefile.in 2024-09-10T08:41:36.311 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/src/ 2024-09-10T08:41:36.313 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/src/mysnprintf.c 2024-09-10T08:41:36.314 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/src/mysnprintf.h 2024-09-10T08:41:36.316 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/src/blogbench.c 2024-09-10T08:41:36.317 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/src/blogbench.h 2024-09-10T08:41:36.319 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/src/fakesnprintf.c 2024-09-10T08:41:36.320 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/src/fakesnprintf.h 2024-09-10T08:41:36.322 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/src/blogbench_p.h 2024-09-10T08:41:36.324 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-09-10T08:41:36.325 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-09-10T08:41:36.327 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/src/Makefile.am 2024-09-10T08:41:36.329 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/src/Makefile.in 2024-09-10T08:41:36.330 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/src/process.c 2024-09-10T08:41:36.332 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/src/globals.h 2024-09-10T08:41:36.333 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/src/commenter.c 2024-09-10T08:41:36.335 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/src/rewriter.c 2024-09-10T08:41:36.336 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/src/reader.c 2024-09-10T08:41:36.341 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/src/writer.c 2024-09-10T08:41:36.341 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/src/helpers.c 2024-09-10T08:41:36.341 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/NEWS 2024-09-10T08:41:36.343 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/compile 2024-09-10T08:41:36.344 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/depcomp 2024-09-10T08:41:36.345 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/aclocal.m4 2024-09-10T08:41:36.347 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/README 2024-09-10T08:41:36.349 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/configure 2024-09-10T08:41:36.354 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/configure.ac 2024-09-10T08:41:36.355 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/config.guess 2024-09-10T08:41:36.357 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/install-sh 2024-09-10T08:41:36.359 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/config.sub 2024-09-10T08:41:36.360 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/missing 2024-09-10T08:41:36.362 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/mkinstalldirs 2024-09-10T08:41:36.363 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/Makefile.am 2024-09-10T08:41:36.364 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/Makefile.in 2024-09-10T08:41:36.366 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/config.h.in 2024-09-10T08:41:36.367 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/AUTHORS 2024-09-10T08:41:36.369 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/INSTALL 2024-09-10T08:41:36.371 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/ChangeLog 2024-09-10T08:41:36.372 INFO:tasks.workunit.client.0.smithi089.stdout:blogbench-1.0/COPYING 2024-09-10T08:41:36.374 INFO:tasks.workunit.client.0.smithi089.stderr:+ cd blogbench-1.0/ 2024-09-10T08:41:36.374 INFO:tasks.workunit.client.0.smithi089.stdout:making blogbench 2024-09-10T08:41:36.375 INFO:tasks.workunit.client.0.smithi089.stderr:+ echo 'making blogbench' 2024-09-10T08:41:36.375 INFO:tasks.workunit.client.0.smithi089.stderr:+ ./configure 2024-09-10T08:41:36.603 INFO:tasks.workunit.client.0.smithi089.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-10T08:41:36.916 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ada3e43a', total seconds elapsed: 34.1 2024-09-10T08:41:37.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:36 smithi111 ceph-mon[35220]: pgmap v443: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-10T08:41:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:36 smithi089 ceph-mon[30757]: pgmap v443: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-10T08:41:37.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:36 smithi149 ceph-mon[36167]: pgmap v443: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 8 op/s 2024-09-10T08:41:37.615 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether build environment is sane... yes 2024-09-10T08:41:37.630 INFO:tasks.workunit.client.0.smithi089.stdout:checking for gawk... gawk 2024-09-10T08:41:37.653 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether make sets $(MAKE)... yes 2024-09-10T08:41:37.670 INFO:tasks.workunit.client.0.smithi089.stdout:checking for gcc... gcc 2024-09-10T08:41:37.757 INFO:tasks.workunit.client.0.smithi089.stdout:checking for C compiler default output file name... a.out 2024-09-10T08:41:37.761 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether the C compiler works... yes 2024-09-10T08:41:37.765 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether we are cross compiling... no 2024-09-10T08:41:37.824 INFO:tasks.workunit.client.0.smithi089.stdout:checking for suffix of executables... 2024-09-10T08:41:37.883 INFO:tasks.workunit.client.0.smithi089.stdout:checking for suffix of object files... o 2024-09-10T08:41:37.942 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether we are using the GNU C compiler... yes 2024-09-10T08:41:38.002 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether gcc accepts -g... yes 2024-09-10T08:41:38.117 INFO:tasks.workunit.client.0.smithi089.stdout:checking for gcc option to accept ANSI C... none needed 2024-09-10T08:41:38.183 INFO:tasks.workunit.client.0.smithi089.stdout:checking for style of include used by make... GNU 2024-09-10T08:41:38.309 INFO:tasks.workunit.client.0.smithi089.stdout:checking dependency style of gcc... gcc3 2024-09-10T08:41:38.309 INFO:tasks.workunit.client.0.smithi089.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-10T08:41:38.430 INFO:tasks.workunit.client.0.smithi089.stdout:checking how to run the C preprocessor... gcc -E 2024-09-10T08:41:38.506 INFO:tasks.workunit.client.0.smithi089.stdout:checking for egrep... grep -E 2024-09-10T08:41:38.526 INFO:tasks.workunit.client.0.smithi089.stdout:checking for AIX... no 2024-09-10T08:41:38.627 INFO:tasks.workunit.client.0.smithi089.stdout:checking for library containing strerror... none required 2024-09-10T08:41:38.872 INFO:tasks.workunit.client.0.smithi089.stdout:checking for ANSI C header files... yes 2024-09-10T08:41:38.942 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-10T08:41:39.022 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sys/types.h... yes 2024-09-10T08:41:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:38 smithi111 ceph-mon[35220]: pgmap v444: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T08:41:39.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:38 smithi089 ceph-mon[30757]: pgmap v444: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T08:41:39.106 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sys/stat.h... yes 2024-09-10T08:41:39.195 INFO:tasks.workunit.client.0.smithi089.stdout:checking for stdlib.h... yes 2024-09-10T08:41:39.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:38 smithi149 ceph-mon[36167]: pgmap v444: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T08:41:39.285 INFO:tasks.workunit.client.0.smithi089.stdout:checking for string.h... yes 2024-09-10T08:41:39.373 INFO:tasks.workunit.client.0.smithi089.stdout:checking for memory.h... yes 2024-09-10T08:41:39.461 INFO:tasks.workunit.client.0.smithi089.stdout:checking for strings.h... yes 2024-09-10T08:41:39.548 INFO:tasks.workunit.client.0.smithi089.stdout:checking for inttypes.h... yes 2024-09-10T08:41:39.636 INFO:tasks.workunit.client.0.smithi089.stdout:checking for stdint.h... yes 2024-09-10T08:41:39.727 INFO:tasks.workunit.client.0.smithi089.stdout:checking for unistd.h... yes 2024-09-10T08:41:39.747 INFO:tasks.workunit.client.0.smithi089.stdout:checking for memory.h... (cached) yes 2024-09-10T08:41:39.761 INFO:tasks.workunit.client.0.smithi089.stdout:checking for strings.h... (cached) yes 2024-09-10T08:41:39.774 INFO:tasks.workunit.client.0.smithi089.stdout:checking for unistd.h... (cached) yes 2024-09-10T08:41:39.859 INFO:tasks.workunit.client.0.smithi089.stdout:checking locale.h usability... yes 2024-09-10T08:41:39.895 INFO:tasks.workunit.client.0.smithi089.stdout:checking locale.h presence... yes 2024-09-10T08:41:39.897 INFO:tasks.workunit.client.0.smithi089.stdout:checking for locale.h... yes 2024-09-10T08:41:39.920 INFO:tasks.workunit.client.0.smithi089.stdout:checking for stdlib.h... (cached) yes 2024-09-10T08:41:39.934 INFO:tasks.workunit.client.0.smithi089.stdout:checking for string.h... (cached) yes 2024-09-10T08:41:40.045 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/socket.h usability... yes 2024-09-10T08:41:40.086 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/socket.h presence... yes 2024-09-10T08:41:40.087 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sys/socket.h... yes 2024-09-10T08:41:40.177 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/time.h usability... yes 2024-09-10T08:41:40.215 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/time.h presence... yes 2024-09-10T08:41:40.216 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sys/time.h... yes 2024-09-10T08:41:40.317 INFO:tasks.workunit.client.0.smithi089.stdout:checking fcntl.h usability... yes 2024-09-10T08:41:40.353 INFO:tasks.workunit.client.0.smithi089.stdout:checking fcntl.h presence... yes 2024-09-10T08:41:40.355 INFO:tasks.workunit.client.0.smithi089.stdout:checking for fcntl.h... yes 2024-09-10T08:41:40.454 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/fcntl.h usability... yes 2024-09-10T08:41:40.491 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/fcntl.h presence... yes 2024-09-10T08:41:40.493 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sys/fcntl.h... yes 2024-09-10T08:41:40.648 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/param.h usability... yes 2024-09-10T08:41:40.690 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/param.h presence... yes 2024-09-10T08:41:40.692 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sys/param.h... yes 2024-09-10T08:41:40.769 INFO:tasks.workunit.client.0.smithi089.stdout:checking ioctl.h usability... no 2024-09-10T08:41:40.806 INFO:tasks.workunit.client.0.smithi089.stdout:checking ioctl.h presence... no 2024-09-10T08:41:40.808 INFO:tasks.workunit.client.0.smithi089.stdout:checking for ioctl.h... no 2024-09-10T08:41:40.891 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/ioctl.h usability... yes 2024-09-10T08:41:40.929 INFO:tasks.workunit.client.0.smithi089.stdout:checking sys/ioctl.h presence... yes 2024-09-10T08:41:40.930 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sys/ioctl.h... yes 2024-09-10T08:41:41.014 INFO:tasks.workunit.client.0.smithi089.stdout:checking return type of signal handlers... void 2024-09-10T08:41:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:40 smithi111 ceph-mon[35220]: pgmap v445: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-10T08:41:41.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:40 smithi089 ceph-mon[30757]: pgmap v445: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-10T08:41:41.091 INFO:tasks.workunit.client.0.smithi089.stdout:checking for size_t... yes 2024-09-10T08:41:41.159 INFO:tasks.workunit.client.0.smithi089.stdout:checking for dev_t... yes 2024-09-10T08:41:41.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:40 smithi149 ceph-mon[36167]: pgmap v445: 129 pgs: 129 active+clean; 678 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-10T08:41:41.236 INFO:tasks.workunit.client.0.smithi089.stdout:checking for short... yes 2024-09-10T08:41:41.348 INFO:tasks.workunit.client.0.smithi089.stdout:checking size of short... 2 2024-09-10T08:41:41.462 INFO:tasks.workunit.client.0.smithi089.stdout:checking for int... yes 2024-09-10T08:41:41.574 INFO:tasks.workunit.client.0.smithi089.stdout:checking size of int... 4 2024-09-10T08:41:41.650 INFO:tasks.workunit.client.0.smithi089.stdout:checking for long... yes 2024-09-10T08:41:41.756 INFO:tasks.workunit.client.0.smithi089.stdout:checking size of long... 8 2024-09-10T08:41:41.835 INFO:tasks.workunit.client.0.smithi089.stdout:checking for long long... yes 2024-09-10T08:41:41.939 INFO:tasks.workunit.client.0.smithi089.stdout:checking size of long long... 8 2024-09-10T08:41:42.013 INFO:tasks.workunit.client.0.smithi089.stdout:checking for ssize_t... yes 2024-09-10T08:41:42.084 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sig_atomic_t... yes 2024-09-10T08:41:42.162 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether gcc needs -traditional... no 2024-09-10T08:41:42.225 INFO:tasks.workunit.client.0.smithi089.stdout:checking for an ANSI C-conforming const... yes 2024-09-10T08:41:42.274 INFO:tasks.workunit.client.0.smithi089.stdout:checking for inline... inline 2024-09-10T08:41:42.408 INFO:tasks.workunit.client.0.smithi089.stdout:checking for working alloca.h... yes 2024-09-10T08:41:42.492 INFO:tasks.workunit.client.0.smithi089.stdout:checking for alloca... yes 2024-09-10T08:41:42.599 INFO:tasks.workunit.client.0.smithi089.stdout:checking for working memcmp... yes 2024-09-10T08:41:42.693 INFO:tasks.workunit.client.0.smithi089.stdout:checking for strftime... yes 2024-09-10T08:41:42.809 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-09-10T08:41:42.907 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether stat accepts an empty string... no 2024-09-10T08:41:43.002 INFO:tasks.workunit.client.0.smithi089.stdout:checking for vprintf... yes 2024-09-10T08:41:43.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:42 smithi111 ceph-mon[35220]: pgmap v446: 129 pgs: 129 active+clean; 1.5 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 77 KiB/s wr, 12 op/s 2024-09-10T08:41:43.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:42 smithi089 ceph-mon[30757]: pgmap v446: 129 pgs: 129 active+clean; 1.5 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 77 KiB/s wr, 12 op/s 2024-09-10T08:41:43.096 INFO:tasks.workunit.client.0.smithi089.stdout:checking for _doprnt... no 2024-09-10T08:41:43.147 INFO:tasks.workunit.client.0.smithi089.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-10T08:41:43.176 INFO:tasks.workunit.client.0.smithi089.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-10T08:41:43.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:42 smithi149 ceph-mon[36167]: pgmap v446: 129 pgs: 129 active+clean; 1.5 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 77 KiB/s wr, 12 op/s 2024-09-10T08:41:43.272 INFO:tasks.workunit.client.0.smithi089.stdout:checking for the pthreads library -lpthreads... no 2024-09-10T08:41:43.362 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether pthreads work without any flags... yes 2024-09-10T08:41:43.446 INFO:tasks.workunit.client.0.smithi089.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-09-10T08:41:43.446 INFO:tasks.workunit.client.0.smithi089.stdout:checking if more special flags are required for pthreads... no 2024-09-10T08:41:43.446 INFO:tasks.workunit.client.0.smithi089.stdout:checking for cc_r... gcc 2024-09-10T08:41:43.559 INFO:tasks.workunit.client.0.smithi089.stdout:checking for setrlimit... yes 2024-09-10T08:41:43.663 INFO:tasks.workunit.client.0.smithi089.stdout:checking for setproctitle... no 2024-09-10T08:41:43.761 INFO:tasks.workunit.client.0.smithi089.stdout:checking for getopt_long... yes 2024-09-10T08:41:43.867 INFO:tasks.workunit.client.0.smithi089.stdout:checking for strtoull... yes 2024-09-10T08:41:43.972 INFO:tasks.workunit.client.0.smithi089.stdout:checking for strtoq... yes 2024-09-10T08:41:44.080 INFO:tasks.workunit.client.0.smithi089.stdout:checking for snprintf... yes 2024-09-10T08:41:44.189 INFO:tasks.workunit.client.0.smithi089.stdout:checking for vsnprintf... yes 2024-09-10T08:41:44.295 INFO:tasks.workunit.client.0.smithi089.stdout:checking for vfprintf... yes 2024-09-10T08:41:44.401 INFO:tasks.workunit.client.0.smithi089.stdout:checking for gethostname... yes 2024-09-10T08:41:44.506 INFO:tasks.workunit.client.0.smithi089.stdout:checking for setlocale... yes 2024-09-10T08:41:44.611 INFO:tasks.workunit.client.0.smithi089.stdout:checking for sync... yes 2024-09-10T08:41:44.718 INFO:tasks.workunit.client.0.smithi089.stdout:checking whether snprintf is C99 conformant... done 2024-09-10T08:41:44.739 INFO:tasks.workunit.client.0.smithi089.stdout:configure: creating ./config.status 2024-09-10T08:41:45.037 INFO:tasks.workunit.client.0.smithi089.stdout:config.status: creating Makefile 2024-09-10T08:41:45.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:44 smithi111 ceph-mon[35220]: pgmap v447: 129 pgs: 129 active+clean; 1.6 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 83 KiB/s wr, 11 op/s 2024-09-10T08:41:45.052 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:44 smithi089 ceph-mon[30757]: pgmap v447: 129 pgs: 129 active+clean; 1.6 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 83 KiB/s wr, 11 op/s 2024-09-10T08:41:45.076 INFO:tasks.workunit.client.0.smithi089.stdout:config.status: creating src/Makefile 2024-09-10T08:41:45.116 INFO:tasks.workunit.client.0.smithi089.stdout:config.status: creating man/Makefile 2024-09-10T08:41:45.137 INFO:tasks.workunit.client.0.smithi089.stdout:config.status: creating config.h 2024-09-10T08:41:45.220 INFO:tasks.workunit.client.0.smithi089.stdout:config.status: executing depfiles commands 2024-09-10T08:41:45.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:44 smithi149 ceph-mon[36167]: pgmap v447: 129 pgs: 129 active+clean; 1.6 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 83 KiB/s wr, 11 op/s 2024-09-10T08:41:45.509 INFO:tasks.workunit.client.0.smithi089.stderr:+ make 2024-09-10T08:41:45.520 INFO:tasks.workunit.client.0.smithi089.stdout:make all-recursive 2024-09-10T08:41:45.524 INFO:tasks.workunit.client.0.smithi089.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-10T08:41:45.529 INFO:tasks.workunit.client.0.smithi089.stdout:Making all in src 2024-09-10T08:41:45.537 INFO:tasks.workunit.client.0.smithi089.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-10T08:41:45.537 INFO:tasks.workunit.client.0.smithi089.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-09-10T08:41:45.537 INFO:tasks.workunit.client.0.smithi089.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-09-10T08:41:45.993 INFO:tasks.workunit.client.0.smithi089.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-09-10T08:41:45.993 INFO:tasks.workunit.client.0.smithi089.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-09-10T08:41:46.120 INFO:tasks.workunit.client.0.smithi089.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-09-10T08:41:46.120 INFO:tasks.workunit.client.0.smithi089.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-09-10T08:41:46.199 INFO:tasks.workunit.client.0.smithi089.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-09-10T08:41:46.199 INFO:tasks.workunit.client.0.smithi089.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-09-10T08:41:46.275 INFO:tasks.workunit.client.0.smithi089.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-09-10T08:41:46.275 INFO:tasks.workunit.client.0.smithi089.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-09-10T08:41:46.350 INFO:tasks.workunit.client.0.smithi089.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-09-10T08:41:46.350 INFO:tasks.workunit.client.0.smithi089.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-09-10T08:41:46.422 INFO:tasks.workunit.client.0.smithi089.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-09-10T08:41:46.423 INFO:tasks.workunit.client.0.smithi089.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-09-10T08:41:46.523 INFO:tasks.workunit.client.0.smithi089.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-09-10T08:41:46.523 INFO:tasks.workunit.client.0.smithi089.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-09-10T08:41:46.584 INFO:tasks.workunit.client.0.smithi089.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-09-10T08:41:46.584 INFO:tasks.workunit.client.0.smithi089.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-09-10T08:41:46.638 INFO:tasks.workunit.client.0.smithi089.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-09-10T08:41:46.639 INFO:tasks.workunit.client.0.smithi089.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-09-10T08:41:46.699 INFO:tasks.workunit.client.0.smithi089.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-09-10T08:41:46.730 INFO:tasks.workunit.client.0.smithi089.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-10T08:41:46.731 INFO:tasks.workunit.client.0.smithi089.stdout:Making all in man 2024-09-10T08:41:46.755 INFO:tasks.workunit.client.0.smithi089.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-10T08:41:46.755 INFO:tasks.workunit.client.0.smithi089.stdout:make[2]: Nothing to be done for 'all'. 2024-09-10T08:41:46.756 INFO:tasks.workunit.client.0.smithi089.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-10T08:41:46.758 INFO:tasks.workunit.client.0.smithi089.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-10T08:41:46.760 INFO:tasks.workunit.client.0.smithi089.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-10T08:41:46.761 INFO:tasks.workunit.client.0.smithi089.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-10T08:41:46.762 INFO:tasks.workunit.client.0.smithi089.stderr:+ cd src 2024-09-10T08:41:46.762 INFO:tasks.workunit.client.0.smithi089.stderr:+ mkdir blogtest_in 2024-09-10T08:41:46.773 INFO:tasks.workunit.client.0.smithi089.stdout:running blogbench 2024-09-10T08:41:46.774 INFO:tasks.workunit.client.0.smithi089.stderr:+ echo 'running blogbench' 2024-09-10T08:41:46.774 INFO:tasks.workunit.client.0.smithi089.stderr:+ ./blogbench -d blogtest_in 2024-09-10T08:41:46.774 INFO:tasks.workunit.client.0.smithi089.stdout: 2024-09-10T08:41:46.775 INFO:tasks.workunit.client.0.smithi089.stdout:Frequency = 10 secs 2024-09-10T08:41:46.775 INFO:tasks.workunit.client.0.smithi089.stdout:Scratch dir = [blogtest_in] 2024-09-10T08:41:47.013 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:46 smithi089 ceph-mon[30757]: pgmap v448: 129 pgs: 129 active+clean; 3.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 230 KiB/s wr, 24 op/s 2024-09-10T08:41:47.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:46 smithi111 ceph-mon[35220]: pgmap v448: 129 pgs: 129 active+clean; 3.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 230 KiB/s wr, 24 op/s 2024-09-10T08:41:47.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:46 smithi149 ceph-mon[36167]: pgmap v448: 129 pgs: 129 active+clean; 3.2 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 230 KiB/s wr, 24 op/s 2024-09-10T08:41:47.924 INFO:tasks.workunit.client.0.smithi089.stdout:Spawning 3 writers... 2024-09-10T08:41:47.924 INFO:tasks.workunit.client.0.smithi089.stdout:Spawning 1 rewriters... 2024-09-10T08:41:47.924 INFO:tasks.workunit.client.0.smithi089.stdout:Spawning 5 commenters... 2024-09-10T08:41:47.924 INFO:tasks.workunit.client.0.smithi089.stdout:Spawning 100 readers... 2024-09-10T08:41:47.930 INFO:tasks.workunit.client.0.smithi089.stdout:Benchmarking for 30 iterations. 2024-09-10T08:41:47.930 INFO:tasks.workunit.client.0.smithi089.stdout:The test will run during 5 minutes. 2024-09-10T08:41:47.930 INFO:tasks.workunit.client.0.smithi089.stdout: 2024-09-10T08:41:47.930 INFO:tasks.workunit.client.0.smithi089.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-09-10T08:41:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.152+0000 7fef0de00640 1 -- 172.21.15.89:0/4193659469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef08104f10 msgr2=0x7fef08107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.152+0000 7fef0de00640 1 --2- 172.21.15.89:0/4193659469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef08104f10 0x7fef08107300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7feef4009900 tx=0x7feef402a410 comp rx=0 tx=0).stop 2024-09-10T08:41:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.152+0000 7fef0de00640 1 -- 172.21.15.89:0/4193659469 shutdown_connections 2024-09-10T08:41:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.152+0000 7fef0de00640 1 --2- 172.21.15.89:0/4193659469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef08107840 0x7fef08109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.152+0000 7fef0de00640 1 --2- 172.21.15.89:0/4193659469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef08104f10 0x7fef08107300 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.152+0000 7fef0de00640 1 --2- 172.21.15.89:0/4193659469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef081025e0 0x7fef081049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.152+0000 7fef0de00640 1 -- 172.21.15.89:0/4193659469 >> 172.21.15.89:0/4193659469 conn(0x7fef080fc1c0 msgr2=0x7fef080fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.152+0000 7fef0de00640 1 -- 172.21.15.89:0/4193659469 shutdown_connections 2024-09-10T08:41:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.152+0000 7fef0de00640 1 -- 172.21.15.89:0/4193659469 wait complete. 2024-09-10T08:41:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.153+0000 7fef0de00640 1 Processor -- start 2024-09-10T08:41:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.153+0000 7fef0de00640 1 -- start start 2024-09-10T08:41:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.154+0000 7fef0de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef081025e0 0x7fef0819a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.154+0000 7fef0de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef08104f10 0x7fef0819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.154+0000 7fef0de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef08107840 0x7fef081a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.154+0000 7fef0de00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fef0810d840 con 0x7fef081025e0 2024-09-10T08:41:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.154+0000 7fef0de00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef0810d6c0 con 0x7fef08104f10 2024-09-10T08:41:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.154+0000 7fef0de00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fef0810d9c0 con 0x7fef08107840 2024-09-10T08:41:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.154+0000 7fef06a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef081025e0 0x7fef0819a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.154+0000 7fef06000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef08104f10 0x7fef0819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.154+0000 7fef06000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef08104f10 0x7fef0819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:40770/0 (socket says 172.21.15.89:40770) 2024-09-10T08:41:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.154+0000 7fef06000640 1 -- 172.21.15.89:0/2332328082 learned_addr learned my addr 172.21.15.89:0/2332328082 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.154+0000 7fef06a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef081025e0 0x7fef0819a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40242/0 (socket says 172.21.15.89:40242) 2024-09-10T08:41:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.154+0000 7fef07400640 1 --2- 172.21.15.89:0/2332328082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef08107840 0x7fef081a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.155+0000 7fef06000640 1 -- 172.21.15.89:0/2332328082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef08107840 msgr2=0x7fef081a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.155+0000 7fef06000640 1 --2- 172.21.15.89:0/2332328082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef08107840 0x7fef081a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.155+0000 7fef06000640 1 -- 172.21.15.89:0/2332328082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef081025e0 msgr2=0x7fef0819a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.155+0000 7fef06000640 1 --2- 172.21.15.89:0/2332328082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef081025e0 0x7fef0819a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.155+0000 7fef06000640 1 -- 172.21.15.89:0/2332328082 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef081a1cc0 con 0x7fef08104f10 2024-09-10T08:41:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.155+0000 7fef06000640 1 --2- 172.21.15.89:0/2332328082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef08104f10 0x7fef0819a5e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feef4031610 tx=0x7feef4032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.155+0000 7fef0ca00640 1 -- 172.21.15.89:0/2332328082 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef403f030 con 0x7fef08104f10 2024-09-10T08:41:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.155+0000 7fef0ca00640 1 -- 172.21.15.89:0/2332328082 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feef4003ce0 con 0x7fef08104f10 2024-09-10T08:41:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.156+0000 7fef0de00640 1 -- 172.21.15.89:0/2332328082 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef081a1f50 con 0x7fef08104f10 2024-09-10T08:41:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.156+0000 7fef0ca00640 1 -- 172.21.15.89:0/2332328082 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4035070 con 0x7fef08104f10 2024-09-10T08:41:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.156+0000 7fef0de00640 1 -- 172.21.15.89:0/2332328082 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef081a23c0 con 0x7fef08104f10 2024-09-10T08:41:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.159+0000 7fef10dc3740 1 Processor -- start 2024-09-10T08:41:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.159+0000 7fef10dc3740 1 -- start start 2024-09-10T08:41:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.159+0000 7fef10dc3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 0x55d76b56e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.159+0000 7fef10dc3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b56e9d0 0x55d76b573490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.159+0000 7fef10dc3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d76b573a60 0x55d76b575f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.159+0000 7fef10dc3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d76b4d3100 con 0x55d76b56e9d0 2024-09-10T08:41:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.159+0000 7fef10dc3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d76b47b1b0 con 0x55d76b56e090 2024-09-10T08:41:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.159+0000 7fef10dc3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d76b4cf0c0 con 0x55d76b573a60 2024-09-10T08:41:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.159+0000 7fef07400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 0x55d76b56e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.160+0000 7fef06a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b56e9d0 0x55d76b573490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.160+0000 7fef07400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 0x55d76b56e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:40778/0 (socket says 172.21.15.89:40778) 2024-09-10T08:41:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.160+0000 7fef07400640 1 -- 172.21.15.89:0/3355948505 learned_addr learned my addr 172.21.15.89:0/3355948505 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.160+0000 7fef0ca00640 1 -- 172.21.15.89:0/2332328082 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7feef4042030 con 0x7fef08104f10 2024-09-10T08:41:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.160+0000 7fef07400640 1 -- 172.21.15.89:0/3355948505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d76b573a60 msgr2=0x55d76b575f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.160+0000 7fef07400640 1 --2- 172.21.15.89:0/3355948505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d76b573a60 0x55d76b575f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.160+0000 7fef07400640 1 -- 172.21.15.89:0/3355948505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b56e9d0 msgr2=0x55d76b573490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.160+0000 7fef07400640 1 --2- 172.21.15.89:0/3355948505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b56e9d0 0x55d76b573490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.160+0000 7fef07400640 1 -- 172.21.15.89:0/3355948505 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d76b1e6210 con 0x55d76b56e090 2024-09-10T08:41:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.160+0000 7fef06a00640 1 --2- 172.21.15.89:0/3355948505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b56e9d0 0x55d76b573490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.160+0000 7fef07400640 1 --2- 172.21.15.89:0/3355948505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 0x55d76b56e490 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fef00009f70 tx=0x7fef0000f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b335740e27e3bd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.160+0000 7feef3e00640 1 -- 172.21.15.89:0/3355948505 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef00015070 con 0x55d76b56e090 2024-09-10T08:41:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.160+0000 7feef3e00640 1 -- 172.21.15.89:0/3355948505 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef0001ab40 con 0x55d76b56e090 2024-09-10T08:41:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.160+0000 7feef3e00640 1 -- 172.21.15.89:0/3355948505 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef00011c30 con 0x55d76b56e090 2024-09-10T08:41:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.161+0000 7fef10dc3740 1 -- 172.21.15.89:0/3355948505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 msgr2=0x55d76b56e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.161+0000 7fef10dc3740 1 --2- 172.21.15.89:0/3355948505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 0x55d76b56e490 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fef00009f70 tx=0x7fef0000f7f0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.161+0000 7fef0ca00640 1 --2- 172.21.15.89:0/2332328082 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7feeec0a2cb0 0x7feeec0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.162+0000 7fef10dc3740 1 -- 172.21.15.89:0/3355948505 shutdown_connections 2024-09-10T08:41:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.162+0000 7fef10dc3740 1 --2- 172.21.15.89:0/3355948505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d76b573a60 0x55d76b575f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.162+0000 7fef10dc3740 1 --2- 172.21.15.89:0/3355948505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b56e9d0 0x55d76b573490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.162+0000 7fef10dc3740 1 --2- 172.21.15.89:0/3355948505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 0x55d76b56e490 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.162+0000 7fef10dc3740 1 -- 172.21.15.89:0/3355948505 >> 172.21.15.89:0/3355948505 conn(0x55d76b569700 msgr2=0x55d76b569b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.162+0000 7fef0ca00640 1 -- 172.21.15.89:0/2332328082 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7feef40ef850 con 0x7fef08104f10 2024-09-10T08:41:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.162+0000 7fef10dc3740 1 -- 172.21.15.89:0/3355948505 shutdown_connections 2024-09-10T08:41:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.162+0000 7fef06a00640 1 --2- 172.21.15.89:0/2332328082 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7feeec0a2cb0 0x7feeec0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.162+0000 7fef10dc3740 1 -- 172.21.15.89:0/3355948505 wait complete. 2024-09-10T08:41:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.162+0000 7fef10dc3740 1 -- start start 2024-09-10T08:41:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.162+0000 7fef10dc3740 1 Processor -- start 2024-09-10T08:41:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.163+0000 7fef06a00640 1 --2- 172.21.15.89:0/2332328082 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7feeec0a2cb0 0x7feeec0a5170 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55d76b56a580 tx=0x7feef8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.163+0000 7fef10dc3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 0x55d76b578390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.163+0000 7fef10dc3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b56e9d0 0x55d76b5788d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.163+0000 7fef10dc3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d76b573a60 0x55d76b57f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.163+0000 7fef10dc3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d76b576a30 con 0x55d76b56e9d0 2024-09-10T08:41:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.163+0000 7fef10dc3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d76b45c090 con 0x55d76b56e090 2024-09-10T08:41:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.163+0000 7fef10dc3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d76b576bb0 con 0x55d76b573a60 2024-09-10T08:41:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.163+0000 7fef10dc3740 10 client.0 _ll_register_callbacks cb 0x55d76b4b0f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.163+0000 7fef06a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 0x55d76b578390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.163+0000 7fef06a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 0x55d76b578390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:40784/0 (socket says 172.21.15.89:40784) 2024-09-10T08:41:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.163+0000 7fef06a00640 1 -- 172.21.15.89:0/661527297 learned_addr learned my addr 172.21.15.89:0/661527297 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.163+0000 7fef07400640 1 --2- 172.21.15.89:0/661527297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d76b573a60 0x55d76b57f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.163+0000 7fef06000640 1 --2- 172.21.15.89:0/661527297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b56e9d0 0x55d76b5788d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.163+0000 7fef06a00640 1 -- 172.21.15.89:0/661527297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d76b573a60 msgr2=0x55d76b57f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.163+0000 7fef06a00640 1 --2- 172.21.15.89:0/661527297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d76b573a60 0x55d76b57f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.164+0000 7fef06a00640 1 -- 172.21.15.89:0/661527297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b56e9d0 msgr2=0x55d76b5788d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.164+0000 7fef06a00640 1 --2- 172.21.15.89:0/661527297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b56e9d0 0x55d76b5788d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.164+0000 7fef06a00640 1 -- 172.21.15.89:0/661527297 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d76b2b2620 con 0x55d76b56e090 2024-09-10T08:41:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.164+0000 7fef06a00640 1 --2- 172.21.15.89:0/661527297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 0x55d76b578390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feef801e6f0 tx=0x7feef8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a47e3e062641d01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.164+0000 7feef0c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:41:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.164+0000 7feef0c00640 1 -- 172.21.15.89:0/661527297 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef801eda0 con 0x55d76b56e090 2024-09-10T08:41:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.164+0000 7feef0c00640 1 -- 172.21.15.89:0/661527297 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7feef8018030 con 0x55d76b56e090 2024-09-10T08:41:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.164+0000 7fef10dc3740 1 -- 172.21.15.89:0/661527297 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef0001ab40 con 0x55d76b56e090 2024-09-10T08:41:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.164+0000 7feef0c00640 1 -- 172.21.15.89:0/661527297 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feef8004020 con 0x55d76b56e090 2024-09-10T08:41:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.164+0000 7fef10dc3740 10 client.25835 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.164+0000 7fef10dc3740 1 -- 172.21.15.89:0/661527297 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d76b5826f0 con 0x55d76b56e090 2024-09-10T08:41:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.165+0000 7feef0c00640 1 -- 172.21.15.89:0/661527297 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef0001ab40 con 0x55d76b56e090 2024-09-10T08:41:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.165+0000 7feef0c00640 1 -- 172.21.15.89:0/661527297 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7feef8004a00 con 0x55d76b56e090 2024-09-10T08:41:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.165+0000 7fef10dc3740 10 client.25835 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.165+0000 7fef10dc3740 10 client.25835 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:41:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.165+0000 7fef10dc3740 20 client.25835 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.165+0000 7fef10dc3740 1 --2- 172.21.15.89:0/661527297 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d76b583dc0 0x55d76b586210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.165+0000 7fef10dc3740 4 client.25835 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:41:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.165+0000 7fef10dc3740 1 -- 172.21.15.89:0/661527297 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d76b4e0830 con 0x55d76b583dc0 2024-09-10T08:41:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.165+0000 7fef06000640 1 --2- 172.21.15.89:0/661527297 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d76b583dc0 0x55d76b586210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.166+0000 7feef0c00640 10 client.25835 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.166+0000 7fef06000640 1 --2- 172.21.15.89:0/661527297 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d76b583dc0 0x55d76b586210 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20ede4f8c0effe68 server_cookie=56bb057245da8a29 in_seq=0 out_seq=0 2024-09-10T08:41:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.167+0000 7feef0c00640 1 -- 172.21.15.89:0/661527297 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d76b4e0830 con 0x55d76b583dc0 2024-09-10T08:41:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.167+0000 7feef0c00640 10 client.25835 handle_command_reply: tid=0 2024-09-10T08:41:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.167+0000 7fef10dc3740 1 client.25835 shutdown 2024-09-10T08:41:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.168+0000 7fef10dc3740 1 -- 172.21.15.89:0/661527297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 msgr2=0x55d76b578390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.168+0000 7fef10dc3740 1 --2- 172.21.15.89:0/661527297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 0x55d76b578390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feef801e6f0 tx=0x7feef8004730 comp rx=0 tx=0).stop 2024-09-10T08:41:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.168+0000 7fef10dc3740 1 -- 172.21.15.89:0/661527297 shutdown_connections 2024-09-10T08:41:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.168+0000 7fef10dc3740 1 --2- 172.21.15.89:0/661527297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d76b573a60 0x55d76b57f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.168+0000 7fef10dc3740 1 --2- 172.21.15.89:0/661527297 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d76b583dc0 0x55d76b586210 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.168+0000 7fef10dc3740 1 --2- 172.21.15.89:0/661527297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b56e9d0 0x55d76b5788d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.168+0000 7fef10dc3740 1 --2- 172.21.15.89:0/661527297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 0x55d76b578390 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.168+0000 7fef10dc3740 1 -- 172.21.15.89:0/661527297 >> 172.21.15.89:0/661527297 conn(0x55d76b569700 msgr2=0x55d76b568880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.168+0000 7feef0c00640 0 client.25835 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.168+0000 7fef10dc3740 1 -- 172.21.15.89:0/661527297 shutdown_connections 2024-09-10T08:41:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.168+0000 7fef10dc3740 1 -- 172.21.15.89:0/661527297 wait complete. 2024-09-10T08:41:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.168+0000 7fef10dc3740 20 client.25835 trim_cache size 0 max 16384 2024-09-10T08:41:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.184+0000 7fef10dc3740 1 Processor -- start 2024-09-10T08:41:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.184+0000 7fef10dc3740 1 -- start start 2024-09-10T08:41:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.185+0000 7fef10dc3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d76b56e090 0x55d76b578960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.185+0000 7fef10dc3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e9d0 0x55d76b57f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.185+0000 7fef10dc3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b573a60 0x55d76b568770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.185+0000 7fef10dc3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fef081af320 con 0x55d76b573a60 2024-09-10T08:41:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.185+0000 7fef10dc3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef081acbf0 con 0x55d76b56e9d0 2024-09-10T08:41:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.185+0000 7fef10dc3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fef081aaa40 con 0x55d76b56e090 2024-09-10T08:41:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.185+0000 7fef06000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e9d0 0x55d76b57f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.185+0000 7fef06000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e9d0 0x55d76b57f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:40792/0 (socket says 172.21.15.89:40792) 2024-09-10T08:41:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.185+0000 7fef06000640 1 -- 172.21.15.89:0/3510897643 learned_addr learned my addr 172.21.15.89:0/3510897643 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.185+0000 7fef07400640 1 --2- 172.21.15.89:0/3510897643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b573a60 0x55d76b568770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.185+0000 7fef06a00640 1 --2- 172.21.15.89:0/3510897643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d76b56e090 0x55d76b578960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.185+0000 7fef06000640 1 -- 172.21.15.89:0/3510897643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d76b56e090 msgr2=0x55d76b578960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.185+0000 7fef06000640 1 --2- 172.21.15.89:0/3510897643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d76b56e090 0x55d76b578960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.185+0000 7fef06000640 1 -- 172.21.15.89:0/3510897643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b573a60 msgr2=0x55d76b568770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.185+0000 7fef06000640 1 --2- 172.21.15.89:0/3510897643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b573a60 0x55d76b568770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.185+0000 7fef06000640 1 -- 172.21.15.89:0/3510897643 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d76b569520 con 0x55d76b56e9d0 2024-09-10T08:41:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.185+0000 7fef06000640 1 --2- 172.21.15.89:0/3510897643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e9d0 0x55d76b57f7f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feef4005ab0 tx=0x7feef4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fde6276a2ab6badc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.186+0000 7feef3400640 1 -- 172.21.15.89:0/3510897643 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4049040 con 0x55d76b56e9d0 2024-09-10T08:41:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.186+0000 7feef3400640 1 -- 172.21.15.89:0/3510897643 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feef40bbc80 con 0x55d76b56e9d0 2024-09-10T08:41:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.186+0000 7feef3400640 1 -- 172.21.15.89:0/3510897643 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef403bd60 con 0x55d76b56e9d0 2024-09-10T08:41:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.186+0000 7fef10dc3740 1 -- 172.21.15.89:0/3510897643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e9d0 msgr2=0x55d76b57f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.186+0000 7fef10dc3740 1 --2- 172.21.15.89:0/3510897643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e9d0 0x55d76b57f7f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feef4005ab0 tx=0x7feef4009260 comp rx=0 tx=0).stop 2024-09-10T08:41:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.186+0000 7fef10dc3740 1 -- 172.21.15.89:0/3510897643 shutdown_connections 2024-09-10T08:41:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.186+0000 7fef10dc3740 1 --2- 172.21.15.89:0/3510897643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b573a60 0x55d76b568770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.186+0000 7fef10dc3740 1 --2- 172.21.15.89:0/3510897643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e9d0 0x55d76b57f7f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.186+0000 7fef10dc3740 1 --2- 172.21.15.89:0/3510897643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d76b56e090 0x55d76b578960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.186+0000 7fef10dc3740 1 -- 172.21.15.89:0/3510897643 >> 172.21.15.89:0/3510897643 conn(0x55d76b569700 msgr2=0x55d76b568060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.186+0000 7fef10dc3740 1 -- 172.21.15.89:0/3510897643 shutdown_connections 2024-09-10T08:41:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.186+0000 7fef10dc3740 1 -- 172.21.15.89:0/3510897643 wait complete. 2024-09-10T08:41:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.187+0000 7fef10dc3740 1 -- start start 2024-09-10T08:41:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.187+0000 7fef10dc3740 1 Processor -- start 2024-09-10T08:41:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.187+0000 7fef10dc3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 0x55d76b57cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.187+0000 7fef10dc3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d76b56e9d0 0x55d76b57d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.187+0000 7fef10dc3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b573a60 0x55d76b57e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.187+0000 7fef10dc3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fef081ab0b0 con 0x55d76b573a60 2024-09-10T08:41:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.187+0000 7fef10dc3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef081ac520 con 0x55d76b56e090 2024-09-10T08:41:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.187+0000 7fef10dc3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fef081ab780 con 0x55d76b56e9d0 2024-09-10T08:41:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.188+0000 7fef10dc3740 10 client.0 _ll_register_callbacks cb 0x7feef8012f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.188+0000 7fef06a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 0x55d76b57cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.188+0000 7fef06a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 0x55d76b57cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:40804/0 (socket says 172.21.15.89:40804) 2024-09-10T08:41:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.188+0000 7fef06a00640 1 -- 172.21.15.89:0/3148171895 learned_addr learned my addr 172.21.15.89:0/3148171895 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.188+0000 7fef07400640 1 --2- 172.21.15.89:0/3148171895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b573a60 0x55d76b57e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.188+0000 7fef06000640 1 --2- 172.21.15.89:0/3148171895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d76b56e9d0 0x55d76b57d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.188+0000 7fef06a00640 1 -- 172.21.15.89:0/3148171895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d76b56e9d0 msgr2=0x55d76b57d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.188+0000 7fef06a00640 1 --2- 172.21.15.89:0/3148171895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d76b56e9d0 0x55d76b57d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.188+0000 7fef06a00640 1 -- 172.21.15.89:0/3148171895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b573a60 msgr2=0x55d76b57e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.188+0000 7fef06a00640 1 --2- 172.21.15.89:0/3148171895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b573a60 0x55d76b57e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.188+0000 7fef06a00640 1 -- 172.21.15.89:0/3148171895 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d76b57e650 con 0x55d76b56e090 2024-09-10T08:41:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.188+0000 7fef07400640 1 --2- 172.21.15.89:0/3148171895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b573a60 0x55d76b57e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:41:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.188+0000 7fef06a00640 1 --2- 172.21.15.89:0/3148171895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 0x55d76b57cb40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feef801e6c0 tx=0x7feef8004460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a142e28bd766a62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.188+0000 7feef0c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:41:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.188+0000 7feef0c00640 1 -- 172.21.15.89:0/3148171895 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef803c020 con 0x55d76b56e090 2024-09-10T08:41:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.188+0000 7feef0c00640 1 -- 172.21.15.89:0/3148171895 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7feef8004af0 con 0x55d76b56e090 2024-09-10T08:41:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.188+0000 7fef10dc3740 1 -- 172.21.15.89:0/3148171895 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feef40bbc80 con 0x55d76b56e090 2024-09-10T08:41:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.189+0000 7feef0c00640 1 -- 172.21.15.89:0/3148171895 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feef8047050 con 0x55d76b56e090 2024-09-10T08:41:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.189+0000 7fef10dc3740 10 client.25841 fetch_fsmap learned FSMap version 26 2024-09-10T08:41:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.189+0000 7fef10dc3740 1 -- 172.21.15.89:0/3148171895 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d76b57f960 con 0x55d76b56e090 2024-09-10T08:41:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.189+0000 7feef0c00640 1 -- 172.21.15.89:0/3148171895 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feef40bbc80 con 0x55d76b56e090 2024-09-10T08:41:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.189+0000 7feef0c00640 1 -- 172.21.15.89:0/3148171895 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7feef80434b0 con 0x55d76b56e090 2024-09-10T08:41:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.189+0000 7fef10dc3740 10 client.25841 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T08:41:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.189+0000 7fef10dc3740 10 client.25841 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:41:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.189+0000 7fef10dc3740 20 client.25841 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.189+0000 7fef10dc3740 1 --2- 172.21.15.89:0/3148171895 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d76b5ef140 0x55d76b5f1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.189+0000 7fef10dc3740 4 client.25841 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:41:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.189+0000 7fef10dc3740 1 -- 172.21.15.89:0/3148171895 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d76b4dab80 con 0x55d76b5ef140 2024-09-10T08:41:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.190+0000 7fef06000640 1 --2- 172.21.15.89:0/3148171895 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d76b5ef140 0x55d76b5f1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.190+0000 7fef06000640 1 --2- 172.21.15.89:0/3148171895 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d76b5ef140 0x55d76b5f1560 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7fa16d227fbdf4 server_cookie=c4021e523e24133f in_seq=0 out_seq=0 2024-09-10T08:41:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:48.190+0000 7feef0c00640 10 client.25841 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:49.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:48 smithi111 ceph-mon[35220]: pgmap v449: 129 pgs: 129 active+clean; 3.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 291 KiB/s wr, 36 op/s 2024-09-10T08:41:49.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:48 smithi089 ceph-mon[30757]: pgmap v449: 129 pgs: 129 active+clean; 3.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 291 KiB/s wr, 36 op/s 2024-09-10T08:41:49.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:48 smithi149 ceph-mon[36167]: pgmap v449: 129 pgs: 129 active+clean; 3.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 291 KiB/s wr, 36 op/s 2024-09-10T08:41:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.938+0000 7feef0c00640 1 -- 172.21.15.89:0/3148171895 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d76b4dab80 con 0x55d76b5ef140 2024-09-10T08:41:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.938+0000 7feef0c00640 10 client.25841 handle_command_reply: tid=0 2024-09-10T08:41:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.938+0000 7fef10dc3740 1 client.25841 shutdown 2024-09-10T08:41:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.938+0000 7fef10dc3740 1 -- 172.21.15.89:0/3148171895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 msgr2=0x55d76b57cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.938+0000 7fef10dc3740 1 --2- 172.21.15.89:0/3148171895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 0x55d76b57cb40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feef801e6c0 tx=0x7feef8004460 comp rx=0 tx=0).stop 2024-09-10T08:41:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.938+0000 7fef10dc3740 1 -- 172.21.15.89:0/3148171895 shutdown_connections 2024-09-10T08:41:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.938+0000 7fef10dc3740 1 --2- 172.21.15.89:0/3148171895 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d76b5ef140 0x55d76b5f1560 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.938+0000 7fef10dc3740 1 --2- 172.21.15.89:0/3148171895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d76b573a60 0x55d76b57e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.938+0000 7feef0c00640 0 client.25841 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.938+0000 7fef10dc3740 1 --2- 172.21.15.89:0/3148171895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d76b56e9d0 0x55d76b57d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.938+0000 7fef10dc3740 1 --2- 172.21.15.89:0/3148171895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d76b56e090 0x55d76b57cb40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.938+0000 7fef10dc3740 1 -- 172.21.15.89:0/3148171895 >> 172.21.15.89:0/3148171895 conn(0x55d76b569700 msgr2=0x55d76b56add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.938+0000 7fef10dc3740 1 -- 172.21.15.89:0/3148171895 shutdown_connections 2024-09-10T08:41:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.938+0000 7fef10dc3740 1 -- 172.21.15.89:0/3148171895 wait complete. 2024-09-10T08:41:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.939+0000 7fef10dc3740 20 client.25841 trim_cache size 0 max 16384 2024-09-10T08:41:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:41:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:41:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:41:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:41:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.940+0000 7fef0de00640 1 -- 172.21.15.89:0/2332328082 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7feeec0a2cb0 msgr2=0x7feeec0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.940+0000 7fef0de00640 1 --2- 172.21.15.89:0/2332328082 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7feeec0a2cb0 0x7feeec0a5170 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55d76b56a580 tx=0x7feef8005f90 comp rx=0 tx=0).stop 2024-09-10T08:41:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.940+0000 7fef0de00640 1 -- 172.21.15.89:0/2332328082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef08104f10 msgr2=0x7fef0819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.940+0000 7fef0de00640 1 --2- 172.21.15.89:0/2332328082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef08104f10 0x7fef0819a5e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feef4031610 tx=0x7feef4032ce0 comp rx=0 tx=0).stop 2024-09-10T08:41:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.940+0000 7fef0de00640 1 -- 172.21.15.89:0/2332328082 shutdown_connections 2024-09-10T08:41:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.940+0000 7fef0de00640 1 --2- 172.21.15.89:0/2332328082 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7feeec0a2cb0 0x7feeec0a5170 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.940+0000 7fef0de00640 1 --2- 172.21.15.89:0/2332328082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef08107840 0x7fef081a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.940+0000 7fef0de00640 1 --2- 172.21.15.89:0/2332328082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef08104f10 0x7fef0819a5e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.940+0000 7fef0de00640 1 --2- 172.21.15.89:0/2332328082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef081025e0 0x7fef0819a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.940+0000 7fef0de00640 1 -- 172.21.15.89:0/2332328082 >> 172.21.15.89:0/2332328082 conn(0x7fef080fc1c0 msgr2=0x7fef08101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.940+0000 7fef0de00640 1 -- 172.21.15.89:0/2332328082 shutdown_connections 2024-09-10T08:41:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:41:50.940+0000 7fef0de00640 1 -- 172.21.15.89:0/2332328082 wait complete. 2024-09-10T08:41:51.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:50 smithi111 ceph-mon[35220]: pgmap v450: 129 pgs: 129 active+clean; 3.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 297 KiB/s wr, 37 op/s 2024-09-10T08:41:51.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:50 smithi089 ceph-mon[30757]: pgmap v450: 129 pgs: 129 active+clean; 3.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 297 KiB/s wr, 37 op/s 2024-09-10T08:41:51.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:50 smithi149 ceph-mon[36167]: pgmap v450: 129 pgs: 129 active+clean; 3.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 297 KiB/s wr, 37 op/s 2024-09-10T08:41:52.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:51 smithi111 ceph-mon[35220]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:41:52.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:51 smithi089 ceph-mon[30757]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:41:52.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:51 smithi149 ceph-mon[36167]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:41:53.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:52 smithi111 ceph-mon[35220]: pgmap v451: 129 pgs: 129 active+clean; 9.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 898 KiB/s wr, 101 op/s 2024-09-10T08:41:53.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:52 smithi089 ceph-mon[30757]: pgmap v451: 129 pgs: 129 active+clean; 9.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 898 KiB/s wr, 101 op/s 2024-09-10T08:41:53.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:52 smithi149 ceph-mon[36167]: pgmap v451: 129 pgs: 129 active+clean; 9.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 898 KiB/s wr, 101 op/s 2024-09-10T08:41:55.023 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:54 smithi149 ceph-mon[36167]: pgmap v452: 129 pgs: 129 active+clean; 9.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 831 KiB/s wr, 107 op/s 2024-09-10T08:41:55.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:54 smithi111 ceph-mon[35220]: pgmap v452: 129 pgs: 129 active+clean; 9.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 831 KiB/s wr, 107 op/s 2024-09-10T08:41:55.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:54 smithi089 ceph-mon[30757]: pgmap v452: 129 pgs: 129 active+clean; 9.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 831 KiB/s wr, 107 op/s 2024-09-10T08:41:57.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:56 smithi111 ceph-mon[35220]: pgmap v453: 129 pgs: 129 active+clean; 13 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-09-10T08:41:57.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:56 smithi111 ceph-mon[35220]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:41:57.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:56 smithi089 ceph-mon[30757]: pgmap v453: 129 pgs: 129 active+clean; 13 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-09-10T08:41:57.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:56 smithi089 ceph-mon[30757]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:41:57.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:56 smithi149 ceph-mon[36167]: pgmap v453: 129 pgs: 129 active+clean; 13 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-09-10T08:41:57.222 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:56 smithi149 ceph-mon[36167]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:41:57.934 INFO:tasks.workunit.client.0.smithi089.stdout: 17 215096 1141 150421 1026 64949 1034 2024-09-10T08:41:58.058 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:41:58.814 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:41:58 smithi089 ceph-mon[30757]: pgmap v454: 129 pgs: 129 active+clean; 28 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 182 op/s 2024-09-10T08:41:59.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:41:58 smithi111 ceph-mon[35220]: pgmap v454: 129 pgs: 129 active+clean; 28 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 182 op/s 2024-09-10T08:41:59.047 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:41:59.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:41:58 smithi149 ceph-mon[36167]: pgmap v454: 129 pgs: 129 active+clean; 28 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 182 op/s 2024-09-10T08:41:59.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.709+0000 7f7cc3c00640 1 -- 172.21.15.89:0/1989257050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cbc101270 msgr2=0x7f7cbc10df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:59.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.709+0000 7f7cc3c00640 1 --2- 172.21.15.89:0/1989257050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cbc101270 0x7f7cbc10df30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009900 tx=0x7f7ca802a400 comp rx=0 tx=0).stop 2024-09-10T08:41:59.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.709+0000 7f7cc3c00640 1 -- 172.21.15.89:0/1989257050 shutdown_connections 2024-09-10T08:41:59.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.709+0000 7f7cc3c00640 1 --2- 172.21.15.89:0/1989257050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cbc10e470 0x7f7cbc110860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.709+0000 7f7cc3c00640 1 --2- 172.21.15.89:0/1989257050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cbc101270 0x7f7cbc10df30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.709+0000 7f7cc3c00640 1 --2- 172.21.15.89:0/1989257050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cbc1008c0 0x7f7cbc100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.709+0000 7f7cc3c00640 1 -- 172.21.15.89:0/1989257050 >> 172.21.15.89:0/1989257050 conn(0x7f7cbc0fc790 msgr2=0x7f7cbc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:59.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.709+0000 7f7cc3c00640 1 -- 172.21.15.89:0/1989257050 shutdown_connections 2024-09-10T08:41:59.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.709+0000 7f7cc3c00640 1 -- 172.21.15.89:0/1989257050 wait complete. 2024-09-10T08:41:59.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.711+0000 7f7cc3c00640 1 Processor -- start 2024-09-10T08:41:59.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.711+0000 7f7cc3c00640 1 -- start start 2024-09-10T08:41:59.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.711+0000 7f7cc3c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cbc1008c0 0x7f7cbc19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:59.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.711+0000 7f7cc3c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cbc101270 0x7f7cbc19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:59.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.711+0000 7f7cc3c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cbc10e470 0x7f7cbc19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:59.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.711+0000 7f7cc3c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc1141d0 con 0x7f7cbc1008c0 2024-09-10T08:41:59.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.711+0000 7f7cc3c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc114050 con 0x7f7cbc10e470 2024-09-10T08:41:59.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.711+0000 7f7cc3c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc114350 con 0x7f7cbc101270 2024-09-10T08:41:59.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.712+0000 7f7cc1c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cbc10e470 0x7f7cbc19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:59.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.712+0000 7f7cc1c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cbc10e470 0x7f7cbc19efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:48762/0 (socket says 172.21.15.89:48762) 2024-09-10T08:41:59.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.712+0000 7f7cc1c00640 1 -- 172.21.15.89:0/4138577571 learned_addr learned my addr 172.21.15.89:0/4138577571 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:59.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.712+0000 7f7cc1200640 1 --2- 172.21.15.89:0/4138577571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cbc1008c0 0x7f7cbc19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:59.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.712+0000 7f7cbbe00640 1 --2- 172.21.15.89:0/4138577571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cbc101270 0x7f7cbc19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:59.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.712+0000 7f7cc1c00640 1 -- 172.21.15.89:0/4138577571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cbc101270 msgr2=0x7f7cbc19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:59.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.712+0000 7f7cc1c00640 1 --2- 172.21.15.89:0/4138577571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cbc101270 0x7f7cbc19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.712+0000 7f7cc1c00640 1 -- 172.21.15.89:0/4138577571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cbc1008c0 msgr2=0x7f7cbc19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:59.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.712+0000 7f7cc1c00640 1 --2- 172.21.15.89:0/4138577571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cbc1008c0 0x7f7cbc19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.712+0000 7f7cc1c00640 1 -- 172.21.15.89:0/4138577571 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cbc19f6e0 con 0x7f7cbc10e470 2024-09-10T08:41:59.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.712+0000 7f7cc1200640 1 --2- 172.21.15.89:0/4138577571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cbc1008c0 0x7f7cbc19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:41:59.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.712+0000 7f7cbbe00640 1 --2- 172.21.15.89:0/4138577571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cbc101270 0x7f7cbc19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:41:59.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.713+0000 7f7cc1c00640 1 --2- 172.21.15.89:0/4138577571 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cbc10e470 0x7f7cbc19efe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400ec50 tx=0x7f7cb400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:59.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.713+0000 7f7cb9600640 1 -- 172.21.15.89:0/4138577571 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4003b70 con 0x7f7cbc10e470 2024-09-10T08:41:59.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.713+0000 7f7cc3c00640 1 -- 172.21.15.89:0/4138577571 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cbc19f9d0 con 0x7f7cbc10e470 2024-09-10T08:41:59.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.713+0000 7f7cc3c00640 1 -- 172.21.15.89:0/4138577571 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cbc06bc10 con 0x7f7cbc10e470 2024-09-10T08:41:59.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.713+0000 7f7cb9600640 1 -- 172.21.15.89:0/4138577571 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cb4003d10 con 0x7f7cbc10e470 2024-09-10T08:41:59.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.713+0000 7f7cb9600640 1 -- 172.21.15.89:0/4138577571 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb40109e0 con 0x7f7cbc10e470 2024-09-10T08:41:59.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.716+0000 7f7cb9600640 1 -- 172.21.15.89:0/4138577571 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7cb4010b80 con 0x7f7cbc10e470 2024-09-10T08:41:59.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.717+0000 7f7cb9600640 1 --2- 172.21.15.89:0/4138577571 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7ca40a2c00 0x7f7ca40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:59.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.717+0000 7f7cc6678740 1 Processor -- start 2024-09-10T08:41:59.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.718+0000 7f7cb9600640 1 -- 172.21.15.89:0/4138577571 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f7cb40cfab0 con 0x7f7cbc10e470 2024-09-10T08:41:59.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.718+0000 7f7cc6678740 1 -- start start 2024-09-10T08:41:59.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.718+0000 7f7cc1200640 1 --2- 172.21.15.89:0/4138577571 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7ca40a2c00 0x7f7ca40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:59.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.718+0000 7f7cc6678740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de508ab0 0x5573de508e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:59.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.718+0000 7f7cc6678740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de5093d0 0x5573de51c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:59.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.718+0000 7f7cc6678740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de51c580 0x5573de51e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:59.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.718+0000 7f7cc6678740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5573de46f3e0 con 0x5573de51c580 2024-09-10T08:41:59.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.718+0000 7f7cc6678740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5573de416d80 con 0x5573de508ab0 2024-09-10T08:41:59.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.718+0000 7f7cc6678740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5573de46b3a0 con 0x5573de5093d0 2024-09-10T08:41:59.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.719+0000 7f7cc1c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de51c580 0x5573de51e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:59.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.719+0000 7f7cc1c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de51c580 0x5573de51e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37626/0 (socket says 172.21.15.89:37626) 2024-09-10T08:41:59.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.719+0000 7f7cc1c00640 1 -- 172.21.15.89:0/908758136 learned_addr learned my addr 172.21.15.89:0/908758136 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:59.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.719+0000 7f7cc1c00640 1 -- 172.21.15.89:0/908758136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de5093d0 msgr2=0x5573de51c040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:41:59.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.719+0000 7f7cc1c00640 1 --2- 172.21.15.89:0/908758136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de5093d0 0x5573de51c040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.719+0000 7f7cc1c00640 1 -- 172.21.15.89:0/908758136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de508ab0 msgr2=0x5573de508e90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:41:59.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.719+0000 7f7cc1c00640 1 --2- 172.21.15.89:0/908758136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de508ab0 0x5573de508e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.719+0000 7f7cc1c00640 1 -- 172.21.15.89:0/908758136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573de435ae0 con 0x5573de51c580 2024-09-10T08:41:59.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.719+0000 7f7cc1c00640 1 --2- 172.21.15.89:0/908758136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de51c580 0x5573de51e970 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400cfd0 tx=0x7f7cb4099250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23434ec9a300c07 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:59.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.720+0000 7f7cae000640 1 -- 172.21.15.89:0/908758136 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb40a2430 con 0x5573de51c580 2024-09-10T08:41:59.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.720+0000 7f7cae000640 1 -- 172.21.15.89:0/908758136 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cb409d5f0 con 0x5573de51c580 2024-09-10T08:41:59.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.720+0000 7f7cae000640 1 -- 172.21.15.89:0/908758136 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4099560 con 0x5573de51c580 2024-09-10T08:41:59.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.720+0000 7f7cc6678740 1 -- 172.21.15.89:0/908758136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de51c580 msgr2=0x5573de51e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:59.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.720+0000 7f7cc1200640 1 --2- 172.21.15.89:0/4138577571 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7ca40a2c00 0x7f7ca40a50c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009cd0 tx=0x7f7cb0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:59.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.720+0000 7f7cc6678740 1 --2- 172.21.15.89:0/908758136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de51c580 0x5573de51e970 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400cfd0 tx=0x7f7cb4099250 comp rx=0 tx=0).stop 2024-09-10T08:41:59.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.721+0000 7f7cc6678740 1 -- 172.21.15.89:0/908758136 shutdown_connections 2024-09-10T08:41:59.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.721+0000 7f7cc6678740 1 --2- 172.21.15.89:0/908758136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de51c580 0x5573de51e970 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.721+0000 7f7cc6678740 1 --2- 172.21.15.89:0/908758136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de5093d0 0x5573de51c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.721+0000 7f7cc6678740 1 --2- 172.21.15.89:0/908758136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de508ab0 0x5573de508e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.721+0000 7f7cc6678740 1 -- 172.21.15.89:0/908758136 >> 172.21.15.89:0/908758136 conn(0x5573de505120 msgr2=0x5573de505500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:59.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.721+0000 7f7cc6678740 1 -- 172.21.15.89:0/908758136 shutdown_connections 2024-09-10T08:41:59.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.721+0000 7f7cc6678740 1 -- 172.21.15.89:0/908758136 wait complete. 2024-09-10T08:41:59.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.721+0000 7f7cc6678740 1 -- start start 2024-09-10T08:41:59.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.722+0000 7f7cc6678740 1 Processor -- start 2024-09-10T08:41:59.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.722+0000 7f7cc6678740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de508ab0 0x5573de519480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:59.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.722+0000 7f7cc6678740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de5093d0 0x5573de5199c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:59.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.722+0000 7f7cc6678740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de51c580 0x5573de519f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:59.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.722+0000 7f7cc6678740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5573de51f130 con 0x5573de5093d0 2024-09-10T08:41:59.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.722+0000 7f7cc6678740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5573de42a0f0 con 0x5573de51c580 2024-09-10T08:41:59.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.722+0000 7f7cc6678740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5573de51f2b0 con 0x5573de508ab0 2024-09-10T08:41:59.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.723+0000 7f7cc6678740 10 client.0 _ll_register_callbacks cb 0x5573de41b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:59.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.723+0000 7f7cc1200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de508ab0 0x5573de519480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:59.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.723+0000 7f7cc1c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de51c580 0x5573de519f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:59.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.723+0000 7f7cc1200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de508ab0 0x5573de519480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:41924/0 (socket says 172.21.15.89:41924) 2024-09-10T08:41:59.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.723+0000 7f7cc1200640 1 -- 172.21.15.89:0/3995179801 learned_addr learned my addr 172.21.15.89:0/3995179801 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:59.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.723+0000 7f7cbbe00640 1 --2- 172.21.15.89:0/3995179801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de5093d0 0x5573de5199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:59.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.723+0000 7f7cc1c00640 1 -- 172.21.15.89:0/3995179801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de508ab0 msgr2=0x5573de519480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:59.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.723+0000 7f7cc1c00640 1 --2- 172.21.15.89:0/3995179801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de508ab0 0x5573de519480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.723+0000 7f7cc1c00640 1 -- 172.21.15.89:0/3995179801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de5093d0 msgr2=0x5573de5199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:59.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.723+0000 7f7cc1c00640 1 --2- 172.21.15.89:0/3995179801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de5093d0 0x5573de5199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.723+0000 7f7cc1c00640 1 -- 172.21.15.89:0/3995179801 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573de1e03c0 con 0x5573de51c580 2024-09-10T08:41:59.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.725+0000 7f7cc1c00640 1 --2- 172.21.15.89:0/3995179801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de51c580 0x5573de519f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x5573de5098d0 tx=0x7f7cb40992c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5512f9b74d467368 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:59.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.725+0000 7f7ca2a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:41:59.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.725+0000 7f7ca2a00640 1 -- 172.21.15.89:0/3995179801 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb40bb020 con 0x5573de51c580 2024-09-10T08:41:59.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.725+0000 7f7ca2a00640 1 -- 172.21.15.89:0/3995179801 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f7cb40a58d0 con 0x5573de51c580 2024-09-10T08:41:59.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.725+0000 7f7cc6678740 1 -- 172.21.15.89:0/3995179801 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cb409d5f0 con 0x5573de51c580 2024-09-10T08:41:59.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.725+0000 7f7ca2a00640 1 -- 172.21.15.89:0/3995179801 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7cb40a12a0 con 0x5573de51c580 2024-09-10T08:41:59.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.725+0000 7f7cc6678740 10 client.25850 fetch_fsmap learned FSMap version 28 2024-09-10T08:41:59.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.725+0000 7f7cc6678740 1 -- 172.21.15.89:0/3995179801 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5573de52aec0 con 0x5573de51c580 2024-09-10T08:41:59.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.725+0000 7f7ca2a00640 1 -- 172.21.15.89:0/3995179801 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cb409d5f0 con 0x5573de51c580 2024-09-10T08:41:59.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.726+0000 7f7ca2a00640 1 -- 172.21.15.89:0/3995179801 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7cb40bbbc0 con 0x5573de51c580 2024-09-10T08:41:59.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.726+0000 7f7cc6678740 10 client.25850 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:41:59.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.726+0000 7f7cc6678740 10 client.25850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:41:59.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.726+0000 7f7cc6678740 20 client.25850 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:59.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.726+0000 7f7cc6678740 1 --2- 172.21.15.89:0/3995179801 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5573de52c210 0x5573de52e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:59.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.726+0000 7f7cc6678740 4 client.25850 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:41:59.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.726+0000 7f7cc6678740 1 -- 172.21.15.89:0/3995179801 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573de47c950 con 0x5573de52c210 2024-09-10T08:41:59.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.726+0000 7f7cc1200640 1 --2- 172.21.15.89:0/3995179801 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5573de52c210 0x5573de52e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:59.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.727+0000 7f7cc1200640 1 --2- 172.21.15.89:0/3995179801 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5573de52c210 0x5573de52e600 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2db0629c77b1163 server_cookie=f67a063030746a7b in_seq=0 out_seq=0 2024-09-10T08:41:59.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.727+0000 7f7ca2a00640 10 client.25850 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:59.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.727+0000 7f7ca2a00640 1 -- 172.21.15.89:0/3995179801 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573de47c950 con 0x5573de52c210 2024-09-10T08:41:59.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.727+0000 7f7ca2a00640 10 client.25850 handle_command_reply: tid=0 2024-09-10T08:41:59.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.727+0000 7f7cc6678740 1 client.25850 shutdown 2024-09-10T08:41:59.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.728+0000 7f7cc6678740 1 -- 172.21.15.89:0/3995179801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de51c580 msgr2=0x5573de519f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:59.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.728+0000 7f7cc6678740 1 --2- 172.21.15.89:0/3995179801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de51c580 0x5573de519f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x5573de5098d0 tx=0x7f7cb40992c0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.728+0000 7f7cc6678740 1 -- 172.21.15.89:0/3995179801 shutdown_connections 2024-09-10T08:41:59.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.728+0000 7f7cc6678740 1 --2- 172.21.15.89:0/3995179801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de51c580 0x5573de519f00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.728+0000 7f7cc6678740 1 --2- 172.21.15.89:0/3995179801 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5573de52c210 0x5573de52e600 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.728+0000 7f7cc6678740 1 --2- 172.21.15.89:0/3995179801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de5093d0 0x5573de5199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.728+0000 7f7cc6678740 1 --2- 172.21.15.89:0/3995179801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de508ab0 0x5573de519480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.728+0000 7f7cc6678740 1 -- 172.21.15.89:0/3995179801 >> 172.21.15.89:0/3995179801 conn(0x5573de505120 msgr2=0x5573de50adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:59.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.728+0000 7f7ca2a00640 0 client.25850 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:59.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.728+0000 7f7cc6678740 1 -- 172.21.15.89:0/3995179801 shutdown_connections 2024-09-10T08:41:59.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.728+0000 7f7cc6678740 1 -- 172.21.15.89:0/3995179801 wait complete. 2024-09-10T08:41:59.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.728+0000 7f7cc6678740 20 client.25850 trim_cache size 0 max 16384 2024-09-10T08:41:59.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.746+0000 7f7cc6678740 1 Processor -- start 2024-09-10T08:41:59.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.746+0000 7f7cc6678740 1 -- start start 2024-09-10T08:41:59.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.747+0000 7f7cc6678740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de508ab0 0x5573de519d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:59.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.747+0000 7f7cc6678740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de5093d0 0x5573de531240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:59.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.747+0000 7f7cc6678740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de51c580 0x5573de531780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:59.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.747+0000 7f7cc6678740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c0124c0 con 0x5573de51c580 2024-09-10T08:41:59.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.747+0000 7f7cc6678740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c00fd90 con 0x5573de508ab0 2024-09-10T08:41:59.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.747+0000 7f7cc6678740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c00dbe0 con 0x5573de5093d0 2024-09-10T08:41:59.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.747+0000 7f7cbbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de5093d0 0x5573de531240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:59.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.747+0000 7f7cbbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de5093d0 0x5573de531240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:41932/0 (socket says 172.21.15.89:41932) 2024-09-10T08:41:59.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.747+0000 7f7cbbe00640 1 -- 172.21.15.89:0/1062741914 learned_addr learned my addr 172.21.15.89:0/1062741914 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:59.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.747+0000 7f7cc1c00640 1 --2- 172.21.15.89:0/1062741914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de51c580 0x5573de531780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:59.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.747+0000 7f7cc1200640 1 --2- 172.21.15.89:0/1062741914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de508ab0 0x5573de519d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:59.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.748+0000 7f7cc1200640 1 -- 172.21.15.89:0/1062741914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de5093d0 msgr2=0x5573de531240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:59.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.748+0000 7f7cc1200640 1 --2- 172.21.15.89:0/1062741914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de5093d0 0x5573de531240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.748+0000 7f7cc1200640 1 -- 172.21.15.89:0/1062741914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de51c580 msgr2=0x5573de531780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:59.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.748+0000 7f7cc1200640 1 --2- 172.21.15.89:0/1062741914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de51c580 0x5573de531780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.748+0000 7f7cc1200640 1 -- 172.21.15.89:0/1062741914 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573de514210 con 0x5573de508ab0 2024-09-10T08:41:59.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.748+0000 7f7cc1c00640 1 --2- 172.21.15.89:0/1062741914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de51c580 0x5573de531780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:59.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.748+0000 7f7cc1200640 1 --2- 172.21.15.89:0/1062741914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de508ab0 0x5573de519d50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0011c40 tx=0x7f7cb0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caab32cd307ef6a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:59.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.748+0000 7f7cad600640 1 -- 172.21.15.89:0/1062741914 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0004020 con 0x5573de508ab0 2024-09-10T08:41:59.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.748+0000 7f7cad600640 1 -- 172.21.15.89:0/1062741914 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cb00041c0 con 0x5573de508ab0 2024-09-10T08:41:59.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.748+0000 7f7cad600640 1 -- 172.21.15.89:0/1062741914 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0029600 con 0x5573de508ab0 2024-09-10T08:41:59.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.749+0000 7f7cc6678740 1 -- 172.21.15.89:0/1062741914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de508ab0 msgr2=0x5573de519d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:59.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.749+0000 7f7cc6678740 1 --2- 172.21.15.89:0/1062741914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de508ab0 0x5573de519d50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0011c40 tx=0x7f7cb0014000 comp rx=0 tx=0).stop 2024-09-10T08:41:59.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.749+0000 7f7cc6678740 1 -- 172.21.15.89:0/1062741914 shutdown_connections 2024-09-10T08:41:59.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.749+0000 7f7cc6678740 1 --2- 172.21.15.89:0/1062741914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de51c580 0x5573de531780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.749+0000 7f7cc6678740 1 --2- 172.21.15.89:0/1062741914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de5093d0 0x5573de531240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.749+0000 7f7cc6678740 1 --2- 172.21.15.89:0/1062741914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de508ab0 0x5573de519d50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.749+0000 7f7cc6678740 1 -- 172.21.15.89:0/1062741914 >> 172.21.15.89:0/1062741914 conn(0x5573de505120 msgr2=0x5573de51b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:59.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.749+0000 7f7cc6678740 1 -- 172.21.15.89:0/1062741914 shutdown_connections 2024-09-10T08:41:59.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.749+0000 7f7cc6678740 1 -- 172.21.15.89:0/1062741914 wait complete. 2024-09-10T08:41:59.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.750+0000 7f7cc6678740 1 -- start start 2024-09-10T08:41:59.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.750+0000 7f7cc6678740 1 Processor -- start 2024-09-10T08:41:59.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.750+0000 7f7cc6678740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de508ab0 0x5573de531730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:59.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.750+0000 7f7cc6678740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de5093d0 0x5573de52ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:59.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.750+0000 7f7cc6678740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de51c580 0x5573de52efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:59.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.750+0000 7f7cc6678740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c00e250 con 0x5573de5093d0 2024-09-10T08:41:59.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.750+0000 7f7cc6678740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c00f6c0 con 0x5573de51c580 2024-09-10T08:41:59.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.750+0000 7f7cc6678740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c00e920 con 0x5573de508ab0 2024-09-10T08:41:59.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.750+0000 7f7cc6678740 10 client.0 _ll_register_callbacks cb 0x7f7cb40a5c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:41:59.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.751+0000 7f7cbbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de5093d0 0x5573de52ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:59.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.751+0000 7f7cbbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de5093d0 0x5573de52ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37656/0 (socket says 172.21.15.89:37656) 2024-09-10T08:41:59.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.751+0000 7f7cbbe00640 1 -- 172.21.15.89:0/3875143606 learned_addr learned my addr 172.21.15.89:0/3875143606 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:41:59.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.751+0000 7f7cc1c00640 1 --2- 172.21.15.89:0/3875143606 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de51c580 0x5573de52efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:59.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.751+0000 7f7cc1200640 1 --2- 172.21.15.89:0/3875143606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de508ab0 0x5573de531730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:59.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.751+0000 7f7cc1c00640 1 -- 172.21.15.89:0/3875143606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de508ab0 msgr2=0x5573de531730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:59.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.751+0000 7f7cc1c00640 1 --2- 172.21.15.89:0/3875143606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de508ab0 0x5573de531730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.751+0000 7f7cc1c00640 1 -- 172.21.15.89:0/3875143606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de5093d0 msgr2=0x5573de52ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:41:59.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.751+0000 7f7cc1c00640 1 --2- 172.21.15.89:0/3875143606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de5093d0 0x5573de52ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.751+0000 7f7cc1c00640 1 -- 172.21.15.89:0/3875143606 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573de52f580 con 0x5573de51c580 2024-09-10T08:41:59.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.751+0000 7f7cbbe00640 1 --2- 172.21.15.89:0/3875143606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de5093d0 0x5573de52ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:41:59.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.751+0000 7f7cc1c00640 1 --2- 172.21.15.89:0/3875143606 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de51c580 0x5573de52efc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40998c0 tx=0x7f7cb4010430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15af4b4478302732 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:59.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.751+0000 7f7ca2a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:41:59.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.752+0000 7f7ca2a00640 1 -- 172.21.15.89:0/3875143606 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb40bb040 con 0x5573de51c580 2024-09-10T08:41:59.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.752+0000 7f7ca2a00640 1 -- 172.21.15.89:0/3875143606 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f7cb40a58d0 con 0x5573de51c580 2024-09-10T08:41:59.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.752+0000 7f7cc6678740 1 -- 172.21.15.89:0/3875143606 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cb00041c0 con 0x5573de51c580 2024-09-10T08:41:59.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.752+0000 7f7ca2a00640 1 -- 172.21.15.89:0/3875143606 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7cb409ca30 con 0x5573de51c580 2024-09-10T08:41:59.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.752+0000 7f7cc6678740 10 client.25856 fetch_fsmap learned FSMap version 28 2024-09-10T08:41:59.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.752+0000 7f7cc6678740 1 -- 172.21.15.89:0/3875143606 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5573de52f580 con 0x5573de51c580 2024-09-10T08:41:59.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.752+0000 7f7ca2a00640 1 -- 172.21.15.89:0/3875143606 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cb00041c0 con 0x5573de51c580 2024-09-10T08:41:59.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.752+0000 7f7ca2a00640 1 -- 172.21.15.89:0/3875143606 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7cb409bdb0 con 0x5573de51c580 2024-09-10T08:41:59.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.752+0000 7f7cc6678740 10 client.25856 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:41:59.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.752+0000 7f7cc6678740 10 client.25856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:41:59.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.752+0000 7f7cc6678740 20 client.25856 populate_metadata read hostname 'smithi089' 2024-09-10T08:41:59.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.752+0000 7f7cc6678740 1 --2- 172.21.15.89:0/3875143606 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5573de596c70 0x5573de599090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:59.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.752+0000 7f7cc6678740 4 client.25856 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:41:59.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.752+0000 7f7cc6678740 1 -- 172.21.15.89:0/3875143606 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5573de476e60 con 0x5573de596c70 2024-09-10T08:41:59.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.753+0000 7f7cc1200640 1 --2- 172.21.15.89:0/3875143606 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5573de596c70 0x5573de599090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:59.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.753+0000 7f7cc1200640 1 --2- 172.21.15.89:0/3875143606 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5573de596c70 0x5573de599090 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6157a95fff3ef81 server_cookie=9472bfb62147f06c in_seq=0 out_seq=0 2024-09-10T08:41:59.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.753+0000 7f7ca2a00640 10 client.25856 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:59.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.753+0000 7f7ca2a00640 1 -- 172.21.15.89:0/3875143606 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5573de476e60 con 0x5573de596c70 2024-09-10T08:41:59.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.753+0000 7f7ca2a00640 10 client.25856 handle_command_reply: tid=0 2024-09-10T08:41:59.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.753+0000 7f7cc6678740 1 client.25856 shutdown 2024-09-10T08:41:59.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.754+0000 7f7cc6678740 1 -- 172.21.15.89:0/3875143606 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de51c580 msgr2=0x5573de52efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:59.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.754+0000 7f7cc6678740 1 --2- 172.21.15.89:0/3875143606 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de51c580 0x5573de52efc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40998c0 tx=0x7f7cb4010430 comp rx=0 tx=0).stop 2024-09-10T08:41:59.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.754+0000 7f7cc6678740 1 -- 172.21.15.89:0/3875143606 shutdown_connections 2024-09-10T08:41:59.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.754+0000 7f7cc6678740 1 --2- 172.21.15.89:0/3875143606 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5573de51c580 0x5573de52efc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.754+0000 7f7cc6678740 1 --2- 172.21.15.89:0/3875143606 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5573de596c70 0x5573de599090 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.754+0000 7f7cc6678740 1 --2- 172.21.15.89:0/3875143606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5573de5093d0 0x5573de52ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.754+0000 7f7cc6678740 1 --2- 172.21.15.89:0/3875143606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573de508ab0 0x5573de531730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.754+0000 7f7ca2a00640 0 client.25856 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:41:59.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.754+0000 7f7cc6678740 1 -- 172.21.15.89:0/3875143606 >> 172.21.15.89:0/3875143606 conn(0x5573de505120 msgr2=0x5573de5275d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:59.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.754+0000 7f7cc6678740 1 -- 172.21.15.89:0/3875143606 shutdown_connections 2024-09-10T08:41:59.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.754+0000 7f7cc6678740 1 -- 172.21.15.89:0/3875143606 wait complete. 2024-09-10T08:41:59.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.754+0000 7f7cc6678740 20 client.25856 trim_cache size 0 max 16384 2024-09-10T08:41:59.759 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:41:59.759 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2024-09-10T08:41:59.759 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2024-09-10T08:41:59.759 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:41:59.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.755+0000 7f7cc3c00640 1 -- 172.21.15.89:0/4138577571 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7ca40a2c00 msgr2=0x7f7ca40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:59.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.755+0000 7f7cc3c00640 1 --2- 172.21.15.89:0/4138577571 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7ca40a2c00 0x7f7ca40a50c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009cd0 tx=0x7f7cb0005f90 comp rx=0 tx=0).stop 2024-09-10T08:41:59.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.755+0000 7f7cc3c00640 1 -- 172.21.15.89:0/4138577571 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cbc10e470 msgr2=0x7f7cbc19efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:59.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.755+0000 7f7cc3c00640 1 --2- 172.21.15.89:0/4138577571 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cbc10e470 0x7f7cbc19efe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400ec50 tx=0x7f7cb400c6a0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.755+0000 7f7cc3c00640 1 -- 172.21.15.89:0/4138577571 shutdown_connections 2024-09-10T08:41:59.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.755+0000 7f7cc3c00640 1 --2- 172.21.15.89:0/4138577571 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7ca40a2c00 0x7f7ca40a50c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.755+0000 7f7cc3c00640 1 --2- 172.21.15.89:0/4138577571 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7cbc10e470 0x7f7cbc19efe0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.755+0000 7f7cc3c00640 1 --2- 172.21.15.89:0/4138577571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cbc101270 0x7f7cbc19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.755+0000 7f7cc3c00640 1 --2- 172.21.15.89:0/4138577571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cbc1008c0 0x7f7cbc19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.755+0000 7f7cc3c00640 1 -- 172.21.15.89:0/4138577571 >> 172.21.15.89:0/4138577571 conn(0x7f7cbc0fc790 msgr2=0x7f7cbc0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:59.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.755+0000 7f7cc3c00640 1 -- 172.21.15.89:0/4138577571 shutdown_connections 2024-09-10T08:41:59.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:41:59.755+0000 7f7cc3c00640 1 -- 172.21.15.89:0/4138577571 wait complete. 2024-09-10T08:42:00.983 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:42:00.984 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:42:01.032 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:00 smithi089 ceph-mon[30757]: pgmap v455: 129 pgs: 129 active+clean; 30 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 177 op/s 2024-09-10T08:42:01.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:00 smithi111 ceph-mon[35220]: pgmap v455: 129 pgs: 129 active+clean; 30 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 177 op/s 2024-09-10T08:42:01.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:00 smithi149 ceph-mon[36167]: pgmap v455: 129 pgs: 129 active+clean; 30 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 177 op/s 2024-09-10T08:42:01.365 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:42:02.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.280+0000 7f2299400640 1 -- 172.21.15.89:0/1979887333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2294073f30 msgr2=0x7f229406bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:02.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.280+0000 7f2299400640 1 --2- 172.21.15.89:0/1979887333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2294073f30 0x7f229406bd20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2284009860 tx=0x7f228402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:42:02.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.280+0000 7f2299400640 1 -- 172.21.15.89:0/1979887333 shutdown_connections 2024-09-10T08:42:02.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.280+0000 7f2299400640 1 --2- 172.21.15.89:0/1979887333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22941137e0 0x7f2294115bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:02.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.280+0000 7f2299400640 1 --2- 172.21.15.89:0/1979887333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f229406c260 0x7f229406c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:02.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.280+0000 7f2299400640 1 --2- 172.21.15.89:0/1979887333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2294073f30 0x7f229406bd20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:02.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.280+0000 7f2299400640 1 -- 172.21.15.89:0/1979887333 >> 172.21.15.89:0/1979887333 conn(0x7f22940fc610 msgr2=0x7f22940fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:02.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.281+0000 7f2299400640 1 -- 172.21.15.89:0/1979887333 shutdown_connections 2024-09-10T08:42:02.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.281+0000 7f2299400640 1 -- 172.21.15.89:0/1979887333 wait complete. 2024-09-10T08:42:02.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.282+0000 7f2299400640 1 Processor -- start 2024-09-10T08:42:02.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.285+0000 7f2299400640 1 -- start start 2024-09-10T08:42:02.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.285+0000 7f2299400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f229406c260 0x7f22941a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:02.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.285+0000 7f2299400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2294073f30 0x7f22941a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:02.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.285+0000 7f2299400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22941137e0 0x7f229419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:02.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.285+0000 7f2299400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f22941183c0 con 0x7f229406c260 2024-09-10T08:42:02.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.285+0000 7f2299400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2294118240 con 0x7f22941137e0 2024-09-10T08:42:02.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.285+0000 7f2299400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2294118540 con 0x7f2294073f30 2024-09-10T08:42:02.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.286+0000 7f2298a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22941137e0 0x7f229419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:02.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.286+0000 7f2298a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22941137e0 0x7f229419e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:48822/0 (socket says 172.21.15.89:48822) 2024-09-10T08:42:02.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.286+0000 7f2298a00640 1 -- 172.21.15.89:0/1347971438 learned_addr learned my addr 172.21.15.89:0/1347971438 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:02.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.286+0000 7f2293e00640 1 --2- 172.21.15.89:0/1347971438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f229406c260 0x7f22941a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:02.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.286+0000 7f2298a00640 1 -- 172.21.15.89:0/1347971438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2294073f30 msgr2=0x7f22941a4b00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:42:02.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.286+0000 7f2298a00640 1 --2- 172.21.15.89:0/1347971438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2294073f30 0x7f22941a4b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:02.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.286+0000 7f2298a00640 1 -- 172.21.15.89:0/1347971438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f229406c260 msgr2=0x7f22941a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:02.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.286+0000 7f2298a00640 1 --2- 172.21.15.89:0/1347971438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f229406c260 0x7f22941a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:02.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.286+0000 7f2298a00640 1 -- 172.21.15.89:0/1347971438 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229419ef00 con 0x7f22941137e0 2024-09-10T08:42:02.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.287+0000 7f2298a00640 1 --2- 172.21.15.89:0/1347971438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22941137e0 0x7f229419e690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f228c0163f0 tx=0x7f228c0168b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:02.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.289+0000 7f2290c00640 1 -- 172.21.15.89:0/1347971438 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c00fbe0 con 0x7f22941137e0 2024-09-10T08:42:02.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.289+0000 7f2290c00640 1 -- 172.21.15.89:0/1347971438 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f228c00fd80 con 0x7f22941137e0 2024-09-10T08:42:02.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.289+0000 7f2290c00640 1 -- 172.21.15.89:0/1347971438 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c01c770 con 0x7f22941137e0 2024-09-10T08:42:02.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.290+0000 7f2299400640 1 -- 172.21.15.89:0/1347971438 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229419f190 con 0x7f22941137e0 2024-09-10T08:42:02.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.291+0000 7f2299400640 1 -- 172.21.15.89:0/1347971438 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22941ab370 con 0x7f22941137e0 2024-09-10T08:42:02.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.301+0000 7f2299400640 1 -- 172.21.15.89:0/1347971438 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f229406d460 con 0x7f22941137e0 2024-09-10T08:42:02.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.301+0000 7f2290c00640 1 -- 172.21.15.89:0/1347971438 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f228c023030 con 0x7f22941137e0 2024-09-10T08:42:02.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.304+0000 7f2290c00640 1 --2- 172.21.15.89:0/1347971438 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f22780a2c00 0x7f22780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:02.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.304+0000 7f2290c00640 1 -- 172.21.15.89:0/1347971438 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f228c0a1660 con 0x7f22941137e0 2024-09-10T08:42:02.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.305+0000 7f2293e00640 1 --2- 172.21.15.89:0/1347971438 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f22780a2c00 0x7f22780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:02.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.307+0000 7f2293e00640 1 --2- 172.21.15.89:0/1347971438 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f22780a2c00 0x7f22780a50c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2284000c00 tx=0x7f2284005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:02.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.313+0000 7f2290c00640 1 -- 172.21.15.89:0/1347971438 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f228c018040 con 0x7f22941137e0 2024-09-10T08:42:02.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.716+0000 7f2299400640 1 -- 172.21.15.89:0/1347971438 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f229402d7c0 con 0x7f22941137e0 2024-09-10T08:42:02.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.717+0000 7f2290c00640 1 -- 172.21.15.89:0/1347971438 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11921 (secure 0 0 0) 0x7f228c018040 con 0x7f22941137e0 2024-09-10T08:42:02.718 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:42:02.718 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":28,"btime":"2024-09-10T08:41:55:762565+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:41:55.148062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24518,24496]},"id":1}]} 2024-09-10T08:42:02.719 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 28 2024-09-10T08:42:02.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.720+0000 7f2299400640 1 -- 172.21.15.89:0/1347971438 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f22780a2c00 msgr2=0x7f22780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:02.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.720+0000 7f2299400640 1 --2- 172.21.15.89:0/1347971438 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f22780a2c00 0x7f22780a50c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2284000c00 tx=0x7f2284005c90 comp rx=0 tx=0).stop 2024-09-10T08:42:02.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.720+0000 7f2299400640 1 -- 172.21.15.89:0/1347971438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22941137e0 msgr2=0x7f229419e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:02.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.720+0000 7f2299400640 1 --2- 172.21.15.89:0/1347971438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22941137e0 0x7f229419e690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f228c0163f0 tx=0x7f228c0168b0 comp rx=0 tx=0).stop 2024-09-10T08:42:02.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.720+0000 7f2299400640 1 -- 172.21.15.89:0/1347971438 shutdown_connections 2024-09-10T08:42:02.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.720+0000 7f2299400640 1 --2- 172.21.15.89:0/1347971438 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f22780a2c00 0x7f22780a50c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:02.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.720+0000 7f2299400640 1 --2- 172.21.15.89:0/1347971438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22941137e0 0x7f229419e690 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:02.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.720+0000 7f2299400640 1 --2- 172.21.15.89:0/1347971438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2294073f30 0x7f22941a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:02.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.720+0000 7f2299400640 1 --2- 172.21.15.89:0/1347971438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f229406c260 0x7f22941a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:02.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.720+0000 7f2299400640 1 -- 172.21.15.89:0/1347971438 >> 172.21.15.89:0/1347971438 conn(0x7f22940fc610 msgr2=0x7f2294072fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:02.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.720+0000 7f2299400640 1 -- 172.21.15.89:0/1347971438 shutdown_connections 2024-09-10T08:42:02.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:02.720+0000 7f2299400640 1 -- 172.21.15.89:0/1347971438 wait complete. 2024-09-10T08:42:02.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:02 smithi089 ceph-mon[30757]: pgmap v456: 129 pgs: 129 active+clean; 55 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 374 op/s 2024-09-10T08:42:02.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:02 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1347971438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:42:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:02 smithi111 ceph-mon[35220]: pgmap v456: 129 pgs: 129 active+clean; 55 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 374 op/s 2024-09-10T08:42:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:02 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1347971438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:42:03.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:02 smithi149 ceph-mon[36167]: pgmap v456: 129 pgs: 129 active+clean; 55 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 374 op/s 2024-09-10T08:42:03.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:02 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1347971438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:42:03.878 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:42:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.083+0000 7f297c600640 1 -- 172.21.15.89:0/1587814226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2974104f80 msgr2=0x7f2974107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.083+0000 7f297c600640 1 --2- 172.21.15.89:0/1587814226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2974104f80 0x7f2974107370 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2960009900 tx=0x7f296002a420 comp rx=0 tx=0).stop 2024-09-10T08:42:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.083+0000 7f297c600640 1 -- 172.21.15.89:0/1587814226 shutdown_connections 2024-09-10T08:42:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.083+0000 7f297c600640 1 --2- 172.21.15.89:0/1587814226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29741078b0 0x7f2974109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.083+0000 7f297c600640 1 --2- 172.21.15.89:0/1587814226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2974104f80 0x7f2974107370 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.083+0000 7f297c600640 1 --2- 172.21.15.89:0/1587814226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2974102650 0x7f2974104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.083+0000 7f297c600640 1 -- 172.21.15.89:0/1587814226 >> 172.21.15.89:0/1587814226 conn(0x7f29740fc1b0 msgr2=0x7f29740fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.084+0000 7f2979c00640 1 --2- 172.21.15.89:0/1587814226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2974102650 0x7f2974104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:42:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.085+0000 7f297c600640 1 -- 172.21.15.89:0/1587814226 shutdown_connections 2024-09-10T08:42:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.085+0000 7f297c600640 1 -- 172.21.15.89:0/1587814226 wait complete. 2024-09-10T08:42:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.086+0000 7f297c600640 1 Processor -- start 2024-09-10T08:42:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.086+0000 7f297c600640 1 -- start start 2024-09-10T08:42:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.086+0000 7f297c600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2974104f80 0x7f2974111030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.086+0000 7f297c600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29741078b0 0x7f2974111570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.087+0000 7f297c600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29741165d0 0x7f2974116a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.087+0000 7f297c600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f297410aaf0 con 0x7f29741078b0 2024-09-10T08:42:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.087+0000 7f297c600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f297410a970 con 0x7f2974104f80 2024-09-10T08:42:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.087+0000 7f297c600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f297410ac70 con 0x7f29741165d0 2024-09-10T08:42:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.087+0000 7f2979c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2974104f80 0x7f2974111030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.087+0000 7f297a600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29741165d0 0x7f2974116a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.087+0000 7f2979200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29741078b0 0x7f2974111570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.087+0000 7f2979c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2974104f80 0x7f2974111030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:37672/0 (socket says 172.21.15.89:37672) 2024-09-10T08:42:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.087+0000 7f297a600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29741165d0 0x7f2974116a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:60226/0 (socket says 172.21.15.89:60226) 2024-09-10T08:42:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.087+0000 7f2979c00640 1 -- 172.21.15.89:0/668676287 learned_addr learned my addr 172.21.15.89:0/668676287 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.087+0000 7f297a600640 1 -- 172.21.15.89:0/668676287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2974104f80 msgr2=0x7f2974111030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.087+0000 7f297a600640 1 --2- 172.21.15.89:0/668676287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2974104f80 0x7f2974111030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.087+0000 7f297a600640 1 -- 172.21.15.89:0/668676287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29741078b0 msgr2=0x7f2974111570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.087+0000 7f297a600640 1 --2- 172.21.15.89:0/668676287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29741078b0 0x7f2974111570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.087+0000 7f297a600640 1 -- 172.21.15.89:0/668676287 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2974117040 con 0x7f29741165d0 2024-09-10T08:42:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.087+0000 7f2979c00640 1 --2- 172.21.15.89:0/668676287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2974104f80 0x7f2974111030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:42:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.088+0000 7f297a600640 1 --2- 172.21.15.89:0/668676287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29741165d0 0x7f2974116a80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f296c005fd0 tx=0x7f296c00edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.088+0000 7f2972000640 1 -- 172.21.15.89:0/668676287 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c004d20 con 0x7f29741165d0 2024-09-10T08:42:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.088+0000 7f2972000640 1 -- 172.21.15.89:0/668676287 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f296c005020 con 0x7f29741165d0 2024-09-10T08:42:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.088+0000 7f2972000640 1 -- 172.21.15.89:0/668676287 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c0116d0 con 0x7f29741165d0 2024-09-10T08:42:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.088+0000 7f297c600640 1 -- 172.21.15.89:0/668676287 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2974117330 con 0x7f29741165d0 2024-09-10T08:42:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.088+0000 7f297c600640 1 -- 172.21.15.89:0/668676287 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2974117880 con 0x7f29741165d0 2024-09-10T08:42:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.093+0000 7f2972000640 1 -- 172.21.15.89:0/668676287 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f296c011930 con 0x7f29741165d0 2024-09-10T08:42:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.093+0000 7f297f5bd740 1 Processor -- start 2024-09-10T08:42:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.093+0000 7f297f5bd740 1 -- start start 2024-09-10T08:42:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.093+0000 7f297f5bd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616c7090 0x5641616c7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.094+0000 7f297f5bd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616c79d0 0x5641616cc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.094+0000 7f297f5bd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641616cca60 0x5641616cef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.094+0000 7f297f5bd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56416162c100 con 0x5641616c79d0 2024-09-10T08:42:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.094+0000 7f297f5bd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5641615d41b0 con 0x5641616c7090 2024-09-10T08:42:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.094+0000 7f297f5bd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5641616280c0 con 0x5641616cca60 2024-09-10T08:42:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.094+0000 7f2979c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616c79d0 0x5641616cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.094+0000 7f2979200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641616cca60 0x5641616cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.094+0000 7f2979c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616c79d0 0x5641616cc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46894/0 (socket says 172.21.15.89:46894) 2024-09-10T08:42:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.094+0000 7f2979200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616c7090 0x5641616c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.094+0000 7f2979c00640 1 -- 172.21.15.89:0/3847198073 learned_addr learned my addr 172.21.15.89:0/3847198073 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.094+0000 7f2979c00640 1 -- 172.21.15.89:0/3847198073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641616cca60 msgr2=0x5641616cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.094+0000 7f2979c00640 1 --2- 172.21.15.89:0/3847198073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641616cca60 0x5641616cef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.094+0000 7f2979c00640 1 -- 172.21.15.89:0/3847198073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616c7090 msgr2=0x5641616c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.096+0000 7f2979c00640 1 --2- 172.21.15.89:0/3847198073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616c7090 0x5641616c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.096+0000 7f2979c00640 1 -- 172.21.15.89:0/3847198073 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56416133f210 con 0x5641616c79d0 2024-09-10T08:42:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.096+0000 7f2979200640 1 --2- 172.21.15.89:0/3847198073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616c7090 0x5641616c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:42:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.096+0000 7f2979c00640 1 --2- 172.21.15.89:0/3847198073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616c79d0 0x5641616cc490 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f296800af70 tx=0x7f296800f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1749e1a689d3c72b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.096+0000 7f2966a00640 1 -- 172.21.15.89:0/3847198073 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296801d070 con 0x5641616c79d0 2024-09-10T08:42:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.097+0000 7f2966a00640 1 -- 172.21.15.89:0/3847198073 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2968010de0 con 0x5641616c79d0 2024-09-10T08:42:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.097+0000 7f2966a00640 1 -- 172.21.15.89:0/3847198073 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968016340 con 0x5641616c79d0 2024-09-10T08:42:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.097+0000 7f2972000640 1 --2- 172.21.15.89:0/668676287 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f295c0a2d30 0x7f295c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.098+0000 7f2972000640 1 -- 172.21.15.89:0/668676287 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f296c0d0480 con 0x7f29741165d0 2024-09-10T08:42:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.098+0000 7f2979200640 1 --2- 172.21.15.89:0/668676287 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f295c0a2d30 0x7f295c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.101+0000 7f297f5bd740 1 -- 172.21.15.89:0/3847198073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616c79d0 msgr2=0x5641616cc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.101+0000 7f2979200640 1 --2- 172.21.15.89:0/668676287 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f295c0a2d30 0x7f295c0a51f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f296002aa10 tx=0x7f29600315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.101+0000 7f297f5bd740 1 --2- 172.21.15.89:0/3847198073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616c79d0 0x5641616cc490 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f296800af70 tx=0x7f296800f850 comp rx=0 tx=0).stop 2024-09-10T08:42:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.101+0000 7f297f5bd740 1 -- 172.21.15.89:0/3847198073 shutdown_connections 2024-09-10T08:42:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.101+0000 7f297f5bd740 1 --2- 172.21.15.89:0/3847198073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641616cca60 0x5641616cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.101+0000 7f297f5bd740 1 --2- 172.21.15.89:0/3847198073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616c79d0 0x5641616cc490 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.101+0000 7f297f5bd740 1 --2- 172.21.15.89:0/3847198073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616c7090 0x5641616c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.101+0000 7f297f5bd740 1 -- 172.21.15.89:0/3847198073 >> 172.21.15.89:0/3847198073 conn(0x5641616c2700 msgr2=0x5641616c2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.101+0000 7f297f5bd740 1 -- 172.21.15.89:0/3847198073 shutdown_connections 2024-09-10T08:42:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.101+0000 7f297f5bd740 1 -- 172.21.15.89:0/3847198073 wait complete. 2024-09-10T08:42:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.102+0000 7f297f5bd740 1 -- start start 2024-09-10T08:42:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.103+0000 7f297f5bd740 1 Processor -- start 2024-09-10T08:42:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.104+0000 7f297f5bd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616c7090 0x5641616e2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.105+0000 7f297f5bd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616c79d0 0x5641616e27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.105+0000 7f297f5bd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641616cca60 0x5641616e97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.105+0000 7f297f5bd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5641616e0a40 con 0x5641616c7090 2024-09-10T08:42:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.105+0000 7f297f5bd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5641615b5090 con 0x5641616c79d0 2024-09-10T08:42:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.105+0000 7f297f5bd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5641616e0bc0 con 0x5641616cca60 2024-09-10T08:42:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.105+0000 7f297f5bd740 10 client.0 _ll_register_callbacks cb 0x564161609f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:42:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.106+0000 7f2979c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616c7090 0x5641616e2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.106+0000 7f2979c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616c7090 0x5641616e2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46900/0 (socket says 172.21.15.89:46900) 2024-09-10T08:42:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.106+0000 7f2979c00640 1 -- 172.21.15.89:0/446759729 learned_addr learned my addr 172.21.15.89:0/446759729 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.106+0000 7f2979c00640 1 -- 172.21.15.89:0/446759729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641616cca60 msgr2=0x5641616e97e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:42:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.106+0000 7f2979c00640 1 --2- 172.21.15.89:0/446759729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641616cca60 0x5641616e97e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.106+0000 7f2979c00640 1 -- 172.21.15.89:0/446759729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616c79d0 msgr2=0x5641616e27a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:42:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.106+0000 7f2979c00640 1 --2- 172.21.15.89:0/446759729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616c79d0 0x5641616e27a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.107+0000 7f2979c00640 1 -- 172.21.15.89:0/446759729 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56416140b620 con 0x5641616c7090 2024-09-10T08:42:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.107+0000 7f2979c00640 1 --2- 172.21.15.89:0/446759729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616c7090 0x5641616e2260 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f296800af40 tx=0x7f2968010930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8002747243062c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.109+0000 7f295b400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:42:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.109+0000 7f295b400640 1 -- 172.21.15.89:0/446759729 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296801d050 con 0x5641616c7090 2024-09-10T08:42:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.109+0000 7f295b400640 1 -- 172.21.15.89:0/446759729 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f2968005040 con 0x5641616c7090 2024-09-10T08:42:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.112+0000 7f297f5bd740 1 -- 172.21.15.89:0/446759729 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2968010de0 con 0x5641616c7090 2024-09-10T08:42:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.113+0000 7f295b400640 1 -- 172.21.15.89:0/446759729 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2968005240 con 0x5641616c7090 2024-09-10T08:42:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.121+0000 7f297f5bd740 10 client.15813 fetch_fsmap learned FSMap version 28 2024-09-10T08:42:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.121+0000 7f297f5bd740 1 -- 172.21.15.89:0/446759729 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5641616ec540 con 0x5641616c7090 2024-09-10T08:42:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.124+0000 7f295b400640 1 -- 172.21.15.89:0/446759729 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2968010de0 con 0x5641616c7090 2024-09-10T08:42:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.124+0000 7f295b400640 1 -- 172.21.15.89:0/446759729 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f296802e450 con 0x5641616c7090 2024-09-10T08:42:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.124+0000 7f297f5bd740 10 client.15813 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:42:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.124+0000 7f297f5bd740 10 client.15813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:42:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.124+0000 7f297f5bd740 20 client.15813 populate_metadata read hostname 'smithi089' 2024-09-10T08:42:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.124+0000 7f297f5bd740 1 --2- 172.21.15.89:0/446759729 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5641616eda90 0x5641616efef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.124+0000 7f297f5bd740 4 client.15813 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:42:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.124+0000 7f297f5bd740 1 -- 172.21.15.89:0/446759729 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564161639830 con 0x5641616eda90 2024-09-10T08:42:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.124+0000 7f2979200640 1 --2- 172.21.15.89:0/446759729 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5641616eda90 0x5641616efef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.125+0000 7f295b400640 10 client.15813 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.126+0000 7f2979200640 1 --2- 172.21.15.89:0/446759729 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5641616eda90 0x5641616efef0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c3e780ca17fc692 server_cookie=fbdaba85bab05136 in_seq=0 out_seq=0 2024-09-10T08:42:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.127+0000 7f295b400640 1 -- 172.21.15.89:0/446759729 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564161639830 con 0x5641616eda90 2024-09-10T08:42:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.127+0000 7f295b400640 10 client.15813 handle_command_reply: tid=0 2024-09-10T08:42:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.128+0000 7f297f5bd740 1 client.15813 shutdown 2024-09-10T08:42:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.128+0000 7f297f5bd740 1 -- 172.21.15.89:0/446759729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616c7090 msgr2=0x5641616e2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.128+0000 7f297f5bd740 1 --2- 172.21.15.89:0/446759729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616c7090 0x5641616e2260 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f296800af40 tx=0x7f2968010930 comp rx=0 tx=0).stop 2024-09-10T08:42:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.134+0000 7f297f5bd740 1 -- 172.21.15.89:0/446759729 shutdown_connections 2024-09-10T08:42:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.134+0000 7f297f5bd740 1 --2- 172.21.15.89:0/446759729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641616cca60 0x5641616e97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.134+0000 7f297f5bd740 1 --2- 172.21.15.89:0/446759729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616c79d0 0x5641616e27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.134+0000 7f297f5bd740 1 --2- 172.21.15.89:0/446759729 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5641616eda90 0x5641616efef0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.134+0000 7f297f5bd740 1 --2- 172.21.15.89:0/446759729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616c7090 0x5641616e2260 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.134+0000 7f297f5bd740 1 -- 172.21.15.89:0/446759729 >> 172.21.15.89:0/446759729 conn(0x5641616c2700 msgr2=0x5641616c3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.134+0000 7f295b400640 0 client.15813 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.134+0000 7f297f5bd740 1 -- 172.21.15.89:0/446759729 shutdown_connections 2024-09-10T08:42:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.134+0000 7f297f5bd740 1 -- 172.21.15.89:0/446759729 wait complete. 2024-09-10T08:42:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.134+0000 7f297f5bd740 20 client.15813 trim_cache size 0 max 16384 2024-09-10T08:42:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.323+0000 7f297f5bd740 1 Processor -- start 2024-09-10T08:42:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.334+0000 7f297f5bd740 1 -- start start 2024-09-10T08:42:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.334+0000 7f297f5bd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641616c7090 0x5641616e9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.334+0000 7f297f5bd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616c79d0 0x5641616e9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.335+0000 7f297f5bd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616cca60 0x5641616c1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.335+0000 7f297f5bd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f29741a0da0 con 0x5641616cca60 2024-09-10T08:42:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.335+0000 7f297f5bd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f297419e6b0 con 0x5641616c79d0 2024-09-10T08:42:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.335+0000 7f297f5bd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2974074050 con 0x5641616c7090 2024-09-10T08:42:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.337+0000 7f297a600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616cca60 0x5641616c1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.338+0000 7f297a600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616cca60 0x5641616c1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46916/0 (socket says 172.21.15.89:46916) 2024-09-10T08:42:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.338+0000 7f297a600640 1 -- 172.21.15.89:0/2521924976 learned_addr learned my addr 172.21.15.89:0/2521924976 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.339+0000 7f2979200640 1 --2- 172.21.15.89:0/2521924976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616c79d0 0x5641616e9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.340+0000 7f2979200640 1 -- 172.21.15.89:0/2521924976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641616c7090 msgr2=0x5641616e9e90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:42:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.340+0000 7f2979200640 1 --2- 172.21.15.89:0/2521924976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641616c7090 0x5641616e9e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.340+0000 7f2979200640 1 -- 172.21.15.89:0/2521924976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616cca60 msgr2=0x5641616c1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.340+0000 7f2979200640 1 --2- 172.21.15.89:0/2521924976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616cca60 0x5641616c1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.340+0000 7f2979200640 1 -- 172.21.15.89:0/2521924976 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641616bdd60 con 0x5641616c79d0 2024-09-10T08:42:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.341+0000 7f2979200640 1 --2- 172.21.15.89:0/2521924976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616c79d0 0x5641616e9750 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2960009230 tx=0x7f296003e070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20c6a7dfa1548a06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.342+0000 7f2966000640 1 -- 172.21.15.89:0/2521924976 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960056070 con 0x5641616c79d0 2024-09-10T08:42:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.351+0000 7f2966000640 1 -- 172.21.15.89:0/2521924976 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2960003ad0 con 0x5641616c79d0 2024-09-10T08:42:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.351+0000 7f2966000640 1 -- 172.21.15.89:0/2521924976 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960035040 con 0x5641616c79d0 2024-09-10T08:42:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.353+0000 7f297f5bd740 1 -- 172.21.15.89:0/2521924976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616c79d0 msgr2=0x5641616e9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.353+0000 7f297f5bd740 1 --2- 172.21.15.89:0/2521924976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616c79d0 0x5641616e9750 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2960009230 tx=0x7f296003e070 comp rx=0 tx=0).stop 2024-09-10T08:42:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.355+0000 7f297f5bd740 1 -- 172.21.15.89:0/2521924976 shutdown_connections 2024-09-10T08:42:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.355+0000 7f297f5bd740 1 --2- 172.21.15.89:0/2521924976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616cca60 0x5641616c1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.355+0000 7f297f5bd740 1 --2- 172.21.15.89:0/2521924976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616c79d0 0x5641616e9750 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.355+0000 7f297f5bd740 1 --2- 172.21.15.89:0/2521924976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641616c7090 0x5641616e9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.355+0000 7f297f5bd740 1 -- 172.21.15.89:0/2521924976 >> 172.21.15.89:0/2521924976 conn(0x5641616c2700 msgr2=0x5641616c1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.358+0000 7f297f5bd740 1 -- 172.21.15.89:0/2521924976 shutdown_connections 2024-09-10T08:42:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.358+0000 7f297f5bd740 1 -- 172.21.15.89:0/2521924976 wait complete. 2024-09-10T08:42:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.359+0000 7f297f5bd740 1 -- start start 2024-09-10T08:42:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.359+0000 7f297f5bd740 1 Processor -- start 2024-09-10T08:42:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.366+0000 7f297f5bd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641616c7090 0x5641616e6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.371+0000 7f297f5bd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616c79d0 0x5641616e6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.371+0000 7f297f5bd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616cca60 0x5641616e7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.371+0000 7f297f5bd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f297419cb70 con 0x5641616c79d0 2024-09-10T08:42:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.371+0000 7f297f5bd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f297419dfe0 con 0x5641616cca60 2024-09-10T08:42:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.371+0000 7f297f5bd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f297419d240 con 0x5641616c7090 2024-09-10T08:42:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.371+0000 7f297f5bd740 10 client.0 _ll_register_callbacks cb 0x7f296801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:42:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.374+0000 7f2979200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616c79d0 0x5641616e6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.374+0000 7f2979200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616c79d0 0x5641616e6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46918/0 (socket says 172.21.15.89:46918) 2024-09-10T08:42:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.374+0000 7f2979200640 1 -- 172.21.15.89:0/1665084675 learned_addr learned my addr 172.21.15.89:0/1665084675 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.374+0000 7f297a600640 1 --2- 172.21.15.89:0/1665084675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616cca60 0x5641616e7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.378+0000 7f2979c00640 1 --2- 172.21.15.89:0/1665084675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641616c7090 0x5641616e6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.379+0000 7f297a600640 1 -- 172.21.15.89:0/1665084675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641616c7090 msgr2=0x5641616e6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.379+0000 7f297a600640 1 --2- 172.21.15.89:0/1665084675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641616c7090 0x5641616e6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.379+0000 7f297a600640 1 -- 172.21.15.89:0/1665084675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616c79d0 msgr2=0x5641616e6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.379+0000 7f297a600640 1 --2- 172.21.15.89:0/1665084675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616c79d0 0x5641616e6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.379+0000 7f297a600640 1 -- 172.21.15.89:0/1665084675 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641616e83c0 con 0x5641616cca60 2024-09-10T08:42:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.380+0000 7f297a600640 1 --2- 172.21.15.89:0/1665084675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616cca60 0x5641616e7db0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f296c099330 tx=0x7f296c09c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9449af1c921bd3f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.380+0000 7f295b400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:42:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.381+0000 7f295b400640 1 -- 172.21.15.89:0/1665084675 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c01f040 con 0x5641616cca60 2024-09-10T08:42:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.381+0000 7f295b400640 1 -- 172.21.15.89:0/1665084675 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f296c09dbe0 con 0x5641616cca60 2024-09-10T08:42:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.381+0000 7f297f5bd740 1 -- 172.21.15.89:0/1665084675 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2960003ad0 con 0x5641616cca60 2024-09-10T08:42:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.382+0000 7f295b400640 1 -- 172.21.15.89:0/1665084675 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f296c0a0c60 con 0x5641616cca60 2024-09-10T08:42:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.382+0000 7f297f5bd740 10 client.25871 fetch_fsmap learned FSMap version 28 2024-09-10T08:42:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.382+0000 7f297f5bd740 1 -- 172.21.15.89:0/1665084675 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5641616e97a0 con 0x5641616cca60 2024-09-10T08:42:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.384+0000 7f295b400640 1 -- 172.21.15.89:0/1665084675 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2960003ad0 con 0x5641616cca60 2024-09-10T08:42:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.384+0000 7f295b400640 1 -- 172.21.15.89:0/1665084675 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f296c00f040 con 0x5641616cca60 2024-09-10T08:42:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.384+0000 7f297f5bd740 10 client.25871 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:42:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.384+0000 7f297f5bd740 10 client.25871 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:42:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.384+0000 7f297f5bd740 20 client.25871 populate_metadata read hostname 'smithi089' 2024-09-10T08:42:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.384+0000 7f297f5bd740 1 --2- 172.21.15.89:0/1665084675 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x564161758b50 0x56416175af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.384+0000 7f297f5bd740 4 client.25871 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:42:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.384+0000 7f297f5bd740 1 -- 172.21.15.89:0/1665084675 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564161633b80 con 0x564161758b50 2024-09-10T08:42:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.385+0000 7f2979c00640 1 --2- 172.21.15.89:0/1665084675 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x564161758b50 0x56416175af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.390+0000 7f295b400640 10 client.25871 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:04.390+0000 7f2979c00640 1 --2- 172.21.15.89:0/1665084675 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x564161758b50 0x56416175af70 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ec5485e791100af server_cookie=d9c3ccf2247cc576 in_seq=0 out_seq=0 2024-09-10T08:42:05.057 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:04 smithi089 ceph-mon[30757]: pgmap v457: 129 pgs: 129 active+clean; 57 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 329 op/s 2024-09-10T08:42:05.057 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:04 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:42:05.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:04 smithi149 ceph-mon[36167]: pgmap v457: 129 pgs: 129 active+clean; 57 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 329 op/s 2024-09-10T08:42:05.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:04 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:42:05.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:04 smithi111 ceph-mon[35220]: pgmap v457: 129 pgs: 129 active+clean; 57 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 329 op/s 2024-09-10T08:42:05.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:04 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:42:05.576 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:42:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.603+0000 7f295b400640 1 -- 172.21.15.89:0/1665084675 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564161633b80 con 0x564161758b50 2024-09-10T08:42:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.603+0000 7f295b400640 10 client.25871 handle_command_reply: tid=0 2024-09-10T08:42:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.604+0000 7f297f5bd740 1 client.25871 shutdown 2024-09-10T08:42:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.605+0000 7f297f5bd740 1 -- 172.21.15.89:0/1665084675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616cca60 msgr2=0x5641616e7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.605+0000 7f297f5bd740 1 --2- 172.21.15.89:0/1665084675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616cca60 0x5641616e7db0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f296c099330 tx=0x7f296c09c560 comp rx=0 tx=0).stop 2024-09-10T08:42:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.605+0000 7f297f5bd740 1 -- 172.21.15.89:0/1665084675 shutdown_connections 2024-09-10T08:42:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.606+0000 7f297f5bd740 1 --2- 172.21.15.89:0/1665084675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5641616cca60 0x5641616e7db0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.606+0000 7f297f5bd740 1 --2- 172.21.15.89:0/1665084675 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x564161758b50 0x56416175af70 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.606+0000 7f297f5bd740 1 --2- 172.21.15.89:0/1665084675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5641616c79d0 0x5641616e6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.606+0000 7f297f5bd740 1 --2- 172.21.15.89:0/1665084675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641616c7090 0x5641616e6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.606+0000 7f297f5bd740 1 -- 172.21.15.89:0/1665084675 >> 172.21.15.89:0/1665084675 conn(0x5641616c2700 msgr2=0x5641616c1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.606+0000 7f295b400640 0 client.25871 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.606+0000 7f297f5bd740 1 -- 172.21.15.89:0/1665084675 shutdown_connections 2024-09-10T08:42:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.606+0000 7f297f5bd740 1 -- 172.21.15.89:0/1665084675 wait complete. 2024-09-10T08:42:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.606+0000 7f297f5bd740 20 client.25871 trim_cache size 0 max 16384 2024-09-10T08:42:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:42:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:42:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:42:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:42:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.609+0000 7f297c600640 1 -- 172.21.15.89:0/668676287 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f295c0a2d30 msgr2=0x7f295c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.609+0000 7f297c600640 1 --2- 172.21.15.89:0/668676287 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f295c0a2d30 0x7f295c0a51f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f296002aa10 tx=0x7f29600315b0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.609+0000 7f297c600640 1 -- 172.21.15.89:0/668676287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29741165d0 msgr2=0x7f2974116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.609+0000 7f297c600640 1 --2- 172.21.15.89:0/668676287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29741165d0 0x7f2974116a80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f296c005fd0 tx=0x7f296c00edd0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.610+0000 7f297c600640 1 -- 172.21.15.89:0/668676287 shutdown_connections 2024-09-10T08:42:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.610+0000 7f297c600640 1 --2- 172.21.15.89:0/668676287 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f295c0a2d30 0x7f295c0a51f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.610+0000 7f297c600640 1 --2- 172.21.15.89:0/668676287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29741165d0 0x7f2974116a80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.610+0000 7f297c600640 1 --2- 172.21.15.89:0/668676287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29741078b0 0x7f2974111570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.610+0000 7f297c600640 1 --2- 172.21.15.89:0/668676287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2974104f80 0x7f2974111030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.610+0000 7f297c600640 1 -- 172.21.15.89:0/668676287 >> 172.21.15.89:0/668676287 conn(0x7f29740fc1b0 msgr2=0x7f2974102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.611+0000 7f297c600640 1 -- 172.21.15.89:0/668676287 shutdown_connections 2024-09-10T08:42:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:06.611+0000 7f297c600640 1 -- 172.21.15.89:0/668676287 wait complete. 2024-09-10T08:42:06.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.639+0000 7f0443e00640 1 -- 172.21.15.89:0/2861748311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f043c004cb0 msgr2=0x7f043c005110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:06.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.639+0000 7f0443e00640 1 --2- 172.21.15.89:0/2861748311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f043c004cb0 0x7f043c005110 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f043400fb60 tx=0x7f04340328b0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.639+0000 7f0443e00640 1 -- 172.21.15.89:0/2861748311 shutdown_connections 2024-09-10T08:42:06.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.639+0000 7f0443e00640 1 --2- 172.21.15.89:0/2861748311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f043c005650 0x7f043c0a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.639+0000 7f0443e00640 1 --2- 172.21.15.89:0/2861748311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f043c004cb0 0x7f043c005110 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.639+0000 7f0443e00640 1 --2- 172.21.15.89:0/2861748311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f043c09b600 0x7f043c09b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.639+0000 7f0443e00640 1 -- 172.21.15.89:0/2861748311 >> 172.21.15.89:0/2861748311 conn(0x7f043c00a1f0 msgr2=0x7f043c00a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:06.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.641+0000 7f0443e00640 1 -- 172.21.15.89:0/2861748311 shutdown_connections 2024-09-10T08:42:06.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.641+0000 7f0443e00640 1 -- 172.21.15.89:0/2861748311 wait complete. 2024-09-10T08:42:06.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.642+0000 7f0443e00640 1 Processor -- start 2024-09-10T08:42:06.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.645+0000 7f0443e00640 1 -- start start 2024-09-10T08:42:06.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.649+0000 7f0443e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f043c004cb0 0x7f043c140860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:06.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.649+0000 7f0443e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f043c005650 0x7f043c140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:06.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.649+0000 7f0442a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f043c004cb0 0x7f043c140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:06.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.649+0000 7f0442a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f043c004cb0 0x7f043c140860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:37706/0 (socket says 172.21.15.89:37706) 2024-09-10T08:42:06.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.649+0000 7f0442a00640 1 -- 172.21.15.89:0/2663239048 learned_addr learned my addr 172.21.15.89:0/2663239048 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:06.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.649+0000 7f0443e00640 1 --2- 172.21.15.89:0/2663239048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f043c09b600 0x7f043c13aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:06.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.649+0000 7f0443e00640 1 -- 172.21.15.89:0/2663239048 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f043c0b4810 con 0x7f043c005650 2024-09-10T08:42:06.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.649+0000 7f0443e00640 1 -- 172.21.15.89:0/2663239048 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f043c0b4690 con 0x7f043c004cb0 2024-09-10T08:42:06.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.649+0000 7f0443e00640 1 -- 172.21.15.89:0/2663239048 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f043c0b4990 con 0x7f043c09b600 2024-09-10T08:42:06.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.650+0000 7f0443400640 1 --2- 172.21.15.89:0/2663239048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f043c09b600 0x7f043c13aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:06.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.650+0000 7f0443400640 1 -- 172.21.15.89:0/2663239048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f043c004cb0 msgr2=0x7f043c140860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:06.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.650+0000 7f0443400640 1 --2- 172.21.15.89:0/2663239048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f043c004cb0 0x7f043c140860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.650+0000 7f0443400640 1 -- 172.21.15.89:0/2663239048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f043c005650 msgr2=0x7f043c140da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:06.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.650+0000 7f0443400640 1 --2- 172.21.15.89:0/2663239048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f043c005650 0x7f043c140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.651+0000 7f0443400640 1 -- 172.21.15.89:0/2663239048 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043c13b150 con 0x7f043c09b600 2024-09-10T08:42:06.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.652+0000 7f0443400640 1 --2- 172.21.15.89:0/2663239048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f043c09b600 0x7f043c13aa20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0430009fd0 tx=0x7f043000eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:06.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.652+0000 7f043b400640 1 -- 172.21.15.89:0/2663239048 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430004220 con 0x7f043c09b600 2024-09-10T08:42:06.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.652+0000 7f0443e00640 1 -- 172.21.15.89:0/2663239048 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f043c13b440 con 0x7f043c09b600 2024-09-10T08:42:06.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.653+0000 7f0443e00640 1 -- 172.21.15.89:0/2663239048 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f043c147730 con 0x7f043c09b600 2024-09-10T08:42:06.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.653+0000 7f043b400640 1 -- 172.21.15.89:0/2663239048 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f043000cd90 con 0x7f043c09b600 2024-09-10T08:42:06.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.653+0000 7f043b400640 1 -- 172.21.15.89:0/2663239048 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430004a00 con 0x7f043c09b600 2024-09-10T08:42:06.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.659+0000 7f044c2d9740 1 Processor -- start 2024-09-10T08:42:06.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.667+0000 7f043b400640 1 -- 172.21.15.89:0/2663239048 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0430004ba0 con 0x7f043c09b600 2024-09-10T08:42:06.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.671+0000 7f044c2d9740 1 -- start start 2024-09-10T08:42:06.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.672+0000 7f044c2d9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062015e70 0x55d062016250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:06.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.672+0000 7f044c2d9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062016790 0x55d062020b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:06.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.672+0000 7f044c2d9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d062021050 0x55d062023440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:06.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.672+0000 7f044c2d9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d061f7c3e0 con 0x55d062015e70 2024-09-10T08:42:06.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.672+0000 7f044c2d9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d061f23d80 con 0x55d062021050 2024-09-10T08:42:06.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.672+0000 7f044c2d9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d061f783a0 con 0x55d062016790 2024-09-10T08:42:06.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.673+0000 7f043b400640 1 --2- 172.21.15.89:0/2663239048 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f04280a2ce0 0x7f04280a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:06.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.673+0000 7f043b400640 1 -- 172.21.15.89:0/2663239048 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f04300cf230 con 0x7f043c09b600 2024-09-10T08:42:06.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.674+0000 7f0442000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062015e70 0x55d062016250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:06.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.674+0000 7f0442a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062016790 0x55d062020b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:06.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.675+0000 7f0442a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062016790 0x55d062020b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:60270/0 (socket says 172.21.15.89:60270) 2024-09-10T08:42:06.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.675+0000 7f0442a00640 1 -- 172.21.15.89:0/4050719194 learned_addr learned my addr 172.21.15.89:0/4050719194 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:06.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.674+0000 7f0443400640 1 --2- 172.21.15.89:0/2663239048 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f04280a2ce0 0x7f04280a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:06.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.676+0000 7f0442a00640 1 -- 172.21.15.89:0/4050719194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d062021050 msgr2=0x55d062023440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:42:06.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.676+0000 7f0442a00640 1 --2- 172.21.15.89:0/4050719194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d062021050 0x55d062023440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.676+0000 7f0442a00640 1 -- 172.21.15.89:0/4050719194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062015e70 msgr2=0x55d062016250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:06.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.676+0000 7f0442a00640 1 --2- 172.21.15.89:0/4050719194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062015e70 0x55d062016250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.676+0000 7f0442a00640 1 -- 172.21.15.89:0/4050719194 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d061f42ae0 con 0x55d062016790 2024-09-10T08:42:06.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.677+0000 7f0442000640 1 --2- 172.21.15.89:0/4050719194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062015e70 0x55d062016250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:42:06.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.677+0000 7f0442a00640 1 --2- 172.21.15.89:0/4050719194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062016790 0x55d062020b10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0444067c70 tx=0x7f04440691c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=299cc8d0c2156bc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:06.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.677+0000 7f0443400640 1 --2- 172.21.15.89:0/2663239048 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f04280a2ce0 0x7f04280a51a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f04300a3da0 tx=0x7f04300cf480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:06.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.678+0000 7f042fe00640 1 -- 172.21.15.89:0/4050719194 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444068800 con 0x55d062016790 2024-09-10T08:42:06.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.678+0000 7f042fe00640 1 -- 172.21.15.89:0/4050719194 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f044406cb90 con 0x55d062016790 2024-09-10T08:42:06.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.678+0000 7f042fe00640 1 -- 172.21.15.89:0/4050719194 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444068800 con 0x55d062016790 2024-09-10T08:42:06.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.679+0000 7f044c2d9740 1 -- 172.21.15.89:0/4050719194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062016790 msgr2=0x55d062020b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:06.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.679+0000 7f044c2d9740 1 --2- 172.21.15.89:0/4050719194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062016790 0x55d062020b10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0444067c70 tx=0x7f04440691c0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.680+0000 7f044c2d9740 1 -- 172.21.15.89:0/4050719194 shutdown_connections 2024-09-10T08:42:06.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.680+0000 7f044c2d9740 1 --2- 172.21.15.89:0/4050719194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d062021050 0x55d062023440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.680+0000 7f044c2d9740 1 --2- 172.21.15.89:0/4050719194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062016790 0x55d062020b10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.680+0000 7f044c2d9740 1 --2- 172.21.15.89:0/4050719194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062015e70 0x55d062016250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.680+0000 7f044c2d9740 1 -- 172.21.15.89:0/4050719194 >> 172.21.15.89:0/4050719194 conn(0x55d062012400 msgr2=0x55d0620127e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:06.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.681+0000 7f044c2d9740 1 -- 172.21.15.89:0/4050719194 shutdown_connections 2024-09-10T08:42:06.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.682+0000 7f044c2d9740 1 -- 172.21.15.89:0/4050719194 wait complete. 2024-09-10T08:42:06.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.683+0000 7f044c2d9740 1 -- start start 2024-09-10T08:42:06.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.683+0000 7f044c2d9740 1 Processor -- start 2024-09-10T08:42:06.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.685+0000 7f044c2d9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062015e70 0x55d06201dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:06.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.685+0000 7f044c2d9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062016790 0x55d06201e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:06.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.686+0000 7f044c2d9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d062021050 0x55d06201ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:06.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.686+0000 7f044c2d9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d062023c20 con 0x55d062015e70 2024-09-10T08:42:06.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.686+0000 7f0442a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062016790 0x55d06201e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:06.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.686+0000 7f0442a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062016790 0x55d06201e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:60286/0 (socket says 172.21.15.89:60286) 2024-09-10T08:42:06.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.686+0000 7f0442a00640 1 -- 172.21.15.89:0/1574639656 learned_addr learned my addr 172.21.15.89:0/1574639656 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:06.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.686+0000 7f044c2d9740 1 -- 172.21.15.89:0/1574639656 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d061f370f0 con 0x55d062021050 2024-09-10T08:42:06.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.686+0000 7f044c2d9740 1 -- 172.21.15.89:0/1574639656 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d062023da0 con 0x55d062016790 2024-09-10T08:42:06.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.687+0000 7f0442000640 1 --2- 172.21.15.89:0/1574639656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062015e70 0x55d06201dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:06.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.687+0000 7f044c2d9740 10 client.0 _ll_register_callbacks cb 0x55d061f1ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:42:06.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.687+0000 7f0442a00640 1 -- 172.21.15.89:0/1574639656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d062021050 msgr2=0x55d06201ea50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:42:06.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.687+0000 7f0442a00640 1 --2- 172.21.15.89:0/1574639656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d062021050 0x55d06201ea50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.687+0000 7f0442a00640 1 -- 172.21.15.89:0/1574639656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062015e70 msgr2=0x55d06201dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:06.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.687+0000 7f0442a00640 1 --2- 172.21.15.89:0/1574639656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062015e70 0x55d06201dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.687+0000 7f0442a00640 1 -- 172.21.15.89:0/1574639656 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d061ced3c0 con 0x55d062016790 2024-09-10T08:42:06.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.687+0000 7f0442000640 1 --2- 172.21.15.89:0/1574639656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062015e70 0x55d06201dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:42:06.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.688+0000 7f0442a00640 1 --2- 172.21.15.89:0/1574639656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062016790 0x55d06201e510 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f044402dd50 tx=0x7f0444078650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad84d10e1f615739 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:06.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.688+0000 7f042cc00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:42:06.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.688+0000 7f042cc00640 1 -- 172.21.15.89:0/1574639656 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444087020 con 0x55d062016790 2024-09-10T08:42:06.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.688+0000 7f042cc00640 1 -- 172.21.15.89:0/1574639656 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f0444074040 con 0x55d062016790 2024-09-10T08:42:06.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.688+0000 7f044c2d9740 1 -- 172.21.15.89:0/1574639656 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f044406cb90 con 0x55d062016790 2024-09-10T08:42:06.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.689+0000 7f042cc00640 1 -- 172.21.15.89:0/1574639656 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0444078e60 con 0x55d062016790 2024-09-10T08:42:06.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.689+0000 7f044c2d9740 10 client.25816 fetch_fsmap learned FSMap version 28 2024-09-10T08:42:06.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.689+0000 7f044c2d9740 1 -- 172.21.15.89:0/1574639656 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d06202fd20 con 0x55d062016790 2024-09-10T08:42:06.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.689+0000 7f042cc00640 1 -- 172.21.15.89:0/1574639656 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f044406cb90 con 0x55d062016790 2024-09-10T08:42:06.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.689+0000 7f042cc00640 1 -- 172.21.15.89:0/1574639656 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f044408b3a0 con 0x55d062016790 2024-09-10T08:42:06.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.689+0000 7f044c2d9740 10 client.25816 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:42:06.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.689+0000 7f044c2d9740 10 client.25816 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:42:06.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.689+0000 7f044c2d9740 20 client.25816 populate_metadata read hostname 'smithi089' 2024-09-10T08:42:06.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.689+0000 7f044c2d9740 1 --2- 172.21.15.89:0/1574639656 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55d0620310d0 0x55d0620334c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:06.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.689+0000 7f044c2d9740 4 client.25816 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:42:06.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.689+0000 7f044c2d9740 1 -- 172.21.15.89:0/1574639656 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d061f89950 con 0x55d0620310d0 2024-09-10T08:42:06.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.691+0000 7f0442000640 1 --2- 172.21.15.89:0/1574639656 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55d0620310d0 0x55d0620334c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:06.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.694+0000 7f0442000640 1 --2- 172.21.15.89:0/1574639656 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55d0620310d0 0x55d0620334c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82eeff2625abda5 server_cookie=ff7b8e9813b319c8 in_seq=0 out_seq=0 2024-09-10T08:42:06.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.694+0000 7f042cc00640 10 client.25816 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:42:06.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.695+0000 7f042cc00640 1 -- 172.21.15.89:0/1574639656 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d061f89950 con 0x55d0620310d0 2024-09-10T08:42:06.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.695+0000 7f042cc00640 10 client.25816 handle_command_reply: tid=0 2024-09-10T08:42:06.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.695+0000 7f044c2d9740 1 client.25816 shutdown 2024-09-10T08:42:06.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.697+0000 7f044c2d9740 1 -- 172.21.15.89:0/1574639656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062016790 msgr2=0x55d06201e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:06.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.697+0000 7f044c2d9740 1 --2- 172.21.15.89:0/1574639656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062016790 0x55d06201e510 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f044402dd50 tx=0x7f0444078650 comp rx=0 tx=0).stop 2024-09-10T08:42:06.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.698+0000 7f044c2d9740 1 -- 172.21.15.89:0/1574639656 shutdown_connections 2024-09-10T08:42:06.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.698+0000 7f044c2d9740 1 --2- 172.21.15.89:0/1574639656 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55d0620310d0 0x55d0620334c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.698+0000 7f044c2d9740 1 --2- 172.21.15.89:0/1574639656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d062021050 0x55d06201ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.698+0000 7f044c2d9740 1 --2- 172.21.15.89:0/1574639656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062016790 0x55d06201e510 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.698+0000 7f044c2d9740 1 --2- 172.21.15.89:0/1574639656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062015e70 0x55d06201dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.698+0000 7f044c2d9740 1 -- 172.21.15.89:0/1574639656 >> 172.21.15.89:0/1574639656 conn(0x55d062012400 msgr2=0x55d062018130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:06.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.698+0000 7f042cc00640 0 client.25816 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:42:06.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.699+0000 7f044c2d9740 1 -- 172.21.15.89:0/1574639656 shutdown_connections 2024-09-10T08:42:06.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.699+0000 7f044c2d9740 1 -- 172.21.15.89:0/1574639656 wait complete. 2024-09-10T08:42:06.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.699+0000 7f044c2d9740 20 client.25816 trim_cache size 0 max 16384 2024-09-10T08:42:06.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.737+0000 7f044c2d9740 1 Processor -- start 2024-09-10T08:42:06.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.737+0000 7f044c2d9740 1 -- start start 2024-09-10T08:42:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.737+0000 7f044c2d9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062015e70 0x55d062019330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.737+0000 7f044c2d9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062016790 0x55d0620188d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.737+0000 7f044c2d9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d062021050 0x55d06201e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.737+0000 7f044c2d9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0424012510 con 0x55d062015e70 2024-09-10T08:42:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.737+0000 7f044c2d9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f042400fde0 con 0x55d062021050 2024-09-10T08:42:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.737+0000 7f044c2d9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f042400dc30 con 0x55d062016790 2024-09-10T08:42:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.737+0000 7f0442000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062015e70 0x55d062019330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:06.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.738+0000 7f0442000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062015e70 0x55d062019330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46962/0 (socket says 172.21.15.89:46962) 2024-09-10T08:42:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.738+0000 7f0442000640 1 -- 172.21.15.89:0/3125146863 learned_addr learned my addr 172.21.15.89:0/3125146863 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.738+0000 7f0442000640 1 --2- 172.21.15.89:0/3125146863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d062021050 0x55d06201e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.738+0000 7f0442000640 1 -- 172.21.15.89:0/3125146863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062016790 msgr2=0x55d0620188d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.738+0000 7f0442000640 1 --2- 172.21.15.89:0/3125146863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062016790 0x55d0620188d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.738+0000 7f0442000640 1 -- 172.21.15.89:0/3125146863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d062021050 msgr2=0x55d06201e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.738+0000 7f0442000640 1 --2- 172.21.15.89:0/3125146863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d062021050 0x55d06201e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.738+0000 7f0442000640 1 -- 172.21.15.89:0/3125146863 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0620208d0 con 0x55d062015e70 2024-09-10T08:42:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.739+0000 7f0442000640 1 --2- 172.21.15.89:0/3125146863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062015e70 0x55d062019330 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0434009250 tx=0x7f043400f860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5c29f0ba1ce1ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.739+0000 7f042f400640 1 -- 172.21.15.89:0/3125146863 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434004040 con 0x55d062015e70 2024-09-10T08:42:06.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.739+0000 7f042f400640 1 -- 172.21.15.89:0/3125146863 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f043403e040 con 0x55d062015e70 2024-09-10T08:42:06.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.739+0000 7f042f400640 1 -- 172.21.15.89:0/3125146863 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434004920 con 0x55d062015e70 2024-09-10T08:42:06.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.740+0000 7f044c2d9740 1 -- 172.21.15.89:0/3125146863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062015e70 msgr2=0x55d062019330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:06.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.740+0000 7f044c2d9740 1 --2- 172.21.15.89:0/3125146863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062015e70 0x55d062019330 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0434009250 tx=0x7f043400f860 comp rx=0 tx=0).stop 2024-09-10T08:42:06.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.740+0000 7f044c2d9740 1 -- 172.21.15.89:0/3125146863 shutdown_connections 2024-09-10T08:42:06.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.740+0000 7f044c2d9740 1 --2- 172.21.15.89:0/3125146863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d062021050 0x55d06201e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.740+0000 7f044c2d9740 1 --2- 172.21.15.89:0/3125146863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062016790 0x55d0620188d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.740+0000 7f044c2d9740 1 --2- 172.21.15.89:0/3125146863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062015e70 0x55d062019330 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.740+0000 7f044c2d9740 1 -- 172.21.15.89:0/3125146863 >> 172.21.15.89:0/3125146863 conn(0x55d062012400 msgr2=0x55d062020050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:06.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.741+0000 7f044c2d9740 1 -- 172.21.15.89:0/3125146863 shutdown_connections 2024-09-10T08:42:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.741+0000 7f044c2d9740 1 -- 172.21.15.89:0/3125146863 wait complete. 2024-09-10T08:42:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.742+0000 7f044c2d9740 1 -- start start 2024-09-10T08:42:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.742+0000 7f044c2d9740 1 Processor -- start 2024-09-10T08:42:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.742+0000 7f044c2d9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062015e70 0x55d062024770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.743+0000 7f044c2d9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062016790 0x55d062024cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.743+0000 7f044c2d9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d062021050 0x55d062093cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.743+0000 7f044c2d9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f042400e2a0 con 0x55d062016790 2024-09-10T08:42:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.743+0000 7f044c2d9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f042400f710 con 0x55d062021050 2024-09-10T08:42:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.743+0000 7f044c2d9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f042400e970 con 0x55d062015e70 2024-09-10T08:42:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.743+0000 7f044c2d9740 10 client.0 _ll_register_callbacks cb 0x7f0430005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:42:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.743+0000 7f0442000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062015e70 0x55d062024770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:06.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.743+0000 7f0442000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062015e70 0x55d062024770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:60302/0 (socket says 172.21.15.89:60302) 2024-09-10T08:42:06.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.743+0000 7f0442000640 1 -- 172.21.15.89:0/2270097233 learned_addr learned my addr 172.21.15.89:0/2270097233 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:06.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.743+0000 7f0442000640 1 --2- 172.21.15.89:0/2270097233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d062021050 0x55d062093cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:06.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.743+0000 7f0442a00640 1 --2- 172.21.15.89:0/2270097233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062016790 0x55d062024cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:06.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.743+0000 7f0442000640 1 -- 172.21.15.89:0/2270097233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d062021050 msgr2=0x55d062093cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:06.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.743+0000 7f0442000640 1 --2- 172.21.15.89:0/2270097233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d062021050 0x55d062093cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.743+0000 7f0442000640 1 -- 172.21.15.89:0/2270097233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062016790 msgr2=0x55d062024cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:06.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.743+0000 7f0442000640 1 --2- 172.21.15.89:0/2270097233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062016790 0x55d062024cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.743+0000 7f0442000640 1 -- 172.21.15.89:0/2270097233 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d062094370 con 0x55d062015e70 2024-09-10T08:42:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.743+0000 7f0442a00640 1 --2- 172.21.15.89:0/2270097233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062016790 0x55d062024cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:42:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.744+0000 7f0442000640 1 --2- 172.21.15.89:0/2270097233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062015e70 0x55d062024770 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0434004640 tx=0x7f043404d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3290e776c486c12 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.744+0000 7f042cc00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:42:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.744+0000 7f042cc00640 1 -- 172.21.15.89:0/2270097233 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434039640 con 0x55d062015e70 2024-09-10T08:42:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.744+0000 7f044c2d9740 1 -- 172.21.15.89:0/2270097233 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f043403e040 con 0x55d062015e70 2024-09-10T08:42:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.744+0000 7f042cc00640 1 -- 172.21.15.89:0/2270097233 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f043405a430 con 0x55d062015e70 2024-09-10T08:42:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.745+0000 7f042cc00640 1 -- 172.21.15.89:0/2270097233 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f04340397e0 con 0x55d062015e70 2024-09-10T08:42:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.745+0000 7f044c2d9740 10 client.25819 fetch_fsmap learned FSMap version 28 2024-09-10T08:42:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.745+0000 7f044c2d9740 1 -- 172.21.15.89:0/2270097233 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d06209b720 con 0x55d062015e70 2024-09-10T08:42:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.745+0000 7f042cc00640 1 -- 172.21.15.89:0/2270097233 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f043403e040 con 0x55d062015e70 2024-09-10T08:42:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.745+0000 7f042cc00640 1 -- 172.21.15.89:0/2270097233 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f043405abf0 con 0x55d062015e70 2024-09-10T08:42:06.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.745+0000 7f044c2d9740 10 client.25819 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:42:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.745+0000 7f044c2d9740 10 client.25819 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:42:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.745+0000 7f044c2d9740 20 client.25819 populate_metadata read hostname 'smithi089' 2024-09-10T08:42:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.745+0000 7f044c2d9740 1 --2- 172.21.15.89:0/2270097233 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55d06209bda0 0x55d06209e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.745+0000 7f044c2d9740 4 client.25819 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:42:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.745+0000 7f044c2d9740 1 -- 172.21.15.89:0/2270097233 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d061f83e60 con 0x55d06209bda0 2024-09-10T08:42:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.746+0000 7f0442a00640 1 --2- 172.21.15.89:0/2270097233 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55d06209bda0 0x55d06209e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:06.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.749+0000 7f0442a00640 1 --2- 172.21.15.89:0/2270097233 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55d06209bda0 0x55d06209e1c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c907eb3da9add6f9 server_cookie=1c0c3d92df0c63ea in_seq=0 out_seq=0 2024-09-10T08:42:06.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.749+0000 7f042cc00640 10 client.25819 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:42:06.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.750+0000 7f042cc00640 1 -- 172.21.15.89:0/2270097233 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d061f83e60 con 0x55d06209bda0 2024-09-10T08:42:06.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.750+0000 7f042cc00640 10 client.25819 handle_command_reply: tid=0 2024-09-10T08:42:06.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.750+0000 7f044c2d9740 1 client.25819 shutdown 2024-09-10T08:42:06.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.751+0000 7f044c2d9740 1 -- 172.21.15.89:0/2270097233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062015e70 msgr2=0x55d062024770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:06.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.751+0000 7f044c2d9740 1 --2- 172.21.15.89:0/2270097233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062015e70 0x55d062024770 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0434004640 tx=0x7f043404d040 comp rx=0 tx=0).stop 2024-09-10T08:42:06.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.751+0000 7f044c2d9740 1 -- 172.21.15.89:0/2270097233 shutdown_connections 2024-09-10T08:42:06.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.751+0000 7f044c2d9740 1 --2- 172.21.15.89:0/2270097233 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x55d06209bda0 0x55d06209e1c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.751+0000 7f044c2d9740 1 --2- 172.21.15.89:0/2270097233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d062021050 0x55d062093cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.751+0000 7f044c2d9740 1 --2- 172.21.15.89:0/2270097233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d062016790 0x55d062024cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.751+0000 7f044c2d9740 1 --2- 172.21.15.89:0/2270097233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d062015e70 0x55d062024770 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.751+0000 7f044c2d9740 1 -- 172.21.15.89:0/2270097233 >> 172.21.15.89:0/2270097233 conn(0x55d062012400 msgr2=0x55d062010d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:06.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.751+0000 7f042cc00640 0 client.25819 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:42:06.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.751+0000 7f044c2d9740 1 -- 172.21.15.89:0/2270097233 shutdown_connections 2024-09-10T08:42:06.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.752+0000 7f044c2d9740 1 -- 172.21.15.89:0/2270097233 wait complete. 2024-09-10T08:42:06.754 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:42:06.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.753+0000 7f044c2d9740 20 client.25819 trim_cache size 0 max 16384 2024-09-10T08:42:06.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.754+0000 7f0443e00640 1 -- 172.21.15.89:0/2663239048 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f04280a2ce0 msgr2=0x7f04280a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:06.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.754+0000 7f0443e00640 1 --2- 172.21.15.89:0/2663239048 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f04280a2ce0 0x7f04280a51a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f04300a3da0 tx=0x7f04300cf480 comp rx=0 tx=0).stop 2024-09-10T08:42:06.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.754+0000 7f0443e00640 1 -- 172.21.15.89:0/2663239048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f043c09b600 msgr2=0x7f043c13aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:06.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.754+0000 7f0443e00640 1 --2- 172.21.15.89:0/2663239048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f043c09b600 0x7f043c13aa20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0430009fd0 tx=0x7f043000eba0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.754+0000 7f0443e00640 1 -- 172.21.15.89:0/2663239048 shutdown_connections 2024-09-10T08:42:06.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.754+0000 7f0443e00640 1 --2- 172.21.15.89:0/2663239048 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f04280a2ce0 0x7f04280a51a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.754+0000 7f0443e00640 1 --2- 172.21.15.89:0/2663239048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f043c09b600 0x7f043c13aa20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.754+0000 7f0443e00640 1 --2- 172.21.15.89:0/2663239048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f043c005650 0x7f043c140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.754+0000 7f0443e00640 1 --2- 172.21.15.89:0/2663239048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f043c004cb0 0x7f043c140860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:06.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.754+0000 7f0443e00640 1 -- 172.21.15.89:0/2663239048 >> 172.21.15.89:0/2663239048 conn(0x7f043c00a1f0 msgr2=0x7f043c093410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:06.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.755+0000 7f0443e00640 1 -- 172.21.15.89:0/2663239048 shutdown_connections 2024-09-10T08:42:06.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:06.755+0000 7f0443e00640 1 -- 172.21.15.89:0/2663239048 wait complete. 2024-09-10T08:42:07.027 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:06 smithi089 ceph-mon[30757]: pgmap v458: 129 pgs: 129 active+clean; 59 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 541 op/s 2024-09-10T08:42:07.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:06 smithi149 ceph-mon[36167]: pgmap v458: 129 pgs: 129 active+clean; 59 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 541 op/s 2024-09-10T08:42:07.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:06 smithi111 ceph-mon[35220]: pgmap v458: 129 pgs: 129 active+clean; 59 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 541 op/s 2024-09-10T08:42:07.810 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:42:07.932 INFO:tasks.workunit.client.0.smithi089.stdout: 26 294129 600 195170 377 79070 960 2024-09-10T08:42:08.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:07 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:08.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:07 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:42:08.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:07 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:42:08.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:07 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:08.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:07 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:42:08.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:07 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:42:08.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:07 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:08.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:07 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:42:08.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:07 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:42:08.294 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:42:09.106 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:08 smithi089 ceph-mon[30757]: pgmap v459: 129 pgs: 129 active+clean; 63 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.9 MiB/s wr, 907 op/s 2024-09-10T08:42:09.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.161+0000 7fb9cb000640 1 -- 172.21.15.89:0/1498267678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9c410e260 msgr2=0x7fb9c4110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:09.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.161+0000 7fb9cb000640 1 --2- 172.21.15.89:0/1498267678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9c410e260 0x7fb9c4110650 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009ca0 tx=0x7fb9bc02a940 comp rx=0 tx=0).stop 2024-09-10T08:42:09.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.164+0000 7fb9cb000640 1 -- 172.21.15.89:0/1498267678 shutdown_connections 2024-09-10T08:42:09.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.164+0000 7fb9cb000640 1 --2- 172.21.15.89:0/1498267678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9c410e260 0x7fb9c4110650 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.164+0000 7fb9cb000640 1 --2- 172.21.15.89:0/1498267678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9c4074ee0 0x7fb9c410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.164+0000 7fb9cb000640 1 --2- 172.21.15.89:0/1498267678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c40745c0 0x7fb9c40749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.164+0000 7fb9cb000640 1 -- 172.21.15.89:0/1498267678 >> 172.21.15.89:0/1498267678 conn(0x7fb9c40fd740 msgr2=0x7fb9c40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:09.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.167+0000 7fb9cb000640 1 -- 172.21.15.89:0/1498267678 shutdown_connections 2024-09-10T08:42:09.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.167+0000 7fb9cb000640 1 -- 172.21.15.89:0/1498267678 wait complete. 2024-09-10T08:42:09.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.169+0000 7fb9cb000640 1 Processor -- start 2024-09-10T08:42:09.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.170+0000 7fb9cb000640 1 -- start start 2024-09-10T08:42:09.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.171+0000 7fb9cb000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9c40745c0 0x7fb9c419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:09.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.171+0000 7fb9cb000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9c4074ee0 0x7fb9c419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:09.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.171+0000 7fb9cb000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c410e260 0x7fb9c419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:09.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.171+0000 7fb9cb000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c4113f70 con 0x7fb9c410e260 2024-09-10T08:42:09.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.171+0000 7fb9cb000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c4113df0 con 0x7fb9c4074ee0 2024-09-10T08:42:09.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.171+0000 7fb9cb000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c41140f0 con 0x7fb9c40745c0 2024-09-10T08:42:09.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.171+0000 7fb9c9200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9c4074ee0 0x7fb9c419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:09.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.171+0000 7fb9c9c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9c40745c0 0x7fb9c419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:09.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.171+0000 7fb9c9200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9c4074ee0 0x7fb9c419a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:37732/0 (socket says 172.21.15.89:37732) 2024-09-10T08:42:09.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.171+0000 7fb9c9200640 1 -- 172.21.15.89:0/2701396469 learned_addr learned my addr 172.21.15.89:0/2701396469 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:09.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.172+0000 7fb9c9200640 1 -- 172.21.15.89:0/2701396469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9c40745c0 msgr2=0x7fb9c419a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:09.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.172+0000 7fb9c9200640 1 --2- 172.21.15.89:0/2701396469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9c40745c0 0x7fb9c419a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.172+0000 7fb9c9200640 1 -- 172.21.15.89:0/2701396469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c410e260 msgr2=0x7fb9c419ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:09.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.172+0000 7fb9c9200640 1 --2- 172.21.15.89:0/2701396469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c410e260 0x7fb9c419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.172+0000 7fb9c9200640 1 -- 172.21.15.89:0/2701396469 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c419f340 con 0x7fb9c4074ee0 2024-09-10T08:42:09.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.172+0000 7fb9c9200640 1 --2- 172.21.15.89:0/2701396469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9c4074ee0 0x7fb9c419a8d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400c920 tx=0x7fb9b400cd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:09.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.173+0000 7fb9c2000640 1 -- 172.21.15.89:0/2701396469 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4007bb0 con 0x7fb9c4074ee0 2024-09-10T08:42:09.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.173+0000 7fb9c2000640 1 -- 172.21.15.89:0/2701396469 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9b4007d50 con 0x7fb9c4074ee0 2024-09-10T08:42:09.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.173+0000 7fb9c2000640 1 -- 172.21.15.89:0/2701396469 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b40187e0 con 0x7fb9c4074ee0 2024-09-10T08:42:09.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.174+0000 7fb9cb000640 1 -- 172.21.15.89:0/2701396469 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c419f630 con 0x7fb9c4074ee0 2024-09-10T08:42:09.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.174+0000 7fb9cb000640 1 -- 172.21.15.89:0/2701396469 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c419fa90 con 0x7fb9c4074ee0 2024-09-10T08:42:09.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.186+0000 7fb9cf090740 1 Processor -- start 2024-09-10T08:42:09.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.186+0000 7fb9c2000640 1 -- 172.21.15.89:0/2701396469 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb9b400f450 con 0x7fb9c4074ee0 2024-09-10T08:42:09.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.187+0000 7fb9cf090740 1 -- start start 2024-09-10T08:42:09.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.187+0000 7fb9cf090740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d1fad60 0x55a49d1fb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:09.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.188+0000 7fb9cf090740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d1fb680 0x55a49d20e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:09.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.188+0000 7fb9cf090740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d20e830 0x55a49d210c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:09.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.188+0000 7fb9cf090740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a49d1613e0 con 0x55a49d20e830 2024-09-10T08:42:09.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.188+0000 7fb9cf090740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a49d108d80 con 0x55a49d1fb680 2024-09-10T08:42:09.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.188+0000 7fb9cf090740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a49d15d3a0 con 0x55a49d1fad60 2024-09-10T08:42:09.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.188+0000 7fb9c2000640 1 --2- 172.21.15.89:0/2701396469 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb9ac0a2cc0 0x7fb9ac0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.188+0000 7fb9ca600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d1fad60 0x55a49d1fb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.188+0000 7fb9c9c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d1fb680 0x55a49d20e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.188+0000 7fb9ca600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d1fad60 0x55a49d1fb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:60342/0 (socket says 172.21.15.89:60342) 2024-09-10T08:42:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.188+0000 7fb9c9c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d1fb680 0x55a49d20e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:37746/0 (socket says 172.21.15.89:37746) 2024-09-10T08:42:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.188+0000 7fb9ca600640 1 -- 172.21.15.89:0/3787130236 learned_addr learned my addr 172.21.15.89:0/3787130236 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.189+0000 7fb9c2000640 1 -- 172.21.15.89:0/2701396469 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb9b40cf370 con 0x7fb9c4074ee0 2024-09-10T08:42:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.189+0000 7fb9ca600640 1 --2- 172.21.15.89:0/2701396469 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb9ac0a2cc0 0x7fb9ac0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.189+0000 7fb9ca600640 1 -- 172.21.15.89:0/3787130236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d1fb680 msgr2=0x55a49d20e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.189+0000 7fb9ca600640 1 --2- 172.21.15.89:0/3787130236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d1fb680 0x55a49d20e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.189+0000 7fb9ca600640 1 -- 172.21.15.89:0/3787130236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d20e830 msgr2=0x55a49d210c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:42:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.189+0000 7fb9ca600640 1 --2- 172.21.15.89:0/3787130236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d20e830 0x55a49d210c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.189+0000 7fb9ca600640 1 -- 172.21.15.89:0/3787130236 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a49d127ae0 con 0x55a49d1fad60 2024-09-10T08:42:09.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.190+0000 7fb9ca600640 1 --2- 172.21.15.89:0/3787130236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d1fad60 0x55a49d1fb140 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009bf0 tx=0x7fb9bc02f580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d57be1e61a6f011 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:09.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.190+0000 7fb9b2a00640 1 -- 172.21.15.89:0/3787130236 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc05c070 con 0x55a49d1fad60 2024-09-10T08:42:09.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.190+0000 7fb9b2a00640 1 -- 172.21.15.89:0/3787130236 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9bc02fe10 con 0x55a49d1fad60 2024-09-10T08:42:09.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.190+0000 7fb9b2a00640 1 -- 172.21.15.89:0/3787130236 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc034040 con 0x55a49d1fad60 2024-09-10T08:42:09.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.194+0000 7fb9cf090740 1 -- 172.21.15.89:0/3787130236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d1fad60 msgr2=0x55a49d1fb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:09.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.194+0000 7fb9ca600640 1 --2- 172.21.15.89:0/2701396469 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb9ac0a2cc0 0x7fb9ac0a5180 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc059000 tx=0x7fb9bc004490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:09.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.194+0000 7fb9cf090740 1 --2- 172.21.15.89:0/3787130236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d1fad60 0x55a49d1fb140 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009bf0 tx=0x7fb9bc02f580 comp rx=0 tx=0).stop 2024-09-10T08:42:09.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.200+0000 7fb9cf090740 1 -- 172.21.15.89:0/3787130236 shutdown_connections 2024-09-10T08:42:09.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.200+0000 7fb9cf090740 1 --2- 172.21.15.89:0/3787130236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d20e830 0x55a49d210c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.200+0000 7fb9cf090740 1 --2- 172.21.15.89:0/3787130236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d1fb680 0x55a49d20e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.200+0000 7fb9cf090740 1 --2- 172.21.15.89:0/3787130236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d1fad60 0x55a49d1fb140 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.200+0000 7fb9cf090740 1 -- 172.21.15.89:0/3787130236 >> 172.21.15.89:0/3787130236 conn(0x55a49d1f73d0 msgr2=0x55a49d1f77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:09.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.200+0000 7fb9cf090740 1 -- 172.21.15.89:0/3787130236 shutdown_connections 2024-09-10T08:42:09.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.200+0000 7fb9cf090740 1 -- 172.21.15.89:0/3787130236 wait complete. 2024-09-10T08:42:09.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.201+0000 7fb9cf090740 1 -- start start 2024-09-10T08:42:09.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.201+0000 7fb9cf090740 1 Processor -- start 2024-09-10T08:42:09.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.202+0000 7fb9cf090740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d1fad60 0x55a49d20b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:09.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.202+0000 7fb9cf090740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d1fb680 0x55a49d20bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:09.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.202+0000 7fb9cf090740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d20e830 0x55a49d20c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:09.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.202+0000 7fb9cf090740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a49d2113a0 con 0x55a49d1fad60 2024-09-10T08:42:09.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.202+0000 7fb9cf090740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a49d11c0f0 con 0x55a49d1fb680 2024-09-10T08:42:09.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.202+0000 7fb9cf090740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a49d211520 con 0x55a49d20e830 2024-09-10T08:42:09.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.202+0000 7fb9cf090740 10 client.0 _ll_register_callbacks cb 0x55a49d103bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:42:09.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.204+0000 7fb9c9200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d1fb680 0x55a49d20bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:09.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.204+0000 7fb9c9200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d1fb680 0x55a49d20bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:37754/0 (socket says 172.21.15.89:37754) 2024-09-10T08:42:09.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.204+0000 7fb9c9200640 1 -- 172.21.15.89:0/1409010367 learned_addr learned my addr 172.21.15.89:0/1409010367 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:09.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.204+0000 7fb9ca600640 1 --2- 172.21.15.89:0/1409010367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d20e830 0x55a49d20c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:09.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.205+0000 7fb9c9c00640 1 --2- 172.21.15.89:0/1409010367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d1fad60 0x55a49d20b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:09.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.205+0000 7fb9ca600640 1 -- 172.21.15.89:0/1409010367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d1fb680 msgr2=0x55a49d20bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:09.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.205+0000 7fb9ca600640 1 --2- 172.21.15.89:0/1409010367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d1fb680 0x55a49d20bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.205+0000 7fb9ca600640 1 -- 172.21.15.89:0/1409010367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d1fad60 msgr2=0x55a49d20b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:09.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.205+0000 7fb9ca600640 1 --2- 172.21.15.89:0/1409010367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d1fad60 0x55a49d20b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.205+0000 7fb9ca600640 1 -- 172.21.15.89:0/1409010367 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a49ced23c0 con 0x55a49d20e830 2024-09-10T08:42:09.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.206+0000 7fb9c9200640 1 --2- 172.21.15.89:0/1409010367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d1fb680 0x55a49d20bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:42:09.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.206+0000 7fb9c9c00640 1 --2- 172.21.15.89:0/1409010367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d1fad60 0x55a49d20b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:42:09.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.207+0000 7fb9a7400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:42:09.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.207+0000 7fb9ca600640 1 --2- 172.21.15.89:0/1409010367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d20e830 0x55a49d20c2e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc05a040 tx=0x7fb9bc005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4399b210b08d3635 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:09.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.207+0000 7fb9a7400640 1 -- 172.21.15.89:0/1409010367 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc067070 con 0x55a49d20e830 2024-09-10T08:42:09.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.208+0000 7fb9a7400640 1 -- 172.21.15.89:0/1409010367 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb9bc003ca0 con 0x55a49d20e830 2024-09-10T08:42:09.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.209+0000 7fb9cf090740 1 -- 172.21.15.89:0/1409010367 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9bc02fe10 con 0x55a49d20e830 2024-09-10T08:42:09.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.209+0000 7fb9a7400640 1 -- 172.21.15.89:0/1409010367 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb9bc067070 con 0x55a49d20e830 2024-09-10T08:42:09.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.209+0000 7fb9cf090740 10 client.25828 fetch_fsmap learned FSMap version 28 2024-09-10T08:42:09.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.209+0000 7fb9cf090740 1 -- 172.21.15.89:0/1409010367 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a49d2113a0 con 0x55a49d20e830 2024-09-10T08:42:09.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.210+0000 7fb9a7400640 1 -- 172.21.15.89:0/1409010367 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9bc02fe10 con 0x55a49d20e830 2024-09-10T08:42:09.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.210+0000 7fb9a7400640 1 -- 172.21.15.89:0/1409010367 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb9bc02f070 con 0x55a49d20e830 2024-09-10T08:42:09.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.210+0000 7fb9cf090740 10 client.25828 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:42:09.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.210+0000 7fb9cf090740 10 client.25828 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:42:09.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.210+0000 7fb9cf090740 20 client.25828 populate_metadata read hostname 'smithi089' 2024-09-10T08:42:09.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.210+0000 7fb9cf090740 1 --2- 172.21.15.89:0/1409010367 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55a49d21e6d0 0x55a49d220ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:09.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.210+0000 7fb9cf090740 4 client.25828 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:42:09.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.210+0000 7fb9cf090740 1 -- 172.21.15.89:0/1409010367 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a49d16e950 con 0x55a49d21e6d0 2024-09-10T08:42:09.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.214+0000 7fb9c9c00640 1 --2- 172.21.15.89:0/1409010367 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55a49d21e6d0 0x55a49d220ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:09.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.218+0000 7fb9a7400640 10 client.25828 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:42:09.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.218+0000 7fb9c9c00640 1 --2- 172.21.15.89:0/1409010367 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55a49d21e6d0 0x55a49d220ac0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=62333bd49a6f394 server_cookie=5b8b6d99cf64e104 in_seq=0 out_seq=0 2024-09-10T08:42:09.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:08 smithi149 ceph-mon[36167]: pgmap v459: 129 pgs: 129 active+clean; 63 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.9 MiB/s wr, 907 op/s 2024-09-10T08:42:09.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.219+0000 7fb9a7400640 1 -- 172.21.15.89:0/1409010367 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a49d16e950 con 0x55a49d21e6d0 2024-09-10T08:42:09.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.221+0000 7fb9a7400640 10 client.25828 handle_command_reply: tid=0 2024-09-10T08:42:09.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.221+0000 7fb9cf090740 1 client.25828 shutdown 2024-09-10T08:42:09.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.221+0000 7fb9cf090740 1 -- 172.21.15.89:0/1409010367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d20e830 msgr2=0x55a49d20c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:09.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.221+0000 7fb9cf090740 1 --2- 172.21.15.89:0/1409010367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d20e830 0x55a49d20c2e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc05a040 tx=0x7fb9bc005f80 comp rx=0 tx=0).stop 2024-09-10T08:42:09.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.221+0000 7fb9cf090740 1 -- 172.21.15.89:0/1409010367 shutdown_connections 2024-09-10T08:42:09.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.221+0000 7fb9cf090740 1 --2- 172.21.15.89:0/1409010367 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55a49d21e6d0 0x55a49d220ac0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.221+0000 7fb9cf090740 1 --2- 172.21.15.89:0/1409010367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d20e830 0x55a49d20c2e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.221+0000 7fb9cf090740 1 --2- 172.21.15.89:0/1409010367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d1fb680 0x55a49d20bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.221+0000 7fb9cf090740 1 --2- 172.21.15.89:0/1409010367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d1fad60 0x55a49d20b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.221+0000 7fb9cf090740 1 -- 172.21.15.89:0/1409010367 >> 172.21.15.89:0/1409010367 conn(0x55a49d1f73d0 msgr2=0x55a49d1fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:09.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.221+0000 7fb9a7400640 0 client.25828 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:42:09.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.222+0000 7fb9cf090740 1 -- 172.21.15.89:0/1409010367 shutdown_connections 2024-09-10T08:42:09.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.222+0000 7fb9cf090740 1 -- 172.21.15.89:0/1409010367 wait complete. 2024-09-10T08:42:09.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.222+0000 7fb9cf090740 20 client.25828 trim_cache size 0 max 16384 2024-09-10T08:42:09.261 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.259+0000 7fb9cf090740 1 Processor -- start 2024-09-10T08:42:09.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.260+0000 7fb9cf090740 1 -- start start 2024-09-10T08:42:09.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.260+0000 7fb9cf090740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d1fad60 0x55a49d20a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:09.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.260+0000 7fb9cf090740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d1fb680 0x55a49d20c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:09.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.260+0000 7fb9cf090740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d20e830 0x55a49d2260f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:09.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.260+0000 7fb9cf090740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c4073dd0 con 0x55a49d1fb680 2024-09-10T08:42:09.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.260+0000 7fb9cf090740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c40716a0 con 0x55a49d20e830 2024-09-10T08:42:09.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.260+0000 7fb9cf090740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c406f550 con 0x55a49d1fad60 2024-09-10T08:42:09.267 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.262+0000 7fb9c9200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d1fb680 0x55a49d20c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.263+0000 7fb9c9c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d1fad60 0x55a49d20a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.263+0000 7fb9c9200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d1fb680 0x55a49d20c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47012/0 (socket says 172.21.15.89:47012) 2024-09-10T08:42:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.263+0000 7fb9c9200640 1 -- 172.21.15.89:0/102779636 learned_addr learned my addr 172.21.15.89:0/102779636 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.263+0000 7fb9ca600640 1 --2- 172.21.15.89:0/102779636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d20e830 0x55a49d2260f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.263+0000 7fb9c9c00640 1 -- 172.21.15.89:0/102779636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d20e830 msgr2=0x55a49d2260f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.264+0000 7fb9c9c00640 1 --2- 172.21.15.89:0/102779636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d20e830 0x55a49d2260f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.264+0000 7fb9c9c00640 1 -- 172.21.15.89:0/102779636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d1fb680 msgr2=0x55a49d20c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.264+0000 7fb9c9c00640 1 --2- 172.21.15.89:0/102779636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d1fb680 0x55a49d20c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.264+0000 7fb9c9c00640 1 -- 172.21.15.89:0/102779636 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a49d1f7f10 con 0x55a49d1fad60 2024-09-10T08:42:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.265+0000 7fb9c9c00640 1 --2- 172.21.15.89:0/102779636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d1fad60 0x55a49d20a130 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55a49d20cde0 tx=0x7fb9b8010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3758d27ff3651512 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.266+0000 7fb9b2000640 1 -- 172.21.15.89:0/102779636 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b802a070 con 0x55a49d1fad60 2024-09-10T08:42:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.266+0000 7fb9b2000640 1 -- 172.21.15.89:0/102779636 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9b801eaf0 con 0x55a49d1fad60 2024-09-10T08:42:09.268 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.266+0000 7fb9b2000640 1 -- 172.21.15.89:0/102779636 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8025b00 con 0x55a49d1fad60 2024-09-10T08:42:09.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.268+0000 7fb9cf090740 1 -- 172.21.15.89:0/102779636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d1fad60 msgr2=0x55a49d20a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.268+0000 7fb9cf090740 1 --2- 172.21.15.89:0/102779636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d1fad60 0x55a49d20a130 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55a49d20cde0 tx=0x7fb9b8010800 comp rx=0 tx=0).stop 2024-09-10T08:42:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.268+0000 7fb9cf090740 1 -- 172.21.15.89:0/102779636 shutdown_connections 2024-09-10T08:42:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.268+0000 7fb9cf090740 1 --2- 172.21.15.89:0/102779636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d20e830 0x55a49d2260f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.268+0000 7fb9cf090740 1 --2- 172.21.15.89:0/102779636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d1fb680 0x55a49d20c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.268+0000 7fb9cf090740 1 --2- 172.21.15.89:0/102779636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d1fad60 0x55a49d20a130 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55a49d20cde0 tx=0x7fb9b8010800 comp rx=0 tx=0).stop 2024-09-10T08:42:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.268+0000 7fb9cf090740 1 -- 172.21.15.89:0/102779636 >> 172.21.15.89:0/102779636 conn(0x55a49d1f73d0 msgr2=0x55a49d20d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.270+0000 7fb9cf090740 1 -- 172.21.15.89:0/102779636 shutdown_connections 2024-09-10T08:42:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.270+0000 7fb9cf090740 1 -- 172.21.15.89:0/102779636 wait complete. 2024-09-10T08:42:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.271+0000 7fb9cf090740 1 -- start start 2024-09-10T08:42:09.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.272+0000 7fb9cf090740 1 Processor -- start 2024-09-10T08:42:09.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.273+0000 7fb9cf090740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d1fb680 0x55a49d208a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:09.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.273+0000 7fb9cf090740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d20e830 0x55a49d208fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:09.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.273+0000 7fb9cf090740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d2094f0 0x55a49d2880c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:09.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.273+0000 7fb9c9c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d1fb680 0x55a49d208a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:09.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.274+0000 7fb9c9c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d1fb680 0x55a49d208a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:37780/0 (socket says 172.21.15.89:37780) 2024-09-10T08:42:09.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.274+0000 7fb9c9200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d20e830 0x55a49d208fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:09.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.274+0000 7fb9c9200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d20e830 0x55a49d208fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:60374/0 (socket says 172.21.15.89:60374) 2024-09-10T08:42:09.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.274+0000 7fb9cf090740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c406fba0 con 0x55a49d2094f0 2024-09-10T08:42:09.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.274+0000 7fb9cf090740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c4070fd0 con 0x55a49d1fb680 2024-09-10T08:42:09.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.274+0000 7fb9cf090740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c4070230 con 0x55a49d20e830 2024-09-10T08:42:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.274+0000 7fb9cf090740 10 client.0 _ll_register_callbacks cb 0x7fb9bc05a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:42:09.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.274+0000 7fb9c9c00640 1 -- 172.21.15.89:0/995697106 learned_addr learned my addr 172.21.15.89:0/995697106 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.274+0000 7fb9ca600640 1 --2- 172.21.15.89:0/995697106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d2094f0 0x55a49d2880c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.278+0000 7fb9c9c00640 1 -- 172.21.15.89:0/995697106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d20e830 msgr2=0x55a49d208fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.278+0000 7fb9c9c00640 1 --2- 172.21.15.89:0/995697106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d20e830 0x55a49d208fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.278+0000 7fb9c9c00640 1 -- 172.21.15.89:0/995697106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d2094f0 msgr2=0x55a49d2880c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.278+0000 7fb9c9c00640 1 --2- 172.21.15.89:0/995697106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d2094f0 0x55a49d2880c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.278+0000 7fb9c9c00640 1 -- 172.21.15.89:0/995697106 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a49d288760 con 0x55a49d1fb680 2024-09-10T08:42:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.278+0000 7fb9ca600640 1 --2- 172.21.15.89:0/995697106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d2094f0 0x55a49d2880c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:42:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.278+0000 7fb9c9200640 1 --2- 172.21.15.89:0/995697106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d20e830 0x55a49d208fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:42:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.279+0000 7fb9a7400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:42:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.279+0000 7fb9c9c00640 1 --2- 172.21.15.89:0/995697106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d1fb680 0x55a49d208a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8005890 tx=0x7fb9b8025630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37cab3965a7ea988 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.279+0000 7fb9a7400640 1 -- 172.21.15.89:0/995697106 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b802a050 con 0x55a49d1fb680 2024-09-10T08:42:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.279+0000 7fb9a7400640 1 -- 172.21.15.89:0/995697106 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb9b801e820 con 0x55a49d1fb680 2024-09-10T08:42:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.279+0000 7fb9cf090740 1 -- 172.21.15.89:0/995697106 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9b801eaf0 con 0x55a49d1fb680 2024-09-10T08:42:09.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.280+0000 7fb9a7400640 1 -- 172.21.15.89:0/995697106 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb9b803f050 con 0x55a49d1fb680 2024-09-10T08:42:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.280+0000 7fb9cf090740 10 client.25901 fetch_fsmap learned FSMap version 28 2024-09-10T08:42:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.280+0000 7fb9cf090740 1 -- 172.21.15.89:0/995697106 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a49d289a20 con 0x55a49d1fb680 2024-09-10T08:42:09.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.280+0000 7fb9a7400640 1 -- 172.21.15.89:0/995697106 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9b801eaf0 con 0x55a49d1fb680 2024-09-10T08:42:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.280+0000 7fb9a7400640 1 -- 172.21.15.89:0/995697106 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb9b80053b0 con 0x55a49d1fb680 2024-09-10T08:42:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.280+0000 7fb9cf090740 10 client.25901 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:42:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.280+0000 7fb9cf090740 10 client.25901 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:42:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.280+0000 7fb9cf090740 20 client.25901 populate_metadata read hostname 'smithi089' 2024-09-10T08:42:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.280+0000 7fb9cf090740 1 --2- 172.21.15.89:0/995697106 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55a49d289f90 0x55a49d28c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.281+0000 7fb9cf090740 4 client.25901 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:42:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.281+0000 7fb9cf090740 1 -- 172.21.15.89:0/995697106 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a49d168e60 con 0x55a49d289f90 2024-09-10T08:42:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.281+0000 7fb9c9200640 1 --2- 172.21.15.89:0/995697106 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55a49d289f90 0x55a49d28c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.281+0000 7fb9c9200640 1 --2- 172.21.15.89:0/995697106 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55a49d289f90 0x55a49d28c3b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aaadbe883ebb6793 server_cookie=e684a85f848548b2 in_seq=0 out_seq=0 2024-09-10T08:42:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.281+0000 7fb9a7400640 10 client.25901 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:42:09.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.282+0000 7fb9a7400640 1 -- 172.21.15.89:0/995697106 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a49d168e60 con 0x55a49d289f90 2024-09-10T08:42:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.282+0000 7fb9a7400640 10 client.25901 handle_command_reply: tid=0 2024-09-10T08:42:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.283+0000 7fb9cf090740 1 client.25901 shutdown 2024-09-10T08:42:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.283+0000 7fb9cf090740 1 -- 172.21.15.89:0/995697106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d1fb680 msgr2=0x55a49d208a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.283+0000 7fb9cf090740 1 --2- 172.21.15.89:0/995697106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d1fb680 0x55a49d208a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8005890 tx=0x7fb9b8025630 comp rx=0 tx=0).stop 2024-09-10T08:42:09.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.283+0000 7fb9cf090740 1 -- 172.21.15.89:0/995697106 shutdown_connections 2024-09-10T08:42:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.283+0000 7fb9cf090740 1 --2- 172.21.15.89:0/995697106 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55a49d289f90 0x55a49d28c3b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.283+0000 7fb9cf090740 1 --2- 172.21.15.89:0/995697106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a49d2094f0 0x55a49d2880c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.283+0000 7fb9cf090740 1 --2- 172.21.15.89:0/995697106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a49d20e830 0x55a49d208fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.283+0000 7fb9cf090740 1 --2- 172.21.15.89:0/995697106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a49d1fb680 0x55a49d208a70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.283+0000 7fb9a7400640 0 client.25901 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:42:09.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.283+0000 7fb9cf090740 1 -- 172.21.15.89:0/995697106 >> 172.21.15.89:0/995697106 conn(0x55a49d1f73d0 msgr2=0x55a49d1fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:09.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.284+0000 7fb9cf090740 1 -- 172.21.15.89:0/995697106 shutdown_connections 2024-09-10T08:42:09.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.284+0000 7fb9cf090740 1 -- 172.21.15.89:0/995697106 wait complete. 2024-09-10T08:42:09.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.284+0000 7fb9cf090740 20 client.25901 trim_cache size 0 max 16384 2024-09-10T08:42:09.286 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:42:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.286+0000 7fb9cb000640 1 -- 172.21.15.89:0/2701396469 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb9ac0a2cc0 msgr2=0x7fb9ac0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.286+0000 7fb9cb000640 1 --2- 172.21.15.89:0/2701396469 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb9ac0a2cc0 0x7fb9ac0a5180 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc059000 tx=0x7fb9bc004490 comp rx=0 tx=0).stop 2024-09-10T08:42:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.286+0000 7fb9cb000640 1 -- 172.21.15.89:0/2701396469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9c4074ee0 msgr2=0x7fb9c419a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.286+0000 7fb9cb000640 1 --2- 172.21.15.89:0/2701396469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9c4074ee0 0x7fb9c419a8d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400c920 tx=0x7fb9b400cd20 comp rx=0 tx=0).stop 2024-09-10T08:42:09.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.286+0000 7fb9cb000640 1 -- 172.21.15.89:0/2701396469 shutdown_connections 2024-09-10T08:42:09.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.286+0000 7fb9cb000640 1 --2- 172.21.15.89:0/2701396469 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb9ac0a2cc0 0x7fb9ac0a5180 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.286+0000 7fb9cb000640 1 --2- 172.21.15.89:0/2701396469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c410e260 0x7fb9c419ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.286+0000 7fb9cb000640 1 --2- 172.21.15.89:0/2701396469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb9c4074ee0 0x7fb9c419a8d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.286+0000 7fb9cb000640 1 --2- 172.21.15.89:0/2701396469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9c40745c0 0x7fb9c419a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:09.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.286+0000 7fb9cb000640 1 -- 172.21.15.89:0/2701396469 >> 172.21.15.89:0/2701396469 conn(0x7fb9c40fd740 msgr2=0x7fb9c40fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:09.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.288+0000 7fb9cb000640 1 -- 172.21.15.89:0/2701396469 shutdown_connections 2024-09-10T08:42:09.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:09.288+0000 7fb9cb000640 1 -- 172.21.15.89:0/2701396469 wait complete. 2024-09-10T08:42:09.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:08 smithi111 ceph-mon[35220]: pgmap v459: 129 pgs: 129 active+clean; 63 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.9 MiB/s wr, 907 op/s 2024-09-10T08:42:10.055 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:09 smithi089 ceph-mon[30757]: pgmap v460: 129 pgs: 129 active+clean; 63 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 851 op/s 2024-09-10T08:42:10.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:09 smithi149 ceph-mon[36167]: pgmap v460: 129 pgs: 129 active+clean; 63 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 851 op/s 2024-09-10T08:42:10.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:09 smithi111 ceph-mon[35220]: pgmap v460: 129 pgs: 129 active+clean; 63 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 851 op/s 2024-09-10T08:42:10.420 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:42:11.350 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:42:12.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.075+0000 7ffa39800640 1 -- 172.21.15.89:0/101738451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa341049a0 msgr2=0x7ffa341089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:12.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.075+0000 7ffa39800640 1 --2- 172.21.15.89:0/101738451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa341049a0 0x7ffa341089f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffa20009900 tx=0x7ffa2002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.075+0000 7ffa39800640 1 -- 172.21.15.89:0/101738451 shutdown_connections 2024-09-10T08:42:12.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.075+0000 7ffa39800640 1 --2- 172.21.15.89:0/101738451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa34109120 0x7ffa34110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.075+0000 7ffa39800640 1 --2- 172.21.15.89:0/101738451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa341049a0 0x7ffa341089f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.075+0000 7ffa39800640 1 --2- 172.21.15.89:0/101738451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa34103ff0 0x7ffa341043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.075+0000 7ffa39800640 1 -- 172.21.15.89:0/101738451 >> 172.21.15.89:0/101738451 conn(0x7ffa340ffa90 msgr2=0x7ffa34101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:12.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.076+0000 7ffa39800640 1 -- 172.21.15.89:0/101738451 shutdown_connections 2024-09-10T08:42:12.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.076+0000 7ffa39800640 1 -- 172.21.15.89:0/101738451 wait complete. 2024-09-10T08:42:12.079 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.077+0000 7ffa39800640 1 Processor -- start 2024-09-10T08:42:12.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.077+0000 7ffa39800640 1 -- start start 2024-09-10T08:42:12.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.078+0000 7ffa39800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa34103ff0 0x7ffa3419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:12.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.078+0000 7ffa39800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa341049a0 0x7ffa3419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:12.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.078+0000 7ffa39800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa34109120 0x7ffa3419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:12.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.078+0000 7ffa39800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ffa34114190 con 0x7ffa34109120 2024-09-10T08:42:12.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.078+0000 7ffa39800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffa34114010 con 0x7ffa34103ff0 2024-09-10T08:42:12.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.078+0000 7ffa39800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffa34114310 con 0x7ffa341049a0 2024-09-10T08:42:12.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.078+0000 7ffa32a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa34103ff0 0x7ffa3419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:12.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.078+0000 7ffa32a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa34103ff0 0x7ffa3419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:37792/0 (socket says 172.21.15.89:37792) 2024-09-10T08:42:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.078+0000 7ffa32a00640 1 -- 172.21.15.89:0/2629828693 learned_addr learned my addr 172.21.15.89:0/2629828693 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.078+0000 7ffa32000640 1 --2- 172.21.15.89:0/2629828693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa341049a0 0x7ffa3419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.078+0000 7ffa33400640 1 --2- 172.21.15.89:0/2629828693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa34109120 0x7ffa3419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.079+0000 7ffa32a00640 1 -- 172.21.15.89:0/2629828693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa341049a0 msgr2=0x7ffa3419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:12.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.079+0000 7ffa32a00640 1 --2- 172.21.15.89:0/2629828693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa341049a0 0x7ffa3419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.079+0000 7ffa32a00640 1 -- 172.21.15.89:0/2629828693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa34109120 msgr2=0x7ffa3419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.079+0000 7ffa32a00640 1 --2- 172.21.15.89:0/2629828693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa34109120 0x7ffa3419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.079+0000 7ffa32a00640 1 -- 172.21.15.89:0/2629828693 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3419f5e0 con 0x7ffa34103ff0 2024-09-10T08:42:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.080+0000 7ffa32a00640 1 --2- 172.21.15.89:0/2629828693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa34103ff0 0x7ffa3419a630 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400ea60 tx=0x7ffa2400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:12.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.080+0000 7ffa2b400640 1 -- 172.21.15.89:0/2629828693 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2400cde0 con 0x7ffa34103ff0 2024-09-10T08:42:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.080+0000 7ffa39800640 1 -- 172.21.15.89:0/2629828693 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3419f8d0 con 0x7ffa34103ff0 2024-09-10T08:42:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.080+0000 7ffa2b400640 1 -- 172.21.15.89:0/2629828693 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa240035f0 con 0x7ffa34103ff0 2024-09-10T08:42:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.080+0000 7ffa2b400640 1 -- 172.21.15.89:0/2629828693 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24010670 con 0x7ffa34103ff0 2024-09-10T08:42:12.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.081+0000 7ffa39800640 1 -- 172.21.15.89:0/2629828693 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa341a71b0 con 0x7ffa34103ff0 2024-09-10T08:42:12.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.083+0000 7ffa2b400640 1 -- 172.21.15.89:0/2629828693 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ffa240037a0 con 0x7ffa34103ff0 2024-09-10T08:42:12.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.085+0000 7ffa2b400640 1 --2- 172.21.15.89:0/2629828693 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffa180a2b80 0x7ffa180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:12.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.085+0000 7ffa3c23b740 1 Processor -- start 2024-09-10T08:42:12.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.085+0000 7ffa2b400640 1 -- 172.21.15.89:0/2629828693 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ffa240cf790 con 0x7ffa34103ff0 2024-09-10T08:42:12.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.085+0000 7ffa32000640 1 --2- 172.21.15.89:0/2629828693 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffa180a2b80 0x7ffa180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:12.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.085+0000 7ffa3c23b740 1 -- start start 2024-09-10T08:42:12.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.086+0000 7ffa3c23b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893785480 0x560893785860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:12.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.086+0000 7ffa3c23b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785da0 0x560893790120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:12.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.086+0000 7ffa3c23b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893790660 0x560893792a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:12.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.086+0000 7ffa3c23b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5608936e33e0 con 0x560893785da0 2024-09-10T08:42:12.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.086+0000 7ffa3c23b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56089368ad80 con 0x560893790660 2024-09-10T08:42:12.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.086+0000 7ffa3c23b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5608936df3a0 con 0x560893785480 2024-09-10T08:42:12.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.086+0000 7ffa33400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893790660 0x560893792a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:12.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.086+0000 7ffa33400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893790660 0x560893792a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:37800/0 (socket says 172.21.15.89:37800) 2024-09-10T08:42:12.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.086+0000 7ffa33400640 1 -- 172.21.15.89:0/2999983220 learned_addr learned my addr 172.21.15.89:0/2999983220 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:12.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.086+0000 7ffa32000640 1 --2- 172.21.15.89:0/2629828693 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffa180a2b80 0x7ffa180a5040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffa20000c00 tx=0x7ffa20005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:12.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.086+0000 7ffa32000640 1 --2- 172.21.15.89:0/2999983220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785da0 0x560893790120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:12.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.086+0000 7ffa32a00640 1 --2- 172.21.15.89:0/2999983220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893785480 0x560893785860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:12.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.086+0000 7ffa33400640 1 -- 172.21.15.89:0/2999983220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893785480 msgr2=0x560893785860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:12.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.086+0000 7ffa33400640 1 --2- 172.21.15.89:0/2999983220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893785480 0x560893785860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.086+0000 7ffa33400640 1 -- 172.21.15.89:0/2999983220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785da0 msgr2=0x560893790120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:12.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.086+0000 7ffa33400640 1 --2- 172.21.15.89:0/2999983220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785da0 0x560893790120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.086+0000 7ffa33400640 1 -- 172.21.15.89:0/2999983220 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608936a9ae0 con 0x560893790660 2024-09-10T08:42:12.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.087+0000 7ffa33400640 1 --2- 172.21.15.89:0/2999983220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893790660 0x560893792a50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009f50 tx=0x7ffa2c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cacc9725507f0034 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:12.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.087+0000 7ffa1fe00640 1 -- 172.21.15.89:0/2999983220 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c004d20 con 0x560893790660 2024-09-10T08:42:12.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.087+0000 7ffa1fe00640 1 -- 172.21.15.89:0/2999983220 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa2c004050 con 0x560893790660 2024-09-10T08:42:12.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.087+0000 7ffa1fe00640 1 -- 172.21.15.89:0/2999983220 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c005280 con 0x560893790660 2024-09-10T08:42:12.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.088+0000 7ffa3c23b740 1 -- 172.21.15.89:0/2999983220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893790660 msgr2=0x560893792a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:12.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.088+0000 7ffa3c23b740 1 --2- 172.21.15.89:0/2999983220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893790660 0x560893792a50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009f50 tx=0x7ffa2c0118d0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.088+0000 7ffa3c23b740 1 -- 172.21.15.89:0/2999983220 shutdown_connections 2024-09-10T08:42:12.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.088+0000 7ffa3c23b740 1 --2- 172.21.15.89:0/2999983220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893790660 0x560893792a50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.088+0000 7ffa3c23b740 1 --2- 172.21.15.89:0/2999983220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785da0 0x560893790120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.088+0000 7ffa3c23b740 1 --2- 172.21.15.89:0/2999983220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893785480 0x560893785860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.088+0000 7ffa3c23b740 1 -- 172.21.15.89:0/2999983220 >> 172.21.15.89:0/2999983220 conn(0x560893781a10 msgr2=0x560893781df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:12.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.089+0000 7ffa3c23b740 1 -- 172.21.15.89:0/2999983220 shutdown_connections 2024-09-10T08:42:12.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.089+0000 7ffa3c23b740 1 -- 172.21.15.89:0/2999983220 wait complete. 2024-09-10T08:42:12.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.089+0000 7ffa3c23b740 1 -- start start 2024-09-10T08:42:12.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.089+0000 7ffa3c23b740 1 Processor -- start 2024-09-10T08:42:12.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.090+0000 7ffa3c23b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785480 0x56089378d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:12.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.090+0000 7ffa3c23b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893785da0 0x56089378da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:12.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.090+0000 7ffa3c23b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893790660 0x56089378df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:12.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.090+0000 7ffa3c23b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560893793230 con 0x560893785480 2024-09-10T08:42:12.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.090+0000 7ffa3c23b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56089369e0f0 con 0x560893790660 2024-09-10T08:42:12.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.090+0000 7ffa3c23b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5608937933b0 con 0x560893785da0 2024-09-10T08:42:12.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.090+0000 7ffa3c23b740 10 client.0 _ll_register_callbacks cb 0x560893685bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:42:12.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.090+0000 7ffa32a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785480 0x56089378d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:12.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.090+0000 7ffa32000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893785da0 0x56089378da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:12.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.090+0000 7ffa32000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893785da0 0x56089378da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:60430/0 (socket says 172.21.15.89:60430) 2024-09-10T08:42:12.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.090+0000 7ffa32000640 1 -- 172.21.15.89:0/1445027199 learned_addr learned my addr 172.21.15.89:0/1445027199 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:12.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.091+0000 7ffa33400640 1 --2- 172.21.15.89:0/1445027199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893790660 0x56089378df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:12.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.091+0000 7ffa32000640 1 -- 172.21.15.89:0/1445027199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893790660 msgr2=0x56089378df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:12.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.091+0000 7ffa32000640 1 --2- 172.21.15.89:0/1445027199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893790660 0x56089378df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.091+0000 7ffa32000640 1 -- 172.21.15.89:0/1445027199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785480 msgr2=0x56089378d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:12.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.091+0000 7ffa32000640 1 --2- 172.21.15.89:0/1445027199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785480 0x56089378d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.091+0000 7ffa32000640 1 -- 172.21.15.89:0/1445027199 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608934543c0 con 0x560893785da0 2024-09-10T08:42:12.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.091+0000 7ffa3c23b740 1 -- 172.21.15.89:0/1445027199 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa2c004050 con 0x560893785da0 2024-09-10T08:42:12.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.091+0000 7ffa32a00640 1 --2- 172.21.15.89:0/1445027199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785480 0x56089378d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:42:12.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.091+0000 7ffa32000640 1 --2- 172.21.15.89:0/1445027199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893785da0 0x56089378da00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffa20031550 tx=0x7ffa20009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55a2802ce8917272 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:12.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.091+0000 7ffa1cc00640 10 client.25846 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:42:12.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.092+0000 7ffa1cc00640 1 -- 172.21.15.89:0/1445027199 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20035040 con 0x560893785da0 2024-09-10T08:42:12.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.092+0000 7ffa1cc00640 1 -- 172.21.15.89:0/1445027199 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ffa20004ad0 con 0x560893785da0 2024-09-10T08:42:12.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.092+0000 7ffa1cc00640 1 -- 172.21.15.89:0/1445027199 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffa20005650 con 0x560893785da0 2024-09-10T08:42:12.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.092+0000 7ffa3c23b740 10 client.25846 fetch_fsmap learned FSMap version 28 2024-09-10T08:42:12.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.092+0000 7ffa3c23b740 1 -- 172.21.15.89:0/1445027199 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5608937b01b0 con 0x560893785da0 2024-09-10T08:42:12.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.092+0000 7ffa1cc00640 1 -- 172.21.15.89:0/1445027199 <== mon.1 v2:172.21.15.149:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ffa20004cd0 con 0x560893785da0 2024-09-10T08:42:12.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.093+0000 7ffa3c23b740 10 client.25846 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:42:12.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.093+0000 7ffa3c23b740 10 client.25846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:42:12.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.093+0000 7ffa3c23b740 20 client.25846 populate_metadata read hostname 'smithi089' 2024-09-10T08:42:12.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.093+0000 7ffa3c23b740 1 --2- 172.21.15.89:0/1445027199 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5608937b1600 0x5608937b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:12.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.093+0000 7ffa3c23b740 4 client.25846 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:42:12.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.093+0000 7ffa3c23b740 1 -- 172.21.15.89:0/1445027199 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608936f0950 con 0x5608937b1600 2024-09-10T08:42:12.108 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.093+0000 7ffa32a00640 1 --2- 172.21.15.89:0/1445027199 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5608937b1600 0x5608937b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:12.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.093+0000 7ffa32a00640 1 --2- 172.21.15.89:0/1445027199 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5608937b1600 0x5608937b3a40 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e2a5a8944939d60 server_cookie=1e26dd849a977807 in_seq=0 out_seq=0 2024-09-10T08:42:12.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.093+0000 7ffa1cc00640 10 client.25846 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:12.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.095+0000 7ffa1cc00640 1 -- 172.21.15.89:0/1445027199 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5608936f0950 con 0x5608937b1600 2024-09-10T08:42:12.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.095+0000 7ffa1cc00640 10 client.25846 handle_command_reply: tid=0 2024-09-10T08:42:12.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.095+0000 7ffa3c23b740 1 client.25846 shutdown 2024-09-10T08:42:12.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.095+0000 7ffa3c23b740 1 -- 172.21.15.89:0/1445027199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893785da0 msgr2=0x56089378da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:12.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.095+0000 7ffa3c23b740 1 --2- 172.21.15.89:0/1445027199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893785da0 0x56089378da00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffa20031550 tx=0x7ffa20009260 comp rx=0 tx=0).stop 2024-09-10T08:42:12.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.095+0000 7ffa3c23b740 1 -- 172.21.15.89:0/1445027199 shutdown_connections 2024-09-10T08:42:12.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.095+0000 7ffa3c23b740 1 --2- 172.21.15.89:0/1445027199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893790660 0x56089378df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.095+0000 7ffa3c23b740 1 --2- 172.21.15.89:0/1445027199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893785da0 0x56089378da00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.095+0000 7ffa3c23b740 1 --2- 172.21.15.89:0/1445027199 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5608937b1600 0x5608937b3a40 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.095+0000 7ffa3c23b740 1 --2- 172.21.15.89:0/1445027199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785480 0x56089378d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.095+0000 7ffa1cc00640 0 client.25846 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:12.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.095+0000 7ffa3c23b740 1 -- 172.21.15.89:0/1445027199 >> 172.21.15.89:0/1445027199 conn(0x560893781a10 msgr2=0x560893778380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:12.115 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.095+0000 7ffa3c23b740 1 -- 172.21.15.89:0/1445027199 shutdown_connections 2024-09-10T08:42:12.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.095+0000 7ffa3c23b740 1 -- 172.21.15.89:0/1445027199 wait complete. 2024-09-10T08:42:12.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.096+0000 7ffa3c23b740 20 client.25846 trim_cache size 0 max 16384 2024-09-10T08:42:12.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.110+0000 7ffa3c23b740 1 Processor -- start 2024-09-10T08:42:12.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.110+0000 7ffa3c23b740 1 -- start start 2024-09-10T08:42:12.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.110+0000 7ffa3c23b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893785480 0x56089378c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:12.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.110+0000 7ffa3c23b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785da0 0x560893787ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:12.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.110+0000 7ffa3c23b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893790660 0x56089378dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:12.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.110+0000 7ffa3c23b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ffa34073c80 con 0x560893785da0 2024-09-10T08:42:12.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.110+0000 7ffa3c23b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffa34071550 con 0x560893785480 2024-09-10T08:42:12.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.110+0000 7ffa3c23b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3406f3c0 con 0x560893790660 2024-09-10T08:42:12.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.110+0000 7ffa32a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893785480 0x56089378c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:12.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.110+0000 7ffa32a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893785480 0x56089378c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:37824/0 (socket says 172.21.15.89:37824) 2024-09-10T08:42:12.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.110+0000 7ffa32a00640 1 -- 172.21.15.89:0/3227570830 learned_addr learned my addr 172.21.15.89:0/3227570830 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:12.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.110+0000 7ffa33400640 1 --2- 172.21.15.89:0/3227570830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893790660 0x56089378dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:12.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.110+0000 7ffa32000640 1 --2- 172.21.15.89:0/3227570830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785da0 0x560893787ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:12.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.111+0000 7ffa33400640 1 -- 172.21.15.89:0/3227570830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893785480 msgr2=0x56089378c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:12.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.111+0000 7ffa33400640 1 --2- 172.21.15.89:0/3227570830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893785480 0x56089378c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.111+0000 7ffa33400640 1 -- 172.21.15.89:0/3227570830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785da0 msgr2=0x560893787ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:12.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.111+0000 7ffa33400640 1 --2- 172.21.15.89:0/3227570830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785da0 0x560893787ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.111+0000 7ffa33400640 1 -- 172.21.15.89:0/3227570830 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560893781840 con 0x560893790660 2024-09-10T08:42:12.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.111+0000 7ffa33400640 1 --2- 172.21.15.89:0/3227570830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893790660 0x56089378dd90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c01b040 tx=0x7ffa2c0042b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=468ccd15f1a6d953 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:12.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.111+0000 7ffa1f400640 1 -- 172.21.15.89:0/3227570830 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c029020 con 0x560893790660 2024-09-10T08:42:12.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.112+0000 7ffa1f400640 1 -- 172.21.15.89:0/3227570830 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa2c003c30 con 0x560893790660 2024-09-10T08:42:12.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.112+0000 7ffa1f400640 1 -- 172.21.15.89:0/3227570830 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c004820 con 0x560893790660 2024-09-10T08:42:12.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.112+0000 7ffa3c23b740 1 -- 172.21.15.89:0/3227570830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893790660 msgr2=0x56089378dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:12.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.112+0000 7ffa3c23b740 1 --2- 172.21.15.89:0/3227570830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893790660 0x56089378dd90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c01b040 tx=0x7ffa2c0042b0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.112+0000 7ffa3c23b740 1 -- 172.21.15.89:0/3227570830 shutdown_connections 2024-09-10T08:42:12.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.112+0000 7ffa3c23b740 1 --2- 172.21.15.89:0/3227570830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893790660 0x56089378dd90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.112+0000 7ffa3c23b740 1 --2- 172.21.15.89:0/3227570830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785da0 0x560893787ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.112+0000 7ffa3c23b740 1 --2- 172.21.15.89:0/3227570830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893785480 0x56089378c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.112+0000 7ffa3c23b740 1 -- 172.21.15.89:0/3227570830 >> 172.21.15.89:0/3227570830 conn(0x560893781a10 msgr2=0x560893778380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:12.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.113+0000 7ffa3c23b740 1 -- 172.21.15.89:0/3227570830 shutdown_connections 2024-09-10T08:42:12.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.113+0000 7ffa3c23b740 1 -- 172.21.15.89:0/3227570830 wait complete. 2024-09-10T08:42:12.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.114+0000 7ffa3c23b740 1 -- start start 2024-09-10T08:42:12.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.114+0000 7ffa3c23b740 1 Processor -- start 2024-09-10T08:42:12.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.115+0000 7ffa3c23b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893785480 0x560893798c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:12.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.115+0000 7ffa3c23b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785da0 0x56089378a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:12.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.115+0000 7ffa3c23b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893790660 0x56089378a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:12.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.115+0000 7ffa3c23b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3406fa10 con 0x560893785da0 2024-09-10T08:42:12.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.115+0000 7ffa3c23b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffa34070e80 con 0x560893790660 2024-09-10T08:42:12.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.115+0000 7ffa3c23b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffa340700e0 con 0x560893785480 2024-09-10T08:42:12.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.115+0000 7ffa3c23b740 10 client.0 _ll_register_callbacks cb 0x7ffa20040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:42:12.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.115+0000 7ffa33400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893790660 0x56089378a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:12.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.115+0000 7ffa32000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785da0 0x56089378a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:12.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.115+0000 7ffa33400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893790660 0x56089378a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:37840/0 (socket says 172.21.15.89:37840) 2024-09-10T08:42:12.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.115+0000 7ffa33400640 1 -- 172.21.15.89:0/3479360661 learned_addr learned my addr 172.21.15.89:0/3479360661 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:12.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.115+0000 7ffa32a00640 1 --2- 172.21.15.89:0/3479360661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893785480 0x560893798c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:12.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.116+0000 7ffa33400640 1 -- 172.21.15.89:0/3479360661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893785480 msgr2=0x560893798c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:12.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.116+0000 7ffa33400640 1 --2- 172.21.15.89:0/3479360661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893785480 0x560893798c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.116+0000 7ffa33400640 1 -- 172.21.15.89:0/3479360661 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785da0 msgr2=0x56089378a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:12.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.116+0000 7ffa33400640 1 --2- 172.21.15.89:0/3479360661 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785da0 0x56089378a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.116+0000 7ffa33400640 1 -- 172.21.15.89:0/3479360661 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560893774770 con 0x560893790660 2024-09-10T08:42:12.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.116+0000 7ffa33400640 1 --2- 172.21.15.89:0/3479360661 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893790660 0x56089378a610 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009f50 tx=0x7ffa2c0056a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a93ecce00f10fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:12.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.116+0000 7ffa1cc00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:42:12.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.116+0000 7ffa1cc00640 1 -- 172.21.15.89:0/3479360661 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c029030 con 0x560893790660 2024-09-10T08:42:12.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.116+0000 7ffa3c23b740 1 -- 172.21.15.89:0/3479360661 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa2c003c30 con 0x560893790660 2024-09-10T08:42:12.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.116+0000 7ffa1cc00640 1 -- 172.21.15.89:0/3479360661 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ffa2c025470 con 0x560893790660 2024-09-10T08:42:12.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.117+0000 7ffa1cc00640 1 -- 172.21.15.89:0/3479360661 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffa2c03f050 con 0x560893790660 2024-09-10T08:42:12.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.117+0000 7ffa3c23b740 10 client.25919 fetch_fsmap learned FSMap version 28 2024-09-10T08:42:12.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.117+0000 7ffa3c23b740 1 -- 172.21.15.89:0/3479360661 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56089381c050 con 0x560893790660 2024-09-10T08:42:12.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.117+0000 7ffa1cc00640 1 -- 172.21.15.89:0/3479360661 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa2c003c30 con 0x560893790660 2024-09-10T08:42:12.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.117+0000 7ffa1cc00640 1 -- 172.21.15.89:0/3479360661 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ffa2c004530 con 0x560893790660 2024-09-10T08:42:12.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.117+0000 7ffa3c23b740 10 client.25919 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:42:12.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.117+0000 7ffa3c23b740 10 client.25919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:42:12.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.117+0000 7ffa3c23b740 20 client.25919 populate_metadata read hostname 'smithi089' 2024-09-10T08:42:12.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.117+0000 7ffa3c23b740 1 --2- 172.21.15.89:0/3479360661 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56089381c530 0x56089381e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:12.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.117+0000 7ffa3c23b740 4 client.25919 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:42:12.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.117+0000 7ffa3c23b740 1 -- 172.21.15.89:0/3479360661 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5608936eae60 con 0x56089381c530 2024-09-10T08:42:12.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.118+0000 7ffa32a00640 1 --2- 172.21.15.89:0/3479360661 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56089381c530 0x56089381e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:12.132 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.118+0000 7ffa32a00640 1 --2- 172.21.15.89:0/3479360661 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56089381c530 0x56089381e920 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86908df8de7188e7 server_cookie=c4b4c80edea499f3 in_seq=0 out_seq=0 2024-09-10T08:42:12.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.118+0000 7ffa1cc00640 10 client.25919 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:12.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.120+0000 7ffa1cc00640 1 -- 172.21.15.89:0/3479360661 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5608936eae60 con 0x56089381c530 2024-09-10T08:42:12.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.120+0000 7ffa1cc00640 10 client.25919 handle_command_reply: tid=0 2024-09-10T08:42:12.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.120+0000 7ffa3c23b740 1 client.25919 shutdown 2024-09-10T08:42:12.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.120+0000 7ffa3c23b740 1 -- 172.21.15.89:0/3479360661 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893790660 msgr2=0x56089378a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:12.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.120+0000 7ffa3c23b740 1 --2- 172.21.15.89:0/3479360661 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893790660 0x56089378a610 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009f50 tx=0x7ffa2c0056a0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.120+0000 7ffa3c23b740 1 -- 172.21.15.89:0/3479360661 shutdown_connections 2024-09-10T08:42:12.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.120+0000 7ffa3c23b740 1 --2- 172.21.15.89:0/3479360661 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560893790660 0x56089378a610 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.120+0000 7ffa3c23b740 1 --2- 172.21.15.89:0/3479360661 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56089381c530 0x56089381e920 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.120+0000 7ffa3c23b740 1 --2- 172.21.15.89:0/3479360661 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560893785da0 0x56089378a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.120+0000 7ffa3c23b740 1 --2- 172.21.15.89:0/3479360661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560893785480 0x560893798c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.120+0000 7ffa3c23b740 1 -- 172.21.15.89:0/3479360661 >> 172.21.15.89:0/3479360661 conn(0x560893781a10 msgr2=0x56089378ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:12.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.120+0000 7ffa1cc00640 0 client.25919 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:12.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.121+0000 7ffa3c23b740 1 -- 172.21.15.89:0/3479360661 shutdown_connections 2024-09-10T08:42:12.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.121+0000 7ffa3c23b740 1 -- 172.21.15.89:0/3479360661 wait complete. 2024-09-10T08:42:12.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.121+0000 7ffa3c23b740 20 client.25919 trim_cache size 0 max 16384 2024-09-10T08:42:12.136 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:42:12.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.122+0000 7ffa39800640 1 -- 172.21.15.89:0/2629828693 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffa180a2b80 msgr2=0x7ffa180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:12.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.122+0000 7ffa39800640 1 --2- 172.21.15.89:0/2629828693 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffa180a2b80 0x7ffa180a5040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffa20000c00 tx=0x7ffa20005b50 comp rx=0 tx=0).stop 2024-09-10T08:42:12.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.122+0000 7ffa39800640 1 -- 172.21.15.89:0/2629828693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa34103ff0 msgr2=0x7ffa3419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:12.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.122+0000 7ffa39800640 1 --2- 172.21.15.89:0/2629828693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa34103ff0 0x7ffa3419a630 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400ea60 tx=0x7ffa2400ef20 comp rx=0 tx=0).stop 2024-09-10T08:42:12.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.122+0000 7ffa39800640 1 -- 172.21.15.89:0/2629828693 shutdown_connections 2024-09-10T08:42:12.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.122+0000 7ffa39800640 1 --2- 172.21.15.89:0/2629828693 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffa180a2b80 0x7ffa180a5040 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.122+0000 7ffa39800640 1 --2- 172.21.15.89:0/2629828693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa34109120 0x7ffa3419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.122+0000 7ffa39800640 1 --2- 172.21.15.89:0/2629828693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa341049a0 0x7ffa3419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.122+0000 7ffa39800640 1 --2- 172.21.15.89:0/2629828693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa34103ff0 0x7ffa3419a630 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:12.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.122+0000 7ffa39800640 1 -- 172.21.15.89:0/2629828693 >> 172.21.15.89:0/2629828693 conn(0x7ffa340ffa90 msgr2=0x7ffa34100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:12.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.122+0000 7ffa39800640 1 -- 172.21.15.89:0/2629828693 shutdown_connections 2024-09-10T08:42:12.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:12.122+0000 7ffa39800640 1 -- 172.21.15.89:0/2629828693 wait complete. 2024-09-10T08:42:12.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:12 smithi089 ceph-mon[30757]: pgmap v461: 129 pgs: 129 active+clean; 47 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 1.21k op/s 2024-09-10T08:42:12.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:12 smithi149 ceph-mon[36167]: pgmap v461: 129 pgs: 129 active+clean; 47 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 1.21k op/s 2024-09-10T08:42:13.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:12 smithi111 ceph-mon[35220]: pgmap v461: 129 pgs: 129 active+clean; 47 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 1.21k op/s 2024-09-10T08:42:14.069 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:42:14.070 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:42:15.040 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:42:15.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:14 smithi111 ceph-mon[35220]: pgmap v462: 129 pgs: 129 active+clean; 47 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 1.03k op/s 2024-09-10T08:42:15.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:14 smithi089 ceph-mon[30757]: pgmap v462: 129 pgs: 129 active+clean; 47 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 1.03k op/s 2024-09-10T08:42:15.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:14 smithi149 ceph-mon[36167]: pgmap v462: 129 pgs: 129 active+clean; 47 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 1.03k op/s 2024-09-10T08:42:15.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.705+0000 7fe4ca200640 1 -- 172.21.15.89:0/1893385102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c41008d0 msgr2=0x7fe4c4100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:15.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.705+0000 7fe4ca200640 1 --2- 172.21.15.89:0/1893385102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c41008d0 0x7fe4c4100d30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009900 tx=0x7fe4b002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.705+0000 7fe4ca200640 1 -- 172.21.15.89:0/1893385102 shutdown_connections 2024-09-10T08:42:15.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.705+0000 7fe4ca200640 1 --2- 172.21.15.89:0/1893385102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4c4101270 0x7fe4c410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.705+0000 7fe4ca200640 1 --2- 172.21.15.89:0/1893385102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c41008d0 0x7fe4c4100d30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.705+0000 7fe4ca200640 1 --2- 172.21.15.89:0/1893385102 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4c41068d0 0x7fe4c4106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.705+0000 7fe4ca200640 1 -- 172.21.15.89:0/1893385102 >> 172.21.15.89:0/1893385102 conn(0x7fe4c40fc610 msgr2=0x7fe4c40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:15.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.705+0000 7fe4ca200640 1 -- 172.21.15.89:0/1893385102 shutdown_connections 2024-09-10T08:42:15.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.705+0000 7fe4ca200640 1 -- 172.21.15.89:0/1893385102 wait complete. 2024-09-10T08:42:15.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.706+0000 7fe4ca200640 1 Processor -- start 2024-09-10T08:42:15.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.707+0000 7fe4ca200640 1 -- start start 2024-09-10T08:42:15.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.707+0000 7fe4ca200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c41008d0 0x7fe4c419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:15.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.707+0000 7fe4ca200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4c4101270 0x7fe4c419b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:15.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.707+0000 7fe4ca200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4c41068d0 0x7fe4c419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:15.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.707+0000 7fe4ca200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c410fc10 con 0x7fe4c41068d0 2024-09-10T08:42:15.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.707+0000 7fe4ca200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c410fa90 con 0x7fe4c41008d0 2024-09-10T08:42:15.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.707+0000 7fe4ca200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c410fd90 con 0x7fe4c4101270 2024-09-10T08:42:15.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.708+0000 7fe4c8e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c41008d0 0x7fe4c419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:15.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.708+0000 7fe4c3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4c4101270 0x7fe4c419b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:15.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.708+0000 7fe4c8e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c41008d0 0x7fe4c419ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:46192/0 (socket says 172.21.15.89:46192) 2024-09-10T08:42:15.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.708+0000 7fe4c3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4c4101270 0x7fe4c419b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44632/0 (socket says 172.21.15.89:44632) 2024-09-10T08:42:15.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.708+0000 7fe4c8e00640 1 -- 172.21.15.89:0/505813904 learned_addr learned my addr 172.21.15.89:0/505813904 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:15.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.708+0000 7fe4c9800640 1 --2- 172.21.15.89:0/505813904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4c41068d0 0x7fe4c419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:15.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.708+0000 7fe4c3e00640 1 -- 172.21.15.89:0/505813904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c41008d0 msgr2=0x7fe4c419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:15.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.708+0000 7fe4c3e00640 1 --2- 172.21.15.89:0/505813904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c41008d0 0x7fe4c419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.708+0000 7fe4c3e00640 1 -- 172.21.15.89:0/505813904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4c41068d0 msgr2=0x7fe4c419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:15.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.708+0000 7fe4c3e00640 1 --2- 172.21.15.89:0/505813904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4c41068d0 0x7fe4c419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.708+0000 7fe4c3e00640 1 -- 172.21.15.89:0/505813904 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c419f410 con 0x7fe4c4101270 2024-09-10T08:42:15.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.709+0000 7fe4c9800640 1 --2- 172.21.15.89:0/505813904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4c41068d0 0x7fe4c419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:42:15.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.709+0000 7fe4c3e00640 1 --2- 172.21.15.89:0/505813904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4c4101270 0x7fe4c419b0d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe4b002a910 tx=0x7fe4b0003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:15.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.709+0000 7fe4c1600640 1 -- 172.21.15.89:0/505813904 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b003f070 con 0x7fe4c4101270 2024-09-10T08:42:15.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.710+0000 7fe4c1600640 1 -- 172.21.15.89:0/505813904 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b0031c20 con 0x7fe4c4101270 2024-09-10T08:42:15.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.710+0000 7fe4c1600640 1 -- 172.21.15.89:0/505813904 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0035030 con 0x7fe4c4101270 2024-09-10T08:42:15.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.710+0000 7fe4ca200640 1 -- 172.21.15.89:0/505813904 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c419f6a0 con 0x7fe4c4101270 2024-09-10T08:42:15.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.710+0000 7fe4ca200640 1 -- 172.21.15.89:0/505813904 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c419fb30 con 0x7fe4c4101270 2024-09-10T08:42:15.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.713+0000 7fe4ce296740 1 Processor -- start 2024-09-10T08:42:15.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.714+0000 7fe4ce296740 1 -- start start 2024-09-10T08:42:15.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.714+0000 7fe4ce296740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c3bd0 0x5555d79c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:15.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.714+0000 7fe4ce296740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79c44f0 0x5555d79d7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:15.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.714+0000 7fe4ce296740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d76a0 0x5555d79d9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:15.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.714+0000 7fe4ce296740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5555d792a290 con 0x5555d79d76a0 2024-09-10T08:42:15.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.714+0000 7fe4ce296740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5555d78d1b40 con 0x5555d79c3bd0 2024-09-10T08:42:15.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.714+0000 7fe4ce296740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5555d7926250 con 0x5555d79c44f0 2024-09-10T08:42:15.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.715+0000 7fe4c1600640 1 -- 172.21.15.89:0/505813904 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe4b0042070 con 0x7fe4c4101270 2024-09-10T08:42:15.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.715+0000 7fe4c9800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c3bd0 0x5555d79c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:15.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.715+0000 7fe4c9800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c3bd0 0x5555d79c3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:46206/0 (socket says 172.21.15.89:46206) 2024-09-10T08:42:15.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.715+0000 7fe4c9800640 1 -- 172.21.15.89:0/421249654 learned_addr learned my addr 172.21.15.89:0/421249654 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:15.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.715+0000 7fe4c3e00640 1 --2- 172.21.15.89:0/421249654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d76a0 0x5555d79d9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:15.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.715+0000 7fe4c8e00640 1 --2- 172.21.15.89:0/421249654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79c44f0 0x5555d79d7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:15.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.716+0000 7fe4c9800640 1 -- 172.21.15.89:0/421249654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79c44f0 msgr2=0x5555d79d7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:15.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.716+0000 7fe4c9800640 1 --2- 172.21.15.89:0/421249654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79c44f0 0x5555d79d7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.716+0000 7fe4c9800640 1 -- 172.21.15.89:0/421249654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d76a0 msgr2=0x5555d79d9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:15.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.716+0000 7fe4c9800640 1 --2- 172.21.15.89:0/421249654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d76a0 0x5555d79d9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.716+0000 7fe4c9800640 1 -- 172.21.15.89:0/421249654 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555d769b6a0 con 0x5555d79c3bd0 2024-09-10T08:42:15.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.716+0000 7fe4c3e00640 1 --2- 172.21.15.89:0/421249654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d76a0 0x5555d79d9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:42:15.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.716+0000 7fe4c9800640 1 --2- 172.21.15.89:0/421249654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c3bd0 0x5555d79c3fb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009f90 tx=0x7fe4bc00f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee598596b54067a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:15.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.716+0000 7fe4b6000640 1 -- 172.21.15.89:0/421249654 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc003b70 con 0x5555d79c3bd0 2024-09-10T08:42:15.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.716+0000 7fe4b6000640 1 -- 172.21.15.89:0/421249654 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4bc003d10 con 0x5555d79c3bd0 2024-09-10T08:42:15.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.716+0000 7fe4b6000640 1 -- 172.21.15.89:0/421249654 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc0119d0 con 0x5555d79c3bd0 2024-09-10T08:42:15.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.717+0000 7fe4c1600640 1 --2- 172.21.15.89:0/505813904 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4ac0a2c00 0x7fe4ac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:15.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.717+0000 7fe4c1600640 1 -- 172.21.15.89:0/505813904 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fe4b00ef930 con 0x7fe4c4101270 2024-09-10T08:42:15.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.717+0000 7fe4c8e00640 1 --2- 172.21.15.89:0/505813904 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4ac0a2c00 0x7fe4ac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:15.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.718+0000 7fe4ce296740 1 -- 172.21.15.89:0/421249654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c3bd0 msgr2=0x5555d79c3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:15.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.718+0000 7fe4ce296740 1 --2- 172.21.15.89:0/421249654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c3bd0 0x5555d79c3fb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009f90 tx=0x7fe4bc00f7f0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.718+0000 7fe4c8e00640 1 --2- 172.21.15.89:0/505813904 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4ac0a2c00 0x7fe4ac0a50c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009620 tx=0x7fe4b8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:15.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.718+0000 7fe4ce296740 1 -- 172.21.15.89:0/421249654 shutdown_connections 2024-09-10T08:42:15.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.718+0000 7fe4ce296740 1 --2- 172.21.15.89:0/421249654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d76a0 0x5555d79d9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.718+0000 7fe4ce296740 1 --2- 172.21.15.89:0/421249654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79c44f0 0x5555d79d7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.718+0000 7fe4ce296740 1 --2- 172.21.15.89:0/421249654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c3bd0 0x5555d79c3fb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.718+0000 7fe4ce296740 1 -- 172.21.15.89:0/421249654 >> 172.21.15.89:0/421249654 conn(0x5555d79c0240 msgr2=0x5555d79c0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:15.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.718+0000 7fe4ce296740 1 -- 172.21.15.89:0/421249654 shutdown_connections 2024-09-10T08:42:15.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.719+0000 7fe4ce296740 1 -- 172.21.15.89:0/421249654 wait complete. 2024-09-10T08:42:15.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.719+0000 7fe4ce296740 1 -- start start 2024-09-10T08:42:15.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.720+0000 7fe4ce296740 1 Processor -- start 2024-09-10T08:42:15.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.720+0000 7fe4ce296740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79c3bd0 0x5555d79d7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:15.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.720+0000 7fe4ce296740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c44f0 0x5555d79d42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:15.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.720+0000 7fe4ce296740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d76a0 0x5555d79d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:15.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.720+0000 7fe4ce296740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5555d79da2b0 con 0x5555d79d76a0 2024-09-10T08:42:15.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.720+0000 7fe4ce296740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5555d78e4fa0 con 0x5555d79c44f0 2024-09-10T08:42:15.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.720+0000 7fe4ce296740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5555d79da430 con 0x5555d79c3bd0 2024-09-10T08:42:15.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.720+0000 7fe4ce296740 10 client.0 _ll_register_callbacks cb 0x5555d7638ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:42:15.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.721+0000 7fe4c3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c44f0 0x5555d79d42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:15.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.721+0000 7fe4c3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c44f0 0x5555d79d42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:46214/0 (socket says 172.21.15.89:46214) 2024-09-10T08:42:15.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.721+0000 7fe4c3e00640 1 -- 172.21.15.89:0/1529753717 learned_addr learned my addr 172.21.15.89:0/1529753717 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:15.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.721+0000 7fe4c9800640 1 --2- 172.21.15.89:0/1529753717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d76a0 0x5555d79d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:15.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.721+0000 7fe4c3e00640 1 -- 172.21.15.89:0/1529753717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79c3bd0 msgr2=0x5555d79d7010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:42:15.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.721+0000 7fe4c3e00640 1 --2- 172.21.15.89:0/1529753717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79c3bd0 0x5555d79d7010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.721+0000 7fe4c3e00640 1 -- 172.21.15.89:0/1529753717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d76a0 msgr2=0x5555d79d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:15.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.721+0000 7fe4c3e00640 1 --2- 172.21.15.89:0/1529753717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d76a0 0x5555d79d4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.721+0000 7fe4c3e00640 1 -- 172.21.15.89:0/1529753717 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555d79d5050 con 0x5555d79c44f0 2024-09-10T08:42:15.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.721+0000 7fe4c9800640 1 --2- 172.21.15.89:0/1529753717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d76a0 0x5555d79d4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:42:15.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.721+0000 7fe4c3e00640 1 --2- 172.21.15.89:0/1529753717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c44f0 0x5555d79d42c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00c4500 tx=0x7fe4b00092f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a44e593b9651f36d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:15.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.721+0000 7fe4aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:42:15.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.722+0000 7fe4aaa00640 1 -- 172.21.15.89:0/1529753717 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b00bb900 con 0x5555d79c44f0 2024-09-10T08:42:15.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.722+0000 7fe4aaa00640 1 -- 172.21.15.89:0/1529753717 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fe4b003b460 con 0x5555d79c44f0 2024-09-10T08:42:15.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.722+0000 7fe4ce296740 1 -- 172.21.15.89:0/1529753717 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4bc003d10 con 0x5555d79c44f0 2024-09-10T08:42:15.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.722+0000 7fe4aaa00640 1 -- 172.21.15.89:0/1529753717 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe4b00bb720 con 0x5555d79c44f0 2024-09-10T08:42:15.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.722+0000 7fe4ce296740 10 client.25931 fetch_fsmap learned FSMap version 28 2024-09-10T08:42:15.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.722+0000 7fe4ce296740 1 -- 172.21.15.89:0/1529753717 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5555d79e6230 con 0x5555d79c44f0 2024-09-10T08:42:15.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.723+0000 7fe4aaa00640 1 -- 172.21.15.89:0/1529753717 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4bc003d10 con 0x5555d79c44f0 2024-09-10T08:42:15.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.723+0000 7fe4aaa00640 1 -- 172.21.15.89:0/1529753717 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe4b00b8430 con 0x5555d79c44f0 2024-09-10T08:42:15.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.723+0000 7fe4ce296740 10 client.25931 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:42:15.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.723+0000 7fe4ce296740 10 client.25931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:42:15.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.723+0000 7fe4ce296740 20 client.25931 populate_metadata read hostname 'smithi089' 2024-09-10T08:42:15.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.723+0000 7fe4ce296740 1 --2- 172.21.15.89:0/1529753717 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5555d79e76e0 0x5555d79e9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:15.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.723+0000 7fe4ce296740 4 client.25931 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:42:15.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.723+0000 7fe4ce296740 1 -- 172.21.15.89:0/1529753717 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555d7937800 con 0x5555d79e76e0 2024-09-10T08:42:15.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.725+0000 7fe4c8e00640 1 --2- 172.21.15.89:0/1529753717 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5555d79e76e0 0x5555d79e9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:15.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.726+0000 7fe4c8e00640 1 --2- 172.21.15.89:0/1529753717 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5555d79e76e0 0x5555d79e9ba0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=418f5f7bd14f160b server_cookie=a62625640c2bb286 in_seq=0 out_seq=0 2024-09-10T08:42:15.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.726+0000 7fe4aaa00640 10 client.25931 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:15.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.733+0000 7fe4aaa00640 1 -- 172.21.15.89:0/1529753717 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555d7937800 con 0x5555d79e76e0 2024-09-10T08:42:15.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.733+0000 7fe4aaa00640 10 client.25931 handle_command_reply: tid=0 2024-09-10T08:42:15.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.733+0000 7fe4ce296740 1 client.25931 shutdown 2024-09-10T08:42:15.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.733+0000 7fe4ce296740 1 -- 172.21.15.89:0/1529753717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c44f0 msgr2=0x5555d79d42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:15.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.733+0000 7fe4ce296740 1 --2- 172.21.15.89:0/1529753717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c44f0 0x5555d79d42c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00c4500 tx=0x7fe4b00092f0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.733+0000 7fe4ce296740 1 -- 172.21.15.89:0/1529753717 shutdown_connections 2024-09-10T08:42:15.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.733+0000 7fe4ce296740 1 --2- 172.21.15.89:0/1529753717 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5555d79e76e0 0x5555d79e9ba0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.733+0000 7fe4ce296740 1 --2- 172.21.15.89:0/1529753717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d76a0 0x5555d79d4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.733+0000 7fe4ce296740 1 --2- 172.21.15.89:0/1529753717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c44f0 0x5555d79d42c0 secure :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00c4500 tx=0x7fe4b00092f0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.733+0000 7fe4aaa00640 0 client.25931 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:15.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.733+0000 7fe4ce296740 1 --2- 172.21.15.89:0/1529753717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79c3bd0 0x5555d79d7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.733+0000 7fe4ce296740 1 -- 172.21.15.89:0/1529753717 >> 172.21.15.89:0/1529753717 conn(0x5555d79c0240 msgr2=0x5555d79c5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:15.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.735+0000 7fe4ce296740 1 -- 172.21.15.89:0/1529753717 shutdown_connections 2024-09-10T08:42:15.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.735+0000 7fe4ce296740 1 -- 172.21.15.89:0/1529753717 wait complete. 2024-09-10T08:42:15.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.735+0000 7fe4ce296740 20 client.25931 trim_cache size 0 max 16384 2024-09-10T08:42:15.846 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:42:15.846 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2024-09-10T08:42:15.847 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "257ffa30-7c72-4daa-b8b0-bbf7a86410dd", 2024-09-10T08:42:15.847 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2024-09-10T08:42:15.847 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:42:15.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.795+0000 7fe4ce296740 1 Processor -- start 2024-09-10T08:42:15.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.796+0000 7fe4ce296740 1 -- start start 2024-09-10T08:42:15.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.796+0000 7fe4ce296740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c3bd0 0x5555d79bf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:15.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.796+0000 7fe4ce296740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79d76a0 0x5555d79d6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:15.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.796+0000 7fe4ce296740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d7070 0x5555d79ef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:15.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.796+0000 7fe4ce296740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41af250 con 0x5555d79d7070 2024-09-10T08:42:15.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.796+0000 7fe4ce296740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41acb20 con 0x5555d79c3bd0 2024-09-10T08:42:15.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.796+0000 7fe4ce296740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41aa9d0 con 0x5555d79d76a0 2024-09-10T08:42:15.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.797+0000 7fe4c3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79d76a0 0x5555d79d6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:15.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.797+0000 7fe4c8e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c3bd0 0x5555d79bf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:15.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.797+0000 7fe4c3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79d76a0 0x5555d79d6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44670/0 (socket says 172.21.15.89:44670) 2024-09-10T08:42:15.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.797+0000 7fe4c3e00640 1 -- 172.21.15.89:0/1362278096 learned_addr learned my addr 172.21.15.89:0/1362278096 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:15.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.797+0000 7fe4c9800640 1 --2- 172.21.15.89:0/1362278096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d7070 0x5555d79ef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:15.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.798+0000 7fe4c3e00640 1 -- 172.21.15.89:0/1362278096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c3bd0 msgr2=0x5555d79bf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:15.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.798+0000 7fe4c3e00640 1 --2- 172.21.15.89:0/1362278096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c3bd0 0x5555d79bf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.798+0000 7fe4c3e00640 1 -- 172.21.15.89:0/1362278096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d7070 msgr2=0x5555d79ef4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:15.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.801+0000 7fe4c3e00640 1 --2- 172.21.15.89:0/1362278096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d7070 0x5555d79ef4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.801+0000 7fe4c3e00640 1 -- 172.21.15.89:0/1362278096 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555d79ea220 con 0x5555d79d76a0 2024-09-10T08:42:15.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.801+0000 7fe4c9800640 1 --2- 172.21.15.89:0/1362278096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d7070 0x5555d79ef4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:42:15.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.801+0000 7fe4c3e00640 1 --2- 172.21.15.89:0/1362278096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79d76a0 0x5555d79d6b30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00bbd50 tx=0x7fe4b0009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a137ed72b8aeed28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:15.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.802+0000 7fe4b5600640 1 -- 172.21.15.89:0/1362278096 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b00bb900 con 0x5555d79d76a0 2024-09-10T08:42:15.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.802+0000 7fe4b5600640 1 -- 172.21.15.89:0/1362278096 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b00bc4c0 con 0x5555d79d76a0 2024-09-10T08:42:15.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.802+0000 7fe4b5600640 1 -- 172.21.15.89:0/1362278096 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b00f5280 con 0x5555d79d76a0 2024-09-10T08:42:15.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.803+0000 7fe4ce296740 1 -- 172.21.15.89:0/1362278096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79d76a0 msgr2=0x5555d79d6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:15.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.803+0000 7fe4ce296740 1 --2- 172.21.15.89:0/1362278096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79d76a0 0x5555d79d6b30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00bbd50 tx=0x7fe4b0009620 comp rx=0 tx=0).stop 2024-09-10T08:42:15.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.803+0000 7fe4ce296740 1 -- 172.21.15.89:0/1362278096 shutdown_connections 2024-09-10T08:42:15.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.803+0000 7fe4ce296740 1 --2- 172.21.15.89:0/1362278096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d7070 0x5555d79ef4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.803+0000 7fe4ce296740 1 --2- 172.21.15.89:0/1362278096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79d76a0 0x5555d79d6b30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.803+0000 7fe4ce296740 1 --2- 172.21.15.89:0/1362278096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c3bd0 0x5555d79bf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.803+0000 7fe4ce296740 1 -- 172.21.15.89:0/1362278096 >> 172.21.15.89:0/1362278096 conn(0x5555d79c0240 msgr2=0x5555d79bea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:15.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.803+0000 7fe4ce296740 1 -- 172.21.15.89:0/1362278096 shutdown_connections 2024-09-10T08:42:15.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.803+0000 7fe4ce296740 1 -- 172.21.15.89:0/1362278096 wait complete. 2024-09-10T08:42:15.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.804+0000 7fe4ce296740 1 -- start start 2024-09-10T08:42:15.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.804+0000 7fe4ce296740 1 Processor -- start 2024-09-10T08:42:15.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.806+0000 7fe4ce296740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c3bd0 0x5555d79dffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:15.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.806+0000 7fe4ce296740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d7070 0x5555d79dc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:15.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.806+0000 7fe4ce296740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79d76a0 0x5555d79dcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:15.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.807+0000 7fe4ce296740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41ab020 con 0x5555d79d7070 2024-09-10T08:42:15.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.807+0000 7fe4ce296740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41ac450 con 0x5555d79c3bd0 2024-09-10T08:42:15.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.807+0000 7fe4ce296740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41ab6b0 con 0x5555d79d76a0 2024-09-10T08:42:15.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.807+0000 7fe4ce296740 10 client.0 _ll_register_callbacks cb 0x5555d79d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:42:15.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.807+0000 7fe4c3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d7070 0x5555d79dc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:15.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.807+0000 7fe4c8e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c3bd0 0x5555d79dffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:15.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.807+0000 7fe4c9800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79d76a0 0x5555d79dcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:15.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.807+0000 7fe4c8e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c3bd0 0x5555d79dffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:46224/0 (socket says 172.21.15.89:46224) 2024-09-10T08:42:15.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.807+0000 7fe4c8e00640 1 -- 172.21.15.89:0/195986203 learned_addr learned my addr 172.21.15.89:0/195986203 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:15.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.807+0000 7fe4c3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d7070 0x5555d79dc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51788/0 (socket says 172.21.15.89:51788) 2024-09-10T08:42:15.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.809+0000 7fe4c3e00640 1 -- 172.21.15.89:0/195986203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79d76a0 msgr2=0x5555d79dcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:15.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.809+0000 7fe4c3e00640 1 --2- 172.21.15.89:0/195986203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79d76a0 0x5555d79dcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.809+0000 7fe4c3e00640 1 -- 172.21.15.89:0/195986203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c3bd0 msgr2=0x5555d79dffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:15.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.809+0000 7fe4c3e00640 1 --2- 172.21.15.89:0/195986203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c3bd0 0x5555d79dffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.809+0000 7fe4c3e00640 1 -- 172.21.15.89:0/195986203 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555d79be880 con 0x5555d79d7070 2024-09-10T08:42:15.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.809+0000 7fe4c8e00640 1 --2- 172.21.15.89:0/195986203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c3bd0 0x5555d79dffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:42:15.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.810+0000 7fe4c3e00640 1 --2- 172.21.15.89:0/195986203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d7070 0x5555d79dc690 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x5555d79da410 tx=0x7fe4b0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfc4baf05ab3da7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:15.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.810+0000 7fe4aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:42:15.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.810+0000 7fe4aaa00640 1 -- 172.21.15.89:0/195986203 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b00f5280 con 0x5555d79d7070 2024-09-10T08:42:15.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.810+0000 7fe4ce296740 1 -- 172.21.15.89:0/195986203 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4b00bc4c0 con 0x5555d79d7070 2024-09-10T08:42:15.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.810+0000 7fe4aaa00640 1 -- 172.21.15.89:0/195986203 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fe4b00bb260 con 0x5555d79d7070 2024-09-10T08:42:15.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.811+0000 7fe4aaa00640 1 -- 172.21.15.89:0/195986203 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe4b00bb900 con 0x5555d79d7070 2024-09-10T08:42:15.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.811+0000 7fe4ce296740 10 client.15867 fetch_fsmap learned FSMap version 28 2024-09-10T08:42:15.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.811+0000 7fe4ce296740 1 -- 172.21.15.89:0/195986203 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5555d79be880 con 0x5555d79d7070 2024-09-10T08:42:15.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.812+0000 7fe4aaa00640 1 -- 172.21.15.89:0/195986203 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b00bc4c0 con 0x5555d79d7070 2024-09-10T08:42:15.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.812+0000 7fe4aaa00640 1 -- 172.21.15.89:0/195986203 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe4b00b7b10 con 0x5555d79d7070 2024-09-10T08:42:15.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.812+0000 7fe4ce296740 10 client.15867 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:42:15.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.812+0000 7fe4ce296740 10 client.15867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:42:15.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.812+0000 7fe4ce296740 20 client.15867 populate_metadata read hostname 'smithi089' 2024-09-10T08:42:15.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.812+0000 7fe4ce296740 1 --2- 172.21.15.89:0/195986203 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5555d7a53760 0x5555d7a55b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:15.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.812+0000 7fe4ce296740 4 client.15867 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:42:15.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.812+0000 7fe4ce296740 1 -- 172.21.15.89:0/195986203 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5555d7931d10 con 0x5555d7a53760 2024-09-10T08:42:15.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.813+0000 7fe4c8e00640 1 --2- 172.21.15.89:0/195986203 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5555d7a53760 0x5555d7a55b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:15.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.813+0000 7fe4c8e00640 1 --2- 172.21.15.89:0/195986203 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5555d7a53760 0x5555d7a55b80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74b76ae218b84b4b server_cookie=f25a9c15ec142b17 in_seq=0 out_seq=0 2024-09-10T08:42:15.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.813+0000 7fe4aaa00640 10 client.15867 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:15.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.815+0000 7fe4aaa00640 1 -- 172.21.15.89:0/195986203 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5555d7931d10 con 0x5555d7a53760 2024-09-10T08:42:15.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.815+0000 7fe4aaa00640 10 client.15867 handle_command_reply: tid=0 2024-09-10T08:42:15.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.815+0000 7fe4ce296740 1 client.15867 shutdown 2024-09-10T08:42:15.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.815+0000 7fe4ce296740 1 -- 172.21.15.89:0/195986203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d7070 msgr2=0x5555d79dc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:15.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.815+0000 7fe4ce296740 1 --2- 172.21.15.89:0/195986203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d7070 0x5555d79dc690 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x5555d79da410 tx=0x7fe4b0005b50 comp rx=0 tx=0).stop 2024-09-10T08:42:15.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.816+0000 7fe4ce296740 1 -- 172.21.15.89:0/195986203 shutdown_connections 2024-09-10T08:42:15.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.816+0000 7fe4ce296740 1 --2- 172.21.15.89:0/195986203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555d79d76a0 0x5555d79dcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.816+0000 7fe4ce296740 1 --2- 172.21.15.89:0/195986203 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5555d7a53760 0x5555d7a55b80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.816+0000 7fe4ce296740 1 --2- 172.21.15.89:0/195986203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555d79d7070 0x5555d79dc690 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.853 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.816+0000 7fe4ce296740 1 --2- 172.21.15.89:0/195986203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5555d79c3bd0 0x5555d79dffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.816+0000 7fe4ce296740 1 -- 172.21.15.89:0/195986203 >> 172.21.15.89:0/195986203 conn(0x5555d79c0240 msgr2=0x5555d79bf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:15.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.816+0000 7fe4aaa00640 0 client.15867 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:15.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.816+0000 7fe4ce296740 1 -- 172.21.15.89:0/195986203 shutdown_connections 2024-09-10T08:42:15.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.816+0000 7fe4ce296740 1 -- 172.21.15.89:0/195986203 wait complete. 2024-09-10T08:42:15.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.817+0000 7fe4ce296740 20 client.15867 trim_cache size 0 max 16384 2024-09-10T08:42:15.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.819+0000 7fe4ca200640 1 -- 172.21.15.89:0/505813904 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4ac0a2c00 msgr2=0x7fe4ac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:15.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.819+0000 7fe4ca200640 1 --2- 172.21.15.89:0/505813904 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4ac0a2c00 0x7fe4ac0a50c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009620 tx=0x7fe4b8005f90 comp rx=0 tx=0).stop 2024-09-10T08:42:15.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.819+0000 7fe4ca200640 1 -- 172.21.15.89:0/505813904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4c4101270 msgr2=0x7fe4c419b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:15.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.819+0000 7fe4ca200640 1 --2- 172.21.15.89:0/505813904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4c4101270 0x7fe4c419b0d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe4b002a910 tx=0x7fe4b0003b20 comp rx=0 tx=0).stop 2024-09-10T08:42:15.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.820+0000 7fe4ca200640 1 -- 172.21.15.89:0/505813904 shutdown_connections 2024-09-10T08:42:15.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.820+0000 7fe4ca200640 1 --2- 172.21.15.89:0/505813904 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe4ac0a2c00 0x7fe4ac0a50c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.820+0000 7fe4ca200640 1 --2- 172.21.15.89:0/505813904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4c41068d0 0x7fe4c419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.854 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.820+0000 7fe4ca200640 1 --2- 172.21.15.89:0/505813904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4c4101270 0x7fe4c419b0d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.820+0000 7fe4ca200640 1 --2- 172.21.15.89:0/505813904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c41008d0 0x7fe4c419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:15.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.820+0000 7fe4ca200640 1 -- 172.21.15.89:0/505813904 >> 172.21.15.89:0/505813904 conn(0x7fe4c40fc610 msgr2=0x7fe4c41104e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:15.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.820+0000 7fe4ca200640 1 -- 172.21.15.89:0/505813904 shutdown_connections 2024-09-10T08:42:15.855 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:15.820+0000 7fe4ca200640 1 -- 172.21.15.89:0/505813904 wait complete. 2024-09-10T08:42:16.848 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:42:17.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:16 smithi111 ceph-mon[35220]: pgmap v463: 129 pgs: 129 active+clean; 54 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 1.03k op/s 2024-09-10T08:42:17.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:16 smithi111 ceph-mon[35220]: scrub queued for path: / 2024-09-10T08:42:17.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:16 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:17.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:16 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:42:17.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:16 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:42:17.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:16 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:17.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:16 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:42:17.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:16 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:17.060 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:16 smithi089 ceph-mon[30757]: pgmap v463: 129 pgs: 129 active+clean; 54 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 1.03k op/s 2024-09-10T08:42:17.060 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:16 smithi089 ceph-mon[30757]: scrub queued for path: / 2024-09-10T08:42:17.060 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:16 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:17.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:16 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:42:17.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:16 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:42:17.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:16 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:17.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:16 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:42:17.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:16 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:17.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:16 smithi149 ceph-mon[36167]: pgmap v463: 129 pgs: 129 active+clean; 54 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 1.03k op/s 2024-09-10T08:42:17.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:16 smithi149 ceph-mon[36167]: scrub queued for path: / 2024-09-10T08:42:17.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:16 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:17.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:16 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:42:17.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:16 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:42:17.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:16 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:17.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:16 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:42:17.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:16 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:17.969 INFO:tasks.workunit.client.0.smithi089.stdout: 34 518258 447 335615 563 203938 1450 2024-09-10T08:42:18.035 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:17 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:42:18.035 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:17 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:18.035 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:17 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:42:18.035 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:17 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:18.035 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:17 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:42:18.035 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:17 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:18.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:17 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:42:18.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:17 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:18.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:17 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:42:18.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:17 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:18.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:17 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:42:18.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:17 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:18.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:17 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:42:18.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:17 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:18.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:17 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:42:18.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:17 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:18.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:17 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:42:18.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:17 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:19.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:18 smithi111 ceph-mon[35220]: pgmap v464: 129 pgs: 129 active+clean; 54 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 846 op/s 2024-09-10T08:42:19.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:18 smithi089 ceph-mon[30757]: pgmap v464: 129 pgs: 129 active+clean; 54 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 846 op/s 2024-09-10T08:42:19.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:18 smithi149 ceph-mon[36167]: pgmap v464: 129 pgs: 129 active+clean; 54 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 846 op/s 2024-09-10T08:42:19.755 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:42:19.957 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:19 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:19 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:20.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:19 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:42:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.301+0000 7f3ad3000640 1 -- 172.21.15.89:0/1376629584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3acc072d20 msgr2=0x7f3acc073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.301+0000 7f3ad3000640 1 --2- 172.21.15.89:0/1376629584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3acc072d20 0x7f3acc073180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009900 tx=0x7f3abc02a440 comp rx=0 tx=0).stop 2024-09-10T08:42:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.305+0000 7f3ad3000640 1 -- 172.21.15.89:0/1376629584 shutdown_connections 2024-09-10T08:42:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.305+0000 7f3ad3000640 1 --2- 172.21.15.89:0/1376629584 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3acc0737d0 0x7f3acc112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.305+0000 7f3ad3000640 1 --2- 172.21.15.89:0/1376629584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3acc072d20 0x7f3acc073180 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.305+0000 7f3ad3000640 1 --2- 172.21.15.89:0/1376629584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3acc109930 0x7f3acc109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.305+0000 7f3ad3000640 1 -- 172.21.15.89:0/1376629584 >> 172.21.15.89:0/1376629584 conn(0x7f3acc06ece0 msgr2=0x7f3acc071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.306+0000 7f3ad3000640 1 -- 172.21.15.89:0/1376629584 shutdown_connections 2024-09-10T08:42:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.307+0000 7f3ad3000640 1 -- 172.21.15.89:0/1376629584 wait complete. 2024-09-10T08:42:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.313+0000 7f3ad3000640 1 Processor -- start 2024-09-10T08:42:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.314+0000 7f3ad3000640 1 -- start start 2024-09-10T08:42:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.314+0000 7f3ad3000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3acc072d20 0x7f3acc11ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.315+0000 7f3ad3000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3acc0737d0 0x7f3acc11b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.315+0000 7f3acb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3acc0737d0 0x7f3acc11b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.315+0000 7f3acb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3acc0737d0 0x7f3acc11b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:46236/0 (socket says 172.21.15.89:46236) 2024-09-10T08:42:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.315+0000 7f3acb400640 1 -- 172.21.15.89:0/1156775083 learned_addr learned my addr 172.21.15.89:0/1156775083 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.315+0000 7f3ad3000640 1 --2- 172.21.15.89:0/1156775083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3acc109930 0x7f3acc119440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.316+0000 7f3ad1000640 1 --2- 172.21.15.89:0/1156775083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3acc109930 0x7f3acc119440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.316+0000 7f3ad3000640 1 -- 172.21.15.89:0/1156775083 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc113dc0 con 0x7f3acc072d20 2024-09-10T08:42:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.316+0000 7f3ad3000640 1 -- 172.21.15.89:0/1156775083 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc113c40 con 0x7f3acc0737d0 2024-09-10T08:42:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.316+0000 7f3ad3000640 1 -- 172.21.15.89:0/1156775083 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc113f40 con 0x7f3acc109930 2024-09-10T08:42:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.321+0000 7f3acbe00640 1 --2- 172.21.15.89:0/1156775083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3acc072d20 0x7f3acc11ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.321+0000 7f3acb400640 1 -- 172.21.15.89:0/1156775083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3acc109930 msgr2=0x7f3acc119440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.321+0000 7f3acb400640 1 --2- 172.21.15.89:0/1156775083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3acc109930 0x7f3acc119440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.321+0000 7f3acb400640 1 -- 172.21.15.89:0/1156775083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3acc072d20 msgr2=0x7f3acc11ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.321+0000 7f3acb400640 1 --2- 172.21.15.89:0/1156775083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3acc072d20 0x7f3acc11ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.321+0000 7f3acb400640 1 -- 172.21.15.89:0/1156775083 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3acc119b70 con 0x7f3acc0737d0 2024-09-10T08:42:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.322+0000 7f3acb400640 1 --2- 172.21.15.89:0/1156775083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3acc0737d0 0x7f3acc11b340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3abc005ab0 tx=0x7f3abc032c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.323+0000 7f3ac8c00640 1 -- 172.21.15.89:0/1156775083 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc03f030 con 0x7f3acc0737d0 2024-09-10T08:42:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.323+0000 7f3ad3000640 1 -- 172.21.15.89:0/1156775083 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3acc119e00 con 0x7f3acc0737d0 2024-09-10T08:42:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.323+0000 7f3ad3000640 1 -- 172.21.15.89:0/1156775083 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3acc1b79b0 con 0x7f3acc0737d0 2024-09-10T08:42:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.323+0000 7f3ac8c00640 1 -- 172.21.15.89:0/1156775083 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3abc032e00 con 0x7f3acc0737d0 2024-09-10T08:42:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.324+0000 7f3ac8c00640 1 -- 172.21.15.89:0/1156775083 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc035070 con 0x7f3acc0737d0 2024-09-10T08:42:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.330+0000 7f3ac8c00640 1 -- 172.21.15.89:0/1156775083 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3abc042030 con 0x7f3acc0737d0 2024-09-10T08:42:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.336+0000 7f3ac8c00640 1 --2- 172.21.15.89:0/1156775083 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3ab00a2e60 0x7f3ab00a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.336+0000 7f3ac8c00640 1 -- 172.21.15.89:0/1156775083 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f3abc0ef950 con 0x7f3acc0737d0 2024-09-10T08:42:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.337+0000 7f3acbe00640 1 --2- 172.21.15.89:0/1156775083 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3ab00a2e60 0x7f3ab00a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.341+0000 7f3acbe00640 1 --2- 172.21.15.89:0/1156775083 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3ab00a2e60 0x7f3ab00a5320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8006fd0 tx=0x7f3ab80090e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.347+0000 7f3ad5fc8740 1 Processor -- start 2024-09-10T08:42:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.347+0000 7f3ad5fc8740 1 -- start start 2024-09-10T08:42:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.347+0000 7f3ad5fc8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 0x5579a8705490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.351+0000 7f3ad5fc8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a87059d0 0x5579a870a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.351+0000 7f3ad5fc8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a870aa60 0x5579a870cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.351+0000 7f3ad5fc8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5579a866a100 con 0x5579a870aa60 2024-09-10T08:42:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.351+0000 7f3ad5fc8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5579a86121b0 con 0x5579a87059d0 2024-09-10T08:42:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.351+0000 7f3ad5fc8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5579a86660c0 con 0x5579a8705090 2024-09-10T08:42:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.351+0000 7f3acbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 0x5579a8705490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.351+0000 7f3acbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 0x5579a8705490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44708/0 (socket says 172.21.15.89:44708) 2024-09-10T08:42:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.351+0000 7f3acbe00640 1 -- 172.21.15.89:0/2958682619 learned_addr learned my addr 172.21.15.89:0/2958682619 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.352+0000 7f3acbe00640 1 -- 172.21.15.89:0/2958682619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a87059d0 msgr2=0x5579a870a490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:42:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.353+0000 7f3acbe00640 1 --2- 172.21.15.89:0/2958682619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a87059d0 0x5579a870a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.353+0000 7f3acbe00640 1 -- 172.21.15.89:0/2958682619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a870aa60 msgr2=0x5579a870cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.353+0000 7f3acbe00640 1 --2- 172.21.15.89:0/2958682619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a870aa60 0x5579a870cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.353+0000 7f3acbe00640 1 -- 172.21.15.89:0/2958682619 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579a837d210 con 0x5579a8705090 2024-09-10T08:42:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.353+0000 7f3acbe00640 1 --2- 172.21.15.89:0/2958682619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 0x5579a8705490 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8006dc0 tx=0x7f3ab8011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f95e7db6efa8cdd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.354+0000 7f3ac1600640 1 -- 172.21.15.89:0/2958682619 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab801b040 con 0x5579a8705090 2024-09-10T08:42:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.354+0000 7f3ac1600640 1 -- 172.21.15.89:0/2958682619 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab8024070 con 0x5579a8705090 2024-09-10T08:42:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.354+0000 7f3ac1600640 1 -- 172.21.15.89:0/2958682619 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab801f650 con 0x5579a8705090 2024-09-10T08:42:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.355+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/2958682619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 msgr2=0x5579a8705490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.355+0000 7f3ad5fc8740 1 --2- 172.21.15.89:0/2958682619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 0x5579a8705490 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8006dc0 tx=0x7f3ab8011000 comp rx=0 tx=0).stop 2024-09-10T08:42:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.355+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/2958682619 shutdown_connections 2024-09-10T08:42:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.356+0000 7f3ad5fc8740 1 --2- 172.21.15.89:0/2958682619 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a870aa60 0x5579a870cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.356+0000 7f3ad5fc8740 1 --2- 172.21.15.89:0/2958682619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a87059d0 0x5579a870a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.356+0000 7f3ad5fc8740 1 --2- 172.21.15.89:0/2958682619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 0x5579a8705490 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.356+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/2958682619 >> 172.21.15.89:0/2958682619 conn(0x5579a8700700 msgr2=0x5579a8700b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.367+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/2958682619 shutdown_connections 2024-09-10T08:42:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.367+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/2958682619 wait complete. 2024-09-10T08:42:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.368+0000 7f3ad5fc8740 1 -- start start 2024-09-10T08:42:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.368+0000 7f3ad5fc8740 1 Processor -- start 2024-09-10T08:42:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.369+0000 7f3ad5fc8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 0x5579a870f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.369+0000 7f3ad5fc8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a87059d0 0x5579a870f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.369+0000 7f3ad5fc8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a870aa60 0x5579a87168b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.369+0000 7f3ad5fc8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5579a870da30 con 0x5579a87059d0 2024-09-10T08:42:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.369+0000 7f3ad5fc8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5579a85f3090 con 0x5579a870aa60 2024-09-10T08:42:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.369+0000 7f3ad5fc8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5579a870dbb0 con 0x5579a8705090 2024-09-10T08:42:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.370+0000 7f3ad5fc8740 10 client.0 _ll_register_callbacks cb 0x5579a8647f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:42:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.370+0000 7f3acb400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a87059d0 0x5579a870f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.370+0000 7f3acbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 0x5579a870f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.370+0000 7f3acbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 0x5579a870f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44714/0 (socket says 172.21.15.89:44714) 2024-09-10T08:42:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.370+0000 7f3acbe00640 1 -- 172.21.15.89:0/720531822 learned_addr learned my addr 172.21.15.89:0/720531822 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.371+0000 7f3acbe00640 1 -- 172.21.15.89:0/720531822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a870aa60 msgr2=0x5579a87168b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:42:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.371+0000 7f3acbe00640 1 --2- 172.21.15.89:0/720531822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a870aa60 0x5579a87168b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.371+0000 7f3acbe00640 1 -- 172.21.15.89:0/720531822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a87059d0 msgr2=0x5579a870f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.371+0000 7f3acbe00640 1 --2- 172.21.15.89:0/720531822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a87059d0 0x5579a870f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.371+0000 7f3acbe00640 1 -- 172.21.15.89:0/720531822 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579a8449620 con 0x5579a8705090 2024-09-10T08:42:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.372+0000 7f3ab6000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:42:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.372+0000 7f3acbe00640 1 --2- 172.21.15.89:0/720531822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 0x5579a870f330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8017010 tx=0x7f3ab8013e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fe9c66bba1db624 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.373+0000 7f3ab6000640 1 -- 172.21.15.89:0/720531822 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8013860 con 0x5579a8705090 2024-09-10T08:42:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.373+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/720531822 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ab8024070 con 0x5579a8705090 2024-09-10T08:42:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.373+0000 7f3ab6000640 1 -- 172.21.15.89:0/720531822 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f3ab8013660 con 0x5579a8705090 2024-09-10T08:42:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.374+0000 7f3ab6000640 1 -- 172.21.15.89:0/720531822 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3ab80283f0 con 0x5579a8705090 2024-09-10T08:42:20.377 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.376+0000 7f3ad5fc8740 10 client.25879 fetch_fsmap learned FSMap version 28 2024-09-10T08:42:20.377 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.376+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/720531822 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5579a873b700 con 0x5579a8705090 2024-09-10T08:42:20.378 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.376+0000 7f3ab6000640 1 -- 172.21.15.89:0/720531822 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab8024070 con 0x5579a8705090 2024-09-10T08:42:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.377+0000 7f3ab6000640 1 -- 172.21.15.89:0/720531822 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3ab801e030 con 0x5579a8705090 2024-09-10T08:42:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.378+0000 7f3ad5fc8740 10 client.25879 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:42:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.378+0000 7f3ad5fc8740 10 client.25879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:42:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.378+0000 7f3ad5fc8740 20 client.25879 populate_metadata read hostname 'smithi089' 2024-09-10T08:42:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.378+0000 7f3ad5fc8740 1 --2- 172.21.15.89:0/720531822 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5579a873d1f0 0x5579a873f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:20.379 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.378+0000 7f3ad5fc8740 4 client.25879 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:42:20.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.378+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/720531822 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579a8677830 con 0x5579a873d1f0 2024-09-10T08:42:20.381 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.381+0000 7f3acb400640 1 --2- 172.21.15.89:0/720531822 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5579a873d1f0 0x5579a873f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:20.382 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.381+0000 7f3ab6000640 10 client.25879 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:20.383 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.381+0000 7f3acb400640 1 --2- 172.21.15.89:0/720531822 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5579a873d1f0 0x5579a873f670 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48e42aa753c91aa3 server_cookie=2e790733ddac6400 in_seq=0 out_seq=0 2024-09-10T08:42:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.382+0000 7f3ab6000640 1 -- 172.21.15.89:0/720531822 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5579a8677830 con 0x5579a873d1f0 2024-09-10T08:42:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.382+0000 7f3ab6000640 10 client.25879 handle_command_reply: tid=0 2024-09-10T08:42:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.382+0000 7f3ad5fc8740 1 client.25879 shutdown 2024-09-10T08:42:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.383+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/720531822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 msgr2=0x5579a870f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.383+0000 7f3ad5fc8740 1 --2- 172.21.15.89:0/720531822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 0x5579a870f330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8017010 tx=0x7f3ab8013e40 comp rx=0 tx=0).stop 2024-09-10T08:42:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.383+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/720531822 shutdown_connections 2024-09-10T08:42:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.383+0000 7f3ad5fc8740 1 --2- 172.21.15.89:0/720531822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a870aa60 0x5579a87168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.383+0000 7f3ad5fc8740 1 --2- 172.21.15.89:0/720531822 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5579a873d1f0 0x5579a873f670 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.383+0000 7f3ad5fc8740 1 --2- 172.21.15.89:0/720531822 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a87059d0 0x5579a870f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.383+0000 7f3ad5fc8740 1 --2- 172.21.15.89:0/720531822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 0x5579a870f330 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.383+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/720531822 >> 172.21.15.89:0/720531822 conn(0x5579a8700700 msgr2=0x5579a8701f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.383+0000 7f3ab6000640 0 client.25879 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.383+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/720531822 shutdown_connections 2024-09-10T08:42:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.383+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/720531822 wait complete. 2024-09-10T08:42:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.384+0000 7f3ad5fc8740 20 client.25879 trim_cache size 0 max 16384 2024-09-10T08:42:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.420+0000 7f3ad5fc8740 1 Processor -- start 2024-09-10T08:42:20.421 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.421+0000 7f3ad5fc8740 1 -- start start 2024-09-10T08:42:20.423 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.422+0000 7f3ad5fc8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a8705090 0x5579a86ff770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.423+0000 7f3ad5fc8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a87059d0 0x5579a87167d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.423+0000 7f3ad5fc8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a870aa60 0x5579a8743f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.423+0000 7f3ad5fc8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac012450 con 0x5579a87059d0 2024-09-10T08:42:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.423+0000 7f3ad5fc8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac00fd20 con 0x5579a8705090 2024-09-10T08:42:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.423+0000 7f3ad5fc8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac00dc20 con 0x5579a870aa60 2024-09-10T08:42:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.423+0000 7f3acbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a8705090 0x5579a86ff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.423+0000 7f3acbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a8705090 0x5579a86ff770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:46254/0 (socket says 172.21.15.89:46254) 2024-09-10T08:42:20.424 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.423+0000 7f3acbe00640 1 -- 172.21.15.89:0/434706079 learned_addr learned my addr 172.21.15.89:0/434706079 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.429+0000 7f3ad1000640 1 --2- 172.21.15.89:0/434706079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a870aa60 0x5579a8743f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.430+0000 7f3acb400640 1 --2- 172.21.15.89:0/434706079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a87059d0 0x5579a87167d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.430+0000 7f3ad1000640 1 -- 172.21.15.89:0/434706079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a8705090 msgr2=0x5579a86ff770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.430+0000 7f3ad1000640 1 --2- 172.21.15.89:0/434706079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a8705090 0x5579a86ff770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.430+0000 7f3ad1000640 1 -- 172.21.15.89:0/434706079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a87059d0 msgr2=0x5579a87167d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.430+0000 7f3ad1000640 1 --2- 172.21.15.89:0/434706079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a87059d0 0x5579a87167d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.430+0000 7f3ad1000640 1 -- 172.21.15.89:0/434706079 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579a86fbd60 con 0x5579a870aa60 2024-09-10T08:42:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.430+0000 7f3acb400640 1 --2- 172.21.15.89:0/434706079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a87059d0 0x5579a87167d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:42:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.433+0000 7f3ad1000640 1 --2- 172.21.15.89:0/434706079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a870aa60 0x5579a8743f30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4005fd0 tx=0x7f3ac4010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d885c1ad88c9333d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.433+0000 7f3ac0c00640 1 -- 172.21.15.89:0/434706079 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4016070 con 0x5579a870aa60 2024-09-10T08:42:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.434+0000 7f3ac0c00640 1 -- 172.21.15.89:0/434706079 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac4010e90 con 0x5579a870aa60 2024-09-10T08:42:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.434+0000 7f3ac0c00640 1 -- 172.21.15.89:0/434706079 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4012ca0 con 0x5579a870aa60 2024-09-10T08:42:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.435+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/434706079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a870aa60 msgr2=0x5579a8743f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.435+0000 7f3ad5fc8740 1 --2- 172.21.15.89:0/434706079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a870aa60 0x5579a8743f30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4005fd0 tx=0x7f3ac4010740 comp rx=0 tx=0).stop 2024-09-10T08:42:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.438+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/434706079 shutdown_connections 2024-09-10T08:42:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.438+0000 7f3ad5fc8740 1 --2- 172.21.15.89:0/434706079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a870aa60 0x5579a8743f30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.438+0000 7f3ad5fc8740 1 --2- 172.21.15.89:0/434706079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a87059d0 0x5579a87167d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.438+0000 7f3ad5fc8740 1 --2- 172.21.15.89:0/434706079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a8705090 0x5579a86ff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.438+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/434706079 >> 172.21.15.89:0/434706079 conn(0x5579a8700700 msgr2=0x5579a86ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.438+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/434706079 shutdown_connections 2024-09-10T08:42:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.439+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/434706079 wait complete. 2024-09-10T08:42:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.440+0000 7f3ad5fc8740 1 -- start start 2024-09-10T08:42:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.440+0000 7f3ad5fc8740 1 Processor -- start 2024-09-10T08:42:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.441+0000 7f3ad5fc8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 0x5579a8713dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.442+0000 7f3ad5fc8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a87059d0 0x5579a8714300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.442+0000 7f3acbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 0x5579a8713dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.442+0000 7f3acbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 0x5579a8713dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44734/0 (socket says 172.21.15.89:44734) 2024-09-10T08:42:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.442+0000 7f3ad5fc8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a870aa60 0x5579a8715230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.442+0000 7f3acb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a87059d0 0x5579a8714300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.442+0000 7f3acb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a87059d0 0x5579a8714300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:46258/0 (socket says 172.21.15.89:46258) 2024-09-10T08:42:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.442+0000 7f3ad1000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a870aa60 0x5579a8715230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.443+0000 7f3ad1000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a870aa60 0x5579a8715230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51832/0 (socket says 172.21.15.89:51832) 2024-09-10T08:42:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.443+0000 7f3ad5fc8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac00e1e0 con 0x5579a870aa60 2024-09-10T08:42:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.443+0000 7f3ad5fc8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac00f650 con 0x5579a87059d0 2024-09-10T08:42:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.443+0000 7f3ad5fc8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac00e8b0 con 0x5579a8705090 2024-09-10T08:42:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.443+0000 7f3ad5fc8740 10 client.0 _ll_register_callbacks cb 0x7f3ab800eab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:42:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.443+0000 7f3acbe00640 1 -- 172.21.15.89:0/1866446992 learned_addr learned my addr 172.21.15.89:0/1866446992 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.444+0000 7f3acb400640 1 -- 172.21.15.89:0/1866446992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 msgr2=0x5579a8713dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.444+0000 7f3acb400640 1 --2- 172.21.15.89:0/1866446992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 0x5579a8713dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.444+0000 7f3acb400640 1 -- 172.21.15.89:0/1866446992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a870aa60 msgr2=0x5579a8715230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.444+0000 7f3acb400640 1 --2- 172.21.15.89:0/1866446992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a870aa60 0x5579a8715230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.444+0000 7f3acb400640 1 -- 172.21.15.89:0/1866446992 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579a8715840 con 0x5579a87059d0 2024-09-10T08:42:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.444+0000 7f3ad1000640 1 --2- 172.21.15.89:0/1866446992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a870aa60 0x5579a8715230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:42:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.444+0000 7f3acbe00640 1 --2- 172.21.15.89:0/1866446992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 0x5579a8713dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:42:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.445+0000 7f3acb400640 1 --2- 172.21.15.89:0/1866446992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a87059d0 0x5579a8714300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x5579a8706400 tx=0x7f3abc0bbef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a810ea313ccf723 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.445+0000 7f3ab6000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:42:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.445+0000 7f3ab6000640 1 -- 172.21.15.89:0/1866446992 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc049040 con 0x5579a87059d0 2024-09-10T08:42:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.445+0000 7f3ab6000640 1 -- 172.21.15.89:0/1866446992 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f3abc03bd00 con 0x5579a87059d0 2024-09-10T08:42:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.445+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/1866446992 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ac4010e90 con 0x5579a87059d0 2024-09-10T08:42:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.446+0000 7f3ab6000640 1 -- 172.21.15.89:0/1866446992 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3abc0c49f0 con 0x5579a87059d0 2024-09-10T08:42:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.446+0000 7f3ad5fc8740 10 client.25946 fetch_fsmap learned FSMap version 28 2024-09-10T08:42:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.446+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/1866446992 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5579a87a7f10 con 0x5579a87059d0 2024-09-10T08:42:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.446+0000 7f3ab6000640 1 -- 172.21.15.89:0/1866446992 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac4010e90 con 0x5579a87059d0 2024-09-10T08:42:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.447+0000 7f3ab6000640 1 -- 172.21.15.89:0/1866446992 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3abc0c4560 con 0x5579a87059d0 2024-09-10T08:42:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.447+0000 7f3ad5fc8740 10 client.25946 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:42:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.447+0000 7f3ad5fc8740 10 client.25946 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:42:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.448+0000 7f3ad5fc8740 20 client.25946 populate_metadata read hostname 'smithi089' 2024-09-10T08:42:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.448+0000 7f3ad5fc8740 1 --2- 172.21.15.89:0/1866446992 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5579a87a8750 0x5579a87aab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.448+0000 7f3ad5fc8740 4 client.25946 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:42:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.448+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/1866446992 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579a8671b80 con 0x5579a87a8750 2024-09-10T08:42:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.452+0000 7f3acbe00640 1 --2- 172.21.15.89:0/1866446992 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5579a87a8750 0x5579a87aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.454+0000 7f3acbe00640 1 --2- 172.21.15.89:0/1866446992 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5579a87a8750 0x5579a87aab70 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=412049eed92bb3d7 server_cookie=1f6f2d1dfa39c691 in_seq=0 out_seq=0 2024-09-10T08:42:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:20.454+0000 7f3ab6000640 10 client.25946 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:21.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:20 smithi111 ceph-mon[35220]: pgmap v465: 129 pgs: 129 active+clean; 54 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.5 MiB/s wr, 447 op/s 2024-09-10T08:42:21.044 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:20 smithi089 ceph-mon[30757]: pgmap v465: 129 pgs: 129 active+clean; 54 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.5 MiB/s wr, 447 op/s 2024-09-10T08:42:21.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:20 smithi149 ceph-mon[36167]: pgmap v465: 129 pgs: 129 active+clean; 54 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.5 MiB/s wr, 447 op/s 2024-09-10T08:42:21.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.234+0000 7fb649a00640 1 -- 172.21.15.89:0/327156064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb644112a30 msgr2=0x7fb644114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:21.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.234+0000 7fb649a00640 1 --2- 172.21.15.89:0/327156064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb644112a30 0x7fb644114e20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009be0 tx=0x7fb63c02a710 comp rx=0 tx=0).stop 2024-09-10T08:42:21.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.236+0000 7fb649a00640 1 -- 172.21.15.89:0/327156064 shutdown_connections 2024-09-10T08:42:21.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.236+0000 7fb649a00640 1 --2- 172.21.15.89:0/327156064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb644112a30 0x7fb644114e20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.236+0000 7fb649a00640 1 --2- 172.21.15.89:0/327156064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb644077040 0x7fb6441124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.236+0000 7fb649a00640 1 --2- 172.21.15.89:0/327156064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb644076720 0x7fb644076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.236+0000 7fb649a00640 1 -- 172.21.15.89:0/327156064 >> 172.21.15.89:0/327156064 conn(0x7fb6440ffab0 msgr2=0x7fb644101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:21.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.236+0000 7fb649a00640 1 -- 172.21.15.89:0/327156064 shutdown_connections 2024-09-10T08:42:21.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.236+0000 7fb649a00640 1 -- 172.21.15.89:0/327156064 wait complete. 2024-09-10T08:42:21.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.238+0000 7fb649a00640 1 Processor -- start 2024-09-10T08:42:21.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.238+0000 7fb649a00640 1 -- start start 2024-09-10T08:42:21.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.238+0000 7fb649a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb644076720 0x7fb6441a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:21.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.238+0000 7fb649a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb644077040 0x7fb6441a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:21.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.239+0000 7fb649a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb644112a30 0x7fb64419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:21.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.239+0000 7fb649a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb644118570 con 0x7fb644112a30 2024-09-10T08:42:21.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.239+0000 7fb649a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441183f0 con 0x7fb644076720 2024-09-10T08:42:21.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.239+0000 7fb649a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441186f0 con 0x7fb644077040 2024-09-10T08:42:21.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.239+0000 7fb642a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb644076720 0x7fb6441a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:21.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.239+0000 7fb642a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb644076720 0x7fb6441a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:46280/0 (socket says 172.21.15.89:46280) 2024-09-10T08:42:21.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.239+0000 7fb642a00640 1 -- 172.21.15.89:0/2987522534 learned_addr learned my addr 172.21.15.89:0/2987522534 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:21.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.242+0000 7fb642000640 1 --2- 172.21.15.89:0/2987522534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb644077040 0x7fb6441a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:21.244 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.242+0000 7fb643400640 1 --2- 172.21.15.89:0/2987522534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb644112a30 0x7fb64419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:21.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.243+0000 7fb642000640 1 -- 172.21.15.89:0/2987522534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb644076720 msgr2=0x7fb6441a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:21.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.243+0000 7fb642000640 1 --2- 172.21.15.89:0/2987522534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb644076720 0x7fb6441a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.245 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.243+0000 7fb642000640 1 -- 172.21.15.89:0/2987522534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb644112a30 msgr2=0x7fb64419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:21.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.243+0000 7fb642000640 1 --2- 172.21.15.89:0/2987522534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb644112a30 0x7fb64419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.243+0000 7fb642000640 1 -- 172.21.15.89:0/2987522534 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64419f010 con 0x7fb644077040 2024-09-10T08:42:21.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.243+0000 7fb642000640 1 --2- 172.21.15.89:0/2987522534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb644077040 0x7fb6441a4c60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb634005fd0 tx=0x7fb63400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:21.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.244+0000 7fb63b400640 1 -- 172.21.15.89:0/2987522534 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634009c30 con 0x7fb644077040 2024-09-10T08:42:21.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.244+0000 7fb63b400640 1 -- 172.21.15.89:0/2987522534 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb634009dd0 con 0x7fb644077040 2024-09-10T08:42:21.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.244+0000 7fb63b400640 1 -- 172.21.15.89:0/2987522534 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634009c30 con 0x7fb644077040 2024-09-10T08:42:21.246 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.244+0000 7fb649a00640 1 -- 172.21.15.89:0/2987522534 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64419f300 con 0x7fb644077040 2024-09-10T08:42:21.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.244+0000 7fb649a00640 1 -- 172.21.15.89:0/2987522534 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6441ab5f0 con 0x7fb644077040 2024-09-10T08:42:21.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.252+0000 7fb64c5e1740 1 Processor -- start 2024-09-10T08:42:21.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.252+0000 7fb63b400640 1 -- 172.21.15.89:0/2987522534 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb634010460 con 0x7fb644077040 2024-09-10T08:42:21.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.254+0000 7fb63b400640 1 --2- 172.21.15.89:0/2987522534 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb6280a2e00 0x7fb6280a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:21.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.254+0000 7fb64c5e1740 1 -- start start 2024-09-10T08:42:21.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.254+0000 7fb64c5e1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa47b90 0x56234fa47f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:21.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.254+0000 7fb64c5e1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa484b0 0x56234fa52830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:21.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.254+0000 7fb64c5e1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa52d70 0x56234fa55160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:21.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.255+0000 7fb64c5e1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56234f9ae3e0 con 0x56234fa47b90 2024-09-10T08:42:21.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.255+0000 7fb64c5e1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56234f955d80 con 0x56234fa484b0 2024-09-10T08:42:21.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.255+0000 7fb64c5e1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56234f9aa3a0 con 0x56234fa52d70 2024-09-10T08:42:21.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.255+0000 7fb63b400640 1 -- 172.21.15.89:0/2987522534 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb6340cc3f0 con 0x7fb644077040 2024-09-10T08:42:21.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.256+0000 7fb643400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa52d70 0x56234fa55160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:21.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.256+0000 7fb643400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa52d70 0x56234fa55160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44762/0 (socket says 172.21.15.89:44762) 2024-09-10T08:42:21.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.256+0000 7fb643400640 1 -- 172.21.15.89:0/2191619042 learned_addr learned my addr 172.21.15.89:0/2191619042 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:21.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.256+0000 7fb643400640 1 --2- 172.21.15.89:0/2987522534 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb6280a2e00 0x7fb6280a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:21.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.256+0000 7fb642a00640 1 --2- 172.21.15.89:0/2191619042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa47b90 0x56234fa47f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:21.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.257+0000 7fb642000640 1 --2- 172.21.15.89:0/2191619042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa484b0 0x56234fa52830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:21.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.257+0000 7fb643400640 1 -- 172.21.15.89:0/2191619042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa484b0 msgr2=0x56234fa52830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:21.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.257+0000 7fb643400640 1 --2- 172.21.15.89:0/2191619042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa484b0 0x56234fa52830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.257+0000 7fb643400640 1 -- 172.21.15.89:0/2191619042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa47b90 msgr2=0x56234fa47f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:21.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.257+0000 7fb643400640 1 --2- 172.21.15.89:0/2191619042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa47b90 0x56234fa47f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.258+0000 7fb643400640 1 -- 172.21.15.89:0/2191619042 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56234f974ae0 con 0x56234fa52d70 2024-09-10T08:42:21.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.258+0000 7fb642a00640 1 --2- 172.21.15.89:0/2191619042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa47b90 0x56234fa47f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:42:21.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.258+0000 7fb642000640 1 --2- 172.21.15.89:0/2191619042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa484b0 0x56234fa52830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:42:21.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.259+0000 7fb643400640 1 --2- 172.21.15.89:0/2191619042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa52d70 0x56234fa55160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb64407aa10 tx=0x7fb63c030da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ca651166394603b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:21.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.259+0000 7fb643400640 1 --2- 172.21.15.89:0/2987522534 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb6280a2e00 0x7fb6280a52c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009be0 tx=0x7fb63c047000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:21.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.259+0000 7fb62fe00640 1 -- 172.21.15.89:0/2191619042 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c04f070 con 0x56234fa52d70 2024-09-10T08:42:21.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.260+0000 7fb62fe00640 1 -- 172.21.15.89:0/2191619042 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63c003e50 con 0x56234fa52d70 2024-09-10T08:42:21.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.260+0000 7fb62fe00640 1 -- 172.21.15.89:0/2191619042 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c036040 con 0x56234fa52d70 2024-09-10T08:42:21.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.261+0000 7fb64c5e1740 1 -- 172.21.15.89:0/2191619042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa52d70 msgr2=0x56234fa55160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:21.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.261+0000 7fb64c5e1740 1 --2- 172.21.15.89:0/2191619042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa52d70 0x56234fa55160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb64407aa10 tx=0x7fb63c030da0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.261+0000 7fb64c5e1740 1 -- 172.21.15.89:0/2191619042 shutdown_connections 2024-09-10T08:42:21.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.261+0000 7fb64c5e1740 1 --2- 172.21.15.89:0/2191619042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa52d70 0x56234fa55160 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.261+0000 7fb64c5e1740 1 --2- 172.21.15.89:0/2191619042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa484b0 0x56234fa52830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.261+0000 7fb64c5e1740 1 --2- 172.21.15.89:0/2191619042 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa47b90 0x56234fa47f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.261+0000 7fb64c5e1740 1 -- 172.21.15.89:0/2191619042 >> 172.21.15.89:0/2191619042 conn(0x56234fa44120 msgr2=0x56234fa44500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:21.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.265+0000 7fb64c5e1740 1 -- 172.21.15.89:0/2191619042 shutdown_connections 2024-09-10T08:42:21.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.265+0000 7fb64c5e1740 1 -- 172.21.15.89:0/2191619042 wait complete. 2024-09-10T08:42:21.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.266+0000 7fb64c5e1740 1 -- start start 2024-09-10T08:42:21.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.266+0000 7fb64c5e1740 1 Processor -- start 2024-09-10T08:42:21.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.268+0000 7fb64c5e1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa47b90 0x56234fa4fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:21.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.268+0000 7fb64c5e1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa484b0 0x56234fa50230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:21.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.268+0000 7fb642a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa47b90 0x56234fa4fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:21.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.268+0000 7fb642a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa47b90 0x56234fa4fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:46304/0 (socket says 172.21.15.89:46304) 2024-09-10T08:42:21.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.269+0000 7fb64c5e1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa52d70 0x56234fa50770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:21.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.269+0000 7fb642a00640 1 -- 172.21.15.89:0/2297684683 learned_addr learned my addr 172.21.15.89:0/2297684683 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:21.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.269+0000 7fb64c5e1740 1 -- 172.21.15.89:0/2297684683 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56234fa669f0 con 0x56234fa484b0 2024-09-10T08:42:21.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.269+0000 7fb64c5e1740 1 -- 172.21.15.89:0/2297684683 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56234f9690f0 con 0x56234fa47b90 2024-09-10T08:42:21.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.269+0000 7fb64c5e1740 1 -- 172.21.15.89:0/2297684683 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56234fa66b70 con 0x56234fa52d70 2024-09-10T08:42:21.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.269+0000 7fb64c5e1740 10 client.0 _ll_register_callbacks cb 0x56234f95a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:42:21.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.269+0000 7fb643400640 1 --2- 172.21.15.89:0/2297684683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa52d70 0x56234fa50770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:21.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.269+0000 7fb642000640 1 --2- 172.21.15.89:0/2297684683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa484b0 0x56234fa50230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:21.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.270+0000 7fb642a00640 1 -- 172.21.15.89:0/2297684683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa52d70 msgr2=0x56234fa50770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:21.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.270+0000 7fb642a00640 1 --2- 172.21.15.89:0/2297684683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa52d70 0x56234fa50770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.270+0000 7fb642a00640 1 -- 172.21.15.89:0/2297684683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa484b0 msgr2=0x56234fa50230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.270+0000 7fb642a00640 1 --2- 172.21.15.89:0/2297684683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa484b0 0x56234fa50230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.270+0000 7fb642a00640 1 -- 172.21.15.89:0/2297684683 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56234f71f3c0 con 0x56234fa47b90 2024-09-10T08:42:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.270+0000 7fb642a00640 1 --2- 172.21.15.89:0/2297684683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa47b90 0x56234fa4fcf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb630009c90 tx=0x7fb6300039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=714a495a4304401d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.270+0000 7fb62cc00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:42:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.271+0000 7fb62cc00640 1 -- 172.21.15.89:0/2297684683 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630012860 con 0x56234fa47b90 2024-09-10T08:42:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.271+0000 7fb62cc00640 1 -- 172.21.15.89:0/2297684683 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb630012a00 con 0x56234fa47b90 2024-09-10T08:42:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.271+0000 7fb64c5e1740 1 -- 172.21.15.89:0/2297684683 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb63c003e50 con 0x56234fa47b90 2024-09-10T08:42:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.273+0000 7fb62cc00640 1 -- 172.21.15.89:0/2297684683 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb630012c40 con 0x56234fa47b90 2024-09-10T08:42:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.273+0000 7fb64c5e1740 10 client.25958 fetch_fsmap learned FSMap version 28 2024-09-10T08:42:21.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.273+0000 7fb64c5e1740 1 -- 172.21.15.89:0/2297684683 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56234fa72740 con 0x56234fa47b90 2024-09-10T08:42:21.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.276+0000 7fb62cc00640 1 -- 172.21.15.89:0/2297684683 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63c003e50 con 0x56234fa47b90 2024-09-10T08:42:21.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.276+0000 7fb62cc00640 1 -- 172.21.15.89:0/2297684683 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb63001a020 con 0x56234fa47b90 2024-09-10T08:42:21.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.278+0000 7fb64c5e1740 10 client.25958 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:42:21.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.278+0000 7fb64c5e1740 10 client.25958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:42:21.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.278+0000 7fb64c5e1740 20 client.25958 populate_metadata read hostname 'smithi089' 2024-09-10T08:42:21.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.278+0000 7fb64c5e1740 1 --2- 172.21.15.89:0/2297684683 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56234fa73c00 0x56234fa75ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:21.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.278+0000 7fb64c5e1740 4 client.25958 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:42:21.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.278+0000 7fb64c5e1740 1 -- 172.21.15.89:0/2297684683 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56234f9bb950 con 0x56234fa73c00 2024-09-10T08:42:21.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.280+0000 7fb642000640 1 --2- 172.21.15.89:0/2297684683 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56234fa73c00 0x56234fa75ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:21.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.283+0000 7fb642000640 1 --2- 172.21.15.89:0/2297684683 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56234fa73c00 0x56234fa75ff0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc6fe61cfb1fe1d server_cookie=a37431a735018420 in_seq=0 out_seq=0 2024-09-10T08:42:21.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.284+0000 7fb62cc00640 10 client.25958 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:21.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.291+0000 7fb62cc00640 1 -- 172.21.15.89:0/2297684683 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56234f9bb950 con 0x56234fa73c00 2024-09-10T08:42:21.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.291+0000 7fb62cc00640 10 client.25958 handle_command_reply: tid=0 2024-09-10T08:42:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.291+0000 7fb64c5e1740 1 client.25958 shutdown 2024-09-10T08:42:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.293+0000 7fb64c5e1740 1 -- 172.21.15.89:0/2297684683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa47b90 msgr2=0x56234fa4fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:21.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.293+0000 7fb64c5e1740 1 --2- 172.21.15.89:0/2297684683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa47b90 0x56234fa4fcf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb630009c90 tx=0x7fb6300039c0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.294+0000 7fb64c5e1740 1 -- 172.21.15.89:0/2297684683 shutdown_connections 2024-09-10T08:42:21.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.294+0000 7fb64c5e1740 1 --2- 172.21.15.89:0/2297684683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa52d70 0x56234fa50770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.294+0000 7fb64c5e1740 1 --2- 172.21.15.89:0/2297684683 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56234fa73c00 0x56234fa75ff0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.294+0000 7fb64c5e1740 1 --2- 172.21.15.89:0/2297684683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa484b0 0x56234fa50230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.294+0000 7fb64c5e1740 1 --2- 172.21.15.89:0/2297684683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa47b90 0x56234fa4fcf0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.294+0000 7fb64c5e1740 1 -- 172.21.15.89:0/2297684683 >> 172.21.15.89:0/2297684683 conn(0x56234fa44120 msgr2=0x56234fa49e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:21.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.294+0000 7fb62cc00640 0 client.25958 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:21.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.294+0000 7fb64c5e1740 1 -- 172.21.15.89:0/2297684683 shutdown_connections 2024-09-10T08:42:21.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.294+0000 7fb64c5e1740 1 -- 172.21.15.89:0/2297684683 wait complete. 2024-09-10T08:42:21.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.294+0000 7fb64c5e1740 20 client.25958 trim_cache size 0 max 16384 2024-09-10T08:42:21.331 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.330+0000 7fb64c5e1740 1 Processor -- start 2024-09-10T08:42:21.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.330+0000 7fb64c5e1740 1 -- start start 2024-09-10T08:42:21.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.331+0000 7fb64c5e1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa47b90 0x56234fa50240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:21.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.331+0000 7fb64c5e1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa484b0 0x56234fa50780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:21.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.331+0000 7fb64c5e1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa52d70 0x56234fa7b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:21.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.331+0000 7fb642a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa47b90 0x56234fa50240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:21.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.331+0000 7fb642a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa47b90 0x56234fa50240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44784/0 (socket says 172.21.15.89:44784) 2024-09-10T08:42:21.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.331+0000 7fb643400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa52d70 0x56234fa7b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:21.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.332+0000 7fb643400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa52d70 0x56234fa7b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51880/0 (socket says 172.21.15.89:51880) 2024-09-10T08:42:21.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.332+0000 7fb64c5e1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441af640 con 0x56234fa52d70 2024-09-10T08:42:21.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.332+0000 7fb64c5e1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441acf10 con 0x56234fa484b0 2024-09-10T08:42:21.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.332+0000 7fb64c5e1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb64410d2d0 con 0x56234fa47b90 2024-09-10T08:42:21.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.332+0000 7fb642a00640 1 -- 172.21.15.89:0/3532083534 learned_addr learned my addr 172.21.15.89:0/3532083534 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:21.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.332+0000 7fb643400640 1 -- 172.21.15.89:0/3532083534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa47b90 msgr2=0x56234fa50240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:21.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.332+0000 7fb643400640 1 --2- 172.21.15.89:0/3532083534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa47b90 0x56234fa50240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.332+0000 7fb643400640 1 -- 172.21.15.89:0/3532083534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa484b0 msgr2=0x56234fa50780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:42:21.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.333+0000 7fb643400640 1 --2- 172.21.15.89:0/3532083534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa484b0 0x56234fa50780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.333+0000 7fb643400640 1 -- 172.21.15.89:0/3532083534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56234fa44db0 con 0x56234fa52d70 2024-09-10T08:42:21.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.333+0000 7fb643400640 1 --2- 172.21.15.89:0/3532083534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa52d70 0x56234fa7b8a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb63c02ad00 tx=0x7fb63c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7136e9ce5385fd91 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:21.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.334+0000 7fb62f400640 1 -- 172.21.15.89:0/3532083534 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c04f040 con 0x56234fa52d70 2024-09-10T08:42:21.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.334+0000 7fb62f400640 1 -- 172.21.15.89:0/3532083534 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63c030570 con 0x56234fa52d70 2024-09-10T08:42:21.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.335+0000 7fb64c5e1740 1 -- 172.21.15.89:0/3532083534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa52d70 msgr2=0x56234fa7b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:21.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.337+0000 7fb62f400640 1 -- 172.21.15.89:0/3532083534 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c04f040 con 0x56234fa52d70 2024-09-10T08:42:21.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.337+0000 7fb64c5e1740 1 --2- 172.21.15.89:0/3532083534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa52d70 0x56234fa7b8a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb63c02ad00 tx=0x7fb63c004b40 comp rx=0 tx=0).stop 2024-09-10T08:42:21.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.337+0000 7fb64c5e1740 1 -- 172.21.15.89:0/3532083534 shutdown_connections 2024-09-10T08:42:21.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.337+0000 7fb64c5e1740 1 --2- 172.21.15.89:0/3532083534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa52d70 0x56234fa7b8a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.337+0000 7fb64c5e1740 1 --2- 172.21.15.89:0/3532083534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa484b0 0x56234fa50780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.337+0000 7fb64c5e1740 1 --2- 172.21.15.89:0/3532083534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa47b90 0x56234fa50240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.337+0000 7fb64c5e1740 1 -- 172.21.15.89:0/3532083534 >> 172.21.15.89:0/3532083534 conn(0x56234fa44120 msgr2=0x56234fa51d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:21.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.337+0000 7fb64c5e1740 1 -- 172.21.15.89:0/3532083534 shutdown_connections 2024-09-10T08:42:21.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.337+0000 7fb64c5e1740 1 -- 172.21.15.89:0/3532083534 wait complete. 2024-09-10T08:42:21.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.338+0000 7fb64c5e1740 1 -- start start 2024-09-10T08:42:21.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.339+0000 7fb64c5e1740 1 Processor -- start 2024-09-10T08:42:21.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.341+0000 7fb64c5e1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa47b90 0x56234fa6c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:21.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.341+0000 7fb64c5e1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa484b0 0x56234fa4c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:21.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.341+0000 7fb64c5e1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa52d70 0x56234fa4ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:21.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.341+0000 7fb642a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa47b90 0x56234fa6c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:21.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.342+0000 7fb642000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa484b0 0x56234fa4c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:21.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.342+0000 7fb642000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa484b0 0x56234fa4c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44788/0 (socket says 172.21.15.89:44788) 2024-09-10T08:42:21.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.342+0000 7fb642a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa47b90 0x56234fa6c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:46326/0 (socket says 172.21.15.89:46326) 2024-09-10T08:42:21.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.342+0000 7fb64c5e1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb64410d9a0 con 0x56234fa52d70 2024-09-10T08:42:21.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.342+0000 7fb64c5e1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441ac840 con 0x56234fa47b90 2024-09-10T08:42:21.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.342+0000 7fb64c5e1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441abaa0 con 0x56234fa484b0 2024-09-10T08:42:21.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.342+0000 7fb64c5e1740 10 client.0 _ll_register_callbacks cb 0x7fb6300098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:42:21.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.342+0000 7fb642000640 1 -- 172.21.15.89:0/941895664 learned_addr learned my addr 172.21.15.89:0/941895664 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:21.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.343+0000 7fb642a00640 1 -- 172.21.15.89:0/941895664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa484b0 msgr2=0x56234fa4c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:21.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.343+0000 7fb642a00640 1 --2- 172.21.15.89:0/941895664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa484b0 0x56234fa4c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.343+0000 7fb642a00640 1 -- 172.21.15.89:0/941895664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa52d70 msgr2=0x56234fa4ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:21.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.343+0000 7fb642a00640 1 --2- 172.21.15.89:0/941895664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa52d70 0x56234fa4ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.344+0000 7fb642a00640 1 -- 172.21.15.89:0/941895664 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56234fa4d550 con 0x56234fa47b90 2024-09-10T08:42:21.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.346+0000 7fb642a00640 1 --2- 172.21.15.89:0/941895664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa47b90 0x56234fa6c400 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb630000c00 tx=0x7fb630005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8256c766155ce6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:21.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.346+0000 7fb62cc00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:42:21.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.346+0000 7fb62cc00640 1 -- 172.21.15.89:0/941895664 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630033020 con 0x56234fa47b90 2024-09-10T08:42:21.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.346+0000 7fb62cc00640 1 -- 172.21.15.89:0/941895664 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb630026c00 con 0x56234fa47b90 2024-09-10T08:42:21.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.347+0000 7fb64c5e1740 1 -- 172.21.15.89:0/941895664 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56234f9b5e60 con 0x56234fa47b90 2024-09-10T08:42:21.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.348+0000 7fb62cc00640 1 -- 172.21.15.89:0/941895664 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb63003b050 con 0x56234fa47b90 2024-09-10T08:42:21.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.348+0000 7fb64c5e1740 10 client.25961 fetch_fsmap learned FSMap version 28 2024-09-10T08:42:21.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.348+0000 7fb64c5e1740 1 -- 172.21.15.89:0/941895664 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56234fade7e0 con 0x56234fa47b90 2024-09-10T08:42:21.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.349+0000 7fb62cc00640 1 -- 172.21.15.89:0/941895664 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56234f9b5e60 con 0x56234fa47b90 2024-09-10T08:42:21.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.350+0000 7fb62cc00640 1 -- 172.21.15.89:0/941895664 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb63002e070 con 0x56234fa47b90 2024-09-10T08:42:21.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.350+0000 7fb64c5e1740 10 client.25961 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:42:21.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.350+0000 7fb64c5e1740 10 client.25961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:42:21.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.350+0000 7fb64c5e1740 20 client.25961 populate_metadata read hostname 'smithi089' 2024-09-10T08:42:21.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.350+0000 7fb64c5e1740 1 --2- 172.21.15.89:0/941895664 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56234fadefe0 0x56234fae1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:21.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.350+0000 7fb64c5e1740 4 client.25961 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:42:21.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.350+0000 7fb64c5e1740 1 -- 172.21.15.89:0/941895664 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56234f9320c0 con 0x56234fadefe0 2024-09-10T08:42:21.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.351+0000 7fb642000640 1 --2- 172.21.15.89:0/941895664 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56234fadefe0 0x56234fae1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:21.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.357+0000 7fb62cc00640 10 client.25961 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:21.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.357+0000 7fb642000640 1 --2- 172.21.15.89:0/941895664 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56234fadefe0 0x56234fae1400 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b36a535d4567b6c1 server_cookie=d552ad03f8727a76 in_seq=0 out_seq=0 2024-09-10T08:42:21.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.376+0000 7fb62cc00640 1 -- 172.21.15.89:0/941895664 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56234f9320c0 con 0x56234fadefe0 2024-09-10T08:42:21.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.376+0000 7fb62cc00640 10 client.25961 handle_command_reply: tid=0 2024-09-10T08:42:21.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.376+0000 7fb64c5e1740 1 client.25961 shutdown 2024-09-10T08:42:21.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.378+0000 7fb64c5e1740 1 -- 172.21.15.89:0/941895664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa47b90 msgr2=0x56234fa6c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:21.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.378+0000 7fb64c5e1740 1 --2- 172.21.15.89:0/941895664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa47b90 0x56234fa6c400 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb630000c00 tx=0x7fb630005be0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.378+0000 7fb64c5e1740 1 -- 172.21.15.89:0/941895664 shutdown_connections 2024-09-10T08:42:21.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.378+0000 7fb64c5e1740 1 --2- 172.21.15.89:0/941895664 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56234fadefe0 0x56234fae1400 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.378+0000 7fb64c5e1740 1 --2- 172.21.15.89:0/941895664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56234fa52d70 0x56234fa4ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.378+0000 7fb64c5e1740 1 --2- 172.21.15.89:0/941895664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56234fa484b0 0x56234fa4c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.378+0000 7fb64c5e1740 1 --2- 172.21.15.89:0/941895664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56234fa47b90 0x56234fa6c400 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.378+0000 7fb64c5e1740 1 -- 172.21.15.89:0/941895664 >> 172.21.15.89:0/941895664 conn(0x56234fa44120 msgr2=0x56234fa429a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:21.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.378+0000 7fb62cc00640 0 client.25961 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:21.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.378+0000 7fb64c5e1740 1 -- 172.21.15.89:0/941895664 shutdown_connections 2024-09-10T08:42:21.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.379+0000 7fb64c5e1740 1 -- 172.21.15.89:0/941895664 wait complete. 2024-09-10T08:42:21.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.379+0000 7fb64c5e1740 20 client.25961 trim_cache size 0 max 16384 2024-09-10T08:42:21.387 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:42:21.387 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:42:21.387 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2024-09-10T08:42:21.387 INFO:teuthology.orchestra.run.smithi089.stdout: "257ffa30-7c72-4daa-b8b0-bbf7a86410dd": { 2024-09-10T08:42:21.387 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2024-09-10T08:42:21.387 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "257ffa30-7c72-4daa-b8b0-bbf7a86410dd", 2024-09-10T08:42:21.387 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2024-09-10T08:42:21.388 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:42:21.388 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:42:21.388 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:42:21.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.380+0000 7fb649a00640 1 -- 172.21.15.89:0/2987522534 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb6280a2e00 msgr2=0x7fb6280a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:21.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.380+0000 7fb649a00640 1 --2- 172.21.15.89:0/2987522534 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb6280a2e00 0x7fb6280a52c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009be0 tx=0x7fb63c047000 comp rx=0 tx=0).stop 2024-09-10T08:42:21.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.380+0000 7fb649a00640 1 -- 172.21.15.89:0/2987522534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb644077040 msgr2=0x7fb6441a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:21.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.380+0000 7fb649a00640 1 --2- 172.21.15.89:0/2987522534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb644077040 0x7fb6441a4c60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb634005fd0 tx=0x7fb63400dde0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.380+0000 7fb649a00640 1 -- 172.21.15.89:0/2987522534 shutdown_connections 2024-09-10T08:42:21.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.380+0000 7fb649a00640 1 --2- 172.21.15.89:0/2987522534 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb6280a2e00 0x7fb6280a52c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.380+0000 7fb649a00640 1 --2- 172.21.15.89:0/2987522534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb644112a30 0x7fb64419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.380+0000 7fb649a00640 1 --2- 172.21.15.89:0/2987522534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb644077040 0x7fb6441a4c60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.380+0000 7fb649a00640 1 --2- 172.21.15.89:0/2987522534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb644076720 0x7fb6441a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:21.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.381+0000 7fb649a00640 1 -- 172.21.15.89:0/2987522534 >> 172.21.15.89:0/2987522534 conn(0x7fb6440ffab0 msgr2=0x7fb644101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:21.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.381+0000 7fb649a00640 1 -- 172.21.15.89:0/2987522534 shutdown_connections 2024-09-10T08:42:21.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:21.381+0000 7fb649a00640 1 -- 172.21.15.89:0/2987522534 wait complete. 2024-09-10T08:42:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.417+0000 7f3ab6000640 1 -- 172.21.15.89:0/1866446992 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5579a8671b80 con 0x5579a87a8750 2024-09-10T08:42:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.417+0000 7f3ab6000640 10 client.25946 handle_command_reply: tid=0 2024-09-10T08:42:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.417+0000 7f3ad5fc8740 1 client.25946 shutdown 2024-09-10T08:42:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.417+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/1866446992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a87059d0 msgr2=0x5579a8714300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.417+0000 7f3ad5fc8740 1 --2- 172.21.15.89:0/1866446992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a87059d0 0x5579a8714300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x5579a8706400 tx=0x7f3abc0bbef0 comp rx=0 tx=0).stop 2024-09-10T08:42:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.417+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/1866446992 shutdown_connections 2024-09-10T08:42:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.417+0000 7f3ad5fc8740 1 --2- 172.21.15.89:0/1866446992 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5579a87a8750 0x5579a87aab70 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.417+0000 7f3ad5fc8740 1 --2- 172.21.15.89:0/1866446992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5579a870aa60 0x5579a8715230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.417+0000 7f3ad5fc8740 1 --2- 172.21.15.89:0/1866446992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5579a87059d0 0x5579a8714300 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.417+0000 7f3ad5fc8740 1 --2- 172.21.15.89:0/1866446992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a8705090 0x5579a8713dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.417+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/1866446992 >> 172.21.15.89:0/1866446992 conn(0x5579a8700700 msgr2=0x5579a86ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.417+0000 7f3ab6000640 0 client.25946 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.418+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/1866446992 shutdown_connections 2024-09-10T08:42:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.418+0000 7f3ad5fc8740 1 -- 172.21.15.89:0/1866446992 wait complete. 2024-09-10T08:42:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.418+0000 7f3ad5fc8740 20 client.25946 trim_cache size 0 max 16384 2024-09-10T08:42:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:42:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:42:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:42:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:42:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.421+0000 7f3ad3000640 1 -- 172.21.15.89:0/1156775083 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3ab00a2e60 msgr2=0x7f3ab00a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.421+0000 7f3ad3000640 1 --2- 172.21.15.89:0/1156775083 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3ab00a2e60 0x7f3ab00a5320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8006fd0 tx=0x7f3ab80090e0 comp rx=0 tx=0).stop 2024-09-10T08:42:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.421+0000 7f3ad3000640 1 -- 172.21.15.89:0/1156775083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3acc0737d0 msgr2=0x7f3acc11b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.421+0000 7f3ad3000640 1 --2- 172.21.15.89:0/1156775083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3acc0737d0 0x7f3acc11b340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3abc005ab0 tx=0x7f3abc032c10 comp rx=0 tx=0).stop 2024-09-10T08:42:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.422+0000 7f3ad3000640 1 -- 172.21.15.89:0/1156775083 shutdown_connections 2024-09-10T08:42:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.422+0000 7f3ad3000640 1 --2- 172.21.15.89:0/1156775083 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3ab00a2e60 0x7f3ab00a5320 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.422+0000 7f3ad3000640 1 --2- 172.21.15.89:0/1156775083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3acc109930 0x7f3acc119440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.422+0000 7f3ad3000640 1 --2- 172.21.15.89:0/1156775083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3acc0737d0 0x7f3acc11b340 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.422+0000 7f3ad3000640 1 --2- 172.21.15.89:0/1156775083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3acc072d20 0x7f3acc11ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.422+0000 7f3ad3000640 1 -- 172.21.15.89:0/1156775083 >> 172.21.15.89:0/1156775083 conn(0x7f3acc06ece0 msgr2=0x7f3acc10e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.423+0000 7f3ad3000640 1 -- 172.21.15.89:0/1156775083 shutdown_connections 2024-09-10T08:42:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:22.423+0000 7f3ad3000640 1 -- 172.21.15.89:0/1156775083 wait complete. 2024-09-10T08:42:22.713 INFO:tasks.cephfs.filesystem:scrub status for tag:257ffa30-7c72-4daa-b8b0-bbf7a86410dd - {'path': '/', 'tag': '257ffa30-7c72-4daa-b8b0-bbf7a86410dd', 'options': 'recursive,force'} 2024-09-10T08:42:23.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:22 smithi111 ceph-mon[35220]: pgmap v466: 129 pgs: 129 active+clean; 97 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.3 MiB/s wr, 1.10k op/s 2024-09-10T08:42:23.050 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:22 smithi089 ceph-mon[30757]: pgmap v466: 129 pgs: 129 active+clean; 97 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.3 MiB/s wr, 1.10k op/s 2024-09-10T08:42:23.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:22 smithi149 ceph-mon[36167]: pgmap v466: 129 pgs: 129 active+clean; 97 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.3 MiB/s wr, 1.10k op/s 2024-09-10T08:42:25.045 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:24 smithi089 ceph-mon[30757]: pgmap v467: 129 pgs: 129 active+clean; 101 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 880 op/s 2024-09-10T08:42:25.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:24 smithi149 ceph-mon[36167]: pgmap v467: 129 pgs: 129 active+clean; 101 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 880 op/s 2024-09-10T08:42:25.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:24 smithi111 ceph-mon[35220]: pgmap v467: 129 pgs: 129 active+clean; 101 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 880 op/s 2024-09-10T08:42:25.934 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:25 smithi089 ceph-mon[30757]: scrub summary: idle 2024-09-10T08:42:25.934 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:25 smithi089 ceph-mon[30757]: scrub summary: idle 2024-09-10T08:42:26.194 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:25 smithi111 ceph-mon[35220]: scrub summary: idle 2024-09-10T08:42:26.194 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:25 smithi111 ceph-mon[35220]: scrub summary: idle 2024-09-10T08:42:26.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:25 smithi149 ceph-mon[36167]: scrub summary: idle 2024-09-10T08:42:26.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:25 smithi149 ceph-mon[36167]: scrub summary: idle 2024-09-10T08:42:27.030 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:26 smithi089 ceph-mon[30757]: scrub summary: idle 2024-09-10T08:42:27.031 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:26 smithi089 ceph-mon[30757]: pgmap v468: 129 pgs: 129 active+clean; 102 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 981 op/s 2024-09-10T08:42:27.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:26 smithi149 ceph-mon[36167]: scrub summary: idle 2024-09-10T08:42:27.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:26 smithi149 ceph-mon[36167]: pgmap v468: 129 pgs: 129 active+clean; 102 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 981 op/s 2024-09-10T08:42:27.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:26 smithi111 ceph-mon[35220]: scrub summary: idle 2024-09-10T08:42:27.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:26 smithi111 ceph-mon[35220]: pgmap v468: 129 pgs: 129 active+clean; 102 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 981 op/s 2024-09-10T08:42:27.969 INFO:tasks.workunit.client.0.smithi089.stdout: 39 266705 202 182521 154 119946 508 2024-09-10T08:42:28.990 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:28 smithi089 ceph-mon[30757]: pgmap v469: 129 pgs: 129 active+clean; 123 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.5 MiB/s wr, 1.85k op/s 2024-09-10T08:42:29.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:28 smithi149 ceph-mon[36167]: pgmap v469: 129 pgs: 129 active+clean; 123 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.5 MiB/s wr, 1.85k op/s 2024-09-10T08:42:29.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:28 smithi111 ceph-mon[35220]: pgmap v469: 129 pgs: 129 active+clean; 123 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.5 MiB/s wr, 1.85k op/s 2024-09-10T08:42:30.970 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:30 smithi089 ceph-mon[30757]: pgmap v470: 129 pgs: 129 active+clean; 123 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.5 MiB/s wr, 1.81k op/s 2024-09-10T08:42:31.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:30 smithi149 ceph-mon[36167]: pgmap v470: 129 pgs: 129 active+clean; 123 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.5 MiB/s wr, 1.81k op/s 2024-09-10T08:42:31.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:30 smithi111 ceph-mon[35220]: pgmap v470: 129 pgs: 129 active+clean; 123 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.5 MiB/s wr, 1.81k op/s 2024-09-10T08:42:32.113 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:31 smithi089 ceph-mon[30757]: pgmap v471: 129 pgs: 129 active+clean; 135 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 10 MiB/s wr, 2.01k op/s 2024-09-10T08:42:32.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:31 smithi149 ceph-mon[36167]: pgmap v471: 129 pgs: 129 active+clean; 135 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 10 MiB/s wr, 2.01k op/s 2024-09-10T08:42:32.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:31 smithi111 ceph-mon[35220]: pgmap v471: 129 pgs: 129 active+clean; 135 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 10 MiB/s wr, 2.01k op/s 2024-09-10T08:42:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:34 smithi111 ceph-mon[35220]: pgmap v472: 129 pgs: 129 active+clean; 137 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.4 MiB/s wr, 1.38k op/s 2024-09-10T08:42:35.077 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:34 smithi089 ceph-mon[30757]: pgmap v472: 129 pgs: 129 active+clean; 137 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.4 MiB/s wr, 1.38k op/s 2024-09-10T08:42:35.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:34 smithi149 ceph-mon[36167]: pgmap v472: 129 pgs: 129 active+clean; 137 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.4 MiB/s wr, 1.38k op/s 2024-09-10T08:42:37.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:36 smithi111 ceph-mon[35220]: pgmap v473: 129 pgs: 129 active+clean; 137 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 1.23k op/s 2024-09-10T08:42:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:36 smithi089 ceph-mon[30757]: pgmap v473: 129 pgs: 129 active+clean; 137 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 1.23k op/s 2024-09-10T08:42:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.112+0000 7f87ba200640 1 -- 172.21.15.89:0/874963022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87b410b4e0 msgr2=0x7f87b410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.112+0000 7f87ba200640 1 --2- 172.21.15.89:0/874963022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87b410b4e0 0x7f87b410d950 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009c30 tx=0x7f87ac02a720 comp rx=0 tx=0).stop 2024-09-10T08:42:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.124+0000 7f87ba200640 1 -- 172.21.15.89:0/874963022 shutdown_connections 2024-09-10T08:42:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.124+0000 7f87ba200640 1 --2- 172.21.15.89:0/874963022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87b410de90 0x7f87b4110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.124+0000 7f87ba200640 1 --2- 172.21.15.89:0/874963022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87b410b4e0 0x7f87b410d950 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.124+0000 7f87ba200640 1 --2- 172.21.15.89:0/874963022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87b4072d80 0x7f87b4073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.124+0000 7f87ba200640 1 -- 172.21.15.89:0/874963022 >> 172.21.15.89:0/874963022 conn(0x7f87b406eb50 msgr2=0x7f87b4070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:37.136 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.136+0000 7f87ba200640 1 -- 172.21.15.89:0/874963022 shutdown_connections 2024-09-10T08:42:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.137+0000 7f87ba200640 1 -- 172.21.15.89:0/874963022 wait complete. 2024-09-10T08:42:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.141+0000 7f87ba200640 1 Processor -- start 2024-09-10T08:42:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.144+0000 7f87ba200640 1 -- start start 2024-09-10T08:42:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.147+0000 7f87ba200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87b4072d80 0x7f87b419e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.148+0000 7f87b3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87b4072d80 0x7f87b419e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.148+0000 7f87b3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87b4072d80 0x7f87b419e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38182/0 (socket says 172.21.15.89:38182) 2024-09-10T08:42:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.148+0000 7f87b3400640 1 -- 172.21.15.89:0/3274509621 learned_addr learned my addr 172.21.15.89:0/3274509621 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.148+0000 7f87ba200640 1 --2- 172.21.15.89:0/3274509621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87b410de90 0x7f87b419ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.148+0000 7f87ba200640 1 --2- 172.21.15.89:0/3274509621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87b41a3b50 0x7f87b41a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.149+0000 7f87b3e00640 1 --2- 172.21.15.89:0/3274509621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87b41a3b50 0x7f87b41a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.149+0000 7f87ba200640 1 -- 172.21.15.89:0/3274509621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41135e0 con 0x7f87b4072d80 2024-09-10T08:42:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.149+0000 7f87ba200640 1 -- 172.21.15.89:0/3274509621 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f87b4113460 con 0x7f87b41a3b50 2024-09-10T08:42:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.149+0000 7f87ba200640 1 -- 172.21.15.89:0/3274509621 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f87b4113760 con 0x7f87b410de90 2024-09-10T08:42:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.154+0000 7f87b2a00640 1 --2- 172.21.15.89:0/3274509621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87b410de90 0x7f87b419ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.154+0000 7f87b3400640 1 -- 172.21.15.89:0/3274509621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87b410de90 msgr2=0x7f87b419ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.154+0000 7f87b3400640 1 --2- 172.21.15.89:0/3274509621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87b410de90 0x7f87b419ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.154+0000 7f87b3400640 1 -- 172.21.15.89:0/3274509621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87b41a3b50 msgr2=0x7f87b41a5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.154+0000 7f87b3400640 1 --2- 172.21.15.89:0/3274509621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87b41a3b50 0x7f87b41a5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.154+0000 7f87b3400640 1 -- 172.21.15.89:0/3274509621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b41a65b0 con 0x7f87b4072d80 2024-09-10T08:42:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.155+0000 7f87b3e00640 1 --2- 172.21.15.89:0/3274509621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87b41a3b50 0x7f87b41a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:42:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.155+0000 7f87b3400640 1 --2- 172.21.15.89:0/3274509621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87b4072d80 0x7f87b419e740 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f87a400b6d0 tx=0x7f87a400bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.156+0000 7f87abe00640 1 -- 172.21.15.89:0/3274509621 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a4019070 con 0x7f87b4072d80 2024-09-10T08:42:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.156+0000 7f87abe00640 1 -- 172.21.15.89:0/3274509621 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87a400ccc0 con 0x7f87b4072d80 2024-09-10T08:42:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.156+0000 7f87abe00640 1 -- 172.21.15.89:0/3274509621 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a4012340 con 0x7f87b4072d80 2024-09-10T08:42:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.160+0000 7f87ba200640 1 -- 172.21.15.89:0/3274509621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b41a68a0 con 0x7f87b4072d80 2024-09-10T08:42:37.166 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.161+0000 7f87ba200640 1 -- 172.21.15.89:0/3274509621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b41a6df0 con 0x7f87b4072d80 2024-09-10T08:42:37.182 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.182+0000 7f87abe00640 1 -- 172.21.15.89:0/3274509621 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f87a400c7e0 con 0x7f87b4072d80 2024-09-10T08:42:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.191+0000 7f87bd22b740 1 Processor -- start 2024-09-10T08:42:37.194 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.193+0000 7f87abe00640 1 --2- 172.21.15.89:0/3274509621 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f87980a2d10 0x7f87980a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.196+0000 7f87abe00640 1 -- 172.21.15.89:0/3274509621 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f87a40ce310 con 0x7f87b4072d80 2024-09-10T08:42:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.197+0000 7f87b3e00640 1 --2- 172.21.15.89:0/3274509621 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f87980a2d10 0x7f87980a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.198+0000 7f87b3e00640 1 --2- 172.21.15.89:0/3274509621 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f87980a2d10 0x7f87980a51d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f87a0005fd0 tx=0x7f87a0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.198+0000 7f87bd22b740 1 -- start start 2024-09-10T08:42:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.199+0000 7f87bd22b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ace8dc8260 0x55ace8dc8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.199+0000 7f87bd22b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8ba0 0x55ace8dcd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.212+0000 7f87bd22b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dcdc30 0x55ace8dd00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.212+0000 7f87bd22b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ace8d2d100 con 0x55ace8dc8ba0 2024-09-10T08:42:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.212+0000 7f87bd22b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ace8cd51b0 con 0x55ace8dc8260 2024-09-10T08:42:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.212+0000 7f87bd22b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ace8d290c0 con 0x55ace8dcdc30 2024-09-10T08:42:37.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.212+0000 7f87b2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8ba0 0x55ace8dcd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.212+0000 7f87b2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8ba0 0x55ace8dcd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38188/0 (socket says 172.21.15.89:38188) 2024-09-10T08:42:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.212+0000 7f87b2a00640 1 -- 172.21.15.89:0/870153355 learned_addr learned my addr 172.21.15.89:0/870153355 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.213+0000 7f87b2a00640 1 -- 172.21.15.89:0/870153355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dcdc30 msgr2=0x55ace8dd00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.213+0000 7f87b2a00640 1 --2- 172.21.15.89:0/870153355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dcdc30 0x55ace8dd00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.213+0000 7f87b2a00640 1 -- 172.21.15.89:0/870153355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ace8dc8260 msgr2=0x55ace8dc8660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:42:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.213+0000 7f87b2a00640 1 --2- 172.21.15.89:0/870153355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ace8dc8260 0x55ace8dc8660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.213+0000 7f87b2a00640 1 -- 172.21.15.89:0/870153355 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ace8a40210 con 0x55ace8dc8ba0 2024-09-10T08:42:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.214+0000 7f87b2a00640 1 --2- 172.21.15.89:0/870153355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8ba0 0x55ace8dcd660 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009b80 tx=0x7f87ac031460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d67d7a68386989 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:37.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:36 smithi149 ceph-mon[36167]: pgmap v473: 129 pgs: 129 active+clean; 137 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.0 MiB/s wr, 1.23k op/s 2024-09-10T08:42:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.215+0000 7f879fe00640 1 -- 172.21.15.89:0/870153355 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac036070 con 0x55ace8dc8ba0 2024-09-10T08:42:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.219+0000 7f879fe00640 1 -- 172.21.15.89:0/870153355 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87ac037040 con 0x55ace8dc8ba0 2024-09-10T08:42:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.219+0000 7f879fe00640 1 -- 172.21.15.89:0/870153355 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac0316e0 con 0x55ace8dc8ba0 2024-09-10T08:42:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.220+0000 7f87bd22b740 1 -- 172.21.15.89:0/870153355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8ba0 msgr2=0x55ace8dcd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.220+0000 7f87bd22b740 1 --2- 172.21.15.89:0/870153355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8ba0 0x55ace8dcd660 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009b80 tx=0x7f87ac031460 comp rx=0 tx=0).stop 2024-09-10T08:42:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.221+0000 7f87bd22b740 1 -- 172.21.15.89:0/870153355 shutdown_connections 2024-09-10T08:42:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.221+0000 7f87bd22b740 1 --2- 172.21.15.89:0/870153355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dcdc30 0x55ace8dd00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.221+0000 7f87bd22b740 1 --2- 172.21.15.89:0/870153355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8ba0 0x55ace8dcd660 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.221+0000 7f87bd22b740 1 --2- 172.21.15.89:0/870153355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ace8dc8260 0x55ace8dc8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.221+0000 7f87bd22b740 1 -- 172.21.15.89:0/870153355 >> 172.21.15.89:0/870153355 conn(0x55ace8dc38d0 msgr2=0x55ace8dc3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.221+0000 7f87bd22b740 1 -- 172.21.15.89:0/870153355 shutdown_connections 2024-09-10T08:42:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.221+0000 7f87bd22b740 1 -- 172.21.15.89:0/870153355 wait complete. 2024-09-10T08:42:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.222+0000 7f87bd22b740 1 -- start start 2024-09-10T08:42:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.223+0000 7f87bd22b740 1 Processor -- start 2024-09-10T08:42:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.223+0000 7f87bd22b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8260 0x55ace8dd23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.225+0000 7f87b3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8260 0x55ace8dd23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.225+0000 7f87b3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8260 0x55ace8dd23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38204/0 (socket says 172.21.15.89:38204) 2024-09-10T08:42:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.225+0000 7f87b3400640 1 -- 172.21.15.89:0/2619021357 learned_addr learned my addr 172.21.15.89:0/2619021357 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.225+0000 7f87bd22b740 1 --2- 172.21.15.89:0/2619021357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dc8ba0 0x55ace8dd2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.225+0000 7f87bd22b740 1 --2- 172.21.15.89:0/2619021357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ace8dcdc30 0x55ace8dd9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:37.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.225+0000 7f87bd22b740 1 -- 172.21.15.89:0/2619021357 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ace8dd0ba0 con 0x55ace8dc8260 2024-09-10T08:42:37.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.225+0000 7f87bd22b740 1 -- 172.21.15.89:0/2619021357 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ace8cb6090 con 0x55ace8dcdc30 2024-09-10T08:42:37.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.225+0000 7f87bd22b740 1 -- 172.21.15.89:0/2619021357 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ace8dd0d20 con 0x55ace8dc8ba0 2024-09-10T08:42:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.226+0000 7f87b3e00640 1 --2- 172.21.15.89:0/2619021357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ace8dcdc30 0x55ace8dd9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.226+0000 7f87bd22b740 10 client.0 _ll_register_callbacks cb 0x55ace8d0af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:42:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.226+0000 7f87b3e00640 1 -- 172.21.15.89:0/2619021357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dc8ba0 msgr2=0x55ace8dd2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.226+0000 7f87b3e00640 1 --2- 172.21.15.89:0/2619021357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dc8ba0 0x55ace8dd2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.226+0000 7f87b3e00640 1 -- 172.21.15.89:0/2619021357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8260 msgr2=0x55ace8dd23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.226+0000 7f87b3e00640 1 --2- 172.21.15.89:0/2619021357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8260 0x55ace8dd23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.226+0000 7f87b3e00640 1 -- 172.21.15.89:0/2619021357 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ace8b0c620 con 0x55ace8dcdc30 2024-09-10T08:42:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.227+0000 7f87b3400640 1 --2- 172.21.15.89:0/2619021357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8260 0x55ace8dd23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:42:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.227+0000 7f87b3e00640 1 --2- 172.21.15.89:0/2619021357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ace8dcdc30 0x55ace8dd9960 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f87a0005ab0 tx=0x7f87a0022d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b61b099b79be216f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.231+0000 7f879cc00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:42:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.231+0000 7f879cc00640 1 -- 172.21.15.89:0/2619021357 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a0033030 con 0x55ace8dcdc30 2024-09-10T08:42:37.232 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.231+0000 7f879cc00640 1 -- 172.21.15.89:0/2619021357 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f87a0019040 con 0x55ace8dcdc30 2024-09-10T08:42:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.237+0000 7f87bd22b740 1 -- 172.21.15.89:0/2619021357 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87ac037040 con 0x55ace8dcdc30 2024-09-10T08:42:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.246+0000 7f879cc00640 1 -- 172.21.15.89:0/2619021357 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f87a00218e0 con 0x55ace8dcdc30 2024-09-10T08:42:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.250+0000 7f87bd22b740 10 client.25967 fetch_fsmap learned FSMap version 28 2024-09-10T08:42:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.250+0000 7f87bd22b740 1 -- 172.21.15.89:0/2619021357 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ace8ddc830 con 0x55ace8dcdc30 2024-09-10T08:42:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.256+0000 7f879cc00640 1 -- 172.21.15.89:0/2619021357 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87ac037040 con 0x55ace8dcdc30 2024-09-10T08:42:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.256+0000 7f879cc00640 1 -- 172.21.15.89:0/2619021357 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f87a002e3a0 con 0x55ace8dcdc30 2024-09-10T08:42:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.256+0000 7f87bd22b740 10 client.25967 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:42:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.256+0000 7f87bd22b740 10 client.25967 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:42:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.257+0000 7f87bd22b740 20 client.25967 populate_metadata read hostname 'smithi089' 2024-09-10T08:42:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.258+0000 7f87bd22b740 1 --2- 172.21.15.89:0/2619021357 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ace8dddf20 0x55ace8de0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.258+0000 7f87bd22b740 4 client.25967 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:42:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.258+0000 7f87bd22b740 1 -- 172.21.15.89:0/2619021357 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ace8d3a830 con 0x55ace8dddf20 2024-09-10T08:42:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.262+0000 7f87b3400640 1 --2- 172.21.15.89:0/2619021357 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ace8dddf20 0x55ace8de0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.265+0000 7f87b3400640 1 --2- 172.21.15.89:0/2619021357 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ace8dddf20 0x55ace8de0360 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15a385a0eea9b0e server_cookie=e01bba0ff83d6efe in_seq=0 out_seq=0 2024-09-10T08:42:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.265+0000 7f879cc00640 10 client.25967 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.267+0000 7f879cc00640 1 -- 172.21.15.89:0/2619021357 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ace8d3a830 con 0x55ace8dddf20 2024-09-10T08:42:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.267+0000 7f879cc00640 10 client.25967 handle_command_reply: tid=0 2024-09-10T08:42:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.267+0000 7f87bd22b740 1 client.25967 shutdown 2024-09-10T08:42:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.271+0000 7f87bd22b740 1 -- 172.21.15.89:0/2619021357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ace8dcdc30 msgr2=0x55ace8dd9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.271+0000 7f87bd22b740 1 --2- 172.21.15.89:0/2619021357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ace8dcdc30 0x55ace8dd9960 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f87a0005ab0 tx=0x7f87a0022d80 comp rx=0 tx=0).stop 2024-09-10T08:42:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.271+0000 7f87bd22b740 1 -- 172.21.15.89:0/2619021357 shutdown_connections 2024-09-10T08:42:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.271+0000 7f87bd22b740 1 --2- 172.21.15.89:0/2619021357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ace8dcdc30 0x55ace8dd9960 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.271+0000 7f87bd22b740 1 --2- 172.21.15.89:0/2619021357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dc8ba0 0x55ace8dd2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.271+0000 7f87bd22b740 1 --2- 172.21.15.89:0/2619021357 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ace8dddf20 0x55ace8de0360 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.271+0000 7f87bd22b740 1 --2- 172.21.15.89:0/2619021357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8260 0x55ace8dd23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.271+0000 7f87bd22b740 1 -- 172.21.15.89:0/2619021357 >> 172.21.15.89:0/2619021357 conn(0x55ace8dc38d0 msgr2=0x55ace8dc50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.271+0000 7f879cc00640 0 client.25967 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.281+0000 7f87bd22b740 1 -- 172.21.15.89:0/2619021357 shutdown_connections 2024-09-10T08:42:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.281+0000 7f87bd22b740 1 -- 172.21.15.89:0/2619021357 wait complete. 2024-09-10T08:42:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.281+0000 7f87bd22b740 20 client.25967 trim_cache size 0 max 16384 2024-09-10T08:42:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.526+0000 7f87bd22b740 1 Processor -- start 2024-09-10T08:42:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.530+0000 7f87bd22b740 1 -- start start 2024-09-10T08:42:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.531+0000 7f87bd22b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ace8dc8260 0x55ace8dc4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.531+0000 7f87bd22b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8ba0 0x55ace8dd9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.531+0000 7f87bd22b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dcdc30 0x55ace8dc4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.531+0000 7f87bd22b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41b3c20 con 0x55ace8dc8ba0 2024-09-10T08:42:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.531+0000 7f87bd22b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41b14f0 con 0x55ace8dc8260 2024-09-10T08:42:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.531+0000 7f87bd22b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41af340 con 0x55ace8dcdc30 2024-09-10T08:42:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.531+0000 7f87b3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dcdc30 0x55ace8dc4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.531+0000 7f87b3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dcdc30 0x55ace8dc4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:49192/0 (socket says 172.21.15.89:49192) 2024-09-10T08:42:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.531+0000 7f87b3e00640 1 -- 172.21.15.89:0/677187419 learned_addr learned my addr 172.21.15.89:0/677187419 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.532+0000 7f87b3e00640 1 -- 172.21.15.89:0/677187419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ace8dc8260 msgr2=0x55ace8dc4780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:42:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.532+0000 7f87b3e00640 1 --2- 172.21.15.89:0/677187419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ace8dc8260 0x55ace8dc4780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.532+0000 7f87b3e00640 1 -- 172.21.15.89:0/677187419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8ba0 msgr2=0x55ace8dd9920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:42:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.532+0000 7f87b3e00640 1 --2- 172.21.15.89:0/677187419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8ba0 0x55ace8dd9920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.532+0000 7f87b3e00640 1 -- 172.21.15.89:0/677187419 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ace8dc36f0 con 0x55ace8dcdc30 2024-09-10T08:42:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.533+0000 7f87b3e00640 1 --2- 172.21.15.89:0/677187419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dcdc30 0x55ace8dc4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f87a001b010 tx=0x7f87a0022690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eee02393bf27fb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.541+0000 7f879f400640 1 -- 172.21.15.89:0/677187419 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a0010070 con 0x55ace8dcdc30 2024-09-10T08:42:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.541+0000 7f879f400640 1 -- 172.21.15.89:0/677187419 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87a0034040 con 0x55ace8dcdc30 2024-09-10T08:42:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.541+0000 7f879f400640 1 -- 172.21.15.89:0/677187419 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a0033070 con 0x55ace8dcdc30 2024-09-10T08:42:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.542+0000 7f87bd22b740 1 -- 172.21.15.89:0/677187419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dcdc30 msgr2=0x55ace8dc4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.542+0000 7f87bd22b740 1 --2- 172.21.15.89:0/677187419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dcdc30 0x55ace8dc4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f87a001b010 tx=0x7f87a0022690 comp rx=0 tx=0).stop 2024-09-10T08:42:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.552+0000 7f87bd22b740 1 -- 172.21.15.89:0/677187419 shutdown_connections 2024-09-10T08:42:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.552+0000 7f87bd22b740 1 --2- 172.21.15.89:0/677187419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dcdc30 0x55ace8dc4cc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.552+0000 7f87bd22b740 1 --2- 172.21.15.89:0/677187419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8ba0 0x55ace8dd9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.552+0000 7f87bd22b740 1 --2- 172.21.15.89:0/677187419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ace8dc8260 0x55ace8dc4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.553 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.552+0000 7f87bd22b740 1 -- 172.21.15.89:0/677187419 >> 172.21.15.89:0/677187419 conn(0x55ace8dc38d0 msgr2=0x55ace8dd23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:37.556 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.552+0000 7f87bd22b740 1 -- 172.21.15.89:0/677187419 shutdown_connections 2024-09-10T08:42:37.556 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.553+0000 7f87bd22b740 1 -- 172.21.15.89:0/677187419 wait complete. 2024-09-10T08:42:37.556 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.555+0000 7f87bd22b740 1 -- start start 2024-09-10T08:42:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.558+0000 7f87bd22b740 1 Processor -- start 2024-09-10T08:42:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.561+0000 7f87bd22b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dc8260 0x55ace8dd6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.563+0000 7f87bd22b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8ba0 0x55ace8dd7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.563+0000 7f87b2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8ba0 0x55ace8dd7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.563+0000 7f87b2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8ba0 0x55ace8dd7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38214/0 (socket says 172.21.15.89:38214) 2024-09-10T08:42:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.563+0000 7f87b2a00640 1 -- 172.21.15.89:0/2355894319 learned_addr learned my addr 172.21.15.89:0/2355894319 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.563+0000 7f87bd22b740 1 --2- 172.21.15.89:0/2355894319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ace8dcdc30 0x55ace8dd8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.564+0000 7f87bd22b740 1 -- 172.21.15.89:0/2355894319 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41af9b0 con 0x55ace8dc8ba0 2024-09-10T08:42:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.564+0000 7f87bd22b740 1 -- 172.21.15.89:0/2355894319 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41b0e20 con 0x55ace8dcdc30 2024-09-10T08:42:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.564+0000 7f87bd22b740 1 -- 172.21.15.89:0/2355894319 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41b0080 con 0x55ace8dc8260 2024-09-10T08:42:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.564+0000 7f87bd22b740 10 client.0 _ll_register_callbacks cb 0x7f87a001a860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:42:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.567+0000 7f87b3400640 1 --2- 172.21.15.89:0/2355894319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dc8260 0x55ace8dd6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.567+0000 7f87b3e00640 1 --2- 172.21.15.89:0/2355894319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ace8dcdc30 0x55ace8dd8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.567+0000 7f87b3400640 1 -- 172.21.15.89:0/2355894319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ace8dcdc30 msgr2=0x55ace8dd8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.567+0000 7f87b3400640 1 --2- 172.21.15.89:0/2355894319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ace8dcdc30 0x55ace8dd8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.567+0000 7f87b3400640 1 -- 172.21.15.89:0/2355894319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8ba0 msgr2=0x55ace8dd7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.567+0000 7f87b3400640 1 --2- 172.21.15.89:0/2355894319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8ba0 0x55ace8dd7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.568+0000 7f87b3400640 1 -- 172.21.15.89:0/2355894319 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ace8dd8770 con 0x55ace8dc8260 2024-09-10T08:42:37.571 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.572+0000 7f87b3400640 1 --2- 172.21.15.89:0/2355894319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dc8260 0x55ace8dd6c10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f87a409a730 tx=0x7f87a400e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d45ea1a4d0159306 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.575+0000 7f879cc00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:42:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.575+0000 7f879cc00640 1 -- 172.21.15.89:0/2355894319 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a409eb50 con 0x55ace8dc8260 2024-09-10T08:42:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.575+0000 7f879cc00640 1 -- 172.21.15.89:0/2355894319 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f87a4098aa0 con 0x55ace8dc8260 2024-09-10T08:42:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.575+0000 7f87bd22b740 1 -- 172.21.15.89:0/2355894319 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87a0034040 con 0x55ace8dc8260 2024-09-10T08:42:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.576+0000 7f879cc00640 1 -- 172.21.15.89:0/2355894319 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f87a40a3410 con 0x55ace8dc8260 2024-09-10T08:42:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.576+0000 7f87bd22b740 10 client.25909 fetch_fsmap learned FSMap version 28 2024-09-10T08:42:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.576+0000 7f87bd22b740 1 -- 172.21.15.89:0/2355894319 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ace8dd9b30 con 0x55ace8dc8260 2024-09-10T08:42:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.576+0000 7f879cc00640 1 -- 172.21.15.89:0/2355894319 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87a0034040 con 0x55ace8dc8260 2024-09-10T08:42:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.576+0000 7f879cc00640 1 -- 172.21.15.89:0/2355894319 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f87a4099bf0 con 0x55ace8dc8260 2024-09-10T08:42:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.576+0000 7f87bd22b740 10 client.25909 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:42:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.576+0000 7f87bd22b740 10 client.25909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:42:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.577+0000 7f87bd22b740 20 client.25909 populate_metadata read hostname 'smithi089' 2024-09-10T08:42:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.577+0000 7f87bd22b740 1 --2- 172.21.15.89:0/2355894319 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ace8e491a0 0x55ace8e4b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.577+0000 7f87bd22b740 4 client.25909 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:42:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.577+0000 7f87bd22b740 1 -- 172.21.15.89:0/2355894319 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ace8d34b80 con 0x55ace8e491a0 2024-09-10T08:42:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.581+0000 7f87b2a00640 1 --2- 172.21.15.89:0/2355894319 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ace8e491a0 0x55ace8e4b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.587+0000 7f87b2a00640 1 --2- 172.21.15.89:0/2355894319 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ace8e491a0 0x55ace8e4b5c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87c06f283683f05a server_cookie=984486df14d8536 in_seq=0 out_seq=0 2024-09-10T08:42:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:37.596+0000 7f879cc00640 10 client.25909 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:37.989 INFO:tasks.workunit.client.0.smithi089.stdout: 47 309303 330 211097 326 146607 996 2024-09-10T08:42:38.769 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:38 smithi089 ceph-mon[30757]: pgmap v474: 129 pgs: 129 active+clean; 142 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.4 MiB/s wr, 1.21k op/s 2024-09-10T08:42:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:38 smithi111 ceph-mon[35220]: pgmap v474: 129 pgs: 129 active+clean; 142 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.4 MiB/s wr, 1.21k op/s 2024-09-10T08:42:39.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:38 smithi149 ceph-mon[36167]: pgmap v474: 129 pgs: 129 active+clean; 142 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.4 MiB/s wr, 1.21k op/s 2024-09-10T08:42:40.326 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.327+0000 7f879cc00640 1 -- 172.21.15.89:0/2355894319 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ace8d34b80 con 0x55ace8e491a0 2024-09-10T08:42:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.327+0000 7f879cc00640 10 client.25909 handle_command_reply: tid=0 2024-09-10T08:42:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.329+0000 7f87bd22b740 1 client.25909 shutdown 2024-09-10T08:42:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:42:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:42:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:42:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:42:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.358+0000 7f87bd22b740 1 -- 172.21.15.89:0/2355894319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dc8260 msgr2=0x55ace8dd6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.358+0000 7f87bd22b740 1 --2- 172.21.15.89:0/2355894319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dc8260 0x55ace8dd6c10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f87a409a730 tx=0x7f87a400e040 comp rx=0 tx=0).stop 2024-09-10T08:42:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.358+0000 7f87bd22b740 1 -- 172.21.15.89:0/2355894319 shutdown_connections 2024-09-10T08:42:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.358+0000 7f87bd22b740 1 --2- 172.21.15.89:0/2355894319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ace8dcdc30 0x55ace8dd8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.358+0000 7f87bd22b740 1 --2- 172.21.15.89:0/2355894319 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ace8e491a0 0x55ace8e4b5c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.358+0000 7f87bd22b740 1 --2- 172.21.15.89:0/2355894319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ace8dc8ba0 0x55ace8dd7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.358+0000 7f87bd22b740 1 --2- 172.21.15.89:0/2355894319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ace8dc8260 0x55ace8dd6c10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.358+0000 7f87bd22b740 1 -- 172.21.15.89:0/2355894319 >> 172.21.15.89:0/2355894319 conn(0x55ace8dc38d0 msgr2=0x55ace8dc21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.358+0000 7f879cc00640 0 client.25909 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.359+0000 7f87bd22b740 1 -- 172.21.15.89:0/2355894319 shutdown_connections 2024-09-10T08:42:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.359+0000 7f87bd22b740 1 -- 172.21.15.89:0/2355894319 wait complete. 2024-09-10T08:42:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.359+0000 7f87bd22b740 20 client.25909 trim_cache size 0 max 16384 2024-09-10T08:42:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.361+0000 7f87ba200640 1 -- 172.21.15.89:0/3274509621 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f87980a2d10 msgr2=0x7f87980a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.361+0000 7f87ba200640 1 --2- 172.21.15.89:0/3274509621 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f87980a2d10 0x7f87980a51d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f87a0005fd0 tx=0x7f87a0005a20 comp rx=0 tx=0).stop 2024-09-10T08:42:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.361+0000 7f87ba200640 1 -- 172.21.15.89:0/3274509621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87b4072d80 msgr2=0x7f87b419e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.361+0000 7f87ba200640 1 --2- 172.21.15.89:0/3274509621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87b4072d80 0x7f87b419e740 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f87a400b6d0 tx=0x7f87a400bb90 comp rx=0 tx=0).stop 2024-09-10T08:42:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.364+0000 7f87ba200640 1 -- 172.21.15.89:0/3274509621 shutdown_connections 2024-09-10T08:42:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.364+0000 7f87ba200640 1 --2- 172.21.15.89:0/3274509621 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f87980a2d10 0x7f87980a51d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.364+0000 7f87ba200640 1 --2- 172.21.15.89:0/3274509621 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87b41a3b50 0x7f87b41a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.364+0000 7f87ba200640 1 --2- 172.21.15.89:0/3274509621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87b410de90 0x7f87b419ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.364+0000 7f87ba200640 1 --2- 172.21.15.89:0/3274509621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87b4072d80 0x7f87b419e740 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.364+0000 7f87ba200640 1 -- 172.21.15.89:0/3274509621 >> 172.21.15.89:0/3274509621 conn(0x7f87b406eb50 msgr2=0x7f87b410c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.365+0000 7f87ba200640 1 -- 172.21.15.89:0/3274509621 shutdown_connections 2024-09-10T08:42:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:40.365+0000 7f87ba200640 1 -- 172.21.15.89:0/3274509621 wait complete. 2024-09-10T08:42:41.035 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:40 smithi089 ceph-mon[30757]: pgmap v475: 129 pgs: 129 active+clean; 142 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 323 op/s 2024-09-10T08:42:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:40 smithi111 ceph-mon[35220]: pgmap v475: 129 pgs: 129 active+clean; 142 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 323 op/s 2024-09-10T08:42:41.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:40 smithi149 ceph-mon[36167]: pgmap v475: 129 pgs: 129 active+clean; 142 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 323 op/s 2024-09-10T08:42:43.034 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:42 smithi089 ceph-mon[30757]: pgmap v476: 129 pgs: 129 active+clean; 131 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 958 op/s 2024-09-10T08:42:43.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:42 smithi111 ceph-mon[35220]: pgmap v476: 129 pgs: 129 active+clean; 131 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 958 op/s 2024-09-10T08:42:43.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:42 smithi149 ceph-mon[36167]: pgmap v476: 129 pgs: 129 active+clean; 131 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 958 op/s 2024-09-10T08:42:45.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:44 smithi111 ceph-mon[35220]: pgmap v477: 129 pgs: 129 active+clean; 126 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 831 op/s 2024-09-10T08:42:45.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:44 smithi089 ceph-mon[30757]: pgmap v477: 129 pgs: 129 active+clean; 126 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 831 op/s 2024-09-10T08:42:45.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:44 smithi149 ceph-mon[36167]: pgmap v477: 129 pgs: 129 active+clean; 126 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 831 op/s 2024-09-10T08:42:47.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:46 smithi111 ceph-mon[35220]: pgmap v478: 129 pgs: 129 active+clean; 126 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 838 op/s 2024-09-10T08:42:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:46 smithi089 ceph-mon[30757]: pgmap v478: 129 pgs: 129 active+clean; 126 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 838 op/s 2024-09-10T08:42:47.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:46 smithi149 ceph-mon[36167]: pgmap v478: 129 pgs: 129 active+clean; 126 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 838 op/s 2024-09-10T08:42:47.987 INFO:tasks.workunit.client.0.smithi089.stdout: 56 482017 614 329134 600 250124 1572 2024-09-10T08:42:49.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:48 smithi111 ceph-mon[35220]: pgmap v479: 129 pgs: 129 active+clean; 133 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 932 op/s 2024-09-10T08:42:49.063 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:48 smithi089 ceph-mon[30757]: pgmap v479: 129 pgs: 129 active+clean; 133 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 932 op/s 2024-09-10T08:42:49.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:48 smithi149 ceph-mon[36167]: pgmap v479: 129 pgs: 129 active+clean; 133 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 932 op/s 2024-09-10T08:42:51.055 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:50 smithi089 ceph-mon[30757]: pgmap v480: 129 pgs: 129 active+clean; 133 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 834 op/s 2024-09-10T08:42:51.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:50 smithi149 ceph-mon[36167]: pgmap v480: 129 pgs: 129 active+clean; 133 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 834 op/s 2024-09-10T08:42:51.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:50 smithi111 ceph-mon[35220]: pgmap v480: 129 pgs: 129 active+clean; 133 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 834 op/s 2024-09-10T08:42:52.715 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:42:53.184 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:52 smithi089 ceph-mon[30757]: pgmap v481: 129 pgs: 129 active+clean; 144 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 899 op/s 2024-09-10T08:42:53.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:52 smithi149 ceph-mon[36167]: pgmap v481: 129 pgs: 129 active+clean; 144 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 899 op/s 2024-09-10T08:42:53.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:52 smithi111 ceph-mon[35220]: pgmap v481: 129 pgs: 129 active+clean; 144 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 899 op/s 2024-09-10T08:42:55.063 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:54 smithi089 ceph-mon[30757]: pgmap v482: 129 pgs: 129 active+clean; 148 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 320 op/s 2024-09-10T08:42:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.165+0000 7f8df5c00640 1 -- 172.21.15.89:0/4226054599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8df010de90 msgr2=0x7f8df0110350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:55.167 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.165+0000 7f8df5c00640 1 --2- 172.21.15.89:0/4226054599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8df010de90 0x7f8df0110350 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8de0009900 tx=0x7f8de002a440 comp rx=0 tx=0).stop 2024-09-10T08:42:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.181+0000 7f8df5c00640 1 -- 172.21.15.89:0/4226054599 shutdown_connections 2024-09-10T08:42:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.181+0000 7f8df5c00640 1 --2- 172.21.15.89:0/4226054599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8df010de90 0x7f8df0110350 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.181+0000 7f8df5c00640 1 --2- 172.21.15.89:0/4226054599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8df010b4e0 0x7f8df010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.181+0000 7f8df5c00640 1 --2- 172.21.15.89:0/4226054599 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8df0072d80 0x7f8df0073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.181+0000 7f8df5c00640 1 -- 172.21.15.89:0/4226054599 >> 172.21.15.89:0/4226054599 conn(0x7f8df006eb50 msgr2=0x7f8df0070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.200+0000 7f8df5c00640 1 -- 172.21.15.89:0/4226054599 shutdown_connections 2024-09-10T08:42:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.203+0000 7f8df5c00640 1 -- 172.21.15.89:0/4226054599 wait complete. 2024-09-10T08:42:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.204+0000 7f8df5c00640 1 Processor -- start 2024-09-10T08:42:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.216+0000 7f8df5c00640 1 -- start start 2024-09-10T08:42:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.216+0000 7f8df5c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8df0072d80 0x7f8df0115820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.216+0000 7f8df5c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8df010b4e0 0x7f8df0115d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.216+0000 7f8df5c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8df010de90 0x7f8df01b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.216+0000 7f8df5c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8df011c480 con 0x7f8df0072d80 2024-09-10T08:42:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.216+0000 7f8df5c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8df011c300 con 0x7f8df010de90 2024-09-10T08:42:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.216+0000 7f8df5c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8df011c600 con 0x7f8df010b4e0 2024-09-10T08:42:55.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:54 smithi149 ceph-mon[36167]: pgmap v482: 129 pgs: 129 active+clean; 148 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 320 op/s 2024-09-10T08:42:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.217+0000 7f8deea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8df0072d80 0x7f8df0115820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.217+0000 7f8deea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8df0072d80 0x7f8df0115820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48240/0 (socket says 172.21.15.89:48240) 2024-09-10T08:42:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.217+0000 7f8deea00640 1 -- 172.21.15.89:0/1288672070 learned_addr learned my addr 172.21.15.89:0/1288672070 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.218+0000 7f8deea00640 1 -- 172.21.15.89:0/1288672070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8df010b4e0 msgr2=0x7f8df0115d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:42:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.218+0000 7f8deea00640 1 --2- 172.21.15.89:0/1288672070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8df010b4e0 0x7f8df0115d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.218+0000 7f8deea00640 1 -- 172.21.15.89:0/1288672070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8df010de90 msgr2=0x7f8df01b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.218+0000 7f8deea00640 1 --2- 172.21.15.89:0/1288672070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8df010de90 0x7f8df01b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.218+0000 7f8deea00640 1 -- 172.21.15.89:0/1288672070 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df0118ee0 con 0x7f8df0072d80 2024-09-10T08:42:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.229+0000 7f8deea00640 1 --2- 172.21.15.89:0/1288672070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8df0072d80 0x7f8df0115820 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8de800b6d0 tx=0x7f8de800bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.240+0000 7f8de7400640 1 -- 172.21.15.89:0/1288672070 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de8019070 con 0x7f8df0072d80 2024-09-10T08:42:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.240+0000 7f8de7400640 1 -- 172.21.15.89:0/1288672070 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8de800cc00 con 0x7f8df0072d80 2024-09-10T08:42:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.240+0000 7f8de7400640 1 -- 172.21.15.89:0/1288672070 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de8012340 con 0x7f8df0072d80 2024-09-10T08:42:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.241+0000 7f8df5c00640 1 -- 172.21.15.89:0/1288672070 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df01b7190 con 0x7f8df0072d80 2024-09-10T08:42:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.241+0000 7f8df5c00640 1 -- 172.21.15.89:0/1288672070 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df01b7660 con 0x7f8df0072d80 2024-09-10T08:42:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.272+0000 7f8df89fd740 1 Processor -- start 2024-09-10T08:42:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.272+0000 7f8df89fd740 1 -- start start 2024-09-10T08:42:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.272+0000 7f8df89fd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500c260 0x55afc500c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.274+0000 7f8de7400640 1 -- 172.21.15.89:0/1288672070 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8de800c720 con 0x7f8df0072d80 2024-09-10T08:42:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.276+0000 7f8de7400640 1 --2- 172.21.15.89:0/1288672070 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8dd00a2d90 0x7f8dd00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.276+0000 7f8de7400640 1 -- 172.21.15.89:0/1288672070 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f8de80ce4b0 con 0x7f8df0072d80 2024-09-10T08:42:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.281+0000 7f8df89fd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc500cba0 0x55afc5011660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.281+0000 7f8def400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500c260 0x55afc500c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.281+0000 7f8def400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500c260 0x55afc500c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48246/0 (socket says 172.21.15.89:48246) 2024-09-10T08:42:55.287 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.285+0000 7f8df89fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afc5011c30 0x55afc50140f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:55.287 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.285+0000 7f8df89fd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55afc4f71100 con 0x55afc500c260 2024-09-10T08:42:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.285+0000 7f8df89fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55afc4f191b0 con 0x55afc5011c30 2024-09-10T08:42:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.285+0000 7f8df89fd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55afc4f6d0c0 con 0x55afc500cba0 2024-09-10T08:42:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.285+0000 7f8def400640 1 -- 172.21.15.89:0/2433483964 learned_addr learned my addr 172.21.15.89:0/2433483964 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.285+0000 7f8dee000640 1 --2- 172.21.15.89:0/2433483964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc500cba0 0x55afc5011660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.286+0000 7f8def400640 1 --2- 172.21.15.89:0/2433483964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afc5011c30 0x55afc50140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.286+0000 7f8def400640 1 -- 172.21.15.89:0/2433483964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc500cba0 msgr2=0x55afc5011660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.286+0000 7f8def400640 1 --2- 172.21.15.89:0/2433483964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc500cba0 0x55afc5011660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.286+0000 7f8def400640 1 -- 172.21.15.89:0/2433483964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afc5011c30 msgr2=0x55afc50140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.286+0000 7f8def400640 1 --2- 172.21.15.89:0/2433483964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afc5011c30 0x55afc50140f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.286+0000 7f8def400640 1 -- 172.21.15.89:0/2433483964 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afc4c84210 con 0x55afc500c260 2024-09-10T08:42:55.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:54 smithi111 ceph-mon[35220]: pgmap v482: 129 pgs: 129 active+clean; 148 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 320 op/s 2024-09-10T08:42:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.294+0000 7f8def400640 1 --2- 172.21.15.89:0/2433483964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500c260 0x55afc500c660 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8de0005c20 tx=0x7f8de0005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9409b948c4a24d7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.299+0000 7f8deea00640 1 --2- 172.21.15.89:0/1288672070 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8dd00a2d90 0x7f8dd00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.302+0000 7f8deea00640 1 --2- 172.21.15.89:0/1288672070 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8dd00a2d90 0x7f8dd00a5250 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8de80cbb80 tx=0x7f8de80046e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.302+0000 7f8ddfe00640 1 -- 172.21.15.89:0/2433483964 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0047070 con 0x55afc500c260 2024-09-10T08:42:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.302+0000 7f8ddfe00640 1 -- 172.21.15.89:0/2433483964 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8de0004420 con 0x55afc500c260 2024-09-10T08:42:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.302+0000 7f8ddfe00640 1 -- 172.21.15.89:0/2433483964 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0003cc0 con 0x55afc500c260 2024-09-10T08:42:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.313+0000 7f8df89fd740 1 -- 172.21.15.89:0/2433483964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500c260 msgr2=0x55afc500c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.313+0000 7f8df89fd740 1 --2- 172.21.15.89:0/2433483964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500c260 0x55afc500c660 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8de0005c20 tx=0x7f8de0005bb0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.332+0000 7f8df89fd740 1 -- 172.21.15.89:0/2433483964 shutdown_connections 2024-09-10T08:42:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.332+0000 7f8df89fd740 1 --2- 172.21.15.89:0/2433483964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afc5011c30 0x55afc50140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.332+0000 7f8df89fd740 1 --2- 172.21.15.89:0/2433483964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc500cba0 0x55afc5011660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.332+0000 7f8df89fd740 1 --2- 172.21.15.89:0/2433483964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500c260 0x55afc500c660 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.332+0000 7f8df89fd740 1 -- 172.21.15.89:0/2433483964 >> 172.21.15.89:0/2433483964 conn(0x55afc50078d0 msgr2=0x55afc5007cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.335+0000 7f8df89fd740 1 -- 172.21.15.89:0/2433483964 shutdown_connections 2024-09-10T08:42:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.335+0000 7f8df89fd740 1 -- 172.21.15.89:0/2433483964 wait complete. 2024-09-10T08:42:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.336+0000 7f8df89fd740 1 -- start start 2024-09-10T08:42:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.336+0000 7f8df89fd740 1 Processor -- start 2024-09-10T08:42:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.344+0000 7f8df89fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afc500c260 0x55afc50164d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.346+0000 7f8df89fd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc500cba0 0x55afc5016a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.346+0000 7f8df89fd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc5011c30 0x55afc501da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.346+0000 7f8df89fd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55afc5014ba0 con 0x55afc5011c30 2024-09-10T08:42:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.346+0000 7f8df89fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55afc4efa090 con 0x55afc500c260 2024-09-10T08:42:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.346+0000 7f8df89fd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55afc5014d20 con 0x55afc500cba0 2024-09-10T08:42:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.346+0000 7f8df89fd740 10 client.0 _ll_register_callbacks cb 0x55afc4f4ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:42:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.348+0000 7f8dee000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc500cba0 0x55afc5016a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.348+0000 7f8def400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afc500c260 0x55afc50164d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.348+0000 7f8def400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afc500c260 0x55afc50164d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:46470/0 (socket says 172.21.15.89:46470) 2024-09-10T08:42:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.348+0000 7f8def400640 1 -- 172.21.15.89:0/405429764 learned_addr learned my addr 172.21.15.89:0/405429764 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.353+0000 7f8dee000640 1 -- 172.21.15.89:0/405429764 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afc500c260 msgr2=0x55afc50164d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.353+0000 7f8dee000640 1 --2- 172.21.15.89:0/405429764 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afc500c260 0x55afc50164d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.353+0000 7f8dee000640 1 -- 172.21.15.89:0/405429764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc5011c30 msgr2=0x55afc501da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.353+0000 7f8dee000640 1 --2- 172.21.15.89:0/405429764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc5011c30 0x55afc501da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.353+0000 7f8dee000640 1 -- 172.21.15.89:0/405429764 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afc4d50620 con 0x55afc500cba0 2024-09-10T08:42:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.353+0000 7f8dee000640 1 --2- 172.21.15.89:0/405429764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc500cba0 0x55afc5016a10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55afc500d5d0 tx=0x7f8dd800ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2d743f3ba01f31c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.353+0000 7f8ddcc00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:42:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.354+0000 7f8ddcc00640 1 -- 172.21.15.89:0/405429764 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8004d20 con 0x55afc500cba0 2024-09-10T08:42:55.355 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.354+0000 7f8ddcc00640 1 -- 172.21.15.89:0/405429764 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f8dd800ca90 con 0x55afc500cba0 2024-09-10T08:42:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.365+0000 7f8df89fd740 1 -- 172.21.15.89:0/405429764 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8de0004420 con 0x55afc500cba0 2024-09-10T08:42:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.368+0000 7f8ddcc00640 1 -- 172.21.15.89:0/405429764 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8dd800ee70 con 0x55afc500cba0 2024-09-10T08:42:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.368+0000 7f8df89fd740 10 client.25915 fetch_fsmap learned FSMap version 28 2024-09-10T08:42:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.368+0000 7f8df89fd740 1 -- 172.21.15.89:0/405429764 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55afc5020910 con 0x55afc500cba0 2024-09-10T08:42:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.376+0000 7f8ddcc00640 1 -- 172.21.15.89:0/405429764 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8de0004420 con 0x55afc500cba0 2024-09-10T08:42:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.376+0000 7f8ddcc00640 1 -- 172.21.15.89:0/405429764 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8dd8024440 con 0x55afc500cba0 2024-09-10T08:42:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.376+0000 7f8df89fd740 10 client.25915 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:42:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.376+0000 7f8df89fd740 10 client.25915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:42:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.376+0000 7f8df89fd740 20 client.25915 populate_metadata read hostname 'smithi089' 2024-09-10T08:42:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.376+0000 7f8df89fd740 1 --2- 172.21.15.89:0/405429764 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55afc5022110 0x55afc5024530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.376+0000 7f8df89fd740 4 client.25915 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:42:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.376+0000 7f8df89fd740 1 -- 172.21.15.89:0/405429764 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afc4f7e830 con 0x55afc5022110 2024-09-10T08:42:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.378+0000 7f8def400640 1 --2- 172.21.15.89:0/405429764 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55afc5022110 0x55afc5024530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.379+0000 7f8def400640 1 --2- 172.21.15.89:0/405429764 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55afc5022110 0x55afc5024530 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cb2a9b5f87df22b server_cookie=affa01f14d45e602 in_seq=0 out_seq=0 2024-09-10T08:42:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.379+0000 7f8ddcc00640 10 client.25915 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.389+0000 7f8ddcc00640 1 -- 172.21.15.89:0/405429764 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55afc4f7e830 con 0x55afc5022110 2024-09-10T08:42:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.389+0000 7f8ddcc00640 10 client.25915 handle_command_reply: tid=0 2024-09-10T08:42:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.389+0000 7f8df89fd740 1 client.25915 shutdown 2024-09-10T08:42:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.390+0000 7f8df89fd740 1 -- 172.21.15.89:0/405429764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc500cba0 msgr2=0x55afc5016a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.390+0000 7f8df89fd740 1 --2- 172.21.15.89:0/405429764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc500cba0 0x55afc5016a10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55afc500d5d0 tx=0x7f8dd800ec20 comp rx=0 tx=0).stop 2024-09-10T08:42:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.390+0000 7f8df89fd740 1 -- 172.21.15.89:0/405429764 shutdown_connections 2024-09-10T08:42:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.390+0000 7f8df89fd740 1 --2- 172.21.15.89:0/405429764 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55afc5022110 0x55afc5024530 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.390+0000 7f8df89fd740 1 --2- 172.21.15.89:0/405429764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc5011c30 0x55afc501da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.390+0000 7f8df89fd740 1 --2- 172.21.15.89:0/405429764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc500cba0 0x55afc5016a10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.390+0000 7f8df89fd740 1 --2- 172.21.15.89:0/405429764 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afc500c260 0x55afc50164d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.390+0000 7f8df89fd740 1 -- 172.21.15.89:0/405429764 >> 172.21.15.89:0/405429764 conn(0x55afc50078d0 msgr2=0x55afc50090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.390+0000 7f8ddcc00640 0 client.25915 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.390+0000 7f8df89fd740 1 -- 172.21.15.89:0/405429764 shutdown_connections 2024-09-10T08:42:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.390+0000 7f8df89fd740 1 -- 172.21.15.89:0/405429764 wait complete. 2024-09-10T08:42:55.394 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.392+0000 7f8df89fd740 20 client.25915 trim_cache size 0 max 16384 2024-09-10T08:42:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.513+0000 7f8df89fd740 1 Processor -- start 2024-09-10T08:42:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.513+0000 7f8df89fd740 1 -- start start 2024-09-10T08:42:55.515 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.514+0000 7f8df89fd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500c260 0x55afc5007cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.515+0000 7f8def400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500c260 0x55afc5007cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.515+0000 7f8def400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500c260 0x55afc5007cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48272/0 (socket says 172.21.15.89:48272) 2024-09-10T08:42:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.515+0000 7f8def400640 1 -- 172.21.15.89:0/2550910612 learned_addr learned my addr 172.21.15.89:0/2550910612 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.515+0000 7f8df89fd740 1 --2- 172.21.15.89:0/2550910612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc500cba0 0x55afc501da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.516+0000 7f8df89fd740 1 --2- 172.21.15.89:0/2550910612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afc5011c30 0x55afc50081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.516+0000 7f8df89fd740 1 -- 172.21.15.89:0/2550910612 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8df01c44d0 con 0x55afc500c260 2024-09-10T08:42:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.516+0000 7f8df89fd740 1 -- 172.21.15.89:0/2550910612 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8df01c1da0 con 0x55afc5011c30 2024-09-10T08:42:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.516+0000 7f8df89fd740 1 -- 172.21.15.89:0/2550910612 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8df01bfbf0 con 0x55afc500cba0 2024-09-10T08:42:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.516+0000 7f8dee000640 1 --2- 172.21.15.89:0/2550910612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc500cba0 0x55afc501da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.516+0000 7f8def400640 1 -- 172.21.15.89:0/2550910612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc500cba0 msgr2=0x55afc501da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.516+0000 7f8def400640 1 --2- 172.21.15.89:0/2550910612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc500cba0 0x55afc501da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.516+0000 7f8def400640 1 -- 172.21.15.89:0/2550910612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afc5011c30 msgr2=0x55afc50081f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:42:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.516+0000 7f8def400640 1 --2- 172.21.15.89:0/2550910612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afc5011c30 0x55afc50081f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.516+0000 7f8def400640 1 -- 172.21.15.89:0/2550910612 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afc50076f0 con 0x55afc500c260 2024-09-10T08:42:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.516+0000 7f8dee000640 1 --2- 172.21.15.89:0/2550910612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc500cba0 0x55afc501da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:42:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.517+0000 7f8def400640 1 --2- 172.21.15.89:0/2550910612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500c260 0x55afc5007cb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8de0004bb0 tx=0x7f8de0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52378f28328db2f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.517+0000 7f8ddf400640 1 -- 172.21.15.89:0/2550910612 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0047040 con 0x55afc500c260 2024-09-10T08:42:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.517+0000 7f8ddf400640 1 -- 172.21.15.89:0/2550910612 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8de0003710 con 0x55afc500c260 2024-09-10T08:42:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.518+0000 7f8ddf400640 1 -- 172.21.15.89:0/2550910612 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0047040 con 0x55afc500c260 2024-09-10T08:42:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.521+0000 7f8df89fd740 1 -- 172.21.15.89:0/2550910612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500c260 msgr2=0x55afc5007cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.521+0000 7f8df89fd740 1 --2- 172.21.15.89:0/2550910612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500c260 0x55afc5007cb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8de0004bb0 tx=0x7f8de0005fd0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.521+0000 7f8df89fd740 1 -- 172.21.15.89:0/2550910612 shutdown_connections 2024-09-10T08:42:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.521+0000 7f8df89fd740 1 --2- 172.21.15.89:0/2550910612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afc5011c30 0x55afc50081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.521+0000 7f8df89fd740 1 --2- 172.21.15.89:0/2550910612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc500cba0 0x55afc501da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.521+0000 7f8df89fd740 1 --2- 172.21.15.89:0/2550910612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500c260 0x55afc5007cb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.521+0000 7f8df89fd740 1 -- 172.21.15.89:0/2550910612 >> 172.21.15.89:0/2550910612 conn(0x55afc50078d0 msgr2=0x55afc50164d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.522+0000 7f8df89fd740 1 -- 172.21.15.89:0/2550910612 shutdown_connections 2024-09-10T08:42:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.522+0000 7f8df89fd740 1 -- 172.21.15.89:0/2550910612 wait complete. 2024-09-10T08:42:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.522+0000 7f8df89fd740 1 -- start start 2024-09-10T08:42:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.522+0000 7f8df89fd740 1 Processor -- start 2024-09-10T08:42:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.527+0000 7f8df89fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afc500c260 0x55afc501d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.527+0000 7f8df89fd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500cba0 0x55afc501a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.527+0000 7f8df89fd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc5011c30 0x55afc501aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.527+0000 7f8df89fd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8df01c0260 con 0x55afc500cba0 2024-09-10T08:42:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.527+0000 7f8df89fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8df01c16d0 con 0x55afc500c260 2024-09-10T08:42:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.527+0000 7f8df89fd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8df01c0930 con 0x55afc5011c30 2024-09-10T08:42:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.527+0000 7f8df89fd740 10 client.0 _ll_register_callbacks cb 0x7f8dd8007460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:42:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.528+0000 7f8dee000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500cba0 0x55afc501a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.528+0000 7f8dee000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500cba0 0x55afc501a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48280/0 (socket says 172.21.15.89:48280) 2024-09-10T08:42:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.528+0000 7f8dee000640 1 -- 172.21.15.89:0/1249303205 learned_addr learned my addr 172.21.15.89:0/1249303205 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.528+0000 7f8dee000640 1 -- 172.21.15.89:0/1249303205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc5011c30 msgr2=0x55afc501aa10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:42:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.528+0000 7f8dee000640 1 --2- 172.21.15.89:0/1249303205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc5011c30 0x55afc501aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.528+0000 7f8dee000640 1 -- 172.21.15.89:0/1249303205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afc500c260 msgr2=0x55afc501d9b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:42:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.528+0000 7f8dee000640 1 --2- 172.21.15.89:0/1249303205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afc500c260 0x55afc501d9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.529+0000 7f8dee000640 1 -- 172.21.15.89:0/1249303205 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afc508ae10 con 0x55afc500cba0 2024-09-10T08:42:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.529+0000 7f8dee000640 1 --2- 172.21.15.89:0/1249303205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500cba0 0x55afc501a450 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8000f80 tx=0x7f8dd8024650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5dbcbb20ad40e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.535+0000 7f8ddcc00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:42:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.535+0000 7f8ddcc00640 1 -- 172.21.15.89:0/1249303205 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8033060 con 0x55afc500cba0 2024-09-10T08:42:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.535+0000 7f8ddcc00640 1 -- 172.21.15.89:0/1249303205 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f8dd800cc90 con 0x55afc500cba0 2024-09-10T08:42:55.544 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.544+0000 7f8df89fd740 1 -- 172.21.15.89:0/1249303205 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8de0003710 con 0x55afc500cba0 2024-09-10T08:42:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.548+0000 7f8ddcc00640 1 -- 172.21.15.89:0/1249303205 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8dd8029070 con 0x55afc500cba0 2024-09-10T08:42:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.548+0000 7f8df89fd740 10 client.15921 fetch_fsmap learned FSMap version 28 2024-09-10T08:42:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.548+0000 7f8df89fd740 1 -- 172.21.15.89:0/1249303205 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55afc508c150 con 0x55afc500cba0 2024-09-10T08:42:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.550+0000 7f8ddcc00640 1 -- 172.21.15.89:0/1249303205 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8de0003710 con 0x55afc500cba0 2024-09-10T08:42:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.550+0000 7f8ddcc00640 1 -- 172.21.15.89:0/1249303205 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8dd8036050 con 0x55afc500cba0 2024-09-10T08:42:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.550+0000 7f8df89fd740 10 client.15921 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:42:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.550+0000 7f8df89fd740 10 client.15921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:42:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.550+0000 7f8df89fd740 20 client.15921 populate_metadata read hostname 'smithi089' 2024-09-10T08:42:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.550+0000 7f8df89fd740 1 --2- 172.21.15.89:0/1249303205 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55afc508cb90 0x55afc508efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.550+0000 7f8df89fd740 4 client.15921 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:42:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.550+0000 7f8df89fd740 1 -- 172.21.15.89:0/1249303205 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55afc4f78b80 con 0x55afc508cb90 2024-09-10T08:42:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.556+0000 7f8def400640 1 --2- 172.21.15.89:0/1249303205 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55afc508cb90 0x55afc508efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.556+0000 7f8def400640 1 --2- 172.21.15.89:0/1249303205 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55afc508cb90 0x55afc508efb0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52da88aec71e6b2c server_cookie=9fb3219ed3fe8451 in_seq=0 out_seq=0 2024-09-10T08:42:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:42:55.560+0000 7f8ddcc00640 10 client.15921 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:56.061 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:42:57.190 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:56 smithi089 ceph-mon[30757]: pgmap v483: 129 pgs: 129 active+clean; 160 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 344 op/s 2024-09-10T08:42:57.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:56 smithi149 ceph-mon[36167]: pgmap v483: 129 pgs: 129 active+clean; 160 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 344 op/s 2024-09-10T08:42:57.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:56 smithi111 ceph-mon[35220]: pgmap v483: 129 pgs: 129 active+clean; 160 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 344 op/s 2024-09-10T08:42:57.987 INFO:tasks.workunit.client.0.smithi089.stdout: 60 428036 280 309086 366 235863 947 2024-09-10T08:42:58.110 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:42:57 smithi089 ceph-mon[30757]: pgmap v484: 129 pgs: 129 active+clean; 178 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.4 MiB/s wr, 638 op/s 2024-09-10T08:42:58.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:42:57 smithi149 ceph-mon[36167]: pgmap v484: 129 pgs: 129 active+clean; 178 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.4 MiB/s wr, 638 op/s 2024-09-10T08:42:58.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:42:57 smithi111 ceph-mon[35220]: pgmap v484: 129 pgs: 129 active+clean; 178 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.4 MiB/s wr, 638 op/s 2024-09-10T08:42:58.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.413+0000 7f9bcf000640 1 -- 172.21.15.89:0/3094220840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bc8112640 msgr2=0x7f9bc8112a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:58.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.413+0000 7f9bcf000640 1 --2- 172.21.15.89:0/3094220840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bc8112640 0x7f9bc8112a20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009c10 tx=0x7f9bc002a710 comp rx=0 tx=0).stop 2024-09-10T08:42:58.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.413+0000 7f9bcf000640 1 -- 172.21.15.89:0/3094220840 shutdown_connections 2024-09-10T08:42:58.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.413+0000 7f9bcf000640 1 --2- 172.21.15.89:0/3094220840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bc810cf30 0x7f9bc8119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.861 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.413+0000 7f9bcf000640 1 --2- 172.21.15.89:0/3094220840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bc810c590 0x7f9bc810c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.413+0000 7f9bcf000640 1 --2- 172.21.15.89:0/3094220840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bc8112640 0x7f9bc8112a20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.413+0000 7f9bcf000640 1 -- 172.21.15.89:0/3094220840 >> 172.21.15.89:0/3094220840 conn(0x7f9bc806cdc0 msgr2=0x7f9bc806d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:58.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.413+0000 7f9bcf000640 1 -- 172.21.15.89:0/3094220840 shutdown_connections 2024-09-10T08:42:58.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.414+0000 7f9bcf000640 1 -- 172.21.15.89:0/3094220840 wait complete. 2024-09-10T08:42:58.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.416+0000 7f9bcf000640 1 Processor -- start 2024-09-10T08:42:58.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.416+0000 7f9bcf000640 1 -- start start 2024-09-10T08:42:58.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.417+0000 7f9bcf000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bc810c590 0x7f9bc81193b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:58.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.417+0000 7f9bcf000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bc810cf30 0x7f9bc8116460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:58.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.417+0000 7f9bcf000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bc8112640 0x7f9bc8116bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:58.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.417+0000 7f9bcf000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc811c9c0 con 0x7f9bc810cf30 2024-09-10T08:42:58.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.417+0000 7f9bcf000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc811c840 con 0x7f9bc8112640 2024-09-10T08:42:58.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.417+0000 7f9bcf000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc811cb40 con 0x7f9bc810c590 2024-09-10T08:42:58.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.418+0000 7f9bcd000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bc8112640 0x7f9bc8116bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:58.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.418+0000 7f9bc7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bc810c590 0x7f9bc81193b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:58.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.422+0000 7f9bcd000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bc8112640 0x7f9bc8116bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:46486/0 (socket says 172.21.15.89:46486) 2024-09-10T08:42:58.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.422+0000 7f9bcd000640 1 -- 172.21.15.89:0/3138652796 learned_addr learned my addr 172.21.15.89:0/3138652796 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:58.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.422+0000 7f9bc7400640 1 --2- 172.21.15.89:0/3138652796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bc810cf30 0x7f9bc8116460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:58.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.422+0000 7f9bc7e00640 1 -- 172.21.15.89:0/3138652796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bc8112640 msgr2=0x7f9bc8116bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:58.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.422+0000 7f9bc7e00640 1 --2- 172.21.15.89:0/3138652796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bc8112640 0x7f9bc8116bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.423+0000 7f9bc7e00640 1 -- 172.21.15.89:0/3138652796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bc810cf30 msgr2=0x7f9bc8116460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:58.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.423+0000 7f9bc7e00640 1 --2- 172.21.15.89:0/3138652796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bc810cf30 0x7f9bc8116460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.423+0000 7f9bc7e00640 1 -- 172.21.15.89:0/3138652796 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc81241d0 con 0x7f9bc810c590 2024-09-10T08:42:58.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.423+0000 7f9bc7e00640 1 --2- 172.21.15.89:0/3138652796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bc810c590 0x7f9bc81193b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009b30 tx=0x7f9bc0030620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:58.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.424+0000 7f9bc4c00640 1 -- 172.21.15.89:0/3138652796 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0041020 con 0x7f9bc810c590 2024-09-10T08:42:58.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.424+0000 7f9bc4c00640 1 -- 172.21.15.89:0/3138652796 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bc0030850 con 0x7f9bc810c590 2024-09-10T08:42:58.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.424+0000 7f9bc4c00640 1 -- 172.21.15.89:0/3138652796 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0036040 con 0x7f9bc810c590 2024-09-10T08:42:58.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.424+0000 7f9bcf000640 1 -- 172.21.15.89:0/3138652796 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc8124460 con 0x7f9bc810c590 2024-09-10T08:42:58.864 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:42:58.864 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2024-09-10T08:42:58.864 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2024-09-10T08:42:58.864 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:42:58.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.424+0000 7f9bcf000640 1 -- 172.21.15.89:0/3138652796 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc8124860 con 0x7f9bc810c590 2024-09-10T08:42:58.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.435+0000 7f9bc4c00640 1 -- 172.21.15.89:0/3138652796 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9bc0044020 con 0x7f9bc810c590 2024-09-10T08:42:58.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.439+0000 7f9bc4c00640 1 --2- 172.21.15.89:0/3138652796 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9bac0a2d10 0x7f9bac0a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:58.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.440+0000 7f9bc4c00640 1 -- 172.21.15.89:0/3138652796 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f9bc00efa70 con 0x7f9bc810c590 2024-09-10T08:42:58.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.442+0000 7f9bc7400640 1 --2- 172.21.15.89:0/3138652796 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9bac0a2d10 0x7f9bac0a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:58.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.444+0000 7f9bd1a83740 1 Processor -- start 2024-09-10T08:42:58.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.444+0000 7f9bd1a83740 1 -- start start 2024-09-10T08:42:58.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.444+0000 7f9bd1a83740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639ba0b90 0x55e639ba0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:58.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.444+0000 7f9bd1a83740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639ba14b0 0x55e639bab830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:58.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.445+0000 7f9bd1a83740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639babd70 0x55e639bae160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:58.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.445+0000 7f9bd1a83740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e639b073e0 con 0x55e639ba0b90 2024-09-10T08:42:58.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.445+0000 7f9bd1a83740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e639aaed80 con 0x55e639ba14b0 2024-09-10T08:42:58.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.445+0000 7f9bd1a83740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e639b033a0 con 0x55e639babd70 2024-09-10T08:42:58.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.447+0000 7f9bcd000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639babd70 0x55e639bae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:58.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.447+0000 7f9bcd000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639babd70 0x55e639bae160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:43466/0 (socket says 172.21.15.89:43466) 2024-09-10T08:42:58.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.447+0000 7f9bcd000640 1 -- 172.21.15.89:0/678277959 learned_addr learned my addr 172.21.15.89:0/678277959 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:58.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.447+0000 7f9bc7e00640 1 --2- 172.21.15.89:0/678277959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639ba0b90 0x55e639ba0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:58.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.448+0000 7f9bc7e00640 1 -- 172.21.15.89:0/678277959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639babd70 msgr2=0x55e639bae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:58.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.448+0000 7f9bc7e00640 1 --2- 172.21.15.89:0/678277959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639babd70 0x55e639bae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.448+0000 7f9bc7e00640 1 -- 172.21.15.89:0/678277959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639ba14b0 msgr2=0x55e639bab830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:42:58.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.448+0000 7f9bc7e00640 1 --2- 172.21.15.89:0/678277959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639ba14b0 0x55e639bab830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.448+0000 7f9bc7e00640 1 -- 172.21.15.89:0/678277959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e639acdae0 con 0x55e639ba0b90 2024-09-10T08:42:58.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.449+0000 7f9bc7400640 1 --2- 172.21.15.89:0/3138652796 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9bac0a2d10 0x7f9bac0a51d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0006fd0 tx=0x7f9bb0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:58.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.449+0000 7f9bc7e00640 1 --2- 172.21.15.89:0/678277959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639ba0b90 0x55e639ba0f70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0031040 tx=0x7f9bc0003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b74bb710baa55a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:58.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.450+0000 7f9bbd600640 1 -- 172.21.15.89:0/678277959 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc00d4070 con 0x55e639ba0b90 2024-09-10T08:42:58.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.451+0000 7f9bbd600640 1 -- 172.21.15.89:0/678277959 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bc00bbf00 con 0x55e639ba0b90 2024-09-10T08:42:58.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.452+0000 7f9bd1a83740 1 -- 172.21.15.89:0/678277959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639ba0b90 msgr2=0x55e639ba0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:58.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.453+0000 7f9bbd600640 1 -- 172.21.15.89:0/678277959 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc00c46e0 con 0x55e639ba0b90 2024-09-10T08:42:58.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.453+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/678277959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639ba0b90 0x55e639ba0f70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0031040 tx=0x7f9bc0003b70 comp rx=0 tx=0).stop 2024-09-10T08:42:58.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.456+0000 7f9bd1a83740 1 -- 172.21.15.89:0/678277959 shutdown_connections 2024-09-10T08:42:58.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.457+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/678277959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639babd70 0x55e639bae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.457+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/678277959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639ba14b0 0x55e639bab830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.457+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/678277959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639ba0b90 0x55e639ba0f70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.457+0000 7f9bd1a83740 1 -- 172.21.15.89:0/678277959 >> 172.21.15.89:0/678277959 conn(0x55e639b9d120 msgr2=0x55e639b9d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:58.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.458+0000 7f9bd1a83740 1 -- 172.21.15.89:0/678277959 shutdown_connections 2024-09-10T08:42:58.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.458+0000 7f9bd1a83740 1 -- 172.21.15.89:0/678277959 wait complete. 2024-09-10T08:42:58.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.460+0000 7f9bd1a83740 1 -- start start 2024-09-10T08:42:58.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.461+0000 7f9bd1a83740 1 Processor -- start 2024-09-10T08:42:58.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.463+0000 7f9bd1a83740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639ba0b90 0x55e639ba8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:58.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.464+0000 7f9bd1a83740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639ba14b0 0x55e639ba91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:58.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.464+0000 7f9bc7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639ba0b90 0x55e639ba8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:58.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.464+0000 7f9bc7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639ba0b90 0x55e639ba8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:43476/0 (socket says 172.21.15.89:43476) 2024-09-10T08:42:58.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.464+0000 7f9bc7e00640 1 -- 172.21.15.89:0/2607802366 learned_addr learned my addr 172.21.15.89:0/2607802366 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:58.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.464+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/2607802366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639babd70 0x55e639ba96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:58.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.464+0000 7f9bd1a83740 1 -- 172.21.15.89:0/2607802366 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e639b073e0 con 0x55e639babd70 2024-09-10T08:42:58.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.464+0000 7f9bd1a83740 1 -- 172.21.15.89:0/2607802366 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e639ac20f0 con 0x55e639ba14b0 2024-09-10T08:42:58.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.464+0000 7f9bd1a83740 1 -- 172.21.15.89:0/2607802366 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e639bae700 con 0x55e639ba0b90 2024-09-10T08:42:58.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.464+0000 7f9bd1a83740 10 client.0 _ll_register_callbacks cb 0x55e639ab39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:42:58.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.465+0000 7f9bc7400640 1 --2- 172.21.15.89:0/2607802366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639ba14b0 0x55e639ba91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:58.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.465+0000 7f9bcd000640 1 --2- 172.21.15.89:0/2607802366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639babd70 0x55e639ba96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:58.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.467+0000 7f9bc7e00640 1 -- 172.21.15.89:0/2607802366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639ba14b0 msgr2=0x55e639ba91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:58.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.467+0000 7f9bc7e00640 1 --2- 172.21.15.89:0/2607802366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639ba14b0 0x55e639ba91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.467+0000 7f9bc7e00640 1 -- 172.21.15.89:0/2607802366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639babd70 msgr2=0x55e639ba96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:58.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.467+0000 7f9bc7e00640 1 --2- 172.21.15.89:0/2607802366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639babd70 0x55e639ba96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.467+0000 7f9bc7e00640 1 -- 172.21.15.89:0/2607802366 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6398783c0 con 0x55e639ba0b90 2024-09-10T08:42:58.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.467+0000 7f9bc7e00640 1 --2- 172.21.15.89:0/2607802366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639ba0b90 0x55e639ba8c60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9bc00efd90 tx=0x7f9bc0003e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6409f347825f9807 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:58.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.467+0000 7f9bb6000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:42:58.870 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.469+0000 7f9bb6000640 1 -- 172.21.15.89:0/2607802366 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc00c9060 con 0x55e639ba0b90 2024-09-10T08:42:58.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.469+0000 7f9bb6000640 1 -- 172.21.15.89:0/2607802366 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f9bc00c48f0 con 0x55e639ba0b90 2024-09-10T08:42:58.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.469+0000 7f9bd1a83740 1 -- 172.21.15.89:0/2607802366 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc00bbf00 con 0x55e639ba0b90 2024-09-10T08:42:58.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.470+0000 7f9bb6000640 1 -- 172.21.15.89:0/2607802366 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9bc00d0050 con 0x55e639ba0b90 2024-09-10T08:42:58.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.470+0000 7f9bd1a83740 10 client.25930 fetch_fsmap learned FSMap version 28 2024-09-10T08:42:58.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.470+0000 7f9bd1a83740 1 -- 172.21.15.89:0/2607802366 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e639bba2d0 con 0x55e639ba0b90 2024-09-10T08:42:58.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.470+0000 7f9bb6000640 1 -- 172.21.15.89:0/2607802366 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bc00bbf00 con 0x55e639ba0b90 2024-09-10T08:42:58.871 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.472+0000 7f9bb6000640 1 -- 172.21.15.89:0/2607802366 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9bc00dacd0 con 0x55e639ba0b90 2024-09-10T08:42:58.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.472+0000 7f9bd1a83740 10 client.25930 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:42:58.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.472+0000 7f9bd1a83740 10 client.25930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:42:58.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.472+0000 7f9bd1a83740 20 client.25930 populate_metadata read hostname 'smithi089' 2024-09-10T08:42:58.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.472+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/2607802366 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e639bbb730 0x55e639bbdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:58.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.473+0000 7f9bd1a83740 4 client.25930 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:42:58.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.473+0000 7f9bd1a83740 1 -- 172.21.15.89:0/2607802366 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e639b14950 con 0x55e639bbb730 2024-09-10T08:42:58.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.473+0000 7f9bc7400640 1 --2- 172.21.15.89:0/2607802366 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e639bbb730 0x55e639bbdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:58.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.474+0000 7f9bc7400640 1 --2- 172.21.15.89:0/2607802366 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e639bbb730 0x55e639bbdb90 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bf0b71b614abe1 server_cookie=91f5d6a87deaa778 in_seq=0 out_seq=0 2024-09-10T08:42:58.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.474+0000 7f9bb6000640 10 client.25930 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:58.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.477+0000 7f9bb6000640 1 -- 172.21.15.89:0/2607802366 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e639b14950 con 0x55e639bbb730 2024-09-10T08:42:58.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.477+0000 7f9bb6000640 10 client.25930 handle_command_reply: tid=0 2024-09-10T08:42:58.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.478+0000 7f9bd1a83740 1 client.25930 shutdown 2024-09-10T08:42:58.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.478+0000 7f9bd1a83740 1 -- 172.21.15.89:0/2607802366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639ba0b90 msgr2=0x55e639ba8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:58.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.478+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/2607802366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639ba0b90 0x55e639ba8c60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9bc00efd90 tx=0x7f9bc0003e30 comp rx=0 tx=0).stop 2024-09-10T08:42:58.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.479+0000 7f9bd1a83740 1 -- 172.21.15.89:0/2607802366 shutdown_connections 2024-09-10T08:42:58.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.479+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/2607802366 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e639bbb730 0x55e639bbdb90 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.479+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/2607802366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639babd70 0x55e639ba96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.479+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/2607802366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639ba14b0 0x55e639ba91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.479+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/2607802366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639ba0b90 0x55e639ba8c60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.479+0000 7f9bd1a83740 1 -- 172.21.15.89:0/2607802366 >> 172.21.15.89:0/2607802366 conn(0x55e639b9d120 msgr2=0x55e639ba2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:58.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.480+0000 7f9bb6000640 0 client.25930 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:58.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.481+0000 7f9bd1a83740 1 -- 172.21.15.89:0/2607802366 shutdown_connections 2024-09-10T08:42:58.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.481+0000 7f9bd1a83740 1 -- 172.21.15.89:0/2607802366 wait complete. 2024-09-10T08:42:58.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.482+0000 7f9bd1a83740 20 client.25930 trim_cache size 0 max 16384 2024-09-10T08:42:58.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.524+0000 7f9bd1a83740 1 Processor -- start 2024-09-10T08:42:58.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.525+0000 7f9bd1a83740 1 -- start start 2024-09-10T08:42:58.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.525+0000 7f9bd1a83740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639ba0b90 0x55e639ba7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:58.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.525+0000 7f9bd1a83740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639ba14b0 0x55e639b9c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:58.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.525+0000 7f9bd1a83740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639babd70 0x55e639bc3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:58.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.525+0000 7f9bd1a83740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc81b7d10 con 0x55e639ba14b0 2024-09-10T08:42:58.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.525+0000 7f9bd1a83740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc81b55e0 con 0x55e639babd70 2024-09-10T08:42:58.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.525+0000 7f9bd1a83740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc81b3430 con 0x55e639ba0b90 2024-09-10T08:42:58.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.526+0000 7f9bc7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639ba0b90 0x55e639ba7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:58.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.526+0000 7f9bc7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639ba0b90 0x55e639ba7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:43482/0 (socket says 172.21.15.89:43482) 2024-09-10T08:42:58.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.526+0000 7f9bc7e00640 1 -- 172.21.15.89:0/1488555594 learned_addr learned my addr 172.21.15.89:0/1488555594 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:58.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.526+0000 7f9bcd000640 1 --2- 172.21.15.89:0/1488555594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639babd70 0x55e639bc3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:58.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.527+0000 7f9bc7400640 1 --2- 172.21.15.89:0/1488555594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639ba14b0 0x55e639b9c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:58.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.528+0000 7f9bcd000640 1 -- 172.21.15.89:0/1488555594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639ba0b90 msgr2=0x55e639ba7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:58.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.528+0000 7f9bcd000640 1 --2- 172.21.15.89:0/1488555594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639ba0b90 0x55e639ba7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.528+0000 7f9bcd000640 1 -- 172.21.15.89:0/1488555594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639ba14b0 msgr2=0x55e639b9c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:58.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.528+0000 7f9bcd000640 1 --2- 172.21.15.89:0/1488555594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639ba14b0 0x55e639b9c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.528+0000 7f9bcd000640 1 -- 172.21.15.89:0/1488555594 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e639ba7c10 con 0x55e639babd70 2024-09-10T08:42:58.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.528+0000 7f9bc7400640 1 --2- 172.21.15.89:0/1488555594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639ba14b0 0x55e639b9c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:42:58.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.528+0000 7f9bc7e00640 1 --2- 172.21.15.89:0/1488555594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639ba0b90 0x55e639ba7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:42:58.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.529+0000 7f9bcd000640 1 --2- 172.21.15.89:0/1488555594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639babd70 0x55e639bc3460 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9bb80096d0 tx=0x7f9bb8010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c05d094136cb403 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:58.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.532+0000 7f9bbcc00640 1 -- 172.21.15.89:0/1488555594 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8005020 con 0x55e639babd70 2024-09-10T08:42:58.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.532+0000 7f9bbcc00640 1 -- 172.21.15.89:0/1488555594 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bb80047a0 con 0x55e639babd70 2024-09-10T08:42:58.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.532+0000 7f9bbcc00640 1 -- 172.21.15.89:0/1488555594 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb801c6c0 con 0x55e639babd70 2024-09-10T08:42:58.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.533+0000 7f9bd1a83740 1 -- 172.21.15.89:0/1488555594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639babd70 msgr2=0x55e639bc3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:58.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.533+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/1488555594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639babd70 0x55e639bc3460 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9bb80096d0 tx=0x7f9bb8010800 comp rx=0 tx=0).stop 2024-09-10T08:42:58.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.533+0000 7f9bd1a83740 1 -- 172.21.15.89:0/1488555594 shutdown_connections 2024-09-10T08:42:58.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.533+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/1488555594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639babd70 0x55e639bc3460 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.533+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/1488555594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639ba14b0 0x55e639b9c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.533+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/1488555594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639ba0b90 0x55e639ba7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.533+0000 7f9bd1a83740 1 -- 172.21.15.89:0/1488555594 >> 172.21.15.89:0/1488555594 conn(0x55e639b9d120 msgr2=0x55e639b9b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:58.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.534+0000 7f9bd1a83740 1 -- 172.21.15.89:0/1488555594 shutdown_connections 2024-09-10T08:42:58.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.534+0000 7f9bd1a83740 1 -- 172.21.15.89:0/1488555594 wait complete. 2024-09-10T08:42:58.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.534+0000 7f9bd1a83740 1 -- start start 2024-09-10T08:42:58.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.535+0000 7f9bd1a83740 1 Processor -- start 2024-09-10T08:42:58.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.536+0000 7f9bd1a83740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639ba0b90 0x55e639bb3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:58.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.536+0000 7f9bd1a83740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639ba14b0 0x55e639bb4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:58.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.536+0000 7f9bd1a83740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639babd70 0x55e639bb09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:58.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.537+0000 7f9bd1a83740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc81b3aa0 con 0x55e639ba0b90 2024-09-10T08:42:58.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.537+0000 7f9bd1a83740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc81b4f10 con 0x55e639ba14b0 2024-09-10T08:42:58.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.537+0000 7f9bd1a83740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc81b4170 con 0x55e639babd70 2024-09-10T08:42:58.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.537+0000 7f9bd1a83740 10 client.0 _ll_register_callbacks cb 0x7f9bc00c0320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:42:58.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.537+0000 7f9bcd000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639babd70 0x55e639bb09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:58.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.537+0000 7f9bcd000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639babd70 0x55e639bb09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:43488/0 (socket says 172.21.15.89:43488) 2024-09-10T08:42:58.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.537+0000 7f9bcd000640 1 -- 172.21.15.89:0/835634627 learned_addr learned my addr 172.21.15.89:0/835634627 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:42:58.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.537+0000 7f9bc7400640 1 --2- 172.21.15.89:0/835634627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639ba14b0 0x55e639bb4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:58.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.538+0000 7f9bc7400640 1 -- 172.21.15.89:0/835634627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639babd70 msgr2=0x55e639bb09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:58.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.538+0000 7f9bc7400640 1 --2- 172.21.15.89:0/835634627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639babd70 0x55e639bb09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.538+0000 7f9bc7400640 1 -- 172.21.15.89:0/835634627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639ba0b90 msgr2=0x55e639bb3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:42:58.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.538+0000 7f9bc7400640 1 --2- 172.21.15.89:0/835634627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639ba0b90 0x55e639bb3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.538+0000 7f9bc7400640 1 -- 172.21.15.89:0/835634627 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e639bb11f0 con 0x55e639ba14b0 2024-09-10T08:42:58.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.539+0000 7f9bc7400640 1 --2- 172.21.15.89:0/835634627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639ba14b0 0x55e639bb4110 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55e639baa2a0 tx=0x7f9bb0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7682cb1a2fbf06f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:58.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.539+0000 7f9bb6000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:42:58.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.540+0000 7f9bb6000640 1 -- 172.21.15.89:0/835634627 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb002f060 con 0x55e639ba14b0 2024-09-10T08:42:58.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.540+0000 7f9bb6000640 1 -- 172.21.15.89:0/835634627 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f9bb000f040 con 0x55e639ba14b0 2024-09-10T08:42:58.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.541+0000 7f9bd1a83740 1 -- 172.21.15.89:0/835634627 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bb80047a0 con 0x55e639ba14b0 2024-09-10T08:42:58.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.541+0000 7f9bb6000640 1 -- 172.21.15.89:0/835634627 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9bb0014c50 con 0x55e639ba14b0 2024-09-10T08:42:58.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.541+0000 7f9bd1a83740 10 client.25988 fetch_fsmap learned FSMap version 28 2024-09-10T08:42:58.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.541+0000 7f9bd1a83740 1 -- 172.21.15.89:0/835634627 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e639c26fd0 con 0x55e639ba14b0 2024-09-10T08:42:58.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.542+0000 7f9bb6000640 1 -- 172.21.15.89:0/835634627 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bb80047a0 con 0x55e639ba14b0 2024-09-10T08:42:58.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.542+0000 7f9bb6000640 1 -- 172.21.15.89:0/835634627 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9bb0013070 con 0x55e639ba14b0 2024-09-10T08:42:58.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.542+0000 7f9bd1a83740 10 client.25988 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:42:58.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.542+0000 7f9bd1a83740 10 client.25988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:42:58.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.542+0000 7f9bd1a83740 20 client.25988 populate_metadata read hostname 'smithi089' 2024-09-10T08:42:58.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.542+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/835634627 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e639c27760 0x55e639c29b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:58.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.542+0000 7f9bd1a83740 4 client.25988 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:42:58.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.542+0000 7f9bd1a83740 1 -- 172.21.15.89:0/835634627 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e639b0ee60 con 0x55e639c27760 2024-09-10T08:42:58.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.544+0000 7f9bc7e00640 1 --2- 172.21.15.89:0/835634627 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e639c27760 0x55e639c29b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:58.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.545+0000 7f9bc7e00640 1 --2- 172.21.15.89:0/835634627 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e639c27760 0x55e639c29b80 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc88ae77b0cd4964 server_cookie=88dc5b5c5274d485 in_seq=0 out_seq=0 2024-09-10T08:42:58.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.545+0000 7f9bb6000640 10 client.25988 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:58.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.548+0000 7f9bb6000640 1 -- 172.21.15.89:0/835634627 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e639b0ee60 con 0x55e639c27760 2024-09-10T08:42:58.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.548+0000 7f9bb6000640 10 client.25988 handle_command_reply: tid=0 2024-09-10T08:42:58.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.548+0000 7f9bd1a83740 1 client.25988 shutdown 2024-09-10T08:42:58.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.548+0000 7f9bd1a83740 1 -- 172.21.15.89:0/835634627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639ba14b0 msgr2=0x55e639bb4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:58.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.548+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/835634627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639ba14b0 0x55e639bb4110 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55e639baa2a0 tx=0x7f9bb0004750 comp rx=0 tx=0).stop 2024-09-10T08:42:58.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.549+0000 7f9bd1a83740 1 -- 172.21.15.89:0/835634627 shutdown_connections 2024-09-10T08:42:58.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.549+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/835634627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e639babd70 0x55e639bb09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.549+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/835634627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e639ba14b0 0x55e639bb4110 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.549+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/835634627 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e639c27760 0x55e639c29b80 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.549+0000 7f9bd1a83740 1 --2- 172.21.15.89:0/835634627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e639ba0b90 0x55e639bb3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.549+0000 7f9bd1a83740 1 -- 172.21.15.89:0/835634627 >> 172.21.15.89:0/835634627 conn(0x55e639b9d120 msgr2=0x55e639b9b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:58.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.549+0000 7f9bb6000640 0 client.25988 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:42:58.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.549+0000 7f9bd1a83740 1 -- 172.21.15.89:0/835634627 shutdown_connections 2024-09-10T08:42:58.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.550+0000 7f9bd1a83740 1 -- 172.21.15.89:0/835634627 wait complete. 2024-09-10T08:42:58.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.550+0000 7f9bd1a83740 20 client.25988 trim_cache size 0 max 16384 2024-09-10T08:42:58.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.553+0000 7f9bcf000640 1 -- 172.21.15.89:0/3138652796 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9bac0a2d10 msgr2=0x7f9bac0a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:58.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.553+0000 7f9bcf000640 1 --2- 172.21.15.89:0/3138652796 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9bac0a2d10 0x7f9bac0a51d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0006fd0 tx=0x7f9bb0008040 comp rx=0 tx=0).stop 2024-09-10T08:42:58.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.553+0000 7f9bcf000640 1 -- 172.21.15.89:0/3138652796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bc810c590 msgr2=0x7f9bc81193b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:58.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.553+0000 7f9bcf000640 1 --2- 172.21.15.89:0/3138652796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bc810c590 0x7f9bc81193b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009b30 tx=0x7f9bc0030620 comp rx=0 tx=0).stop 2024-09-10T08:42:58.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.554+0000 7f9bcf000640 1 -- 172.21.15.89:0/3138652796 shutdown_connections 2024-09-10T08:42:58.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.554+0000 7f9bcf000640 1 --2- 172.21.15.89:0/3138652796 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9bac0a2d10 0x7f9bac0a51d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.554+0000 7f9bcf000640 1 --2- 172.21.15.89:0/3138652796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bc8112640 0x7f9bc8116bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.554+0000 7f9bcf000640 1 --2- 172.21.15.89:0/3138652796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bc810cf30 0x7f9bc8116460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.554+0000 7f9bcf000640 1 --2- 172.21.15.89:0/3138652796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bc810c590 0x7f9bc81193b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:58.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.554+0000 7f9bcf000640 1 -- 172.21.15.89:0/3138652796 >> 172.21.15.89:0/3138652796 conn(0x7f9bc806cdc0 msgr2=0x7f9bc810af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:58.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.554+0000 7f9bcf000640 1 -- 172.21.15.89:0/3138652796 shutdown_connections 2024-09-10T08:42:58.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:42:57.555+0000 7f9bcf000640 1 -- 172.21.15.89:0/3138652796 wait complete. 2024-09-10T08:43:00.143 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:43:00.144 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:43:00.906 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:01.007 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:00 smithi089 ceph-mon[30757]: pgmap v485: 129 pgs: 129 active+clean; 183 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 685 op/s 2024-09-10T08:43:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.024+0000 7f8ddcc00640 1 -- 172.21.15.89:0/1249303205 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55afc4f78b80 con 0x55afc508cb90 2024-09-10T08:43:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.024+0000 7f8ddcc00640 10 client.15921 handle_command_reply: tid=0 2024-09-10T08:43:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.024+0000 7f8df89fd740 1 client.15921 shutdown 2024-09-10T08:43:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.025+0000 7f8df89fd740 1 -- 172.21.15.89:0/1249303205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500cba0 msgr2=0x55afc501a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.025+0000 7f8df89fd740 1 --2- 172.21.15.89:0/1249303205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500cba0 0x55afc501a450 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8000f80 tx=0x7f8dd8024650 comp rx=0 tx=0).stop 2024-09-10T08:43:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.025+0000 7f8df89fd740 1 -- 172.21.15.89:0/1249303205 shutdown_connections 2024-09-10T08:43:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.025+0000 7f8df89fd740 1 --2- 172.21.15.89:0/1249303205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55afc5011c30 0x55afc501aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.025+0000 7f8df89fd740 1 --2- 172.21.15.89:0/1249303205 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55afc508cb90 0x55afc508efb0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.025+0000 7f8df89fd740 1 --2- 172.21.15.89:0/1249303205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55afc500cba0 0x55afc501a450 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.025+0000 7f8df89fd740 1 --2- 172.21.15.89:0/1249303205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afc500c260 0x55afc501d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.025+0000 7f8ddcc00640 0 client.15921 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.025+0000 7f8df89fd740 1 -- 172.21.15.89:0/1249303205 >> 172.21.15.89:0/1249303205 conn(0x55afc50078d0 msgr2=0x55afc50061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.026+0000 7f8df89fd740 1 -- 172.21.15.89:0/1249303205 shutdown_connections 2024-09-10T08:43:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.026+0000 7f8df89fd740 1 -- 172.21.15.89:0/1249303205 wait complete. 2024-09-10T08:43:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.026+0000 7f8df89fd740 20 client.15921 trim_cache size 0 max 16384 2024-09-10T08:43:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:43:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:43:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:43:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:43:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.027+0000 7f8df5c00640 1 -- 172.21.15.89:0/1288672070 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8dd00a2d90 msgr2=0x7f8dd00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.027+0000 7f8df5c00640 1 --2- 172.21.15.89:0/1288672070 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8dd00a2d90 0x7f8dd00a5250 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8de80cbb80 tx=0x7f8de80046e0 comp rx=0 tx=0).stop 2024-09-10T08:43:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.027+0000 7f8df5c00640 1 -- 172.21.15.89:0/1288672070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8df0072d80 msgr2=0x7f8df0115820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.027+0000 7f8df5c00640 1 --2- 172.21.15.89:0/1288672070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8df0072d80 0x7f8df0115820 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8de800b6d0 tx=0x7f8de800bb90 comp rx=0 tx=0).stop 2024-09-10T08:43:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.027+0000 7f8df5c00640 1 -- 172.21.15.89:0/1288672070 shutdown_connections 2024-09-10T08:43:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.027+0000 7f8df5c00640 1 --2- 172.21.15.89:0/1288672070 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8dd00a2d90 0x7f8dd00a5250 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.027+0000 7f8df5c00640 1 --2- 172.21.15.89:0/1288672070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8df010de90 0x7f8df01b6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.028+0000 7f8df5c00640 1 --2- 172.21.15.89:0/1288672070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8df010b4e0 0x7f8df0115d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.028+0000 7f8df5c00640 1 --2- 172.21.15.89:0/1288672070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8df0072d80 0x7f8df0115820 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.028+0000 7f8df5c00640 1 -- 172.21.15.89:0/1288672070 >> 172.21.15.89:0/1288672070 conn(0x7f8df006eb50 msgr2=0x7f8df010cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.028+0000 7f8df5c00640 1 -- 172.21.15.89:0/1288672070 shutdown_connections 2024-09-10T08:43:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:01.028+0000 7f8df5c00640 1 -- 172.21.15.89:0/1288672070 wait complete. 2024-09-10T08:43:01.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:00 smithi111 ceph-mon[35220]: pgmap v485: 129 pgs: 129 active+clean; 183 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 685 op/s 2024-09-10T08:43:01.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:00 smithi149 ceph-mon[36167]: pgmap v485: 129 pgs: 129 active+clean; 183 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 685 op/s 2024-09-10T08:43:02.790 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:02 smithi089 ceph-mon[30757]: pgmap v486: 129 pgs: 129 active+clean; 214 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.4 MiB/s wr, 1.51k op/s 2024-09-10T08:43:03.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.033+0000 7f0060a00640 1 -- 172.21.15.89:0/175449667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f005c1012a0 msgr2=0x7f005c105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:03.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.033+0000 7f0060a00640 1 --2- 172.21.15.89:0/175449667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f005c1012a0 0x7f005c105770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f004c009900 tx=0x7f004c02a470 comp rx=0 tx=0).stop 2024-09-10T08:43:03.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.033+0000 7f0060a00640 1 -- 172.21.15.89:0/175449667 shutdown_connections 2024-09-10T08:43:03.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.033+0000 7f0060a00640 1 --2- 172.21.15.89:0/175449667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f005c105d90 0x7f005c106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:03.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.033+0000 7f0060a00640 1 --2- 172.21.15.89:0/175449667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f005c1012a0 0x7f005c105770 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:03.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.033+0000 7f0060a00640 1 --2- 172.21.15.89:0/175449667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f005c100980 0x7f005c100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:03.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.033+0000 7f0060a00640 1 -- 172.21.15.89:0/175449667 >> 172.21.15.89:0/175449667 conn(0x7f005c0fc610 msgr2=0x7f005c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:03.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.033+0000 7f0060a00640 1 -- 172.21.15.89:0/175449667 shutdown_connections 2024-09-10T08:43:03.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.033+0000 7f0060a00640 1 -- 172.21.15.89:0/175449667 wait complete. 2024-09-10T08:43:03.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.035+0000 7f0060a00640 1 Processor -- start 2024-09-10T08:43:03.037 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.036+0000 7f0060a00640 1 -- start start 2024-09-10T08:43:03.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.036+0000 7f0060a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f005c100980 0x7f005c114b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:03.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.036+0000 7f0060a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f005c1012a0 0x7f005c10dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:03.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.037+0000 7f005a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f005c100980 0x7f005c114b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:03.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.037+0000 7f005a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f005c100980 0x7f005c114b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:42646/0 (socket says 172.21.15.89:42646) 2024-09-10T08:43:03.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.037+0000 7f005a800640 1 -- 172.21.15.89:0/1254284594 learned_addr learned my addr 172.21.15.89:0/1254284594 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:03.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.037+0000 7f0060a00640 1 --2- 172.21.15.89:0/1254284594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f005c105d90 0x7f005c10e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:03.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.038+0000 7f0060a00640 1 -- 172.21.15.89:0/1254284594 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1183e0 con 0x7f005c105d90 2024-09-10T08:43:03.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.038+0000 7f0060a00640 1 -- 172.21.15.89:0/1254284594 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f005c118260 con 0x7f005c100980 2024-09-10T08:43:03.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.038+0000 7f0060a00640 1 -- 172.21.15.89:0/1254284594 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f005c118560 con 0x7f005c1012a0 2024-09-10T08:43:03.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.038+0000 7f0059e00640 1 --2- 172.21.15.89:0/1254284594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f005c1012a0 0x7f005c10dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:02 smithi111 ceph-mon[35220]: pgmap v486: 129 pgs: 129 active+clean; 214 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.4 MiB/s wr, 1.51k op/s 2024-09-10T08:43:03.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.039+0000 7f005b200640 1 --2- 172.21.15.89:0/1254284594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f005c105d90 0x7f005c10e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:03.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.039+0000 7f0059e00640 1 -- 172.21.15.89:0/1254284594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f005c100980 msgr2=0x7f005c114b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:03.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.039+0000 7f0059e00640 1 --2- 172.21.15.89:0/1254284594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f005c100980 0x7f005c114b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:03.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.039+0000 7f0059e00640 1 -- 172.21.15.89:0/1254284594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f005c105d90 msgr2=0x7f005c10e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:03.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.040+0000 7f0059e00640 1 --2- 172.21.15.89:0/1254284594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f005c105d90 0x7f005c10e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:03.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.040+0000 7f0059e00640 1 -- 172.21.15.89:0/1254284594 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005c10e720 con 0x7f005c1012a0 2024-09-10T08:43:03.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.040+0000 7f005b200640 1 --2- 172.21.15.89:0/1254284594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f005c105d90 0x7f005c10e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:43:03.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.041+0000 7f0059e00640 1 --2- 172.21.15.89:0/1254284594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f005c1012a0 0x7f005c10dc20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f004c02aa60 tx=0x7f004c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:03.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.043+0000 7f0053400640 1 -- 172.21.15.89:0/1254284594 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c03f070 con 0x7f005c1012a0 2024-09-10T08:43:03.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.044+0000 7f0060a00640 1 -- 172.21.15.89:0/1254284594 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005c10e9b0 con 0x7f005c1012a0 2024-09-10T08:43:03.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.045+0000 7f0053400640 1 -- 172.21.15.89:0/1254284594 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004c031c90 con 0x7f005c1012a0 2024-09-10T08:43:03.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.045+0000 7f0053400640 1 -- 172.21.15.89:0/1254284594 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c03f070 con 0x7f005c1012a0 2024-09-10T08:43:03.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.045+0000 7f0060a00640 1 -- 172.21.15.89:0/1254284594 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005c112940 con 0x7f005c1012a0 2024-09-10T08:43:03.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.051+0000 7f0060a00640 1 -- 172.21.15.89:0/1254284594 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f003c005290 con 0x7f005c1012a0 2024-09-10T08:43:03.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.055+0000 7f0053400640 1 -- 172.21.15.89:0/1254284594 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f004c042070 con 0x7f005c1012a0 2024-09-10T08:43:03.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.058+0000 7f0053400640 1 --2- 172.21.15.89:0/1254284594 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f00400a2ed0 0x7f00400a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:03.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.058+0000 7f0053400640 1 -- 172.21.15.89:0/1254284594 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f004c0eef80 con 0x7f005c1012a0 2024-09-10T08:43:03.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.059+0000 7f005a800640 1 --2- 172.21.15.89:0/1254284594 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f00400a2ed0 0x7f00400a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:03.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.061+0000 7f0053400640 1 -- 172.21.15.89:0/1254284594 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f004c0bbab0 con 0x7f005c1012a0 2024-09-10T08:43:03.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.061+0000 7f005a800640 1 --2- 172.21.15.89:0/1254284594 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f00400a2ed0 0x7f00400a5390 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0048005fd0 tx=0x7f004800b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:03.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:02 smithi149 ceph-mon[36167]: pgmap v486: 129 pgs: 129 active+clean; 214 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.4 MiB/s wr, 1.51k op/s 2024-09-10T08:43:03.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.532+0000 7f0060a00640 1 -- 172.21.15.89:0/1254284594 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f003c005580 con 0x7f005c1012a0 2024-09-10T08:43:03.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.532+0000 7f0053400640 1 -- 172.21.15.89:0/1254284594 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11921 (secure 0 0 0) 0x7f004c0bb3b0 con 0x7f005c1012a0 2024-09-10T08:43:03.540 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:43:03.540 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":28,"btime":"2024-09-10T08:41:55:762565+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:41:55.148062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24518,24496]},"id":1}]} 2024-09-10T08:43:03.540 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 28 2024-09-10T08:43:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.541+0000 7f0060a00640 1 -- 172.21.15.89:0/1254284594 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f00400a2ed0 msgr2=0x7f00400a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.541+0000 7f0060a00640 1 --2- 172.21.15.89:0/1254284594 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f00400a2ed0 0x7f00400a5390 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0048005fd0 tx=0x7f004800b480 comp rx=0 tx=0).stop 2024-09-10T08:43:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.541+0000 7f0060a00640 1 -- 172.21.15.89:0/1254284594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f005c1012a0 msgr2=0x7f005c10dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.541+0000 7f0060a00640 1 --2- 172.21.15.89:0/1254284594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f005c1012a0 0x7f005c10dc20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f004c02aa60 tx=0x7f004c003b90 comp rx=0 tx=0).stop 2024-09-10T08:43:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.541+0000 7f0060a00640 1 -- 172.21.15.89:0/1254284594 shutdown_connections 2024-09-10T08:43:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.541+0000 7f0060a00640 1 --2- 172.21.15.89:0/1254284594 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f00400a2ed0 0x7f00400a5390 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.541+0000 7f0060a00640 1 --2- 172.21.15.89:0/1254284594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f005c105d90 0x7f005c10e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.541+0000 7f0060a00640 1 --2- 172.21.15.89:0/1254284594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f005c1012a0 0x7f005c10dc20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:03.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.541+0000 7f0060a00640 1 --2- 172.21.15.89:0/1254284594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f005c100980 0x7f005c114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:03.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.541+0000 7f0060a00640 1 -- 172.21.15.89:0/1254284594 >> 172.21.15.89:0/1254284594 conn(0x7f005c0fc610 msgr2=0x7f005c0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:03.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.545+0000 7f0060a00640 1 -- 172.21.15.89:0/1254284594 shutdown_connections 2024-09-10T08:43:03.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:03.545+0000 7f0060a00640 1 -- 172.21.15.89:0/1254284594 wait complete. 2024-09-10T08:43:03.764 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:03 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1254284594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:04.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:03 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1254284594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:04.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:03 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1254284594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:04.879 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:04 smithi089 ceph-mon[30757]: pgmap v487: 129 pgs: 129 active+clean; 221 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.4 MiB/s wr, 1.73k op/s 2024-09-10T08:43:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:04 smithi111 ceph-mon[35220]: pgmap v487: 129 pgs: 129 active+clean; 221 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.4 MiB/s wr, 1.73k op/s 2024-09-10T08:43:05.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:04 smithi149 ceph-mon[36167]: pgmap v487: 129 pgs: 129 active+clean; 221 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.4 MiB/s wr, 1.73k op/s 2024-09-10T08:43:05.592 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:43:06.778 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:06 smithi089 ceph-mon[30757]: pgmap v488: 129 pgs: 129 active+clean; 224 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.2 MiB/s wr, 1.71k op/s 2024-09-10T08:43:07.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:06 smithi111 ceph-mon[35220]: pgmap v488: 129 pgs: 129 active+clean; 224 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.2 MiB/s wr, 1.71k op/s 2024-09-10T08:43:07.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:06 smithi149 ceph-mon[36167]: pgmap v488: 129 pgs: 129 active+clean; 224 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.2 MiB/s wr, 1.71k op/s 2024-09-10T08:43:07.799 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:07 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:43:08.033 INFO:tasks.workunit.client.0.smithi089.stdout: 66 364597 340 264475 294 196543 956 2024-09-10T08:43:08.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:07 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:43:08.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:07 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:43:09.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:08 smithi089 ceph-mon[30757]: pgmap v489: 129 pgs: 129 active+clean; 203 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.2 MiB/s wr, 1.69k op/s 2024-09-10T08:43:09.421 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:08 smithi111 ceph-mon[35220]: pgmap v489: 129 pgs: 129 active+clean; 203 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.2 MiB/s wr, 1.69k op/s 2024-09-10T08:43:09.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:08 smithi149 ceph-mon[36167]: pgmap v489: 129 pgs: 129 active+clean; 203 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.2 MiB/s wr, 1.69k op/s 2024-09-10T08:43:10.809 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:11.084 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:10 smithi089 ceph-mon[30757]: pgmap v490: 129 pgs: 129 active+clean; 207 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 1.43k op/s 2024-09-10T08:43:11.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:10 smithi149 ceph-mon[36167]: pgmap v490: 129 pgs: 129 active+clean; 207 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 1.43k op/s 2024-09-10T08:43:11.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:10 smithi111 ceph-mon[35220]: pgmap v490: 129 pgs: 129 active+clean; 207 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 1.43k op/s 2024-09-10T08:43:12.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.731+0000 7ff1ab000640 1 -- 172.21.15.89:0/2079195871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a4100100 msgr2=0x7ff1a4100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:12.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.731+0000 7ff1ab000640 1 --2- 172.21.15.89:0/2079195871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a4100100 0x7ff1a4100520 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff190009900 tx=0x7ff19002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.736+0000 7ff1ab000640 1 -- 172.21.15.89:0/2079195871 shutdown_connections 2024-09-10T08:43:12.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.736+0000 7ff1ab000640 1 --2- 172.21.15.89:0/2079195871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1a4100c80 0x7ff1a4110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.736+0000 7ff1ab000640 1 --2- 172.21.15.89:0/2079195871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a4100100 0x7ff1a4100520 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.736+0000 7ff1ab000640 1 --2- 172.21.15.89:0/2079195871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1a4102a50 0x7ff1a4102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.736+0000 7ff1ab000640 1 -- 172.21.15.89:0/2079195871 >> 172.21.15.89:0/2079195871 conn(0x7ff1a4077230 msgr2=0x7ff1a40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:12.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.738+0000 7ff1ab000640 1 -- 172.21.15.89:0/2079195871 shutdown_connections 2024-09-10T08:43:12.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.738+0000 7ff1ab000640 1 -- 172.21.15.89:0/2079195871 wait complete. 2024-09-10T08:43:12.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.740+0000 7ff1ab000640 1 Processor -- start 2024-09-10T08:43:12.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.740+0000 7ff1ab000640 1 -- start start 2024-09-10T08:43:12.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.741+0000 7ff1ab000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1a4100100 0x7ff1a4073d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:12.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.742+0000 7ff1ab000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1a4100c80 0x7ff1a406bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:12.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.742+0000 7ff1ab000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a4102a50 0x7ff1a406c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:12.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.742+0000 7ff1ab000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a4114150 con 0x7ff1a4100c80 2024-09-10T08:43:12.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.742+0000 7ff1ab000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a4113fd0 con 0x7ff1a4100100 2024-09-10T08:43:12.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.742+0000 7ff1ab000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a41142d0 con 0x7ff1a4102a50 2024-09-10T08:43:12.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.743+0000 7ff1a3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1a4100100 0x7ff1a4073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:12.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.743+0000 7ff1a3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1a4100100 0x7ff1a4073d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:42664/0 (socket says 172.21.15.89:42664) 2024-09-10T08:43:12.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.743+0000 7ff1a3e00640 1 -- 172.21.15.89:0/1459124756 learned_addr learned my addr 172.21.15.89:0/1459124756 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:12.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.743+0000 7ff1a3400640 1 --2- 172.21.15.89:0/1459124756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1a4100c80 0x7ff1a406bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:12.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.743+0000 7ff1a9000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a4102a50 0x7ff1a406c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:12.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.744+0000 7ff1a9000640 1 -- 172.21.15.89:0/1459124756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1a4100100 msgr2=0x7ff1a4073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:12.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.744+0000 7ff1a9000640 1 --2- 172.21.15.89:0/1459124756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1a4100100 0x7ff1a4073d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.744+0000 7ff1a9000640 1 -- 172.21.15.89:0/1459124756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1a4100c80 msgr2=0x7ff1a406bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:12.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.744+0000 7ff1a9000640 1 --2- 172.21.15.89:0/1459124756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1a4100c80 0x7ff1a406bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.744+0000 7ff1a9000640 1 -- 172.21.15.89:0/1459124756 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a406ca40 con 0x7ff1a4102a50 2024-09-10T08:43:12.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.748+0000 7ff1a9000640 1 --2- 172.21.15.89:0/1459124756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a4102a50 0x7ff1a406c220 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00ca30 tx=0x7ff19c00cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:12.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.748+0000 7ff1a0c00640 1 -- 172.21.15.89:0/1459124756 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c004d20 con 0x7ff1a4102a50 2024-09-10T08:43:12.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.751+0000 7ff1ab000640 1 -- 172.21.15.89:0/1459124756 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a4103480 con 0x7ff1a4102a50 2024-09-10T08:43:12.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.751+0000 7ff1ab000640 1 -- 172.21.15.89:0/1459124756 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a4103990 con 0x7ff1a4102a50 2024-09-10T08:43:12.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.755+0000 7ff1a0c00640 1 -- 172.21.15.89:0/1459124756 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff19c015e20 con 0x7ff1a4102a50 2024-09-10T08:43:12.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.755+0000 7ff1a0c00640 1 -- 172.21.15.89:0/1459124756 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c01c3e0 con 0x7ff1a4102a50 2024-09-10T08:43:12.777 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.767+0000 7ff1ada20740 1 Processor -- start 2024-09-10T08:43:12.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.767+0000 7ff1a0c00640 1 -- 172.21.15.89:0/1459124756 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff19c01c580 con 0x7ff1a4102a50 2024-09-10T08:43:12.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.768+0000 7ff1ada20740 1 -- start start 2024-09-10T08:43:12.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.769+0000 7ff1ada20740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc01c70 0x562e9dc02050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:12.778 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.769+0000 7ff1ada20740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e9dc02590 0x562e9dc0c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:12.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.771+0000 7ff1a3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc01c70 0x562e9dc02050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:12.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.771+0000 7ff1a3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc01c70 0x562e9dc02050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:42672/0 (socket says 172.21.15.89:42672) 2024-09-10T08:43:12.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.772+0000 7ff1a3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e9dc02590 0x562e9dc0c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:12.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.772+0000 7ff1a3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e9dc02590 0x562e9dc0c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:43294/0 (socket says 172.21.15.89:43294) 2024-09-10T08:43:12.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.772+0000 7ff1ada20740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc0ce50 0x562e9dc0f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:12.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.772+0000 7ff1ada20740 1 -- 172.21.15.89:0/2559203582 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562e9db683e0 con 0x562e9dc02590 2024-09-10T08:43:12.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.772+0000 7ff1a3400640 1 -- 172.21.15.89:0/2559203582 learned_addr learned my addr 172.21.15.89:0/2559203582 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:12.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.772+0000 7ff1ada20740 1 -- 172.21.15.89:0/2559203582 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562e9db0fd80 con 0x562e9dc01c70 2024-09-10T08:43:12.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.772+0000 7ff1ada20740 1 -- 172.21.15.89:0/2559203582 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562e9db643a0 con 0x562e9dc0ce50 2024-09-10T08:43:12.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.774+0000 7ff1a3400640 1 --2- 172.21.15.89:0/2559203582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc0ce50 0x562e9dc0f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:12.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.774+0000 7ff1a3400640 1 -- 172.21.15.89:0/2559203582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc0ce50 msgr2=0x562e9dc0f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:12.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.774+0000 7ff1a3400640 1 --2- 172.21.15.89:0/2559203582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc0ce50 0x562e9dc0f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.774+0000 7ff1a3400640 1 -- 172.21.15.89:0/2559203582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e9dc02590 msgr2=0x562e9dc0c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:12.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.774+0000 7ff1a3400640 1 --2- 172.21.15.89:0/2559203582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e9dc02590 0x562e9dc0c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.774+0000 7ff1a3400640 1 -- 172.21.15.89:0/2559203582 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e9db2eae0 con 0x562e9dc01c70 2024-09-10T08:43:12.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.775+0000 7ff1a3400640 1 --2- 172.21.15.89:0/2559203582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc01c70 0x562e9dc02050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff1900096c0 tx=0x7ff190009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44ce2b1463a4fcf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:12.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.776+0000 7ff1a0c00640 1 --2- 172.21.15.89:0/1459124756 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff1880a2c00 0x7ff1880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:12.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.776+0000 7ff199600640 1 -- 172.21.15.89:0/2559203582 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff190043030 con 0x562e9dc01c70 2024-09-10T08:43:12.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.777+0000 7ff199600640 1 -- 172.21.15.89:0/2559203582 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff190005500 con 0x562e9dc01c70 2024-09-10T08:43:12.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.777+0000 7ff199600640 1 -- 172.21.15.89:0/2559203582 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff190035030 con 0x562e9dc01c70 2024-09-10T08:43:12.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.777+0000 7ff1a0c00640 1 -- 172.21.15.89:0/1459124756 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff19c0d7ae0 con 0x7ff1a4102a50 2024-09-10T08:43:12.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.778+0000 7ff1ada20740 1 -- 172.21.15.89:0/2559203582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc01c70 msgr2=0x562e9dc02050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:12.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.778+0000 7ff1ada20740 1 --2- 172.21.15.89:0/2559203582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc01c70 0x562e9dc02050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff1900096c0 tx=0x7ff190009650 comp rx=0 tx=0).stop 2024-09-10T08:43:12.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.778+0000 7ff1ada20740 1 -- 172.21.15.89:0/2559203582 shutdown_connections 2024-09-10T08:43:12.792 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.778+0000 7ff1ada20740 1 --2- 172.21.15.89:0/2559203582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc0ce50 0x562e9dc0f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.779+0000 7ff1ada20740 1 --2- 172.21.15.89:0/2559203582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e9dc02590 0x562e9dc0c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.779+0000 7ff1ada20740 1 --2- 172.21.15.89:0/2559203582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc01c70 0x562e9dc02050 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.779+0000 7ff1ada20740 1 -- 172.21.15.89:0/2559203582 >> 172.21.15.89:0/2559203582 conn(0x562e9dbfe200 msgr2=0x562e9dbfe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:12.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.779+0000 7ff1a3e00640 1 --2- 172.21.15.89:0/1459124756 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff1880a2c00 0x7ff1880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:12.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.779+0000 7ff1ada20740 1 -- 172.21.15.89:0/2559203582 shutdown_connections 2024-09-10T08:43:12.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.779+0000 7ff1ada20740 1 -- 172.21.15.89:0/2559203582 wait complete. 2024-09-10T08:43:12.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.780+0000 7ff1ada20740 1 -- start start 2024-09-10T08:43:12.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.782+0000 7ff1a3e00640 1 --2- 172.21.15.89:0/1459124756 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff1880a2c00 0x7ff1880a50c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x562e9dbff060 tx=0x7ff19402a480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:12.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.783+0000 7ff1ada20740 1 Processor -- start 2024-09-10T08:43:12.796 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.787+0000 7ff1ada20740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 0x562e9dc09cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:12.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.788+0000 7ff1ada20740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e9dc02590 0x562e9dc0a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:12.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.788+0000 7ff1ada20740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc0ce50 0x562e9dc0a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:12.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.788+0000 7ff1a3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 0x562e9dc09cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:12.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.788+0000 7ff1a3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 0x562e9dc09cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:43520/0 (socket says 172.21.15.89:43520) 2024-09-10T08:43:12.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.788+0000 7ff1ada20740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562e9dc0fa20 con 0x562e9dc02590 2024-09-10T08:43:12.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.788+0000 7ff1a3e00640 1 -- 172.21.15.89:0/4022187856 learned_addr learned my addr 172.21.15.89:0/4022187856 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:12.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.788+0000 7ff1ada20740 1 -- 172.21.15.89:0/4022187856 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562e9db230f0 con 0x562e9dc0ce50 2024-09-10T08:43:12.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.788+0000 7ff1ada20740 1 -- 172.21.15.89:0/4022187856 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562e9dc0fba0 con 0x562e9dc01c70 2024-09-10T08:43:12.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.789+0000 7ff1ada20740 10 client.0 _ll_register_callbacks cb 0x562e9db0abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:43:12.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.789+0000 7ff1a3400640 1 --2- 172.21.15.89:0/4022187856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e9dc02590 0x562e9dc0a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:12.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.789+0000 7ff1a9000640 1 --2- 172.21.15.89:0/4022187856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc0ce50 0x562e9dc0a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:12.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.790+0000 7ff1a3e00640 1 -- 172.21.15.89:0/4022187856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc0ce50 msgr2=0x562e9dc0a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:12.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.790+0000 7ff1a3e00640 1 --2- 172.21.15.89:0/4022187856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc0ce50 0x562e9dc0a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.797 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.790+0000 7ff1a3e00640 1 -- 172.21.15.89:0/4022187856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e9dc02590 msgr2=0x562e9dc0a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:12.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.790+0000 7ff1a3e00640 1 --2- 172.21.15.89:0/4022187856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e9dc02590 0x562e9dc0a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.790+0000 7ff1a3e00640 1 -- 172.21.15.89:0/4022187856 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e9d8d93c0 con 0x562e9dc01c70 2024-09-10T08:43:12.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.791+0000 7ff18e000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:43:12.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.791+0000 7ff1a3e00640 1 --2- 172.21.15.89:0/4022187856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 0x562e9dc09cc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff194030940 tx=0x7ff1940046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11cba42059da66cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:12.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.793+0000 7ff18e000640 1 -- 172.21.15.89:0/4022187856 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194030e00 con 0x562e9dc01c70 2024-09-10T08:43:12.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.793+0000 7ff18e000640 1 -- 172.21.15.89:0/4022187856 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff19403fb40 con 0x562e9dc01c70 2024-09-10T08:43:12.798 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.794+0000 7ff1ada20740 1 -- 172.21.15.89:0/4022187856 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff190005500 con 0x562e9dc01c70 2024-09-10T08:43:12.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.797+0000 7ff18e000640 1 -- 172.21.15.89:0/4022187856 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff19403fd40 con 0x562e9dc01c70 2024-09-10T08:43:12.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.797+0000 7ff1ada20740 10 client.25954 fetch_fsmap learned FSMap version 28 2024-09-10T08:43:12.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.797+0000 7ff1ada20740 1 -- 172.21.15.89:0/4022187856 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e9dc1bb00 con 0x562e9dc01c70 2024-09-10T08:43:12.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.797+0000 7ff18e000640 1 -- 172.21.15.89:0/4022187856 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff190005500 con 0x562e9dc01c70 2024-09-10T08:43:12.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.797+0000 7ff18e000640 1 -- 172.21.15.89:0/4022187856 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff19403d670 con 0x562e9dc01c70 2024-09-10T08:43:12.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.797+0000 7ff1ada20740 10 client.25954 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:43:12.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.797+0000 7ff1ada20740 10 client.25954 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:43:12.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.798+0000 7ff1ada20740 20 client.25954 populate_metadata read hostname 'smithi089' 2024-09-10T08:43:12.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.798+0000 7ff1ada20740 1 --2- 172.21.15.89:0/4022187856 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x562e9dc1d060 0x562e9dc1f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:12.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.798+0000 7ff1ada20740 4 client.25954 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:43:12.799 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.798+0000 7ff1ada20740 1 -- 172.21.15.89:0/4022187856 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e9db75950 con 0x562e9dc1d060 2024-09-10T08:43:12.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.799+0000 7ff1a3400640 1 --2- 172.21.15.89:0/4022187856 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x562e9dc1d060 0x562e9dc1f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:12.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.802+0000 7ff1a3400640 1 --2- 172.21.15.89:0/4022187856 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x562e9dc1d060 0x562e9dc1f520 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d86ca99b2a3707f0 server_cookie=a9613991369291d8 in_seq=0 out_seq=0 2024-09-10T08:43:12.802 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.802+0000 7ff18e000640 10 client.25954 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:43:12.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.806+0000 7ff18e000640 1 -- 172.21.15.89:0/4022187856 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e9db75950 con 0x562e9dc1d060 2024-09-10T08:43:12.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.806+0000 7ff18e000640 10 client.25954 handle_command_reply: tid=0 2024-09-10T08:43:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.806+0000 7ff1ada20740 1 client.25954 shutdown 2024-09-10T08:43:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.806+0000 7ff1ada20740 1 -- 172.21.15.89:0/4022187856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 msgr2=0x562e9dc09cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.807+0000 7ff1ada20740 1 --2- 172.21.15.89:0/4022187856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 0x562e9dc09cc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff194030940 tx=0x7ff1940046c0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.807+0000 7ff1ada20740 1 -- 172.21.15.89:0/4022187856 shutdown_connections 2024-09-10T08:43:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.807+0000 7ff1ada20740 1 --2- 172.21.15.89:0/4022187856 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x562e9dc1d060 0x562e9dc1f520 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.807+0000 7ff1ada20740 1 --2- 172.21.15.89:0/4022187856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc0ce50 0x562e9dc0a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.807+0000 7ff1ada20740 1 --2- 172.21.15.89:0/4022187856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e9dc02590 0x562e9dc0a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.807+0000 7ff18e000640 0 client.25954 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:43:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.807+0000 7ff1ada20740 1 --2- 172.21.15.89:0/4022187856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 0x562e9dc09cc0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.807+0000 7ff1ada20740 1 -- 172.21.15.89:0/4022187856 >> 172.21.15.89:0/4022187856 conn(0x562e9dbfe200 msgr2=0x562e9dbfd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:12.808 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.808+0000 7ff1ada20740 1 -- 172.21.15.89:0/4022187856 shutdown_connections 2024-09-10T08:43:12.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.810+0000 7ff1ada20740 1 -- 172.21.15.89:0/4022187856 wait complete. 2024-09-10T08:43:12.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.810+0000 7ff1ada20740 20 client.25954 trim_cache size 0 max 16384 2024-09-10T08:43:12.896 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:43:12.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.856+0000 7ff1ada20740 1 Processor -- start 2024-09-10T08:43:12.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.857+0000 7ff1ada20740 1 -- start start 2024-09-10T08:43:12.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.858+0000 7ff1ada20740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 0x562e9dc046d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:12.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.858+0000 7ff1ada20740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e9dc02590 0x562e9dc24df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:12.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.858+0000 7ff1ada20740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc0ce50 0x562e9dc25330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:12.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.859+0000 7ff1ada20740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff184012630 con 0x562e9dc02590 2024-09-10T08:43:12.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.859+0000 7ff1ada20740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff18400ff00 con 0x562e9dc0ce50 2024-09-10T08:43:12.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.859+0000 7ff1ada20740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff18400dd50 con 0x562e9dc01c70 2024-09-10T08:43:12.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.859+0000 7ff1a3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 0x562e9dc046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:12.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.859+0000 7ff1a3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 0x562e9dc046d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:34078/0 (socket says 172.21.15.89:34078) 2024-09-10T08:43:12.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.859+0000 7ff1a3e00640 1 -- 172.21.15.89:0/2804134088 learned_addr learned my addr 172.21.15.89:0/2804134088 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:12.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.859+0000 7ff1a9000640 1 --2- 172.21.15.89:0/2804134088 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc0ce50 0x562e9dc25330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:12.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.859+0000 7ff1a3e00640 1 -- 172.21.15.89:0/2804134088 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc0ce50 msgr2=0x562e9dc25330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:12.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.859+0000 7ff1a3e00640 1 --2- 172.21.15.89:0/2804134088 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc0ce50 0x562e9dc25330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.859+0000 7ff1a3e00640 1 -- 172.21.15.89:0/2804134088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e9dc02590 msgr2=0x562e9dc24df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:12.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.859+0000 7ff1a3e00640 1 --2- 172.21.15.89:0/2804134088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e9dc02590 0x562e9dc24df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.859+0000 7ff1a3e00640 1 -- 172.21.15.89:0/2804134088 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e9dbfee90 con 0x562e9dc01c70 2024-09-10T08:43:12.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.860+0000 7ff1a3e00640 1 --2- 172.21.15.89:0/2804134088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 0x562e9dc046d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff194038040 tx=0x7ff19403f8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4987d3a34290e823 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:12.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.862+0000 7ff198c00640 1 -- 172.21.15.89:0/2804134088 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19405f070 con 0x562e9dc01c70 2024-09-10T08:43:12.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.862+0000 7ff198c00640 1 -- 172.21.15.89:0/2804134088 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff19403fb40 con 0x562e9dc01c70 2024-09-10T08:43:12.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.862+0000 7ff198c00640 1 -- 172.21.15.89:0/2804134088 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194056030 con 0x562e9dc01c70 2024-09-10T08:43:12.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.863+0000 7ff1ada20740 1 -- 172.21.15.89:0/2804134088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 msgr2=0x562e9dc046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:12.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.863+0000 7ff1ada20740 1 --2- 172.21.15.89:0/2804134088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 0x562e9dc046d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff194038040 tx=0x7ff19403f8b0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.863+0000 7ff1ada20740 1 -- 172.21.15.89:0/2804134088 shutdown_connections 2024-09-10T08:43:12.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.863+0000 7ff1ada20740 1 --2- 172.21.15.89:0/2804134088 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc0ce50 0x562e9dc25330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.863+0000 7ff1ada20740 1 --2- 172.21.15.89:0/2804134088 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e9dc02590 0x562e9dc24df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.863+0000 7ff1ada20740 1 --2- 172.21.15.89:0/2804134088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 0x562e9dc046d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.864+0000 7ff1ada20740 1 -- 172.21.15.89:0/2804134088 >> 172.21.15.89:0/2804134088 conn(0x562e9dbfe200 msgr2=0x562e9dbfd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:12.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.864+0000 7ff1ada20740 1 -- 172.21.15.89:0/2804134088 shutdown_connections 2024-09-10T08:43:12.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.864+0000 7ff1ada20740 1 -- 172.21.15.89:0/2804134088 wait complete. 2024-09-10T08:43:12.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.865+0000 7ff1ada20740 1 -- start start 2024-09-10T08:43:12.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.865+0000 7ff1ada20740 1 Processor -- start 2024-09-10T08:43:12.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.867+0000 7ff1ada20740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 0x562e9dc15230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:12.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.867+0000 7ff1ada20740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e9dc02590 0x562e9dc15770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:12.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.867+0000 7ff1ada20740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc0ce50 0x562e9dc107b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:12.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.867+0000 7ff1ada20740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff18400e3c0 con 0x562e9dc02590 2024-09-10T08:43:12.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.867+0000 7ff1ada20740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff18400f830 con 0x562e9dc0ce50 2024-09-10T08:43:12.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.867+0000 7ff1ada20740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff18400ea90 con 0x562e9dc01c70 2024-09-10T08:43:12.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.867+0000 7ff1ada20740 10 client.0 _ll_register_callbacks cb 0x7ff19402a9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:43:12.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.868+0000 7ff1a3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 0x562e9dc15230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:12.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.868+0000 7ff1a3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 0x562e9dc15230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:34092/0 (socket says 172.21.15.89:34092) 2024-09-10T08:43:12.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.868+0000 7ff1a9000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc0ce50 0x562e9dc107b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:12.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.868+0000 7ff1a9000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc0ce50 0x562e9dc107b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:38720/0 (socket says 172.21.15.89:38720) 2024-09-10T08:43:12.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.868+0000 7ff1a9000640 1 -- 172.21.15.89:0/2188059007 learned_addr learned my addr 172.21.15.89:0/2188059007 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:12.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.868+0000 7ff1a3400640 1 --2- 172.21.15.89:0/2188059007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e9dc02590 0x562e9dc15770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:12.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.869+0000 7ff1a3e00640 1 -- 172.21.15.89:0/2188059007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc0ce50 msgr2=0x562e9dc107b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:12.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.869+0000 7ff1a3e00640 1 --2- 172.21.15.89:0/2188059007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc0ce50 0x562e9dc107b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.869+0000 7ff1a3e00640 1 -- 172.21.15.89:0/2188059007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e9dc02590 msgr2=0x562e9dc15770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:12.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.869+0000 7ff1a3e00640 1 --2- 172.21.15.89:0/2188059007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e9dc02590 0x562e9dc15770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.869+0000 7ff1a3e00640 1 -- 172.21.15.89:0/2188059007 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e9dc10fe0 con 0x562e9dc01c70 2024-09-10T08:43:12.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.869+0000 7ff1a3e00640 1 --2- 172.21.15.89:0/2188059007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 0x562e9dc15230 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff19403dbb0 tx=0x7ff1940052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee4f8139121f4c0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:12.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.869+0000 7ff18e000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:43:12.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.869+0000 7ff18e000640 1 -- 172.21.15.89:0/2188059007 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19405f030 con 0x562e9dc01c70 2024-09-10T08:43:12.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.869+0000 7ff18e000640 1 -- 172.21.15.89:0/2188059007 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff19402a040 con 0x562e9dc01c70 2024-09-10T08:43:12.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.870+0000 7ff1ada20740 1 -- 172.21.15.89:0/2188059007 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff19403fb40 con 0x562e9dc01c70 2024-09-10T08:43:12.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.870+0000 7ff18e000640 1 -- 172.21.15.89:0/2188059007 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff194056070 con 0x562e9dc01c70 2024-09-10T08:43:12.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.871+0000 7ff1ada20740 10 client.25960 fetch_fsmap learned FSMap version 28 2024-09-10T08:43:12.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.871+0000 7ff1ada20740 1 -- 172.21.15.89:0/2188059007 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e9dc87a30 con 0x562e9dc01c70 2024-09-10T08:43:12.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.872+0000 7ff18e000640 1 -- 172.21.15.89:0/2188059007 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff19403fb40 con 0x562e9dc01c70 2024-09-10T08:43:12.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.873+0000 7ff18e000640 1 -- 172.21.15.89:0/2188059007 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff194004cc0 con 0x562e9dc01c70 2024-09-10T08:43:12.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.873+0000 7ff1ada20740 10 client.25960 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:43:12.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.873+0000 7ff1ada20740 10 client.25960 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:43:12.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.873+0000 7ff1ada20740 20 client.25960 populate_metadata read hostname 'smithi089' 2024-09-10T08:43:12.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.873+0000 7ff1ada20740 1 --2- 172.21.15.89:0/2188059007 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x562e9dc88080 0x562e9dc8a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:12.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.873+0000 7ff1ada20740 4 client.25960 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:43:12.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.873+0000 7ff1ada20740 1 -- 172.21.15.89:0/2188059007 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e9db6fe60 con 0x562e9dc88080 2024-09-10T08:43:12.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.873+0000 7ff1a3400640 1 --2- 172.21.15.89:0/2188059007 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x562e9dc88080 0x562e9dc8a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:12.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.874+0000 7ff1a3400640 1 --2- 172.21.15.89:0/2188059007 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x562e9dc88080 0x562e9dc8a470 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b572ddee3adb357 server_cookie=2f98e4367efa8f60 in_seq=0 out_seq=0 2024-09-10T08:43:12.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.874+0000 7ff18e000640 10 client.25960 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:43:12.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.874+0000 7ff18e000640 1 -- 172.21.15.89:0/2188059007 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e9db6fe60 con 0x562e9dc88080 2024-09-10T08:43:12.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.874+0000 7ff18e000640 10 client.25960 handle_command_reply: tid=0 2024-09-10T08:43:12.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.874+0000 7ff1ada20740 1 client.25960 shutdown 2024-09-10T08:43:12.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.875+0000 7ff1ada20740 1 -- 172.21.15.89:0/2188059007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 msgr2=0x562e9dc15230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:12.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.875+0000 7ff1ada20740 1 --2- 172.21.15.89:0/2188059007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 0x562e9dc15230 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff19403dbb0 tx=0x7ff1940052f0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.875+0000 7ff1ada20740 1 -- 172.21.15.89:0/2188059007 shutdown_connections 2024-09-10T08:43:12.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.875+0000 7ff1ada20740 1 --2- 172.21.15.89:0/2188059007 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x562e9dc88080 0x562e9dc8a470 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.875+0000 7ff1ada20740 1 --2- 172.21.15.89:0/2188059007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562e9dc0ce50 0x562e9dc107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.875+0000 7ff1ada20740 1 --2- 172.21.15.89:0/2188059007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e9dc02590 0x562e9dc15770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.875+0000 7ff1ada20740 1 --2- 172.21.15.89:0/2188059007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562e9dc01c70 0x562e9dc15230 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.875+0000 7ff1ada20740 1 -- 172.21.15.89:0/2188059007 >> 172.21.15.89:0/2188059007 conn(0x562e9dbfe200 msgr2=0x562e9dc09c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:12.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.875+0000 7ff18e000640 0 client.25960 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:43:12.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.875+0000 7ff1ada20740 1 -- 172.21.15.89:0/2188059007 shutdown_connections 2024-09-10T08:43:12.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.875+0000 7ff1ada20740 1 -- 172.21.15.89:0/2188059007 wait complete. 2024-09-10T08:43:12.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.876+0000 7ff1ada20740 20 client.25960 trim_cache size 0 max 16384 2024-09-10T08:43:12.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.878+0000 7ff1ab000640 1 -- 172.21.15.89:0/1459124756 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff1880a2c00 msgr2=0x7ff1880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:12.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.878+0000 7ff1ab000640 1 --2- 172.21.15.89:0/1459124756 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff1880a2c00 0x7ff1880a50c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x562e9dbff060 tx=0x7ff19402a480 comp rx=0 tx=0).stop 2024-09-10T08:43:12.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.878+0000 7ff1ab000640 1 -- 172.21.15.89:0/1459124756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a4102a50 msgr2=0x7ff1a406c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:12.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.878+0000 7ff1ab000640 1 --2- 172.21.15.89:0/1459124756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a4102a50 0x7ff1a406c220 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00ca30 tx=0x7ff19c00cef0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.878+0000 7ff1ab000640 1 -- 172.21.15.89:0/1459124756 shutdown_connections 2024-09-10T08:43:12.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.878+0000 7ff1ab000640 1 --2- 172.21.15.89:0/1459124756 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff1880a2c00 0x7ff1880a50c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.878+0000 7ff1ab000640 1 --2- 172.21.15.89:0/1459124756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a4102a50 0x7ff1a406c220 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.878+0000 7ff1ab000640 1 --2- 172.21.15.89:0/1459124756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1a4100c80 0x7ff1a406bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.878+0000 7ff1ab000640 1 --2- 172.21.15.89:0/1459124756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1a4100100 0x7ff1a4073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:12.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.878+0000 7ff1ab000640 1 -- 172.21.15.89:0/1459124756 >> 172.21.15.89:0/1459124756 conn(0x7ff1a4077230 msgr2=0x7ff1a40fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:12.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.878+0000 7ff1ab000640 1 -- 172.21.15.89:0/1459124756 shutdown_connections 2024-09-10T08:43:12.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:12.879+0000 7ff1ab000640 1 -- 172.21.15.89:0/1459124756 wait complete. 2024-09-10T08:43:13.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:12 smithi149 ceph-mon[36167]: pgmap v491: 129 pgs: 129 active+clean; 210 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.6 MiB/s wr, 1.40k op/s 2024-09-10T08:43:13.257 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:12 smithi089 ceph-mon[30757]: pgmap v491: 129 pgs: 129 active+clean; 210 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.6 MiB/s wr, 1.40k op/s 2024-09-10T08:43:13.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:12 smithi111 ceph-mon[35220]: pgmap v491: 129 pgs: 129 active+clean; 210 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.6 MiB/s wr, 1.40k op/s 2024-09-10T08:43:14.332 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:43:14.536 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.535+0000 7efc33e00640 1 -- 172.21.15.89:0/291789181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc34072cb0 msgr2=0x7efc340730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:14.536 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.535+0000 7efc33e00640 1 --2- 172.21.15.89:0/291789181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc34072cb0 0x7efc340730b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7efc24009900 tx=0x7efc2402a420 comp rx=0 tx=0).stop 2024-09-10T08:43:14.539 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.540+0000 7efc33e00640 1 -- 172.21.15.89:0/291789181 shutdown_connections 2024-09-10T08:43:14.540 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.540+0000 7efc33e00640 1 --2- 172.21.15.89:0/291789181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc3410dbb0 0x7efc34110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.540 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.540+0000 7efc33e00640 1 --2- 172.21.15.89:0/291789181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc34073680 0x7efc3410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.540+0000 7efc33e00640 1 --2- 172.21.15.89:0/291789181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc34072cb0 0x7efc340730b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.540+0000 7efc33e00640 1 -- 172.21.15.89:0/291789181 >> 172.21.15.89:0/291789181 conn(0x7efc3406eb50 msgr2=0x7efc34070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.544+0000 7efc33e00640 1 -- 172.21.15.89:0/291789181 shutdown_connections 2024-09-10T08:43:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.546+0000 7efc33e00640 1 -- 172.21.15.89:0/291789181 wait complete. 2024-09-10T08:43:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.547+0000 7efc33e00640 1 Processor -- start 2024-09-10T08:43:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.547+0000 7efc33e00640 1 -- start start 2024-09-10T08:43:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.548+0000 7efc33e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc34072cb0 0x7efc341a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.548+0000 7efc33e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc34073680 0x7efc341a7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.548+0000 7efc33e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc3410dbb0 0x7efc341ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.548+0000 7efc33e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efc3411c4a0 con 0x7efc34072cb0 2024-09-10T08:43:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.548+0000 7efc33e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efc3411c320 con 0x7efc34073680 2024-09-10T08:43:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.548+0000 7efc33e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efc3411c620 con 0x7efc3410dbb0 2024-09-10T08:43:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.548+0000 7efc32000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc34073680 0x7efc341a7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.548+0000 7efc32000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc34073680 0x7efc341a7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:38728/0 (socket says 172.21.15.89:38728) 2024-09-10T08:43:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.548+0000 7efc33400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc3410dbb0 0x7efc341ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.548+0000 7efc32000640 1 -- 172.21.15.89:0/3460281693 learned_addr learned my addr 172.21.15.89:0/3460281693 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.548+0000 7efc33400640 1 -- 172.21.15.89:0/3460281693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc34073680 msgr2=0x7efc341a7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.549+0000 7efc33400640 1 --2- 172.21.15.89:0/3460281693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc34073680 0x7efc341a7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.549+0000 7efc33400640 1 -- 172.21.15.89:0/3460281693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc34072cb0 msgr2=0x7efc341a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:14.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.551+0000 7efc32a00640 1 --2- 172.21.15.89:0/3460281693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc34072cb0 0x7efc341a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:14.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.551+0000 7efc33400640 1 --2- 172.21.15.89:0/3460281693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc34072cb0 0x7efc341a6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.551+0000 7efc33400640 1 -- 172.21.15.89:0/3460281693 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc34110a50 con 0x7efc3410dbb0 2024-09-10T08:43:14.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.551+0000 7efc32a00640 1 --2- 172.21.15.89:0/3460281693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc34072cb0 0x7efc341a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:43:14.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.551+0000 7efc33400640 1 --2- 172.21.15.89:0/3460281693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc3410dbb0 0x7efc341ae0d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efc2000d6b0 tx=0x7efc2000db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:14.558 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.554+0000 7efc2b400640 1 -- 172.21.15.89:0/3460281693 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20004230 con 0x7efc3410dbb0 2024-09-10T08:43:14.558 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.554+0000 7efc2b400640 1 -- 172.21.15.89:0/3460281693 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc200043d0 con 0x7efc3410dbb0 2024-09-10T08:43:14.558 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.554+0000 7efc2b400640 1 -- 172.21.15.89:0/3460281693 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20003e40 con 0x7efc3410dbb0 2024-09-10T08:43:14.558 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.558+0000 7efc33e00640 1 -- 172.21.15.89:0/3460281693 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc34110d40 con 0x7efc3410dbb0 2024-09-10T08:43:14.558 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.558+0000 7efc33e00640 1 -- 172.21.15.89:0/3460281693 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc34111210 con 0x7efc3410dbb0 2024-09-10T08:43:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.585+0000 7efc3c4c9740 1 Processor -- start 2024-09-10T08:43:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.589+0000 7efc2b400640 1 -- 172.21.15.89:0/3460281693 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7efc20009820 con 0x7efc3410dbb0 2024-09-10T08:43:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.591+0000 7efc2b400640 1 --2- 172.21.15.89:0/3460281693 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efc180a2d90 0x7efc180a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.592+0000 7efc2b400640 1 -- 172.21.15.89:0/3460281693 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7efc200cea70 con 0x7efc3410dbb0 2024-09-10T08:43:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.595+0000 7efc32000640 1 --2- 172.21.15.89:0/3460281693 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efc180a2d90 0x7efc180a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.596+0000 7efc32000640 1 --2- 172.21.15.89:0/3460281693 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efc180a2d90 0x7efc180a5250 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efc341a8030 tx=0x7efc2c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.599+0000 7efc3c4c9740 1 -- start start 2024-09-10T08:43:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.599+0000 7efc3c4c9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439260 0x56275d439660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.599+0000 7efc3c4c9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56275d439ba0 0x56275d43e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.599+0000 7efc3c4c9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d4410f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.599+0000 7efc3c4c9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56275d39e100 con 0x56275d43ec30 2024-09-10T08:43:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.599+0000 7efc3c4c9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56275d3461b0 con 0x56275d439260 2024-09-10T08:43:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.599+0000 7efc3c4c9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56275d39a0c0 con 0x56275d439ba0 2024-09-10T08:43:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.600+0000 7efc33400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d4410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.600+0000 7efc33400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d4410f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54922/0 (socket says 172.21.15.89:54922) 2024-09-10T08:43:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.600+0000 7efc33400640 1 -- 172.21.15.89:0/1829609544 learned_addr learned my addr 172.21.15.89:0/1829609544 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.601+0000 7efc33400640 1 -- 172.21.15.89:0/1829609544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56275d439ba0 msgr2=0x56275d43e660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:43:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.601+0000 7efc33400640 1 --2- 172.21.15.89:0/1829609544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56275d439ba0 0x56275d43e660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.601+0000 7efc33400640 1 -- 172.21.15.89:0/1829609544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439260 msgr2=0x56275d439660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:43:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.601+0000 7efc33400640 1 --2- 172.21.15.89:0/1829609544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439260 0x56275d439660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.601+0000 7efc33400640 1 -- 172.21.15.89:0/1829609544 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56275d0b1210 con 0x56275d43ec30 2024-09-10T08:43:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.601+0000 7efc33400640 1 --2- 172.21.15.89:0/1829609544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d4410f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efc200978f0 tx=0x7efc200036b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fad24d6020321d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.606+0000 7efc1f400640 1 -- 172.21.15.89:0/1829609544 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc200a3b90 con 0x56275d43ec30 2024-09-10T08:43:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.606+0000 7efc1f400640 1 -- 172.21.15.89:0/1829609544 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc200cc250 con 0x56275d43ec30 2024-09-10T08:43:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.606+0000 7efc1f400640 1 -- 172.21.15.89:0/1829609544 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc200a3960 con 0x56275d43ec30 2024-09-10T08:43:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.607+0000 7efc3c4c9740 1 -- 172.21.15.89:0/1829609544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 msgr2=0x56275d4410f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.607+0000 7efc3c4c9740 1 --2- 172.21.15.89:0/1829609544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d4410f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efc200978f0 tx=0x7efc200036b0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.619+0000 7efc3c4c9740 1 -- 172.21.15.89:0/1829609544 shutdown_connections 2024-09-10T08:43:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.619+0000 7efc3c4c9740 1 --2- 172.21.15.89:0/1829609544 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d4410f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.619+0000 7efc3c4c9740 1 --2- 172.21.15.89:0/1829609544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56275d439ba0 0x56275d43e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.619+0000 7efc3c4c9740 1 --2- 172.21.15.89:0/1829609544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439260 0x56275d439660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.619+0000 7efc3c4c9740 1 -- 172.21.15.89:0/1829609544 >> 172.21.15.89:0/1829609544 conn(0x56275d4348d0 msgr2=0x56275d434cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.630+0000 7efc3c4c9740 1 -- 172.21.15.89:0/1829609544 shutdown_connections 2024-09-10T08:43:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.633+0000 7efc3c4c9740 1 -- 172.21.15.89:0/1829609544 wait complete. 2024-09-10T08:43:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.634+0000 7efc3c4c9740 1 -- start start 2024-09-10T08:43:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.634+0000 7efc3c4c9740 1 Processor -- start 2024-09-10T08:43:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.641+0000 7efc3c4c9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56275d439260 0x56275d443620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.641+0000 7efc3c4c9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439ba0 0x56275d443b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.641+0000 7efc3c4c9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d44ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.641+0000 7efc3c4c9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56275d441c00 con 0x56275d43ec30 2024-09-10T08:43:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.641+0000 7efc3c4c9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56275d327090 con 0x56275d439ba0 2024-09-10T08:43:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.641+0000 7efc3c4c9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56275d441d80 con 0x56275d439260 2024-09-10T08:43:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.641+0000 7efc3c4c9740 10 client.0 _ll_register_callbacks cb 0x56275d37bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:43:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.642+0000 7efc32000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439ba0 0x56275d443b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.642+0000 7efc32000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439ba0 0x56275d443b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:38742/0 (socket says 172.21.15.89:38742) 2024-09-10T08:43:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.642+0000 7efc32000640 1 -- 172.21.15.89:0/1446479018 learned_addr learned my addr 172.21.15.89:0/1446479018 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.642+0000 7efc32000640 1 -- 172.21.15.89:0/1446479018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56275d439260 msgr2=0x56275d443620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.642+0000 7efc32000640 1 --2- 172.21.15.89:0/1446479018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56275d439260 0x56275d443620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.642+0000 7efc32000640 1 -- 172.21.15.89:0/1446479018 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 msgr2=0x56275d44ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.642+0000 7efc33400640 1 --2- 172.21.15.89:0/1446479018 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d44ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.642+0000 7efc32000640 1 --2- 172.21.15.89:0/1446479018 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d44ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.642+0000 7efc32000640 1 -- 172.21.15.89:0/1446479018 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56275d17d620 con 0x56275d439ba0 2024-09-10T08:43:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.643+0000 7efc33400640 1 --2- 172.21.15.89:0/1446479018 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d44ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:43:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.647+0000 7efc32000640 1 --2- 172.21.15.89:0/1446479018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439ba0 0x56275d443b60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0070b0 tx=0x7efc2c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37abb92ff767d447 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.650+0000 7efc13e00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:43:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.650+0000 7efc13e00640 1 -- 172.21.15.89:0/1446479018 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c023020 con 0x56275d439ba0 2024-09-10T08:43:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.650+0000 7efc13e00640 1 -- 172.21.15.89:0/1446479018 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7efc2c013d50 con 0x56275d439ba0 2024-09-10T08:43:14.665 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.655+0000 7efc3c4c9740 1 -- 172.21.15.89:0/1446479018 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc200cc250 con 0x56275d439ba0 2024-09-10T08:43:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.657+0000 7efc13e00640 1 -- 172.21.15.89:0/1446479018 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efc2c01b040 con 0x56275d439ba0 2024-09-10T08:43:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.657+0000 7efc3c4c9740 10 client.26015 fetch_fsmap learned FSMap version 28 2024-09-10T08:43:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.657+0000 7efc3c4c9740 1 -- 172.21.15.89:0/1446479018 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56275d44da40 con 0x56275d439ba0 2024-09-10T08:43:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.658+0000 7efc13e00640 1 -- 172.21.15.89:0/1446479018 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc200cc250 con 0x56275d439ba0 2024-09-10T08:43:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.658+0000 7efc13e00640 1 -- 172.21.15.89:0/1446479018 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efc2c030020 con 0x56275d439ba0 2024-09-10T08:43:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.660+0000 7efc3c4c9740 10 client.26015 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:43:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.660+0000 7efc3c4c9740 10 client.26015 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:43:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.660+0000 7efc3c4c9740 20 client.26015 populate_metadata read hostname 'smithi089' 2024-09-10T08:43:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.660+0000 7efc3c4c9740 1 --2- 172.21.15.89:0/1446479018 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56275d44f130 0x56275d451590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.660+0000 7efc3c4c9740 4 client.26015 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:43:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.660+0000 7efc3c4c9740 1 -- 172.21.15.89:0/1446479018 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56275d3ab830 con 0x56275d44f130 2024-09-10T08:43:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.664+0000 7efc32a00640 1 --2- 172.21.15.89:0/1446479018 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56275d44f130 0x56275d451590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.676+0000 7efc32a00640 1 --2- 172.21.15.89:0/1446479018 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56275d44f130 0x56275d451590 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6c58ddd4cb6d7bb server_cookie=2d45b3b9c11e91db in_seq=0 out_seq=0 2024-09-10T08:43:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.681+0000 7efc13e00640 10 client.26015 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.685+0000 7efc13e00640 1 -- 172.21.15.89:0/1446479018 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56275d3ab830 con 0x56275d44f130 2024-09-10T08:43:14.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.685+0000 7efc13e00640 10 client.26015 handle_command_reply: tid=0 2024-09-10T08:43:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.687+0000 7efc3c4c9740 1 client.26015 shutdown 2024-09-10T08:43:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.702+0000 7efc3c4c9740 1 -- 172.21.15.89:0/1446479018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439ba0 msgr2=0x56275d443b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.702+0000 7efc3c4c9740 1 --2- 172.21.15.89:0/1446479018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439ba0 0x56275d443b60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0070b0 tx=0x7efc2c004ad0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.702+0000 7efc3c4c9740 1 -- 172.21.15.89:0/1446479018 shutdown_connections 2024-09-10T08:43:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.702+0000 7efc3c4c9740 1 --2- 172.21.15.89:0/1446479018 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56275d44f130 0x56275d451590 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.702+0000 7efc3c4c9740 1 --2- 172.21.15.89:0/1446479018 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d44ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.702+0000 7efc3c4c9740 1 --2- 172.21.15.89:0/1446479018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439ba0 0x56275d443b60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.702+0000 7efc3c4c9740 1 --2- 172.21.15.89:0/1446479018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56275d439260 0x56275d443620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.702+0000 7efc13e00640 0 client.26015 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.702+0000 7efc3c4c9740 1 -- 172.21.15.89:0/1446479018 >> 172.21.15.89:0/1446479018 conn(0x56275d4348d0 msgr2=0x56275d4360e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.725+0000 7efc3c4c9740 1 -- 172.21.15.89:0/1446479018 shutdown_connections 2024-09-10T08:43:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.736+0000 7efc3c4c9740 1 -- 172.21.15.89:0/1446479018 wait complete. 2024-09-10T08:43:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:14.739+0000 7efc3c4c9740 20 client.26015 trim_cache size 0 max 16384 2024-09-10T08:43:14.991 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:14 smithi089 ceph-mon[30757]: pgmap v492: 129 pgs: 129 active+clean; 210 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 629 op/s 2024-09-10T08:43:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.038+0000 7efc3c4c9740 1 Processor -- start 2024-09-10T08:43:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.041+0000 7efc3c4c9740 1 -- start start 2024-09-10T08:43:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.041+0000 7efc3c4c9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439260 0x56275d443620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.041+0000 7efc3c4c9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56275d439ba0 0x56275d443b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.041+0000 7efc3c4c9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d44aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.041+0000 7efc3c4c9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efc341c3fc0 con 0x56275d43ec30 2024-09-10T08:43:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.041+0000 7efc3c4c9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efc341c4690 con 0x56275d439260 2024-09-10T08:43:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.041+0000 7efc3c4c9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efc341c1f60 con 0x56275d439ba0 2024-09-10T08:43:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.042+0000 7efc32a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439260 0x56275d443620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.042+0000 7efc32a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439260 0x56275d443620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:38756/0 (socket says 172.21.15.89:38756) 2024-09-10T08:43:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.042+0000 7efc32a00640 1 -- 172.21.15.89:0/1382544323 learned_addr learned my addr 172.21.15.89:0/1382544323 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.042+0000 7efc33400640 1 --2- 172.21.15.89:0/1382544323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d44aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.042+0000 7efc33400640 1 -- 172.21.15.89:0/1382544323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56275d439ba0 msgr2=0x56275d443b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:43:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.042+0000 7efc33400640 1 --2- 172.21.15.89:0/1382544323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56275d439ba0 0x56275d443b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.042+0000 7efc33400640 1 -- 172.21.15.89:0/1382544323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439260 msgr2=0x56275d443620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.042+0000 7efc33400640 1 --2- 172.21.15.89:0/1382544323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439260 0x56275d443620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.042+0000 7efc33400640 1 -- 172.21.15.89:0/1382544323 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56275d4346f0 con 0x56275d43ec30 2024-09-10T08:43:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.043+0000 7efc33400640 1 --2- 172.21.15.89:0/1382544323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d44aaf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efc20097380 tx=0x7efc2009b0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5182789157333b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.051+0000 7efc1ea00640 1 -- 172.21.15.89:0/1382544323 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc200992e0 con 0x56275d43ec30 2024-09-10T08:43:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.051+0000 7efc1ea00640 1 -- 172.21.15.89:0/1382544323 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc20097590 con 0x56275d43ec30 2024-09-10T08:43:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.051+0000 7efc1ea00640 1 -- 172.21.15.89:0/1382544323 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2009a370 con 0x56275d43ec30 2024-09-10T08:43:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.056+0000 7efc3c4c9740 1 -- 172.21.15.89:0/1382544323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 msgr2=0x56275d44aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.056+0000 7efc3c4c9740 1 --2- 172.21.15.89:0/1382544323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d44aaf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efc20097380 tx=0x7efc2009b0a0 comp rx=0 tx=0).stop 2024-09-10T08:43:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.065+0000 7efc3c4c9740 1 -- 172.21.15.89:0/1382544323 shutdown_connections 2024-09-10T08:43:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.065+0000 7efc3c4c9740 1 --2- 172.21.15.89:0/1382544323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d44aaf0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.065+0000 7efc3c4c9740 1 --2- 172.21.15.89:0/1382544323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56275d439ba0 0x56275d443b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.065+0000 7efc3c4c9740 1 --2- 172.21.15.89:0/1382544323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439260 0x56275d443620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.065+0000 7efc3c4c9740 1 -- 172.21.15.89:0/1382544323 >> 172.21.15.89:0/1382544323 conn(0x56275d4348d0 msgr2=0x56275d43afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.066+0000 7efc3c4c9740 1 -- 172.21.15.89:0/1382544323 shutdown_connections 2024-09-10T08:43:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.066+0000 7efc3c4c9740 1 -- 172.21.15.89:0/1382544323 wait complete. 2024-09-10T08:43:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.067+0000 7efc3c4c9740 1 -- start start 2024-09-10T08:43:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.067+0000 7efc3c4c9740 1 Processor -- start 2024-09-10T08:43:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.068+0000 7efc3c4c9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439260 0x56275d447bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.068+0000 7efc3c4c9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56275d439ba0 0x56275d448100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.068+0000 7efc3c4c9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d4490a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.069+0000 7efc3c4c9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efc341c0420 con 0x56275d43ec30 2024-09-10T08:43:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.069+0000 7efc3c4c9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efc341c1890 con 0x56275d439260 2024-09-10T08:43:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.069+0000 7efc3c4c9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efc341c0af0 con 0x56275d439ba0 2024-09-10T08:43:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.069+0000 7efc3c4c9740 10 client.0 _ll_register_callbacks cb 0x7efc2c009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:43:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.069+0000 7efc33400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d4490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.069+0000 7efc33400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d4490a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54944/0 (socket says 172.21.15.89:54944) 2024-09-10T08:43:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.069+0000 7efc33400640 1 -- 172.21.15.89:0/3975641799 learned_addr learned my addr 172.21.15.89:0/3975641799 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.069+0000 7efc32000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56275d439ba0 0x56275d448100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.069+0000 7efc32a00640 1 --2- 172.21.15.89:0/3975641799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439260 0x56275d447bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.070+0000 7efc33400640 1 -- 172.21.15.89:0/3975641799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56275d439ba0 msgr2=0x56275d448100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.070+0000 7efc33400640 1 --2- 172.21.15.89:0/3975641799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56275d439ba0 0x56275d448100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.070+0000 7efc33400640 1 -- 172.21.15.89:0/3975641799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439260 msgr2=0x56275d447bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.070+0000 7efc33400640 1 --2- 172.21.15.89:0/3975641799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439260 0x56275d447bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.070+0000 7efc33400640 1 -- 172.21.15.89:0/3975641799 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56275d4496b0 con 0x56275d43ec30 2024-09-10T08:43:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.070+0000 7efc33400640 1 --2- 172.21.15.89:0/3975641799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d4490a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7efc200a3660 tx=0x7efc200977b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653ec00e606c5e24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.070+0000 7efc13e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:43:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.071+0000 7efc13e00640 1 -- 172.21.15.89:0/3975641799 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc200a8070 con 0x56275d43ec30 2024-09-10T08:43:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.071+0000 7efc13e00640 1 -- 172.21.15.89:0/3975641799 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7efc2009a370 con 0x56275d43ec30 2024-09-10T08:43:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.071+0000 7efc3c4c9740 1 -- 172.21.15.89:0/3975641799 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc20097590 con 0x56275d43ec30 2024-09-10T08:43:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.071+0000 7efc13e00640 1 -- 172.21.15.89:0/3975641799 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efc2009b380 con 0x56275d43ec30 2024-09-10T08:43:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.071+0000 7efc3c4c9740 10 client.15963 fetch_fsmap learned FSMap version 28 2024-09-10T08:43:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.071+0000 7efc3c4c9740 1 -- 172.21.15.89:0/3975641799 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56275d44a9c0 con 0x56275d43ec30 2024-09-10T08:43:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.072+0000 7efc13e00640 1 -- 172.21.15.89:0/3975641799 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc20097590 con 0x56275d43ec30 2024-09-10T08:43:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.072+0000 7efc13e00640 1 -- 172.21.15.89:0/3975641799 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efc2009e030 con 0x56275d43ec30 2024-09-10T08:43:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.072+0000 7efc3c4c9740 10 client.15963 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:43:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.072+0000 7efc3c4c9740 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:43:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.072+0000 7efc3c4c9740 20 client.15963 populate_metadata read hostname 'smithi089' 2024-09-10T08:43:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.072+0000 7efc3c4c9740 1 --2- 172.21.15.89:0/3975641799 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56275d4b9e70 0x56275d4bc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.072+0000 7efc3c4c9740 4 client.15963 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:43:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.072+0000 7efc3c4c9740 1 -- 172.21.15.89:0/3975641799 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56275d3a5b80 con 0x56275d4b9e70 2024-09-10T08:43:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.073+0000 7efc32a00640 1 --2- 172.21.15.89:0/3975641799 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56275d4b9e70 0x56275d4bc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.074+0000 7efc32a00640 1 --2- 172.21.15.89:0/3975641799 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56275d4b9e70 0x56275d4bc260 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a412437645cf9798 server_cookie=415d64a9af831f24 in_seq=0 out_seq=0 2024-09-10T08:43:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:15.074+0000 7efc13e00640 10 client.15963 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:15.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:14 smithi149 ceph-mon[36167]: pgmap v492: 129 pgs: 129 active+clean; 210 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 629 op/s 2024-09-10T08:43:15.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:14 smithi111 ceph-mon[35220]: pgmap v492: 129 pgs: 129 active+clean; 210 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 629 op/s 2024-09-10T08:43:17.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:16 smithi149 ceph-mon[36167]: pgmap v493: 129 pgs: 129 active+clean; 214 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 368 op/s 2024-09-10T08:43:17.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:16 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:17.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:16 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:43:17.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:16 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:43:17.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:16 smithi111 ceph-mon[35220]: pgmap v493: 129 pgs: 129 active+clean; 214 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 368 op/s 2024-09-10T08:43:17.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:16 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:17.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:16 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:43:17.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:16 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:43:17.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:16 smithi089 ceph-mon[30757]: pgmap v493: 129 pgs: 129 active+clean; 214 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 368 op/s 2024-09-10T08:43:17.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:16 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:17.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:16 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:43:17.312 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:16 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:43:18.033 INFO:tasks.workunit.client.0.smithi089.stdout: 74 553125 613 398966 433 290782 1529 2024-09-10T08:43:18.904 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:18 smithi089 ceph-mon[30757]: pgmap v494: 129 pgs: 129 active+clean; 215 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.7 MiB/s wr, 460 op/s 2024-09-10T08:43:19.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:18 smithi149 ceph-mon[36167]: pgmap v494: 129 pgs: 129 active+clean; 215 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.7 MiB/s wr, 460 op/s 2024-09-10T08:43:19.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:18 smithi111 ceph-mon[35220]: pgmap v494: 129 pgs: 129 active+clean; 215 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.7 MiB/s wr, 460 op/s 2024-09-10T08:43:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.755+0000 7efc13e00640 1 -- 172.21.15.89:0/3975641799 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56275d3a5b80 con 0x56275d4b9e70 2024-09-10T08:43:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.755+0000 7efc13e00640 10 client.15963 handle_command_reply: tid=0 2024-09-10T08:43:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.755+0000 7efc3c4c9740 1 client.15963 shutdown 2024-09-10T08:43:20.767 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.767+0000 7efc3c4c9740 1 -- 172.21.15.89:0/3975641799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 msgr2=0x56275d4490a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:20.768 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.767+0000 7efc3c4c9740 1 --2- 172.21.15.89:0/3975641799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d4490a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7efc200a3660 tx=0x7efc200977b0 comp rx=0 tx=0).stop 2024-09-10T08:43:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.768+0000 7efc3c4c9740 1 -- 172.21.15.89:0/3975641799 shutdown_connections 2024-09-10T08:43:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.768+0000 7efc3c4c9740 1 --2- 172.21.15.89:0/3975641799 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56275d4b9e70 0x56275d4bc260 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.768+0000 7efc3c4c9740 1 --2- 172.21.15.89:0/3975641799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56275d43ec30 0x56275d4490a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.768+0000 7efc3c4c9740 1 --2- 172.21.15.89:0/3975641799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56275d439ba0 0x56275d448100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.768+0000 7efc3c4c9740 1 --2- 172.21.15.89:0/3975641799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56275d439260 0x56275d447bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.768+0000 7efc3c4c9740 1 -- 172.21.15.89:0/3975641799 >> 172.21.15.89:0/3975641799 conn(0x56275d4348d0 msgr2=0x56275d4433a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.768+0000 7efc13e00640 0 client.15963 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:20.773 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.773+0000 7efc3c4c9740 1 -- 172.21.15.89:0/3975641799 shutdown_connections 2024-09-10T08:43:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.795+0000 7efc3c4c9740 1 -- 172.21.15.89:0/3975641799 wait complete. 2024-09-10T08:43:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.798+0000 7efc3c4c9740 20 client.15963 trim_cache size 0 max 16384 2024-09-10T08:43:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:43:20.800 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:43:20.800 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:43:20.800 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:43:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.802+0000 7efc33e00640 1 -- 172.21.15.89:0/3460281693 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efc180a2d90 msgr2=0x7efc180a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.802+0000 7efc33e00640 1 --2- 172.21.15.89:0/3460281693 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efc180a2d90 0x7efc180a5250 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efc341a8030 tx=0x7efc2c00b040 comp rx=0 tx=0).stop 2024-09-10T08:43:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.802+0000 7efc33e00640 1 -- 172.21.15.89:0/3460281693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc3410dbb0 msgr2=0x7efc341ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.802+0000 7efc33e00640 1 --2- 172.21.15.89:0/3460281693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc3410dbb0 0x7efc341ae0d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efc2000d6b0 tx=0x7efc2000db70 comp rx=0 tx=0).stop 2024-09-10T08:43:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.803+0000 7efc33e00640 1 -- 172.21.15.89:0/3460281693 shutdown_connections 2024-09-10T08:43:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.803+0000 7efc33e00640 1 --2- 172.21.15.89:0/3460281693 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efc180a2d90 0x7efc180a5250 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.803+0000 7efc33e00640 1 --2- 172.21.15.89:0/3460281693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc3410dbb0 0x7efc341ae0d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.803+0000 7efc33e00640 1 --2- 172.21.15.89:0/3460281693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc34073680 0x7efc341a7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.803+0000 7efc33e00640 1 --2- 172.21.15.89:0/3460281693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc34072cb0 0x7efc341a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.803+0000 7efc33e00640 1 -- 172.21.15.89:0/3460281693 >> 172.21.15.89:0/3460281693 conn(0x7efc3406eb50 msgr2=0x7efc3406f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.803+0000 7efc33e00640 1 -- 172.21.15.89:0/3460281693 shutdown_connections 2024-09-10T08:43:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:20.803+0000 7efc33e00640 1 -- 172.21.15.89:0/3460281693 wait complete. 2024-09-10T08:43:21.008 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:20 smithi089 ceph-mon[30757]: pgmap v495: 129 pgs: 129 active+clean; 217 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 568 op/s 2024-09-10T08:43:21.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:20 smithi149 ceph-mon[36167]: pgmap v495: 129 pgs: 129 active+clean; 217 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 568 op/s 2024-09-10T08:43:21.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:20 smithi111 ceph-mon[35220]: pgmap v495: 129 pgs: 129 active+clean; 217 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 568 op/s 2024-09-10T08:43:21.659 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:23.070 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:22 smithi089 ceph-mon[30757]: pgmap v496: 129 pgs: 129 active+clean; 221 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 1.20k op/s 2024-09-10T08:43:23.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:22 smithi149 ceph-mon[36167]: pgmap v496: 129 pgs: 129 active+clean; 221 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 1.20k op/s 2024-09-10T08:43:23.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:22 smithi111 ceph-mon[35220]: pgmap v496: 129 pgs: 129 active+clean; 221 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 1.20k op/s 2024-09-10T08:43:24.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.023+0000 7fdd5e400640 1 -- 172.21.15.89:0/1988493623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd58110190 msgr2=0x7fdd5806e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:24.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.023+0000 7fdd5e400640 1 --2- 172.21.15.89:0/1988493623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd58110190 0x7fdd5806e4f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009860 tx=0x7fdd4802a380 comp rx=0 tx=0).stop 2024-09-10T08:43:24.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.023+0000 7fdd5e400640 1 -- 172.21.15.89:0/1988493623 shutdown_connections 2024-09-10T08:43:24.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.023+0000 7fdd5e400640 1 --2- 172.21.15.89:0/1988493623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd5811b140 0x7fdd5811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.023+0000 7fdd5e400640 1 --2- 172.21.15.89:0/1988493623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd5806ea30 0x7fdd5806ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.023+0000 7fdd5e400640 1 --2- 172.21.15.89:0/1988493623 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd58110190 0x7fdd5806e4f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.023+0000 7fdd5e400640 1 -- 172.21.15.89:0/1988493623 >> 172.21.15.89:0/1988493623 conn(0x7fdd58073180 msgr2=0x7fdd58073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:24.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.026+0000 7fdd5e400640 1 -- 172.21.15.89:0/1988493623 shutdown_connections 2024-09-10T08:43:24.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.027+0000 7fdd5e400640 1 -- 172.21.15.89:0/1988493623 wait complete. 2024-09-10T08:43:24.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.028+0000 7fdd5e400640 1 Processor -- start 2024-09-10T08:43:24.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.028+0000 7fdd5e400640 1 -- start start 2024-09-10T08:43:24.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.029+0000 7fdd5e400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd5806ea30 0x7fdd581a72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.029+0000 7fdd5e400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd5811b140 0x7fdd581a7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.029+0000 7fdd5d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd5806ea30 0x7fdd581a72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:24.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.029+0000 7fdd5d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd5806ea30 0x7fdd581a72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:36492/0 (socket says 172.21.15.89:36492) 2024-09-10T08:43:24.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.029+0000 7fdd57e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd5811b140 0x7fdd581a7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:24.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.029+0000 7fdd57e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd5811b140 0x7fdd581a7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57096/0 (socket says 172.21.15.89:57096) 2024-09-10T08:43:24.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.030+0000 7fdd5d000640 1 -- 172.21.15.89:0/2911137647 learned_addr learned my addr 172.21.15.89:0/2911137647 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:24.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.030+0000 7fdd5e400640 1 --2- 172.21.15.89:0/2911137647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd581abba0 0x7fdd581ac050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.030+0000 7fdd5e400640 1 -- 172.21.15.89:0/2911137647 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdd58120b90 con 0x7fdd581abba0 2024-09-10T08:43:24.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.030+0000 7fdd5e400640 1 -- 172.21.15.89:0/2911137647 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdd58120a10 con 0x7fdd5811b140 2024-09-10T08:43:24.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.031+0000 7fdd5e400640 1 -- 172.21.15.89:0/2911137647 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdd58120d10 con 0x7fdd5806ea30 2024-09-10T08:43:24.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.033+0000 7fdd5d000640 1 -- 172.21.15.89:0/2911137647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd5811b140 msgr2=0x7fdd581a7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:24.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.034+0000 7fdd5d000640 1 --2- 172.21.15.89:0/2911137647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd5811b140 0x7fdd581a7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.034+0000 7fdd5d000640 1 -- 172.21.15.89:0/2911137647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd581abba0 msgr2=0x7fdd581ac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:24.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.034+0000 7fdd5d000640 1 --2- 172.21.15.89:0/2911137647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd581abba0 0x7fdd581ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.035+0000 7fdd5d000640 1 -- 172.21.15.89:0/2911137647 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd581ac660 con 0x7fdd5806ea30 2024-09-10T08:43:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.036+0000 7fdd5d000640 1 --2- 172.21.15.89:0/2911137647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd5806ea30 0x7fdd581a72d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdd48035040 tx=0x7fdd48033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.036+0000 7fdd55600640 1 -- 172.21.15.89:0/2911137647 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48037040 con 0x7fdd5806ea30 2024-09-10T08:43:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.040+0000 7fdd5e400640 1 -- 172.21.15.89:0/2911137647 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd581ac8f0 con 0x7fdd5806ea30 2024-09-10T08:43:24.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.040+0000 7fdd5e400640 1 -- 172.21.15.89:0/2911137647 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd581b40d0 con 0x7fdd5806ea30 2024-09-10T08:43:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.049+0000 7fdd55600640 1 -- 172.21.15.89:0/2911137647 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd48004240 con 0x7fdd5806ea30 2024-09-10T08:43:24.055 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.049+0000 7fdd55600640 1 -- 172.21.15.89:0/2911137647 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48037040 con 0x7fdd5806ea30 2024-09-10T08:43:24.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.057+0000 7fdd625fc740 1 Processor -- start 2024-09-10T08:43:24.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.059+0000 7fdd625fc740 1 -- start start 2024-09-10T08:43:24.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.059+0000 7fdd625fc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa69e40 0x55c67fa6a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.068 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.059+0000 7fdd625fc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67fa6a760 0x55c67fa74ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.059+0000 7fdd625fc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa75020 0x55c67fa77410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.069 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.059+0000 7fdd625fc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c67f9d03e0 con 0x55c67fa6a760 2024-09-10T08:43:24.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.059+0000 7fdd625fc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c67f977d80 con 0x55c67fa69e40 2024-09-10T08:43:24.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.059+0000 7fdd625fc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c67f9cc3a0 con 0x55c67fa75020 2024-09-10T08:43:24.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.061+0000 7fdd5da00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa69e40 0x55c67fa6a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:24.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.061+0000 7fdd57e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa75020 0x55c67fa77410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:24.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.061+0000 7fdd57e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa75020 0x55c67fa77410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:36502/0 (socket says 172.21.15.89:36502) 2024-09-10T08:43:24.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.061+0000 7fdd57e00640 1 -- 172.21.15.89:0/161255536 learned_addr learned my addr 172.21.15.89:0/161255536 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:24.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.061+0000 7fdd5da00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa69e40 0x55c67fa6a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57104/0 (socket says 172.21.15.89:57104) 2024-09-10T08:43:24.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.062+0000 7fdd57e00640 1 -- 172.21.15.89:0/161255536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa69e40 msgr2=0x55c67fa6a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:24.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.062+0000 7fdd57e00640 1 --2- 172.21.15.89:0/161255536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa69e40 0x55c67fa6a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.062+0000 7fdd57e00640 1 -- 172.21.15.89:0/161255536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67fa6a760 msgr2=0x55c67fa74ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:43:24.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.062+0000 7fdd57e00640 1 --2- 172.21.15.89:0/161255536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67fa6a760 0x55c67fa74ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.070 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.062+0000 7fdd57e00640 1 -- 172.21.15.89:0/161255536 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c67f996ae0 con 0x55c67fa75020 2024-09-10T08:43:24.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.066+0000 7fdd57e00640 1 --2- 172.21.15.89:0/161255536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa75020 0x55c67fa77410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdd4400bcb0 tx=0x7fdd440107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd949be758c05daa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:24.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.066+0000 7fdd4e000640 1 -- 172.21.15.89:0/161255536 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd44004030 con 0x55c67fa75020 2024-09-10T08:43:24.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.067+0000 7fdd4e000640 1 -- 172.21.15.89:0/161255536 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd440041d0 con 0x55c67fa75020 2024-09-10T08:43:24.072 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.070+0000 7fdd4e000640 1 -- 172.21.15.89:0/161255536 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd44004030 con 0x55c67fa75020 2024-09-10T08:43:24.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.072+0000 7fdd625fc740 1 -- 172.21.15.89:0/161255536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa75020 msgr2=0x55c67fa77410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:24.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.072+0000 7fdd625fc740 1 --2- 172.21.15.89:0/161255536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa75020 0x55c67fa77410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdd4400bcb0 tx=0x7fdd440107a0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.073+0000 7fdd625fc740 1 -- 172.21.15.89:0/161255536 shutdown_connections 2024-09-10T08:43:24.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.073+0000 7fdd625fc740 1 --2- 172.21.15.89:0/161255536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa75020 0x55c67fa77410 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.073+0000 7fdd625fc740 1 --2- 172.21.15.89:0/161255536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67fa6a760 0x55c67fa74ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.073+0000 7fdd625fc740 1 --2- 172.21.15.89:0/161255536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa69e40 0x55c67fa6a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.077+0000 7fdd625fc740 1 -- 172.21.15.89:0/161255536 >> 172.21.15.89:0/161255536 conn(0x55c67fa663d0 msgr2=0x55c67fa667b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:24.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.083+0000 7fdd55600640 1 -- 172.21.15.89:0/2911137647 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdd48005330 con 0x7fdd5806ea30 2024-09-10T08:43:24.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.085+0000 7fdd55600640 1 --2- 172.21.15.89:0/2911137647 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdd3c0a2d10 0x7fdd3c0a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.085+0000 7fdd55600640 1 -- 172.21.15.89:0/2911137647 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fdd480ee950 con 0x7fdd5806ea30 2024-09-10T08:43:24.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.087+0000 7fdd625fc740 1 -- 172.21.15.89:0/161255536 shutdown_connections 2024-09-10T08:43:24.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.087+0000 7fdd625fc740 1 -- 172.21.15.89:0/161255536 wait complete. 2024-09-10T08:43:24.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.087+0000 7fdd625fc740 1 -- start start 2024-09-10T08:43:24.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.088+0000 7fdd625fc740 1 Processor -- start 2024-09-10T08:43:24.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.089+0000 7fdd625fc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa69e40 0x55c67fa72080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.089+0000 7fdd625fc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67fa75020 0x55c67fa725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.089+0000 7fdd625fc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa72b00 0x55c67fa810f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.090+0000 7fdd5da00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa72b00 0x55c67fa810f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:24.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.090+0000 7fdd5da00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa72b00 0x55c67fa810f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57114/0 (socket says 172.21.15.89:57114) 2024-09-10T08:43:24.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.090+0000 7fdd5da00640 1 -- 172.21.15.89:0/2371823162 learned_addr learned my addr 172.21.15.89:0/2371823162 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:24.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.091+0000 7fdd5d000640 1 --2- 172.21.15.89:0/2371823162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa69e40 0x55c67fa72080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:24.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.092+0000 7fdd57e00640 1 --2- 172.21.15.89:0/2911137647 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdd3c0a2d10 0x7fdd3c0a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:24.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.092+0000 7fdd57e00640 1 --2- 172.21.15.89:0/2371823162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67fa75020 0x55c67fa725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:24.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.096+0000 7fdd625fc740 1 -- 172.21.15.89:0/2371823162 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c67fa77bf0 con 0x55c67fa75020 2024-09-10T08:43:24.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.096+0000 7fdd625fc740 1 -- 172.21.15.89:0/2371823162 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c67f98b0f0 con 0x55c67fa72b00 2024-09-10T08:43:24.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.096+0000 7fdd625fc740 1 -- 172.21.15.89:0/2371823162 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c67fa77d70 con 0x55c67fa69e40 2024-09-10T08:43:24.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.097+0000 7fdd625fc740 10 client.0 _ll_register_callbacks cb 0x55c67f972bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:43:24.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.098+0000 7fdd5d000640 1 -- 172.21.15.89:0/2371823162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa72b00 msgr2=0x55c67fa810f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:24.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.098+0000 7fdd5d000640 1 --2- 172.21.15.89:0/2371823162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa72b00 0x55c67fa810f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.098+0000 7fdd5d000640 1 -- 172.21.15.89:0/2371823162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67fa75020 msgr2=0x55c67fa725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:24.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.098+0000 7fdd5d000640 1 --2- 172.21.15.89:0/2371823162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67fa75020 0x55c67fa725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.098+0000 7fdd5d000640 1 -- 172.21.15.89:0/2371823162 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c67f7413c0 con 0x55c67fa69e40 2024-09-10T08:43:24.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.099+0000 7fdd42a00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:43:24.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.099+0000 7fdd5d000640 1 --2- 172.21.15.89:0/2371823162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa69e40 0x55c67fa72080 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55c67fa67230 tx=0x7fdd480b7240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f7fd38de4362931 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:24.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.099+0000 7fdd42a00640 1 -- 172.21.15.89:0/2371823162 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd480bae70 con 0x55c67fa69e40 2024-09-10T08:43:24.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.100+0000 7fdd42a00640 1 -- 172.21.15.89:0/2371823162 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fdd480b8f10 con 0x55c67fa69e40 2024-09-10T08:43:24.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.100+0000 7fdd625fc740 1 -- 172.21.15.89:0/2371823162 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd440041d0 con 0x55c67fa69e40 2024-09-10T08:43:24.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.100+0000 7fdd57e00640 1 --2- 172.21.15.89:0/2371823162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67fa75020 0x55c67fa725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:43:24.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.104+0000 7fdd42a00640 1 -- 172.21.15.89:0/2371823162 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd480bae70 con 0x55c67fa69e40 2024-09-10T08:43:24.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.104+0000 7fdd57e00640 1 --2- 172.21.15.89:0/2911137647 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdd3c0a2d10 0x7fdd3c0a51d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55c67fa71910 tx=0x7fdd44003eb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:24.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.105+0000 7fdd625fc740 10 client.25981 fetch_fsmap learned FSMap version 28 2024-09-10T08:43:24.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.105+0000 7fdd625fc740 1 -- 172.21.15.89:0/2371823162 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55c67fa77c50 con 0x55c67fa69e40 2024-09-10T08:43:24.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.106+0000 7fdd42a00640 1 -- 172.21.15.89:0/2371823162 <== mon.1 v2:172.21.15.149:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdd48099ce0 con 0x55c67fa69e40 2024-09-10T08:43:24.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.106+0000 7fdd625fc740 10 client.25981 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:43:24.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.106+0000 7fdd625fc740 10 client.25981 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:43:24.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.106+0000 7fdd625fc740 20 client.25981 populate_metadata read hostname 'smithi089' 2024-09-10T08:43:24.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.106+0000 7fdd625fc740 1 --2- 172.21.15.89:0/2371823162 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55c67fa853a0 0x55c67fa87840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.106+0000 7fdd625fc740 4 client.25981 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:43:24.124 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.106+0000 7fdd625fc740 1 -- 172.21.15.89:0/2371823162 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdd440041d0 con 0x55c67fa853a0 2024-09-10T08:43:24.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.107+0000 7fdd57e00640 1 --2- 172.21.15.89:0/2371823162 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55c67fa853a0 0x55c67fa87840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:24.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.113+0000 7fdd57e00640 1 --2- 172.21.15.89:0/2371823162 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55c67fa853a0 0x55c67fa87840 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1c1bdb5e61870ab server_cookie=35f7e0da71b78951 in_seq=0 out_seq=0 2024-09-10T08:43:24.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.115+0000 7fdd42a00640 10 client.25981 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:43:24.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.115+0000 7fdd42a00640 1 -- 172.21.15.89:0/2371823162 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fdd440041d0 con 0x55c67fa853a0 2024-09-10T08:43:24.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.115+0000 7fdd42a00640 10 client.25981 handle_command_reply: tid=0 2024-09-10T08:43:24.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.115+0000 7fdd625fc740 1 client.25981 shutdown 2024-09-10T08:43:24.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.116+0000 7fdd625fc740 1 -- 172.21.15.89:0/2371823162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa69e40 msgr2=0x55c67fa72080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:24.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.116+0000 7fdd625fc740 1 --2- 172.21.15.89:0/2371823162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa69e40 0x55c67fa72080 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55c67fa67230 tx=0x7fdd480b7240 comp rx=0 tx=0).stop 2024-09-10T08:43:24.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.118+0000 7fdd625fc740 1 -- 172.21.15.89:0/2371823162 shutdown_connections 2024-09-10T08:43:24.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.118+0000 7fdd625fc740 1 --2- 172.21.15.89:0/2371823162 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55c67fa853a0 0x55c67fa87840 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.118+0000 7fdd625fc740 1 --2- 172.21.15.89:0/2371823162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa72b00 0x55c67fa810f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.118+0000 7fdd625fc740 1 --2- 172.21.15.89:0/2371823162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67fa75020 0x55c67fa725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.118+0000 7fdd625fc740 1 --2- 172.21.15.89:0/2371823162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa69e40 0x55c67fa72080 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.118+0000 7fdd625fc740 1 -- 172.21.15.89:0/2371823162 >> 172.21.15.89:0/2371823162 conn(0x55c67fa663d0 msgr2=0x55c67fa6c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:24.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.119+0000 7fdd42a00640 0 client.25981 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:43:24.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.119+0000 7fdd625fc740 1 -- 172.21.15.89:0/2371823162 shutdown_connections 2024-09-10T08:43:24.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.120+0000 7fdd625fc740 1 -- 172.21.15.89:0/2371823162 wait complete. 2024-09-10T08:43:24.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.120+0000 7fdd625fc740 20 client.25981 trim_cache size 0 max 16384 2024-09-10T08:43:24.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.161+0000 7fdd625fc740 1 Processor -- start 2024-09-10T08:43:24.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.161+0000 7fdd625fc740 1 -- start start 2024-09-10T08:43:24.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.162+0000 7fdd625fc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa69e40 0x55c67fa80ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.162+0000 7fdd625fc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67fa72b00 0x55c67fa8d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.162+0000 7fdd625fc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa75020 0x55c67fa8d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.162+0000 7fdd625fc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdd38012450 con 0x55c67fa72b00 2024-09-10T08:43:24.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.162+0000 7fdd625fc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3800fd20 con 0x55c67fa75020 2024-09-10T08:43:24.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.162+0000 7fdd625fc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3800db70 con 0x55c67fa69e40 2024-09-10T08:43:24.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.162+0000 7fdd5d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa69e40 0x55c67fa80ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:24.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.162+0000 7fdd5d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa69e40 0x55c67fa80ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:36528/0 (socket says 172.21.15.89:36528) 2024-09-10T08:43:24.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.162+0000 7fdd5d000640 1 -- 172.21.15.89:0/3096341 learned_addr learned my addr 172.21.15.89:0/3096341 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:24.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.162+0000 7fdd5da00640 1 --2- 172.21.15.89:0/3096341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa75020 0x55c67fa8d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:24.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.163+0000 7fdd5d000640 1 -- 172.21.15.89:0/3096341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa75020 msgr2=0x55c67fa8d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:24.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.163+0000 7fdd5d000640 1 --2- 172.21.15.89:0/3096341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa75020 0x55c67fa8d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.163+0000 7fdd5d000640 1 -- 172.21.15.89:0/3096341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67fa72b00 msgr2=0x55c67fa8d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:24.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.163+0000 7fdd5d000640 1 --2- 172.21.15.89:0/3096341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67fa72b00 0x55c67fa8d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.163+0000 7fdd5d000640 1 -- 172.21.15.89:0/3096341 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c67fa705d0 con 0x55c67fa69e40 2024-09-10T08:43:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.163+0000 7fdd5d000640 1 --2- 172.21.15.89:0/3096341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa69e40 0x55c67fa80ee0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdd480bae00 tx=0x7fdd48045000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcce209f8e4b23b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.163+0000 7fdd4d600640 1 -- 172.21.15.89:0/3096341 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd480d8070 con 0x55c67fa69e40 2024-09-10T08:43:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.163+0000 7fdd4d600640 1 -- 172.21.15.89:0/3096341 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd4809d070 con 0x55c67fa69e40 2024-09-10T08:43:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.163+0000 7fdd4d600640 1 -- 172.21.15.89:0/3096341 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48030810 con 0x55c67fa69e40 2024-09-10T08:43:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.164+0000 7fdd625fc740 1 -- 172.21.15.89:0/3096341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa69e40 msgr2=0x55c67fa80ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.164+0000 7fdd625fc740 1 --2- 172.21.15.89:0/3096341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa69e40 0x55c67fa80ee0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdd480bae00 tx=0x7fdd48045000 comp rx=0 tx=0).stop 2024-09-10T08:43:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.168+0000 7fdd625fc740 1 -- 172.21.15.89:0/3096341 shutdown_connections 2024-09-10T08:43:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.168+0000 7fdd625fc740 1 --2- 172.21.15.89:0/3096341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa75020 0x55c67fa8d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.168+0000 7fdd625fc740 1 --2- 172.21.15.89:0/3096341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67fa72b00 0x55c67fa8d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.168+0000 7fdd625fc740 1 --2- 172.21.15.89:0/3096341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa69e40 0x55c67fa80ee0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.168+0000 7fdd625fc740 1 -- 172.21.15.89:0/3096341 >> 172.21.15.89:0/3096341 conn(0x55c67fa663d0 msgr2=0x55c67fa70950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.169+0000 7fdd625fc740 1 -- 172.21.15.89:0/3096341 shutdown_connections 2024-09-10T08:43:24.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.169+0000 7fdd625fc740 1 -- 172.21.15.89:0/3096341 wait complete. 2024-09-10T08:43:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.170+0000 7fdd625fc740 1 -- start start 2024-09-10T08:43:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.170+0000 7fdd625fc740 1 Processor -- start 2024-09-10T08:43:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.174+0000 7fdd625fc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa69e40 0x55c67fa7d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.174+0000 7fdd625fc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa72b00 0x55c67fa78790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.174+0000 7fdd625fc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67fa75020 0x55c67fa78cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.175+0000 7fdd625fc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3800e1e0 con 0x55c67fa75020 2024-09-10T08:43:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.175+0000 7fdd625fc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3800f650 con 0x55c67fa69e40 2024-09-10T08:43:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.175+0000 7fdd625fc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3800e8b0 con 0x55c67fa72b00 2024-09-10T08:43:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.175+0000 7fdd5d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa69e40 0x55c67fa7d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.175+0000 7fdd5d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa69e40 0x55c67fa7d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57144/0 (socket says 172.21.15.89:57144) 2024-09-10T08:43:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.175+0000 7fdd5d000640 1 -- 172.21.15.89:0/489025079 learned_addr learned my addr 172.21.15.89:0/489025079 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.175+0000 7fdd625fc740 10 client.0 _ll_register_callbacks cb 0x7fdd48009420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:43:24.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.175+0000 7fdd5da00640 1 --2- 172.21.15.89:0/489025079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67fa75020 0x55c67fa78cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:24.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.176+0000 7fdd57e00640 1 --2- 172.21.15.89:0/489025079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa72b00 0x55c67fa78790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:24.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.176+0000 7fdd5d000640 1 -- 172.21.15.89:0/489025079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa72b00 msgr2=0x55c67fa78790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:24.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.176+0000 7fdd5d000640 1 --2- 172.21.15.89:0/489025079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa72b00 0x55c67fa78790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.176+0000 7fdd5d000640 1 -- 172.21.15.89:0/489025079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67fa75020 msgr2=0x55c67fa78cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:24.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.177+0000 7fdd5d000640 1 --2- 172.21.15.89:0/489025079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67fa75020 0x55c67fa78cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.179+0000 7fdd5d000640 1 -- 172.21.15.89:0/489025079 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c67fa79210 con 0x55c67fa69e40 2024-09-10T08:43:24.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.189+0000 7fdd5d000640 1 --2- 172.21.15.89:0/489025079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa69e40 0x55c67fa7d460 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdd480094d0 tx=0x7fdd48005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=753fabb17947439 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:24.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.189+0000 7fdd42a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:43:24.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.190+0000 7fdd42a00640 1 -- 172.21.15.89:0/489025079 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd480d8070 con 0x55c67fa69e40 2024-09-10T08:43:24.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.190+0000 7fdd42a00640 1 -- 172.21.15.89:0/489025079 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fdd480ba4d0 con 0x55c67fa69e40 2024-09-10T08:43:24.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.190+0000 7fdd625fc740 1 -- 172.21.15.89:0/489025079 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd4809d070 con 0x55c67fa69e40 2024-09-10T08:43:24.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.190+0000 7fdd42a00640 1 -- 172.21.15.89:0/489025079 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd480c3810 con 0x55c67fa69e40 2024-09-10T08:43:24.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.191+0000 7fdd625fc740 10 client.26039 fetch_fsmap learned FSMap version 28 2024-09-10T08:43:24.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.191+0000 7fdd625fc740 1 -- 172.21.15.89:0/489025079 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c67fa79210 con 0x55c67fa69e40 2024-09-10T08:43:24.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.191+0000 7fdd42a00640 1 -- 172.21.15.89:0/489025079 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd4809d070 con 0x55c67fa69e40 2024-09-10T08:43:24.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.192+0000 7fdd42a00640 1 -- 172.21.15.89:0/489025079 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdd48099ce0 con 0x55c67fa69e40 2024-09-10T08:43:24.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.192+0000 7fdd625fc740 10 client.26039 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:43:24.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.192+0000 7fdd625fc740 10 client.26039 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:43:24.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.192+0000 7fdd625fc740 20 client.26039 populate_metadata read hostname 'smithi089' 2024-09-10T08:43:24.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.192+0000 7fdd625fc740 1 --2- 172.21.15.89:0/489025079 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55c67faefff0 0x55c67faf23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.192+0000 7fdd625fc740 4 client.26039 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:43:24.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.192+0000 7fdd625fc740 1 -- 172.21.15.89:0/489025079 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c67f9dd950 con 0x55c67faefff0 2024-09-10T08:43:24.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.208+0000 7fdd57e00640 1 --2- 172.21.15.89:0/489025079 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55c67faefff0 0x55c67faf23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:24.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.238+0000 7fdd42a00640 10 client.26039 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:43:24.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.238+0000 7fdd57e00640 1 --2- 172.21.15.89:0/489025079 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55c67faefff0 0x55c67faf23e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73f5b9a543091988 server_cookie=ffe0edf090caf345 in_seq=0 out_seq=0 2024-09-10T08:43:24.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.250+0000 7fdd42a00640 1 -- 172.21.15.89:0/489025079 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c67f9dd950 con 0x55c67faefff0 2024-09-10T08:43:24.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.250+0000 7fdd42a00640 10 client.26039 handle_command_reply: tid=0 2024-09-10T08:43:24.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.250+0000 7fdd625fc740 1 client.26039 shutdown 2024-09-10T08:43:24.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.250+0000 7fdd625fc740 1 -- 172.21.15.89:0/489025079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa69e40 msgr2=0x55c67fa7d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:24.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.250+0000 7fdd625fc740 1 --2- 172.21.15.89:0/489025079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa69e40 0x55c67fa7d460 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdd480094d0 tx=0x7fdd48005f90 comp rx=0 tx=0).stop 2024-09-10T08:43:24.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.251+0000 7fdd625fc740 1 -- 172.21.15.89:0/489025079 shutdown_connections 2024-09-10T08:43:24.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.251+0000 7fdd625fc740 1 --2- 172.21.15.89:0/489025079 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55c67faefff0 0x55c67faf23e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.251+0000 7fdd625fc740 1 --2- 172.21.15.89:0/489025079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67fa75020 0x55c67fa78cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.251+0000 7fdd625fc740 1 --2- 172.21.15.89:0/489025079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67fa72b00 0x55c67fa78790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.251+0000 7fdd625fc740 1 --2- 172.21.15.89:0/489025079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67fa69e40 0x55c67fa7d460 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.256 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:43:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.251+0000 7fdd625fc740 1 -- 172.21.15.89:0/489025079 >> 172.21.15.89:0/489025079 conn(0x55c67fa663d0 msgr2=0x55c67fa6b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.251+0000 7fdd42a00640 0 client.26039 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:43:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.251+0000 7fdd625fc740 1 -- 172.21.15.89:0/489025079 shutdown_connections 2024-09-10T08:43:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.251+0000 7fdd625fc740 1 -- 172.21.15.89:0/489025079 wait complete. 2024-09-10T08:43:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.251+0000 7fdd625fc740 20 client.26039 trim_cache size 0 max 16384 2024-09-10T08:43:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.252+0000 7fdd5e400640 1 -- 172.21.15.89:0/2911137647 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdd3c0a2d10 msgr2=0x7fdd3c0a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.252+0000 7fdd5e400640 1 --2- 172.21.15.89:0/2911137647 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdd3c0a2d10 0x7fdd3c0a51d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55c67fa71910 tx=0x7fdd44003eb0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.252+0000 7fdd5e400640 1 -- 172.21.15.89:0/2911137647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd5806ea30 msgr2=0x7fdd581a72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.252+0000 7fdd5e400640 1 --2- 172.21.15.89:0/2911137647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd5806ea30 0x7fdd581a72d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdd48035040 tx=0x7fdd48033d90 comp rx=0 tx=0).stop 2024-09-10T08:43:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.253+0000 7fdd5e400640 1 -- 172.21.15.89:0/2911137647 shutdown_connections 2024-09-10T08:43:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.253+0000 7fdd5e400640 1 --2- 172.21.15.89:0/2911137647 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdd3c0a2d10 0x7fdd3c0a51d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.253+0000 7fdd5e400640 1 --2- 172.21.15.89:0/2911137647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd581abba0 0x7fdd581ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.253+0000 7fdd5e400640 1 --2- 172.21.15.89:0/2911137647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd5811b140 0x7fdd581a7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.253+0000 7fdd5e400640 1 --2- 172.21.15.89:0/2911137647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd5806ea30 0x7fdd581a72d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.253+0000 7fdd5e400640 1 -- 172.21.15.89:0/2911137647 >> 172.21.15.89:0/2911137647 conn(0x7fdd58073180 msgr2=0x7fdd58119130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:24.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.253+0000 7fdd5e400640 1 -- 172.21.15.89:0/2911137647 shutdown_connections 2024-09-10T08:43:24.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:24.253+0000 7fdd5e400640 1 -- 172.21.15.89:0/2911137647 wait complete. 2024-09-10T08:43:24.952 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:24 smithi089 ceph-mon[30757]: pgmap v497: 129 pgs: 129 active+clean; 222 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 1.13k op/s 2024-09-10T08:43:25.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:24 smithi149 ceph-mon[36167]: pgmap v497: 129 pgs: 129 active+clean; 222 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 1.13k op/s 2024-09-10T08:43:25.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:24 smithi111 ceph-mon[35220]: pgmap v497: 129 pgs: 129 active+clean; 222 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 1.13k op/s 2024-09-10T08:43:26.042 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:25 smithi089 ceph-mon[30757]: pgmap v498: 129 pgs: 129 active+clean; 222 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 1.33k op/s 2024-09-10T08:43:26.061 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:43:26.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:25 smithi149 ceph-mon[36167]: pgmap v498: 129 pgs: 129 active+clean; 222 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 1.33k op/s 2024-09-10T08:43:26.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:25 smithi111 ceph-mon[35220]: pgmap v498: 129 pgs: 129 active+clean; 222 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 1.33k op/s 2024-09-10T08:43:26.590 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:28.029 INFO:tasks.workunit.client.0.smithi089.stdout: 77 356085 238 253270 143 192858 623 2024-09-10T08:43:28.345 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T08:43:28.345 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-09-10T08:43:28.345 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:43:28.828 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:28 smithi089 ceph-mon[30757]: pgmap v499: 129 pgs: 129 active+clean; 250 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 1.50k op/s 2024-09-10T08:43:28.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8799a00640 1 -- 172.21.15.89:0/1501173893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8794074ee0 msgr2=0x7f879410def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:28.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8799a00640 1 --2- 172.21.15.89:0/1501173893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8794074ee0 0x7f879410def0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8780009900 tx=0x7f878002a480 comp rx=0 tx=0).stop 2024-09-10T08:43:28.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8799a00640 1 -- 172.21.15.89:0/1501173893 shutdown_connections 2024-09-10T08:43:28.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8799a00640 1 --2- 172.21.15.89:0/1501173893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f879410e430 0x7f8794110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8799a00640 1 --2- 172.21.15.89:0/1501173893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8794074ee0 0x7f879410def0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8799a00640 1 --2- 172.21.15.89:0/1501173893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87940745c0 0x7f87940749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8799a00640 1 -- 172.21.15.89:0/1501173893 >> 172.21.15.89:0/1501173893 conn(0x7f87940fd8b0 msgr2=0x7f87940ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:28.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8799a00640 1 -- 172.21.15.89:0/1501173893 shutdown_connections 2024-09-10T08:43:28.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8799a00640 1 -- 172.21.15.89:0/1501173893 wait complete. 2024-09-10T08:43:28.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8799a00640 1 Processor -- start 2024-09-10T08:43:28.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8799a00640 1 -- start start 2024-09-10T08:43:28.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8799a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87940745c0 0x7f879410db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8799a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8794074ee0 0x7f8794107c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8799a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f879410e430 0x7f8794108140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8799a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f879406c660 con 0x7f8794074ee0 2024-09-10T08:43:28.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8799a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f879406c4e0 con 0x7f879410e430 2024-09-10T08:43:28.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8799a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f879406c7e0 con 0x7f87940745c0 2024-09-10T08:43:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8792000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8794074ee0 0x7f8794107c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8792000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8794074ee0 0x7f8794107c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52076/0 (socket says 172.21.15.89:52076) 2024-09-10T08:43:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8792000640 1 -- 172.21.15.89:0/1148869645 learned_addr learned my addr 172.21.15.89:0/1148869645 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8792a00640 1 --2- 172.21.15.89:0/1148869645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87940745c0 0x7f879410db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8793400640 1 --2- 172.21.15.89:0/1148869645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f879410e430 0x7f8794108140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8793400640 1 -- 172.21.15.89:0/1148869645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87940745c0 msgr2=0x7f879410db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8793400640 1 --2- 172.21.15.89:0/1148869645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87940745c0 0x7f879410db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8793400640 1 -- 172.21.15.89:0/1148869645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8794074ee0 msgr2=0x7f8794107c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8793400640 1 --2- 172.21.15.89:0/1148869645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8794074ee0 0x7f8794107c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8793400640 1 -- 172.21.15.89:0/1148869645 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8794108870 con 0x7f879410e430 2024-09-10T08:43:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.066+0000 7f8793400640 1 --2- 172.21.15.89:0/1148869645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f879410e430 0x7f8794108140 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f878c005fd0 tx=0x7f878c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.071+0000 7f878b400640 1 -- 172.21.15.89:0/1148869645 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c00cdf0 con 0x7f879410e430 2024-09-10T08:43:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.071+0000 7f878b400640 1 -- 172.21.15.89:0/1148869645 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f878c005020 con 0x7f879410e430 2024-09-10T08:43:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.071+0000 7f8799a00640 1 -- 172.21.15.89:0/1148869645 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87941af4c0 con 0x7f879410e430 2024-09-10T08:43:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.071+0000 7f878b400640 1 -- 172.21.15.89:0/1148869645 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c0106d0 con 0x7f879410e430 2024-09-10T08:43:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.071+0000 7f8799a00640 1 -- 172.21.15.89:0/1148869645 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87941af920 con 0x7f879410e430 2024-09-10T08:43:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.071+0000 7f879c472740 1 Processor -- start 2024-09-10T08:43:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.071+0000 7f878b400640 1 -- 172.21.15.89:0/1148869645 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f878c0054d0 con 0x7f879410e430 2024-09-10T08:43:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.071+0000 7f879c472740 1 -- start start 2024-09-10T08:43:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.076+0000 7f879c472740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71cdec70 0x55ea71cdf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.076+0000 7f879c472740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71cdf590 0x55ea71ce9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.076+0000 7f879c472740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71ce9e50 0x55ea71cec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.076+0000 7f879c472740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ea71c453e0 con 0x55ea71ce9e50 2024-09-10T08:43:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.076+0000 7f879c472740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ea71becd80 con 0x55ea71cdf590 2024-09-10T08:43:28.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.076+0000 7f879c472740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ea71c413a0 con 0x55ea71cdec70 2024-09-10T08:43:28.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.076+0000 7f8792a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71cdf590 0x55ea71ce9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.076+0000 7f8792a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71cdf590 0x55ea71ce9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57180/0 (socket says 172.21.15.89:57180) 2024-09-10T08:43:28.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.076+0000 7f8792a00640 1 -- 172.21.15.89:0/1226485246 learned_addr learned my addr 172.21.15.89:0/1226485246 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:28.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.076+0000 7f8792000640 1 --2- 172.21.15.89:0/1226485246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71ce9e50 0x55ea71cec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8792a00640 1 -- 172.21.15.89:0/1226485246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71cdec70 msgr2=0x55ea71cdf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:28.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f878b400640 1 --2- 172.21.15.89:0/1148869645 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f87780a2b80 0x7f87780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8792000640 1 --2- 172.21.15.89:0/1226485246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71cdec70 0x55ea71cdf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8792a00640 1 --2- 172.21.15.89:0/1226485246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71cdec70 0x55ea71cdf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8792a00640 1 -- 172.21.15.89:0/1226485246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71ce9e50 msgr2=0x55ea71cec240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:28.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8792a00640 1 --2- 172.21.15.89:0/1226485246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71ce9e50 0x55ea71cec240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8792a00640 1 -- 172.21.15.89:0/1226485246 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea71c0bae0 con 0x55ea71cdf590 2024-09-10T08:43:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8792000640 1 --2- 172.21.15.89:0/1226485246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71ce9e50 0x55ea71cec240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:43:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f878b400640 1 -- 172.21.15.89:0/1148869645 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f878c0cfae0 con 0x7f879410e430 2024-09-10T08:43:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8792a00640 1 --2- 172.21.15.89:0/1226485246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71cdf590 0x55ea71ce9910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8784009cf0 tx=0x7f8784011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c4fb3ab37fc79d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8793400640 1 --2- 172.21.15.89:0/1148869645 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f87780a2b80 0x7f87780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f877fe00640 1 -- 172.21.15.89:0/1226485246 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784004230 con 0x55ea71cdf590 2024-09-10T08:43:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f877fe00640 1 -- 172.21.15.89:0/1226485246 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87840043d0 con 0x55ea71cdf590 2024-09-10T08:43:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f877fe00640 1 -- 172.21.15.89:0/1226485246 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784019960 con 0x55ea71cdf590 2024-09-10T08:43:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f879c472740 1 -- 172.21.15.89:0/1226485246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71cdf590 msgr2=0x55ea71ce9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8793400640 1 --2- 172.21.15.89:0/1148869645 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f87780a2b80 0x7f87780a5040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f878c09bc10 tx=0x7f878c005440 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f879c472740 1 --2- 172.21.15.89:0/1226485246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71cdf590 0x55ea71ce9910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8784009cf0 tx=0x7f8784011810 comp rx=0 tx=0).stop 2024-09-10T08:43:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f879c472740 1 -- 172.21.15.89:0/1226485246 shutdown_connections 2024-09-10T08:43:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f879c472740 1 --2- 172.21.15.89:0/1226485246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71ce9e50 0x55ea71cec240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f879c472740 1 --2- 172.21.15.89:0/1226485246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71cdf590 0x55ea71ce9910 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.880 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f879c472740 1 --2- 172.21.15.89:0/1226485246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71cdec70 0x55ea71cdf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f879c472740 1 -- 172.21.15.89:0/1226485246 >> 172.21.15.89:0/1226485246 conn(0x55ea71cdb200 msgr2=0x55ea71cdb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f879c472740 1 -- 172.21.15.89:0/1226485246 shutdown_connections 2024-09-10T08:43:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f879c472740 1 -- 172.21.15.89:0/1226485246 wait complete. 2024-09-10T08:43:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f879c472740 1 -- start start 2024-09-10T08:43:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f879c472740 1 Processor -- start 2024-09-10T08:43:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f879c472740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71cdec70 0x55ea71ce6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f879c472740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71cdf590 0x55ea71ce72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f879c472740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71ce9e50 0x55ea71ce7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f879c472740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ea71ceca20 con 0x55ea71cdec70 2024-09-10T08:43:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f879c472740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ea71c000f0 con 0x55ea71cdf590 2024-09-10T08:43:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f879c472740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ea71cecba0 con 0x55ea71ce9e50 2024-09-10T08:43:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f879c472740 10 client.0 _ll_register_callbacks cb 0x55ea71be7bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:43:28.881 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8792a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71cdf590 0x55ea71ce72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8792a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71cdf590 0x55ea71ce72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57192/0 (socket says 172.21.15.89:57192) 2024-09-10T08:43:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8792a00640 1 -- 172.21.15.89:0/2620689069 learned_addr learned my addr 172.21.15.89:0/2620689069 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8792000640 1 --2- 172.21.15.89:0/2620689069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71cdec70 0x55ea71ce6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8792000640 1 --2- 172.21.15.89:0/2620689069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71ce9e50 0x55ea71ce7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8792a00640 1 -- 172.21.15.89:0/2620689069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71ce9e50 msgr2=0x55ea71ce7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8792a00640 1 --2- 172.21.15.89:0/2620689069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71ce9e50 0x55ea71ce7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8792a00640 1 -- 172.21.15.89:0/2620689069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71cdec70 msgr2=0x55ea71ce6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8792a00640 1 --2- 172.21.15.89:0/2620689069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71cdec70 0x55ea71ce6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8792a00640 1 -- 172.21.15.89:0/2620689069 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea719b63c0 con 0x55ea71cdf590 2024-09-10T08:43:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.077+0000 7f8792000640 1 --2- 172.21.15.89:0/2620689069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71ce9e50 0x55ea71ce7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:43:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f8792a00640 1 --2- 172.21.15.89:0/2620689069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71cdf590 0x55ea71ce72d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8784000c00 tx=0x7f8784005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=147126ad4f804d46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f877cc00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:43:28.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f877cc00640 1 -- 172.21.15.89:0/2620689069 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784027070 con 0x55ea71cdf590 2024-09-10T08:43:28.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f877cc00640 1 -- 172.21.15.89:0/2620689069 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f87840037f0 con 0x55ea71cdf590 2024-09-10T08:43:28.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f879c472740 1 -- 172.21.15.89:0/2620689069 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87840043d0 con 0x55ea71cdf590 2024-09-10T08:43:28.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f877cc00640 1 -- 172.21.15.89:0/2620689069 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8784033050 con 0x55ea71cdf590 2024-09-10T08:43:28.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f879c472740 10 client.26051 fetch_fsmap learned FSMap version 28 2024-09-10T08:43:28.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f879c472740 1 -- 172.21.15.89:0/2620689069 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ea71cf8ab0 con 0x55ea71cdf590 2024-09-10T08:43:28.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f877cc00640 1 -- 172.21.15.89:0/2620689069 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87840043d0 con 0x55ea71cdf590 2024-09-10T08:43:28.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f877cc00640 1 -- 172.21.15.89:0/2620689069 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8784004ba0 con 0x55ea71cdf590 2024-09-10T08:43:28.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f879c472740 10 client.26051 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:43:28.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f879c472740 10 client.26051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:43:28.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f879c472740 20 client.26051 populate_metadata read hostname 'smithi089' 2024-09-10T08:43:28.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f879c472740 1 --2- 172.21.15.89:0/2620689069 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ea71cf9e60 0x55ea71cfc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f879c472740 4 client.26051 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:43:28.883 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f879c472740 1 -- 172.21.15.89:0/2620689069 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea71c52950 con 0x55ea71cf9e60 2024-09-10T08:43:28.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f8792000640 1 --2- 172.21.15.89:0/2620689069 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ea71cf9e60 0x55ea71cfc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f8792000640 1 --2- 172.21.15.89:0/2620689069 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ea71cf9e60 0x55ea71cfc250 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f9c194f8414a9cc server_cookie=9fc9e6a581660d6f in_seq=0 out_seq=0 2024-09-10T08:43:28.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f877cc00640 10 client.26051 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:28.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f877cc00640 1 -- 172.21.15.89:0/2620689069 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea71c52950 con 0x55ea71cf9e60 2024-09-10T08:43:28.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f877cc00640 10 client.26051 handle_command_reply: tid=0 2024-09-10T08:43:28.884 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.082+0000 7f879c472740 1 client.26051 shutdown 2024-09-10T08:43:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.085+0000 7f879c472740 1 -- 172.21.15.89:0/2620689069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71cdf590 msgr2=0x55ea71ce72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.085+0000 7f879c472740 1 --2- 172.21.15.89:0/2620689069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71cdf590 0x55ea71ce72d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8784000c00 tx=0x7f8784005270 comp rx=0 tx=0).stop 2024-09-10T08:43:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.085+0000 7f879c472740 1 -- 172.21.15.89:0/2620689069 shutdown_connections 2024-09-10T08:43:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.085+0000 7f879c472740 1 --2- 172.21.15.89:0/2620689069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71ce9e50 0x55ea71ce7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.085+0000 7f879c472740 1 --2- 172.21.15.89:0/2620689069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71cdf590 0x55ea71ce72d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.085+0000 7f879c472740 1 --2- 172.21.15.89:0/2620689069 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ea71cf9e60 0x55ea71cfc250 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.085+0000 7f879c472740 1 --2- 172.21.15.89:0/2620689069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71cdec70 0x55ea71ce6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.085+0000 7f879c472740 1 -- 172.21.15.89:0/2620689069 >> 172.21.15.89:0/2620689069 conn(0x55ea71cdb200 msgr2=0x55ea71ce0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.085+0000 7f877cc00640 0 client.26051 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.085+0000 7f879c472740 1 -- 172.21.15.89:0/2620689069 shutdown_connections 2024-09-10T08:43:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.085+0000 7f879c472740 1 -- 172.21.15.89:0/2620689069 wait complete. 2024-09-10T08:43:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.085+0000 7f879c472740 20 client.26051 trim_cache size 0 max 16384 2024-09-10T08:43:28.885 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f879c472740 1 Processor -- start 2024-09-10T08:43:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f879c472740 1 -- start start 2024-09-10T08:43:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f879c472740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71cdec70 0x55ea71ce8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f879c472740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71cdf590 0x55ea71d019e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f879c472740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71ce9e50 0x55ea71d01f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f879c472740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f87941bc0b0 con 0x55ea71cdf590 2024-09-10T08:43:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f879c472740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f87941b9980 con 0x55ea71ce9e50 2024-09-10T08:43:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f879c472740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f87941b77d0 con 0x55ea71cdec70 2024-09-10T08:43:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f8792a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71cdf590 0x55ea71d019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f8792000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71cdec70 0x55ea71ce8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f8792000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71ce9e50 0x55ea71d01f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f8792000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71ce9e50 0x55ea71d01f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57202/0 (socket says 172.21.15.89:57202) 2024-09-10T08:43:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f8792a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71cdf590 0x55ea71d019e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52094/0 (socket says 172.21.15.89:52094) 2024-09-10T08:43:28.886 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f8792000640 1 -- 172.21.15.89:0/140680390 learned_addr learned my addr 172.21.15.89:0/140680390 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:28.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f8792000640 1 -- 172.21.15.89:0/140680390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71cdec70 msgr2=0x55ea71ce8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:28.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f8792000640 1 --2- 172.21.15.89:0/140680390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71cdec70 0x55ea71ce8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f8792000640 1 -- 172.21.15.89:0/140680390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71cdf590 msgr2=0x55ea71d019e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:28.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f8792000640 1 --2- 172.21.15.89:0/140680390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71cdf590 0x55ea71d019e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f8792000640 1 -- 172.21.15.89:0/140680390 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea71cd6770 con 0x55ea71ce9e50 2024-09-10T08:43:28.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f8792000640 1 --2- 172.21.15.89:0/140680390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71ce9e50 0x55ea71d01f20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f878002aa40 tx=0x7f8780009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b76e5b2b7e90666b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:28.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f877f400640 1 -- 172.21.15.89:0/140680390 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780035040 con 0x55ea71ce9e50 2024-09-10T08:43:28.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f877f400640 1 -- 172.21.15.89:0/140680390 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8780005500 con 0x55ea71ce9e50 2024-09-10T08:43:28.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.099+0000 7f877f400640 1 -- 172.21.15.89:0/140680390 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780032670 con 0x55ea71ce9e50 2024-09-10T08:43:28.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 -- 172.21.15.89:0/140680390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71ce9e50 msgr2=0x55ea71d01f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:28.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 --2- 172.21.15.89:0/140680390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71ce9e50 0x55ea71d01f20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f878002aa40 tx=0x7f8780009260 comp rx=0 tx=0).stop 2024-09-10T08:43:28.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 -- 172.21.15.89:0/140680390 shutdown_connections 2024-09-10T08:43:28.887 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 --2- 172.21.15.89:0/140680390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71ce9e50 0x55ea71d01f20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 --2- 172.21.15.89:0/140680390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71cdf590 0x55ea71d019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 --2- 172.21.15.89:0/140680390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71cdec70 0x55ea71ce8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 -- 172.21.15.89:0/140680390 >> 172.21.15.89:0/140680390 conn(0x55ea71cdb200 msgr2=0x55ea71ce85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:28.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 -- 172.21.15.89:0/140680390 shutdown_connections 2024-09-10T08:43:28.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 -- 172.21.15.89:0/140680390 wait complete. 2024-09-10T08:43:28.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 -- start start 2024-09-10T08:43:28.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 Processor -- start 2024-09-10T08:43:28.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71cdec70 0x55ea71cf2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71cdf590 0x55ea71ced870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.889 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:43:28.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71ce9e50 0x55ea71ceddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f87941b7e40 con 0x55ea71cdec70 2024-09-10T08:43:28.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f87941b92b0 con 0x55ea71cdf590 2024-09-10T08:43:28.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f87941b8510 con 0x55ea71ce9e50 2024-09-10T08:43:28.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 10 client.0 _ll_register_callbacks cb 0x7f8784005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:43:28.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f8792000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71cdec70 0x55ea71cf2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f8792000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71cdec70 0x55ea71cf2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52100/0 (socket says 172.21.15.89:52100) 2024-09-10T08:43:28.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f8792000640 1 -- 172.21.15.89:0/1362463106 learned_addr learned my addr 172.21.15.89:0/1362463106 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:28.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f8792a00640 1 --2- 172.21.15.89:0/1362463106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71cdf590 0x55ea71ced870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f8792000640 1 --2- 172.21.15.89:0/1362463106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71ce9e50 0x55ea71ceddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f8792a00640 1 -- 172.21.15.89:0/1362463106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71ce9e50 msgr2=0x55ea71ceddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:28.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f8792a00640 1 --2- 172.21.15.89:0/1362463106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71ce9e50 0x55ea71ceddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f8792a00640 1 -- 172.21.15.89:0/1362463106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71cdec70 msgr2=0x55ea71cf2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:28.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f8792a00640 1 --2- 172.21.15.89:0/1362463106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71cdec70 0x55ea71cf2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f8792a00640 1 -- 172.21.15.89:0/1362463106 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea71cee590 con 0x55ea71cdf590 2024-09-10T08:43:28.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f8792000640 1 --2- 172.21.15.89:0/1362463106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71cdec70 0x55ea71cf2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:43:28.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f8792a00640 1 --2- 172.21.15.89:0/1362463106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71cdf590 0x55ea71ced870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55ea71ce7e80 tx=0x7f87840198f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98c3edddeb5dc46a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:28.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f877cc00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:43:28.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f877cc00640 1 -- 172.21.15.89:0/1362463106 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878403c060 con 0x55ea71cdf590 2024-09-10T08:43:28.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 -- 172.21.15.89:0/1362463106 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8780005500 con 0x55ea71cdf590 2024-09-10T08:43:28.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f877cc00640 1 -- 172.21.15.89:0/1362463106 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f8784003dd0 con 0x55ea71cdf590 2024-09-10T08:43:28.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f877cc00640 1 -- 172.21.15.89:0/1362463106 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8784043050 con 0x55ea71cdf590 2024-09-10T08:43:28.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 10 client.26057 fetch_fsmap learned FSMap version 28 2024-09-10T08:43:28.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 -- 172.21.15.89:0/1362463106 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ea71d649c0 con 0x55ea71cdf590 2024-09-10T08:43:28.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f877cc00640 1 -- 172.21.15.89:0/1362463106 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8780005500 con 0x55ea71cdf590 2024-09-10T08:43:28.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f877cc00640 1 -- 172.21.15.89:0/1362463106 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8784004640 con 0x55ea71cdf590 2024-09-10T08:43:28.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 10 client.26057 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:43:28.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 10 client.26057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:43:28.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 20 client.26057 populate_metadata read hostname 'smithi089' 2024-09-10T08:43:28.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 --2- 172.21.15.89:0/1362463106 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ea71d651d0 0x55ea71d675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 4 client.26057 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:43:28.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f879c472740 1 -- 172.21.15.89:0/1362463106 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea71c4ce60 con 0x55ea71d651d0 2024-09-10T08:43:28.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f8792000640 1 --2- 172.21.15.89:0/1362463106 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ea71d651d0 0x55ea71d675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f8792000640 1 --2- 172.21.15.89:0/1362463106 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ea71d651d0 0x55ea71d675f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3b7039f79c56652 server_cookie=c5059f15540b15a7 in_seq=0 out_seq=0 2024-09-10T08:43:28.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.101+0000 7f877cc00640 10 client.26057 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:28.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f877cc00640 1 -- 172.21.15.89:0/1362463106 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea71c4ce60 con 0x55ea71d651d0 2024-09-10T08:43:28.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f877cc00640 10 client.26057 handle_command_reply: tid=0 2024-09-10T08:43:28.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f879c472740 1 client.26057 shutdown 2024-09-10T08:43:28.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f879c472740 1 -- 172.21.15.89:0/1362463106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71cdf590 msgr2=0x55ea71ced870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:28.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f879c472740 1 --2- 172.21.15.89:0/1362463106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71cdf590 0x55ea71ced870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55ea71ce7e80 tx=0x7f87840198f0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f879c472740 1 -- 172.21.15.89:0/1362463106 shutdown_connections 2024-09-10T08:43:28.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f879c472740 1 --2- 172.21.15.89:0/1362463106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea71ce9e50 0x55ea71ceddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f879c472740 1 --2- 172.21.15.89:0/1362463106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea71cdf590 0x55ea71ced870 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f879c472740 1 --2- 172.21.15.89:0/1362463106 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ea71d651d0 0x55ea71d675f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f879c472740 1 --2- 172.21.15.89:0/1362463106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea71cdec70 0x55ea71cf2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f879c472740 1 -- 172.21.15.89:0/1362463106 >> 172.21.15.89:0/1362463106 conn(0x55ea71cdb200 msgr2=0x55ea71ce0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:28.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f877cc00640 0 client.26057 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:28.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f879c472740 1 -- 172.21.15.89:0/1362463106 shutdown_connections 2024-09-10T08:43:28.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f879c472740 1 -- 172.21.15.89:0/1362463106 wait complete. 2024-09-10T08:43:28.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f879c472740 20 client.26057 trim_cache size 0 max 16384 2024-09-10T08:43:28.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f8799a00640 1 -- 172.21.15.89:0/1148869645 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f87780a2b80 msgr2=0x7f87780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:28.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f8799a00640 1 --2- 172.21.15.89:0/1148869645 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f87780a2b80 0x7f87780a5040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f878c09bc10 tx=0x7f878c005440 comp rx=0 tx=0).stop 2024-09-10T08:43:28.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f8799a00640 1 -- 172.21.15.89:0/1148869645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f879410e430 msgr2=0x7f8794108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:28.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f8799a00640 1 --2- 172.21.15.89:0/1148869645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f879410e430 0x7f8794108140 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f878c005fd0 tx=0x7f878c00ef10 comp rx=0 tx=0).stop 2024-09-10T08:43:28.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f8799a00640 1 -- 172.21.15.89:0/1148869645 shutdown_connections 2024-09-10T08:43:28.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f8799a00640 1 --2- 172.21.15.89:0/1148869645 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f87780a2b80 0x7f87780a5040 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f8799a00640 1 --2- 172.21.15.89:0/1148869645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f879410e430 0x7f8794108140 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f8799a00640 1 --2- 172.21.15.89:0/1148869645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8794074ee0 0x7f8794107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f8799a00640 1 --2- 172.21.15.89:0/1148869645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87940745c0 0x7f879410db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f8799a00640 1 -- 172.21.15.89:0/1148869645 >> 172.21.15.89:0/1148869645 conn(0x7f87940fd8b0 msgr2=0x7f87940ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:28.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f8799a00640 1 -- 172.21.15.89:0/1148869645 shutdown_connections 2024-09-10T08:43:28.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:28.106+0000 7f8799a00640 1 -- 172.21.15.89:0/1148869645 wait complete. 2024-09-10T08:43:29.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:28 smithi111 ceph-mon[35220]: pgmap v499: 129 pgs: 129 active+clean; 250 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 1.50k op/s 2024-09-10T08:43:29.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:28 smithi149 ceph-mon[36167]: pgmap v499: 129 pgs: 129 active+clean; 250 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 1.50k op/s 2024-09-10T08:43:30.872 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:30 smithi089 ceph-mon[30757]: pgmap v500: 129 pgs: 129 active+clean; 261 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.3 MiB/s wr, 1.47k op/s 2024-09-10T08:43:31.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:30 smithi111 ceph-mon[35220]: pgmap v500: 129 pgs: 129 active+clean; 261 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.3 MiB/s wr, 1.47k op/s 2024-09-10T08:43:31.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:30 smithi149 ceph-mon[36167]: pgmap v500: 129 pgs: 129 active+clean; 261 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.3 MiB/s wr, 1.47k op/s 2024-09-10T08:43:31.599 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:32.450 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:43:32.450 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:43:33.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.039+0000 7f8af9e00640 1 -- 172.21.15.89:0/3083441424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af4106f10 msgr2=0x7f8af410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:33.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.039+0000 7f8af9e00640 1 --2- 172.21.15.89:0/3083441424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af4106f10 0x7f8af410ea90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009ca0 tx=0x7f8aec02a8f0 comp rx=0 tx=0).stop 2024-09-10T08:43:33.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.040+0000 7f8af9e00640 1 -- 172.21.15.89:0/3083441424 shutdown_connections 2024-09-10T08:43:33.042 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:32 smithi089 ceph-mon[30757]: pgmap v501: 129 pgs: 129 active+clean; 268 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.0 MiB/s wr, 1.38k op/s 2024-09-10T08:43:33.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.040+0000 7f8af9e00640 1 --2- 172.21.15.89:0/3083441424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af4106f10 0x7f8af410ea90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:33.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.042+0000 7f8af9e00640 1 --2- 172.21.15.89:0/3083441424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8af4102790 0x7f8af41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:33.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.042+0000 7f8af9e00640 1 --2- 172.21.15.89:0/3083441424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8af4101de0 0x7f8af41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:33.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.042+0000 7f8af9e00640 1 -- 172.21.15.89:0/3083441424 >> 172.21.15.89:0/3083441424 conn(0x7f8af40fd8e0 msgr2=0x7f8af40ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:33.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:32 smithi111 ceph-mon[35220]: pgmap v501: 129 pgs: 129 active+clean; 268 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.0 MiB/s wr, 1.38k op/s 2024-09-10T08:43:33.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.042+0000 7f8af9e00640 1 -- 172.21.15.89:0/3083441424 shutdown_connections 2024-09-10T08:43:33.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.042+0000 7f8af9e00640 1 -- 172.21.15.89:0/3083441424 wait complete. 2024-09-10T08:43:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.044+0000 7f8af9e00640 1 Processor -- start 2024-09-10T08:43:33.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.044+0000 7f8af9e00640 1 -- start start 2024-09-10T08:43:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.045+0000 7f8af9e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8af4101de0 0x7f8af4116370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.046+0000 7f8af9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af4102790 0x7f8af41168b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.046+0000 7f8af2a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8af4101de0 0x7f8af4116370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.046+0000 7f8af2a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8af4101de0 0x7f8af4116370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57038/0 (socket says 172.21.15.89:57038) 2024-09-10T08:43:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.047+0000 7f8af2a00640 1 -- 172.21.15.89:0/776948437 learned_addr learned my addr 172.21.15.89:0/776948437 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:33.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.047+0000 7f8af2000640 1 --2- 172.21.15.89:0/776948437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af4102790 0x7f8af41168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:33.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.047+0000 7f8af9e00640 1 --2- 172.21.15.89:0/776948437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8af4106f10 0x7f8af411c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:33.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.047+0000 7f8af9e00640 1 -- 172.21.15.89:0/776948437 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8af40761c0 con 0x7f8af4102790 2024-09-10T08:43:33.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.047+0000 7f8af9e00640 1 -- 172.21.15.89:0/776948437 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8af4076040 con 0x7f8af4101de0 2024-09-10T08:43:33.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.047+0000 7f8af3400640 1 --2- 172.21.15.89:0/776948437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8af4106f10 0x7f8af411c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:33.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.047+0000 7f8af9e00640 1 -- 172.21.15.89:0/776948437 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8af4076340 con 0x7f8af4106f10 2024-09-10T08:43:33.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.048+0000 7f8af2a00640 1 -- 172.21.15.89:0/776948437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8af4106f10 msgr2=0x7f8af411c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:33.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.048+0000 7f8af2a00640 1 --2- 172.21.15.89:0/776948437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8af4106f10 0x7f8af411c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:33.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.048+0000 7f8af2a00640 1 -- 172.21.15.89:0/776948437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af4102790 msgr2=0x7f8af41168b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:33.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.048+0000 7f8af2a00640 1 --2- 172.21.15.89:0/776948437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af4102790 0x7f8af41168b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:33.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.048+0000 7f8af2a00640 1 -- 172.21.15.89:0/776948437 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af411c9a0 con 0x7f8af4101de0 2024-09-10T08:43:33.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.049+0000 7f8af2000640 1 --2- 172.21.15.89:0/776948437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af4102790 0x7f8af41168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:43:33.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.049+0000 7f8af2a00640 1 --2- 172.21.15.89:0/776948437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8af4101de0 0x7f8af4116370 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400b450 tx=0x7f8ae400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:33.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.049+0000 7f8af8a00640 1 -- 172.21.15.89:0/776948437 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4015020 con 0x7f8af4101de0 2024-09-10T08:43:33.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.049+0000 7f8af8a00640 1 -- 172.21.15.89:0/776948437 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ae4011d70 con 0x7f8af4101de0 2024-09-10T08:43:33.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.049+0000 7f8af8a00640 1 -- 172.21.15.89:0/776948437 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae400f340 con 0x7f8af4101de0 2024-09-10T08:43:33.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.049+0000 7f8af9e00640 1 -- 172.21.15.89:0/776948437 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af411cc90 con 0x7f8af4101de0 2024-09-10T08:43:33.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.050+0000 7f8af9e00640 1 -- 172.21.15.89:0/776948437 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af41117c0 con 0x7f8af4101de0 2024-09-10T08:43:33.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.055+0000 7f8af9e00640 1 -- 172.21.15.89:0/776948437 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad4005290 con 0x7f8af4101de0 2024-09-10T08:43:33.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.062+0000 7f8af8a00640 1 -- 172.21.15.89:0/776948437 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8ae4018020 con 0x7f8af4101de0 2024-09-10T08:43:33.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.068+0000 7f8af8a00640 1 --2- 172.21.15.89:0/776948437 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8ad80a2cc0 0x7f8ad80a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:33.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.069+0000 7f8af8a00640 1 -- 172.21.15.89:0/776948437 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f8ae40ced10 con 0x7f8af4101de0 2024-09-10T08:43:33.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.069+0000 7f8af2000640 1 --2- 172.21.15.89:0/776948437 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8ad80a2cc0 0x7f8ad80a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:33.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.072+0000 7f8af8a00640 1 -- 172.21.15.89:0/776948437 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ae401f020 con 0x7f8af4101de0 2024-09-10T08:43:33.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.073+0000 7f8af2000640 1 --2- 172.21.15.89:0/776948437 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8ad80a2cc0 0x7f8ad80a5180 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0006fd0 tx=0x7f8ae0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:33.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:32 smithi149 ceph-mon[36167]: pgmap v501: 129 pgs: 129 active+clean; 268 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.0 MiB/s wr, 1.38k op/s 2024-09-10T08:43:33.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.515+0000 7f8af9e00640 1 -- 172.21.15.89:0/776948437 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ad4005580 con 0x7f8af4101de0 2024-09-10T08:43:33.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.517+0000 7f8af8a00640 1 -- 172.21.15.89:0/776948437 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11921 (secure 0 0 0) 0x7f8ae400d040 con 0x7f8af4101de0 2024-09-10T08:43:33.542 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:43:33.542 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":28,"btime":"2024-09-10T08:41:55:762565+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:41:55.148062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24518,24496]},"id":1}]} 2024-09-10T08:43:33.542 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 28 2024-09-10T08:43:33.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.523+0000 7f8af9e00640 1 -- 172.21.15.89:0/776948437 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8ad80a2cc0 msgr2=0x7f8ad80a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:33.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.523+0000 7f8af9e00640 1 --2- 172.21.15.89:0/776948437 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8ad80a2cc0 0x7f8ad80a5180 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0006fd0 tx=0x7f8ae0008040 comp rx=0 tx=0).stop 2024-09-10T08:43:33.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.524+0000 7f8af9e00640 1 -- 172.21.15.89:0/776948437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8af4101de0 msgr2=0x7f8af4116370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:33.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.524+0000 7f8af9e00640 1 --2- 172.21.15.89:0/776948437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8af4101de0 0x7f8af4116370 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400b450 tx=0x7f8ae400b910 comp rx=0 tx=0).stop 2024-09-10T08:43:33.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.527+0000 7f8af9e00640 1 -- 172.21.15.89:0/776948437 shutdown_connections 2024-09-10T08:43:33.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.527+0000 7f8af9e00640 1 --2- 172.21.15.89:0/776948437 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8ad80a2cc0 0x7f8ad80a5180 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:33.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.527+0000 7f8af9e00640 1 --2- 172.21.15.89:0/776948437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8af4106f10 0x7f8af411c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:33.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.527+0000 7f8af9e00640 1 --2- 172.21.15.89:0/776948437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af4102790 0x7f8af41168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:33.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.527+0000 7f8af9e00640 1 --2- 172.21.15.89:0/776948437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8af4101de0 0x7f8af4116370 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:33.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.527+0000 7f8af9e00640 1 -- 172.21.15.89:0/776948437 >> 172.21.15.89:0/776948437 conn(0x7f8af40fd8e0 msgr2=0x7f8af40fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:33.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.527+0000 7f8af9e00640 1 -- 172.21.15.89:0/776948437 shutdown_connections 2024-09-10T08:43:33.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:33.527+0000 7f8af9e00640 1 -- 172.21.15.89:0/776948437 wait complete. 2024-09-10T08:43:33.966 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:33 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/776948437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:34.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:33 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/776948437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:34.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:33 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/776948437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.581+0000 7f45d3a00640 1 -- 172.21.15.89:0/3032951058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45cc109700 msgr2=0x7f45cc109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.581+0000 7f45d3a00640 1 --2- 172.21.15.89:0/3032951058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45cc109700 0x7f45cc109b00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f45b80098b0 tx=0x7f45b802a3b0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.589+0000 7f45d3a00640 1 -- 172.21.15.89:0/3032951058 shutdown_connections 2024-09-10T08:43:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.589+0000 7f45d3a00640 1 --2- 172.21.15.89:0/3032951058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45cc0736c0 0x7f45cc119130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.589+0000 7f45d3a00640 1 --2- 172.21.15.89:0/3032951058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45cc072d20 0x7f45cc073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.589+0000 7f45d3a00640 1 --2- 172.21.15.89:0/3032951058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45cc109700 0x7f45cc109b00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.589+0000 7f45d3a00640 1 -- 172.21.15.89:0/3032951058 >> 172.21.15.89:0/3032951058 conn(0x7f45cc06ece0 msgr2=0x7f45cc071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.600+0000 7f45d3a00640 1 -- 172.21.15.89:0/3032951058 shutdown_connections 2024-09-10T08:43:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.607+0000 7f45d3a00640 1 -- 172.21.15.89:0/3032951058 wait complete. 2024-09-10T08:43:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.608+0000 7f45d3a00640 1 Processor -- start 2024-09-10T08:43:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.610+0000 7f45d3a00640 1 -- start start 2024-09-10T08:43:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.610+0000 7f45d3a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45cc072d20 0x7f45cc19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.611+0000 7f45d3a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45cc0736c0 0x7f45cc19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.614+0000 7f45d3a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45cc109700 0x7f45cc1a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.614+0000 7f45d3a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc1135e0 con 0x7f45cc109700 2024-09-10T08:43:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.614+0000 7f45d3a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc113460 con 0x7f45cc0736c0 2024-09-10T08:43:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.614+0000 7f45d3a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc113760 con 0x7f45cc072d20 2024-09-10T08:43:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.614+0000 7f45d1a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45cc109700 0x7f45cc1a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.614+0000 7f45d1a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45cc109700 0x7f45cc1a5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34638/0 (socket says 172.21.15.89:34638) 2024-09-10T08:43:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.614+0000 7f45d1a00640 1 -- 172.21.15.89:0/3538205135 learned_addr learned my addr 172.21.15.89:0/3538205135 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.615+0000 7f45cbe00640 1 --2- 172.21.15.89:0/3538205135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45cc0736c0 0x7f45cc19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.613+0000 7f45d1000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45cc072d20 0x7f45cc19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.616+0000 7f45d1000640 1 -- 172.21.15.89:0/3538205135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45cc0736c0 msgr2=0x7f45cc19ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.616+0000 7f45d1000640 1 --2- 172.21.15.89:0/3538205135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45cc0736c0 0x7f45cc19ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.616+0000 7f45d1000640 1 -- 172.21.15.89:0/3538205135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45cc109700 msgr2=0x7f45cc1a5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.616+0000 7f45d1000640 1 --2- 172.21.15.89:0/3538205135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45cc109700 0x7f45cc1a5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.616+0000 7f45d1000640 1 -- 172.21.15.89:0/3538205135 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45cc1a5eb0 con 0x7f45cc072d20 2024-09-10T08:43:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.620+0000 7f45d1000640 1 --2- 172.21.15.89:0/3538205135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45cc072d20 0x7f45cc19e8f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f45b802a8d0 tx=0x7f45b8030860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.620+0000 7f45c9600640 1 -- 172.21.15.89:0/3538205135 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8037040 con 0x7f45cc072d20 2024-09-10T08:43:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.620+0000 7f45c9600640 1 -- 172.21.15.89:0/3538205135 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b8030a50 con 0x7f45cc072d20 2024-09-10T08:43:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.620+0000 7f45c9600640 1 -- 172.21.15.89:0/3538205135 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b803b3f0 con 0x7f45cc072d20 2024-09-10T08:43:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.621+0000 7f45d3a00640 1 -- 172.21.15.89:0/3538205135 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45cc1a6140 con 0x7f45cc072d20 2024-09-10T08:43:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.621+0000 7f45d3a00640 1 -- 172.21.15.89:0/3538205135 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45cc1a6630 con 0x7f45cc072d20 2024-09-10T08:43:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.624+0000 7f45d1a00640 1 --2- 172.21.15.89:0/3538205135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45cc109700 0x7f45cc1a5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:43:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.628+0000 7f45d6836740 1 Processor -- start 2024-09-10T08:43:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.631+0000 7f45c9600640 1 -- 172.21.15.89:0/3538205135 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f45b803b590 con 0x7f45cc072d20 2024-09-10T08:43:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.633+0000 7f45c9600640 1 --2- 172.21.15.89:0/3538205135 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f45b00a2d80 0x7f45b00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.633+0000 7f45c9600640 1 -- 172.21.15.89:0/3538205135 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f45b8004230 con 0x7f45cc072d20 2024-09-10T08:43:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.634+0000 7f45d1a00640 1 --2- 172.21.15.89:0/3538205135 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f45b00a2d80 0x7f45b00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.636+0000 7f45d1a00640 1 --2- 172.21.15.89:0/3538205135 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f45b00a2d80 0x7f45b00a5240 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f45bc01ccb0 tx=0x7f45bc005f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.651+0000 7f45d6836740 1 -- start start 2024-09-10T08:43:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.651+0000 7f45d6836740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead893090 0x560ead893490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.651+0000 7f45d6836740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8939d0 0x560ead898490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.651+0000 7f45d6836740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ead898a60 0x560ead89af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.651+0000 7f45d6836740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560ead7f8100 con 0x560ead893090 2024-09-10T08:43:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.651+0000 7f45d6836740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560ead7a01b0 con 0x560ead8939d0 2024-09-10T08:43:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.651+0000 7f45d6836740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560ead7f40c0 con 0x560ead898a60 2024-09-10T08:43:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.652+0000 7f45d1a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ead898a60 0x560ead89af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.652+0000 7f45d1a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ead898a60 0x560ead89af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44268/0 (socket says 172.21.15.89:44268) 2024-09-10T08:43:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.652+0000 7f45d1a00640 1 -- 172.21.15.89:0/76266011 learned_addr learned my addr 172.21.15.89:0/76266011 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.652+0000 7f45cbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8939d0 0x560ead898490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.652+0000 7f45cbe00640 1 -- 172.21.15.89:0/76266011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ead898a60 msgr2=0x560ead89af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.652+0000 7f45cbe00640 1 --2- 172.21.15.89:0/76266011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ead898a60 0x560ead89af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.652+0000 7f45cbe00640 1 -- 172.21.15.89:0/76266011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead893090 msgr2=0x560ead893490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:43:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.652+0000 7f45cbe00640 1 --2- 172.21.15.89:0/76266011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead893090 0x560ead893490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.652+0000 7f45cbe00640 1 -- 172.21.15.89:0/76266011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ead50b210 con 0x560ead8939d0 2024-09-10T08:43:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.653+0000 7f45cbe00640 1 --2- 172.21.15.89:0/76266011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8939d0 0x560ead898490 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009ed0 tx=0x7f45c400c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0110a4d9d366b49 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.657+0000 7f45c2000640 1 -- 172.21.15.89:0/76266011 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c403c070 con 0x560ead8939d0 2024-09-10T08:43:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.657+0000 7f45c2000640 1 -- 172.21.15.89:0/76266011 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c400aad0 con 0x560ead8939d0 2024-09-10T08:43:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.657+0000 7f45c2000640 1 -- 172.21.15.89:0/76266011 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4037cf0 con 0x560ead8939d0 2024-09-10T08:43:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.658+0000 7f45d6836740 1 -- 172.21.15.89:0/76266011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8939d0 msgr2=0x560ead898490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.658+0000 7f45d6836740 1 --2- 172.21.15.89:0/76266011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8939d0 0x560ead898490 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009ed0 tx=0x7f45c400c7d0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.666+0000 7f45d6836740 1 -- 172.21.15.89:0/76266011 shutdown_connections 2024-09-10T08:43:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.666+0000 7f45d6836740 1 --2- 172.21.15.89:0/76266011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ead898a60 0x560ead89af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.668 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.666+0000 7f45d6836740 1 --2- 172.21.15.89:0/76266011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8939d0 0x560ead898490 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.668 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.666+0000 7f45d6836740 1 --2- 172.21.15.89:0/76266011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead893090 0x560ead893490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.668 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.666+0000 7f45d6836740 1 -- 172.21.15.89:0/76266011 >> 172.21.15.89:0/76266011 conn(0x560ead88e700 msgr2=0x560ead88eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.667+0000 7f45d6836740 1 -- 172.21.15.89:0/76266011 shutdown_connections 2024-09-10T08:43:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.667+0000 7f45d6836740 1 -- 172.21.15.89:0/76266011 wait complete. 2024-09-10T08:43:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.668+0000 7f45d6836740 1 -- start start 2024-09-10T08:43:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.668+0000 7f45d6836740 1 Processor -- start 2024-09-10T08:43:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.668+0000 7f45d6836740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ead893090 0x560ead89d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.672+0000 7f45d6836740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8939d0 0x560ead89d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.672+0000 7f45d6836740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead898a60 0x560ead8a4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.673+0000 7f45cbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8939d0 0x560ead89d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.673+0000 7f45cbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8939d0 0x560ead89d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57070/0 (socket says 172.21.15.89:57070) 2024-09-10T08:43:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.673+0000 7f45cbe00640 1 -- 172.21.15.89:0/554333670 learned_addr learned my addr 172.21.15.89:0/554333670 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.673+0000 7f45d6836740 1 -- 172.21.15.89:0/554333670 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560ead89ba30 con 0x560ead898a60 2024-09-10T08:43:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.673+0000 7f45d6836740 1 -- 172.21.15.89:0/554333670 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560ead781090 con 0x560ead8939d0 2024-09-10T08:43:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.673+0000 7f45d6836740 1 -- 172.21.15.89:0/554333670 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560ead89bbb0 con 0x560ead893090 2024-09-10T08:43:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.673+0000 7f45d6836740 10 client.0 _ll_register_callbacks cb 0x560ead7d5f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:43:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.673+0000 7f45cbe00640 1 -- 172.21.15.89:0/554333670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ead893090 msgr2=0x560ead89d2d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:43:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.673+0000 7f45cbe00640 1 --2- 172.21.15.89:0/554333670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ead893090 0x560ead89d2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.673+0000 7f45cbe00640 1 -- 172.21.15.89:0/554333670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead898a60 msgr2=0x560ead8a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:34.675 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.673+0000 7f45cbe00640 1 --2- 172.21.15.89:0/554333670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead898a60 0x560ead8a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.673+0000 7f45cbe00640 1 -- 172.21.15.89:0/554333670 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ead5d7620 con 0x560ead8939d0 2024-09-10T08:43:34.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.674+0000 7f45cbe00640 1 --2- 172.21.15.89:0/554333670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8939d0 0x560ead89d810 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f45c400ccd0 tx=0x7f45c4037690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=969d54368b94a14a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:34.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.674+0000 7f45b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:43:34.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.674+0000 7f45b6a00640 1 -- 172.21.15.89:0/554333670 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c403c040 con 0x560ead8939d0 2024-09-10T08:43:34.676 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.674+0000 7f45b6a00640 1 -- 172.21.15.89:0/554333670 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f45c40379f0 con 0x560ead8939d0 2024-09-10T08:43:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.677+0000 7f45d6836740 1 -- 172.21.15.89:0/554333670 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45c400aad0 con 0x560ead8939d0 2024-09-10T08:43:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.679+0000 7f45b6a00640 1 -- 172.21.15.89:0/554333670 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f45c4009b50 con 0x560ead8939d0 2024-09-10T08:43:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.681+0000 7f45d6836740 10 client.26072 fetch_fsmap learned FSMap version 28 2024-09-10T08:43:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.681+0000 7f45d6836740 1 -- 172.21.15.89:0/554333670 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560ead8a7650 con 0x560ead8939d0 2024-09-10T08:43:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.683+0000 7f45b6a00640 1 -- 172.21.15.89:0/554333670 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c400aad0 con 0x560ead8939d0 2024-09-10T08:43:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.683+0000 7f45b6a00640 1 -- 172.21.15.89:0/554333670 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f45c404a030 con 0x560ead8939d0 2024-09-10T08:43:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.684+0000 7f45d6836740 10 client.26072 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:43:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.684+0000 7f45d6836740 10 client.26072 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:43:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.684+0000 7f45d6836740 20 client.26072 populate_metadata read hostname 'smithi089' 2024-09-10T08:43:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.684+0000 7f45d6836740 1 --2- 172.21.15.89:0/554333670 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560ead8a8a60 0x560ead8aae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.684+0000 7f45d6836740 4 client.26072 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:43:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.684+0000 7f45d6836740 1 -- 172.21.15.89:0/554333670 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ead805830 con 0x560ead8a8a60 2024-09-10T08:43:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.685+0000 7f45d1000640 1 --2- 172.21.15.89:0/554333670 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560ead8a8a60 0x560ead8aae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.691+0000 7f45b6a00640 10 client.26072 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.691+0000 7f45d1000640 1 --2- 172.21.15.89:0/554333670 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560ead8a8a60 0x560ead8aae90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31a1f493b8f8e358 server_cookie=5687e4550d20deed in_seq=0 out_seq=0 2024-09-10T08:43:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.696+0000 7f45b6a00640 1 -- 172.21.15.89:0/554333670 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ead805830 con 0x560ead8a8a60 2024-09-10T08:43:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.696+0000 7f45b6a00640 10 client.26072 handle_command_reply: tid=0 2024-09-10T08:43:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.696+0000 7f45d6836740 1 client.26072 shutdown 2024-09-10T08:43:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.698+0000 7f45d6836740 1 -- 172.21.15.89:0/554333670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8939d0 msgr2=0x560ead89d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.698+0000 7f45d6836740 1 --2- 172.21.15.89:0/554333670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8939d0 0x560ead89d810 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f45c400ccd0 tx=0x7f45c4037690 comp rx=0 tx=0).stop 2024-09-10T08:43:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.698+0000 7f45d6836740 1 -- 172.21.15.89:0/554333670 shutdown_connections 2024-09-10T08:43:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.698+0000 7f45d6836740 1 --2- 172.21.15.89:0/554333670 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560ead8a8a60 0x560ead8aae90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.698+0000 7f45d6836740 1 --2- 172.21.15.89:0/554333670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead898a60 0x560ead8a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.698+0000 7f45d6836740 1 --2- 172.21.15.89:0/554333670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8939d0 0x560ead89d810 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.698+0000 7f45d6836740 1 --2- 172.21.15.89:0/554333670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ead893090 0x560ead89d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.698+0000 7f45d6836740 1 -- 172.21.15.89:0/554333670 >> 172.21.15.89:0/554333670 conn(0x560ead88e700 msgr2=0x560ead88ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.698+0000 7f45b6a00640 0 client.26072 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.702+0000 7f45d6836740 1 -- 172.21.15.89:0/554333670 shutdown_connections 2024-09-10T08:43:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.702+0000 7f45d6836740 1 -- 172.21.15.89:0/554333670 wait complete. 2024-09-10T08:43:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.702+0000 7f45d6836740 20 client.26072 trim_cache size 0 max 16384 2024-09-10T08:43:34.820 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:34 smithi089 ceph-mon[30757]: pgmap v502: 129 pgs: 129 active+clean; 268 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.2 MiB/s wr, 696 op/s 2024-09-10T08:43:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.849+0000 7f45d6836740 1 Processor -- start 2024-09-10T08:43:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.855+0000 7f45d6836740 1 -- start start 2024-09-10T08:43:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.856+0000 7f45d6836740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ead893090 0x560ead89d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.856+0000 7f45d6836740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8939d0 0x560ead89d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.856+0000 7f45d6836740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead898a60 0x560ead8a4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.857+0000 7f45d1a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead898a60 0x560ead8a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.857+0000 7f45d1a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead898a60 0x560ead8a4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34668/0 (socket says 172.21.15.89:34668) 2024-09-10T08:43:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.857+0000 7f45d1a00640 1 -- 172.21.15.89:0/3271599703 learned_addr learned my addr 172.21.15.89:0/3271599703 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.857+0000 7f45d6836740 1 -- 172.21.15.89:0/3271599703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac012590 con 0x560ead898a60 2024-09-10T08:43:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.857+0000 7f45d6836740 1 -- 172.21.15.89:0/3271599703 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac00fe60 con 0x560ead8939d0 2024-09-10T08:43:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.857+0000 7f45d6836740 1 -- 172.21.15.89:0/3271599703 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac00dcb0 con 0x560ead893090 2024-09-10T08:43:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.857+0000 7f45d1a00640 1 -- 172.21.15.89:0/3271599703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ead893090 msgr2=0x560ead89d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.857+0000 7f45d1a00640 1 --2- 172.21.15.89:0/3271599703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ead893090 0x560ead89d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.857+0000 7f45d1a00640 1 -- 172.21.15.89:0/3271599703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8939d0 msgr2=0x560ead89d810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:43:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.857+0000 7f45d1a00640 1 --2- 172.21.15.89:0/3271599703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8939d0 0x560ead89d810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.857+0000 7f45d1a00640 1 -- 172.21.15.89:0/3271599703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ead893df0 con 0x560ead898a60 2024-09-10T08:43:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.858+0000 7f45d1a00640 1 --2- 172.21.15.89:0/3271599703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead898a60 0x560ead8a4810 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f45bc022450 tx=0x7f45bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41536b8e92284c4c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.863+0000 7f45c1600640 1 -- 172.21.15.89:0/3271599703 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc011020 con 0x560ead898a60 2024-09-10T08:43:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.863+0000 7f45c1600640 1 -- 172.21.15.89:0/3271599703 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45bc00db50 con 0x560ead898a60 2024-09-10T08:43:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.863+0000 7f45c1600640 1 -- 172.21.15.89:0/3271599703 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc021c20 con 0x560ead898a60 2024-09-10T08:43:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.867+0000 7f45d6836740 1 -- 172.21.15.89:0/3271599703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead898a60 msgr2=0x560ead8a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.867+0000 7f45d6836740 1 --2- 172.21.15.89:0/3271599703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead898a60 0x560ead8a4810 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f45bc022450 tx=0x7f45bc004730 comp rx=0 tx=0).stop 2024-09-10T08:43:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.868+0000 7f45d6836740 1 -- 172.21.15.89:0/3271599703 shutdown_connections 2024-09-10T08:43:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.868+0000 7f45d6836740 1 --2- 172.21.15.89:0/3271599703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead898a60 0x560ead8a4810 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.869 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.868+0000 7f45d6836740 1 --2- 172.21.15.89:0/3271599703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8939d0 0x560ead89d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.868+0000 7f45d6836740 1 --2- 172.21.15.89:0/3271599703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ead893090 0x560ead89d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.868+0000 7f45d6836740 1 -- 172.21.15.89:0/3271599703 >> 172.21.15.89:0/3271599703 conn(0x560ead88e700 msgr2=0x560ead894120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.875+0000 7f45d6836740 1 -- 172.21.15.89:0/3271599703 shutdown_connections 2024-09-10T08:43:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.875+0000 7f45d6836740 1 -- 172.21.15.89:0/3271599703 wait complete. 2024-09-10T08:43:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.875+0000 7f45d6836740 1 -- start start 2024-09-10T08:43:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.875+0000 7f45d6836740 1 Processor -- start 2024-09-10T08:43:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.876+0000 7f45d6836740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ead893090 0x560ead8a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.876+0000 7f45d6836740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead898a60 0x560ead8a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.876+0000 7f45d6836740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8a2f50 0x560ead8a33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.876+0000 7f45d6836740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac00e320 con 0x560ead898a60 2024-09-10T08:43:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.876+0000 7f45d6836740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac00f790 con 0x560ead8a2f50 2024-09-10T08:43:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.876+0000 7f45d6836740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac00e9f0 con 0x560ead893090 2024-09-10T08:43:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.876+0000 7f45d6836740 10 client.0 _ll_register_callbacks cb 0x7f45c403a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:43:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.877+0000 7f45cbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead898a60 0x560ead8a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.877+0000 7f45cbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead898a60 0x560ead8a1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34676/0 (socket says 172.21.15.89:34676) 2024-09-10T08:43:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.877+0000 7f45cbe00640 1 -- 172.21.15.89:0/787764597 learned_addr learned my addr 172.21.15.89:0/787764597 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.878+0000 7f45cbe00640 1 -- 172.21.15.89:0/787764597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ead893090 msgr2=0x560ead8a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.878+0000 7f45cbe00640 1 --2- 172.21.15.89:0/787764597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ead893090 0x560ead8a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.878+0000 7f45cbe00640 1 -- 172.21.15.89:0/787764597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8a2f50 msgr2=0x560ead8a33b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:43:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.878+0000 7f45cbe00640 1 --2- 172.21.15.89:0/787764597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8a2f50 0x560ead8a33b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.878+0000 7f45cbe00640 1 -- 172.21.15.89:0/787764597 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ead8a3940 con 0x560ead898a60 2024-09-10T08:43:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.878+0000 7f45cbe00640 1 --2- 172.21.15.89:0/787764597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead898a60 0x560ead8a1f40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009d10 tx=0x7f45c4042870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f9ac8b67779c0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.883+0000 7f45b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:43:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.883+0000 7f45b6a00640 1 -- 172.21.15.89:0/787764597 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c403c070 con 0x560ead898a60 2024-09-10T08:43:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.883+0000 7f45b6a00640 1 -- 172.21.15.89:0/787764597 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f45c4042c80 con 0x560ead898a60 2024-09-10T08:43:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.884+0000 7f45d6836740 1 -- 172.21.15.89:0/787764597 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45bc00db50 con 0x560ead898a60 2024-09-10T08:43:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.884+0000 7f45b6a00640 1 -- 172.21.15.89:0/787764597 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f45c400aad0 con 0x560ead898a60 2024-09-10T08:43:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.885+0000 7f45d6836740 10 client.16008 fetch_fsmap learned FSMap version 28 2024-09-10T08:43:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.885+0000 7f45d6836740 1 -- 172.21.15.89:0/787764597 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560ead913770 con 0x560ead898a60 2024-09-10T08:43:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.885+0000 7f45b6a00640 1 -- 172.21.15.89:0/787764597 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45bc00db50 con 0x560ead898a60 2024-09-10T08:43:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.885+0000 7f45b6a00640 1 -- 172.21.15.89:0/787764597 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f45c404a070 con 0x560ead898a60 2024-09-10T08:43:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.885+0000 7f45d6836740 10 client.16008 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:43:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.885+0000 7f45d6836740 10 client.16008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:43:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.885+0000 7f45d6836740 20 client.16008 populate_metadata read hostname 'smithi089' 2024-09-10T08:43:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.886+0000 7f45d6836740 1 --2- 172.21.15.89:0/787764597 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560ead913fe0 0x560ead916400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.886+0000 7f45d6836740 4 client.16008 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:43:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.886+0000 7f45d6836740 1 -- 172.21.15.89:0/787764597 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ead7ffb80 con 0x560ead913fe0 2024-09-10T08:43:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.889+0000 7f45d1000640 1 --2- 172.21.15.89:0/787764597 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560ead913fe0 0x560ead916400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.897+0000 7f45d1000640 1 --2- 172.21.15.89:0/787764597 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560ead913fe0 0x560ead916400 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce4447db10853a5f server_cookie=822e181e37aca3bf in_seq=0 out_seq=0 2024-09-10T08:43:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:34.897+0000 7f45b6a00640 10 client.16008 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:34.985 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24554 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-09-10T08:43:34.986 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24554 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-09-10T08:43:35.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:34 smithi111 ceph-mon[35220]: pgmap v502: 129 pgs: 129 active+clean; 268 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.2 MiB/s wr, 696 op/s 2024-09-10T08:43:35.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:34 smithi149 ceph-mon[36167]: pgmap v502: 129 pgs: 129 active+clean; 268 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.2 MiB/s wr, 696 op/s 2024-09-10T08:43:35.564 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:37.001 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:36 smithi089 ceph-mon[30757]: pgmap v503: 129 pgs: 129 active+clean; 268 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.2 MiB/s wr, 690 op/s 2024-09-10T08:43:37.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:36 smithi149 ceph-mon[36167]: pgmap v503: 129 pgs: 129 active+clean; 268 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.2 MiB/s wr, 690 op/s 2024-09-10T08:43:37.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:36 smithi111 ceph-mon[35220]: pgmap v503: 129 pgs: 129 active+clean; 268 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.2 MiB/s wr, 690 op/s 2024-09-10T08:43:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.560+0000 7f1513000640 1 -- 172.21.15.89:0/4177076976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f150c1047e0 msgr2=0x7f150c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.560+0000 7f1513000640 1 --2- 172.21.15.89:0/4177076976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f150c1047e0 0x7f150c108830 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1500009900 tx=0x7f150002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.560+0000 7f1513000640 1 -- 172.21.15.89:0/4177076976 shutdown_connections 2024-09-10T08:43:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.560+0000 7f1513000640 1 --2- 172.21.15.89:0/4177076976 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f150c108f60 0x7f150c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.560+0000 7f1513000640 1 --2- 172.21.15.89:0/4177076976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f150c1047e0 0x7f150c108830 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.560+0000 7f1513000640 1 --2- 172.21.15.89:0/4177076976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f150c103e30 0x7f150c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.560+0000 7f1513000640 1 -- 172.21.15.89:0/4177076976 >> 172.21.15.89:0/4177076976 conn(0x7f150c0ff910 msgr2=0x7f150c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.560+0000 7f1513000640 1 -- 172.21.15.89:0/4177076976 shutdown_connections 2024-09-10T08:43:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.560+0000 7f1513000640 1 -- 172.21.15.89:0/4177076976 wait complete. 2024-09-10T08:43:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.562+0000 7f1513000640 1 Processor -- start 2024-09-10T08:43:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.564+0000 7f1513000640 1 -- start start 2024-09-10T08:43:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.564+0000 7f1513000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f150c103e30 0x7f150c110930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.564+0000 7f1513000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f150c1047e0 0x7f150c10d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:37.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.564+0000 7f1513000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f150c108f60 0x7f150c10df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.564+0000 7f1513000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f150c114030 con 0x7f150c103e30 2024-09-10T08:43:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.564+0000 7f1513000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f150c113eb0 con 0x7f150c108f60 2024-09-10T08:43:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.564+0000 7f1513000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1141b0 con 0x7f150c1047e0 2024-09-10T08:43:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.565+0000 7f1511200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f150c1047e0 0x7f150c10d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.565+0000 7f1511200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f150c1047e0 0x7f150c10d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44318/0 (socket says 172.21.15.89:44318) 2024-09-10T08:43:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.565+0000 7f1511200640 1 -- 172.21.15.89:0/417148535 learned_addr learned my addr 172.21.15.89:0/417148535 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.566+0000 7f1511200640 1 -- 172.21.15.89:0/417148535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f150c108f60 msgr2=0x7f150c10df20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:43:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.566+0000 7f1511200640 1 --2- 172.21.15.89:0/417148535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f150c108f60 0x7f150c10df20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.566+0000 7f1511200640 1 -- 172.21.15.89:0/417148535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f150c103e30 msgr2=0x7f150c110930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:43:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.566+0000 7f1511200640 1 --2- 172.21.15.89:0/417148535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f150c103e30 0x7f150c110930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.566+0000 7f1511200640 1 -- 172.21.15.89:0/417148535 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150c10e790 con 0x7f150c1047e0 2024-09-10T08:43:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.566+0000 7f1511200640 1 --2- 172.21.15.89:0/417148535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f150c1047e0 0x7f150c10d9e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1500006010 tx=0x7f1500003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.567+0000 7f150a000640 1 -- 172.21.15.89:0/417148535 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150003f070 con 0x7f150c1047e0 2024-09-10T08:43:37.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.567+0000 7f150a000640 1 -- 172.21.15.89:0/417148535 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1500031c90 con 0x7f150c1047e0 2024-09-10T08:43:37.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.567+0000 7f150a000640 1 -- 172.21.15.89:0/417148535 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150003f070 con 0x7f150c1047e0 2024-09-10T08:43:37.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.567+0000 7f1513000640 1 -- 172.21.15.89:0/417148535 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150c1a25c0 con 0x7f150c1047e0 2024-09-10T08:43:37.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.567+0000 7f1513000640 1 -- 172.21.15.89:0/417148535 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f150c1a2960 con 0x7f150c1047e0 2024-09-10T08:43:37.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.573+0000 7f150a000640 1 -- 172.21.15.89:0/417148535 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1500042070 con 0x7f150c1047e0 2024-09-10T08:43:37.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.581+0000 7f15170d6740 1 Processor -- start 2024-09-10T08:43:37.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.584+0000 7f150a000640 1 --2- 172.21.15.89:0/417148535 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14f40a2e00 0x7f14f40a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:37.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.588+0000 7f1512600640 1 --2- 172.21.15.89:0/417148535 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14f40a2e00 0x7f14f40a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:37.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.590+0000 7f15170d6740 1 -- start start 2024-09-10T08:43:37.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.590+0000 7f15170d6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde121cb0 0x562cde122090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:37.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.590+0000 7f15170d6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde1225d0 0x562cde12c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:37.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.590+0000 7f15170d6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cde12ce90 0x562cde12f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:37.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.590+0000 7f15170d6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562cde088290 con 0x562cde12ce90 2024-09-10T08:43:37.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.590+0000 7f15170d6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562cde02fb40 con 0x562cde121cb0 2024-09-10T08:43:37.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.590+0000 7f15170d6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562cde084250 con 0x562cde1225d0 2024-09-10T08:43:37.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.591+0000 7f150a000640 1 -- 172.21.15.89:0/417148535 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f15000ef5a0 con 0x7f150c1047e0 2024-09-10T08:43:37.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.591+0000 7f1511c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde121cb0 0x562cde122090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:37.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.591+0000 7f1512600640 1 --2- 172.21.15.89:0/417148535 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14f40a2e00 0x7f14f40a52c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f150c108870 tx=0x7f14f8005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:37.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.591+0000 7f1511c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde121cb0 0x562cde122090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57118/0 (socket says 172.21.15.89:57118) 2024-09-10T08:43:37.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.591+0000 7f1511c00640 1 -- 172.21.15.89:0/3195772505 learned_addr learned my addr 172.21.15.89:0/3195772505 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:37.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.592+0000 7f1512600640 1 --2- 172.21.15.89:0/3195772505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cde12ce90 0x562cde12f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:37.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.592+0000 7f1511200640 1 --2- 172.21.15.89:0/3195772505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde1225d0 0x562cde12c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:37.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.596+0000 7f1511c00640 1 -- 172.21.15.89:0/3195772505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde1225d0 msgr2=0x562cde12c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:37.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.596+0000 7f1511c00640 1 --2- 172.21.15.89:0/3195772505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde1225d0 0x562cde12c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.596+0000 7f1511c00640 1 -- 172.21.15.89:0/3195772505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cde12ce90 msgr2=0x562cde12f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:37.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.596+0000 7f1511c00640 1 --2- 172.21.15.89:0/3195772505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cde12ce90 0x562cde12f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.596+0000 7f1511c00640 1 -- 172.21.15.89:0/3195772505 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cdddf96a0 con 0x562cde121cb0 2024-09-10T08:43:37.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.597+0000 7f1511c00640 1 --2- 172.21.15.89:0/3195772505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde121cb0 0x562cde122090 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1504009d80 tx=0x7f150400b420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acf8a1abba9c3cb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:37.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.598+0000 7f14fea00640 1 -- 172.21.15.89:0/3195772505 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150400bd40 con 0x562cde121cb0 2024-09-10T08:43:37.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.598+0000 7f14fea00640 1 -- 172.21.15.89:0/3195772505 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1504017070 con 0x562cde121cb0 2024-09-10T08:43:37.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.598+0000 7f14fea00640 1 -- 172.21.15.89:0/3195772505 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504011760 con 0x562cde121cb0 2024-09-10T08:43:37.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.600+0000 7f15170d6740 1 -- 172.21.15.89:0/3195772505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde121cb0 msgr2=0x562cde122090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:37.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.600+0000 7f15170d6740 1 --2- 172.21.15.89:0/3195772505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde121cb0 0x562cde122090 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1504009d80 tx=0x7f150400b420 comp rx=0 tx=0).stop 2024-09-10T08:43:37.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.600+0000 7f15170d6740 1 -- 172.21.15.89:0/3195772505 shutdown_connections 2024-09-10T08:43:37.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.600+0000 7f15170d6740 1 --2- 172.21.15.89:0/3195772505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cde12ce90 0x562cde12f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.600+0000 7f15170d6740 1 --2- 172.21.15.89:0/3195772505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde1225d0 0x562cde12c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.600+0000 7f15170d6740 1 --2- 172.21.15.89:0/3195772505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde121cb0 0x562cde122090 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.600+0000 7f15170d6740 1 -- 172.21.15.89:0/3195772505 >> 172.21.15.89:0/3195772505 conn(0x562cde11e240 msgr2=0x562cde11e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:37.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.600+0000 7f15170d6740 1 -- 172.21.15.89:0/3195772505 shutdown_connections 2024-09-10T08:43:37.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.600+0000 7f15170d6740 1 -- 172.21.15.89:0/3195772505 wait complete. 2024-09-10T08:43:37.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.602+0000 7f15170d6740 1 -- start start 2024-09-10T08:43:37.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.603+0000 7f15170d6740 1 Processor -- start 2024-09-10T08:43:37.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.604+0000 7f15170d6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde121cb0 0x562cde12c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:37.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.604+0000 7f15170d6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde1225d0 0x562cde1299d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:37.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.604+0000 7f15170d6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cde12ce90 0x562cde129f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:37.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.606+0000 7f15170d6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562cde12fb00 con 0x562cde12ce90 2024-09-10T08:43:37.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.606+0000 7f1511c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde121cb0 0x562cde12c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:37.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.606+0000 7f15170d6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562cde042fa0 con 0x562cde1225d0 2024-09-10T08:43:37.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.606+0000 7f15170d6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562cde12fc80 con 0x562cde121cb0 2024-09-10T08:43:37.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.606+0000 7f1511200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde1225d0 0x562cde1299d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:37.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.606+0000 7f1511200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde1225d0 0x562cde1299d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57120/0 (socket says 172.21.15.89:57120) 2024-09-10T08:43:37.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.606+0000 7f1511200640 1 -- 172.21.15.89:0/2127930581 learned_addr learned my addr 172.21.15.89:0/2127930581 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:37.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.606+0000 7f1512600640 1 --2- 172.21.15.89:0/2127930581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cde12ce90 0x562cde129f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:37.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.607+0000 7f1511200640 1 -- 172.21.15.89:0/2127930581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde121cb0 msgr2=0x562cde12c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:37.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.607+0000 7f1511200640 1 --2- 172.21.15.89:0/2127930581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde121cb0 0x562cde12c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.607+0000 7f1511200640 1 -- 172.21.15.89:0/2127930581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cde12ce90 msgr2=0x562cde129f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:37.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.607+0000 7f1511200640 1 --2- 172.21.15.89:0/2127930581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cde12ce90 0x562cde129f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.607+0000 7f1511200640 1 -- 172.21.15.89:0/2127930581 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cde12a6f0 con 0x562cde1225d0 2024-09-10T08:43:37.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.607+0000 7f1511200640 1 --2- 172.21.15.89:0/2127930581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde1225d0 0x562cde1299d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f15000eccd0 tx=0x7f1500031f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d45ff49efded7d4b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:37.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.607+0000 7f14f3400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:43:37.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.608+0000 7f14f3400640 1 -- 172.21.15.89:0/2127930581 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500049040 con 0x562cde1225d0 2024-09-10T08:43:37.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.608+0000 7f14f3400640 1 -- 172.21.15.89:0/2127930581 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f15000317b0 con 0x562cde1225d0 2024-09-10T08:43:37.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.608+0000 7f14f3400640 10 client.0 inactive, discarding osd_map(96..96 src has 1..96) 2024-09-10T08:43:37.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.613+0000 7f1511c00640 1 --2- 172.21.15.89:0/2127930581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde121cb0 0x562cde12c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:43:37.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.613+0000 7f1512600640 1 --2- 172.21.15.89:0/2127930581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cde12ce90 0x562cde129f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:43:37.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.614+0000 7f15170d6740 10 client.0 _ll_register_callbacks cb 0x562cddd96ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:43:37.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.614+0000 7f15170d6740 1 -- 172.21.15.89:0/2127930581 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1504017070 con 0x562cde1225d0 2024-09-10T08:43:37.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.617+0000 7f14f3400640 1 -- 172.21.15.89:0/2127930581 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f15000bfde0 con 0x562cde1225d0 2024-09-10T08:43:37.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.620+0000 7f15170d6740 10 client.26078 fetch_fsmap learned FSMap version 28 2024-09-10T08:43:37.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.620+0000 7f15170d6740 1 -- 172.21.15.89:0/2127930581 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({fsmap=28}) -- 0x562cde13b510 con 0x562cde1225d0 2024-09-10T08:43:37.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.625+0000 7f14f3400640 1 -- 172.21.15.89:0/2127930581 <== mon.2 v2:172.21.15.111:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f15000b80a0 con 0x562cde1225d0 2024-09-10T08:43:37.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.625+0000 7f15170d6740 10 client.26078 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:43:37.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.625+0000 7f15170d6740 10 client.26078 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:43:37.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.625+0000 7f15170d6740 20 client.26078 populate_metadata read hostname 'smithi089' 2024-09-10T08:43:37.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.625+0000 7f15170d6740 1 --2- 172.21.15.89:0/2127930581 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x562cde13cb40 0x562cde13f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:37.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.625+0000 7f15170d6740 4 client.26078 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:43:37.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.625+0000 7f15170d6740 1 -- 172.21.15.89:0/2127930581 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cde095800 con 0x562cde13cb40 2024-09-10T08:43:37.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.626+0000 7f1511c00640 1 --2- 172.21.15.89:0/2127930581 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x562cde13cb40 0x562cde13f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:37.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.627+0000 7f1511c00640 1 --2- 172.21.15.89:0/2127930581 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x562cde13cb40 0x562cde13f000 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=928a260e0d6a8be7 server_cookie=d8dbc64fa605b20f in_seq=0 out_seq=0 2024-09-10T08:43:37.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.627+0000 7f14f3400640 10 client.26078 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:37.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.644+0000 7f14f3400640 1 -- 172.21.15.89:0/2127930581 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562cde095800 con 0x562cde13cb40 2024-09-10T08:43:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.644+0000 7f14f3400640 10 client.26078 handle_command_reply: tid=0 2024-09-10T08:43:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.644+0000 7f15170d6740 1 client.26078 shutdown 2024-09-10T08:43:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.644+0000 7f15170d6740 1 -- 172.21.15.89:0/2127930581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde1225d0 msgr2=0x562cde1299d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.644+0000 7f15170d6740 1 --2- 172.21.15.89:0/2127930581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde1225d0 0x562cde1299d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f15000eccd0 tx=0x7f1500031f80 comp rx=0 tx=0).stop 2024-09-10T08:43:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.644+0000 7f15170d6740 1 -- 172.21.15.89:0/2127930581 shutdown_connections 2024-09-10T08:43:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.644+0000 7f15170d6740 1 --2- 172.21.15.89:0/2127930581 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x562cde13cb40 0x562cde13f000 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.644+0000 7f15170d6740 1 --2- 172.21.15.89:0/2127930581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cde12ce90 0x562cde129f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.644+0000 7f15170d6740 1 --2- 172.21.15.89:0/2127930581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde1225d0 0x562cde1299d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.644+0000 7f14f3400640 0 client.26078 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.644+0000 7f15170d6740 1 --2- 172.21.15.89:0/2127930581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde121cb0 0x562cde12c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.644+0000 7f15170d6740 1 -- 172.21.15.89:0/2127930581 >> 172.21.15.89:0/2127930581 conn(0x562cde11e240 msgr2=0x562cde123f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.645+0000 7f15170d6740 1 -- 172.21.15.89:0/2127930581 shutdown_connections 2024-09-10T08:43:37.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.645+0000 7f15170d6740 1 -- 172.21.15.89:0/2127930581 wait complete. 2024-09-10T08:43:37.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.645+0000 7f15170d6740 20 client.26078 trim_cache size 0 max 16384 2024-09-10T08:43:37.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.683+0000 7f15170d6740 1 Processor -- start 2024-09-10T08:43:37.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.683+0000 7f15170d6740 1 -- start start 2024-09-10T08:43:37.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.683+0000 7f15170d6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cde121cb0 0x562cde12b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:37.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.683+0000 7f15170d6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde1225d0 0x562cde138460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:37.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.684+0000 7f15170d6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde12ce90 0x562cde1389a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:37.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.684+0000 7f15170d6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1af210 con 0x562cde121cb0 2024-09-10T08:43:37.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.684+0000 7f15170d6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1acae0 con 0x562cde1225d0 2024-09-10T08:43:37.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.684+0000 7f15170d6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1aa930 con 0x562cde12ce90 2024-09-10T08:43:37.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.684+0000 7f1511200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde1225d0 0x562cde138460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:37.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.684+0000 7f1511200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde1225d0 0x562cde138460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57132/0 (socket says 172.21.15.89:57132) 2024-09-10T08:43:37.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.684+0000 7f1511200640 1 -- 172.21.15.89:0/2823665107 learned_addr learned my addr 172.21.15.89:0/2823665107 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:37.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.684+0000 7f1512600640 1 --2- 172.21.15.89:0/2823665107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde12ce90 0x562cde1389a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:37.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.685+0000 7f1511200640 1 -- 172.21.15.89:0/2823665107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde12ce90 msgr2=0x562cde1389a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:37.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.685+0000 7f1511200640 1 --2- 172.21.15.89:0/2823665107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde12ce90 0x562cde1389a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.685+0000 7f1511200640 1 -- 172.21.15.89:0/2823665107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cde121cb0 msgr2=0x562cde12b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:37.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.685+0000 7f1511200640 1 --2- 172.21.15.89:0/2823665107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cde121cb0 0x562cde12b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.685+0000 7f1511200640 1 -- 172.21.15.89:0/2823665107 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cde11f3b0 con 0x562cde1225d0 2024-09-10T08:43:37.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.685+0000 7f1511200640 1 --2- 172.21.15.89:0/2823665107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde1225d0 0x562cde138460 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f15000ed040 tx=0x7f15000be000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ba325213929a0a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:37.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.685+0000 7f14fe000640 1 -- 172.21.15.89:0/2823665107 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500049070 con 0x562cde1225d0 2024-09-10T08:43:37.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.686+0000 7f14fe000640 1 -- 172.21.15.89:0/2823665107 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1504017070 con 0x562cde1225d0 2024-09-10T08:43:37.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.686+0000 7f14fe000640 1 -- 172.21.15.89:0/2823665107 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500049070 con 0x562cde1225d0 2024-09-10T08:43:37.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.687+0000 7f15170d6740 1 -- 172.21.15.89:0/2823665107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde1225d0 msgr2=0x562cde138460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:37.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.687+0000 7f15170d6740 1 --2- 172.21.15.89:0/2823665107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde1225d0 0x562cde138460 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f15000ed040 tx=0x7f15000be000 comp rx=0 tx=0).stop 2024-09-10T08:43:37.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.692+0000 7f15170d6740 1 -- 172.21.15.89:0/2823665107 shutdown_connections 2024-09-10T08:43:37.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.692+0000 7f15170d6740 1 --2- 172.21.15.89:0/2823665107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde12ce90 0x562cde1389a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.692+0000 7f15170d6740 1 --2- 172.21.15.89:0/2823665107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde1225d0 0x562cde138460 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.692+0000 7f15170d6740 1 --2- 172.21.15.89:0/2823665107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cde121cb0 0x562cde12b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.692+0000 7f15170d6740 1 -- 172.21.15.89:0/2823665107 >> 172.21.15.89:0/2823665107 conn(0x562cde11e240 msgr2=0x562cde137d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:37.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.692+0000 7f15170d6740 1 -- 172.21.15.89:0/2823665107 shutdown_connections 2024-09-10T08:43:37.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.692+0000 7f15170d6740 1 -- 172.21.15.89:0/2823665107 wait complete. 2024-09-10T08:43:37.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.692+0000 7f15170d6740 1 -- start start 2024-09-10T08:43:37.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.693+0000 7f15170d6740 1 Processor -- start 2024-09-10T08:43:37.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.694+0000 7f15170d6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cde121cb0 0x562cde1354c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:37.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.694+0000 7f15170d6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde1225d0 0x562cde133870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:37.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.695+0000 7f15170d6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde12ce90 0x562cde133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:37.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.695+0000 7f1511200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde1225d0 0x562cde133870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:37.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.695+0000 7f1511200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde1225d0 0x562cde133870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44370/0 (socket says 172.21.15.89:44370) 2024-09-10T08:43:37.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.695+0000 7f15170d6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1aafa0 con 0x562cde121cb0 2024-09-10T08:43:37.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.695+0000 7f15170d6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1ac410 con 0x562cde12ce90 2024-09-10T08:43:37.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.695+0000 7f1511200640 1 -- 172.21.15.89:0/1230696247 learned_addr learned my addr 172.21.15.89:0/1230696247 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:37.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.695+0000 7f15170d6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f150c1ab670 con 0x562cde1225d0 2024-09-10T08:43:37.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.695+0000 7f1512600640 1 --2- 172.21.15.89:0/1230696247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde12ce90 0x562cde133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:37.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.695+0000 7f15170d6740 10 client.0 _ll_register_callbacks cb 0x7f15000bb800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:43:37.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.696+0000 7f1511c00640 1 --2- 172.21.15.89:0/1230696247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cde121cb0 0x562cde1354c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:37.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.696+0000 7f1512600640 1 -- 172.21.15.89:0/1230696247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde1225d0 msgr2=0x562cde133870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:37.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.696+0000 7f1512600640 1 --2- 172.21.15.89:0/1230696247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde1225d0 0x562cde133870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.696+0000 7f1512600640 1 -- 172.21.15.89:0/1230696247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cde121cb0 msgr2=0x562cde1354c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:37.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.696+0000 7f1512600640 1 --2- 172.21.15.89:0/1230696247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cde121cb0 0x562cde1354c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:37.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.696+0000 7f1512600640 1 -- 172.21.15.89:0/1230696247 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cde1344c0 con 0x562cde12ce90 2024-09-10T08:43:37.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.700+0000 7f1512600640 1 --2- 172.21.15.89:0/1230696247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde12ce90 0x562cde133db0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x562cde12be20 tx=0x7f14f8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3795794f8e7555b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:37.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.700+0000 7f14f3400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:43:37.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.701+0000 7f14f3400640 1 -- 172.21.15.89:0/1230696247 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f8005220 con 0x562cde12ce90 2024-09-10T08:43:37.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.701+0000 7f14f3400640 1 -- 172.21.15.89:0/1230696247 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f14f80053c0 con 0x562cde12ce90 2024-09-10T08:43:37.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.701+0000 7f15170d6740 1 -- 172.21.15.89:0/1230696247 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1504017070 con 0x562cde12ce90 2024-09-10T08:43:37.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.703+0000 7f14f3400640 1 -- 172.21.15.89:0/1230696247 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14f80055c0 con 0x562cde12ce90 2024-09-10T08:43:37.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.704+0000 7f15170d6740 10 client.26084 fetch_fsmap learned FSMap version 28 2024-09-10T08:43:37.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.704+0000 7f15170d6740 1 -- 172.21.15.89:0/1230696247 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562cde1a88e0 con 0x562cde12ce90 2024-09-10T08:43:37.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.704+0000 7f14f3400640 1 -- 172.21.15.89:0/1230696247 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1504017070 con 0x562cde12ce90 2024-09-10T08:43:37.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.704+0000 7f14f3400640 1 -- 172.21.15.89:0/1230696247 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f14f801d040 con 0x562cde12ce90 2024-09-10T08:43:37.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.705+0000 7f15170d6740 10 client.26084 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:43:37.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.705+0000 7f15170d6740 10 client.26084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:43:37.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.705+0000 7f15170d6740 20 client.26084 populate_metadata read hostname 'smithi089' 2024-09-10T08:43:37.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.705+0000 7f15170d6740 1 --2- 172.21.15.89:0/1230696247 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x562cde1a9080 0x562cde1ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:37.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.705+0000 7f15170d6740 4 client.26084 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:43:37.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.705+0000 7f15170d6740 1 -- 172.21.15.89:0/1230696247 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562cde12a2a0 con 0x562cde1a9080 2024-09-10T08:43:37.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.710+0000 7f1511c00640 1 --2- 172.21.15.89:0/1230696247 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x562cde1a9080 0x562cde1ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:37.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.711+0000 7f1511c00640 1 --2- 172.21.15.89:0/1230696247 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x562cde1a9080 0x562cde1ab4a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8c9a68e7a88d19c server_cookie=c62b3dd3e926b70b in_seq=0 out_seq=0 2024-09-10T08:43:37.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:37.711+0000 7f14f3400640 10 client.26084 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:38.041 INFO:tasks.workunit.client.0.smithi089.stdout: 85 500090 475 351066 399 274440 1540 2024-09-10T08:43:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.072+0000 7f14f3400640 1 -- 172.21.15.89:0/1230696247 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562cde12a2a0 con 0x562cde1a9080 2024-09-10T08:43:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.072+0000 7f14f3400640 10 client.26084 handle_command_reply: tid=0 2024-09-10T08:43:38.074 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.073+0000 7f15170d6740 1 client.26084 shutdown 2024-09-10T08:43:38.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.073+0000 7f15170d6740 1 -- 172.21.15.89:0/1230696247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde12ce90 msgr2=0x562cde133db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:38.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.073+0000 7f15170d6740 1 --2- 172.21.15.89:0/1230696247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde12ce90 0x562cde133db0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x562cde12be20 tx=0x7f14f8004730 comp rx=0 tx=0).stop 2024-09-10T08:43:38.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.073+0000 7f15170d6740 1 -- 172.21.15.89:0/1230696247 shutdown_connections 2024-09-10T08:43:38.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.073+0000 7f15170d6740 1 --2- 172.21.15.89:0/1230696247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562cde12ce90 0x562cde133db0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:38.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.073+0000 7f15170d6740 1 --2- 172.21.15.89:0/1230696247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cde1225d0 0x562cde133870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:38.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.073+0000 7f15170d6740 1 --2- 172.21.15.89:0/1230696247 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x562cde1a9080 0x562cde1ab4a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:38.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.073+0000 7f15170d6740 1 --2- 172.21.15.89:0/1230696247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cde121cb0 0x562cde1354c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:38.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.073+0000 7f15170d6740 1 -- 172.21.15.89:0/1230696247 >> 172.21.15.89:0/1230696247 conn(0x562cde11e240 msgr2=0x562cde137d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:38.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.073+0000 7f14f3400640 0 client.26084 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:38.076 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.074+0000 7f15170d6740 1 -- 172.21.15.89:0/1230696247 shutdown_connections 2024-09-10T08:43:38.077 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:43:38.077 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2024-09-10T08:43:38.077 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "c243eb93-7473-40b5-b951-e020536a2130", 2024-09-10T08:43:38.077 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2024-09-10T08:43:38.077 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:43:38.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.074+0000 7f15170d6740 1 -- 172.21.15.89:0/1230696247 wait complete. 2024-09-10T08:43:38.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.074+0000 7f15170d6740 20 client.26084 trim_cache size 0 max 16384 2024-09-10T08:43:38.078 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.076+0000 7f1513000640 1 -- 172.21.15.89:0/417148535 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14f40a2e00 msgr2=0x7f14f40a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:38.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.076+0000 7f1513000640 1 --2- 172.21.15.89:0/417148535 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14f40a2e00 0x7f14f40a52c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f150c108870 tx=0x7f14f8005be0 comp rx=0 tx=0).stop 2024-09-10T08:43:38.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.076+0000 7f1513000640 1 -- 172.21.15.89:0/417148535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f150c1047e0 msgr2=0x7f150c10d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:38.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.076+0000 7f1513000640 1 --2- 172.21.15.89:0/417148535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f150c1047e0 0x7f150c10d9e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1500006010 tx=0x7f1500003ba0 comp rx=0 tx=0).stop 2024-09-10T08:43:38.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.076+0000 7f1513000640 1 -- 172.21.15.89:0/417148535 shutdown_connections 2024-09-10T08:43:38.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.076+0000 7f1513000640 1 --2- 172.21.15.89:0/417148535 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f14f40a2e00 0x7f14f40a52c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:38.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.076+0000 7f1513000640 1 --2- 172.21.15.89:0/417148535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f150c108f60 0x7f150c10df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:38.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.076+0000 7f1513000640 1 --2- 172.21.15.89:0/417148535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f150c1047e0 0x7f150c10d9e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:38.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.076+0000 7f1513000640 1 --2- 172.21.15.89:0/417148535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f150c103e30 0x7f150c110930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:38.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.076+0000 7f1513000640 1 -- 172.21.15.89:0/417148535 >> 172.21.15.89:0/417148535 conn(0x7f150c0ff910 msgr2=0x7f150c1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:38.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.077+0000 7f1513000640 1 -- 172.21.15.89:0/417148535 shutdown_connections 2024-09-10T08:43:38.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:38.077+0000 7f1513000640 1 -- 172.21.15.89:0/417148535 wait complete. 2024-09-10T08:43:39.061 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:38 smithi089 ceph-mon[30757]: pgmap v504: 129 pgs: 129 active+clean; 270 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 481 op/s 2024-09-10T08:43:39.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:38 smithi149 ceph-mon[36167]: pgmap v504: 129 pgs: 129 active+clean; 270 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 481 op/s 2024-09-10T08:43:39.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:38 smithi149 ceph-mon[36167]: scrub queued for path: / 2024-09-10T08:43:39.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:38 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:43:39.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:38 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:43:39.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:38 smithi111 ceph-mon[35220]: pgmap v504: 129 pgs: 129 active+clean; 270 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 481 op/s 2024-09-10T08:43:39.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:38 smithi111 ceph-mon[35220]: scrub queued for path: / 2024-09-10T08:43:39.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:38 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:43:39.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:38 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:43:39.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:39 smithi089 ceph-mon[30757]: scrub queued for path: / 2024-09-10T08:43:39.554 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:39 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:43:39.554 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:39 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:43:40.985 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:43:40.997 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:41.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:40 smithi089 ceph-mon[30757]: pgmap v505: 129 pgs: 129 active+clean; 272 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 493 op/s 2024-09-10T08:43:41.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:40 smithi149 ceph-mon[36167]: pgmap v505: 129 pgs: 129 active+clean; 272 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 493 op/s 2024-09-10T08:43:41.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:40 smithi111 ceph-mon[35220]: pgmap v505: 129 pgs: 129 active+clean; 272 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 493 op/s 2024-09-10T08:43:42.471 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.470+0000 7f45b6a00640 1 -- 172.21.15.89:0/787764597 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560ead7ffb80 con 0x560ead913fe0 2024-09-10T08:43:42.471 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.470+0000 7f45b6a00640 10 client.16008 handle_command_reply: tid=0 2024-09-10T08:43:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.483+0000 7f45d6836740 1 client.16008 shutdown 2024-09-10T08:43:42.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.218+0000 7f7411800640 1 -- 172.21.15.89:0/1708663184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f740c077040 msgr2=0x7f740c1124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:42.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.218+0000 7f7411800640 1 --2- 172.21.15.89:0/1708663184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f740c077040 0x7f740c1124f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009860 tx=0x7f73fc02a360 comp rx=0 tx=0).stop 2024-09-10T08:43:42.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.221+0000 7f7411800640 1 -- 172.21.15.89:0/1708663184 shutdown_connections 2024-09-10T08:43:42.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.221+0000 7f7411800640 1 --2- 172.21.15.89:0/1708663184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f740c112a30 0x7f740c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.221+0000 7f7411800640 1 --2- 172.21.15.89:0/1708663184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f740c077040 0x7f740c1124f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.221+0000 7f7411800640 1 --2- 172.21.15.89:0/1708663184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f740c076720 0x7f740c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.221+0000 7f7411800640 1 -- 172.21.15.89:0/1708663184 >> 172.21.15.89:0/1708663184 conn(0x7f740c0ffa90 msgr2=0x7f740c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:42.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.221+0000 7f7411800640 1 -- 172.21.15.89:0/1708663184 shutdown_connections 2024-09-10T08:43:42.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.221+0000 7f7411800640 1 -- 172.21.15.89:0/1708663184 wait complete. 2024-09-10T08:43:42.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.223+0000 7f7411800640 1 Processor -- start 2024-09-10T08:43:42.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.223+0000 7f7411800640 1 -- start start 2024-09-10T08:43:42.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.227+0000 7f7411800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f740c076720 0x7f740c073ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:42.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.227+0000 7f7411800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f740c077040 0x7f740c06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:42.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.228+0000 7f7411800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f740c112a30 0x7f740c06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:42.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.228+0000 7f740a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f740c077040 0x7f740c06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:42.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.228+0000 7f740a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f740c077040 0x7f740c06dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44388/0 (socket says 172.21.15.89:44388) 2024-09-10T08:43:42.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.228+0000 7f740a000640 1 -- 172.21.15.89:0/1817291195 learned_addr learned my addr 172.21.15.89:0/1817291195 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:42.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.228+0000 7f7411800640 1 -- 172.21.15.89:0/1817291195 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f740c06c660 con 0x7f740c076720 2024-09-10T08:43:42.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.228+0000 7f7411800640 1 -- 172.21.15.89:0/1817291195 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f740c06c4e0 con 0x7f740c112a30 2024-09-10T08:43:42.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.229+0000 7f7411800640 1 -- 172.21.15.89:0/1817291195 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f740c06c7e0 con 0x7f740c077040 2024-09-10T08:43:42.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.232+0000 7f740a000640 1 -- 172.21.15.89:0/1817291195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f740c112a30 msgr2=0x7f740c06e2a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:43:42.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.232+0000 7f740a000640 1 --2- 172.21.15.89:0/1817291195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f740c112a30 0x7f740c06e2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.232+0000 7f740a000640 1 -- 172.21.15.89:0/1817291195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f740c076720 msgr2=0x7f740c073ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:42.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.232+0000 7f740a000640 1 --2- 172.21.15.89:0/1817291195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f740c076720 0x7f740c073ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.232+0000 7f740a000640 1 -- 172.21.15.89:0/1817291195 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740c06eb30 con 0x7f740c077040 2024-09-10T08:43:42.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.232+0000 7f740a000640 1 --2- 172.21.15.89:0/1817291195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f740c077040 0x7f740c06dd60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f73fc035040 tx=0x7f73fc0327d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:42.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.233+0000 7f7403400640 1 -- 172.21.15.89:0/1817291195 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc037040 con 0x7f740c077040 2024-09-10T08:43:42.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.233+0000 7f7403400640 1 -- 172.21.15.89:0/1817291195 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73fc040070 con 0x7f740c077040 2024-09-10T08:43:42.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.233+0000 7f7411800640 1 -- 172.21.15.89:0/1817291195 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740c1b39e0 con 0x7f740c077040 2024-09-10T08:43:42.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.234+0000 7f7403400640 1 -- 172.21.15.89:0/1817291195 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc003900 con 0x7f740c077040 2024-09-10T08:43:42.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.234+0000 7f7411800640 1 -- 172.21.15.89:0/1817291195 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f740c1b3f20 con 0x7f740c077040 2024-09-10T08:43:42.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.246+0000 7f7403400640 1 -- 172.21.15.89:0/1817291195 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f73fc003aa0 con 0x7f740c077040 2024-09-10T08:43:42.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.246+0000 7f741441d740 1 Processor -- start 2024-09-10T08:43:42.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.246+0000 7f741441d740 1 -- start start 2024-09-10T08:43:42.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.247+0000 7f741441d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a455a4e0 0x5557a455a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:42.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.247+0000 7f741441d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455ae00 0x5557a4565180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:42.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.247+0000 7f741441d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a45656c0 0x5557a4567ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:42.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.247+0000 7f741441d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5557a44bf260 con 0x5557a45656c0 2024-09-10T08:43:42.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.247+0000 7f741441d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5557a4466500 con 0x5557a455a4e0 2024-09-10T08:43:42.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.247+0000 7f741441d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5557a44bb220 con 0x5557a455ae00 2024-09-10T08:43:42.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.248+0000 7f740b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a455a4e0 0x5557a455a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:42.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.248+0000 7f740b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a455a4e0 0x5557a455a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57158/0 (socket says 172.21.15.89:57158) 2024-09-10T08:43:42.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.248+0000 7f740b400640 1 -- 172.21.15.89:0/1809200031 learned_addr learned my addr 172.21.15.89:0/1809200031 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:42.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.248+0000 7f740a000640 1 --2- 172.21.15.89:0/1809200031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a45656c0 0x5557a4567ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:42.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.248+0000 7f740aa00640 1 --2- 172.21.15.89:0/1809200031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455ae00 0x5557a4565180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:42.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.248+0000 7f740aa00640 1 -- 172.21.15.89:0/1809200031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a455a4e0 msgr2=0x5557a455a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:42.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.248+0000 7f740aa00640 1 --2- 172.21.15.89:0/1809200031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a455a4e0 0x5557a455a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.248+0000 7f740aa00640 1 -- 172.21.15.89:0/1809200031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a45656c0 msgr2=0x5557a4567ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:42.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.248+0000 7f740aa00640 1 --2- 172.21.15.89:0/1809200031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a45656c0 0x5557a4567ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.249+0000 7f740aa00640 1 -- 172.21.15.89:0/1809200031 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557a4230ba0 con 0x5557a455ae00 2024-09-10T08:43:42.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.249+0000 7f740aa00640 1 --2- 172.21.15.89:0/1809200031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455ae00 0x5557a4565180 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7404009d60 tx=0x7f740400b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2852ebfb16ba5437 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:42.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.249+0000 7f73f7e00640 1 -- 172.21.15.89:0/1809200031 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404018070 con 0x5557a455ae00 2024-09-10T08:43:42.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.249+0000 7f73f7e00640 1 -- 172.21.15.89:0/1809200031 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7404011a00 con 0x5557a455ae00 2024-09-10T08:43:42.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.249+0000 7f73f7e00640 1 -- 172.21.15.89:0/1809200031 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404013ab0 con 0x5557a455ae00 2024-09-10T08:43:42.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.250+0000 7f741441d740 1 -- 172.21.15.89:0/1809200031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455ae00 msgr2=0x5557a4565180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:42.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.250+0000 7f741441d740 1 --2- 172.21.15.89:0/1809200031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455ae00 0x5557a4565180 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7404009d60 tx=0x7f740400b7e0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.251+0000 7f741441d740 1 -- 172.21.15.89:0/1809200031 shutdown_connections 2024-09-10T08:43:42.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.251+0000 7f741441d740 1 --2- 172.21.15.89:0/1809200031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a45656c0 0x5557a4567ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.251+0000 7f741441d740 1 --2- 172.21.15.89:0/1809200031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455ae00 0x5557a4565180 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.251+0000 7f741441d740 1 --2- 172.21.15.89:0/1809200031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a455a4e0 0x5557a455a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.251+0000 7f741441d740 1 -- 172.21.15.89:0/1809200031 >> 172.21.15.89:0/1809200031 conn(0x5557a4556a70 msgr2=0x5557a4556e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:42.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.252+0000 7f741441d740 1 -- 172.21.15.89:0/1809200031 shutdown_connections 2024-09-10T08:43:42.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.252+0000 7f741441d740 1 -- 172.21.15.89:0/1809200031 wait complete. 2024-09-10T08:43:42.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.253+0000 7f741441d740 1 -- start start 2024-09-10T08:43:42.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.254+0000 7f741441d740 1 Processor -- start 2024-09-10T08:43:42.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.254+0000 7f7403400640 1 --2- 172.21.15.89:0/1817291195 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f73f00a2b60 0x7f73f00a5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:42.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.254+0000 7f7403400640 1 -- 172.21.15.89:0/1817291195 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f73fc0efe70 con 0x7f740c077040 2024-09-10T08:43:42.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.256+0000 7f740b400640 1 --2- 172.21.15.89:0/1817291195 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f73f00a2b60 0x7f73f00a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:42.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.258+0000 7f741441d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a455a4e0 0x5557a45625c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:42.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.259+0000 7f741441d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455ae00 0x5557a4562b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:42.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.259+0000 7f741441d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a4563040 0x5557a4571700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:42.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.259+0000 7f741441d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5557a4568290 con 0x5557a455a4e0 2024-09-10T08:43:42.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.259+0000 7f741441d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5557a4479bd0 con 0x5557a4563040 2024-09-10T08:43:42.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.259+0000 7f741441d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5557a4568410 con 0x5557a455ae00 2024-09-10T08:43:42.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.260+0000 7f741441d740 10 client.0 _ll_register_callbacks cb 0x5557a41ce0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:43:42.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.260+0000 7f740b400640 1 --2- 172.21.15.89:0/1817291195 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f73f00a2b60 0x7f73f00a5020 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f73f8004650 tx=0x7f73f802d2a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:42.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.261+0000 7f740b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a4563040 0x5557a4571700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:42.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.261+0000 7f740b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a4563040 0x5557a4571700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57170/0 (socket says 172.21.15.89:57170) 2024-09-10T08:43:42.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.261+0000 7f740b400640 1 -- 172.21.15.89:0/2158090663 learned_addr learned my addr 172.21.15.89:0/2158090663 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:42.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.261+0000 7f740a000640 1 --2- 172.21.15.89:0/2158090663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455ae00 0x5557a4562b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:42.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.261+0000 7f740b400640 1 -- 172.21.15.89:0/2158090663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455ae00 msgr2=0x5557a4562b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:42.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.261+0000 7f740aa00640 1 --2- 172.21.15.89:0/2158090663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a455a4e0 0x5557a45625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:42.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.261+0000 7f740b400640 1 --2- 172.21.15.89:0/2158090663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455ae00 0x5557a4562b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.261+0000 7f740b400640 1 -- 172.21.15.89:0/2158090663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a455a4e0 msgr2=0x5557a45625c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:42.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.261+0000 7f740b400640 1 --2- 172.21.15.89:0/2158090663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a455a4e0 0x5557a45625c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.261+0000 7f740b400640 1 -- 172.21.15.89:0/2158090663 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557a4571c40 con 0x5557a4563040 2024-09-10T08:43:42.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.261+0000 7f740a000640 1 --2- 172.21.15.89:0/2158090663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455ae00 0x5557a4562b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:43:42.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.261+0000 7f740aa00640 1 --2- 172.21.15.89:0/2158090663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a455a4e0 0x5557a45625c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:43:42.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.262+0000 7f740b400640 1 --2- 172.21.15.89:0/2158090663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a4563040 0x5557a4571700 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f73f80336b0 tx=0x7f73f8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=576aab3dda779a30 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:42.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.262+0000 7f73f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:43:42.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.262+0000 7f73f4c00640 1 -- 172.21.15.89:0/2158090663 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f80408f0 con 0x5557a4563040 2024-09-10T08:43:42.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.262+0000 7f73f4c00640 1 -- 172.21.15.89:0/2158090663 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f73f8040a90 con 0x5557a4563040 2024-09-10T08:43:42.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.262+0000 7f741441d740 1 -- 172.21.15.89:0/2158090663 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7404011a00 con 0x5557a4563040 2024-09-10T08:43:42.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.263+0000 7f73f4c00640 1 -- 172.21.15.89:0/2158090663 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f73f8040c90 con 0x5557a4563040 2024-09-10T08:43:42.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.265+0000 7f741441d740 10 client.26093 fetch_fsmap learned FSMap version 28 2024-09-10T08:43:42.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.265+0000 7f741441d740 1 -- 172.21.15.89:0/2158090663 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557a4574530 con 0x5557a4563040 2024-09-10T08:43:42.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.266+0000 7f73f4c00640 1 -- 172.21.15.89:0/2158090663 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7404011a00 con 0x5557a4563040 2024-09-10T08:43:42.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.266+0000 7f73f4c00640 1 -- 172.21.15.89:0/2158090663 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f73f8030040 con 0x5557a4563040 2024-09-10T08:43:42.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.266+0000 7f741441d740 10 client.26093 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:43:42.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.266+0000 7f741441d740 10 client.26093 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:43:42.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.266+0000 7f741441d740 20 client.26093 populate_metadata read hostname 'smithi089' 2024-09-10T08:43:42.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.266+0000 7f741441d740 1 --2- 172.21.15.89:0/2158090663 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5557a45759f0 0x5557a4577e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:42.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.266+0000 7f741441d740 4 client.26093 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:43:42.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.266+0000 7f741441d740 1 -- 172.21.15.89:0/2158090663 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557a44cc990 con 0x5557a45759f0 2024-09-10T08:43:42.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.266+0000 7f740aa00640 1 --2- 172.21.15.89:0/2158090663 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5557a45759f0 0x5557a4577e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:42.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.267+0000 7f740aa00640 1 --2- 172.21.15.89:0/2158090663 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5557a45759f0 0x5557a4577e40 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34736698b6dcac54 server_cookie=2f4878b5c0b878c3 in_seq=0 out_seq=0 2024-09-10T08:43:42.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.268+0000 7f73f4c00640 10 client.26093 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:42.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.271+0000 7f73f4c00640 1 -- 172.21.15.89:0/2158090663 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557a44cc990 con 0x5557a45759f0 2024-09-10T08:43:42.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.271+0000 7f73f4c00640 10 client.26093 handle_command_reply: tid=0 2024-09-10T08:43:42.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.271+0000 7f741441d740 1 client.26093 shutdown 2024-09-10T08:43:42.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.272+0000 7f741441d740 1 -- 172.21.15.89:0/2158090663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a4563040 msgr2=0x5557a4571700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:42.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.272+0000 7f741441d740 1 --2- 172.21.15.89:0/2158090663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a4563040 0x5557a4571700 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f73f80336b0 tx=0x7f73f8004730 comp rx=0 tx=0).stop 2024-09-10T08:43:42.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.272+0000 7f741441d740 1 -- 172.21.15.89:0/2158090663 shutdown_connections 2024-09-10T08:43:42.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.272+0000 7f741441d740 1 --2- 172.21.15.89:0/2158090663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a4563040 0x5557a4571700 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.272+0000 7f741441d740 1 --2- 172.21.15.89:0/2158090663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455ae00 0x5557a4562b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.272+0000 7f741441d740 1 --2- 172.21.15.89:0/2158090663 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5557a45759f0 0x5557a4577e40 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.272+0000 7f741441d740 1 --2- 172.21.15.89:0/2158090663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a455a4e0 0x5557a45625c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.272+0000 7f73f4c00640 0 client.26093 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:42.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.272+0000 7f741441d740 1 -- 172.21.15.89:0/2158090663 >> 172.21.15.89:0/2158090663 conn(0x5557a4556a70 msgr2=0x5557a4555bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:42.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.272+0000 7f741441d740 1 -- 172.21.15.89:0/2158090663 shutdown_connections 2024-09-10T08:43:42.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.272+0000 7f741441d740 1 -- 172.21.15.89:0/2158090663 wait complete. 2024-09-10T08:43:42.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.273+0000 7f741441d740 20 client.26093 trim_cache size 0 max 16384 2024-09-10T08:43:42.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.315+0000 7f741441d740 1 Processor -- start 2024-09-10T08:43:42.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.318+0000 7f741441d740 1 -- start start 2024-09-10T08:43:42.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.318+0000 7f741441d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a4556a70 0x5557a45714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:42.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.318+0000 7f741441d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455a4e0 0x5557a4564530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:42.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.319+0000 7f741441d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a455ae00 0x5557a457d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:42.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.319+0000 7f741441d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f740c124150 con 0x5557a455ae00 2024-09-10T08:43:42.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.319+0000 7f741441d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f740c121a20 con 0x5557a4556a70 2024-09-10T08:43:42.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.319+0000 7f741441d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f740c11f870 con 0x5557a455a4e0 2024-09-10T08:43:42.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.319+0000 7f740a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455a4e0 0x5557a4564530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:42.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.319+0000 7f740a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455a4e0 0x5557a4564530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:44428/0 (socket says 172.21.15.89:44428) 2024-09-10T08:43:42.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.319+0000 7f740aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a4556a70 0x5557a45714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:42.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.319+0000 7f740aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a4556a70 0x5557a45714f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57178/0 (socket says 172.21.15.89:57178) 2024-09-10T08:43:42.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.319+0000 7f740aa00640 1 -- 172.21.15.89:0/3478919474 learned_addr learned my addr 172.21.15.89:0/3478919474 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:42.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.319+0000 7f740b400640 1 --2- 172.21.15.89:0/3478919474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a455ae00 0x5557a457d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:42.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.320+0000 7f740aa00640 1 -- 172.21.15.89:0/3478919474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455a4e0 msgr2=0x5557a4564530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:42.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.320+0000 7f740aa00640 1 --2- 172.21.15.89:0/3478919474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455a4e0 0x5557a4564530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.320+0000 7f740aa00640 1 -- 172.21.15.89:0/3478919474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a455ae00 msgr2=0x5557a457d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:42.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.320+0000 7f740aa00640 1 --2- 172.21.15.89:0/3478919474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a455ae00 0x5557a457d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.320+0000 7f740aa00640 1 -- 172.21.15.89:0/3478919474 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557a45784c0 con 0x5557a4556a70 2024-09-10T08:43:42.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.320+0000 7f740aa00640 1 --2- 172.21.15.89:0/3478919474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a4556a70 0x5557a45714f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7404009c80 tx=0x7f74040134b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9f59d09d05f451 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:42.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.320+0000 7f73f7400640 1 -- 172.21.15.89:0/3478919474 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404018040 con 0x5557a4556a70 2024-09-10T08:43:42.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.320+0000 7f73f7400640 1 -- 172.21.15.89:0/3478919474 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7404024070 con 0x5557a4556a70 2024-09-10T08:43:42.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.320+0000 7f73f7400640 1 -- 172.21.15.89:0/3478919474 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404013bc0 con 0x5557a4556a70 2024-09-10T08:43:42.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.321+0000 7f741441d740 1 -- 172.21.15.89:0/3478919474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a4556a70 msgr2=0x5557a45714f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:42.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.321+0000 7f741441d740 1 --2- 172.21.15.89:0/3478919474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a4556a70 0x5557a45714f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7404009c80 tx=0x7f74040134b0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.322+0000 7f741441d740 1 -- 172.21.15.89:0/3478919474 shutdown_connections 2024-09-10T08:43:42.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.322+0000 7f741441d740 1 --2- 172.21.15.89:0/3478919474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a455ae00 0x5557a457d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.322+0000 7f741441d740 1 --2- 172.21.15.89:0/3478919474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455a4e0 0x5557a4564530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.322+0000 7f741441d740 1 --2- 172.21.15.89:0/3478919474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a4556a70 0x5557a45714f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.322+0000 7f741441d740 1 -- 172.21.15.89:0/3478919474 >> 172.21.15.89:0/3478919474 conn(0x5557a4562c60 msgr2=0x5557a4563e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:42.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.322+0000 7f741441d740 1 -- 172.21.15.89:0/3478919474 shutdown_connections 2024-09-10T08:43:42.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.322+0000 7f741441d740 1 -- 172.21.15.89:0/3478919474 wait complete. 2024-09-10T08:43:42.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.323+0000 7f741441d740 1 -- start start 2024-09-10T08:43:42.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.323+0000 7f741441d740 1 Processor -- start 2024-09-10T08:43:42.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.323+0000 7f741441d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a4556a70 0x5557a45d9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:42.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.324+0000 7f740aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a4556a70 0x5557a45d9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:42.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.324+0000 7f740aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a4556a70 0x5557a45d9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:34780/0 (socket says 172.21.15.89:34780) 2024-09-10T08:43:42.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.328+0000 7f741441d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a455a4e0 0x5557a45da230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:42.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.328+0000 7f740aa00640 1 -- 172.21.15.89:0/3716621990 learned_addr learned my addr 172.21.15.89:0/3716621990 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:42.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.328+0000 7f741441d740 1 --2- 172.21.15.89:0/3716621990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455ae00 0x5557a456d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:42.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.328+0000 7f740a000640 1 --2- 172.21.15.89:0/3716621990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a455a4e0 0x5557a45da230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:42.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.328+0000 7f741441d740 1 -- 172.21.15.89:0/3716621990 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f740c11fee0 con 0x5557a4556a70 2024-09-10T08:43:42.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.328+0000 7f741441d740 1 -- 172.21.15.89:0/3716621990 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f740c121350 con 0x5557a455a4e0 2024-09-10T08:43:42.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.328+0000 7f741441d740 1 -- 172.21.15.89:0/3716621990 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f740c1205b0 con 0x5557a455ae00 2024-09-10T08:43:42.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.328+0000 7f741441d740 10 client.0 _ll_register_callbacks cb 0x7f73f802d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:43:42.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.328+0000 7f740b400640 1 --2- 172.21.15.89:0/3716621990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455ae00 0x5557a456d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:42.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.329+0000 7f740b400640 1 -- 172.21.15.89:0/3716621990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a455a4e0 msgr2=0x5557a45da230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:42.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.329+0000 7f740b400640 1 --2- 172.21.15.89:0/3716621990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a455a4e0 0x5557a45da230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.329+0000 7f740b400640 1 -- 172.21.15.89:0/3716621990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a4556a70 msgr2=0x5557a45d9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:42.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.329+0000 7f740b400640 1 --2- 172.21.15.89:0/3716621990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a4556a70 0x5557a45d9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.329+0000 7f740b400640 1 -- 172.21.15.89:0/3716621990 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557a456dba0 con 0x5557a455ae00 2024-09-10T08:43:42.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.329+0000 7f73f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:43:42.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.329+0000 7f740b400640 1 --2- 172.21.15.89:0/3716621990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455ae00 0x5557a456d660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f73f803f590 tx=0x7f73f803fa30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7ce4ddab37c5c09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:42.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.330+0000 7f73f4c00640 1 -- 172.21.15.89:0/3716621990 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8032c80 con 0x5557a455ae00 2024-09-10T08:43:42.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.330+0000 7f73f4c00640 1 -- 172.21.15.89:0/3716621990 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f73f8040900 con 0x5557a455ae00 2024-09-10T08:43:42.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.330+0000 7f740a000640 1 --2- 172.21.15.89:0/3716621990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a455a4e0 0x5557a45da230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:43:42.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.330+0000 7f741441d740 1 -- 172.21.15.89:0/3716621990 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7404024070 con 0x5557a455ae00 2024-09-10T08:43:42.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.331+0000 7f73f4c00640 1 -- 172.21.15.89:0/3716621990 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f73f8032c80 con 0x5557a455ae00 2024-09-10T08:43:42.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.331+0000 7f741441d740 10 client.26038 fetch_fsmap learned FSMap version 28 2024-09-10T08:43:42.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.331+0000 7f741441d740 1 -- 172.21.15.89:0/3716621990 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557a45f2070 con 0x5557a455ae00 2024-09-10T08:43:42.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.331+0000 7f73f4c00640 1 -- 172.21.15.89:0/3716621990 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7404024070 con 0x5557a455ae00 2024-09-10T08:43:42.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.331+0000 7f73f4c00640 1 -- 172.21.15.89:0/3716621990 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f73f8056400 con 0x5557a455ae00 2024-09-10T08:43:42.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.332+0000 7f741441d740 10 client.26038 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:43:42.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.332+0000 7f741441d740 10 client.26038 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:43:42.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.332+0000 7f741441d740 20 client.26038 populate_metadata read hostname 'smithi089' 2024-09-10T08:43:42.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.332+0000 7f741441d740 1 --2- 172.21.15.89:0/3716621990 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5557a4563040 0x5557a45f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:42.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.332+0000 7f741441d740 4 client.26038 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-09-10T08:43:42.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.332+0000 7f741441d740 1 -- 172.21.15.89:0/3716621990 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x5557a44c6ce0 con 0x5557a4563040 2024-09-10T08:43:42.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.333+0000 7f740aa00640 1 --2- 172.21.15.89:0/3716621990 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5557a4563040 0x5557a45f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:42.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.335+0000 7f73f4c00640 10 client.26038 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:42.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:42.335+0000 7f740aa00640 1 --2- 172.21.15.89:0/3716621990 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5557a4563040 0x5557a45f48a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dd94faef49847ea server_cookie=23b8a4b0cfa69475 in_seq=0 out_seq=0 2024-09-10T08:43:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.515+0000 7f45d6836740 1 -- 172.21.15.89:0/787764597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead898a60 msgr2=0x560ead8a1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.515+0000 7f45d6836740 1 --2- 172.21.15.89:0/787764597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead898a60 0x560ead8a1f40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009d10 tx=0x7f45c4042870 comp rx=0 tx=0).stop 2024-09-10T08:43:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.526+0000 7f45d6836740 1 -- 172.21.15.89:0/787764597 shutdown_connections 2024-09-10T08:43:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.526+0000 7f45d6836740 1 --2- 172.21.15.89:0/787764597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560ead8a2f50 0x560ead8a33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.526+0000 7f45d6836740 1 --2- 172.21.15.89:0/787764597 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560ead913fe0 0x560ead916400 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.527+0000 7f45d6836740 1 --2- 172.21.15.89:0/787764597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ead898a60 0x560ead8a1f40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.527+0000 7f45d6836740 1 --2- 172.21.15.89:0/787764597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ead893090 0x560ead8a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.527+0000 7f45d6836740 1 -- 172.21.15.89:0/787764597 >> 172.21.15.89:0/787764597 conn(0x560ead88e700 msgr2=0x560ead894c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.528+0000 7f45b6a00640 0 client.16008 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.537+0000 7f45d6836740 1 -- 172.21.15.89:0/787764597 shutdown_connections 2024-09-10T08:43:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.542+0000 7f45d6836740 1 -- 172.21.15.89:0/787764597 wait complete. 2024-09-10T08:43:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.547+0000 7f45d6836740 20 client.16008 trim_cache size 0 max 16384 2024-09-10T08:43:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:43:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:43:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:43:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:43:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.624+0000 7f45d3a00640 1 -- 172.21.15.89:0/3538205135 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f45b00a2d80 msgr2=0x7f45b00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.624+0000 7f45d3a00640 1 --2- 172.21.15.89:0/3538205135 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f45b00a2d80 0x7f45b00a5240 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f45bc01ccb0 tx=0x7f45bc005f70 comp rx=0 tx=0).stop 2024-09-10T08:43:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.624+0000 7f45d3a00640 1 -- 172.21.15.89:0/3538205135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45cc072d20 msgr2=0x7f45cc19e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.624+0000 7f45d3a00640 1 --2- 172.21.15.89:0/3538205135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45cc072d20 0x7f45cc19e8f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f45b802a8d0 tx=0x7f45b8030860 comp rx=0 tx=0).stop 2024-09-10T08:43:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.629+0000 7f45d3a00640 1 -- 172.21.15.89:0/3538205135 shutdown_connections 2024-09-10T08:43:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.629+0000 7f45d3a00640 1 --2- 172.21.15.89:0/3538205135 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f45b00a2d80 0x7f45b00a5240 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.629+0000 7f45d3a00640 1 --2- 172.21.15.89:0/3538205135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45cc109700 0x7f45cc1a5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.629+0000 7f45d3a00640 1 --2- 172.21.15.89:0/3538205135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45cc0736c0 0x7f45cc19ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.629+0000 7f45d3a00640 1 --2- 172.21.15.89:0/3538205135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45cc072d20 0x7f45cc19e8f0 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f45b802a8d0 tx=0x7f45b8030860 comp rx=0 tx=0).stop 2024-09-10T08:43:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.629+0000 7f45d3a00640 1 -- 172.21.15.89:0/3538205135 >> 172.21.15.89:0/3538205135 conn(0x7f45cc06ece0 msgr2=0x7f45cc071b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.665+0000 7f45d3a00640 1 -- 172.21.15.89:0/3538205135 shutdown_connections 2024-09-10T08:43:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:42.673+0000 7f45d3a00640 1 -- 172.21.15.89:0/3538205135 wait complete. 2024-09-10T08:43:43.168 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:42 smithi089 ceph-mon[30757]: pgmap v506: 129 pgs: 129 active+clean; 275 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 972 op/s 2024-09-10T08:43:43.196 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:42 smithi111 ceph-mon[35220]: pgmap v506: 129 pgs: 129 active+clean; 275 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 972 op/s 2024-09-10T08:43:43.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:42 smithi149 ceph-mon[36167]: pgmap v506: 129 pgs: 129 active+clean; 275 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 972 op/s 2024-09-10T08:43:43.543 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:43:43 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-b[65063]: 2024-09-10T08:43:43.195+0000 7fb147ed3640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001741 [...2,head] /volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-33/ auth{1=1} fragtree_t(*^3) v151480 ap=2 RANDEPHEMERALPIN f(v8 m2024-09-10T08:43:22.292097+0000 220=220+0) n(v9 rc2024-09-10T08:43:22.292097+0000 b4435004 221=220+1) (inest sync->mix g=1) caps={25157=pAsLsXsFs/pAsLsXsFsx@231} mcw={1=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x557facfdc580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000001741:[<0x10000000399/blog-33 v44350>,<0x10000000208/blogtest_in v473>,<0x10000000201/src v1692>,<0x10000000006/blogbench-1.0 v1470>,<0x10000000005/tmp v97>,<0x100000001fb/client.0 v88>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v104>,<0x10000000001/sv_1 v120>,<0x10000000000/qa v110>,<0x1/volumes v107>]//[]","memoryvalue":"(2)0x10000001741:[<0x10000000399/blog-33 v151480>,<0x10000000208/blogtest_in v1002>,<0x10000000201/src v1835>,<0x10000000006/blogbench-1.0 v1614>,<0x10000000005/tmp v234>,<0x100000001fb/client.0 v224>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v233>,<0x10000000001/sv_1 v249>,<0x10000000000/qa v234>,<0x1/volumes v235>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T08:43:12.690321+0000 217=217+0)","ondisk_value.rstat":"n(v0 rc2024-09-10T08:43:22.292097+0000 b4435004 221=220+1)","memory_value.dirstat":"f(v8 m2024-09-10T08:43:22.292097+0000 220=220+0)","memory_value.rstat":"n(v9 rc2024-09-10T08:43:22.292097+0000 b4435004 221=220+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-10T08:43:43.689 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:44.107 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:43 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:43:44.107 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:43 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:43:44.107 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:43 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:43:44.107 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:43 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:43:44.107 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:43 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:43:44.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:43 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:43:44.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:43 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:43:44.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:43 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:43:44.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:43 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:43:44.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:43 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:43:44.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:43 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:43:44.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:43 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:43:44.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:43 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:43:44.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:43 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:43:44.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:43 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:43:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.492+0000 7f21b3600640 1 -- 172.21.15.89:0/3093313045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21ac102d20 msgr2=0x7f21ac103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:44.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.492+0000 7f21b3600640 1 --2- 172.21.15.89:0/3093313045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21ac102d20 0x7f21ac103100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009860 tx=0x7f21a002a3d0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.501+0000 7f21b3600640 1 -- 172.21.15.89:0/3093313045 shutdown_connections 2024-09-10T08:43:44.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.501+0000 7f21b3600640 1 --2- 172.21.15.89:0/3093313045 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21ac10c150 0x7f21ac10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.501+0000 7f21b3600640 1 --2- 172.21.15.89:0/3093313045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21ac103640 0x7f21ac10bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.501+0000 7f21b3600640 1 --2- 172.21.15.89:0/3093313045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21ac102d20 0x7f21ac103100 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.501+0000 7f21b3600640 1 -- 172.21.15.89:0/3093313045 >> 172.21.15.89:0/3093313045 conn(0x7f21ac0fc780 msgr2=0x7f21ac0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:44.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.502+0000 7f21b3600640 1 -- 172.21.15.89:0/3093313045 shutdown_connections 2024-09-10T08:43:44.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.515+0000 7f21b3600640 1 -- 172.21.15.89:0/3093313045 wait complete. 2024-09-10T08:43:44.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.516+0000 7f21b3600640 1 Processor -- start 2024-09-10T08:43:44.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.517+0000 7f21b3600640 1 -- start start 2024-09-10T08:43:44.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.518+0000 7f21b3600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21ac102d20 0x7f21ac100980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:44.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.518+0000 7f21b3600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21ac103640 0x7f21ac100ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:44.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.518+0000 7f21b3600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21ac10c150 0x7f21ac101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:44.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.518+0000 7f21b3600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac111fa0 con 0x7f21ac102d20 2024-09-10T08:43:44.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.518+0000 7f21b3600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac111e20 con 0x7f21ac10c150 2024-09-10T08:43:44.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.518+0000 7f21b3600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac112120 con 0x7f21ac103640 2024-09-10T08:43:44.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.519+0000 7f21b0c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21ac102d20 0x7f21ac100980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:44.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.519+0000 7f21b1600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21ac10c150 0x7f21ac101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:44.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.519+0000 7f21b1600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21ac10c150 0x7f21ac101400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:56750/0 (socket says 172.21.15.89:56750) 2024-09-10T08:43:44.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.519+0000 7f21b1600640 1 -- 172.21.15.89:0/4235872289 learned_addr learned my addr 172.21.15.89:0/4235872289 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:44.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.520+0000 7f21b1600640 1 -- 172.21.15.89:0/4235872289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21ac103640 msgr2=0x7f21ac100ec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:43:44.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.520+0000 7f21b1600640 1 --2- 172.21.15.89:0/4235872289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21ac103640 0x7f21ac100ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.520+0000 7f21b1600640 1 -- 172.21.15.89:0/4235872289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21ac102d20 msgr2=0x7f21ac100980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:44.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.520+0000 7f21b1600640 1 --2- 172.21.15.89:0/4235872289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21ac102d20 0x7f21ac100980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.520+0000 7f21b1600640 1 -- 172.21.15.89:0/4235872289 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ac19c5e0 con 0x7f21ac10c150 2024-09-10T08:43:44.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.520+0000 7f21b1600640 1 --2- 172.21.15.89:0/4235872289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21ac10c150 0x7f21ac101400 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f21a400cce0 tx=0x7f21a400a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:44.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.521+0000 7f21a9600640 1 -- 172.21.15.89:0/4235872289 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a400f450 con 0x7f21ac10c150 2024-09-10T08:43:44.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.521+0000 7f21a9600640 1 -- 172.21.15.89:0/4235872289 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21a400fad0 con 0x7f21ac10c150 2024-09-10T08:43:44.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.521+0000 7f21a9600640 1 -- 172.21.15.89:0/4235872289 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a4018c10 con 0x7f21ac10c150 2024-09-10T08:43:44.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.523+0000 7f21b3600640 1 -- 172.21.15.89:0/4235872289 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21ac19c8d0 con 0x7f21ac10c150 2024-09-10T08:43:44.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.524+0000 7f21b3600640 1 -- 172.21.15.89:0/4235872289 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21ac19cd90 con 0x7f21ac10c150 2024-09-10T08:43:44.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.534+0000 7f21a9600640 1 -- 172.21.15.89:0/4235872289 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f21a400fc80 con 0x7f21ac10c150 2024-09-10T08:43:44.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.537+0000 7f21a9600640 1 --2- 172.21.15.89:0/4235872289 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f21940a2c00 0x7f21940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:44.540 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.539+0000 7f21a9600640 1 -- 172.21.15.89:0/4235872289 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f21a40cf6f0 con 0x7f21ac10c150 2024-09-10T08:43:44.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.541+0000 7f21b608c740 1 Processor -- start 2024-09-10T08:43:44.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.541+0000 7f21b0c00640 1 --2- 172.21.15.89:0/4235872289 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f21940a2c00 0x7f21940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:44.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.543+0000 7f21b608c740 1 -- start start 2024-09-10T08:43:44.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.544+0000 7f21b0c00640 1 --2- 172.21.15.89:0/4235872289 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f21940a2c00 0x7f21940a50c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f21a002a9c0 tx=0x7f21a0005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:44.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.545+0000 7f21b608c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6fa9b90 0x5624f6fa9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:44.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.545+0000 7f21b608c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624f6faa4b0 0x5624f6fb4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:44.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.545+0000 7f21b608c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6fb4d70 0x5624f6fb7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:44.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.545+0000 7f21b608c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5624f6f103e0 con 0x5624f6faa4b0 2024-09-10T08:43:44.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.545+0000 7f21b608c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5624f6eb7d80 con 0x5624f6fb4d70 2024-09-10T08:43:44.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.545+0000 7f21b608c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5624f6f0c3a0 con 0x5624f6fa9b90 2024-09-10T08:43:44.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.546+0000 7f21b1600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6fb4d70 0x5624f6fb7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.546+0000 7f21b1600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6fb4d70 0x5624f6fb7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:56766/0 (socket says 172.21.15.89:56766) 2024-09-10T08:43:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.546+0000 7f21b1600640 1 -- 172.21.15.89:0/410860203 learned_addr learned my addr 172.21.15.89:0/410860203 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:44.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.546+0000 7f21abe00640 1 --2- 172.21.15.89:0/410860203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624f6faa4b0 0x5624f6fb4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:44.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.547+0000 7f21b1600640 1 -- 172.21.15.89:0/410860203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6fa9b90 msgr2=0x5624f6fa9f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:43:44.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.547+0000 7f21b1600640 1 --2- 172.21.15.89:0/410860203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6fa9b90 0x5624f6fa9f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.547+0000 7f21b1600640 1 -- 172.21.15.89:0/410860203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624f6faa4b0 msgr2=0x5624f6fb4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:44.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.547+0000 7f21b1600640 1 --2- 172.21.15.89:0/410860203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624f6faa4b0 0x5624f6fb4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.548+0000 7f21b1600640 1 -- 172.21.15.89:0/410860203 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624f6ed6ae0 con 0x5624f6fb4d70 2024-09-10T08:43:44.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.549+0000 7f21b1600640 1 --2- 172.21.15.89:0/410860203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6fb4d70 0x5624f6fb7160 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f21a4098550 tx=0x7f21a409c280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9e0a7b410c55811 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:44.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.549+0000 7f219dc00640 1 -- 172.21.15.89:0/410860203 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a40a47f0 con 0x5624f6fb4d70 2024-09-10T08:43:44.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.549+0000 7f219dc00640 1 -- 172.21.15.89:0/410860203 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21a4018db0 con 0x5624f6fb4d70 2024-09-10T08:43:44.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.550+0000 7f219dc00640 1 -- 172.21.15.89:0/410860203 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a409b230 con 0x5624f6fb4d70 2024-09-10T08:43:44.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.552+0000 7f21b608c740 1 -- 172.21.15.89:0/410860203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6fb4d70 msgr2=0x5624f6fb7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:44.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.552+0000 7f21b608c740 1 --2- 172.21.15.89:0/410860203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6fb4d70 0x5624f6fb7160 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f21a4098550 tx=0x7f21a409c280 comp rx=0 tx=0).stop 2024-09-10T08:43:44.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.553+0000 7f21b608c740 1 -- 172.21.15.89:0/410860203 shutdown_connections 2024-09-10T08:43:44.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.553+0000 7f21b608c740 1 --2- 172.21.15.89:0/410860203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6fb4d70 0x5624f6fb7160 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.553+0000 7f21b608c740 1 --2- 172.21.15.89:0/410860203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624f6faa4b0 0x5624f6fb4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.553+0000 7f21b608c740 1 --2- 172.21.15.89:0/410860203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6fa9b90 0x5624f6fa9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.553+0000 7f21b608c740 1 -- 172.21.15.89:0/410860203 >> 172.21.15.89:0/410860203 conn(0x5624f6fa6120 msgr2=0x5624f6fa6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:44.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.555+0000 7f21b608c740 1 -- 172.21.15.89:0/410860203 shutdown_connections 2024-09-10T08:43:44.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.555+0000 7f21b608c740 1 -- 172.21.15.89:0/410860203 wait complete. 2024-09-10T08:43:44.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.556+0000 7f21b608c740 1 -- start start 2024-09-10T08:43:44.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.557+0000 7f21b608c740 1 Processor -- start 2024-09-10T08:43:44.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.558+0000 7f21b608c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624f6fa9b90 0x5624f6fb1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:44.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.558+0000 7f21b608c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6faa4b0 0x5624f6fb22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:44.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.558+0000 7f21b608c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6fb4d70 0x5624f6fb2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:44.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.558+0000 7f21b608c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5624f6fb7980 con 0x5624f6fa9b90 2024-09-10T08:43:44.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.558+0000 7f21b608c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5624f6ecb0f0 con 0x5624f6faa4b0 2024-09-10T08:43:44.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.558+0000 7f21b608c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5624f6fb7b00 con 0x5624f6fb4d70 2024-09-10T08:43:44.560 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.558+0000 7f21b608c740 10 client.0 _ll_register_callbacks cb 0x5624f6ebc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:43:44.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.560+0000 7f21abe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6faa4b0 0x5624f6fb22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:44.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.560+0000 7f21abe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6faa4b0 0x5624f6fb22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:56778/0 (socket says 172.21.15.89:56778) 2024-09-10T08:43:44.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.560+0000 7f21abe00640 1 -- 172.21.15.89:0/2667309066 learned_addr learned my addr 172.21.15.89:0/2667309066 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:44.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.561+0000 7f21b1600640 1 --2- 172.21.15.89:0/2667309066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6fb4d70 0x5624f6fb2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:44.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.561+0000 7f21b0c00640 1 --2- 172.21.15.89:0/2667309066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624f6fa9b90 0x5624f6fb1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:44.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.562+0000 7f21abe00640 1 -- 172.21.15.89:0/2667309066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6fb4d70 msgr2=0x5624f6fb2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:44.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.562+0000 7f21abe00640 1 --2- 172.21.15.89:0/2667309066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6fb4d70 0x5624f6fb2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.562+0000 7f21abe00640 1 -- 172.21.15.89:0/2667309066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624f6fa9b90 msgr2=0x5624f6fb1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:44.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.562+0000 7f21abe00640 1 --2- 172.21.15.89:0/2667309066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624f6fa9b90 0x5624f6fb1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.562+0000 7f21abe00640 1 -- 172.21.15.89:0/2667309066 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624f6c813c0 con 0x5624f6faa4b0 2024-09-10T08:43:44.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.563+0000 7f21b1600640 1 --2- 172.21.15.89:0/2667309066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6fb4d70 0x5624f6fb2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:43:44.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.563+0000 7f21abe00640 1 --2- 172.21.15.89:0/2667309066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6faa4b0 0x5624f6fb22f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f21980113e0 tx=0x7f21980118a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f28e63a7fa2b9180 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:44.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.564+0000 7f2192000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:43:44.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.564+0000 7f2192000640 1 -- 172.21.15.89:0/2667309066 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198012920 con 0x5624f6faa4b0 2024-09-10T08:43:44.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.565+0000 7f2192000640 1 -- 172.21.15.89:0/2667309066 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f2198012ac0 con 0x5624f6faa4b0 2024-09-10T08:43:44.566 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.565+0000 7f21b608c740 1 -- 172.21.15.89:0/2667309066 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21a4018db0 con 0x5624f6faa4b0 2024-09-10T08:43:44.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.567+0000 7f2192000640 1 -- 172.21.15.89:0/2667309066 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2198012cc0 con 0x5624f6faa4b0 2024-09-10T08:43:44.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.567+0000 7f21b608c740 10 client.26108 fetch_fsmap learned FSMap version 28 2024-09-10T08:43:44.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.567+0000 7f21b608c740 1 -- 172.21.15.89:0/2667309066 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5624f6fc37d0 con 0x5624f6faa4b0 2024-09-10T08:43:44.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.568+0000 7f2192000640 1 -- 172.21.15.89:0/2667309066 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21a4018db0 con 0x5624f6faa4b0 2024-09-10T08:43:44.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.568+0000 7f2192000640 1 -- 172.21.15.89:0/2667309066 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f219801a480 con 0x5624f6faa4b0 2024-09-10T08:43:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.569+0000 7f21b608c740 10 client.26108 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:43:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.569+0000 7f21b608c740 10 client.26108 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:43:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.569+0000 7f21b608c740 20 client.26108 populate_metadata read hostname 'smithi089' 2024-09-10T08:43:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.569+0000 7f21b608c740 1 --2- 172.21.15.89:0/2667309066 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5624f6fc4c90 0x5624f6fc70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.569+0000 7f21b608c740 4 client.26108 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:43:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.569+0000 7f21b608c740 1 -- 172.21.15.89:0/2667309066 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624f6f1d950 con 0x5624f6fc4c90 2024-09-10T08:43:44.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.570+0000 7f21b0c00640 1 --2- 172.21.15.89:0/2667309066 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5624f6fc4c90 0x5624f6fc70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:44.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.575+0000 7f21b0c00640 1 --2- 172.21.15.89:0/2667309066 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5624f6fc4c90 0x5624f6fc70a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6844bc995062826c server_cookie=4398d62e3fc8bf3b in_seq=0 out_seq=0 2024-09-10T08:43:44.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.576+0000 7f2192000640 10 client.26108 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:44.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.578+0000 7f2192000640 1 -- 172.21.15.89:0/2667309066 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624f6f1d950 con 0x5624f6fc4c90 2024-09-10T08:43:44.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.579+0000 7f2192000640 10 client.26108 handle_command_reply: tid=0 2024-09-10T08:43:44.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.579+0000 7f21b608c740 1 client.26108 shutdown 2024-09-10T08:43:44.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.579+0000 7f21b608c740 1 -- 172.21.15.89:0/2667309066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6faa4b0 msgr2=0x5624f6fb22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:44.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.579+0000 7f21b608c740 1 --2- 172.21.15.89:0/2667309066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6faa4b0 0x5624f6fb22f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f21980113e0 tx=0x7f21980118a0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.580+0000 7f21b608c740 1 -- 172.21.15.89:0/2667309066 shutdown_connections 2024-09-10T08:43:44.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.580+0000 7f21b608c740 1 --2- 172.21.15.89:0/2667309066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6fb4d70 0x5624f6fb2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.580+0000 7f21b608c740 1 --2- 172.21.15.89:0/2667309066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6faa4b0 0x5624f6fb22f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.580+0000 7f21b608c740 1 --2- 172.21.15.89:0/2667309066 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5624f6fc4c90 0x5624f6fc70a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.580+0000 7f21b608c740 1 --2- 172.21.15.89:0/2667309066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624f6fa9b90 0x5624f6fb1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.580+0000 7f2192000640 0 client.26108 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:44.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.580+0000 7f21b608c740 1 -- 172.21.15.89:0/2667309066 >> 172.21.15.89:0/2667309066 conn(0x5624f6fa6120 msgr2=0x5624f6fabe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:44.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.586+0000 7f21b608c740 1 -- 172.21.15.89:0/2667309066 shutdown_connections 2024-09-10T08:43:44.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.586+0000 7f21b608c740 1 -- 172.21.15.89:0/2667309066 wait complete. 2024-09-10T08:43:44.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.586+0000 7f21b608c740 20 client.26108 trim_cache size 0 max 16384 2024-09-10T08:43:44.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.663+0000 7f21b608c740 1 Processor -- start 2024-09-10T08:43:44.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.663+0000 7f21b608c740 1 -- start start 2024-09-10T08:43:44.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.663+0000 7f21b608c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6fa9b90 0x5624f6fb3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:44.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.663+0000 7f21b608c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624f6faa4b0 0x5624f6fcc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:44.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.663+0000 7f21b608c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6fb4d70 0x5624f6fccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:44.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.664+0000 7f21b608c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac1c0530 con 0x5624f6faa4b0 2024-09-10T08:43:44.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.664+0000 7f21b608c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac1bde00 con 0x5624f6fb4d70 2024-09-10T08:43:44.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.664+0000 7f21b608c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac1a4bc0 con 0x5624f6fa9b90 2024-09-10T08:43:44.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.664+0000 7f21b1600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6fb4d70 0x5624f6fccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:44.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.664+0000 7f21b1600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6fb4d70 0x5624f6fccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:56790/0 (socket says 172.21.15.89:56790) 2024-09-10T08:43:44.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.664+0000 7f21b1600640 1 -- 172.21.15.89:0/1926302718 learned_addr learned my addr 172.21.15.89:0/1926302718 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:44.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.665+0000 7f21b0c00640 1 --2- 172.21.15.89:0/1926302718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6fa9b90 0x5624f6fb3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:44.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.665+0000 7f21b0c00640 1 -- 172.21.15.89:0/1926302718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6fb4d70 msgr2=0x5624f6fccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:44.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.665+0000 7f21b0c00640 1 --2- 172.21.15.89:0/1926302718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6fb4d70 0x5624f6fccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.665+0000 7f21b0c00640 1 -- 172.21.15.89:0/1926302718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624f6faa4b0 msgr2=0x5624f6fcc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:44.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.665+0000 7f21b0c00640 1 --2- 172.21.15.89:0/1926302718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624f6faa4b0 0x5624f6fcc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.665+0000 7f21b0c00640 1 -- 172.21.15.89:0/1926302718 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624f6fb7390 con 0x5624f6fa9b90 2024-09-10T08:43:44.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.666+0000 7f21b0c00640 1 --2- 172.21.15.89:0/1926302718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6fa9b90 0x5624f6fb3d20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009830 tx=0x7f21a003f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a381b69923f1c822 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:44.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.666+0000 7f219d200640 1 -- 172.21.15.89:0/1926302718 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0058070 con 0x5624f6fa9b90 2024-09-10T08:43:44.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.666+0000 7f219d200640 1 -- 172.21.15.89:0/1926302718 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21a0041070 con 0x5624f6fa9b90 2024-09-10T08:43:44.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.667+0000 7f219d200640 1 -- 172.21.15.89:0/1926302718 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0004b80 con 0x5624f6fa9b90 2024-09-10T08:43:44.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.668+0000 7f21b608c740 1 -- 172.21.15.89:0/1926302718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6fa9b90 msgr2=0x5624f6fb3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:44.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.668+0000 7f21b608c740 1 --2- 172.21.15.89:0/1926302718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6fa9b90 0x5624f6fb3d20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009830 tx=0x7f21a003f000 comp rx=0 tx=0).stop 2024-09-10T08:43:44.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.668+0000 7f21b608c740 1 -- 172.21.15.89:0/1926302718 shutdown_connections 2024-09-10T08:43:44.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.668+0000 7f21b608c740 1 --2- 172.21.15.89:0/1926302718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6fb4d70 0x5624f6fccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.668+0000 7f21b608c740 1 --2- 172.21.15.89:0/1926302718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624f6faa4b0 0x5624f6fcc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.668+0000 7f21b608c740 1 --2- 172.21.15.89:0/1926302718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6fa9b90 0x5624f6fb3d20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.668+0000 7f21b608c740 1 -- 172.21.15.89:0/1926302718 >> 172.21.15.89:0/1926302718 conn(0x5624f6fa6120 msgr2=0x5624f6fb3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:44.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.670+0000 7f21b608c740 1 -- 172.21.15.89:0/1926302718 shutdown_connections 2024-09-10T08:43:44.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.671+0000 7f21b608c740 1 -- 172.21.15.89:0/1926302718 wait complete. 2024-09-10T08:43:44.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.672+0000 7f21b608c740 1 -- start start 2024-09-10T08:43:44.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.672+0000 7f21b608c740 1 Processor -- start 2024-09-10T08:43:44.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.676+0000 7f21b608c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6fa9b90 0x5624f6fbcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:44.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.676+0000 7f21b608c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6faa4b0 0x5624f6fbd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:44.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.676+0000 7f21b0c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6fa9b90 0x5624f6fbcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:44.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.676+0000 7f21b0c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6fa9b90 0x5624f6fbcd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:56796/0 (socket says 172.21.15.89:56796) 2024-09-10T08:43:44.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.676+0000 7f21b0c00640 1 -- 172.21.15.89:0/791020435 learned_addr learned my addr 172.21.15.89:0/791020435 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:44.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.676+0000 7f21b608c740 1 --2- 172.21.15.89:0/791020435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624f6fb4d70 0x5624f6fb9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:44.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.677+0000 7f21abe00640 1 --2- 172.21.15.89:0/791020435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6faa4b0 0x5624f6fbd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:44.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.677+0000 7f21b608c740 1 -- 172.21.15.89:0/791020435 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac1bc2c0 con 0x5624f6fb4d70 2024-09-10T08:43:44.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.677+0000 7f21b608c740 1 -- 172.21.15.89:0/791020435 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac1bd730 con 0x5624f6fa9b90 2024-09-10T08:43:44.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.677+0000 7f21b608c740 1 -- 172.21.15.89:0/791020435 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac1bc990 con 0x5624f6faa4b0 2024-09-10T08:43:44.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.677+0000 7f21b608c740 10 client.0 _ll_register_callbacks cb 0x7f2198009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:43:44.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.678+0000 7f21b1600640 1 --2- 172.21.15.89:0/791020435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624f6fb4d70 0x5624f6fb9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:44.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.678+0000 7f21b0c00640 1 -- 172.21.15.89:0/791020435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6faa4b0 msgr2=0x5624f6fbd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:44.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.678+0000 7f21b0c00640 1 --2- 172.21.15.89:0/791020435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6faa4b0 0x5624f6fbd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.679+0000 7f21b0c00640 1 -- 172.21.15.89:0/791020435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624f6fb4d70 msgr2=0x5624f6fb9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:44.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.679+0000 7f21b0c00640 1 --2- 172.21.15.89:0/791020435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624f6fb4d70 0x5624f6fb9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.679+0000 7f21b0c00640 1 -- 172.21.15.89:0/791020435 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624f6fba260 con 0x5624f6fa9b90 2024-09-10T08:43:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.679+0000 7f21b1600640 1 --2- 172.21.15.89:0/791020435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624f6fb4d70 0x5624f6fb9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:43:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.679+0000 7f21b0c00640 1 --2- 172.21.15.89:0/791020435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6fa9b90 0x5624f6fbcd50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f21a00058c0 tx=0x7f21a0030e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a537e2735b0b884 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.679+0000 7f2192000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:43:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.680+0000 7f2192000640 1 -- 172.21.15.89:0/791020435 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0058040 con 0x5624f6fa9b90 2024-09-10T08:43:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.680+0000 7f2192000640 1 -- 172.21.15.89:0/791020435 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f21a0004420 con 0x5624f6fa9b90 2024-09-10T08:43:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.681+0000 7f21b608c740 1 -- 172.21.15.89:0/791020435 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21a0041070 con 0x5624f6fa9b90 2024-09-10T08:43:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.681+0000 7f2192000640 1 -- 172.21.15.89:0/791020435 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f21a00035f0 con 0x5624f6fa9b90 2024-09-10T08:43:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.681+0000 7f21b608c740 10 client.26114 fetch_fsmap learned FSMap version 28 2024-09-10T08:43:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.681+0000 7f21b608c740 1 -- 172.21.15.89:0/791020435 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5624f70300e0 con 0x5624f6fa9b90 2024-09-10T08:43:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.682+0000 7f2192000640 1 -- 172.21.15.89:0/791020435 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21a0041070 con 0x5624f6fa9b90 2024-09-10T08:43:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.682+0000 7f2192000640 1 -- 172.21.15.89:0/791020435 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f21a005b070 con 0x5624f6fa9b90 2024-09-10T08:43:44.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.682+0000 7f21b608c740 10 client.26114 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:43:44.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.682+0000 7f21b608c740 10 client.26114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:43:44.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.682+0000 7f21b608c740 20 client.26114 populate_metadata read hostname 'smithi089' 2024-09-10T08:43:44.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.682+0000 7f21b608c740 1 --2- 172.21.15.89:0/791020435 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5624f7030760 0x5624f7032b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:44.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.682+0000 7f21b608c740 4 client.26114 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:43:44.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.682+0000 7f21b608c740 1 -- 172.21.15.89:0/791020435 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5624f6f17e60 con 0x5624f7030760 2024-09-10T08:43:44.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.682+0000 7f21abe00640 1 --2- 172.21.15.89:0/791020435 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5624f7030760 0x5624f7032b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:44.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.683+0000 7f21abe00640 1 --2- 172.21.15.89:0/791020435 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5624f7030760 0x5624f7032b80 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=854cc1984f431236 server_cookie=4bd96f1c30b6c67e in_seq=0 out_seq=0 2024-09-10T08:43:44.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.683+0000 7f2192000640 10 client.26114 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.691+0000 7f2192000640 1 -- 172.21.15.89:0/791020435 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x5624f6f17e60 con 0x5624f7030760 2024-09-10T08:43:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.691+0000 7f2192000640 10 client.26114 handle_command_reply: tid=0 2024-09-10T08:43:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.692+0000 7f21b608c740 1 client.26114 shutdown 2024-09-10T08:43:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.692+0000 7f21b608c740 1 -- 172.21.15.89:0/791020435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6fa9b90 msgr2=0x5624f6fbcd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.692+0000 7f21b608c740 1 --2- 172.21.15.89:0/791020435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6fa9b90 0x5624f6fbcd50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f21a00058c0 tx=0x7f21a0030e90 comp rx=0 tx=0).stop 2024-09-10T08:43:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.693+0000 7f21b608c740 1 -- 172.21.15.89:0/791020435 shutdown_connections 2024-09-10T08:43:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.693+0000 7f21b608c740 1 --2- 172.21.15.89:0/791020435 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5624f7030760 0x5624f7032b80 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.693+0000 7f21b608c740 1 --2- 172.21.15.89:0/791020435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624f6fb4d70 0x5624f6fb9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.693+0000 7f21b608c740 1 --2- 172.21.15.89:0/791020435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624f6faa4b0 0x5624f6fbd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.693+0000 7f21b608c740 1 --2- 172.21.15.89:0/791020435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624f6fa9b90 0x5624f6fbcd50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.693+0000 7f21b608c740 1 -- 172.21.15.89:0/791020435 >> 172.21.15.89:0/791020435 conn(0x5624f6fa6120 msgr2=0x5624f6fabe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:44.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.693+0000 7f2192000640 0 client.26114 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:44.700 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:43:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "scrub active (12178 inodes in the stack)", 2024-09-10T08:43:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2024-09-10T08:43:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: "c243eb93-7473-40b5-b951-e020536a2130": { 2024-09-10T08:43:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2024-09-10T08:43:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "c243eb93-7473-40b5-b951-e020536a2130", 2024-09-10T08:43:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2024-09-10T08:43:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:43:44.700 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:43:44.701 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:43:44.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.694+0000 7f21b608c740 1 -- 172.21.15.89:0/791020435 shutdown_connections 2024-09-10T08:43:44.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.694+0000 7f21b608c740 1 -- 172.21.15.89:0/791020435 wait complete. 2024-09-10T08:43:44.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.695+0000 7f21b608c740 20 client.26114 trim_cache size 0 max 16384 2024-09-10T08:43:44.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.696+0000 7f21b3600640 1 -- 172.21.15.89:0/4235872289 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f21940a2c00 msgr2=0x7f21940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:44.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.696+0000 7f21b3600640 1 --2- 172.21.15.89:0/4235872289 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f21940a2c00 0x7f21940a50c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f21a002a9c0 tx=0x7f21a0005c90 comp rx=0 tx=0).stop 2024-09-10T08:43:44.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.697+0000 7f21b3600640 1 -- 172.21.15.89:0/4235872289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21ac10c150 msgr2=0x7f21ac101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:44.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.697+0000 7f21b3600640 1 --2- 172.21.15.89:0/4235872289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21ac10c150 0x7f21ac101400 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f21a400cce0 tx=0x7f21a400a620 comp rx=0 tx=0).stop 2024-09-10T08:43:44.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.697+0000 7f21b3600640 1 -- 172.21.15.89:0/4235872289 shutdown_connections 2024-09-10T08:43:44.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.697+0000 7f21b3600640 1 --2- 172.21.15.89:0/4235872289 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f21940a2c00 0x7f21940a50c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.697+0000 7f21b3600640 1 --2- 172.21.15.89:0/4235872289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21ac10c150 0x7f21ac101400 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.697+0000 7f21b3600640 1 --2- 172.21.15.89:0/4235872289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21ac103640 0x7f21ac100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.697+0000 7f21b3600640 1 --2- 172.21.15.89:0/4235872289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f21ac102d20 0x7f21ac100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:44.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.697+0000 7f21b3600640 1 -- 172.21.15.89:0/4235872289 >> 172.21.15.89:0/4235872289 conn(0x7f21ac0fc780 msgr2=0x7f21ac0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:44.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.698+0000 7f21b3600640 1 -- 172.21.15.89:0/4235872289 shutdown_connections 2024-09-10T08:43:44.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:43:44.698+0000 7f21b3600640 1 -- 172.21.15.89:0/4235872289 wait complete. 2024-09-10T08:43:44.932 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:44 smithi089 ceph-mon[30757]: Scrub error on inode 0x10000001741 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-33) see mds.b log and `damage ls` output for details 2024-09-10T08:43:44.932 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:44 smithi089 ceph-mon[30757]: pgmap v507: 129 pgs: 129 active+clean; 275 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 975 op/s 2024-09-10T08:43:45.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:44 smithi149 ceph-mon[36167]: Scrub error on inode 0x10000001741 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-33) see mds.b log and `damage ls` output for details 2024-09-10T08:43:45.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:44 smithi149 ceph-mon[36167]: pgmap v507: 129 pgs: 129 active+clean; 275 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 975 op/s 2024-09-10T08:43:45.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:44 smithi111 ceph-mon[35220]: Scrub error on inode 0x10000001741 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-33) see mds.b log and `damage ls` output for details 2024-09-10T08:43:45.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:44 smithi111 ceph-mon[35220]: pgmap v507: 129 pgs: 129 active+clean; 275 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 975 op/s 2024-09-10T08:43:45.720 INFO:tasks.cephfs.filesystem:scrub status for tag:c243eb93-7473-40b5-b951-e020536a2130 - {'path': '/', 'tag': 'c243eb93-7473-40b5-b951-e020536a2130', 'options': 'recursive,force'} 2024-09-10T08:43:47.107 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:46 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:43:47.107 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:46 smithi089 ceph-mon[30757]: pgmap v508: 129 pgs: 129 active+clean; 280 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 1.23k op/s 2024-09-10T08:43:47.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:46 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:43:47.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:46 smithi149 ceph-mon[36167]: pgmap v508: 129 pgs: 129 active+clean; 280 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 1.23k op/s 2024-09-10T08:43:47.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:46 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:43:47.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:46 smithi111 ceph-mon[35220]: pgmap v508: 129 pgs: 129 active+clean; 280 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 1.23k op/s 2024-09-10T08:43:48.042 INFO:tasks.workunit.client.0.smithi089.stdout: 87 289654 170 198821 103 162846 896 2024-09-10T08:43:49.032 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:48 smithi089 ceph-mon[30757]: pgmap v509: 129 pgs: 129 active+clean; 281 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 1.39k op/s 2024-09-10T08:43:49.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:48 smithi149 ceph-mon[36167]: pgmap v509: 129 pgs: 129 active+clean; 281 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 1.39k op/s 2024-09-10T08:43:49.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:48 smithi111 ceph-mon[35220]: pgmap v509: 129 pgs: 129 active+clean; 281 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 1.39k op/s 2024-09-10T08:43:50.973 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:50 smithi089 ceph-mon[30757]: pgmap v510: 129 pgs: 129 active+clean; 282 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 1.37k op/s 2024-09-10T08:43:51.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:50 smithi149 ceph-mon[36167]: pgmap v510: 129 pgs: 129 active+clean; 282 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 1.37k op/s 2024-09-10T08:43:51.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:50 smithi111 ceph-mon[35220]: pgmap v510: 129 pgs: 129 active+clean; 282 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 1.37k op/s 2024-09-10T08:43:53.130 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:52 smithi089 ceph-mon[30757]: pgmap v511: 129 pgs: 129 active+clean; 305 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 1.67k op/s 2024-09-10T08:43:53.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:52 smithi149 ceph-mon[36167]: pgmap v511: 129 pgs: 129 active+clean; 305 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 1.67k op/s 2024-09-10T08:43:53.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:52 smithi111 ceph-mon[35220]: pgmap v511: 129 pgs: 129 active+clean; 305 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 1.67k op/s 2024-09-10T08:43:53.978 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:53 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:43:54.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:53 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:43:54.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:53 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:43:55.134 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:54 smithi089 ceph-mon[30757]: pgmap v512: 129 pgs: 129 active+clean; 310 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 1.16k op/s 2024-09-10T08:43:55.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:54 smithi149 ceph-mon[36167]: pgmap v512: 129 pgs: 129 active+clean; 310 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 1.16k op/s 2024-09-10T08:43:55.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:54 smithi111 ceph-mon[35220]: pgmap v512: 129 pgs: 129 active+clean; 310 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 1.16k op/s 2024-09-10T08:43:56.010 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:55 smithi089 ceph-mon[30757]: scrub summary: idle 2024-09-10T08:43:56.010 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:55 smithi089 ceph-mon[30757]: scrub summary: idle 2024-09-10T08:43:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.075+0000 7f45f8e00640 1 -- 172.21.15.89:0/3073730866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f4107620 msgr2=0x7f45f4109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.075+0000 7f45f8e00640 1 --2- 172.21.15.89:0/3073730866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f4107620 0x7f45f4109a90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009900 tx=0x7f45ec02a410 comp rx=0 tx=0).stop 2024-09-10T08:43:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.075+0000 7f45f8e00640 1 -- 172.21.15.89:0/3073730866 shutdown_connections 2024-09-10T08:43:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.075+0000 7f45f8e00640 1 --2- 172.21.15.89:0/3073730866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f4107620 0x7f45f4109a90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.075+0000 7f45f8e00640 1 --2- 172.21.15.89:0/3073730866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45f4104cf0 0x7f45f41070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.075+0000 7f45f8e00640 1 --2- 172.21.15.89:0/3073730866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45f41023c0 0x7f45f41047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.075+0000 7f45f8e00640 1 -- 172.21.15.89:0/3073730866 >> 172.21.15.89:0/3073730866 conn(0x7f45f40fbfa0 msgr2=0x7f45f40fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.075+0000 7f45f8e00640 1 -- 172.21.15.89:0/3073730866 shutdown_connections 2024-09-10T08:43:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.075+0000 7f45f8e00640 1 -- 172.21.15.89:0/3073730866 wait complete. 2024-09-10T08:43:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.076+0000 7f45f8e00640 1 Processor -- start 2024-09-10T08:43:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.076+0000 7f45f8e00640 1 -- start start 2024-09-10T08:43:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.076+0000 7f45f8e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f41023c0 0x7f45f4197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.076+0000 7f45f8e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45f4104cf0 0x7f45f4198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.076+0000 7f45f8e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45f4107620 0x7f45f419f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.076+0000 7f45f8e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f45f410d680 con 0x7f45f41023c0 2024-09-10T08:43:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.076+0000 7f45f8e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f45f410d500 con 0x7f45f4104cf0 2024-09-10T08:43:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.076+0000 7f45f8e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f45f410d800 con 0x7f45f4107620 2024-09-10T08:43:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.076+0000 7f45f3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f41023c0 0x7f45f4197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.076+0000 7f45f3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f41023c0 0x7f45f4197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56620/0 (socket says 172.21.15.89:56620) 2024-09-10T08:43:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.076+0000 7f45f3400640 1 -- 172.21.15.89:0/547744377 learned_addr learned my addr 172.21.15.89:0/547744377 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.077+0000 7f45eaa00640 1 --2- 172.21.15.89:0/547744377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45f4104cf0 0x7f45f4198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.077+0000 7f45f3400640 1 -- 172.21.15.89:0/547744377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45f4107620 msgr2=0x7f45f419f2f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T08:43:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.077+0000 7f45f3400640 1 --2- 172.21.15.89:0/547744377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45f4107620 0x7f45f419f2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.077+0000 7f45f3400640 1 -- 172.21.15.89:0/547744377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45f4104cf0 msgr2=0x7f45f4198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.077+0000 7f45f3400640 1 --2- 172.21.15.89:0/547744377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45f4104cf0 0x7f45f4198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.077+0000 7f45f3400640 1 -- 172.21.15.89:0/547744377 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f419f960 con 0x7f45f41023c0 2024-09-10T08:43:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.077+0000 7f45eaa00640 1 --2- 172.21.15.89:0/547744377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45f4104cf0 0x7f45f4198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:43:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.077+0000 7f45f3400640 1 --2- 172.21.15.89:0/547744377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f41023c0 0x7f45f4197d40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f45e400ba50 tx=0x7f45e400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.077+0000 7f45f0c00640 1 -- 172.21.15.89:0/547744377 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4019070 con 0x7f45f41023c0 2024-09-10T08:43:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.078+0000 7f45f8e00640 1 -- 172.21.15.89:0/547744377 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f419fc50 con 0x7f45f41023c0 2024-09-10T08:43:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.078+0000 7f45f0c00640 1 -- 172.21.15.89:0/547744377 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45e400cc10 con 0x7f45f41023c0 2024-09-10T08:43:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.078+0000 7f45f0c00640 1 -- 172.21.15.89:0/547744377 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e40076a0 con 0x7f45f41023c0 2024-09-10T08:43:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.078+0000 7f45f8e00640 1 -- 172.21.15.89:0/547744377 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f41a0150 con 0x7f45f41023c0 2024-09-10T08:43:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.081+0000 7f45f0c00640 1 -- 172.21.15.89:0/547744377 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f45e401c070 con 0x7f45f41023c0 2024-09-10T08:43:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.081+0000 7f45fd434740 1 Processor -- start 2024-09-10T08:43:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.081+0000 7f45fd434740 1 -- start start 2024-09-10T08:43:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.082+0000 7f45fd434740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e0982b1670 0x55e0982b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.082+0000 7f45fd434740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1fb0 0x55e0982b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.082+0000 7f45fd434740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b7040 0x55e0982b9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.082+0000 7f45fd434740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e098212100 con 0x55e0982b1fb0 2024-09-10T08:43:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.082+0000 7f45fd434740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e0981ba1b0 con 0x55e0982b7040 2024-09-10T08:43:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.082+0000 7f45fd434740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e09820e0c0 con 0x55e0982b1670 2024-09-10T08:43:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.082+0000 7f45f3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1fb0 0x55e0982b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.082+0000 7f45eaa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b7040 0x55e0982b9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.082+0000 7f45eaa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b7040 0x55e0982b9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:49782/0 (socket says 172.21.15.89:49782) 2024-09-10T08:43:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.082+0000 7f45f3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1fb0 0x55e0982b6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56632/0 (socket says 172.21.15.89:56632) 2024-09-10T08:43:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.082+0000 7f45eaa00640 1 -- 172.21.15.89:0/2253072665 learned_addr learned my addr 172.21.15.89:0/2253072665 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.082+0000 7f45eaa00640 1 -- 172.21.15.89:0/2253072665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e0982b1670 msgr2=0x55e0982b1a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:43:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.082+0000 7f45eaa00640 1 --2- 172.21.15.89:0/2253072665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e0982b1670 0x55e0982b1a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.082+0000 7f45f0c00640 1 --2- 172.21.15.89:0/547744377 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f45d80a2b80 0x7f45d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.082+0000 7f45eaa00640 1 -- 172.21.15.89:0/2253072665 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1fb0 msgr2=0x55e0982b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.082+0000 7f45eaa00640 1 --2- 172.21.15.89:0/2253072665 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1fb0 0x55e0982b6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.082+0000 7f45eaa00640 1 -- 172.21.15.89:0/2253072665 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e097f25210 con 0x55e0982b7040 2024-09-10T08:43:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.083+0000 7f45f0c00640 1 -- 172.21.15.89:0/547744377 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f45e40ceb10 con 0x7f45f41023c0 2024-09-10T08:43:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.083+0000 7f45eaa00640 1 --2- 172.21.15.89:0/2253072665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b7040 0x55e0982b9500 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0114f0 tx=0x7f45dc0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27f94b3c1f901fa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.083+0000 7f45f3e00640 1 --2- 172.21.15.89:0/547744377 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f45d80a2b80 0x7f45d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.083+0000 7f45e8c00640 1 -- 172.21.15.89:0/2253072665 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc005020 con 0x55e0982b7040 2024-09-10T08:43:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.083+0000 7f45e8c00640 1 -- 172.21.15.89:0/2253072665 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45dc0047a0 con 0x55e0982b7040 2024-09-10T08:43:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.083+0000 7f45e8c00640 1 -- 172.21.15.89:0/2253072665 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc00f950 con 0x55e0982b7040 2024-09-10T08:43:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.084+0000 7f45fd434740 1 -- 172.21.15.89:0/2253072665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b7040 msgr2=0x55e0982b9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.084+0000 7f45f3e00640 1 --2- 172.21.15.89:0/547744377 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f45d80a2b80 0x7f45d80a5040 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f45ec005c20 tx=0x7f45ec005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.084+0000 7f45fd434740 1 --2- 172.21.15.89:0/2253072665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b7040 0x55e0982b9500 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0114f0 tx=0x7f45dc0119b0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.084+0000 7f45fd434740 1 -- 172.21.15.89:0/2253072665 shutdown_connections 2024-09-10T08:43:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.084+0000 7f45fd434740 1 --2- 172.21.15.89:0/2253072665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b7040 0x55e0982b9500 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.084+0000 7f45fd434740 1 --2- 172.21.15.89:0/2253072665 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1fb0 0x55e0982b6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.084+0000 7f45fd434740 1 --2- 172.21.15.89:0/2253072665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e0982b1670 0x55e0982b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.084+0000 7f45fd434740 1 -- 172.21.15.89:0/2253072665 >> 172.21.15.89:0/2253072665 conn(0x55e0982acce0 msgr2=0x55e0982ad0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.084+0000 7f45fd434740 1 -- 172.21.15.89:0/2253072665 shutdown_connections 2024-09-10T08:43:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.084+0000 7f45fd434740 1 -- 172.21.15.89:0/2253072665 wait complete. 2024-09-10T08:43:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.084+0000 7f45fd434740 1 -- start start 2024-09-10T08:43:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.085+0000 7f45fd434740 1 Processor -- start 2024-09-10T08:43:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.085+0000 7f45fd434740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1670 0x55e0982bb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.085+0000 7f45fd434740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e0982b1fb0 0x55e0982bbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.085+0000 7f45fd434740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b7040 0x55e0982c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.085+0000 7f45fd434740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e0982b9fb0 con 0x55e0982b1670 2024-09-10T08:43:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.085+0000 7f45fd434740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e09819b090 con 0x55e0982b7040 2024-09-10T08:43:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.085+0000 7f45fd434740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e0982ba130 con 0x55e0982b1fb0 2024-09-10T08:43:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.085+0000 7f45fd434740 10 client.0 _ll_register_callbacks cb 0x55e0981eff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:43:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.085+0000 7f45f3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1670 0x55e0982bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.085+0000 7f45f3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1670 0x55e0982bb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56644/0 (socket says 172.21.15.89:56644) 2024-09-10T08:43:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.085+0000 7f45f3400640 1 -- 172.21.15.89:0/928819384 learned_addr learned my addr 172.21.15.89:0/928819384 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.085+0000 7f45f3e00640 1 --2- 172.21.15.89:0/928819384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b7040 0x55e0982c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.086+0000 7f45f3400640 1 -- 172.21.15.89:0/928819384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e0982b1fb0 msgr2=0x55e0982bbca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:43:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.086+0000 7f45f3400640 1 --2- 172.21.15.89:0/928819384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e0982b1fb0 0x55e0982bbca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.086+0000 7f45f3400640 1 -- 172.21.15.89:0/928819384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b7040 msgr2=0x55e0982c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.086+0000 7f45f3400640 1 --2- 172.21.15.89:0/928819384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b7040 0x55e0982c2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.086+0000 7f45f3400640 1 -- 172.21.15.89:0/928819384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e097ff1620 con 0x55e0982b1670 2024-09-10T08:43:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.086+0000 7f45f3400640 1 --2- 172.21.15.89:0/928819384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1670 0x55e0982bb760 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f45e409af80 tx=0x7f45e409abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770ab2ad62883101 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.086+0000 7f45e1600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:43:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.086+0000 7f45e1600640 1 -- 172.21.15.89:0/928819384 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e409f2f0 con 0x55e0982b1670 2024-09-10T08:43:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.086+0000 7f45e1600640 1 -- 172.21.15.89:0/928819384 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f45e4097660 con 0x55e0982b1670 2024-09-10T08:43:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.086+0000 7f45fd434740 1 -- 172.21.15.89:0/928819384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45dc0047a0 con 0x55e0982b1670 2024-09-10T08:43:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.086+0000 7f45e1600640 1 -- 172.21.15.89:0/928819384 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f45e40a3bb0 con 0x55e0982b1670 2024-09-10T08:43:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.086+0000 7f45fd434740 10 client.16044 fetch_fsmap learned FSMap version 28 2024-09-10T08:43:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.086+0000 7f45fd434740 1 -- 172.21.15.89:0/928819384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e0982c5b40 con 0x55e0982b1670 2024-09-10T08:43:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.086+0000 7f45e1600640 1 -- 172.21.15.89:0/928819384 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45dc0047a0 con 0x55e0982b1670 2024-09-10T08:43:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.087+0000 7f45e1600640 1 -- 172.21.15.89:0/928819384 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f45e40b9ca0 con 0x55e0982b1670 2024-09-10T08:43:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.087+0000 7f45fd434740 10 client.16044 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:43:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.087+0000 7f45fd434740 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:43:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.087+0000 7f45fd434740 20 client.16044 populate_metadata read hostname 'smithi089' 2024-09-10T08:43:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.087+0000 7f45fd434740 1 --2- 172.21.15.89:0/928819384 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e0982c7230 0x55e0982c9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.087+0000 7f45fd434740 4 client.16044 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:43:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.087+0000 7f45fd434740 1 -- 172.21.15.89:0/928819384 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e09821f830 con 0x55e0982c7230 2024-09-10T08:43:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.104+0000 7f45eaa00640 1 --2- 172.21.15.89:0/928819384 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e0982c7230 0x55e0982c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.163+0000 7f45eaa00640 1 --2- 172.21.15.89:0/928819384 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e0982c7230 0x55e0982c9640 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67ad17840b130abf server_cookie=97ffd839db9c9639 in_seq=0 out_seq=0 2024-09-10T08:43:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.163+0000 7f45e1600640 10 client.16044 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.186+0000 7f45e1600640 1 -- 172.21.15.89:0/928819384 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e09821f830 con 0x55e0982c7230 2024-09-10T08:43:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.186+0000 7f45e1600640 10 client.16044 handle_command_reply: tid=0 2024-09-10T08:43:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.187+0000 7f45fd434740 1 client.16044 shutdown 2024-09-10T08:43:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.187+0000 7f45fd434740 1 -- 172.21.15.89:0/928819384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1670 msgr2=0x55e0982bb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.187+0000 7f45fd434740 1 --2- 172.21.15.89:0/928819384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1670 0x55e0982bb760 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f45e409af80 tx=0x7f45e409abf0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.187+0000 7f45fd434740 1 -- 172.21.15.89:0/928819384 shutdown_connections 2024-09-10T08:43:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.187+0000 7f45fd434740 1 --2- 172.21.15.89:0/928819384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b7040 0x55e0982c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.187+0000 7f45fd434740 1 --2- 172.21.15.89:0/928819384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e0982b1fb0 0x55e0982bbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.187+0000 7f45fd434740 1 --2- 172.21.15.89:0/928819384 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e0982c7230 0x55e0982c9640 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.187+0000 7f45fd434740 1 --2- 172.21.15.89:0/928819384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1670 0x55e0982bb760 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.187+0000 7f45fd434740 1 -- 172.21.15.89:0/928819384 >> 172.21.15.89:0/928819384 conn(0x55e0982acce0 msgr2=0x55e0982ae4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.187+0000 7f45e1600640 0 client.16044 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.187+0000 7f45fd434740 1 -- 172.21.15.89:0/928819384 shutdown_connections 2024-09-10T08:43:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.187+0000 7f45fd434740 1 -- 172.21.15.89:0/928819384 wait complete. 2024-09-10T08:43:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.187+0000 7f45fd434740 20 client.16044 trim_cache size 0 max 16384 2024-09-10T08:43:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.202+0000 7f45fd434740 1 Processor -- start 2024-09-10T08:43:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.202+0000 7f45fd434740 1 -- start start 2024-09-10T08:43:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.202+0000 7f45fd434740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1670 0x55e0982bbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.202+0000 7f45fd434740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b1fb0 0x55e0982c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.202+0000 7f45fd434740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e0982b7040 0x55e0982adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.202+0000 7f45fd434740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f45f41aae80 con 0x55e0982b1670 2024-09-10T08:43:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.202+0000 7f45fd434740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f45f41a8750 con 0x55e0982b1fb0 2024-09-10T08:43:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.202+0000 7f45fd434740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f45f41a65a0 con 0x55e0982b7040 2024-09-10T08:43:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.203+0000 7f45f3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1670 0x55e0982bbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.203+0000 7f45f3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e0982b7040 0x55e0982adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.203+0000 7f45f3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e0982b7040 0x55e0982adb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:56926/0 (socket says 172.21.15.89:56926) 2024-09-10T08:43:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.203+0000 7f45f3e00640 1 -- 172.21.15.89:0/327935718 learned_addr learned my addr 172.21.15.89:0/327935718 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.203+0000 7f45eaa00640 1 --2- 172.21.15.89:0/327935718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b1fb0 0x55e0982c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.203+0000 7f45f3400640 1 -- 172.21.15.89:0/327935718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e0982b7040 msgr2=0x55e0982adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.203+0000 7f45f3400640 1 --2- 172.21.15.89:0/327935718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e0982b7040 0x55e0982adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.203+0000 7f45f3400640 1 -- 172.21.15.89:0/327935718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b1fb0 msgr2=0x55e0982c2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.203+0000 7f45f3400640 1 --2- 172.21.15.89:0/327935718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b1fb0 0x55e0982c2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.203+0000 7f45f3400640 1 -- 172.21.15.89:0/327935718 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0982b2490 con 0x55e0982b1670 2024-09-10T08:43:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.203+0000 7f45eaa00640 1 --2- 172.21.15.89:0/327935718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b1fb0 0x55e0982c2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:43:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.203+0000 7f45f3400640 1 --2- 172.21.15.89:0/327935718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1670 0x55e0982bbc20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f45e4097450 tx=0x7f45e40bb170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b6308deb835e0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.204+0000 7f45e3e00640 1 -- 172.21.15.89:0/327935718 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e409f2f0 con 0x55e0982b1670 2024-09-10T08:43:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.204+0000 7f45e3e00640 1 -- 172.21.15.89:0/327935718 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45e40d5070 con 0x55e0982b1670 2024-09-10T08:43:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.204+0000 7f45e3e00640 1 -- 172.21.15.89:0/327935718 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e409a510 con 0x55e0982b1670 2024-09-10T08:43:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.204+0000 7f45fd434740 1 -- 172.21.15.89:0/327935718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1670 msgr2=0x55e0982bbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.204+0000 7f45fd434740 1 --2- 172.21.15.89:0/327935718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1670 0x55e0982bbc20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f45e4097450 tx=0x7f45e40bb170 comp rx=0 tx=0).stop 2024-09-10T08:43:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.204+0000 7f45fd434740 1 -- 172.21.15.89:0/327935718 shutdown_connections 2024-09-10T08:43:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.204+0000 7f45fd434740 1 --2- 172.21.15.89:0/327935718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e0982b7040 0x55e0982adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.204+0000 7f45fd434740 1 --2- 172.21.15.89:0/327935718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b1fb0 0x55e0982c2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.204+0000 7f45fd434740 1 --2- 172.21.15.89:0/327935718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1670 0x55e0982bbc20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.204+0000 7f45fd434740 1 -- 172.21.15.89:0/327935718 >> 172.21.15.89:0/327935718 conn(0x55e0982acce0 msgr2=0x55e0982bb510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.204+0000 7f45fd434740 1 -- 172.21.15.89:0/327935718 shutdown_connections 2024-09-10T08:43:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.204+0000 7f45fd434740 1 -- 172.21.15.89:0/327935718 wait complete. 2024-09-10T08:43:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.205+0000 7f45fd434740 1 -- start start 2024-09-10T08:43:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.205+0000 7f45fd434740 1 Processor -- start 2024-09-10T08:43:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.205+0000 7f45fd434740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b1670 0x55e0982bfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.205+0000 7f45fd434740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1fb0 0x55e0982c0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.205+0000 7f45fd434740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e0982b7040 0x55e0982c1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.205+0000 7f45fd434740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f45f41a6c10 con 0x55e0982b1fb0 2024-09-10T08:43:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.205+0000 7f45fd434740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f45f41a8080 con 0x55e0982b1670 2024-09-10T08:43:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.205+0000 7f45fd434740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f45f41a72e0 con 0x55e0982b7040 2024-09-10T08:43:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.205+0000 7f45fd434740 10 client.0 _ll_register_callbacks cb 0x7f45e4097300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:43:56.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.206+0000 7f45eaa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1fb0 0x55e0982c0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.206+0000 7f45eaa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1fb0 0x55e0982c0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56654/0 (socket says 172.21.15.89:56654) 2024-09-10T08:43:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.206+0000 7f45eaa00640 1 -- 172.21.15.89:0/746119389 learned_addr learned my addr 172.21.15.89:0/746119389 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:43:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.206+0000 7f45f3400640 1 --2- 172.21.15.89:0/746119389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b1670 0x55e0982bfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.206+0000 7f45eaa00640 1 -- 172.21.15.89:0/746119389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e0982b7040 msgr2=0x55e0982c1320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:43:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.206+0000 7f45eaa00640 1 --2- 172.21.15.89:0/746119389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e0982b7040 0x55e0982c1320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.206+0000 7f45eaa00640 1 -- 172.21.15.89:0/746119389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b1670 msgr2=0x55e0982bfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:43:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.206+0000 7f45eaa00640 1 --2- 172.21.15.89:0/746119389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b1670 0x55e0982bfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.206+0000 7f45eaa00640 1 -- 172.21.15.89:0/746119389 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0982c1930 con 0x55e0982b1fb0 2024-09-10T08:43:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.206+0000 7f45eaa00640 1 --2- 172.21.15.89:0/746119389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1fb0 0x55e0982c0310 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f45dc014040 tx=0x7f45dc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808f643209badbde server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.206+0000 7f45e1600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:43:56.209 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.206+0000 7f45e1600640 1 -- 172.21.15.89:0/746119389 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc004dc0 con 0x55e0982b1fb0 2024-09-10T08:43:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.207+0000 7f45e1600640 1 -- 172.21.15.89:0/746119389 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f45dc011040 con 0x55e0982b1fb0 2024-09-10T08:43:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.207+0000 7f45fd434740 1 -- 172.21.15.89:0/746119389 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45e40d5070 con 0x55e0982b1fb0 2024-09-10T08:43:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.207+0000 7f45e1600640 1 -- 172.21.15.89:0/746119389 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f45dc03b050 con 0x55e0982b1fb0 2024-09-10T08:43:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.207+0000 7f45fd434740 10 client.16050 fetch_fsmap learned FSMap version 28 2024-09-10T08:43:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.207+0000 7f45fd434740 1 -- 172.21.15.89:0/746119389 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e0982c2c40 con 0x55e0982b1fb0 2024-09-10T08:43:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.207+0000 7f45e1600640 1 -- 172.21.15.89:0/746119389 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45e40d5070 con 0x55e0982b1fb0 2024-09-10T08:43:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.207+0000 7f45e1600640 1 -- 172.21.15.89:0/746119389 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f45dc004320 con 0x55e0982b1fb0 2024-09-10T08:43:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.207+0000 7f45fd434740 10 client.16050 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:43:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.207+0000 7f45fd434740 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:43:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.207+0000 7f45fd434740 20 client.16050 populate_metadata read hostname 'smithi089' 2024-09-10T08:43:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.207+0000 7f45fd434740 1 --2- 172.21.15.89:0/746119389 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e0983321a0 0x55e0983345c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.207+0000 7f45fd434740 4 client.16050 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:43:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.207+0000 7f45fd434740 1 -- 172.21.15.89:0/746119389 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e098219b80 con 0x55e0983321a0 2024-09-10T08:43:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.208+0000 7f45f3400640 1 --2- 172.21.15.89:0/746119389 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e0983321a0 0x55e0983345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:56.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.208+0000 7f45f3400640 1 --2- 172.21.15.89:0/746119389 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e0983321a0 0x55e0983345c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3383a30ac07c7a52 server_cookie=1e403a761c7bea70 in_seq=0 out_seq=0 2024-09-10T08:43:56.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:56.208+0000 7f45e1600640 10 client.16050 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:56.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:55 smithi149 ceph-mon[36167]: scrub summary: idle 2024-09-10T08:43:56.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:55 smithi149 ceph-mon[36167]: scrub summary: idle 2024-09-10T08:43:56.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:55 smithi111 ceph-mon[35220]: scrub summary: idle 2024-09-10T08:43:56.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:55 smithi111 ceph-mon[35220]: scrub summary: idle 2024-09-10T08:43:57.109 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:56 smithi089 ceph-mon[30757]: scrub summary: idle 2024-09-10T08:43:57.109 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:56 smithi089 ceph-mon[30757]: pgmap v513: 129 pgs: 129 active+clean; 313 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 1.41k op/s 2024-09-10T08:43:57.187 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.187+0000 7f45e1600640 1 -- 172.21.15.89:0/746119389 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e098219b80 con 0x55e0983321a0 2024-09-10T08:43:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.187+0000 7f45e1600640 10 client.16050 handle_command_reply: tid=0 2024-09-10T08:43:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.188+0000 7f45fd434740 1 client.16050 shutdown 2024-09-10T08:43:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.188+0000 7f45fd434740 1 -- 172.21.15.89:0/746119389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1fb0 msgr2=0x55e0982c0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.188+0000 7f45fd434740 1 --2- 172.21.15.89:0/746119389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1fb0 0x55e0982c0310 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f45dc014040 tx=0x7f45dc005850 comp rx=0 tx=0).stop 2024-09-10T08:43:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.188+0000 7f45fd434740 1 -- 172.21.15.89:0/746119389 shutdown_connections 2024-09-10T08:43:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.188+0000 7f45fd434740 1 --2- 172.21.15.89:0/746119389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e0982b7040 0x55e0982c1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.188+0000 7f45fd434740 1 --2- 172.21.15.89:0/746119389 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e0983321a0 0x55e0983345c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.188+0000 7f45fd434740 1 --2- 172.21.15.89:0/746119389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0982b1fb0 0x55e0982c0310 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.188+0000 7f45fd434740 1 --2- 172.21.15.89:0/746119389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0982b1670 0x55e0982bfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.188+0000 7f45fd434740 1 -- 172.21.15.89:0/746119389 >> 172.21.15.89:0/746119389 conn(0x55e0982acce0 msgr2=0x55e0982a71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.188+0000 7f45e1600640 0 client.16050 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:43:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.190+0000 7f45fd434740 1 -- 172.21.15.89:0/746119389 shutdown_connections 2024-09-10T08:43:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.190+0000 7f45fd434740 1 -- 172.21.15.89:0/746119389 wait complete. 2024-09-10T08:43:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.191+0000 7f45fd434740 20 client.16050 trim_cache size 0 max 16384 2024-09-10T08:43:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:43:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:43:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:43:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:43:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.191+0000 7f45f8e00640 1 -- 172.21.15.89:0/547744377 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f45d80a2b80 msgr2=0x7f45d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.191+0000 7f45f8e00640 1 --2- 172.21.15.89:0/547744377 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f45d80a2b80 0x7f45d80a5040 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f45ec005c20 tx=0x7f45ec005fd0 comp rx=0 tx=0).stop 2024-09-10T08:43:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.191+0000 7f45f8e00640 1 -- 172.21.15.89:0/547744377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f41023c0 msgr2=0x7f45f4197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.191+0000 7f45f8e00640 1 --2- 172.21.15.89:0/547744377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f41023c0 0x7f45f4197d40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f45e400ba50 tx=0x7f45e400bf10 comp rx=0 tx=0).stop 2024-09-10T08:43:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.192+0000 7f45f8e00640 1 -- 172.21.15.89:0/547744377 shutdown_connections 2024-09-10T08:43:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.192+0000 7f45f8e00640 1 --2- 172.21.15.89:0/547744377 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f45d80a2b80 0x7f45d80a5040 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.192+0000 7f45f8e00640 1 --2- 172.21.15.89:0/547744377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45f4107620 0x7f45f419f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.192+0000 7f45f8e00640 1 --2- 172.21.15.89:0/547744377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45f4104cf0 0x7f45f4198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.192+0000 7f45f8e00640 1 --2- 172.21.15.89:0/547744377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45f41023c0 0x7f45f4197d40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.192+0000 7f45f8e00640 1 -- 172.21.15.89:0/547744377 >> 172.21.15.89:0/547744377 conn(0x7f45f40fbfa0 msgr2=0x7f45f4102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.192+0000 7f45f8e00640 1 -- 172.21.15.89:0/547744377 shutdown_connections 2024-09-10T08:43:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:43:57.192+0000 7f45f8e00640 1 -- 172.21.15.89:0/547744377 wait complete. 2024-09-10T08:43:57.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:56 smithi149 ceph-mon[36167]: scrub summary: idle 2024-09-10T08:43:57.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:56 smithi149 ceph-mon[36167]: pgmap v513: 129 pgs: 129 active+clean; 313 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 1.41k op/s 2024-09-10T08:43:57.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:56 smithi111 ceph-mon[35220]: scrub summary: idle 2024-09-10T08:43:57.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:56 smithi111 ceph-mon[35220]: pgmap v513: 129 pgs: 129 active+clean; 313 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 1.41k op/s 2024-09-10T08:43:58.042 INFO:tasks.workunit.client.0.smithi089.stdout: 87 0 0 0 0 0 0 2024-09-10T08:43:58.217 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:43:57 smithi089 ceph-mon[30757]: pgmap v514: 129 pgs: 129 active+clean; 313 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 1.46k op/s 2024-09-10T08:43:58.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:43:57 smithi149 ceph-mon[36167]: pgmap v514: 129 pgs: 129 active+clean; 313 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 1.46k op/s 2024-09-10T08:43:58.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:43:57 smithi111 ceph-mon[35220]: pgmap v514: 129 pgs: 129 active+clean; 313 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 1.46k op/s 2024-09-10T08:44:01.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:00 smithi111 ceph-mon[35220]: pgmap v515: 129 pgs: 129 active+clean; 313 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 1.31k op/s 2024-09-10T08:44:01.046 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:00 smithi089 ceph-mon[30757]: pgmap v515: 129 pgs: 129 active+clean; 313 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 1.31k op/s 2024-09-10T08:44:01.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:00 smithi149 ceph-mon[36167]: pgmap v515: 129 pgs: 129 active+clean; 313 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 1.31k op/s 2024-09-10T08:44:03.022 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:02 smithi089 ceph-mon[30757]: pgmap v516: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 1.68k op/s 2024-09-10T08:44:03.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:02 smithi111 ceph-mon[35220]: pgmap v516: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 1.68k op/s 2024-09-10T08:44:03.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:02 smithi149 ceph-mon[36167]: pgmap v516: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 1.68k op/s 2024-09-10T08:44:04.953 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:04 smithi089 ceph-mon[30757]: pgmap v517: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 951 KiB/s wr, 1.21k op/s 2024-09-10T08:44:05.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:04 smithi111 ceph-mon[35220]: pgmap v517: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 951 KiB/s wr, 1.21k op/s 2024-09-10T08:44:05.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:04 smithi149 ceph-mon[36167]: pgmap v517: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 951 KiB/s wr, 1.21k op/s 2024-09-10T08:44:06.905 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:06 smithi089 ceph-mon[30757]: pgmap v518: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 529 KiB/s wr, 1.13k op/s 2024-09-10T08:44:07.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:06 smithi111 ceph-mon[35220]: pgmap v518: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 529 KiB/s wr, 1.13k op/s 2024-09-10T08:44:07.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:06 smithi149 ceph-mon[36167]: pgmap v518: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 529 KiB/s wr, 1.13k op/s 2024-09-10T08:44:08.042 INFO:tasks.workunit.client.0.smithi089.stdout: 87 0 0 0 0 0 0 2024-09-10T08:44:08.906 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:08 smithi089 ceph-mon[30757]: pgmap v519: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 284 KiB/s wr, 785 op/s 2024-09-10T08:44:09.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:08 smithi111 ceph-mon[35220]: pgmap v519: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 284 KiB/s wr, 785 op/s 2024-09-10T08:44:09.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:08 smithi149 ceph-mon[36167]: pgmap v519: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 284 KiB/s wr, 785 op/s 2024-09-10T08:44:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.305+0000 7f724ee00640 1 -- 172.21.15.89:0/793723616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7248105340 msgr2=0x7f7248110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.305+0000 7f724ee00640 1 --2- 172.21.15.89:0/793723616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7248105340 0x7f7248110490 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7240009ca0 tx=0x7f724002a8d0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.305+0000 7f724ee00640 1 -- 172.21.15.89:0/793723616 shutdown_connections 2024-09-10T08:44:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.305+0000 7f724ee00640 1 --2- 172.21.15.89:0/793723616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7248105340 0x7f7248110490 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.305+0000 7f724ee00640 1 --2- 172.21.15.89:0/793723616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7248074fc0 0x7f724810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.305+0000 7f724ee00640 1 --2- 172.21.15.89:0/793723616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7248074680 0x7f7248074a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.305+0000 7f724ee00640 1 -- 172.21.15.89:0/793723616 >> 172.21.15.89:0/793723616 conn(0x7f72480fd0d0 msgr2=0x7f72480ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.305+0000 7f724ee00640 1 -- 172.21.15.89:0/793723616 shutdown_connections 2024-09-10T08:44:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.305+0000 7f724ee00640 1 -- 172.21.15.89:0/793723616 wait complete. 2024-09-10T08:44:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.306+0000 7f724ee00640 1 Processor -- start 2024-09-10T08:44:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.306+0000 7f724ee00640 1 -- start start 2024-09-10T08:44:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.306+0000 7f724ee00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7248074680 0x7f724819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.307+0000 7f724ee00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7248074fc0 0x7f724819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.307+0000 7f724ee00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72481a39b0 0x7f72481a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.307+0000 7f724ee00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7248113d80 con 0x7f7248074680 2024-09-10T08:44:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.307+0000 7f724ee00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7248113c00 con 0x7f7248074fc0 2024-09-10T08:44:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.307+0000 7f724ee00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7248113f00 con 0x7f72481a39b0 2024-09-10T08:44:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.307+0000 7f724da00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7248074680 0x7f724819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.307+0000 7f724da00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7248074680 0x7f724819e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41366/0 (socket says 172.21.15.89:41366) 2024-09-10T08:44:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.307+0000 7f724da00640 1 -- 172.21.15.89:0/2341350291 learned_addr learned my addr 172.21.15.89:0/2341350291 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.307+0000 7f724e400640 1 --2- 172.21.15.89:0/2341350291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72481a39b0 0x7f72481a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.307+0000 7f724da00640 1 -- 172.21.15.89:0/2341350291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72481a39b0 msgr2=0x7f72481a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.307+0000 7f724da00640 1 --2- 172.21.15.89:0/2341350291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72481a39b0 0x7f72481a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.307+0000 7f724da00640 1 -- 172.21.15.89:0/2341350291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7248074fc0 msgr2=0x7f724819e950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T08:44:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.307+0000 7f724da00640 1 --2- 172.21.15.89:0/2341350291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7248074fc0 0x7f724819e950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.307+0000 7f724da00640 1 -- 172.21.15.89:0/2341350291 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72481a6410 con 0x7f7248074680 2024-09-10T08:44:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.307+0000 7f724e400640 1 --2- 172.21.15.89:0/2341350291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72481a39b0 0x7f72481a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:44:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.307+0000 7f724da00640 1 --2- 172.21.15.89:0/2341350291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7248074680 0x7f724819e410 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f723c00b450 tx=0x7f723c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.308+0000 7f7246000640 1 -- 172.21.15.89:0/2341350291 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c015020 con 0x7f7248074680 2024-09-10T08:44:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.308+0000 7f7246000640 1 -- 172.21.15.89:0/2341350291 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f723c011d70 con 0x7f7248074680 2024-09-10T08:44:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.308+0000 7f7246000640 1 -- 172.21.15.89:0/2341350291 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c00f340 con 0x7f7248074680 2024-09-10T08:44:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.308+0000 7f724ee00640 1 -- 172.21.15.89:0/2341350291 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72481a6700 con 0x7f7248074680 2024-09-10T08:44:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.308+0000 7f724ee00640 1 -- 172.21.15.89:0/2341350291 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72481a6c80 con 0x7f7248074680 2024-09-10T08:44:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.311+0000 7f72532ad740 1 Processor -- start 2024-09-10T08:44:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.311+0000 7f72532ad740 1 -- start start 2024-09-10T08:44:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.311+0000 7f7246000640 1 -- 172.21.15.89:0/2341350291 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f723c018020 con 0x7f7248074680 2024-09-10T08:44:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.311+0000 7f72532ad740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a846086260 0x55a846086660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.311+0000 7f72532ad740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a846086ba0 0x55a84608b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.311+0000 7f72532ad740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a84608bc30 0x55a84608e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.311+0000 7f72532ad740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a845feb100 con 0x55a846086260 2024-09-10T08:44:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.312+0000 7f72532ad740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a845f931b0 con 0x55a846086ba0 2024-09-10T08:44:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.312+0000 7f72532ad740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a845fe70c0 con 0x55a84608bc30 2024-09-10T08:44:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.312+0000 7f724e400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a846086260 0x55a846086660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.312+0000 7f724e400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a846086260 0x55a846086660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41374/0 (socket says 172.21.15.89:41374) 2024-09-10T08:44:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.312+0000 7f724e400640 1 -- 172.21.15.89:0/2376745485 learned_addr learned my addr 172.21.15.89:0/2376745485 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.312+0000 7f724d000640 1 --2- 172.21.15.89:0/2376745485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a84608bc30 0x55a84608e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.312+0000 7f724da00640 1 --2- 172.21.15.89:0/2376745485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a846086ba0 0x55a84608b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.312+0000 7f724e400640 1 -- 172.21.15.89:0/2376745485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a84608bc30 msgr2=0x55a84608e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.312+0000 7f724e400640 1 --2- 172.21.15.89:0/2376745485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a84608bc30 0x55a84608e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.312+0000 7f724e400640 1 -- 172.21.15.89:0/2376745485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a846086ba0 msgr2=0x55a84608b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.312+0000 7f724e400640 1 --2- 172.21.15.89:0/2376745485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a846086ba0 0x55a84608b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.312+0000 7f724e400640 1 -- 172.21.15.89:0/2376745485 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a845cfe210 con 0x55a846086260 2024-09-10T08:44:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.313+0000 7f724e400640 1 --2- 172.21.15.89:0/2376745485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a846086260 0x55a846086660 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7240009420 tx=0x7f724002f320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e5a69bd6a77c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.313+0000 7f723aa00640 1 -- 172.21.15.89:0/2376745485 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7240031460 con 0x55a846086260 2024-09-10T08:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.313+0000 7f723aa00640 1 -- 172.21.15.89:0/2376745485 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7240036070 con 0x55a846086260 2024-09-10T08:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.313+0000 7f723aa00640 1 -- 172.21.15.89:0/2376745485 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7240004570 con 0x55a846086260 2024-09-10T08:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.313+0000 7f7246000640 1 --2- 172.21.15.89:0/2341350291 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f72300a2c00 0x7f72300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.313+0000 7f7246000640 1 -- 172.21.15.89:0/2341350291 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f723c0cef40 con 0x7f7248074680 2024-09-10T08:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.313+0000 7f724da00640 1 --2- 172.21.15.89:0/2341350291 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f72300a2c00 0x7f72300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.314+0000 7f72532ad740 1 -- 172.21.15.89:0/2376745485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a846086260 msgr2=0x55a846086660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.314+0000 7f72532ad740 1 --2- 172.21.15.89:0/2376745485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a846086260 0x55a846086660 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7240009420 tx=0x7f724002f320 comp rx=0 tx=0).stop 2024-09-10T08:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.314+0000 7f72532ad740 1 -- 172.21.15.89:0/2376745485 shutdown_connections 2024-09-10T08:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.314+0000 7f72532ad740 1 --2- 172.21.15.89:0/2376745485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a84608bc30 0x55a84608e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.314+0000 7f72532ad740 1 --2- 172.21.15.89:0/2376745485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a846086ba0 0x55a84608b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.314+0000 7f72532ad740 1 --2- 172.21.15.89:0/2376745485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a846086260 0x55a846086660 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.314+0000 7f72532ad740 1 -- 172.21.15.89:0/2376745485 >> 172.21.15.89:0/2376745485 conn(0x55a8460818d0 msgr2=0x55a846081cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.314+0000 7f724da00640 1 --2- 172.21.15.89:0/2341350291 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f72300a2c00 0x7f72300a50c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f723c0cc9e0 tx=0x7f723c0d2b00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.314+0000 7f72532ad740 1 -- 172.21.15.89:0/2376745485 shutdown_connections 2024-09-10T08:44:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.314+0000 7f72532ad740 1 -- 172.21.15.89:0/2376745485 wait complete. 2024-09-10T08:44:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.314+0000 7f72532ad740 1 -- start start 2024-09-10T08:44:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.315+0000 7f72532ad740 1 Processor -- start 2024-09-10T08:44:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.315+0000 7f72532ad740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a846086260 0x55a8460903b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.315+0000 7f72532ad740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a846086ba0 0x55a8460908f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.315+0000 7f72532ad740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a84608bc30 0x55a846097930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.315+0000 7f72532ad740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a84608eba0 con 0x55a846086260 2024-09-10T08:44:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.315+0000 7f72532ad740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a845f74090 con 0x55a84608bc30 2024-09-10T08:44:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.315+0000 7f72532ad740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a84608ed20 con 0x55a846086ba0 2024-09-10T08:44:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.315+0000 7f72532ad740 10 client.0 _ll_register_callbacks cb 0x55a845fc8f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.315+0000 7f724e400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a846086260 0x55a8460903b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.315+0000 7f724e400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a846086260 0x55a8460903b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41386/0 (socket says 172.21.15.89:41386) 2024-09-10T08:44:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.315+0000 7f724d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a846086ba0 0x55a8460908f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.315+0000 7f724e400640 1 -- 172.21.15.89:0/3607296347 learned_addr learned my addr 172.21.15.89:0/3607296347 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.316+0000 7f724e400640 1 --2- 172.21.15.89:0/3607296347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a84608bc30 0x55a846097930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.316+0000 7f724d000640 1 -- 172.21.15.89:0/3607296347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a84608bc30 msgr2=0x55a846097930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.316+0000 7f724d000640 1 --2- 172.21.15.89:0/3607296347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a84608bc30 0x55a846097930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.316+0000 7f724d000640 1 -- 172.21.15.89:0/3607296347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a846086260 msgr2=0x55a8460903b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.316+0000 7f724d000640 1 --2- 172.21.15.89:0/3607296347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a846086260 0x55a8460903b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.316+0000 7f724d000640 1 -- 172.21.15.89:0/3607296347 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a845dca620 con 0x55a846086ba0 2024-09-10T08:44:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.316+0000 7f724e400640 1 --2- 172.21.15.89:0/3607296347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a84608bc30 0x55a846097930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.316+0000 7f724d000640 1 --2- 172.21.15.89:0/3607296347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a846086ba0 0x55a8460908f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f724819f920 tx=0x7f7234011920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1658a749533b5052 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.316+0000 7f722f400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:44:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.316+0000 7f722f400640 1 -- 172.21.15.89:0/3607296347 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7234005020 con 0x55a846086ba0 2024-09-10T08:44:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.316+0000 7f72532ad740 1 -- 172.21.15.89:0/3607296347 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7240036070 con 0x55a846086ba0 2024-09-10T08:44:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.316+0000 7f722f400640 1 -- 172.21.15.89:0/3607296347 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f72340051c0 con 0x55a846086ba0 2024-09-10T08:44:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.317+0000 7f722f400640 1 -- 172.21.15.89:0/3607296347 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7234023050 con 0x55a846086ba0 2024-09-10T08:44:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.317+0000 7f72532ad740 10 client.26059 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.317+0000 7f72532ad740 1 -- 172.21.15.89:0/3607296347 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a84609a7f0 con 0x55a846086ba0 2024-09-10T08:44:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.317+0000 7f722f400640 1 -- 172.21.15.89:0/3607296347 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7240036070 con 0x55a846086ba0 2024-09-10T08:44:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.317+0000 7f722f400640 1 -- 172.21.15.89:0/3607296347 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7234003710 con 0x55a846086ba0 2024-09-10T08:44:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.317+0000 7f72532ad740 10 client.26059 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.317+0000 7f72532ad740 10 client.26059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:44:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.317+0000 7f72532ad740 20 client.26059 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.317+0000 7f72532ad740 1 --2- 172.21.15.89:0/3607296347 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a84609bef0 0x55a84609e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.317+0000 7f72532ad740 4 client.26059 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:44:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.317+0000 7f72532ad740 1 -- 172.21.15.89:0/3607296347 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a845ff8830 con 0x55a84609bef0 2024-09-10T08:44:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.318+0000 7f724e400640 1 --2- 172.21.15.89:0/3607296347 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a84609bef0 0x55a84609e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.318+0000 7f724e400640 1 --2- 172.21.15.89:0/3607296347 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a84609bef0 0x55a84609e370 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a13fd0f044798cd server_cookie=8da61760effbc1e1 in_seq=0 out_seq=0 2024-09-10T08:44:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.318+0000 7f722f400640 10 client.26059 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.319+0000 7f722f400640 1 -- 172.21.15.89:0/3607296347 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a845ff8830 con 0x55a84609bef0 2024-09-10T08:44:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.319+0000 7f722f400640 10 client.26059 handle_command_reply: tid=0 2024-09-10T08:44:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.319+0000 7f72532ad740 1 client.26059 shutdown 2024-09-10T08:44:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.319+0000 7f72532ad740 1 -- 172.21.15.89:0/3607296347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a846086ba0 msgr2=0x55a8460908f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.319+0000 7f72532ad740 1 --2- 172.21.15.89:0/3607296347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a846086ba0 0x55a8460908f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f724819f920 tx=0x7f7234011920 comp rx=0 tx=0).stop 2024-09-10T08:44:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.319+0000 7f72532ad740 1 -- 172.21.15.89:0/3607296347 shutdown_connections 2024-09-10T08:44:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.319+0000 7f72532ad740 1 --2- 172.21.15.89:0/3607296347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a84608bc30 0x55a846097930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.319+0000 7f72532ad740 1 --2- 172.21.15.89:0/3607296347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a846086ba0 0x55a8460908f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.319+0000 7f72532ad740 1 --2- 172.21.15.89:0/3607296347 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a84609bef0 0x55a84609e370 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.319+0000 7f72532ad740 1 --2- 172.21.15.89:0/3607296347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a846086260 0x55a8460903b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.319+0000 7f72532ad740 1 -- 172.21.15.89:0/3607296347 >> 172.21.15.89:0/3607296347 conn(0x55a8460818d0 msgr2=0x55a8460830e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.319+0000 7f722f400640 0 client.26059 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.320+0000 7f72532ad740 1 -- 172.21.15.89:0/3607296347 shutdown_connections 2024-09-10T08:44:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.320+0000 7f72532ad740 1 -- 172.21.15.89:0/3607296347 wait complete. 2024-09-10T08:44:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.320+0000 7f72532ad740 20 client.26059 trim_cache size 0 max 16384 2024-09-10T08:44:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.335+0000 7f72532ad740 1 Processor -- start 2024-09-10T08:44:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.335+0000 7f72532ad740 1 -- start start 2024-09-10T08:44:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.335+0000 7f72532ad740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a846086260 0x55a846086640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.335+0000 7f72532ad740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a846086ba0 0x55a8460978f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.335+0000 7f72532ad740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a84608bc30 0x55a846080900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.335+0000 7f72532ad740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f72481b3660 con 0x55a84608bc30 2024-09-10T08:44:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.335+0000 7f72532ad740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f72481b0f30 con 0x55a846086ba0 2024-09-10T08:44:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.335+0000 7f72532ad740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f72481aed80 con 0x55a846086260 2024-09-10T08:44:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.335+0000 7f724e400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a84608bc30 0x55a846080900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.335+0000 7f724e400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a84608bc30 0x55a846080900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41398/0 (socket says 172.21.15.89:41398) 2024-09-10T08:44:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.335+0000 7f724e400640 1 -- 172.21.15.89:0/2087028046 learned_addr learned my addr 172.21.15.89:0/2087028046 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.335+0000 7f724e400640 1 --2- 172.21.15.89:0/2087028046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a846086260 0x55a846086640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.335+0000 7f724d000640 1 --2- 172.21.15.89:0/2087028046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a846086ba0 0x55a8460978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.336+0000 7f724e400640 1 -- 172.21.15.89:0/2087028046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a846086260 msgr2=0x55a846086640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.336+0000 7f724e400640 1 --2- 172.21.15.89:0/2087028046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a846086260 0x55a846086640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.336+0000 7f724e400640 1 -- 172.21.15.89:0/2087028046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a846086ba0 msgr2=0x55a8460978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.336+0000 7f724e400640 1 --2- 172.21.15.89:0/2087028046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a846086ba0 0x55a8460978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.336+0000 7f724e400640 1 -- 172.21.15.89:0/2087028046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a846087830 con 0x55a84608bc30 2024-09-10T08:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.336+0000 7f724e400640 1 --2- 172.21.15.89:0/2087028046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a84608bc30 0x55a846080900 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7240005dc0 tx=0x7f724002a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3b4f1c4ea39205 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.336+0000 7f723a000640 1 -- 172.21.15.89:0/2087028046 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7240031860 con 0x55a84608bc30 2024-09-10T08:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.336+0000 7f723a000640 1 -- 172.21.15.89:0/2087028046 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7240036040 con 0x55a84608bc30 2024-09-10T08:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.336+0000 7f723a000640 1 -- 172.21.15.89:0/2087028046 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724002f670 con 0x55a84608bc30 2024-09-10T08:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.337+0000 7f72532ad740 1 -- 172.21.15.89:0/2087028046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a84608bc30 msgr2=0x55a846080900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.337+0000 7f72532ad740 1 --2- 172.21.15.89:0/2087028046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a84608bc30 0x55a846080900 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7240005dc0 tx=0x7f724002a8d0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.337+0000 7f72532ad740 1 -- 172.21.15.89:0/2087028046 shutdown_connections 2024-09-10T08:44:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.337+0000 7f72532ad740 1 --2- 172.21.15.89:0/2087028046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a84608bc30 0x55a846080900 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.337+0000 7f72532ad740 1 --2- 172.21.15.89:0/2087028046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a846086ba0 0x55a8460978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.337+0000 7f72532ad740 1 --2- 172.21.15.89:0/2087028046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a846086260 0x55a846086640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.337+0000 7f72532ad740 1 -- 172.21.15.89:0/2087028046 >> 172.21.15.89:0/2087028046 conn(0x55a8460818d0 msgr2=0x55a8460801f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.337+0000 7f72532ad740 1 -- 172.21.15.89:0/2087028046 shutdown_connections 2024-09-10T08:44:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.337+0000 7f72532ad740 1 -- 172.21.15.89:0/2087028046 wait complete. 2024-09-10T08:44:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.337+0000 7f72532ad740 1 -- start start 2024-09-10T08:44:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.337+0000 7f72532ad740 1 Processor -- start 2024-09-10T08:44:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.338+0000 7f72532ad740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a846086260 0x55a846097810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.338+0000 7f72532ad740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a846086ba0 0x55a8460942b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.338+0000 7f72532ad740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a84608bc30 0x55a8460947f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.338+0000 7f72532ad740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f72481af3f0 con 0x55a846086ba0 2024-09-10T08:44:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.338+0000 7f72532ad740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f72481b0860 con 0x55a846086260 2024-09-10T08:44:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.338+0000 7f72532ad740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f72481afac0 con 0x55a84608bc30 2024-09-10T08:44:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.338+0000 7f72532ad740 10 client.0 _ll_register_callbacks cb 0x7f72340096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.338+0000 7f724e400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a846086260 0x55a846097810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.338+0000 7f724d000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a846086ba0 0x55a8460942b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.338+0000 7f724e400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a846086260 0x55a846097810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:40132/0 (socket says 172.21.15.89:40132) 2024-09-10T08:44:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.338+0000 7f724e400640 1 -- 172.21.15.89:0/3852200829 learned_addr learned my addr 172.21.15.89:0/3852200829 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.338+0000 7f724e400640 1 --2- 172.21.15.89:0/3852200829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a84608bc30 0x55a8460947f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.338+0000 7f724d000640 1 -- 172.21.15.89:0/3852200829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a84608bc30 msgr2=0x55a8460947f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.338+0000 7f724d000640 1 --2- 172.21.15.89:0/3852200829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a84608bc30 0x55a8460947f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.338+0000 7f724d000640 1 -- 172.21.15.89:0/3852200829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a846086260 msgr2=0x55a846097810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.338+0000 7f724d000640 1 --2- 172.21.15.89:0/3852200829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a846086260 0x55a846097810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.338+0000 7f724d000640 1 -- 172.21.15.89:0/3852200829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a846104c00 con 0x55a846086ba0 2024-09-10T08:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.338+0000 7f724e400640 1 --2- 172.21.15.89:0/3852200829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a84608bc30 0x55a8460947f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.339+0000 7f724d000640 1 --2- 172.21.15.89:0/3852200829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a846086ba0 0x55a8460942b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7234003cd0 tx=0x7f7234003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9656634ca9cb8103 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.339+0000 7f722f400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.339+0000 7f722f400640 1 -- 172.21.15.89:0/3852200829 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7234000e60 con 0x55a846086ba0 2024-09-10T08:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.339+0000 7f722f400640 1 -- 172.21.15.89:0/3852200829 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f7234003870 con 0x55a846086ba0 2024-09-10T08:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.339+0000 7f72532ad740 1 -- 172.21.15.89:0/3852200829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7240036040 con 0x55a846086ba0 2024-09-10T08:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.339+0000 7f722f400640 1 -- 172.21.15.89:0/3852200829 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f72340053a0 con 0x55a846086ba0 2024-09-10T08:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.339+0000 7f72532ad740 10 client.16068 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.339+0000 7f72532ad740 1 -- 172.21.15.89:0/3852200829 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a846105f00 con 0x55a846086ba0 2024-09-10T08:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.340+0000 7f722f400640 1 -- 172.21.15.89:0/3852200829 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7240036040 con 0x55a846086ba0 2024-09-10T08:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.340+0000 7f722f400640 1 -- 172.21.15.89:0/3852200829 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7234019430 con 0x55a846086ba0 2024-09-10T08:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.340+0000 7f72532ad740 10 client.16068 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.340+0000 7f72532ad740 10 client.16068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.340+0000 7f72532ad740 20 client.16068 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.340+0000 7f72532ad740 1 --2- 172.21.15.89:0/3852200829 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a846106990 0x55a846108db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.340+0000 7f72532ad740 4 client.16068 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.340+0000 7f72532ad740 1 -- 172.21.15.89:0/3852200829 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a845ff2b80 con 0x55a846106990 2024-09-10T08:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.340+0000 7f724e400640 1 --2- 172.21.15.89:0/3852200829 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a846106990 0x55a846108db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.341+0000 7f722f400640 10 client.16068 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.341+0000 7f724e400640 1 --2- 172.21.15.89:0/3852200829 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a846106990 0x55a846108db0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2c5b9813b185d09 server_cookie=bd60ae73b73a98fa in_seq=0 out_seq=0 2024-09-10T08:44:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.356+0000 7f722f400640 1 -- 172.21.15.89:0/3852200829 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a845ff2b80 con 0x55a846106990 2024-09-10T08:44:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.356+0000 7f722f400640 10 client.16068 handle_command_reply: tid=0 2024-09-10T08:44:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.356+0000 7f72532ad740 1 client.16068 shutdown 2024-09-10T08:44:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.356+0000 7f72532ad740 1 -- 172.21.15.89:0/3852200829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a846086ba0 msgr2=0x55a8460942b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.356+0000 7f72532ad740 1 --2- 172.21.15.89:0/3852200829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a846086ba0 0x55a8460942b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7234003cd0 tx=0x7f7234003f80 comp rx=0 tx=0).stop 2024-09-10T08:44:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.356+0000 7f72532ad740 1 -- 172.21.15.89:0/3852200829 shutdown_connections 2024-09-10T08:44:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.356+0000 7f72532ad740 1 --2- 172.21.15.89:0/3852200829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a84608bc30 0x55a8460947f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.356+0000 7f72532ad740 1 --2- 172.21.15.89:0/3852200829 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a846106990 0x55a846108db0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.356+0000 7f72532ad740 1 --2- 172.21.15.89:0/3852200829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a846086ba0 0x55a8460942b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.356+0000 7f72532ad740 1 --2- 172.21.15.89:0/3852200829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a846086260 0x55a846097810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.356+0000 7f722f400640 0 client.16068 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.356+0000 7f72532ad740 1 -- 172.21.15.89:0/3852200829 >> 172.21.15.89:0/3852200829 conn(0x55a8460818d0 msgr2=0x55a846080a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.356+0000 7f72532ad740 1 -- 172.21.15.89:0/3852200829 shutdown_connections 2024-09-10T08:44:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.356+0000 7f72532ad740 1 -- 172.21.15.89:0/3852200829 wait complete. 2024-09-10T08:44:10.358 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.356+0000 7f72532ad740 20 client.16068 trim_cache size 0 max 16384 2024-09-10T08:44:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:44:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:44:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:44:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:44:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.357+0000 7f724ee00640 1 -- 172.21.15.89:0/2341350291 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f72300a2c00 msgr2=0x7f72300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.357+0000 7f724ee00640 1 --2- 172.21.15.89:0/2341350291 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f72300a2c00 0x7f72300a50c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f723c0cc9e0 tx=0x7f723c0d2b00 comp rx=0 tx=0).stop 2024-09-10T08:44:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.357+0000 7f724ee00640 1 -- 172.21.15.89:0/2341350291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7248074680 msgr2=0x7f724819e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.357+0000 7f724ee00640 1 --2- 172.21.15.89:0/2341350291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7248074680 0x7f724819e410 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f723c00b450 tx=0x7f723c00b910 comp rx=0 tx=0).stop 2024-09-10T08:44:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.357+0000 7f724ee00640 1 -- 172.21.15.89:0/2341350291 shutdown_connections 2024-09-10T08:44:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.357+0000 7f724ee00640 1 --2- 172.21.15.89:0/2341350291 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f72300a2c00 0x7f72300a50c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.357+0000 7f724ee00640 1 --2- 172.21.15.89:0/2341350291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72481a39b0 0x7f72481a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.360 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.357+0000 7f724ee00640 1 --2- 172.21.15.89:0/2341350291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7248074fc0 0x7f724819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.357+0000 7f724ee00640 1 --2- 172.21.15.89:0/2341350291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7248074680 0x7f724819e410 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.357+0000 7f724ee00640 1 -- 172.21.15.89:0/2341350291 >> 172.21.15.89:0/2341350291 conn(0x7f72480fd0d0 msgr2=0x7f7248110010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.357+0000 7f724ee00640 1 -- 172.21.15.89:0/2341350291 shutdown_connections 2024-09-10T08:44:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:10.357+0000 7f724ee00640 1 -- 172.21.15.89:0/2341350291 wait complete. 2024-09-10T08:44:10.944 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:10 smithi089 ceph-mon[30757]: pgmap v520: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 242 KiB/s wr, 473 op/s 2024-09-10T08:44:11.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:10 smithi149 ceph-mon[36167]: pgmap v520: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 242 KiB/s wr, 473 op/s 2024-09-10T08:44:11.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:10 smithi111 ceph-mon[35220]: pgmap v520: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 242 KiB/s wr, 473 op/s 2024-09-10T08:44:12.954 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:12 smithi089 ceph-mon[30757]: pgmap v521: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 218 KiB/s wr, 422 op/s 2024-09-10T08:44:13.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:12 smithi149 ceph-mon[36167]: pgmap v521: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 218 KiB/s wr, 422 op/s 2024-09-10T08:44:13.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:12 smithi111 ceph-mon[35220]: pgmap v521: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 218 KiB/s wr, 422 op/s 2024-09-10T08:44:14.978 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:14 smithi089 ceph-mon[30757]: pgmap v522: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 29 op/s 2024-09-10T08:44:15.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:14 smithi149 ceph-mon[36167]: pgmap v522: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 29 op/s 2024-09-10T08:44:15.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:14 smithi111 ceph-mon[35220]: pgmap v522: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 29 op/s 2024-09-10T08:44:15.721 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:44:16.103 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:16.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.808+0000 7fa5c1e00640 1 -- 172.21.15.89:0/2636041456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5bc1019c0 msgr2=0x7fa5bc101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:16.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.808+0000 7fa5c1e00640 1 --2- 172.21.15.89:0/2636041456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5bc1019c0 0x7fa5bc101e20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009930 tx=0x7fa5a802a420 comp rx=0 tx=0).stop 2024-09-10T08:44:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.809+0000 7fa5c1e00640 1 -- 172.21.15.89:0/2636041456 shutdown_connections 2024-09-10T08:44:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.809+0000 7fa5c1e00640 1 --2- 172.21.15.89:0/2636041456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5bc102360 0x7fa5bc10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.809+0000 7fa5c1e00640 1 --2- 172.21.15.89:0/2636041456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5bc1019c0 0x7fa5bc101e20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.809+0000 7fa5c1e00640 1 --2- 172.21.15.89:0/2636041456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5bc1079c0 0x7fa5bc107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.809+0000 7fa5c1e00640 1 -- 172.21.15.89:0/2636041456 >> 172.21.15.89:0/2636041456 conn(0x7fa5bc0fd740 msgr2=0x7fa5bc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.809+0000 7fa5c1e00640 1 -- 172.21.15.89:0/2636041456 shutdown_connections 2024-09-10T08:44:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.809+0000 7fa5c1e00640 1 -- 172.21.15.89:0/2636041456 wait complete. 2024-09-10T08:44:16.811 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.810+0000 7fa5c1e00640 1 Processor -- start 2024-09-10T08:44:16.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.810+0000 7fa5c1e00640 1 -- start start 2024-09-10T08:44:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.810+0000 7fa5c1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5bc1019c0 0x7fa5bc19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.810+0000 7fa5c1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5bc102360 0x7fa5bc19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.810+0000 7fa5c1e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5bc1079c0 0x7fa5bc19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.810+0000 7fa5c1e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc111dc0 con 0x7fa5bc102360 2024-09-10T08:44:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.810+0000 7fa5c1e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc111c40 con 0x7fa5bc1019c0 2024-09-10T08:44:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.810+0000 7fa5c1e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc111f40 con 0x7fa5bc1079c0 2024-09-10T08:44:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.811+0000 7fa5bbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5bc102360 0x7fa5bc19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.811+0000 7fa5bbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5bc102360 0x7fa5bc19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60956/0 (socket says 172.21.15.89:60956) 2024-09-10T08:44:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.811+0000 7fa5bbe00640 1 -- 172.21.15.89:0/743597722 learned_addr learned my addr 172.21.15.89:0/743597722 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.811+0000 7fa5c0a00640 1 --2- 172.21.15.89:0/743597722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5bc1019c0 0x7fa5bc19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.811+0000 7fa5c1400640 1 --2- 172.21.15.89:0/743597722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5bc1079c0 0x7fa5bc19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.811+0000 7fa5c0a00640 1 -- 172.21.15.89:0/743597722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5bc1079c0 msgr2=0x7fa5bc19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:16.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.811+0000 7fa5c0a00640 1 --2- 172.21.15.89:0/743597722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5bc1079c0 0x7fa5bc19ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.811+0000 7fa5c0a00640 1 -- 172.21.15.89:0/743597722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5bc102360 msgr2=0x7fa5bc19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:16.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.811+0000 7fa5c0a00640 1 --2- 172.21.15.89:0/743597722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5bc102360 0x7fa5bc19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.811+0000 7fa5c0a00640 1 -- 172.21.15.89:0/743597722 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5bc19f460 con 0x7fa5bc1019c0 2024-09-10T08:44:16.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.811+0000 7fa5c0a00640 1 --2- 172.21.15.89:0/743597722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5bc1019c0 0x7fa5bc19a4b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000e9f0 tx=0x7fa5b000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:16.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.812+0000 7fa5b9600640 1 -- 172.21.15.89:0/743597722 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b000ccc0 con 0x7fa5bc1019c0 2024-09-10T08:44:16.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.812+0000 7fa5c1e00640 1 -- 172.21.15.89:0/743597722 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5bc19f750 con 0x7fa5bc1019c0 2024-09-10T08:44:16.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.812+0000 7fa5b9600640 1 -- 172.21.15.89:0/743597722 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b0010910 con 0x7fa5bc1019c0 2024-09-10T08:44:16.814 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.812+0000 7fa5b9600640 1 -- 172.21.15.89:0/743597722 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0018960 con 0x7fa5bc1019c0 2024-09-10T08:44:16.815 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.812+0000 7fa5c1e00640 1 -- 172.21.15.89:0/743597722 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5bc1a7030 con 0x7fa5bc1019c0 2024-09-10T08:44:16.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.815+0000 7fa5c5dbb740 1 Processor -- start 2024-09-10T08:44:16.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.815+0000 7fa5c5dbb740 1 -- start start 2024-09-10T08:44:16.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.815+0000 7fa5c5dbb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38e2d50 0x5594b38e3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:16.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.815+0000 7fa5c5dbb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594b38e3670 0x5594b38ed9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:16.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.815+0000 7fa5c5dbb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38edf30 0x5594b38f0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:16.816 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.815+0000 7fa5c5dbb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5594b38493e0 con 0x5594b38edf30 2024-09-10T08:44:16.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.815+0000 7fa5c5dbb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5594b37f0d80 con 0x5594b38e3670 2024-09-10T08:44:16.817 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.815+0000 7fa5c5dbb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5594b38453a0 con 0x5594b38e2d50 2024-09-10T08:44:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.816+0000 7fa5c1400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38e2d50 0x5594b38e3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.816+0000 7fa5bbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38edf30 0x5594b38f0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.816+0000 7fa5b9600640 1 -- 172.21.15.89:0/743597722 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa5b0010ac0 con 0x7fa5bc1019c0 2024-09-10T08:44:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.816+0000 7fa5c1400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38e2d50 0x5594b38e3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:43266/0 (socket says 172.21.15.89:43266) 2024-09-10T08:44:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.816+0000 7fa5c1400640 1 -- 172.21.15.89:0/2281368733 learned_addr learned my addr 172.21.15.89:0/2281368733 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.816+0000 7fa5c1400640 1 -- 172.21.15.89:0/2281368733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594b38e3670 msgr2=0x5594b38ed9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.816+0000 7fa5c1400640 1 --2- 172.21.15.89:0/2281368733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594b38e3670 0x5594b38ed9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.816+0000 7fa5c1400640 1 -- 172.21.15.89:0/2281368733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38edf30 msgr2=0x5594b38f0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.816+0000 7fa5c1400640 1 --2- 172.21.15.89:0/2281368733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38edf30 0x5594b38f0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.816+0000 7fa5c1400640 1 -- 172.21.15.89:0/2281368733 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594b380fae0 con 0x5594b38e2d50 2024-09-10T08:44:16.818 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.817+0000 7fa5c1400640 1 --2- 172.21.15.89:0/2281368733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38e2d50 0x5594b38e3130 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009df0 tx=0x7fa5b400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1699e39d4833488 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:16.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.817+0000 7fa5adc00640 1 -- 172.21.15.89:0/2281368733 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4003970 con 0x5594b38e2d50 2024-09-10T08:44:16.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.817+0000 7fa5adc00640 1 -- 172.21.15.89:0/2281368733 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b4004030 con 0x5594b38e2d50 2024-09-10T08:44:16.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.817+0000 7fa5adc00640 1 -- 172.21.15.89:0/2281368733 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4011800 con 0x5594b38e2d50 2024-09-10T08:44:16.819 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.818+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/2281368733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38e2d50 msgr2=0x5594b38e3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.818+0000 7fa5c5dbb740 1 --2- 172.21.15.89:0/2281368733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38e2d50 0x5594b38e3130 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009df0 tx=0x7fa5b400f730 comp rx=0 tx=0).stop 2024-09-10T08:44:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.818+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/2281368733 shutdown_connections 2024-09-10T08:44:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.818+0000 7fa5c5dbb740 1 --2- 172.21.15.89:0/2281368733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38edf30 0x5594b38f0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.818+0000 7fa5c5dbb740 1 --2- 172.21.15.89:0/2281368733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594b38e3670 0x5594b38ed9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.818+0000 7fa5c5dbb740 1 --2- 172.21.15.89:0/2281368733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38e2d50 0x5594b38e3130 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.818+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/2281368733 >> 172.21.15.89:0/2281368733 conn(0x5594b38df2e0 msgr2=0x5594b38df6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.818+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/2281368733 shutdown_connections 2024-09-10T08:44:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.818+0000 7fa5b9600640 1 --2- 172.21.15.89:0/743597722 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5a40a2b10 0x7fa5a40a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.818+0000 7fa5b9600640 1 -- 172.21.15.89:0/743597722 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa5b00cf7d0 con 0x7fa5bc1019c0 2024-09-10T08:44:16.820 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.819+0000 7fa5bbe00640 1 --2- 172.21.15.89:0/743597722 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5a40a2b10 0x7fa5a40a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:16.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.819+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/2281368733 wait complete. 2024-09-10T08:44:16.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.819+0000 7fa5bbe00640 1 --2- 172.21.15.89:0/743597722 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5a40a2b10 0x7fa5a40a4fd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc10ecb0 tx=0x7fa5a8009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:16.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.819+0000 7fa5c5dbb740 1 -- start start 2024-09-10T08:44:16.821 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.820+0000 7fa5c5dbb740 1 Processor -- start 2024-09-10T08:44:16.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.820+0000 7fa5c5dbb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38e2d50 0x5594b38eae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:16.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.820+0000 7fa5c5dbb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594b38e3670 0x5594b38eb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:16.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.820+0000 7fa5c5dbb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38edf30 0x5594b38eb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:16.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.820+0000 7fa5c5dbb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5594b38f0ba0 con 0x5594b38e2d50 2024-09-10T08:44:16.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.820+0000 7fa5c5dbb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5594b38040f0 con 0x5594b38e3670 2024-09-10T08:44:16.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.820+0000 7fa5c5dbb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5594b38f0d20 con 0x5594b38edf30 2024-09-10T08:44:16.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.820+0000 7fa5c5dbb740 10 client.0 _ll_register_callbacks cb 0x5594b37f59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:16.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.821+0000 7fa5c0a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38e2d50 0x5594b38eae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:16.822 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.821+0000 7fa5c0a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38e2d50 0x5594b38eae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60970/0 (socket says 172.21.15.89:60970) 2024-09-10T08:44:16.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.821+0000 7fa5c0a00640 1 -- 172.21.15.89:0/575185767 learned_addr learned my addr 172.21.15.89:0/575185767 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:16.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.821+0000 7fa5c1400640 1 --2- 172.21.15.89:0/575185767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38edf30 0x5594b38eb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.821+0000 7fa5c0a00640 1 -- 172.21.15.89:0/575185767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38edf30 msgr2=0x5594b38eb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.821+0000 7fa5c0a00640 1 --2- 172.21.15.89:0/575185767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38edf30 0x5594b38eb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.821+0000 7fa5c0a00640 1 -- 172.21.15.89:0/575185767 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594b38e3670 msgr2=0x5594b38eb3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:44:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.821+0000 7fa5c0a00640 1 --2- 172.21.15.89:0/575185767 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594b38e3670 0x5594b38eb3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.821+0000 7fa5c0a00640 1 -- 172.21.15.89:0/575185767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594b35ba3c0 con 0x5594b38e2d50 2024-09-10T08:44:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.821+0000 7fa5c0a00640 1 --2- 172.21.15.89:0/575185767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38e2d50 0x5594b38eae80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00985d0 tx=0x7fa5b009c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a14ba0f453423cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.821+0000 7fa5a2000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:44:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.822+0000 7fa5a2000640 1 -- 172.21.15.89:0/575185767 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b00b2060 con 0x5594b38e2d50 2024-09-10T08:44:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.822+0000 7fa5a2000640 1 -- 172.21.15.89:0/575185767 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa5b00cfdc0 con 0x5594b38e2d50 2024-09-10T08:44:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.822+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/575185767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5b4004030 con 0x5594b38e2d50 2024-09-10T08:44:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.822+0000 7fa5a2000640 1 -- 172.21.15.89:0/575185767 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa5b00b9050 con 0x5594b38e2d50 2024-09-10T08:44:16.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.822+0000 7fa5c5dbb740 10 client.16080 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:16.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.822+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/575185767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5594b38fc8b0 con 0x5594b38e2d50 2024-09-10T08:44:16.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.822+0000 7fa5a2000640 1 -- 172.21.15.89:0/575185767 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b4004030 con 0x5594b38e2d50 2024-09-10T08:44:16.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.823+0000 7fa5a2000640 1 -- 172.21.15.89:0/575185767 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa5b0018c20 con 0x5594b38e2d50 2024-09-10T08:44:16.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.823+0000 7fa5c5dbb740 10 client.16080 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:16.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.823+0000 7fa5c5dbb740 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:44:16.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.823+0000 7fa5c5dbb740 20 client.16080 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:16.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.823+0000 7fa5c5dbb740 1 --2- 172.21.15.89:0/575185767 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5594b38fdcc0 0x5594b39000f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:16.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.823+0000 7fa5c5dbb740 4 client.16080 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:44:16.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.823+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/575185767 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594b3856950 con 0x5594b38fdcc0 2024-09-10T08:44:16.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.823+0000 7fa5bbe00640 1 --2- 172.21.15.89:0/575185767 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5594b38fdcc0 0x5594b39000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:16.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.863+0000 7fa5bbe00640 1 --2- 172.21.15.89:0/575185767 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5594b38fdcc0 0x5594b39000f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b60982537c732e8d server_cookie=830cc6836b48bcd5 in_seq=0 out_seq=0 2024-09-10T08:44:16.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.863+0000 7fa5a2000640 10 client.16080 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:16.888 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.887+0000 7fa5a2000640 1 -- 172.21.15.89:0/575185767 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594b3856950 con 0x5594b38fdcc0 2024-09-10T08:44:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.887+0000 7fa5a2000640 10 client.16080 handle_command_reply: tid=0 2024-09-10T08:44:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.887+0000 7fa5c5dbb740 1 client.16080 shutdown 2024-09-10T08:44:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.887+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/575185767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38e2d50 msgr2=0x5594b38eae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.887+0000 7fa5c5dbb740 1 --2- 172.21.15.89:0/575185767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38e2d50 0x5594b38eae80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00985d0 tx=0x7fa5b009c300 comp rx=0 tx=0).stop 2024-09-10T08:44:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.887+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/575185767 shutdown_connections 2024-09-10T08:44:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.887+0000 7fa5c5dbb740 1 --2- 172.21.15.89:0/575185767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38edf30 0x5594b38eb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.887+0000 7fa5c5dbb740 1 --2- 172.21.15.89:0/575185767 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594b38e3670 0x5594b38eb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.887+0000 7fa5c5dbb740 1 --2- 172.21.15.89:0/575185767 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5594b38fdcc0 0x5594b39000f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.887+0000 7fa5c5dbb740 1 --2- 172.21.15.89:0/575185767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38e2d50 0x5594b38eae80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.887+0000 7fa5a2000640 0 client.16080 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.887+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/575185767 >> 172.21.15.89:0/575185767 conn(0x5594b38df2e0 msgr2=0x5594b38e4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:16.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.887+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/575185767 shutdown_connections 2024-09-10T08:44:16.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.888+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/575185767 wait complete. 2024-09-10T08:44:16.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.888+0000 7fa5c5dbb740 20 client.16080 trim_cache size 0 max 16384 2024-09-10T08:44:16.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.905+0000 7fa5c5dbb740 1 Processor -- start 2024-09-10T08:44:16.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.905+0000 7fa5c5dbb740 1 -- start start 2024-09-10T08:44:16.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.906+0000 7fa5c5dbb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38e2d50 0x5594b38e57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:16.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.906+0000 7fa5c5dbb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594b38e3670 0x5594b38de270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:16.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.906+0000 7fa5c5dbb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38edf30 0x5594b39059c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:16.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.906+0000 7fa5c5dbb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1af3b0 con 0x5594b38edf30 2024-09-10T08:44:16.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.906+0000 7fa5c5dbb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1acc80 con 0x5594b38e3670 2024-09-10T08:44:16.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.906+0000 7fa5c5dbb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1aaab0 con 0x5594b38e2d50 2024-09-10T08:44:16.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.906+0000 7fa5c1400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38edf30 0x5594b39059c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:16.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.906+0000 7fa5c1400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38edf30 0x5594b39059c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60978/0 (socket says 172.21.15.89:60978) 2024-09-10T08:44:16.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.906+0000 7fa5c0a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38e2d50 0x5594b38e57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:16.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.906+0000 7fa5c1400640 1 -- 172.21.15.89:0/2759133525 learned_addr learned my addr 172.21.15.89:0/2759133525 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:16.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.906+0000 7fa5bbe00640 1 --2- 172.21.15.89:0/2759133525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594b38e3670 0x5594b38de270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:16.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.906+0000 7fa5c1400640 1 -- 172.21.15.89:0/2759133525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38e2d50 msgr2=0x5594b38e57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:16.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.906+0000 7fa5c1400640 1 --2- 172.21.15.89:0/2759133525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38e2d50 0x5594b38e57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.906+0000 7fa5c1400640 1 -- 172.21.15.89:0/2759133525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594b38e3670 msgr2=0x5594b38de270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:16.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.906+0000 7fa5c1400640 1 --2- 172.21.15.89:0/2759133525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594b38e3670 0x5594b38de270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.906+0000 7fa5c1400640 1 -- 172.21.15.89:0/2759133525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594b3900770 con 0x5594b38edf30 2024-09-10T08:44:16.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.906+0000 7fa5bbe00640 1 --2- 172.21.15.89:0/2759133525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594b38e3670 0x5594b38de270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:16.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.907+0000 7fa5c1400640 1 --2- 172.21.15.89:0/2759133525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38edf30 0x5594b39059c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009d10 tx=0x7fa5b4011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f631178c497af5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:16.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.907+0000 7fa5ad200640 1 -- 172.21.15.89:0/2759133525 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b402b070 con 0x5594b38edf30 2024-09-10T08:44:16.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.907+0000 7fa5ad200640 1 -- 172.21.15.89:0/2759133525 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b4004da0 con 0x5594b38edf30 2024-09-10T08:44:16.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.907+0000 7fa5ad200640 1 -- 172.21.15.89:0/2759133525 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40114b0 con 0x5594b38edf30 2024-09-10T08:44:16.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.908+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/2759133525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38edf30 msgr2=0x5594b39059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:16.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.908+0000 7fa5c5dbb740 1 --2- 172.21.15.89:0/2759133525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38edf30 0x5594b39059c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009d10 tx=0x7fa5b4011440 comp rx=0 tx=0).stop 2024-09-10T08:44:16.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.908+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/2759133525 shutdown_connections 2024-09-10T08:44:16.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.908+0000 7fa5c5dbb740 1 --2- 172.21.15.89:0/2759133525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38edf30 0x5594b39059c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.908+0000 7fa5c5dbb740 1 --2- 172.21.15.89:0/2759133525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594b38e3670 0x5594b38de270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.908+0000 7fa5c5dbb740 1 --2- 172.21.15.89:0/2759133525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38e2d50 0x5594b38e57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.908+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/2759133525 >> 172.21.15.89:0/2759133525 conn(0x5594b38df2e0 msgr2=0x5594b38ddb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:16.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.908+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/2759133525 shutdown_connections 2024-09-10T08:44:16.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.908+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/2759133525 wait complete. 2024-09-10T08:44:16.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.909+0000 7fa5c5dbb740 1 -- start start 2024-09-10T08:44:16.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.909+0000 7fa5c5dbb740 1 Processor -- start 2024-09-10T08:44:16.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.909+0000 7fa5c5dbb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38e2d50 0x5594b38f6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:16.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.909+0000 7fa5c5dbb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594b38e3670 0x5594b38f6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:16.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.909+0000 7fa5c5dbb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38edf30 0x5594b38f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:16.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.909+0000 7fa5c5dbb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1ab140 con 0x5594b38e2d50 2024-09-10T08:44:16.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.909+0000 7fa5c5dbb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1ac5b0 con 0x5594b38e3670 2024-09-10T08:44:16.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.909+0000 7fa5c5dbb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1ab810 con 0x5594b38edf30 2024-09-10T08:44:16.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.909+0000 7fa5c5dbb740 10 client.0 _ll_register_callbacks cb 0x7fa5b009b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:16.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.910+0000 7fa5c0a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38e2d50 0x5594b38f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:16.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.910+0000 7fa5bbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594b38e3670 0x5594b38f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:16.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.910+0000 7fa5c0a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38e2d50 0x5594b38f6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60984/0 (socket says 172.21.15.89:60984) 2024-09-10T08:44:16.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.910+0000 7fa5c0a00640 1 -- 172.21.15.89:0/2344837805 learned_addr learned my addr 172.21.15.89:0/2344837805 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:16.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.910+0000 7fa5c1400640 1 --2- 172.21.15.89:0/2344837805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38edf30 0x5594b38f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:16.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.910+0000 7fa5c0a00640 1 -- 172.21.15.89:0/2344837805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38edf30 msgr2=0x5594b38f2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:16.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.910+0000 7fa5c0a00640 1 --2- 172.21.15.89:0/2344837805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38edf30 0x5594b38f2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.910+0000 7fa5c0a00640 1 -- 172.21.15.89:0/2344837805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594b38e3670 msgr2=0x5594b38f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:16.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.910+0000 7fa5c0a00640 1 --2- 172.21.15.89:0/2344837805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594b38e3670 0x5594b38f6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.910+0000 7fa5c0a00640 1 -- 172.21.15.89:0/2344837805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594b38f3740 con 0x5594b38e2d50 2024-09-10T08:44:16.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.910+0000 7fa5bbe00640 1 --2- 172.21.15.89:0/2344837805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594b38e3670 0x5594b38f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:16.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.910+0000 7fa5c0a00640 1 --2- 172.21.15.89:0/2344837805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38e2d50 0x5594b38f6150 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000e9c0 tx=0x7fa5b009b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5b820e5c04c73f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:16.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.910+0000 7fa5a2000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:44:16.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.911+0000 7fa5a2000640 1 -- 172.21.15.89:0/2344837805 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b00b2070 con 0x5594b38e2d50 2024-09-10T08:44:16.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.911+0000 7fa5a2000640 1 -- 172.21.15.89:0/2344837805 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa5b0010640 con 0x5594b38e2d50 2024-09-10T08:44:16.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.911+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/2344837805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5b4004da0 con 0x5594b38e2d50 2024-09-10T08:44:16.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.911+0000 7fa5a2000640 1 -- 172.21.15.89:0/2344837805 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa5b0097a90 con 0x5594b38e2d50 2024-09-10T08:44:16.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.911+0000 7fa5c5dbb740 10 client.16086 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:16.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.911+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/2344837805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5594b3969430 con 0x5594b38e2d50 2024-09-10T08:44:16.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.911+0000 7fa5a2000640 1 -- 172.21.15.89:0/2344837805 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b4004da0 con 0x5594b38e2d50 2024-09-10T08:44:16.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.911+0000 7fa5a2000640 1 -- 172.21.15.89:0/2344837805 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa5b009aac0 con 0x5594b38e2d50 2024-09-10T08:44:16.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.911+0000 7fa5c5dbb740 10 client.16086 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:16.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.911+0000 7fa5c5dbb740 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:44:16.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.911+0000 7fa5c5dbb740 20 client.16086 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:16.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.911+0000 7fa5c5dbb740 1 --2- 172.21.15.89:0/2344837805 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5594b3969aa0 0x5594b396bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:16.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.911+0000 7fa5c5dbb740 4 client.16086 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:44:16.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.911+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/2344837805 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5594b3850e60 con 0x5594b3969aa0 2024-09-10T08:44:16.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.912+0000 7fa5bbe00640 1 --2- 172.21.15.89:0/2344837805 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5594b3969aa0 0x5594b396bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:16.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.912+0000 7fa5bbe00640 1 --2- 172.21.15.89:0/2344837805 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5594b3969aa0 0x5594b396bec0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd21ea3706abaeaf server_cookie=e861c81c35adec98 in_seq=0 out_seq=0 2024-09-10T08:44:16.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.912+0000 7fa5a2000640 10 client.16086 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:16.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.912+0000 7fa5a2000640 1 -- 172.21.15.89:0/2344837805 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5594b3850e60 con 0x5594b3969aa0 2024-09-10T08:44:16.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.912+0000 7fa5a2000640 10 client.16086 handle_command_reply: tid=0 2024-09-10T08:44:16.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.913+0000 7fa5c5dbb740 1 client.16086 shutdown 2024-09-10T08:44:16.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.913+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/2344837805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38e2d50 msgr2=0x5594b38f6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:16.916 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.913+0000 7fa5c5dbb740 1 --2- 172.21.15.89:0/2344837805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38e2d50 0x5594b38f6150 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000e9c0 tx=0x7fa5b009b280 comp rx=0 tx=0).stop 2024-09-10T08:44:16.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.913+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/2344837805 shutdown_connections 2024-09-10T08:44:16.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.913+0000 7fa5c5dbb740 1 --2- 172.21.15.89:0/2344837805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594b38edf30 0x5594b38f2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.913+0000 7fa5c5dbb740 1 --2- 172.21.15.89:0/2344837805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594b38e3670 0x5594b38f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.913+0000 7fa5c5dbb740 1 --2- 172.21.15.89:0/2344837805 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5594b3969aa0 0x5594b396bec0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.913+0000 7fa5c5dbb740 1 --2- 172.21.15.89:0/2344837805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5594b38e2d50 0x5594b38f6150 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.913+0000 7fa5a2000640 0 client.16086 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:16.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.913+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/2344837805 >> 172.21.15.89:0/2344837805 conn(0x5594b38df2e0 msgr2=0x5594b38eb660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:16.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.913+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/2344837805 shutdown_connections 2024-09-10T08:44:16.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.913+0000 7fa5c5dbb740 1 -- 172.21.15.89:0/2344837805 wait complete. 2024-09-10T08:44:16.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.913+0000 7fa5c5dbb740 20 client.16086 trim_cache size 0 max 16384 2024-09-10T08:44:16.917 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:16 smithi089 ceph-mon[30757]: pgmap v523: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-09-10T08:44:16.918 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:44:16.918 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2024-09-10T08:44:16.918 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2024-09-10T08:44:16.918 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:44:16.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.914+0000 7fa5c1e00640 1 -- 172.21.15.89:0/743597722 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5a40a2b10 msgr2=0x7fa5a40a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:16.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.914+0000 7fa5c1e00640 1 --2- 172.21.15.89:0/743597722 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5a40a2b10 0x7fa5a40a4fd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc10ecb0 tx=0x7fa5a8009560 comp rx=0 tx=0).stop 2024-09-10T08:44:16.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.914+0000 7fa5c1e00640 1 -- 172.21.15.89:0/743597722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5bc1019c0 msgr2=0x7fa5bc19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:16.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.914+0000 7fa5c1e00640 1 --2- 172.21.15.89:0/743597722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5bc1019c0 0x7fa5bc19a4b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000e9f0 tx=0x7fa5b000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.914+0000 7fa5c1e00640 1 -- 172.21.15.89:0/743597722 shutdown_connections 2024-09-10T08:44:16.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.914+0000 7fa5c1e00640 1 --2- 172.21.15.89:0/743597722 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa5a40a2b10 0x7fa5a40a4fd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.914+0000 7fa5c1e00640 1 --2- 172.21.15.89:0/743597722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5bc1079c0 0x7fa5bc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.914+0000 7fa5c1e00640 1 --2- 172.21.15.89:0/743597722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5bc102360 0x7fa5bc19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.914+0000 7fa5c1e00640 1 --2- 172.21.15.89:0/743597722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5bc1019c0 0x7fa5bc19a4b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.914+0000 7fa5c1e00640 1 -- 172.21.15.89:0/743597722 >> 172.21.15.89:0/743597722 conn(0x7fa5bc0fd740 msgr2=0x7fa5bc0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:16.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.914+0000 7fa5c1e00640 1 -- 172.21.15.89:0/743597722 shutdown_connections 2024-09-10T08:44:16.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:16.915+0000 7fa5c1e00640 1 -- 172.21.15.89:0/743597722 wait complete. 2024-09-10T08:44:17.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:16 smithi149 ceph-mon[36167]: pgmap v523: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-09-10T08:44:17.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:16 smithi111 ceph-mon[35220]: pgmap v523: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-09-10T08:44:17.915 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:44:17.915 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:44:18.042 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:17 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:44:18.042 INFO:tasks.workunit.client.0.smithi089.stdout: 87 0 0 0 0 0 0 2024-09-10T08:44:18.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:17 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:44:18.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:17 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:44:18.321 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:18.913 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:18 smithi089 ceph-mon[30757]: pgmap v524: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-10T08:44:18.913 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:18 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:44:18.913 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:18 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:44:18.913 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:18 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:44:19.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.091+0000 7f4e5e800640 1 -- 172.21.15.89:0/1868105604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e58074f40 msgr2=0x7f4e5810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:19.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.091+0000 7f4e5e800640 1 --2- 172.21.15.89:0/1868105604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e58074f40 0x7f4e5810dcd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009900 tx=0x7f4e4402a480 comp rx=0 tx=0).stop 2024-09-10T08:44:19.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.091+0000 7f4e5e800640 1 -- 172.21.15.89:0/1868105604 shutdown_connections 2024-09-10T08:44:19.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.091+0000 7f4e5e800640 1 --2- 172.21.15.89:0/1868105604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e5810e210 0x7f4e58110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:19.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.091+0000 7f4e5e800640 1 --2- 172.21.15.89:0/1868105604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e58074f40 0x7f4e5810dcd0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:19.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.091+0000 7f4e5e800640 1 --2- 172.21.15.89:0/1868105604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e58074620 0x7f4e58074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:19.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.091+0000 7f4e5e800640 1 -- 172.21.15.89:0/1868105604 >> 172.21.15.89:0/1868105604 conn(0x7f4e580fd710 msgr2=0x7f4e580ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:19.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.092+0000 7f4e5e800640 1 -- 172.21.15.89:0/1868105604 shutdown_connections 2024-09-10T08:44:19.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.092+0000 7f4e5e800640 1 -- 172.21.15.89:0/1868105604 wait complete. 2024-09-10T08:44:19.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.093+0000 7f4e5e800640 1 Processor -- start 2024-09-10T08:44:19.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.093+0000 7f4e5e800640 1 -- start start 2024-09-10T08:44:19.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.093+0000 7f4e5e800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e58074620 0x7f4e5819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:19.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.093+0000 7f4e5e800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e58074f40 0x7f4e5819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:19.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.094+0000 7f4e5e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e5810e210 0x7f4e5819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:19.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.094+0000 7f4e5e800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58114010 con 0x7f4e5810e210 2024-09-10T08:44:19.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.094+0000 7f4e5e800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58113e90 con 0x7f4e58074f40 2024-09-10T08:44:19.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.094+0000 7f4e5e800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58114190 con 0x7f4e58074620 2024-09-10T08:44:19.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.094+0000 7f4e5d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e58074620 0x7f4e5819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:19.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.094+0000 7f4e5d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e58074620 0x7f4e5819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:43316/0 (socket says 172.21.15.89:43316) 2024-09-10T08:44:19.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.094+0000 7f4e5d400640 1 -- 172.21.15.89:0/582258452 learned_addr learned my addr 172.21.15.89:0/582258452 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:19.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.094+0000 7f4e5ca00640 1 --2- 172.21.15.89:0/582258452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e58074f40 0x7f4e5819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:19.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.094+0000 7f4e5de00640 1 --2- 172.21.15.89:0/582258452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e5810e210 0x7f4e5819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:19.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.094+0000 7f4e5ca00640 1 -- 172.21.15.89:0/582258452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e58074620 msgr2=0x7f4e5819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:19.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.094+0000 7f4e5ca00640 1 --2- 172.21.15.89:0/582258452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e58074620 0x7f4e5819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:19.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.094+0000 7f4e5ca00640 1 -- 172.21.15.89:0/582258452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e5810e210 msgr2=0x7f4e5819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:19.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.094+0000 7f4e5ca00640 1 --2- 172.21.15.89:0/582258452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e5810e210 0x7f4e5819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:19.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.094+0000 7f4e5ca00640 1 -- 172.21.15.89:0/582258452 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5819f4a0 con 0x7f4e58074f40 2024-09-10T08:44:19.096 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.095+0000 7f4e5ca00640 1 --2- 172.21.15.89:0/582258452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e58074f40 0x7f4e5819aa30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4e4402a9a0 tx=0x7f4e44003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:19.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.095+0000 7f4e56000640 1 -- 172.21.15.89:0/582258452 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4403f070 con 0x7f4e58074f40 2024-09-10T08:44:19.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.095+0000 7f4e56000640 1 -- 172.21.15.89:0/582258452 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e44031c90 con 0x7f4e58074f40 2024-09-10T08:44:19.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.095+0000 7f4e56000640 1 -- 172.21.15.89:0/582258452 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e44035030 con 0x7f4e58074f40 2024-09-10T08:44:19.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.095+0000 7f4e5e800640 1 -- 172.21.15.89:0/582258452 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5819f730 con 0x7f4e58074f40 2024-09-10T08:44:19.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.095+0000 7f4e5e800640 1 -- 172.21.15.89:0/582258452 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e5819fbc0 con 0x7f4e58074f40 2024-09-10T08:44:19.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.096+0000 7f4e5e800640 1 -- 172.21.15.89:0/582258452 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e58074a00 con 0x7f4e58074f40 2024-09-10T08:44:19.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.100+0000 7f4e56000640 1 -- 172.21.15.89:0/582258452 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4e44042070 con 0x7f4e58074f40 2024-09-10T08:44:19.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.101+0000 7f4e56000640 1 --2- 172.21.15.89:0/582258452 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f4e400a2c00 0x7f4e400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:19.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.102+0000 7f4e56000640 1 -- 172.21.15.89:0/582258452 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f4e440f05a0 con 0x7f4e58074f40 2024-09-10T08:44:19.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.102+0000 7f4e5d400640 1 --2- 172.21.15.89:0/582258452 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f4e400a2c00 0x7f4e400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:19.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.102+0000 7f4e56000640 1 -- 172.21.15.89:0/582258452 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e440bd0d0 con 0x7f4e58074f40 2024-09-10T08:44:19.103 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.103+0000 7f4e5d400640 1 --2- 172.21.15.89:0/582258452 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f4e400a2c00 0x7f4e400a50c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009c90 tx=0x7f4e4c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:19.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:18 smithi149 ceph-mon[36167]: pgmap v524: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-10T08:44:19.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:18 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:44:19.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:18 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:44:19.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:18 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:44:19.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:18 smithi111 ceph-mon[35220]: pgmap v524: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-10T08:44:19.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:18 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:44:19.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:18 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:44:19.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:18 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:44:19.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.321+0000 7f4e5e800640 1 -- 172.21.15.89:0/582258452 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e5819b850 con 0x7f4e58074f40 2024-09-10T08:44:19.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.322+0000 7f4e56000640 1 -- 172.21.15.89:0/582258452 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11921 (secure 0 0 0) 0x7f4e440bc9d0 con 0x7f4e58074f40 2024-09-10T08:44:19.324 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:44:19.324 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":28,"btime":"2024-09-10T08:41:55:762565+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:41:55.148062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24518,24496]},"id":1}]} 2024-09-10T08:44:19.324 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 28 2024-09-10T08:44:19.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.325+0000 7f4e5e800640 1 -- 172.21.15.89:0/582258452 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f4e400a2c00 msgr2=0x7f4e400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:19.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.325+0000 7f4e5e800640 1 --2- 172.21.15.89:0/582258452 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f4e400a2c00 0x7f4e400a50c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009c90 tx=0x7f4e4c005f90 comp rx=0 tx=0).stop 2024-09-10T08:44:19.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.325+0000 7f4e5e800640 1 -- 172.21.15.89:0/582258452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e58074f40 msgr2=0x7f4e5819aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:19.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.325+0000 7f4e5e800640 1 --2- 172.21.15.89:0/582258452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e58074f40 0x7f4e5819aa30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4e4402a9a0 tx=0x7f4e44003b20 comp rx=0 tx=0).stop 2024-09-10T08:44:19.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.325+0000 7f4e5e800640 1 -- 172.21.15.89:0/582258452 shutdown_connections 2024-09-10T08:44:19.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.325+0000 7f4e5e800640 1 --2- 172.21.15.89:0/582258452 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f4e400a2c00 0x7f4e400a50c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:19.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.325+0000 7f4e5e800640 1 --2- 172.21.15.89:0/582258452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e5810e210 0x7f4e5819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:19.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.325+0000 7f4e5e800640 1 --2- 172.21.15.89:0/582258452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e58074f40 0x7f4e5819aa30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:19.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.325+0000 7f4e5e800640 1 --2- 172.21.15.89:0/582258452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e58074620 0x7f4e5819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:19.326 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.325+0000 7f4e5e800640 1 -- 172.21.15.89:0/582258452 >> 172.21.15.89:0/582258452 conn(0x7f4e580fd710 msgr2=0x7f4e580fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:19.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.325+0000 7f4e5e800640 1 -- 172.21.15.89:0/582258452 shutdown_connections 2024-09-10T08:44:19.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:19.325+0000 7f4e5e800640 1 -- 172.21.15.89:0/582258452 wait complete. 2024-09-10T08:44:19.987 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:19 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/582258452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:20.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:19 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/582258452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:20.268 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:44:20.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:19 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/582258452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:20.636 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:21.070 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:20 smithi089 ceph-mon[30757]: pgmap v525: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-09-10T08:44:21.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:20 smithi149 ceph-mon[36167]: pgmap v525: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-09-10T08:44:21.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:20 smithi111 ceph-mon[35220]: pgmap v525: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-09-10T08:44:21.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.462+0000 7f2f7be00640 1 -- 172.21.15.89:0/2552514308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f7c074ee0 msgr2=0x7f2f7c10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:21.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.462+0000 7f2f7be00640 1 --2- 172.21.15.89:0/2552514308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f7c074ee0 0x7f2f7c10dd20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009900 tx=0x7f2f6402a480 comp rx=0 tx=0).stop 2024-09-10T08:44:21.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.462+0000 7f2f7be00640 1 -- 172.21.15.89:0/2552514308 shutdown_connections 2024-09-10T08:44:21.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.462+0000 7f2f7be00640 1 --2- 172.21.15.89:0/2552514308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f7c10e260 0x7f2f7c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.462+0000 7f2f7be00640 1 --2- 172.21.15.89:0/2552514308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f7c074ee0 0x7f2f7c10dd20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.462+0000 7f2f7be00640 1 --2- 172.21.15.89:0/2552514308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f7c0745c0 0x7f2f7c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.462+0000 7f2f7be00640 1 -- 172.21.15.89:0/2552514308 >> 172.21.15.89:0/2552514308 conn(0x7f2f7c0fd740 msgr2=0x7f2f7c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:21.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.462+0000 7f2f7be00640 1 -- 172.21.15.89:0/2552514308 shutdown_connections 2024-09-10T08:44:21.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.462+0000 7f2f7be00640 1 -- 172.21.15.89:0/2552514308 wait complete. 2024-09-10T08:44:21.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.463+0000 7f2f7be00640 1 Processor -- start 2024-09-10T08:44:21.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.463+0000 7f2f7be00640 1 -- start start 2024-09-10T08:44:21.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.464+0000 7f2f7be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f7c0745c0 0x7f2f7c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:21.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.464+0000 7f2f7be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f7c074ee0 0x7f2f7c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:21.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.464+0000 7f2f7be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f7c10e260 0x7f2f7c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:21.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.464+0000 7f2f7be00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c113fb0 con 0x7f2f7c074ee0 2024-09-10T08:44:21.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.464+0000 7f2f7be00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c113e30 con 0x7f2f7c0745c0 2024-09-10T08:44:21.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.464+0000 7f2f7be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c114130 con 0x7f2f7c10e260 2024-09-10T08:44:21.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.464+0000 7f2f72000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f7c074ee0 0x7f2f7c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:21.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.464+0000 7f2f7aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f7c0745c0 0x7f2f7c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:21.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.464+0000 7f2f72000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f7c074ee0 0x7f2f7c19a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32778/0 (socket says 172.21.15.89:32778) 2024-09-10T08:44:21.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.464+0000 7f2f72000640 1 -- 172.21.15.89:0/3420000082 learned_addr learned my addr 172.21.15.89:0/3420000082 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:21.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.464+0000 7f2f7b400640 1 --2- 172.21.15.89:0/3420000082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f7c10e260 0x7f2f7c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:21.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.464+0000 7f2f7b400640 1 -- 172.21.15.89:0/3420000082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f7c0745c0 msgr2=0x7f2f7c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:21.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.464+0000 7f2f7b400640 1 --2- 172.21.15.89:0/3420000082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f7c0745c0 0x7f2f7c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.464+0000 7f2f7b400640 1 -- 172.21.15.89:0/3420000082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f7c074ee0 msgr2=0x7f2f7c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:21.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.464+0000 7f2f7b400640 1 --2- 172.21.15.89:0/3420000082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f7c074ee0 0x7f2f7c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.464+0000 7f2f7b400640 1 -- 172.21.15.89:0/3420000082 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f7c19f440 con 0x7f2f7c10e260 2024-09-10T08:44:21.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.464+0000 7f2f72000640 1 --2- 172.21.15.89:0/3420000082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f7c074ee0 0x7f2f7c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:21.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.465+0000 7f2f7b400640 1 --2- 172.21.15.89:0/3420000082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f7c10e260 0x7f2f7c19ed60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2f74005fd0 tx=0x7f2f7400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:21.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.465+0000 7f2f80a00640 1 -- 172.21.15.89:0/3420000082 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7400cdf0 con 0x7f2f7c10e260 2024-09-10T08:44:21.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.465+0000 7f2f80a00640 1 -- 172.21.15.89:0/3420000082 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f74005020 con 0x7f2f7c10e260 2024-09-10T08:44:21.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.465+0000 7f2f80a00640 1 -- 172.21.15.89:0/3420000082 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f740106d0 con 0x7f2f7c10e260 2024-09-10T08:44:21.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.465+0000 7f2f7be00640 1 -- 172.21.15.89:0/3420000082 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f7c19f730 con 0x7f2f7c10e260 2024-09-10T08:44:21.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.465+0000 7f2f7be00640 1 -- 172.21.15.89:0/3420000082 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f7c1a7010 con 0x7f2f7c10e260 2024-09-10T08:44:21.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.469+0000 7f2f80a00640 1 -- 172.21.15.89:0/3420000082 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2f740054d0 con 0x7f2f7c10e260 2024-09-10T08:44:21.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.469+0000 7f2f84a68740 1 Processor -- start 2024-09-10T08:44:21.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.469+0000 7f2f84a68740 1 -- start start 2024-09-10T08:44:21.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.470+0000 7f2f84a68740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a399be40 0x5599a399c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.470+0000 7f2f84a68740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a399c760 0x5599a39a6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.470+0000 7f2f84a68740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599a39a7020 0x5599a39a9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.470+0000 7f2f84a68740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5599a39023e0 con 0x5599a399c760 2024-09-10T08:44:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.470+0000 7f2f84a68740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5599a38a9d80 con 0x5599a399be40 2024-09-10T08:44:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.470+0000 7f2f84a68740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5599a38fe3a0 con 0x5599a39a7020 2024-09-10T08:44:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.470+0000 7f2f80a00640 1 --2- 172.21.15.89:0/3420000082 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2f600a2c00 0x7f2f600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.470+0000 7f2f80a00640 1 -- 172.21.15.89:0/3420000082 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f2f740cf9b0 con 0x7f2f7c10e260 2024-09-10T08:44:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.470+0000 7f2f7aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a399c760 0x5599a39a6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.470+0000 7f2f7aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a399c760 0x5599a39a6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32786/0 (socket says 172.21.15.89:32786) 2024-09-10T08:44:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.470+0000 7f2f7aa00640 1 -- 172.21.15.89:0/3551138211 learned_addr learned my addr 172.21.15.89:0/3551138211 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.471+0000 7f2f72000640 1 --2- 172.21.15.89:0/3551138211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a399be40 0x5599a399c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:21.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.471+0000 7f2f72000640 1 --2- 172.21.15.89:0/3551138211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599a39a7020 0x5599a39a9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:21.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.471+0000 7f2f7aa00640 1 -- 172.21.15.89:0/3551138211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599a39a7020 msgr2=0x5599a39a9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:21.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.471+0000 7f2f7b400640 1 --2- 172.21.15.89:0/3420000082 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2f600a2c00 0x7f2f600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:21.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.471+0000 7f2f7aa00640 1 --2- 172.21.15.89:0/3551138211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599a39a7020 0x5599a39a9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.471+0000 7f2f7aa00640 1 -- 172.21.15.89:0/3551138211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a399be40 msgr2=0x5599a399c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:21.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.471+0000 7f2f7aa00640 1 --2- 172.21.15.89:0/3551138211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a399be40 0x5599a399c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.471+0000 7f2f7aa00640 1 -- 172.21.15.89:0/3551138211 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599a38c8ae0 con 0x5599a399c760 2024-09-10T08:44:21.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.471+0000 7f2f72000640 1 --2- 172.21.15.89:0/3551138211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599a39a7020 0x5599a39a9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:44:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.471+0000 7f2f7aa00640 1 --2- 172.21.15.89:0/3551138211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a399c760 0x5599a39a6ae0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00bcd0 tx=0x7f2f6c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3f61eb036f473c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:21.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.472+0000 7f2f7b400640 1 --2- 172.21.15.89:0/3420000082 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2f600a2c00 0x7f2f600a50c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2f74005910 tx=0x7f2f7400cf90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.472+0000 7f2f6be00640 1 -- 172.21.15.89:0/3551138211 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c005220 con 0x5599a399c760 2024-09-10T08:44:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.472+0000 7f2f6be00640 1 -- 172.21.15.89:0/3551138211 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f6c004c80 con 0x5599a399c760 2024-09-10T08:44:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.472+0000 7f2f6be00640 1 -- 172.21.15.89:0/3551138211 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c0117a0 con 0x5599a399c760 2024-09-10T08:44:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.472+0000 7f2f84a68740 1 -- 172.21.15.89:0/3551138211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a399c760 msgr2=0x5599a39a6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.472+0000 7f2f84a68740 1 --2- 172.21.15.89:0/3551138211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a399c760 0x5599a39a6ae0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00bcd0 tx=0x7f2f6c0107a0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.472+0000 7f2f84a68740 1 -- 172.21.15.89:0/3551138211 shutdown_connections 2024-09-10T08:44:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.472+0000 7f2f84a68740 1 --2- 172.21.15.89:0/3551138211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599a39a7020 0x5599a39a9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.472+0000 7f2f84a68740 1 --2- 172.21.15.89:0/3551138211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a399c760 0x5599a39a6ae0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.472+0000 7f2f84a68740 1 --2- 172.21.15.89:0/3551138211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a399be40 0x5599a399c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.472+0000 7f2f84a68740 1 -- 172.21.15.89:0/3551138211 >> 172.21.15.89:0/3551138211 conn(0x5599a39983d0 msgr2=0x5599a39987b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.473+0000 7f2f84a68740 1 -- 172.21.15.89:0/3551138211 shutdown_connections 2024-09-10T08:44:21.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.473+0000 7f2f84a68740 1 -- 172.21.15.89:0/3551138211 wait complete. 2024-09-10T08:44:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.473+0000 7f2f84a68740 1 -- start start 2024-09-10T08:44:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.473+0000 7f2f84a68740 1 Processor -- start 2024-09-10T08:44:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.474+0000 7f2f84a68740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599a399be40 0x5599a39a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.474+0000 7f2f84a68740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a399c760 0x5599a39a44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.474+0000 7f2f84a68740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a39a7020 0x5599a39a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.474+0000 7f2f84a68740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5599a39a9bf0 con 0x5599a39a7020 2024-09-10T08:44:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.474+0000 7f2f84a68740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5599a38bd0f0 con 0x5599a399c760 2024-09-10T08:44:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.474+0000 7f2f84a68740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5599a39a9d70 con 0x5599a399be40 2024-09-10T08:44:21.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.474+0000 7f2f84a68740 10 client.0 _ll_register_callbacks cb 0x5599a38a4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.474+0000 7f2f72000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a39a7020 0x5599a39a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.474+0000 7f2f7aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a399c760 0x5599a39a44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.474+0000 7f2f7aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a399c760 0x5599a39a44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:33218/0 (socket says 172.21.15.89:33218) 2024-09-10T08:44:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.474+0000 7f2f7aa00640 1 -- 172.21.15.89:0/1701905784 learned_addr learned my addr 172.21.15.89:0/1701905784 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:21.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.474+0000 7f2f7aa00640 1 -- 172.21.15.89:0/1701905784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599a399be40 msgr2=0x5599a39a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.474+0000 7f2f7aa00640 1 --2- 172.21.15.89:0/1701905784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599a399be40 0x5599a39a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.474+0000 7f2f7aa00640 1 -- 172.21.15.89:0/1701905784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a39a7020 msgr2=0x5599a39a4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.474+0000 7f2f7aa00640 1 --2- 172.21.15.89:0/1701905784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a39a7020 0x5599a39a4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.474+0000 7f2f7aa00640 1 -- 172.21.15.89:0/1701905784 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599a36733c0 con 0x5599a399c760 2024-09-10T08:44:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.475+0000 7f2f7aa00640 1 --2- 172.21.15.89:0/1701905784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a399c760 0x5599a39a44f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c011010 tx=0x7f2f6c0048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bf214dad4a76cce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.475+0000 7f2f68c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:44:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.475+0000 7f2f68c00640 1 -- 172.21.15.89:0/1701905784 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c00be80 con 0x5599a399c760 2024-09-10T08:44:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.475+0000 7f2f84a68740 1 -- 172.21.15.89:0/1701905784 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f6c004c80 con 0x5599a399c760 2024-09-10T08:44:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.475+0000 7f2f68c00640 1 -- 172.21.15.89:0/1701905784 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f2f6c004020 con 0x5599a399c760 2024-09-10T08:44:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.476+0000 7f2f68c00640 1 -- 172.21.15.89:0/1701905784 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2f6c02f050 con 0x5599a399c760 2024-09-10T08:44:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.476+0000 7f2f84a68740 10 client.26177 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.476+0000 7f2f84a68740 1 -- 172.21.15.89:0/1701905784 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5599a39b5ca0 con 0x5599a399c760 2024-09-10T08:44:21.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.476+0000 7f2f68c00640 1 -- 172.21.15.89:0/1701905784 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f6c004c80 con 0x5599a399c760 2024-09-10T08:44:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.476+0000 7f2f68c00640 1 -- 172.21.15.89:0/1701905784 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2f6c003ae0 con 0x5599a399c760 2024-09-10T08:44:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.476+0000 7f2f84a68740 10 client.26177 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.476+0000 7f2f84a68740 10 client.26177 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:44:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.476+0000 7f2f84a68740 20 client.26177 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.476+0000 7f2f84a68740 1 --2- 172.21.15.89:0/1701905784 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5599a39b70b0 0x5599a39b94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.476+0000 7f2f84a68740 4 client.26177 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:44:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.476+0000 7f2f84a68740 1 -- 172.21.15.89:0/1701905784 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599a390f950 con 0x5599a39b70b0 2024-09-10T08:44:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.477+0000 7f2f72000640 1 --2- 172.21.15.89:0/1701905784 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5599a39b70b0 0x5599a39b94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.477+0000 7f2f68c00640 10 client.26177 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:44:21.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.477+0000 7f2f72000640 1 --2- 172.21.15.89:0/1701905784 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5599a39b70b0 0x5599a39b94a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c96d3bbd1f3bd81 server_cookie=7bea139999b33fde in_seq=0 out_seq=0 2024-09-10T08:44:21.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.478+0000 7f2f68c00640 1 -- 172.21.15.89:0/1701905784 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599a390f950 con 0x5599a39b70b0 2024-09-10T08:44:21.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.478+0000 7f2f68c00640 10 client.26177 handle_command_reply: tid=0 2024-09-10T08:44:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.478+0000 7f2f84a68740 1 client.26177 shutdown 2024-09-10T08:44:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.478+0000 7f2f84a68740 1 -- 172.21.15.89:0/1701905784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a399c760 msgr2=0x5599a39a44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.478+0000 7f2f84a68740 1 --2- 172.21.15.89:0/1701905784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a399c760 0x5599a39a44f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c011010 tx=0x7f2f6c0048f0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.478+0000 7f2f84a68740 1 -- 172.21.15.89:0/1701905784 shutdown_connections 2024-09-10T08:44:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.478+0000 7f2f84a68740 1 --2- 172.21.15.89:0/1701905784 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a39a7020 0x5599a39a4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.479+0000 7f2f84a68740 1 --2- 172.21.15.89:0/1701905784 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5599a39b70b0 0x5599a39b94a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.479+0000 7f2f84a68740 1 --2- 172.21.15.89:0/1701905784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a399c760 0x5599a39a44f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.479+0000 7f2f84a68740 1 --2- 172.21.15.89:0/1701905784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599a399be40 0x5599a39a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.479+0000 7f2f68c00640 0 client.26177 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:44:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.479+0000 7f2f84a68740 1 -- 172.21.15.89:0/1701905784 >> 172.21.15.89:0/1701905784 conn(0x5599a39983d0 msgr2=0x5599a399e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.479+0000 7f2f84a68740 1 -- 172.21.15.89:0/1701905784 shutdown_connections 2024-09-10T08:44:21.482 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.479+0000 7f2f84a68740 1 -- 172.21.15.89:0/1701905784 wait complete. 2024-09-10T08:44:21.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.479+0000 7f2f84a68740 20 client.26177 trim_cache size 0 max 16384 2024-09-10T08:44:21.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.504+0000 7f2f84a68740 1 Processor -- start 2024-09-10T08:44:21.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.504+0000 7f2f84a68740 1 -- start start 2024-09-10T08:44:21.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.504+0000 7f2f84a68740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599a399be40 0x5599a39a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:21.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.505+0000 7f2f84a68740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a399c760 0x5599a39beaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:21.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.505+0000 7f2f84a68740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a39a7020 0x5599a39befe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:21.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.505+0000 7f2f84a68740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c1b37b0 con 0x5599a39a7020 2024-09-10T08:44:21.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.505+0000 7f2f84a68740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c1b1080 con 0x5599a399c760 2024-09-10T08:44:21.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.505+0000 7f2f84a68740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c1aeed0 con 0x5599a399be40 2024-09-10T08:44:21.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.505+0000 7f2f72000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a39a7020 0x5599a39befe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:21.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.505+0000 7f2f72000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a39a7020 0x5599a39befe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32798/0 (socket says 172.21.15.89:32798) 2024-09-10T08:44:21.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.505+0000 7f2f72000640 1 -- 172.21.15.89:0/1474110346 learned_addr learned my addr 172.21.15.89:0/1474110346 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:21.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.505+0000 7f2f7aa00640 1 --2- 172.21.15.89:0/1474110346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a399c760 0x5599a39beaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:21.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.505+0000 7f2f72000640 1 -- 172.21.15.89:0/1474110346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599a399be40 msgr2=0x5599a39a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:21.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.505+0000 7f2f72000640 1 --2- 172.21.15.89:0/1474110346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599a399be40 0x5599a39a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.505+0000 7f2f72000640 1 -- 172.21.15.89:0/1474110346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a399c760 msgr2=0x5599a39beaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:21.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.505+0000 7f2f72000640 1 --2- 172.21.15.89:0/1474110346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a399c760 0x5599a39beaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.505+0000 7f2f72000640 1 -- 172.21.15.89:0/1474110346 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599a39b2e50 con 0x5599a39a7020 2024-09-10T08:44:21.507 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.506+0000 7f2f72000640 1 --2- 172.21.15.89:0/1474110346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a39a7020 0x5599a39befe0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2f64031550 tx=0x7f2f64009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955e50b77093a04d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:21.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.506+0000 7f2f6b400640 1 -- 172.21.15.89:0/1474110346 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f64035040 con 0x5599a39a7020 2024-09-10T08:44:21.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.506+0000 7f2f6b400640 1 -- 172.21.15.89:0/1474110346 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f64005510 con 0x5599a39a7020 2024-09-10T08:44:21.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.506+0000 7f2f6b400640 1 -- 172.21.15.89:0/1474110346 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f64047590 con 0x5599a39a7020 2024-09-10T08:44:21.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.507+0000 7f2f84a68740 1 -- 172.21.15.89:0/1474110346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a39a7020 msgr2=0x5599a39befe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:21.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.507+0000 7f2f84a68740 1 --2- 172.21.15.89:0/1474110346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a39a7020 0x5599a39befe0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2f64031550 tx=0x7f2f64009670 comp rx=0 tx=0).stop 2024-09-10T08:44:21.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.507+0000 7f2f84a68740 1 -- 172.21.15.89:0/1474110346 shutdown_connections 2024-09-10T08:44:21.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.507+0000 7f2f84a68740 1 --2- 172.21.15.89:0/1474110346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a39a7020 0x5599a39befe0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.507+0000 7f2f84a68740 1 --2- 172.21.15.89:0/1474110346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a399c760 0x5599a39beaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.507+0000 7f2f84a68740 1 --2- 172.21.15.89:0/1474110346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599a399be40 0x5599a39a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.507+0000 7f2f84a68740 1 -- 172.21.15.89:0/1474110346 >> 172.21.15.89:0/1474110346 conn(0x5599a39983d0 msgr2=0x5599a39a5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:21.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.507+0000 7f2f84a68740 1 -- 172.21.15.89:0/1474110346 shutdown_connections 2024-09-10T08:44:21.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.507+0000 7f2f84a68740 1 -- 172.21.15.89:0/1474110346 wait complete. 2024-09-10T08:44:21.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.507+0000 7f2f84a68740 1 -- start start 2024-09-10T08:44:21.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.507+0000 7f2f84a68740 1 Processor -- start 2024-09-10T08:44:21.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.508+0000 7f2f84a68740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a399be40 0x5599a39aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:21.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.508+0000 7f2f84a68740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599a399c760 0x5599a39af500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:21.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.508+0000 7f2f84a68740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a39a7020 0x5599a39abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:21.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.508+0000 7f2f84a68740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c1af540 con 0x5599a399be40 2024-09-10T08:44:21.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.508+0000 7f2f84a68740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c1b09b0 con 0x5599a39a7020 2024-09-10T08:44:21.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.508+0000 7f2f84a68740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c1afc10 con 0x5599a399c760 2024-09-10T08:44:21.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.508+0000 7f2f84a68740 10 client.0 _ll_register_callbacks cb 0x7f2f6c00bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:21.510 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.508+0000 7f2f72000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a399be40 0x5599a39aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:21.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.508+0000 7f2f72000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a399be40 0x5599a39aefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:32804/0 (socket says 172.21.15.89:32804) 2024-09-10T08:44:21.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.508+0000 7f2f72000640 1 -- 172.21.15.89:0/1301980849 learned_addr learned my addr 172.21.15.89:0/1301980849 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:21.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.508+0000 7f2f7aa00640 1 --2- 172.21.15.89:0/1301980849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599a399c760 0x5599a39af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:21.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.508+0000 7f2f72000640 1 -- 172.21.15.89:0/1301980849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599a399c760 msgr2=0x5599a39af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:21.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.508+0000 7f2f72000640 1 --2- 172.21.15.89:0/1301980849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599a399c760 0x5599a39af500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.509+0000 7f2f72000640 1 -- 172.21.15.89:0/1301980849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a39a7020 msgr2=0x5599a39abf90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:44:21.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.509+0000 7f2f72000640 1 --2- 172.21.15.89:0/1301980849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a39a7020 0x5599a39abf90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.509+0000 7f2f72000640 1 -- 172.21.15.89:0/1301980849 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599a39ac4d0 con 0x5599a399be40 2024-09-10T08:44:21.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.509+0000 7f2f7aa00640 1 --2- 172.21.15.89:0/1301980849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599a399c760 0x5599a39af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:21.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.509+0000 7f2f72000640 1 --2- 172.21.15.89:0/1301980849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a399be40 0x5599a39aefc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2f64005a60 tx=0x7f2f640325c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7318512a3691f417 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:21.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.509+0000 7f2f68c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:44:21.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.509+0000 7f2f68c00640 1 -- 172.21.15.89:0/1301980849 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f64035030 con 0x5599a399be40 2024-09-10T08:44:21.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.509+0000 7f2f84a68740 1 -- 172.21.15.89:0/1301980849 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f64005510 con 0x5599a399be40 2024-09-10T08:44:21.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.509+0000 7f2f68c00640 1 -- 172.21.15.89:0/1301980849 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f2f640052f0 con 0x5599a399be40 2024-09-10T08:44:21.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.509+0000 7f2f68c00640 1 -- 172.21.15.89:0/1301980849 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2f64004760 con 0x5599a399be40 2024-09-10T08:44:21.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.509+0000 7f2f84a68740 10 client.16110 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:21.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.509+0000 7f2f84a68740 1 -- 172.21.15.89:0/1301980849 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5599a3a22170 con 0x5599a399be40 2024-09-10T08:44:21.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.510+0000 7f2f68c00640 1 -- 172.21.15.89:0/1301980849 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f64005510 con 0x5599a399be40 2024-09-10T08:44:21.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.510+0000 7f2f68c00640 1 -- 172.21.15.89:0/1301980849 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2f64003990 con 0x5599a399be40 2024-09-10T08:44:21.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.510+0000 7f2f84a68740 10 client.16110 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:21.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.510+0000 7f2f84a68740 10 client.16110 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:44:21.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.510+0000 7f2f84a68740 20 client.16110 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:21.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.510+0000 7f2f84a68740 1 --2- 172.21.15.89:0/1301980849 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5599a3a22860 0x5599a3a24c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:21.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.510+0000 7f2f84a68740 4 client.16110 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:44:21.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.510+0000 7f2f84a68740 1 -- 172.21.15.89:0/1301980849 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599a3909e60 con 0x5599a3a22860 2024-09-10T08:44:21.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.510+0000 7f2f7aa00640 1 --2- 172.21.15.89:0/1301980849 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5599a3a22860 0x5599a3a24c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:21.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.511+0000 7f2f7aa00640 1 --2- 172.21.15.89:0/1301980849 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5599a3a22860 0x5599a3a24c80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e33fa500c1d8aee1 server_cookie=e118a4542d3a354f in_seq=0 out_seq=0 2024-09-10T08:44:21.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.511+0000 7f2f68c00640 10 client.16110 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:44:21.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.511+0000 7f2f68c00640 1 -- 172.21.15.89:0/1301980849 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599a3909e60 con 0x5599a3a22860 2024-09-10T08:44:21.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.511+0000 7f2f68c00640 10 client.16110 handle_command_reply: tid=0 2024-09-10T08:44:21.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.511+0000 7f2f84a68740 1 client.16110 shutdown 2024-09-10T08:44:21.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.511+0000 7f2f84a68740 1 -- 172.21.15.89:0/1301980849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a399be40 msgr2=0x5599a39aefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:21.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.511+0000 7f2f84a68740 1 --2- 172.21.15.89:0/1301980849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a399be40 0x5599a39aefc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2f64005a60 tx=0x7f2f640325c0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.512+0000 7f2f84a68740 1 -- 172.21.15.89:0/1301980849 shutdown_connections 2024-09-10T08:44:21.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.512+0000 7f2f84a68740 1 --2- 172.21.15.89:0/1301980849 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5599a3a22860 0x5599a3a24c80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.512+0000 7f2f84a68740 1 --2- 172.21.15.89:0/1301980849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599a39a7020 0x5599a39abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.512+0000 7f2f84a68740 1 --2- 172.21.15.89:0/1301980849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599a399c760 0x5599a39af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.512+0000 7f2f84a68740 1 --2- 172.21.15.89:0/1301980849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599a399be40 0x5599a39aefc0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.512+0000 7f2f84a68740 1 -- 172.21.15.89:0/1301980849 >> 172.21.15.89:0/1301980849 conn(0x5599a39983d0 msgr2=0x5599a39b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:21.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.512+0000 7f2f68c00640 0 client.16110 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:44:21.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.512+0000 7f2f84a68740 1 -- 172.21.15.89:0/1301980849 shutdown_connections 2024-09-10T08:44:21.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.512+0000 7f2f84a68740 1 -- 172.21.15.89:0/1301980849 wait complete. 2024-09-10T08:44:21.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.512+0000 7f2f84a68740 20 client.16110 trim_cache size 0 max 16384 2024-09-10T08:44:21.517 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:44:21.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.513+0000 7f2f7be00640 1 -- 172.21.15.89:0/3420000082 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2f600a2c00 msgr2=0x7f2f600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:21.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.513+0000 7f2f7be00640 1 --2- 172.21.15.89:0/3420000082 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2f600a2c00 0x7f2f600a50c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2f74005910 tx=0x7f2f7400cf90 comp rx=0 tx=0).stop 2024-09-10T08:44:21.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.513+0000 7f2f7be00640 1 -- 172.21.15.89:0/3420000082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f7c10e260 msgr2=0x7f2f7c19ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:21.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.513+0000 7f2f7be00640 1 --2- 172.21.15.89:0/3420000082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f7c10e260 0x7f2f7c19ed60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2f74005fd0 tx=0x7f2f7400ef10 comp rx=0 tx=0).stop 2024-09-10T08:44:21.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.513+0000 7f2f7be00640 1 -- 172.21.15.89:0/3420000082 shutdown_connections 2024-09-10T08:44:21.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.513+0000 7f2f7be00640 1 --2- 172.21.15.89:0/3420000082 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2f600a2c00 0x7f2f600a50c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.513+0000 7f2f7be00640 1 --2- 172.21.15.89:0/3420000082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f7c10e260 0x7f2f7c19ed60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.513+0000 7f2f7be00640 1 --2- 172.21.15.89:0/3420000082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f7c074ee0 0x7f2f7c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.513+0000 7f2f7be00640 1 --2- 172.21.15.89:0/3420000082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f7c0745c0 0x7f2f7c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:21.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.513+0000 7f2f7be00640 1 -- 172.21.15.89:0/3420000082 >> 172.21.15.89:0/3420000082 conn(0x7f2f7c0fd740 msgr2=0x7f2f7c0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:21.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.513+0000 7f2f7be00640 1 -- 172.21.15.89:0/3420000082 shutdown_connections 2024-09-10T08:44:21.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:21.513+0000 7f2f7be00640 1 -- 172.21.15.89:0/3420000082 wait complete. 2024-09-10T08:44:22.476 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:44:22.848 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:23.016 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:22 smithi089 ceph-mon[30757]: pgmap v526: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-09-10T08:44:23.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:22 smithi149 ceph-mon[36167]: pgmap v526: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-09-10T08:44:23.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:22 smithi111 ceph-mon[35220]: pgmap v526: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-09-10T08:44:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.476+0000 7fdd22c00640 1 -- 172.21.15.89:0/1970123167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd1c104d60 msgr2=0x7fdd1c1051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.476+0000 7fdd22c00640 1 --2- 172.21.15.89:0/1970123167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd1c104d60 0x7fdd1c1051e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009900 tx=0x7fdd0802a410 comp rx=0 tx=0).stop 2024-09-10T08:44:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.476+0000 7fdd22c00640 1 -- 172.21.15.89:0/1970123167 shutdown_connections 2024-09-10T08:44:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.476+0000 7fdd22c00640 1 --2- 172.21.15.89:0/1970123167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd1c105720 0x7fdd1c10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.476+0000 7fdd22c00640 1 --2- 172.21.15.89:0/1970123167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd1c104d60 0x7fdd1c1051e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.476+0000 7fdd22c00640 1 --2- 172.21.15.89:0/1970123167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd1c103b70 0x7fdd1c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.476+0000 7fdd22c00640 1 -- 172.21.15.89:0/1970123167 >> 172.21.15.89:0/1970123167 conn(0x7fdd1c0ff2f0 msgr2=0x7fdd1c101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.476+0000 7fdd22c00640 1 -- 172.21.15.89:0/1970123167 shutdown_connections 2024-09-10T08:44:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.476+0000 7fdd22c00640 1 -- 172.21.15.89:0/1970123167 wait complete. 2024-09-10T08:44:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.477+0000 7fdd22c00640 1 Processor -- start 2024-09-10T08:44:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.477+0000 7fdd22c00640 1 -- start start 2024-09-10T08:44:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.477+0000 7fdd22c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd1c103b70 0x7fdd1c199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.477+0000 7fdd22c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd1c104d60 0x7fdd1c19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.477+0000 7fdd22c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd1c105720 0x7fdd1c1a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.477+0000 7fdd22c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c10f840 con 0x7fdd1c105720 2024-09-10T08:44:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.477+0000 7fdd22c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c10f6c0 con 0x7fdd1c104d60 2024-09-10T08:44:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.477+0000 7fdd22c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c10f9c0 con 0x7fdd1c103b70 2024-09-10T08:44:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.478+0000 7fdd22200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd1c105720 0x7fdd1c1a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.478+0000 7fdd22200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd1c105720 0x7fdd1c1a1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36130/0 (socket says 172.21.15.89:36130) 2024-09-10T08:44:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.478+0000 7fdd22200640 1 -- 172.21.15.89:0/3404969306 learned_addr learned my addr 172.21.15.89:0/3404969306 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.478+0000 7fdd21800640 1 --2- 172.21.15.89:0/3404969306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd1c103b70 0x7fdd1c199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.478+0000 7fdd20e00640 1 --2- 172.21.15.89:0/3404969306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd1c104d60 0x7fdd1c19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.478+0000 7fdd22200640 1 -- 172.21.15.89:0/3404969306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd1c103b70 msgr2=0x7fdd1c199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.478+0000 7fdd22200640 1 --2- 172.21.15.89:0/3404969306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd1c103b70 0x7fdd1c199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.478+0000 7fdd22200640 1 -- 172.21.15.89:0/3404969306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd1c104d60 msgr2=0x7fdd1c19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.478+0000 7fdd22200640 1 --2- 172.21.15.89:0/3404969306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd1c104d60 0x7fdd1c19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.478+0000 7fdd22200640 1 -- 172.21.15.89:0/3404969306 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1c1a1a80 con 0x7fdd1c105720 2024-09-10T08:44:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.479+0000 7fdd22200640 1 --2- 172.21.15.89:0/3404969306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd1c105720 0x7fdd1c1a1410 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400eb10 tx=0x7fdd1400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.479+0000 7fdd1a000640 1 -- 172.21.15.89:0/3404969306 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14004b20 con 0x7fdd1c105720 2024-09-10T08:44:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.479+0000 7fdd1a000640 1 -- 172.21.15.89:0/3404969306 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd14004cc0 con 0x7fdd1c105720 2024-09-10T08:44:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.479+0000 7fdd22c00640 1 -- 172.21.15.89:0/3404969306 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1c1a1d70 con 0x7fdd1c105720 2024-09-10T08:44:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.479+0000 7fdd1a000640 1 -- 172.21.15.89:0/3404969306 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14010760 con 0x7fdd1c105720 2024-09-10T08:44:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.479+0000 7fdd22c00640 1 -- 172.21.15.89:0/3404969306 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd1c1a2280 con 0x7fdd1c105720 2024-09-10T08:44:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.482+0000 7fdd1a000640 1 -- 172.21.15.89:0/3404969306 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdd14005020 con 0x7fdd1c105720 2024-09-10T08:44:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.483+0000 7fdd2726f740 1 Processor -- start 2024-09-10T08:44:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.483+0000 7fdd2726f740 1 -- start start 2024-09-10T08:44:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.483+0000 7fdd2726f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560589c95260 0x560589c95660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.483+0000 7fdd2726f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560589c95ba0 0x560589c9a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.483+0000 7fdd2726f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c9ac30 0x560589c9d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.483+0000 7fdd2726f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560589bfa100 con 0x560589c9ac30 2024-09-10T08:44:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.483+0000 7fdd2726f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560589ba21b0 con 0x560589c95ba0 2024-09-10T08:44:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.483+0000 7fdd2726f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560589bf60c0 con 0x560589c95260 2024-09-10T08:44:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.483+0000 7fdd20e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560589c95260 0x560589c95660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.483+0000 7fdd20e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c9ac30 0x560589c9d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.483+0000 7fdd1a000640 1 --2- 172.21.15.89:0/3404969306 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdd040a2c00 0x7fdd040a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.483+0000 7fdd21800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560589c95ba0 0x560589c9a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.503 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.483+0000 7fdd20e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c9ac30 0x560589c9d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36138/0 (socket says 172.21.15.89:36138) 2024-09-10T08:44:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.483+0000 7fdd20e00640 1 -- 172.21.15.89:0/1715903274 learned_addr learned my addr 172.21.15.89:0/1715903274 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.484+0000 7fdd1a000640 1 -- 172.21.15.89:0/3404969306 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fdd140cfa90 con 0x7fdd1c105720 2024-09-10T08:44:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.484+0000 7fdd22200640 1 --2- 172.21.15.89:0/3404969306 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdd040a2c00 0x7fdd040a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.484+0000 7fdd20e00640 1 -- 172.21.15.89:0/1715903274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560589c95260 msgr2=0x560589c95660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:23.504 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.484+0000 7fdd20e00640 1 --2- 172.21.15.89:0/1715903274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560589c95260 0x560589c95660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.484+0000 7fdd20e00640 1 -- 172.21.15.89:0/1715903274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560589c95ba0 msgr2=0x560589c9a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.484+0000 7fdd20e00640 1 --2- 172.21.15.89:0/1715903274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560589c95ba0 0x560589c9a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.484+0000 7fdd20e00640 1 -- 172.21.15.89:0/1715903274 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56058990d210 con 0x560589c9ac30 2024-09-10T08:44:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.484+0000 7fdd20e00640 1 --2- 172.21.15.89:0/1715903274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c9ac30 0x560589c9d0f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdd080314e0 tx=0x7fdd080315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d3be0dbd767424 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.484+0000 7fdd22200640 1 --2- 172.21.15.89:0/3404969306 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdd040a2c00 0x7fdd040a50c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdd14004f90 tx=0x7fdd140058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:23.505 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.485+0000 7fdd0ea00640 1 -- 172.21.15.89:0/1715903274 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08035050 con 0x560589c9ac30 2024-09-10T08:44:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.485+0000 7fdd0ea00640 1 -- 172.21.15.89:0/1715903274 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd08032e50 con 0x560589c9ac30 2024-09-10T08:44:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.485+0000 7fdd0ea00640 1 -- 172.21.15.89:0/1715903274 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08003ea0 con 0x560589c9ac30 2024-09-10T08:44:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.485+0000 7fdd2726f740 1 -- 172.21.15.89:0/1715903274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c9ac30 msgr2=0x560589c9d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.485+0000 7fdd2726f740 1 --2- 172.21.15.89:0/1715903274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c9ac30 0x560589c9d0f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdd080314e0 tx=0x7fdd080315b0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.485+0000 7fdd2726f740 1 -- 172.21.15.89:0/1715903274 shutdown_connections 2024-09-10T08:44:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.485+0000 7fdd2726f740 1 --2- 172.21.15.89:0/1715903274 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c9ac30 0x560589c9d0f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.485+0000 7fdd2726f740 1 --2- 172.21.15.89:0/1715903274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560589c95ba0 0x560589c9a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.485+0000 7fdd2726f740 1 --2- 172.21.15.89:0/1715903274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560589c95260 0x560589c95660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.507 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.485+0000 7fdd2726f740 1 -- 172.21.15.89:0/1715903274 >> 172.21.15.89:0/1715903274 conn(0x560589c908d0 msgr2=0x560589c90cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.485+0000 7fdd2726f740 1 -- 172.21.15.89:0/1715903274 shutdown_connections 2024-09-10T08:44:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.485+0000 7fdd2726f740 1 -- 172.21.15.89:0/1715903274 wait complete. 2024-09-10T08:44:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.486+0000 7fdd2726f740 1 -- start start 2024-09-10T08:44:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.486+0000 7fdd2726f740 1 Processor -- start 2024-09-10T08:44:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.486+0000 7fdd2726f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c95260 0x560589c9f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.486+0000 7fdd2726f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560589c95ba0 0x560589c9f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.486+0000 7fdd2726f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560589c9ac30 0x560589ca6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.486+0000 7fdd2726f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560589c9dba0 con 0x560589c95260 2024-09-10T08:44:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.486+0000 7fdd2726f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560589b83090 con 0x560589c9ac30 2024-09-10T08:44:23.509 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.486+0000 7fdd2726f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560589c9dd20 con 0x560589c95ba0 2024-09-10T08:44:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.486+0000 7fdd2726f740 10 client.0 _ll_register_callbacks cb 0x560589bd7f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.487+0000 7fdd20e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c95260 0x560589c9f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.487+0000 7fdd20e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c95260 0x560589c9f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36142/0 (socket says 172.21.15.89:36142) 2024-09-10T08:44:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.487+0000 7fdd20e00640 1 -- 172.21.15.89:0/2832992810 learned_addr learned my addr 172.21.15.89:0/2832992810 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.487+0000 7fdd21800640 1 --2- 172.21.15.89:0/2832992810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560589c95ba0 0x560589c9f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.510 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.487+0000 7fdd20e00640 1 -- 172.21.15.89:0/2832992810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560589c95ba0 msgr2=0x560589c9f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.487+0000 7fdd20e00640 1 --2- 172.21.15.89:0/2832992810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560589c95ba0 0x560589c9f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.487+0000 7fdd20e00640 1 -- 172.21.15.89:0/2832992810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560589c9ac30 msgr2=0x560589ca6960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:44:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.487+0000 7fdd20e00640 1 --2- 172.21.15.89:0/2832992810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560589c9ac30 0x560589ca6960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.487+0000 7fdd20e00640 1 -- 172.21.15.89:0/2832992810 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605899d9620 con 0x560589c95260 2024-09-10T08:44:23.511 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.487+0000 7fdd20e00640 1 --2- 172.21.15.89:0/2832992810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c95260 0x560589c9f3e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdd080096c0 tx=0x7fdd08005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ea7b3ec6daf286 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.487+0000 7fdd03400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:44:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.488+0000 7fdd03400640 1 -- 172.21.15.89:0/2832992810 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0804e070 con 0x560589c95260 2024-09-10T08:44:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.488+0000 7fdd2726f740 1 -- 172.21.15.89:0/2832992810 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd08032e50 con 0x560589c95260 2024-09-10T08:44:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.488+0000 7fdd03400640 1 -- 172.21.15.89:0/2832992810 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fdd08003e40 con 0x560589c95260 2024-09-10T08:44:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.488+0000 7fdd03400640 1 -- 172.21.15.89:0/2832992810 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd0805a050 con 0x560589c95260 2024-09-10T08:44:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.488+0000 7fdd2726f740 10 client.16122 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.488+0000 7fdd2726f740 1 -- 172.21.15.89:0/2832992810 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560589ca9750 con 0x560589c95260 2024-09-10T08:44:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.488+0000 7fdd03400640 1 -- 172.21.15.89:0/2832992810 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd08032e50 con 0x560589c95260 2024-09-10T08:44:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.488+0000 7fdd03400640 1 -- 172.21.15.89:0/2832992810 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdd0803b8d0 con 0x560589c95260 2024-09-10T08:44:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.489+0000 7fdd2726f740 10 client.16122 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.489+0000 7fdd2726f740 10 client.16122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:44:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.489+0000 7fdd2726f740 20 client.16122 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.489+0000 7fdd2726f740 1 --2- 172.21.15.89:0/2832992810 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560589caaaa0 0x560589cace90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.489+0000 7fdd2726f740 4 client.16122 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:44:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.489+0000 7fdd2726f740 1 -- 172.21.15.89:0/2832992810 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560589c07830 con 0x560589caaaa0 2024-09-10T08:44:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.489+0000 7fdd21800640 1 --2- 172.21.15.89:0/2832992810 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560589caaaa0 0x560589cace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.489+0000 7fdd21800640 1 --2- 172.21.15.89:0/2832992810 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560589caaaa0 0x560589cace90 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e677ae8ddcf7540 server_cookie=fce27211f9d804bc in_seq=0 out_seq=0 2024-09-10T08:44:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.489+0000 7fdd03400640 10 client.16122 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.490+0000 7fdd03400640 1 -- 172.21.15.89:0/2832992810 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560589c07830 con 0x560589caaaa0 2024-09-10T08:44:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.490+0000 7fdd03400640 10 client.16122 handle_command_reply: tid=0 2024-09-10T08:44:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.490+0000 7fdd2726f740 1 client.16122 shutdown 2024-09-10T08:44:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.490+0000 7fdd2726f740 1 -- 172.21.15.89:0/2832992810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c95260 msgr2=0x560589c9f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.490+0000 7fdd2726f740 1 --2- 172.21.15.89:0/2832992810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c95260 0x560589c9f3e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdd080096c0 tx=0x7fdd08005d70 comp rx=0 tx=0).stop 2024-09-10T08:44:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.490+0000 7fdd2726f740 1 -- 172.21.15.89:0/2832992810 shutdown_connections 2024-09-10T08:44:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.490+0000 7fdd2726f740 1 --2- 172.21.15.89:0/2832992810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560589c9ac30 0x560589ca6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.490+0000 7fdd2726f740 1 --2- 172.21.15.89:0/2832992810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560589c95ba0 0x560589c9f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.490+0000 7fdd2726f740 1 --2- 172.21.15.89:0/2832992810 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560589caaaa0 0x560589cace90 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.490+0000 7fdd2726f740 1 --2- 172.21.15.89:0/2832992810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c95260 0x560589c9f3e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.490+0000 7fdd2726f740 1 -- 172.21.15.89:0/2832992810 >> 172.21.15.89:0/2832992810 conn(0x560589c908d0 msgr2=0x560589c920e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.491+0000 7fdd03400640 0 client.16122 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.491+0000 7fdd2726f740 1 -- 172.21.15.89:0/2832992810 shutdown_connections 2024-09-10T08:44:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.491+0000 7fdd2726f740 1 -- 172.21.15.89:0/2832992810 wait complete. 2024-09-10T08:44:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.491+0000 7fdd2726f740 20 client.16122 trim_cache size 0 max 16384 2024-09-10T08:44:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.505+0000 7fdd2726f740 1 Processor -- start 2024-09-10T08:44:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.505+0000 7fdd2726f740 1 -- start start 2024-09-10T08:44:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.506+0000 7fdd2726f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560589c95260 0x560589c95640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.506+0000 7fdd2726f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c95ba0 0x560589ca68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.506+0000 7fdd2726f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560589c9ac30 0x560589c9f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.506+0000 7fdd2726f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c1af140 con 0x560589c95ba0 2024-09-10T08:44:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.506+0000 7fdd2726f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c1aca10 con 0x560589c95260 2024-09-10T08:44:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.506+0000 7fdd2726f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c1aa860 con 0x560589c9ac30 2024-09-10T08:44:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.506+0000 7fdd21800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c95ba0 0x560589ca68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.506+0000 7fdd21800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c95ba0 0x560589ca68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36152/0 (socket says 172.21.15.89:36152) 2024-09-10T08:44:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.506+0000 7fdd21800640 1 -- 172.21.15.89:0/1475375588 learned_addr learned my addr 172.21.15.89:0/1475375588 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.506+0000 7fdd20e00640 1 --2- 172.21.15.89:0/1475375588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560589c95260 0x560589c95640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.506+0000 7fdd20e00640 1 --2- 172.21.15.89:0/1475375588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560589c9ac30 0x560589c9f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.506+0000 7fdd21800640 1 -- 172.21.15.89:0/1475375588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560589c9ac30 msgr2=0x560589c9f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.506+0000 7fdd21800640 1 --2- 172.21.15.89:0/1475375588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560589c9ac30 0x560589c9f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.506+0000 7fdd21800640 1 -- 172.21.15.89:0/1475375588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560589c95260 msgr2=0x560589c95640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.506+0000 7fdd21800640 1 --2- 172.21.15.89:0/1475375588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560589c95260 0x560589c95640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.506+0000 7fdd21800640 1 -- 172.21.15.89:0/1475375588 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560589c906f0 con 0x560589c95ba0 2024-09-10T08:44:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.507+0000 7fdd21800640 1 --2- 172.21.15.89:0/1475375588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c95ba0 0x560589ca68f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdd10004700 tx=0x7fdd10005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632869dad7db169d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.507+0000 7fdd0e000640 1 -- 172.21.15.89:0/1475375588 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10022070 con 0x560589c95ba0 2024-09-10T08:44:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.507+0000 7fdd0e000640 1 -- 172.21.15.89:0/1475375588 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd10004250 con 0x560589c95ba0 2024-09-10T08:44:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.507+0000 7fdd0e000640 1 -- 172.21.15.89:0/1475375588 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1001f7b0 con 0x560589c95ba0 2024-09-10T08:44:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.507+0000 7fdd2726f740 1 -- 172.21.15.89:0/1475375588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c95ba0 msgr2=0x560589ca68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.507+0000 7fdd2726f740 1 --2- 172.21.15.89:0/1475375588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c95ba0 0x560589ca68f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdd10004700 tx=0x7fdd10005f90 comp rx=0 tx=0).stop 2024-09-10T08:44:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.508+0000 7fdd2726f740 1 -- 172.21.15.89:0/1475375588 shutdown_connections 2024-09-10T08:44:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.508+0000 7fdd2726f740 1 --2- 172.21.15.89:0/1475375588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560589c9ac30 0x560589c9f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.508+0000 7fdd2726f740 1 --2- 172.21.15.89:0/1475375588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c95ba0 0x560589ca68f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.508+0000 7fdd2726f740 1 --2- 172.21.15.89:0/1475375588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560589c95260 0x560589c95640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.508+0000 7fdd2726f740 1 -- 172.21.15.89:0/1475375588 >> 172.21.15.89:0/1475375588 conn(0x560589c908d0 msgr2=0x560589c9a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.508+0000 7fdd2726f740 1 -- 172.21.15.89:0/1475375588 shutdown_connections 2024-09-10T08:44:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.508+0000 7fdd2726f740 1 -- 172.21.15.89:0/1475375588 wait complete. 2024-09-10T08:44:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.508+0000 7fdd2726f740 1 -- start start 2024-09-10T08:44:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.508+0000 7fdd2726f740 1 Processor -- start 2024-09-10T08:44:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.508+0000 7fdd2726f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560589c95260 0x560589ca3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.508+0000 7fdd2726f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560589c95ba0 0x560589ca3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.508+0000 7fdd2726f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c9ac30 0x560589d136f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.508+0000 7fdd2726f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c1aaed0 con 0x560589c9ac30 2024-09-10T08:44:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.509+0000 7fdd2726f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c1ac340 con 0x560589c95260 2024-09-10T08:44:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.509+0000 7fdd2726f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c1ab5a0 con 0x560589c95ba0 2024-09-10T08:44:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.509+0000 7fdd2726f740 10 client.0 _ll_register_callbacks cb 0x7fdd100097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.509+0000 7fdd20e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c9ac30 0x560589d136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.509+0000 7fdd20e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c9ac30 0x560589d136f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36154/0 (socket says 172.21.15.89:36154) 2024-09-10T08:44:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.509+0000 7fdd20e00640 1 -- 172.21.15.89:0/39118464 learned_addr learned my addr 172.21.15.89:0/39118464 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.509+0000 7fdd21800640 1 --2- 172.21.15.89:0/39118464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560589c95ba0 0x560589ca3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.509+0000 7fdd20e00640 1 --2- 172.21.15.89:0/39118464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560589c95260 0x560589ca3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.509+0000 7fdd20e00640 1 -- 172.21.15.89:0/39118464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560589c95ba0 msgr2=0x560589ca3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.509+0000 7fdd20e00640 1 --2- 172.21.15.89:0/39118464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560589c95ba0 0x560589ca3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.509+0000 7fdd20e00640 1 -- 172.21.15.89:0/39118464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560589c95260 msgr2=0x560589ca3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.509+0000 7fdd20e00640 1 --2- 172.21.15.89:0/39118464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560589c95260 0x560589ca3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.509+0000 7fdd20e00640 1 -- 172.21.15.89:0/39118464 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560589ca6bb0 con 0x560589c9ac30 2024-09-10T08:44:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.510+0000 7fdd20e00640 1 --2- 172.21.15.89:0/39118464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c9ac30 0x560589d136f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdd0803d010 tx=0x7fdd0802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb9029dedd3f99d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.510+0000 7fdd03400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:44:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.510+0000 7fdd03400640 1 -- 172.21.15.89:0/39118464 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0804e040 con 0x560589c9ac30 2024-09-10T08:44:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.510+0000 7fdd03400640 1 -- 172.21.15.89:0/39118464 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fdd0803b460 con 0x560589c9ac30 2024-09-10T08:44:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.510+0000 7fdd2726f740 1 -- 172.21.15.89:0/39118464 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd10004250 con 0x560589c9ac30 2024-09-10T08:44:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.510+0000 7fdd03400640 1 -- 172.21.15.89:0/39118464 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd08035040 con 0x560589c9ac30 2024-09-10T08:44:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.510+0000 7fdd2726f740 10 client.16128 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.510+0000 7fdd2726f740 1 -- 172.21.15.89:0/39118464 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560589d14ca0 con 0x560589c9ac30 2024-09-10T08:44:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.510+0000 7fdd03400640 1 -- 172.21.15.89:0/39118464 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd10004250 con 0x560589c9ac30 2024-09-10T08:44:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.511+0000 7fdd03400640 1 -- 172.21.15.89:0/39118464 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdd08031c00 con 0x560589c9ac30 2024-09-10T08:44:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.511+0000 7fdd2726f740 10 client.16128 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.511+0000 7fdd2726f740 10 client.16128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:44:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.511+0000 7fdd2726f740 20 client.16128 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.511+0000 7fdd2726f740 1 --2- 172.21.15.89:0/39118464 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560589d15620 0x560589d17a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.511+0000 7fdd2726f740 4 client.16128 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:44:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.511+0000 7fdd2726f740 1 -- 172.21.15.89:0/39118464 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560589c01b80 con 0x560589d15620 2024-09-10T08:44:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.511+0000 7fdd21800640 1 --2- 172.21.15.89:0/39118464 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560589d15620 0x560589d17a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.512+0000 7fdd03400640 10 client.16128 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.512+0000 7fdd21800640 1 --2- 172.21.15.89:0/39118464 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560589d15620 0x560589d17a40 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de9d0d2a3726029f server_cookie=5e7bb6b79e4460b4 in_seq=0 out_seq=0 2024-09-10T08:44:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.524+0000 7fdd03400640 1 -- 172.21.15.89:0/39118464 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560589c01b80 con 0x560589d15620 2024-09-10T08:44:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.524+0000 7fdd03400640 10 client.16128 handle_command_reply: tid=0 2024-09-10T08:44:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.524+0000 7fdd2726f740 1 client.16128 shutdown 2024-09-10T08:44:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.524+0000 7fdd2726f740 1 -- 172.21.15.89:0/39118464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c9ac30 msgr2=0x560589d136f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.524+0000 7fdd2726f740 1 --2- 172.21.15.89:0/39118464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c9ac30 0x560589d136f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdd0803d010 tx=0x7fdd0802a410 comp rx=0 tx=0).stop 2024-09-10T08:44:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.524+0000 7fdd2726f740 1 -- 172.21.15.89:0/39118464 shutdown_connections 2024-09-10T08:44:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.524+0000 7fdd2726f740 1 --2- 172.21.15.89:0/39118464 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560589d15620 0x560589d17a40 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.524+0000 7fdd2726f740 1 --2- 172.21.15.89:0/39118464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560589c9ac30 0x560589d136f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.524+0000 7fdd2726f740 1 --2- 172.21.15.89:0/39118464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560589c95ba0 0x560589ca3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.524+0000 7fdd03400640 0 client.16128 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.524+0000 7fdd2726f740 1 --2- 172.21.15.89:0/39118464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560589c95260 0x560589ca3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.524+0000 7fdd2726f740 1 -- 172.21.15.89:0/39118464 >> 172.21.15.89:0/39118464 conn(0x560589c908d0 msgr2=0x560589c9f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.524+0000 7fdd2726f740 1 -- 172.21.15.89:0/39118464 shutdown_connections 2024-09-10T08:44:23.543 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.524+0000 7fdd2726f740 1 -- 172.21.15.89:0/39118464 wait complete. 2024-09-10T08:44:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.524+0000 7fdd2726f740 20 client.16128 trim_cache size 0 max 16384 2024-09-10T08:44:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:44:23.544 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:44:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:44:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:44:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.525+0000 7fdd22c00640 1 -- 172.21.15.89:0/3404969306 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdd040a2c00 msgr2=0x7fdd040a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.525+0000 7fdd22c00640 1 --2- 172.21.15.89:0/3404969306 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdd040a2c00 0x7fdd040a50c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdd14004f90 tx=0x7fdd140058d0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.525+0000 7fdd22c00640 1 -- 172.21.15.89:0/3404969306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd1c105720 msgr2=0x7fdd1c1a1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.525+0000 7fdd22c00640 1 --2- 172.21.15.89:0/3404969306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd1c105720 0x7fdd1c1a1410 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400eb10 tx=0x7fdd1400c2d0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.525+0000 7fdd22c00640 1 -- 172.21.15.89:0/3404969306 shutdown_connections 2024-09-10T08:44:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.525+0000 7fdd22c00640 1 --2- 172.21.15.89:0/3404969306 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdd040a2c00 0x7fdd040a50c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.525+0000 7fdd22c00640 1 --2- 172.21.15.89:0/3404969306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd1c105720 0x7fdd1c1a1410 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.525+0000 7fdd22c00640 1 --2- 172.21.15.89:0/3404969306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd1c104d60 0x7fdd1c19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.525+0000 7fdd22c00640 1 --2- 172.21.15.89:0/3404969306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd1c103b70 0x7fdd1c199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.525+0000 7fdd22c00640 1 -- 172.21.15.89:0/3404969306 >> 172.21.15.89:0/3404969306 conn(0x7fdd1c0ff2f0 msgr2=0x7fdd1c1080b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.525+0000 7fdd22c00640 1 -- 172.21.15.89:0/3404969306 shutdown_connections 2024-09-10T08:44:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:23.525+0000 7fdd22c00640 1 -- 172.21.15.89:0/3404969306 wait complete. 2024-09-10T08:44:23.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.708+0000 7f8d2b200640 1 -- 172.21.15.89:0/2947565115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d2c10e260 msgr2=0x7f8d2c110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.708+0000 7f8d2b200640 1 --2- 172.21.15.89:0/2947565115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d2c10e260 0x7f8d2c110650 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8d24009ca0 tx=0x7f8d2402a950 comp rx=0 tx=0).stop 2024-09-10T08:44:23.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.708+0000 7f8d2b200640 1 -- 172.21.15.89:0/2947565115 shutdown_connections 2024-09-10T08:44:23.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.708+0000 7f8d2b200640 1 --2- 172.21.15.89:0/2947565115 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d2c10e260 0x7f8d2c110650 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.708+0000 7f8d2b200640 1 --2- 172.21.15.89:0/2947565115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d2c074ee0 0x7f8d2c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.708+0000 7f8d2b200640 1 --2- 172.21.15.89:0/2947565115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d2c0745c0 0x7f8d2c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.708+0000 7f8d2b200640 1 -- 172.21.15.89:0/2947565115 >> 172.21.15.89:0/2947565115 conn(0x7f8d2c0fd740 msgr2=0x7f8d2c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.708+0000 7f8d2b200640 1 -- 172.21.15.89:0/2947565115 shutdown_connections 2024-09-10T08:44:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.708+0000 7f8d2b200640 1 -- 172.21.15.89:0/2947565115 wait complete. 2024-09-10T08:44:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.709+0000 7f8d2b200640 1 Processor -- start 2024-09-10T08:44:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.710+0000 7f8d2b200640 1 -- start start 2024-09-10T08:44:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.710+0000 7f8d2b200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d2c0745c0 0x7f8d2c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.710+0000 7f8d2b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d2c074ee0 0x7f8d2c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.710+0000 7f8d2b200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d2c10e260 0x7f8d2c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.710+0000 7f8d2b200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c113fc0 con 0x7f8d2c10e260 2024-09-10T08:44:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.710+0000 7f8d2b200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c113e40 con 0x7f8d2c0745c0 2024-09-10T08:44:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.710+0000 7f8d2b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c114140 con 0x7f8d2c074ee0 2024-09-10T08:44:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.710+0000 7f8d2a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d2c10e260 0x7f8d2c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.710+0000 7f8d29400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d2c074ee0 0x7f8d2c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.710+0000 7f8d29400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d2c074ee0 0x7f8d2c19a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:35570/0 (socket says 172.21.15.89:35570) 2024-09-10T08:44:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.710+0000 7f8d29400640 1 -- 172.21.15.89:0/212078241 learned_addr learned my addr 172.21.15.89:0/212078241 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.710+0000 7f8d29e00640 1 --2- 172.21.15.89:0/212078241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d2c0745c0 0x7f8d2c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.711+0000 7f8d2a800640 1 -- 172.21.15.89:0/212078241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d2c074ee0 msgr2=0x7f8d2c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.711+0000 7f8d2a800640 1 --2- 172.21.15.89:0/212078241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d2c074ee0 0x7f8d2c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.711+0000 7f8d2a800640 1 -- 172.21.15.89:0/212078241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d2c0745c0 msgr2=0x7f8d2c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.711+0000 7f8d2a800640 1 --2- 172.21.15.89:0/212078241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d2c0745c0 0x7f8d2c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.711+0000 7f8d2a800640 1 -- 172.21.15.89:0/212078241 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2c19f420 con 0x7f8d2c10e260 2024-09-10T08:44:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.711+0000 7f8d2a800640 1 --2- 172.21.15.89:0/212078241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d2c10e260 0x7f8d2c19ed40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8d2402af40 tx=0x7f8d24003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.712+0000 7f8d22a00640 1 -- 172.21.15.89:0/212078241 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24034040 con 0x7f8d2c10e260 2024-09-10T08:44:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.712+0000 7f8d22a00640 1 -- 172.21.15.89:0/212078241 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d2402f940 con 0x7f8d2c10e260 2024-09-10T08:44:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.712+0000 7f8d2b200640 1 -- 172.21.15.89:0/212078241 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2c19f6b0 con 0x7f8d2c10e260 2024-09-10T08:44:23.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.712+0000 7f8d22a00640 1 -- 172.21.15.89:0/212078241 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d240045a0 con 0x7f8d2c10e260 2024-09-10T08:44:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.712+0000 7f8d2b200640 1 -- 172.21.15.89:0/212078241 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d2c1a6ff0 con 0x7f8d2c10e260 2024-09-10T08:44:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.715+0000 7f8d335f8740 1 Processor -- start 2024-09-10T08:44:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.716+0000 7f8d22a00640 1 -- 172.21.15.89:0/212078241 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8d24037020 con 0x7f8d2c10e260 2024-09-10T08:44:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.716+0000 7f8d335f8740 1 -- start start 2024-09-10T08:44:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.716+0000 7f8d335f8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 0x5570b60cf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.716+0000 7f8d335f8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60cf760 0x5570b60d9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.716+0000 7f8d335f8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570b60da020 0x5570b60dc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.716+0000 7f8d335f8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5570b60353e0 con 0x5570b60cee40 2024-09-10T08:44:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.716+0000 7f8d335f8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5570b5fdcd80 con 0x5570b60cf760 2024-09-10T08:44:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.716+0000 7f8d335f8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5570b60313a0 con 0x5570b60da020 2024-09-10T08:44:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.717+0000 7f8d29400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 0x5570b60cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.717+0000 7f8d29400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 0x5570b60cf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36178/0 (socket says 172.21.15.89:36178) 2024-09-10T08:44:23.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.717+0000 7f8d29400640 1 -- 172.21.15.89:0/628404598 learned_addr learned my addr 172.21.15.89:0/628404598 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:23.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.717+0000 7f8d29e00640 1 --2- 172.21.15.89:0/628404598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60cf760 0x5570b60d9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.717+0000 7f8d29400640 1 -- 172.21.15.89:0/628404598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570b60da020 msgr2=0x5570b60dc410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:44:23.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.717+0000 7f8d29400640 1 --2- 172.21.15.89:0/628404598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570b60da020 0x5570b60dc410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.717+0000 7f8d29400640 1 -- 172.21.15.89:0/628404598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60cf760 msgr2=0x5570b60d9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:23.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.717+0000 7f8d29400640 1 --2- 172.21.15.89:0/628404598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60cf760 0x5570b60d9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.717+0000 7f8d29400640 1 -- 172.21.15.89:0/628404598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570b5ffbae0 con 0x5570b60cee40 2024-09-10T08:44:23.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.717+0000 7f8d29e00640 1 --2- 172.21.15.89:0/628404598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60cf760 0x5570b60d9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:23.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.717+0000 7f8d22a00640 1 --2- 172.21.15.89:0/212078241 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8d100a2c00 0x7f8d100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.717+0000 7f8d29400640 1 --2- 172.21.15.89:0/628404598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 0x5570b60cf220 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009c10 tx=0x7f8d180117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21dc70e7977fbf73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:23.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.718+0000 7f8d22a00640 1 -- 172.21.15.89:0/212078241 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f8d240f0150 con 0x7f8d2c10e260 2024-09-10T08:44:23.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.718+0000 7f8d29e00640 1 --2- 172.21.15.89:0/212078241 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8d100a2c00 0x7f8d100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.718+0000 7f8d17000640 1 -- 172.21.15.89:0/628404598 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d18004030 con 0x5570b60cee40 2024-09-10T08:44:23.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.718+0000 7f8d17000640 1 -- 172.21.15.89:0/628404598 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d180041d0 con 0x5570b60cee40 2024-09-10T08:44:23.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.718+0000 7f8d17000640 1 -- 172.21.15.89:0/628404598 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d18012780 con 0x5570b60cee40 2024-09-10T08:44:23.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.719+0000 7f8d335f8740 1 -- 172.21.15.89:0/628404598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 msgr2=0x5570b60cf220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.719+0000 7f8d29e00640 1 --2- 172.21.15.89:0/212078241 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8d100a2c00 0x7f8d100a50c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x5570b60cc230 tx=0x7f8d1c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:23.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.719+0000 7f8d335f8740 1 --2- 172.21.15.89:0/628404598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 0x5570b60cf220 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009c10 tx=0x7f8d180117b0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.719+0000 7f8d335f8740 1 -- 172.21.15.89:0/628404598 shutdown_connections 2024-09-10T08:44:23.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.719+0000 7f8d335f8740 1 --2- 172.21.15.89:0/628404598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570b60da020 0x5570b60dc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.719+0000 7f8d335f8740 1 --2- 172.21.15.89:0/628404598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60cf760 0x5570b60d9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.719+0000 7f8d335f8740 1 --2- 172.21.15.89:0/628404598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 0x5570b60cf220 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.719+0000 7f8d335f8740 1 -- 172.21.15.89:0/628404598 >> 172.21.15.89:0/628404598 conn(0x5570b60cb3d0 msgr2=0x5570b60cb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:23.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.719+0000 7f8d335f8740 1 -- 172.21.15.89:0/628404598 shutdown_connections 2024-09-10T08:44:23.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.719+0000 7f8d335f8740 1 -- 172.21.15.89:0/628404598 wait complete. 2024-09-10T08:44:23.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.719+0000 7f8d335f8740 1 -- start start 2024-09-10T08:44:23.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.720+0000 7f8d335f8740 1 Processor -- start 2024-09-10T08:44:23.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.720+0000 7f8d335f8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 0x5570b60d6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.720+0000 7f8d335f8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570b60cf760 0x5570b60d74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.720+0000 7f8d335f8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60da020 0x5570b60d7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.720+0000 7f8d335f8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5570b60dcbf0 con 0x5570b60cee40 2024-09-10T08:44:23.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.720+0000 7f8d335f8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5570b5ff00f0 con 0x5570b60da020 2024-09-10T08:44:23.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.720+0000 7f8d335f8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5570b60dcd70 con 0x5570b60cf760 2024-09-10T08:44:23.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.720+0000 7f8d335f8740 10 client.0 _ll_register_callbacks cb 0x5570b5fd7bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:23.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.720+0000 7f8d29e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 0x5570b60d6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.720+0000 7f8d29e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 0x5570b60d6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36184/0 (socket says 172.21.15.89:36184) 2024-09-10T08:44:23.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.720+0000 7f8d29400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570b60cf760 0x5570b60d74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.720+0000 7f8d29e00640 1 -- 172.21.15.89:0/981316995 learned_addr learned my addr 172.21.15.89:0/981316995 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:23.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.720+0000 7f8d2a800640 1 --2- 172.21.15.89:0/981316995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60da020 0x5570b60d7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.721+0000 7f8d29400640 1 -- 172.21.15.89:0/981316995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60da020 msgr2=0x5570b60d7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:23.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.721+0000 7f8d29400640 1 --2- 172.21.15.89:0/981316995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60da020 0x5570b60d7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.721+0000 7f8d29400640 1 -- 172.21.15.89:0/981316995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 msgr2=0x5570b60d6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:23.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.721+0000 7f8d29400640 1 --2- 172.21.15.89:0/981316995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 0x5570b60d6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.721+0000 7f8d29400640 1 -- 172.21.15.89:0/981316995 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570b5da63c0 con 0x5570b60cf760 2024-09-10T08:44:23.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.721+0000 7f8d2a800640 1 --2- 172.21.15.89:0/981316995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60da020 0x5570b60d7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:23.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.721+0000 7f8d29e00640 1 --2- 172.21.15.89:0/981316995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 0x5570b60d6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:23.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.721+0000 7f8d29400640 1 --2- 172.21.15.89:0/981316995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570b60cf760 0x5570b60d74f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8d180095f0 tx=0x7f8d18009980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa76611bd9ec223 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:23.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.721+0000 7f8d0b400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:44:23.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.721+0000 7f8d0b400640 1 -- 172.21.15.89:0/981316995 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d18029060 con 0x5570b60cf760 2024-09-10T08:44:23.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.721+0000 7f8d335f8740 1 -- 172.21.15.89:0/981316995 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d180041d0 con 0x5570b60cf760 2024-09-10T08:44:23.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.721+0000 7f8d0b400640 1 -- 172.21.15.89:0/981316995 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f8d180035f0 con 0x5570b60cf760 2024-09-10T08:44:23.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.722+0000 7f8d0b400640 1 -- 172.21.15.89:0/981316995 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8d18030050 con 0x5570b60cf760 2024-09-10T08:44:23.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.722+0000 7f8d335f8740 10 client.26122 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:23.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.722+0000 7f8d335f8740 1 -- 172.21.15.89:0/981316995 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570b60e8cd0 con 0x5570b60cf760 2024-09-10T08:44:23.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.722+0000 7f8d0b400640 1 -- 172.21.15.89:0/981316995 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d180041d0 con 0x5570b60cf760 2024-09-10T08:44:23.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.722+0000 7f8d0b400640 1 -- 172.21.15.89:0/981316995 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8d18004750 con 0x5570b60cf760 2024-09-10T08:44:23.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.723+0000 7f8d335f8740 10 client.26122 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:23.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.723+0000 7f8d335f8740 10 client.26122 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:44:23.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.723+0000 7f8d335f8740 20 client.26122 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:23.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.723+0000 7f8d335f8740 1 --2- 172.21.15.89:0/981316995 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x5570b60ea0e0 0x5570b60ec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.723+0000 7f8d335f8740 4 client.26122 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:44:23.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.723+0000 7f8d335f8740 1 -- 172.21.15.89:0/981316995 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570b6042950 con 0x5570b60ea0e0 2024-09-10T08:44:23.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.723+0000 7f8d29e00640 1 --2- 172.21.15.89:0/981316995 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x5570b60ea0e0 0x5570b60ec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.724+0000 7f8d29e00640 1 --2- 172.21.15.89:0/981316995 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x5570b60ea0e0 0x5570b60ec4d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=57133c988392ed7f server_cookie=5218b3865f031330 in_seq=0 out_seq=0 2024-09-10T08:44:23.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.724+0000 7f8d0b400640 10 client.26122 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:44:23.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.724+0000 7f8d0b400640 1 -- 172.21.15.89:0/981316995 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570b6042950 con 0x5570b60ea0e0 2024-09-10T08:44:23.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.724+0000 7f8d0b400640 10 client.26122 handle_command_reply: tid=0 2024-09-10T08:44:23.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.724+0000 7f8d335f8740 1 client.26122 shutdown 2024-09-10T08:44:23.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.725+0000 7f8d335f8740 1 -- 172.21.15.89:0/981316995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570b60cf760 msgr2=0x5570b60d74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.725+0000 7f8d335f8740 1 --2- 172.21.15.89:0/981316995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570b60cf760 0x5570b60d74f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8d180095f0 tx=0x7f8d18009980 comp rx=0 tx=0).stop 2024-09-10T08:44:23.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.725+0000 7f8d335f8740 1 -- 172.21.15.89:0/981316995 shutdown_connections 2024-09-10T08:44:23.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.725+0000 7f8d335f8740 1 --2- 172.21.15.89:0/981316995 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x5570b60ea0e0 0x5570b60ec4d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.725+0000 7f8d335f8740 1 --2- 172.21.15.89:0/981316995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60da020 0x5570b60d7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.725+0000 7f8d335f8740 1 --2- 172.21.15.89:0/981316995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570b60cf760 0x5570b60d74f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.725+0000 7f8d335f8740 1 --2- 172.21.15.89:0/981316995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 0x5570b60d6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.725+0000 7f8d335f8740 1 -- 172.21.15.89:0/981316995 >> 172.21.15.89:0/981316995 conn(0x5570b60cb3d0 msgr2=0x5570b60d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:23.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.725+0000 7f8d0b400640 0 client.26122 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:44:23.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.725+0000 7f8d335f8740 1 -- 172.21.15.89:0/981316995 shutdown_connections 2024-09-10T08:44:23.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.725+0000 7f8d335f8740 1 -- 172.21.15.89:0/981316995 wait complete. 2024-09-10T08:44:23.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.725+0000 7f8d335f8740 20 client.26122 trim_cache size 0 max 16384 2024-09-10T08:44:23.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.738+0000 7f8d335f8740 1 Processor -- start 2024-09-10T08:44:23.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.739+0000 7f8d335f8740 1 -- start start 2024-09-10T08:44:23.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.739+0000 7f8d335f8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 0x5570b60d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.739+0000 7f8d335f8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570b60cf760 0x5570b60d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.739+0000 7f8d335f8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60da020 0x5570b60d4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.739+0000 7f8d335f8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c1b3640 con 0x5570b60cee40 2024-09-10T08:44:23.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.739+0000 7f8d335f8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c1b0f10 con 0x5570b60da020 2024-09-10T08:44:23.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.739+0000 7f8d335f8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c1aed60 con 0x5570b60cf760 2024-09-10T08:44:23.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.739+0000 7f8d29e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 0x5570b60d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.739+0000 7f8d29e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 0x5570b60d18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36194/0 (socket says 172.21.15.89:36194) 2024-09-10T08:44:23.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.739+0000 7f8d29e00640 1 -- 172.21.15.89:0/3064097295 learned_addr learned my addr 172.21.15.89:0/3064097295 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:23.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.739+0000 7f8d29400640 1 --2- 172.21.15.89:0/3064097295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570b60cf760 0x5570b60d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.739+0000 7f8d29e00640 1 -- 172.21.15.89:0/3064097295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570b60cf760 msgr2=0x5570b60d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:23.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.740+0000 7f8d29e00640 1 --2- 172.21.15.89:0/3064097295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570b60cf760 0x5570b60d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.740+0000 7f8d29e00640 1 -- 172.21.15.89:0/3064097295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60da020 msgr2=0x5570b60d4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:23.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.739+0000 7f8d2a800640 1 --2- 172.21.15.89:0/3064097295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60da020 0x5570b60d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.740+0000 7f8d29e00640 1 --2- 172.21.15.89:0/3064097295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60da020 0x5570b60d4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.740+0000 7f8d29e00640 1 -- 172.21.15.89:0/3064097295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570b60e6990 con 0x5570b60cee40 2024-09-10T08:44:23.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.740+0000 7f8d2a800640 1 --2- 172.21.15.89:0/3064097295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60da020 0x5570b60d4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:44:23.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.740+0000 7f8d29e00640 1 --2- 172.21.15.89:0/3064097295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 0x5570b60d18a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c012cb0 tx=0x7f8d1c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52c8b136a0e57b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:23.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.740+0000 7f8d16600640 1 -- 172.21.15.89:0/3064097295 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c005020 con 0x5570b60cee40 2024-09-10T08:44:23.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.740+0000 7f8d16600640 1 -- 172.21.15.89:0/3064097295 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d1c0047a0 con 0x5570b60cee40 2024-09-10T08:44:23.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.740+0000 7f8d16600640 1 -- 172.21.15.89:0/3064097295 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c02b900 con 0x5570b60cee40 2024-09-10T08:44:23.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.741+0000 7f8d335f8740 1 -- 172.21.15.89:0/3064097295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 msgr2=0x5570b60d18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.741+0000 7f8d335f8740 1 --2- 172.21.15.89:0/3064097295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 0x5570b60d18a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c012cb0 tx=0x7f8d1c01b000 comp rx=0 tx=0).stop 2024-09-10T08:44:23.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.741+0000 7f8d335f8740 1 -- 172.21.15.89:0/3064097295 shutdown_connections 2024-09-10T08:44:23.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.741+0000 7f8d335f8740 1 --2- 172.21.15.89:0/3064097295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60da020 0x5570b60d4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.741+0000 7f8d335f8740 1 --2- 172.21.15.89:0/3064097295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570b60cf760 0x5570b60d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.741+0000 7f8d335f8740 1 --2- 172.21.15.89:0/3064097295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60cee40 0x5570b60d18a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.741+0000 7f8d335f8740 1 -- 172.21.15.89:0/3064097295 >> 172.21.15.89:0/3064097295 conn(0x5570b60cb3d0 msgr2=0x5570b60cb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:23.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.741+0000 7f8d335f8740 1 -- 172.21.15.89:0/3064097295 shutdown_connections 2024-09-10T08:44:23.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.741+0000 7f8d335f8740 1 -- 172.21.15.89:0/3064097295 wait complete. 2024-09-10T08:44:23.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.742+0000 7f8d335f8740 1 -- start start 2024-09-10T08:44:23.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.742+0000 7f8d335f8740 1 Processor -- start 2024-09-10T08:44:23.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.742+0000 7f8d335f8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570b60cee40 0x5570b60ec210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.742+0000 7f8d335f8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60cf760 0x5570b60ec750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.742+0000 7f8d335f8740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60da020 0x5570b6152a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.742+0000 7f8d335f8740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c1af3d0 con 0x5570b60da020 2024-09-10T08:44:23.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.742+0000 7f8d335f8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c1b0840 con 0x5570b60cf760 2024-09-10T08:44:23.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.742+0000 7f8d335f8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c1afaa0 con 0x5570b60cee40 2024-09-10T08:44:23.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.742+0000 7f8d335f8740 10 client.0 _ll_register_callbacks cb 0x7f8d1c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:23.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.743+0000 7f8d29400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60cf760 0x5570b60ec750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.743+0000 7f8d29400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60cf760 0x5570b60ec750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:59086/0 (socket says 172.21.15.89:59086) 2024-09-10T08:44:23.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.743+0000 7f8d29400640 1 -- 172.21.15.89:0/1151410984 learned_addr learned my addr 172.21.15.89:0/1151410984 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:23.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.743+0000 7f8d29e00640 1 --2- 172.21.15.89:0/1151410984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570b60cee40 0x5570b60ec210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.743+0000 7f8d2a800640 1 --2- 172.21.15.89:0/1151410984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60da020 0x5570b6152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.743+0000 7f8d29400640 1 -- 172.21.15.89:0/1151410984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570b60cee40 msgr2=0x5570b60ec210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:23.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.743+0000 7f8d29400640 1 --2- 172.21.15.89:0/1151410984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570b60cee40 0x5570b60ec210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.743+0000 7f8d29400640 1 -- 172.21.15.89:0/1151410984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60da020 msgr2=0x5570b6152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:23.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.743+0000 7f8d29400640 1 --2- 172.21.15.89:0/1151410984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60da020 0x5570b6152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.743+0000 7f8d29400640 1 -- 172.21.15.89:0/1151410984 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570b60ed030 con 0x5570b60cf760 2024-09-10T08:44:23.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.743+0000 7f8d2a800640 1 --2- 172.21.15.89:0/1151410984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60da020 0x5570b6152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:44:23.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.743+0000 7f8d29400640 1 --2- 172.21.15.89:0/1151410984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60cf760 0x5570b60ec750 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8d1800a010 tx=0x7f8d180047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bdfbb014b5f9a3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:23.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.743+0000 7f8d0b400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:44:23.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.744+0000 7f8d0b400640 1 -- 172.21.15.89:0/1151410984 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d18039060 con 0x5570b60cf760 2024-09-10T08:44:23.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.744+0000 7f8d335f8740 1 -- 172.21.15.89:0/1151410984 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d1c0047a0 con 0x5570b60cf760 2024-09-10T08:44:23.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.744+0000 7f8d0b400640 1 -- 172.21.15.89:0/1151410984 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f8d18004050 con 0x5570b60cf760 2024-09-10T08:44:23.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.744+0000 7f8d0b400640 1 -- 172.21.15.89:0/1151410984 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8d18040050 con 0x5570b60cf760 2024-09-10T08:44:23.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.744+0000 7f8d335f8740 10 client.26210 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:23.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.744+0000 7f8d335f8740 1 -- 172.21.15.89:0/1151410984 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570b6153fd0 con 0x5570b60cf760 2024-09-10T08:44:23.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.745+0000 7f8d0b400640 1 -- 172.21.15.89:0/1151410984 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d1c0047a0 con 0x5570b60cf760 2024-09-10T08:44:23.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.745+0000 7f8d0b400640 1 -- 172.21.15.89:0/1151410984 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8d18004e60 con 0x5570b60cf760 2024-09-10T08:44:23.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.745+0000 7f8d335f8740 10 client.26210 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:23.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.745+0000 7f8d335f8740 10 client.26210 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:44:23.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.745+0000 7f8d335f8740 20 client.26210 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:23.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.745+0000 7f8d335f8740 1 --2- 172.21.15.89:0/1151410984 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x5570b6154720 0x5570b6156b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.745+0000 7f8d335f8740 4 client.26210 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:44:23.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.745+0000 7f8d335f8740 1 -- 172.21.15.89:0/1151410984 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570b603ce60 con 0x5570b6154720 2024-09-10T08:44:23.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.745+0000 7f8d29e00640 1 --2- 172.21.15.89:0/1151410984 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x5570b6154720 0x5570b6156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.746+0000 7f8d29e00640 1 --2- 172.21.15.89:0/1151410984 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x5570b6154720 0x5570b6156b40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a7c439d9e86fa6 server_cookie=342bb986f4946ed6 in_seq=0 out_seq=0 2024-09-10T08:44:23.756 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.746+0000 7f8d0b400640 10 client.26210 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:44:23.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.746+0000 7f8d0b400640 1 -- 172.21.15.89:0/1151410984 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570b603ce60 con 0x5570b6154720 2024-09-10T08:44:23.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.746+0000 7f8d0b400640 10 client.26210 handle_command_reply: tid=0 2024-09-10T08:44:23.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.746+0000 7f8d335f8740 1 client.26210 shutdown 2024-09-10T08:44:23.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.746+0000 7f8d335f8740 1 -- 172.21.15.89:0/1151410984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60cf760 msgr2=0x5570b60ec750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.746+0000 7f8d335f8740 1 --2- 172.21.15.89:0/1151410984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60cf760 0x5570b60ec750 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8d1800a010 tx=0x7f8d180047c0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.746+0000 7f8d335f8740 1 -- 172.21.15.89:0/1151410984 shutdown_connections 2024-09-10T08:44:23.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.746+0000 7f8d335f8740 1 --2- 172.21.15.89:0/1151410984 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x5570b6154720 0x5570b6156b40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.746+0000 7f8d335f8740 1 --2- 172.21.15.89:0/1151410984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5570b60da020 0x5570b6152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.746+0000 7f8d335f8740 1 --2- 172.21.15.89:0/1151410984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b60cf760 0x5570b60ec750 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.746+0000 7f8d335f8740 1 --2- 172.21.15.89:0/1151410984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570b60cee40 0x5570b60ec210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.746+0000 7f8d335f8740 1 -- 172.21.15.89:0/1151410984 >> 172.21.15.89:0/1151410984 conn(0x5570b60cb3d0 msgr2=0x5570b60d6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:23.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.746+0000 7f8d0b400640 0 client.26210 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:44:23.757 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.746+0000 7f8d335f8740 1 -- 172.21.15.89:0/1151410984 shutdown_connections 2024-09-10T08:44:23.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.747+0000 7f8d335f8740 1 -- 172.21.15.89:0/1151410984 wait complete. 2024-09-10T08:44:23.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.747+0000 7f8d335f8740 20 client.26210 trim_cache size 0 max 16384 2024-09-10T08:44:23.758 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:44:23.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.747+0000 7f8d2b200640 1 -- 172.21.15.89:0/212078241 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8d100a2c00 msgr2=0x7f8d100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.747+0000 7f8d2b200640 1 --2- 172.21.15.89:0/212078241 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8d100a2c00 0x7f8d100a50c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x5570b60cc230 tx=0x7f8d1c005f90 comp rx=0 tx=0).stop 2024-09-10T08:44:23.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.747+0000 7f8d2b200640 1 -- 172.21.15.89:0/212078241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d2c10e260 msgr2=0x7f8d2c19ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.747+0000 7f8d2b200640 1 --2- 172.21.15.89:0/212078241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d2c10e260 0x7f8d2c19ed40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8d2402af40 tx=0x7f8d24003b80 comp rx=0 tx=0).stop 2024-09-10T08:44:23.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.747+0000 7f8d2b200640 1 -- 172.21.15.89:0/212078241 shutdown_connections 2024-09-10T08:44:23.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.747+0000 7f8d2b200640 1 --2- 172.21.15.89:0/212078241 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f8d100a2c00 0x7f8d100a50c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.747+0000 7f8d2b200640 1 --2- 172.21.15.89:0/212078241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8d2c10e260 0x7f8d2c19ed40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.747+0000 7f8d2b200640 1 --2- 172.21.15.89:0/212078241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d2c074ee0 0x7f8d2c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.747+0000 7f8d2b200640 1 --2- 172.21.15.89:0/212078241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d2c0745c0 0x7f8d2c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.747+0000 7f8d2b200640 1 -- 172.21.15.89:0/212078241 >> 172.21.15.89:0/212078241 conn(0x7f8d2c0fd740 msgr2=0x7f8d2c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:23.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.748+0000 7f8d2b200640 1 -- 172.21.15.89:0/212078241 shutdown_connections 2024-09-10T08:44:23.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:23.748+0000 7f8d2b200640 1 -- 172.21.15.89:0/212078241 wait complete. 2024-09-10T08:44:24.848 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:44:24.987 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:24 smithi089 ceph-mon[30757]: pgmap v527: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-09-10T08:44:25.219 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:25.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:24 smithi149 ceph-mon[36167]: pgmap v527: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-09-10T08:44:25.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:24 smithi111 ceph-mon[35220]: pgmap v527: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-09-10T08:44:26.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.002+0000 7f7d92800640 1 -- 172.21.15.89:0/184790336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d8c074eb0 msgr2=0x7f7d8c1124a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:26.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.002+0000 7f7d92800640 1 --2- 172.21.15.89:0/184790336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d8c074eb0 0x7f7d8c1124a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009900 tx=0x7f7d7802a4b0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.003+0000 7f7d92800640 1 -- 172.21.15.89:0/184790336 shutdown_connections 2024-09-10T08:44:26.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.003+0000 7f7d92800640 1 --2- 172.21.15.89:0/184790336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d8c1129e0 0x7f7d8c114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.003+0000 7f7d92800640 1 --2- 172.21.15.89:0/184790336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d8c074eb0 0x7f7d8c1124a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.003+0000 7f7d92800640 1 --2- 172.21.15.89:0/184790336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d8c074590 0x7f7d8c074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.003+0000 7f7d92800640 1 -- 172.21.15.89:0/184790336 >> 172.21.15.89:0/184790336 conn(0x7f7d8c0fd710 msgr2=0x7f7d8c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:26.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.003+0000 7f7d92800640 1 -- 172.21.15.89:0/184790336 shutdown_connections 2024-09-10T08:44:26.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.003+0000 7f7d92800640 1 -- 172.21.15.89:0/184790336 wait complete. 2024-09-10T08:44:26.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.004+0000 7f7d92800640 1 Processor -- start 2024-09-10T08:44:26.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.004+0000 7f7d92800640 1 -- start start 2024-09-10T08:44:26.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.005+0000 7f7d92800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d8c074590 0x7f7d8c1a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:26.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.005+0000 7f7d92800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d8c074eb0 0x7f7d8c1a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:26.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.005+0000 7f7d92800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d8c1129e0 0x7f7d8c19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:26.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.005+0000 7f7d92800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c118420 con 0x7f7d8c1129e0 2024-09-10T08:44:26.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.005+0000 7f7d92800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c1182a0 con 0x7f7d8c074eb0 2024-09-10T08:44:26.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.005+0000 7f7d92800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c1185a0 con 0x7f7d8c074590 2024-09-10T08:44:26.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.005+0000 7f7d91400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d8c074590 0x7f7d8c1a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:26.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.005+0000 7f7d90a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d8c074eb0 0x7f7d8c1a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:26.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.005+0000 7f7d91400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d8c074590 0x7f7d8c1a4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:35628/0 (socket says 172.21.15.89:35628) 2024-09-10T08:44:26.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.005+0000 7f7d91400640 1 -- 172.21.15.89:0/1848119715 learned_addr learned my addr 172.21.15.89:0/1848119715 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:26.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.005+0000 7f7d91e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d8c1129e0 0x7f7d8c19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:26.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.006+0000 7f7d91400640 1 -- 172.21.15.89:0/1848119715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d8c074eb0 msgr2=0x7f7d8c1a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:26.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.006+0000 7f7d91400640 1 --2- 172.21.15.89:0/1848119715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d8c074eb0 0x7f7d8c1a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.006+0000 7f7d91400640 1 -- 172.21.15.89:0/1848119715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d8c1129e0 msgr2=0x7f7d8c19e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:26.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.006+0000 7f7d91400640 1 --2- 172.21.15.89:0/1848119715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d8c1129e0 0x7f7d8c19e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.006+0000 7f7d91400640 1 -- 172.21.15.89:0/1848119715 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8c19ef50 con 0x7f7d8c074590 2024-09-10T08:44:26.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.006+0000 7f7d91400640 1 --2- 172.21.15.89:0/1848119715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d8c074590 0x7f7d8c1a4630 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7d8000e9f0 tx=0x7f7d8000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:26.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.006+0000 7f7d8a000640 1 -- 172.21.15.89:0/1848119715 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8000cd80 con 0x7f7d8c074590 2024-09-10T08:44:26.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.006+0000 7f7d92800640 1 -- 172.21.15.89:0/1848119715 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8c19f240 con 0x7f7d8c074590 2024-09-10T08:44:26.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.006+0000 7f7d8a000640 1 -- 172.21.15.89:0/1848119715 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d80010910 con 0x7f7d8c074590 2024-09-10T08:44:26.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.006+0000 7f7d8a000640 1 -- 172.21.15.89:0/1848119715 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d800188f0 con 0x7f7d8c074590 2024-09-10T08:44:26.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.006+0000 7f7d92800640 1 -- 172.21.15.89:0/1848119715 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d8c1ab490 con 0x7f7d8c074590 2024-09-10T08:44:26.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.011+0000 7f7d967ee740 1 Processor -- start 2024-09-10T08:44:26.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.011+0000 7f7d967ee740 1 -- start start 2024-09-10T08:44:26.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.011+0000 7f7d8a000640 1 -- 172.21.15.89:0/1848119715 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7d80010ac0 con 0x7f7d8c074590 2024-09-10T08:44:26.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.011+0000 7f7d967ee740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6cdce40 0x55f7b6cdd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:26.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.011+0000 7f7d967ee740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7b6cdd760 0x55f7b6ce7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:26.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.011+0000 7f7d967ee740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6ce8020 0x55f7b6cea410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:26.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.011+0000 7f7d967ee740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f7b6c433e0 con 0x55f7b6ce8020 2024-09-10T08:44:26.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.011+0000 7f7d967ee740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f7b6bead80 con 0x55f7b6cdce40 2024-09-10T08:44:26.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.011+0000 7f7d967ee740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f7b6c3f3a0 con 0x55f7b6cdd760 2024-09-10T08:44:26.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.012+0000 7f7d90a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6ce8020 0x55f7b6cea410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:26.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.012+0000 7f7d91e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6cdce40 0x55f7b6cdd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:26.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.012+0000 7f7d91e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6cdce40 0x55f7b6cdd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:59108/0 (socket says 172.21.15.89:59108) 2024-09-10T08:44:26.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.012+0000 7f7d91400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7b6cdd760 0x55f7b6ce7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:26.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.012+0000 7f7d91e00640 1 -- 172.21.15.89:0/4103412195 learned_addr learned my addr 172.21.15.89:0/4103412195 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:26.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.012+0000 7f7d90a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6ce8020 0x55f7b6cea410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36224/0 (socket says 172.21.15.89:36224) 2024-09-10T08:44:26.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.012+0000 7f7d91e00640 1 -- 172.21.15.89:0/4103412195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7b6cdd760 msgr2=0x55f7b6ce7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:26.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.012+0000 7f7d91e00640 1 --2- 172.21.15.89:0/4103412195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7b6cdd760 0x55f7b6ce7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.012+0000 7f7d91e00640 1 -- 172.21.15.89:0/4103412195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6ce8020 msgr2=0x55f7b6cea410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:26.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.012+0000 7f7d91e00640 1 --2- 172.21.15.89:0/4103412195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6ce8020 0x55f7b6cea410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.012+0000 7f7d91e00640 1 -- 172.21.15.89:0/4103412195 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7b6c09ae0 con 0x55f7b6cdce40 2024-09-10T08:44:26.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.013+0000 7f7d91e00640 1 --2- 172.21.15.89:0/4103412195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6cdce40 0x55f7b6cdd220 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400a4b0 tx=0x7f7d84009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edd93351e60f91d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:26.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.013+0000 7f7d7e600640 1 -- 172.21.15.89:0/4103412195 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84013790 con 0x55f7b6cdce40 2024-09-10T08:44:26.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.013+0000 7f7d7e600640 1 -- 172.21.15.89:0/4103412195 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d84019070 con 0x55f7b6cdce40 2024-09-10T08:44:26.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.013+0000 7f7d7e600640 1 -- 172.21.15.89:0/4103412195 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84014dd0 con 0x55f7b6cdce40 2024-09-10T08:44:26.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.013+0000 7f7d8a000640 1 --2- 172.21.15.89:0/1848119715 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7d740a2b80 0x7f7d740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:26.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.013+0000 7f7d8a000640 1 -- 172.21.15.89:0/1848119715 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f7d800cf7a0 con 0x7f7d8c074590 2024-09-10T08:44:26.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.013+0000 7f7d91400640 1 --2- 172.21.15.89:0/1848119715 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7d740a2b80 0x7f7d740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:26.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.014+0000 7f7d967ee740 1 -- 172.21.15.89:0/4103412195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6cdce40 msgr2=0x55f7b6cdd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:26.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.014+0000 7f7d967ee740 1 --2- 172.21.15.89:0/4103412195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6cdce40 0x55f7b6cdd220 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400a4b0 tx=0x7f7d84009650 comp rx=0 tx=0).stop 2024-09-10T08:44:26.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.014+0000 7f7d967ee740 1 -- 172.21.15.89:0/4103412195 shutdown_connections 2024-09-10T08:44:26.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.014+0000 7f7d967ee740 1 --2- 172.21.15.89:0/4103412195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6ce8020 0x55f7b6cea410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.014+0000 7f7d967ee740 1 --2- 172.21.15.89:0/4103412195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7b6cdd760 0x55f7b6ce7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.014+0000 7f7d967ee740 1 --2- 172.21.15.89:0/4103412195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6cdce40 0x55f7b6cdd220 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.014+0000 7f7d967ee740 1 -- 172.21.15.89:0/4103412195 >> 172.21.15.89:0/4103412195 conn(0x55f7b6cd93d0 msgr2=0x55f7b6cd97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:26.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.014+0000 7f7d91400640 1 --2- 172.21.15.89:0/1848119715 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7d740a2b80 0x7f7d740a5040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7d800cd240 tx=0x7f7d800d5550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:26.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.014+0000 7f7d967ee740 1 -- 172.21.15.89:0/4103412195 shutdown_connections 2024-09-10T08:44:26.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.014+0000 7f7d967ee740 1 -- 172.21.15.89:0/4103412195 wait complete. 2024-09-10T08:44:26.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.014+0000 7f7d967ee740 1 -- start start 2024-09-10T08:44:26.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.015+0000 7f7d967ee740 1 Processor -- start 2024-09-10T08:44:26.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.015+0000 7f7d967ee740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7b6cdce40 0x55f7b6ce4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:26.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.015+0000 7f7d967ee740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6cdd760 0x55f7b6ce5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:26.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.015+0000 7f7d967ee740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6ce8020 0x55f7b6ce5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:26.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.015+0000 7f7d967ee740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f7b6ceabf0 con 0x55f7b6ce8020 2024-09-10T08:44:26.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.015+0000 7f7d967ee740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f7b6bfe0f0 con 0x55f7b6cdd760 2024-09-10T08:44:26.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.015+0000 7f7d967ee740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f7b6cead70 con 0x55f7b6cdce40 2024-09-10T08:44:26.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.015+0000 7f7d967ee740 10 client.0 _ll_register_callbacks cb 0x55f7b6be5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:26.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.015+0000 7f7d91e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6ce8020 0x55f7b6ce5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:26.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.016+0000 7f7d91e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6ce8020 0x55f7b6ce5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36226/0 (socket says 172.21.15.89:36226) 2024-09-10T08:44:26.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.016+0000 7f7d91e00640 1 -- 172.21.15.89:0/2937593748 learned_addr learned my addr 172.21.15.89:0/2937593748 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:26.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.016+0000 7f7d90a00640 1 --2- 172.21.15.89:0/2937593748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6cdd760 0x55f7b6ce5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:26.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.016+0000 7f7d91e00640 1 -- 172.21.15.89:0/2937593748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7b6cdce40 msgr2=0x55f7b6ce4ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:44:26.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.016+0000 7f7d91e00640 1 --2- 172.21.15.89:0/2937593748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7b6cdce40 0x55f7b6ce4ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.016+0000 7f7d91e00640 1 -- 172.21.15.89:0/2937593748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6cdd760 msgr2=0x55f7b6ce5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:26.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.016+0000 7f7d91e00640 1 --2- 172.21.15.89:0/2937593748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6cdd760 0x55f7b6ce5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.016+0000 7f7d91e00640 1 -- 172.21.15.89:0/2937593748 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7b69b43c0 con 0x55f7b6ce8020 2024-09-10T08:44:26.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.016+0000 7f7d91e00640 1 --2- 172.21.15.89:0/2937593748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6ce8020 0x55f7b6ce5960 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400ae00 tx=0x7f7d84004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8189df4aff203d19 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:26.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.016+0000 7f7d72a00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:44:26.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.017+0000 7f7d72a00640 1 -- 172.21.15.89:0/2937593748 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d840139a0 con 0x55f7b6ce8020 2024-09-10T08:44:26.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.017+0000 7f7d967ee740 1 -- 172.21.15.89:0/2937593748 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d84019070 con 0x55f7b6ce8020 2024-09-10T08:44:26.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.017+0000 7f7d72a00640 1 -- 172.21.15.89:0/2937593748 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f7d84013b40 con 0x55f7b6ce8020 2024-09-10T08:44:26.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.017+0000 7f7d72a00640 1 -- 172.21.15.89:0/2937593748 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7d84030050 con 0x55f7b6ce8020 2024-09-10T08:44:26.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.017+0000 7f7d967ee740 10 client.16155 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:26.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.017+0000 7f7d967ee740 1 -- 172.21.15.89:0/2937593748 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f7b6cf6c00 con 0x55f7b6ce8020 2024-09-10T08:44:26.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.017+0000 7f7d72a00640 1 -- 172.21.15.89:0/2937593748 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d84019070 con 0x55f7b6ce8020 2024-09-10T08:44:26.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.017+0000 7f7d72a00640 1 -- 172.21.15.89:0/2937593748 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7d8401fbc0 con 0x55f7b6ce8020 2024-09-10T08:44:26.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.018+0000 7f7d967ee740 10 client.16155 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:26.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.018+0000 7f7d967ee740 10 client.16155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:44:26.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.018+0000 7f7d967ee740 20 client.16155 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:26.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.018+0000 7f7d967ee740 1 --2- 172.21.15.89:0/2937593748 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f7b6cf7fb0 0x55f7b6cfa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:26.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.018+0000 7f7d967ee740 4 client.16155 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:44:26.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.018+0000 7f7d967ee740 1 -- 172.21.15.89:0/2937593748 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7b6c50950 con 0x55f7b6cf7fb0 2024-09-10T08:44:26.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.034+0000 7f7d90a00640 1 --2- 172.21.15.89:0/2937593748 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f7b6cf7fb0 0x55f7b6cfa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:26.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.093+0000 7f7d90a00640 1 --2- 172.21.15.89:0/2937593748 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f7b6cf7fb0 0x55f7b6cfa3a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=738e203fa1978fa1 server_cookie=9bfd3926ab73e5d3 in_seq=0 out_seq=0 2024-09-10T08:44:26.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.093+0000 7f7d72a00640 10 client.16155 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:26.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.117+0000 7f7d72a00640 1 -- 172.21.15.89:0/2937593748 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7b6c50950 con 0x55f7b6cf7fb0 2024-09-10T08:44:26.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.117+0000 7f7d72a00640 10 client.16155 handle_command_reply: tid=0 2024-09-10T08:44:26.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.117+0000 7f7d967ee740 1 client.16155 shutdown 2024-09-10T08:44:26.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.117+0000 7f7d967ee740 1 -- 172.21.15.89:0/2937593748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6ce8020 msgr2=0x55f7b6ce5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:26.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.117+0000 7f7d967ee740 1 --2- 172.21.15.89:0/2937593748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6ce8020 0x55f7b6ce5960 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400ae00 tx=0x7f7d84004ad0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.117+0000 7f7d967ee740 1 -- 172.21.15.89:0/2937593748 shutdown_connections 2024-09-10T08:44:26.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.117+0000 7f7d967ee740 1 --2- 172.21.15.89:0/2937593748 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f7b6cf7fb0 0x55f7b6cfa3a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.117+0000 7f7d967ee740 1 --2- 172.21.15.89:0/2937593748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6ce8020 0x55f7b6ce5960 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.118+0000 7f7d967ee740 1 --2- 172.21.15.89:0/2937593748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6cdd760 0x55f7b6ce5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.118+0000 7f7d967ee740 1 --2- 172.21.15.89:0/2937593748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7b6cdce40 0x55f7b6ce4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.118+0000 7f7d967ee740 1 -- 172.21.15.89:0/2937593748 >> 172.21.15.89:0/2937593748 conn(0x55f7b6cd93d0 msgr2=0x55f7b6cdf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:26.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.118+0000 7f7d72a00640 0 client.16155 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:26.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.118+0000 7f7d967ee740 1 -- 172.21.15.89:0/2937593748 shutdown_connections 2024-09-10T08:44:26.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.118+0000 7f7d967ee740 1 -- 172.21.15.89:0/2937593748 wait complete. 2024-09-10T08:44:26.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.118+0000 7f7d967ee740 20 client.16155 trim_cache size 0 max 16384 2024-09-10T08:44:26.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.131+0000 7f7d967ee740 1 Processor -- start 2024-09-10T08:44:26.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.132+0000 7f7d967ee740 1 -- start start 2024-09-10T08:44:26.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.132+0000 7f7d967ee740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6cdce40 0x55f7b6ce47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:26.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.132+0000 7f7d967ee740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7b6cdd760 0x55f7b6cff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:26.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.132+0000 7f7d967ee740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6ce8020 0x55f7b6cffee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:26.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.132+0000 7f7d967ee740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c1b7b30 con 0x55f7b6cdce40 2024-09-10T08:44:26.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.132+0000 7f7d967ee740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c1b5400 con 0x55f7b6ce8020 2024-09-10T08:44:26.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.132+0000 7f7d967ee740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c1b3250 con 0x55f7b6cdd760 2024-09-10T08:44:26.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.132+0000 7f7d91e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6cdce40 0x55f7b6ce47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:26.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.132+0000 7f7d91e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6cdce40 0x55f7b6ce47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36230/0 (socket says 172.21.15.89:36230) 2024-09-10T08:44:26.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.132+0000 7f7d91e00640 1 -- 172.21.15.89:0/218131550 learned_addr learned my addr 172.21.15.89:0/218131550 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:26.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.132+0000 7f7d90a00640 1 --2- 172.21.15.89:0/218131550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7b6cdd760 0x55f7b6cff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:26.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.133+0000 7f7d91e00640 1 --2- 172.21.15.89:0/218131550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6ce8020 0x55f7b6cffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:26.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.133+0000 7f7d91e00640 1 -- 172.21.15.89:0/218131550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7b6cdd760 msgr2=0x55f7b6cff9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:26.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.133+0000 7f7d91e00640 1 --2- 172.21.15.89:0/218131550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7b6cdd760 0x55f7b6cff9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.133+0000 7f7d91e00640 1 -- 172.21.15.89:0/218131550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6ce8020 msgr2=0x55f7b6cffee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:26.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.133+0000 7f7d91e00640 1 --2- 172.21.15.89:0/218131550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6ce8020 0x55f7b6cffee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.133+0000 7f7d91e00640 1 -- 172.21.15.89:0/218131550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7b6cdfcb0 con 0x55f7b6cdce40 2024-09-10T08:44:26.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.133+0000 7f7d90a00640 1 --2- 172.21.15.89:0/218131550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7b6cdd760 0x55f7b6cff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:26.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.133+0000 7f7d91e00640 1 --2- 172.21.15.89:0/218131550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6cdce40 0x55f7b6ce47e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7d8402a680 tx=0x7f7d84007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d38f33d3717398 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:26.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.133+0000 7f7d7dc00640 1 -- 172.21.15.89:0/218131550 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d840149d0 con 0x55f7b6cdce40 2024-09-10T08:44:26.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.133+0000 7f7d7dc00640 1 -- 172.21.15.89:0/218131550 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d84019050 con 0x55f7b6cdce40 2024-09-10T08:44:26.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.133+0000 7f7d7dc00640 1 -- 172.21.15.89:0/218131550 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84003c70 con 0x55f7b6cdce40 2024-09-10T08:44:26.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.134+0000 7f7d967ee740 1 -- 172.21.15.89:0/218131550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6cdce40 msgr2=0x55f7b6ce47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:26.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.134+0000 7f7d967ee740 1 --2- 172.21.15.89:0/218131550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6cdce40 0x55f7b6ce47e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7d8402a680 tx=0x7f7d84007040 comp rx=0 tx=0).stop 2024-09-10T08:44:26.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.134+0000 7f7d967ee740 1 -- 172.21.15.89:0/218131550 shutdown_connections 2024-09-10T08:44:26.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.134+0000 7f7d967ee740 1 --2- 172.21.15.89:0/218131550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6ce8020 0x55f7b6cffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.134+0000 7f7d967ee740 1 --2- 172.21.15.89:0/218131550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7b6cdd760 0x55f7b6cff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.134+0000 7f7d967ee740 1 --2- 172.21.15.89:0/218131550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6cdce40 0x55f7b6ce47e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.134+0000 7f7d967ee740 1 -- 172.21.15.89:0/218131550 >> 172.21.15.89:0/218131550 conn(0x55f7b6cd93d0 msgr2=0x55f7b6ce6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:26.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.134+0000 7f7d967ee740 1 -- 172.21.15.89:0/218131550 shutdown_connections 2024-09-10T08:44:26.137 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.134+0000 7f7d967ee740 1 -- 172.21.15.89:0/218131550 wait complete. 2024-09-10T08:44:26.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.135+0000 7f7d967ee740 1 -- start start 2024-09-10T08:44:26.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.135+0000 7f7d967ee740 1 Processor -- start 2024-09-10T08:44:26.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.136+0000 7f7d967ee740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6cdce40 0x55f7b6cf01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:26.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.136+0000 7f7d967ee740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6cdd760 0x55f7b6cf06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:26.138 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.136+0000 7f7d967ee740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7b6ce8020 0x55f7b6ce1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.136+0000 7f7d967ee740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c1b38c0 con 0x55f7b6cdce40 2024-09-10T08:44:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.136+0000 7f7d967ee740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c1b4d30 con 0x55f7b6cdd760 2024-09-10T08:44:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.136+0000 7f7d967ee740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c1b3f90 con 0x55f7b6ce8020 2024-09-10T08:44:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.136+0000 7f7d967ee740 10 client.0 _ll_register_callbacks cb 0x7f7d78009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.136+0000 7f7d91e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6cdce40 0x55f7b6cf01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.136+0000 7f7d91e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6cdce40 0x55f7b6cf01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36246/0 (socket says 172.21.15.89:36246) 2024-09-10T08:44:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.136+0000 7f7d91e00640 1 -- 172.21.15.89:0/3307505974 learned_addr learned my addr 172.21.15.89:0/3307505974 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.136+0000 7f7d90a00640 1 --2- 172.21.15.89:0/3307505974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6cdd760 0x55f7b6cf06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.136+0000 7f7d91e00640 1 --2- 172.21.15.89:0/3307505974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7b6ce8020 0x55f7b6ce1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.136+0000 7f7d91e00640 1 -- 172.21.15.89:0/3307505974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7b6ce8020 msgr2=0x55f7b6ce1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.136+0000 7f7d91e00640 1 --2- 172.21.15.89:0/3307505974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7b6ce8020 0x55f7b6ce1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.139 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.136+0000 7f7d91e00640 1 -- 172.21.15.89:0/3307505974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6cdd760 msgr2=0x55f7b6cf06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:26.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.136+0000 7f7d91e00640 1 --2- 172.21.15.89:0/3307505974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6cdd760 0x55f7b6cf06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.136+0000 7f7d91e00640 1 -- 172.21.15.89:0/3307505974 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7b6ce1f40 con 0x55f7b6cdce40 2024-09-10T08:44:26.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.136+0000 7f7d90a00640 1 --2- 172.21.15.89:0/3307505974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6cdd760 0x55f7b6cf06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:26.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.137+0000 7f7d91e00640 1 --2- 172.21.15.89:0/3307505974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6cdce40 0x55f7b6cf01a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7d840145e0 tx=0x7f7d84021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae200bfa3b672410 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:26.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.137+0000 7f7d72a00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:44:26.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.137+0000 7f7d72a00640 1 -- 172.21.15.89:0/3307505974 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8403b030 con 0x55f7b6cdce40 2024-09-10T08:44:26.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.137+0000 7f7d72a00640 1 -- 172.21.15.89:0/3307505974 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f7d8402a0d0 con 0x55f7b6cdce40 2024-09-10T08:44:26.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.137+0000 7f7d967ee740 1 -- 172.21.15.89:0/3307505974 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d84019050 con 0x55f7b6cdce40 2024-09-10T08:44:26.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.137+0000 7f7d72a00640 1 -- 172.21.15.89:0/3307505974 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7d84014d80 con 0x55f7b6cdce40 2024-09-10T08:44:26.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.137+0000 7f7d967ee740 10 client.16161 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:26.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.137+0000 7f7d967ee740 1 -- 172.21.15.89:0/3307505974 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f7b6d623d0 con 0x55f7b6cdce40 2024-09-10T08:44:26.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.138+0000 7f7d72a00640 1 -- 172.21.15.89:0/3307505974 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d84019050 con 0x55f7b6cdce40 2024-09-10T08:44:26.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.138+0000 7f7d72a00640 1 -- 172.21.15.89:0/3307505974 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7d84013b10 con 0x55f7b6cdce40 2024-09-10T08:44:26.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.138+0000 7f7d967ee740 10 client.16161 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:26.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.138+0000 7f7d967ee740 10 client.16161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:44:26.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.138+0000 7f7d967ee740 20 client.16161 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:26.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.138+0000 7f7d967ee740 1 --2- 172.21.15.89:0/3307505974 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f7b6d627b0 0x55f7b6d64bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:26.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.138+0000 7f7d967ee740 4 client.16161 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:44:26.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.138+0000 7f7d967ee740 1 -- 172.21.15.89:0/3307505974 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f7b6c4ae60 con 0x55f7b6d627b0 2024-09-10T08:44:26.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.138+0000 7f7d90a00640 1 --2- 172.21.15.89:0/3307505974 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f7b6d627b0 0x55f7b6d64bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:26.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.139+0000 7f7d90a00640 1 --2- 172.21.15.89:0/3307505974 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f7b6d627b0 0x55f7b6d64bd0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ba296726ffcd511 server_cookie=8b206061c8b1a4e in_seq=0 out_seq=0 2024-09-10T08:44:26.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.139+0000 7f7d72a00640 10 client.16161 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:26.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.139+0000 7f7d72a00640 1 -- 172.21.15.89:0/3307505974 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f7b6c4ae60 con 0x55f7b6d627b0 2024-09-10T08:44:26.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.139+0000 7f7d72a00640 10 client.16161 handle_command_reply: tid=0 2024-09-10T08:44:26.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.139+0000 7f7d967ee740 1 client.16161 shutdown 2024-09-10T08:44:26.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.139+0000 7f7d967ee740 1 -- 172.21.15.89:0/3307505974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6cdce40 msgr2=0x55f7b6cf01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:26.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.139+0000 7f7d967ee740 1 --2- 172.21.15.89:0/3307505974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6cdce40 0x55f7b6cf01a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7d840145e0 tx=0x7f7d84021040 comp rx=0 tx=0).stop 2024-09-10T08:44:26.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.139+0000 7f7d967ee740 1 -- 172.21.15.89:0/3307505974 shutdown_connections 2024-09-10T08:44:26.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.139+0000 7f7d967ee740 1 --2- 172.21.15.89:0/3307505974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7b6ce8020 0x55f7b6ce1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.139+0000 7f7d967ee740 1 --2- 172.21.15.89:0/3307505974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7b6cdd760 0x55f7b6cf06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.139+0000 7f7d967ee740 1 --2- 172.21.15.89:0/3307505974 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f7b6d627b0 0x55f7b6d64bd0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.139+0000 7f7d967ee740 1 --2- 172.21.15.89:0/3307505974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7b6cdce40 0x55f7b6cf01a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.139+0000 7f7d967ee740 1 -- 172.21.15.89:0/3307505974 >> 172.21.15.89:0/3307505974 conn(0x55f7b6cd93d0 msgr2=0x55f7b6ce6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:26.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.139+0000 7f7d72a00640 0 client.16161 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:26.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.140+0000 7f7d967ee740 1 -- 172.21.15.89:0/3307505974 shutdown_connections 2024-09-10T08:44:26.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.140+0000 7f7d967ee740 1 -- 172.21.15.89:0/3307505974 wait complete. 2024-09-10T08:44:26.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.140+0000 7f7d967ee740 20 client.16161 trim_cache size 0 max 16384 2024-09-10T08:44:26.144 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:44:26.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.140+0000 7f7d92800640 1 -- 172.21.15.89:0/1848119715 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7d740a2b80 msgr2=0x7f7d740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:26.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.140+0000 7f7d92800640 1 --2- 172.21.15.89:0/1848119715 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7d740a2b80 0x7f7d740a5040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7d800cd240 tx=0x7f7d800d5550 comp rx=0 tx=0).stop 2024-09-10T08:44:26.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.140+0000 7f7d92800640 1 -- 172.21.15.89:0/1848119715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d8c074590 msgr2=0x7f7d8c1a4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:26.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.140+0000 7f7d92800640 1 --2- 172.21.15.89:0/1848119715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d8c074590 0x7f7d8c1a4630 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7d8000e9f0 tx=0x7f7d8000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.140+0000 7f7d92800640 1 -- 172.21.15.89:0/1848119715 shutdown_connections 2024-09-10T08:44:26.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.140+0000 7f7d92800640 1 --2- 172.21.15.89:0/1848119715 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7d740a2b80 0x7f7d740a5040 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.140+0000 7f7d92800640 1 --2- 172.21.15.89:0/1848119715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d8c1129e0 0x7f7d8c19e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.140+0000 7f7d92800640 1 --2- 172.21.15.89:0/1848119715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d8c074eb0 0x7f7d8c1a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.140+0000 7f7d92800640 1 --2- 172.21.15.89:0/1848119715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d8c074590 0x7f7d8c1a4630 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:26.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.140+0000 7f7d92800640 1 -- 172.21.15.89:0/1848119715 >> 172.21.15.89:0/1848119715 conn(0x7f7d8c0fd710 msgr2=0x7f7d8c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:26.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.141+0000 7f7d92800640 1 -- 172.21.15.89:0/1848119715 shutdown_connections 2024-09-10T08:44:26.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:26.141+0000 7f7d92800640 1 -- 172.21.15.89:0/1848119715 wait complete. 2024-09-10T08:44:26.954 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:26 smithi089 ceph-mon[30757]: pgmap v528: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-10T08:44:27.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:26 smithi149 ceph-mon[36167]: pgmap v528: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-10T08:44:27.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:26 smithi111 ceph-mon[35220]: pgmap v528: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-10T08:44:27.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:27.338+0000 7f73f4c00640 1 -- 172.21.15.89:0/3716621990 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: -115 ) ==== 8+0+593 (crc 0 0 0) 0x5557a44c6ce0 con 0x5557a4563040 2024-09-10T08:44:27.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:27.338+0000 7f73f4c00640 10 client.26038 handle_command_reply: tid=0 2024-09-10T08:44:27.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:27.339+0000 7f741441d740 1 client.26038 shutdown 2024-09-10T08:44:27.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:27.339+0000 7f741441d740 1 -- 172.21.15.89:0/3716621990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455ae00 msgr2=0x5557a456d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:27.339+0000 7f741441d740 1 --2- 172.21.15.89:0/3716621990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455ae00 0x5557a456d660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f73f803f590 tx=0x7f73f803fa30 comp rx=0 tx=0).stop 2024-09-10T08:44:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:27.339+0000 7f741441d740 1 -- 172.21.15.89:0/3716621990 shutdown_connections 2024-09-10T08:44:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:27.339+0000 7f741441d740 1 --2- 172.21.15.89:0/3716621990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a455ae00 0x5557a456d660 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:27.339+0000 7f741441d740 1 --2- 172.21.15.89:0/3716621990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557a455a4e0 0x5557a45da230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:27.339+0000 7f741441d740 1 --2- 172.21.15.89:0/3716621990 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5557a4563040 0x5557a45f48a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:27.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:27.339+0000 7f741441d740 1 --2- 172.21.15.89:0/3716621990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5557a4556a70 0x5557a45d9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:27.339+0000 7f73f4c00640 0 client.26038 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:27.339+0000 7f741441d740 1 -- 172.21.15.89:0/3716621990 >> 172.21.15.89:0/3716621990 conn(0x5557a4562c60 msgr2=0x5557a4555bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:27.339+0000 7f741441d740 1 -- 172.21.15.89:0/3716621990 shutdown_connections 2024-09-10T08:44:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:27.339+0000 7f741441d740 1 -- 172.21.15.89:0/3716621990 wait complete. 2024-09-10T08:44:27.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:27.339+0000 7f741441d740 20 client.26038 trim_cache size 0 max 16384 2024-09-10T08:44:27.342 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:44:27.342 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 28, 2024-09-10T08:44:27.342 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24554, 2024-09-10T08:44:27.342 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 9, 2024-09-10T08:44:27.343 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": { 2024-09-10T08:44:27.343 INFO:teuthology.orchestra.run.smithi089.stdout: "a577fe06": { 2024-09-10T08:44:27.343 INFO:teuthology.orchestra.run.smithi089.stdout: "version": 9, 2024-09-10T08:44:27.343 INFO:teuthology.orchestra.run.smithi089.stdout: "age_ref": 0.0, 2024-09-10T08:44:27.343 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:44:27.343 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCING", 2024-09-10T08:44:27.343 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 45.0 2024-09-10T08:44:27.344 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:44:27.344 INFO:teuthology.orchestra.run.smithi089.stdout: "timeout": 90.0, 2024-09-10T08:44:27.344 INFO:teuthology.orchestra.run.smithi089.stdout: "expiration": 142.8, 2024-09-10T08:44:27.344 INFO:teuthology.orchestra.run.smithi089.stdout: "members": { 2024-09-10T08:44:27.344 INFO:teuthology.orchestra.run.smithi089.stdout: "file:/": { 2024-09-10T08:44:27.344 INFO:teuthology.orchestra.run.smithi089.stdout: "excluded": false, 2024-09-10T08:44:27.344 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:44:27.344 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCING", 2024-09-10T08:44:27.344 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 45.0 2024-09-10T08:44:27.344 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:44:27.344 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:44:27.345 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:44:27.345 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:44:27.345 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:44:27.345 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:44:27.345 INFO:teuthology.orchestra.run.smithi089.stderr:Error EINPROGRESS: 2024-09-10T08:44:28.042 INFO:tasks.workunit.client.0.smithi089.stdout: 87 0 0 0 0 0 0 2024-09-10T08:44:28.168 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:44:28.169 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:44:28.401 DEBUG:teuthology.orchestra.run:got remote process result: 115 2024-09-10T08:44:28.402 WARNING:tasks.quiescer.fs.[cephfs]:Set 'a577fe06' hasn't quiesced after 60.1 seconds (timeout: 90). Dumping ops with locks from all ranks. 2024-09-10T08:44:28.403 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:44:28.559 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:28.896 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:28 smithi089 ceph-mon[30757]: pgmap v529: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-10T08:44:29.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:28 smithi149 ceph-mon[36167]: pgmap v529: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-10T08:44:29.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:28 smithi111 ceph-mon[35220]: pgmap v529: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-10T08:44:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.429+0000 7f42eae00640 1 -- 172.21.15.89:0/3673753827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42e40ffb60 msgr2=0x7f42e40fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.429+0000 7f42eae00640 1 --2- 172.21.15.89:0/3673753827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42e40ffb60 0x7f42e40fffc0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009900 tx=0x7f42d002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.429+0000 7f42eae00640 1 -- 172.21.15.89:0/3673753827 shutdown_connections 2024-09-10T08:44:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.429+0000 7f42eae00640 1 --2- 172.21.15.89:0/3673753827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e4100500 0x7f42e410ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.429+0000 7f42eae00640 1 --2- 172.21.15.89:0/3673753827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42e40ffb60 0x7f42e40fffc0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.429+0000 7f42eae00640 1 --2- 172.21.15.89:0/3673753827 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42e4106b80 0x7f42e4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.429+0000 7f42eae00640 1 -- 172.21.15.89:0/3673753827 >> 172.21.15.89:0/3673753827 conn(0x7f42e40fba90 msgr2=0x7f42e40fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:29.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.430+0000 7f42eae00640 1 -- 172.21.15.89:0/3673753827 shutdown_connections 2024-09-10T08:44:29.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.430+0000 7f42eae00640 1 -- 172.21.15.89:0/3673753827 wait complete. 2024-09-10T08:44:29.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.431+0000 7f42eae00640 1 Processor -- start 2024-09-10T08:44:29.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.431+0000 7f42eae00640 1 -- start start 2024-09-10T08:44:29.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.431+0000 7f42eae00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e40ffb60 0x7f42e419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:29.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.431+0000 7f42eae00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42e4100500 0x7f42e419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.431+0000 7f42eae00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42e4106b80 0x7f42e419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.431+0000 7f42eae00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f42e4111ff0 con 0x7f42e4100500 2024-09-10T08:44:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.431+0000 7f42eae00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f42e4111e70 con 0x7f42e4106b80 2024-09-10T08:44:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.431+0000 7f42eae00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f42e4112170 con 0x7f42e40ffb60 2024-09-10T08:44:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.432+0000 7f42e8e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42e4106b80 0x7f42e419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.432+0000 7f42e8e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42e4106b80 0x7f42e419ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:59146/0 (socket says 172.21.15.89:59146) 2024-09-10T08:44:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.432+0000 7f42e8e00640 1 -- 172.21.15.89:0/2462242114 learned_addr learned my addr 172.21.15.89:0/2462242114 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.432+0000 7f42e3400640 1 --2- 172.21.15.89:0/2462242114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42e4100500 0x7f42e419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.432+0000 7f42e3e00640 1 --2- 172.21.15.89:0/2462242114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e40ffb60 0x7f42e419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.432+0000 7f42e8e00640 1 -- 172.21.15.89:0/2462242114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e40ffb60 msgr2=0x7f42e419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.432+0000 7f42e8e00640 1 --2- 172.21.15.89:0/2462242114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e40ffb60 0x7f42e419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.432+0000 7f42e8e00640 1 -- 172.21.15.89:0/2462242114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42e4100500 msgr2=0x7f42e419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:29.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.433+0000 7f42e8e00640 1 --2- 172.21.15.89:0/2462242114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42e4100500 0x7f42e419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.433+0000 7f42e8e00640 1 -- 172.21.15.89:0/2462242114 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e419f560 con 0x7f42e4106b80 2024-09-10T08:44:29.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.433+0000 7f42e3400640 1 --2- 172.21.15.89:0/2462242114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42e4100500 0x7f42e419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:29.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.433+0000 7f42e8e00640 1 --2- 172.21.15.89:0/2462242114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42e4106b80 0x7f42e419ee80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00dac0 tx=0x7f42dc00df80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:29.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.433+0000 7f42e0c00640 1 -- 172.21.15.89:0/2462242114 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc017070 con 0x7f42e4106b80 2024-09-10T08:44:29.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.433+0000 7f42eae00640 1 -- 172.21.15.89:0/2462242114 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e419f850 con 0x7f42e4106b80 2024-09-10T08:44:29.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.433+0000 7f42e0c00640 1 -- 172.21.15.89:0/2462242114 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42dc00be30 con 0x7f42e4106b80 2024-09-10T08:44:29.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.433+0000 7f42e0c00640 1 -- 172.21.15.89:0/2462242114 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc00f890 con 0x7f42e4106b80 2024-09-10T08:44:29.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.434+0000 7f42eae00640 1 -- 172.21.15.89:0/2462242114 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e41a7130 con 0x7f42e4106b80 2024-09-10T08:44:29.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.438+0000 7f42ed85a740 1 Processor -- start 2024-09-10T08:44:29.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.438+0000 7f42e0c00640 1 -- 172.21.15.89:0/2462242114 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f42dc00fad0 con 0x7f42e4106b80 2024-09-10T08:44:29.439 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.438+0000 7f42ed85a740 1 -- start start 2024-09-10T08:44:29.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.439+0000 7f42ed85a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642fecf1ac0 0x5642fecf1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:29.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.439+0000 7f42ed85a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf23e0 0x5642fecfc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:29.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.439+0000 7f42ed85a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecfcca0 0x5642fecff090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:29.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.439+0000 7f42ed85a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5642fec58290 con 0x5642fecfcca0 2024-09-10T08:44:29.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.439+0000 7f42ed85a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5642febffb40 con 0x5642fecf1ac0 2024-09-10T08:44:29.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.439+0000 7f42ed85a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5642fec54250 con 0x5642fecf23e0 2024-09-10T08:44:29.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.439+0000 7f42e3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecfcca0 0x5642fecff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:29.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.439+0000 7f42e3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf23e0 0x5642fecfc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:29.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.439+0000 7f42e3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf23e0 0x5642fecfc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:35708/0 (socket says 172.21.15.89:35708) 2024-09-10T08:44:29.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.439+0000 7f42e3e00640 1 -- 172.21.15.89:0/3433647969 learned_addr learned my addr 172.21.15.89:0/3433647969 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:29.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.440+0000 7f42e3400640 1 --2- 172.21.15.89:0/3433647969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642fecf1ac0 0x5642fecf1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:29.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.440+0000 7f42e3e00640 1 -- 172.21.15.89:0/3433647969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642fecf1ac0 msgr2=0x5642fecf1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:29.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.440+0000 7f42e3e00640 1 --2- 172.21.15.89:0/3433647969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642fecf1ac0 0x5642fecf1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.440+0000 7f42e0c00640 1 --2- 172.21.15.89:0/2462242114 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f42c80a2b80 0x7f42c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:29.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.440+0000 7f42e3e00640 1 -- 172.21.15.89:0/3433647969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecfcca0 msgr2=0x5642fecff090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:29.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.440+0000 7f42e3e00640 1 --2- 172.21.15.89:0/3433647969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecfcca0 0x5642fecff090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.440+0000 7f42e3e00640 1 -- 172.21.15.89:0/3433647969 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642fe9c96a0 con 0x5642fecf23e0 2024-09-10T08:44:29.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.440+0000 7f42e3400640 1 --2- 172.21.15.89:0/3433647969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecfcca0 0x5642fecff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:29.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.440+0000 7f42e0c00640 1 -- 172.21.15.89:0/2462242114 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f42dc0cf440 con 0x7f42e4106b80 2024-09-10T08:44:29.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.441+0000 7f42e8e00640 1 --2- 172.21.15.89:0/2462242114 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f42c80a2b80 0x7f42c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:29.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.441+0000 7f42e3e00640 1 --2- 172.21.15.89:0/3433647969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf23e0 0x5642fecfc760 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009c00 tx=0x7f42d40117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89f3f76cb82463e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:29.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.441+0000 7f42d9600640 1 -- 172.21.15.89:0/3433647969 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4012450 con 0x5642fecf23e0 2024-09-10T08:44:29.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.441+0000 7f42d9600640 1 -- 172.21.15.89:0/3433647969 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42d4012ad0 con 0x5642fecf23e0 2024-09-10T08:44:29.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.441+0000 7f42d9600640 1 -- 172.21.15.89:0/3433647969 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d400fb80 con 0x5642fecf23e0 2024-09-10T08:44:29.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.442+0000 7f42ed85a740 1 -- 172.21.15.89:0/3433647969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf23e0 msgr2=0x5642fecfc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.442+0000 7f42e8e00640 1 --2- 172.21.15.89:0/2462242114 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f42c80a2b80 0x7f42c80a5040 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f42dc097cd0 tx=0x7f42dc010e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.442+0000 7f42ed85a740 1 --2- 172.21.15.89:0/3433647969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf23e0 0x5642fecfc760 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009c00 tx=0x7f42d40117b0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.442+0000 7f42ed85a740 1 -- 172.21.15.89:0/3433647969 shutdown_connections 2024-09-10T08:44:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.442+0000 7f42ed85a740 1 --2- 172.21.15.89:0/3433647969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecfcca0 0x5642fecff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.442+0000 7f42ed85a740 1 --2- 172.21.15.89:0/3433647969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf23e0 0x5642fecfc760 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.442+0000 7f42ed85a740 1 --2- 172.21.15.89:0/3433647969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642fecf1ac0 0x5642fecf1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.442+0000 7f42ed85a740 1 -- 172.21.15.89:0/3433647969 >> 172.21.15.89:0/3433647969 conn(0x5642fecee050 msgr2=0x5642fecee430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.442+0000 7f42ed85a740 1 -- 172.21.15.89:0/3433647969 shutdown_connections 2024-09-10T08:44:29.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.443+0000 7f42ed85a740 1 -- 172.21.15.89:0/3433647969 wait complete. 2024-09-10T08:44:29.446 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.443+0000 7f42ed85a740 1 -- start start 2024-09-10T08:44:29.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.443+0000 7f42ed85a740 1 Processor -- start 2024-09-10T08:44:29.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.444+0000 7f42ed85a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecf1ac0 0x5642fecfc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:29.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.444+0000 7f42ed85a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf23e0 0x5642fecf9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:29.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.444+0000 7f42ed85a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642fecfcca0 0x5642fecf9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:29.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.444+0000 7f42ed85a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5642fecff910 con 0x5642fecf1ac0 2024-09-10T08:44:29.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.444+0000 7f42ed85a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5642fec12fa0 con 0x5642fecfcca0 2024-09-10T08:44:29.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.444+0000 7f42ed85a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5642fecffa90 con 0x5642fecf23e0 2024-09-10T08:44:29.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.444+0000 7f42ed85a740 10 client.0 _ll_register_callbacks cb 0x5642fe966ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:29.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.444+0000 7f42e3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecf1ac0 0x5642fecfc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:29.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.444+0000 7f42e3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecf1ac0 0x5642fecfc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36290/0 (socket says 172.21.15.89:36290) 2024-09-10T08:44:29.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.444+0000 7f42e3400640 1 -- 172.21.15.89:0/3503252803 learned_addr learned my addr 172.21.15.89:0/3503252803 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:29.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.444+0000 7f42e3400640 1 --2- 172.21.15.89:0/3503252803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642fecfcca0 0x5642fecf9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:29.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.445+0000 7f42e3e00640 1 --2- 172.21.15.89:0/3503252803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf23e0 0x5642fecf9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:29.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.445+0000 7f42e3400640 1 -- 172.21.15.89:0/3503252803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf23e0 msgr2=0x5642fecf9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:29.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.445+0000 7f42e3400640 1 --2- 172.21.15.89:0/3503252803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf23e0 0x5642fecf9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.445+0000 7f42e3400640 1 -- 172.21.15.89:0/3503252803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642fecfcca0 msgr2=0x5642fecf9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:29.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.445+0000 7f42e3400640 1 --2- 172.21.15.89:0/3503252803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642fecfcca0 0x5642fecf9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.445+0000 7f42e3400640 1 -- 172.21.15.89:0/3503252803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642fecfa5a0 con 0x5642fecf1ac0 2024-09-10T08:44:29.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.446+0000 7f42e3400640 1 --2- 172.21.15.89:0/3503252803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecf1ac0 0x5642fecfc5b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009230 tx=0x7f42d0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ecdd4c35b72b51 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:29.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.446+0000 7f42ce000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:44:29.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.446+0000 7f42ce000640 1 -- 172.21.15.89:0/3503252803 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0044030 con 0x5642fecf1ac0 2024-09-10T08:44:29.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.446+0000 7f42ce000640 1 -- 172.21.15.89:0/3503252803 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f42d0004970 con 0x5642fecf1ac0 2024-09-10T08:44:29.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.446+0000 7f42ed85a740 1 -- 172.21.15.89:0/3503252803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42d4012ad0 con 0x5642fecf1ac0 2024-09-10T08:44:29.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.446+0000 7f42ce000640 1 -- 172.21.15.89:0/3503252803 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f42d0052050 con 0x5642fecf1ac0 2024-09-10T08:44:29.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.446+0000 7f42ed85a740 10 client.16173 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:29.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.446+0000 7f42ed85a740 1 -- 172.21.15.89:0/3503252803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5642fed0b7f0 con 0x5642fecf1ac0 2024-09-10T08:44:29.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.447+0000 7f42ce000640 1 -- 172.21.15.89:0/3503252803 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42d4012ad0 con 0x5642fecf1ac0 2024-09-10T08:44:29.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.447+0000 7f42ce000640 1 -- 172.21.15.89:0/3503252803 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f42d0031540 con 0x5642fecf1ac0 2024-09-10T08:44:29.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.447+0000 7f42ed85a740 10 client.16173 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:29.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.447+0000 7f42ed85a740 10 client.16173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:44:29.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.447+0000 7f42ed85a740 20 client.16173 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:29.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.447+0000 7f42ed85a740 1 --2- 172.21.15.89:0/3503252803 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5642fed0cc00 0x5642fed0f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:29.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.447+0000 7f42ed85a740 4 client.16173 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:44:29.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.447+0000 7f42ed85a740 1 -- 172.21.15.89:0/3503252803 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642fec65800 con 0x5642fed0cc00 2024-09-10T08:44:29.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.448+0000 7f42e3e00640 1 --2- 172.21.15.89:0/3503252803 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5642fed0cc00 0x5642fed0f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:29.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.448+0000 7f42e3e00640 1 --2- 172.21.15.89:0/3503252803 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5642fed0cc00 0x5642fed0f050 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6358ec9738c756cc server_cookie=1296289d3630aabf in_seq=0 out_seq=0 2024-09-10T08:44:29.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.448+0000 7f42ce000640 10 client.16173 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:29.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.449+0000 7f42ce000640 1 -- 172.21.15.89:0/3503252803 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642fec65800 con 0x5642fed0cc00 2024-09-10T08:44:29.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.449+0000 7f42ce000640 10 client.16173 handle_command_reply: tid=0 2024-09-10T08:44:29.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.449+0000 7f42ed85a740 1 client.16173 shutdown 2024-09-10T08:44:29.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.449+0000 7f42ed85a740 1 -- 172.21.15.89:0/3503252803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecf1ac0 msgr2=0x5642fecfc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:29.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.449+0000 7f42ed85a740 1 --2- 172.21.15.89:0/3503252803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecf1ac0 0x5642fecfc5b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009230 tx=0x7f42d0005b50 comp rx=0 tx=0).stop 2024-09-10T08:44:29.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.449+0000 7f42ed85a740 1 -- 172.21.15.89:0/3503252803 shutdown_connections 2024-09-10T08:44:29.456 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.449+0000 7f42ed85a740 1 --2- 172.21.15.89:0/3503252803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642fecfcca0 0x5642fecf9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.449+0000 7f42ed85a740 1 --2- 172.21.15.89:0/3503252803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf23e0 0x5642fecf9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.449+0000 7f42ed85a740 1 --2- 172.21.15.89:0/3503252803 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5642fed0cc00 0x5642fed0f050 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.449+0000 7f42ed85a740 1 --2- 172.21.15.89:0/3503252803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecf1ac0 0x5642fecfc5b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.449+0000 7f42ed85a740 1 -- 172.21.15.89:0/3503252803 >> 172.21.15.89:0/3503252803 conn(0x5642fecee050 msgr2=0x5642fecf3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:29.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.450+0000 7f42ce000640 0 client.16173 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:29.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.450+0000 7f42ed85a740 1 -- 172.21.15.89:0/3503252803 shutdown_connections 2024-09-10T08:44:29.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.450+0000 7f42ed85a740 1 -- 172.21.15.89:0/3503252803 wait complete. 2024-09-10T08:44:29.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.450+0000 7f42ed85a740 20 client.16173 trim_cache size 0 max 16384 2024-09-10T08:44:29.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.468+0000 7f42ed85a740 1 Processor -- start 2024-09-10T08:44:29.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.468+0000 7f42ed85a740 1 -- start start 2024-09-10T08:44:29.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.468+0000 7f42ed85a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642fecf23e0 0x5642fecfb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:29.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.468+0000 7f42ed85a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecfcca0 0x5642feced0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:29.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.468+0000 7f42ed85a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf8950 0x5642fed04ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:29.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.468+0000 7f42ed85a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41b3880 con 0x5642fecfcca0 2024-09-10T08:44:29.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.468+0000 7f42ed85a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41b1150 con 0x5642fecf23e0 2024-09-10T08:44:29.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.468+0000 7f42ed85a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41aefa0 con 0x5642fecf8950 2024-09-10T08:44:29.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.469+0000 7f42e3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecfcca0 0x5642feced0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:29.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.469+0000 7f42e3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642fecf23e0 0x5642fecfb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:29.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.469+0000 7f42e3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecfcca0 0x5642feced0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36294/0 (socket says 172.21.15.89:36294) 2024-09-10T08:44:29.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.469+0000 7f42e3e00640 1 -- 172.21.15.89:0/3325615497 learned_addr learned my addr 172.21.15.89:0/3325615497 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:29.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.469+0000 7f42e3400640 1 --2- 172.21.15.89:0/3325615497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf8950 0x5642fed04ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:29.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.469+0000 7f42e3e00640 1 -- 172.21.15.89:0/3325615497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf8950 msgr2=0x5642fed04ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:29.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.469+0000 7f42e3e00640 1 --2- 172.21.15.89:0/3325615497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf8950 0x5642fed04ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.469+0000 7f42e3e00640 1 -- 172.21.15.89:0/3325615497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642fecf23e0 msgr2=0x5642fecfb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:29.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.469+0000 7f42e3e00640 1 --2- 172.21.15.89:0/3325615497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642fecf23e0 0x5642fecfb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.469+0000 7f42e3e00640 1 -- 172.21.15.89:0/3325615497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642fecf1840 con 0x5642fecfcca0 2024-09-10T08:44:29.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.470+0000 7f42e3e00640 1 --2- 172.21.15.89:0/3325615497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecfcca0 0x5642feced0b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f42d400f790 tx=0x7f42d400f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d54a73cac3149 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:29.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.470+0000 7f42d8c00640 1 -- 172.21.15.89:0/3325615497 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4009e90 con 0x5642fecfcca0 2024-09-10T08:44:29.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.470+0000 7f42d8c00640 1 -- 172.21.15.89:0/3325615497 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42d4009630 con 0x5642fecfcca0 2024-09-10T08:44:29.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.470+0000 7f42d8c00640 1 -- 172.21.15.89:0/3325615497 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d400fd20 con 0x5642fecfcca0 2024-09-10T08:44:29.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.471+0000 7f42ed85a740 1 -- 172.21.15.89:0/3325615497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecfcca0 msgr2=0x5642feced0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.471+0000 7f42ed85a740 1 --2- 172.21.15.89:0/3325615497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecfcca0 0x5642feced0b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f42d400f790 tx=0x7f42d400f680 comp rx=0 tx=0).stop 2024-09-10T08:44:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.471+0000 7f42ed85a740 1 -- 172.21.15.89:0/3325615497 shutdown_connections 2024-09-10T08:44:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.471+0000 7f42ed85a740 1 --2- 172.21.15.89:0/3325615497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf8950 0x5642fed04ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.471+0000 7f42ed85a740 1 --2- 172.21.15.89:0/3325615497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecfcca0 0x5642feced0b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.471+0000 7f42ed85a740 1 --2- 172.21.15.89:0/3325615497 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642fecf23e0 0x5642fecfb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.471+0000 7f42ed85a740 1 -- 172.21.15.89:0/3325615497 >> 172.21.15.89:0/3325615497 conn(0x5642fecee050 msgr2=0x5642fecec9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.472+0000 7f42ed85a740 1 -- 172.21.15.89:0/3325615497 shutdown_connections 2024-09-10T08:44:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.472+0000 7f42ed85a740 1 -- 172.21.15.89:0/3325615497 wait complete. 2024-09-10T08:44:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.472+0000 7f42ed85a740 1 -- start start 2024-09-10T08:44:29.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.472+0000 7f42ed85a740 1 Processor -- start 2024-09-10T08:44:29.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.473+0000 7f42ed85a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf23e0 0x5642fed03bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:29.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.473+0000 7f42ed85a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecf8950 0x5642fed04110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:29.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.473+0000 7f42ed85a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642fecfcca0 0x5642fed04650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:29.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.473+0000 7f42ed85a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41af610 con 0x5642fecf8950 2024-09-10T08:44:29.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.473+0000 7f42ed85a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41b0a80 con 0x5642fecfcca0 2024-09-10T08:44:29.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.473+0000 7f42ed85a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41afce0 con 0x5642fecf23e0 2024-09-10T08:44:29.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.473+0000 7f42ed85a740 10 client.0 _ll_register_callbacks cb 0x7f42d00044d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:29.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.473+0000 7f42e3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf23e0 0x5642fed03bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:29.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.473+0000 7f42e3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf23e0 0x5642fed03bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:35732/0 (socket says 172.21.15.89:35732) 2024-09-10T08:44:29.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.473+0000 7f42e3400640 1 -- 172.21.15.89:0/3856498421 learned_addr learned my addr 172.21.15.89:0/3856498421 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:29.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.474+0000 7f42e3400640 1 --2- 172.21.15.89:0/3856498421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642fecfcca0 0x5642fed04650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:29.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.474+0000 7f42e3e00640 1 --2- 172.21.15.89:0/3856498421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecf8950 0x5642fed04110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:29.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.474+0000 7f42e3400640 1 -- 172.21.15.89:0/3856498421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642fecfcca0 msgr2=0x5642fed04650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:29.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.474+0000 7f42e3400640 1 --2- 172.21.15.89:0/3856498421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642fecfcca0 0x5642fed04650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.474+0000 7f42e3400640 1 -- 172.21.15.89:0/3856498421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecf8950 msgr2=0x5642fed04110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:29.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.474+0000 7f42e3400640 1 --2- 172.21.15.89:0/3856498421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecf8950 0x5642fed04110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.474+0000 7f42e3400640 1 -- 172.21.15.89:0/3856498421 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642fed779f0 con 0x5642fecf23e0 2024-09-10T08:44:29.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.474+0000 7f42e3400640 1 --2- 172.21.15.89:0/3856498421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf23e0 0x5642fed03bd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f42d0005ec0 tx=0x7f42d00095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4322bbe4ae318903 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:29.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.474+0000 7f42ce000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:44:29.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.475+0000 7f42ce000640 1 -- 172.21.15.89:0/3856498421 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0044040 con 0x5642fecf23e0 2024-09-10T08:44:29.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.475+0000 7f42ed85a740 1 -- 172.21.15.89:0/3856498421 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42d4009630 con 0x5642fecf23e0 2024-09-10T08:44:29.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.475+0000 7f42ce000640 1 -- 172.21.15.89:0/3856498421 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f42d005dba0 con 0x5642fecf23e0 2024-09-10T08:44:29.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.475+0000 7f42ce000640 1 -- 172.21.15.89:0/3856498421 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f42d0035040 con 0x5642fecf23e0 2024-09-10T08:44:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.475+0000 7f42ed85a740 10 client.26158 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.475+0000 7f42ed85a740 1 -- 172.21.15.89:0/3856498421 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5642fed78d30 con 0x5642fecf23e0 2024-09-10T08:44:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.475+0000 7f42ce000640 1 -- 172.21.15.89:0/3856498421 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42d4009630 con 0x5642fecf23e0 2024-09-10T08:44:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.476+0000 7f42ce000640 1 -- 172.21.15.89:0/3856498421 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f42d0003680 con 0x5642fecf23e0 2024-09-10T08:44:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.476+0000 7f42ed85a740 10 client.26158 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.476+0000 7f42ed85a740 10 client.26158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:44:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.476+0000 7f42ed85a740 20 client.26158 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.476+0000 7f42ed85a740 1 --2- 172.21.15.89:0/3856498421 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5642fed793a0 0x5642fed7b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.476+0000 7f42ed85a740 4 client.26158 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:44:29.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.476+0000 7f42ed85a740 1 -- 172.21.15.89:0/3856498421 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5642fecfa130 con 0x5642fed793a0 2024-09-10T08:44:29.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.476+0000 7f42e3e00640 1 --2- 172.21.15.89:0/3856498421 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5642fed793a0 0x5642fed7b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:29.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.477+0000 7f42e3e00640 1 --2- 172.21.15.89:0/3856498421 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5642fed793a0 0x5642fed7b7c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4eecfea816fe011 server_cookie=bb8fe026574d4c36 in_seq=0 out_seq=0 2024-09-10T08:44:29.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.477+0000 7f42ce000640 10 client.26158 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:29.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.478+0000 7f42ce000640 1 -- 172.21.15.89:0/3856498421 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5642fecfa130 con 0x5642fed793a0 2024-09-10T08:44:29.480 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.478+0000 7f42ce000640 10 client.26158 handle_command_reply: tid=0 2024-09-10T08:44:29.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.478+0000 7f42ed85a740 1 client.26158 shutdown 2024-09-10T08:44:29.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.479+0000 7f42ed85a740 1 -- 172.21.15.89:0/3856498421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf23e0 msgr2=0x5642fed03bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:29.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.479+0000 7f42ed85a740 1 --2- 172.21.15.89:0/3856498421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf23e0 0x5642fed03bd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f42d0005ec0 tx=0x7f42d00095a0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.479+0000 7f42ed85a740 1 -- 172.21.15.89:0/3856498421 shutdown_connections 2024-09-10T08:44:29.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.479+0000 7f42ed85a740 1 --2- 172.21.15.89:0/3856498421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642fecfcca0 0x5642fed04650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.479+0000 7f42ed85a740 1 --2- 172.21.15.89:0/3856498421 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5642fed793a0 0x5642fed7b7c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.479+0000 7f42ed85a740 1 --2- 172.21.15.89:0/3856498421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642fecf8950 0x5642fed04110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.479+0000 7f42ed85a740 1 --2- 172.21.15.89:0/3856498421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fecf23e0 0x5642fed03bd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.479+0000 7f42ed85a740 1 -- 172.21.15.89:0/3856498421 >> 172.21.15.89:0/3856498421 conn(0x5642fecee050 msgr2=0x5642feced1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:29.481 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.479+0000 7f42ce000640 0 client.26158 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:29.482 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:44:29.482 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2024-09-10T08:44:29.482 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "dd495ccf-1cb7-4de1-96ec-55a0464e3e40", 2024-09-10T08:44:29.482 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2024-09-10T08:44:29.482 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:44:29.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.479+0000 7f42ed85a740 1 -- 172.21.15.89:0/3856498421 shutdown_connections 2024-09-10T08:44:29.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.479+0000 7f42ed85a740 1 -- 172.21.15.89:0/3856498421 wait complete. 2024-09-10T08:44:29.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.479+0000 7f42ed85a740 20 client.26158 trim_cache size 0 max 16384 2024-09-10T08:44:29.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.480+0000 7f42eae00640 1 -- 172.21.15.89:0/2462242114 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f42c80a2b80 msgr2=0x7f42c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:29.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.480+0000 7f42eae00640 1 --2- 172.21.15.89:0/2462242114 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f42c80a2b80 0x7f42c80a5040 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f42dc097cd0 tx=0x7f42dc010e30 comp rx=0 tx=0).stop 2024-09-10T08:44:29.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.480+0000 7f42eae00640 1 -- 172.21.15.89:0/2462242114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42e4106b80 msgr2=0x7f42e419ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:29.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.480+0000 7f42eae00640 1 --2- 172.21.15.89:0/2462242114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42e4106b80 0x7f42e419ee80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00dac0 tx=0x7f42dc00df80 comp rx=0 tx=0).stop 2024-09-10T08:44:29.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.480+0000 7f42eae00640 1 -- 172.21.15.89:0/2462242114 shutdown_connections 2024-09-10T08:44:29.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.480+0000 7f42eae00640 1 --2- 172.21.15.89:0/2462242114 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f42c80a2b80 0x7f42c80a5040 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.480+0000 7f42eae00640 1 --2- 172.21.15.89:0/2462242114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42e4106b80 0x7f42e419ee80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.480+0000 7f42eae00640 1 --2- 172.21.15.89:0/2462242114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42e4100500 0x7f42e419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.480+0000 7f42eae00640 1 --2- 172.21.15.89:0/2462242114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e40ffb60 0x7f42e419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:29.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.480+0000 7f42eae00640 1 -- 172.21.15.89:0/2462242114 >> 172.21.15.89:0/2462242114 conn(0x7f42e40fba90 msgr2=0x7f42e410cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:29.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.481+0000 7f42eae00640 1 -- 172.21.15.89:0/2462242114 shutdown_connections 2024-09-10T08:44:29.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:29.481+0000 7f42eae00640 1 -- 172.21.15.89:0/2462242114 wait complete. 2024-09-10T08:44:29.724 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:30.043 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:44:29 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-b[65063]: 2024-09-10T08:44:29.616+0000 7fb147ed3640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000356e [...2,head] /volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-65/ auth{1=1} fragtree_t(*^3) v174153 ap=3 RANDEPHEMERALPIN f(v3 m2024-09-10T08:43:28.763947+0000 143=143+0) n(v5 rc2024-09-10T08:43:28.763947+0000 b3795327 144=143+1) (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:271 nref=3)) caps={25157=pAsLsXsFs/pAsLsXsFsx@255} | dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x557fab76f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000356e:[<0x10000000399/blog-65 v121927>,<0x10000000208/blogtest_in v813>,<0x10000000201/src v1787>,<0x10000000006/blogbench-1.0 v1562>,<0x10000000005/tmp v185>,<0x100000001fb/client.0 v172>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v184>,<0x10000000001/sv_1 v201>,<0x10000000000/qa v190>,<0x1/volumes v185>]//[]","memoryvalue":"(2)0x1000000356e:[<0x10000000399/blog-65 v174153>,<0x10000000208/blogtest_in v1110>,<0x10000000201/src v1908>,<0x10000000006/blogbench-1.0 v1669>,<0x10000000005/tmp v271>,<0x100000001fb/client.0 v265>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v266>,<0x10000000001/sv_1 v290>,<0x10000000000/qa v267>,<0x1/volumes v267>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T08:43:17.516209+0000 142=142+0)","ondisk_value.rstat":"n(v0 rc2024-09-10T08:43:28.763947+0000 b3795327 144=143+1)","memory_value.dirstat":"f(v3 m2024-09-10T08:43:28.763947+0000 143=143+0)","memory_value.rstat":"n(v5 rc2024-09-10T08:43:28.763947+0000 b3795327 144=143+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-10T08:44:30.043 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:44:29 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-b[65063]: 2024-09-10T08:44:29.762+0000 7fb147ed3640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000023a0 [...2,head] /volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-47/ auth{1=1} fragtree_t(*^3) v174157 ap=3 RANDEPHEMERALPIN f(v4 m2024-09-10T08:43:33.887827+0000 147=147+0) n(v7 rc2024-09-10T08:43:33.887827+0000 b2082915 148=147+1) (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:311 nref=3)) caps={25157=pAsLsXsFs/pAsLsXsFsx@126} | dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x557fac55f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000023a0:[<0x10000000399/blog-47 v66310>,<0x10000000208/blogtest_in v563>,<0x10000000201/src v1720>,<0x10000000006/blogbench-1.0 v1498>,<0x10000000005/tmp v123>,<0x100000001fb/client.0 v111>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v128>,<0x10000000001/sv_1 v142>,<0x10000000000/qa v133>,<0x1/volumes v127>]//[]","memoryvalue":"(2)0x100000023a0:[<0x10000000399/blog-47 v174157>,<0x10000000208/blogtest_in v1118>,<0x10000000201/src v1908>,<0x10000000006/blogbench-1.0 v1669>,<0x10000000005/tmp v271>,<0x100000001fb/client.0 v265>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v266>,<0x10000000001/sv_1 v290>,<0x10000000000/qa v267>,<0x1/volumes v267>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T08:43:17.440210+0000 144=144+0)","ondisk_value.rstat":"n(v0 rc2024-09-10T08:43:33.887827+0000 b2082915 148=147+1)","memory_value.dirstat":"f(v4 m2024-09-10T08:43:33.887827+0000 147=147+0)","memory_value.rstat":"n(v7 rc2024-09-10T08:43:33.887827+0000 b2082915 148=147+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-10T08:44:30.705 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:44:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.036+0000 7f85d5800640 1 -- 172.21.15.89:0/3129941108 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85d0102360 msgr2=0x7f85d010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.036+0000 7f85d5800640 1 --2- 172.21.15.89:0/3129941108 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85d0102360 0x7f85d010e890 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009be0 tx=0x7f85c802a6c0 comp rx=0 tx=0).stop 2024-09-10T08:44:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.036+0000 7f85d5800640 1 -- 172.21.15.89:0/3129941108 shutdown_connections 2024-09-10T08:44:31.038 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.036+0000 7f85d5800640 1 --2- 172.21.15.89:0/3129941108 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85d0102360 0x7f85d010e890 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:31.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.036+0000 7f85d5800640 1 --2- 172.21.15.89:0/3129941108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85d01019c0 0x7f85d0101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:31.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.036+0000 7f85d5800640 1 --2- 172.21.15.89:0/3129941108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85d01079c0 0x7f85d0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:31.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.036+0000 7f85d5800640 1 -- 172.21.15.89:0/3129941108 >> 172.21.15.89:0/3129941108 conn(0x7f85d00fd740 msgr2=0x7f85d00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:31.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.037+0000 7f85d5800640 1 -- 172.21.15.89:0/3129941108 shutdown_connections 2024-09-10T08:44:31.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.037+0000 7f85d5800640 1 -- 172.21.15.89:0/3129941108 wait complete. 2024-09-10T08:44:31.039 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.038+0000 7f85d5800640 1 Processor -- start 2024-09-10T08:44:31.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.038+0000 7f85d5800640 1 -- start start 2024-09-10T08:44:31.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.039+0000 7f85d5800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85d01019c0 0x7f85d010ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:31.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.039+0000 7f85d5800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85d0102360 0x7f85d010bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:31.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.039+0000 7f85d5800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85d01079c0 0x7f85d010c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:31.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.039+0000 7f85d5800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f85d00760e0 con 0x7f85d01079c0 2024-09-10T08:44:31.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.039+0000 7f85d5800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f85d0075f60 con 0x7f85d01019c0 2024-09-10T08:44:31.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.039+0000 7f85d5800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f85d0076260 con 0x7f85d0102360 2024-09-10T08:44:31.040 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.039+0000 7f85d4e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85d01079c0 0x7f85d010c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.039+0000 7f85d4e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85d01079c0 0x7f85d010c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36334/0 (socket says 172.21.15.89:36334) 2024-09-10T08:44:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.039+0000 7f85d4e00640 1 -- 172.21.15.89:0/2207053687 learned_addr learned my addr 172.21.15.89:0/2207053687 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.040+0000 7f85d4e00640 1 -- 172.21.15.89:0/2207053687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85d0102360 msgr2=0x7f85d010bf50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T08:44:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.040+0000 7f85d4e00640 1 --2- 172.21.15.89:0/2207053687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85d0102360 0x7f85d010bf50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.040+0000 7f85d4e00640 1 -- 172.21.15.89:0/2207053687 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85d01019c0 msgr2=0x7f85d010ba10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T08:44:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.040+0000 7f85d4e00640 1 --2- 172.21.15.89:0/2207053687 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85d01019c0 0x7f85d010ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:31.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.040+0000 7f85d4e00640 1 -- 172.21.15.89:0/2207053687 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d01aaae0 con 0x7f85d01079c0 2024-09-10T08:44:31.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.041+0000 7f85d4e00640 1 --2- 172.21.15.89:0/2207053687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85d01079c0 0x7f85d010c490 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009b30 tx=0x7f85c8030c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:31.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.042+0000 7f85ccc00640 1 -- 172.21.15.89:0/2207053687 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c803f020 con 0x7f85d01079c0 2024-09-10T08:44:31.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.042+0000 7f85ccc00640 1 -- 172.21.15.89:0/2207053687 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c8030eb0 con 0x7f85d01079c0 2024-09-10T08:44:31.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.042+0000 7f85d5800640 1 -- 172.21.15.89:0/2207053687 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d01aad70 con 0x7f85d01079c0 2024-09-10T08:44:31.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.042+0000 7f85ccc00640 1 -- 172.21.15.89:0/2207053687 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8036040 con 0x7f85d01079c0 2024-09-10T08:44:31.043 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.042+0000 7f85d5800640 1 -- 172.21.15.89:0/2207053687 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d01ab2a0 con 0x7f85d01079c0 2024-09-10T08:44:31.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.043+0000 7f85d5800640 1 -- 172.21.15.89:0/2207053687 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d0103100 con 0x7f85d01079c0 2024-09-10T08:44:31.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.045+0000 7f85ccc00640 1 -- 172.21.15.89:0/2207053687 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f85c8042020 con 0x7f85d01079c0 2024-09-10T08:44:31.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.047+0000 7f85ccc00640 1 --2- 172.21.15.89:0/2207053687 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f85b40a2c00 0x7f85b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:31.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.048+0000 7f85ccc00640 1 -- 172.21.15.89:0/2207053687 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f85c80efc30 con 0x7f85d01079c0 2024-09-10T08:44:31.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.049+0000 7f85cfe00640 1 --2- 172.21.15.89:0/2207053687 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f85b40a2c00 0x7f85b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:31.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.050+0000 7f85cfe00640 1 --2- 172.21.15.89:0/2207053687 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f85b40a2c00 0x7f85b40a50c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f85c0005fd0 tx=0x7f85c0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:31.057 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.056+0000 7f85ccc00640 1 -- 172.21.15.89:0/2207053687 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85c8049020 con 0x7f85d01079c0 2024-09-10T08:44:31.081 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:30 smithi089 ceph-mon[30757]: scrub queued for path: / 2024-09-10T08:44:31.081 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:30 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:44:31.081 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:30 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:44:31.081 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:30 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:44:31.081 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:30 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:44:31.081 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:30 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:44:31.081 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:30 smithi089 ceph-mon[30757]: Scrub error on inode 0x1000000356e (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-65) see mds.b log and `damage ls` output for details 2024-09-10T08:44:31.081 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:30 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:44:31.081 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:30 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:44:31.081 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:30 smithi089 ceph-mon[30757]: pgmap v530: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-09-10T08:44:31.081 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:30 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:44:31.081 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:30 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:44:31.081 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:30 smithi089 ceph-mon[30757]: Scrub error on inode 0x100000023a0 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-47) see mds.b log and `damage ls` output for details 2024-09-10T08:44:31.082 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:30 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:44:31.174 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:31.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:30 smithi149 ceph-mon[36167]: scrub queued for path: / 2024-09-10T08:44:31.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:30 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:44:31.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:30 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:44:31.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:30 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:44:31.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:30 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:44:31.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:30 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:44:31.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:30 smithi149 ceph-mon[36167]: Scrub error on inode 0x1000000356e (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-65) see mds.b log and `damage ls` output for details 2024-09-10T08:44:31.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:30 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:44:31.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:30 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:44:31.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:30 smithi149 ceph-mon[36167]: pgmap v530: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-09-10T08:44:31.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:30 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:44:31.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:30 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:44:31.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:30 smithi149 ceph-mon[36167]: Scrub error on inode 0x100000023a0 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-47) see mds.b log and `damage ls` output for details 2024-09-10T08:44:31.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:30 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:44:31.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:30 smithi111 ceph-mon[35220]: scrub queued for path: / 2024-09-10T08:44:31.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:30 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:44:31.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:30 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:44:31.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:30 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:44:31.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:30 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:44:31.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:30 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:44:31.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:30 smithi111 ceph-mon[35220]: Scrub error on inode 0x1000000356e (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-65) see mds.b log and `damage ls` output for details 2024-09-10T08:44:31.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:30 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:44:31.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:30 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:44:31.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:30 smithi111 ceph-mon[35220]: pgmap v530: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-09-10T08:44:31.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:30 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:44:31.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:30 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:44:31.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:30 smithi111 ceph-mon[35220]: Scrub error on inode 0x100000023a0 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-47) see mds.b log and `damage ls` output for details 2024-09-10T08:44:31.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:30 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:44:31.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.323+0000 7f85d5800640 1 -- 172.21.15.89:0/2207053687 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85d002d7c0 con 0x7f85d01079c0 2024-09-10T08:44:31.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.324+0000 7f85ccc00640 1 -- 172.21.15.89:0/2207053687 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11921 (secure 0 0 0) 0x7f85c804b300 con 0x7f85d01079c0 2024-09-10T08:44:31.325 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:44:31.325 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":28,"btime":"2024-09-10T08:41:55:762565+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:41:55.148062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24518,24496]},"id":1}]} 2024-09-10T08:44:31.325 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 28 2024-09-10T08:44:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.326+0000 7f85d5800640 1 -- 172.21.15.89:0/2207053687 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f85b40a2c00 msgr2=0x7f85b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.326+0000 7f85d5800640 1 --2- 172.21.15.89:0/2207053687 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f85b40a2c00 0x7f85b40a50c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f85c0005fd0 tx=0x7f85c0009260 comp rx=0 tx=0).stop 2024-09-10T08:44:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.326+0000 7f85d5800640 1 -- 172.21.15.89:0/2207053687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85d01079c0 msgr2=0x7f85d010c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.326+0000 7f85d5800640 1 --2- 172.21.15.89:0/2207053687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85d01079c0 0x7f85d010c490 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009b30 tx=0x7f85c8030c50 comp rx=0 tx=0).stop 2024-09-10T08:44:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.326+0000 7f85d5800640 1 -- 172.21.15.89:0/2207053687 shutdown_connections 2024-09-10T08:44:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.326+0000 7f85d5800640 1 --2- 172.21.15.89:0/2207053687 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f85b40a2c00 0x7f85b40a50c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.326+0000 7f85d5800640 1 --2- 172.21.15.89:0/2207053687 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85d01079c0 0x7f85d010c490 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.326+0000 7f85d5800640 1 --2- 172.21.15.89:0/2207053687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85d0102360 0x7f85d010bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.326+0000 7f85d5800640 1 --2- 172.21.15.89:0/2207053687 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85d01019c0 0x7f85d010ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.327+0000 7f85d5800640 1 -- 172.21.15.89:0/2207053687 >> 172.21.15.89:0/2207053687 conn(0x7f85d00fd740 msgr2=0x7f85d00fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:31.328 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.327+0000 7f85d5800640 1 -- 172.21.15.89:0/2207053687 shutdown_connections 2024-09-10T08:44:31.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:31.327+0000 7f85d5800640 1 -- 172.21.15.89:0/2207053687 wait complete. 2024-09-10T08:44:32.154 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:31 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2207053687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:32.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.168+0000 7f4135000640 1 -- 172.21.15.89:0/1717484163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4130103ff0 msgr2=0x7f41301043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:32.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.168+0000 7f4135000640 1 --2- 172.21.15.89:0/1717484163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4130103ff0 0x7f41301043d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4120009900 tx=0x7f412002a410 comp rx=0 tx=0).stop 2024-09-10T08:44:32.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.169+0000 7f4135000640 1 -- 172.21.15.89:0/1717484163 shutdown_connections 2024-09-10T08:44:32.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.169+0000 7f4135000640 1 --2- 172.21.15.89:0/1717484163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4130109120 0x7f4130110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.169+0000 7f4135000640 1 --2- 172.21.15.89:0/1717484163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41301049a0 0x7f41301089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.169+0000 7f4135000640 1 --2- 172.21.15.89:0/1717484163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4130103ff0 0x7f41301043d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.169+0000 7f4135000640 1 -- 172.21.15.89:0/1717484163 >> 172.21.15.89:0/1717484163 conn(0x7f41300ffa90 msgr2=0x7f4130101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:32.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.169+0000 7f4135000640 1 -- 172.21.15.89:0/1717484163 shutdown_connections 2024-09-10T08:44:32.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.169+0000 7f4135000640 1 -- 172.21.15.89:0/1717484163 wait complete. 2024-09-10T08:44:32.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.170+0000 7f4135000640 1 Processor -- start 2024-09-10T08:44:32.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.170+0000 7f4135000640 1 -- start start 2024-09-10T08:44:32.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.170+0000 7f4135000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4130103ff0 0x7f413019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:32.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.170+0000 7f4135000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41301049a0 0x7f413019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:32.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.170+0000 7f4135000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4130109120 0x7f413019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:32.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.170+0000 7f4135000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f41301141c0 con 0x7f41301049a0 2024-09-10T08:44:32.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.170+0000 7f4135000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4130114040 con 0x7f4130109120 2024-09-10T08:44:32.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.170+0000 7f4135000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4130114340 con 0x7f4130103ff0 2024-09-10T08:44:32.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.171+0000 7f412de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4130103ff0 0x7f413019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:32.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.171+0000 7f412de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4130103ff0 0x7f413019a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:35774/0 (socket says 172.21.15.89:35774) 2024-09-10T08:44:32.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.171+0000 7f412de00640 1 -- 172.21.15.89:0/79077703 learned_addr learned my addr 172.21.15.89:0/79077703 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:32.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.171+0000 7f412e800640 1 --2- 172.21.15.89:0/79077703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4130109120 0x7f413019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:32.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.171+0000 7f4127e00640 1 --2- 172.21.15.89:0/79077703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41301049a0 0x7f413019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:32.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.171+0000 7f412de00640 1 -- 172.21.15.89:0/79077703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4130109120 msgr2=0x7f413019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:32.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.171+0000 7f412de00640 1 --2- 172.21.15.89:0/79077703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4130109120 0x7f413019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.171+0000 7f412de00640 1 -- 172.21.15.89:0/79077703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41301049a0 msgr2=0x7f413019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:32.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.171+0000 7f412de00640 1 --2- 172.21.15.89:0/79077703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41301049a0 0x7f413019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.171+0000 7f412de00640 1 -- 172.21.15.89:0/79077703 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413019f630 con 0x7f4130103ff0 2024-09-10T08:44:32.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.171+0000 7f4127e00640 1 --2- 172.21.15.89:0/79077703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41301049a0 0x7f413019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:44:32.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.172+0000 7f412de00640 1 --2- 172.21.15.89:0/79077703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4130103ff0 0x7f413019a680 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4120006010 tx=0x7f41200042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:32.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.172+0000 7f4126a00640 1 -- 172.21.15.89:0/79077703 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412003f030 con 0x7f4130103ff0 2024-09-10T08:44:32.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.172+0000 7f4126a00640 1 -- 172.21.15.89:0/79077703 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4120003ce0 con 0x7f4130103ff0 2024-09-10T08:44:32.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.172+0000 7f4126a00640 1 -- 172.21.15.89:0/79077703 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120035070 con 0x7f4130103ff0 2024-09-10T08:44:32.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.172+0000 7f4135000640 1 -- 172.21.15.89:0/79077703 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f413019f8c0 con 0x7f4130103ff0 2024-09-10T08:44:32.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.172+0000 7f4135000640 1 -- 172.21.15.89:0/79077703 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f413019fd50 con 0x7f4130103ff0 2024-09-10T08:44:32.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.175+0000 7f4126a00640 1 -- 172.21.15.89:0/79077703 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4120042030 con 0x7f4130103ff0 2024-09-10T08:44:32.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.176+0000 7f4137a27740 1 Processor -- start 2024-09-10T08:44:32.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.176+0000 7f4137a27740 1 -- start start 2024-09-10T08:44:32.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.176+0000 7f4137a27740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e12087b90 0x564e12087f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:32.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.177+0000 7f4137a27740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e120884b0 0x564e12092830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:32.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.177+0000 7f4137a27740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e12092d70 0x564e12095160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:32.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.177+0000 7f4137a27740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564e11fee3e0 con 0x564e12092d70 2024-09-10T08:44:32.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.177+0000 7f4137a27740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564e11f95d80 con 0x564e120884b0 2024-09-10T08:44:32.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.177+0000 7f4137a27740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564e11fea3a0 con 0x564e12087b90 2024-09-10T08:44:32.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.177+0000 7f4127e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e12092d70 0x564e12095160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:32.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.177+0000 7f412de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e120884b0 0x564e12092830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:32.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.177+0000 7f4127e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e12092d70 0x564e12095160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36366/0 (socket says 172.21.15.89:36366) 2024-09-10T08:44:32.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.177+0000 7f412de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e120884b0 0x564e12092830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:59230/0 (socket says 172.21.15.89:59230) 2024-09-10T08:44:32.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.177+0000 7f4127e00640 1 -- 172.21.15.89:0/484108404 learned_addr learned my addr 172.21.15.89:0/484108404 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:32.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.177+0000 7f4126a00640 1 --2- 172.21.15.89:0/79077703 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f41140a2c00 0x7f41140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:32.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.177+0000 7f4127e00640 1 -- 172.21.15.89:0/484108404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e12087b90 msgr2=0x564e12087f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:44:32.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.177+0000 7f4127e00640 1 --2- 172.21.15.89:0/484108404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e12087b90 0x564e12087f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.177+0000 7f4127e00640 1 -- 172.21.15.89:0/484108404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e120884b0 msgr2=0x564e12092830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:32.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.177+0000 7f4127e00640 1 --2- 172.21.15.89:0/484108404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e120884b0 0x564e12092830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.177+0000 7f4126a00640 1 -- 172.21.15.89:0/79077703 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f41200efce0 con 0x7f4130103ff0 2024-09-10T08:44:32.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.177+0000 7f4127e00640 1 -- 172.21.15.89:0/484108404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e11fb4ae0 con 0x564e12092d70 2024-09-10T08:44:32.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.177+0000 7f412e800640 1 --2- 172.21.15.89:0/79077703 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f41140a2c00 0x7f41140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:32.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.178+0000 7f4127e00640 1 --2- 172.21.15.89:0/484108404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e12092d70 0x564e12095160 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f411800a980 tx=0x7f411800ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f72a72a02afd62b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:32.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.178+0000 7f411f400640 1 -- 172.21.15.89:0/484108404 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118015070 con 0x564e12092d70 2024-09-10T08:44:32.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.178+0000 7f411f400640 1 -- 172.21.15.89:0/484108404 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f411800cc00 con 0x564e12092d70 2024-09-10T08:44:32.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.178+0000 7f411f400640 1 -- 172.21.15.89:0/484108404 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118018690 con 0x564e12092d70 2024-09-10T08:44:32.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.178+0000 7f412e800640 1 --2- 172.21.15.89:0/79077703 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f41140a2c00 0x7f41140a50c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4128005fd0 tx=0x7f4128009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:32.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.179+0000 7f4137a27740 1 -- 172.21.15.89:0/484108404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e12092d70 msgr2=0x564e12095160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:32.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.179+0000 7f4137a27740 1 --2- 172.21.15.89:0/484108404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e12092d70 0x564e12095160 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f411800a980 tx=0x7f411800ae40 comp rx=0 tx=0).stop 2024-09-10T08:44:32.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.179+0000 7f4137a27740 1 -- 172.21.15.89:0/484108404 shutdown_connections 2024-09-10T08:44:32.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.179+0000 7f4137a27740 1 --2- 172.21.15.89:0/484108404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e12092d70 0x564e12095160 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.179+0000 7f4137a27740 1 --2- 172.21.15.89:0/484108404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e120884b0 0x564e12092830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.179+0000 7f4137a27740 1 --2- 172.21.15.89:0/484108404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e12087b90 0x564e12087f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.180+0000 7f4137a27740 1 -- 172.21.15.89:0/484108404 >> 172.21.15.89:0/484108404 conn(0x564e12084120 msgr2=0x564e12084500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:32.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.180+0000 7f4137a27740 1 -- 172.21.15.89:0/484108404 shutdown_connections 2024-09-10T08:44:32.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.180+0000 7f4137a27740 1 -- 172.21.15.89:0/484108404 wait complete. 2024-09-10T08:44:32.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.180+0000 7f4137a27740 1 -- start start 2024-09-10T08:44:32.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.180+0000 7f4137a27740 1 Processor -- start 2024-09-10T08:44:32.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.181+0000 7f4137a27740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e12087b90 0x564e1208fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:32.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.181+0000 7f4137a27740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e120884b0 0x564e12090240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:32.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.181+0000 7f4137a27740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e12092d70 0x564e12090780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:32.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.181+0000 7f4137a27740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564e120959e0 con 0x564e12087b90 2024-09-10T08:44:32.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.181+0000 7f4137a27740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564e11fa90f0 con 0x564e12092d70 2024-09-10T08:44:32.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.181+0000 7f4137a27740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564e12095b60 con 0x564e120884b0 2024-09-10T08:44:32.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.181+0000 7f4137a27740 10 client.0 _ll_register_callbacks cb 0x564e11f9a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:32.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.181+0000 7f412de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e12087b90 0x564e1208fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:32.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.181+0000 7f412de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e12087b90 0x564e1208fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36368/0 (socket says 172.21.15.89:36368) 2024-09-10T08:44:32.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.181+0000 7f412de00640 1 -- 172.21.15.89:0/2202652706 learned_addr learned my addr 172.21.15.89:0/2202652706 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:32.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.181+0000 7f4127e00640 1 --2- 172.21.15.89:0/2202652706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e120884b0 0x564e12090240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:32.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.182+0000 7f412de00640 1 -- 172.21.15.89:0/2202652706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e120884b0 msgr2=0x564e12090240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:32.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.182+0000 7f412de00640 1 --2- 172.21.15.89:0/2202652706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e120884b0 0x564e12090240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.182+0000 7f412de00640 1 -- 172.21.15.89:0/2202652706 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e12092d70 msgr2=0x564e12090780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:44:32.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.182+0000 7f412de00640 1 --2- 172.21.15.89:0/2202652706 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e12092d70 0x564e12090780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.182+0000 7f412de00640 1 -- 172.21.15.89:0/2202652706 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e11d5f3c0 con 0x564e12087b90 2024-09-10T08:44:32.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.182+0000 7f4127e00640 1 --2- 172.21.15.89:0/2202652706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e120884b0 0x564e12090240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:32.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.182+0000 7f412de00640 1 --2- 172.21.15.89:0/2202652706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e12087b90 0x564e1208fd00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f41200c4850 tx=0x7f4120009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afc5f56f1be7813 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:32.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.182+0000 7f4113e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:44:32.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.182+0000 7f4113e00640 1 -- 172.21.15.89:0/2202652706 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120049040 con 0x564e12087b90 2024-09-10T08:44:32.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.182+0000 7f4113e00640 1 -- 172.21.15.89:0/2202652706 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f41200b8780 con 0x564e12087b90 2024-09-10T08:44:32.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.182+0000 7f4137a27740 1 -- 172.21.15.89:0/2202652706 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f411800cc00 con 0x564e12087b90 2024-09-10T08:44:32.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.183+0000 7f4113e00640 1 -- 172.21.15.89:0/2202652706 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f41200bba70 con 0x564e12087b90 2024-09-10T08:44:32.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.183+0000 7f4137a27740 10 client.16194 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:32.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.183+0000 7f4137a27740 1 -- 172.21.15.89:0/2202652706 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564e120a1730 con 0x564e12087b90 2024-09-10T08:44:32.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.183+0000 7f4113e00640 1 -- 172.21.15.89:0/2202652706 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f411800cc00 con 0x564e12087b90 2024-09-10T08:44:32.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.184+0000 7f4113e00640 1 -- 172.21.15.89:0/2202652706 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f41200c0110 con 0x564e12087b90 2024-09-10T08:44:32.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.184+0000 7f4137a27740 10 client.16194 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:32.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.184+0000 7f4137a27740 10 client.16194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:44:32.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.184+0000 7f4137a27740 20 client.16194 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:32.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.184+0000 7f4137a27740 1 --2- 172.21.15.89:0/2202652706 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x564e120a2b20 0x564e120a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:32.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.184+0000 7f4137a27740 4 client.16194 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:44:32.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.184+0000 7f4137a27740 1 -- 172.21.15.89:0/2202652706 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e11ffb950 con 0x564e120a2b20 2024-09-10T08:44:32.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.184+0000 7f4127e00640 1 --2- 172.21.15.89:0/2202652706 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x564e120a2b20 0x564e120a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:32.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.185+0000 7f4127e00640 1 --2- 172.21.15.89:0/2202652706 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x564e120a2b20 0x564e120a4f90 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd9e750705b3cc6d server_cookie=63ac1f1e63af0727 in_seq=0 out_seq=0 2024-09-10T08:44:32.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.185+0000 7f4113e00640 10 client.16194 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:32.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.186+0000 7f4113e00640 1 -- 172.21.15.89:0/2202652706 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e11ffb950 con 0x564e120a2b20 2024-09-10T08:44:32.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.186+0000 7f4113e00640 10 client.16194 handle_command_reply: tid=0 2024-09-10T08:44:32.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.186+0000 7f4137a27740 1 client.16194 shutdown 2024-09-10T08:44:32.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.186+0000 7f4137a27740 1 -- 172.21.15.89:0/2202652706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e12087b90 msgr2=0x564e1208fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:32.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.186+0000 7f4137a27740 1 --2- 172.21.15.89:0/2202652706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e12087b90 0x564e1208fd00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f41200c4850 tx=0x7f4120009560 comp rx=0 tx=0).stop 2024-09-10T08:44:32.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.186+0000 7f4137a27740 1 -- 172.21.15.89:0/2202652706 shutdown_connections 2024-09-10T08:44:32.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.186+0000 7f4137a27740 1 --2- 172.21.15.89:0/2202652706 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e12092d70 0x564e12090780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.186+0000 7f4137a27740 1 --2- 172.21.15.89:0/2202652706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e120884b0 0x564e12090240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.186+0000 7f4137a27740 1 --2- 172.21.15.89:0/2202652706 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x564e120a2b20 0x564e120a4f90 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.186+0000 7f4137a27740 1 --2- 172.21.15.89:0/2202652706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e12087b90 0x564e1208fd00 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.186+0000 7f4137a27740 1 -- 172.21.15.89:0/2202652706 >> 172.21.15.89:0/2202652706 conn(0x564e12084120 msgr2=0x564e12089e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:32.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.186+0000 7f4113e00640 0 client.16194 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:32.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.186+0000 7f4137a27740 1 -- 172.21.15.89:0/2202652706 shutdown_connections 2024-09-10T08:44:32.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.186+0000 7f4137a27740 1 -- 172.21.15.89:0/2202652706 wait complete. 2024-09-10T08:44:32.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.186+0000 7f4137a27740 20 client.16194 trim_cache size 0 max 16384 2024-09-10T08:44:32.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.205+0000 7f4137a27740 1 Processor -- start 2024-09-10T08:44:32.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.205+0000 7f4137a27740 1 -- start start 2024-09-10T08:44:32.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.205+0000 7f4137a27740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e12087b90 0x564e12089cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:32.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.205+0000 7f4137a27740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e120884b0 0x564e120aa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:32.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.205+0000 7f4137a27740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e12092d70 0x564e120aade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:32.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.205+0000 7f4137a27740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f41301b3810 con 0x564e12087b90 2024-09-10T08:44:32.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.205+0000 7f4137a27740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f41301b10e0 con 0x564e120884b0 2024-09-10T08:44:32.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.205+0000 7f4137a27740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f41301aef30 con 0x564e12092d70 2024-09-10T08:44:32.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.206+0000 7f412de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e12087b90 0x564e12089cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:32.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.206+0000 7f412de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e12087b90 0x564e12089cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36384/0 (socket says 172.21.15.89:36384) 2024-09-10T08:44:32.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.206+0000 7f412de00640 1 -- 172.21.15.89:0/2243773757 learned_addr learned my addr 172.21.15.89:0/2243773757 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:32.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.206+0000 7f4127e00640 1 --2- 172.21.15.89:0/2243773757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e120884b0 0x564e120aa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:32.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.206+0000 7f412e800640 1 --2- 172.21.15.89:0/2243773757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e12092d70 0x564e120aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:32.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.206+0000 7f4127e00640 1 -- 172.21.15.89:0/2243773757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e12092d70 msgr2=0x564e120aade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:32.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.206+0000 7f4127e00640 1 --2- 172.21.15.89:0/2243773757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e12092d70 0x564e120aade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.206+0000 7f4127e00640 1 -- 172.21.15.89:0/2243773757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e12087b90 msgr2=0x564e12089cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:32.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.206+0000 7f4127e00640 1 --2- 172.21.15.89:0/2243773757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e12087b90 0x564e12089cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.206+0000 7f4127e00640 1 -- 172.21.15.89:0/2243773757 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e12087910 con 0x564e120884b0 2024-09-10T08:44:32.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.206+0000 7f412e800640 1 --2- 172.21.15.89:0/2243773757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e12092d70 0x564e120aade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:32.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.212+0000 7f4127e00640 1 --2- 172.21.15.89:0/2243773757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e120884b0 0x564e120aa8a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4118011ee0 tx=0x7f411800a280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=625e2f5a21766b29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:32.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.212+0000 7f411ea00640 1 -- 172.21.15.89:0/2243773757 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118015040 con 0x564e120884b0 2024-09-10T08:44:32.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.212+0000 7f411ea00640 1 -- 172.21.15.89:0/2243773757 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f411800cde0 con 0x564e120884b0 2024-09-10T08:44:32.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.212+0000 7f411ea00640 1 -- 172.21.15.89:0/2243773757 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118003690 con 0x564e120884b0 2024-09-10T08:44:32.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.213+0000 7f4137a27740 1 -- 172.21.15.89:0/2243773757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e120884b0 msgr2=0x564e120aa8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:32.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.213+0000 7f4137a27740 1 --2- 172.21.15.89:0/2243773757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e120884b0 0x564e120aa8a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4118011ee0 tx=0x7f411800a280 comp rx=0 tx=0).stop 2024-09-10T08:44:32.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.213+0000 7f4137a27740 1 -- 172.21.15.89:0/2243773757 shutdown_connections 2024-09-10T08:44:32.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.213+0000 7f4137a27740 1 --2- 172.21.15.89:0/2243773757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e12092d70 0x564e120aade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.213+0000 7f4137a27740 1 --2- 172.21.15.89:0/2243773757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e120884b0 0x564e120aa8a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.213+0000 7f4137a27740 1 --2- 172.21.15.89:0/2243773757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e12087b90 0x564e12089cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.213+0000 7f4137a27740 1 -- 172.21.15.89:0/2243773757 >> 172.21.15.89:0/2243773757 conn(0x564e12084120 msgr2=0x564e120895e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:32.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.213+0000 7f4137a27740 1 -- 172.21.15.89:0/2243773757 shutdown_connections 2024-09-10T08:44:32.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.213+0000 7f4137a27740 1 -- 172.21.15.89:0/2243773757 wait complete. 2024-09-10T08:44:32.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.214+0000 7f4137a27740 1 -- start start 2024-09-10T08:44:32.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.214+0000 7f4137a27740 1 Processor -- start 2024-09-10T08:44:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.214+0000 7f4137a27740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e12087b90 0x564e1209adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.215+0000 7f4137a27740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e120884b0 0x564e1209b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.215+0000 7f4137a27740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e12092d70 0x564e12097d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.215+0000 7f4137a27740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f41301af5a0 con 0x564e120884b0 2024-09-10T08:44:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.215+0000 7f4137a27740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f41301b0a10 con 0x564e12087b90 2024-09-10T08:44:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.215+0000 7f4137a27740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f41301afc70 con 0x564e12092d70 2024-09-10T08:44:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.215+0000 7f4137a27740 10 client.0 _ll_register_callbacks cb 0x7f41200f3250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.215+0000 7f412de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e12087b90 0x564e1209adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.215+0000 7f412de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e12087b90 0x564e1209adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:59272/0 (socket says 172.21.15.89:59272) 2024-09-10T08:44:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.215+0000 7f412de00640 1 -- 172.21.15.89:0/2525680044 learned_addr learned my addr 172.21.15.89:0/2525680044 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.215+0000 7f4127e00640 1 --2- 172.21.15.89:0/2525680044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e120884b0 0x564e1209b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.215+0000 7f412e800640 1 --2- 172.21.15.89:0/2525680044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e12092d70 0x564e12097d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.215+0000 7f412de00640 1 -- 172.21.15.89:0/2525680044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e12092d70 msgr2=0x564e12097d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.215+0000 7f412de00640 1 --2- 172.21.15.89:0/2525680044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e12092d70 0x564e12097d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.215+0000 7f412de00640 1 -- 172.21.15.89:0/2525680044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e120884b0 msgr2=0x564e1209b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.215+0000 7f412de00640 1 --2- 172.21.15.89:0/2525680044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e120884b0 0x564e1209b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.215+0000 7f412de00640 1 -- 172.21.15.89:0/2525680044 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e120982c0 con 0x564e12087b90 2024-09-10T08:44:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.215+0000 7f4127e00640 1 --2- 172.21.15.89:0/2525680044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e120884b0 0x564e1209b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.215+0000 7f412de00640 1 --2- 172.21.15.89:0/2525680044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e12087b90 0x564e1209adb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4120004650 tx=0x7f41200c4df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ad7b80e058bb16a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.215+0000 7f4113e00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:44:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.216+0000 7f4113e00640 1 -- 172.21.15.89:0/2525680044 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120049040 con 0x564e12087b90 2024-09-10T08:44:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.216+0000 7f4113e00640 1 -- 172.21.15.89:0/2525680044 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f41200c0110 con 0x564e12087b90 2024-09-10T08:44:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.216+0000 7f4137a27740 1 -- 172.21.15.89:0/2525680044 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f411800cde0 con 0x564e12087b90 2024-09-10T08:44:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.216+0000 7f4113e00640 1 -- 172.21.15.89:0/2525680044 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f41200f2040 con 0x564e12087b90 2024-09-10T08:44:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.216+0000 7f4137a27740 10 client.26261 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.216+0000 7f4137a27740 1 -- 172.21.15.89:0/2525680044 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564e120982c0 con 0x564e12087b90 2024-09-10T08:44:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.216+0000 7f4113e00640 1 -- 172.21.15.89:0/2525680044 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f411800cde0 con 0x564e12087b90 2024-09-10T08:44:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.216+0000 7f4113e00640 1 -- 172.21.15.89:0/2525680044 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f41200c48f0 con 0x564e12087b90 2024-09-10T08:44:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.216+0000 7f4137a27740 10 client.26261 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.216+0000 7f4137a27740 10 client.26261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:44:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.216+0000 7f4137a27740 20 client.26261 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.216+0000 7f4137a27740 1 --2- 172.21.15.89:0/2525680044 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x564e1210e390 0x564e121107b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.216+0000 7f4137a27740 4 client.26261 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:44:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.216+0000 7f4137a27740 1 -- 172.21.15.89:0/2525680044 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564e11ff5e60 con 0x564e1210e390 2024-09-10T08:44:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.217+0000 7f4127e00640 1 --2- 172.21.15.89:0/2525680044 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x564e1210e390 0x564e121107b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.217+0000 7f4127e00640 1 --2- 172.21.15.89:0/2525680044 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x564e1210e390 0x564e121107b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf1b9665dbf27cb4 server_cookie=dd5bd68b7b58da5f in_seq=0 out_seq=0 2024-09-10T08:44:32.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.217+0000 7f4113e00640 10 client.26261 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.217+0000 7f4113e00640 1 -- 172.21.15.89:0/2525680044 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564e11ff5e60 con 0x564e1210e390 2024-09-10T08:44:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.217+0000 7f4113e00640 10 client.26261 handle_command_reply: tid=0 2024-09-10T08:44:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.217+0000 7f4137a27740 1 client.26261 shutdown 2024-09-10T08:44:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.218+0000 7f4137a27740 1 -- 172.21.15.89:0/2525680044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e12087b90 msgr2=0x564e1209adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.218+0000 7f4137a27740 1 --2- 172.21.15.89:0/2525680044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e12087b90 0x564e1209adb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4120004650 tx=0x7f41200c4df0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.218+0000 7f4137a27740 1 -- 172.21.15.89:0/2525680044 shutdown_connections 2024-09-10T08:44:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.218+0000 7f4137a27740 1 --2- 172.21.15.89:0/2525680044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564e12092d70 0x564e12097d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.218+0000 7f4137a27740 1 --2- 172.21.15.89:0/2525680044 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x564e1210e390 0x564e121107b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.218+0000 7f4137a27740 1 --2- 172.21.15.89:0/2525680044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e120884b0 0x564e1209b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.218+0000 7f4137a27740 1 --2- 172.21.15.89:0/2525680044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e12087b90 0x564e1209adb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.218+0000 7f4113e00640 0 client.26261 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:32.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.218+0000 7f4137a27740 1 -- 172.21.15.89:0/2525680044 >> 172.21.15.89:0/2525680044 conn(0x564e12084120 msgr2=0x564e1208e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:32.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.218+0000 7f4137a27740 1 -- 172.21.15.89:0/2525680044 shutdown_connections 2024-09-10T08:44:32.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.218+0000 7f4137a27740 1 -- 172.21.15.89:0/2525680044 wait complete. 2024-09-10T08:44:32.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.218+0000 7f4137a27740 20 client.26261 trim_cache size 0 max 16384 2024-09-10T08:44:32.226 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:44:32.226 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:44:32.227 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2024-09-10T08:44:32.227 INFO:teuthology.orchestra.run.smithi089.stdout: "dd495ccf-1cb7-4de1-96ec-55a0464e3e40": { 2024-09-10T08:44:32.227 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2024-09-10T08:44:32.227 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "dd495ccf-1cb7-4de1-96ec-55a0464e3e40", 2024-09-10T08:44:32.227 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2024-09-10T08:44:32.227 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:44:32.228 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:44:32.228 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:44:32.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.219+0000 7f4135000640 1 -- 172.21.15.89:0/79077703 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f41140a2c00 msgr2=0x7f41140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:32.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.219+0000 7f4135000640 1 --2- 172.21.15.89:0/79077703 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f41140a2c00 0x7f41140a50c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4128005fd0 tx=0x7f4128009500 comp rx=0 tx=0).stop 2024-09-10T08:44:32.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.219+0000 7f4135000640 1 -- 172.21.15.89:0/79077703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4130103ff0 msgr2=0x7f413019a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:32.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.219+0000 7f4135000640 1 --2- 172.21.15.89:0/79077703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4130103ff0 0x7f413019a680 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4120006010 tx=0x7f41200042f0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.219+0000 7f4135000640 1 -- 172.21.15.89:0/79077703 shutdown_connections 2024-09-10T08:44:32.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.219+0000 7f4135000640 1 --2- 172.21.15.89:0/79077703 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f41140a2c00 0x7f41140a50c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.219+0000 7f4135000640 1 --2- 172.21.15.89:0/79077703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4130109120 0x7f413019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.219+0000 7f4135000640 1 --2- 172.21.15.89:0/79077703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f41301049a0 0x7f413019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.219+0000 7f4135000640 1 --2- 172.21.15.89:0/79077703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4130103ff0 0x7f413019a680 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.219+0000 7f4135000640 1 -- 172.21.15.89:0/79077703 >> 172.21.15.89:0/79077703 conn(0x7f41300ffa90 msgr2=0x7f4130100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:32.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.219+0000 7f4135000640 1 -- 172.21.15.89:0/79077703 shutdown_connections 2024-09-10T08:44:32.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:32.219+0000 7f4135000640 1 -- 172.21.15.89:0/79077703 wait complete. 2024-09-10T08:44:32.231 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:31 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2207053687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:32.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:31 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2207053687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:32.687 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 2 (i) to a remote file /var/run/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ops-a577fe06-1-mds.i.json 2024-09-10T08:44:32.688 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 ops --flags=locks --path=/var/run/ceph/ops-a577fe06-1-mds.i.json 2024-09-10T08:44:32.978 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:32 smithi089 ceph-mon[30757]: pgmap v531: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 47 KiB/s wr, 309 op/s 2024-09-10T08:44:32.979 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:32 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:44:33.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:32 smithi149 ceph-mon[36167]: pgmap v531: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 47 KiB/s wr, 309 op/s 2024-09-10T08:44:33.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:32 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:44:33.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:32 smithi111 ceph-mon[35220]: pgmap v531: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 47 KiB/s wr, 309 op/s 2024-09-10T08:44:33.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:32 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:44:33.663 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:33.689 INFO:tasks.cephfs.filesystem:scrub status for tag:dd495ccf-1cb7-4de1-96ec-55a0464e3e40 - {'path': '/', 'tag': 'dd495ccf-1cb7-4de1-96ec-55a0464e3e40', 'options': 'recursive,force'} 2024-09-10T08:44:34.119 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:33 smithi089 ceph-mon[30757]: pgmap v532: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 40 KiB/s wr, 504 op/s 2024-09-10T08:44:34.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.197+0000 7fcf6be00640 1 -- 172.21.15.89:0/1351382284 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf6c1012a0 msgr2=0x7fcf6c105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:34.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.197+0000 7fcf6be00640 1 --2- 172.21.15.89:0/1351382284 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf6c1012a0 0x7fcf6c105770 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009900 tx=0x7fcf5802a490 comp rx=0 tx=0).stop 2024-09-10T08:44:34.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.197+0000 7fcf6be00640 1 -- 172.21.15.89:0/1351382284 shutdown_connections 2024-09-10T08:44:34.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.197+0000 7fcf6be00640 1 --2- 172.21.15.89:0/1351382284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf6c105d90 0x7fcf6c106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.197+0000 7fcf6be00640 1 --2- 172.21.15.89:0/1351382284 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf6c1012a0 0x7fcf6c105770 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.198+0000 7fcf6be00640 1 --2- 172.21.15.89:0/1351382284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf6c100980 0x7fcf6c100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.198+0000 7fcf6be00640 1 -- 172.21.15.89:0/1351382284 >> 172.21.15.89:0/1351382284 conn(0x7fcf6c0fc610 msgr2=0x7fcf6c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:34.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.198+0000 7fcf6be00640 1 -- 172.21.15.89:0/1351382284 shutdown_connections 2024-09-10T08:44:34.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.198+0000 7fcf6be00640 1 -- 172.21.15.89:0/1351382284 wait complete. 2024-09-10T08:44:34.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.199+0000 7fcf6be00640 1 Processor -- start 2024-09-10T08:44:34.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.199+0000 7fcf6be00640 1 -- start start 2024-09-10T08:44:34.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.199+0000 7fcf6be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf6c100980 0x7fcf6c1a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:34.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.199+0000 7fcf6be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf6c1012a0 0x7fcf6c1a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:34.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.199+0000 7fcf6be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf6c105d90 0x7fcf6c19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:34.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.199+0000 7fcf6be00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c1184a0 con 0x7fcf6c100980 2024-09-10T08:44:34.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.199+0000 7fcf6be00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c118320 con 0x7fcf6c1012a0 2024-09-10T08:44:34.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.200+0000 7fcf6be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c118620 con 0x7fcf6c105d90 2024-09-10T08:44:34.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.200+0000 7fcf6a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf6c1012a0 0x7fcf6c1a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:34.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.200+0000 7fcf6aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf6c100980 0x7fcf6c1a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:34.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.200+0000 7fcf6a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf6c1012a0 0x7fcf6c1a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:43104/0 (socket says 172.21.15.89:43104) 2024-09-10T08:44:34.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.200+0000 7fcf6a000640 1 -- 172.21.15.89:0/1238771570 learned_addr learned my addr 172.21.15.89:0/1238771570 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:34.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.200+0000 7fcf6aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf6c100980 0x7fcf6c1a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38436/0 (socket says 172.21.15.89:38436) 2024-09-10T08:44:34.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.200+0000 7fcf6b400640 1 --2- 172.21.15.89:0/1238771570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf6c105d90 0x7fcf6c19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:34.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.200+0000 7fcf6aa00640 1 -- 172.21.15.89:0/1238771570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf6c105d90 msgr2=0x7fcf6c19e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:34.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.200+0000 7fcf6aa00640 1 --2- 172.21.15.89:0/1238771570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf6c105d90 0x7fcf6c19e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.200+0000 7fcf6aa00640 1 -- 172.21.15.89:0/1238771570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf6c1012a0 msgr2=0x7fcf6c1a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:34.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.200+0000 7fcf6aa00640 1 --2- 172.21.15.89:0/1238771570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf6c1012a0 0x7fcf6c1a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.200+0000 7fcf6aa00640 1 -- 172.21.15.89:0/1238771570 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6c19efd0 con 0x7fcf6c100980 2024-09-10T08:44:34.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.201+0000 7fcf6aa00640 1 --2- 172.21.15.89:0/1238771570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf6c100980 0x7fcf6c1a4690 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00e9f0 tx=0x7fcf5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:34.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.201+0000 7fcf63400640 1 -- 172.21.15.89:0/1238771570 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c00ccc0 con 0x7fcf6c100980 2024-09-10T08:44:34.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.201+0000 7fcf63400640 1 -- 172.21.15.89:0/1238771570 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5c005020 con 0x7fcf6c100980 2024-09-10T08:44:34.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.201+0000 7fcf6be00640 1 -- 172.21.15.89:0/1238771570 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf6c19f2c0 con 0x7fcf6c100980 2024-09-10T08:44:34.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.201+0000 7fcf63400640 1 -- 172.21.15.89:0/1238771570 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c0049e0 con 0x7fcf6c100980 2024-09-10T08:44:34.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.201+0000 7fcf6be00640 1 -- 172.21.15.89:0/1238771570 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf6c1ab610 con 0x7fcf6c100980 2024-09-10T08:44:34.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.205+0000 7fcf74265740 1 Processor -- start 2024-09-10T08:44:34.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.206+0000 7fcf74265740 1 -- start start 2024-09-10T08:44:34.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.206+0000 7fcf74265740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb732a0 0x555a4eb73680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:34.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.206+0000 7fcf74265740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb73bc0 0x555a4eb7df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:34.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.206+0000 7fcf74265740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb7e480 0x555a4eb80870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:34.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.206+0000 7fcf74265740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555a4ead7cf0 con 0x555a4eb732a0 2024-09-10T08:44:34.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.206+0000 7fcf74265740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555a4ea7f680 con 0x555a4eb73bc0 2024-09-10T08:44:34.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.206+0000 7fcf74265740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555a4ead3cb0 con 0x555a4eb7e480 2024-09-10T08:44:34.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.206+0000 7fcf6b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb732a0 0x555a4eb73680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:34.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.207+0000 7fcf6a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb7e480 0x555a4eb80870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:34.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.207+0000 7fcf6a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb7e480 0x555a4eb80870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:36922/0 (socket says 172.21.15.89:36922) 2024-09-10T08:44:34.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.207+0000 7fcf6a000640 1 -- 172.21.15.89:0/177221362 learned_addr learned my addr 172.21.15.89:0/177221362 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:34.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.207+0000 7fcf63400640 1 -- 172.21.15.89:0/1238771570 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fcf5c005560 con 0x7fcf6c100980 2024-09-10T08:44:34.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.207+0000 7fcf6a000640 1 -- 172.21.15.89:0/177221362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb73bc0 msgr2=0x555a4eb7df40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:44:34.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.207+0000 7fcf6a000640 1 --2- 172.21.15.89:0/177221362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb73bc0 0x555a4eb7df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.207+0000 7fcf6a000640 1 -- 172.21.15.89:0/177221362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb732a0 msgr2=0x555a4eb73680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:34.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.207+0000 7fcf6a000640 1 --2- 172.21.15.89:0/177221362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb732a0 0x555a4eb73680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.207+0000 7fcf6a000640 1 -- 172.21.15.89:0/177221362 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a4ea8bb70 con 0x555a4eb7e480 2024-09-10T08:44:34.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.207+0000 7fcf6b400640 1 --2- 172.21.15.89:0/177221362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb732a0 0x555a4eb73680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:34.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.209+0000 7fcf63400640 1 --2- 172.21.15.89:0/1238771570 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcf500a2c80 0x7fcf500a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:34.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.209+0000 7fcf63400640 1 -- 172.21.15.89:0/1238771570 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fcf5c0cf390 con 0x7fcf6c100980 2024-09-10T08:44:34.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.209+0000 7fcf6b400640 1 --2- 172.21.15.89:0/1238771570 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcf500a2c80 0x7fcf500a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:34.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.210+0000 7fcf6b400640 1 --2- 172.21.15.89:0/1238771570 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcf500a2c80 0x7fcf500a5140 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009fd0 tx=0x7fcf6400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:34.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.214+0000 7fcf6a000640 1 --2- 172.21.15.89:0/177221362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb7e480 0x555a4eb80870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009a20 tx=0x7fcf58004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f69969b04971ad9b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:34.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.214+0000 7fcf57400640 1 -- 172.21.15.89:0/177221362 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58040070 con 0x555a4eb7e480 2024-09-10T08:44:34.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.214+0000 7fcf57400640 1 -- 172.21.15.89:0/177221362 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf58005500 con 0x555a4eb7e480 2024-09-10T08:44:34.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.214+0000 7fcf57400640 1 -- 172.21.15.89:0/177221362 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58035040 con 0x555a4eb7e480 2024-09-10T08:44:34.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.215+0000 7fcf74265740 1 -- 172.21.15.89:0/177221362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb7e480 msgr2=0x555a4eb80870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:34.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.215+0000 7fcf74265740 1 --2- 172.21.15.89:0/177221362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb7e480 0x555a4eb80870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009a20 tx=0x7fcf58004040 comp rx=0 tx=0).stop 2024-09-10T08:44:34.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.215+0000 7fcf74265740 1 -- 172.21.15.89:0/177221362 shutdown_connections 2024-09-10T08:44:34.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.215+0000 7fcf74265740 1 --2- 172.21.15.89:0/177221362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb7e480 0x555a4eb80870 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.215+0000 7fcf74265740 1 --2- 172.21.15.89:0/177221362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb73bc0 0x555a4eb7df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.215+0000 7fcf74265740 1 --2- 172.21.15.89:0/177221362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb732a0 0x555a4eb73680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.215+0000 7fcf74265740 1 -- 172.21.15.89:0/177221362 >> 172.21.15.89:0/177221362 conn(0x555a4eb6f830 msgr2=0x555a4eb6fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:34.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.215+0000 7fcf74265740 1 -- 172.21.15.89:0/177221362 shutdown_connections 2024-09-10T08:44:34.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.215+0000 7fcf74265740 1 -- 172.21.15.89:0/177221362 wait complete. 2024-09-10T08:44:34.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.216+0000 7fcf74265740 1 -- start start 2024-09-10T08:44:34.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.216+0000 7fcf74265740 1 Processor -- start 2024-09-10T08:44:34.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.216+0000 7fcf74265740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb732a0 0x555a4eb7dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:34.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.217+0000 7fcf74265740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb73bc0 0x555a4eb7b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:34.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.217+0000 7fcf74265740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb7e480 0x555a4eb7b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:34.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.217+0000 7fcf74265740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555a4eb81050 con 0x555a4eb732a0 2024-09-10T08:44:34.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.217+0000 7fcf74265740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555a4ea607b0 con 0x555a4eb7e480 2024-09-10T08:44:34.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.217+0000 7fcf74265740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555a4eb811d0 con 0x555a4eb73bc0 2024-09-10T08:44:34.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.217+0000 7fcf74265740 10 client.0 _ll_register_callbacks cb 0x555a4ea5b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:34.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.217+0000 7fcf6aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb732a0 0x555a4eb7dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:34.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.217+0000 7fcf6aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb732a0 0x555a4eb7dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38456/0 (socket says 172.21.15.89:38456) 2024-09-10T08:44:34.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.217+0000 7fcf6aa00640 1 -- 172.21.15.89:0/1766610192 learned_addr learned my addr 172.21.15.89:0/1766610192 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:34.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.217+0000 7fcf6b400640 1 --2- 172.21.15.89:0/1766610192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb7e480 0x555a4eb7b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:34.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.217+0000 7fcf6a000640 1 --2- 172.21.15.89:0/1766610192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb73bc0 0x555a4eb7b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:34.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.217+0000 7fcf6aa00640 1 -- 172.21.15.89:0/1766610192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb73bc0 msgr2=0x555a4eb7b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:34.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.217+0000 7fcf6aa00640 1 --2- 172.21.15.89:0/1766610192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb73bc0 0x555a4eb7b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.217+0000 7fcf6aa00640 1 -- 172.21.15.89:0/1766610192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb7e480 msgr2=0x555a4eb7b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:34.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.217+0000 7fcf6aa00640 1 --2- 172.21.15.89:0/1766610192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb7e480 0x555a4eb7b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.217+0000 7fcf6aa00640 1 -- 172.21.15.89:0/1766610192 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a4e8ac490 con 0x555a4eb732a0 2024-09-10T08:44:34.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.218+0000 7fcf6aa00640 1 --2- 172.21.15.89:0/1766610192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb732a0 0x555a4eb7dec0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c098190 tx=0x7fcf5c09b120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd6fd6c17d5e44e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:34.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.218+0000 7fcf4be00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:44:34.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.218+0000 7fcf4be00640 1 -- 172.21.15.89:0/1766610192 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c01c040 con 0x555a4eb732a0 2024-09-10T08:44:34.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.218+0000 7fcf4be00640 1 -- 172.21.15.89:0/1766610192 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fcf5c004c60 con 0x555a4eb732a0 2024-09-10T08:44:34.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.218+0000 7fcf74265740 1 -- 172.21.15.89:0/1766610192 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf58005500 con 0x555a4eb732a0 2024-09-10T08:44:34.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.218+0000 7fcf4be00640 1 -- 172.21.15.89:0/1766610192 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcf5c004e60 con 0x555a4eb732a0 2024-09-10T08:44:34.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.218+0000 7fcf74265740 10 client.16212 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:34.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.218+0000 7fcf74265740 1 -- 172.21.15.89:0/1766610192 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555a4eb8cb30 con 0x555a4eb732a0 2024-09-10T08:44:34.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.219+0000 7fcf4be00640 1 -- 172.21.15.89:0/1766610192 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf58005500 con 0x555a4eb732a0 2024-09-10T08:44:34.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.219+0000 7fcf4be00640 1 -- 172.21.15.89:0/1766610192 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcf5c0afce0 con 0x555a4eb732a0 2024-09-10T08:44:34.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.219+0000 7fcf74265740 10 client.16212 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:34.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.219+0000 7fcf74265740 10 client.16212 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:44:34.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.219+0000 7fcf74265740 20 client.16212 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:34.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:33 smithi149 ceph-mon[36167]: pgmap v532: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 40 KiB/s wr, 504 op/s 2024-09-10T08:44:34.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.219+0000 7fcf74265740 1 --2- 172.21.15.89:0/1766610192 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x555a4eb8e040 0x555a4eb90450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:34.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.219+0000 7fcf74265740 4 client.16212 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:44:34.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.219+0000 7fcf74265740 1 -- 172.21.15.89:0/1766610192 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a4eae5420 con 0x555a4eb8e040 2024-09-10T08:44:34.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.220+0000 7fcf6a000640 1 --2- 172.21.15.89:0/1766610192 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x555a4eb8e040 0x555a4eb90450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:34.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.220+0000 7fcf4be00640 10 client.16212 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:44:34.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.220+0000 7fcf6a000640 1 --2- 172.21.15.89:0/1766610192 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x555a4eb8e040 0x555a4eb90450 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=267fcee407d124b2 server_cookie=525f6e51250ab181 in_seq=0 out_seq=0 2024-09-10T08:44:34.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.221+0000 7fcf4be00640 1 -- 172.21.15.89:0/1766610192 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a4eae5420 con 0x555a4eb8e040 2024-09-10T08:44:34.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.221+0000 7fcf4be00640 10 client.16212 handle_command_reply: tid=0 2024-09-10T08:44:34.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.221+0000 7fcf74265740 1 client.16212 shutdown 2024-09-10T08:44:34.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.221+0000 7fcf74265740 1 -- 172.21.15.89:0/1766610192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb732a0 msgr2=0x555a4eb7dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:34.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.221+0000 7fcf74265740 1 --2- 172.21.15.89:0/1766610192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb732a0 0x555a4eb7dec0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c098190 tx=0x7fcf5c09b120 comp rx=0 tx=0).stop 2024-09-10T08:44:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.221+0000 7fcf74265740 1 -- 172.21.15.89:0/1766610192 shutdown_connections 2024-09-10T08:44:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.221+0000 7fcf74265740 1 --2- 172.21.15.89:0/1766610192 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x555a4eb8e040 0x555a4eb90450 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.221+0000 7fcf74265740 1 --2- 172.21.15.89:0/1766610192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb7e480 0x555a4eb7b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.221+0000 7fcf74265740 1 --2- 172.21.15.89:0/1766610192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb73bc0 0x555a4eb7b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.221+0000 7fcf74265740 1 --2- 172.21.15.89:0/1766610192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb732a0 0x555a4eb7dec0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.221+0000 7fcf4be00640 0 client.16212 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:44:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.221+0000 7fcf74265740 1 -- 172.21.15.89:0/1766610192 >> 172.21.15.89:0/1766610192 conn(0x555a4eb6f830 msgr2=0x555a4eb6eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.222+0000 7fcf74265740 1 -- 172.21.15.89:0/1766610192 shutdown_connections 2024-09-10T08:44:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.222+0000 7fcf74265740 1 -- 172.21.15.89:0/1766610192 wait complete. 2024-09-10T08:44:34.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.222+0000 7fcf74265740 20 client.16212 trim_cache size 0 max 16384 2024-09-10T08:44:34.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.248+0000 7fcf74265740 1 Processor -- start 2024-09-10T08:44:34.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.249+0000 7fcf74265740 1 -- start start 2024-09-10T08:44:34.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.249+0000 7fcf74265740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb732a0 0x555a4eb95d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:34.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.249+0000 7fcf74265740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb73bc0 0x555a4eb96260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:34.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.249+0000 7fcf74265740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb7e480 0x555a4eb78450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:34.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.249+0000 7fcf74265740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c1b7b00 con 0x555a4eb7e480 2024-09-10T08:44:34.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.249+0000 7fcf74265740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c1b53d0 con 0x555a4eb73bc0 2024-09-10T08:44:34.249 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.249+0000 7fcf74265740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c1b3220 con 0x555a4eb732a0 2024-09-10T08:44:34.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.249+0000 7fcf6aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb732a0 0x555a4eb95d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:34.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.249+0000 7fcf6aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb732a0 0x555a4eb95d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:36932/0 (socket says 172.21.15.89:36932) 2024-09-10T08:44:34.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.249+0000 7fcf6aa00640 1 -- 172.21.15.89:0/814794865 learned_addr learned my addr 172.21.15.89:0/814794865 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:34.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.249+0000 7fcf6a000640 1 --2- 172.21.15.89:0/814794865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb73bc0 0x555a4eb96260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:34.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.249+0000 7fcf6b400640 1 --2- 172.21.15.89:0/814794865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb7e480 0x555a4eb78450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:34.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.250+0000 7fcf6aa00640 1 -- 172.21.15.89:0/814794865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb73bc0 msgr2=0x555a4eb96260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:34.250 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.250+0000 7fcf6aa00640 1 --2- 172.21.15.89:0/814794865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb73bc0 0x555a4eb96260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.250+0000 7fcf6aa00640 1 -- 172.21.15.89:0/814794865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb7e480 msgr2=0x555a4eb78450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:34.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.250+0000 7fcf6aa00640 1 --2- 172.21.15.89:0/814794865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb7e480 0x555a4eb78450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.250+0000 7fcf6aa00640 1 -- 172.21.15.89:0/814794865 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a4eb90ad0 con 0x555a4eb732a0 2024-09-10T08:44:34.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.250+0000 7fcf6a000640 1 --2- 172.21.15.89:0/814794865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb73bc0 0x555a4eb96260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:34.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.250+0000 7fcf6aa00640 1 --2- 172.21.15.89:0/814794865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb732a0 0x555a4eb95d20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c097c20 tx=0x7fcf5c009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5e2ad74124c4de2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:34.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.250+0000 7fcf56a00640 1 -- 172.21.15.89:0/814794865 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c01c070 con 0x555a4eb732a0 2024-09-10T08:44:34.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.250+0000 7fcf56a00640 1 -- 172.21.15.89:0/814794865 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5c0ba1a0 con 0x555a4eb732a0 2024-09-10T08:44:34.251 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.250+0000 7fcf56a00640 1 -- 172.21.15.89:0/814794865 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c09ac60 con 0x555a4eb732a0 2024-09-10T08:44:34.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.251+0000 7fcf74265740 1 -- 172.21.15.89:0/814794865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb732a0 msgr2=0x555a4eb95d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:34.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.251+0000 7fcf74265740 1 --2- 172.21.15.89:0/814794865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb732a0 0x555a4eb95d20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c097c20 tx=0x7fcf5c009f60 comp rx=0 tx=0).stop 2024-09-10T08:44:34.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.251+0000 7fcf74265740 1 -- 172.21.15.89:0/814794865 shutdown_connections 2024-09-10T08:44:34.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.251+0000 7fcf74265740 1 --2- 172.21.15.89:0/814794865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb7e480 0x555a4eb78450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.251+0000 7fcf74265740 1 --2- 172.21.15.89:0/814794865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb73bc0 0x555a4eb96260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.251+0000 7fcf74265740 1 --2- 172.21.15.89:0/814794865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb732a0 0x555a4eb95d20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.252 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.251+0000 7fcf74265740 1 -- 172.21.15.89:0/814794865 >> 172.21.15.89:0/814794865 conn(0x555a4eb6f830 msgr2=0x555a4eb79aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:34.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.251+0000 7fcf74265740 1 -- 172.21.15.89:0/814794865 shutdown_connections 2024-09-10T08:44:34.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.252+0000 7fcf74265740 1 -- 172.21.15.89:0/814794865 wait complete. 2024-09-10T08:44:34.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.252+0000 7fcf74265740 1 -- start start 2024-09-10T08:44:34.253 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.252+0000 7fcf74265740 1 Processor -- start 2024-09-10T08:44:34.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.253+0000 7fcf74265740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb732a0 0x555a4eb78c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:34.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.253+0000 7fcf74265740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb73bc0 0x555a4eb86450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:34.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.253+0000 7fcf74265740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb7e480 0x555a4eb86990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:34.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.253+0000 7fcf74265740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c1b3890 con 0x555a4eb73bc0 2024-09-10T08:44:34.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.253+0000 7fcf74265740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c1b4d00 con 0x555a4eb732a0 2024-09-10T08:44:34.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.253+0000 7fcf74265740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c1b3f60 con 0x555a4eb7e480 2024-09-10T08:44:34.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.253+0000 7fcf74265740 10 client.0 _ll_register_callbacks cb 0x7fcf5c09b590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:34.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.253+0000 7fcf6aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb732a0 0x555a4eb78c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:34.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.253+0000 7fcf6aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb732a0 0x555a4eb78c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:43144/0 (socket says 172.21.15.89:43144) 2024-09-10T08:44:34.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.253+0000 7fcf6aa00640 1 -- 172.21.15.89:0/1579642329 learned_addr learned my addr 172.21.15.89:0/1579642329 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:34.254 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.253+0000 7fcf6a000640 1 --2- 172.21.15.89:0/1579642329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb73bc0 0x555a4eb86450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:34.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.253+0000 7fcf6b400640 1 --2- 172.21.15.89:0/1579642329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb7e480 0x555a4eb86990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:34.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.254+0000 7fcf6aa00640 1 -- 172.21.15.89:0/1579642329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb7e480 msgr2=0x555a4eb86990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:34.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.254+0000 7fcf6aa00640 1 --2- 172.21.15.89:0/1579642329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb7e480 0x555a4eb86990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.254+0000 7fcf6aa00640 1 -- 172.21.15.89:0/1579642329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb73bc0 msgr2=0x555a4eb86450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:34.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.254+0000 7fcf6aa00640 1 --2- 172.21.15.89:0/1579642329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb73bc0 0x555a4eb86450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.254+0000 7fcf6aa00640 1 -- 172.21.15.89:0/1579642329 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a4eb77440 con 0x555a4eb732a0 2024-09-10T08:44:34.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.254+0000 7fcf6a000640 1 --2- 172.21.15.89:0/1579642329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb73bc0 0x555a4eb86450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:34.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.254+0000 7fcf6aa00640 1 --2- 172.21.15.89:0/1579642329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb732a0 0x555a4eb78c80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c098190 tx=0x7fcf5c09b120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b180bec030d626c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:34.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.254+0000 7fcf4be00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:44:34.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.254+0000 7fcf4be00640 1 -- 172.21.15.89:0/1579642329 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c07e060 con 0x555a4eb732a0 2024-09-10T08:44:34.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.254+0000 7fcf4be00640 1 -- 172.21.15.89:0/1579642329 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fcf5c09ae70 con 0x555a4eb732a0 2024-09-10T08:44:34.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.254+0000 7fcf74265740 1 -- 172.21.15.89:0/1579642329 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf5c0ba1a0 con 0x555a4eb732a0 2024-09-10T08:44:34.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.255+0000 7fcf4be00640 1 -- 172.21.15.89:0/1579642329 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcf5c01c070 con 0x555a4eb732a0 2024-09-10T08:44:34.256 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.255+0000 7fcf74265740 10 client.26273 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:34.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.255+0000 7fcf74265740 1 -- 172.21.15.89:0/1579642329 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555a4ebf9e50 con 0x555a4eb732a0 2024-09-10T08:44:34.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.255+0000 7fcf4be00640 1 -- 172.21.15.89:0/1579642329 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5c0ba1a0 con 0x555a4eb732a0 2024-09-10T08:44:34.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.255+0000 7fcf4be00640 1 -- 172.21.15.89:0/1579642329 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcf5c0afce0 con 0x555a4eb732a0 2024-09-10T08:44:34.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.256+0000 7fcf74265740 10 client.26273 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:34.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.256+0000 7fcf74265740 10 client.26273 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:44:34.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.256+0000 7fcf74265740 20 client.26273 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:34.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.256+0000 7fcf74265740 1 --2- 172.21.15.89:0/1579642329 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x555a4ebfa250 0x555a4ebfc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:34.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.256+0000 7fcf74265740 4 client.26273 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-a577fe06-1-mds.i.json"}] 2024-09-10T08:44:34.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.256+0000 7fcf74265740 1 -- 172.21.15.89:0/1579642329 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-a577fe06-1-mds.i.json"}) -- 0x555a4eadf770 con 0x555a4ebfa250 2024-09-10T08:44:34.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.256+0000 7fcf6a000640 1 --2- 172.21.15.89:0/1579642329 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x555a4ebfa250 0x555a4ebfc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:34.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.257+0000 7fcf6a000640 1 --2- 172.21.15.89:0/1579642329 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x555a4ebfa250 0x555a4ebfc640 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65a014ff4b063f32 server_cookie=ac970ec0303941b9 in_seq=0 out_seq=0 2024-09-10T08:44:34.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.257+0000 7fcf4be00640 10 client.26273 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:44:34.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.276+0000 7fcf4be00640 1 -- 172.21.15.89:0/1579642329 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x555a4eadf770 con 0x555a4ebfa250 2024-09-10T08:44:34.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.276+0000 7fcf4be00640 10 client.26273 handle_command_reply: tid=0 2024-09-10T08:44:34.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.276+0000 7fcf74265740 1 client.26273 shutdown 2024-09-10T08:44:34.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.276+0000 7fcf74265740 1 -- 172.21.15.89:0/1579642329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb732a0 msgr2=0x555a4eb78c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:34.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.276+0000 7fcf74265740 1 --2- 172.21.15.89:0/1579642329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb732a0 0x555a4eb78c80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c098190 tx=0x7fcf5c09b120 comp rx=0 tx=0).stop 2024-09-10T08:44:34.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.276+0000 7fcf74265740 1 -- 172.21.15.89:0/1579642329 shutdown_connections 2024-09-10T08:44:34.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.276+0000 7fcf74265740 1 --2- 172.21.15.89:0/1579642329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555a4eb7e480 0x555a4eb86990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.276+0000 7fcf74265740 1 --2- 172.21.15.89:0/1579642329 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a4eb73bc0 0x555a4eb86450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.276+0000 7fcf74265740 1 --2- 172.21.15.89:0/1579642329 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x555a4ebfa250 0x555a4ebfc640 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.276+0000 7fcf74265740 1 --2- 172.21.15.89:0/1579642329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555a4eb732a0 0x555a4eb78c80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.276+0000 7fcf4be00640 0 client.26273 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:44:34.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.276+0000 7fcf74265740 1 -- 172.21.15.89:0/1579642329 >> 172.21.15.89:0/1579642329 conn(0x555a4eb6f830 msgr2=0x555a4eb892d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:34.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.276+0000 7fcf74265740 1 -- 172.21.15.89:0/1579642329 shutdown_connections 2024-09-10T08:44:34.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.277+0000 7fcf74265740 1 -- 172.21.15.89:0/1579642329 wait complete. 2024-09-10T08:44:34.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.277+0000 7fcf74265740 20 client.26273 trim_cache size 0 max 16384 2024-09-10T08:44:34.280 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:44:34.280 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/var/run/ceph/ops-a577fe06-1-mds.i.json" 2024-09-10T08:44:34.281 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:44:34.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.277+0000 7fcf6be00640 1 -- 172.21.15.89:0/1238771570 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcf500a2c80 msgr2=0x7fcf500a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:34.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.278+0000 7fcf6be00640 1 --2- 172.21.15.89:0/1238771570 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcf500a2c80 0x7fcf500a5140 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009fd0 tx=0x7fcf6400d040 comp rx=0 tx=0).stop 2024-09-10T08:44:34.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.278+0000 7fcf6be00640 1 -- 172.21.15.89:0/1238771570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf6c100980 msgr2=0x7fcf6c1a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:34.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.278+0000 7fcf6be00640 1 --2- 172.21.15.89:0/1238771570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf6c100980 0x7fcf6c1a4690 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00e9f0 tx=0x7fcf5c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.278+0000 7fcf6be00640 1 -- 172.21.15.89:0/1238771570 shutdown_connections 2024-09-10T08:44:34.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.278+0000 7fcf6be00640 1 --2- 172.21.15.89:0/1238771570 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcf500a2c80 0x7fcf500a5140 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.278+0000 7fcf6be00640 1 --2- 172.21.15.89:0/1238771570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf6c105d90 0x7fcf6c19e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.278+0000 7fcf6be00640 1 --2- 172.21.15.89:0/1238771570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf6c1012a0 0x7fcf6c1a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.278+0000 7fcf6be00640 1 --2- 172.21.15.89:0/1238771570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcf6c100980 0x7fcf6c1a4690 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:34.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.278+0000 7fcf6be00640 1 -- 172.21.15.89:0/1238771570 >> 172.21.15.89:0/1238771570 conn(0x7fcf6c0fc610 msgr2=0x7fcf6c10f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:34.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.278+0000 7fcf6be00640 1 -- 172.21.15.89:0/1238771570 shutdown_connections 2024-09-10T08:44:34.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:34.278+0000 7fcf6be00640 1 -- 172.21.15.89:0/1238771570 wait complete. 2024-09-10T08:44:34.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:33 smithi111 ceph-mon[35220]: pgmap v532: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 40 KiB/s wr, 504 op/s 2024-09-10T08:44:35.251 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 1 (h) to a remote file /var/run/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ops-a577fe06-1-mds.h.json 2024-09-10T08:44:35.251 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 ops --flags=locks --path=/var/run/ceph/ops-a577fe06-1-mds.h.json 2024-09-10T08:44:35.520 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:35 smithi089 ceph-mon[30757]: scrub summary: idle 2024-09-10T08:44:35.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:35 smithi111 ceph-mon[35220]: scrub summary: idle 2024-09-10T08:44:35.655 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:35.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:35 smithi149 ceph-mon[36167]: scrub summary: idle 2024-09-10T08:44:36.332 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:36 smithi089 ceph-mon[30757]: scrub summary: idle 2024-09-10T08:44:36.332 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:36 smithi089 ceph-mon[30757]: scrub summary: idle 2024-09-10T08:44:36.332 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:36 smithi089 ceph-mon[30757]: pgmap v533: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 40 KiB/s wr, 995 op/s 2024-09-10T08:44:36.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.350+0000 7fe7c7e00640 1 -- 172.21.15.89:0/3463890538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7c8074510 msgr2=0x7fe7c80748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:36.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.350+0000 7fe7c7e00640 1 --2- 172.21.15.89:0/3463890538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7c8074510 0x7fe7c80748f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009900 tx=0x7fe7b802a4a0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.350+0000 7fe7c7e00640 1 -- 172.21.15.89:0/3463890538 shutdown_connections 2024-09-10T08:44:36.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.350+0000 7fe7c7e00640 1 --2- 172.21.15.89:0/3463890538 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7c810e280 0x7fe7c8110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.350+0000 7fe7c7e00640 1 --2- 172.21.15.89:0/3463890538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7c8074e30 0x7fe7c810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.350+0000 7fe7c7e00640 1 --2- 172.21.15.89:0/3463890538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7c8074510 0x7fe7c80748f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.350+0000 7fe7c7e00640 1 -- 172.21.15.89:0/3463890538 >> 172.21.15.89:0/3463890538 conn(0x7fe7c80fd740 msgr2=0x7fe7c80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:36.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.351+0000 7fe7c7e00640 1 -- 172.21.15.89:0/3463890538 shutdown_connections 2024-09-10T08:44:36.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.351+0000 7fe7c7e00640 1 -- 172.21.15.89:0/3463890538 wait complete. 2024-09-10T08:44:36.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.351+0000 7fe7c7e00640 1 Processor -- start 2024-09-10T08:44:36.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.352+0000 7fe7c7e00640 1 -- start start 2024-09-10T08:44:36.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.352+0000 7fe7c7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7c8074510 0x7fe7c819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.352+0000 7fe7c7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7c8074e30 0x7fe7c819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.352+0000 7fe7c7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7c810e280 0x7fe7c819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.352+0000 7fe7c7e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c8114060 con 0x7fe7c8074510 2024-09-10T08:44:36.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.352+0000 7fe7c7e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c8113ee0 con 0x7fe7c810e280 2024-09-10T08:44:36.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.352+0000 7fe7c7e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81141e0 con 0x7fe7c8074e30 2024-09-10T08:44:36.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.352+0000 7fe7c6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7c8074510 0x7fe7c819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.352+0000 7fe7c6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7c8074510 0x7fe7c819a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38468/0 (socket says 172.21.15.89:38468) 2024-09-10T08:44:36.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.352+0000 7fe7c6a00640 1 -- 172.21.15.89:0/1459724435 learned_addr learned my addr 172.21.15.89:0/1459724435 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:36.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.352+0000 7fe7c7400640 1 --2- 172.21.15.89:0/1459724435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7c810e280 0x7fe7c819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.352+0000 7fe7bfe00640 1 --2- 172.21.15.89:0/1459724435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7c8074e30 0x7fe7c819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.353+0000 7fe7c6a00640 1 -- 172.21.15.89:0/1459724435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7c8074e30 msgr2=0x7fe7c819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:36.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.353+0000 7fe7c6a00640 1 --2- 172.21.15.89:0/1459724435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7c8074e30 0x7fe7c819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.353+0000 7fe7c6a00640 1 -- 172.21.15.89:0/1459724435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7c810e280 msgr2=0x7fe7c819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:36.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.353+0000 7fe7c6a00640 1 --2- 172.21.15.89:0/1459724435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7c810e280 0x7fe7c819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.353+0000 7fe7c6a00640 1 -- 172.21.15.89:0/1459724435 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c819f4f0 con 0x7fe7c8074510 2024-09-10T08:44:36.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.353+0000 7fe7c6a00640 1 --2- 172.21.15.89:0/1459724435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7c8074510 0x7fe7c819a540 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8004770 tx=0x7fe7b8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:36.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.353+0000 7fe7bf400640 1 -- 172.21.15.89:0/1459724435 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b803f070 con 0x7fe7c8074510 2024-09-10T08:44:36.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.353+0000 7fe7c7e00640 1 -- 172.21.15.89:0/1459724435 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c819f780 con 0x7fe7c8074510 2024-09-10T08:44:36.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.353+0000 7fe7bf400640 1 -- 172.21.15.89:0/1459724435 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7b8031ca0 con 0x7fe7c8074510 2024-09-10T08:44:36.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.354+0000 7fe7bf400640 1 -- 172.21.15.89:0/1459724435 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8035030 con 0x7fe7c8074510 2024-09-10T08:44:36.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.354+0000 7fe7c7e00640 1 -- 172.21.15.89:0/1459724435 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c81a70c0 con 0x7fe7c8074510 2024-09-10T08:44:36.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.357+0000 7fe7bf400640 1 -- 172.21.15.89:0/1459724435 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe7b8042070 con 0x7fe7c8074510 2024-09-10T08:44:36.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.357+0000 7fe7d062f740 1 Processor -- start 2024-09-10T08:44:36.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.357+0000 7fe7d062f740 1 -- start start 2024-09-10T08:44:36.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.358+0000 7fe7d062f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f32a0 0x555cdb9f3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.358+0000 7fe7d062f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f3bc0 0x555cdb9fdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.358+0000 7fe7d062f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdba00870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.358+0000 7fe7d062f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555cdb957cf0 con 0x555cdb9fe480 2024-09-10T08:44:36.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.358+0000 7fe7d062f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555cdb8ff680 con 0x555cdb9f32a0 2024-09-10T08:44:36.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.358+0000 7fe7d062f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555cdb953cb0 con 0x555cdb9f3bc0 2024-09-10T08:44:36.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.358+0000 7fe7bfe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdba00870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.358+0000 7fe7bfe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdba00870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38476/0 (socket says 172.21.15.89:38476) 2024-09-10T08:44:36.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.358+0000 7fe7bfe00640 1 -- 172.21.15.89:0/791370860 learned_addr learned my addr 172.21.15.89:0/791370860 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:36.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.358+0000 7fe7c7400640 1 --2- 172.21.15.89:0/791370860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f32a0 0x555cdb9f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.359+0000 7fe7bfe00640 1 -- 172.21.15.89:0/791370860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f3bc0 msgr2=0x555cdb9fdf40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:44:36.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.359+0000 7fe7bfe00640 1 --2- 172.21.15.89:0/791370860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f3bc0 0x555cdb9fdf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.359+0000 7fe7bfe00640 1 -- 172.21.15.89:0/791370860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f32a0 msgr2=0x555cdb9f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:36.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.359+0000 7fe7bfe00640 1 --2- 172.21.15.89:0/791370860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f32a0 0x555cdb9f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.359+0000 7fe7bfe00640 1 -- 172.21.15.89:0/791370860 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cdb90bb70 con 0x555cdb9fe480 2024-09-10T08:44:36.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.359+0000 7fe7c7400640 1 --2- 172.21.15.89:0/791370860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f32a0 0x555cdb9f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:36.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.359+0000 7fe7bf400640 1 --2- 172.21.15.89:0/1459724435 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe7ac0a2b80 0x7fe7ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.359+0000 7fe7bfe00640 1 --2- 172.21.15.89:0/791370860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdba00870 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009a10 tx=0x7fe7b0009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d787550e8ff474 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:36.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.359+0000 7fe7bf400640 1 -- 172.21.15.89:0/1459724435 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fe7b80efc10 con 0x7fe7c8074510 2024-09-10T08:44:36.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.359+0000 7fe7c7400640 1 --2- 172.21.15.89:0/1459724435 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe7ac0a2b80 0x7fe7ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.359+0000 7fe7b7e00640 1 -- 172.21.15.89:0/791370860 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0017070 con 0x555cdb9fe480 2024-09-10T08:44:36.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.360+0000 7fe7b7e00640 1 -- 172.21.15.89:0/791370860 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7b0013470 con 0x555cdb9fe480 2024-09-10T08:44:36.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.360+0000 7fe7b7e00640 1 -- 172.21.15.89:0/791370860 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0012630 con 0x555cdb9fe480 2024-09-10T08:44:36.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.360+0000 7fe7d062f740 1 -- 172.21.15.89:0/791370860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 msgr2=0x555cdba00870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:36.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.360+0000 7fe7c7400640 1 --2- 172.21.15.89:0/1459724435 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe7ac0a2b80 0x7fe7ac0a5040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80754f0 tx=0x7fe7c0009740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:36.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.360+0000 7fe7d062f740 1 --2- 172.21.15.89:0/791370860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdba00870 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009a10 tx=0x7fe7b0009ed0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.361+0000 7fe7d062f740 1 -- 172.21.15.89:0/791370860 shutdown_connections 2024-09-10T08:44:36.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.361+0000 7fe7d062f740 1 --2- 172.21.15.89:0/791370860 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdba00870 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.361+0000 7fe7d062f740 1 --2- 172.21.15.89:0/791370860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f3bc0 0x555cdb9fdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.361+0000 7fe7d062f740 1 --2- 172.21.15.89:0/791370860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f32a0 0x555cdb9f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.361+0000 7fe7d062f740 1 -- 172.21.15.89:0/791370860 >> 172.21.15.89:0/791370860 conn(0x555cdb9ef830 msgr2=0x555cdb9efc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:36.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.361+0000 7fe7d062f740 1 -- 172.21.15.89:0/791370860 shutdown_connections 2024-09-10T08:44:36.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.361+0000 7fe7d062f740 1 -- 172.21.15.89:0/791370860 wait complete. 2024-09-10T08:44:36.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.361+0000 7fe7d062f740 1 -- start start 2024-09-10T08:44:36.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.362+0000 7fe7d062f740 1 Processor -- start 2024-09-10T08:44:36.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.362+0000 7fe7d062f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f32a0 0x555cdb9fde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.362+0000 7fe7d062f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f3bc0 0x555cdb9fb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.362+0000 7fe7d062f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdb9fb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.362+0000 7fe7d062f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555cdba01050 con 0x555cdb9fe480 2024-09-10T08:44:36.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.362+0000 7fe7d062f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555cdb8e07b0 con 0x555cdb9f3bc0 2024-09-10T08:44:36.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.362+0000 7fe7d062f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555cdba011d0 con 0x555cdb9f32a0 2024-09-10T08:44:36.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.362+0000 7fe7d062f740 10 client.0 _ll_register_callbacks cb 0x555cdb8db790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:36.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.362+0000 7fe7c7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdb9fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.362+0000 7fe7c6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f32a0 0x555cdb9fde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.363+0000 7fe7c6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f32a0 0x555cdb9fde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:36978/0 (socket says 172.21.15.89:36978) 2024-09-10T08:44:36.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.363+0000 7fe7c7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdb9fb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38488/0 (socket says 172.21.15.89:38488) 2024-09-10T08:44:36.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.363+0000 7fe7c6a00640 1 -- 172.21.15.89:0/718122808 learned_addr learned my addr 172.21.15.89:0/718122808 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:36.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.363+0000 7fe7c6a00640 1 -- 172.21.15.89:0/718122808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f3bc0 msgr2=0x555cdb9fb130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:44:36.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.363+0000 7fe7c6a00640 1 --2- 172.21.15.89:0/718122808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f3bc0 0x555cdb9fb130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.363+0000 7fe7c6a00640 1 -- 172.21.15.89:0/718122808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 msgr2=0x555cdb9fb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:36.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.363+0000 7fe7c6a00640 1 --2- 172.21.15.89:0/718122808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdb9fb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.363+0000 7fe7c6a00640 1 -- 172.21.15.89:0/718122808 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cdb72c490 con 0x555cdb9f32a0 2024-09-10T08:44:36.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.363+0000 7fe7c7400640 1 --2- 172.21.15.89:0/718122808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdb9fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:36.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.363+0000 7fe7c6a00640 1 --2- 172.21.15.89:0/718122808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f32a0 0x555cdb9fde80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80bbd40 tx=0x7fe7b80b83f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbbe7436ba888aab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:36.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.363+0000 7fe7b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:44:36.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.363+0000 7fe7b4c00640 1 -- 172.21.15.89:0/718122808 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8049050 con 0x555cdb9f32a0 2024-09-10T08:44:36.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.363+0000 7fe7b4c00640 1 -- 172.21.15.89:0/718122808 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fe7b80b86b0 con 0x555cdb9f32a0 2024-09-10T08:44:36.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.363+0000 7fe7d062f740 1 -- 172.21.15.89:0/718122808 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7b0013470 con 0x555cdb9f32a0 2024-09-10T08:44:36.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.364+0000 7fe7b4c00640 1 -- 172.21.15.89:0/718122808 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe7b80d5020 con 0x555cdb9f32a0 2024-09-10T08:44:36.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.364+0000 7fe7d062f740 10 client.26194 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:36.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.364+0000 7fe7d062f740 1 -- 172.21.15.89:0/718122808 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555cdba010b0 con 0x555cdb9f32a0 2024-09-10T08:44:36.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.364+0000 7fe7b4c00640 1 -- 172.21.15.89:0/718122808 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7b0013470 con 0x555cdb9f32a0 2024-09-10T08:44:36.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.364+0000 7fe7b4c00640 1 -- 172.21.15.89:0/718122808 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe7b80bc2a0 con 0x555cdb9f32a0 2024-09-10T08:44:36.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.364+0000 7fe7d062f740 10 client.26194 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:36.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.364+0000 7fe7d062f740 10 client.26194 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:44:36.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.364+0000 7fe7d062f740 20 client.26194 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:36.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.364+0000 7fe7d062f740 1 --2- 172.21.15.89:0/718122808 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x555cdba0dee0 0x555cdba10340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.364+0000 7fe7d062f740 4 client.26194 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:44:36.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.364+0000 7fe7d062f740 1 -- 172.21.15.89:0/718122808 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cdb965420 con 0x555cdba0dee0 2024-09-10T08:44:36.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.365+0000 7fe7bfe00640 1 --2- 172.21.15.89:0/718122808 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x555cdba0dee0 0x555cdba10340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.365+0000 7fe7b4c00640 10 client.26194 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:44:36.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.365+0000 7fe7bfe00640 1 --2- 172.21.15.89:0/718122808 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x555cdba0dee0 0x555cdba10340 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf82c194c4de05b4 server_cookie=a15b4c5fdc7ba4ce in_seq=0 out_seq=0 2024-09-10T08:44:36.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.366+0000 7fe7b4c00640 1 -- 172.21.15.89:0/718122808 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555cdb965420 con 0x555cdba0dee0 2024-09-10T08:44:36.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.366+0000 7fe7b4c00640 10 client.26194 handle_command_reply: tid=0 2024-09-10T08:44:36.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.366+0000 7fe7d062f740 1 client.26194 shutdown 2024-09-10T08:44:36.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.366+0000 7fe7d062f740 1 -- 172.21.15.89:0/718122808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f32a0 msgr2=0x555cdb9fde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:36.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.366+0000 7fe7d062f740 1 --2- 172.21.15.89:0/718122808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f32a0 0x555cdb9fde80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80bbd40 tx=0x7fe7b80b83f0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.366+0000 7fe7d062f740 1 -- 172.21.15.89:0/718122808 shutdown_connections 2024-09-10T08:44:36.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.366+0000 7fe7d062f740 1 --2- 172.21.15.89:0/718122808 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x555cdba0dee0 0x555cdba10340 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.366+0000 7fe7d062f740 1 --2- 172.21.15.89:0/718122808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdb9fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.366+0000 7fe7b4c00640 0 client.26194 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:44:36.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.366+0000 7fe7d062f740 1 --2- 172.21.15.89:0/718122808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f3bc0 0x555cdb9fb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.366+0000 7fe7d062f740 1 --2- 172.21.15.89:0/718122808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f32a0 0x555cdb9fde80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.366+0000 7fe7d062f740 1 -- 172.21.15.89:0/718122808 >> 172.21.15.89:0/718122808 conn(0x555cdb9ef830 msgr2=0x555cdb9eeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:36.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.367+0000 7fe7d062f740 1 -- 172.21.15.89:0/718122808 shutdown_connections 2024-09-10T08:44:36.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.367+0000 7fe7d062f740 1 -- 172.21.15.89:0/718122808 wait complete. 2024-09-10T08:44:36.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.367+0000 7fe7d062f740 20 client.26194 trim_cache size 0 max 16384 2024-09-10T08:44:36.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.383+0000 7fe7d062f740 1 Processor -- start 2024-09-10T08:44:36.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.383+0000 7fe7d062f740 1 -- start start 2024-09-10T08:44:36.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.383+0000 7fe7d062f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f32a0 0x555cdba0a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.383+0000 7fe7d062f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f3bc0 0x555cdb9ff870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.383+0000 7fe7d062f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdb9ffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.383+0000 7fe7d062f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81b3700 con 0x555cdb9fe480 2024-09-10T08:44:36.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.384+0000 7fe7d062f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81b0fd0 con 0x555cdb9f32a0 2024-09-10T08:44:36.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.384+0000 7fe7d062f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81aee20 con 0x555cdb9f3bc0 2024-09-10T08:44:36.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.384+0000 7fe7c6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f32a0 0x555cdba0a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.384+0000 7fe7c6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f32a0 0x555cdba0a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:43188/0 (socket says 172.21.15.89:43188) 2024-09-10T08:44:36.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.384+0000 7fe7c6a00640 1 -- 172.21.15.89:0/2535770286 learned_addr learned my addr 172.21.15.89:0/2535770286 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:36.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.384+0000 7fe7bfe00640 1 --2- 172.21.15.89:0/2535770286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f3bc0 0x555cdb9ff870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.384+0000 7fe7c6a00640 1 -- 172.21.15.89:0/2535770286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f3bc0 msgr2=0x555cdb9ff870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:36.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.384+0000 7fe7c6a00640 1 --2- 172.21.15.89:0/2535770286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f3bc0 0x555cdb9ff870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.384+0000 7fe7c6a00640 1 -- 172.21.15.89:0/2535770286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 msgr2=0x555cdb9ffdb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:44:36.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.384+0000 7fe7c7400640 1 --2- 172.21.15.89:0/2535770286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdb9ffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.385+0000 7fe7c6a00640 1 --2- 172.21.15.89:0/2535770286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdb9ffdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.385+0000 7fe7c6a00640 1 -- 172.21.15.89:0/2535770286 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cdb9fba90 con 0x555cdb9f32a0 2024-09-10T08:44:36.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.385+0000 7fe7c7400640 1 --2- 172.21.15.89:0/2535770286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdb9ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:44:36.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.385+0000 7fe7c6a00640 1 --2- 172.21.15.89:0/2535770286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f32a0 0x555cdba0a2b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8000c00 tx=0x7fe7b80efeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3242c3573142b300 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:36.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.385+0000 7fe7b7400640 1 -- 172.21.15.89:0/2535770286 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b809f070 con 0x555cdb9f32a0 2024-09-10T08:44:36.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.385+0000 7fe7b7400640 1 -- 172.21.15.89:0/2535770286 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7b80bc740 con 0x555cdb9f32a0 2024-09-10T08:44:36.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.385+0000 7fe7b7400640 1 -- 172.21.15.89:0/2535770286 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8049070 con 0x555cdb9f32a0 2024-09-10T08:44:36.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.386+0000 7fe7d062f740 1 -- 172.21.15.89:0/2535770286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f32a0 msgr2=0x555cdba0a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:36.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.386+0000 7fe7d062f740 1 --2- 172.21.15.89:0/2535770286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f32a0 0x555cdba0a2b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8000c00 tx=0x7fe7b80efeb0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.386+0000 7fe7d062f740 1 -- 172.21.15.89:0/2535770286 shutdown_connections 2024-09-10T08:44:36.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.386+0000 7fe7d062f740 1 --2- 172.21.15.89:0/2535770286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdb9ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.386+0000 7fe7d062f740 1 --2- 172.21.15.89:0/2535770286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f3bc0 0x555cdb9ff870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.386+0000 7fe7d062f740 1 --2- 172.21.15.89:0/2535770286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f32a0 0x555cdba0a2b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.386+0000 7fe7d062f740 1 -- 172.21.15.89:0/2535770286 >> 172.21.15.89:0/2535770286 conn(0x555cdb9ef830 msgr2=0x555cdb9fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:36.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.386+0000 7fe7d062f740 1 -- 172.21.15.89:0/2535770286 shutdown_connections 2024-09-10T08:44:36.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.386+0000 7fe7d062f740 1 -- 172.21.15.89:0/2535770286 wait complete. 2024-09-10T08:44:36.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.386+0000 7fe7d062f740 1 -- start start 2024-09-10T08:44:36.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.387+0000 7fe7d062f740 1 Processor -- start 2024-09-10T08:44:36.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.387+0000 7fe7d062f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f32a0 0x555cdba06610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.387+0000 7fe7d062f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f3bc0 0x555cdba06b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.387+0000 7fe7d062f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdba013e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.387+0000 7fe7d062f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81af490 con 0x555cdb9fe480 2024-09-10T08:44:36.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.387+0000 7fe7d062f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81b0900 con 0x555cdb9f3bc0 2024-09-10T08:44:36.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.387+0000 7fe7d062f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c81afb60 con 0x555cdb9f32a0 2024-09-10T08:44:36.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.387+0000 7fe7d062f740 10 client.0 _ll_register_callbacks cb 0x7fe7b8005a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:36.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.387+0000 7fe7bfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f3bc0 0x555cdba06b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.387+0000 7fe7bfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f3bc0 0x555cdba06b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:43196/0 (socket says 172.21.15.89:43196) 2024-09-10T08:44:36.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.387+0000 7fe7bfe00640 1 -- 172.21.15.89:0/879541306 learned_addr learned my addr 172.21.15.89:0/879541306 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:36.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.387+0000 7fe7c7400640 1 --2- 172.21.15.89:0/879541306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdba013e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.387+0000 7fe7c6a00640 1 --2- 172.21.15.89:0/879541306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f32a0 0x555cdba06610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.388+0000 7fe7c6a00640 1 -- 172.21.15.89:0/879541306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f3bc0 msgr2=0x555cdba06b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:36.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.388+0000 7fe7c6a00640 1 --2- 172.21.15.89:0/879541306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f3bc0 0x555cdba06b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.388+0000 7fe7c6a00640 1 -- 172.21.15.89:0/879541306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 msgr2=0x555cdba013e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:36.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.388+0000 7fe7c6a00640 1 --2- 172.21.15.89:0/879541306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdba013e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.388+0000 7fe7c6a00640 1 -- 172.21.15.89:0/879541306 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cdba01b20 con 0x555cdb9f32a0 2024-09-10T08:44:36.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.388+0000 7fe7c6a00640 1 --2- 172.21.15.89:0/879541306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f32a0 0x555cdba06610 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80b8a10 tx=0x7fe7b80bc640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d92d87fb13e822d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:36.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.388+0000 7fe7b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:44:36.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.388+0000 7fe7b4c00640 1 -- 172.21.15.89:0/879541306 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b809f020 con 0x555cdb9f32a0 2024-09-10T08:44:36.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.389+0000 7fe7b4c00640 1 -- 172.21.15.89:0/879541306 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fe7b80b6db0 con 0x555cdb9f32a0 2024-09-10T08:44:36.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.389+0000 7fe7d062f740 1 -- 172.21.15.89:0/879541306 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7b80bc740 con 0x555cdb9f32a0 2024-09-10T08:44:36.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.389+0000 7fe7b4c00640 1 -- 172.21.15.89:0/879541306 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe7b8049070 con 0x555cdb9f32a0 2024-09-10T08:44:36.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.389+0000 7fe7d062f740 10 client.26200 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:36.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.389+0000 7fe7d062f740 1 -- 172.21.15.89:0/879541306 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555cdba01b20 con 0x555cdb9f32a0 2024-09-10T08:44:36.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.389+0000 7fe7b4c00640 1 -- 172.21.15.89:0/879541306 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7b80bc740 con 0x555cdb9f32a0 2024-09-10T08:44:36.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.389+0000 7fe7b4c00640 1 -- 172.21.15.89:0/879541306 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe7b80bb500 con 0x555cdb9f32a0 2024-09-10T08:44:36.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.389+0000 7fe7d062f740 10 client.26200 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:36.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.389+0000 7fe7d062f740 10 client.26200 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:44:36.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.389+0000 7fe7d062f740 20 client.26200 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:36.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.389+0000 7fe7d062f740 1 --2- 172.21.15.89:0/879541306 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x555cdba797d0 0x555cdba7bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.389+0000 7fe7d062f740 4 client.26200 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-a577fe06-1-mds.h.json"}] 2024-09-10T08:44:36.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.389+0000 7fe7d062f740 1 -- 172.21.15.89:0/879541306 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-a577fe06-1-mds.h.json"}) -- 0x555cdb95f770 con 0x555cdba797d0 2024-09-10T08:44:36.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.390+0000 7fe7bfe00640 1 --2- 172.21.15.89:0/879541306 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x555cdba797d0 0x555cdba7bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.390+0000 7fe7b4c00640 10 client.26200 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:44:36.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.390+0000 7fe7bfe00640 1 --2- 172.21.15.89:0/879541306 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x555cdba797d0 0x555cdba7bbc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd16750a466555bf server_cookie=56fa97e16029ab80 in_seq=0 out_seq=0 2024-09-10T08:44:36.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.504+0000 7fe7b4c00640 1 -- 172.21.15.89:0/879541306 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x555cdb95f770 con 0x555cdba797d0 2024-09-10T08:44:36.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.504+0000 7fe7b4c00640 10 client.26200 handle_command_reply: tid=0 2024-09-10T08:44:36.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.504+0000 7fe7d062f740 1 client.26200 shutdown 2024-09-10T08:44:36.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.505+0000 7fe7d062f740 1 -- 172.21.15.89:0/879541306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f32a0 msgr2=0x555cdba06610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:36.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.505+0000 7fe7d062f740 1 --2- 172.21.15.89:0/879541306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f32a0 0x555cdba06610 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80b8a10 tx=0x7fe7b80bc640 comp rx=0 tx=0).stop 2024-09-10T08:44:36.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.505+0000 7fe7d062f740 1 -- 172.21.15.89:0/879541306 shutdown_connections 2024-09-10T08:44:36.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.505+0000 7fe7d062f740 1 --2- 172.21.15.89:0/879541306 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x555cdba797d0 0x555cdba7bbc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.505+0000 7fe7d062f740 1 --2- 172.21.15.89:0/879541306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555cdb9fe480 0x555cdba013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.505+0000 7fe7d062f740 1 --2- 172.21.15.89:0/879541306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555cdb9f3bc0 0x555cdba06b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.505+0000 7fe7d062f740 1 --2- 172.21.15.89:0/879541306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cdb9f32a0 0x555cdba06610 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.505+0000 7fe7d062f740 1 -- 172.21.15.89:0/879541306 >> 172.21.15.89:0/879541306 conn(0x555cdb9ef830 msgr2=0x555cdb9f5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:36.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.505+0000 7fe7b4c00640 0 client.26200 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:44:36.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.505+0000 7fe7d062f740 1 -- 172.21.15.89:0/879541306 shutdown_connections 2024-09-10T08:44:36.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.505+0000 7fe7d062f740 1 -- 172.21.15.89:0/879541306 wait complete. 2024-09-10T08:44:36.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.505+0000 7fe7d062f740 20 client.26200 trim_cache size 0 max 16384 2024-09-10T08:44:36.507 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:44:36.507 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/var/run/ceph/ops-a577fe06-1-mds.h.json" 2024-09-10T08:44:36.507 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:44:36.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.506+0000 7fe7c7e00640 1 -- 172.21.15.89:0/1459724435 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe7ac0a2b80 msgr2=0x7fe7ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:36.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.506+0000 7fe7c7e00640 1 --2- 172.21.15.89:0/1459724435 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe7ac0a2b80 0x7fe7ac0a5040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80754f0 tx=0x7fe7c0009740 comp rx=0 tx=0).stop 2024-09-10T08:44:36.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.506+0000 7fe7c7e00640 1 -- 172.21.15.89:0/1459724435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7c8074510 msgr2=0x7fe7c819a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:36.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.506+0000 7fe7c7e00640 1 --2- 172.21.15.89:0/1459724435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7c8074510 0x7fe7c819a540 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8004770 tx=0x7fe7b8003c10 comp rx=0 tx=0).stop 2024-09-10T08:44:36.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.506+0000 7fe7c7e00640 1 -- 172.21.15.89:0/1459724435 shutdown_connections 2024-09-10T08:44:36.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.506+0000 7fe7c7e00640 1 --2- 172.21.15.89:0/1459724435 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe7ac0a2b80 0x7fe7ac0a5040 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.506+0000 7fe7c7e00640 1 --2- 172.21.15.89:0/1459724435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7c810e280 0x7fe7c819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.506+0000 7fe7c7e00640 1 --2- 172.21.15.89:0/1459724435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7c8074e30 0x7fe7c819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.506+0000 7fe7c7e00640 1 --2- 172.21.15.89:0/1459724435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7c8074510 0x7fe7c819a540 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.506+0000 7fe7c7e00640 1 -- 172.21.15.89:0/1459724435 >> 172.21.15.89:0/1459724435 conn(0x7fe7c80fd740 msgr2=0x7fe7c80fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:36.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.506+0000 7fe7c7e00640 1 -- 172.21.15.89:0/1459724435 shutdown_connections 2024-09-10T08:44:36.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:36.506+0000 7fe7c7e00640 1 -- 172.21.15.89:0/1459724435 wait complete. 2024-09-10T08:44:36.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:36 smithi111 ceph-mon[35220]: scrub summary: idle 2024-09-10T08:44:36.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:36 smithi111 ceph-mon[35220]: scrub summary: idle 2024-09-10T08:44:36.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:36 smithi111 ceph-mon[35220]: pgmap v533: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 40 KiB/s wr, 995 op/s 2024-09-10T08:44:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.643+0000 7f44ae200640 1 -- 172.21.15.89:0/2630787243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44a80761b0 msgr2=0x7f44a8074600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.643+0000 7f44ae200640 1 --2- 172.21.15.89:0/2630787243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44a80761b0 0x7f44a8074600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4498009900 tx=0x7f449802a4a0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.643+0000 7f44ae200640 1 -- 172.21.15.89:0/2630787243 shutdown_connections 2024-09-10T08:44:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.643+0000 7f44ae200640 1 --2- 172.21.15.89:0/2630787243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44a8109a10 0x7f44a810bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.643+0000 7f44ae200640 1 --2- 172.21.15.89:0/2630787243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44a8074bd0 0x7f44a8075030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.643+0000 7f44ae200640 1 --2- 172.21.15.89:0/2630787243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44a80761b0 0x7f44a8074600 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.643+0000 7f44ae200640 1 -- 172.21.15.89:0/2630787243 >> 172.21.15.89:0/2630787243 conn(0x7f44a80fd2e0 msgr2=0x7f44a80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.643+0000 7f44ae200640 1 -- 172.21.15.89:0/2630787243 shutdown_connections 2024-09-10T08:44:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.643+0000 7f44ae200640 1 -- 172.21.15.89:0/2630787243 wait complete. 2024-09-10T08:44:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44ae200640 1 Processor -- start 2024-09-10T08:44:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44ae200640 1 -- start start 2024-09-10T08:44:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44ae200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44a8074bd0 0x7f44a819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44ae200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44a80761b0 0x7f44a819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44ae200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44a8109a10 0x7f44a81a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44ae200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f44a810fab0 con 0x7f44a80761b0 2024-09-10T08:44:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44ae200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f44a810f930 con 0x7f44a8109a10 2024-09-10T08:44:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44ae200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f44a810fc30 con 0x7f44a8074bd0 2024-09-10T08:44:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44a7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44a8074bd0 0x7f44a819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44a6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44a80761b0 0x7f44a819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44a7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44a8074bd0 0x7f44a819a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:37006/0 (socket says 172.21.15.89:37006) 2024-09-10T08:44:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44a7400640 1 -- 172.21.15.89:0/2232348648 learned_addr learned my addr 172.21.15.89:0/2232348648 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44a7e00640 1 --2- 172.21.15.89:0/2232348648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44a8109a10 0x7f44a81a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44a7400640 1 -- 172.21.15.89:0/2232348648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44a8109a10 msgr2=0x7f44a81a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44a7400640 1 --2- 172.21.15.89:0/2232348648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44a8109a10 0x7f44a81a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44a7400640 1 -- 172.21.15.89:0/2232348648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44a80761b0 msgr2=0x7f44a819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44a7400640 1 --2- 172.21.15.89:0/2232348648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44a80761b0 0x7f44a819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44a7400640 1 -- 172.21.15.89:0/2232348648 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a81a1c90 con 0x7f44a8074bd0 2024-09-10T08:44:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44a6a00640 1 --2- 172.21.15.89:0/2232348648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44a80761b0 0x7f44a819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44a7400640 1 --2- 172.21.15.89:0/2232348648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44a8074bd0 0x7f44a819a070 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4498031440 tx=0x7f4498032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f449fe00640 1 -- 172.21.15.89:0/2232348648 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449803f030 con 0x7f44a8074bd0 2024-09-10T08:44:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f449fe00640 1 -- 172.21.15.89:0/2232348648 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4498003ce0 con 0x7f44a8074bd0 2024-09-10T08:44:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44ae200640 1 -- 172.21.15.89:0/2232348648 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a81a1f20 con 0x7f44a8074bd0 2024-09-10T08:44:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f449fe00640 1 -- 172.21.15.89:0/2232348648 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498035070 con 0x7f44a8074bd0 2024-09-10T08:44:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.644+0000 7f44ae200640 1 -- 172.21.15.89:0/2232348648 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a81a2400 con 0x7f44a8074bd0 2024-09-10T08:44:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.649+0000 7f449fe00640 1 -- 172.21.15.89:0/2232348648 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4498042030 con 0x7f44a8074bd0 2024-09-10T08:44:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 Processor -- start 2024-09-10T08:44:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 -- start start 2024-09-10T08:44:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 0x55f1737b6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737b69d0 0x55f1737bb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737bba60 0x55f1737bdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f17371b100 con 0x55f1737bba60 2024-09-10T08:44:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f1736c31b0 con 0x55f1737b69d0 2024-09-10T08:44:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f1737170c0 con 0x55f1737b6090 2024-09-10T08:44:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737b69d0 0x55f1737bb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 0x55f1737b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737b69d0 0x55f1737bb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:43228/0 (socket says 172.21.15.89:43228) 2024-09-10T08:44:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a7400640 1 -- 172.21.15.89:0/4084212160 learned_addr learned my addr 172.21.15.89:0/4084212160 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a6a00640 1 --2- 172.21.15.89:0/4084212160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737bba60 0x55f1737bdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a7400640 1 -- 172.21.15.89:0/4084212160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 msgr2=0x55f1737b6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a7400640 1 --2- 172.21.15.89:0/4084212160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 0x55f1737b6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a7400640 1 -- 172.21.15.89:0/4084212160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737bba60 msgr2=0x55f1737bdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a7400640 1 --2- 172.21.15.89:0/4084212160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737bba60 0x55f1737bdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a7400640 1 -- 172.21.15.89:0/4084212160 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17342e210 con 0x55f1737b69d0 2024-09-10T08:44:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f449fe00640 1 --2- 172.21.15.89:0/2232348648 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f448c0a2cb0 0x7f448c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a7e00640 1 --2- 172.21.15.89:0/2232348648 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f448c0a2cb0 0x7f448c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f449fe00640 1 -- 172.21.15.89:0/2232348648 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f44980ef8c0 con 0x7f44a8074bd0 2024-09-10T08:44:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a7400640 1 --2- 172.21.15.89:0/4084212160 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737b69d0 0x55f1737bb490 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f44980f5c20 tx=0x7f44980f5d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9088f078daa5cdc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f4493e00640 1 -- 172.21.15.89:0/4084212160 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498049030 con 0x55f1737b69d0 2024-09-10T08:44:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f4493e00640 1 -- 172.21.15.89:0/4084212160 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44980f5e40 con 0x55f1737b69d0 2024-09-10T08:44:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f4493e00640 1 -- 172.21.15.89:0/4084212160 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44980bbb70 con 0x55f1737b69d0 2024-09-10T08:44:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 -- 172.21.15.89:0/4084212160 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737b69d0 msgr2=0x55f1737bb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a7e00640 1 --2- 172.21.15.89:0/2232348648 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f448c0a2cb0 0x7f448c0a5170 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f44a0005fd0 tx=0x7f44a0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 --2- 172.21.15.89:0/4084212160 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737b69d0 0x55f1737bb490 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f44980f5c20 tx=0x7f44980f5d10 comp rx=0 tx=0).stop 2024-09-10T08:44:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 -- 172.21.15.89:0/4084212160 shutdown_connections 2024-09-10T08:44:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 --2- 172.21.15.89:0/4084212160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737bba60 0x55f1737bdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 --2- 172.21.15.89:0/4084212160 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737b69d0 0x55f1737bb490 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 --2- 172.21.15.89:0/4084212160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 0x55f1737b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 -- 172.21.15.89:0/4084212160 >> 172.21.15.89:0/4084212160 conn(0x55f1737b1700 msgr2=0x55f1737b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 -- 172.21.15.89:0/4084212160 shutdown_connections 2024-09-10T08:44:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 -- 172.21.15.89:0/4084212160 wait complete. 2024-09-10T08:44:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 -- start start 2024-09-10T08:44:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 Processor -- start 2024-09-10T08:44:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 0x55f1737bff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737b69d0 0x55f1737c04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737bba60 0x55f1737c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f1737be7f0 con 0x55f1737b69d0 2024-09-10T08:44:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f1736a4090 con 0x55f1737bba60 2024-09-10T08:44:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f1737be970 con 0x55f1737b6090 2024-09-10T08:44:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 10 client.0 _ll_register_callbacks cb 0x55f1736f8f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737b69d0 0x55f1737c04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737b69d0 0x55f1737c04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38546/0 (socket says 172.21.15.89:38546) 2024-09-10T08:44:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a6a00640 1 -- 172.21.15.89:0/3775306448 learned_addr learned my addr 172.21.15.89:0/3775306448 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a7e00640 1 --2- 172.21.15.89:0/3775306448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737bba60 0x55f1737c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a7400640 1 --2- 172.21.15.89:0/3775306448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 0x55f1737bff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a6a00640 1 -- 172.21.15.89:0/3775306448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 msgr2=0x55f1737bff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a6a00640 1 --2- 172.21.15.89:0/3775306448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 0x55f1737bff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a6a00640 1 -- 172.21.15.89:0/3775306448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737bba60 msgr2=0x55f1737c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a6a00640 1 --2- 172.21.15.89:0/3775306448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737bba60 0x55f1737c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a6a00640 1 -- 172.21.15.89:0/3775306448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1734fa620 con 0x55f1737b69d0 2024-09-10T08:44:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a7e00640 1 --2- 172.21.15.89:0/3775306448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737bba60 0x55f1737c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44a6a00640 1 --2- 172.21.15.89:0/3775306448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737b69d0 0x55f1737c04a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f44940114e0 tx=0x7f44940119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75046c911701a53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f4490c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:44:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f4490c00640 1 -- 172.21.15.89:0/3775306448 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494015870 con 0x55f1737b69d0 2024-09-10T08:44:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f44b109d740 1 -- 172.21.15.89:0/3775306448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f173728830 con 0x55f1737b69d0 2024-09-10T08:44:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.650+0000 7f4490c00640 1 -- 172.21.15.89:0/3775306448 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f4494015a10 con 0x55f1737b69d0 2024-09-10T08:44:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.656+0000 7f4490c00640 1 -- 172.21.15.89:0/3775306448 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4494026050 con 0x55f1737b69d0 2024-09-10T08:44:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.656+0000 7f44b109d740 10 client.16245 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.656+0000 7f44b109d740 1 -- 172.21.15.89:0/3775306448 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1737ca3f0 con 0x55f1737b69d0 2024-09-10T08:44:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.656+0000 7f4490c00640 1 -- 172.21.15.89:0/3775306448 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f173728830 con 0x55f1737b69d0 2024-09-10T08:44:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.656+0000 7f4490c00640 1 -- 172.21.15.89:0/3775306448 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4494013400 con 0x55f1737b69d0 2024-09-10T08:44:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.656+0000 7f44b109d740 10 client.16245 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.656+0000 7f44b109d740 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:44:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.656+0000 7f44b109d740 20 client.16245 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.656+0000 7f44b109d740 1 --2- 172.21.15.89:0/3775306448 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f1737cbb40 0x55f1737cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.656+0000 7f44b109d740 4 client.16245 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:44:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.656+0000 7f44b109d740 1 -- 172.21.15.89:0/3775306448 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f173722b80 con 0x55f1737cbb40 2024-09-10T08:44:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.673+0000 7f44a7400640 1 --2- 172.21.15.89:0/3775306448 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f1737cbb40 0x55f1737cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:36 smithi149 ceph-mon[36167]: scrub summary: idle 2024-09-10T08:44:36.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:36 smithi149 ceph-mon[36167]: scrub summary: idle 2024-09-10T08:44:36.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:36 smithi149 ceph-mon[36167]: pgmap v533: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 40 KiB/s wr, 995 op/s 2024-09-10T08:44:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.732+0000 7f44a7400640 1 --2- 172.21.15.89:0/3775306448 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f1737cbb40 0x55f1737cdfa0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cd01ddfb373567e server_cookie=d4bd9b92056e18e9 in_seq=0 out_seq=0 2024-09-10T08:44:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.732+0000 7f4490c00640 10 client.16245 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.755+0000 7f4490c00640 1 -- 172.21.15.89:0/3775306448 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f173722b80 con 0x55f1737cbb40 2024-09-10T08:44:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.755+0000 7f4490c00640 10 client.16245 handle_command_reply: tid=0 2024-09-10T08:44:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.755+0000 7f44b109d740 1 client.16245 shutdown 2024-09-10T08:44:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.755+0000 7f44b109d740 1 -- 172.21.15.89:0/3775306448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737b69d0 msgr2=0x55f1737c04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.755+0000 7f44b109d740 1 --2- 172.21.15.89:0/3775306448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737b69d0 0x55f1737c04a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f44940114e0 tx=0x7f44940119a0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.755+0000 7f44b109d740 1 -- 172.21.15.89:0/3775306448 shutdown_connections 2024-09-10T08:44:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.755+0000 7f44b109d740 1 --2- 172.21.15.89:0/3775306448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737bba60 0x55f1737c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.755+0000 7f44b109d740 1 --2- 172.21.15.89:0/3775306448 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f1737cbb40 0x55f1737cdfa0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.755+0000 7f44b109d740 1 --2- 172.21.15.89:0/3775306448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737b69d0 0x55f1737c04a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.755+0000 7f44b109d740 1 --2- 172.21.15.89:0/3775306448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 0x55f1737bff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.755+0000 7f4490c00640 0 client.16245 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.755+0000 7f44b109d740 1 -- 172.21.15.89:0/3775306448 >> 172.21.15.89:0/3775306448 conn(0x55f1737b1700 msgr2=0x55f1737b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.755+0000 7f44b109d740 1 -- 172.21.15.89:0/3775306448 shutdown_connections 2024-09-10T08:44:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.755+0000 7f44b109d740 1 -- 172.21.15.89:0/3775306448 wait complete. 2024-09-10T08:44:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.755+0000 7f44b109d740 20 client.16245 trim_cache size 0 max 16384 2024-09-10T08:44:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.767+0000 7f44b109d740 1 Processor -- start 2024-09-10T08:44:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.767+0000 7f44b109d740 1 -- start start 2024-09-10T08:44:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.767+0000 7f44b109d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 0x55f1737b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.767+0000 7f44b109d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737b69d0 0x55f1737c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.767+0000 7f44b109d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737bba60 0x55f1737b2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.767+0000 7f44b109d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f44a81af2d0 con 0x55f1737b69d0 2024-09-10T08:44:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.767+0000 7f44b109d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f44a81acba0 con 0x55f1737bba60 2024-09-10T08:44:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.767+0000 7f44b109d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f44a81aa9f0 con 0x55f1737b6090 2024-09-10T08:44:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.767+0000 7f44a6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737b69d0 0x55f1737c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.767+0000 7f44a6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737b69d0 0x55f1737c74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38552/0 (socket says 172.21.15.89:38552) 2024-09-10T08:44:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.767+0000 7f44a6a00640 1 -- 172.21.15.89:0/4158261294 learned_addr learned my addr 172.21.15.89:0/4158261294 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.767+0000 7f44a7400640 1 --2- 172.21.15.89:0/4158261294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 0x55f1737b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.767+0000 7f44a7e00640 1 --2- 172.21.15.89:0/4158261294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737bba60 0x55f1737b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44a6a00640 1 -- 172.21.15.89:0/4158261294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 msgr2=0x55f1737b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44a6a00640 1 --2- 172.21.15.89:0/4158261294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 0x55f1737b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44a6a00640 1 -- 172.21.15.89:0/4158261294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737bba60 msgr2=0x55f1737b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44a6a00640 1 --2- 172.21.15.89:0/4158261294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737bba60 0x55f1737b2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44a6a00640 1 -- 172.21.15.89:0/4158261294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1737befd0 con 0x55f1737b69d0 2024-09-10T08:44:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44a7400640 1 --2- 172.21.15.89:0/4158261294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 0x55f1737b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44a7e00640 1 --2- 172.21.15.89:0/4158261294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737bba60 0x55f1737b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44a6a00640 1 --2- 172.21.15.89:0/4158261294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737b69d0 0x55f1737c74a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4494013850 tx=0x7f4494004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff342b717774d19 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f4493400640 1 -- 172.21.15.89:0/4158261294 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494030020 con 0x55f1737b69d0 2024-09-10T08:44:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f4493400640 1 -- 172.21.15.89:0/4158261294 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4494013400 con 0x55f1737b69d0 2024-09-10T08:44:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f4493400640 1 -- 172.21.15.89:0/4158261294 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449402c6a0 con 0x55f1737b69d0 2024-09-10T08:44:36.778 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 -- 172.21.15.89:0/4158261294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737b69d0 msgr2=0x55f1737c74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 --2- 172.21.15.89:0/4158261294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737b69d0 0x55f1737c74a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4494013850 tx=0x7f4494004740 comp rx=0 tx=0).stop 2024-09-10T08:44:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 -- 172.21.15.89:0/4158261294 shutdown_connections 2024-09-10T08:44:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 --2- 172.21.15.89:0/4158261294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737bba60 0x55f1737b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 --2- 172.21.15.89:0/4158261294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737b69d0 0x55f1737c74a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 --2- 172.21.15.89:0/4158261294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 0x55f1737b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 -- 172.21.15.89:0/4158261294 >> 172.21.15.89:0/4158261294 conn(0x55f1737b1700 msgr2=0x55f1737b7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 -- 172.21.15.89:0/4158261294 shutdown_connections 2024-09-10T08:44:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 -- 172.21.15.89:0/4158261294 wait complete. 2024-09-10T08:44:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 -- start start 2024-09-10T08:44:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 Processor -- start 2024-09-10T08:44:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 0x55f1737c4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737b69d0 0x55f1737c4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737bba60 0x55f173834830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f44a81ab060 con 0x55f1737bba60 2024-09-10T08:44:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f44a81ac4d0 con 0x55f1737b69d0 2024-09-10T08:44:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f44a81ab730 con 0x55f1737b6090 2024-09-10T08:44:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 10 client.0 _ll_register_callbacks cb 0x7f4494009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44a7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737bba60 0x55f173834830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44a7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737bba60 0x55f173834830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38564/0 (socket says 172.21.15.89:38564) 2024-09-10T08:44:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44a7e00640 1 -- 172.21.15.89:0/3199591638 learned_addr learned my addr 172.21.15.89:0/3199591638 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44a7400640 1 --2- 172.21.15.89:0/3199591638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 0x55f1737c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44a6a00640 1 --2- 172.21.15.89:0/3199591638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737b69d0 0x55f1737c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44a7400640 1 -- 172.21.15.89:0/3199591638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737b69d0 msgr2=0x55f1737c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44a7400640 1 --2- 172.21.15.89:0/3199591638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737b69d0 0x55f1737c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44a7400640 1 -- 172.21.15.89:0/3199591638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737bba60 msgr2=0x55f173834830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44a7400640 1 --2- 172.21.15.89:0/3199591638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737bba60 0x55f173834830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44a7400640 1 -- 172.21.15.89:0/3199591638 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1737c7710 con 0x55f1737b6090 2024-09-10T08:44:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44a7e00640 1 --2- 172.21.15.89:0/3199591638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737bba60 0x55f173834830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44a7400640 1 --2- 172.21.15.89:0/3199591638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 0x55f1737c4130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f44980f7670 tx=0x7f4498005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a2402be1bd1ba0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f4490c00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:44:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f4490c00640 1 -- 172.21.15.89:0/3199591638 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498049040 con 0x55f1737b6090 2024-09-10T08:44:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f4490c00640 1 -- 172.21.15.89:0/3199591638 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f449803bd60 con 0x55f1737b6090 2024-09-10T08:44:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 -- 172.21.15.89:0/3199591638 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4494013400 con 0x55f1737b6090 2024-09-10T08:44:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f4490c00640 1 -- 172.21.15.89:0/3199591638 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4498005d50 con 0x55f1737b6090 2024-09-10T08:44:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 10 client.26218 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 -- 172.21.15.89:0/3199591638 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f173835e10 con 0x55f1737b6090 2024-09-10T08:44:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f4490c00640 1 -- 172.21.15.89:0/3199591638 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4494013400 con 0x55f1737b6090 2024-09-10T08:44:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f4490c00640 1 -- 172.21.15.89:0/3199591638 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4498003800 con 0x55f1737b6090 2024-09-10T08:44:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 10 client.26218 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 10 client.26218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:44:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 20 client.26218 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 --2- 172.21.15.89:0/3199591638 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f173836440 0x55f173838830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 4 client.26218 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:44:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.773+0000 7f44b109d740 1 -- 172.21.15.89:0/3199591638 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f17369f7c0 con 0x55f173836440 2024-09-10T08:44:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.778+0000 7f44a6a00640 1 --2- 172.21.15.89:0/3199591638 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f173836440 0x55f173838830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.778+0000 7f44a6a00640 1 --2- 172.21.15.89:0/3199591638 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f173836440 0x55f173838830 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2c2deb9e4505452 server_cookie=72f2d209fbad930b in_seq=0 out_seq=0 2024-09-10T08:44:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.778+0000 7f4490c00640 10 client.26218 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f4490c00640 1 -- 172.21.15.89:0/3199591638 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f17369f7c0 con 0x55f173836440 2024-09-10T08:44:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f4490c00640 10 client.26218 handle_command_reply: tid=0 2024-09-10T08:44:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44b109d740 1 client.26218 shutdown 2024-09-10T08:44:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44b109d740 1 -- 172.21.15.89:0/3199591638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 msgr2=0x55f1737c4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44b109d740 1 --2- 172.21.15.89:0/3199591638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 0x55f1737c4130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f44980f7670 tx=0x7f4498005c90 comp rx=0 tx=0).stop 2024-09-10T08:44:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44b109d740 1 -- 172.21.15.89:0/3199591638 shutdown_connections 2024-09-10T08:44:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44b109d740 1 --2- 172.21.15.89:0/3199591638 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55f173836440 0x55f173838830 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44b109d740 1 --2- 172.21.15.89:0/3199591638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f1737bba60 0x55f173834830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44b109d740 1 --2- 172.21.15.89:0/3199591638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1737b69d0 0x55f1737c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f4490c00640 0 client.26218 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44b109d740 1 --2- 172.21.15.89:0/3199591638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1737b6090 0x55f1737c4130 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44b109d740 1 -- 172.21.15.89:0/3199591638 >> 172.21.15.89:0/3199591638 conn(0x55f1737b1700 msgr2=0x55f1737b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44b109d740 1 -- 172.21.15.89:0/3199591638 shutdown_connections 2024-09-10T08:44:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44b109d740 1 -- 172.21.15.89:0/3199591638 wait complete. 2024-09-10T08:44:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44b109d740 20 client.26218 trim_cache size 0 max 16384 2024-09-10T08:44:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:44:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:44:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:44:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:44:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44ae200640 1 -- 172.21.15.89:0/2232348648 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f448c0a2cb0 msgr2=0x7f448c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44ae200640 1 --2- 172.21.15.89:0/2232348648 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f448c0a2cb0 0x7f448c0a5170 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f44a0005fd0 tx=0x7f44a0009500 comp rx=0 tx=0).stop 2024-09-10T08:44:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44ae200640 1 -- 172.21.15.89:0/2232348648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44a8074bd0 msgr2=0x7f44a819a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44ae200640 1 --2- 172.21.15.89:0/2232348648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44a8074bd0 0x7f44a819a070 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4498031440 tx=0x7f4498032ce0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44ae200640 1 -- 172.21.15.89:0/2232348648 shutdown_connections 2024-09-10T08:44:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44ae200640 1 --2- 172.21.15.89:0/2232348648 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f448c0a2cb0 0x7f448c0a5170 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44ae200640 1 --2- 172.21.15.89:0/2232348648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44a8109a10 0x7f44a81a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44ae200640 1 --2- 172.21.15.89:0/2232348648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44a80761b0 0x7f44a819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44ae200640 1 --2- 172.21.15.89:0/2232348648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44a8074bd0 0x7f44a819a070 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44ae200640 1 -- 172.21.15.89:0/2232348648 >> 172.21.15.89:0/2232348648 conn(0x7f44a80fd2e0 msgr2=0x7f44a81059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44ae200640 1 -- 172.21.15.89:0/2232348648 shutdown_connections 2024-09-10T08:44:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:36.791+0000 7f44ae200640 1 -- 172.21.15.89:0/2232348648 wait complete. 2024-09-10T08:44:37.851 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 0 (b) to a remote file /var/run/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ops-a577fe06-1-mds.b.json 2024-09-10T08:44:37.852 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 ops --flags=locks --path=/var/run/ceph/ops-a577fe06-1-mds.b.json 2024-09-10T08:44:38.042 INFO:tasks.workunit.client.0.smithi089.stdout: 87 0 0 0 0 0 0 2024-09-10T08:44:38.229 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:38.894 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:38 smithi089 ceph-mon[30757]: pgmap v534: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 48 KiB/s wr, 1.10k op/s 2024-09-10T08:44:38.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.897+0000 7f9d94c00640 1 -- 172.21.15.89:0/1323220688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d901079d0 msgr2=0x7f9d90107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:38.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.897+0000 7f9d94c00640 1 --2- 172.21.15.89:0/1323220688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d901079d0 0x7f9d90107db0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9d80009cd0 tx=0x7f9d8002a8e0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.897+0000 7f9d94c00640 1 -- 172.21.15.89:0/1323220688 shutdown_connections 2024-09-10T08:44:38.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.897+0000 7f9d94c00640 1 --2- 172.21.15.89:0/1323220688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d90102370 0x7f9d9010e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.897+0000 7f9d94c00640 1 --2- 172.21.15.89:0/1323220688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d901019d0 0x7f9d90101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.897+0000 7f9d94c00640 1 --2- 172.21.15.89:0/1323220688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d901079d0 0x7f9d90107db0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.897+0000 7f9d94c00640 1 -- 172.21.15.89:0/1323220688 >> 172.21.15.89:0/1323220688 conn(0x7f9d900fd710 msgr2=0x7f9d900ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:38.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.897+0000 7f9d94c00640 1 -- 172.21.15.89:0/1323220688 shutdown_connections 2024-09-10T08:44:38.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.897+0000 7f9d94c00640 1 -- 172.21.15.89:0/1323220688 wait complete. 2024-09-10T08:44:38.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d94c00640 1 Processor -- start 2024-09-10T08:44:38.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d94c00640 1 -- start start 2024-09-10T08:44:38.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d94c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d901019d0 0x7f9d9019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:38.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d94c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d90102370 0x7f9d9019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:38.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d94c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d901079d0 0x7f9d9019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:38.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d94c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9d90111e20 con 0x7f9d901019d0 2024-09-10T08:44:38.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d94c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9d90111ca0 con 0x7f9d90102370 2024-09-10T08:44:38.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d94c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9d90111fa0 con 0x7f9d901079d0 2024-09-10T08:44:38.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d8f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d901019d0 0x7f9d9019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:38.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d8f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d901019d0 0x7f9d9019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38584/0 (socket says 172.21.15.89:38584) 2024-09-10T08:44:38.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d8f400640 1 -- 172.21.15.89:0/1087519679 learned_addr learned my addr 172.21.15.89:0/1087519679 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:38.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d86a00640 1 --2- 172.21.15.89:0/1087519679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d90102370 0x7f9d9019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:38.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d8f400640 1 -- 172.21.15.89:0/1087519679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d901079d0 msgr2=0x7f9d9019ed90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T08:44:38.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d8f400640 1 --2- 172.21.15.89:0/1087519679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d901079d0 0x7f9d9019ed90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d8f400640 1 -- 172.21.15.89:0/1087519679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d90102370 msgr2=0x7f9d9019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:38.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d8f400640 1 --2- 172.21.15.89:0/1087519679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d90102370 0x7f9d9019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d8f400640 1 -- 172.21.15.89:0/1087519679 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9019f470 con 0x7f9d901019d0 2024-09-10T08:44:38.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d86a00640 1 --2- 172.21.15.89:0/1087519679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d90102370 0x7f9d9019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:38.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d8f400640 1 --2- 172.21.15.89:0/1087519679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d901019d0 0x7f9d9019a4c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9d80033040 tx=0x7f9d800093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:38.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d8cc00640 1 -- 172.21.15.89:0/1087519679 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80004040 con 0x7f9d901019d0 2024-09-10T08:44:38.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d8cc00640 1 -- 172.21.15.89:0/1087519679 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d800046c0 con 0x7f9d901019d0 2024-09-10T08:44:38.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d8cc00640 1 -- 172.21.15.89:0/1087519679 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80030ec0 con 0x7f9d901019d0 2024-09-10T08:44:38.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d94c00640 1 -- 172.21.15.89:0/1087519679 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9019f700 con 0x7f9d901019d0 2024-09-10T08:44:38.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.903+0000 7f9d94c00640 1 -- 172.21.15.89:0/1087519679 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d901a7040 con 0x7f9d901019d0 2024-09-10T08:44:38.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.908+0000 7f9d8cc00640 1 -- 172.21.15.89:0/1087519679 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9d800305a0 con 0x7f9d901019d0 2024-09-10T08:44:38.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.908+0000 7f9d98dc4740 1 Processor -- start 2024-09-10T08:44:38.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.908+0000 7f9d98dc4740 1 -- start start 2024-09-10T08:44:38.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.908+0000 7f9d98dc4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55837040c2a0 0x55837040c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:38.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.908+0000 7f9d98dc4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040cbc0 0x558370416f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:38.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.908+0000 7f9d98dc4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558370417480 0x558370419870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:38.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.908+0000 7f9d98dc4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558370370cf0 con 0x558370417480 2024-09-10T08:44:38.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.908+0000 7f9d98dc4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558370318680 con 0x55837040cbc0 2024-09-10T08:44:38.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.908+0000 7f9d98dc4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55837036ccb0 con 0x55837040c2a0 2024-09-10T08:44:38.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d86a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558370417480 0x558370419870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:38.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d8f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040cbc0 0x558370416f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:38.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d8fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55837040c2a0 0x55837040c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:38.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d8fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55837040c2a0 0x55837040c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:37060/0 (socket says 172.21.15.89:37060) 2024-09-10T08:44:38.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d8fe00640 1 -- 172.21.15.89:0/362741254 learned_addr learned my addr 172.21.15.89:0/362741254 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:38.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d86a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558370417480 0x558370419870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38600/0 (socket says 172.21.15.89:38600) 2024-09-10T08:44:38.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d86a00640 1 -- 172.21.15.89:0/362741254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55837040c2a0 msgr2=0x55837040c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:38.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d86a00640 1 --2- 172.21.15.89:0/362741254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55837040c2a0 0x55837040c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d86a00640 1 -- 172.21.15.89:0/362741254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040cbc0 msgr2=0x558370416f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:38.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d86a00640 1 --2- 172.21.15.89:0/362741254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040cbc0 0x558370416f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d86a00640 1 -- 172.21.15.89:0/362741254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558370324b70 con 0x558370417480 2024-09-10T08:44:38.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d8cc00640 1 --2- 172.21.15.89:0/1087519679 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9d740a2da0 0x7f9d740a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:38.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d8fe00640 1 --2- 172.21.15.89:0/362741254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55837040c2a0 0x55837040c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:38.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d8cc00640 1 -- 172.21.15.89:0/1087519679 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f9d8002e030 con 0x7f9d901019d0 2024-09-10T08:44:38.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d86a00640 1 --2- 172.21.15.89:0/362741254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558370417480 0x558370419870 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009fa0 tx=0x7f9d780118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3a91c73589a804 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:38.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d8fe00640 1 --2- 172.21.15.89:0/1087519679 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9d740a2da0 0x7f9d740a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:38.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d7fe00640 1 -- 172.21.15.89:0/362741254 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78004310 con 0x558370417480 2024-09-10T08:44:38.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d7fe00640 1 -- 172.21.15.89:0/362741254 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d780044b0 con 0x558370417480 2024-09-10T08:44:38.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d7fe00640 1 -- 172.21.15.89:0/362741254 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78003650 con 0x558370417480 2024-09-10T08:44:38.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d98dc4740 1 -- 172.21.15.89:0/362741254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558370417480 msgr2=0x558370419870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:38.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d98dc4740 1 --2- 172.21.15.89:0/362741254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558370417480 0x558370419870 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009fa0 tx=0x7f9d780118d0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d8fe00640 1 --2- 172.21.15.89:0/1087519679 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9d740a2da0 0x7f9d740a5260 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009de0 tx=0x7f9d88005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:38.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d98dc4740 1 -- 172.21.15.89:0/362741254 shutdown_connections 2024-09-10T08:44:38.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d98dc4740 1 --2- 172.21.15.89:0/362741254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558370417480 0x558370419870 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d98dc4740 1 --2- 172.21.15.89:0/362741254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040cbc0 0x558370416f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d98dc4740 1 --2- 172.21.15.89:0/362741254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55837040c2a0 0x55837040c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d98dc4740 1 -- 172.21.15.89:0/362741254 >> 172.21.15.89:0/362741254 conn(0x558370408830 msgr2=0x558370408c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:38.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d98dc4740 1 -- 172.21.15.89:0/362741254 shutdown_connections 2024-09-10T08:44:38.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d98dc4740 1 -- 172.21.15.89:0/362741254 wait complete. 2024-09-10T08:44:38.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d98dc4740 1 -- start start 2024-09-10T08:44:38.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d98dc4740 1 Processor -- start 2024-09-10T08:44:38.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d98dc4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55837040c2a0 0x558370416ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:38.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d98dc4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040cbc0 0x558370414170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:38.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d98dc4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558370417480 0x558370414900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:38.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d98dc4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55837041a050 con 0x55837040c2a0 2024-09-10T08:44:38.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d98dc4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5583702f97b0 con 0x55837040cbc0 2024-09-10T08:44:38.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d98dc4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55837041a1d0 con 0x558370417480 2024-09-10T08:44:38.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d98dc4740 10 client.0 _ll_register_callbacks cb 0x5583702f4790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:38.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d8f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55837040c2a0 0x558370416ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:38.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d8f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55837040c2a0 0x558370416ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38610/0 (socket says 172.21.15.89:38610) 2024-09-10T08:44:38.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d8f400640 1 -- 172.21.15.89:0/3320111008 learned_addr learned my addr 172.21.15.89:0/3320111008 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:38.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.909+0000 7f9d8fe00640 1 --2- 172.21.15.89:0/3320111008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558370417480 0x558370414900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:38.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d8f400640 1 -- 172.21.15.89:0/3320111008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558370417480 msgr2=0x558370414900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:38.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d8f400640 1 --2- 172.21.15.89:0/3320111008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558370417480 0x558370414900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d8f400640 1 -- 172.21.15.89:0/3320111008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040cbc0 msgr2=0x558370414170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:44:38.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d8f400640 1 --2- 172.21.15.89:0/3320111008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040cbc0 0x558370414170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d8f400640 1 -- 172.21.15.89:0/3320111008 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558370145490 con 0x55837040c2a0 2024-09-10T08:44:38.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d8fe00640 1 --2- 172.21.15.89:0/3320111008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558370417480 0x558370414900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:38.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d8f400640 1 --2- 172.21.15.89:0/3320111008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55837040c2a0 0x558370416ec0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9d800bcb70 tx=0x7f9d80004930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f884168f707ae0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:38.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:44:38.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d7cc00640 1 -- 172.21.15.89:0/3320111008 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d800bcc40 con 0x55837040c2a0 2024-09-10T08:44:38.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d7cc00640 1 -- 172.21.15.89:0/3320111008 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f9d800bc4c0 con 0x55837040c2a0 2024-09-10T08:44:38.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 1 -- 172.21.15.89:0/3320111008 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d780044b0 con 0x55837040c2a0 2024-09-10T08:44:38.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d7cc00640 1 -- 172.21.15.89:0/3320111008 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9d800bc7d0 con 0x55837040c2a0 2024-09-10T08:44:38.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 10 client.16263 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:38.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 1 -- 172.21.15.89:0/3320111008 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558370425b00 con 0x55837040c2a0 2024-09-10T08:44:38.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d7cc00640 1 -- 172.21.15.89:0/3320111008 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d780044b0 con 0x55837040c2a0 2024-09-10T08:44:38.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d7cc00640 1 -- 172.21.15.89:0/3320111008 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9d8004e070 con 0x55837040c2a0 2024-09-10T08:44:38.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 10 client.16263 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:38.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 10 client.16263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:44:38.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 20 client.16263 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:38.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 1 --2- 172.21.15.89:0/3320111008 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x558370426d20 0x5583704291e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:38.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 4 client.16263 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:44:38.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 1 -- 172.21.15.89:0/3320111008 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55837037e420 con 0x558370426d20 2024-09-10T08:44:38.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d86a00640 1 --2- 172.21.15.89:0/3320111008 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x558370426d20 0x5583704291e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:38.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d86a00640 1 --2- 172.21.15.89:0/3320111008 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x558370426d20 0x5583704291e0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8945f3cef9fdbf30 server_cookie=c2bf027d89804907 in_seq=0 out_seq=0 2024-09-10T08:44:38.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d7cc00640 10 client.16263 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:38.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d7cc00640 1 -- 172.21.15.89:0/3320111008 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55837037e420 con 0x558370426d20 2024-09-10T08:44:38.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d7cc00640 10 client.16263 handle_command_reply: tid=0 2024-09-10T08:44:38.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 1 client.16263 shutdown 2024-09-10T08:44:38.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 1 -- 172.21.15.89:0/3320111008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55837040c2a0 msgr2=0x558370416ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:38.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 1 --2- 172.21.15.89:0/3320111008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55837040c2a0 0x558370416ec0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9d800bcb70 tx=0x7f9d80004930 comp rx=0 tx=0).stop 2024-09-10T08:44:38.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 1 -- 172.21.15.89:0/3320111008 shutdown_connections 2024-09-10T08:44:38.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 1 --2- 172.21.15.89:0/3320111008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558370417480 0x558370414900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 1 --2- 172.21.15.89:0/3320111008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040cbc0 0x558370414170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 1 --2- 172.21.15.89:0/3320111008 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x558370426d20 0x5583704291e0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 1 --2- 172.21.15.89:0/3320111008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55837040c2a0 0x558370416ec0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 1 -- 172.21.15.89:0/3320111008 >> 172.21.15.89:0/3320111008 conn(0x558370408830 msgr2=0x558370407b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:38.932 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d7cc00640 0 client.16263 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:38.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 1 -- 172.21.15.89:0/3320111008 shutdown_connections 2024-09-10T08:44:38.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 1 -- 172.21.15.89:0/3320111008 wait complete. 2024-09-10T08:44:38.933 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.914+0000 7f9d98dc4740 20 client.16263 trim_cache size 0 max 16384 2024-09-10T08:44:38.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d98dc4740 1 Processor -- start 2024-09-10T08:44:38.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d98dc4740 1 -- start start 2024-09-10T08:44:38.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d98dc4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55837040c2a0 0x558370412a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:38.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d98dc4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040cbc0 0x558370412f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:38.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d98dc4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558370417480 0x55837042eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:38.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d98dc4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9d901b3600 con 0x558370417480 2024-09-10T08:44:38.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d98dc4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9d901b0ed0 con 0x55837040cbc0 2024-09-10T08:44:38.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d98dc4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9d901aed20 con 0x55837040c2a0 2024-09-10T08:44:38.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d86a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040cbc0 0x558370412f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:38.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d8f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55837040c2a0 0x558370412a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:38.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d8fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558370417480 0x55837042eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:38.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d86a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040cbc0 0x558370412f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:43296/0 (socket says 172.21.15.89:43296) 2024-09-10T08:44:38.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d86a00640 1 -- 172.21.15.89:0/4089929433 learned_addr learned my addr 172.21.15.89:0/4089929433 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:38.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d86a00640 1 -- 172.21.15.89:0/4089929433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55837040c2a0 msgr2=0x558370412a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:38.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d86a00640 1 --2- 172.21.15.89:0/4089929433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55837040c2a0 0x558370412a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d86a00640 1 -- 172.21.15.89:0/4089929433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558370417480 msgr2=0x55837042eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:38.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d86a00640 1 --2- 172.21.15.89:0/4089929433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558370417480 0x55837042eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d86a00640 1 -- 172.21.15.89:0/4089929433 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558370414c70 con 0x55837040cbc0 2024-09-10T08:44:38.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d8f400640 1 --2- 172.21.15.89:0/4089929433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55837040c2a0 0x558370412a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:38.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d86a00640 1 --2- 172.21.15.89:0/4089929433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040cbc0 0x558370412f80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9d78004be0 tx=0x7f9d78004c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b4564b00496e127 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:38.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d7f400640 1 -- 172.21.15.89:0/4089929433 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7801d040 con 0x55837040cbc0 2024-09-10T08:44:38.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d7f400640 1 -- 172.21.15.89:0/4089929433 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d78029070 con 0x55837040cbc0 2024-09-10T08:44:38.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d7f400640 1 -- 172.21.15.89:0/4089929433 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d780035f0 con 0x55837040cbc0 2024-09-10T08:44:38.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d98dc4740 1 -- 172.21.15.89:0/4089929433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040cbc0 msgr2=0x558370412f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:38.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d98dc4740 1 --2- 172.21.15.89:0/4089929433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040cbc0 0x558370412f80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9d78004be0 tx=0x7f9d78004c10 comp rx=0 tx=0).stop 2024-09-10T08:44:38.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d98dc4740 1 -- 172.21.15.89:0/4089929433 shutdown_connections 2024-09-10T08:44:38.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d98dc4740 1 --2- 172.21.15.89:0/4089929433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558370417480 0x55837042eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d98dc4740 1 --2- 172.21.15.89:0/4089929433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040cbc0 0x558370412f80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d98dc4740 1 --2- 172.21.15.89:0/4089929433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55837040c2a0 0x558370412a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d98dc4740 1 -- 172.21.15.89:0/4089929433 >> 172.21.15.89:0/4089929433 conn(0x558370408830 msgr2=0x558370403a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:38.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d98dc4740 1 -- 172.21.15.89:0/4089929433 shutdown_connections 2024-09-10T08:44:38.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d98dc4740 1 -- 172.21.15.89:0/4089929433 wait complete. 2024-09-10T08:44:38.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d98dc4740 1 -- start start 2024-09-10T08:44:38.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.938+0000 7f9d98dc4740 1 Processor -- start 2024-09-10T08:44:38.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.943+0000 7f9d98dc4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040c2a0 0x55837041f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:38.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.943+0000 7f9d98dc4740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55837040cbc0 0x55837041fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:38.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.943+0000 7f9d98dc4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558370417480 0x55837041a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:38.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.943+0000 7f9d98dc4740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9d901af390 con 0x55837040cbc0 2024-09-10T08:44:38.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.943+0000 7f9d98dc4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9d901b0800 con 0x55837040c2a0 2024-09-10T08:44:38.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.943+0000 7f9d98dc4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9d901afa60 con 0x558370417480 2024-09-10T08:44:38.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.943+0000 7f9d98dc4740 10 client.0 _ll_register_callbacks cb 0x7f9d800f6220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:38.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.943+0000 7f9d8fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558370417480 0x55837041a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:38.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.943+0000 7f9d8fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558370417480 0x55837041a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:37100/0 (socket says 172.21.15.89:37100) 2024-09-10T08:44:38.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.943+0000 7f9d8fe00640 1 -- 172.21.15.89:0/3576110082 learned_addr learned my addr 172.21.15.89:0/3576110082 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:38.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.943+0000 7f9d86a00640 1 --2- 172.21.15.89:0/3576110082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55837040cbc0 0x55837041fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:38.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.943+0000 7f9d8f400640 1 --2- 172.21.15.89:0/3576110082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040c2a0 0x55837041f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:38.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.943+0000 7f9d8fe00640 1 -- 172.21.15.89:0/3576110082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040c2a0 msgr2=0x55837041f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:38.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.943+0000 7f9d8fe00640 1 --2- 172.21.15.89:0/3576110082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040c2a0 0x55837041f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.943+0000 7f9d8fe00640 1 -- 172.21.15.89:0/3576110082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55837040cbc0 msgr2=0x55837041fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:38.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.943+0000 7f9d8fe00640 1 --2- 172.21.15.89:0/3576110082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55837040cbc0 0x55837041fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:38.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.943+0000 7f9d8fe00640 1 -- 172.21.15.89:0/3576110082 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55837041aa80 con 0x558370417480 2024-09-10T08:44:38.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.943+0000 7f9d86a00640 1 --2- 172.21.15.89:0/3576110082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55837040cbc0 0x55837041fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:38.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.944+0000 7f9d8fe00640 1 --2- 172.21.15.89:0/3576110082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558370417480 0x55837041a3e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9d8801ca00 tx=0x7f9d88004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1201fcd0b8f25b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:38.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.944+0000 7f9d7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:44:38.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.944+0000 7f9d7cc00640 1 -- 172.21.15.89:0/3576110082 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d880037f0 con 0x558370417480 2024-09-10T08:44:38.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.944+0000 7f9d7cc00640 1 -- 172.21.15.89:0/3576110082 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f9d88003990 con 0x558370417480 2024-09-10T08:44:38.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.944+0000 7f9d98dc4740 1 -- 172.21.15.89:0/3576110082 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d78029070 con 0x558370417480 2024-09-10T08:44:38.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.944+0000 7f9d7cc00640 1 -- 172.21.15.89:0/3576110082 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9d88003bd0 con 0x558370417480 2024-09-10T08:44:38.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.944+0000 7f9d98dc4740 10 client.26233 fetch_fsmap learned FSMap version 28 2024-09-10T08:44:38.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.944+0000 7f9d98dc4740 1 -- 172.21.15.89:0/3576110082 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558370492420 con 0x558370417480 2024-09-10T08:44:38.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.944+0000 7f9d7cc00640 1 -- 172.21.15.89:0/3576110082 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d78029070 con 0x558370417480 2024-09-10T08:44:38.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.944+0000 7f9d7cc00640 1 -- 172.21.15.89:0/3576110082 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9d880273f0 con 0x558370417480 2024-09-10T08:44:38.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.944+0000 7f9d98dc4740 10 client.26233 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:44:38.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.944+0000 7f9d98dc4740 10 client.26233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:44:38.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.944+0000 7f9d98dc4740 20 client.26233 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:38.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.944+0000 7f9d98dc4740 1 --2- 172.21.15.89:0/3576110082 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x558370492cf0 0x558370495110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:38.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.944+0000 7f9d98dc4740 4 client.26233 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-a577fe06-1-mds.b.json"}] 2024-09-10T08:44:38.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.944+0000 7f9d98dc4740 1 -- 172.21.15.89:0/3576110082 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-a577fe06-1-mds.b.json"}) -- 0x558370378770 con 0x558370492cf0 2024-09-10T08:44:38.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.944+0000 7f9d8f400640 1 --2- 172.21.15.89:0/3576110082 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x558370492cf0 0x558370495110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:38.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.944+0000 7f9d7cc00640 10 client.26233 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:38.949 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:38.944+0000 7f9d8f400640 1 --2- 172.21.15.89:0/3576110082 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x558370492cf0 0x558370495110 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84fa55928a2a0e7 server_cookie=7c862d692807e4f9 in_seq=0 out_seq=0 2024-09-10T08:44:39.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:38 smithi111 ceph-mon[35220]: pgmap v534: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 48 KiB/s wr, 1.10k op/s 2024-09-10T08:44:39.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:38 smithi149 ceph-mon[36167]: pgmap v534: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 48 KiB/s wr, 1.10k op/s 2024-09-10T08:44:39.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d7cc00640 1 -- 172.21.15.89:0/3576110082 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x558370378770 con 0x558370492cf0 2024-09-10T08:44:39.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d7cc00640 10 client.26233 handle_command_reply: tid=0 2024-09-10T08:44:39.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d98dc4740 1 client.26233 shutdown 2024-09-10T08:44:39.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d98dc4740 1 -- 172.21.15.89:0/3576110082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558370417480 msgr2=0x55837041a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:39.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d98dc4740 1 --2- 172.21.15.89:0/3576110082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558370417480 0x55837041a3e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9d8801ca00 tx=0x7f9d88004730 comp rx=0 tx=0).stop 2024-09-10T08:44:39.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d98dc4740 1 -- 172.21.15.89:0/3576110082 shutdown_connections 2024-09-10T08:44:39.889 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d98dc4740 1 --2- 172.21.15.89:0/3576110082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558370417480 0x55837041a3e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:39.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d98dc4740 1 --2- 172.21.15.89:0/3576110082 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x558370492cf0 0x558370495110 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:39.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d98dc4740 1 --2- 172.21.15.89:0/3576110082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55837040cbc0 0x55837041fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:39.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d98dc4740 1 --2- 172.21.15.89:0/3576110082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55837040c2a0 0x55837041f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:39.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d7cc00640 0 client.26233 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:39.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d98dc4740 1 -- 172.21.15.89:0/3576110082 >> 172.21.15.89:0/3576110082 conn(0x558370408830 msgr2=0x5583704129c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:39.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d98dc4740 1 -- 172.21.15.89:0/3576110082 shutdown_connections 2024-09-10T08:44:39.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d98dc4740 1 -- 172.21.15.89:0/3576110082 wait complete. 2024-09-10T08:44:39.890 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d98dc4740 20 client.26233 trim_cache size 0 max 16384 2024-09-10T08:44:39.890 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:44:39.890 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/var/run/ceph/ops-a577fe06-1-mds.b.json" 2024-09-10T08:44:39.891 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:44:39.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d94c00640 1 -- 172.21.15.89:0/1087519679 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9d740a2da0 msgr2=0x7f9d740a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:39.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d94c00640 1 --2- 172.21.15.89:0/1087519679 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9d740a2da0 0x7f9d740a5260 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009de0 tx=0x7f9d88005f90 comp rx=0 tx=0).stop 2024-09-10T08:44:39.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d94c00640 1 -- 172.21.15.89:0/1087519679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d901019d0 msgr2=0x7f9d9019a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:39.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d94c00640 1 --2- 172.21.15.89:0/1087519679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d901019d0 0x7f9d9019a4c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9d80033040 tx=0x7f9d800093d0 comp rx=0 tx=0).stop 2024-09-10T08:44:39.891 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d94c00640 1 -- 172.21.15.89:0/1087519679 shutdown_connections 2024-09-10T08:44:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d94c00640 1 --2- 172.21.15.89:0/1087519679 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f9d740a2da0 0x7f9d740a5260 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d94c00640 1 --2- 172.21.15.89:0/1087519679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d901079d0 0x7f9d9019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d94c00640 1 --2- 172.21.15.89:0/1087519679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d90102370 0x7f9d9019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d94c00640 1 --2- 172.21.15.89:0/1087519679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d901019d0 0x7f9d9019a4c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d94c00640 1 -- 172.21.15.89:0/1087519679 >> 172.21.15.89:0/1087519679 conn(0x7f9d900fd710 msgr2=0x7f9d900fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d94c00640 1 -- 172.21.15.89:0/1087519679 shutdown_connections 2024-09-10T08:44:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:39.888+0000 7f9d94c00640 1 -- 172.21.15.89:0/1087519679 wait complete. 2024-09-10T08:44:40.730 DEBUG:teuthology.orchestra.run.smithi149:> mktemp 2024-09-10T08:44:40.750 INFO:teuthology.orchestra.run.smithi149.stdout:/tmp/tmp.UGmRTSzWS9 2024-09-10T08:44:40.750 DEBUG:teuthology.orchestra.run.smithi149:> sudo cp /var/run/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ops-a577fe06-1-mds.i.json /tmp/tmp.UGmRTSzWS9 2024-09-10T08:44:40.823 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 0666 /tmp/tmp.UGmRTSzWS9 2024-09-10T08:44:40.827 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:40 smithi089 ceph-mon[30757]: pgmap v535: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 37 KiB/s wr, 1.09k op/s 2024-09-10T08:44:40.954 DEBUG:teuthology.orchestra.remote:smithi149:/tmp/tmp.UGmRTSzWS9 is 660KB 2024-09-10T08:44:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:40 smithi111 ceph-mon[35220]: pgmap v535: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 37 KiB/s wr, 1.09k op/s 2024-09-10T08:44:41.053 DEBUG:teuthology.orchestra.run.smithi149:> rm -fr /tmp/tmp.UGmRTSzWS9 2024-09-10T08:44:41.071 DEBUG:tasks.quiescer.fs.[cephfs]:read 676215B of ops from '/var/run/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ops-a577fe06-1-mds.i.json' on mds.2 (i) 2024-09-10T08:44:41.121 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 2 (i) for a577fe06-1: 'client_request(client.25157:73984 create owner_uid=1000, owner_gid=1333 #0x10000001106/article-44.xml.tmp 2024-09-10T08:43:42.831619+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})' 2024-09-10T08:44:41.146 INFO:tasks.quiescer.fs.[cephfs]:Pulled 240 ops from rank 2 (i) into home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898917/ops-a577fe06-1-mds.i.json 2024-09-10T08:44:41.146 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -- /var/run/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ops-a577fe06-1-mds.i.json 2024-09-10T08:44:41.169 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:40 smithi149 ceph-mon[36167]: pgmap v535: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 37 KiB/s wr, 1.09k op/s 2024-09-10T08:44:41.173 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2024-09-10T08:44:41.192 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.iindcSeuht 2024-09-10T08:44:41.193 DEBUG:teuthology.orchestra.run.smithi111:> sudo cp /var/run/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ops-a577fe06-1-mds.h.json /tmp/tmp.iindcSeuht 2024-09-10T08:44:41.269 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0666 /tmp/tmp.iindcSeuht 2024-09-10T08:44:41.400 DEBUG:teuthology.orchestra.remote:smithi111:/tmp/tmp.iindcSeuht is 4MB 2024-09-10T08:44:41.580 DEBUG:teuthology.orchestra.run.smithi111:> rm -fr /tmp/tmp.iindcSeuht 2024-09-10T08:44:41.606 DEBUG:tasks.quiescer.fs.[cephfs]:read 4399363B of ops from '/var/run/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ops-a577fe06-1-mds.h.json' on mds.1 (h) 2024-09-10T08:44:41.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (h) for a577fe06-1: 'client_request(client.25157:74410 create owner_uid=1000, owner_gid=1333 #0x10000001741/comment-29.xml.tmp 2024-09-10T08:43:44.106589+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})' 2024-09-10T08:44:42.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 1 (h) for a577fe06-1: 'client_request(client.25157:73857 rename #0x10000004cdd/article-17.xml #0x10000004cdd/article-17.xml.tmp 2024-09-10T08:43:42.341631+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})' 2024-09-10T08:44:42.285 INFO:tasks.quiescer.fs.[cephfs]:Pulled 1558 ops from rank 1 (h) into home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898917/ops-a577fe06-1-mds.h.json 2024-09-10T08:44:42.285 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -- /var/run/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ops-a577fe06-1-mds.h.json 2024-09-10T08:44:42.286 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:41 smithi149 ceph-mon[36167]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:44:42.287 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:41 smithi089 ceph-mon[30757]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:44:42.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:41 smithi111 ceph-mon[35220]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:44:42.316 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2024-09-10T08:44:42.375 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.zqxSNd5VLx 2024-09-10T08:44:42.375 DEBUG:teuthology.orchestra.run.smithi111:> sudo cp /var/run/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ops-a577fe06-1-mds.b.json /tmp/tmp.zqxSNd5VLx 2024-09-10T08:44:42.611 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0666 /tmp/tmp.zqxSNd5VLx 2024-09-10T08:44:42.702 DEBUG:teuthology.orchestra.remote:smithi111:/tmp/tmp.zqxSNd5VLx is 34MB 2024-09-10T08:44:43.072 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:42 smithi111 ceph-mon[35220]: pgmap v536: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 38 KiB/s wr, 1.10k op/s 2024-09-10T08:44:43.228 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:42 smithi149 ceph-mon[36167]: pgmap v536: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 38 KiB/s wr, 1.10k op/s 2024-09-10T08:44:43.317 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:42 smithi089 ceph-mon[30757]: pgmap v536: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 38 KiB/s wr, 1.10k op/s 2024-09-10T08:44:43.425 DEBUG:teuthology.orchestra.run.smithi111:> rm -fr /tmp/tmp.zqxSNd5VLx 2024-09-10T08:44:43.803 DEBUG:tasks.quiescer.fs.[cephfs]:read 35987479B of ops from '/var/run/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ops-a577fe06-1-mds.b.json' on mds.0 (b) 2024-09-10T08:44:44.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:267 fp=#0x10000002864' 2024-09-10T08:44:44.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:299 fp=#0x1000000233c' 2024-09-10T08:44:44.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:331 fp=#0x10000004ab3' 2024-09-10T08:44:44.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:419 fp=#0x10000004b31' 2024-09-10T08:44:44.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:451 fp=#0x10000004b4d' 2024-09-10T08:44:44.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:483 fp=#0x100000034ee' 2024-09-10T08:44:44.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:515 fp=#0x10000003331' 2024-09-10T08:44:44.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:579 fp=#0x10000003509' 2024-09-10T08:44:44.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:611 fp=#0x100000032ef' 2024-09-10T08:44:44.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:771 fp=#0x10000000d88' 2024-09-10T08:44:44.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:803 fp=#0x100000044ef' 2024-09-10T08:44:44.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:835 fp=#0x10000004559' 2024-09-10T08:44:44.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:867 fp=#0x10000004597' 2024-09-10T08:44:44.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:899 fp=#0x10000000dbd' 2024-09-10T08:44:44.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:931 fp=#0x10000000dc9' 2024-09-10T08:44:44.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:963 fp=#0x10000000de5' 2024-09-10T08:44:44.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:995 fp=#0x10000000dcf' 2024-09-10T08:44:44.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1027 fp=#0x100000044cb' 2024-09-10T08:44:44.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1123 fp=#0x1000000423a' 2024-09-10T08:44:44.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1155 fp=#0x10000004143' 2024-09-10T08:44:44.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1219 fp=#0x100000032e2' 2024-09-10T08:44:44.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1251 fp=#0x10000001ff5' 2024-09-10T08:44:44.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1283 fp=#0x1000000362b' 2024-09-10T08:44:44.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1315 fp=#0x10000002016' 2024-09-10T08:44:44.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1507 fp=#0x100000042f9' 2024-09-10T08:44:44.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1667 fp=#0x100000045bb' 2024-09-10T08:44:44.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1891 fp=#0x10000002788' 2024-09-10T08:44:44.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1955 fp=#0x100000027cc' 2024-09-10T08:44:44.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2051 fp=#0x100000021d4' 2024-09-10T08:44:44.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2083 fp=#0x10000002035' 2024-09-10T08:44:44.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2211 fp=#0x10000001ff0' 2024-09-10T08:44:44.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2307 fp=#0x100000025d8' 2024-09-10T08:44:44.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2339 fp=#0x1000000260b' 2024-09-10T08:44:44.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2563 fp=#0x10000001887' 2024-09-10T08:44:44.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2595 fp=#0x10000001048' 2024-09-10T08:44:44.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2627 fp=#0x10000003b0b' 2024-09-10T08:44:44.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2659 fp=#0x100000011cd' 2024-09-10T08:44:44.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2691 fp=#0x100000018ba' 2024-09-10T08:44:44.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2723 fp=#0x10000001d4c' 2024-09-10T08:44:44.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2755 fp=#0x10000003b7d' 2024-09-10T08:44:44.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2787 fp=#0x100000047d7' 2024-09-10T08:44:44.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2819 fp=#0x1000000345d' 2024-09-10T08:44:44.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2851 fp=#0x100000040c7' 2024-09-10T08:44:44.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2915 fp=#0x10000003666' 2024-09-10T08:44:44.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2947 fp=#0x100000035bb' 2024-09-10T08:44:44.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2979 fp=#0x1000000461a' 2024-09-10T08:44:44.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3011 fp=#0x10000004a27' 2024-09-10T08:44:44.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3043 fp=#0x100000019b0' 2024-09-10T08:44:44.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3075 fp=#0x10000000dca' 2024-09-10T08:44:44.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3107 fp=#0x10000000e86' 2024-09-10T08:44:44.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3171 fp=#0x100000010c2' 2024-09-10T08:44:44.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3203 fp=#0x1000000194a' 2024-09-10T08:44:44.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3235 fp=#0x1000000073a' 2024-09-10T08:44:44.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3267 fp=#0x1000000076e' 2024-09-10T08:44:44.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3299 fp=#0x10000003d90' 2024-09-10T08:44:44.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3331 fp=#0x1000000077b' 2024-09-10T08:44:44.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3363 fp=#0x10000004199' 2024-09-10T08:44:44.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3395 fp=#0x1000000074f' 2024-09-10T08:44:44.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3427 fp=#0x100000006fd' 2024-09-10T08:44:44.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3459 fp=#0x10000004132' 2024-09-10T08:44:44.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3523 fp=#0x10000003cc2' 2024-09-10T08:44:44.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3555 fp=#0x10000003821' 2024-09-10T08:44:44.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3651 fp=#0x10000004adc' 2024-09-10T08:44:44.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3683 fp=#0x10000004c3b' 2024-09-10T08:44:44.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3715 fp=#0x10000004b85' 2024-09-10T08:44:44.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3747 fp=#0x10000003fca' 2024-09-10T08:44:44.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3779 fp=#0x100000024f9' 2024-09-10T08:44:44.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3811 fp=#0x100000025c1' 2024-09-10T08:44:44.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3843 fp=#0x100000026c0' 2024-09-10T08:44:44.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3875 fp=#0x100000027a7' 2024-09-10T08:44:44.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3907 fp=#0x100000041bc' 2024-09-10T08:44:44.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3939 fp=#0x100000006a5' 2024-09-10T08:44:44.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3971 fp=#0x1000000065b' 2024-09-10T08:44:44.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4035 fp=#0x10000004ea0' 2024-09-10T08:44:44.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4067 fp=#0x10000000f9a' 2024-09-10T08:44:44.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4099 fp=#0x10000004e41' 2024-09-10T08:44:44.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4131 fp=#0x1000000100f' 2024-09-10T08:44:44.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4163 fp=#0x10000000fe7' 2024-09-10T08:44:44.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4195 fp=#0x10000004dc6' 2024-09-10T08:44:44.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4227 fp=#0x10000003ab1' 2024-09-10T08:44:44.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4259 fp=#0x10000003b48' 2024-09-10T08:44:44.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4291 fp=#0x100000039d8' 2024-09-10T08:44:44.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4323 fp=#0x10000003b31' 2024-09-10T08:44:44.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4355 fp=#0x1000000399e' 2024-09-10T08:44:44.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4387 fp=#0x10000001f37' 2024-09-10T08:44:44.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4483 fp=#0x100000017a5' 2024-09-10T08:44:44.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4515 fp=#0x1000000511d' 2024-09-10T08:44:44.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4547 fp=#0x100000033de' 2024-09-10T08:44:44.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4579 fp=#0x10000003839' 2024-09-10T08:44:44.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4611 fp=#0x100000038ce' 2024-09-10T08:44:44.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4643 fp=#0x10000001542' 2024-09-10T08:44:44.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4675 fp=#0x100000015bc' 2024-09-10T08:44:44.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4707 fp=#0x10000001413' 2024-09-10T08:44:44.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4739 fp=#0x1000000133f' 2024-09-10T08:44:44.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4771 fp=#0x100000012bd' 2024-09-10T08:44:44.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4803 fp=#0x1000000382b' 2024-09-10T08:44:44.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4835 fp=#0x100000037f9' 2024-09-10T08:44:44.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4867 fp=#0x10000004ab0' 2024-09-10T08:44:44.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4899 fp=#0x100000037ff' 2024-09-10T08:44:44.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4931 fp=#0x100000036e1' 2024-09-10T08:44:44.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4963 fp=#0x100000050e1' 2024-09-10T08:44:44.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4995 fp=#0x10000005110' 2024-09-10T08:44:44.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5027 fp=#0x10000005051' 2024-09-10T08:44:44.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5059 fp=#0x1000000516f' 2024-09-10T08:44:44.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5091 fp=#0x100000051b8' 2024-09-10T08:44:44.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5123 fp=#0x10000001e75' 2024-09-10T08:44:44.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5155 fp=#0x1000000417d' 2024-09-10T08:44:44.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5187 fp=#0x10000001c99' 2024-09-10T08:44:44.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5219 fp=#0x10000001cbe' 2024-09-10T08:44:44.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5251 fp=#0x100000038b7' 2024-09-10T08:44:44.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5283 fp=#0x100000038e7' 2024-09-10T08:44:44.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5315 fp=#0x10000003697' 2024-09-10T08:44:44.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5347 fp=#0x10000003d4c' 2024-09-10T08:44:44.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5379 fp=#0x1000000362c' 2024-09-10T08:44:44.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5411 fp=#0x100000037ae' 2024-09-10T08:44:44.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5443 fp=#0x10000001285' 2024-09-10T08:44:44.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5475 fp=#0x10000001254' 2024-09-10T08:44:44.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5539 fp=#0x100000041a2' 2024-09-10T08:44:44.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5571 fp=#0x100000010d5' 2024-09-10T08:44:44.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5603 fp=#0x100000010f1' 2024-09-10T08:44:44.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5635 fp=#0x100000010e7' 2024-09-10T08:44:44.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5667 fp=#0x1000000510d' 2024-09-10T08:44:44.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5699 fp=#0x10000000fcf' 2024-09-10T08:44:44.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5731 fp=#0x100000014a9' 2024-09-10T08:44:44.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5763 fp=#0x10000000fff' 2024-09-10T08:44:44.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5795 fp=#0x10000004520' 2024-09-10T08:44:44.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5827 fp=#0x10000004c81' 2024-09-10T08:44:44.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5859 fp=#0x10000004d41' 2024-09-10T08:44:44.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5891 fp=#0x10000004d75' 2024-09-10T08:44:44.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5923 fp=#0x10000004449' 2024-09-10T08:44:44.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5955 fp=#0x10000004b74' 2024-09-10T08:44:44.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6083 fp=#0x10000003580' 2024-09-10T08:44:44.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6179 fp=#0x100000034fa' 2024-09-10T08:44:44.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6211 fp=#0x1000000353e' 2024-09-10T08:44:44.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6275 fp=#0x10000001fae' 2024-09-10T08:44:44.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6307 fp=#0x10000001eb7' 2024-09-10T08:44:44.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6339 fp=#0x10000003419' 2024-09-10T08:44:44.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6371 fp=#0x10000001ed9' 2024-09-10T08:44:44.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6403 fp=#0x10000004aef' 2024-09-10T08:44:44.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6435 fp=#0x10000004c12' 2024-09-10T08:44:44.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6467 fp=#0x10000004c07' 2024-09-10T08:44:44.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6499 fp=#0x10000004b68' 2024-09-10T08:44:44.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6531 fp=#0x10000004b5b' 2024-09-10T08:44:44.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6627 fp=#0x10000004ff4' 2024-09-10T08:44:44.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6659 fp=#0x10000003b83' 2024-09-10T08:44:44.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6691 fp=#0x1000000501d' 2024-09-10T08:44:44.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6723 fp=#0x10000005433' 2024-09-10T08:44:44.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6755 fp=#0x10000003abe' 2024-09-10T08:44:44.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6787 fp=#0x10000004531' 2024-09-10T08:44:44.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6819 fp=#0x10000004961' 2024-09-10T08:44:44.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6851 fp=#0x1000000477c' 2024-09-10T08:44:44.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6883 fp=#0x1000000456a' 2024-09-10T08:44:44.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6915 fp=#0x10000004680' 2024-09-10T08:44:44.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6947 fp=#0x10000005338' 2024-09-10T08:44:44.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6979 fp=#0x1000000062c' 2024-09-10T08:44:44.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7011 fp=#0x1000000064d' 2024-09-10T08:44:44.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7043 fp=#0x10000000696' 2024-09-10T08:44:44.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7075 fp=#0x10000005064' 2024-09-10T08:44:44.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7107 fp=#0x1000000065c' 2024-09-10T08:44:44.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7139 fp=#0x10000000617' 2024-09-10T08:44:44.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7171 fp=#0x1000000068c' 2024-09-10T08:44:44.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7267 fp=#0x10000005157' 2024-09-10T08:44:44.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7395 fp=#0x10000002351' 2024-09-10T08:44:44.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7427 fp=#0x10000002397' 2024-09-10T08:44:44.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7459 fp=#0x10000004915' 2024-09-10T08:44:44.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7491 fp=#0x10000002389' 2024-09-10T08:44:44.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7523 fp=#0x100000048f6' 2024-09-10T08:44:44.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7587 fp=#0x10000001317' 2024-09-10T08:44:44.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7619 fp=#0x100000051bb' 2024-09-10T08:44:44.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7715 fp=#0x10000004f86' 2024-09-10T08:44:44.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7747 fp=#0x10000004e9c' 2024-09-10T08:44:44.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7779 fp=#0x10000004f2f' 2024-09-10T08:44:44.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7811 fp=#0x10000004cbd' 2024-09-10T08:44:44.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7843 fp=#0x100000016b6' 2024-09-10T08:44:44.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7875 fp=#0x10000004d86' 2024-09-10T08:44:44.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7907 fp=#0x10000004ed6' 2024-09-10T08:44:44.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7939 fp=#0x10000004d21' 2024-09-10T08:44:44.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7971 fp=#0x10000004d4d' 2024-09-10T08:44:44.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8003 fp=#0x1000000202a' 2024-09-10T08:44:44.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8035 fp=#0x10000001708' 2024-09-10T08:44:44.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8067 fp=#0x10000001b9d' 2024-09-10T08:44:44.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8099 fp=#0x10000001a23' 2024-09-10T08:44:44.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8131 fp=#0x1000000387b' 2024-09-10T08:44:44.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8195 fp=#0x10000004cbc' 2024-09-10T08:44:44.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8227 fp=#0x10000002621' 2024-09-10T08:44:44.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8259 fp=#0x100000043bb' 2024-09-10T08:44:44.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8291 fp=#0x1000000461b' 2024-09-10T08:44:44.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8323 fp=#0x1000000526d' 2024-09-10T08:44:44.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8355 fp=#0x1000000441e' 2024-09-10T08:44:44.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8387 fp=#0x1000000467e' 2024-09-10T08:44:44.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8419 fp=#0x1000000526b' 2024-09-10T08:44:44.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8451 fp=#0x10000003dca' 2024-09-10T08:44:44.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8483 fp=#0x10000003e0f' 2024-09-10T08:44:44.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8515 fp=#0x100000041d0' 2024-09-10T08:44:44.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8547 fp=#0x10000003e1c' 2024-09-10T08:44:44.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8579 fp=#0x1000000411f' 2024-09-10T08:44:44.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8611 fp=#0x1000000451b' 2024-09-10T08:44:44.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8643 fp=#0x10000003d38' 2024-09-10T08:44:44.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8675 fp=#0x100000052cc' 2024-09-10T08:44:44.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8707 fp=#0x100000052d1' 2024-09-10T08:44:44.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8739 fp=#0x10000004f4b' 2024-09-10T08:44:44.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8771 fp=#0x10000003e77' 2024-09-10T08:44:44.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8803 fp=#0x100000052f5' 2024-09-10T08:44:44.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8835 fp=#0x10000001b8e' 2024-09-10T08:44:44.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8867 fp=#0x100000043c9' 2024-09-10T08:44:44.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8899 fp=#0x10000004336' 2024-09-10T08:44:44.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8931 fp=#0x10000001bde' 2024-09-10T08:44:44.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8963 fp=#0x10000001c58' 2024-09-10T08:44:44.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8995 fp=#0x10000001b22' 2024-09-10T08:44:44.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9027 fp=#0x10000000dc0' 2024-09-10T08:44:44.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9059 fp=#0x1000000488d' 2024-09-10T08:44:44.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9091 fp=#0x1000000428f' 2024-09-10T08:44:44.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9123 fp=#0x10000000d54' 2024-09-10T08:44:44.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9187 fp=#0x10000000d95' 2024-09-10T08:44:44.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9219 fp=#0x10000000d10' 2024-09-10T08:44:44.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9251 fp=#0x10000000b00' 2024-09-10T08:44:44.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9283 fp=#0x10000004c78' 2024-09-10T08:44:44.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9315 fp=#0x10000001c24' 2024-09-10T08:44:44.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9347 fp=#0x10000002608' 2024-09-10T08:44:44.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9379 fp=#0x1000000253a' 2024-09-10T08:44:44.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9411 fp=#0x10000004ca4' 2024-09-10T08:44:44.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9443 fp=#0x10000000b9c' 2024-09-10T08:44:44.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9475 fp=#0x10000004c8c' 2024-09-10T08:44:44.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9507 fp=#0x1000000430e' 2024-09-10T08:44:44.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9539 fp=#0x10000000ee0' 2024-09-10T08:44:44.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9571 fp=#0x100000049ae' 2024-09-10T08:44:44.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9603 fp=#0x10000001c89' 2024-09-10T08:44:44.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9635 fp=#0x10000000ee5' 2024-09-10T08:44:44.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9667 fp=#0x100000048b9' 2024-09-10T08:44:44.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9699 fp=#0x100000047bb' 2024-09-10T08:44:44.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9731 fp=#0x10000004469' 2024-09-10T08:44:44.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9763 fp=#0x10000004434' 2024-09-10T08:44:44.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9795 fp=#0x10000001827' 2024-09-10T08:44:44.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9827 fp=#0x100000043db' 2024-09-10T08:44:44.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9859 fp=#0x1000000439f' 2024-09-10T08:44:44.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9891 fp=#0x10000003f5d' 2024-09-10T08:44:44.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9923 fp=#0x10000001617' 2024-09-10T08:44:44.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9955 fp=#0x100000016ae' 2024-09-10T08:44:44.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9987 fp=#0x100000015c2' 2024-09-10T08:44:44.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10019 fp=#0x10000001603' 2024-09-10T08:44:44.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10051 fp=#0x10000001678' 2024-09-10T08:44:44.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10083 fp=#0x10000001ecb' 2024-09-10T08:44:44.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10115 fp=#0x10000003a65' 2024-09-10T08:44:44.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10147 fp=#0x10000003978' 2024-09-10T08:44:44.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10179 fp=#0x100000039dc' 2024-09-10T08:44:44.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10211 fp=#0x10000003a00' 2024-09-10T08:44:44.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10243 fp=#0x10000003a9f' 2024-09-10T08:44:44.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10275 fp=#0x100000024d9' 2024-09-10T08:44:44.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10307 fp=#0x1000000236b' 2024-09-10T08:44:44.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10339 fp=#0x100000012e5' 2024-09-10T08:44:44.907 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10371 fp=#0x100000014db' 2024-09-10T08:44:44.907 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10403 fp=#0x100000039d9' 2024-09-10T08:44:44.907 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10435 fp=#0x10000003a82' 2024-09-10T08:44:44.908 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10467 fp=#0x100000014cf' 2024-09-10T08:44:44.908 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10499 fp=#0x1000000134c' 2024-09-10T08:44:44.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10563 fp=#0x100000039f1' 2024-09-10T08:44:44.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10595 fp=#0x10000004948' 2024-09-10T08:44:44.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10627 fp=#0x100000049a6' 2024-09-10T08:44:44.910 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10659 fp=#0x10000004909' 2024-09-10T08:44:44.910 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10723 fp=#0x100000048a4' 2024-09-10T08:44:44.910 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10755 fp=#0x10000004939' 2024-09-10T08:44:44.911 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10787 fp=#0x1000000491f' 2024-09-10T08:44:44.911 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10819 fp=#0x100000005fe' 2024-09-10T08:44:44.912 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10851 fp=#0x10000000927' 2024-09-10T08:44:44.912 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10915 fp=#0x10000000a7b' 2024-09-10T08:44:44.912 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10947 fp=#0x10000000a39' 2024-09-10T08:44:44.913 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11011 fp=#0x1000000053c' 2024-09-10T08:44:44.913 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11043 fp=#0x100000038e6' 2024-09-10T08:44:44.914 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11075 fp=#0x100000048b5' 2024-09-10T08:44:44.914 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11107 fp=#0x1000000507a' 2024-09-10T08:44:44.914 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11139 fp=#0x10000003e08' 2024-09-10T08:44:44.915 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11171 fp=#0x10000002372' 2024-09-10T08:44:44.915 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11203 fp=#0x10000002089' 2024-09-10T08:44:44.915 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11235 fp=#0x100000022ae' 2024-09-10T08:44:44.916 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11267 fp=#0x10000003e6d' 2024-09-10T08:44:44.916 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11299 fp=#0x10000003e3b' 2024-09-10T08:44:44.917 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11363 fp=#0x10000001d4a' 2024-09-10T08:44:44.917 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11395 fp=#0x10000001c4d' 2024-09-10T08:44:44.917 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11427 fp=#0x10000001a39' 2024-09-10T08:44:44.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11459 fp=#0x10000001d23' 2024-09-10T08:44:44.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11491 fp=#0x10000001a95' 2024-09-10T08:44:44.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11523 fp=#0x10000001a7c' 2024-09-10T08:44:44.920 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11683 fp=#0x1000000518a' 2024-09-10T08:44:44.920 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11715 fp=#0x10000000a28' 2024-09-10T08:44:44.920 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11747 fp=#0x1000000339f' 2024-09-10T08:44:44.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11779 fp=#0x10000004275' 2024-09-10T08:44:44.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11811 fp=#0x1000000422c' 2024-09-10T08:44:44.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11843 fp=#0x1000000425d' 2024-09-10T08:44:44.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11875 fp=#0x10000004de2' 2024-09-10T08:44:44.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11907 fp=#0x100000010c3' 2024-09-10T08:44:44.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11939 fp=#0x10000001108' 2024-09-10T08:44:44.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11971 fp=#0x10000001114' 2024-09-10T08:44:44.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12003 fp=#0x10000000f0e' 2024-09-10T08:44:44.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12035 fp=#0x100000010d6' 2024-09-10T08:44:44.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12067 fp=#0x1000000344f' 2024-09-10T08:44:44.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12099 fp=#0x100000034c8' 2024-09-10T08:44:44.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12131 fp=#0x100000043ba' 2024-09-10T08:44:44.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12163 fp=#0x1000000341e' 2024-09-10T08:44:44.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12195 fp=#0x1000000457e' 2024-09-10T08:44:44.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12227 fp=#0x10000000d61' 2024-09-10T08:44:44.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12291 fp=#0x100000039eb' 2024-09-10T08:44:44.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12323 fp=#0x10000003b1c' 2024-09-10T08:44:44.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12355 fp=#0x10000001133' 2024-09-10T08:44:44.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12387 fp=#0x10000000d24' 2024-09-10T08:44:44.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12419 fp=#0x10000003abb' 2024-09-10T08:44:44.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12515 fp=#0x10000001dd5' 2024-09-10T08:44:44.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12547 fp=#0x10000004c5e' 2024-09-10T08:44:44.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12579 fp=#0x10000004b0d' 2024-09-10T08:44:44.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12611 fp=#0x10000004b1a' 2024-09-10T08:44:44.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12643 fp=#0x10000004ca1' 2024-09-10T08:44:44.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12675 fp=#0x1000000490f' 2024-09-10T08:44:44.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12707 fp=#0x10000004817' 2024-09-10T08:44:44.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12739 fp=#0x10000004835' 2024-09-10T08:44:44.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12771 fp=#0x1000000541f' 2024-09-10T08:44:44.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12803 fp=#0x1000000533a' 2024-09-10T08:44:44.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12835 fp=#0x100000020c8' 2024-09-10T08:44:44.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12867 fp=#0x10000002303' 2024-09-10T08:44:44.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12899 fp=#0x100000022fe' 2024-09-10T08:44:44.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12931 fp=#0x1000000219a' 2024-09-10T08:44:44.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12963 fp=#0x10000004dce' 2024-09-10T08:44:44.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12995 fp=#0x100000053ce' 2024-09-10T08:44:44.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13051 fp=#0x10000003936' 2024-09-10T08:44:44.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13083 fp=#0x10000004f87' 2024-09-10T08:44:44.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13115 fp=#0x10000003914' 2024-09-10T08:44:44.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13147 fp=#0x100000039e2' 2024-09-10T08:44:44.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:140 fp=#0x10000000000' 2024-09-10T08:44:44.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:268 fp=#0x10000004ab9' 2024-09-10T08:44:44.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:300 fp=#0x100000012d8' 2024-09-10T08:44:44.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:388 fp=#0x10000004b7c' 2024-09-10T08:44:44.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:420 fp=#0x10000004ae4' 2024-09-10T08:44:44.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:452 fp=#0x10000004668' 2024-09-10T08:44:44.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:516 fp=#0x100000034ce' 2024-09-10T08:44:44.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:580 fp=#0x1000000337c' 2024-09-10T08:44:44.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:612 fp=#0x10000003345' 2024-09-10T08:44:44.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:772 fp=#0x10000000ebf' 2024-09-10T08:44:44.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:804 fp=#0x10000004401' 2024-09-10T08:44:44.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:836 fp=#0x10000004553' 2024-09-10T08:44:44.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:868 fp=#0x1000000441a' 2024-09-10T08:44:44.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:900 fp=#0x100000045aa' 2024-09-10T08:44:44.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:932 fp=#0x100000044e7' 2024-09-10T08:44:44.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:964 fp=#0x10000000ec1' 2024-09-10T08:44:44.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:996 fp=#0x10000000eb1' 2024-09-10T08:44:44.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1028 fp=#0x1000000442a' 2024-09-10T08:44:44.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1060 fp=#0x1000000427a' 2024-09-10T08:44:44.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1092 fp=#0x10000004281' 2024-09-10T08:44:44.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1220 fp=#0x10000001f7f' 2024-09-10T08:44:44.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1252 fp=#0x10000001fb7' 2024-09-10T08:44:44.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1284 fp=#0x100000035e7' 2024-09-10T08:44:44.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1316 fp=#0x10000001fa0' 2024-09-10T08:44:44.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1668 fp=#0x100000045a6' 2024-09-10T08:44:44.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1700 fp=#0x10000003368' 2024-09-10T08:44:44.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2052 fp=#0x100000021aa' 2024-09-10T08:44:44.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2148 fp=#0x1000000214e' 2024-09-10T08:44:44.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2212 fp=#0x10000002007' 2024-09-10T08:44:44.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2308 fp=#0x1000000258c' 2024-09-10T08:44:44.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2340 fp=#0x1000000277e' 2024-09-10T08:44:44.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2564 fp=#0x10000001181' 2024-09-10T08:44:44.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2596 fp=#0x1000000188d' 2024-09-10T08:44:44.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2628 fp=#0x100000010e2' 2024-09-10T08:44:44.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2660 fp=#0x100000010e6' 2024-09-10T08:44:44.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2692 fp=#0x100000010ea' 2024-09-10T08:44:44.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2724 fp=#0x100000010cd' 2024-09-10T08:44:44.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2820 fp=#0x10000004718' 2024-09-10T08:44:44.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2852 fp=#0x100000033ee' 2024-09-10T08:44:44.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2884 fp=#0x1000000346e' 2024-09-10T08:44:44.961 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2948 fp=#0x10000003567' 2024-09-10T08:44:44.961 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2980 fp=#0x1000000473c' 2024-09-10T08:44:44.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3044 fp=#0x100000019b6' 2024-09-10T08:44:44.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3076 fp=#0x10000000de2' 2024-09-10T08:44:44.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3172 fp=#0x10000003ec8' 2024-09-10T08:44:44.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3204 fp=#0x10000000df6' 2024-09-10T08:44:44.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3236 fp=#0x100000007d3' 2024-09-10T08:44:44.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3268 fp=#0x1000000078b' 2024-09-10T08:44:44.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3300 fp=#0x10000003c65' 2024-09-10T08:44:44.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3332 fp=#0x100000007a7' 2024-09-10T08:44:44.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3364 fp=#0x10000000cff' 2024-09-10T08:44:44.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3396 fp=#0x10000003c7d' 2024-09-10T08:44:44.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3428 fp=#0x100000007a2' 2024-09-10T08:44:44.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3460 fp=#0x100000040ea' 2024-09-10T08:44:44.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3492 fp=#0x10000004029' 2024-09-10T08:44:44.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3524 fp=#0x10000003c64' 2024-09-10T08:44:44.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3556 fp=#0x10000004149' 2024-09-10T08:44:44.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3588 fp=#0x10000003c12' 2024-09-10T08:44:44.968 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3620 fp=#0x10000003cd9' 2024-09-10T08:44:44.968 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3652 fp=#0x10000004bcd' 2024-09-10T08:44:44.968 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3684 fp=#0x10000004abe' 2024-09-10T08:44:44.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3716 fp=#0x10000004b28' 2024-09-10T08:44:44.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3748 fp=#0x100000032b4' 2024-09-10T08:44:44.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3780 fp=#0x1000000254e' 2024-09-10T08:44:44.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3812 fp=#0x10000002687' 2024-09-10T08:44:44.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3876 fp=#0x10000002770' 2024-09-10T08:44:44.971 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3908 fp=#0x10000004dbe' 2024-09-10T08:44:44.971 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3940 fp=#0x1000000056b' 2024-09-10T08:44:44.971 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3972 fp=#0x100000006ba' 2024-09-10T08:44:44.972 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4004 fp=#0x10000004e78' 2024-09-10T08:44:44.972 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4036 fp=#0x10000000fd6' 2024-09-10T08:44:44.972 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4068 fp=#0x10000001cc1' 2024-09-10T08:44:44.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4100 fp=#0x10000004650' 2024-09-10T08:44:44.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4132 fp=#0x100000046a0' 2024-09-10T08:44:44.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4164 fp=#0x10000003404' 2024-09-10T08:44:44.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4196 fp=#0x10000000fbc' 2024-09-10T08:44:44.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4228 fp=#0x10000003c66' 2024-09-10T08:44:44.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4260 fp=#0x1000000410c' 2024-09-10T08:44:44.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4292 fp=#0x100000039f5' 2024-09-10T08:44:44.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4324 fp=#0x10000003bbf' 2024-09-10T08:44:44.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4356 fp=#0x1000000416f' 2024-09-10T08:44:44.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4388 fp=#0x10000001fdc' 2024-09-10T08:44:44.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4452 fp=#0x10000001fc9' 2024-09-10T08:44:44.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4484 fp=#0x10000003850' 2024-09-10T08:44:44.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4516 fp=#0x10000001bbb' 2024-09-10T08:44:44.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4548 fp=#0x10000003336' 2024-09-10T08:44:44.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4580 fp=#0x10000001573' 2024-09-10T08:44:44.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4612 fp=#0x10000001599' 2024-09-10T08:44:44.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4644 fp=#0x100000042df' 2024-09-10T08:44:44.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4676 fp=#0x10000003910' 2024-09-10T08:44:44.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4708 fp=#0x10000001304' 2024-09-10T08:44:44.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4740 fp=#0x10000001590' 2024-09-10T08:44:44.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4772 fp=#0x10000001276' 2024-09-10T08:44:44.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4804 fp=#0x10000003810' 2024-09-10T08:44:44.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4836 fp=#0x100000037dd' 2024-09-10T08:44:44.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4868 fp=#0x10000004a9a' 2024-09-10T08:44:44.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4900 fp=#0x100000036e6' 2024-09-10T08:44:44.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4932 fp=#0x1000000379b' 2024-09-10T08:44:44.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4964 fp=#0x1000000534e' 2024-09-10T08:44:44.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4996 fp=#0x100000051b9' 2024-09-10T08:44:44.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5028 fp=#0x10000005068' 2024-09-10T08:44:44.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5060 fp=#0x10000005136' 2024-09-10T08:44:44.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5092 fp=#0x100000050e4' 2024-09-10T08:44:44.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5156 fp=#0x10000001e55' 2024-09-10T08:44:44.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5188 fp=#0x10000001bdc' 2024-09-10T08:44:44.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5220 fp=#0x10000001d0c' 2024-09-10T08:44:44.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5252 fp=#0x1000000389b' 2024-09-10T08:44:44.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5284 fp=#0x100000035f4' 2024-09-10T08:44:44.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5316 fp=#0x10000003ff0' 2024-09-10T08:44:44.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5348 fp=#0x10000003741' 2024-09-10T08:44:44.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5380 fp=#0x100000036b6' 2024-09-10T08:44:44.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5412 fp=#0x100000035f9' 2024-09-10T08:44:44.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5476 fp=#0x10000001228' 2024-09-10T08:44:44.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5540 fp=#0x10000001eef' 2024-09-10T08:44:44.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5572 fp=#0x10000002117' 2024-09-10T08:44:44.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5604 fp=#0x10000000f44' 2024-09-10T08:44:44.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5636 fp=#0x10000001078' 2024-09-10T08:44:44.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5668 fp=#0x100000050ef' 2024-09-10T08:44:44.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5700 fp=#0x10000001006' 2024-09-10T08:44:44.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5732 fp=#0x10000005123' 2024-09-10T08:44:44.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5764 fp=#0x100000050bf' 2024-09-10T08:44:44.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5796 fp=#0x1000000435f' 2024-09-10T08:44:44.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5828 fp=#0x1000000436e' 2024-09-10T08:44:44.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5860 fp=#0x10000004b39' 2024-09-10T08:44:44.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5892 fp=#0x10000004604' 2024-09-10T08:44:44.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5924 fp=#0x100000044ca' 2024-09-10T08:44:44.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5956 fp=#0x10000004c0e' 2024-09-10T08:44:44.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6084 fp=#0x100000034aa' 2024-09-10T08:44:44.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6116 fp=#0x100000034ae' 2024-09-10T08:44:44.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6148 fp=#0x100000035b3' 2024-09-10T08:44:44.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6212 fp=#0x1000000385d' 2024-09-10T08:44:44.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6276 fp=#0x10000001ebe' 2024-09-10T08:44:44.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6308 fp=#0x10000001e88' 2024-09-10T08:44:44.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6340 fp=#0x10000001f11' 2024-09-10T08:44:44.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6372 fp=#0x1000000332f' 2024-09-10T08:44:44.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6404 fp=#0x10000004dd7' 2024-09-10T08:44:44.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6436 fp=#0x10000004eb4' 2024-09-10T08:44:44.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6468 fp=#0x10000004c3a' 2024-09-10T08:44:44.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6500 fp=#0x10000004c57' 2024-09-10T08:44:44.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6532 fp=#0x10000004dcf' 2024-09-10T08:44:44.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6628 fp=#0x10000003a7f' 2024-09-10T08:44:44.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6660 fp=#0x10000003d0f' 2024-09-10T08:44:44.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6692 fp=#0x10000005299' 2024-09-10T08:44:45.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6724 fp=#0x10000003ca8' 2024-09-10T08:44:45.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6756 fp=#0x10000003cdf' 2024-09-10T08:44:45.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6788 fp=#0x10000004811' 2024-09-10T08:44:45.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6820 fp=#0x100000045d0' 2024-09-10T08:44:45.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6852 fp=#0x1000000532e' 2024-09-10T08:44:45.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6884 fp=#0x1000000457b' 2024-09-10T08:44:45.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6916 fp=#0x10000004550' 2024-09-10T08:44:45.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6948 fp=#0x1000000528f' 2024-09-10T08:44:45.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6980 fp=#0x1000000080f' 2024-09-10T08:44:45.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7012 fp=#0x1000000067e' 2024-09-10T08:44:45.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7044 fp=#0x1000000502b' 2024-09-10T08:44:45.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7076 fp=#0x10000001df8' 2024-09-10T08:44:45.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7108 fp=#0x10000000602' 2024-09-10T08:44:45.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7140 fp=#0x10000000694' 2024-09-10T08:44:45.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7172 fp=#0x1000000066e' 2024-09-10T08:44:45.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7236 fp=#0x10000004211' 2024-09-10T08:44:45.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7268 fp=#0x10000005084' 2024-09-10T08:44:45.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7396 fp=#0x10000004924' 2024-09-10T08:44:45.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7428 fp=#0x100000048a0' 2024-09-10T08:44:45.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7460 fp=#0x100000048e0' 2024-09-10T08:44:45.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7492 fp=#0x10000002453' 2024-09-10T08:44:45.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7524 fp=#0x100000047c4' 2024-09-10T08:44:45.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7556 fp=#0x10000001381' 2024-09-10T08:44:45.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7588 fp=#0x10000003327' 2024-09-10T08:44:45.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7620 fp=#0x1000000131d' 2024-09-10T08:44:45.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7652 fp=#0x10000001341' 2024-09-10T08:44:45.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7716 fp=#0x10000004e5f' 2024-09-10T08:44:45.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7748 fp=#0x10000004e56' 2024-09-10T08:44:45.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7780 fp=#0x10000004e95' 2024-09-10T08:44:45.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7812 fp=#0x1000000201f' 2024-09-10T08:44:45.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7844 fp=#0x10000001658' 2024-09-10T08:44:45.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7876 fp=#0x10000004d54' 2024-09-10T08:44:45.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7940 fp=#0x100000016bc' 2024-09-10T08:44:45.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7972 fp=#0x10000004da4' 2024-09-10T08:44:45.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8004 fp=#0x1000000145b' 2024-09-10T08:44:45.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8036 fp=#0x10000001712' 2024-09-10T08:44:45.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8068 fp=#0x1000000386d' 2024-09-10T08:44:45.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8100 fp=#0x10000001618' 2024-09-10T08:44:45.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8132 fp=#0x1000000167c' 2024-09-10T08:44:45.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8164 fp=#0x1000000256d' 2024-09-10T08:44:45.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8196 fp=#0x10000002562' 2024-09-10T08:44:45.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8228 fp=#0x100000026b4' 2024-09-10T08:44:45.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8260 fp=#0x1000000465d' 2024-09-10T08:44:45.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8292 fp=#0x100000045cf' 2024-09-10T08:44:45.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8324 fp=#0x100000041c0' 2024-09-10T08:44:45.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8356 fp=#0x100000042de' 2024-09-10T08:44:45.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8388 fp=#0x100000044fe' 2024-09-10T08:44:45.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8420 fp=#0x10000005266' 2024-09-10T08:44:45.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8452 fp=#0x10000003f16' 2024-09-10T08:44:45.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8484 fp=#0x10000003ef3' 2024-09-10T08:44:45.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8516 fp=#0x10000004264' 2024-09-10T08:44:45.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8548 fp=#0x10000004187' 2024-09-10T08:44:45.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8580 fp=#0x100000041a9' 2024-09-10T08:44:45.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8612 fp=#0x10000003db9' 2024-09-10T08:44:45.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8644 fp=#0x10000004092' 2024-09-10T08:44:45.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8676 fp=#0x100000052c7' 2024-09-10T08:44:45.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8708 fp=#0x10000005323' 2024-09-10T08:44:45.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8740 fp=#0x10000003bc4' 2024-09-10T08:44:45.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8772 fp=#0x10000004f7d' 2024-09-10T08:44:45.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8804 fp=#0x100000052d5' 2024-09-10T08:44:45.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8836 fp=#0x1000000448d' 2024-09-10T08:44:45.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8868 fp=#0x10000002242' 2024-09-10T08:44:45.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8900 fp=#0x10000001d0b' 2024-09-10T08:44:45.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8932 fp=#0x10000001b31' 2024-09-10T08:44:45.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8964 fp=#0x10000004394' 2024-09-10T08:44:45.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8996 fp=#0x10000001d14' 2024-09-10T08:44:45.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9028 fp=#0x10000000d1f' 2024-09-10T08:44:45.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9060 fp=#0x10000000da2' 2024-09-10T08:44:45.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9124 fp=#0x10000004173' 2024-09-10T08:44:45.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9156 fp=#0x10000000d90' 2024-09-10T08:44:45.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9188 fp=#0x10000004079' 2024-09-10T08:44:45.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9220 fp=#0x100000041e2' 2024-09-10T08:44:45.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9252 fp=#0x10000004d4a' 2024-09-10T08:44:45.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9284 fp=#0x10000000b44' 2024-09-10T08:44:45.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9316 fp=#0x100000025f7' 2024-09-10T08:44:45.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9348 fp=#0x10000004e30' 2024-09-10T08:44:45.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9380 fp=#0x10000000b91' 2024-09-10T08:44:45.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9412 fp=#0x10000000b5b' 2024-09-10T08:44:45.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9444 fp=#0x10000000ac8' 2024-09-10T08:44:45.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9508 fp=#0x10000004896' 2024-09-10T08:44:45.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9540 fp=#0x10000000ee4' 2024-09-10T08:44:45.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9572 fp=#0x10000001a1a' 2024-09-10T08:44:45.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9604 fp=#0x100000048f0' 2024-09-10T08:44:45.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9636 fp=#0x100000048f7' 2024-09-10T08:44:45.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9668 fp=#0x100000048ac' 2024-09-10T08:44:45.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9700 fp=#0x100000047cf' 2024-09-10T08:44:45.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9732 fp=#0x100000043a9' 2024-09-10T08:44:45.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9764 fp=#0x10000001983' 2024-09-10T08:44:45.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9796 fp=#0x10000004414' 2024-09-10T08:44:45.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9828 fp=#0x10000001774' 2024-09-10T08:44:45.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9860 fp=#0x100000043df' 2024-09-10T08:44:45.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9892 fp=#0x1000000448b' 2024-09-10T08:44:45.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9924 fp=#0x10000001609' 2024-09-10T08:44:45.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9956 fp=#0x1000000196b' 2024-09-10T08:44:45.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9988 fp=#0x100000015e5' 2024-09-10T08:44:45.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10020 fp=#0x1000000175d' 2024-09-10T08:44:45.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10052 fp=#0x100000019bd' 2024-09-10T08:44:45.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10084 fp=#0x10000001de7' 2024-09-10T08:44:45.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10116 fp=#0x10000003aa7' 2024-09-10T08:44:45.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10148 fp=#0x1000000393f' 2024-09-10T08:44:45.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10180 fp=#0x10000003b76' 2024-09-10T08:44:45.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10212 fp=#0x10000001e47' 2024-09-10T08:44:45.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10244 fp=#0x10000004e06' 2024-09-10T08:44:45.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10276 fp=#0x10000004590' 2024-09-10T08:44:45.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10308 fp=#0x10000002464' 2024-09-10T08:44:45.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10340 fp=#0x1000000144e' 2024-09-10T08:44:45.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10404 fp=#0x100000039df' 2024-09-10T08:44:45.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10436 fp=#0x10000003b27' 2024-09-10T08:44:45.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10468 fp=#0x100000013ee' 2024-09-10T08:44:45.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10500 fp=#0x10000001397' 2024-09-10T08:44:45.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10532 fp=#0x100000013b9' 2024-09-10T08:44:45.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10564 fp=#0x10000003955' 2024-09-10T08:44:45.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10596 fp=#0x10000004868' 2024-09-10T08:44:45.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10660 fp=#0x100000010eb' 2024-09-10T08:44:45.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10724 fp=#0x100000007c5' 2024-09-10T08:44:45.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10788 fp=#0x1000000487b' 2024-09-10T08:44:45.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10820 fp=#0x10000000a62' 2024-09-10T08:44:45.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10852 fp=#0x100000018ac' 2024-09-10T08:44:45.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10884 fp=#0x100000009ff' 2024-09-10T08:44:45.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10916 fp=#0x10000001d34' 2024-09-10T08:44:45.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10948 fp=#0x10000001015' 2024-09-10T08:44:45.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11012 fp=#0x10000000a4d' 2024-09-10T08:44:45.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11044 fp=#0x100000039c3' 2024-09-10T08:44:45.045 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11076 fp=#0x100000038f4' 2024-09-10T08:44:45.045 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11108 fp=#0x10000003ac7' 2024-09-10T08:44:45.045 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11140 fp=#0x1000000236a' 2024-09-10T08:44:45.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11172 fp=#0x1000000224c' 2024-09-10T08:44:45.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11204 fp=#0x10000003ee4' 2024-09-10T08:44:45.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11236 fp=#0x10000002194' 2024-09-10T08:44:45.047 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11268 fp=#0x100000020cb' 2024-09-10T08:44:45.047 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11300 fp=#0x10000001d13' 2024-09-10T08:44:45.047 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11332 fp=#0x10000001d56' 2024-09-10T08:44:45.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11364 fp=#0x10000001ce5' 2024-09-10T08:44:45.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11396 fp=#0x10000003dbf' 2024-09-10T08:44:45.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11428 fp=#0x10000003ece' 2024-09-10T08:44:45.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11460 fp=#0x10000001bf0' 2024-09-10T08:44:45.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11492 fp=#0x10000001d08' 2024-09-10T08:44:45.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11524 fp=#0x10000001aa5' 2024-09-10T08:44:45.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11620 fp=#0x1000000515c' 2024-09-10T08:44:45.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11684 fp=#0x1000000386f' 2024-09-10T08:44:45.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11716 fp=#0x100000026ba' 2024-09-10T08:44:45.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11748 fp=#0x10000001fc2' 2024-09-10T08:44:45.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11780 fp=#0x10000000958' 2024-09-10T08:44:45.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11812 fp=#0x10000000895' 2024-09-10T08:44:45.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11844 fp=#0x10000000a02' 2024-09-10T08:44:45.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11876 fp=#0x10000004db9' 2024-09-10T08:44:45.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11908 fp=#0x10000000ef3' 2024-09-10T08:44:45.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11940 fp=#0x10000000efc' 2024-09-10T08:44:45.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11972 fp=#0x1000000186f' 2024-09-10T08:44:45.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12004 fp=#0x10000001128' 2024-09-10T08:44:45.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12036 fp=#0x10000000f09' 2024-09-10T08:44:45.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12068 fp=#0x100000044ff' 2024-09-10T08:44:45.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12100 fp=#0x10000004532' 2024-09-10T08:44:45.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12132 fp=#0x1000000446f' 2024-09-10T08:44:45.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12164 fp=#0x100000033b2' 2024-09-10T08:44:45.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12196 fp=#0x10000004540' 2024-09-10T08:44:45.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12228 fp=#0x10000000c41' 2024-09-10T08:44:45.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12260 fp=#0x10000003a5b' 2024-09-10T08:44:45.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12292 fp=#0x10000003a63' 2024-09-10T08:44:45.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12324 fp=#0x10000003aa9' 2024-09-10T08:44:45.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12356 fp=#0x10000003acc' 2024-09-10T08:44:45.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12388 fp=#0x10000003b52' 2024-09-10T08:44:45.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12420 fp=#0x10000003b09' 2024-09-10T08:44:45.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12452 fp=#0x100000010d0' 2024-09-10T08:44:45.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12580 fp=#0x10000004c0a' 2024-09-10T08:44:45.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12612 fp=#0x10000004c18' 2024-09-10T08:44:45.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12644 fp=#0x10000004af5' 2024-09-10T08:44:45.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12676 fp=#0x10000004841' 2024-09-10T08:44:45.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12708 fp=#0x10000005353' 2024-09-10T08:44:45.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12740 fp=#0x1000000496a' 2024-09-10T08:44:45.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12772 fp=#0x100000053e1' 2024-09-10T08:44:45.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12804 fp=#0x100000020eb' 2024-09-10T08:44:45.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12836 fp=#0x100000052dd' 2024-09-10T08:44:45.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12868 fp=#0x10000004519' 2024-09-10T08:44:45.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12900 fp=#0x100000021d0' 2024-09-10T08:44:45.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12932 fp=#0x1000000216e' 2024-09-10T08:44:45.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12964 fp=#0x10000002783' 2024-09-10T08:44:45.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12996 fp=#0x10000002754' 2024-09-10T08:44:45.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13052 fp=#0x10000003996' 2024-09-10T08:44:45.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13084 fp=#0x100000022c2' 2024-09-10T08:44:45.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13148 fp=#0x100000022ef' 2024-09-10T08:44:45.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:269 fp=#0x100000024db' 2024-09-10T08:44:45.070 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:301 fp=#0x10000004dcb' 2024-09-10T08:44:45.070 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'client_request(client.25157:74426 create owner_uid=1000, owner_gid=1333 #0x10000005387/article-29.xml.tmp 2024-09-10T08:43:46.112543+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})' 2024-09-10T08:44:45.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:485 fp=#0x10000003285' 2024-09-10T08:44:45.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:517 fp=#0x10000003451' 2024-09-10T08:44:45.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:581 fp=#0x100000032fe' 2024-09-10T08:44:45.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:613 fp=#0x1000000333f' 2024-09-10T08:44:45.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:741 fp=#0x10000003967' 2024-09-10T08:44:45.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:773 fp=#0x1000000446a' 2024-09-10T08:44:45.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:805 fp=#0x10000000f19' 2024-09-10T08:44:45.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:837 fp=#0x10000000d8e' 2024-09-10T08:44:45.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:869 fp=#0x10000000e69' 2024-09-10T08:44:45.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:901 fp=#0x10000000d93' 2024-09-10T08:44:45.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:933 fp=#0x10000004418' 2024-09-10T08:44:45.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:965 fp=#0x10000000e81' 2024-09-10T08:44:45.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:997 fp=#0x1000000406a' 2024-09-10T08:44:45.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1029 fp=#0x10000000dc7' 2024-09-10T08:44:45.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1061 fp=#0x1000000415e' 2024-09-10T08:44:45.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1093 fp=#0x100000041d4' 2024-09-10T08:44:45.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1317 fp=#0x10000003609' 2024-09-10T08:44:45.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1349 fp=#0x10000001f9c' 2024-09-10T08:44:45.080 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1381 fp=#0x100000032a0' 2024-09-10T08:44:45.081 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1509 fp=#0x10000004533' 2024-09-10T08:44:45.082 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1669 fp=#0x100000045c2' 2024-09-10T08:44:45.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1861 fp=#0x10000002791' 2024-09-10T08:44:45.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2245 fp=#0x100000027c2' 2024-09-10T08:44:45.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2277 fp=#0x100000027b6' 2024-09-10T08:44:45.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2565 fp=#0x100000010ba' 2024-09-10T08:44:45.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2597 fp=#0x100000010a6' 2024-09-10T08:44:45.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2629 fp=#0x100000010f2' 2024-09-10T08:44:45.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2661 fp=#0x1000000108b' 2024-09-10T08:44:45.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2693 fp=#0x100000010a2' 2024-09-10T08:44:45.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2725 fp=#0x100000010f7' 2024-09-10T08:44:45.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2757 fp=#0x10000003b4e' 2024-09-10T08:44:45.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2789 fp=#0x10000004763' 2024-09-10T08:44:45.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2821 fp=#0x1000000471f' 2024-09-10T08:44:45.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2853 fp=#0x100000045fa' 2024-09-10T08:44:45.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2885 fp=#0x100000033e8' 2024-09-10T08:44:45.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2917 fp=#0x1000000354f' 2024-09-10T08:44:45.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2949 fp=#0x10000003431' 2024-09-10T08:44:45.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2981 fp=#0x10000004697' 2024-09-10T08:44:45.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3013 fp=#0x100000049cf' 2024-09-10T08:44:45.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3045 fp=#0x10000004b6c' 2024-09-10T08:44:45.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3077 fp=#0x100000018b1' 2024-09-10T08:44:45.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3109 fp=#0x10000000e19' 2024-09-10T08:44:45.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3141 fp=#0x10000000e10' 2024-09-10T08:44:45.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3173 fp=#0x10000000e74' 2024-09-10T08:44:45.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3205 fp=#0x10000000e02' 2024-09-10T08:44:45.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3237 fp=#0x10000003d44' 2024-09-10T08:44:45.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3269 fp=#0x10000003cf0' 2024-09-10T08:44:45.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3301 fp=#0x10000000723' 2024-09-10T08:44:45.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3333 fp=#0x1000000076c' 2024-09-10T08:44:45.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3365 fp=#0x10000000753' 2024-09-10T08:44:45.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3397 fp=#0x10000003cab' 2024-09-10T08:44:45.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3429 fp=#0x100000041e5' 2024-09-10T08:44:45.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3493 fp=#0x1000000402d' 2024-09-10T08:44:45.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3589 fp=#0x10000004185' 2024-09-10T08:44:45.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3621 fp=#0x10000004049' 2024-09-10T08:44:45.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3653 fp=#0x10000004b24' 2024-09-10T08:44:45.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3685 fp=#0x10000004bb7' 2024-09-10T08:44:45.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3717 fp=#0x10000004c67' 2024-09-10T08:44:45.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3781 fp=#0x1000000250a' 2024-09-10T08:44:45.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3813 fp=#0x10000002739' 2024-09-10T08:44:45.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3845 fp=#0x10000002588' 2024-09-10T08:44:45.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3877 fp=#0x1000000255b' 2024-09-10T08:44:45.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3909 fp=#0x100000027c1' 2024-09-10T08:44:45.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3941 fp=#0x10000000697' 2024-09-10T08:44:45.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3973 fp=#0x10000002048' 2024-09-10T08:44:45.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4005 fp=#0x10000005291' 2024-09-10T08:44:45.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4037 fp=#0x10000004da7' 2024-09-10T08:44:45.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4069 fp=#0x10000000f37' 2024-09-10T08:44:45.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4101 fp=#0x10000000f67' 2024-09-10T08:44:45.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4133 fp=#0x10000004e9b' 2024-09-10T08:44:45.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4165 fp=#0x10000000f61' 2024-09-10T08:44:45.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4197 fp=#0x10000004138' 2024-09-10T08:44:45.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4229 fp=#0x10000003c86' 2024-09-10T08:44:45.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4261 fp=#0x10000004103' 2024-09-10T08:44:45.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4293 fp=#0x1000000406f' 2024-09-10T08:44:45.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4325 fp=#0x10000004035' 2024-09-10T08:44:45.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4357 fp=#0x1000000403c' 2024-09-10T08:44:45.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4389 fp=#0x10000001ef8' 2024-09-10T08:44:45.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4453 fp=#0x10000004d39' 2024-09-10T08:44:45.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4485 fp=#0x10000003eda' 2024-09-10T08:44:45.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4517 fp=#0x1000000510a' 2024-09-10T08:44:45.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4549 fp=#0x1000000403f' 2024-09-10T08:44:45.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4581 fp=#0x100000012a2' 2024-09-10T08:44:45.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4613 fp=#0x100000015cc' 2024-09-10T08:44:45.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4645 fp=#0x10000001fc5' 2024-09-10T08:44:45.112 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4677 fp=#0x100000046b2' 2024-09-10T08:44:45.112 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4709 fp=#0x100000012c6' 2024-09-10T08:44:45.112 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4741 fp=#0x100000014b4' 2024-09-10T08:44:45.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4773 fp=#0x1000000396e' 2024-09-10T08:44:45.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4805 fp=#0x100000036c7' 2024-09-10T08:44:45.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4837 fp=#0x10000003783' 2024-09-10T08:44:45.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4869 fp=#0x10000004a94' 2024-09-10T08:44:45.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4901 fp=#0x100000036d0' 2024-09-10T08:44:45.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4933 fp=#0x1000000370e' 2024-09-10T08:44:45.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4965 fp=#0x10000004fe4' 2024-09-10T08:44:45.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4997 fp=#0x10000005167' 2024-09-10T08:44:45.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5029 fp=#0x100000050c8' 2024-09-10T08:44:45.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5061 fp=#0x1000000507b' 2024-09-10T08:44:45.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5093 fp=#0x10000005328' 2024-09-10T08:44:45.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5125 fp=#0x10000001ec0' 2024-09-10T08:44:45.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5157 fp=#0x10000001c88' 2024-09-10T08:44:45.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5189 fp=#0x10000001cc4' 2024-09-10T08:44:45.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5221 fp=#0x10000001bc6' 2024-09-10T08:44:45.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5253 fp=#0x100000035cc' 2024-09-10T08:44:45.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5285 fp=#0x10000003606' 2024-09-10T08:44:45.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5317 fp=#0x100000036a0' 2024-09-10T08:44:45.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5349 fp=#0x10000004aa3' 2024-09-10T08:44:45.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5381 fp=#0x100000038aa' 2024-09-10T08:44:45.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5413 fp=#0x10000004a9c' 2024-09-10T08:44:45.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5445 fp=#0x10000001281' 2024-09-10T08:44:45.121 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5573 fp=#0x100000010a7' 2024-09-10T08:44:45.121 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5605 fp=#0x10000002104' 2024-09-10T08:44:45.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5637 fp=#0x10000000f98' 2024-09-10T08:44:45.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5669 fp=#0x10000000f4f' 2024-09-10T08:44:45.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5701 fp=#0x100000026a7' 2024-09-10T08:44:45.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5733 fp=#0x10000000f7a' 2024-09-10T08:44:45.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5765 fp=#0x10000001028' 2024-09-10T08:44:45.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5797 fp=#0x1000000445c' 2024-09-10T08:44:45.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5829 fp=#0x1000000496c' 2024-09-10T08:44:45.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5861 fp=#0x100000042e6' 2024-09-10T08:44:45.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5893 fp=#0x100000044e2' 2024-09-10T08:44:45.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5925 fp=#0x100000042d6' 2024-09-10T08:44:45.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5957 fp=#0x100000043f7' 2024-09-10T08:44:45.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6117 fp=#0x10000004576' 2024-09-10T08:44:45.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6181 fp=#0x100000049ea' 2024-09-10T08:44:45.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6277 fp=#0x10000001dbe' 2024-09-10T08:44:45.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6309 fp=#0x1000000334a' 2024-09-10T08:44:45.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6341 fp=#0x10000001e71' 2024-09-10T08:44:45.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6373 fp=#0x1000000335b' 2024-09-10T08:44:45.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6405 fp=#0x10000004d40' 2024-09-10T08:44:45.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6437 fp=#0x10000004b3f' 2024-09-10T08:44:45.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6469 fp=#0x10000004d64' 2024-09-10T08:44:45.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6501 fp=#0x10000004d0d' 2024-09-10T08:44:45.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6533 fp=#0x10000004f4e' 2024-09-10T08:44:45.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6629 fp=#0x10000003d78' 2024-09-10T08:44:45.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6661 fp=#0x10000003b3c' 2024-09-10T08:44:45.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6693 fp=#0x10000004f6b' 2024-09-10T08:44:45.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6725 fp=#0x10000003aec' 2024-09-10T08:44:45.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6757 fp=#0x10000004fe2' 2024-09-10T08:44:45.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6789 fp=#0x100000045a3' 2024-09-10T08:44:45.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6821 fp=#0x100000045df' 2024-09-10T08:44:45.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6853 fp=#0x1000000453e' 2024-09-10T08:44:45.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6885 fp=#0x100000046a8' 2024-09-10T08:44:45.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6917 fp=#0x1000000537f' 2024-09-10T08:44:45.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6949 fp=#0x100000052b0' 2024-09-10T08:44:45.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6981 fp=#0x10000005060' 2024-09-10T08:44:45.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7013 fp=#0x10000000659' 2024-09-10T08:44:45.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7045 fp=#0x100000005e7' 2024-09-10T08:44:45.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7077 fp=#0x10000004f78' 2024-09-10T08:44:45.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7109 fp=#0x100000005b5' 2024-09-10T08:44:45.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7141 fp=#0x1000000058c' 2024-09-10T08:44:45.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7173 fp=#0x1000000065a' 2024-09-10T08:44:45.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7365 fp=#0x10000005120' 2024-09-10T08:44:45.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7397 fp=#0x100000023c0' 2024-09-10T08:44:45.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7429 fp=#0x10000004894' 2024-09-10T08:44:45.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7461 fp=#0x100000047ef' 2024-09-10T08:44:45.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7493 fp=#0x100000047bc' 2024-09-10T08:44:45.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7525 fp=#0x1000000469f' 2024-09-10T08:44:45.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7557 fp=#0x1000000139c' 2024-09-10T08:44:45.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7621 fp=#0x100000051ce' 2024-09-10T08:44:45.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7685 fp=#0x10000004536' 2024-09-10T08:44:45.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7717 fp=#0x10000004fa7' 2024-09-10T08:44:45.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7749 fp=#0x10000004ebe' 2024-09-10T08:44:45.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7781 fp=#0x10000004df5' 2024-09-10T08:44:45.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7813 fp=#0x1000000171a' 2024-09-10T08:44:45.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7845 fp=#0x1000000154b' 2024-09-10T08:44:45.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7877 fp=#0x10000004f25' 2024-09-10T08:44:45.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7909 fp=#0x10000004f07' 2024-09-10T08:44:45.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7941 fp=#0x100000014f0' 2024-09-10T08:44:45.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7973 fp=#0x10000004d2f' 2024-09-10T08:44:45.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8005 fp=#0x1000000146a' 2024-09-10T08:44:45.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8069 fp=#0x1000000384b' 2024-09-10T08:44:45.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8101 fp=#0x10000001614' 2024-09-10T08:44:45.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8165 fp=#0x1000000263c' 2024-09-10T08:44:45.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8197 fp=#0x1000000250e' 2024-09-10T08:44:45.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8229 fp=#0x10000002644' 2024-09-10T08:44:45.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8261 fp=#0x10000004189' 2024-09-10T08:44:45.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8293 fp=#0x10000004411' 2024-09-10T08:44:45.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8325 fp=#0x1000000526a' 2024-09-10T08:44:45.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8357 fp=#0x1000000525d' 2024-09-10T08:44:45.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8389 fp=#0x1000000524e' 2024-09-10T08:44:45.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8421 fp=#0x10000004119' 2024-09-10T08:44:45.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8453 fp=#0x10000004504' 2024-09-10T08:44:45.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8485 fp=#0x10000004269' 2024-09-10T08:44:45.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8517 fp=#0x10000003e34' 2024-09-10T08:44:45.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8549 fp=#0x1000000422a' 2024-09-10T08:44:45.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8581 fp=#0x10000003ea0' 2024-09-10T08:44:45.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8613 fp=#0x100000040ee' 2024-09-10T08:44:45.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8645 fp=#0x100000052ff' 2024-09-10T08:44:45.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8677 fp=#0x10000003c7e' 2024-09-10T08:44:45.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8709 fp=#0x10000004fa6' 2024-09-10T08:44:45.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8741 fp=#0x10000003bd7' 2024-09-10T08:44:45.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8773 fp=#0x10000003cb0' 2024-09-10T08:44:45.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8805 fp=#0x100000052bd' 2024-09-10T08:44:45.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8837 fp=#0x10000001d47' 2024-09-10T08:44:45.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8869 fp=#0x10000001d52' 2024-09-10T08:44:45.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8901 fp=#0x10000004446' 2024-09-10T08:44:45.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8933 fp=#0x100000043e7' 2024-09-10T08:44:45.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8965 fp=#0x10000001ae6' 2024-09-10T08:44:45.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8997 fp=#0x10000001b17' 2024-09-10T08:44:45.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9029 fp=#0x10000004cf0' 2024-09-10T08:44:45.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9061 fp=#0x10000000d09' 2024-09-10T08:44:45.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9093 fp=#0x10000000cfe' 2024-09-10T08:44:45.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9125 fp=#0x10000000d3e' 2024-09-10T08:44:45.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9157 fp=#0x10000000d1d' 2024-09-10T08:44:45.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9189 fp=#0x100000040f4' 2024-09-10T08:44:45.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9221 fp=#0x10000000d76' 2024-09-10T08:44:45.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9253 fp=#0x100000026a1' 2024-09-10T08:44:45.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9285 fp=#0x10000004d18' 2024-09-10T08:44:45.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9317 fp=#0x100000010d7' 2024-09-10T08:44:45.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9349 fp=#0x100000010be' 2024-09-10T08:44:45.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9381 fp=#0x10000004c47' 2024-09-10T08:44:45.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9413 fp=#0x100000032f2' 2024-09-10T08:44:45.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9445 fp=#0x10000004df0' 2024-09-10T08:44:45.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9477 fp=#0x10000001eac' 2024-09-10T08:44:45.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9509 fp=#0x1000000473f' 2024-09-10T08:44:45.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9541 fp=#0x10000004976' 2024-09-10T08:44:45.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9573 fp=#0x10000002046' 2024-09-10T08:44:45.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9605 fp=#0x10000000ec2' 2024-09-10T08:44:45.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9669 fp=#0x1000000490d' 2024-09-10T08:44:45.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9701 fp=#0x100000047a8' 2024-09-10T08:44:45.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9733 fp=#0x10000001ac6' 2024-09-10T08:44:45.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9765 fp=#0x100000043ae' 2024-09-10T08:44:45.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9797 fp=#0x100000043c4' 2024-09-10T08:44:45.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9829 fp=#0x10000004466' 2024-09-10T08:44:45.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9861 fp=#0x10000004385' 2024-09-10T08:44:45.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9893 fp=#0x1000000194b' 2024-09-10T08:44:45.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9925 fp=#0x1000000159a' 2024-09-10T08:44:45.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9957 fp=#0x1000000159f' 2024-09-10T08:44:45.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9989 fp=#0x1000000164d' 2024-09-10T08:44:45.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10021 fp=#0x100000015fe' 2024-09-10T08:44:45.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10085 fp=#0x10000003b7b' 2024-09-10T08:44:45.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10117 fp=#0x10000003a2e' 2024-09-10T08:44:45.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10149 fp=#0x10000003b82' 2024-09-10T08:44:45.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10181 fp=#0x100000038e9' 2024-09-10T08:44:45.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10213 fp=#0x10000001db0' 2024-09-10T08:44:45.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10245 fp=#0x10000001e11' 2024-09-10T08:44:45.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10277 fp=#0x1000000257d' 2024-09-10T08:44:45.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10309 fp=#0x100000044d6' 2024-09-10T08:44:45.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10341 fp=#0x10000001313' 2024-09-10T08:44:45.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10373 fp=#0x1000000155e' 2024-09-10T08:44:45.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10405 fp=#0x10000001403' 2024-09-10T08:44:45.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10437 fp=#0x100000039e9' 2024-09-10T08:44:45.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10469 fp=#0x10000001380' 2024-09-10T08:44:45.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10501 fp=#0x1000000147e' 2024-09-10T08:44:45.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10533 fp=#0x10000001435' 2024-09-10T08:44:45.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10565 fp=#0x100000013a3' 2024-09-10T08:44:45.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10661 fp=#0x10000004d3c' 2024-09-10T08:44:45.179 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10949 fp=#0x10000001d10' 2024-09-10T08:44:45.179 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11013 fp=#0x1000000051b' 2024-09-10T08:44:45.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11045 fp=#0x10000003ac0' 2024-09-10T08:44:45.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11077 fp=#0x100000039fb' 2024-09-10T08:44:45.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11109 fp=#0x100000038f0' 2024-09-10T08:44:45.181 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11141 fp=#0x1000000239b' 2024-09-10T08:44:45.181 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11173 fp=#0x1000000210e' 2024-09-10T08:44:45.181 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11205 fp=#0x10000003fad' 2024-09-10T08:44:45.182 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11237 fp=#0x10000003e53' 2024-09-10T08:44:45.182 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11269 fp=#0x100000040d4' 2024-09-10T08:44:45.182 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11301 fp=#0x10000001a9c' 2024-09-10T08:44:45.183 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11333 fp=#0x100000046f2' 2024-09-10T08:44:45.183 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11365 fp=#0x10000001a46' 2024-09-10T08:44:45.183 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11397 fp=#0x10000001bd4' 2024-09-10T08:44:45.184 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11429 fp=#0x10000003ef9' 2024-09-10T08:44:45.184 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11461 fp=#0x10000001c61' 2024-09-10T08:44:45.184 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11493 fp=#0x10000003d9e' 2024-09-10T08:44:45.185 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11525 fp=#0x10000001cc0' 2024-09-10T08:44:45.185 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11589 fp=#0x1000000519a' 2024-09-10T08:44:45.186 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11621 fp=#0x1000000516e' 2024-09-10T08:44:45.186 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11653 fp=#0x100000051c0' 2024-09-10T08:44:45.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11717 fp=#0x10000000a22' 2024-09-10T08:44:45.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11749 fp=#0x100000009f2' 2024-09-10T08:44:45.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11781 fp=#0x10000000982' 2024-09-10T08:44:45.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11813 fp=#0x100000008a7' 2024-09-10T08:44:45.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11845 fp=#0x100000022b5' 2024-09-10T08:44:45.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11877 fp=#0x100000008b8' 2024-09-10T08:44:45.189 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11909 fp=#0x10000001d86' 2024-09-10T08:44:45.189 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11941 fp=#0x100000010e0' 2024-09-10T08:44:45.189 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11973 fp=#0x100000052dc' 2024-09-10T08:44:45.190 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12005 fp=#0x10000000eee' 2024-09-10T08:44:45.190 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12037 fp=#0x100000010e1' 2024-09-10T08:44:45.190 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12069 fp=#0x10000004081' 2024-09-10T08:44:45.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12101 fp=#0x10000004528' 2024-09-10T08:44:45.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12133 fp=#0x1000000440c' 2024-09-10T08:44:45.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12165 fp=#0x10000004443' 2024-09-10T08:44:45.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12197 fp=#0x10000004513' 2024-09-10T08:44:45.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12229 fp=#0x10000002580' 2024-09-10T08:44:45.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12261 fp=#0x10000000d32' 2024-09-10T08:44:45.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12293 fp=#0x1000000258b' 2024-09-10T08:44:45.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12325 fp=#0x10000000ce6' 2024-09-10T08:44:45.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12357 fp=#0x10000000bd8' 2024-09-10T08:44:45.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12389 fp=#0x10000003b63' 2024-09-10T08:44:45.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12421 fp=#0x10000002585' 2024-09-10T08:44:45.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12453 fp=#0x10000004c89' 2024-09-10T08:44:45.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12485 fp=#0x10000004bcc' 2024-09-10T08:44:45.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12517 fp=#0x10000004bc0' 2024-09-10T08:44:45.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12549 fp=#0x10000004ba6' 2024-09-10T08:44:45.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12581 fp=#0x10000004af1' 2024-09-10T08:44:45.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12613 fp=#0x10000004b60' 2024-09-10T08:44:45.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12645 fp=#0x10000004cb4' 2024-09-10T08:44:45.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12677 fp=#0x10000004772' 2024-09-10T08:44:45.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12709 fp=#0x10000004890' 2024-09-10T08:44:45.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12741 fp=#0x100000053a0' 2024-09-10T08:44:45.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12773 fp=#0x100000053f6' 2024-09-10T08:44:45.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12805 fp=#0x100000053c2' 2024-09-10T08:44:45.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12837 fp=#0x100000053b5' 2024-09-10T08:44:45.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12869 fp=#0x100000020bd' 2024-09-10T08:44:45.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12901 fp=#0x1000000535a' 2024-09-10T08:44:45.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12933 fp=#0x100000053de' 2024-09-10T08:44:45.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12965 fp=#0x1000000277d' 2024-09-10T08:44:45.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12997 fp=#0x10000003f85' 2024-09-10T08:44:45.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'client_request(client.25157:74437 create owner_uid=1000, owner_gid=1333 #0x10000004747/comment-53.xml.tmp 2024-09-10T08:43:46.434535+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})' 2024-09-10T08:44:45.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13053 fp=#0x100000022c5' 2024-09-10T08:44:45.201 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13085 fp=#0x100000039c7' 2024-09-10T08:44:45.201 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13117 fp=#0x10000003a1e' 2024-09-10T08:44:45.201 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13149 fp=#0x10000003721' 2024-09-10T08:44:45.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:164 fp=#0x10000000006' 2024-09-10T08:44:45.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:270 fp=#0x10000000504' 2024-09-10T08:44:45.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:302 fp=#0x100000013a0' 2024-09-10T08:44:45.204 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:390 fp=#0x1000000466f' 2024-09-10T08:44:45.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:486 fp=#0x10000003552' 2024-09-10T08:44:45.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:518 fp=#0x10000003315' 2024-09-10T08:44:45.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:550 fp=#0x10000003a15' 2024-09-10T08:44:45.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:582 fp=#0x1000000351f' 2024-09-10T08:44:45.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:614 fp=#0x10000003448' 2024-09-10T08:44:45.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:774 fp=#0x10000000eb5' 2024-09-10T08:44:45.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:806 fp=#0x10000003fc4' 2024-09-10T08:44:45.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:838 fp=#0x10000000ea7' 2024-09-10T08:44:45.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:870 fp=#0x10000000ea0' 2024-09-10T08:44:45.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:966 fp=#0x10000004492' 2024-09-10T08:44:45.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:998 fp=#0x10000004518' 2024-09-10T08:44:45.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1030 fp=#0x10000000dd3' 2024-09-10T08:44:45.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1094 fp=#0x10000004256' 2024-09-10T08:44:45.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1190 fp=#0x1000000420e' 2024-09-10T08:44:45.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1222 fp=#0x10000002053' 2024-09-10T08:44:45.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1318 fp=#0x10000001f8c' 2024-09-10T08:44:45.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1350 fp=#0x10000001fe5' 2024-09-10T08:44:45.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1574 fp=#0x10000003ca4' 2024-09-10T08:44:45.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1670 fp=#0x10000004600' 2024-09-10T08:44:45.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1958 fp=#0x100000027c6' 2024-09-10T08:44:45.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1990 fp=#0x10000003ee0' 2024-09-10T08:44:45.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2246 fp=#0x1000000268b' 2024-09-10T08:44:45.221 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2310 fp=#0x1000000259e' 2024-09-10T08:44:45.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2694 fp=#0x10000002220' 2024-09-10T08:44:45.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2758 fp=#0x10000003b56' 2024-09-10T08:44:45.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2790 fp=#0x10000004672' 2024-09-10T08:44:45.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2822 fp=#0x10000003614' 2024-09-10T08:44:45.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2854 fp=#0x10000003690' 2024-09-10T08:44:45.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2886 fp=#0x10000004777' 2024-09-10T08:44:45.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2918 fp=#0x10000003456' 2024-09-10T08:44:45.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2950 fp=#0x1000000364b' 2024-09-10T08:44:45.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2982 fp=#0x10000003421' 2024-09-10T08:44:45.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3014 fp=#0x10000001926' 2024-09-10T08:44:45.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3046 fp=#0x1000000192c' 2024-09-10T08:44:45.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3078 fp=#0x10000000e92' 2024-09-10T08:44:45.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3110 fp=#0x10000000ddf' 2024-09-10T08:44:45.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3142 fp=#0x10000000e84' 2024-09-10T08:44:45.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3174 fp=#0x100000018f7' 2024-09-10T08:44:45.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3206 fp=#0x10000000e63' 2024-09-10T08:44:45.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3238 fp=#0x10000003d05' 2024-09-10T08:44:45.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3270 fp=#0x10000003ceb' 2024-09-10T08:44:45.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3302 fp=#0x10000004a59' 2024-09-10T08:44:45.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3334 fp=#0x10000000759' 2024-09-10T08:44:45.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3366 fp=#0x10000003bf5' 2024-09-10T08:44:45.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3398 fp=#0x10000003c84' 2024-09-10T08:44:45.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3430 fp=#0x10000003d3d' 2024-09-10T08:44:45.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3462 fp=#0x1000000350c' 2024-09-10T08:44:45.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3526 fp=#0x10000004150' 2024-09-10T08:44:45.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3622 fp=#0x10000003c6b' 2024-09-10T08:44:45.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3654 fp=#0x10000004bda' 2024-09-10T08:44:45.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3686 fp=#0x10000004bd2' 2024-09-10T08:44:45.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3718 fp=#0x10000004b01' 2024-09-10T08:44:45.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3750 fp=#0x10000003eb2' 2024-09-10T08:44:45.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3782 fp=#0x10000003f54' 2024-09-10T08:44:45.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3814 fp=#0x10000003f0b' 2024-09-10T08:44:45.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3846 fp=#0x10000003eaa' 2024-09-10T08:44:45.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3910 fp=#0x1000000274f' 2024-09-10T08:44:45.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3942 fp=#0x10000000685' 2024-09-10T08:44:45.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3974 fp=#0x1000000058b' 2024-09-10T08:44:45.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4006 fp=#0x1000000069d' 2024-09-10T08:44:45.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4038 fp=#0x100000013d3' 2024-09-10T08:44:45.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4070 fp=#0x10000000fc6' 2024-09-10T08:44:45.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4102 fp=#0x10000000fb5' 2024-09-10T08:44:45.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4134 fp=#0x100000033fb' 2024-09-10T08:44:45.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4166 fp=#0x10000000f7c' 2024-09-10T08:44:45.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4198 fp=#0x10000000f6d' 2024-09-10T08:44:45.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4230 fp=#0x1000000397f' 2024-09-10T08:44:45.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4262 fp=#0x10000004a4a' 2024-09-10T08:44:45.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4294 fp=#0x1000000413d' 2024-09-10T08:44:45.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4326 fp=#0x10000003a16' 2024-09-10T08:44:45.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4358 fp=#0x10000004075' 2024-09-10T08:44:45.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4390 fp=#0x10000001fb4' 2024-09-10T08:44:45.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4422 fp=#0x10000004d00' 2024-09-10T08:44:45.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4454 fp=#0x10000001f75' 2024-09-10T08:44:45.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4486 fp=#0x10000001882' 2024-09-10T08:44:45.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4518 fp=#0x10000001888' 2024-09-10T08:44:45.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4550 fp=#0x1000000404b' 2024-09-10T08:44:45.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4582 fp=#0x10000001327' 2024-09-10T08:44:45.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4614 fp=#0x10000003957' 2024-09-10T08:44:45.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4646 fp=#0x1000000151c' 2024-09-10T08:44:45.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4678 fp=#0x10000001468' 2024-09-10T08:44:45.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4710 fp=#0x10000001452' 2024-09-10T08:44:45.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4742 fp=#0x100000012d2' 2024-09-10T08:44:45.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4774 fp=#0x10000003937' 2024-09-10T08:44:45.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4806 fp=#0x10000004ab7' 2024-09-10T08:44:45.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4838 fp=#0x100000036f5' 2024-09-10T08:44:45.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4870 fp=#0x10000004aaa' 2024-09-10T08:44:45.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4902 fp=#0x10000003822' 2024-09-10T08:44:45.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4934 fp=#0x10000004a62' 2024-09-10T08:44:45.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4966 fp=#0x10000005044' 2024-09-10T08:44:45.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4998 fp=#0x100000051c2' 2024-09-10T08:44:45.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5030 fp=#0x1000000506b' 2024-09-10T08:44:45.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5062 fp=#0x10000005015' 2024-09-10T08:44:45.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5094 fp=#0x10000005097' 2024-09-10T08:44:45.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5126 fp=#0x10000001dce' 2024-09-10T08:44:45.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5158 fp=#0x10000001e25' 2024-09-10T08:44:45.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5190 fp=#0x10000001bfb' 2024-09-10T08:44:45.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5222 fp=#0x10000001cb7' 2024-09-10T08:44:45.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5254 fp=#0x100000036da' 2024-09-10T08:44:45.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5286 fp=#0x10000003cf7' 2024-09-10T08:44:45.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5318 fp=#0x10000003d97' 2024-09-10T08:44:45.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5350 fp=#0x100000038fc' 2024-09-10T08:44:45.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5382 fp=#0x1000000373d' 2024-09-10T08:44:45.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5414 fp=#0x10000004a81' 2024-09-10T08:44:45.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5542 fp=#0x1000000127a' 2024-09-10T08:44:45.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5574 fp=#0x10000000fc2' 2024-09-10T08:44:45.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5606 fp=#0x10000000f48' 2024-09-10T08:44:45.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5638 fp=#0x100000020ca' 2024-09-10T08:44:45.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5670 fp=#0x10000000f59' 2024-09-10T08:44:45.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5702 fp=#0x100000020ec' 2024-09-10T08:44:45.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5734 fp=#0x10000001054' 2024-09-10T08:44:45.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5766 fp=#0x1000000511c' 2024-09-10T08:44:45.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5798 fp=#0x100000044d0' 2024-09-10T08:44:45.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5830 fp=#0x100000044bd' 2024-09-10T08:44:45.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5862 fp=#0x10000004d1f' 2024-09-10T08:44:45.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5894 fp=#0x100000044a1' 2024-09-10T08:44:45.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5926 fp=#0x100000047b9' 2024-09-10T08:44:45.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5958 fp=#0x10000004382' 2024-09-10T08:44:45.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6150 fp=#0x1000000348b' 2024-09-10T08:44:45.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6214 fp=#0x10000004bc4' 2024-09-10T08:44:45.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6278 fp=#0x10000001e4b' 2024-09-10T08:44:45.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6310 fp=#0x10000001eec' 2024-09-10T08:44:45.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6342 fp=#0x1000000334e' 2024-09-10T08:44:45.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6374 fp=#0x10000003335' 2024-09-10T08:44:45.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6406 fp=#0x10000004f66' 2024-09-10T08:44:45.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6438 fp=#0x10000004b5f' 2024-09-10T08:44:45.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6470 fp=#0x10000004b10' 2024-09-10T08:44:45.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6502 fp=#0x10000004f1a' 2024-09-10T08:44:45.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6534 fp=#0x10000004bba' 2024-09-10T08:44:45.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6630 fp=#0x10000003baf' 2024-09-10T08:44:45.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6662 fp=#0x10000003c95' 2024-09-10T08:44:45.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6694 fp=#0x10000003a06' 2024-09-10T08:44:45.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6726 fp=#0x10000003b54' 2024-09-10T08:44:45.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6758 fp=#0x10000003a0d' 2024-09-10T08:44:45.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6790 fp=#0x10000004703' 2024-09-10T08:44:45.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6822 fp=#0x100000046cf' 2024-09-10T08:44:45.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6854 fp=#0x10000005315' 2024-09-10T08:44:45.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6886 fp=#0x100000045ea' 2024-09-10T08:44:45.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6918 fp=#0x10000004664' 2024-09-10T08:44:45.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6950 fp=#0x100000046dc' 2024-09-10T08:44:45.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6982 fp=#0x10000005027' 2024-09-10T08:44:45.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7014 fp=#0x10000002750' 2024-09-10T08:44:45.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7046 fp=#0x1000000501e' 2024-09-10T08:44:45.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7078 fp=#0x10000005082' 2024-09-10T08:44:45.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7110 fp=#0x10000004d0e' 2024-09-10T08:44:45.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7142 fp=#0x10000000645' 2024-09-10T08:44:45.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7174 fp=#0x1000000508f' 2024-09-10T08:44:45.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7270 fp=#0x1000000467d' 2024-09-10T08:44:45.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7302 fp=#0x100000050b3' 2024-09-10T08:44:45.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7366 fp=#0x100000050f7' 2024-09-10T08:44:45.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7398 fp=#0x10000004858' 2024-09-10T08:44:45.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7430 fp=#0x10000003d92' 2024-09-10T08:44:45.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7462 fp=#0x100000048d0' 2024-09-10T08:44:45.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7494 fp=#0x100000047c9' 2024-09-10T08:44:45.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7526 fp=#0x100000047f6' 2024-09-10T08:44:45.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7558 fp=#0x100000051be' 2024-09-10T08:44:45.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7622 fp=#0x1000000516b' 2024-09-10T08:44:45.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7654 fp=#0x10000001324' 2024-09-10T08:44:45.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7718 fp=#0x10000004f9d' 2024-09-10T08:44:45.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7750 fp=#0x10000004fb8' 2024-09-10T08:44:45.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7782 fp=#0x10000004f05' 2024-09-10T08:44:45.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7814 fp=#0x10000001488' 2024-09-10T08:44:45.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7846 fp=#0x10000001518' 2024-09-10T08:44:45.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7878 fp=#0x10000001426' 2024-09-10T08:44:45.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7910 fp=#0x1000000141b' 2024-09-10T08:44:45.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7942 fp=#0x100000015a5' 2024-09-10T08:44:45.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7974 fp=#0x10000004f39' 2024-09-10T08:44:45.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8006 fp=#0x100000014fb' 2024-09-10T08:44:45.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8038 fp=#0x100000049f1' 2024-09-10T08:44:45.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8070 fp=#0x100000015ee' 2024-09-10T08:44:45.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8102 fp=#0x100000016bd' 2024-09-10T08:44:45.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8134 fp=#0x10000004bab' 2024-09-10T08:44:45.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8166 fp=#0x10000002699' 2024-09-10T08:44:45.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8198 fp=#0x1000000255a' 2024-09-10T08:44:45.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8230 fp=#0x1000000268d' 2024-09-10T08:44:45.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8262 fp=#0x10000004285' 2024-09-10T08:44:45.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8294 fp=#0x1000000453a' 2024-09-10T08:44:45.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8326 fp=#0x1000000527e' 2024-09-10T08:44:45.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8358 fp=#0x10000005264' 2024-09-10T08:44:45.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8390 fp=#0x10000004232' 2024-09-10T08:44:45.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8422 fp=#0x100000042bc' 2024-09-10T08:44:45.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8454 fp=#0x10000003e52' 2024-09-10T08:44:45.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8486 fp=#0x10000003df4' 2024-09-10T08:44:45.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8518 fp=#0x10000003fd6' 2024-09-10T08:44:45.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8550 fp=#0x10000003efe' 2024-09-10T08:44:45.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8582 fp=#0x10000003dd5' 2024-09-10T08:44:45.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8614 fp=#0x10000003ef7' 2024-09-10T08:44:45.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8646 fp=#0x100000052d9' 2024-09-10T08:44:45.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8678 fp=#0x10000004016' 2024-09-10T08:44:45.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8710 fp=#0x10000003cdc' 2024-09-10T08:44:45.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8742 fp=#0x10000003d48' 2024-09-10T08:44:45.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8774 fp=#0x10000004f8b' 2024-09-10T08:44:45.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8806 fp=#0x10000003bed' 2024-09-10T08:44:45.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8838 fp=#0x10000001ad6' 2024-09-10T08:44:45.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8870 fp=#0x1000000405c' 2024-09-10T08:44:45.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8902 fp=#0x10000001ab5' 2024-09-10T08:44:45.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8934 fp=#0x10000004410' 2024-09-10T08:44:45.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8966 fp=#0x1000000439d' 2024-09-10T08:44:45.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8998 fp=#0x10000001b27' 2024-09-10T08:44:45.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9030 fp=#0x10000000da8' 2024-09-10T08:44:45.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9062 fp=#0x10000000d8d' 2024-09-10T08:44:45.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9094 fp=#0x10000000d0d' 2024-09-10T08:44:45.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9158 fp=#0x10000000ce8' 2024-09-10T08:44:45.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9190 fp=#0x10000000d25' 2024-09-10T08:44:45.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9222 fp=#0x10000000d50' 2024-09-10T08:44:45.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9254 fp=#0x10000000c4f' 2024-09-10T08:44:45.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9286 fp=#0x10000004d6b' 2024-09-10T08:44:45.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9318 fp=#0x10000000b37' 2024-09-10T08:44:45.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9350 fp=#0x10000004dda' 2024-09-10T08:44:45.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9382 fp=#0x100000049a5' 2024-09-10T08:44:45.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9414 fp=#0x10000000b25' 2024-09-10T08:44:45.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9446 fp=#0x10000004d96' 2024-09-10T08:44:45.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9478 fp=#0x10000000ed4' 2024-09-10T08:44:45.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9510 fp=#0x10000001a79' 2024-09-10T08:44:45.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9542 fp=#0x10000001abb' 2024-09-10T08:44:45.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9574 fp=#0x10000000ee1' 2024-09-10T08:44:45.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9606 fp=#0x10000001a86' 2024-09-10T08:44:45.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9638 fp=#0x100000049bf' 2024-09-10T08:44:45.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9670 fp=#0x100000047e4' 2024-09-10T08:44:45.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9702 fp=#0x10000004865' 2024-09-10T08:44:45.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9734 fp=#0x1000000431d' 2024-09-10T08:44:45.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9766 fp=#0x10000001899' 2024-09-10T08:44:45.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9798 fp=#0x100000017d1' 2024-09-10T08:44:45.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9830 fp=#0x100000043b7' 2024-09-10T08:44:45.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9894 fp=#0x10000001867' 2024-09-10T08:44:45.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9926 fp=#0x100000017a6' 2024-09-10T08:44:45.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9958 fp=#0x100000018ab' 2024-09-10T08:44:45.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9990 fp=#0x100000026b0' 2024-09-10T08:44:45.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10022 fp=#0x10000001591' 2024-09-10T08:44:45.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10054 fp=#0x10000001612' 2024-09-10T08:44:45.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10086 fp=#0x10000001eb9' 2024-09-10T08:44:45.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10118 fp=#0x1000000392b' 2024-09-10T08:44:45.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10150 fp=#0x10000001e37' 2024-09-10T08:44:45.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10182 fp=#0x10000001f18' 2024-09-10T08:44:45.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10214 fp=#0x10000004e00' 2024-09-10T08:44:45.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10246 fp=#0x10000003e33' 2024-09-10T08:44:45.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10278 fp=#0x10000002483' 2024-09-10T08:44:45.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10310 fp=#0x10000003828' 2024-09-10T08:44:45.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10342 fp=#0x100000013bb' 2024-09-10T08:44:45.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10374 fp=#0x10000003ab3' 2024-09-10T08:44:45.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10406 fp=#0x10000003512' 2024-09-10T08:44:45.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10438 fp=#0x1000000486f' 2024-09-10T08:44:45.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10470 fp=#0x10000003a25' 2024-09-10T08:44:45.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10502 fp=#0x100000012e2' 2024-09-10T08:44:45.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10534 fp=#0x100000012eb' 2024-09-10T08:44:45.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10566 fp=#0x100000013be' 2024-09-10T08:44:45.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10630 fp=#0x100000048f2' 2024-09-10T08:44:45.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10726 fp=#0x100000008c2' 2024-09-10T08:44:45.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10886 fp=#0x1000000055e' 2024-09-10T08:44:45.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10918 fp=#0x10000000a5c' 2024-09-10T08:44:45.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10950 fp=#0x10000000713' 2024-09-10T08:44:45.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10982 fp=#0x10000000984' 2024-09-10T08:44:45.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11046 fp=#0x1000000453b' 2024-09-10T08:44:45.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11078 fp=#0x10000003942' 2024-09-10T08:44:45.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11110 fp=#0x10000003922' 2024-09-10T08:44:45.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11142 fp=#0x10000003fb1' 2024-09-10T08:44:45.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11174 fp=#0x10000003e14' 2024-09-10T08:44:45.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11206 fp=#0x10000002129' 2024-09-10T08:44:45.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11238 fp=#0x10000002175' 2024-09-10T08:44:45.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11270 fp=#0x10000002066' 2024-09-10T08:44:45.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11302 fp=#0x10000003f92' 2024-09-10T08:44:45.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11334 fp=#0x10000003e2e' 2024-09-10T08:44:45.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11366 fp=#0x10000003d96' 2024-09-10T08:44:45.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11398 fp=#0x1000000199f' 2024-09-10T08:44:45.315 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11430 fp=#0x10000003e35' 2024-09-10T08:44:45.315 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11462 fp=#0x10000005352' 2024-09-10T08:44:45.315 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11494 fp=#0x10000003edf' 2024-09-10T08:44:45.316 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11526 fp=#0x10000001c5b' 2024-09-10T08:44:45.316 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11558 fp=#0x1000000515e' 2024-09-10T08:44:45.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11686 fp=#0x1000000519f' 2024-09-10T08:44:45.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11718 fp=#0x10000004e2d' 2024-09-10T08:44:45.318 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11750 fp=#0x10000004261' 2024-09-10T08:44:45.318 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11782 fp=#0x100000023ee' 2024-09-10T08:44:45.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11814 fp=#0x10000000994' 2024-09-10T08:44:45.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11846 fp=#0x10000001052' 2024-09-10T08:44:45.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11878 fp=#0x1000000173e' 2024-09-10T08:44:45.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11910 fp=#0x10000000efd' 2024-09-10T08:44:45.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11942 fp=#0x1000000110b' 2024-09-10T08:44:45.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12006 fp=#0x10000000eed' 2024-09-10T08:44:45.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12070 fp=#0x10000003508' 2024-09-10T08:44:45.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12102 fp=#0x100000033d3' 2024-09-10T08:44:45.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12134 fp=#0x100000045a4' 2024-09-10T08:44:45.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12166 fp=#0x10000003396' 2024-09-10T08:44:45.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12230 fp=#0x10000001111' 2024-09-10T08:44:45.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12262 fp=#0x10000000d5b' 2024-09-10T08:44:45.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12294 fp=#0x10000003bb8' 2024-09-10T08:44:45.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12326 fp=#0x100000051f6' 2024-09-10T08:44:45.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12358 fp=#0x10000003bac' 2024-09-10T08:44:45.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12390 fp=#0x10000000c14' 2024-09-10T08:44:45.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12422 fp=#0x10000004dde' 2024-09-10T08:44:45.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12454 fp=#0x10000004bf4' 2024-09-10T08:44:45.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12486 fp=#0x10000001f41' 2024-09-10T08:44:45.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12582 fp=#0x10000004cad' 2024-09-10T08:44:45.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12614 fp=#0x10000004c48' 2024-09-10T08:44:45.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12646 fp=#0x10000004c10' 2024-09-10T08:44:45.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12678 fp=#0x100000048c5' 2024-09-10T08:44:45.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12710 fp=#0x10000005368' 2024-09-10T08:44:45.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12742 fp=#0x100000053f7' 2024-09-10T08:44:45.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12774 fp=#0x100000053fc' 2024-09-10T08:44:45.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12806 fp=#0x10000005400' 2024-09-10T08:44:45.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12838 fp=#0x10000002127' 2024-09-10T08:44:45.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12870 fp=#0x100000053bc' 2024-09-10T08:44:45.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12902 fp=#0x10000005367' 2024-09-10T08:44:45.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12934 fp=#0x10000002205' 2024-09-10T08:44:45.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12966 fp=#0x10000005413' 2024-09-10T08:44:45.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'client_request(client.25157:74438 create owner_uid=1000, owner_gid=1333 #0x100000053d8/article-4.xml.tmp 2024-09-10T08:43:46.435535+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})' 2024-09-10T08:44:45.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13054 fp=#0x1000000541c' 2024-09-10T08:44:45.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13086 fp=#0x100000039b2' 2024-09-10T08:44:45.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13118 fp=#0x100000022de' 2024-09-10T08:44:45.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13150 fp=#0x1000000394a' 2024-09-10T08:44:45.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:303 fp=#0x100000015cf' 2024-09-10T08:44:45.334 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:455 fp=#0x10000004b03' 2024-09-10T08:44:45.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:487 fp=#0x1000000343a' 2024-09-10T08:44:45.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:519 fp=#0x100000033d2' 2024-09-10T08:44:45.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:583 fp=#0x100000033b3' 2024-09-10T08:44:45.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:615 fp=#0x1000000398c' 2024-09-10T08:44:45.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:775 fp=#0x10000000dd1' 2024-09-10T08:44:45.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:807 fp=#0x10000000f2b' 2024-09-10T08:44:45.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:839 fp=#0x10000003fbc' 2024-09-10T08:44:45.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:871 fp=#0x1000000457a' 2024-09-10T08:44:45.339 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:967 fp=#0x10000000fd5' 2024-09-10T08:44:45.339 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:999 fp=#0x1000000443f' 2024-09-10T08:44:45.340 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1063 fp=#0x10000004272' 2024-09-10T08:44:45.340 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1095 fp=#0x10000004223' 2024-09-10T08:44:45.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1191 fp=#0x10000004156' 2024-09-10T08:44:45.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1255 fp=#0x10000001fea' 2024-09-10T08:44:45.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1287 fp=#0x10000003655' 2024-09-10T08:44:45.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1319 fp=#0x100000035b0' 2024-09-10T08:44:45.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1639 fp=#0x1000000459b' 2024-09-10T08:44:45.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1671 fp=#0x100000045e3' 2024-09-10T08:44:45.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1735 fp=#0x1000000336f' 2024-09-10T08:44:45.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1799 fp=#0x10000002772' 2024-09-10T08:44:45.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2247 fp=#0x10000002664' 2024-09-10T08:44:45.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2279 fp=#0x100000025c6' 2024-09-10T08:44:45.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2567 fp=#0x10000001178' 2024-09-10T08:44:45.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2631 fp=#0x10000003298' 2024-09-10T08:44:45.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2695 fp=#0x10000001082' 2024-09-10T08:44:45.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2727 fp=#0x100000010fb' 2024-09-10T08:44:45.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2759 fp=#0x10000003b86' 2024-09-10T08:44:45.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2791 fp=#0x10000003677' 2024-09-10T08:44:45.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2823 fp=#0x10000004676' 2024-09-10T08:44:45.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2855 fp=#0x100000047ab' 2024-09-10T08:44:45.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2887 fp=#0x100000047a4' 2024-09-10T08:44:45.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2919 fp=#0x10000003673' 2024-09-10T08:44:45.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2951 fp=#0x10000004634' 2024-09-10T08:44:45.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2983 fp=#0x10000003439' 2024-09-10T08:44:45.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3015 fp=#0x10000000e88' 2024-09-10T08:44:45.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3079 fp=#0x10000004a07' 2024-09-10T08:44:45.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3111 fp=#0x100000049da' 2024-09-10T08:44:45.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3143 fp=#0x10000000e51' 2024-09-10T08:44:45.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3175 fp=#0x100000049ed' 2024-09-10T08:44:45.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3239 fp=#0x10000000772' 2024-09-10T08:44:45.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3271 fp=#0x1000000070e' 2024-09-10T08:44:45.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3335 fp=#0x10000000917' 2024-09-10T08:44:45.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3367 fp=#0x1000000074e' 2024-09-10T08:44:45.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3399 fp=#0x1000000078e' 2024-09-10T08:44:45.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3431 fp=#0x10000000768' 2024-09-10T08:44:45.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3463 fp=#0x1000000406b' 2024-09-10T08:44:45.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3495 fp=#0x10000003c4b' 2024-09-10T08:44:45.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3527 fp=#0x1000000410f' 2024-09-10T08:44:45.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3591 fp=#0x1000000495f' 2024-09-10T08:44:45.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3655 fp=#0x10000004c2a' 2024-09-10T08:44:45.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3687 fp=#0x10000004b55' 2024-09-10T08:44:45.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3719 fp=#0x10000004c01' 2024-09-10T08:44:45.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3751 fp=#0x10000003ec4' 2024-09-10T08:44:45.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3783 fp=#0x1000000257e' 2024-09-10T08:44:45.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3815 fp=#0x100000026dd' 2024-09-10T08:44:45.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3847 fp=#0x1000000256c' 2024-09-10T08:44:45.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3879 fp=#0x10000003f6a' 2024-09-10T08:44:45.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3911 fp=#0x10000003edd' 2024-09-10T08:44:45.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3943 fp=#0x10000000583' 2024-09-10T08:44:45.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3975 fp=#0x10000003f69' 2024-09-10T08:44:45.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4007 fp=#0x1000000206c' 2024-09-10T08:44:45.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4039 fp=#0x1000000100b' 2024-09-10T08:44:45.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4071 fp=#0x10000000fa6' 2024-09-10T08:44:45.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4103 fp=#0x10000004e12' 2024-09-10T08:44:45.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4135 fp=#0x1000000478d' 2024-09-10T08:44:45.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4167 fp=#0x10000000ff2' 2024-09-10T08:44:45.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4199 fp=#0x10000000f3d' 2024-09-10T08:44:45.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4231 fp=#0x10000003aee' 2024-09-10T08:44:45.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4263 fp=#0x10000003b41' 2024-09-10T08:44:45.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4295 fp=#0x10000004b90' 2024-09-10T08:44:45.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4327 fp=#0x10000004044' 2024-09-10T08:44:45.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4359 fp=#0x10000003ad4' 2024-09-10T08:44:45.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4391 fp=#0x10000001f51' 2024-09-10T08:44:45.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4487 fp=#0x10000001247' 2024-09-10T08:44:45.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4519 fp=#0x100000050ff' 2024-09-10T08:44:45.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4551 fp=#0x1000000122d' 2024-09-10T08:44:45.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4583 fp=#0x100000039ac' 2024-09-10T08:44:45.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4615 fp=#0x10000001342' 2024-09-10T08:44:45.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4647 fp=#0x100000012b5' 2024-09-10T08:44:45.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4679 fp=#0x100000012c9' 2024-09-10T08:44:45.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4711 fp=#0x100000014c8' 2024-09-10T08:44:45.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4743 fp=#0x10000001351' 2024-09-10T08:44:45.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4775 fp=#0x10000003917' 2024-09-10T08:44:45.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4807 fp=#0x1000000380c' 2024-09-10T08:44:45.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4839 fp=#0x100000037bd' 2024-09-10T08:44:45.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4871 fp=#0x10000003729' 2024-09-10T08:44:45.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4903 fp=#0x10000003787' 2024-09-10T08:44:45.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4935 fp=#0x100000037db' 2024-09-10T08:44:45.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4967 fp=#0x10000005098' 2024-09-10T08:44:45.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4999 fp=#0x1000000505e' 2024-09-10T08:44:45.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5031 fp=#0x10000005140' 2024-09-10T08:44:45.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5063 fp=#0x100000051e2' 2024-09-10T08:44:45.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5095 fp=#0x100000051c4' 2024-09-10T08:44:45.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5127 fp=#0x10000001eca' 2024-09-10T08:44:45.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5159 fp=#0x10000001d46' 2024-09-10T08:44:45.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5191 fp=#0x10000001cf4' 2024-09-10T08:44:45.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5223 fp=#0x10000001e69' 2024-09-10T08:44:45.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5255 fp=#0x10000004ab1' 2024-09-10T08:44:45.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5287 fp=#0x100000036bd' 2024-09-10T08:44:45.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5319 fp=#0x100000036ef' 2024-09-10T08:44:45.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5351 fp=#0x10000003d09' 2024-09-10T08:44:45.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5383 fp=#0x100000036fc' 2024-09-10T08:44:45.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5415 fp=#0x10000003dcc' 2024-09-10T08:44:45.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5447 fp=#0x100000012b8' 2024-09-10T08:44:45.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5479 fp=#0x1000000124d' 2024-09-10T08:44:45.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5511 fp=#0x100000012bc' 2024-09-10T08:44:45.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5543 fp=#0x100000014e6' 2024-09-10T08:44:45.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5575 fp=#0x10000005115' 2024-09-10T08:44:45.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5607 fp=#0x1000000211b' 2024-09-10T08:44:45.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5639 fp=#0x10000000fba' 2024-09-10T08:44:45.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5671 fp=#0x10000005095' 2024-09-10T08:44:45.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5703 fp=#0x10000000f69' 2024-09-10T08:44:45.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5735 fp=#0x1000000111d' 2024-09-10T08:44:45.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5767 fp=#0x100000050e2' 2024-09-10T08:44:45.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5799 fp=#0x10000004308' 2024-09-10T08:44:45.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5831 fp=#0x100000044a9' 2024-09-10T08:44:45.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5863 fp=#0x100000043b5' 2024-09-10T08:44:45.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5895 fp=#0x10000004422' 2024-09-10T08:44:45.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5927 fp=#0x100000044db' 2024-09-10T08:44:45.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5959 fp=#0x100000042f6' 2024-09-10T08:44:45.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6087 fp=#0x100000035eb' 2024-09-10T08:44:45.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6183 fp=#0x100000034df' 2024-09-10T08:44:45.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6215 fp=#0x1000000359d' 2024-09-10T08:44:45.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6279 fp=#0x10000001e2e' 2024-09-10T08:44:45.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6311 fp=#0x10000003427' 2024-09-10T08:44:45.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6343 fp=#0x10000003376' 2024-09-10T08:44:45.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6375 fp=#0x10000002563' 2024-09-10T08:44:45.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6407 fp=#0x10000004c40' 2024-09-10T08:44:45.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6439 fp=#0x10000004f92' 2024-09-10T08:44:45.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6471 fp=#0x10000004b13' 2024-09-10T08:44:45.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6503 fp=#0x10000004df6' 2024-09-10T08:44:45.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6535 fp=#0x10000004cba' 2024-09-10T08:44:45.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6567 fp=#0x10000004110' 2024-09-10T08:44:45.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6599 fp=#0x10000003fa5' 2024-09-10T08:44:45.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6631 fp=#0x10000003d88' 2024-09-10T08:44:45.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6663 fp=#0x10000003bd3' 2024-09-10T08:44:45.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6695 fp=#0x10000003a53' 2024-09-10T08:44:45.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6727 fp=#0x100000039ee' 2024-09-10T08:44:45.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6759 fp=#0x10000003b6e' 2024-09-10T08:44:45.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6791 fp=#0x10000005365' 2024-09-10T08:44:45.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6823 fp=#0x100000052d4' 2024-09-10T08:44:45.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6855 fp=#0x10000004609' 2024-09-10T08:44:45.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6887 fp=#0x100000046e6' 2024-09-10T08:44:45.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6919 fp=#0x10000004567' 2024-09-10T08:44:45.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6951 fp=#0x1000000470a' 2024-09-10T08:44:45.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6983 fp=#0x1000000065f' 2024-09-10T08:44:45.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7015 fp=#0x10000005019' 2024-09-10T08:44:45.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7047 fp=#0x10000005093' 2024-09-10T08:44:45.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7079 fp=#0x1000000066c' 2024-09-10T08:44:45.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7111 fp=#0x100000006a4' 2024-09-10T08:44:45.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7143 fp=#0x10000004fce' 2024-09-10T08:44:45.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7175 fp=#0x10000005030' 2024-09-10T08:44:45.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7303 fp=#0x100000050d3' 2024-09-10T08:44:45.402 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7335 fp=#0x1000000506c' 2024-09-10T08:44:45.402 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7367 fp=#0x100000050dd' 2024-09-10T08:44:45.402 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7399 fp=#0x100000049db' 2024-09-10T08:44:45.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7431 fp=#0x10000002394' 2024-09-10T08:44:45.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7463 fp=#0x1000000490e' 2024-09-10T08:44:45.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7495 fp=#0x100000023a7' 2024-09-10T08:44:45.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7527 fp=#0x100000052eb' 2024-09-10T08:44:45.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7559 fp=#0x10000001356' 2024-09-10T08:44:45.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7655 fp=#0x10000001375' 2024-09-10T08:44:45.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7687 fp=#0x100000051c5' 2024-09-10T08:44:45.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7719 fp=#0x10000004e47' 2024-09-10T08:44:45.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7751 fp=#0x10000004e81' 2024-09-10T08:44:45.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7783 fp=#0x10000004e8e' 2024-09-10T08:44:45.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7815 fp=#0x1000000170f' 2024-09-10T08:44:45.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7847 fp=#0x100000015c5' 2024-09-10T08:44:45.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7879 fp=#0x1000000151e' 2024-09-10T08:44:45.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7911 fp=#0x10000001572' 2024-09-10T08:44:45.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7943 fp=#0x100000015d1' 2024-09-10T08:44:45.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7975 fp=#0x10000004cae' 2024-09-10T08:44:45.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8039 fp=#0x1000000168d' 2024-09-10T08:44:45.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8071 fp=#0x10000004017' 2024-09-10T08:44:45.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8103 fp=#0x100000016a5' 2024-09-10T08:44:45.410 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8135 fp=#0x100000015da' 2024-09-10T08:44:45.410 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8167 fp=#0x100000024f7' 2024-09-10T08:44:45.410 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8199 fp=#0x1000000258a' 2024-09-10T08:44:45.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8231 fp=#0x10000004cc4' 2024-09-10T08:44:45.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8263 fp=#0x1000000425c' 2024-09-10T08:44:45.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8295 fp=#0x1000000456f' 2024-09-10T08:44:45.412 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8327 fp=#0x10000005254' 2024-09-10T08:44:45.412 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8359 fp=#0x10000005247' 2024-09-10T08:44:45.412 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8391 fp=#0x10000005250' 2024-09-10T08:44:45.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8423 fp=#0x1000000460c' 2024-09-10T08:44:45.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8455 fp=#0x10000004021' 2024-09-10T08:44:45.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8487 fp=#0x10000004239' 2024-09-10T08:44:45.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8519 fp=#0x10000003e7f' 2024-09-10T08:44:45.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8551 fp=#0x10000003fba' 2024-09-10T08:44:45.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8583 fp=#0x10000004106' 2024-09-10T08:44:45.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8615 fp=#0x10000003f1a' 2024-09-10T08:44:45.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8647 fp=#0x10000003bc9' 2024-09-10T08:44:45.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8679 fp=#0x10000003ddf' 2024-09-10T08:44:45.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8711 fp=#0x10000003d2a' 2024-09-10T08:44:45.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8743 fp=#0x10000003c5e' 2024-09-10T08:44:45.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8775 fp=#0x10000003df6' 2024-09-10T08:44:45.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8807 fp=#0x10000003d0b' 2024-09-10T08:44:45.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8839 fp=#0x1000000440b' 2024-09-10T08:44:45.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8871 fp=#0x10000001d29' 2024-09-10T08:44:45.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8903 fp=#0x10000004432' 2024-09-10T08:44:45.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8935 fp=#0x1000000436c' 2024-09-10T08:44:45.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8967 fp=#0x10000001be5' 2024-09-10T08:44:45.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8999 fp=#0x10000001bb4' 2024-09-10T08:44:45.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9031 fp=#0x100000011f5' 2024-09-10T08:44:45.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9063 fp=#0x10000000d78' 2024-09-10T08:44:45.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9095 fp=#0x10000004248' 2024-09-10T08:44:45.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9127 fp=#0x10000002448' 2024-09-10T08:44:45.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9159 fp=#0x10000000d60' 2024-09-10T08:44:45.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9191 fp=#0x10000004055' 2024-09-10T08:44:45.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9223 fp=#0x10000001fd8' 2024-09-10T08:44:45.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9255 fp=#0x10000000ba6' 2024-09-10T08:44:45.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9287 fp=#0x10000000bbd' 2024-09-10T08:44:45.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9319 fp=#0x10000001c13' 2024-09-10T08:44:45.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9351 fp=#0x10000000ae3' 2024-09-10T08:44:45.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9383 fp=#0x10000004cf4' 2024-09-10T08:44:45.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9415 fp=#0x10000004d38' 2024-09-10T08:44:45.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9447 fp=#0x10000004d61' 2024-09-10T08:44:45.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9479 fp=#0x100000049b2' 2024-09-10T08:44:45.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9511 fp=#0x10000000ee3' 2024-09-10T08:44:45.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9543 fp=#0x100000035ab' 2024-09-10T08:44:45.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9575 fp=#0x10000000ec4' 2024-09-10T08:44:45.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9607 fp=#0x10000001af2' 2024-09-10T08:44:45.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9639 fp=#0x1000000495c' 2024-09-10T08:44:45.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9671 fp=#0x10000004836' 2024-09-10T08:44:45.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9703 fp=#0x10000004844' 2024-09-10T08:44:45.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9735 fp=#0x10000001b73' 2024-09-10T08:44:45.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9767 fp=#0x10000001ae7' 2024-09-10T08:44:45.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9799 fp=#0x10000004cb3' 2024-09-10T08:44:45.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9831 fp=#0x10000004cac' 2024-09-10T08:44:45.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9863 fp=#0x10000004ce6' 2024-09-10T08:44:45.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9895 fp=#0x100000017bd' 2024-09-10T08:44:45.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9927 fp=#0x10000001a40' 2024-09-10T08:44:45.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9959 fp=#0x1000000157a' 2024-09-10T08:44:45.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9991 fp=#0x100000015eb' 2024-09-10T08:44:45.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10023 fp=#0x100000019ae' 2024-09-10T08:44:45.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10055 fp=#0x1000000193a' 2024-09-10T08:44:45.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10087 fp=#0x10000004053' 2024-09-10T08:44:45.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10119 fp=#0x10000001eda' 2024-09-10T08:44:45.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10151 fp=#0x10000001e54' 2024-09-10T08:44:45.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10183 fp=#0x10000002667' 2024-09-10T08:44:45.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10215 fp=#0x100000027c4' 2024-09-10T08:44:45.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10247 fp=#0x10000003a81' 2024-09-10T08:44:45.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10279 fp=#0x10000002387' 2024-09-10T08:44:45.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10311 fp=#0x10000002375' 2024-09-10T08:44:45.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10343 fp=#0x10000003b15' 2024-09-10T08:44:45.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10375 fp=#0x100000013dc' 2024-09-10T08:44:45.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10407 fp=#0x10000003934' 2024-09-10T08:44:45.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10439 fp=#0x1000000136a' 2024-09-10T08:44:45.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10471 fp=#0x10000003a58' 2024-09-10T08:44:45.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10503 fp=#0x100000013da' 2024-09-10T08:44:45.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10535 fp=#0x100000038e8' 2024-09-10T08:44:45.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10567 fp=#0x10000001331' 2024-09-10T08:44:45.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10631 fp=#0x100000048fb' 2024-09-10T08:44:45.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10823 fp=#0x10000001e6c' 2024-09-10T08:44:45.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10855 fp=#0x10000000562' 2024-09-10T08:44:45.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10919 fp=#0x10000001044' 2024-09-10T08:44:45.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10951 fp=#0x10000001ca1' 2024-09-10T08:44:45.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10983 fp=#0x1000000090f' 2024-09-10T08:44:45.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11047 fp=#0x100000039de' 2024-09-10T08:44:45.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11079 fp=#0x10000003a1b' 2024-09-10T08:44:45.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11111 fp=#0x10000002077' 2024-09-10T08:44:45.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11143 fp=#0x100000021a0' 2024-09-10T08:44:45.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11175 fp=#0x10000002158' 2024-09-10T08:44:45.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11207 fp=#0x100000048f3' 2024-09-10T08:44:45.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11239 fp=#0x10000002085' 2024-09-10T08:44:45.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11271 fp=#0x10000003fc7' 2024-09-10T08:44:45.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11303 fp=#0x10000005284' 2024-09-10T08:44:45.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11335 fp=#0x10000003e42' 2024-09-10T08:44:45.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11367 fp=#0x10000003de8' 2024-09-10T08:44:45.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11399 fp=#0x10000003e62' 2024-09-10T08:44:45.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11431 fp=#0x10000001b1f' 2024-09-10T08:44:45.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11495 fp=#0x10000003e0c' 2024-09-10T08:44:45.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11527 fp=#0x10000003e67' 2024-09-10T08:44:45.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11559 fp=#0x10000005175' 2024-09-10T08:44:45.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11687 fp=#0x10000005131' 2024-09-10T08:44:45.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11719 fp=#0x10000004dea' 2024-09-10T08:44:45.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11751 fp=#0x10000004266' 2024-09-10T08:44:45.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11783 fp=#0x10000004db3' 2024-09-10T08:44:45.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11815 fp=#0x10000000908' 2024-09-10T08:44:45.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11847 fp=#0x1000000096f' 2024-09-10T08:44:45.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11879 fp=#0x100000008c3' 2024-09-10T08:44:45.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11911 fp=#0x100000052c6' 2024-09-10T08:44:45.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11943 fp=#0x10000001d7f' 2024-09-10T08:44:45.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11975 fp=#0x100000052b2' 2024-09-10T08:44:45.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12007 fp=#0x10000004884' 2024-09-10T08:44:45.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12039 fp=#0x10000001121' 2024-09-10T08:44:45.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12071 fp=#0x10000003528' 2024-09-10T08:44:45.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12103 fp=#0x1000000448c' 2024-09-10T08:44:45.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12135 fp=#0x1000000456c' 2024-09-10T08:44:45.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12167 fp=#0x100000043b2' 2024-09-10T08:44:45.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12199 fp=#0x10000003a55' 2024-09-10T08:44:45.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12231 fp=#0x10000002576' 2024-09-10T08:44:45.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12263 fp=#0x10000000c52' 2024-09-10T08:44:45.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12295 fp=#0x10000000bf7' 2024-09-10T08:44:45.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12327 fp=#0x100000051ee' 2024-09-10T08:44:45.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12359 fp=#0x10000003a95' 2024-09-10T08:44:45.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12391 fp=#0x10000000cbe' 2024-09-10T08:44:45.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12423 fp=#0x100000051f0' 2024-09-10T08:44:45.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12455 fp=#0x1000000177a' 2024-09-10T08:44:45.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12519 fp=#0x10000004c74' 2024-09-10T08:44:45.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12551 fp=#0x10000004542' 2024-09-10T08:44:45.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12583 fp=#0x10000004c91' 2024-09-10T08:44:45.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12615 fp=#0x10000004b75' 2024-09-10T08:44:45.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12647 fp=#0x10000004c37' 2024-09-10T08:44:45.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12679 fp=#0x100000048dd' 2024-09-10T08:44:45.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12711 fp=#0x100000049f0' 2024-09-10T08:44:45.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12743 fp=#0x1000000537c' 2024-09-10T08:44:45.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12775 fp=#0x100000053da' 2024-09-10T08:44:45.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12807 fp=#0x10000002266' 2024-09-10T08:44:45.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12839 fp=#0x1000000530e' 2024-09-10T08:44:45.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12871 fp=#0x100000053ec' 2024-09-10T08:44:45.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12903 fp=#0x1000000215c' 2024-09-10T08:44:45.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12935 fp=#0x10000002555' 2024-09-10T08:44:45.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12967 fp=#0x100000053ba' 2024-09-10T08:44:45.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12999 fp=#0x1000000276b' 2024-09-10T08:44:45.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'client_request(client.25157:74439 create owner_uid=1000, owner_gid=1333 #0x100000020b6/comment-19.xml.tmp 2024-09-10T08:43:46.435535+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})' 2024-09-10T08:44:45.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13087 fp=#0x10000003a51' 2024-09-10T08:44:45.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13119 fp=#0x1000000226d' 2024-09-10T08:44:45.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13151 fp=#0x10000005386' 2024-09-10T08:44:45.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:304 fp=#0x100000024ec' 2024-09-10T08:44:45.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:392 fp=#0x10000004657' 2024-09-10T08:44:45.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:424 fp=#0x10000004b70' 2024-09-10T08:44:45.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:488 fp=#0x10000003560' 2024-09-10T08:44:45.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:520 fp=#0x10000003401' 2024-09-10T08:44:45.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:552 fp=#0x1000000334b' 2024-09-10T08:44:45.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:776 fp=#0x100000045bc' 2024-09-10T08:44:45.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:808 fp=#0x1000000458a' 2024-09-10T08:44:45.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:840 fp=#0x10000000de9' 2024-09-10T08:44:45.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:872 fp=#0x10000000d8c' 2024-09-10T08:44:45.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:904 fp=#0x10000003fb5' 2024-09-10T08:44:45.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:936 fp=#0x10000000d97' 2024-09-10T08:44:45.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:968 fp=#0x1000000449e' 2024-09-10T08:44:45.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1000 fp=#0x10000004477' 2024-09-10T08:44:45.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1032 fp=#0x10000000e99' 2024-09-10T08:44:45.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1096 fp=#0x100000041bd' 2024-09-10T08:44:45.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1128 fp=#0x100000040e7' 2024-09-10T08:44:45.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1160 fp=#0x1000000425e' 2024-09-10T08:44:45.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1192 fp=#0x1000000426c' 2024-09-10T08:44:45.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1256 fp=#0x10000001faf' 2024-09-10T08:44:45.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1320 fp=#0x1000000206f' 2024-09-10T08:44:45.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1640 fp=#0x10000004a2d' 2024-09-10T08:44:45.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1672 fp=#0x10000004a40' 2024-09-10T08:44:45.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2088 fp=#0x100000021ae' 2024-09-10T08:44:45.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2216 fp=#0x100000026a2' 2024-09-10T08:44:45.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2248 fp=#0x1000000264b' 2024-09-10T08:44:45.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2600 fp=#0x1000000111e' 2024-09-10T08:44:45.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2632 fp=#0x100000041ed' 2024-09-10T08:44:45.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2664 fp=#0x10000001881' 2024-09-10T08:44:45.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2760 fp=#0x10000003b46' 2024-09-10T08:44:45.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2792 fp=#0x100000045ed' 2024-09-10T08:44:45.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2824 fp=#0x1000000353f' 2024-09-10T08:44:45.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2856 fp=#0x10000003537' 2024-09-10T08:44:45.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2888 fp=#0x100000046cb' 2024-09-10T08:44:45.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2920 fp=#0x10000003563' 2024-09-10T08:44:45.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2952 fp=#0x1000000478a' 2024-09-10T08:44:45.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3016 fp=#0x10000000e9a' 2024-09-10T08:44:45.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3112 fp=#0x10000004a02' 2024-09-10T08:44:45.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3144 fp=#0x1000000199d' 2024-09-10T08:44:45.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3176 fp=#0x10000001957' 2024-09-10T08:44:45.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3208 fp=#0x10000001997' 2024-09-10T08:44:45.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3240 fp=#0x10000003c8f' 2024-09-10T08:44:45.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3272 fp=#0x10000000799' 2024-09-10T08:44:45.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3304 fp=#0x100000007e1' 2024-09-10T08:44:45.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3336 fp=#0x10000003c9c' 2024-09-10T08:44:45.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3368 fp=#0x1000000070f' 2024-09-10T08:44:45.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3400 fp=#0x10000000787' 2024-09-10T08:44:45.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3432 fp=#0x10000003c41' 2024-09-10T08:44:45.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3464 fp=#0x10000003c42' 2024-09-10T08:44:45.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3528 fp=#0x10000004043' 2024-09-10T08:44:45.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3592 fp=#0x10000004054' 2024-09-10T08:44:45.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3624 fp=#0x10000004a9e' 2024-09-10T08:44:45.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3656 fp=#0x10000004ad8' 2024-09-10T08:44:45.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3688 fp=#0x10000004b2e' 2024-09-10T08:44:45.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3720 fp=#0x10000004ba5' 2024-09-10T08:44:45.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3752 fp=#0x10000002536' 2024-09-10T08:44:45.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3784 fp=#0x10000003e89' 2024-09-10T08:44:45.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3816 fp=#0x1000000275f' 2024-09-10T08:44:45.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3848 fp=#0x10000002529' 2024-09-10T08:44:45.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3880 fp=#0x10000003e8d' 2024-09-10T08:44:45.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3912 fp=#0x100000006ad' 2024-09-10T08:44:45.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3944 fp=#0x1000000055b' 2024-09-10T08:44:45.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3976 fp=#0x10000000563' 2024-09-10T08:44:45.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4008 fp=#0x10000003ad9' 2024-09-10T08:44:45.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4040 fp=#0x10000000fa0' 2024-09-10T08:44:45.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4072 fp=#0x100000052ef' 2024-09-10T08:44:45.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4104 fp=#0x10000004df3' 2024-09-10T08:44:45.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4136 fp=#0x1000000412f' 2024-09-10T08:44:45.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4168 fp=#0x10000000f82' 2024-09-10T08:44:45.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4200 fp=#0x10000004d9e' 2024-09-10T08:44:45.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4232 fp=#0x10000003b92' 2024-09-10T08:44:45.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4264 fp=#0x10000004052' 2024-09-10T08:44:45.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4296 fp=#0x1000000418c' 2024-09-10T08:44:45.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4328 fp=#0x1000000414b' 2024-09-10T08:44:45.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4360 fp=#0x10000003aff' 2024-09-10T08:44:45.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4392 fp=#0x10000001ee5' 2024-09-10T08:44:45.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4424 fp=#0x10000001f0b' 2024-09-10T08:44:45.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4488 fp=#0x10000005127' 2024-09-10T08:44:45.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4520 fp=#0x10000005150' 2024-09-10T08:44:45.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4552 fp=#0x100000050dc' 2024-09-10T08:44:45.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4584 fp=#0x1000000156c' 2024-09-10T08:44:45.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4616 fp=#0x10000001392' 2024-09-10T08:44:45.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4648 fp=#0x10000001496' 2024-09-10T08:44:45.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4680 fp=#0x10000001447' 2024-09-10T08:44:45.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4712 fp=#0x10000001319' 2024-09-10T08:44:45.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4744 fp=#0x1000000130d' 2024-09-10T08:44:45.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4776 fp=#0x10000004041' 2024-09-10T08:44:45.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4808 fp=#0x10000004a66' 2024-09-10T08:44:45.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4840 fp=#0x10000003753' 2024-09-10T08:44:45.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4872 fp=#0x1000000374e' 2024-09-10T08:44:45.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4904 fp=#0x100000037ac' 2024-09-10T08:44:45.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4936 fp=#0x10000003794' 2024-09-10T08:44:45.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4968 fp=#0x10000005087' 2024-09-10T08:44:45.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5000 fp=#0x100000051dc' 2024-09-10T08:44:45.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5032 fp=#0x10000005164' 2024-09-10T08:44:45.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5064 fp=#0x10000005100' 2024-09-10T08:44:45.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5096 fp=#0x10000005035' 2024-09-10T08:44:45.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5128 fp=#0x10000002417' 2024-09-10T08:44:45.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5160 fp=#0x10000001ead' 2024-09-10T08:44:45.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5192 fp=#0x10000001d75' 2024-09-10T08:44:45.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5224 fp=#0x10000003717' 2024-09-10T08:44:45.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5256 fp=#0x10000003896' 2024-09-10T08:44:45.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5288 fp=#0x10000003da6' 2024-09-10T08:44:45.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5320 fp=#0x10000004ab6' 2024-09-10T08:44:45.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5352 fp=#0x10000004aaf' 2024-09-10T08:44:45.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5384 fp=#0x100000038bc' 2024-09-10T08:44:45.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5416 fp=#0x100000035c7' 2024-09-10T08:44:45.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5448 fp=#0x100000012bf' 2024-09-10T08:44:45.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5480 fp=#0x10000001223' 2024-09-10T08:44:45.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5576 fp=#0x100000010a1' 2024-09-10T08:44:45.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5608 fp=#0x10000000fb1' 2024-09-10T08:44:45.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5640 fp=#0x10000000f9e' 2024-09-10T08:44:45.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5672 fp=#0x100000050e6' 2024-09-10T08:44:45.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5704 fp=#0x10000000f66' 2024-09-10T08:44:45.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5736 fp=#0x10000000fcc' 2024-09-10T08:44:45.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5768 fp=#0x1000000117e' 2024-09-10T08:44:45.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5800 fp=#0x10000004396' 2024-09-10T08:44:45.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5832 fp=#0x10000004499' 2024-09-10T08:44:45.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5864 fp=#0x10000004d09' 2024-09-10T08:44:45.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5896 fp=#0x10000004ca9' 2024-09-10T08:44:45.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5928 fp=#0x10000004c42' 2024-09-10T08:44:45.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5960 fp=#0x10000004d46' 2024-09-10T08:44:45.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6152 fp=#0x10000003524' 2024-09-10T08:44:45.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6280 fp=#0x10000001f45' 2024-09-10T08:44:45.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6312 fp=#0x10000001e0d' 2024-09-10T08:44:45.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6344 fp=#0x10000001e8d' 2024-09-10T08:44:45.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6376 fp=#0x10000001ec4' 2024-09-10T08:44:45.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6408 fp=#0x10000004bdc' 2024-09-10T08:44:45.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6440 fp=#0x10000004e48' 2024-09-10T08:44:45.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6472 fp=#0x10000004ca6' 2024-09-10T08:44:45.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6504 fp=#0x10000004c79' 2024-09-10T08:44:45.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6536 fp=#0x10000004b8e' 2024-09-10T08:44:45.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6600 fp=#0x10000004769' 2024-09-10T08:44:45.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6632 fp=#0x10000003935' 2024-09-10T08:44:45.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6664 fp=#0x10000003b1f' 2024-09-10T08:44:45.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6696 fp=#0x10000003d6f' 2024-09-10T08:44:45.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6728 fp=#0x10000003a24' 2024-09-10T08:44:45.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6760 fp=#0x10000003960' 2024-09-10T08:44:45.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6792 fp=#0x100000047f7' 2024-09-10T08:44:45.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6824 fp=#0x100000052f8' 2024-09-10T08:44:45.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6856 fp=#0x10000005309' 2024-09-10T08:44:45.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6888 fp=#0x100000052c4' 2024-09-10T08:44:45.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6920 fp=#0x10000004717' 2024-09-10T08:44:45.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6952 fp=#0x100000045db' 2024-09-10T08:44:45.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6984 fp=#0x10000000594' 2024-09-10T08:44:45.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7016 fp=#0x10000005014' 2024-09-10T08:44:45.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7048 fp=#0x10000004fb5' 2024-09-10T08:44:45.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7080 fp=#0x100000007ec' 2024-09-10T08:44:45.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7112 fp=#0x10000000649' 2024-09-10T08:44:45.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7144 fp=#0x10000004fec' 2024-09-10T08:44:45.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7176 fp=#0x100000005c9' 2024-09-10T08:44:45.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7240 fp=#0x100000050ed' 2024-09-10T08:44:45.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7272 fp=#0x100000050d7' 2024-09-10T08:44:45.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7304 fp=#0x1000000504f' 2024-09-10T08:44:45.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7368 fp=#0x10000004a04' 2024-09-10T08:44:45.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7400 fp=#0x1000000249c' 2024-09-10T08:44:45.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7432 fp=#0x100000047d1' 2024-09-10T08:44:45.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7464 fp=#0x100000047e0' 2024-09-10T08:44:45.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7496 fp=#0x10000004847' 2024-09-10T08:44:45.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7528 fp=#0x10000004742' 2024-09-10T08:44:45.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7560 fp=#0x100000012e3' 2024-09-10T08:44:45.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7592 fp=#0x10000005183' 2024-09-10T08:44:45.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7656 fp=#0x10000001353' 2024-09-10T08:44:45.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7688 fp=#0x100000051b0' 2024-09-10T08:44:45.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7720 fp=#0x10000004f99' 2024-09-10T08:44:45.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7752 fp=#0x10000004f95' 2024-09-10T08:44:45.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7784 fp=#0x10000004e32' 2024-09-10T08:44:45.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7816 fp=#0x10000004efc' 2024-09-10T08:44:45.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7848 fp=#0x100000015f9' 2024-09-10T08:44:45.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7880 fp=#0x100000032bc' 2024-09-10T08:44:45.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7912 fp=#0x10000004e21' 2024-09-10T08:44:45.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7944 fp=#0x10000004d97' 2024-09-10T08:44:45.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7976 fp=#0x10000002207' 2024-09-10T08:44:45.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8008 fp=#0x1000000153e' 2024-09-10T08:44:45.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8040 fp=#0x100000015df' 2024-09-10T08:44:45.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8072 fp=#0x10000004628' 2024-09-10T08:44:45.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8104 fp=#0x100000015f8' 2024-09-10T08:44:45.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8136 fp=#0x100000026e2' 2024-09-10T08:44:45.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8168 fp=#0x10000002673' 2024-09-10T08:44:45.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8200 fp=#0x10000002635' 2024-09-10T08:44:45.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8232 fp=#0x100000025ed' 2024-09-10T08:44:45.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8264 fp=#0x10000004146' 2024-09-10T08:44:45.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8296 fp=#0x10000005269' 2024-09-10T08:44:45.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8328 fp=#0x10000004273' 2024-09-10T08:44:45.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8360 fp=#0x100000040e8' 2024-09-10T08:44:45.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8392 fp=#0x10000004426' 2024-09-10T08:44:45.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8424 fp=#0x10000004129' 2024-09-10T08:44:45.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8456 fp=#0x100000045fd' 2024-09-10T08:44:45.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8488 fp=#0x10000004516' 2024-09-10T08:44:45.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8520 fp=#0x10000003ec2' 2024-09-10T08:44:45.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8552 fp=#0x10000003ff4' 2024-09-10T08:44:45.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8584 fp=#0x100000044f6' 2024-09-10T08:44:45.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8616 fp=#0x100000040c4' 2024-09-10T08:44:45.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8648 fp=#0x10000004fca' 2024-09-10T08:44:45.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8680 fp=#0x10000004f94' 2024-09-10T08:44:45.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8712 fp=#0x100000040ad' 2024-09-10T08:44:45.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8744 fp=#0x10000003ecf' 2024-09-10T08:44:45.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8776 fp=#0x10000003fd1' 2024-09-10T08:44:45.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8808 fp=#0x10000004008' 2024-09-10T08:44:45.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8840 fp=#0x10000001b54' 2024-09-10T08:44:45.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8872 fp=#0x10000001cf5' 2024-09-10T08:44:45.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8904 fp=#0x1000000449f' 2024-09-10T08:44:45.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8936 fp=#0x10000001b4a' 2024-09-10T08:44:45.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8968 fp=#0x100000035b7' 2024-09-10T08:44:45.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9000 fp=#0x10000001bf7' 2024-09-10T08:44:45.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9032 fp=#0x10000003286' 2024-09-10T08:44:45.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9064 fp=#0x10000000d48' 2024-09-10T08:44:45.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9096 fp=#0x100000040b7' 2024-09-10T08:44:45.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9160 fp=#0x10000001170' 2024-09-10T08:44:45.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9192 fp=#0x1000000244e' 2024-09-10T08:44:45.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9224 fp=#0x1000000429d' 2024-09-10T08:44:45.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9256 fp=#0x10000000c6a' 2024-09-10T08:44:45.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9288 fp=#0x10000004cd7' 2024-09-10T08:44:45.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9320 fp=#0x10000001c57' 2024-09-10T08:44:45.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9352 fp=#0x10000000af8' 2024-09-10T08:44:45.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9384 fp=#0x100000025fd' 2024-09-10T08:44:45.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9416 fp=#0x10000000aa8' 2024-09-10T08:44:45.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9448 fp=#0x10000000a9c' 2024-09-10T08:44:45.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9480 fp=#0x10000000ea1' 2024-09-10T08:44:45.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9512 fp=#0x10000004b3c' 2024-09-10T08:44:45.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9544 fp=#0x10000004955' 2024-09-10T08:44:45.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9576 fp=#0x100000048cc' 2024-09-10T08:44:45.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9608 fp=#0x10000001a5f' 2024-09-10T08:44:45.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9640 fp=#0x10000004984' 2024-09-10T08:44:45.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9672 fp=#0x100000047f1' 2024-09-10T08:44:45.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9704 fp=#0x10000004799' 2024-09-10T08:44:45.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9736 fp=#0x10000004c64' 2024-09-10T08:44:45.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9768 fp=#0x100000019c5' 2024-09-10T08:44:45.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9800 fp=#0x10000004c6a' 2024-09-10T08:44:45.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9832 fp=#0x10000004377' 2024-09-10T08:44:45.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9864 fp=#0x10000004316' 2024-09-10T08:44:45.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9896 fp=#0x10000004cd0' 2024-09-10T08:44:45.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9928 fp=#0x10000001641' 2024-09-10T08:44:45.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9960 fp=#0x10000001a31' 2024-09-10T08:44:45.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9992 fp=#0x10000001733' 2024-09-10T08:44:45.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10024 fp=#0x100000041ab' 2024-09-10T08:44:45.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10056 fp=#0x100000015ab' 2024-09-10T08:44:45.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10088 fp=#0x10000003a89' 2024-09-10T08:44:45.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10120 fp=#0x10000001e4e' 2024-09-10T08:44:45.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10152 fp=#0x10000003b2b' 2024-09-10T08:44:45.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10184 fp=#0x10000001d7d' 2024-09-10T08:44:45.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10216 fp=#0x10000003a7a' 2024-09-10T08:44:45.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10248 fp=#0x10000003b88' 2024-09-10T08:44:45.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10280 fp=#0x10000002514' 2024-09-10T08:44:45.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10312 fp=#0x10000004974' 2024-09-10T08:44:45.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10344 fp=#0x1000000143d' 2024-09-10T08:44:45.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10376 fp=#0x100000034e1' 2024-09-10T08:44:45.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10408 fp=#0x10000004990' 2024-09-10T08:44:45.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10440 fp=#0x10000001382' 2024-09-10T08:44:45.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10472 fp=#0x10000003990' 2024-09-10T08:44:45.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10504 fp=#0x100000014bc' 2024-09-10T08:44:45.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10536 fp=#0x10000003a3a' 2024-09-10T08:44:45.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10568 fp=#0x100000039ba' 2024-09-10T08:44:45.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10600 fp=#0x100000049dc' 2024-09-10T08:44:45.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10664 fp=#0x10000000a86' 2024-09-10T08:44:45.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10696 fp=#0x10000004957' 2024-09-10T08:44:45.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10856 fp=#0x10000000a27' 2024-09-10T08:44:45.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10920 fp=#0x10000001d65' 2024-09-10T08:44:45.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10952 fp=#0x10000000fb0' 2024-09-10T08:44:45.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10984 fp=#0x100000006f7' 2024-09-10T08:44:45.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11048 fp=#0x10000003979' 2024-09-10T08:44:45.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11080 fp=#0x10000003958' 2024-09-10T08:44:45.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11112 fp=#0x10000002358' 2024-09-10T08:44:45.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11144 fp=#0x100000020c1' 2024-09-10T08:44:45.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11176 fp=#0x1000000211c' 2024-09-10T08:44:45.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11208 fp=#0x10000002091' 2024-09-10T08:44:45.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11240 fp=#0x10000002137' 2024-09-10T08:44:45.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11272 fp=#0x10000003f95' 2024-09-10T08:44:45.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11304 fp=#0x100000019c6' 2024-09-10T08:44:45.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11336 fp=#0x10000001a77' 2024-09-10T08:44:45.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11368 fp=#0x10000001a88' 2024-09-10T08:44:45.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11400 fp=#0x10000003dfa' 2024-09-10T08:44:45.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11432 fp=#0x10000001a59' 2024-09-10T08:44:45.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11464 fp=#0x10000001c72' 2024-09-10T08:44:45.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11496 fp=#0x10000003e48' 2024-09-10T08:44:45.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11592 fp=#0x10000005187' 2024-09-10T08:44:45.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11688 fp=#0x10000005149' 2024-09-10T08:44:45.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11720 fp=#0x1000000089c' 2024-09-10T08:44:45.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11752 fp=#0x100000008df' 2024-09-10T08:44:45.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11784 fp=#0x100000041b1' 2024-09-10T08:44:45.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11816 fp=#0x10000004212' 2024-09-10T08:44:45.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11848 fp=#0x10000004d94' 2024-09-10T08:44:45.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11880 fp=#0x1000000096a' 2024-09-10T08:44:45.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11912 fp=#0x100000052a1' 2024-09-10T08:44:45.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11944 fp=#0x10000000f07' 2024-09-10T08:44:45.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12008 fp=#0x10000000f11' 2024-09-10T08:44:45.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12040 fp=#0x1000000114d' 2024-09-10T08:44:45.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12072 fp=#0x1000000337b' 2024-09-10T08:44:45.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12104 fp=#0x10000004491' 2024-09-10T08:44:45.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12136 fp=#0x100000044a4' 2024-09-10T08:44:45.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12168 fp=#0x10000003359' 2024-09-10T08:44:45.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12200 fp=#0x100000033ad' 2024-09-10T08:44:45.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12232 fp=#0x10000000cef' 2024-09-10T08:44:45.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12264 fp=#0x10000000c17' 2024-09-10T08:44:45.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12296 fp=#0x10000003ab6' 2024-09-10T08:44:45.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12328 fp=#0x10000003b85' 2024-09-10T08:44:45.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12360 fp=#0x10000000c49' 2024-09-10T08:44:45.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12392 fp=#0x10000000cf4' 2024-09-10T08:44:45.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12424 fp=#0x100000051e8' 2024-09-10T08:44:45.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12456 fp=#0x10000001026' 2024-09-10T08:44:45.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12520 fp=#0x10000001f3b' 2024-09-10T08:44:45.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12552 fp=#0x100000034ea' 2024-09-10T08:44:45.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12584 fp=#0x10000004c75' 2024-09-10T08:44:45.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12616 fp=#0x10000004b6e' 2024-09-10T08:44:45.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12648 fp=#0x10000004b09' 2024-09-10T08:44:45.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12680 fp=#0x1000000494e' 2024-09-10T08:44:45.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12712 fp=#0x1000000484f' 2024-09-10T08:44:45.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12744 fp=#0x10000004926' 2024-09-10T08:44:45.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12776 fp=#0x100000053ef' 2024-09-10T08:44:45.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12808 fp=#0x10000005395' 2024-09-10T08:44:45.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12840 fp=#0x100000052e9' 2024-09-10T08:44:45.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12872 fp=#0x1000000532f' 2024-09-10T08:44:45.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12904 fp=#0x100000020e2' 2024-09-10T08:44:45.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12936 fp=#0x100000022e9' 2024-09-10T08:44:45.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12968 fp=#0x1000000541b' 2024-09-10T08:44:45.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13000 fp=#0x1000000273e' 2024-09-10T08:44:45.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'client_request(client.25157:74440 create owner_uid=1000, owner_gid=1333 #0x100000026f7/comment-67.xml.tmp 2024-09-10T08:43:46.438535+0000 caller_uid=1000, caller_gid=1333{6,36,1000,1333,})' 2024-09-10T08:44:45.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13056 fp=#0x1000000229e' 2024-09-10T08:44:45.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13088 fp=#0x10000005405' 2024-09-10T08:44:45.625 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13120 fp=#0x10000002286' 2024-09-10T08:44:45.625 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13152 fp=#0x100000053bf' 2024-09-10T08:44:45.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:241 fp=#0x10000004647' 2024-09-10T08:44:45.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:273 fp=#0x10000000f2c' 2024-09-10T08:44:45.627 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:305 fp=#0x100000040d2' 2024-09-10T08:44:45.628 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:457 fp=#0x10000004b7f' 2024-09-10T08:44:45.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:553 fp=#0x1000000411e' 2024-09-10T08:44:45.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:585 fp=#0x10000003329' 2024-09-10T08:44:45.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:617 fp=#0x10000003502' 2024-09-10T08:44:45.630 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:745 fp=#0x1000000481e' 2024-09-10T08:44:45.631 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:777 fp=#0x10000000eb3' 2024-09-10T08:44:45.631 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:809 fp=#0x10000000de7' 2024-09-10T08:44:45.631 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:841 fp=#0x100000044a3' 2024-09-10T08:44:45.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:873 fp=#0x10000004562' 2024-09-10T08:44:45.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:905 fp=#0x10000000e73' 2024-09-10T08:44:45.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:937 fp=#0x100000045b0' 2024-09-10T08:44:45.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:969 fp=#0x10000000df1' 2024-09-10T08:44:45.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1001 fp=#0x10000000da1' 2024-09-10T08:44:45.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1033 fp=#0x10000000d80' 2024-09-10T08:44:45.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1065 fp=#0x10000004118' 2024-09-10T08:44:45.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1129 fp=#0x100000042b9' 2024-09-10T08:44:45.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1161 fp=#0x10000004181' 2024-09-10T08:44:45.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1225 fp=#0x100000020a3' 2024-09-10T08:44:45.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1257 fp=#0x10000003642' 2024-09-10T08:44:45.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1321 fp=#0x10000001fa4' 2024-09-10T08:44:45.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1577 fp=#0x10000003f14' 2024-09-10T08:44:45.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1641 fp=#0x10000004631' 2024-09-10T08:44:45.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1673 fp=#0x100000045a8' 2024-09-10T08:44:45.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1769 fp=#0x100000049b6' 2024-09-10T08:44:45.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2057 fp=#0x100000021a5' 2024-09-10T08:44:45.644 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2089 fp=#0x1000000213b' 2024-09-10T08:44:45.644 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2153 fp=#0x10000002154' 2024-09-10T08:44:45.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2217 fp=#0x10000002745' 2024-09-10T08:44:45.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2281 fp=#0x100000027a9' 2024-09-10T08:44:45.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2313 fp=#0x10000002617' 2024-09-10T08:44:45.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2665 fp=#0x10000001098' 2024-09-10T08:44:45.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2761 fp=#0x10000003b9c' 2024-09-10T08:44:45.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2793 fp=#0x100000036a3' 2024-09-10T08:44:45.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2825 fp=#0x1000000367f' 2024-09-10T08:44:45.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2857 fp=#0x10000004653' 2024-09-10T08:44:45.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2921 fp=#0x10000004704' 2024-09-10T08:44:45.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2953 fp=#0x100000046a4' 2024-09-10T08:44:45.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2985 fp=#0x1000000360f' 2024-09-10T08:44:45.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3017 fp=#0x10000004a3b' 2024-09-10T08:44:45.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3049 fp=#0x10000000e8c' 2024-09-10T08:44:45.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3081 fp=#0x1000000190f' 2024-09-10T08:44:45.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3113 fp=#0x10000001978' 2024-09-10T08:44:45.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3145 fp=#0x100000019a9' 2024-09-10T08:44:45.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3177 fp=#0x10000001987' 2024-09-10T08:44:45.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3209 fp=#0x10000000709' 2024-09-10T08:44:45.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3241 fp=#0x100000007a8' 2024-09-10T08:44:45.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3273 fp=#0x10000000756' 2024-09-10T08:44:45.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3305 fp=#0x10000003cd5' 2024-09-10T08:44:45.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3337 fp=#0x10000003cce' 2024-09-10T08:44:45.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3369 fp=#0x100000045e6' 2024-09-10T08:44:45.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3401 fp=#0x1000000074b' 2024-09-10T08:44:45.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3433 fp=#0x10000003c71' 2024-09-10T08:44:45.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3529 fp=#0x100000040ff' 2024-09-10T08:44:45.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3593 fp=#0x10000004086' 2024-09-10T08:44:45.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3657 fp=#0x10000004c06' 2024-09-10T08:44:45.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3689 fp=#0x10000004d30' 2024-09-10T08:44:45.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3753 fp=#0x10000003f5f' 2024-09-10T08:44:45.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3785 fp=#0x10000005337' 2024-09-10T08:44:45.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3817 fp=#0x1000000531b' 2024-09-10T08:44:45.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3881 fp=#0x10000003eca' 2024-09-10T08:44:45.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3913 fp=#0x1000000119d' 2024-09-10T08:44:45.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3945 fp=#0x10000000668' 2024-09-10T08:44:45.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4009 fp=#0x1000000068b' 2024-09-10T08:44:45.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4041 fp=#0x10000001014' 2024-09-10T08:44:45.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4073 fp=#0x10000001030' 2024-09-10T08:44:45.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4105 fp=#0x10000004eba' 2024-09-10T08:44:45.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4137 fp=#0x100000027c9' 2024-09-10T08:44:45.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4169 fp=#0x100000046b6' 2024-09-10T08:44:45.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4201 fp=#0x10000000fa3' 2024-09-10T08:44:45.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4233 fp=#0x10000003f8c' 2024-09-10T08:44:45.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4265 fp=#0x10000004058' 2024-09-10T08:44:45.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4297 fp=#0x100000039fc' 2024-09-10T08:44:45.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4329 fp=#0x10000003ac1' 2024-09-10T08:44:45.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4361 fp=#0x100000039b6' 2024-09-10T08:44:45.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4393 fp=#0x10000001f1e' 2024-09-10T08:44:45.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4425 fp=#0x10000004cfa' 2024-09-10T08:44:45.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4489 fp=#0x10000005107' 2024-09-10T08:44:45.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4521 fp=#0x1000000405f' 2024-09-10T08:44:45.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4553 fp=#0x10000002714' 2024-09-10T08:44:45.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4585 fp=#0x100000012cf' 2024-09-10T08:44:45.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4649 fp=#0x1000000129d' 2024-09-10T08:44:45.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4681 fp=#0x1000000399c' 2024-09-10T08:44:45.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4713 fp=#0x10000001379' 2024-09-10T08:44:45.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4745 fp=#0x1000000140a' 2024-09-10T08:44:45.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4777 fp=#0x100000012ec' 2024-09-10T08:44:45.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4809 fp=#0x1000000385f' 2024-09-10T08:44:45.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4841 fp=#0x100000037c8' 2024-09-10T08:44:45.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4873 fp=#0x1000000379f' 2024-09-10T08:44:45.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4905 fp=#0x100000036e8' 2024-09-10T08:44:45.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4937 fp=#0x10000003758' 2024-09-10T08:44:45.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4969 fp=#0x1000000501f' 2024-09-10T08:44:45.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5001 fp=#0x1000000510c' 2024-09-10T08:44:45.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5033 fp=#0x1000000519e' 2024-09-10T08:44:45.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5065 fp=#0x100000050f0' 2024-09-10T08:44:45.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5097 fp=#0x10000004fcf' 2024-09-10T08:44:45.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5129 fp=#0x10000001c39' 2024-09-10T08:44:45.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5161 fp=#0x10000001e4f' 2024-09-10T08:44:45.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5193 fp=#0x10000001c2b' 2024-09-10T08:44:45.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5225 fp=#0x10000001bb8' 2024-09-10T08:44:45.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5257 fp=#0x10000004854' 2024-09-10T08:44:45.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5289 fp=#0x10000004a73' 2024-09-10T08:44:45.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5321 fp=#0x10000003641' 2024-09-10T08:44:45.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5353 fp=#0x1000000469b' 2024-09-10T08:44:45.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5385 fp=#0x10000003754' 2024-09-10T08:44:45.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5417 fp=#0x10000003681' 2024-09-10T08:44:45.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5449 fp=#0x10000001297' 2024-09-10T08:44:45.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5577 fp=#0x10000000f5c' 2024-09-10T08:44:45.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5609 fp=#0x100000050b1' 2024-09-10T08:44:45.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5641 fp=#0x100000020af' 2024-09-10T08:44:45.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5673 fp=#0x10000000fa4' 2024-09-10T08:44:45.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5705 fp=#0x10000000f36' 2024-09-10T08:44:45.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5737 fp=#0x1000000209a' 2024-09-10T08:44:45.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5769 fp=#0x100000010b8' 2024-09-10T08:44:45.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5801 fp=#0x100000043fc' 2024-09-10T08:44:45.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5833 fp=#0x10000004d02' 2024-09-10T08:44:45.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5865 fp=#0x10000004d90' 2024-09-10T08:44:45.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5897 fp=#0x10000004cdb' 2024-09-10T08:44:45.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5929 fp=#0x100000043e1' 2024-09-10T08:44:45.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5961 fp=#0x10000004d11' 2024-09-10T08:44:45.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6089 fp=#0x100000033e9' 2024-09-10T08:44:45.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6121 fp=#0x1000000345c' 2024-09-10T08:44:45.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6153 fp=#0x100000034c1' 2024-09-10T08:44:45.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6185 fp=#0x1000000357c' 2024-09-10T08:44:45.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6281 fp=#0x10000001e83' 2024-09-10T08:44:45.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6313 fp=#0x10000001f0c' 2024-09-10T08:44:45.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6345 fp=#0x10000001dc3' 2024-09-10T08:44:45.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6377 fp=#0x1000000335e' 2024-09-10T08:44:45.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6409 fp=#0x10000004d91' 2024-09-10T08:44:45.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6441 fp=#0x10000004f0a' 2024-09-10T08:44:45.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6473 fp=#0x10000004d45' 2024-09-10T08:44:45.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6505 fp=#0x10000004c6c' 2024-09-10T08:44:45.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6537 fp=#0x10000004cd2' 2024-09-10T08:44:45.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6569 fp=#0x10000004770' 2024-09-10T08:44:45.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6633 fp=#0x10000003ab8' 2024-09-10T08:44:45.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6665 fp=#0x10000003b65' 2024-09-10T08:44:45.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6697 fp=#0x10000005000' 2024-09-10T08:44:45.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6729 fp=#0x10000003930' 2024-09-10T08:44:45.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6761 fp=#0x1000000504c' 2024-09-10T08:44:45.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6793 fp=#0x10000004623' 2024-09-10T08:44:45.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6825 fp=#0x10000004555' 2024-09-10T08:44:45.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6857 fp=#0x10000005378' 2024-09-10T08:44:45.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6889 fp=#0x1000000462f' 2024-09-10T08:44:45.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6921 fp=#0x100000052ba' 2024-09-10T08:44:45.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6953 fp=#0x10000005285' 2024-09-10T08:44:45.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6985 fp=#0x10000004fdd' 2024-09-10T08:44:45.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7017 fp=#0x100000005ea' 2024-09-10T08:44:45.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7049 fp=#0x10000003521' 2024-09-10T08:44:45.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7081 fp=#0x10000000656' 2024-09-10T08:44:45.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7113 fp=#0x10000004f4f' 2024-09-10T08:44:45.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7145 fp=#0x10000003460' 2024-09-10T08:44:45.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7177 fp=#0x100000006b0' 2024-09-10T08:44:45.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7209 fp=#0x100000049fe' 2024-09-10T08:44:45.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7241 fp=#0x100000050e8' 2024-09-10T08:44:45.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7337 fp=#0x10000005073' 2024-09-10T08:44:45.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7401 fp=#0x100000024c0' 2024-09-10T08:44:45.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7433 fp=#0x100000047da' 2024-09-10T08:44:45.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7465 fp=#0x100000046b3' 2024-09-10T08:44:45.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7497 fp=#0x10000004706' 2024-09-10T08:44:45.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7529 fp=#0x10000002340' 2024-09-10T08:44:45.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7561 fp=#0x1000000138a' 2024-09-10T08:44:45.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7593 fp=#0x1000000134a' 2024-09-10T08:44:45.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7625 fp=#0x100000051cc' 2024-09-10T08:44:45.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7657 fp=#0x1000000137c' 2024-09-10T08:44:45.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7689 fp=#0x100000051a0' 2024-09-10T08:44:45.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7721 fp=#0x10000004fc3' 2024-09-10T08:44:45.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7753 fp=#0x10000004e1c' 2024-09-10T08:44:45.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7785 fp=#0x10000004f7c' 2024-09-10T08:44:45.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7817 fp=#0x10000004ec4' 2024-09-10T08:44:45.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7849 fp=#0x10000004eb5' 2024-09-10T08:44:45.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7881 fp=#0x10000004ee9' 2024-09-10T08:44:45.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7913 fp=#0x10000004d05' 2024-09-10T08:44:45.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7945 fp=#0x1000000143a' 2024-09-10T08:44:45.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7977 fp=#0x1000000167b' 2024-09-10T08:44:45.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8009 fp=#0x10000004e8d' 2024-09-10T08:44:45.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8041 fp=#0x1000000162e' 2024-09-10T08:44:45.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8073 fp=#0x10000001650' 2024-09-10T08:44:45.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8105 fp=#0x1000000166c' 2024-09-10T08:44:45.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8137 fp=#0x100000026bf' 2024-09-10T08:44:45.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8169 fp=#0x100000025fe' 2024-09-10T08:44:45.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8201 fp=#0x10000004959' 2024-09-10T08:44:45.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8233 fp=#0x10000002523' 2024-09-10T08:44:45.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8265 fp=#0x100000046e4' 2024-09-10T08:44:45.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8297 fp=#0x10000005268' 2024-09-10T08:44:45.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8329 fp=#0x100000042e4' 2024-09-10T08:44:45.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8361 fp=#0x10000004101' 2024-09-10T08:44:45.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8393 fp=#0x1000000437f' 2024-09-10T08:44:45.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8425 fp=#0x1000000429e' 2024-09-10T08:44:45.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8457 fp=#0x10000004563' 2024-09-10T08:44:45.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8489 fp=#0x10000003fe9' 2024-09-10T08:44:45.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8521 fp=#0x1000000403d' 2024-09-10T08:44:45.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8553 fp=#0x10000003e5e' 2024-09-10T08:44:45.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8585 fp=#0x10000003eb6' 2024-09-10T08:44:45.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8617 fp=#0x10000004255' 2024-09-10T08:44:45.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8649 fp=#0x10000004ef4' 2024-09-10T08:44:45.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8681 fp=#0x10000004f9b' 2024-09-10T08:44:45.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8713 fp=#0x100000052b4' 2024-09-10T08:44:45.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8745 fp=#0x10000004f31' 2024-09-10T08:44:45.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8777 fp=#0x10000003d01' 2024-09-10T08:44:45.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8809 fp=#0x10000003c03' 2024-09-10T08:44:45.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8841 fp=#0x10000001d9a' 2024-09-10T08:44:45.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8873 fp=#0x10000001bd2' 2024-09-10T08:44:45.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8905 fp=#0x100000043ad' 2024-09-10T08:44:45.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8937 fp=#0x1000000405a' 2024-09-10T08:44:45.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8969 fp=#0x10000001b1c' 2024-09-10T08:44:45.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9001 fp=#0x10000004404' 2024-09-10T08:44:45.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9033 fp=#0x100000011c3' 2024-09-10T08:44:45.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9065 fp=#0x10000000cfa' 2024-09-10T08:44:45.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9097 fp=#0x10000000d4e' 2024-09-10T08:44:45.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9129 fp=#0x10000004263' 2024-09-10T08:44:45.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9161 fp=#0x100000040e2' 2024-09-10T08:44:45.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9193 fp=#0x100000042af' 2024-09-10T08:44:45.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9225 fp=#0x10000004230' 2024-09-10T08:44:45.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9257 fp=#0x10000000ab0' 2024-09-10T08:44:45.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9289 fp=#0x1000000261e' 2024-09-10T08:44:45.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9321 fp=#0x10000001c0f' 2024-09-10T08:44:45.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9353 fp=#0x10000004cc8' 2024-09-10T08:44:45.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9385 fp=#0x100000041fb' 2024-09-10T08:44:45.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9417 fp=#0x10000000bd6' 2024-09-10T08:44:45.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9449 fp=#0x10000000b28' 2024-09-10T08:44:45.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9481 fp=#0x10000004901' 2024-09-10T08:44:45.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9513 fp=#0x10000002709' 2024-09-10T08:44:45.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9545 fp=#0x10000004940' 2024-09-10T08:44:45.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9577 fp=#0x10000004908' 2024-09-10T08:44:45.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9609 fp=#0x100000049d1' 2024-09-10T08:44:45.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9641 fp=#0x10000001ecd' 2024-09-10T08:44:45.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9673 fp=#0x10000004878' 2024-09-10T08:44:45.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9705 fp=#0x100000047c2' 2024-09-10T08:44:45.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9737 fp=#0x10000001732' 2024-09-10T08:44:45.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9769 fp=#0x1000000430c' 2024-09-10T08:44:45.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9801 fp=#0x10000004444' 2024-09-10T08:44:45.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9833 fp=#0x100000019f9' 2024-09-10T08:44:45.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9865 fp=#0x1000000198f' 2024-09-10T08:44:45.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9897 fp=#0x100000044c8' 2024-09-10T08:44:45.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9929 fp=#0x10000001a2b' 2024-09-10T08:44:45.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9961 fp=#0x10000003b61' 2024-09-10T08:44:45.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9993 fp=#0x10000001890' 2024-09-10T08:44:45.732 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10025 fp=#0x10000001686' 2024-09-10T08:44:45.732 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10057 fp=#0x1000000268f' 2024-09-10T08:44:45.732 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10089 fp=#0x10000001da4' 2024-09-10T08:44:45.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10121 fp=#0x10000001e5f' 2024-09-10T08:44:45.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10153 fp=#0x10000003961' 2024-09-10T08:44:45.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10185 fp=#0x10000003a5d' 2024-09-10T08:44:45.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10217 fp=#0x10000001e97' 2024-09-10T08:44:45.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10249 fp=#0x10000003ade' 2024-09-10T08:44:45.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10281 fp=#0x10000002428' 2024-09-10T08:44:45.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10313 fp=#0x10000002459' 2024-09-10T08:44:45.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10345 fp=#0x10000001355' 2024-09-10T08:44:45.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10377 fp=#0x10000001515' 2024-09-10T08:44:45.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10409 fp=#0x1000000130e' 2024-09-10T08:44:45.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10473 fp=#0x10000001301' 2024-09-10T08:44:45.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10505 fp=#0x100000039c2' 2024-09-10T08:44:45.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10537 fp=#0x10000003962' 2024-09-10T08:44:45.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10633 fp=#0x1000000486e' 2024-09-10T08:44:45.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10665 fp=#0x1000000498f' 2024-09-10T08:44:45.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10697 fp=#0x10000004988' 2024-09-10T08:44:45.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10729 fp=#0x1000000081f' 2024-09-10T08:44:45.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10825 fp=#0x10000001cc9' 2024-09-10T08:44:45.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10857 fp=#0x10000003ed2' 2024-09-10T08:44:45.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10921 fp=#0x100000005de' 2024-09-10T08:44:45.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10953 fp=#0x1000000062d' 2024-09-10T08:44:45.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10985 fp=#0x10000000a40' 2024-09-10T08:44:45.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11017 fp=#0x100000035ce' 2024-09-10T08:44:45.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11049 fp=#0x100000039d6' 2024-09-10T08:44:45.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11081 fp=#0x1000000392d' 2024-09-10T08:44:45.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11113 fp=#0x10000003e61' 2024-09-10T08:44:45.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11145 fp=#0x1000000237e' 2024-09-10T08:44:45.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11177 fp=#0x10000003f15' 2024-09-10T08:44:45.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11209 fp=#0x10000003eb9' 2024-09-10T08:44:45.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11241 fp=#0x1000000232d' 2024-09-10T08:44:45.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11273 fp=#0x10000003f66' 2024-09-10T08:44:45.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11305 fp=#0x10000003ef4' 2024-09-10T08:44:45.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11337 fp=#0x10000003ed3' 2024-09-10T08:44:45.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11369 fp=#0x10000001a53' 2024-09-10T08:44:45.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11401 fp=#0x10000001b65' 2024-09-10T08:44:45.747 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11433 fp=#0x10000001b7f' 2024-09-10T08:44:45.747 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11465 fp=#0x10000003e6c' 2024-09-10T08:44:45.747 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11497 fp=#0x10000001a01' 2024-09-10T08:44:45.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11561 fp=#0x1000000514b' 2024-09-10T08:44:45.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11593 fp=#0x10000003be8' 2024-09-10T08:44:45.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11625 fp=#0x10000005156' 2024-09-10T08:44:45.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11657 fp=#0x1000000514f' 2024-09-10T08:44:45.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11721 fp=#0x10000004296' 2024-09-10T08:44:45.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11753 fp=#0x100000041d8' 2024-09-10T08:44:45.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11785 fp=#0x10000000916' 2024-09-10T08:44:45.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11817 fp=#0x100000009b3' 2024-09-10T08:44:45.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11849 fp=#0x100000008c9' 2024-09-10T08:44:45.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11881 fp=#0x10000000977' 2024-09-10T08:44:45.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11913 fp=#0x10000000f2f' 2024-09-10T08:44:45.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11945 fp=#0x10000000efe' 2024-09-10T08:44:45.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11977 fp=#0x100000052aa' 2024-09-10T08:44:45.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12009 fp=#0x10000001138' 2024-09-10T08:44:45.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12041 fp=#0x10000000ef2' 2024-09-10T08:44:45.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12073 fp=#0x1000000349d' 2024-09-10T08:44:45.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12105 fp=#0x100000043f0' 2024-09-10T08:44:45.755 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12137 fp=#0x1000000455f' 2024-09-10T08:44:45.755 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12169 fp=#0x1000000348c' 2024-09-10T08:44:45.755 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12201 fp=#0x100000033fd' 2024-09-10T08:44:45.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12233 fp=#0x10000000d6f' 2024-09-10T08:44:45.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12265 fp=#0x10000000fd9' 2024-09-10T08:44:45.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12297 fp=#0x10000003a70' 2024-09-10T08:44:45.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12329 fp=#0x100000051ec' 2024-09-10T08:44:45.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12361 fp=#0x10000000c56' 2024-09-10T08:44:45.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12393 fp=#0x10000003bdc' 2024-09-10T08:44:45.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12425 fp=#0x100000051e1' 2024-09-10T08:44:45.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12457 fp=#0x1000000256f' 2024-09-10T08:44:45.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12489 fp=#0x10000004c90' 2024-09-10T08:44:45.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12585 fp=#0x10000004b36' 2024-09-10T08:44:45.760 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12617 fp=#0x10000004ad7' 2024-09-10T08:44:45.760 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12649 fp=#0x100000049a2' 2024-09-10T08:44:45.760 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12681 fp=#0x1000000475c' 2024-09-10T08:44:45.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12713 fp=#0x1000000534c' 2024-09-10T08:44:45.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12745 fp=#0x10000005339' 2024-09-10T08:44:45.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12777 fp=#0x100000022f8' 2024-09-10T08:44:45.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12809 fp=#0x10000002735' 2024-09-10T08:44:45.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12841 fp=#0x10000002150' 2024-09-10T08:44:45.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12873 fp=#0x100000052ab' 2024-09-10T08:44:45.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12905 fp=#0x100000053d6' 2024-09-10T08:44:45.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12937 fp=#0x10000005361' 2024-09-10T08:44:45.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12969 fp=#0x10000002720' 2024-09-10T08:44:45.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13001 fp=#0x100000053e4' 2024-09-10T08:44:45.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13025 fp=#0x10000002311' 2024-09-10T08:44:45.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13057 fp=#0x100000022fd' 2024-09-10T08:44:45.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13089 fp=#0x10000002337' 2024-09-10T08:44:45.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13121 fp=#0x1000000538c' 2024-09-10T08:44:45.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13153 fp=#0x100000024e0' 2024-09-10T08:44:45.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_path:mds.0:138 fp=#0x1' 2024-09-10T08:44:45.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:242 fp=#0x100000030bf' 2024-09-10T08:44:45.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:274 fp=#0x1000000396a' 2024-09-10T08:44:45.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:306 fp=#0x10000003d98' 2024-09-10T08:44:45.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:490 fp=#0x100000032b2' 2024-09-10T08:44:45.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:522 fp=#0x100000034c9' 2024-09-10T08:44:45.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:554 fp=#0x1000000338f' 2024-09-10T08:44:45.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:586 fp=#0x10000003993' 2024-09-10T08:44:45.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:778 fp=#0x10000000d94' 2024-09-10T08:44:45.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:842 fp=#0x100000044ad' 2024-09-10T08:44:45.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:874 fp=#0x100000045cb' 2024-09-10T08:44:45.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:906 fp=#0x100000044b6' 2024-09-10T08:44:45.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:938 fp=#0x1000000408c' 2024-09-10T08:44:45.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:970 fp=#0x10000000dc3' 2024-09-10T08:44:45.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1002 fp=#0x10000003fe8' 2024-09-10T08:44:45.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1034 fp=#0x10000000e79' 2024-09-10T08:44:45.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1066 fp=#0x100000042bf' 2024-09-10T08:44:45.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1226 fp=#0x100000020ab' 2024-09-10T08:44:45.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1258 fp=#0x10000003647' 2024-09-10T08:44:45.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1322 fp=#0x1000000207b' 2024-09-10T08:44:45.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1354 fp=#0x1000000206a' 2024-09-10T08:44:45.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1642 fp=#0x1000000463f' 2024-09-10T08:44:45.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1674 fp=#0x10000004595' 2024-09-10T08:44:45.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2058 fp=#0x100000021c8' 2024-09-10T08:44:45.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2090 fp=#0x100000021b5' 2024-09-10T08:44:45.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2122 fp=#0x10000001fd4' 2024-09-10T08:44:45.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2154 fp=#0x10000002028' 2024-09-10T08:44:45.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2186 fp=#0x10000002002' 2024-09-10T08:44:45.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2218 fp=#0x10000002683' 2024-09-10T08:44:45.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2314 fp=#0x100000027cb' 2024-09-10T08:44:45.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2570 fp=#0x1000000109d' 2024-09-10T08:44:45.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2602 fp=#0x100000018ec' 2024-09-10T08:44:45.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2634 fp=#0x10000001056' 2024-09-10T08:44:45.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2666 fp=#0x1000000104d' 2024-09-10T08:44:45.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2698 fp=#0x10000001165' 2024-09-10T08:44:45.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2794 fp=#0x100000047f4' 2024-09-10T08:44:45.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2826 fp=#0x10000004b49' 2024-09-10T08:44:45.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2890 fp=#0x10000003405' 2024-09-10T08:44:45.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2922 fp=#0x100000033a1' 2024-09-10T08:44:45.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2954 fp=#0x100000046e8' 2024-09-10T08:44:45.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2986 fp=#0x10000003549' 2024-09-10T08:44:45.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3018 fp=#0x10000000e12' 2024-09-10T08:44:45.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3050 fp=#0x10000000e6b' 2024-09-10T08:44:45.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3146 fp=#0x10000000e1b' 2024-09-10T08:44:45.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3178 fp=#0x10000000dcc' 2024-09-10T08:44:45.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3210 fp=#0x100000007a5' 2024-09-10T08:44:45.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3242 fp=#0x100000007f3' 2024-09-10T08:44:45.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3274 fp=#0x10000003c51' 2024-09-10T08:44:45.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3306 fp=#0x10000000780' 2024-09-10T08:44:45.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3338 fp=#0x10000003c05' 2024-09-10T08:44:45.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3370 fp=#0x10000003393' 2024-09-10T08:44:45.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3402 fp=#0x1000000079e' 2024-09-10T08:44:45.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3434 fp=#0x10000003bba' 2024-09-10T08:44:45.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3530 fp=#0x10000004140' 2024-09-10T08:44:45.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3562 fp=#0x10000003c7c' 2024-09-10T08:44:45.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3594 fp=#0x10000004060' 2024-09-10T08:44:45.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3658 fp=#0x10000004b9f' 2024-09-10T08:44:45.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3690 fp=#0x10000004c44' 2024-09-10T08:44:45.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3722 fp=#0x10000003ebe' 2024-09-10T08:44:45.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3754 fp=#0x10000003f98' 2024-09-10T08:44:45.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3786 fp=#0x100000025a1' 2024-09-10T08:44:45.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3850 fp=#0x10000002710' 2024-09-10T08:44:45.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3882 fp=#0x10000002795' 2024-09-10T08:44:45.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3914 fp=#0x100000006a3' 2024-09-10T08:44:45.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3946 fp=#0x10000000672' 2024-09-10T08:44:45.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3978 fp=#0x10000000651' 2024-09-10T08:44:45.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4010 fp=#0x10000000540' 2024-09-10T08:44:45.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4042 fp=#0x100000026e5' 2024-09-10T08:44:45.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4074 fp=#0x1000000102a' 2024-09-10T08:44:45.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4138 fp=#0x10000004d6a' 2024-09-10T08:44:45.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4170 fp=#0x100000052f1' 2024-09-10T08:44:45.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4202 fp=#0x10000000f6b' 2024-09-10T08:44:45.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4234 fp=#0x100000039c6' 2024-09-10T08:44:45.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4266 fp=#0x1000000398e' 2024-09-10T08:44:45.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4298 fp=#0x10000004b88' 2024-09-10T08:44:45.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4330 fp=#0x10000003a5e' 2024-09-10T08:44:45.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4362 fp=#0x1000000402e' 2024-09-10T08:44:45.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4394 fp=#0x10000001f2b' 2024-09-10T08:44:45.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4426 fp=#0x1000000401d' 2024-09-10T08:44:45.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4458 fp=#0x10000001f3c' 2024-09-10T08:44:45.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4490 fp=#0x100000050b4' 2024-09-10T08:44:45.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4522 fp=#0x100000012be' 2024-09-10T08:44:45.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4554 fp=#0x10000003b45' 2024-09-10T08:44:45.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4586 fp=#0x10000003904' 2024-09-10T08:44:45.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4618 fp=#0x100000015b7' 2024-09-10T08:44:45.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4650 fp=#0x100000015a2' 2024-09-10T08:44:45.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4682 fp=#0x10000001293' 2024-09-10T08:44:45.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4714 fp=#0x1000000138b' 2024-09-10T08:44:45.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4746 fp=#0x10000001462' 2024-09-10T08:44:45.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4778 fp=#0x10000001536' 2024-09-10T08:44:45.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4810 fp=#0x100000037d8' 2024-09-10T08:44:45.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4842 fp=#0x10000004a7c' 2024-09-10T08:44:45.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4874 fp=#0x1000000376f' 2024-09-10T08:44:45.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4906 fp=#0x10000003832' 2024-09-10T08:44:45.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4938 fp=#0x10000004ac1' 2024-09-10T08:44:45.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4970 fp=#0x10000005182' 2024-09-10T08:44:45.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5002 fp=#0x100000051d1' 2024-09-10T08:44:45.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5034 fp=#0x1000000517f' 2024-09-10T08:44:45.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5066 fp=#0x10000005065' 2024-09-10T08:44:45.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5098 fp=#0x10000005005' 2024-09-10T08:44:45.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5130 fp=#0x100000027b2' 2024-09-10T08:44:45.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5162 fp=#0x10000001be8' 2024-09-10T08:44:45.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5194 fp=#0x10000002412' 2024-09-10T08:44:45.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5226 fp=#0x10000001e32' 2024-09-10T08:44:45.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5258 fp=#0x10000003dbc' 2024-09-10T08:44:45.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5290 fp=#0x10000003845' 2024-09-10T08:44:45.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5322 fp=#0x1000000370d' 2024-09-10T08:44:45.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5354 fp=#0x10000003712' 2024-09-10T08:44:45.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5386 fp=#0x10000003d24' 2024-09-10T08:44:45.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5418 fp=#0x10000003859' 2024-09-10T08:44:45.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5514 fp=#0x100000012a4' 2024-09-10T08:44:45.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5546 fp=#0x10000003e6a' 2024-09-10T08:44:45.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5578 fp=#0x100000050c1' 2024-09-10T08:44:45.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5610 fp=#0x100000050cc' 2024-09-10T08:44:45.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5642 fp=#0x100000050a5' 2024-09-10T08:44:45.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5674 fp=#0x10000005105' 2024-09-10T08:44:45.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5706 fp=#0x10000001091' 2024-09-10T08:44:45.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5738 fp=#0x1000000509c' 2024-09-10T08:44:45.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5770 fp=#0x10000004f8a' 2024-09-10T08:44:45.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5802 fp=#0x100000043eb' 2024-09-10T08:44:45.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5834 fp=#0x10000004cfc' 2024-09-10T08:44:45.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5866 fp=#0x10000004c6d' 2024-09-10T08:44:45.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5898 fp=#0x10000004bea' 2024-09-10T08:44:45.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5930 fp=#0x10000004c4f' 2024-09-10T08:44:45.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5962 fp=#0x10000004ce9' 2024-09-10T08:44:45.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6122 fp=#0x100000034b3' 2024-09-10T08:44:45.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6154 fp=#0x10000003611' 2024-09-10T08:44:45.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6186 fp=#0x100000034b8' 2024-09-10T08:44:45.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6218 fp=#0x1000000349f' 2024-09-10T08:44:45.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6250 fp=#0x100000035a0' 2024-09-10T08:44:45.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6282 fp=#0x100000033f8' 2024-09-10T08:44:45.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6314 fp=#0x10000001f24' 2024-09-10T08:44:45.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6346 fp=#0x10000001eb3' 2024-09-10T08:44:45.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6378 fp=#0x10000001e44' 2024-09-10T08:44:45.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6410 fp=#0x1000000536c' 2024-09-10T08:44:45.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6442 fp=#0x10000004e2a' 2024-09-10T08:44:45.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6474 fp=#0x10000005364' 2024-09-10T08:44:45.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6506 fp=#0x10000004b29' 2024-09-10T08:44:45.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6538 fp=#0x10000004c4c' 2024-09-10T08:44:45.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6634 fp=#0x10000003d29' 2024-09-10T08:44:45.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6666 fp=#0x10000005017' 2024-09-10T08:44:45.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6698 fp=#0x100000039aa' 2024-09-10T08:44:45.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6730 fp=#0x10000003cb7' 2024-09-10T08:44:45.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6762 fp=#0x10000005023' 2024-09-10T08:44:45.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6794 fp=#0x100000052ea' 2024-09-10T08:44:45.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6826 fp=#0x10000004547' 2024-09-10T08:44:45.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6858 fp=#0x100000052a9' 2024-09-10T08:44:45.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6890 fp=#0x10000005351' 2024-09-10T08:44:45.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6922 fp=#0x100000052d8' 2024-09-10T08:44:45.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6954 fp=#0x10000001f01' 2024-09-10T08:44:45.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6986 fp=#0x100000007fc' 2024-09-10T08:44:45.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7018 fp=#0x1000000069e' 2024-09-10T08:44:45.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7050 fp=#0x10000002461' 2024-09-10T08:44:45.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7082 fp=#0x1000000064f' 2024-09-10T08:44:45.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7114 fp=#0x10000002466' 2024-09-10T08:44:45.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7146 fp=#0x10000004fd3' 2024-09-10T08:44:45.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7178 fp=#0x100000005a0' 2024-09-10T08:44:45.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7434 fp=#0x1000000471c' 2024-09-10T08:44:45.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7466 fp=#0x100000046c8' 2024-09-10T08:44:45.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7498 fp=#0x1000000476d' 2024-09-10T08:44:45.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7530 fp=#0x10000002356' 2024-09-10T08:44:45.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7626 fp=#0x10000001395' 2024-09-10T08:44:45.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7658 fp=#0x100000019f7' 2024-09-10T08:44:45.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7690 fp=#0x10000001d30' 2024-09-10T08:44:45.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7722 fp=#0x10000004fab' 2024-09-10T08:44:45.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7754 fp=#0x10000004dd8' 2024-09-10T08:44:45.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7786 fp=#0x10000004dfe' 2024-09-10T08:44:45.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7818 fp=#0x100000013a6' 2024-09-10T08:44:45.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7850 fp=#0x10000004eae' 2024-09-10T08:44:45.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7882 fp=#0x10000004d71' 2024-09-10T08:44:45.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7914 fp=#0x1000000159c' 2024-09-10T08:44:45.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7946 fp=#0x10000001436' 2024-09-10T08:44:45.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7978 fp=#0x10000004ce8' 2024-09-10T08:44:45.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8010 fp=#0x10000001649' 2024-09-10T08:44:45.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8042 fp=#0x1000000161e' 2024-09-10T08:44:45.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8074 fp=#0x10000001701' 2024-09-10T08:44:45.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8106 fp=#0x100000016c3' 2024-09-10T08:44:45.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8138 fp=#0x1000000496e' 2024-09-10T08:44:45.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8170 fp=#0x10000002706' 2024-09-10T08:44:45.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8202 fp=#0x100000026a9' 2024-09-10T08:44:45.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8234 fp=#0x10000002660' 2024-09-10T08:44:45.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8266 fp=#0x10000004307' 2024-09-10T08:44:45.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8298 fp=#0x100000041d1' 2024-09-10T08:44:45.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8330 fp=#0x100000046f9' 2024-09-10T08:44:45.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8362 fp=#0x1000000431b' 2024-09-10T08:44:45.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8394 fp=#0x100000042aa' 2024-09-10T08:44:45.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8426 fp=#0x100000042c8' 2024-09-10T08:44:45.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8458 fp=#0x10000004245' 2024-09-10T08:44:45.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8490 fp=#0x10000003f84' 2024-09-10T08:44:45.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8522 fp=#0x1000000455a' 2024-09-10T08:44:45.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8554 fp=#0x10000004282' 2024-09-10T08:44:45.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8586 fp=#0x100000040d8' 2024-09-10T08:44:45.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8618 fp=#0x10000003e09' 2024-09-10T08:44:45.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8650 fp=#0x10000004088' 2024-09-10T08:44:45.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8682 fp=#0x10000003fc3' 2024-09-10T08:44:45.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8714 fp=#0x10000004ed7' 2024-09-10T08:44:45.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8746 fp=#0x10000003d84' 2024-09-10T08:44:45.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8778 fp=#0x100000052a3' 2024-09-10T08:44:45.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8810 fp=#0x10000003cf9' 2024-09-10T08:44:45.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8842 fp=#0x10000001ce9' 2024-09-10T08:44:45.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8874 fp=#0x10000001c75' 2024-09-10T08:44:45.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8906 fp=#0x10000001aff' 2024-09-10T08:44:45.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8938 fp=#0x10000004351' 2024-09-10T08:44:45.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8970 fp=#0x100000043ff' 2024-09-10T08:44:45.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9002 fp=#0x10000001cef' 2024-09-10T08:44:45.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9034 fp=#0x10000000d40' 2024-09-10T08:44:45.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9066 fp=#0x10000000d66' 2024-09-10T08:44:45.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9098 fp=#0x10000003bf7' 2024-09-10T08:44:45.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9130 fp=#0x10000000fe5' 2024-09-10T08:44:45.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9162 fp=#0x10000000d42' 2024-09-10T08:44:45.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9194 fp=#0x10000000d7f' 2024-09-10T08:44:45.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9226 fp=#0x10000000da4' 2024-09-10T08:44:45.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9258 fp=#0x10000000b4a' 2024-09-10T08:44:45.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9290 fp=#0x10000001c51' 2024-09-10T08:44:45.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9322 fp=#0x10000004d06' 2024-09-10T08:44:45.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9354 fp=#0x10000004dc3' 2024-09-10T08:44:45.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9386 fp=#0x10000000b05' 2024-09-10T08:44:45.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9418 fp=#0x10000000b2c' 2024-09-10T08:44:45.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9450 fp=#0x10000000bc0' 2024-09-10T08:44:45.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9482 fp=#0x10000000ecb' 2024-09-10T08:44:45.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9514 fp=#0x10000004938' 2024-09-10T08:44:45.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9546 fp=#0x100000049ca' 2024-09-10T08:44:45.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9578 fp=#0x10000001ea7' 2024-09-10T08:44:45.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9610 fp=#0x100000019ce' 2024-09-10T08:44:45.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9642 fp=#0x10000001a0b' 2024-09-10T08:44:45.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9674 fp=#0x10000004759' 2024-09-10T08:44:45.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9706 fp=#0x100000047b3' 2024-09-10T08:44:45.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9738 fp=#0x10000001b03' 2024-09-10T08:44:45.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9770 fp=#0x10000004360' 2024-09-10T08:44:45.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9802 fp=#0x100000044cc' 2024-09-10T08:44:45.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9834 fp=#0x10000001e22' 2024-09-10T08:44:45.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9866 fp=#0x10000004340' 2024-09-10T08:44:45.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9898 fp=#0x10000001a75' 2024-09-10T08:44:45.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9930 fp=#0x100000026a0' 2024-09-10T08:44:45.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9962 fp=#0x1000000163c' 2024-09-10T08:44:45.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9994 fp=#0x10000001799' 2024-09-10T08:44:45.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10026 fp=#0x100000015d0' 2024-09-10T08:44:45.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10058 fp=#0x10000001580' 2024-09-10T08:44:45.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10090 fp=#0x10000001f3a' 2024-09-10T08:44:45.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10122 fp=#0x10000001e30' 2024-09-10T08:44:45.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10154 fp=#0x10000003ad5' 2024-09-10T08:44:45.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10186 fp=#0x10000003a91' 2024-09-10T08:44:45.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10218 fp=#0x10000001d79' 2024-09-10T08:44:45.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10250 fp=#0x1000000404c' 2024-09-10T08:44:45.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10282 fp=#0x100000024a7' 2024-09-10T08:44:45.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10314 fp=#0x1000000241c' 2024-09-10T08:44:45.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10346 fp=#0x10000003a52' 2024-09-10T08:44:45.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10378 fp=#0x1000000132b' 2024-09-10T08:44:45.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10410 fp=#0x100000014fa' 2024-09-10T08:44:45.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10442 fp=#0x10000001525' 2024-09-10T08:44:45.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10474 fp=#0x10000001309' 2024-09-10T08:44:45.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10506 fp=#0x10000001ec6' 2024-09-10T08:44:45.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10538 fp=#0x100000039a7' 2024-09-10T08:44:45.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10570 fp=#0x100000048b3' 2024-09-10T08:44:45.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10666 fp=#0x10000004d36' 2024-09-10T08:44:45.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10698 fp=#0x100000048b1' 2024-09-10T08:44:45.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10762 fp=#0x1000000496b' 2024-09-10T08:44:45.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10794 fp=#0x100000048cb' 2024-09-10T08:44:45.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10826 fp=#0x10000000a5f' 2024-09-10T08:44:45.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10890 fp=#0x10000000592' 2024-09-10T08:44:45.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10954 fp=#0x10000000a50' 2024-09-10T08:44:45.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10986 fp=#0x1000000089a' 2024-09-10T08:44:45.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11018 fp=#0x10000001ced' 2024-09-10T08:44:45.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11050 fp=#0x10000003970' 2024-09-10T08:44:45.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11082 fp=#0x100000038d8' 2024-09-10T08:44:45.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11114 fp=#0x100000022fa' 2024-09-10T08:44:45.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11178 fp=#0x10000002189' 2024-09-10T08:44:45.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11210 fp=#0x100000048ed' 2024-09-10T08:44:45.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11242 fp=#0x10000002199' 2024-09-10T08:44:45.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11274 fp=#0x1000000203b' 2024-09-10T08:44:45.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11306 fp=#0x10000001a64' 2024-09-10T08:44:45.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11338 fp=#0x10000003f0c' 2024-09-10T08:44:45.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11370 fp=#0x10000003d66' 2024-09-10T08:44:45.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11402 fp=#0x10000003f7f' 2024-09-10T08:44:45.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11434 fp=#0x10000001c8e' 2024-09-10T08:44:45.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11466 fp=#0x100000019d8' 2024-09-10T08:44:45.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11498 fp=#0x10000003d56' 2024-09-10T08:44:45.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11594 fp=#0x1000000517e' 2024-09-10T08:44:45.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11658 fp=#0x10000005160' 2024-09-10T08:44:45.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11722 fp=#0x100000041d3' 2024-09-10T08:44:45.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11754 fp=#0x10000000b39' 2024-09-10T08:44:45.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11786 fp=#0x10000004249' 2024-09-10T08:44:45.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11818 fp=#0x10000004292' 2024-09-10T08:44:45.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11850 fp=#0x10000001809' 2024-09-10T08:44:45.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11882 fp=#0x100000009e0' 2024-09-10T08:44:45.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11946 fp=#0x100000010ce' 2024-09-10T08:44:45.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11978 fp=#0x10000000f00' 2024-09-10T08:44:45.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12010 fp=#0x10000000f0c' 2024-09-10T08:44:45.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12042 fp=#0x100000010c6' 2024-09-10T08:44:45.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12106 fp=#0x100000034ec' 2024-09-10T08:44:45.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12138 fp=#0x1000000334d' 2024-09-10T08:44:45.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12170 fp=#0x100000033b9' 2024-09-10T08:44:45.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12202 fp=#0x100000044f4' 2024-09-10T08:44:45.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12234 fp=#0x1000000221b' 2024-09-10T08:44:45.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12266 fp=#0x10000000e43' 2024-09-10T08:44:45.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12298 fp=#0x100000051f9' 2024-09-10T08:44:45.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12330 fp=#0x100000051f4' 2024-09-10T08:44:45.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12362 fp=#0x100000051f5' 2024-09-10T08:44:45.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12394 fp=#0x10000000cb1' 2024-09-10T08:44:45.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12426 fp=#0x100000017c2' 2024-09-10T08:44:45.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12458 fp=#0x10000002670' 2024-09-10T08:44:45.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12490 fp=#0x1000000101d' 2024-09-10T08:44:45.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12522 fp=#0x10000003dec' 2024-09-10T08:44:45.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12554 fp=#0x10000004b1f' 2024-09-10T08:44:45.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12586 fp=#0x10000004c6e' 2024-09-10T08:44:45.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12618 fp=#0x10000004c5a' 2024-09-10T08:44:45.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12650 fp=#0x10000005411' 2024-09-10T08:44:45.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12682 fp=#0x1000000491c' 2024-09-10T08:44:45.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12714 fp=#0x100000049ba' 2024-09-10T08:44:45.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12746 fp=#0x10000005375' 2024-09-10T08:44:45.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12778 fp=#0x10000005334' 2024-09-10T08:44:45.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12810 fp=#0x100000052da' 2024-09-10T08:44:45.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12842 fp=#0x10000002168' 2024-09-10T08:44:45.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12874 fp=#0x10000002496' 2024-09-10T08:44:45.907 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12906 fp=#0x100000022d0' 2024-09-10T08:44:45.907 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12938 fp=#0x1000000538f' 2024-09-10T08:44:45.907 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12970 fp=#0x10000005396' 2024-09-10T08:44:45.908 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13002 fp=#0x100000026fb' 2024-09-10T08:44:45.908 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13026 fp=#0x100000053b2' 2024-09-10T08:44:45.908 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13058 fp=#0x100000022d8' 2024-09-10T08:44:45.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13090 fp=#0x1000000397d' 2024-09-10T08:44:45.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13122 fp=#0x10000004000' 2024-09-10T08:44:45.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13154 fp=#0x100000039fe' 2024-09-10T08:44:45.910 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:139 fp=#0x1' 2024-09-10T08:44:45.911 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:275 fp=#0x10000001ed4' 2024-09-10T08:44:45.911 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:307 fp=#0x10000003bb7' 2024-09-10T08:44:45.912 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:427 fp=#0x10000004b62' 2024-09-10T08:44:45.912 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:459 fp=#0x10000004b82' 2024-09-10T08:44:45.913 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:491 fp=#0x10000003510' 2024-09-10T08:44:45.913 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:523 fp=#0x1000000330f' 2024-09-10T08:44:45.914 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:555 fp=#0x1000000355a' 2024-09-10T08:44:45.915 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:619 fp=#0x1000000329c' 2024-09-10T08:44:45.916 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:779 fp=#0x1000000452d' 2024-09-10T08:44:45.916 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:811 fp=#0x10000000d9f' 2024-09-10T08:44:45.917 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:843 fp=#0x10000000e9b' 2024-09-10T08:44:45.917 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:907 fp=#0x10000000e50' 2024-09-10T08:44:45.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:939 fp=#0x10000000eab' 2024-09-10T08:44:45.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1003 fp=#0x10000000e8d' 2024-09-10T08:44:45.919 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1035 fp=#0x1000000448f' 2024-09-10T08:44:45.919 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1067 fp=#0x1000000421a' 2024-09-10T08:44:45.920 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1227 fp=#0x10000001f78' 2024-09-10T08:44:45.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1259 fp=#0x100000035d5' 2024-09-10T08:44:45.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1291 fp=#0x1000000364c' 2024-09-10T08:44:45.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1323 fp=#0x10000003635' 2024-09-10T08:44:45.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1547 fp=#0x10000003f2a' 2024-09-10T08:44:45.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1579 fp=#0x10000003cb4' 2024-09-10T08:44:45.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1643 fp=#0x10000004732' 2024-09-10T08:44:45.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1675 fp=#0x100000045b5' 2024-09-10T08:44:45.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2059 fp=#0x1000000220e' 2024-09-10T08:44:45.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2187 fp=#0x10000002160' 2024-09-10T08:44:45.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2219 fp=#0x10000002605' 2024-09-10T08:44:45.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2251 fp=#0x10000002643' 2024-09-10T08:44:45.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2283 fp=#0x100000026dc' 2024-09-10T08:44:45.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2571 fp=#0x10000001216' 2024-09-10T08:44:45.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2603 fp=#0x1000000103b' 2024-09-10T08:44:45.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2667 fp=#0x10000001445' 2024-09-10T08:44:45.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2699 fp=#0x10000001089' 2024-09-10T08:44:45.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2731 fp=#0x100000041f4' 2024-09-10T08:44:45.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2763 fp=#0x1000000366e' 2024-09-10T08:44:45.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2795 fp=#0x10000003459' 2024-09-10T08:44:45.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2827 fp=#0x100000033f3' 2024-09-10T08:44:45.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2859 fp=#0x100000046f5' 2024-09-10T08:44:45.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2923 fp=#0x10000004712' 2024-09-10T08:44:45.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2955 fp=#0x100000033c1' 2024-09-10T08:44:45.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2987 fp=#0x100000046fb' 2024-09-10T08:44:45.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3019 fp=#0x10000000dc8' 2024-09-10T08:44:45.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3051 fp=#0x10000000de4' 2024-09-10T08:44:45.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3115 fp=#0x10000000dea' 2024-09-10T08:44:45.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3147 fp=#0x10000000df2' 2024-09-10T08:44:45.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3179 fp=#0x100000018bd' 2024-09-10T08:44:45.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3211 fp=#0x10000003c0d' 2024-09-10T08:44:45.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3243 fp=#0x10000003d21' 2024-09-10T08:44:45.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3275 fp=#0x10000003c58' 2024-09-10T08:44:45.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3307 fp=#0x10000000778' 2024-09-10T08:44:45.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3339 fp=#0x10000000d03' 2024-09-10T08:44:45.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3371 fp=#0x1000000077a' 2024-09-10T08:44:45.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3403 fp=#0x10000003cb2' 2024-09-10T08:44:45.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3435 fp=#0x10000000747' 2024-09-10T08:44:45.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3467 fp=#0x10000004107' 2024-09-10T08:44:45.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3499 fp=#0x10000003c52' 2024-09-10T08:44:45.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3659 fp=#0x10000004ca0' 2024-09-10T08:44:45.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3691 fp=#0x10000004aca' 2024-09-10T08:44:45.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3755 fp=#0x1000000492a' 2024-09-10T08:44:45.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3787 fp=#0x100000025b5' 2024-09-10T08:44:45.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3819 fp=#0x10000003ed1' 2024-09-10T08:44:45.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3851 fp=#0x10000002757' 2024-09-10T08:44:45.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3883 fp=#0x10000003fd7' 2024-09-10T08:44:45.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3915 fp=#0x10000000537' 2024-09-10T08:44:45.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3947 fp=#0x100000035b1' 2024-09-10T08:44:45.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3979 fp=#0x10000000cfc' 2024-09-10T08:44:45.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4011 fp=#0x1000000067c' 2024-09-10T08:44:45.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4043 fp=#0x10000004a8a' 2024-09-10T08:44:45.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4075 fp=#0x10000001d91' 2024-09-10T08:44:45.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4107 fp=#0x100000046e2' 2024-09-10T08:44:45.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4139 fp=#0x10000004daf' 2024-09-10T08:44:45.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4171 fp=#0x100000046c6' 2024-09-10T08:44:45.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4203 fp=#0x10000000f46' 2024-09-10T08:44:45.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4235 fp=#0x10000003a48' 2024-09-10T08:44:45.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4267 fp=#0x10000003c0b' 2024-09-10T08:44:45.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4299 fp=#0x10000004062' 2024-09-10T08:44:45.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4331 fp=#0x100000040c8' 2024-09-10T08:44:45.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4363 fp=#0x10000001ff1' 2024-09-10T08:44:45.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4459 fp=#0x10000001f19' 2024-09-10T08:44:45.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4491 fp=#0x10000001263' 2024-09-10T08:44:45.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4523 fp=#0x10000001255' 2024-09-10T08:44:45.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4555 fp=#0x100000017ae' 2024-09-10T08:44:45.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4587 fp=#0x1000000139e' 2024-09-10T08:44:45.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4619 fp=#0x100000012ba' 2024-09-10T08:44:45.956 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4651 fp=#0x10000001374' 2024-09-10T08:44:45.956 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4683 fp=#0x1000000395d' 2024-09-10T08:44:45.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4715 fp=#0x100000012d1' 2024-09-10T08:44:45.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4747 fp=#0x100000012ea' 2024-09-10T08:44:45.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4779 fp=#0x100000012a5' 2024-09-10T08:44:45.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4811 fp=#0x10000003870' 2024-09-10T08:44:45.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4843 fp=#0x10000004a78' 2024-09-10T08:44:45.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4875 fp=#0x100000037b2' 2024-09-10T08:44:45.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4907 fp=#0x100000037b9' 2024-09-10T08:44:45.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4939 fp=#0x10000004a91' 2024-09-10T08:44:45.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4971 fp=#0x10000005381' 2024-09-10T08:44:45.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5003 fp=#0x10000005024' 2024-09-10T08:44:45.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5035 fp=#0x100000051ad' 2024-09-10T08:44:45.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5067 fp=#0x100000050ce' 2024-09-10T08:44:45.961 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5099 fp=#0x100000050a7' 2024-09-10T08:44:45.961 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5131 fp=#0x10000001bb2' 2024-09-10T08:44:45.961 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5163 fp=#0x10000001c31' 2024-09-10T08:44:45.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5195 fp=#0x10000001b83' 2024-09-10T08:44:45.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5227 fp=#0x1000000438c' 2024-09-10T08:44:45.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5259 fp=#0x100000036b1' 2024-09-10T08:44:45.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5291 fp=#0x1000000356f' 2024-09-10T08:44:45.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5323 fp=#0x10000003724' 2024-09-10T08:44:45.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5355 fp=#0x1000000368f' 2024-09-10T08:44:45.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5387 fp=#0x1000000364d' 2024-09-10T08:44:45.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5419 fp=#0x10000004f8e' 2024-09-10T08:44:45.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5451 fp=#0x10000003476' 2024-09-10T08:44:45.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5483 fp=#0x100000014f8' 2024-09-10T08:44:45.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5515 fp=#0x10000001232' 2024-09-10T08:44:45.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5547 fp=#0x1000000272b' 2024-09-10T08:44:45.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5579 fp=#0x10000003c1e' 2024-09-10T08:44:45.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5611 fp=#0x10000001065' 2024-09-10T08:44:45.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5643 fp=#0x10000000fd3' 2024-09-10T08:44:45.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5675 fp=#0x10000000f5b' 2024-09-10T08:44:45.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5707 fp=#0x10000000fdb' 2024-09-10T08:44:45.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5739 fp=#0x100000050b5' 2024-09-10T08:44:45.968 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5771 fp=#0x100000050f9' 2024-09-10T08:44:45.968 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5803 fp=#0x10000004366' 2024-09-10T08:44:45.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5835 fp=#0x100000043b9' 2024-09-10T08:44:45.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5867 fp=#0x100000043f3' 2024-09-10T08:44:45.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5899 fp=#0x10000004315' 2024-09-10T08:44:45.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5931 fp=#0x100000044a5' 2024-09-10T08:44:45.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5963 fp=#0x10000004342' 2024-09-10T08:44:45.971 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6091 fp=#0x100000034f4' 2024-09-10T08:44:45.972 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6155 fp=#0x100000035ee' 2024-09-10T08:44:45.972 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6219 fp=#0x100000034e5' 2024-09-10T08:44:45.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6283 fp=#0x100000033a7' 2024-09-10T08:44:45.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6315 fp=#0x100000033e0' 2024-09-10T08:44:45.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6347 fp=#0x1000000333c' 2024-09-10T08:44:45.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6379 fp=#0x100000033ea' 2024-09-10T08:44:45.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6411 fp=#0x10000005379' 2024-09-10T08:44:45.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6443 fp=#0x10000004c2c' 2024-09-10T08:44:45.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6475 fp=#0x10000004bfc' 2024-09-10T08:44:45.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6507 fp=#0x10000004f2e' 2024-09-10T08:44:45.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6539 fp=#0x10000004c0d' 2024-09-10T08:44:45.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6603 fp=#0x10000003fab' 2024-09-10T08:44:45.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6635 fp=#0x10000003985' 2024-09-10T08:44:45.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6667 fp=#0x1000000500f' 2024-09-10T08:44:45.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6699 fp=#0x10000005282' 2024-09-10T08:44:45.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6731 fp=#0x10000003b07' 2024-09-10T08:44:45.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6763 fp=#0x10000005007' 2024-09-10T08:44:45.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6795 fp=#0x100000046b4' 2024-09-10T08:44:45.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6827 fp=#0x100000052b8' 2024-09-10T08:44:45.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6859 fp=#0x1000000464f' 2024-09-10T08:44:45.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6891 fp=#0x1000000479a' 2024-09-10T08:44:45.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6923 fp=#0x100000052bf' 2024-09-10T08:44:45.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6955 fp=#0x100000007d8' 2024-09-10T08:44:45.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6987 fp=#0x10000000859' 2024-09-10T08:44:45.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7019 fp=#0x10000000558' 2024-09-10T08:44:45.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7051 fp=#0x1000000060e' 2024-09-10T08:44:45.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7083 fp=#0x10000000626' 2024-09-10T08:44:45.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7115 fp=#0x100000005ed' 2024-09-10T08:44:45.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7147 fp=#0x10000000688' 2024-09-10T08:44:45.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7179 fp=#0x10000005058' 2024-09-10T08:44:45.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7403 fp=#0x10000002421' 2024-09-10T08:44:45.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7435 fp=#0x10000002475' 2024-09-10T08:44:45.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7467 fp=#0x10000004907' 2024-09-10T08:44:45.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7499 fp=#0x100000046d9' 2024-09-10T08:44:45.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7531 fp=#0x100000047b5' 2024-09-10T08:44:45.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7659 fp=#0x1000000515f' 2024-09-10T08:44:45.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7691 fp=#0x100000012fe' 2024-09-10T08:44:45.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7723 fp=#0x10000004f28' 2024-09-10T08:44:45.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7755 fp=#0x10000004e15' 2024-09-10T08:44:45.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7787 fp=#0x10000004f0f' 2024-09-10T08:44:45.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7819 fp=#0x10000004e60' 2024-09-10T08:44:45.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7851 fp=#0x1000000140d' 2024-09-10T08:44:45.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7883 fp=#0x1000000155d' 2024-09-10T08:44:45.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7915 fp=#0x10000004d1a' 2024-09-10T08:44:45.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7947 fp=#0x100000013c5' 2024-09-10T08:44:45.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7979 fp=#0x100000014e9' 2024-09-10T08:44:45.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8011 fp=#0x10000001564' 2024-09-10T08:44:45.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8043 fp=#0x10000001659' 2024-09-10T08:44:45.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8075 fp=#0x10000004e24' 2024-09-10T08:44:45.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8107 fp=#0x10000001610' 2024-09-10T08:44:45.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8139 fp=#0x100000026bb' 2024-09-10T08:44:45.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8171 fp=#0x10000002693' 2024-09-10T08:44:45.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8235 fp=#0x100000025e6' 2024-09-10T08:44:45.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8267 fp=#0x100000046b9' 2024-09-10T08:44:45.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8299 fp=#0x10000004617' 2024-09-10T08:44:46.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8331 fp=#0x10000005265' 2024-09-10T08:44:46.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8363 fp=#0x10000004195' 2024-09-10T08:44:46.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8395 fp=#0x10000005261' 2024-09-10T08:44:46.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8427 fp=#0x100000043f2' 2024-09-10T08:44:46.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8459 fp=#0x1000000456e' 2024-09-10T08:44:46.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8491 fp=#0x1000000427d' 2024-09-10T08:44:46.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8523 fp=#0x100000041e6' 2024-09-10T08:44:46.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8555 fp=#0x10000004078' 2024-09-10T08:44:46.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8587 fp=#0x10000003e93' 2024-09-10T08:44:46.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8619 fp=#0x100000041f3' 2024-09-10T08:44:46.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8651 fp=#0x100000040ce' 2024-09-10T08:44:46.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8683 fp=#0x10000003c26' 2024-09-10T08:44:46.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8715 fp=#0x10000003c74' 2024-09-10T08:44:46.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8747 fp=#0x10000004fc4' 2024-09-10T08:44:46.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8779 fp=#0x10000003c0a' 2024-09-10T08:44:46.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8811 fp=#0x10000003d11' 2024-09-10T08:44:46.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8843 fp=#0x10000001da5' 2024-09-10T08:44:46.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8875 fp=#0x10000001ca6' 2024-09-10T08:44:46.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8907 fp=#0x10000001b35' 2024-09-10T08:44:46.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8939 fp=#0x100000039a5' 2024-09-10T08:44:46.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8971 fp=#0x10000001d3b' 2024-09-10T08:44:46.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9003 fp=#0x1000000449a' 2024-09-10T08:44:46.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9035 fp=#0x10000000db6' 2024-09-10T08:44:46.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9067 fp=#0x10000000d33' 2024-09-10T08:44:46.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9099 fp=#0x1000000422b' 2024-09-10T08:44:46.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9131 fp=#0x1000000412a' 2024-09-10T08:44:46.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9163 fp=#0x10000000d37' 2024-09-10T08:44:46.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9195 fp=#0x10000000ce5' 2024-09-10T08:44:46.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9227 fp=#0x10000004254' 2024-09-10T08:44:46.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9259 fp=#0x10000000c03' 2024-09-10T08:44:46.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9291 fp=#0x10000000b9f' 2024-09-10T08:44:46.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9323 fp=#0x100000011a9' 2024-09-10T08:44:46.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9355 fp=#0x10000000b3d' 2024-09-10T08:44:46.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9387 fp=#0x10000004e26' 2024-09-10T08:44:46.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9419 fp=#0x10000001c5f' 2024-09-10T08:44:46.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9451 fp=#0x10000000aa4' 2024-09-10T08:44:46.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9483 fp=#0x10000000ee8' 2024-09-10T08:44:46.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9515 fp=#0x10000004930' 2024-09-10T08:44:46.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9547 fp=#0x10000001ca5' 2024-09-10T08:44:46.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9579 fp=#0x100000014b0' 2024-09-10T08:44:46.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9611 fp=#0x10000004a82' 2024-09-10T08:44:46.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9643 fp=#0x1000000488f' 2024-09-10T08:44:46.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9675 fp=#0x100000048be' 2024-09-10T08:44:46.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9707 fp=#0x100000047c8' 2024-09-10T08:44:46.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9739 fp=#0x100000043f5' 2024-09-10T08:44:46.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9771 fp=#0x100000043b1' 2024-09-10T08:44:46.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9803 fp=#0x10000004ca5' 2024-09-10T08:44:46.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9835 fp=#0x100000018c6' 2024-09-10T08:44:46.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9867 fp=#0x1000000437e' 2024-09-10T08:44:46.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9899 fp=#0x10000004c8e' 2024-09-10T08:44:46.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9931 fp=#0x1000000177d' 2024-09-10T08:44:46.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9995 fp=#0x100000018d5' 2024-09-10T08:44:46.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10027 fp=#0x10000001657' 2024-09-10T08:44:46.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10059 fp=#0x10000001627' 2024-09-10T08:44:46.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10091 fp=#0x10000001f49' 2024-09-10T08:44:46.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10123 fp=#0x10000003ae7' 2024-09-10T08:44:46.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10155 fp=#0x10000003abc' 2024-09-10T08:44:46.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10187 fp=#0x100000039b8' 2024-09-10T08:44:46.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10219 fp=#0x10000003939' 2024-09-10T08:44:46.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10251 fp=#0x10000003b40' 2024-09-10T08:44:46.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10283 fp=#0x10000002493' 2024-09-10T08:44:46.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10315 fp=#0x1000000247e' 2024-09-10T08:44:46.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10347 fp=#0x10000001393' 2024-09-10T08:44:46.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10379 fp=#0x1000000398a' 2024-09-10T08:44:46.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10411 fp=#0x1000000141a' 2024-09-10T08:44:46.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10443 fp=#0x10000003a60' 2024-09-10T08:44:46.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10475 fp=#0x1000000138c' 2024-09-10T08:44:46.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10507 fp=#0x100000039d1' 2024-09-10T08:44:46.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10539 fp=#0x1000000136c' 2024-09-10T08:44:46.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10635 fp=#0x10000003b84' 2024-09-10T08:44:46.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10763 fp=#0x1000000488a' 2024-09-10T08:44:46.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10795 fp=#0x10000000552' 2024-09-10T08:44:46.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10827 fp=#0x10000000a83' 2024-09-10T08:44:46.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10891 fp=#0x10000001d69' 2024-09-10T08:44:46.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10955 fp=#0x100000008b1' 2024-09-10T08:44:46.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11019 fp=#0x1000000063e' 2024-09-10T08:44:46.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11051 fp=#0x100000039b0' 2024-09-10T08:44:46.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11083 fp=#0x10000003ad1' 2024-09-10T08:44:46.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11115 fp=#0x10000002041' 2024-09-10T08:44:46.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11147 fp=#0x10000003e58' 2024-09-10T08:44:46.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11179 fp=#0x10000003e32' 2024-09-10T08:44:46.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11211 fp=#0x10000002390' 2024-09-10T08:44:46.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11243 fp=#0x1000000235f' 2024-09-10T08:44:46.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11275 fp=#0x100000020f1' 2024-09-10T08:44:46.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11307 fp=#0x10000001d63' 2024-09-10T08:44:46.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11339 fp=#0x10000001cda' 2024-09-10T08:44:46.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11371 fp=#0x10000003dc9' 2024-09-10T08:44:46.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11403 fp=#0x10000003f51' 2024-09-10T08:44:46.045 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11435 fp=#0x10000003e1a' 2024-09-10T08:44:46.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11467 fp=#0x10000003e79' 2024-09-10T08:44:46.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11499 fp=#0x10000001c9a' 2024-09-10T08:44:46.047 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11531 fp=#0x100000051b3' 2024-09-10T08:44:46.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11595 fp=#0x100000051bd' 2024-09-10T08:44:46.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11627 fp=#0x100000051ab' 2024-09-10T08:44:46.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11659 fp=#0x10000005153' 2024-09-10T08:44:46.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11723 fp=#0x10000000a1c' 2024-09-10T08:44:46.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11755 fp=#0x10000000afd' 2024-09-10T08:44:46.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11787 fp=#0x100000041f1' 2024-09-10T08:44:46.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11819 fp=#0x100000041e3' 2024-09-10T08:44:46.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11851 fp=#0x1000000181a' 2024-09-10T08:44:46.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11883 fp=#0x1000000474f' 2024-09-10T08:44:46.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11915 fp=#0x10000001105' 2024-09-10T08:44:46.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11947 fp=#0x100000010fc' 2024-09-10T08:44:46.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11979 fp=#0x100000052ce' 2024-09-10T08:44:46.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12011 fp=#0x100000010f3' 2024-09-10T08:44:46.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12043 fp=#0x100000052bc' 2024-09-10T08:44:46.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12075 fp=#0x100000034f2' 2024-09-10T08:44:46.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12139 fp=#0x1000000459c' 2024-09-10T08:44:46.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12171 fp=#0x100000034b1' 2024-09-10T08:44:46.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12203 fp=#0x100000045bf' 2024-09-10T08:44:46.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12235 fp=#0x10000000d6b' 2024-09-10T08:44:46.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12267 fp=#0x10000000cf9' 2024-09-10T08:44:46.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12299 fp=#0x10000000de0' 2024-09-10T08:44:46.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12331 fp=#0x10000000c7c' 2024-09-10T08:44:46.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12363 fp=#0x10000003a2d' 2024-09-10T08:44:46.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12395 fp=#0x10000000d53' 2024-09-10T08:44:46.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12427 fp=#0x10000001137' 2024-09-10T08:44:46.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12459 fp=#0x10000004802' 2024-09-10T08:44:46.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12491 fp=#0x10000004c53' 2024-09-10T08:44:46.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12555 fp=#0x10000004ccb' 2024-09-10T08:44:46.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12587 fp=#0x10000004aed' 2024-09-10T08:44:46.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12619 fp=#0x10000004c3c' 2024-09-10T08:44:46.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12651 fp=#0x1000000497f' 2024-09-10T08:44:46.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12683 fp=#0x100000053b3' 2024-09-10T08:44:46.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12715 fp=#0x10000004794' 2024-09-10T08:44:46.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12747 fp=#0x10000005345' 2024-09-10T08:44:46.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12779 fp=#0x100000022df' 2024-09-10T08:44:46.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12811 fp=#0x100000021f4' 2024-09-10T08:44:46.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12843 fp=#0x10000005301' 2024-09-10T08:44:46.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12875 fp=#0x100000020cc' 2024-09-10T08:44:46.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12907 fp=#0x1000000255c' 2024-09-10T08:44:46.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12939 fp=#0x10000005308' 2024-09-10T08:44:46.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12971 fp=#0x100000053f9' 2024-09-10T08:44:46.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13003 fp=#0x10000002771' 2024-09-10T08:44:46.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13027 fp=#0x100000022f7' 2024-09-10T08:44:46.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13059 fp=#0x100000022b2' 2024-09-10T08:44:46.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13091 fp=#0x1000000224d' 2024-09-10T08:44:46.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13123 fp=#0x100000022e3' 2024-09-10T08:44:46.070 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13155 fp=#0x100000039b9' 2024-09-10T08:44:46.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:244 fp=#0x10000002de8' 2024-09-10T08:44:46.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:276 fp=#0x10000004747' 2024-09-10T08:44:46.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:308 fp=#0x10000001a9f' 2024-09-10T08:44:46.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:524 fp=#0x10000003475' 2024-09-10T08:44:46.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:556 fp=#0x10000003525' 2024-09-10T08:44:46.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:620 fp=#0x100000032c8' 2024-09-10T08:44:46.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:812 fp=#0x10000000f20' 2024-09-10T08:44:46.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:844 fp=#0x10000004405' 2024-09-10T08:44:46.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:876 fp=#0x100000044bb' 2024-09-10T08:44:46.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:908 fp=#0x10000004039' 2024-09-10T08:44:46.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:972 fp=#0x10000000e59' 2024-09-10T08:44:46.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1004 fp=#0x10000000d7e' 2024-09-10T08:44:46.080 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1068 fp=#0x100000042ca' 2024-09-10T08:44:46.081 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1164 fp=#0x10000004262' 2024-09-10T08:44:46.082 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1228 fp=#0x10000002074' 2024-09-10T08:44:46.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1260 fp=#0x1000000208e' 2024-09-10T08:44:46.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1292 fp=#0x10000003573' 2024-09-10T08:44:46.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1324 fp=#0x1000000363d' 2024-09-10T08:44:46.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1388 fp=#0x100000032dc' 2024-09-10T08:44:46.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1516 fp=#0x10000003271' 2024-09-10T08:44:46.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1580 fp=#0x10000003ccf' 2024-09-10T08:44:46.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1644 fp=#0x1000000461c' 2024-09-10T08:44:46.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1676 fp=#0x100000045f6' 2024-09-10T08:44:46.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1708 fp=#0x1000000326b' 2024-09-10T08:44:46.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1964 fp=#0x10000002797' 2024-09-10T08:44:46.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1996 fp=#0x1000000436f' 2024-09-10T08:44:46.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2124 fp=#0x1000000219d' 2024-09-10T08:44:46.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2156 fp=#0x1000000200a' 2024-09-10T08:44:46.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2252 fp=#0x1000000279a' 2024-09-10T08:44:46.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2284 fp=#0x1000000276f' 2024-09-10T08:44:46.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2572 fp=#0x10000001220' 2024-09-10T08:44:46.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2604 fp=#0x1000000118a' 2024-09-10T08:44:46.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2668 fp=#0x1000000186b' 2024-09-10T08:44:46.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2700 fp=#0x10000001870' 2024-09-10T08:44:46.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2732 fp=#0x10000001198' 2024-09-10T08:44:46.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2764 fp=#0x100000046ce' 2024-09-10T08:44:46.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2796 fp=#0x100000033ba' 2024-09-10T08:44:46.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2828 fp=#0x10000004630' 2024-09-10T08:44:46.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2860 fp=#0x1000000468e' 2024-09-10T08:44:46.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2892 fp=#0x1000000357b' 2024-09-10T08:44:46.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2924 fp=#0x10000003474' 2024-09-10T08:44:46.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2956 fp=#0x10000004605' 2024-09-10T08:44:46.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2988 fp=#0x10000000e82' 2024-09-10T08:44:46.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3020 fp=#0x10000000e90' 2024-09-10T08:44:46.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3052 fp=#0x100000018ef' 2024-09-10T08:44:46.121 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3084 fp=#0x10000001992' 2024-09-10T08:44:46.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3180 fp=#0x10000000e5a' 2024-09-10T08:44:46.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3212 fp=#0x100000007a0' 2024-09-10T08:44:46.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3244 fp=#0x10000000755' 2024-09-10T08:44:46.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3276 fp=#0x10000000793' 2024-09-10T08:44:46.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3308 fp=#0x1000000070c' 2024-09-10T08:44:46.124 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3340 fp=#0x1000000076b' 2024-09-10T08:44:46.124 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3372 fp=#0x10000000783' 2024-09-10T08:44:46.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3404 fp=#0x10000003bd4' 2024-09-10T08:44:46.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3436 fp=#0x1000000074d' 2024-09-10T08:44:46.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3532 fp=#0x10000004958' 2024-09-10T08:44:46.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3628 fp=#0x10000003cc9' 2024-09-10T08:44:46.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3660 fp=#0x10000004c25' 2024-09-10T08:44:46.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3692 fp=#0x10000004c6f' 2024-09-10T08:44:46.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3724 fp=#0x1000000348f' 2024-09-10T08:44:46.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3756 fp=#0x10000004dc5' 2024-09-10T08:44:46.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3788 fp=#0x10000003fd3' 2024-09-10T08:44:46.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3820 fp=#0x1000000532c' 2024-09-10T08:44:46.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3852 fp=#0x1000000258f' 2024-09-10T08:44:46.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3884 fp=#0x10000003fae' 2024-09-10T08:44:46.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3916 fp=#0x1000000204c' 2024-09-10T08:44:46.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3948 fp=#0x10000000695' 2024-09-10T08:44:46.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3980 fp=#0x1000000066f' 2024-09-10T08:44:46.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4012 fp=#0x10000000664' 2024-09-10T08:44:46.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4044 fp=#0x10000000fe9' 2024-09-10T08:44:46.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4076 fp=#0x10000000f95' 2024-09-10T08:44:46.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4108 fp=#0x10000001f07' 2024-09-10T08:44:46.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4140 fp=#0x10000004687' 2024-09-10T08:44:46.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4172 fp=#0x10000000f34' 2024-09-10T08:44:46.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4204 fp=#0x10000004eb3' 2024-09-10T08:44:46.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4236 fp=#0x10000003a29' 2024-09-10T08:44:46.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4268 fp=#0x10000003a8b' 2024-09-10T08:44:46.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4300 fp=#0x10000004183' 2024-09-10T08:44:46.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4332 fp=#0x1000000396d' 2024-09-10T08:44:46.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4364 fp=#0x10000001fd7' 2024-09-10T08:44:46.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4428 fp=#0x10000004d16' 2024-09-10T08:44:46.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4460 fp=#0x10000001fcd' 2024-09-10T08:44:46.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4492 fp=#0x10000005113' 2024-09-10T08:44:46.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4524 fp=#0x100000050eb' 2024-09-10T08:44:46.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4556 fp=#0x10000005154' 2024-09-10T08:44:46.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4588 fp=#0x10000001308' 2024-09-10T08:44:46.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4620 fp=#0x100000012c3' 2024-09-10T08:44:46.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4652 fp=#0x1000000153c' 2024-09-10T08:44:46.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4684 fp=#0x10000001d44' 2024-09-10T08:44:46.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4716 fp=#0x10000003995' 2024-09-10T08:44:46.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4748 fp=#0x100000038f6' 2024-09-10T08:44:46.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4780 fp=#0x100000012e7' 2024-09-10T08:44:46.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4812 fp=#0x10000003722' 2024-09-10T08:44:46.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4844 fp=#0x100000037f6' 2024-09-10T08:44:46.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4876 fp=#0x10000004aa7' 2024-09-10T08:44:46.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4908 fp=#0x10000003702' 2024-09-10T08:44:46.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4940 fp=#0x1000000381b' 2024-09-10T08:44:46.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4972 fp=#0x100000050be' 2024-09-10T08:44:46.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5004 fp=#0x100000051a4' 2024-09-10T08:44:46.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5036 fp=#0x100000053bb' 2024-09-10T08:44:46.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5068 fp=#0x10000005347' 2024-09-10T08:44:46.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5100 fp=#0x100000051d8' 2024-09-10T08:44:46.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5132 fp=#0x10000001b99' 2024-09-10T08:44:46.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5164 fp=#0x10000001bf5' 2024-09-10T08:44:46.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5196 fp=#0x10000001e61' 2024-09-10T08:44:46.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5228 fp=#0x10000004387' 2024-09-10T08:44:46.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5260 fp=#0x100000038f3' 2024-09-10T08:44:46.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5292 fp=#0x10000003d6a' 2024-09-10T08:44:46.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5324 fp=#0x10000004a99' 2024-09-10T08:44:46.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5356 fp=#0x10000003d9f' 2024-09-10T08:44:46.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5388 fp=#0x10000003d32' 2024-09-10T08:44:46.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5420 fp=#0x10000003679' 2024-09-10T08:44:46.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5452 fp=#0x1000000254d' 2024-09-10T08:44:46.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5484 fp=#0x100000012ad' 2024-09-10T08:44:46.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5580 fp=#0x100000010f9' 2024-09-10T08:44:46.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5612 fp=#0x100000014a3' 2024-09-10T08:44:46.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5644 fp=#0x100000020e3' 2024-09-10T08:44:46.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5676 fp=#0x10000005090' 2024-09-10T08:44:46.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5708 fp=#0x100000020fe' 2024-09-10T08:44:46.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5740 fp=#0x1000000509f' 2024-09-10T08:44:46.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5772 fp=#0x10000005081' 2024-09-10T08:44:46.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5804 fp=#0x100000044fa' 2024-09-10T08:44:46.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5836 fp=#0x10000004493' 2024-09-10T08:44:46.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5868 fp=#0x10000004428' 2024-09-10T08:44:46.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5900 fp=#0x1000000432b' 2024-09-10T08:44:46.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5932 fp=#0x1000000444b' 2024-09-10T08:44:46.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5964 fp=#0x10000004415' 2024-09-10T08:44:46.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6188 fp=#0x1000000360b' 2024-09-10T08:44:46.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6220 fp=#0x1000000347d' 2024-09-10T08:44:46.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6284 fp=#0x10000003340' 2024-09-10T08:44:46.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6316 fp=#0x100000033af' 2024-09-10T08:44:46.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6348 fp=#0x10000001e15' 2024-09-10T08:44:46.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6380 fp=#0x100000033a0' 2024-09-10T08:44:46.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6412 fp=#0x10000004b32' 2024-09-10T08:44:46.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6444 fp=#0x10000004c1f' 2024-09-10T08:44:46.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6476 fp=#0x10000004bfa' 2024-09-10T08:44:46.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6508 fp=#0x10000004e0d' 2024-09-10T08:44:46.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6540 fp=#0x10000004ae5' 2024-09-10T08:44:46.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6636 fp=#0x100000044cd' 2024-09-10T08:44:46.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6668 fp=#0x100000039b7' 2024-09-10T08:44:46.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6700 fp=#0x10000005041' 2024-09-10T08:44:46.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6732 fp=#0x10000003adb' 2024-09-10T08:44:46.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6764 fp=#0x10000003916' 2024-09-10T08:44:46.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6796 fp=#0x10000004987' 2024-09-10T08:44:46.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6828 fp=#0x100000045c4' 2024-09-10T08:44:46.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6860 fp=#0x1000000468a' 2024-09-10T08:44:46.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6892 fp=#0x1000000471d' 2024-09-10T08:44:46.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6924 fp=#0x100000046fd' 2024-09-10T08:44:46.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6956 fp=#0x10000004f63' 2024-09-10T08:44:46.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6988 fp=#0x10000005039' 2024-09-10T08:44:46.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7020 fp=#0x1000000064c' 2024-09-10T08:44:46.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7052 fp=#0x10000001e00' 2024-09-10T08:44:46.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7084 fp=#0x10000004fe3' 2024-09-10T08:44:46.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7116 fp=#0x10000000636' 2024-09-10T08:44:46.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7148 fp=#0x1000000067d' 2024-09-10T08:44:46.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7180 fp=#0x10000004f98' 2024-09-10T08:44:46.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7212 fp=#0x1000000512a' 2024-09-10T08:44:46.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7244 fp=#0x100000047d8' 2024-09-10T08:44:46.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7276 fp=#0x10000005066' 2024-09-10T08:44:46.179 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7404 fp=#0x10000002482' 2024-09-10T08:44:46.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7436 fp=#0x100000046b8' 2024-09-10T08:44:46.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7468 fp=#0x100000024a9' 2024-09-10T08:44:46.181 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7500 fp=#0x10000004944' 2024-09-10T08:44:46.181 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7532 fp=#0x1000000137a' 2024-09-10T08:44:46.181 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7564 fp=#0x1000000227c' 2024-09-10T08:44:46.182 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7596 fp=#0x100000051d9' 2024-09-10T08:44:46.182 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7628 fp=#0x100000051d5' 2024-09-10T08:44:46.183 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7660 fp=#0x10000005163' 2024-09-10T08:44:46.183 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7692 fp=#0x100000012f8' 2024-09-10T08:44:46.184 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7724 fp=#0x10000004ef1' 2024-09-10T08:44:46.184 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7756 fp=#0x10000004e4f' 2024-09-10T08:44:46.185 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7788 fp=#0x10000004eaf' 2024-09-10T08:44:46.185 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7820 fp=#0x10000004d59' 2024-09-10T08:44:46.185 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7852 fp=#0x100000016b1' 2024-09-10T08:44:46.186 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7884 fp=#0x1000000169b' 2024-09-10T08:44:46.186 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7916 fp=#0x10000004f03' 2024-09-10T08:44:46.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7948 fp=#0x100000016bf' 2024-09-10T08:44:46.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7980 fp=#0x10000004d29' 2024-09-10T08:44:46.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8012 fp=#0x10000004f2a' 2024-09-10T08:44:46.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8044 fp=#0x10000003881' 2024-09-10T08:44:46.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8076 fp=#0x10000003fee' 2024-09-10T08:44:46.189 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8108 fp=#0x10000001f60' 2024-09-10T08:44:46.189 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8140 fp=#0x10000002724' 2024-09-10T08:44:46.190 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8204 fp=#0x1000000262d' 2024-09-10T08:44:46.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8236 fp=#0x10000002574' 2024-09-10T08:44:46.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8268 fp=#0x10000005275' 2024-09-10T08:44:46.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8300 fp=#0x10000004408' 2024-09-10T08:44:46.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8332 fp=#0x1000000447b' 2024-09-10T08:44:46.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8364 fp=#0x1000000455e' 2024-09-10T08:44:46.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8396 fp=#0x10000004131' 2024-09-10T08:44:46.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8428 fp=#0x10000004467' 2024-09-10T08:44:46.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8460 fp=#0x10000004095' 2024-09-10T08:44:46.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8492 fp=#0x10000004250' 2024-09-10T08:44:46.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8524 fp=#0x100000041b6' 2024-09-10T08:44:46.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8556 fp=#0x10000003f2c' 2024-09-10T08:44:46.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8588 fp=#0x100000045f5' 2024-09-10T08:44:46.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8620 fp=#0x10000003c63' 2024-09-10T08:44:46.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8652 fp=#0x100000052ec' 2024-09-10T08:44:46.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8684 fp=#0x10000004efb' 2024-09-10T08:44:46.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8716 fp=#0x10000004f70' 2024-09-10T08:44:46.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8748 fp=#0x10000004f06' 2024-09-10T08:44:46.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8780 fp=#0x10000004030' 2024-09-10T08:44:46.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8812 fp=#0x10000003dc8' 2024-09-10T08:44:46.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8844 fp=#0x10000001b7d' 2024-09-10T08:44:46.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8876 fp=#0x1000000442d' 2024-09-10T08:44:46.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8908 fp=#0x10000001d93' 2024-09-10T08:44:46.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8940 fp=#0x100000043d7' 2024-09-10T08:44:46.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8972 fp=#0x10000001b2b' 2024-09-10T08:44:46.201 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9004 fp=#0x1000000444a' 2024-09-10T08:44:46.201 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9036 fp=#0x10000000ca9' 2024-09-10T08:44:46.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9068 fp=#0x10000000d6c' 2024-09-10T08:44:46.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9100 fp=#0x10000004219' 2024-09-10T08:44:46.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9132 fp=#0x1000000427f' 2024-09-10T08:44:46.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9164 fp=#0x10000000d1b' 2024-09-10T08:44:46.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9196 fp=#0x1000000404f' 2024-09-10T08:44:46.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9228 fp=#0x10000000d27' 2024-09-10T08:44:46.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9260 fp=#0x1000000113b' 2024-09-10T08:44:46.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9292 fp=#0x10000000bf3' 2024-09-10T08:44:46.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9324 fp=#0x10000002612' 2024-09-10T08:44:46.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9356 fp=#0x10000004e1e' 2024-09-10T08:44:46.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9388 fp=#0x10000001c30' 2024-09-10T08:44:46.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9420 fp=#0x10000004c50' 2024-09-10T08:44:46.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9452 fp=#0x10000000b22' 2024-09-10T08:44:46.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9484 fp=#0x10000004a7e' 2024-09-10T08:44:46.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9516 fp=#0x100000048c7' 2024-09-10T08:44:46.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9548 fp=#0x10000001ee7' 2024-09-10T08:44:46.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9580 fp=#0x10000000ec9' 2024-09-10T08:44:46.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9612 fp=#0x10000000eb8' 2024-09-10T08:44:46.214 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9644 fp=#0x10000001744' 2024-09-10T08:44:46.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9676 fp=#0x100000048dc' 2024-09-10T08:44:46.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9708 fp=#0x10000004881' 2024-09-10T08:44:46.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9740 fp=#0x10000004407' 2024-09-10T08:44:46.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9772 fp=#0x10000004339' 2024-09-10T08:44:46.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9804 fp=#0x100000017a0' 2024-09-10T08:44:46.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9836 fp=#0x100000018f1' 2024-09-10T08:44:46.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9868 fp=#0x10000004454' 2024-09-10T08:44:46.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9900 fp=#0x10000004425' 2024-09-10T08:44:46.221 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9932 fp=#0x10000001a24' 2024-09-10T08:44:46.221 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9964 fp=#0x10000004db5' 2024-09-10T08:44:46.221 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9996 fp=#0x100000026a6' 2024-09-10T08:44:46.222 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10028 fp=#0x100000019ec' 2024-09-10T08:44:46.222 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10060 fp=#0x10000001982' 2024-09-10T08:44:46.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10092 fp=#0x10000001dd9' 2024-09-10T08:44:46.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10124 fp=#0x10000003ab4' 2024-09-10T08:44:46.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10156 fp=#0x10000003b8f' 2024-09-10T08:44:46.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10188 fp=#0x100000045c1' 2024-09-10T08:44:46.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10220 fp=#0x10000003999' 2024-09-10T08:44:46.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10252 fp=#0x10000002015' 2024-09-10T08:44:46.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10284 fp=#0x1000000248a' 2024-09-10T08:44:46.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10316 fp=#0x100000024b4' 2024-09-10T08:44:46.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10348 fp=#0x10000001534' 2024-09-10T08:44:46.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10380 fp=#0x10000001334' 2024-09-10T08:44:46.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10412 fp=#0x10000001385' 2024-09-10T08:44:46.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10444 fp=#0x1000000390f' 2024-09-10T08:44:46.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10476 fp=#0x1000000133a' 2024-09-10T08:44:46.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10508 fp=#0x100000013f2' 2024-09-10T08:44:46.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10604 fp=#0x10000000e48' 2024-09-10T08:44:46.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10700 fp=#0x10000001033' 2024-09-10T08:44:46.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10764 fp=#0x100000048bd' 2024-09-10T08:44:46.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10796 fp=#0x10000000a56' 2024-09-10T08:44:46.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10892 fp=#0x10000000a73' 2024-09-10T08:44:46.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10924 fp=#0x10000001d1a' 2024-09-10T08:44:46.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10956 fp=#0x1000000073d' 2024-09-10T08:44:46.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10988 fp=#0x10000001cf9' 2024-09-10T08:44:46.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11020 fp=#0x1000000070a' 2024-09-10T08:44:46.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11052 fp=#0x10000005313' 2024-09-10T08:44:46.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11084 fp=#0x1000000395f' 2024-09-10T08:44:46.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11116 fp=#0x10000002169' 2024-09-10T08:44:46.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11148 fp=#0x1000000206e' 2024-09-10T08:44:46.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11180 fp=#0x10000004923' 2024-09-10T08:44:46.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11212 fp=#0x10000003e73' 2024-09-10T08:44:46.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11244 fp=#0x10000003e8a' 2024-09-10T08:44:46.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11276 fp=#0x1000000222b' 2024-09-10T08:44:46.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11308 fp=#0x10000001cad' 2024-09-10T08:44:46.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11340 fp=#0x10000003ea8' 2024-09-10T08:44:46.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11372 fp=#0x10000003d5e' 2024-09-10T08:44:46.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11404 fp=#0x1000000199a' 2024-09-10T08:44:46.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11436 fp=#0x10000001a82' 2024-09-10T08:44:46.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11468 fp=#0x10000004989' 2024-09-10T08:44:46.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11500 fp=#0x10000003f5b' 2024-09-10T08:44:46.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11596 fp=#0x10000005141' 2024-09-10T08:44:46.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11724 fp=#0x10000000a3e' 2024-09-10T08:44:46.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11756 fp=#0x10000003e9c' 2024-09-10T08:44:46.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11788 fp=#0x10000004e4b' 2024-09-10T08:44:46.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11820 fp=#0x1000000093c' 2024-09-10T08:44:46.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11852 fp=#0x100000041c9' 2024-09-10T08:44:46.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11884 fp=#0x10000004d9b' 2024-09-10T08:44:46.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11916 fp=#0x10000000f2a' 2024-09-10T08:44:46.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11948 fp=#0x100000010ca' 2024-09-10T08:44:46.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11980 fp=#0x1000000222e' 2024-09-10T08:44:46.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12012 fp=#0x10000002639' 2024-09-10T08:44:46.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12044 fp=#0x1000000529e' 2024-09-10T08:44:46.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12076 fp=#0x1000000333b' 2024-09-10T08:44:46.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12108 fp=#0x10000003334' 2024-09-10T08:44:46.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12140 fp=#0x10000004497' 2024-09-10T08:44:46.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12172 fp=#0x10000004478' 2024-09-10T08:44:46.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12204 fp=#0x10000004557' 2024-09-10T08:44:46.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12236 fp=#0x100000051f2' 2024-09-10T08:44:46.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12300 fp=#0x10000000c0d' 2024-09-10T08:44:46.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12332 fp=#0x10000000cda' 2024-09-10T08:44:46.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12364 fp=#0x10000000d4f' 2024-09-10T08:44:46.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12396 fp=#0x10000003a6b' 2024-09-10T08:44:46.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12428 fp=#0x10000000bfa' 2024-09-10T08:44:46.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12460 fp=#0x10000004c63' 2024-09-10T08:44:46.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12524 fp=#0x10000004705' 2024-09-10T08:44:46.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12556 fp=#0x10000004ca7' 2024-09-10T08:44:46.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12588 fp=#0x10000004ab4' 2024-09-10T08:44:46.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12620 fp=#0x10000004b3b' 2024-09-10T08:44:46.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12652 fp=#0x10000004979' 2024-09-10T08:44:46.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12684 fp=#0x1000000487a' 2024-09-10T08:44:46.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12716 fp=#0x1000000540a' 2024-09-10T08:44:46.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12748 fp=#0x100000048f5' 2024-09-10T08:44:46.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12780 fp=#0x1000000210b' 2024-09-10T08:44:46.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12812 fp=#0x100000022e4' 2024-09-10T08:44:46.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12844 fp=#0x1000000218c' 2024-09-10T08:44:46.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12876 fp=#0x100000021e5' 2024-09-10T08:44:46.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12908 fp=#0x100000052e5' 2024-09-10T08:44:46.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12940 fp=#0x1000000354c' 2024-09-10T08:44:46.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12972 fp=#0x10000005390' 2024-09-10T08:44:46.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13004 fp=#0x10000002794' 2024-09-10T08:44:46.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13028 fp=#0x10000003a36' 2024-09-10T08:44:46.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13060 fp=#0x10000002225' 2024-09-10T08:44:46.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13124 fp=#0x1000000539f' 2024-09-10T08:44:46.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13156 fp=#0x10000003988' 2024-09-10T08:44:46.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:277 fp=#0x1000000122a' 2024-09-10T08:44:46.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:365 fp=#0x10000004b1b' 2024-09-10T08:44:46.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:397 fp=#0x10000004651' 2024-09-10T08:44:46.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:429 fp=#0x1000000472e' 2024-09-10T08:44:46.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:557 fp=#0x10000003321' 2024-09-10T08:44:46.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:589 fp=#0x100000034e0' 2024-09-10T08:44:46.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:621 fp=#0x10000003462' 2024-09-10T08:44:46.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:781 fp=#0x10000000dbf' 2024-09-10T08:44:46.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:813 fp=#0x10000004461' 2024-09-10T08:44:46.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:845 fp=#0x10000000e77' 2024-09-10T08:44:46.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:909 fp=#0x10000000e7d' 2024-09-10T08:44:46.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:941 fp=#0x10000000ddb' 2024-09-10T08:44:46.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:973 fp=#0x10000000da9' 2024-09-10T08:44:46.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1005 fp=#0x10000004599' 2024-09-10T08:44:46.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1069 fp=#0x1000000414d' 2024-09-10T08:44:46.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1101 fp=#0x10000004252' 2024-09-10T08:44:46.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1165 fp=#0x10000004198' 2024-09-10T08:44:46.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1229 fp=#0x10000002093' 2024-09-10T08:44:46.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1261 fp=#0x10000003578' 2024-09-10T08:44:46.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1293 fp=#0x10000003593' 2024-09-10T08:44:46.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1325 fp=#0x10000001fc8' 2024-09-10T08:44:46.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1357 fp=#0x1000000363a' 2024-09-10T08:44:46.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1389 fp=#0x10000003e8b' 2024-09-10T08:44:46.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1485 fp=#0x10000003266' 2024-09-10T08:44:46.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1517 fp=#0x1000000434f' 2024-09-10T08:44:46.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1549 fp=#0x10000003f36' 2024-09-10T08:44:46.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1645 fp=#0x10000004610' 2024-09-10T08:44:46.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2061 fp=#0x1000000202f' 2024-09-10T08:44:46.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2093 fp=#0x10000001ff8' 2024-09-10T08:44:46.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2221 fp=#0x1000000267e' 2024-09-10T08:44:46.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2253 fp=#0x100000027a3' 2024-09-10T08:44:46.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2573 fp=#0x10000001032' 2024-09-10T08:44:46.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2605 fp=#0x100000010ee' 2024-09-10T08:44:46.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2637 fp=#0x100000018da' 2024-09-10T08:44:46.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2669 fp=#0x1000000450e' 2024-09-10T08:44:46.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2765 fp=#0x10000003656' 2024-09-10T08:44:46.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2797 fp=#0x1000000368b' 2024-09-10T08:44:46.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2829 fp=#0x1000000467c' 2024-09-10T08:44:46.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2893 fp=#0x100000046b1' 2024-09-10T08:44:46.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2925 fp=#0x10000003417' 2024-09-10T08:44:46.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2957 fp=#0x10000003599' 2024-09-10T08:44:46.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2989 fp=#0x1000000197d' 2024-09-10T08:44:46.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3021 fp=#0x10000001971' 2024-09-10T08:44:46.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3053 fp=#0x10000001933' 2024-09-10T08:44:46.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3085 fp=#0x10000001944' 2024-09-10T08:44:46.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3117 fp=#0x10000000e57' 2024-09-10T08:44:46.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3181 fp=#0x10000004a47' 2024-09-10T08:44:46.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3213 fp=#0x100000045ee' 2024-09-10T08:44:46.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3277 fp=#0x100000006fb' 2024-09-10T08:44:46.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3309 fp=#0x10000003c2a' 2024-09-10T08:44:46.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3341 fp=#0x1000000073e' 2024-09-10T08:44:46.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3373 fp=#0x10000001f54' 2024-09-10T08:44:46.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3405 fp=#0x10000003c1c' 2024-09-10T08:44:46.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3437 fp=#0x10000000766' 2024-09-10T08:44:46.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3501 fp=#0x10000003c2f' 2024-09-10T08:44:46.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3629 fp=#0x10000004c7f' 2024-09-10T08:44:46.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3661 fp=#0x10000004ac4' 2024-09-10T08:44:46.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3693 fp=#0x10000004b71' 2024-09-10T08:44:46.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3725 fp=#0x10000005330' 2024-09-10T08:44:46.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3757 fp=#0x10000003f33' 2024-09-10T08:44:46.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3789 fp=#0x10000003e72' 2024-09-10T08:44:46.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3821 fp=#0x1000000271a' 2024-09-10T08:44:46.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3885 fp=#0x10000003f91' 2024-09-10T08:44:46.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3917 fp=#0x100000006a1' 2024-09-10T08:44:46.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3949 fp=#0x10000000557' 2024-09-10T08:44:46.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3981 fp=#0x1000000052b' 2024-09-10T08:44:46.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4013 fp=#0x10000000534' 2024-09-10T08:44:46.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4045 fp=#0x10000000f56' 2024-09-10T08:44:46.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4077 fp=#0x10000004de7' 2024-09-10T08:44:46.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4109 fp=#0x10000001008' 2024-09-10T08:44:46.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4141 fp=#0x100000049be' 2024-09-10T08:44:46.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4173 fp=#0x10000000f5a' 2024-09-10T08:44:46.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4205 fp=#0x10000004e19' 2024-09-10T08:44:46.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4237 fp=#0x10000003986' 2024-09-10T08:44:46.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4269 fp=#0x10000003ae5' 2024-09-10T08:44:46.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4301 fp=#0x10000004667' 2024-09-10T08:44:46.315 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4333 fp=#0x10000003a75' 2024-09-10T08:44:46.315 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4365 fp=#0x100000021e3' 2024-09-10T08:44:46.316 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4397 fp=#0x10000001fb1' 2024-09-10T08:44:46.316 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4429 fp=#0x10000001ff9' 2024-09-10T08:44:46.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4493 fp=#0x10000001251' 2024-09-10T08:44:46.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4525 fp=#0x10000004033' 2024-09-10T08:44:46.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4557 fp=#0x100000050a9' 2024-09-10T08:44:46.318 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4589 fp=#0x100000015c0' 2024-09-10T08:44:46.318 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4653 fp=#0x100000012fa' 2024-09-10T08:44:46.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4685 fp=#0x100000012cd' 2024-09-10T08:44:46.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4717 fp=#0x10000001552' 2024-09-10T08:44:46.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4781 fp=#0x100000012ef' 2024-09-10T08:44:46.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4813 fp=#0x100000036be' 2024-09-10T08:44:46.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4845 fp=#0x100000036f1' 2024-09-10T08:44:46.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4877 fp=#0x1000000386a' 2024-09-10T08:44:46.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4909 fp=#0x1000000373f' 2024-09-10T08:44:46.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4941 fp=#0x10000004aa2' 2024-09-10T08:44:46.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4973 fp=#0x10000005372' 2024-09-10T08:44:46.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5005 fp=#0x10000005004' 2024-09-10T08:44:46.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5037 fp=#0x10000005193' 2024-09-10T08:44:46.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5069 fp=#0x10000005191' 2024-09-10T08:44:46.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5101 fp=#0x1000000500d' 2024-09-10T08:44:46.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5133 fp=#0x10000001d01' 2024-09-10T08:44:46.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5165 fp=#0x10000001c66' 2024-09-10T08:44:46.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5197 fp=#0x100000027b8' 2024-09-10T08:44:46.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5229 fp=#0x10000001d66' 2024-09-10T08:44:46.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5261 fp=#0x100000037e2' 2024-09-10T08:44:46.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5293 fp=#0x100000046a3' 2024-09-10T08:44:46.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5325 fp=#0x10000003831' 2024-09-10T08:44:46.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5357 fp=#0x10000003dc1' 2024-09-10T08:44:46.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5389 fp=#0x10000003718' 2024-09-10T08:44:46.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5421 fp=#0x10000003686' 2024-09-10T08:44:46.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5453 fp=#0x100000012b1' 2024-09-10T08:44:46.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5485 fp=#0x100000014fe' 2024-09-10T08:44:46.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5581 fp=#0x1000000101e' 2024-09-10T08:44:46.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5613 fp=#0x1000000509a' 2024-09-10T08:44:46.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5645 fp=#0x100000020b7' 2024-09-10T08:44:46.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5677 fp=#0x1000000208f' 2024-09-10T08:44:46.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5709 fp=#0x1000000117a' 2024-09-10T08:44:46.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5741 fp=#0x10000001019' 2024-09-10T08:44:46.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5773 fp=#0x10000005096' 2024-09-10T08:44:46.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5805 fp=#0x10000004430' 2024-09-10T08:44:46.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5837 fp=#0x10000004c59' 2024-09-10T08:44:46.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5869 fp=#0x10000004508' 2024-09-10T08:44:46.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5901 fp=#0x10000004439' 2024-09-10T08:44:46.334 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5933 fp=#0x10000004412' 2024-09-10T08:44:46.334 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5965 fp=#0x1000000434c' 2024-09-10T08:44:46.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6093 fp=#0x1000000350d' 2024-09-10T08:44:46.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6125 fp=#0x1000000457d' 2024-09-10T08:44:46.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6189 fp=#0x10000003470' 2024-09-10T08:44:46.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6221 fp=#0x1000000362e' 2024-09-10T08:44:46.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6253 fp=#0x100000049e2' 2024-09-10T08:44:46.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6317 fp=#0x10000004383' 2024-09-10T08:44:46.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6349 fp=#0x1000000340a' 2024-09-10T08:44:46.339 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6381 fp=#0x10000003383' 2024-09-10T08:44:46.339 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6413 fp=#0x10000004dc8' 2024-09-10T08:44:46.340 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6445 fp=#0x10000004c19' 2024-09-10T08:44:46.340 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6477 fp=#0x10000004d1b' 2024-09-10T08:44:46.340 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6509 fp=#0x10000004de9' 2024-09-10T08:44:46.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6541 fp=#0x10000004d0a' 2024-09-10T08:44:46.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6573 fp=#0x100000052bb' 2024-09-10T08:44:46.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6605 fp=#0x10000003c0e' 2024-09-10T08:44:46.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6637 fp=#0x100000039f7' 2024-09-10T08:44:46.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6669 fp=#0x10000003c80' 2024-09-10T08:44:46.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6701 fp=#0x100000039a3' 2024-09-10T08:44:46.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6733 fp=#0x100000039bd' 2024-09-10T08:44:46.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6765 fp=#0x10000003a33' 2024-09-10T08:44:46.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6797 fp=#0x100000049ac' 2024-09-10T08:44:46.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6829 fp=#0x1000000482d' 2024-09-10T08:44:46.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6861 fp=#0x10000004973' 2024-09-10T08:44:46.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6893 fp=#0x1000000455d' 2024-09-10T08:44:46.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6925 fp=#0x100000047d6' 2024-09-10T08:44:46.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6957 fp=#0x1000000275b' 2024-09-10T08:44:46.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6989 fp=#0x1000000064b' 2024-09-10T08:44:46.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7021 fp=#0x10000004fa4' 2024-09-10T08:44:46.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7053 fp=#0x10000000839' 2024-09-10T08:44:46.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7085 fp=#0x10000004fff' 2024-09-10T08:44:46.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7117 fp=#0x100000006b2' 2024-09-10T08:44:46.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7149 fp=#0x1000000080a' 2024-09-10T08:44:46.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7181 fp=#0x10000004fc0' 2024-09-10T08:44:46.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7213 fp=#0x100000050fc' 2024-09-10T08:44:46.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7245 fp=#0x10000005079' 2024-09-10T08:44:46.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7309 fp=#0x1000000503e' 2024-09-10T08:44:46.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7405 fp=#0x10000002465' 2024-09-10T08:44:46.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7437 fp=#0x10000004723' 2024-09-10T08:44:46.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7469 fp=#0x100000047a7' 2024-09-10T08:44:46.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7501 fp=#0x1000000469a' 2024-09-10T08:44:46.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7565 fp=#0x10000005189' 2024-09-10T08:44:46.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7597 fp=#0x1000000135f' 2024-09-10T08:44:46.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7661 fp=#0x10000001363' 2024-09-10T08:44:46.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7693 fp=#0x10000001303' 2024-09-10T08:44:46.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7725 fp=#0x10000004f74' 2024-09-10T08:44:46.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7757 fp=#0x10000004edf' 2024-09-10T08:44:46.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7789 fp=#0x100000013fb' 2024-09-10T08:44:46.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7821 fp=#0x10000001cf7' 2024-09-10T08:44:46.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7853 fp=#0x100000024fd' 2024-09-10T08:44:46.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7885 fp=#0x10000004ec8' 2024-09-10T08:44:46.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7917 fp=#0x10000001730' 2024-09-10T08:44:46.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7949 fp=#0x10000001500' 2024-09-10T08:44:46.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7981 fp=#0x10000004e4c' 2024-09-10T08:44:46.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8013 fp=#0x10000004ece' 2024-09-10T08:44:46.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8045 fp=#0x10000003846' 2024-09-10T08:44:46.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8077 fp=#0x100000018a5' 2024-09-10T08:44:46.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8109 fp=#0x10000001672' 2024-09-10T08:44:46.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8141 fp=#0x1000000265a' 2024-09-10T08:44:46.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8173 fp=#0x10000004975' 2024-09-10T08:44:46.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8205 fp=#0x100000025b4' 2024-09-10T08:44:46.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8237 fp=#0x1000000497c' 2024-09-10T08:44:46.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8269 fp=#0x1000000526c' 2024-09-10T08:44:46.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8301 fp=#0x10000005251' 2024-09-10T08:44:46.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8333 fp=#0x100000041af' 2024-09-10T08:44:46.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8365 fp=#0x10000005243' 2024-09-10T08:44:46.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8397 fp=#0x1000000462e' 2024-09-10T08:44:46.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8429 fp=#0x10000004435' 2024-09-10T08:44:46.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8461 fp=#0x10000003de5' 2024-09-10T08:44:46.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8493 fp=#0x100000041bb' 2024-09-10T08:44:46.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8525 fp=#0x10000003e2d' 2024-09-10T08:44:46.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8557 fp=#0x10000004209' 2024-09-10T08:44:46.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8589 fp=#0x10000003da3' 2024-09-10T08:44:46.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8621 fp=#0x10000004070' 2024-09-10T08:44:46.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8653 fp=#0x10000003dda' 2024-09-10T08:44:46.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8685 fp=#0x10000003e0d' 2024-09-10T08:44:46.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8717 fp=#0x10000004f14' 2024-09-10T08:44:46.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8749 fp=#0x10000003ca9' 2024-09-10T08:44:46.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8781 fp=#0x10000003d1a' 2024-09-10T08:44:46.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8813 fp=#0x10000003e07' 2024-09-10T08:44:46.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8845 fp=#0x10000001d62' 2024-09-10T08:44:46.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8877 fp=#0x10000004427' 2024-09-10T08:44:46.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8909 fp=#0x10000004416' 2024-09-10T08:44:46.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8941 fp=#0x10000004331' 2024-09-10T08:44:46.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8973 fp=#0x10000004358' 2024-09-10T08:44:46.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9005 fp=#0x10000001d58' 2024-09-10T08:44:46.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9037 fp=#0x10000000db2' 2024-09-10T08:44:46.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9069 fp=#0x10000004214' 2024-09-10T08:44:46.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9101 fp=#0x100000042a8' 2024-09-10T08:44:46.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9133 fp=#0x10000000d4c' 2024-09-10T08:44:46.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9165 fp=#0x10000000d92' 2024-09-10T08:44:46.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9197 fp=#0x100000041a4' 2024-09-10T08:44:46.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9229 fp=#0x1000000414a' 2024-09-10T08:44:46.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9261 fp=#0x10000004aa8' 2024-09-10T08:44:46.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9293 fp=#0x10000000b5c' 2024-09-10T08:44:46.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9325 fp=#0x10000001c6b' 2024-09-10T08:44:46.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9357 fp=#0x10000000afc' 2024-09-10T08:44:46.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9389 fp=#0x10000000bcb' 2024-09-10T08:44:46.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9421 fp=#0x10000002618' 2024-09-10T08:44:46.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9453 fp=#0x10000000b2a' 2024-09-10T08:44:46.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9485 fp=#0x10000000ed2' 2024-09-10T08:44:46.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9517 fp=#0x100000019c8' 2024-09-10T08:44:46.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9549 fp=#0x10000001787' 2024-09-10T08:44:46.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9581 fp=#0x10000001aeb' 2024-09-10T08:44:46.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9613 fp=#0x10000001a04' 2024-09-10T08:44:46.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9645 fp=#0x10000001acc' 2024-09-10T08:44:46.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9677 fp=#0x10000004792' 2024-09-10T08:44:46.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9709 fp=#0x100000048e6' 2024-09-10T08:44:46.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9741 fp=#0x1000000447a' 2024-09-10T08:44:46.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9773 fp=#0x1000000439a' 2024-09-10T08:44:46.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9805 fp=#0x1000000187e' 2024-09-10T08:44:46.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9837 fp=#0x100000043d6' 2024-09-10T08:44:46.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9869 fp=#0x1000000447d' 2024-09-10T08:44:46.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9901 fp=#0x10000004348' 2024-09-10T08:44:46.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9933 fp=#0x10000001a97' 2024-09-10T08:44:46.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9965 fp=#0x10000001645' 2024-09-10T08:44:46.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9997 fp=#0x10000001a8c' 2024-09-10T08:44:46.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10029 fp=#0x100000017fd' 2024-09-10T08:44:46.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10061 fp=#0x10000001589' 2024-09-10T08:44:46.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10093 fp=#0x10000001f09' 2024-09-10T08:44:46.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10125 fp=#0x10000003aad' 2024-09-10T08:44:46.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10157 fp=#0x10000003a1c' 2024-09-10T08:44:46.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10189 fp=#0x10000001d9b' 2024-09-10T08:44:46.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10221 fp=#0x10000003918' 2024-09-10T08:44:46.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10253 fp=#0x100000039f9' 2024-09-10T08:44:46.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10285 fp=#0x100000024eb' 2024-09-10T08:44:46.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10317 fp=#0x10000002370' 2024-09-10T08:44:46.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10349 fp=#0x10000001551' 2024-09-10T08:44:46.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10381 fp=#0x100000039a0' 2024-09-10T08:44:46.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10413 fp=#0x100000038e2' 2024-09-10T08:44:46.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10445 fp=#0x1000000279e' 2024-09-10T08:44:46.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10477 fp=#0x10000003973' 2024-09-10T08:44:46.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10509 fp=#0x10000004760' 2024-09-10T08:44:46.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10541 fp=#0x1000000132e' 2024-09-10T08:44:46.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10573 fp=#0x10000001e0b' 2024-09-10T08:44:46.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10605 fp=#0x1000000143e' 2024-09-10T08:44:46.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10637 fp=#0x100000045d4' 2024-09-10T08:44:46.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10669 fp=#0x100000048c6' 2024-09-10T08:44:46.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10797 fp=#0x10000001d1f' 2024-09-10T08:44:46.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10829 fp=#0x10000003494' 2024-09-10T08:44:46.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10861 fp=#0x10000001012' 2024-09-10T08:44:46.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10957 fp=#0x10000000a58' 2024-09-10T08:44:46.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10989 fp=#0x10000001d2f' 2024-09-10T08:44:46.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11021 fp=#0x100000008e4' 2024-09-10T08:44:46.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11053 fp=#0x10000003b05' 2024-09-10T08:44:46.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11085 fp=#0x10000003ae8' 2024-09-10T08:44:46.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11117 fp=#0x1000000491a' 2024-09-10T08:44:46.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11149 fp=#0x10000003e69' 2024-09-10T08:44:46.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11181 fp=#0x1000000209b' 2024-09-10T08:44:46.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11213 fp=#0x1000000490c' 2024-09-10T08:44:46.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11245 fp=#0x100000020a5' 2024-09-10T08:44:46.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11277 fp=#0x10000002045' 2024-09-10T08:44:46.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11309 fp=#0x10000001d6d' 2024-09-10T08:44:46.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11341 fp=#0x10000003dc3' 2024-09-10T08:44:46.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11373 fp=#0x10000003ec5' 2024-09-10T08:44:46.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11405 fp=#0x100000019b8' 2024-09-10T08:44:46.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11437 fp=#0x10000003eba' 2024-09-10T08:44:46.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11469 fp=#0x10000001a71' 2024-09-10T08:44:46.402 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11501 fp=#0x10000003f1e' 2024-09-10T08:44:46.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11693 fp=#0x1000000512b' 2024-09-10T08:44:46.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11725 fp=#0x10000004e09' 2024-09-10T08:44:46.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11757 fp=#0x10000000987' 2024-09-10T08:44:46.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11821 fp=#0x10000004e44' 2024-09-10T08:44:46.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11853 fp=#0x10000004217' 2024-09-10T08:44:46.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11885 fp=#0x10000004e68' 2024-09-10T08:44:46.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11917 fp=#0x100000052ad' 2024-09-10T08:44:46.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12013 fp=#0x10000000f12' 2024-09-10T08:44:46.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12045 fp=#0x1000000528a' 2024-09-10T08:44:46.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12077 fp=#0x100000040a2' 2024-09-10T08:44:46.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12109 fp=#0x10000004384' 2024-09-10T08:44:46.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12141 fp=#0x10000004398' 2024-09-10T08:44:46.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12173 fp=#0x10000004486' 2024-09-10T08:44:46.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12205 fp=#0x1000000347e' 2024-09-10T08:44:46.410 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12237 fp=#0x100000051e9' 2024-09-10T08:44:46.410 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12269 fp=#0x10000000d28' 2024-09-10T08:44:46.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12301 fp=#0x10000000c1b' 2024-09-10T08:44:46.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12333 fp=#0x10000000d0f' 2024-09-10T08:44:46.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12365 fp=#0x10000000c6f' 2024-09-10T08:44:46.412 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12397 fp=#0x10000003ad3' 2024-09-10T08:44:46.412 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12429 fp=#0x10000000d0c' 2024-09-10T08:44:46.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12461 fp=#0x10000004c2b' 2024-09-10T08:44:46.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12493 fp=#0x10000004c32' 2024-09-10T08:44:46.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12557 fp=#0x10000004ac9' 2024-09-10T08:44:46.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12589 fp=#0x10000004d2e' 2024-09-10T08:44:46.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12621 fp=#0x10000004bfe' 2024-09-10T08:44:46.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12653 fp=#0x100000049eb' 2024-09-10T08:44:46.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12685 fp=#0x10000004f6c' 2024-09-10T08:44:46.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12717 fp=#0x10000004d01' 2024-09-10T08:44:46.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12749 fp=#0x100000048af' 2024-09-10T08:44:46.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12781 fp=#0x10000004512' 2024-09-10T08:44:46.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12845 fp=#0x10000002250' 2024-09-10T08:44:46.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12877 fp=#0x10000005319' 2024-09-10T08:44:46.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12909 fp=#0x1000000531f' 2024-09-10T08:44:46.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12941 fp=#0x10000002112' 2024-09-10T08:44:46.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12973 fp=#0x10000002761' 2024-09-10T08:44:46.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13005 fp=#0x10000002704' 2024-09-10T08:44:46.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13029 fp=#0x100000022f2' 2024-09-10T08:44:46.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13061 fp=#0x10000002280' 2024-09-10T08:44:46.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13093 fp=#0x1000000221f' 2024-09-10T08:44:46.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13157 fp=#0x100000022ca' 2024-09-10T08:44:46.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:165 fp=#0x10000000201' 2024-09-10T08:44:46.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:246 fp=#0x10000000d7d' 2024-09-10T08:44:46.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:278 fp=#0x10000001266' 2024-09-10T08:44:46.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:310 fp=#0x10000000ca6' 2024-09-10T08:44:46.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:398 fp=#0x10000004b54' 2024-09-10T08:44:46.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:430 fp=#0x10000004b2b' 2024-09-10T08:44:46.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:462 fp=#0x10000004af3' 2024-09-10T08:44:46.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:526 fp=#0x10000003357' 2024-09-10T08:44:46.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:782 fp=#0x10000000f25' 2024-09-10T08:44:46.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:814 fp=#0x10000000d82' 2024-09-10T08:44:46.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:846 fp=#0x10000000f1c' 2024-09-10T08:44:46.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:910 fp=#0x10000004076' 2024-09-10T08:44:46.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:942 fp=#0x10000000e7f' 2024-09-10T08:44:46.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:974 fp=#0x1000000451e' 2024-09-10T08:44:46.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1006 fp=#0x1000000456d' 2024-09-10T08:44:46.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1070 fp=#0x1000000428d' 2024-09-10T08:44:46.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1102 fp=#0x100000041b3' 2024-09-10T08:44:46.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1230 fp=#0x10000002088' 2024-09-10T08:44:46.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1262 fp=#0x100000035da' 2024-09-10T08:44:46.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1326 fp=#0x10000001f5d' 2024-09-10T08:44:46.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1358 fp=#0x1000000355e' 2024-09-10T08:44:46.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1390 fp=#0x100000032ca' 2024-09-10T08:44:46.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1646 fp=#0x1000000460b' 2024-09-10T08:44:46.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1678 fp=#0x10000004a37' 2024-09-10T08:44:46.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2062 fp=#0x100000021da' 2024-09-10T08:44:46.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2158 fp=#0x10000001fdf' 2024-09-10T08:44:46.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2286 fp=#0x100000034f1' 2024-09-10T08:44:46.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2318 fp=#0x100000025ff' 2024-09-10T08:44:46.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2574 fp=#0x10000004aab' 2024-09-10T08:44:46.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2606 fp=#0x10000001148' 2024-09-10T08:44:46.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2638 fp=#0x10000001076' 2024-09-10T08:44:46.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2670 fp=#0x100000018f9' 2024-09-10T08:44:46.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2766 fp=#0x10000003435' 2024-09-10T08:44:46.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2798 fp=#0x100000046c4' 2024-09-10T08:44:46.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2830 fp=#0x10000003429' 2024-09-10T08:44:46.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2862 fp=#0x10000004842' 2024-09-10T08:44:46.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2894 fp=#0x10000003425' 2024-09-10T08:44:46.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2958 fp=#0x1000000480a' 2024-09-10T08:44:46.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2990 fp=#0x10000000e78' 2024-09-10T08:44:46.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3022 fp=#0x10000001981' 2024-09-10T08:44:46.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3054 fp=#0x10000000dfa' 2024-09-10T08:44:46.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3118 fp=#0x10000001962' 2024-09-10T08:44:46.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3150 fp=#0x10000000dd4' 2024-09-10T08:44:46.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3182 fp=#0x10000004a22' 2024-09-10T08:44:46.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3214 fp=#0x1000000074c' 2024-09-10T08:44:46.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3246 fp=#0x10000000722' 2024-09-10T08:44:46.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3278 fp=#0x10000000781' 2024-09-10T08:44:46.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3310 fp=#0x10000003bad' 2024-09-10T08:44:46.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3342 fp=#0x10000000706' 2024-09-10T08:44:46.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3374 fp=#0x100000007a6' 2024-09-10T08:44:46.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3406 fp=#0x10000000724' 2024-09-10T08:44:46.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3438 fp=#0x10000000792' 2024-09-10T08:44:46.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3470 fp=#0x10000003c98' 2024-09-10T08:44:46.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3502 fp=#0x10000004071' 2024-09-10T08:44:46.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3534 fp=#0x10000004034' 2024-09-10T08:44:46.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3630 fp=#0x10000004c5c' 2024-09-10T08:44:46.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3662 fp=#0x10000004ae0' 2024-09-10T08:44:46.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3694 fp=#0x10000004bac' 2024-09-10T08:44:46.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3758 fp=#0x1000000279c' 2024-09-10T08:44:46.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3790 fp=#0x10000002721' 2024-09-10T08:44:46.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3822 fp=#0x100000041b2' 2024-09-10T08:44:46.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3854 fp=#0x10000002769' 2024-09-10T08:44:46.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3886 fp=#0x10000004317' 2024-09-10T08:44:46.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3918 fp=#0x1000000051c' 2024-09-10T08:44:46.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3950 fp=#0x10000000689' 2024-09-10T08:44:46.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3982 fp=#0x10000001ddc' 2024-09-10T08:44:46.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4014 fp=#0x1000000069f' 2024-09-10T08:44:46.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4046 fp=#0x10000001002' 2024-09-10T08:44:46.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4078 fp=#0x10000004ddf' 2024-09-10T08:44:46.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4110 fp=#0x10000004dbb' 2024-09-10T08:44:46.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4142 fp=#0x10000001115' 2024-09-10T08:44:46.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4174 fp=#0x10000004d95' 2024-09-10T08:44:46.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4238 fp=#0x10000003c22' 2024-09-10T08:44:46.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4270 fp=#0x10000003a9b' 2024-09-10T08:44:46.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4302 fp=#0x100000041a6' 2024-09-10T08:44:46.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4334 fp=#0x100000041fe' 2024-09-10T08:44:46.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4366 fp=#0x10000004d1e' 2024-09-10T08:44:46.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4398 fp=#0x1000000381c' 2024-09-10T08:44:46.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4430 fp=#0x10000001ef0' 2024-09-10T08:44:46.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4462 fp=#0x10000001f30' 2024-09-10T08:44:46.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4494 fp=#0x10000003b55' 2024-09-10T08:44:46.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4526 fp=#0x10000005138' 2024-09-10T08:44:46.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4558 fp=#0x100000050cd' 2024-09-10T08:44:46.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4590 fp=#0x100000014e7' 2024-09-10T08:44:46.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4622 fp=#0x100000038e0' 2024-09-10T08:44:46.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4654 fp=#0x1000000131c' 2024-09-10T08:44:46.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4686 fp=#0x10000003950' 2024-09-10T08:44:46.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4718 fp=#0x10000004def' 2024-09-10T08:44:46.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4750 fp=#0x10000001401' 2024-09-10T08:44:46.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4782 fp=#0x10000001339' 2024-09-10T08:44:46.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4814 fp=#0x10000003826' 2024-09-10T08:44:46.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4846 fp=#0x100000037e3' 2024-09-10T08:44:46.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4878 fp=#0x10000004ab2' 2024-09-10T08:44:46.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4910 fp=#0x10000003779' 2024-09-10T08:44:46.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4942 fp=#0x10000003739' 2024-09-10T08:44:46.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4974 fp=#0x10000005179' 2024-09-10T08:44:46.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5006 fp=#0x1000000537a' 2024-09-10T08:44:46.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5038 fp=#0x10000005362' 2024-09-10T08:44:46.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5070 fp=#0x100000051aa' 2024-09-10T08:44:46.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5102 fp=#0x1000000509e' 2024-09-10T08:44:46.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5134 fp=#0x10000001eb1' 2024-09-10T08:44:46.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5166 fp=#0x10000001b7c' 2024-09-10T08:44:46.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5198 fp=#0x10000001d5c' 2024-09-10T08:44:46.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5230 fp=#0x10000001c3f' 2024-09-10T08:44:46.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5262 fp=#0x1000000484e' 2024-09-10T08:44:46.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5294 fp=#0x100000037c4' 2024-09-10T08:44:46.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5326 fp=#0x10000004a89' 2024-09-10T08:44:46.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5358 fp=#0x100000035a4' 2024-09-10T08:44:46.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5390 fp=#0x100000036b8' 2024-09-10T08:44:46.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5422 fp=#0x100000036f4' 2024-09-10T08:44:46.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5454 fp=#0x100000012aa' 2024-09-10T08:44:46.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5486 fp=#0x100000014e0' 2024-09-10T08:44:46.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5582 fp=#0x10000001100' 2024-09-10T08:44:46.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5614 fp=#0x10000000f41' 2024-09-10T08:44:46.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5646 fp=#0x10000005118' 2024-09-10T08:44:46.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5678 fp=#0x10000005102' 2024-09-10T08:44:46.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5710 fp=#0x10000000fc8' 2024-09-10T08:44:46.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5742 fp=#0x10000001001' 2024-09-10T08:44:46.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5774 fp=#0x100000020df' 2024-09-10T08:44:46.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5806 fp=#0x100000044ee' 2024-09-10T08:44:46.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5838 fp=#0x10000004c5f' 2024-09-10T08:44:46.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5870 fp=#0x10000004ce3' 2024-09-10T08:44:46.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5902 fp=#0x1000000439c' 2024-09-10T08:44:46.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5934 fp=#0x10000004cc7' 2024-09-10T08:44:46.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5966 fp=#0x1000000440d' 2024-09-10T08:44:46.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6062 fp=#0x10000003503' 2024-09-10T08:44:46.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6126 fp=#0x10000003498' 2024-09-10T08:44:46.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6158 fp=#0x10000003562' 2024-09-10T08:44:46.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6254 fp=#0x10000003626' 2024-09-10T08:44:46.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6286 fp=#0x100000033ce' 2024-09-10T08:44:46.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6318 fp=#0x1000000339a' 2024-09-10T08:44:46.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6350 fp=#0x10000001ee3' 2024-09-10T08:44:46.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6382 fp=#0x10000004b83' 2024-09-10T08:44:46.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6414 fp=#0x10000004fba' 2024-09-10T08:44:46.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6446 fp=#0x10000004ef6' 2024-09-10T08:44:46.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6478 fp=#0x10000004cb2' 2024-09-10T08:44:46.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6510 fp=#0x10000004c24' 2024-09-10T08:44:46.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6542 fp=#0x10000004ecf' 2024-09-10T08:44:46.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6574 fp=#0x10000004108' 2024-09-10T08:44:46.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6606 fp=#0x10000004a55' 2024-09-10T08:44:46.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6638 fp=#0x1000000393c' 2024-09-10T08:44:46.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6670 fp=#0x10000003b18' 2024-09-10T08:44:46.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6702 fp=#0x100000039c4' 2024-09-10T08:44:46.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6734 fp=#0x10000003a87' 2024-09-10T08:44:46.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6766 fp=#0x10000003c5b' 2024-09-10T08:44:46.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6798 fp=#0x100000045b3' 2024-09-10T08:44:46.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6830 fp=#0x10000004776' 2024-09-10T08:44:46.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6862 fp=#0x100000052f0' 2024-09-10T08:44:46.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6894 fp=#0x100000046d3' 2024-09-10T08:44:46.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6926 fp=#0x10000004544' 2024-09-10T08:44:46.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6958 fp=#0x100000006ac' 2024-09-10T08:44:46.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6990 fp=#0x1000000060b' 2024-09-10T08:44:46.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7022 fp=#0x10000004fa9' 2024-09-10T08:44:46.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7054 fp=#0x10000005003' 2024-09-10T08:44:46.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7086 fp=#0x10000004f5b' 2024-09-10T08:44:46.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7118 fp=#0x10000000560' 2024-09-10T08:44:46.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7150 fp=#0x100000006a8' 2024-09-10T08:44:46.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7182 fp=#0x100000010a4' 2024-09-10T08:44:46.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7246 fp=#0x1000000507c' 2024-09-10T08:44:46.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7278 fp=#0x1000000513d' 2024-09-10T08:44:46.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7406 fp=#0x100000048ff' 2024-09-10T08:44:46.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7438 fp=#0x10000002348' 2024-09-10T08:44:46.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7470 fp=#0x1000000491d' 2024-09-10T08:44:46.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7502 fp=#0x1000000236e' 2024-09-10T08:44:46.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7534 fp=#0x10000002256' 2024-09-10T08:44:46.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7566 fp=#0x100000051a3' 2024-09-10T08:44:46.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7598 fp=#0x10000001368' 2024-09-10T08:44:46.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7630 fp=#0x1000000137e' 2024-09-10T08:44:46.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7662 fp=#0x10000001326' 2024-09-10T08:44:46.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7694 fp=#0x1000000134d' 2024-09-10T08:44:46.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7726 fp=#0x10000004de8' 2024-09-10T08:44:46.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7758 fp=#0x10000004f49' 2024-09-10T08:44:46.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7790 fp=#0x100000016f6' 2024-09-10T08:44:46.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7822 fp=#0x10000004cb6' 2024-09-10T08:44:46.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7854 fp=#0x10000004ca8' 2024-09-10T08:44:46.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7886 fp=#0x10000001408' 2024-09-10T08:44:46.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7918 fp=#0x10000004ddd' 2024-09-10T08:44:46.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7950 fp=#0x10000004dad' 2024-09-10T08:44:46.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7982 fp=#0x10000004e7a' 2024-09-10T08:44:46.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8014 fp=#0x100000013dd' 2024-09-10T08:44:46.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8046 fp=#0x10000001719' 2024-09-10T08:44:46.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8078 fp=#0x10000003866' 2024-09-10T08:44:46.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8110 fp=#0x1000000163b' 2024-09-10T08:44:46.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8142 fp=#0x10000002736' 2024-09-10T08:44:46.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8174 fp=#0x10000002507' 2024-09-10T08:44:46.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8206 fp=#0x10000003dd9' 2024-09-10T08:44:46.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8238 fp=#0x10000003dd3' 2024-09-10T08:44:46.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8270 fp=#0x100000040f0' 2024-09-10T08:44:46.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8302 fp=#0x10000005252' 2024-09-10T08:44:46.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8334 fp=#0x1000000524c' 2024-09-10T08:44:46.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8366 fp=#0x100000043af' 2024-09-10T08:44:46.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8398 fp=#0x100000042d3' 2024-09-10T08:44:46.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8430 fp=#0x10000005262' 2024-09-10T08:44:46.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8462 fp=#0x10000003f05' 2024-09-10T08:44:46.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8494 fp=#0x10000003f62' 2024-09-10T08:44:46.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8526 fp=#0x10000004179' 2024-09-10T08:44:46.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8558 fp=#0x10000004222' 2024-09-10T08:44:46.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8590 fp=#0x10000003f27' 2024-09-10T08:44:46.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8622 fp=#0x10000004f52' 2024-09-10T08:44:46.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8654 fp=#0x10000003cd6' 2024-09-10T08:44:46.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8686 fp=#0x10000003be5' 2024-09-10T08:44:46.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8718 fp=#0x10000005314' 2024-09-10T08:44:46.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8750 fp=#0x1000000530f' 2024-09-10T08:44:46.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8782 fp=#0x10000004ee8' 2024-09-10T08:44:46.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8814 fp=#0x100000052a8' 2024-09-10T08:44:46.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8846 fp=#0x10000003571' 2024-09-10T08:44:46.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8878 fp=#0x10000001aba' 2024-09-10T08:44:46.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8910 fp=#0x10000001c01' 2024-09-10T08:44:46.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8942 fp=#0x10000001ae1' 2024-09-10T08:44:46.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9006 fp=#0x10000003f35' 2024-09-10T08:44:46.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9038 fp=#0x1000000425a' 2024-09-10T08:44:46.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9070 fp=#0x1000000420d' 2024-09-10T08:44:46.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9102 fp=#0x10000004182' 2024-09-10T08:44:46.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9166 fp=#0x100000024f6' 2024-09-10T08:44:46.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9198 fp=#0x100000041d2' 2024-09-10T08:44:46.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9262 fp=#0x10000000ad5' 2024-09-10T08:44:46.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9294 fp=#0x10000003e8f' 2024-09-10T08:44:46.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9326 fp=#0x10000004e4e' 2024-09-10T08:44:46.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9358 fp=#0x10000000b6b' 2024-09-10T08:44:46.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9390 fp=#0x10000000b51' 2024-09-10T08:44:46.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9422 fp=#0x10000001c3d' 2024-09-10T08:44:46.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9454 fp=#0x10000000b53' 2024-09-10T08:44:46.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9486 fp=#0x100000019e9' 2024-09-10T08:44:46.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9518 fp=#0x10000000ede' 2024-09-10T08:44:46.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9550 fp=#0x10000000ec5' 2024-09-10T08:44:46.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9582 fp=#0x10000000ed0' 2024-09-10T08:44:46.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9614 fp=#0x10000000ed7' 2024-09-10T08:44:46.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9646 fp=#0x10000000ecf' 2024-09-10T08:44:46.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9678 fp=#0x100000047dc' 2024-09-10T08:44:46.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9710 fp=#0x10000004828' 2024-09-10T08:44:46.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9742 fp=#0x100000044a2' 2024-09-10T08:44:46.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9774 fp=#0x10000004334' 2024-09-10T08:44:46.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9806 fp=#0x1000000443d' 2024-09-10T08:44:46.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9838 fp=#0x10000001811' 2024-09-10T08:44:46.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9902 fp=#0x10000004448' 2024-09-10T08:44:46.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9934 fp=#0x1000000266e' 2024-09-10T08:44:46.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9998 fp=#0x100000026d0' 2024-09-10T08:44:46.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10030 fp=#0x100000018c7' 2024-09-10T08:44:46.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10062 fp=#0x1000000161f' 2024-09-10T08:44:46.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10094 fp=#0x100000038b8' 2024-09-10T08:44:46.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10126 fp=#0x10000004309' 2024-09-10T08:44:46.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10158 fp=#0x10000004031' 2024-09-10T08:44:46.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10190 fp=#0x1000000398f' 2024-09-10T08:44:46.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10222 fp=#0x10000003b97' 2024-09-10T08:44:46.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10254 fp=#0x100000038dd' 2024-09-10T08:44:46.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10286 fp=#0x100000023da' 2024-09-10T08:44:46.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10318 fp=#0x100000024a4' 2024-09-10T08:44:46.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10350 fp=#0x1000000131e' 2024-09-10T08:44:46.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10382 fp=#0x1000000139f' 2024-09-10T08:44:46.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10414 fp=#0x1000000393a' 2024-09-10T08:44:46.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10446 fp=#0x100000039b4' 2024-09-10T08:44:46.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10478 fp=#0x100000013a7' 2024-09-10T08:44:46.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10510 fp=#0x10000001376' 2024-09-10T08:44:46.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10542 fp=#0x100000012f0' 2024-09-10T08:44:46.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10574 fp=#0x10000004a18' 2024-09-10T08:44:46.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10606 fp=#0x100000048aa' 2024-09-10T08:44:46.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10670 fp=#0x10000004875' 2024-09-10T08:44:46.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10734 fp=#0x100000048d2' 2024-09-10T08:44:46.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10798 fp=#0x10000000a48' 2024-09-10T08:44:46.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10830 fp=#0x1000000058e' 2024-09-10T08:44:46.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10862 fp=#0x10000000536' 2024-09-10T08:44:46.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10894 fp=#0x10000000fac' 2024-09-10T08:44:46.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10926 fp=#0x10000001150' 2024-09-10T08:44:46.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10958 fp=#0x100000008f3' 2024-09-10T08:44:46.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10990 fp=#0x10000000596' 2024-09-10T08:44:46.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11022 fp=#0x100000035c9' 2024-09-10T08:44:46.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11054 fp=#0x100000039d2' 2024-09-10T08:44:46.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11086 fp=#0x10000004a92' 2024-09-10T08:44:46.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11118 fp=#0x100000022ec' 2024-09-10T08:44:46.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11150 fp=#0x10000002146' 2024-09-10T08:44:46.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11182 fp=#0x100000021ac' 2024-09-10T08:44:46.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11214 fp=#0x10000002364' 2024-09-10T08:44:46.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11246 fp=#0x10000003ea1' 2024-09-10T08:44:46.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11278 fp=#0x100000020da' 2024-09-10T08:44:46.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11310 fp=#0x10000004982' 2024-09-10T08:44:46.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11342 fp=#0x1000000455c' 2024-09-10T08:44:46.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11374 fp=#0x10000001ac8' 2024-09-10T08:44:46.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11406 fp=#0x10000003e9d' 2024-09-10T08:44:46.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11438 fp=#0x10000003def' 2024-09-10T08:44:46.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11470 fp=#0x10000001b13' 2024-09-10T08:44:46.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11502 fp=#0x10000003f04' 2024-09-10T08:44:46.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11534 fp=#0x1000000516a' 2024-09-10T08:44:46.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11694 fp=#0x1000000090d' 2024-09-10T08:44:46.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11726 fp=#0x100000009ed' 2024-09-10T08:44:46.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11758 fp=#0x100000015b3' 2024-09-10T08:44:46.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11790 fp=#0x10000004e0f' 2024-09-10T08:44:46.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11822 fp=#0x1000000418e' 2024-09-10T08:44:46.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11854 fp=#0x100000026b2' 2024-09-10T08:44:46.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11886 fp=#0x10000004dfa' 2024-09-10T08:44:46.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11918 fp=#0x10000000f13' 2024-09-10T08:44:46.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11950 fp=#0x100000040b4' 2024-09-10T08:44:46.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11982 fp=#0x100000010dd' 2024-09-10T08:44:46.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12046 fp=#0x10000000ef7' 2024-09-10T08:44:46.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12078 fp=#0x1000000458d' 2024-09-10T08:44:46.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12110 fp=#0x100000043f6' 2024-09-10T08:44:46.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12142 fp=#0x10000004594' 2024-09-10T08:44:46.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12174 fp=#0x100000033eb' 2024-09-10T08:44:46.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12206 fp=#0x10000004586' 2024-09-10T08:44:46.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12238 fp=#0x10000000ce9' 2024-09-10T08:44:46.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12270 fp=#0x10000000cdf' 2024-09-10T08:44:46.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12302 fp=#0x100000051ef' 2024-09-10T08:44:46.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12334 fp=#0x10000003b79' 2024-09-10T08:44:46.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12366 fp=#0x10000000bf5' 2024-09-10T08:44:46.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12398 fp=#0x10000000c8f' 2024-09-10T08:44:46.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12430 fp=#0x10000000c1e' 2024-09-10T08:44:46.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12462 fp=#0x1000000482e' 2024-09-10T08:44:46.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12494 fp=#0x10000004c20' 2024-09-10T08:44:46.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12558 fp=#0x10000004c2f' 2024-09-10T08:44:46.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12590 fp=#0x10000004bd9' 2024-09-10T08:44:46.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12622 fp=#0x10000004cbb' 2024-09-10T08:44:46.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12654 fp=#0x100000048fe' 2024-09-10T08:44:46.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12686 fp=#0x100000049ce' 2024-09-10T08:44:46.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12718 fp=#0x1000000480e' 2024-09-10T08:44:46.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12750 fp=#0x10000004971' 2024-09-10T08:44:46.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12782 fp=#0x100000053e5' 2024-09-10T08:44:46.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12814 fp=#0x10000002161' 2024-09-10T08:44:46.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12846 fp=#0x10000005414' 2024-09-10T08:44:46.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12878 fp=#0x100000020f5' 2024-09-10T08:44:46.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12910 fp=#0x100000052ca' 2024-09-10T08:44:46.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12942 fp=#0x10000002210' 2024-09-10T08:44:46.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12974 fp=#0x10000005406' 2024-09-10T08:44:46.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13006 fp=#0x10000003b42' 2024-09-10T08:44:46.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13030 fp=#0x10000003966' 2024-09-10T08:44:46.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13062 fp=#0x10000002294' 2024-09-10T08:44:46.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13094 fp=#0x100000053cd' 2024-09-10T08:44:46.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13126 fp=#0x10000002305' 2024-09-10T08:44:46.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13158 fp=#0x10000002245' 2024-09-10T08:44:46.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:279 fp=#0x100000026f7' 2024-09-10T08:44:46.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:399 fp=#0x10000004b5c' 2024-09-10T08:44:46.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:463 fp=#0x10000004b26' 2024-09-10T08:44:46.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:623 fp=#0x1000000336a' 2024-09-10T08:44:46.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:783 fp=#0x10000000e93' 2024-09-10T08:44:46.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:815 fp=#0x1000000446c' 2024-09-10T08:44:46.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:879 fp=#0x10000001baf' 2024-09-10T08:44:46.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:911 fp=#0x100000045a5' 2024-09-10T08:44:46.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:943 fp=#0x10000000fd0' 2024-09-10T08:44:46.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1007 fp=#0x10000004546' 2024-09-10T08:44:46.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1071 fp=#0x100000042ae' 2024-09-10T08:44:46.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1167 fp=#0x1000000416d' 2024-09-10T08:44:46.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1263 fp=#0x100000022ab' 2024-09-10T08:44:46.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1295 fp=#0x100000020a8' 2024-09-10T08:44:46.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1327 fp=#0x10000003696' 2024-09-10T08:44:46.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1359 fp=#0x10000003659' 2024-09-10T08:44:46.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1391 fp=#0x100000042bd' 2024-09-10T08:44:46.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1647 fp=#0x10000004642' 2024-09-10T08:44:46.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1679 fp=#0x10000004583' 2024-09-10T08:44:46.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2031 fp=#0x10000002759' 2024-09-10T08:44:46.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2159 fp=#0x10000002182' 2024-09-10T08:44:46.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2223 fp=#0x100000025cc' 2024-09-10T08:44:46.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2255 fp=#0x100000026bd' 2024-09-10T08:44:46.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2287 fp=#0x10000002656' 2024-09-10T08:44:46.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2319 fp=#0x10000002677' 2024-09-10T08:44:46.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2607 fp=#0x100000010fe' 2024-09-10T08:44:46.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2639 fp=#0x1000000108e' 2024-09-10T08:44:46.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2671 fp=#0x10000001069' 2024-09-10T08:44:46.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2703 fp=#0x1000000107d' 2024-09-10T08:44:46.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2735 fp=#0x100000018b4' 2024-09-10T08:44:46.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2767 fp=#0x10000004853' 2024-09-10T08:44:46.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2799 fp=#0x100000046d7' 2024-09-10T08:44:46.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2831 fp=#0x10000003400' 2024-09-10T08:44:46.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2863 fp=#0x100000047fc' 2024-09-10T08:44:46.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2895 fp=#0x1000000475a' 2024-09-10T08:44:46.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2959 fp=#0x100000047d0' 2024-09-10T08:44:46.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2991 fp=#0x10000000e00' 2024-09-10T08:44:46.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3023 fp=#0x10000000df4' 2024-09-10T08:44:46.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3055 fp=#0x10000000de8' 2024-09-10T08:44:46.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3087 fp=#0x10000004a5d' 2024-09-10T08:44:46.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3119 fp=#0x10000000df8' 2024-09-10T08:44:46.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3151 fp=#0x100000049f4' 2024-09-10T08:44:46.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3183 fp=#0x10000004a0b' 2024-09-10T08:44:46.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3215 fp=#0x10000003d7a' 2024-09-10T08:44:46.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3247 fp=#0x100000007dd' 2024-09-10T08:44:46.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3279 fp=#0x100000006ef' 2024-09-10T08:44:46.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3311 fp=#0x10000000771' 2024-09-10T08:44:46.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3343 fp=#0x10000000773' 2024-09-10T08:44:46.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3375 fp=#0x10000000786' 2024-09-10T08:44:46.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3407 fp=#0x1000000078c' 2024-09-10T08:44:46.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3439 fp=#0x10000003bbe' 2024-09-10T08:44:46.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3471 fp=#0x10000003cae' 2024-09-10T08:44:46.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3599 fp=#0x1000000408d' 2024-09-10T08:44:46.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3631 fp=#0x10000004c55' 2024-09-10T08:44:46.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3663 fp=#0x10000004af4' 2024-09-10T08:44:46.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3695 fp=#0x10000004acd' 2024-09-10T08:44:46.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3727 fp=#0x10000004ae3' 2024-09-10T08:44:46.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3823 fp=#0x1000000265c' 2024-09-10T08:44:46.625 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3855 fp=#0x10000003ee7' 2024-09-10T08:44:46.625 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3887 fp=#0x10000004ccc' 2024-09-10T08:44:46.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3919 fp=#0x100000027a2' 2024-09-10T08:44:46.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3951 fp=#0x100000005a9' 2024-09-10T08:44:46.627 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3983 fp=#0x10000000699' 2024-09-10T08:44:46.627 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4015 fp=#0x100000010fa' 2024-09-10T08:44:46.628 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4079 fp=#0x100000044e6' 2024-09-10T08:44:46.628 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4111 fp=#0x10000000f73' 2024-09-10T08:44:46.628 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4143 fp=#0x10000000f50' 2024-09-10T08:44:46.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4175 fp=#0x10000000fce' 2024-09-10T08:44:46.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4207 fp=#0x10000000fe2' 2024-09-10T08:44:46.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4239 fp=#0x10000003a93' 2024-09-10T08:44:46.630 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4271 fp=#0x1000000409f' 2024-09-10T08:44:46.630 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4303 fp=#0x10000003bce' 2024-09-10T08:44:46.630 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4335 fp=#0x10000003a4f' 2024-09-10T08:44:46.631 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4367 fp=#0x10000001fd2' 2024-09-10T08:44:46.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4495 fp=#0x1000000401a' 2024-09-10T08:44:46.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4527 fp=#0x10000005117' 2024-09-10T08:44:46.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4559 fp=#0x100000050ac' 2024-09-10T08:44:46.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4591 fp=#0x100000015c8' 2024-09-10T08:44:46.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4623 fp=#0x1000000391f' 2024-09-10T08:44:46.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4655 fp=#0x10000001300' 2024-09-10T08:44:46.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4687 fp=#0x10000001305' 2024-09-10T08:44:46.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4719 fp=#0x10000001333' 2024-09-10T08:44:46.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4751 fp=#0x1000000155f' 2024-09-10T08:44:46.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4783 fp=#0x10000003351' 2024-09-10T08:44:46.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4815 fp=#0x10000003773' 2024-09-10T08:44:46.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4847 fp=#0x100000037a6' 2024-09-10T08:44:46.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4879 fp=#0x1000000378a' 2024-09-10T08:44:46.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4911 fp=#0x10000004a5e' 2024-09-10T08:44:46.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4943 fp=#0x10000004a30' 2024-09-10T08:44:46.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4975 fp=#0x10000004ff2' 2024-09-10T08:44:46.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5007 fp=#0x100000051da' 2024-09-10T08:44:46.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5039 fp=#0x10000005158' 2024-09-10T08:44:46.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5071 fp=#0x10000005103' 2024-09-10T08:44:46.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5103 fp=#0x100000050aa' 2024-09-10T08:44:46.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5135 fp=#0x10000001bee' 2024-09-10T08:44:46.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5167 fp=#0x10000001c91' 2024-09-10T08:44:46.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5199 fp=#0x10000001d70' 2024-09-10T08:44:46.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5231 fp=#0x10000001e6d' 2024-09-10T08:44:46.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5263 fp=#0x100000035df' 2024-09-10T08:44:46.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5295 fp=#0x100000038a0' 2024-09-10T08:44:46.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5327 fp=#0x10000003621' 2024-09-10T08:44:46.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5359 fp=#0x100000035b4' 2024-09-10T08:44:46.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5391 fp=#0x1000000367d' 2024-09-10T08:44:46.644 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5423 fp=#0x1000000372a' 2024-09-10T08:44:46.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5519 fp=#0x100000017da' 2024-09-10T08:44:46.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5551 fp=#0x10000001ef3' 2024-09-10T08:44:46.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5583 fp=#0x10000000f39' 2024-09-10T08:44:46.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5615 fp=#0x100000020aa' 2024-09-10T08:44:46.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5647 fp=#0x1000000142e' 2024-09-10T08:44:46.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5679 fp=#0x100000010fd' 2024-09-10T08:44:46.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5711 fp=#0x100000020a4' 2024-09-10T08:44:46.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5743 fp=#0x10000000fbf' 2024-09-10T08:44:46.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5775 fp=#0x10000000fb4' 2024-09-10T08:44:46.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5807 fp=#0x10000004d56' 2024-09-10T08:44:46.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5839 fp=#0x10000004368' 2024-09-10T08:44:46.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5871 fp=#0x10000004964' 2024-09-10T08:44:46.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5903 fp=#0x10000004ba1' 2024-09-10T08:44:46.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5935 fp=#0x1000000434e' 2024-09-10T08:44:46.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5967 fp=#0x10000004419' 2024-09-10T08:44:46.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6095 fp=#0x1000000358b' 2024-09-10T08:44:46.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6127 fp=#0x10000003584' 2024-09-10T08:44:46.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6159 fp=#0x100000035e6' 2024-09-10T08:44:46.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6255 fp=#0x100000035e1' 2024-09-10T08:44:46.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6287 fp=#0x10000001eff' 2024-09-10T08:44:46.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6319 fp=#0x10000003434' 2024-09-10T08:44:46.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6351 fp=#0x10000001e28' 2024-09-10T08:44:46.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6383 fp=#0x10000001e58' 2024-09-10T08:44:46.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6415 fp=#0x10000004bc2' 2024-09-10T08:44:46.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6447 fp=#0x10000004e23' 2024-09-10T08:44:46.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6479 fp=#0x10000004be7' 2024-09-10T08:44:46.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6511 fp=#0x10000004d9a' 2024-09-10T08:44:46.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6543 fp=#0x10000004b00' 2024-09-10T08:44:46.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6639 fp=#0x10000003d60' 2024-09-10T08:44:46.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6671 fp=#0x10000004f55' 2024-09-10T08:44:46.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6703 fp=#0x10000004f72' 2024-09-10T08:44:46.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6735 fp=#0x10000003d1f' 2024-09-10T08:44:46.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6767 fp=#0x10000003923' 2024-09-10T08:44:46.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6799 fp=#0x100000046f6' 2024-09-10T08:44:46.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6831 fp=#0x10000004795' 2024-09-10T08:44:46.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6863 fp=#0x10000004602' 2024-09-10T08:44:46.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6895 fp=#0x10000004537' 2024-09-10T08:44:46.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6927 fp=#0x100000052e0' 2024-09-10T08:44:46.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6959 fp=#0x100000005ab' 2024-09-10T08:44:46.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6991 fp=#0x10000000830' 2024-09-10T08:44:46.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7023 fp=#0x10000000692' 2024-09-10T08:44:46.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7055 fp=#0x10000000673' 2024-09-10T08:44:46.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7087 fp=#0x10000000590' 2024-09-10T08:44:46.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7119 fp=#0x1000000066a' 2024-09-10T08:44:46.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7151 fp=#0x10000000570' 2024-09-10T08:44:46.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7183 fp=#0x1000000063c' 2024-09-10T08:44:46.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7215 fp=#0x100000050ad' 2024-09-10T08:44:46.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7311 fp=#0x100000050ab' 2024-09-10T08:44:46.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7343 fp=#0x100000050c6' 2024-09-10T08:44:46.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7375 fp=#0x100000050ca' 2024-09-10T08:44:46.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7407 fp=#0x100000048bc' 2024-09-10T08:44:46.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7439 fp=#0x10000004691' 2024-09-10T08:44:46.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7471 fp=#0x100000046ac' 2024-09-10T08:44:46.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7503 fp=#0x100000046d2' 2024-09-10T08:44:46.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7535 fp=#0x1000000136d' 2024-09-10T08:44:46.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7599 fp=#0x100000051c3' 2024-09-10T08:44:46.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7631 fp=#0x10000001361' 2024-09-10T08:44:46.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7663 fp=#0x100000012fb' 2024-09-10T08:44:46.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7695 fp=#0x1000000136b' 2024-09-10T08:44:46.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7727 fp=#0x10000004f42' 2024-09-10T08:44:46.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7759 fp=#0x10000004ec5' 2024-09-10T08:44:46.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7791 fp=#0x10000004dc4' 2024-09-10T08:44:46.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7823 fp=#0x1000000173b' 2024-09-10T08:44:46.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7855 fp=#0x10000001511' 2024-09-10T08:44:46.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7887 fp=#0x10000004eb9' 2024-09-10T08:44:46.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7919 fp=#0x10000004f18' 2024-09-10T08:44:46.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7983 fp=#0x10000004e54' 2024-09-10T08:44:46.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8015 fp=#0x100000016ea' 2024-09-10T08:44:46.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8047 fp=#0x10000003862' 2024-09-10T08:44:46.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8079 fp=#0x100000015ea' 2024-09-10T08:44:46.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8111 fp=#0x10000002211' 2024-09-10T08:44:46.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8143 fp=#0x1000000252a' 2024-09-10T08:44:46.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8175 fp=#0x1000000251d' 2024-09-10T08:44:46.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8207 fp=#0x1000000257a' 2024-09-10T08:44:46.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8239 fp=#0x10000002545' 2024-09-10T08:44:46.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8271 fp=#0x1000000525b' 2024-09-10T08:44:46.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8303 fp=#0x100000045f4' 2024-09-10T08:44:46.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8335 fp=#0x1000000523e' 2024-09-10T08:44:46.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8367 fp=#0x1000000525a' 2024-09-10T08:44:46.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8399 fp=#0x10000005244' 2024-09-10T08:44:46.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8431 fp=#0x100000045c9' 2024-09-10T08:44:46.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8463 fp=#0x10000004529' 2024-09-10T08:44:46.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8495 fp=#0x1000000421b' 2024-09-10T08:44:46.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8527 fp=#0x10000003daf' 2024-09-10T08:44:46.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8559 fp=#0x100000041a0' 2024-09-10T08:44:46.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8591 fp=#0x1000000423f' 2024-09-10T08:44:46.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8623 fp=#0x10000004045' 2024-09-10T08:44:46.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8655 fp=#0x100000040a6' 2024-09-10T08:44:46.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8687 fp=#0x10000004099' 2024-09-10T08:44:46.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8719 fp=#0x10000003bcf' 2024-09-10T08:44:46.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8751 fp=#0x10000004022' 2024-09-10T08:44:46.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8783 fp=#0x10000004f37' 2024-09-10T08:44:46.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8815 fp=#0x10000003f88' 2024-09-10T08:44:46.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8847 fp=#0x10000001b09' 2024-09-10T08:44:46.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8879 fp=#0x10000001d24' 2024-09-10T08:44:46.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8911 fp=#0x10000001aaf' 2024-09-10T08:44:46.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8943 fp=#0x1000000447c' 2024-09-10T08:44:46.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8975 fp=#0x100000044a6' 2024-09-10T08:44:46.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9007 fp=#0x10000004464' 2024-09-10T08:44:46.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9039 fp=#0x10000004225' 2024-09-10T08:44:46.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9071 fp=#0x10000000d8a' 2024-09-10T08:44:46.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9103 fp=#0x1000000209c' 2024-09-10T08:44:46.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9135 fp=#0x10000000dae' 2024-09-10T08:44:46.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9167 fp=#0x100000040fc' 2024-09-10T08:44:46.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9231 fp=#0x10000004192' 2024-09-10T08:44:46.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9263 fp=#0x10000000b1d' 2024-09-10T08:44:46.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9295 fp=#0x10000000bae' 2024-09-10T08:44:46.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9327 fp=#0x10000000b03' 2024-09-10T08:44:46.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9359 fp=#0x10000004cb8' 2024-09-10T08:44:46.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9391 fp=#0x10000000b81' 2024-09-10T08:44:46.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9423 fp=#0x10000004e6b' 2024-09-10T08:44:46.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9455 fp=#0x10000000b69' 2024-09-10T08:44:46.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9487 fp=#0x10000000eb6' 2024-09-10T08:44:46.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9519 fp=#0x10000001ed7' 2024-09-10T08:44:46.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9551 fp=#0x100000019e2' 2024-09-10T08:44:46.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9583 fp=#0x100000048a9' 2024-09-10T08:44:46.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9615 fp=#0x10000001a55' 2024-09-10T08:44:46.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9647 fp=#0x10000000ed5' 2024-09-10T08:44:46.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9679 fp=#0x1000000491b' 2024-09-10T08:44:46.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9711 fp=#0x10000004856' 2024-09-10T08:44:46.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9743 fp=#0x100000019a4' 2024-09-10T08:44:46.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9775 fp=#0x1000000446e' 2024-09-10T08:44:46.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9807 fp=#0x100000017f9' 2024-09-10T08:44:46.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9839 fp=#0x100000017c6' 2024-09-10T08:44:46.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9871 fp=#0x10000004459' 2024-09-10T08:44:46.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9903 fp=#0x100000043ca' 2024-09-10T08:44:46.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9935 fp=#0x100000016a7' 2024-09-10T08:44:46.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9967 fp=#0x10000001692' 2024-09-10T08:44:46.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9999 fp=#0x10000001810' 2024-09-10T08:44:46.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10031 fp=#0x100000016a1' 2024-09-10T08:44:46.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10063 fp=#0x1000000162b' 2024-09-10T08:44:46.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10095 fp=#0x1000000332d' 2024-09-10T08:44:46.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10127 fp=#0x10000003897' 2024-09-10T08:44:46.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10159 fp=#0x10000003e2c' 2024-09-10T08:44:46.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10191 fp=#0x10000001dcb' 2024-09-10T08:44:46.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10223 fp=#0x1000000389f' 2024-09-10T08:44:46.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10255 fp=#0x10000003a28' 2024-09-10T08:44:46.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10287 fp=#0x1000000496d' 2024-09-10T08:44:46.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10319 fp=#0x10000002422' 2024-09-10T08:44:46.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10351 fp=#0x10000001ec2' 2024-09-10T08:44:46.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10383 fp=#0x10000003a2c' 2024-09-10T08:44:46.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10415 fp=#0x100000013ab' 2024-09-10T08:44:46.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10447 fp=#0x1000000394e' 2024-09-10T08:44:46.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10479 fp=#0x10000003a11' 2024-09-10T08:44:46.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10511 fp=#0x10000003a42' 2024-09-10T08:44:46.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10543 fp=#0x10000003ad0' 2024-09-10T08:44:46.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10607 fp=#0x100000048b7' 2024-09-10T08:44:46.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10639 fp=#0x10000004871' 2024-09-10T08:44:46.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10671 fp=#0x10000001439' 2024-09-10T08:44:46.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10703 fp=#0x10000004897' 2024-09-10T08:44:46.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10735 fp=#0x100000048e8' 2024-09-10T08:44:46.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10767 fp=#0x100000049ef' 2024-09-10T08:44:46.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10799 fp=#0x10000001f15' 2024-09-10T08:44:46.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10831 fp=#0x10000000a6f' 2024-09-10T08:44:46.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10863 fp=#0x10000001ccf' 2024-09-10T08:44:46.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10895 fp=#0x10000001ca7' 2024-09-10T08:44:46.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10927 fp=#0x10000000613' 2024-09-10T08:44:46.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10959 fp=#0x10000001d43' 2024-09-10T08:44:46.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10991 fp=#0x1000000096e' 2024-09-10T08:44:46.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11023 fp=#0x10000000545' 2024-09-10T08:44:46.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11055 fp=#0x100000038b2' 2024-09-10T08:44:46.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11087 fp=#0x10000003992' 2024-09-10T08:44:46.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11119 fp=#0x10000002096' 2024-09-10T08:44:46.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11151 fp=#0x10000003f26' 2024-09-10T08:44:46.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11183 fp=#0x100000021c7' 2024-09-10T08:44:46.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11215 fp=#0x100000021b3' 2024-09-10T08:44:46.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11247 fp=#0x100000021b7' 2024-09-10T08:44:46.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11279 fp=#0x100000020f7' 2024-09-10T08:44:46.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11311 fp=#0x100000039c5' 2024-09-10T08:44:46.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11343 fp=#0x10000003e4b' 2024-09-10T08:44:46.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11375 fp=#0x10000003e06' 2024-09-10T08:44:46.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11407 fp=#0x10000003f40' 2024-09-10T08:44:46.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11439 fp=#0x100000019d2' 2024-09-10T08:44:46.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11471 fp=#0x10000001bc2' 2024-09-10T08:44:46.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11503 fp=#0x10000001a21' 2024-09-10T08:44:46.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11695 fp=#0x10000000a13' 2024-09-10T08:44:46.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11727 fp=#0x10000000d0a' 2024-09-10T08:44:46.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11759 fp=#0x100000041dd' 2024-09-10T08:44:46.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11791 fp=#0x10000000912' 2024-09-10T08:44:46.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11823 fp=#0x100000009d6' 2024-09-10T08:44:46.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11855 fp=#0x1000000419f' 2024-09-10T08:44:46.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11887 fp=#0x100000008eb' 2024-09-10T08:44:46.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11919 fp=#0x10000000ef0' 2024-09-10T08:44:46.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11951 fp=#0x1000000111c' 2024-09-10T08:44:46.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11983 fp=#0x100000010e5' 2024-09-10T08:44:46.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12015 fp=#0x10000001169' 2024-09-10T08:44:46.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12047 fp=#0x10000000ef5' 2024-09-10T08:44:46.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12079 fp=#0x10000004545' 2024-09-10T08:44:46.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12111 fp=#0x10000003375' 2024-09-10T08:44:46.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12143 fp=#0x10000003513' 2024-09-10T08:44:46.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12175 fp=#0x10000003361' 2024-09-10T08:44:46.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12207 fp=#0x10000004481' 2024-09-10T08:44:46.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12239 fp=#0x10000003f61' 2024-09-10T08:44:46.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12271 fp=#0x10000000d2e' 2024-09-10T08:44:46.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12303 fp=#0x10000003a9d' 2024-09-10T08:44:46.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12335 fp=#0x10000000ced' 2024-09-10T08:44:46.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12367 fp=#0x10000002427' 2024-09-10T08:44:46.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12399 fp=#0x10000003a3d' 2024-09-10T08:44:46.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12431 fp=#0x10000003be2' 2024-09-10T08:44:46.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12463 fp=#0x10000004bc7' 2024-09-10T08:44:46.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12495 fp=#0x10000004c9c' 2024-09-10T08:44:46.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12527 fp=#0x10000004be9' 2024-09-10T08:44:46.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12559 fp=#0x10000004b52' 2024-09-10T08:44:46.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12591 fp=#0x10000004c1d' 2024-09-10T08:44:46.732 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12623 fp=#0x10000004c61' 2024-09-10T08:44:46.732 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12655 fp=#0x100000049f7' 2024-09-10T08:44:46.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12687 fp=#0x10000004946' 2024-09-10T08:44:46.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12719 fp=#0x100000048d1' 2024-09-10T08:44:46.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12751 fp=#0x10000005360' 2024-09-10T08:44:46.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12783 fp=#0x100000020d8' 2024-09-10T08:44:46.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12815 fp=#0x100000022b9' 2024-09-10T08:44:46.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12847 fp=#0x10000005382' 2024-09-10T08:44:46.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12879 fp=#0x100000053a8' 2024-09-10T08:44:46.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12911 fp=#0x10000002237' 2024-09-10T08:44:46.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12943 fp=#0x10000003a5f' 2024-09-10T08:44:46.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13007 fp=#0x100000053ff' 2024-09-10T08:44:46.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13031 fp=#0x1000000233f' 2024-09-10T08:44:46.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13063 fp=#0x100000039ab' 2024-09-10T08:44:46.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13127 fp=#0x1000000222c' 2024-09-10T08:44:46.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13159 fp=#0x10000002218' 2024-09-10T08:44:46.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:248 fp=#0x100000040d7' 2024-09-10T08:44:46.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:280 fp=#0x100000036a6' 2024-09-10T08:44:46.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:312 fp=#0x10000000a89' 2024-09-10T08:44:46.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:400 fp=#0x10000004ad0' 2024-09-10T08:44:46.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:528 fp=#0x10000003536' 2024-09-10T08:44:46.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:784 fp=#0x10000000f29' 2024-09-10T08:44:46.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:816 fp=#0x10000000dc5' 2024-09-10T08:44:46.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:848 fp=#0x10000000ea6' 2024-09-10T08:44:46.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:880 fp=#0x10000000db3' 2024-09-10T08:44:46.747 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:912 fp=#0x10000000f27' 2024-09-10T08:44:46.747 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:944 fp=#0x10000004445' 2024-09-10T08:44:46.747 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:976 fp=#0x10000000eaf' 2024-09-10T08:44:46.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1008 fp=#0x100000011d1' 2024-09-10T08:44:46.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1072 fp=#0x100000040f9' 2024-09-10T08:44:46.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1264 fp=#0x100000035c1' 2024-09-10T08:44:46.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1296 fp=#0x10000003627' 2024-09-10T08:44:46.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1328 fp=#0x1000000203c' 2024-09-10T08:44:46.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1424 fp=#0x100000032d3' 2024-09-10T08:44:46.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1456 fp=#0x100000032fc' 2024-09-10T08:44:46.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1520 fp=#0x10000004810' 2024-09-10T08:44:46.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1584 fp=#0x10000003f24' 2024-09-10T08:44:46.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1616 fp=#0x10000003cc1' 2024-09-10T08:44:46.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1648 fp=#0x100000045c7' 2024-09-10T08:44:46.755 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1680 fp=#0x1000000459f' 2024-09-10T08:44:46.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1808 fp=#0x100000027ad' 2024-09-10T08:44:46.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1840 fp=#0x1000000275d' 2024-09-10T08:44:46.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1904 fp=#0x1000000277f' 2024-09-10T08:44:46.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1936 fp=#0x10000003a04' 2024-09-10T08:44:46.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2096 fp=#0x10000002208' 2024-09-10T08:44:46.760 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2192 fp=#0x10000002202' 2024-09-10T08:44:46.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2224 fp=#0x100000026d4' 2024-09-10T08:44:46.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2256 fp=#0x100000025ae' 2024-09-10T08:44:46.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2576 fp=#0x10000001e2f' 2024-09-10T08:44:46.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2608 fp=#0x100000010de' 2024-09-10T08:44:46.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2640 fp=#0x10000001212' 2024-09-10T08:44:46.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2704 fp=#0x1000000110a' 2024-09-10T08:44:46.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2736 fp=#0x100000018e5' 2024-09-10T08:44:46.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2768 fp=#0x10000003650' 2024-09-10T08:44:46.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2800 fp=#0x1000000341b' 2024-09-10T08:44:46.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2832 fp=#0x1000000460f' 2024-09-10T08:44:46.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2864 fp=#0x10000004735' 2024-09-10T08:44:46.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2896 fp=#0x10000004683' 2024-09-10T08:44:46.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2928 fp=#0x100000033d1' 2024-09-10T08:44:46.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2960 fp=#0x100000047b0' 2024-09-10T08:44:46.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2992 fp=#0x10000004a63' 2024-09-10T08:44:46.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3024 fp=#0x100000049df' 2024-09-10T08:44:46.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3056 fp=#0x100000018c8' 2024-09-10T08:44:46.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3088 fp=#0x10000004a3f' 2024-09-10T08:44:46.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3120 fp=#0x100000049c9' 2024-09-10T08:44:46.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3184 fp=#0x10000000dd6' 2024-09-10T08:44:46.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3216 fp=#0x1000000079f' 2024-09-10T08:44:46.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3248 fp=#0x100000006fe' 2024-09-10T08:44:46.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3280 fp=#0x100000007d6' 2024-09-10T08:44:46.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3312 fp=#0x10000003d69' 2024-09-10T08:44:46.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3344 fp=#0x1000000078a' 2024-09-10T08:44:46.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3376 fp=#0x10000003c3a' 2024-09-10T08:44:46.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3408 fp=#0x10000000700' 2024-09-10T08:44:46.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3440 fp=#0x10000000782' 2024-09-10T08:44:46.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3472 fp=#0x10000004082' 2024-09-10T08:44:46.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3504 fp=#0x10000003c70' 2024-09-10T08:44:46.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3568 fp=#0x10000003c90' 2024-09-10T08:44:46.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3600 fp=#0x10000003c76' 2024-09-10T08:44:46.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3632 fp=#0x10000004cbf' 2024-09-10T08:44:46.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3664 fp=#0x10000004b7d' 2024-09-10T08:44:46.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3696 fp=#0x10000004c87' 2024-09-10T08:44:46.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3728 fp=#0x10000003f7c' 2024-09-10T08:44:46.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3760 fp=#0x100000025b0' 2024-09-10T08:44:46.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3792 fp=#0x10000003fbd' 2024-09-10T08:44:46.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3824 fp=#0x100000025a7' 2024-09-10T08:44:46.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3856 fp=#0x10000003f12' 2024-09-10T08:44:46.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3888 fp=#0x100000024dd' 2024-09-10T08:44:46.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3920 fp=#0x10000000680' 2024-09-10T08:44:46.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3952 fp=#0x100000006b5' 2024-09-10T08:44:46.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3984 fp=#0x10000000543' 2024-09-10T08:44:46.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4016 fp=#0x10000000675' 2024-09-10T08:44:46.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4048 fp=#0x10000000ff9' 2024-09-10T08:44:46.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4080 fp=#0x10000004e64' 2024-09-10T08:44:46.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4112 fp=#0x10000004621' 2024-09-10T08:44:46.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4144 fp=#0x10000000f91' 2024-09-10T08:44:46.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4176 fp=#0x10000004632' 2024-09-10T08:44:46.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4208 fp=#0x10000004e55' 2024-09-10T08:44:46.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4240 fp=#0x10000003bf6' 2024-09-10T08:44:46.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4272 fp=#0x10000003b1a' 2024-09-10T08:44:46.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4304 fp=#0x10000003b57' 2024-09-10T08:44:46.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4336 fp=#0x100000049d3' 2024-09-10T08:44:46.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4496 fp=#0x100000050c0' 2024-09-10T08:44:46.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4528 fp=#0x1000000509d' 2024-09-10T08:44:46.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4560 fp=#0x100000050d9' 2024-09-10T08:44:46.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4592 fp=#0x100000026f6' 2024-09-10T08:44:46.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4656 fp=#0x100000038d3' 2024-09-10T08:44:46.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4688 fp=#0x1000000398d' 2024-09-10T08:44:46.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4720 fp=#0x100000014ad' 2024-09-10T08:44:46.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4752 fp=#0x100000012f1' 2024-09-10T08:44:46.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4784 fp=#0x1000000135a' 2024-09-10T08:44:46.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4816 fp=#0x1000000384d' 2024-09-10T08:44:46.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4848 fp=#0x10000003848' 2024-09-10T08:44:46.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4880 fp=#0x10000003836' 2024-09-10T08:44:46.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4912 fp=#0x1000000373c' 2024-09-10T08:44:46.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4944 fp=#0x100000036ac' 2024-09-10T08:44:46.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4976 fp=#0x1000000536b' 2024-09-10T08:44:46.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5008 fp=#0x100000050ba' 2024-09-10T08:44:46.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5040 fp=#0x10000005119' 2024-09-10T08:44:46.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5072 fp=#0x1000000535c' 2024-09-10T08:44:46.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5104 fp=#0x100000050fe' 2024-09-10T08:44:46.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5168 fp=#0x10000001cab' 2024-09-10T08:44:46.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5200 fp=#0x100000020b4' 2024-09-10T08:44:46.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5232 fp=#0x10000001bab' 2024-09-10T08:44:46.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5264 fp=#0x100000036ce' 2024-09-10T08:44:46.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5296 fp=#0x10000003720' 2024-09-10T08:44:46.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5328 fp=#0x10000004a7d' 2024-09-10T08:44:46.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5360 fp=#0x100000036a5' 2024-09-10T08:44:46.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5392 fp=#0x10000003db1' 2024-09-10T08:44:46.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5424 fp=#0x10000003db7' 2024-09-10T08:44:46.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5488 fp=#0x1000000151a' 2024-09-10T08:44:46.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5552 fp=#0x1000000127e' 2024-09-10T08:44:46.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5616 fp=#0x10000001016' 2024-09-10T08:44:46.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5648 fp=#0x10000000fee' 2024-09-10T08:44:46.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5680 fp=#0x10000002124' 2024-09-10T08:44:46.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5712 fp=#0x10000000f57' 2024-09-10T08:44:46.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5744 fp=#0x1000000109c' 2024-09-10T08:44:46.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5776 fp=#0x10000000fc5' 2024-09-10T08:44:46.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5808 fp=#0x10000004d17' 2024-09-10T08:44:46.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5840 fp=#0x10000004b12' 2024-09-10T08:44:46.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5872 fp=#0x100000043b0' 2024-09-10T08:44:46.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5904 fp=#0x10000004455' 2024-09-10T08:44:46.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5936 fp=#0x1000000441d' 2024-09-10T08:44:46.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5968 fp=#0x1000000447f' 2024-09-10T08:44:46.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6096 fp=#0x100000035bf' 2024-09-10T08:44:46.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6224 fp=#0x100000033ac' 2024-09-10T08:44:46.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6288 fp=#0x10000001e34' 2024-09-10T08:44:46.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6320 fp=#0x10000003308' 2024-09-10T08:44:46.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6352 fp=#0x10000004722' 2024-09-10T08:44:46.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6384 fp=#0x10000001e68' 2024-09-10T08:44:46.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6416 fp=#0x10000004cca' 2024-09-10T08:44:46.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6448 fp=#0x10000004aea' 2024-09-10T08:44:46.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6480 fp=#0x10000004c30' 2024-09-10T08:44:46.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6512 fp=#0x10000004fae' 2024-09-10T08:44:46.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6544 fp=#0x10000004cee' 2024-09-10T08:44:46.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6640 fp=#0x10000003a61' 2024-09-10T08:44:46.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6672 fp=#0x10000004f5c' 2024-09-10T08:44:46.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6704 fp=#0x10000005036' 2024-09-10T08:44:46.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6736 fp=#0x10000003b0e' 2024-09-10T08:44:46.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6768 fp=#0x10000003a1f' 2024-09-10T08:44:46.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6800 fp=#0x10000004886' 2024-09-10T08:44:46.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6832 fp=#0x10000004754' 2024-09-10T08:44:46.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6864 fp=#0x10000005357' 2024-09-10T08:44:46.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6896 fp=#0x10000004639' 2024-09-10T08:44:46.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6928 fp=#0x10000004655' 2024-09-10T08:44:46.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6960 fp=#0x10000000642' 2024-09-10T08:44:46.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6992 fp=#0x100000005cc' 2024-09-10T08:44:46.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7024 fp=#0x100000005c2' 2024-09-10T08:44:46.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7056 fp=#0x100000017b6' 2024-09-10T08:44:46.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7088 fp=#0x1000000059c' 2024-09-10T08:44:46.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7120 fp=#0x10000000677' 2024-09-10T08:44:46.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7152 fp=#0x10000004ff0' 2024-09-10T08:44:46.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7184 fp=#0x10000003467' 2024-09-10T08:44:46.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7280 fp=#0x10000005116' 2024-09-10T08:44:46.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7408 fp=#0x1000000481d' 2024-09-10T08:44:46.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7440 fp=#0x100000046fe' 2024-09-10T08:44:46.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7472 fp=#0x1000000245a' 2024-09-10T08:44:46.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7504 fp=#0x10000004805' 2024-09-10T08:44:46.824 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7536 fp=#0x1000000133b' 2024-09-10T08:44:46.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7568 fp=#0x10000005174' 2024-09-10T08:44:46.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7632 fp=#0x1000000208a' 2024-09-10T08:44:46.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7664 fp=#0x10000005192' 2024-09-10T08:44:46.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7696 fp=#0x10000005194' 2024-09-10T08:44:46.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7728 fp=#0x10000004f0b' 2024-09-10T08:44:46.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7760 fp=#0x10000004eda' 2024-09-10T08:44:46.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7792 fp=#0x100000016d7' 2024-09-10T08:44:46.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7824 fp=#0x10000004edb' 2024-09-10T08:44:46.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7856 fp=#0x10000004e14' 2024-09-10T08:44:46.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7888 fp=#0x10000004f32' 2024-09-10T08:44:46.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7920 fp=#0x10000001557' 2024-09-10T08:44:46.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7952 fp=#0x10000004d35' 2024-09-10T08:44:46.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7984 fp=#0x10000004cf7' 2024-09-10T08:44:46.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8016 fp=#0x100000013b2' 2024-09-10T08:44:46.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8048 fp=#0x100000016ce' 2024-09-10T08:44:46.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8080 fp=#0x10000001a29' 2024-09-10T08:44:46.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8112 fp=#0x10000001646' 2024-09-10T08:44:46.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8144 fp=#0x100000026da' 2024-09-10T08:44:46.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8176 fp=#0x100000025e0' 2024-09-10T08:44:46.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8208 fp=#0x100000026ac' 2024-09-10T08:44:46.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8240 fp=#0x1000000253b' 2024-09-10T08:44:46.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8272 fp=#0x100000046a2' 2024-09-10T08:44:46.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8304 fp=#0x1000000414e' 2024-09-10T08:44:46.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8336 fp=#0x10000005279' 2024-09-10T08:44:46.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8368 fp=#0x100000045b8' 2024-09-10T08:44:46.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8400 fp=#0x1000000527f' 2024-09-10T08:44:46.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8432 fp=#0x1000000444f' 2024-09-10T08:44:46.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8464 fp=#0x10000003f97' 2024-09-10T08:44:46.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8496 fp=#0x10000004163' 2024-09-10T08:44:46.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8528 fp=#0x10000003e27' 2024-09-10T08:44:46.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8560 fp=#0x10000003e6f' 2024-09-10T08:44:46.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8592 fp=#0x10000004215' 2024-09-10T08:44:46.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8624 fp=#0x10000003bf3' 2024-09-10T08:44:46.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8656 fp=#0x10000003c3e' 2024-09-10T08:44:46.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8688 fp=#0x1000000400f' 2024-09-10T08:44:46.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8720 fp=#0x100000052f2' 2024-09-10T08:44:46.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8752 fp=#0x10000004a16' 2024-09-10T08:44:46.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8784 fp=#0x10000004eca' 2024-09-10T08:44:46.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8816 fp=#0x10000005320' 2024-09-10T08:44:46.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8848 fp=#0x10000001af5' 2024-09-10T08:44:46.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8880 fp=#0x10000001bcd' 2024-09-10T08:44:46.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8912 fp=#0x10000004389' 2024-09-10T08:44:46.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8944 fp=#0x10000001d2e' 2024-09-10T08:44:46.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8976 fp=#0x1000000243a' 2024-09-10T08:44:46.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9008 fp=#0x10000004399' 2024-09-10T08:44:46.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9040 fp=#0x10000000d3a' 2024-09-10T08:44:46.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9072 fp=#0x10000000cf8' 2024-09-10T08:44:46.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9104 fp=#0x10000000d18' 2024-09-10T08:44:46.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9136 fp=#0x10000000d2f' 2024-09-10T08:44:46.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9168 fp=#0x10000000ce3' 2024-09-10T08:44:46.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9200 fp=#0x100000041ad' 2024-09-10T08:44:46.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9232 fp=#0x10000003bfe' 2024-09-10T08:44:46.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9264 fp=#0x10000000b12' 2024-09-10T08:44:46.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9296 fp=#0x10000001750' 2024-09-10T08:44:46.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9328 fp=#0x10000001a17' 2024-09-10T08:44:46.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9360 fp=#0x10000001c23' 2024-09-10T08:44:46.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9392 fp=#0x10000000b8e' 2024-09-10T08:44:46.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9424 fp=#0x10000000b1f' 2024-09-10T08:44:46.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9456 fp=#0x10000000b5f' 2024-09-10T08:44:46.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9488 fp=#0x10000004b43' 2024-09-10T08:44:46.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9520 fp=#0x10000001edc' 2024-09-10T08:44:46.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9552 fp=#0x10000000ee7' 2024-09-10T08:44:46.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9584 fp=#0x10000001ab7' 2024-09-10T08:44:46.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9616 fp=#0x1000000489d' 2024-09-10T08:44:46.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9648 fp=#0x10000001ec3' 2024-09-10T08:44:46.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9680 fp=#0x10000004800' 2024-09-10T08:44:46.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9712 fp=#0x10000001e29' 2024-09-10T08:44:46.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9744 fp=#0x10000004451' 2024-09-10T08:44:46.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9776 fp=#0x100000034b5' 2024-09-10T08:44:46.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9808 fp=#0x1000000435c' 2024-09-10T08:44:46.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9840 fp=#0x10000001a1c' 2024-09-10T08:44:46.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9872 fp=#0x1000000434d' 2024-09-10T08:44:46.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9904 fp=#0x10000001970' 2024-09-10T08:44:46.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9936 fp=#0x1000000160b' 2024-09-10T08:44:46.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9968 fp=#0x100000026ea' 2024-09-10T08:44:46.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10000 fp=#0x10000001a19' 2024-09-10T08:44:46.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10032 fp=#0x10000001a10' 2024-09-10T08:44:46.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10064 fp=#0x10000001ffb' 2024-09-10T08:44:46.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10096 fp=#0x100000038ab' 2024-09-10T08:44:46.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10128 fp=#0x10000004809' 2024-09-10T08:44:46.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10160 fp=#0x100000038b1' 2024-09-10T08:44:46.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10192 fp=#0x10000003a44' 2024-09-10T08:44:46.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10224 fp=#0x100000038d7' 2024-09-10T08:44:46.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10256 fp=#0x10000003972' 2024-09-10T08:44:46.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10288 fp=#0x10000003bef' 2024-09-10T08:44:46.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10320 fp=#0x100000023a9' 2024-09-10T08:44:46.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10352 fp=#0x10000004998' 2024-09-10T08:44:46.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10384 fp=#0x10000003a6f' 2024-09-10T08:44:46.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10416 fp=#0x10000001389' 2024-09-10T08:44:46.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10448 fp=#0x10000003afc' 2024-09-10T08:44:46.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10480 fp=#0x10000003941' 2024-09-10T08:44:46.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10512 fp=#0x10000001318' 2024-09-10T08:44:46.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10544 fp=#0x10000003a8a' 2024-09-10T08:44:46.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10576 fp=#0x10000004980' 2024-09-10T08:44:46.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10672 fp=#0x10000004a0e' 2024-09-10T08:44:46.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10768 fp=#0x100000049bd' 2024-09-10T08:44:46.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10800 fp=#0x10000000703' 2024-09-10T08:44:46.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10832 fp=#0x1000000053f' 2024-09-10T08:44:46.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10864 fp=#0x10000000835' 2024-09-10T08:44:46.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10896 fp=#0x10000003ecd' 2024-09-10T08:44:46.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10928 fp=#0x1000000054e' 2024-09-10T08:44:46.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10960 fp=#0x1000000057e' 2024-09-10T08:44:46.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10992 fp=#0x10000000930' 2024-09-10T08:44:46.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11024 fp=#0x100000018b8' 2024-09-10T08:44:46.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11056 fp=#0x100000038f9' 2024-09-10T08:44:46.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11088 fp=#0x100000038de' 2024-09-10T08:44:46.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11120 fp=#0x10000003f55' 2024-09-10T08:44:46.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11152 fp=#0x10000003f71' 2024-09-10T08:44:46.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11184 fp=#0x10000003fbf' 2024-09-10T08:44:46.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11216 fp=#0x10000003863' 2024-09-10T08:44:46.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11248 fp=#0x1000000213f' 2024-09-10T08:44:46.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11280 fp=#0x1000000218f' 2024-09-10T08:44:46.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11344 fp=#0x10000001b00' 2024-09-10T08:44:46.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11376 fp=#0x10000003dd7' 2024-09-10T08:44:46.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11408 fp=#0x10000001a60' 2024-09-10T08:44:46.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11440 fp=#0x10000003f79' 2024-09-10T08:44:46.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11472 fp=#0x10000001a1b' 2024-09-10T08:44:46.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11504 fp=#0x10000001cdf' 2024-09-10T08:44:46.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11600 fp=#0x10000005171' 2024-09-10T08:44:46.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11664 fp=#0x10000005162' 2024-09-10T08:44:46.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11696 fp=#0x10000004231' 2024-09-10T08:44:46.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11728 fp=#0x10000000b06' 2024-09-10T08:44:46.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11760 fp=#0x10000001d5b' 2024-09-10T08:44:46.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11792 fp=#0x10000000929' 2024-09-10T08:44:46.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11824 fp=#0x10000000885' 2024-09-10T08:44:46.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11888 fp=#0x100000008ac' 2024-09-10T08:44:46.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11920 fp=#0x10000000f21' 2024-09-10T08:44:46.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11952 fp=#0x10000001102' 2024-09-10T08:44:46.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11984 fp=#0x100000052c8' 2024-09-10T08:44:46.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12048 fp=#0x10000000ef8' 2024-09-10T08:44:46.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12112 fp=#0x10000003342' 2024-09-10T08:44:46.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12144 fp=#0x1000000446b' 2024-09-10T08:44:46.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12176 fp=#0x100000044eb' 2024-09-10T08:44:46.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12208 fp=#0x100000044aa' 2024-09-10T08:44:46.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12240 fp=#0x10000003bb4' 2024-09-10T08:44:46.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12272 fp=#0x100000051e6' 2024-09-10T08:44:46.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12304 fp=#0x10000000d04' 2024-09-10T08:44:46.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12336 fp=#0x10000003a12' 2024-09-10T08:44:46.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12368 fp=#0x10000000d2a' 2024-09-10T08:44:46.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12400 fp=#0x100000051fc' 2024-09-10T08:44:46.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12432 fp=#0x10000000d38' 2024-09-10T08:44:46.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12496 fp=#0x10000004b98' 2024-09-10T08:44:46.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12528 fp=#0x1000000104f' 2024-09-10T08:44:46.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12560 fp=#0x10000004bf8' 2024-09-10T08:44:46.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12592 fp=#0x10000004be4' 2024-09-10T08:44:46.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12624 fp=#0x10000004cd8' 2024-09-10T08:44:46.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12656 fp=#0x1000000499d' 2024-09-10T08:44:46.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12688 fp=#0x10000005359' 2024-09-10T08:44:46.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12720 fp=#0x100000053f0' 2024-09-10T08:44:46.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12752 fp=#0x100000053e2' 2024-09-10T08:44:46.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12816 fp=#0x10000002209' 2024-09-10T08:44:46.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12848 fp=#0x100000020be' 2024-09-10T08:44:46.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12880 fp=#0x100000052fc' 2024-09-10T08:44:46.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12912 fp=#0x1000000211e' 2024-09-10T08:44:46.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12944 fp=#0x100000021b8' 2024-09-10T08:44:46.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12976 fp=#0x10000002787' 2024-09-10T08:44:46.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13008 fp=#0x100000053dd' 2024-09-10T08:44:46.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13032 fp=#0x100000022a7' 2024-09-10T08:44:46.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13064 fp=#0x1000000399f' 2024-09-10T08:44:46.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13096 fp=#0x10000003a26' 2024-09-10T08:44:46.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13128 fp=#0x100000039d0' 2024-09-10T08:44:46.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13160 fp=#0x1000000395c' 2024-09-10T08:44:46.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:281 fp=#0x10000004fc7' 2024-09-10T08:44:46.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:313 fp=#0x10000000e9c' 2024-09-10T08:44:46.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:401 fp=#0x1000000470b' 2024-09-10T08:44:46.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:433 fp=#0x10000004663' 2024-09-10T08:44:46.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:497 fp=#0x1000000353d' 2024-09-10T08:44:46.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:561 fp=#0x10000003278' 2024-09-10T08:44:46.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:593 fp=#0x10000003484' 2024-09-10T08:44:46.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:721 fp=#0x100000042f3' 2024-09-10T08:44:46.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:817 fp=#0x10000004507' 2024-09-10T08:44:46.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:849 fp=#0x10000003fb8' 2024-09-10T08:44:46.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:913 fp=#0x10000004482' 2024-09-10T08:44:46.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:945 fp=#0x10000000e56' 2024-09-10T08:44:46.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:977 fp=#0x10000000daf' 2024-09-10T08:44:46.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1009 fp=#0x10000000ea4' 2024-09-10T08:44:46.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1073 fp=#0x100000042a4' 2024-09-10T08:44:46.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1105 fp=#0x1000000422e' 2024-09-10T08:44:46.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1169 fp=#0x10000004121' 2024-09-10T08:44:46.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1201 fp=#0x10000004247' 2024-09-10T08:44:46.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1233 fp=#0x10000003676' 2024-09-10T08:44:46.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1265 fp=#0x1000000367e' 2024-09-10T08:44:46.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1329 fp=#0x100000035c6' 2024-09-10T08:44:46.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1361 fp=#0x100000035ea' 2024-09-10T08:44:46.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1393 fp=#0x100000032c3' 2024-09-10T08:44:46.911 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1553 fp=#0x10000003f18' 2024-09-10T08:44:46.912 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1649 fp=#0x10000004625' 2024-09-10T08:44:46.915 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1937 fp=#0x1000000276c' 2024-09-10T08:44:46.916 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2065 fp=#0x10000001feb' 2024-09-10T08:44:46.917 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2161 fp=#0x1000000202d' 2024-09-10T08:44:46.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2225 fp=#0x100000026e7' 2024-09-10T08:44:46.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2257 fp=#0x100000025f1' 2024-09-10T08:44:46.919 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2321 fp=#0x10000002768' 2024-09-10T08:44:46.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2577 fp=#0x1000000107b' 2024-09-10T08:44:46.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2673 fp=#0x10000001151' 2024-09-10T08:44:46.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2737 fp=#0x10000001852' 2024-09-10T08:44:46.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2769 fp=#0x1000000479c' 2024-09-10T08:44:46.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2801 fp=#0x1000000477e' 2024-09-10T08:44:46.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2833 fp=#0x1000000465f' 2024-09-10T08:44:46.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2865 fp=#0x100000047ec' 2024-09-10T08:44:46.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2897 fp=#0x1000000339b' 2024-09-10T08:44:46.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2929 fp=#0x10000004790' 2024-09-10T08:44:46.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2961 fp=#0x100000033d8' 2024-09-10T08:44:46.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2993 fp=#0x10000000e76' 2024-09-10T08:44:46.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3025 fp=#0x10000004a10' 2024-09-10T08:44:46.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3057 fp=#0x10000001917' 2024-09-10T08:44:46.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3089 fp=#0x100000049a1' 2024-09-10T08:44:46.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3121 fp=#0x10000001939' 2024-09-10T08:44:46.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3153 fp=#0x10000000e72' 2024-09-10T08:44:46.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3185 fp=#0x10000000e6a' 2024-09-10T08:44:46.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3217 fp=#0x10000000715' 2024-09-10T08:44:46.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3249 fp=#0x10000003bfc' 2024-09-10T08:44:46.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3281 fp=#0x10000000798' 2024-09-10T08:44:46.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3313 fp=#0x10000003d26' 2024-09-10T08:44:46.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3345 fp=#0x10000003be6' 2024-09-10T08:44:46.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3377 fp=#0x10000000fef' 2024-09-10T08:44:46.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3409 fp=#0x10000003d36' 2024-09-10T08:44:46.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3441 fp=#0x10000003c32' 2024-09-10T08:44:46.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3473 fp=#0x100000040c2' 2024-09-10T08:44:46.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3505 fp=#0x10000003c3c' 2024-09-10T08:44:46.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3537 fp=#0x10000004139' 2024-09-10T08:44:46.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3601 fp=#0x10000003cb5' 2024-09-10T08:44:46.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3633 fp=#0x10000004bf1' 2024-09-10T08:44:46.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3665 fp=#0x10000004c13' 2024-09-10T08:44:46.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3697 fp=#0x10000004bbd' 2024-09-10T08:44:46.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3729 fp=#0x10000002522' 2024-09-10T08:44:46.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3793 fp=#0x10000003fa7' 2024-09-10T08:44:46.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3825 fp=#0x10000003591' 2024-09-10T08:44:46.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3857 fp=#0x10000003efa' 2024-09-10T08:44:46.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3889 fp=#0x10000002583' 2024-09-10T08:44:46.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3921 fp=#0x100000006bb' 2024-09-10T08:44:46.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3953 fp=#0x10000000693' 2024-09-10T08:44:46.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3985 fp=#0x1000000065e' 2024-09-10T08:44:46.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4017 fp=#0x100000011dd' 2024-09-10T08:44:46.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4049 fp=#0x10000004e62' 2024-09-10T08:44:46.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4081 fp=#0x100000046cc' 2024-09-10T08:44:46.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4113 fp=#0x10000004714' 2024-09-10T08:44:46.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4145 fp=#0x10000004673' 2024-09-10T08:44:46.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4177 fp=#0x10000004677' 2024-09-10T08:44:46.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4209 fp=#0x100000046bf' 2024-09-10T08:44:46.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4241 fp=#0x1000000417b' 2024-09-10T08:44:46.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4273 fp=#0x100000041b7' 2024-09-10T08:44:46.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4305 fp=#0x10000003a7b' 2024-09-10T08:44:46.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4337 fp=#0x10000004085' 2024-09-10T08:44:46.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4369 fp=#0x10000002022' 2024-09-10T08:44:46.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4401 fp=#0x10000001f62' 2024-09-10T08:44:46.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4433 fp=#0x1000000208d' 2024-09-10T08:44:46.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4497 fp=#0x100000050f4' 2024-09-10T08:44:46.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4529 fp=#0x10000005112' 2024-09-10T08:44:46.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4561 fp=#0x10000004024' 2024-09-10T08:44:46.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4593 fp=#0x100000012af' 2024-09-10T08:44:46.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4625 fp=#0x1000000127d' 2024-09-10T08:44:46.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4657 fp=#0x10000001336' 2024-09-10T08:44:46.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4689 fp=#0x100000012ce' 2024-09-10T08:44:46.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4721 fp=#0x100000014c3' 2024-09-10T08:44:46.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4753 fp=#0x100000013e7' 2024-09-10T08:44:46.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4785 fp=#0x100000038fe' 2024-09-10T08:44:46.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4817 fp=#0x10000003841' 2024-09-10T08:44:46.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4849 fp=#0x100000037f3' 2024-09-10T08:44:46.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4881 fp=#0x1000000374b' 2024-09-10T08:44:46.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4913 fp=#0x10000003804' 2024-09-10T08:44:46.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4945 fp=#0x100000036cc' 2024-09-10T08:44:46.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4977 fp=#0x100000053d0' 2024-09-10T08:44:46.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5009 fp=#0x100000050de' 2024-09-10T08:44:46.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5041 fp=#0x100000050b2' 2024-09-10T08:44:46.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5073 fp=#0x100000051b5' 2024-09-10T08:44:46.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5105 fp=#0x1000000519b' 2024-09-10T08:44:46.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5137 fp=#0x10000001cee' 2024-09-10T08:44:46.952 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5169 fp=#0x10000001d27' 2024-09-10T08:44:46.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5201 fp=#0x10000001d07' 2024-09-10T08:44:46.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5233 fp=#0x10000001d95' 2024-09-10T08:44:46.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5265 fp=#0x100000038b0' 2024-09-10T08:44:46.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5297 fp=#0x10000003877' 2024-09-10T08:44:46.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5329 fp=#0x10000004abf' 2024-09-10T08:44:46.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5361 fp=#0x10000004658' 2024-09-10T08:44:46.956 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5393 fp=#0x100000035d6' 2024-09-10T08:44:46.956 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5425 fp=#0x100000037ed' 2024-09-10T08:44:46.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5489 fp=#0x100000014ec' 2024-09-10T08:44:46.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5585 fp=#0x100000014c6' 2024-09-10T08:44:46.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5617 fp=#0x100000010ae' 2024-09-10T08:44:46.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5649 fp=#0x10000000ffc' 2024-09-10T08:44:46.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5681 fp=#0x1000000510f' 2024-09-10T08:44:46.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5713 fp=#0x10000000faa' 2024-09-10T08:44:46.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5745 fp=#0x1000000208b' 2024-09-10T08:44:46.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5777 fp=#0x10000001041' 2024-09-10T08:44:46.961 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5809 fp=#0x1000000431a' 2024-09-10T08:44:46.961 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5841 fp=#0x10000004335' 2024-09-10T08:44:46.961 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5873 fp=#0x10000004c27' 2024-09-10T08:44:46.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5905 fp=#0x10000004c94' 2024-09-10T08:44:46.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5937 fp=#0x10000004981' 2024-09-10T08:44:46.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5969 fp=#0x100000044b5' 2024-09-10T08:44:46.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6065 fp=#0x1000000361b' 2024-09-10T08:44:46.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6129 fp=#0x100000034cd' 2024-09-10T08:44:46.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6161 fp=#0x10000003607' 2024-09-10T08:44:46.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6225 fp=#0x1000000485b' 2024-09-10T08:44:46.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6257 fp=#0x100000035fa' 2024-09-10T08:44:46.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6289 fp=#0x10000001f63' 2024-09-10T08:44:46.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6321 fp=#0x10000001e3c' 2024-09-10T08:44:46.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6353 fp=#0x10000001e21' 2024-09-10T08:44:46.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6385 fp=#0x10000004add' 2024-09-10T08:44:46.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6417 fp=#0x10000004c85' 2024-09-10T08:44:46.968 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6449 fp=#0x10000004af8' 2024-09-10T08:44:46.968 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6481 fp=#0x10000004b21' 2024-09-10T08:44:46.968 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6513 fp=#0x10000004e13' 2024-09-10T08:44:46.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6545 fp=#0x10000004d13' 2024-09-10T08:44:46.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6641 fp=#0x10000003d51' 2024-09-10T08:44:46.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6673 fp=#0x10000003bfd' 2024-09-10T08:44:46.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6705 fp=#0x10000003a78' 2024-09-10T08:44:46.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6737 fp=#0x10000003d37' 2024-09-10T08:44:46.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6769 fp=#0x10000003a3b' 2024-09-10T08:44:46.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6801 fp=#0x10000004561' 2024-09-10T08:44:46.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6833 fp=#0x10000005305' 2024-09-10T08:44:46.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6865 fp=#0x10000004577' 2024-09-10T08:44:46.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6897 fp=#0x100000045ae' 2024-09-10T08:44:46.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6929 fp=#0x100000045d5' 2024-09-10T08:44:46.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6961 fp=#0x10000005088' 2024-09-10T08:44:46.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6993 fp=#0x10000004f54' 2024-09-10T08:44:46.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7025 fp=#0x10000000674' 2024-09-10T08:44:46.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7057 fp=#0x10000004d14' 2024-09-10T08:44:46.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7089 fp=#0x10000000630' 2024-09-10T08:44:46.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7121 fp=#0x1000000064a' 2024-09-10T08:44:46.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7153 fp=#0x10000004f71' 2024-09-10T08:44:46.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7185 fp=#0x100000042a2' 2024-09-10T08:44:46.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7313 fp=#0x10000005139' 2024-09-10T08:44:46.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7377 fp=#0x10000003a6a' 2024-09-10T08:44:46.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7409 fp=#0x10000004715' 2024-09-10T08:44:46.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7441 fp=#0x10000004824' 2024-09-10T08:44:46.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7473 fp=#0x100000023a3' 2024-09-10T08:44:46.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7505 fp=#0x10000004838' 2024-09-10T08:44:46.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7537 fp=#0x100000051d7' 2024-09-10T08:44:46.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7601 fp=#0x1000000436a' 2024-09-10T08:44:46.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7633 fp=#0x10000005186' 2024-09-10T08:44:46.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7665 fp=#0x10000001dbf' 2024-09-10T08:44:46.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7697 fp=#0x100000051db' 2024-09-10T08:44:46.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7729 fp=#0x10000004e77' 2024-09-10T08:44:46.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7761 fp=#0x10000004e65' 2024-09-10T08:44:46.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7793 fp=#0x100000013bc' 2024-09-10T08:44:46.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7825 fp=#0x1000000150b' 2024-09-10T08:44:46.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7857 fp=#0x10000004e1b' 2024-09-10T08:44:46.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7889 fp=#0x10000004e2c' 2024-09-10T08:44:46.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7921 fp=#0x10000004cfe' 2024-09-10T08:44:46.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7953 fp=#0x10000001411' 2024-09-10T08:44:46.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7985 fp=#0x10000001632' 2024-09-10T08:44:46.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8017 fp=#0x10000004ef5' 2024-09-10T08:44:46.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8049 fp=#0x10000003b9f' 2024-09-10T08:44:46.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8081 fp=#0x10000003857' 2024-09-10T08:44:46.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8113 fp=#0x1000000401e' 2024-09-10T08:44:46.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8145 fp=#0x10000003a31' 2024-09-10T08:44:46.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8177 fp=#0x10000002680' 2024-09-10T08:44:46.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8241 fp=#0x1000000266c' 2024-09-10T08:44:46.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8273 fp=#0x10000004736' 2024-09-10T08:44:46.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8305 fp=#0x10000004489' 2024-09-10T08:44:46.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8337 fp=#0x10000004780' 2024-09-10T08:44:46.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8369 fp=#0x10000005278' 2024-09-10T08:44:46.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8401 fp=#0x10000004290' 2024-09-10T08:44:46.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8433 fp=#0x1000000527d' 2024-09-10T08:44:46.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8465 fp=#0x10000003d9c' 2024-09-10T08:44:46.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8497 fp=#0x1000000422f' 2024-09-10T08:44:46.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8529 fp=#0x10000003e46' 2024-09-10T08:44:46.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8561 fp=#0x10000004172' 2024-09-10T08:44:46.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8593 fp=#0x10000003f09' 2024-09-10T08:44:46.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8625 fp=#0x10000003da2' 2024-09-10T08:44:47.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8657 fp=#0x10000004f47' 2024-09-10T08:44:47.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8689 fp=#0x10000003d31' 2024-09-10T08:44:47.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8721 fp=#0x10000003c6a' 2024-09-10T08:44:47.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8753 fp=#0x10000003e64' 2024-09-10T08:44:47.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8785 fp=#0x10000005326' 2024-09-10T08:44:47.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8817 fp=#0x100000052e4' 2024-09-10T08:44:47.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8849 fp=#0x10000001d88' 2024-09-10T08:44:47.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8881 fp=#0x100000043a3' 2024-09-10T08:44:47.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8913 fp=#0x10000004338' 2024-09-10T08:44:47.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8945 fp=#0x10000001ba6' 2024-09-10T08:44:47.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8977 fp=#0x10000001b11' 2024-09-10T08:44:47.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9009 fp=#0x100000043c3' 2024-09-10T08:44:47.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9041 fp=#0x100000041ba' 2024-09-10T08:44:47.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9073 fp=#0x10000004159' 2024-09-10T08:44:47.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9105 fp=#0x10000000dba' 2024-09-10T08:44:47.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9137 fp=#0x10000000d02' 2024-09-10T08:44:47.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9169 fp=#0x1000000405e' 2024-09-10T08:44:47.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9233 fp=#0x100000040c9' 2024-09-10T08:44:47.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9265 fp=#0x10000000ab4' 2024-09-10T08:44:47.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9297 fp=#0x10000002658' 2024-09-10T08:44:47.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9329 fp=#0x10000000b5e' 2024-09-10T08:44:47.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9361 fp=#0x10000000b7e' 2024-09-10T08:44:47.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9393 fp=#0x1000000262c' 2024-09-10T08:44:47.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9425 fp=#0x1000000499f' 2024-09-10T08:44:47.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9457 fp=#0x10000004d44' 2024-09-10T08:44:47.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9489 fp=#0x100000019db' 2024-09-10T08:44:47.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9521 fp=#0x10000001aa4' 2024-09-10T08:44:47.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9553 fp=#0x10000004917' 2024-09-10T08:44:47.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9585 fp=#0x10000000ea5' 2024-09-10T08:44:47.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9617 fp=#0x100000048e2' 2024-09-10T08:44:47.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9649 fp=#0x1000000484b' 2024-09-10T08:44:47.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9681 fp=#0x1000000492b' 2024-09-10T08:44:47.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9713 fp=#0x10000001a9e' 2024-09-10T08:44:47.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9745 fp=#0x10000004397' 2024-09-10T08:44:47.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9777 fp=#0x100000043ce' 2024-09-10T08:44:47.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9809 fp=#0x100000018bf' 2024-09-10T08:44:47.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9841 fp=#0x10000001728' 2024-09-10T08:44:47.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9873 fp=#0x1000000191f' 2024-09-10T08:44:47.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9905 fp=#0x100000018a7' 2024-09-10T08:44:47.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9937 fp=#0x10000001a48' 2024-09-10T08:44:47.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9969 fp=#0x100000015f0' 2024-09-10T08:44:47.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10001 fp=#0x10000001727' 2024-09-10T08:44:47.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10033 fp=#0x100000018e4' 2024-09-10T08:44:47.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10065 fp=#0x10000001880' 2024-09-10T08:44:47.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10097 fp=#0x10000001f29' 2024-09-10T08:44:47.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10129 fp=#0x10000003a07' 2024-09-10T08:44:47.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10161 fp=#0x10000001f35' 2024-09-10T08:44:47.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10193 fp=#0x1000000395a' 2024-09-10T08:44:47.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10225 fp=#0x10000001dfc' 2024-09-10T08:44:47.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10257 fp=#0x10000001dc0' 2024-09-10T08:44:47.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10289 fp=#0x100000023e9' 2024-09-10T08:44:47.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10321 fp=#0x100000023c9' 2024-09-10T08:44:47.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10353 fp=#0x10000001499' 2024-09-10T08:44:47.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10385 fp=#0x10000001410' 2024-09-10T08:44:47.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10417 fp=#0x100000012e1' 2024-09-10T08:44:47.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10449 fp=#0x10000001306' 2024-09-10T08:44:47.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10481 fp=#0x10000003f9f' 2024-09-10T08:44:47.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10513 fp=#0x1000000142f' 2024-09-10T08:44:47.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10545 fp=#0x10000003a68' 2024-09-10T08:44:47.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10609 fp=#0x1000000495e' 2024-09-10T08:44:47.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10641 fp=#0x10000000a7d' 2024-09-10T08:44:47.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10673 fp=#0x100000049e3' 2024-09-10T08:44:47.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10705 fp=#0x10000004941' 2024-09-10T08:44:47.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10769 fp=#0x10000004996' 2024-09-10T08:44:47.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10833 fp=#0x10000001d15' 2024-09-10T08:44:47.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10865 fp=#0x100000045e0' 2024-09-10T08:44:47.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10897 fp=#0x100000018c0' 2024-09-10T08:44:47.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10929 fp=#0x10000000948' 2024-09-10T08:44:47.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10993 fp=#0x1000000118d' 2024-09-10T08:44:47.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11025 fp=#0x10000001d3e' 2024-09-10T08:44:47.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11057 fp=#0x100000038c0' 2024-09-10T08:44:47.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11089 fp=#0x1000000388d' 2024-09-10T08:44:47.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11121 fp=#0x10000002131' 2024-09-10T08:44:47.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11153 fp=#0x10000002213' 2024-09-10T08:44:47.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11185 fp=#0x10000003e1e' 2024-09-10T08:44:47.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11217 fp=#0x10000003f1d' 2024-09-10T08:44:47.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11249 fp=#0x100000048d9' 2024-09-10T08:44:47.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11281 fp=#0x100000021cc' 2024-09-10T08:44:47.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11313 fp=#0x10000001d28' 2024-09-10T08:44:47.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11345 fp=#0x10000001cf1' 2024-09-10T08:44:47.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11377 fp=#0x10000003f99' 2024-09-10T08:44:47.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11409 fp=#0x10000003f72' 2024-09-10T08:44:47.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11441 fp=#0x10000003daa' 2024-09-10T08:44:47.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11473 fp=#0x10000003ee6' 2024-09-10T08:44:47.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11505 fp=#0x10000001a13' 2024-09-10T08:44:47.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11697 fp=#0x10000000867' 2024-09-10T08:44:47.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11729 fp=#0x10000000a2b' 2024-09-10T08:44:47.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11761 fp=#0x10000000990' 2024-09-10T08:44:47.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11793 fp=#0x10000004278' 2024-09-10T08:44:47.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11825 fp=#0x1000000420c' 2024-09-10T08:44:47.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11857 fp=#0x10000004e75' 2024-09-10T08:44:47.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11889 fp=#0x10000004e25' 2024-09-10T08:44:47.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11921 fp=#0x1000000112b' 2024-09-10T08:44:47.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11953 fp=#0x100000052f7' 2024-09-10T08:44:47.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12017 fp=#0x10000000f18' 2024-09-10T08:44:47.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12049 fp=#0x10000000f10' 2024-09-10T08:44:47.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12081 fp=#0x100000044c0' 2024-09-10T08:44:47.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12113 fp=#0x100000043c8' 2024-09-10T08:44:47.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12145 fp=#0x100000045ab' 2024-09-10T08:44:47.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12177 fp=#0x10000003430' 2024-09-10T08:44:47.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12209 fp=#0x10000004413' 2024-09-10T08:44:47.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12241 fp=#0x10000003a7d' 2024-09-10T08:44:47.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12273 fp=#0x100000051e5' 2024-09-10T08:44:47.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12305 fp=#0x10000002442' 2024-09-10T08:44:47.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12337 fp=#0x1000000110e' 2024-09-10T08:44:47.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12369 fp=#0x10000003ac2' 2024-09-10T08:44:47.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12401 fp=#0x1000000256b' 2024-09-10T08:44:47.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12465 fp=#0x1000000102b' 2024-09-10T08:44:47.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12497 fp=#0x1000000183d' 2024-09-10T08:44:47.045 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12561 fp=#0x10000004d79' 2024-09-10T08:44:47.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12593 fp=#0x10000004c4e' 2024-09-10T08:44:47.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12625 fp=#0x10000004b66' 2024-09-10T08:44:47.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12657 fp=#0x10000004d08' 2024-09-10T08:44:47.047 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12689 fp=#0x100000053b9' 2024-09-10T08:44:47.047 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12721 fp=#0x100000053c0' 2024-09-10T08:44:47.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12753 fp=#0x1000000541e' 2024-09-10T08:44:47.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12785 fp=#0x10000002204' 2024-09-10T08:44:47.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12817 fp=#0x1000000217a' 2024-09-10T08:44:47.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12881 fp=#0x100000052be' 2024-09-10T08:44:47.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12913 fp=#0x100000052c3' 2024-09-10T08:44:47.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12945 fp=#0x10000005354' 2024-09-10T08:44:47.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12977 fp=#0x100000027a0' 2024-09-10T08:44:47.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13033 fp=#0x10000002347' 2024-09-10T08:44:47.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13065 fp=#0x100000022d2' 2024-09-10T08:44:47.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13097 fp=#0x100000039f0' 2024-09-10T08:44:47.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13129 fp=#0x10000002315' 2024-09-10T08:44:47.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13161 fp=#0x100000022ac' 2024-09-10T08:44:47.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:145 fp=#0x10000000001' 2024-09-10T08:44:47.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:250 fp=#0x100000020b6' 2024-09-10T08:44:47.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:282 fp=#0x10000001b7a' 2024-09-10T08:44:47.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:314 fp=#0x1000000474c' 2024-09-10T08:44:47.201 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:370 fp=#0x10000004728' 2024-09-10T08:44:47.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:498 fp=#0x10000003268' 2024-09-10T08:44:47.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:530 fp=#0x100000034c3' 2024-09-10T08:44:47.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:818 fp=#0x1000000454e' 2024-09-10T08:44:47.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:850 fp=#0x10000000db5' 2024-09-10T08:44:47.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:882 fp=#0x10000003fdd' 2024-09-10T08:44:47.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:914 fp=#0x100000045b9' 2024-09-10T08:44:47.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:946 fp=#0x10000000d9c' 2024-09-10T08:44:47.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:978 fp=#0x100000040cc' 2024-09-10T08:44:47.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1010 fp=#0x10000000da7' 2024-09-10T08:44:47.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1138 fp=#0x10000004111' 2024-09-10T08:44:47.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1170 fp=#0x1000000428a' 2024-09-10T08:44:47.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1202 fp=#0x10000004191' 2024-09-10T08:44:47.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1234 fp=#0x100000035d0' 2024-09-10T08:44:47.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1266 fp=#0x10000001f98' 2024-09-10T08:44:47.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1298 fp=#0x1000000356a' 2024-09-10T08:44:47.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1330 fp=#0x10000003605' 2024-09-10T08:44:47.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1362 fp=#0x10000001f69' 2024-09-10T08:44:47.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1394 fp=#0x1000000330c' 2024-09-10T08:44:47.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1458 fp=#0x100000032bb' 2024-09-10T08:44:47.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1554 fp=#0x10000003f0f' 2024-09-10T08:44:47.214 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1650 fp=#0x100000045e8' 2024-09-10T08:44:47.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2034 fp=#0x1000000218e' 2024-09-10T08:44:47.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2098 fp=#0x1000000201c' 2024-09-10T08:44:47.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2130 fp=#0x100000021f7' 2024-09-10T08:44:47.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2162 fp=#0x10000002195' 2024-09-10T08:44:47.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2226 fp=#0x100000025f9' 2024-09-10T08:44:47.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2578 fp=#0x10000001072' 2024-09-10T08:44:47.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2642 fp=#0x10000001053' 2024-09-10T08:44:47.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2674 fp=#0x100000010c7' 2024-09-10T08:44:47.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2706 fp=#0x100000010c4' 2024-09-10T08:44:47.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2738 fp=#0x100000010aa' 2024-09-10T08:44:47.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2770 fp=#0x1000000340c' 2024-09-10T08:44:47.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2802 fp=#0x100000047b6' 2024-09-10T08:44:47.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2834 fp=#0x1000000460a' 2024-09-10T08:44:47.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2866 fp=#0x10000003407' 2024-09-10T08:44:47.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2898 fp=#0x10000004839' 2024-09-10T08:44:47.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2962 fp=#0x10000003623' 2024-09-10T08:44:47.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2994 fp=#0x100000049fd' 2024-09-10T08:44:47.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3026 fp=#0x10000000e68' 2024-09-10T08:44:47.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3058 fp=#0x100000018c2' 2024-09-10T08:44:47.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3090 fp=#0x10000004a36' 2024-09-10T08:44:47.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3122 fp=#0x10000004a5b' 2024-09-10T08:44:47.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3154 fp=#0x10000000ddc' 2024-09-10T08:44:47.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3186 fp=#0x10000000dd2' 2024-09-10T08:44:47.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3218 fp=#0x10000003cc5' 2024-09-10T08:44:47.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3250 fp=#0x10000000704' 2024-09-10T08:44:47.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3282 fp=#0x10000003fe5' 2024-09-10T08:44:47.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3346 fp=#0x10000000748' 2024-09-10T08:44:47.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3378 fp=#0x10000003c4a' 2024-09-10T08:44:47.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3410 fp=#0x10000003cfd' 2024-09-10T08:44:47.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3506 fp=#0x10000004169' 2024-09-10T08:44:47.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3570 fp=#0x10000003c1a' 2024-09-10T08:44:47.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3634 fp=#0x10000004cd1' 2024-09-10T08:44:47.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3666 fp=#0x10000004cb7' 2024-09-10T08:44:47.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3698 fp=#0x10000004ae9' 2024-09-10T08:44:47.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3762 fp=#0x100000027bb' 2024-09-10T08:44:47.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3794 fp=#0x10000005342' 2024-09-10T08:44:47.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3826 fp=#0x10000003f5a' 2024-09-10T08:44:47.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3858 fp=#0x100000025ac' 2024-09-10T08:44:47.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3890 fp=#0x10000002732' 2024-09-10T08:44:47.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3922 fp=#0x10000003f70' 2024-09-10T08:44:47.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3986 fp=#0x1000000066b' 2024-09-10T08:44:47.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4018 fp=#0x10000000691' 2024-09-10T08:44:47.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4050 fp=#0x10000004e0a' 2024-09-10T08:44:47.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4082 fp=#0x100000046d6' 2024-09-10T08:44:47.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4114 fp=#0x100000046a7' 2024-09-10T08:44:47.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4146 fp=#0x10000004d51' 2024-09-10T08:44:47.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4178 fp=#0x10000004618' 2024-09-10T08:44:47.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4210 fp=#0x100000046f0' 2024-09-10T08:44:47.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4242 fp=#0x10000004124' 2024-09-10T08:44:47.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4274 fp=#0x10000003a0a' 2024-09-10T08:44:47.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4306 fp=#0x100000040a8' 2024-09-10T08:44:47.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4338 fp=#0x100000040b8' 2024-09-10T08:44:47.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4370 fp=#0x10000001f43' 2024-09-10T08:44:47.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4402 fp=#0x1000000200c' 2024-09-10T08:44:47.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4434 fp=#0x10000001f26' 2024-09-10T08:44:47.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4466 fp=#0x10000004662' 2024-09-10T08:44:47.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4530 fp=#0x10000005130' 2024-09-10T08:44:47.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4562 fp=#0x10000001ccd' 2024-09-10T08:44:47.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4626 fp=#0x10000001273' 2024-09-10T08:44:47.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4658 fp=#0x100000039b3' 2024-09-10T08:44:47.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4690 fp=#0x100000013d2' 2024-09-10T08:44:47.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4722 fp=#0x10000001282' 2024-09-10T08:44:47.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4818 fp=#0x10000004aa4' 2024-09-10T08:44:47.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4850 fp=#0x100000036b9' 2024-09-10T08:44:47.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4882 fp=#0x100000036fd' 2024-09-10T08:44:47.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4914 fp=#0x10000003735' 2024-09-10T08:44:47.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4946 fp=#0x1000000372e' 2024-09-10T08:44:47.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4978 fp=#0x10000005108' 2024-09-10T08:44:47.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5010 fp=#0x100000050c3' 2024-09-10T08:44:47.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5042 fp=#0x10000004fed' 2024-09-10T08:44:47.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5074 fp=#0x10000005078' 2024-09-10T08:44:47.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5106 fp=#0x10000005124' 2024-09-10T08:44:47.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5138 fp=#0x10000001ea4' 2024-09-10T08:44:47.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5170 fp=#0x10000001e3a' 2024-09-10T08:44:47.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5202 fp=#0x10000001b93' 2024-09-10T08:44:47.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5234 fp=#0x10000004186' 2024-09-10T08:44:47.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5266 fp=#0x1000000359b' 2024-09-10T08:44:47.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5298 fp=#0x100000037fa' 2024-09-10T08:44:47.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5330 fp=#0x10000003d53' 2024-09-10T08:44:47.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5362 fp=#0x10000003798' 2024-09-10T08:44:47.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5394 fp=#0x1000000388c' 2024-09-10T08:44:47.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5426 fp=#0x10000004abc' 2024-09-10T08:44:47.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5522 fp=#0x10000001503' 2024-09-10T08:44:47.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5586 fp=#0x100000010c9' 2024-09-10T08:44:47.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5618 fp=#0x10000000ff8' 2024-09-10T08:44:47.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5650 fp=#0x100000050f1' 2024-09-10T08:44:47.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5682 fp=#0x100000010dc' 2024-09-10T08:44:47.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5714 fp=#0x100000010bf' 2024-09-10T08:44:47.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5746 fp=#0x100000050b8' 2024-09-10T08:44:47.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5778 fp=#0x10000000f47' 2024-09-10T08:44:47.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5810 fp=#0x10000004c9b' 2024-09-10T08:44:47.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5842 fp=#0x100000044f3' 2024-09-10T08:44:47.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5874 fp=#0x10000004b48' 2024-09-10T08:44:47.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5906 fp=#0x10000004b2c' 2024-09-10T08:44:47.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5938 fp=#0x10000004bb2' 2024-09-10T08:44:47.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5970 fp=#0x10000004ccd' 2024-09-10T08:44:47.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6098 fp=#0x1000000356d' 2024-09-10T08:44:47.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6130 fp=#0x100000034a4' 2024-09-10T08:44:47.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6194 fp=#0x100000034d9' 2024-09-10T08:44:47.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6226 fp=#0x10000003477' 2024-09-10T08:44:47.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6258 fp=#0x10000003538' 2024-09-10T08:44:47.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6290 fp=#0x10000001db9' 2024-09-10T08:44:47.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6322 fp=#0x10000001e74' 2024-09-10T08:44:47.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6354 fp=#0x10000001ef6' 2024-09-10T08:44:47.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6386 fp=#0x10000001f31' 2024-09-10T08:44:47.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6418 fp=#0x10000004b42' 2024-09-10T08:44:47.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6450 fp=#0x10000004e72' 2024-09-10T08:44:47.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6482 fp=#0x10000004c98' 2024-09-10T08:44:47.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6514 fp=#0x10000004fcc' 2024-09-10T08:44:47.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6546 fp=#0x10000004e05' 2024-09-10T08:44:47.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6642 fp=#0x10000005059' 2024-09-10T08:44:47.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6674 fp=#0x10000003943' 2024-09-10T08:44:47.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6706 fp=#0x10000005062' 2024-09-10T08:44:47.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6738 fp=#0x100000039e0' 2024-09-10T08:44:47.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6770 fp=#0x10000003b02' 2024-09-10T08:44:47.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6802 fp=#0x100000045e5' 2024-09-10T08:44:47.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6834 fp=#0x100000052fd' 2024-09-10T08:44:47.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6866 fp=#0x10000005329' 2024-09-10T08:44:47.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6898 fp=#0x100000047a6' 2024-09-10T08:44:47.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6930 fp=#0x1000000454b' 2024-09-10T08:44:47.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6962 fp=#0x100000006aa' 2024-09-10T08:44:47.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6994 fp=#0x10000004cf8' 2024-09-10T08:44:47.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7026 fp=#0x100000005a4' 2024-09-10T08:44:47.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7058 fp=#0x10000004f7e' 2024-09-10T08:44:47.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7090 fp=#0x100000005db' 2024-09-10T08:44:47.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7122 fp=#0x100000007e0' 2024-09-10T08:44:47.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7154 fp=#0x10000003527' 2024-09-10T08:44:47.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7186 fp=#0x10000000690' 2024-09-10T08:44:47.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7218 fp=#0x10000005111' 2024-09-10T08:44:47.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7410 fp=#0x10000002460' 2024-09-10T08:44:47.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7442 fp=#0x10000002379' 2024-09-10T08:44:47.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7474 fp=#0x1000000482c' 2024-09-10T08:44:47.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7506 fp=#0x1000000480c' 2024-09-10T08:44:47.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7538 fp=#0x10000005197' 2024-09-10T08:44:47.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7570 fp=#0x10000001370' 2024-09-10T08:44:47.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7602 fp=#0x100000051b4' 2024-09-10T08:44:47.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7634 fp=#0x10000005199' 2024-09-10T08:44:47.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7666 fp=#0x10000001307' 2024-09-10T08:44:47.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7698 fp=#0x100000051b7' 2024-09-10T08:44:47.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7730 fp=#0x10000004ed4' 2024-09-10T08:44:47.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7762 fp=#0x10000004e39' 2024-09-10T08:44:47.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7794 fp=#0x100000014d6' 2024-09-10T08:44:47.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7826 fp=#0x10000001444' 2024-09-10T08:44:47.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7858 fp=#0x1000000166d' 2024-09-10T08:44:47.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7890 fp=#0x10000001448' 2024-09-10T08:44:47.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7922 fp=#0x10000001703' 2024-09-10T08:44:47.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7954 fp=#0x100000014a4' 2024-09-10T08:44:47.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7986 fp=#0x100000015bb' 2024-09-10T08:44:47.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8018 fp=#0x10000004e01' 2024-09-10T08:44:47.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8050 fp=#0x100000016fe' 2024-09-10T08:44:47.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8082 fp=#0x10000001857' 2024-09-10T08:44:47.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8114 fp=#0x10000001633' 2024-09-10T08:44:47.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8146 fp=#0x100000025a4' 2024-09-10T08:44:47.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8178 fp=#0x10000004983' 2024-09-10T08:44:47.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8210 fp=#0x100000026d5' 2024-09-10T08:44:47.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8242 fp=#0x10000002604' 2024-09-10T08:44:47.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8274 fp=#0x10000005270' 2024-09-10T08:44:47.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8306 fp=#0x10000004112' 2024-09-10T08:44:47.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8338 fp=#0x10000005271' 2024-09-10T08:44:47.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8370 fp=#0x10000004622' 2024-09-10T08:44:47.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8402 fp=#0x100000045fc' 2024-09-10T08:44:47.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8434 fp=#0x1000000526e' 2024-09-10T08:44:47.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8466 fp=#0x10000003e15' 2024-09-10T08:44:47.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8498 fp=#0x10000003e3a' 2024-09-10T08:44:47.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8530 fp=#0x10000003edc' 2024-09-10T08:44:47.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8562 fp=#0x10000004050' 2024-09-10T08:44:47.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8594 fp=#0x100000044fc' 2024-09-10T08:44:47.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8626 fp=#0x10000005318' 2024-09-10T08:44:47.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8658 fp=#0x10000003c54' 2024-09-10T08:44:47.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8690 fp=#0x10000004f01' 2024-09-10T08:44:47.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8722 fp=#0x10000003c8c' 2024-09-10T08:44:47.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8754 fp=#0x10000003ebd' 2024-09-10T08:44:47.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8786 fp=#0x100000047ee' 2024-09-10T08:44:47.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8818 fp=#0x10000003c36' 2024-09-10T08:44:47.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8850 fp=#0x10000001b40' 2024-09-10T08:44:47.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8882 fp=#0x100000043a6' 2024-09-10T08:44:47.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8914 fp=#0x10000004483' 2024-09-10T08:44:47.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8946 fp=#0x10000001c43' 2024-09-10T08:44:47.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8978 fp=#0x10000001d06' 2024-09-10T08:44:47.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9010 fp=#0x1000000433a' 2024-09-10T08:44:47.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9042 fp=#0x10000004087' 2024-09-10T08:44:47.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9074 fp=#0x10000004160' 2024-09-10T08:44:47.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9106 fp=#0x100000041f6' 2024-09-10T08:44:47.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9138 fp=#0x100000011d7' 2024-09-10T08:44:47.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9170 fp=#0x10000000cc9' 2024-09-10T08:44:47.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9202 fp=#0x10000000d72' 2024-09-10T08:44:47.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9234 fp=#0x10000004295' 2024-09-10T08:44:47.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9266 fp=#0x10000004aa5' 2024-09-10T08:44:47.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9298 fp=#0x100000010af' 2024-09-10T08:44:47.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9330 fp=#0x10000004de4' 2024-09-10T08:44:47.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9362 fp=#0x10000001c2c' 2024-09-10T08:44:47.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9394 fp=#0x10000000b0f' 2024-09-10T08:44:47.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9426 fp=#0x10000001c16' 2024-09-10T08:44:47.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9458 fp=#0x10000000baa' 2024-09-10T08:44:47.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9490 fp=#0x10000000ec7' 2024-09-10T08:44:47.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9522 fp=#0x10000003dcd' 2024-09-10T08:44:47.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9554 fp=#0x10000001a6f' 2024-09-10T08:44:47.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9586 fp=#0x100000049a0' 2024-09-10T08:44:47.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9618 fp=#0x100000048a2' 2024-09-10T08:44:47.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9650 fp=#0x1000000474e' 2024-09-10T08:44:47.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9682 fp=#0x100000048f4' 2024-09-10T08:44:47.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9714 fp=#0x10000004402' 2024-09-10T08:44:47.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9746 fp=#0x1000000191a' 2024-09-10T08:44:47.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9778 fp=#0x100000043a0' 2024-09-10T08:44:47.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9810 fp=#0x10000004371' 2024-09-10T08:44:47.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9842 fp=#0x100000043e4' 2024-09-10T08:44:47.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9874 fp=#0x100000018db' 2024-09-10T08:44:47.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9906 fp=#0x1000000441c' 2024-09-10T08:44:47.315 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9938 fp=#0x100000015aa' 2024-09-10T08:44:47.318 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9970 fp=#0x100000016b7' 2024-09-10T08:44:47.318 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10002 fp=#0x100000016bb' 2024-09-10T08:44:47.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10034 fp=#0x100000015fa' 2024-09-10T08:44:47.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10066 fp=#0x100000016cf' 2024-09-10T08:44:47.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10098 fp=#0x10000001d8e' 2024-09-10T08:44:47.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10130 fp=#0x10000003a0e' 2024-09-10T08:44:47.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10162 fp=#0x10000003a99' 2024-09-10T08:44:47.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10194 fp=#0x1000000278f' 2024-09-10T08:44:47.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10226 fp=#0x10000003912' 2024-09-10T08:44:47.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10258 fp=#0x1000000234a' 2024-09-10T08:44:47.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10290 fp=#0x10000002469' 2024-09-10T08:44:47.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10322 fp=#0x1000000239f' 2024-09-10T08:44:47.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10354 fp=#0x100000034db' 2024-09-10T08:44:47.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10386 fp=#0x100000012f6' 2024-09-10T08:44:47.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10450 fp=#0x10000003abd' 2024-09-10T08:44:47.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10482 fp=#0x10000003af5' 2024-09-10T08:44:47.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10514 fp=#0x1000000137d' 2024-09-10T08:44:47.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10546 fp=#0x1000000130f' 2024-09-10T08:44:47.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10610 fp=#0x1000000499e' 2024-09-10T08:44:47.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10674 fp=#0x10000000dd7' 2024-09-10T08:44:47.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10706 fp=#0x10000004872' 2024-09-10T08:44:47.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10802 fp=#0x10000001d73' 2024-09-10T08:44:47.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10834 fp=#0x10000000548' 2024-09-10T08:44:47.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10866 fp=#0x10000000a66' 2024-09-10T08:44:47.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10898 fp=#0x10000001d3a' 2024-09-10T08:44:47.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10930 fp=#0x10000001d03' 2024-09-10T08:44:47.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10962 fp=#0x100000007b7' 2024-09-10T08:44:47.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11026 fp=#0x10000003ae3' 2024-09-10T08:44:47.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11058 fp=#0x10000003947' 2024-09-10T08:44:47.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11090 fp=#0x10000003a45' 2024-09-10T08:44:47.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11122 fp=#0x1000000238b' 2024-09-10T08:44:47.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11154 fp=#0x10000003ef0' 2024-09-10T08:44:47.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11186 fp=#0x100000021f6' 2024-09-10T08:44:47.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11218 fp=#0x10000003f5c' 2024-09-10T08:44:47.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11250 fp=#0x100000020de' 2024-09-10T08:44:47.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11282 fp=#0x10000003f4c' 2024-09-10T08:44:47.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11314 fp=#0x10000003d4e' 2024-09-10T08:44:47.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11346 fp=#0x10000001cbb' 2024-09-10T08:44:47.334 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11378 fp=#0x100000041df' 2024-09-10T08:44:47.334 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11410 fp=#0x100000019bf' 2024-09-10T08:44:47.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11442 fp=#0x10000001cf6' 2024-09-10T08:44:47.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11474 fp=#0x10000001d02' 2024-09-10T08:44:47.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11506 fp=#0x10000001a2a' 2024-09-10T08:44:47.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11570 fp=#0x1000000517b' 2024-09-10T08:44:47.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11602 fp=#0x10000005128' 2024-09-10T08:44:47.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11698 fp=#0x100000008be' 2024-09-10T08:44:47.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11730 fp=#0x100000008f1' 2024-09-10T08:44:47.339 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11762 fp=#0x10000000863' 2024-09-10T08:44:47.339 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11794 fp=#0x10000004e02' 2024-09-10T08:44:47.340 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11826 fp=#0x1000000094a' 2024-09-10T08:44:47.340 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11858 fp=#0x10000000965' 2024-09-10T08:44:47.340 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11890 fp=#0x10000004258' 2024-09-10T08:44:47.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11954 fp=#0x10000001875' 2024-09-10T08:44:47.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11986 fp=#0x10000001152' 2024-09-10T08:44:47.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12018 fp=#0x10000000f03' 2024-09-10T08:44:47.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12050 fp=#0x10000000f1f' 2024-09-10T08:44:47.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12082 fp=#0x100000043e8' 2024-09-10T08:44:47.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12114 fp=#0x100000044c6' 2024-09-10T08:44:47.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12146 fp=#0x100000034c2' 2024-09-10T08:44:47.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12178 fp=#0x1000000331f' 2024-09-10T08:44:47.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12210 fp=#0x1000000336c' 2024-09-10T08:44:47.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12242 fp=#0x10000004cdc' 2024-09-10T08:44:47.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12274 fp=#0x10000000d4b' 2024-09-10T08:44:47.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12306 fp=#0x10000000d43' 2024-09-10T08:44:47.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12338 fp=#0x10000003a27' 2024-09-10T08:44:47.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12370 fp=#0x10000000cf1' 2024-09-10T08:44:47.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12402 fp=#0x10000001832' 2024-09-10T08:44:47.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12434 fp=#0x10000003b20' 2024-09-10T08:44:47.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12466 fp=#0x10000004c41' 2024-09-10T08:44:47.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12498 fp=#0x10000004c00' 2024-09-10T08:44:47.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12530 fp=#0x10000004ca3' 2024-09-10T08:44:47.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12562 fp=#0x10000004da6' 2024-09-10T08:44:47.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12594 fp=#0x10000004b59' 2024-09-10T08:44:47.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12626 fp=#0x10000004c8b' 2024-09-10T08:44:47.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12658 fp=#0x10000004822' 2024-09-10T08:44:47.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12690 fp=#0x10000005333' 2024-09-10T08:44:47.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12722 fp=#0x1000000536f' 2024-09-10T08:44:47.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12754 fp=#0x100000048e4' 2024-09-10T08:44:47.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12786 fp=#0x10000002335' 2024-09-10T08:44:47.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12818 fp=#0x1000000214a' 2024-09-10T08:44:47.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12850 fp=#0x10000002326' 2024-09-10T08:44:47.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12882 fp=#0x100000022c7' 2024-09-10T08:44:47.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12914 fp=#0x1000000539c' 2024-09-10T08:44:47.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12946 fp=#0x10000003da1' 2024-09-10T08:44:47.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13010 fp=#0x1000000539d' 2024-09-10T08:44:47.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13034 fp=#0x1000000230c' 2024-09-10T08:44:47.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13066 fp=#0x100000053f1' 2024-09-10T08:44:47.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13098 fp=#0x1000000540d' 2024-09-10T08:44:47.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13130 fp=#0x10000002273' 2024-09-10T08:44:47.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13162 fp=#0x10000002240' 2024-09-10T08:44:47.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:150 fp=#0x100000001fa' 2024-09-10T08:44:47.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:251 fp=#0x10000001f5b' 2024-09-10T08:44:47.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:283 fp=#0x1000000356b' 2024-09-10T08:44:47.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:315 fp=#0x1000000171f' 2024-09-10T08:44:47.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:371 fp=#0x1000000473a' 2024-09-10T08:44:47.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:403 fp=#0x10000004ad5' 2024-09-10T08:44:47.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:499 fp=#0x10000003530' 2024-09-10T08:44:47.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:531 fp=#0x10000003267' 2024-09-10T08:44:47.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:563 fp=#0x100000033f4' 2024-09-10T08:44:47.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:595 fp=#0x100000032d8' 2024-09-10T08:44:47.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:723 fp=#0x100000042ad' 2024-09-10T08:44:47.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:819 fp=#0x10000000ea2' 2024-09-10T08:44:47.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:851 fp=#0x10000004042' 2024-09-10T08:44:47.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:883 fp=#0x10000000f23' 2024-09-10T08:44:47.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:915 fp=#0x10000000e66' 2024-09-10T08:44:47.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:979 fp=#0x1000000445a' 2024-09-10T08:44:47.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1011 fp=#0x10000000dad' 2024-09-10T08:44:47.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1107 fp=#0x100000042a9' 2024-09-10T08:44:47.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1171 fp=#0x100000041a1' 2024-09-10T08:44:47.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1203 fp=#0x100000041de' 2024-09-10T08:44:47.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1235 fp=#0x10000002083' 2024-09-10T08:44:47.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1267 fp=#0x100000035ef' 2024-09-10T08:44:47.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1299 fp=#0x1000000207f' 2024-09-10T08:44:47.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1331 fp=#0x100000035a3' 2024-09-10T08:44:47.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1363 fp=#0x1000000358a' 2024-09-10T08:44:47.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1491 fp=#0x10000003299' 2024-09-10T08:44:47.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1651 fp=#0x100000045d3' 2024-09-10T08:44:47.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1747 fp=#0x1000000337f' 2024-09-10T08:44:47.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2003 fp=#0x1000000274d' 2024-09-10T08:44:47.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2227 fp=#0x1000000269b' 2024-09-10T08:44:47.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2291 fp=#0x1000000272a' 2024-09-10T08:44:47.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2579 fp=#0x10000001035' 2024-09-10T08:44:47.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2611 fp=#0x10000001059' 2024-09-10T08:44:47.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2643 fp=#0x100000018a1' 2024-09-10T08:44:47.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2675 fp=#0x10000001864' 2024-09-10T08:44:47.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2707 fp=#0x100000018ee' 2024-09-10T08:44:47.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2739 fp=#0x100000010bd' 2024-09-10T08:44:47.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2771 fp=#0x10000003694' 2024-09-10T08:44:47.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2803 fp=#0x1000000361f' 2024-09-10T08:44:47.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2835 fp=#0x1000000476b' 2024-09-10T08:44:47.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2867 fp=#0x10000004827' 2024-09-10T08:44:47.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2899 fp=#0x1000000463e' 2024-09-10T08:44:47.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2931 fp=#0x10000003643' 2024-09-10T08:44:47.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2963 fp=#0x100000033cd' 2024-09-10T08:44:47.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2995 fp=#0x10000000dee' 2024-09-10T08:44:47.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3027 fp=#0x100000049b3' 2024-09-10T08:44:47.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3059 fp=#0x100000049d5' 2024-09-10T08:44:47.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3091 fp=#0x1000000114c' 2024-09-10T08:44:47.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3123 fp=#0x10000001902' 2024-09-10T08:44:47.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3155 fp=#0x1000000191b' 2024-09-10T08:44:47.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3187 fp=#0x10000000dd9' 2024-09-10T08:44:47.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3251 fp=#0x10000003c14' 2024-09-10T08:44:47.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3283 fp=#0x10000000749' 2024-09-10T08:44:47.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3315 fp=#0x10000000711' 2024-09-10T08:44:47.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3347 fp=#0x10000003c89' 2024-09-10T08:44:47.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3379 fp=#0x1000000075a' 2024-09-10T08:44:47.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3411 fp=#0x10000003cdd' 2024-09-10T08:44:47.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3443 fp=#0x10000003ca6' 2024-09-10T08:44:47.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3539 fp=#0x10000003cbc' 2024-09-10T08:44:47.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3603 fp=#0x10000004120' 2024-09-10T08:44:47.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3635 fp=#0x10000004c77' 2024-09-10T08:44:47.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3667 fp=#0x10000004b99' 2024-09-10T08:44:47.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3699 fp=#0x10000004b0c' 2024-09-10T08:44:47.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3731 fp=#0x100000027ae' 2024-09-10T08:44:47.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3763 fp=#0x10000002708' 2024-09-10T08:44:47.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3795 fp=#0x10000003f21' 2024-09-10T08:44:47.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3827 fp=#0x10000003f83' 2024-09-10T08:44:47.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3859 fp=#0x10000002530' 2024-09-10T08:44:47.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3891 fp=#0x100000025bb' 2024-09-10T08:44:47.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3923 fp=#0x10000001419' 2024-09-10T08:44:47.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3955 fp=#0x1000000052e' 2024-09-10T08:44:47.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3987 fp=#0x10000000577' 2024-09-10T08:44:47.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4019 fp=#0x10000000a77' 2024-09-10T08:44:47.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4051 fp=#0x10000000f4e' 2024-09-10T08:44:47.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4083 fp=#0x10000000fad' 2024-09-10T08:44:47.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4115 fp=#0x10000004654' 2024-09-10T08:44:47.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4147 fp=#0x1000000464c' 2024-09-10T08:44:47.402 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4179 fp=#0x10000004ec3' 2024-09-10T08:44:47.402 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4211 fp=#0x10000004659' 2024-09-10T08:44:47.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4243 fp=#0x10000004098' 2024-09-10T08:44:47.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4275 fp=#0x10000003b50' 2024-09-10T08:44:47.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4307 fp=#0x100000040e1' 2024-09-10T08:44:47.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4339 fp=#0x1000000408b' 2024-09-10T08:44:47.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4371 fp=#0x1000000202e' 2024-09-10T08:44:47.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4403 fp=#0x10000001fab' 2024-09-10T08:44:47.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4467 fp=#0x10000001fa7' 2024-09-10T08:44:47.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4499 fp=#0x100000050a6' 2024-09-10T08:44:47.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4531 fp=#0x1000000405b' 2024-09-10T08:44:47.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4563 fp=#0x10000005122' 2024-09-10T08:44:47.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4595 fp=#0x10000002702' 2024-09-10T08:44:47.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4627 fp=#0x1000000135d' 2024-09-10T08:44:47.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4659 fp=#0x100000012d0' 2024-09-10T08:44:47.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4691 fp=#0x10000001427' 2024-09-10T08:44:47.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4723 fp=#0x100000026d3' 2024-09-10T08:44:47.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4755 fp=#0x10000001279' 2024-09-10T08:44:47.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4787 fp=#0x10000003833' 2024-09-10T08:44:47.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4819 fp=#0x10000004a88' 2024-09-10T08:44:47.410 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4851 fp=#0x10000004a4c' 2024-09-10T08:44:47.410 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4883 fp=#0x10000004a50' 2024-09-10T08:44:47.410 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4915 fp=#0x10000004ac8' 2024-09-10T08:44:47.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4947 fp=#0x100000037e0' 2024-09-10T08:44:47.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4979 fp=#0x10000005129' 2024-09-10T08:44:47.412 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5011 fp=#0x100000050fa' 2024-09-10T08:44:47.412 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5043 fp=#0x10000004ffe' 2024-09-10T08:44:47.412 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5075 fp=#0x1000000511a' 2024-09-10T08:44:47.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5107 fp=#0x100000051cb' 2024-09-10T08:44:47.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5139 fp=#0x10000001ed0' 2024-09-10T08:44:47.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5171 fp=#0x100000020ac' 2024-09-10T08:44:47.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5203 fp=#0x10000001ce3' 2024-09-10T08:44:47.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5235 fp=#0x10000001bcb' 2024-09-10T08:44:47.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5267 fp=#0x1000000360a' 2024-09-10T08:44:47.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5299 fp=#0x10000003709' 2024-09-10T08:44:47.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5331 fp=#0x10000003665' 2024-09-10T08:44:47.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5363 fp=#0x100000036c5' 2024-09-10T08:44:47.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5395 fp=#0x1000000368a' 2024-09-10T08:44:47.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5427 fp=#0x10000004a8d' 2024-09-10T08:44:47.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5459 fp=#0x100000014f3' 2024-09-10T08:44:47.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5491 fp=#0x1000000122e' 2024-09-10T08:44:47.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5523 fp=#0x100000017d2' 2024-09-10T08:44:47.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5587 fp=#0x10000000f4a' 2024-09-10T08:44:47.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5619 fp=#0x10000001086' 2024-09-10T08:44:47.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5651 fp=#0x100000020d4' 2024-09-10T08:44:47.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5683 fp=#0x10000000ff1' 2024-09-10T08:44:47.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5715 fp=#0x100000047df' 2024-09-10T08:44:47.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5747 fp=#0x100000020e7' 2024-09-10T08:44:47.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5779 fp=#0x1000000212e' 2024-09-10T08:44:47.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5811 fp=#0x10000004b05' 2024-09-10T08:44:47.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5843 fp=#0x1000000442b' 2024-09-10T08:44:47.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5875 fp=#0x10000004d6d' 2024-09-10T08:44:47.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5907 fp=#0x100000043d4' 2024-09-10T08:44:47.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5939 fp=#0x10000004954' 2024-09-10T08:44:47.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5971 fp=#0x10000004d8a' 2024-09-10T08:44:47.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6099 fp=#0x10000003492' 2024-09-10T08:44:47.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6131 fp=#0x10000003620' 2024-09-10T08:44:47.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6227 fp=#0x10000003575' 2024-09-10T08:44:47.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6259 fp=#0x10000003569' 2024-09-10T08:44:47.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6323 fp=#0x1000000437c' 2024-09-10T08:44:47.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6355 fp=#0x10000001ed3' 2024-09-10T08:44:47.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6387 fp=#0x10000003369' 2024-09-10T08:44:47.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6419 fp=#0x10000005348' 2024-09-10T08:44:47.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6451 fp=#0x10000004b51' 2024-09-10T08:44:47.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6483 fp=#0x10000004d8b' 2024-09-10T08:44:47.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6515 fp=#0x10000005343' 2024-09-10T08:44:47.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6547 fp=#0x10000004e57' 2024-09-10T08:44:47.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6611 fp=#0x100000044da' 2024-09-10T08:44:47.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6643 fp=#0x10000005028' 2024-09-10T08:44:47.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6675 fp=#0x100000049ad' 2024-09-10T08:44:47.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6707 fp=#0x100000039cc' 2024-09-10T08:44:47.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6739 fp=#0x10000004f51' 2024-09-10T08:44:47.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6771 fp=#0x10000003b4c' 2024-09-10T08:44:47.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6803 fp=#0x1000000465e' 2024-09-10T08:44:47.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6835 fp=#0x1000000460e' 2024-09-10T08:44:47.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6867 fp=#0x100000045f7' 2024-09-10T08:44:47.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6899 fp=#0x10000004731' 2024-09-10T08:44:47.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6931 fp=#0x10000005331' 2024-09-10T08:44:47.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6963 fp=#0x100000005f6' 2024-09-10T08:44:47.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6995 fp=#0x10000004f6a' 2024-09-10T08:44:47.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7027 fp=#0x1000000081e' 2024-09-10T08:44:47.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7059 fp=#0x10000004328' 2024-09-10T08:44:47.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7091 fp=#0x10000000660' 2024-09-10T08:44:47.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7123 fp=#0x1000000067b' 2024-09-10T08:44:47.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7155 fp=#0x10000000684' 2024-09-10T08:44:47.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7187 fp=#0x100000005d2' 2024-09-10T08:44:47.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7283 fp=#0x10000005086' 2024-09-10T08:44:47.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7347 fp=#0x10000005043' 2024-09-10T08:44:47.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7379 fp=#0x10000005151' 2024-09-10T08:44:47.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7411 fp=#0x100000024e2' 2024-09-10T08:44:47.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7443 fp=#0x10000004748' 2024-09-10T08:44:47.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7475 fp=#0x10000004863' 2024-09-10T08:44:47.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7507 fp=#0x100000046a5' 2024-09-10T08:44:47.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7539 fp=#0x10000001321' 2024-09-10T08:44:47.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7571 fp=#0x10000001ee6' 2024-09-10T08:44:47.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7603 fp=#0x100000051ac' 2024-09-10T08:44:47.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7699 fp=#0x10000001372' 2024-09-10T08:44:47.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7731 fp=#0x10000004e2f' 2024-09-10T08:44:47.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7763 fp=#0x10000004f50' 2024-09-10T08:44:47.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7795 fp=#0x10000004f1e' 2024-09-10T08:44:47.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7827 fp=#0x1000000171d' 2024-09-10T08:44:47.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7859 fp=#0x100000013d5' 2024-09-10T08:44:47.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7891 fp=#0x1000000147d' 2024-09-10T08:44:47.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7923 fp=#0x10000001524' 2024-09-10T08:44:47.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7955 fp=#0x100000016ee' 2024-09-10T08:44:47.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7987 fp=#0x100000014d1' 2024-09-10T08:44:47.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8019 fp=#0x10000004cf1' 2024-09-10T08:44:47.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8051 fp=#0x100000015d3' 2024-09-10T08:44:47.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8083 fp=#0x10000003fdb' 2024-09-10T08:44:47.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8115 fp=#0x100000016ab' 2024-09-10T08:44:47.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8147 fp=#0x1000000270f' 2024-09-10T08:44:47.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8179 fp=#0x10000003de0' 2024-09-10T08:44:47.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8243 fp=#0x1000000419e' 2024-09-10T08:44:47.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8275 fp=#0x10000004403' 2024-09-10T08:44:47.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8307 fp=#0x100000046c9' 2024-09-10T08:44:47.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8339 fp=#0x100000041a7' 2024-09-10T08:44:47.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8371 fp=#0x100000043e5' 2024-09-10T08:44:47.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8403 fp=#0x100000040d6' 2024-09-10T08:44:47.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8435 fp=#0x1000000415b' 2024-09-10T08:44:47.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8467 fp=#0x10000004b57' 2024-09-10T08:44:47.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8499 fp=#0x10000004279' 2024-09-10T08:44:47.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8531 fp=#0x10000003e4d' 2024-09-10T08:44:47.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8563 fp=#0x10000003e6b' 2024-09-10T08:44:47.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8595 fp=#0x10000003e84' 2024-09-10T08:44:47.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8627 fp=#0x1000000403e' 2024-09-10T08:44:47.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8659 fp=#0x10000004f57' 2024-09-10T08:44:47.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8691 fp=#0x10000003d8d' 2024-09-10T08:44:47.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8723 fp=#0x10000004fad' 2024-09-10T08:44:47.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8755 fp=#0x10000004f2b' 2024-09-10T08:44:47.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8787 fp=#0x10000003fe6' 2024-09-10T08:44:47.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8819 fp=#0x10000004056' 2024-09-10T08:44:47.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8851 fp=#0x10000001d7c' 2024-09-10T08:44:47.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8883 fp=#0x10000001d00' 2024-09-10T08:44:47.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8915 fp=#0x10000004457' 2024-09-10T08:44:47.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8947 fp=#0x10000001c87' 2024-09-10T08:44:47.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8979 fp=#0x10000001ba2' 2024-09-10T08:44:47.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9011 fp=#0x10000003c33' 2024-09-10T08:44:47.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9043 fp=#0x10000003280' 2024-09-10T08:44:47.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9075 fp=#0x10000000d7c' 2024-09-10T08:44:47.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9107 fp=#0x10000000d56' 2024-09-10T08:44:47.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9139 fp=#0x100000041c8' 2024-09-10T08:44:47.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9171 fp=#0x10000000da6' 2024-09-10T08:44:47.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9203 fp=#0x10000000d64' 2024-09-10T08:44:47.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9235 fp=#0x10000000ce0' 2024-09-10T08:44:47.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9267 fp=#0x10000000c36' 2024-09-10T08:44:47.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9331 fp=#0x10000002601' 2024-09-10T08:44:47.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9363 fp=#0x10000004e10' 2024-09-10T08:44:47.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9395 fp=#0x10000000b0b' 2024-09-10T08:44:47.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9427 fp=#0x10000000b50' 2024-09-10T08:44:47.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9459 fp=#0x10000000b65' 2024-09-10T08:44:47.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9491 fp=#0x1000000498b' 2024-09-10T08:44:47.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9523 fp=#0x10000001c7c' 2024-09-10T08:44:47.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9555 fp=#0x10000001ebf' 2024-09-10T08:44:47.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9587 fp=#0x10000001a4b' 2024-09-10T08:44:47.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9619 fp=#0x10000001a12' 2024-09-10T08:44:47.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9651 fp=#0x10000004807' 2024-09-10T08:44:47.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9683 fp=#0x100000048c4' 2024-09-10T08:44:47.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9715 fp=#0x10000001a5b' 2024-09-10T08:44:47.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9747 fp=#0x10000001a6a' 2024-09-10T08:44:47.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9779 fp=#0x10000004cc9' 2024-09-10T08:44:47.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9811 fp=#0x10000004c9f' 2024-09-10T08:44:47.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9843 fp=#0x1000000438b' 2024-09-10T08:44:47.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9875 fp=#0x10000001a3f' 2024-09-10T08:44:47.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9907 fp=#0x100000015be' 2024-09-10T08:44:47.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9939 fp=#0x100000015b1' 2024-09-10T08:44:47.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9971 fp=#0x100000016cd' 2024-09-10T08:44:47.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10003 fp=#0x10000001634' 2024-09-10T08:44:47.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10035 fp=#0x100000016be' 2024-09-10T08:44:47.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10067 fp=#0x10000001954' 2024-09-10T08:44:47.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10099 fp=#0x10000001f20' 2024-09-10T08:44:47.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10131 fp=#0x1000000394d' 2024-09-10T08:44:47.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10163 fp=#0x10000003925' 2024-09-10T08:44:47.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10195 fp=#0x10000003b66' 2024-09-10T08:44:47.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10227 fp=#0x10000003a35' 2024-09-10T08:44:47.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10259 fp=#0x10000001dee' 2024-09-10T08:44:47.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10291 fp=#0x10000002366' 2024-09-10T08:44:47.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10323 fp=#0x1000000371b' 2024-09-10T08:44:47.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10387 fp=#0x100000012ed' 2024-09-10T08:44:47.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10419 fp=#0x1000000391a' 2024-09-10T08:44:47.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10451 fp=#0x10000001340' 2024-09-10T08:44:47.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10483 fp=#0x100000038f7' 2024-09-10T08:44:47.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10515 fp=#0x10000003921' 2024-09-10T08:44:47.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10547 fp=#0x10000001378' 2024-09-10T08:44:47.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10643 fp=#0x10000004851' 2024-09-10T08:44:47.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10675 fp=#0x10000004916' 2024-09-10T08:44:47.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10771 fp=#0x10000003b91' 2024-09-10T08:44:47.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10803 fp=#0x10000000572' 2024-09-10T08:44:47.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10835 fp=#0x10000001d25' 2024-09-10T08:44:47.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10867 fp=#0x10000000a20' 2024-09-10T08:44:47.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10899 fp=#0x10000001ce6' 2024-09-10T08:44:47.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10931 fp=#0x100000006cc' 2024-09-10T08:44:47.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10963 fp=#0x10000000633' 2024-09-10T08:44:47.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11027 fp=#0x10000003aa3' 2024-09-10T08:44:47.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11059 fp=#0x10000005076' 2024-09-10T08:44:47.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11091 fp=#0x100000038a1' 2024-09-10T08:44:47.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11123 fp=#0x10000002395' 2024-09-10T08:44:47.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11155 fp=#0x1000000205c' 2024-09-10T08:44:47.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11187 fp=#0x10000003fa3' 2024-09-10T08:44:47.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11219 fp=#0x10000003e5d' 2024-09-10T08:44:47.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11251 fp=#0x10000003f46' 2024-09-10T08:44:47.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11283 fp=#0x1000000492c' 2024-09-10T08:44:47.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11315 fp=#0x10000001a08' 2024-09-10T08:44:47.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11347 fp=#0x10000001be1' 2024-09-10T08:44:47.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11379 fp=#0x10000001af1' 2024-09-10T08:44:47.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11443 fp=#0x10000001ac2' 2024-09-10T08:44:47.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11475 fp=#0x10000001a30' 2024-09-10T08:44:47.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11507 fp=#0x10000001acf' 2024-09-10T08:44:47.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11539 fp=#0x100000047bf' 2024-09-10T08:44:47.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11571 fp=#0x10000005178' 2024-09-10T08:44:47.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11603 fp=#0x100000051ae' 2024-09-10T08:44:47.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11667 fp=#0x100000047c6' 2024-09-10T08:44:47.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11699 fp=#0x100000009c3' 2024-09-10T08:44:47.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11731 fp=#0x10000004226' 2024-09-10T08:44:47.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11763 fp=#0x100000041c1' 2024-09-10T08:44:47.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11795 fp=#0x10000000d0e' 2024-09-10T08:44:47.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11827 fp=#0x10000003ea9' 2024-09-10T08:44:47.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11859 fp=#0x10000001057' 2024-09-10T08:44:47.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11891 fp=#0x1000000427c' 2024-09-10T08:44:47.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11923 fp=#0x1000000113c' 2024-09-10T08:44:47.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11955 fp=#0x10000000f1d' 2024-09-10T08:44:47.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11987 fp=#0x100000052df' 2024-09-10T08:44:47.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12019 fp=#0x10000000ef6' 2024-09-10T08:44:47.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12083 fp=#0x100000034d5' 2024-09-10T08:44:47.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12115 fp=#0x100000033a8' 2024-09-10T08:44:47.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12147 fp=#0x100000033e6' 2024-09-10T08:44:47.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12179 fp=#0x100000043d8' 2024-09-10T08:44:47.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12211 fp=#0x100000040a7' 2024-09-10T08:44:47.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12243 fp=#0x10000002570' 2024-09-10T08:44:47.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12275 fp=#0x10000004ce4' 2024-09-10T08:44:47.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12307 fp=#0x100000051ea' 2024-09-10T08:44:47.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12339 fp=#0x100000051f3' 2024-09-10T08:44:47.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12371 fp=#0x10000000ce4' 2024-09-10T08:44:47.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12403 fp=#0x10000000d41' 2024-09-10T08:44:47.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12435 fp=#0x10000000c3e' 2024-09-10T08:44:47.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12467 fp=#0x1000000267c' 2024-09-10T08:44:47.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12531 fp=#0x10000004bb3' 2024-09-10T08:44:47.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12563 fp=#0x10000004ae1' 2024-09-10T08:44:47.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12595 fp=#0x10000004c7e' 2024-09-10T08:44:47.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12627 fp=#0x10000004c04' 2024-09-10T08:44:47.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12659 fp=#0x100000049d6' 2024-09-10T08:44:47.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12691 fp=#0x1000000539a' 2024-09-10T08:44:47.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12723 fp=#0x1000000495d' 2024-09-10T08:44:47.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12755 fp=#0x1000000493f' 2024-09-10T08:44:47.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12787 fp=#0x10000002773' 2024-09-10T08:44:47.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12819 fp=#0x100000021db' 2024-09-10T08:44:47.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12851 fp=#0x100000053f3' 2024-09-10T08:44:47.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12883 fp=#0x10000002123' 2024-09-10T08:44:47.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12915 fp=#0x100000053c8' 2024-09-10T08:44:47.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12947 fp=#0x10000002731' 2024-09-10T08:44:47.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13011 fp=#0x1000000274b' 2024-09-10T08:44:47.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13035 fp=#0x100000024e9' 2024-09-10T08:44:47.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13067 fp=#0x100000053f8' 2024-09-10T08:44:47.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13099 fp=#0x100000039dd' 2024-09-10T08:44:47.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13131 fp=#0x100000053dc' 2024-09-10T08:44:47.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13163 fp=#0x10000003a4c' 2024-09-10T08:44:47.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:252 fp=#0x100000053d8' 2024-09-10T08:44:47.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:284 fp=#0x10000001217' 2024-09-10T08:44:47.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:316 fp=#0x10000001568' 2024-09-10T08:44:47.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:436 fp=#0x10000004b17' 2024-09-10T08:44:47.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:468 fp=#0x1000000467a' 2024-09-10T08:44:47.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:532 fp=#0x100000032ab' 2024-09-10T08:44:47.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:564 fp=#0x100000032a3' 2024-09-10T08:44:47.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:596 fp=#0x1000000342b' 2024-09-10T08:44:47.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:788 fp=#0x10000000dcd' 2024-09-10T08:44:47.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:852 fp=#0x1000000440e' 2024-09-10T08:44:47.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:884 fp=#0x1000000453d' 2024-09-10T08:44:47.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:916 fp=#0x10000004025' 2024-09-10T08:44:47.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1012 fp=#0x10000000de1' 2024-09-10T08:44:47.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1108 fp=#0x1000000424e' 2024-09-10T08:44:47.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1172 fp=#0x10000004297' 2024-09-10T08:44:47.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1236 fp=#0x100000020b0' 2024-09-10T08:44:47.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1268 fp=#0x100000035cb' 2024-09-10T08:44:47.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1300 fp=#0x10000001fbf' 2024-09-10T08:44:47.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1652 fp=#0x100000045cd' 2024-09-10T08:44:47.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2036 fp=#0x1000000216c' 2024-09-10T08:44:47.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2068 fp=#0x1000000212f' 2024-09-10T08:44:47.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2228 fp=#0x100000026c3' 2024-09-10T08:44:47.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2260 fp=#0x10000002624' 2024-09-10T08:44:47.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2292 fp=#0x100000025c0' 2024-09-10T08:44:47.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2580 fp=#0x100000011fc' 2024-09-10T08:44:47.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2612 fp=#0x1000000113f' 2024-09-10T08:44:47.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2644 fp=#0x1000000184c' 2024-09-10T08:44:47.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2676 fp=#0x10000001070' 2024-09-10T08:44:47.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2740 fp=#0x10000001101' 2024-09-10T08:44:47.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2772 fp=#0x100000035d7' 2024-09-10T08:44:47.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2804 fp=#0x10000004744' 2024-09-10T08:44:47.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2836 fp=#0x1000000344a' 2024-09-10T08:44:47.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2900 fp=#0x10000003638' 2024-09-10T08:44:47.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2932 fp=#0x100000033e3' 2024-09-10T08:44:47.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2964 fp=#0x100000033dc' 2024-09-10T08:44:47.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2996 fp=#0x10000000e94' 2024-09-10T08:44:47.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3028 fp=#0x10000001914' 2024-09-10T08:44:47.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3060 fp=#0x10000000e0a' 2024-09-10T08:44:47.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3092 fp=#0x10000004a57' 2024-09-10T08:44:47.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3124 fp=#0x10000000e6f' 2024-09-10T08:44:47.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3156 fp=#0x10000001969' 2024-09-10T08:44:47.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3188 fp=#0x10000000e0c' 2024-09-10T08:44:47.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3220 fp=#0x10000000742' 2024-09-10T08:44:47.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3252 fp=#0x10000000746' 2024-09-10T08:44:47.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3284 fp=#0x1000000073c' 2024-09-10T08:44:47.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3316 fp=#0x10000003c6c' 2024-09-10T08:44:47.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3348 fp=#0x10000003b9d' 2024-09-10T08:44:47.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3380 fp=#0x1000000264c' 2024-09-10T08:44:47.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3412 fp=#0x100000006f0' 2024-09-10T08:44:47.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3476 fp=#0x1000000404d' 2024-09-10T08:44:47.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3540 fp=#0x10000004158' 2024-09-10T08:44:47.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3604 fp=#0x100000040e3' 2024-09-10T08:44:47.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3636 fp=#0x10000004b4f' 2024-09-10T08:44:47.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3668 fp=#0x10000004b63' 2024-09-10T08:44:47.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3700 fp=#0x10000004b5d' 2024-09-10T08:44:47.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3764 fp=#0x1000000259d' 2024-09-10T08:44:47.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3796 fp=#0x100000024ef' 2024-09-10T08:44:47.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3828 fp=#0x10000003eb8' 2024-09-10T08:44:47.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3860 fp=#0x10000002631' 2024-09-10T08:44:47.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3892 fp=#0x10000003f75' 2024-09-10T08:44:47.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3924 fp=#0x100000027ab' 2024-09-10T08:44:47.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4020 fp=#0x10000000549' 2024-09-10T08:44:47.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4052 fp=#0x10000004793' 2024-09-10T08:44:47.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4084 fp=#0x10000004641' 2024-09-10T08:44:47.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4116 fp=#0x10000000f4c' 2024-09-10T08:44:47.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4148 fp=#0x10000004e8c' 2024-09-10T08:44:47.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4180 fp=#0x10000000fb2' 2024-09-10T08:44:47.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4244 fp=#0x10000003be3' 2024-09-10T08:44:47.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4276 fp=#0x10000003b7f' 2024-09-10T08:44:47.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4308 fp=#0x10000003aa2' 2024-09-10T08:44:47.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4340 fp=#0x10000003af3' 2024-09-10T08:44:47.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4372 fp=#0x10000001fec' 2024-09-10T08:44:47.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4436 fp=#0x10000004d07' 2024-09-10T08:44:47.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4468 fp=#0x10000001f00' 2024-09-10T08:44:47.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4500 fp=#0x10000001239' 2024-09-10T08:44:47.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4532 fp=#0x1000000512c' 2024-09-10T08:44:47.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4564 fp=#0x10000005104' 2024-09-10T08:44:47.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4596 fp=#0x10000001588' 2024-09-10T08:44:47.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4628 fp=#0x10000003358' 2024-09-10T08:44:47.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4660 fp=#0x1000000134b' 2024-09-10T08:44:47.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4692 fp=#0x10000001489' 2024-09-10T08:44:47.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4724 fp=#0x10000004df7' 2024-09-10T08:44:47.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4756 fp=#0x100000039ca' 2024-09-10T08:44:47.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4788 fp=#0x1000000143c' 2024-09-10T08:44:47.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4820 fp=#0x10000003767' 2024-09-10T08:44:47.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4852 fp=#0x100000036db' 2024-09-10T08:44:47.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4884 fp=#0x10000004a6a' 2024-09-10T08:44:47.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4916 fp=#0x100000037e6' 2024-09-10T08:44:47.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4948 fp=#0x100000037ab' 2024-09-10T08:44:47.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4980 fp=#0x1000000508d' 2024-09-10T08:44:47.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5012 fp=#0x10000005071' 2024-09-10T08:44:47.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5044 fp=#0x10000005155' 2024-09-10T08:44:47.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5076 fp=#0x100000050d1' 2024-09-10T08:44:47.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5108 fp=#0x10000005198' 2024-09-10T08:44:47.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5140 fp=#0x10000001c81' 2024-09-10T08:44:47.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5172 fp=#0x10000001ba5' 2024-09-10T08:44:47.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5204 fp=#0x10000001b8c' 2024-09-10T08:44:47.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5236 fp=#0x10000001e48' 2024-09-10T08:44:47.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5268 fp=#0x10000003671' 2024-09-10T08:44:47.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5300 fp=#0x1000000376e' 2024-09-10T08:44:47.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5332 fp=#0x10000003692' 2024-09-10T08:44:47.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5364 fp=#0x10000003d83' 2024-09-10T08:44:47.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5396 fp=#0x10000004ac2' 2024-09-10T08:44:47.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5428 fp=#0x100000035f0' 2024-09-10T08:44:47.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5460 fp=#0x100000012a8' 2024-09-10T08:44:47.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5492 fp=#0x10000001244' 2024-09-10T08:44:47.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5588 fp=#0x10000000f77' 2024-09-10T08:44:47.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5620 fp=#0x10000001037' 2024-09-10T08:44:47.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5652 fp=#0x1000000105e' 2024-09-10T08:44:47.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5684 fp=#0x10000000f93' 2024-09-10T08:44:47.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5716 fp=#0x10000000fdf' 2024-09-10T08:44:47.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5748 fp=#0x10000002122' 2024-09-10T08:44:47.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5812 fp=#0x100000044e9' 2024-09-10T08:44:47.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5844 fp=#0x10000004626' 2024-09-10T08:44:47.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5876 fp=#0x10000004327' 2024-09-10T08:44:47.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5908 fp=#0x10000004d68' 2024-09-10T08:44:47.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5940 fp=#0x100000042fd' 2024-09-10T08:44:47.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5972 fp=#0x10000004d26' 2024-09-10T08:44:47.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6068 fp=#0x100000035a9' 2024-09-10T08:44:47.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6164 fp=#0x10000003556' 2024-09-10T08:44:47.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6228 fp=#0x10000003590' 2024-09-10T08:44:47.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6260 fp=#0x100000034c7' 2024-09-10T08:44:47.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6292 fp=#0x10000003355' 2024-09-10T08:44:47.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6324 fp=#0x10000003380' 2024-09-10T08:44:47.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6356 fp=#0x10000001df0' 2024-09-10T08:44:47.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6388 fp=#0x10000004002' 2024-09-10T08:44:47.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6420 fp=#0x10000004d67' 2024-09-10T08:44:47.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6452 fp=#0x10000004ac5' 2024-09-10T08:44:47.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6484 fp=#0x10000004cf9' 2024-09-10T08:44:47.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6516 fp=#0x10000004ad6' 2024-09-10T08:44:47.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6548 fp=#0x10000004bd3' 2024-09-10T08:44:47.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6612 fp=#0x10000003c06' 2024-09-10T08:44:47.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6644 fp=#0x10000004fa8' 2024-09-10T08:44:47.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6676 fp=#0x10000003d5a' 2024-09-10T08:44:47.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6708 fp=#0x10000004f64' 2024-09-10T08:44:47.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6740 fp=#0x10000003b34' 2024-09-10T08:44:47.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6772 fp=#0x10000004fee' 2024-09-10T08:44:47.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6804 fp=#0x10000004644' 2024-09-10T08:44:47.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6836 fp=#0x1000000478c' 2024-09-10T08:44:47.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6868 fp=#0x10000004619' 2024-09-10T08:44:47.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6900 fp=#0x10000004710' 2024-09-10T08:44:47.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6932 fp=#0x1000000530c' 2024-09-10T08:44:47.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6964 fp=#0x10000004ffb' 2024-09-10T08:44:47.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6996 fp=#0x1000000063a' 2024-09-10T08:44:47.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7028 fp=#0x10000004a95' 2024-09-10T08:44:47.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7060 fp=#0x1000000507d' 2024-09-10T08:44:47.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7092 fp=#0x100000007f7' 2024-09-10T08:44:47.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7124 fp=#0x10000004f90' 2024-09-10T08:44:47.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7156 fp=#0x10000000b23' 2024-09-10T08:44:47.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7188 fp=#0x10000000676' 2024-09-10T08:44:47.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7284 fp=#0x1000000510b' 2024-09-10T08:44:47.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7380 fp=#0x10000005106' 2024-09-10T08:44:47.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7412 fp=#0x100000048d7' 2024-09-10T08:44:47.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7444 fp=#0x1000000470d' 2024-09-10T08:44:47.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7476 fp=#0x10000004786' 2024-09-10T08:44:47.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7508 fp=#0x1000000234d' 2024-09-10T08:44:47.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7540 fp=#0x10000001390' 2024-09-10T08:44:47.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7572 fp=#0x100000019fe' 2024-09-10T08:44:47.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7604 fp=#0x100000051dd' 2024-09-10T08:44:47.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7636 fp=#0x1000000131a' 2024-09-10T08:44:47.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7700 fp=#0x10000001fed' 2024-09-10T08:44:47.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7732 fp=#0x10000004ee6' 2024-09-10T08:44:47.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7764 fp=#0x10000004f81' 2024-09-10T08:44:47.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7796 fp=#0x100000016d1' 2024-09-10T08:44:47.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7828 fp=#0x10000001895' 2024-09-10T08:44:47.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7860 fp=#0x10000004d5f' 2024-09-10T08:44:47.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7892 fp=#0x10000004cc5' 2024-09-10T08:44:47.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7924 fp=#0x10000001475' 2024-09-10T08:44:47.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7956 fp=#0x10000001550' 2024-09-10T08:44:47.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7988 fp=#0x100000016cc' 2024-09-10T08:44:47.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8020 fp=#0x10000004e3d' 2024-09-10T08:44:47.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8052 fp=#0x10000001d04' 2024-09-10T08:44:47.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8084 fp=#0x10000001600' 2024-09-10T08:44:47.616 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8116 fp=#0x10000001691' 2024-09-10T08:44:47.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8180 fp=#0x1000000271d' 2024-09-10T08:44:47.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8212 fp=#0x100000025d4' 2024-09-10T08:44:47.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8244 fp=#0x1000000466a' 2024-09-10T08:44:47.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8276 fp=#0x10000004267' 2024-09-10T08:44:47.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8308 fp=#0x10000004612' 2024-09-10T08:44:47.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8340 fp=#0x100000041ee' 2024-09-10T08:44:47.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8372 fp=#0x100000044e5' 2024-09-10T08:44:47.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8404 fp=#0x100000043de' 2024-09-10T08:44:47.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8436 fp=#0x1000000464e' 2024-09-10T08:44:47.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8468 fp=#0x10000003e74' 2024-09-10T08:44:47.620 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8500 fp=#0x10000003f56' 2024-09-10T08:44:47.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8532 fp=#0x100000040af' 2024-09-10T08:44:47.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8564 fp=#0x10000003ddb' 2024-09-10T08:44:47.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8596 fp=#0x10000004234' 2024-09-10T08:44:47.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8628 fp=#0x10000003c99' 2024-09-10T08:44:47.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8660 fp=#0x10000003d74' 2024-09-10T08:44:47.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8692 fp=#0x10000003d40' 2024-09-10T08:44:47.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8724 fp=#0x100000052e7' 2024-09-10T08:44:47.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8756 fp=#0x10000003cc6' 2024-09-10T08:44:47.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8788 fp=#0x10000003bbc' 2024-09-10T08:44:47.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8820 fp=#0x10000003bde' 2024-09-10T08:44:47.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8852 fp=#0x1000000446d' 2024-09-10T08:44:47.624 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8884 fp=#0x10000003c2c' 2024-09-10T08:44:47.625 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8916 fp=#0x10000001aa9' 2024-09-10T08:44:47.625 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8948 fp=#0x10000004345' 2024-09-10T08:44:47.625 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8980 fp=#0x100000043cd' 2024-09-10T08:44:47.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9012 fp=#0x10000001b94' 2024-09-10T08:44:47.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9044 fp=#0x10000000dc2' 2024-09-10T08:44:47.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9076 fp=#0x10000004097' 2024-09-10T08:44:47.627 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9108 fp=#0x10000004066' 2024-09-10T08:44:47.627 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9140 fp=#0x100000041f0' 2024-09-10T08:44:47.627 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9172 fp=#0x10000000d4a' 2024-09-10T08:44:47.628 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9204 fp=#0x10000001ea0' 2024-09-10T08:44:47.628 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9236 fp=#0x10000000d23' 2024-09-10T08:44:47.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9268 fp=#0x10000000b35' 2024-09-10T08:44:47.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9300 fp=#0x10000004d7e' 2024-09-10T08:44:47.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9332 fp=#0x10000000b61' 2024-09-10T08:44:47.630 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9364 fp=#0x10000000c5b' 2024-09-10T08:44:47.630 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9396 fp=#0x10000000aac' 2024-09-10T08:44:47.630 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9428 fp=#0x10000001c37' 2024-09-10T08:44:47.631 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9460 fp=#0x10000004e5a' 2024-09-10T08:44:47.631 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9492 fp=#0x1000000494f' 2024-09-10T08:44:47.631 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9524 fp=#0x100000019c1' 2024-09-10T08:44:47.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9556 fp=#0x10000001ac1' 2024-09-10T08:44:47.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9588 fp=#0x10000000ed8' 2024-09-10T08:44:47.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9620 fp=#0x10000000ed3' 2024-09-10T08:44:47.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9652 fp=#0x10000004922' 2024-09-10T08:44:47.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9684 fp=#0x100000047d5' 2024-09-10T08:44:47.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9716 fp=#0x10000004cc1' 2024-09-10T08:44:47.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9748 fp=#0x10000001793' 2024-09-10T08:44:47.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9780 fp=#0x10000001743' 2024-09-10T08:44:47.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9812 fp=#0x10000001b67' 2024-09-10T08:44:47.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9844 fp=#0x10000001a83' 2024-09-10T08:44:47.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9876 fp=#0x1000000436d' 2024-09-10T08:44:47.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9908 fp=#0x10000001a07' 2024-09-10T08:44:47.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9940 fp=#0x10000001660' 2024-09-10T08:44:47.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9972 fp=#0x10000002685' 2024-09-10T08:44:47.636 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10004 fp=#0x1000000156d' 2024-09-10T08:44:47.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10036 fp=#0x1000000165b' 2024-09-10T08:44:47.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10068 fp=#0x10000001871' 2024-09-10T08:44:47.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10100 fp=#0x10000003b4a' 2024-09-10T08:44:47.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10132 fp=#0x10000001f25' 2024-09-10T08:44:47.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10164 fp=#0x100000039e3' 2024-09-10T08:44:47.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10196 fp=#0x1000000390c' 2024-09-10T08:44:47.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10228 fp=#0x10000003a6c' 2024-09-10T08:44:47.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10260 fp=#0x10000003f39' 2024-09-10T08:44:47.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10292 fp=#0x10000002499' 2024-09-10T08:44:47.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10324 fp=#0x1000000245e' 2024-09-10T08:44:47.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10356 fp=#0x10000001dca' 2024-09-10T08:44:47.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10388 fp=#0x100000039fd' 2024-09-10T08:44:47.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10452 fp=#0x1000000134f' 2024-09-10T08:44:47.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10484 fp=#0x1000000142a' 2024-09-10T08:44:47.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10516 fp=#0x1000000397a' 2024-09-10T08:44:47.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10548 fp=#0x10000001415' 2024-09-10T08:44:47.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10612 fp=#0x10000000ddd' 2024-09-10T08:44:47.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10676 fp=#0x100000049d8' 2024-09-10T08:44:47.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10708 fp=#0x10000004a0a' 2024-09-10T08:44:47.644 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10740 fp=#0x10000004972' 2024-09-10T08:44:47.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10836 fp=#0x1000000063b' 2024-09-10T08:44:47.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10868 fp=#0x10000001a34' 2024-09-10T08:44:47.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10964 fp=#0x100000005bf' 2024-09-10T08:44:47.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11028 fp=#0x1000000391c' 2024-09-10T08:44:47.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11060 fp=#0x10000003982' 2024-09-10T08:44:47.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11092 fp=#0x1000000507f' 2024-09-10T08:44:47.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11124 fp=#0x1000000335d' 2024-09-10T08:44:47.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11156 fp=#0x10000002309' 2024-09-10T08:44:47.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11188 fp=#0x10000002081' 2024-09-10T08:44:47.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11220 fp=#0x10000002121' 2024-09-10T08:44:47.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11252 fp=#0x1000000204a' 2024-09-10T08:44:47.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11284 fp=#0x10000003fa6' 2024-09-10T08:44:47.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11316 fp=#0x100000019ed' 2024-09-10T08:44:47.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11348 fp=#0x10000001aea' 2024-09-10T08:44:47.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11380 fp=#0x10000001b06' 2024-09-10T08:44:47.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11412 fp=#0x10000001c47' 2024-09-10T08:44:47.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11444 fp=#0x10000001c6d' 2024-09-10T08:44:47.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11476 fp=#0x10000003dcf' 2024-09-10T08:44:47.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11508 fp=#0x10000001c40' 2024-09-10T08:44:47.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11604 fp=#0x10000005195' 2024-09-10T08:44:47.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11668 fp=#0x1000000519d' 2024-09-10T08:44:47.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11700 fp=#0x10000004e5c' 2024-09-10T08:44:47.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11732 fp=#0x100000008fe' 2024-09-10T08:44:47.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11764 fp=#0x1000000429b' 2024-09-10T08:44:47.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11796 fp=#0x10000004e3e' 2024-09-10T08:44:47.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11828 fp=#0x10000000946' 2024-09-10T08:44:47.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11860 fp=#0x100000009d2' 2024-09-10T08:44:47.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11892 fp=#0x100000041f7' 2024-09-10T08:44:47.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11924 fp=#0x10000001149' 2024-09-10T08:44:47.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11956 fp=#0x100000052b6' 2024-09-10T08:44:47.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11988 fp=#0x100000010da' 2024-09-10T08:44:47.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12052 fp=#0x10000005300' 2024-09-10T08:44:47.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12084 fp=#0x10000003564' 2024-09-10T08:44:47.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12116 fp=#0x1000000441f' 2024-09-10T08:44:47.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12148 fp=#0x10000003325' 2024-09-10T08:44:47.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12180 fp=#0x1000000443a' 2024-09-10T08:44:47.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12212 fp=#0x10000004473' 2024-09-10T08:44:47.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12244 fp=#0x10000000d7b' 2024-09-10T08:44:47.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12276 fp=#0x10000002565' 2024-09-10T08:44:47.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12308 fp=#0x100000051e7' 2024-09-10T08:44:47.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12340 fp=#0x10000000d77' 2024-09-10T08:44:47.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12372 fp=#0x10000000c6b' 2024-09-10T08:44:47.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12404 fp=#0x10000005302' 2024-09-10T08:44:47.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12436 fp=#0x100000051fa' 2024-09-10T08:44:47.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12468 fp=#0x1000000170b' 2024-09-10T08:44:47.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12532 fp=#0x10000004bdd' 2024-09-10T08:44:47.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12564 fp=#0x10000004c23' 2024-09-10T08:44:47.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12596 fp=#0x10000004c99' 2024-09-10T08:44:47.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12628 fp=#0x10000004bb6' 2024-09-10T08:44:47.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12660 fp=#0x100000049c7' 2024-09-10T08:44:47.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12692 fp=#0x1000000532d' 2024-09-10T08:44:47.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12724 fp=#0x10000004898' 2024-09-10T08:44:47.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12756 fp=#0x10000004867' 2024-09-10T08:44:47.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12788 fp=#0x10000002101' 2024-09-10T08:44:47.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12820 fp=#0x100000021eb' 2024-09-10T08:44:47.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12852 fp=#0x100000021ad' 2024-09-10T08:44:47.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12884 fp=#0x100000021ca' 2024-09-10T08:44:47.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12916 fp=#0x100000053a2' 2024-09-10T08:44:47.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12980 fp=#0x100000053eb' 2024-09-10T08:44:47.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13012 fp=#0x100000053d5' 2024-09-10T08:44:47.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13036 fp=#0x1000000232e' 2024-09-10T08:44:47.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13068 fp=#0x100000022bb' 2024-09-10T08:44:47.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13100 fp=#0x10000003a17' 2024-09-10T08:44:47.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13132 fp=#0x100000053ea' 2024-09-10T08:44:47.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13164 fp=#0x100000039cd' 2024-09-10T08:44:47.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:223 fp=#0x10000000399' 2024-09-10T08:44:47.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:253 fp=#0x10000002c85' 2024-09-10T08:44:47.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:317 fp=#0x10000001d76' 2024-09-10T08:44:47.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:373 fp=#0x10000004711' 2024-09-10T08:44:47.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:405 fp=#0x10000004aee' 2024-09-10T08:44:47.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:437 fp=#0x10000004aff' 2024-09-10T08:44:47.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:501 fp=#0x1000000352c' 2024-09-10T08:44:47.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:533 fp=#0x10000003270' 2024-09-10T08:44:47.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:597 fp=#0x10000004b5e' 2024-09-10T08:44:47.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:789 fp=#0x10000000ebd' 2024-09-10T08:44:47.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:885 fp=#0x10000000e75' 2024-09-10T08:44:47.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:917 fp=#0x10000000ec3' 2024-09-10T08:44:47.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:981 fp=#0x1000000409b' 2024-09-10T08:44:47.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1013 fp=#0x10000000e70' 2024-09-10T08:44:47.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1109 fp=#0x100000040f1' 2024-09-10T08:44:47.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1205 fp=#0x1000000421f' 2024-09-10T08:44:47.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1301 fp=#0x10000003683' 2024-09-10T08:44:47.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1333 fp=#0x10000001f8f' 2024-09-10T08:44:47.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1365 fp=#0x10000003630' 2024-09-10T08:44:47.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1557 fp=#0x10000003f3b' 2024-09-10T08:44:47.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1653 fp=#0x10000004616' 2024-09-10T08:44:47.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2037 fp=#0x10000002005' 2024-09-10T08:44:47.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2229 fp=#0x100000025b3' 2024-09-10T08:44:47.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2293 fp=#0x100000025e9' 2024-09-10T08:44:47.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2581 fp=#0x10000001093' 2024-09-10T08:44:47.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2613 fp=#0x100000039f4' 2024-09-10T08:44:47.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2677 fp=#0x1000000106e' 2024-09-10T08:44:47.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2709 fp=#0x100000018f6' 2024-09-10T08:44:47.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2741 fp=#0x10000002226' 2024-09-10T08:44:47.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2773 fp=#0x1000000369e' 2024-09-10T08:44:47.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2805 fp=#0x1000000465a' 2024-09-10T08:44:47.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2837 fp=#0x1000000469e' 2024-09-10T08:44:47.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2901 fp=#0x10000003446' 2024-09-10T08:44:47.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2933 fp=#0x10000004666' 2024-09-10T08:44:47.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2965 fp=#0x1000000344e' 2024-09-10T08:44:47.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2997 fp=#0x10000000e5e' 2024-09-10T08:44:47.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3029 fp=#0x10000000e5b' 2024-09-10T08:44:47.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3061 fp=#0x10000001950' 2024-09-10T08:44:47.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3125 fp=#0x10000000e08' 2024-09-10T08:44:47.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3157 fp=#0x10000001907' 2024-09-10T08:44:47.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3221 fp=#0x100000007e5' 2024-09-10T08:44:47.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3253 fp=#0x10000003ca3' 2024-09-10T08:44:47.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3285 fp=#0x10000003c97' 2024-09-10T08:44:47.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3317 fp=#0x10000003d19' 2024-09-10T08:44:47.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3349 fp=#0x1000000075e' 2024-09-10T08:44:47.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3381 fp=#0x1000000076a' 2024-09-10T08:44:47.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3413 fp=#0x1000000072e' 2024-09-10T08:44:47.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3477 fp=#0x10000003c27' 2024-09-10T08:44:47.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3509 fp=#0x10000004117' 2024-09-10T08:44:47.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3605 fp=#0x100000040ba' 2024-09-10T08:44:47.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3637 fp=#0x10000004caa' 2024-09-10T08:44:47.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3669 fp=#0x10000004b34' 2024-09-10T08:44:47.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3701 fp=#0x10000004b07' 2024-09-10T08:44:47.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3733 fp=#0x10000002553' 2024-09-10T08:44:47.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3765 fp=#0x10000002779' 2024-09-10T08:44:47.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3829 fp=#0x10000002648' 2024-09-10T08:44:47.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3861 fp=#0x100000026f8' 2024-09-10T08:44:47.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3893 fp=#0x100000027b3' 2024-09-10T08:44:47.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3925 fp=#0x100000006b6' 2024-09-10T08:44:47.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3957 fp=#0x10000000546' 2024-09-10T08:44:47.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3989 fp=#0x1000000069b' 2024-09-10T08:44:47.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4021 fp=#0x10000000597' 2024-09-10T08:44:47.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4053 fp=#0x10000004d5a' 2024-09-10T08:44:47.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4085 fp=#0x10000000f76' 2024-09-10T08:44:47.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4117 fp=#0x10000004e93' 2024-09-10T08:44:47.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4149 fp=#0x1000000119e' 2024-09-10T08:44:47.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4181 fp=#0x100000011a4' 2024-09-10T08:44:47.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4213 fp=#0x10000000fa9' 2024-09-10T08:44:47.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4245 fp=#0x10000003c96' 2024-09-10T08:44:47.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4277 fp=#0x10000004162' 2024-09-10T08:44:47.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4309 fp=#0x100000039ce' 2024-09-10T08:44:47.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4341 fp=#0x100000040bf' 2024-09-10T08:44:47.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4373 fp=#0x10000002010' 2024-09-10T08:44:47.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4405 fp=#0x10000004d24' 2024-09-10T08:44:47.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4437 fp=#0x10000001fbc' 2024-09-10T08:44:47.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4501 fp=#0x10000001cde' 2024-09-10T08:44:47.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4533 fp=#0x1000000125c' 2024-09-10T08:44:47.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4565 fp=#0x100000050ee' 2024-09-10T08:44:47.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4597 fp=#0x1000000136e' 2024-09-10T08:44:47.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4629 fp=#0x100000014da' 2024-09-10T08:44:47.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4661 fp=#0x10000001357' 2024-09-10T08:44:47.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4693 fp=#0x10000001455' 2024-09-10T08:44:47.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4725 fp=#0x1000000126a' 2024-09-10T08:44:47.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4757 fp=#0x100000012e4' 2024-09-10T08:44:47.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4789 fp=#0x10000001369' 2024-09-10T08:44:47.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4821 fp=#0x10000004a49' 2024-09-10T08:44:47.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4853 fp=#0x10000004abd' 2024-09-10T08:44:47.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4885 fp=#0x10000003747' 2024-09-10T08:44:47.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4917 fp=#0x100000036d5' 2024-09-10T08:44:47.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4949 fp=#0x1000000504e' 2024-09-10T08:44:47.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4981 fp=#0x10000005388' 2024-09-10T08:44:47.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5013 fp=#0x10000005114' 2024-09-10T08:44:47.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5045 fp=#0x100000050c5' 2024-09-10T08:44:47.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5077 fp=#0x10000005152' 2024-09-10T08:44:47.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5109 fp=#0x1000000516c' 2024-09-10T08:44:47.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5141 fp=#0x10000001b9f' 2024-09-10T08:44:47.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5173 fp=#0x10000001be2' 2024-09-10T08:44:47.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5205 fp=#0x10000001e65' 2024-09-10T08:44:47.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5237 fp=#0x10000002476' 2024-09-10T08:44:47.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5269 fp=#0x1000000365d' 2024-09-10T08:44:47.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5301 fp=#0x100000037a4' 2024-09-10T08:44:47.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5333 fp=#0x100000038ed' 2024-09-10T08:44:47.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5365 fp=#0x10000003648' 2024-09-10T08:44:47.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5397 fp=#0x10000003674' 2024-09-10T08:44:47.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5429 fp=#0x100000038a5' 2024-09-10T08:44:47.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5461 fp=#0x100000012c2' 2024-09-10T08:44:47.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5525 fp=#0x10000004e52' 2024-09-10T08:44:47.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5589 fp=#0x10000000f6c' 2024-09-10T08:44:47.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5621 fp=#0x10000000fea' 2024-09-10T08:44:47.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5653 fp=#0x100000050df' 2024-09-10T08:44:47.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5685 fp=#0x100000050a8' 2024-09-10T08:44:47.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5717 fp=#0x100000010d9' 2024-09-10T08:44:47.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5749 fp=#0x10000000f74' 2024-09-10T08:44:47.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5813 fp=#0x10000004526' 2024-09-10T08:44:47.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5845 fp=#0x10000004b1e' 2024-09-10T08:44:47.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5877 fp=#0x10000004d3a' 2024-09-10T08:44:47.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5909 fp=#0x10000004943' 2024-09-10T08:44:47.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5941 fp=#0x10000004d99' 2024-09-10T08:44:47.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5973 fp=#0x10000004374' 2024-09-10T08:44:47.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6165 fp=#0x10000003458' 2024-09-10T08:44:47.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6229 fp=#0x10000003579' 2024-09-10T08:44:47.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6293 fp=#0x10000003388' 2024-09-10T08:44:47.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6325 fp=#0x10000001dda' 2024-09-10T08:44:47.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6357 fp=#0x1000000331a' 2024-09-10T08:44:47.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6389 fp=#0x1000000342f' 2024-09-10T08:44:47.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6421 fp=#0x10000005332' 2024-09-10T08:44:47.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6453 fp=#0x1000000533c' 2024-09-10T08:44:47.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6485 fp=#0x10000004c36' 2024-09-10T08:44:47.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6517 fp=#0x10000004e40' 2024-09-10T08:44:47.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6549 fp=#0x10000004d5c' 2024-09-10T08:44:47.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6645 fp=#0x10000003d4a' 2024-09-10T08:44:47.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6677 fp=#0x10000003c69' 2024-09-10T08:44:47.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6709 fp=#0x1000000505d' 2024-09-10T08:44:47.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6741 fp=#0x10000003a8d' 2024-09-10T08:44:47.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6773 fp=#0x10000003bc2' 2024-09-10T08:44:47.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6805 fp=#0x100000045b7' 2024-09-10T08:44:47.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6837 fp=#0x10000004749' 2024-09-10T08:44:47.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6869 fp=#0x100000052ac' 2024-09-10T08:44:47.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6901 fp=#0x10000004614' 2024-09-10T08:44:47.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6933 fp=#0x100000052f3' 2024-09-10T08:44:47.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6965 fp=#0x10000000634' 2024-09-10T08:44:47.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6997 fp=#0x10000004fe8' 2024-09-10T08:44:47.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7029 fp=#0x100000010e8' 2024-09-10T08:44:47.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7061 fp=#0x10000000598' 2024-09-10T08:44:47.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7093 fp=#0x10000004f43' 2024-09-10T08:44:47.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7125 fp=#0x100000005ff' 2024-09-10T08:44:47.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7157 fp=#0x10000000578' 2024-09-10T08:44:47.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7189 fp=#0x10000004f88' 2024-09-10T08:44:47.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7349 fp=#0x1000000515a' 2024-09-10T08:44:47.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7413 fp=#0x1000000238e' 2024-09-10T08:44:47.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7445 fp=#0x1000000492d' 2024-09-10T08:44:47.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7477 fp=#0x10000003aa6' 2024-09-10T08:44:47.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7509 fp=#0x100000046c0' 2024-09-10T08:44:47.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7541 fp=#0x10000001398' 2024-09-10T08:44:47.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7573 fp=#0x1000000135c' 2024-09-10T08:44:47.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7605 fp=#0x1000000516d' 2024-09-10T08:44:47.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7637 fp=#0x10000003320' 2024-09-10T08:44:47.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7701 fp=#0x100000051c7' 2024-09-10T08:44:47.747 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7733 fp=#0x10000004f3b' 2024-09-10T08:44:47.747 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7765 fp=#0x10000004e7b' 2024-09-10T08:44:47.747 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7797 fp=#0x10000001421' 2024-09-10T08:44:47.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7829 fp=#0x100000013e0' 2024-09-10T08:44:47.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7861 fp=#0x1000000157d' 2024-09-10T08:44:47.748 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7893 fp=#0x10000001724' 2024-09-10T08:44:47.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7925 fp=#0x10000004e6d' 2024-09-10T08:44:47.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7957 fp=#0x10000001593' 2024-09-10T08:44:47.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7989 fp=#0x10000004d15' 2024-09-10T08:44:47.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8021 fp=#0x10000004d81' 2024-09-10T08:44:47.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8053 fp=#0x10000001677' 2024-09-10T08:44:47.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8085 fp=#0x10000003fd8' 2024-09-10T08:44:47.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8117 fp=#0x1000000162a' 2024-09-10T08:44:47.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8149 fp=#0x100000026fe' 2024-09-10T08:44:47.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8181 fp=#0x100000025f8' 2024-09-10T08:44:47.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8213 fp=#0x1000000260a' 2024-09-10T08:44:47.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8245 fp=#0x10000005248' 2024-09-10T08:44:47.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8277 fp=#0x100000046eb' 2024-09-10T08:44:47.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8309 fp=#0x100000042f1' 2024-09-10T08:44:47.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8341 fp=#0x10000005255' 2024-09-10T08:44:47.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8373 fp=#0x1000000454a' 2024-09-10T08:44:47.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8405 fp=#0x10000005258' 2024-09-10T08:44:47.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8437 fp=#0x1000000410a' 2024-09-10T08:44:47.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8469 fp=#0x10000003deb' 2024-09-10T08:44:47.755 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8501 fp=#0x1000000425b' 2024-09-10T08:44:47.755 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8533 fp=#0x1000000415c' 2024-09-10T08:44:47.755 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8565 fp=#0x10000004288' 2024-09-10T08:44:47.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8597 fp=#0x10000003e02' 2024-09-10T08:44:47.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8629 fp=#0x100000052ae' 2024-09-10T08:44:47.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8661 fp=#0x10000004fda' 2024-09-10T08:44:47.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8693 fp=#0x10000004ee1' 2024-09-10T08:44:47.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8725 fp=#0x10000003d50' 2024-09-10T08:44:47.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8757 fp=#0x10000003cb9' 2024-09-10T08:44:47.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8789 fp=#0x10000003db8' 2024-09-10T08:44:47.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8821 fp=#0x100000052fa' 2024-09-10T08:44:47.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8853 fp=#0x1000000443c' 2024-09-10T08:44:47.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8885 fp=#0x10000001d83' 2024-09-10T08:44:47.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8917 fp=#0x10000001aca' 2024-09-10T08:44:47.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8949 fp=#0x10000001af0' 2024-09-10T08:44:47.760 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8981 fp=#0x100000043f1' 2024-09-10T08:44:47.760 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9013 fp=#0x10000001cfb' 2024-09-10T08:44:47.760 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9045 fp=#0x1000000423b' 2024-09-10T08:44:47.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9077 fp=#0x10000001e98' 2024-09-10T08:44:47.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9109 fp=#0x10000000d46' 2024-09-10T08:44:47.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9141 fp=#0x1000000410b' 2024-09-10T08:44:47.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9173 fp=#0x10000000d5c' 2024-09-10T08:44:47.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9205 fp=#0x10000000d99' 2024-09-10T08:44:47.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9237 fp=#0x10000000d7a' 2024-09-10T08:44:47.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9269 fp=#0x10000000c29' 2024-09-10T08:44:47.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9301 fp=#0x10000004d73' 2024-09-10T08:44:47.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9333 fp=#0x10000000bba' 2024-09-10T08:44:47.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9365 fp=#0x10000004c80' 2024-09-10T08:44:47.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9397 fp=#0x10000000bd1' 2024-09-10T08:44:47.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9429 fp=#0x10000004ce1' 2024-09-10T08:44:47.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9461 fp=#0x10000004db6' 2024-09-10T08:44:47.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9493 fp=#0x100000048b8' 2024-09-10T08:44:47.765 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9525 fp=#0x10000000ec6' 2024-09-10T08:44:47.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9557 fp=#0x100000049a7' 2024-09-10T08:44:47.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9589 fp=#0x10000001ae2' 2024-09-10T08:44:47.766 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9621 fp=#0x10000001a38' 2024-09-10T08:44:47.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9653 fp=#0x10000004933' 2024-09-10T08:44:47.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9685 fp=#0x100000047fb' 2024-09-10T08:44:47.767 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9717 fp=#0x10000001a4d' 2024-09-10T08:44:47.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9749 fp=#0x10000001759' 2024-09-10T08:44:47.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9781 fp=#0x1000000175f' 2024-09-10T08:44:47.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9813 fp=#0x1000000440f' 2024-09-10T08:44:47.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9845 fp=#0x10000001842' 2024-09-10T08:44:47.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9877 fp=#0x10000004480' 2024-09-10T08:44:47.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9909 fp=#0x100000019cf' 2024-09-10T08:44:47.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9941 fp=#0x1000000197e' 2024-09-10T08:44:47.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9973 fp=#0x10000001862' 2024-09-10T08:44:47.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10005 fp=#0x1000000171b' 2024-09-10T08:44:47.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10037 fp=#0x10000001707' 2024-09-10T08:44:47.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10069 fp=#0x1000000174e' 2024-09-10T08:44:47.771 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10101 fp=#0x10000003acb' 2024-09-10T08:44:47.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10133 fp=#0x100000039af' 2024-09-10T08:44:47.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10165 fp=#0x10000003969' 2024-09-10T08:44:47.772 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10197 fp=#0x10000001e6f' 2024-09-10T08:44:47.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10229 fp=#0x10000003a3c' 2024-09-10T08:44:47.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10261 fp=#0x100000039e7' 2024-09-10T08:44:47.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10293 fp=#0x100000024ab' 2024-09-10T08:44:47.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10325 fp=#0x10000002446' 2024-09-10T08:44:47.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10357 fp=#0x10000001337' 2024-09-10T08:44:47.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10389 fp=#0x10000001325' 2024-09-10T08:44:47.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10421 fp=#0x10000003a20' 2024-09-10T08:44:47.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10453 fp=#0x100000039ed' 2024-09-10T08:44:47.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10485 fp=#0x10000001dc5' 2024-09-10T08:44:47.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10517 fp=#0x10000003903' 2024-09-10T08:44:47.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10549 fp=#0x10000001315' 2024-09-10T08:44:47.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10613 fp=#0x10000002262' 2024-09-10T08:44:47.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10645 fp=#0x10000004950' 2024-09-10T08:44:47.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10805 fp=#0x100000010a8' 2024-09-10T08:44:47.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10869 fp=#0x10000001cd5' 2024-09-10T08:44:47.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10901 fp=#0x10000000b95' 2024-09-10T08:44:47.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10933 fp=#0x10000001cdc' 2024-09-10T08:44:47.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10997 fp=#0x100000018b3' 2024-09-10T08:44:47.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11029 fp=#0x10000005074' 2024-09-10T08:44:47.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11061 fp=#0x100000039a2' 2024-09-10T08:44:47.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11093 fp=#0x10000003882' 2024-09-10T08:44:47.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11125 fp=#0x10000002052' 2024-09-10T08:44:47.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11157 fp=#0x100000021c1' 2024-09-10T08:44:47.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11189 fp=#0x10000003f8b' 2024-09-10T08:44:47.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11221 fp=#0x10000002076' 2024-09-10T08:44:47.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11253 fp=#0x100000022c8' 2024-09-10T08:44:47.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11285 fp=#0x1000000205e' 2024-09-10T08:44:47.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11317 fp=#0x10000003d3e' 2024-09-10T08:44:47.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11349 fp=#0x10000001c7e' 2024-09-10T08:44:47.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11381 fp=#0x10000003f58' 2024-09-10T08:44:47.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11413 fp=#0x10000003e92' 2024-09-10T08:44:47.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11445 fp=#0x10000001c86' 2024-09-10T08:44:47.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11477 fp=#0x10000003e1f' 2024-09-10T08:44:47.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11509 fp=#0x10000001b0b' 2024-09-10T08:44:47.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11669 fp=#0x10000005181' 2024-09-10T08:44:47.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11701 fp=#0x10000000932' 2024-09-10T08:44:47.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11733 fp=#0x10000004237' 2024-09-10T08:44:47.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11765 fp=#0x100000009db' 2024-09-10T08:44:47.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11797 fp=#0x100000023f6' 2024-09-10T08:44:47.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11829 fp=#0x10000003540' 2024-09-10T08:44:47.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11861 fp=#0x10000004196' 2024-09-10T08:44:47.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11893 fp=#0x100000009b0' 2024-09-10T08:44:47.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11925 fp=#0x10000000f04' 2024-09-10T08:44:47.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11957 fp=#0x10000001110' 2024-09-10T08:44:47.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11989 fp=#0x10000000f17' 2024-09-10T08:44:47.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12021 fp=#0x10000000ef9' 2024-09-10T08:44:47.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12053 fp=#0x100000052a7' 2024-09-10T08:44:47.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12085 fp=#0x10000004565' 2024-09-10T08:44:47.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12117 fp=#0x100000043e0' 2024-09-10T08:44:47.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12149 fp=#0x100000044cf' 2024-09-10T08:44:47.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12181 fp=#0x100000043aa' 2024-09-10T08:44:47.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12213 fp=#0x10000000d63' 2024-09-10T08:44:47.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12245 fp=#0x100000051ed' 2024-09-10T08:44:47.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12277 fp=#0x10000000c07' 2024-09-10T08:44:47.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12309 fp=#0x100000051fb' 2024-09-10T08:44:47.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12341 fp=#0x10000003b11' 2024-09-10T08:44:47.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12373 fp=#0x100000017c7' 2024-09-10T08:44:47.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12405 fp=#0x100000039f2' 2024-09-10T08:44:47.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12437 fp=#0x100000051eb' 2024-09-10T08:44:47.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12501 fp=#0x10000004c7b' 2024-09-10T08:44:47.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12533 fp=#0x10000004bb9' 2024-09-10T08:44:47.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12565 fp=#0x10000004cc3' 2024-09-10T08:44:47.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12597 fp=#0x10000004b30' 2024-09-10T08:44:47.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12629 fp=#0x10000004ad2' 2024-09-10T08:44:47.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12661 fp=#0x100000047f9' 2024-09-10T08:44:47.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12693 fp=#0x100000053db' 2024-09-10T08:44:47.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12725 fp=#0x100000048ca' 2024-09-10T08:44:47.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12757 fp=#0x100000047ae' 2024-09-10T08:44:47.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12789 fp=#0x1000000276a' 2024-09-10T08:44:47.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12821 fp=#0x1000000540e' 2024-09-10T08:44:47.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12853 fp=#0x100000052d7' 2024-09-10T08:44:47.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12885 fp=#0x100000021df' 2024-09-10T08:44:47.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12917 fp=#0x100000021fd' 2024-09-10T08:44:47.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12949 fp=#0x100000027c5' 2024-09-10T08:44:47.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13013 fp=#0x100000053a9' 2024-09-10T08:44:47.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13037 fp=#0x10000002323' 2024-09-10T08:44:47.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13101 fp=#0x10000002248' 2024-09-10T08:44:47.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13133 fp=#0x10000003817' 2024-09-10T08:44:47.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13165 fp=#0x10000002301' 2024-09-10T08:44:47.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:254 fp=#0x10000003ca1' 2024-09-10T08:44:47.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:286 fp=#0x10000000f2d' 2024-09-10T08:44:47.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:318 fp=#0x10000002357' 2024-09-10T08:44:47.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:374 fp=#0x100000046fc' 2024-09-10T08:44:47.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:502 fp=#0x10000003566' 2024-09-10T08:44:47.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:534 fp=#0x100000032e7' 2024-09-10T08:44:47.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:566 fp=#0x100000034d3' 2024-09-10T08:44:47.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:598 fp=#0x100000032b7' 2024-09-10T08:44:47.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:822 fp=#0x100000044d3' 2024-09-10T08:44:47.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:854 fp=#0x10000000dcb' 2024-09-10T08:44:47.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:886 fp=#0x100000040b2' 2024-09-10T08:44:47.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:950 fp=#0x10000000d9a' 2024-09-10T08:44:47.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:982 fp=#0x100000045d2' 2024-09-10T08:44:47.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1014 fp=#0x10000000df7' 2024-09-10T08:44:47.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1078 fp=#0x1000000429f' 2024-09-10T08:44:47.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1238 fp=#0x10000001fa9' 2024-09-10T08:44:47.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1270 fp=#0x10000004951' 2024-09-10T08:44:47.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1334 fp=#0x1000000359e' 2024-09-10T08:44:47.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1398 fp=#0x100000032e4' 2024-09-10T08:44:47.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1654 fp=#0x1000000458b' 2024-09-10T08:44:47.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1910 fp=#0x1000000277a' 2024-09-10T08:44:47.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2134 fp=#0x1000000219b' 2024-09-10T08:44:47.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2166 fp=#0x1000000204d' 2024-09-10T08:44:47.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2198 fp=#0x10000002135' 2024-09-10T08:44:47.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2230 fp=#0x100000026b5' 2024-09-10T08:44:47.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2262 fp=#0x10000002650' 2024-09-10T08:44:47.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2326 fp=#0x10000002593' 2024-09-10T08:44:47.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2614 fp=#0x100000018c1' 2024-09-10T08:44:47.825 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2678 fp=#0x1000000102e' 2024-09-10T08:44:47.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2710 fp=#0x100000010b7' 2024-09-10T08:44:47.826 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2742 fp=#0x100000018a6' 2024-09-10T08:44:47.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2806 fp=#0x100000035a8' 2024-09-10T08:44:47.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2838 fp=#0x1000000466c' 2024-09-10T08:44:47.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2870 fp=#0x10000003531' 2024-09-10T08:44:47.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2902 fp=#0x1000000355b' 2024-09-10T08:44:47.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2934 fp=#0x100000046b7' 2024-09-10T08:44:47.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2998 fp=#0x10000000e98' 2024-09-10T08:44:47.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3030 fp=#0x10000000e7a' 2024-09-10T08:44:47.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3062 fp=#0x10000001921' 2024-09-10T08:44:47.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3126 fp=#0x10000000e4f' 2024-09-10T08:44:47.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3190 fp=#0x10000000e21' 2024-09-10T08:44:47.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3222 fp=#0x100000045eb' 2024-09-10T08:44:47.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3254 fp=#0x10000003ce4' 2024-09-10T08:44:47.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3286 fp=#0x10000000751' 2024-09-10T08:44:47.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3318 fp=#0x1000000073f' 2024-09-10T08:44:47.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3350 fp=#0x1000000071a' 2024-09-10T08:44:47.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3382 fp=#0x10000000777' 2024-09-10T08:44:47.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3414 fp=#0x10000000797' 2024-09-10T08:44:47.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3446 fp=#0x1000000415f' 2024-09-10T08:44:47.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3510 fp=#0x10000003c60' 2024-09-10T08:44:47.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3606 fp=#0x100000032cd' 2024-09-10T08:44:47.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3638 fp=#0x10000004c9a' 2024-09-10T08:44:47.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3670 fp=#0x10000004b1d' 2024-09-10T08:44:47.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3702 fp=#0x10000004c4a' 2024-09-10T08:44:47.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3766 fp=#0x1000000269d' 2024-09-10T08:44:47.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3798 fp=#0x10000003f8a' 2024-09-10T08:44:47.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3830 fp=#0x10000002515' 2024-09-10T08:44:47.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3894 fp=#0x10000002728' 2024-09-10T08:44:47.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3926 fp=#0x10000004a51' 2024-09-10T08:44:47.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3958 fp=#0x10000002050' 2024-09-10T08:44:47.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3990 fp=#0x1000000054c' 2024-09-10T08:44:47.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4022 fp=#0x10000000573' 2024-09-10T08:44:47.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4054 fp=#0x1000000463c' 2024-09-10T08:44:47.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4086 fp=#0x10000001000' 2024-09-10T08:44:47.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4118 fp=#0x10000004e37' 2024-09-10T08:44:47.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4150 fp=#0x10000000fc9' 2024-09-10T08:44:47.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4182 fp=#0x10000004d3e' 2024-09-10T08:44:47.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4214 fp=#0x10000001cc7' 2024-09-10T08:44:47.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4246 fp=#0x10000004144' 2024-09-10T08:44:47.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4278 fp=#0x1000000412c' 2024-09-10T08:44:47.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4310 fp=#0x10000003adc' 2024-09-10T08:44:47.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4342 fp=#0x10000003a3f' 2024-09-10T08:44:47.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4374 fp=#0x10000002006' 2024-09-10T08:44:47.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4406 fp=#0x10000001eea' 2024-09-10T08:44:47.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4438 fp=#0x10000001f0f' 2024-09-10T08:44:47.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4470 fp=#0x10000001f4d' 2024-09-10T08:44:47.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4502 fp=#0x100000050fb' 2024-09-10T08:44:47.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4534 fp=#0x1000000402b' 2024-09-10T08:44:47.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4566 fp=#0x10000001c55' 2024-09-10T08:44:47.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4598 fp=#0x10000001286' 2024-09-10T08:44:47.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4630 fp=#0x1000000181f' 2024-09-10T08:44:47.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4662 fp=#0x10000003963' 2024-09-10T08:44:47.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4694 fp=#0x100000014a7' 2024-09-10T08:44:47.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4726 fp=#0x10000001442' 2024-09-10T08:44:47.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4758 fp=#0x10000001522' 2024-09-10T08:44:47.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4790 fp=#0x100000039bb' 2024-09-10T08:44:47.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4822 fp=#0x1000000383c' 2024-09-10T08:44:47.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4854 fp=#0x100000036fa' 2024-09-10T08:44:47.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4886 fp=#0x10000003769' 2024-09-10T08:44:47.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4918 fp=#0x10000004a3a' 2024-09-10T08:44:47.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4950 fp=#0x10000005355' 2024-09-10T08:44:47.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4982 fp=#0x10000005032' 2024-09-10T08:44:47.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5014 fp=#0x10000005121' 2024-09-10T08:44:47.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5046 fp=#0x10000005049' 2024-09-10T08:44:47.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5078 fp=#0x10000005188' 2024-09-10T08:44:47.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5110 fp=#0x1000000502e' 2024-09-10T08:44:47.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5142 fp=#0x10000001eb5' 2024-09-10T08:44:47.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5174 fp=#0x10000001eba' 2024-09-10T08:44:47.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5206 fp=#0x10000001c71' 2024-09-10T08:44:47.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5238 fp=#0x10000001bd0' 2024-09-10T08:44:47.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5270 fp=#0x100000035e3' 2024-09-10T08:44:47.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5302 fp=#0x10000004a79' 2024-09-10T08:44:47.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5334 fp=#0x10000003dc7' 2024-09-10T08:44:47.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5366 fp=#0x10000004aa9' 2024-09-10T08:44:47.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5398 fp=#0x10000003820' 2024-09-10T08:44:47.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5430 fp=#0x100000035b9' 2024-09-10T08:44:47.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5494 fp=#0x10000001508' 2024-09-10T08:44:47.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5526 fp=#0x10000002723' 2024-09-10T08:44:47.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5590 fp=#0x10000004f7a' 2024-09-10T08:44:47.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5622 fp=#0x10000001003' 2024-09-10T08:44:47.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5654 fp=#0x10000000f54' 2024-09-10T08:44:47.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5686 fp=#0x100000050c4' 2024-09-10T08:44:47.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5718 fp=#0x10000002134' 2024-09-10T08:44:47.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5750 fp=#0x10000001011' 2024-09-10T08:44:47.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5814 fp=#0x10000004d34' 2024-09-10T08:44:47.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5846 fp=#0x10000004456' 2024-09-10T08:44:47.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5878 fp=#0x100000043a8' 2024-09-10T08:44:47.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5910 fp=#0x10000004479' 2024-09-10T08:44:47.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5942 fp=#0x10000004409' 2024-09-10T08:44:47.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5974 fp=#0x100000044c5' 2024-09-10T08:44:47.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6294 fp=#0x10000003326' 2024-09-10T08:44:47.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6326 fp=#0x1000000340f' 2024-09-10T08:44:47.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6358 fp=#0x10000001efa' 2024-09-10T08:44:47.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6390 fp=#0x100000033b5' 2024-09-10T08:44:47.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6422 fp=#0x10000004ce5' 2024-09-10T08:44:47.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6454 fp=#0x10000004ee4' 2024-09-10T08:44:47.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6486 fp=#0x10000004afd' 2024-09-10T08:44:47.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6518 fp=#0x10000004da1' 2024-09-10T08:44:47.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6550 fp=#0x10000004c8a' 2024-09-10T08:44:47.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6646 fp=#0x10000003d91' 2024-09-10T08:44:47.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6678 fp=#0x100000052c5' 2024-09-10T08:44:47.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6710 fp=#0x10000003d80' 2024-09-10T08:44:47.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6742 fp=#0x10000003a2b' 2024-09-10T08:44:47.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6774 fp=#0x10000005298' 2024-09-10T08:44:47.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6806 fp=#0x10000004949' 2024-09-10T08:44:47.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6838 fp=#0x10000005295' 2024-09-10T08:44:47.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6870 fp=#0x10000005321' 2024-09-10T08:44:47.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6902 fp=#0x100000046ba' 2024-09-10T08:44:47.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6934 fp=#0x1000000459d' 2024-09-10T08:44:47.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6966 fp=#0x10000000841' 2024-09-10T08:44:47.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6998 fp=#0x10000005010' 2024-09-10T08:44:47.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7030 fp=#0x10000001efd' 2024-09-10T08:44:47.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7062 fp=#0x1000000503f' 2024-09-10T08:44:47.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7094 fp=#0x1000000063d' 2024-09-10T08:44:47.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7126 fp=#0x10000004f9e' 2024-09-10T08:44:47.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7158 fp=#0x10000005054' 2024-09-10T08:44:47.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7222 fp=#0x1000000504b' 2024-09-10T08:44:47.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7286 fp=#0x10000005133' 2024-09-10T08:44:47.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7318 fp=#0x1000000503b' 2024-09-10T08:44:47.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7382 fp=#0x10000005126' 2024-09-10T08:44:47.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7414 fp=#0x100000035c4' 2024-09-10T08:44:47.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7446 fp=#0x1000000239d' 2024-09-10T08:44:47.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7478 fp=#0x10000004766' 2024-09-10T08:44:47.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7510 fp=#0x10000004840' 2024-09-10T08:44:47.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7542 fp=#0x10000001335' 2024-09-10T08:44:47.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7574 fp=#0x100000051a9' 2024-09-10T08:44:47.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7606 fp=#0x10000001db8' 2024-09-10T08:44:47.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7670 fp=#0x100000051df' 2024-09-10T08:44:47.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7702 fp=#0x10000005161' 2024-09-10T08:44:47.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7734 fp=#0x10000004f13' 2024-09-10T08:44:47.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7766 fp=#0x10000004eb6' 2024-09-10T08:44:47.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7798 fp=#0x10000004e92' 2024-09-10T08:44:47.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7830 fp=#0x10000004db4' 2024-09-10T08:44:47.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7862 fp=#0x100000013b4' 2024-09-10T08:44:47.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7894 fp=#0x10000004e9d' 2024-09-10T08:44:47.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7926 fp=#0x10000004e97' 2024-09-10T08:44:47.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7958 fp=#0x10000001505' 2024-09-10T08:44:47.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7990 fp=#0x10000004e80' 2024-09-10T08:44:47.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8022 fp=#0x100000014b9' 2024-09-10T08:44:47.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8054 fp=#0x100000033e4' 2024-09-10T08:44:47.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8086 fp=#0x1000000170e' 2024-09-10T08:44:47.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8118 fp=#0x100000015d7' 2024-09-10T08:44:47.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8150 fp=#0x10000002584' 2024-09-10T08:44:47.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8182 fp=#0x100000025cb' 2024-09-10T08:44:47.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8214 fp=#0x100000026e9' 2024-09-10T08:44:47.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8246 fp=#0x1000000463d' 2024-09-10T08:44:47.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8278 fp=#0x10000004164' 2024-09-10T08:44:47.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8310 fp=#0x100000043b6' 2024-09-10T08:44:47.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8342 fp=#0x1000000526f' 2024-09-10T08:44:47.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8374 fp=#0x10000005242' 2024-09-10T08:44:47.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8406 fp=#0x1000000525f' 2024-09-10T08:44:47.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8438 fp=#0x10000005274' 2024-09-10T08:44:47.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8470 fp=#0x10000004197' 2024-09-10T08:44:47.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8502 fp=#0x10000003dee' 2024-09-10T08:44:47.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8534 fp=#0x100000041e0' 2024-09-10T08:44:47.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8566 fp=#0x10000003f42' 2024-09-10T08:44:47.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8598 fp=#0x10000003dc6' 2024-09-10T08:44:47.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8630 fp=#0x10000003d5b' 2024-09-10T08:44:47.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8662 fp=#0x100000052c2' 2024-09-10T08:44:47.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8694 fp=#0x10000004fd1' 2024-09-10T08:44:47.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8726 fp=#0x1000000530a' 2024-09-10T08:44:47.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8758 fp=#0x10000003c13' 2024-09-10T08:44:47.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8790 fp=#0x10000004fbb' 2024-09-10T08:44:47.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8822 fp=#0x10000004f83' 2024-09-10T08:44:47.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8854 fp=#0x100000043dc' 2024-09-10T08:44:47.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8886 fp=#0x10000001d19' 2024-09-10T08:44:47.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8918 fp=#0x100000043b3' 2024-09-10T08:44:47.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8982 fp=#0x100000043d2' 2024-09-10T08:44:47.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9014 fp=#0x10000001c0b' 2024-09-10T08:44:47.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9046 fp=#0x10000000cf5' 2024-09-10T08:44:47.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9078 fp=#0x10000001159' 2024-09-10T08:44:47.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9110 fp=#0x10000004133' 2024-09-10T08:44:47.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9142 fp=#0x100000014cc' 2024-09-10T08:44:47.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9174 fp=#0x10000000cf2' 2024-09-10T08:44:47.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9238 fp=#0x10000004cea' 2024-09-10T08:44:47.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9270 fp=#0x10000004e07' 2024-09-10T08:44:47.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9302 fp=#0x10000000aef' 2024-09-10T08:44:47.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9334 fp=#0x10000004dac' 2024-09-10T08:44:47.898 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9366 fp=#0x10000004e3a' 2024-09-10T08:44:47.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9398 fp=#0x10000000b54' 2024-09-10T08:44:47.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9430 fp=#0x10000004d25' 2024-09-10T08:44:47.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9462 fp=#0x10000000ac0' 2024-09-10T08:44:47.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9494 fp=#0x10000001a9a' 2024-09-10T08:44:47.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9526 fp=#0x10000001a2d' 2024-09-10T08:44:47.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9558 fp=#0x1000000496f' 2024-09-10T08:44:47.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9590 fp=#0x10000000eac' 2024-09-10T08:44:47.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9622 fp=#0x100000048ea' 2024-09-10T08:44:47.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9654 fp=#0x1000000483e' 2024-09-10T08:44:47.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9686 fp=#0x10000004830' 2024-09-10T08:44:47.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9718 fp=#0x10000004490' 2024-09-10T08:44:47.902 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9750 fp=#0x100000019eb' 2024-09-10T08:44:47.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9782 fp=#0x10000004c76' 2024-09-10T08:44:47.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9814 fp=#0x10000004cb9' 2024-09-10T08:44:47.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9846 fp=#0x100000018b2' 2024-09-10T08:44:47.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9878 fp=#0x100000043d1' 2024-09-10T08:44:47.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9910 fp=#0x10000001584' 2024-09-10T08:44:47.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9942 fp=#0x100000015a3' 2024-09-10T08:44:47.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9974 fp=#0x1000000197b' 2024-09-10T08:44:47.905 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10006 fp=#0x1000000164a' 2024-09-10T08:44:47.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10038 fp=#0x100000017c4' 2024-09-10T08:44:47.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10070 fp=#0x100000026f4' 2024-09-10T08:44:47.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10102 fp=#0x10000003a4d' 2024-09-10T08:44:47.907 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10134 fp=#0x1000000402c' 2024-09-10T08:44:47.907 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10166 fp=#0x10000003b6f' 2024-09-10T08:44:47.908 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10198 fp=#0x10000001e03' 2024-09-10T08:44:47.908 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10230 fp=#0x100000038e4' 2024-09-10T08:44:47.908 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10294 fp=#0x1000000238c' 2024-09-10T08:44:47.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10326 fp=#0x10000003a8c' 2024-09-10T08:44:47.909 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10358 fp=#0x1000000136f' 2024-09-10T08:44:47.910 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10390 fp=#0x10000003915' 2024-09-10T08:44:47.910 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10422 fp=#0x100000013ce' 2024-09-10T08:44:47.910 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10454 fp=#0x10000003aac' 2024-09-10T08:44:47.911 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10486 fp=#0x100000013c8' 2024-09-10T08:44:47.911 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10518 fp=#0x100000012f3' 2024-09-10T08:44:47.911 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10550 fp=#0x10000001373' 2024-09-10T08:44:47.912 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10614 fp=#0x10000004928' 2024-09-10T08:44:47.912 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10678 fp=#0x10000004a05' 2024-09-10T08:44:47.913 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10774 fp=#0x1000000105a' 2024-09-10T08:44:47.914 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10806 fp=#0x10000000627' 2024-09-10T08:44:47.914 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10870 fp=#0x100000006e4' 2024-09-10T08:44:47.915 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10934 fp=#0x10000003a9e' 2024-09-10T08:44:47.915 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10966 fp=#0x1000000052a' 2024-09-10T08:44:47.915 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10998 fp=#0x100000005c7' 2024-09-10T08:44:47.916 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11030 fp=#0x10000003a85' 2024-09-10T08:44:47.916 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11062 fp=#0x100000039bc' 2024-09-10T08:44:47.916 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11094 fp=#0x100000039b5' 2024-09-10T08:44:47.917 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11158 fp=#0x10000002278' 2024-09-10T08:44:47.917 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11190 fp=#0x10000004934' 2024-09-10T08:44:47.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11222 fp=#0x10000003e45' 2024-09-10T08:44:47.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11254 fp=#0x100000020ff' 2024-09-10T08:44:47.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11286 fp=#0x1000000231c' 2024-09-10T08:44:47.919 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11318 fp=#0x10000001ba0' 2024-09-10T08:44:47.919 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11350 fp=#0x10000001b45' 2024-09-10T08:44:47.919 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11382 fp=#0x10000003df5' 2024-09-10T08:44:47.920 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11414 fp=#0x10000001b90' 2024-09-10T08:44:47.920 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11446 fp=#0x10000003e13' 2024-09-10T08:44:47.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11478 fp=#0x10000003db3' 2024-09-10T08:44:47.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11510 fp=#0x10000003eeb' 2024-09-10T08:44:47.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11574 fp=#0x10000005137' 2024-09-10T08:44:47.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11606 fp=#0x10000005145' 2024-09-10T08:44:47.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11670 fp=#0x10000005173' 2024-09-10T08:44:47.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11702 fp=#0x10000004da3' 2024-09-10T08:44:47.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11734 fp=#0x10000003b71' 2024-09-10T08:44:47.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11766 fp=#0x10000004200' 2024-09-10T08:44:47.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11798 fp=#0x1000000426d' 2024-09-10T08:44:47.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11830 fp=#0x100000009e5' 2024-09-10T08:44:47.924 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11862 fp=#0x10000004289' 2024-09-10T08:44:47.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11894 fp=#0x100000009c7' 2024-09-10T08:44:47.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11990 fp=#0x10000000efa' 2024-09-10T08:44:47.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12022 fp=#0x100000010ff' 2024-09-10T08:44:47.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12054 fp=#0x100000052c0' 2024-09-10T08:44:47.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12086 fp=#0x10000003a38' 2024-09-10T08:44:47.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12118 fp=#0x100000045b6' 2024-09-10T08:44:47.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12150 fp=#0x10000004506' 2024-09-10T08:44:47.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12182 fp=#0x10000004d93' 2024-09-10T08:44:47.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12214 fp=#0x10000003af8' 2024-09-10T08:44:47.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12246 fp=#0x10000000c02' 2024-09-10T08:44:47.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12278 fp=#0x10000003b6c' 2024-09-10T08:44:47.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12310 fp=#0x10000000c4d' 2024-09-10T08:44:47.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12342 fp=#0x100000051f7' 2024-09-10T08:44:47.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12374 fp=#0x10000000c3a' 2024-09-10T08:44:47.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12406 fp=#0x100000042c7' 2024-09-10T08:44:47.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12438 fp=#0x10000001e7a' 2024-09-10T08:44:47.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12502 fp=#0x10000004c4d' 2024-09-10T08:44:47.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12534 fp=#0x10000003df0' 2024-09-10T08:44:47.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12566 fp=#0x10000004b14' 2024-09-10T08:44:47.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12598 fp=#0x10000004b94' 2024-09-10T08:44:47.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12630 fp=#0x10000004bc8' 2024-09-10T08:44:47.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12662 fp=#0x100000053ad' 2024-09-10T08:44:47.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12694 fp=#0x100000053a6' 2024-09-10T08:44:47.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12726 fp=#0x100000053e9' 2024-09-10T08:44:47.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12758 fp=#0x10000004903' 2024-09-10T08:44:47.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12822 fp=#0x10000005407' 2024-09-10T08:44:47.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12854 fp=#0x10000002174' 2024-09-10T08:44:47.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12886 fp=#0x10000005317' 2024-09-10T08:44:47.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12918 fp=#0x100000021d5' 2024-09-10T08:44:47.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12950 fp=#0x10000004d19' 2024-09-10T08:44:47.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12982 fp=#0x10000004dd6' 2024-09-10T08:44:47.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13014 fp=#0x1000000272d' 2024-09-10T08:44:47.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13038 fp=#0x10000003a0c' 2024-09-10T08:44:47.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13070 fp=#0x10000004f82' 2024-09-10T08:44:47.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13102 fp=#0x100000053fe' 2024-09-10T08:44:47.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13134 fp=#0x10000003954' 2024-09-10T08:44:47.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13166 fp=#0x10000003a05' 2024-09-10T08:44:47.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:255 fp=#0x10000004580' 2024-09-10T08:44:47.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:319 fp=#0x100000012d5' 2024-09-10T08:44:47.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:439 fp=#0x10000004adb' 2024-09-10T08:44:47.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:503 fp=#0x10000003303' 2024-09-10T08:44:47.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:567 fp=#0x100000032df' 2024-09-10T08:44:47.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:599 fp=#0x100000034e6' 2024-09-10T08:44:47.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:791 fp=#0x10000003fe0' 2024-09-10T08:44:47.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:823 fp=#0x100000040bc' 2024-09-10T08:44:47.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:855 fp=#0x10000000db9' 2024-09-10T08:44:47.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:887 fp=#0x10000000e5c' 2024-09-10T08:44:47.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:919 fp=#0x10000000def' 2024-09-10T08:44:47.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:983 fp=#0x10000003fc0' 2024-09-10T08:44:47.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1015 fp=#0x10000004525' 2024-09-10T08:44:47.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1079 fp=#0x100000042ce' 2024-09-10T08:44:47.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1111 fp=#0x10000004147' 2024-09-10T08:44:47.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1143 fp=#0x100000042c4' 2024-09-10T08:44:47.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1175 fp=#0x100000041cf' 2024-09-10T08:44:47.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1239 fp=#0x10000002098' 2024-09-10T08:44:47.950 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1271 fp=#0x10000001fe0' 2024-09-10T08:44:47.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1303 fp=#0x1000000360e' 2024-09-10T08:44:47.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1335 fp=#0x10000003663' 2024-09-10T08:44:47.951 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1367 fp=#0x100000032a6' 2024-09-10T08:44:47.953 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1559 fp=#0x10000003f1c' 2024-09-10T08:44:47.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1655 fp=#0x100000045b1' 2024-09-10T08:44:47.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1687 fp=#0x10000003377' 2024-09-10T08:44:47.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1751 fp=#0x1000000483c' 2024-09-10T08:44:47.956 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1911 fp=#0x1000000279d' 2024-09-10T08:44:47.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2039 fp=#0x10000002166' 2024-09-10T08:44:47.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2103 fp=#0x10000002044' 2024-09-10T08:44:47.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2135 fp=#0x10000002049' 2024-09-10T08:44:47.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2167 fp=#0x10000001fe4' 2024-09-10T08:44:47.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2199 fp=#0x10000002025' 2024-09-10T08:44:47.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2327 fp=#0x10000002597' 2024-09-10T08:44:47.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2615 fp=#0x100000018c5' 2024-09-10T08:44:47.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2679 fp=#0x1000000115a' 2024-09-10T08:44:47.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2711 fp=#0x1000000116f' 2024-09-10T08:44:47.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2743 fp=#0x10000003baa' 2024-09-10T08:44:47.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2775 fp=#0x1000000342e' 2024-09-10T08:44:47.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2807 fp=#0x1000000365b' 2024-09-10T08:44:47.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2839 fp=#0x1000000485f' 2024-09-10T08:44:47.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2871 fp=#0x10000004725' 2024-09-10T08:44:47.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2903 fp=#0x10000003576' 2024-09-10T08:44:47.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2967 fp=#0x1000000348a' 2024-09-10T08:44:47.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3031 fp=#0x10000000e1d' 2024-09-10T08:44:47.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3063 fp=#0x10000004a65' 2024-09-10T08:44:47.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3127 fp=#0x10000000e55' 2024-09-10T08:44:47.968 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3191 fp=#0x100000049f9' 2024-09-10T08:44:47.968 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3223 fp=#0x100000007ee' 2024-09-10T08:44:47.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3255 fp=#0x10000000796' 2024-09-10T08:44:47.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3287 fp=#0x10000003bca' 2024-09-10T08:44:47.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3319 fp=#0x10000003d54' 2024-09-10T08:44:47.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3351 fp=#0x10000003d63' 2024-09-10T08:44:47.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3383 fp=#0x10000000921' 2024-09-10T08:44:47.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3415 fp=#0x100000006f2' 2024-09-10T08:44:47.972 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3639 fp=#0x10000004b3a' 2024-09-10T08:44:47.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3671 fp=#0x10000004b93' 2024-09-10T08:44:47.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3703 fp=#0x10000004c0c' 2024-09-10T08:44:47.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3767 fp=#0x10000002790' 2024-09-10T08:44:47.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3799 fp=#0x1000000273f' 2024-09-10T08:44:47.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3831 fp=#0x100000026d7' 2024-09-10T08:44:47.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3863 fp=#0x10000003f6f' 2024-09-10T08:44:47.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3895 fp=#0x100000026b6' 2024-09-10T08:44:47.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3927 fp=#0x100000006b4' 2024-09-10T08:44:47.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3959 fp=#0x10000002484' 2024-09-10T08:44:47.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3991 fp=#0x1000000053d' 2024-09-10T08:44:47.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4023 fp=#0x10000000683' 2024-09-10T08:44:47.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4055 fp=#0x10000000ff6' 2024-09-10T08:44:47.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4087 fp=#0x10000000fc0' 2024-09-10T08:44:47.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4119 fp=#0x100000046e9' 2024-09-10T08:44:47.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4151 fp=#0x1000000101c' 2024-09-10T08:44:47.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4183 fp=#0x10000004d7d' 2024-09-10T08:44:47.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4215 fp=#0x10000000f79' 2024-09-10T08:44:47.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4247 fp=#0x10000003b38' 2024-09-10T08:44:47.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4279 fp=#0x10000004671' 2024-09-10T08:44:47.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4311 fp=#0x1000000408e' 2024-09-10T08:44:47.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4343 fp=#0x100000039ad' 2024-09-10T08:44:47.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4375 fp=#0x10000004cf3' 2024-09-10T08:44:47.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4407 fp=#0x10000001efc' 2024-09-10T08:44:47.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4439 fp=#0x10000001f21' 2024-09-10T08:44:47.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4471 fp=#0x10000004d2a' 2024-09-10T08:44:47.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4503 fp=#0x100000050f8' 2024-09-10T08:44:47.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4535 fp=#0x1000000513c' 2024-09-10T08:44:47.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4567 fp=#0x10000001242' 2024-09-10T08:44:47.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4599 fp=#0x100000012c5' 2024-09-10T08:44:47.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4631 fp=#0x10000001384' 2024-09-10T08:44:47.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4663 fp=#0x1000000147b' 2024-09-10T08:44:47.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4695 fp=#0x10000003984' 2024-09-10T08:44:47.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4727 fp=#0x100000013b7' 2024-09-10T08:44:47.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4759 fp=#0x10000001289' 2024-09-10T08:44:47.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4791 fp=#0x1000000394b' 2024-09-10T08:44:47.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4823 fp=#0x10000003878' 2024-09-10T08:44:47.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4855 fp=#0x1000000370a' 2024-09-10T08:44:47.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4887 fp=#0x100000037c5' 2024-09-10T08:44:47.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4919 fp=#0x10000004a58' 2024-09-10T08:44:47.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4951 fp=#0x10000005335' 2024-09-10T08:44:47.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4983 fp=#0x10000005040' 2024-09-10T08:44:47.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5015 fp=#0x10000005196' 2024-09-10T08:44:47.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5047 fp=#0x1000000517c' 2024-09-10T08:44:47.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5079 fp=#0x100000050a3' 2024-09-10T08:44:47.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5111 fp=#0x10000005341' 2024-09-10T08:44:47.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5143 fp=#0x10000001c22' 2024-09-10T08:44:47.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5175 fp=#0x10000001d40' 2024-09-10T08:44:47.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5207 fp=#0x10000001c52' 2024-09-10T08:44:47.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5239 fp=#0x10000001c6c' 2024-09-10T08:44:47.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5271 fp=#0x100000038df' 2024-09-10T08:44:47.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5303 fp=#0x10000004a76' 2024-09-10T08:44:47.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5335 fp=#0x1000000361d' 2024-09-10T08:44:47.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5367 fp=#0x1000000387f' 2024-09-10T08:44:47.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5399 fp=#0x1000000359f' 2024-09-10T08:44:47.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5431 fp=#0x10000004a9f' 2024-09-10T08:44:47.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5463 fp=#0x1000000121e' 2024-09-10T08:44:47.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5591 fp=#0x100000010ed' 2024-09-10T08:44:47.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5623 fp=#0x1000000100d' 2024-09-10T08:44:47.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5655 fp=#0x100000050d4' 2024-09-10T08:44:47.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5687 fp=#0x1000000210d' 2024-09-10T08:44:47.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5719 fp=#0x10000000f63' 2024-09-10T08:44:47.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5751 fp=#0x10000000f45' 2024-09-10T08:44:47.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5783 fp=#0x100000043a1' 2024-09-10T08:44:47.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5815 fp=#0x1000000448e' 2024-09-10T08:44:47.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5847 fp=#0x10000004433' 2024-09-10T08:44:47.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5879 fp=#0x100000043c2' 2024-09-10T08:44:47.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5911 fp=#0x10000004303' 2024-09-10T08:44:47.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5943 fp=#0x1000000430a' 2024-09-10T08:44:47.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5975 fp=#0x10000004330' 2024-09-10T08:44:47.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6135 fp=#0x100000035f2' 2024-09-10T08:44:47.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6167 fp=#0x10000003468' 2024-09-10T08:44:47.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6199 fp=#0x100000034ed' 2024-09-10T08:44:47.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6295 fp=#0x10000001f1a' 2024-09-10T08:44:48.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6327 fp=#0x100000033d7' 2024-09-10T08:44:48.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6359 fp=#0x100000032d9' 2024-09-10T08:44:48.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6391 fp=#0x100000033f0' 2024-09-10T08:44:48.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6423 fp=#0x10000004c71' 2024-09-10T08:44:48.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6455 fp=#0x10000004d55' 2024-09-10T08:44:48.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6487 fp=#0x10000004b06' 2024-09-10T08:44:48.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6519 fp=#0x10000004cff' 2024-09-10T08:44:48.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6551 fp=#0x10000004b6d' 2024-09-10T08:44:48.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6647 fp=#0x1000000503c' 2024-09-10T08:44:48.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6679 fp=#0x10000003aaf' 2024-09-10T08:44:48.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6711 fp=#0x10000005286' 2024-09-10T08:44:48.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6743 fp=#0x10000004fe7' 2024-09-10T08:44:48.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6775 fp=#0x10000005031' 2024-09-10T08:44:48.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6807 fp=#0x10000004696' 2024-09-10T08:44:48.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6839 fp=#0x100000052a0' 2024-09-10T08:44:48.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6871 fp=#0x100000046ae' 2024-09-10T08:44:48.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6903 fp=#0x10000004860' 2024-09-10T08:44:48.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6935 fp=#0x10000004675' 2024-09-10T08:44:48.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6967 fp=#0x1000000068a' 2024-09-10T08:44:48.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6999 fp=#0x10000000698' 2024-09-10T08:44:48.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7031 fp=#0x10000004a98' 2024-09-10T08:44:48.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7063 fp=#0x100000005e1' 2024-09-10T08:44:48.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7095 fp=#0x10000004fd9' 2024-09-10T08:44:48.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7127 fp=#0x10000004a71' 2024-09-10T08:44:48.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7159 fp=#0x10000005022' 2024-09-10T08:44:48.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7223 fp=#0x10000005057' 2024-09-10T08:44:48.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7255 fp=#0x100000050b0' 2024-09-10T08:44:48.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7415 fp=#0x1000000248f' 2024-09-10T08:44:48.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7447 fp=#0x100000023b2' 2024-09-10T08:44:48.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7479 fp=#0x10000002345' 2024-09-10T08:44:48.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7511 fp=#0x10000004753' 2024-09-10T08:44:48.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7543 fp=#0x10000001377' 2024-09-10T08:44:48.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7575 fp=#0x100000051a6' 2024-09-10T08:44:48.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7607 fp=#0x100000017f5' 2024-09-10T08:44:48.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7639 fp=#0x1000000226c' 2024-09-10T08:44:48.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7671 fp=#0x10000003275' 2024-09-10T08:44:48.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7735 fp=#0x10000004eaa' 2024-09-10T08:44:48.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7767 fp=#0x10000004f8f' 2024-09-10T08:44:48.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7799 fp=#0x1000000149e' 2024-09-10T08:44:48.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7831 fp=#0x100000013ed' 2024-09-10T08:44:48.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7863 fp=#0x10000001713' 2024-09-10T08:44:48.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7895 fp=#0x10000001577' 2024-09-10T08:44:48.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7927 fp=#0x10000004dbd' 2024-09-10T08:44:48.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7959 fp=#0x100000016fd' 2024-09-10T08:44:48.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7991 fp=#0x10000004d8e' 2024-09-10T08:44:48.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8023 fp=#0x100000014dd' 2024-09-10T08:44:48.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8055 fp=#0x100000016eb' 2024-09-10T08:44:48.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8087 fp=#0x10000003ff5' 2024-09-10T08:44:48.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8119 fp=#0x10000003874' 2024-09-10T08:44:48.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8151 fp=#0x100000026f9' 2024-09-10T08:44:48.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8183 fp=#0x1000000257f' 2024-09-10T08:44:48.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8215 fp=#0x1000000261a' 2024-09-10T08:44:48.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8247 fp=#0x10000004122' 2024-09-10T08:44:48.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8279 fp=#0x10000005246' 2024-09-10T08:44:48.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8311 fp=#0x1000000430d' 2024-09-10T08:44:48.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8343 fp=#0x10000004350' 2024-09-10T08:44:48.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8375 fp=#0x1000000424f' 2024-09-10T08:44:48.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8407 fp=#0x10000005259' 2024-09-10T08:44:48.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8439 fp=#0x1000000524f' 2024-09-10T08:44:48.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8471 fp=#0x10000004552' 2024-09-10T08:44:48.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8503 fp=#0x100000041ec' 2024-09-10T08:44:48.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8535 fp=#0x1000000418f' 2024-09-10T08:44:48.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8567 fp=#0x10000003f20' 2024-09-10T08:44:48.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8599 fp=#0x10000003ea6' 2024-09-10T08:44:48.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8631 fp=#0x100000040b6' 2024-09-10T08:44:48.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8663 fp=#0x10000004003' 2024-09-10T08:44:48.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8695 fp=#0x10000003dc2' 2024-09-10T08:44:48.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8727 fp=#0x10000003c92' 2024-09-10T08:44:48.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8759 fp=#0x1000000402a' 2024-09-10T08:44:48.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8791 fp=#0x1000000529f' 2024-09-10T08:44:48.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8823 fp=#0x10000004fb2' 2024-09-10T08:44:48.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8855 fp=#0x10000001d78' 2024-09-10T08:44:48.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8887 fp=#0x100000049b1' 2024-09-10T08:44:48.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8919 fp=#0x1000000444e' 2024-09-10T08:44:48.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8951 fp=#0x10000001aec' 2024-09-10T08:44:48.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8983 fp=#0x10000001cd8' 2024-09-10T08:44:48.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9015 fp=#0x10000004390' 2024-09-10T08:44:48.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9047 fp=#0x10000001c4e' 2024-09-10T08:44:48.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9079 fp=#0x10000000db4' 2024-09-10T08:44:48.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9111 fp=#0x100000040c0' 2024-09-10T08:44:48.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9143 fp=#0x10000000d2d' 2024-09-10T08:44:48.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9175 fp=#0x10000000dac' 2024-09-10T08:44:48.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9207 fp=#0x100000041d7' 2024-09-10T08:44:48.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9239 fp=#0x1000000413b' 2024-09-10T08:44:48.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9271 fp=#0x10000004d9f' 2024-09-10T08:44:48.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9303 fp=#0x10000000bc6' 2024-09-10T08:44:48.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9335 fp=#0x10000000aec' 2024-09-10T08:44:48.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9367 fp=#0x10000000b60' 2024-09-10T08:44:48.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9399 fp=#0x10000004c5b' 2024-09-10T08:44:48.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9431 fp=#0x10000004ceb' 2024-09-10T08:44:48.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9463 fp=#0x10000000bf8' 2024-09-10T08:44:48.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9495 fp=#0x100000019d5' 2024-09-10T08:44:48.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9527 fp=#0x10000001aad' 2024-09-10T08:44:48.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9559 fp=#0x10000002055' 2024-09-10T08:44:48.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9591 fp=#0x10000000eaa' 2024-09-10T08:44:48.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9623 fp=#0x10000001ada' 2024-09-10T08:44:48.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9655 fp=#0x100000047a0' 2024-09-10T08:44:48.034 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9687 fp=#0x10000004762' 2024-09-10T08:44:48.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9719 fp=#0x10000001b16' 2024-09-10T08:44:48.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9751 fp=#0x1000000195c' 2024-09-10T08:44:48.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9783 fp=#0x100000043be' 2024-09-10T08:44:48.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9815 fp=#0x1000000184d' 2024-09-10T08:44:48.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9847 fp=#0x1000000172d' 2024-09-10T08:44:48.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9879 fp=#0x1000000440a' 2024-09-10T08:44:48.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9911 fp=#0x1000000271c' 2024-09-10T08:44:48.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9943 fp=#0x10000001940' 2024-09-10T08:44:48.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9975 fp=#0x10000003b6a' 2024-09-10T08:44:48.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10007 fp=#0x10000002084' 2024-09-10T08:44:48.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10039 fp=#0x10000001622' 2024-09-10T08:44:48.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10071 fp=#0x100000015a7' 2024-09-10T08:44:48.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10103 fp=#0x10000003946' 2024-09-10T08:44:48.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10135 fp=#0x10000003893' 2024-09-10T08:44:48.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10167 fp=#0x10000003af9' 2024-09-10T08:44:48.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10199 fp=#0x10000003987' 2024-09-10T08:44:48.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10231 fp=#0x10000001de0' 2024-09-10T08:44:48.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10263 fp=#0x10000004596' 2024-09-10T08:44:48.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10295 fp=#0x100000023d4' 2024-09-10T08:44:48.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10327 fp=#0x10000002392' 2024-09-10T08:44:48.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10391 fp=#0x10000001463' 2024-09-10T08:44:48.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10423 fp=#0x1000000395b' 2024-09-10T08:44:48.042 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10455 fp=#0x1000000138f' 2024-09-10T08:44:48.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10487 fp=#0x100000012f9' 2024-09-10T08:44:48.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10519 fp=#0x10000003b2f' 2024-09-10T08:44:48.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10551 fp=#0x1000000137b' 2024-09-10T08:44:48.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10583 fp=#0x100000010ef' 2024-09-10T08:44:48.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10615 fp=#0x10000004849' 2024-09-10T08:44:48.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10647 fp=#0x10000004882' 2024-09-10T08:44:48.045 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10743 fp=#0x1000000082c' 2024-09-10T08:44:48.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10775 fp=#0x1000000497a' 2024-09-10T08:44:48.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10807 fp=#0x100000045d9' 2024-09-10T08:44:48.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10839 fp=#0x10000000a10' 2024-09-10T08:44:48.047 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10871 fp=#0x10000000608' 2024-09-10T08:44:48.047 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10903 fp=#0x10000000556' 2024-09-10T08:44:48.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10999 fp=#0x10000000a19' 2024-09-10T08:44:48.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11031 fp=#0x100000038d1' 2024-09-10T08:44:48.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11063 fp=#0x1000000507e' 2024-09-10T08:44:48.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11095 fp=#0x1000000393b' 2024-09-10T08:44:48.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11127 fp=#0x1000000233e' 2024-09-10T08:44:48.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11159 fp=#0x100000021a7' 2024-09-10T08:44:48.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11191 fp=#0x10000003ed9' 2024-09-10T08:44:48.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11223 fp=#0x10000002152' 2024-09-10T08:44:48.050 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11255 fp=#0x10000003e4c' 2024-09-10T08:44:48.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11287 fp=#0x100000048e1' 2024-09-10T08:44:48.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11319 fp=#0x10000001d50' 2024-09-10T08:44:48.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11351 fp=#0x10000001c93' 2024-09-10T08:44:48.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11383 fp=#0x10000001c34' 2024-09-10T08:44:48.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11415 fp=#0x10000003d8e' 2024-09-10T08:44:48.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11447 fp=#0x10000001bb0' 2024-09-10T08:44:48.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11479 fp=#0x10000003fa9' 2024-09-10T08:44:48.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11511 fp=#0x10000001cb3' 2024-09-10T08:44:48.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11575 fp=#0x1000000513b' 2024-09-10T08:44:48.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11607 fp=#0x1000000518d' 2024-09-10T08:44:48.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11703 fp=#0x1000000099d' 2024-09-10T08:44:48.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11735 fp=#0x10000004d7f' 2024-09-10T08:44:48.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11767 fp=#0x100000041ce' 2024-09-10T08:44:48.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11799 fp=#0x10000001fbb' 2024-09-10T08:44:48.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11831 fp=#0x10000004220' 2024-09-10T08:44:48.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11863 fp=#0x10000004d85' 2024-09-10T08:44:48.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11895 fp=#0x100000008fa' 2024-09-10T08:44:48.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11927 fp=#0x1000000528e' 2024-09-10T08:44:48.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12023 fp=#0x10000001134' 2024-09-10T08:44:48.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12087 fp=#0x100000034f9' 2024-09-10T08:44:48.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12119 fp=#0x100000033dd' 2024-09-10T08:44:48.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12151 fp=#0x1000000444d' 2024-09-10T08:44:48.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12183 fp=#0x100000045d1' 2024-09-10T08:44:48.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12215 fp=#0x10000000d59' 2024-09-10T08:44:48.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12247 fp=#0x10000001e76' 2024-09-10T08:44:48.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12279 fp=#0x10000003b75' 2024-09-10T08:44:48.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12311 fp=#0x10000003b80' 2024-09-10T08:44:48.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12343 fp=#0x10000000d01' 2024-09-10T08:44:48.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12375 fp=#0x10000000d51' 2024-09-10T08:44:48.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12407 fp=#0x10000003a34' 2024-09-10T08:44:48.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12439 fp=#0x10000000d5d' 2024-09-10T08:44:48.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12471 fp=#0x10000004bd8' 2024-09-10T08:44:48.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12503 fp=#0x10000004ba0' 2024-09-10T08:44:48.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12535 fp=#0x10000004be3' 2024-09-10T08:44:48.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12567 fp=#0x10000004ba3' 2024-09-10T08:44:48.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12599 fp=#0x10000004bd5' 2024-09-10T08:44:48.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12631 fp=#0x10000004baa' 2024-09-10T08:44:48.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12663 fp=#0x100000049c1' 2024-09-10T08:44:48.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12695 fp=#0x1000000478f' 2024-09-10T08:44:48.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12727 fp=#0x1000000482a' 2024-09-10T08:44:48.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12759 fp=#0x1000000485e' 2024-09-10T08:44:48.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12791 fp=#0x10000002317' 2024-09-10T08:44:48.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12823 fp=#0x1000000229f' 2024-09-10T08:44:48.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12855 fp=#0x1000000534d' 2024-09-10T08:44:48.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12887 fp=#0x10000003db6' 2024-09-10T08:44:48.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12919 fp=#0x10000002107' 2024-09-10T08:44:48.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12951 fp=#0x10000002726' 2024-09-10T08:44:48.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13015 fp=#0x100000053c1' 2024-09-10T08:44:48.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13039 fp=#0x100000039c0' 2024-09-10T08:44:48.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13071 fp=#0x10000002328' 2024-09-10T08:44:48.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13103 fp=#0x1000000342d' 2024-09-10T08:44:48.070 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13135 fp=#0x100000022e8' 2024-09-10T08:44:48.070 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13167 fp=#0x100000053e3' 2024-09-10T08:44:48.070 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:168 fp=#0x10000000208' 2024-09-10T08:44:48.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:256 fp=#0x10000003246' 2024-09-10T08:44:48.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:288 fp=#0x100000042d4' 2024-09-10T08:44:48.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:320 fp=#0x1000000039d' 2024-09-10T08:44:48.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:408 fp=#0x1000000465c' 2024-09-10T08:44:48.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:536 fp=#0x1000000331b' 2024-09-10T08:44:48.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:568 fp=#0x100000033a5' 2024-09-10T08:44:48.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:792 fp=#0x10000000d84' 2024-09-10T08:44:48.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:824 fp=#0x10000000e8f' 2024-09-10T08:44:48.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:856 fp=#0x10000003ff2' 2024-09-10T08:44:48.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:888 fp=#0x10000000d8b' 2024-09-10T08:44:48.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:920 fp=#0x10000000db1' 2024-09-10T08:44:48.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:952 fp=#0x10000000ead' 2024-09-10T08:44:48.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:984 fp=#0x10000000da5' 2024-09-10T08:44:48.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1016 fp=#0x10000000e95' 2024-09-10T08:44:48.080 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1112 fp=#0x10000004100' 2024-09-10T08:44:48.080 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1176 fp=#0x1000000412b' 2024-09-10T08:44:48.081 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1208 fp=#0x10000004284' 2024-09-10T08:44:48.081 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1240 fp=#0x1000000360c' 2024-09-10T08:44:48.082 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1336 fp=#0x1000000368c' 2024-09-10T08:44:48.082 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1368 fp=#0x10000003283' 2024-09-10T08:44:48.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1496 fp=#0x100000032ad' 2024-09-10T08:44:48.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1656 fp=#0x10000004a32' 2024-09-10T08:44:48.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1688 fp=#0x10000003384' 2024-09-10T08:44:48.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1720 fp=#0x1000000335c' 2024-09-10T08:44:48.088 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2072 fp=#0x10000002172' 2024-09-10T08:44:48.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2104 fp=#0x10000002054' 2024-09-10T08:44:48.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2200 fp=#0x10000002057' 2024-09-10T08:44:48.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2328 fp=#0x10000002629' 2024-09-10T08:44:48.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2616 fp=#0x10000001104' 2024-09-10T08:44:48.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2648 fp=#0x10000003b49' 2024-09-10T08:44:48.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2680 fp=#0x100000010cf' 2024-09-10T08:44:48.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2776 fp=#0x1000000366a' 2024-09-10T08:44:48.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2808 fp=#0x1000000347c' 2024-09-10T08:44:48.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2840 fp=#0x10000003395' 2024-09-10T08:44:48.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2904 fp=#0x100000046ab' 2024-09-10T08:44:48.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2936 fp=#0x10000004648' 2024-09-10T08:44:48.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2968 fp=#0x10000004796' 2024-09-10T08:44:48.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3000 fp=#0x10000000dfc' 2024-09-10T08:44:48.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3032 fp=#0x10000000e6e' 2024-09-10T08:44:48.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3064 fp=#0x10000000e17' 2024-09-10T08:44:48.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3096 fp=#0x10000004992' 2024-09-10T08:44:48.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3128 fp=#0x1000000195a' 2024-09-10T08:44:48.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3160 fp=#0x10000004a6f' 2024-09-10T08:44:48.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3192 fp=#0x10000000e60' 2024-09-10T08:44:48.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3288 fp=#0x10000003d82' 2024-09-10T08:44:48.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3320 fp=#0x10000000714' 2024-09-10T08:44:48.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3352 fp=#0x1000000079b' 2024-09-10T08:44:48.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3384 fp=#0x10000003cba' 2024-09-10T08:44:48.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3416 fp=#0x1000000072a' 2024-09-10T08:44:48.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3448 fp=#0x100000040a3' 2024-09-10T08:44:48.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3640 fp=#0x10000004c95' 2024-09-10T08:44:48.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3672 fp=#0x10000004b47' 2024-09-10T08:44:48.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3704 fp=#0x10000004abb' 2024-09-10T08:44:48.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3736 fp=#0x100000026e8' 2024-09-10T08:44:48.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3768 fp=#0x10000003fa1' 2024-09-10T08:44:48.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3800 fp=#0x10000002700' 2024-09-10T08:44:48.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3832 fp=#0x10000002789' 2024-09-10T08:44:48.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3864 fp=#0x1000000261b' 2024-09-10T08:44:48.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3896 fp=#0x1000000533d' 2024-09-10T08:44:48.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3928 fp=#0x1000000179d' 2024-09-10T08:44:48.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3960 fp=#0x1000000056f' 2024-09-10T08:44:48.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3992 fp=#0x1000000051f' 2024-09-10T08:44:48.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4024 fp=#0x10000000593' 2024-09-10T08:44:48.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4056 fp=#0x10000001018' 2024-09-10T08:44:48.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4088 fp=#0x10000001190' 2024-09-10T08:44:48.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4120 fp=#0x10000004e22' 2024-09-10T08:44:48.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4152 fp=#0x10000000f80' 2024-09-10T08:44:48.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4184 fp=#0x10000004d47' 2024-09-10T08:44:48.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4216 fp=#0x100000046af' 2024-09-10T08:44:48.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4248 fp=#0x10000003a30' 2024-09-10T08:44:48.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4280 fp=#0x10000004114' 2024-09-10T08:44:48.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4312 fp=#0x10000003a0f' 2024-09-10T08:44:48.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4344 fp=#0x1000000416a' 2024-09-10T08:44:48.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4440 fp=#0x100000035c0' 2024-09-10T08:44:48.112 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4504 fp=#0x10000003856' 2024-09-10T08:44:48.112 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4536 fp=#0x100000050bc' 2024-09-10T08:44:48.112 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4568 fp=#0x100000050e0' 2024-09-10T08:44:48.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4600 fp=#0x100000012e0' 2024-09-10T08:44:48.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4632 fp=#0x100000026e0' 2024-09-10T08:44:48.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4664 fp=#0x1000000126e' 2024-09-10T08:44:48.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4696 fp=#0x1000000403a' 2024-09-10T08:44:48.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4728 fp=#0x10000001825' 2024-09-10T08:44:48.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4760 fp=#0x10000001330' 2024-09-10T08:44:48.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4792 fp=#0x1000000497b' 2024-09-10T08:44:48.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4824 fp=#0x10000004a97' 2024-09-10T08:44:48.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4856 fp=#0x10000004a9d' 2024-09-10T08:44:48.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4888 fp=#0x100000036b3' 2024-09-10T08:44:48.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4920 fp=#0x10000004a3e' 2024-09-10T08:44:48.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4952 fp=#0x10000005009' 2024-09-10T08:44:48.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4984 fp=#0x100000050d5' 2024-09-10T08:44:48.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5016 fp=#0x100000051d3' 2024-09-10T08:44:48.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5048 fp=#0x100000051bf' 2024-09-10T08:44:48.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5080 fp=#0x10000005011' 2024-09-10T08:44:48.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5112 fp=#0x10000005002' 2024-09-10T08:44:48.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5144 fp=#0x10000001e41' 2024-09-10T08:44:48.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5176 fp=#0x10000001d11' 2024-09-10T08:44:48.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5208 fp=#0x10000001c1b' 2024-09-10T08:44:48.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5240 fp=#0x10000001d31' 2024-09-10T08:44:48.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5272 fp=#0x10000003f2b' 2024-09-10T08:44:48.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5304 fp=#0x10000003625' 2024-09-10T08:44:48.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5336 fp=#0x10000003d12' 2024-09-10T08:44:48.121 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5368 fp=#0x100000036ed' 2024-09-10T08:44:48.121 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5400 fp=#0x100000036e9' 2024-09-10T08:44:48.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5432 fp=#0x10000003d2c' 2024-09-10T08:44:48.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5464 fp=#0x100000012b4' 2024-09-10T08:44:48.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5528 fp=#0x10000003a40' 2024-09-10T08:44:48.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5592 fp=#0x10000000f9b' 2024-09-10T08:44:48.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5624 fp=#0x100000050d2' 2024-09-10T08:44:48.124 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5656 fp=#0x1000000511f' 2024-09-10T08:44:48.124 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5688 fp=#0x10000000fa1' 2024-09-10T08:44:48.124 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5720 fp=#0x10000005085' 2024-09-10T08:44:48.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5784 fp=#0x100000044d5' 2024-09-10T08:44:48.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5816 fp=#0x100000043ee' 2024-09-10T08:44:48.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5848 fp=#0x100000045ba' 2024-09-10T08:44:48.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5880 fp=#0x10000004c73' 2024-09-10T08:44:48.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5912 fp=#0x10000004b91' 2024-09-10T08:44:48.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5944 fp=#0x100000045ce' 2024-09-10T08:44:48.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5976 fp=#0x10000004d4c' 2024-09-10T08:44:48.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6136 fp=#0x100000034bd' 2024-09-10T08:44:48.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6168 fp=#0x10000003603' 2024-09-10T08:44:48.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6296 fp=#0x10000001f08' 2024-09-10T08:44:48.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6328 fp=#0x10000001dd2' 2024-09-10T08:44:48.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6360 fp=#0x10000001e1b' 2024-09-10T08:44:48.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6392 fp=#0x100000032e1' 2024-09-10T08:44:48.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6424 fp=#0x10000004b77' 2024-09-10T08:44:48.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6456 fp=#0x10000004dc0' 2024-09-10T08:44:48.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6488 fp=#0x10000004cf2' 2024-09-10T08:44:48.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6520 fp=#0x10000004bc9' 2024-09-10T08:44:48.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6552 fp=#0x10000004fa0' 2024-09-10T08:44:48.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6584 fp=#0x100000047fd' 2024-09-10T08:44:48.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6648 fp=#0x10000003b13' 2024-09-10T08:44:48.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6680 fp=#0x10000003a69' 2024-09-10T08:44:48.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6712 fp=#0x10000003d42' 2024-09-10T08:44:48.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6744 fp=#0x10000003929' 2024-09-10T08:44:48.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6776 fp=#0x1000000394f' 2024-09-10T08:44:48.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6808 fp=#0x10000004918' 2024-09-10T08:44:48.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6840 fp=#0x100000045be' 2024-09-10T08:44:48.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6872 fp=#0x1000000534a' 2024-09-10T08:44:48.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6904 fp=#0x10000004738' 2024-09-10T08:44:48.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6936 fp=#0x100000045a7' 2024-09-10T08:44:48.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6968 fp=#0x10000000852' 2024-09-10T08:44:48.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7000 fp=#0x100000010cc' 2024-09-10T08:44:48.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7032 fp=#0x1000000062e' 2024-09-10T08:44:48.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7064 fp=#0x100000005f0' 2024-09-10T08:44:48.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7096 fp=#0x100000005fc' 2024-09-10T08:44:48.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7128 fp=#0x10000001a3b' 2024-09-10T08:44:48.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7160 fp=#0x10000005008' 2024-09-10T08:44:48.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7192 fp=#0x10000005052' 2024-09-10T08:44:48.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7384 fp=#0x1000000506f' 2024-09-10T08:44:48.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7416 fp=#0x1000000477d' 2024-09-10T08:44:48.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7448 fp=#0x100000048e7' 2024-09-10T08:44:48.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7480 fp=#0x10000004814' 2024-09-10T08:44:48.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7512 fp=#0x1000000479e' 2024-09-10T08:44:48.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7576 fp=#0x10000001359' 2024-09-10T08:44:48.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7608 fp=#0x10000001310' 2024-09-10T08:44:48.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7640 fp=#0x1000000517d' 2024-09-10T08:44:48.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7704 fp=#0x10000005172' 2024-09-10T08:44:48.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7736 fp=#0x10000004fa2' 2024-09-10T08:44:48.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7768 fp=#0x10000004ec9' 2024-09-10T08:44:48.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7800 fp=#0x1000000172c' 2024-09-10T08:44:48.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7832 fp=#0x10000004dcc' 2024-09-10T08:44:48.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7864 fp=#0x10000001ecf' 2024-09-10T08:44:48.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7896 fp=#0x10000004d3b' 2024-09-10T08:44:48.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7928 fp=#0x100000013a2' 2024-09-10T08:44:48.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7960 fp=#0x100000015b2' 2024-09-10T08:44:48.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7992 fp=#0x10000004dd5' 2024-09-10T08:44:48.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8024 fp=#0x10000001662' 2024-09-10T08:44:48.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8056 fp=#0x100000016d5' 2024-09-10T08:44:48.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8088 fp=#0x100000016e1' 2024-09-10T08:44:48.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8120 fp=#0x1000000383f' 2024-09-10T08:44:48.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8152 fp=#0x1000000273d' 2024-09-10T08:44:48.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8184 fp=#0x10000004960' 2024-09-10T08:44:48.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8216 fp=#0x100000042b8' 2024-09-10T08:44:48.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8248 fp=#0x100000043c5' 2024-09-10T08:44:48.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8280 fp=#0x10000004180' 2024-09-10T08:44:48.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8312 fp=#0x1000000525c' 2024-09-10T08:44:48.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8344 fp=#0x10000005277' 2024-09-10T08:44:48.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8376 fp=#0x1000000423e' 2024-09-10T08:44:48.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8408 fp=#0x100000045bd' 2024-09-10T08:44:48.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8440 fp=#0x10000005256' 2024-09-10T08:44:48.155 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8472 fp=#0x10000003ecb' 2024-09-10T08:44:48.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8504 fp=#0x10000003dbd' 2024-09-10T08:44:48.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8536 fp=#0x10000004201' 2024-09-10T08:44:48.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8568 fp=#0x10000003e3f' 2024-09-10T08:44:48.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8600 fp=#0x1000000426e' 2024-09-10T08:44:48.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8632 fp=#0x10000003f6c' 2024-09-10T08:44:48.157 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8664 fp=#0x10000004fb7' 2024-09-10T08:44:48.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8696 fp=#0x10000004f20' 2024-09-10T08:44:48.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8728 fp=#0x10000004fa1' 2024-09-10T08:44:48.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8760 fp=#0x10000003dce' 2024-09-10T08:44:48.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8792 fp=#0x10000004fbf' 2024-09-10T08:44:48.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8824 fp=#0x10000004f19' 2024-09-10T08:44:48.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8856 fp=#0x10000001daa' 2024-09-10T08:44:48.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8888 fp=#0x10000004326' 2024-09-10T08:44:48.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8920 fp=#0x10000001b4f' 2024-09-10T08:44:48.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8952 fp=#0x10000001d35' 2024-09-10T08:44:48.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8984 fp=#0x10000001c69' 2024-09-10T08:44:48.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9016 fp=#0x10000000daa' 2024-09-10T08:44:48.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9048 fp=#0x10000000db8' 2024-09-10T08:44:48.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9080 fp=#0x10000000d87' 2024-09-10T08:44:48.162 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9112 fp=#0x10000001155' 2024-09-10T08:44:48.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9144 fp=#0x10000000cb6' 2024-09-10T08:44:48.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9176 fp=#0x10000004165' 2024-09-10T08:44:48.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9208 fp=#0x100000040d9' 2024-09-10T08:44:48.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9272 fp=#0x10000004cfb' 2024-09-10T08:44:48.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9304 fp=#0x10000000b57' 2024-09-10T08:44:48.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9336 fp=#0x10000004d88' 2024-09-10T08:44:48.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9368 fp=#0x10000001c21' 2024-09-10T08:44:48.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9400 fp=#0x10000004203' 2024-09-10T08:44:48.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9432 fp=#0x10000001c1f' 2024-09-10T08:44:48.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9464 fp=#0x10000002542' 2024-09-10T08:44:48.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9496 fp=#0x10000000eeb' 2024-09-10T08:44:48.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9528 fp=#0x10000000ed1' 2024-09-10T08:44:48.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9560 fp=#0x10000004888' 2024-09-10T08:44:48.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9592 fp=#0x10000000edb' 2024-09-10T08:44:48.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9624 fp=#0x10000001eb4' 2024-09-10T08:44:48.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9656 fp=#0x10000004781' 2024-09-10T08:44:48.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9688 fp=#0x1000000486c' 2024-09-10T08:44:48.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9720 fp=#0x10000001931' 2024-09-10T08:44:48.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9752 fp=#0x10000001837' 2024-09-10T08:44:48.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9784 fp=#0x10000001858' 2024-09-10T08:44:48.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9816 fp=#0x10000004357' 2024-09-10T08:44:48.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9848 fp=#0x10000001aab' 2024-09-10T08:44:48.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9880 fp=#0x100000024cc' 2024-09-10T08:44:48.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9912 fp=#0x100000026df' 2024-09-10T08:44:48.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9944 fp=#0x100000019d7' 2024-09-10T08:44:48.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9976 fp=#0x10000002701' 2024-09-10T08:44:48.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10008 fp=#0x1000000178a' 2024-09-10T08:44:48.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10040 fp=#0x1000000158d' 2024-09-10T08:44:48.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10072 fp=#0x100000019de' 2024-09-10T08:44:48.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10104 fp=#0x1000000265f' 2024-09-10T08:44:48.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10136 fp=#0x10000001d87' 2024-09-10T08:44:48.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10168 fp=#0x10000003a23' 2024-09-10T08:44:48.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10200 fp=#0x10000001e87' 2024-09-10T08:44:48.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10232 fp=#0x10000001ea6' 2024-09-10T08:44:48.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10264 fp=#0x1000000244c' 2024-09-10T08:44:48.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10296 fp=#0x10000002526' 2024-09-10T08:44:48.179 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10328 fp=#0x100000023bf' 2024-09-10T08:44:48.179 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10360 fp=#0x10000001320' 2024-09-10T08:44:48.179 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10392 fp=#0x10000004876' 2024-09-10T08:44:48.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10424 fp=#0x10000003928' 2024-09-10T08:44:48.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10456 fp=#0x10000003aed' 2024-09-10T08:44:48.181 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10488 fp=#0x1000000396b' 2024-09-10T08:44:48.181 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10552 fp=#0x100000013d1' 2024-09-10T08:44:48.182 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10648 fp=#0x10000004861' 2024-09-10T08:44:48.183 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10712 fp=#0x100000007ed' 2024-09-10T08:44:48.183 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10744 fp=#0x10000004859' 2024-09-10T08:44:48.184 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10808 fp=#0x10000000995' 2024-09-10T08:44:48.184 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10872 fp=#0x10000001cb4' 2024-09-10T08:44:48.185 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10904 fp=#0x1000000056a' 2024-09-10T08:44:48.185 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10936 fp=#0x10000000a35' 2024-09-10T08:44:48.185 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10968 fp=#0x10000000a3d' 2024-09-10T08:44:48.186 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11032 fp=#0x10000003b1e' 2024-09-10T08:44:48.186 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11064 fp=#0x100000038ac' 2024-09-10T08:44:48.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11096 fp=#0x100000048ab' 2024-09-10T08:44:48.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11128 fp=#0x10000002071' 2024-09-10T08:44:48.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11160 fp=#0x10000003f81' 2024-09-10T08:44:48.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11192 fp=#0x100000020c5' 2024-09-10T08:44:48.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11224 fp=#0x10000003f08' 2024-09-10T08:44:48.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11256 fp=#0x1000000493b' 2024-09-10T08:44:48.189 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11288 fp=#0x10000003fc9' 2024-09-10T08:44:48.189 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11320 fp=#0x10000001a4c' 2024-09-10T08:44:48.189 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11352 fp=#0x10000003f13' 2024-09-10T08:44:48.190 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11384 fp=#0x10000001cd3' 2024-09-10T08:44:48.190 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11416 fp=#0x10000001c68' 2024-09-10T08:44:48.190 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11448 fp=#0x100000019e6' 2024-09-10T08:44:48.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11480 fp=#0x10000001a0d' 2024-09-10T08:44:48.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11512 fp=#0x10000001c3a' 2024-09-10T08:44:48.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11704 fp=#0x10000000a34' 2024-09-10T08:44:48.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11736 fp=#0x10000004dc1' 2024-09-10T08:44:48.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11768 fp=#0x10000004e61' 2024-09-10T08:44:48.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11800 fp=#0x100000015a1' 2024-09-10T08:44:48.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11832 fp=#0x100000022b0' 2024-09-10T08:44:48.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11864 fp=#0x1000000428e' 2024-09-10T08:44:48.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11896 fp=#0x1000000424c' 2024-09-10T08:44:48.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11960 fp=#0x100000052a6' 2024-09-10T08:44:48.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12024 fp=#0x100000010f0' 2024-09-10T08:44:48.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12056 fp=#0x100000010d3' 2024-09-10T08:44:48.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12120 fp=#0x100000033f7' 2024-09-10T08:44:48.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12152 fp=#0x10000003349' 2024-09-10T08:44:48.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12184 fp=#0x1000000339c' 2024-09-10T08:44:48.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12216 fp=#0x10000000be8' 2024-09-10T08:44:48.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12248 fp=#0x10000000d71' 2024-09-10T08:44:48.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12280 fp=#0x10000000d3f' 2024-09-10T08:44:48.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12312 fp=#0x10000000c2a' 2024-09-10T08:44:48.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12344 fp=#0x10000003a08' 2024-09-10T08:44:48.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12376 fp=#0x100000046ea' 2024-09-10T08:44:48.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12408 fp=#0x10000003b9b' 2024-09-10T08:44:48.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12440 fp=#0x100000051f1' 2024-09-10T08:44:48.201 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12472 fp=#0x10000004c0b' 2024-09-10T08:44:48.201 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12504 fp=#0x10000004c3d' 2024-09-10T08:44:48.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12568 fp=#0x10000004b86' 2024-09-10T08:44:48.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12600 fp=#0x10000004d65' 2024-09-10T08:44:48.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12632 fp=#0x100000052db' 2024-09-10T08:44:48.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12664 fp=#0x100000049ff' 2024-09-10T08:44:48.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12696 fp=#0x100000047aa' 2024-09-10T08:44:48.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12728 fp=#0x100000047c3' 2024-09-10T08:44:48.204 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12760 fp=#0x100000047b4' 2024-09-10T08:44:48.204 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12792 fp=#0x100000020c6' 2024-09-10T08:44:48.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12824 fp=#0x10000002144' 2024-09-10T08:44:48.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12856 fp=#0x10000005304' 2024-09-10T08:44:48.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12888 fp=#0x100000022cb' 2024-09-10T08:44:48.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12920 fp=#0x100000022d4' 2024-09-10T08:44:48.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12952 fp=#0x10000005383' 2024-09-10T08:44:48.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12984 fp=#0x100000053b4' 2024-09-10T08:44:48.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13016 fp=#0x1000000538a' 2024-09-10T08:44:48.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13040 fp=#0x1000000391b' 2024-09-10T08:44:48.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13072 fp=#0x100000022ce' 2024-09-10T08:44:48.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13104 fp=#0x10000003a10' 2024-09-10T08:44:48.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13168 fp=#0x10000005415' 2024-09-10T08:44:48.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:257 fp=#0x10000002743' 2024-09-10T08:44:48.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:321 fp=#0x100000004ef' 2024-09-10T08:44:48.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:409 fp=#0x10000004674' 2024-09-10T08:44:48.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:505 fp=#0x10000003544' 2024-09-10T08:44:48.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:537 fp=#0x10000003339' 2024-09-10T08:44:48.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:569 fp=#0x100000033e5' 2024-09-10T08:44:48.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:697 fp=#0x100000042eb' 2024-09-10T08:44:48.214 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:793 fp=#0x10000000da3' 2024-09-10T08:44:48.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:825 fp=#0x10000000eb7' 2024-09-10T08:44:48.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:857 fp=#0x100000040c3' 2024-09-10T08:44:48.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:889 fp=#0x10000004487' 2024-09-10T08:44:48.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:921 fp=#0x10000004501' 2024-09-10T08:44:48.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:953 fp=#0x10000000de3' 2024-09-10T08:44:48.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:985 fp=#0x10000000e9e' 2024-09-10T08:44:48.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1017 fp=#0x10000000e6d' 2024-09-10T08:44:48.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1081 fp=#0x100000041aa' 2024-09-10T08:44:48.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1273 fp=#0x100000035e2' 2024-09-10T08:44:48.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1305 fp=#0x10000003601' 2024-09-10T08:44:48.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1337 fp=#0x1000000366c' 2024-09-10T08:44:48.221 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1433 fp=#0x10000004572' 2024-09-10T08:44:48.222 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1593 fp=#0x10000003f2f' 2024-09-10T08:44:48.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1657 fp=#0x100000045d8' 2024-09-10T08:44:48.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2041 fp=#0x10000001fd9' 2024-09-10T08:44:48.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2073 fp=#0x10000002051' 2024-09-10T08:44:48.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2297 fp=#0x1000000261f' 2024-09-10T08:44:48.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2329 fp=#0x10000002672' 2024-09-10T08:44:48.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2617 fp=#0x10000001038' 2024-09-10T08:44:48.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2649 fp=#0x100000018e1' 2024-09-10T08:44:48.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2681 fp=#0x10000001878' 2024-09-10T08:44:48.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2713 fp=#0x1000000185e' 2024-09-10T08:44:48.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2745 fp=#0x10000003bb1' 2024-09-10T08:44:48.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2777 fp=#0x100000045fe' 2024-09-10T08:44:48.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2809 fp=#0x10000003634' 2024-09-10T08:44:48.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2841 fp=#0x1000000346a' 2024-09-10T08:44:48.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2873 fp=#0x1000000462b' 2024-09-10T08:44:48.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2937 fp=#0x10000004879' 2024-09-10T08:44:48.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2969 fp=#0x100000035e4' 2024-09-10T08:44:48.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3001 fp=#0x10000000e80' 2024-09-10T08:44:48.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3033 fp=#0x10000000e61' 2024-09-10T08:44:48.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3065 fp=#0x10000000e1f' 2024-09-10T08:44:48.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3161 fp=#0x10000000dd0' 2024-09-10T08:44:48.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3225 fp=#0x100000006f8' 2024-09-10T08:44:48.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3257 fp=#0x10000003c77' 2024-09-10T08:44:48.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3289 fp=#0x10000000717' 2024-09-10T08:44:48.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3321 fp=#0x10000000765' 2024-09-10T08:44:48.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3353 fp=#0x10000000763' 2024-09-10T08:44:48.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3385 fp=#0x100000006f3' 2024-09-10T08:44:48.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3417 fp=#0x10000000730' 2024-09-10T08:44:48.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3513 fp=#0x100000040f2' 2024-09-10T08:44:48.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3545 fp=#0x1000000407b' 2024-09-10T08:44:48.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3577 fp=#0x10000003cd0' 2024-09-10T08:44:48.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3641 fp=#0x10000004cd6' 2024-09-10T08:44:48.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3673 fp=#0x10000004bb1' 2024-09-10T08:44:48.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3705 fp=#0x10000004d27' 2024-09-10T08:44:48.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3737 fp=#0x10000003ee8' 2024-09-10T08:44:48.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3769 fp=#0x10000003f9c' 2024-09-10T08:44:48.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3801 fp=#0x10000004cd3' 2024-09-10T08:44:48.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3833 fp=#0x10000003e9b' 2024-09-10T08:44:48.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3897 fp=#0x10000003fc2' 2024-09-10T08:44:48.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3929 fp=#0x100000006b3' 2024-09-10T08:44:48.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3961 fp=#0x1000000058f' 2024-09-10T08:44:48.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3993 fp=#0x100000006a9' 2024-09-10T08:44:48.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4025 fp=#0x10000002056' 2024-09-10T08:44:48.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4057 fp=#0x10000004e29' 2024-09-10T08:44:48.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4089 fp=#0x10000004645' 2024-09-10T08:44:48.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4121 fp=#0x10000004e82' 2024-09-10T08:44:48.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4153 fp=#0x10000000f8e' 2024-09-10T08:44:48.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4185 fp=#0x10000000f9d' 2024-09-10T08:44:48.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4249 fp=#0x10000003c38' 2024-09-10T08:44:48.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4281 fp=#0x100000041c7' 2024-09-10T08:44:48.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4313 fp=#0x10000004193' 2024-09-10T08:44:48.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4345 fp=#0x1000000411b' 2024-09-10T08:44:48.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4409 fp=#0x10000001f86' 2024-09-10T08:44:48.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4441 fp=#0x10000001fc0' 2024-09-10T08:44:48.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4473 fp=#0x10000001fe1' 2024-09-10T08:44:48.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4537 fp=#0x100000050e7' 2024-09-10T08:44:48.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4569 fp=#0x1000000514d' 2024-09-10T08:44:48.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4601 fp=#0x100000012d7' 2024-09-10T08:44:48.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4633 fp=#0x10000001549' 2024-09-10T08:44:48.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4665 fp=#0x10000001529' 2024-09-10T08:44:48.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4697 fp=#0x100000039a4' 2024-09-10T08:44:48.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4729 fp=#0x100000012ac' 2024-09-10T08:44:48.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4761 fp=#0x100000038ea' 2024-09-10T08:44:48.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4793 fp=#0x1000000157f' 2024-09-10T08:44:48.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4825 fp=#0x100000036ec' 2024-09-10T08:44:48.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4857 fp=#0x1000000378f' 2024-09-10T08:44:48.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4889 fp=#0x10000004a45' 2024-09-10T08:44:48.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4921 fp=#0x100000037d6' 2024-09-10T08:44:48.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4953 fp=#0x100000050e9' 2024-09-10T08:44:48.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4985 fp=#0x100000051d6' 2024-09-10T08:44:48.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5017 fp=#0x1000000505a' 2024-09-10T08:44:48.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5049 fp=#0x100000050db' 2024-09-10T08:44:48.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5081 fp=#0x1000000514e' 2024-09-10T08:44:48.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5113 fp=#0x1000000514a' 2024-09-10T08:44:48.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5145 fp=#0x100000034a3' 2024-09-10T08:44:48.257 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5177 fp=#0x10000001ce8' 2024-09-10T08:44:48.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5209 fp=#0x10000001e86' 2024-09-10T08:44:48.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5241 fp=#0x10000001cb2' 2024-09-10T08:44:48.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5273 fp=#0x100000038d2' 2024-09-10T08:44:48.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5305 fp=#0x10000003871' 2024-09-10T08:44:48.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5337 fp=#0x10000004aac' 2024-09-10T08:44:48.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5369 fp=#0x10000003746' 2024-09-10T08:44:48.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5401 fp=#0x10000003d73' 2024-09-10T08:44:48.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5433 fp=#0x10000003d5f' 2024-09-10T08:44:48.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5529 fp=#0x10000001272' 2024-09-10T08:44:48.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5561 fp=#0x1000000346f' 2024-09-10T08:44:48.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5593 fp=#0x100000010e3' 2024-09-10T08:44:48.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5625 fp=#0x100000050d0' 2024-09-10T08:44:48.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5657 fp=#0x10000005092' 2024-09-10T08:44:48.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5689 fp=#0x10000000f4b' 2024-09-10T08:44:48.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5721 fp=#0x10000000ff5' 2024-09-10T08:44:48.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5753 fp=#0x100000050f5' 2024-09-10T08:44:48.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5785 fp=#0x10000004bf5' 2024-09-10T08:44:48.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5817 fp=#0x10000004503' 2024-09-10T08:44:48.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5849 fp=#0x10000004978' 2024-09-10T08:44:48.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5881 fp=#0x10000004d2d' 2024-09-10T08:44:48.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5913 fp=#0x100000043e9' 2024-09-10T08:44:48.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5945 fp=#0x10000004476' 2024-09-10T08:44:48.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5977 fp=#0x10000004c49' 2024-09-10T08:44:48.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6201 fp=#0x10000003485' 2024-09-10T08:44:48.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6297 fp=#0x10000001f1f' 2024-09-10T08:44:48.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6329 fp=#0x10000001df6' 2024-09-10T08:44:48.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6361 fp=#0x100000032ea' 2024-09-10T08:44:48.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6393 fp=#0x10000003316' 2024-09-10T08:44:48.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6425 fp=#0x10000004fd8' 2024-09-10T08:44:48.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6457 fp=#0x10000004d74' 2024-09-10T08:44:48.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6489 fp=#0x10000005373' 2024-09-10T08:44:48.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6521 fp=#0x10000004b18' 2024-09-10T08:44:48.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6553 fp=#0x10000004ab8' 2024-09-10T08:44:48.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6649 fp=#0x100000039ff' 2024-09-10T08:44:48.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6681 fp=#0x10000004fb1' 2024-09-10T08:44:48.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6713 fp=#0x10000003b2c' 2024-09-10T08:44:48.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6745 fp=#0x10000003ccc' 2024-09-10T08:44:48.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6777 fp=#0x10000003d17' 2024-09-10T08:44:48.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6809 fp=#0x1000000533e' 2024-09-10T08:44:48.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6841 fp=#0x10000004591' 2024-09-10T08:44:48.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6873 fp=#0x1000000461f' 2024-09-10T08:44:48.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6905 fp=#0x100000048b6' 2024-09-10T08:44:48.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6937 fp=#0x10000004669' 2024-09-10T08:44:48.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6969 fp=#0x10000003802' 2024-09-10T08:44:48.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7001 fp=#0x100000042a6' 2024-09-10T08:44:48.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7033 fp=#0x1000000055c' 2024-09-10T08:44:48.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7065 fp=#0x10000004fbc' 2024-09-10T08:44:48.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7097 fp=#0x10000005077' 2024-09-10T08:44:48.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7129 fp=#0x10000000666' 2024-09-10T08:44:48.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7161 fp=#0x100000005a8' 2024-09-10T08:44:48.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7225 fp=#0x100000050b7' 2024-09-10T08:44:48.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7257 fp=#0x1000000514c' 2024-09-10T08:44:48.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7353 fp=#0x10000005094' 2024-09-10T08:44:48.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7385 fp=#0x1000000508a' 2024-09-10T08:44:48.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7417 fp=#0x10000004a6d' 2024-09-10T08:44:48.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7449 fp=#0x10000004734' 2024-09-10T08:44:48.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7481 fp=#0x1000000472a' 2024-09-10T08:44:48.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7513 fp=#0x10000003ab2' 2024-09-10T08:44:48.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7545 fp=#0x10000001eeb' 2024-09-10T08:44:48.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7577 fp=#0x10000005166' 2024-09-10T08:44:48.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7609 fp=#0x1000000138d' 2024-09-10T08:44:48.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7673 fp=#0x10000001344' 2024-09-10T08:44:48.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7737 fp=#0x10000004f35' 2024-09-10T08:44:48.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7769 fp=#0x10000004de1' 2024-09-10T08:44:48.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7801 fp=#0x1000000163f' 2024-09-10T08:44:48.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7833 fp=#0x100000014b3' 2024-09-10T08:44:48.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7865 fp=#0x10000001cff' 2024-09-10T08:44:48.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7897 fp=#0x10000001544' 2024-09-10T08:44:48.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7929 fp=#0x100000013ae' 2024-09-10T08:44:48.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7961 fp=#0x10000004df9' 2024-09-10T08:44:48.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7993 fp=#0x1000000144d' 2024-09-10T08:44:48.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8025 fp=#0x100000013e9' 2024-09-10T08:44:48.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8057 fp=#0x100000015f4' 2024-09-10T08:44:48.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8089 fp=#0x1000000164b' 2024-09-10T08:44:48.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8121 fp=#0x100000049f8' 2024-09-10T08:44:48.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8153 fp=#0x100000024f3' 2024-09-10T08:44:48.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8185 fp=#0x10000004952' 2024-09-10T08:44:48.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8217 fp=#0x10000002552' 2024-09-10T08:44:48.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8249 fp=#0x1000000527b' 2024-09-10T08:44:48.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8281 fp=#0x10000005249' 2024-09-10T08:44:48.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8313 fp=#0x100000043ef' 2024-09-10T08:44:48.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8345 fp=#0x100000041fd' 2024-09-10T08:44:48.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8377 fp=#0x1000000413a' 2024-09-10T08:44:48.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8409 fp=#0x1000000452b' 2024-09-10T08:44:48.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8441 fp=#0x1000000523f' 2024-09-10T08:44:48.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8473 fp=#0x10000004548' 2024-09-10T08:44:48.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8505 fp=#0x1000000454c' 2024-09-10T08:44:48.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8537 fp=#0x10000004568' 2024-09-10T08:44:48.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8569 fp=#0x10000003ee9' 2024-09-10T08:44:48.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8601 fp=#0x10000003dd1' 2024-09-10T08:44:48.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8633 fp=#0x100000040c6' 2024-09-10T08:44:48.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8665 fp=#0x10000003c2e' 2024-09-10T08:44:48.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8697 fp=#0x10000004eee' 2024-09-10T08:44:48.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8729 fp=#0x10000003e54' 2024-09-10T08:44:48.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8761 fp=#0x10000004f3f' 2024-09-10T08:44:48.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8793 fp=#0x10000003f44' 2024-09-10T08:44:48.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8825 fp=#0x10000003d7c' 2024-09-10T08:44:48.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8857 fp=#0x10000004452' 2024-09-10T08:44:48.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8889 fp=#0x100000022dc' 2024-09-10T08:44:48.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8921 fp=#0x10000004475' 2024-09-10T08:44:48.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8953 fp=#0x10000001bba' 2024-09-10T08:44:48.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8985 fp=#0x10000001b88' 2024-09-10T08:44:48.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9017 fp=#0x10000004113' 2024-09-10T08:44:48.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9049 fp=#0x10000004102' 2024-09-10T08:44:48.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9081 fp=#0x10000001fe7' 2024-09-10T08:44:48.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9113 fp=#0x10000004276' 2024-09-10T08:44:48.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9145 fp=#0x10000000d3c' 2024-09-10T08:44:48.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9177 fp=#0x10000004142' 2024-09-10T08:44:48.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9209 fp=#0x10000004123' 2024-09-10T08:44:48.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9273 fp=#0x10000001c70' 2024-09-10T08:44:48.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9305 fp=#0x10000004cab' 2024-09-10T08:44:48.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9337 fp=#0x10000004e63' 2024-09-10T08:44:48.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9369 fp=#0x10000000c18' 2024-09-10T08:44:48.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9401 fp=#0x10000001c42' 2024-09-10T08:44:48.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9433 fp=#0x10000000ba3' 2024-09-10T08:44:48.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9465 fp=#0x10000004dfb' 2024-09-10T08:44:48.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9497 fp=#0x10000004968' 2024-09-10T08:44:48.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9529 fp=#0x10000000ec8' 2024-09-10T08:44:48.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9561 fp=#0x10000004962' 2024-09-10T08:44:48.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9593 fp=#0x10000001e9c' 2024-09-10T08:44:48.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9625 fp=#0x10000000eb4' 2024-09-10T08:44:48.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9657 fp=#0x10000004910' 2024-09-10T08:44:48.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9689 fp=#0x10000004779' 2024-09-10T08:44:48.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9721 fp=#0x100000023dd' 2024-09-10T08:44:48.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9753 fp=#0x1000000173c' 2024-09-10T08:44:48.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9785 fp=#0x1000000199b' 2024-09-10T08:44:48.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9817 fp=#0x10000001abf' 2024-09-10T08:44:48.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9849 fp=#0x100000018cf' 2024-09-10T08:44:48.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9881 fp=#0x1000000432e' 2024-09-10T08:44:48.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9913 fp=#0x100000019c2' 2024-09-10T08:44:48.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9945 fp=#0x10000004a7a' 2024-09-10T08:44:48.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10009 fp=#0x1000000192e' 2024-09-10T08:44:48.307 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10041 fp=#0x100000016f8' 2024-09-10T08:44:48.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10073 fp=#0x10000001574' 2024-09-10T08:44:48.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10105 fp=#0x100000038a4' 2024-09-10T08:44:48.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10137 fp=#0x100000038fa' 2024-09-10T08:44:48.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10169 fp=#0x10000003ae4' 2024-09-10T08:44:48.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10201 fp=#0x10000001e17' 2024-09-10T08:44:48.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10233 fp=#0x100000045c6' 2024-09-10T08:44:48.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10265 fp=#0x10000002360' 2024-09-10T08:44:48.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10297 fp=#0x10000003a84' 2024-09-10T08:44:48.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10329 fp=#0x10000004d8c' 2024-09-10T08:44:48.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10393 fp=#0x100000013f7' 2024-09-10T08:44:48.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10425 fp=#0x10000003b19' 2024-09-10T08:44:48.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10457 fp=#0x10000001352' 2024-09-10T08:44:48.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10489 fp=#0x10000001362' 2024-09-10T08:44:48.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10521 fp=#0x1000000228f' 2024-09-10T08:44:48.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10553 fp=#0x100000013fe' 2024-09-10T08:44:48.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10617 fp=#0x10000000dda' 2024-09-10T08:44:48.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10649 fp=#0x10000004a1c' 2024-09-10T08:44:48.315 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10745 fp=#0x1000000489e' 2024-09-10T08:44:48.315 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10777 fp=#0x100000008b4' 2024-09-10T08:44:48.316 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10809 fp=#0x10000000a81' 2024-09-10T08:44:48.316 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10841 fp=#0x100000006ce' 2024-09-10T08:44:48.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10937 fp=#0x10000000721' 2024-09-10T08:44:48.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10969 fp=#0x10000003812' 2024-09-10T08:44:48.318 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11001 fp=#0x1000000200e' 2024-09-10T08:44:48.318 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11033 fp=#0x10000003b2d' 2024-09-10T08:44:48.318 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11065 fp=#0x10000003b17' 2024-09-10T08:44:48.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11097 fp=#0x1000000530d' 2024-09-10T08:44:48.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11129 fp=#0x100000020d1' 2024-09-10T08:44:48.319 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11161 fp=#0x10000003f77' 2024-09-10T08:44:48.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11193 fp=#0x10000002109' 2024-09-10T08:44:48.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11225 fp=#0x10000003e1b' 2024-09-10T08:44:48.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11257 fp=#0x100000020d5' 2024-09-10T08:44:48.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11289 fp=#0x10000003e97' 2024-09-10T08:44:48.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11321 fp=#0x10000001d45' 2024-09-10T08:44:48.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11353 fp=#0x10000003f10' 2024-09-10T08:44:48.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11385 fp=#0x10000003f29' 2024-09-10T08:44:48.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11417 fp=#0x10000003da5' 2024-09-10T08:44:48.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11449 fp=#0x10000001b5b' 2024-09-10T08:44:48.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11481 fp=#0x10000001ccc' 2024-09-10T08:44:48.323 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11513 fp=#0x10000001b50' 2024-09-10T08:44:48.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11609 fp=#0x100000051a8' 2024-09-10T08:44:48.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11705 fp=#0x10000000a3a' 2024-09-10T08:44:48.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11737 fp=#0x10000000881' 2024-09-10T08:44:48.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11769 fp=#0x10000000938' 2024-09-10T08:44:48.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11801 fp=#0x100000015ca' 2024-09-10T08:44:48.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11833 fp=#0x100000009cc' 2024-09-10T08:44:48.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11865 fp=#0x100000009c0' 2024-09-10T08:44:48.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11897 fp=#0x10000001144' 2024-09-10T08:44:48.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11929 fp=#0x10000004e7d' 2024-09-10T08:44:48.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11961 fp=#0x1000000529b' 2024-09-10T08:44:48.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11993 fp=#0x10000001141' 2024-09-10T08:44:48.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12025 fp=#0x10000001124' 2024-09-10T08:44:48.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12089 fp=#0x10000004447' 2024-09-10T08:44:48.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12121 fp=#0x10000004579' 2024-09-10T08:44:48.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12153 fp=#0x10000004436' 2024-09-10T08:44:48.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12185 fp=#0x100000043a2' 2024-09-10T08:44:48.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12217 fp=#0x100000051f8' 2024-09-10T08:44:48.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12249 fp=#0x100000020a7' 2024-09-10T08:44:48.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12281 fp=#0x1000000112e' 2024-09-10T08:44:48.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12313 fp=#0x10000000c30' 2024-09-10T08:44:48.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12345 fp=#0x10000000d67' 2024-09-10T08:44:48.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12377 fp=#0x10000003add' 2024-09-10T08:44:48.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12409 fp=#0x10000003bcc' 2024-09-10T08:44:48.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12441 fp=#0x10000000c9e' 2024-09-10T08:44:48.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12473 fp=#0x10000004757' 2024-09-10T08:44:48.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12505 fp=#0x10000004c70' 2024-09-10T08:44:48.334 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12569 fp=#0x10000004ae7' 2024-09-10T08:44:48.334 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12601 fp=#0x10000004b2a' 2024-09-10T08:44:48.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12633 fp=#0x10000004c43' 2024-09-10T08:44:48.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12665 fp=#0x10000004755' 2024-09-10T08:44:48.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12697 fp=#0x100000053c6' 2024-09-10T08:44:48.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12729 fp=#0x10000004965' 2024-09-10T08:44:48.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12761 fp=#0x1000000498e' 2024-09-10T08:44:48.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12793 fp=#0x1000000230e' 2024-09-10T08:44:48.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12825 fp=#0x1000000536e' 2024-09-10T08:44:48.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12857 fp=#0x10000003dbb' 2024-09-10T08:44:48.340 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12889 fp=#0x10000002132' 2024-09-10T08:44:48.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12921 fp=#0x100000052b5' 2024-09-10T08:44:48.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12953 fp=#0x100000044ed' 2024-09-10T08:44:48.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12985 fp=#0x100000053c7' 2024-09-10T08:44:48.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13017 fp=#0x10000002765' 2024-09-10T08:44:48.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13105 fp=#0x10000002342' 2024-09-10T08:44:48.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13137 fp=#0x10000005399' 2024-09-10T08:44:48.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:258 fp=#0x10000001fcc' 2024-09-10T08:44:48.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:322 fp=#0x1000000387e' 2024-09-10T08:44:48.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:378 fp=#0x10000004acc' 2024-09-10T08:44:48.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:474 fp=#0x10000004b44' 2024-09-10T08:44:48.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:570 fp=#0x10000004127' 2024-09-10T08:44:48.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:602 fp=#0x10000003a0b' 2024-09-10T08:44:48.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:794 fp=#0x10000000d9d' 2024-09-10T08:44:48.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:858 fp=#0x10000003fed' 2024-09-10T08:44:48.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:890 fp=#0x10000004420' 2024-09-10T08:44:48.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:922 fp=#0x10000004535' 2024-09-10T08:44:48.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:954 fp=#0x1000000445d' 2024-09-10T08:44:48.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:986 fp=#0x10000004006' 2024-09-10T08:44:48.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1082 fp=#0x100000042b2' 2024-09-10T08:44:48.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1146 fp=#0x1000000429a' 2024-09-10T08:44:48.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1178 fp=#0x10000004268' 2024-09-10T08:44:48.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1242 fp=#0x1000000361e' 2024-09-10T08:44:48.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1306 fp=#0x100000032eb' 2024-09-10T08:44:48.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1338 fp=#0x10000003565' 2024-09-10T08:44:48.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1658 fp=#0x100000045ef' 2024-09-10T08:44:48.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1690 fp=#0x10000004834' 2024-09-10T08:44:48.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1946 fp=#0x10000003e75' 2024-09-10T08:44:48.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2042 fp=#0x100000021ed' 2024-09-10T08:44:48.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2074 fp=#0x10000002148' 2024-09-10T08:44:48.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2170 fp=#0x1000000215a' 2024-09-10T08:44:48.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2234 fp=#0x100000026ae' 2024-09-10T08:44:48.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2298 fp=#0x100000046f8' 2024-09-10T08:44:48.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2586 fp=#0x10000001901' 2024-09-10T08:44:48.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2618 fp=#0x100000011f1' 2024-09-10T08:44:48.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2682 fp=#0x100000010c1' 2024-09-10T08:44:48.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2714 fp=#0x10000001894' 2024-09-10T08:44:48.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2746 fp=#0x10000003b5f' 2024-09-10T08:44:48.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2778 fp=#0x100000033c6' 2024-09-10T08:44:48.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2810 fp=#0x100000035f5' 2024-09-10T08:44:48.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2842 fp=#0x10000003483' 2024-09-10T08:44:48.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2874 fp=#0x10000003555' 2024-09-10T08:44:48.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2906 fp=#0x100000033ae' 2024-09-10T08:44:48.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2938 fp=#0x100000033c9' 2024-09-10T08:44:48.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3002 fp=#0x100000018b6' 2024-09-10T08:44:48.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3066 fp=#0x10000004a43' 2024-09-10T08:44:48.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3098 fp=#0x100000018ea' 2024-09-10T08:44:48.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3162 fp=#0x10000000e65' 2024-09-10T08:44:48.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3226 fp=#0x1000000074a' 2024-09-10T08:44:48.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3258 fp=#0x100000006fc' 2024-09-10T08:44:48.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3290 fp=#0x10000000758' 2024-09-10T08:44:48.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3322 fp=#0x1000000076d' 2024-09-10T08:44:48.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3354 fp=#0x10000000750' 2024-09-10T08:44:48.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3386 fp=#0x10000000769' 2024-09-10T08:44:48.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3418 fp=#0x10000000752' 2024-09-10T08:44:48.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3514 fp=#0x100000040dd' 2024-09-10T08:44:48.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3546 fp=#0x100000040ab' 2024-09-10T08:44:48.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3578 fp=#0x10000003ce1' 2024-09-10T08:44:48.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3642 fp=#0x10000004ad4' 2024-09-10T08:44:48.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3674 fp=#0x10000004cc6' 2024-09-10T08:44:48.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3706 fp=#0x10000004bed' 2024-09-10T08:44:48.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3738 fp=#0x10000003e80' 2024-09-10T08:44:48.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3770 fp=#0x10000004313' 2024-09-10T08:44:48.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3802 fp=#0x10000003ed7' 2024-09-10T08:44:48.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3834 fp=#0x10000002547' 2024-09-10T08:44:48.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3866 fp=#0x100000024e3' 2024-09-10T08:44:48.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3930 fp=#0x100000006bc' 2024-09-10T08:44:48.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3962 fp=#0x1000000068d' 2024-09-10T08:44:48.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3994 fp=#0x10000000587' 2024-09-10T08:44:48.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4026 fp=#0x1000000141d' 2024-09-10T08:44:48.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4058 fp=#0x10000001004' 2024-09-10T08:44:48.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4090 fp=#0x10000004a86' 2024-09-10T08:44:48.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4122 fp=#0x10000001d36' 2024-09-10T08:44:48.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4154 fp=#0x1000000466b' 2024-09-10T08:44:48.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4186 fp=#0x10000000f86' 2024-09-10T08:44:48.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4250 fp=#0x100000039a6' 2024-09-10T08:44:48.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4282 fp=#0x10000004068' 2024-09-10T08:44:48.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4314 fp=#0x10000003b16' 2024-09-10T08:44:48.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4346 fp=#0x100000040f3' 2024-09-10T08:44:48.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4378 fp=#0x10000001fff' 2024-09-10T08:44:48.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4442 fp=#0x10000001fe6' 2024-09-10T08:44:48.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4474 fp=#0x10000004cec' 2024-09-10T08:44:48.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4506 fp=#0x10000001ce7' 2024-09-10T08:44:48.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4538 fp=#0x10000003b5a' 2024-09-10T08:44:48.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4570 fp=#0x1000000510e' 2024-09-10T08:44:48.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4602 fp=#0x100000012b2' 2024-09-10T08:44:48.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4634 fp=#0x1000000150f' 2024-09-10T08:44:48.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4666 fp=#0x10000003944' 2024-09-10T08:44:48.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4698 fp=#0x10000001478' 2024-09-10T08:44:48.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4730 fp=#0x10000001364' 2024-09-10T08:44:48.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4762 fp=#0x100000042d8' 2024-09-10T08:44:48.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4794 fp=#0x10000001299' 2024-09-10T08:44:48.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4826 fp=#0x10000003853' 2024-09-10T08:44:48.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4858 fp=#0x100000037e9' 2024-09-10T08:44:48.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4890 fp=#0x10000004aad' 2024-09-10T08:44:48.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4922 fp=#0x100000037bc' 2024-09-10T08:44:48.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4954 fp=#0x1000000532b' 2024-09-10T08:44:48.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4986 fp=#0x10000004fdc' 2024-09-10T08:44:48.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5018 fp=#0x10000005091' 2024-09-10T08:44:48.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5050 fp=#0x10000005075' 2024-09-10T08:44:48.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5082 fp=#0x100000051e4' 2024-09-10T08:44:48.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5114 fp=#0x100000050d6' 2024-09-10T08:44:48.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5146 fp=#0x10000001da8' 2024-09-10T08:44:48.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5178 fp=#0x10000001c11' 2024-09-10T08:44:48.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5210 fp=#0x10000003ac3' 2024-09-10T08:44:48.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5242 fp=#0x10000001de4' 2024-09-10T08:44:48.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5274 fp=#0x10000004aa1' 2024-09-10T08:44:48.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5306 fp=#0x10000003705' 2024-09-10T08:44:48.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5338 fp=#0x100000035ae' 2024-09-10T08:44:48.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5370 fp=#0x10000003782' 2024-09-10T08:44:48.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5402 fp=#0x10000003da9' 2024-09-10T08:44:48.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5434 fp=#0x100000036c0' 2024-09-10T08:44:48.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5594 fp=#0x100000050fd' 2024-09-10T08:44:48.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5626 fp=#0x10000000f55' 2024-09-10T08:44:48.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5658 fp=#0x10000000f71' 2024-09-10T08:44:48.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5690 fp=#0x10000001009' 2024-09-10T08:44:48.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5722 fp=#0x10000000fae' 2024-09-10T08:44:48.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5754 fp=#0x100000050d8' 2024-09-10T08:44:48.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5786 fp=#0x100000042dd' 2024-09-10T08:44:48.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5818 fp=#0x10000004379' 2024-09-10T08:44:48.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5850 fp=#0x100000045ac' 2024-09-10T08:44:48.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5882 fp=#0x1000000443e' 2024-09-10T08:44:48.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5914 fp=#0x100000042e3' 2024-09-10T08:44:48.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5946 fp=#0x1000000494b' 2024-09-10T08:44:48.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5978 fp=#0x100000045e2' 2024-09-10T08:44:48.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6170 fp=#0x1000000351d' 2024-09-10T08:44:48.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6234 fp=#0x10000003461' 2024-09-10T08:44:48.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6266 fp=#0x1000000352e' 2024-09-10T08:44:48.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6298 fp=#0x10000003394' 2024-09-10T08:44:48.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6330 fp=#0x10000003310' 2024-09-10T08:44:48.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6362 fp=#0x100000033c3' 2024-09-10T08:44:48.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6394 fp=#0x10000001eaf' 2024-09-10T08:44:48.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6426 fp=#0x10000005380' 2024-09-10T08:44:48.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6458 fp=#0x10000004b8c' 2024-09-10T08:44:48.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6490 fp=#0x10000004d2b' 2024-09-10T08:44:48.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6522 fp=#0x10000004f3e' 2024-09-10T08:44:48.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6554 fp=#0x10000004cd9' 2024-09-10T08:44:48.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6650 fp=#0x10000003d67' 2024-09-10T08:44:48.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6682 fp=#0x10000003af6' 2024-09-10T08:44:48.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6714 fp=#0x10000003a4b' 2024-09-10T08:44:48.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6746 fp=#0x10000003a43' 2024-09-10T08:44:48.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6778 fp=#0x10000005053' 2024-09-10T08:44:48.410 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6810 fp=#0x10000005311' 2024-09-10T08:44:48.410 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6842 fp=#0x10000004929' 2024-09-10T08:44:48.410 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6874 fp=#0x100000052a4' 2024-09-10T08:44:48.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6906 fp=#0x1000000528b' 2024-09-10T08:44:48.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6938 fp=#0x10000004679' 2024-09-10T08:44:48.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6970 fp=#0x10000000640' 2024-09-10T08:44:48.412 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7002 fp=#0x10000004fc6' 2024-09-10T08:44:48.412 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7034 fp=#0x10000004fe1' 2024-09-10T08:44:48.412 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7066 fp=#0x10000005033' 2024-09-10T08:44:48.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7098 fp=#0x100000006a2' 2024-09-10T08:44:48.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7130 fp=#0x10000001e4a' 2024-09-10T08:44:48.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7162 fp=#0x10000000620' 2024-09-10T08:44:48.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7226 fp=#0x100000050e3' 2024-09-10T08:44:48.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7290 fp=#0x10000005080' 2024-09-10T08:44:48.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7354 fp=#0x100000050bd' 2024-09-10T08:44:48.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7386 fp=#0x10000003e18' 2024-09-10T08:44:48.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7418 fp=#0x10000004797' 2024-09-10T08:44:48.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7450 fp=#0x100000048c3' 2024-09-10T08:44:48.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7482 fp=#0x10000004911' 2024-09-10T08:44:48.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7514 fp=#0x100000048ef' 2024-09-10T08:44:48.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7546 fp=#0x100000012f2' 2024-09-10T08:44:48.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7578 fp=#0x10000005169' 2024-09-10T08:44:48.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7610 fp=#0x1000000519c' 2024-09-10T08:44:48.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7706 fp=#0x10000002435' 2024-09-10T08:44:48.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7738 fp=#0x10000004dd0' 2024-09-10T08:44:48.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7770 fp=#0x10000004e0c' 2024-09-10T08:44:48.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7802 fp=#0x10000001736' 2024-09-10T08:44:48.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7834 fp=#0x10000004e73' 2024-09-10T08:44:48.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7866 fp=#0x10000004d0b' 2024-09-10T08:44:48.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7898 fp=#0x10000001404' 2024-09-10T08:44:48.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7930 fp=#0x10000001492' 2024-09-10T08:44:48.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7994 fp=#0x10000004cd4' 2024-09-10T08:44:48.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8026 fp=#0x10000001586' 2024-09-10T08:44:48.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8058 fp=#0x10000001e72' 2024-09-10T08:44:48.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8090 fp=#0x10000004bb5' 2024-09-10T08:44:48.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8122 fp=#0x10000001640' 2024-09-10T08:44:48.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8154 fp=#0x10000002716' 2024-09-10T08:44:48.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8186 fp=#0x10000002532' 2024-09-10T08:44:48.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8218 fp=#0x100000025c5' 2024-09-10T08:44:48.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8250 fp=#0x10000004638' 2024-09-10T08:44:48.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8282 fp=#0x1000000438d' 2024-09-10T08:44:48.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8314 fp=#0x10000005240' 2024-09-10T08:44:48.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8346 fp=#0x10000004472' 2024-09-10T08:44:48.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8378 fp=#0x10000004629' 2024-09-10T08:44:48.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8410 fp=#0x100000043bf' 2024-09-10T08:44:48.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8442 fp=#0x100000043a7' 2024-09-10T08:44:48.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8474 fp=#0x1000000417e' 2024-09-10T08:44:48.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8506 fp=#0x10000003fac' 2024-09-10T08:44:48.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8538 fp=#0x100000041cb' 2024-09-10T08:44:48.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8570 fp=#0x10000004522' 2024-09-10T08:44:48.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8602 fp=#0x10000003e9a' 2024-09-10T08:44:48.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8634 fp=#0x10000003c19' 2024-09-10T08:44:48.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8666 fp=#0x10000004edd' 2024-09-10T08:44:48.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8698 fp=#0x1000000531d' 2024-09-10T08:44:48.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8730 fp=#0x10000005303' 2024-09-10T08:44:48.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8762 fp=#0x10000003db2' 2024-09-10T08:44:48.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8794 fp=#0x10000003e91' 2024-09-10T08:44:48.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8826 fp=#0x10000003dac' 2024-09-10T08:44:48.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8858 fp=#0x10000001bc8' 2024-09-10T08:44:48.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8890 fp=#0x10000001b44' 2024-09-10T08:44:48.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8922 fp=#0x10000002251' 2024-09-10T08:44:48.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8954 fp=#0x1000000435b' 2024-09-10T08:44:48.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8986 fp=#0x10000001d41' 2024-09-10T08:44:48.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9018 fp=#0x10000000d9e' 2024-09-10T08:44:48.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9050 fp=#0x1000000411a' 2024-09-10T08:44:48.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9114 fp=#0x10000004242' 2024-09-10T08:44:48.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9146 fp=#0x10000000d6a' 2024-09-10T08:44:48.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9178 fp=#0x10000004895' 2024-09-10T08:44:48.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9210 fp=#0x10000000d6e' 2024-09-10T08:44:48.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9274 fp=#0x10000000c22' 2024-09-10T08:44:48.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9306 fp=#0x10000004cb1' 2024-09-10T08:44:48.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9338 fp=#0x10000004cc0' 2024-09-10T08:44:48.436 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9370 fp=#0x10000000b5a' 2024-09-10T08:44:48.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9402 fp=#0x10000000af3' 2024-09-10T08:44:48.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9434 fp=#0x10000000b87' 2024-09-10T08:44:48.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9466 fp=#0x10000004c9d' 2024-09-10T08:44:48.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9498 fp=#0x10000000edd' 2024-09-10T08:44:48.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9530 fp=#0x100000019fd' 2024-09-10T08:44:48.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9594 fp=#0x10000000ed6' 2024-09-10T08:44:48.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9626 fp=#0x100000019f0' 2024-09-10T08:44:48.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9658 fp=#0x100000047eb' 2024-09-10T08:44:48.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9690 fp=#0x10000004773' 2024-09-10T08:44:48.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9722 fp=#0x10000001a8f' 2024-09-10T08:44:48.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9754 fp=#0x100000018e6' 2024-09-10T08:44:48.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9786 fp=#0x100000044c2' 2024-09-10T08:44:48.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9818 fp=#0x10000001790' 2024-09-10T08:44:48.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9850 fp=#0x10000004421' 2024-09-10T08:44:48.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9882 fp=#0x10000001910' 2024-09-10T08:44:48.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9914 fp=#0x100000015ed' 2024-09-10T08:44:48.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9946 fp=#0x10000001999' 2024-09-10T08:44:48.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9978 fp=#0x1000000160e' 2024-09-10T08:44:48.443 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10010 fp=#0x10000001669' 2024-09-10T08:44:48.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10042 fp=#0x100000019b5' 2024-09-10T08:44:48.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10074 fp=#0x10000002697' 2024-09-10T08:44:48.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10106 fp=#0x10000001f56' 2024-09-10T08:44:48.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10138 fp=#0x10000003953' 2024-09-10T08:44:48.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10170 fp=#0x10000001d9f' 2024-09-10T08:44:48.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10202 fp=#0x100000039a1' 2024-09-10T08:44:48.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10234 fp=#0x100000038ff' 2024-09-10T08:44:48.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10266 fp=#0x10000002408' 2024-09-10T08:44:48.446 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10298 fp=#0x100000023f9' 2024-09-10T08:44:48.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10330 fp=#0x100000023a6' 2024-09-10T08:44:48.447 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10394 fp=#0x10000001425' 2024-09-10T08:44:48.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10426 fp=#0x10000001358' 2024-09-10T08:44:48.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10458 fp=#0x100000039e4' 2024-09-10T08:44:48.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10490 fp=#0x100000012e6' 2024-09-10T08:44:48.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10554 fp=#0x10000003a4a' 2024-09-10T08:44:48.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10586 fp=#0x100000008cd' 2024-09-10T08:44:48.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10682 fp=#0x10000004902' 2024-09-10T08:44:48.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10714 fp=#0x100000007ca' 2024-09-10T08:44:48.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10810 fp=#0x10000004321' 2024-09-10T08:44:48.452 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10874 fp=#0x10000000729' 2024-09-10T08:44:48.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10938 fp=#0x10000002009' 2024-09-10T08:44:48.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10970 fp=#0x10000001cba' 2024-09-10T08:44:48.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11002 fp=#0x100000005d0' 2024-09-10T08:44:48.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11034 fp=#0x10000003901' 2024-09-10T08:44:48.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11066 fp=#0x10000003dc5' 2024-09-10T08:44:48.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11098 fp=#0x10000004a8e' 2024-09-10T08:44:48.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11130 fp=#0x10000003ba6' 2024-09-10T08:44:48.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11162 fp=#0x10000002082' 2024-09-10T08:44:48.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11226 fp=#0x100000020a0' 2024-09-10T08:44:48.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11258 fp=#0x100000021e8' 2024-09-10T08:44:48.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11290 fp=#0x10000003eea' 2024-09-10T08:44:48.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11322 fp=#0x10000003f65' 2024-09-10T08:44:48.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11354 fp=#0x100000019aa' 2024-09-10T08:44:48.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11386 fp=#0x100000019a3' 2024-09-10T08:44:48.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11418 fp=#0x10000003f6d' 2024-09-10T08:44:48.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11450 fp=#0x100000019df' 2024-09-10T08:44:48.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11482 fp=#0x10000003d6e' 2024-09-10T08:44:48.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11514 fp=#0x10000003f9d' 2024-09-10T08:44:48.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11706 fp=#0x100000008d3' 2024-09-10T08:44:48.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11738 fp=#0x10000000874' 2024-09-10T08:44:48.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11770 fp=#0x100000009fa' 2024-09-10T08:44:48.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11802 fp=#0x10000004e36' 2024-09-10T08:44:48.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11834 fp=#0x100000041a8' 2024-09-10T08:44:48.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11866 fp=#0x10000000870' 2024-09-10T08:44:48.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11930 fp=#0x100000010f8' 2024-09-10T08:44:48.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11962 fp=#0x10000005306' 2024-09-10T08:44:48.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11994 fp=#0x10000001493' 2024-09-10T08:44:48.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12026 fp=#0x1000000530b' 2024-09-10T08:44:48.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12058 fp=#0x10000001118' 2024-09-10T08:44:48.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12090 fp=#0x1000000445e' 2024-09-10T08:44:48.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12122 fp=#0x10000004406' 2024-09-10T08:44:48.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12154 fp=#0x10000004d9d' 2024-09-10T08:44:48.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12186 fp=#0x100000044b4' 2024-09-10T08:44:48.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12218 fp=#0x10000000d57' 2024-09-10T08:44:48.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12250 fp=#0x10000003ae6' 2024-09-10T08:44:48.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12282 fp=#0x10000003a86' 2024-09-10T08:44:48.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12346 fp=#0x10000000cd1' 2024-09-10T08:44:48.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12378 fp=#0x10000000c26' 2024-09-10T08:44:48.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12410 fp=#0x100000052fe' 2024-09-10T08:44:48.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12442 fp=#0x10000003b5c' 2024-09-10T08:44:48.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12474 fp=#0x10000004bad' 2024-09-10T08:44:48.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12506 fp=#0x10000004b92' 2024-09-10T08:44:48.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12538 fp=#0x10000002575' 2024-09-10T08:44:48.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12570 fp=#0x10000004d3f' 2024-09-10T08:44:48.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12602 fp=#0x10000004b25' 2024-09-10T08:44:48.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12634 fp=#0x10000004c29' 2024-09-10T08:44:48.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12666 fp=#0x100000049de' 2024-09-10T08:44:48.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12698 fp=#0x10000004857' 2024-09-10T08:44:48.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12730 fp=#0x100000053fd' 2024-09-10T08:44:48.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12762 fp=#0x10000005340' 2024-09-10T08:44:48.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12794 fp=#0x10000005420' 2024-09-10T08:44:48.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12826 fp=#0x10000005374' 2024-09-10T08:44:48.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12858 fp=#0x10000005389' 2024-09-10T08:44:48.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12890 fp=#0x10000005325' 2024-09-10T08:44:48.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12922 fp=#0x100000053cf' 2024-09-10T08:44:48.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12954 fp=#0x100000039e8' 2024-09-10T08:44:48.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13018 fp=#0x100000027a4' 2024-09-10T08:44:48.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13042 fp=#0x10000002321' 2024-09-10T08:44:48.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13074 fp=#0x1000000490b' 2024-09-10T08:44:48.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13106 fp=#0x100000053c5' 2024-09-10T08:44:48.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13170 fp=#0x10000003ff8' 2024-09-10T08:44:48.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:156 fp=#0x100000001fb' 2024-09-10T08:44:48.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:259 fp=#0x10000002589' 2024-09-10T08:44:48.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:291 fp=#0x10000002ab9' 2024-09-10T08:44:48.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:323 fp=#0x10000002034' 2024-09-10T08:44:48.478 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:379 fp=#0x100000046f4' 2024-09-10T08:44:48.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:411 fp=#0x10000004adf' 2024-09-10T08:44:48.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:443 fp=#0x10000004af9' 2024-09-10T08:44:48.479 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:475 fp=#0x10000004b79' 2024-09-10T08:44:48.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:539 fp=#0x10000003497' 2024-09-10T08:44:48.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:795 fp=#0x10000003fc8' 2024-09-10T08:44:48.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:827 fp=#0x10000000df3' 2024-09-10T08:44:48.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:859 fp=#0x100000044f8' 2024-09-10T08:44:48.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:891 fp=#0x10000000e46' 2024-09-10T08:44:48.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:923 fp=#0x10000004465' 2024-09-10T08:44:48.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:987 fp=#0x10000000dd8' 2024-09-10T08:44:48.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1115 fp=#0x1000000418a' 2024-09-10T08:44:48.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1147 fp=#0x100000041cc' 2024-09-10T08:44:48.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1179 fp=#0x100000040db' 2024-09-10T08:44:48.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1243 fp=#0x10000001f65' 2024-09-10T08:44:48.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1307 fp=#0x10000003672' 2024-09-10T08:44:48.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1339 fp=#0x10000001f7c' 2024-09-10T08:44:48.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1371 fp=#0x10000003279' 2024-09-10T08:44:48.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1531 fp=#0x10000003e83' 2024-09-10T08:44:48.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1659 fp=#0x10000004a3c' 2024-09-10T08:44:48.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1691 fp=#0x10000003356' 2024-09-10T08:44:48.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1787 fp=#0x100000034d4' 2024-09-10T08:44:48.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1819 fp=#0x10000002763' 2024-09-10T08:44:48.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1883 fp=#0x100000027b4' 2024-09-10T08:44:48.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2107 fp=#0x1000000212a' 2024-09-10T08:44:48.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2139 fp=#0x10000001fcf' 2024-09-10T08:44:48.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2235 fp=#0x100000026ef' 2024-09-10T08:44:48.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2299 fp=#0x10000002611' 2024-09-10T08:44:48.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2587 fp=#0x100000011da' 2024-09-10T08:44:48.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2619 fp=#0x10000001132' 2024-09-10T08:44:48.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2651 fp=#0x1000000109a' 2024-09-10T08:44:48.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2715 fp=#0x10000001079' 2024-09-10T08:44:48.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2747 fp=#0x10000003b8e' 2024-09-10T08:44:48.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2779 fp=#0x100000033b4' 2024-09-10T08:44:48.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2843 fp=#0x10000004803' 2024-09-10T08:44:48.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2875 fp=#0x1000000463a' 2024-09-10T08:44:48.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2939 fp=#0x10000003618' 2024-09-10T08:44:48.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3003 fp=#0x10000000dce' 2024-09-10T08:44:48.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3035 fp=#0x100000018d7' 2024-09-10T08:44:48.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3067 fp=#0x1000000193f' 2024-09-10T08:44:48.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3099 fp=#0x10000000e4e' 2024-09-10T08:44:48.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3163 fp=#0x100000018cd' 2024-09-10T08:44:48.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3195 fp=#0x10000004a68' 2024-09-10T08:44:48.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3227 fp=#0x100000007a4' 2024-09-10T08:44:48.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3259 fp=#0x10000001861' 2024-09-10T08:44:48.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3291 fp=#0x1000000075c' 2024-09-10T08:44:48.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3323 fp=#0x10000000feb' 2024-09-10T08:44:48.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3355 fp=#0x10000000785' 2024-09-10T08:44:48.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3387 fp=#0x1000000079d' 2024-09-10T08:44:48.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3419 fp=#0x1000000077e' 2024-09-10T08:44:48.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3515 fp=#0x1000000417a' 2024-09-10T08:44:48.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3547 fp=#0x10000003c9f' 2024-09-10T08:44:48.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3611 fp=#0x1000000409a' 2024-09-10T08:44:48.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3643 fp=#0x10000004caf' 2024-09-10T08:44:48.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3675 fp=#0x10000004b78' 2024-09-10T08:44:48.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3707 fp=#0x10000004bc5' 2024-09-10T08:44:48.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3771 fp=#0x10000002598' 2024-09-10T08:44:48.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3803 fp=#0x10000003e78' 2024-09-10T08:44:48.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3835 fp=#0x10000003f45' 2024-09-10T08:44:48.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3867 fp=#0x10000003ea2' 2024-09-10T08:44:48.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3899 fp=#0x1000000337e' 2024-09-10T08:44:48.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3931 fp=#0x10000000553' 2024-09-10T08:44:48.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3963 fp=#0x10000002065' 2024-09-10T08:44:48.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3995 fp=#0x10000000658' 2024-09-10T08:44:48.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4027 fp=#0x10000003acd' 2024-09-10T08:44:48.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4059 fp=#0x10000004681' 2024-09-10T08:44:48.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4091 fp=#0x10000001e1c' 2024-09-10T08:44:48.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4123 fp=#0x10000000f40' 2024-09-10T08:44:48.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4155 fp=#0x10000000f58' 2024-09-10T08:44:48.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4187 fp=#0x10000000fd2' 2024-09-10T08:44:48.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4251 fp=#0x100000039bf' 2024-09-10T08:44:48.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4283 fp=#0x10000003a39' 2024-09-10T08:44:48.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4315 fp=#0x10000003a22' 2024-09-10T08:44:48.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4347 fp=#0x10000003f25' 2024-09-10T08:44:48.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4379 fp=#0x10000002030' 2024-09-10T08:44:48.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4443 fp=#0x10000001f05' 2024-09-10T08:44:48.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4507 fp=#0x10000004051' 2024-09-10T08:44:48.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4539 fp=#0x10000003b3d' 2024-09-10T08:44:48.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4571 fp=#0x1000000333d' 2024-09-10T08:44:48.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4603 fp=#0x1000000145a' 2024-09-10T08:44:48.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4635 fp=#0x1000000146d' 2024-09-10T08:44:48.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4667 fp=#0x100000015ac' 2024-09-10T08:44:48.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4699 fp=#0x10000001595' 2024-09-10T08:44:48.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4731 fp=#0x10000001558' 2024-09-10T08:44:48.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4763 fp=#0x100000014e1' 2024-09-10T08:44:48.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4795 fp=#0x10000003976' 2024-09-10T08:44:48.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4827 fp=#0x100000036a9' 2024-09-10T08:44:48.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4859 fp=#0x10000004a8c' 2024-09-10T08:44:48.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4891 fp=#0x100000037fb' 2024-09-10T08:44:48.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4923 fp=#0x100000036c3' 2024-09-10T08:44:48.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4955 fp=#0x1000000506e' 2024-09-10T08:44:48.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4987 fp=#0x10000004fc9' 2024-09-10T08:44:48.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5019 fp=#0x100000050b6' 2024-09-10T08:44:48.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5051 fp=#0x100000051e0' 2024-09-10T08:44:48.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5083 fp=#0x100000050ae' 2024-09-10T08:44:48.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5115 fp=#0x100000050ec' 2024-09-10T08:44:48.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5147 fp=#0x10000001e7b' 2024-09-10T08:44:48.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5179 fp=#0x10000001cd1' 2024-09-10T08:44:48.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5211 fp=#0x10000003aba' 2024-09-10T08:44:48.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5243 fp=#0x1000000246f' 2024-09-10T08:44:48.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5275 fp=#0x10000004a85' 2024-09-10T08:44:48.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5307 fp=#0x10000003d3a' 2024-09-10T08:44:48.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5339 fp=#0x10000004a96' 2024-09-10T08:44:48.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5371 fp=#0x10000003596' 2024-09-10T08:44:48.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5403 fp=#0x10000003d7b' 2024-09-10T08:44:48.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5435 fp=#0x100000044f1' 2024-09-10T08:44:48.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5499 fp=#0x100000012b6' 2024-09-10T08:44:48.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5531 fp=#0x10000001292' 2024-09-10T08:44:48.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5595 fp=#0x100000050da' 2024-09-10T08:44:48.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5627 fp=#0x1000000107f' 2024-09-10T08:44:48.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5659 fp=#0x10000000f96' 2024-09-10T08:44:48.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5691 fp=#0x100000020f3' 2024-09-10T08:44:48.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5723 fp=#0x10000000f83' 2024-09-10T08:44:48.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5755 fp=#0x100000050c9' 2024-09-10T08:44:48.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5787 fp=#0x100000044b9' 2024-09-10T08:44:48.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5819 fp=#0x10000004be0' 2024-09-10T08:44:48.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5851 fp=#0x10000004ca2' 2024-09-10T08:44:48.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5883 fp=#0x10000004d62' 2024-09-10T08:44:48.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5915 fp=#0x100000043e6' 2024-09-10T08:44:48.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5947 fp=#0x10000004c86' 2024-09-10T08:44:48.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5979 fp=#0x10000004c7a' 2024-09-10T08:44:48.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6139 fp=#0x100000035dd' 2024-09-10T08:44:48.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6171 fp=#0x10000003617' 2024-09-10T08:44:48.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6235 fp=#0x10000003595' 2024-09-10T08:44:48.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6299 fp=#0x1000000338c' 2024-09-10T08:44:48.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6331 fp=#0x100000033c0' 2024-09-10T08:44:48.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6363 fp=#0x10000001ea2' 2024-09-10T08:44:48.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6395 fp=#0x10000001ece' 2024-09-10T08:44:48.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6427 fp=#0x10000004c9e' 2024-09-10T08:44:48.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6459 fp=#0x10000004dff' 2024-09-10T08:44:48.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6491 fp=#0x10000004fd2' 2024-09-10T08:44:48.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6523 fp=#0x10000004d22' 2024-09-10T08:44:48.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6555 fp=#0x10000004ad1' 2024-09-10T08:44:48.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6619 fp=#0x1000000399b' 2024-09-10T08:44:48.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6651 fp=#0x10000003956' 2024-09-10T08:44:48.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6683 fp=#0x1000000528d' 2024-09-10T08:44:48.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6715 fp=#0x10000004fbe' 2024-09-10T08:44:48.547 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6747 fp=#0x1000000529c' 2024-09-10T08:44:48.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6779 fp=#0x10000004f4a' 2024-09-10T08:44:48.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6811 fp=#0x10000004574' 2024-09-10T08:44:48.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6843 fp=#0x10000004690' 2024-09-10T08:44:48.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6875 fp=#0x10000005344' 2024-09-10T08:44:48.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6907 fp=#0x10000004635' 2024-09-10T08:44:48.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6939 fp=#0x1000000472b' 2024-09-10T08:44:48.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6971 fp=#0x1000000206b' 2024-09-10T08:44:48.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7003 fp=#0x100000005c6' 2024-09-10T08:44:48.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7035 fp=#0x10000000647' 2024-09-10T08:44:48.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7067 fp=#0x10000000632' 2024-09-10T08:44:48.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7099 fp=#0x10000000646' 2024-09-10T08:44:48.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7131 fp=#0x10000004ff5' 2024-09-10T08:44:48.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7163 fp=#0x1000000069a' 2024-09-10T08:44:48.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7195 fp=#0x10000005147' 2024-09-10T08:44:48.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7323 fp=#0x10000004678' 2024-09-10T08:44:48.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7355 fp=#0x100000050a4' 2024-09-10T08:44:48.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7419 fp=#0x1000000351a' 2024-09-10T08:44:48.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7451 fp=#0x100000048a6' 2024-09-10T08:44:48.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7483 fp=#0x100000046cd' 2024-09-10T08:44:48.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7515 fp=#0x100000048ad' 2024-09-10T08:44:48.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7547 fp=#0x100000012de' 2024-09-10T08:44:48.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7611 fp=#0x10000001d2a' 2024-09-10T08:44:48.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7643 fp=#0x100000051d0' 2024-09-10T08:44:48.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7675 fp=#0x10000005177' 2024-09-10T08:44:48.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7739 fp=#0x10000004ded' 2024-09-10T08:44:48.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7771 fp=#0x10000004e69' 2024-09-10T08:44:48.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7803 fp=#0x100000013ca' 2024-09-10T08:44:48.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7835 fp=#0x10000004ea8' 2024-09-10T08:44:48.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7867 fp=#0x10000004d66' 2024-09-10T08:44:48.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7899 fp=#0x10000004de5' 2024-09-10T08:44:48.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7931 fp=#0x100000013f6' 2024-09-10T08:44:48.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7963 fp=#0x10000004e08' 2024-09-10T08:44:48.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7995 fp=#0x100000021ee' 2024-09-10T08:44:48.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8027 fp=#0x10000004df1' 2024-09-10T08:44:48.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8059 fp=#0x100000016d0' 2024-09-10T08:44:48.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8091 fp=#0x10000001661' 2024-09-10T08:44:48.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8123 fp=#0x10000001637' 2024-09-10T08:44:48.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8155 fp=#0x10000002688' 2024-09-10T08:44:48.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8187 fp=#0x10000002567' 2024-09-10T08:44:48.562 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8219 fp=#0x1000000254c' 2024-09-10T08:44:48.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8251 fp=#0x10000004210' 2024-09-10T08:44:48.563 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8283 fp=#0x1000000525e' 2024-09-10T08:44:48.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8315 fp=#0x1000000527c' 2024-09-10T08:44:48.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8347 fp=#0x10000005253' 2024-09-10T08:44:48.564 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8379 fp=#0x100000043fb' 2024-09-10T08:44:48.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8411 fp=#0x10000004633' 2024-09-10T08:44:48.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8443 fp=#0x100000043d5' 2024-09-10T08:44:48.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8475 fp=#0x10000003dab' 2024-09-10T08:44:48.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8507 fp=#0x10000004274' 2024-09-10T08:44:48.566 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8539 fp=#0x1000000453f' 2024-09-10T08:44:48.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8571 fp=#0x100000041c3' 2024-09-10T08:44:48.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8603 fp=#0x10000003ead' 2024-09-10T08:44:48.567 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8635 fp=#0x10000003fbe' 2024-09-10T08:44:48.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8667 fp=#0x10000003f1b' 2024-09-10T08:44:48.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8699 fp=#0x10000003c9e' 2024-09-10T08:44:48.568 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8731 fp=#0x100000040be' 2024-09-10T08:44:48.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8763 fp=#0x10000004ed0' 2024-09-10T08:44:48.569 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8795 fp=#0x10000003d9d' 2024-09-10T08:44:48.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8827 fp=#0x10000003fa2' 2024-09-10T08:44:48.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8859 fp=#0x10000001afb' 2024-09-10T08:44:48.570 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8891 fp=#0x100000043e2' 2024-09-10T08:44:48.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8923 fp=#0x10000001bfc' 2024-09-10T08:44:48.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8955 fp=#0x100000022d7' 2024-09-10T08:44:48.571 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8987 fp=#0x100000043fa' 2024-09-10T08:44:48.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9019 fp=#0x10000000cde' 2024-09-10T08:44:48.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9051 fp=#0x10000000cce' 2024-09-10T08:44:48.572 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9083 fp=#0x100000040a5' 2024-09-10T08:44:48.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9115 fp=#0x1000000349e' 2024-09-10T08:44:48.573 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9147 fp=#0x100000040eb' 2024-09-10T08:44:48.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9179 fp=#0x10000004151' 2024-09-10T08:44:48.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9211 fp=#0x10000004178' 2024-09-10T08:44:48.574 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9243 fp=#0x10000000adf' 2024-09-10T08:44:48.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9275 fp=#0x100000010d4' 2024-09-10T08:44:48.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9307 fp=#0x10000000bb5' 2024-09-10T08:44:48.575 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9339 fp=#0x10000001760' 2024-09-10T08:44:48.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9371 fp=#0x10000000b2e' 2024-09-10T08:44:48.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9403 fp=#0x10000000b41' 2024-09-10T08:44:48.576 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9435 fp=#0x10000000b4c' 2024-09-10T08:44:48.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9467 fp=#0x10000001c19' 2024-09-10T08:44:48.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9499 fp=#0x10000001424' 2024-09-10T08:44:48.577 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9531 fp=#0x10000001a80' 2024-09-10T08:44:48.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9563 fp=#0x10000004999' 2024-09-10T08:44:48.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9595 fp=#0x10000001a67' 2024-09-10T08:44:48.578 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9627 fp=#0x10000001cb1' 2024-09-10T08:44:48.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9659 fp=#0x10000004906' 2024-09-10T08:44:48.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9691 fp=#0x1000000488c' 2024-09-10T08:44:48.579 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9723 fp=#0x10000004329' 2024-09-10T08:44:48.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9755 fp=#0x100000017af' 2024-09-10T08:44:48.580 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9787 fp=#0x100000044b3' 2024-09-10T08:44:48.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9819 fp=#0x10000001749' 2024-09-10T08:44:48.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9851 fp=#0x1000000442c' 2024-09-10T08:44:48.581 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9883 fp=#0x100000023e4' 2024-09-10T08:44:48.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9915 fp=#0x10000001652' 2024-09-10T08:44:48.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9947 fp=#0x1000000195b' 2024-09-10T08:44:48.582 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9979 fp=#0x10000001638' 2024-09-10T08:44:48.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10011 fp=#0x100000015f7' 2024-09-10T08:44:48.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10043 fp=#0x1000000162f' 2024-09-10T08:44:48.583 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10075 fp=#0x100000016e4' 2024-09-10T08:44:48.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10107 fp=#0x10000003af2' 2024-09-10T08:44:48.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10139 fp=#0x10000001dd1' 2024-09-10T08:44:48.584 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10171 fp=#0x10000003b5e' 2024-09-10T08:44:48.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10203 fp=#0x10000001df4' 2024-09-10T08:44:48.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10235 fp=#0x10000003a73' 2024-09-10T08:44:48.585 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10267 fp=#0x10000002541' 2024-09-10T08:44:48.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10299 fp=#0x1000000235c' 2024-09-10T08:44:48.586 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10363 fp=#0x10000001509' 2024-09-10T08:44:48.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10395 fp=#0x100000013e5' 2024-09-10T08:44:48.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10427 fp=#0x100000013ea' 2024-09-10T08:44:48.587 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10459 fp=#0x10000002796' 2024-09-10T08:44:48.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10491 fp=#0x10000002288' 2024-09-10T08:44:48.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10523 fp=#0x10000001407' 2024-09-10T08:44:48.588 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10555 fp=#0x10000003b21' 2024-09-10T08:44:48.589 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10587 fp=#0x10000001f36' 2024-09-10T08:44:48.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10715 fp=#0x10000004891' 2024-09-10T08:44:48.590 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10779 fp=#0x10000004a21' 2024-09-10T08:44:48.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10811 fp=#0x100000006ed' 2024-09-10T08:44:48.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10843 fp=#0x10000000539' 2024-09-10T08:44:48.591 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10875 fp=#0x10000001cf3' 2024-09-10T08:44:48.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10907 fp=#0x1000000061f' 2024-09-10T08:44:48.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10939 fp=#0x10000000937' 2024-09-10T08:44:48.592 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10971 fp=#0x10000001d09' 2024-09-10T08:44:48.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11003 fp=#0x1000000072f' 2024-09-10T08:44:48.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11035 fp=#0x10000003ada' 2024-09-10T08:44:48.593 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11067 fp=#0x100000038cc' 2024-09-10T08:44:48.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11099 fp=#0x1000000390b' 2024-09-10T08:44:48.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11131 fp=#0x10000004a0d' 2024-09-10T08:44:48.594 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11163 fp=#0x1000000225f' 2024-09-10T08:44:48.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11195 fp=#0x10000002384' 2024-09-10T08:44:48.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11227 fp=#0x100000048fc' 2024-09-10T08:44:48.595 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11259 fp=#0x1000000217d' 2024-09-10T08:44:48.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11291 fp=#0x10000003e2b' 2024-09-10T08:44:48.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11323 fp=#0x10000003f23' 2024-09-10T08:44:48.596 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11355 fp=#0x10000001ceb' 2024-09-10T08:44:48.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11387 fp=#0x10000003f17' 2024-09-10T08:44:48.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11419 fp=#0x10000001d68' 2024-09-10T08:44:48.597 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11451 fp=#0x10000005288' 2024-09-10T08:44:48.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11483 fp=#0x10000001abc' 2024-09-10T08:44:48.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11515 fp=#0x10000003f31' 2024-09-10T08:44:48.598 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11547 fp=#0x100000051a2' 2024-09-10T08:44:48.599 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11643 fp=#0x1000000512e' 2024-09-10T08:44:48.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11707 fp=#0x100000009bb' 2024-09-10T08:44:48.600 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11739 fp=#0x10000004283' 2024-09-10T08:44:48.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11771 fp=#0x10000000a0b' 2024-09-10T08:44:48.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11803 fp=#0x10000000a37' 2024-09-10T08:44:48.601 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11835 fp=#0x10000000924' 2024-09-10T08:44:48.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11867 fp=#0x100000015bf' 2024-09-10T08:44:48.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11899 fp=#0x10000000f0a' 2024-09-10T08:44:48.602 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11931 fp=#0x10000000f1b' 2024-09-10T08:44:48.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11995 fp=#0x10000000eff' 2024-09-10T08:44:48.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12027 fp=#0x10000000f3e' 2024-09-10T08:44:48.603 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12059 fp=#0x10000000f05' 2024-09-10T08:44:48.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12091 fp=#0x10000003367' 2024-09-10T08:44:48.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12123 fp=#0x1000000454f' 2024-09-10T08:44:48.604 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12155 fp=#0x100000044bf' 2024-09-10T08:44:48.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12187 fp=#0x100000044dc' 2024-09-10T08:44:48.605 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12219 fp=#0x10000003beb' 2024-09-10T08:44:48.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12251 fp=#0x10000003b39' 2024-09-10T08:44:48.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12283 fp=#0x10000000d17' 2024-09-10T08:44:48.606 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12315 fp=#0x10000000c66' 2024-09-10T08:44:48.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12347 fp=#0x10000001434' 2024-09-10T08:44:48.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12379 fp=#0x10000000bef' 2024-09-10T08:44:48.607 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12411 fp=#0x10000003ba2' 2024-09-10T08:44:48.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12443 fp=#0x100000039f8' 2024-09-10T08:44:48.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12475 fp=#0x100000012c4' 2024-09-10T08:44:48.608 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12507 fp=#0x10000004c68' 2024-09-10T08:44:48.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12539 fp=#0x10000004c58' 2024-09-10T08:44:48.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12571 fp=#0x10000004bde' 2024-09-10T08:44:48.609 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12603 fp=#0x10000004b45' 2024-09-10T08:44:48.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12635 fp=#0x10000004b9c' 2024-09-10T08:44:48.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12667 fp=#0x10000005404' 2024-09-10T08:44:48.610 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12699 fp=#0x1000000492f' 2024-09-10T08:44:48.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12731 fp=#0x10000005384' 2024-09-10T08:44:48.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12763 fp=#0x100000053d3' 2024-09-10T08:44:48.611 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12795 fp=#0x100000053af' 2024-09-10T08:44:48.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12827 fp=#0x1000000221e' 2024-09-10T08:44:48.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12859 fp=#0x100000020d0' 2024-09-10T08:44:48.612 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12891 fp=#0x10000002118' 2024-09-10T08:44:48.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12923 fp=#0x10000005310' 2024-09-10T08:44:48.613 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12987 fp=#0x100000027aa' 2024-09-10T08:44:48.614 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13043 fp=#0x10000002352' 2024-09-10T08:44:48.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13107 fp=#0x10000003a2f' 2024-09-10T08:44:48.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13139 fp=#0x10000003a5a' 2024-09-10T08:44:48.615 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13171 fp=#0x100000022b7' 2024-09-10T08:44:48.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:292 fp=#0x10000001daf' 2024-09-10T08:44:48.617 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:324 fp=#0x10000001991' 2024-09-10T08:44:48.618 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:380 fp=#0x10000004743' 2024-09-10T08:44:48.619 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:476 fp=#0x10000004aeb' 2024-09-10T08:44:48.621 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:796 fp=#0x10000000e53' 2024-09-10T08:44:48.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:828 fp=#0x10000000ea9' 2024-09-10T08:44:48.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:860 fp=#0x10000000dde' 2024-09-10T08:44:48.622 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:892 fp=#0x100000045b4' 2024-09-10T08:44:48.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:956 fp=#0x10000000dd5' 2024-09-10T08:44:48.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:988 fp=#0x10000004094' 2024-09-10T08:44:48.623 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1020 fp=#0x100000045c5' 2024-09-10T08:44:48.625 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1148 fp=#0x100000040e0' 2024-09-10T08:44:48.625 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1244 fp=#0x10000003687' 2024-09-10T08:44:48.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1308 fp=#0x100000035de' 2024-09-10T08:44:48.626 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1340 fp=#0x10000002047' 2024-09-10T08:44:48.628 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1564 fp=#0x10000003cc8' 2024-09-10T08:44:48.629 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1660 fp=#0x1000000462c' 2024-09-10T08:44:48.631 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1916 fp=#0x100000027a5' 2024-09-10T08:44:48.632 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2044 fp=#0x100000021fc' 2024-09-10T08:44:48.633 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2076 fp=#0x10000002037' 2024-09-10T08:44:48.634 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2268 fp=#0x100000027b0' 2024-09-10T08:44:48.635 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2300 fp=#0x100000025d2' 2024-09-10T08:44:48.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2588 fp=#0x10000001225' 2024-09-10T08:44:48.637 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2620 fp=#0x10000001090' 2024-09-10T08:44:48.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2684 fp=#0x1000000109f' 2024-09-10T08:44:48.638 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2716 fp=#0x10000001043' 2024-09-10T08:44:48.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2748 fp=#0x10000004904' 2024-09-10T08:44:48.639 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2812 fp=#0x10000003463' 2024-09-10T08:44:48.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2844 fp=#0x10000004689' 2024-09-10T08:44:48.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2876 fp=#0x10000004820' 2024-09-10T08:44:48.640 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2908 fp=#0x100000033a9' 2024-09-10T08:44:48.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2940 fp=#0x10000004615' 2024-09-10T08:44:48.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2972 fp=#0x10000004866' 2024-09-10T08:44:48.641 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3004 fp=#0x10000000dc6' 2024-09-10T08:44:48.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3036 fp=#0x10000000e54' 2024-09-10T08:44:48.642 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3100 fp=#0x100000018fa' 2024-09-10T08:44:48.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3132 fp=#0x10000000e71' 2024-09-10T08:44:48.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3164 fp=#0x10000000e06' 2024-09-10T08:44:48.643 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3196 fp=#0x10000004a31' 2024-09-10T08:44:48.644 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3260 fp=#0x100000007a1' 2024-09-10T08:44:48.644 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3292 fp=#0x10000003d2e' 2024-09-10T08:44:48.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3324 fp=#0x10000003c5f' 2024-09-10T08:44:48.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3356 fp=#0x10000000789' 2024-09-10T08:44:48.645 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3388 fp=#0x10000000732' 2024-09-10T08:44:48.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3420 fp=#0x1000000075d' 2024-09-10T08:44:48.646 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3516 fp=#0x1000000403b' 2024-09-10T08:44:48.647 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3644 fp=#0x10000004c1a' 2024-09-10T08:44:48.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3676 fp=#0x10000004c8f' 2024-09-10T08:44:48.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3708 fp=#0x10000004c51' 2024-09-10T08:44:48.648 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3740 fp=#0x10000005312' 2024-09-10T08:44:48.649 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3772 fp=#0x10000003f01' 2024-09-10T08:44:48.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3836 fp=#0x10000005322' 2024-09-10T08:44:48.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3868 fp=#0x100000025eb' 2024-09-10T08:44:48.650 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3900 fp=#0x10000003fdf' 2024-09-10T08:44:48.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3932 fp=#0x10000005294' 2024-09-10T08:44:48.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3964 fp=#0x10000000b0d' 2024-09-10T08:44:48.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3996 fp=#0x1000000054f' 2024-09-10T08:44:48.651 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4028 fp=#0x10000003e57' 2024-09-10T08:44:48.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4060 fp=#0x10000004693' 2024-09-10T08:44:48.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4092 fp=#0x1000000462d' 2024-09-10T08:44:48.652 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4124 fp=#0x10000000f70' 2024-09-10T08:44:48.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4156 fp=#0x10000004e71' 2024-09-10T08:44:48.653 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4188 fp=#0x10000001194' 2024-09-10T08:44:48.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4252 fp=#0x10000003aca' 2024-09-10T08:44:48.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4284 fp=#0x10000003a83' 2024-09-10T08:44:48.654 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4316 fp=#0x10000004a46' 2024-09-10T08:44:48.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4348 fp=#0x10000003f94' 2024-09-10T08:44:48.655 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4412 fp=#0x10000004d32' 2024-09-10T08:44:48.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4444 fp=#0x10000001ed8' 2024-09-10T08:44:48.656 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4508 fp=#0x10000004057' 2024-09-10T08:44:48.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4540 fp=#0x100000050af' 2024-09-10T08:44:48.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4572 fp=#0x10000005125' 2024-09-10T08:44:48.657 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4604 fp=#0x100000015b0' 2024-09-10T08:44:48.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4636 fp=#0x10000001348' 2024-09-10T08:44:48.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4668 fp=#0x10000003931' 2024-09-10T08:44:48.658 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4700 fp=#0x1000000141e' 2024-09-10T08:44:48.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4732 fp=#0x1000000133c' 2024-09-10T08:44:48.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4764 fp=#0x100000014b8' 2024-09-10T08:44:48.659 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4796 fp=#0x100000013f5' 2024-09-10T08:44:48.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4828 fp=#0x10000004a54' 2024-09-10T08:44:48.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4860 fp=#0x10000004a80' 2024-09-10T08:44:48.660 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4892 fp=#0x10000003719' 2024-09-10T08:44:48.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4924 fp=#0x10000003808' 2024-09-10T08:44:48.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4956 fp=#0x1000000512f' 2024-09-10T08:44:48.661 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4988 fp=#0x10000005026' 2024-09-10T08:44:48.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5020 fp=#0x10000005132' 2024-09-10T08:44:48.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5052 fp=#0x1000000513a' 2024-09-10T08:44:48.662 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5084 fp=#0x10000005148' 2024-09-10T08:44:48.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5116 fp=#0x10000005144' 2024-09-10T08:44:48.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5148 fp=#0x10000001ca4' 2024-09-10T08:44:48.663 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5180 fp=#0x10000001d38' 2024-09-10T08:44:48.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5212 fp=#0x10000001d51' 2024-09-10T08:44:48.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5244 fp=#0x10000001cdd' 2024-09-10T08:44:48.664 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5276 fp=#0x100000035aa' 2024-09-10T08:44:48.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5308 fp=#0x10000003d41' 2024-09-10T08:44:48.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5340 fp=#0x10000003619' 2024-09-10T08:44:48.665 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5372 fp=#0x10000004921' 2024-09-10T08:44:48.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5404 fp=#0x100000037d2' 2024-09-10T08:44:48.666 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5436 fp=#0x10000003806' 2024-09-10T08:44:48.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5532 fp=#0x100000033d5' 2024-09-10T08:44:48.667 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5564 fp=#0x1000000128c' 2024-09-10T08:44:48.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5596 fp=#0x100000050a2' 2024-09-10T08:44:48.668 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5628 fp=#0x10000000fe6' 2024-09-10T08:44:48.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5692 fp=#0x10000000fb7' 2024-09-10T08:44:48.669 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5724 fp=#0x100000050bb' 2024-09-10T08:44:48.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5756 fp=#0x10000004d7c' 2024-09-10T08:44:48.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5788 fp=#0x10000004344' 2024-09-10T08:44:48.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5820 fp=#0x1000000438e' 2024-09-10T08:44:48.670 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5852 fp=#0x1000000447e' 2024-09-10T08:44:48.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5884 fp=#0x100000043c7' 2024-09-10T08:44:48.671 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5916 fp=#0x10000004355' 2024-09-10T08:44:48.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5948 fp=#0x10000004cb5' 2024-09-10T08:44:48.672 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5980 fp=#0x10000004b56' 2024-09-10T08:44:48.673 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6108 fp=#0x100000035ca' 2024-09-10T08:44:48.674 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6236 fp=#0x100000034d2' 2024-09-10T08:44:48.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6268 fp=#0x10000003506' 2024-09-10T08:44:48.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6300 fp=#0x10000001de9' 2024-09-10T08:44:48.675 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6332 fp=#0x10000001e92' 2024-09-10T08:44:48.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6364 fp=#0x10000002568' 2024-09-10T08:44:48.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6396 fp=#0x10000001db3' 2024-09-10T08:44:48.676 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6428 fp=#0x10000004e1a' 2024-09-10T08:44:48.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6460 fp=#0x10000004d4e' 2024-09-10T08:44:48.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6492 fp=#0x10000004e38' 2024-09-10T08:44:48.677 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6524 fp=#0x10000005336' 2024-09-10T08:44:48.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6556 fp=#0x10000004b35' 2024-09-10T08:44:48.678 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6620 fp=#0x10000003d2f' 2024-09-10T08:44:48.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6652 fp=#0x1000000396f' 2024-09-10T08:44:48.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6684 fp=#0x100000039d7' 2024-09-10T08:44:48.679 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6716 fp=#0x10000004ffa' 2024-09-10T08:44:48.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6748 fp=#0x10000003a19' 2024-09-10T08:44:48.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6780 fp=#0x10000004f79' 2024-09-10T08:44:48.680 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6812 fp=#0x100000052cb' 2024-09-10T08:44:48.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6844 fp=#0x100000046ee' 2024-09-10T08:44:48.681 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6876 fp=#0x10000004997' 2024-09-10T08:44:48.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6908 fp=#0x10000005292' 2024-09-10T08:44:48.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6940 fp=#0x10000004785' 2024-09-10T08:44:48.682 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6972 fp=#0x10000000588' 2024-09-10T08:44:48.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7036 fp=#0x1000000508b' 2024-09-10T08:44:48.683 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7068 fp=#0x10000005072' 2024-09-10T08:44:48.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7100 fp=#0x1000000067a' 2024-09-10T08:44:48.684 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7132 fp=#0x100000005b1' 2024-09-10T08:44:48.685 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7228 fp=#0x1000000508e' 2024-09-10T08:44:48.686 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7420 fp=#0x10000004831' 2024-09-10T08:44:48.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7452 fp=#0x1000000493d' 2024-09-10T08:44:48.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7484 fp=#0x10000002373' 2024-09-10T08:44:48.687 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7516 fp=#0x10000004883' 2024-09-10T08:44:48.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7548 fp=#0x10000001388' 2024-09-10T08:44:48.688 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7580 fp=#0x10000002272' 2024-09-10T08:44:48.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7676 fp=#0x1000000518c' 2024-09-10T08:44:48.689 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7708 fp=#0x1000000515d' 2024-09-10T08:44:48.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7740 fp=#0x10000004e87' 2024-09-10T08:44:48.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7772 fp=#0x10000004e04' 2024-09-10T08:44:48.690 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7804 fp=#0x100000014c9' 2024-09-10T08:44:48.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7836 fp=#0x100000016a2' 2024-09-10T08:44:48.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7868 fp=#0x100000014c4' 2024-09-10T08:44:48.691 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7900 fp=#0x10000004d78' 2024-09-10T08:44:48.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7932 fp=#0x100000015e6' 2024-09-10T08:44:48.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7964 fp=#0x1000000152b' 2024-09-10T08:44:48.692 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7996 fp=#0x100000013e4' 2024-09-10T08:44:48.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8028 fp=#0x100000015e7' 2024-09-10T08:44:48.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8060 fp=#0x10000001654' 2024-09-10T08:44:48.693 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8092 fp=#0x100000015e2' 2024-09-10T08:44:48.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8124 fp=#0x10000001642' 2024-09-10T08:44:48.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8156 fp=#0x1000000264a' 2024-09-10T08:44:48.694 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8188 fp=#0x100000025f3' 2024-09-10T08:44:48.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8220 fp=#0x100000026af' 2024-09-10T08:44:48.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8252 fp=#0x10000005263' 2024-09-10T08:44:48.695 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8284 fp=#0x10000005267' 2024-09-10T08:44:48.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8316 fp=#0x1000000421e' 2024-09-10T08:44:48.696 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8348 fp=#0x100000044be' 2024-09-10T08:44:48.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8380 fp=#0x10000005245' 2024-09-10T08:44:48.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8412 fp=#0x10000005260' 2024-09-10T08:44:48.697 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8444 fp=#0x100000045f0' 2024-09-10T08:44:48.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8476 fp=#0x10000004543' 2024-09-10T08:44:48.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8508 fp=#0x10000003ec7' 2024-09-10T08:44:48.698 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8540 fp=#0x10000004154' 2024-09-10T08:44:48.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8572 fp=#0x100000041fa' 2024-09-10T08:44:48.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8604 fp=#0x10000003f3d' 2024-09-10T08:44:48.699 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8636 fp=#0x10000003d95' 2024-09-10T08:44:48.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8668 fp=#0x1000000404a' 2024-09-10T08:44:48.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8700 fp=#0x10000003de9' 2024-09-10T08:44:48.700 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8732 fp=#0x10000004f68' 2024-09-10T08:44:48.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8764 fp=#0x10000003c85' 2024-09-10T08:44:48.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8796 fp=#0x10000004036' 2024-09-10T08:44:48.701 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8828 fp=#0x10000003c4c' 2024-09-10T08:44:48.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8860 fp=#0x10000001d8d' 2024-09-10T08:44:48.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8892 fp=#0x10000001bad' 2024-09-10T08:44:48.702 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8924 fp=#0x100000043ab' 2024-09-10T08:44:48.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8956 fp=#0x10000001d1b' 2024-09-10T08:44:48.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8988 fp=#0x1000000433f' 2024-09-10T08:44:48.703 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9020 fp=#0x10000004287' 2024-09-10T08:44:48.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9052 fp=#0x100000041cd' 2024-09-10T08:44:48.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9084 fp=#0x10000001174' 2024-09-10T08:44:48.704 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9116 fp=#0x1000000418b' 2024-09-10T08:44:48.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9148 fp=#0x10000000cd5' 2024-09-10T08:44:48.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9180 fp=#0x10000002068' 2024-09-10T08:44:48.705 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9212 fp=#0x1000000424d' 2024-09-10T08:44:48.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9244 fp=#0x10000000c0a' 2024-09-10T08:44:48.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9276 fp=#0x10000004dcd' 2024-09-10T08:44:48.706 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9308 fp=#0x10000000ab8' 2024-09-10T08:44:48.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9340 fp=#0x10000000b30' 2024-09-10T08:44:48.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9372 fp=#0x10000004d0f' 2024-09-10T08:44:48.707 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9404 fp=#0x10000000bfd' 2024-09-10T08:44:48.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9436 fp=#0x10000000bdc' 2024-09-10T08:44:48.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9468 fp=#0x10000004cce' 2024-09-10T08:44:48.708 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9500 fp=#0x10000000ee6' 2024-09-10T08:44:48.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9532 fp=#0x100000019f6' 2024-09-10T08:44:48.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9564 fp=#0x10000000e9f' 2024-09-10T08:44:48.709 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9596 fp=#0x10000000ecc' 2024-09-10T08:44:48.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9628 fp=#0x1000000497d' 2024-09-10T08:44:48.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9660 fp=#0x100000048fd' 2024-09-10T08:44:48.710 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9692 fp=#0x1000000485d' 2024-09-10T08:44:48.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9724 fp=#0x10000004320' 2024-09-10T08:44:48.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9756 fp=#0x10000001903' 2024-09-10T08:44:48.711 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9788 fp=#0x10000004cde' 2024-09-10T08:44:48.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9820 fp=#0x10000001ef7' 2024-09-10T08:44:48.712 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9852 fp=#0x10000001754' 2024-09-10T08:44:48.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9884 fp=#0x10000004c83' 2024-09-10T08:44:48.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9916 fp=#0x10000001a58' 2024-09-10T08:44:48.713 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9948 fp=#0x100000017eb' 2024-09-10T08:44:48.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9980 fp=#0x10000001a52' 2024-09-10T08:44:48.714 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10012 fp=#0x100000026b8' 2024-09-10T08:44:48.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10044 fp=#0x100000016d6' 2024-09-10T08:44:48.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10076 fp=#0x10000001664' 2024-09-10T08:44:48.715 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10108 fp=#0x10000003488' 2024-09-10T08:44:48.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10140 fp=#0x10000001d94' 2024-09-10T08:44:48.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10172 fp=#0x100000039be' 2024-09-10T08:44:48.716 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10204 fp=#0x10000003920' 2024-09-10T08:44:48.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10236 fp=#0x10000003980' 2024-09-10T08:44:48.717 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10300 fp=#0x10000002558' 2024-09-10T08:44:48.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10332 fp=#0x10000002471' 2024-09-10T08:44:48.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10364 fp=#0x1000000148d' 2024-09-10T08:44:48.718 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10396 fp=#0x10000003a7c' 2024-09-10T08:44:48.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10428 fp=#0x100000013fa' 2024-09-10T08:44:48.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10460 fp=#0x100000038db' 2024-09-10T08:44:48.719 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10492 fp=#0x10000001360' 2024-09-10T08:44:48.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10524 fp=#0x100000012ff' 2024-09-10T08:44:48.720 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10556 fp=#0x10000003aa1' 2024-09-10T08:44:48.721 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10652 fp=#0x100000049e9' 2024-09-10T08:44:48.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10748 fp=#0x10000004a2b' 2024-09-10T08:44:48.722 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10780 fp=#0x100000049d2' 2024-09-10T08:44:48.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10812 fp=#0x10000000a53' 2024-09-10T08:44:48.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10844 fp=#0x10000001cac' 2024-09-10T08:44:48.723 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10876 fp=#0x10000001cc3' 2024-09-10T08:44:48.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10940 fp=#0x10000001047' 2024-09-10T08:44:48.724 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10972 fp=#0x10000001c9c' 2024-09-10T08:44:48.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11004 fp=#0x10000000875' 2024-09-10T08:44:48.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11036 fp=#0x10000003b0f' 2024-09-10T08:44:48.725 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11068 fp=#0x10000003b26' 2024-09-10T08:44:48.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11100 fp=#0x10000003a67' 2024-09-10T08:44:48.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11132 fp=#0x10000002377' 2024-09-10T08:44:48.726 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11164 fp=#0x100000021a2' 2024-09-10T08:44:48.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11196 fp=#0x10000003f5e' 2024-09-10T08:44:48.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11228 fp=#0x10000002297' 2024-09-10T08:44:48.727 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11260 fp=#0x10000003ef8' 2024-09-10T08:44:48.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11292 fp=#0x1000000214c' 2024-09-10T08:44:48.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11324 fp=#0x10000001b74' 2024-09-10T08:44:48.728 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11356 fp=#0x10000001ae5' 2024-09-10T08:44:48.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11388 fp=#0x10000003fa4' 2024-09-10T08:44:48.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11420 fp=#0x10000003e5c' 2024-09-10T08:44:48.729 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11452 fp=#0x10000001cfc' 2024-09-10T08:44:48.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11484 fp=#0x10000003dae' 2024-09-10T08:44:48.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11516 fp=#0x10000004564' 2024-09-10T08:44:48.730 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11548 fp=#0x10000005184' 2024-09-10T08:44:48.731 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11612 fp=#0x1000000513f' 2024-09-10T08:44:48.732 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11708 fp=#0x10000000a11' 2024-09-10T08:44:48.732 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11740 fp=#0x100000010b5' 2024-09-10T08:44:48.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11772 fp=#0x10000004e17' 2024-09-10T08:44:48.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11804 fp=#0x10000004241' 2024-09-10T08:44:48.733 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11836 fp=#0x100000009a1' 2024-09-10T08:44:48.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11868 fp=#0x10000004d8d' 2024-09-10T08:44:48.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11900 fp=#0x100000052e2' 2024-09-10T08:44:48.734 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11932 fp=#0x10000000f15' 2024-09-10T08:44:48.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11964 fp=#0x10000001849' 2024-09-10T08:44:48.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11996 fp=#0x10000002236' 2024-09-10T08:44:48.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12028 fp=#0x10000000ef4' 2024-09-10T08:44:48.735 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12060 fp=#0x10000000f38' 2024-09-10T08:44:48.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12092 fp=#0x100000044d4' 2024-09-10T08:44:48.736 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12156 fp=#0x100000043fe' 2024-09-10T08:44:48.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12188 fp=#0x100000044b8' 2024-09-10T08:44:48.737 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12220 fp=#0x10000000c61' 2024-09-10T08:44:48.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12252 fp=#0x10000003b01' 2024-09-10T08:44:48.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12284 fp=#0x10000000f85' 2024-09-10T08:44:48.738 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12316 fp=#0x1000000257b' 2024-09-10T08:44:48.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12348 fp=#0x10000003bc0' 2024-09-10T08:44:48.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12380 fp=#0x10000003a46' 2024-09-10T08:44:48.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12412 fp=#0x10000003a01' 2024-09-10T08:44:48.739 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12444 fp=#0x10000003b8c' 2024-09-10T08:44:48.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12476 fp=#0x10000002505' 2024-09-10T08:44:48.740 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12508 fp=#0x10000001023' 2024-09-10T08:44:48.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12540 fp=#0x10000004c26' 2024-09-10T08:44:48.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12572 fp=#0x10000004d1d' 2024-09-10T08:44:48.741 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12604 fp=#0x10000004bf3' 2024-09-10T08:44:48.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12636 fp=#0x10000004b02' 2024-09-10T08:44:48.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12668 fp=#0x10000005418' 2024-09-10T08:44:48.742 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12700 fp=#0x100000049e4' 2024-09-10T08:44:48.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12732 fp=#0x10000004f65' 2024-09-10T08:44:48.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12764 fp=#0x10000004912' 2024-09-10T08:44:48.743 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12796 fp=#0x100000052ed' 2024-09-10T08:44:48.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12828 fp=#0x100000052f6' 2024-09-10T08:44:48.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12860 fp=#0x100000021a9' 2024-09-10T08:44:48.744 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12892 fp=#0x100000052cf' 2024-09-10T08:44:48.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12924 fp=#0x1000000533f' 2024-09-10T08:44:48.745 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12956 fp=#0x1000000540c' 2024-09-10T08:44:48.746 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13020 fp=#0x100000052e8' 2024-09-10T08:44:48.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13044 fp=#0x100000039ea' 2024-09-10T08:44:48.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13076 fp=#0x10000003927' 2024-09-10T08:44:48.749 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13108 fp=#0x10000002232' 2024-09-10T08:44:48.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13140 fp=#0x1000000343c' 2024-09-10T08:44:48.750 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13172 fp=#0x1000000225a' 2024-09-10T08:44:48.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:261 fp=#0x10000002215' 2024-09-10T08:44:48.751 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:293 fp=#0x10000004ab5' 2024-09-10T08:44:48.752 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:325 fp=#0x10000002c9b' 2024-09-10T08:44:48.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:445 fp=#0x10000004b0a' 2024-09-10T08:44:48.753 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:477 fp=#0x10000004b37' 2024-09-10T08:44:48.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:509 fp=#0x100000034a7' 2024-09-10T08:44:48.754 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:541 fp=#0x10000003f19' 2024-09-10T08:44:48.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:733 fp=#0x10000004500' 2024-09-10T08:44:48.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:765 fp=#0x10000003e00' 2024-09-10T08:44:48.756 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:797 fp=#0x10000000f1e' 2024-09-10T08:44:48.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:829 fp=#0x10000000e97' 2024-09-10T08:44:48.757 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:861 fp=#0x10000004437' 2024-09-10T08:44:48.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:957 fp=#0x100000040a9' 2024-09-10T08:44:48.758 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:989 fp=#0x100000044d9' 2024-09-10T08:44:48.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1021 fp=#0x1000000457f' 2024-09-10T08:44:48.759 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1085 fp=#0x10000004304' 2024-09-10T08:44:48.760 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1213 fp=#0x1000000413c' 2024-09-10T08:44:48.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1277 fp=#0x10000003653' 2024-09-10T08:44:48.761 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1341 fp=#0x10000003583' 2024-09-10T08:44:48.762 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1469 fp=#0x1000000384a' 2024-09-10T08:44:48.763 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1533 fp=#0x10000003292' 2024-09-10T08:44:48.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1597 fp=#0x10000003cbb' 2024-09-10T08:44:48.764 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1661 fp=#0x1000000463b' 2024-09-10T08:44:48.768 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2045 fp=#0x10000001ffe' 2024-09-10T08:44:48.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2205 fp=#0x10000002178' 2024-09-10T08:44:48.769 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2237 fp=#0x1000000263d' 2024-09-10T08:44:48.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2269 fp=#0x100000025ab' 2024-09-10T08:44:48.770 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2301 fp=#0x10000002696' 2024-09-10T08:44:48.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2621 fp=#0x100000018cb' 2024-09-10T08:44:48.773 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2653 fp=#0x10000001062' 2024-09-10T08:44:48.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2685 fp=#0x100000011a2' 2024-09-10T08:44:48.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2749 fp=#0x10000003b67' 2024-09-10T08:44:48.774 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2781 fp=#0x100000045f8' 2024-09-10T08:44:48.775 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2813 fp=#0x10000003570' 2024-09-10T08:44:48.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2877 fp=#0x10000003698' 2024-09-10T08:44:48.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2909 fp=#0x1000000363e' 2024-09-10T08:44:48.776 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2941 fp=#0x10000003442' 2024-09-10T08:44:48.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2973 fp=#0x100000047e5' 2024-09-10T08:44:48.777 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3005 fp=#0x100000018a9' 2024-09-10T08:44:48.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3069 fp=#0x10000000e67' 2024-09-10T08:44:48.778 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3101 fp=#0x10000004a53' 2024-09-10T08:44:48.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3133 fp=#0x10000000e0e' 2024-09-10T08:44:48.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3165 fp=#0x10000001905' 2024-09-10T08:44:48.779 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3197 fp=#0x10000000e7c' 2024-09-10T08:44:48.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3229 fp=#0x10000000705' 2024-09-10T08:44:48.780 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3261 fp=#0x1000000075b' 2024-09-10T08:44:48.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3293 fp=#0x10000003ba5' 2024-09-10T08:44:48.781 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3325 fp=#0x10000000774' 2024-09-10T08:44:48.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3357 fp=#0x10000003bdf' 2024-09-10T08:44:48.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3389 fp=#0x10000003bee' 2024-09-10T08:44:48.782 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3421 fp=#0x10000003cc0' 2024-09-10T08:44:48.783 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3453 fp=#0x10000004a1a' 2024-09-10T08:44:48.784 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3613 fp=#0x100000032c4' 2024-09-10T08:44:48.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3645 fp=#0x10000004be8' 2024-09-10T08:44:48.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3677 fp=#0x10000004b19' 2024-09-10T08:44:48.785 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3709 fp=#0x10000004be1' 2024-09-10T08:44:48.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3741 fp=#0x10000003fb4' 2024-09-10T08:44:48.786 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3773 fp=#0x10000003f06' 2024-09-10T08:44:48.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3805 fp=#0x10000005327' 2024-09-10T08:44:48.787 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3869 fp=#0x1000000534b' 2024-09-10T08:44:48.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3901 fp=#0x10000002602' 2024-09-10T08:44:48.788 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3933 fp=#0x100000006b7' 2024-09-10T08:44:48.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3997 fp=#0x100000006af' 2024-09-10T08:44:48.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4029 fp=#0x1000000053a' 2024-09-10T08:44:48.789 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4061 fp=#0x10000001025' 2024-09-10T08:44:48.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4093 fp=#0x10000004d60' 2024-09-10T08:44:48.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4125 fp=#0x100000046dd' 2024-09-10T08:44:48.790 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4157 fp=#0x10000000f49' 2024-09-10T08:44:48.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4189 fp=#0x10000004d87' 2024-09-10T08:44:48.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4221 fp=#0x100000039d3' 2024-09-10T08:44:48.791 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4253 fp=#0x100000040b0' 2024-09-10T08:44:48.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4285 fp=#0x10000003c50' 2024-09-10T08:44:48.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4317 fp=#0x10000003a1d' 2024-09-10T08:44:48.792 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4349 fp=#0x100000047f5' 2024-09-10T08:44:48.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4381 fp=#0x10000001edd' 2024-09-10T08:44:48.793 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4445 fp=#0x10000001ff4' 2024-09-10T08:44:48.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4477 fp=#0x10000001f5c' 2024-09-10T08:44:48.794 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4541 fp=#0x10000005134' 2024-09-10T08:44:48.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4573 fp=#0x10000004046' 2024-09-10T08:44:48.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4605 fp=#0x100000012fd' 2024-09-10T08:44:48.795 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4637 fp=#0x100000014be' 2024-09-10T08:44:48.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4669 fp=#0x100000039c1' 2024-09-10T08:44:48.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4701 fp=#0x1000000134e' 2024-09-10T08:44:48.796 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4733 fp=#0x1000000132d' 2024-09-10T08:44:48.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4765 fp=#0x10000001323' 2024-09-10T08:44:48.797 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4797 fp=#0x10000001316' 2024-09-10T08:44:48.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4829 fp=#0x10000004a75' 2024-09-10T08:44:48.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4861 fp=#0x10000004ac3' 2024-09-10T08:44:48.798 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4893 fp=#0x100000036d8' 2024-09-10T08:44:48.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4925 fp=#0x10000004a5c' 2024-09-10T08:44:48.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4957 fp=#0x100000050cb' 2024-09-10T08:44:48.799 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4989 fp=#0x100000051c6' 2024-09-10T08:44:48.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5021 fp=#0x10000005398' 2024-09-10T08:44:48.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5053 fp=#0x100000050f6' 2024-09-10T08:44:48.800 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5085 fp=#0x10000004fd4' 2024-09-10T08:44:48.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5117 fp=#0x100000051cf' 2024-09-10T08:44:48.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5149 fp=#0x10000001e5a' 2024-09-10T08:44:48.801 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5181 fp=#0x10000001cfa' 2024-09-10T08:44:48.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5213 fp=#0x10000002782' 2024-09-10T08:44:48.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5245 fp=#0x10000001e80' 2024-09-10T08:44:48.802 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5277 fp=#0x10000003d8f' 2024-09-10T08:44:48.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5309 fp=#0x10000003811' 2024-09-10T08:44:48.803 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5341 fp=#0x10000003632' 2024-09-10T08:44:48.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5373 fp=#0x10000003651' 2024-09-10T08:44:48.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5405 fp=#0x10000003761' 2024-09-10T08:44:48.804 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5437 fp=#0x100000035fd' 2024-09-10T08:44:48.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5469 fp=#0x10000004e59' 2024-09-10T08:44:48.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5501 fp=#0x100000014d3' 2024-09-10T08:44:48.805 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5533 fp=#0x10000001dea' 2024-09-10T08:44:48.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5597 fp=#0x100000020c0' 2024-09-10T08:44:48.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5629 fp=#0x10000005089' 2024-09-10T08:44:48.806 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5661 fp=#0x10000003c16' 2024-09-10T08:44:48.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5693 fp=#0x100000050a1' 2024-09-10T08:44:48.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5725 fp=#0x10000000f33' 2024-09-10T08:44:48.807 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5757 fp=#0x10000000f87' 2024-09-10T08:44:48.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5789 fp=#0x10000004463' 2024-09-10T08:44:48.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5821 fp=#0x10000004c02' 2024-09-10T08:44:48.808 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5853 fp=#0x10000004b6b' 2024-09-10T08:44:48.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5885 fp=#0x10000004c69' 2024-09-10T08:44:48.809 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5917 fp=#0x10000004484' 2024-09-10T08:44:48.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5949 fp=#0x10000004c8d' 2024-09-10T08:44:48.810 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5981 fp=#0x10000004453' 2024-09-10T08:44:48.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6109 fp=#0x100000035c5' 2024-09-10T08:44:48.811 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6141 fp=#0x100000035cf' 2024-09-10T08:44:48.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6173 fp=#0x10000003455' 2024-09-10T08:44:48.812 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6237 fp=#0x10000003514' 2024-09-10T08:44:48.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6269 fp=#0x10000004009' 2024-09-10T08:44:48.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6301 fp=#0x10000003f3f' 2024-09-10T08:44:48.813 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6333 fp=#0x10000003406' 2024-09-10T08:44:48.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6365 fp=#0x10000003365' 2024-09-10T08:44:48.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6397 fp=#0x100000032d1' 2024-09-10T08:44:48.814 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6429 fp=#0x10000004c92' 2024-09-10T08:44:48.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6461 fp=#0x10000004c46' 2024-09-10T08:44:48.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6493 fp=#0x10000004d33' 2024-09-10T08:44:48.815 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6525 fp=#0x10000004d70' 2024-09-10T08:44:48.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6557 fp=#0x10000004db0' 2024-09-10T08:44:48.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6589 fp=#0x100000039ef' 2024-09-10T08:44:48.816 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6621 fp=#0x10000003d07' 2024-09-10T08:44:48.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6653 fp=#0x10000003a94' 2024-09-10T08:44:48.817 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6685 fp=#0x10000003b25' 2024-09-10T08:44:48.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6717 fp=#0x10000003a72' 2024-09-10T08:44:48.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6749 fp=#0x10000003c10' 2024-09-10T08:44:48.818 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6781 fp=#0x10000003a9c' 2024-09-10T08:44:48.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6813 fp=#0x100000048f8' 2024-09-10T08:44:48.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6845 fp=#0x1000000469d' 2024-09-10T08:44:48.819 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6877 fp=#0x1000000535e' 2024-09-10T08:44:48.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6909 fp=#0x1000000476e' 2024-09-10T08:44:48.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6941 fp=#0x100000052e6' 2024-09-10T08:44:48.820 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6973 fp=#0x100000005d8' 2024-09-10T08:44:48.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7005 fp=#0x100000006ae' 2024-09-10T08:44:48.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7037 fp=#0x100000005f9' 2024-09-10T08:44:48.821 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7069 fp=#0x100000005f3' 2024-09-10T08:44:48.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7101 fp=#0x1000000067f' 2024-09-10T08:44:48.822 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7133 fp=#0x100000006a6' 2024-09-10T08:44:48.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7165 fp=#0x100000027c0' 2024-09-10T08:44:48.823 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7261 fp=#0x1000000511b' 2024-09-10T08:44:48.827 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7293 fp=#0x1000000512d' 2024-09-10T08:44:48.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7325 fp=#0x10000005047' 2024-09-10T08:44:48.828 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7421 fp=#0x1000000488b' 2024-09-10T08:44:48.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7453 fp=#0x100000047e7' 2024-09-10T08:44:48.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7485 fp=#0x1000000475e' 2024-09-10T08:44:48.829 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7517 fp=#0x1000000237f' 2024-09-10T08:44:48.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7549 fp=#0x10000001314' 2024-09-10T08:44:48.830 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7581 fp=#0x10000001386' 2024-09-10T08:44:48.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7645 fp=#0x100000052af' 2024-09-10T08:44:48.831 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7709 fp=#0x10000004f89' 2024-09-10T08:44:48.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7741 fp=#0x10000004f23' 2024-09-10T08:44:48.832 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7773 fp=#0x10000004f59' 2024-09-10T08:44:48.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7805 fp=#0x100000016f3' 2024-09-10T08:44:48.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7837 fp=#0x10000004e45' 2024-09-10T08:44:48.833 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7869 fp=#0x1000000148c' 2024-09-10T08:44:48.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7901 fp=#0x10000004f12' 2024-09-10T08:44:48.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7933 fp=#0x10000004d9c' 2024-09-10T08:44:48.834 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7965 fp=#0x100000014ac' 2024-09-10T08:44:48.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7997 fp=#0x10000004eef' 2024-09-10T08:44:48.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8029 fp=#0x10000004e2b' 2024-09-10T08:44:48.835 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8061 fp=#0x10000001685' 2024-09-10T08:44:48.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8093 fp=#0x10000004e1d' 2024-09-10T08:44:48.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8125 fp=#0x100000016b9' 2024-09-10T08:44:48.836 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8157 fp=#0x100000026c7' 2024-09-10T08:44:48.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8189 fp=#0x10000002599' 2024-09-10T08:44:48.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8221 fp=#0x10000004f56' 2024-09-10T08:44:48.837 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8253 fp=#0x1000000435a' 2024-09-10T08:44:48.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8285 fp=#0x10000004606' 2024-09-10T08:44:48.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8317 fp=#0x10000004417' 2024-09-10T08:44:48.838 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8349 fp=#0x10000005276' 2024-09-10T08:44:48.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8381 fp=#0x100000043da' 2024-09-10T08:44:48.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8413 fp=#0x10000005241' 2024-09-10T08:44:48.839 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8445 fp=#0x10000004174' 2024-09-10T08:44:48.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8477 fp=#0x10000004573' 2024-09-10T08:44:48.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8509 fp=#0x10000003f02' 2024-09-10T08:44:48.840 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8541 fp=#0x10000003f76' 2024-09-10T08:44:48.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8573 fp=#0x10000003f48' 2024-09-10T08:44:48.841 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8605 fp=#0x100000041db' 2024-09-10T08:44:48.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8637 fp=#0x10000004069' 2024-09-10T08:44:48.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8669 fp=#0x10000003dfb' 2024-09-10T08:44:48.842 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8701 fp=#0x10000003e01' 2024-09-10T08:44:48.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8733 fp=#0x10000005316' 2024-09-10T08:44:48.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8765 fp=#0x10000003d62' 2024-09-10T08:44:48.843 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8797 fp=#0x100000052a2' 2024-09-10T08:44:48.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8829 fp=#0x100000047e8' 2024-09-10T08:44:48.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8861 fp=#0x10000004361' 2024-09-10T08:44:48.844 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8893 fp=#0x1000000433d' 2024-09-10T08:44:48.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8925 fp=#0x10000001c36' 2024-09-10T08:44:48.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8957 fp=#0x10000001ac4' 2024-09-10T08:44:48.845 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8989 fp=#0x1000000437b' 2024-09-10T08:44:48.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9021 fp=#0x10000000d9b' 2024-09-10T08:44:48.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9053 fp=#0x10000004206' 2024-09-10T08:44:48.846 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9085 fp=#0x10000000caf' 2024-09-10T08:44:48.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9117 fp=#0x10000004235' 2024-09-10T08:44:48.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9149 fp=#0x100000041b5' 2024-09-10T08:44:48.847 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9181 fp=#0x10000000ccc' 2024-09-10T08:44:48.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9213 fp=#0x1000000421d' 2024-09-10T08:44:48.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9245 fp=#0x10000004c72' 2024-09-10T08:44:48.848 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9277 fp=#0x10000000b56' 2024-09-10T08:44:48.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9309 fp=#0x100000025f2' 2024-09-10T08:44:48.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9341 fp=#0x100000010bb' 2024-09-10T08:44:48.849 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9373 fp=#0x10000004d4f' 2024-09-10T08:44:48.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9405 fp=#0x10000000b79' 2024-09-10T08:44:48.850 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9437 fp=#0x10000001c25' 2024-09-10T08:44:48.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9469 fp=#0x1000000260d' 2024-09-10T08:44:48.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9501 fp=#0x10000001ea1' 2024-09-10T08:44:48.851 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9533 fp=#0x100000048bf' 2024-09-10T08:44:48.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9565 fp=#0x10000001ee0' 2024-09-10T08:44:48.852 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9597 fp=#0x10000000edc' 2024-09-10T08:44:48.853 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9629 fp=#0x10000004947' 2024-09-10T08:44:48.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9661 fp=#0x1000000493c' 2024-09-10T08:44:48.854 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9693 fp=#0x1000000480f' 2024-09-10T08:44:48.855 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9725 fp=#0x100000024c5' 2024-09-10T08:44:48.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9757 fp=#0x100000044b2' 2024-09-10T08:44:48.856 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9789 fp=#0x10000004441' 2024-09-10T08:44:48.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9821 fp=#0x10000004474' 2024-09-10T08:44:48.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9853 fp=#0x10000001770' 2024-09-10T08:44:48.857 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9885 fp=#0x100000044b7' 2024-09-10T08:44:48.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9917 fp=#0x1000000194c' 2024-09-10T08:44:48.858 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9949 fp=#0x10000002282' 2024-09-10T08:44:48.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9981 fp=#0x1000000189d' 2024-09-10T08:44:48.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10013 fp=#0x10000001ff3' 2024-09-10T08:44:48.859 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10045 fp=#0x1000000270d' 2024-09-10T08:44:48.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10077 fp=#0x10000001682' 2024-09-10T08:44:48.860 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10109 fp=#0x10000003f53' 2024-09-10T08:44:48.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10141 fp=#0x10000004a20' 2024-09-10T08:44:48.861 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10173 fp=#0x1000000234e' 2024-09-10T08:44:48.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10205 fp=#0x10000003b03' 2024-09-10T08:44:48.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10237 fp=#0x100000039d4' 2024-09-10T08:44:48.862 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10269 fp=#0x10000002440' 2024-09-10T08:44:48.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10301 fp=#0x100000024b8' 2024-09-10T08:44:48.863 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10333 fp=#0x100000024b0' 2024-09-10T08:44:48.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10365 fp=#0x100000014ed' 2024-09-10T08:44:48.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10397 fp=#0x10000003a76' 2024-09-10T08:44:48.864 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10429 fp=#0x10000003ac6' 2024-09-10T08:44:48.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10461 fp=#0x100000038ee' 2024-09-10T08:44:48.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10493 fp=#0x1000000131b' 2024-09-10T08:44:48.865 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10557 fp=#0x10000001328' 2024-09-10T08:44:48.866 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10653 fp=#0x10000004874' 2024-09-10T08:44:48.867 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10717 fp=#0x1000000531a' 2024-09-10T08:44:48.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10781 fp=#0x10000004ba4' 2024-09-10T08:44:48.868 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10845 fp=#0x10000000976' 2024-09-10T08:44:48.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10909 fp=#0x100000048e9' 2024-09-10T08:44:48.869 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10941 fp=#0x10000001cfe' 2024-09-10T08:44:48.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11005 fp=#0x100000007e9' 2024-09-10T08:44:48.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11037 fp=#0x100000038b9' 2024-09-10T08:44:48.870 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11069 fp=#0x10000005070' 2024-09-10T08:44:48.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11101 fp=#0x10000003895' 2024-09-10T08:44:48.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11133 fp=#0x10000002115' 2024-09-10T08:44:48.871 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11165 fp=#0x10000003ea5' 2024-09-10T08:44:48.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11197 fp=#0x1000000207e' 2024-09-10T08:44:48.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11229 fp=#0x10000002183' 2024-09-10T08:44:48.872 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11261 fp=#0x10000003f30' 2024-09-10T08:44:48.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11293 fp=#0x1000000216f' 2024-09-10T08:44:48.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11325 fp=#0x10000003ebf' 2024-09-10T08:44:48.873 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11357 fp=#0x10000003e7c' 2024-09-10T08:44:48.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11389 fp=#0x10000003e98' 2024-09-10T08:44:48.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11421 fp=#0x10000003f86' 2024-09-10T08:44:48.874 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11453 fp=#0x10000001ad5' 2024-09-10T08:44:48.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11485 fp=#0x10000003eaf' 2024-09-10T08:44:48.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11517 fp=#0x10000001d1e' 2024-09-10T08:44:48.875 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11549 fp=#0x100000035bc' 2024-09-10T08:44:48.876 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11645 fp=#0x10000005135' 2024-09-10T08:44:48.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11709 fp=#0x1000000088a' 2024-09-10T08:44:48.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11741 fp=#0x100000041b8' 2024-09-10T08:44:48.877 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11773 fp=#0x100000041e9' 2024-09-10T08:44:48.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11805 fp=#0x10000004e53' 2024-09-10T08:44:48.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11837 fp=#0x1000000172a' 2024-09-10T08:44:48.878 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11869 fp=#0x10000004205' 2024-09-10T08:44:48.879 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11933 fp=#0x10000005296' 2024-09-10T08:44:48.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11997 fp=#0x1000000112f' 2024-09-10T08:44:48.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12029 fp=#0x100000010d1' 2024-09-10T08:44:48.880 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12093 fp=#0x1000000451f' 2024-09-10T08:44:48.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12125 fp=#0x10000004538' 2024-09-10T08:44:48.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12157 fp=#0x100000045a1' 2024-09-10T08:44:48.881 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12189 fp=#0x10000003382' 2024-09-10T08:44:48.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12221 fp=#0x10000000d75' 2024-09-10T08:44:48.882 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12253 fp=#0x10000003bc7' 2024-09-10T08:44:48.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12285 fp=#0x10000000c44' 2024-09-10T08:44:48.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12317 fp=#0x10000000d73' 2024-09-10T08:44:48.883 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12349 fp=#0x10000003bf2' 2024-09-10T08:44:48.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12381 fp=#0x10000000d5f' 2024-09-10T08:44:48.884 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12413 fp=#0x10000000d34' 2024-09-10T08:44:48.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12445 fp=#0x10000003ab0' 2024-09-10T08:44:48.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12477 fp=#0x10000004c16' 2024-09-10T08:44:48.885 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12541 fp=#0x10000004c05' 2024-09-10T08:44:48.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12573 fp=#0x10000004d04' 2024-09-10T08:44:48.886 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12605 fp=#0x10000004b4b' 2024-09-10T08:44:48.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12637 fp=#0x10000004aba' 2024-09-10T08:44:48.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12669 fp=#0x10000004937' 2024-09-10T08:44:48.887 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12701 fp=#0x10000004873' 2024-09-10T08:44:48.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12733 fp=#0x10000004848' 2024-09-10T08:44:48.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12765 fp=#0x10000004880' 2024-09-10T08:44:48.888 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12797 fp=#0x10000002307' 2024-09-10T08:44:48.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12829 fp=#0x100000021b2' 2024-09-10T08:44:48.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12861 fp=#0x1000000537b' 2024-09-10T08:44:48.889 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12893 fp=#0x100000020dd' 2024-09-10T08:44:48.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12925 fp=#0x100000021f1' 2024-09-10T08:44:48.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12957 fp=#0x100000027ee' 2024-09-10T08:44:48.890 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12989 fp=#0x100000053ae' 2024-09-10T08:44:48.891 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13045 fp=#0x10000004686' 2024-09-10T08:44:48.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13109 fp=#0x1000000231a' 2024-09-10T08:44:48.892 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13141 fp=#0x100000022bf' 2024-09-10T08:44:48.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13173 fp=#0x100000053d4' 2024-09-10T08:44:48.893 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:163 fp=#0x10000000005' 2024-09-10T08:44:48.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:262 fp=#0x1000000102d' 2024-09-10T08:44:48.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:294 fp=#0x10000002bb4' 2024-09-10T08:44:48.894 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:326 fp=#0x1000000085d' 2024-09-10T08:44:48.895 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:382 fp=#0x1000000464a' 2024-09-10T08:44:48.896 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:574 fp=#0x100000034da' 2024-09-10T08:44:48.897 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:606 fp=#0x10000003f1f' 2024-09-10T08:44:48.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:798 fp=#0x10000000db7' 2024-09-10T08:44:48.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:830 fp=#0x100000016de' 2024-09-10T08:44:48.899 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:862 fp=#0x100000040d3' 2024-09-10T08:44:48.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:894 fp=#0x1000000458e' 2024-09-10T08:44:48.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:926 fp=#0x10000000dbb' 2024-09-10T08:44:48.900 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:958 fp=#0x10000000d8f' 2024-09-10T08:44:48.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:990 fp=#0x1000000450f' 2024-09-10T08:44:48.901 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1022 fp=#0x10000000eb9' 2024-09-10T08:44:48.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1214 fp=#0x10000004166' 2024-09-10T08:44:48.903 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1310 fp=#0x10000003598' 2024-09-10T08:44:48.904 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1342 fp=#0x10000003691' 2024-09-10T08:44:48.906 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1630 fp=#0x10000003cd4' 2024-09-10T08:44:48.907 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1662 fp=#0x10000004a28' 2024-09-10T08:44:48.910 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2046 fp=#0x10000002188' 2024-09-10T08:44:48.910 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2142 fp=#0x10000001ff2' 2024-09-10T08:44:48.911 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2174 fp=#0x100000045d6' 2024-09-10T08:44:48.911 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2238 fp=#0x100000026c8' 2024-09-10T08:44:48.912 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2270 fp=#0x100000026ce' 2024-09-10T08:44:48.912 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2302 fp=#0x10000002637' 2024-09-10T08:44:48.913 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2334 fp=#0x10000002691' 2024-09-10T08:44:48.915 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2590 fp=#0x100000010d8' 2024-09-10T08:44:48.915 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2622 fp=#0x10000001095' 2024-09-10T08:44:48.915 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2654 fp=#0x100000018d6' 2024-09-10T08:44:48.916 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2686 fp=#0x100000010d2' 2024-09-10T08:44:48.916 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2750 fp=#0x10000003b3e' 2024-09-10T08:44:48.917 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2846 fp=#0x100000040bb' 2024-09-10T08:44:48.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2878 fp=#0x100000046ed' 2024-09-10T08:44:48.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2910 fp=#0x10000003453' 2024-09-10T08:44:48.918 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2942 fp=#0x10000004643' 2024-09-10T08:44:48.919 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2974 fp=#0x10000003660' 2024-09-10T08:44:48.919 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3038 fp=#0x10000000e58' 2024-09-10T08:44:48.920 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3070 fp=#0x10000004a2c' 2024-09-10T08:44:48.920 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3102 fp=#0x10000000e96' 2024-09-10T08:44:48.920 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3134 fp=#0x1000000196e' 2024-09-10T08:44:48.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3166 fp=#0x100000049bb' 2024-09-10T08:44:48.921 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3230 fp=#0x10000000725' 2024-09-10T08:44:48.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3262 fp=#0x10000001f4e' 2024-09-10T08:44:48.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3294 fp=#0x1000000077c' 2024-09-10T08:44:48.922 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3326 fp=#0x10000000784' 2024-09-10T08:44:48.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3358 fp=#0x10000000776' 2024-09-10T08:44:48.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3390 fp=#0x10000000767' 2024-09-10T08:44:48.923 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3422 fp=#0x10000000790' 2024-09-10T08:44:48.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3582 fp=#0x10000004093' 2024-09-10T08:44:48.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3646 fp=#0x10000004c62' 2024-09-10T08:44:48.925 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3678 fp=#0x10000004ae6' 2024-09-10T08:44:48.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3710 fp=#0x10000004c1e' 2024-09-10T08:44:48.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3742 fp=#0x100000026d1' 2024-09-10T08:44:48.926 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3774 fp=#0x100000027c7' 2024-09-10T08:44:48.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3806 fp=#0x10000003fce' 2024-09-10T08:44:48.927 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3838 fp=#0x1000000399d' 2024-09-10T08:44:48.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3902 fp=#0x10000003ef1' 2024-09-10T08:44:48.928 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3934 fp=#0x10000000525' 2024-09-10T08:44:48.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3966 fp=#0x100000006b8' 2024-09-10T08:44:48.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3998 fp=#0x10000000661' 2024-09-10T08:44:48.929 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4030 fp=#0x10000000687' 2024-09-10T08:44:48.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4062 fp=#0x10000004d72' 2024-09-10T08:44:48.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4094 fp=#0x10000004660' 2024-09-10T08:44:48.930 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4126 fp=#0x10000000f52' 2024-09-10T08:44:48.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4158 fp=#0x10000000fcb' 2024-09-10T08:44:48.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4190 fp=#0x10000004665' 2024-09-10T08:44:48.931 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4222 fp=#0x10000003ba9' 2024-09-10T08:44:48.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4254 fp=#0x100000040fb' 2024-09-10T08:44:48.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4286 fp=#0x100000040e9' 2024-09-10T08:44:48.932 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4318 fp=#0x10000003977' 2024-09-10T08:44:48.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4350 fp=#0x10000003b24' 2024-09-10T08:44:48.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4382 fp=#0x1000000201a' 2024-09-10T08:44:48.933 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4414 fp=#0x10000004026' 2024-09-10T08:44:48.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4446 fp=#0x10000001f13' 2024-09-10T08:44:48.934 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4510 fp=#0x10000001c5c' 2024-09-10T08:44:48.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4542 fp=#0x10000003b36' 2024-09-10T08:44:48.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4574 fp=#0x100000050a0' 2024-09-10T08:44:48.935 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4606 fp=#0x1000000159e' 2024-09-10T08:44:48.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4638 fp=#0x10000001432' 2024-09-10T08:44:48.936 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4670 fp=#0x1000000137f' 2024-09-10T08:44:48.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4702 fp=#0x100000012f4' 2024-09-10T08:44:48.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4734 fp=#0x100000046bb' 2024-09-10T08:44:48.937 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4766 fp=#0x10000001565' 2024-09-10T08:44:48.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4798 fp=#0x1000000132a' 2024-09-10T08:44:48.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4830 fp=#0x100000037ec' 2024-09-10T08:44:48.938 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4862 fp=#0x1000000385a' 2024-09-10T08:44:48.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4894 fp=#0x100000037b6' 2024-09-10T08:44:48.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4926 fp=#0x100000037cb' 2024-09-10T08:44:48.939 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4958 fp=#0x100000053ab' 2024-09-10T08:44:48.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4990 fp=#0x1000000533b' 2024-09-10T08:44:48.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5022 fp=#0x100000051a1' 2024-09-10T08:44:48.940 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5054 fp=#0x100000051b1' 2024-09-10T08:44:48.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5086 fp=#0x1000000509b' 2024-09-10T08:44:48.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5118 fp=#0x1000000503a' 2024-09-10T08:44:48.941 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5150 fp=#0x10000001dfb' 2024-09-10T08:44:48.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5182 fp=#0x10000002778' 2024-09-10T08:44:48.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5214 fp=#0x10000001e99' 2024-09-10T08:44:48.942 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5246 fp=#0x10000001c27' 2024-09-10T08:44:48.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5278 fp=#0x100000038cd' 2024-09-10T08:44:48.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5310 fp=#0x100000035e8' 2024-09-10T08:44:48.943 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5342 fp=#0x10000003d58' 2024-09-10T08:44:48.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5374 fp=#0x100000035db' 2024-09-10T08:44:48.944 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5406 fp=#0x100000036d3' 2024-09-10T08:44:48.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5438 fp=#0x10000003733' 2024-09-10T08:44:48.945 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5470 fp=#0x10000003876' 2024-09-10T08:44:48.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5598 fp=#0x100000010df' 2024-09-10T08:44:48.946 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5630 fp=#0x1000000508c' 2024-09-10T08:44:48.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5662 fp=#0x100000050c7' 2024-09-10T08:44:48.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5694 fp=#0x100000020cf' 2024-09-10T08:44:48.947 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5726 fp=#0x10000001097' 2024-09-10T08:44:48.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5758 fp=#0x1000000106c' 2024-09-10T08:44:48.948 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5790 fp=#0x10000004d7a' 2024-09-10T08:44:48.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5822 fp=#0x1000000444c' 2024-09-10T08:44:48.949 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5854 fp=#0x10000004af7' 2024-09-10T08:44:48.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5886 fp=#0x10000004613' 2024-09-10T08:44:48.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5918 fp=#0x10000004bcb' 2024-09-10T08:44:48.954 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5950 fp=#0x100000045f3' 2024-09-10T08:44:48.955 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5982 fp=#0x10000004468' 2024-09-10T08:44:48.956 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6142 fp=#0x1000000362a' 2024-09-10T08:44:48.957 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6270 fp=#0x10000003422' 2024-09-10T08:44:48.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6302 fp=#0x100000032f1' 2024-09-10T08:44:48.958 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6334 fp=#0x10000001dfe' 2024-09-10T08:44:48.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6430 fp=#0x1000000535b' 2024-09-10T08:44:48.959 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6462 fp=#0x10000004e31' 2024-09-10T08:44:48.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6494 fp=#0x10000004b9e' 2024-09-10T08:44:48.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6526 fp=#0x10000004ce2' 2024-09-10T08:44:48.960 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6558 fp=#0x10000004de0' 2024-09-10T08:44:48.961 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6622 fp=#0x1000000391d' 2024-09-10T08:44:48.961 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6654 fp=#0x10000004fcb' 2024-09-10T08:44:48.961 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6686 fp=#0x10000003b44' 2024-09-10T08:44:48.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6718 fp=#0x10000003994' 2024-09-10T08:44:48.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6750 fp=#0x10000003ae1' 2024-09-10T08:44:48.962 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6782 fp=#0x10000003ad2' 2024-09-10T08:44:48.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6814 fp=#0x100000049b8' 2024-09-10T08:44:48.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6846 fp=#0x100000052d0' 2024-09-10T08:44:48.963 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6878 fp=#0x100000048df' 2024-09-10T08:44:48.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6910 fp=#0x100000046c2' 2024-09-10T08:44:48.964 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6942 fp=#0x10000005371' 2024-09-10T08:44:48.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6974 fp=#0x10000000614' 2024-09-10T08:44:48.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7006 fp=#0x100000017bb' 2024-09-10T08:44:48.965 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7038 fp=#0x1000000064e' 2024-09-10T08:44:48.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7070 fp=#0x10000000657' 2024-09-10T08:44:48.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7102 fp=#0x10000000584' 2024-09-10T08:44:48.966 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7134 fp=#0x10000000624' 2024-09-10T08:44:48.967 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7198 fp=#0x100000050cf' 2024-09-10T08:44:48.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7422 fp=#0x1000000263f' 2024-09-10T08:44:48.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7454 fp=#0x10000004a69' 2024-09-10T08:44:48.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7486 fp=#0x10000002385' 2024-09-10T08:44:48.969 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7518 fp=#0x100000023b7' 2024-09-10T08:44:48.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7550 fp=#0x10000001383' 2024-09-10T08:44:48.970 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7582 fp=#0x10000004a12' 2024-09-10T08:44:48.971 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7646 fp=#0x10000001cd9' 2024-09-10T08:44:48.971 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7678 fp=#0x1000000517a' 2024-09-10T08:44:48.972 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7710 fp=#0x10000004f6e' 2024-09-10T08:44:48.972 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7742 fp=#0x10000004faf' 2024-09-10T08:44:48.972 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7774 fp=#0x10000004f1f' 2024-09-10T08:44:48.972 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7806 fp=#0x10000004ccf' 2024-09-10T08:44:48.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7838 fp=#0x100000013d8' 2024-09-10T08:44:48.973 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7870 fp=#0x10000004ce0' 2024-09-10T08:44:48.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7902 fp=#0x100000013a9' 2024-09-10T08:44:48.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7934 fp=#0x100000014bf' 2024-09-10T08:44:48.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7966 fp=#0x10000001ed5' 2024-09-10T08:44:48.974 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7998 fp=#0x10000004ebf' 2024-09-10T08:44:48.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8030 fp=#0x100000052f9' 2024-09-10T08:44:48.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8062 fp=#0x1000000165d' 2024-09-10T08:44:48.975 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8094 fp=#0x10000003386' 2024-09-10T08:44:48.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8126 fp=#0x100000016f9' 2024-09-10T08:44:48.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8158 fp=#0x10000002517' 2024-09-10T08:44:48.976 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8190 fp=#0x1000000260f' 2024-09-10T08:44:48.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8222 fp=#0x1000000253f' 2024-09-10T08:44:48.977 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8254 fp=#0x10000004719' 2024-09-10T08:44:48.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8286 fp=#0x10000005257' 2024-09-10T08:44:48.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8318 fp=#0x1000000442e' 2024-09-10T08:44:48.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8350 fp=#0x1000000471e' 2024-09-10T08:44:48.978 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8382 fp=#0x100000041e1' 2024-09-10T08:44:48.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8414 fp=#0x1000000524a' 2024-09-10T08:44:48.979 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8446 fp=#0x1000000449b' 2024-09-10T08:44:48.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8478 fp=#0x10000003f4f' 2024-09-10T08:44:48.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8510 fp=#0x10000003ed0' 2024-09-10T08:44:48.980 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8542 fp=#0x10000003ebc' 2024-09-10T08:44:48.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8574 fp=#0x10000004539' 2024-09-10T08:44:48.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8606 fp=#0x10000003f90' 2024-09-10T08:44:48.981 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8638 fp=#0x10000003c5a' 2024-09-10T08:44:48.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8670 fp=#0x10000004a2e' 2024-09-10T08:44:48.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8702 fp=#0x100000052f4' 2024-09-10T08:44:48.982 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8734 fp=#0x1000000407f' 2024-09-10T08:44:48.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8766 fp=#0x10000004061' 2024-09-10T08:44:48.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8798 fp=#0x10000004f26' 2024-09-10T08:44:48.983 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8830 fp=#0x10000004f77' 2024-09-10T08:44:48.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8862 fp=#0x10000004375' 2024-09-10T08:44:48.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8894 fp=#0x10000004495' 2024-09-10T08:44:48.984 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8926 fp=#0x10000004471' 2024-09-10T08:44:48.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8958 fp=#0x10000001b59' 2024-09-10T08:44:48.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8990 fp=#0x10000001b04' 2024-09-10T08:44:48.985 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9022 fp=#0x100000041e7' 2024-09-10T08:44:48.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9054 fp=#0x10000003554' 2024-09-10T08:44:48.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9086 fp=#0x10000004072' 2024-09-10T08:44:48.986 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9118 fp=#0x10000000cbc' 2024-09-10T08:44:48.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9150 fp=#0x100000040ae' 2024-09-10T08:44:48.987 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9182 fp=#0x10000000d2b' 2024-09-10T08:44:48.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9214 fp=#0x100000041ff' 2024-09-10T08:44:48.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9246 fp=#0x10000000bec' 2024-09-10T08:44:48.988 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9278 fp=#0x10000000b15' 2024-09-10T08:44:48.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9310 fp=#0x10000002516' 2024-09-10T08:44:48.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9342 fp=#0x10000004d58' 2024-09-10T08:44:48.989 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9374 fp=#0x10000004c6b' 2024-09-10T08:44:48.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9406 fp=#0x10000000b55' 2024-09-10T08:44:48.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9438 fp=#0x10000004d2c' 2024-09-10T08:44:48.990 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9470 fp=#0x10000000b45' 2024-09-10T08:44:48.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9502 fp=#0x10000001eb0' 2024-09-10T08:44:48.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9534 fp=#0x10000000ed9' 2024-09-10T08:44:48.991 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9566 fp=#0x10000001a22' 2024-09-10T08:44:48.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9598 fp=#0x10000001ec8' 2024-09-10T08:44:48.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9630 fp=#0x10000004920' 2024-09-10T08:44:48.992 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9662 fp=#0x100000048a3' 2024-09-10T08:44:48.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9694 fp=#0x10000004899' 2024-09-10T08:44:48.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9726 fp=#0x10000001af7' 2024-09-10T08:44:48.993 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9758 fp=#0x100000044ac' 2024-09-10T08:44:48.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9790 fp=#0x10000004324' 2024-09-10T08:44:48.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9822 fp=#0x10000004494' 2024-09-10T08:44:48.994 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9854 fp=#0x1000000181b' 2024-09-10T08:44:48.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9886 fp=#0x1000000449d' 2024-09-10T08:44:48.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9918 fp=#0x10000001a93' 2024-09-10T08:44:48.995 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9950 fp=#0x10000001900' 2024-09-10T08:44:48.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9982 fp=#0x100000017d3' 2024-09-10T08:44:48.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10014 fp=#0x1000000168c' 2024-09-10T08:44:48.996 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10046 fp=#0x1000000161b' 2024-09-10T08:44:48.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10078 fp=#0x10000001961' 2024-09-10T08:44:48.997 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10110 fp=#0x1000000201b' 2024-09-10T08:44:48.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10142 fp=#0x100000038d0' 2024-09-10T08:44:48.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10174 fp=#0x10000003a14' 2024-09-10T08:44:48.998 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10206 fp=#0x10000001f03' 2024-09-10T08:44:48.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10238 fp=#0x100000038bf' 2024-09-10T08:44:48.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10270 fp=#0x10000004d84' 2024-09-10T08:44:48.999 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10302 fp=#0x1000000249f' 2024-09-10T08:44:49.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10334 fp=#0x1000000242e' 2024-09-10T08:44:49.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10366 fp=#0x10000003adf' 2024-09-10T08:44:49.000 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10398 fp=#0x10000001349' 2024-09-10T08:44:49.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10430 fp=#0x10000003948' 2024-09-10T08:44:49.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10462 fp=#0x10000003ae9' 2024-09-10T08:44:49.001 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10494 fp=#0x1000000142d' 2024-09-10T08:44:49.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10526 fp=#0x100000013c7' 2024-09-10T08:44:49.002 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10558 fp=#0x1000000146e' 2024-09-10T08:44:49.003 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10718 fp=#0x10000001036' 2024-09-10T08:44:49.004 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10814 fp=#0x1000000054b' 2024-09-10T08:44:49.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10846 fp=#0x10000000a4a' 2024-09-10T08:44:49.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10878 fp=#0x10000000576' 2024-09-10T08:44:49.005 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10910 fp=#0x10000001e56' 2024-09-10T08:44:49.006 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10942 fp=#0x1000000431c' 2024-09-10T08:44:49.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11038 fp=#0x10000003afe' 2024-09-10T08:44:49.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11070 fp=#0x10000003dc0' 2024-09-10T08:44:49.007 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11102 fp=#0x100000039cb' 2024-09-10T08:44:49.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11134 fp=#0x1000000236f' 2024-09-10T08:44:49.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11166 fp=#0x10000003eb0' 2024-09-10T08:44:49.008 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11198 fp=#0x10000003e7a' 2024-09-10T08:44:49.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11230 fp=#0x10000003364' 2024-09-10T08:44:49.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11262 fp=#0x10000003f00' 2024-09-10T08:44:49.009 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11294 fp=#0x100000021d2' 2024-09-10T08:44:49.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11326 fp=#0x10000003ddd' 2024-09-10T08:44:49.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11358 fp=#0x10000003e86' 2024-09-10T08:44:49.010 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11390 fp=#0x100000019fa' 2024-09-10T08:44:49.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11422 fp=#0x10000001c08' 2024-09-10T08:44:49.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11454 fp=#0x10000003e26' 2024-09-10T08:44:49.011 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11486 fp=#0x10000003ed8' 2024-09-10T08:44:49.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11518 fp=#0x10000003f60' 2024-09-10T08:44:49.012 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11550 fp=#0x10000005159' 2024-09-10T08:44:49.013 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11614 fp=#0x100000051a5' 2024-09-10T08:44:49.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11710 fp=#0x10000000904' 2024-09-10T08:44:49.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11742 fp=#0x10000000861' 2024-09-10T08:44:49.014 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11774 fp=#0x1000000086c' 2024-09-10T08:44:49.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11806 fp=#0x10000000a1a' 2024-09-10T08:44:49.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11838 fp=#0x10000000b27' 2024-09-10T08:44:49.015 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11870 fp=#0x1000000423c' 2024-09-10T08:44:49.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11902 fp=#0x10000004bbc' 2024-09-10T08:44:49.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11934 fp=#0x10000005293' 2024-09-10T08:44:49.016 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11966 fp=#0x10000000efb' 2024-09-10T08:44:49.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11998 fp=#0x10000000f06' 2024-09-10T08:44:49.017 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12062 fp=#0x100000034a8' 2024-09-10T08:44:49.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12094 fp=#0x100000044b0' 2024-09-10T08:44:49.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12126 fp=#0x100000045c8' 2024-09-10T08:44:49.018 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12158 fp=#0x1000000439b' 2024-09-10T08:44:49.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12190 fp=#0x10000003441' 2024-09-10T08:44:49.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12222 fp=#0x10000000d36' 2024-09-10T08:44:49.019 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12254 fp=#0x100000051e3' 2024-09-10T08:44:49.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12286 fp=#0x10000000d49' 2024-09-10T08:44:49.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12318 fp=#0x10000003bf9' 2024-09-10T08:44:49.020 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12350 fp=#0x10000003af1' 2024-09-10T08:44:49.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12382 fp=#0x10000004de6' 2024-09-10T08:44:49.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12414 fp=#0x10000000d26' 2024-09-10T08:44:49.021 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12446 fp=#0x10000000d1c' 2024-09-10T08:44:49.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12478 fp=#0x10000004c11' 2024-09-10T08:44:49.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12510 fp=#0x10000004c82' 2024-09-10T08:44:49.022 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12542 fp=#0x1000000250d' 2024-09-10T08:44:49.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12574 fp=#0x10000004dbc' 2024-09-10T08:44:49.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12606 fp=#0x10000004d92' 2024-09-10T08:44:49.023 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12638 fp=#0x10000004bbb' 2024-09-10T08:44:49.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12670 fp=#0x10000005393' 2024-09-10T08:44:49.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12702 fp=#0x1000000538d' 2024-09-10T08:44:49.024 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12734 fp=#0x10000004889' 2024-09-10T08:44:49.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12766 fp=#0x10000005403' 2024-09-10T08:44:49.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12798 fp=#0x10000002338' 2024-09-10T08:44:49.025 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12830 fp=#0x10000002312' 2024-09-10T08:44:49.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12862 fp=#0x100000021c5' 2024-09-10T08:44:49.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12894 fp=#0x100000022f0' 2024-09-10T08:44:49.026 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12926 fp=#0x1000000541a' 2024-09-10T08:44:49.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12958 fp=#0x10000002777' 2024-09-10T08:44:49.027 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12990 fp=#0x1000000278d' 2024-09-10T08:44:49.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13022 fp=#0x100000053a3' 2024-09-10T08:44:49.028 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13110 fp=#0x1000000392e' 2024-09-10T08:44:49.029 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13142 fp=#0x10000005422' 2024-09-10T08:44:49.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:263 fp=#0x10000003b3b' 2024-09-10T08:44:49.030 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:295 fp=#0x1000000390e' 2024-09-10T08:44:49.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:327 fp=#0x10000000eec' 2024-09-10T08:44:49.031 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:383 fp=#0x10000004716' 2024-09-10T08:44:49.032 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:415 fp=#0x10000004b69' 2024-09-10T08:44:49.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:575 fp=#0x100000032f7' 2024-09-10T08:44:49.033 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:607 fp=#0x100000034b2' 2024-09-10T08:44:49.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:799 fp=#0x10000000f1a' 2024-09-10T08:44:49.035 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:831 fp=#0x10000000deb' 2024-09-10T08:44:49.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:863 fp=#0x100000045c0' 2024-09-10T08:44:49.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:895 fp=#0x10000000d89' 2024-09-10T08:44:49.036 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:927 fp=#0x10000000d86' 2024-09-10T08:44:49.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:959 fp=#0x10000000dc1' 2024-09-10T08:44:49.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:991 fp=#0x100000044e0' 2024-09-10T08:44:49.037 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1023 fp=#0x1000000459e' 2024-09-10T08:44:49.038 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1119 fp=#0x100000041c5' 2024-09-10T08:44:49.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1215 fp=#0x10000004229' 2024-09-10T08:44:49.039 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1247 fp=#0x1000000365c' 2024-09-10T08:44:49.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1311 fp=#0x10000002003' 2024-09-10T08:44:49.040 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1343 fp=#0x100000035f3' 2024-09-10T08:44:49.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1375 fp=#0x100000032f4' 2024-09-10T08:44:49.041 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1407 fp=#0x1000000328c' 2024-09-10T08:44:49.043 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1663 fp=#0x10000004607' 2024-09-10T08:44:49.044 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1727 fp=#0x10000003272' 2024-09-10T08:44:49.045 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1919 fp=#0x100000039fa' 2024-09-10T08:44:49.046 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2047 fp=#0x100000021f0' 2024-09-10T08:44:49.048 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2175 fp=#0x100000021c3' 2024-09-10T08:44:49.049 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2303 fp=#0x100000026a8' 2024-09-10T08:44:49.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2591 fp=#0x10000001074' 2024-09-10T08:44:49.051 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2623 fp=#0x10000001904' 2024-09-10T08:44:49.052 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2687 fp=#0x100000017e5' 2024-09-10T08:44:49.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2751 fp=#0x10000003b77' 2024-09-10T08:44:49.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2783 fp=#0x10000003684' 2024-09-10T08:44:49.053 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2815 fp=#0x10000003585' 2024-09-10T08:44:49.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2847 fp=#0x100000047de' 2024-09-10T08:44:49.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2879 fp=#0x1000000484c' 2024-09-10T08:44:49.054 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2911 fp=#0x100000046bd' 2024-09-10T08:44:49.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2943 fp=#0x1000000474a' 2024-09-10T08:44:49.055 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3007 fp=#0x10000001146' 2024-09-10T08:44:49.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3039 fp=#0x10000004a19' 2024-09-10T08:44:49.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3071 fp=#0x10000004a1d' 2024-09-10T08:44:49.056 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3103 fp=#0x1000000198c' 2024-09-10T08:44:49.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3135 fp=#0x10000000e04' 2024-09-10T08:44:49.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3167 fp=#0x100000049e6' 2024-09-10T08:44:49.057 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3199 fp=#0x10000004a4b' 2024-09-10T08:44:49.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3231 fp=#0x10000000720' 2024-09-10T08:44:49.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3263 fp=#0x10000000791' 2024-09-10T08:44:49.058 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3295 fp=#0x10000000795' 2024-09-10T08:44:49.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3327 fp=#0x10000003d4b' 2024-09-10T08:44:49.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3359 fp=#0x1000000071d' 2024-09-10T08:44:49.059 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3391 fp=#0x10000001869' 2024-09-10T08:44:49.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3423 fp=#0x1000000077d' 2024-09-10T08:44:49.060 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3455 fp=#0x100000041c4' 2024-09-10T08:44:49.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3519 fp=#0x100000040b3' 2024-09-10T08:44:49.061 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3551 fp=#0x10000003c83' 2024-09-10T08:44:49.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3615 fp=#0x10000004171' 2024-09-10T08:44:49.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3647 fp=#0x10000004af0' 2024-09-10T08:44:49.062 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3679 fp=#0x10000004d20' 2024-09-10T08:44:49.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3711 fp=#0x10000004ac6' 2024-09-10T08:44:49.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3743 fp=#0x100000052c1' 2024-09-10T08:44:49.063 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3775 fp=#0x10000004932' 2024-09-10T08:44:49.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3807 fp=#0x10000003f28' 2024-09-10T08:44:49.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3839 fp=#0x10000002764' 2024-09-10T08:44:49.064 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3871 fp=#0x10000002573' 2024-09-10T08:44:49.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3903 fp=#0x10000003f2d' 2024-09-10T08:44:49.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3935 fp=#0x1000000068f' 2024-09-10T08:44:49.065 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3967 fp=#0x1000000059f' 2024-09-10T08:44:49.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3999 fp=#0x100000011fa' 2024-09-10T08:44:49.066 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4031 fp=#0x10000000653' 2024-09-10T08:44:49.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4063 fp=#0x10000004dd4' 2024-09-10T08:44:49.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4095 fp=#0x10000004637' 2024-09-10T08:44:49.067 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4127 fp=#0x1000000118b' 2024-09-10T08:44:49.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4159 fp=#0x1000000471b' 2024-09-10T08:44:49.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4191 fp=#0x10000004699' 2024-09-10T08:44:49.068 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4223 fp=#0x10000003afb' 2024-09-10T08:44:49.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4255 fp=#0x10000004603' 2024-09-10T08:44:49.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4287 fp=#0x100000039db' 2024-09-10T08:44:49.069 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4319 fp=#0x10000003b2a' 2024-09-10T08:44:49.070 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4351 fp=#0x10000003b8b' 2024-09-10T08:44:49.070 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4415 fp=#0x10000002020' 2024-09-10T08:44:49.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4479 fp=#0x10000003b62' 2024-09-10T08:44:49.071 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4543 fp=#0x10000003597' 2024-09-10T08:44:49.072 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4607 fp=#0x1000000397c' 2024-09-10T08:44:49.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4639 fp=#0x100000014ce' 2024-09-10T08:44:49.073 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4703 fp=#0x100000038da' 2024-09-10T08:44:49.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4735 fp=#0x1000000130a' 2024-09-10T08:44:49.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4767 fp=#0x10000001270' 2024-09-10T08:44:49.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4799 fp=#0x100000013de' 2024-09-10T08:44:49.074 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4831 fp=#0x10000004a35' 2024-09-10T08:44:49.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4863 fp=#0x10000004a6e' 2024-09-10T08:44:49.075 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4895 fp=#0x100000037c2' 2024-09-10T08:44:49.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4927 fp=#0x10000004aa0' 2024-09-10T08:44:49.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4959 fp=#0x100000051ca' 2024-09-10T08:44:49.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4991 fp=#0x1000000511e' 2024-09-10T08:44:49.076 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5023 fp=#0x1000000518b' 2024-09-10T08:44:49.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5055 fp=#0x100000051bc' 2024-09-10T08:44:49.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5087 fp=#0x100000051d2' 2024-09-10T08:44:49.077 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5119 fp=#0x100000051a7' 2024-09-10T08:44:49.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5151 fp=#0x10000001d17' 2024-09-10T08:44:49.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5183 fp=#0x10000001ec5' 2024-09-10T08:44:49.078 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5215 fp=#0x10000001d61' 2024-09-10T08:44:49.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5247 fp=#0x10000001c4b' 2024-09-10T08:44:49.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5279 fp=#0x10000003908' 2024-09-10T08:44:49.079 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5311 fp=#0x1000000372f' 2024-09-10T08:44:49.080 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5343 fp=#0x10000004a90' 2024-09-10T08:44:49.080 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5375 fp=#0x10000004652' 2024-09-10T08:44:49.081 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5407 fp=#0x10000003891' 2024-09-10T08:44:49.081 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5439 fp=#0x10000003d18' 2024-09-10T08:44:49.082 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5503 fp=#0x1000000150e' 2024-09-10T08:44:49.082 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5535 fp=#0x1000000121a' 2024-09-10T08:44:49.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5599 fp=#0x10000001103' 2024-09-10T08:44:49.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5631 fp=#0x1000000104b' 2024-09-10T08:44:49.083 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5663 fp=#0x100000020a6' 2024-09-10T08:44:49.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5727 fp=#0x10000000f90' 2024-09-10T08:44:49.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5759 fp=#0x10000000f30' 2024-09-10T08:44:49.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5791 fp=#0x10000004a77' 2024-09-10T08:44:49.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5823 fp=#0x10000004cb0' 2024-09-10T08:44:49.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5855 fp=#0x10000004d83' 2024-09-10T08:44:49.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5887 fp=#0x10000004646' 2024-09-10T08:44:49.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5919 fp=#0x100000047a2' 2024-09-10T08:44:49.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5951 fp=#0x10000004460' 2024-09-10T08:44:49.086 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5983 fp=#0x10000004386' 2024-09-10T08:44:49.088 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6143 fp=#0x100000033a6' 2024-09-10T08:44:49.088 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6175 fp=#0x10000004829' 2024-09-10T08:44:49.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6271 fp=#0x10000001e05' 2024-09-10T08:44:49.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6303 fp=#0x100000032fa' 2024-09-10T08:44:49.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6367 fp=#0x10000001e6b' 2024-09-10T08:44:49.090 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6431 fp=#0x10000004b81' 2024-09-10T08:44:49.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6463 fp=#0x10000004e50' 2024-09-10T08:44:49.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6495 fp=#0x10000004bee' 2024-09-10T08:44:49.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6527 fp=#0x10000004e88' 2024-09-10T08:44:49.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6559 fp=#0x10000004e5e' 2024-09-10T08:44:49.092 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6623 fp=#0x10000003ac9' 2024-09-10T08:44:49.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6655 fp=#0x1000000500b' 2024-09-10T08:44:49.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6687 fp=#0x10000005048' 2024-09-10T08:44:49.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6719 fp=#0x10000003b5b' 2024-09-10T08:44:49.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6751 fp=#0x10000004f8d' 2024-09-10T08:44:49.094 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6783 fp=#0x10000003be9' 2024-09-10T08:44:49.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6815 fp=#0x10000005324' 2024-09-10T08:44:49.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6847 fp=#0x10000004724' 2024-09-10T08:44:49.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6879 fp=#0x10000004765' 2024-09-10T08:44:49.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6911 fp=#0x10000004670' 2024-09-10T08:44:49.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6943 fp=#0x1000000531c' 2024-09-10T08:44:49.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6975 fp=#0x10000002070' 2024-09-10T08:44:49.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7007 fp=#0x10000000652' 2024-09-10T08:44:49.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7039 fp=#0x10000000654' 2024-09-10T08:44:49.097 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7071 fp=#0x10000000671' 2024-09-10T08:44:49.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7103 fp=#0x10000004f40' 2024-09-10T08:44:49.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7135 fp=#0x10000004f48' 2024-09-10T08:44:49.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7167 fp=#0x10000000648' 2024-09-10T08:44:49.099 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7231 fp=#0x10000004527' 2024-09-10T08:44:49.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7295 fp=#0x10000005061' 2024-09-10T08:44:49.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7391 fp=#0x10000005069' 2024-09-10T08:44:49.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7423 fp=#0x100000047fe' 2024-09-10T08:44:49.101 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7455 fp=#0x1000000487c' 2024-09-10T08:44:49.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7487 fp=#0x10000004775' 2024-09-10T08:44:49.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7519 fp=#0x10000004850' 2024-09-10T08:44:49.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7551 fp=#0x100000051c9' 2024-09-10T08:44:49.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7615 fp=#0x100000051d4' 2024-09-10T08:44:49.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7711 fp=#0x10000004f67' 2024-09-10T08:44:49.104 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7743 fp=#0x10000004ea2' 2024-09-10T08:44:49.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7775 fp=#0x10000004ef7' 2024-09-10T08:44:49.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7807 fp=#0x100000013f1' 2024-09-10T08:44:49.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7839 fp=#0x10000004d42' 2024-09-10T08:44:49.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7871 fp=#0x10000004d49' 2024-09-10T08:44:49.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7903 fp=#0x10000004ee2' 2024-09-10T08:44:49.106 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7935 fp=#0x10000004e5b' 2024-09-10T08:44:49.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7967 fp=#0x10000004e86' 2024-09-10T08:44:49.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7999 fp=#0x10000004ea4' 2024-09-10T08:44:49.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8031 fp=#0x10000001699' 2024-09-10T08:44:49.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8063 fp=#0x100000016b4' 2024-09-10T08:44:49.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8095 fp=#0x100000018a0' 2024-09-10T08:44:49.108 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8127 fp=#0x1000000400e' 2024-09-10T08:44:49.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8159 fp=#0x100000026a4' 2024-09-10T08:44:49.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8191 fp=#0x1000000494c' 2024-09-10T08:44:49.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8223 fp=#0x1000000498a' 2024-09-10T08:44:49.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8255 fp=#0x100000045c3' 2024-09-10T08:44:49.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8287 fp=#0x10000004155' 2024-09-10T08:44:49.110 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8319 fp=#0x10000005272' 2024-09-10T08:44:49.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8351 fp=#0x1000000524b' 2024-09-10T08:44:49.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8383 fp=#0x10000004373' 2024-09-10T08:44:49.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8415 fp=#0x1000000427b' 2024-09-10T08:44:49.112 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8447 fp=#0x1000000524d' 2024-09-10T08:44:49.112 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8479 fp=#0x10000004010' 2024-09-10T08:44:49.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8511 fp=#0x10000003f32' 2024-09-10T08:44:49.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8543 fp=#0x10000003da8' 2024-09-10T08:44:49.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8575 fp=#0x1000000420f' 2024-09-10T08:44:49.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8607 fp=#0x1000000424a' 2024-09-10T08:44:49.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8639 fp=#0x10000003ce9' 2024-09-10T08:44:49.114 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8671 fp=#0x10000003f96' 2024-09-10T08:44:49.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8703 fp=#0x10000004f0c' 2024-09-10T08:44:49.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8735 fp=#0x10000003df1' 2024-09-10T08:44:49.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8767 fp=#0x10000004a33' 2024-09-10T08:44:49.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8799 fp=#0x10000004f60' 2024-09-10T08:44:49.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8831 fp=#0x10000001aa2' 2024-09-10T08:44:49.116 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8863 fp=#0x10000001b64' 2024-09-10T08:44:49.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8895 fp=#0x10000001adc' 2024-09-10T08:44:49.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8927 fp=#0x10000001da0' 2024-09-10T08:44:49.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8959 fp=#0x10000001d5d' 2024-09-10T08:44:49.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8991 fp=#0x10000001cb8' 2024-09-10T08:44:49.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9023 fp=#0x10000000d14' 2024-09-10T08:44:49.118 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9055 fp=#0x10000001e3d' 2024-09-10T08:44:49.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9087 fp=#0x100000041c2' 2024-09-10T08:44:49.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9119 fp=#0x1000000426a' 2024-09-10T08:44:49.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9151 fp=#0x100000011bd' 2024-09-10T08:44:49.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9183 fp=#0x10000000d44' 2024-09-10T08:44:49.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9215 fp=#0x100000026c2' 2024-09-10T08:44:49.120 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9247 fp=#0x10000000aa0' 2024-09-10T08:44:49.121 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9279 fp=#0x10000000c3d' 2024-09-10T08:44:49.121 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9311 fp=#0x10000000a98' 2024-09-10T08:44:49.121 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9343 fp=#0x10000000b6f' 2024-09-10T08:44:49.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9375 fp=#0x1000000103a' 2024-09-10T08:44:49.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9407 fp=#0x100000011a3' 2024-09-10T08:44:49.122 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9439 fp=#0x10000002622' 2024-09-10T08:44:49.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9471 fp=#0x10000000b4e' 2024-09-10T08:44:49.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9503 fp=#0x10000000ece' 2024-09-10T08:44:49.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9535 fp=#0x10000001ad2' 2024-09-10T08:44:49.124 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9567 fp=#0x10000001a42' 2024-09-10T08:44:49.124 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9599 fp=#0x10000000ea8' 2024-09-10T08:44:49.124 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9631 fp=#0x100000048d3' 2024-09-10T08:44:49.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9663 fp=#0x10000004788' 2024-09-10T08:44:49.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9695 fp=#0x100000048ee' 2024-09-10T08:44:49.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9727 fp=#0x10000001ad7' 2024-09-10T08:44:49.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9759 fp=#0x100000019dc' 2024-09-10T08:44:49.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9791 fp=#0x10000001b0d' 2024-09-10T08:44:49.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9823 fp=#0x100000043f8' 2024-09-10T08:44:49.126 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9855 fp=#0x10000004485' 2024-09-10T08:44:49.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9887 fp=#0x100000019b7' 2024-09-10T08:44:49.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9919 fp=#0x10000001596' 2024-09-10T08:44:49.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9951 fp=#0x10000001740' 2024-09-10T08:44:49.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9983 fp=#0x100000015c4' 2024-09-10T08:44:49.128 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10015 fp=#0x100000018e0' 2024-09-10T08:44:49.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10047 fp=#0x10000004dae' 2024-09-10T08:44:49.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10079 fp=#0x100000015ad' 2024-09-10T08:44:49.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10111 fp=#0x10000001f2e' 2024-09-10T08:44:49.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10143 fp=#0x10000003933' 2024-09-10T08:44:49.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10175 fp=#0x100000038f5' 2024-09-10T08:44:49.130 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10207 fp=#0x10000001e1e' 2024-09-10T08:44:49.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10239 fp=#0x10000003703' 2024-09-10T08:44:49.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10271 fp=#0x10000002399' 2024-09-10T08:44:49.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10303 fp=#0x100000023ce' 2024-09-10T08:44:49.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10335 fp=#0x10000002452' 2024-09-10T08:44:49.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10367 fp=#0x10000003a92' 2024-09-10T08:44:49.132 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10399 fp=#0x10000001420' 2024-09-10T08:44:49.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10431 fp=#0x100000012e9' 2024-09-10T08:44:49.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10463 fp=#0x10000001545' 2024-09-10T08:44:49.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10495 fp=#0x100000013d6' 2024-09-10T08:44:49.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10527 fp=#0x100000038fd' 2024-09-10T08:44:49.134 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10559 fp=#0x1000000130b' 2024-09-10T08:44:49.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10623 fp=#0x10000001f2f' 2024-09-10T08:44:49.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10687 fp=#0x10000004a00' 2024-09-10T08:44:49.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10719 fp=#0x10000000d12' 2024-09-10T08:44:49.136 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10783 fp=#0x100000049f6' 2024-09-10T08:44:49.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10815 fp=#0x1000000059a' 2024-09-10T08:44:49.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10847 fp=#0x100000007bb' 2024-09-10T08:44:49.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10879 fp=#0x10000002062' 2024-09-10T08:44:49.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10911 fp=#0x10000002628' 2024-09-10T08:44:49.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10943 fp=#0x10000000586' 2024-09-10T08:44:49.138 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10975 fp=#0x1000000059e' 2024-09-10T08:44:49.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11007 fp=#0x100000009a7' 2024-09-10T08:44:49.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11039 fp=#0x10000003af7' 2024-09-10T08:44:49.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11071 fp=#0x10000003aef' 2024-09-10T08:44:49.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11103 fp=#0x10000003888' 2024-09-10T08:44:49.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11135 fp=#0x10000003fcd' 2024-09-10T08:44:49.140 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11167 fp=#0x100000021d7' 2024-09-10T08:44:49.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11199 fp=#0x10000003e24' 2024-09-10T08:44:49.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11231 fp=#0x10000003e3e' 2024-09-10T08:44:49.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11263 fp=#0x100000040de' 2024-09-10T08:44:49.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11295 fp=#0x10000002069' 2024-09-10T08:44:49.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11327 fp=#0x10000001d3f' 2024-09-10T08:44:49.142 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11359 fp=#0x10000001cc6' 2024-09-10T08:44:49.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11391 fp=#0x10000001a3e' 2024-09-10T08:44:49.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11423 fp=#0x10000003d87' 2024-09-10T08:44:49.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11455 fp=#0x10000001ab3' 2024-09-10T08:44:49.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11487 fp=#0x10000003eb5' 2024-09-10T08:44:49.144 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11519 fp=#0x10000003e71' 2024-09-10T08:44:49.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11583 fp=#0x1000000515b' 2024-09-10T08:44:49.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11679 fp=#0x10000005168' 2024-09-10T08:44:49.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11711 fp=#0x10000000960' 2024-09-10T08:44:49.146 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11743 fp=#0x10000000a07' 2024-09-10T08:44:49.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11775 fp=#0x10000004df2' 2024-09-10T08:44:49.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11807 fp=#0x100000009fe' 2024-09-10T08:44:49.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11839 fp=#0x100000009ea' 2024-09-10T08:44:49.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11871 fp=#0x1000000095d' 2024-09-10T08:44:49.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11903 fp=#0x10000000f24' 2024-09-10T08:44:49.148 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11935 fp=#0x10000003534' 2024-09-10T08:44:49.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11967 fp=#0x10000000f22' 2024-09-10T08:44:49.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12031 fp=#0x100000010ec' 2024-09-10T08:44:49.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12063 fp=#0x10000004458' 2024-09-10T08:44:49.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12095 fp=#0x100000043c1' 2024-09-10T08:44:49.150 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12127 fp=#0x100000044a0' 2024-09-10T08:44:49.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12159 fp=#0x10000003971' 2024-09-10T08:44:49.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12191 fp=#0x1000000438f' 2024-09-10T08:44:49.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12223 fp=#0x10000000d79' 2024-09-10T08:44:49.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12255 fp=#0x10000000d45' 2024-09-10T08:44:49.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12287 fp=#0x10000003a18' 2024-09-10T08:44:49.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12319 fp=#0x10000000d13' 2024-09-10T08:44:49.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12351 fp=#0x10000002607' 2024-09-10T08:44:49.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12383 fp=#0x10000000cfd' 2024-09-10T08:44:49.153 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12415 fp=#0x10000000be4' 2024-09-10T08:44:49.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12447 fp=#0x10000000c85' 2024-09-10T08:44:49.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12479 fp=#0x1000000177f' 2024-09-10T08:44:49.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12511 fp=#0x100000012c1' 2024-09-10T08:44:49.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12543 fp=#0x10000001021' 2024-09-10T08:44:49.159 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12575 fp=#0x10000004c66' 2024-09-10T08:44:49.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12607 fp=#0x10000004ced' 2024-09-10T08:44:49.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12639 fp=#0x10000004ace' 2024-09-10T08:44:49.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12671 fp=#0x100000048ec' 2024-09-10T08:44:49.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12703 fp=#0x1000000483a' 2024-09-10T08:44:49.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12735 fp=#0x1000000474b' 2024-09-10T08:44:49.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12767 fp=#0x10000005412' 2024-09-10T08:44:49.164 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12799 fp=#0x1000000213e' 2024-09-10T08:44:49.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12831 fp=#0x100000040ef' 2024-09-10T08:44:49.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12863 fp=#0x100000052b9' 2024-09-10T08:44:49.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12895 fp=#0x1000000249a' 2024-09-10T08:44:49.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12927 fp=#0x100000053fa' 2024-09-10T08:44:49.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12959 fp=#0x100000052e1' 2024-09-10T08:44:49.166 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12991 fp=#0x10000003d9a' 2024-09-10T08:44:49.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13023 fp=#0x10000002712' 2024-09-10T08:44:49.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13047 fp=#0x10000002332' 2024-09-10T08:44:49.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13079 fp=#0x10000003a41' 2024-09-10T08:44:49.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13111 fp=#0x10000003974' 2024-09-10T08:44:49.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13143 fp=#0x10000003940' 2024-09-10T08:44:49.169 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:264 fp=#0x10000003390' 2024-09-10T08:44:49.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:296 fp=#0x1000000452e' 2024-09-10T08:44:49.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:328 fp=#0x10000003313' 2024-09-10T08:44:49.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:384 fp=#0x10000004b0f' 2024-09-10T08:44:49.171 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:416 fp=#0x10000004721' 2024-09-10T08:44:49.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:480 fp=#0x100000034f7' 2024-09-10T08:44:49.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:544 fp=#0x1000000354b' 2024-09-10T08:44:49.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:576 fp=#0x100000032d0' 2024-09-10T08:44:49.174 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:800 fp=#0x100000045a2' 2024-09-10T08:44:49.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:832 fp=#0x10000000e85' 2024-09-10T08:44:49.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:896 fp=#0x10000000d91' 2024-09-10T08:44:49.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:928 fp=#0x10000000e4d' 2024-09-10T08:44:49.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:960 fp=#0x10000000ebb' 2024-09-10T08:44:49.176 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:992 fp=#0x1000000401f' 2024-09-10T08:44:49.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1088 fp=#0x10000004243' 2024-09-10T08:44:49.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1120 fp=#0x100000041d9' 2024-09-10T08:44:49.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1152 fp=#0x10000004109' 2024-09-10T08:44:49.178 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1184 fp=#0x10000004233' 2024-09-10T08:44:49.179 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1248 fp=#0x100000035bd' 2024-09-10T08:44:49.179 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1280 fp=#0x1000000209e' 2024-09-10T08:44:49.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1312 fp=#0x10000001f92' 2024-09-10T08:44:49.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1344 fp=#0x10000003622' 2024-09-10T08:44:49.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1408 fp=#0x1000000434a' 2024-09-10T08:44:49.182 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1536 fp=#0x100000042c1' 2024-09-10T08:44:49.182 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1568 fp=#0x10000003cac' 2024-09-10T08:44:49.183 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1664 fp=#0x100000045dd' 2024-09-10T08:44:49.183 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1696 fp=#0x10000003344' 2024-09-10T08:44:49.186 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2112 fp=#0x100000021e0' 2024-09-10T08:44:49.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2176 fp=#0x10000002125' 2024-09-10T08:44:49.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2240 fp=#0x10000002715' 2024-09-10T08:44:49.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2272 fp=#0x10000002776' 2024-09-10T08:44:49.188 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2304 fp=#0x10000002785' 2024-09-10T08:44:49.189 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2336 fp=#0x1000000278b' 2024-09-10T08:44:49.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2592 fp=#0x1000000120e' 2024-09-10T08:44:49.191 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2624 fp=#0x100000010cb' 2024-09-10T08:44:49.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2656 fp=#0x1000000121b' 2024-09-10T08:44:49.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2720 fp=#0x10000001084' 2024-09-10T08:44:49.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2752 fp=#0x10000003b94' 2024-09-10T08:44:49.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2816 fp=#0x100000047c5' 2024-09-10T08:44:49.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2848 fp=#0x100000047ca' 2024-09-10T08:44:49.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2912 fp=#0x1000000343e' 2024-09-10T08:44:49.194 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2944 fp=#0x10000004784' 2024-09-10T08:44:49.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2976 fp=#0x10000004812' 2024-09-10T08:44:49.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3008 fp=#0x10000000de6' 2024-09-10T08:44:49.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3040 fp=#0x10000004a14' 2024-09-10T08:44:49.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3072 fp=#0x10000004a6c' 2024-09-10T08:44:49.196 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3104 fp=#0x10000004a60' 2024-09-10T08:44:49.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3136 fp=#0x1000000190a' 2024-09-10T08:44:49.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3168 fp=#0x100000049c3' 2024-09-10T08:44:49.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3200 fp=#0x100000019a2' 2024-09-10T08:44:49.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3232 fp=#0x100000006f9' 2024-09-10T08:44:49.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3264 fp=#0x10000000779' 2024-09-10T08:44:49.198 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3296 fp=#0x10000003d0c' 2024-09-10T08:44:49.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3328 fp=#0x100000007e8' 2024-09-10T08:44:49.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3360 fp=#0x10000000770' 2024-09-10T08:44:49.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3392 fp=#0x10000003d99' 2024-09-10T08:44:49.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3424 fp=#0x1000000076f' 2024-09-10T08:44:49.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3488 fp=#0x100000040d1' 2024-09-10T08:44:49.201 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3616 fp=#0x100000040f8' 2024-09-10T08:44:49.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3648 fp=#0x10000004c33' 2024-09-10T08:44:49.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3680 fp=#0x10000004b11' 2024-09-10T08:44:49.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3712 fp=#0x10000004b8b' 2024-09-10T08:44:49.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3744 fp=#0x10000003f4d' 2024-09-10T08:44:49.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3776 fp=#0x1000000274a' 2024-09-10T08:44:49.203 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3808 fp=#0x10000003fc6' 2024-09-10T08:44:49.204 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3840 fp=#0x10000002780' 2024-09-10T08:44:49.204 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3872 fp=#0x10000003f38' 2024-09-10T08:44:49.204 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3904 fp=#0x10000004f73' 2024-09-10T08:44:49.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3936 fp=#0x10000002268' 2024-09-10T08:44:49.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3968 fp=#0x1000000057f' 2024-09-10T08:44:49.205 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4000 fp=#0x10000000567' 2024-09-10T08:44:49.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4032 fp=#0x10000000fed' 2024-09-10T08:44:49.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4064 fp=#0x10000000fb9' 2024-09-10T08:44:49.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4096 fp=#0x10000004ea9' 2024-09-10T08:44:49.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4128 fp=#0x100000044df' 2024-09-10T08:44:49.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4160 fp=#0x1000000467b' 2024-09-10T08:44:49.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4224 fp=#0x100000041b0' 2024-09-10T08:44:49.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4256 fp=#0x1000000407e' 2024-09-10T08:44:49.208 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4288 fp=#0x10000004027' 2024-09-10T08:44:49.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4320 fp=#0x10000003b06' 2024-09-10T08:44:49.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4352 fp=#0x100000040cf' 2024-09-10T08:44:49.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4480 fp=#0x10000005307' 2024-09-10T08:44:49.210 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4512 fp=#0x100000050b9' 2024-09-10T08:44:49.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4544 fp=#0x10000001c63' 2024-09-10T08:44:49.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4576 fp=#0x10000004037' 2024-09-10T08:44:49.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4608 fp=#0x1000000393d' 2024-09-10T08:44:49.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4640 fp=#0x1000000392a' 2024-09-10T08:44:49.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4672 fp=#0x100000012db' 2024-09-10T08:44:49.212 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4704 fp=#0x1000000390a' 2024-09-10T08:44:49.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4736 fp=#0x10000001583' 2024-09-10T08:44:49.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4768 fp=#0x100000038ef' 2024-09-10T08:44:49.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4800 fp=#0x10000001516' 2024-09-10T08:44:49.214 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4832 fp=#0x100000037d1' 2024-09-10T08:44:49.214 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4864 fp=#0x100000037a2' 2024-09-10T08:44:49.214 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4896 fp=#0x10000003777' 2024-09-10T08:44:49.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4928 fp=#0x10000004a84' 2024-09-10T08:44:49.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4960 fp=#0x100000053c9' 2024-09-10T08:44:49.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4992 fp=#0x100000051de' 2024-09-10T08:44:49.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5024 fp=#0x10000005056' 2024-09-10T08:44:49.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5056 fp=#0x10000004fe0' 2024-09-10T08:44:49.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5088 fp=#0x10000004fe9' 2024-09-10T08:44:49.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5120 fp=#0x10000001e91' 2024-09-10T08:44:49.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5152 fp=#0x10000001c8d' 2024-09-10T08:44:49.217 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5184 fp=#0x10000001d22' 2024-09-10T08:44:49.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5216 fp=#0x10000001b86' 2024-09-10T08:44:49.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5248 fp=#0x10000001e10' 2024-09-10T08:44:49.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5280 fp=#0x10000004a93' 2024-09-10T08:44:49.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5312 fp=#0x100000035c2' 2024-09-10T08:44:49.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5344 fp=#0x100000036af' 2024-09-10T08:44:49.219 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5376 fp=#0x10000003887' 2024-09-10T08:44:49.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5408 fp=#0x10000003dad' 2024-09-10T08:44:49.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5440 fp=#0x10000001ede' 2024-09-10T08:44:49.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5472 fp=#0x100000014d9' 2024-09-10T08:44:49.221 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5536 fp=#0x100000012a1' 2024-09-10T08:44:49.221 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5568 fp=#0x10000001514' 2024-09-10T08:44:49.222 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5600 fp=#0x100000050ea' 2024-09-10T08:44:49.222 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5632 fp=#0x10000000f3f' 2024-09-10T08:44:49.222 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5664 fp=#0x10000000f51' 2024-09-10T08:44:49.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5696 fp=#0x10000005109' 2024-09-10T08:44:49.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5728 fp=#0x100000020b3' 2024-09-10T08:44:49.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5760 fp=#0x10000000f7d' 2024-09-10T08:44:49.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5792 fp=#0x10000004cd5' 2024-09-10T08:44:49.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5824 fp=#0x10000004cf5' 2024-09-10T08:44:49.224 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5856 fp=#0x100000043d0' 2024-09-10T08:44:49.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5888 fp=#0x10000004d5b' 2024-09-10T08:44:49.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5920 fp=#0x10000004470' 2024-09-10T08:44:49.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5952 fp=#0x100000042ec' 2024-09-10T08:44:49.226 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5984 fp=#0x10000004c34' 2024-09-10T08:44:49.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6144 fp=#0x1000000355d' 2024-09-10T08:44:49.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6240 fp=#0x100000035f7' 2024-09-10T08:44:49.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6272 fp=#0x10000003370' 2024-09-10T08:44:49.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6304 fp=#0x100000033b8' 2024-09-10T08:44:49.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6336 fp=#0x10000003300' 2024-09-10T08:44:49.229 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6368 fp=#0x10000003437' 2024-09-10T08:44:49.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6400 fp=#0x10000004bdf' 2024-09-10T08:44:49.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6432 fp=#0x10000004bb4' 2024-09-10T08:44:49.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6464 fp=#0x10000004cc2' 2024-09-10T08:44:49.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6496 fp=#0x10000004d80' 2024-09-10T08:44:49.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6528 fp=#0x10000004dee' 2024-09-10T08:44:49.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6560 fp=#0x10000004b9a' 2024-09-10T08:44:49.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6624 fp=#0x10000005289' 2024-09-10T08:44:49.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6656 fp=#0x100000044c7' 2024-09-10T08:44:49.232 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6688 fp=#0x10000003c2b' 2024-09-10T08:44:49.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6720 fp=#0x10000003aa5' 2024-09-10T08:44:49.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6752 fp=#0x10000004fdb' 2024-09-10T08:44:49.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6784 fp=#0x10000003965' 2024-09-10T08:44:49.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6816 fp=#0x10000004741' 2024-09-10T08:44:49.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6848 fp=#0x1000000536a' 2024-09-10T08:44:49.234 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6880 fp=#0x10000004685' 2024-09-10T08:44:49.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6912 fp=#0x100000052e3' 2024-09-10T08:44:49.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6944 fp=#0x100000045ca' 2024-09-10T08:44:49.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6976 fp=#0x100000005be' 2024-09-10T08:44:49.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7008 fp=#0x100000006a0' 2024-09-10T08:44:49.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7040 fp=#0x10000005046' 2024-09-10T08:44:49.236 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7072 fp=#0x10000004fb0' 2024-09-10T08:44:49.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7104 fp=#0x10000001e43' 2024-09-10T08:44:49.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7136 fp=#0x100000010c8' 2024-09-10T08:44:49.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7168 fp=#0x10000000670' 2024-09-10T08:44:49.238 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7264 fp=#0x10000005101' 2024-09-10T08:44:49.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7296 fp=#0x10000005142' 2024-09-10T08:44:49.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7360 fp=#0x100000050c2' 2024-09-10T08:44:49.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7392 fp=#0x10000002443' 2024-09-10T08:44:49.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7424 fp=#0x100000046f3' 2024-09-10T08:44:49.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7456 fp=#0x100000046e3' 2024-09-10T08:44:49.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7488 fp=#0x10000002367' 2024-09-10T08:44:49.241 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7520 fp=#0x10000004935' 2024-09-10T08:44:49.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7552 fp=#0x100000051b2' 2024-09-10T08:44:49.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7616 fp=#0x10000001c98' 2024-09-10T08:44:49.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7648 fp=#0x100000012dc' 2024-09-10T08:44:49.243 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7680 fp=#0x10000001338' 2024-09-10T08:44:49.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7712 fp=#0x10000004fb4' 2024-09-10T08:44:49.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7744 fp=#0x10000004e6f' 2024-09-10T08:44:49.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7776 fp=#0x10000004ed1' 2024-09-10T08:44:49.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7808 fp=#0x10000001453' 2024-09-10T08:44:49.245 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7840 fp=#0x10000001626' 2024-09-10T08:44:49.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7872 fp=#0x10000004cda' 2024-09-10T08:44:49.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7904 fp=#0x100000013ff' 2024-09-10T08:44:49.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7936 fp=#0x10000004d6c' 2024-09-10T08:44:49.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7968 fp=#0x100000013c3' 2024-09-10T08:44:49.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8000 fp=#0x100000013b8' 2024-09-10T08:44:49.247 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8032 fp=#0x10000003ffe' 2024-09-10T08:44:49.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8064 fp=#0x10000004007' 2024-09-10T08:44:49.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8096 fp=#0x1000000160c' 2024-09-10T08:44:49.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8128 fp=#0x10000003851' 2024-09-10T08:44:49.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8160 fp=#0x10000003a2a' 2024-09-10T08:44:49.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8192 fp=#0x1000000272c' 2024-09-10T08:44:49.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8224 fp=#0x1000000267a' 2024-09-10T08:44:49.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8256 fp=#0x10000004727' 2024-09-10T08:44:49.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8288 fp=#0x100000045af' 2024-09-10T08:44:49.250 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8320 fp=#0x1000000523d' 2024-09-10T08:44:49.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8352 fp=#0x1000000527a' 2024-09-10T08:44:49.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8384 fp=#0x10000004656' 2024-09-10T08:44:49.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8416 fp=#0x1000000458c' 2024-09-10T08:44:49.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8448 fp=#0x10000003e23' 2024-09-10T08:44:49.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8480 fp=#0x10000003e7b' 2024-09-10T08:44:49.252 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8512 fp=#0x10000003e59' 2024-09-10T08:44:49.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8544 fp=#0x10000003db5' 2024-09-10T08:44:49.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8576 fp=#0x10000003e63' 2024-09-10T08:44:49.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8608 fp=#0x100000041d6' 2024-09-10T08:44:49.254 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8640 fp=#0x10000003ccd' 2024-09-10T08:44:49.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8672 fp=#0x10000003de4' 2024-09-10T08:44:49.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8704 fp=#0x10000003ffc' 2024-09-10T08:44:49.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8736 fp=#0x10000003d20' 2024-09-10T08:44:49.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8768 fp=#0x10000003e44' 2024-09-10T08:44:49.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8800 fp=#0x10000003ce3' 2024-09-10T08:44:49.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8832 fp=#0x10000001d67' 2024-09-10T08:44:49.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8864 fp=#0x100000043ec' 2024-09-10T08:44:49.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8896 fp=#0x10000001bd9' 2024-09-10T08:44:49.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8928 fp=#0x10000004381' 2024-09-10T08:44:49.259 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8960 fp=#0x10000001c28' 2024-09-10T08:44:49.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8992 fp=#0x1000000445f' 2024-09-10T08:44:49.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9024 fp=#0x10000000cc3' 2024-09-10T08:44:49.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9056 fp=#0x1000000355c' 2024-09-10T08:44:49.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9120 fp=#0x100000020a1' 2024-09-10T08:44:49.261 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9152 fp=#0x10000004291' 2024-09-10T08:44:49.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9184 fp=#0x10000000fe8' 2024-09-10T08:44:49.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9216 fp=#0x100000026cc' 2024-09-10T08:44:49.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9248 fp=#0x10000000b4d' 2024-09-10T08:44:49.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9280 fp=#0x10000003e96' 2024-09-10T08:44:49.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9312 fp=#0x10000000c2f' 2024-09-10T08:44:49.263 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9344 fp=#0x10000001c4a' 2024-09-10T08:44:49.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9376 fp=#0x10000000a90' 2024-09-10T08:44:49.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9408 fp=#0x10000000b84' 2024-09-10T08:44:49.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9440 fp=#0x10000004c7c' 2024-09-10T08:44:49.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9472 fp=#0x10000000bb2' 2024-09-10T08:44:49.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9504 fp=#0x100000048da' 2024-09-10T08:44:49.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9536 fp=#0x10000001a90' 2024-09-10T08:44:49.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9568 fp=#0x10000004991' 2024-09-10T08:44:49.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9600 fp=#0x10000001ed2' 2024-09-10T08:44:49.266 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9632 fp=#0x10000000eba' 2024-09-10T08:44:49.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9664 fp=#0x10000004913' 2024-09-10T08:44:49.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9696 fp=#0x1000000476a' 2024-09-10T08:44:49.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9728 fp=#0x10000004c97' 2024-09-10T08:44:49.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9760 fp=#0x10000004310' 2024-09-10T08:44:49.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9792 fp=#0x10000004498' 2024-09-10T08:44:49.268 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9824 fp=#0x10000001785' 2024-09-10T08:44:49.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9856 fp=#0x100000032db' 2024-09-10T08:44:49.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9920 fp=#0x10000001673' 2024-09-10T08:44:49.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9952 fp=#0x10000003343' 2024-09-10T08:44:49.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9984 fp=#0x1000000176f' 2024-09-10T08:44:49.270 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10016 fp=#0x10000001928' 2024-09-10T08:44:49.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10048 fp=#0x10000001689' 2024-09-10T08:44:49.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10080 fp=#0x100000018b9' 2024-09-10T08:44:49.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10112 fp=#0x100000039a8' 2024-09-10T08:44:49.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10144 fp=#0x100000038c9' 2024-09-10T08:44:49.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10176 fp=#0x10000003906' 2024-09-10T08:44:49.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10208 fp=#0x10000001f14' 2024-09-10T08:44:49.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10240 fp=#0x10000003f59' 2024-09-10T08:44:49.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10272 fp=#0x10000002381' 2024-09-10T08:44:49.273 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10304 fp=#0x100000023ef' 2024-09-10T08:44:49.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10336 fp=#0x1000000133d' 2024-09-10T08:44:49.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10368 fp=#0x10000001302' 2024-09-10T08:44:49.274 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10400 fp=#0x100000012df' 2024-09-10T08:44:49.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10432 fp=#0x1000000140c' 2024-09-10T08:44:49.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10464 fp=#0x100000039ae' 2024-09-10T08:44:49.275 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10496 fp=#0x10000003981' 2024-09-10T08:44:49.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10528 fp=#0x10000003a02' 2024-09-10T08:44:49.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10560 fp=#0x10000003ad6' 2024-09-10T08:44:49.276 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10592 fp=#0x100000049fb' 2024-09-10T08:44:49.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10624 fp=#0x1000000225b' 2024-09-10T08:44:49.277 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10656 fp=#0x100000049cb' 2024-09-10T08:44:49.278 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10816 fp=#0x1000000058a' 2024-09-10T08:44:49.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10848 fp=#0x10000000a2d' 2024-09-10T08:44:49.279 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10880 fp=#0x100000017cd' 2024-09-10T08:44:49.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10976 fp=#0x100000009ee' 2024-09-10T08:44:49.280 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11008 fp=#0x100000007f5' 2024-09-10T08:44:49.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11040 fp=#0x10000003b33' 2024-09-10T08:44:49.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11072 fp=#0x10000003989' 2024-09-10T08:44:49.281 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11104 fp=#0x1000000389c' 2024-09-10T08:44:49.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11136 fp=#0x10000003f9b' 2024-09-10T08:44:49.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11168 fp=#0x1000000205b' 2024-09-10T08:44:49.282 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11200 fp=#0x10000004905' 2024-09-10T08:44:49.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11232 fp=#0x100000020b1' 2024-09-10T08:44:49.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11264 fp=#0x100000022dd' 2024-09-10T08:44:49.283 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11296 fp=#0x10000003e81' 2024-09-10T08:44:49.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11328 fp=#0x10000001d72' 2024-09-10T08:44:49.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11360 fp=#0x10000003de2' 2024-09-10T08:44:49.284 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11392 fp=#0x10000003d46' 2024-09-10T08:44:49.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11424 fp=#0x10000001d37' 2024-09-10T08:44:49.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11456 fp=#0x10000003dba' 2024-09-10T08:44:49.285 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11488 fp=#0x10000003d7f' 2024-09-10T08:44:49.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11520 fp=#0x10000003d77' 2024-09-10T08:44:49.286 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11616 fp=#0x100000051af' 2024-09-10T08:44:49.287 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11712 fp=#0x1000000094f' 2024-09-10T08:44:49.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11744 fp=#0x100000009f6' 2024-09-10T08:44:49.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11776 fp=#0x1000000091c' 2024-09-10T08:44:49.288 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11808 fp=#0x1000000097e' 2024-09-10T08:44:49.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11840 fp=#0x10000000a16' 2024-09-10T08:44:49.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11872 fp=#0x100000008f5' 2024-09-10T08:44:49.289 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11904 fp=#0x10000000f28' 2024-09-10T08:44:49.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11936 fp=#0x1000000184f' 2024-09-10T08:44:49.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11968 fp=#0x10000005434' 2024-09-10T08:44:49.290 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12000 fp=#0x1000000110d' 2024-09-10T08:44:49.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12064 fp=#0x10000003466' 2024-09-10T08:44:49.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12096 fp=#0x10000003471' 2024-09-10T08:44:49.291 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12128 fp=#0x1000000332c' 2024-09-10T08:44:49.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12160 fp=#0x100000033a2' 2024-09-10T08:44:49.292 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12192 fp=#0x100000044e3' 2024-09-10T08:44:49.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12256 fp=#0x10000004da5' 2024-09-10T08:44:49.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12288 fp=#0x10000003a8e' 2024-09-10T08:44:49.293 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12320 fp=#0x10000003a50' 2024-09-10T08:44:49.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12352 fp=#0x10000000ccb' 2024-09-10T08:44:49.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12384 fp=#0x10000000d1e' 2024-09-10T08:44:49.294 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12416 fp=#0x10000000d55' 2024-09-10T08:44:49.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12448 fp=#0x10000000d3d' 2024-09-10T08:44:49.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12480 fp=#0x10000004700' 2024-09-10T08:44:49.295 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12512 fp=#0x10000004bd1' 2024-09-10T08:44:49.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12544 fp=#0x10000004bef' 2024-09-10T08:44:49.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12576 fp=#0x10000004b8d' 2024-09-10T08:44:49.296 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12608 fp=#0x10000004c84' 2024-09-10T08:44:49.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12640 fp=#0x10000004baf' 2024-09-10T08:44:49.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12672 fp=#0x10000004986' 2024-09-10T08:44:49.297 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12704 fp=#0x100000053cc' 2024-09-10T08:44:49.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12736 fp=#0x100000049a9' 2024-09-10T08:44:49.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12768 fp=#0x1000000540b' 2024-09-10T08:44:49.298 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12800 fp=#0x1000000231f' 2024-09-10T08:44:49.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12864 fp=#0x10000002330' 2024-09-10T08:44:49.299 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12896 fp=#0x1000000217f' 2024-09-10T08:44:49.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12928 fp=#0x100000040f7' 2024-09-10T08:44:49.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12960 fp=#0x1000000270a' 2024-09-10T08:44:49.300 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12992 fp=#0x10000002718' 2024-09-10T08:44:49.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13024 fp=#0x10000002738' 2024-09-10T08:44:49.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13048 fp=#0x100000053a5' 2024-09-10T08:44:49.301 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13080 fp=#0x100000039f6' 2024-09-10T08:44:49.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13112 fp=#0x100000053ac' 2024-09-10T08:44:49.302 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13144 fp=#0x10000003905' 2024-09-10T08:44:49.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:265 fp=#0x10000000dc4' 2024-09-10T08:44:49.303 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:297 fp=#0x10000000555' 2024-09-10T08:44:49.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:329 fp=#0x10000000bd5' 2024-09-10T08:44:49.304 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:385 fp=#0x10000004b20' 2024-09-10T08:44:49.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:481 fp=#0x10000004b65' 2024-09-10T08:44:49.305 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:513 fp=#0x10000003420' 2024-09-10T08:44:49.306 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:545 fp=#0x10000003350' 2024-09-10T08:44:49.308 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:801 fp=#0x10000004575' 2024-09-10T08:44:49.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:865 fp=#0x100000016e5' 2024-09-10T08:44:49.309 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:897 fp=#0x100000044c3' 2024-09-10T08:44:49.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:929 fp=#0x10000000e89' 2024-09-10T08:44:49.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:961 fp=#0x10000000e62' 2024-09-10T08:44:49.310 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:993 fp=#0x10000000e91' 2024-09-10T08:44:49.311 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1025 fp=#0x10000004496' 2024-09-10T08:44:49.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1185 fp=#0x10000004176' 2024-09-10T08:44:49.312 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1217 fp=#0x10000002078' 2024-09-10T08:44:49.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1249 fp=#0x10000002067' 2024-09-10T08:44:49.313 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1313 fp=#0x100000035f8' 2024-09-10T08:44:49.314 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1377 fp=#0x10000003304' 2024-09-10T08:44:49.315 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1441 fp=#0x100000032ed' 2024-09-10T08:44:49.316 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1665 fp=#0x10000004587' 2024-09-10T08:44:49.317 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1697 fp=#0x10000003389' 2024-09-10T08:44:49.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2081 fp=#0x10000002142' 2024-09-10T08:44:49.320 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2113 fp=#0x100000021e6' 2024-09-10T08:44:49.321 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2241 fp=#0x10000002755' 2024-09-10T08:44:49.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2305 fp=#0x10000002792' 2024-09-10T08:44:49.322 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2337 fp=#0x100000027bd' 2024-09-10T08:44:49.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2561 fp=#0x1000000104e' 2024-09-10T08:44:49.324 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2593 fp=#0x10000001040' 2024-09-10T08:44:49.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2625 fp=#0x10000001859' 2024-09-10T08:44:49.325 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2657 fp=#0x1000000189a' 2024-09-10T08:44:49.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2689 fp=#0x100000018ad' 2024-09-10T08:44:49.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2721 fp=#0x10000001127' 2024-09-10T08:44:49.326 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2785 fp=#0x1000000367b' 2024-09-10T08:44:49.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2817 fp=#0x100000047bd' 2024-09-10T08:44:49.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2849 fp=#0x1000000486b' 2024-09-10T08:44:49.327 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2881 fp=#0x10000004624' 2024-09-10T08:44:49.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2913 fp=#0x1000000464d' 2024-09-10T08:44:49.328 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2945 fp=#0x10000004751' 2024-09-10T08:44:49.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3009 fp=#0x10000000e8a' 2024-09-10T08:44:49.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3041 fp=#0x10000000e6c' 2024-09-10T08:44:49.329 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3073 fp=#0x1000000499a' 2024-09-10T08:44:49.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3137 fp=#0x10000000df0' 2024-09-10T08:44:49.330 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3169 fp=#0x100000018d9' 2024-09-10T08:44:49.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3201 fp=#0x100000018d0' 2024-09-10T08:44:49.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3233 fp=#0x10000000775' 2024-09-10T08:44:49.331 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3265 fp=#0x10000000757' 2024-09-10T08:44:49.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3297 fp=#0x1000000070d' 2024-09-10T08:44:49.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3329 fp=#0x10000003c24' 2024-09-10T08:44:49.332 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3361 fp=#0x10000000762' 2024-09-10T08:44:49.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3393 fp=#0x10000000728' 2024-09-10T08:44:49.333 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3425 fp=#0x10000003d8a' 2024-09-10T08:44:49.334 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3489 fp=#0x100000040cb' 2024-09-10T08:44:49.334 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3521 fp=#0x10000003c21' 2024-09-10T08:44:49.334 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3553 fp=#0x10000004065' 2024-09-10T08:44:49.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3585 fp=#0x10000003c35' 2024-09-10T08:44:49.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3649 fp=#0x10000004bfb' 2024-09-10T08:44:49.335 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3681 fp=#0x10000004b41' 2024-09-10T08:44:49.336 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3713 fp=#0x10000004afa' 2024-09-10T08:44:49.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3809 fp=#0x1000000251b' 2024-09-10T08:44:49.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3841 fp=#0x10000003e94' 2024-09-10T08:44:49.337 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3873 fp=#0x10000003f64' 2024-09-10T08:44:49.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3905 fp=#0x10000003e7d' 2024-09-10T08:44:49.338 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3937 fp=#0x10000001de2' 2024-09-10T08:44:49.339 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4001 fp=#0x10000000522' 2024-09-10T08:44:49.339 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4033 fp=#0x1000000468c' 2024-09-10T08:44:49.339 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4065 fp=#0x100000046f7' 2024-09-10T08:44:49.340 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4097 fp=#0x10000000f2e' 2024-09-10T08:44:49.340 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4129 fp=#0x10000001199' 2024-09-10T08:44:49.340 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4161 fp=#0x1000000461d' 2024-09-10T08:44:49.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4193 fp=#0x10000004e4d' 2024-09-10T08:44:49.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4225 fp=#0x10000003a03' 2024-09-10T08:44:49.341 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4257 fp=#0x10000003997' 2024-09-10T08:44:49.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4289 fp=#0x1000000419d' 2024-09-10T08:44:49.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4321 fp=#0x100000039ec' 2024-09-10T08:44:49.342 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4353 fp=#0x100000041bf' 2024-09-10T08:44:49.343 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4385 fp=#0x10000001f9a' 2024-09-10T08:44:49.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4481 fp=#0x10000005143' 2024-09-10T08:44:49.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4513 fp=#0x10000005146' 2024-09-10T08:44:49.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4545 fp=#0x100000012bb' 2024-09-10T08:44:49.344 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4577 fp=#0x100000012c7' 2024-09-10T08:44:49.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4609 fp=#0x1000000144c' 2024-09-10T08:44:49.345 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4641 fp=#0x10000003924' 2024-09-10T08:44:49.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4673 fp=#0x100000012f7' 2024-09-10T08:44:49.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4705 fp=#0x1000000128e' 2024-09-10T08:44:49.346 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4737 fp=#0x10000001473' 2024-09-10T08:44:49.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4769 fp=#0x100000012dd' 2024-09-10T08:44:49.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4801 fp=#0x100000012c0' 2024-09-10T08:44:49.347 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4833 fp=#0x10000003816' 2024-09-10T08:44:49.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4865 fp=#0x1000000377d' 2024-09-10T08:44:49.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4897 fp=#0x1000000375e' 2024-09-10T08:44:49.348 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4929 fp=#0x10000004a72' 2024-09-10T08:44:49.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4961 fp=#0x1000000513e' 2024-09-10T08:44:49.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4993 fp=#0x100000051cd' 2024-09-10T08:44:49.349 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5025 fp=#0x10000005176' 2024-09-10T08:44:49.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5057 fp=#0x10000005185' 2024-09-10T08:44:49.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5089 fp=#0x100000050f2' 2024-09-10T08:44:49.350 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5121 fp=#0x10000001e79' 2024-09-10T08:44:49.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5153 fp=#0x10000001d6b' 2024-09-10T08:44:49.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5185 fp=#0x10000001e9e' 2024-09-10T08:44:49.351 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5217 fp=#0x10000001c9f' 2024-09-10T08:44:49.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5249 fp=#0x10000001d4b' 2024-09-10T08:44:49.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5281 fp=#0x10000003700' 2024-09-10T08:44:49.352 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5313 fp=#0x100000038d9' 2024-09-10T08:44:49.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5345 fp=#0x10000004aa6' 2024-09-10T08:44:49.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5377 fp=#0x100000036f8' 2024-09-10T08:44:49.353 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5409 fp=#0x10000003d02' 2024-09-10T08:44:49.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5441 fp=#0x1000000129c' 2024-09-10T08:44:49.354 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5473 fp=#0x10000001288' 2024-09-10T08:44:49.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5601 fp=#0x10000000fe3' 2024-09-10T08:44:49.355 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5633 fp=#0x100000020c4' 2024-09-10T08:44:49.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5665 fp=#0x10000002128' 2024-09-10T08:44:49.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5697 fp=#0x10000002095' 2024-09-10T08:44:49.356 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5729 fp=#0x100000020f9' 2024-09-10T08:44:49.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5761 fp=#0x10000000f8c' 2024-09-10T08:44:49.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5793 fp=#0x10000004442' 2024-09-10T08:44:49.357 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5825 fp=#0x10000004300' 2024-09-10T08:44:49.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5857 fp=#0x1000000449c' 2024-09-10T08:44:49.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5889 fp=#0x1000000451a' 2024-09-10T08:44:49.358 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5921 fp=#0x1000000495a' 2024-09-10T08:44:49.359 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5953 fp=#0x10000004636' 2024-09-10T08:44:49.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6081 fp=#0x100000035fe' 2024-09-10T08:44:49.360 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6113 fp=#0x10000003547' 2024-09-10T08:44:49.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6177 fp=#0x10000003587' 2024-09-10T08:44:49.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6209 fp=#0x10000003532' 2024-09-10T08:44:49.361 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6241 fp=#0x100000035d4' 2024-09-10T08:44:49.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6273 fp=#0x10000001e96' 2024-09-10T08:44:49.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6305 fp=#0x10000001f38' 2024-09-10T08:44:49.362 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6337 fp=#0x100000033fe' 2024-09-10T08:44:49.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6401 fp=#0x10000004fc5' 2024-09-10T08:44:49.363 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6433 fp=#0x10000004da8' 2024-09-10T08:44:49.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6465 fp=#0x10000004e9f' 2024-09-10T08:44:49.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6497 fp=#0x10000004bcf' 2024-09-10T08:44:49.364 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6529 fp=#0x10000004b4c' 2024-09-10T08:44:49.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6625 fp=#0x10000003cff' 2024-09-10T08:44:49.365 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6657 fp=#0x10000004f7f' 2024-09-10T08:44:49.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6689 fp=#0x1000000502c' 2024-09-10T08:44:49.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6721 fp=#0x10000004f9c' 2024-09-10T08:44:49.366 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6753 fp=#0x10000004f96' 2024-09-10T08:44:49.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6785 fp=#0x100000045f1' 2024-09-10T08:44:49.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6817 fp=#0x100000046c5' 2024-09-10T08:44:49.367 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6849 fp=#0x1000000462a' 2024-09-10T08:44:49.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6881 fp=#0x100000052d3' 2024-09-10T08:44:49.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6913 fp=#0x100000045a9' 2024-09-10T08:44:49.368 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6945 fp=#0x10000004846' 2024-09-10T08:44:49.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6977 fp=#0x10000000663' 2024-09-10T08:44:49.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7009 fp=#0x1000000068e' 2024-09-10T08:44:49.369 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7041 fp=#0x1000000066d' 2024-09-10T08:44:49.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7073 fp=#0x10000000662' 2024-09-10T08:44:49.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7105 fp=#0x10000000605' 2024-09-10T08:44:49.370 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7137 fp=#0x1000000057c' 2024-09-10T08:44:49.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7169 fp=#0x10000000655' 2024-09-10T08:44:49.371 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7233 fp=#0x100000047cd' 2024-09-10T08:44:49.372 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7361 fp=#0x10000005037' 2024-09-10T08:44:49.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7393 fp=#0x1000000478b' 2024-09-10T08:44:49.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7425 fp=#0x100000023df' 2024-09-10T08:44:49.373 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7457 fp=#0x100000047ad' 2024-09-10T08:44:49.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7489 fp=#0x100000046df' 2024-09-10T08:44:49.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7521 fp=#0x100000048c9' 2024-09-10T08:44:49.374 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7553 fp=#0x10000001cd2' 2024-09-10T08:44:49.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7585 fp=#0x100000035fb' 2024-09-10T08:44:49.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7617 fp=#0x100000051c1' 2024-09-10T08:44:49.375 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7649 fp=#0x100000012da' 2024-09-10T08:44:49.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7713 fp=#0x10000004efe' 2024-09-10T08:44:49.376 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7745 fp=#0x10000004e3f' 2024-09-10T08:44:49.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7777 fp=#0x10000004f5f' 2024-09-10T08:44:49.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7809 fp=#0x1000000143f' 2024-09-10T08:44:49.377 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7841 fp=#0x100000016e3' 2024-09-10T08:44:49.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7873 fp=#0x10000004e67' 2024-09-10T08:44:49.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7905 fp=#0x10000001416' 2024-09-10T08:44:49.378 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7937 fp=#0x1000000142b' 2024-09-10T08:44:49.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7969 fp=#0x100000016c6' 2024-09-10T08:44:49.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8001 fp=#0x10000001465' 2024-09-10T08:44:49.379 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8033 fp=#0x10000001604' 2024-09-10T08:44:49.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8065 fp=#0x1000000160a' 2024-09-10T08:44:49.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8097 fp=#0x10000003fe3' 2024-09-10T08:44:49.380 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8129 fp=#0x10000004620' 2024-09-10T08:44:49.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8161 fp=#0x100000025cf' 2024-09-10T08:44:49.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8193 fp=#0x10000004967' 2024-09-10T08:44:49.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8225 fp=#0x10000003de6' 2024-09-10T08:44:49.381 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8257 fp=#0x100000045a0' 2024-09-10T08:44:49.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8289 fp=#0x1000000464b' 2024-09-10T08:44:49.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8321 fp=#0x10000004440' 2024-09-10T08:44:49.382 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8353 fp=#0x1000000443b' 2024-09-10T08:44:49.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8385 fp=#0x100000044ae' 2024-09-10T08:44:49.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8417 fp=#0x1000000416c' 2024-09-10T08:44:49.383 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8449 fp=#0x1000000452f' 2024-09-10T08:44:49.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8481 fp=#0x10000004556' 2024-09-10T08:44:49.384 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8513 fp=#0x10000003f67' 2024-09-10T08:44:49.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8545 fp=#0x10000004130' 2024-09-10T08:44:49.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8577 fp=#0x10000003f7d' 2024-09-10T08:44:49.385 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8609 fp=#0x1000000416b' 2024-09-10T08:44:49.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8641 fp=#0x10000003c1f' 2024-09-10T08:44:49.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8673 fp=#0x10000003eab' 2024-09-10T08:44:49.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8705 fp=#0x100000052de' 2024-09-10T08:44:49.386 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8737 fp=#0x10000004fd7' 2024-09-10T08:44:49.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8769 fp=#0x10000004864' 2024-09-10T08:44:49.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8801 fp=#0x10000003ef6' 2024-09-10T08:44:49.387 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8833 fp=#0x10000004370' 2024-09-10T08:44:49.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8865 fp=#0x10000004423' 2024-09-10T08:44:49.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8897 fp=#0x10000001bf1' 2024-09-10T08:44:49.388 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8929 fp=#0x1000000448a' 2024-09-10T08:44:49.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8961 fp=#0x10000001bc0' 2024-09-10T08:44:49.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8993 fp=#0x10000004438' 2024-09-10T08:44:49.389 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9025 fp=#0x10000000dbc' 2024-09-10T08:44:49.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9057 fp=#0x1000000419b' 2024-09-10T08:44:49.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9089 fp=#0x100000040d0' 2024-09-10T08:44:49.390 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9121 fp=#0x10000000d21' 2024-09-10T08:44:49.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9153 fp=#0x100000016fb' 2024-09-10T08:44:49.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9185 fp=#0x100000041dc' 2024-09-10T08:44:49.391 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9217 fp=#0x10000000cee' 2024-09-10T08:44:49.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9249 fp=#0x10000004e46' 2024-09-10T08:44:49.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9281 fp=#0x10000004c65' 2024-09-10T08:44:49.392 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9313 fp=#0x10000000bc3' 2024-09-10T08:44:49.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9345 fp=#0x10000000ae8' 2024-09-10T08:44:49.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9377 fp=#0x10000000b3b' 2024-09-10T08:44:49.393 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9409 fp=#0x10000004c93' 2024-09-10T08:44:49.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9473 fp=#0x10000000acc' 2024-09-10T08:44:49.394 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9505 fp=#0x10000004927' 2024-09-10T08:44:49.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9537 fp=#0x100000049b7' 2024-09-10T08:44:49.395 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9569 fp=#0x10000000ee9' 2024-09-10T08:44:49.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9633 fp=#0x10000000ec0' 2024-09-10T08:44:49.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9665 fp=#0x10000004821' 2024-09-10T08:44:49.396 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9697 fp=#0x100000048ce' 2024-09-10T08:44:49.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9729 fp=#0x100000044bc' 2024-09-10T08:44:49.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9761 fp=#0x10000001737' 2024-09-10T08:44:49.397 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9793 fp=#0x1000000188c' 2024-09-10T08:44:49.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9825 fp=#0x100000032d5' 2024-09-10T08:44:49.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9857 fp=#0x1000000176d' 2024-09-10T08:44:49.398 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9889 fp=#0x10000001722' 2024-09-10T08:44:49.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9921 fp=#0x10000001a00' 2024-09-10T08:44:49.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9953 fp=#0x100000015cd' 2024-09-10T08:44:49.399 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9985 fp=#0x100000026c4' 2024-09-10T08:44:49.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10049 fp=#0x10000001946' 2024-09-10T08:44:49.400 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10081 fp=#0x10000001f0e' 2024-09-10T08:44:49.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10113 fp=#0x100000039cf' 2024-09-10T08:44:49.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10145 fp=#0x10000003a54' 2024-09-10T08:44:49.401 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10177 fp=#0x10000003b51' 2024-09-10T08:44:49.402 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10209 fp=#0x100000039f3' 2024-09-10T08:44:49.402 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10241 fp=#0x10000003b35' 2024-09-10T08:44:49.402 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10273 fp=#0x10000002500' 2024-09-10T08:44:49.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10305 fp=#0x10000002400' 2024-09-10T08:44:49.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10337 fp=#0x100000014aa' 2024-09-10T08:44:49.403 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10369 fp=#0x10000003a1a' 2024-09-10T08:44:49.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10433 fp=#0x10000003a98' 2024-09-10T08:44:49.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10465 fp=#0x10000003b0c' 2024-09-10T08:44:49.404 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10497 fp=#0x100000013c4' 2024-09-10T08:44:49.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10529 fp=#0x10000003a32' 2024-09-10T08:44:49.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10561 fp=#0x100000039c9' 2024-09-10T08:44:49.405 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10593 fp=#0x100000049c4' 2024-09-10T08:44:49.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10657 fp=#0x100000049b5' 2024-09-10T08:44:49.406 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10689 fp=#0x100000048c1' 2024-09-10T08:44:49.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10721 fp=#0x100000007bf' 2024-09-10T08:44:49.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10753 fp=#0x10000004931' 2024-09-10T08:44:49.407 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10785 fp=#0x100000048d8' 2024-09-10T08:44:49.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10817 fp=#0x1000000262f' 2024-09-10T08:44:49.408 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10849 fp=#0x100000009b8' 2024-09-10T08:44:49.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10913 fp=#0x100000008c5' 2024-09-10T08:44:49.409 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10945 fp=#0x10000000582' 2024-09-10T08:44:49.410 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11041 fp=#0x10000003968' 2024-09-10T08:44:49.410 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11073 fp=#0x100000038a7' 2024-09-10T08:44:49.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11105 fp=#0x10000003951' 2024-09-10T08:44:49.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11137 fp=#0x10000004a08' 2024-09-10T08:44:49.411 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11169 fp=#0x10000003e0e' 2024-09-10T08:44:49.412 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11201 fp=#0x10000003fb9' 2024-09-10T08:44:49.412 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11233 fp=#0x100000020ad' 2024-09-10T08:44:49.412 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11265 fp=#0x10000003f3a' 2024-09-10T08:44:49.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11297 fp=#0x10000003e51' 2024-09-10T08:44:49.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11329 fp=#0x10000003f4a' 2024-09-10T08:44:49.413 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11361 fp=#0x10000001c54' 2024-09-10T08:44:49.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11393 fp=#0x10000001d5e' 2024-09-10T08:44:49.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11425 fp=#0x10000001b2e' 2024-09-10T08:44:49.414 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11457 fp=#0x100000019b1' 2024-09-10T08:44:49.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11489 fp=#0x10000001ca8' 2024-09-10T08:44:49.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11521 fp=#0x10000003ea3' 2024-09-10T08:44:49.415 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11553 fp=#0x10000005190' 2024-09-10T08:44:49.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11585 fp=#0x100000051b6' 2024-09-10T08:44:49.416 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11649 fp=#0x10000005165' 2024-09-10T08:44:49.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11713 fp=#0x10000000b8b' 2024-09-10T08:44:49.417 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11745 fp=#0x10000004dd3' 2024-09-10T08:44:49.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11777 fp=#0x10000004ddc' 2024-09-10T08:44:49.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11809 fp=#0x100000042a0' 2024-09-10T08:44:49.418 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11841 fp=#0x10000004251' 2024-09-10T08:44:49.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11873 fp=#0x10000000879' 2024-09-10T08:44:49.419 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11905 fp=#0x10000000f08' 2024-09-10T08:44:49.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11969 fp=#0x100000010f6' 2024-09-10T08:44:49.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12001 fp=#0x100000052d2' 2024-09-10T08:44:49.420 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12065 fp=#0x1000000338a' 2024-09-10T08:44:49.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12097 fp=#0x100000033f1' 2024-09-10T08:44:49.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12129 fp=#0x10000003500' 2024-09-10T08:44:49.421 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12161 fp=#0x1000000442f' 2024-09-10T08:44:49.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12193 fp=#0x10000004424' 2024-09-10T08:44:49.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12225 fp=#0x10000000c33' 2024-09-10T08:44:49.422 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12257 fp=#0x10000000c37' 2024-09-10T08:44:49.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12289 fp=#0x10000000c2c' 2024-09-10T08:44:49.423 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12321 fp=#0x10000000cf7' 2024-09-10T08:44:49.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12353 fp=#0x10000001119' 2024-09-10T08:44:49.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12385 fp=#0x10000000bff' 2024-09-10T08:44:49.424 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12417 fp=#0x10000003a21' 2024-09-10T08:44:49.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12449 fp=#0x10000000d1a' 2024-09-10T08:44:49.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12513 fp=#0x10000004bf9' 2024-09-10T08:44:49.425 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12545 fp=#0x10000004c96' 2024-09-10T08:44:49.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12577 fp=#0x10000004d53' 2024-09-10T08:44:49.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12609 fp=#0x10000004bce' 2024-09-10T08:44:49.426 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12641 fp=#0x10000004bc1' 2024-09-10T08:44:49.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12673 fp=#0x100000047a3' 2024-09-10T08:44:49.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12705 fp=#0x100000047e1' 2024-09-10T08:44:49.427 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12737 fp=#0x100000048a8' 2024-09-10T08:44:49.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12769 fp=#0x100000053e8' 2024-09-10T08:44:49.428 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12833 fp=#0x100000052fb' 2024-09-10T08:44:49.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12865 fp=#0x10000005346' 2024-09-10T08:44:49.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12897 fp=#0x100000021bd' 2024-09-10T08:44:49.429 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12929 fp=#0x10000002279' 2024-09-10T08:44:49.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12961 fp=#0x100000053f2' 2024-09-10T08:44:49.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12993 fp=#0x10000005421' 2024-09-10T08:44:49.430 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13049 fp=#0x100000053b8' 2024-09-10T08:44:49.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13113 fp=#0x1000000228e' 2024-09-10T08:44:49.431 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13145 fp=#0x100000022a3' 2024-09-10T08:44:49.432 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:266 fp=#0x100000006ee' 2024-09-10T08:44:49.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:298 fp=#0x100000027fd' 2024-09-10T08:44:49.433 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:330 fp=#0x1000000101b' 2024-09-10T08:44:49.434 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:450 fp=#0x10000004b3d' 2024-09-10T08:44:49.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:482 fp=#0x100000032c0' 2024-09-10T08:44:49.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:514 fp=#0x100000034fd' 2024-09-10T08:44:49.435 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:578 fp=#0x10000003288' 2024-09-10T08:44:49.437 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:802 fp=#0x10000000e4a' 2024-09-10T08:44:49.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:834 fp=#0x10000000e8b' 2024-09-10T08:44:49.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:866 fp=#0x10000000df5' 2024-09-10T08:44:49.438 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:898 fp=#0x10000004585' 2024-09-10T08:44:49.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:930 fp=#0x10000004450' 2024-09-10T08:44:49.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:962 fp=#0x10000000e7b' 2024-09-10T08:44:49.439 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:994 fp=#0x10000000e83' 2024-09-10T08:44:49.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1026 fp=#0x1000000400d' 2024-09-10T08:44:49.440 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1058 fp=#0x10000004293' 2024-09-10T08:44:49.441 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1218 fp=#0x10000003613' 2024-09-10T08:44:49.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1250 fp=#0x10000001fd5' 2024-09-10T08:44:49.442 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1282 fp=#0x100000035b6' 2024-09-10T08:44:49.444 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1506 fp=#0x1000000326a' 2024-09-10T08:44:49.445 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:1666 fp=#0x100000045ad' 2024-09-10T08:44:49.448 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2050 fp=#0x100000021ce' 2024-09-10T08:44:49.449 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2082 fp=#0x100000021bf' 2024-09-10T08:44:49.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2178 fp=#0x1000000200f' 2024-09-10T08:44:49.450 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2242 fp=#0x1000000340d' 2024-09-10T08:44:49.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2306 fp=#0x1000000265b' 2024-09-10T08:44:49.451 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2338 fp=#0x100000025e3' 2024-09-10T08:44:49.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2562 fp=#0x1000000450c' 2024-09-10T08:44:49.453 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2594 fp=#0x1000000187b' 2024-09-10T08:44:49.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2690 fp=#0x1000000329f' 2024-09-10T08:44:49.454 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2722 fp=#0x10000001080' 2024-09-10T08:44:49.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2754 fp=#0x10000003ba3' 2024-09-10T08:44:49.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2786 fp=#0x10000004819' 2024-09-10T08:44:49.455 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2818 fp=#0x1000000362f' 2024-09-10T08:44:49.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2850 fp=#0x1000000472c' 2024-09-10T08:44:49.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2882 fp=#0x10000003675' 2024-09-10T08:44:49.456 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2914 fp=#0x10000003412' 2024-09-10T08:44:49.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2946 fp=#0x1000000461e' 2024-09-10T08:44:49.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:2978 fp=#0x1000000470c' 2024-09-10T08:44:49.457 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3010 fp=#0x10000004a4f' 2024-09-10T08:44:49.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3042 fp=#0x10000000e14' 2024-09-10T08:44:49.458 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3106 fp=#0x100000018e3' 2024-09-10T08:44:49.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3138 fp=#0x10000000e7e' 2024-09-10T08:44:49.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3170 fp=#0x10000000e5d' 2024-09-10T08:44:49.459 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3202 fp=#0x100000049a8' 2024-09-10T08:44:49.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3234 fp=#0x100000007d9' 2024-09-10T08:44:49.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3266 fp=#0x10000000788' 2024-09-10T08:44:49.460 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3298 fp=#0x10000003cf6' 2024-09-10T08:44:49.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3330 fp=#0x10000003d72' 2024-09-10T08:44:49.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3362 fp=#0x1000000071c' 2024-09-10T08:44:49.461 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3394 fp=#0x10000000794' 2024-09-10T08:44:49.462 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3426 fp=#0x10000003d13' 2024-09-10T08:44:49.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3554 fp=#0x10000003c59' 2024-09-10T08:44:49.463 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3586 fp=#0x10000003c8a' 2024-09-10T08:44:49.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3618 fp=#0x10000004128' 2024-09-10T08:44:49.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3650 fp=#0x10000004b6a' 2024-09-10T08:44:49.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3682 fp=#0x10000004bd6' 2024-09-10T08:44:49.464 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3714 fp=#0x10000004bf6' 2024-09-10T08:44:49.465 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3810 fp=#0x1000000531e' 2024-09-10T08:44:49.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3842 fp=#0x100000026f0' 2024-09-10T08:44:49.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3874 fp=#0x10000003f3e' 2024-09-10T08:44:49.466 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3906 fp=#0x1000000253d' 2024-09-10T08:44:49.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3938 fp=#0x100000010f5' 2024-09-10T08:44:49.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:3970 fp=#0x1000000059b' 2024-09-10T08:44:49.467 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4002 fp=#0x1000000057b' 2024-09-10T08:44:49.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4034 fp=#0x10000002684' 2024-09-10T08:44:49.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4066 fp=#0x10000004627' 2024-09-10T08:44:49.468 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4098 fp=#0x10000000f3a' 2024-09-10T08:44:49.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4130 fp=#0x10000004e6e' 2024-09-10T08:44:49.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4162 fp=#0x10000001d3c' 2024-09-10T08:44:49.469 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4194 fp=#0x10000004dfd' 2024-09-10T08:44:49.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4226 fp=#0x100000040da' 2024-09-10T08:44:49.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4258 fp=#0x10000003b69' 2024-09-10T08:44:49.470 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4290 fp=#0x1000000415a' 2024-09-10T08:44:49.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4322 fp=#0x10000003ab9' 2024-09-10T08:44:49.471 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4354 fp=#0x10000004135' 2024-09-10T08:44:49.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4418 fp=#0x10000004d10' 2024-09-10T08:44:49.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4450 fp=#0x10000001ef2' 2024-09-10T08:44:49.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4482 fp=#0x100000050e5' 2024-09-10T08:44:49.472 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4514 fp=#0x1000000123d' 2024-09-10T08:44:49.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4546 fp=#0x10000003b4d' 2024-09-10T08:44:49.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4578 fp=#0x10000001e7e' 2024-09-10T08:44:49.473 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4610 fp=#0x10000001290' 2024-09-10T08:44:49.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4642 fp=#0x100000012cb' 2024-09-10T08:44:49.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4674 fp=#0x100000013ad' 2024-09-10T08:44:49.474 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4706 fp=#0x10000001296' 2024-09-10T08:44:49.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4738 fp=#0x100000014a1' 2024-09-10T08:44:49.475 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4770 fp=#0x100000014d4' 2024-09-10T08:44:49.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4802 fp=#0x1000000149d' 2024-09-10T08:44:49.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4834 fp=#0x10000003707' 2024-09-10T08:44:49.476 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4866 fp=#0x10000003764' 2024-09-10T08:44:49.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4898 fp=#0x10000003713' 2024-09-10T08:44:49.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4930 fp=#0x100000037f0' 2024-09-10T08:44:49.477 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4962 fp=#0x1000000501a' 2024-09-10T08:44:49.480 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:4994 fp=#0x1000000518e' 2024-09-10T08:44:49.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5026 fp=#0x10000004ff9' 2024-09-10T08:44:49.481 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5058 fp=#0x10000005083' 2024-09-10T08:44:49.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5090 fp=#0x100000051c8' 2024-09-10T08:44:49.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5122 fp=#0x10000003e11' 2024-09-10T08:44:49.482 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5154 fp=#0x10000001cd6' 2024-09-10T08:44:49.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5186 fp=#0x10000001cca' 2024-09-10T08:44:49.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5218 fp=#0x10000001c1e' 2024-09-10T08:44:49.483 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5250 fp=#0x10000003615' 2024-09-10T08:44:49.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5282 fp=#0x10000003661' 2024-09-10T08:44:49.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5314 fp=#0x10000003869' 2024-09-10T08:44:49.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5346 fp=#0x10000003637' 2024-09-10T08:44:49.484 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5378 fp=#0x10000003e66' 2024-09-10T08:44:49.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5410 fp=#0x10000003668' 2024-09-10T08:44:49.485 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5474 fp=#0x10000004a9b' 2024-09-10T08:44:49.486 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5570 fp=#0x10000000f4d' 2024-09-10T08:44:49.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5602 fp=#0x10000000f6e' 2024-09-10T08:44:49.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5634 fp=#0x10000004d77' 2024-09-10T08:44:49.487 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5666 fp=#0x100000020db' 2024-09-10T08:44:49.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5698 fp=#0x1000000108c' 2024-09-10T08:44:49.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5730 fp=#0x1000000209f' 2024-09-10T08:44:49.488 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5762 fp=#0x10000000fd7' 2024-09-10T08:44:49.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5794 fp=#0x10000004514' 2024-09-10T08:44:49.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5826 fp=#0x100000042f2' 2024-09-10T08:44:49.489 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5858 fp=#0x100000044af' 2024-09-10T08:44:49.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5890 fp=#0x10000004c1b' 2024-09-10T08:44:49.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5922 fp=#0x100000043bd' 2024-09-10T08:44:49.490 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:5954 fp=#0x10000004cbe' 2024-09-10T08:44:49.491 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6082 fp=#0x100000035d9' 2024-09-10T08:44:49.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6114 fp=#0x10000003550' 2024-09-10T08:44:49.492 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6178 fp=#0x100000035ad' 2024-09-10T08:44:49.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6242 fp=#0x100000035ba' 2024-09-10T08:44:49.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6274 fp=#0x10000004b7e' 2024-09-10T08:44:49.493 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6306 fp=#0x10000001e7f' 2024-09-10T08:44:49.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6338 fp=#0x10000001f9b' 2024-09-10T08:44:49.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6370 fp=#0x10000001e51' 2024-09-10T08:44:49.494 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6402 fp=#0x10000004f7b' 2024-09-10T08:44:49.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6434 fp=#0x10000004ba7' 2024-09-10T08:44:49.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6466 fp=#0x10000004c7d' 2024-09-10T08:44:49.495 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6498 fp=#0x10000004db8' 2024-09-10T08:44:49.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6530 fp=#0x10000004bb0' 2024-09-10T08:44:49.496 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6626 fp=#0x100000039d5' 2024-09-10T08:44:49.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6658 fp=#0x10000003949' 2024-09-10T08:44:49.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6690 fp=#0x100000039b1' 2024-09-10T08:44:49.497 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6722 fp=#0x100000039da' 2024-09-10T08:44:49.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6754 fp=#0x10000003b96' 2024-09-10T08:44:49.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6786 fp=#0x1000000489c' 2024-09-10T08:44:49.498 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6818 fp=#0x10000004649' 2024-09-10T08:44:49.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6850 fp=#0x10000005281' 2024-09-10T08:44:49.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6882 fp=#0x100000052ee' 2024-09-10T08:44:49.499 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6914 fp=#0x100000047ba' 2024-09-10T08:44:49.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6946 fp=#0x10000004581' 2024-09-10T08:44:49.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:6978 fp=#0x10000000827' 2024-09-10T08:44:49.500 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7010 fp=#0x10000000669' 2024-09-10T08:44:49.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7042 fp=#0x1000000056c' 2024-09-10T08:44:49.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7074 fp=#0x10000000679' 2024-09-10T08:44:49.501 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7106 fp=#0x1000000432c' 2024-09-10T08:44:49.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7138 fp=#0x10000005099' 2024-09-10T08:44:49.502 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7170 fp=#0x10000004ff8' 2024-09-10T08:44:49.503 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7266 fp=#0x100000050f3' 2024-09-10T08:44:49.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7362 fp=#0x1000000505c' 2024-09-10T08:44:49.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7394 fp=#0x100000023c7' 2024-09-10T08:44:49.504 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7426 fp=#0x100000023ad' 2024-09-10T08:44:49.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7458 fp=#0x1000000473b' 2024-09-10T08:44:49.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7490 fp=#0x1000000240c' 2024-09-10T08:44:49.505 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7522 fp=#0x100000023f2' 2024-09-10T08:44:49.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7554 fp=#0x10000005180' 2024-09-10T08:44:49.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7586 fp=#0x10000001332' 2024-09-10T08:44:49.506 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7618 fp=#0x10000005170' 2024-09-10T08:44:49.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7650 fp=#0x100000035ff' 2024-09-10T08:44:49.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7682 fp=#0x1000000518f' 2024-09-10T08:44:49.507 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7714 fp=#0x10000004fbd' 2024-09-10T08:44:49.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7746 fp=#0x10000004e27' 2024-09-10T08:44:49.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7778 fp=#0x10000004eec' 2024-09-10T08:44:49.508 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7810 fp=#0x100000013d0' 2024-09-10T08:44:49.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7842 fp=#0x10000001531' 2024-09-10T08:44:49.509 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7874 fp=#0x100000014f5' 2024-09-10T08:44:49.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7906 fp=#0x10000004e35' 2024-09-10T08:44:49.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7938 fp=#0x10000004f0d' 2024-09-10T08:44:49.510 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:7970 fp=#0x10000001470' 2024-09-10T08:44:49.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8002 fp=#0x100000016a8' 2024-09-10T08:44:49.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8034 fp=#0x10000001668' 2024-09-10T08:44:49.511 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8066 fp=#0x100000016db' 2024-09-10T08:44:49.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8098 fp=#0x100000016b0' 2024-09-10T08:44:49.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8130 fp=#0x10000003b99' 2024-09-10T08:44:49.512 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8162 fp=#0x100000024ff' 2024-09-10T08:44:49.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8194 fp=#0x100000025da' 2024-09-10T08:44:49.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8226 fp=#0x10000002651' 2024-09-10T08:44:49.513 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8258 fp=#0x10000004141' 2024-09-10T08:44:49.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8290 fp=#0x10000004429' 2024-09-10T08:44:49.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8322 fp=#0x100000045ff' 2024-09-10T08:44:49.514 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8354 fp=#0x1000000468f' 2024-09-10T08:44:49.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8386 fp=#0x10000004400' 2024-09-10T08:44:49.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8418 fp=#0x10000005273' 2024-09-10T08:44:49.515 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8450 fp=#0x10000004509' 2024-09-10T08:44:49.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8482 fp=#0x10000003dfc' 2024-09-10T08:44:49.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8514 fp=#0x10000003f8d' 2024-09-10T08:44:49.516 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8546 fp=#0x10000003ed6' 2024-09-10T08:44:49.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8578 fp=#0x10000003eee' 2024-09-10T08:44:49.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8610 fp=#0x10000003df7' 2024-09-10T08:44:49.517 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8642 fp=#0x100000040a0' 2024-09-10T08:44:49.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8674 fp=#0x10000003ee2' 2024-09-10T08:44:49.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8706 fp=#0x10000003c79' 2024-09-10T08:44:49.518 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8738 fp=#0x100000052b1' 2024-09-10T08:44:49.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8770 fp=#0x10000004077' 2024-09-10T08:44:49.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8802 fp=#0x10000003c45' 2024-09-10T08:44:49.519 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8834 fp=#0x10000001d4d' 2024-09-10T08:44:49.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8866 fp=#0x10000001d0f' 2024-09-10T08:44:49.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8898 fp=#0x1000000445b' 2024-09-10T08:44:49.520 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8930 fp=#0x10000001d71' 2024-09-10T08:44:49.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8962 fp=#0x10000001d6c' 2024-09-10T08:44:49.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:8994 fp=#0x1000000441b' 2024-09-10T08:44:49.521 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9026 fp=#0x10000000d74' 2024-09-10T08:44:49.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9058 fp=#0x1000000406c' 2024-09-10T08:44:49.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9090 fp=#0x10000004080' 2024-09-10T08:44:49.522 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9122 fp=#0x10000000d31' 2024-09-10T08:44:49.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9154 fp=#0x1000000426f' 2024-09-10T08:44:49.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9186 fp=#0x10000000d29' 2024-09-10T08:44:49.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9218 fp=#0x10000000d52' 2024-09-10T08:44:49.523 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9250 fp=#0x10000000be5' 2024-09-10T08:44:49.524 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9314 fp=#0x10000000b33' 2024-09-10T08:44:49.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9346 fp=#0x10000004c88' 2024-09-10T08:44:49.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9378 fp=#0x10000000ada' 2024-09-10T08:44:49.525 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9410 fp=#0x10000000b08' 2024-09-10T08:44:49.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9442 fp=#0x10000001c0a' 2024-09-10T08:44:49.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9474 fp=#0x10000002695' 2024-09-10T08:44:49.526 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9506 fp=#0x10000000f53' 2024-09-10T08:44:49.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9538 fp=#0x10000000ea3' 2024-09-10T08:44:49.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9570 fp=#0x10000001ebb' 2024-09-10T08:44:49.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9602 fp=#0x10000000ecd' 2024-09-10T08:44:49.527 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9634 fp=#0x10000000eda' 2024-09-10T08:44:49.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9666 fp=#0x100000048d5' 2024-09-10T08:44:49.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9698 fp=#0x1000000481a' 2024-09-10T08:44:49.528 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9730 fp=#0x100000017de' 2024-09-10T08:44:49.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9762 fp=#0x10000004431' 2024-09-10T08:44:49.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9794 fp=#0x10000004392' 2024-09-10T08:44:49.529 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9826 fp=#0x100000018fb' 2024-09-10T08:44:49.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9858 fp=#0x10000001a32' 2024-09-10T08:44:49.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9890 fp=#0x10000001876' 2024-09-10T08:44:49.530 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9922 fp=#0x100000015b5' 2024-09-10T08:44:49.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9954 fp=#0x1000000198d' 2024-09-10T08:44:49.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:9986 fp=#0x10000002679' 2024-09-10T08:44:49.531 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10018 fp=#0x100000017b3' 2024-09-10T08:44:49.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10050 fp=#0x100000018f0' 2024-09-10T08:44:49.532 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10082 fp=#0x100000039c8' 2024-09-10T08:44:49.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10114 fp=#0x10000001db4' 2024-09-10T08:44:49.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10146 fp=#0x10000001e09' 2024-09-10T08:44:49.533 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10178 fp=#0x10000001f42' 2024-09-10T08:44:49.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10210 fp=#0x10000002491' 2024-09-10T08:44:49.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10242 fp=#0x10000003ac4' 2024-09-10T08:44:49.534 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10274 fp=#0x100000023e2' 2024-09-10T08:44:49.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10306 fp=#0x10000002434' 2024-09-10T08:44:49.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10338 fp=#0x10000003998' 2024-09-10T08:44:49.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10370 fp=#0x1000000392f' 2024-09-10T08:44:49.535 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10402 fp=#0x100000013e1' 2024-09-10T08:44:49.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10434 fp=#0x1000000487d' 2024-09-10T08:44:49.536 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10498 fp=#0x10000003909' 2024-09-10T08:44:49.537 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10530 fp=#0x10000003a09' 2024-09-10T08:44:49.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10626 fp=#0x100000049b0' 2024-09-10T08:44:49.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10658 fp=#0x10000004a26' 2024-09-10T08:44:49.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10690 fp=#0x10000004a13' 2024-09-10T08:44:49.538 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10722 fp=#0x10000000a8e' 2024-09-10T08:44:49.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10754 fp=#0x10000004966' 2024-09-10T08:44:49.539 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10818 fp=#0x10000000a6a' 2024-09-10T08:44:49.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10850 fp=#0x100000007ae' 2024-09-10T08:44:49.540 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10882 fp=#0x10000001ce2' 2024-09-10T08:44:49.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10946 fp=#0x100000010ab' 2024-09-10T08:44:49.541 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:10978 fp=#0x100000015af' 2024-09-10T08:44:49.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11042 fp=#0x10000003b12' 2024-09-10T08:44:49.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11074 fp=#0x100000038c6' 2024-09-10T08:44:49.542 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11106 fp=#0x100000039a9' 2024-09-10T08:44:49.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11138 fp=#0x10000003f0e' 2024-09-10T08:44:49.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11170 fp=#0x10000002036' 2024-09-10T08:44:49.543 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11202 fp=#0x1000000207a' 2024-09-10T08:44:49.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11234 fp=#0x1000000204f' 2024-09-10T08:44:49.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11266 fp=#0x10000003e38' 2024-09-10T08:44:49.544 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11298 fp=#0x10000001d2d' 2024-09-10T08:44:49.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11330 fp=#0x10000001ae0' 2024-09-10T08:44:49.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11362 fp=#0x10000004012' 2024-09-10T08:44:49.545 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11394 fp=#0x10000003eff' 2024-09-10T08:44:49.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11426 fp=#0x10000001af6' 2024-09-10T08:44:49.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11458 fp=#0x10000001a6c' 2024-09-10T08:44:49.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11490 fp=#0x10000001c79' 2024-09-10T08:44:49.546 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11522 fp=#0x10000003dff' 2024-09-10T08:44:49.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11650 fp=#0x10000003868' 2024-09-10T08:44:49.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11682 fp=#0x100000051ba' 2024-09-10T08:44:49.548 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11714 fp=#0x10000000b32' 2024-09-10T08:44:49.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11746 fp=#0x10000004dca' 2024-09-10T08:44:49.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11778 fp=#0x10000004e6c' 2024-09-10T08:44:49.549 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11810 fp=#0x10000004dab' 2024-09-10T08:44:49.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11842 fp=#0x10000000941' 2024-09-10T08:44:49.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11874 fp=#0x10000004e1f' 2024-09-10T08:44:49.550 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:11906 fp=#0x10000000f3b' 2024-09-10T08:44:49.551 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12002 fp=#0x100000010e9' 2024-09-10T08:44:49.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12066 fp=#0x100000045b2' 2024-09-10T08:44:49.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12098 fp=#0x10000003402' 2024-09-10T08:44:49.552 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12130 fp=#0x1000000450d' 2024-09-10T08:44:49.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12194 fp=#0x10000004462' 2024-09-10T08:44:49.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12226 fp=#0x10000000be1' 2024-09-10T08:44:49.553 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12258 fp=#0x10000003b93' 2024-09-10T08:44:49.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12290 fp=#0x10000000bdb' 2024-09-10T08:44:49.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12322 fp=#0x10000000f89' 2024-09-10T08:44:49.554 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12354 fp=#0x1000000182c' 2024-09-10T08:44:49.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12386 fp=#0x10000000d65' 2024-09-10T08:44:49.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12418 fp=#0x10000003bd6' 2024-09-10T08:44:49.555 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12450 fp=#0x10000003b2e' 2024-09-10T08:44:49.556 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12546 fp=#0x10000004c38' 2024-09-10T08:44:49.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12578 fp=#0x10000004afc' 2024-09-10T08:44:49.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12610 fp=#0x10000004bec' 2024-09-10T08:44:49.557 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12642 fp=#0x10000004c54' 2024-09-10T08:44:49.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12674 fp=#0x10000004956' 2024-09-10T08:44:49.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12706 fp=#0x1000000486d' 2024-09-10T08:44:49.558 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12738 fp=#0x1000000479b' 2024-09-10T08:44:49.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12770 fp=#0x10000005419' 2024-09-10T08:44:49.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12802 fp=#0x1000000532a' 2024-09-10T08:44:49.559 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12834 fp=#0x100000020fb' 2024-09-10T08:44:49.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12866 fp=#0x10000002185' 2024-09-10T08:44:49.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12898 fp=#0x10000002139' 2024-09-10T08:44:49.560 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12930 fp=#0x1000000212c' 2024-09-10T08:44:49.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:12962 fp=#0x1000000279b' 2024-09-10T08:44:49.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13050 fp=#0x1000000468b' 2024-09-10T08:44:49.561 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13082 fp=#0x1000000223a' 2024-09-10T08:44:49.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13114 fp=#0x10000005392' 2024-09-10T08:44:49.565 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for a577fe06-1: 'internal op quiesce_inode:mds.0:13146 fp=#0x1000000227a' 2024-09-10T08:44:49.566 INFO:tasks.quiescer.fs.[cephfs]:Pulled 13028 ops from rank 0 (b) into home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898917/ops-a577fe06-1-mds.b.json 2024-09-10T08:44:49.566 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -- /var/run/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ops-a577fe06-1-mds.b.json 2024-09-10T08:44:49.569 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:44 smithi089 ceph-mon[30757]: pgmap v537: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 14 KiB/s wr, 798 op/s 2024-09-10T08:44:49.569 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:46 smithi089 ceph-mon[30757]: pgmap v538: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 11 KiB/s wr, 606 op/s 2024-09-10T08:44:49.569 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:44 smithi149 ceph-mon[36167]: pgmap v537: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 14 KiB/s wr, 798 op/s 2024-09-10T08:44:49.569 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:46 smithi149 ceph-mon[36167]: pgmap v538: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 11 KiB/s wr, 606 op/s 2024-09-10T08:44:49.569 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:48 smithi149 ceph-mon[36167]: pgmap v539: 129 pgs: 129 active+clean; 303 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 945 KiB/s wr, 327 op/s 2024-09-10T08:44:49.569 INFO:tasks.workunit.client.0.smithi089.stdout: 87 13 0 19 0 66 0 2024-09-10T08:44:49.570 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:44 smithi111 ceph-mon[35220]: pgmap v537: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 14 KiB/s wr, 798 op/s 2024-09-10T08:44:49.570 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:46 smithi111 ceph-mon[35220]: pgmap v538: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 11 KiB/s wr, 606 op/s 2024-09-10T08:44:49.570 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:48 smithi111 ceph-mon[35220]: pgmap v539: 129 pgs: 129 active+clean; 303 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 945 KiB/s wr, 327 op/s 2024-09-10T08:44:49.570 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:48 smithi089 ceph-mon[30757]: pgmap v539: 129 pgs: 129 active+clean; 303 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 945 KiB/s wr, 327 op/s 2024-09-10T08:44:49.686 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:44:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.928+0000 7fc5f1c00640 1 -- 172.21.15.89:0/3423854736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5ec1023c0 msgr2=0x7fc5ec1047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.928+0000 7fc5f1c00640 1 --2- 172.21.15.89:0/3423854736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5ec1023c0 0x7fc5ec1047b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc009900 tx=0x7fc5dc02a410 comp rx=0 tx=0).stop 2024-09-10T08:44:49.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.928+0000 7fc5f1c00640 1 -- 172.21.15.89:0/3423854736 shutdown_connections 2024-09-10T08:44:49.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.928+0000 7fc5f1c00640 1 --2- 172.21.15.89:0/3423854736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5ec107620 0x7fc5ec109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.928+0000 7fc5f1c00640 1 --2- 172.21.15.89:0/3423854736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5ec104cf0 0x7fc5ec1070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.928+0000 7fc5f1c00640 1 --2- 172.21.15.89:0/3423854736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5ec1023c0 0x7fc5ec1047b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.928+0000 7fc5f1c00640 1 -- 172.21.15.89:0/3423854736 >> 172.21.15.89:0/3423854736 conn(0x7fc5ec0fbfa0 msgr2=0x7fc5ec0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:49.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.928+0000 7fc5f1c00640 1 -- 172.21.15.89:0/3423854736 shutdown_connections 2024-09-10T08:44:49.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.928+0000 7fc5f1c00640 1 -- 172.21.15.89:0/3423854736 wait complete. 2024-09-10T08:44:49.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.929+0000 7fc5f1c00640 1 Processor -- start 2024-09-10T08:44:49.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.930+0000 7fc5f1c00640 1 -- start start 2024-09-10T08:44:49.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.930+0000 7fc5f1c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5ec1023c0 0x7fc5ec199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:49.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.930+0000 7fc5f1c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5ec104cf0 0x7fc5ec19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:49.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.930+0000 7fc5f1c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5ec107620 0x7fc5ec1a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:49.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.930+0000 7fc5f1c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec10d650 con 0x7fc5ec107620 2024-09-10T08:44:49.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.930+0000 7fc5f1c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec10d4d0 con 0x7fc5ec104cf0 2024-09-10T08:44:49.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.930+0000 7fc5f1c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec10d7d0 con 0x7fc5ec1023c0 2024-09-10T08:44:49.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.930+0000 7fc5ebe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5ec1023c0 0x7fc5ec199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:49.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.930+0000 7fc5ebe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5ec1023c0 0x7fc5ec199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:45694/0 (socket says 172.21.15.89:45694) 2024-09-10T08:44:49.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.930+0000 7fc5ebe00640 1 -- 172.21.15.89:0/3457857979 learned_addr learned my addr 172.21.15.89:0/3457857979 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:49.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.930+0000 7fc5eb400640 1 --2- 172.21.15.89:0/3457857979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5ec104cf0 0x7fc5ec19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:49.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.930+0000 7fc5f1200640 1 --2- 172.21.15.89:0/3457857979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5ec107620 0x7fc5ec1a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.931+0000 7fc5ebe00640 1 -- 172.21.15.89:0/3457857979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5ec104cf0 msgr2=0x7fc5ec19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.931+0000 7fc5ebe00640 1 --2- 172.21.15.89:0/3457857979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5ec104cf0 0x7fc5ec19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.931+0000 7fc5ebe00640 1 -- 172.21.15.89:0/3457857979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5ec107620 msgr2=0x7fc5ec1a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.931+0000 7fc5ebe00640 1 --2- 172.21.15.89:0/3457857979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5ec107620 0x7fc5ec1a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.931+0000 7fc5ebe00640 1 -- 172.21.15.89:0/3457857979 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ec1a1b10 con 0x7fc5ec1023c0 2024-09-10T08:44:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.931+0000 7fc5ebe00640 1 --2- 172.21.15.89:0/3457857979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5ec1023c0 0x7fc5ec199ef0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc031440 tx=0x7fc5dc032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.931+0000 7fc5e8c00640 1 -- 172.21.15.89:0/3457857979 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc03f030 con 0x7fc5ec1023c0 2024-09-10T08:44:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.931+0000 7fc5e8c00640 1 -- 172.21.15.89:0/3457857979 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5dc003ce0 con 0x7fc5ec1023c0 2024-09-10T08:44:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.931+0000 7fc5e8c00640 1 -- 172.21.15.89:0/3457857979 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc035070 con 0x7fc5ec1023c0 2024-09-10T08:44:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.931+0000 7fc5f1c00640 1 -- 172.21.15.89:0/3457857979 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5ec1a1da0 con 0x7fc5ec1023c0 2024-09-10T08:44:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.932+0000 7fc5f1c00640 1 -- 172.21.15.89:0/3457857979 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5ec1a2180 con 0x7fc5ec1023c0 2024-09-10T08:44:49.934 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.935+0000 7fc5f6005740 1 Processor -- start 2024-09-10T08:44:49.934 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.935+0000 7fc5f6005740 1 -- start start 2024-09-10T08:44:49.935 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.935+0000 7fc5f6005740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea47260 0x55c7eea47660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:49.935 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.935+0000 7fc5f6005740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47ba0 0x55c7eea4c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:49.935 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.935+0000 7fc5f6005740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea4cc30 0x55c7eea4f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:49.935 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.935+0000 7fc5f6005740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c7ee9ac100 con 0x55c7eea47260 2024-09-10T08:44:49.935 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.935+0000 7fc5f6005740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c7ee9541b0 con 0x55c7eea4cc30 2024-09-10T08:44:49.935 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.935+0000 7fc5f6005740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c7ee9a80c0 con 0x55c7eea47ba0 2024-09-10T08:44:49.936 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.935+0000 7fc5f1200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea47260 0x55c7eea47660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:49.936 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.936+0000 7fc5eb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea4cc30 0x55c7eea4f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:49.936 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.936+0000 7fc5eb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea4cc30 0x55c7eea4f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:36462/0 (socket says 172.21.15.89:36462) 2024-09-10T08:44:49.936 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.936+0000 7fc5f1200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea47260 0x55c7eea47660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:45624/0 (socket says 172.21.15.89:45624) 2024-09-10T08:44:49.936 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.936+0000 7fc5eb400640 1 -- 172.21.15.89:0/2540372482 learned_addr learned my addr 172.21.15.89:0/2540372482 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:49.936 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.936+0000 7fc5e8c00640 1 -- 172.21.15.89:0/3457857979 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc5dc042030 con 0x7fc5ec1023c0 2024-09-10T08:44:49.936 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.936+0000 7fc5ebe00640 1 --2- 172.21.15.89:0/2540372482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47ba0 0x55c7eea4c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.936+0000 7fc5f1200640 1 -- 172.21.15.89:0/2540372482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47ba0 msgr2=0x55c7eea4c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.936+0000 7fc5f1200640 1 --2- 172.21.15.89:0/2540372482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47ba0 0x55c7eea4c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.936+0000 7fc5f1200640 1 -- 172.21.15.89:0/2540372482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea4cc30 msgr2=0x55c7eea4f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.936+0000 7fc5f1200640 1 --2- 172.21.15.89:0/2540372482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea4cc30 0x55c7eea4f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.936+0000 7fc5f1200640 1 -- 172.21.15.89:0/2540372482 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7ee6bf210 con 0x55c7eea47260 2024-09-10T08:44:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.936+0000 7fc5eb400640 1 --2- 172.21.15.89:0/2540372482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea4cc30 0x55c7eea4f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.937+0000 7fc5f1200640 1 --2- 172.21.15.89:0/2540372482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea47260 0x55c7eea47660 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4005fd0 tx=0x7fc5e4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2456b586e24a35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.937+0000 7fc5e1600640 1 -- 172.21.15.89:0/2540372482 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4004d20 con 0x55c7eea47260 2024-09-10T08:44:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.937+0000 7fc5e1600640 1 -- 172.21.15.89:0/2540372482 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5e4004030 con 0x55c7eea47260 2024-09-10T08:44:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.937+0000 7fc5e1600640 1 -- 172.21.15.89:0/2540372482 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4005280 con 0x55c7eea47260 2024-09-10T08:44:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.938+0000 7fc5e8c00640 1 --2- 172.21.15.89:0/3457857979 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc5d00a2c00 0x7fc5d00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.938+0000 7fc5e8c00640 1 -- 172.21.15.89:0/3457857979 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fc5dc0efcc0 con 0x7fc5ec1023c0 2024-09-10T08:44:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.938+0000 7fc5f6005740 1 -- 172.21.15.89:0/2540372482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea47260 msgr2=0x55c7eea47660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.938+0000 7fc5f6005740 1 --2- 172.21.15.89:0/2540372482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea47260 0x55c7eea47660 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4005fd0 tx=0x7fc5e4011810 comp rx=0 tx=0).stop 2024-09-10T08:44:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.938+0000 7fc5ebe00640 1 --2- 172.21.15.89:0/3457857979 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc5d00a2c00 0x7fc5d00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.938+0000 7fc5f6005740 1 -- 172.21.15.89:0/2540372482 shutdown_connections 2024-09-10T08:44:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.938+0000 7fc5f6005740 1 --2- 172.21.15.89:0/2540372482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea4cc30 0x55c7eea4f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.938+0000 7fc5f6005740 1 --2- 172.21.15.89:0/2540372482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47ba0 0x55c7eea4c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.938+0000 7fc5f6005740 1 --2- 172.21.15.89:0/2540372482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea47260 0x55c7eea47660 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.938+0000 7fc5f6005740 1 -- 172.21.15.89:0/2540372482 >> 172.21.15.89:0/2540372482 conn(0x55c7eea428d0 msgr2=0x55c7eea42cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.938+0000 7fc5f6005740 1 -- 172.21.15.89:0/2540372482 shutdown_connections 2024-09-10T08:44:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.938+0000 7fc5f6005740 1 -- 172.21.15.89:0/2540372482 wait complete. 2024-09-10T08:44:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.939+0000 7fc5ebe00640 1 --2- 172.21.15.89:0/3457857979 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc5d00a2c00 0x7fc5d00a50c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0b8b70 tx=0x7fc5dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.939+0000 7fc5f6005740 1 -- start start 2024-09-10T08:44:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.939+0000 7fc5f6005740 1 Processor -- start 2024-09-10T08:44:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.939+0000 7fc5f6005740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea47260 0x55c7eea513f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.939+0000 7fc5f6005740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47ba0 0x55c7eea51930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.940+0000 7fc5f6005740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea4cc30 0x55c7eea58970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.940+0000 7fc5f6005740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c7eea4fc00 con 0x55c7eea47260 2024-09-10T08:44:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.940+0000 7fc5f6005740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c7ee935090 con 0x55c7eea4cc30 2024-09-10T08:44:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.940+0000 7fc5f6005740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c7eea4fd80 con 0x55c7eea47ba0 2024-09-10T08:44:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.940+0000 7fc5f6005740 10 client.0 _ll_register_callbacks cb 0x55c7ee989f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.940+0000 7fc5eb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47ba0 0x55c7eea51930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.940+0000 7fc5eb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47ba0 0x55c7eea51930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:45708/0 (socket says 172.21.15.89:45708) 2024-09-10T08:44:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.940+0000 7fc5f1200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea4cc30 0x55c7eea58970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.940+0000 7fc5eb400640 1 -- 172.21.15.89:0/1433061140 learned_addr learned my addr 172.21.15.89:0/1433061140 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.940+0000 7fc5f1200640 1 --2- 172.21.15.89:0/1433061140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea47260 0x55c7eea513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.940+0000 7fc5eb400640 1 -- 172.21.15.89:0/1433061140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea4cc30 msgr2=0x55c7eea58970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.940+0000 7fc5eb400640 1 --2- 172.21.15.89:0/1433061140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea4cc30 0x55c7eea58970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.940+0000 7fc5eb400640 1 -- 172.21.15.89:0/1433061140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea47260 msgr2=0x55c7eea513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.940+0000 7fc5eb400640 1 --2- 172.21.15.89:0/1433061140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea47260 0x55c7eea513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.940+0000 7fc5eb400640 1 -- 172.21.15.89:0/1433061140 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7ee78b620 con 0x55c7eea47ba0 2024-09-10T08:44:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.941+0000 7fc5eb400640 1 --2- 172.21.15.89:0/1433061140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47ba0 0x55c7eea51930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80126e0 tx=0x7fc5d8012ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b39af51fb63d00ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.941+0000 7fc5d6000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:44:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.941+0000 7fc5d6000640 1 -- 172.21.15.89:0/1433061140 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80121d0 con 0x55c7eea47ba0 2024-09-10T08:44:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.941+0000 7fc5f6005740 1 -- 172.21.15.89:0/1433061140 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5e4004030 con 0x55c7eea47ba0 2024-09-10T08:44:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.941+0000 7fc5d6000640 1 -- 172.21.15.89:0/1433061140 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fc5d8004050 con 0x55c7eea47ba0 2024-09-10T08:44:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.941+0000 7fc5d6000640 1 -- 172.21.15.89:0/1433061140 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc5d8025050 con 0x55c7eea47ba0 2024-09-10T08:44:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.941+0000 7fc5f6005740 10 client.26245 fetch_fsmap learned FSMap version 29 2024-09-10T08:44:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.941+0000 7fc5f6005740 1 -- 172.21.15.89:0/1433061140 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c7eea5b830 con 0x55c7eea47ba0 2024-09-10T08:44:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.941+0000 7fc5d6000640 1 -- 172.21.15.89:0/1433061140 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5e4004030 con 0x55c7eea47ba0 2024-09-10T08:44:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.942+0000 7fc5d6000640 1 -- 172.21.15.89:0/1433061140 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fc5d8003980 con 0x55c7eea47ba0 2024-09-10T08:44:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.942+0000 7fc5f6005740 10 client.26245 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:44:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.942+0000 7fc5f6005740 10 client.26245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:44:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.942+0000 7fc5f6005740 20 client.26245 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.942+0000 7fc5f6005740 1 --2- 172.21.15.89:0/1433061140 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c7eea5cf00 0x55c7eea5f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.942+0000 7fc5f6005740 4 client.26245 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:44:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.942+0000 7fc5f6005740 1 -- 172.21.15.89:0/1433061140 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7ee9b9830 con 0x55c7eea5cf00 2024-09-10T08:44:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.942+0000 7fc5f1200640 1 --2- 172.21.15.89:0/1433061140 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c7eea5cf00 0x55c7eea5f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.943+0000 7fc5f1200640 1 --2- 172.21.15.89:0/1433061140 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c7eea5cf00 0x55c7eea5f310 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8e087a736556c48 server_cookie=ca8c629363095964 in_seq=0 out_seq=0 2024-09-10T08:44:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.943+0000 7fc5d6000640 10 client.26245 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.946+0000 7fc5d6000640 1 -- 172.21.15.89:0/1433061140 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7ee9b9830 con 0x55c7eea5cf00 2024-09-10T08:44:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.946+0000 7fc5d6000640 10 client.26245 handle_command_reply: tid=0 2024-09-10T08:44:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.946+0000 7fc5f6005740 1 client.26245 shutdown 2024-09-10T08:44:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.946+0000 7fc5f6005740 1 -- 172.21.15.89:0/1433061140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47ba0 msgr2=0x55c7eea51930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.946+0000 7fc5f6005740 1 --2- 172.21.15.89:0/1433061140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47ba0 0x55c7eea51930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80126e0 tx=0x7fc5d8012ba0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.946+0000 7fc5f6005740 1 -- 172.21.15.89:0/1433061140 shutdown_connections 2024-09-10T08:44:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.946+0000 7fc5f6005740 1 --2- 172.21.15.89:0/1433061140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea4cc30 0x55c7eea58970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.946+0000 7fc5f6005740 1 --2- 172.21.15.89:0/1433061140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47ba0 0x55c7eea51930 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.946+0000 7fc5f6005740 1 --2- 172.21.15.89:0/1433061140 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c7eea5cf00 0x55c7eea5f310 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.946+0000 7fc5f6005740 1 --2- 172.21.15.89:0/1433061140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea47260 0x55c7eea513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.946+0000 7fc5d6000640 0 client.26245 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.946+0000 7fc5f6005740 1 -- 172.21.15.89:0/1433061140 >> 172.21.15.89:0/1433061140 conn(0x55c7eea428d0 msgr2=0x55c7eea440e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.946+0000 7fc5f6005740 1 -- 172.21.15.89:0/1433061140 shutdown_connections 2024-09-10T08:44:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.946+0000 7fc5f6005740 1 -- 172.21.15.89:0/1433061140 wait complete. 2024-09-10T08:44:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.947+0000 7fc5f6005740 20 client.26245 trim_cache size 0 max 16384 2024-09-10T08:44:49.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.971+0000 7fc5f6005740 1 Processor -- start 2024-09-10T08:44:49.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.971+0000 7fc5f6005740 1 -- start start 2024-09-10T08:44:49.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.971+0000 7fc5f6005740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47260 0x55c7eea4c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:49.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.971+0000 7fc5f6005740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea47ba0 0x55c7eea58930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:49.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.971+0000 7fc5f6005740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea4cc30 0x55c7eea41900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:49.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.971+0000 7fc5f6005740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec1af130 con 0x55c7eea4cc30 2024-09-10T08:44:49.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.971+0000 7fc5f6005740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec1aca00 con 0x55c7eea47ba0 2024-09-10T08:44:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.971+0000 7fc5f6005740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec1aa850 con 0x55c7eea47260 2024-09-10T08:44:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.972+0000 7fc5eb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea47ba0 0x55c7eea58930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.972+0000 7fc5eb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea47ba0 0x55c7eea58930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:36494/0 (socket says 172.21.15.89:36494) 2024-09-10T08:44:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.972+0000 7fc5eb400640 1 -- 172.21.15.89:0/29810648 learned_addr learned my addr 172.21.15.89:0/29810648 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.972+0000 7fc5f1200640 1 --2- 172.21.15.89:0/29810648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47260 0x55c7eea4c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.972+0000 7fc5f1200640 1 --2- 172.21.15.89:0/29810648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea4cc30 0x55c7eea41900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.972+0000 7fc5eb400640 1 -- 172.21.15.89:0/29810648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47260 msgr2=0x55c7eea4c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.972+0000 7fc5eb400640 1 --2- 172.21.15.89:0/29810648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47260 0x55c7eea4c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.972+0000 7fc5eb400640 1 -- 172.21.15.89:0/29810648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea4cc30 msgr2=0x55c7eea41900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.972+0000 7fc5eb400640 1 --2- 172.21.15.89:0/29810648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea4cc30 0x55c7eea41900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.972+0000 7fc5eb400640 1 -- 172.21.15.89:0/29810648 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7eea48080 con 0x55c7eea47ba0 2024-09-10T08:44:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.972+0000 7fc5f1200640 1 --2- 172.21.15.89:0/29810648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea4cc30 0x55c7eea41900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.972+0000 7fc5eb400640 1 --2- 172.21.15.89:0/29810648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea47ba0 0x55c7eea58930 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8001eb0 tx=0x7fc5d800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c17efba6e97e4898 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.973+0000 7fc5e0c00640 1 -- 172.21.15.89:0/29810648 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8003fc0 con 0x55c7eea47ba0 2024-09-10T08:44:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.973+0000 7fc5e0c00640 1 -- 172.21.15.89:0/29810648 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5d8003ad0 con 0x55c7eea47ba0 2024-09-10T08:44:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.973+0000 7fc5e0c00640 1 -- 172.21.15.89:0/29810648 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80052d0 con 0x55c7eea47ba0 2024-09-10T08:44:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.974+0000 7fc5f6005740 1 -- 172.21.15.89:0/29810648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea47ba0 msgr2=0x55c7eea58930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.974+0000 7fc5f6005740 1 --2- 172.21.15.89:0/29810648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea47ba0 0x55c7eea58930 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8001eb0 tx=0x7fc5d800f790 comp rx=0 tx=0).stop 2024-09-10T08:44:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.974+0000 7fc5f6005740 1 -- 172.21.15.89:0/29810648 shutdown_connections 2024-09-10T08:44:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.974+0000 7fc5f6005740 1 --2- 172.21.15.89:0/29810648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea4cc30 0x55c7eea41900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.974+0000 7fc5f6005740 1 --2- 172.21.15.89:0/29810648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea47ba0 0x55c7eea58930 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.974+0000 7fc5f6005740 1 --2- 172.21.15.89:0/29810648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47260 0x55c7eea4c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.974+0000 7fc5f6005740 1 -- 172.21.15.89:0/29810648 >> 172.21.15.89:0/29810648 conn(0x55c7eea428d0 msgr2=0x55c7eea411f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.974+0000 7fc5f6005740 1 -- 172.21.15.89:0/29810648 shutdown_connections 2024-09-10T08:44:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.974+0000 7fc5f6005740 1 -- 172.21.15.89:0/29810648 wait complete. 2024-09-10T08:44:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.974+0000 7fc5f6005740 1 -- start start 2024-09-10T08:44:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.974+0000 7fc5f6005740 1 Processor -- start 2024-09-10T08:44:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.975+0000 7fc5f6005740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea47260 0x55c7eea55ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.975+0000 7fc5f6005740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47ba0 0x55c7eea563e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.975+0000 7fc5f6005740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea4cc30 0x55c7eea573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.975+0000 7fc5f6005740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec1aaec0 con 0x55c7eea4cc30 2024-09-10T08:44:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.975+0000 7fc5f6005740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec1ac330 con 0x55c7eea47260 2024-09-10T08:44:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.975+0000 7fc5f6005740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ec1ab590 con 0x55c7eea47ba0 2024-09-10T08:44:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.975+0000 7fc5f6005740 10 client.0 _ll_register_callbacks cb 0x7fc5d8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.975+0000 7fc5eb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47ba0 0x55c7eea563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.975+0000 7fc5eb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47ba0 0x55c7eea563e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:45720/0 (socket says 172.21.15.89:45720) 2024-09-10T08:44:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.975+0000 7fc5f1200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea47260 0x55c7eea55ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.975+0000 7fc5eb400640 1 -- 172.21.15.89:0/3143945136 learned_addr learned my addr 172.21.15.89:0/3143945136 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.975+0000 7fc5f1200640 1 --2- 172.21.15.89:0/3143945136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea4cc30 0x55c7eea573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.976+0000 7fc5eb400640 1 -- 172.21.15.89:0/3143945136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea47260 msgr2=0x55c7eea55ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.976+0000 7fc5eb400640 1 --2- 172.21.15.89:0/3143945136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea47260 0x55c7eea55ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.976+0000 7fc5eb400640 1 -- 172.21.15.89:0/3143945136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea4cc30 msgr2=0x55c7eea573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.976+0000 7fc5eb400640 1 --2- 172.21.15.89:0/3143945136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea4cc30 0x55c7eea573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.976+0000 7fc5eb400640 1 -- 172.21.15.89:0/3143945136 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7eea57a00 con 0x55c7eea47ba0 2024-09-10T08:44:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.976+0000 7fc5eb400640 1 --2- 172.21.15.89:0/3143945136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47ba0 0x55c7eea563e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8003d50 tx=0x7fc5d8003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfe7ccb156c375f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.976+0000 7fc5d6000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:44:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.976+0000 7fc5d6000640 1 -- 172.21.15.89:0/3143945136 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8005460 con 0x55c7eea47ba0 2024-09-10T08:44:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.976+0000 7fc5f6005740 1 -- 172.21.15.89:0/3143945136 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5d8003ad0 con 0x55c7eea47ba0 2024-09-10T08:44:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.976+0000 7fc5d6000640 1 -- 172.21.15.89:0/3143945136 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fc5d8005600 con 0x55c7eea47ba0 2024-09-10T08:44:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.976+0000 7fc5d6000640 1 -- 172.21.15.89:0/3143945136 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc5d803f050 con 0x55c7eea47ba0 2024-09-10T08:44:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.977+0000 7fc5f6005740 10 client.26251 fetch_fsmap learned FSMap version 29 2024-09-10T08:44:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.977+0000 7fc5f6005740 1 -- 172.21.15.89:0/3143945136 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c7eeac7bf0 con 0x55c7eea47ba0 2024-09-10T08:44:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.977+0000 7fc5d6000640 1 -- 172.21.15.89:0/3143945136 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5d8003ad0 con 0x55c7eea47ba0 2024-09-10T08:44:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.977+0000 7fc5d6000640 1 -- 172.21.15.89:0/3143945136 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fc5d8029820 con 0x55c7eea47ba0 2024-09-10T08:44:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.977+0000 7fc5f6005740 10 client.26251 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:44:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.977+0000 7fc5f6005740 10 client.26251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:44:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.977+0000 7fc5f6005740 20 client.26251 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.977+0000 7fc5f6005740 1 --2- 172.21.15.89:0/3143945136 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c7eeac8430 0x55c7eeaca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.977+0000 7fc5f6005740 4 client.26251 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:44:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.977+0000 7fc5f6005740 1 -- 172.21.15.89:0/3143945136 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c7ee9b3b80 con 0x55c7eeac8430 2024-09-10T08:44:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.977+0000 7fc5f1200640 1 --2- 172.21.15.89:0/3143945136 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c7eeac8430 0x55c7eeaca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.978+0000 7fc5d6000640 10 client.26251 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:49.978+0000 7fc5f1200640 1 --2- 172.21.15.89:0/3143945136 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c7eeac8430 0x55c7eeaca850 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21fb7a018220af17 server_cookie=74d7970c189f529 in_seq=0 out_seq=0 2024-09-10T08:44:50.112 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:50.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.756+0000 7fa28e200640 1 -- 172.21.15.89:0/3306771913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2881034b0 msgr2=0x7fa28810b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:50.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.756+0000 7fa28e200640 1 --2- 172.21.15.89:0/3306771913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2881034b0 0x7fa28810b9a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa274009900 tx=0x7fa27402a490 comp rx=0 tx=0).stop 2024-09-10T08:44:50.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.757+0000 7fa28e200640 1 -- 172.21.15.89:0/3306771913 shutdown_connections 2024-09-10T08:44:50.758 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.757+0000 7fa28e200640 1 --2- 172.21.15.89:0/3306771913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa28810bfc0 0x7fa28810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:50.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.757+0000 7fa28e200640 1 --2- 172.21.15.89:0/3306771913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2881034b0 0x7fa28810b9a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:50.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.757+0000 7fa28e200640 1 --2- 172.21.15.89:0/3306771913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa288102b90 0x7fa288102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:50.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.757+0000 7fa28e200640 1 -- 172.21.15.89:0/3306771913 >> 172.21.15.89:0/3306771913 conn(0x7fa2880fc610 msgr2=0x7fa2880fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:50.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.757+0000 7fa28e200640 1 -- 172.21.15.89:0/3306771913 shutdown_connections 2024-09-10T08:44:50.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.757+0000 7fa28e200640 1 -- 172.21.15.89:0/3306771913 wait complete. 2024-09-10T08:44:50.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.758+0000 7fa28e200640 1 Processor -- start 2024-09-10T08:44:50.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.758+0000 7fa28e200640 1 -- start start 2024-09-10T08:44:50.761 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.758+0000 7fa28e200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa288102b90 0x7fa288198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:50.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.758+0000 7fa28e200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2881034b0 0x7fa2881987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:50.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.759+0000 7fa28e200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa28810bfc0 0x7fa28819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:50.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.759+0000 7fa28e200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa288111e10 con 0x7fa2881034b0 2024-09-10T08:44:50.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.759+0000 7fa28e200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa288111c90 con 0x7fa28810bfc0 2024-09-10T08:44:50.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.759+0000 7fa28e200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa288111f90 con 0x7fa288102b90 2024-09-10T08:44:50.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.759+0000 7fa28ce00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa288102b90 0x7fa288198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:50.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.759+0000 7fa28d800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa28810bfc0 0x7fa28819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:50.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.759+0000 7fa287e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2881034b0 0x7fa2881987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:50.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.759+0000 7fa28ce00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa288102b90 0x7fa288198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:45736/0 (socket says 172.21.15.89:45736) 2024-09-10T08:44:50.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.759+0000 7fa28ce00640 1 -- 172.21.15.89:0/936716996 learned_addr learned my addr 172.21.15.89:0/936716996 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:50.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.759+0000 7fa28d800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa28810bfc0 0x7fa28819cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:36516/0 (socket says 172.21.15.89:36516) 2024-09-10T08:44:50.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.759+0000 7fa28ce00640 1 -- 172.21.15.89:0/936716996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa28810bfc0 msgr2=0x7fa28819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:50.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.759+0000 7fa28ce00640 1 --2- 172.21.15.89:0/936716996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa28810bfc0 0x7fa28819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:50.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.759+0000 7fa28ce00640 1 -- 172.21.15.89:0/936716996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2881034b0 msgr2=0x7fa2881987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:50.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.759+0000 7fa28ce00640 1 --2- 172.21.15.89:0/936716996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2881034b0 0x7fa2881987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.759+0000 7fa28ce00640 1 -- 172.21.15.89:0/936716996 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28819d240 con 0x7fa288102b90 2024-09-10T08:44:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.759+0000 7fa287e00640 1 --2- 172.21.15.89:0/936716996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2881034b0 0x7fa2881987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.760+0000 7fa28ce00640 1 --2- 172.21.15.89:0/936716996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa288102b90 0x7fa288198290 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00e9f0 tx=0x7fa27c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.760+0000 7fa285600640 1 -- 172.21.15.89:0/936716996 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c00cd80 con 0x7fa288102b90 2024-09-10T08:44:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.760+0000 7fa285600640 1 -- 172.21.15.89:0/936716996 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa27c010910 con 0x7fa288102b90 2024-09-10T08:44:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.760+0000 7fa285600640 1 -- 172.21.15.89:0/936716996 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c018990 con 0x7fa288102b90 2024-09-10T08:44:50.764 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.760+0000 7fa28e200640 1 -- 172.21.15.89:0/936716996 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28819d530 con 0x7fa288102b90 2024-09-10T08:44:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.760+0000 7fa28e200640 1 -- 172.21.15.89:0/936716996 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2881a4e10 con 0x7fa288102b90 2024-09-10T08:44:50.765 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.761+0000 7fa28e200640 1 -- 172.21.15.89:0/936716996 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa288102f70 con 0x7fa288102b90 2024-09-10T08:44:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.764+0000 7fa285600640 1 -- 172.21.15.89:0/936716996 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa27c010ac0 con 0x7fa288102b90 2024-09-10T08:44:50.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.765+0000 7fa285600640 1 --2- 172.21.15.89:0/936716996 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa2700a2b80 0x7fa2700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:50.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.765+0000 7fa285600640 1 -- 172.21.15.89:0/936716996 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa27c0cf780 con 0x7fa288102b90 2024-09-10T08:44:50.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.765+0000 7fa287e00640 1 --2- 172.21.15.89:0/936716996 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa2700a2b80 0x7fa2700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:50.767 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.766+0000 7fa287e00640 1 --2- 172.21.15.89:0/936716996 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa2700a2b80 0x7fa2700a5040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa274000c00 tx=0x7fa274031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:50.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:50.769+0000 7fa285600640 1 -- 172.21.15.89:0/936716996 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa27c09bc60 con 0x7fa288102b90 2024-09-10T08:44:51.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:51.024+0000 7fa28e200640 1 -- 172.21.15.89:0/936716996 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2881995f0 con 0x7fa288102b90 2024-09-10T08:44:51.025 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:50 smithi089 ceph-mon[30757]: pgmap v540: 129 pgs: 129 active+clean; 305 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 1.2 MiB/s wr, 308 op/s 2024-09-10T08:44:51.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:51.025+0000 7fa285600640 1 -- 172.21.15.89:0/936716996 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11919 (secure 0 0 0) 0x7fa27c09ff60 con 0x7fa288102b90 2024-09-10T08:44:51.026 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:44:51.026 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"btime":"2024-09-10T08:44:40:778845+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:44:40.174532+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24496,24518,24554]},"id":1}]} 2024-09-10T08:44:51.026 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 29 2024-09-10T08:44:51.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:51.028+0000 7fa28e200640 1 -- 172.21.15.89:0/936716996 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa2700a2b80 msgr2=0x7fa2700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:51.028+0000 7fa28e200640 1 --2- 172.21.15.89:0/936716996 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa2700a2b80 0x7fa2700a5040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa274000c00 tx=0x7fa274031510 comp rx=0 tx=0).stop 2024-09-10T08:44:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:51.028+0000 7fa28e200640 1 -- 172.21.15.89:0/936716996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa288102b90 msgr2=0x7fa288198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:51.028+0000 7fa28e200640 1 --2- 172.21.15.89:0/936716996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa288102b90 0x7fa288198290 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00e9f0 tx=0x7fa27c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:44:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:51.028+0000 7fa28e200640 1 -- 172.21.15.89:0/936716996 shutdown_connections 2024-09-10T08:44:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:51.028+0000 7fa28e200640 1 --2- 172.21.15.89:0/936716996 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa2700a2b80 0x7fa2700a5040 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:51.028+0000 7fa28e200640 1 --2- 172.21.15.89:0/936716996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa28810bfc0 0x7fa28819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:51.028+0000 7fa28e200640 1 --2- 172.21.15.89:0/936716996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2881034b0 0x7fa2881987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:51.028+0000 7fa28e200640 1 --2- 172.21.15.89:0/936716996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa288102b90 0x7fa288198290 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:51.028+0000 7fa28e200640 1 -- 172.21.15.89:0/936716996 >> 172.21.15.89:0/936716996 conn(0x7fa2880fc610 msgr2=0x7fa2880fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:51.029+0000 7fa28e200640 1 -- 172.21.15.89:0/936716996 shutdown_connections 2024-09-10T08:44:51.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:51.029+0000 7fa28e200640 1 -- 172.21.15.89:0/936716996 wait complete. 2024-09-10T08:44:51.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:50 smithi149 ceph-mon[36167]: pgmap v540: 129 pgs: 129 active+clean; 305 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 1.2 MiB/s wr, 308 op/s 2024-09-10T08:44:51.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:50 smithi111 ceph-mon[35220]: pgmap v540: 129 pgs: 129 active+clean; 305 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 1.2 MiB/s wr, 308 op/s 2024-09-10T08:44:51.831 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24554 quiesce db --set-id a577fe06 --await-for 45.0' 2024-09-10T08:44:51.832 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24554 quiesce db --set-id a577fe06 --await-for 45.0 2024-09-10T08:44:52.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:51 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/936716996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:52.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:51 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/936716996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:52.238 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.263+0000 7fc5d6000640 1 -- 172.21.15.89:0/3143945136 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c7ee9b3b80 con 0x55c7eeac8430 2024-09-10T08:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.263+0000 7fc5d6000640 10 client.26251 handle_command_reply: tid=0 2024-09-10T08:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.263+0000 7fc5f6005740 1 client.26251 shutdown 2024-09-10T08:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.263+0000 7fc5f6005740 1 -- 172.21.15.89:0/3143945136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47ba0 msgr2=0x55c7eea563e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.263+0000 7fc5f6005740 1 --2- 172.21.15.89:0/3143945136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47ba0 0x55c7eea563e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8003d50 tx=0x7fc5d8003820 comp rx=0 tx=0).stop 2024-09-10T08:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.263+0000 7fc5f6005740 1 -- 172.21.15.89:0/3143945136 shutdown_connections 2024-09-10T08:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.263+0000 7fc5f6005740 1 --2- 172.21.15.89:0/3143945136 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c7eeac8430 0x55c7eeaca850 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.263+0000 7fc5f6005740 1 --2- 172.21.15.89:0/3143945136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c7eea4cc30 0x55c7eea573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.263+0000 7fc5f6005740 1 --2- 172.21.15.89:0/3143945136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c7eea47ba0 0x55c7eea563e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.263+0000 7fc5f6005740 1 --2- 172.21.15.89:0/3143945136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7eea47260 0x55c7eea55ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.263+0000 7fc5f6005740 1 -- 172.21.15.89:0/3143945136 >> 172.21.15.89:0/3143945136 conn(0x55c7eea428d0 msgr2=0x55c7eea41a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.263+0000 7fc5d6000640 0 client.26251 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.263+0000 7fc5f6005740 1 -- 172.21.15.89:0/3143945136 shutdown_connections 2024-09-10T08:44:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.263+0000 7fc5f6005740 1 -- 172.21.15.89:0/3143945136 wait complete. 2024-09-10T08:44:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.263+0000 7fc5f6005740 20 client.26251 trim_cache size 0 max 16384 2024-09-10T08:44:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:44:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:44:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:44:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:44:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.264+0000 7fc5f1c00640 1 -- 172.21.15.89:0/3457857979 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc5d00a2c00 msgr2=0x7fc5d00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.264+0000 7fc5f1c00640 1 --2- 172.21.15.89:0/3457857979 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc5d00a2c00 0x7fc5d00a50c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0b8b70 tx=0x7fc5dc009260 comp rx=0 tx=0).stop 2024-09-10T08:44:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.264+0000 7fc5f1c00640 1 -- 172.21.15.89:0/3457857979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5ec1023c0 msgr2=0x7fc5ec199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.264+0000 7fc5f1c00640 1 --2- 172.21.15.89:0/3457857979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5ec1023c0 0x7fc5ec199ef0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc031440 tx=0x7fc5dc032ce0 comp rx=0 tx=0).stop 2024-09-10T08:44:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.264+0000 7fc5f1c00640 1 -- 172.21.15.89:0/3457857979 shutdown_connections 2024-09-10T08:44:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.264+0000 7fc5f1c00640 1 --2- 172.21.15.89:0/3457857979 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc5d00a2c00 0x7fc5d00a50c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.264+0000 7fc5f1c00640 1 --2- 172.21.15.89:0/3457857979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5ec107620 0x7fc5ec1a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.264+0000 7fc5f1c00640 1 --2- 172.21.15.89:0/3457857979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5ec104cf0 0x7fc5ec19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.264+0000 7fc5f1c00640 1 --2- 172.21.15.89:0/3457857979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5ec1023c0 0x7fc5ec199ef0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.264+0000 7fc5f1c00640 1 -- 172.21.15.89:0/3457857979 >> 172.21.15.89:0/3457857979 conn(0x7fc5ec0fbfa0 msgr2=0x7fc5ec101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.264+0000 7fc5f1c00640 1 -- 172.21.15.89:0/3457857979 shutdown_connections 2024-09-10T08:44:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:44:52.264+0000 7fc5f1c00640 1 -- 172.21.15.89:0/3457857979 wait complete. 2024-09-10T08:44:52.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:51 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/936716996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:53.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:52 smithi089 ceph-mon[30757]: pgmap v541: 129 pgs: 129 active+clean; 308 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 562 op/s 2024-09-10T08:44:53.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:52 smithi149 ceph-mon[36167]: pgmap v541: 129 pgs: 129 active+clean; 308 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 562 op/s 2024-09-10T08:44:53.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:52 smithi111 ceph-mon[35220]: pgmap v541: 129 pgs: 129 active+clean; 308 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 562 op/s 2024-09-10T08:44:53.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.332+0000 7fddfde00640 1 -- 172.21.15.89:0/2076826760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddf81047e0 msgr2=0x7fddf8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:53.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.332+0000 7fddfde00640 1 --2- 172.21.15.89:0/2076826760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddf81047e0 0x7fddf8108830 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009900 tx=0x7fdde402a410 comp rx=0 tx=0).stop 2024-09-10T08:44:53.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.332+0000 7fddfde00640 1 -- 172.21.15.89:0/2076826760 shutdown_connections 2024-09-10T08:44:53.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.332+0000 7fddfde00640 1 --2- 172.21.15.89:0/2076826760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fddf8108f60 0x7fddf8110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.334 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.332+0000 7fddfde00640 1 --2- 172.21.15.89:0/2076826760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddf81047e0 0x7fddf8108830 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.332+0000 7fddfde00640 1 --2- 172.21.15.89:0/2076826760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddf8103e30 0x7fddf8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.332+0000 7fddfde00640 1 -- 172.21.15.89:0/2076826760 >> 172.21.15.89:0/2076826760 conn(0x7fddf80ff910 msgr2=0x7fddf8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:53.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.332+0000 7fddfde00640 1 -- 172.21.15.89:0/2076826760 shutdown_connections 2024-09-10T08:44:53.335 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.332+0000 7fddfde00640 1 -- 172.21.15.89:0/2076826760 wait complete. 2024-09-10T08:44:53.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.333+0000 7fddfde00640 1 Processor -- start 2024-09-10T08:44:53.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.333+0000 7fddfde00640 1 -- start start 2024-09-10T08:44:53.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.334+0000 7fddfde00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddf8103e30 0x7fddf819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:53.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.334+0000 7fddfde00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddf81047e0 0x7fddf819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:53.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.334+0000 7fddfde00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fddf8108f60 0x7fddf819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:53.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.334+0000 7fddfde00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fddf8113fc0 con 0x7fddf8103e30 2024-09-10T08:44:53.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.334+0000 7fddfde00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fddf8113e40 con 0x7fddf81047e0 2024-09-10T08:44:53.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.334+0000 7fddfde00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fddf8114140 con 0x7fddf8108f60 2024-09-10T08:44:53.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.334+0000 7fddfca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddf8103e30 0x7fddf819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:53.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.334+0000 7fddf7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddf81047e0 0x7fddf819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:53.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.334+0000 7fddf7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddf81047e0 0x7fddf819a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57370/0 (socket says 172.21.15.89:57370) 2024-09-10T08:44:53.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.334+0000 7fddf7e00640 1 -- 172.21.15.89:0/1239127536 learned_addr learned my addr 172.21.15.89:0/1239127536 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:53.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.334+0000 7fddfca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddf8103e30 0x7fddf819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52686/0 (socket says 172.21.15.89:52686) 2024-09-10T08:44:53.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.334+0000 7fddfd400640 1 --2- 172.21.15.89:0/1239127536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fddf8108f60 0x7fddf819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:53.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.335+0000 7fddf7e00640 1 -- 172.21.15.89:0/1239127536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fddf8108f60 msgr2=0x7fddf819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:53.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.335+0000 7fddf7e00640 1 --2- 172.21.15.89:0/1239127536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fddf8108f60 0x7fddf819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.335+0000 7fddf7e00640 1 -- 172.21.15.89:0/1239127536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddf8103e30 msgr2=0x7fddf819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:53.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.335+0000 7fddf7e00640 1 --2- 172.21.15.89:0/1239127536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddf8103e30 0x7fddf819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.335+0000 7fddf7e00640 1 -- 172.21.15.89:0/1239127536 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf819f430 con 0x7fddf81047e0 2024-09-10T08:44:53.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.335+0000 7fddf7e00640 1 --2- 172.21.15.89:0/1239127536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddf81047e0 0x7fddf819a9c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdde4006010 tx=0x7fdde4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:53.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.335+0000 7fddf5600640 1 -- 172.21.15.89:0/1239127536 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde403f030 con 0x7fddf81047e0 2024-09-10T08:44:53.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.335+0000 7fddfde00640 1 -- 172.21.15.89:0/1239127536 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf819f6c0 con 0x7fddf81047e0 2024-09-10T08:44:53.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.335+0000 7fddf5600640 1 -- 172.21.15.89:0/1239127536 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdde4004440 con 0x7fddf81047e0 2024-09-10T08:44:53.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.335+0000 7fddf5600640 1 -- 172.21.15.89:0/1239127536 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4035070 con 0x7fddf81047e0 2024-09-10T08:44:53.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.336+0000 7fddfde00640 1 -- 172.21.15.89:0/1239127536 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf81a7000 con 0x7fddf81047e0 2024-09-10T08:44:53.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.339+0000 7fde0207b740 1 Processor -- start 2024-09-10T08:44:53.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.339+0000 7fde0207b740 1 -- start start 2024-09-10T08:44:53.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.339+0000 7fddf5600640 1 -- 172.21.15.89:0/1239127536 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdde4042030 con 0x7fddf81047e0 2024-09-10T08:44:53.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.339+0000 7fde0207b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557660510c60 0x557660511040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:53.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.339+0000 7fde0207b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557660511580 0x55766051b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:53.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.339+0000 7fde0207b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55766051be40 0x55766051e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:53.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.339+0000 7fde0207b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557660477530 con 0x557660511580 2024-09-10T08:44:53.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.339+0000 7fde0207b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55766041ecc0 con 0x557660510c60 2024-09-10T08:44:53.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.339+0000 7fde0207b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5576604734f0 con 0x55766051be40 2024-09-10T08:44:53.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.340+0000 7fddfd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557660510c60 0x557660511040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:53.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.340+0000 7fddfd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557660510c60 0x557660511040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57386/0 (socket says 172.21.15.89:57386) 2024-09-10T08:44:53.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.340+0000 7fddfd400640 1 -- 172.21.15.89:0/1388027762 learned_addr learned my addr 172.21.15.89:0/1388027762 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:53.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.340+0000 7fddf7e00640 1 --2- 172.21.15.89:0/1388027762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55766051be40 0x55766051e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:53.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.340+0000 7fddfca00640 1 --2- 172.21.15.89:0/1388027762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557660511580 0x55766051b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:53.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.340+0000 7fddfd400640 1 -- 172.21.15.89:0/1388027762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55766051be40 msgr2=0x55766051e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:53.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.340+0000 7fddfd400640 1 --2- 172.21.15.89:0/1388027762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55766051be40 0x55766051e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.340+0000 7fddfd400640 1 -- 172.21.15.89:0/1388027762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557660511580 msgr2=0x55766051b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:53.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.340+0000 7fddfd400640 1 --2- 172.21.15.89:0/1388027762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557660511580 0x55766051b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.340+0000 7fddfd400640 1 -- 172.21.15.89:0/1388027762 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576601e8870 con 0x557660510c60 2024-09-10T08:44:53.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.340+0000 7fddf7e00640 1 --2- 172.21.15.89:0/1388027762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55766051be40 0x55766051e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:53.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.340+0000 7fddfd400640 1 --2- 172.21.15.89:0/1388027762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557660510c60 0x557660511040 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009f50 tx=0x7fddf00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2676b939df1d741 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:53.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.341+0000 7fdde9c00640 1 -- 172.21.15.89:0/1388027762 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0004b20 con 0x557660510c60 2024-09-10T08:44:53.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.341+0000 7fdde9c00640 1 -- 172.21.15.89:0/1388027762 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddf0004cc0 con 0x557660510c60 2024-09-10T08:44:53.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.341+0000 7fdde9c00640 1 -- 172.21.15.89:0/1388027762 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0005020 con 0x557660510c60 2024-09-10T08:44:53.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.341+0000 7fde0207b740 1 -- 172.21.15.89:0/1388027762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557660510c60 msgr2=0x557660511040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:53.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.341+0000 7fde0207b740 1 --2- 172.21.15.89:0/1388027762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557660510c60 0x557660511040 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009f50 tx=0x7fddf00118d0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.342+0000 7fde0207b740 1 -- 172.21.15.89:0/1388027762 shutdown_connections 2024-09-10T08:44:53.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.342+0000 7fde0207b740 1 --2- 172.21.15.89:0/1388027762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55766051be40 0x55766051e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.342+0000 7fde0207b740 1 --2- 172.21.15.89:0/1388027762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557660511580 0x55766051b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.342+0000 7fde0207b740 1 --2- 172.21.15.89:0/1388027762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557660510c60 0x557660511040 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.342+0000 7fde0207b740 1 -- 172.21.15.89:0/1388027762 >> 172.21.15.89:0/1388027762 conn(0x55766050d1f0 msgr2=0x55766050d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:53.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.342+0000 7fde0207b740 1 -- 172.21.15.89:0/1388027762 shutdown_connections 2024-09-10T08:44:53.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.342+0000 7fde0207b740 1 -- 172.21.15.89:0/1388027762 wait complete. 2024-09-10T08:44:53.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.342+0000 7fddf5600640 1 --2- 172.21.15.89:0/1239127536 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdde00a2a90 0x7fdde00a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:53.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.342+0000 7fddf5600640 1 -- 172.21.15.89:0/1239127536 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fdde40ef750 con 0x7fddf81047e0 2024-09-10T08:44:53.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.342+0000 7fde0207b740 1 -- start start 2024-09-10T08:44:53.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.342+0000 7fde0207b740 1 Processor -- start 2024-09-10T08:44:53.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.343+0000 7fddfca00640 1 --2- 172.21.15.89:0/1239127536 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdde00a2a90 0x7fdde00a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:53.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.343+0000 7fde0207b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557660510c60 0x557660518d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:53.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.343+0000 7fde0207b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557660511580 0x557660519270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:53.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.343+0000 7fde0207b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55766051be40 0x5576605197b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:53.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.343+0000 7fde0207b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55766051eab0 con 0x557660511580 2024-09-10T08:44:53.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.343+0000 7fde0207b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5576604320a0 con 0x55766051be40 2024-09-10T08:44:53.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.343+0000 7fde0207b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55766051ec30 con 0x557660510c60 2024-09-10T08:44:53.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.343+0000 7fde0207b740 10 client.0 _ll_register_callbacks cb 0x557660430880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:53.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.343+0000 7fddf7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557660511580 0x557660519270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:53.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.343+0000 7fddf7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557660511580 0x557660519270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52704/0 (socket says 172.21.15.89:52704) 2024-09-10T08:44:53.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.343+0000 7fddf7e00640 1 -- 172.21.15.89:0/2217112891 learned_addr learned my addr 172.21.15.89:0/2217112891 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:53.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.343+0000 7fddfd400640 1 --2- 172.21.15.89:0/2217112891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55766051be40 0x5576605197b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:53.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.344+0000 7fddfca00640 1 --2- 172.21.15.89:0/2217112891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557660510c60 0x557660518d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:53.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.344+0000 7fddf7e00640 1 -- 172.21.15.89:0/2217112891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557660510c60 msgr2=0x557660518d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:53.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.344+0000 7fddf7e00640 1 --2- 172.21.15.89:0/2217112891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557660510c60 0x557660518d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.344+0000 7fddf7e00640 1 -- 172.21.15.89:0/2217112891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55766051be40 msgr2=0x5576605197b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:53.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.344+0000 7fddf7e00640 1 --2- 172.21.15.89:0/2217112891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55766051be40 0x5576605197b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.344+0000 7fddf7e00640 1 -- 172.21.15.89:0/2217112891 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55766042f4b0 con 0x557660511580 2024-09-10T08:44:53.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.344+0000 7fddfca00640 1 --2- 172.21.15.89:0/2217112891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557660510c60 0x557660518d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:53.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.344+0000 7fddfd400640 1 --2- 172.21.15.89:0/2217112891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55766051be40 0x5576605197b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:53.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.344+0000 7fddfca00640 1 --2- 172.21.15.89:0/1239127536 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdde00a2a90 0x7fdde00a4f50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55766050e050 tx=0x7fddec009f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:53.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.344+0000 7fddf7e00640 1 --2- 172.21.15.89:0/2217112891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557660511580 0x557660519270 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdde40b80f0 tx=0x7fdde40b7f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc494a1c2e1f7430 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:53.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.344+0000 7fddde000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:44:53.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.344+0000 7fddde000640 1 -- 172.21.15.89:0/2217112891 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4049070 con 0x557660511580 2024-09-10T08:44:53.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.344+0000 7fddde000640 1 -- 172.21.15.89:0/2217112891 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fdde40ef440 con 0x557660511580 2024-09-10T08:44:53.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.344+0000 7fde0207b740 1 -- 172.21.15.89:0/2217112891 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddf0004cc0 con 0x557660511580 2024-09-10T08:44:53.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.345+0000 7fddde000640 1 -- 172.21.15.89:0/2217112891 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdde40bbc90 con 0x557660511580 2024-09-10T08:44:53.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.345+0000 7fde0207b740 10 client.16305 fetch_fsmap learned FSMap version 29 2024-09-10T08:44:53.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.345+0000 7fde0207b740 1 -- 172.21.15.89:0/2217112891 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55766052aac0 con 0x557660511580 2024-09-10T08:44:53.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.345+0000 7fddde000640 1 -- 172.21.15.89:0/2217112891 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddf0004cc0 con 0x557660511580 2024-09-10T08:44:53.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.345+0000 7fddde000640 1 -- 172.21.15.89:0/2217112891 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fdde40bbef0 con 0x557660511580 2024-09-10T08:44:53.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.345+0000 7fde0207b740 10 client.16305 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:44:53.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.345+0000 7fde0207b740 10 client.16305 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:44:53.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.345+0000 7fde0207b740 20 client.16305 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:53.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.345+0000 7fde0207b740 1 --2- 172.21.15.89:0/2217112891 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55766052be60 0x55766052e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:53.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.345+0000 7fde0207b740 4 client.16305 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:44:53.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.345+0000 7fde0207b740 1 -- 172.21.15.89:0/2217112891 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557660484aa0 con 0x55766052be60 2024-09-10T08:44:53.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.345+0000 7fddfca00640 1 --2- 172.21.15.89:0/2217112891 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55766052be60 0x55766052e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:53.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.346+0000 7fddfca00640 1 --2- 172.21.15.89:0/2217112891 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55766052be60 0x55766052e290 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8de9dc8b31773e4a server_cookie=338f727c4f0d85ad in_seq=0 out_seq=0 2024-09-10T08:44:53.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.346+0000 7fddde000640 10 client.16305 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:53.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.348+0000 7fddde000640 1 -- 172.21.15.89:0/2217112891 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557660484aa0 con 0x55766052be60 2024-09-10T08:44:53.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.348+0000 7fddde000640 10 client.16305 handle_command_reply: tid=0 2024-09-10T08:44:53.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.348+0000 7fde0207b740 1 client.16305 shutdown 2024-09-10T08:44:53.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.348+0000 7fde0207b740 1 -- 172.21.15.89:0/2217112891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557660511580 msgr2=0x557660519270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:53.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.348+0000 7fde0207b740 1 --2- 172.21.15.89:0/2217112891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557660511580 0x557660519270 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdde40b80f0 tx=0x7fdde40b7f30 comp rx=0 tx=0).stop 2024-09-10T08:44:53.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.348+0000 7fde0207b740 1 -- 172.21.15.89:0/2217112891 shutdown_connections 2024-09-10T08:44:53.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.348+0000 7fde0207b740 1 --2- 172.21.15.89:0/2217112891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55766051be40 0x5576605197b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.348+0000 7fde0207b740 1 --2- 172.21.15.89:0/2217112891 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55766052be60 0x55766052e290 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.348+0000 7fde0207b740 1 --2- 172.21.15.89:0/2217112891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557660511580 0x557660519270 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.348+0000 7fde0207b740 1 --2- 172.21.15.89:0/2217112891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557660510c60 0x557660518d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.348+0000 7fde0207b740 1 -- 172.21.15.89:0/2217112891 >> 172.21.15.89:0/2217112891 conn(0x55766050d1f0 msgr2=0x557660512f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:53.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.348+0000 7fddde000640 0 client.16305 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:53.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.348+0000 7fde0207b740 1 -- 172.21.15.89:0/2217112891 shutdown_connections 2024-09-10T08:44:53.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.348+0000 7fde0207b740 1 -- 172.21.15.89:0/2217112891 wait complete. 2024-09-10T08:44:53.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.348+0000 7fde0207b740 20 client.16305 trim_cache size 0 max 16384 2024-09-10T08:44:53.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.366+0000 7fde0207b740 1 Processor -- start 2024-09-10T08:44:53.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.366+0000 7fde0207b740 1 -- start start 2024-09-10T08:44:53.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.366+0000 7fde0207b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557660510c60 0x557660533b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:53.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.366+0000 7fde0207b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557660511580 0x5576605340a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:53.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.366+0000 7fde0207b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55766051be40 0x557660515fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:53.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.366+0000 7fde0207b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fddf81b3630 con 0x557660510c60 2024-09-10T08:44:53.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.366+0000 7fde0207b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fddf81b0f00 con 0x55766051be40 2024-09-10T08:44:53.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.366+0000 7fde0207b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fddf81aed50 con 0x557660511580 2024-09-10T08:44:53.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.366+0000 7fddfca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557660510c60 0x557660533b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:53.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.366+0000 7fddf7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557660511580 0x5576605340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:53.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.367+0000 7fddfca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557660510c60 0x557660533b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52720/0 (socket says 172.21.15.89:52720) 2024-09-10T08:44:53.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.367+0000 7fddfca00640 1 -- 172.21.15.89:0/3941933742 learned_addr learned my addr 172.21.15.89:0/3941933742 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:53.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.367+0000 7fddfd400640 1 --2- 172.21.15.89:0/3941933742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55766051be40 0x557660515fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:53.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.367+0000 7fddfca00640 1 -- 172.21.15.89:0/3941933742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557660511580 msgr2=0x5576605340a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:53.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.367+0000 7fddfca00640 1 --2- 172.21.15.89:0/3941933742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557660511580 0x5576605340a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.367+0000 7fddfca00640 1 -- 172.21.15.89:0/3941933742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55766051be40 msgr2=0x557660515fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:53.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.367+0000 7fddfca00640 1 --2- 172.21.15.89:0/3941933742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55766051be40 0x557660515fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.367+0000 7fddfca00640 1 -- 172.21.15.89:0/3941933742 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55766051b6a0 con 0x557660510c60 2024-09-10T08:44:53.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.367+0000 7fddfca00640 1 --2- 172.21.15.89:0/3941933742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557660510c60 0x557660533b60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x55766052eb00 tx=0x7fddec01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ef6a14b07fddcc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:53.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.367+0000 7fdde9200640 1 -- 172.21.15.89:0/3941933742 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec037070 con 0x557660510c60 2024-09-10T08:44:53.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.368+0000 7fdde9200640 1 -- 172.21.15.89:0/3941933742 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddec0328f0 con 0x557660510c60 2024-09-10T08:44:53.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.368+0000 7fdde9200640 1 -- 172.21.15.89:0/3941933742 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec02a910 con 0x557660510c60 2024-09-10T08:44:53.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.368+0000 7fde0207b740 1 -- 172.21.15.89:0/3941933742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557660510c60 msgr2=0x557660533b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:53.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.368+0000 7fde0207b740 1 --2- 172.21.15.89:0/3941933742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557660510c60 0x557660533b60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x55766052eb00 tx=0x7fddec01f000 comp rx=0 tx=0).stop 2024-09-10T08:44:53.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.368+0000 7fde0207b740 1 -- 172.21.15.89:0/3941933742 shutdown_connections 2024-09-10T08:44:53.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.368+0000 7fde0207b740 1 --2- 172.21.15.89:0/3941933742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55766051be40 0x557660515fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.368+0000 7fde0207b740 1 --2- 172.21.15.89:0/3941933742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557660511580 0x5576605340a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.368+0000 7fde0207b740 1 --2- 172.21.15.89:0/3941933742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557660510c60 0x557660533b60 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.368+0000 7fde0207b740 1 -- 172.21.15.89:0/3941933742 >> 172.21.15.89:0/3941933742 conn(0x55766050d1f0 msgr2=0x55766051adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:53.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.368+0000 7fde0207b740 1 -- 172.21.15.89:0/3941933742 shutdown_connections 2024-09-10T08:44:53.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.368+0000 7fde0207b740 1 -- 172.21.15.89:0/3941933742 wait complete. 2024-09-10T08:44:53.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.369+0000 7fde0207b740 1 -- start start 2024-09-10T08:44:53.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.369+0000 7fde0207b740 1 Processor -- start 2024-09-10T08:44:53.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.369+0000 7fde0207b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557660510c60 0x557660523e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:53.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.369+0000 7fde0207b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557660511580 0x5576605243c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:53.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.369+0000 7fde0207b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55766051be40 0x557660520e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:53.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.369+0000 7fde0207b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fddf81af3c0 con 0x55766051be40 2024-09-10T08:44:53.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.369+0000 7fde0207b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fddf81b0830 con 0x557660510c60 2024-09-10T08:44:53.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.369+0000 7fde0207b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fddf81afa90 con 0x557660511580 2024-09-10T08:44:53.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.369+0000 7fde0207b740 10 client.0 _ll_register_callbacks cb 0x7fdde4005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:53.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.369+0000 7fddfca00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557660510c60 0x557660523e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:53.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.370+0000 7fddfca00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557660510c60 0x557660523e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57406/0 (socket says 172.21.15.89:57406) 2024-09-10T08:44:53.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.370+0000 7fddfca00640 1 -- 172.21.15.89:0/2518916067 learned_addr learned my addr 172.21.15.89:0/2518916067 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:53.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.370+0000 7fddfd400640 1 --2- 172.21.15.89:0/2518916067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55766051be40 0x557660520e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:53.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.370+0000 7fddf7e00640 1 --2- 172.21.15.89:0/2518916067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557660511580 0x5576605243c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:53.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.370+0000 7fddfca00640 1 -- 172.21.15.89:0/2518916067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557660511580 msgr2=0x5576605243c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:53.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.370+0000 7fddfca00640 1 --2- 172.21.15.89:0/2518916067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557660511580 0x5576605243c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.370+0000 7fddfca00640 1 -- 172.21.15.89:0/2518916067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55766051be40 msgr2=0x557660520e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:53.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.370+0000 7fddfca00640 1 --2- 172.21.15.89:0/2518916067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55766051be40 0x557660520e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.370+0000 7fddfca00640 1 -- 172.21.15.89:0/2518916067 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557660521490 con 0x557660510c60 2024-09-10T08:44:53.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.370+0000 7fddfd400640 1 --2- 172.21.15.89:0/2518916067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55766051be40 0x557660520e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:53.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.370+0000 7fddf7e00640 1 --2- 172.21.15.89:0/2518916067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557660511580 0x5576605243c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:53.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.370+0000 7fddfca00640 1 --2- 172.21.15.89:0/2518916067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557660510c60 0x557660523e80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fddec02a450 tx=0x7fddec02a540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a13f4799fe02a34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:53.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.370+0000 7fddde000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:44:53.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.370+0000 7fddde000640 1 -- 172.21.15.89:0/2518916067 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec037070 con 0x557660510c60 2024-09-10T08:44:53.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.371+0000 7fddde000640 1 -- 172.21.15.89:0/2518916067 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fddec032aa0 con 0x557660510c60 2024-09-10T08:44:53.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.371+0000 7fde0207b740 1 -- 172.21.15.89:0/2518916067 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddec0328f0 con 0x557660510c60 2024-09-10T08:44:53.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.371+0000 7fddde000640 1 -- 172.21.15.89:0/2518916067 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fddec032ca0 con 0x557660510c60 2024-09-10T08:44:53.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.371+0000 7fde0207b740 10 client.26357 fetch_fsmap learned FSMap version 29 2024-09-10T08:44:53.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.371+0000 7fde0207b740 1 -- 172.21.15.89:0/2518916067 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557660597470 con 0x557660510c60 2024-09-10T08:44:53.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.371+0000 7fddde000640 1 -- 172.21.15.89:0/2518916067 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddec0328f0 con 0x557660510c60 2024-09-10T08:44:53.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.371+0000 7fddde000640 1 -- 172.21.15.89:0/2518916067 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fddec02ac40 con 0x557660510c60 2024-09-10T08:44:53.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.371+0000 7fde0207b740 10 client.26357 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:44:53.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.371+0000 7fde0207b740 10 client.26357 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:44:53.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.371+0000 7fde0207b740 20 client.26357 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:53.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.371+0000 7fde0207b740 1 --2- 172.21.15.89:0/2518916067 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x557660597b00 0x557660599f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:53.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.371+0000 7fde0207b740 4 client.26357 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a577fe06", "await_for": 45.0}] 2024-09-10T08:44:53.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.371+0000 7fde0207b740 1 -- 172.21.15.89:0/2518916067 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a577fe06", "await_for": 45.0}) -- 0x55766047efb0 con 0x557660597b00 2024-09-10T08:44:53.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.372+0000 7fddf7e00640 1 --2- 172.21.15.89:0/2518916067 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x557660597b00 0x557660599f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:53.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.372+0000 7fddf7e00640 1 --2- 172.21.15.89:0/2518916067 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x557660597b00 0x557660599f20 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5967455530562d85 server_cookie=a43070f0bfce6ca0 in_seq=0 out_seq=0 2024-09-10T08:44:53.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.372+0000 7fddde000640 10 client.26357 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:53.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.963+0000 7fddde000640 1 -- 172.21.15.89:0/2518916067 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55766047efb0 con 0x557660597b00 2024-09-10T08:44:53.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.963+0000 7fddde000640 10 client.26357 handle_command_reply: tid=0 2024-09-10T08:44:53.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.963+0000 7fde0207b740 1 client.26357 shutdown 2024-09-10T08:44:53.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.963+0000 7fde0207b740 1 -- 172.21.15.89:0/2518916067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557660510c60 msgr2=0x557660523e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:53.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.963+0000 7fde0207b740 1 --2- 172.21.15.89:0/2518916067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557660510c60 0x557660523e80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fddec02a450 tx=0x7fddec02a540 comp rx=0 tx=0).stop 2024-09-10T08:44:53.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.963+0000 7fde0207b740 1 -- 172.21.15.89:0/2518916067 shutdown_connections 2024-09-10T08:44:53.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.963+0000 7fde0207b740 1 --2- 172.21.15.89:0/2518916067 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x557660597b00 0x557660599f20 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.963+0000 7fde0207b740 1 --2- 172.21.15.89:0/2518916067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55766051be40 0x557660520e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.963+0000 7fddde000640 0 client.26357 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:53.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.963+0000 7fde0207b740 1 --2- 172.21.15.89:0/2518916067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557660511580 0x5576605243c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.963+0000 7fde0207b740 1 --2- 172.21.15.89:0/2518916067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557660510c60 0x557660523e80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.963+0000 7fde0207b740 1 -- 172.21.15.89:0/2518916067 >> 172.21.15.89:0/2518916067 conn(0x55766050d1f0 msgr2=0x55766051a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:53.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.963+0000 7fde0207b740 1 -- 172.21.15.89:0/2518916067 shutdown_connections 2024-09-10T08:44:53.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.963+0000 7fde0207b740 1 -- 172.21.15.89:0/2518916067 wait complete. 2024-09-10T08:44:53.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.964+0000 7fde0207b740 20 client.26357 trim_cache size 0 max 16384 2024-09-10T08:44:53.966 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:44:53.966 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 29, 2024-09-10T08:44:53.966 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24554, 2024-09-10T08:44:53.966 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 10, 2024-09-10T08:44:53.966 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": { 2024-09-10T08:44:53.966 INFO:teuthology.orchestra.run.smithi089.stdout: "a577fe06": { 2024-09-10T08:44:53.966 INFO:teuthology.orchestra.run.smithi089.stdout: "version": 10, 2024-09-10T08:44:53.966 INFO:teuthology.orchestra.run.smithi089.stdout: "age_ref": 0.0, 2024-09-10T08:44:53.967 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:44:53.967 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCED", 2024-09-10T08:44:53.967 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:44:53.967 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:44:53.967 INFO:teuthology.orchestra.run.smithi089.stdout: "timeout": 90.0, 2024-09-10T08:44:53.967 INFO:teuthology.orchestra.run.smithi089.stdout: "expiration": 142.8, 2024-09-10T08:44:53.967 INFO:teuthology.orchestra.run.smithi089.stdout: "members": { 2024-09-10T08:44:53.967 INFO:teuthology.orchestra.run.smithi089.stdout: "file:/": { 2024-09-10T08:44:53.967 INFO:teuthology.orchestra.run.smithi089.stdout: "excluded": false, 2024-09-10T08:44:53.967 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:44:53.967 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCED", 2024-09-10T08:44:53.967 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:44:53.967 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:44:53.967 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:44:53.967 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:44:53.968 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:44:53.968 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:44:53.968 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:44:53.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.965+0000 7fddfde00640 1 -- 172.21.15.89:0/1239127536 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdde00a2a90 msgr2=0x7fdde00a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:53.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.965+0000 7fddfde00640 1 --2- 172.21.15.89:0/1239127536 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdde00a2a90 0x7fdde00a4f50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55766050e050 tx=0x7fddec009f80 comp rx=0 tx=0).stop 2024-09-10T08:44:53.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.965+0000 7fddfde00640 1 -- 172.21.15.89:0/1239127536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddf81047e0 msgr2=0x7fddf819a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:53.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.965+0000 7fddfde00640 1 --2- 172.21.15.89:0/1239127536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddf81047e0 0x7fddf819a9c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdde4006010 tx=0x7fdde4004040 comp rx=0 tx=0).stop 2024-09-10T08:44:53.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.965+0000 7fddfde00640 1 -- 172.21.15.89:0/1239127536 shutdown_connections 2024-09-10T08:44:53.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.965+0000 7fddfde00640 1 --2- 172.21.15.89:0/1239127536 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fdde00a2a90 0x7fdde00a4f50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.965+0000 7fddfde00640 1 --2- 172.21.15.89:0/1239127536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fddf8108f60 0x7fddf819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.965+0000 7fddfde00640 1 --2- 172.21.15.89:0/1239127536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddf81047e0 0x7fddf819a9c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.965+0000 7fddfde00640 1 --2- 172.21.15.89:0/1239127536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fddf8103e30 0x7fddf819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.965+0000 7fddfde00640 1 -- 172.21.15.89:0/1239127536 >> 172.21.15.89:0/1239127536 conn(0x7fddf80ff910 msgr2=0x7fddf8100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:53.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.965+0000 7fddfde00640 1 -- 172.21.15.89:0/1239127536 shutdown_connections 2024-09-10T08:44:53.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:53.966+0000 7fddfde00640 1 -- 172.21.15.89:0/1239127536 wait complete. 2024-09-10T08:44:55.048 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a577fe06', quiesce took 86.7 seconds. Will release after: -63.95 2024-09-10T08:44:55.048 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:44:55.109 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:54 smithi089 ceph-mon[30757]: pgmap v542: 129 pgs: 129 active+clean; 300 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 568 op/s 2024-09-10T08:44:55.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:54 smithi149 ceph-mon[36167]: pgmap v542: 129 pgs: 129 active+clean; 300 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 568 op/s 2024-09-10T08:44:55.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:54 smithi111 ceph-mon[35220]: pgmap v542: 129 pgs: 129 active+clean; 300 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 568 op/s 2024-09-10T08:44:55.523 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:56.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.350+0000 7f3a0ca00640 1 -- 172.21.15.89:0/1284253360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a081049a0 msgr2=0x7f3a081089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:56.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.350+0000 7f3a0ca00640 1 --2- 172.21.15.89:0/1284253360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a081049a0 0x7f3a081089f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f39f0009900 tx=0x7f39f002a410 comp rx=0 tx=0).stop 2024-09-10T08:44:56.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.350+0000 7f3a0ca00640 1 -- 172.21.15.89:0/1284253360 shutdown_connections 2024-09-10T08:44:56.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.350+0000 7f3a0ca00640 1 --2- 172.21.15.89:0/1284253360 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a08109120 0x7f3a08110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:56.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.350+0000 7f3a0ca00640 1 --2- 172.21.15.89:0/1284253360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a081049a0 0x7f3a081089f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:56.352 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.350+0000 7f3a0ca00640 1 --2- 172.21.15.89:0/1284253360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a08103ff0 0x7f3a081043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:56.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.350+0000 7f3a0ca00640 1 -- 172.21.15.89:0/1284253360 >> 172.21.15.89:0/1284253360 conn(0x7f3a080ffa90 msgr2=0x7f3a08101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:56.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.350+0000 7f3a0ca00640 1 -- 172.21.15.89:0/1284253360 shutdown_connections 2024-09-10T08:44:56.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.350+0000 7f3a0ca00640 1 -- 172.21.15.89:0/1284253360 wait complete. 2024-09-10T08:44:56.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.351+0000 7f3a0ca00640 1 Processor -- start 2024-09-10T08:44:56.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.351+0000 7f3a0ca00640 1 -- start start 2024-09-10T08:44:56.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.352+0000 7f3a0ca00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a08103ff0 0x7f3a0819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:56.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.352+0000 7f3a0ca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a081049a0 0x7f3a0819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:56.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.352+0000 7f3a0ca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a08109120 0x7f3a0819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:56.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.352+0000 7f3a0ca00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3a08114190 con 0x7f3a08109120 2024-09-10T08:44:56.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.352+0000 7f3a0ca00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3a08114010 con 0x7f3a08103ff0 2024-09-10T08:44:56.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.352+0000 7f3a0ca00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3a08114310 con 0x7f3a081049a0 2024-09-10T08:44:56.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.352+0000 7f3a05e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a08103ff0 0x7f3a0819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:56.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.352+0000 7f3a05e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a08103ff0 0x7f3a0819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57432/0 (socket says 172.21.15.89:57432) 2024-09-10T08:44:56.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.352+0000 7f3a05e00640 1 -- 172.21.15.89:0/3764511930 learned_addr learned my addr 172.21.15.89:0/3764511930 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:56.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.352+0000 7f39ffe00640 1 --2- 172.21.15.89:0/3764511930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a081049a0 0x7f3a0819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:56.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.352+0000 7f3a06800640 1 --2- 172.21.15.89:0/3764511930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a08109120 0x7f3a0819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:56.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.353+0000 7f39ffe00640 1 -- 172.21.15.89:0/3764511930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a08103ff0 msgr2=0x7f3a0819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:56.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.353+0000 7f39ffe00640 1 --2- 172.21.15.89:0/3764511930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a08103ff0 0x7f3a0819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:56.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.353+0000 7f39ffe00640 1 -- 172.21.15.89:0/3764511930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a08109120 msgr2=0x7f3a0819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:56.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.353+0000 7f39ffe00640 1 --2- 172.21.15.89:0/3764511930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a08109120 0x7f3a0819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:56.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.353+0000 7f39ffe00640 1 -- 172.21.15.89:0/3764511930 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a0819f620 con 0x7f3a081049a0 2024-09-10T08:44:56.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.354+0000 7f39ffe00640 1 --2- 172.21.15.89:0/3764511930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a081049a0 0x7f3a0819abb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f39f002a930 tx=0x7f39f0003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.354+0000 7f39fea00640 1 -- 172.21.15.89:0/3764511930 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f003f070 con 0x7f3a081049a0 2024-09-10T08:44:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.354+0000 7f39fea00640 1 -- 172.21.15.89:0/3764511930 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39f0031c90 con 0x7f3a081049a0 2024-09-10T08:44:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.354+0000 7f39fea00640 1 -- 172.21.15.89:0/3764511930 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f0035030 con 0x7f3a081049a0 2024-09-10T08:44:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.354+0000 7f3a0ca00640 1 -- 172.21.15.89:0/3764511930 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a0819f8b0 con 0x7f3a081049a0 2024-09-10T08:44:56.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.354+0000 7f3a0ca00640 1 -- 172.21.15.89:0/3764511930 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a0819fd40 con 0x7f3a081049a0 2024-09-10T08:44:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.356+0000 7f3a0ca00640 1 -- 172.21.15.89:0/3764511930 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a081043d0 con 0x7f3a081049a0 2024-09-10T08:44:56.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.357+0000 7f39fea00640 1 -- 172.21.15.89:0/3764511930 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f39f0042070 con 0x7f3a081049a0 2024-09-10T08:44:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.359+0000 7f39fea00640 1 --2- 172.21.15.89:0/3764511930 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f39ec0a2c00 0x7f39ec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.359+0000 7f39fea00640 1 -- 172.21.15.89:0/3764511930 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f39f00efcf0 con 0x7f3a081049a0 2024-09-10T08:44:56.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.360+0000 7f3a05e00640 1 --2- 172.21.15.89:0/3764511930 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f39ec0a2c00 0x7f39ec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:56.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.360+0000 7f3a05e00640 1 --2- 172.21.15.89:0/3764511930 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f39ec0a2c00 0x7f39ec0a50c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f39f8009c90 tx=0x7f39f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:56.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.362+0000 7f39fea00640 1 -- 172.21.15.89:0/3764511930 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39f0049020 con 0x7f3a081049a0 2024-09-10T08:44:56.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.637+0000 7f3a0ca00640 1 -- 172.21.15.89:0/3764511930 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a0810c360 con 0x7f3a081049a0 2024-09-10T08:44:56.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.638+0000 7f39fea00640 1 -- 172.21.15.89:0/3764511930 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11919 (secure 0 0 0) 0x7f39f00bc820 con 0x7f3a081049a0 2024-09-10T08:44:56.639 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:44:56.640 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"btime":"2024-09-10T08:44:40:778845+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:44:40.174532+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24496,24518,24554]},"id":1}]} 2024-09-10T08:44:56.640 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 29 2024-09-10T08:44:56.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.641+0000 7f3a0ca00640 1 -- 172.21.15.89:0/3764511930 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f39ec0a2c00 msgr2=0x7f39ec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:56.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.641+0000 7f3a0ca00640 1 --2- 172.21.15.89:0/3764511930 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f39ec0a2c00 0x7f39ec0a50c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f39f8009c90 tx=0x7f39f8005f90 comp rx=0 tx=0).stop 2024-09-10T08:44:56.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.641+0000 7f3a0ca00640 1 -- 172.21.15.89:0/3764511930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a081049a0 msgr2=0x7f3a0819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:56.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.641+0000 7f3a0ca00640 1 --2- 172.21.15.89:0/3764511930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a081049a0 0x7f3a0819abb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f39f002a930 tx=0x7f39f0003b20 comp rx=0 tx=0).stop 2024-09-10T08:44:56.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.641+0000 7f3a0ca00640 1 -- 172.21.15.89:0/3764511930 shutdown_connections 2024-09-10T08:44:56.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.641+0000 7f3a0ca00640 1 --2- 172.21.15.89:0/3764511930 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f39ec0a2c00 0x7f39ec0a50c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:56.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.641+0000 7f3a0ca00640 1 --2- 172.21.15.89:0/3764511930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a08109120 0x7f3a0819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:56.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.641+0000 7f3a0ca00640 1 --2- 172.21.15.89:0/3764511930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a081049a0 0x7f3a0819abb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:56.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.641+0000 7f3a0ca00640 1 --2- 172.21.15.89:0/3764511930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a08103ff0 0x7f3a0819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:56.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.641+0000 7f3a0ca00640 1 -- 172.21.15.89:0/3764511930 >> 172.21.15.89:0/3764511930 conn(0x7f3a080ffa90 msgr2=0x7f3a08100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:56.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.642+0000 7f3a0ca00640 1 -- 172.21.15.89:0/3764511930 shutdown_connections 2024-09-10T08:44:56.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:56.642+0000 7f3a0ca00640 1 -- 172.21.15.89:0/3764511930 wait complete. 2024-09-10T08:44:57.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:56 smithi089 ceph-mon[30757]: pgmap v543: 129 pgs: 129 active+clean; 298 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 823 op/s 2024-09-10T08:44:57.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:56 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3764511930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:57.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:56 smithi149 ceph-mon[36167]: pgmap v543: 129 pgs: 129 active+clean; 298 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 823 op/s 2024-09-10T08:44:57.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:56 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3764511930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:57.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:56 smithi111 ceph-mon[35220]: pgmap v543: 129 pgs: 129 active+clean; 298 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 823 op/s 2024-09-10T08:44:57.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:56 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3764511930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:57.587 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24554 quiesce db --set-id a577fe06 --release --await' 2024-09-10T08:44:57.587 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24554 quiesce db --set-id a577fe06 --release --await 2024-09-10T08:44:58.042 INFO:tasks.workunit.client.0.smithi089.stdout: 87 5886 0 5043 0 5573 0 2024-09-10T08:44:58.052 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:58.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.658+0000 7fadb7e00640 1 -- 172.21.15.89:0/2530003214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadb8103e30 msgr2=0x7fadb8104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:58.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.658+0000 7fadb7e00640 1 --2- 172.21.15.89:0/2530003214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadb8103e30 0x7fadb8104210 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fada8009900 tx=0x7fada802a410 comp rx=0 tx=0).stop 2024-09-10T08:44:58.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.658+0000 7fadb7e00640 1 -- 172.21.15.89:0/2530003214 shutdown_connections 2024-09-10T08:44:58.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.658+0000 7fadb7e00640 1 --2- 172.21.15.89:0/2530003214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadb8108f60 0x7fadb8110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.658+0000 7fadb7e00640 1 --2- 172.21.15.89:0/2530003214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadb81047e0 0x7fadb8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.658+0000 7fadb7e00640 1 --2- 172.21.15.89:0/2530003214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadb8103e30 0x7fadb8104210 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.658+0000 7fadb7e00640 1 -- 172.21.15.89:0/2530003214 >> 172.21.15.89:0/2530003214 conn(0x7fadb80ff910 msgr2=0x7fadb8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:58.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.658+0000 7fadb7e00640 1 -- 172.21.15.89:0/2530003214 shutdown_connections 2024-09-10T08:44:58.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.658+0000 7fadb7e00640 1 -- 172.21.15.89:0/2530003214 wait complete. 2024-09-10T08:44:58.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.660+0000 7fadb7e00640 1 Processor -- start 2024-09-10T08:44:58.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.660+0000 7fadb7e00640 1 -- start start 2024-09-10T08:44:58.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.661+0000 7fadb7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadb8103e30 0x7fadb819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.661+0000 7fadb7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadb81047e0 0x7fadb819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.661+0000 7fadb7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadb8108f60 0x7fadb819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.661+0000 7fadb7e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8114010 con 0x7fadb8103e30 2024-09-10T08:44:58.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.661+0000 7fadb7e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8113e90 con 0x7fadb8108f60 2024-09-10T08:44:58.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.661+0000 7fadb7e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8114190 con 0x7fadb81047e0 2024-09-10T08:44:58.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.661+0000 7fadb6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadb8103e30 0x7fadb819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.661+0000 7fadb6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadb8103e30 0x7fadb819a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52766/0 (socket says 172.21.15.89:52766) 2024-09-10T08:44:58.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.661+0000 7fadb6a00640 1 -- 172.21.15.89:0/2341755953 learned_addr learned my addr 172.21.15.89:0/2341755953 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:58.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.661+0000 7fadb6000640 1 --2- 172.21.15.89:0/2341755953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadb81047e0 0x7fadb819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.661+0000 7fadb6a00640 1 -- 172.21.15.89:0/2341755953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadb81047e0 msgr2=0x7fadb819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:58.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.661+0000 7fadb6a00640 1 --2- 172.21.15.89:0/2341755953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadb81047e0 0x7fadb819a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.661+0000 7fadb6a00640 1 -- 172.21.15.89:0/2341755953 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadb8108f60 msgr2=0x7fadb819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:58.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.661+0000 7fadb6a00640 1 --2- 172.21.15.89:0/2341755953 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadb8108f60 0x7fadb819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.661+0000 7fadb6a00640 1 -- 172.21.15.89:0/2341755953 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb819f3c0 con 0x7fadb8103e30 2024-09-10T08:44:58.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.662+0000 7fadb6a00640 1 --2- 172.21.15.89:0/2341755953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadb8103e30 0x7fadb819a410 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fada8004770 tx=0x7fada8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:58.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.662+0000 7fadbca00640 1 -- 172.21.15.89:0/2341755953 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada803f070 con 0x7fadb8103e30 2024-09-10T08:44:58.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.662+0000 7fadbca00640 1 -- 172.21.15.89:0/2341755953 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada8031ca0 con 0x7fadb8103e30 2024-09-10T08:44:58.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.662+0000 7fadbca00640 1 -- 172.21.15.89:0/2341755953 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8035030 con 0x7fadb8103e30 2024-09-10T08:44:58.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.662+0000 7fadb7e00640 1 -- 172.21.15.89:0/2341755953 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb819f650 con 0x7fadb8103e30 2024-09-10T08:44:58.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.662+0000 7fadb7e00640 1 -- 172.21.15.89:0/2341755953 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb819fae0 con 0x7fadb8103e30 2024-09-10T08:44:58.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.666+0000 7fadbca00640 1 -- 172.21.15.89:0/2341755953 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fada8042070 con 0x7fadb8103e30 2024-09-10T08:44:58.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.667+0000 7fadc09f9740 1 Processor -- start 2024-09-10T08:44:58.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.667+0000 7fadc09f9740 1 -- start start 2024-09-10T08:44:58.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.667+0000 7fadc09f9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e262f520 0x5599e262f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.667+0000 7fadc09f9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e262fe40 0x5599e263a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.667+0000 7fadc09f9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e263a700 0x5599e263caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.667+0000 7fadc09f9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5599e2595650 con 0x5599e262f520 2024-09-10T08:44:58.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.667+0000 7fadc09f9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5599e253cf10 con 0x5599e263a700 2024-09-10T08:44:58.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.667+0000 7fadc09f9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5599e2591610 con 0x5599e262fe40 2024-09-10T08:44:58.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.668+0000 7fadb6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e263a700 0x5599e263caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.668+0000 7fadb6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e263a700 0x5599e263caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57462/0 (socket says 172.21.15.89:57462) 2024-09-10T08:44:58.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.668+0000 7fadb6000640 1 -- 172.21.15.89:0/3024538218 learned_addr learned my addr 172.21.15.89:0/3024538218 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:58.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.668+0000 7fadb7400640 1 --2- 172.21.15.89:0/3024538218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e262f520 0x5599e262f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.668+0000 7fadb6a00640 1 --2- 172.21.15.89:0/3024538218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e262fe40 0x5599e263a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.669+0000 7fadb6000640 1 -- 172.21.15.89:0/3024538218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e262fe40 msgr2=0x5599e263a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:58.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.668+0000 7fadbca00640 1 --2- 172.21.15.89:0/2341755953 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fad9c0a2b80 0x7fad9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.669+0000 7fadb6000640 1 --2- 172.21.15.89:0/3024538218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e262fe40 0x5599e263a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.669+0000 7fadb6000640 1 -- 172.21.15.89:0/3024538218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e262f520 msgr2=0x5599e262f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:58.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.669+0000 7fadb6000640 1 --2- 172.21.15.89:0/3024538218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e262f520 0x5599e262f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.669+0000 7fadb6000640 1 -- 172.21.15.89:0/3024538218 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599e22a4580 con 0x5599e263a700 2024-09-10T08:44:58.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.669+0000 7fadb7400640 1 --2- 172.21.15.89:0/3024538218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e262f520 0x5599e262f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:44:58.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.669+0000 7fadbca00640 1 -- 172.21.15.89:0/2341755953 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fada80ef7d0 con 0x7fadb8103e30 2024-09-10T08:44:58.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.669+0000 7fadb6a00640 1 --2- 172.21.15.89:0/3024538218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e262fe40 0x5599e263a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:58.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.670+0000 7fadb7400640 1 --2- 172.21.15.89:0/2341755953 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fad9c0a2b80 0x7fad9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.670+0000 7fadb6000640 1 --2- 172.21.15.89:0/3024538218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e263a700 0x5599e263caf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fada4009c00 tx=0x7fada4010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e30ad7d02da73ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:58.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.670+0000 7fada3e00640 1 -- 172.21.15.89:0/3024538218 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4004020 con 0x5599e263a700 2024-09-10T08:44:58.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.670+0000 7fada3e00640 1 -- 172.21.15.89:0/3024538218 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada40041c0 con 0x5599e263a700 2024-09-10T08:44:58.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.671+0000 7fada3e00640 1 -- 172.21.15.89:0/3024538218 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4012670 con 0x5599e263a700 2024-09-10T08:44:58.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.671+0000 7fadc09f9740 1 -- 172.21.15.89:0/3024538218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e263a700 msgr2=0x5599e263caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:58.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.671+0000 7fadb7400640 1 --2- 172.21.15.89:0/2341755953 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fad9c0a2b80 0x7fad9c0a5040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009270 tx=0x7fadb0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:58.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.671+0000 7fadc09f9740 1 --2- 172.21.15.89:0/3024538218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e263a700 0x5599e263caf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fada4009c00 tx=0x7fada4010740 comp rx=0 tx=0).stop 2024-09-10T08:44:58.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.672+0000 7fadc09f9740 1 -- 172.21.15.89:0/3024538218 shutdown_connections 2024-09-10T08:44:58.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.672+0000 7fadc09f9740 1 --2- 172.21.15.89:0/3024538218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e263a700 0x5599e263caf0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.672+0000 7fadc09f9740 1 --2- 172.21.15.89:0/3024538218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e262fe40 0x5599e263a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.672+0000 7fadc09f9740 1 --2- 172.21.15.89:0/3024538218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e262f520 0x5599e262f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.672+0000 7fadc09f9740 1 -- 172.21.15.89:0/3024538218 >> 172.21.15.89:0/3024538218 conn(0x5599e262bab0 msgr2=0x5599e262be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:58.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.672+0000 7fadc09f9740 1 -- 172.21.15.89:0/3024538218 shutdown_connections 2024-09-10T08:44:58.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.672+0000 7fadc09f9740 1 -- 172.21.15.89:0/3024538218 wait complete. 2024-09-10T08:44:58.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.673+0000 7fadc09f9740 1 -- start start 2024-09-10T08:44:58.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.673+0000 7fadc09f9740 1 Processor -- start 2024-09-10T08:44:58.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.673+0000 7fadc09f9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e262f520 0x5599e2637660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.673+0000 7fadc09f9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e262fe40 0x5599e2637ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.673+0000 7fadc09f9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e263a700 0x5599e26380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.673+0000 7fadc09f9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5599e263d2d0 con 0x5599e263a700 2024-09-10T08:44:58.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.673+0000 7fadc09f9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5599e2550350 con 0x5599e262f520 2024-09-10T08:44:58.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.673+0000 7fadc09f9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5599e263d450 con 0x5599e262fe40 2024-09-10T08:44:58.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.673+0000 7fadc09f9740 10 client.0 _ll_register_callbacks cb 0x5599e2573290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:58.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.674+0000 7fadb6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e262f520 0x5599e2637660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.674+0000 7fadb6000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e262fe40 0x5599e2637ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.674+0000 7fadb7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e263a700 0x5599e26380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.674+0000 7fadb6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e262f520 0x5599e2637660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57474/0 (socket says 172.21.15.89:57474) 2024-09-10T08:44:58.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.674+0000 7fadb6a00640 1 -- 172.21.15.89:0/3561041289 learned_addr learned my addr 172.21.15.89:0/3561041289 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:58.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.674+0000 7fadb6a00640 1 -- 172.21.15.89:0/3561041289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e262fe40 msgr2=0x5599e2637ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:58.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.674+0000 7fadb6a00640 1 --2- 172.21.15.89:0/3561041289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e262fe40 0x5599e2637ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.674+0000 7fadb6a00640 1 -- 172.21.15.89:0/3561041289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e263a700 msgr2=0x5599e26380e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:58.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.674+0000 7fadb6a00640 1 --2- 172.21.15.89:0/3561041289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e263a700 0x5599e26380e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.674+0000 7fadb6a00640 1 -- 172.21.15.89:0/3561041289 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599e236e960 con 0x5599e262f520 2024-09-10T08:44:58.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.674+0000 7fadb6a00640 1 --2- 172.21.15.89:0/3561041289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e262f520 0x5599e2637660 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fada80bb9c0 tx=0x7fada802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3628cad60796deb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:58.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.674+0000 7fada0c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:44:58.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.675+0000 7fada0c00640 1 -- 172.21.15.89:0/3561041289 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8049070 con 0x5599e262f520 2024-09-10T08:44:58.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.675+0000 7fada0c00640 1 -- 172.21.15.89:0/3561041289 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fada80ecfb0 con 0x5599e262f520 2024-09-10T08:44:58.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.675+0000 7fadc09f9740 1 -- 172.21.15.89:0/3561041289 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fada40041c0 con 0x5599e262f520 2024-09-10T08:44:58.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.675+0000 7fada0c00640 1 -- 172.21.15.89:0/3561041289 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fada80bc3c0 con 0x5599e262f520 2024-09-10T08:44:58.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.675+0000 7fadc09f9740 10 client.26372 fetch_fsmap learned FSMap version 29 2024-09-10T08:44:58.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.675+0000 7fadc09f9740 1 -- 172.21.15.89:0/3561041289 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5599e2648ec0 con 0x5599e262f520 2024-09-10T08:44:58.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.675+0000 7fada0c00640 1 -- 172.21.15.89:0/3561041289 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada40041c0 con 0x5599e262f520 2024-09-10T08:44:58.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.675+0000 7fada0c00640 1 -- 172.21.15.89:0/3561041289 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fada804c060 con 0x5599e262f520 2024-09-10T08:44:58.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.675+0000 7fadc09f9740 10 client.26372 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:44:58.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.675+0000 7fadc09f9740 10 client.26372 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:44:58.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.676+0000 7fadc09f9740 20 client.26372 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:58.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.676+0000 7fadc09f9740 1 --2- 172.21.15.89:0/3561041289 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5599e264a2b0 0x5599e264c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.676+0000 7fadc09f9740 4 client.26372 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:44:58.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.676+0000 7fadc09f9740 1 -- 172.21.15.89:0/3561041289 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599e25a2bc0 con 0x5599e264a2b0 2024-09-10T08:44:58.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.676+0000 7fadb6000640 1 --2- 172.21.15.89:0/3561041289 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5599e264a2b0 0x5599e264c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.677+0000 7fadb6000640 1 --2- 172.21.15.89:0/3561041289 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5599e264a2b0 0x5599e264c720 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fde87147574e88b server_cookie=63cb74ee75a7870d in_seq=0 out_seq=0 2024-09-10T08:44:58.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.677+0000 7fada0c00640 10 client.26372 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:58.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.678+0000 7fada0c00640 1 -- 172.21.15.89:0/3561041289 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599e25a2bc0 con 0x5599e264a2b0 2024-09-10T08:44:58.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.678+0000 7fada0c00640 10 client.26372 handle_command_reply: tid=0 2024-09-10T08:44:58.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.679+0000 7fadc09f9740 1 client.26372 shutdown 2024-09-10T08:44:58.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.679+0000 7fadc09f9740 1 -- 172.21.15.89:0/3561041289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e262f520 msgr2=0x5599e2637660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:58.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.679+0000 7fadc09f9740 1 --2- 172.21.15.89:0/3561041289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e262f520 0x5599e2637660 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fada80bb9c0 tx=0x7fada802a410 comp rx=0 tx=0).stop 2024-09-10T08:44:58.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.679+0000 7fadc09f9740 1 -- 172.21.15.89:0/3561041289 shutdown_connections 2024-09-10T08:44:58.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.679+0000 7fadc09f9740 1 --2- 172.21.15.89:0/3561041289 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5599e264a2b0 0x5599e264c720 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.679+0000 7fadc09f9740 1 --2- 172.21.15.89:0/3561041289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e263a700 0x5599e26380e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.679+0000 7fadc09f9740 1 --2- 172.21.15.89:0/3561041289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e262fe40 0x5599e2637ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.679+0000 7fadc09f9740 1 --2- 172.21.15.89:0/3561041289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e262f520 0x5599e2637660 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.679+0000 7fada0c00640 0 client.26372 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:58.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.679+0000 7fadc09f9740 1 -- 172.21.15.89:0/3561041289 >> 172.21.15.89:0/3561041289 conn(0x5599e262bab0 msgr2=0x5599e262acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:58.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.679+0000 7fadc09f9740 1 -- 172.21.15.89:0/3561041289 shutdown_connections 2024-09-10T08:44:58.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.679+0000 7fadc09f9740 1 -- 172.21.15.89:0/3561041289 wait complete. 2024-09-10T08:44:58.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.679+0000 7fadc09f9740 20 client.26372 trim_cache size 0 max 16384 2024-09-10T08:44:58.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.701+0000 7fadc09f9740 1 Processor -- start 2024-09-10T08:44:58.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.701+0000 7fadc09f9740 1 -- start start 2024-09-10T08:44:58.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.702+0000 7fadc09f9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e262f520 0x5599e262ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.702+0000 7fadc09f9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e262fe40 0x5599e2631fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.702+0000 7fadc09f9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e263a700 0x5599e2635f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.702+0000 7fadc09f9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fadb81b36d0 con 0x5599e262fe40 2024-09-10T08:44:58.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.702+0000 7fadc09f9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fadb81b0fa0 con 0x5599e263a700 2024-09-10T08:44:58.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.702+0000 7fadc09f9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fadb81aedf0 con 0x5599e262f520 2024-09-10T08:44:58.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.702+0000 7fadb6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e262f520 0x5599e262ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.702+0000 7fadb6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e262f520 0x5599e262ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:42332/0 (socket says 172.21.15.89:42332) 2024-09-10T08:44:58.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.702+0000 7fadb6a00640 1 -- 172.21.15.89:0/1030403147 learned_addr learned my addr 172.21.15.89:0/1030403147 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:58.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.702+0000 7fadb7400640 1 --2- 172.21.15.89:0/1030403147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e263a700 0x5599e2635f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.702+0000 7fadb6000640 1 --2- 172.21.15.89:0/1030403147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e262fe40 0x5599e2631fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.702+0000 7fadb6a00640 1 -- 172.21.15.89:0/1030403147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e263a700 msgr2=0x5599e2635f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:58.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.702+0000 7fadb6a00640 1 --2- 172.21.15.89:0/1030403147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e263a700 0x5599e2635f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.702+0000 7fadb6a00640 1 -- 172.21.15.89:0/1030403147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e262fe40 msgr2=0x5599e2631fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:58.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.702+0000 7fadb6a00640 1 --2- 172.21.15.89:0/1030403147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e262fe40 0x5599e2631fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.702+0000 7fadb6a00640 1 -- 172.21.15.89:0/1030403147 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599e262b8e0 con 0x5599e262f520 2024-09-10T08:44:58.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.703+0000 7fadb7400640 1 --2- 172.21.15.89:0/1030403147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e263a700 0x5599e2635f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:58.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.703+0000 7fadb6a00640 1 --2- 172.21.15.89:0/1030403147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e262f520 0x5599e262ab40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fada80b9f70 tx=0x7fada80efab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cb221cc32cb32c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:58.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.703+0000 7fada3400640 1 -- 172.21.15.89:0/1030403147 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8049070 con 0x5599e262f520 2024-09-10T08:44:58.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.703+0000 7fada3400640 1 -- 172.21.15.89:0/1030403147 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada80c0070 con 0x5599e262f520 2024-09-10T08:44:58.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.703+0000 7fada3400640 1 -- 172.21.15.89:0/1030403147 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada80c4930 con 0x5599e262f520 2024-09-10T08:44:58.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.704+0000 7fadc09f9740 1 -- 172.21.15.89:0/1030403147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e262f520 msgr2=0x5599e262ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:58.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.704+0000 7fadc09f9740 1 --2- 172.21.15.89:0/1030403147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e262f520 0x5599e262ab40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fada80b9f70 tx=0x7fada80efab0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.704+0000 7fadc09f9740 1 -- 172.21.15.89:0/1030403147 shutdown_connections 2024-09-10T08:44:58.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.704+0000 7fadc09f9740 1 --2- 172.21.15.89:0/1030403147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e263a700 0x5599e2635f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.704+0000 7fadc09f9740 1 --2- 172.21.15.89:0/1030403147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e262fe40 0x5599e2631fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.704+0000 7fadc09f9740 1 --2- 172.21.15.89:0/1030403147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e262f520 0x5599e262ab40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.704+0000 7fadc09f9740 1 -- 172.21.15.89:0/1030403147 >> 172.21.15.89:0/1030403147 conn(0x5599e262bab0 msgr2=0x5599e26395d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:58.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.704+0000 7fadc09f9740 1 -- 172.21.15.89:0/1030403147 shutdown_connections 2024-09-10T08:44:58.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.704+0000 7fadc09f9740 1 -- 172.21.15.89:0/1030403147 wait complete. 2024-09-10T08:44:58.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.705+0000 7fadc09f9740 1 -- start start 2024-09-10T08:44:58.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.705+0000 7fadc09f9740 1 Processor -- start 2024-09-10T08:44:58.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.706+0000 7fadc09f9740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e262f520 0x5599e2642a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.706+0000 7fadc09f9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e262fe40 0x5599e2642f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.706+0000 7fadc09f9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e263a700 0x5599e263efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.706+0000 7fadc09f9740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fadb81af460 con 0x5599e262f520 2024-09-10T08:44:58.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.706+0000 7fadc09f9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fadb81b08d0 con 0x5599e262fe40 2024-09-10T08:44:58.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.706+0000 7fadc09f9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fadb81afb30 con 0x5599e263a700 2024-09-10T08:44:58.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.706+0000 7fadc09f9740 10 client.0 _ll_register_callbacks cb 0x7fada8031e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:58.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.706+0000 7fadb6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e262f520 0x5599e2642a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.706+0000 7fadb7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e263a700 0x5599e263efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.706+0000 7fadb7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e263a700 0x5599e263efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:42346/0 (socket says 172.21.15.89:42346) 2024-09-10T08:44:58.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.706+0000 7fadb7400640 1 -- 172.21.15.89:0/2938832548 learned_addr learned my addr 172.21.15.89:0/2938832548 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:44:58.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.706+0000 7fadb6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e262fe40 0x5599e2642f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.707+0000 7fadb6000640 1 -- 172.21.15.89:0/2938832548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e263a700 msgr2=0x5599e263efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:58.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.707+0000 7fadb6000640 1 --2- 172.21.15.89:0/2938832548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e263a700 0x5599e263efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.707+0000 7fadb6000640 1 -- 172.21.15.89:0/2938832548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e262f520 msgr2=0x5599e2642a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:58.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.707+0000 7fadb6000640 1 --2- 172.21.15.89:0/2938832548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e262f520 0x5599e2642a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.707+0000 7fadb6000640 1 -- 172.21.15.89:0/2938832548 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599e263f7d0 con 0x5599e262fe40 2024-09-10T08:44:58.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.707+0000 7fadb6a00640 1 --2- 172.21.15.89:0/2938832548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e262f520 0x5599e2642a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:58.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.707+0000 7fadb6000640 1 --2- 172.21.15.89:0/2938832548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e262fe40 0x5599e2642f50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fada4000c00 tx=0x7fada4003e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cef45c9cde0a8128 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:58.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.707+0000 7fada0c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:44:58.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.708+0000 7fada0c00640 1 -- 172.21.15.89:0/2938832548 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada402c020 con 0x5599e262fe40 2024-09-10T08:44:58.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.708+0000 7fada0c00640 1 -- 172.21.15.89:0/2938832548 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fada40037f0 con 0x5599e262fe40 2024-09-10T08:44:58.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.708+0000 7fadc09f9740 1 -- 172.21.15.89:0/2938832548 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fada80c0070 con 0x5599e262fe40 2024-09-10T08:44:58.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.708+0000 7fada0c00640 1 -- 172.21.15.89:0/2938832548 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fada4034050 con 0x5599e262fe40 2024-09-10T08:44:58.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.709+0000 7fadc09f9740 10 client.26378 fetch_fsmap learned FSMap version 29 2024-09-10T08:44:58.710 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.709+0000 7fadc09f9740 1 -- 172.21.15.89:0/2938832548 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5599e26b5c20 con 0x5599e262fe40 2024-09-10T08:44:58.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.709+0000 7fada0c00640 1 -- 172.21.15.89:0/2938832548 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada80c0070 con 0x5599e262fe40 2024-09-10T08:44:58.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.709+0000 7fada0c00640 1 -- 172.21.15.89:0/2938832548 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fada400f040 con 0x5599e262fe40 2024-09-10T08:44:58.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.710+0000 7fadc09f9740 10 client.26378 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:44:58.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.710+0000 7fadc09f9740 10 client.26378 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:44:58.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.710+0000 7fadc09f9740 20 client.26378 populate_metadata read hostname 'smithi089' 2024-09-10T08:44:58.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.710+0000 7fadc09f9740 1 --2- 172.21.15.89:0/2938832548 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5599e26b6070 0x5599e26b8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.710+0000 7fadc09f9740 4 client.26378 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a577fe06", "release": true, "await": true}] 2024-09-10T08:44:58.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.710+0000 7fadc09f9740 1 -- 172.21.15.89:0/2938832548 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a577fe06", "release": true, "await": true}) -- 0x5599e259d0d0 con 0x5599e26b6070 2024-09-10T08:44:58.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.710+0000 7fadb6a00640 1 --2- 172.21.15.89:0/2938832548 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5599e26b6070 0x5599e26b8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.711+0000 7fadb6a00640 1 --2- 172.21.15.89:0/2938832548 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5599e26b6070 0x5599e26b8460 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a61c5494037ae65c server_cookie=cdb6d79b873248f3 in_seq=0 out_seq=0 2024-09-10T08:44:58.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:58.711+0000 7fada0c00640 10 client.26378 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:59.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:44:58 smithi149 ceph-mon[36167]: pgmap v544: 129 pgs: 129 active+clean; 303 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.0 MiB/s wr, 1.01k op/s 2024-09-10T08:44:59.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:44:58 smithi111 ceph-mon[35220]: pgmap v544: 129 pgs: 129 active+clean; 303 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.0 MiB/s wr, 1.01k op/s 2024-09-10T08:44:59.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:44:58 smithi089 ceph-mon[30757]: pgmap v544: 129 pgs: 129 active+clean; 303 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.0 MiB/s wr, 1.01k op/s 2024-09-10T08:44:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.305+0000 7fada0c00640 1 -- 172.21.15.89:0/2938832548 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5599e259d0d0 con 0x5599e26b6070 2024-09-10T08:44:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.306+0000 7fada0c00640 10 client.26378 handle_command_reply: tid=0 2024-09-10T08:44:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.306+0000 7fadc09f9740 1 client.26378 shutdown 2024-09-10T08:44:59.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.306+0000 7fadc09f9740 1 -- 172.21.15.89:0/2938832548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e262fe40 msgr2=0x5599e2642f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.306+0000 7fadc09f9740 1 --2- 172.21.15.89:0/2938832548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e262fe40 0x5599e2642f50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fada4000c00 tx=0x7fada4003e80 comp rx=0 tx=0).stop 2024-09-10T08:44:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.306+0000 7fadc09f9740 1 -- 172.21.15.89:0/2938832548 shutdown_connections 2024-09-10T08:44:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.306+0000 7fadc09f9740 1 --2- 172.21.15.89:0/2938832548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599e263a700 0x5599e263efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.307+0000 7fadc09f9740 1 --2- 172.21.15.89:0/2938832548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5599e262fe40 0x5599e2642f50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.307+0000 7fadc09f9740 1 --2- 172.21.15.89:0/2938832548 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5599e26b6070 0x5599e26b8460 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.307+0000 7fadc09f9740 1 --2- 172.21.15.89:0/2938832548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5599e262f520 0x5599e2642a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.307+0000 7fadc09f9740 1 -- 172.21.15.89:0/2938832548 >> 172.21.15.89:0/2938832548 conn(0x5599e262bab0 msgr2=0x5599e2645d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:59.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.307+0000 7fada0c00640 0 client.26378 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:44:59.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.307+0000 7fadc09f9740 1 -- 172.21.15.89:0/2938832548 shutdown_connections 2024-09-10T08:44:59.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.307+0000 7fadc09f9740 1 -- 172.21.15.89:0/2938832548 wait complete. 2024-09-10T08:44:59.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.307+0000 7fadc09f9740 20 client.26378 trim_cache size 0 max 16384 2024-09-10T08:44:59.310 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:44:59.310 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 29, 2024-09-10T08:44:59.310 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24554, 2024-09-10T08:44:59.310 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 12, 2024-09-10T08:44:59.310 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": { 2024-09-10T08:44:59.310 INFO:teuthology.orchestra.run.smithi089.stdout: "a577fe06": { 2024-09-10T08:44:59.310 INFO:teuthology.orchestra.run.smithi089.stdout: "version": 12, 2024-09-10T08:44:59.310 INFO:teuthology.orchestra.run.smithi089.stdout: "age_ref": 0.0, 2024-09-10T08:44:59.310 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:44:59.310 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RELEASED", 2024-09-10T08:44:59.310 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:44:59.310 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:44:59.310 INFO:teuthology.orchestra.run.smithi089.stdout: "timeout": 90.0, 2024-09-10T08:44:59.311 INFO:teuthology.orchestra.run.smithi089.stdout: "expiration": 142.8, 2024-09-10T08:44:59.311 INFO:teuthology.orchestra.run.smithi089.stdout: "members": { 2024-09-10T08:44:59.311 INFO:teuthology.orchestra.run.smithi089.stdout: "file:/": { 2024-09-10T08:44:59.311 INFO:teuthology.orchestra.run.smithi089.stdout: "excluded": false, 2024-09-10T08:44:59.311 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:44:59.311 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RELEASED", 2024-09-10T08:44:59.311 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:44:59.311 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:44:59.311 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:44:59.311 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:44:59.311 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:44:59.311 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:44:59.311 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:44:59.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.308+0000 7fadb7e00640 1 -- 172.21.15.89:0/2341755953 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fad9c0a2b80 msgr2=0x7fad9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:59.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.308+0000 7fadb7e00640 1 --2- 172.21.15.89:0/2341755953 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fad9c0a2b80 0x7fad9c0a5040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009270 tx=0x7fadb0009500 comp rx=0 tx=0).stop 2024-09-10T08:44:59.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.309+0000 7fadb7e00640 1 -- 172.21.15.89:0/2341755953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadb8103e30 msgr2=0x7fadb819a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:59.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.309+0000 7fadb7e00640 1 --2- 172.21.15.89:0/2341755953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadb8103e30 0x7fadb819a410 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fada8004770 tx=0x7fada8003c10 comp rx=0 tx=0).stop 2024-09-10T08:44:59.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.309+0000 7fadb7e00640 1 -- 172.21.15.89:0/2341755953 shutdown_connections 2024-09-10T08:44:59.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.309+0000 7fadb7e00640 1 --2- 172.21.15.89:0/2341755953 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fad9c0a2b80 0x7fad9c0a5040 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:59.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.309+0000 7fadb7e00640 1 --2- 172.21.15.89:0/2341755953 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadb8108f60 0x7fadb819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:59.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.309+0000 7fadb7e00640 1 --2- 172.21.15.89:0/2341755953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadb81047e0 0x7fadb819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:59.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.309+0000 7fadb7e00640 1 --2- 172.21.15.89:0/2341755953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadb8103e30 0x7fadb819a410 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:59.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.309+0000 7fadb7e00640 1 -- 172.21.15.89:0/2341755953 >> 172.21.15.89:0/2341755953 conn(0x7fadb80ff910 msgr2=0x7fadb8100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:59.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.309+0000 7fadb7e00640 1 -- 172.21.15.89:0/2341755953 shutdown_connections 2024-09-10T08:44:59.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:44:59.309+0000 7fadb7e00640 1 -- 172.21.15.89:0/2341755953 wait complete. 2024-09-10T08:45:00.245 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a577fe06', total seconds elapsed: 91.9 2024-09-10T08:45:00.246 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-09-10T08:45:00.246 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:45:00.700 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:01.144 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:00 smithi089 ceph-mon[30757]: pgmap v545: 129 pgs: 129 active+clean; 303 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 836 op/s 2024-09-10T08:45:01.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:00 smithi149 ceph-mon[36167]: pgmap v545: 129 pgs: 129 active+clean; 303 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 836 op/s 2024-09-10T08:45:01.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:00 smithi111 ceph-mon[35220]: pgmap v545: 129 pgs: 129 active+clean; 303 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 836 op/s 2024-09-10T08:45:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.293+0000 7fab73000640 1 -- 172.21.15.89:0/573904975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab6c11b140 msgr2=0x7fab6c11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.293+0000 7fab73000640 1 --2- 172.21.15.89:0/573904975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab6c11b140 0x7fab6c11d530 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fab64009ca0 tx=0x7fab6402a8b0 comp rx=0 tx=0).stop 2024-09-10T08:45:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.293+0000 7fab73000640 1 -- 172.21.15.89:0/573904975 shutdown_connections 2024-09-10T08:45:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.293+0000 7fab73000640 1 --2- 172.21.15.89:0/573904975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab6c11b140 0x7fab6c11d530 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.293+0000 7fab73000640 1 --2- 172.21.15.89:0/573904975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab6c06ea30 0x7fab6c06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.293+0000 7fab73000640 1 --2- 172.21.15.89:0/573904975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab6c110190 0x7fab6c06e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.293+0000 7fab73000640 1 -- 172.21.15.89:0/573904975 >> 172.21.15.89:0/573904975 conn(0x7fab6c073180 msgr2=0x7fab6c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.294+0000 7fab73000640 1 -- 172.21.15.89:0/573904975 shutdown_connections 2024-09-10T08:45:01.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.294+0000 7fab73000640 1 -- 172.21.15.89:0/573904975 wait complete. 2024-09-10T08:45:01.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.295+0000 7fab73000640 1 Processor -- start 2024-09-10T08:45:01.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.295+0000 7fab73000640 1 -- start start 2024-09-10T08:45:01.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.296+0000 7fab73000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab6c06ea30 0x7fab6c1a7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:01.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.296+0000 7fab73000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab6c110190 0x7fab6c1a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:01.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.296+0000 7fab73000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab6c11b140 0x7fab6c1ab950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:01.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.296+0000 7fab73000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c120be0 con 0x7fab6c110190 2024-09-10T08:45:01.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.296+0000 7fab73000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c120a60 con 0x7fab6c11b140 2024-09-10T08:45:01.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.296+0000 7fab73000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c120d60 con 0x7fab6c06ea30 2024-09-10T08:45:01.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.297+0000 7fab71c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab6c06ea30 0x7fab6c1a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:01.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.297+0000 7fab71c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab6c06ea30 0x7fab6c1a7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:42362/0 (socket says 172.21.15.89:42362) 2024-09-10T08:45:01.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.297+0000 7fab71c00640 1 -- 172.21.15.89:0/70860195 learned_addr learned my addr 172.21.15.89:0/70860195 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:01.298 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.297+0000 7fab71200640 1 --2- 172.21.15.89:0/70860195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab6c110190 0x7fab6c1a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:01.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.297+0000 7fab72600640 1 --2- 172.21.15.89:0/70860195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab6c11b140 0x7fab6c1ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:01.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.297+0000 7fab71c00640 1 -- 172.21.15.89:0/70860195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab6c11b140 msgr2=0x7fab6c1ab950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:01.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.297+0000 7fab71c00640 1 --2- 172.21.15.89:0/70860195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab6c11b140 0x7fab6c1ab950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:01.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.297+0000 7fab71c00640 1 -- 172.21.15.89:0/70860195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab6c110190 msgr2=0x7fab6c1a75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:01.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.298+0000 7fab71c00640 1 --2- 172.21.15.89:0/70860195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab6c110190 0x7fab6c1a75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:01.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.298+0000 7fab71c00640 1 -- 172.21.15.89:0/70860195 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6c1ac030 con 0x7fab6c06ea30 2024-09-10T08:45:01.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.299+0000 7fab72600640 1 --2- 172.21.15.89:0/70860195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab6c11b140 0x7fab6c1ab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:01.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.299+0000 7fab71200640 1 --2- 172.21.15.89:0/70860195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab6c110190 0x7fab6c1a75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:01.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.299+0000 7fab71c00640 1 --2- 172.21.15.89:0/70860195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab6c06ea30 0x7fab6c1a7080 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fab6000e9f0 tx=0x7fab6000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:01.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.299+0000 7fab6a000640 1 -- 172.21.15.89:0/70860195 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6000cd80 con 0x7fab6c06ea30 2024-09-10T08:45:01.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.299+0000 7fab6a000640 1 -- 172.21.15.89:0/70860195 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab60005500 con 0x7fab6c06ea30 2024-09-10T08:45:01.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.299+0000 7fab6a000640 1 -- 172.21.15.89:0/70860195 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab60010670 con 0x7fab6c06ea30 2024-09-10T08:45:01.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.299+0000 7fab73000640 1 -- 172.21.15.89:0/70860195 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab6c1ac320 con 0x7fab6c06ea30 2024-09-10T08:45:01.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.300+0000 7fab73000640 1 -- 172.21.15.89:0/70860195 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab6c1b3c00 con 0x7fab6c06ea30 2024-09-10T08:45:01.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.301+0000 7fab73000640 1 -- 172.21.15.89:0/70860195 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab6c06ee70 con 0x7fab6c06ea30 2024-09-10T08:45:01.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.304+0000 7fab6a000640 1 -- 172.21.15.89:0/70860195 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fab600046c0 con 0x7fab6c06ea30 2024-09-10T08:45:01.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.307+0000 7fab6a000640 1 --2- 172.21.15.89:0/70860195 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fab540a2cd0 0x7fab540a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:01.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.307+0000 7fab6a000640 1 -- 172.21.15.89:0/70860195 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fab600cf750 con 0x7fab6c06ea30 2024-09-10T08:45:01.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.308+0000 7fab71200640 1 --2- 172.21.15.89:0/70860195 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fab540a2cd0 0x7fab540a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:01.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.309+0000 7fab71200640 1 --2- 172.21.15.89:0/70860195 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fab540a2cd0 0x7fab540a5190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fab6c06e540 tx=0x7fab58005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:01.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.310+0000 7fab6a000640 1 -- 172.21.15.89:0/70860195 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab6009c280 con 0x7fab6c06ea30 2024-09-10T08:45:01.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.596+0000 7fab73000640 1 -- 172.21.15.89:0/70860195 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab6c1a83e0 con 0x7fab6c06ea30 2024-09-10T08:45:01.598 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.597+0000 7fab6a000640 1 -- 172.21.15.89:0/70860195 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11919 (secure 0 0 0) 0x7fab6009bb80 con 0x7fab6c06ea30 2024-09-10T08:45:01.598 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:45:01.598 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"btime":"2024-09-10T08:44:40:778845+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:44:40.174532+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24496,24518,24554]},"id":1}]} 2024-09-10T08:45:01.598 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 29 2024-09-10T08:45:01.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.599+0000 7fab73000640 1 -- 172.21.15.89:0/70860195 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fab540a2cd0 msgr2=0x7fab540a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:01.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.599+0000 7fab73000640 1 --2- 172.21.15.89:0/70860195 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fab540a2cd0 0x7fab540a5190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fab6c06e540 tx=0x7fab58005f90 comp rx=0 tx=0).stop 2024-09-10T08:45:01.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.600+0000 7fab73000640 1 -- 172.21.15.89:0/70860195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab6c06ea30 msgr2=0x7fab6c1a7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:01.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.600+0000 7fab73000640 1 --2- 172.21.15.89:0/70860195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab6c06ea30 0x7fab6c1a7080 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fab6000e9f0 tx=0x7fab6000edf0 comp rx=0 tx=0).stop 2024-09-10T08:45:01.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.600+0000 7fab73000640 1 -- 172.21.15.89:0/70860195 shutdown_connections 2024-09-10T08:45:01.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.600+0000 7fab73000640 1 --2- 172.21.15.89:0/70860195 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fab540a2cd0 0x7fab540a5190 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:01.601 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.600+0000 7fab73000640 1 --2- 172.21.15.89:0/70860195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab6c11b140 0x7fab6c1ab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:01.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.600+0000 7fab73000640 1 --2- 172.21.15.89:0/70860195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab6c110190 0x7fab6c1a75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:01.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.600+0000 7fab73000640 1 --2- 172.21.15.89:0/70860195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab6c06ea30 0x7fab6c1a7080 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:01.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.600+0000 7fab73000640 1 -- 172.21.15.89:0/70860195 >> 172.21.15.89:0/70860195 conn(0x7fab6c073180 msgr2=0x7fab6c10e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:01.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.600+0000 7fab73000640 1 -- 172.21.15.89:0/70860195 shutdown_connections 2024-09-10T08:45:01.602 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:01.600+0000 7fab73000640 1 -- 172.21.15.89:0/70860195 wait complete. 2024-09-10T08:45:02.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:01 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/70860195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:02.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:01 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/70860195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:02.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:01 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/70860195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:02.507 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24554 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-09-10T08:45:02.508 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24554 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-09-10T08:45:03.010 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:03.155 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:02 smithi089 ceph-mon[30757]: pgmap v546: 129 pgs: 129 active+clean; 312 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.6 MiB/s wr, 1.19k op/s 2024-09-10T08:45:03.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:02 smithi149 ceph-mon[36167]: pgmap v546: 129 pgs: 129 active+clean; 312 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.6 MiB/s wr, 1.19k op/s 2024-09-10T08:45:03.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:02 smithi111 ceph-mon[35220]: pgmap v546: 129 pgs: 129 active+clean; 312 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.6 MiB/s wr, 1.19k op/s 2024-09-10T08:45:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.640+0000 7f58be600640 1 -- 172.21.15.89:0/138991755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b810e440 msgr2=0x7f58b8110830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.640+0000 7f58be600640 1 --2- 172.21.15.89:0/138991755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b810e440 0x7f58b8110830 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009860 tx=0x7f58a402a3d0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.641+0000 7f58be600640 1 -- 172.21.15.89:0/138991755 shutdown_connections 2024-09-10T08:45:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.641+0000 7f58be600640 1 --2- 172.21.15.89:0/138991755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b810e440 0x7f58b8110830 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.641+0000 7f58be600640 1 --2- 172.21.15.89:0/138991755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b8074e30 0x7f58b810df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.641+0000 7f58be600640 1 --2- 172.21.15.89:0/138991755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58b8074510 0x7f58b80748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.641+0000 7f58be600640 1 -- 172.21.15.89:0/138991755 >> 172.21.15.89:0/138991755 conn(0x7f58b80fd8e0 msgr2=0x7f58b80ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.641+0000 7f58be600640 1 -- 172.21.15.89:0/138991755 shutdown_connections 2024-09-10T08:45:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.641+0000 7f58be600640 1 -- 172.21.15.89:0/138991755 wait complete. 2024-09-10T08:45:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.643+0000 7f58be600640 1 Processor -- start 2024-09-10T08:45:03.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.643+0000 7f58be600640 1 -- start start 2024-09-10T08:45:03.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.644+0000 7f58be600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b8074510 0x7f58b8073820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.644+0000 7f58be600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58b8074e30 0x7f58b8073d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.644+0000 7f58be600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b810e440 0x7f58b806c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.644+0000 7f58be600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f58b81141a0 con 0x7f58b8074510 2024-09-10T08:45:03.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.644+0000 7f58be600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f58b8114020 con 0x7f58b810e440 2024-09-10T08:45:03.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.644+0000 7f58be600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f58b8114320 con 0x7f58b8074e30 2024-09-10T08:45:03.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.644+0000 7f58b7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b8074510 0x7f58b8073820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:03.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.644+0000 7f58b7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b810e440 0x7f58b806c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:03.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.645+0000 7f58b7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b810e440 0x7f58b806c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50320/0 (socket says 172.21.15.89:50320) 2024-09-10T08:45:03.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.645+0000 7f58b7e00640 1 -- 172.21.15.89:0/4287148171 learned_addr learned my addr 172.21.15.89:0/4287148171 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:03.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.645+0000 7f58b6a00640 1 --2- 172.21.15.89:0/4287148171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58b8074e30 0x7f58b8073d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:03.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.645+0000 7f58b7e00640 1 -- 172.21.15.89:0/4287148171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58b8074e30 msgr2=0x7f58b8073d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:03.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.645+0000 7f58b7e00640 1 --2- 172.21.15.89:0/4287148171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58b8074e30 0x7f58b8073d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.645+0000 7f58b7e00640 1 -- 172.21.15.89:0/4287148171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b8074510 msgr2=0x7f58b8073820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:03.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.645+0000 7f58b7e00640 1 --2- 172.21.15.89:0/4287148171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b8074510 0x7f58b8073820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.645+0000 7f58b7e00640 1 -- 172.21.15.89:0/4287148171 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b806d1d0 con 0x7f58b810e440 2024-09-10T08:45:03.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.646+0000 7f58b7e00640 1 --2- 172.21.15.89:0/4287148171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b810e440 0x7f58b806c910 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f58a402a8f0 tx=0x7f58a40305b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:03.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.646+0000 7f58afe00640 1 -- 172.21.15.89:0/4287148171 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4005c90 con 0x7f58b810e440 2024-09-10T08:45:03.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.646+0000 7f58afe00640 1 -- 172.21.15.89:0/4287148171 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58a4031400 con 0x7f58b810e440 2024-09-10T08:45:03.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.646+0000 7f58afe00640 1 -- 172.21.15.89:0/4287148171 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4004b40 con 0x7f58b810e440 2024-09-10T08:45:03.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.648+0000 7f58be600640 1 -- 172.21.15.89:0/4287148171 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b806d460 con 0x7f58b810e440 2024-09-10T08:45:03.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.648+0000 7f58be600640 1 -- 172.21.15.89:0/4287148171 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b8071350 con 0x7f58b810e440 2024-09-10T08:45:03.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.654+0000 7f58c1086740 1 Processor -- start 2024-09-10T08:45:03.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.656+0000 7f58afe00640 1 -- 172.21.15.89:0/4287148171 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f58a4039040 con 0x7f58b810e440 2024-09-10T08:45:03.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.657+0000 7f58c1086740 1 -- start start 2024-09-10T08:45:03.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.657+0000 7f58c1086740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a81e0554e0 0x55a81e0558c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.657+0000 7f58c1086740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e055e00 0x55a81e060180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.657+0000 7f58c1086740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e0606c0 0x55a81e062ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.657+0000 7f58c1086740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a81dfba260 con 0x55a81e0606c0 2024-09-10T08:45:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.657+0000 7f58c1086740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a81df61500 con 0x55a81e0554e0 2024-09-10T08:45:03.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.657+0000 7f58c1086740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a81dfb6220 con 0x55a81e055e00 2024-09-10T08:45:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.658+0000 7f58afe00640 1 --2- 172.21.15.89:0/4287148171 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f589c0a2c80 0x7f589c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.658+0000 7f58b6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e0606c0 0x55a81e062ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.658+0000 7f58b6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a81e0554e0 0x55a81e0558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.658+0000 7f58b7e00640 1 --2- 172.21.15.89:0/4287148171 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f589c0a2c80 0x7f589c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.658+0000 7f58b6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e0606c0 0x55a81e062ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60524/0 (socket says 172.21.15.89:60524) 2024-09-10T08:45:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.658+0000 7f58b6a00640 1 -- 172.21.15.89:0/59408554 learned_addr learned my addr 172.21.15.89:0/59408554 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.659+0000 7f58b6a00640 1 -- 172.21.15.89:0/59408554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e055e00 msgr2=0x55a81e060180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:45:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.659+0000 7f58b6a00640 1 --2- 172.21.15.89:0/59408554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e055e00 0x55a81e060180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.659+0000 7f58b6a00640 1 -- 172.21.15.89:0/59408554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a81e0554e0 msgr2=0x55a81e0558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.659+0000 7f58b6a00640 1 --2- 172.21.15.89:0/59408554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a81e0554e0 0x55a81e0558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.659+0000 7f58b6a00640 1 -- 172.21.15.89:0/59408554 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a81dd2bba0 con 0x55a81e0606c0 2024-09-10T08:45:03.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.659+0000 7f58afe00640 1 -- 172.21.15.89:0/4287148171 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f58a40eff90 con 0x7f58b810e440 2024-09-10T08:45:03.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.659+0000 7f58b6a00640 1 --2- 172.21.15.89:0/59408554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e0606c0 0x55a81e062ab0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f58a8011480 tx=0x7f58a8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f44b4c83285313 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:03.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.660+0000 7f58b7e00640 1 --2- 172.21.15.89:0/4287148171 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f589c0a2c80 0x7f589c0a5140 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f58a40ed630 tx=0x7f58a40f3660 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:03.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.662+0000 7f58acc00640 1 -- 172.21.15.89:0/59408554 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8021030 con 0x55a81e0606c0 2024-09-10T08:45:03.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.662+0000 7f58acc00640 1 -- 172.21.15.89:0/59408554 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58a80258a0 con 0x55a81e0606c0 2024-09-10T08:45:03.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.662+0000 7f58acc00640 1 -- 172.21.15.89:0/59408554 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a800f820 con 0x55a81e0606c0 2024-09-10T08:45:03.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.667+0000 7f58c1086740 1 -- 172.21.15.89:0/59408554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e0606c0 msgr2=0x55a81e062ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.667+0000 7f58c1086740 1 --2- 172.21.15.89:0/59408554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e0606c0 0x55a81e062ab0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f58a8011480 tx=0x7f58a8005850 comp rx=0 tx=0).stop 2024-09-10T08:45:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.667+0000 7f58c1086740 1 -- 172.21.15.89:0/59408554 shutdown_connections 2024-09-10T08:45:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.667+0000 7f58c1086740 1 --2- 172.21.15.89:0/59408554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e0606c0 0x55a81e062ab0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.667+0000 7f58c1086740 1 --2- 172.21.15.89:0/59408554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e055e00 0x55a81e060180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.667+0000 7f58c1086740 1 --2- 172.21.15.89:0/59408554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a81e0554e0 0x55a81e0558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.667+0000 7f58c1086740 1 -- 172.21.15.89:0/59408554 >> 172.21.15.89:0/59408554 conn(0x55a81e051a70 msgr2=0x55a81e051e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:03.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.667+0000 7f58c1086740 1 -- 172.21.15.89:0/59408554 shutdown_connections 2024-09-10T08:45:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.667+0000 7f58c1086740 1 -- 172.21.15.89:0/59408554 wait complete. 2024-09-10T08:45:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.668+0000 7f58c1086740 1 -- start start 2024-09-10T08:45:03.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.668+0000 7f58c1086740 1 Processor -- start 2024-09-10T08:45:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.671+0000 7f58c1086740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e0554e0 0x55a81e05d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.671+0000 7f58c1086740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e055e00 0x55a81e05db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.671+0000 7f58c1086740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a81e0606c0 0x55a81e05e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.671+0000 7f58c1086740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a81e063290 con 0x55a81e0554e0 2024-09-10T08:45:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.671+0000 7f58c1086740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a81df74bd0 con 0x55a81e0606c0 2024-09-10T08:45:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.671+0000 7f58c1086740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a81e063410 con 0x55a81e055e00 2024-09-10T08:45:03.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.671+0000 7f58c1086740 10 client.0 _ll_register_callbacks cb 0x55a81dcc90a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.672+0000 7f58b7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e055e00 0x55a81e05db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.672+0000 7f58b7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e055e00 0x55a81e05db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:57160/0 (socket says 172.21.15.89:57160) 2024-09-10T08:45:03.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.672+0000 7f58b7400640 1 -- 172.21.15.89:0/3565820238 learned_addr learned my addr 172.21.15.89:0/3565820238 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:03.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.673+0000 7f58b6a00640 1 --2- 172.21.15.89:0/3565820238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e0554e0 0x55a81e05d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:03.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.674+0000 7f58b7400640 1 -- 172.21.15.89:0/3565820238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a81e0606c0 msgr2=0x55a81e05e0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:45:03.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.674+0000 7f58b7400640 1 --2- 172.21.15.89:0/3565820238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a81e0606c0 0x55a81e05e0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.674+0000 7f58b7400640 1 -- 172.21.15.89:0/3565820238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e0554e0 msgr2=0x55a81e05d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:03.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.674+0000 7f58b7400640 1 --2- 172.21.15.89:0/3565820238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e0554e0 0x55a81e05d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.674+0000 7f58b7400640 1 -- 172.21.15.89:0/3565820238 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a81e06c780 con 0x55a81e055e00 2024-09-10T08:45:03.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.674+0000 7f58b6a00640 1 --2- 172.21.15.89:0/3565820238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e0554e0 0x55a81e05d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:03.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.675+0000 7f58a1600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:45:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.675+0000 7f58b7400640 1 --2- 172.21.15.89:0/3565820238 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e055e00 0x55a81e05db70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f58b000fc00 tx=0x7f58b000c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16187c26ea9c3097 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.675+0000 7f58a1600640 1 -- 172.21.15.89:0/3565820238 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b000ce70 con 0x55a81e055e00 2024-09-10T08:45:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.675+0000 7f58a1600640 1 -- 172.21.15.89:0/3565820238 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f58b0003970 con 0x55a81e055e00 2024-09-10T08:45:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.676+0000 7f58c1086740 1 -- 172.21.15.89:0/3565820238 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58a80258a0 con 0x55a81e055e00 2024-09-10T08:45:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.676+0000 7f58a1600640 1 -- 172.21.15.89:0/3565820238 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f58b0003b70 con 0x55a81e055e00 2024-09-10T08:45:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.676+0000 7f58c1086740 10 client.26311 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:03.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.676+0000 7f58c1086740 1 -- 172.21.15.89:0/3565820238 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a81e06f1a0 con 0x55a81e055e00 2024-09-10T08:45:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.677+0000 7f58a1600640 1 -- 172.21.15.89:0/3565820238 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58a80258a0 con 0x55a81e055e00 2024-09-10T08:45:03.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.677+0000 7f58a1600640 1 -- 172.21.15.89:0/3565820238 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f58b000b040 con 0x55a81e055e00 2024-09-10T08:45:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.677+0000 7f58c1086740 10 client.26311 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.677+0000 7f58c1086740 10 client.26311 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:45:03.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.677+0000 7f58c1086740 20 client.26311 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:03.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.677+0000 7f58c1086740 1 --2- 172.21.15.89:0/3565820238 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a81e070700 0x55a81e072b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.677+0000 7f58c1086740 4 client.26311 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:45:03.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.677+0000 7f58c1086740 1 -- 172.21.15.89:0/3565820238 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a81dfc7990 con 0x55a81e070700 2024-09-10T08:45:03.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.678+0000 7f58b6a00640 1 --2- 172.21.15.89:0/3565820238 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a81e070700 0x55a81e072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:03.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.679+0000 7f58b6a00640 1 --2- 172.21.15.89:0/3565820238 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a81e070700 0x55a81e072b60 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3492a5903d1d867f server_cookie=4ec479e54f3996 in_seq=0 out_seq=0 2024-09-10T08:45:03.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.679+0000 7f58a1600640 10 client.26311 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:03.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.681+0000 7f58a1600640 1 -- 172.21.15.89:0/3565820238 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a81dfc7990 con 0x55a81e070700 2024-09-10T08:45:03.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.681+0000 7f58a1600640 10 client.26311 handle_command_reply: tid=0 2024-09-10T08:45:03.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.681+0000 7f58c1086740 1 client.26311 shutdown 2024-09-10T08:45:03.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.682+0000 7f58c1086740 1 -- 172.21.15.89:0/3565820238 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e055e00 msgr2=0x55a81e05db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:03.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.682+0000 7f58c1086740 1 --2- 172.21.15.89:0/3565820238 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e055e00 0x55a81e05db70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f58b000fc00 tx=0x7f58b000c740 comp rx=0 tx=0).stop 2024-09-10T08:45:03.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.682+0000 7f58c1086740 1 -- 172.21.15.89:0/3565820238 shutdown_connections 2024-09-10T08:45:03.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.682+0000 7f58c1086740 1 --2- 172.21.15.89:0/3565820238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a81e0606c0 0x55a81e05e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.682+0000 7f58c1086740 1 --2- 172.21.15.89:0/3565820238 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e055e00 0x55a81e05db70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.682+0000 7f58c1086740 1 --2- 172.21.15.89:0/3565820238 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a81e070700 0x55a81e072b60 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.682+0000 7f58c1086740 1 --2- 172.21.15.89:0/3565820238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e0554e0 0x55a81e05d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.682+0000 7f58a1600640 0 client.26311 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:03.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.682+0000 7f58c1086740 1 -- 172.21.15.89:0/3565820238 >> 172.21.15.89:0/3565820238 conn(0x55a81e051a70 msgr2=0x55a81e0577a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:03.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.682+0000 7f58c1086740 1 -- 172.21.15.89:0/3565820238 shutdown_connections 2024-09-10T08:45:03.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.682+0000 7f58c1086740 1 -- 172.21.15.89:0/3565820238 wait complete. 2024-09-10T08:45:03.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.683+0000 7f58c1086740 20 client.26311 trim_cache size 0 max 16384 2024-09-10T08:45:03.690 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:45:03.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.713+0000 7f58c1086740 1 Processor -- start 2024-09-10T08:45:03.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.713+0000 7f58c1086740 1 -- start start 2024-09-10T08:45:03.715 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.714+0000 7f58c1086740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e0554e0 0x55a81e078470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.716 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.714+0000 7f58c1086740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e055e00 0x55a81e0789b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.716+0000 7f58b6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e0554e0 0x55a81e078470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:03.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.716+0000 7f58c1086740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a81e0606c0 0x55a81e05a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.716+0000 7f58c1086740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f58b81b7cc0 con 0x55a81e055e00 2024-09-10T08:45:03.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.716+0000 7f58c1086740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f58b81b5590 con 0x55a81e0606c0 2024-09-10T08:45:03.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.716+0000 7f58c1086740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f58b81b33e0 con 0x55a81e0554e0 2024-09-10T08:45:03.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.716+0000 7f58b6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e0554e0 0x55a81e078470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:57172/0 (socket says 172.21.15.89:57172) 2024-09-10T08:45:03.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.717+0000 7f58b6a00640 1 -- 172.21.15.89:0/1099300146 learned_addr learned my addr 172.21.15.89:0/1099300146 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:03.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.717+0000 7f58b7400640 1 --2- 172.21.15.89:0/1099300146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e055e00 0x55a81e0789b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:03.718 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.717+0000 7f58b6a00640 1 --2- 172.21.15.89:0/1099300146 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a81e0606c0 0x55a81e05a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:03.719 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.719+0000 7f58b6a00640 1 -- 172.21.15.89:0/1099300146 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a81e0606c0 msgr2=0x55a81e05a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:03.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.719+0000 7f58b6a00640 1 --2- 172.21.15.89:0/1099300146 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a81e0606c0 0x55a81e05a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.719+0000 7f58b6a00640 1 -- 172.21.15.89:0/1099300146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e055e00 msgr2=0x55a81e0789b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:03.720 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.719+0000 7f58b6a00640 1 --2- 172.21.15.89:0/1099300146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e055e00 0x55a81e0789b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.719+0000 7f58b6a00640 1 -- 172.21.15.89:0/1099300146 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a81e055c30 con 0x55a81e0554e0 2024-09-10T08:45:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.719+0000 7f58b7400640 1 --2- 172.21.15.89:0/1099300146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e055e00 0x55a81e0789b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:03.721 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.720+0000 7f58b6a00640 1 --2- 172.21.15.89:0/1099300146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e0554e0 0x55a81e078470 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f58a800f4b0 tx=0x7f58a8005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a061f1d47b1214fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:03.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.720+0000 7f58a3e00640 1 -- 172.21.15.89:0/1099300146 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8021040 con 0x55a81e0554e0 2024-09-10T08:45:03.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.720+0000 7f58a3e00640 1 -- 172.21.15.89:0/1099300146 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58a80257b0 con 0x55a81e0554e0 2024-09-10T08:45:03.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.720+0000 7f58a3e00640 1 -- 172.21.15.89:0/1099300146 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a80048c0 con 0x55a81e0554e0 2024-09-10T08:45:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.721+0000 7f58c1086740 1 -- 172.21.15.89:0/1099300146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e0554e0 msgr2=0x55a81e078470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.721+0000 7f58c1086740 1 --2- 172.21.15.89:0/1099300146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e0554e0 0x55a81e078470 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f58a800f4b0 tx=0x7f58a8005c50 comp rx=0 tx=0).stop 2024-09-10T08:45:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.721+0000 7f58c1086740 1 -- 172.21.15.89:0/1099300146 shutdown_connections 2024-09-10T08:45:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.721+0000 7f58c1086740 1 --2- 172.21.15.89:0/1099300146 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a81e0606c0 0x55a81e05a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.721+0000 7f58c1086740 1 --2- 172.21.15.89:0/1099300146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e055e00 0x55a81e0789b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.721+0000 7f58c1086740 1 --2- 172.21.15.89:0/1099300146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e0554e0 0x55a81e078470 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.721+0000 7f58c1086740 1 -- 172.21.15.89:0/1099300146 >> 172.21.15.89:0/1099300146 conn(0x55a81e051a70 msgr2=0x55a81e05f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:03.723 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.722+0000 7f58c1086740 1 -- 172.21.15.89:0/1099300146 shutdown_connections 2024-09-10T08:45:03.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.722+0000 7f58c1086740 1 -- 172.21.15.89:0/1099300146 wait complete. 2024-09-10T08:45:03.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.723+0000 7f58c1086740 1 -- start start 2024-09-10T08:45:03.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.723+0000 7f58c1086740 1 Processor -- start 2024-09-10T08:45:03.724 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.724+0000 7f58c1086740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a81e0554e0 0x55a81e05a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.724+0000 7f58c1086740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e055e00 0x55a81e05abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.724+0000 7f58c1086740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e0606c0 0x55a81e05b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.724+0000 7f58c1086740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f58b81b3a50 con 0x55a81e055e00 2024-09-10T08:45:03.725 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.724+0000 7f58c1086740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f58b81b4ec0 con 0x55a81e0554e0 2024-09-10T08:45:03.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.724+0000 7f58c1086740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f58b81b4120 con 0x55a81e0606c0 2024-09-10T08:45:03.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.724+0000 7f58c1086740 10 client.0 _ll_register_callbacks cb 0x7f58b0009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:03.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.725+0000 7f58b6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e0606c0 0x55a81e05b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:03.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.725+0000 7f58b6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e0606c0 0x55a81e05b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:57184/0 (socket says 172.21.15.89:57184) 2024-09-10T08:45:03.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.725+0000 7f58b6a00640 1 -- 172.21.15.89:0/1065509286 learned_addr learned my addr 172.21.15.89:0/1065509286 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:03.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.725+0000 7f58b7400640 1 --2- 172.21.15.89:0/1065509286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e055e00 0x55a81e05abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:03.726 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.725+0000 7f58b6a00640 1 --2- 172.21.15.89:0/1065509286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a81e0554e0 0x55a81e05a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:03.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.726+0000 7f58b6a00640 1 -- 172.21.15.89:0/1065509286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a81e0554e0 msgr2=0x55a81e05a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:03.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.726+0000 7f58b6a00640 1 --2- 172.21.15.89:0/1065509286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a81e0554e0 0x55a81e05a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.726+0000 7f58b6a00640 1 -- 172.21.15.89:0/1065509286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e055e00 msgr2=0x55a81e05abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:03.727 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.726+0000 7f58b6a00640 1 --2- 172.21.15.89:0/1065509286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e055e00 0x55a81e05abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.726+0000 7f58b6a00640 1 -- 172.21.15.89:0/1065509286 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a81e0da470 con 0x55a81e0606c0 2024-09-10T08:45:03.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.726+0000 7f58b7400640 1 --2- 172.21.15.89:0/1065509286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e055e00 0x55a81e05abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:03.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.727+0000 7f58b6a00640 1 --2- 172.21.15.89:0/1065509286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e0606c0 0x55a81e05b120 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f58a800e4a0 tx=0x7f58a800e890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a94f2cc974d0c99b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:03.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.727+0000 7f58a1600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:45:03.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.727+0000 7f58a1600640 1 -- 172.21.15.89:0/1065509286 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8021070 con 0x55a81e0606c0 2024-09-10T08:45:03.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.727+0000 7f58a1600640 1 -- 172.21.15.89:0/1065509286 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f58a8011020 con 0x55a81e0606c0 2024-09-10T08:45:03.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.727+0000 7f58c1086740 1 -- 172.21.15.89:0/1065509286 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58a80257b0 con 0x55a81e0606c0 2024-09-10T08:45:03.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.728+0000 7f58a1600640 1 -- 172.21.15.89:0/1065509286 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f58a8011220 con 0x55a81e0606c0 2024-09-10T08:45:03.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.728+0000 7f58c1086740 10 client.26317 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:03.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.728+0000 7f58c1086740 1 -- 172.21.15.89:0/1065509286 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a81e0db710 con 0x55a81e0606c0 2024-09-10T08:45:03.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.728+0000 7f58a1600640 1 -- 172.21.15.89:0/1065509286 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58a80257b0 con 0x55a81e0606c0 2024-09-10T08:45:03.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.729+0000 7f58a1600640 1 -- 172.21.15.89:0/1065509286 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f58a803c070 con 0x55a81e0606c0 2024-09-10T08:45:03.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.729+0000 7f58c1086740 10 client.26317 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:03.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.729+0000 7f58c1086740 10 client.26317 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:45:03.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.729+0000 7f58c1086740 20 client.26317 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:03.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.729+0000 7f58c1086740 1 --2- 172.21.15.89:0/1065509286 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a81e0dbb00 0x55a81e0ddef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.730+0000 7f58c1086740 4 client.26317 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-09-10T08:45:03.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.730+0000 7f58c1086740 1 -- 172.21.15.89:0/1065509286 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x55a81dfc1ce0 con 0x55a81e0dbb00 2024-09-10T08:45:03.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.732+0000 7f58b7400640 1 --2- 172.21.15.89:0/1065509286 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a81e0dbb00 0x55a81e0ddef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:03.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.733+0000 7f58b7400640 1 --2- 172.21.15.89:0/1065509286 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a81e0dbb00 0x55a81e0ddef0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c9e2bedb633dbce server_cookie=223807cb903009b9 in_seq=0 out_seq=0 2024-09-10T08:45:03.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:03.733+0000 7f58a1600640 10 client.26317 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:04.249 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:05.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.010+0000 7fe774a00640 1 -- 172.21.15.89:0/2346241513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe770109120 msgr2=0x7fe770110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.010+0000 7fe774a00640 1 --2- 172.21.15.89:0/2346241513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe770109120 0x7fe770110ca0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe768009ca0 tx=0x7fe76802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.010+0000 7fe774a00640 1 -- 172.21.15.89:0/2346241513 shutdown_connections 2024-09-10T08:45:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.010+0000 7fe774a00640 1 --2- 172.21.15.89:0/2346241513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe770109120 0x7fe770110ca0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.010+0000 7fe774a00640 1 --2- 172.21.15.89:0/2346241513 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7701049a0 0x7fe7701089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.010+0000 7fe774a00640 1 --2- 172.21.15.89:0/2346241513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe770103ff0 0x7fe7701043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.010+0000 7fe774a00640 1 -- 172.21.15.89:0/2346241513 >> 172.21.15.89:0/2346241513 conn(0x7fe7700ffa90 msgr2=0x7fe770101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.010+0000 7fe774a00640 1 -- 172.21.15.89:0/2346241513 shutdown_connections 2024-09-10T08:45:05.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.010+0000 7fe774a00640 1 -- 172.21.15.89:0/2346241513 wait complete. 2024-09-10T08:45:05.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.011+0000 7fe774a00640 1 Processor -- start 2024-09-10T08:45:05.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.011+0000 7fe774a00640 1 -- start start 2024-09-10T08:45:05.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.012+0000 7fe774a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe770103ff0 0x7fe77019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.012+0000 7fe774a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7701049a0 0x7fe77019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.012+0000 7fe774a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe770109120 0x7fe77019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.012+0000 7fe774a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe7701141a0 con 0x7fe770109120 2024-09-10T08:45:05.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.012+0000 7fe774a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe770114020 con 0x7fe7701049a0 2024-09-10T08:45:05.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.012+0000 7fe774a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe770114320 con 0x7fe770103ff0 2024-09-10T08:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.012+0000 7fe767e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7701049a0 0x7fe77019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.012+0000 7fe767e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7701049a0 0x7fe77019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50378/0 (socket says 172.21.15.89:50378) 2024-09-10T08:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.012+0000 7fe767e00640 1 -- 172.21.15.89:0/4008313878 learned_addr learned my addr 172.21.15.89:0/4008313878 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.012+0000 7fe76de00640 1 --2- 172.21.15.89:0/4008313878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe770103ff0 0x7fe77019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.012+0000 7fe76e800640 1 --2- 172.21.15.89:0/4008313878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe770109120 0x7fe77019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.013+0000 7fe76de00640 1 -- 172.21.15.89:0/4008313878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7701049a0 msgr2=0x7fe77019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.013+0000 7fe76de00640 1 --2- 172.21.15.89:0/4008313878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7701049a0 0x7fe77019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.013+0000 7fe76de00640 1 -- 172.21.15.89:0/4008313878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe770109120 msgr2=0x7fe77019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.013+0000 7fe76de00640 1 --2- 172.21.15.89:0/4008313878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe770109120 0x7fe77019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.013+0000 7fe76de00640 1 -- 172.21.15.89:0/4008313878 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe77019f590 con 0x7fe770103ff0 2024-09-10T08:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.013+0000 7fe76e800640 1 --2- 172.21.15.89:0/4008313878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe770109120 0x7fe77019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:05.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.013+0000 7fe76de00640 1 --2- 172.21.15.89:0/4008313878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe770103ff0 0x7fe77019a5e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe76000d720 tx=0x7fe76000dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.013+0000 7fe766a00640 1 -- 172.21.15.89:0/4008313878 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760017070 con 0x7fe770103ff0 2024-09-10T08:45:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.013+0000 7fe766a00640 1 -- 172.21.15.89:0/4008313878 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe76000de30 con 0x7fe770103ff0 2024-09-10T08:45:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.013+0000 7fe774a00640 1 -- 172.21.15.89:0/4008313878 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe77019f880 con 0x7fe770103ff0 2024-09-10T08:45:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.014+0000 7fe774a00640 1 -- 172.21.15.89:0/4008313878 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7701a7160 con 0x7fe770103ff0 2024-09-10T08:45:05.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.014+0000 7fe766a00640 1 -- 172.21.15.89:0/4008313878 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7600047a0 con 0x7fe770103ff0 2024-09-10T08:45:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.017+0000 7fe7775d0740 1 Processor -- start 2024-09-10T08:45:05.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.017+0000 7fe7775d0740 1 -- start start 2024-09-10T08:45:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.017+0000 7fe7775d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d18b90 0x563275d18f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.017+0000 7fe7775d0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d194b0 0x563275d23830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.017+0000 7fe7775d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d23d70 0x563275d26160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.017+0000 7fe7775d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563275c7f3e0 con 0x563275d18b90 2024-09-10T08:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.017+0000 7fe7775d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563275c26d80 con 0x563275d23d70 2024-09-10T08:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.017+0000 7fe7775d0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563275c7b3a0 con 0x563275d194b0 2024-09-10T08:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.018+0000 7fe76e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d18b90 0x563275d18f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.018+0000 7fe76e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d18b90 0x563275d18f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60578/0 (socket says 172.21.15.89:60578) 2024-09-10T08:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.018+0000 7fe76e800640 1 -- 172.21.15.89:0/346786159 learned_addr learned my addr 172.21.15.89:0/346786159 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.018+0000 7fe76e800640 1 -- 172.21.15.89:0/346786159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d194b0 msgr2=0x563275d23830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.018+0000 7fe76e800640 1 --2- 172.21.15.89:0/346786159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d194b0 0x563275d23830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.018+0000 7fe76e800640 1 -- 172.21.15.89:0/346786159 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d23d70 msgr2=0x563275d26160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.018+0000 7fe76e800640 1 --2- 172.21.15.89:0/346786159 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d23d70 0x563275d26160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.018+0000 7fe76e800640 1 -- 172.21.15.89:0/346786159 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563275c45ae0 con 0x563275d18b90 2024-09-10T08:45:05.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.018+0000 7fe766a00640 1 -- 172.21.15.89:0/4008313878 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe760004940 con 0x7fe770103ff0 2024-09-10T08:45:05.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.018+0000 7fe76e800640 1 --2- 172.21.15.89:0/346786159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d18b90 0x563275d18f70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe768009ca0 tx=0x7fe768033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d1924956724bfe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.019+0000 7fe75f000640 1 -- 172.21.15.89:0/346786159 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768003b70 con 0x563275d18b90 2024-09-10T08:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.019+0000 7fe75f000640 1 -- 172.21.15.89:0/346786159 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe768036040 con 0x563275d18b90 2024-09-10T08:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.019+0000 7fe75f000640 1 -- 172.21.15.89:0/346786159 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768004650 con 0x563275d18b90 2024-09-10T08:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.020+0000 7fe7775d0740 1 -- 172.21.15.89:0/346786159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d18b90 msgr2=0x563275d18f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.020+0000 7fe7775d0740 1 --2- 172.21.15.89:0/346786159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d18b90 0x563275d18f70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe768009ca0 tx=0x7fe768033000 comp rx=0 tx=0).stop 2024-09-10T08:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.020+0000 7fe7775d0740 1 -- 172.21.15.89:0/346786159 shutdown_connections 2024-09-10T08:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.020+0000 7fe7775d0740 1 --2- 172.21.15.89:0/346786159 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d23d70 0x563275d26160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.020+0000 7fe7775d0740 1 --2- 172.21.15.89:0/346786159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d194b0 0x563275d23830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.020+0000 7fe7775d0740 1 --2- 172.21.15.89:0/346786159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d18b90 0x563275d18f70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.020+0000 7fe7775d0740 1 -- 172.21.15.89:0/346786159 >> 172.21.15.89:0/346786159 conn(0x563275d15120 msgr2=0x563275d15500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:05.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.020+0000 7fe766a00640 1 --2- 172.21.15.89:0/4008313878 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe7540a2c00 0x7fe7540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.020+0000 7fe766a00640 1 -- 172.21.15.89:0/4008313878 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fe7600cee00 con 0x7fe770103ff0 2024-09-10T08:45:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.021+0000 7fe767e00640 1 --2- 172.21.15.89:0/4008313878 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe7540a2c00 0x7fe7540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.021+0000 7fe7775d0740 1 -- 172.21.15.89:0/346786159 shutdown_connections 2024-09-10T08:45:05.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.021+0000 7fe7775d0740 1 -- 172.21.15.89:0/346786159 wait complete. 2024-09-10T08:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.021+0000 7fe7775d0740 1 -- start start 2024-09-10T08:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.021+0000 7fe767e00640 1 --2- 172.21.15.89:0/4008313878 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe7540a2c00 0x7fe7540a50c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x563275d164a0 tx=0x7fe758005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.022+0000 7fe7775d0740 1 Processor -- start 2024-09-10T08:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.022+0000 7fe7775d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d18b90 0x563275d20c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.022+0000 7fe7775d0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d194b0 0x563275d211a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.022+0000 7fe7775d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d23d70 0x563275d216e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.022+0000 7fe7775d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563275d26860 con 0x563275d18b90 2024-09-10T08:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.022+0000 7fe7775d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563275c3a0f0 con 0x563275d23d70 2024-09-10T08:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.022+0000 7fe7775d0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563275d269e0 con 0x563275d194b0 2024-09-10T08:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.022+0000 7fe7775d0740 10 client.0 _ll_register_callbacks cb 0x563275c2b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.023+0000 7fe76de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d18b90 0x563275d20c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.023+0000 7fe767e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d194b0 0x563275d211a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.023+0000 7fe767e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d194b0 0x563275d211a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:57214/0 (socket says 172.21.15.89:57214) 2024-09-10T08:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.023+0000 7fe767e00640 1 -- 172.21.15.89:0/3101966995 learned_addr learned my addr 172.21.15.89:0/3101966995 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.023+0000 7fe76de00640 1 -- 172.21.15.89:0/3101966995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d194b0 msgr2=0x563275d211a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.023+0000 7fe76de00640 1 --2- 172.21.15.89:0/3101966995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d194b0 0x563275d211a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.023+0000 7fe76e800640 1 --2- 172.21.15.89:0/3101966995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d23d70 0x563275d216e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.023+0000 7fe76de00640 1 -- 172.21.15.89:0/3101966995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d23d70 msgr2=0x563275d216e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.023+0000 7fe76de00640 1 --2- 172.21.15.89:0/3101966995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d23d70 0x563275d216e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.023+0000 7fe76de00640 1 -- 172.21.15.89:0/3101966995 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632759f03c0 con 0x563275d18b90 2024-09-10T08:45:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.023+0000 7fe76de00640 1 --2- 172.21.15.89:0/3101966995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d18b90 0x563275d20c60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe7600cc500 tx=0x7fe76009ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0846aca0926839 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.023+0000 7fe76e800640 1 --2- 172.21.15.89:0/3101966995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d23d70 0x563275d216e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.023+0000 7fe753400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:45:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.024+0000 7fe753400640 1 -- 172.21.15.89:0/3101966995 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7600a3f00 con 0x563275d18b90 2024-09-10T08:45:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.024+0000 7fe753400640 1 -- 172.21.15.89:0/3101966995 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fe7600050d0 con 0x563275d18b90 2024-09-10T08:45:05.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.024+0000 7fe7775d0740 1 -- 172.21.15.89:0/3101966995 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe768036040 con 0x563275d18b90 2024-09-10T08:45:05.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.024+0000 7fe753400640 1 -- 172.21.15.89:0/3101966995 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe7600b4220 con 0x563275d18b90 2024-09-10T08:45:05.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.024+0000 7fe7775d0740 10 client.16362 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:05.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.024+0000 7fe7775d0740 1 -- 172.21.15.89:0/3101966995 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563275d325a0 con 0x563275d18b90 2024-09-10T08:45:05.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.024+0000 7fe753400640 1 -- 172.21.15.89:0/3101966995 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe768036040 con 0x563275d18b90 2024-09-10T08:45:05.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.025+0000 7fe753400640 1 -- 172.21.15.89:0/3101966995 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fe7600a3b00 con 0x563275d18b90 2024-09-10T08:45:05.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.025+0000 7fe7775d0740 10 client.16362 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:05.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.025+0000 7fe7775d0740 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:45:05.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.025+0000 7fe7775d0740 20 client.16362 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:05.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.025+0000 7fe7775d0740 1 --2- 172.21.15.89:0/3101966995 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563275d33a50 0x563275d35e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.025+0000 7fe7775d0740 4 client.16362 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:45:05.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.025+0000 7fe7775d0740 1 -- 172.21.15.89:0/3101966995 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563275c8c950 con 0x563275d33a50 2024-09-10T08:45:05.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.025+0000 7fe767e00640 1 --2- 172.21.15.89:0/3101966995 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563275d33a50 0x563275d35e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.026+0000 7fe767e00640 1 --2- 172.21.15.89:0/3101966995 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563275d33a50 0x563275d35e70 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4a81f85a0bd232d server_cookie=f14b1f88b0ddc099 in_seq=0 out_seq=0 2024-09-10T08:45:05.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.026+0000 7fe753400640 10 client.16362 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:05.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.063+0000 7fe753400640 1 -- 172.21.15.89:0/3101966995 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563275c8c950 con 0x563275d33a50 2024-09-10T08:45:05.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.063+0000 7fe753400640 10 client.16362 handle_command_reply: tid=0 2024-09-10T08:45:05.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.063+0000 7fe7775d0740 1 client.16362 shutdown 2024-09-10T08:45:05.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.063+0000 7fe7775d0740 1 -- 172.21.15.89:0/3101966995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d18b90 msgr2=0x563275d20c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:05.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.063+0000 7fe7775d0740 1 --2- 172.21.15.89:0/3101966995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d18b90 0x563275d20c60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe7600cc500 tx=0x7fe76009ba50 comp rx=0 tx=0).stop 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.063+0000 7fe7775d0740 1 -- 172.21.15.89:0/3101966995 shutdown_connections 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.063+0000 7fe7775d0740 1 --2- 172.21.15.89:0/3101966995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d23d70 0x563275d216e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.063+0000 7fe7775d0740 1 --2- 172.21.15.89:0/3101966995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d194b0 0x563275d211a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.063+0000 7fe7775d0740 1 --2- 172.21.15.89:0/3101966995 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563275d33a50 0x563275d35e70 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.063+0000 7fe7775d0740 1 --2- 172.21.15.89:0/3101966995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d18b90 0x563275d20c60 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.063+0000 7fe7775d0740 1 -- 172.21.15.89:0/3101966995 >> 172.21.15.89:0/3101966995 conn(0x563275d15120 msgr2=0x563275d1ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.063+0000 7fe753400640 0 client.16362 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.063+0000 7fe7775d0740 1 -- 172.21.15.89:0/3101966995 shutdown_connections 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.063+0000 7fe7775d0740 1 -- 172.21.15.89:0/3101966995 wait complete. 2024-09-10T08:45:05.067 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.064+0000 7fe7775d0740 20 client.16362 trim_cache size 0 max 16384 2024-09-10T08:45:05.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.088+0000 7fe7775d0740 1 Processor -- start 2024-09-10T08:45:05.089 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:04 smithi089 ceph-mon[30757]: pgmap v547: 129 pgs: 129 active+clean; 312 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.0 MiB/s wr, 1.03k op/s 2024-09-10T08:45:05.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.088+0000 7fe7775d0740 1 -- start start 2024-09-10T08:45:05.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.088+0000 7fe7775d0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d18b90 0x563275d1acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.088+0000 7fe7775d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d194b0 0x563275d21530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.088+0000 7fe7775d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d23d70 0x563275d3b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.088+0000 7fe7775d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c012500 con 0x563275d194b0 2024-09-10T08:45:05.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.088+0000 7fe7775d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c00fdd0 con 0x563275d23d70 2024-09-10T08:45:05.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.088+0000 7fe7775d0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c00dc20 con 0x563275d18b90 2024-09-10T08:45:05.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.089+0000 7fe767e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d194b0 0x563275d21530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.089+0000 7fe767e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d194b0 0x563275d21530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60586/0 (socket says 172.21.15.89:60586) 2024-09-10T08:45:05.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.089+0000 7fe767e00640 1 -- 172.21.15.89:0/2446586898 learned_addr learned my addr 172.21.15.89:0/2446586898 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:05.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.089+0000 7fe76de00640 1 --2- 172.21.15.89:0/2446586898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d18b90 0x563275d1acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.089+0000 7fe76de00640 1 -- 172.21.15.89:0/2446586898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d23d70 msgr2=0x563275d3b720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:45:05.091 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.089+0000 7fe76de00640 1 --2- 172.21.15.89:0/2446586898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d23d70 0x563275d3b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.089+0000 7fe76de00640 1 -- 172.21.15.89:0/2446586898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d194b0 msgr2=0x563275d21530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:05.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.089+0000 7fe76de00640 1 --2- 172.21.15.89:0/2446586898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d194b0 0x563275d21530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.089+0000 7fe76de00640 1 -- 172.21.15.89:0/2446586898 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563275d1fc10 con 0x563275d18b90 2024-09-10T08:45:05.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.089+0000 7fe767e00640 1 --2- 172.21.15.89:0/2446586898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d194b0 0x563275d21530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:05.092 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.090+0000 7fe76de00640 1 --2- 172.21.15.89:0/2446586898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d18b90 0x563275d1acf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe76009af90 tx=0x7fe7600997a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb3c74afaf32ea30 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:05.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.090+0000 7fe75e600640 1 -- 172.21.15.89:0/2446586898 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76009b060 con 0x563275d18b90 2024-09-10T08:45:05.093 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.090+0000 7fe75e600640 1 -- 172.21.15.89:0/2446586898 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7600b8020 con 0x563275d18b90 2024-09-10T08:45:05.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.090+0000 7fe75e600640 1 -- 172.21.15.89:0/2446586898 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76009f260 con 0x563275d18b90 2024-09-10T08:45:05.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.091+0000 7fe7775d0740 1 -- 172.21.15.89:0/2446586898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d18b90 msgr2=0x563275d1acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:05.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.091+0000 7fe7775d0740 1 --2- 172.21.15.89:0/2446586898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d18b90 0x563275d1acf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe76009af90 tx=0x7fe7600997a0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.091+0000 7fe7775d0740 1 -- 172.21.15.89:0/2446586898 shutdown_connections 2024-09-10T08:45:05.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.091+0000 7fe7775d0740 1 --2- 172.21.15.89:0/2446586898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d23d70 0x563275d3b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.091+0000 7fe7775d0740 1 --2- 172.21.15.89:0/2446586898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d194b0 0x563275d21530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.091+0000 7fe7775d0740 1 --2- 172.21.15.89:0/2446586898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d18b90 0x563275d1acf0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.091+0000 7fe7775d0740 1 -- 172.21.15.89:0/2446586898 >> 172.21.15.89:0/2446586898 conn(0x563275d15120 msgr2=0x563275d1a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:05.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.091+0000 7fe7775d0740 1 -- 172.21.15.89:0/2446586898 shutdown_connections 2024-09-10T08:45:05.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.091+0000 7fe7775d0740 1 -- 172.21.15.89:0/2446586898 wait complete. 2024-09-10T08:45:05.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.091+0000 7fe7775d0740 1 -- start start 2024-09-10T08:45:05.094 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.092+0000 7fe7775d0740 1 Processor -- start 2024-09-10T08:45:05.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.092+0000 7fe7775d0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d18b90 0x563275d2c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.092+0000 7fe7775d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d194b0 0x563275d27740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.092+0000 7fe7775d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d23d70 0x563275d27c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.092+0000 7fe7775d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c00e290 con 0x563275d23d70 2024-09-10T08:45:05.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.092+0000 7fe7775d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c00f700 con 0x563275d194b0 2024-09-10T08:45:05.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.092+0000 7fe7775d0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c00e960 con 0x563275d18b90 2024-09-10T08:45:05.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.092+0000 7fe7775d0740 10 client.0 _ll_register_callbacks cb 0x7fe76009add0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:05.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.092+0000 7fe767e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d194b0 0x563275d27740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.092+0000 7fe767e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d194b0 0x563275d27740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50424/0 (socket says 172.21.15.89:50424) 2024-09-10T08:45:05.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.092+0000 7fe76de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d18b90 0x563275d2c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.095 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.092+0000 7fe767e00640 1 -- 172.21.15.89:0/112097882 learned_addr learned my addr 172.21.15.89:0/112097882 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:05.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.093+0000 7fe76e800640 1 --2- 172.21.15.89:0/112097882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d23d70 0x563275d27c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.097 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.093+0000 7fe767e00640 1 -- 172.21.15.89:0/112097882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d18b90 msgr2=0x563275d2c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:05.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.093+0000 7fe767e00640 1 --2- 172.21.15.89:0/112097882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d18b90 0x563275d2c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.093+0000 7fe767e00640 1 -- 172.21.15.89:0/112097882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d23d70 msgr2=0x563275d27c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:05.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.093+0000 7fe767e00640 1 --2- 172.21.15.89:0/112097882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d23d70 0x563275d27c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.093+0000 7fe767e00640 1 -- 172.21.15.89:0/112097882 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563275d28460 con 0x563275d194b0 2024-09-10T08:45:05.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.093+0000 7fe76de00640 1 --2- 172.21.15.89:0/112097882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d18b90 0x563275d2c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:05.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.093+0000 7fe76e800640 1 --2- 172.21.15.89:0/112097882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d23d70 0x563275d27c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:05.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.093+0000 7fe767e00640 1 --2- 172.21.15.89:0/112097882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d194b0 0x563275d27740 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe758004520 tx=0x7fe758013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8628c8a12676b6b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:05.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.093+0000 7fe753400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:45:05.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.093+0000 7fe753400640 1 -- 172.21.15.89:0/112097882 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758005230 con 0x563275d194b0 2024-09-10T08:45:05.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.094+0000 7fe7775d0740 1 -- 172.21.15.89:0/112097882 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7600b8020 con 0x563275d194b0 2024-09-10T08:45:05.098 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.094+0000 7fe753400640 1 -- 172.21.15.89:0/112097882 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fe7580053d0 con 0x563275d194b0 2024-09-10T08:45:05.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.094+0000 7fe753400640 1 -- 172.21.15.89:0/112097882 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe758034050 con 0x563275d194b0 2024-09-10T08:45:05.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.094+0000 7fe7775d0740 10 client.26405 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:05.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.094+0000 7fe7775d0740 1 -- 172.21.15.89:0/112097882 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563275da6fb0 con 0x563275d194b0 2024-09-10T08:45:05.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.094+0000 7fe753400640 1 -- 172.21.15.89:0/112097882 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7600b8020 con 0x563275d194b0 2024-09-10T08:45:05.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.094+0000 7fe753400640 1 -- 172.21.15.89:0/112097882 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fe75801e930 con 0x563275d194b0 2024-09-10T08:45:05.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.094+0000 7fe7775d0740 10 client.26405 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:05.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.094+0000 7fe7775d0740 10 client.26405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:45:05.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.094+0000 7fe7775d0740 20 client.26405 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:05.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.094+0000 7fe7775d0740 1 --2- 172.21.15.89:0/112097882 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563275da7790 0x563275da9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.099 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.094+0000 7fe7775d0740 4 client.26405 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:45:05.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.094+0000 7fe7775d0740 1 -- 172.21.15.89:0/112097882 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563275c86e60 con 0x563275da7790 2024-09-10T08:45:05.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.095+0000 7fe76de00640 1 --2- 172.21.15.89:0/112097882 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563275da7790 0x563275da9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.095+0000 7fe76de00640 1 --2- 172.21.15.89:0/112097882 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563275da7790 0x563275da9bb0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e06b7c71dfc3c5a4 server_cookie=46b5be4470e75c6e in_seq=0 out_seq=0 2024-09-10T08:45:05.100 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.095+0000 7fe753400640 10 client.26405 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:05.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.138+0000 7fe753400640 1 -- 172.21.15.89:0/112097882 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563275c86e60 con 0x563275da7790 2024-09-10T08:45:05.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.138+0000 7fe753400640 10 client.26405 handle_command_reply: tid=0 2024-09-10T08:45:05.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.138+0000 7fe7775d0740 1 client.26405 shutdown 2024-09-10T08:45:05.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.138+0000 7fe7775d0740 1 -- 172.21.15.89:0/112097882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d194b0 msgr2=0x563275d27740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:05.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.138+0000 7fe7775d0740 1 --2- 172.21.15.89:0/112097882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d194b0 0x563275d27740 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe758004520 tx=0x7fe758013040 comp rx=0 tx=0).stop 2024-09-10T08:45:05.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.139+0000 7fe7775d0740 1 -- 172.21.15.89:0/112097882 shutdown_connections 2024-09-10T08:45:05.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.139+0000 7fe7775d0740 1 --2- 172.21.15.89:0/112097882 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563275da7790 0x563275da9bb0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.139+0000 7fe7775d0740 1 --2- 172.21.15.89:0/112097882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563275d23d70 0x563275d27c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.139+0000 7fe7775d0740 1 --2- 172.21.15.89:0/112097882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563275d194b0 0x563275d27740 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.139+0000 7fe7775d0740 1 --2- 172.21.15.89:0/112097882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563275d18b90 0x563275d2c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.139+0000 7fe7775d0740 1 -- 172.21.15.89:0/112097882 >> 172.21.15.89:0/112097882 conn(0x563275d15120 msgr2=0x563275d1ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:05.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.139+0000 7fe753400640 0 client.26405 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:05.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.139+0000 7fe7775d0740 1 -- 172.21.15.89:0/112097882 shutdown_connections 2024-09-10T08:45:05.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.139+0000 7fe7775d0740 1 -- 172.21.15.89:0/112097882 wait complete. 2024-09-10T08:45:05.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.139+0000 7fe7775d0740 20 client.26405 trim_cache size 0 max 16384 2024-09-10T08:45:05.141 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:45:05.141 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2024-09-10T08:45:05.142 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2024-09-10T08:45:05.142 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:45:05.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.140+0000 7fe774a00640 1 -- 172.21.15.89:0/4008313878 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe7540a2c00 msgr2=0x7fe7540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:05.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.140+0000 7fe774a00640 1 --2- 172.21.15.89:0/4008313878 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe7540a2c00 0x7fe7540a50c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x563275d164a0 tx=0x7fe758005f90 comp rx=0 tx=0).stop 2024-09-10T08:45:05.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.140+0000 7fe774a00640 1 -- 172.21.15.89:0/4008313878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe770103ff0 msgr2=0x7fe77019a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:05.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.140+0000 7fe774a00640 1 --2- 172.21.15.89:0/4008313878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe770103ff0 0x7fe77019a5e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe76000d720 tx=0x7fe76000dbe0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.140+0000 7fe774a00640 1 -- 172.21.15.89:0/4008313878 shutdown_connections 2024-09-10T08:45:05.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.140+0000 7fe774a00640 1 --2- 172.21.15.89:0/4008313878 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe7540a2c00 0x7fe7540a50c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.140+0000 7fe774a00640 1 --2- 172.21.15.89:0/4008313878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe770109120 0x7fe77019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.140+0000 7fe774a00640 1 --2- 172.21.15.89:0/4008313878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7701049a0 0x7fe77019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.140+0000 7fe774a00640 1 --2- 172.21.15.89:0/4008313878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe770103ff0 0x7fe77019a5e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.142 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.140+0000 7fe774a00640 1 -- 172.21.15.89:0/4008313878 >> 172.21.15.89:0/4008313878 conn(0x7fe7700ffa90 msgr2=0x7fe7701005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:05.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.140+0000 7fe774a00640 1 -- 172.21.15.89:0/4008313878 shutdown_connections 2024-09-10T08:45:05.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:05.140+0000 7fe774a00640 1 -- 172.21.15.89:0/4008313878 wait complete. 2024-09-10T08:45:05.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:04 smithi149 ceph-mon[36167]: pgmap v547: 129 pgs: 129 active+clean; 312 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.0 MiB/s wr, 1.03k op/s 2024-09-10T08:45:05.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:04 smithi111 ceph-mon[35220]: pgmap v547: 129 pgs: 129 active+clean; 312 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.0 MiB/s wr, 1.03k op/s 2024-09-10T08:45:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.398+0000 7ff46b200640 1 -- 172.21.15.89:0/2169918071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff46c103050 msgr2=0x7ff46c103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.398+0000 7ff46b200640 1 --2- 172.21.15.89:0/2169918071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff46c103050 0x7ff46c103450 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009900 tx=0x7ff45c02a410 comp rx=0 tx=0).stop 2024-09-10T08:45:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.398+0000 7ff46b200640 1 -- 172.21.15.89:0/2169918071 shutdown_connections 2024-09-10T08:45:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.398+0000 7ff46b200640 1 --2- 172.21.15.89:0/2169918071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff46c101f40 0x7ff46c109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.398+0000 7ff46b200640 1 --2- 172.21.15.89:0/2169918071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff46c101580 0x7ff46c101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.398+0000 7ff46b200640 1 --2- 172.21.15.89:0/2169918071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff46c103050 0x7ff46c103450 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.398+0000 7ff46b200640 1 -- 172.21.15.89:0/2169918071 >> 172.21.15.89:0/2169918071 conn(0x7ff46c0fd0d0 msgr2=0x7ff46c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.398+0000 7ff46b200640 1 -- 172.21.15.89:0/2169918071 shutdown_connections 2024-09-10T08:45:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.398+0000 7ff46b200640 1 -- 172.21.15.89:0/2169918071 wait complete. 2024-09-10T08:45:05.402 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.399+0000 7ff46b200640 1 Processor -- start 2024-09-10T08:45:05.402 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.399+0000 7ff46b200640 1 -- start start 2024-09-10T08:45:05.402 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.400+0000 7ff46b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff46c101580 0x7ff46c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.400+0000 7ff46b200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff46c101f40 0x7ff46c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.400+0000 7ff46b200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff46c103050 0x7ff46c19f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.400+0000 7ff46b200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c10d6f0 con 0x7ff46c101f40 2024-09-10T08:45:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.400+0000 7ff46b200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c10d570 con 0x7ff46c103050 2024-09-10T08:45:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.400+0000 7ff46b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c10d870 con 0x7ff46c101580 2024-09-10T08:45:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.400+0000 7ff46a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff46c103050 0x7ff46c19f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.400+0000 7ff46a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff46c103050 0x7ff46c19f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50442/0 (socket says 172.21.15.89:50442) 2024-09-10T08:45:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.400+0000 7ff46a800640 1 -- 172.21.15.89:0/1805389844 learned_addr learned my addr 172.21.15.89:0/1805389844 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.400+0000 7ff469400640 1 --2- 172.21.15.89:0/1805389844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff46c101f40 0x7ff46c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.400+0000 7ff469e00640 1 --2- 172.21.15.89:0/1805389844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff46c101580 0x7ff46c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.400+0000 7ff46a800640 1 -- 172.21.15.89:0/1805389844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff46c101580 msgr2=0x7ff46c197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.400+0000 7ff46a800640 1 --2- 172.21.15.89:0/1805389844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff46c101580 0x7ff46c197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.400+0000 7ff46a800640 1 -- 172.21.15.89:0/1805389844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff46c101f40 msgr2=0x7ff46c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.400+0000 7ff46a800640 1 --2- 172.21.15.89:0/1805389844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff46c101f40 0x7ff46c198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.400+0000 7ff46a800640 1 -- 172.21.15.89:0/1805389844 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46c19fa10 con 0x7ff46c103050 2024-09-10T08:45:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.401+0000 7ff46a800640 1 --2- 172.21.15.89:0/1805389844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff46c103050 0x7ff46c19f3a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff46400eb10 tx=0x7ff46400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.401+0000 7ff462a00640 1 -- 172.21.15.89:0/1805389844 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464004b20 con 0x7ff46c103050 2024-09-10T08:45:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.401+0000 7ff462a00640 1 -- 172.21.15.89:0/1805389844 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff464004cc0 con 0x7ff46c103050 2024-09-10T08:45:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.401+0000 7ff46b200640 1 -- 172.21.15.89:0/1805389844 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff46c19fd00 con 0x7ff46c103050 2024-09-10T08:45:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.401+0000 7ff462a00640 1 -- 172.21.15.89:0/1805389844 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464010760 con 0x7ff46c103050 2024-09-10T08:45:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.401+0000 7ff46b200640 1 -- 172.21.15.89:0/1805389844 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff46c1a01a0 con 0x7ff46c103050 2024-09-10T08:45:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.404+0000 7ff473e88740 1 Processor -- start 2024-09-10T08:45:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.404+0000 7ff462a00640 1 -- 172.21.15.89:0/1805389844 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff464005020 con 0x7ff46c103050 2024-09-10T08:45:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.404+0000 7ff473e88740 1 -- start start 2024-09-10T08:45:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.405+0000 7ff473e88740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562316787670 0x562316787a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.405+0000 7ff473e88740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316787fb0 0x56231678ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.405+0000 7ff473e88740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56231678d040 0x56231678f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.405+0000 7ff473e88740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5623166e8100 con 0x562316787fb0 2024-09-10T08:45:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.405+0000 7ff473e88740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5623166901b0 con 0x562316787670 2024-09-10T08:45:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.405+0000 7ff473e88740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5623166e40c0 con 0x56231678d040 2024-09-10T08:45:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.405+0000 7ff469e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316787fb0 0x56231678ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.405+0000 7ff469400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562316787670 0x562316787a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.405+0000 7ff469e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316787fb0 0x56231678ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60624/0 (socket says 172.21.15.89:60624) 2024-09-10T08:45:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.405+0000 7ff469e00640 1 -- 172.21.15.89:0/703636662 learned_addr learned my addr 172.21.15.89:0/703636662 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.405+0000 7ff469400640 1 --2- 172.21.15.89:0/703636662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56231678d040 0x56231678f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.405+0000 7ff469e00640 1 -- 172.21.15.89:0/703636662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56231678d040 msgr2=0x56231678f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.405+0000 7ff469e00640 1 --2- 172.21.15.89:0/703636662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56231678d040 0x56231678f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.405+0000 7ff469e00640 1 -- 172.21.15.89:0/703636662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562316787670 msgr2=0x562316787a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.405+0000 7ff469e00640 1 --2- 172.21.15.89:0/703636662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562316787670 0x562316787a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.405+0000 7ff469e00640 1 -- 172.21.15.89:0/703636662 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623163fb210 con 0x562316787fb0 2024-09-10T08:45:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.405+0000 7ff469400640 1 --2- 172.21.15.89:0/703636662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562316787670 0x562316787a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.406+0000 7ff469e00640 1 --2- 172.21.15.89:0/703636662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316787fb0 0x56231678ca70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff45c031440 tx=0x7ff45c005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98f60b561f4e59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.406+0000 7ff457400640 1 -- 172.21.15.89:0/703636662 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c03f070 con 0x562316787fb0 2024-09-10T08:45:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.406+0000 7ff462a00640 1 --2- 172.21.15.89:0/1805389844 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff4500a2b80 0x7ff4500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.406+0000 7ff457400640 1 -- 172.21.15.89:0/703636662 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff45c004030 con 0x562316787fb0 2024-09-10T08:45:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.406+0000 7ff457400640 1 -- 172.21.15.89:0/703636662 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c035070 con 0x562316787fb0 2024-09-10T08:45:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.406+0000 7ff469400640 1 --2- 172.21.15.89:0/1805389844 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff4500a2b80 0x7ff4500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.407+0000 7ff462a00640 1 -- 172.21.15.89:0/1805389844 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff4640cfa70 con 0x7ff46c103050 2024-09-10T08:45:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.407+0000 7ff473e88740 1 -- 172.21.15.89:0/703636662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316787fb0 msgr2=0x56231678ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.407+0000 7ff473e88740 1 --2- 172.21.15.89:0/703636662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316787fb0 0x56231678ca70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff45c031440 tx=0x7ff45c005e60 comp rx=0 tx=0).stop 2024-09-10T08:45:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.407+0000 7ff473e88740 1 -- 172.21.15.89:0/703636662 shutdown_connections 2024-09-10T08:45:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.407+0000 7ff473e88740 1 --2- 172.21.15.89:0/703636662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56231678d040 0x56231678f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.407+0000 7ff473e88740 1 --2- 172.21.15.89:0/703636662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316787fb0 0x56231678ca70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.407+0000 7ff473e88740 1 --2- 172.21.15.89:0/703636662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562316787670 0x562316787a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.407+0000 7ff473e88740 1 -- 172.21.15.89:0/703636662 >> 172.21.15.89:0/703636662 conn(0x562316782ce0 msgr2=0x5623167830e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.407+0000 7ff469400640 1 --2- 172.21.15.89:0/1805389844 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff4500a2b80 0x7ff4500a5040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff458009710 tx=0x7ff4580096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.407+0000 7ff473e88740 1 -- 172.21.15.89:0/703636662 shutdown_connections 2024-09-10T08:45:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.407+0000 7ff473e88740 1 -- 172.21.15.89:0/703636662 wait complete. 2024-09-10T08:45:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.408+0000 7ff473e88740 1 -- start start 2024-09-10T08:45:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.408+0000 7ff473e88740 1 Processor -- start 2024-09-10T08:45:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.408+0000 7ff473e88740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316787670 0x562316791780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.408+0000 7ff473e88740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562316787fb0 0x562316791cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.408+0000 7ff473e88740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56231678d040 0x562316798d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.408+0000 7ff473e88740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56231678ffb0 con 0x562316787670 2024-09-10T08:45:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.408+0000 7ff473e88740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562316671090 con 0x56231678d040 2024-09-10T08:45:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.408+0000 7ff473e88740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562316790130 con 0x562316787fb0 2024-09-10T08:45:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.408+0000 7ff473e88740 10 client.0 _ll_register_callbacks cb 0x5623166c5f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.408+0000 7ff469e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316787670 0x562316791780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.408+0000 7ff46a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56231678d040 0x562316798d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.408+0000 7ff469e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316787670 0x562316791780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60634/0 (socket says 172.21.15.89:60634) 2024-09-10T08:45:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.408+0000 7ff469e00640 1 -- 172.21.15.89:0/2196646063 learned_addr learned my addr 172.21.15.89:0/2196646063 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.409+0000 7ff469400640 1 --2- 172.21.15.89:0/2196646063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562316787fb0 0x562316791cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.409+0000 7ff46a800640 1 -- 172.21.15.89:0/2196646063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562316787fb0 msgr2=0x562316791cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.409+0000 7ff46a800640 1 --2- 172.21.15.89:0/2196646063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562316787fb0 0x562316791cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.409+0000 7ff46a800640 1 -- 172.21.15.89:0/2196646063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316787670 msgr2=0x562316791780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.409+0000 7ff46a800640 1 --2- 172.21.15.89:0/2196646063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316787670 0x562316791780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.409+0000 7ff46a800640 1 -- 172.21.15.89:0/2196646063 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623164c7620 con 0x56231678d040 2024-09-10T08:45:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.409+0000 7ff469400640 1 --2- 172.21.15.89:0/2196646063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562316787fb0 0x562316791cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.409+0000 7ff469e00640 1 --2- 172.21.15.89:0/2196646063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316787670 0x562316791780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.409+0000 7ff46a800640 1 --2- 172.21.15.89:0/2196646063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56231678d040 0x562316798d00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff4640cd1f0 tx=0x7ff46409b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35e0a6b98844aae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.409+0000 7ff44be00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:45:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.409+0000 7ff44be00640 1 -- 172.21.15.89:0/2196646063 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4640a48e0 con 0x56231678d040 2024-09-10T08:45:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.409+0000 7ff473e88740 1 -- 172.21.15.89:0/2196646063 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff45c004030 con 0x56231678d040 2024-09-10T08:45:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.409+0000 7ff44be00640 1 -- 172.21.15.89:0/2196646063 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff464005310 con 0x56231678d040 2024-09-10T08:45:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.409+0000 7ff44be00640 1 -- 172.21.15.89:0/2196646063 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff4640a02b0 con 0x56231678d040 2024-09-10T08:45:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.410+0000 7ff473e88740 10 client.26417 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.410+0000 7ff473e88740 1 -- 172.21.15.89:0/2196646063 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56231679bbb0 con 0x56231678d040 2024-09-10T08:45:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.410+0000 7ff44be00640 1 -- 172.21.15.89:0/2196646063 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff45c004030 con 0x56231678d040 2024-09-10T08:45:05.417 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.410+0000 7ff44be00640 1 -- 172.21.15.89:0/2196646063 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7ff4640bacd0 con 0x56231678d040 2024-09-10T08:45:05.417 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.410+0000 7ff473e88740 10 client.26417 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:05.417 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.410+0000 7ff473e88740 10 client.26417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:45:05.417 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.410+0000 7ff473e88740 20 client.26417 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:05.417 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.410+0000 7ff473e88740 1 --2- 172.21.15.89:0/2196646063 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56231679d280 0x56231679f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.417 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.410+0000 7ff473e88740 4 client.26417 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:45:05.417 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.410+0000 7ff473e88740 1 -- 172.21.15.89:0/2196646063 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623166f5830 con 0x56231679d280 2024-09-10T08:45:05.417 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.410+0000 7ff469e00640 1 --2- 172.21.15.89:0/2196646063 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56231679d280 0x56231679f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.417 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.411+0000 7ff469e00640 1 --2- 172.21.15.89:0/2196646063 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56231679d280 0x56231679f6f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fa0787a4209e501 server_cookie=1e798954806850d6 in_seq=0 out_seq=0 2024-09-10T08:45:05.417 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.411+0000 7ff44be00640 10 client.26417 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.461+0000 7ff44be00640 1 -- 172.21.15.89:0/2196646063 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623166f5830 con 0x56231679d280 2024-09-10T08:45:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.461+0000 7ff44be00640 10 client.26417 handle_command_reply: tid=0 2024-09-10T08:45:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.461+0000 7ff473e88740 1 client.26417 shutdown 2024-09-10T08:45:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.461+0000 7ff473e88740 1 -- 172.21.15.89:0/2196646063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56231678d040 msgr2=0x562316798d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.461+0000 7ff473e88740 1 --2- 172.21.15.89:0/2196646063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56231678d040 0x562316798d00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff4640cd1f0 tx=0x7ff46409b860 comp rx=0 tx=0).stop 2024-09-10T08:45:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.461+0000 7ff473e88740 1 -- 172.21.15.89:0/2196646063 shutdown_connections 2024-09-10T08:45:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.461+0000 7ff473e88740 1 --2- 172.21.15.89:0/2196646063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56231678d040 0x562316798d00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.461+0000 7ff473e88740 1 --2- 172.21.15.89:0/2196646063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562316787fb0 0x562316791cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.461+0000 7ff473e88740 1 --2- 172.21.15.89:0/2196646063 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56231679d280 0x56231679f6f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.461+0000 7ff473e88740 1 --2- 172.21.15.89:0/2196646063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316787670 0x562316791780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.461+0000 7ff473e88740 1 -- 172.21.15.89:0/2196646063 >> 172.21.15.89:0/2196646063 conn(0x562316782ce0 msgr2=0x5623167844f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.461+0000 7ff44be00640 0 client.26417 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.461+0000 7ff473e88740 1 -- 172.21.15.89:0/2196646063 shutdown_connections 2024-09-10T08:45:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.461+0000 7ff473e88740 1 -- 172.21.15.89:0/2196646063 wait complete. 2024-09-10T08:45:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.461+0000 7ff473e88740 20 client.26417 trim_cache size 0 max 16384 2024-09-10T08:45:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.477+0000 7ff473e88740 1 Processor -- start 2024-09-10T08:45:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.477+0000 7ff473e88740 1 -- start start 2024-09-10T08:45:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.477+0000 7ff473e88740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316782ce0 0x56231677da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.477+0000 7ff473e88740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562316787670 0x562316798cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.477+0000 7ff473e88740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562316787fb0 0x5623167840d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.477+0000 7ff473e88740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c1aaca0 con 0x562316782ce0 2024-09-10T08:45:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.477+0000 7ff473e88740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c1a8570 con 0x562316787670 2024-09-10T08:45:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.477+0000 7ff473e88740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c1a63c0 con 0x562316787fb0 2024-09-10T08:45:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.477+0000 7ff469e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316782ce0 0x56231677da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.477+0000 7ff469400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562316787670 0x562316798cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.477+0000 7ff469400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562316787670 0x562316798cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50464/0 (socket says 172.21.15.89:50464) 2024-09-10T08:45:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.477+0000 7ff469400640 1 -- 172.21.15.89:0/1391428369 learned_addr learned my addr 172.21.15.89:0/1391428369 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.477+0000 7ff469e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316782ce0 0x56231677da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60636/0 (socket says 172.21.15.89:60636) 2024-09-10T08:45:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.477+0000 7ff46a800640 1 --2- 172.21.15.89:0/1391428369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562316787fb0 0x5623167840d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.478+0000 7ff469e00640 1 -- 172.21.15.89:0/1391428369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562316787fb0 msgr2=0x5623167840d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.478+0000 7ff469e00640 1 --2- 172.21.15.89:0/1391428369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562316787fb0 0x5623167840d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.478+0000 7ff469e00640 1 -- 172.21.15.89:0/1391428369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562316787670 msgr2=0x562316798cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.478+0000 7ff469e00640 1 --2- 172.21.15.89:0/1391428369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562316787670 0x562316798cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.478+0000 7ff469e00640 1 -- 172.21.15.89:0/1391428369 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562316791e60 con 0x562316782ce0 2024-09-10T08:45:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.478+0000 7ff469400640 1 --2- 172.21.15.89:0/1391428369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562316787670 0x562316798cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.478+0000 7ff469e00640 1 --2- 172.21.15.89:0/1391428369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316782ce0 0x56231677da10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x562316790d50 tx=0x7ff45c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c5aa043924e5e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.478+0000 7ff456a00640 1 -- 172.21.15.89:0/1391428369 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c035040 con 0x562316782ce0 2024-09-10T08:45:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.479+0000 7ff456a00640 1 -- 172.21.15.89:0/1391428369 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff45c032d70 con 0x562316782ce0 2024-09-10T08:45:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.479+0000 7ff456a00640 1 -- 172.21.15.89:0/1391428369 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c03f040 con 0x562316782ce0 2024-09-10T08:45:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.479+0000 7ff473e88740 1 -- 172.21.15.89:0/1391428369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316782ce0 msgr2=0x56231677da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.479+0000 7ff473e88740 1 --2- 172.21.15.89:0/1391428369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316782ce0 0x56231677da10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x562316790d50 tx=0x7ff45c005fd0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.480+0000 7ff473e88740 1 -- 172.21.15.89:0/1391428369 shutdown_connections 2024-09-10T08:45:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.480+0000 7ff473e88740 1 --2- 172.21.15.89:0/1391428369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562316787fb0 0x5623167840d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.480+0000 7ff473e88740 1 --2- 172.21.15.89:0/1391428369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562316787670 0x562316798cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.480+0000 7ff473e88740 1 --2- 172.21.15.89:0/1391428369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316782ce0 0x56231677da10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.480+0000 7ff473e88740 1 -- 172.21.15.89:0/1391428369 >> 172.21.15.89:0/1391428369 conn(0x56231678cbd0 msgr2=0x5623167796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.480+0000 7ff473e88740 1 -- 172.21.15.89:0/1391428369 shutdown_connections 2024-09-10T08:45:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.480+0000 7ff473e88740 1 -- 172.21.15.89:0/1391428369 wait complete. 2024-09-10T08:45:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.480+0000 7ff473e88740 1 -- start start 2024-09-10T08:45:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.480+0000 7ff473e88740 1 Processor -- start 2024-09-10T08:45:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.481+0000 7ff473e88740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316782ce0 0x562316796040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.481+0000 7ff473e88740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562316787670 0x562316796580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.481+0000 7ff473e88740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562316787fb0 0x562316797590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.481+0000 7ff473e88740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c1a6a30 con 0x562316782ce0 2024-09-10T08:45:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.481+0000 7ff473e88740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c1a7ea0 con 0x562316787670 2024-09-10T08:45:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.481+0000 7ff473e88740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c1a7100 con 0x562316787fb0 2024-09-10T08:45:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.481+0000 7ff473e88740 10 client.0 _ll_register_callbacks cb 0x7ff46409bcd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.481+0000 7ff469e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316782ce0 0x562316796040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.481+0000 7ff469400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562316787670 0x562316796580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.481+0000 7ff46a800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562316787fb0 0x562316797590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.481+0000 7ff469400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562316787670 0x562316796580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50472/0 (socket says 172.21.15.89:50472) 2024-09-10T08:45:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.481+0000 7ff469400640 1 -- 172.21.15.89:0/626051567 learned_addr learned my addr 172.21.15.89:0/626051567 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.482+0000 7ff46a800640 1 -- 172.21.15.89:0/626051567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562316787670 msgr2=0x562316796580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.482+0000 7ff46a800640 1 --2- 172.21.15.89:0/626051567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562316787670 0x562316796580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.482+0000 7ff46a800640 1 -- 172.21.15.89:0/626051567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316782ce0 msgr2=0x562316796040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.482+0000 7ff46a800640 1 --2- 172.21.15.89:0/626051567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316782ce0 0x562316796040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.482+0000 7ff46a800640 1 -- 172.21.15.89:0/626051567 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562316797ba0 con 0x562316787fb0 2024-09-10T08:45:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.482+0000 7ff469400640 1 --2- 172.21.15.89:0/626051567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562316787670 0x562316796580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.482+0000 7ff469e00640 1 --2- 172.21.15.89:0/626051567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316782ce0 0x562316796040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.482+0000 7ff46a800640 1 --2- 172.21.15.89:0/626051567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562316787fb0 0x562316797590 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff464098880 tx=0x7ff46409b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14542309ee409565 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.482+0000 7ff44be00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:45:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.482+0000 7ff44be00640 1 -- 172.21.15.89:0/626051567 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464066060 con 0x562316787fb0 2024-09-10T08:45:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.482+0000 7ff44be00640 1 -- 172.21.15.89:0/626051567 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff4640bab40 con 0x562316787fb0 2024-09-10T08:45:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.482+0000 7ff473e88740 1 -- 172.21.15.89:0/626051567 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff45c032d70 con 0x562316787fb0 2024-09-10T08:45:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.482+0000 7ff44be00640 1 -- 172.21.15.89:0/626051567 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff4640981a0 con 0x562316787fb0 2024-09-10T08:45:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.483+0000 7ff473e88740 10 client.26350 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.483+0000 7ff473e88740 1 -- 172.21.15.89:0/626051567 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562316798f20 con 0x562316787fb0 2024-09-10T08:45:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.483+0000 7ff44be00640 1 -- 172.21.15.89:0/626051567 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff45c032d70 con 0x562316787fb0 2024-09-10T08:45:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.483+0000 7ff44be00640 1 -- 172.21.15.89:0/626051567 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7ff4640a1440 con 0x562316787fb0 2024-09-10T08:45:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.483+0000 7ff473e88740 10 client.26350 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.483+0000 7ff473e88740 10 client.26350 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:45:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.483+0000 7ff473e88740 20 client.26350 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.483+0000 7ff473e88740 1 --2- 172.21.15.89:0/626051567 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56231678d040 0x56231680a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.483+0000 7ff473e88740 4 client.26350 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:45:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.483+0000 7ff473e88740 1 -- 172.21.15.89:0/626051567 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5623166efb80 con 0x56231678d040 2024-09-10T08:45:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.483+0000 7ff469e00640 1 --2- 172.21.15.89:0/626051567 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56231678d040 0x56231680a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.484+0000 7ff469e00640 1 --2- 172.21.15.89:0/626051567 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56231678d040 0x56231680a570 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e864551c849c323 server_cookie=a01466f13695c60e in_seq=0 out_seq=0 2024-09-10T08:45:05.488 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:05.484+0000 7ff44be00640 10 client.26350 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:06.026 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:45:06.027 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:45:06.462 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.606+0000 7ff44be00640 1 -- 172.21.15.89:0/626051567 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5623166efb80 con 0x56231678d040 2024-09-10T08:45:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.606+0000 7ff44be00640 10 client.26350 handle_command_reply: tid=0 2024-09-10T08:45:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.606+0000 7ff473e88740 1 client.26350 shutdown 2024-09-10T08:45:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.606+0000 7ff473e88740 1 -- 172.21.15.89:0/626051567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562316787fb0 msgr2=0x562316797590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.606+0000 7ff473e88740 1 --2- 172.21.15.89:0/626051567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562316787fb0 0x562316797590 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff464098880 tx=0x7ff46409b990 comp rx=0 tx=0).stop 2024-09-10T08:45:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.606+0000 7ff473e88740 1 -- 172.21.15.89:0/626051567 shutdown_connections 2024-09-10T08:45:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.606+0000 7ff473e88740 1 --2- 172.21.15.89:0/626051567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562316787fb0 0x562316797590 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.606+0000 7ff473e88740 1 --2- 172.21.15.89:0/626051567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562316787670 0x562316796580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.606+0000 7ff473e88740 1 --2- 172.21.15.89:0/626051567 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56231678d040 0x56231680a570 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.606+0000 7ff473e88740 1 --2- 172.21.15.89:0/626051567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562316782ce0 0x562316796040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.606+0000 7ff44be00640 0 client.26350 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.606+0000 7ff473e88740 1 -- 172.21.15.89:0/626051567 >> 172.21.15.89:0/626051567 conn(0x56231678cbd0 msgr2=0x56231678c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.606+0000 7ff473e88740 1 -- 172.21.15.89:0/626051567 shutdown_connections 2024-09-10T08:45:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.606+0000 7ff473e88740 1 -- 172.21.15.89:0/626051567 wait complete. 2024-09-10T08:45:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.607+0000 7ff473e88740 20 client.26350 trim_cache size 0 max 16384 2024-09-10T08:45:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:45:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:45:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:45:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:45:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.607+0000 7ff46b200640 1 -- 172.21.15.89:0/1805389844 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff4500a2b80 msgr2=0x7ff4500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.607+0000 7ff46b200640 1 --2- 172.21.15.89:0/1805389844 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff4500a2b80 0x7ff4500a5040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff458009710 tx=0x7ff4580096a0 comp rx=0 tx=0).stop 2024-09-10T08:45:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.607+0000 7ff46b200640 1 -- 172.21.15.89:0/1805389844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff46c103050 msgr2=0x7ff46c19f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.607+0000 7ff46b200640 1 --2- 172.21.15.89:0/1805389844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff46c103050 0x7ff46c19f3a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff46400eb10 tx=0x7ff46400c2d0 comp rx=0 tx=0).stop 2024-09-10T08:45:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.608+0000 7ff46b200640 1 -- 172.21.15.89:0/1805389844 shutdown_connections 2024-09-10T08:45:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.608+0000 7ff46b200640 1 --2- 172.21.15.89:0/1805389844 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff4500a2b80 0x7ff4500a5040 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.608+0000 7ff46b200640 1 --2- 172.21.15.89:0/1805389844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff46c103050 0x7ff46c19f3a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.608+0000 7ff46b200640 1 --2- 172.21.15.89:0/1805389844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff46c101f40 0x7ff46c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.608+0000 7ff46b200640 1 --2- 172.21.15.89:0/1805389844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff46c101580 0x7ff46c197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.608+0000 7ff46b200640 1 -- 172.21.15.89:0/1805389844 >> 172.21.15.89:0/1805389844 conn(0x7ff46c0fd0d0 msgr2=0x7ff46c106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.608+0000 7ff46b200640 1 -- 172.21.15.89:0/1805389844 shutdown_connections 2024-09-10T08:45:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:06.608+0000 7ff46b200640 1 -- 172.21.15.89:0/1805389844 wait complete. 2024-09-10T08:45:07.024 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.021+0000 7f62cf000640 1 -- 172.21.15.89:0/538484670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62c8108f60 msgr2=0x7f62c8110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:07.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.021+0000 7f62cf000640 1 --2- 172.21.15.89:0/538484670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62c8108f60 0x7f62c8110ae0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009ca0 tx=0x7f62c002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:45:07.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.021+0000 7f62cf000640 1 -- 172.21.15.89:0/538484670 shutdown_connections 2024-09-10T08:45:07.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.021+0000 7f62cf000640 1 --2- 172.21.15.89:0/538484670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62c8108f60 0x7f62c8110ae0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:07.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.021+0000 7f62cf000640 1 --2- 172.21.15.89:0/538484670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62c81047e0 0x7f62c8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:07.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.021+0000 7f62cf000640 1 --2- 172.21.15.89:0/538484670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62c8103e30 0x7f62c8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:07.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.021+0000 7f62cf000640 1 -- 172.21.15.89:0/538484670 >> 172.21.15.89:0/538484670 conn(0x7f62c80ff8d0 msgr2=0x7f62c8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:07.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.021+0000 7f62cf000640 1 -- 172.21.15.89:0/538484670 shutdown_connections 2024-09-10T08:45:07.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.021+0000 7f62cf000640 1 -- 172.21.15.89:0/538484670 wait complete. 2024-09-10T08:45:07.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.022+0000 7f62cf000640 1 Processor -- start 2024-09-10T08:45:07.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.022+0000 7f62cf000640 1 -- start start 2024-09-10T08:45:07.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.023+0000 7f62cf000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62c8103e30 0x7f62c819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:07.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.023+0000 7f62cf000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62c81047e0 0x7f62c819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:07.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.023+0000 7f62cf000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62c8108f60 0x7f62c819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:07.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.023+0000 7f62cf000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f62c8113fc0 con 0x7f62c8108f60 2024-09-10T08:45:07.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.023+0000 7f62cf000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f62c8113e40 con 0x7f62c8103e30 2024-09-10T08:45:07.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.023+0000 7f62cf000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f62c8114140 con 0x7f62c81047e0 2024-09-10T08:45:07.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.023+0000 7f62cdc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62c8103e30 0x7f62c819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:07.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.023+0000 7f62cdc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62c8103e30 0x7f62c819a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50488/0 (socket says 172.21.15.89:50488) 2024-09-10T08:45:07.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.023+0000 7f62cdc00640 1 -- 172.21.15.89:0/83331416 learned_addr learned my addr 172.21.15.89:0/83331416 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:07.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.023+0000 7f62ce600640 1 --2- 172.21.15.89:0/83331416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62c8108f60 0x7f62c819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:07.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.023+0000 7f62cd200640 1 --2- 172.21.15.89:0/83331416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62c81047e0 0x7f62c819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:07.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.023+0000 7f62cdc00640 1 -- 172.21.15.89:0/83331416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62c81047e0 msgr2=0x7f62c819a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:07.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.023+0000 7f62cdc00640 1 --2- 172.21.15.89:0/83331416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62c81047e0 0x7f62c819a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:07.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.023+0000 7f62cdc00640 1 -- 172.21.15.89:0/83331416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62c8108f60 msgr2=0x7f62c819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:07.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.023+0000 7f62cdc00640 1 --2- 172.21.15.89:0/83331416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62c8108f60 0x7f62c819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:07.026 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.023+0000 7f62cdc00640 1 -- 172.21.15.89:0/83331416 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c819f3d0 con 0x7f62c8103e30 2024-09-10T08:45:07.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.024+0000 7f62cdc00640 1 --2- 172.21.15.89:0/83331416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62c8103e30 0x7f62c819a420 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00ea60 tx=0x7f62bc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:07.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.024+0000 7f62c6000640 1 -- 172.21.15.89:0/83331416 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc00cde0 con 0x7f62c8103e30 2024-09-10T08:45:07.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.024+0000 7f62cf000640 1 -- 172.21.15.89:0/83331416 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c819f6c0 con 0x7f62c8103e30 2024-09-10T08:45:07.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.024+0000 7f62c6000640 1 -- 172.21.15.89:0/83331416 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62bc0035f0 con 0x7f62c8103e30 2024-09-10T08:45:07.027 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.024+0000 7f62c6000640 1 -- 172.21.15.89:0/83331416 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc010670 con 0x7f62c8103e30 2024-09-10T08:45:07.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.024+0000 7f62cf000640 1 -- 172.21.15.89:0/83331416 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c81a6fa0 con 0x7f62c8103e30 2024-09-10T08:45:07.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.025+0000 7f62cf000640 1 -- 172.21.15.89:0/83331416 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62c8104210 con 0x7f62c8103e30 2024-09-10T08:45:07.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.027+0000 7f62c6000640 1 -- 172.21.15.89:0/83331416 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f62bc0037a0 con 0x7f62c8103e30 2024-09-10T08:45:07.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.029+0000 7f62c6000640 1 --2- 172.21.15.89:0/83331416 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f62b00a2b80 0x7f62b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:07.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.029+0000 7f62c6000640 1 -- 172.21.15.89:0/83331416 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f62bc0cf750 con 0x7f62c8103e30 2024-09-10T08:45:07.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.029+0000 7f62cd200640 1 --2- 172.21.15.89:0/83331416 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f62b00a2b80 0x7f62b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:07.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.030+0000 7f62cd200640 1 --2- 172.21.15.89:0/83331416 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f62b00a2b80 0x7f62b00a5040 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f62c810a930 tx=0x7f62b4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:07.031 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.030+0000 7f62c6000640 1 -- 172.21.15.89:0/83331416 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62bc09c280 con 0x7f62c8103e30 2024-09-10T08:45:07.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:06 smithi149 ceph-mon[36167]: pgmap v548: 129 pgs: 129 active+clean; 317 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.4 MiB/s wr, 1.25k op/s 2024-09-10T08:45:07.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.268+0000 7f62cf000640 1 -- 172.21.15.89:0/83331416 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62c819b780 con 0x7f62c8103e30 2024-09-10T08:45:07.269 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:06 smithi089 ceph-mon[30757]: pgmap v548: 129 pgs: 129 active+clean; 317 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.4 MiB/s wr, 1.25k op/s 2024-09-10T08:45:07.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.269+0000 7f62c6000640 1 -- 172.21.15.89:0/83331416 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11919 (secure 0 0 0) 0x7f62bc09bb80 con 0x7f62c8103e30 2024-09-10T08:45:07.270 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:45:07.270 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"btime":"2024-09-10T08:44:40:778845+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:44:40.174532+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24496,24518,24554]},"id":1}]} 2024-09-10T08:45:07.270 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 29 2024-09-10T08:45:07.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.271+0000 7f62cf000640 1 -- 172.21.15.89:0/83331416 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f62b00a2b80 msgr2=0x7f62b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:07.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.271+0000 7f62cf000640 1 --2- 172.21.15.89:0/83331416 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f62b00a2b80 0x7f62b00a5040 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f62c810a930 tx=0x7f62b4005f90 comp rx=0 tx=0).stop 2024-09-10T08:45:07.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.271+0000 7f62cf000640 1 -- 172.21.15.89:0/83331416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62c8103e30 msgr2=0x7f62c819a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:07.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.271+0000 7f62cf000640 1 --2- 172.21.15.89:0/83331416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62c8103e30 0x7f62c819a420 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00ea60 tx=0x7f62bc00ef20 comp rx=0 tx=0).stop 2024-09-10T08:45:07.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.271+0000 7f62cf000640 1 -- 172.21.15.89:0/83331416 shutdown_connections 2024-09-10T08:45:07.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.271+0000 7f62cf000640 1 --2- 172.21.15.89:0/83331416 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f62b00a2b80 0x7f62b00a5040 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:07.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.271+0000 7f62cf000640 1 --2- 172.21.15.89:0/83331416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f62c8108f60 0x7f62c819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:07.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.271+0000 7f62cf000640 1 --2- 172.21.15.89:0/83331416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62c81047e0 0x7f62c819a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:07.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.271+0000 7f62cf000640 1 --2- 172.21.15.89:0/83331416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62c8103e30 0x7f62c819a420 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:07.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.271+0000 7f62cf000640 1 -- 172.21.15.89:0/83331416 >> 172.21.15.89:0/83331416 conn(0x7f62c80ff8d0 msgr2=0x7f62c8100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:07.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.272+0000 7f62cf000640 1 -- 172.21.15.89:0/83331416 shutdown_connections 2024-09-10T08:45:07.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:07.272+0000 7f62cf000640 1 -- 172.21.15.89:0/83331416 wait complete. 2024-09-10T08:45:07.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:06 smithi111 ceph-mon[35220]: pgmap v548: 129 pgs: 129 active+clean; 317 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.4 MiB/s wr, 1.25k op/s 2024-09-10T08:45:07.964 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:07 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/83331416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:08.042 INFO:tasks.workunit.client.0.smithi089.stdout: 89 35597 169 24277 111 21021 227 2024-09-10T08:45:08.157 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:45:08.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:07 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/83331416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:08.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:07 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/83331416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:08.533 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:09.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:08 smithi089 ceph-mon[30757]: pgmap v549: 129 pgs: 129 active+clean; 324 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.8 MiB/s wr, 1.19k op/s 2024-09-10T08:45:09.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:08 smithi149 ceph-mon[36167]: pgmap v549: 129 pgs: 129 active+clean; 324 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.8 MiB/s wr, 1.19k op/s 2024-09-10T08:45:09.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:08 smithi111 ceph-mon[35220]: pgmap v549: 129 pgs: 129 active+clean; 324 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.8 MiB/s wr, 1.19k op/s 2024-09-10T08:45:09.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.831+0000 7fa516600640 1 -- 172.21.15.89:0/719857036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5101047e0 msgr2=0x7fa510108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:09.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.831+0000 7fa516600640 1 --2- 172.21.15.89:0/719857036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5101047e0 0x7fa510108830 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc009900 tx=0x7fa4fc02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.831+0000 7fa516600640 1 -- 172.21.15.89:0/719857036 shutdown_connections 2024-09-10T08:45:09.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.831+0000 7fa516600640 1 --2- 172.21.15.89:0/719857036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa510108f60 0x7fa510110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.831+0000 7fa516600640 1 --2- 172.21.15.89:0/719857036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5101047e0 0x7fa510108830 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.831+0000 7fa516600640 1 --2- 172.21.15.89:0/719857036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa510103e30 0x7fa510104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.831+0000 7fa516600640 1 -- 172.21.15.89:0/719857036 >> 172.21.15.89:0/719857036 conn(0x7fa5100ff910 msgr2=0x7fa510101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:09.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.831+0000 7fa516600640 1 -- 172.21.15.89:0/719857036 shutdown_connections 2024-09-10T08:45:09.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.831+0000 7fa516600640 1 -- 172.21.15.89:0/719857036 wait complete. 2024-09-10T08:45:09.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.833+0000 7fa516600640 1 Processor -- start 2024-09-10T08:45:09.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.833+0000 7fa516600640 1 -- start start 2024-09-10T08:45:09.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.833+0000 7fa516600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa510103e30 0x7fa51019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.833+0000 7fa516600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5101047e0 0x7fa51019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.833+0000 7fa516600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa510108f60 0x7fa51019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.833+0000 7fa516600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa510113f80 con 0x7fa510108f60 2024-09-10T08:45:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.833+0000 7fa516600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa510113e00 con 0x7fa5101047e0 2024-09-10T08:45:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.833+0000 7fa516600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa510114100 con 0x7fa510103e30 2024-09-10T08:45:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.834+0000 7fa50fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5101047e0 0x7fa51019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.834+0000 7fa50fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5101047e0 0x7fa51019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50496/0 (socket says 172.21.15.89:50496) 2024-09-10T08:45:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.834+0000 7fa50fe00640 1 -- 172.21.15.89:0/721335402 learned_addr learned my addr 172.21.15.89:0/721335402 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.834+0000 7fa515c00640 1 --2- 172.21.15.89:0/721335402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa510108f60 0x7fa51019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.834+0000 7fa515200640 1 --2- 172.21.15.89:0/721335402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa510103e30 0x7fa51019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.834+0000 7fa50fe00640 1 -- 172.21.15.89:0/721335402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa510103e30 msgr2=0x7fa51019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.834+0000 7fa50fe00640 1 --2- 172.21.15.89:0/721335402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa510103e30 0x7fa51019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.834+0000 7fa50fe00640 1 -- 172.21.15.89:0/721335402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa510108f60 msgr2=0x7fa51019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.834+0000 7fa50fe00640 1 --2- 172.21.15.89:0/721335402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa510108f60 0x7fa51019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.834+0000 7fa50fe00640 1 -- 172.21.15.89:0/721335402 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51019f450 con 0x7fa5101047e0 2024-09-10T08:45:09.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.834+0000 7fa50fe00640 1 --2- 172.21.15.89:0/721335402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5101047e0 0x7fa51019a9e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc006010 tx=0x7fa4fc0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.835+0000 7fa50d600640 1 -- 172.21.15.89:0/721335402 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc03f030 con 0x7fa5101047e0 2024-09-10T08:45:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.835+0000 7fa50d600640 1 -- 172.21.15.89:0/721335402 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4fc003ce0 con 0x7fa5101047e0 2024-09-10T08:45:09.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.835+0000 7fa516600640 1 -- 172.21.15.89:0/721335402 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa51019f6e0 con 0x7fa5101047e0 2024-09-10T08:45:09.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.835+0000 7fa50d600640 1 -- 172.21.15.89:0/721335402 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc035070 con 0x7fa5101047e0 2024-09-10T08:45:09.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.835+0000 7fa516600640 1 -- 172.21.15.89:0/721335402 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5101a7020 con 0x7fa5101047e0 2024-09-10T08:45:09.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.838+0000 7fa51a64d740 1 Processor -- start 2024-09-10T08:45:09.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.838+0000 7fa50d600640 1 -- 172.21.15.89:0/721335402 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa4fc042030 con 0x7fa5101047e0 2024-09-10T08:45:09.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.838+0000 7fa51a64d740 1 -- start start 2024-09-10T08:45:09.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.838+0000 7fa51a64d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17db1e40 0x560e17db2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.838+0000 7fa51a64d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17db2760 0x560e17dbcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.838+0000 7fa51a64d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17dbd020 0x560e17dbf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.838+0000 7fa51a64d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560e17d183e0 con 0x560e17db1e40 2024-09-10T08:45:09.840 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.838+0000 7fa51a64d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560e17cbfd80 con 0x560e17dbd020 2024-09-10T08:45:09.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.838+0000 7fa51a64d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560e17d143a0 con 0x560e17db2760 2024-09-10T08:45:09.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.839+0000 7fa515c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17db1e40 0x560e17db2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.839+0000 7fa515c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17db1e40 0x560e17db2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60692/0 (socket says 172.21.15.89:60692) 2024-09-10T08:45:09.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.839+0000 7fa515c00640 1 -- 172.21.15.89:0/1577166930 learned_addr learned my addr 172.21.15.89:0/1577166930 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:09.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.839+0000 7fa50fe00640 1 --2- 172.21.15.89:0/1577166930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17dbd020 0x560e17dbf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.839+0000 7fa515200640 1 --2- 172.21.15.89:0/1577166930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17db2760 0x560e17dbcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.839+0000 7fa515c00640 1 -- 172.21.15.89:0/1577166930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17db2760 msgr2=0x560e17dbcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:09.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.839+0000 7fa515c00640 1 --2- 172.21.15.89:0/1577166930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17db2760 0x560e17dbcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.839+0000 7fa515c00640 1 -- 172.21.15.89:0/1577166930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17dbd020 msgr2=0x560e17dbf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:09.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.839+0000 7fa515c00640 1 --2- 172.21.15.89:0/1577166930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17dbd020 0x560e17dbf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.839+0000 7fa515c00640 1 -- 172.21.15.89:0/1577166930 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e17cdeae0 con 0x560e17db1e40 2024-09-10T08:45:09.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.840+0000 7fa515c00640 1 --2- 172.21.15.89:0/1577166930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17db1e40 0x560e17db2220 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa508009f90 tx=0x7fa508010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca378150db739f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:09.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.840+0000 7fa50d600640 1 --2- 172.21.15.89:0/721335402 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa4f80a2b80 0x7fa4f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.840+0000 7fa50d600640 1 -- 172.21.15.89:0/721335402 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa4fc0ef9a0 con 0x7fa5101047e0 2024-09-10T08:45:09.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.840+0000 7fa502000640 1 -- 172.21.15.89:0/1577166930 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508011700 con 0x560e17db1e40 2024-09-10T08:45:09.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.840+0000 7fa502000640 1 -- 172.21.15.89:0/1577166930 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa508011d80 con 0x560e17db1e40 2024-09-10T08:45:09.843 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.840+0000 7fa515200640 1 --2- 172.21.15.89:0/721335402 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa4f80a2b80 0x7fa4f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.840+0000 7fa502000640 1 -- 172.21.15.89:0/1577166930 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508019e40 con 0x560e17db1e40 2024-09-10T08:45:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.841+0000 7fa51a64d740 1 -- 172.21.15.89:0/1577166930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17db1e40 msgr2=0x560e17db2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.841+0000 7fa51a64d740 1 --2- 172.21.15.89:0/1577166930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17db1e40 0x560e17db2220 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa508009f90 tx=0x7fa508010860 comp rx=0 tx=0).stop 2024-09-10T08:45:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.841+0000 7fa515200640 1 --2- 172.21.15.89:0/721335402 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa4f80a2b80 0x7fa4f80a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x560e17daf230 tx=0x7fa504005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.841+0000 7fa51a64d740 1 -- 172.21.15.89:0/1577166930 shutdown_connections 2024-09-10T08:45:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.841+0000 7fa51a64d740 1 --2- 172.21.15.89:0/1577166930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17dbd020 0x560e17dbf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.841+0000 7fa51a64d740 1 --2- 172.21.15.89:0/1577166930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17db2760 0x560e17dbcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.841+0000 7fa51a64d740 1 --2- 172.21.15.89:0/1577166930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17db1e40 0x560e17db2220 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.841+0000 7fa51a64d740 1 -- 172.21.15.89:0/1577166930 >> 172.21.15.89:0/1577166930 conn(0x560e17dae3d0 msgr2=0x560e17dae7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.841+0000 7fa51a64d740 1 -- 172.21.15.89:0/1577166930 shutdown_connections 2024-09-10T08:45:09.844 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.841+0000 7fa51a64d740 1 -- 172.21.15.89:0/1577166930 wait complete. 2024-09-10T08:45:09.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.842+0000 7fa51a64d740 1 -- start start 2024-09-10T08:45:09.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.842+0000 7fa51a64d740 1 Processor -- start 2024-09-10T08:45:09.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.842+0000 7fa51a64d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17db1e40 0x560e17db9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.842+0000 7fa51a64d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17db2760 0x560e17dba460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.842+0000 7fa51a64d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17dbd020 0x560e17dba9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.842+0000 7fa51a64d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560e17dbfbf0 con 0x560e17dbd020 2024-09-10T08:45:09.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.842+0000 7fa51a64d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560e17cd30f0 con 0x560e17db1e40 2024-09-10T08:45:09.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.842+0000 7fa51a64d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560e17dbfd70 con 0x560e17db2760 2024-09-10T08:45:09.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.843+0000 7fa51a64d740 10 client.0 _ll_register_callbacks cb 0x560e17cbabd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:09.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.843+0000 7fa515200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17db1e40 0x560e17db9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.843+0000 7fa515200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17db1e40 0x560e17db9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50516/0 (socket says 172.21.15.89:50516) 2024-09-10T08:45:09.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.843+0000 7fa515c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17dbd020 0x560e17dba9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.843+0000 7fa515200640 1 -- 172.21.15.89:0/80162678 learned_addr learned my addr 172.21.15.89:0/80162678 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:09.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.843+0000 7fa50fe00640 1 --2- 172.21.15.89:0/80162678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17db2760 0x560e17dba460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.843+0000 7fa515200640 1 -- 172.21.15.89:0/80162678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17db2760 msgr2=0x560e17dba460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:09.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.843+0000 7fa515200640 1 --2- 172.21.15.89:0/80162678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17db2760 0x560e17dba460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.843+0000 7fa515200640 1 -- 172.21.15.89:0/80162678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17dbd020 msgr2=0x560e17dba9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:09.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.843+0000 7fa515200640 1 --2- 172.21.15.89:0/80162678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17dbd020 0x560e17dba9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.843+0000 7fa515200640 1 -- 172.21.15.89:0/80162678 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e17a893c0 con 0x560e17db1e40 2024-09-10T08:45:09.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.843+0000 7fa515200640 1 --2- 172.21.15.89:0/80162678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17db1e40 0x560e17db9f20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa5040045f0 tx=0x7fa504004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d196d46cc4747cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:09.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.844+0000 7fa4f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:45:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.844+0000 7fa4f6a00640 1 -- 172.21.15.89:0/80162678 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50401f640 con 0x560e17db1e40 2024-09-10T08:45:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.844+0000 7fa4f6a00640 1 -- 172.21.15.89:0/80162678 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa50401f7e0 con 0x560e17db1e40 2024-09-10T08:45:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.844+0000 7fa51a64d740 1 -- 172.21.15.89:0/80162678 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa508011d80 con 0x560e17db1e40 2024-09-10T08:45:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.844+0000 7fa4f6a00640 1 -- 172.21.15.89:0/80162678 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa50401f9e0 con 0x560e17db1e40 2024-09-10T08:45:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.844+0000 7fa51a64d740 10 client.26441 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.844+0000 7fa51a64d740 1 -- 172.21.15.89:0/80162678 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560e17dcbca0 con 0x560e17db1e40 2024-09-10T08:45:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.844+0000 7fa4f6a00640 1 -- 172.21.15.89:0/80162678 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa508011d80 con 0x560e17db1e40 2024-09-10T08:45:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.845+0000 7fa4f6a00640 1 -- 172.21.15.89:0/80162678 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fa504027020 con 0x560e17db1e40 2024-09-10T08:45:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.845+0000 7fa51a64d740 10 client.26441 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.845+0000 7fa51a64d740 10 client.26441 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:45:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.845+0000 7fa51a64d740 20 client.26441 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.845+0000 7fa51a64d740 1 --2- 172.21.15.89:0/80162678 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x560e17dcd160 0x560e17dcf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.849 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.845+0000 7fa51a64d740 4 client.26441 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:45:09.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.845+0000 7fa51a64d740 1 -- 172.21.15.89:0/80162678 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e17d25950 con 0x560e17dcd160 2024-09-10T08:45:09.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.845+0000 7fa50fe00640 1 --2- 172.21.15.89:0/80162678 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x560e17dcd160 0x560e17dcf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.846+0000 7fa50fe00640 1 --2- 172.21.15.89:0/80162678 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x560e17dcd160 0x560e17dcf570 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=58b7e4ba3f34ef03 server_cookie=9a7c2839b07bb8c6 in_seq=0 out_seq=0 2024-09-10T08:45:09.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.846+0000 7fa4f6a00640 10 client.26441 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:45:09.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.847+0000 7fa4f6a00640 1 -- 172.21.15.89:0/80162678 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e17d25950 con 0x560e17dcd160 2024-09-10T08:45:09.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.847+0000 7fa4f6a00640 10 client.26441 handle_command_reply: tid=0 2024-09-10T08:45:09.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.847+0000 7fa51a64d740 1 client.26441 shutdown 2024-09-10T08:45:09.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.847+0000 7fa51a64d740 1 -- 172.21.15.89:0/80162678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17db1e40 msgr2=0x560e17db9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:09.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.847+0000 7fa51a64d740 1 --2- 172.21.15.89:0/80162678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17db1e40 0x560e17db9f20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa5040045f0 tx=0x7fa504004730 comp rx=0 tx=0).stop 2024-09-10T08:45:09.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.847+0000 7fa51a64d740 1 -- 172.21.15.89:0/80162678 shutdown_connections 2024-09-10T08:45:09.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.847+0000 7fa51a64d740 1 --2- 172.21.15.89:0/80162678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17dbd020 0x560e17dba9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.847+0000 7fa51a64d740 1 --2- 172.21.15.89:0/80162678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17db2760 0x560e17dba460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.847+0000 7fa51a64d740 1 --2- 172.21.15.89:0/80162678 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x560e17dcd160 0x560e17dcf570 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.847+0000 7fa51a64d740 1 --2- 172.21.15.89:0/80162678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17db1e40 0x560e17db9f20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.847+0000 7fa51a64d740 1 -- 172.21.15.89:0/80162678 >> 172.21.15.89:0/80162678 conn(0x560e17dae3d0 msgr2=0x560e17db40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:09.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.847+0000 7fa4f6a00640 0 client.26441 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:45:09.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.848+0000 7fa51a64d740 1 -- 172.21.15.89:0/80162678 shutdown_connections 2024-09-10T08:45:09.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.848+0000 7fa51a64d740 1 -- 172.21.15.89:0/80162678 wait complete. 2024-09-10T08:45:09.852 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.848+0000 7fa51a64d740 20 client.26441 trim_cache size 0 max 16384 2024-09-10T08:45:09.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.862+0000 7fa51a64d740 1 Processor -- start 2024-09-10T08:45:09.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.862+0000 7fa51a64d740 1 -- start start 2024-09-10T08:45:09.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.862+0000 7fa51a64d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17db1e40 0x560e17db48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.862+0000 7fa51a64d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17db2760 0x560e17dbbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.862+0000 7fa51a64d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17dbd020 0x560e17dd4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.862+0000 7fa51a64d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa5101b3700 con 0x560e17db1e40 2024-09-10T08:45:09.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.862+0000 7fa51a64d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa5101b0fd0 con 0x560e17db2760 2024-09-10T08:45:09.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.862+0000 7fa51a64d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa5101aee20 con 0x560e17dbd020 2024-09-10T08:45:09.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.862+0000 7fa515200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17db1e40 0x560e17db48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.862+0000 7fa50fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17db2760 0x560e17dbbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.862+0000 7fa515200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17db1e40 0x560e17db48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60710/0 (socket says 172.21.15.89:60710) 2024-09-10T08:45:09.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.862+0000 7fa515200640 1 -- 172.21.15.89:0/3383676577 learned_addr learned my addr 172.21.15.89:0/3383676577 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:09.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.863+0000 7fa515c00640 1 --2- 172.21.15.89:0/3383676577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17dbd020 0x560e17dd4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.863+0000 7fa515200640 1 -- 172.21.15.89:0/3383676577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17dbd020 msgr2=0x560e17dd4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:09.866 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.863+0000 7fa515200640 1 --2- 172.21.15.89:0/3383676577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17dbd020 0x560e17dd4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.863+0000 7fa515200640 1 -- 172.21.15.89:0/3383676577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17db2760 msgr2=0x560e17dbbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:09.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.863+0000 7fa515200640 1 --2- 172.21.15.89:0/3383676577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17db2760 0x560e17dbbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.863+0000 7fa515200640 1 -- 172.21.15.89:0/3383676577 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e17dbc890 con 0x560e17db1e40 2024-09-10T08:45:09.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.863+0000 7fa515c00640 1 --2- 172.21.15.89:0/3383676577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17dbd020 0x560e17dd4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:09.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.863+0000 7fa50fe00640 1 --2- 172.21.15.89:0/3383676577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17db2760 0x560e17dbbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:09.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.863+0000 7fa515200640 1 --2- 172.21.15.89:0/3383676577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17db1e40 0x560e17db48a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa504012d50 tx=0x7fa5040092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7792f175c20af102 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:09.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.863+0000 7fa501600640 1 -- 172.21.15.89:0/3383676577 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50401f840 con 0x560e17db1e40 2024-09-10T08:45:09.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.863+0000 7fa501600640 1 -- 172.21.15.89:0/3383676577 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa504005500 con 0x560e17db1e40 2024-09-10T08:45:09.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.863+0000 7fa501600640 1 -- 172.21.15.89:0/3383676577 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504004d10 con 0x560e17db1e40 2024-09-10T08:45:09.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.864+0000 7fa51a64d740 1 -- 172.21.15.89:0/3383676577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17db1e40 msgr2=0x560e17db48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:09.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.864+0000 7fa51a64d740 1 --2- 172.21.15.89:0/3383676577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17db1e40 0x560e17db48a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa504012d50 tx=0x7fa5040092a0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.864+0000 7fa51a64d740 1 -- 172.21.15.89:0/3383676577 shutdown_connections 2024-09-10T08:45:09.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.864+0000 7fa51a64d740 1 --2- 172.21.15.89:0/3383676577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17dbd020 0x560e17dd4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.864+0000 7fa51a64d740 1 --2- 172.21.15.89:0/3383676577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17db2760 0x560e17dbbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.864+0000 7fa51a64d740 1 --2- 172.21.15.89:0/3383676577 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17db1e40 0x560e17db48a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.864+0000 7fa51a64d740 1 -- 172.21.15.89:0/3383676577 >> 172.21.15.89:0/3383676577 conn(0x560e17dae3d0 msgr2=0x560e17dbb780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:09.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.864+0000 7fa51a64d740 1 -- 172.21.15.89:0/3383676577 shutdown_connections 2024-09-10T08:45:09.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.864+0000 7fa51a64d740 1 -- 172.21.15.89:0/3383676577 wait complete. 2024-09-10T08:45:09.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.865+0000 7fa51a64d740 1 -- start start 2024-09-10T08:45:09.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.865+0000 7fa51a64d740 1 Processor -- start 2024-09-10T08:45:09.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.865+0000 7fa51a64d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17db1e40 0x560e17dc5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.865+0000 7fa51a64d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17db2760 0x560e17dc09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.866+0000 7fa51a64d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17dbd020 0x560e17dc0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.866+0000 7fa51a64d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa5101af490 con 0x560e17db2760 2024-09-10T08:45:09.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.866+0000 7fa51a64d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa5101b0900 con 0x560e17db1e40 2024-09-10T08:45:09.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.866+0000 7fa51a64d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa5101afb60 con 0x560e17dbd020 2024-09-10T08:45:09.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.866+0000 7fa51a64d740 10 client.0 _ll_register_callbacks cb 0x7fa504009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:09.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.866+0000 7fa515200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17db1e40 0x560e17dc5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.866+0000 7fa515200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17db1e40 0x560e17dc5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50536/0 (socket says 172.21.15.89:50536) 2024-09-10T08:45:09.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.866+0000 7fa50fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17db2760 0x560e17dc09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.866+0000 7fa515200640 1 -- 172.21.15.89:0/1886854091 learned_addr learned my addr 172.21.15.89:0/1886854091 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:09.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.866+0000 7fa515c00640 1 --2- 172.21.15.89:0/1886854091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17dbd020 0x560e17dc0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.866+0000 7fa515200640 1 -- 172.21.15.89:0/1886854091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17dbd020 msgr2=0x560e17dc0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:09.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.866+0000 7fa515200640 1 --2- 172.21.15.89:0/1886854091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17dbd020 0x560e17dc0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.866+0000 7fa515200640 1 -- 172.21.15.89:0/1886854091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17db2760 msgr2=0x560e17dc09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:09.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.866+0000 7fa515200640 1 --2- 172.21.15.89:0/1886854091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17db2760 0x560e17dc09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.866+0000 7fa515200640 1 -- 172.21.15.89:0/1886854091 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e17dc1700 con 0x560e17db1e40 2024-09-10T08:45:09.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.867+0000 7fa515200640 1 --2- 172.21.15.89:0/1886854091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17db1e40 0x560e17dc5680 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa504038040 tx=0x7fa50401fdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=276e58065cf12f7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:09.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.867+0000 7fa4f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:45:09.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.867+0000 7fa4f6a00640 1 -- 172.21.15.89:0/1886854091 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504043060 con 0x560e17db1e40 2024-09-10T08:45:09.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.867+0000 7fa4f6a00640 1 -- 172.21.15.89:0/1886854091 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa504004a50 con 0x560e17db1e40 2024-09-10T08:45:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.867+0000 7fa51a64d740 1 -- 172.21.15.89:0/1886854091 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa504005500 con 0x560e17db1e40 2024-09-10T08:45:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.867+0000 7fa4f6a00640 1 -- 172.21.15.89:0/1886854091 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa50404a050 con 0x560e17db1e40 2024-09-10T08:45:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.867+0000 7fa51a64d740 10 client.26447 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.867+0000 7fa51a64d740 1 -- 172.21.15.89:0/1886854091 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560e17e37d80 con 0x560e17db1e40 2024-09-10T08:45:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.867+0000 7fa4f6a00640 1 -- 172.21.15.89:0/1886854091 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa504005500 con 0x560e17db1e40 2024-09-10T08:45:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.868+0000 7fa4f6a00640 1 -- 172.21.15.89:0/1886854091 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fa504027070 con 0x560e17db1e40 2024-09-10T08:45:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.868+0000 7fa51a64d740 10 client.26447 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.868+0000 7fa51a64d740 10 client.26447 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:45:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.868+0000 7fa51a64d740 20 client.26447 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.868+0000 7fa51a64d740 1 --2- 172.21.15.89:0/1886854091 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x560e17e383e0 0x560e17e3a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.868+0000 7fa51a64d740 4 client.26447 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:45:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.868+0000 7fa51a64d740 1 -- 172.21.15.89:0/1886854091 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e17d1fe60 con 0x560e17e383e0 2024-09-10T08:45:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.868+0000 7fa50fe00640 1 --2- 172.21.15.89:0/1886854091 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x560e17e383e0 0x560e17e3a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.869+0000 7fa50fe00640 1 --2- 172.21.15.89:0/1886854091 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x560e17e383e0 0x560e17e3a7d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3bb8cad9d338f193 server_cookie=a6f8ba698b4c9775 in_seq=0 out_seq=0 2024-09-10T08:45:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.869+0000 7fa4f6a00640 10 client.26447 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:45:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.869+0000 7fa4f6a00640 1 -- 172.21.15.89:0/1886854091 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e17d1fe60 con 0x560e17e383e0 2024-09-10T08:45:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.869+0000 7fa4f6a00640 10 client.26447 handle_command_reply: tid=0 2024-09-10T08:45:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.869+0000 7fa51a64d740 1 client.26447 shutdown 2024-09-10T08:45:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.869+0000 7fa51a64d740 1 -- 172.21.15.89:0/1886854091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17db1e40 msgr2=0x560e17dc5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.869+0000 7fa51a64d740 1 --2- 172.21.15.89:0/1886854091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17db1e40 0x560e17dc5680 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa504038040 tx=0x7fa50401fdb0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.869+0000 7fa51a64d740 1 -- 172.21.15.89:0/1886854091 shutdown_connections 2024-09-10T08:45:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.869+0000 7fa51a64d740 1 --2- 172.21.15.89:0/1886854091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560e17dbd020 0x560e17dc0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.869+0000 7fa51a64d740 1 --2- 172.21.15.89:0/1886854091 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e17db2760 0x560e17dc09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.869+0000 7fa51a64d740 1 --2- 172.21.15.89:0/1886854091 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x560e17e383e0 0x560e17e3a7d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.869+0000 7fa51a64d740 1 --2- 172.21.15.89:0/1886854091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e17db1e40 0x560e17dc5680 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.869+0000 7fa51a64d740 1 -- 172.21.15.89:0/1886854091 >> 172.21.15.89:0/1886854091 conn(0x560e17dae3d0 msgr2=0x560e17dad490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:09.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.869+0000 7fa4f6a00640 0 client.26447 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:45:09.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.869+0000 7fa51a64d740 1 -- 172.21.15.89:0/1886854091 shutdown_connections 2024-09-10T08:45:09.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.869+0000 7fa51a64d740 1 -- 172.21.15.89:0/1886854091 wait complete. 2024-09-10T08:45:09.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.870+0000 7fa51a64d740 20 client.26447 trim_cache size 0 max 16384 2024-09-10T08:45:09.876 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:45:09.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.870+0000 7fa516600640 1 -- 172.21.15.89:0/721335402 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa4f80a2b80 msgr2=0x7fa4f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:09.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.870+0000 7fa516600640 1 --2- 172.21.15.89:0/721335402 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa4f80a2b80 0x7fa4f80a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x560e17daf230 tx=0x7fa504005f90 comp rx=0 tx=0).stop 2024-09-10T08:45:09.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.870+0000 7fa516600640 1 -- 172.21.15.89:0/721335402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5101047e0 msgr2=0x7fa51019a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:09.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.870+0000 7fa516600640 1 --2- 172.21.15.89:0/721335402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5101047e0 0x7fa51019a9e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc006010 tx=0x7fa4fc0042f0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.870+0000 7fa516600640 1 -- 172.21.15.89:0/721335402 shutdown_connections 2024-09-10T08:45:09.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.870+0000 7fa516600640 1 --2- 172.21.15.89:0/721335402 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa4f80a2b80 0x7fa4f80a5040 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.870+0000 7fa516600640 1 --2- 172.21.15.89:0/721335402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa510108f60 0x7fa51019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.870+0000 7fa516600640 1 --2- 172.21.15.89:0/721335402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa5101047e0 0x7fa51019a9e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.870+0000 7fa516600640 1 --2- 172.21.15.89:0/721335402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa510103e30 0x7fa51019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.870+0000 7fa516600640 1 -- 172.21.15.89:0/721335402 >> 172.21.15.89:0/721335402 conn(0x7fa5100ff910 msgr2=0x7fa510100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:09.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.870+0000 7fa516600640 1 -- 172.21.15.89:0/721335402 shutdown_connections 2024-09-10T08:45:09.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:09.871+0000 7fa516600640 1 -- 172.21.15.89:0/721335402 wait complete. 2024-09-10T08:45:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.528+0000 7f58a1600640 1 -- 172.21.15.89:0/1065509286 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a81dfc1ce0 con 0x55a81e0dbb00 2024-09-10T08:45:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.528+0000 7f58a1600640 10 client.26317 handle_command_reply: tid=0 2024-09-10T08:45:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.528+0000 7f58c1086740 1 client.26317 shutdown 2024-09-10T08:45:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.528+0000 7f58c1086740 1 -- 172.21.15.89:0/1065509286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e0606c0 msgr2=0x55a81e05b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.528+0000 7f58c1086740 1 --2- 172.21.15.89:0/1065509286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e0606c0 0x55a81e05b120 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f58a800e4a0 tx=0x7f58a800e890 comp rx=0 tx=0).stop 2024-09-10T08:45:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.529+0000 7f58c1086740 1 -- 172.21.15.89:0/1065509286 shutdown_connections 2024-09-10T08:45:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.529+0000 7f58c1086740 1 --2- 172.21.15.89:0/1065509286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a81e0606c0 0x55a81e05b120 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.529+0000 7f58c1086740 1 --2- 172.21.15.89:0/1065509286 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55a81e0dbb00 0x55a81e0ddef0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:10.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.529+0000 7f58c1086740 1 --2- 172.21.15.89:0/1065509286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a81e055e00 0x55a81e05abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:10.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.529+0000 7f58c1086740 1 --2- 172.21.15.89:0/1065509286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a81e0554e0 0x55a81e05a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:10.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.529+0000 7f58a1600640 0 client.26317 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:10.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.529+0000 7f58c1086740 1 -- 172.21.15.89:0/1065509286 >> 172.21.15.89:0/1065509286 conn(0x55a81e051a70 msgr2=0x55a81e05d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:10.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.529+0000 7f58c1086740 1 -- 172.21.15.89:0/1065509286 shutdown_connections 2024-09-10T08:45:10.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.529+0000 7f58c1086740 1 -- 172.21.15.89:0/1065509286 wait complete. 2024-09-10T08:45:10.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.529+0000 7f58c1086740 20 client.26317 trim_cache size 0 max 16384 2024-09-10T08:45:10.531 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:45:10.532 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 29, 2024-09-10T08:45:10.532 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24554, 2024-09-10T08:45:10.532 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 14, 2024-09-10T08:45:10.532 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": { 2024-09-10T08:45:10.532 INFO:teuthology.orchestra.run.smithi089.stdout: "2809171f": { 2024-09-10T08:45:10.532 INFO:teuthology.orchestra.run.smithi089.stdout: "version": 14, 2024-09-10T08:45:10.532 INFO:teuthology.orchestra.run.smithi089.stdout: "age_ref": 0.0, 2024-09-10T08:45:10.532 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:45:10.532 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCED", 2024-09-10T08:45:10.532 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:45:10.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:45:10.532 INFO:teuthology.orchestra.run.smithi089.stdout: "timeout": 90.0, 2024-09-10T08:45:10.532 INFO:teuthology.orchestra.run.smithi089.stdout: "expiration": 142.8, 2024-09-10T08:45:10.533 INFO:teuthology.orchestra.run.smithi089.stdout: "members": { 2024-09-10T08:45:10.533 INFO:teuthology.orchestra.run.smithi089.stdout: "file:/": { 2024-09-10T08:45:10.533 INFO:teuthology.orchestra.run.smithi089.stdout: "excluded": false, 2024-09-10T08:45:10.533 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:45:10.533 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCED", 2024-09-10T08:45:10.533 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:45:10.533 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:45:10.533 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:45:10.533 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:45:10.533 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:45:10.533 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:45:10.533 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:45:10.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.530+0000 7f58be600640 1 -- 172.21.15.89:0/4287148171 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f589c0a2c80 msgr2=0x7f589c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:10.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.530+0000 7f58be600640 1 --2- 172.21.15.89:0/4287148171 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f589c0a2c80 0x7f589c0a5140 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f58a40ed630 tx=0x7f58a40f3660 comp rx=0 tx=0).stop 2024-09-10T08:45:10.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.530+0000 7f58be600640 1 -- 172.21.15.89:0/4287148171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b810e440 msgr2=0x7f58b806c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:10.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.530+0000 7f58be600640 1 --2- 172.21.15.89:0/4287148171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b810e440 0x7f58b806c910 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f58a402a8f0 tx=0x7f58a40305b0 comp rx=0 tx=0).stop 2024-09-10T08:45:10.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.530+0000 7f58be600640 1 -- 172.21.15.89:0/4287148171 shutdown_connections 2024-09-10T08:45:10.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.530+0000 7f58be600640 1 --2- 172.21.15.89:0/4287148171 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f589c0a2c80 0x7f589c0a5140 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:10.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.530+0000 7f58be600640 1 --2- 172.21.15.89:0/4287148171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58b810e440 0x7f58b806c910 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:10.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.530+0000 7f58be600640 1 --2- 172.21.15.89:0/4287148171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58b8074e30 0x7f58b8073d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:10.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.530+0000 7f58be600640 1 --2- 172.21.15.89:0/4287148171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58b8074510 0x7f58b8073820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:10.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.530+0000 7f58be600640 1 -- 172.21.15.89:0/4287148171 >> 172.21.15.89:0/4287148171 conn(0x7f58b80fd8e0 msgr2=0x7f58b810fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:10.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.530+0000 7f58be600640 1 -- 172.21.15.89:0/4287148171 shutdown_connections 2024-09-10T08:45:10.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:10.530+0000 7f58be600640 1 -- 172.21.15.89:0/4287148171 wait complete. 2024-09-10T08:45:11.212 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:10 smithi089 ceph-mon[30757]: pgmap v550: 129 pgs: 129 active+clean; 320 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.1 MiB/s wr, 1.02k op/s 2024-09-10T08:45:11.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:10 smithi149 ceph-mon[36167]: pgmap v550: 129 pgs: 129 active+clean; 320 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.1 MiB/s wr, 1.02k op/s 2024-09-10T08:45:11.254 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:45:11.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:10 smithi111 ceph-mon[35220]: pgmap v550: 129 pgs: 129 active+clean; 320 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.1 MiB/s wr, 1.02k op/s 2024-09-10T08:45:11.782 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2809171f', quiesce took 11.5 seconds. Will release after: 11.25 2024-09-10T08:45:11.825 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:12.483 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.482+0000 7f7a2ce00640 1 -- 172.21.15.89:0/111532216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a28103e30 msgr2=0x7f7a28104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:12.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.482+0000 7f7a2ce00640 1 --2- 172.21.15.89:0/111532216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a28103e30 0x7f7a28104210 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009900 tx=0x7f7a1802a400 comp rx=0 tx=0).stop 2024-09-10T08:45:12.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.482+0000 7f7a2ce00640 1 -- 172.21.15.89:0/111532216 shutdown_connections 2024-09-10T08:45:12.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.482+0000 7f7a2ce00640 1 --2- 172.21.15.89:0/111532216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a28108f60 0x7f7a28110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.482+0000 7f7a2ce00640 1 --2- 172.21.15.89:0/111532216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a281047e0 0x7f7a28108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.482+0000 7f7a2ce00640 1 --2- 172.21.15.89:0/111532216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a28103e30 0x7f7a28104210 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.482+0000 7f7a2ce00640 1 -- 172.21.15.89:0/111532216 >> 172.21.15.89:0/111532216 conn(0x7f7a280ff910 msgr2=0x7f7a28101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:12.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.482+0000 7f7a2ce00640 1 -- 172.21.15.89:0/111532216 shutdown_connections 2024-09-10T08:45:12.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.482+0000 7f7a2ce00640 1 -- 172.21.15.89:0/111532216 wait complete. 2024-09-10T08:45:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.483+0000 7f7a2ce00640 1 Processor -- start 2024-09-10T08:45:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.483+0000 7f7a2ce00640 1 -- start start 2024-09-10T08:45:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.484+0000 7f7a2ce00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a28103e30 0x7f7a2819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.484+0000 7f7a2ce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a281047e0 0x7f7a2819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.484+0000 7f7a2ce00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a28108f60 0x7f7a2819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.484+0000 7f7a2ce00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7a28113fc0 con 0x7f7a281047e0 2024-09-10T08:45:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.484+0000 7f7a2ce00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7a28113e40 con 0x7f7a28103e30 2024-09-10T08:45:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.484+0000 7f7a2ce00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7a28114140 con 0x7f7a28108f60 2024-09-10T08:45:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.484+0000 7f7a27400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a28103e30 0x7f7a2819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.484+0000 7f7a27400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a28103e30 0x7f7a2819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50552/0 (socket says 172.21.15.89:50552) 2024-09-10T08:45:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.484+0000 7f7a27400640 1 -- 172.21.15.89:0/537301956 learned_addr learned my addr 172.21.15.89:0/537301956 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.484+0000 7f7a26a00640 1 --2- 172.21.15.89:0/537301956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a281047e0 0x7f7a2819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.484+0000 7f7a27e00640 1 --2- 172.21.15.89:0/537301956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a28108f60 0x7f7a2819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.484+0000 7f7a27400640 1 -- 172.21.15.89:0/537301956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a28108f60 msgr2=0x7f7a2819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:12.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.484+0000 7f7a27400640 1 --2- 172.21.15.89:0/537301956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a28108f60 0x7f7a2819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.484+0000 7f7a27400640 1 -- 172.21.15.89:0/537301956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a281047e0 msgr2=0x7f7a2819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:12.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.484+0000 7f7a27400640 1 --2- 172.21.15.89:0/537301956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a281047e0 0x7f7a2819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.484+0000 7f7a27400640 1 -- 172.21.15.89:0/537301956 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2819f490 con 0x7f7a28103e30 2024-09-10T08:45:12.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.485+0000 7f7a27400640 1 --2- 172.21.15.89:0/537301956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a28103e30 0x7f7a2819a4e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7a18006010 tx=0x7f7a18004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:12.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.485+0000 7f7a1fe00640 1 -- 172.21.15.89:0/537301956 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1803f030 con 0x7f7a28103e30 2024-09-10T08:45:12.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.485+0000 7f7a2ce00640 1 -- 172.21.15.89:0/537301956 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2819f720 con 0x7f7a28103e30 2024-09-10T08:45:12.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.485+0000 7f7a1fe00640 1 -- 172.21.15.89:0/537301956 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a18004440 con 0x7f7a28103e30 2024-09-10T08:45:12.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.485+0000 7f7a1fe00640 1 -- 172.21.15.89:0/537301956 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18035070 con 0x7f7a28103e30 2024-09-10T08:45:12.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.485+0000 7f7a2ce00640 1 -- 172.21.15.89:0/537301956 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a281a7060 con 0x7f7a28103e30 2024-09-10T08:45:12.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.488+0000 7f7a30de6740 1 Processor -- start 2024-09-10T08:45:12.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.488+0000 7f7a30de6740 1 -- start start 2024-09-10T08:45:12.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.489+0000 7f7a1fe00640 1 -- 172.21.15.89:0/537301956 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7a18042030 con 0x7f7a28103e30 2024-09-10T08:45:12.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.489+0000 7f7a30de6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b429e40 0x55b52b42a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.489+0000 7f7a30de6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b52b42a760 0x55b52b434ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.489+0000 7f7a30de6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b435020 0x55b52b437410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.489+0000 7f7a30de6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b52b3903e0 con 0x55b52b429e40 2024-09-10T08:45:12.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.489+0000 7f7a30de6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b52b337d80 con 0x55b52b435020 2024-09-10T08:45:12.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.489+0000 7f7a30de6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b52b38c3a0 con 0x55b52b42a760 2024-09-10T08:45:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.489+0000 7f7a26a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b435020 0x55b52b437410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.489+0000 7f7a27400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b52b42a760 0x55b52b434ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.489+0000 7f7a27e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b429e40 0x55b52b42a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.489+0000 7f7a26a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b435020 0x55b52b437410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50554/0 (socket says 172.21.15.89:50554) 2024-09-10T08:45:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.489+0000 7f7a26a00640 1 -- 172.21.15.89:0/1208020222 learned_addr learned my addr 172.21.15.89:0/1208020222 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.490+0000 7f7a26a00640 1 -- 172.21.15.89:0/1208020222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b52b42a760 msgr2=0x55b52b434ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.490+0000 7f7a26a00640 1 --2- 172.21.15.89:0/1208020222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b52b42a760 0x55b52b434ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.490+0000 7f7a26a00640 1 -- 172.21.15.89:0/1208020222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b429e40 msgr2=0x55b52b42a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.490+0000 7f7a26a00640 1 --2- 172.21.15.89:0/1208020222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b429e40 0x55b52b42a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.490+0000 7f7a26a00640 1 -- 172.21.15.89:0/1208020222 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b52b356ae0 con 0x55b52b435020 2024-09-10T08:45:12.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.490+0000 7f7a27e00640 1 --2- 172.21.15.89:0/1208020222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b429e40 0x55b52b42a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.490+0000 7f7a26a00640 1 --2- 172.21.15.89:0/1208020222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b435020 0x55b52b437410 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009c90 tx=0x7f7a14011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b519604c55a5a88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.490+0000 7f7a1cc00640 1 -- 172.21.15.89:0/1208020222 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1401e070 con 0x55b52b435020 2024-09-10T08:45:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.490+0000 7f7a1cc00640 1 -- 172.21.15.89:0/1208020222 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a14019b10 con 0x55b52b435020 2024-09-10T08:45:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.490+0000 7f7a1cc00640 1 -- 172.21.15.89:0/1208020222 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1400ea00 con 0x55b52b435020 2024-09-10T08:45:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.491+0000 7f7a1fe00640 1 --2- 172.21.15.89:0/537301956 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7a0c0a2c00 0x7f7a0c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.491+0000 7f7a1fe00640 1 -- 172.21.15.89:0/537301956 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f7a180ef670 con 0x7f7a28103e30 2024-09-10T08:45:12.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.491+0000 7f7a30de6740 1 -- 172.21.15.89:0/1208020222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b435020 msgr2=0x55b52b437410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.491+0000 7f7a30de6740 1 --2- 172.21.15.89:0/1208020222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b435020 0x55b52b437410 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009c90 tx=0x7f7a14011810 comp rx=0 tx=0).stop 2024-09-10T08:45:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.491+0000 7f7a27e00640 1 --2- 172.21.15.89:0/537301956 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7a0c0a2c00 0x7f7a0c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.491+0000 7f7a30de6740 1 -- 172.21.15.89:0/1208020222 shutdown_connections 2024-09-10T08:45:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.491+0000 7f7a30de6740 1 --2- 172.21.15.89:0/1208020222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b435020 0x55b52b437410 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.491+0000 7f7a30de6740 1 --2- 172.21.15.89:0/1208020222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b52b42a760 0x55b52b434ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.491+0000 7f7a30de6740 1 --2- 172.21.15.89:0/1208020222 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b429e40 0x55b52b42a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.491+0000 7f7a30de6740 1 -- 172.21.15.89:0/1208020222 >> 172.21.15.89:0/1208020222 conn(0x55b52b4263d0 msgr2=0x55b52b4267b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.492+0000 7f7a30de6740 1 -- 172.21.15.89:0/1208020222 shutdown_connections 2024-09-10T08:45:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.492+0000 7f7a30de6740 1 -- 172.21.15.89:0/1208020222 wait complete. 2024-09-10T08:45:12.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.492+0000 7f7a27e00640 1 --2- 172.21.15.89:0/537301956 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7a0c0a2c00 0x7f7a0c0a50c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7a200099a0 tx=0x7f7a20009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.492+0000 7f7a30de6740 1 -- start start 2024-09-10T08:45:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.492+0000 7f7a30de6740 1 Processor -- start 2024-09-10T08:45:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.493+0000 7f7a30de6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b429e40 0x55b52b432170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.493+0000 7f7a30de6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b42a760 0x55b52b4326b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.493+0000 7f7a30de6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b52b432bf0 0x55b52b441090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.493+0000 7f7a30de6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b52b437bf0 con 0x55b52b42a760 2024-09-10T08:45:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.493+0000 7f7a30de6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b52b34b0f0 con 0x55b52b429e40 2024-09-10T08:45:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.493+0000 7f7a30de6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b52b437d70 con 0x55b52b432bf0 2024-09-10T08:45:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.493+0000 7f7a30de6740 10 client.0 _ll_register_callbacks cb 0x55b52b332bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:12.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.493+0000 7f7a27400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b429e40 0x55b52b432170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.493+0000 7f7a27400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b429e40 0x55b52b432170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50562/0 (socket says 172.21.15.89:50562) 2024-09-10T08:45:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.493+0000 7f7a27400640 1 -- 172.21.15.89:0/3193310942 learned_addr learned my addr 172.21.15.89:0/3193310942 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.493+0000 7f7a27e00640 1 --2- 172.21.15.89:0/3193310942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b52b432bf0 0x55b52b441090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.493+0000 7f7a26a00640 1 --2- 172.21.15.89:0/3193310942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b42a760 0x55b52b4326b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.493+0000 7f7a27400640 1 -- 172.21.15.89:0/3193310942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b52b432bf0 msgr2=0x55b52b441090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.493+0000 7f7a27400640 1 --2- 172.21.15.89:0/3193310942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b52b432bf0 0x55b52b441090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.493+0000 7f7a27400640 1 -- 172.21.15.89:0/3193310942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b42a760 msgr2=0x55b52b4326b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.493+0000 7f7a27400640 1 --2- 172.21.15.89:0/3193310942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b42a760 0x55b52b4326b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.493+0000 7f7a27400640 1 -- 172.21.15.89:0/3193310942 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b52b1013c0 con 0x55b52b429e40 2024-09-10T08:45:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.493+0000 7f7a26a00640 1 --2- 172.21.15.89:0/3193310942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b42a760 0x55b52b4326b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.494+0000 7f7a27400640 1 --2- 172.21.15.89:0/3193310942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b429e40 0x55b52b432170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7a180c41e0 tx=0x7f7a180f2ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb97c49f638048d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.494+0000 7f7a11600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:45:12.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.494+0000 7f7a11600640 1 -- 172.21.15.89:0/3193310942 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18049070 con 0x55b52b429e40 2024-09-10T08:45:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.494+0000 7f7a30de6740 1 -- 172.21.15.89:0/3193310942 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a14019b10 con 0x55b52b429e40 2024-09-10T08:45:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.494+0000 7f7a11600640 1 -- 172.21.15.89:0/3193310942 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f7a180ef2b0 con 0x55b52b429e40 2024-09-10T08:45:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.494+0000 7f7a11600640 1 -- 172.21.15.89:0/3193310942 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7a180b7d40 con 0x55b52b429e40 2024-09-10T08:45:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.494+0000 7f7a30de6740 10 client.26459 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.494+0000 7f7a30de6740 1 -- 172.21.15.89:0/3193310942 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b52b4442a0 con 0x55b52b429e40 2024-09-10T08:45:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.494+0000 7f7a11600640 1 -- 172.21.15.89:0/3193310942 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a14019b10 con 0x55b52b429e40 2024-09-10T08:45:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.495+0000 7f7a11600640 1 -- 172.21.15.89:0/3193310942 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f7a18003c60 con 0x55b52b429e40 2024-09-10T08:45:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.495+0000 7f7a30de6740 10 client.26459 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.495+0000 7f7a30de6740 10 client.26459 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:45:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.495+0000 7f7a30de6740 20 client.26459 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.495+0000 7f7a30de6740 1 --2- 172.21.15.89:0/3193310942 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55b52b445750 0x55b52b447bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.495+0000 7f7a30de6740 4 client.26459 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:45:12.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.495+0000 7f7a30de6740 1 -- 172.21.15.89:0/3193310942 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b52b39d950 con 0x55b52b445750 2024-09-10T08:45:12.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.495+0000 7f7a26a00640 1 --2- 172.21.15.89:0/3193310942 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55b52b445750 0x55b52b447bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.496+0000 7f7a26a00640 1 --2- 172.21.15.89:0/3193310942 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55b52b445750 0x55b52b447bf0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=379db64fc7e25f79 server_cookie=bdd0db763eafe3a6 in_seq=0 out_seq=0 2024-09-10T08:45:12.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.496+0000 7f7a11600640 10 client.26459 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:45:12.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.496+0000 7f7a11600640 1 -- 172.21.15.89:0/3193310942 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b52b39d950 con 0x55b52b445750 2024-09-10T08:45:12.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.496+0000 7f7a11600640 10 client.26459 handle_command_reply: tid=0 2024-09-10T08:45:12.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.496+0000 7f7a30de6740 1 client.26459 shutdown 2024-09-10T08:45:12.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.497+0000 7f7a30de6740 1 -- 172.21.15.89:0/3193310942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b429e40 msgr2=0x55b52b432170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:12.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.497+0000 7f7a30de6740 1 --2- 172.21.15.89:0/3193310942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b429e40 0x55b52b432170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7a180c41e0 tx=0x7f7a180f2ca0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.497+0000 7f7a30de6740 1 -- 172.21.15.89:0/3193310942 shutdown_connections 2024-09-10T08:45:12.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.497+0000 7f7a30de6740 1 --2- 172.21.15.89:0/3193310942 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55b52b445750 0x55b52b447bf0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.497+0000 7f7a30de6740 1 --2- 172.21.15.89:0/3193310942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b52b432bf0 0x55b52b441090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.497+0000 7f7a30de6740 1 --2- 172.21.15.89:0/3193310942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b42a760 0x55b52b4326b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.497+0000 7f7a30de6740 1 --2- 172.21.15.89:0/3193310942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b429e40 0x55b52b432170 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.497+0000 7f7a30de6740 1 -- 172.21.15.89:0/3193310942 >> 172.21.15.89:0/3193310942 conn(0x55b52b4263d0 msgr2=0x55b52b42c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:12.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.497+0000 7f7a11600640 0 client.26459 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:45:12.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.497+0000 7f7a30de6740 1 -- 172.21.15.89:0/3193310942 shutdown_connections 2024-09-10T08:45:12.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.497+0000 7f7a30de6740 1 -- 172.21.15.89:0/3193310942 wait complete. 2024-09-10T08:45:12.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.497+0000 7f7a30de6740 20 client.26459 trim_cache size 0 max 16384 2024-09-10T08:45:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.511+0000 7f7a30de6740 1 Processor -- start 2024-09-10T08:45:12.512 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.512+0000 7f7a30de6740 1 -- start start 2024-09-10T08:45:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.512+0000 7f7a30de6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b429e40 0x55b52b42c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.512+0000 7f7a30de6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b52b42a760 0x55b52b440e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.512+0000 7f7a30de6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b432bf0 0x55b52b44d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.512+0000 7f7a30de6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281b3750 con 0x55b52b432bf0 2024-09-10T08:45:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.512+0000 7f7a30de6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281b1020 con 0x55b52b429e40 2024-09-10T08:45:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.512+0000 7f7a30de6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281aee70 con 0x55b52b42a760 2024-09-10T08:45:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.512+0000 7f7a27400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b429e40 0x55b52b42c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.514 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.512+0000 7f7a27400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b429e40 0x55b52b42c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50566/0 (socket says 172.21.15.89:50566) 2024-09-10T08:45:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.512+0000 7f7a27400640 1 -- 172.21.15.89:0/3232508002 learned_addr learned my addr 172.21.15.89:0/3232508002 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.512+0000 7f7a27e00640 1 --2- 172.21.15.89:0/3232508002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b432bf0 0x55b52b44d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.512+0000 7f7a26a00640 1 --2- 172.21.15.89:0/3232508002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b52b42a760 0x55b52b440e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.512+0000 7f7a27400640 1 -- 172.21.15.89:0/3232508002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b52b42a760 msgr2=0x55b52b440e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.512+0000 7f7a27400640 1 --2- 172.21.15.89:0/3232508002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b52b42a760 0x55b52b440e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.512+0000 7f7a27400640 1 -- 172.21.15.89:0/3232508002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b432bf0 msgr2=0x55b52b44d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.513+0000 7f7a27400640 1 --2- 172.21.15.89:0/3232508002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b432bf0 0x55b52b44d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.513+0000 7f7a27400640 1 -- 172.21.15.89:0/3232508002 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b52b42a590 con 0x55b52b429e40 2024-09-10T08:45:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.513+0000 7f7a27400640 1 --2- 172.21.15.89:0/3232508002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b429e40 0x55b52b42c920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7a18005ab0 tx=0x7f7a18005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a81569c68b53e07b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.513+0000 7f7a13e00640 1 -- 172.21.15.89:0/3232508002 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a180d9070 con 0x55b52b429e40 2024-09-10T08:45:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.513+0000 7f7a13e00640 1 -- 172.21.15.89:0/3232508002 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a180f2e50 con 0x55b52b429e40 2024-09-10T08:45:12.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.513+0000 7f7a13e00640 1 -- 172.21.15.89:0/3232508002 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18049070 con 0x55b52b429e40 2024-09-10T08:45:12.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.514+0000 7f7a30de6740 1 -- 172.21.15.89:0/3232508002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b429e40 msgr2=0x55b52b42c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:12.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.514+0000 7f7a30de6740 1 --2- 172.21.15.89:0/3232508002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b429e40 0x55b52b42c920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7a18005ab0 tx=0x7f7a18005b50 comp rx=0 tx=0).stop 2024-09-10T08:45:12.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.514+0000 7f7a30de6740 1 -- 172.21.15.89:0/3232508002 shutdown_connections 2024-09-10T08:45:12.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.514+0000 7f7a30de6740 1 --2- 172.21.15.89:0/3232508002 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b432bf0 0x55b52b44d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.514+0000 7f7a30de6740 1 --2- 172.21.15.89:0/3232508002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b52b42a760 0x55b52b440e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.514+0000 7f7a30de6740 1 --2- 172.21.15.89:0/3232508002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b429e40 0x55b52b42c920 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.514+0000 7f7a30de6740 1 -- 172.21.15.89:0/3232508002 >> 172.21.15.89:0/3232508002 conn(0x55b52b4263d0 msgr2=0x55b52b4267b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:12.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.514+0000 7f7a30de6740 1 -- 172.21.15.89:0/3232508002 shutdown_connections 2024-09-10T08:45:12.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.514+0000 7f7a30de6740 1 -- 172.21.15.89:0/3232508002 wait complete. 2024-09-10T08:45:12.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.515+0000 7f7a30de6740 1 -- start start 2024-09-10T08:45:12.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.515+0000 7f7a30de6740 1 Processor -- start 2024-09-10T08:45:12.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.515+0000 7f7a30de6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b52b429e40 0x55b52b43d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.515+0000 7f7a30de6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b42a760 0x55b52b43d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.515+0000 7f7a30de6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b432bf0 0x55b52b42ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.515+0000 7f7a30de6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281af4e0 con 0x55b52b42a760 2024-09-10T08:45:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.515+0000 7f7a30de6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281b0950 con 0x55b52b432bf0 2024-09-10T08:45:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.515+0000 7f7a30de6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281afbb0 con 0x55b52b429e40 2024-09-10T08:45:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.515+0000 7f7a30de6740 10 client.0 _ll_register_callbacks cb 0x7f7a180f2da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.515+0000 7f7a27e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b432bf0 0x55b52b42ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.516+0000 7f7a27e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b432bf0 0x55b52b42ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50570/0 (socket says 172.21.15.89:50570) 2024-09-10T08:45:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.516+0000 7f7a26a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b42a760 0x55b52b43d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.516+0000 7f7a27e00640 1 -- 172.21.15.89:0/176989472 learned_addr learned my addr 172.21.15.89:0/176989472 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.516+0000 7f7a27400640 1 --2- 172.21.15.89:0/176989472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b52b429e40 0x55b52b43d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.516+0000 7f7a27e00640 1 -- 172.21.15.89:0/176989472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b52b429e40 msgr2=0x55b52b43d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.516+0000 7f7a27e00640 1 --2- 172.21.15.89:0/176989472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b52b429e40 0x55b52b43d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.516+0000 7f7a27e00640 1 -- 172.21.15.89:0/176989472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b42a760 msgr2=0x55b52b43d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.516+0000 7f7a27e00640 1 --2- 172.21.15.89:0/176989472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b42a760 0x55b52b43d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.516+0000 7f7a27e00640 1 -- 172.21.15.89:0/176989472 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b52b433110 con 0x55b52b432bf0 2024-09-10T08:45:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.516+0000 7f7a27e00640 1 --2- 172.21.15.89:0/176989472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b432bf0 0x55b52b42ea60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7a20004270 tx=0x7f7a20004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1628684b15372f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.516+0000 7f7a11600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:45:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.516+0000 7f7a11600640 1 -- 172.21.15.89:0/176989472 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2002e060 con 0x55b52b432bf0 2024-09-10T08:45:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.517+0000 7f7a30de6740 1 -- 172.21.15.89:0/176989472 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a180f2e50 con 0x55b52b432bf0 2024-09-10T08:45:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.517+0000 7f7a11600640 1 -- 172.21.15.89:0/176989472 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f7a2001fa90 con 0x55b52b432bf0 2024-09-10T08:45:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.517+0000 7f7a11600640 1 -- 172.21.15.89:0/176989472 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7a20035050 con 0x55b52b432bf0 2024-09-10T08:45:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.517+0000 7f7a30de6740 10 client.26465 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:12.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.517+0000 7f7a30de6740 1 -- 172.21.15.89:0/176989472 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b52b4afec0 con 0x55b52b432bf0 2024-09-10T08:45:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.517+0000 7f7a11600640 1 -- 172.21.15.89:0/176989472 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a180f2e50 con 0x55b52b432bf0 2024-09-10T08:45:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.517+0000 7f7a11600640 1 -- 172.21.15.89:0/176989472 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f7a2001e3f0 con 0x55b52b432bf0 2024-09-10T08:45:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.517+0000 7f7a30de6740 10 client.26465 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.517+0000 7f7a30de6740 10 client.26465 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:45:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.517+0000 7f7a30de6740 20 client.26465 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.517+0000 7f7a30de6740 1 --2- 172.21.15.89:0/176989472 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55b52b4b05f0 0x55b52b4b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.517+0000 7f7a30de6740 4 client.26465 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:45:12.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.517+0000 7f7a30de6740 1 -- 172.21.15.89:0/176989472 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b52b397e60 con 0x55b52b4b05f0 2024-09-10T08:45:12.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.518+0000 7f7a27400640 1 --2- 172.21.15.89:0/176989472 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55b52b4b05f0 0x55b52b4b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.518+0000 7f7a27400640 1 --2- 172.21.15.89:0/176989472 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55b52b4b05f0 0x55b52b4b2a10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=525dc6a1cbc1abea server_cookie=b349aa9d059c6d7a in_seq=0 out_seq=0 2024-09-10T08:45:12.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.518+0000 7f7a11600640 10 client.26465 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:45:12.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.518+0000 7f7a11600640 1 -- 172.21.15.89:0/176989472 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b52b397e60 con 0x55b52b4b05f0 2024-09-10T08:45:12.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.518+0000 7f7a11600640 10 client.26465 handle_command_reply: tid=0 2024-09-10T08:45:12.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.518+0000 7f7a30de6740 1 client.26465 shutdown 2024-09-10T08:45:12.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.519+0000 7f7a30de6740 1 -- 172.21.15.89:0/176989472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b432bf0 msgr2=0x55b52b42ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:12.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.519+0000 7f7a30de6740 1 --2- 172.21.15.89:0/176989472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b432bf0 0x55b52b42ea60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7a20004270 tx=0x7f7a20004a40 comp rx=0 tx=0).stop 2024-09-10T08:45:12.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.519+0000 7f7a30de6740 1 -- 172.21.15.89:0/176989472 shutdown_connections 2024-09-10T08:45:12.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.519+0000 7f7a30de6740 1 --2- 172.21.15.89:0/176989472 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55b52b4b05f0 0x55b52b4b2a10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.519+0000 7f7a30de6740 1 --2- 172.21.15.89:0/176989472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b52b432bf0 0x55b52b42ea60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.519+0000 7f7a30de6740 1 --2- 172.21.15.89:0/176989472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b52b42a760 0x55b52b43d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.519+0000 7f7a30de6740 1 --2- 172.21.15.89:0/176989472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b52b429e40 0x55b52b43d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.519+0000 7f7a11600640 0 client.26465 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:45:12.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.519+0000 7f7a30de6740 1 -- 172.21.15.89:0/176989472 >> 172.21.15.89:0/176989472 conn(0x55b52b4263d0 msgr2=0x55b52b424c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:12.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.519+0000 7f7a30de6740 1 -- 172.21.15.89:0/176989472 shutdown_connections 2024-09-10T08:45:12.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.519+0000 7f7a30de6740 1 -- 172.21.15.89:0/176989472 wait complete. 2024-09-10T08:45:12.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.519+0000 7f7a30de6740 20 client.26465 trim_cache size 0 max 16384 2024-09-10T08:45:12.524 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:45:12.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.520+0000 7f7a2ce00640 1 -- 172.21.15.89:0/537301956 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7a0c0a2c00 msgr2=0x7f7a0c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:12.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.520+0000 7f7a2ce00640 1 --2- 172.21.15.89:0/537301956 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7a0c0a2c00 0x7f7a0c0a50c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7a200099a0 tx=0x7f7a20009500 comp rx=0 tx=0).stop 2024-09-10T08:45:12.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.520+0000 7f7a2ce00640 1 -- 172.21.15.89:0/537301956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a28103e30 msgr2=0x7f7a2819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:12.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.520+0000 7f7a2ce00640 1 --2- 172.21.15.89:0/537301956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a28103e30 0x7f7a2819a4e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7a18006010 tx=0x7f7a18004040 comp rx=0 tx=0).stop 2024-09-10T08:45:12.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.520+0000 7f7a2ce00640 1 -- 172.21.15.89:0/537301956 shutdown_connections 2024-09-10T08:45:12.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.520+0000 7f7a2ce00640 1 --2- 172.21.15.89:0/537301956 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7a0c0a2c00 0x7f7a0c0a50c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.520+0000 7f7a2ce00640 1 --2- 172.21.15.89:0/537301956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a28108f60 0x7f7a2819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.520+0000 7f7a2ce00640 1 --2- 172.21.15.89:0/537301956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a281047e0 0x7f7a2819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.520+0000 7f7a2ce00640 1 --2- 172.21.15.89:0/537301956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a28103e30 0x7f7a2819a4e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.520+0000 7f7a2ce00640 1 -- 172.21.15.89:0/537301956 >> 172.21.15.89:0/537301956 conn(0x7f7a280ff910 msgr2=0x7f7a28100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:12.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.520+0000 7f7a2ce00640 1 -- 172.21.15.89:0/537301956 shutdown_connections 2024-09-10T08:45:12.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:12.520+0000 7f7a2ce00640 1 -- 172.21.15.89:0/537301956 wait complete. 2024-09-10T08:45:13.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:12 smithi149 ceph-mon[36167]: pgmap v551: 129 pgs: 129 active+clean; 301 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.3 MiB/s wr, 1.12k op/s 2024-09-10T08:45:13.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:12 smithi111 ceph-mon[35220]: pgmap v551: 129 pgs: 129 active+clean; 301 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.3 MiB/s wr, 1.12k op/s 2024-09-10T08:45:13.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:12 smithi089 ceph-mon[30757]: pgmap v551: 129 pgs: 129 active+clean; 301 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.3 MiB/s wr, 1.12k op/s 2024-09-10T08:45:13.472 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:45:13.875 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.482+0000 7fe967200640 1 -- 172.21.15.89:0/3554988673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe968077040 msgr2=0x7fe968112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.482+0000 7fe967200640 1 --2- 172.21.15.89:0/3554988673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe968077040 0x7fe968112330 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe950009900 tx=0x7fe95002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.483+0000 7fe967200640 1 -- 172.21.15.89:0/3554988673 shutdown_connections 2024-09-10T08:45:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.483+0000 7fe967200640 1 --2- 172.21.15.89:0/3554988673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe968112870 0x7fe968114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.483+0000 7fe967200640 1 --2- 172.21.15.89:0/3554988673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe968077040 0x7fe968112330 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.483+0000 7fe967200640 1 --2- 172.21.15.89:0/3554988673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe968076720 0x7fe968076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.484 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.483+0000 7fe967200640 1 -- 172.21.15.89:0/3554988673 >> 172.21.15.89:0/3554988673 conn(0x7fe9680ff8d0 msgr2=0x7fe968101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.483+0000 7fe967200640 1 -- 172.21.15.89:0/3554988673 shutdown_connections 2024-09-10T08:45:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.483+0000 7fe967200640 1 -- 172.21.15.89:0/3554988673 wait complete. 2024-09-10T08:45:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.484+0000 7fe967200640 1 Processor -- start 2024-09-10T08:45:14.485 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.484+0000 7fe967200640 1 -- start start 2024-09-10T08:45:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.484+0000 7fe967200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe968076720 0x7fe9681a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.484+0000 7fe967200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe968077040 0x7fe9681a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.484+0000 7fe967200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe968112870 0x7fe96819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.484+0000 7fe967200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe968118450 con 0x7fe968077040 2024-09-10T08:45:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.484+0000 7fe967200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe9681182d0 con 0x7fe968112870 2024-09-10T08:45:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.484+0000 7fe967200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe9681185d0 con 0x7fe968076720 2024-09-10T08:45:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.484+0000 7fe966800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe968112870 0x7fe96819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.485+0000 7fe95d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe968077040 0x7fe9681a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.485+0000 7fe965e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe968076720 0x7fe9681a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.485+0000 7fe966800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe968112870 0x7fe96819e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35904/0 (socket says 172.21.15.89:35904) 2024-09-10T08:45:14.486 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.485+0000 7fe966800640 1 -- 172.21.15.89:0/752048942 learned_addr learned my addr 172.21.15.89:0/752048942 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.485+0000 7fe966800640 1 -- 172.21.15.89:0/752048942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe968076720 msgr2=0x7fe9681a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.485+0000 7fe966800640 1 --2- 172.21.15.89:0/752048942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe968076720 0x7fe9681a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.485+0000 7fe966800640 1 -- 172.21.15.89:0/752048942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe968077040 msgr2=0x7fe9681a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:14.487 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.485+0000 7fe966800640 1 --2- 172.21.15.89:0/752048942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe968077040 0x7fe9681a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.485+0000 7fe966800640 1 -- 172.21.15.89:0/752048942 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96819efa0 con 0x7fe968112870 2024-09-10T08:45:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.485+0000 7fe966800640 1 --2- 172.21.15.89:0/752048942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe968112870 0x7fe96819e730 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe96000e940 tx=0x7fe96000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.485+0000 7fe95f400640 1 -- 172.21.15.89:0/752048942 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96000ccc0 con 0x7fe968112870 2024-09-10T08:45:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.486+0000 7fe967200640 1 -- 172.21.15.89:0/752048942 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe96819f290 con 0x7fe968112870 2024-09-10T08:45:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.486+0000 7fe967200640 1 -- 172.21.15.89:0/752048942 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9681ab5e0 con 0x7fe968112870 2024-09-10T08:45:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.486+0000 7fe95f400640 1 -- 172.21.15.89:0/752048942 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe960005500 con 0x7fe968112870 2024-09-10T08:45:14.488 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.486+0000 7fe95f400640 1 -- 172.21.15.89:0/752048942 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960010430 con 0x7fe968112870 2024-09-10T08:45:14.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.489+0000 7fe96fc5a740 1 Processor -- start 2024-09-10T08:45:14.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.489+0000 7fe95f400640 1 -- 172.21.15.89:0/752048942 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe9600046c0 con 0x7fe968112870 2024-09-10T08:45:14.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.489+0000 7fe96fc5a740 1 -- start start 2024-09-10T08:45:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.490+0000 7fe96fc5a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe5fbe40 0x5610fe5fc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.490+0000 7fe96fc5a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe5fc760 0x5610fe606ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.490+0000 7fe96fc5a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe607020 0x5610fe609410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.490+0000 7fe96fc5a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5610fe5623e0 con 0x5610fe5fbe40 2024-09-10T08:45:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.490+0000 7fe96fc5a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5610fe509d80 con 0x5610fe5fc760 2024-09-10T08:45:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.490+0000 7fe96fc5a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5610fe55e3a0 con 0x5610fe607020 2024-09-10T08:45:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.490+0000 7fe965e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe5fc760 0x5610fe606ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.490+0000 7fe965e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe5fc760 0x5610fe606ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35912/0 (socket says 172.21.15.89:35912) 2024-09-10T08:45:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.490+0000 7fe965e00640 1 -- 172.21.15.89:0/1119773403 learned_addr learned my addr 172.21.15.89:0/1119773403 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:14.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.490+0000 7fe95d400640 1 --2- 172.21.15.89:0/1119773403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe5fbe40 0x5610fe5fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.490+0000 7fe95d400640 1 --2- 172.21.15.89:0/1119773403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe607020 0x5610fe609410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.490+0000 7fe965e00640 1 -- 172.21.15.89:0/1119773403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe607020 msgr2=0x5610fe609410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.490+0000 7fe965e00640 1 --2- 172.21.15.89:0/1119773403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe607020 0x5610fe609410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.490+0000 7fe965e00640 1 -- 172.21.15.89:0/1119773403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe5fbe40 msgr2=0x5610fe5fc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.491+0000 7fe965e00640 1 --2- 172.21.15.89:0/1119773403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe5fbe40 0x5610fe5fc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.491+0000 7fe965e00640 1 -- 172.21.15.89:0/1119773403 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610fe528ae0 con 0x5610fe5fc760 2024-09-10T08:45:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.491+0000 7fe95d400640 1 --2- 172.21.15.89:0/1119773403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe5fbe40 0x5610fe5fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.491+0000 7fe965e00640 1 --2- 172.21.15.89:0/1119773403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe5fc760 0x5610fe606ae0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe9580103f0 tx=0x7fe9580108b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8573e8b8e39c108 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.491+0000 7fe95f400640 1 --2- 172.21.15.89:0/752048942 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe94c0a2b80 0x7fe94c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.491+0000 7fe95f400640 1 -- 172.21.15.89:0/752048942 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fe9600cf540 con 0x7fe968112870 2024-09-10T08:45:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.491+0000 7fe95d400640 1 --2- 172.21.15.89:0/752048942 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe94c0a2b80 0x7fe94c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.491+0000 7fe957400640 1 -- 172.21.15.89:0/1119773403 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958016070 con 0x5610fe5fc760 2024-09-10T08:45:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.491+0000 7fe957400640 1 -- 172.21.15.89:0/1119773403 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe958004d20 con 0x5610fe5fc760 2024-09-10T08:45:14.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.491+0000 7fe957400640 1 -- 172.21.15.89:0/1119773403 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9580054c0 con 0x5610fe5fc760 2024-09-10T08:45:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.492+0000 7fe96fc5a740 1 -- 172.21.15.89:0/1119773403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe5fc760 msgr2=0x5610fe606ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.492+0000 7fe95d400640 1 --2- 172.21.15.89:0/752048942 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe94c0a2b80 0x7fe94c0a5040 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe950000c00 tx=0x7fe95002a3f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.492+0000 7fe96fc5a740 1 --2- 172.21.15.89:0/1119773403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe5fc760 0x5610fe606ae0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe9580103f0 tx=0x7fe9580108b0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.492+0000 7fe96fc5a740 1 -- 172.21.15.89:0/1119773403 shutdown_connections 2024-09-10T08:45:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.492+0000 7fe96fc5a740 1 --2- 172.21.15.89:0/1119773403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe607020 0x5610fe609410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.492+0000 7fe96fc5a740 1 --2- 172.21.15.89:0/1119773403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe5fc760 0x5610fe606ae0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.492+0000 7fe96fc5a740 1 --2- 172.21.15.89:0/1119773403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe5fbe40 0x5610fe5fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.492+0000 7fe96fc5a740 1 -- 172.21.15.89:0/1119773403 >> 172.21.15.89:0/1119773403 conn(0x5610fe5f83d0 msgr2=0x5610fe5f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.492+0000 7fe96fc5a740 1 -- 172.21.15.89:0/1119773403 shutdown_connections 2024-09-10T08:45:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.492+0000 7fe96fc5a740 1 -- 172.21.15.89:0/1119773403 wait complete. 2024-09-10T08:45:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.493+0000 7fe96fc5a740 1 -- start start 2024-09-10T08:45:14.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.493+0000 7fe96fc5a740 1 Processor -- start 2024-09-10T08:45:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.493+0000 7fe96fc5a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe5fbe40 0x5610fe603ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.493+0000 7fe96fc5a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe5fc760 0x5610fe604400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.493+0000 7fe96fc5a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe607020 0x5610fe604940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.494+0000 7fe96fc5a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5610fe609bf0 con 0x5610fe5fc760 2024-09-10T08:45:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.494+0000 7fe96fc5a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5610fe51d0f0 con 0x5610fe607020 2024-09-10T08:45:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.494+0000 7fe96fc5a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5610fe609d70 con 0x5610fe5fbe40 2024-09-10T08:45:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.494+0000 7fe96fc5a740 10 client.0 _ll_register_callbacks cb 0x5610fe504bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.494+0000 7fe965e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe5fbe40 0x5610fe603ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.494+0000 7fe95d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe5fc760 0x5610fe604400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.494+0000 7fe965e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe5fbe40 0x5610fe603ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:41142/0 (socket says 172.21.15.89:41142) 2024-09-10T08:45:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.494+0000 7fe965e00640 1 -- 172.21.15.89:0/249669173 learned_addr learned my addr 172.21.15.89:0/249669173 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:14.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.494+0000 7fe966800640 1 --2- 172.21.15.89:0/249669173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe607020 0x5610fe604940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.494+0000 7fe965e00640 1 -- 172.21.15.89:0/249669173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe607020 msgr2=0x5610fe604940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.494+0000 7fe965e00640 1 --2- 172.21.15.89:0/249669173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe607020 0x5610fe604940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.494+0000 7fe965e00640 1 -- 172.21.15.89:0/249669173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe5fc760 msgr2=0x5610fe604400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.494+0000 7fe965e00640 1 --2- 172.21.15.89:0/249669173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe5fc760 0x5610fe604400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.494+0000 7fe965e00640 1 -- 172.21.15.89:0/249669173 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610fe2d33c0 con 0x5610fe5fbe40 2024-09-10T08:45:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.494+0000 7fe966800640 1 --2- 172.21.15.89:0/249669173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe607020 0x5610fe604940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.495+0000 7fe965e00640 1 --2- 172.21.15.89:0/249669173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe5fbe40 0x5610fe603ec0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe958005120 tx=0x7fe958003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18879d667cbf4df1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.495+0000 7fe94be00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:45:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.495+0000 7fe94be00640 1 -- 172.21.15.89:0/249669173 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958029060 con 0x5610fe5fbe40 2024-09-10T08:45:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.495+0000 7fe96fc5a740 1 -- 172.21.15.89:0/249669173 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe958004d20 con 0x5610fe5fbe40 2024-09-10T08:45:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.495+0000 7fe94be00640 1 -- 172.21.15.89:0/249669173 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fe9580055d0 con 0x5610fe5fbe40 2024-09-10T08:45:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.495+0000 7fe94be00640 1 -- 172.21.15.89:0/249669173 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe958030050 con 0x5610fe5fbe40 2024-09-10T08:45:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.495+0000 7fe96fc5a740 10 client.26404 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.495+0000 7fe96fc5a740 1 -- 172.21.15.89:0/249669173 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5610fe615ca0 con 0x5610fe5fbe40 2024-09-10T08:45:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.496+0000 7fe94be00640 1 -- 172.21.15.89:0/249669173 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe958004d20 con 0x5610fe5fbe40 2024-09-10T08:45:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.496+0000 7fe94be00640 1 -- 172.21.15.89:0/249669173 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fe95800f040 con 0x5610fe5fbe40 2024-09-10T08:45:14.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.496+0000 7fe96fc5a740 10 client.26404 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.496+0000 7fe96fc5a740 10 client.26404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:45:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.496+0000 7fe96fc5a740 20 client.26404 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.496+0000 7fe96fc5a740 1 --2- 172.21.15.89:0/249669173 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5610fe6170b0 0x5610fe6194a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.496+0000 7fe96fc5a740 4 client.26404 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:45:14.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.496+0000 7fe96fc5a740 1 -- 172.21.15.89:0/249669173 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610fe56f950 con 0x5610fe6170b0 2024-09-10T08:45:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.496+0000 7fe95d400640 1 --2- 172.21.15.89:0/249669173 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5610fe6170b0 0x5610fe6194a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.497+0000 7fe95d400640 1 --2- 172.21.15.89:0/249669173 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5610fe6170b0 0x5610fe6194a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e393d3046db9c6ba server_cookie=9066b3facbcbd738 in_seq=0 out_seq=0 2024-09-10T08:45:14.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.497+0000 7fe94be00640 10 client.26404 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.499+0000 7fe94be00640 1 -- 172.21.15.89:0/249669173 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610fe56f950 con 0x5610fe6170b0 2024-09-10T08:45:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.499+0000 7fe94be00640 10 client.26404 handle_command_reply: tid=0 2024-09-10T08:45:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.499+0000 7fe96fc5a740 1 client.26404 shutdown 2024-09-10T08:45:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.499+0000 7fe96fc5a740 1 -- 172.21.15.89:0/249669173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe5fbe40 msgr2=0x5610fe603ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.499+0000 7fe96fc5a740 1 --2- 172.21.15.89:0/249669173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe5fbe40 0x5610fe603ec0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe958005120 tx=0x7fe958003c10 comp rx=0 tx=0).stop 2024-09-10T08:45:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.499+0000 7fe96fc5a740 1 -- 172.21.15.89:0/249669173 shutdown_connections 2024-09-10T08:45:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.499+0000 7fe96fc5a740 1 --2- 172.21.15.89:0/249669173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe607020 0x5610fe604940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.499+0000 7fe96fc5a740 1 --2- 172.21.15.89:0/249669173 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5610fe6170b0 0x5610fe6194a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.499+0000 7fe96fc5a740 1 --2- 172.21.15.89:0/249669173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe5fc760 0x5610fe604400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.499+0000 7fe96fc5a740 1 --2- 172.21.15.89:0/249669173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe5fbe40 0x5610fe603ec0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.499+0000 7fe94be00640 0 client.26404 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.499+0000 7fe96fc5a740 1 -- 172.21.15.89:0/249669173 >> 172.21.15.89:0/249669173 conn(0x5610fe5f83d0 msgr2=0x5610fe5fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.499+0000 7fe96fc5a740 1 -- 172.21.15.89:0/249669173 shutdown_connections 2024-09-10T08:45:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.499+0000 7fe96fc5a740 1 -- 172.21.15.89:0/249669173 wait complete. 2024-09-10T08:45:14.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.500+0000 7fe96fc5a740 20 client.26404 trim_cache size 0 max 16384 2024-09-10T08:45:14.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.516+0000 7fe96fc5a740 1 Processor -- start 2024-09-10T08:45:14.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.516+0000 7fe96fc5a740 1 -- start start 2024-09-10T08:45:14.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.516+0000 7fe96fc5a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe5fbe40 0x5610fe5fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.516+0000 7fe96fc5a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe5fc760 0x5610fe604590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.516+0000 7fe96fc5a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe607020 0x5610fe604ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.516+0000 7fe96fc5a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe944012560 con 0x5610fe607020 2024-09-10T08:45:14.517 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.516+0000 7fe96fc5a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe94400fe30 con 0x5610fe5fbe40 2024-09-10T08:45:14.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.516+0000 7fe96fc5a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe94400dc80 con 0x5610fe5fc760 2024-09-10T08:45:14.522 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.516+0000 7fe965e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe5fbe40 0x5610fe5fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.516+0000 7fe965e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe5fbe40 0x5610fe5fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35932/0 (socket says 172.21.15.89:35932) 2024-09-10T08:45:14.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.516+0000 7fe965e00640 1 -- 172.21.15.89:0/849591455 learned_addr learned my addr 172.21.15.89:0/849591455 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:14.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.516+0000 7fe966800640 1 --2- 172.21.15.89:0/849591455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe607020 0x5610fe604ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.516+0000 7fe95d400640 1 --2- 172.21.15.89:0/849591455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe5fc760 0x5610fe604590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.517+0000 7fe965e00640 1 -- 172.21.15.89:0/849591455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe5fc760 msgr2=0x5610fe604590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:14.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.517+0000 7fe965e00640 1 --2- 172.21.15.89:0/849591455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe5fc760 0x5610fe604590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.517+0000 7fe965e00640 1 -- 172.21.15.89:0/849591455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe607020 msgr2=0x5610fe604ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:14.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.517+0000 7fe965e00640 1 --2- 172.21.15.89:0/849591455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe607020 0x5610fe604ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.517+0000 7fe965e00640 1 -- 172.21.15.89:0/849591455 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610fe605330 con 0x5610fe5fbe40 2024-09-10T08:45:14.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.517+0000 7fe965e00640 1 --2- 172.21.15.89:0/849591455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe5fbe40 0x5610fe5fe8f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe95800a010 tx=0x7fe958009d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfc5a92854b6153b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:14.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.517+0000 7fe956a00640 1 -- 172.21.15.89:0/849591455 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958039070 con 0x5610fe5fbe40 2024-09-10T08:45:14.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.517+0000 7fe956a00640 1 -- 172.21.15.89:0/849591455 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe958005df0 con 0x5610fe5fbe40 2024-09-10T08:45:14.523 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.517+0000 7fe956a00640 1 -- 172.21.15.89:0/849591455 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958016070 con 0x5610fe5fbe40 2024-09-10T08:45:14.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.518+0000 7fe96fc5a740 1 -- 172.21.15.89:0/849591455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe5fbe40 msgr2=0x5610fe5fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:14.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.518+0000 7fe96fc5a740 1 --2- 172.21.15.89:0/849591455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe5fbe40 0x5610fe5fe8f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe95800a010 tx=0x7fe958009d80 comp rx=0 tx=0).stop 2024-09-10T08:45:14.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.518+0000 7fe96fc5a740 1 -- 172.21.15.89:0/849591455 shutdown_connections 2024-09-10T08:45:14.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.518+0000 7fe96fc5a740 1 --2- 172.21.15.89:0/849591455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe607020 0x5610fe604ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.518+0000 7fe96fc5a740 1 --2- 172.21.15.89:0/849591455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe5fc760 0x5610fe604590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.518+0000 7fe96fc5a740 1 --2- 172.21.15.89:0/849591455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe5fbe40 0x5610fe5fe8f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.518+0000 7fe96fc5a740 1 -- 172.21.15.89:0/849591455 >> 172.21.15.89:0/849591455 conn(0x5610fe5f83d0 msgr2=0x5610fe603e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:14.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.518+0000 7fe96fc5a740 1 -- 172.21.15.89:0/849591455 shutdown_connections 2024-09-10T08:45:14.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.518+0000 7fe96fc5a740 1 -- 172.21.15.89:0/849591455 wait complete. 2024-09-10T08:45:14.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.519+0000 7fe96fc5a740 1 -- start start 2024-09-10T08:45:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.519+0000 7fe96fc5a740 1 Processor -- start 2024-09-10T08:45:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.520+0000 7fe96fc5a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe5fbe40 0x5610fe601050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.520+0000 7fe96fc5a740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe5fc760 0x5610fe601590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.520+0000 7fe96fc5a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe607020 0x5610fe601ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.520+0000 7fe96fc5a740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe94400e2f0 con 0x5610fe5fc760 2024-09-10T08:45:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.520+0000 7fe96fc5a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe94400f760 con 0x5610fe607020 2024-09-10T08:45:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.520+0000 7fe96fc5a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe94400e9c0 con 0x5610fe5fbe40 2024-09-10T08:45:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.520+0000 7fe96fc5a740 10 client.0 _ll_register_callbacks cb 0x7fe958009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.520+0000 7fe965e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe5fbe40 0x5610fe601050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.520+0000 7fe966800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe607020 0x5610fe601ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.520+0000 7fe95d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe5fc760 0x5610fe601590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.520+0000 7fe966800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe607020 0x5610fe601ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35946/0 (socket says 172.21.15.89:35946) 2024-09-10T08:45:14.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.520+0000 7fe965e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe5fbe40 0x5610fe601050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:41164/0 (socket says 172.21.15.89:41164) 2024-09-10T08:45:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.520+0000 7fe966800640 1 -- 172.21.15.89:0/3292884540 learned_addr learned my addr 172.21.15.89:0/3292884540 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.521+0000 7fe966800640 1 -- 172.21.15.89:0/3292884540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe5fbe40 msgr2=0x5610fe601050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.521+0000 7fe966800640 1 --2- 172.21.15.89:0/3292884540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe5fbe40 0x5610fe601050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.521+0000 7fe966800640 1 -- 172.21.15.89:0/3292884540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe5fc760 msgr2=0x5610fe601590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.521+0000 7fe966800640 1 --2- 172.21.15.89:0/3292884540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe5fc760 0x5610fe601590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.521+0000 7fe966800640 1 -- 172.21.15.89:0/3292884540 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610fe680c30 con 0x5610fe607020 2024-09-10T08:45:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.521+0000 7fe966800640 1 --2- 172.21.15.89:0/3292884540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe607020 0x5610fe601ad0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe96009b960 tx=0x7fe960005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dcca71716ab1c94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.521+0000 7fe94be00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:45:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.521+0000 7fe94be00640 1 -- 172.21.15.89:0/3292884540 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960005190 con 0x5610fe607020 2024-09-10T08:45:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.521+0000 7fe94be00640 1 -- 172.21.15.89:0/3292884540 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fe9600a41b0 con 0x5610fe607020 2024-09-10T08:45:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.521+0000 7fe96fc5a740 1 -- 172.21.15.89:0/3292884540 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe958005df0 con 0x5610fe607020 2024-09-10T08:45:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.521+0000 7fe94be00640 1 -- 172.21.15.89:0/3292884540 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe96009b9d0 con 0x5610fe607020 2024-09-10T08:45:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.522+0000 7fe96fc5a740 10 client.26483 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:14.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.522+0000 7fe96fc5a740 1 -- 172.21.15.89:0/3292884540 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5610fe681f50 con 0x5610fe607020 2024-09-10T08:45:14.527 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.522+0000 7fe94be00640 1 -- 172.21.15.89:0/3292884540 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe958005df0 con 0x5610fe607020 2024-09-10T08:45:14.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.522+0000 7fe94be00640 1 -- 172.21.15.89:0/3292884540 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fe9600b4bd0 con 0x5610fe607020 2024-09-10T08:45:14.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.522+0000 7fe96fc5a740 10 client.26483 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:14.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.522+0000 7fe96fc5a740 10 client.26483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:45:14.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.522+0000 7fe96fc5a740 20 client.26483 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:14.528 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.522+0000 7fe96fc5a740 1 --2- 172.21.15.89:0/3292884540 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5610fe682670 0x5610fe684a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.522+0000 7fe96fc5a740 4 client.26483 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:45:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.522+0000 7fe96fc5a740 1 -- 172.21.15.89:0/3292884540 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610fe569e60 con 0x5610fe682670 2024-09-10T08:45:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.523+0000 7fe965e00640 1 --2- 172.21.15.89:0/3292884540 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5610fe682670 0x5610fe684a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.523+0000 7fe965e00640 1 --2- 172.21.15.89:0/3292884540 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5610fe682670 0x5610fe684a90 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=868d23481ddc605f server_cookie=f0074d871ecc465e in_seq=0 out_seq=0 2024-09-10T08:45:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.523+0000 7fe94be00640 10 client.26483 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.524+0000 7fe94be00640 1 -- 172.21.15.89:0/3292884540 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610fe569e60 con 0x5610fe682670 2024-09-10T08:45:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.524+0000 7fe94be00640 10 client.26483 handle_command_reply: tid=0 2024-09-10T08:45:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.524+0000 7fe96fc5a740 1 client.26483 shutdown 2024-09-10T08:45:14.529 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.524+0000 7fe96fc5a740 1 -- 172.21.15.89:0/3292884540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe607020 msgr2=0x5610fe601ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:14.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.525+0000 7fe96fc5a740 1 --2- 172.21.15.89:0/3292884540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe607020 0x5610fe601ad0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe96009b960 tx=0x7fe960005770 comp rx=0 tx=0).stop 2024-09-10T08:45:14.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.525+0000 7fe96fc5a740 1 -- 172.21.15.89:0/3292884540 shutdown_connections 2024-09-10T08:45:14.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.525+0000 7fe96fc5a740 1 --2- 172.21.15.89:0/3292884540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610fe607020 0x5610fe601ad0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.525+0000 7fe96fc5a740 1 --2- 172.21.15.89:0/3292884540 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5610fe682670 0x5610fe684a90 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.525+0000 7fe96fc5a740 1 --2- 172.21.15.89:0/3292884540 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5610fe5fc760 0x5610fe601590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.525+0000 7fe96fc5a740 1 --2- 172.21.15.89:0/3292884540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610fe5fbe40 0x5610fe601050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.525+0000 7fe94be00640 0 client.26483 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:14.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.525+0000 7fe96fc5a740 1 -- 172.21.15.89:0/3292884540 >> 172.21.15.89:0/3292884540 conn(0x5610fe5f83d0 msgr2=0x5610fe5f7490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:14.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.525+0000 7fe96fc5a740 1 -- 172.21.15.89:0/3292884540 shutdown_connections 2024-09-10T08:45:14.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.525+0000 7fe96fc5a740 1 -- 172.21.15.89:0/3292884540 wait complete. 2024-09-10T08:45:14.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.525+0000 7fe96fc5a740 20 client.26483 trim_cache size 0 max 16384 2024-09-10T08:45:14.531 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:45:14.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.526+0000 7fe967200640 1 -- 172.21.15.89:0/752048942 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe94c0a2b80 msgr2=0x7fe94c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:14.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.526+0000 7fe967200640 1 --2- 172.21.15.89:0/752048942 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe94c0a2b80 0x7fe94c0a5040 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe950000c00 tx=0x7fe95002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.526+0000 7fe967200640 1 -- 172.21.15.89:0/752048942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe968112870 msgr2=0x7fe96819e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:14.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.526+0000 7fe967200640 1 --2- 172.21.15.89:0/752048942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe968112870 0x7fe96819e730 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe96000e940 tx=0x7fe96000ee00 comp rx=0 tx=0).stop 2024-09-10T08:45:14.531 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.526+0000 7fe967200640 1 -- 172.21.15.89:0/752048942 shutdown_connections 2024-09-10T08:45:14.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.526+0000 7fe967200640 1 --2- 172.21.15.89:0/752048942 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fe94c0a2b80 0x7fe94c0a5040 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.526+0000 7fe967200640 1 --2- 172.21.15.89:0/752048942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe968112870 0x7fe96819e730 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.526+0000 7fe967200640 1 --2- 172.21.15.89:0/752048942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe968077040 0x7fe9681a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.526+0000 7fe967200640 1 --2- 172.21.15.89:0/752048942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe968076720 0x7fe9681a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.526+0000 7fe967200640 1 -- 172.21.15.89:0/752048942 >> 172.21.15.89:0/752048942 conn(0x7fe9680ff8d0 msgr2=0x7fe968101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:14.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.526+0000 7fe967200640 1 -- 172.21.15.89:0/752048942 shutdown_connections 2024-09-10T08:45:14.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:14.526+0000 7fe967200640 1 -- 172.21.15.89:0/752048942 wait complete. 2024-09-10T08:45:15.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:14 smithi149 ceph-mon[36167]: pgmap v552: 129 pgs: 129 active+clean; 301 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.5 MiB/s wr, 695 op/s 2024-09-10T08:45:15.251 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:14 smithi089 ceph-mon[30757]: pgmap v552: 129 pgs: 129 active+clean; 301 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.5 MiB/s wr, 695 op/s 2024-09-10T08:45:15.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:14 smithi111 ceph-mon[35220]: pgmap v552: 129 pgs: 129 active+clean; 301 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.5 MiB/s wr, 695 op/s 2024-09-10T08:45:16.291 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:45:16.291 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:45:16.690 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:17.163 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:16 smithi089 ceph-mon[30757]: pgmap v553: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 650 op/s 2024-09-10T08:45:17.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:16 smithi149 ceph-mon[36167]: pgmap v553: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 650 op/s 2024-09-10T08:45:17.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:16 smithi111 ceph-mon[35220]: pgmap v553: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 650 op/s 2024-09-10T08:45:17.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.304+0000 7f4e83000640 1 -- 172.21.15.89:0/2066312873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e7c10e260 msgr2=0x7f4e7c110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:17.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.304+0000 7f4e83000640 1 --2- 172.21.15.89:0/2066312873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e7c10e260 0x7f4e7c110650 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4e74009900 tx=0x7f4e7402a480 comp rx=0 tx=0).stop 2024-09-10T08:45:17.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.304+0000 7f4e83000640 1 -- 172.21.15.89:0/2066312873 shutdown_connections 2024-09-10T08:45:17.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.304+0000 7f4e83000640 1 --2- 172.21.15.89:0/2066312873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e7c10e260 0x7f4e7c110650 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.304+0000 7f4e83000640 1 --2- 172.21.15.89:0/2066312873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e7c074ee0 0x7f4e7c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.304+0000 7f4e83000640 1 --2- 172.21.15.89:0/2066312873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e7c0745c0 0x7f4e7c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.304+0000 7f4e83000640 1 -- 172.21.15.89:0/2066312873 >> 172.21.15.89:0/2066312873 conn(0x7f4e7c0fd740 msgr2=0x7f4e7c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:17.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.304+0000 7f4e83000640 1 -- 172.21.15.89:0/2066312873 shutdown_connections 2024-09-10T08:45:17.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.304+0000 7f4e83000640 1 -- 172.21.15.89:0/2066312873 wait complete. 2024-09-10T08:45:17.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.305+0000 7f4e83000640 1 Processor -- start 2024-09-10T08:45:17.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.305+0000 7f4e83000640 1 -- start start 2024-09-10T08:45:17.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.306+0000 7f4e83000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e7c0745c0 0x7f4e7c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.306+0000 7f4e83000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e7c074ee0 0x7f4e7c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.306+0000 7f4e83000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e7c10e260 0x7f4e7c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.306+0000 7f4e83000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c114000 con 0x7f4e7c074ee0 2024-09-10T08:45:17.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.306+0000 7f4e83000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c113e80 con 0x7f4e7c0745c0 2024-09-10T08:45:17.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.306+0000 7f4e83000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c114180 con 0x7f4e7c10e260 2024-09-10T08:45:17.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.306+0000 7f4e81c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e7c0745c0 0x7f4e7c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.306+0000 7f4e81200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e7c074ee0 0x7f4e7c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.306+0000 7f4e81c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e7c0745c0 0x7f4e7c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35974/0 (socket says 172.21.15.89:35974) 2024-09-10T08:45:17.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.306+0000 7f4e81c00640 1 -- 172.21.15.89:0/1819458164 learned_addr learned my addr 172.21.15.89:0/1819458164 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:17.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.306+0000 7f4e82600640 1 --2- 172.21.15.89:0/1819458164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e7c10e260 0x7f4e7c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.306+0000 7f4e81c00640 1 -- 172.21.15.89:0/1819458164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e7c10e260 msgr2=0x7f4e7c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:17.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.306+0000 7f4e81c00640 1 --2- 172.21.15.89:0/1819458164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e7c10e260 0x7f4e7c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.306+0000 7f4e81c00640 1 -- 172.21.15.89:0/1819458164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e7c074ee0 msgr2=0x7f4e7c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:17.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.306+0000 7f4e81c00640 1 --2- 172.21.15.89:0/1819458164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e7c074ee0 0x7f4e7c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.306+0000 7f4e81c00640 1 -- 172.21.15.89:0/1819458164 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c19f440 con 0x7f4e7c0745c0 2024-09-10T08:45:17.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.306+0000 7f4e81200640 1 --2- 172.21.15.89:0/1819458164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e7c074ee0 0x7f4e7c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:17.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.307+0000 7f4e81c00640 1 --2- 172.21.15.89:0/1819458164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e7c0745c0 0x7f4e7c19a490 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000eb90 tx=0x7f4e7000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:17.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.307+0000 7f4e7a000640 1 -- 172.21.15.89:0/1819458164 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70004d20 con 0x7f4e7c0745c0 2024-09-10T08:45:17.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.307+0000 7f4e7a000640 1 -- 172.21.15.89:0/1819458164 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e7000ee90 con 0x7f4e7c0745c0 2024-09-10T08:45:17.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.307+0000 7f4e7a000640 1 -- 172.21.15.89:0/1819458164 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70010830 con 0x7f4e7c0745c0 2024-09-10T08:45:17.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.307+0000 7f4e83000640 1 -- 172.21.15.89:0/1819458164 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7c19f730 con 0x7f4e7c0745c0 2024-09-10T08:45:17.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.307+0000 7f4e83000640 1 -- 172.21.15.89:0/1819458164 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7c1a7010 con 0x7f4e7c0745c0 2024-09-10T08:45:17.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.311+0000 7f4e7a000640 1 -- 172.21.15.89:0/1819458164 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4e70005020 con 0x7f4e7c0745c0 2024-09-10T08:45:17.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.311+0000 7f4e8721c740 1 Processor -- start 2024-09-10T08:45:17.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.311+0000 7f4e8721c740 1 -- start start 2024-09-10T08:45:17.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.311+0000 7f4e8721c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9cbd0 0x55ec1cd9cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.312+0000 7f4e8721c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cd9d4f0 0x55ec1cdb0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.312+0000 7f4e8721c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec1cdb06a0 0x55ec1cdb2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.312+0000 7f4e8721c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ec1cd03290 con 0x55ec1cdb06a0 2024-09-10T08:45:17.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.312+0000 7f4e8721c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ec1ccaab40 con 0x55ec1cd9cbd0 2024-09-10T08:45:17.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.312+0000 7f4e8721c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ec1ccff250 con 0x55ec1cd9d4f0 2024-09-10T08:45:17.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.312+0000 7f4e81c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cd9d4f0 0x55ec1cdb0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.312+0000 7f4e82600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9cbd0 0x55ec1cd9cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.312+0000 7f4e82600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9cbd0 0x55ec1cd9cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35982/0 (socket says 172.21.15.89:35982) 2024-09-10T08:45:17.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.312+0000 7f4e82600640 1 -- 172.21.15.89:0/1814974530 learned_addr learned my addr 172.21.15.89:0/1814974530 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:17.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.312+0000 7f4e81200640 1 --2- 172.21.15.89:0/1814974530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec1cdb06a0 0x55ec1cdb2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.312+0000 7f4e82600640 1 -- 172.21.15.89:0/1814974530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cd9d4f0 msgr2=0x55ec1cdb0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:17.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.312+0000 7f4e82600640 1 --2- 172.21.15.89:0/1814974530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cd9d4f0 0x55ec1cdb0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.312+0000 7f4e82600640 1 -- 172.21.15.89:0/1814974530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec1cdb06a0 msgr2=0x55ec1cdb2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:17.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.312+0000 7f4e82600640 1 --2- 172.21.15.89:0/1814974530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec1cdb06a0 0x55ec1cdb2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.312+0000 7f4e82600640 1 -- 172.21.15.89:0/1814974530 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec1ca746a0 con 0x55ec1cd9cbd0 2024-09-10T08:45:17.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.313+0000 7f4e7a000640 1 --2- 172.21.15.89:0/1819458164 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f4e640a2b80 0x7f4e640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.313+0000 7f4e82600640 1 --2- 172.21.15.89:0/1814974530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9cbd0 0x55ec1cd9cfb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4e74006010 tx=0x7f4e74005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10c1480f3fbf15d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:17.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.313+0000 7f4e7a000640 1 -- 172.21.15.89:0/1819458164 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f4e700cfab0 con 0x7f4e7c0745c0 2024-09-10T08:45:17.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.313+0000 7f4e6ea00640 1 -- 172.21.15.89:0/1814974530 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e74040070 con 0x55ec1cd9cbd0 2024-09-10T08:45:17.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.313+0000 7f4e81c00640 1 --2- 172.21.15.89:0/1819458164 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f4e640a2b80 0x7f4e640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.313+0000 7f4e6ea00640 1 -- 172.21.15.89:0/1814974530 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e74004300 con 0x55ec1cd9cbd0 2024-09-10T08:45:17.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.313+0000 7f4e6ea00640 1 -- 172.21.15.89:0/1814974530 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e74035040 con 0x55ec1cd9cbd0 2024-09-10T08:45:17.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.314+0000 7f4e8721c740 1 -- 172.21.15.89:0/1814974530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9cbd0 msgr2=0x55ec1cd9cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:17.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.314+0000 7f4e81c00640 1 --2- 172.21.15.89:0/1819458164 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f4e640a2b80 0x7f4e640a5040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4e700988b0 tx=0x7f4e700d34f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:17.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.314+0000 7f4e8721c740 1 --2- 172.21.15.89:0/1814974530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9cbd0 0x55ec1cd9cfb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4e74006010 tx=0x7f4e74005be0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.314+0000 7f4e8721c740 1 -- 172.21.15.89:0/1814974530 shutdown_connections 2024-09-10T08:45:17.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.314+0000 7f4e8721c740 1 --2- 172.21.15.89:0/1814974530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec1cdb06a0 0x55ec1cdb2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.314+0000 7f4e8721c740 1 --2- 172.21.15.89:0/1814974530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cd9d4f0 0x55ec1cdb0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.314+0000 7f4e8721c740 1 --2- 172.21.15.89:0/1814974530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9cbd0 0x55ec1cd9cfb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.314+0000 7f4e8721c740 1 -- 172.21.15.89:0/1814974530 >> 172.21.15.89:0/1814974530 conn(0x55ec1cd99240 msgr2=0x55ec1cd99620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:17.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.314+0000 7f4e8721c740 1 -- 172.21.15.89:0/1814974530 shutdown_connections 2024-09-10T08:45:17.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.314+0000 7f4e8721c740 1 -- 172.21.15.89:0/1814974530 wait complete. 2024-09-10T08:45:17.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.315+0000 7f4e8721c740 1 -- start start 2024-09-10T08:45:17.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.315+0000 7f4e8721c740 1 Processor -- start 2024-09-10T08:45:17.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.316+0000 7f4e8721c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec1cd9cbd0 0x55ec1cdb00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.316+0000 7f4e8721c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9d4f0 0x55ec1cdad360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.316+0000 7f4e8721c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cdb06a0 0x55ec1cdad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.316+0000 7f4e8721c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ec1cdb3250 con 0x55ec1cd9cbd0 2024-09-10T08:45:17.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.316+0000 7f4e8721c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ec1ccbdfa0 con 0x55ec1cd9d4f0 2024-09-10T08:45:17.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.316+0000 7f4e8721c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ec1cdb33d0 con 0x55ec1cdb06a0 2024-09-10T08:45:17.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.316+0000 7f4e8721c740 10 client.0 _ll_register_callbacks cb 0x55ec1ca11ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:17.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.316+0000 7f4e81200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9d4f0 0x55ec1cdad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.316+0000 7f4e82600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec1cd9cbd0 0x55ec1cdb00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.316+0000 7f4e81200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9d4f0 0x55ec1cdad360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35994/0 (socket says 172.21.15.89:35994) 2024-09-10T08:45:17.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.316+0000 7f4e81200640 1 -- 172.21.15.89:0/2539075503 learned_addr learned my addr 172.21.15.89:0/2539075503 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:17.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.317+0000 7f4e82600640 1 --2- 172.21.15.89:0/2539075503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cdb06a0 0x55ec1cdad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.317+0000 7f4e81200640 1 -- 172.21.15.89:0/2539075503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cdb06a0 msgr2=0x55ec1cdad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:17.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.317+0000 7f4e81200640 1 --2- 172.21.15.89:0/2539075503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cdb06a0 0x55ec1cdad970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.317+0000 7f4e81200640 1 -- 172.21.15.89:0/2539075503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec1cd9cbd0 msgr2=0x55ec1cdb00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:17.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.317+0000 7f4e81200640 1 --2- 172.21.15.89:0/2539075503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec1cd9cbd0 0x55ec1cdb00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.317+0000 7f4e81200640 1 -- 172.21.15.89:0/2539075503 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec1cdae1a0 con 0x55ec1cd9d4f0 2024-09-10T08:45:17.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.317+0000 7f4e81200640 1 --2- 172.21.15.89:0/2539075503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9d4f0 0x55ec1cdad360 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4e68012d20 tx=0x7f4e6800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d496c278993e6cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:17.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.317+0000 7f4e63400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:45:17.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.318+0000 7f4e63400640 1 -- 172.21.15.89:0/2539075503 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68012110 con 0x55ec1cd9d4f0 2024-09-10T08:45:17.320 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.318+0000 7f4e63400640 1 -- 172.21.15.89:0/2539075503 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f4e68004050 con 0x55ec1cd9d4f0 2024-09-10T08:45:17.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.318+0000 7f4e8721c740 1 -- 172.21.15.89:0/2539075503 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e74004300 con 0x55ec1cd9d4f0 2024-09-10T08:45:17.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.318+0000 7f4e63400640 1 -- 172.21.15.89:0/2539075503 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4e68025050 con 0x55ec1cd9d4f0 2024-09-10T08:45:17.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.318+0000 7f4e8721c740 10 client.26495 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:17.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.318+0000 7f4e8721c740 1 -- 172.21.15.89:0/2539075503 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ec1cdbf2d0 con 0x55ec1cd9d4f0 2024-09-10T08:45:17.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.318+0000 7f4e63400640 1 -- 172.21.15.89:0/2539075503 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e74004300 con 0x55ec1cd9d4f0 2024-09-10T08:45:17.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.319+0000 7f4e63400640 1 -- 172.21.15.89:0/2539075503 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f4e68010430 con 0x55ec1cd9d4f0 2024-09-10T08:45:17.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.319+0000 7f4e8721c740 10 client.26495 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:17.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.319+0000 7f4e8721c740 10 client.26495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:45:17.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.319+0000 7f4e8721c740 20 client.26495 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:17.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.319+0000 7f4e8721c740 1 --2- 172.21.15.89:0/2539075503 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ec1cdc0790 0x55ec1cdc2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.321 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.319+0000 7f4e8721c740 4 client.26495 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:45:17.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.319+0000 7f4e8721c740 1 -- 172.21.15.89:0/2539075503 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec1cd10800 con 0x55ec1cdc0790 2024-09-10T08:45:17.322 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.319+0000 7f4e82600640 1 --2- 172.21.15.89:0/2539075503 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ec1cdc0790 0x55ec1cdc2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.320+0000 7f4e82600640 1 --2- 172.21.15.89:0/2539075503 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ec1cdc0790 0x55ec1cdc2c10 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6059d68e52c45a1f server_cookie=73188ad15423135d in_seq=0 out_seq=0 2024-09-10T08:45:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.320+0000 7f4e63400640 10 client.26495 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.321+0000 7f4e63400640 1 -- 172.21.15.89:0/2539075503 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec1cd10800 con 0x55ec1cdc0790 2024-09-10T08:45:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.321+0000 7f4e63400640 10 client.26495 handle_command_reply: tid=0 2024-09-10T08:45:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.321+0000 7f4e8721c740 1 client.26495 shutdown 2024-09-10T08:45:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.322+0000 7f4e8721c740 1 -- 172.21.15.89:0/2539075503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9d4f0 msgr2=0x55ec1cdad360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:17.323 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.322+0000 7f4e8721c740 1 --2- 172.21.15.89:0/2539075503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9d4f0 0x55ec1cdad360 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4e68012d20 tx=0x7f4e6800f790 comp rx=0 tx=0).stop 2024-09-10T08:45:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.322+0000 7f4e8721c740 1 -- 172.21.15.89:0/2539075503 shutdown_connections 2024-09-10T08:45:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.322+0000 7f4e8721c740 1 --2- 172.21.15.89:0/2539075503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cdb06a0 0x55ec1cdad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.322+0000 7f4e8721c740 1 --2- 172.21.15.89:0/2539075503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9d4f0 0x55ec1cdad360 secure :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4e68012d20 tx=0x7f4e6800f790 comp rx=0 tx=0).stop 2024-09-10T08:45:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.322+0000 7f4e8721c740 1 --2- 172.21.15.89:0/2539075503 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ec1cdc0790 0x55ec1cdc2c10 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.322+0000 7f4e8721c740 1 --2- 172.21.15.89:0/2539075503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec1cd9cbd0 0x55ec1cdb00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.322+0000 7f4e8721c740 1 -- 172.21.15.89:0/2539075503 >> 172.21.15.89:0/2539075503 conn(0x55ec1cd99240 msgr2=0x55ec1cd9ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.322+0000 7f4e63400640 0 client.26495 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.322+0000 7f4e8721c740 1 -- 172.21.15.89:0/2539075503 shutdown_connections 2024-09-10T08:45:17.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.322+0000 7f4e8721c740 1 -- 172.21.15.89:0/2539075503 wait complete. 2024-09-10T08:45:17.325 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.323+0000 7f4e8721c740 20 client.26495 trim_cache size 0 max 16384 2024-09-10T08:45:17.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.345+0000 7f4e8721c740 1 Processor -- start 2024-09-10T08:45:17.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.345+0000 7f4e8721c740 1 -- start start 2024-09-10T08:45:17.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.345+0000 7f4e8721c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9cbd0 0x55ec1cdaf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.345+0000 7f4e8721c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cdb06a0 0x55ec1cdc8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.345+0000 7f4e8721c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec1cdaf810 0x55ec1cdc8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.345+0000 7f4e8721c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1af2b0 con 0x55ec1cdaf810 2024-09-10T08:45:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.345+0000 7f4e8721c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1acb80 con 0x55ec1cd9cbd0 2024-09-10T08:45:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.345+0000 7f4e8721c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1aa9d0 con 0x55ec1cdb06a0 2024-09-10T08:45:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.346+0000 7f4e82600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9cbd0 0x55ec1cdaf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.346+0000 7f4e82600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9cbd0 0x55ec1cdaf2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:36008/0 (socket says 172.21.15.89:36008) 2024-09-10T08:45:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.346+0000 7f4e82600640 1 -- 172.21.15.89:0/22807164 learned_addr learned my addr 172.21.15.89:0/22807164 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.346+0000 7f4e81200640 1 --2- 172.21.15.89:0/22807164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cdb06a0 0x55ec1cdc8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.346+0000 7f4e81200640 1 -- 172.21.15.89:0/22807164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9cbd0 msgr2=0x55ec1cdaf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.346+0000 7f4e81200640 1 --2- 172.21.15.89:0/22807164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9cbd0 0x55ec1cdaf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.346+0000 7f4e81200640 1 -- 172.21.15.89:0/22807164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec1cdaf810 msgr2=0x55ec1cdc8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.346+0000 7f4e81200640 1 --2- 172.21.15.89:0/22807164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec1cdaf810 0x55ec1cdc8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.346+0000 7f4e81200640 1 -- 172.21.15.89:0/22807164 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec1cd9dd70 con 0x55ec1cdb06a0 2024-09-10T08:45:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.346+0000 7f4e82600640 1 --2- 172.21.15.89:0/22807164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9cbd0 0x55ec1cdaf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:17.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.347+0000 7f4e81200640 1 --2- 172.21.15.89:0/22807164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cdb06a0 0x55ec1cdc8520 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55ec1cdae450 tx=0x7f4e6800fc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46eb9b8710b2595e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:17.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.347+0000 7f4e6e000640 1 -- 172.21.15.89:0/22807164 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68012110 con 0x55ec1cdb06a0 2024-09-10T08:45:17.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.347+0000 7f4e6e000640 1 -- 172.21.15.89:0/22807164 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e680055b0 con 0x55ec1cdb06a0 2024-09-10T08:45:17.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.347+0000 7f4e6e000640 1 -- 172.21.15.89:0/22807164 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e680035f0 con 0x55ec1cdb06a0 2024-09-10T08:45:17.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.348+0000 7f4e8721c740 1 -- 172.21.15.89:0/22807164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cdb06a0 msgr2=0x55ec1cdc8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:17.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.348+0000 7f4e8721c740 1 --2- 172.21.15.89:0/22807164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cdb06a0 0x55ec1cdc8520 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55ec1cdae450 tx=0x7f4e6800fc90 comp rx=0 tx=0).stop 2024-09-10T08:45:17.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.349+0000 7f4e8721c740 1 -- 172.21.15.89:0/22807164 shutdown_connections 2024-09-10T08:45:17.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.349+0000 7f4e8721c740 1 --2- 172.21.15.89:0/22807164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec1cdaf810 0x55ec1cdc8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.349+0000 7f4e8721c740 1 --2- 172.21.15.89:0/22807164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cdb06a0 0x55ec1cdc8520 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.349+0000 7f4e8721c740 1 --2- 172.21.15.89:0/22807164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cd9cbd0 0x55ec1cdaf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.349+0000 7f4e8721c740 1 -- 172.21.15.89:0/22807164 >> 172.21.15.89:0/22807164 conn(0x55ec1cd99240 msgr2=0x55ec1cdaebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:17.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.349+0000 7f4e8721c740 1 -- 172.21.15.89:0/22807164 shutdown_connections 2024-09-10T08:45:17.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.349+0000 7f4e8721c740 1 -- 172.21.15.89:0/22807164 wait complete. 2024-09-10T08:45:17.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.350+0000 7f4e8721c740 1 -- start start 2024-09-10T08:45:17.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.350+0000 7f4e8721c740 1 Processor -- start 2024-09-10T08:45:17.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.351+0000 7f4e8721c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cd9cbd0 0x55ec1cdb55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.351+0000 7f4e8721c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec1cdaf810 0x55ec1cdb5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.351+0000 7f4e8721c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cdb06a0 0x55ec1cdb6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.351+0000 7f4e8721c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1ab040 con 0x55ec1cdaf810 2024-09-10T08:45:17.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.351+0000 7f4e8721c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1ac4b0 con 0x55ec1cdb06a0 2024-09-10T08:45:17.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.351+0000 7f4e8721c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1ab710 con 0x55ec1cd9cbd0 2024-09-10T08:45:17.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.351+0000 7f4e8721c740 10 client.0 _ll_register_callbacks cb 0x7f4e68009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:17.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.351+0000 7f4e82600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cdb06a0 0x55ec1cdb6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.351+0000 7f4e82600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cdb06a0 0x55ec1cdb6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:36010/0 (socket says 172.21.15.89:36010) 2024-09-10T08:45:17.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.351+0000 7f4e82600640 1 -- 172.21.15.89:0/201112043 learned_addr learned my addr 172.21.15.89:0/201112043 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:17.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.351+0000 7f4e81200640 1 --2- 172.21.15.89:0/201112043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec1cdaf810 0x55ec1cdb5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.351+0000 7f4e82600640 1 --2- 172.21.15.89:0/201112043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cd9cbd0 0x55ec1cdb55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.352+0000 7f4e82600640 1 -- 172.21.15.89:0/201112043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cd9cbd0 msgr2=0x55ec1cdb55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:17.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.352+0000 7f4e82600640 1 --2- 172.21.15.89:0/201112043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cd9cbd0 0x55ec1cdb55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.352+0000 7f4e82600640 1 -- 172.21.15.89:0/201112043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec1cdaf810 msgr2=0x55ec1cdb5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:17.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.352+0000 7f4e82600640 1 --2- 172.21.15.89:0/201112043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec1cdaf810 0x55ec1cdb5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.352+0000 7f4e82600640 1 -- 172.21.15.89:0/201112043 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec1ce2b6a0 con 0x55ec1cdb06a0 2024-09-10T08:45:17.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.352+0000 7f4e82600640 1 --2- 172.21.15.89:0/201112043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cdb06a0 0x55ec1cdb6040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4e74009a20 tx=0x7f4e7402a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b427c910ef6697e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:17.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.352+0000 7f4e63400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:45:17.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.352+0000 7f4e63400640 1 -- 172.21.15.89:0/201112043 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e74035040 con 0x55ec1cdb06a0 2024-09-10T08:45:17.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.353+0000 7f4e63400640 1 -- 172.21.15.89:0/201112043 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f4e74003a00 con 0x55ec1cdb06a0 2024-09-10T08:45:17.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.353+0000 7f4e8721c740 1 -- 172.21.15.89:0/201112043 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e680055b0 con 0x55ec1cdb06a0 2024-09-10T08:45:17.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.353+0000 7f4e63400640 1 -- 172.21.15.89:0/201112043 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4e74040040 con 0x55ec1cdb06a0 2024-09-10T08:45:17.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.353+0000 7f4e8721c740 10 client.26501 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:17.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.353+0000 7f4e8721c740 1 -- 172.21.15.89:0/201112043 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ec1ce2c9c0 con 0x55ec1cdb06a0 2024-09-10T08:45:17.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.354+0000 7f4e63400640 1 -- 172.21.15.89:0/201112043 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e680055b0 con 0x55ec1cdb06a0 2024-09-10T08:45:17.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.354+0000 7f4e63400640 1 -- 172.21.15.89:0/201112043 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f4e740503c0 con 0x55ec1cdb06a0 2024-09-10T08:45:17.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.354+0000 7f4e8721c740 10 client.26501 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:17.356 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.354+0000 7f4e8721c740 10 client.26501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:45:17.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.354+0000 7f4e8721c740 20 client.26501 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:17.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.354+0000 7f4e8721c740 1 --2- 172.21.15.89:0/201112043 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ec1ce2ccb0 0x55ec1ce2f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.354+0000 7f4e8721c740 4 client.26501 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:45:17.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.354+0000 7f4e8721c740 1 -- 172.21.15.89:0/201112043 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ec1cd0ad10 con 0x55ec1ce2ccb0 2024-09-10T08:45:17.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.354+0000 7f4e81200640 1 --2- 172.21.15.89:0/201112043 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ec1ce2ccb0 0x55ec1ce2f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.355+0000 7f4e63400640 10 client.26501 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:17.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.355+0000 7f4e81200640 1 --2- 172.21.15.89:0/201112043 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ec1ce2ccb0 0x55ec1ce2f0a0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dee567a5d2671bc server_cookie=2197ae55540e8953 in_seq=0 out_seq=0 2024-09-10T08:45:17.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.357+0000 7f4e63400640 1 -- 172.21.15.89:0/201112043 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ec1cd0ad10 con 0x55ec1ce2ccb0 2024-09-10T08:45:17.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.357+0000 7f4e63400640 10 client.26501 handle_command_reply: tid=0 2024-09-10T08:45:17.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.357+0000 7f4e8721c740 1 client.26501 shutdown 2024-09-10T08:45:17.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.358+0000 7f4e8721c740 1 -- 172.21.15.89:0/201112043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cdb06a0 msgr2=0x55ec1cdb6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:17.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.358+0000 7f4e8721c740 1 --2- 172.21.15.89:0/201112043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cdb06a0 0x55ec1cdb6040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4e74009a20 tx=0x7f4e7402a480 comp rx=0 tx=0).stop 2024-09-10T08:45:17.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.358+0000 7f4e8721c740 1 -- 172.21.15.89:0/201112043 shutdown_connections 2024-09-10T08:45:17.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.358+0000 7f4e8721c740 1 --2- 172.21.15.89:0/201112043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec1cdb06a0 0x55ec1cdb6040 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.358+0000 7f4e8721c740 1 --2- 172.21.15.89:0/201112043 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ec1ce2ccb0 0x55ec1ce2f0a0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.358+0000 7f4e8721c740 1 --2- 172.21.15.89:0/201112043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec1cdaf810 0x55ec1cdb5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.358+0000 7f4e8721c740 1 --2- 172.21.15.89:0/201112043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec1cd9cbd0 0x55ec1cdb55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.358+0000 7f4e8721c740 1 -- 172.21.15.89:0/201112043 >> 172.21.15.89:0/201112043 conn(0x55ec1cd99240 msgr2=0x55ec1cda8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:17.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.358+0000 7f4e63400640 0 client.26501 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:17.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.358+0000 7f4e8721c740 1 -- 172.21.15.89:0/201112043 shutdown_connections 2024-09-10T08:45:17.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.358+0000 7f4e8721c740 1 -- 172.21.15.89:0/201112043 wait complete. 2024-09-10T08:45:17.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.358+0000 7f4e8721c740 20 client.26501 trim_cache size 0 max 16384 2024-09-10T08:45:17.361 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:45:17.362 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2024-09-10T08:45:17.362 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "586d0657-64b8-465e-b9a0-0767a07ac56b", 2024-09-10T08:45:17.362 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2024-09-10T08:45:17.362 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:45:17.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.359+0000 7f4e83000640 1 -- 172.21.15.89:0/1819458164 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f4e640a2b80 msgr2=0x7f4e640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:17.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.359+0000 7f4e83000640 1 --2- 172.21.15.89:0/1819458164 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f4e640a2b80 0x7f4e640a5040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4e700988b0 tx=0x7f4e700d34f0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.359+0000 7f4e83000640 1 -- 172.21.15.89:0/1819458164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e7c0745c0 msgr2=0x7f4e7c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:17.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.359+0000 7f4e83000640 1 --2- 172.21.15.89:0/1819458164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e7c0745c0 0x7f4e7c19a490 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000eb90 tx=0x7f4e7000c6a0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.359+0000 7f4e83000640 1 -- 172.21.15.89:0/1819458164 shutdown_connections 2024-09-10T08:45:17.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.359+0000 7f4e83000640 1 --2- 172.21.15.89:0/1819458164 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f4e640a2b80 0x7f4e640a5040 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.359+0000 7f4e83000640 1 --2- 172.21.15.89:0/1819458164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e7c10e260 0x7f4e7c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.359+0000 7f4e83000640 1 --2- 172.21.15.89:0/1819458164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e7c074ee0 0x7f4e7c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.359+0000 7f4e83000640 1 --2- 172.21.15.89:0/1819458164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e7c0745c0 0x7f4e7c19a490 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.359+0000 7f4e83000640 1 -- 172.21.15.89:0/1819458164 >> 172.21.15.89:0/1819458164 conn(0x7f4e7c0fd740 msgr2=0x7f4e7c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.360+0000 7f4e83000640 1 -- 172.21.15.89:0/1819458164 shutdown_connections 2024-09-10T08:45:17.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:17.360+0000 7f4e83000640 1 -- 172.21.15.89:0/1819458164 wait complete. 2024-09-10T08:45:17.952 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:45:17 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-b[65063]: 2024-09-10T08:45:17.565+0000 7fb147ed3640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000356e [...2,head] /volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-65/ auth{1=1} fragtree_t(*^3) v174153 ap=3 RANDEPHEMERALPIN f(v3 m2024-09-10T08:43:28.763947+0000 143=143+0) n(v5 rc2024-09-10T08:43:28.763947+0000 b3795327 144=143+1) (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:13310 nref=3)) caps={25157=pAsLsXsFs/pAsLsXsFsx@276} mcw={1=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x557fab76f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000356e:[<0x10000000399/blog-65 v121927>,<0x10000000208/blogtest_in v813>,<0x10000000201/src v1787>,<0x10000000006/blogbench-1.0 v1562>,<0x10000000005/tmp v185>,<0x100000001fb/client.0 v172>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v184>,<0x10000000001/sv_1 v201>,<0x10000000000/qa v190>,<0x1/volumes v185>]//[]","memoryvalue":"(2)0x1000000356e:[<0x10000000399/blog-65 v174153>,<0x10000000208/blogtest_in v1205>,<0x10000000201/src v1963>,<0x10000000006/blogbench-1.0 v1721>,<0x10000000005/tmp v321>,<0x100000001fb/client.0 v312>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v315>,<0x10000000001/sv_1 v341>,<0x10000000000/qa v315>,<0x1/volumes v318>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T08:43:17.516209+0000 142=142+0)","ondisk_value.rstat":"n(v0 rc2024-09-10T08:43:28.763947+0000 b3795327 144=143+1)","memory_value.dirstat":"f(v3 m2024-09-10T08:43:28.763947+0000 143=143+0)","memory_value.rstat":"n(v5 rc2024-09-10T08:43:28.763947+0000 b3795327 144=143+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-10T08:45:17.953 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:45:17 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-b[65063]: 2024-09-10T08:45:17.607+0000 7fb147ed3640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000006bd [...2,head] /volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/ auth{2=1} fragtree_t(*^3) v174221 ap=3 RANDEPHEMERALPIN f(v11 m2024-09-10T08:43:41.846642+0000 154=154+0) n(v14 rc2024-09-10T08:43:41.846642+0000 b2261379 155=154+1) (inest sync->mix g=2) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:13326 nref=3)) caps={25157=pAsLsXsFs/pAsLsXsFs@1068} mcw={2=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x557fab447180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000006bd:[<0x10000000399/blog-6 v16039>,<0x10000000208/blogtest_in v353>,<0x10000000201/src v1661>,<0x10000000006/blogbench-1.0 v1440>,<0x10000000005/tmp v65>,<0x100000001fb/client.0 v58>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v72>,<0x10000000001/sv_1 v89>,<0x10000000000/qa v80>,<0x1/volumes v76>]//[]","memoryvalue":"(2)0x100000006bd:[<0x10000000399/blog-6 v174221>,<0x10000000208/blogtest_in v1205>,<0x10000000201/src v1963>,<0x10000000006/blogbench-1.0 v1721>,<0x10000000005/tmp v321>,<0x100000001fb/client.0 v312>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v315>,<0x10000000001/sv_1 v341>,<0x10000000000/qa v315>,<0x1/volumes v318>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T08:43:22.515092+0000 152=152+0)","ondisk_value.rstat":"n(v0 rc2024-09-10T08:43:41.846642+0000 b2261379 155=154+1)","memory_value.dirstat":"f(v11 m2024-09-10T08:43:41.846642+0000 154=154+0)","memory_value.rstat":"n(v14 rc2024-09-10T08:43:41.846642+0000 b2261379 155=154+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-10T08:45:17.953 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:45:17 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-b[65063]: 2024-09-10T08:45:17.627+0000 7fb147ed3640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007f6 [...2,head] /volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-8/ auth{1=8} fragtree_t(*^3) v174223 ap=3 RANDEPHEMERALPIN f(v10 m2024-09-10T08:43:29.695925+0000 244=244+0) n(v15 rc2024-09-10T08:43:29.695925+0000 b4369368 245=244+1) (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:13324 nref=3)) caps={25157=pAsLsXsFs/pAsLsXsFs@566} mcw={1=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x557fab8aa000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000007f6:[<0x10000000399/blog-8 v23913>,<0x10000000208/blogtest_in v353>,<0x10000000201/src v1661>,<0x10000000006/blogbench-1.0 v1440>,<0x10000000005/tmp v65>,<0x100000001fb/client.0 v58>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v72>,<0x10000000001/sv_1 v89>,<0x10000000000/qa v80>,<0x1/volumes v76>]//[]","memoryvalue":"(2)0x100000007f6:[<0x10000000399/blog-8 v174223>,<0x10000000208/blogtest_in v1205>,<0x10000000201/src v1963>,<0x10000000006/blogbench-1.0 v1721>,<0x10000000005/tmp v321>,<0x100000001fb/client.0 v312>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v315>,<0x10000000001/sv_1 v341>,<0x10000000000/qa v315>,<0x1/volumes v318>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-10T08:43:13.681298+0000 243=243+0)","ondisk_value.rstat":"n(v0 rc2024-09-10T08:43:29.695925+0000 b4369368 245=244+1)","memory_value.dirstat":"f(v10 m2024-09-10T08:43:29.695925+0000 244=244+0)","memory_value.rstat":"n(v15 rc2024-09-10T08:43:29.695925+0000 b4369368 245=244+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-10T08:45:18.045 INFO:tasks.workunit.client.0.smithi089.stdout: 89 12878 0 9248 0 7871 0 2024-09-10T08:45:18.409 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:17 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:45:18.409 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:17 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:18.409 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:17 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:45:18.409 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:17 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:18.409 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:17 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:45:18.409 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:17 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:18.410 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:17 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:45:18.410 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:17 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:18.410 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:17 smithi111 ceph-mon[35220]: pgmap v554: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 420 op/s 2024-09-10T08:45:18.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:17 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:45:18.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:17 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:18.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:17 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:45:18.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:17 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:18.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:17 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:45:18.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:17 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:18.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:17 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:45:18.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:17 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:18.410 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:17 smithi149 ceph-mon[36167]: pgmap v554: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 420 op/s 2024-09-10T08:45:18.411 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:17 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:45:18.411 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:17 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:18.411 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:17 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:45:18.411 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:17 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:18.411 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:17 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:45:18.411 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:17 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:18.411 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:17 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:45:18.411 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:17 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:18.411 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:17 smithi089 ceph-mon[30757]: pgmap v554: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 420 op/s 2024-09-10T08:45:18.412 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:45:18.851 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:19.213 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:18 smithi089 ceph-mon[30757]: scrub queued for path: / 2024-09-10T08:45:19.213 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:18 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:19.213 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:18 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:45:19.214 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:18 smithi089 ceph-mon[30757]: Scrub error on inode 0x1000000356e (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-65) see mds.b log and `damage ls` output for details 2024-09-10T08:45:19.214 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:18 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:45:19.214 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:18 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:19.214 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:18 smithi089 ceph-mon[30757]: Scrub error on inode 0x100000006bd (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6) see mds.b log and `damage ls` output for details 2024-09-10T08:45:19.214 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:18 smithi089 ceph-mon[30757]: Scrub error on inode 0x100000007f6 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-8) see mds.b log and `damage ls` output for details 2024-09-10T08:45:19.214 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:18 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:45:19.214 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:18 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:19.214 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:18 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:45:19.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:18 smithi149 ceph-mon[36167]: scrub queued for path: / 2024-09-10T08:45:19.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:18 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:19.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:18 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:45:19.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:18 smithi149 ceph-mon[36167]: Scrub error on inode 0x1000000356e (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-65) see mds.b log and `damage ls` output for details 2024-09-10T08:45:19.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:18 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:45:19.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:18 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:19.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:18 smithi149 ceph-mon[36167]: Scrub error on inode 0x100000006bd (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6) see mds.b log and `damage ls` output for details 2024-09-10T08:45:19.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:18 smithi149 ceph-mon[36167]: Scrub error on inode 0x100000007f6 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-8) see mds.b log and `damage ls` output for details 2024-09-10T08:45:19.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:18 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:45:19.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:18 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:19.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:18 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:45:19.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:18 smithi111 ceph-mon[35220]: scrub queued for path: / 2024-09-10T08:45:19.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:18 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:19.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:18 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:45:19.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:18 smithi111 ceph-mon[35220]: Scrub error on inode 0x1000000356e (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-65) see mds.b log and `damage ls` output for details 2024-09-10T08:45:19.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:18 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:45:19.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:18 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:19.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:18 smithi111 ceph-mon[35220]: Scrub error on inode 0x100000006bd (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6) see mds.b log and `damage ls` output for details 2024-09-10T08:45:19.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:18 smithi111 ceph-mon[35220]: Scrub error on inode 0x100000007f6 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-8) see mds.b log and `damage ls` output for details 2024-09-10T08:45:19.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:18 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:45:19.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:18 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:19.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:18 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:45:19.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.450+0000 7fb2e7200640 1 -- 172.21.15.89:0/1808142237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2e8109090 msgr2=0x7fb2e8110c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.450+0000 7fb2e7200640 1 --2- 172.21.15.89:0/1808142237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2e8109090 0x7fb2e8110c10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009ca0 tx=0x7fb2e002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.450+0000 7fb2e7200640 1 -- 172.21.15.89:0/1808142237 shutdown_connections 2024-09-10T08:45:19.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.450+0000 7fb2e7200640 1 --2- 172.21.15.89:0/1808142237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2e8109090 0x7fb2e8110c10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.450+0000 7fb2e7200640 1 --2- 172.21.15.89:0/1808142237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2e8104800 0x7fb2e8108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.450+0000 7fb2e7200640 1 --2- 172.21.15.89:0/1808142237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2e8103e50 0x7fb2e8104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.450+0000 7fb2e7200640 1 -- 172.21.15.89:0/1808142237 >> 172.21.15.89:0/1808142237 conn(0x7fb2e80ff8b0 msgr2=0x7fb2e8101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:19.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.450+0000 7fb2e7200640 1 -- 172.21.15.89:0/1808142237 shutdown_connections 2024-09-10T08:45:19.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.450+0000 7fb2e7200640 1 -- 172.21.15.89:0/1808142237 wait complete. 2024-09-10T08:45:19.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.452+0000 7fb2e7200640 1 Processor -- start 2024-09-10T08:45:19.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.452+0000 7fb2e7200640 1 -- start start 2024-09-10T08:45:19.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.452+0000 7fb2e7200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2e8103e50 0x7fb2e819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.452+0000 7fb2e7200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2e8104800 0x7fb2e819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.452+0000 7fb2e7200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2e8109090 0x7fb2e819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.452+0000 7fb2e7200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81140f0 con 0x7fb2e8109090 2024-09-10T08:45:19.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.452+0000 7fb2e7200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e8113f70 con 0x7fb2e8103e50 2024-09-10T08:45:19.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.452+0000 7fb2e7200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e8114270 con 0x7fb2e8104800 2024-09-10T08:45:19.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.452+0000 7fb2e5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2e8103e50 0x7fb2e819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.452+0000 7fb2e5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2e8103e50 0x7fb2e819a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:36036/0 (socket says 172.21.15.89:36036) 2024-09-10T08:45:19.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.452+0000 7fb2e5e00640 1 -- 172.21.15.89:0/1557739722 learned_addr learned my addr 172.21.15.89:0/1557739722 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:19.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.452+0000 7fb2e6800640 1 --2- 172.21.15.89:0/1557739722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2e8109090 0x7fb2e819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.453+0000 7fb2dfe00640 1 --2- 172.21.15.89:0/1557739722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2e8104800 0x7fb2e819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.453+0000 7fb2e5e00640 1 -- 172.21.15.89:0/1557739722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2e8104800 msgr2=0x7fb2e819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.453+0000 7fb2e5e00640 1 --2- 172.21.15.89:0/1557739722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2e8104800 0x7fb2e819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.453+0000 7fb2e5e00640 1 -- 172.21.15.89:0/1557739722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2e8109090 msgr2=0x7fb2e819ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.453+0000 7fb2e5e00640 1 --2- 172.21.15.89:0/1557739722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2e8109090 0x7fb2e819ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.453+0000 7fb2e5e00640 1 -- 172.21.15.89:0/1557739722 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e819f500 con 0x7fb2e8103e50 2024-09-10T08:45:19.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.453+0000 7fb2e5e00640 1 --2- 172.21.15.89:0/1557739722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2e8103e50 0x7fb2e819a550 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800e9f0 tx=0x7fb2d800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:19.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.453+0000 7fb2dea00640 1 -- 172.21.15.89:0/1557739722 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d800cd80 con 0x7fb2e8103e50 2024-09-10T08:45:19.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.453+0000 7fb2e7200640 1 -- 172.21.15.89:0/1557739722 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e819f7f0 con 0x7fb2e8103e50 2024-09-10T08:45:19.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.454+0000 7fb2dea00640 1 -- 172.21.15.89:0/1557739722 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2d8010910 con 0x7fb2e8103e50 2024-09-10T08:45:19.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.454+0000 7fb2dea00640 1 -- 172.21.15.89:0/1557739722 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d80188f0 con 0x7fb2e8103e50 2024-09-10T08:45:19.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.454+0000 7fb2e7200640 1 -- 172.21.15.89:0/1557739722 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e81a70d0 con 0x7fb2e8103e50 2024-09-10T08:45:19.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.457+0000 7fb2efbdc740 1 Processor -- start 2024-09-10T08:45:19.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.457+0000 7fb2efbdc740 1 -- start start 2024-09-10T08:45:19.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.457+0000 7fb2dea00640 1 -- 172.21.15.89:0/1557739722 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb2d8010ac0 con 0x7fb2e8103e50 2024-09-10T08:45:19.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.457+0000 7fb2efbdc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403951d50 0x55e403952130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.457+0000 7fb2efbdc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e403952670 0x55e40395c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.458+0000 7fb2efbdc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40395cf30 0x55e40395f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.458+0000 7fb2efbdc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e4038b83e0 con 0x55e40395cf30 2024-09-10T08:45:19.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.458+0000 7fb2efbdc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e40385fd80 con 0x55e403952670 2024-09-10T08:45:19.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.458+0000 7fb2efbdc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e4038b43a0 con 0x55e403951d50 2024-09-10T08:45:19.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.458+0000 7fb2e5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e403952670 0x55e40395c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.458+0000 7fb2e6800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403951d50 0x55e403952130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.458+0000 7fb2e6800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403951d50 0x55e403952130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:41274/0 (socket says 172.21.15.89:41274) 2024-09-10T08:45:19.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.458+0000 7fb2e6800640 1 -- 172.21.15.89:0/407076149 learned_addr learned my addr 172.21.15.89:0/407076149 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:19.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.458+0000 7fb2dfe00640 1 --2- 172.21.15.89:0/407076149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40395cf30 0x55e40395f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.458+0000 7fb2e5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e403952670 0x55e40395c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:36042/0 (socket says 172.21.15.89:36042) 2024-09-10T08:45:19.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.458+0000 7fb2e6800640 1 -- 172.21.15.89:0/407076149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e403952670 msgr2=0x55e40395c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:19.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.458+0000 7fb2e6800640 1 --2- 172.21.15.89:0/407076149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e403952670 0x55e40395c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.458+0000 7fb2e6800640 1 -- 172.21.15.89:0/407076149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40395cf30 msgr2=0x55e40395f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:19.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.458+0000 7fb2e6800640 1 --2- 172.21.15.89:0/407076149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40395cf30 0x55e40395f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.458+0000 7fb2e6800640 1 -- 172.21.15.89:0/407076149 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e40387eae0 con 0x55e403951d50 2024-09-10T08:45:19.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.459+0000 7fb2e6800640 1 --2- 172.21.15.89:0/407076149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403951d50 0x55e403952130 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009ca0 tx=0x7fb2e00093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed4699c9b71e5644 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:19.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.459+0000 7fb2dea00640 1 --2- 172.21.15.89:0/1557739722 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb2cc0a2b80 0x7fb2cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.459+0000 7fb2d7400640 1 -- 172.21.15.89:0/407076149 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0031db0 con 0x55e403951d50 2024-09-10T08:45:19.461 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.459+0000 7fb2d7400640 1 -- 172.21.15.89:0/407076149 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2e0036030 con 0x55e403951d50 2024-09-10T08:45:19.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.459+0000 7fb2d7400640 1 -- 172.21.15.89:0/407076149 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0004330 con 0x55e403951d50 2024-09-10T08:45:19.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.459+0000 7fb2dea00640 1 -- 172.21.15.89:0/1557739722 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb2d80cf7a0 con 0x7fb2e8103e50 2024-09-10T08:45:19.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.460+0000 7fb2e5e00640 1 --2- 172.21.15.89:0/1557739722 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb2cc0a2b80 0x7fb2cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.460+0000 7fb2efbdc740 1 -- 172.21.15.89:0/407076149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403951d50 msgr2=0x55e403952130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.460+0000 7fb2efbdc740 1 --2- 172.21.15.89:0/407076149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403951d50 0x55e403952130 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009ca0 tx=0x7fb2e00093e0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.460+0000 7fb2efbdc740 1 -- 172.21.15.89:0/407076149 shutdown_connections 2024-09-10T08:45:19.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.460+0000 7fb2efbdc740 1 --2- 172.21.15.89:0/407076149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e40395cf30 0x55e40395f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.460+0000 7fb2efbdc740 1 --2- 172.21.15.89:0/407076149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e403952670 0x55e40395c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.460+0000 7fb2efbdc740 1 --2- 172.21.15.89:0/407076149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403951d50 0x55e403952130 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.460+0000 7fb2efbdc740 1 -- 172.21.15.89:0/407076149 >> 172.21.15.89:0/407076149 conn(0x55e40394e2e0 msgr2=0x55e40394e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:19.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.460+0000 7fb2efbdc740 1 -- 172.21.15.89:0/407076149 shutdown_connections 2024-09-10T08:45:19.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.460+0000 7fb2efbdc740 1 -- 172.21.15.89:0/407076149 wait complete. 2024-09-10T08:45:19.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.460+0000 7fb2e5e00640 1 --2- 172.21.15.89:0/1557739722 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb2cc0a2b80 0x7fb2cc0a5040 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80cd240 tx=0x7fb2d801f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:19.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.461+0000 7fb2efbdc740 1 -- start start 2024-09-10T08:45:19.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.461+0000 7fb2efbdc740 1 Processor -- start 2024-09-10T08:45:19.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.461+0000 7fb2efbdc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e403951d50 0x55e403959eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.462+0000 7fb2efbdc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403952670 0x55e40395a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.462+0000 7fb2efbdc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e40395cf30 0x55e40395a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.462+0000 7fb2efbdc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e40395fa20 con 0x55e403951d50 2024-09-10T08:45:19.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.462+0000 7fb2efbdc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e4038730f0 con 0x55e40395cf30 2024-09-10T08:45:19.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.462+0000 7fb2efbdc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e40395fba0 con 0x55e403952670 2024-09-10T08:45:19.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.462+0000 7fb2efbdc740 10 client.0 _ll_register_callbacks cb 0x55e4038649e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:19.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.462+0000 7fb2dfe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403952670 0x55e40395a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.462+0000 7fb2dfe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403952670 0x55e40395a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:41284/0 (socket says 172.21.15.89:41284) 2024-09-10T08:45:19.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.462+0000 7fb2dfe00640 1 -- 172.21.15.89:0/1981290871 learned_addr learned my addr 172.21.15.89:0/1981290871 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:19.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.462+0000 7fb2e6800640 1 --2- 172.21.15.89:0/1981290871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e403951d50 0x55e403959eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.462+0000 7fb2dfe00640 1 -- 172.21.15.89:0/1981290871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e40395cf30 msgr2=0x55e40395a930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:45:19.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.462+0000 7fb2dfe00640 1 --2- 172.21.15.89:0/1981290871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e40395cf30 0x55e40395a930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.462+0000 7fb2dfe00640 1 -- 172.21.15.89:0/1981290871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e403951d50 msgr2=0x55e403959eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:19.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.462+0000 7fb2dfe00640 1 --2- 172.21.15.89:0/1981290871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e403951d50 0x55e403959eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.462+0000 7fb2dfe00640 1 -- 172.21.15.89:0/1981290871 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4036293c0 con 0x55e403952670 2024-09-10T08:45:19.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.463+0000 7fb2dfe00640 1 --2- 172.21.15.89:0/1981290871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403952670 0x55e40395a3f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0012700 tx=0x7fb2d0012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa06171fc39a8932 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:19.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.463+0000 7fb2cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:45:19.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.463+0000 7fb2cbe00640 1 -- 172.21.15.89:0/1981290871 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d000f8c0 con 0x55e403952670 2024-09-10T08:45:19.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.463+0000 7fb2cbe00640 1 -- 172.21.15.89:0/1981290871 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb2d000fa60 con 0x55e403952670 2024-09-10T08:45:19.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.463+0000 7fb2efbdc740 1 -- 172.21.15.89:0/1981290871 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2e0036030 con 0x55e403952670 2024-09-10T08:45:19.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.464+0000 7fb2cbe00640 1 -- 172.21.15.89:0/1981290871 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb2d000fc60 con 0x55e403952670 2024-09-10T08:45:19.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.464+0000 7fb2efbdc740 10 client.26437 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:19.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.464+0000 7fb2efbdc740 1 -- 172.21.15.89:0/1981290871 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e40396b800 con 0x55e403952670 2024-09-10T08:45:19.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.464+0000 7fb2cbe00640 1 -- 172.21.15.89:0/1981290871 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2e0036030 con 0x55e403952670 2024-09-10T08:45:19.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.464+0000 7fb2cbe00640 1 -- 172.21.15.89:0/1981290871 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fb2d0010550 con 0x55e403952670 2024-09-10T08:45:19.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.465+0000 7fb2efbdc740 10 client.26437 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:19.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.465+0000 7fb2efbdc740 10 client.26437 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:45:19.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.465+0000 7fb2efbdc740 20 client.26437 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:19.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.465+0000 7fb2efbdc740 1 --2- 172.21.15.89:0/1981290871 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e40396ccc0 0x55e40396f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.465+0000 7fb2efbdc740 4 client.26437 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:45:19.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.465+0000 7fb2efbdc740 1 -- 172.21.15.89:0/1981290871 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4038c5950 con 0x55e40396ccc0 2024-09-10T08:45:19.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.465+0000 7fb2e6800640 1 --2- 172.21.15.89:0/1981290871 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e40396ccc0 0x55e40396f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.466+0000 7fb2e6800640 1 --2- 172.21.15.89:0/1981290871 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e40396ccc0 0x55e40396f0b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1671e32bc42b8765 server_cookie=9b9085d9ec47aa21 in_seq=0 out_seq=0 2024-09-10T08:45:19.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.466+0000 7fb2cbe00640 10 client.26437 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:19.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.467+0000 7fb2cbe00640 1 -- 172.21.15.89:0/1981290871 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4038c5950 con 0x55e40396ccc0 2024-09-10T08:45:19.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.467+0000 7fb2cbe00640 10 client.26437 handle_command_reply: tid=0 2024-09-10T08:45:19.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.468+0000 7fb2efbdc740 1 client.26437 shutdown 2024-09-10T08:45:19.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.468+0000 7fb2efbdc740 1 -- 172.21.15.89:0/1981290871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403952670 msgr2=0x55e40395a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.468+0000 7fb2efbdc740 1 --2- 172.21.15.89:0/1981290871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403952670 0x55e40395a3f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0012700 tx=0x7fb2d0012bc0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.468+0000 7fb2efbdc740 1 -- 172.21.15.89:0/1981290871 shutdown_connections 2024-09-10T08:45:19.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.468+0000 7fb2efbdc740 1 --2- 172.21.15.89:0/1981290871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e40395cf30 0x55e40395a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.468+0000 7fb2efbdc740 1 --2- 172.21.15.89:0/1981290871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403952670 0x55e40395a3f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.468+0000 7fb2efbdc740 1 --2- 172.21.15.89:0/1981290871 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e40396ccc0 0x55e40396f0b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.468+0000 7fb2cbe00640 0 client.26437 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:19.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.468+0000 7fb2efbdc740 1 --2- 172.21.15.89:0/1981290871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e403951d50 0x55e403959eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.468+0000 7fb2efbdc740 1 -- 172.21.15.89:0/1981290871 >> 172.21.15.89:0/1981290871 conn(0x55e40394e2e0 msgr2=0x55e403953fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:19.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.468+0000 7fb2efbdc740 1 -- 172.21.15.89:0/1981290871 shutdown_connections 2024-09-10T08:45:19.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.468+0000 7fb2efbdc740 1 -- 172.21.15.89:0/1981290871 wait complete. 2024-09-10T08:45:19.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.469+0000 7fb2efbdc740 20 client.26437 trim_cache size 0 max 16384 2024-09-10T08:45:19.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.492+0000 7fb2efbdc740 1 Processor -- start 2024-09-10T08:45:19.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.492+0000 7fb2efbdc740 1 -- start start 2024-09-10T08:45:19.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.492+0000 7fb2efbdc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e403951d50 0x55e403949060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.492+0000 7fb2efbdc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e403952670 0x55e40395be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.492+0000 7fb2efbdc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e40395cf30 0x55e403974960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.492+0000 7fb2efbdc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81b3710 con 0x55e403951d50 2024-09-10T08:45:19.492 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.492+0000 7fb2efbdc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81b0fe0 con 0x55e403952670 2024-09-10T08:45:19.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.492+0000 7fb2efbdc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81aee30 con 0x55e40395cf30 2024-09-10T08:45:19.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.492+0000 7fb2dfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e403952670 0x55e40395be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.492+0000 7fb2e6800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e403951d50 0x55e403949060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.492+0000 7fb2dfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e403952670 0x55e40395be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:36070/0 (socket says 172.21.15.89:36070) 2024-09-10T08:45:19.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.493+0000 7fb2dfe00640 1 -- 172.21.15.89:0/1127317605 learned_addr learned my addr 172.21.15.89:0/1127317605 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:19.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.493+0000 7fb2e6800640 1 --2- 172.21.15.89:0/1127317605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e40395cf30 0x55e403974960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.493+0000 7fb2dfe00640 1 -- 172.21.15.89:0/1127317605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e40395cf30 msgr2=0x55e403974960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:19.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.493+0000 7fb2dfe00640 1 --2- 172.21.15.89:0/1127317605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e40395cf30 0x55e403974960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.493+0000 7fb2dfe00640 1 -- 172.21.15.89:0/1127317605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e403951d50 msgr2=0x55e403949060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:19.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.493+0000 7fb2dfe00640 1 --2- 172.21.15.89:0/1127317605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e403951d50 0x55e403949060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.493+0000 7fb2dfe00640 1 -- 172.21.15.89:0/1127317605 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4039524a0 con 0x55e403952670 2024-09-10T08:45:19.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.493+0000 7fb2e6800640 1 --2- 172.21.15.89:0/1127317605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e403951d50 0x55e403949060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:19.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.493+0000 7fb2dfe00640 1 --2- 172.21.15.89:0/1127317605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e403952670 0x55e40395be20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0010b10 tx=0x7fb2d0011fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d685beaa733dc952 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:19.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.493+0000 7fb2d6a00640 1 -- 172.21.15.89:0/1127317605 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d002e070 con 0x55e403952670 2024-09-10T08:45:19.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.493+0000 7fb2d6a00640 1 -- 172.21.15.89:0/1127317605 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2d0010d10 con 0x55e403952670 2024-09-10T08:45:19.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.494+0000 7fb2d6a00640 1 -- 172.21.15.89:0/1127317605 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d000faa0 con 0x55e403952670 2024-09-10T08:45:19.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.494+0000 7fb2efbdc740 1 -- 172.21.15.89:0/1127317605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e403952670 msgr2=0x55e40395be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.494+0000 7fb2efbdc740 1 --2- 172.21.15.89:0/1127317605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e403952670 0x55e40395be20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0010b10 tx=0x7fb2d0011fd0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.494+0000 7fb2efbdc740 1 -- 172.21.15.89:0/1127317605 shutdown_connections 2024-09-10T08:45:19.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.494+0000 7fb2efbdc740 1 --2- 172.21.15.89:0/1127317605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e40395cf30 0x55e403974960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.494+0000 7fb2efbdc740 1 --2- 172.21.15.89:0/1127317605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e403952670 0x55e40395be20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.494+0000 7fb2efbdc740 1 --2- 172.21.15.89:0/1127317605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e403951d50 0x55e403949060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.494+0000 7fb2efbdc740 1 -- 172.21.15.89:0/1127317605 >> 172.21.15.89:0/1127317605 conn(0x55e40394e2e0 msgr2=0x55e40395b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:19.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.494+0000 7fb2efbdc740 1 -- 172.21.15.89:0/1127317605 shutdown_connections 2024-09-10T08:45:19.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.495+0000 7fb2efbdc740 1 -- 172.21.15.89:0/1127317605 wait complete. 2024-09-10T08:45:19.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.495+0000 7fb2efbdc740 1 -- start start 2024-09-10T08:45:19.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.495+0000 7fb2efbdc740 1 Processor -- start 2024-09-10T08:45:19.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.495+0000 7fb2efbdc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403951d50 0x55e403964df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.495+0000 7fb2efbdc740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e403952670 0x55e403965330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.496+0000 7fb2efbdc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e40395cf30 0x55e403961dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.496+0000 7fb2efbdc740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81af4a0 con 0x55e403952670 2024-09-10T08:45:19.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.496+0000 7fb2efbdc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81b0910 con 0x55e40395cf30 2024-09-10T08:45:19.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.496+0000 7fb2efbdc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81afb70 con 0x55e403951d50 2024-09-10T08:45:19.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.496+0000 7fb2efbdc740 10 client.0 _ll_register_callbacks cb 0x7fb2d0009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:19.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.496+0000 7fb2e6800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403951d50 0x55e403964df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.496+0000 7fb2e6800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403951d50 0x55e403964df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:41294/0 (socket says 172.21.15.89:41294) 2024-09-10T08:45:19.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.496+0000 7fb2e6800640 1 -- 172.21.15.89:0/830115987 learned_addr learned my addr 172.21.15.89:0/830115987 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:19.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.496+0000 7fb2dfe00640 1 --2- 172.21.15.89:0/830115987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e403952670 0x55e403965330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.496+0000 7fb2e6800640 1 --2- 172.21.15.89:0/830115987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e40395cf30 0x55e403961dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.496+0000 7fb2e6800640 1 -- 172.21.15.89:0/830115987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e40395cf30 msgr2=0x55e403961dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:19.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.496+0000 7fb2e6800640 1 --2- 172.21.15.89:0/830115987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e40395cf30 0x55e403961dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.496+0000 7fb2e6800640 1 -- 172.21.15.89:0/830115987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e403952670 msgr2=0x55e403965330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:19.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.496+0000 7fb2e6800640 1 --2- 172.21.15.89:0/830115987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e403952670 0x55e403965330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.496+0000 7fb2e6800640 1 -- 172.21.15.89:0/830115987 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e403965a70 con 0x55e403951d50 2024-09-10T08:45:19.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.497+0000 7fb2e6800640 1 --2- 172.21.15.89:0/830115987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403951d50 0x55e403964df0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0004af0 tx=0x7fb2e0005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55b4aac6b99e9109 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:19.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.497+0000 7fb2cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:45:19.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.497+0000 7fb2cbe00640 1 -- 172.21.15.89:0/830115987 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e002f610 con 0x55e403951d50 2024-09-10T08:45:19.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.497+0000 7fb2efbdc740 1 -- 172.21.15.89:0/830115987 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2d0010d10 con 0x55e403951d50 2024-09-10T08:45:19.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.497+0000 7fb2cbe00640 1 -- 172.21.15.89:0/830115987 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb2e002f7b0 con 0x55e403951d50 2024-09-10T08:45:19.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.497+0000 7fb2cbe00640 1 -- 172.21.15.89:0/830115987 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb2e002f9b0 con 0x55e403951d50 2024-09-10T08:45:19.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.497+0000 7fb2efbdc740 10 client.26443 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:19.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.497+0000 7fb2efbdc740 1 -- 172.21.15.89:0/830115987 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e4039d7ff0 con 0x55e403951d50 2024-09-10T08:45:19.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.497+0000 7fb2cbe00640 1 -- 172.21.15.89:0/830115987 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2d0010d10 con 0x55e403951d50 2024-09-10T08:45:19.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.498+0000 7fb2cbe00640 1 -- 172.21.15.89:0/830115987 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fb2e0043070 con 0x55e403951d50 2024-09-10T08:45:19.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.498+0000 7fb2efbdc740 10 client.26443 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:19.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.498+0000 7fb2efbdc740 10 client.26443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:45:19.500 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.498+0000 7fb2efbdc740 20 client.26443 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:19.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.498+0000 7fb2efbdc740 1 --2- 172.21.15.89:0/830115987 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e40396ccc0 0x55e4039da3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.498+0000 7fb2efbdc740 4 client.26443 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:45:19.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.498+0000 7fb2efbdc740 1 -- 172.21.15.89:0/830115987 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e4038bfe60 con 0x55e40396ccc0 2024-09-10T08:45:19.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.498+0000 7fb2dfe00640 1 --2- 172.21.15.89:0/830115987 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e40396ccc0 0x55e4039da3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.499+0000 7fb2cbe00640 10 client.26443 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:19.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.499+0000 7fb2dfe00640 1 --2- 172.21.15.89:0/830115987 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e40396ccc0 0x55e4039da3b0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=660695c35c8bc443 server_cookie=2e5469d8969114e7 in_seq=0 out_seq=0 2024-09-10T08:45:19.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.501+0000 7fb2cbe00640 1 -- 172.21.15.89:0/830115987 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55e4038bfe60 con 0x55e40396ccc0 2024-09-10T08:45:19.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.501+0000 7fb2cbe00640 10 client.26443 handle_command_reply: tid=0 2024-09-10T08:45:19.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.501+0000 7fb2efbdc740 1 client.26443 shutdown 2024-09-10T08:45:19.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.501+0000 7fb2efbdc740 1 -- 172.21.15.89:0/830115987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403951d50 msgr2=0x55e403964df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.501+0000 7fb2efbdc740 1 --2- 172.21.15.89:0/830115987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403951d50 0x55e403964df0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0004af0 tx=0x7fb2e0005d80 comp rx=0 tx=0).stop 2024-09-10T08:45:19.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.501+0000 7fb2efbdc740 1 -- 172.21.15.89:0/830115987 shutdown_connections 2024-09-10T08:45:19.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.501+0000 7fb2efbdc740 1 --2- 172.21.15.89:0/830115987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e40395cf30 0x55e403961dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.501+0000 7fb2efbdc740 1 --2- 172.21.15.89:0/830115987 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55e40396ccc0 0x55e4039da3b0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.501+0000 7fb2efbdc740 1 --2- 172.21.15.89:0/830115987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e403952670 0x55e403965330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.501+0000 7fb2efbdc740 1 --2- 172.21.15.89:0/830115987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e403951d50 0x55e403964df0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.501+0000 7fb2efbdc740 1 -- 172.21.15.89:0/830115987 >> 172.21.15.89:0/830115987 conn(0x55e40394e2e0 msgr2=0x55e40395a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:19.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.501+0000 7fb2cbe00640 0 client.26443 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:19.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.502+0000 7fb2efbdc740 1 -- 172.21.15.89:0/830115987 shutdown_connections 2024-09-10T08:45:19.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.502+0000 7fb2efbdc740 1 -- 172.21.15.89:0/830115987 wait complete. 2024-09-10T08:45:19.503 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.502+0000 7fb2efbdc740 20 client.26443 trim_cache size 0 max 16384 2024-09-10T08:45:19.503 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:45:19.503 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "scrub active (9979 inodes in the stack)", 2024-09-10T08:45:19.504 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2024-09-10T08:45:19.504 INFO:teuthology.orchestra.run.smithi089.stdout: "586d0657-64b8-465e-b9a0-0767a07ac56b": { 2024-09-10T08:45:19.504 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2024-09-10T08:45:19.504 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "586d0657-64b8-465e-b9a0-0767a07ac56b", 2024-09-10T08:45:19.504 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2024-09-10T08:45:19.504 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:45:19.504 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:45:19.504 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:45:19.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.503+0000 7fb2e7200640 1 -- 172.21.15.89:0/1557739722 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb2cc0a2b80 msgr2=0x7fb2cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.503+0000 7fb2e7200640 1 --2- 172.21.15.89:0/1557739722 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb2cc0a2b80 0x7fb2cc0a5040 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80cd240 tx=0x7fb2d801f000 comp rx=0 tx=0).stop 2024-09-10T08:45:19.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.503+0000 7fb2e7200640 1 -- 172.21.15.89:0/1557739722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2e8103e50 msgr2=0x7fb2e819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.503+0000 7fb2e7200640 1 --2- 172.21.15.89:0/1557739722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2e8103e50 0x7fb2e819a550 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800e9f0 tx=0x7fb2d800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.503+0000 7fb2e7200640 1 -- 172.21.15.89:0/1557739722 shutdown_connections 2024-09-10T08:45:19.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.503+0000 7fb2e7200640 1 --2- 172.21.15.89:0/1557739722 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb2cc0a2b80 0x7fb2cc0a5040 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.503+0000 7fb2e7200640 1 --2- 172.21.15.89:0/1557739722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb2e8109090 0x7fb2e819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.503+0000 7fb2e7200640 1 --2- 172.21.15.89:0/1557739722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2e8104800 0x7fb2e819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.503+0000 7fb2e7200640 1 --2- 172.21.15.89:0/1557739722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2e8103e50 0x7fb2e819a550 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.503+0000 7fb2e7200640 1 -- 172.21.15.89:0/1557739722 >> 172.21.15.89:0/1557739722 conn(0x7fb2e80ff8b0 msgr2=0x7fb2e8100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:19.505 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.504+0000 7fb2e7200640 1 -- 172.21.15.89:0/1557739722 shutdown_connections 2024-09-10T08:45:19.506 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:19.504+0000 7fb2e7200640 1 -- 172.21.15.89:0/1557739722 wait complete. 2024-09-10T08:45:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.740+0000 7f5714a00640 1 -- 172.21.15.89:0/3056824180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5710101b60 msgr2=0x7f5710101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.740+0000 7f5714a00640 1 --2- 172.21.15.89:0/3056824180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5710101b60 0x7f5710101f60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5700009ca0 tx=0x7f570002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.740+0000 7f5714a00640 1 -- 172.21.15.89:0/3056824180 shutdown_connections 2024-09-10T08:45:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.740+0000 7f5714a00640 1 --2- 172.21.15.89:0/3056824180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5710103710 0x7f5710109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.740+0000 7f5714a00640 1 --2- 172.21.15.89:0/3056824180 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5710102d50 0x7f57101031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.740+0000 7f5714a00640 1 --2- 172.21.15.89:0/3056824180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5710101b60 0x7f5710101f60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.740+0000 7f5714a00640 1 -- 172.21.15.89:0/3056824180 >> 172.21.15.89:0/3056824180 conn(0x7f57100fd2e0 msgr2=0x7f57100ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.740+0000 7f5714a00640 1 -- 172.21.15.89:0/3056824180 shutdown_connections 2024-09-10T08:45:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.741+0000 7f5714a00640 1 -- 172.21.15.89:0/3056824180 wait complete. 2024-09-10T08:45:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.741+0000 7f5714a00640 1 Processor -- start 2024-09-10T08:45:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.742+0000 7f5714a00640 1 -- start start 2024-09-10T08:45:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.742+0000 7f5714a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5710101b60 0x7f571019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.742+0000 7f5714a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5710102d50 0x7f571019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.742+0000 7f5714a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5710103710 0x7f57101a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.742+0000 7f5714a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f571010d810 con 0x7f5710103710 2024-09-10T08:45:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.742+0000 7f5714a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f571010d690 con 0x7f5710101b60 2024-09-10T08:45:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.742+0000 7f5714a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f571010d990 con 0x7f5710102d50 2024-09-10T08:45:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.742+0000 7f570de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5710101b60 0x7f571019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.742+0000 7f570de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5710101b60 0x7f571019a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:36096/0 (socket says 172.21.15.89:36096) 2024-09-10T08:45:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.742+0000 7f570de00640 1 -- 172.21.15.89:0/286111385 learned_addr learned my addr 172.21.15.89:0/286111385 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.742+0000 7f5705400640 1 --2- 172.21.15.89:0/286111385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5710102d50 0x7f571019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.742+0000 7f570e800640 1 --2- 172.21.15.89:0/286111385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5710103710 0x7f57101a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.743+0000 7f5705400640 1 -- 172.21.15.89:0/286111385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5710101b60 msgr2=0x7f571019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.743+0000 7f5705400640 1 --2- 172.21.15.89:0/286111385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5710101b60 0x7f571019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.743+0000 7f5705400640 1 -- 172.21.15.89:0/286111385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5710103710 msgr2=0x7f57101a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.743+0000 7f5705400640 1 --2- 172.21.15.89:0/286111385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5710103710 0x7f57101a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.743+0000 7f5705400640 1 -- 172.21.15.89:0/286111385 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57101a1c30 con 0x7f5710102d50 2024-09-10T08:45:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.743+0000 7f570de00640 1 --2- 172.21.15.89:0/286111385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5710101b60 0x7f571019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.743+0000 7f5705400640 1 --2- 172.21.15.89:0/286111385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5710102d50 0x7f571019a550 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f56f800e9f0 tx=0x7f56f800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.743+0000 7f5707400640 1 -- 172.21.15.89:0/286111385 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f800ccc0 con 0x7f5710102d50 2024-09-10T08:45:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.743+0000 7f5707400640 1 -- 172.21.15.89:0/286111385 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f8010910 con 0x7f5710102d50 2024-09-10T08:45:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.743+0000 7f5714a00640 1 -- 172.21.15.89:0/286111385 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57101a1f20 con 0x7f5710102d50 2024-09-10T08:45:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.743+0000 7f5707400640 1 -- 172.21.15.89:0/286111385 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f8018800 con 0x7f5710102d50 2024-09-10T08:45:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.743+0000 7f5714a00640 1 -- 172.21.15.89:0/286111385 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57101a23e0 con 0x7f5710102d50 2024-09-10T08:45:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.747+0000 7f5707400640 1 -- 172.21.15.89:0/286111385 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f56f8010ac0 con 0x7f5710102d50 2024-09-10T08:45:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.747+0000 7f57178ac740 1 Processor -- start 2024-09-10T08:45:19.747 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.747+0000 7f57178ac740 1 -- start start 2024-09-10T08:45:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.747+0000 7f57178ac740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1332a090 0x563e1332a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.747+0000 7f57178ac740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332a9d0 0x563e1332f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.747+0000 7f57178ac740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332fa60 0x563e13331f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.747+0000 7f57178ac740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563e1328f100 con 0x563e1332fa60 2024-09-10T08:45:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.747+0000 7f57178ac740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563e132371b0 con 0x563e1332a9d0 2024-09-10T08:45:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.747+0000 7f57178ac740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563e1328b0c0 con 0x563e1332a090 2024-09-10T08:45:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.748+0000 7f570de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332a9d0 0x563e1332f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.748+0000 7f570e800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1332a090 0x563e1332a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.748+0000 7f570de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332a9d0 0x563e1332f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:36112/0 (socket says 172.21.15.89:36112) 2024-09-10T08:45:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.748+0000 7f570de00640 1 -- 172.21.15.89:0/2880188053 learned_addr learned my addr 172.21.15.89:0/2880188053 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.748+0000 7f5705400640 1 --2- 172.21.15.89:0/2880188053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332fa60 0x563e13331f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.748+0000 7f570e800640 1 -- 172.21.15.89:0/2880188053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332a9d0 msgr2=0x563e1332f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.748+0000 7f570e800640 1 --2- 172.21.15.89:0/2880188053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332a9d0 0x563e1332f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.748+0000 7f570e800640 1 -- 172.21.15.89:0/2880188053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332fa60 msgr2=0x563e13331f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.748+0000 7f570e800640 1 --2- 172.21.15.89:0/2880188053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332fa60 0x563e13331f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.748+0000 7f570e800640 1 -- 172.21.15.89:0/2880188053 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e12fa2210 con 0x563e1332a090 2024-09-10T08:45:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.748+0000 7f570e800640 1 --2- 172.21.15.89:0/2880188053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1332a090 0x563e1332a490 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5708009b90 tx=0x7f5708011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae24788f003ec627 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.748+0000 7f5707400640 1 --2- 172.21.15.89:0/286111385 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f56f40a2c30 0x7f56f40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.748+0000 7f56ff000640 1 -- 172.21.15.89:0/2880188053 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5708004030 con 0x563e1332a090 2024-09-10T08:45:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.748+0000 7f56ff000640 1 -- 172.21.15.89:0/2880188053 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57080041d0 con 0x563e1332a090 2024-09-10T08:45:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.748+0000 7f56ff000640 1 -- 172.21.15.89:0/2880188053 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5708019760 con 0x563e1332a090 2024-09-10T08:45:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.748+0000 7f5707400640 1 -- 172.21.15.89:0/286111385 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f56f80cf720 con 0x7f5710102d50 2024-09-10T08:45:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.749+0000 7f570de00640 1 --2- 172.21.15.89:0/286111385 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f56f40a2c30 0x7f56f40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.749+0000 7f57178ac740 1 -- 172.21.15.89:0/2880188053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1332a090 msgr2=0x563e1332a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.749+0000 7f57178ac740 1 --2- 172.21.15.89:0/2880188053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1332a090 0x563e1332a490 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5708009b90 tx=0x7f5708011810 comp rx=0 tx=0).stop 2024-09-10T08:45:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.749+0000 7f570de00640 1 --2- 172.21.15.89:0/286111385 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f56f40a2c30 0x7f56f40a50f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f570002f540 tx=0x7f570002f390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.749+0000 7f57178ac740 1 -- 172.21.15.89:0/2880188053 shutdown_connections 2024-09-10T08:45:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.749+0000 7f57178ac740 1 --2- 172.21.15.89:0/2880188053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332fa60 0x563e13331f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.749+0000 7f57178ac740 1 --2- 172.21.15.89:0/2880188053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332a9d0 0x563e1332f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.749+0000 7f57178ac740 1 --2- 172.21.15.89:0/2880188053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1332a090 0x563e1332a490 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.749+0000 7f57178ac740 1 -- 172.21.15.89:0/2880188053 >> 172.21.15.89:0/2880188053 conn(0x563e13325700 msgr2=0x563e13325b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.749+0000 7f57178ac740 1 -- 172.21.15.89:0/2880188053 shutdown_connections 2024-09-10T08:45:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.749+0000 7f57178ac740 1 -- 172.21.15.89:0/2880188053 wait complete. 2024-09-10T08:45:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.750+0000 7f57178ac740 1 -- start start 2024-09-10T08:45:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.750+0000 7f57178ac740 1 Processor -- start 2024-09-10T08:45:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.750+0000 7f57178ac740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332a090 0x563e133341f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.750+0000 7f57178ac740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1332a9d0 0x563e13334730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.750+0000 7f57178ac740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332fa60 0x563e1333b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.750+0000 7f57178ac740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563e13332a30 con 0x563e1332a090 2024-09-10T08:45:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.750+0000 7f57178ac740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563e13218090 con 0x563e1332fa60 2024-09-10T08:45:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.750+0000 7f57178ac740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563e13332bb0 con 0x563e1332a9d0 2024-09-10T08:45:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.751+0000 7f57178ac740 10 client.0 _ll_register_callbacks cb 0x563e1326cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.751+0000 7f570de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332a090 0x563e133341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.751+0000 7f5705400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1332a9d0 0x563e13334730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.751+0000 7f570de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332a090 0x563e133341f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57988/0 (socket says 172.21.15.89:57988) 2024-09-10T08:45:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.751+0000 7f570de00640 1 -- 172.21.15.89:0/3130369513 learned_addr learned my addr 172.21.15.89:0/3130369513 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.751+0000 7f570e800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332fa60 0x563e1333b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.751+0000 7f5705400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1332a9d0 0x563e13334730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:41332/0 (socket says 172.21.15.89:41332) 2024-09-10T08:45:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.751+0000 7f5705400640 1 -- 172.21.15.89:0/3130369513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332fa60 msgr2=0x563e1333b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.751+0000 7f5705400640 1 --2- 172.21.15.89:0/3130369513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332fa60 0x563e1333b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.751+0000 7f5705400640 1 -- 172.21.15.89:0/3130369513 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332a090 msgr2=0x563e133341f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.751+0000 7f5705400640 1 --2- 172.21.15.89:0/3130369513 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332a090 0x563e133341f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.751+0000 7f5705400640 1 -- 172.21.15.89:0/3130369513 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e1306e620 con 0x563e1332a9d0 2024-09-10T08:45:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.751+0000 7f5705400640 1 --2- 172.21.15.89:0/3130369513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1332a9d0 0x563e13334730 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f56f800c6a0 tx=0x7f56f809b4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29b132953a1b8c22 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.751+0000 7f56f3400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:45:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.752+0000 7f56f3400640 1 -- 172.21.15.89:0/3130369513 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f809ff00 con 0x563e1332a9d0 2024-09-10T08:45:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.752+0000 7f56f3400640 1 -- 172.21.15.89:0/3130369513 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f56f80d1be0 con 0x563e1332a9d0 2024-09-10T08:45:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.752+0000 7f57178ac740 1 -- 172.21.15.89:0/3130369513 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57080041d0 con 0x563e1332a9d0 2024-09-10T08:45:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.752+0000 7f56f3400640 1 -- 172.21.15.89:0/3130369513 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f56f80b3020 con 0x563e1332a9d0 2024-09-10T08:45:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.752+0000 7f57178ac740 10 client.26455 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.752+0000 7f57178ac740 1 -- 172.21.15.89:0/3130369513 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563e1333e5d0 con 0x563e1332a9d0 2024-09-10T08:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.752+0000 7f56f3400640 1 -- 172.21.15.89:0/3130369513 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57080041d0 con 0x563e1332a9d0 2024-09-10T08:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.753+0000 7f56f3400640 1 -- 172.21.15.89:0/3130369513 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f56f80cfb10 con 0x563e1332a9d0 2024-09-10T08:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.753+0000 7f57178ac740 10 client.26455 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.753+0000 7f57178ac740 10 client.26455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.753+0000 7f57178ac740 20 client.26455 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.753+0000 7f57178ac740 1 --2- 172.21.15.89:0/3130369513 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563e1333fc50 0x563e13342040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.753+0000 7f57178ac740 4 client.26455 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.753+0000 7f57178ac740 1 -- 172.21.15.89:0/3130369513 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e1329c830 con 0x563e1333fc50 2024-09-10T08:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.753+0000 7f570de00640 1 --2- 172.21.15.89:0/3130369513 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563e1333fc50 0x563e13342040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.754+0000 7f56f3400640 10 client.26455 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.754+0000 7f570de00640 1 --2- 172.21.15.89:0/3130369513 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563e1333fc50 0x563e13342040 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe1fae502b5f6b93 server_cookie=5a2825fd1b0d16db in_seq=0 out_seq=0 2024-09-10T08:45:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.756+0000 7f56f3400640 1 -- 172.21.15.89:0/3130369513 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e1329c830 con 0x563e1333fc50 2024-09-10T08:45:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.756+0000 7f56f3400640 10 client.26455 handle_command_reply: tid=0 2024-09-10T08:45:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.756+0000 7f57178ac740 1 client.26455 shutdown 2024-09-10T08:45:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.756+0000 7f57178ac740 1 -- 172.21.15.89:0/3130369513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1332a9d0 msgr2=0x563e13334730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.756+0000 7f57178ac740 1 --2- 172.21.15.89:0/3130369513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1332a9d0 0x563e13334730 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f56f800c6a0 tx=0x7f56f809b4b0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.756+0000 7f57178ac740 1 -- 172.21.15.89:0/3130369513 shutdown_connections 2024-09-10T08:45:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.756+0000 7f57178ac740 1 --2- 172.21.15.89:0/3130369513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332fa60 0x563e1333b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.756+0000 7f57178ac740 1 --2- 172.21.15.89:0/3130369513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1332a9d0 0x563e13334730 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.756+0000 7f57178ac740 1 --2- 172.21.15.89:0/3130369513 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563e1333fc50 0x563e13342040 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.756+0000 7f57178ac740 1 --2- 172.21.15.89:0/3130369513 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332a090 0x563e133341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.756+0000 7f57178ac740 1 -- 172.21.15.89:0/3130369513 >> 172.21.15.89:0/3130369513 conn(0x563e13325700 msgr2=0x563e13326f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.756+0000 7f56f3400640 0 client.26455 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.756+0000 7f57178ac740 1 -- 172.21.15.89:0/3130369513 shutdown_connections 2024-09-10T08:45:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.756+0000 7f57178ac740 1 -- 172.21.15.89:0/3130369513 wait complete. 2024-09-10T08:45:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.757+0000 7f57178ac740 20 client.26455 trim_cache size 0 max 16384 2024-09-10T08:45:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.774+0000 7f57178ac740 1 Processor -- start 2024-09-10T08:45:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.774+0000 7f57178ac740 1 -- start start 2024-09-10T08:45:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.774+0000 7f57178ac740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1332a090 0x563e133208d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.775+0000 7f57178ac740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332a9d0 0x563e1333b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.775+0000 7f57178ac740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332fa60 0x563e13324770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.775+0000 7f57178ac740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f57101af330 con 0x563e1332a9d0 2024-09-10T08:45:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.775+0000 7f57178ac740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f57101acc00 con 0x563e1332fa60 2024-09-10T08:45:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.775+0000 7f57178ac740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f57101aaa50 con 0x563e1332a090 2024-09-10T08:45:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.775+0000 7f570de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1332a090 0x563e133208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.775+0000 7f570de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1332a090 0x563e133208d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:41348/0 (socket says 172.21.15.89:41348) 2024-09-10T08:45:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.775+0000 7f570de00640 1 -- 172.21.15.89:0/2465774476 learned_addr learned my addr 172.21.15.89:0/2465774476 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.775+0000 7f5705400640 1 --2- 172.21.15.89:0/2465774476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332a9d0 0x563e1333b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.775+0000 7f570e800640 1 --2- 172.21.15.89:0/2465774476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332fa60 0x563e13324770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.775+0000 7f570de00640 1 -- 172.21.15.89:0/2465774476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332fa60 msgr2=0x563e13324770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.775+0000 7f570de00640 1 --2- 172.21.15.89:0/2465774476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332fa60 0x563e13324770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.775+0000 7f570de00640 1 -- 172.21.15.89:0/2465774476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332a9d0 msgr2=0x563e1333b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.775+0000 7f570de00640 1 --2- 172.21.15.89:0/2465774476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332a9d0 0x563e1333b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.775+0000 7f570de00640 1 -- 172.21.15.89:0/2465774476 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e133426c0 con 0x563e1332a090 2024-09-10T08:45:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.776+0000 7f5705400640 1 --2- 172.21.15.89:0/2465774476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332a9d0 0x563e1333b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.776+0000 7f570e800640 1 --2- 172.21.15.89:0/2465774476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332fa60 0x563e13324770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.776+0000 7f570de00640 1 --2- 172.21.15.89:0/2465774476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1332a090 0x563e133208d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5700000f80 tx=0x7f5700042000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8ddf92306aabe87 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.776+0000 7f56fe600640 1 -- 172.21.15.89:0/2465774476 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700003af0 con 0x563e1332a090 2024-09-10T08:45:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.776+0000 7f56fe600640 1 -- 172.21.15.89:0/2465774476 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5700036040 con 0x563e1332a090 2024-09-10T08:45:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.776+0000 7f56fe600640 1 -- 172.21.15.89:0/2465774476 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700055720 con 0x563e1332a090 2024-09-10T08:45:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.777+0000 7f57178ac740 1 -- 172.21.15.89:0/2465774476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1332a090 msgr2=0x563e133208d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.777+0000 7f57178ac740 1 --2- 172.21.15.89:0/2465774476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1332a090 0x563e133208d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5700000f80 tx=0x7f5700042000 comp rx=0 tx=0).stop 2024-09-10T08:45:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.777+0000 7f57178ac740 1 -- 172.21.15.89:0/2465774476 shutdown_connections 2024-09-10T08:45:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.777+0000 7f57178ac740 1 --2- 172.21.15.89:0/2465774476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332fa60 0x563e13324770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.777+0000 7f57178ac740 1 --2- 172.21.15.89:0/2465774476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332a9d0 0x563e1333b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.777+0000 7f57178ac740 1 --2- 172.21.15.89:0/2465774476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1332a090 0x563e133208d0 secure :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5700000f80 tx=0x7f5700042000 comp rx=0 tx=0).stop 2024-09-10T08:45:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.777+0000 7f57178ac740 1 -- 172.21.15.89:0/2465774476 >> 172.21.15.89:0/2465774476 conn(0x563e13325700 msgr2=0x563e13324060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.777+0000 7f57178ac740 1 -- 172.21.15.89:0/2465774476 shutdown_connections 2024-09-10T08:45:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.777+0000 7f57178ac740 1 -- 172.21.15.89:0/2465774476 wait complete. 2024-09-10T08:45:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.778+0000 7f57178ac740 1 -- start start 2024-09-10T08:45:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.778+0000 7f57178ac740 1 Processor -- start 2024-09-10T08:45:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.779+0000 7f57178ac740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332a9d0 0x563e13338ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.779+0000 7f57178ac740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332fa60 0x563e133390e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.779+0000 7f57178ac740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1333a0f0 0x563e1333a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.779+0000 7f57178ac740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f57101ab0c0 con 0x563e1332fa60 2024-09-10T08:45:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.779+0000 7f57178ac740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f57101ac530 con 0x563e1332a9d0 2024-09-10T08:45:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.779+0000 7f57178ac740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f57101ab790 con 0x563e1333a0f0 2024-09-10T08:45:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.779+0000 7f57178ac740 10 client.0 _ll_register_callbacks cb 0x7f56f800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.779+0000 7f570de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332a9d0 0x563e13338ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.779+0000 7f570de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332a9d0 0x563e13338ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:36146/0 (socket says 172.21.15.89:36146) 2024-09-10T08:45:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.779+0000 7f570de00640 1 -- 172.21.15.89:0/137845681 learned_addr learned my addr 172.21.15.89:0/137845681 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.779+0000 7f570e800640 1 --2- 172.21.15.89:0/137845681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1333a0f0 0x563e1333a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.779+0000 7f5705400640 1 --2- 172.21.15.89:0/137845681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332fa60 0x563e133390e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.779+0000 7f570de00640 1 -- 172.21.15.89:0/137845681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1333a0f0 msgr2=0x563e1333a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.779+0000 7f570de00640 1 --2- 172.21.15.89:0/137845681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1333a0f0 0x563e1333a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.779+0000 7f570de00640 1 -- 172.21.15.89:0/137845681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332fa60 msgr2=0x563e133390e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.779+0000 7f570de00640 1 --2- 172.21.15.89:0/137845681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332fa60 0x563e133390e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.779+0000 7f570de00640 1 -- 172.21.15.89:0/137845681 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e1333aae0 con 0x563e1332a9d0 2024-09-10T08:45:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.779+0000 7f570e800640 1 --2- 172.21.15.89:0/137845681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1333a0f0 0x563e1333a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.780+0000 7f570de00640 1 --2- 172.21.15.89:0/137845681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332a9d0 0x563e13338ba0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f570002aeb0 tx=0x7f57000316a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f0806eab3dcb2d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.780+0000 7f56f3400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:45:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.780+0000 7f56f3400640 1 -- 172.21.15.89:0/137845681 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700064020 con 0x563e1332a9d0 2024-09-10T08:45:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.780+0000 7f57178ac740 1 -- 172.21.15.89:0/137845681 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5700036040 con 0x563e1332a9d0 2024-09-10T08:45:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.780+0000 7f56f3400640 1 -- 172.21.15.89:0/137845681 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f5700055720 con 0x563e1332a9d0 2024-09-10T08:45:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.780+0000 7f56f3400640 1 -- 172.21.15.89:0/137845681 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5700055920 con 0x563e1332a9d0 2024-09-10T08:45:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.780+0000 7f57178ac740 10 client.26531 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.780+0000 7f57178ac740 1 -- 172.21.15.89:0/137845681 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563e133b3240 con 0x563e1332a9d0 2024-09-10T08:45:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.780+0000 7f56f3400640 1 -- 172.21.15.89:0/137845681 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5700036040 con 0x563e1332a9d0 2024-09-10T08:45:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.781+0000 7f56f3400640 1 -- 172.21.15.89:0/137845681 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f57000305c0 con 0x563e1332a9d0 2024-09-10T08:45:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.781+0000 7f57178ac740 10 client.26531 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.781+0000 7f57178ac740 10 client.26531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:45:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.781+0000 7f57178ac740 20 client.26531 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.781+0000 7f57178ac740 1 --2- 172.21.15.89:0/137845681 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563e133b3c70 0x563e133b6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.781+0000 7f57178ac740 4 client.26531 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:45:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.781+0000 7f57178ac740 1 -- 172.21.15.89:0/137845681 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e13296b80 con 0x563e133b3c70 2024-09-10T08:45:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.781+0000 7f5705400640 1 --2- 172.21.15.89:0/137845681 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563e133b3c70 0x563e133b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.782+0000 7f56f3400640 10 client.26531 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:19.784 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.782+0000 7f5705400640 1 --2- 172.21.15.89:0/137845681 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563e133b3c70 0x563e133b6090 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6deb62b6b98fa53f server_cookie=4f7cc90bd4f4999a in_seq=0 out_seq=0 2024-09-10T08:45:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.895+0000 7f56f3400640 1 -- 172.21.15.89:0/137845681 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563e13296b80 con 0x563e133b3c70 2024-09-10T08:45:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.895+0000 7f56f3400640 10 client.26531 handle_command_reply: tid=0 2024-09-10T08:45:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.895+0000 7f57178ac740 1 client.26531 shutdown 2024-09-10T08:45:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.896+0000 7f57178ac740 1 -- 172.21.15.89:0/137845681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332a9d0 msgr2=0x563e13338ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.896+0000 7f57178ac740 1 --2- 172.21.15.89:0/137845681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332a9d0 0x563e13338ba0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f570002aeb0 tx=0x7f57000316a0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.896+0000 7f57178ac740 1 -- 172.21.15.89:0/137845681 shutdown_connections 2024-09-10T08:45:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.896+0000 7f57178ac740 1 --2- 172.21.15.89:0/137845681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563e1333a0f0 0x563e1333a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.896+0000 7f57178ac740 1 --2- 172.21.15.89:0/137845681 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x563e133b3c70 0x563e133b6090 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.896+0000 7f57178ac740 1 --2- 172.21.15.89:0/137845681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563e1332fa60 0x563e133390e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.896+0000 7f57178ac740 1 --2- 172.21.15.89:0/137845681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e1332a9d0 0x563e13338ba0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.896+0000 7f56f3400640 0 client.26531 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.896+0000 7f57178ac740 1 -- 172.21.15.89:0/137845681 >> 172.21.15.89:0/137845681 conn(0x563e13325700 msgr2=0x563e13324880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.896+0000 7f57178ac740 1 -- 172.21.15.89:0/137845681 shutdown_connections 2024-09-10T08:45:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.896+0000 7f57178ac740 1 -- 172.21.15.89:0/137845681 wait complete. 2024-09-10T08:45:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.896+0000 7f57178ac740 20 client.26531 trim_cache size 0 max 16384 2024-09-10T08:45:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:45:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:45:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:45:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:45:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.897+0000 7f5714a00640 1 -- 172.21.15.89:0/286111385 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f56f40a2c30 msgr2=0x7f56f40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.897+0000 7f5714a00640 1 --2- 172.21.15.89:0/286111385 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f56f40a2c30 0x7f56f40a50f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f570002f540 tx=0x7f570002f390 comp rx=0 tx=0).stop 2024-09-10T08:45:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.897+0000 7f5714a00640 1 -- 172.21.15.89:0/286111385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5710102d50 msgr2=0x7f571019a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.897+0000 7f5714a00640 1 --2- 172.21.15.89:0/286111385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5710102d50 0x7f571019a550 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f56f800e9f0 tx=0x7f56f800edf0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.897+0000 7f5714a00640 1 -- 172.21.15.89:0/286111385 shutdown_connections 2024-09-10T08:45:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.897+0000 7f5714a00640 1 --2- 172.21.15.89:0/286111385 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f56f40a2c30 0x7f56f40a50f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.897+0000 7f5714a00640 1 --2- 172.21.15.89:0/286111385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5710103710 0x7f57101a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.897+0000 7f5714a00640 1 --2- 172.21.15.89:0/286111385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5710102d50 0x7f571019a550 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.897+0000 7f5714a00640 1 --2- 172.21.15.89:0/286111385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5710101b60 0x7f571019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.897+0000 7f5714a00640 1 -- 172.21.15.89:0/286111385 >> 172.21.15.89:0/286111385 conn(0x7f57100fd2e0 msgr2=0x7f5710074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.897+0000 7f5714a00640 1 -- 172.21.15.89:0/286111385 shutdown_connections 2024-09-10T08:45:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:19.897+0000 7f5714a00640 1 -- 172.21.15.89:0/286111385 wait complete. 2024-09-10T08:45:20.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:19 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:45:20.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:19 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:45:20.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:19 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:45:20.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:19 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:45:20.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:19 smithi149 ceph-mon[36167]: Adjusting osd_memory_target on smithi089 to 176.4M 2024-09-10T08:45:20.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:19 smithi149 ceph-mon[36167]: Unable to set osd_memory_target on smithi089 to 184978841: error parsing value: Value '184978841' is below minimum 939524096 2024-09-10T08:45:20.222 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:19 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:45:20.222 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:19 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:45:20.222 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:19 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:45:20.222 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:19 smithi149 ceph-mon[36167]: pgmap v555: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 360 KiB/s wr, 234 op/s 2024-09-10T08:45:20.282 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:19 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:45:20.282 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:19 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:45:20.282 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:19 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:45:20.282 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:19 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:45:20.282 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:19 smithi089 ceph-mon[30757]: Adjusting osd_memory_target on smithi089 to 176.4M 2024-09-10T08:45:20.282 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:19 smithi089 ceph-mon[30757]: Unable to set osd_memory_target on smithi089 to 184978841: error parsing value: Value '184978841' is below minimum 939524096 2024-09-10T08:45:20.282 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:19 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:45:20.282 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:19 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:45:20.283 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:19 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:45:20.283 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:19 smithi089 ceph-mon[30757]: pgmap v555: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 360 KiB/s wr, 234 op/s 2024-09-10T08:45:20.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:19 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:45:20.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:19 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:45:20.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:19 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:45:20.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:19 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:45:20.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:19 smithi111 ceph-mon[35220]: Adjusting osd_memory_target on smithi089 to 176.4M 2024-09-10T08:45:20.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:19 smithi111 ceph-mon[35220]: Unable to set osd_memory_target on smithi089 to 184978841: error parsing value: Value '184978841' is below minimum 939524096 2024-09-10T08:45:20.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:19 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:45:20.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:19 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:45:20.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:19 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:45:20.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:19 smithi111 ceph-mon[35220]: pgmap v555: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 360 KiB/s wr, 234 op/s 2024-09-10T08:45:20.323 INFO:tasks.cephfs.filesystem:scrub status for tag:586d0657-64b8-465e-b9a0-0767a07ac56b - {'path': '/', 'tag': '586d0657-64b8-465e-b9a0-0767a07ac56b', 'options': 'recursive,force'} 2024-09-10T08:45:23.032 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:45:23.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:22 smithi111 ceph-mon[35220]: pgmap v556: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 328 KiB/s wr, 573 op/s 2024-09-10T08:45:23.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:22 smithi089 ceph-mon[30757]: pgmap v556: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 328 KiB/s wr, 573 op/s 2024-09-10T08:45:23.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:22 smithi149 ceph-mon[36167]: pgmap v556: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 328 KiB/s wr, 573 op/s 2024-09-10T08:45:23.448 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:24.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.345+0000 7f1c1e200640 1 -- 172.21.15.89:0/3218957514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c18103ff0 msgr2=0x7f1c181043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:24.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.345+0000 7f1c1e200640 1 --2- 172.21.15.89:0/3218957514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c18103ff0 0x7f1c181043d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009900 tx=0x7f1c0802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:45:24.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.345+0000 7f1c1e200640 1 -- 172.21.15.89:0/3218957514 shutdown_connections 2024-09-10T08:45:24.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.345+0000 7f1c1e200640 1 --2- 172.21.15.89:0/3218957514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c18109120 0x7f1c18110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:24.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.345+0000 7f1c1e200640 1 --2- 172.21.15.89:0/3218957514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c181049a0 0x7f1c181089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:24.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.345+0000 7f1c1e200640 1 --2- 172.21.15.89:0/3218957514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c18103ff0 0x7f1c181043d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:24.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.345+0000 7f1c1e200640 1 -- 172.21.15.89:0/3218957514 >> 172.21.15.89:0/3218957514 conn(0x7f1c180ffa90 msgr2=0x7f1c18101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:24.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.346+0000 7f1c1e200640 1 -- 172.21.15.89:0/3218957514 shutdown_connections 2024-09-10T08:45:24.346 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.346+0000 7f1c1e200640 1 -- 172.21.15.89:0/3218957514 wait complete. 2024-09-10T08:45:24.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.347+0000 7f1c1e200640 1 Processor -- start 2024-09-10T08:45:24.347 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.347+0000 7f1c1e200640 1 -- start start 2024-09-10T08:45:24.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.348+0000 7f1c1e200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c18103ff0 0x7f1c1819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:24.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.348+0000 7f1c1e200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c181049a0 0x7f1c1819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:24.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.348+0000 7f1c1e200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c18109120 0x7f1c1819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:24.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.348+0000 7f1c1e200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1c18114120 con 0x7f1c181049a0 2024-09-10T08:45:24.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.348+0000 7f1c1e200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1c18113fa0 con 0x7f1c18109120 2024-09-10T08:45:24.348 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.348+0000 7f1c1e200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181142a0 con 0x7f1c18103ff0 2024-09-10T08:45:24.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.348+0000 7f1c17e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c18109120 0x7f1c1819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:24.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.348+0000 7f1c16a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c181049a0 0x7f1c1819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:24.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.348+0000 7f1c17e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c18109120 0x7f1c1819eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:53056/0 (socket says 172.21.15.89:53056) 2024-09-10T08:45:24.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.348+0000 7f1c17e00640 1 -- 172.21.15.89:0/496664040 learned_addr learned my addr 172.21.15.89:0/496664040 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:24.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.348+0000 7f1c17400640 1 --2- 172.21.15.89:0/496664040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c18103ff0 0x7f1c1819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:24.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.349+0000 7f1c17e00640 1 -- 172.21.15.89:0/496664040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c18103ff0 msgr2=0x7f1c1819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:24.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.349+0000 7f1c17e00640 1 --2- 172.21.15.89:0/496664040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c18103ff0 0x7f1c1819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:24.349 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.349+0000 7f1c17e00640 1 -- 172.21.15.89:0/496664040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c181049a0 msgr2=0x7f1c1819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:24.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.349+0000 7f1c17e00640 1 --2- 172.21.15.89:0/496664040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c181049a0 0x7f1c1819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:24.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.349+0000 7f1c17e00640 1 -- 172.21.15.89:0/496664040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1819f5b0 con 0x7f1c18109120 2024-09-10T08:45:24.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.349+0000 7f1c17e00640 1 --2- 172.21.15.89:0/496664040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c18109120 0x7f1c1819eed0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000efd0 tx=0x7f1c1000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:24.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.349+0000 7f1c0fe00640 1 -- 172.21.15.89:0/496664040 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10003b70 con 0x7f1c18109120 2024-09-10T08:45:24.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.349+0000 7f1c0fe00640 1 -- 172.21.15.89:0/496664040 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c10003d10 con 0x7f1c18109120 2024-09-10T08:45:24.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.349+0000 7f1c0fe00640 1 -- 172.21.15.89:0/496664040 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10010960 con 0x7f1c18109120 2024-09-10T08:45:24.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.349+0000 7f1c1e200640 1 -- 172.21.15.89:0/496664040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1819f8a0 con 0x7f1c18109120 2024-09-10T08:45:24.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.350+0000 7f1c1e200640 1 -- 172.21.15.89:0/496664040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c18076630 con 0x7f1c18109120 2024-09-10T08:45:24.351 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.351+0000 7f1c1e200640 1 -- 172.21.15.89:0/496664040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c181054d0 con 0x7f1c18109120 2024-09-10T08:45:24.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.353+0000 7f1c0fe00640 1 -- 172.21.15.89:0/496664040 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1c10004020 con 0x7f1c18109120 2024-09-10T08:45:24.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.354+0000 7f1c0fe00640 1 --2- 172.21.15.89:0/496664040 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1bfc0a2c80 0x7f1bfc0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:24.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.355+0000 7f1c0fe00640 1 -- 172.21.15.89:0/496664040 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f1c100cf990 con 0x7f1c18109120 2024-09-10T08:45:24.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.355+0000 7f1c17400640 1 --2- 172.21.15.89:0/496664040 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1bfc0a2c80 0x7f1bfc0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:24.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.356+0000 7f1c17400640 1 --2- 172.21.15.89:0/496664040 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1bfc0a2c80 0x7f1bfc0a5140 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009230 tx=0x7f1c080058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:24.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.358+0000 7f1c0fe00640 1 -- 172.21.15.89:0/496664040 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c1009c4c0 con 0x7f1c18109120 2024-09-10T08:45:24.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.608+0000 7f1c1e200640 1 -- 172.21.15.89:0/496664040 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c1810c300 con 0x7f1c18109120 2024-09-10T08:45:24.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.609+0000 7f1c0fe00640 1 -- 172.21.15.89:0/496664040 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11919 (secure 0 0 0) 0x7f1c1009bdc0 con 0x7f1c18109120 2024-09-10T08:45:24.609 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:45:24.609 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"btime":"2024-09-10T08:44:40:778845+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:44:40.174532+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24496,24518,24554]},"id":1}]} 2024-09-10T08:45:24.609 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 29 2024-09-10T08:45:24.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.612+0000 7f1c1e200640 1 -- 172.21.15.89:0/496664040 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1bfc0a2c80 msgr2=0x7f1bfc0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:24.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.612+0000 7f1c1e200640 1 --2- 172.21.15.89:0/496664040 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1bfc0a2c80 0x7f1bfc0a5140 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009230 tx=0x7f1c080058d0 comp rx=0 tx=0).stop 2024-09-10T08:45:24.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.612+0000 7f1c1e200640 1 -- 172.21.15.89:0/496664040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c18109120 msgr2=0x7f1c1819eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:24.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.612+0000 7f1c1e200640 1 --2- 172.21.15.89:0/496664040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c18109120 0x7f1c1819eed0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000efd0 tx=0x7f1c1000c6a0 comp rx=0 tx=0).stop 2024-09-10T08:45:24.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.612+0000 7f1c1e200640 1 -- 172.21.15.89:0/496664040 shutdown_connections 2024-09-10T08:45:24.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.612+0000 7f1c1e200640 1 --2- 172.21.15.89:0/496664040 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f1bfc0a2c80 0x7f1bfc0a5140 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:24.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.612+0000 7f1c1e200640 1 --2- 172.21.15.89:0/496664040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c18109120 0x7f1c1819eed0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:24.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.612+0000 7f1c1e200640 1 --2- 172.21.15.89:0/496664040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c181049a0 0x7f1c1819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:24.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.612+0000 7f1c1e200640 1 --2- 172.21.15.89:0/496664040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c18103ff0 0x7f1c1819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:24.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.612+0000 7f1c1e200640 1 -- 172.21.15.89:0/496664040 >> 172.21.15.89:0/496664040 conn(0x7f1c180ffa90 msgr2=0x7f1c18100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:24.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.612+0000 7f1c1e200640 1 -- 172.21.15.89:0/496664040 shutdown_connections 2024-09-10T08:45:24.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:24.612+0000 7f1c1e200640 1 -- 172.21.15.89:0/496664040 wait complete. 2024-09-10T08:45:25.030 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:24 smithi089 ceph-mon[30757]: pgmap v557: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 107 KiB/s wr, 552 op/s 2024-09-10T08:45:25.030 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:24 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/496664040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:25.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:24 smithi111 ceph-mon[35220]: pgmap v557: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 107 KiB/s wr, 552 op/s 2024-09-10T08:45:25.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:24 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/496664040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:25.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:24 smithi149 ceph-mon[36167]: pgmap v557: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 107 KiB/s wr, 552 op/s 2024-09-10T08:45:25.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:24 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/496664040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:25.606 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24554 quiesce db --set-id 2809171f --release --await' 2024-09-10T08:45:25.607 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24554 quiesce db --set-id 2809171f --release --await 2024-09-10T08:45:26.073 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:26.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.668+0000 7fd92d000640 1 -- 172.21.15.89:0/309448842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd928076720 msgr2=0x7fd928076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.668+0000 7fd92d000640 1 --2- 172.21.15.89:0/309448842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd928076720 0x7fd928076b00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd918009900 tx=0x7fd91802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.669+0000 7fd92d000640 1 -- 172.21.15.89:0/309448842 shutdown_connections 2024-09-10T08:45:26.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.669+0000 7fd92d000640 1 --2- 172.21.15.89:0/309448842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd928112a30 0x7fd928114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.669+0000 7fd92d000640 1 --2- 172.21.15.89:0/309448842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd928077040 0x7fd9281124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.669+0000 7fd92d000640 1 --2- 172.21.15.89:0/309448842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd928076720 0x7fd928076b00 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.669+0000 7fd92d000640 1 -- 172.21.15.89:0/309448842 >> 172.21.15.89:0/309448842 conn(0x7fd9280ffa90 msgr2=0x7fd928101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:26.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.669+0000 7fd92d000640 1 -- 172.21.15.89:0/309448842 shutdown_connections 2024-09-10T08:45:26.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.669+0000 7fd92d000640 1 -- 172.21.15.89:0/309448842 wait complete. 2024-09-10T08:45:26.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.670+0000 7fd92d000640 1 Processor -- start 2024-09-10T08:45:26.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.670+0000 7fd92d000640 1 -- start start 2024-09-10T08:45:26.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.670+0000 7fd92d000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd928076720 0x7fd9281a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.670+0000 7fd92d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd928077040 0x7fd9281a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.670+0000 7fd92d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd928112a30 0x7fd92819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.670+0000 7fd92d000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd9281185c0 con 0x7fd928076720 2024-09-10T08:45:26.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.670+0000 7fd92d000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd928118440 con 0x7fd928077040 2024-09-10T08:45:26.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.670+0000 7fd92d000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd928118740 con 0x7fd928112a30 2024-09-10T08:45:26.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.670+0000 7fd925e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd928076720 0x7fd9281a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.670+0000 7fd91fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd928077040 0x7fd9281a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.670+0000 7fd925e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd928076720 0x7fd9281a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57728/0 (socket says 172.21.15.89:57728) 2024-09-10T08:45:26.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.670+0000 7fd925e00640 1 -- 172.21.15.89:0/395702470 learned_addr learned my addr 172.21.15.89:0/395702470 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:26.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.671+0000 7fd926800640 1 --2- 172.21.15.89:0/395702470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd928112a30 0x7fd92819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.671+0000 7fd91fe00640 1 -- 172.21.15.89:0/395702470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd928112a30 msgr2=0x7fd92819e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.671+0000 7fd91fe00640 1 --2- 172.21.15.89:0/395702470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd928112a30 0x7fd92819e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.671+0000 7fd91fe00640 1 -- 172.21.15.89:0/395702470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd928076720 msgr2=0x7fd9281a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.671+0000 7fd91fe00640 1 --2- 172.21.15.89:0/395702470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd928076720 0x7fd9281a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.671+0000 7fd91fe00640 1 -- 172.21.15.89:0/395702470 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd92819efd0 con 0x7fd928077040 2024-09-10T08:45:26.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.671+0000 7fd926800640 1 --2- 172.21.15.89:0/395702470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd928112a30 0x7fd92819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:26.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.671+0000 7fd925e00640 1 --2- 172.21.15.89:0/395702470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd928076720 0x7fd9281a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:26.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.671+0000 7fd91fe00640 1 --2- 172.21.15.89:0/395702470 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd928077040 0x7fd9281a4c20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd91000e9f0 tx=0x7fd91000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:26.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.671+0000 7fd91ea00640 1 -- 172.21.15.89:0/395702470 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91000ccc0 con 0x7fd928077040 2024-09-10T08:45:26.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.672+0000 7fd92d000640 1 -- 172.21.15.89:0/395702470 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd92819f2c0 con 0x7fd928077040 2024-09-10T08:45:26.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.672+0000 7fd91ea00640 1 -- 172.21.15.89:0/395702470 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd910010910 con 0x7fd928077040 2024-09-10T08:45:26.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.672+0000 7fd91ea00640 1 -- 172.21.15.89:0/395702470 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd910018960 con 0x7fd928077040 2024-09-10T08:45:26.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.672+0000 7fd92d000640 1 -- 172.21.15.89:0/395702470 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9281ab5b0 con 0x7fd928077040 2024-09-10T08:45:26.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.675+0000 7fd91ea00640 1 -- 172.21.15.89:0/395702470 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd910010ac0 con 0x7fd928077040 2024-09-10T08:45:26.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.676+0000 7fd92fc6e740 1 Processor -- start 2024-09-10T08:45:26.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.677+0000 7fd92fc6e740 1 -- start start 2024-09-10T08:45:26.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.677+0000 7fd91ea00640 1 --2- 172.21.15.89:0/395702470 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd90c0a2b80 0x7fd90c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.677+0000 7fd91ea00640 1 -- 172.21.15.89:0/395702470 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fd9100cf8b0 con 0x7fd928077040 2024-09-10T08:45:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.677+0000 7fd92fc6e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c496310 0x55c67c4966f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.677+0000 7fd92fc6e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c496c30 0x55c67c4a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.677+0000 7fd92fc6e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c4a14f0 0x55c67c4a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.677+0000 7fd92fc6e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c67c3fc650 con 0x55c67c4a14f0 2024-09-10T08:45:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.677+0000 7fd92fc6e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c67c3a3f10 con 0x55c67c496c30 2024-09-10T08:45:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.677+0000 7fd92fc6e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c67c3f8610 con 0x55c67c496310 2024-09-10T08:45:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.677+0000 7fd91fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c496c30 0x55c67c4a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.677+0000 7fd925e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c496310 0x55c67c4966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.677+0000 7fd91fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c496c30 0x55c67c4a0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:53088/0 (socket says 172.21.15.89:53088) 2024-09-10T08:45:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.677+0000 7fd91fe00640 1 -- 172.21.15.89:0/4290564073 learned_addr learned my addr 172.21.15.89:0/4290564073 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.677+0000 7fd925e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c496310 0x55c67c4966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:35282/0 (socket says 172.21.15.89:35282) 2024-09-10T08:45:26.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.678+0000 7fd926800640 1 --2- 172.21.15.89:0/395702470 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd90c0a2b80 0x7fd90c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.678+0000 7fd91fe00640 1 -- 172.21.15.89:0/4290564073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c496310 msgr2=0x55c67c4966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.678+0000 7fd91fe00640 1 --2- 172.21.15.89:0/4290564073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c496310 0x55c67c4966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.678+0000 7fd91fe00640 1 -- 172.21.15.89:0/4290564073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c4a14f0 msgr2=0x55c67c4a38e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:45:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.678+0000 7fd91fe00640 1 --2- 172.21.15.89:0/4290564073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c4a14f0 0x55c67c4a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.678+0000 7fd91fe00640 1 -- 172.21.15.89:0/4290564073 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c67c10b580 con 0x55c67c496c30 2024-09-10T08:45:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.678+0000 7fd91fe00640 1 --2- 172.21.15.89:0/4290564073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c496c30 0x55c67c4a0fb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd9100986b0 tx=0x7fd91009c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a01bcdf80272231c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.679+0000 7fd926800640 1 --2- 172.21.15.89:0/395702470 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd90c0a2b80 0x7fd90c0a5040 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55c67c493c20 tx=0x7fd920009ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.679+0000 7fd917400640 1 -- 172.21.15.89:0/4290564073 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9100a0090 con 0x55c67c496c30 2024-09-10T08:45:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.679+0000 7fd917400640 1 -- 172.21.15.89:0/4290564073 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9100a4950 con 0x55c67c496c30 2024-09-10T08:45:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.679+0000 7fd917400640 1 -- 172.21.15.89:0/4290564073 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd910018d60 con 0x55c67c496c30 2024-09-10T08:45:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.680+0000 7fd92fc6e740 1 -- 172.21.15.89:0/4290564073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c496c30 msgr2=0x55c67c4a0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.680+0000 7fd92fc6e740 1 --2- 172.21.15.89:0/4290564073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c496c30 0x55c67c4a0fb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd9100986b0 tx=0x7fd91009c3e0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.680+0000 7fd92fc6e740 1 -- 172.21.15.89:0/4290564073 shutdown_connections 2024-09-10T08:45:26.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.680+0000 7fd92fc6e740 1 --2- 172.21.15.89:0/4290564073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c4a14f0 0x55c67c4a38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.680+0000 7fd92fc6e740 1 --2- 172.21.15.89:0/4290564073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c496c30 0x55c67c4a0fb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.680+0000 7fd92fc6e740 1 --2- 172.21.15.89:0/4290564073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c496310 0x55c67c4966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.680+0000 7fd92fc6e740 1 -- 172.21.15.89:0/4290564073 >> 172.21.15.89:0/4290564073 conn(0x55c67c4928a0 msgr2=0x55c67c492c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:26.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.680+0000 7fd92fc6e740 1 -- 172.21.15.89:0/4290564073 shutdown_connections 2024-09-10T08:45:26.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.680+0000 7fd92fc6e740 1 -- 172.21.15.89:0/4290564073 wait complete. 2024-09-10T08:45:26.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.681+0000 7fd92fc6e740 1 -- start start 2024-09-10T08:45:26.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.681+0000 7fd92fc6e740 1 Processor -- start 2024-09-10T08:45:26.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.682+0000 7fd92fc6e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c496310 0x55c67c49e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.682+0000 7fd92fc6e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c496c30 0x55c67c49ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.682+0000 7fd92fc6e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c4a14f0 0x55c67c49ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.682+0000 7fd92fc6e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c67c4a40c0 con 0x55c67c4a14f0 2024-09-10T08:45:26.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.682+0000 7fd92fc6e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c67c3b7350 con 0x55c67c496310 2024-09-10T08:45:26.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.682+0000 7fd92fc6e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c67c4a4240 con 0x55c67c496c30 2024-09-10T08:45:26.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.682+0000 7fd92fc6e740 10 client.0 _ll_register_callbacks cb 0x55c67c3da290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:26.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.682+0000 7fd925e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c496310 0x55c67c49e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.682+0000 7fd925e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c496310 0x55c67c49e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:53092/0 (socket says 172.21.15.89:53092) 2024-09-10T08:45:26.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.682+0000 7fd925e00640 1 -- 172.21.15.89:0/1477283883 learned_addr learned my addr 172.21.15.89:0/1477283883 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:26.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.682+0000 7fd91fe00640 1 --2- 172.21.15.89:0/1477283883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c496c30 0x55c67c49ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.682+0000 7fd926800640 1 --2- 172.21.15.89:0/1477283883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c4a14f0 0x55c67c49ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.683+0000 7fd925e00640 1 -- 172.21.15.89:0/1477283883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c496c30 msgr2=0x55c67c49ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:26.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.683+0000 7fd925e00640 1 --2- 172.21.15.89:0/1477283883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c496c30 0x55c67c49ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.683+0000 7fd925e00640 1 -- 172.21.15.89:0/1477283883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c4a14f0 msgr2=0x55c67c49ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:26.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.683+0000 7fd925e00640 1 --2- 172.21.15.89:0/1477283883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c4a14f0 0x55c67c49ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.683+0000 7fd925e00640 1 -- 172.21.15.89:0/1477283883 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c67c1d5960 con 0x55c67c496310 2024-09-10T08:45:26.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.683+0000 7fd926800640 1 --2- 172.21.15.89:0/1477283883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c4a14f0 0x55c67c49ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:26.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.683+0000 7fd91fe00640 1 --2- 172.21.15.89:0/1477283883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c496c30 0x55c67c49ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:26.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.683+0000 7fd925e00640 1 --2- 172.21.15.89:0/1477283883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c496310 0x55c67c49e500 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd918009a20 tx=0x7fd918031920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1709e5353926fb40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:26.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.683+0000 7fd90be00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:45:26.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.683+0000 7fd90be00640 1 -- 172.21.15.89:0/1477283883 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd918035040 con 0x55c67c496310 2024-09-10T08:45:26.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.683+0000 7fd92fc6e740 1 -- 172.21.15.89:0/1477283883 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9100a4950 con 0x55c67c496310 2024-09-10T08:45:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.683+0000 7fd90be00640 1 -- 172.21.15.89:0/1477283883 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fd918005030 con 0x55c67c496310 2024-09-10T08:45:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.684+0000 7fd90be00640 1 -- 172.21.15.89:0/1477283883 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd91804e050 con 0x55c67c496310 2024-09-10T08:45:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.684+0000 7fd92fc6e740 10 client.26549 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.684+0000 7fd92fc6e740 1 -- 172.21.15.89:0/1477283883 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c67c4afe40 con 0x55c67c496310 2024-09-10T08:45:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.684+0000 7fd90be00640 1 -- 172.21.15.89:0/1477283883 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9100a4950 con 0x55c67c496310 2024-09-10T08:45:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.684+0000 7fd90be00640 1 -- 172.21.15.89:0/1477283883 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fd9180035f0 con 0x55c67c496310 2024-09-10T08:45:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.684+0000 7fd92fc6e740 10 client.26549 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.684+0000 7fd92fc6e740 10 client.26549 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:45:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.684+0000 7fd92fc6e740 20 client.26549 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.684+0000 7fd92fc6e740 1 --2- 172.21.15.89:0/1477283883 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c67c4b1250 0x55c67c4b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.685+0000 7fd92fc6e740 4 client.26549 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:45:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.685+0000 7fd92fc6e740 1 -- 172.21.15.89:0/1477283883 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c67c409bc0 con 0x55c67c4b1250 2024-09-10T08:45:26.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.685+0000 7fd91fe00640 1 --2- 172.21.15.89:0/1477283883 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c67c4b1250 0x55c67c4b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.685+0000 7fd90be00640 10 client.26549 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:26.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.685+0000 7fd91fe00640 1 --2- 172.21.15.89:0/1477283883 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c67c4b1250 0x55c67c4b36a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e15850e354f4234 server_cookie=6ced7b6382cb4d3d in_seq=0 out_seq=0 2024-09-10T08:45:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.686+0000 7fd90be00640 1 -- 172.21.15.89:0/1477283883 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c67c409bc0 con 0x55c67c4b1250 2024-09-10T08:45:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.686+0000 7fd90be00640 10 client.26549 handle_command_reply: tid=0 2024-09-10T08:45:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.686+0000 7fd92fc6e740 1 client.26549 shutdown 2024-09-10T08:45:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.686+0000 7fd92fc6e740 1 -- 172.21.15.89:0/1477283883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c496310 msgr2=0x55c67c49e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.687+0000 7fd92fc6e740 1 --2- 172.21.15.89:0/1477283883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c496310 0x55c67c49e500 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd918009a20 tx=0x7fd918031920 comp rx=0 tx=0).stop 2024-09-10T08:45:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.687+0000 7fd92fc6e740 1 -- 172.21.15.89:0/1477283883 shutdown_connections 2024-09-10T08:45:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.687+0000 7fd92fc6e740 1 --2- 172.21.15.89:0/1477283883 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c67c4b1250 0x55c67c4b36a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.687+0000 7fd92fc6e740 1 --2- 172.21.15.89:0/1477283883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c4a14f0 0x55c67c49ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.687+0000 7fd92fc6e740 1 --2- 172.21.15.89:0/1477283883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c496c30 0x55c67c49ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.687+0000 7fd92fc6e740 1 --2- 172.21.15.89:0/1477283883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c496310 0x55c67c49e500 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.687+0000 7fd92fc6e740 1 -- 172.21.15.89:0/1477283883 >> 172.21.15.89:0/1477283883 conn(0x55c67c4928a0 msgr2=0x55c67c491aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:26.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.687+0000 7fd90be00640 0 client.26549 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:26.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.687+0000 7fd92fc6e740 1 -- 172.21.15.89:0/1477283883 shutdown_connections 2024-09-10T08:45:26.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.687+0000 7fd92fc6e740 1 -- 172.21.15.89:0/1477283883 wait complete. 2024-09-10T08:45:26.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.687+0000 7fd92fc6e740 20 client.26549 trim_cache size 0 max 16384 2024-09-10T08:45:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.705+0000 7fd92fc6e740 1 Processor -- start 2024-09-10T08:45:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.705+0000 7fd92fc6e740 1 -- start start 2024-09-10T08:45:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.705+0000 7fd92fc6e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c496310 0x55c67c49edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.705+0000 7fd92fc6e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c496c30 0x55c67c498470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.705+0000 7fd92fc6e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c4a14f0 0x55c67c4b8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.705+0000 7fd92fc6e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd9281b7d20 con 0x55c67c496c30 2024-09-10T08:45:26.705 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.705+0000 7fd92fc6e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd9281b55f0 con 0x55c67c4a14f0 2024-09-10T08:45:26.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.705+0000 7fd92fc6e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd9281b3440 con 0x55c67c496310 2024-09-10T08:45:26.706 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.706+0000 7fd91fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c496c30 0x55c67c498470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.706+0000 7fd925e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c496310 0x55c67c49edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.706+0000 7fd925e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c496310 0x55c67c49edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:35298/0 (socket says 172.21.15.89:35298) 2024-09-10T08:45:26.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.706+0000 7fd925e00640 1 -- 172.21.15.89:0/1737894652 learned_addr learned my addr 172.21.15.89:0/1737894652 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:26.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.706+0000 7fd926800640 1 --2- 172.21.15.89:0/1737894652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c4a14f0 0x55c67c4b8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.706+0000 7fd925e00640 1 -- 172.21.15.89:0/1737894652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c4a14f0 msgr2=0x55c67c4b8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:26.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.706+0000 7fd925e00640 1 --2- 172.21.15.89:0/1737894652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c4a14f0 0x55c67c4b8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.706+0000 7fd925e00640 1 -- 172.21.15.89:0/1737894652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c496c30 msgr2=0x55c67c498470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:26.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.706+0000 7fd925e00640 1 --2- 172.21.15.89:0/1737894652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c496c30 0x55c67c498470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.706+0000 7fd925e00640 1 -- 172.21.15.89:0/1737894652 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c67c493530 con 0x55c67c496310 2024-09-10T08:45:26.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.706+0000 7fd926800640 1 --2- 172.21.15.89:0/1737894652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c4a14f0 0x55c67c4b8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:26.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.706+0000 7fd925e00640 1 --2- 172.21.15.89:0/1737894652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c496310 0x55c67c49edd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd918005770 tx=0x7fd918031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=393a7429f457d024 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:26.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.707+0000 7fd916a00640 1 -- 172.21.15.89:0/1737894652 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd918057070 con 0x55c67c496310 2024-09-10T08:45:26.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.707+0000 7fd916a00640 1 -- 172.21.15.89:0/1737894652 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd918041070 con 0x55c67c496310 2024-09-10T08:45:26.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.707+0000 7fd916a00640 1 -- 172.21.15.89:0/1737894652 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd918035030 con 0x55c67c496310 2024-09-10T08:45:26.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.707+0000 7fd92fc6e740 1 -- 172.21.15.89:0/1737894652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c496310 msgr2=0x55c67c49edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.707+0000 7fd92fc6e740 1 --2- 172.21.15.89:0/1737894652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c496310 0x55c67c49edd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd918005770 tx=0x7fd918031400 comp rx=0 tx=0).stop 2024-09-10T08:45:26.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.708+0000 7fd92fc6e740 1 -- 172.21.15.89:0/1737894652 shutdown_connections 2024-09-10T08:45:26.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.708+0000 7fd92fc6e740 1 --2- 172.21.15.89:0/1737894652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c4a14f0 0x55c67c4b8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.708+0000 7fd92fc6e740 1 --2- 172.21.15.89:0/1737894652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c496c30 0x55c67c498470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.708+0000 7fd92fc6e740 1 --2- 172.21.15.89:0/1737894652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c496310 0x55c67c49edd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.708+0000 7fd92fc6e740 1 -- 172.21.15.89:0/1737894652 >> 172.21.15.89:0/1737894652 conn(0x55c67c4928a0 msgr2=0x55c67c497d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:26.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.708+0000 7fd92fc6e740 1 -- 172.21.15.89:0/1737894652 shutdown_connections 2024-09-10T08:45:26.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.708+0000 7fd92fc6e740 1 -- 172.21.15.89:0/1737894652 wait complete. 2024-09-10T08:45:26.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.708+0000 7fd92fc6e740 1 -- start start 2024-09-10T08:45:26.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.708+0000 7fd92fc6e740 1 Processor -- start 2024-09-10T08:45:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.709+0000 7fd92fc6e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c496310 0x55c67c4a5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.709+0000 7fd92fc6e740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c496c30 0x55c67c4a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.709+0000 7fd92fc6e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c4a14f0 0x55c67c51af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.709+0000 7fd92fc6e740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd9281b3ab0 con 0x55c67c496c30 2024-09-10T08:45:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.709+0000 7fd92fc6e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd9281b4f20 con 0x55c67c496310 2024-09-10T08:45:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.709+0000 7fd92fc6e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd9281b4180 con 0x55c67c4a14f0 2024-09-10T08:45:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.709+0000 7fd92fc6e740 10 client.0 _ll_register_callbacks cb 0x7fd918009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.709+0000 7fd926800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c4a14f0 0x55c67c51af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.709+0000 7fd91fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c496c30 0x55c67c4a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.709+0000 7fd926800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c4a14f0 0x55c67c51af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:35312/0 (socket says 172.21.15.89:35312) 2024-09-10T08:45:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.709+0000 7fd926800640 1 -- 172.21.15.89:0/4148623196 learned_addr learned my addr 172.21.15.89:0/4148623196 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:26.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.710+0000 7fd91fe00640 1 -- 172.21.15.89:0/4148623196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c4a14f0 msgr2=0x55c67c51af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:26.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.710+0000 7fd91fe00640 1 --2- 172.21.15.89:0/4148623196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c4a14f0 0x55c67c51af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.710+0000 7fd91fe00640 1 -- 172.21.15.89:0/4148623196 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c496310 msgr2=0x55c67c4a5980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:45:26.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.710+0000 7fd91fe00640 1 --2- 172.21.15.89:0/4148623196 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c496310 0x55c67c4a5980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.710+0000 7fd91fe00640 1 -- 172.21.15.89:0/4148623196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c67c51b530 con 0x55c67c496c30 2024-09-10T08:45:26.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.710+0000 7fd91fe00640 1 --2- 172.21.15.89:0/4148623196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c496c30 0x55c67c4a5ec0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd91009bc70 tx=0x7fd910009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc92845c3c190afb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:26.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.710+0000 7fd90be00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:45:26.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.710+0000 7fd90be00640 1 -- 172.21.15.89:0/4148623196 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9100a4950 con 0x55c67c496c30 2024-09-10T08:45:26.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.710+0000 7fd90be00640 1 -- 172.21.15.89:0/4148623196 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fd91009b130 con 0x55c67c496c30 2024-09-10T08:45:26.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.710+0000 7fd92fc6e740 1 -- 172.21.15.89:0/4148623196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd918041070 con 0x55c67c496c30 2024-09-10T08:45:26.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.711+0000 7fd90be00640 1 -- 172.21.15.89:0/4148623196 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd9100a0090 con 0x55c67c496c30 2024-09-10T08:45:26.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.711+0000 7fd92fc6e740 10 client.16512 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:26.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.711+0000 7fd92fc6e740 1 -- 172.21.15.89:0/4148623196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c67c51c810 con 0x55c67c496c30 2024-09-10T08:45:26.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.711+0000 7fd90be00640 1 -- 172.21.15.89:0/4148623196 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd918041070 con 0x55c67c496c30 2024-09-10T08:45:26.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.711+0000 7fd90be00640 1 -- 172.21.15.89:0/4148623196 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fd910097b30 con 0x55c67c496c30 2024-09-10T08:45:26.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.711+0000 7fd92fc6e740 10 client.16512 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:26.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.711+0000 7fd92fc6e740 10 client.16512 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:45:26.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.711+0000 7fd92fc6e740 20 client.16512 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:26.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.711+0000 7fd92fc6e740 1 --2- 172.21.15.89:0/4148623196 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c67c51caf0 0x55c67c51ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.711+0000 7fd92fc6e740 4 client.16512 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2809171f", "release": true, "await": true}] 2024-09-10T08:45:26.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.711+0000 7fd92fc6e740 1 -- 172.21.15.89:0/4148623196 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2809171f", "release": true, "await": true}) -- 0x55c67c4040d0 con 0x55c67c51caf0 2024-09-10T08:45:26.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.712+0000 7fd925e00640 1 --2- 172.21.15.89:0/4148623196 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c67c51caf0 0x55c67c51ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.712+0000 7fd925e00640 1 --2- 172.21.15.89:0/4148623196 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c67c51caf0 0x55c67c51ef10 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de9452663d0db9ee server_cookie=ec6e8c99e9eb372c in_seq=0 out_seq=0 2024-09-10T08:45:26.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:26.712+0000 7fd90be00640 10 client.16512 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:27.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:26 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:27.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:26 smithi089 ceph-mon[30757]: pgmap v558: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 110 KiB/s wr, 974 op/s 2024-09-10T08:45:27.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:26 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:27.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:26 smithi149 ceph-mon[36167]: pgmap v558: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 110 KiB/s wr, 974 op/s 2024-09-10T08:45:27.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:26 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:45:27.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:26 smithi111 ceph-mon[35220]: pgmap v558: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 110 KiB/s wr, 974 op/s 2024-09-10T08:45:27.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.378+0000 7fd90be00640 1 -- 172.21.15.89:0/4148623196 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c67c4040d0 con 0x55c67c51caf0 2024-09-10T08:45:27.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.378+0000 7fd90be00640 10 client.16512 handle_command_reply: tid=0 2024-09-10T08:45:27.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.378+0000 7fd92fc6e740 1 client.16512 shutdown 2024-09-10T08:45:27.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.379+0000 7fd92fc6e740 1 -- 172.21.15.89:0/4148623196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c496c30 msgr2=0x55c67c4a5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:27.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.379+0000 7fd92fc6e740 1 --2- 172.21.15.89:0/4148623196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c496c30 0x55c67c4a5ec0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd91009bc70 tx=0x7fd910009700 comp rx=0 tx=0).stop 2024-09-10T08:45:27.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.379+0000 7fd92fc6e740 1 -- 172.21.15.89:0/4148623196 shutdown_connections 2024-09-10T08:45:27.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.379+0000 7fd92fc6e740 1 --2- 172.21.15.89:0/4148623196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c67c4a14f0 0x55c67c51af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:27.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.379+0000 7fd92fc6e740 1 --2- 172.21.15.89:0/4148623196 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c67c51caf0 0x55c67c51ef10 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:27.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.379+0000 7fd92fc6e740 1 --2- 172.21.15.89:0/4148623196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c67c496c30 0x55c67c4a5ec0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:27.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.379+0000 7fd92fc6e740 1 --2- 172.21.15.89:0/4148623196 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c67c496310 0x55c67c4a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:27.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.379+0000 7fd92fc6e740 1 -- 172.21.15.89:0/4148623196 >> 172.21.15.89:0/4148623196 conn(0x55c67c4928a0 msgr2=0x55c67c491aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:27.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.379+0000 7fd90be00640 0 client.16512 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:27.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.379+0000 7fd92fc6e740 1 -- 172.21.15.89:0/4148623196 shutdown_connections 2024-09-10T08:45:27.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.379+0000 7fd92fc6e740 1 -- 172.21.15.89:0/4148623196 wait complete. 2024-09-10T08:45:27.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.379+0000 7fd92fc6e740 20 client.16512 trim_cache size 0 max 16384 2024-09-10T08:45:27.380 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:45:27.380 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 29, 2024-09-10T08:45:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24554, 2024-09-10T08:45:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 16, 2024-09-10T08:45:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": { 2024-09-10T08:45:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "2809171f": { 2024-09-10T08:45:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "version": 16, 2024-09-10T08:45:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "age_ref": 0.0, 2024-09-10T08:45:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:45:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RELEASED", 2024-09-10T08:45:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:45:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:45:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "timeout": 90.0, 2024-09-10T08:45:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "expiration": 142.8, 2024-09-10T08:45:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "members": { 2024-09-10T08:45:27.381 INFO:teuthology.orchestra.run.smithi089.stdout: "file:/": { 2024-09-10T08:45:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "excluded": false, 2024-09-10T08:45:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:45:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RELEASED", 2024-09-10T08:45:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:45:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:45:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:45:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:45:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:45:27.382 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:45:27.382 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:45:27.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.380+0000 7fd92d000640 1 -- 172.21.15.89:0/395702470 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd90c0a2b80 msgr2=0x7fd90c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:27.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.380+0000 7fd92d000640 1 --2- 172.21.15.89:0/395702470 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd90c0a2b80 0x7fd90c0a5040 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55c67c493c20 tx=0x7fd920009ab0 comp rx=0 tx=0).stop 2024-09-10T08:45:27.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.380+0000 7fd92d000640 1 -- 172.21.15.89:0/395702470 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd928077040 msgr2=0x7fd9281a4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:27.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.380+0000 7fd92d000640 1 --2- 172.21.15.89:0/395702470 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd928077040 0x7fd9281a4c20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd91000e9f0 tx=0x7fd91000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:45:27.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.380+0000 7fd92d000640 1 -- 172.21.15.89:0/395702470 shutdown_connections 2024-09-10T08:45:27.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.380+0000 7fd92d000640 1 --2- 172.21.15.89:0/395702470 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd90c0a2b80 0x7fd90c0a5040 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:27.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.380+0000 7fd92d000640 1 --2- 172.21.15.89:0/395702470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd928112a30 0x7fd92819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:27.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.380+0000 7fd92d000640 1 --2- 172.21.15.89:0/395702470 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd928077040 0x7fd9281a4c20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:27.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.380+0000 7fd92d000640 1 --2- 172.21.15.89:0/395702470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd928076720 0x7fd9281a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:27.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.380+0000 7fd92d000640 1 -- 172.21.15.89:0/395702470 >> 172.21.15.89:0/395702470 conn(0x7fd9280ffa90 msgr2=0x7fd928101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:27.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.380+0000 7fd92d000640 1 -- 172.21.15.89:0/395702470 shutdown_connections 2024-09-10T08:45:27.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:27.380+0000 7fd92d000640 1 -- 172.21.15.89:0/395702470 wait complete. 2024-09-10T08:45:28.042 INFO:tasks.workunit.client.0.smithi089.stdout: 89 142630 20 97617 7 77252 64 2024-09-10T08:45:28.263 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2809171f', total seconds elapsed: 28.0 2024-09-10T08:45:29.095 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:28 smithi111 ceph-mon[35220]: pgmap v559: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 38 KiB/s wr, 1.12k op/s 2024-09-10T08:45:29.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:28 smithi149 ceph-mon[36167]: pgmap v559: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 38 KiB/s wr, 1.12k op/s 2024-09-10T08:45:29.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:28 smithi089 ceph-mon[30757]: pgmap v559: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 38 KiB/s wr, 1.12k op/s 2024-09-10T08:45:31.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:30 smithi149 ceph-mon[36167]: pgmap v560: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 38 KiB/s wr, 1.14k op/s 2024-09-10T08:45:31.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:30 smithi149 ceph-mon[36167]: scrub summary: idle 2024-09-10T08:45:31.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:30 smithi149 ceph-mon[36167]: scrub summary: idle 2024-09-10T08:45:31.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:30 smithi111 ceph-mon[35220]: pgmap v560: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 38 KiB/s wr, 1.14k op/s 2024-09-10T08:45:31.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:30 smithi111 ceph-mon[35220]: scrub summary: idle 2024-09-10T08:45:31.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:30 smithi111 ceph-mon[35220]: scrub summary: idle 2024-09-10T08:45:31.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:30 smithi089 ceph-mon[30757]: pgmap v560: 129 pgs: 129 active+clean; 302 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 38 KiB/s wr, 1.14k op/s 2024-09-10T08:45:31.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:30 smithi089 ceph-mon[30757]: scrub summary: idle 2024-09-10T08:45:31.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:30 smithi089 ceph-mon[30757]: scrub summary: idle 2024-09-10T08:45:32.204 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:31 smithi089 ceph-mon[30757]: scrub summary: idle 2024-09-10T08:45:32.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:31 smithi149 ceph-mon[36167]: scrub summary: idle 2024-09-10T08:45:32.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:31 smithi111 ceph-mon[35220]: scrub summary: idle 2024-09-10T08:45:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.082+0000 7ffb7be00640 1 -- 172.21.15.89:0/2451156703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb7c1023f0 msgr2=0x7ffb7c108c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.082+0000 7ffb7be00640 1 --2- 172.21.15.89:0/2451156703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb7c1023f0 0x7ffb7c108c80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009be0 tx=0x7ffb7402a6f0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.082+0000 7ffb7be00640 1 -- 172.21.15.89:0/2451156703 shutdown_connections 2024-09-10T08:45:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.082+0000 7ffb7be00640 1 --2- 172.21.15.89:0/2451156703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb7c1023f0 0x7ffb7c108c80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.082+0000 7ffb7be00640 1 --2- 172.21.15.89:0/2451156703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb7c101a30 0x7ffb7c101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.082+0000 7ffb7be00640 1 --2- 172.21.15.89:0/2451156703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb7c100840 0x7ffb7c100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.082+0000 7ffb7be00640 1 -- 172.21.15.89:0/2451156703 >> 172.21.15.89:0/2451156703 conn(0x7ffb7c0fbfa0 msgr2=0x7ffb7c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.082+0000 7ffb7be00640 1 -- 172.21.15.89:0/2451156703 shutdown_connections 2024-09-10T08:45:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.082+0000 7ffb7be00640 1 -- 172.21.15.89:0/2451156703 wait complete. 2024-09-10T08:45:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.083+0000 7ffb7be00640 1 Processor -- start 2024-09-10T08:45:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.084+0000 7ffb7be00640 1 -- start start 2024-09-10T08:45:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.084+0000 7ffb7be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb7c100840 0x7ffb7c199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.084+0000 7ffb7be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb7c101a30 0x7ffb7c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.084+0000 7ffb7be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb7c1023f0 0x7ffb7c1a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.084+0000 7ffb7be00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c10c540 con 0x7ffb7c100840 2024-09-10T08:45:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.084+0000 7ffb7be00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c10c3c0 con 0x7ffb7c101a30 2024-09-10T08:45:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.084+0000 7ffb7be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c10c6c0 con 0x7ffb7c1023f0 2024-09-10T08:45:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.085+0000 7ffb7aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb7c100840 0x7ffb7c199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.085+0000 7ffb7b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb7c1023f0 0x7ffb7c1a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.085+0000 7ffb7b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb7c1023f0 0x7ffb7c1a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:40224/0 (socket says 172.21.15.89:40224) 2024-09-10T08:45:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.085+0000 7ffb7aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb7c100840 0x7ffb7c199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49750/0 (socket says 172.21.15.89:49750) 2024-09-10T08:45:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.085+0000 7ffb7b400640 1 -- 172.21.15.89:0/19696352 learned_addr learned my addr 172.21.15.89:0/19696352 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.085+0000 7ffb7aa00640 1 -- 172.21.15.89:0/19696352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb7c1023f0 msgr2=0x7ffb7c1a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.085+0000 7ffb7aa00640 1 --2- 172.21.15.89:0/19696352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb7c1023f0 0x7ffb7c1a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.085+0000 7ffb7aa00640 1 -- 172.21.15.89:0/19696352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb7c101a30 msgr2=0x7ffb7c19a460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:45:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.085+0000 7ffb7aa00640 1 --2- 172.21.15.89:0/19696352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb7c101a30 0x7ffb7c19a460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.085+0000 7ffb7aa00640 1 -- 172.21.15.89:0/19696352 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7c1a1b40 con 0x7ffb7c100840 2024-09-10T08:45:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.085+0000 7ffb7b400640 1 --2- 172.21.15.89:0/19696352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb7c1023f0 0x7ffb7c1a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.085+0000 7ffb7aa00640 1 --2- 172.21.15.89:0/19696352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb7c100840 0x7ffb7c199f20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00b3c0 tx=0x7ffb6c00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.086+0000 7ffb73400640 1 -- 172.21.15.89:0/19696352 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c011830 con 0x7ffb7c100840 2024-09-10T08:45:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.086+0000 7ffb73400640 1 -- 172.21.15.89:0/19696352 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb6c016070 con 0x7ffb7c100840 2024-09-10T08:45:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.086+0000 7ffb73400640 1 -- 172.21.15.89:0/19696352 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c00f2e0 con 0x7ffb7c100840 2024-09-10T08:45:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.086+0000 7ffb7be00640 1 -- 172.21.15.89:0/19696352 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb7c1a1e30 con 0x7ffb7c100840 2024-09-10T08:45:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.086+0000 7ffb7be00640 1 -- 172.21.15.89:0/19696352 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb7c1a22c0 con 0x7ffb7c100840 2024-09-10T08:45:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.090+0000 7ffb73400640 1 -- 172.21.15.89:0/19696352 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ffb6c0119d0 con 0x7ffb7c100840 2024-09-10T08:45:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.090+0000 7ffb84634740 1 Processor -- start 2024-09-10T08:45:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.090+0000 7ffb84634740 1 -- start start 2024-09-10T08:45:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.090+0000 7ffb84634740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63c9650 0x55faa63c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.090+0000 7ffb84634740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63c9f90 0x55faa63cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.090+0000 7ffb84634740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63cf070 0x55faa63d1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.090+0000 7ffb84634740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55faa6333100 con 0x55faa63c9650 2024-09-10T08:45:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.090+0000 7ffb84634740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55faa62db1b0 con 0x55faa63c9f90 2024-09-10T08:45:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.090+0000 7ffb84634740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55faa632f0c0 con 0x55faa63cf070 2024-09-10T08:45:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.091+0000 7ffb7b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63c9650 0x55faa63c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.091+0000 7ffb7aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63c9f90 0x55faa63cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.091+0000 7ffb7a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63cf070 0x55faa63d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.091+0000 7ffb7b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63c9650 0x55faa63c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49758/0 (socket says 172.21.15.89:49758) 2024-09-10T08:45:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.091+0000 7ffb7b400640 1 -- 172.21.15.89:0/3433886246 learned_addr learned my addr 172.21.15.89:0/3433886246 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.091+0000 7ffb7aa00640 1 -- 172.21.15.89:0/3433886246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63cf070 msgr2=0x55faa63d1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.091+0000 7ffb7aa00640 1 --2- 172.21.15.89:0/3433886246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63cf070 0x55faa63d1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.091+0000 7ffb7aa00640 1 -- 172.21.15.89:0/3433886246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63c9650 msgr2=0x55faa63c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.091+0000 7ffb7aa00640 1 --2- 172.21.15.89:0/3433886246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63c9650 0x55faa63c9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.091+0000 7ffb7aa00640 1 -- 172.21.15.89:0/3433886246 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa6046210 con 0x55faa63c9f90 2024-09-10T08:45:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.092+0000 7ffb73400640 1 --2- 172.21.15.89:0/19696352 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffb600a2c40 0x7ffb600a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.092+0000 7ffb7aa00640 1 --2- 172.21.15.89:0/3433886246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63c9f90 0x55faa63cea50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0cc000 tx=0x7ffb6c0d4ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93134972622b3501 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.092+0000 7ffb67e00640 1 -- 172.21.15.89:0/3433886246 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c097060 con 0x55faa63c9f90 2024-09-10T08:45:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.092+0000 7ffb73400640 1 -- 172.21.15.89:0/19696352 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ffb6c0ce990 con 0x7ffb7c100840 2024-09-10T08:45:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.092+0000 7ffb7b400640 1 --2- 172.21.15.89:0/19696352 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffb600a2c40 0x7ffb600a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.092+0000 7ffb67e00640 1 -- 172.21.15.89:0/3433886246 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb6c09b4c0 con 0x55faa63c9f90 2024-09-10T08:45:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.092+0000 7ffb67e00640 1 -- 172.21.15.89:0/3433886246 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c068dd0 con 0x55faa63c9f90 2024-09-10T08:45:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.093+0000 7ffb84634740 1 -- 172.21.15.89:0/3433886246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63c9f90 msgr2=0x55faa63cea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.093+0000 7ffb7b400640 1 --2- 172.21.15.89:0/19696352 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffb600a2c40 0x7ffb600a5100 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ffb74030e40 tx=0x7ffb74009960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.093+0000 7ffb84634740 1 --2- 172.21.15.89:0/3433886246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63c9f90 0x55faa63cea50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0cc000 tx=0x7ffb6c0d4ef0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.093+0000 7ffb84634740 1 -- 172.21.15.89:0/3433886246 shutdown_connections 2024-09-10T08:45:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.093+0000 7ffb84634740 1 --2- 172.21.15.89:0/3433886246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63cf070 0x55faa63d1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.093+0000 7ffb84634740 1 --2- 172.21.15.89:0/3433886246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63c9f90 0x55faa63cea50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.093+0000 7ffb84634740 1 --2- 172.21.15.89:0/3433886246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63c9650 0x55faa63c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.093+0000 7ffb84634740 1 -- 172.21.15.89:0/3433886246 >> 172.21.15.89:0/3433886246 conn(0x55faa63d20e0 msgr2=0x55faa63d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.093+0000 7ffb84634740 1 -- 172.21.15.89:0/3433886246 shutdown_connections 2024-09-10T08:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.093+0000 7ffb84634740 1 -- 172.21.15.89:0/3433886246 wait complete. 2024-09-10T08:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.093+0000 7ffb84634740 1 -- start start 2024-09-10T08:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.094+0000 7ffb84634740 1 Processor -- start 2024-09-10T08:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.094+0000 7ffb84634740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63c9650 0x55faa63e94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.094+0000 7ffb84634740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63c9f90 0x55faa63e99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.094+0000 7ffb84634740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63cf070 0x55faa63f0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.094+0000 7ffb84634740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55faa63e7be0 con 0x55faa63c9f90 2024-09-10T08:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.094+0000 7ffb84634740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55faa62bc090 con 0x55faa63cf070 2024-09-10T08:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.094+0000 7ffb84634740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55faa63e7d60 con 0x55faa63c9650 2024-09-10T08:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.094+0000 7ffb84634740 10 client.0 _ll_register_callbacks cb 0x55faa6310f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.094+0000 7ffb7a000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63c9f90 0x55faa63e99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.094+0000 7ffb7a000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63c9f90 0x55faa63e99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49760/0 (socket says 172.21.15.89:49760) 2024-09-10T08:45:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.094+0000 7ffb7a000640 1 -- 172.21.15.89:0/1527824285 learned_addr learned my addr 172.21.15.89:0/1527824285 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.094+0000 7ffb7b400640 1 --2- 172.21.15.89:0/1527824285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63cf070 0x55faa63f0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.095+0000 7ffb7aa00640 1 --2- 172.21.15.89:0/1527824285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63c9650 0x55faa63e94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.095+0000 7ffb7a000640 1 -- 172.21.15.89:0/1527824285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63c9650 msgr2=0x55faa63e94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.095+0000 7ffb7a000640 1 --2- 172.21.15.89:0/1527824285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63c9650 0x55faa63e94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.095+0000 7ffb7a000640 1 -- 172.21.15.89:0/1527824285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63cf070 msgr2=0x55faa63f0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.095+0000 7ffb7a000640 1 --2- 172.21.15.89:0/1527824285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63cf070 0x55faa63f0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.095+0000 7ffb7a000640 1 -- 172.21.15.89:0/1527824285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa6112620 con 0x55faa63c9f90 2024-09-10T08:45:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.095+0000 7ffb7a000640 1 --2- 172.21.15.89:0/1527824285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63c9f90 0x55faa63e99f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800bd80 tx=0x7ffb680107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a532cc02c05ce7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.095+0000 7ffb64c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:45:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.095+0000 7ffb64c00640 1 -- 172.21.15.89:0/1527824285 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb68011850 con 0x55faa63c9f90 2024-09-10T08:45:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.095+0000 7ffb64c00640 1 -- 172.21.15.89:0/1527824285 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ffb680119f0 con 0x55faa63c9f90 2024-09-10T08:45:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.095+0000 7ffb84634740 1 -- 172.21.15.89:0/1527824285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb6c09b4c0 con 0x55faa63c9f90 2024-09-10T08:45:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.096+0000 7ffb64c00640 1 -- 172.21.15.89:0/1527824285 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffb68011bf0 con 0x55faa63c9f90 2024-09-10T08:45:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.096+0000 7ffb84634740 10 client.16524 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.096+0000 7ffb84634740 1 -- 172.21.15.89:0/1527824285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55faa63f38f0 con 0x55faa63c9f90 2024-09-10T08:45:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.096+0000 7ffb64c00640 1 -- 172.21.15.89:0/1527824285 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb6c09b4c0 con 0x55faa63c9f90 2024-09-10T08:45:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.096+0000 7ffb64c00640 1 -- 172.21.15.89:0/1527824285 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7ffb6801a380 con 0x55faa63c9f90 2024-09-10T08:45:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.096+0000 7ffb84634740 10 client.16524 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.096+0000 7ffb84634740 10 client.16524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:45:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.096+0000 7ffb84634740 20 client.16524 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.096+0000 7ffb84634740 1 --2- 172.21.15.89:0/1527824285 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55faa63f4f70 0x55faa63f7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.096+0000 7ffb84634740 4 client.16524 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:45:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.096+0000 7ffb84634740 1 -- 172.21.15.89:0/1527824285 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faa6340830 con 0x55faa63f4f70 2024-09-10T08:45:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.097+0000 7ffb7aa00640 1 --2- 172.21.15.89:0/1527824285 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55faa63f4f70 0x55faa63f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.097+0000 7ffb7aa00640 1 --2- 172.21.15.89:0/1527824285 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55faa63f4f70 0x55faa63f7380 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d497f885381d91a6 server_cookie=38db0bfd8674ed65 in_seq=0 out_seq=0 2024-09-10T08:45:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.097+0000 7ffb64c00640 10 client.16524 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.099+0000 7ffb64c00640 1 -- 172.21.15.89:0/1527824285 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55faa6340830 con 0x55faa63f4f70 2024-09-10T08:45:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.099+0000 7ffb64c00640 10 client.16524 handle_command_reply: tid=0 2024-09-10T08:45:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.099+0000 7ffb84634740 1 client.16524 shutdown 2024-09-10T08:45:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.099+0000 7ffb84634740 1 -- 172.21.15.89:0/1527824285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63c9f90 msgr2=0x55faa63e99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.099+0000 7ffb84634740 1 --2- 172.21.15.89:0/1527824285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63c9f90 0x55faa63e99f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800bd80 tx=0x7ffb680107a0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.099+0000 7ffb84634740 1 -- 172.21.15.89:0/1527824285 shutdown_connections 2024-09-10T08:45:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.099+0000 7ffb84634740 1 --2- 172.21.15.89:0/1527824285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63cf070 0x55faa63f0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.099+0000 7ffb84634740 1 --2- 172.21.15.89:0/1527824285 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55faa63f4f70 0x55faa63f7380 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.099+0000 7ffb84634740 1 --2- 172.21.15.89:0/1527824285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63c9f90 0x55faa63e99f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.099+0000 7ffb64c00640 0 client.16524 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.099+0000 7ffb84634740 1 --2- 172.21.15.89:0/1527824285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63c9650 0x55faa63e94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.099+0000 7ffb84634740 1 -- 172.21.15.89:0/1527824285 >> 172.21.15.89:0/1527824285 conn(0x55faa63d20e0 msgr2=0x55faa63d38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.099+0000 7ffb84634740 1 -- 172.21.15.89:0/1527824285 shutdown_connections 2024-09-10T08:45:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.100+0000 7ffb84634740 1 -- 172.21.15.89:0/1527824285 wait complete. 2024-09-10T08:45:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.100+0000 7ffb84634740 20 client.16524 trim_cache size 0 max 16384 2024-09-10T08:45:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.115+0000 7ffb84634740 1 Processor -- start 2024-09-10T08:45:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.115+0000 7ffb84634740 1 -- start start 2024-09-10T08:45:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.115+0000 7ffb84634740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63c9650 0x55faa63fbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.115+0000 7ffb84634740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63c9f90 0x55faa63f09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.115+0000 7ffb84634740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63cf070 0x55faa63fc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.115+0000 7ffb84634740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c1aaf40 con 0x55faa63cf070 2024-09-10T08:45:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.115+0000 7ffb84634740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c1a8810 con 0x55faa63c9650 2024-09-10T08:45:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.115+0000 7ffb84634740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c1a6660 con 0x55faa63c9f90 2024-09-10T08:45:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.115+0000 7ffb7aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63c9650 0x55faa63fbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.115+0000 7ffb7aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63c9650 0x55faa63fbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:55508/0 (socket says 172.21.15.89:55508) 2024-09-10T08:45:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.115+0000 7ffb7aa00640 1 -- 172.21.15.89:0/2795920809 learned_addr learned my addr 172.21.15.89:0/2795920809 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.115+0000 7ffb7b400640 1 --2- 172.21.15.89:0/2795920809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63cf070 0x55faa63fc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.115+0000 7ffb7a000640 1 --2- 172.21.15.89:0/2795920809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63c9f90 0x55faa63f09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.116+0000 7ffb7aa00640 1 -- 172.21.15.89:0/2795920809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63c9f90 msgr2=0x55faa63f09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.116+0000 7ffb7aa00640 1 --2- 172.21.15.89:0/2795920809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63c9f90 0x55faa63f09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.116+0000 7ffb7aa00640 1 -- 172.21.15.89:0/2795920809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63cf070 msgr2=0x55faa63fc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.116+0000 7ffb7aa00640 1 --2- 172.21.15.89:0/2795920809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63cf070 0x55faa63fc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.116+0000 7ffb7aa00640 1 -- 172.21.15.89:0/2795920809 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa63ca3d0 con 0x55faa63c9650 2024-09-10T08:45:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.116+0000 7ffb7a000640 1 --2- 172.21.15.89:0/2795920809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63c9f90 0x55faa63f09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.116+0000 7ffb7aa00640 1 --2- 172.21.15.89:0/2795920809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63c9650 0x55faa63fbcd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c000c00 tx=0x7ffb6c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a82bbe0bfee7cd29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.116+0000 7ffb67400640 1 -- 172.21.15.89:0/2795920809 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c069030 con 0x55faa63c9650 2024-09-10T08:45:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.116+0000 7ffb67400640 1 -- 172.21.15.89:0/2795920809 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb6c09b4c0 con 0x55faa63c9650 2024-09-10T08:45:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.116+0000 7ffb67400640 1 -- 172.21.15.89:0/2795920809 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c069290 con 0x55faa63c9650 2024-09-10T08:45:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.117+0000 7ffb84634740 1 -- 172.21.15.89:0/2795920809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63c9650 msgr2=0x55faa63fbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.117+0000 7ffb84634740 1 --2- 172.21.15.89:0/2795920809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63c9650 0x55faa63fbcd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c000c00 tx=0x7ffb6c0046c0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.117+0000 7ffb84634740 1 -- 172.21.15.89:0/2795920809 shutdown_connections 2024-09-10T08:45:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.117+0000 7ffb84634740 1 --2- 172.21.15.89:0/2795920809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63cf070 0x55faa63fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.117+0000 7ffb84634740 1 --2- 172.21.15.89:0/2795920809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63c9f90 0x55faa63f09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.117+0000 7ffb84634740 1 --2- 172.21.15.89:0/2795920809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63c9650 0x55faa63fbcd0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.117+0000 7ffb84634740 1 -- 172.21.15.89:0/2795920809 >> 172.21.15.89:0/2795920809 conn(0x55faa63d20e0 msgr2=0x55faa63d3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.117+0000 7ffb84634740 1 -- 172.21.15.89:0/2795920809 shutdown_connections 2024-09-10T08:45:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.117+0000 7ffb84634740 1 -- 172.21.15.89:0/2795920809 wait complete. 2024-09-10T08:45:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.117+0000 7ffb84634740 1 -- start start 2024-09-10T08:45:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.117+0000 7ffb84634740 1 Processor -- start 2024-09-10T08:45:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.118+0000 7ffb84634740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63c9650 0x55faa63ee240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.118+0000 7ffb84634740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63c9f90 0x55faa63ee780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.118+0000 7ffb84634740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63cf070 0x55faa63ef790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.118+0000 7ffb84634740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c1a6cd0 con 0x55faa63cf070 2024-09-10T08:45:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.118+0000 7ffb84634740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c1a8140 con 0x55faa63c9650 2024-09-10T08:45:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.118+0000 7ffb84634740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c1a73a0 con 0x55faa63c9f90 2024-09-10T08:45:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.118+0000 7ffb84634740 10 client.0 _ll_register_callbacks cb 0x7ffb6800b5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.118+0000 7ffb7aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63c9650 0x55faa63ee240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.118+0000 7ffb7aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63c9650 0x55faa63ee240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:55524/0 (socket says 172.21.15.89:55524) 2024-09-10T08:45:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.118+0000 7ffb7aa00640 1 -- 172.21.15.89:0/965137116 learned_addr learned my addr 172.21.15.89:0/965137116 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.118+0000 7ffb7b400640 1 --2- 172.21.15.89:0/965137116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63cf070 0x55faa63ef790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.118+0000 7ffb7a000640 1 --2- 172.21.15.89:0/965137116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63c9f90 0x55faa63ee780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.118+0000 7ffb7aa00640 1 -- 172.21.15.89:0/965137116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63c9f90 msgr2=0x55faa63ee780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.118+0000 7ffb7aa00640 1 --2- 172.21.15.89:0/965137116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63c9f90 0x55faa63ee780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.118+0000 7ffb7aa00640 1 -- 172.21.15.89:0/965137116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63cf070 msgr2=0x55faa63ef790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.118+0000 7ffb7aa00640 1 --2- 172.21.15.89:0/965137116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63cf070 0x55faa63ef790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.118+0000 7ffb7aa00640 1 -- 172.21.15.89:0/965137116 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa63efda0 con 0x55faa63c9650 2024-09-10T08:45:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.118+0000 7ffb7b400640 1 --2- 172.21.15.89:0/965137116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63cf070 0x55faa63ef790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.118+0000 7ffb7a000640 1 --2- 172.21.15.89:0/965137116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63c9f90 0x55faa63ee780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.119+0000 7ffb7aa00640 1 --2- 172.21.15.89:0/965137116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63c9650 0x55faa63ee240 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c010f10 tx=0x7ffb6c010780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb38af8c203a7159 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.119+0000 7ffb64c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:45:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.119+0000 7ffb64c00640 1 -- 172.21.15.89:0/965137116 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c065020 con 0x55faa63c9650 2024-09-10T08:45:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.119+0000 7ffb84634740 1 -- 172.21.15.89:0/965137116 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb6c09b4c0 con 0x55faa63c9650 2024-09-10T08:45:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.119+0000 7ffb64c00640 1 -- 172.21.15.89:0/965137116 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ffb6c0cc170 con 0x55faa63c9650 2024-09-10T08:45:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.119+0000 7ffb64c00640 1 -- 172.21.15.89:0/965137116 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffb6c0698f0 con 0x55faa63c9650 2024-09-10T08:45:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.119+0000 7ffb84634740 10 client.26564 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.119+0000 7ffb84634740 1 -- 172.21.15.89:0/965137116 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55faa645fe30 con 0x55faa63c9650 2024-09-10T08:45:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.119+0000 7ffb64c00640 1 -- 172.21.15.89:0/965137116 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb6c09b4c0 con 0x55faa63c9650 2024-09-10T08:45:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.119+0000 7ffb64c00640 1 -- 172.21.15.89:0/965137116 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7ffb6c0d4220 con 0x55faa63c9650 2024-09-10T08:45:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.119+0000 7ffb84634740 10 client.26564 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.119+0000 7ffb84634740 10 client.26564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:45:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.119+0000 7ffb84634740 20 client.26564 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.119+0000 7ffb84634740 1 --2- 172.21.15.89:0/965137116 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55faa64602f0 0x55faa6462710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.119+0000 7ffb84634740 4 client.26564 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:45:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.119+0000 7ffb84634740 1 -- 172.21.15.89:0/965137116 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55faa63e88d0 con 0x55faa64602f0 2024-09-10T08:45:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.120+0000 7ffb7a000640 1 --2- 172.21.15.89:0/965137116 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55faa64602f0 0x55faa6462710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.120+0000 7ffb7a000640 1 --2- 172.21.15.89:0/965137116 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55faa64602f0 0x55faa6462710 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6069187da611f28 server_cookie=8b9f87a6fe8a0609 in_seq=0 out_seq=0 2024-09-10T08:45:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:33.120+0000 7ffb64c00640 10 client.26564 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:33.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:32 smithi149 ceph-mon[36167]: pgmap v561: 129 pgs: 129 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 146 KiB/s wr, 1.14k op/s 2024-09-10T08:45:33.221 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:32 smithi089 ceph-mon[30757]: pgmap v561: 129 pgs: 129 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 146 KiB/s wr, 1.14k op/s 2024-09-10T08:45:33.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:32 smithi111 ceph-mon[35220]: pgmap v561: 129 pgs: 129 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 146 KiB/s wr, 1.14k op/s 2024-09-10T08:45:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.033+0000 7ffb64c00640 1 -- 172.21.15.89:0/965137116 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55faa63e88d0 con 0x55faa64602f0 2024-09-10T08:45:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.033+0000 7ffb64c00640 10 client.26564 handle_command_reply: tid=0 2024-09-10T08:45:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.033+0000 7ffb84634740 1 client.26564 shutdown 2024-09-10T08:45:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.034+0000 7ffb84634740 1 -- 172.21.15.89:0/965137116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63c9650 msgr2=0x55faa63ee240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.034+0000 7ffb84634740 1 --2- 172.21.15.89:0/965137116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63c9650 0x55faa63ee240 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c010f10 tx=0x7ffb6c010780 comp rx=0 tx=0).stop 2024-09-10T08:45:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.034+0000 7ffb84634740 1 -- 172.21.15.89:0/965137116 shutdown_connections 2024-09-10T08:45:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.034+0000 7ffb84634740 1 --2- 172.21.15.89:0/965137116 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55faa64602f0 0x55faa6462710 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.034+0000 7ffb84634740 1 --2- 172.21.15.89:0/965137116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55faa63cf070 0x55faa63ef790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.034+0000 7ffb84634740 1 --2- 172.21.15.89:0/965137116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faa63c9f90 0x55faa63ee780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.034+0000 7ffb84634740 1 --2- 172.21.15.89:0/965137116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa63c9650 0x55faa63ee240 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.034+0000 7ffb64c00640 0 client.26564 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.034+0000 7ffb84634740 1 -- 172.21.15.89:0/965137116 >> 172.21.15.89:0/965137116 conn(0x55faa63d20e0 msgr2=0x55faa63c81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.034+0000 7ffb84634740 1 -- 172.21.15.89:0/965137116 shutdown_connections 2024-09-10T08:45:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.034+0000 7ffb84634740 1 -- 172.21.15.89:0/965137116 wait complete. 2024-09-10T08:45:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.034+0000 7ffb84634740 20 client.26564 trim_cache size 0 max 16384 2024-09-10T08:45:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:45:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:45:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:45:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:45:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.035+0000 7ffb7be00640 1 -- 172.21.15.89:0/19696352 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffb600a2c40 msgr2=0x7ffb600a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.035+0000 7ffb7be00640 1 --2- 172.21.15.89:0/19696352 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffb600a2c40 0x7ffb600a5100 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ffb74030e40 tx=0x7ffb74009960 comp rx=0 tx=0).stop 2024-09-10T08:45:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.035+0000 7ffb7be00640 1 -- 172.21.15.89:0/19696352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb7c100840 msgr2=0x7ffb7c199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.035+0000 7ffb7be00640 1 --2- 172.21.15.89:0/19696352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb7c100840 0x7ffb7c199f20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00b3c0 tx=0x7ffb6c00b880 comp rx=0 tx=0).stop 2024-09-10T08:45:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.035+0000 7ffb7be00640 1 -- 172.21.15.89:0/19696352 shutdown_connections 2024-09-10T08:45:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.035+0000 7ffb7be00640 1 --2- 172.21.15.89:0/19696352 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ffb600a2c40 0x7ffb600a5100 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.035+0000 7ffb7be00640 1 --2- 172.21.15.89:0/19696352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb7c1023f0 0x7ffb7c1a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.035+0000 7ffb7be00640 1 --2- 172.21.15.89:0/19696352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb7c101a30 0x7ffb7c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.035+0000 7ffb7be00640 1 --2- 172.21.15.89:0/19696352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb7c100840 0x7ffb7c199f20 secure :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00b3c0 tx=0x7ffb6c00b880 comp rx=0 tx=0).stop 2024-09-10T08:45:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.035+0000 7ffb7be00640 1 -- 172.21.15.89:0/19696352 >> 172.21.15.89:0/19696352 conn(0x7ffb7c0fbfa0 msgr2=0x7ffb7c0698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.036+0000 7ffb7be00640 1 -- 172.21.15.89:0/19696352 shutdown_connections 2024-09-10T08:45:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:34.036+0000 7ffb7be00640 1 -- 172.21.15.89:0/19696352 wait complete. 2024-09-10T08:45:35.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:34 smithi149 ceph-mon[36167]: pgmap v562: 129 pgs: 129 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 153 KiB/s wr, 783 op/s 2024-09-10T08:45:35.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:34 smithi111 ceph-mon[35220]: pgmap v562: 129 pgs: 129 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 153 KiB/s wr, 783 op/s 2024-09-10T08:45:35.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:34 smithi089 ceph-mon[30757]: pgmap v562: 129 pgs: 129 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 153 KiB/s wr, 783 op/s 2024-09-10T08:45:37.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:36 smithi149 ceph-mon[36167]: pgmap v563: 129 pgs: 129 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 572 KiB/s wr, 777 op/s 2024-09-10T08:45:37.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:36 smithi111 ceph-mon[35220]: pgmap v563: 129 pgs: 129 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 572 KiB/s wr, 777 op/s 2024-09-10T08:45:37.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:36 smithi089 ceph-mon[30757]: pgmap v563: 129 pgs: 129 active+clean; 303 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 572 KiB/s wr, 777 op/s 2024-09-10T08:45:38.043 INFO:tasks.workunit.client.0.smithi089.stdout: 92 65663 116 45422 188 36902 603 2024-09-10T08:45:39.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:38 smithi149 ceph-mon[36167]: pgmap v564: 129 pgs: 129 active+clean; 308 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.0 MiB/s wr, 422 op/s 2024-09-10T08:45:39.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:38 smithi111 ceph-mon[35220]: pgmap v564: 129 pgs: 129 active+clean; 308 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.0 MiB/s wr, 422 op/s 2024-09-10T08:45:39.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:38 smithi089 ceph-mon[30757]: pgmap v564: 129 pgs: 129 active+clean; 308 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.0 MiB/s wr, 422 op/s 2024-09-10T08:45:41.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:40 smithi149 ceph-mon[36167]: pgmap v565: 129 pgs: 129 active+clean; 308 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.0 MiB/s wr, 249 op/s 2024-09-10T08:45:41.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:40 smithi111 ceph-mon[35220]: pgmap v565: 129 pgs: 129 active+clean; 308 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.0 MiB/s wr, 249 op/s 2024-09-10T08:45:41.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:40 smithi089 ceph-mon[30757]: pgmap v565: 129 pgs: 129 active+clean; 308 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.0 MiB/s wr, 249 op/s 2024-09-10T08:45:43.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:42 smithi149 ceph-mon[36167]: pgmap v566: 129 pgs: 129 active+clean; 309 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 1.2 MiB/s wr, 274 op/s 2024-09-10T08:45:43.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:42 smithi111 ceph-mon[35220]: pgmap v566: 129 pgs: 129 active+clean; 309 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 1.2 MiB/s wr, 274 op/s 2024-09-10T08:45:43.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:42 smithi089 ceph-mon[30757]: pgmap v566: 129 pgs: 129 active+clean; 309 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 1.2 MiB/s wr, 274 op/s 2024-09-10T08:45:45.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:44 smithi149 ceph-mon[36167]: pgmap v567: 129 pgs: 129 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 1.3 MiB/s wr, 277 op/s 2024-09-10T08:45:45.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:44 smithi111 ceph-mon[35220]: pgmap v567: 129 pgs: 129 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 1.3 MiB/s wr, 277 op/s 2024-09-10T08:45:45.306 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:44 smithi089 ceph-mon[30757]: pgmap v567: 129 pgs: 129 active+clean; 310 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 1.3 MiB/s wr, 277 op/s 2024-09-10T08:45:47.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:46 smithi149 ceph-mon[36167]: pgmap v568: 129 pgs: 129 active+clean; 309 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 1.2 MiB/s wr, 296 op/s 2024-09-10T08:45:47.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:46 smithi111 ceph-mon[35220]: pgmap v568: 129 pgs: 129 active+clean; 309 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 1.2 MiB/s wr, 296 op/s 2024-09-10T08:45:47.321 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:46 smithi089 ceph-mon[30757]: pgmap v568: 129 pgs: 129 active+clean; 309 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 1.2 MiB/s wr, 296 op/s 2024-09-10T08:45:48.069 INFO:tasks.workunit.client.0.smithi089.stdout: 96 358793 271 249174 304 214880 879 2024-09-10T08:45:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.009+0000 7fa12d600640 1 -- 172.21.15.89:0/2961806739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa128072d20 msgr2=0x7fa128073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:49.539 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.009+0000 7fa12d600640 1 --2- 172.21.15.89:0/2961806739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa128072d20 0x7fa128073180 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa114009900 tx=0x7fa11402a440 comp rx=0 tx=0).stop 2024-09-10T08:45:49.541 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:48 smithi111 ceph-mon[35220]: pgmap v569: 129 pgs: 129 active+clean; 311 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.0 MiB/s wr, 197 op/s 2024-09-10T08:45:49.541 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:48 smithi149 ceph-mon[36167]: pgmap v569: 129 pgs: 129 active+clean; 311 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.0 MiB/s wr, 197 op/s 2024-09-10T08:45:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.040+0000 7fa12d600640 1 -- 172.21.15.89:0/2961806739 shutdown_connections 2024-09-10T08:45:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.040+0000 7fa12d600640 1 --2- 172.21.15.89:0/2961806739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1280736c0 0x7fa128119130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.040+0000 7fa12d600640 1 --2- 172.21.15.89:0/2961806739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa128072d20 0x7fa128073180 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.040+0000 7fa12d600640 1 --2- 172.21.15.89:0/2961806739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa128109700 0x7fa128109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.040+0000 7fa12d600640 1 -- 172.21.15.89:0/2961806739 >> 172.21.15.89:0/2961806739 conn(0x7fa12806ece0 msgr2=0x7fa128071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:49.542 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:48 smithi089 ceph-mon[30757]: pgmap v569: 129 pgs: 129 active+clean; 311 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.0 MiB/s wr, 197 op/s 2024-09-10T08:45:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.086+0000 7fa12d600640 1 -- 172.21.15.89:0/2961806739 shutdown_connections 2024-09-10T08:45:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.087+0000 7fa12d600640 1 -- 172.21.15.89:0/2961806739 wait complete. 2024-09-10T08:45:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.088+0000 7fa12d600640 1 Processor -- start 2024-09-10T08:45:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.092+0000 7fa12d600640 1 -- start start 2024-09-10T08:45:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.104+0000 7fa12d600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa128072d20 0x7fa1281a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.106+0000 7fa126a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa128072d20 0x7fa1281a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.107+0000 7fa126a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa128072d20 0x7fa1281a6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:45944/0 (socket says 172.21.15.89:45944) 2024-09-10T08:45:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.108+0000 7fa126a00640 1 -- 172.21.15.89:0/1552719108 learned_addr learned my addr 172.21.15.89:0/1552719108 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.108+0000 7fa12d600640 1 --2- 172.21.15.89:0/1552719108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1280736c0 0x7fa1281a70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.112+0000 7fa126000640 1 --2- 172.21.15.89:0/1552719108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1280736c0 0x7fa1281a70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.113+0000 7fa12d600640 1 --2- 172.21.15.89:0/1552719108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa128109700 0x7fa12807a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.114+0000 7fa127400640 1 --2- 172.21.15.89:0/1552719108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa128109700 0x7fa12807a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.114+0000 7fa12d600640 1 -- 172.21.15.89:0/1552719108 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa12811c5e0 con 0x7fa1280736c0 2024-09-10T08:45:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.114+0000 7fa12d600640 1 -- 172.21.15.89:0/1552719108 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa12811c460 con 0x7fa128072d20 2024-09-10T08:45:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.114+0000 7fa12d600640 1 -- 172.21.15.89:0/1552719108 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa12811c760 con 0x7fa128109700 2024-09-10T08:45:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.121+0000 7fa126000640 1 -- 172.21.15.89:0/1552719108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa128109700 msgr2=0x7fa12807a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.121+0000 7fa126000640 1 --2- 172.21.15.89:0/1552719108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa128109700 0x7fa12807a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.121+0000 7fa126000640 1 -- 172.21.15.89:0/1552719108 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa128072d20 msgr2=0x7fa1281a6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.121+0000 7fa126000640 1 --2- 172.21.15.89:0/1552719108 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa128072d20 0x7fa1281a6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.121+0000 7fa126000640 1 -- 172.21.15.89:0/1552719108 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12807aaf0 con 0x7fa1280736c0 2024-09-10T08:45:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.156+0000 7fa126000640 1 --2- 172.21.15.89:0/1552719108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1280736c0 0x7fa1281a70d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa114009850 tx=0x7fa114032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.168+0000 7fa11f400640 1 -- 172.21.15.89:0/1552719108 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114050030 con 0x7fa1280736c0 2024-09-10T08:45:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.168+0000 7fa11f400640 1 -- 172.21.15.89:0/1552719108 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa114032e00 con 0x7fa1280736c0 2024-09-10T08:45:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.168+0000 7fa11f400640 1 -- 172.21.15.89:0/1552719108 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114035070 con 0x7fa1280736c0 2024-09-10T08:45:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.169+0000 7fa12d600640 1 -- 172.21.15.89:0/1552719108 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa128110920 con 0x7fa1280736c0 2024-09-10T08:45:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.169+0000 7fa12d600640 1 -- 172.21.15.89:0/1552719108 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa128110d90 con 0x7fa1280736c0 2024-09-10T08:45:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.208+0000 7fa130654740 1 Processor -- start 2024-09-10T08:45:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.216+0000 7fa130654740 1 -- start start 2024-09-10T08:45:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.216+0000 7fa130654740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c89090 0x55ea66c89490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.227+0000 7fa130654740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c899d0 0x55ea66c8e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.227+0000 7fa130654740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea66c8ea60 0x55ea66c90f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.227+0000 7fa130654740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ea66bee100 con 0x55ea66c8ea60 2024-09-10T08:45:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.227+0000 7fa130654740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ea66b961b0 con 0x55ea66c89090 2024-09-10T08:45:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.227+0000 7fa130654740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ea66bea0c0 con 0x55ea66c899d0 2024-09-10T08:45:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.227+0000 7fa127400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c89090 0x55ea66c89490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.228+0000 7fa127400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c89090 0x55ea66c89490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:45956/0 (socket says 172.21.15.89:45956) 2024-09-10T08:45:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.228+0000 7fa127400640 1 -- 172.21.15.89:0/2095071147 learned_addr learned my addr 172.21.15.89:0/2095071147 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.230+0000 7fa127400640 1 -- 172.21.15.89:0/2095071147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c899d0 msgr2=0x55ea66c8e490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:45:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.230+0000 7fa127400640 1 --2- 172.21.15.89:0/2095071147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c899d0 0x55ea66c8e490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.230+0000 7fa127400640 1 -- 172.21.15.89:0/2095071147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea66c8ea60 msgr2=0x55ea66c90f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:45:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.230+0000 7fa127400640 1 --2- 172.21.15.89:0/2095071147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea66c8ea60 0x55ea66c90f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.230+0000 7fa127400640 1 -- 172.21.15.89:0/2095071147 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea66901210 con 0x55ea66c89090 2024-09-10T08:45:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.250+0000 7fa127400640 1 --2- 172.21.15.89:0/2095071147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c89090 0x55ea66c89490 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa120009ee0 tx=0x7fa120010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=555b932a08c8664b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.256+0000 7fa113400640 1 -- 172.21.15.89:0/2095071147 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120016070 con 0x55ea66c89090 2024-09-10T08:45:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.256+0000 7fa113400640 1 -- 172.21.15.89:0/2095071147 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa120004d20 con 0x55ea66c89090 2024-09-10T08:45:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.256+0000 7fa113400640 1 -- 172.21.15.89:0/2095071147 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120012810 con 0x55ea66c89090 2024-09-10T08:45:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.263+0000 7fa11f400640 1 -- 172.21.15.89:0/1552719108 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa114053030 con 0x7fa1280736c0 2024-09-10T08:45:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.282+0000 7fa11f400640 1 --2- 172.21.15.89:0/1552719108 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa10c0a2e70 0x7fa10c0a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.283+0000 7fa11f400640 1 -- 172.21.15.89:0/1552719108 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa114100550 con 0x7fa1280736c0 2024-09-10T08:45:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.288+0000 7fa126a00640 1 --2- 172.21.15.89:0/1552719108 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa10c0a2e70 0x7fa10c0a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.307+0000 7fa130654740 1 -- 172.21.15.89:0/2095071147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c89090 msgr2=0x55ea66c89490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.307+0000 7fa130654740 1 --2- 172.21.15.89:0/2095071147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c89090 0x55ea66c89490 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa120009ee0 tx=0x7fa120010740 comp rx=0 tx=0).stop 2024-09-10T08:45:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.307+0000 7fa130654740 1 -- 172.21.15.89:0/2095071147 shutdown_connections 2024-09-10T08:45:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.307+0000 7fa130654740 1 --2- 172.21.15.89:0/2095071147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea66c8ea60 0x55ea66c90f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.307+0000 7fa130654740 1 --2- 172.21.15.89:0/2095071147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c899d0 0x55ea66c8e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.307+0000 7fa130654740 1 --2- 172.21.15.89:0/2095071147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c89090 0x55ea66c89490 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.307+0000 7fa130654740 1 -- 172.21.15.89:0/2095071147 >> 172.21.15.89:0/2095071147 conn(0x55ea66c84700 msgr2=0x55ea66c84b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.308+0000 7fa126a00640 1 --2- 172.21.15.89:0/1552719108 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa10c0a2e70 0x7fa10c0a5330 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x55ea66c8a400 tx=0x7fa11800b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.308+0000 7fa130654740 1 -- 172.21.15.89:0/2095071147 shutdown_connections 2024-09-10T08:45:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.308+0000 7fa130654740 1 -- 172.21.15.89:0/2095071147 wait complete. 2024-09-10T08:45:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.309+0000 7fa130654740 1 -- start start 2024-09-10T08:45:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.311+0000 7fa130654740 1 Processor -- start 2024-09-10T08:45:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.313+0000 7fa130654740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c89090 0x55ea66c93360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.319+0000 7fa126a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c89090 0x55ea66c93360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.319+0000 7fa126a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c89090 0x55ea66c93360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:53354/0 (socket says 172.21.15.89:53354) 2024-09-10T08:45:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.319+0000 7fa126a00640 1 -- 172.21.15.89:0/1538399291 learned_addr learned my addr 172.21.15.89:0/1538399291 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.319+0000 7fa130654740 1 --2- 172.21.15.89:0/1538399291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea66c899d0 0x55ea66c938a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.320+0000 7fa130654740 1 --2- 172.21.15.89:0/1538399291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c8ea60 0x55ea66c9a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.321+0000 7fa130654740 1 -- 172.21.15.89:0/1538399291 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ea66c91a30 con 0x55ea66c899d0 2024-09-10T08:45:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.321+0000 7fa130654740 1 -- 172.21.15.89:0/1538399291 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ea66b77090 con 0x55ea66c8ea60 2024-09-10T08:45:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.321+0000 7fa130654740 1 -- 172.21.15.89:0/1538399291 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ea66c91bb0 con 0x55ea66c89090 2024-09-10T08:45:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.321+0000 7fa127400640 1 --2- 172.21.15.89:0/1538399291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c8ea60 0x55ea66c9a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.321+0000 7fa130654740 10 client.0 _ll_register_callbacks cb 0x55ea66bcbf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.322+0000 7fa127400640 1 -- 172.21.15.89:0/1538399291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c89090 msgr2=0x55ea66c93360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.322+0000 7fa127400640 1 --2- 172.21.15.89:0/1538399291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c89090 0x55ea66c93360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.322+0000 7fa127400640 1 -- 172.21.15.89:0/1538399291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea66c899d0 msgr2=0x55ea66c938a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.322+0000 7fa127400640 1 --2- 172.21.15.89:0/1538399291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea66c899d0 0x55ea66c938a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.322+0000 7fa127400640 1 -- 172.21.15.89:0/1538399291 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea669cd620 con 0x55ea66c8ea60 2024-09-10T08:45:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.324+0000 7fa130654740 1 -- 172.21.15.89:0/1538399291 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa120004d20 con 0x55ea66c8ea60 2024-09-10T08:45:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.342+0000 7fa127400640 1 --2- 172.21.15.89:0/1538399291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c8ea60 0x55ea66c9a8e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa120009ee0 tx=0x7fa120004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40970cbff730f6b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.353+0000 7fa107e00640 10 client.26576 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:45:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.353+0000 7fa107e00640 1 -- 172.21.15.89:0/1538399291 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120016050 con 0x55ea66c8ea60 2024-09-10T08:45:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.353+0000 7fa107e00640 1 -- 172.21.15.89:0/1538399291 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa1200050e0 con 0x55ea66c8ea60 2024-09-10T08:45:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.354+0000 7fa107e00640 1 -- 172.21.15.89:0/1538399291 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa120005340 con 0x55ea66c8ea60 2024-09-10T08:45:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.357+0000 7fa130654740 10 client.26576 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.357+0000 7fa130654740 1 -- 172.21.15.89:0/1538399291 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55ea66c9d5c0 con 0x55ea66c8ea60 2024-09-10T08:45:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.358+0000 7fa107e00640 1 -- 172.21.15.89:0/1538399291 <== mon.2 v2:172.21.15.111:3300/0 4 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fa12000f040 con 0x55ea66c8ea60 2024-09-10T08:45:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.359+0000 7fa130654740 10 client.26576 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.359+0000 7fa130654740 10 client.26576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:45:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.359+0000 7fa130654740 20 client.26576 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.359+0000 7fa130654740 1 --2- 172.21.15.89:0/1538399291 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ea66c9e910 0x55ea66ca0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.359+0000 7fa130654740 4 client.26576 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:45:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.359+0000 7fa130654740 1 -- 172.21.15.89:0/1538399291 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea66bfb830 con 0x55ea66c9e910 2024-09-10T08:45:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.366+0000 7fa126a00640 1 --2- 172.21.15.89:0/1538399291 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ea66c9e910 0x55ea66ca0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.368+0000 7fa126a00640 1 --2- 172.21.15.89:0/1538399291 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ea66c9e910 0x55ea66ca0d00 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9617f2782401e430 server_cookie=fd03415e4d68167b in_seq=0 out_seq=0 2024-09-10T08:45:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.369+0000 7fa107e00640 10 client.26576 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.415+0000 7fa107e00640 1 -- 172.21.15.89:0/1538399291 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea66bfb830 con 0x55ea66c9e910 2024-09-10T08:45:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.415+0000 7fa107e00640 10 client.26576 handle_command_reply: tid=0 2024-09-10T08:45:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.439+0000 7fa130654740 1 client.26576 shutdown 2024-09-10T08:45:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.443+0000 7fa130654740 1 -- 172.21.15.89:0/1538399291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c8ea60 msgr2=0x55ea66c9a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.443+0000 7fa130654740 1 --2- 172.21.15.89:0/1538399291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c8ea60 0x55ea66c9a8e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa120009ee0 tx=0x7fa120004c70 comp rx=0 tx=0).stop 2024-09-10T08:45:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.443+0000 7fa130654740 1 -- 172.21.15.89:0/1538399291 shutdown_connections 2024-09-10T08:45:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.443+0000 7fa130654740 1 --2- 172.21.15.89:0/1538399291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c8ea60 0x55ea66c9a8e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.443+0000 7fa130654740 1 --2- 172.21.15.89:0/1538399291 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ea66c9e910 0x55ea66ca0d00 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.443+0000 7fa130654740 1 --2- 172.21.15.89:0/1538399291 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea66c899d0 0x55ea66c938a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.443+0000 7fa130654740 1 --2- 172.21.15.89:0/1538399291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c89090 0x55ea66c93360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.443+0000 7fa130654740 1 -- 172.21.15.89:0/1538399291 >> 172.21.15.89:0/1538399291 conn(0x55ea66c84700 msgr2=0x55ea66c85f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.443+0000 7fa107e00640 0 client.26576 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.466+0000 7fa130654740 1 -- 172.21.15.89:0/1538399291 shutdown_connections 2024-09-10T08:45:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.480+0000 7fa130654740 1 -- 172.21.15.89:0/1538399291 wait complete. 2024-09-10T08:45:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.481+0000 7fa130654740 20 client.26576 trim_cache size 0 max 16384 2024-09-10T08:45:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.941+0000 7fa130654740 1 Processor -- start 2024-09-10T08:45:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.942+0000 7fa130654740 1 -- start start 2024-09-10T08:45:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.942+0000 7fa130654740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea66c89090 0x55ea66c84ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.943+0000 7fa130654740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c899d0 0x55ea66c9a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.944+0000 7fa130654740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c8ea60 0x55ea66c85020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.944+0000 7fa126a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea66c89090 0x55ea66c84ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.944+0000 7fa126a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea66c89090 0x55ea66c84ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:58056/0 (socket says 172.21.15.89:58056) 2024-09-10T08:45:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.944+0000 7fa126000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c899d0 0x55ea66c9a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.944+0000 7fa126000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c899d0 0x55ea66c9a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:53358/0 (socket says 172.21.15.89:53358) 2024-09-10T08:45:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.944+0000 7fa127400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c8ea60 0x55ea66c85020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.944+0000 7fa127400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c8ea60 0x55ea66c85020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:45964/0 (socket says 172.21.15.89:45964) 2024-09-10T08:45:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.945+0000 7fa126a00640 1 -- 172.21.15.89:0/2925283268 learned_addr learned my addr 172.21.15.89:0/2925283268 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.945+0000 7fa130654740 1 -- 172.21.15.89:0/2925283268 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa108012650 con 0x55ea66c89090 2024-09-10T08:45:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.945+0000 7fa130654740 1 -- 172.21.15.89:0/2925283268 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa10800ff20 con 0x55ea66c8ea60 2024-09-10T08:45:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.945+0000 7fa130654740 1 -- 172.21.15.89:0/2925283268 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa10800dd70 con 0x55ea66c899d0 2024-09-10T08:45:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.946+0000 7fa127400640 1 -- 172.21.15.89:0/2925283268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c899d0 msgr2=0x55ea66c9a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.946+0000 7fa127400640 1 --2- 172.21.15.89:0/2925283268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c899d0 0x55ea66c9a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.946+0000 7fa127400640 1 -- 172.21.15.89:0/2925283268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea66c89090 msgr2=0x55ea66c84ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.946+0000 7fa127400640 1 --2- 172.21.15.89:0/2925283268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea66c89090 0x55ea66c84ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.946+0000 7fa127400640 1 -- 172.21.15.89:0/2925283268 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea66c84520 con 0x55ea66c8ea60 2024-09-10T08:45:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.947+0000 7fa127400640 1 --2- 172.21.15.89:0/2925283268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c8ea60 0x55ea66c85020 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa120009e30 tx=0x7fa12001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b8c3e703a7dfbee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.947+0000 7fa112a00640 1 -- 172.21.15.89:0/2925283268 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120016070 con 0x55ea66c8ea60 2024-09-10T08:45:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.948+0000 7fa112a00640 1 -- 172.21.15.89:0/2925283268 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa120004d20 con 0x55ea66c8ea60 2024-09-10T08:45:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.948+0000 7fa112a00640 1 -- 172.21.15.89:0/2925283268 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120012cc0 con 0x55ea66c8ea60 2024-09-10T08:45:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.948+0000 7fa126a00640 1 --2- 172.21.15.89:0/2925283268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea66c89090 0x55ea66c84ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.958+0000 7fa130654740 1 -- 172.21.15.89:0/2925283268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c8ea60 msgr2=0x55ea66c85020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.958+0000 7fa130654740 1 --2- 172.21.15.89:0/2925283268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c8ea60 0x55ea66c85020 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa120009e30 tx=0x7fa12001f000 comp rx=0 tx=0).stop 2024-09-10T08:45:49.971 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.969+0000 7fa130654740 1 -- 172.21.15.89:0/2925283268 shutdown_connections 2024-09-10T08:45:49.971 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.969+0000 7fa130654740 1 --2- 172.21.15.89:0/2925283268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c8ea60 0x55ea66c85020 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.971 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.969+0000 7fa130654740 1 --2- 172.21.15.89:0/2925283268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c899d0 0x55ea66c9a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.971 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.969+0000 7fa130654740 1 --2- 172.21.15.89:0/2925283268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea66c89090 0x55ea66c84ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.971 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.969+0000 7fa130654740 1 -- 172.21.15.89:0/2925283268 >> 172.21.15.89:0/2925283268 conn(0x55ea66c84700 msgr2=0x55ea66c8ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.971+0000 7fa130654740 1 -- 172.21.15.89:0/2925283268 shutdown_connections 2024-09-10T08:45:49.974 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.972+0000 7fa130654740 1 -- 172.21.15.89:0/2925283268 wait complete. 2024-09-10T08:45:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.982+0000 7fa130654740 1 -- start start 2024-09-10T08:45:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.983+0000 7fa130654740 1 Processor -- start 2024-09-10T08:45:49.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.984+0000 7fa130654740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c89090 0x55ea66c97760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:49.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.984+0000 7fa130654740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c899d0 0x55ea66c97ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:49.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.985+0000 7fa126a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c89090 0x55ea66c97760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:49.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.985+0000 7fa126a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c89090 0x55ea66c97760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:53374/0 (socket says 172.21.15.89:53374) 2024-09-10T08:45:49.989 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.985+0000 7fa126a00640 1 -- 172.21.15.89:0/4165350504 learned_addr learned my addr 172.21.15.89:0/4165350504 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.985+0000 7fa130654740 1 --2- 172.21.15.89:0/4165350504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea66c8ea60 0x55ea66d07490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.985+0000 7fa127400640 1 --2- 172.21.15.89:0/4165350504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea66c8ea60 0x55ea66d07490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.986+0000 7fa126000640 1 --2- 172.21.15.89:0/4165350504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c899d0 0x55ea66c97ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.986+0000 7fa130654740 1 -- 172.21.15.89:0/4165350504 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa10800e3e0 con 0x55ea66c8ea60 2024-09-10T08:45:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.986+0000 7fa130654740 1 -- 172.21.15.89:0/4165350504 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa10800f850 con 0x55ea66c899d0 2024-09-10T08:45:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.986+0000 7fa130654740 1 -- 172.21.15.89:0/4165350504 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa10800eab0 con 0x55ea66c89090 2024-09-10T08:45:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.987+0000 7fa126000640 1 -- 172.21.15.89:0/4165350504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c89090 msgr2=0x55ea66c97760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.987+0000 7fa126000640 1 --2- 172.21.15.89:0/4165350504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c89090 0x55ea66c97760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.987+0000 7fa126000640 1 -- 172.21.15.89:0/4165350504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea66c8ea60 msgr2=0x55ea66d07490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.987+0000 7fa126000640 1 --2- 172.21.15.89:0/4165350504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea66c8ea60 0x55ea66d07490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.987+0000 7fa126000640 1 -- 172.21.15.89:0/4165350504 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea66d079d0 con 0x55ea66c899d0 2024-09-10T08:45:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.988+0000 7fa130654740 10 client.0 _ll_register_callbacks cb 0x7fa120009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.988+0000 7fa130654740 1 -- 172.21.15.89:0/4165350504 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa120004d20 con 0x55ea66c899d0 2024-09-10T08:45:49.993 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:49.988+0000 7fa126000640 1 --2- 172.21.15.89:0/4165350504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c899d0 0x55ea66c97ca0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55ea66c94840 tx=0x7fa1140cc3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a81ba7ef3ef091b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:50.007+0000 7fa107e00640 10 client.26582 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:45:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:50.007+0000 7fa107e00640 1 -- 172.21.15.89:0/4165350504 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1140e4070 con 0x55ea66c899d0 2024-09-10T08:45:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:50.007+0000 7fa107e00640 1 -- 172.21.15.89:0/4165350504 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa11404cd20 con 0x55ea66c899d0 2024-09-10T08:45:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:50.007+0000 7fa107e00640 1 -- 172.21.15.89:0/4165350504 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa1140cc980 con 0x55ea66c899d0 2024-09-10T08:45:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:50.013+0000 7fa130654740 10 client.26582 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:50.015+0000 7fa130654740 1 -- 172.21.15.89:0/4165350504 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55ea66d08a80 con 0x55ea66c899d0 2024-09-10T08:45:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:50.015+0000 7fa107e00640 1 -- 172.21.15.89:0/4165350504 <== mon.2 v2:172.21.15.111:3300/0 4 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fa1140c8ff0 con 0x55ea66c899d0 2024-09-10T08:45:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:50.015+0000 7fa130654740 10 client.26582 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:50.015+0000 7fa130654740 10 client.26582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:45:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:50.015+0000 7fa130654740 20 client.26582 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:50.015+0000 7fa130654740 1 --2- 172.21.15.89:0/4165350504 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ea66d093e0 0x55ea66d0b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:50.015+0000 7fa130654740 4 client.26582 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:45:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:50.015+0000 7fa130654740 1 -- 172.21.15.89:0/4165350504 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ea66bf5b80 con 0x55ea66d093e0 2024-09-10T08:45:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:50.016+0000 7fa126a00640 1 --2- 172.21.15.89:0/4165350504 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ea66d093e0 0x55ea66d0b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:50.017+0000 7fa126a00640 1 --2- 172.21.15.89:0/4165350504 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ea66d093e0 0x55ea66d0b8a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49ddf7316fa91b2d server_cookie=2a55b17f969a3c14 in_seq=0 out_seq=0 2024-09-10T08:45:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:50.017+0000 7fa107e00640 10 client.26582 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:50.325 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:45:51.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:50 smithi089 ceph-mon[30757]: pgmap v570: 129 pgs: 129 active+clean; 312 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 632 KiB/s wr, 122 op/s 2024-09-10T08:45:51.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:50 smithi149 ceph-mon[36167]: pgmap v570: 129 pgs: 129 active+clean; 312 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 632 KiB/s wr, 122 op/s 2024-09-10T08:45:51.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:50 smithi111 ceph-mon[35220]: pgmap v570: 129 pgs: 129 active+clean; 312 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 632 KiB/s wr, 122 op/s 2024-09-10T08:45:53.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:52 smithi149 ceph-mon[36167]: pgmap v571: 129 pgs: 129 active+clean; 324 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 235 op/s 2024-09-10T08:45:53.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:52 smithi111 ceph-mon[35220]: pgmap v571: 129 pgs: 129 active+clean; 324 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 235 op/s 2024-09-10T08:45:53.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:52 smithi089 ceph-mon[30757]: pgmap v571: 129 pgs: 129 active+clean; 324 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 235 op/s 2024-09-10T08:45:54.915 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:54 smithi089 ceph-mon[30757]: pgmap v572: 129 pgs: 129 active+clean; 326 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.9 MiB/s wr, 221 op/s 2024-09-10T08:45:55.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:54 smithi149 ceph-mon[36167]: pgmap v572: 129 pgs: 129 active+clean; 326 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.9 MiB/s wr, 221 op/s 2024-09-10T08:45:55.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:54 smithi111 ceph-mon[35220]: pgmap v572: 129 pgs: 129 active+clean; 326 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 1.9 MiB/s wr, 221 op/s 2024-09-10T08:45:55.408 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.840+0000 7fa107e00640 1 -- 172.21.15.89:0/4165350504 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ea66bf5b80 con 0x55ea66d093e0 2024-09-10T08:45:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.840+0000 7fa107e00640 10 client.26582 handle_command_reply: tid=0 2024-09-10T08:45:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.866+0000 7fa130654740 1 client.26582 shutdown 2024-09-10T08:45:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.888+0000 7fa130654740 1 -- 172.21.15.89:0/4165350504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c899d0 msgr2=0x55ea66c97ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.888+0000 7fa130654740 1 --2- 172.21.15.89:0/4165350504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c899d0 0x55ea66c97ca0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55ea66c94840 tx=0x7fa1140cc3a0 comp rx=0 tx=0).stop 2024-09-10T08:45:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.917+0000 7fa130654740 1 -- 172.21.15.89:0/4165350504 shutdown_connections 2024-09-10T08:45:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.917+0000 7fa130654740 1 --2- 172.21.15.89:0/4165350504 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55ea66d093e0 0x55ea66d0b8a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.917+0000 7fa130654740 1 --2- 172.21.15.89:0/4165350504 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ea66c8ea60 0x55ea66d07490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.917+0000 7fa130654740 1 --2- 172.21.15.89:0/4165350504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea66c899d0 0x55ea66c97ca0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.917+0000 7fa130654740 1 --2- 172.21.15.89:0/4165350504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66c89090 0x55ea66c97760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:56.918 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.917+0000 7fa130654740 1 -- 172.21.15.89:0/4165350504 >> 172.21.15.89:0/4165350504 conn(0x55ea66c84700 msgr2=0x55ea66c83060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.917+0000 7fa107e00640 0 client.26582 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.920+0000 7fa130654740 1 -- 172.21.15.89:0/4165350504 shutdown_connections 2024-09-10T08:45:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.920+0000 7fa130654740 1 -- 172.21.15.89:0/4165350504 wait complete. 2024-09-10T08:45:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:45:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:45:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:45:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:45:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.926+0000 7fa130654740 20 client.26582 trim_cache size 0 max 16384 2024-09-10T08:45:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.981+0000 7fa12d600640 1 -- 172.21.15.89:0/1552719108 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa10c0a2e70 msgr2=0x7fa10c0a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.981+0000 7fa12d600640 1 --2- 172.21.15.89:0/1552719108 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa10c0a2e70 0x7fa10c0a5330 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x55ea66c8a400 tx=0x7fa11800b5b0 comp rx=0 tx=0).stop 2024-09-10T08:45:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.981+0000 7fa12d600640 1 -- 172.21.15.89:0/1552719108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1280736c0 msgr2=0x7fa1281a70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.981+0000 7fa12d600640 1 --2- 172.21.15.89:0/1552719108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1280736c0 0x7fa1281a70d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa114009850 tx=0x7fa114032c10 comp rx=0 tx=0).stop 2024-09-10T08:45:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.982+0000 7fa12d600640 1 -- 172.21.15.89:0/1552719108 shutdown_connections 2024-09-10T08:45:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.982+0000 7fa12d600640 1 --2- 172.21.15.89:0/1552719108 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa10c0a2e70 0x7fa10c0a5330 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.982+0000 7fa12d600640 1 --2- 172.21.15.89:0/1552719108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa128109700 0x7fa12807a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.982+0000 7fa12d600640 1 --2- 172.21.15.89:0/1552719108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1280736c0 0x7fa1281a70d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.982+0000 7fa12d600640 1 --2- 172.21.15.89:0/1552719108 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa128072d20 0x7fa1281a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.982+0000 7fa12d600640 1 -- 172.21.15.89:0/1552719108 >> 172.21.15.89:0/1552719108 conn(0x7fa12806ece0 msgr2=0x7fa1280716c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.983+0000 7fa12d600640 1 -- 172.21.15.89:0/1552719108 shutdown_connections 2024-09-10T08:45:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:45:56.991+0000 7fa12d600640 1 -- 172.21.15.89:0/1552719108 wait complete. 2024-09-10T08:45:57.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:56 smithi149 ceph-mon[36167]: pgmap v573: 129 pgs: 129 active+clean; 341 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 512 op/s 2024-09-10T08:45:57.277 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:56 smithi089 ceph-mon[30757]: pgmap v573: 129 pgs: 129 active+clean; 341 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 512 op/s 2024-09-10T08:45:57.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:56 smithi111 ceph-mon[35220]: pgmap v573: 129 pgs: 129 active+clean; 341 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 512 op/s 2024-09-10T08:45:58.071 INFO:tasks.workunit.client.0.smithi089.stdout: 105 502601 338 352894 416 290568 688 2024-09-10T08:45:58.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.704+0000 7f30dca00640 1 -- 172.21.15.89:0/3453760818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30d8074eb0 msgr2=0x7f30d8112480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:58.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.704+0000 7f30dca00640 1 --2- 172.21.15.89:0/3453760818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30d8074eb0 0x7f30d8112480 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009900 tx=0x7f30c402a4d0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.705+0000 7f30dca00640 1 -- 172.21.15.89:0/3453760818 shutdown_connections 2024-09-10T08:45:58.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.705+0000 7f30dca00640 1 --2- 172.21.15.89:0/3453760818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30d81129c0 0x7f30d8114db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.711 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.705+0000 7f30dca00640 1 --2- 172.21.15.89:0/3453760818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30d8074eb0 0x7f30d8112480 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.705+0000 7f30dca00640 1 --2- 172.21.15.89:0/3453760818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30d8074590 0x7f30d8074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.705+0000 7f30dca00640 1 -- 172.21.15.89:0/3453760818 >> 172.21.15.89:0/3453760818 conn(0x7f30d80fd6b0 msgr2=0x7f30d80ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:58.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.705+0000 7f30dca00640 1 -- 172.21.15.89:0/3453760818 shutdown_connections 2024-09-10T08:45:58.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.705+0000 7f30dca00640 1 -- 172.21.15.89:0/3453760818 wait complete. 2024-09-10T08:45:58.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.707+0000 7f30dca00640 1 Processor -- start 2024-09-10T08:45:58.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.707+0000 7f30dca00640 1 -- start start 2024-09-10T08:45:58.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.708+0000 7f30dca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30d8074590 0x7f30d80738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.708+0000 7f30dca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30d8074eb0 0x7f30d8073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.712 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.708+0000 7f30dca00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30d81129c0 0x7f30d806cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.709+0000 7f30d7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30d81129c0 0x7f30d806cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.709+0000 7f30d7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30d81129c0 0x7f30d806cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:33686/0 (socket says 172.21.15.89:33686) 2024-09-10T08:45:58.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.709+0000 7f30d7e00640 1 -- 172.21.15.89:0/1341380433 learned_addr learned my addr 172.21.15.89:0/1341380433 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:58.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.709+0000 7f30dca00640 1 -- 172.21.15.89:0/1341380433 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f30d8118460 con 0x7f30d8074590 2024-09-10T08:45:58.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.709+0000 7f30dca00640 1 -- 172.21.15.89:0/1341380433 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f30d81182e0 con 0x7f30d81129c0 2024-09-10T08:45:58.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.709+0000 7f30dca00640 1 -- 172.21.15.89:0/1341380433 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f30d81185e0 con 0x7f30d8074eb0 2024-09-10T08:45:58.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.709+0000 7f30d7400640 1 --2- 172.21.15.89:0/1341380433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30d8074590 0x7f30d80738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.713 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.710+0000 7f30d6a00640 1 --2- 172.21.15.89:0/1341380433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30d8074eb0 0x7f30d8073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.710+0000 7f30d7e00640 1 -- 172.21.15.89:0/1341380433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30d8074eb0 msgr2=0x7f30d8073e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:58.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.710+0000 7f30d7e00640 1 --2- 172.21.15.89:0/1341380433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30d8074eb0 0x7f30d8073e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.710+0000 7f30d7e00640 1 -- 172.21.15.89:0/1341380433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30d8074590 msgr2=0x7f30d80738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:58.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.710+0000 7f30d7e00640 1 --2- 172.21.15.89:0/1341380433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30d8074590 0x7f30d80738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.710+0000 7f30d7e00640 1 -- 172.21.15.89:0/1341380433 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d806d390 con 0x7f30d81129c0 2024-09-10T08:45:58.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.711+0000 7f30d7400640 1 --2- 172.21.15.89:0/1341380433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30d8074590 0x7f30d80738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:58.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.711+0000 7f30d7e00640 1 --2- 172.21.15.89:0/1341380433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30d81129c0 0x7f30d806cad0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f30d000dac0 tx=0x7f30d000df80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:58.714 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.711+0000 7f30cfe00640 1 -- 172.21.15.89:0/1341380433 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0017070 con 0x7f30d81129c0 2024-09-10T08:45:58.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.713+0000 7f30cfe00640 1 -- 172.21.15.89:0/1341380433 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30d000be30 con 0x7f30d81129c0 2024-09-10T08:45:58.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.713+0000 7f30dca00640 1 -- 172.21.15.89:0/1341380433 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d806d680 con 0x7f30d81129c0 2024-09-10T08:45:58.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.713+0000 7f30cfe00640 1 -- 172.21.15.89:0/1341380433 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d000f890 con 0x7f30d81129c0 2024-09-10T08:45:58.717 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.713+0000 7f30dca00640 1 -- 172.21.15.89:0/1341380433 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d80715d0 con 0x7f30d81129c0 2024-09-10T08:45:58.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.720+0000 7f30e09bf740 1 Processor -- start 2024-09-10T08:45:58.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.726+0000 7f30cfe00640 1 -- 172.21.15.89:0/1341380433 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f30d000fa30 con 0x7f30d81129c0 2024-09-10T08:45:58.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.728+0000 7f30e09bf740 1 -- start start 2024-09-10T08:45:58.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.729+0000 7f30e09bf740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b7dda0 0x55b6d5b7e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.729+0000 7f30e09bf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5b88a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.730+0000 7f30e09bf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b88f80 0x55b6d5b8b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.730+0000 7f30e09bf740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b6d5ae43e0 con 0x55b6d5b7dda0 2024-09-10T08:45:58.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.730+0000 7f30e09bf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b6d5a8bd80 con 0x55b6d5b88f80 2024-09-10T08:45:58.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.730+0000 7f30d6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b7dda0 0x55b6d5b7e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.730+0000 7f30cfe00640 1 --2- 172.21.15.89:0/1341380433 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30bc0a2c80 0x7f30bc0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.730+0000 7f30e09bf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b6d5ae03a0 con 0x55b6d5b7e6c0 2024-09-10T08:45:58.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.730+0000 7f30d7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5b88a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.730+0000 7f30d6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b7dda0 0x55b6d5b7e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40014/0 (socket says 172.21.15.89:40014) 2024-09-10T08:45:58.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.731+0000 7f30d6a00640 1 -- 172.21.15.89:0/1856100235 learned_addr learned my addr 172.21.15.89:0/1856100235 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:58.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.731+0000 7f30d6a00640 1 --2- 172.21.15.89:0/1856100235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b88f80 0x55b6d5b8b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.731+0000 7f30d7e00640 1 --2- 172.21.15.89:0/1341380433 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30bc0a2c80 0x7f30bc0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.732+0000 7f30d6a00640 1 -- 172.21.15.89:0/1856100235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b7e6c0 msgr2=0x55b6d5b88a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:58.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.732+0000 7f30d6a00640 1 --2- 172.21.15.89:0/1856100235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5b88a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.732+0000 7f30d6a00640 1 -- 172.21.15.89:0/1856100235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b7dda0 msgr2=0x55b6d5b7e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:58.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.732+0000 7f30d6a00640 1 --2- 172.21.15.89:0/1856100235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b7dda0 0x55b6d5b7e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.732+0000 7f30d6a00640 1 -- 172.21.15.89:0/1856100235 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6d5aaaae0 con 0x55b6d5b88f80 2024-09-10T08:45:58.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.732+0000 7f30d7e00640 1 --2- 172.21.15.89:0/1341380433 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30bc0a2c80 0x7f30bc0a5140 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f30d00cca10 tx=0x7f30d00d2a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:58.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.732+0000 7f30d6a00640 1 --2- 172.21.15.89:0/1856100235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b88f80 0x55b6d5b8b370 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f30c402aac0 tx=0x7f30c402a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74fd557af3fa66cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:58.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.732+0000 7f30cfe00640 1 -- 172.21.15.89:0/1341380433 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f30d00cf370 con 0x7f30d81129c0 2024-09-10T08:45:58.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.733+0000 7f30d7400640 1 --2- 172.21.15.89:0/1856100235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5b88a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:58.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.733+0000 7f30c3e00640 1 -- 172.21.15.89:0/1856100235 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4004040 con 0x55b6d5b88f80 2024-09-10T08:45:58.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.733+0000 7f30c3e00640 1 -- 172.21.15.89:0/1856100235 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c40054e0 con 0x55b6d5b88f80 2024-09-10T08:45:58.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.733+0000 7f30c3e00640 1 -- 172.21.15.89:0/1856100235 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4003a60 con 0x55b6d5b88f80 2024-09-10T08:45:58.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.735+0000 7f30e09bf740 1 -- 172.21.15.89:0/1856100235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b88f80 msgr2=0x55b6d5b8b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:58.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.735+0000 7f30e09bf740 1 --2- 172.21.15.89:0/1856100235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b88f80 0x55b6d5b8b370 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f30c402aac0 tx=0x7f30c402a4d0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.735+0000 7f30e09bf740 1 -- 172.21.15.89:0/1856100235 shutdown_connections 2024-09-10T08:45:58.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.735+0000 7f30e09bf740 1 --2- 172.21.15.89:0/1856100235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b88f80 0x55b6d5b8b370 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.735+0000 7f30e09bf740 1 --2- 172.21.15.89:0/1856100235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5b88a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.735+0000 7f30e09bf740 1 --2- 172.21.15.89:0/1856100235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b7dda0 0x55b6d5b7e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.735+0000 7f30e09bf740 1 -- 172.21.15.89:0/1856100235 >> 172.21.15.89:0/1856100235 conn(0x55b6d5b7a330 msgr2=0x55b6d5b7a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:58.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.735+0000 7f30e09bf740 1 -- 172.21.15.89:0/1856100235 shutdown_connections 2024-09-10T08:45:58.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.735+0000 7f30e09bf740 1 -- 172.21.15.89:0/1856100235 wait complete. 2024-09-10T08:45:58.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.736+0000 7f30e09bf740 1 -- start start 2024-09-10T08:45:58.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.737+0000 7f30e09bf740 1 Processor -- start 2024-09-10T08:45:58.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.738+0000 7f30e09bf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b7dda0 0x55b6d5b85dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.738+0000 7f30e09bf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5b86300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.738+0000 7f30e09bf740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b88f80 0x55b6d5b86840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.738+0000 7f30e09bf740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b6d5b8bc40 con 0x55b6d5b88f80 2024-09-10T08:45:58.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.738+0000 7f30e09bf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b6d5a9f0f0 con 0x55b6d5b7e6c0 2024-09-10T08:45:58.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.738+0000 7f30e09bf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b6d5b8bdc0 con 0x55b6d5b7dda0 2024-09-10T08:45:58.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.738+0000 7f30e09bf740 10 client.0 _ll_register_callbacks cb 0x55b6d5a909e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:58.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.739+0000 7f30d6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b7dda0 0x55b6d5b85dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.739+0000 7f30d6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b7dda0 0x55b6d5b85dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:36988/0 (socket says 172.21.15.89:36988) 2024-09-10T08:45:58.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.739+0000 7f30d7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5b86300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.739+0000 7f30d7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5b86300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:33704/0 (socket says 172.21.15.89:33704) 2024-09-10T08:45:58.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.739+0000 7f30d7400640 1 -- 172.21.15.89:0/3778222735 learned_addr learned my addr 172.21.15.89:0/3778222735 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:58.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.740+0000 7f30d6a00640 1 --2- 172.21.15.89:0/3778222735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b88f80 0x55b6d5b86840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.740+0000 7f30d7400640 1 -- 172.21.15.89:0/3778222735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b7dda0 msgr2=0x55b6d5b85dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:58.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.740+0000 7f30d7400640 1 --2- 172.21.15.89:0/3778222735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b7dda0 0x55b6d5b85dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.740+0000 7f30d7400640 1 -- 172.21.15.89:0/3778222735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b88f80 msgr2=0x55b6d5b86840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:58.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.740+0000 7f30d7400640 1 --2- 172.21.15.89:0/3778222735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b88f80 0x55b6d5b86840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.740+0000 7f30d7400640 1 -- 172.21.15.89:0/3778222735 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6d58553c0 con 0x55b6d5b7e6c0 2024-09-10T08:45:58.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.740+0000 7f30d6a00640 1 --2- 172.21.15.89:0/3778222735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b88f80 0x55b6d5b86840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:58.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.740+0000 7f30d7400640 1 --2- 172.21.15.89:0/3778222735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5b86300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f30c800f3e0 tx=0x7f30c800f8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=589358b580fde66b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:58.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.741+0000 7f30c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:45:58.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.743+0000 7f30c0c00640 1 -- 172.21.15.89:0/3778222735 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8011910 con 0x55b6d5b7e6c0 2024-09-10T08:45:58.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.743+0000 7f30c0c00640 1 -- 172.21.15.89:0/3778222735 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f30c8011ab0 con 0x55b6d5b7e6c0 2024-09-10T08:45:58.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.743+0000 7f30e09bf740 1 -- 172.21.15.89:0/3778222735 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30c40054e0 con 0x55b6d5b7e6c0 2024-09-10T08:45:58.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.746+0000 7f30c0c00640 1 -- 172.21.15.89:0/3778222735 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f30c8011cb0 con 0x55b6d5b7e6c0 2024-09-10T08:45:58.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.746+0000 7f30e09bf740 10 client.26594 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:58.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.746+0000 7f30e09bf740 1 -- 172.21.15.89:0/3778222735 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6d5b97860 con 0x55b6d5b7e6c0 2024-09-10T08:45:58.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.746+0000 7f30c0c00640 1 -- 172.21.15.89:0/3778222735 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c40054e0 con 0x55b6d5b7e6c0 2024-09-10T08:45:58.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.748+0000 7f30c0c00640 1 -- 172.21.15.89:0/3778222735 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f30c80104c0 con 0x55b6d5b7e6c0 2024-09-10T08:45:58.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.748+0000 7f30e09bf740 10 client.26594 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:58.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.748+0000 7f30e09bf740 10 client.26594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:45:58.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.748+0000 7f30e09bf740 20 client.26594 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:58.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.748+0000 7f30e09bf740 1 --2- 172.21.15.89:0/3778222735 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b6d5b98d20 0x55b6d5b9b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.748+0000 7f30e09bf740 4 client.26594 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:45:58.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.748+0000 7f30e09bf740 1 -- 172.21.15.89:0/3778222735 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6d5af1950 con 0x55b6d5b98d20 2024-09-10T08:45:58.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.749+0000 7f30d6a00640 1 --2- 172.21.15.89:0/3778222735 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b6d5b98d20 0x55b6d5b9b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.753+0000 7f30c0c00640 10 client.26594 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:58.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.753+0000 7f30d6a00640 1 --2- 172.21.15.89:0/3778222735 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b6d5b98d20 0x55b6d5b9b140 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f546578e6527086b server_cookie=f38d76854a8e5f0d in_seq=0 out_seq=0 2024-09-10T08:45:58.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.757+0000 7f30c0c00640 1 -- 172.21.15.89:0/3778222735 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6d5af1950 con 0x55b6d5b98d20 2024-09-10T08:45:58.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.757+0000 7f30c0c00640 10 client.26594 handle_command_reply: tid=0 2024-09-10T08:45:58.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.757+0000 7f30e09bf740 1 client.26594 shutdown 2024-09-10T08:45:58.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.758+0000 7f30e09bf740 1 -- 172.21.15.89:0/3778222735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b7e6c0 msgr2=0x55b6d5b86300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:58.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.758+0000 7f30e09bf740 1 --2- 172.21.15.89:0/3778222735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5b86300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f30c800f3e0 tx=0x7f30c800f8a0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.758+0000 7f30e09bf740 1 -- 172.21.15.89:0/3778222735 shutdown_connections 2024-09-10T08:45:58.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.758+0000 7f30e09bf740 1 --2- 172.21.15.89:0/3778222735 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b6d5b98d20 0x55b6d5b9b140 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.758+0000 7f30e09bf740 1 --2- 172.21.15.89:0/3778222735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b88f80 0x55b6d5b86840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.779 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.758+0000 7f30e09bf740 1 --2- 172.21.15.89:0/3778222735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5b86300 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.758+0000 7f30e09bf740 1 --2- 172.21.15.89:0/3778222735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b7dda0 0x55b6d5b85dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.758+0000 7f30e09bf740 1 -- 172.21.15.89:0/3778222735 >> 172.21.15.89:0/3778222735 conn(0x55b6d5b7a330 msgr2=0x55b6d5b80060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:58.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.758+0000 7f30c0c00640 0 client.26594 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:58.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.760+0000 7f30e09bf740 1 -- 172.21.15.89:0/3778222735 shutdown_connections 2024-09-10T08:45:58.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.760+0000 7f30e09bf740 1 -- 172.21.15.89:0/3778222735 wait complete. 2024-09-10T08:45:58.780 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.761+0000 7f30e09bf740 20 client.26594 trim_cache size 0 max 16384 2024-09-10T08:45:58.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.809+0000 7f30e09bf740 1 Processor -- start 2024-09-10T08:45:58.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.813+0000 7f30e09bf740 1 -- start start 2024-09-10T08:45:58.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.814+0000 7f30e09bf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b7dda0 0x55b6d5ba09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.824 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.814+0000 7f30e09bf740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5ba0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.814+0000 7f30d6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b7dda0 0x55b6d5ba09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.814+0000 7f30d6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b7dda0 0x55b6d5ba09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:36994/0 (socket says 172.21.15.89:36994) 2024-09-10T08:45:58.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.814+0000 7f30e09bf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b88f80 0x55b6d5b7f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.814+0000 7f30e09bf740 1 -- 172.21.15.89:0/1964065984 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f30d81c0380 con 0x55b6d5b7e6c0 2024-09-10T08:45:58.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.814+0000 7f30d6a00640 1 -- 172.21.15.89:0/1964065984 learned_addr learned my addr 172.21.15.89:0/1964065984 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:58.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.814+0000 7f30e09bf740 1 -- 172.21.15.89:0/1964065984 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f30d81bdc50 con 0x55b6d5b88f80 2024-09-10T08:45:58.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.814+0000 7f30e09bf740 1 -- 172.21.15.89:0/1964065984 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f30d81bbaa0 con 0x55b6d5b7dda0 2024-09-10T08:45:58.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.815+0000 7f30d7400640 1 --2- 172.21.15.89:0/1964065984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5ba0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.815+0000 7f30d6a00640 1 --2- 172.21.15.89:0/1964065984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b88f80 0x55b6d5b7f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.815+0000 7f30d6a00640 1 -- 172.21.15.89:0/1964065984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b88f80 msgr2=0x55b6d5b7f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:58.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.816+0000 7f30d6a00640 1 --2- 172.21.15.89:0/1964065984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b88f80 0x55b6d5b7f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.816+0000 7f30d6a00640 1 -- 172.21.15.89:0/1964065984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b7e6c0 msgr2=0x55b6d5ba0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:58.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.816+0000 7f30d6a00640 1 --2- 172.21.15.89:0/1964065984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5ba0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.816+0000 7f30d6a00640 1 -- 172.21.15.89:0/1964065984 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6d5b79dd0 con 0x55b6d5b7dda0 2024-09-10T08:45:58.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.816+0000 7f30d7400640 1 --2- 172.21.15.89:0/1964065984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5ba0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:58.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.817+0000 7f30d6a00640 1 --2- 172.21.15.89:0/1964065984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b7dda0 0x55b6d5ba09f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f30c402a9f0 tx=0x7f30c40047e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8b4f9487fe3f9b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:58.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.817+0000 7f30c3400640 1 -- 172.21.15.89:0/1964065984 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c40051b0 con 0x55b6d5b7dda0 2024-09-10T08:45:58.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.817+0000 7f30c3400640 1 -- 172.21.15.89:0/1964065984 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c4057040 con 0x55b6d5b7dda0 2024-09-10T08:45:58.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.817+0000 7f30c3400640 1 -- 172.21.15.89:0/1964065984 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c404c6a0 con 0x55b6d5b7dda0 2024-09-10T08:45:58.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.818+0000 7f30e09bf740 1 -- 172.21.15.89:0/1964065984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b7dda0 msgr2=0x55b6d5ba09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:58.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.818+0000 7f30e09bf740 1 --2- 172.21.15.89:0/1964065984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b7dda0 0x55b6d5ba09f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f30c402a9f0 tx=0x7f30c40047e0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.818+0000 7f30e09bf740 1 -- 172.21.15.89:0/1964065984 shutdown_connections 2024-09-10T08:45:58.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.818+0000 7f30e09bf740 1 --2- 172.21.15.89:0/1964065984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b88f80 0x55b6d5b7f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.819+0000 7f30e09bf740 1 --2- 172.21.15.89:0/1964065984 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5ba0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.819+0000 7f30e09bf740 1 --2- 172.21.15.89:0/1964065984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b7dda0 0x55b6d5ba09f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.819+0000 7f30e09bf740 1 -- 172.21.15.89:0/1964065984 >> 172.21.15.89:0/1964065984 conn(0x55b6d5b7a330 msgr2=0x55b6d5b84eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:58.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.819+0000 7f30e09bf740 1 -- 172.21.15.89:0/1964065984 shutdown_connections 2024-09-10T08:45:58.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.819+0000 7f30e09bf740 1 -- 172.21.15.89:0/1964065984 wait complete. 2024-09-10T08:45:58.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.820+0000 7f30e09bf740 1 -- start start 2024-09-10T08:45:58.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.820+0000 7f30e09bf740 1 Processor -- start 2024-09-10T08:45:58.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.824+0000 7f30e09bf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b7dda0 0x55b6d5b915d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.824+0000 7f30e09bf740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5b8c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.824+0000 7f30e09bf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b88f80 0x55b6d5b8ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.824+0000 7f30e09bf740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f30d81bc110 con 0x55b6d5b7e6c0 2024-09-10T08:45:58.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.824+0000 7f30e09bf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f30d81bd580 con 0x55b6d5b7dda0 2024-09-10T08:45:58.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.824+0000 7f30e09bf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f30d81bc7e0 con 0x55b6d5b88f80 2024-09-10T08:45:58.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.824+0000 7f30e09bf740 10 client.0 _ll_register_callbacks cb 0x7f30c8009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:58.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.825+0000 7f30d6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b7dda0 0x55b6d5b915d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.825+0000 7f30d6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b7dda0 0x55b6d5b915d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:33708/0 (socket says 172.21.15.89:33708) 2024-09-10T08:45:58.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.825+0000 7f30d6a00640 1 -- 172.21.15.89:0/857673183 learned_addr learned my addr 172.21.15.89:0/857673183 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:45:58.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.825+0000 7f30d7400640 1 --2- 172.21.15.89:0/857673183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5b8c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.826+0000 7f30d6a00640 1 --2- 172.21.15.89:0/857673183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b88f80 0x55b6d5b8ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.826+0000 7f30d7400640 1 -- 172.21.15.89:0/857673183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b88f80 msgr2=0x55b6d5b8ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:58.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.826+0000 7f30d7400640 1 --2- 172.21.15.89:0/857673183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b88f80 0x55b6d5b8ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.826+0000 7f30d7400640 1 -- 172.21.15.89:0/857673183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b7dda0 msgr2=0x55b6d5b915d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:58.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.826+0000 7f30d7400640 1 --2- 172.21.15.89:0/857673183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b7dda0 0x55b6d5b915d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.826+0000 7f30d7400640 1 -- 172.21.15.89:0/857673183 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6d5b8d480 con 0x55b6d5b7e6c0 2024-09-10T08:45:58.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.827+0000 7f30c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:45:58.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.828+0000 7f30d7400640 1 --2- 172.21.15.89:0/857673183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5b8c900 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f30c80108e0 tx=0x7f30c8011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f566abce799c230c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:58.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.828+0000 7f30c0c00640 1 -- 172.21.15.89:0/857673183 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c800fda0 con 0x55b6d5b7e6c0 2024-09-10T08:45:58.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.828+0000 7f30c0c00640 1 -- 172.21.15.89:0/857673183 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f30c8009350 con 0x55b6d5b7e6c0 2024-09-10T08:45:58.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.828+0000 7f30e09bf740 1 -- 172.21.15.89:0/857673183 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30c4057040 con 0x55b6d5b7e6c0 2024-09-10T08:45:58.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.829+0000 7f30c0c00640 1 -- 172.21.15.89:0/857673183 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f30c8009760 con 0x55b6d5b7e6c0 2024-09-10T08:45:58.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.829+0000 7f30e09bf740 10 client.16554 fetch_fsmap learned FSMap version 29 2024-09-10T08:45:58.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.829+0000 7f30e09bf740 1 -- 172.21.15.89:0/857673183 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6d5c03840 con 0x55b6d5b7e6c0 2024-09-10T08:45:58.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.830+0000 7f30c0c00640 1 -- 172.21.15.89:0/857673183 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c4057040 con 0x55b6d5b7e6c0 2024-09-10T08:45:58.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.830+0000 7f30c0c00640 1 -- 172.21.15.89:0/857673183 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f30c8004ba0 con 0x55b6d5b7e6c0 2024-09-10T08:45:58.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.830+0000 7f30e09bf740 10 client.16554 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:45:58.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.830+0000 7f30e09bf740 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:45:58.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.830+0000 7f30e09bf740 20 client.16554 populate_metadata read hostname 'smithi089' 2024-09-10T08:45:58.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.830+0000 7f30e09bf740 1 --2- 172.21.15.89:0/857673183 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b6d5c03ff0 0x55b6d5c06410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.830+0000 7f30e09bf740 4 client.16554 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:45:58.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.830+0000 7f30e09bf740 1 -- 172.21.15.89:0/857673183 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b6d5aebe60 con 0x55b6d5c03ff0 2024-09-10T08:45:58.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.833+0000 7f30d6a00640 1 --2- 172.21.15.89:0/857673183 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b6d5c03ff0 0x55b6d5c06410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.833+0000 7f30d6a00640 1 --2- 172.21.15.89:0/857673183 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b6d5c03ff0 0x55b6d5c06410 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bff819087289680f server_cookie=6bf04fe0ac58af5b in_seq=0 out_seq=0 2024-09-10T08:45:58.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.833+0000 7f30c0c00640 10 client.16554 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:58.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.834+0000 7f30c0c00640 1 -- 172.21.15.89:0/857673183 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b6d5aebe60 con 0x55b6d5c03ff0 2024-09-10T08:45:58.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.834+0000 7f30c0c00640 10 client.16554 handle_command_reply: tid=0 2024-09-10T08:45:58.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.834+0000 7f30e09bf740 1 client.16554 shutdown 2024-09-10T08:45:58.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.835+0000 7f30e09bf740 1 -- 172.21.15.89:0/857673183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b7e6c0 msgr2=0x55b6d5b8c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:58.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.835+0000 7f30e09bf740 1 --2- 172.21.15.89:0/857673183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5b8c900 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f30c80108e0 tx=0x7f30c8011940 comp rx=0 tx=0).stop 2024-09-10T08:45:58.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.835+0000 7f30e09bf740 1 -- 172.21.15.89:0/857673183 shutdown_connections 2024-09-10T08:45:58.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.835+0000 7f30e09bf740 1 --2- 172.21.15.89:0/857673183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b6d5b88f80 0x55b6d5b8ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.835+0000 7f30e09bf740 1 --2- 172.21.15.89:0/857673183 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b6d5c03ff0 0x55b6d5c06410 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.835+0000 7f30e09bf740 1 --2- 172.21.15.89:0/857673183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6d5b7e6c0 0x55b6d5b8c900 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.835+0000 7f30e09bf740 1 --2- 172.21.15.89:0/857673183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6d5b7dda0 0x55b6d5b915d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.835+0000 7f30e09bf740 1 -- 172.21.15.89:0/857673183 >> 172.21.15.89:0/857673183 conn(0x55b6d5b7a330 msgr2=0x55b6d5b7f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:58.838 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.835+0000 7f30c0c00640 0 client.16554 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:45:58.841 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:45:58.841 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2024-09-10T08:45:58.841 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2024-09-10T08:45:58.841 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:45:58.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.836+0000 7f30e09bf740 1 -- 172.21.15.89:0/857673183 shutdown_connections 2024-09-10T08:45:58.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.836+0000 7f30e09bf740 1 -- 172.21.15.89:0/857673183 wait complete. 2024-09-10T08:45:58.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.836+0000 7f30e09bf740 20 client.16554 trim_cache size 0 max 16384 2024-09-10T08:45:58.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.837+0000 7f30dca00640 1 -- 172.21.15.89:0/1341380433 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30bc0a2c80 msgr2=0x7f30bc0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:58.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.837+0000 7f30dca00640 1 --2- 172.21.15.89:0/1341380433 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30bc0a2c80 0x7f30bc0a5140 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f30d00cca10 tx=0x7f30d00d2a40 comp rx=0 tx=0).stop 2024-09-10T08:45:58.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.837+0000 7f30dca00640 1 -- 172.21.15.89:0/1341380433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30d81129c0 msgr2=0x7f30d806cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:58.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.837+0000 7f30dca00640 1 --2- 172.21.15.89:0/1341380433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30d81129c0 0x7f30d806cad0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f30d000dac0 tx=0x7f30d000df80 comp rx=0 tx=0).stop 2024-09-10T08:45:58.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.838+0000 7f30dca00640 1 -- 172.21.15.89:0/1341380433 shutdown_connections 2024-09-10T08:45:58.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.838+0000 7f30dca00640 1 --2- 172.21.15.89:0/1341380433 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f30bc0a2c80 0x7f30bc0a5140 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.838+0000 7f30dca00640 1 --2- 172.21.15.89:0/1341380433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30d81129c0 0x7f30d806cad0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.838+0000 7f30dca00640 1 --2- 172.21.15.89:0/1341380433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30d8074eb0 0x7f30d8073e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.838+0000 7f30dca00640 1 --2- 172.21.15.89:0/1341380433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30d8074590 0x7f30d80738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.838+0000 7f30dca00640 1 -- 172.21.15.89:0/1341380433 >> 172.21.15.89:0/1341380433 conn(0x7f30d80fd6b0 msgr2=0x7f30d80fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:58.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.840+0000 7f30dca00640 1 -- 172.21.15.89:0/1341380433 shutdown_connections 2024-09-10T08:45:58.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:45:58.840+0000 7f30dca00640 1 -- 172.21.15.89:0/1341380433 wait complete. 2024-09-10T08:45:59.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:45:58 smithi149 ceph-mon[36167]: pgmap v574: 129 pgs: 129 active+clean; 345 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 742 op/s 2024-09-10T08:45:59.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:45:58 smithi111 ceph-mon[35220]: pgmap v574: 129 pgs: 129 active+clean; 345 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 742 op/s 2024-09-10T08:45:59.305 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:45:58 smithi089 ceph-mon[30757]: pgmap v574: 129 pgs: 129 active+clean; 345 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 742 op/s 2024-09-10T08:45:59.908 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:45:59.908 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:46:01.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:00 smithi149 ceph-mon[36167]: pgmap v575: 129 pgs: 129 active+clean; 346 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 772 op/s 2024-09-10T08:46:01.258 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:00 smithi089 ceph-mon[30757]: pgmap v575: 129 pgs: 129 active+clean; 346 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 772 op/s 2024-09-10T08:46:01.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:00 smithi111 ceph-mon[35220]: pgmap v575: 129 pgs: 129 active+clean; 346 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 772 op/s 2024-09-10T08:46:02.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:01 smithi111 ceph-mon[35220]: pgmap v576: 129 pgs: 129 active+clean; 343 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 926 op/s 2024-09-10T08:46:02.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:01 smithi089 ceph-mon[30757]: pgmap v576: 129 pgs: 129 active+clean; 343 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 926 op/s 2024-09-10T08:46:02.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:01 smithi149 ceph-mon[36167]: pgmap v576: 129 pgs: 129 active+clean; 343 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 926 op/s 2024-09-10T08:46:03.253 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:05.080 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:04 smithi089 ceph-mon[30757]: pgmap v577: 129 pgs: 129 active+clean; 339 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 813 op/s 2024-09-10T08:46:05.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:04 smithi149 ceph-mon[36167]: pgmap v577: 129 pgs: 129 active+clean; 339 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 813 op/s 2024-09-10T08:46:05.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.269+0000 7fa068c00640 1 -- 172.21.15.89:0/245658877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0641047e0 msgr2=0x7fa064108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:05.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.269+0000 7fa068c00640 1 --2- 172.21.15.89:0/245658877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0641047e0 0x7fa064108830 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa05001a910 tx=0x7fa05003b430 comp rx=0 tx=0).stop 2024-09-10T08:46:05.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.269+0000 7fa068c00640 1 -- 172.21.15.89:0/245658877 shutdown_connections 2024-09-10T08:46:05.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.269+0000 7fa068c00640 1 --2- 172.21.15.89:0/245658877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa064108f60 0x7fa064110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.269+0000 7fa068c00640 1 --2- 172.21.15.89:0/245658877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0641047e0 0x7fa064108830 secure :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa05001a910 tx=0x7fa05003b430 comp rx=0 tx=0).stop 2024-09-10T08:46:05.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.269+0000 7fa068c00640 1 --2- 172.21.15.89:0/245658877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa064103e30 0x7fa064104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.270+0000 7fa068c00640 1 -- 172.21.15.89:0/245658877 >> 172.21.15.89:0/245658877 conn(0x7fa0640ff910 msgr2=0x7fa064101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:05.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.271+0000 7fa068c00640 1 -- 172.21.15.89:0/245658877 shutdown_connections 2024-09-10T08:46:05.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.271+0000 7fa068c00640 1 -- 172.21.15.89:0/245658877 wait complete. 2024-09-10T08:46:05.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.272+0000 7fa068c00640 1 Processor -- start 2024-09-10T08:46:05.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.274+0000 7fa068c00640 1 -- start start 2024-09-10T08:46:05.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.275+0000 7fa068c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa064103e30 0x7fa064110840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.275+0000 7fa068c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa064108f60 0x7fa06410c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.275+0000 7fa068c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa06410ce20 0x7fa06410d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.276+0000 7fa062a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa064108f60 0x7fa06410c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.276+0000 7fa062a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa064108f60 0x7fa06410c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:45488/0 (socket says 172.21.15.89:45488) 2024-09-10T08:46:05.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.276+0000 7fa068c00640 1 -- 172.21.15.89:0/2469712967 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa06406e5a0 con 0x7fa06410ce20 2024-09-10T08:46:05.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.276+0000 7fa068c00640 1 -- 172.21.15.89:0/2469712967 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa06406e420 con 0x7fa064103e30 2024-09-10T08:46:05.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.276+0000 7fa068c00640 1 -- 172.21.15.89:0/2469712967 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa06406e720 con 0x7fa064108f60 2024-09-10T08:46:05.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.276+0000 7fa062a00640 1 -- 172.21.15.89:0/2469712967 learned_addr learned my addr 172.21.15.89:0/2469712967 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:05.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.276+0000 7fa063400640 1 --2- 172.21.15.89:0/2469712967 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa064103e30 0x7fa064110840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.277+0000 7fa063e00640 1 --2- 172.21.15.89:0/2469712967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa06410ce20 0x7fa06410d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.277+0000 7fa063400640 1 -- 172.21.15.89:0/2469712967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa064108f60 msgr2=0x7fa06410c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:05.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.278+0000 7fa063400640 1 --2- 172.21.15.89:0/2469712967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa064108f60 0x7fa06410c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.278+0000 7fa063400640 1 -- 172.21.15.89:0/2469712967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa06410ce20 msgr2=0x7fa06410d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:05.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.278+0000 7fa063400640 1 --2- 172.21.15.89:0/2469712967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa06410ce20 0x7fa06410d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.278+0000 7fa063400640 1 -- 172.21.15.89:0/2469712967 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06419e460 con 0x7fa064103e30 2024-09-10T08:46:05.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.278+0000 7fa062a00640 1 --2- 172.21.15.89:0/2469712967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa064108f60 0x7fa06410c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:46:05.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.279+0000 7fa063400640 1 --2- 172.21.15.89:0/2469712967 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa064103e30 0x7fa064110840 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa05400e9f0 tx=0x7fa05400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:05.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.279+0000 7fa05be00640 1 -- 172.21.15.89:0/2469712967 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05400ccc0 con 0x7fa064103e30 2024-09-10T08:46:05.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.279+0000 7fa05be00640 1 -- 172.21.15.89:0/2469712967 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa054010910 con 0x7fa064103e30 2024-09-10T08:46:05.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.279+0000 7fa05be00640 1 -- 172.21.15.89:0/2469712967 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054018990 con 0x7fa064103e30 2024-09-10T08:46:05.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.281+0000 7fa068c00640 1 -- 172.21.15.89:0/2469712967 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06419e750 con 0x7fa064103e30 2024-09-10T08:46:05.282 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.281+0000 7fa068c00640 1 -- 172.21.15.89:0/2469712967 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa06419ec40 con 0x7fa064103e30 2024-09-10T08:46:05.286 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.282+0000 7fa068c00640 1 -- 172.21.15.89:0/2469712967 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa064105310 con 0x7fa064103e30 2024-09-10T08:46:05.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:04 smithi111 ceph-mon[35220]: pgmap v577: 129 pgs: 129 active+clean; 339 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 813 op/s 2024-09-10T08:46:05.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.289+0000 7fa05be00640 1 -- 172.21.15.89:0/2469712967 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa054010ac0 con 0x7fa064103e30 2024-09-10T08:46:05.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.295+0000 7fa05be00640 1 --2- 172.21.15.89:0/2469712967 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa0480a2d50 0x7fa0480a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.295+0000 7fa05be00640 1 -- 172.21.15.89:0/2469712967 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa0540cfbc0 con 0x7fa064103e30 2024-09-10T08:46:05.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.296+0000 7fa05be00640 1 -- 172.21.15.89:0/2469712967 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa0540fe1b0 con 0x7fa064103e30 2024-09-10T08:46:05.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.298+0000 7fa062a00640 1 --2- 172.21.15.89:0/2469712967 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa0480a2d50 0x7fa0480a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.324 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.302+0000 7fa062a00640 1 --2- 172.21.15.89:0/2469712967 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa0480a2d50 0x7fa0480a5210 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa06410e0a0 tx=0x7fa0500405d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:05.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.746+0000 7fa068c00640 1 -- 172.21.15.89:0/2469712967 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa064108830 con 0x7fa064103e30 2024-09-10T08:46:05.749 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:46:05.749 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"btime":"2024-09-10T08:44:40:778845+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:44:40.174532+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24496,24518,24554]},"id":1}]} 2024-09-10T08:46:05.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.747+0000 7fa05be00640 1 -- 172.21.15.89:0/2469712967 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11919 (secure 0 0 0) 0x7fa064108830 con 0x7fa064103e30 2024-09-10T08:46:05.749 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 29 2024-09-10T08:46:05.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.751+0000 7fa068c00640 1 -- 172.21.15.89:0/2469712967 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa0480a2d50 msgr2=0x7fa0480a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:05.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.751+0000 7fa068c00640 1 --2- 172.21.15.89:0/2469712967 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa0480a2d50 0x7fa0480a5210 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa06410e0a0 tx=0x7fa0500405d0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.751+0000 7fa068c00640 1 -- 172.21.15.89:0/2469712967 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa064103e30 msgr2=0x7fa064110840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:05.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.751+0000 7fa068c00640 1 --2- 172.21.15.89:0/2469712967 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa064103e30 0x7fa064110840 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa05400e9f0 tx=0x7fa05400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.752+0000 7fa068c00640 1 -- 172.21.15.89:0/2469712967 shutdown_connections 2024-09-10T08:46:05.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.752+0000 7fa068c00640 1 --2- 172.21.15.89:0/2469712967 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa0480a2d50 0x7fa0480a5210 secure :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa06410e0a0 tx=0x7fa0500405d0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.752+0000 7fa068c00640 1 --2- 172.21.15.89:0/2469712967 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa06410ce20 0x7fa06410d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.752+0000 7fa068c00640 1 --2- 172.21.15.89:0/2469712967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa064108f60 0x7fa06410c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.752+0000 7fa068c00640 1 --2- 172.21.15.89:0/2469712967 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa064103e30 0x7fa064110840 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.754 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.752+0000 7fa068c00640 1 -- 172.21.15.89:0/2469712967 >> 172.21.15.89:0/2469712967 conn(0x7fa0640ff910 msgr2=0x7fa064100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:05.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.753+0000 7fa068c00640 1 -- 172.21.15.89:0/2469712967 shutdown_connections 2024-09-10T08:46:05.755 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:05.753+0000 7fa068c00640 1 -- 172.21.15.89:0/2469712967 wait complete. 2024-09-10T08:46:06.057 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:05 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2469712967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:06.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:05 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2469712967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:06.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:05 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2469712967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:07.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:06 smithi149 ceph-mon[36167]: pgmap v578: 129 pgs: 129 active+clean; 337 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 816 op/s 2024-09-10T08:46:07.282 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:06 smithi089 ceph-mon[30757]: pgmap v578: 129 pgs: 129 active+clean; 337 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 816 op/s 2024-09-10T08:46:07.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:06 smithi111 ceph-mon[35220]: pgmap v578: 129 pgs: 129 active+clean; 337 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 816 op/s 2024-09-10T08:46:07.340 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:46:08.071 INFO:tasks.workunit.client.0.smithi089.stdout: 112 588846 486 419698 369 360474 1480 2024-09-10T08:46:09.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:08 smithi149 ceph-mon[36167]: pgmap v579: 129 pgs: 129 active+clean; 344 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 542 op/s 2024-09-10T08:46:09.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:08 smithi111 ceph-mon[35220]: pgmap v579: 129 pgs: 129 active+clean; 344 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 542 op/s 2024-09-10T08:46:09.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:08 smithi089 ceph-mon[30757]: pgmap v579: 129 pgs: 129 active+clean; 344 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 542 op/s 2024-09-10T08:46:11.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:10 smithi149 ceph-mon[36167]: pgmap v580: 129 pgs: 129 active+clean; 344 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 294 op/s 2024-09-10T08:46:11.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:10 smithi111 ceph-mon[35220]: pgmap v580: 129 pgs: 129 active+clean; 344 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 294 op/s 2024-09-10T08:46:11.307 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:10 smithi089 ceph-mon[30757]: pgmap v580: 129 pgs: 129 active+clean; 344 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 294 op/s 2024-09-10T08:46:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.308+0000 7f2378a00640 1 -- 172.21.15.89:0/2994524324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2374073680 msgr2=0x7f237410d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.308+0000 7f2378a00640 1 --2- 172.21.15.89:0/2994524324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2374073680 0x7f237410d4f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f236c009c30 tx=0x7f236c02a720 comp rx=0 tx=0).stop 2024-09-10T08:46:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.335+0000 7f2378a00640 1 -- 172.21.15.89:0/2994524324 shutdown_connections 2024-09-10T08:46:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.335+0000 7f2378a00640 1 --2- 172.21.15.89:0/2994524324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f237410dbb0 0x7f2374110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.335+0000 7f2378a00640 1 --2- 172.21.15.89:0/2994524324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2374073680 0x7f237410d4f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.335+0000 7f2378a00640 1 --2- 172.21.15.89:0/2994524324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2374072cb0 0x7f23740730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.335+0000 7f2378a00640 1 -- 172.21.15.89:0/2994524324 >> 172.21.15.89:0/2994524324 conn(0x7f237406eb50 msgr2=0x7f2374070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.349+0000 7f2378a00640 1 -- 172.21.15.89:0/2994524324 shutdown_connections 2024-09-10T08:46:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.354+0000 7f2378a00640 1 -- 172.21.15.89:0/2994524324 wait complete. 2024-09-10T08:46:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.355+0000 7f2378a00640 1 Processor -- start 2024-09-10T08:46:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.362+0000 7f2378a00640 1 -- start start 2024-09-10T08:46:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.362+0000 7f2378a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2374072cb0 0x7f2374115950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.362+0000 7f2378a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2374073680 0x7f2374115e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.362+0000 7f2378a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f237410dbb0 0x7f23741b6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.362+0000 7f2378a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f237411c400 con 0x7f2374073680 2024-09-10T08:46:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.362+0000 7f2378a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f237411c280 con 0x7f2374072cb0 2024-09-10T08:46:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.362+0000 7f2378a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f237411c580 con 0x7f237410dbb0 2024-09-10T08:46:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.363+0000 7f2371e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2374073680 0x7f2374115e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.363+0000 7f2371e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2374073680 0x7f2374115e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47866/0 (socket says 172.21.15.89:47866) 2024-09-10T08:46:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.363+0000 7f2371e00640 1 -- 172.21.15.89:0/849306317 learned_addr learned my addr 172.21.15.89:0/849306317 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.364+0000 7f2371e00640 1 -- 172.21.15.89:0/849306317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f237410dbb0 msgr2=0x7f23741b6b10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:46:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.364+0000 7f2371e00640 1 --2- 172.21.15.89:0/849306317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f237410dbb0 0x7f23741b6b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.364+0000 7f2371e00640 1 -- 172.21.15.89:0/849306317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2374072cb0 msgr2=0x7f2374115950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:46:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.364+0000 7f2371e00640 1 --2- 172.21.15.89:0/849306317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2374072cb0 0x7f2374115950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.364+0000 7f2371e00640 1 -- 172.21.15.89:0/849306317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23741b7050 con 0x7f2374073680 2024-09-10T08:46:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.365+0000 7f2371e00640 1 --2- 172.21.15.89:0/849306317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2374073680 0x7f2374115e90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f236c009b80 tx=0x7f236c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.369+0000 7f236b400640 1 -- 172.21.15.89:0/849306317 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c041020 con 0x7f2374073680 2024-09-10T08:46:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.369+0000 7f236b400640 1 -- 172.21.15.89:0/849306317 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f236c037040 con 0x7f2374073680 2024-09-10T08:46:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.369+0000 7f236b400640 1 -- 172.21.15.89:0/849306317 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c036070 con 0x7f2374073680 2024-09-10T08:46:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.377+0000 7f2378a00640 1 -- 172.21.15.89:0/849306317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23741b7220 con 0x7f2374073680 2024-09-10T08:46:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.377+0000 7f2378a00640 1 -- 172.21.15.89:0/849306317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23741b7690 con 0x7f2374073680 2024-09-10T08:46:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.398+0000 7f237c437740 1 Processor -- start 2024-09-10T08:46:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.403+0000 7f236b400640 1 -- 172.21.15.89:0/849306317 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f236c044020 con 0x7f2374073680 2024-09-10T08:46:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.405+0000 7f236b400640 1 --2- 172.21.15.89:0/849306317 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23580a2d90 0x7f23580a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.405+0000 7f237c437740 1 -- start start 2024-09-10T08:46:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.406+0000 7f237c437740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa04b61280 0x55aa04b61680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.406+0000 7f237c437740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61bc0 0x55aa04b66680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.406+0000 7f237c437740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aa04b66c50 0x55aa04b69110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.406+0000 7f237c437740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55aa04ac6100 con 0x55aa04b61bc0 2024-09-10T08:46:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.406+0000 7f237c437740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55aa04a6e1b0 con 0x55aa04b66c50 2024-09-10T08:46:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.406+0000 7f237c437740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55aa04ac20c0 con 0x55aa04b61280 2024-09-10T08:46:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.407+0000 7f2371e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61bc0 0x55aa04b66680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.407+0000 7f2371e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61bc0 0x55aa04b66680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47876/0 (socket says 172.21.15.89:47876) 2024-09-10T08:46:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.407+0000 7f2371e00640 1 -- 172.21.15.89:0/2232828696 learned_addr learned my addr 172.21.15.89:0/2232828696 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.408+0000 7f2371e00640 1 -- 172.21.15.89:0/2232828696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa04b61280 msgr2=0x55aa04b61680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:46:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.408+0000 7f2371e00640 1 --2- 172.21.15.89:0/2232828696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa04b61280 0x55aa04b61680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.408+0000 7f2371e00640 1 -- 172.21.15.89:0/2232828696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aa04b66c50 msgr2=0x55aa04b69110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:46:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.408+0000 7f2371e00640 1 --2- 172.21.15.89:0/2232828696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aa04b66c50 0x55aa04b69110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.408+0000 7f2371e00640 1 -- 172.21.15.89:0/2232828696 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa047d9210 con 0x55aa04b61bc0 2024-09-10T08:46:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.408+0000 7f2371e00640 1 --2- 172.21.15.89:0/2232828696 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61bc0 0x55aa04b66680 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f236c0f3dd0 tx=0x7f236c0f3ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45291a0a0635f26e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.409+0000 7f236b400640 1 -- 172.21.15.89:0/849306317 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f236c0efed0 con 0x7f2374073680 2024-09-10T08:46:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.409+0000 7f2373200640 1 --2- 172.21.15.89:0/849306317 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23580a2d90 0x7f23580a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.414+0000 7f2378a00640 1 -- 172.21.15.89:0/2232828696 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c0bbd20 con 0x55aa04b61bc0 2024-09-10T08:46:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.414+0000 7f2378a00640 1 -- 172.21.15.89:0/2232828696 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f236c0bcac0 con 0x55aa04b61bc0 2024-09-10T08:46:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.414+0000 7f2378a00640 1 -- 172.21.15.89:0/2232828696 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c0bbf00 con 0x55aa04b61bc0 2024-09-10T08:46:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.414+0000 7f2373200640 1 --2- 172.21.15.89:0/849306317 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23580a2d90 0x7f23580a5250 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2364005dc0 tx=0x7f2364005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.425+0000 7f237c437740 1 -- 172.21.15.89:0/2232828696 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61bc0 msgr2=0x55aa04b66680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.426+0000 7f237c437740 1 --2- 172.21.15.89:0/2232828696 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61bc0 0x55aa04b66680 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f236c0f3dd0 tx=0x7f236c0f3ec0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.430+0000 7f237c437740 1 -- 172.21.15.89:0/2232828696 shutdown_connections 2024-09-10T08:46:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.430+0000 7f237c437740 1 --2- 172.21.15.89:0/2232828696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aa04b66c50 0x55aa04b69110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.430+0000 7f237c437740 1 --2- 172.21.15.89:0/2232828696 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61bc0 0x55aa04b66680 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.430+0000 7f237c437740 1 --2- 172.21.15.89:0/2232828696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa04b61280 0x55aa04b61680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.430+0000 7f237c437740 1 -- 172.21.15.89:0/2232828696 >> 172.21.15.89:0/2232828696 conn(0x55aa04b5cbe0 msgr2=0x55aa04b5cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.433+0000 7f237c437740 1 -- 172.21.15.89:0/2232828696 shutdown_connections 2024-09-10T08:46:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.441+0000 7f237c437740 1 -- 172.21.15.89:0/2232828696 wait complete. 2024-09-10T08:46:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.441+0000 7f237c437740 1 -- start start 2024-09-10T08:46:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.442+0000 7f237c437740 1 Processor -- start 2024-09-10T08:46:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.450+0000 7f237c437740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa04b61280 0x55aa04b6b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.450+0000 7f237c437740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61bc0 0x55aa04b6b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.450+0000 7f237c437740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aa04b66c50 0x55aa04b725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.450+0000 7f237c437740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55aa04ac6100 con 0x55aa04b61bc0 2024-09-10T08:46:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.450+0000 7f237c437740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55aa04a4f090 con 0x55aa04b66c50 2024-09-10T08:46:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.450+0000 7f237c437740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55aa04b699e0 con 0x55aa04b61280 2024-09-10T08:46:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.450+0000 7f237c437740 10 client.0 _ll_register_callbacks cb 0x55aa04aa3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.451+0000 7f2371e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61bc0 0x55aa04b6b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.451+0000 7f2371e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61bc0 0x55aa04b6b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47888/0 (socket says 172.21.15.89:47888) 2024-09-10T08:46:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.451+0000 7f2371e00640 1 -- 172.21.15.89:0/4108814969 learned_addr learned my addr 172.21.15.89:0/4108814969 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.452+0000 7f2371e00640 1 -- 172.21.15.89:0/4108814969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa04b61280 msgr2=0x55aa04b6b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.452+0000 7f2371e00640 1 --2- 172.21.15.89:0/4108814969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa04b61280 0x55aa04b6b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.452+0000 7f2371e00640 1 -- 172.21.15.89:0/4108814969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aa04b66c50 msgr2=0x55aa04b725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.452+0000 7f2371e00640 1 --2- 172.21.15.89:0/4108814969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aa04b66c50 0x55aa04b725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.452+0000 7f2371e00640 1 -- 172.21.15.89:0/4108814969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa048a5620 con 0x55aa04b61bc0 2024-09-10T08:46:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.453+0000 7f2371e00640 1 --2- 172.21.15.89:0/4108814969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61bc0 0x55aa04b6b570 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f236c0f3da0 tx=0x7f236c02a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7619dacc5e71a600 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.458+0000 7f2353e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:46:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.458+0000 7f2353e00640 1 -- 172.21.15.89:0/4108814969 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c0bc190 con 0x55aa04b61bc0 2024-09-10T08:46:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.458+0000 7f2353e00640 1 -- 172.21.15.89:0/4108814969 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f236c0ba6c0 con 0x55aa04b61bc0 2024-09-10T08:46:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.462+0000 7f237c437740 1 -- 172.21.15.89:0/4108814969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f236c0bcac0 con 0x55aa04b61bc0 2024-09-10T08:46:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.477+0000 7f2353e00640 1 -- 172.21.15.89:0/4108814969 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f236c0b8660 con 0x55aa04b61bc0 2024-09-10T08:46:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.482+0000 7f237c437740 10 client.16569 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.482+0000 7f237c437740 1 -- 172.21.15.89:0/4108814969 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aa04b753a0 con 0x55aa04b61bc0 2024-09-10T08:46:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.483+0000 7f2353e00640 1 -- 172.21.15.89:0/4108814969 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f236c0bcac0 con 0x55aa04b61bc0 2024-09-10T08:46:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.483+0000 7f2353e00640 1 -- 172.21.15.89:0/4108814969 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f236c0432c0 con 0x55aa04b61bc0 2024-09-10T08:46:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.484+0000 7f237c437740 10 client.16569 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.484+0000 7f237c437740 10 client.16569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:46:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.484+0000 7f237c437740 20 client.16569 populate_metadata read hostname 'smithi089' 2024-09-10T08:46:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.484+0000 7f237c437740 1 --2- 172.21.15.89:0/4108814969 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55aa04b76a10 0x55aa04b78e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.484+0000 7f237c437740 4 client.16569 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.484+0000 7f237c437740 1 -- 172.21.15.89:0/4108814969 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa04ad3830 con 0x55aa04b76a10 2024-09-10T08:46:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.484+0000 7f2372800640 1 --2- 172.21.15.89:0/4108814969 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55aa04b76a10 0x55aa04b78e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.485+0000 7f2372800640 1 --2- 172.21.15.89:0/4108814969 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55aa04b76a10 0x55aa04b78e80 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae6c1d0bbe2007ed server_cookie=41345aa7a3873466 in_seq=0 out_seq=0 2024-09-10T08:46:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.485+0000 7f2353e00640 10 client.16569 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.493+0000 7f2353e00640 1 -- 172.21.15.89:0/4108814969 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aa04ad3830 con 0x55aa04b76a10 2024-09-10T08:46:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.493+0000 7f2353e00640 10 client.16569 handle_command_reply: tid=0 2024-09-10T08:46:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.493+0000 7f237c437740 1 client.16569 shutdown 2024-09-10T08:46:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.494+0000 7f237c437740 1 -- 172.21.15.89:0/4108814969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61bc0 msgr2=0x55aa04b6b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.494+0000 7f237c437740 1 --2- 172.21.15.89:0/4108814969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61bc0 0x55aa04b6b570 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f236c0f3da0 tx=0x7f236c02a720 comp rx=0 tx=0).stop 2024-09-10T08:46:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.494+0000 7f237c437740 1 -- 172.21.15.89:0/4108814969 shutdown_connections 2024-09-10T08:46:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.494+0000 7f237c437740 1 --2- 172.21.15.89:0/4108814969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aa04b66c50 0x55aa04b725b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.494+0000 7f237c437740 1 --2- 172.21.15.89:0/4108814969 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55aa04b76a10 0x55aa04b78e80 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.494+0000 7f237c437740 1 --2- 172.21.15.89:0/4108814969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61bc0 0x55aa04b6b570 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.494+0000 7f237c437740 1 --2- 172.21.15.89:0/4108814969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa04b61280 0x55aa04b6b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.494+0000 7f237c437740 1 -- 172.21.15.89:0/4108814969 >> 172.21.15.89:0/4108814969 conn(0x55aa04b5cbe0 msgr2=0x55aa04b5e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.494+0000 7f2353e00640 0 client.16569 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.502+0000 7f237c437740 1 -- 172.21.15.89:0/4108814969 shutdown_connections 2024-09-10T08:46:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.505+0000 7f237c437740 1 -- 172.21.15.89:0/4108814969 wait complete. 2024-09-10T08:46:12.515 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.512+0000 7f237c437740 20 client.16569 trim_cache size 0 max 16384 2024-09-10T08:46:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.764+0000 7f237c437740 1 Processor -- start 2024-09-10T08:46:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.768+0000 7f237c437740 1 -- start start 2024-09-10T08:46:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.768+0000 7f237c437740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aa04b61280 0x55aa04b5da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.768+0000 7f237c437740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa04b61bc0 0x55aa04b72570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.768+0000 7f237c437740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b66c50 0x55aa04b5dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.769+0000 7f237c437740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f23741c4620 con 0x55aa04b66c50 2024-09-10T08:46:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.769+0000 7f237c437740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f23741c1ef0 con 0x55aa04b61280 2024-09-10T08:46:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.769+0000 7f237c437740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f23741bfd40 con 0x55aa04b61bc0 2024-09-10T08:46:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.772+0000 7f2371e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa04b61bc0 0x55aa04b72570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.772+0000 7f2371e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa04b61bc0 0x55aa04b72570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:45526/0 (socket says 172.21.15.89:45526) 2024-09-10T08:46:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.772+0000 7f2371e00640 1 -- 172.21.15.89:0/2580809797 learned_addr learned my addr 172.21.15.89:0/2580809797 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.772+0000 7f2372800640 1 --2- 172.21.15.89:0/2580809797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aa04b61280 0x55aa04b5da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.772+0000 7f2371e00640 1 -- 172.21.15.89:0/2580809797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aa04b61280 msgr2=0x55aa04b5da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.772+0000 7f2371e00640 1 --2- 172.21.15.89:0/2580809797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aa04b61280 0x55aa04b5da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.772+0000 7f2371e00640 1 -- 172.21.15.89:0/2580809797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b66c50 msgr2=0x55aa04b5dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.772+0000 7f2371e00640 1 --2- 172.21.15.89:0/2580809797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b66c50 0x55aa04b5dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.772+0000 7f2371e00640 1 -- 172.21.15.89:0/2580809797 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa04b79500 con 0x55aa04b61bc0 2024-09-10T08:46:12.773 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.773+0000 7f2371e00640 1 --2- 172.21.15.89:0/2580809797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa04b61bc0 0x55aa04b72570 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f236c0f3dd0 tx=0x7f236c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=673d32a5289cc4d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.777+0000 7f235ea00640 1 -- 172.21.15.89:0/2580809797 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c0bbd20 con 0x55aa04b61bc0 2024-09-10T08:46:12.783 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.777+0000 7f235ea00640 1 -- 172.21.15.89:0/2580809797 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f236c031b20 con 0x55aa04b61bc0 2024-09-10T08:46:12.783 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.777+0000 7f235ea00640 1 -- 172.21.15.89:0/2580809797 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c004720 con 0x55aa04b61bc0 2024-09-10T08:46:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.789+0000 7f237c437740 1 -- 172.21.15.89:0/2580809797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa04b61bc0 msgr2=0x55aa04b72570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.789+0000 7f237c437740 1 --2- 172.21.15.89:0/2580809797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa04b61bc0 0x55aa04b72570 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f236c0f3dd0 tx=0x7f236c004ab0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.799+0000 7f237c437740 1 -- 172.21.15.89:0/2580809797 shutdown_connections 2024-09-10T08:46:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.799+0000 7f237c437740 1 --2- 172.21.15.89:0/2580809797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b66c50 0x55aa04b5dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.799+0000 7f237c437740 1 --2- 172.21.15.89:0/2580809797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa04b61bc0 0x55aa04b72570 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.799+0000 7f237c437740 1 --2- 172.21.15.89:0/2580809797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aa04b61280 0x55aa04b5da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.799+0000 7f237c437740 1 -- 172.21.15.89:0/2580809797 >> 172.21.15.89:0/2580809797 conn(0x55aa04b5cbe0 msgr2=0x55aa04b62e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.815+0000 7f237c437740 1 -- 172.21.15.89:0/2580809797 shutdown_connections 2024-09-10T08:46:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.819+0000 7f237c437740 1 -- 172.21.15.89:0/2580809797 wait complete. 2024-09-10T08:46:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.819+0000 7f237c437740 1 -- start start 2024-09-10T08:46:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.819+0000 7f237c437740 1 Processor -- start 2024-09-10T08:46:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.821+0000 7f237c437740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61280 0x55aa04b6fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.821+0000 7f237c437740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa04b61bc0 0x55aa04b6ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.821+0000 7f237c437740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aa04b66c50 0x55aa04b70fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.821+0000 7f237c437740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f23741c03b0 con 0x55aa04b61280 2024-09-10T08:46:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.821+0000 7f237c437740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f23741c1820 con 0x55aa04b66c50 2024-09-10T08:46:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.821+0000 7f237c437740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f23741c0a80 con 0x55aa04b61bc0 2024-09-10T08:46:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.821+0000 7f237c437740 10 client.0 _ll_register_callbacks cb 0x7f236c0f3bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.822+0000 7f2372800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61280 0x55aa04b6fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.822+0000 7f2372800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61280 0x55aa04b6fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41326/0 (socket says 172.21.15.89:41326) 2024-09-10T08:46:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.822+0000 7f2372800640 1 -- 172.21.15.89:0/1830093019 learned_addr learned my addr 172.21.15.89:0/1830093019 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.823+0000 7f2372800640 1 -- 172.21.15.89:0/1830093019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa04b61bc0 msgr2=0x55aa04b6ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.823+0000 7f2372800640 1 --2- 172.21.15.89:0/1830093019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa04b61bc0 0x55aa04b6ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.823+0000 7f2372800640 1 -- 172.21.15.89:0/1830093019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aa04b66c50 msgr2=0x55aa04b70fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:46:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.823+0000 7f2372800640 1 --2- 172.21.15.89:0/1830093019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aa04b66c50 0x55aa04b70fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.823+0000 7f2372800640 1 -- 172.21.15.89:0/1830093019 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa04b715c0 con 0x55aa04b61280 2024-09-10T08:46:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.823+0000 7f2372800640 1 --2- 172.21.15.89:0/1830093019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61280 0x55aa04b6fa60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2360005d00 tx=0x7f2360013820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f8bbb58e1b33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.830+0000 7f2353e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:46:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.833+0000 7f2353e00640 1 -- 172.21.15.89:0/1830093019 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360004260 con 0x55aa04b61280 2024-09-10T08:46:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.833+0000 7f2353e00640 1 -- 172.21.15.89:0/1830093019 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f2360004400 con 0x55aa04b61280 2024-09-10T08:46:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.834+0000 7f237c437740 1 -- 172.21.15.89:0/1830093019 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f236c031b20 con 0x55aa04b61280 2024-09-10T08:46:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.841+0000 7f2353e00640 1 -- 172.21.15.89:0/1830093019 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f23600035f0 con 0x55aa04b61280 2024-09-10T08:46:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.846+0000 7f237c437740 10 client.16572 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.846+0000 7f237c437740 1 -- 172.21.15.89:0/1830093019 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aa04be1760 con 0x55aa04b61280 2024-09-10T08:46:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.846+0000 7f2353e00640 1 -- 172.21.15.89:0/1830093019 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f236c031b20 con 0x55aa04b61280 2024-09-10T08:46:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.846+0000 7f2353e00640 1 -- 172.21.15.89:0/1830093019 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f23600108d0 con 0x55aa04b61280 2024-09-10T08:46:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.847+0000 7f237c437740 10 client.16572 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.847+0000 7f237c437740 10 client.16572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:46:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.847+0000 7f237c437740 20 client.16572 populate_metadata read hostname 'smithi089' 2024-09-10T08:46:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.847+0000 7f237c437740 1 --2- 172.21.15.89:0/1830093019 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55aa04be2060 0x55aa04be4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.847+0000 7f237c437740 4 client.16572 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:46:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.847+0000 7f237c437740 1 -- 172.21.15.89:0/1830093019 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aa04acdb80 con 0x55aa04be2060 2024-09-10T08:46:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.851+0000 7f2371e00640 1 --2- 172.21.15.89:0/1830093019 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55aa04be2060 0x55aa04be4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.862+0000 7f2371e00640 1 --2- 172.21.15.89:0/1830093019 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55aa04be2060 0x55aa04be4480 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4a704d9afc6c21b server_cookie=c2ba108d004447fd in_seq=0 out_seq=0 2024-09-10T08:46:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:12.870+0000 7f2353e00640 10 client.16572 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:13.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:12 smithi149 ceph-mon[36167]: pgmap v581: 129 pgs: 129 active+clean; 348 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 329 op/s 2024-09-10T08:46:13.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:12 smithi111 ceph-mon[35220]: pgmap v581: 129 pgs: 129 active+clean; 348 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 329 op/s 2024-09-10T08:46:13.313 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:12 smithi089 ceph-mon[30757]: pgmap v581: 129 pgs: 129 active+clean; 348 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 329 op/s 2024-09-10T08:46:14.216 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:15.072 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:14 smithi089 ceph-mon[30757]: pgmap v582: 129 pgs: 129 active+clean; 348 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 183 op/s 2024-09-10T08:46:15.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:14 smithi149 ceph-mon[36167]: pgmap v582: 129 pgs: 129 active+clean; 348 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 183 op/s 2024-09-10T08:46:15.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:14 smithi111 ceph-mon[35220]: pgmap v582: 129 pgs: 129 active+clean; 348 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 183 op/s 2024-09-10T08:46:16.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.969+0000 7fbfe1600640 1 -- 172.21.15.89:0/1309389803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfdc108f60 msgr2=0x7fbfdc110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:16.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.969+0000 7fbfe1600640 1 --2- 172.21.15.89:0/1309389803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfdc108f60 0x7fbfdc110ae0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4009ca0 tx=0x7fbfd402a8f0 comp rx=0 tx=0).stop 2024-09-10T08:46:16.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.971+0000 7fbfe1600640 1 -- 172.21.15.89:0/1309389803 shutdown_connections 2024-09-10T08:46:16.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.971+0000 7fbfe1600640 1 --2- 172.21.15.89:0/1309389803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfdc108f60 0x7fbfdc110ae0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:16.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.971+0000 7fbfe1600640 1 --2- 172.21.15.89:0/1309389803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfdc1047e0 0x7fbfdc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:16.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.971+0000 7fbfe1600640 1 --2- 172.21.15.89:0/1309389803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfdc103e30 0x7fbfdc104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:16.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.971+0000 7fbfe1600640 1 -- 172.21.15.89:0/1309389803 >> 172.21.15.89:0/1309389803 conn(0x7fbfdc0ff910 msgr2=0x7fbfdc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:16.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.971+0000 7fbfe1600640 1 -- 172.21.15.89:0/1309389803 shutdown_connections 2024-09-10T08:46:16.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.971+0000 7fbfe1600640 1 -- 172.21.15.89:0/1309389803 wait complete. 2024-09-10T08:46:17.005 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.973+0000 7fbfe1600640 1 Processor -- start 2024-09-10T08:46:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.974+0000 7fbfe1600640 1 -- start start 2024-09-10T08:46:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.974+0000 7fbfe1600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfdc103e30 0x7fbfdc1109e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.975+0000 7fbfe1600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfdc1047e0 0x7fbfdc10da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.975+0000 7fbfe1600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfdc108f60 0x7fbfdc10e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.975+0000 7fbfe1600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc1140a0 con 0x7fbfdc108f60 2024-09-10T08:46:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.975+0000 7fbfe1600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc113f20 con 0x7fbfdc103e30 2024-09-10T08:46:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.975+0000 7fbfe1600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc114220 con 0x7fbfdc1047e0 2024-09-10T08:46:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.975+0000 7fbfdbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfdc103e30 0x7fbfdc1109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.975+0000 7fbfdbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfdc103e30 0x7fbfdc1109e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:54774/0 (socket says 172.21.15.89:54774) 2024-09-10T08:46:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.975+0000 7fbfdbe00640 1 -- 172.21.15.89:0/422365960 learned_addr learned my addr 172.21.15.89:0/422365960 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.976+0000 7fbfdbe00640 1 -- 172.21.15.89:0/422365960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfdc1047e0 msgr2=0x7fbfdc10da90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:46:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.976+0000 7fbfdbe00640 1 --2- 172.21.15.89:0/422365960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfdc1047e0 0x7fbfdc10da90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.976+0000 7fbfdbe00640 1 -- 172.21.15.89:0/422365960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfdc108f60 msgr2=0x7fbfdc10e030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:46:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.976+0000 7fbfdbe00640 1 --2- 172.21.15.89:0/422365960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfdc108f60 0x7fbfdc10e030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.976+0000 7fbfdbe00640 1 -- 172.21.15.89:0/422365960 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfdc1a05c0 con 0x7fbfdc103e30 2024-09-10T08:46:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.976+0000 7fbfdbe00640 1 --2- 172.21.15.89:0/422365960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfdc103e30 0x7fbfdc1109e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00b450 tx=0x7fbfcc00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.977+0000 7fbfd8c00640 1 -- 172.21.15.89:0/422365960 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc015020 con 0x7fbfdc103e30 2024-09-10T08:46:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.977+0000 7fbfd8c00640 1 -- 172.21.15.89:0/422365960 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfcc011d70 con 0x7fbfdc103e30 2024-09-10T08:46:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.977+0000 7fbfd8c00640 1 -- 172.21.15.89:0/422365960 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc015020 con 0x7fbfdc103e30 2024-09-10T08:46:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.982+0000 7fbfe1600640 1 -- 172.21.15.89:0/422365960 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfdc1a08b0 con 0x7fbfdc103e30 2024-09-10T08:46:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.992+0000 7fbfe1600640 1 -- 172.21.15.89:0/422365960 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfdc1a0d10 con 0x7fbfdc103e30 2024-09-10T08:46:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.997+0000 7fbfe5683740 1 Processor -- start 2024-09-10T08:46:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.997+0000 7fbfd8c00640 1 -- 172.21.15.89:0/422365960 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbfcc018020 con 0x7fbfdc103e30 2024-09-10T08:46:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:16.997+0000 7fbfe5683740 1 -- start start 2024-09-10T08:46:17.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.000+0000 7fbfe5683740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d32e40 0x558bc3d33220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.001+0000 7fbfe5683740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d33760 0x558bc3d3dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.009 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.001+0000 7fbfe5683740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d3e020 0x558bc3d40410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.001+0000 7fbfe5683740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558bc3c993e0 con 0x558bc3d32e40 2024-09-10T08:46:17.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.001+0000 7fbfe5683740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558bc3c40d80 con 0x558bc3d33760 2024-09-10T08:46:17.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.001+0000 7fbfe5683740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558bc3c953a0 con 0x558bc3d3e020 2024-09-10T08:46:17.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.001+0000 7fbfdb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d3e020 0x558bc3d40410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:17.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.002+0000 7fbfdb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d3e020 0x558bc3d40410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:49304/0 (socket says 172.21.15.89:49304) 2024-09-10T08:46:17.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.002+0000 7fbfdb400640 1 -- 172.21.15.89:0/4203415681 learned_addr learned my addr 172.21.15.89:0/4203415681 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:17.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.002+0000 7fbfdb400640 1 -- 172.21.15.89:0/4203415681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d33760 msgr2=0x558bc3d3dae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:46:17.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.002+0000 7fbfdb400640 1 --2- 172.21.15.89:0/4203415681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d33760 0x558bc3d3dae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.002+0000 7fbfdb400640 1 -- 172.21.15.89:0/4203415681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d32e40 msgr2=0x558bc3d33220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:17.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.002+0000 7fbfdb400640 1 --2- 172.21.15.89:0/4203415681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d32e40 0x558bc3d33220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.002+0000 7fbfdb400640 1 -- 172.21.15.89:0/4203415681 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bc3c5fae0 con 0x558bc3d3e020 2024-09-10T08:46:17.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.003+0000 7fbfdb400640 1 --2- 172.21.15.89:0/4203415681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d3e020 0x558bc3d40410 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80096f0 tx=0x7fbfc8009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f3b7e6138e03770 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:17.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.003+0000 7fbfd1200640 1 -- 172.21.15.89:0/4203415681 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8015020 con 0x558bc3d3e020 2024-09-10T08:46:17.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.003+0000 7fbfd1200640 1 -- 172.21.15.89:0/4203415681 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfc800bdb0 con 0x558bc3d3e020 2024-09-10T08:46:17.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.003+0000 7fbfd1200640 1 -- 172.21.15.89:0/4203415681 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8015020 con 0x558bc3d3e020 2024-09-10T08:46:17.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.003+0000 7fbfd8c00640 1 --2- 172.21.15.89:0/422365960 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbfc00a2e00 0x7fbfc00a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.004+0000 7fbfd8c00640 1 -- 172.21.15.89:0/422365960 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fbfcc0d1170 con 0x7fbfdc103e30 2024-09-10T08:46:17.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.004+0000 7fbfe5683740 1 -- 172.21.15.89:0/4203415681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d3e020 msgr2=0x558bc3d40410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:17.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.004+0000 7fbfe5683740 1 --2- 172.21.15.89:0/4203415681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d3e020 0x558bc3d40410 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80096f0 tx=0x7fbfc8009bb0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.005+0000 7fbfe5683740 1 -- 172.21.15.89:0/4203415681 shutdown_connections 2024-09-10T08:46:17.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.005+0000 7fbfe5683740 1 --2- 172.21.15.89:0/4203415681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d3e020 0x558bc3d40410 secure :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80096f0 tx=0x7fbfc8009bb0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.005+0000 7fbfe5683740 1 --2- 172.21.15.89:0/4203415681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d33760 0x558bc3d3dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.005+0000 7fbfe5683740 1 --2- 172.21.15.89:0/4203415681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d32e40 0x558bc3d33220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.005+0000 7fbfe5683740 1 -- 172.21.15.89:0/4203415681 >> 172.21.15.89:0/4203415681 conn(0x558bc3d2f3d0 msgr2=0x558bc3d2f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:17.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.005+0000 7fbfe0c00640 1 --2- 172.21.15.89:0/422365960 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbfc00a2e00 0x7fbfc00a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:17.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.008+0000 7fbfe5683740 1 -- 172.21.15.89:0/4203415681 shutdown_connections 2024-09-10T08:46:17.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.009+0000 7fbfe0c00640 1 --2- 172.21.15.89:0/422365960 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbfc00a2e00 0x7fbfc00a52c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4009430 tx=0x7fbfd4009860 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:17.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.010+0000 7fbfe5683740 1 -- 172.21.15.89:0/4203415681 wait complete. 2024-09-10T08:46:17.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.010+0000 7fbfe5683740 1 -- start start 2024-09-10T08:46:17.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.011+0000 7fbfe5683740 1 Processor -- start 2024-09-10T08:46:17.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.011+0000 7fbfe5683740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d32e40 0x558bc3d3b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.012+0000 7fbfe5683740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d33760 0x558bc3d3b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.012+0000 7fbfdbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d32e40 0x558bc3d3b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.012+0000 7fbfdbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d32e40 0x558bc3d3b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:54778/0 (socket says 172.21.15.89:54778) 2024-09-10T08:46:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.012+0000 7fbfe5683740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d3bb50 0x558bc3d4a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.012+0000 7fbfdb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d33760 0x558bc3d3b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.012+0000 7fbfdb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d33760 0x558bc3d3b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:49314/0 (socket says 172.21.15.89:49314) 2024-09-10T08:46:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.012+0000 7fbfe0c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d3bb50 0x558bc3d4a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.013+0000 7fbfe0c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d3bb50 0x558bc3d4a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41346/0 (socket says 172.21.15.89:41346) 2024-09-10T08:46:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.013+0000 7fbfdbe00640 1 -- 172.21.15.89:0/4116334101 learned_addr learned my addr 172.21.15.89:0/4116334101 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.013+0000 7fbfe5683740 1 -- 172.21.15.89:0/4116334101 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558bc3d40bf0 con 0x558bc3d3bb50 2024-09-10T08:46:17.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.013+0000 7fbfe5683740 1 -- 172.21.15.89:0/4116334101 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558bc3c540f0 con 0x558bc3d32e40 2024-09-10T08:46:17.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.013+0000 7fbfe5683740 1 -- 172.21.15.89:0/4116334101 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558bc3d40d70 con 0x558bc3d33760 2024-09-10T08:46:17.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.013+0000 7fbfe5683740 10 client.0 _ll_register_callbacks cb 0x558bc3c3bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:17.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.013+0000 7fbfdbe00640 1 -- 172.21.15.89:0/4116334101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d33760 msgr2=0x558bc3d3b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:17.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.013+0000 7fbfdbe00640 1 --2- 172.21.15.89:0/4116334101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d33760 0x558bc3d3b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.013+0000 7fbfdbe00640 1 -- 172.21.15.89:0/4116334101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d3bb50 msgr2=0x558bc3d4a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:17.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.013+0000 7fbfdbe00640 1 --2- 172.21.15.89:0/4116334101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d3bb50 0x558bc3d4a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.013+0000 7fbfdbe00640 1 -- 172.21.15.89:0/4116334101 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bc3a0a3c0 con 0x558bc3d32e40 2024-09-10T08:46:17.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.014+0000 7fbfdbe00640 1 --2- 172.21.15.89:0/4116334101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d32e40 0x558bc3d3b0d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009fd0 tx=0x7fbfcc09a9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90c2b22d7976a67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:17.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.014+0000 7fbfc5600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:46:17.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.014+0000 7fbfc5600640 1 -- 172.21.15.89:0/4116334101 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc0b3060 con 0x558bc3d32e40 2024-09-10T08:46:17.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.014+0000 7fbfc5600640 1 -- 172.21.15.89:0/4116334101 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fbfcc010d10 con 0x558bc3d32e40 2024-09-10T08:46:17.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.014+0000 7fbfe5683740 1 -- 172.21.15.89:0/4116334101 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfc800bdb0 con 0x558bc3d32e40 2024-09-10T08:46:17.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.015+0000 7fbfc5600640 1 -- 172.21.15.89:0/4116334101 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbfcc094e60 con 0x558bc3d32e40 2024-09-10T08:46:17.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.015+0000 7fbfe5683740 10 client.26618 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:17.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.015+0000 7fbfe5683740 1 -- 172.21.15.89:0/4116334101 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558bc3d4d230 con 0x558bc3d32e40 2024-09-10T08:46:17.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.015+0000 7fbfc5600640 1 -- 172.21.15.89:0/4116334101 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfc800bdb0 con 0x558bc3d32e40 2024-09-10T08:46:17.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.015+0000 7fbfc5600640 1 -- 172.21.15.89:0/4116334101 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fbfcc0cdb90 con 0x558bc3d32e40 2024-09-10T08:46:17.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.016+0000 7fbfe5683740 10 client.26618 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:17.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.016+0000 7fbfe5683740 10 client.26618 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:46:17.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.016+0000 7fbfe5683740 20 client.26618 populate_metadata read hostname 'smithi089' 2024-09-10T08:46:17.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.016+0000 7fbfe5683740 1 --2- 172.21.15.89:0/4116334101 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x558bc3d4e640 0x558bc3d50a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.016+0000 7fbfe5683740 4 client.26618 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.016+0000 7fbfe5683740 1 -- 172.21.15.89:0/4116334101 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bc3ca6950 con 0x558bc3d4e640 2024-09-10T08:46:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.016+0000 7fbfdb400640 1 --2- 172.21.15.89:0/4116334101 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x558bc3d4e640 0x558bc3d50a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.017+0000 7fbfdb400640 1 --2- 172.21.15.89:0/4116334101 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x558bc3d4e640 0x558bc3d50a30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e87cb11502acafae server_cookie=78883db50ef73933 in_seq=0 out_seq=0 2024-09-10T08:46:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.017+0000 7fbfc5600640 10 client.26618 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:46:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.017+0000 7fbfc5600640 1 -- 172.21.15.89:0/4116334101 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558bc3ca6950 con 0x558bc3d4e640 2024-09-10T08:46:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.017+0000 7fbfc5600640 10 client.26618 handle_command_reply: tid=0 2024-09-10T08:46:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.018+0000 7fbfe5683740 1 client.26618 shutdown 2024-09-10T08:46:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.018+0000 7fbfe5683740 1 -- 172.21.15.89:0/4116334101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d32e40 msgr2=0x558bc3d3b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.018+0000 7fbfe5683740 1 --2- 172.21.15.89:0/4116334101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d32e40 0x558bc3d3b0d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009fd0 tx=0x7fbfcc09a9f0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.018+0000 7fbfe5683740 1 -- 172.21.15.89:0/4116334101 shutdown_connections 2024-09-10T08:46:17.052 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.018+0000 7fbfe5683740 1 --2- 172.21.15.89:0/4116334101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d3bb50 0x558bc3d4a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.018+0000 7fbfe5683740 1 --2- 172.21.15.89:0/4116334101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d33760 0x558bc3d3b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.018+0000 7fbfe5683740 1 --2- 172.21.15.89:0/4116334101 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x558bc3d4e640 0x558bc3d50a30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.018+0000 7fbfe5683740 1 --2- 172.21.15.89:0/4116334101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d32e40 0x558bc3d3b0d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.018+0000 7fbfc5600640 0 client.26618 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:46:17.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.018+0000 7fbfe5683740 1 -- 172.21.15.89:0/4116334101 >> 172.21.15.89:0/4116334101 conn(0x558bc3d2f3d0 msgr2=0x558bc3d35130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:17.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.019+0000 7fbfe5683740 1 -- 172.21.15.89:0/4116334101 shutdown_connections 2024-09-10T08:46:17.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.020+0000 7fbfe5683740 1 -- 172.21.15.89:0/4116334101 wait complete. 2024-09-10T08:46:17.053 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.020+0000 7fbfe5683740 20 client.26618 trim_cache size 0 max 16384 2024-09-10T08:46:17.062 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.062+0000 7fbfe5683740 1 Processor -- start 2024-09-10T08:46:17.063 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.063+0000 7fbfe5683740 1 -- start start 2024-09-10T08:46:17.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.064+0000 7fbfe5683740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d32e40 0x558bc3d49eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.064+0000 7fbfe5683740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d33760 0x558bc3d56300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.064 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.064+0000 7fbfe5683740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d3bb50 0x558bc3d56840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.064+0000 7fbfe5683740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc1c04a0 con 0x558bc3d3bb50 2024-09-10T08:46:17.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.064+0000 7fbfdbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d32e40 0x558bc3d49eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:17.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.064+0000 7fbfdbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d32e40 0x558bc3d49eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:54784/0 (socket says 172.21.15.89:54784) 2024-09-10T08:46:17.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.064+0000 7fbfdbe00640 1 -- 172.21.15.89:0/3717292821 learned_addr learned my addr 172.21.15.89:0/3717292821 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:17.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.064+0000 7fbfe5683740 1 -- 172.21.15.89:0/3717292821 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc1bdd70 con 0x558bc3d32e40 2024-09-10T08:46:17.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.065+0000 7fbfe5683740 1 -- 172.21.15.89:0/3717292821 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc1bbbc0 con 0x558bc3d33760 2024-09-10T08:46:17.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.065+0000 7fbfdb400640 1 --2- 172.21.15.89:0/3717292821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d33760 0x558bc3d56300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:17.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.065+0000 7fbfe0c00640 1 --2- 172.21.15.89:0/3717292821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d3bb50 0x558bc3d56840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:17.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.066+0000 7fbfdbe00640 1 -- 172.21.15.89:0/3717292821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d33760 msgr2=0x558bc3d56300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:17.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.066+0000 7fbfdbe00640 1 --2- 172.21.15.89:0/3717292821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d33760 0x558bc3d56300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.081 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.066+0000 7fbfdbe00640 1 -- 172.21.15.89:0/3717292821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d3bb50 msgr2=0x558bc3d56840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:17.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.066+0000 7fbfdbe00640 1 --2- 172.21.15.89:0/3717292821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d3bb50 0x558bc3d56840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.066+0000 7fbfdbe00640 1 -- 172.21.15.89:0/3717292821 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bc3d32bf0 con 0x558bc3d32e40 2024-09-10T08:46:17.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.066+0000 7fbfdbe00640 1 --2- 172.21.15.89:0/3717292821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d32e40 0x558bc3d49eb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0985a0 tx=0x7fbfcc098800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=870aa6cbfc70f108 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:17.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.068+0000 7fbfc7e00640 1 -- 172.21.15.89:0/3717292821 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc0b3050 con 0x558bc3d32e40 2024-09-10T08:46:17.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.068+0000 7fbfc7e00640 1 -- 172.21.15.89:0/3717292821 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfcc0a2f60 con 0x558bc3d32e40 2024-09-10T08:46:17.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.068+0000 7fbfc7e00640 1 -- 172.21.15.89:0/3717292821 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc0d2060 con 0x558bc3d32e40 2024-09-10T08:46:17.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.069+0000 7fbfe5683740 1 -- 172.21.15.89:0/3717292821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d32e40 msgr2=0x558bc3d49eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:17.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.069+0000 7fbfe5683740 1 --2- 172.21.15.89:0/3717292821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d32e40 0x558bc3d49eb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0985a0 tx=0x7fbfcc098800 comp rx=0 tx=0).stop 2024-09-10T08:46:17.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.069+0000 7fbfe5683740 1 -- 172.21.15.89:0/3717292821 shutdown_connections 2024-09-10T08:46:17.082 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.069+0000 7fbfe5683740 1 --2- 172.21.15.89:0/3717292821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d3bb50 0x558bc3d56840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.069+0000 7fbfe5683740 1 --2- 172.21.15.89:0/3717292821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d33760 0x558bc3d56300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.069+0000 7fbfe5683740 1 --2- 172.21.15.89:0/3717292821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d32e40 0x558bc3d49eb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.069+0000 7fbfe5683740 1 -- 172.21.15.89:0/3717292821 >> 172.21.15.89:0/3717292821 conn(0x558bc3d2f3d0 msgr2=0x558bc3d399a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:17.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.070+0000 7fbfe5683740 1 -- 172.21.15.89:0/3717292821 shutdown_connections 2024-09-10T08:46:17.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.070+0000 7fbfe5683740 1 -- 172.21.15.89:0/3717292821 wait complete. 2024-09-10T08:46:17.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.070+0000 7fbfe5683740 1 -- start start 2024-09-10T08:46:17.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.071+0000 7fbfe5683740 1 Processor -- start 2024-09-10T08:46:17.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.071+0000 7fbfe5683740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d32e40 0x558bc3d462e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.071+0000 7fbfe5683740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d33760 0x558bc3d46820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.071+0000 7fbfe5683740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d3bb50 0x558bc3d416c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.072+0000 7fbfdbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d32e40 0x558bc3d462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:17.083 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.072+0000 7fbfdbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d32e40 0x558bc3d462e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:49326/0 (socket says 172.21.15.89:49326) 2024-09-10T08:46:17.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.072+0000 7fbfe0c00640 1 --2- 172.21.15.89:0/2467320752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d3bb50 0x558bc3d416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:17.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.073+0000 7fbfdb400640 1 --2- 172.21.15.89:0/2467320752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d33760 0x558bc3d46820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:17.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.073+0000 7fbfe5683740 1 -- 172.21.15.89:0/2467320752 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc1bc230 con 0x558bc3d33760 2024-09-10T08:46:17.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.073+0000 7fbfe5683740 1 -- 172.21.15.89:0/2467320752 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc1bd6a0 con 0x558bc3d3bb50 2024-09-10T08:46:17.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.073+0000 7fbfe5683740 1 -- 172.21.15.89:0/2467320752 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc1bc900 con 0x558bc3d32e40 2024-09-10T08:46:17.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.073+0000 7fbfdbe00640 1 -- 172.21.15.89:0/2467320752 learned_addr learned my addr 172.21.15.89:0/2467320752 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:17.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.073+0000 7fbfe0c00640 1 -- 172.21.15.89:0/2467320752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d32e40 msgr2=0x558bc3d462e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:17.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.074+0000 7fbfe5683740 10 client.0 _ll_register_callbacks cb 0x7fbfcc0d0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:17.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.074+0000 7fbfe0c00640 1 --2- 172.21.15.89:0/2467320752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d32e40 0x558bc3d462e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.084 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.074+0000 7fbfe0c00640 1 -- 172.21.15.89:0/2467320752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d33760 msgr2=0x558bc3d46820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:17.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.074+0000 7fbfe0c00640 1 --2- 172.21.15.89:0/2467320752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d33760 0x558bc3d46820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.074+0000 7fbfe0c00640 1 -- 172.21.15.89:0/2467320752 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bc3d41ef0 con 0x558bc3d3bb50 2024-09-10T08:46:17.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.077+0000 7fbfdbe00640 1 --2- 172.21.15.89:0/2467320752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d32e40 0x558bc3d462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:17.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.077+0000 7fbfe5683740 1 -- 172.21.15.89:0/2467320752 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfcc0a2f60 con 0x558bc3d3bb50 2024-09-10T08:46:17.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.077+0000 7fbfdb400640 1 --2- 172.21.15.89:0/2467320752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d33760 0x558bc3d46820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:17.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.077+0000 7fbfc5600640 10 client.26624 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:46:17.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.077+0000 7fbfe0c00640 1 --2- 172.21.15.89:0/2467320752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d3bb50 0x558bc3d416c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x558bc3d50f40 tx=0x7fbfd402a8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31c7049f3ae983c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:17.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.078+0000 7fbfc5600640 1 -- 172.21.15.89:0/2467320752 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4004450 con 0x558bc3d3bb50 2024-09-10T08:46:17.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.078+0000 7fbfc5600640 1 -- 172.21.15.89:0/2467320752 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fbfd4004250 con 0x558bc3d3bb50 2024-09-10T08:46:17.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.078+0000 7fbfc5600640 1 -- 172.21.15.89:0/2467320752 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbfd4004450 con 0x558bc3d3bb50 2024-09-10T08:46:17.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.078+0000 7fbfe5683740 10 client.26624 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:17.085 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.078+0000 7fbfe5683740 1 -- 172.21.15.89:0/2467320752 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({fsmap=29}) -- 0x558bc3db8da0 con 0x558bc3d3bb50 2024-09-10T08:46:17.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.079+0000 7fbfc5600640 1 -- 172.21.15.89:0/2467320752 <== mon.2 v2:172.21.15.111:3300/0 4 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fbfd40045f0 con 0x558bc3d3bb50 2024-09-10T08:46:17.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.079+0000 7fbfe5683740 10 client.26624 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:17.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.079+0000 7fbfe5683740 10 client.26624 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:46:17.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.079+0000 7fbfe5683740 20 client.26624 populate_metadata read hostname 'smithi089' 2024-09-10T08:46:17.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.079+0000 7fbfe5683740 1 --2- 172.21.15.89:0/2467320752 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x558bc3db9360 0x558bc3dbb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.079+0000 7fbfe5683740 4 client.26624 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:46:17.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.079+0000 7fbfe5683740 1 -- 172.21.15.89:0/2467320752 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558bc3ca0e60 con 0x558bc3db9360 2024-09-10T08:46:17.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.081+0000 7fbfdbe00640 1 --2- 172.21.15.89:0/2467320752 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x558bc3db9360 0x558bc3dbb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:17.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.081+0000 7fbfdbe00640 1 --2- 172.21.15.89:0/2467320752 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x558bc3db9360 0x558bc3dbb750 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14791494e08888b9 server_cookie=67a4356a7ccea2d7 in_seq=0 out_seq=0 2024-09-10T08:46:17.086 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.081+0000 7fbfc5600640 10 client.26624 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:46:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.082+0000 7fbfc5600640 1 -- 172.21.15.89:0/2467320752 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558bc3ca0e60 con 0x558bc3db9360 2024-09-10T08:46:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.082+0000 7fbfc5600640 10 client.26624 handle_command_reply: tid=0 2024-09-10T08:46:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.082+0000 7fbfe5683740 1 client.26624 shutdown 2024-09-10T08:46:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.082+0000 7fbfe5683740 1 -- 172.21.15.89:0/2467320752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d3bb50 msgr2=0x558bc3d416c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.082+0000 7fbfe5683740 1 --2- 172.21.15.89:0/2467320752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d3bb50 0x558bc3d416c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x558bc3d50f40 tx=0x7fbfd402a8f0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.082+0000 7fbfe5683740 1 -- 172.21.15.89:0/2467320752 shutdown_connections 2024-09-10T08:46:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.082+0000 7fbfe5683740 1 --2- 172.21.15.89:0/2467320752 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x558bc3db9360 0x558bc3dbb750 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.087 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.082+0000 7fbfe5683740 1 --2- 172.21.15.89:0/2467320752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558bc3d3bb50 0x558bc3d416c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.088 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:46:17.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.082+0000 7fbfe5683740 1 --2- 172.21.15.89:0/2467320752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558bc3d33760 0x558bc3d46820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.082+0000 7fbfe5683740 1 --2- 172.21.15.89:0/2467320752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558bc3d32e40 0x558bc3d462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.082+0000 7fbfe5683740 1 -- 172.21.15.89:0/2467320752 >> 172.21.15.89:0/2467320752 conn(0x558bc3d2f3d0 msgr2=0x558bc3d2e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:17.088 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.082+0000 7fbfc5600640 0 client.26624 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:46:17.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.083+0000 7fbfe5683740 1 -- 172.21.15.89:0/2467320752 shutdown_connections 2024-09-10T08:46:17.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.083+0000 7fbfe5683740 1 -- 172.21.15.89:0/2467320752 wait complete. 2024-09-10T08:46:17.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.083+0000 7fbfe5683740 20 client.26624 trim_cache size 0 max 16384 2024-09-10T08:46:17.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.084+0000 7fbfe1600640 1 -- 172.21.15.89:0/422365960 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbfc00a2e00 msgr2=0x7fbfc00a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:17.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.084+0000 7fbfe1600640 1 --2- 172.21.15.89:0/422365960 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbfc00a2e00 0x7fbfc00a52c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4009430 tx=0x7fbfd4009860 comp rx=0 tx=0).stop 2024-09-10T08:46:17.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.084+0000 7fbfe1600640 1 -- 172.21.15.89:0/422365960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfdc103e30 msgr2=0x7fbfdc1109e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:17.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.084+0000 7fbfe1600640 1 --2- 172.21.15.89:0/422365960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfdc103e30 0x7fbfdc1109e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00b450 tx=0x7fbfcc00b910 comp rx=0 tx=0).stop 2024-09-10T08:46:17.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.084+0000 7fbfe1600640 1 -- 172.21.15.89:0/422365960 shutdown_connections 2024-09-10T08:46:17.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.084+0000 7fbfe1600640 1 --2- 172.21.15.89:0/422365960 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbfc00a2e00 0x7fbfc00a52c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.084+0000 7fbfe1600640 1 --2- 172.21.15.89:0/422365960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfdc108f60 0x7fbfdc10e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.089 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.084+0000 7fbfe1600640 1 --2- 172.21.15.89:0/422365960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfdc1047e0 0x7fbfdc10da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.084+0000 7fbfe1600640 1 --2- 172.21.15.89:0/422365960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfdc103e30 0x7fbfdc1109e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.084+0000 7fbfe1600640 1 -- 172.21.15.89:0/422365960 >> 172.21.15.89:0/422365960 conn(0x7fbfdc0ff910 msgr2=0x7fbfdc100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:17.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.085+0000 7fbfe1600640 1 -- 172.21.15.89:0/422365960 shutdown_connections 2024-09-10T08:46:17.090 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:17.085+0000 7fbfe1600640 1 -- 172.21.15.89:0/422365960 wait complete. 2024-09-10T08:46:17.148 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:16 smithi089 ceph-mon[30757]: pgmap v583: 129 pgs: 129 active+clean; 356 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 339 op/s 2024-09-10T08:46:17.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:16 smithi149 ceph-mon[36167]: pgmap v583: 129 pgs: 129 active+clean; 356 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 339 op/s 2024-09-10T08:46:17.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:16 smithi111 ceph-mon[35220]: pgmap v583: 129 pgs: 129 active+clean; 356 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 339 op/s 2024-09-10T08:46:18.076 INFO:tasks.workunit.client.0.smithi089.stdout: 119 501777 304 360716 458 308992 1118 2024-09-10T08:46:19.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:18 smithi149 ceph-mon[36167]: pgmap v584: 129 pgs: 129 active+clean; 378 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 860 op/s 2024-09-10T08:46:19.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:18 smithi111 ceph-mon[35220]: pgmap v584: 129 pgs: 129 active+clean; 378 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 860 op/s 2024-09-10T08:46:19.307 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:18 smithi089 ceph-mon[30757]: pgmap v584: 129 pgs: 129 active+clean; 378 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 860 op/s 2024-09-10T08:46:19.655 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:46:19.941 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:19 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:46:20.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:19 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:46:20.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:19 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:46:21.214 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:20 smithi089 ceph-mon[30757]: pgmap v585: 129 pgs: 129 active+clean; 378 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 867 op/s 2024-09-10T08:46:21.215 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:20 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:46:21.215 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:20 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:46:21.215 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:20 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:46:21.215 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:20 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-09-10T08:46:21.215 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:20 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:46:21.215 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:20 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:46:21.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:20 smithi149 ceph-mon[36167]: pgmap v585: 129 pgs: 129 active+clean; 378 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 867 op/s 2024-09-10T08:46:21.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:20 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:46:21.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:20 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:46:21.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:20 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:46:21.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:20 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-09-10T08:46:21.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:20 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:46:21.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:20 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:46:21.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:20 smithi111 ceph-mon[35220]: pgmap v585: 129 pgs: 129 active+clean; 378 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 867 op/s 2024-09-10T08:46:21.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:20 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:46:21.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:20 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:46:21.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:20 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:46:21.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:20 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-09-10T08:46:21.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:20 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:46:21.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:20 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:46:22.157 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:21 smithi089 ceph-mon[30757]: Adjusting osd_memory_target on smithi111 to 176.0M 2024-09-10T08:46:22.157 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:21 smithi089 ceph-mon[30757]: Unable to set osd_memory_target on smithi111 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-10T08:46:22.157 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:21 smithi089 ceph-mon[30757]: pgmap v586: 129 pgs: 129 active+clean; 406 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 1.39k op/s 2024-09-10T08:46:22.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:21 smithi149 ceph-mon[36167]: Adjusting osd_memory_target on smithi111 to 176.0M 2024-09-10T08:46:22.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:21 smithi149 ceph-mon[36167]: Unable to set osd_memory_target on smithi111 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-10T08:46:22.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:21 smithi149 ceph-mon[36167]: pgmap v586: 129 pgs: 129 active+clean; 406 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 1.39k op/s 2024-09-10T08:46:22.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:21 smithi111 ceph-mon[35220]: Adjusting osd_memory_target on smithi111 to 176.0M 2024-09-10T08:46:22.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:21 smithi111 ceph-mon[35220]: Unable to set osd_memory_target on smithi111 to 184590336: error parsing value: Value '184590336' is below minimum 939524096 2024-09-10T08:46:22.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:21 smithi111 ceph-mon[35220]: pgmap v586: 129 pgs: 129 active+clean; 406 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 1.39k op/s 2024-09-10T08:46:25.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:24 smithi149 ceph-mon[36167]: pgmap v587: 129 pgs: 129 active+clean; 417 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.3 MiB/s wr, 1.52k op/s 2024-09-10T08:46:25.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:24 smithi111 ceph-mon[35220]: pgmap v587: 129 pgs: 129 active+clean; 417 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.3 MiB/s wr, 1.52k op/s 2024-09-10T08:46:25.306 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:24 smithi089 ceph-mon[30757]: pgmap v587: 129 pgs: 129 active+clean; 417 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.3 MiB/s wr, 1.52k op/s 2024-09-10T08:46:27.097 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:27.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:26 smithi149 ceph-mon[36167]: pgmap v588: 129 pgs: 129 active+clean; 427 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.1 MiB/s wr, 1.57k op/s 2024-09-10T08:46:27.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:26 smithi111 ceph-mon[35220]: pgmap v588: 129 pgs: 129 active+clean; 427 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.1 MiB/s wr, 1.57k op/s 2024-09-10T08:46:27.304 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:26 smithi089 ceph-mon[30757]: pgmap v588: 129 pgs: 129 active+clean; 427 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.1 MiB/s wr, 1.57k op/s 2024-09-10T08:46:28.073 INFO:tasks.workunit.client.0.smithi089.stdout: 124 391362 168 282523 252 246476 915 2024-09-10T08:46:29.095 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:28 smithi111 ceph-mon[35220]: pgmap v589: 129 pgs: 129 active+clean; 446 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.0 MiB/s wr, 1.58k op/s 2024-09-10T08:46:29.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:28 smithi149 ceph-mon[36167]: pgmap v589: 129 pgs: 129 active+clean; 446 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.0 MiB/s wr, 1.58k op/s 2024-09-10T08:46:29.270 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:28 smithi089 ceph-mon[30757]: pgmap v589: 129 pgs: 129 active+clean; 446 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.0 MiB/s wr, 1.58k op/s 2024-09-10T08:46:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.585+0000 7f2353e00640 1 -- 172.21.15.89:0/1830093019 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aa04acdb80 con 0x55aa04be2060 2024-09-10T08:46:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.585+0000 7f2353e00640 10 client.16572 handle_command_reply: tid=0 2024-09-10T08:46:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.586+0000 7f237c437740 1 client.16572 shutdown 2024-09-10T08:46:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.618+0000 7f237c437740 1 -- 172.21.15.89:0/1830093019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61280 msgr2=0x55aa04b6fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.618+0000 7f237c437740 1 --2- 172.21.15.89:0/1830093019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61280 0x55aa04b6fa60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2360005d00 tx=0x7f2360013820 comp rx=0 tx=0).stop 2024-09-10T08:46:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.636+0000 7f237c437740 1 -- 172.21.15.89:0/1830093019 shutdown_connections 2024-09-10T08:46:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.636+0000 7f237c437740 1 --2- 172.21.15.89:0/1830093019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aa04b66c50 0x55aa04b70fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.636+0000 7f237c437740 1 --2- 172.21.15.89:0/1830093019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa04b61bc0 0x55aa04b6ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.636+0000 7f237c437740 1 --2- 172.21.15.89:0/1830093019 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55aa04be2060 0x55aa04be4480 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.636+0000 7f237c437740 1 --2- 172.21.15.89:0/1830093019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa04b61280 0x55aa04b6fa60 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.636+0000 7f237c437740 1 -- 172.21.15.89:0/1830093019 >> 172.21.15.89:0/1830093019 conn(0x55aa04b5cbe0 msgr2=0x55aa04b5b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.636+0000 7f2353e00640 0 client.16572 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.665+0000 7f237c437740 1 -- 172.21.15.89:0/1830093019 shutdown_connections 2024-09-10T08:46:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.672+0000 7f237c437740 1 -- 172.21.15.89:0/1830093019 wait complete. 2024-09-10T08:46:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.677+0000 7f237c437740 20 client.16572 trim_cache size 0 max 16384 2024-09-10T08:46:29.727 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:46:29.727 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:46:29.727 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:46:29.727 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:46:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.785+0000 7f235fe00640 1 -- 172.21.15.89:0/849306317 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23580a2d90 msgr2=0x7f23580a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.785+0000 7f235fe00640 1 --2- 172.21.15.89:0/849306317 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23580a2d90 0x7f23580a5250 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2364005dc0 tx=0x7f2364005d50 comp rx=0 tx=0).stop 2024-09-10T08:46:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.785+0000 7f235fe00640 1 -- 172.21.15.89:0/849306317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2374073680 msgr2=0x7f2374115e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.785+0000 7f235fe00640 1 --2- 172.21.15.89:0/849306317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2374073680 0x7f2374115e90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f236c009b80 tx=0x7f236c004a40 comp rx=0 tx=0).stop 2024-09-10T08:46:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.786+0000 7f235fe00640 1 -- 172.21.15.89:0/849306317 shutdown_connections 2024-09-10T08:46:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.786+0000 7f235fe00640 1 --2- 172.21.15.89:0/849306317 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f23580a2d90 0x7f23580a5250 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.786+0000 7f235fe00640 1 --2- 172.21.15.89:0/849306317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f237410dbb0 0x7f23741b6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.786+0000 7f235fe00640 1 --2- 172.21.15.89:0/849306317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2374073680 0x7f2374115e90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.786+0000 7f235fe00640 1 --2- 172.21.15.89:0/849306317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2374072cb0 0x7f2374115950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.786+0000 7f235fe00640 1 -- 172.21.15.89:0/849306317 >> 172.21.15.89:0/849306317 conn(0x7f237406eb50 msgr2=0x7f2374070900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.786+0000 7f235fe00640 1 -- 172.21.15.89:0/849306317 shutdown_connections 2024-09-10T08:46:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:29.794+0000 7f235fe00640 1 -- 172.21.15.89:0/849306317 wait complete. 2024-09-10T08:46:30.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.110+0000 7fc183a00640 1 -- 172.21.15.89:0/462596396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc17c1011e0 msgr2=0x7fc17c105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:30.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.110+0000 7fc183a00640 1 --2- 172.21.15.89:0/462596396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc17c1011e0 0x7fc17c105980 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc168009890 tx=0x7fc16802a370 comp rx=0 tx=0).stop 2024-09-10T08:46:30.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.112+0000 7fc183a00640 1 -- 172.21.15.89:0/462596396 shutdown_connections 2024-09-10T08:46:30.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.112+0000 7fc183a00640 1 --2- 172.21.15.89:0/462596396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc17c105ec0 0x7fc17c106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.112+0000 7fc183a00640 1 --2- 172.21.15.89:0/462596396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc17c1011e0 0x7fc17c105980 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.112+0000 7fc183a00640 1 --2- 172.21.15.89:0/462596396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc17c1008c0 0x7fc17c100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.112+0000 7fc183a00640 1 -- 172.21.15.89:0/462596396 >> 172.21.15.89:0/462596396 conn(0x7fc17c0fc7d0 msgr2=0x7fc17c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:30.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.112+0000 7fc183a00640 1 -- 172.21.15.89:0/462596396 shutdown_connections 2024-09-10T08:46:30.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.113+0000 7fc183a00640 1 -- 172.21.15.89:0/462596396 wait complete. 2024-09-10T08:46:30.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.114+0000 7fc183a00640 1 Processor -- start 2024-09-10T08:46:30.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.115+0000 7fc183a00640 1 -- start start 2024-09-10T08:46:30.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.116+0000 7fc183a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc17c1008c0 0x7fc17c114d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.116+0000 7fc183a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc17c1011e0 0x7fc17c10dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.116+0000 7fc183a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc17c105ec0 0x7fc17c10e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.117+0000 7fc181000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc17c1008c0 0x7fc17c114d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.117+0000 7fc181000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc17c1008c0 0x7fc17c114d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:43362/0 (socket says 172.21.15.89:43362) 2024-09-10T08:46:30.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.118+0000 7fc17be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc17c1011e0 0x7fc17c10dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.118+0000 7fc17be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc17c1011e0 0x7fc17c10dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37460/0 (socket says 172.21.15.89:37460) 2024-09-10T08:46:30.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.118+0000 7fc181a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc17c105ec0 0x7fc17c10e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.118+0000 7fc181a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc17c105ec0 0x7fc17c10e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:39024/0 (socket says 172.21.15.89:39024) 2024-09-10T08:46:30.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.121+0000 7fc181000640 1 -- 172.21.15.89:0/1147148490 learned_addr learned my addr 172.21.15.89:0/1147148490 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:30.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.121+0000 7fc183a00640 1 -- 172.21.15.89:0/1147148490 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c118660 con 0x7fc17c1011e0 2024-09-10T08:46:30.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.121+0000 7fc183a00640 1 -- 172.21.15.89:0/1147148490 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c1184e0 con 0x7fc17c105ec0 2024-09-10T08:46:30.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.121+0000 7fc183a00640 1 -- 172.21.15.89:0/1147148490 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c1187e0 con 0x7fc17c1008c0 2024-09-10T08:46:30.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.122+0000 7fc181a00640 1 -- 172.21.15.89:0/1147148490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc17c1008c0 msgr2=0x7fc17c114d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:30.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.122+0000 7fc181a00640 1 --2- 172.21.15.89:0/1147148490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc17c1008c0 0x7fc17c114d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.122+0000 7fc181a00640 1 -- 172.21.15.89:0/1147148490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc17c1011e0 msgr2=0x7fc17c10dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:30.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.122+0000 7fc181a00640 1 --2- 172.21.15.89:0/1147148490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc17c1011e0 0x7fc17c10dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.122+0000 7fc181a00640 1 -- 172.21.15.89:0/1147148490 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17c10e8b0 con 0x7fc17c105ec0 2024-09-10T08:46:30.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.122+0000 7fc181000640 1 --2- 172.21.15.89:0/1147148490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc17c1008c0 0x7fc17c114d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:30.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.122+0000 7fc17be00640 1 --2- 172.21.15.89:0/1147148490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc17c1011e0 0x7fc17c10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:30.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.122+0000 7fc181a00640 1 --2- 172.21.15.89:0/1147148490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc17c105ec0 0x7fc17c10e2c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc174009fd0 tx=0x7fc17400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:30.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.123+0000 7fc179600640 1 -- 172.21.15.89:0/1147148490 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17400ce20 con 0x7fc17c105ec0 2024-09-10T08:46:30.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.123+0000 7fc183a00640 1 -- 172.21.15.89:0/1147148490 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc17c10eba0 con 0x7fc17c105ec0 2024-09-10T08:46:30.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.123+0000 7fc183a00640 1 -- 172.21.15.89:0/1147148490 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc17c112bd0 con 0x7fc17c105ec0 2024-09-10T08:46:30.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.124+0000 7fc179600640 1 -- 172.21.15.89:0/1147148490 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc174005020 con 0x7fc17c105ec0 2024-09-10T08:46:30.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.124+0000 7fc179600640 1 -- 172.21.15.89:0/1147148490 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17400ce20 con 0x7fc17c105ec0 2024-09-10T08:46:30.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.131+0000 7fc179600640 1 -- 172.21.15.89:0/1147148490 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc174010430 con 0x7fc17c105ec0 2024-09-10T08:46:30.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.134+0000 7fc179600640 1 --2- 172.21.15.89:0/1147148490 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc1600a2ed0 0x7fc1600a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.135+0000 7fc179600640 1 -- 172.21.15.89:0/1147148490 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fc174004b80 con 0x7fc17c105ec0 2024-09-10T08:46:30.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.138+0000 7fc181000640 1 --2- 172.21.15.89:0/1147148490 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc1600a2ed0 0x7fc1600a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.139+0000 7fc186447740 1 Processor -- start 2024-09-10T08:46:30.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.139+0000 7fc186447740 1 -- start start 2024-09-10T08:46:30.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.139+0000 7fc186447740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0908c70 0x561cc0909050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.139+0000 7fc186447740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0909590 0x561cc0913910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.139+0000 7fc186447740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0913e50 0x561cc0916240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.140+0000 7fc17be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0909590 0x561cc0913910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.140+0000 7fc17be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0909590 0x561cc0913910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:43370/0 (socket says 172.21.15.89:43370) 2024-09-10T08:46:30.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.140+0000 7fc181a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0913e50 0x561cc0916240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.141+0000 7fc181a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0913e50 0x561cc0916240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:37470/0 (socket says 172.21.15.89:37470) 2024-09-10T08:46:30.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.141+0000 7fc17be00640 1 -- 172.21.15.89:0/1615425243 learned_addr learned my addr 172.21.15.89:0/1615425243 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.141+0000 7fc186447740 1 -- 172.21.15.89:0/1615425243 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561cc086f3e0 con 0x561cc0913e50 2024-09-10T08:46:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.141+0000 7fc186447740 1 -- 172.21.15.89:0/1615425243 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561cc0816d80 con 0x561cc0908c70 2024-09-10T08:46:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.141+0000 7fc186447740 1 -- 172.21.15.89:0/1615425243 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561cc086b3a0 con 0x561cc0909590 2024-09-10T08:46:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.141+0000 7fc17be00640 1 -- 172.21.15.89:0/1615425243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0908c70 msgr2=0x561cc0909050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:46:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.142+0000 7fc181000640 1 --2- 172.21.15.89:0/1615425243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0908c70 0x561cc0909050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.142+0000 7fc17be00640 1 --2- 172.21.15.89:0/1615425243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0908c70 0x561cc0909050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.142+0000 7fc17be00640 1 -- 172.21.15.89:0/1615425243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0913e50 msgr2=0x561cc0916240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.142+0000 7fc17be00640 1 --2- 172.21.15.89:0/1615425243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0913e50 0x561cc0916240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.142+0000 7fc17be00640 1 -- 172.21.15.89:0/1615425243 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cc0835ae0 con 0x561cc0909590 2024-09-10T08:46:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.142+0000 7fc181000640 1 --2- 172.21.15.89:0/1615425243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0908c70 0x561cc0909050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.142+0000 7fc181000640 1 --2- 172.21.15.89:0/1147148490 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc1600a2ed0 0x7fc1600a5390 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0058c0 tx=0x7fc16c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.143+0000 7fc17be00640 1 --2- 172.21.15.89:0/1615425243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0909590 0x561cc0913910 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc168009890 tx=0x7fc1680303d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=558b4338fda8f7d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.143+0000 7fc172000640 1 -- 172.21.15.89:0/1615425243 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc168037040 con 0x561cc0909590 2024-09-10T08:46:30.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.143+0000 7fc172000640 1 -- 172.21.15.89:0/1615425243 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc168041070 con 0x561cc0909590 2024-09-10T08:46:30.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.144+0000 7fc186447740 1 -- 172.21.15.89:0/1615425243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0909590 msgr2=0x561cc0913910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:30.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.144+0000 7fc186447740 1 --2- 172.21.15.89:0/1615425243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0909590 0x561cc0913910 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc168009890 tx=0x7fc1680303d0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.146+0000 7fc186447740 1 -- 172.21.15.89:0/1615425243 shutdown_connections 2024-09-10T08:46:30.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.146+0000 7fc186447740 1 --2- 172.21.15.89:0/1615425243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0913e50 0x561cc0916240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.146+0000 7fc186447740 1 --2- 172.21.15.89:0/1615425243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0909590 0x561cc0913910 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.146+0000 7fc186447740 1 --2- 172.21.15.89:0/1615425243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0908c70 0x561cc0909050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.146+0000 7fc186447740 1 -- 172.21.15.89:0/1615425243 >> 172.21.15.89:0/1615425243 conn(0x561cc0905200 msgr2=0x561cc09055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:30.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.146+0000 7fc186447740 1 -- 172.21.15.89:0/1615425243 shutdown_connections 2024-09-10T08:46:30.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.146+0000 7fc186447740 1 -- 172.21.15.89:0/1615425243 wait complete. 2024-09-10T08:46:30.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.147+0000 7fc186447740 1 -- start start 2024-09-10T08:46:30.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.147+0000 7fc186447740 1 Processor -- start 2024-09-10T08:46:30.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.149+0000 7fc186447740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0908c70 0x561cc0910bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.149+0000 7fc186447740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0909590 0x561cc0911130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.149+0000 7fc186447740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0913e50 0x561cc0911670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.150+0000 7fc186447740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561cc0916ac0 con 0x561cc0909590 2024-09-10T08:46:30.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.150+0000 7fc181a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0913e50 0x561cc0911670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.150+0000 7fc181a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0913e50 0x561cc0911670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:43378/0 (socket says 172.21.15.89:43378) 2024-09-10T08:46:30.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.150+0000 7fc181a00640 1 -- 172.21.15.89:0/3523012071 learned_addr learned my addr 172.21.15.89:0/3523012071 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:30.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.150+0000 7fc186447740 1 -- 172.21.15.89:0/3523012071 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561cc082a0f0 con 0x561cc0908c70 2024-09-10T08:46:30.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.150+0000 7fc186447740 1 -- 172.21.15.89:0/3523012071 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561cc0916c40 con 0x561cc0913e50 2024-09-10T08:46:30.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.150+0000 7fc181a00640 1 -- 172.21.15.89:0/3523012071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0908c70 msgr2=0x561cc0910bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:46:30.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.150+0000 7fc181a00640 1 --2- 172.21.15.89:0/3523012071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0908c70 0x561cc0910bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.150+0000 7fc181a00640 1 -- 172.21.15.89:0/3523012071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0909590 msgr2=0x561cc0911130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:46:30.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.151+0000 7fc17be00640 1 --2- 172.21.15.89:0/3523012071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0909590 0x561cc0911130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.150+0000 7fc186447740 10 client.0 _ll_register_callbacks cb 0x561cc0811bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:30.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.151+0000 7fc181a00640 1 --2- 172.21.15.89:0/3523012071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0909590 0x561cc0911130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.151+0000 7fc181a00640 1 -- 172.21.15.89:0/3523012071 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cc05e03c0 con 0x561cc0913e50 2024-09-10T08:46:30.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.152+0000 7fc186447740 1 -- 172.21.15.89:0/3523012071 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc168041070 con 0x561cc0913e50 2024-09-10T08:46:30.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.152+0000 7fc166a00640 10 client.26566 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:46:30.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.152+0000 7fc181a00640 1 --2- 172.21.15.89:0/3523012071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0913e50 0x561cc0911670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc17400c760 tx=0x7fc174004ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c82bc29c415580b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:30.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.152+0000 7fc166a00640 1 -- 172.21.15.89:0/3523012071 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1740b4070 con 0x561cc0913e50 2024-09-10T08:46:30.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.152+0000 7fc166a00640 1 -- 172.21.15.89:0/3523012071 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fc1740ba060 con 0x561cc0913e50 2024-09-10T08:46:30.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.153+0000 7fc166a00640 1 -- 172.21.15.89:0/3523012071 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc1740b4070 con 0x561cc0913e50 2024-09-10T08:46:30.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.153+0000 7fc186447740 10 client.26566 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:30.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.153+0000 7fc186447740 1 -- 172.21.15.89:0/3523012071 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({fsmap=29}) -- 0x561cc0922950 con 0x561cc0913e50 2024-09-10T08:46:30.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.154+0000 7fc166a00640 1 -- 172.21.15.89:0/3523012071 <== mon.1 v2:172.21.15.149:3300/0 4 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fc1740a5330 con 0x561cc0913e50 2024-09-10T08:46:30.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.154+0000 7fc186447740 10 client.26566 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:30.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.154+0000 7fc186447740 10 client.26566 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:46:30.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.154+0000 7fc186447740 20 client.26566 populate_metadata read hostname 'smithi089' 2024-09-10T08:46:30.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.155+0000 7fc186447740 1 --2- 172.21.15.89:0/3523012071 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x561cc0923dc0 0x561cc0926250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.155+0000 7fc186447740 4 client.26566 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:30.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.155+0000 7fc186447740 1 -- 172.21.15.89:0/3523012071 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cc087c950 con 0x561cc0923dc0 2024-09-10T08:46:30.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.156+0000 7fc181000640 1 --2- 172.21.15.89:0/3523012071 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x561cc0923dc0 0x561cc0926250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.157+0000 7fc181000640 1 --2- 172.21.15.89:0/3523012071 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x561cc0923dc0 0x561cc0926250 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c2cd952a02c4f473 server_cookie=18a553deb9b5b665 in_seq=0 out_seq=0 2024-09-10T08:46:30.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.157+0000 7fc166a00640 10 client.26566 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:46:30.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.158+0000 7fc166a00640 1 -- 172.21.15.89:0/3523012071 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561cc087c950 con 0x561cc0923dc0 2024-09-10T08:46:30.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.158+0000 7fc166a00640 10 client.26566 handle_command_reply: tid=0 2024-09-10T08:46:30.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.158+0000 7fc186447740 1 client.26566 shutdown 2024-09-10T08:46:30.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.159+0000 7fc186447740 1 -- 172.21.15.89:0/3523012071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0913e50 msgr2=0x561cc0911670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:30.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.159+0000 7fc186447740 1 --2- 172.21.15.89:0/3523012071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0913e50 0x561cc0911670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc17400c760 tx=0x7fc174004ed0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.160+0000 7fc186447740 1 -- 172.21.15.89:0/3523012071 shutdown_connections 2024-09-10T08:46:30.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.160+0000 7fc186447740 1 --2- 172.21.15.89:0/3523012071 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x561cc0923dc0 0x561cc0926250 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.160+0000 7fc186447740 1 --2- 172.21.15.89:0/3523012071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0913e50 0x561cc0911670 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.160+0000 7fc186447740 1 --2- 172.21.15.89:0/3523012071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0909590 0x561cc0911130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.160+0000 7fc186447740 1 --2- 172.21.15.89:0/3523012071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0908c70 0x561cc0910bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.160+0000 7fc186447740 1 -- 172.21.15.89:0/3523012071 >> 172.21.15.89:0/3523012071 conn(0x561cc0905200 msgr2=0x561cc090aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:30.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.160+0000 7fc166a00640 0 client.26566 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:46:30.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.161+0000 7fc186447740 1 -- 172.21.15.89:0/3523012071 shutdown_connections 2024-09-10T08:46:30.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.161+0000 7fc186447740 1 -- 172.21.15.89:0/3523012071 wait complete. 2024-09-10T08:46:30.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.161+0000 7fc186447740 20 client.26566 trim_cache size 0 max 16384 2024-09-10T08:46:30.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.207+0000 7fc186447740 1 Processor -- start 2024-09-10T08:46:30.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.211+0000 7fc186447740 1 -- start start 2024-09-10T08:46:30.288 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:46:30.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.211+0000 7fc186447740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0908c70 0x561cc092bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.211+0000 7fc186447740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0909590 0x561cc092c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.211+0000 7fc186447740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0913e50 0x561cc0903b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.211+0000 7fc186447740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c0125f0 con 0x561cc0909590 2024-09-10T08:46:30.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.211+0000 7fc186447740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c00fec0 con 0x561cc0913e50 2024-09-10T08:46:30.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.211+0000 7fc186447740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c00dd10 con 0x561cc0908c70 2024-09-10T08:46:30.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.212+0000 7fc181000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0908c70 0x561cc092bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.212+0000 7fc181000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0908c70 0x561cc092bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:43382/0 (socket says 172.21.15.89:43382) 2024-09-10T08:46:30.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.212+0000 7fc181000640 1 -- 172.21.15.89:0/3684457078 learned_addr learned my addr 172.21.15.89:0/3684457078 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.213+0000 7fc181a00640 1 --2- 172.21.15.89:0/3684457078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0913e50 0x561cc0903b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.213+0000 7fc17be00640 1 --2- 172.21.15.89:0/3684457078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0909590 0x561cc092c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.213+0000 7fc181000640 1 -- 172.21.15.89:0/3684457078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0913e50 msgr2=0x561cc0903b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.213+0000 7fc181000640 1 --2- 172.21.15.89:0/3684457078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0913e50 0x561cc0903b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.213+0000 7fc181000640 1 -- 172.21.15.89:0/3684457078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0909590 msgr2=0x561cc092c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.213+0000 7fc181000640 1 --2- 172.21.15.89:0/3684457078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0909590 0x561cc092c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.213+0000 7fc181000640 1 -- 172.21.15.89:0/3684457078 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cc0905030 con 0x561cc0908c70 2024-09-10T08:46:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.213+0000 7fc181000640 1 --2- 172.21.15.89:0/3684457078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0908c70 0x561cc092bb20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x561cc0926a90 tx=0x7fc16c014070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af5b59c36f59be13 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.215+0000 7fc171600640 1 -- 172.21.15.89:0/3684457078 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c027540 con 0x561cc0908c70 2024-09-10T08:46:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.215+0000 7fc171600640 1 -- 172.21.15.89:0/3684457078 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc16c027bc0 con 0x561cc0908c70 2024-09-10T08:46:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.215+0000 7fc171600640 1 -- 172.21.15.89:0/3684457078 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c01e420 con 0x561cc0908c70 2024-09-10T08:46:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.216+0000 7fc186447740 1 -- 172.21.15.89:0/3684457078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0908c70 msgr2=0x561cc092bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:30.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.216+0000 7fc186447740 1 --2- 172.21.15.89:0/3684457078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0908c70 0x561cc092bb20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x561cc0926a90 tx=0x7fc16c014070 comp rx=0 tx=0).stop 2024-09-10T08:46:30.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.217+0000 7fc186447740 1 -- 172.21.15.89:0/3684457078 shutdown_connections 2024-09-10T08:46:30.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.217+0000 7fc186447740 1 --2- 172.21.15.89:0/3684457078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0913e50 0x561cc0903b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.217+0000 7fc186447740 1 --2- 172.21.15.89:0/3684457078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0909590 0x561cc092c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.217+0000 7fc186447740 1 --2- 172.21.15.89:0/3684457078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0908c70 0x561cc092bb20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.217+0000 7fc186447740 1 -- 172.21.15.89:0/3684457078 >> 172.21.15.89:0/3684457078 conn(0x561cc0905200 msgr2=0x561cc0912c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:30.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.221+0000 7fc186447740 1 -- 172.21.15.89:0/3684457078 shutdown_connections 2024-09-10T08:46:30.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.221+0000 7fc186447740 1 -- 172.21.15.89:0/3684457078 wait complete. 2024-09-10T08:46:30.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.221+0000 7fc186447740 1 -- start start 2024-09-10T08:46:30.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.222+0000 7fc186447740 1 Processor -- start 2024-09-10T08:46:30.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.224+0000 7fc186447740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0908c70 0x561cc0914fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.224+0000 7fc186447740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0909590 0x561cc0915510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.224+0000 7fc186447740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0913e50 0x561cc0915a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.224+0000 7fc186447740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c00e380 con 0x561cc0908c70 2024-09-10T08:46:30.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.224+0000 7fc186447740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c00f7f0 con 0x561cc0909590 2024-09-10T08:46:30.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.224+0000 7fc186447740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c00ea50 con 0x561cc0913e50 2024-09-10T08:46:30.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.224+0000 7fc186447740 10 client.0 _ll_register_callbacks cb 0x7fc174098df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:30.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.225+0000 7fc17be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0909590 0x561cc0915510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.225+0000 7fc181a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0913e50 0x561cc0915a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.225+0000 7fc181a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0913e50 0x561cc0915a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:43390/0 (socket says 172.21.15.89:43390) 2024-09-10T08:46:30.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.225+0000 7fc181a00640 1 -- 172.21.15.89:0/2056185885 learned_addr learned my addr 172.21.15.89:0/2056185885 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:30.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.226+0000 7fc181000640 1 --2- 172.21.15.89:0/2056185885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0908c70 0x561cc0914fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.228+0000 7fc17be00640 1 -- 172.21.15.89:0/2056185885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0913e50 msgr2=0x561cc0915a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:30.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.228+0000 7fc17be00640 1 --2- 172.21.15.89:0/2056185885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0913e50 0x561cc0915a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.228+0000 7fc17be00640 1 -- 172.21.15.89:0/2056185885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0908c70 msgr2=0x561cc0914fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:30.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.228+0000 7fc17be00640 1 --2- 172.21.15.89:0/2056185885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0908c70 0x561cc0914fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.228+0000 7fc17be00640 1 -- 172.21.15.89:0/2056185885 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cc0900770 con 0x561cc0909590 2024-09-10T08:46:30.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.231+0000 7fc166a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:46:30.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.232+0000 7fc17be00640 1 --2- 172.21.15.89:0/2056185885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0909590 0x561cc0915510 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x561cc0912230 tx=0x7fc168005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5636a844e415f0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:30.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.232+0000 7fc166a00640 1 -- 172.21.15.89:0/2056185885 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc168037050 con 0x561cc0909590 2024-09-10T08:46:30.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.232+0000 7fc166a00640 1 -- 172.21.15.89:0/2056185885 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fc168030580 con 0x561cc0909590 2024-09-10T08:46:30.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.232+0000 7fc186447740 1 -- 172.21.15.89:0/2056185885 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc16c027bc0 con 0x561cc0909590 2024-09-10T08:46:30.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.233+0000 7fc166a00640 1 -- 172.21.15.89:0/2056185885 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc168037050 con 0x561cc0909590 2024-09-10T08:46:30.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.233+0000 7fc186447740 10 client.26636 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:30.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.233+0000 7fc186447740 1 -- 172.21.15.89:0/2056185885 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561cc098ec00 con 0x561cc0909590 2024-09-10T08:46:30.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.233+0000 7fc166a00640 1 -- 172.21.15.89:0/2056185885 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc168030780 con 0x561cc0909590 2024-09-10T08:46:30.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.234+0000 7fc166a00640 1 -- 172.21.15.89:0/2056185885 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fc168003ac0 con 0x561cc0909590 2024-09-10T08:46:30.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.236+0000 7fc186447740 10 client.26636 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:30.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.236+0000 7fc186447740 10 client.26636 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:46:30.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.236+0000 7fc186447740 20 client.26636 populate_metadata read hostname 'smithi089' 2024-09-10T08:46:30.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.236+0000 7fc186447740 1 --2- 172.21.15.89:0/2056185885 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x561cc098ef90 0x561cc0991380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.236+0000 7fc186447740 4 client.26636 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:46:30.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.236+0000 7fc186447740 1 -- 172.21.15.89:0/2056185885 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc16c027bc0 con 0x561cc098ef90 2024-09-10T08:46:30.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.236+0000 7fc181000640 1 --2- 172.21.15.89:0/2056185885 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x561cc098ef90 0x561cc0991380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.237+0000 7fc166a00640 10 client.26636 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:46:30.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.237+0000 7fc181000640 1 --2- 172.21.15.89:0/2056185885 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x561cc098ef90 0x561cc0991380 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=86ddc8d3f9e2df2a server_cookie=fe21f004d3a8997 in_seq=0 out_seq=0 2024-09-10T08:46:30.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.237+0000 7fc166a00640 1 -- 172.21.15.89:0/2056185885 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc16c027bc0 con 0x561cc098ef90 2024-09-10T08:46:30.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.237+0000 7fc166a00640 10 client.26636 handle_command_reply: tid=0 2024-09-10T08:46:30.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.238+0000 7fc186447740 1 client.26636 shutdown 2024-09-10T08:46:30.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.239+0000 7fc186447740 1 -- 172.21.15.89:0/2056185885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0909590 msgr2=0x561cc0915510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:30.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.239+0000 7fc186447740 1 --2- 172.21.15.89:0/2056185885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0909590 0x561cc0915510 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x561cc0912230 tx=0x7fc168005f10 comp rx=0 tx=0).stop 2024-09-10T08:46:30.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.239+0000 7fc186447740 1 -- 172.21.15.89:0/2056185885 shutdown_connections 2024-09-10T08:46:30.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.239+0000 7fc186447740 1 --2- 172.21.15.89:0/2056185885 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x561cc098ef90 0x561cc0991380 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.239+0000 7fc186447740 1 --2- 172.21.15.89:0/2056185885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc0913e50 0x561cc0915a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.239+0000 7fc166a00640 0 client.26636 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:46:30.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.239+0000 7fc186447740 1 --2- 172.21.15.89:0/2056185885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561cc0909590 0x561cc0915510 secure :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0x561cc0912230 tx=0x7fc168005f10 comp rx=0 tx=0).stop 2024-09-10T08:46:30.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.239+0000 7fc186447740 1 --2- 172.21.15.89:0/2056185885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cc0908c70 0x561cc0914fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.239+0000 7fc186447740 1 -- 172.21.15.89:0/2056185885 >> 172.21.15.89:0/2056185885 conn(0x561cc0905200 msgr2=0x561cc0910bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:30.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.241+0000 7fc186447740 1 -- 172.21.15.89:0/2056185885 shutdown_connections 2024-09-10T08:46:30.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.241+0000 7fc186447740 1 -- 172.21.15.89:0/2056185885 wait complete. 2024-09-10T08:46:30.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.242+0000 7fc186447740 20 client.26636 trim_cache size 0 max 16384 2024-09-10T08:46:30.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.243+0000 7fc183a00640 1 -- 172.21.15.89:0/1147148490 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc1600a2ed0 msgr2=0x7fc1600a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:30.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.243+0000 7fc183a00640 1 --2- 172.21.15.89:0/1147148490 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc1600a2ed0 0x7fc1600a5390 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0058c0 tx=0x7fc16c009260 comp rx=0 tx=0).stop 2024-09-10T08:46:30.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.243+0000 7fc183a00640 1 -- 172.21.15.89:0/1147148490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc17c105ec0 msgr2=0x7fc17c10e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:30.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.243+0000 7fc183a00640 1 --2- 172.21.15.89:0/1147148490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc17c105ec0 0x7fc17c10e2c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc174009fd0 tx=0x7fc17400ef10 comp rx=0 tx=0).stop 2024-09-10T08:46:30.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.243+0000 7fc183a00640 1 -- 172.21.15.89:0/1147148490 shutdown_connections 2024-09-10T08:46:30.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.243+0000 7fc183a00640 1 --2- 172.21.15.89:0/1147148490 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc1600a2ed0 0x7fc1600a5390 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.243+0000 7fc183a00640 1 --2- 172.21.15.89:0/1147148490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc17c105ec0 0x7fc17c10e2c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.243+0000 7fc183a00640 1 --2- 172.21.15.89:0/1147148490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc17c1011e0 0x7fc17c10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.243+0000 7fc183a00640 1 --2- 172.21.15.89:0/1147148490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc17c1008c0 0x7fc17c114d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.243+0000 7fc183a00640 1 -- 172.21.15.89:0/1147148490 >> 172.21.15.89:0/1147148490 conn(0x7fc17c0fc7d0 msgr2=0x7fc17c106ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:30.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.243+0000 7fc183a00640 1 -- 172.21.15.89:0/1147148490 shutdown_connections 2024-09-10T08:46:30.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:30.243+0000 7fc183a00640 1 -- 172.21.15.89:0/1147148490 wait complete. 2024-09-10T08:46:31.048 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:30 smithi089 ceph-mon[30757]: pgmap v590: 129 pgs: 129 active+clean; 438 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.0 MiB/s wr, 1.03k op/s 2024-09-10T08:46:31.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:30 smithi149 ceph-mon[36167]: pgmap v590: 129 pgs: 129 active+clean; 438 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.0 MiB/s wr, 1.03k op/s 2024-09-10T08:46:31.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:30 smithi111 ceph-mon[35220]: pgmap v590: 129 pgs: 129 active+clean; 438 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.0 MiB/s wr, 1.03k op/s 2024-09-10T08:46:33.144 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:32 smithi089 ceph-mon[30757]: pgmap v591: 129 pgs: 129 active+clean; 423 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.6 MiB/s wr, 1.18k op/s 2024-09-10T08:46:33.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:32 smithi149 ceph-mon[36167]: pgmap v591: 129 pgs: 129 active+clean; 423 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.6 MiB/s wr, 1.18k op/s 2024-09-10T08:46:33.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:32 smithi111 ceph-mon[35220]: pgmap v591: 129 pgs: 129 active+clean; 423 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.6 MiB/s wr, 1.18k op/s 2024-09-10T08:46:33.466 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:46:35.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:34 smithi149 ceph-mon[36167]: pgmap v592: 129 pgs: 129 active+clean; 415 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 740 op/s 2024-09-10T08:46:35.273 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:34 smithi089 ceph-mon[30757]: pgmap v592: 129 pgs: 129 active+clean; 415 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 740 op/s 2024-09-10T08:46:35.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:34 smithi111 ceph-mon[35220]: pgmap v592: 129 pgs: 129 active+clean; 415 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 740 op/s 2024-09-10T08:46:36.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:35 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:46:36.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:35 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:46:36.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:35 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:46:36.250 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:35 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:46:36.250 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:35 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:46:36.250 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:35 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:46:36.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:35 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:46:36.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:35 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:46:36.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:35 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:46:37.033 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:36 smithi089 ceph-mon[30757]: pgmap v593: 129 pgs: 129 active+clean; 413 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 565 op/s 2024-09-10T08:46:37.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:36 smithi149 ceph-mon[36167]: pgmap v593: 129 pgs: 129 active+clean; 413 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 565 op/s 2024-09-10T08:46:37.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:36 smithi111 ceph-mon[35220]: pgmap v593: 129 pgs: 129 active+clean; 413 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 565 op/s 2024-09-10T08:46:38.073 INFO:tasks.workunit.client.0.smithi089.stdout: 130 490830 587 354693 445 307534 1400 2024-09-10T08:46:38.510 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:39.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:38 smithi149 ceph-mon[36167]: pgmap v594: 129 pgs: 129 active+clean; 417 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 614 op/s 2024-09-10T08:46:39.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:38 smithi111 ceph-mon[35220]: pgmap v594: 129 pgs: 129 active+clean; 417 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 614 op/s 2024-09-10T08:46:39.309 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:38 smithi089 ceph-mon[30757]: pgmap v594: 129 pgs: 129 active+clean; 417 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 614 op/s 2024-09-10T08:46:40.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.490+0000 7f31a8a00640 1 -- 172.21.15.89:0/261323187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31a4106f20 msgr2=0x7f31a410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:40.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.490+0000 7f31a8a00640 1 --2- 172.21.15.89:0/261323187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31a4106f20 0x7f31a410eaa0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f319c009ca0 tx=0x7f319c02a8d0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.491+0000 7f31a8a00640 1 -- 172.21.15.89:0/261323187 shutdown_connections 2024-09-10T08:46:40.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.491+0000 7f31a8a00640 1 --2- 172.21.15.89:0/261323187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31a4106f20 0x7f31a410eaa0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.491+0000 7f31a8a00640 1 --2- 172.21.15.89:0/261323187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31a41027a0 0x7f31a41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.491+0000 7f31a8a00640 1 --2- 172.21.15.89:0/261323187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31a4101df0 0x7f31a41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.491+0000 7f31a8a00640 1 -- 172.21.15.89:0/261323187 >> 172.21.15.89:0/261323187 conn(0x7f31a40fd8b0 msgr2=0x7f31a40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:40.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.492+0000 7f31a8a00640 1 -- 172.21.15.89:0/261323187 shutdown_connections 2024-09-10T08:46:40.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.492+0000 7f31a8a00640 1 -- 172.21.15.89:0/261323187 wait complete. 2024-09-10T08:46:40.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.494+0000 7f31a8a00640 1 Processor -- start 2024-09-10T08:46:40.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.494+0000 7f31a8a00640 1 -- start start 2024-09-10T08:46:40.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.495+0000 7f31a8a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31a4101df0 0x7f31a4116370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.496+0000 7f31a8a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31a41027a0 0x7f31a41168b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.496+0000 7f31a1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31a4101df0 0x7f31a4116370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.496+0000 7f31a1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31a4101df0 0x7f31a4116370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:54236/0 (socket says 172.21.15.89:54236) 2024-09-10T08:46:40.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.496+0000 7f31a8a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31a4106f20 0x7f31a411c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.508 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.496+0000 7f31a8a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f31a40761a0 con 0x7f31a41027a0 2024-09-10T08:46:40.509 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.496+0000 7f31a8a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f31a4076020 con 0x7f31a4101df0 2024-09-10T08:46:40.511 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.496+0000 7f31a8a00640 1 -- 172.21.15.89:0/2431254947 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f31a4076320 con 0x7f31a4106f20 2024-09-10T08:46:40.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.496+0000 7f31a1e00640 1 -- 172.21.15.89:0/2431254947 learned_addr learned my addr 172.21.15.89:0/2431254947 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:40.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.497+0000 7f31a1e00640 1 -- 172.21.15.89:0/2431254947 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31a4106f20 msgr2=0x7f31a411c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:40.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.497+0000 7f31a1e00640 1 --2- 172.21.15.89:0/2431254947 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31a4106f20 0x7f31a411c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.513 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.497+0000 7f31a1e00640 1 -- 172.21.15.89:0/2431254947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31a41027a0 msgr2=0x7f31a41168b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:40.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.497+0000 7f31a1400640 1 --2- 172.21.15.89:0/2431254947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31a41027a0 0x7f31a41168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.498+0000 7f31a1e00640 1 --2- 172.21.15.89:0/2431254947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31a41027a0 0x7f31a41168b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.498+0000 7f31a1e00640 1 -- 172.21.15.89:0/2431254947 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a411c950 con 0x7f31a4101df0 2024-09-10T08:46:40.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.498+0000 7f31a1e00640 1 --2- 172.21.15.89:0/2431254947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31a4101df0 0x7f31a4116370 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f319000b450 tx=0x7f319000b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:40.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.499+0000 7f319aa00640 1 -- 172.21.15.89:0/2431254947 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190015020 con 0x7f31a4101df0 2024-09-10T08:46:40.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.499+0000 7f31a8a00640 1 -- 172.21.15.89:0/2431254947 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a411cc40 con 0x7f31a4101df0 2024-09-10T08:46:40.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.499+0000 7f31a8a00640 1 -- 172.21.15.89:0/2431254947 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a4111760 con 0x7f31a4101df0 2024-09-10T08:46:40.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.504+0000 7f31ab463740 1 Processor -- start 2024-09-10T08:46:40.518 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.504+0000 7f31ab463740 1 -- start start 2024-09-10T08:46:40.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.505+0000 7f31ab463740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a17bc70 0x55d65a17c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.505+0000 7f319aa00640 1 -- 172.21.15.89:0/2431254947 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3190011d70 con 0x7f31a4101df0 2024-09-10T08:46:40.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.506+0000 7f319aa00640 1 -- 172.21.15.89:0/2431254947 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319000f340 con 0x7f31a4101df0 2024-09-10T08:46:40.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.506+0000 7f31a2800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a17bc70 0x55d65a17c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.507+0000 7f31ab463740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17c590 0x55d65a186910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.507+0000 7f31a2800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a17bc70 0x55d65a17c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40338/0 (socket says 172.21.15.89:40338) 2024-09-10T08:46:40.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.507+0000 7f31ab463740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a186e50 0x55d65a189240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.507+0000 7f31ab463740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d65a0e23e0 con 0x55d65a17bc70 2024-09-10T08:46:40.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.507+0000 7f31ab463740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d65a089d80 con 0x55d65a17c590 2024-09-10T08:46:40.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.507+0000 7f31ab463740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d65a0de3a0 con 0x55d65a186e50 2024-09-10T08:46:40.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.507+0000 7f31a2800640 1 -- 172.21.15.89:0/2548822300 learned_addr learned my addr 172.21.15.89:0/2548822300 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:40.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.508+0000 7f31a1400640 1 --2- 172.21.15.89:0/2548822300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a186e50 0x55d65a189240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.508+0000 7f31a2800640 1 -- 172.21.15.89:0/2548822300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a186e50 msgr2=0x55d65a189240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:40.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.508+0000 7f31a2800640 1 --2- 172.21.15.89:0/2548822300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a186e50 0x55d65a189240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.508+0000 7f31a2800640 1 -- 172.21.15.89:0/2548822300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17c590 msgr2=0x55d65a186910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:46:40.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.508+0000 7f31a2800640 1 --2- 172.21.15.89:0/2548822300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17c590 0x55d65a186910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.508+0000 7f31a2800640 1 -- 172.21.15.89:0/2548822300 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d65a0a8ae0 con 0x55d65a17bc70 2024-09-10T08:46:40.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.508+0000 7f31a1400640 1 --2- 172.21.15.89:0/2548822300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a186e50 0x55d65a189240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:40.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.509+0000 7f31a2800640 1 --2- 172.21.15.89:0/2548822300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a17bc70 0x55d65a17c050 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f319c02aec0 tx=0x7f319c009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6cf146acf88517 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:40.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.510+0000 7f318f000640 1 -- 172.21.15.89:0/2548822300 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c031d80 con 0x55d65a17bc70 2024-09-10T08:46:40.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.510+0000 7f318f000640 1 -- 172.21.15.89:0/2548822300 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f319c03f070 con 0x55d65a17bc70 2024-09-10T08:46:40.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.510+0000 7f318f000640 1 -- 172.21.15.89:0/2548822300 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c031d80 con 0x55d65a17bc70 2024-09-10T08:46:40.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.511+0000 7f31ab463740 1 -- 172.21.15.89:0/2548822300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a17bc70 msgr2=0x55d65a17c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:40.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.511+0000 7f31ab463740 1 --2- 172.21.15.89:0/2548822300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a17bc70 0x55d65a17c050 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f319c02aec0 tx=0x7f319c009830 comp rx=0 tx=0).stop 2024-09-10T08:46:40.521 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.518+0000 7f31ab463740 1 -- 172.21.15.89:0/2548822300 shutdown_connections 2024-09-10T08:46:40.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.518+0000 7f31ab463740 1 --2- 172.21.15.89:0/2548822300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a186e50 0x55d65a189240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.518+0000 7f31ab463740 1 --2- 172.21.15.89:0/2548822300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17c590 0x55d65a186910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.518+0000 7f31ab463740 1 --2- 172.21.15.89:0/2548822300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a17bc70 0x55d65a17c050 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.518+0000 7f31ab463740 1 -- 172.21.15.89:0/2548822300 >> 172.21.15.89:0/2548822300 conn(0x55d65a178200 msgr2=0x55d65a1785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:40.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.519+0000 7f31ab463740 1 -- 172.21.15.89:0/2548822300 shutdown_connections 2024-09-10T08:46:40.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.519+0000 7f31ab463740 1 -- 172.21.15.89:0/2548822300 wait complete. 2024-09-10T08:46:40.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.520+0000 7f319aa00640 1 -- 172.21.15.89:0/2431254947 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3190018020 con 0x7f31a4101df0 2024-09-10T08:46:40.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.520+0000 7f31ab463740 1 -- start start 2024-09-10T08:46:40.524 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.520+0000 7f31ab463740 1 Processor -- start 2024-09-10T08:46:40.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.521+0000 7f31ab463740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17bc70 0x55d65a183d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.521+0000 7f31ab463740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a17c590 0x55d65a184250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.521+0000 7f31ab463740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a186e50 0x55d65a184790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.521+0000 7f31ab463740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d65a189a70 con 0x55d65a186e50 2024-09-10T08:46:40.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.521+0000 7f31ab463740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d65a09d0f0 con 0x55d65a17bc70 2024-09-10T08:46:40.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.521+0000 7f31ab463740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d65a189bf0 con 0x55d65a17c590 2024-09-10T08:46:40.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.522+0000 7f31a2800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17bc70 0x55d65a183d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.522+0000 7f31a2800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17bc70 0x55d65a183d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:54240/0 (socket says 172.21.15.89:54240) 2024-09-10T08:46:40.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.522+0000 7f31a2800640 1 -- 172.21.15.89:0/3550504343 learned_addr learned my addr 172.21.15.89:0/3550504343 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:40.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.522+0000 7f31a1e00640 1 --2- 172.21.15.89:0/3550504343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a17c590 0x55d65a184250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.525 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.522+0000 7f31ab463740 10 client.0 _ll_register_callbacks cb 0x55d65a084bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:40.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.523+0000 7f31a1e00640 1 -- 172.21.15.89:0/3550504343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17bc70 msgr2=0x55d65a183d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:40.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.523+0000 7f31a1e00640 1 --2- 172.21.15.89:0/3550504343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17bc70 0x55d65a183d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.523+0000 7f31a1e00640 1 -- 172.21.15.89:0/3550504343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a186e50 msgr2=0x55d65a184790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:40.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.523+0000 7f31a1400640 1 --2- 172.21.15.89:0/3550504343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a186e50 0x55d65a184790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.523+0000 7f31a1e00640 1 --2- 172.21.15.89:0/3550504343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a186e50 0x55d65a184790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.523+0000 7f31a1e00640 1 -- 172.21.15.89:0/3550504343 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d659e533c0 con 0x55d65a17c590 2024-09-10T08:46:40.526 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.523+0000 7f31a1400640 1 --2- 172.21.15.89:0/3550504343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a186e50 0x55d65a184790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:40.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.532+0000 7f31a1e00640 1 --2- 172.21.15.89:0/3550504343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a17c590 0x55d65a184250 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f31900cc5a0 tx=0x7f31900d53d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25c84b2c6a5e2a6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:40.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.534+0000 7f3183400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:46:40.535 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.534+0000 7f3183400640 1 -- 172.21.15.89:0/3550504343 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31900e03f0 con 0x55d65a17c590 2024-09-10T08:46:40.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.534+0000 7f3183400640 1 -- 172.21.15.89:0/3550504343 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f31900e0590 con 0x55d65a17c590 2024-09-10T08:46:40.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.534+0000 7f31ab463740 1 -- 172.21.15.89:0/3550504343 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f319c03f070 con 0x55d65a17c590 2024-09-10T08:46:40.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.535+0000 7f3183400640 1 -- 172.21.15.89:0/3550504343 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f31900e0790 con 0x55d65a17c590 2024-09-10T08:46:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.535+0000 7f31ab463740 10 client.26575 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.535+0000 7f319aa00640 1 --2- 172.21.15.89:0/2431254947 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f31880a2b40 0x7f31880a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.535+0000 7f31ab463740 1 -- 172.21.15.89:0/3550504343 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d65a1a6b30 con 0x55d65a17c590 2024-09-10T08:46:40.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.536+0000 7f31a2800640 1 --2- 172.21.15.89:0/2431254947 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f31880a2b40 0x7f31880a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.537+0000 7f319aa00640 1 -- 172.21.15.89:0/2431254947 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f31900cef00 con 0x7f31a4101df0 2024-09-10T08:46:40.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.538+0000 7f3183400640 1 -- 172.21.15.89:0/3550504343 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f319c03f070 con 0x55d65a17c590 2024-09-10T08:46:40.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.538+0000 7f3183400640 1 -- 172.21.15.89:0/3550504343 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f31900de070 con 0x55d65a17c590 2024-09-10T08:46:40.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.538+0000 7f31ab463740 10 client.26575 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.538+0000 7f31ab463740 10 client.26575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:46:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.538+0000 7f31ab463740 20 client.26575 populate_metadata read hostname 'smithi089' 2024-09-10T08:46:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.538+0000 7f31ab463740 1 --2- 172.21.15.89:0/3550504343 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d65a1a8090 0x55d65a1aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.538+0000 7f31ab463740 4 client.26575 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.538+0000 7f31ab463740 1 -- 172.21.15.89:0/3550504343 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d65a0ef950 con 0x55d65a1a8090 2024-09-10T08:46:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.539+0000 7f31a1400640 1 --2- 172.21.15.89:0/3550504343 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d65a1a8090 0x55d65a1aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.539+0000 7f31a2800640 1 --2- 172.21.15.89:0/2431254947 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f31880a2b40 0x7f31880a5000 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f319c031aa0 tx=0x7f319c046040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.540+0000 7f3183400640 10 client.26575 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.540+0000 7f31a1400640 1 --2- 172.21.15.89:0/3550504343 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d65a1a8090 0x55d65a1aa4b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9371ea49e9d3fa2d server_cookie=3e5b1d26a6422589 in_seq=0 out_seq=0 2024-09-10T08:46:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.542+0000 7f3183400640 1 -- 172.21.15.89:0/3550504343 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d65a0ef950 con 0x55d65a1a8090 2024-09-10T08:46:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.542+0000 7f3183400640 10 client.26575 handle_command_reply: tid=0 2024-09-10T08:46:40.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.543+0000 7f31ab463740 1 client.26575 shutdown 2024-09-10T08:46:40.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.549+0000 7f31ab463740 1 -- 172.21.15.89:0/3550504343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a17c590 msgr2=0x55d65a184250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:40.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.549+0000 7f31ab463740 1 --2- 172.21.15.89:0/3550504343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a17c590 0x55d65a184250 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f31900cc5a0 tx=0x7f31900d53d0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.550+0000 7f31ab463740 1 -- 172.21.15.89:0/3550504343 shutdown_connections 2024-09-10T08:46:40.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.550+0000 7f31ab463740 1 --2- 172.21.15.89:0/3550504343 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d65a1a8090 0x55d65a1aa4b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.550+0000 7f31ab463740 1 --2- 172.21.15.89:0/3550504343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a186e50 0x55d65a184790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.550+0000 7f31ab463740 1 --2- 172.21.15.89:0/3550504343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a17c590 0x55d65a184250 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.550+0000 7f31ab463740 1 --2- 172.21.15.89:0/3550504343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17bc70 0x55d65a183d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.550+0000 7f31ab463740 1 -- 172.21.15.89:0/3550504343 >> 172.21.15.89:0/3550504343 conn(0x55d65a178200 msgr2=0x55d65a17df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:40.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.554+0000 7f3183400640 0 client.26575 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:40.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.554+0000 7f31ab463740 1 -- 172.21.15.89:0/3550504343 shutdown_connections 2024-09-10T08:46:40.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.554+0000 7f31ab463740 1 -- 172.21.15.89:0/3550504343 wait complete. 2024-09-10T08:46:40.557 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.555+0000 7f31ab463740 20 client.26575 trim_cache size 0 max 16384 2024-09-10T08:46:40.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.610+0000 7f31ab463740 1 Processor -- start 2024-09-10T08:46:40.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.610+0000 7f31ab463740 1 -- start start 2024-09-10T08:46:40.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.611+0000 7f31ab463740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17bc70 0x55d65a17e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.611+0000 7f31ab463740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a17c590 0x55d65a1825e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.611+0000 7f31ab463740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a186e50 0x55d65a1845e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.611+0000 7f31ab463740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3184012670 con 0x55d65a17c590 2024-09-10T08:46:40.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.611+0000 7f31ab463740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f318400ff40 con 0x55d65a17bc70 2024-09-10T08:46:40.626 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.611+0000 7f31ab463740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f318400dd90 con 0x55d65a186e50 2024-09-10T08:46:40.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.612+0000 7f31a2800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a186e50 0x55d65a1845e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.612+0000 7f31a2800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a186e50 0x55d65a1845e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:49688/0 (socket says 172.21.15.89:49688) 2024-09-10T08:46:40.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.612+0000 7f31a2800640 1 -- 172.21.15.89:0/3679283597 learned_addr learned my addr 172.21.15.89:0/3679283597 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:40.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.612+0000 7f31a1e00640 1 --2- 172.21.15.89:0/3679283597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17bc70 0x55d65a17e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.613+0000 7f31a2800640 1 -- 172.21.15.89:0/3679283597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17bc70 msgr2=0x55d65a17e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:40.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.613+0000 7f31a2800640 1 --2- 172.21.15.89:0/3679283597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17bc70 0x55d65a17e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.613+0000 7f31a2800640 1 -- 172.21.15.89:0/3679283597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a17c590 msgr2=0x55d65a1825e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:40.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.613+0000 7f31a2800640 1 --2- 172.21.15.89:0/3679283597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a17c590 0x55d65a1825e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.613+0000 7f31a2800640 1 -- 172.21.15.89:0/3679283597 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d65a178030 con 0x55d65a186e50 2024-09-10T08:46:40.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.615+0000 7f31a2800640 1 --2- 172.21.15.89:0/3679283597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a186e50 0x55d65a1845e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f319c031970 tx=0x7f319c009830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16022eda7a9bb2e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:40.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.616+0000 7f318e600640 1 -- 172.21.15.89:0/3679283597 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c004450 con 0x55d65a186e50 2024-09-10T08:46:40.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.616+0000 7f318e600640 1 -- 172.21.15.89:0/3679283597 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f319c03f040 con 0x55d65a186e50 2024-09-10T08:46:40.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.616+0000 7f318e600640 1 -- 172.21.15.89:0/3679283597 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c004450 con 0x55d65a186e50 2024-09-10T08:46:40.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.617+0000 7f31ab463740 1 -- 172.21.15.89:0/3679283597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a186e50 msgr2=0x55d65a1845e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:40.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.617+0000 7f31ab463740 1 --2- 172.21.15.89:0/3679283597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a186e50 0x55d65a1845e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f319c031970 tx=0x7f319c009830 comp rx=0 tx=0).stop 2024-09-10T08:46:40.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.619+0000 7f31ab463740 1 -- 172.21.15.89:0/3679283597 shutdown_connections 2024-09-10T08:46:40.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.619+0000 7f31ab463740 1 --2- 172.21.15.89:0/3679283597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a186e50 0x55d65a1845e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.619+0000 7f31ab463740 1 --2- 172.21.15.89:0/3679283597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a17c590 0x55d65a1825e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.619+0000 7f31ab463740 1 --2- 172.21.15.89:0/3679283597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17bc70 0x55d65a17e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.620+0000 7f31ab463740 1 -- 172.21.15.89:0/3679283597 >> 172.21.15.89:0/3679283597 conn(0x55d65a178200 msgr2=0x55d65a185d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:40.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.621+0000 7f31ab463740 1 -- 172.21.15.89:0/3679283597 shutdown_connections 2024-09-10T08:46:40.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.621+0000 7f31ab463740 1 -- 172.21.15.89:0/3679283597 wait complete. 2024-09-10T08:46:40.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.622+0000 7f31ab463740 1 -- start start 2024-09-10T08:46:40.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.625+0000 7f31ab463740 1 Processor -- start 2024-09-10T08:46:40.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.628+0000 7f31ab463740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17bc70 0x55d65a18f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.632+0000 7f31ab463740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a17c590 0x55d65a18a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.632+0000 7f31a1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17bc70 0x55d65a18f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.636+0000 7f31ab463740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a186e50 0x55d65a18ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.636+0000 7f31ab463740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f318400e400 con 0x55d65a17c590 2024-09-10T08:46:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.636+0000 7f31ab463740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f318400f870 con 0x55d65a17bc70 2024-09-10T08:46:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.636+0000 7f31ab463740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f318400ead0 con 0x55d65a186e50 2024-09-10T08:46:40.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.636+0000 7f31ab463740 10 client.0 _ll_register_callbacks cb 0x7f31900d2700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:40.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.636+0000 7f31a1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17bc70 0x55d65a18f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:54244/0 (socket says 172.21.15.89:54244) 2024-09-10T08:46:40.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.636+0000 7f31a1e00640 1 -- 172.21.15.89:0/3679161785 learned_addr learned my addr 172.21.15.89:0/3679161785 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:40.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.636+0000 7f31a1e00640 1 -- 172.21.15.89:0/3679161785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a186e50 msgr2=0x55d65a18ada0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:46:40.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.636+0000 7f31a1e00640 1 --2- 172.21.15.89:0/3679161785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a186e50 0x55d65a18ada0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.636+0000 7f31a1e00640 1 -- 172.21.15.89:0/3679161785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a17c590 msgr2=0x55d65a18a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.636+0000 7f31a1e00640 1 --2- 172.21.15.89:0/3679161785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a17c590 0x55d65a18a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.637+0000 7f31a1e00640 1 -- 172.21.15.89:0/3679161785 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d65a173770 con 0x55d65a17bc70 2024-09-10T08:46:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.637+0000 7f31a1e00640 1 --2- 172.21.15.89:0/3679161785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17bc70 0x55d65a18f530 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3190009fd0 tx=0x7f319009b3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9d650eea3612206 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.637+0000 7f3183400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:46:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.638+0000 7f3183400640 1 -- 172.21.15.89:0/3679161785 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31900975d0 con 0x55d65a17bc70 2024-09-10T08:46:40.640 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.638+0000 7f3183400640 1 -- 172.21.15.89:0/3679161785 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f3190099630 con 0x55d65a17bc70 2024-09-10T08:46:40.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.638+0000 7f31ab463740 1 -- 172.21.15.89:0/3679161785 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f319c03f040 con 0x55d65a17bc70 2024-09-10T08:46:40.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.639+0000 7f3183400640 1 -- 172.21.15.89:0/3679161785 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f319009ae70 con 0x55d65a17bc70 2024-09-10T08:46:40.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.639+0000 7f31ab463740 10 client.26648 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:40.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.639+0000 7f31ab463740 1 -- 172.21.15.89:0/3679161785 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d65a212a40 con 0x55d65a17bc70 2024-09-10T08:46:40.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.639+0000 7f3183400640 1 -- 172.21.15.89:0/3679161785 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f319c03f040 con 0x55d65a17bc70 2024-09-10T08:46:40.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.644+0000 7f3183400640 1 -- 172.21.15.89:0/3679161785 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f31900de070 con 0x55d65a17bc70 2024-09-10T08:46:40.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.644+0000 7f31ab463740 10 client.26648 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:40.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.644+0000 7f31ab463740 10 client.26648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:46:40.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.644+0000 7f31ab463740 20 client.26648 populate_metadata read hostname 'smithi089' 2024-09-10T08:46:40.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.644+0000 7f31ab463740 1 --2- 172.21.15.89:0/3679161785 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d65a213100 0x55d65a215570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.644+0000 7f31ab463740 4 client.26648 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:46:40.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.644+0000 7f31ab463740 1 -- 172.21.15.89:0/3679161785 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d65a0e9e60 con 0x55d65a213100 2024-09-10T08:46:40.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.645+0000 7f31a1400640 1 --2- 172.21.15.89:0/3679161785 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d65a213100 0x55d65a215570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.646+0000 7f31a1400640 1 --2- 172.21.15.89:0/3679161785 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d65a213100 0x55d65a215570 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81cb315fff02b66e server_cookie=1988673d2e42ac1f in_seq=0 out_seq=0 2024-09-10T08:46:40.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.646+0000 7f3183400640 10 client.26648 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.649+0000 7f3183400640 1 -- 172.21.15.89:0/3679161785 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d65a0e9e60 con 0x55d65a213100 2024-09-10T08:46:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.649+0000 7f3183400640 10 client.26648 handle_command_reply: tid=0 2024-09-10T08:46:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.649+0000 7f31ab463740 1 client.26648 shutdown 2024-09-10T08:46:40.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.649+0000 7f31ab463740 1 -- 172.21.15.89:0/3679161785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17bc70 msgr2=0x55d65a18f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.649+0000 7f31ab463740 1 --2- 172.21.15.89:0/3679161785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17bc70 0x55d65a18f530 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3190009fd0 tx=0x7f319009b3d0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.649+0000 7f31ab463740 1 -- 172.21.15.89:0/3679161785 shutdown_connections 2024-09-10T08:46:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.649+0000 7f31ab463740 1 --2- 172.21.15.89:0/3679161785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d65a186e50 0x55d65a18ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.649+0000 7f31ab463740 1 --2- 172.21.15.89:0/3679161785 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d65a213100 0x55d65a215570 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.649+0000 7f31ab463740 1 --2- 172.21.15.89:0/3679161785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d65a17c590 0x55d65a18a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.649+0000 7f31ab463740 1 --2- 172.21.15.89:0/3679161785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65a17bc70 0x55d65a18f530 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.649+0000 7f31ab463740 1 -- 172.21.15.89:0/3679161785 >> 172.21.15.89:0/3679161785 conn(0x55d65a178200 msgr2=0x55d65a191da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.650+0000 7f3183400640 0 client.26648 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.650+0000 7f31ab463740 1 -- 172.21.15.89:0/3679161785 shutdown_connections 2024-09-10T08:46:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.650+0000 7f31ab463740 1 -- 172.21.15.89:0/3679161785 wait complete. 2024-09-10T08:46:40.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.650+0000 7f31ab463740 20 client.26648 trim_cache size 0 max 16384 2024-09-10T08:46:40.653 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:46:40.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.651+0000 7f31a8a00640 1 -- 172.21.15.89:0/2431254947 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f31880a2b40 msgr2=0x7f31880a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.651+0000 7f31a8a00640 1 --2- 172.21.15.89:0/2431254947 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f31880a2b40 0x7f31880a5000 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f319c031aa0 tx=0x7f319c046040 comp rx=0 tx=0).stop 2024-09-10T08:46:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.651+0000 7f31a8a00640 1 -- 172.21.15.89:0/2431254947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31a4101df0 msgr2=0x7f31a4116370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.651+0000 7f31a8a00640 1 --2- 172.21.15.89:0/2431254947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31a4101df0 0x7f31a4116370 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f319000b450 tx=0x7f319000b910 comp rx=0 tx=0).stop 2024-09-10T08:46:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.651+0000 7f31a8a00640 1 -- 172.21.15.89:0/2431254947 shutdown_connections 2024-09-10T08:46:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.651+0000 7f31a8a00640 1 --2- 172.21.15.89:0/2431254947 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f31880a2b40 0x7f31880a5000 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.651+0000 7f31a8a00640 1 --2- 172.21.15.89:0/2431254947 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31a4106f20 0x7f31a411c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.651+0000 7f31a8a00640 1 --2- 172.21.15.89:0/2431254947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31a41027a0 0x7f31a41168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.651+0000 7f31a8a00640 1 --2- 172.21.15.89:0/2431254947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31a4101df0 0x7f31a4116370 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.651+0000 7f31a8a00640 1 -- 172.21.15.89:0/2431254947 >> 172.21.15.89:0/2431254947 conn(0x7f31a40fd8b0 msgr2=0x7f31a40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.652+0000 7f31a8a00640 1 -- 172.21.15.89:0/2431254947 shutdown_connections 2024-09-10T08:46:40.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:40.652+0000 7f31a8a00640 1 -- 172.21.15.89:0/2431254947 wait complete. 2024-09-10T08:46:41.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:40 smithi149 ceph-mon[36167]: pgmap v595: 129 pgs: 129 active+clean; 417 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 444 op/s 2024-09-10T08:46:41.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:40 smithi111 ceph-mon[35220]: pgmap v595: 129 pgs: 129 active+clean; 417 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 444 op/s 2024-09-10T08:46:41.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:40 smithi089 ceph-mon[30757]: pgmap v595: 129 pgs: 129 active+clean; 417 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 444 op/s 2024-09-10T08:46:43.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:42 smithi089 ceph-mon[30757]: pgmap v596: 129 pgs: 129 active+clean; 425 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 550 op/s 2024-09-10T08:46:43.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:42 smithi149 ceph-mon[36167]: pgmap v596: 129 pgs: 129 active+clean; 425 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 550 op/s 2024-09-10T08:46:43.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:42 smithi111 ceph-mon[35220]: pgmap v596: 129 pgs: 129 active+clean; 425 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 550 op/s 2024-09-10T08:46:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.652+0000 7feaa4a00640 1 -- 172.21.15.89:0/1120361555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa0109590 msgr2=0x7feaa0109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.652+0000 7feaa4a00640 1 --2- 172.21.15.89:0/1120361555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa0109590 0x7feaa0109a10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fea8c007380 tx=0x7fea8c02aff0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.652+0000 7feaa4a00640 1 -- 172.21.15.89:0/1120361555 shutdown_connections 2024-09-10T08:46:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.652+0000 7feaa4a00640 1 --2- 172.21.15.89:0/1120361555 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feaa0109f50 0x7feaa01107e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.652+0000 7feaa4a00640 1 --2- 172.21.15.89:0/1120361555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa0109590 0x7feaa0109a10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.652+0000 7feaa4a00640 1 --2- 172.21.15.89:0/1120361555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feaa00735d0 0x7feaa00739d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.652+0000 7feaa4a00640 1 -- 172.21.15.89:0/1120361555 >> 172.21.15.89:0/1120361555 conn(0x7feaa006ead0 msgr2=0x7feaa0070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.686+0000 7feaa4a00640 1 -- 172.21.15.89:0/1120361555 shutdown_connections 2024-09-10T08:46:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.691+0000 7feaa4a00640 1 -- 172.21.15.89:0/1120361555 wait complete. 2024-09-10T08:46:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.692+0000 7feaa4a00640 1 Processor -- start 2024-09-10T08:46:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.708+0000 7feaa4a00640 1 -- start start 2024-09-10T08:46:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.708+0000 7feaa4a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feaa0109590 0x7feaa019e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.708+0000 7feaa4a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feaa0109f50 0x7feaa019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.708+0000 7feaa4a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa01b4b00 0x7feaa01b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.708+0000 7feaa4a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7feaa0113bf0 con 0x7feaa01b4b00 2024-09-10T08:46:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.708+0000 7feaa4a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7feaa0113a70 con 0x7feaa0109590 2024-09-10T08:46:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.708+0000 7feaa4a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7feaa0113d70 con 0x7feaa0109f50 2024-09-10T08:46:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.708+0000 7fea9d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feaa0109f50 0x7feaa019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.709+0000 7fea9d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feaa0109f50 0x7feaa019e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:40260/0 (socket says 172.21.15.89:40260) 2024-09-10T08:46:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.709+0000 7fea9d400640 1 -- 172.21.15.89:0/3068850560 learned_addr learned my addr 172.21.15.89:0/3068850560 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.709+0000 7fea9de00640 1 --2- 172.21.15.89:0/3068850560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feaa0109590 0x7feaa019e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.709+0000 7fea9f200640 1 --2- 172.21.15.89:0/3068850560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa01b4b00 0x7feaa01b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.710+0000 7fea9f200640 1 -- 172.21.15.89:0/3068850560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feaa0109f50 msgr2=0x7feaa019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.710+0000 7fea9f200640 1 --2- 172.21.15.89:0/3068850560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feaa0109f50 0x7feaa019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.710+0000 7fea9f200640 1 -- 172.21.15.89:0/3068850560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feaa0109590 msgr2=0x7feaa019e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.710+0000 7fea9f200640 1 --2- 172.21.15.89:0/3068850560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feaa0109590 0x7feaa019e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.710+0000 7fea9f200640 1 -- 172.21.15.89:0/3068850560 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa01b7430 con 0x7feaa01b4b00 2024-09-10T08:46:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.711+0000 7fea9f200640 1 --2- 172.21.15.89:0/3068850560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa01b4b00 0x7feaa01b6ef0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fea9800aef0 tx=0x7fea9800d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.720+0000 7fea96a00640 1 -- 172.21.15.89:0/3068850560 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9803e070 con 0x7feaa01b4b00 2024-09-10T08:46:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.720+0000 7fea96a00640 1 -- 172.21.15.89:0/3068850560 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea9800cb30 con 0x7feaa01b4b00 2024-09-10T08:46:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.720+0000 7feaa4a00640 1 -- 172.21.15.89:0/3068850560 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa0074480 con 0x7feaa01b4b00 2024-09-10T08:46:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.720+0000 7feaa4a00640 1 -- 172.21.15.89:0/3068850560 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa0074a50 con 0x7feaa01b4b00 2024-09-10T08:46:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.728+0000 7fea96a00640 1 -- 172.21.15.89:0/3068850560 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98039bb0 con 0x7feaa01b4b00 2024-09-10T08:46:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.738+0000 7fea96a00640 1 -- 172.21.15.89:0/3068850560 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fea98041070 con 0x7feaa01b4b00 2024-09-10T08:46:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.742+0000 7feaa7a6f740 1 Processor -- start 2024-09-10T08:46:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.746+0000 7feaa7a6f740 1 -- start start 2024-09-10T08:46:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.747+0000 7feaa7a6f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ba3b17260 0x559ba3b17660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.747+0000 7feaa7a6f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17ba0 0x559ba3b1c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.747+0000 7feaa7a6f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559ba3b1cc30 0x559ba3b1f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.747+0000 7feaa7a6f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559ba3a7c100 con 0x559ba3b17260 2024-09-10T08:46:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.747+0000 7feaa7a6f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559ba3a241b0 con 0x559ba3b1cc30 2024-09-10T08:46:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.747+0000 7feaa7a6f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559ba3a780c0 con 0x559ba3b17ba0 2024-09-10T08:46:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.748+0000 7fea9de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17ba0 0x559ba3b1c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.748+0000 7fea9de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17ba0 0x559ba3b1c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:40268/0 (socket says 172.21.15.89:40268) 2024-09-10T08:46:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.748+0000 7fea9de00640 1 -- 172.21.15.89:0/4020054778 learned_addr learned my addr 172.21.15.89:0/4020054778 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.748+0000 7fea96a00640 1 --2- 172.21.15.89:0/3068850560 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fea840a2d00 0x7fea840a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.749+0000 7fea9de00640 1 -- 172.21.15.89:0/4020054778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559ba3b1cc30 msgr2=0x559ba3b1f0f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:46:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.749+0000 7fea9de00640 1 --2- 172.21.15.89:0/4020054778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559ba3b1cc30 0x559ba3b1f0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.749+0000 7fea9de00640 1 -- 172.21.15.89:0/4020054778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ba3b17260 msgr2=0x559ba3b17660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:46:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.749+0000 7fea9de00640 1 --2- 172.21.15.89:0/4020054778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ba3b17260 0x559ba3b17660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.749+0000 7fea9de00640 1 -- 172.21.15.89:0/4020054778 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ba378f210 con 0x559ba3b17ba0 2024-09-10T08:46:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.749+0000 7fea9de00640 1 --2- 172.21.15.89:0/4020054778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17ba0 0x559ba3b1c660 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fea90009990 tx=0x7fea90009e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=302c833a3405517a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.752+0000 7fea96a00640 1 -- 172.21.15.89:0/3068850560 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fea9800c6f0 con 0x7feaa01b4b00 2024-09-10T08:46:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.755+0000 7fea9f200640 1 --2- 172.21.15.89:0/3068850560 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fea840a2d00 0x7fea840a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.757+0000 7fea9f200640 1 --2- 172.21.15.89:0/3068850560 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fea840a2d00 0x7fea840a51c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fea980be900 tx=0x7fea98034000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.758+0000 7fea8b000640 1 -- 172.21.15.89:0/4020054778 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90017070 con 0x559ba3b17ba0 2024-09-10T08:46:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.758+0000 7fea8b000640 1 -- 172.21.15.89:0/4020054778 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea90013470 con 0x559ba3b17ba0 2024-09-10T08:46:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.758+0000 7fea8b000640 1 -- 172.21.15.89:0/4020054778 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90012630 con 0x559ba3b17ba0 2024-09-10T08:46:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.764+0000 7feaa7a6f740 1 -- 172.21.15.89:0/4020054778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17ba0 msgr2=0x559ba3b1c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.764+0000 7feaa7a6f740 1 --2- 172.21.15.89:0/4020054778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17ba0 0x559ba3b1c660 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fea90009990 tx=0x7fea90009e50 comp rx=0 tx=0).stop 2024-09-10T08:46:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.772+0000 7feaa7a6f740 1 -- 172.21.15.89:0/4020054778 shutdown_connections 2024-09-10T08:46:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.772+0000 7feaa7a6f740 1 --2- 172.21.15.89:0/4020054778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559ba3b1cc30 0x559ba3b1f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.772+0000 7feaa7a6f740 1 --2- 172.21.15.89:0/4020054778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17ba0 0x559ba3b1c660 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.772+0000 7feaa7a6f740 1 --2- 172.21.15.89:0/4020054778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ba3b17260 0x559ba3b17660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.772+0000 7feaa7a6f740 1 -- 172.21.15.89:0/4020054778 >> 172.21.15.89:0/4020054778 conn(0x559ba3b128d0 msgr2=0x559ba3b12cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.776+0000 7feaa7a6f740 1 -- 172.21.15.89:0/4020054778 shutdown_connections 2024-09-10T08:46:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.784+0000 7feaa7a6f740 1 -- 172.21.15.89:0/4020054778 wait complete. 2024-09-10T08:46:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.784+0000 7feaa7a6f740 1 -- start start 2024-09-10T08:46:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.784+0000 7feaa7a6f740 1 Processor -- start 2024-09-10T08:46:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.792+0000 7feaa7a6f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559ba3b17260 0x559ba3b21470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.792+0000 7feaa7a6f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17ba0 0x559ba3b219b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.792+0000 7feaa7a6f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ba3b1cc30 0x559ba3b289f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.792+0000 7feaa7a6f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559ba3b1fc00 con 0x559ba3b1cc30 2024-09-10T08:46:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.792+0000 7feaa7a6f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559ba3a05090 con 0x559ba3b17260 2024-09-10T08:46:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.792+0000 7feaa7a6f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559ba3b1fd80 con 0x559ba3b17ba0 2024-09-10T08:46:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.793+0000 7feaa7a6f740 10 client.0 _ll_register_callbacks cb 0x559ba3a59f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.793+0000 7fea9de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17ba0 0x559ba3b219b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.793+0000 7fea9de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17ba0 0x559ba3b219b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:40280/0 (socket says 172.21.15.89:40280) 2024-09-10T08:46:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.793+0000 7fea9de00640 1 -- 172.21.15.89:0/3088710575 learned_addr learned my addr 172.21.15.89:0/3088710575 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.793+0000 7fea9de00640 1 -- 172.21.15.89:0/3088710575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559ba3b17260 msgr2=0x559ba3b21470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:46:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.793+0000 7fea9de00640 1 --2- 172.21.15.89:0/3088710575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559ba3b17260 0x559ba3b21470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.793+0000 7fea9de00640 1 -- 172.21.15.89:0/3088710575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ba3b1cc30 msgr2=0x559ba3b289f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:46:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.793+0000 7fea9de00640 1 --2- 172.21.15.89:0/3088710575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ba3b1cc30 0x559ba3b289f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.793+0000 7fea9de00640 1 -- 172.21.15.89:0/3088710575 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ba385b620 con 0x559ba3b17ba0 2024-09-10T08:46:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.799+0000 7fea9de00640 1 --2- 172.21.15.89:0/3088710575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17ba0 0x559ba3b219b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fea9000ec90 tx=0x7fea9000ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc75153323b4ee03 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.805+0000 7fea7f400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:46:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.805+0000 7fea7f400640 1 -- 172.21.15.89:0/3088710575 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90017050 con 0x559ba3b17ba0 2024-09-10T08:46:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.805+0000 7fea7f400640 1 -- 172.21.15.89:0/3088710575 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fea9001c650 con 0x559ba3b17ba0 2024-09-10T08:46:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.814+0000 7feaa7a6f740 1 -- 172.21.15.89:0/3088710575 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea90013470 con 0x559ba3b17ba0 2024-09-10T08:46:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.814+0000 7fea7f400640 1 -- 172.21.15.89:0/3088710575 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fea9001c850 con 0x559ba3b17ba0 2024-09-10T08:46:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.814+0000 7feaa7a6f740 10 client.26587 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.814+0000 7feaa7a6f740 1 -- 172.21.15.89:0/3088710575 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559ba3b2b810 con 0x559ba3b17ba0 2024-09-10T08:46:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.829+0000 7fea7f400640 1 -- 172.21.15.89:0/3088710575 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea90013470 con 0x559ba3b17ba0 2024-09-10T08:46:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.829+0000 7fea7f400640 1 -- 172.21.15.89:0/3088710575 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fea90026410 con 0x559ba3b17ba0 2024-09-10T08:46:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.829+0000 7feaa7a6f740 10 client.26587 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.829+0000 7feaa7a6f740 10 client.26587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:46:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.829+0000 7feaa7a6f740 20 client.26587 populate_metadata read hostname 'smithi089' 2024-09-10T08:46:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.829+0000 7feaa7a6f740 1 --2- 172.21.15.89:0/3088710575 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x559ba3b2cc70 0x559ba3b2f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.829+0000 7feaa7a6f740 4 client.26587 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.829+0000 7feaa7a6f740 1 -- 172.21.15.89:0/3088710575 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ba3a89830 con 0x559ba3b2cc70 2024-09-10T08:46:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.829+0000 7fea9d400640 1 --2- 172.21.15.89:0/3088710575 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x559ba3b2cc70 0x559ba3b2f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.831+0000 7fea7f400640 10 client.26587 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.831+0000 7fea9d400640 1 --2- 172.21.15.89:0/3088710575 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x559ba3b2cc70 0x559ba3b2f060 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2601e6287553eb7e server_cookie=aa9a1c75ad1f16a6 in_seq=0 out_seq=0 2024-09-10T08:46:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.833+0000 7fea7f400640 1 -- 172.21.15.89:0/3088710575 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ba3a89830 con 0x559ba3b2cc70 2024-09-10T08:46:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.833+0000 7fea7f400640 10 client.26587 handle_command_reply: tid=0 2024-09-10T08:46:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.833+0000 7feaa7a6f740 1 client.26587 shutdown 2024-09-10T08:46:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.837+0000 7feaa7a6f740 1 -- 172.21.15.89:0/3088710575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17ba0 msgr2=0x559ba3b219b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.837+0000 7feaa7a6f740 1 --2- 172.21.15.89:0/3088710575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17ba0 0x559ba3b219b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fea9000ec90 tx=0x7fea9000ecc0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.837+0000 7feaa7a6f740 1 -- 172.21.15.89:0/3088710575 shutdown_connections 2024-09-10T08:46:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.837+0000 7feaa7a6f740 1 --2- 172.21.15.89:0/3088710575 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x559ba3b2cc70 0x559ba3b2f060 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.837+0000 7feaa7a6f740 1 --2- 172.21.15.89:0/3088710575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ba3b1cc30 0x559ba3b289f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.837+0000 7feaa7a6f740 1 --2- 172.21.15.89:0/3088710575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17ba0 0x559ba3b219b0 secure :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fea9000ec90 tx=0x7fea9000ecc0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.838+0000 7feaa7a6f740 1 --2- 172.21.15.89:0/3088710575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559ba3b17260 0x559ba3b21470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.838+0000 7feaa7a6f740 1 -- 172.21.15.89:0/3088710575 >> 172.21.15.89:0/3088710575 conn(0x559ba3b128d0 msgr2=0x559ba3b140e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.838+0000 7fea7f400640 0 client.26587 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.838+0000 7feaa7a6f740 1 -- 172.21.15.89:0/3088710575 shutdown_connections 2024-09-10T08:46:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.843+0000 7feaa7a6f740 1 -- 172.21.15.89:0/3088710575 wait complete. 2024-09-10T08:46:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.843+0000 7feaa7a6f740 20 client.26587 trim_cache size 0 max 16384 2024-09-10T08:46:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.930+0000 7feaa7a6f740 1 Processor -- start 2024-09-10T08:46:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.952+0000 7feaa7a6f740 1 -- start start 2024-09-10T08:46:43.953 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.952+0000 7feaa7a6f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17260 0x559ba3b12cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.952+0000 7feaa7a6f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ba3b1cc30 0x559ba3b289b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.952+0000 7feaa7a6f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559ba3b2f390 0x559ba3b131f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.952+0000 7feaa7a6f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fea800126f0 con 0x559ba3b1cc30 2024-09-10T08:46:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.952+0000 7feaa7a6f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fea8000ffc0 con 0x559ba3b2f390 2024-09-10T08:46:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.952+0000 7feaa7a6f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fea8000ddd0 con 0x559ba3b17260 2024-09-10T08:46:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.953+0000 7fea9d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17260 0x559ba3b12cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.953+0000 7fea9d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17260 0x559ba3b12cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:40296/0 (socket says 172.21.15.89:40296) 2024-09-10T08:46:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.953+0000 7fea9d400640 1 -- 172.21.15.89:0/578075653 learned_addr learned my addr 172.21.15.89:0/578075653 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.953+0000 7fea9d400640 1 --2- 172.21.15.89:0/578075653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559ba3b2f390 0x559ba3b131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.954+0000 7fea9d400640 1 -- 172.21.15.89:0/578075653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559ba3b2f390 msgr2=0x559ba3b131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.954+0000 7fea9d400640 1 --2- 172.21.15.89:0/578075653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559ba3b2f390 0x559ba3b131f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.954+0000 7fea9d400640 1 -- 172.21.15.89:0/578075653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ba3b1cc30 msgr2=0x559ba3b289b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.954+0000 7fea9d400640 1 --2- 172.21.15.89:0/578075653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ba3b1cc30 0x559ba3b289b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.954+0000 7fea9d400640 1 -- 172.21.15.89:0/578075653 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ba3b222f0 con 0x559ba3b17260 2024-09-10T08:46:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.954+0000 7fea9d400640 1 --2- 172.21.15.89:0/578075653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17260 0x559ba3b12cb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7feaa019f870 tx=0x7fea8c02aff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=796dbc5f411f16e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.977+0000 7fea8a600640 1 -- 172.21.15.89:0/578075653 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c007a50 con 0x559ba3b17260 2024-09-10T08:46:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.977+0000 7fea8a600640 1 -- 172.21.15.89:0/578075653 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea8c007bf0 con 0x559ba3b17260 2024-09-10T08:46:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.977+0000 7fea8a600640 1 -- 172.21.15.89:0/578075653 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c033d30 con 0x559ba3b17260 2024-09-10T08:46:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.984+0000 7feaa7a6f740 1 -- 172.21.15.89:0/578075653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17260 msgr2=0x559ba3b12cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:43.984+0000 7feaa7a6f740 1 --2- 172.21.15.89:0/578075653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17260 0x559ba3b12cb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7feaa019f870 tx=0x7fea8c02aff0 comp rx=0 tx=0).stop 2024-09-10T08:46:44.011 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.008+0000 7feaa7a6f740 1 -- 172.21.15.89:0/578075653 shutdown_connections 2024-09-10T08:46:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.008+0000 7feaa7a6f740 1 --2- 172.21.15.89:0/578075653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559ba3b2f390 0x559ba3b131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.008+0000 7feaa7a6f740 1 --2- 172.21.15.89:0/578075653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ba3b1cc30 0x559ba3b289b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.008+0000 7feaa7a6f740 1 --2- 172.21.15.89:0/578075653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b17260 0x559ba3b12cb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.008+0000 7feaa7a6f740 1 -- 172.21.15.89:0/578075653 >> 172.21.15.89:0/578075653 conn(0x559ba3b128d0 msgr2=0x559ba3b2ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.010+0000 7feaa7a6f740 1 -- 172.21.15.89:0/578075653 shutdown_connections 2024-09-10T08:46:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.011+0000 7feaa7a6f740 1 -- 172.21.15.89:0/578075653 wait complete. 2024-09-10T08:46:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.011+0000 7feaa7a6f740 1 -- start start 2024-09-10T08:46:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.011+0000 7feaa7a6f740 1 Processor -- start 2024-09-10T08:46:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.044+0000 7feaa7a6f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559ba3b17260 0x559ba3b26060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.044+0000 7feaa7a6f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b1cc30 0x559ba3b265a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.044+0000 7feaa7a6f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ba3b2f390 0x559ba3b27540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.044+0000 7feaa7a6f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fea8000e480 con 0x559ba3b2f390 2024-09-10T08:46:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.044+0000 7feaa7a6f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fea8000f8f0 con 0x559ba3b17260 2024-09-10T08:46:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.044+0000 7feaa7a6f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fea8000eb50 con 0x559ba3b1cc30 2024-09-10T08:46:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.044+0000 7feaa7a6f740 10 client.0 _ll_register_callbacks cb 0x7fea90009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.045+0000 7fea9d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559ba3b17260 0x559ba3b26060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.045+0000 7fea9de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b1cc30 0x559ba3b265a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.045+0000 7fea9de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b1cc30 0x559ba3b265a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:40310/0 (socket says 172.21.15.89:40310) 2024-09-10T08:46:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.045+0000 7fea9de00640 1 -- 172.21.15.89:0/1285293191 learned_addr learned my addr 172.21.15.89:0/1285293191 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.046+0000 7fea9de00640 1 -- 172.21.15.89:0/1285293191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559ba3b17260 msgr2=0x559ba3b26060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.046+0000 7fea9de00640 1 --2- 172.21.15.89:0/1285293191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559ba3b17260 0x559ba3b26060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.046+0000 7fea9de00640 1 -- 172.21.15.89:0/1285293191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ba3b2f390 msgr2=0x559ba3b27540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.046+0000 7fea9de00640 1 --2- 172.21.15.89:0/1285293191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ba3b2f390 0x559ba3b27540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.046+0000 7fea9de00640 1 -- 172.21.15.89:0/1285293191 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ba3b27b50 con 0x559ba3b1cc30 2024-09-10T08:46:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.061+0000 7fea9de00640 1 --2- 172.21.15.89:0/1285293191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b1cc30 0x559ba3b265a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fea90026980 tx=0x7fea90013f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd4d9b1f8ed358af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.067+0000 7fea7f400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:46:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.067+0000 7fea7f400640 1 -- 172.21.15.89:0/1285293191 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9001c850 con 0x559ba3b1cc30 2024-09-10T08:46:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.067+0000 7fea7f400640 1 -- 172.21.15.89:0/1285293191 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fea9001c450 con 0x559ba3b1cc30 2024-09-10T08:46:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.078+0000 7feaa7a6f740 1 -- 172.21.15.89:0/1285293191 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea8c007bf0 con 0x559ba3b1cc30 2024-09-10T08:46:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.084+0000 7fea7f400640 1 -- 172.21.15.89:0/1285293191 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fea90026c10 con 0x559ba3b1cc30 2024-09-10T08:46:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.085+0000 7feaa7a6f740 10 client.26593 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.085+0000 7feaa7a6f740 1 -- 172.21.15.89:0/1285293191 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559ba3b978a0 con 0x559ba3b1cc30 2024-09-10T08:46:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.099+0000 7fea7f400640 1 -- 172.21.15.89:0/1285293191 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea8c007bf0 con 0x559ba3b1cc30 2024-09-10T08:46:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.099+0000 7fea7f400640 1 -- 172.21.15.89:0/1285293191 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fea90030900 con 0x559ba3b1cc30 2024-09-10T08:46:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.116+0000 7feaa7a6f740 10 client.26593 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.116+0000 7feaa7a6f740 10 client.26593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:46:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.116+0000 7feaa7a6f740 20 client.26593 populate_metadata read hostname 'smithi089' 2024-09-10T08:46:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.116+0000 7feaa7a6f740 1 --2- 172.21.15.89:0/1285293191 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x559ba3b981d0 0x559ba3b9a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.116+0000 7feaa7a6f740 4 client.26593 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:46:44.123 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.116+0000 7feaa7a6f740 1 -- 172.21.15.89:0/1285293191 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ba3a83b80 con 0x559ba3b981d0 2024-09-10T08:46:44.123 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.122+0000 7fea9d400640 1 --2- 172.21.15.89:0/1285293191 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x559ba3b981d0 0x559ba3b9a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.137+0000 7fea9d400640 1 --2- 172.21.15.89:0/1285293191 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x559ba3b981d0 0x559ba3b9a5f0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e651335a20da772 server_cookie=1386a90e0d6f5dba in_seq=0 out_seq=0 2024-09-10T08:46:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:44.145+0000 7fea7f400640 10 client.26593 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:44.795 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:46:44.796 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:46:45.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:44 smithi149 ceph-mon[36167]: pgmap v597: 129 pgs: 129 active+clean; 430 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 388 op/s 2024-09-10T08:46:45.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:44 smithi111 ceph-mon[35220]: pgmap v597: 129 pgs: 129 active+clean; 430 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 388 op/s 2024-09-10T08:46:45.319 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:44 smithi089 ceph-mon[30757]: pgmap v597: 129 pgs: 129 active+clean; 430 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 388 op/s 2024-09-10T08:46:47.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:46 smithi149 ceph-mon[36167]: pgmap v598: 129 pgs: 129 active+clean; 434 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 299 op/s 2024-09-10T08:46:47.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:46 smithi111 ceph-mon[35220]: pgmap v598: 129 pgs: 129 active+clean; 434 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 299 op/s 2024-09-10T08:46:47.314 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:46 smithi089 ceph-mon[30757]: pgmap v598: 129 pgs: 129 active+clean; 434 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 299 op/s 2024-09-10T08:46:48.073 INFO:tasks.workunit.client.0.smithi089.stdout: 136 490036 347 358937 401 302466 1267 2024-09-10T08:46:48.469 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:49.139 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:48 smithi089 ceph-mon[30757]: pgmap v599: 129 pgs: 129 active+clean; 459 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 505 op/s 2024-09-10T08:46:49.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:48 smithi149 ceph-mon[36167]: pgmap v599: 129 pgs: 129 active+clean; 459 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 505 op/s 2024-09-10T08:46:49.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:48 smithi111 ceph-mon[35220]: pgmap v599: 129 pgs: 129 active+clean; 459 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 505 op/s 2024-09-10T08:46:49.461 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:46:49.461 INFO:teuthology.orchestra.run.smithi089.stdout: "return_code": 0, 2024-09-10T08:46:49.461 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_tag": "8bf06df9-b26a-4aca-9618-8557aebc03ea", 2024-09-10T08:46:49.461 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": "asynchronous" 2024-09-10T08:46:49.461 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:46:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.242+0000 7f2fb5c00640 1 -- 172.21.15.89:0/2775054561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fb01005c0 msgr2=0x7f2fb010ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.242+0000 7f2fb5c00640 1 --2- 172.21.15.89:0/2775054561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fb01005c0 0x7f2fb010ea40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009900 tx=0x7f2fa002a440 comp rx=0 tx=0).stop 2024-09-10T08:46:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.242+0000 7f2fb5c00640 1 -- 172.21.15.89:0/2775054561 shutdown_connections 2024-09-10T08:46:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.242+0000 7f2fb5c00640 1 --2- 172.21.15.89:0/2775054561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fb01005c0 0x7f2fb010ea40 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.242+0000 7f2fb5c00640 1 --2- 172.21.15.89:0/2775054561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fb00ffc20 0x7f2fb0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.242+0000 7f2fb5c00640 1 --2- 172.21.15.89:0/2775054561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb0106ba0 0x7f2fb0106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.242+0000 7f2fb5c00640 1 -- 172.21.15.89:0/2775054561 >> 172.21.15.89:0/2775054561 conn(0x7f2fb00fba90 msgr2=0x7f2fb00fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.243+0000 7f2fb5c00640 1 -- 172.21.15.89:0/2775054561 shutdown_connections 2024-09-10T08:46:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.243+0000 7f2fb5c00640 1 -- 172.21.15.89:0/2775054561 wait complete. 2024-09-10T08:46:49.462 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.246+0000 7f2fb5c00640 1 Processor -- start 2024-09-10T08:46:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.246+0000 7f2fb5c00640 1 -- start start 2024-09-10T08:46:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.246+0000 7f2fb5c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb00ffc20 0x7f2fb0073950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.246+0000 7f2fb5c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fb01005c0 0x7f2fb0073e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.246+0000 7f2fb5c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fb0106ba0 0x7f2fb006cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.246+0000 7f2fb5c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb010fe80 con 0x7f2fb0106ba0 2024-09-10T08:46:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.246+0000 7f2fb5c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb010fd00 con 0x7f2fb01005c0 2024-09-10T08:46:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.246+0000 7f2fb5c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb0110000 con 0x7f2fb00ffc20 2024-09-10T08:46:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.247+0000 7f2fae000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fb01005c0 0x7f2fb0073e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.247+0000 7f2faea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb00ffc20 0x7f2fb0073950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.247+0000 7f2faea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb00ffc20 0x7f2fb0073950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:40328/0 (socket says 172.21.15.89:40328) 2024-09-10T08:46:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.247+0000 7f2faea00640 1 -- 172.21.15.89:0/824556918 learned_addr learned my addr 172.21.15.89:0/824556918 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.248+0000 7f2faf400640 1 --2- 172.21.15.89:0/824556918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fb0106ba0 0x7f2fb006cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.463 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.249+0000 7f2faea00640 1 -- 172.21.15.89:0/824556918 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fb01005c0 msgr2=0x7f2fb0073e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.249+0000 7f2faea00640 1 --2- 172.21.15.89:0/824556918 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fb01005c0 0x7f2fb0073e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.249+0000 7f2faea00640 1 -- 172.21.15.89:0/824556918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fb0106ba0 msgr2=0x7f2fb006cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.249+0000 7f2faea00640 1 --2- 172.21.15.89:0/824556918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fb0106ba0 0x7f2fb006cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.249+0000 7f2faea00640 1 -- 172.21.15.89:0/824556918 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb006d430 con 0x7f2fb00ffc20 2024-09-10T08:46:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.249+0000 7f2faf400640 1 --2- 172.21.15.89:0/824556918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fb0106ba0 0x7f2fb006cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.251+0000 7f2faea00640 1 --2- 172.21.15.89:0/824556918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb00ffc20 0x7f2fb0073950 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800ba50 tx=0x7f2fa800bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.252+0000 7f2fa7400640 1 -- 172.21.15.89:0/824556918 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8019070 con 0x7f2fb00ffc20 2024-09-10T08:46:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.252+0000 7f2fa7400640 1 -- 172.21.15.89:0/824556918 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fa800cc10 con 0x7f2fb00ffc20 2024-09-10T08:46:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.252+0000 7f2fa7400640 1 -- 172.21.15.89:0/824556918 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa80076a0 con 0x7f2fb00ffc20 2024-09-10T08:46:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.252+0000 7f2fb5c00640 1 -- 172.21.15.89:0/824556918 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb006d720 con 0x7f2fb00ffc20 2024-09-10T08:46:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.252+0000 7f2fb5c00640 1 -- 172.21.15.89:0/824556918 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb00716b0 con 0x7f2fb00ffc20 2024-09-10T08:46:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.258+0000 7f2fa7400640 1 -- 172.21.15.89:0/824556918 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2fa801c070 con 0x7f2fb00ffc20 2024-09-10T08:46:49.464 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.259+0000 7f2fb86b3740 1 Processor -- start 2024-09-10T08:46:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.260+0000 7f2fb86b3740 1 -- start start 2024-09-10T08:46:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.260+0000 7f2fb86b3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461bcac0 0x5627461bcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.260+0000 7f2fb86b3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461bd3e0 0x5627461c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.260+0000 7f2fb86b3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461c7ca0 0x5627461ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.261+0000 7f2fa7400640 1 --2- 172.21.15.89:0/824556918 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2f940a71a0 0x7f2f940a9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.261+0000 7f2fb86b3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562746123290 con 0x5627461c7ca0 2024-09-10T08:46:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.261+0000 7f2fb86b3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5627460cab40 con 0x5627461bcac0 2024-09-10T08:46:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.261+0000 7f2fb86b3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56274611f250 con 0x5627461bd3e0 2024-09-10T08:46:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.261+0000 7f2faf400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461bcac0 0x5627461bcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.261+0000 7f2faf400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461bcac0 0x5627461bcea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:38696/0 (socket says 172.21.15.89:38696) 2024-09-10T08:46:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.261+0000 7f2faf400640 1 -- 172.21.15.89:0/1211223276 learned_addr learned my addr 172.21.15.89:0/1211223276 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.262+0000 7f2fae000640 1 --2- 172.21.15.89:0/1211223276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461c7ca0 0x5627461ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.465 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.262+0000 7f2faea00640 1 --2- 172.21.15.89:0/1211223276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461bd3e0 0x5627461c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.262+0000 7f2faf400640 1 --2- 172.21.15.89:0/824556918 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2f940a71a0 0x7f2f940a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.263+0000 7f2fa7400640 1 -- 172.21.15.89:0/824556918 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f2fa80ce9b0 con 0x7f2fb00ffc20 2024-09-10T08:46:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.264+0000 7f2faea00640 1 -- 172.21.15.89:0/1211223276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461bcac0 msgr2=0x5627461bcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.264+0000 7f2faea00640 1 --2- 172.21.15.89:0/1211223276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461bcac0 0x5627461bcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.264+0000 7f2faea00640 1 -- 172.21.15.89:0/1211223276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461c7ca0 msgr2=0x5627461ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.264+0000 7f2faea00640 1 --2- 172.21.15.89:0/1211223276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461c7ca0 0x5627461ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.264+0000 7f2faea00640 1 -- 172.21.15.89:0/1211223276 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562745e946a0 con 0x5627461bd3e0 2024-09-10T08:46:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.264+0000 7f2faea00640 1 --2- 172.21.15.89:0/1211223276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461bd3e0 0x5627461c7760 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8000f80 tx=0x7f2fa809ae90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecd57c163438e56d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.265+0000 7f2f9be00640 1 -- 172.21.15.89:0/1211223276 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa80a3a50 con 0x5627461bd3e0 2024-09-10T08:46:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.265+0000 7f2f9be00640 1 -- 172.21.15.89:0/1211223276 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fa809e070 con 0x5627461bd3e0 2024-09-10T08:46:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.266+0000 7f2fb86b3740 1 -- 172.21.15.89:0/1211223276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461bd3e0 msgr2=0x5627461c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.266+0000 7f2fb86b3740 1 --2- 172.21.15.89:0/1211223276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461bd3e0 0x5627461c7760 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8000f80 tx=0x7f2fa809ae90 comp rx=0 tx=0).stop 2024-09-10T08:46:49.466 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.266+0000 7f2faf400640 1 --2- 172.21.15.89:0/824556918 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2f940a71a0 0x7f2f940a9660 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0031550 tx=0x7f2fa002a440 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.267+0000 7f2f9be00640 1 -- 172.21.15.89:0/1211223276 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa80a35c0 con 0x5627461bd3e0 2024-09-10T08:46:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.267+0000 7f2fb86b3740 1 -- 172.21.15.89:0/1211223276 shutdown_connections 2024-09-10T08:46:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.267+0000 7f2fb86b3740 1 --2- 172.21.15.89:0/1211223276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461c7ca0 0x5627461ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.267+0000 7f2fb86b3740 1 --2- 172.21.15.89:0/1211223276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461bd3e0 0x5627461c7760 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.267+0000 7f2fb86b3740 1 --2- 172.21.15.89:0/1211223276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461bcac0 0x5627461bcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.267+0000 7f2fb86b3740 1 -- 172.21.15.89:0/1211223276 >> 172.21.15.89:0/1211223276 conn(0x5627461b9050 msgr2=0x5627461b9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.267+0000 7f2fb86b3740 1 -- 172.21.15.89:0/1211223276 shutdown_connections 2024-09-10T08:46:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.267+0000 7f2fb86b3740 1 -- 172.21.15.89:0/1211223276 wait complete. 2024-09-10T08:46:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.268+0000 7f2fb86b3740 1 -- start start 2024-09-10T08:46:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.268+0000 7f2fb86b3740 1 Processor -- start 2024-09-10T08:46:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.270+0000 7f2fb86b3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461bcac0 0x5627461c7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.270+0000 7f2fb86b3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461bd3e0 0x5627461c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.270+0000 7f2fb86b3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461c7ca0 0x5627461c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.467 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.270+0000 7f2fb86b3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5627461ca9b0 con 0x5627461bcac0 2024-09-10T08:46:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.270+0000 7f2fb86b3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5627460ddfa0 con 0x5627461c7ca0 2024-09-10T08:46:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.270+0000 7f2fb86b3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5627461cab30 con 0x5627461bd3e0 2024-09-10T08:46:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.270+0000 7f2fb86b3740 10 client.0 _ll_register_callbacks cb 0x562745e31ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.271+0000 7f2fae000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461bd3e0 0x5627461c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.271+0000 7f2fae000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461bd3e0 0x5627461c47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:40344/0 (socket says 172.21.15.89:40344) 2024-09-10T08:46:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.271+0000 7f2fae000640 1 -- 172.21.15.89:0/811799985 learned_addr learned my addr 172.21.15.89:0/811799985 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.271+0000 7f2fae000640 1 -- 172.21.15.89:0/811799985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461c7ca0 msgr2=0x5627461c4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.271+0000 7f2fae000640 1 --2- 172.21.15.89:0/811799985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461c7ca0 0x5627461c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.271+0000 7f2fae000640 1 -- 172.21.15.89:0/811799985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461bcac0 msgr2=0x5627461c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.271+0000 7f2fae000640 1 --2- 172.21.15.89:0/811799985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461bcac0 0x5627461c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.271+0000 7f2fae000640 1 -- 172.21.15.89:0/811799985 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627461c5530 con 0x5627461bd3e0 2024-09-10T08:46:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.272+0000 7f2fae000640 1 --2- 172.21.15.89:0/811799985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461bd3e0 0x5627461c47f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c010240 tx=0x7f2f9c010700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7edea7e49c6fbdd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:49.468 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.272+0000 7f2f98c00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:46:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.272+0000 7f2f98c00640 1 -- 172.21.15.89:0/811799985 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c018070 con 0x5627461bd3e0 2024-09-10T08:46:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.273+0000 7f2f98c00640 1 -- 172.21.15.89:0/811799985 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f2f9c014800 con 0x5627461bd3e0 2024-09-10T08:46:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.273+0000 7f2fb86b3740 1 -- 172.21.15.89:0/811799985 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fa809e070 con 0x5627461bd3e0 2024-09-10T08:46:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.273+0000 7f2f98c00640 1 -- 172.21.15.89:0/811799985 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2f9c024050 con 0x5627461bd3e0 2024-09-10T08:46:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.273+0000 7f2fb86b3740 10 client.26602 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.274+0000 7f2fb86b3740 1 -- 172.21.15.89:0/811799985 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5627461df0f0 con 0x5627461bd3e0 2024-09-10T08:46:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.274+0000 7f2f98c00640 1 -- 172.21.15.89:0/811799985 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fa809e070 con 0x5627461bd3e0 2024-09-10T08:46:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.276+0000 7f2f98c00640 1 -- 172.21.15.89:0/811799985 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f2f9c01cba0 con 0x5627461bd3e0 2024-09-10T08:46:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.276+0000 7f2fb86b3740 10 client.26602 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.276+0000 7f2fb86b3740 10 client.26602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:46:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.276+0000 7f2fb86b3740 20 client.26602 populate_metadata read hostname 'smithi089' 2024-09-10T08:46:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.276+0000 7f2fb86b3740 1 --2- 172.21.15.89:0/811799985 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5627461e0550 0x5627461e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.469 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.276+0000 7f2fb86b3740 4 client.26602 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:49.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.276+0000 7f2fb86b3740 1 -- 172.21.15.89:0/811799985 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562746130800 con 0x5627461e0550 2024-09-10T08:46:49.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.277+0000 7f2faea00640 1 --2- 172.21.15.89:0/811799985 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5627461e0550 0x5627461e2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.280+0000 7f2faea00640 1 --2- 172.21.15.89:0/811799985 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5627461e0550 0x5627461e2a10 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaf52365dbfd30a1 server_cookie=188fe43abb6284e7 in_seq=0 out_seq=0 2024-09-10T08:46:49.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.280+0000 7f2f98c00640 10 client.26602 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:49.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.282+0000 7f2f98c00640 1 -- 172.21.15.89:0/811799985 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562746130800 con 0x5627461e0550 2024-09-10T08:46:49.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.282+0000 7f2f98c00640 10 client.26602 handle_command_reply: tid=0 2024-09-10T08:46:49.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.282+0000 7f2fb86b3740 1 client.26602 shutdown 2024-09-10T08:46:49.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.284+0000 7f2fb86b3740 1 -- 172.21.15.89:0/811799985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461bd3e0 msgr2=0x5627461c47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:49.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.284+0000 7f2fb86b3740 1 --2- 172.21.15.89:0/811799985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461bd3e0 0x5627461c47f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c010240 tx=0x7f2f9c010700 comp rx=0 tx=0).stop 2024-09-10T08:46:49.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.284+0000 7f2fb86b3740 1 -- 172.21.15.89:0/811799985 shutdown_connections 2024-09-10T08:46:49.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.284+0000 7f2fb86b3740 1 --2- 172.21.15.89:0/811799985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461c7ca0 0x5627461c4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.284+0000 7f2fb86b3740 1 --2- 172.21.15.89:0/811799985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461bd3e0 0x5627461c47f0 secure :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c010240 tx=0x7f2f9c010700 comp rx=0 tx=0).stop 2024-09-10T08:46:49.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.284+0000 7f2fb86b3740 1 --2- 172.21.15.89:0/811799985 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5627461e0550 0x5627461e2a10 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.284+0000 7f2fb86b3740 1 --2- 172.21.15.89:0/811799985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461bcac0 0x5627461c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.284+0000 7f2fb86b3740 1 -- 172.21.15.89:0/811799985 >> 172.21.15.89:0/811799985 conn(0x5627461b9050 msgr2=0x5627461b81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:49.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.284+0000 7f2f98c00640 0 client.26602 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:49.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.286+0000 7f2fb86b3740 1 -- 172.21.15.89:0/811799985 shutdown_connections 2024-09-10T08:46:49.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.286+0000 7f2fb86b3740 1 -- 172.21.15.89:0/811799985 wait complete. 2024-09-10T08:46:49.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.286+0000 7f2fb86b3740 20 client.26602 trim_cache size 0 max 16384 2024-09-10T08:46:49.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.325+0000 7f2fb86b3740 1 Processor -- start 2024-09-10T08:46:49.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.325+0000 7f2fb86b3740 1 -- start start 2024-09-10T08:46:49.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.326+0000 7f2fb86b3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461bcac0 0x5627461bfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.326+0000 7f2fb86b3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461c7ca0 0x5627461e82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.327+0000 7f2fb86b3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461c38e0 0x5627461e8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.327+0000 7f2fb86b3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb01a0e10 con 0x5627461bcac0 2024-09-10T08:46:49.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.327+0000 7f2fb86b3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb019e720 con 0x5627461c38e0 2024-09-10T08:46:49.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.327+0000 7f2fb86b3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb0100080 con 0x5627461c7ca0 2024-09-10T08:46:49.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.327+0000 7f2faea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461bcac0 0x5627461bfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.327+0000 7f2fae000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461c7ca0 0x5627461e82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.330+0000 7f2faea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461bcac0 0x5627461bfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:48656/0 (socket says 172.21.15.89:48656) 2024-09-10T08:46:49.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.330+0000 7f2faea00640 1 -- 172.21.15.89:0/3629711670 learned_addr learned my addr 172.21.15.89:0/3629711670 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:49.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.330+0000 7f2fae000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461c7ca0 0x5627461e82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:40346/0 (socket says 172.21.15.89:40346) 2024-09-10T08:46:49.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.327+0000 7f2faf400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461c38e0 0x5627461e8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.330+0000 7f2faea00640 1 -- 172.21.15.89:0/3629711670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461c7ca0 msgr2=0x5627461e82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:49.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.330+0000 7f2faea00640 1 --2- 172.21.15.89:0/3629711670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461c7ca0 0x5627461e82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.330+0000 7f2faea00640 1 -- 172.21.15.89:0/3629711670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461c38e0 msgr2=0x5627461e8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:49.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.330+0000 7f2faea00640 1 --2- 172.21.15.89:0/3629711670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461c38e0 0x5627461e8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.330+0000 7f2faea00640 1 -- 172.21.15.89:0/3629711670 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627461b9c00 con 0x5627461bcac0 2024-09-10T08:46:49.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.331+0000 7f2faea00640 1 --2- 172.21.15.89:0/3629711670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461bcac0 0x5627461bfe10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8000f80 tx=0x7f2fa809ad90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a46bcd8cb3bb0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:49.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.332+0000 7f2f9b400640 1 -- 172.21.15.89:0/3629711670 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa80ab070 con 0x5627461bcac0 2024-09-10T08:46:49.472 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.333+0000 7f2f9b400640 1 -- 172.21.15.89:0/3629711670 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fa809e040 con 0x5627461bcac0 2024-09-10T08:46:49.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.333+0000 7f2f9b400640 1 -- 172.21.15.89:0/3629711670 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa80a7070 con 0x5627461bcac0 2024-09-10T08:46:49.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.337+0000 7f2fb86b3740 1 -- 172.21.15.89:0/3629711670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461bcac0 msgr2=0x5627461bfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:49.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.337+0000 7f2fb86b3740 1 --2- 172.21.15.89:0/3629711670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461bcac0 0x5627461bfe10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8000f80 tx=0x7f2fa809ad90 comp rx=0 tx=0).stop 2024-09-10T08:46:49.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.337+0000 7f2fb86b3740 1 -- 172.21.15.89:0/3629711670 shutdown_connections 2024-09-10T08:46:49.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.337+0000 7f2fb86b3740 1 --2- 172.21.15.89:0/3629711670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461c38e0 0x5627461e8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.337+0000 7f2fb86b3740 1 --2- 172.21.15.89:0/3629711670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461c7ca0 0x5627461e82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.337+0000 7f2fb86b3740 1 --2- 172.21.15.89:0/3629711670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461bcac0 0x5627461bfe10 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.337+0000 7f2fb86b3740 1 -- 172.21.15.89:0/3629711670 >> 172.21.15.89:0/3629711670 conn(0x5627461b9050 msgr2=0x5627461bf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:49.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.337+0000 7f2fb86b3740 1 -- 172.21.15.89:0/3629711670 shutdown_connections 2024-09-10T08:46:49.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.337+0000 7f2fb86b3740 1 -- 172.21.15.89:0/3629711670 wait complete. 2024-09-10T08:46:49.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.338+0000 7f2fb86b3740 1 -- start start 2024-09-10T08:46:49.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.338+0000 7f2fb86b3740 1 Processor -- start 2024-09-10T08:46:49.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.339+0000 7f2fb86b3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461c38e0 0x5627461d4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.339+0000 7f2fb86b3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461c7ca0 0x5627461d4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.339+0000 7f2fb86b3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461d4dc0 0x56274624a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.339+0000 7f2fb86b3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb019cbe0 con 0x5627461d4dc0 2024-09-10T08:46:49.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.339+0000 7f2fb86b3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb019e050 con 0x5627461c7ca0 2024-09-10T08:46:49.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.339+0000 7f2fb86b3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb019d2b0 con 0x5627461c38e0 2024-09-10T08:46:49.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.339+0000 7f2fb86b3740 10 client.0 _ll_register_callbacks cb 0x7f2f9c0093b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:49.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.339+0000 7f2faea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461c38e0 0x5627461d4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.340+0000 7f2faea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461c38e0 0x5627461d4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:40360/0 (socket says 172.21.15.89:40360) 2024-09-10T08:46:49.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.340+0000 7f2faea00640 1 -- 172.21.15.89:0/4000763832 learned_addr learned my addr 172.21.15.89:0/4000763832 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:49.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.340+0000 7f2fae000640 1 --2- 172.21.15.89:0/4000763832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461c7ca0 0x5627461d4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.340+0000 7f2faf400640 1 --2- 172.21.15.89:0/4000763832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461d4dc0 0x56274624a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.341+0000 7f2fae000640 1 -- 172.21.15.89:0/4000763832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461c38e0 msgr2=0x5627461d4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:49.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.341+0000 7f2fae000640 1 --2- 172.21.15.89:0/4000763832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461c38e0 0x5627461d4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.341+0000 7f2fae000640 1 -- 172.21.15.89:0/4000763832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461d4dc0 msgr2=0x56274624a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:49.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.341+0000 7f2fae000640 1 --2- 172.21.15.89:0/4000763832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461d4dc0 0x56274624a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.341+0000 7f2fae000640 1 -- 172.21.15.89:0/4000763832 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56274624a7f0 con 0x5627461c7ca0 2024-09-10T08:46:49.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.341+0000 7f2faf400640 1 --2- 172.21.15.89:0/4000763832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461d4dc0 0x56274624a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:49.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.343+0000 7f2fae000640 1 --2- 172.21.15.89:0/4000763832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461c7ca0 0x5627461d4880 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x5627461c5da0 tx=0x7f2f9c01cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f97954ab337f155 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:49.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.343+0000 7f2f98c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:46:49.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.345+0000 7f2f98c00640 1 -- 172.21.15.89:0/4000763832 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c02c020 con 0x5627461c7ca0 2024-09-10T08:46:49.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.345+0000 7f2f98c00640 1 -- 172.21.15.89:0/4000763832 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f2f9c004220 con 0x5627461c7ca0 2024-09-10T08:46:49.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.345+0000 7f2fb86b3740 1 -- 172.21.15.89:0/4000763832 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fa809e040 con 0x5627461c7ca0 2024-09-10T08:46:49.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.346+0000 7f2f98c00640 1 -- 172.21.15.89:0/4000763832 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2f9c018040 con 0x5627461c7ca0 2024-09-10T08:46:49.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.346+0000 7f2fb86b3740 10 client.26669 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:49.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.346+0000 7f2fb86b3740 1 -- 172.21.15.89:0/4000763832 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56274624b9d0 con 0x5627461c7ca0 2024-09-10T08:46:49.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.346+0000 7f2f98c00640 1 -- 172.21.15.89:0/4000763832 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fa809e040 con 0x5627461c7ca0 2024-09-10T08:46:49.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.347+0000 7f2f98c00640 1 -- 172.21.15.89:0/4000763832 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f2f9c02f020 con 0x5627461c7ca0 2024-09-10T08:46:49.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.347+0000 7f2fb86b3740 10 client.26669 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:49.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.347+0000 7f2fb86b3740 10 client.26669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:46:49.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.347+0000 7f2fb86b3740 20 client.26669 populate_metadata read hostname 'smithi089' 2024-09-10T08:46:49.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.347+0000 7f2fb86b3740 1 --2- 172.21.15.89:0/4000763832 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56274624c0a0 0x56274624e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.347+0000 7f2fb86b3740 4 client.26669 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:46:49.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.347+0000 7f2fb86b3740 1 -- 172.21.15.89:0/4000763832 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56274612ad10 con 0x56274624c0a0 2024-09-10T08:46:49.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.348+0000 7f2faea00640 1 --2- 172.21.15.89:0/4000763832 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56274624c0a0 0x56274624e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.350+0000 7f2faea00640 1 --2- 172.21.15.89:0/4000763832 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56274624c0a0 0x56274624e4c0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10ce672e7a76f1d5 server_cookie=dfb1ea3bb44ddf4a in_seq=0 out_seq=0 2024-09-10T08:46:49.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.350+0000 7f2f98c00640 10 client.26669 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:49.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.355+0000 7f2f98c00640 1 -- 172.21.15.89:0/4000763832 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56274612ad10 con 0x56274624c0a0 2024-09-10T08:46:49.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.355+0000 7f2f98c00640 10 client.26669 handle_command_reply: tid=0 2024-09-10T08:46:49.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.355+0000 7f2fb86b3740 1 client.26669 shutdown 2024-09-10T08:46:49.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.355+0000 7f2fb86b3740 1 -- 172.21.15.89:0/4000763832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461c7ca0 msgr2=0x5627461d4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:49.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.355+0000 7f2fb86b3740 1 --2- 172.21.15.89:0/4000763832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461c7ca0 0x5627461d4880 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x5627461c5da0 tx=0x7f2f9c01cbd0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.355+0000 7f2fb86b3740 1 -- 172.21.15.89:0/4000763832 shutdown_connections 2024-09-10T08:46:49.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.355+0000 7f2fb86b3740 1 --2- 172.21.15.89:0/4000763832 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56274624c0a0 0x56274624e4c0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.355+0000 7f2fb86b3740 1 --2- 172.21.15.89:0/4000763832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5627461d4dc0 0x56274624a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.355+0000 7f2fb86b3740 1 --2- 172.21.15.89:0/4000763832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5627461c7ca0 0x5627461d4880 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.355+0000 7f2f98c00640 0 client.26669 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:49.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.355+0000 7f2fb86b3740 1 --2- 172.21.15.89:0/4000763832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627461c38e0 0x5627461d4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.355+0000 7f2fb86b3740 1 -- 172.21.15.89:0/4000763832 >> 172.21.15.89:0/4000763832 conn(0x5627461b9050 msgr2=0x5627461bed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:49.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.356+0000 7f2fb86b3740 1 -- 172.21.15.89:0/4000763832 shutdown_connections 2024-09-10T08:46:49.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.358+0000 7f2fb86b3740 1 -- 172.21.15.89:0/4000763832 wait complete. 2024-09-10T08:46:49.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.358+0000 7f2fb86b3740 20 client.26669 trim_cache size 0 max 16384 2024-09-10T08:46:49.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.359+0000 7f2fb5c00640 1 -- 172.21.15.89:0/824556918 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2f940a71a0 msgr2=0x7f2f940a9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:49.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.359+0000 7f2fb5c00640 1 --2- 172.21.15.89:0/824556918 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2f940a71a0 0x7f2f940a9660 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0031550 tx=0x7f2fa002a440 comp rx=0 tx=0).stop 2024-09-10T08:46:49.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.359+0000 7f2fb5c00640 1 -- 172.21.15.89:0/824556918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb00ffc20 msgr2=0x7f2fb0073950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:49.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.359+0000 7f2fb5c00640 1 --2- 172.21.15.89:0/824556918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb00ffc20 0x7f2fb0073950 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800ba50 tx=0x7f2fa800bf10 comp rx=0 tx=0).stop 2024-09-10T08:46:49.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.359+0000 7f2fb5c00640 1 -- 172.21.15.89:0/824556918 shutdown_connections 2024-09-10T08:46:49.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.359+0000 7f2fb5c00640 1 --2- 172.21.15.89:0/824556918 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f2f940a71a0 0x7f2f940a9660 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.359+0000 7f2fb5c00640 1 --2- 172.21.15.89:0/824556918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fb0106ba0 0x7f2fb006cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.359+0000 7f2fb5c00640 1 --2- 172.21.15.89:0/824556918 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fb01005c0 0x7f2fb0073e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.359+0000 7f2fb5c00640 1 --2- 172.21.15.89:0/824556918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb00ffc20 0x7f2fb0073950 secure :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800ba50 tx=0x7f2fa800bf10 comp rx=0 tx=0).stop 2024-09-10T08:46:49.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.359+0000 7f2fb5c00640 1 -- 172.21.15.89:0/824556918 >> 172.21.15.89:0/824556918 conn(0x7f2fb00fba90 msgr2=0x7f2fb00fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:49.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.360+0000 7f2fb5c00640 1 -- 172.21.15.89:0/824556918 shutdown_connections 2024-09-10T08:46:49.479 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:49.360+0000 7f2fb5c00640 1 -- 172.21.15.89:0/824556918 wait complete. 2024-09-10T08:46:50.744 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:46:51.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:50 smithi149 ceph-mon[36167]: scrub queued for path: / 2024-09-10T08:46:51.349 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:50 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:51.349 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:50 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:46:51.349 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:50 smithi149 ceph-mon[36167]: pgmap v600: 129 pgs: 129 active+clean; 459 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 442 op/s 2024-09-10T08:46:51.350 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:50 smithi111 ceph-mon[35220]: scrub queued for path: / 2024-09-10T08:46:51.350 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:50 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:51.350 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:50 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:46:51.350 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:50 smithi111 ceph-mon[35220]: pgmap v600: 129 pgs: 129 active+clean; 459 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 442 op/s 2024-09-10T08:46:51.350 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:50 smithi089 ceph-mon[30757]: scrub queued for path: / 2024-09-10T08:46:51.350 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:50 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:51.350 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:50 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:46:51.350 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:50 smithi089 ceph-mon[30757]: pgmap v600: 129 pgs: 129 active+clean; 459 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 442 op/s 2024-09-10T08:46:51.495 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:52.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.268+0000 7f5b76200640 1 -- 172.21.15.89:0/3775986141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b70102b90 msgr2=0x7f5b70102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:52.269 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.268+0000 7f5b76200640 1 --2- 172.21.15.89:0/3775986141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b70102b90 0x7f5b70102f70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009900 tx=0x7f5b6002a4a0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.269+0000 7f5b76200640 1 -- 172.21.15.89:0/3775986141 shutdown_connections 2024-09-10T08:46:52.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.269+0000 7f5b76200640 1 --2- 172.21.15.89:0/3775986141 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b7010bfc0 0x7f5b7010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.269+0000 7f5b76200640 1 --2- 172.21.15.89:0/3775986141 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b701034b0 0x7f5b7010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.269+0000 7f5b76200640 1 --2- 172.21.15.89:0/3775986141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b70102b90 0x7f5b70102f70 secure :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009900 tx=0x7f5b6002a4a0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.269+0000 7f5b76200640 1 -- 172.21.15.89:0/3775986141 >> 172.21.15.89:0/3775986141 conn(0x7f5b700fc610 msgr2=0x7f5b700fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:52.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.269+0000 7f5b76200640 1 -- 172.21.15.89:0/3775986141 shutdown_connections 2024-09-10T08:46:52.270 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.269+0000 7f5b76200640 1 -- 172.21.15.89:0/3775986141 wait complete. 2024-09-10T08:46:52.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.271+0000 7f5b76200640 1 Processor -- start 2024-09-10T08:46:52.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.271+0000 7f5b76200640 1 -- start start 2024-09-10T08:46:52.271 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.272+0000 7f5b76200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b701034b0 0x7f5b7019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:52.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.272+0000 7f5b76200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b7010bfc0 0x7f5b7019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:52.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.273+0000 7f5b6fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b7010bfc0 0x7f5b7019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:52.272 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.273+0000 7f5b6fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b7010bfc0 0x7f5b7019ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:40368/0 (socket says 172.21.15.89:40368) 2024-09-10T08:46:52.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.273+0000 7f5b6fe00640 1 -- 172.21.15.89:0/826884098 learned_addr learned my addr 172.21.15.89:0/826884098 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:52.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.273+0000 7f5b76200640 1 --2- 172.21.15.89:0/826884098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b7019efa0 0x7f5b7019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:52.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.273+0000 7f5b76200640 1 -- 172.21.15.89:0/826884098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5b70111e10 con 0x7f5b701034b0 2024-09-10T08:46:52.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.273+0000 7f5b76200640 1 -- 172.21.15.89:0/826884098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5b70111c90 con 0x7f5b7019efa0 2024-09-10T08:46:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.274+0000 7f5b76200640 1 -- 172.21.15.89:0/826884098 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5b70111f90 con 0x7f5b7010bfc0 2024-09-10T08:46:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.274+0000 7f5b75800640 1 --2- 172.21.15.89:0/826884098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b7019efa0 0x7f5b7019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.274+0000 7f5b74e00640 1 --2- 172.21.15.89:0/826884098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b701034b0 0x7f5b7019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.274+0000 7f5b6fe00640 1 -- 172.21.15.89:0/826884098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b7019efa0 msgr2=0x7f5b7019f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:52.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.274+0000 7f5b6fe00640 1 --2- 172.21.15.89:0/826884098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b7019efa0 0x7f5b7019f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.275+0000 7f5b6fe00640 1 -- 172.21.15.89:0/826884098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b701034b0 msgr2=0x7f5b7019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.275+0000 7f5b6fe00640 1 --2- 172.21.15.89:0/826884098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b701034b0 0x7f5b7019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.275+0000 7f5b6fe00640 1 -- 172.21.15.89:0/826884098 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7019fa60 con 0x7f5b7010bfc0 2024-09-10T08:46:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.275+0000 7f5b75800640 1 --2- 172.21.15.89:0/826884098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b7019efa0 0x7f5b7019f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:46:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.275+0000 7f5b6fe00640 1 --2- 172.21.15.89:0/826884098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b7010bfc0 0x7f5b7019ac10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00d950 tx=0x7f5b5c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.275+0000 7f5b6d600640 1 -- 172.21.15.89:0/826884098 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c017030 con 0x7f5b7010bfc0 2024-09-10T08:46:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.275+0000 7f5b6d600640 1 -- 172.21.15.89:0/826884098 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b5c004020 con 0x7f5b7010bfc0 2024-09-10T08:46:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.276+0000 7f5b6d600640 1 -- 172.21.15.89:0/826884098 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c003c90 con 0x7f5b7010bfc0 2024-09-10T08:46:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.276+0000 7f5b74e00640 1 --2- 172.21.15.89:0/826884098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b701034b0 0x7f5b7019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.276+0000 7f5b76200640 1 -- 172.21.15.89:0/826884098 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7019fd50 con 0x7f5b7010bfc0 2024-09-10T08:46:52.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.276+0000 7f5b76200640 1 -- 172.21.15.89:0/826884098 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b701a74e0 con 0x7f5b7010bfc0 2024-09-10T08:46:52.287 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.287+0000 7f5b7a29d740 1 Processor -- start 2024-09-10T08:46:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.288+0000 7f5b7a29d740 1 -- start start 2024-09-10T08:46:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.289+0000 7f5b7a29d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818034c70 0x55d818035050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.289+0000 7f5b7a29d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818035590 0x55d818048200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.289+0000 7f5b7a29d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818048740 0x55d81804ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.289+0000 7f5b7a29d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d817f9b3e0 con 0x55d818035590 2024-09-10T08:46:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.289+0000 7f5b7a29d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d817f42d80 con 0x55d818034c70 2024-09-10T08:46:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.289+0000 7f5b7a29d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d817f973a0 con 0x55d818048740 2024-09-10T08:46:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.292+0000 7f5b75800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818034c70 0x55d818035050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:52.292 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.292+0000 7f5b75800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818034c70 0x55d818035050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:38766/0 (socket says 172.21.15.89:38766) 2024-09-10T08:46:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.292+0000 7f5b75800640 1 -- 172.21.15.89:0/4066079600 learned_addr learned my addr 172.21.15.89:0/4066079600 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.293+0000 7f5b75800640 1 -- 172.21.15.89:0/4066079600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818048740 msgr2=0x55d81804ab30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:46:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.293+0000 7f5b74e00640 1 --2- 172.21.15.89:0/4066079600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818035590 0x55d818048200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.293+0000 7f5b75800640 1 --2- 172.21.15.89:0/4066079600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818048740 0x55d81804ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.293+0000 7f5b75800640 1 -- 172.21.15.89:0/4066079600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818035590 msgr2=0x55d818048200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:52.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.293+0000 7f5b75800640 1 --2- 172.21.15.89:0/4066079600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818035590 0x55d818048200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.294+0000 7f5b6d600640 1 -- 172.21.15.89:0/826884098 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5b5c01a070 con 0x7f5b7010bfc0 2024-09-10T08:46:52.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.298+0000 7f5b6d600640 1 --2- 172.21.15.89:0/826884098 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5b540a2c00 0x7f5b540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:52.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.299+0000 7f5b6d600640 1 -- 172.21.15.89:0/826884098 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f5b5c0ceee0 con 0x7f5b7010bfc0 2024-09-10T08:46:52.299 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.299+0000 7f5b75800640 1 -- 172.21.15.89:0/4066079600 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d817f61ae0 con 0x55d818034c70 2024-09-10T08:46:52.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.299+0000 7f5b74e00640 1 --2- 172.21.15.89:0/4066079600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818035590 0x55d818048200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:52.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.300+0000 7f5b74e00640 1 --2- 172.21.15.89:0/826884098 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5b540a2c00 0x7f5b540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:52.300 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.300+0000 7f5b75800640 1 --2- 172.21.15.89:0/4066079600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818034c70 0x55d818035050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5b68011790 tx=0x7f5b68011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a163ebbcc8a6de2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:52.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.301+0000 7f5b66000640 1 -- 172.21.15.89:0/4066079600 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68003c30 con 0x55d818034c70 2024-09-10T08:46:52.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.301+0000 7f5b66000640 1 -- 172.21.15.89:0/4066079600 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b6800fe80 con 0x55d818034c70 2024-09-10T08:46:52.301 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.301+0000 7f5b66000640 1 -- 172.21.15.89:0/4066079600 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68003c30 con 0x55d818034c70 2024-09-10T08:46:52.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.302+0000 7f5b7a29d740 1 -- 172.21.15.89:0/4066079600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818034c70 msgr2=0x55d818035050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:52.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.302+0000 7f5b7a29d740 1 --2- 172.21.15.89:0/4066079600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818034c70 0x55d818035050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5b68011790 tx=0x7f5b68011c50 comp rx=0 tx=0).stop 2024-09-10T08:46:52.302 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.302+0000 7f5b7a29d740 1 -- 172.21.15.89:0/4066079600 shutdown_connections 2024-09-10T08:46:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.302+0000 7f5b7a29d740 1 --2- 172.21.15.89:0/4066079600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818048740 0x55d81804ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.302+0000 7f5b7a29d740 1 --2- 172.21.15.89:0/4066079600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818035590 0x55d818048200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.302+0000 7f5b7a29d740 1 --2- 172.21.15.89:0/4066079600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818034c70 0x55d818035050 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.302+0000 7f5b7a29d740 1 -- 172.21.15.89:0/4066079600 >> 172.21.15.89:0/4066079600 conn(0x55d8180312e0 msgr2=0x55d8180316c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:52.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.303+0000 7f5b7a29d740 1 -- 172.21.15.89:0/4066079600 shutdown_connections 2024-09-10T08:46:52.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.303+0000 7f5b7a29d740 1 -- 172.21.15.89:0/4066079600 wait complete. 2024-09-10T08:46:52.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.303+0000 7f5b74e00640 1 --2- 172.21.15.89:0/826884098 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5b540a2c00 0x7f5b540a50c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5b60006010 tx=0x7f5b60009590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:52.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.304+0000 7f5b7a29d740 1 -- start start 2024-09-10T08:46:52.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.304+0000 7f5b7a29d740 1 Processor -- start 2024-09-10T08:46:52.305 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.305+0000 7f5b7a29d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818034c70 0x55d818042350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.305+0000 7f5b7a29d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818035590 0x55d818048060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.305+0000 7f5b7a29d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818048740 0x55d818042890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.305+0000 7f5b74e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818034c70 0x55d818042350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.306+0000 7f5b74e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818034c70 0x55d818042350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:40376/0 (socket says 172.21.15.89:40376) 2024-09-10T08:46:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.306+0000 7f5b74e00640 1 -- 172.21.15.89:0/1396245163 learned_addr learned my addr 172.21.15.89:0/1396245163 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:52.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.306+0000 7f5b7a29d740 1 -- 172.21.15.89:0/1396245163 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d818037c70 con 0x55d818035590 2024-09-10T08:46:52.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.306+0000 7f5b7a29d740 1 -- 172.21.15.89:0/1396245163 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d817f560f0 con 0x55d818048740 2024-09-10T08:46:52.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.306+0000 7f5b7a29d740 1 -- 172.21.15.89:0/1396245163 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d818037df0 con 0x55d818034c70 2024-09-10T08:46:52.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.306+0000 7f5b7a29d740 10 client.0 _ll_register_callbacks cb 0x55d817f479e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:52.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.306+0000 7f5b6fe00640 1 --2- 172.21.15.89:0/1396245163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818035590 0x55d818048060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:52.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.306+0000 7f5b74e00640 1 -- 172.21.15.89:0/1396245163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818048740 msgr2=0x55d818042890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:46:52.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.306+0000 7f5b74e00640 1 --2- 172.21.15.89:0/1396245163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818048740 0x55d818042890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.306+0000 7f5b74e00640 1 -- 172.21.15.89:0/1396245163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818035590 msgr2=0x55d818048060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:52.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.306+0000 7f5b74e00640 1 --2- 172.21.15.89:0/1396245163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818035590 0x55d818048060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.306+0000 7f5b74e00640 1 -- 172.21.15.89:0/1396245163 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d817d0c3c0 con 0x55d818034c70 2024-09-10T08:46:52.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.306+0000 7f5b6fe00640 1 --2- 172.21.15.89:0/1396245163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818035590 0x55d818048060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:52.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.307+0000 7f5b74e00640 1 --2- 172.21.15.89:0/1396245163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818034c70 0x55d818042350 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5b60033000 tx=0x7f5b6002a4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=315954c6fa350c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:52.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.307+0000 7f5b5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:46:52.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.308+0000 7f5b5aa00640 1 -- 172.21.15.89:0/1396245163 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b600048c0 con 0x55d818034c70 2024-09-10T08:46:52.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.308+0000 7f5b5aa00640 1 -- 172.21.15.89:0/1396245163 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f5b60004a60 con 0x55d818034c70 2024-09-10T08:46:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.308+0000 7f5b7a29d740 1 -- 172.21.15.89:0/1396245163 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d817fa8950 con 0x55d818034c70 2024-09-10T08:46:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.309+0000 7f5b5aa00640 1 -- 172.21.15.89:0/1396245163 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5b60004c60 con 0x55d818034c70 2024-09-10T08:46:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.309+0000 7f5b7a29d740 10 client.26617 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:52.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.309+0000 7f5b7a29d740 1 -- 172.21.15.89:0/1396245163 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d81805f8d0 con 0x55d818034c70 2024-09-10T08:46:52.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.310+0000 7f5b5aa00640 1 -- 172.21.15.89:0/1396245163 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d817fa8950 con 0x55d818034c70 2024-09-10T08:46:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.310+0000 7f5b5aa00640 1 -- 172.21.15.89:0/1396245163 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f5b600315a0 con 0x55d818034c70 2024-09-10T08:46:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.311+0000 7f5b7a29d740 10 client.26617 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.311+0000 7f5b7a29d740 10 client.26617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:46:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.311+0000 7f5b7a29d740 20 client.26617 populate_metadata read hostname 'smithi089' 2024-09-10T08:46:52.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.311+0000 7f5b7a29d740 1 --2- 172.21.15.89:0/1396245163 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d818060e30 0x55d8180632b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:52.313 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.311+0000 7f5b7a29d740 4 client.26617 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:52.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.311+0000 7f5b7a29d740 1 -- 172.21.15.89:0/1396245163 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d817fa2e60 con 0x55d818060e30 2024-09-10T08:46:52.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.312+0000 7f5b6fe00640 1 --2- 172.21.15.89:0/1396245163 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d818060e30 0x55d8180632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:52.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.313+0000 7f5b6fe00640 1 --2- 172.21.15.89:0/1396245163 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d818060e30 0x55d8180632b0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=272d68858f9d3293 server_cookie=a4b0ab3502b05271 in_seq=0 out_seq=0 2024-09-10T08:46:52.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.313+0000 7f5b5aa00640 10 client.26617 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:52.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.314+0000 7f5b5aa00640 1 -- 172.21.15.89:0/1396245163 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d817fa2e60 con 0x55d818060e30 2024-09-10T08:46:52.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.314+0000 7f5b5aa00640 10 client.26617 handle_command_reply: tid=0 2024-09-10T08:46:52.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.314+0000 7f5b7a29d740 1 client.26617 shutdown 2024-09-10T08:46:52.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.315+0000 7f5b7a29d740 1 -- 172.21.15.89:0/1396245163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818034c70 msgr2=0x55d818042350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:52.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.315+0000 7f5b7a29d740 1 --2- 172.21.15.89:0/1396245163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818034c70 0x55d818042350 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5b60033000 tx=0x7f5b6002a4a0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.315+0000 7f5b7a29d740 1 -- 172.21.15.89:0/1396245163 shutdown_connections 2024-09-10T08:46:52.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.316+0000 7f5b7a29d740 1 --2- 172.21.15.89:0/1396245163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818048740 0x55d818042890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.316+0000 7f5b7a29d740 1 --2- 172.21.15.89:0/1396245163 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d818060e30 0x55d8180632b0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.316+0000 7f5b7a29d740 1 --2- 172.21.15.89:0/1396245163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818035590 0x55d818048060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.316+0000 7f5b7a29d740 1 --2- 172.21.15.89:0/1396245163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818034c70 0x55d818042350 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.316+0000 7f5b7a29d740 1 -- 172.21.15.89:0/1396245163 >> 172.21.15.89:0/1396245163 conn(0x55d8180312e0 msgr2=0x55d818036f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:52.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.316+0000 7f5b5aa00640 0 client.26617 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:52.318 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.316+0000 7f5b7a29d740 1 -- 172.21.15.89:0/1396245163 shutdown_connections 2024-09-10T08:46:52.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.316+0000 7f5b7a29d740 1 -- 172.21.15.89:0/1396245163 wait complete. 2024-09-10T08:46:52.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.316+0000 7f5b7a29d740 20 client.26617 trim_cache size 0 max 16384 2024-09-10T08:46:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.355+0000 7f5b7a29d740 1 Processor -- start 2024-09-10T08:46:52.355 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.355+0000 7f5b7a29d740 1 -- start start 2024-09-10T08:46:52.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.356+0000 7f5b7a29d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818034c70 0x55d818047ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:52.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.356+0000 7f5b7a29d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818035590 0x55d818068b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:52.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.356+0000 7f5b7a29d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818048740 0x55d8180690c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:52.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.356+0000 7f5b7a29d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701af890 con 0x55d818035590 2024-09-10T08:46:52.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.356+0000 7f5b7a29d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701ad160 con 0x55d818034c70 2024-09-10T08:46:52.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.356+0000 7f5b7a29d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701ab010 con 0x55d818048740 2024-09-10T08:46:52.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.356+0000 7f5b75800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818048740 0x55d8180690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:52.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.357+0000 7f5b75800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818048740 0x55d8180690c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:40378/0 (socket says 172.21.15.89:40378) 2024-09-10T08:46:52.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.357+0000 7f5b75800640 1 -- 172.21.15.89:0/1494258779 learned_addr learned my addr 172.21.15.89:0/1494258779 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:52.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.357+0000 7f5b74e00640 1 --2- 172.21.15.89:0/1494258779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818034c70 0x55d818047ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:52.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.357+0000 7f5b75800640 1 -- 172.21.15.89:0/1494258779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818034c70 msgr2=0x55d818047ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:52.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.357+0000 7f5b75800640 1 --2- 172.21.15.89:0/1494258779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818034c70 0x55d818047ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.357+0000 7f5b75800640 1 -- 172.21.15.89:0/1494258779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818035590 msgr2=0x55d818068b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:52.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.357+0000 7f5b75800640 1 --2- 172.21.15.89:0/1494258779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818035590 0x55d818068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.357+0000 7f5b75800640 1 -- 172.21.15.89:0/1494258779 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d818034540 con 0x55d818048740 2024-09-10T08:46:52.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.357+0000 7f5b75800640 1 --2- 172.21.15.89:0/1494258779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818048740 0x55d8180690c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009fd0 tx=0x7f5b68009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72e0013b55638bc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:52.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.358+0000 7f5b65600640 1 -- 172.21.15.89:0/1494258779 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68029070 con 0x55d818048740 2024-09-10T08:46:52.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.358+0000 7f5b65600640 1 -- 172.21.15.89:0/1494258779 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b6800fa60 con 0x55d818048740 2024-09-10T08:46:52.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.358+0000 7f5b65600640 1 -- 172.21.15.89:0/1494258779 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b680052a0 con 0x55d818048740 2024-09-10T08:46:52.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.359+0000 7f5b7a29d740 1 -- 172.21.15.89:0/1494258779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818048740 msgr2=0x55d8180690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:52.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.359+0000 7f5b7a29d740 1 --2- 172.21.15.89:0/1494258779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818048740 0x55d8180690c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009fd0 tx=0x7f5b68009500 comp rx=0 tx=0).stop 2024-09-10T08:46:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.359+0000 7f5b7a29d740 1 -- 172.21.15.89:0/1494258779 shutdown_connections 2024-09-10T08:46:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.359+0000 7f5b7a29d740 1 --2- 172.21.15.89:0/1494258779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818048740 0x55d8180690c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.359+0000 7f5b7a29d740 1 --2- 172.21.15.89:0/1494258779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818035590 0x55d818068b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.359+0000 7f5b7a29d740 1 --2- 172.21.15.89:0/1494258779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818034c70 0x55d818047ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.359+0000 7f5b7a29d740 1 -- 172.21.15.89:0/1494258779 >> 172.21.15.89:0/1494258779 conn(0x55d8180312e0 msgr2=0x55d8180303a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.359+0000 7f5b7a29d740 1 -- 172.21.15.89:0/1494258779 shutdown_connections 2024-09-10T08:46:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.359+0000 7f5b7a29d740 1 -- 172.21.15.89:0/1494258779 wait complete. 2024-09-10T08:46:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.360+0000 7f5b7a29d740 1 -- start start 2024-09-10T08:46:52.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.360+0000 7f5b7a29d740 1 Processor -- start 2024-09-10T08:46:52.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.362+0000 7f5b7a29d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818034c70 0x55d8180748b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.362+0000 7f5b7a29d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818035590 0x55d818074df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.362+0000 7f5b7a29d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818048740 0x55d818071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.363+0000 7f5b74e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818034c70 0x55d8180748b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.363+0000 7f5b74e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818034c70 0x55d8180748b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:40392/0 (socket says 172.21.15.89:40392) 2024-09-10T08:46:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.363+0000 7f5b74e00640 1 -- 172.21.15.89:0/3469088076 learned_addr learned my addr 172.21.15.89:0/3469088076 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.363+0000 7f5b6fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818035590 0x55d818074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.364+0000 7f5b7a29d740 1 -- 172.21.15.89:0/3469088076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701ab660 con 0x55d818048740 2024-09-10T08:46:52.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.364+0000 7f5b7a29d740 1 -- 172.21.15.89:0/3469088076 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701aca90 con 0x55d818035590 2024-09-10T08:46:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.364+0000 7f5b7a29d740 1 -- 172.21.15.89:0/3469088076 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5b701abcf0 con 0x55d818034c70 2024-09-10T08:46:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.364+0000 7f5b7a29d740 10 client.0 _ll_register_callbacks cb 0x7f5b60009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.364+0000 7f5b75800640 1 --2- 172.21.15.89:0/3469088076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818048740 0x55d818071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.365+0000 7f5b74e00640 1 -- 172.21.15.89:0/3469088076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818035590 msgr2=0x55d818074df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.365+0000 7f5b74e00640 1 --2- 172.21.15.89:0/3469088076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818035590 0x55d818074df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.365+0000 7f5b74e00640 1 -- 172.21.15.89:0/3469088076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818048740 msgr2=0x55d818071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.365+0000 7f5b74e00640 1 --2- 172.21.15.89:0/3469088076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818048740 0x55d818071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.365+0000 7f5b74e00640 1 -- 172.21.15.89:0/3469088076 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d818042c00 con 0x55d818034c70 2024-09-10T08:46:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.365+0000 7f5b75800640 1 --2- 172.21.15.89:0/3469088076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818048740 0x55d818071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.366+0000 7f5b74e00640 1 --2- 172.21.15.89:0/3469088076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818034c70 0x55d8180748b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5b60000c00 tx=0x7f5b60004290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a877abafef756adf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.366+0000 7f5b5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:46:52.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.366+0000 7f5b5aa00640 1 -- 172.21.15.89:0/3469088076 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60031780 con 0x55d818034c70 2024-09-10T08:46:52.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.366+0000 7f5b5aa00640 1 -- 172.21.15.89:0/3469088076 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f5b600035f0 con 0x55d818034c70 2024-09-10T08:46:52.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.367+0000 7f5b7a29d740 1 -- 172.21.15.89:0/3469088076 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b6800fa60 con 0x55d818034c70 2024-09-10T08:46:52.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.367+0000 7f5b5aa00640 1 -- 172.21.15.89:0/3469088076 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5b600037f0 con 0x55d818034c70 2024-09-10T08:46:52.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.367+0000 7f5b7a29d740 10 client.26623 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:52.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.367+0000 7f5b7a29d740 1 -- 172.21.15.89:0/3469088076 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d8180cc020 con 0x55d818034c70 2024-09-10T08:46:52.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.367+0000 7f5b5aa00640 1 -- 172.21.15.89:0/3469088076 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b6800fa60 con 0x55d818034c70 2024-09-10T08:46:52.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.368+0000 7f5b5aa00640 1 -- 172.21.15.89:0/3469088076 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f5b60049a40 con 0x55d818034c70 2024-09-10T08:46:52.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.368+0000 7f5b7a29d740 10 client.26623 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:52.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.368+0000 7f5b7a29d740 10 client.26623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:46:52.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.368+0000 7f5b7a29d740 20 client.26623 populate_metadata read hostname 'smithi089' 2024-09-10T08:46:52.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.368+0000 7f5b7a29d740 1 --2- 172.21.15.89:0/3469088076 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d8180cc7d0 0x55d8180cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:52.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.368+0000 7f5b7a29d740 4 client.26623 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:46:52.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.368+0000 7f5b7a29d740 1 -- 172.21.15.89:0/3469088076 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d817f1f0c0 con 0x55d8180cc7d0 2024-09-10T08:46:52.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.368+0000 7f5b6fe00640 1 --2- 172.21.15.89:0/3469088076 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d8180cc7d0 0x55d8180cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:52.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.369+0000 7f5b5aa00640 10 client.26623 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:52.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.370+0000 7f5b6fe00640 1 --2- 172.21.15.89:0/3469088076 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d8180cc7d0 0x55d8180cebf0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1a38c732a28c95b server_cookie=4efb42b367c47a00 in_seq=0 out_seq=0 2024-09-10T08:46:52.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.371+0000 7f5b5aa00640 1 -- 172.21.15.89:0/3469088076 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55d817f1f0c0 con 0x55d8180cc7d0 2024-09-10T08:46:52.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.371+0000 7f5b5aa00640 10 client.26623 handle_command_reply: tid=0 2024-09-10T08:46:52.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.371+0000 7f5b7a29d740 1 client.26623 shutdown 2024-09-10T08:46:52.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.372+0000 7f5b7a29d740 1 -- 172.21.15.89:0/3469088076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818034c70 msgr2=0x55d8180748b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:52.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.372+0000 7f5b7a29d740 1 --2- 172.21.15.89:0/3469088076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818034c70 0x55d8180748b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5b60000c00 tx=0x7f5b60004290 comp rx=0 tx=0).stop 2024-09-10T08:46:52.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.372+0000 7f5b7a29d740 1 -- 172.21.15.89:0/3469088076 shutdown_connections 2024-09-10T08:46:52.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.372+0000 7f5b7a29d740 1 --2- 172.21.15.89:0/3469088076 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55d8180cc7d0 0x55d8180cebf0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.372+0000 7f5b7a29d740 1 --2- 172.21.15.89:0/3469088076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d818048740 0x55d818071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.372+0000 7f5b7a29d740 1 --2- 172.21.15.89:0/3469088076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d818035590 0x55d818074df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.372+0000 7f5b7a29d740 1 --2- 172.21.15.89:0/3469088076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d818034c70 0x55d8180748b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.372+0000 7f5b7a29d740 1 -- 172.21.15.89:0/3469088076 >> 172.21.15.89:0/3469088076 conn(0x55d8180312e0 msgr2=0x55d8180366c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:52.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.373+0000 7f5b5aa00640 0 client.26623 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:52.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.373+0000 7f5b7a29d740 1 -- 172.21.15.89:0/3469088076 shutdown_connections 2024-09-10T08:46:52.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.373+0000 7f5b7a29d740 1 -- 172.21.15.89:0/3469088076 wait complete. 2024-09-10T08:46:52.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.374+0000 7f5b7a29d740 20 client.26623 trim_cache size 0 max 16384 2024-09-10T08:46:52.375 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:46:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "scrub active (1 inodes in the stack)", 2024-09-10T08:46:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": { 2024-09-10T08:46:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: "8bf06df9-b26a-4aca-9618-8557aebc03ea": { 2024-09-10T08:46:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/", 2024-09-10T08:46:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: "tag": "8bf06df9-b26a-4aca-9618-8557aebc03ea", 2024-09-10T08:46:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: "options": "recursive,force" 2024-09-10T08:46:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:46:52.375 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:46:52.375 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:46:52.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.375+0000 7f5b76200640 1 -- 172.21.15.89:0/826884098 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5b540a2c00 msgr2=0x7f5b540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:52.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.375+0000 7f5b76200640 1 --2- 172.21.15.89:0/826884098 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5b540a2c00 0x7f5b540a50c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5b60006010 tx=0x7f5b60009590 comp rx=0 tx=0).stop 2024-09-10T08:46:52.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.375+0000 7f5b76200640 1 -- 172.21.15.89:0/826884098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b7010bfc0 msgr2=0x7f5b7019ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:52.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.375+0000 7f5b76200640 1 --2- 172.21.15.89:0/826884098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b7010bfc0 0x7f5b7019ac10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00d950 tx=0x7f5b5c00de10 comp rx=0 tx=0).stop 2024-09-10T08:46:52.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.375+0000 7f5b76200640 1 -- 172.21.15.89:0/826884098 shutdown_connections 2024-09-10T08:46:52.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.375+0000 7f5b76200640 1 --2- 172.21.15.89:0/826884098 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f5b540a2c00 0x7f5b540a50c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.375+0000 7f5b76200640 1 --2- 172.21.15.89:0/826884098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b7019efa0 0x7f5b7019f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.375+0000 7f5b76200640 1 --2- 172.21.15.89:0/826884098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b7010bfc0 0x7f5b7019ac10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.375+0000 7f5b76200640 1 --2- 172.21.15.89:0/826884098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b701034b0 0x7f5b7019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.375+0000 7f5b76200640 1 -- 172.21.15.89:0/826884098 >> 172.21.15.89:0/826884098 conn(0x7f5b700fc610 msgr2=0x7f5b700fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:52.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.376+0000 7f5b76200640 1 -- 172.21.15.89:0/826884098 shutdown_connections 2024-09-10T08:46:52.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:52.376+0000 7f5b76200640 1 -- 172.21.15.89:0/826884098 wait complete. 2024-09-10T08:46:52.765 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T08:46:52.765 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-10T08:46:52.765 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:46:53.098 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:52 smithi089 ceph-mon[30757]: pgmap v601: 129 pgs: 129 active+clean; 483 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.7 MiB/s wr, 1.48k op/s 2024-09-10T08:46:53.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:52 smithi149 ceph-mon[36167]: pgmap v601: 129 pgs: 129 active+clean; 483 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.7 MiB/s wr, 1.48k op/s 2024-09-10T08:46:53.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:52 smithi111 ceph-mon[35220]: pgmap v601: 129 pgs: 129 active+clean; 483 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.7 MiB/s wr, 1.48k op/s 2024-09-10T08:46:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.451+0000 7fea7f400640 1 -- 172.21.15.89:0/1285293191 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559ba3a83b80 con 0x559ba3b981d0 2024-09-10T08:46:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.451+0000 7fea7f400640 10 client.26593 handle_command_reply: tid=0 2024-09-10T08:46:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.451+0000 7feaa7a6f740 1 client.26593 shutdown 2024-09-10T08:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.452+0000 7feaa7a6f740 1 -- 172.21.15.89:0/1285293191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b1cc30 msgr2=0x559ba3b265a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.452+0000 7feaa7a6f740 1 --2- 172.21.15.89:0/1285293191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b1cc30 0x559ba3b265a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fea90026980 tx=0x7fea90013f80 comp rx=0 tx=0).stop 2024-09-10T08:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.453+0000 7feaa7a6f740 1 -- 172.21.15.89:0/1285293191 shutdown_connections 2024-09-10T08:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.453+0000 7feaa7a6f740 1 --2- 172.21.15.89:0/1285293191 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x559ba3b981d0 0x559ba3b9a5f0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.453+0000 7feaa7a6f740 1 --2- 172.21.15.89:0/1285293191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ba3b2f390 0x559ba3b27540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.453+0000 7feaa7a6f740 1 --2- 172.21.15.89:0/1285293191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ba3b1cc30 0x559ba3b265a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.453+0000 7feaa7a6f740 1 --2- 172.21.15.89:0/1285293191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559ba3b17260 0x559ba3b26060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.453+0000 7feaa7a6f740 1 -- 172.21.15.89:0/1285293191 >> 172.21.15.89:0/1285293191 conn(0x559ba3b128d0 msgr2=0x559ba3b111f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.453+0000 7fea7f400640 0 client.26593 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.453+0000 7feaa7a6f740 1 -- 172.21.15.89:0/1285293191 shutdown_connections 2024-09-10T08:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.453+0000 7feaa7a6f740 1 -- 172.21.15.89:0/1285293191 wait complete. 2024-09-10T08:46:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.453+0000 7feaa7a6f740 20 client.26593 trim_cache size 0 max 16384 2024-09-10T08:46:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:46:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:46:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:46:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:46:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.455+0000 7feaa4a00640 1 -- 172.21.15.89:0/3068850560 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fea840a2d00 msgr2=0x7fea840a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.455+0000 7feaa4a00640 1 --2- 172.21.15.89:0/3068850560 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fea840a2d00 0x7fea840a51c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fea980be900 tx=0x7fea98034000 comp rx=0 tx=0).stop 2024-09-10T08:46:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.455+0000 7feaa4a00640 1 -- 172.21.15.89:0/3068850560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa01b4b00 msgr2=0x7feaa01b6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.455+0000 7feaa4a00640 1 --2- 172.21.15.89:0/3068850560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa01b4b00 0x7feaa01b6ef0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fea9800aef0 tx=0x7fea9800d9e0 comp rx=0 tx=0).stop 2024-09-10T08:46:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.456+0000 7feaa4a00640 1 -- 172.21.15.89:0/3068850560 shutdown_connections 2024-09-10T08:46:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.456+0000 7feaa4a00640 1 --2- 172.21.15.89:0/3068850560 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fea840a2d00 0x7fea840a51c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.456+0000 7feaa4a00640 1 --2- 172.21.15.89:0/3068850560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feaa01b4b00 0x7feaa01b6ef0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.456+0000 7feaa4a00640 1 --2- 172.21.15.89:0/3068850560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feaa0109f50 0x7feaa019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.456+0000 7feaa4a00640 1 --2- 172.21.15.89:0/3068850560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feaa0109590 0x7feaa019e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.456+0000 7feaa4a00640 1 -- 172.21.15.89:0/3068850560 >> 172.21.15.89:0/3068850560 conn(0x7feaa006ead0 msgr2=0x7feaa010c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.458+0000 7feaa4a00640 1 -- 172.21.15.89:0/3068850560 shutdown_connections 2024-09-10T08:46:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:46:53.458+0000 7feaa4a00640 1 -- 172.21.15.89:0/3068850560 wait complete. 2024-09-10T08:46:53.592 INFO:tasks.cephfs.filesystem:scrub status for tag:8bf06df9-b26a-4aca-9618-8557aebc03ea - {'path': '/', 'tag': '8bf06df9-b26a-4aca-9618-8557aebc03ea', 'options': 'recursive,force'} 2024-09-10T08:46:53.685 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:54.104 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:53 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:46:54.104 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:53 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:54.104 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:53 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:46:54.104 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:53 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:54.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:53 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:46:54.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:53 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:54.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:53 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:46:54.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:53 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:54.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.277+0000 7f1892e00640 1 -- 172.21.15.89:0/3677377818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f188c1091f0 msgr2=0x7f188c1095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:54.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.277+0000 7f1892e00640 1 --2- 172.21.15.89:0/3677377818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f188c1091f0 0x7f188c1095d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f187c009900 tx=0x7f187c02a4b0 comp rx=0 tx=0).stop 2024-09-10T08:46:54.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.278+0000 7f1892e00640 1 -- 172.21.15.89:0/3677377818 shutdown_connections 2024-09-10T08:46:54.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.278+0000 7f1892e00640 1 --2- 172.21.15.89:0/3677377818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f188c114ab0 0x7f188c116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:54.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.278+0000 7f1892e00640 1 --2- 172.21.15.89:0/3677377818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f188c109b10 0x7f188c114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:54.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.278+0000 7f1892e00640 1 --2- 172.21.15.89:0/3677377818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f188c1091f0 0x7f188c1095d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:54.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.278+0000 7f1892e00640 1 -- 172.21.15.89:0/3677377818 >> 172.21.15.89:0/3677377818 conn(0x7f188c06e140 msgr2=0x7f188c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:54.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.278+0000 7f1892e00640 1 -- 172.21.15.89:0/3677377818 shutdown_connections 2024-09-10T08:46:54.279 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.279+0000 7f1892e00640 1 -- 172.21.15.89:0/3677377818 wait complete. 2024-09-10T08:46:54.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.280+0000 7f1892e00640 1 Processor -- start 2024-09-10T08:46:54.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.280+0000 7f1892e00640 1 -- start start 2024-09-10T08:46:54.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.280+0000 7f1892e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f188c1091f0 0x7f188c10c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:54.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.281+0000 7f1892e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f188c109b10 0x7f188c10cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:54.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.281+0000 7f1892e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f188c114ab0 0x7f188c10d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:54.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.281+0000 7f1892e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f188c11a850 con 0x7f188c1091f0 2024-09-10T08:46:54.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.281+0000 7f1892e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f188c11a6d0 con 0x7f188c109b10 2024-09-10T08:46:54.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.281+0000 7f1892e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f188c11a9d0 con 0x7f188c114ab0 2024-09-10T08:46:54.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.281+0000 7f188b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f188c109b10 0x7f188c10cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:54.282 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:53 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:46:54.282 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:53 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:54.282 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:53 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:46:54.282 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:53 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:54.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.281+0000 7f188b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f188c109b10 0x7f188c10cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50526/0 (socket says 172.21.15.89:50526) 2024-09-10T08:46:54.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.281+0000 7f1890e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f188c114ab0 0x7f188c10d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:54.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.281+0000 7f1890e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f188c114ab0 0x7f188c10d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:60764/0 (socket says 172.21.15.89:60764) 2024-09-10T08:46:54.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.281+0000 7f1890e00640 1 -- 172.21.15.89:0/3058626067 learned_addr learned my addr 172.21.15.89:0/3058626067 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:54.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.282+0000 7f1890e00640 1 -- 172.21.15.89:0/3058626067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f188c109b10 msgr2=0x7f188c10cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:54.283 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.282+0000 7f1890e00640 1 --2- 172.21.15.89:0/3058626067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f188c109b10 0x7f188c10cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:54.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.282+0000 7f1890e00640 1 -- 172.21.15.89:0/3058626067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f188c1091f0 msgr2=0x7f188c10c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:54.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.282+0000 7f1890e00640 1 --2- 172.21.15.89:0/3058626067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f188c1091f0 0x7f188c10c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:54.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.282+0000 7f1890e00640 1 -- 172.21.15.89:0/3058626067 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188c1a2cf0 con 0x7f188c114ab0 2024-09-10T08:46:54.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.283+0000 7f188b400640 1 --2- 172.21.15.89:0/3058626067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f188c109b10 0x7f188c10cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:54.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.283+0000 7f1890e00640 1 --2- 172.21.15.89:0/3058626067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f188c114ab0 0x7f188c10d060 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f188400ba10 tx=0x7f188400bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:54.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.285+0000 7f1888c00640 1 -- 172.21.15.89:0/3058626067 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884019070 con 0x7f188c114ab0 2024-09-10T08:46:54.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.285+0000 7f1888c00640 1 -- 172.21.15.89:0/3058626067 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1884014440 con 0x7f188c114ab0 2024-09-10T08:46:54.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.285+0000 7f1892e00640 1 -- 172.21.15.89:0/3058626067 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188c1a2fe0 con 0x7f188c114ab0 2024-09-10T08:46:54.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.285+0000 7f1888c00640 1 -- 172.21.15.89:0/3058626067 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18840125a0 con 0x7f188c114ab0 2024-09-10T08:46:54.285 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.285+0000 7f1892e00640 1 -- 172.21.15.89:0/3058626067 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f188c1a3520 con 0x7f188c114ab0 2024-09-10T08:46:54.288 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.287+0000 7f1892e00640 1 -- 172.21.15.89:0/3058626067 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f188c073a40 con 0x7f188c114ab0 2024-09-10T08:46:54.289 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.289+0000 7f1888c00640 1 -- 172.21.15.89:0/3058626067 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f188401c070 con 0x7f188c114ab0 2024-09-10T08:46:54.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.291+0000 7f1888c00640 1 --2- 172.21.15.89:0/3058626067 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f18700a2c40 0x7f18700a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:54.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.292+0000 7f1888c00640 1 -- 172.21.15.89:0/3058626067 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f18840ce920 con 0x7f188c114ab0 2024-09-10T08:46:54.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.293+0000 7f188be00640 1 --2- 172.21.15.89:0/3058626067 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f18700a2c40 0x7f18700a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:54.295 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.295+0000 7f188be00640 1 --2- 172.21.15.89:0/3058626067 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f18700a2c40 0x7f18700a5100 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f187c005c20 tx=0x7f187c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:54.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.296+0000 7f1888c00640 1 -- 172.21.15.89:0/3058626067 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f188409b4b0 con 0x7f188c114ab0 2024-09-10T08:46:54.544 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h[64759]: 2024-09-10T08:46:54.280+0000 7fac2fed3640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000005ef8 [2,head] /volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-4.jpg auth v126 ap=1 s=57026 n(v0 rc2024-09-10T08:45:50.296686+0000 b57026 1=1+0) (ifile excl) cr={25157=0-4194304@1} caps={25157=pAsLsXsFsxcrwb/pFscr@4},l=25157 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c6b107bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000005ef8:[<0x10000005ed5/picture-4.jpg v64>,<0x10000000399/blog-102 v219048>,<0x10000000208/blogtest_in v1326>,<0x10000000201/src v2013>,<0x10000000006/blogbench-1.0 v1769>,<0x10000000005/tmp v372>,<0x100000001fb/client.0 v358>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v362>,<0x10000000001/sv_1 v383>,<0x10000000000/qa v359>,<0x1/volumes v358>]//[3]","memoryvalue":"(4)0x10000005ef8:[<0x10000005ed5/picture-4.jpg v126>,<0x10000000399/blog-102 v237249>,<0x10000000208/blogtest_in v1676>,<0x10000000201/src v2113>,<0x10000000006/blogbench-1.0 v1861>,<0x10000000005/tmp v465>,<0x100000001fb/client.0 v445>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v455>,<0x10000000001/sv_1 v459>,<0x10000000000/qa v303>,<0x1/volumes v440>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-10T08:46:54.544 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h[64759]: 2024-09-10T08:46:54.280+0000 7fac2fed3640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000005eed [2,head] /volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-7.jpg auth v138 ap=1 s=45557 n(v0 rc2024-09-10T08:45:50.267686+0000 b45557 1=1+0) caps={25157=pAsLsXsFsc/-@1},l=25157(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c6b107cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000005eed:[<0x10000005ed5/picture-7.jpg v46>,<0x10000000399/blog-102 v219048>,<0x10000000208/blogtest_in v1326>,<0x10000000201/src v2013>,<0x10000000006/blogbench-1.0 v1769>,<0x10000000005/tmp v372>,<0x100000001fb/client.0 v358>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v362>,<0x10000000001/sv_1 v383>,<0x10000000000/qa v359>,<0x1/volumes v358>]//[3]","memoryvalue":"(4)0x10000005eed:[<0x10000005ed5/picture-7.jpg v138>,<0x10000000399/blog-102 v237249>,<0x10000000208/blogtest_in v1676>,<0x10000000201/src v2113>,<0x10000000006/blogbench-1.0 v1861>,<0x10000000005/tmp v465>,<0x100000001fb/client.0 v445>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v455>,<0x10000000001/sv_1 v459>,<0x10000000000/qa v303>,<0x1/volumes v440>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-10T08:46:54.545 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h[64759]: 2024-09-10T08:46:54.282+0000 7fac2fed3640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000005ef0 [2,head] /volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-6.jpg auth v134 ap=1 s=64525 n(v0 rc2024-09-10T08:45:50.275686+0000 b64525 1=1+0) (ifile excl) cr={25157=0-4194304@1} caps={25157=pAsLsXsFsxcrwb/pFscr@4},l=25157 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c6b1084b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000005ef0:[<0x10000005ed5/picture-6.jpg v52>,<0x10000000399/blog-102 v219048>,<0x10000000208/blogtest_in v1326>,<0x10000000201/src v2013>,<0x10000000006/blogbench-1.0 v1769>,<0x10000000005/tmp v372>,<0x100000001fb/client.0 v358>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v362>,<0x10000000001/sv_1 v383>,<0x10000000000/qa v359>,<0x1/volumes v358>]//[3]","memoryvalue":"(4)0x10000005ef0:[<0x10000005ed5/picture-6.jpg v134>,<0x10000000399/blog-102 v237249>,<0x10000000208/blogtest_in v1676>,<0x10000000201/src v2113>,<0x10000000006/blogbench-1.0 v1861>,<0x10000000005/tmp v465>,<0x100000001fb/client.0 v445>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v455>,<0x10000000001/sv_1 v459>,<0x10000000000/qa v303>,<0x1/volumes v440>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-10T08:46:54.545 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h[64759]: 2024-09-10T08:46:54.282+0000 7fac2fed3640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000005eeb [2,head] /volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-8.jpg auth v140 ap=1 s=57141 n(v0 rc2024-09-10T08:45:50.259687+0000 b57141 1=1+0) caps={25157=pAsLsXsFsc/-@1},l=25157(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c6b1085b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000005eeb:[<0x10000005ed5/picture-8.jpg v40>,<0x10000000399/blog-102 v219048>,<0x10000000208/blogtest_in v1326>,<0x10000000201/src v2013>,<0x10000000006/blogbench-1.0 v1769>,<0x10000000005/tmp v372>,<0x100000001fb/client.0 v358>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v362>,<0x10000000001/sv_1 v383>,<0x10000000000/qa v359>,<0x1/volumes v358>]//[3]","memoryvalue":"(4)0x10000005eeb:[<0x10000005ed5/picture-8.jpg v140>,<0x10000000399/blog-102 v237249>,<0x10000000208/blogtest_in v1676>,<0x10000000201/src v2113>,<0x10000000006/blogbench-1.0 v1861>,<0x10000000005/tmp v465>,<0x100000001fb/client.0 v445>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v455>,<0x10000000001/sv_1 v459>,<0x10000000000/qa v303>,<0x1/volumes v440>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-10T08:46:54.545 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h[64759]: 2024-09-10T08:46:54.282+0000 7fac2fed3640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000005ee0 [2,head] /volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-3.xml auth v124 ap=1 s=8522 n(v0 rc2024-09-10T08:45:50.229687+0000 b8522 1=1+0) (ifile excl) cr={25157=0-4194304@1} caps={25157=pAsLsXsFsxcrwb/pFscr@3},l=25157 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c6b1085600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000005ee0:[<0x10000005ed5/article-3.xml v16>,<0x10000000399/blog-102 v219048>,<0x10000000208/blogtest_in v1326>,<0x10000000201/src v2013>,<0x10000000006/blogbench-1.0 v1769>,<0x10000000005/tmp v372>,<0x100000001fb/client.0 v358>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v362>,<0x10000000001/sv_1 v383>,<0x10000000000/qa v359>,<0x1/volumes v358>]//[3]","memoryvalue":"(4)0x10000005ee0:[<0x10000005ed5/article-3.xml v124>,<0x10000000399/blog-102 v237249>,<0x10000000208/blogtest_in v1676>,<0x10000000201/src v2113>,<0x10000000006/blogbench-1.0 v1861>,<0x10000000005/tmp v465>,<0x100000001fb/client.0 v445>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v455>,<0x10000000001/sv_1 v459>,<0x10000000000/qa v303>,<0x1/volumes v440>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-10T08:46:54.545 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h[64759]: 2024-09-10T08:46:54.282+0000 7fac2fed3640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000005ee7 [2,head] /volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-9.jpg auth v136 ap=1 s=48817 n(v0 rc2024-09-10T08:45:50.249687+0000 b48817 1=1+0) caps={25157=pAsLsXsFsc/-@1},l=25157(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c6b1084000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000005ee7:[<0x10000005ed5/picture-9.jpg v34>,<0x10000000399/blog-102 v219048>,<0x10000000208/blogtest_in v1326>,<0x10000000201/src v2013>,<0x10000000006/blogbench-1.0 v1769>,<0x10000000005/tmp v372>,<0x100000001fb/client.0 v358>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v362>,<0x10000000001/sv_1 v383>,<0x10000000000/qa v359>,<0x1/volumes v358>]//[3]","memoryvalue":"(4)0x10000005ee7:[<0x10000005ed5/picture-9.jpg v136>,<0x10000000399/blog-102 v237249>,<0x10000000208/blogtest_in v1676>,<0x10000000201/src v2113>,<0x10000000006/blogbench-1.0 v1861>,<0x10000000005/tmp v465>,<0x100000001fb/client.0 v445>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v455>,<0x10000000001/sv_1 v459>,<0x10000000000/qa v303>,<0x1/volumes v440>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-10T08:46:54.545 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h[64759]: 2024-09-10T08:46:54.283+0000 7fac2fed3640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000005ef4 [2,head] /volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-5.jpg auth v130 ap=1 s=58132 n(v0 rc2024-09-10T08:45:50.282686+0000 b58132 1=1+0) (ifile excl) cr={25157=0-4194304@1} caps={25157=pAsLsXsFsxcrwb/pFscr@3},l=25157 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c6b1741b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000005ef4:[<0x10000005ed5/picture-5.jpg v58>,<0x10000000399/blog-102 v219048>,<0x10000000208/blogtest_in v1326>,<0x10000000201/src v2013>,<0x10000000006/blogbench-1.0 v1769>,<0x10000000005/tmp v372>,<0x100000001fb/client.0 v358>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v362>,<0x10000000001/sv_1 v383>,<0x10000000000/qa v359>,<0x1/volumes v358>]//[3]","memoryvalue":"(4)0x10000005ef4:[<0x10000005ed5/picture-5.jpg v130>,<0x10000000399/blog-102 v237249>,<0x10000000208/blogtest_in v1676>,<0x10000000201/src v2113>,<0x10000000006/blogbench-1.0 v1861>,<0x10000000005/tmp v465>,<0x100000001fb/client.0 v445>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v455>,<0x10000000001/sv_1 v459>,<0x10000000000/qa v303>,<0x1/volumes v440>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-10T08:46:54.545 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h[64759]: 2024-09-10T08:46:54.284+0000 7fac2fed3640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000005ed7 [2,head] /volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-5.xml auth v132 ap=1 s=5456 n(v0 rc2024-09-10T08:45:50.205688+0000 b5456 1=1+0) (ifile excl) cr={25157=0-4194304@1} caps={25157=pAsLsXsFsxcrwb/pFscr@3},l=25157 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c6b206a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000005ed7:[<0x10000005ed5/article-5.xml v4>,<0x10000000399/blog-102 v219048>,<0x10000000208/blogtest_in v1326>,<0x10000000201/src v2013>,<0x10000000006/blogbench-1.0 v1769>,<0x10000000005/tmp v372>,<0x100000001fb/client.0 v358>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v362>,<0x10000000001/sv_1 v383>,<0x10000000000/qa v359>,<0x1/volumes v358>]//[3]","memoryvalue":"(4)0x10000005ed7:[<0x10000005ed5/article-5.xml v132>,<0x10000000399/blog-102 v237249>,<0x10000000208/blogtest_in v1676>,<0x10000000201/src v2113>,<0x10000000006/blogbench-1.0 v1861>,<0x10000000005/tmp v465>,<0x100000001fb/client.0 v445>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v455>,<0x10000000001/sv_1 v459>,<0x10000000000/qa v303>,<0x1/volumes v440>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-10T08:46:54.545 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h[64759]: 2024-09-10T08:46:54.285+0000 7fac2fed3640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000005edb [2,head] /volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-4.xml auth v128 ap=1 s=17358 n(v0 rc2024-09-10T08:45:50.221687+0000 b17358 1=1+0) (ifile excl) cr={25157=0-4194304@1} caps={25157=pAsLsXsFsxcrwb/pFscr@4},l=25157 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c6b206a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000005edb:[<0x10000005ed5/article-4.xml v10>,<0x10000000399/blog-102 v219048>,<0x10000000208/blogtest_in v1326>,<0x10000000201/src v2013>,<0x10000000006/blogbench-1.0 v1769>,<0x10000000005/tmp v372>,<0x100000001fb/client.0 v358>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v362>,<0x10000000001/sv_1 v383>,<0x10000000000/qa v359>,<0x1/volumes v358>]//[3]","memoryvalue":"(4)0x10000005edb:[<0x10000005ed5/article-4.xml v128>,<0x10000000399/blog-102 v237249>,<0x10000000208/blogtest_in v1676>,<0x10000000201/src v2113>,<0x10000000006/blogbench-1.0 v1861>,<0x10000000005/tmp v465>,<0x100000001fb/client.0 v445>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v455>,<0x10000000001/sv_1 v459>,<0x10000000000/qa v303>,<0x1/volumes v440>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-10T08:46:54.545 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h[64759]: 2024-09-10T08:46:54.286+0000 7fac2fed3640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000005ee3 [2,head] /volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-2.xml auth v120 ap=1 s=28608 n(v0 rc2024-09-10T08:45:50.234687+0000 b28608 1=1+0) (ifile excl) cr={25157=0-4194304@1} caps={25157=pAsLsXsFsxcrwb/pFscr@4},l=25157 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c6b1086100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000005ee3:[<0x10000005ed5/article-2.xml v22>,<0x10000000399/blog-102 v219048>,<0x10000000208/blogtest_in v1326>,<0x10000000201/src v2013>,<0x10000000006/blogbench-1.0 v1769>,<0x10000000005/tmp v372>,<0x100000001fb/client.0 v358>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v362>,<0x10000000001/sv_1 v383>,<0x10000000000/qa v359>,<0x1/volumes v358>]//[3]","memoryvalue":"(4)0x10000005ee3:[<0x10000005ed5/article-2.xml v120>,<0x10000000399/blog-102 v237249>,<0x10000000208/blogtest_in v1676>,<0x10000000201/src v2113>,<0x10000000006/blogbench-1.0 v1861>,<0x10000000005/tmp v465>,<0x100000001fb/client.0 v445>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v455>,<0x10000000001/sv_1 v459>,<0x10000000000/qa v303>,<0x1/volumes v440>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-10T08:46:54.545 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h[64759]: 2024-09-10T08:46:54.287+0000 7fac2fed3640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000005ee4 [2,head] /volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-1.xml auth v116 ap=1 s=12897 n(v0 rc2024-09-10T08:45:50.243687+0000 b12897 1=1+0) (ifile excl) cr={25157=0-4194304@1} caps={25157=pAsLsXsFsxcrwb/pFscr@3},l=25157 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c6b206ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000005ee4:[<0x10000005ed5/article-1.xml v28>,<0x10000000399/blog-102 v219048>,<0x10000000208/blogtest_in v1326>,<0x10000000201/src v2013>,<0x10000000006/blogbench-1.0 v1769>,<0x10000000005/tmp v372>,<0x100000001fb/client.0 v358>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v362>,<0x10000000001/sv_1 v383>,<0x10000000000/qa v359>,<0x1/volumes v358>]//[3]","memoryvalue":"(4)0x10000005ee4:[<0x10000005ed5/article-1.xml v116>,<0x10000000399/blog-102 v237249>,<0x10000000208/blogtest_in v1676>,<0x10000000201/src v2113>,<0x10000000006/blogbench-1.0 v1861>,<0x10000000005/tmp v465>,<0x100000001fb/client.0 v445>,<0x100000001fa/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e v455>,<0x10000000001/sv_1 v459>,<0x10000000000/qa v303>,<0x1/volumes v440>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-10T08:46:54.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.604+0000 7f1892e00640 1 -- 172.21.15.89:0/3058626067 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f188c1a37e0 con 0x7f188c114ab0 2024-09-10T08:46:54.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.604+0000 7f1888c00640 1 -- 172.21.15.89:0/3058626067 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11919 (secure 0 0 0) 0x7f188409adb0 con 0x7f188c114ab0 2024-09-10T08:46:54.605 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:46:54.605 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 29 2024-09-10T08:46:54.605 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":29,"btime":"2024-09-10T08:44:40:778845+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:44:40.174532+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":20,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24496,24518,24554]},"id":1}]} 2024-09-10T08:46:54.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.607+0000 7f1892e00640 1 -- 172.21.15.89:0/3058626067 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f18700a2c40 msgr2=0x7f18700a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:54.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.607+0000 7f1892e00640 1 --2- 172.21.15.89:0/3058626067 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f18700a2c40 0x7f18700a5100 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f187c005c20 tx=0x7f187c000bc0 comp rx=0 tx=0).stop 2024-09-10T08:46:54.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.607+0000 7f1892e00640 1 -- 172.21.15.89:0/3058626067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f188c114ab0 msgr2=0x7f188c10d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:54.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.607+0000 7f1892e00640 1 --2- 172.21.15.89:0/3058626067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f188c114ab0 0x7f188c10d060 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f188400ba10 tx=0x7f188400bed0 comp rx=0 tx=0).stop 2024-09-10T08:46:54.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.607+0000 7f1892e00640 1 -- 172.21.15.89:0/3058626067 shutdown_connections 2024-09-10T08:46:54.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.607+0000 7f1892e00640 1 --2- 172.21.15.89:0/3058626067 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f18700a2c40 0x7f18700a5100 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:54.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.607+0000 7f1892e00640 1 --2- 172.21.15.89:0/3058626067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f188c114ab0 0x7f188c10d060 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:54.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.607+0000 7f1892e00640 1 --2- 172.21.15.89:0/3058626067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f188c109b10 0x7f188c10cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:54.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.607+0000 7f1892e00640 1 --2- 172.21.15.89:0/3058626067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f188c1091f0 0x7f188c10c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:54.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.607+0000 7f1892e00640 1 -- 172.21.15.89:0/3058626067 >> 172.21.15.89:0/3058626067 conn(0x7f188c06e140 msgr2=0x7f188c10a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:54.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.608+0000 7f1892e00640 1 -- 172.21.15.89:0/3058626067 shutdown_connections 2024-09-10T08:46:54.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:54.608+0000 7f1892e00640 1 -- 172.21.15.89:0/3058626067 wait complete. 2024-09-10T08:46:55.198 INFO:tasks.workunit.client.0.smithi089.stdout: 2024-09-10T08:46:55.199 INFO:tasks.workunit.client.0.smithi089.stdout:Final score for writes: 136 2024-09-10T08:46:55.199 INFO:tasks.workunit.client.0.smithi089.stdout:Final score for reads : 175775 2024-09-10T08:46:55.199 INFO:tasks.workunit.client.0.smithi089.stdout: 2024-09-10T08:46:55.202 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T08:46:55.202 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T08:46:55.291 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: pgmap v602: 129 pgs: 129 active+clean; 493 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.8 MiB/s wr, 1.89k op/s 2024-09-10T08:46:55.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:46:55.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:46:55.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:55.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:55.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:46:55.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: Scrub error on inode 0x10000005ef8 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-4.jpg) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: Scrub error on inode 0x10000005eed (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-7.jpg) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: Scrub error on inode 0x10000005ef0 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-6.jpg) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: Scrub error on inode 0x10000005eeb (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-8.jpg) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: Scrub error on inode 0x10000005ee0 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-3.xml) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: Scrub error on inode 0x10000005ee7 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-9.jpg) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.292 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: Scrub error on inode 0x10000005ef4 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-5.jpg) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: Scrub error on inode 0x10000005ed7 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-5.xml) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: Scrub error on inode 0x10000005edb (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-4.xml) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: Scrub error on inode 0x10000005ee3 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-2.xml) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: Scrub error on inode 0x10000005ee4 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-1.xml) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:55.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:46:55.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:55.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:46:55.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: scrub summary: active paths [/] 2024-09-10T08:46:55.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:55.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3058626067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:55.293 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:54 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:55.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: pgmap v602: 129 pgs: 129 active+clean; 493 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.8 MiB/s wr, 1.89k op/s 2024-09-10T08:46:55.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:46:55.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:46:55.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:55.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:55.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:46:55.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: Scrub error on inode 0x10000005ef8 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-4.jpg) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: Scrub error on inode 0x10000005eed (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-7.jpg) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: Scrub error on inode 0x10000005ef0 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-6.jpg) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: Scrub error on inode 0x10000005eeb (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-8.jpg) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: Scrub error on inode 0x10000005ee0 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-3.xml) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: Scrub error on inode 0x10000005ee7 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-9.jpg) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: Scrub error on inode 0x10000005ef4 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-5.jpg) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: Scrub error on inode 0x10000005ed7 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-5.xml) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: Scrub error on inode 0x10000005edb (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-4.xml) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: Scrub error on inode 0x10000005ee3 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-2.xml) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: Scrub error on inode 0x10000005ee4 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-1.xml) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:55.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:46:55.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:55.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:46:55.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: scrub summary: active paths [/] 2024-09-10T08:46:55.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:55.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3058626067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:55.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:54 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:55.408 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24554 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-10T08:46:55.409 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24554 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-10T08:46:55.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: pgmap v602: 129 pgs: 129 active+clean; 493 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.8 MiB/s wr, 1.89k op/s 2024-09-10T08:46:55.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:46:55.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:46:55.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:55.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:55.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:46:55.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: Scrub error on inode 0x10000005ef8 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-4.jpg) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: Scrub error on inode 0x10000005eed (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-7.jpg) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: Scrub error on inode 0x10000005ef0 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-6.jpg) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: Scrub error on inode 0x10000005eeb (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-8.jpg) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: Scrub error on inode 0x10000005ee0 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-3.xml) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: Scrub error on inode 0x10000005ee7 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-9.jpg) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.472 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: Scrub error on inode 0x10000005ef4 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-5.jpg) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.472 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: Scrub error on inode 0x10000005ed7 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-5.xml) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.472 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: Scrub error on inode 0x10000005edb (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-4.xml) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.472 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: Scrub error on inode 0x10000005ee3 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-2.xml) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.472 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: Scrub error on inode 0x10000005ee4 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-1.xml) see mds.h log and `damage ls` output for details 2024-09-10T08:46:55.472 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:55.472 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:46:55.472 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:55.472 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:46:55.472 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: scrub summary: active paths [/] 2024-09-10T08:46:55.472 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:55.472 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3058626067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:55.472 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:54 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:55.829 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:56.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.534+0000 7fd783e00640 1 -- 172.21.15.89:0/760460130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd77c1049a0 msgr2=0x7fd77c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:56.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.534+0000 7fd783e00640 1 --2- 172.21.15.89:0/760460130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd77c1049a0 0x7fd77c1089f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd768009900 tx=0x7fd76802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.534+0000 7fd783e00640 1 -- 172.21.15.89:0/760460130 shutdown_connections 2024-09-10T08:46:56.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.534+0000 7fd783e00640 1 --2- 172.21.15.89:0/760460130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd77c109120 0x7fd77c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.534+0000 7fd783e00640 1 --2- 172.21.15.89:0/760460130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd77c1049a0 0x7fd77c1089f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.534+0000 7fd783e00640 1 --2- 172.21.15.89:0/760460130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c103ff0 0x7fd77c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.534+0000 7fd783e00640 1 -- 172.21.15.89:0/760460130 >> 172.21.15.89:0/760460130 conn(0x7fd77c0ffa90 msgr2=0x7fd77c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:56.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.534+0000 7fd783e00640 1 -- 172.21.15.89:0/760460130 shutdown_connections 2024-09-10T08:46:56.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.534+0000 7fd783e00640 1 -- 172.21.15.89:0/760460130 wait complete. 2024-09-10T08:46:56.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.535+0000 7fd783e00640 1 Processor -- start 2024-09-10T08:46:56.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.535+0000 7fd783e00640 1 -- start start 2024-09-10T08:46:56.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.535+0000 7fd783e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c103ff0 0x7fd77c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:56.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.535+0000 7fd783e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd77c1049a0 0x7fd77c19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:56.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.536+0000 7fd783e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd77c109120 0x7fd77c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:56.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.536+0000 7fd783e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c114120 con 0x7fd77c109120 2024-09-10T08:46:56.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.536+0000 7fd783e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c113fa0 con 0x7fd77c1049a0 2024-09-10T08:46:56.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.536+0000 7fd783e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1142a0 con 0x7fd77c103ff0 2024-09-10T08:46:56.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.536+0000 7fd780a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd77c1049a0 0x7fd77c19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:56.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.536+0000 7fd781400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c103ff0 0x7fd77c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:56.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.536+0000 7fd780a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd77c1049a0 0x7fd77c19ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50556/0 (socket says 172.21.15.89:50556) 2024-09-10T08:46:56.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.536+0000 7fd781400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c103ff0 0x7fd77c19a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:60794/0 (socket says 172.21.15.89:60794) 2024-09-10T08:46:56.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.536+0000 7fd780a00640 1 -- 172.21.15.89:0/72834518 learned_addr learned my addr 172.21.15.89:0/72834518 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:56.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.536+0000 7fd781e00640 1 --2- 172.21.15.89:0/72834518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd77c109120 0x7fd77c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:56.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.536+0000 7fd780a00640 1 -- 172.21.15.89:0/72834518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c103ff0 msgr2=0x7fd77c19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:56.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.536+0000 7fd780a00640 1 --2- 172.21.15.89:0/72834518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c103ff0 0x7fd77c19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.536+0000 7fd780a00640 1 -- 172.21.15.89:0/72834518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd77c109120 msgr2=0x7fd77c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:56.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.536+0000 7fd780a00640 1 --2- 172.21.15.89:0/72834518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd77c109120 0x7fd77c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.536+0000 7fd780a00640 1 -- 172.21.15.89:0/72834518 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77c19f670 con 0x7fd77c1049a0 2024-09-10T08:46:56.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.537+0000 7fd780a00640 1 --2- 172.21.15.89:0/72834518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd77c1049a0 0x7fd77c19ac00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd768006010 tx=0x7fd768004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:56.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.537+0000 7fd77a000640 1 -- 172.21.15.89:0/72834518 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76803f030 con 0x7fd77c1049a0 2024-09-10T08:46:56.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.537+0000 7fd77a000640 1 -- 172.21.15.89:0/72834518 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd768004440 con 0x7fd77c1049a0 2024-09-10T08:46:56.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.537+0000 7fd77a000640 1 -- 172.21.15.89:0/72834518 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768035070 con 0x7fd77c1049a0 2024-09-10T08:46:56.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.537+0000 7fd783e00640 1 -- 172.21.15.89:0/72834518 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd77c19f900 con 0x7fd77c1049a0 2024-09-10T08:46:56.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.537+0000 7fd783e00640 1 -- 172.21.15.89:0/72834518 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd77c19fd90 con 0x7fd77c1049a0 2024-09-10T08:46:56.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.540+0000 7fd7863fd740 1 Processor -- start 2024-09-10T08:46:56.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.540+0000 7fd7863fd740 1 -- start start 2024-09-10T08:46:56.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.540+0000 7fd77a000640 1 -- 172.21.15.89:0/72834518 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd768042030 con 0x7fd77c1049a0 2024-09-10T08:46:56.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.540+0000 7fd7863fd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f63b0 0x55b4b87f6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:56.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.541+0000 7fd7863fd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b87f6cd0 0x55b4b8801050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:56.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.541+0000 7fd7863fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b8801590 0x55b4b8803980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:56.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.541+0000 7fd7863fd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b4b875c530 con 0x55b4b87f6cd0 2024-09-10T08:46:56.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.541+0000 7fd7863fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b4b8703860 con 0x55b4b8801590 2024-09-10T08:46:56.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.541+0000 7fd7863fd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b4b87584f0 con 0x55b4b87f63b0 2024-09-10T08:46:56.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.541+0000 7fd781e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f63b0 0x55b4b87f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:56.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.541+0000 7fd780a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b8801590 0x55b4b8803980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:56.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.541+0000 7fd781e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f63b0 0x55b4b87f6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:60810/0 (socket says 172.21.15.89:60810) 2024-09-10T08:46:56.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.541+0000 7fd781e00640 1 -- 172.21.15.89:0/1038453394 learned_addr learned my addr 172.21.15.89:0/1038453394 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:56.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.541+0000 7fd781400640 1 --2- 172.21.15.89:0/1038453394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b87f6cd0 0x55b4b8801050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:56.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.541+0000 7fd781e00640 1 -- 172.21.15.89:0/1038453394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b8801590 msgr2=0x55b4b8803980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:56.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.541+0000 7fd781e00640 1 --2- 172.21.15.89:0/1038453394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b8801590 0x55b4b8803980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.541+0000 7fd781e00640 1 -- 172.21.15.89:0/1038453394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b87f6cd0 msgr2=0x55b4b8801050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:56.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.541+0000 7fd781e00640 1 --2- 172.21.15.89:0/1038453394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b87f6cd0 0x55b4b8801050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.541+0000 7fd781e00640 1 -- 172.21.15.89:0/1038453394 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4b84cdb90 con 0x55b4b87f63b0 2024-09-10T08:46:56.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.542+0000 7fd781e00640 1 --2- 172.21.15.89:0/1038453394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f63b0 0x55b4b87f6790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd774009f50 tx=0x7fd7740118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1deff658726cdf71 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:56.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.542+0000 7fd76e600640 1 -- 172.21.15.89:0/1038453394 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd774004d20 con 0x55b4b87f63b0 2024-09-10T08:46:56.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.542+0000 7fd76e600640 1 -- 172.21.15.89:0/1038453394 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd774004050 con 0x55b4b87f63b0 2024-09-10T08:46:56.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.542+0000 7fd76e600640 1 -- 172.21.15.89:0/1038453394 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd774005280 con 0x55b4b87f63b0 2024-09-10T08:46:56.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.542+0000 7fd77a000640 1 --2- 172.21.15.89:0/72834518 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd7640a2c00 0x7fd7640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:56.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.543+0000 7fd77a000640 1 -- 172.21.15.89:0/72834518 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fd7680ef730 con 0x7fd77c1049a0 2024-09-10T08:46:56.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.543+0000 7fd781400640 1 --2- 172.21.15.89:0/72834518 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd7640a2c00 0x7fd7640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:56.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.543+0000 7fd7863fd740 1 -- 172.21.15.89:0/1038453394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f63b0 msgr2=0x55b4b87f6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:56.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.543+0000 7fd7863fd740 1 --2- 172.21.15.89:0/1038453394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f63b0 0x55b4b87f6790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd774009f50 tx=0x7fd7740118d0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.543+0000 7fd7863fd740 1 -- 172.21.15.89:0/1038453394 shutdown_connections 2024-09-10T08:46:56.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.543+0000 7fd7863fd740 1 --2- 172.21.15.89:0/1038453394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b8801590 0x55b4b8803980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.543+0000 7fd7863fd740 1 --2- 172.21.15.89:0/1038453394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b87f6cd0 0x55b4b8801050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.543+0000 7fd7863fd740 1 --2- 172.21.15.89:0/1038453394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f63b0 0x55b4b87f6790 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.543+0000 7fd7863fd740 1 -- 172.21.15.89:0/1038453394 >> 172.21.15.89:0/1038453394 conn(0x55b4b87f2940 msgr2=0x55b4b87f2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:56.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.543+0000 7fd7863fd740 1 -- 172.21.15.89:0/1038453394 shutdown_connections 2024-09-10T08:46:56.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.544+0000 7fd7863fd740 1 -- 172.21.15.89:0/1038453394 wait complete. 2024-09-10T08:46:56.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.544+0000 7fd781400640 1 --2- 172.21.15.89:0/72834518 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd7640a2c00 0x7fd7640a50c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55b4b87f37a0 tx=0x7fd770005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:56.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.544+0000 7fd7863fd740 1 -- start start 2024-09-10T08:46:56.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.545+0000 7fd7863fd740 1 Processor -- start 2024-09-10T08:46:56.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.545+0000 7fd7863fd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f63b0 0x55b4b87fe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:56.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.545+0000 7fd7863fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b87f6cd0 0x55b4b87fec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:56.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.545+0000 7fd7863fd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b8801590 0x55b4b87ff190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:56.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.545+0000 7fd7863fd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b4b8803fb0 con 0x55b4b8801590 2024-09-10T08:46:56.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.545+0000 7fd7863fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b4b8716e60 con 0x55b4b87f6cd0 2024-09-10T08:46:56.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.545+0000 7fd7863fd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b4b8804130 con 0x55b4b87f63b0 2024-09-10T08:46:56.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.545+0000 7fd7863fd740 10 client.0 _ll_register_callbacks cb 0x55b4b871bcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:56.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.545+0000 7fd780a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b87f6cd0 0x55b4b87fec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:56.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.545+0000 7fd780a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b87f6cd0 0x55b4b87fec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50586/0 (socket says 172.21.15.89:50586) 2024-09-10T08:46:56.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.545+0000 7fd781400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f63b0 0x55b4b87fe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:56.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.545+0000 7fd780a00640 1 -- 172.21.15.89:0/1673794454 learned_addr learned my addr 172.21.15.89:0/1673794454 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:56.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.546+0000 7fd781e00640 1 --2- 172.21.15.89:0/1673794454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b8801590 0x55b4b87ff190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:56.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.546+0000 7fd780a00640 1 -- 172.21.15.89:0/1673794454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f63b0 msgr2=0x55b4b87fe710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:56.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.546+0000 7fd780a00640 1 --2- 172.21.15.89:0/1673794454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f63b0 0x55b4b87fe710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.546+0000 7fd780a00640 1 -- 172.21.15.89:0/1673794454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b8801590 msgr2=0x55b4b87ff190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:56.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.546+0000 7fd780a00640 1 --2- 172.21.15.89:0/1673794454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b8801590 0x55b4b87ff190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.546+0000 7fd780a00640 1 -- 172.21.15.89:0/1673794454 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4b8722990 con 0x55b4b87f6cd0 2024-09-10T08:46:56.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.546+0000 7fd781e00640 1 --2- 172.21.15.89:0/1673794454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b8801590 0x55b4b87ff190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:56.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.546+0000 7fd780a00640 1 --2- 172.21.15.89:0/1673794454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b87f6cd0 0x55b4b87fec50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd7680c42a0 tx=0x7fd7680f3040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec0b6d3a943646ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:56.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.546+0000 7fd762a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:46:56.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.546+0000 7fd762a00640 1 -- 172.21.15.89:0/1673794454 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768049070 con 0x55b4b87f6cd0 2024-09-10T08:46:56.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.547+0000 7fd762a00640 1 -- 172.21.15.89:0/1673794454 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fd7680b8280 con 0x55b4b87f6cd0 2024-09-10T08:46:56.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.547+0000 7fd7863fd740 1 -- 172.21.15.89:0/1673794454 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd774004050 con 0x55b4b87f6cd0 2024-09-10T08:46:56.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.547+0000 7fd762a00640 1 -- 172.21.15.89:0/1673794454 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd7680d3050 con 0x55b4b87f6cd0 2024-09-10T08:46:56.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.547+0000 7fd7863fd740 10 client.26696 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:56.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.547+0000 7fd7863fd740 1 -- 172.21.15.89:0/1673794454 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b4b880fdd0 con 0x55b4b87f6cd0 2024-09-10T08:46:56.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.547+0000 7fd762a00640 1 -- 172.21.15.89:0/1673794454 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd774004050 con 0x55b4b87f6cd0 2024-09-10T08:46:56.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.547+0000 7fd762a00640 1 -- 172.21.15.89:0/1673794454 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fd7680ef370 con 0x55b4b87f6cd0 2024-09-10T08:46:56.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.547+0000 7fd7863fd740 10 client.26696 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:56.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.547+0000 7fd7863fd740 10 client.26696 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:46:56.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.547+0000 7fd7863fd740 20 client.26696 populate_metadata read hostname 'smithi089' 2024-09-10T08:46:56.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.547+0000 7fd7863fd740 1 --2- 172.21.15.89:0/1673794454 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b4b8811280 0x55b4b88136e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:56.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.547+0000 7fd7863fd740 4 client.26696 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:56.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.548+0000 7fd7863fd740 1 -- 172.21.15.89:0/1673794454 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4b86ff310 con 0x55b4b8811280 2024-09-10T08:46:56.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.548+0000 7fd781400640 1 --2- 172.21.15.89:0/1673794454 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b4b8811280 0x55b4b88136e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:56.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.548+0000 7fd781400640 1 --2- 172.21.15.89:0/1673794454 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b4b8811280 0x55b4b88136e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b93000af183c8abb server_cookie=9ed98ef43f0cdea9 in_seq=0 out_seq=0 2024-09-10T08:46:56.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.548+0000 7fd762a00640 10 client.26696 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:56.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.550+0000 7fd762a00640 1 -- 172.21.15.89:0/1673794454 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4b86ff310 con 0x55b4b8811280 2024-09-10T08:46:56.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.550+0000 7fd762a00640 10 client.26696 handle_command_reply: tid=0 2024-09-10T08:46:56.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.550+0000 7fd7863fd740 1 client.26696 shutdown 2024-09-10T08:46:56.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.550+0000 7fd7863fd740 1 -- 172.21.15.89:0/1673794454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b87f6cd0 msgr2=0x55b4b87fec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:56.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.550+0000 7fd7863fd740 1 --2- 172.21.15.89:0/1673794454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b87f6cd0 0x55b4b87fec50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd7680c42a0 tx=0x7fd7680f3040 comp rx=0 tx=0).stop 2024-09-10T08:46:56.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.550+0000 7fd7863fd740 1 -- 172.21.15.89:0/1673794454 shutdown_connections 2024-09-10T08:46:56.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.550+0000 7fd7863fd740 1 --2- 172.21.15.89:0/1673794454 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b4b8811280 0x55b4b88136e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.550+0000 7fd7863fd740 1 --2- 172.21.15.89:0/1673794454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b8801590 0x55b4b87ff190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.550+0000 7fd7863fd740 1 --2- 172.21.15.89:0/1673794454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b87f6cd0 0x55b4b87fec50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.550+0000 7fd7863fd740 1 --2- 172.21.15.89:0/1673794454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f63b0 0x55b4b87fe710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.550+0000 7fd762a00640 0 client.26696 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:56.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.550+0000 7fd7863fd740 1 -- 172.21.15.89:0/1673794454 >> 172.21.15.89:0/1673794454 conn(0x55b4b87f2940 msgr2=0x55b4b87f1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:56.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.550+0000 7fd7863fd740 1 -- 172.21.15.89:0/1673794454 shutdown_connections 2024-09-10T08:46:56.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.550+0000 7fd7863fd740 1 -- 172.21.15.89:0/1673794454 wait complete. 2024-09-10T08:46:56.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.550+0000 7fd7863fd740 20 client.26696 trim_cache size 0 max 16384 2024-09-10T08:46:56.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.570+0000 7fd7863fd740 1 Processor -- start 2024-09-10T08:46:56.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.570+0000 7fd7863fd740 1 -- start start 2024-09-10T08:46:56.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.570+0000 7fd7863fd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f63b0 0x55b4b8818fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:56.669 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.570+0000 7fd7863fd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b87f6cd0 0x55b4b88194f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:56.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.570+0000 7fd7863fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b8801590 0x55b4b87fb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:56.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.570+0000 7fd7863fd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1b3890 con 0x55b4b87f6cd0 2024-09-10T08:46:56.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.570+0000 7fd7863fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1b1160 con 0x55b4b8801590 2024-09-10T08:46:56.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.570+0000 7fd7863fd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1aefb0 con 0x55b4b87f63b0 2024-09-10T08:46:56.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.571+0000 7fd781e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b8801590 0x55b4b87fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:56.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.571+0000 7fd781400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f63b0 0x55b4b8818fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:56.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.571+0000 7fd781e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b8801590 0x55b4b87fb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50588/0 (socket says 172.21.15.89:50588) 2024-09-10T08:46:56.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.571+0000 7fd780a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b87f6cd0 0x55b4b88194f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:56.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.571+0000 7fd781e00640 1 -- 172.21.15.89:0/267265536 learned_addr learned my addr 172.21.15.89:0/267265536 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:56.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.571+0000 7fd781400640 1 -- 172.21.15.89:0/267265536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b8801590 msgr2=0x55b4b87fb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:56.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.571+0000 7fd781400640 1 --2- 172.21.15.89:0/267265536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b8801590 0x55b4b87fb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.571+0000 7fd781400640 1 -- 172.21.15.89:0/267265536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b87f6cd0 msgr2=0x55b4b88194f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:56.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.571+0000 7fd781400640 1 --2- 172.21.15.89:0/267265536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b87f6cd0 0x55b4b88194f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.571+0000 7fd781400640 1 -- 172.21.15.89:0/267265536 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4b8803cd0 con 0x55b4b87f63b0 2024-09-10T08:46:56.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.571+0000 7fd781400640 1 --2- 172.21.15.89:0/267265536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f63b0 0x55b4b8818fb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55b4b8811660 tx=0x7fd770016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ced764c002d4c428 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:56.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.572+0000 7fd76dc00640 1 -- 172.21.15.89:0/267265536 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770005220 con 0x55b4b87f63b0 2024-09-10T08:46:56.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.572+0000 7fd76dc00640 1 -- 172.21.15.89:0/267265536 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd770004c80 con 0x55b4b87f63b0 2024-09-10T08:46:56.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.572+0000 7fd76dc00640 1 -- 172.21.15.89:0/267265536 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770003970 con 0x55b4b87f63b0 2024-09-10T08:46:56.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.573+0000 7fd7863fd740 1 -- 172.21.15.89:0/267265536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f63b0 msgr2=0x55b4b8818fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:56.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.573+0000 7fd7863fd740 1 --2- 172.21.15.89:0/267265536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f63b0 0x55b4b8818fb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55b4b8811660 tx=0x7fd770016000 comp rx=0 tx=0).stop 2024-09-10T08:46:56.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.573+0000 7fd7863fd740 1 -- 172.21.15.89:0/267265536 shutdown_connections 2024-09-10T08:46:56.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.573+0000 7fd7863fd740 1 --2- 172.21.15.89:0/267265536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b8801590 0x55b4b87fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.573+0000 7fd7863fd740 1 --2- 172.21.15.89:0/267265536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b87f6cd0 0x55b4b88194f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.573+0000 7fd7863fd740 1 --2- 172.21.15.89:0/267265536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f63b0 0x55b4b8818fb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.573+0000 7fd7863fd740 1 -- 172.21.15.89:0/267265536 >> 172.21.15.89:0/267265536 conn(0x55b4b87f2940 msgr2=0x55b4b87f1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:56.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.574+0000 7fd7863fd740 1 -- 172.21.15.89:0/267265536 shutdown_connections 2024-09-10T08:46:56.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.574+0000 7fd7863fd740 1 -- 172.21.15.89:0/267265536 wait complete. 2024-09-10T08:46:56.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.575+0000 7fd7863fd740 1 -- start start 2024-09-10T08:46:56.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.575+0000 7fd7863fd740 1 Processor -- start 2024-09-10T08:46:56.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.575+0000 7fd7863fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b87f63b0 0x55b4b8809680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:56.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.575+0000 7fd7863fd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f6cd0 0x55b4b8809bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:56.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.575+0000 7fd7863fd740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b8801590 0x55b4b8806490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:56.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.575+0000 7fd7863fd740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1af620 con 0x55b4b8801590 2024-09-10T08:46:56.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.575+0000 7fd7863fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1b0a90 con 0x55b4b87f63b0 2024-09-10T08:46:56.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.575+0000 7fd7863fd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1afcf0 con 0x55b4b87f6cd0 2024-09-10T08:46:56.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.575+0000 7fd7863fd740 10 client.0 _ll_register_callbacks cb 0x7fd7680f5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:56.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.576+0000 7fd781400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b87f63b0 0x55b4b8809680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:56.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.576+0000 7fd780a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f6cd0 0x55b4b8809bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:56.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.576+0000 7fd781400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b87f63b0 0x55b4b8809680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:50602/0 (socket says 172.21.15.89:50602) 2024-09-10T08:46:56.673 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.576+0000 7fd781400640 1 -- 172.21.15.89:0/1423527487 learned_addr learned my addr 172.21.15.89:0/1423527487 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:46:56.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.576+0000 7fd781e00640 1 --2- 172.21.15.89:0/1423527487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b8801590 0x55b4b8806490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:56.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.576+0000 7fd780a00640 1 -- 172.21.15.89:0/1423527487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b87f63b0 msgr2=0x55b4b8809680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:56.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.576+0000 7fd780a00640 1 --2- 172.21.15.89:0/1423527487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b87f63b0 0x55b4b8809680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.576+0000 7fd780a00640 1 -- 172.21.15.89:0/1423527487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b8801590 msgr2=0x55b4b8806490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:56.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.576+0000 7fd780a00640 1 --2- 172.21.15.89:0/1423527487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b8801590 0x55b4b8806490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:56.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.576+0000 7fd780a00640 1 -- 172.21.15.89:0/1423527487 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4b8806c90 con 0x55b4b87f6cd0 2024-09-10T08:46:56.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.576+0000 7fd780a00640 1 --2- 172.21.15.89:0/1423527487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f6cd0 0x55b4b8809bc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd768000c00 tx=0x7fd7680b8510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dace92eaf4ee517 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:56.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.576+0000 7fd762a00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:46:56.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.577+0000 7fd762a00640 1 -- 172.21.15.89:0/1423527487 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768049040 con 0x55b4b87f6cd0 2024-09-10T08:46:56.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.577+0000 7fd762a00640 1 -- 172.21.15.89:0/1423527487 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fd768031bd0 con 0x55b4b87f6cd0 2024-09-10T08:46:56.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.577+0000 7fd7863fd740 1 -- 172.21.15.89:0/1423527487 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd770004c80 con 0x55b4b87f6cd0 2024-09-10T08:46:56.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.577+0000 7fd762a00640 1 -- 172.21.15.89:0/1423527487 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd7680bb570 con 0x55b4b87f6cd0 2024-09-10T08:46:56.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.577+0000 7fd7863fd740 10 client.26644 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:56.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.577+0000 7fd7863fd740 1 -- 172.21.15.89:0/1423527487 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b4b8806c90 con 0x55b4b87f6cd0 2024-09-10T08:46:56.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.577+0000 7fd762a00640 1 -- 172.21.15.89:0/1423527487 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd770004c80 con 0x55b4b87f6cd0 2024-09-10T08:46:56.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.577+0000 7fd762a00640 1 -- 172.21.15.89:0/1423527487 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fd7680bb060 con 0x55b4b87f6cd0 2024-09-10T08:46:56.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.577+0000 7fd7863fd740 10 client.26644 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:56.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.577+0000 7fd7863fd740 10 client.26644 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:46:56.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.577+0000 7fd7863fd740 20 client.26644 populate_metadata read hostname 'smithi089' 2024-09-10T08:46:56.675 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.577+0000 7fd7863fd740 1 --2- 172.21.15.89:0/1423527487 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b4b887d030 0x55b4b887f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.577+0000 7fd7863fd740 4 client.26644 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-10T08:46:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.577+0000 7fd7863fd740 1 -- 172.21.15.89:0/1423527487 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x55b4b8769c60 con 0x55b4b887d030 2024-09-10T08:46:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.578+0000 7fd781400640 1 --2- 172.21.15.89:0/1423527487 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b4b887d030 0x55b4b887f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.578+0000 7fd781400640 1 --2- 172.21.15.89:0/1423527487 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b4b887d030 0x55b4b887f450 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59fead9356720a0c server_cookie=fdc7ecf72a10fc7f in_seq=0 out_seq=0 2024-09-10T08:46:56.676 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:46:56.578+0000 7fd762a00640 10 client.26644 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:46:56.913 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:56 smithi089 ceph-mon[30757]: pgmap v603: 129 pgs: 129 active+clean; 489 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 6.3 MiB/s wr, 1.86k op/s 2024-09-10T08:46:57.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:56 smithi111 ceph-mon[35220]: pgmap v603: 129 pgs: 129 active+clean; 489 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 6.3 MiB/s wr, 1.86k op/s 2024-09-10T08:46:57.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:56 smithi149 ceph-mon[36167]: pgmap v603: 129 pgs: 129 active+clean; 489 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 6.3 MiB/s wr, 1.86k op/s 2024-09-10T08:46:59.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:58 smithi149 ceph-mon[36167]: pgmap v604: 129 pgs: 129 active+clean; 449 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.9 MiB/s wr, 2.48k op/s 2024-09-10T08:46:59.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:58 smithi111 ceph-mon[35220]: pgmap v604: 129 pgs: 129 active+clean; 449 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.9 MiB/s wr, 2.48k op/s 2024-09-10T08:46:59.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:58 smithi089 ceph-mon[30757]: pgmap v604: 129 pgs: 129 active+clean; 449 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.9 MiB/s wr, 2.48k op/s 2024-09-10T08:46:59.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:58 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-a[30753]: 2024-09-10T08:46:58.833+0000 7f4551e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-10T08:47:00.197 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:59 smithi111 ceph-mon[35220]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-10T08:47:00.197 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:59 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] up:active 2024-09-10T08:47:00.197 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:46:59 smithi111 ceph-mon[35220]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:47:00.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:59 smithi149 ceph-mon[36167]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-10T08:47:00.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:59 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] up:active 2024-09-10T08:47:00.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:46:59 smithi149 ceph-mon[36167]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:47:00.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:59 smithi089 ceph-mon[30757]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-10T08:47:00.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:59 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] up:active 2024-09-10T08:47:00.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:46:59 smithi089 ceph-mon[30757]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:47:01.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:00 smithi149 ceph-mon[36167]: scrub summary: idle+waiting paths [/] 2024-09-10T08:47:01.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:00 smithi149 ceph-mon[36167]: pgmap v605: 129 pgs: 129 active+clean; 450 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 2.24k op/s 2024-09-10T08:47:01.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:00 smithi149 ceph-mon[36167]: scrub summary: idle 2024-09-10T08:47:01.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:00 smithi149 ceph-mon[36167]: scrub summary: idle 2024-09-10T08:47:01.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:00 smithi111 ceph-mon[35220]: scrub summary: idle+waiting paths [/] 2024-09-10T08:47:01.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:00 smithi111 ceph-mon[35220]: pgmap v605: 129 pgs: 129 active+clean; 450 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 2.24k op/s 2024-09-10T08:47:01.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:00 smithi111 ceph-mon[35220]: scrub summary: idle 2024-09-10T08:47:01.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:00 smithi111 ceph-mon[35220]: scrub summary: idle 2024-09-10T08:47:01.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:00 smithi089 ceph-mon[30757]: scrub summary: idle+waiting paths [/] 2024-09-10T08:47:01.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:00 smithi089 ceph-mon[30757]: pgmap v605: 129 pgs: 129 active+clean; 450 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 2.24k op/s 2024-09-10T08:47:01.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:00 smithi089 ceph-mon[30757]: scrub summary: idle 2024-09-10T08:47:01.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:00 smithi089 ceph-mon[30757]: scrub summary: idle 2024-09-10T08:47:02.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:01 smithi149 ceph-mon[36167]: scrub summary: idle 2024-09-10T08:47:02.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:01 smithi111 ceph-mon[35220]: scrub summary: idle 2024-09-10T08:47:02.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:01 smithi089 ceph-mon[30757]: scrub summary: idle 2024-09-10T08:47:03.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:02 smithi149 ceph-mon[36167]: pgmap v606: 129 pgs: 129 active+clean; 447 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 3.34k op/s 2024-09-10T08:47:03.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:02 smithi111 ceph-mon[35220]: pgmap v606: 129 pgs: 129 active+clean; 447 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 3.34k op/s 2024-09-10T08:47:03.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:02 smithi089 ceph-mon[30757]: pgmap v606: 129 pgs: 129 active+clean; 447 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 3.34k op/s 2024-09-10T08:47:05.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:04 smithi149 ceph-mon[36167]: pgmap v607: 129 pgs: 129 active+clean; 446 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 2.72k op/s 2024-09-10T08:47:05.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:04 smithi111 ceph-mon[35220]: pgmap v607: 129 pgs: 129 active+clean; 446 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 2.72k op/s 2024-09-10T08:47:05.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:04 smithi089 ceph-mon[30757]: pgmap v607: 129 pgs: 129 active+clean; 446 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 2.72k op/s 2024-09-10T08:47:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.636+0000 7f6101800640 1 -- 172.21.15.89:0/1865884331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60fc1036f0 msgr2=0x7f60fc109f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.636+0000 7f6101800640 1 --2- 172.21.15.89:0/1865884331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60fc1036f0 0x7f60fc109f80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f60f4009be0 tx=0x7f60f402a6e0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.637+0000 7f6101800640 1 -- 172.21.15.89:0/1865884331 shutdown_connections 2024-09-10T08:47:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.637+0000 7f6101800640 1 --2- 172.21.15.89:0/1865884331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60fc1036f0 0x7f60fc109f80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.637+0000 7f6101800640 1 --2- 172.21.15.89:0/1865884331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60fc102d30 0x7f60fc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.637+0000 7f6101800640 1 --2- 172.21.15.89:0/1865884331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60fc101b40 0x7f60fc101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.637+0000 7f6101800640 1 -- 172.21.15.89:0/1865884331 >> 172.21.15.89:0/1865884331 conn(0x7f60fc0fd2e0 msgr2=0x7f60fc0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.637+0000 7f6101800640 1 -- 172.21.15.89:0/1865884331 shutdown_connections 2024-09-10T08:47:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.638+0000 7f6101800640 1 -- 172.21.15.89:0/1865884331 wait complete. 2024-09-10T08:47:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.639+0000 7f6101800640 1 Processor -- start 2024-09-10T08:47:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.640+0000 7f6101800640 1 -- start start 2024-09-10T08:47:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.640+0000 7f6101800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60fc101b40 0x7f60fc19e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.640+0000 7f6101800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60fc102d30 0x7f60fc19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.640+0000 7f6101800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60fc1036f0 0x7f60fc1a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.641+0000 7f60fb400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60fc1036f0 0x7f60fc1a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.641+0000 7f60fa000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60fc102d30 0x7f60fc19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.641+0000 7f60faa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60fc101b40 0x7f60fc19e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.641+0000 7f6101800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc10b630 con 0x7f60fc1036f0 2024-09-10T08:47:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.641+0000 7f6101800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc10b4b0 con 0x7f60fc101b40 2024-09-10T08:47:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.641+0000 7f6101800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc10b7b0 con 0x7f60fc102d30 2024-09-10T08:47:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.641+0000 7f60faa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60fc101b40 0x7f60fc19e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:39764/0 (socket says 172.21.15.89:39764) 2024-09-10T08:47:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.641+0000 7f60faa00640 1 -- 172.21.15.89:0/3945148791 learned_addr learned my addr 172.21.15.89:0/3945148791 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.642+0000 7f60fa000640 1 -- 172.21.15.89:0/3945148791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60fc101b40 msgr2=0x7f60fc19e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.642+0000 7f60fa000640 1 --2- 172.21.15.89:0/3945148791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60fc101b40 0x7f60fc19e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.642+0000 7f60fa000640 1 -- 172.21.15.89:0/3945148791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60fc1036f0 msgr2=0x7f60fc1a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.642+0000 7f60fa000640 1 --2- 172.21.15.89:0/3945148791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60fc1036f0 0x7f60fc1a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.642+0000 7f60fa000640 1 -- 172.21.15.89:0/3945148791 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc1a5fc0 con 0x7f60fc102d30 2024-09-10T08:47:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.642+0000 7f60faa00640 1 --2- 172.21.15.89:0/3945148791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60fc101b40 0x7f60fc19e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.642+0000 7f60fa000640 1 --2- 172.21.15.89:0/3945148791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60fc102d30 0x7f60fc19e8e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f60e8005fd0 tx=0x7f60e800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.643+0000 7f60f3400640 1 -- 172.21.15.89:0/3945148791 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e800cd00 con 0x7f60fc102d30 2024-09-10T08:47:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.643+0000 7f60f3400640 1 -- 172.21.15.89:0/3945148791 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e80035f0 con 0x7f60fc102d30 2024-09-10T08:47:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.643+0000 7f60f3400640 1 -- 172.21.15.89:0/3945148791 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8010690 con 0x7f60fc102d30 2024-09-10T08:47:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.643+0000 7f6101800640 1 -- 172.21.15.89:0/3945148791 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60fc1a62b0 con 0x7f60fc102d30 2024-09-10T08:47:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.644+0000 7f6101800640 1 -- 172.21.15.89:0/3945148791 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60fc1a6850 con 0x7f60fc102d30 2024-09-10T08:47:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.650+0000 7f61045f7740 1 Processor -- start 2024-09-10T08:47:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.651+0000 7f60f3400640 1 -- 172.21.15.89:0/3945148791 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f60e8004020 con 0x7f60fc102d30 2024-09-10T08:47:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.652+0000 7f61045f7740 1 -- start start 2024-09-10T08:47:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.653+0000 7f61045f7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 0x56194012a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.653+0000 7f61045f7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012a9d0 0x56194012f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.653+0000 7f60fb400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 0x56194012a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.653+0000 7f60fb400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 0x56194012a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:46520/0 (socket says 172.21.15.89:46520) 2024-09-10T08:47:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.654+0000 7f60fb400640 1 -- 172.21.15.89:0/2144271013 learned_addr learned my addr 172.21.15.89:0/2144271013 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.654+0000 7f61045f7740 1 --2- 172.21.15.89:0/2144271013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012fa60 0x561940131f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.654+0000 7f61045f7740 1 -- 172.21.15.89:0/2144271013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56194008f100 con 0x56194012a090 2024-09-10T08:47:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.654+0000 7f61045f7740 1 -- 172.21.15.89:0/2144271013 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5619400371b0 con 0x56194012fa60 2024-09-10T08:47:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.654+0000 7f61045f7740 1 -- 172.21.15.89:0/2144271013 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56194008b0c0 con 0x56194012a9d0 2024-09-10T08:47:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.654+0000 7f60fa000640 1 --2- 172.21.15.89:0/2144271013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012fa60 0x561940131f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.654+0000 7f60faa00640 1 --2- 172.21.15.89:0/2144271013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012a9d0 0x56194012f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.655+0000 7f60fb400640 1 -- 172.21.15.89:0/2144271013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012a9d0 msgr2=0x56194012f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.655+0000 7f60fb400640 1 --2- 172.21.15.89:0/2144271013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012a9d0 0x56194012f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.655+0000 7f60fb400640 1 -- 172.21.15.89:0/2144271013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012fa60 msgr2=0x561940131f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.655+0000 7f60fb400640 1 --2- 172.21.15.89:0/2144271013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012fa60 0x561940131f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.655+0000 7f60fb400640 1 -- 172.21.15.89:0/2144271013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56193fda2210 con 0x56194012a090 2024-09-10T08:47:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.655+0000 7f60fa000640 1 --2- 172.21.15.89:0/2144271013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012fa60 0x561940131f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.655+0000 7f60f3400640 1 --2- 172.21.15.89:0/3945148791 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f60e00a2cd0 0x7f60e00a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.656+0000 7f60f3400640 1 -- 172.21.15.89:0/3945148791 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f60e80cf770 con 0x7f60fc102d30 2024-09-10T08:47:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.656+0000 7f60faa00640 1 --2- 172.21.15.89:0/3945148791 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f60e00a2cd0 0x7f60e00a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.657+0000 7f60fb400640 1 --2- 172.21.15.89:0/2144271013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 0x56194012a490 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f60f4006010 tx=0x7f60f4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa258fdc077cfc51 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.657+0000 7f60e7e00640 1 -- 172.21.15.89:0/2144271013 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f4040030 con 0x56194012a090 2024-09-10T08:47:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.658+0000 7f60faa00640 1 --2- 172.21.15.89:0/3945148791 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f60e00a2cd0 0x7f60e00a5190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x56194012b400 tx=0x7f60ec009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.658+0000 7f60e7e00640 1 -- 172.21.15.89:0/2144271013 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60f40303a0 con 0x56194012a090 2024-09-10T08:47:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.659+0000 7f61045f7740 1 -- 172.21.15.89:0/2144271013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 msgr2=0x56194012a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.659+0000 7f61045f7740 1 --2- 172.21.15.89:0/2144271013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 0x56194012a490 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f60f4006010 tx=0x7f60f4009310 comp rx=0 tx=0).stop 2024-09-10T08:47:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.659+0000 7f61045f7740 1 -- 172.21.15.89:0/2144271013 shutdown_connections 2024-09-10T08:47:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.659+0000 7f61045f7740 1 --2- 172.21.15.89:0/2144271013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012fa60 0x561940131f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.659+0000 7f61045f7740 1 --2- 172.21.15.89:0/2144271013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012a9d0 0x56194012f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.659+0000 7f61045f7740 1 --2- 172.21.15.89:0/2144271013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 0x56194012a490 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.659+0000 7f61045f7740 1 -- 172.21.15.89:0/2144271013 >> 172.21.15.89:0/2144271013 conn(0x561940125700 msgr2=0x561940125b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.660+0000 7f61045f7740 1 -- 172.21.15.89:0/2144271013 shutdown_connections 2024-09-10T08:47:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.660+0000 7f61045f7740 1 -- 172.21.15.89:0/2144271013 wait complete. 2024-09-10T08:47:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.661+0000 7f61045f7740 1 -- start start 2024-09-10T08:47:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.661+0000 7f61045f7740 1 Processor -- start 2024-09-10T08:47:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.664+0000 7f61045f7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 0x5619401341b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.664+0000 7f61045f7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012a9d0 0x5619401346f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.664+0000 7f61045f7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012fa60 0x56194013b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.664+0000 7f61045f7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561940132a70 con 0x56194012a090 2024-09-10T08:47:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.664+0000 7f61045f7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561940018090 con 0x56194012fa60 2024-09-10T08:47:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.664+0000 7f61045f7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561940132bf0 con 0x56194012a9d0 2024-09-10T08:47:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.664+0000 7f61045f7740 10 client.0 _ll_register_callbacks cb 0x56194006cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.664+0000 7f60fa000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012a9d0 0x5619401346f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.664+0000 7f60fa000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012a9d0 0x5619401346f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:60310/0 (socket says 172.21.15.89:60310) 2024-09-10T08:47:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.664+0000 7f60fa000640 1 -- 172.21.15.89:0/729450240 learned_addr learned my addr 172.21.15.89:0/729450240 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.664+0000 7f60fb400640 1 --2- 172.21.15.89:0/729450240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012fa60 0x56194013b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.665+0000 7f60fa000640 1 -- 172.21.15.89:0/729450240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012fa60 msgr2=0x56194013b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.665+0000 7f60fa000640 1 --2- 172.21.15.89:0/729450240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012fa60 0x56194013b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.665+0000 7f60fa000640 1 -- 172.21.15.89:0/729450240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 msgr2=0x5619401341b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.665+0000 7f60fa000640 1 --2- 172.21.15.89:0/729450240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 0x5619401341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.665+0000 7f60fa000640 1 -- 172.21.15.89:0/729450240 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56193fe6e620 con 0x56194012a9d0 2024-09-10T08:47:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.665+0000 7f60fa000640 1 --2- 172.21.15.89:0/729450240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012a9d0 0x5619401346f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f60e809b900 tx=0x7f60e8097f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=326dbb9dbabaccb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.665+0000 7f60e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:47:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.666+0000 7f60e4c00640 1 -- 172.21.15.89:0/729450240 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8009530 con 0x56194012a9d0 2024-09-10T08:47:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.666+0000 7f60e4c00640 1 -- 172.21.15.89:0/729450240 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f60e80037a0 con 0x56194012a9d0 2024-09-10T08:47:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.666+0000 7f61045f7740 1 -- 172.21.15.89:0/729450240 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60f40303a0 con 0x56194012a9d0 2024-09-10T08:47:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.667+0000 7f60e4c00640 1 -- 172.21.15.89:0/729450240 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f60e809c300 con 0x56194012a9d0 2024-09-10T08:47:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.667+0000 7f61045f7740 10 client.26656 fetch_fsmap learned FSMap version 30 2024-09-10T08:47:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.667+0000 7f61045f7740 1 -- 172.21.15.89:0/729450240 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56194013e590 con 0x56194012a9d0 2024-09-10T08:47:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.667+0000 7f60e4c00640 1 -- 172.21.15.89:0/729450240 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60f40303a0 con 0x56194012a9d0 2024-09-10T08:47:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.668+0000 7f60e4c00640 1 -- 172.21.15.89:0/729450240 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f60e809b050 con 0x56194012a9d0 2024-09-10T08:47:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.668+0000 7f61045f7740 10 client.26656 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T08:47:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.668+0000 7f61045f7740 10 client.26656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.668+0000 7f61045f7740 20 client.26656 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.668+0000 7f61045f7740 1 --2- 172.21.15.89:0/729450240 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56194013fcd0 0x5619401420c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.668+0000 7f61045f7740 4 client.26656 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.668+0000 7f61045f7740 1 -- 172.21.15.89:0/729450240 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56194009c830 con 0x56194013fcd0 2024-09-10T08:47:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.670+0000 7f60faa00640 1 --2- 172.21.15.89:0/729450240 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56194013fcd0 0x5619401420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.671+0000 7f60faa00640 1 --2- 172.21.15.89:0/729450240 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56194013fcd0 0x5619401420c0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce5e4f4469d65b79 server_cookie=ff39c5bb56e5f2a2 in_seq=0 out_seq=0 2024-09-10T08:47:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.671+0000 7f60e4c00640 10 client.26656 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.676+0000 7f60e4c00640 1 -- 172.21.15.89:0/729450240 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56194009c830 con 0x56194013fcd0 2024-09-10T08:47:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.676+0000 7f60e4c00640 10 client.26656 handle_command_reply: tid=0 2024-09-10T08:47:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.677+0000 7f61045f7740 1 client.26656 shutdown 2024-09-10T08:47:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.677+0000 7f61045f7740 1 -- 172.21.15.89:0/729450240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012a9d0 msgr2=0x5619401346f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.677+0000 7f61045f7740 1 --2- 172.21.15.89:0/729450240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012a9d0 0x5619401346f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f60e809b900 tx=0x7f60e8097f60 comp rx=0 tx=0).stop 2024-09-10T08:47:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.677+0000 7f61045f7740 1 -- 172.21.15.89:0/729450240 shutdown_connections 2024-09-10T08:47:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.677+0000 7f61045f7740 1 --2- 172.21.15.89:0/729450240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012fa60 0x56194013b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.677+0000 7f61045f7740 1 --2- 172.21.15.89:0/729450240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012a9d0 0x5619401346f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.677+0000 7f61045f7740 1 --2- 172.21.15.89:0/729450240 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x56194013fcd0 0x5619401420c0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.677+0000 7f61045f7740 1 --2- 172.21.15.89:0/729450240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 0x5619401341b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.677+0000 7f61045f7740 1 -- 172.21.15.89:0/729450240 >> 172.21.15.89:0/729450240 conn(0x561940125700 msgr2=0x561940126f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.677+0000 7f60e4c00640 0 client.26656 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.678+0000 7f61045f7740 1 -- 172.21.15.89:0/729450240 shutdown_connections 2024-09-10T08:47:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.678+0000 7f61045f7740 1 -- 172.21.15.89:0/729450240 wait complete. 2024-09-10T08:47:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.679+0000 7f61045f7740 20 client.26656 trim_cache size 0 max 16384 2024-09-10T08:47:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.704+0000 7f61045f7740 1 Processor -- start 2024-09-10T08:47:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.704+0000 7f61045f7740 1 -- start start 2024-09-10T08:47:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.704+0000 7f61045f7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 0x5619401336f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.704+0000 7f61045f7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012a9d0 0x56194013b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.704+0000 7f61045f7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012fa60 0x5619401265b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.704+0000 7f61045f7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc1ab030 con 0x56194012a090 2024-09-10T08:47:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.704+0000 7f61045f7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc1a8900 con 0x56194012a9d0 2024-09-10T08:47:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.704+0000 7f61045f7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc1990b0 con 0x56194012fa60 2024-09-10T08:47:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.705+0000 7f60faa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 0x5619401336f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.705+0000 7f60fb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012fa60 0x5619401265b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.706+0000 7f60fb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012fa60 0x5619401265b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:60324/0 (socket says 172.21.15.89:60324) 2024-09-10T08:47:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.706+0000 7f60fb400640 1 -- 172.21.15.89:0/4004711255 learned_addr learned my addr 172.21.15.89:0/4004711255 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.706+0000 7f60fa000640 1 --2- 172.21.15.89:0/4004711255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012a9d0 0x56194013b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.706+0000 7f60fb400640 1 -- 172.21.15.89:0/4004711255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012a9d0 msgr2=0x56194013b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.706+0000 7f60fb400640 1 --2- 172.21.15.89:0/4004711255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012a9d0 0x56194013b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.706+0000 7f60fb400640 1 -- 172.21.15.89:0/4004711255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 msgr2=0x5619401336f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.706+0000 7f60fb400640 1 --2- 172.21.15.89:0/4004711255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 0x5619401336f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.706+0000 7f60fb400640 1 -- 172.21.15.89:0/4004711255 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561940142740 con 0x56194012fa60 2024-09-10T08:47:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.706+0000 7f60faa00640 1 --2- 172.21.15.89:0/4004711255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 0x5619401336f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.706+0000 7f60fa000640 1 --2- 172.21.15.89:0/4004711255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012a9d0 0x56194013b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.706+0000 7f60fb400640 1 --2- 172.21.15.89:0/4004711255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012fa60 0x5619401265b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f60f4009be0 tx=0x7f60f402a6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e47108fce0498af2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.707+0000 7f60e7400640 1 -- 172.21.15.89:0/4004711255 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f4053070 con 0x56194012fa60 2024-09-10T08:47:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.707+0000 7f60e7400640 1 -- 172.21.15.89:0/4004711255 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60f40303a0 con 0x56194012fa60 2024-09-10T08:47:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.707+0000 7f60e7400640 1 -- 172.21.15.89:0/4004711255 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f4036040 con 0x56194012fa60 2024-09-10T08:47:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.708+0000 7f61045f7740 1 -- 172.21.15.89:0/4004711255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012fa60 msgr2=0x5619401265b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.708+0000 7f61045f7740 1 --2- 172.21.15.89:0/4004711255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012fa60 0x5619401265b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f60f4009be0 tx=0x7f60f402a6e0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.708+0000 7f61045f7740 1 -- 172.21.15.89:0/4004711255 shutdown_connections 2024-09-10T08:47:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.708+0000 7f61045f7740 1 --2- 172.21.15.89:0/4004711255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012fa60 0x5619401265b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.709+0000 7f61045f7740 1 --2- 172.21.15.89:0/4004711255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012a9d0 0x56194013b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.709+0000 7f61045f7740 1 --2- 172.21.15.89:0/4004711255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 0x5619401336f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.709+0000 7f61045f7740 1 -- 172.21.15.89:0/4004711255 >> 172.21.15.89:0/4004711255 conn(0x561940125700 msgr2=0x561940132fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.709+0000 7f61045f7740 1 -- 172.21.15.89:0/4004711255 shutdown_connections 2024-09-10T08:47:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.709+0000 7f61045f7740 1 -- 172.21.15.89:0/4004711255 wait complete. 2024-09-10T08:47:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.710+0000 7f61045f7740 1 -- start start 2024-09-10T08:47:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.710+0000 7f61045f7740 1 Processor -- start 2024-09-10T08:47:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.710+0000 7f61045f7740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 0x561940138b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.711+0000 7f61045f7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012a9d0 0x561940139080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.711+0000 7f61045f7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012fa60 0x56194013a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.711+0000 7f61045f7740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc1a6dc0 con 0x56194012a090 2024-09-10T08:47:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.711+0000 7f61045f7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc1a8230 con 0x56194012a9d0 2024-09-10T08:47:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.711+0000 7f61045f7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc1a7490 con 0x56194012fa60 2024-09-10T08:47:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.711+0000 7f61045f7740 10 client.0 _ll_register_callbacks cb 0x7f60e800cea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.711+0000 7f60fb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012fa60 0x56194013a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.711+0000 7f60fb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012fa60 0x56194013a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:60332/0 (socket says 172.21.15.89:60332) 2024-09-10T08:47:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.711+0000 7f60fb400640 1 -- 172.21.15.89:0/3861757649 learned_addr learned my addr 172.21.15.89:0/3861757649 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.711+0000 7f60fa000640 1 --2- 172.21.15.89:0/3861757649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012a9d0 0x561940139080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.711+0000 7f60faa00640 1 --2- 172.21.15.89:0/3861757649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 0x561940138b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.712+0000 7f60fb400640 1 -- 172.21.15.89:0/3861757649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012a9d0 msgr2=0x561940139080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.712+0000 7f60fb400640 1 --2- 172.21.15.89:0/3861757649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012a9d0 0x561940139080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.712+0000 7f60fb400640 1 -- 172.21.15.89:0/3861757649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 msgr2=0x561940138b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.712+0000 7f60fb400640 1 --2- 172.21.15.89:0/3861757649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 0x561940138b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.712+0000 7f60fb400640 1 -- 172.21.15.89:0/3861757649 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56194013a6a0 con 0x56194012fa60 2024-09-10T08:47:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.712+0000 7f60faa00640 1 --2- 172.21.15.89:0/3861757649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 0x561940138b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.712+0000 7f60fb400640 1 --2- 172.21.15.89:0/3861757649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012fa60 0x56194013a090 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f60f4051040 tx=0x7f60f4009870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a0bb34a04210dc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.712+0000 7f60e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:47:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.712+0000 7f60e4c00640 1 -- 172.21.15.89:0/3861757649 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f4036070 con 0x56194012fa60 2024-09-10T08:47:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.713+0000 7f60e4c00640 1 -- 172.21.15.89:0/3861757649 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f60f4030b70 con 0x56194012fa60 2024-09-10T08:47:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.713+0000 7f61045f7740 1 -- 172.21.15.89:0/3861757649 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60f40303a0 con 0x56194012fa60 2024-09-10T08:47:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.713+0000 7f60e4c00640 1 -- 172.21.15.89:0/3861757649 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f60f4053070 con 0x56194012fa60 2024-09-10T08:47:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.713+0000 7f61045f7740 10 client.26662 fetch_fsmap learned FSMap version 30 2024-09-10T08:47:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.713+0000 7f61045f7740 1 -- 172.21.15.89:0/3861757649 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56194013b920 con 0x56194012fa60 2024-09-10T08:47:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.713+0000 7f60e4c00640 1 -- 172.21.15.89:0/3861757649 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60f40303a0 con 0x56194012fa60 2024-09-10T08:47:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.714+0000 7f60e4c00640 1 -- 172.21.15.89:0/3861757649 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f60f40594c0 con 0x56194012fa60 2024-09-10T08:47:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.714+0000 7f61045f7740 10 client.26662 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T08:47:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.714+0000 7f61045f7740 10 client.26662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.714+0000 7f61045f7740 20 client.26662 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.714+0000 7f61045f7740 1 --2- 172.21.15.89:0/3861757649 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5619401aabb0 0x5619401acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.714+0000 7f61045f7740 4 client.26662 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:47:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.714+0000 7f61045f7740 1 -- 172.21.15.89:0/3861757649 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561940096b80 con 0x5619401aabb0 2024-09-10T08:47:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.714+0000 7f60faa00640 1 --2- 172.21.15.89:0/3861757649 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5619401aabb0 0x5619401acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.715+0000 7f60faa00640 1 --2- 172.21.15.89:0/3861757649 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5619401aabb0 0x5619401acfd0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4aff442a4dd58d7 server_cookie=a6a6de250febddd4 in_seq=0 out_seq=0 2024-09-10T08:47:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:06.715+0000 7f60e4c00640 10 client.26662 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:07.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:06 smithi149 ceph-mon[36167]: pgmap v608: 129 pgs: 129 active+clean; 448 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.6 MiB/s wr, 2.28k op/s 2024-09-10T08:47:07.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:06 smithi111 ceph-mon[35220]: pgmap v608: 129 pgs: 129 active+clean; 448 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.6 MiB/s wr, 2.28k op/s 2024-09-10T08:47:07.305 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:06 smithi089 ceph-mon[30757]: pgmap v608: 129 pgs: 129 active+clean; 448 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.6 MiB/s wr, 2.28k op/s 2024-09-10T08:47:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.771+0000 7f60e4c00640 1 -- 172.21.15.89:0/3861757649 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561940096b80 con 0x5619401aabb0 2024-09-10T08:47:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.771+0000 7f60e4c00640 10 client.26662 handle_command_reply: tid=0 2024-09-10T08:47:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.771+0000 7f61045f7740 1 client.26662 shutdown 2024-09-10T08:47:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.771+0000 7f61045f7740 1 -- 172.21.15.89:0/3861757649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012fa60 msgr2=0x56194013a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.771+0000 7f61045f7740 1 --2- 172.21.15.89:0/3861757649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012fa60 0x56194013a090 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f60f4051040 tx=0x7f60f4009870 comp rx=0 tx=0).stop 2024-09-10T08:47:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.771+0000 7f61045f7740 1 -- 172.21.15.89:0/3861757649 shutdown_connections 2024-09-10T08:47:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.771+0000 7f61045f7740 1 --2- 172.21.15.89:0/3861757649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56194012fa60 0x56194013a090 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.771+0000 7f61045f7740 1 --2- 172.21.15.89:0/3861757649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56194012a9d0 0x561940139080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.771+0000 7f61045f7740 1 --2- 172.21.15.89:0/3861757649 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5619401aabb0 0x5619401acfd0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.771+0000 7f61045f7740 1 --2- 172.21.15.89:0/3861757649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56194012a090 0x561940138b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.771+0000 7f60e4c00640 0 client.26662 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.771+0000 7f61045f7740 1 -- 172.21.15.89:0/3861757649 >> 172.21.15.89:0/3861757649 conn(0x561940125700 msgr2=0x561940124880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.771+0000 7f61045f7740 1 -- 172.21.15.89:0/3861757649 shutdown_connections 2024-09-10T08:47:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.771+0000 7f61045f7740 1 -- 172.21.15.89:0/3861757649 wait complete. 2024-09-10T08:47:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.771+0000 7f61045f7740 20 client.26662 trim_cache size 0 max 16384 2024-09-10T08:47:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:47:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:47:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:47:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:47:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.772+0000 7f6101800640 1 -- 172.21.15.89:0/3945148791 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f60e00a2cd0 msgr2=0x7f60e00a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.772+0000 7f6101800640 1 --2- 172.21.15.89:0/3945148791 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f60e00a2cd0 0x7f60e00a5190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x56194012b400 tx=0x7f60ec009560 comp rx=0 tx=0).stop 2024-09-10T08:47:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.772+0000 7f6101800640 1 -- 172.21.15.89:0/3945148791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60fc102d30 msgr2=0x7f60fc19e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.772+0000 7f6101800640 1 --2- 172.21.15.89:0/3945148791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60fc102d30 0x7f60fc19e8e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f60e8005fd0 tx=0x7f60e800ee50 comp rx=0 tx=0).stop 2024-09-10T08:47:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.772+0000 7f6101800640 1 -- 172.21.15.89:0/3945148791 shutdown_connections 2024-09-10T08:47:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.772+0000 7f6101800640 1 --2- 172.21.15.89:0/3945148791 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f60e00a2cd0 0x7f60e00a5190 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.772+0000 7f6101800640 1 --2- 172.21.15.89:0/3945148791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f60fc1036f0 0x7f60fc1a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.772+0000 7f6101800640 1 --2- 172.21.15.89:0/3945148791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60fc102d30 0x7f60fc19e8e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.772+0000 7f6101800640 1 --2- 172.21.15.89:0/3945148791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60fc101b40 0x7f60fc19e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.772+0000 7f6101800640 1 -- 172.21.15.89:0/3945148791 >> 172.21.15.89:0/3945148791 conn(0x7f60fc0fd2e0 msgr2=0x7f60fc106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.772+0000 7f6101800640 1 -- 172.21.15.89:0/3945148791 shutdown_connections 2024-09-10T08:47:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:07.772+0000 7f6101800640 1 -- 172.21.15.89:0/3945148791 wait complete. 2024-09-10T08:47:09.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:08 smithi149 ceph-mon[36167]: pgmap v609: 129 pgs: 129 active+clean; 454 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 2.47k op/s 2024-09-10T08:47:09.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:08 smithi111 ceph-mon[35220]: pgmap v609: 129 pgs: 129 active+clean; 454 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 2.47k op/s 2024-09-10T08:47:09.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:08 smithi089 ceph-mon[30757]: pgmap v609: 129 pgs: 129 active+clean; 454 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 2.47k op/s 2024-09-10T08:47:11.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:10 smithi149 ceph-mon[36167]: pgmap v610: 129 pgs: 129 active+clean; 451 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 1.88k op/s 2024-09-10T08:47:11.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:10 smithi111 ceph-mon[35220]: pgmap v610: 129 pgs: 129 active+clean; 451 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 1.88k op/s 2024-09-10T08:47:11.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:10 smithi089 ceph-mon[30757]: pgmap v610: 129 pgs: 129 active+clean; 451 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 1.88k op/s 2024-09-10T08:47:13.121 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:12 smithi089 ceph-mon[30757]: pgmap v611: 129 pgs: 129 active+clean; 448 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 2.36k op/s 2024-09-10T08:47:13.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:12 smithi149 ceph-mon[36167]: pgmap v611: 129 pgs: 129 active+clean; 448 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 2.36k op/s 2024-09-10T08:47:13.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:12 smithi111 ceph-mon[35220]: pgmap v611: 129 pgs: 129 active+clean; 448 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 2.36k op/s 2024-09-10T08:47:15.123 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:14 smithi089 ceph-mon[30757]: pgmap v612: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 1.25k op/s 2024-09-10T08:47:15.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:14 smithi149 ceph-mon[36167]: pgmap v612: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 1.25k op/s 2024-09-10T08:47:15.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:14 smithi111 ceph-mon[35220]: pgmap v612: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 1.25k op/s 2024-09-10T08:47:17.194 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:16 smithi089 ceph-mon[30757]: pgmap v613: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.4 MiB/s wr, 820 op/s 2024-09-10T08:47:17.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:16 smithi149 ceph-mon[36167]: pgmap v613: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.4 MiB/s wr, 820 op/s 2024-09-10T08:47:17.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:16 smithi111 ceph-mon[35220]: pgmap v613: 129 pgs: 129 active+clean; 446 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.4 MiB/s wr, 820 op/s 2024-09-10T08:47:18.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.335+0000 7fd762a00640 1 -- 172.21.15.89:0/1423527487 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b4b8769c60 con 0x55b4b887d030 2024-09-10T08:47:18.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.335+0000 7fd762a00640 10 client.26644 handle_command_reply: tid=0 2024-09-10T08:47:18.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.336+0000 7fd7863fd740 1 client.26644 shutdown 2024-09-10T08:47:18.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.336+0000 7fd7863fd740 1 -- 172.21.15.89:0/1423527487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f6cd0 msgr2=0x55b4b8809bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:18.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.336+0000 7fd7863fd740 1 --2- 172.21.15.89:0/1423527487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f6cd0 0x55b4b8809bc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd768000c00 tx=0x7fd7680b8510 comp rx=0 tx=0).stop 2024-09-10T08:47:18.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.336+0000 7fd7863fd740 1 -- 172.21.15.89:0/1423527487 shutdown_connections 2024-09-10T08:47:18.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.336+0000 7fd7863fd740 1 --2- 172.21.15.89:0/1423527487 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55b4b887d030 0x55b4b887f450 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:18.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.336+0000 7fd7863fd740 1 --2- 172.21.15.89:0/1423527487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4b8801590 0x55b4b8806490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:18.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.336+0000 7fd7863fd740 1 --2- 172.21.15.89:0/1423527487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b87f6cd0 0x55b4b8809bc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:18.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.336+0000 7fd762a00640 0 client.26644 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:18.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.336+0000 7fd7863fd740 1 --2- 172.21.15.89:0/1423527487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4b87f63b0 0x55b4b8809680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:18.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.336+0000 7fd7863fd740 1 -- 172.21.15.89:0/1423527487 >> 172.21.15.89:0/1423527487 conn(0x55b4b87f2940 msgr2=0x55b4b87f7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:18.338 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.336+0000 7fd7863fd740 1 -- 172.21.15.89:0/1423527487 shutdown_connections 2024-09-10T08:47:18.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.336+0000 7fd7863fd740 1 -- 172.21.15.89:0/1423527487 wait complete. 2024-09-10T08:47:18.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.336+0000 7fd7863fd740 20 client.26644 trim_cache size 0 max 16384 2024-09-10T08:47:18.339 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:47:18.339 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 30, 2024-09-10T08:47:18.339 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24554, 2024-09-10T08:47:18.339 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 18, 2024-09-10T08:47:18.339 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": { 2024-09-10T08:47:18.339 INFO:teuthology.orchestra.run.smithi089.stdout: "7ca83d3b": { 2024-09-10T08:47:18.340 INFO:teuthology.orchestra.run.smithi089.stdout: "version": 18, 2024-09-10T08:47:18.340 INFO:teuthology.orchestra.run.smithi089.stdout: "age_ref": 0.0, 2024-09-10T08:47:18.340 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:47:18.340 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCED", 2024-09-10T08:47:18.340 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:47:18.340 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:47:18.340 INFO:teuthology.orchestra.run.smithi089.stdout: "timeout": 90.0, 2024-09-10T08:47:18.340 INFO:teuthology.orchestra.run.smithi089.stdout: "expiration": 140.1, 2024-09-10T08:47:18.340 INFO:teuthology.orchestra.run.smithi089.stdout: "members": { 2024-09-10T08:47:18.340 INFO:teuthology.orchestra.run.smithi089.stdout: "file:/": { 2024-09-10T08:47:18.340 INFO:teuthology.orchestra.run.smithi089.stdout: "excluded": false, 2024-09-10T08:47:18.340 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:47:18.340 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCED", 2024-09-10T08:47:18.340 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:47:18.341 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:18.341 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:18.341 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:18.341 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:18.341 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:18.341 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:47:18.341 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.337+0000 7fd783e00640 1 -- 172.21.15.89:0/72834518 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd7640a2c00 msgr2=0x7fd7640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:18.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.337+0000 7fd783e00640 1 --2- 172.21.15.89:0/72834518 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd7640a2c00 0x7fd7640a50c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55b4b87f37a0 tx=0x7fd770005f90 comp rx=0 tx=0).stop 2024-09-10T08:47:18.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.337+0000 7fd783e00640 1 -- 172.21.15.89:0/72834518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd77c1049a0 msgr2=0x7fd77c19ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:18.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.337+0000 7fd783e00640 1 --2- 172.21.15.89:0/72834518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd77c1049a0 0x7fd77c19ac00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd768006010 tx=0x7fd768004040 comp rx=0 tx=0).stop 2024-09-10T08:47:18.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.337+0000 7fd783e00640 1 -- 172.21.15.89:0/72834518 shutdown_connections 2024-09-10T08:47:18.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.337+0000 7fd783e00640 1 --2- 172.21.15.89:0/72834518 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd7640a2c00 0x7fd7640a50c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:18.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.337+0000 7fd783e00640 1 --2- 172.21.15.89:0/72834518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd77c109120 0x7fd77c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:18.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.337+0000 7fd783e00640 1 --2- 172.21.15.89:0/72834518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd77c1049a0 0x7fd77c19ac00 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:18.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.337+0000 7fd783e00640 1 --2- 172.21.15.89:0/72834518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd77c103ff0 0x7fd77c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:18.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.337+0000 7fd783e00640 1 -- 172.21.15.89:0/72834518 >> 172.21.15.89:0/72834518 conn(0x7fd77c0ffa90 msgr2=0x7fd77c100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:18.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.338+0000 7fd783e00640 1 -- 172.21.15.89:0/72834518 shutdown_connections 2024-09-10T08:47:18.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:18.338+0000 7fd783e00640 1 -- 172.21.15.89:0/72834518 wait complete. 2024-09-10T08:47:19.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:18 smithi149 ceph-mon[36167]: pgmap v614: 129 pgs: 129 active+clean; 460 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 1.05k op/s 2024-09-10T08:47:19.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:18 smithi111 ceph-mon[35220]: pgmap v614: 129 pgs: 129 active+clean; 460 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 1.05k op/s 2024-09-10T08:47:19.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:18 smithi089 ceph-mon[30757]: pgmap v614: 129 pgs: 129 active+clean; 460 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 1.05k op/s 2024-09-10T08:47:20.576 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7ca83d3b', quiesce took 27.8 seconds. Will release after: -7.699999999999999 2024-09-10T08:47:20.577 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:47:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.890+0000 7f397c000640 1 -- 172.21.15.89:0/4284057879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3974104f00 msgr2=0x7f39741072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.890+0000 7f397c000640 1 --2- 172.21.15.89:0/4284057879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3974104f00 0x7f39741072f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3960009900 tx=0x7f396002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.890+0000 7f397c000640 1 -- 172.21.15.89:0/4284057879 shutdown_connections 2024-09-10T08:47:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.890+0000 7f397c000640 1 --2- 172.21.15.89:0/4284057879 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3974107830 0x7f3974109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.890+0000 7f397c000640 1 --2- 172.21.15.89:0/4284057879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3974104f00 0x7f39741072f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.890+0000 7f397c000640 1 --2- 172.21.15.89:0/4284057879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39741025d0 0x7f39741049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.890+0000 7f397c000640 1 -- 172.21.15.89:0/4284057879 >> 172.21.15.89:0/4284057879 conn(0x7f39740fc1b0 msgr2=0x7f39740fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.890+0000 7f397c000640 1 -- 172.21.15.89:0/4284057879 shutdown_connections 2024-09-10T08:47:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.890+0000 7f397c000640 1 -- 172.21.15.89:0/4284057879 wait complete. 2024-09-10T08:47:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.891+0000 7f397c000640 1 Processor -- start 2024-09-10T08:47:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.891+0000 7f397c000640 1 -- start start 2024-09-10T08:47:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.891+0000 7f397c000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39741025d0 0x7f397419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.892+0000 7f397c000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3974104f00 0x7f397419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.892+0000 7f397c000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3974107830 0x7f39741a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.892+0000 7f397c000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f397410d830 con 0x7f3974104f00 2024-09-10T08:47:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.892+0000 7f397c000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f397410d6b0 con 0x7f39741025d0 2024-09-10T08:47:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.892+0000 7f397c000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f397410d9b0 con 0x7f3974107830 2024-09-10T08:47:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.892+0000 7f3978c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3974104f00 0x7f397419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.892+0000 7f3978c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3974104f00 0x7f397419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35378/0 (socket says 172.21.15.89:35378) 2024-09-10T08:47:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.892+0000 7f3978c00640 1 -- 172.21.15.89:0/1497117605 learned_addr learned my addr 172.21.15.89:0/1497117605 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.892+0000 7f397a000640 1 --2- 172.21.15.89:0/1497117605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3974107830 0x7f39741a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.892+0000 7f3979600640 1 --2- 172.21.15.89:0/1497117605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39741025d0 0x7f397419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.892+0000 7f3978c00640 1 -- 172.21.15.89:0/1497117605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3974107830 msgr2=0x7f39741a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.892+0000 7f3978c00640 1 --2- 172.21.15.89:0/1497117605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3974107830 0x7f39741a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.892+0000 7f3978c00640 1 -- 172.21.15.89:0/1497117605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39741025d0 msgr2=0x7f397419a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.892+0000 7f3978c00640 1 --2- 172.21.15.89:0/1497117605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39741025d0 0x7f397419a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.892+0000 7f3978c00640 1 -- 172.21.15.89:0/1497117605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39741a1c90 con 0x7f3974104f00 2024-09-10T08:47:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.892+0000 7f3979600640 1 --2- 172.21.15.89:0/1497117605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39741025d0 0x7f397419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.893+0000 7f3978c00640 1 --2- 172.21.15.89:0/1497117605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3974104f00 0x7f397419a5b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3960031440 tx=0x7f3960032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.893+0000 7f3972000640 1 -- 172.21.15.89:0/1497117605 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396003f030 con 0x7f3974104f00 2024-09-10T08:47:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.893+0000 7f3972000640 1 -- 172.21.15.89:0/1497117605 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3960003ce0 con 0x7f3974104f00 2024-09-10T08:47:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.893+0000 7f3972000640 1 -- 172.21.15.89:0/1497117605 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3960035070 con 0x7f3974104f00 2024-09-10T08:47:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.893+0000 7f397c000640 1 -- 172.21.15.89:0/1497117605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39741a1f20 con 0x7f3974104f00 2024-09-10T08:47:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.894+0000 7f397c000640 1 -- 172.21.15.89:0/1497117605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39741a2330 con 0x7f3974104f00 2024-09-10T08:47:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.896+0000 7f397f04d740 1 Processor -- start 2024-09-10T08:47:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.897+0000 7f397f04d740 1 -- start start 2024-09-10T08:47:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.897+0000 7f397f04d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a84090 0x556511a84490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.897+0000 7f397f04d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a849d0 0x556511a89490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.897+0000 7f397f04d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a89a60 0x556511a8bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.897+0000 7f3972000640 1 -- 172.21.15.89:0/1497117605 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3960042030 con 0x7f3974104f00 2024-09-10T08:47:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.897+0000 7f397f04d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5565119e9100 con 0x556511a849d0 2024-09-10T08:47:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.897+0000 7f397f04d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5565119911b0 con 0x556511a89a60 2024-09-10T08:47:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.897+0000 7f397f04d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5565119e50c0 con 0x556511a84090 2024-09-10T08:47:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.897+0000 7f3979600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a849d0 0x556511a89490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.897+0000 7f397a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a84090 0x556511a84490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.897+0000 7f397a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a84090 0x556511a84490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:60574/0 (socket says 172.21.15.89:60574) 2024-09-10T08:47:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.897+0000 7f397a000640 1 -- 172.21.15.89:0/2657809854 learned_addr learned my addr 172.21.15.89:0/2657809854 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.897+0000 7f3978c00640 1 --2- 172.21.15.89:0/2657809854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a89a60 0x556511a8bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.898+0000 7f397a000640 1 -- 172.21.15.89:0/2657809854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a89a60 msgr2=0x556511a8bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.898+0000 7f397a000640 1 --2- 172.21.15.89:0/2657809854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a89a60 0x556511a8bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.898+0000 7f397a000640 1 -- 172.21.15.89:0/2657809854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a849d0 msgr2=0x556511a89490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.898+0000 7f397a000640 1 --2- 172.21.15.89:0/2657809854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a849d0 0x556511a89490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.898+0000 7f397a000640 1 -- 172.21.15.89:0/2657809854 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565116fc210 con 0x556511a84090 2024-09-10T08:47:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.898+0000 7f397a000640 1 --2- 172.21.15.89:0/2657809854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a84090 0x556511a84490 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f396c009fa0 tx=0x7f396c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6ea8440217a6770 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.898+0000 7f3966600640 1 -- 172.21.15.89:0/2657809854 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c004d20 con 0x556511a84090 2024-09-10T08:47:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.898+0000 7f3966600640 1 -- 172.21.15.89:0/2657809854 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f396c004050 con 0x556511a84090 2024-09-10T08:47:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.898+0000 7f3966600640 1 -- 172.21.15.89:0/2657809854 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c005280 con 0x556511a84090 2024-09-10T08:47:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.899+0000 7f3972000640 1 --2- 172.21.15.89:0/1497117605 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f395c0a2cb0 0x7f395c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.899+0000 7f3972000640 1 -- 172.21.15.89:0/1497117605 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f39600ef870 con 0x7f3974104f00 2024-09-10T08:47:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.899+0000 7f3979600640 1 --2- 172.21.15.89:0/1497117605 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f395c0a2cb0 0x7f395c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.899+0000 7f397f04d740 1 -- 172.21.15.89:0/2657809854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a84090 msgr2=0x556511a84490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.899+0000 7f397f04d740 1 --2- 172.21.15.89:0/2657809854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a84090 0x556511a84490 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f396c009fa0 tx=0x7f396c0118d0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.899+0000 7f397f04d740 1 -- 172.21.15.89:0/2657809854 shutdown_connections 2024-09-10T08:47:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.899+0000 7f397f04d740 1 --2- 172.21.15.89:0/2657809854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a89a60 0x556511a8bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.899+0000 7f397f04d740 1 --2- 172.21.15.89:0/2657809854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a849d0 0x556511a89490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.899+0000 7f397f04d740 1 --2- 172.21.15.89:0/2657809854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a84090 0x556511a84490 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.899+0000 7f397f04d740 1 -- 172.21.15.89:0/2657809854 >> 172.21.15.89:0/2657809854 conn(0x556511a7f700 msgr2=0x556511a7fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.899+0000 7f397f04d740 1 -- 172.21.15.89:0/2657809854 shutdown_connections 2024-09-10T08:47:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.900+0000 7f397f04d740 1 -- 172.21.15.89:0/2657809854 wait complete. 2024-09-10T08:47:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.900+0000 7f3979600640 1 --2- 172.21.15.89:0/1497117605 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f395c0a2cb0 0x7f395c0a5170 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x556511a85400 tx=0x7f3968005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.900+0000 7f397f04d740 1 -- start start 2024-09-10T08:47:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.900+0000 7f397f04d740 1 Processor -- start 2024-09-10T08:47:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.901+0000 7f397f04d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a84090 0x556511a8e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.901+0000 7f397f04d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a849d0 0x556511a8e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.901+0000 7f397f04d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a89a60 0x556511a95820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.901+0000 7f397f04d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556511a8ca30 con 0x556511a84090 2024-09-10T08:47:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.901+0000 7f397f04d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556511972090 con 0x556511a849d0 2024-09-10T08:47:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.901+0000 7f397f04d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556511a8cbb0 con 0x556511a89a60 2024-09-10T08:47:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.901+0000 7f397f04d740 10 client.0 _ll_register_callbacks cb 0x5565119c6f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.901+0000 7f3979600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a84090 0x556511a8e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.901+0000 7f3979600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a84090 0x556511a8e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35398/0 (socket says 172.21.15.89:35398) 2024-09-10T08:47:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.901+0000 7f3979600640 1 -- 172.21.15.89:0/1617162898 learned_addr learned my addr 172.21.15.89:0/1617162898 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.901+0000 7f397a000640 1 --2- 172.21.15.89:0/1617162898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a89a60 0x556511a95820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.901+0000 7f3978c00640 1 --2- 172.21.15.89:0/1617162898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a849d0 0x556511a8e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.901+0000 7f3979600640 1 -- 172.21.15.89:0/1617162898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a89a60 msgr2=0x556511a95820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.901+0000 7f3979600640 1 --2- 172.21.15.89:0/1617162898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a89a60 0x556511a95820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.901+0000 7f3979600640 1 -- 172.21.15.89:0/1617162898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a849d0 msgr2=0x556511a8e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.901+0000 7f3979600640 1 --2- 172.21.15.89:0/1617162898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a849d0 0x556511a8e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.902+0000 7f3979600640 1 -- 172.21.15.89:0/1617162898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565117c8620 con 0x556511a84090 2024-09-10T08:47:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.902+0000 7f397a000640 1 --2- 172.21.15.89:0/1617162898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a89a60 0x556511a95820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.902+0000 7f3978c00640 1 --2- 172.21.15.89:0/1617162898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a849d0 0x556511a8e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.902+0000 7f3979600640 1 --2- 172.21.15.89:0/1617162898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a84090 0x556511a8e2a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f396800fac0 tx=0x7f3968004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ae3c9e196317fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.902+0000 7f395aa00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:47:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.902+0000 7f395aa00640 1 -- 172.21.15.89:0/1617162898 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968004020 con 0x556511a84090 2024-09-10T08:47:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.902+0000 7f397f04d740 1 -- 172.21.15.89:0/1617162898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f396c004050 con 0x556511a84090 2024-09-10T08:47:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.902+0000 7f395aa00640 1 -- 172.21.15.89:0/1617162898 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f3968018030 con 0x556511a84090 2024-09-10T08:47:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.902+0000 7f395aa00640 1 -- 172.21.15.89:0/1617162898 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3968034050 con 0x556511a84090 2024-09-10T08:47:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.902+0000 7f397f04d740 10 client.16662 fetch_fsmap learned FSMap version 30 2024-09-10T08:47:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.902+0000 7f397f04d740 1 -- 172.21.15.89:0/1617162898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556511aa0f40 con 0x556511a84090 2024-09-10T08:47:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.903+0000 7f395aa00640 1 -- 172.21.15.89:0/1617162898 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f396c004050 con 0x556511a84090 2024-09-10T08:47:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.903+0000 7f395aa00640 1 -- 172.21.15.89:0/1617162898 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f39680035f0 con 0x556511a84090 2024-09-10T08:47:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.903+0000 7f397f04d740 10 client.16662 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T08:47:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.903+0000 7f397f04d740 10 client.16662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.903+0000 7f397f04d740 20 client.16662 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.903+0000 7f397f04d740 1 --2- 172.21.15.89:0/1617162898 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x556511aa2630 0x556511aa4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.903+0000 7f397f04d740 4 client.16662 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.903+0000 7f397f04d740 1 -- 172.21.15.89:0/1617162898 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565119f6830 con 0x556511aa2630 2024-09-10T08:47:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.904+0000 7f3978c00640 1 --2- 172.21.15.89:0/1617162898 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x556511aa2630 0x556511aa4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.904+0000 7f395aa00640 10 client.16662 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.904+0000 7f3978c00640 1 --2- 172.21.15.89:0/1617162898 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x556511aa2630 0x556511aa4ab0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7941bf1131c8f42f server_cookie=3a4efaba4522a885 in_seq=0 out_seq=0 2024-09-10T08:47:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.905+0000 7f395aa00640 1 -- 172.21.15.89:0/1617162898 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565119f6830 con 0x556511aa2630 2024-09-10T08:47:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.905+0000 7f395aa00640 10 client.16662 handle_command_reply: tid=0 2024-09-10T08:47:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.905+0000 7f397f04d740 1 client.16662 shutdown 2024-09-10T08:47:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.905+0000 7f397f04d740 1 -- 172.21.15.89:0/1617162898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a84090 msgr2=0x556511a8e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.905+0000 7f397f04d740 1 --2- 172.21.15.89:0/1617162898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a84090 0x556511a8e2a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f396800fac0 tx=0x7f3968004730 comp rx=0 tx=0).stop 2024-09-10T08:47:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.905+0000 7f397f04d740 1 -- 172.21.15.89:0/1617162898 shutdown_connections 2024-09-10T08:47:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.905+0000 7f397f04d740 1 --2- 172.21.15.89:0/1617162898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a89a60 0x556511a95820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.905+0000 7f397f04d740 1 --2- 172.21.15.89:0/1617162898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a849d0 0x556511a8e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.905+0000 7f397f04d740 1 --2- 172.21.15.89:0/1617162898 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x556511aa2630 0x556511aa4ab0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.905+0000 7f397f04d740 1 --2- 172.21.15.89:0/1617162898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a84090 0x556511a8e2a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.905+0000 7f397f04d740 1 -- 172.21.15.89:0/1617162898 >> 172.21.15.89:0/1617162898 conn(0x556511a7f700 msgr2=0x556511a80f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.905+0000 7f395aa00640 0 client.16662 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.906+0000 7f397f04d740 1 -- 172.21.15.89:0/1617162898 shutdown_connections 2024-09-10T08:47:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.906+0000 7f397f04d740 1 -- 172.21.15.89:0/1617162898 wait complete. 2024-09-10T08:47:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.906+0000 7f397f04d740 20 client.16662 trim_cache size 0 max 16384 2024-09-10T08:47:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.921+0000 7f397f04d740 1 Processor -- start 2024-09-10T08:47:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.921+0000 7f397f04d740 1 -- start start 2024-09-10T08:47:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.922+0000 7f397f04d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a84090 0x556511a89480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.922+0000 7f397f04d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a849d0 0x556511a957e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.922+0000 7f397f04d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a89a60 0x556511a7e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.922+0000 7f397f04d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f39741aaf50 con 0x556511a84090 2024-09-10T08:47:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.922+0000 7f397f04d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f39741a8820 con 0x556511a849d0 2024-09-10T08:47:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.922+0000 7f397f04d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f39741a6670 con 0x556511a89a60 2024-09-10T08:47:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.922+0000 7f3979600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a84090 0x556511a89480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.922+0000 7f3978c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a849d0 0x556511a957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.922+0000 7f3979600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a84090 0x556511a89480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35408/0 (socket says 172.21.15.89:35408) 2024-09-10T08:47:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.922+0000 7f3978c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a849d0 0x556511a957e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:51432/0 (socket says 172.21.15.89:51432) 2024-09-10T08:47:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.922+0000 7f3979600640 1 -- 172.21.15.89:0/587304809 learned_addr learned my addr 172.21.15.89:0/587304809 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.922+0000 7f397a000640 1 --2- 172.21.15.89:0/587304809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a89a60 0x556511a7e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.922+0000 7f3978c00640 1 -- 172.21.15.89:0/587304809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a89a60 msgr2=0x556511a7e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.922+0000 7f3978c00640 1 --2- 172.21.15.89:0/587304809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a89a60 0x556511a7e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.922+0000 7f3978c00640 1 -- 172.21.15.89:0/587304809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a84090 msgr2=0x556511a89480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.922+0000 7f3978c00640 1 --2- 172.21.15.89:0/587304809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a84090 0x556511a89480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.922+0000 7f3978c00640 1 -- 172.21.15.89:0/587304809 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556511a88ed0 con 0x556511a849d0 2024-09-10T08:47:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.922+0000 7f3979600640 1 --2- 172.21.15.89:0/587304809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a84090 0x556511a89480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.923+0000 7f3978c00640 1 --2- 172.21.15.89:0/587304809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a849d0 0x556511a957e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3960006010 tx=0x7f39600f1cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16286f5098a6b726 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.923+0000 7f3965c00640 1 -- 172.21.15.89:0/587304809 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3960049040 con 0x556511a849d0 2024-09-10T08:47:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.923+0000 7f3965c00640 1 -- 172.21.15.89:0/587304809 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39600bc3a0 con 0x556511a849d0 2024-09-10T08:47:20.925 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.923+0000 7f3965c00640 1 -- 172.21.15.89:0/587304809 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396003b460 con 0x556511a849d0 2024-09-10T08:47:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.924+0000 7f397f04d740 1 -- 172.21.15.89:0/587304809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a849d0 msgr2=0x556511a957e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.924+0000 7f397f04d740 1 --2- 172.21.15.89:0/587304809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a849d0 0x556511a957e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3960006010 tx=0x7f39600f1cc0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.924+0000 7f397f04d740 1 -- 172.21.15.89:0/587304809 shutdown_connections 2024-09-10T08:47:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.924+0000 7f397f04d740 1 --2- 172.21.15.89:0/587304809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a89a60 0x556511a7e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.924+0000 7f397f04d740 1 --2- 172.21.15.89:0/587304809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a849d0 0x556511a957e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.924+0000 7f397f04d740 1 --2- 172.21.15.89:0/587304809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a84090 0x556511a89480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.924+0000 7f397f04d740 1 -- 172.21.15.89:0/587304809 >> 172.21.15.89:0/587304809 conn(0x556511a7f700 msgr2=0x556511a7e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.924+0000 7f397f04d740 1 -- 172.21.15.89:0/587304809 shutdown_connections 2024-09-10T08:47:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.924+0000 7f397f04d740 1 -- 172.21.15.89:0/587304809 wait complete. 2024-09-10T08:47:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.924+0000 7f397f04d740 1 -- start start 2024-09-10T08:47:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.924+0000 7f397f04d740 1 Processor -- start 2024-09-10T08:47:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.925+0000 7f397f04d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a84090 0x556511a936b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.925+0000 7f397f04d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a849d0 0x556511a93bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.925+0000 7f397f04d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a89a60 0x556511a94c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.925+0000 7f397f04d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f39741a6ce0 con 0x556511a84090 2024-09-10T08:47:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.925+0000 7f397f04d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f39741a8150 con 0x556511a89a60 2024-09-10T08:47:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.925+0000 7f397f04d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f39741a73b0 con 0x556511a849d0 2024-09-10T08:47:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.925+0000 7f397f04d740 10 client.0 _ll_register_callbacks cb 0x7f3968009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.925+0000 7f397a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a89a60 0x556511a94c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.925+0000 7f3979600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a84090 0x556511a936b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.925+0000 7f397a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a89a60 0x556511a94c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:51434/0 (socket says 172.21.15.89:51434) 2024-09-10T08:47:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.925+0000 7f397a000640 1 -- 172.21.15.89:0/1927378427 learned_addr learned my addr 172.21.15.89:0/1927378427 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.925+0000 7f3978c00640 1 --2- 172.21.15.89:0/1927378427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a849d0 0x556511a93bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.925+0000 7f397a000640 1 -- 172.21.15.89:0/1927378427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a849d0 msgr2=0x556511a93bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.926+0000 7f397a000640 1 --2- 172.21.15.89:0/1927378427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a849d0 0x556511a93bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.926+0000 7f397a000640 1 -- 172.21.15.89:0/1927378427 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a84090 msgr2=0x556511a936b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.926+0000 7f397a000640 1 --2- 172.21.15.89:0/1927378427 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a84090 0x556511a936b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.926+0000 7f397a000640 1 -- 172.21.15.89:0/1927378427 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556511a95210 con 0x556511a89a60 2024-09-10T08:47:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.926+0000 7f3979600640 1 --2- 172.21.15.89:0/1927378427 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a84090 0x556511a936b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.926+0000 7f397a000640 1 --2- 172.21.15.89:0/1927378427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a89a60 0x556511a94c00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f396c000f80 tx=0x7f396c003f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b086e7278f4aca2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.926+0000 7f395aa00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:47:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.926+0000 7f395aa00640 1 -- 172.21.15.89:0/1927378427 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c01d040 con 0x556511a89a60 2024-09-10T08:47:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.926+0000 7f395aa00640 1 -- 172.21.15.89:0/1927378427 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f396c005450 con 0x556511a89a60 2024-09-10T08:47:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.926+0000 7f397f04d740 1 -- 172.21.15.89:0/1927378427 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39600bc3a0 con 0x556511a89a60 2024-09-10T08:47:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.926+0000 7f395aa00640 1 -- 172.21.15.89:0/1927378427 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f396c005670 con 0x556511a89a60 2024-09-10T08:47:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.926+0000 7f397f04d740 10 client.26732 fetch_fsmap learned FSMap version 30 2024-09-10T08:47:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.926+0000 7f397f04d740 1 -- 172.21.15.89:0/1927378427 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556511b0da80 con 0x556511a89a60 2024-09-10T08:47:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.927+0000 7f395aa00640 1 -- 172.21.15.89:0/1927378427 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39600bc3a0 con 0x556511a89a60 2024-09-10T08:47:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.927+0000 7f395aa00640 1 -- 172.21.15.89:0/1927378427 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f396c019430 con 0x556511a89a60 2024-09-10T08:47:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.927+0000 7f397f04d740 10 client.26732 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T08:47:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.927+0000 7f397f04d740 10 client.26732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.927+0000 7f397f04d740 20 client.26732 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.927+0000 7f397f04d740 1 --2- 172.21.15.89:0/1927378427 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x556511b0e2d0 0x556511b106f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.927+0000 7f397f04d740 4 client.26732 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:47:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.927+0000 7f397f04d740 1 -- 172.21.15.89:0/1927378427 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565119f0b80 con 0x556511b0e2d0 2024-09-10T08:47:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.927+0000 7f3979600640 1 --2- 172.21.15.89:0/1927378427 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x556511b0e2d0 0x556511b106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.928+0000 7f3979600640 1 --2- 172.21.15.89:0/1927378427 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x556511b0e2d0 0x556511b106f0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dee5064fa073c23 server_cookie=b57fea9e6c0bb4ae in_seq=0 out_seq=0 2024-09-10T08:47:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:20.928+0000 7f395aa00640 10 client.26732 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:20.981 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:21.142 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:20 smithi089 ceph-mon[30757]: pgmap v615: 129 pgs: 129 active+clean; 460 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 862 op/s 2024-09-10T08:47:21.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:20 smithi149 ceph-mon[36167]: pgmap v615: 129 pgs: 129 active+clean; 460 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 862 op/s 2024-09-10T08:47:21.289 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.286+0000 7f395aa00640 1 -- 172.21.15.89:0/1927378427 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5565119f0b80 con 0x556511b0e2d0 2024-09-10T08:47:21.289 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.286+0000 7f395aa00640 10 client.26732 handle_command_reply: tid=0 2024-09-10T08:47:21.289 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.287+0000 7f397f04d740 1 client.26732 shutdown 2024-09-10T08:47:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:47:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:47:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:47:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:47:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.287+0000 7f397f04d740 1 -- 172.21.15.89:0/1927378427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a89a60 msgr2=0x556511a94c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.287+0000 7f397f04d740 1 --2- 172.21.15.89:0/1927378427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a89a60 0x556511a94c00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f396c000f80 tx=0x7f396c003f30 comp rx=0 tx=0).stop 2024-09-10T08:47:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.287+0000 7f397f04d740 1 -- 172.21.15.89:0/1927378427 shutdown_connections 2024-09-10T08:47:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.287+0000 7f397f04d740 1 --2- 172.21.15.89:0/1927378427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556511a89a60 0x556511a94c00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.287+0000 7f397f04d740 1 --2- 172.21.15.89:0/1927378427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556511a849d0 0x556511a93bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.287+0000 7f397f04d740 1 --2- 172.21.15.89:0/1927378427 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x556511b0e2d0 0x556511b106f0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.287+0000 7f397f04d740 1 --2- 172.21.15.89:0/1927378427 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556511a84090 0x556511a936b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.287+0000 7f397f04d740 1 -- 172.21.15.89:0/1927378427 >> 172.21.15.89:0/1927378427 conn(0x556511a7f700 msgr2=0x556511a7e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.287+0000 7f395aa00640 0 client.26732 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.287+0000 7f397f04d740 1 -- 172.21.15.89:0/1927378427 shutdown_connections 2024-09-10T08:47:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.287+0000 7f397f04d740 1 -- 172.21.15.89:0/1927378427 wait complete. 2024-09-10T08:47:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.287+0000 7f397f04d740 20 client.26732 trim_cache size 0 max 16384 2024-09-10T08:47:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.291+0000 7f397c000640 1 -- 172.21.15.89:0/1497117605 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f395c0a2cb0 msgr2=0x7f395c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.291+0000 7f397c000640 1 --2- 172.21.15.89:0/1497117605 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f395c0a2cb0 0x7f395c0a5170 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x556511a85400 tx=0x7f3968005f90 comp rx=0 tx=0).stop 2024-09-10T08:47:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.291+0000 7f397c000640 1 -- 172.21.15.89:0/1497117605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3974104f00 msgr2=0x7f397419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.291+0000 7f397c000640 1 --2- 172.21.15.89:0/1497117605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3974104f00 0x7f397419a5b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3960031440 tx=0x7f3960032ce0 comp rx=0 tx=0).stop 2024-09-10T08:47:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.292+0000 7f397c000640 1 -- 172.21.15.89:0/1497117605 shutdown_connections 2024-09-10T08:47:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.292+0000 7f397c000640 1 --2- 172.21.15.89:0/1497117605 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f395c0a2cb0 0x7f395c0a5170 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.292+0000 7f397c000640 1 --2- 172.21.15.89:0/1497117605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3974107830 0x7f39741a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.292+0000 7f397c000640 1 --2- 172.21.15.89:0/1497117605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3974104f00 0x7f397419a5b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.292+0000 7f397c000640 1 --2- 172.21.15.89:0/1497117605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39741025d0 0x7f397419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.292+0000 7f397c000640 1 -- 172.21.15.89:0/1497117605 >> 172.21.15.89:0/1497117605 conn(0x7f39740fc1b0 msgr2=0x7f3974101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:21.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:20 smithi111 ceph-mon[35220]: pgmap v615: 129 pgs: 129 active+clean; 460 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 862 op/s 2024-09-10T08:47:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.292+0000 7f397c000640 1 -- 172.21.15.89:0/1497117605 shutdown_connections 2024-09-10T08:47:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:21.292+0000 7f397c000640 1 -- 172.21.15.89:0/1497117605 wait complete. 2024-09-10T08:47:21.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.544+0000 7fb005200640 1 -- 172.21.15.89:0/4115667844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0001019c0 msgr2=0x7fb000101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:21.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.544+0000 7fb005200640 1 --2- 172.21.15.89:0/4115667844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0001019c0 0x7fb000101e20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fafec009930 tx=0x7fafec02a420 comp rx=0 tx=0).stop 2024-09-10T08:47:21.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.544+0000 7fb005200640 1 -- 172.21.15.89:0/4115667844 shutdown_connections 2024-09-10T08:47:21.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.544+0000 7fb005200640 1 --2- 172.21.15.89:0/4115667844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb000102360 0x7fb00010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:21.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.544+0000 7fb005200640 1 --2- 172.21.15.89:0/4115667844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0001019c0 0x7fb000101e20 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:21.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.544+0000 7fb005200640 1 --2- 172.21.15.89:0/4115667844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0001079c0 0x7fb000107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:21.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.544+0000 7fb005200640 1 -- 172.21.15.89:0/4115667844 >> 172.21.15.89:0/4115667844 conn(0x7fb0000fd740 msgr2=0x7fb0000ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:21.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.544+0000 7fb005200640 1 -- 172.21.15.89:0/4115667844 shutdown_connections 2024-09-10T08:47:21.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.544+0000 7fb005200640 1 -- 172.21.15.89:0/4115667844 wait complete. 2024-09-10T08:47:21.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.545+0000 7fb005200640 1 Processor -- start 2024-09-10T08:47:21.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.545+0000 7fb005200640 1 -- start start 2024-09-10T08:47:21.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.546+0000 7fb005200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0001019c0 0x7fb0000744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:21.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.546+0000 7fb005200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb000102360 0x7fb000074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:21.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.546+0000 7fb005200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0001079c0 0x7fb000074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:21.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.546+0000 7fb005200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb000111dc0 con 0x7fb0001019c0 2024-09-10T08:47:21.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.546+0000 7fb005200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb000111c40 con 0x7fb0001079c0 2024-09-10T08:47:21.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.546+0000 7fb005200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb000111f40 con 0x7fb000102360 2024-09-10T08:47:21.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.546+0000 7fafff400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0001019c0 0x7fb0000744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:21.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.546+0000 7fafffe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0001079c0 0x7fb000074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:21.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.546+0000 7fafffe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0001079c0 0x7fb000074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:51454/0 (socket says 172.21.15.89:51454) 2024-09-10T08:47:21.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.546+0000 7fafff400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0001019c0 0x7fb0000744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35426/0 (socket says 172.21.15.89:35426) 2024-09-10T08:47:21.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.546+0000 7fafffe00640 1 -- 172.21.15.89:0/4210725355 learned_addr learned my addr 172.21.15.89:0/4210725355 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:21.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.546+0000 7faffea00640 1 --2- 172.21.15.89:0/4210725355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb000102360 0x7fb000074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:21.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.546+0000 7fafffe00640 1 -- 172.21.15.89:0/4210725355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb000102360 msgr2=0x7fb000074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:21.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.546+0000 7fafffe00640 1 --2- 172.21.15.89:0/4210725355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb000102360 0x7fb000074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:21.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.546+0000 7fafffe00640 1 -- 172.21.15.89:0/4210725355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0001019c0 msgr2=0x7fb0000744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:21.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.546+0000 7fafffe00640 1 --2- 172.21.15.89:0/4210725355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0001019c0 0x7fb0000744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:21.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.546+0000 7fafffe00640 1 -- 172.21.15.89:0/4210725355 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0001aacd0 con 0x7fb0001079c0 2024-09-10T08:47:21.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.547+0000 7fafffe00640 1 --2- 172.21.15.89:0/4210725355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0001079c0 0x7fb000074f70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faff800eb10 tx=0x7faff800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:21.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.547+0000 7faff7e00640 1 -- 172.21.15.89:0/4210725355 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff8004b20 con 0x7fb0001079c0 2024-09-10T08:47:21.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.547+0000 7faff7e00640 1 -- 172.21.15.89:0/4210725355 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faff8004cc0 con 0x7fb0001079c0 2024-09-10T08:47:21.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.547+0000 7fb005200640 1 -- 172.21.15.89:0/4210725355 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0001aaf00 con 0x7fb0001079c0 2024-09-10T08:47:21.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.547+0000 7faff7e00640 1 -- 172.21.15.89:0/4210725355 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff8010760 con 0x7fb0001079c0 2024-09-10T08:47:21.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.547+0000 7fb005200640 1 -- 172.21.15.89:0/4210725355 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0001ab360 con 0x7fb0001079c0 2024-09-10T08:47:21.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.548+0000 7fb005200640 1 -- 172.21.15.89:0/4210725355 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb000103100 con 0x7fb0001079c0 2024-09-10T08:47:21.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.550+0000 7faff7e00640 1 -- 172.21.15.89:0/4210725355 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7faff8005020 con 0x7fb0001079c0 2024-09-10T08:47:21.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.551+0000 7faff7e00640 1 --2- 172.21.15.89:0/4210725355 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fafe40a2b80 0x7fafe40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:21.552 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.551+0000 7faff7e00640 1 -- 172.21.15.89:0/4210725355 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7faff80cfa00 con 0x7fb0001079c0 2024-09-10T08:47:21.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.552+0000 7fafff400640 1 --2- 172.21.15.89:0/4210725355 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fafe40a2b80 0x7fafe40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:21.553 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.552+0000 7fafff400640 1 --2- 172.21.15.89:0/4210725355 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fafe40a2b80 0x7fafe40a5040 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faff0009b50 tx=0x7faff0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:21.556 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.555+0000 7faff7e00640 1 -- 172.21.15.89:0/4210725355 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faff809c530 con 0x7fb0001079c0 2024-09-10T08:47:21.768 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.767+0000 7fb005200640 1 -- 172.21.15.89:0/4210725355 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb000109f60 con 0x7fb0001079c0 2024-09-10T08:47:21.769 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.768+0000 7faff7e00640 1 -- 172.21.15.89:0/4210725355 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11920 (secure 0 0 0) 0x7faff809be30 con 0x7fb0001079c0 2024-09-10T08:47:21.769 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:47:21.770 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":30,"btime":"2024-09-10T08:46:58:834163+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:46:58.731037+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":158,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24518,24496]},"id":1}]} 2024-09-10T08:47:21.770 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 30 2024-09-10T08:47:21.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.770+0000 7fb005200640 1 -- 172.21.15.89:0/4210725355 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fafe40a2b80 msgr2=0x7fafe40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:21.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.770+0000 7fb005200640 1 --2- 172.21.15.89:0/4210725355 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fafe40a2b80 0x7fafe40a5040 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faff0009b50 tx=0x7faff0005f90 comp rx=0 tx=0).stop 2024-09-10T08:47:21.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.770+0000 7fb005200640 1 -- 172.21.15.89:0/4210725355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0001079c0 msgr2=0x7fb000074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:21.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.770+0000 7fb005200640 1 --2- 172.21.15.89:0/4210725355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0001079c0 0x7fb000074f70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faff800eb10 tx=0x7faff800c2d0 comp rx=0 tx=0).stop 2024-09-10T08:47:21.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.771+0000 7fb005200640 1 -- 172.21.15.89:0/4210725355 shutdown_connections 2024-09-10T08:47:21.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.771+0000 7fb005200640 1 --2- 172.21.15.89:0/4210725355 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fafe40a2b80 0x7fafe40a5040 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:21.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.771+0000 7fb005200640 1 --2- 172.21.15.89:0/4210725355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0001079c0 0x7fb000074f70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:21.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.771+0000 7fb005200640 1 --2- 172.21.15.89:0/4210725355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb000102360 0x7fb000074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:21.772 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.771+0000 7fb005200640 1 --2- 172.21.15.89:0/4210725355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0001019c0 0x7fb0000744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:21.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.771+0000 7fb005200640 1 -- 172.21.15.89:0/4210725355 >> 172.21.15.89:0/4210725355 conn(0x7fb0000fd740 msgr2=0x7fb0000fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:21.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.771+0000 7fb005200640 1 -- 172.21.15.89:0/4210725355 shutdown_connections 2024-09-10T08:47:21.773 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:21.771+0000 7fb005200640 1 -- 172.21.15.89:0/4210725355 wait complete. 2024-09-10T08:47:22.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:21 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/4210725355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:22.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:21 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/4210725355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:22.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:21 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/4210725355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:22.677 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24554 quiesce db --set-id 7ca83d3b --release --await' 2024-09-10T08:47:22.677 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24554 quiesce db --set-id 7ca83d3b --release --await 2024-09-10T08:47:23.046 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:23.184 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:22 smithi089 ceph-mon[30757]: pgmap v616: 129 pgs: 129 active+clean; 465 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 984 op/s 2024-09-10T08:47:23.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:22 smithi149 ceph-mon[36167]: pgmap v616: 129 pgs: 129 active+clean; 465 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 984 op/s 2024-09-10T08:47:23.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:22 smithi111 ceph-mon[35220]: pgmap v616: 129 pgs: 129 active+clean; 465 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 984 op/s 2024-09-10T08:47:23.593 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:47:23.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.640+0000 7efee1e00640 1 -- 172.21.15.89:0/432475244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efedc101130 msgr2=0x7efedc10ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:23.641 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.640+0000 7efee1e00640 1 --2- 172.21.15.89:0/432475244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efedc101130 0x7efedc10ddf0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7efec8009900 tx=0x7efec802a410 comp rx=0 tx=0).stop 2024-09-10T08:47:23.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.640+0000 7efee1e00640 1 -- 172.21.15.89:0/432475244 shutdown_connections 2024-09-10T08:47:23.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.640+0000 7efee1e00640 1 --2- 172.21.15.89:0/432475244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efedc10e330 0x7efedc110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.640+0000 7efee1e00640 1 --2- 172.21.15.89:0/432475244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efedc101130 0x7efedc10ddf0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.640+0000 7efee1e00640 1 --2- 172.21.15.89:0/432475244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efedc100780 0x7efedc100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.640+0000 7efee1e00640 1 -- 172.21.15.89:0/432475244 >> 172.21.15.89:0/432475244 conn(0x7efedc0fc610 msgr2=0x7efedc0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:23.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.641+0000 7efee1e00640 1 -- 172.21.15.89:0/432475244 shutdown_connections 2024-09-10T08:47:23.642 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.641+0000 7efee1e00640 1 -- 172.21.15.89:0/432475244 wait complete. 2024-09-10T08:47:23.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.642+0000 7efee1e00640 1 Processor -- start 2024-09-10T08:47:23.643 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.642+0000 7efee1e00640 1 -- start start 2024-09-10T08:47:23.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.643+0000 7efee1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efedc100780 0x7efedc19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:23.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.643+0000 7efee1e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efedc101130 0x7efedc19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:23.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.643+0000 7efee1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efedc10e330 0x7efedc19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:23.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.643+0000 7efee1e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efedc113fe0 con 0x7efedc100780 2024-09-10T08:47:23.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.643+0000 7efee1e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efedc113e60 con 0x7efedc10e330 2024-09-10T08:47:23.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.643+0000 7efee1e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efedc114160 con 0x7efedc101130 2024-09-10T08:47:23.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.643+0000 7efee0a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efedc100780 0x7efedc19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:23.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.643+0000 7efee0a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efedc100780 0x7efedc19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51920/0 (socket says 172.21.15.89:51920) 2024-09-10T08:47:23.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.643+0000 7efee0a00640 1 -- 172.21.15.89:0/1133747659 learned_addr learned my addr 172.21.15.89:0/1133747659 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:23.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.643+0000 7efedbe00640 1 --2- 172.21.15.89:0/1133747659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efedc101130 0x7efedc19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:23.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.643+0000 7efee1400640 1 --2- 172.21.15.89:0/1133747659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efedc10e330 0x7efedc19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:23.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.644+0000 7efee0a00640 1 -- 172.21.15.89:0/1133747659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efedc101130 msgr2=0x7efedc19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:23.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.644+0000 7efee0a00640 1 --2- 172.21.15.89:0/1133747659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efedc101130 0x7efedc19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.644+0000 7efee0a00640 1 -- 172.21.15.89:0/1133747659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efedc10e330 msgr2=0x7efedc19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:23.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.644+0000 7efee0a00640 1 --2- 172.21.15.89:0/1133747659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efedc10e330 0x7efedc19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.644+0000 7efee0a00640 1 -- 172.21.15.89:0/1133747659 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efedc19f450 con 0x7efedc100780 2024-09-10T08:47:23.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.644+0000 7efedbe00640 1 --2- 172.21.15.89:0/1133747659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efedc101130 0x7efedc19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:23.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.644+0000 7efee0a00640 1 --2- 172.21.15.89:0/1133747659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efedc100780 0x7efedc19a4a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7efed000e9f0 tx=0x7efed000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:23.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.644+0000 7efed9600640 1 -- 172.21.15.89:0/1133747659 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed000cd80 con 0x7efedc100780 2024-09-10T08:47:23.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.644+0000 7efed9600640 1 -- 172.21.15.89:0/1133747659 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed0010910 con 0x7efedc100780 2024-09-10T08:47:23.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.644+0000 7efee1e00640 1 -- 172.21.15.89:0/1133747659 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efedc19f740 con 0x7efedc100780 2024-09-10T08:47:23.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.644+0000 7efed9600640 1 -- 172.21.15.89:0/1133747659 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0018990 con 0x7efedc100780 2024-09-10T08:47:23.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.645+0000 7efee1e00640 1 -- 172.21.15.89:0/1133747659 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efedc1a7020 con 0x7efedc100780 2024-09-10T08:47:23.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.648+0000 7efed9600640 1 -- 172.21.15.89:0/1133747659 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7efed0010ac0 con 0x7efedc100780 2024-09-10T08:47:23.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.648+0000 7efee5ec0740 1 Processor -- start 2024-09-10T08:47:23.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.648+0000 7efee5ec0740 1 -- start start 2024-09-10T08:47:23.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.648+0000 7efee5ec0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f24511520 0x560f24511900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:23.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.648+0000 7efee5ec0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f24511e40 0x560f2451c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:23.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.648+0000 7efee5ec0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f2451c700 0x560f2451eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:23.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.648+0000 7efee5ec0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560f24477650 con 0x560f24511520 2024-09-10T08:47:23.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.648+0000 7efee5ec0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560f2441ef10 con 0x560f24511e40 2024-09-10T08:47:23.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.648+0000 7efee5ec0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560f24473610 con 0x560f2451c700 2024-09-10T08:47:23.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.649+0000 7efee0a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f24511e40 0x560f2451c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:23.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.649+0000 7efee0a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f24511e40 0x560f2451c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:34394/0 (socket says 172.21.15.89:34394) 2024-09-10T08:47:23.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.649+0000 7efee0a00640 1 -- 172.21.15.89:0/3099994701 learned_addr learned my addr 172.21.15.89:0/3099994701 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:23.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.649+0000 7efee1400640 1 --2- 172.21.15.89:0/3099994701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f24511520 0x560f24511900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:23.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.649+0000 7efedbe00640 1 --2- 172.21.15.89:0/3099994701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f2451c700 0x560f2451eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:23.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.649+0000 7efee0a00640 1 -- 172.21.15.89:0/3099994701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f2451c700 msgr2=0x560f2451eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:23.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.649+0000 7efee0a00640 1 --2- 172.21.15.89:0/3099994701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f2451c700 0x560f2451eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.649+0000 7efee0a00640 1 -- 172.21.15.89:0/3099994701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f24511520 msgr2=0x560f24511900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:23.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.649+0000 7efee0a00640 1 --2- 172.21.15.89:0/3099994701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f24511520 0x560f24511900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.649+0000 7efee0a00640 1 -- 172.21.15.89:0/3099994701 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f24186580 con 0x560f24511e40 2024-09-10T08:47:23.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.649+0000 7efee1400640 1 --2- 172.21.15.89:0/3099994701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f24511520 0x560f24511900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:23.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.650+0000 7efee0a00640 1 --2- 172.21.15.89:0/3099994701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f24511e40 0x560f2451c1c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efed00d5c20 tx=0x7efed00d7670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=612cc7e3013c9c8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:23.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.650+0000 7efecdc00640 1 -- 172.21.15.89:0/3099994701 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed00de070 con 0x560f24511e40 2024-09-10T08:47:23.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.650+0000 7efed9600640 1 --2- 172.21.15.89:0/1133747659 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efec40a2b80 0x7efec40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:23.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.650+0000 7efecdc00640 1 -- 172.21.15.89:0/3099994701 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed00cfd70 con 0x560f24511e40 2024-09-10T08:47:23.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.650+0000 7efecdc00640 1 -- 172.21.15.89:0/3099994701 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed00da790 con 0x560f24511e40 2024-09-10T08:47:23.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.650+0000 7efed9600640 1 -- 172.21.15.89:0/1133747659 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7efed00cf780 con 0x7efedc100780 2024-09-10T08:47:23.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.651+0000 7efee5ec0740 1 -- 172.21.15.89:0/3099994701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f24511e40 msgr2=0x560f2451c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:23.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.651+0000 7efee5ec0740 1 --2- 172.21.15.89:0/3099994701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f24511e40 0x560f2451c1c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efed00d5c20 tx=0x7efed00d7670 comp rx=0 tx=0).stop 2024-09-10T08:47:23.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.651+0000 7efee1400640 1 --2- 172.21.15.89:0/1133747659 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efec40a2b80 0x7efec40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:23.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.651+0000 7efee5ec0740 1 -- 172.21.15.89:0/3099994701 shutdown_connections 2024-09-10T08:47:23.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.651+0000 7efee5ec0740 1 --2- 172.21.15.89:0/3099994701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f2451c700 0x560f2451eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.651+0000 7efee5ec0740 1 --2- 172.21.15.89:0/3099994701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f24511e40 0x560f2451c1c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.651+0000 7efee5ec0740 1 --2- 172.21.15.89:0/3099994701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f24511520 0x560f24511900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.651+0000 7efee5ec0740 1 -- 172.21.15.89:0/3099994701 >> 172.21.15.89:0/3099994701 conn(0x560f2450dab0 msgr2=0x560f2450de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:23.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.651+0000 7efee5ec0740 1 -- 172.21.15.89:0/3099994701 shutdown_connections 2024-09-10T08:47:23.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.651+0000 7efee5ec0740 1 -- 172.21.15.89:0/3099994701 wait complete. 2024-09-10T08:47:23.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.652+0000 7efee1400640 1 --2- 172.21.15.89:0/1133747659 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efec40a2b80 0x7efec40a5040 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efedc102960 tx=0x7efed40096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:23.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.652+0000 7efee5ec0740 1 -- start start 2024-09-10T08:47:23.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.652+0000 7efee5ec0740 1 Processor -- start 2024-09-10T08:47:23.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.653+0000 7efee5ec0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f24511520 0x560f245195a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:23.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.653+0000 7efee5ec0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f24511e40 0x560f24519ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:23.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.653+0000 7efee5ec0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f2451c700 0x560f2451a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:23.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.653+0000 7efee5ec0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560f2451f2d0 con 0x560f24511e40 2024-09-10T08:47:23.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.653+0000 7efee5ec0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560f24432350 con 0x560f2451c700 2024-09-10T08:47:23.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.653+0000 7efee5ec0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560f2451f450 con 0x560f24511520 2024-09-10T08:47:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.653+0000 7efee5ec0740 10 client.0 _ll_register_callbacks cb 0x560f24455290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.653+0000 7efee0a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f24511520 0x560f245195a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.653+0000 7efee0a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f24511520 0x560f245195a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:56474/0 (socket says 172.21.15.89:56474) 2024-09-10T08:47:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.653+0000 7efee0a00640 1 -- 172.21.15.89:0/983536554 learned_addr learned my addr 172.21.15.89:0/983536554 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.653+0000 7efee1400640 1 --2- 172.21.15.89:0/983536554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f2451c700 0x560f2451a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.653+0000 7efedbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f24511e40 0x560f24519ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.654+0000 7efee1400640 1 -- 172.21.15.89:0/983536554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f24511520 msgr2=0x560f245195a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.654+0000 7efee1400640 1 --2- 172.21.15.89:0/983536554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f24511520 0x560f245195a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.654+0000 7efee1400640 1 -- 172.21.15.89:0/983536554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f24511e40 msgr2=0x560f24519ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.654+0000 7efee1400640 1 --2- 172.21.15.89:0/983536554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f24511e40 0x560f24519ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.654+0000 7efee1400640 1 -- 172.21.15.89:0/983536554 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f24250960 con 0x560f2451c700 2024-09-10T08:47:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.654+0000 7efee1400640 1 --2- 172.21.15.89:0/983536554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f2451c700 0x560f2451a020 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efed4010920 tx=0x7efed4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6daa3d90d0b8b7a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.654+0000 7efec2000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:47:23.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.654+0000 7efec2000640 1 -- 172.21.15.89:0/983536554 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed400fac0 con 0x560f2451c700 2024-09-10T08:47:23.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.655+0000 7efee5ec0740 1 -- 172.21.15.89:0/983536554 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed00cfd70 con 0x560f2451c700 2024-09-10T08:47:23.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.655+0000 7efec2000640 1 -- 172.21.15.89:0/983536554 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7efed400fc60 con 0x560f2451c700 2024-09-10T08:47:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.655+0000 7efec2000640 1 -- 172.21.15.89:0/983536554 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efed4030050 con 0x560f2451c700 2024-09-10T08:47:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.655+0000 7efee5ec0740 10 client.26750 fetch_fsmap learned FSMap version 30 2024-09-10T08:47:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.655+0000 7efee5ec0740 1 -- 172.21.15.89:0/983536554 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560f2452af50 con 0x560f2451c700 2024-09-10T08:47:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.656+0000 7efec2000640 1 -- 172.21.15.89:0/983536554 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed00cfd70 con 0x560f2451c700 2024-09-10T08:47:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.656+0000 7efec2000640 1 -- 172.21.15.89:0/983536554 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7efed401f4a0 con 0x560f2451c700 2024-09-10T08:47:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.656+0000 7efee5ec0740 10 client.26750 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T08:47:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.656+0000 7efee5ec0740 10 client.26750 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:47:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.656+0000 7efee5ec0740 20 client.26750 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.656+0000 7efee5ec0740 1 --2- 172.21.15.89:0/983536554 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560f2452c360 0x560f2452e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.656+0000 7efee5ec0740 4 client.26750 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.656+0000 7efee5ec0740 1 -- 172.21.15.89:0/983536554 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f24484bc0 con 0x560f2452c360 2024-09-10T08:47:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.656+0000 7efee0a00640 1 --2- 172.21.15.89:0/983536554 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560f2452c360 0x560f2452e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:23.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.657+0000 7efee0a00640 1 --2- 172.21.15.89:0/983536554 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560f2452c360 0x560f2452e770 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bdd9a3693f9278c server_cookie=4f7c43d690a3b7a2 in_seq=0 out_seq=0 2024-09-10T08:47:23.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.657+0000 7efec2000640 10 client.26750 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:23.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.658+0000 7efec2000640 1 -- 172.21.15.89:0/983536554 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f24484bc0 con 0x560f2452c360 2024-09-10T08:47:23.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.658+0000 7efec2000640 10 client.26750 handle_command_reply: tid=0 2024-09-10T08:47:23.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.658+0000 7efee5ec0740 1 client.26750 shutdown 2024-09-10T08:47:23.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.658+0000 7efee5ec0740 1 -- 172.21.15.89:0/983536554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f2451c700 msgr2=0x560f2451a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:23.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.658+0000 7efee5ec0740 1 --2- 172.21.15.89:0/983536554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f2451c700 0x560f2451a020 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efed4010920 tx=0x7efed4004a40 comp rx=0 tx=0).stop 2024-09-10T08:47:23.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.658+0000 7efee5ec0740 1 -- 172.21.15.89:0/983536554 shutdown_connections 2024-09-10T08:47:23.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.658+0000 7efee5ec0740 1 --2- 172.21.15.89:0/983536554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f2451c700 0x560f2451a020 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.658+0000 7efee5ec0740 1 --2- 172.21.15.89:0/983536554 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560f2452c360 0x560f2452e770 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.658+0000 7efee5ec0740 1 --2- 172.21.15.89:0/983536554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f24511e40 0x560f24519ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.658+0000 7efee5ec0740 1 --2- 172.21.15.89:0/983536554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f24511520 0x560f245195a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.658+0000 7efec2000640 0 client.26750 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:23.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.658+0000 7efee5ec0740 1 -- 172.21.15.89:0/983536554 >> 172.21.15.89:0/983536554 conn(0x560f2450dab0 msgr2=0x560f2450ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:23.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.658+0000 7efee5ec0740 1 -- 172.21.15.89:0/983536554 shutdown_connections 2024-09-10T08:47:23.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.658+0000 7efee5ec0740 1 -- 172.21.15.89:0/983536554 wait complete. 2024-09-10T08:47:23.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.658+0000 7efee5ec0740 20 client.26750 trim_cache size 0 max 16384 2024-09-10T08:47:23.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.676+0000 7efee5ec0740 1 Processor -- start 2024-09-10T08:47:23.677 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.676+0000 7efee5ec0740 1 -- start start 2024-09-10T08:47:23.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.676+0000 7efee5ec0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f24511520 0x560f2451b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:23.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.677+0000 7efee5ec0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f24511e40 0x560f2450c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:23.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.677+0000 7efee5ec0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2451c700 0x560f24518e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:23.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.677+0000 7efee5ec0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efedc1b3780 con 0x560f2451c700 2024-09-10T08:47:23.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.677+0000 7efee5ec0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efedc1b1050 con 0x560f24511e40 2024-09-10T08:47:23.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.677+0000 7efee5ec0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efedc1aeea0 con 0x560f24511520 2024-09-10T08:47:23.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.677+0000 7efee1400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2451c700 0x560f24518e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:23.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.677+0000 7efee0a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f24511520 0x560f2451b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:23.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.677+0000 7efee0a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f24511520 0x560f2451b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:56482/0 (socket says 172.21.15.89:56482) 2024-09-10T08:47:23.678 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.677+0000 7efee1400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2451c700 0x560f24518e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51944/0 (socket says 172.21.15.89:51944) 2024-09-10T08:47:23.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.677+0000 7efedbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f24511e40 0x560f2450c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:23.679 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.677+0000 7efee0a00640 1 -- 172.21.15.89:0/2571156806 learned_addr learned my addr 172.21.15.89:0/2571156806 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:23.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.677+0000 7efee0a00640 1 -- 172.21.15.89:0/2571156806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f24511e40 msgr2=0x560f2450c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:23.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.677+0000 7efee0a00640 1 --2- 172.21.15.89:0/2571156806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f24511e40 0x560f2450c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.677+0000 7efee0a00640 1 -- 172.21.15.89:0/2571156806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2451c700 msgr2=0x560f24518e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:23.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.677+0000 7efee0a00640 1 --2- 172.21.15.89:0/2571156806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2451c700 0x560f24518e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.677+0000 7efee0a00640 1 -- 172.21.15.89:0/2571156806 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f2450d550 con 0x560f24511520 2024-09-10T08:47:23.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.677+0000 7efee1400640 1 --2- 172.21.15.89:0/2571156806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2451c700 0x560f24518e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:23.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.678+0000 7efee0a00640 1 --2- 172.21.15.89:0/2571156806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f24511520 0x560f2451b360 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efed00d5c20 tx=0x7efed00d1a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=335b9173b3e71f83 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:23.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.678+0000 7efecd200640 1 -- 172.21.15.89:0/2571156806 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed00de070 con 0x560f24511520 2024-09-10T08:47:23.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.678+0000 7efecd200640 1 -- 172.21.15.89:0/2571156806 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed009bc60 con 0x560f24511520 2024-09-10T08:47:23.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.678+0000 7efecd200640 1 -- 172.21.15.89:0/2571156806 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed00a4820 con 0x560f24511520 2024-09-10T08:47:23.680 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.679+0000 7efee5ec0740 1 -- 172.21.15.89:0/2571156806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f24511520 msgr2=0x560f2451b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:23.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.679+0000 7efee5ec0740 1 --2- 172.21.15.89:0/2571156806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f24511520 0x560f2451b360 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efed00d5c20 tx=0x7efed00d1a10 comp rx=0 tx=0).stop 2024-09-10T08:47:23.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.679+0000 7efee5ec0740 1 -- 172.21.15.89:0/2571156806 shutdown_connections 2024-09-10T08:47:23.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.679+0000 7efee5ec0740 1 --2- 172.21.15.89:0/2571156806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f2451c700 0x560f24518e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.679+0000 7efee5ec0740 1 --2- 172.21.15.89:0/2571156806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f24511e40 0x560f2450c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.679+0000 7efee5ec0740 1 --2- 172.21.15.89:0/2571156806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f24511520 0x560f2451b360 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.679+0000 7efee5ec0740 1 -- 172.21.15.89:0/2571156806 >> 172.21.15.89:0/2571156806 conn(0x560f2450dab0 msgr2=0x560f24519d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:23.681 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.679+0000 7efee5ec0740 1 -- 172.21.15.89:0/2571156806 shutdown_connections 2024-09-10T08:47:23.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.679+0000 7efee5ec0740 1 -- 172.21.15.89:0/2571156806 wait complete. 2024-09-10T08:47:23.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.679+0000 7efee5ec0740 1 -- start start 2024-09-10T08:47:23.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.680+0000 7efee5ec0740 1 Processor -- start 2024-09-10T08:47:23.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.680+0000 7efee5ec0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f24511520 0x560f24524b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:23.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.680+0000 7efee5ec0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f24511e40 0x560f2451fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:23.682 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.680+0000 7efee5ec0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f2451c700 0x560f245203e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:23.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.680+0000 7efee5ec0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efedc1af510 con 0x560f24511520 2024-09-10T08:47:23.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.680+0000 7efee5ec0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efedc1b0980 con 0x560f24511e40 2024-09-10T08:47:23.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.680+0000 7efee5ec0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efedc1afbe0 con 0x560f2451c700 2024-09-10T08:47:23.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.680+0000 7efee5ec0740 10 client.0 _ll_register_callbacks cb 0x7efed40105a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:23.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.680+0000 7efee1400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f2451c700 0x560f245203e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:23.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.680+0000 7efee1400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f2451c700 0x560f245203e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:56494/0 (socket says 172.21.15.89:56494) 2024-09-10T08:47:23.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.680+0000 7efee1400640 1 -- 172.21.15.89:0/3269211613 learned_addr learned my addr 172.21.15.89:0/3269211613 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:23.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.680+0000 7efee0a00640 1 --2- 172.21.15.89:0/3269211613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f24511520 0x560f24524b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:23.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.680+0000 7efedbe00640 1 --2- 172.21.15.89:0/3269211613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f24511e40 0x560f2451fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:23.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.681+0000 7efee1400640 1 -- 172.21.15.89:0/3269211613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f24511e40 msgr2=0x560f2451fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:23.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.681+0000 7efee1400640 1 --2- 172.21.15.89:0/3269211613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f24511e40 0x560f2451fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.681+0000 7efee1400640 1 -- 172.21.15.89:0/3269211613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f24511520 msgr2=0x560f24524b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:23.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.681+0000 7efee1400640 1 --2- 172.21.15.89:0/3269211613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f24511520 0x560f24524b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.681+0000 7efee1400640 1 -- 172.21.15.89:0/3269211613 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f245209a0 con 0x560f2451c700 2024-09-10T08:47:23.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.681+0000 7efee1400640 1 --2- 172.21.15.89:0/3269211613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f2451c700 0x560f245203e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efed40108f0 tx=0x7efed401fd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61dfab22bf3bcc1c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:23.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.681+0000 7efec2000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:47:23.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.681+0000 7efec2000640 1 -- 172.21.15.89:0/3269211613 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed402d040 con 0x560f2451c700 2024-09-10T08:47:23.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.681+0000 7efee5ec0740 1 -- 172.21.15.89:0/3269211613 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed009bc60 con 0x560f2451c700 2024-09-10T08:47:23.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.681+0000 7efec2000640 1 -- 172.21.15.89:0/3269211613 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7efed400f990 con 0x560f2451c700 2024-09-10T08:47:23.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.682+0000 7efec2000640 1 -- 172.21.15.89:0/3269211613 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efed401f680 con 0x560f2451c700 2024-09-10T08:47:23.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.682+0000 7efee5ec0740 10 client.26704 fetch_fsmap learned FSMap version 30 2024-09-10T08:47:23.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.682+0000 7efee5ec0740 1 -- 172.21.15.89:0/3269211613 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560f24596dc0 con 0x560f2451c700 2024-09-10T08:47:23.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.682+0000 7efec2000640 1 -- 172.21.15.89:0/3269211613 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed009bc60 con 0x560f2451c700 2024-09-10T08:47:23.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.682+0000 7efec2000640 1 -- 172.21.15.89:0/3269211613 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7efed403fc90 con 0x560f2451c700 2024-09-10T08:47:23.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.682+0000 7efee5ec0740 10 client.26704 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T08:47:23.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.682+0000 7efee5ec0740 10 client.26704 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:47:23.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.682+0000 7efee5ec0740 20 client.26704 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:23.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.682+0000 7efee5ec0740 1 --2- 172.21.15.89:0/3269211613 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560f245974b0 0x560f245998d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:23.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.682+0000 7efee5ec0740 4 client.26704 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7ca83d3b", "release": true, "await": true}] 2024-09-10T08:47:23.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.682+0000 7efee5ec0740 1 -- 172.21.15.89:0/3269211613 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7ca83d3b", "release": true, "await": true}) -- 0x560f2447f0d0 con 0x560f245974b0 2024-09-10T08:47:23.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.683+0000 7efee0a00640 1 --2- 172.21.15.89:0/3269211613 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560f245974b0 0x560f245998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:23.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.683+0000 7efee0a00640 1 --2- 172.21.15.89:0/3269211613 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560f245974b0 0x560f245998d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59640d4a09ba17ae server_cookie=9bdba65673a1d7c in_seq=0 out_seq=0 2024-09-10T08:47:23.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:23.683+0000 7efec2000640 10 client.26704 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:23.968 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:24.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.513+0000 7efec2000640 1 -- 172.21.15.89:0/3269211613 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560f2447f0d0 con 0x560f245974b0 2024-09-10T08:47:24.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.513+0000 7efec2000640 10 client.26704 handle_command_reply: tid=0 2024-09-10T08:47:24.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.513+0000 7efee5ec0740 1 client.26704 shutdown 2024-09-10T08:47:24.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.514+0000 7efee5ec0740 1 -- 172.21.15.89:0/3269211613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f2451c700 msgr2=0x560f245203e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.514+0000 7efee5ec0740 1 --2- 172.21.15.89:0/3269211613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f2451c700 0x560f245203e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efed40108f0 tx=0x7efed401fd90 comp rx=0 tx=0).stop 2024-09-10T08:47:24.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.514+0000 7efee5ec0740 1 -- 172.21.15.89:0/3269211613 shutdown_connections 2024-09-10T08:47:24.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.514+0000 7efee5ec0740 1 --2- 172.21.15.89:0/3269211613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f2451c700 0x560f245203e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.515 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.514+0000 7efee5ec0740 1 --2- 172.21.15.89:0/3269211613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f24511e40 0x560f2451fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.514+0000 7efee5ec0740 1 --2- 172.21.15.89:0/3269211613 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x560f245974b0 0x560f245998d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.514+0000 7efee5ec0740 1 --2- 172.21.15.89:0/3269211613 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f24511520 0x560f24524b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.514+0000 7efee5ec0740 1 -- 172.21.15.89:0/3269211613 >> 172.21.15.89:0/3269211613 conn(0x560f2450dab0 msgr2=0x560f24527520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:24.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.514+0000 7efec2000640 0 client.26704 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:24.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.514+0000 7efee5ec0740 1 -- 172.21.15.89:0/3269211613 shutdown_connections 2024-09-10T08:47:24.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.514+0000 7efee5ec0740 1 -- 172.21.15.89:0/3269211613 wait complete. 2024-09-10T08:47:24.516 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.514+0000 7efee5ec0740 20 client.26704 trim_cache size 0 max 16384 2024-09-10T08:47:24.516 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:47:24.517 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 30, 2024-09-10T08:47:24.517 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24554, 2024-09-10T08:47:24.517 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 20, 2024-09-10T08:47:24.517 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": { 2024-09-10T08:47:24.517 INFO:teuthology.orchestra.run.smithi089.stdout: "7ca83d3b": { 2024-09-10T08:47:24.517 INFO:teuthology.orchestra.run.smithi089.stdout: "version": 20, 2024-09-10T08:47:24.517 INFO:teuthology.orchestra.run.smithi089.stdout: "age_ref": 0.0, 2024-09-10T08:47:24.517 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:47:24.517 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RELEASED", 2024-09-10T08:47:24.517 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:47:24.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:47:24.517 INFO:teuthology.orchestra.run.smithi089.stdout: "timeout": 90.0, 2024-09-10T08:47:24.517 INFO:teuthology.orchestra.run.smithi089.stdout: "expiration": 140.1, 2024-09-10T08:47:24.517 INFO:teuthology.orchestra.run.smithi089.stdout: "members": { 2024-09-10T08:47:24.518 INFO:teuthology.orchestra.run.smithi089.stdout: "file:/": { 2024-09-10T08:47:24.518 INFO:teuthology.orchestra.run.smithi089.stdout: "excluded": false, 2024-09-10T08:47:24.518 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:47:24.518 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RELEASED", 2024-09-10T08:47:24.518 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:47:24.518 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:24.518 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:24.518 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:24.518 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:24.518 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:24.518 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:47:24.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.515+0000 7efee1e00640 1 -- 172.21.15.89:0/1133747659 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efec40a2b80 msgr2=0x7efec40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.515+0000 7efee1e00640 1 --2- 172.21.15.89:0/1133747659 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efec40a2b80 0x7efec40a5040 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efedc102960 tx=0x7efed40096a0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.515+0000 7efee1e00640 1 -- 172.21.15.89:0/1133747659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efedc100780 msgr2=0x7efedc19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.515+0000 7efee1e00640 1 --2- 172.21.15.89:0/1133747659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efedc100780 0x7efedc19a4a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7efed000e9f0 tx=0x7efed000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.515+0000 7efee1e00640 1 -- 172.21.15.89:0/1133747659 shutdown_connections 2024-09-10T08:47:24.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.515+0000 7efee1e00640 1 --2- 172.21.15.89:0/1133747659 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7efec40a2b80 0x7efec40a5040 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.515+0000 7efee1e00640 1 --2- 172.21.15.89:0/1133747659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efedc10e330 0x7efedc19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.515+0000 7efee1e00640 1 --2- 172.21.15.89:0/1133747659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efedc101130 0x7efedc19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.519 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.515+0000 7efee1e00640 1 --2- 172.21.15.89:0/1133747659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efedc100780 0x7efedc19a4a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.515+0000 7efee1e00640 1 -- 172.21.15.89:0/1133747659 >> 172.21.15.89:0/1133747659 conn(0x7efedc0fc610 msgr2=0x7efedc0fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:24.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.516+0000 7efee1e00640 1 -- 172.21.15.89:0/1133747659 shutdown_connections 2024-09-10T08:47:24.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.516+0000 7efee1e00640 1 -- 172.21.15.89:0/1133747659 wait complete. 2024-09-10T08:47:24.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.619+0000 7f683f200640 1 -- 172.21.15.89:0/2826370130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68401079c0 msgr2=0x7f6840107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.619+0000 7f683f200640 1 --2- 172.21.15.89:0/2826370130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68401079c0 0x7f6840107da0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6830009900 tx=0x7f683002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.619+0000 7f683f200640 1 -- 172.21.15.89:0/2826370130 shutdown_connections 2024-09-10T08:47:24.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.619+0000 7f683f200640 1 --2- 172.21.15.89:0/2826370130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6840102360 0x7f684010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.619+0000 7f683f200640 1 --2- 172.21.15.89:0/2826370130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68401019c0 0x7f6840101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.619+0000 7f683f200640 1 --2- 172.21.15.89:0/2826370130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68401079c0 0x7f6840107da0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.619+0000 7f683f200640 1 -- 172.21.15.89:0/2826370130 >> 172.21.15.89:0/2826370130 conn(0x7f68400fd740 msgr2=0x7f68400ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:24.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.619+0000 7f683f200640 1 -- 172.21.15.89:0/2826370130 shutdown_connections 2024-09-10T08:47:24.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.619+0000 7f683f200640 1 -- 172.21.15.89:0/2826370130 wait complete. 2024-09-10T08:47:24.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.620+0000 7f683f200640 1 Processor -- start 2024-09-10T08:47:24.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.620+0000 7f683f200640 1 -- start start 2024-09-10T08:47:24.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.621+0000 7f683f200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68401019c0 0x7f68401982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.621+0000 7f683f200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6840102360 0x7f68401987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.621+0000 7f683f200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68401079c0 0x7f684019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.621+0000 7f683f200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6840111dc0 con 0x7f68401079c0 2024-09-10T08:47:24.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.621+0000 7f683f200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6840111c40 con 0x7f68401019c0 2024-09-10T08:47:24.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.621+0000 7f683f200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6840111f40 con 0x7f6840102360 2024-09-10T08:47:24.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.621+0000 7f683de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68401019c0 0x7f68401982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.621+0000 7f683de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68401019c0 0x7f68401982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:34432/0 (socket says 172.21.15.89:34432) 2024-09-10T08:47:24.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.621+0000 7f683de00640 1 -- 172.21.15.89:0/831050650 learned_addr learned my addr 172.21.15.89:0/831050650 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:24.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.621+0000 7f683d400640 1 --2- 172.21.15.89:0/831050650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6840102360 0x7f68401987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.621+0000 7f683e800640 1 --2- 172.21.15.89:0/831050650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68401079c0 0x7f684019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.621+0000 7f683de00640 1 -- 172.21.15.89:0/831050650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6840102360 msgr2=0x7f68401987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.621+0000 7f683de00640 1 --2- 172.21.15.89:0/831050650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6840102360 0x7f68401987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.621+0000 7f683de00640 1 -- 172.21.15.89:0/831050650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68401079c0 msgr2=0x7f684019cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.621+0000 7f683de00640 1 --2- 172.21.15.89:0/831050650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68401079c0 0x7f684019cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.621+0000 7f683de00640 1 -- 172.21.15.89:0/831050650 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684019d250 con 0x7f68401019c0 2024-09-10T08:47:24.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.621+0000 7f683d400640 1 --2- 172.21.15.89:0/831050650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6840102360 0x7f68401987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:24.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.622+0000 7f683de00640 1 --2- 172.21.15.89:0/831050650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68401019c0 0x7f68401982a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6830006010 tx=0x7f6830003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:24.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.622+0000 7f6836a00640 1 -- 172.21.15.89:0/831050650 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683003f070 con 0x7f68401019c0 2024-09-10T08:47:24.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.622+0000 7f6836a00640 1 -- 172.21.15.89:0/831050650 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6830031c30 con 0x7f68401019c0 2024-09-10T08:47:24.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.622+0000 7f683f200640 1 -- 172.21.15.89:0/831050650 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f684019d4e0 con 0x7f68401019c0 2024-09-10T08:47:24.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.622+0000 7f6836a00640 1 -- 172.21.15.89:0/831050650 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830035030 con 0x7f68401019c0 2024-09-10T08:47:24.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.622+0000 7f683f200640 1 -- 172.21.15.89:0/831050650 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68401a4e20 con 0x7f68401019c0 2024-09-10T08:47:24.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.626+0000 7f6847a6d740 1 Processor -- start 2024-09-10T08:47:24.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.626+0000 7f6847a6d740 1 -- start start 2024-09-10T08:47:24.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.627+0000 7f6836a00640 1 -- 172.21.15.89:0/831050650 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6830042070 con 0x7f68401019c0 2024-09-10T08:47:24.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.627+0000 7f6847a6d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aa3d50 0x555668aa4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.627+0000 7f6847a6d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668aa4670 0x555668aae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.627+0000 7f6847a6d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aaef30 0x555668ab1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.627+0000 7f6847a6d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555668a0a3e0 con 0x555668aaef30 2024-09-10T08:47:24.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.627+0000 7f6847a6d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5556689b1d80 con 0x555668aa4670 2024-09-10T08:47:24.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.627+0000 7f6847a6d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555668a063a0 con 0x555668aa3d50 2024-09-10T08:47:24.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.627+0000 7f683e800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aa3d50 0x555668aa4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.627+0000 7f683de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668aa4670 0x555668aae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.627+0000 7f683e800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aa3d50 0x555668aa4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:56534/0 (socket says 172.21.15.89:56534) 2024-09-10T08:47:24.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.627+0000 7f683e800640 1 -- 172.21.15.89:0/1169988914 learned_addr learned my addr 172.21.15.89:0/1169988914 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:24.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.627+0000 7f683d400640 1 --2- 172.21.15.89:0/1169988914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aaef30 0x555668ab1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.627+0000 7f683e800640 1 -- 172.21.15.89:0/1169988914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668aa4670 msgr2=0x555668aae9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.628+0000 7f683e800640 1 --2- 172.21.15.89:0/1169988914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668aa4670 0x555668aae9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.628+0000 7f683e800640 1 -- 172.21.15.89:0/1169988914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aaef30 msgr2=0x555668ab1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.628+0000 7f683e800640 1 --2- 172.21.15.89:0/1169988914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aaef30 0x555668ab1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.628+0000 7f683e800640 1 -- 172.21.15.89:0/1169988914 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556689d0ae0 con 0x555668aa3d50 2024-09-10T08:47:24.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.628+0000 7f683de00640 1 --2- 172.21.15.89:0/1169988914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668aa4670 0x555668aae9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:24.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.628+0000 7f683e800640 1 --2- 172.21.15.89:0/1169988914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aa3d50 0x555668aa4130 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6838009f60 tx=0x7f6838009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=795e95987be60c52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:24.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.628+0000 7f682b400640 1 -- 172.21.15.89:0/1169988914 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683801e070 con 0x555668aa3d50 2024-09-10T08:47:24.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.628+0000 7f682b400640 1 -- 172.21.15.89:0/1169988914 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6838003c30 con 0x555668aa3d50 2024-09-10T08:47:24.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.628+0000 7f682b400640 1 -- 172.21.15.89:0/1169988914 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683800fa20 con 0x555668aa3d50 2024-09-10T08:47:24.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.629+0000 7f6836a00640 1 --2- 172.21.15.89:0/831050650 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f68240a2c00 0x7f68240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.629+0000 7f6836a00640 1 -- 172.21.15.89:0/831050650 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f68300ef720 con 0x7f68401019c0 2024-09-10T08:47:24.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.629+0000 7f683de00640 1 --2- 172.21.15.89:0/831050650 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f68240a2c00 0x7f68240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.629+0000 7f6847a6d740 1 -- 172.21.15.89:0/1169988914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aa3d50 msgr2=0x555668aa4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.629+0000 7f6847a6d740 1 --2- 172.21.15.89:0/1169988914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aa3d50 0x555668aa4130 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6838009f60 tx=0x7f6838009f90 comp rx=0 tx=0).stop 2024-09-10T08:47:24.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.629+0000 7f6847a6d740 1 -- 172.21.15.89:0/1169988914 shutdown_connections 2024-09-10T08:47:24.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.629+0000 7f6847a6d740 1 --2- 172.21.15.89:0/1169988914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aaef30 0x555668ab1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.629+0000 7f6847a6d740 1 --2- 172.21.15.89:0/1169988914 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668aa4670 0x555668aae9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.629+0000 7f6847a6d740 1 --2- 172.21.15.89:0/1169988914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aa3d50 0x555668aa4130 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.629+0000 7f6847a6d740 1 -- 172.21.15.89:0/1169988914 >> 172.21.15.89:0/1169988914 conn(0x555668aa02e0 msgr2=0x555668aa06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:24.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.630+0000 7f6847a6d740 1 -- 172.21.15.89:0/1169988914 shutdown_connections 2024-09-10T08:47:24.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.630+0000 7f683de00640 1 --2- 172.21.15.89:0/831050650 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f68240a2c00 0x7f68240a50c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f68300ed1c0 tx=0x7f68300f5550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:24.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.630+0000 7f6847a6d740 1 -- 172.21.15.89:0/1169988914 wait complete. 2024-09-10T08:47:24.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.630+0000 7f6847a6d740 1 -- start start 2024-09-10T08:47:24.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.630+0000 7f6847a6d740 1 Processor -- start 2024-09-10T08:47:24.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.631+0000 7f6847a6d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668aa3d50 0x555668aabea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.631+0000 7f6847a6d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aa4670 0x555668aac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.631+0000 7f6847a6d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aaef30 0x555668aac920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.631+0000 7f6847a6d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555668ab1ba0 con 0x555668aaef30 2024-09-10T08:47:24.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.631+0000 7f6847a6d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5556689c50f0 con 0x555668aa3d50 2024-09-10T08:47:24.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.631+0000 7f6847a6d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555668ab1d20 con 0x555668aa4670 2024-09-10T08:47:24.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.631+0000 7f6847a6d740 10 client.0 _ll_register_callbacks cb 0x5556689b69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:24.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.631+0000 7f683d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aa4670 0x555668aac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.631+0000 7f683d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aa4670 0x555668aac3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:56550/0 (socket says 172.21.15.89:56550) 2024-09-10T08:47:24.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.631+0000 7f683d400640 1 -- 172.21.15.89:0/1753154930 learned_addr learned my addr 172.21.15.89:0/1753154930 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:24.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.631+0000 7f683e800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668aa3d50 0x555668aabea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.632+0000 7f683e800640 1 --2- 172.21.15.89:0/1753154930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aaef30 0x555668aac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.632+0000 7f683d400640 1 -- 172.21.15.89:0/1753154930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668aa3d50 msgr2=0x555668aabea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.632+0000 7f683d400640 1 --2- 172.21.15.89:0/1753154930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668aa3d50 0x555668aabea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.632+0000 7f683d400640 1 -- 172.21.15.89:0/1753154930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aaef30 msgr2=0x555668aac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.632+0000 7f683d400640 1 --2- 172.21.15.89:0/1753154930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aaef30 0x555668aac920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.632+0000 7f683d400640 1 -- 172.21.15.89:0/1753154930 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55566877b3c0 con 0x555668aa4670 2024-09-10T08:47:24.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.632+0000 7f683e800640 1 --2- 172.21.15.89:0/1753154930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668aa3d50 0x555668aabea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:24.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.632+0000 7f683d400640 1 --2- 172.21.15.89:0/1753154930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aa4670 0x555668aac3e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f682c00b960 tx=0x7f682c00be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=503d722c2330dd9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:24.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.632+0000 7f681fe00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:47:24.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.633+0000 7f681fe00640 1 -- 172.21.15.89:0/1753154930 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c01b440 con 0x555668aa4670 2024-09-10T08:47:24.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.633+0000 7f681fe00640 1 -- 172.21.15.89:0/1753154930 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f682c01b5e0 con 0x555668aa4670 2024-09-10T08:47:24.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.633+0000 7f6847a6d740 1 -- 172.21.15.89:0/1753154930 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6838003c30 con 0x555668aa4670 2024-09-10T08:47:24.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.633+0000 7f681fe00640 1 -- 172.21.15.89:0/1753154930 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f682c024050 con 0x555668aa4670 2024-09-10T08:47:24.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.633+0000 7f6847a6d740 10 client.26716 fetch_fsmap learned FSMap version 30 2024-09-10T08:47:24.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.633+0000 7f6847a6d740 1 -- 172.21.15.89:0/1753154930 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555668ac1d70 con 0x555668aa4670 2024-09-10T08:47:24.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.633+0000 7f681fe00640 1 -- 172.21.15.89:0/1753154930 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6838003c30 con 0x555668aa4670 2024-09-10T08:47:24.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.634+0000 7f681fe00640 1 -- 172.21.15.89:0/1753154930 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f682c019020 con 0x555668aa4670 2024-09-10T08:47:24.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.634+0000 7f6847a6d740 10 client.26716 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T08:47:24.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.634+0000 7f6847a6d740 10 client.26716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:24.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.634+0000 7f6847a6d740 20 client.26716 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:24.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.634+0000 7f6847a6d740 1 --2- 172.21.15.89:0/1753154930 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x555668ac32a0 0x555668ac56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.634+0000 7f6847a6d740 4 client.26716 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:24.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.634+0000 7f6847a6d740 1 -- 172.21.15.89:0/1753154930 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555668a17950 con 0x555668ac32a0 2024-09-10T08:47:24.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.634+0000 7f683e800640 1 --2- 172.21.15.89:0/1753154930 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x555668ac32a0 0x555668ac56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.635+0000 7f681fe00640 10 client.26716 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:24.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.635+0000 7f683e800640 1 --2- 172.21.15.89:0/1753154930 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x555668ac32a0 0x555668ac56c0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1186dff915997c9b server_cookie=8155cea64b8efe31 in_seq=0 out_seq=0 2024-09-10T08:47:24.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.635+0000 7f681fe00640 1 -- 172.21.15.89:0/1753154930 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555668a17950 con 0x555668ac32a0 2024-09-10T08:47:24.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.635+0000 7f681fe00640 10 client.26716 handle_command_reply: tid=0 2024-09-10T08:47:24.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.635+0000 7f6847a6d740 1 client.26716 shutdown 2024-09-10T08:47:24.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.636+0000 7f6847a6d740 1 -- 172.21.15.89:0/1753154930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aa4670 msgr2=0x555668aac3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.636+0000 7f6847a6d740 1 --2- 172.21.15.89:0/1753154930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aa4670 0x555668aac3e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f682c00b960 tx=0x7f682c00be20 comp rx=0 tx=0).stop 2024-09-10T08:47:24.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.636+0000 7f6847a6d740 1 -- 172.21.15.89:0/1753154930 shutdown_connections 2024-09-10T08:47:24.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.636+0000 7f6847a6d740 1 --2- 172.21.15.89:0/1753154930 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x555668ac32a0 0x555668ac56c0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.636+0000 7f6847a6d740 1 --2- 172.21.15.89:0/1753154930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aaef30 0x555668aac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.636+0000 7f6847a6d740 1 --2- 172.21.15.89:0/1753154930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aa4670 0x555668aac3e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.636+0000 7f6847a6d740 1 --2- 172.21.15.89:0/1753154930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668aa3d50 0x555668aabea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.636+0000 7f6847a6d740 1 -- 172.21.15.89:0/1753154930 >> 172.21.15.89:0/1753154930 conn(0x555668aa02e0 msgr2=0x555668aa5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:24.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.636+0000 7f681fe00640 0 client.26716 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:24.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.636+0000 7f6847a6d740 1 -- 172.21.15.89:0/1753154930 shutdown_connections 2024-09-10T08:47:24.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.636+0000 7f6847a6d740 1 -- 172.21.15.89:0/1753154930 wait complete. 2024-09-10T08:47:24.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.636+0000 7f6847a6d740 20 client.26716 trim_cache size 0 max 16384 2024-09-10T08:47:24.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.654+0000 7f6847a6d740 1 Processor -- start 2024-09-10T08:47:24.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.654+0000 7f6847a6d740 1 -- start start 2024-09-10T08:47:24.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.654+0000 7f6847a6d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aa3d50 0x555668acafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.654+0000 7f6847a6d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aa4670 0x555668acb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.654+0000 7f6847a6d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668aaef30 0x555668aa9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.655+0000 7f6847a6d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f68401af310 con 0x555668aa4670 2024-09-10T08:47:24.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.655+0000 7f6847a6d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f68401acbe0 con 0x555668aaef30 2024-09-10T08:47:24.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.655+0000 7f6847a6d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f68401aaa30 con 0x555668aa3d50 2024-09-10T08:47:24.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.655+0000 7f683d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aa4670 0x555668acb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.655+0000 7f683d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aa4670 0x555668acb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52010/0 (socket says 172.21.15.89:52010) 2024-09-10T08:47:24.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.655+0000 7f683d400640 1 -- 172.21.15.89:0/396012529 learned_addr learned my addr 172.21.15.89:0/396012529 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:24.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.655+0000 7f683e800640 1 --2- 172.21.15.89:0/396012529 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668aaef30 0x555668aa9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.655+0000 7f683e800640 1 --2- 172.21.15.89:0/396012529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aa3d50 0x555668acafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.655+0000 7f683d400640 1 -- 172.21.15.89:0/396012529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aa3d50 msgr2=0x555668acafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.655+0000 7f683d400640 1 --2- 172.21.15.89:0/396012529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aa3d50 0x555668acafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.655+0000 7f683d400640 1 -- 172.21.15.89:0/396012529 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668aaef30 msgr2=0x555668aa9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.655+0000 7f683d400640 1 --2- 172.21.15.89:0/396012529 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668aaef30 0x555668aa9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.655+0000 7f683d400640 1 -- 172.21.15.89:0/396012529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555668aa0f70 con 0x555668aa4670 2024-09-10T08:47:24.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.656+0000 7f683d400640 1 --2- 172.21.15.89:0/396012529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aa4670 0x555668acb520 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f682c006010 tx=0x7f682c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e21ceb102bfc80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:24.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.656+0000 7f682aa00640 1 -- 172.21.15.89:0/396012529 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c0039c0 con 0x555668aa4670 2024-09-10T08:47:24.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.656+0000 7f682aa00640 1 -- 172.21.15.89:0/396012529 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f682c003f60 con 0x555668aa4670 2024-09-10T08:47:24.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.656+0000 7f682aa00640 1 -- 172.21.15.89:0/396012529 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c0048e0 con 0x555668aa4670 2024-09-10T08:47:24.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.657+0000 7f6847a6d740 1 -- 172.21.15.89:0/396012529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aa4670 msgr2=0x555668acb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.659 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.657+0000 7f6847a6d740 1 --2- 172.21.15.89:0/396012529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aa4670 0x555668acb520 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f682c006010 tx=0x7f682c000bc0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.657+0000 7f6847a6d740 1 -- 172.21.15.89:0/396012529 shutdown_connections 2024-09-10T08:47:24.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.657+0000 7f6847a6d740 1 --2- 172.21.15.89:0/396012529 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668aaef30 0x555668aa9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.657+0000 7f6847a6d740 1 --2- 172.21.15.89:0/396012529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aa4670 0x555668acb520 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.657+0000 7f6847a6d740 1 --2- 172.21.15.89:0/396012529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aa3d50 0x555668acafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.657+0000 7f6847a6d740 1 -- 172.21.15.89:0/396012529 >> 172.21.15.89:0/396012529 conn(0x555668aa02e0 msgr2=0x555668aa5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:24.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.657+0000 7f6847a6d740 1 -- 172.21.15.89:0/396012529 shutdown_connections 2024-09-10T08:47:24.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.657+0000 7f6847a6d740 1 -- 172.21.15.89:0/396012529 wait complete. 2024-09-10T08:47:24.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.657+0000 7f6847a6d740 1 -- start start 2024-09-10T08:47:24.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.658+0000 7f6847a6d740 1 Processor -- start 2024-09-10T08:47:24.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.658+0000 7f6847a6d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aa3d50 0x555668abbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.658+0000 7f6847a6d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aaef30 0x555668ab6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.658+0000 7f6847a6d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668ab7280 0x555668ab7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.660 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.658+0000 7f6847a6d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f68401ab0a0 con 0x555668aa3d50 2024-09-10T08:47:24.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.658+0000 7f6847a6d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f68401ac510 con 0x555668ab7280 2024-09-10T08:47:24.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.658+0000 7f6847a6d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f68401ab770 con 0x555668aaef30 2024-09-10T08:47:24.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.658+0000 7f6847a6d740 10 client.0 _ll_register_callbacks cb 0x7f682c0041f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:24.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.658+0000 7f683e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aa3d50 0x555668abbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.659+0000 7f683e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aa3d50 0x555668abbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52016/0 (socket says 172.21.15.89:52016) 2024-09-10T08:47:24.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.659+0000 7f683e800640 1 -- 172.21.15.89:0/2989512769 learned_addr learned my addr 172.21.15.89:0/2989512769 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:24.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.659+0000 7f683d400640 1 --2- 172.21.15.89:0/2989512769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aaef30 0x555668ab6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.659+0000 7f683e800640 1 --2- 172.21.15.89:0/2989512769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668ab7280 0x555668ab7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.659+0000 7f683d400640 1 -- 172.21.15.89:0/2989512769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668ab7280 msgr2=0x555668ab7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.659+0000 7f683d400640 1 --2- 172.21.15.89:0/2989512769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668ab7280 0x555668ab7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.659+0000 7f683d400640 1 -- 172.21.15.89:0/2989512769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aa3d50 msgr2=0x555668abbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.659+0000 7f683d400640 1 --2- 172.21.15.89:0/2989512769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aa3d50 0x555668abbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.659+0000 7f683d400640 1 -- 172.21.15.89:0/2989512769 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555668b2d000 con 0x555668aaef30 2024-09-10T08:47:24.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.659+0000 7f683e800640 1 --2- 172.21.15.89:0/2989512769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668ab7280 0x555668ab7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:24.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.659+0000 7f683d400640 1 --2- 172.21.15.89:0/2989512769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aaef30 0x555668ab6d40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x555668aaa680 tx=0x7f682c003a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f1f4179fd8cca07 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:24.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.659+0000 7f681fe00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:47:24.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.660+0000 7f681fe00640 1 -- 172.21.15.89:0/2989512769 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c035020 con 0x555668aaef30 2024-09-10T08:47:24.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.660+0000 7f681fe00640 1 -- 172.21.15.89:0/2989512769 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f682c00fc50 con 0x555668aaef30 2024-09-10T08:47:24.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.660+0000 7f6847a6d740 1 -- 172.21.15.89:0/2989512769 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f682c003f60 con 0x555668aaef30 2024-09-10T08:47:24.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.660+0000 7f681fe00640 1 -- 172.21.15.89:0/2989512769 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f682c03d050 con 0x555668aaef30 2024-09-10T08:47:24.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.660+0000 7f6847a6d740 10 client.26722 fetch_fsmap learned FSMap version 30 2024-09-10T08:47:24.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.660+0000 7f6847a6d740 1 -- 172.21.15.89:0/2989512769 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555668b2e2d0 con 0x555668aaef30 2024-09-10T08:47:24.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.660+0000 7f681fe00640 1 -- 172.21.15.89:0/2989512769 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f682c003f60 con 0x555668aaef30 2024-09-10T08:47:24.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.660+0000 7f681fe00640 1 -- 172.21.15.89:0/2989512769 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f682c019070 con 0x555668aaef30 2024-09-10T08:47:24.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.660+0000 7f6847a6d740 10 client.26722 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T08:47:24.664 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.660+0000 7f6847a6d740 10 client.26722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.660+0000 7f6847a6d740 20 client.26722 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.660+0000 7f6847a6d740 1 --2- 172.21.15.89:0/2989512769 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x555668b2e960 0x555668b30d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.661+0000 7f6847a6d740 4 client.26722 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:47:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.661+0000 7f6847a6d740 1 -- 172.21.15.89:0/2989512769 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555668a11e60 con 0x555668b2e960 2024-09-10T08:47:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.661+0000 7f683e800640 1 --2- 172.21.15.89:0/2989512769 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x555668b2e960 0x555668b30d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.661+0000 7f683e800640 1 --2- 172.21.15.89:0/2989512769 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x555668b2e960 0x555668b30d80 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4bad9ee7b8b64ba server_cookie=b57a81fc25f04e2f in_seq=0 out_seq=0 2024-09-10T08:47:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.661+0000 7f681fe00640 10 client.26722 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.661+0000 7f681fe00640 1 -- 172.21.15.89:0/2989512769 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555668a11e60 con 0x555668b2e960 2024-09-10T08:47:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.661+0000 7f681fe00640 10 client.26722 handle_command_reply: tid=0 2024-09-10T08:47:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.661+0000 7f6847a6d740 1 client.26722 shutdown 2024-09-10T08:47:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.662+0000 7f6847a6d740 1 -- 172.21.15.89:0/2989512769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aaef30 msgr2=0x555668ab6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.662+0000 7f6847a6d740 1 --2- 172.21.15.89:0/2989512769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aaef30 0x555668ab6d40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x555668aaa680 tx=0x7f682c003a40 comp rx=0 tx=0).stop 2024-09-10T08:47:24.665 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.662+0000 7f6847a6d740 1 -- 172.21.15.89:0/2989512769 shutdown_connections 2024-09-10T08:47:24.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.662+0000 7f6847a6d740 1 --2- 172.21.15.89:0/2989512769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555668ab7280 0x555668ab7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.662+0000 7f6847a6d740 1 --2- 172.21.15.89:0/2989512769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555668aaef30 0x555668ab6d40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.662+0000 7f6847a6d740 1 --2- 172.21.15.89:0/2989512769 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x555668b2e960 0x555668b30d80 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.662+0000 7f6847a6d740 1 --2- 172.21.15.89:0/2989512769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555668aa3d50 0x555668abbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.662+0000 7f681fe00640 0 client.26722 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:24.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.662+0000 7f6847a6d740 1 -- 172.21.15.89:0/2989512769 >> 172.21.15.89:0/2989512769 conn(0x555668aa02e0 msgr2=0x555668aaa770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:24.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.662+0000 7f6847a6d740 1 -- 172.21.15.89:0/2989512769 shutdown_connections 2024-09-10T08:47:24.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.662+0000 7f6847a6d740 1 -- 172.21.15.89:0/2989512769 wait complete. 2024-09-10T08:47:24.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.662+0000 7f6847a6d740 20 client.26722 trim_cache size 0 max 16384 2024-09-10T08:47:24.666 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:47:24.666 INFO:teuthology.orchestra.run.smithi089.stdout: "status": "no active scrubs running", 2024-09-10T08:47:24.667 INFO:teuthology.orchestra.run.smithi089.stdout: "scrubs": {} 2024-09-10T08:47:24.667 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:47:24.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.663+0000 7f683f200640 1 -- 172.21.15.89:0/831050650 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f68240a2c00 msgr2=0x7f68240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.663+0000 7f683f200640 1 --2- 172.21.15.89:0/831050650 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f68240a2c00 0x7f68240a50c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f68300ed1c0 tx=0x7f68300f5550 comp rx=0 tx=0).stop 2024-09-10T08:47:24.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.663+0000 7f683f200640 1 -- 172.21.15.89:0/831050650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68401019c0 msgr2=0x7f68401982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.663+0000 7f683f200640 1 --2- 172.21.15.89:0/831050650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68401019c0 0x7f68401982a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6830006010 tx=0x7f6830003c10 comp rx=0 tx=0).stop 2024-09-10T08:47:24.667 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.663+0000 7f683f200640 1 -- 172.21.15.89:0/831050650 shutdown_connections 2024-09-10T08:47:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.663+0000 7f683f200640 1 --2- 172.21.15.89:0/831050650 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f68240a2c00 0x7f68240a50c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.663+0000 7f683f200640 1 --2- 172.21.15.89:0/831050650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68401079c0 0x7f684019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.663+0000 7f683f200640 1 --2- 172.21.15.89:0/831050650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6840102360 0x7f68401987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.663+0000 7f683f200640 1 --2- 172.21.15.89:0/831050650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68401019c0 0x7f68401982a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.663+0000 7f683f200640 1 -- 172.21.15.89:0/831050650 >> 172.21.15.89:0/831050650 conn(0x7f68400fd740 msgr2=0x7f68400fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.663+0000 7f683f200640 1 -- 172.21.15.89:0/831050650 shutdown_connections 2024-09-10T08:47:24.668 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:24.663+0000 7f683f200640 1 -- 172.21.15.89:0/831050650 wait complete. 2024-09-10T08:47:25.209 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:24 smithi089 ceph-mon[30757]: pgmap v617: 129 pgs: 129 active+clean; 465 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 551 op/s 2024-09-10T08:47:25.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:24 smithi149 ceph-mon[36167]: pgmap v617: 129 pgs: 129 active+clean; 465 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 551 op/s 2024-09-10T08:47:25.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:24 smithi111 ceph-mon[35220]: pgmap v617: 129 pgs: 129 active+clean; 465 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 551 op/s 2024-09-10T08:47:25.609 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:47:25.610 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:47:26.202 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7ca83d3b', total seconds elapsed: 33.4 2024-09-10T08:47:26.203 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-10T08:47:26.203 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:47:26.343 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:26.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.974+0000 7f6d2de00640 1 -- 172.21.15.89:0/2705843079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d281079c0 msgr2=0x7f6d28107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.974+0000 7f6d2de00640 1 --2- 172.21.15.89:0/2705843079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d281079c0 0x7f6d28107da0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009900 tx=0x7f6d1c02a410 comp rx=0 tx=0).stop 2024-09-10T08:47:26.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.974+0000 7f6d2de00640 1 -- 172.21.15.89:0/2705843079 shutdown_connections 2024-09-10T08:47:26.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.974+0000 7f6d2de00640 1 --2- 172.21.15.89:0/2705843079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d28102360 0x7f6d2810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.974+0000 7f6d2de00640 1 --2- 172.21.15.89:0/2705843079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d281019c0 0x7f6d28101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.974+0000 7f6d2de00640 1 --2- 172.21.15.89:0/2705843079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d281079c0 0x7f6d28107da0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.974+0000 7f6d2de00640 1 -- 172.21.15.89:0/2705843079 >> 172.21.15.89:0/2705843079 conn(0x7f6d280fd740 msgr2=0x7f6d280ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:26.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.974+0000 7f6d2de00640 1 -- 172.21.15.89:0/2705843079 shutdown_connections 2024-09-10T08:47:26.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.975+0000 7f6d2de00640 1 -- 172.21.15.89:0/2705843079 wait complete. 2024-09-10T08:47:26.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.976+0000 7f6d2de00640 1 Processor -- start 2024-09-10T08:47:26.977 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.976+0000 7f6d2de00640 1 -- start start 2024-09-10T08:47:26.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.976+0000 7f6d2de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d281019c0 0x7f6d280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.976+0000 7f6d2de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d28102360 0x7f6d28074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.976+0000 7f6d2de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d281079c0 0x7f6d28074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.976+0000 7f6d2de00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6d28111dd0 con 0x7f6d281079c0 2024-09-10T08:47:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.976+0000 7f6d2de00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6d28111c50 con 0x7f6d281019c0 2024-09-10T08:47:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.976+0000 7f6d2de00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6d28111f50 con 0x7f6d28102360 2024-09-10T08:47:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.977+0000 7f6d2ca00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d281019c0 0x7f6d280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.977+0000 7f6d27e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d28102360 0x7f6d28074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.977+0000 7f6d2ca00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d281019c0 0x7f6d280744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:34480/0 (socket says 172.21.15.89:34480) 2024-09-10T08:47:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.977+0000 7f6d2ca00640 1 -- 172.21.15.89:0/2636591295 learned_addr learned my addr 172.21.15.89:0/2636591295 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.977+0000 7f6d2d400640 1 --2- 172.21.15.89:0/2636591295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d281079c0 0x7f6d28074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.977+0000 7f6d2ca00640 1 -- 172.21.15.89:0/2636591295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d28102360 msgr2=0x7f6d28074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.977+0000 7f6d2ca00640 1 --2- 172.21.15.89:0/2636591295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d28102360 0x7f6d28074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.977+0000 7f6d2ca00640 1 -- 172.21.15.89:0/2636591295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d281079c0 msgr2=0x7f6d28074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.977+0000 7f6d2ca00640 1 --2- 172.21.15.89:0/2636591295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d281079c0 0x7f6d28074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.977+0000 7f6d2ca00640 1 -- 172.21.15.89:0/2636591295 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d281a6730 con 0x7f6d281019c0 2024-09-10T08:47:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.977+0000 7f6d2d400640 1 --2- 172.21.15.89:0/2636591295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d281079c0 0x7f6d28074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.978+0000 7f6d2ca00640 1 --2- 172.21.15.89:0/2636591295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d281019c0 0x7f6d280744f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c02aa00 tx=0x7f6d1c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.978+0000 7f6d25600640 1 -- 172.21.15.89:0/2636591295 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c03f070 con 0x7f6d281019c0 2024-09-10T08:47:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.978+0000 7f6d25600640 1 -- 172.21.15.89:0/2636591295 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d1c031c30 con 0x7f6d281019c0 2024-09-10T08:47:26.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.978+0000 7f6d25600640 1 -- 172.21.15.89:0/2636591295 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c035030 con 0x7f6d281019c0 2024-09-10T08:47:26.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.979+0000 7f6d2de00640 1 -- 172.21.15.89:0/2636591295 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d281a69c0 con 0x7f6d281019c0 2024-09-10T08:47:26.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.979+0000 7f6d2de00640 1 -- 172.21.15.89:0/2636591295 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d281a6e20 con 0x7f6d281019c0 2024-09-10T08:47:26.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.980+0000 7f6d2de00640 1 -- 172.21.15.89:0/2636591295 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d28101e20 con 0x7f6d281019c0 2024-09-10T08:47:26.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.982+0000 7f6d25600640 1 -- 172.21.15.89:0/2636591295 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6d1c042070 con 0x7f6d281019c0 2024-09-10T08:47:26.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.983+0000 7f6d25600640 1 --2- 172.21.15.89:0/2636591295 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6d100a2c80 0x7f6d100a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.983+0000 7f6d25600640 1 -- 172.21.15.89:0/2636591295 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f6d1c0efd90 con 0x7f6d281019c0 2024-09-10T08:47:26.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.984+0000 7f6d27e00640 1 --2- 172.21.15.89:0/2636591295 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6d100a2c80 0x7f6d100a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.984+0000 7f6d27e00640 1 --2- 172.21.15.89:0/2636591295 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6d100a2c80 0x7f6d100a5140 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6d28075f80 tx=0x7f6d18005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:26.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:26.987+0000 7f6d25600640 1 -- 172.21.15.89:0/2636591295 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6d1c049030 con 0x7f6d281019c0 2024-09-10T08:47:27.021 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:27.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:26 smithi089 ceph-mon[30757]: pgmap v618: 129 pgs: 129 active+clean; 465 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 537 op/s 2024-09-10T08:47:27.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:26 smithi149 ceph-mon[36167]: pgmap v618: 129 pgs: 129 active+clean; 465 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 537 op/s 2024-09-10T08:47:27.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:26 smithi111 ceph-mon[35220]: pgmap v618: 129 pgs: 129 active+clean; 465 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 537 op/s 2024-09-10T08:47:27.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.361+0000 7f6d2de00640 1 -- 172.21.15.89:0/2636591295 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d2802d7c0 con 0x7f6d281019c0 2024-09-10T08:47:27.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.362+0000 7f6d25600640 1 -- 172.21.15.89:0/2636591295 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11920 (secure 0 0 0) 0x7f6d1c0bc8c0 con 0x7f6d281019c0 2024-09-10T08:47:27.364 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:47:27.364 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":30,"btime":"2024-09-10T08:46:58:834163+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:46:58.731037+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":158,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24518,24496]},"id":1}]} 2024-09-10T08:47:27.364 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 30 2024-09-10T08:47:27.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.365+0000 7f6d2de00640 1 -- 172.21.15.89:0/2636591295 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6d100a2c80 msgr2=0x7f6d100a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:27.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.365+0000 7f6d2de00640 1 --2- 172.21.15.89:0/2636591295 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6d100a2c80 0x7f6d100a5140 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6d28075f80 tx=0x7f6d18005f90 comp rx=0 tx=0).stop 2024-09-10T08:47:27.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.365+0000 7f6d2de00640 1 -- 172.21.15.89:0/2636591295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d281019c0 msgr2=0x7f6d280744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:27.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.365+0000 7f6d2de00640 1 --2- 172.21.15.89:0/2636591295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d281019c0 0x7f6d280744f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c02aa00 tx=0x7f6d1c003c10 comp rx=0 tx=0).stop 2024-09-10T08:47:27.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.365+0000 7f6d2de00640 1 -- 172.21.15.89:0/2636591295 shutdown_connections 2024-09-10T08:47:27.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.365+0000 7f6d2de00640 1 --2- 172.21.15.89:0/2636591295 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6d100a2c80 0x7f6d100a5140 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:27.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.365+0000 7f6d2de00640 1 --2- 172.21.15.89:0/2636591295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d281079c0 0x7f6d28074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:27.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.365+0000 7f6d2de00640 1 --2- 172.21.15.89:0/2636591295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d28102360 0x7f6d28074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:27.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.365+0000 7f6d2de00640 1 --2- 172.21.15.89:0/2636591295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d281019c0 0x7f6d280744f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:27.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.365+0000 7f6d2de00640 1 -- 172.21.15.89:0/2636591295 >> 172.21.15.89:0/2636591295 conn(0x7f6d280fd740 msgr2=0x7f6d280fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:27.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.365+0000 7f6d2de00640 1 -- 172.21.15.89:0/2636591295 shutdown_connections 2024-09-10T08:47:27.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.365+0000 7f6d2de00640 1 -- 172.21.15.89:0/2636591295 wait complete. 2024-09-10T08:47:27.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.899+0000 7f3826c00640 1 -- 172.21.15.89:0/1666484850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3820102d20 msgr2=0x7f3820103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:27.900 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.899+0000 7f3826c00640 1 --2- 172.21.15.89:0/1666484850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3820102d20 0x7f3820103100 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f381000fb10 tx=0x7f3810032a20 comp rx=0 tx=0).stop 2024-09-10T08:47:27.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.899+0000 7f3826c00640 1 -- 172.21.15.89:0/1666484850 shutdown_connections 2024-09-10T08:47:27.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.899+0000 7f3826c00640 1 --2- 172.21.15.89:0/1666484850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f382010c150 0x7f382010e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:27.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.899+0000 7f3826c00640 1 --2- 172.21.15.89:0/1666484850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3820103640 0x7f382010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:27.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.899+0000 7f3826c00640 1 --2- 172.21.15.89:0/1666484850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3820102d20 0x7f3820103100 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:27.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.899+0000 7f3826c00640 1 -- 172.21.15.89:0/1666484850 >> 172.21.15.89:0/1666484850 conn(0x7f38200fc780 msgr2=0x7f38200feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:27.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.900+0000 7f3826c00640 1 -- 172.21.15.89:0/1666484850 shutdown_connections 2024-09-10T08:47:27.902 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.900+0000 7f3826c00640 1 -- 172.21.15.89:0/1666484850 wait complete. 2024-09-10T08:47:27.903 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.902+0000 7f3826c00640 1 Processor -- start 2024-09-10T08:47:27.904 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.903+0000 7f3826c00640 1 -- start start 2024-09-10T08:47:27.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.905+0000 7f3826c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3820102d20 0x7f38200739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:27.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.905+0000 7f3826c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3820103640 0x7f382006ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:27.906 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.905+0000 7f3826c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f382010c150 0x7f382006cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:27.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.906+0000 7f3826c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f382010fdd0 con 0x7f3820102d20 2024-09-10T08:47:27.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.906+0000 7f3826c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f382010fc50 con 0x7f3820103640 2024-09-10T08:47:27.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.906+0000 7f3826c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f382010ff50 con 0x7f382010c150 2024-09-10T08:47:27.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.906+0000 7f381f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3820103640 0x7f382006ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:27.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.906+0000 7f381f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3820103640 0x7f382006ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:34500/0 (socket says 172.21.15.89:34500) 2024-09-10T08:47:27.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.906+0000 7f381f400640 1 -- 172.21.15.89:0/269362099 learned_addr learned my addr 172.21.15.89:0/269362099 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:27.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.906+0000 7f3824c00640 1 --2- 172.21.15.89:0/269362099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f382010c150 0x7f382006cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:27.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.907+0000 7f381fe00640 1 --2- 172.21.15.89:0/269362099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3820102d20 0x7f38200739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:27.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.907+0000 7f381f400640 1 -- 172.21.15.89:0/269362099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f382010c150 msgr2=0x7f382006cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:27.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.907+0000 7f381f400640 1 --2- 172.21.15.89:0/269362099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f382010c150 0x7f382006cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:27.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.908+0000 7f381f400640 1 -- 172.21.15.89:0/269362099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3820102d20 msgr2=0x7f38200739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:27.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.908+0000 7f381f400640 1 --2- 172.21.15.89:0/269362099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3820102d20 0x7f38200739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:27.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.908+0000 7f381f400640 1 -- 172.21.15.89:0/269362099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382006d550 con 0x7f3820103640 2024-09-10T08:47:27.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.908+0000 7f381fe00640 1 --2- 172.21.15.89:0/269362099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3820102d20 0x7f38200739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:27.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.908+0000 7f381f400640 1 --2- 172.21.15.89:0/269362099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3820103640 0x7f382006ca50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f380c00b3f0 tx=0x7f380c00b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:27.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.909+0000 7f381cc00640 1 -- 172.21.15.89:0/269362099 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c015020 con 0x7f3820103640 2024-09-10T08:47:27.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.909+0000 7f381cc00640 1 -- 172.21.15.89:0/269362099 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f380c011d10 con 0x7f3820103640 2024-09-10T08:47:27.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.909+0000 7f381cc00640 1 -- 172.21.15.89:0/269362099 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c00f2e0 con 0x7f3820103640 2024-09-10T08:47:27.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.909+0000 7f3826c00640 1 -- 172.21.15.89:0/269362099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f382006d840 con 0x7f3820103640 2024-09-10T08:47:27.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.909+0000 7f3826c00640 1 -- 172.21.15.89:0/269362099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3820071810 con 0x7f3820103640 2024-09-10T08:47:27.913 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.912+0000 7f3826c00640 1 -- 172.21.15.89:0/269362099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f382006b070 con 0x7f3820103640 2024-09-10T08:47:27.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.914+0000 7f381cc00640 1 -- 172.21.15.89:0/269362099 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f380c018020 con 0x7f3820103640 2024-09-10T08:47:27.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.916+0000 7f381cc00640 1 --2- 172.21.15.89:0/269362099 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f38040a2cd0 0x7f38040a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:27.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.917+0000 7f381fe00640 1 --2- 172.21.15.89:0/269362099 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f38040a2cd0 0x7f38040a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:27.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.917+0000 7f381cc00640 1 -- 172.21.15.89:0/269362099 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f380c0cc3b0 con 0x7f3820103640 2024-09-10T08:47:27.919 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.918+0000 7f381fe00640 1 --2- 172.21.15.89:0/269362099 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f38040a2cd0 0x7f38040a5190 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3810005ab0 tx=0x7f3810041040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:27.926 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:27.925+0000 7f381cc00640 1 -- 172.21.15.89:0/269362099 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f380c01f020 con 0x7f3820103640 2024-09-10T08:47:27.985 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:27 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2636591295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:28.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:27 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2636591295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:28.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:27 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2636591295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:28.303 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:28.302+0000 7f3826c00640 1 -- 172.21.15.89:0/269362099 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3820071b30 con 0x7f3820103640 2024-09-10T08:47:28.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:28.303+0000 7f381cc00640 1 -- 172.21.15.89:0/269362099 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11920 (secure 0 0 0) 0x7f380c09b700 con 0x7f3820103640 2024-09-10T08:47:28.304 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:47:28.304 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":30,"btime":"2024-09-10T08:46:58:834163+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:46:58.731037+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":158,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24518,24496]},"id":1}]} 2024-09-10T08:47:28.305 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 30 2024-09-10T08:47:28.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:28.305+0000 7f3826c00640 1 -- 172.21.15.89:0/269362099 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f38040a2cd0 msgr2=0x7f38040a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:28.305+0000 7f3826c00640 1 --2- 172.21.15.89:0/269362099 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f38040a2cd0 0x7f38040a5190 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3810005ab0 tx=0x7f3810041040 comp rx=0 tx=0).stop 2024-09-10T08:47:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:28.305+0000 7f3826c00640 1 -- 172.21.15.89:0/269362099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3820103640 msgr2=0x7f382006ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:28.305+0000 7f3826c00640 1 --2- 172.21.15.89:0/269362099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3820103640 0x7f382006ca50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f380c00b3f0 tx=0x7f380c00b8b0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:28.305+0000 7f3826c00640 1 -- 172.21.15.89:0/269362099 shutdown_connections 2024-09-10T08:47:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:28.305+0000 7f3826c00640 1 --2- 172.21.15.89:0/269362099 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f38040a2cd0 0x7f38040a5190 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:28.305+0000 7f3826c00640 1 --2- 172.21.15.89:0/269362099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f382010c150 0x7f382006cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:28.305+0000 7f3826c00640 1 --2- 172.21.15.89:0/269362099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3820103640 0x7f382006ca50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:28.305+0000 7f3826c00640 1 --2- 172.21.15.89:0/269362099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3820102d20 0x7f38200739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:28.305+0000 7f3826c00640 1 -- 172.21.15.89:0/269362099 >> 172.21.15.89:0/269362099 conn(0x7f38200fc780 msgr2=0x7f38200fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:28.305+0000 7f3826c00640 1 -- 172.21.15.89:0/269362099 shutdown_connections 2024-09-10T08:47:28.308 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:28.305+0000 7f3826c00640 1 -- 172.21.15.89:0/269362099 wait complete. 2024-09-10T08:47:28.422 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:47:29.056 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:28 smithi089 ceph-mon[30757]: pgmap v619: 129 pgs: 129 active+clean; 451 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 651 op/s 2024-09-10T08:47:29.056 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:28 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/269362099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:29.170 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24554 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-10T08:47:29.170 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24554 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-10T08:47:29.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:28 smithi149 ceph-mon[36167]: pgmap v619: 129 pgs: 129 active+clean; 451 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 651 op/s 2024-09-10T08:47:29.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:28 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/269362099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:29.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:28 smithi111 ceph-mon[35220]: pgmap v619: 129 pgs: 129 active+clean; 451 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 651 op/s 2024-09-10T08:47:29.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:28 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/269362099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:29.306 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:29.937 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.937+0000 7f920b200640 1 -- 172.21.15.89:0/3228877264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9204103e30 msgr2=0x7f9204104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:29.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.937+0000 7f920b200640 1 --2- 172.21.15.89:0/3228877264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9204103e30 0x7f9204104210 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009900 tx=0x7f91f802a410 comp rx=0 tx=0).stop 2024-09-10T08:47:29.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.937+0000 7f920b200640 1 -- 172.21.15.89:0/3228877264 shutdown_connections 2024-09-10T08:47:29.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.937+0000 7f920b200640 1 --2- 172.21.15.89:0/3228877264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9204108f60 0x7f9204110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.937+0000 7f920b200640 1 --2- 172.21.15.89:0/3228877264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92041047e0 0x7f9204108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.937+0000 7f920b200640 1 --2- 172.21.15.89:0/3228877264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9204103e30 0x7f9204104210 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.937+0000 7f920b200640 1 -- 172.21.15.89:0/3228877264 >> 172.21.15.89:0/3228877264 conn(0x7f92040ff8f0 msgr2=0x7f9204101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:29.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.937+0000 7f920b200640 1 -- 172.21.15.89:0/3228877264 shutdown_connections 2024-09-10T08:47:29.938 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.937+0000 7f920b200640 1 -- 172.21.15.89:0/3228877264 wait complete. 2024-09-10T08:47:29.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.938+0000 7f920b200640 1 Processor -- start 2024-09-10T08:47:29.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.938+0000 7f920b200640 1 -- start start 2024-09-10T08:47:29.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.939+0000 7f920b200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9204103e30 0x7f920419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:29.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.939+0000 7f920b200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92041047e0 0x7f920419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:29.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.939+0000 7f920b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9204108f60 0x7f920419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:29.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.939+0000 7f920b200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9204113fd0 con 0x7f92041047e0 2024-09-10T08:47:29.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.939+0000 7f920b200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9204113e50 con 0x7f9204103e30 2024-09-10T08:47:29.939 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.939+0000 7f920b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9204114150 con 0x7f9204108f60 2024-09-10T08:47:29.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.939+0000 7f9209e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9204103e30 0x7f920419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:29.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.939+0000 7f9209e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9204103e30 0x7f920419a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:34522/0 (socket says 172.21.15.89:34522) 2024-09-10T08:47:29.940 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.939+0000 7f9209e00640 1 -- 172.21.15.89:0/2322028588 learned_addr learned my addr 172.21.15.89:0/2322028588 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:29.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.939+0000 7f9209400640 1 --2- 172.21.15.89:0/2322028588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92041047e0 0x7f920419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:29.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.939+0000 7f920a800640 1 --2- 172.21.15.89:0/2322028588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9204108f60 0x7f920419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:29.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.940+0000 7f9209e00640 1 -- 172.21.15.89:0/2322028588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9204108f60 msgr2=0x7f920419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:29.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.940+0000 7f9209e00640 1 --2- 172.21.15.89:0/2322028588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9204108f60 0x7f920419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.940+0000 7f9209e00640 1 -- 172.21.15.89:0/2322028588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92041047e0 msgr2=0x7f920419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:29.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.940+0000 7f9209e00640 1 --2- 172.21.15.89:0/2322028588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92041047e0 0x7f920419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.940+0000 7f9209e00640 1 -- 172.21.15.89:0/2322028588 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920419f3e0 con 0x7f9204103e30 2024-09-10T08:47:29.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.940+0000 7f9209e00640 1 --2- 172.21.15.89:0/2322028588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9204103e30 0x7f920419a430 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f91f8004770 tx=0x7f91f8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:29.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.940+0000 7f91fea00640 1 -- 172.21.15.89:0/2322028588 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f803f070 con 0x7f9204103e30 2024-09-10T08:47:29.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.941+0000 7f91fea00640 1 -- 172.21.15.89:0/2322028588 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f8031ca0 con 0x7f9204103e30 2024-09-10T08:47:29.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.941+0000 7f920b200640 1 -- 172.21.15.89:0/2322028588 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920419f670 con 0x7f9204103e30 2024-09-10T08:47:29.941 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.941+0000 7f91fea00640 1 -- 172.21.15.89:0/2322028588 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8035030 con 0x7f9204103e30 2024-09-10T08:47:29.942 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.941+0000 7f920b200640 1 -- 172.21.15.89:0/2322028588 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92041a6fb0 con 0x7f9204103e30 2024-09-10T08:47:29.943 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.944+0000 7f91fea00640 1 -- 172.21.15.89:0/2322028588 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f91f8042070 con 0x7f9204103e30 2024-09-10T08:47:29.944 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.944+0000 7f920f281740 1 Processor -- start 2024-09-10T08:47:29.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.945+0000 7f920f281740 1 -- start start 2024-09-10T08:47:29.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.945+0000 7f920f281740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14e2e40 0x5649d14e3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:29.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.945+0000 7f920f281740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e3760 0x5649d14edae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:29.945 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.945+0000 7f920f281740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14ee020 0x5649d14f0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:29.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.945+0000 7f920f281740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5649d14493e0 con 0x5649d14e2e40 2024-09-10T08:47:29.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.945+0000 7f920f281740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5649d13f0d80 con 0x5649d14e3760 2024-09-10T08:47:29.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.945+0000 7f920f281740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5649d14453a0 con 0x5649d14ee020 2024-09-10T08:47:29.946 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.945+0000 7f91fea00640 1 --2- 172.21.15.89:0/2322028588 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f91ec0a2c00 0x7f91ec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:29.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.945+0000 7f91fea00640 1 -- 172.21.15.89:0/2322028588 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f91f80ef9b0 con 0x7f9204103e30 2024-09-10T08:47:29.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.945+0000 7f9209e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e3760 0x5649d14edae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:29.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.945+0000 7f9209e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e3760 0x5649d14edae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:34530/0 (socket says 172.21.15.89:34530) 2024-09-10T08:47:29.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.945+0000 7f9209e00640 1 -- 172.21.15.89:0/1285431653 learned_addr learned my addr 172.21.15.89:0/1285431653 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:29.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.945+0000 7f920a800640 1 --2- 172.21.15.89:0/1285431653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14e2e40 0x5649d14e3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:29.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.946+0000 7f9209400640 1 --2- 172.21.15.89:0/1285431653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14ee020 0x5649d14f0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:29.947 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.946+0000 7f9209e00640 1 -- 172.21.15.89:0/1285431653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14ee020 msgr2=0x5649d14f0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:29.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.946+0000 7f9209e00640 1 --2- 172.21.15.89:0/1285431653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14ee020 0x5649d14f0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.946+0000 7f9209e00640 1 -- 172.21.15.89:0/1285431653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14e2e40 msgr2=0x5649d14e3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:29.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.946+0000 7f9209e00640 1 --2- 172.21.15.89:0/1285431653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14e2e40 0x5649d14e3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.946+0000 7f9209e00640 1 -- 172.21.15.89:0/1285431653 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649d140fae0 con 0x5649d14e3760 2024-09-10T08:47:29.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.946+0000 7f920a800640 1 --2- 172.21.15.89:0/2322028588 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f91ec0a2c00 0x7f91ec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:29.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.946+0000 7f9209e00640 1 --2- 172.21.15.89:0/1285431653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e3760 0x5649d14edae0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f91f80ed080 tx=0x7f91f802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35a9f1d079de521d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:29.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.946+0000 7f91f3000640 1 -- 172.21.15.89:0/1285431653 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8049030 con 0x5649d14e3760 2024-09-10T08:47:29.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.947+0000 7f91f3000640 1 -- 172.21.15.89:0/1285431653 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f80c0250 con 0x5649d14e3760 2024-09-10T08:47:29.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.947+0000 7f91f3000640 1 -- 172.21.15.89:0/1285431653 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f803b4f0 con 0x5649d14e3760 2024-09-10T08:47:29.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.948+0000 7f920f281740 1 -- 172.21.15.89:0/1285431653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e3760 msgr2=0x5649d14edae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:29.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.948+0000 7f920f281740 1 --2- 172.21.15.89:0/1285431653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e3760 0x5649d14edae0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f91f80ed080 tx=0x7f91f802a410 comp rx=0 tx=0).stop 2024-09-10T08:47:29.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.948+0000 7f920a800640 1 --2- 172.21.15.89:0/2322028588 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f91ec0a2c00 0x7f91ec0a50c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9200005fd0 tx=0x7f9200014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:29.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.948+0000 7f920f281740 1 -- 172.21.15.89:0/1285431653 shutdown_connections 2024-09-10T08:47:29.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.948+0000 7f920f281740 1 --2- 172.21.15.89:0/1285431653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14ee020 0x5649d14f0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.948+0000 7f920f281740 1 --2- 172.21.15.89:0/1285431653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e3760 0x5649d14edae0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.948+0000 7f920f281740 1 --2- 172.21.15.89:0/1285431653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14e2e40 0x5649d14e3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.948+0000 7f920f281740 1 -- 172.21.15.89:0/1285431653 >> 172.21.15.89:0/1285431653 conn(0x5649d14df3d0 msgr2=0x5649d14df7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.948+0000 7f920f281740 1 -- 172.21.15.89:0/1285431653 shutdown_connections 2024-09-10T08:47:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.948+0000 7f920f281740 1 -- 172.21.15.89:0/1285431653 wait complete. 2024-09-10T08:47:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.948+0000 7f920f281740 1 -- start start 2024-09-10T08:47:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.949+0000 7f920f281740 1 Processor -- start 2024-09-10T08:47:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.949+0000 7f920f281740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 0x5649d14eaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.949+0000 7f920f281740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14e3760 0x5649d14eb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.949+0000 7f920f281740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14ee020 0x5649d14eb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.949+0000 7f920f281740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5649d14f09b0 con 0x5649d14ee020 2024-09-10T08:47:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.949+0000 7f920f281740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5649d14040f0 con 0x5649d14e2e40 2024-09-10T08:47:29.952 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.949+0000 7f920f281740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5649d14f0b30 con 0x5649d14e3760 2024-09-10T08:47:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.949+0000 7f920f281740 10 client.0 _ll_register_callbacks cb 0x5649d13ebbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.949+0000 7f9209e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 0x5649d14eaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.949+0000 7f9209e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 0x5649d14eaec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:34534/0 (socket says 172.21.15.89:34534) 2024-09-10T08:47:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.949+0000 7f9209e00640 1 -- 172.21.15.89:0/1650783411 learned_addr learned my addr 172.21.15.89:0/1650783411 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.949+0000 7f9209400640 1 --2- 172.21.15.89:0/1650783411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14e3760 0x5649d14eb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.949+0000 7f920a800640 1 --2- 172.21.15.89:0/1650783411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14ee020 0x5649d14eb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.950+0000 7f9209e00640 1 -- 172.21.15.89:0/1650783411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14e3760 msgr2=0x5649d14eb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.950+0000 7f9209e00640 1 --2- 172.21.15.89:0/1650783411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14e3760 0x5649d14eb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.950+0000 7f9209e00640 1 -- 172.21.15.89:0/1650783411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14ee020 msgr2=0x5649d14eb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.950+0000 7f9209e00640 1 --2- 172.21.15.89:0/1650783411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14ee020 0x5649d14eb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.950+0000 7f9209e00640 1 -- 172.21.15.89:0/1650783411 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649d11ba3c0 con 0x5649d14e2e40 2024-09-10T08:47:29.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.950+0000 7f920a800640 1 --2- 172.21.15.89:0/1650783411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14ee020 0x5649d14eb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:29.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.950+0000 7f9209400640 1 --2- 172.21.15.89:0/1650783411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14e3760 0x5649d14eb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:29.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.950+0000 7f9209e00640 1 --2- 172.21.15.89:0/1650783411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 0x5649d14eaec0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f91f80bbc50 tx=0x7f91f80c4c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55a5f424ade9638 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:29.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.950+0000 7f91e7400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:47:29.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.950+0000 7f91e7400640 1 -- 172.21.15.89:0/1650783411 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8049020 con 0x5649d14e2e40 2024-09-10T08:47:29.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.950+0000 7f91e7400640 1 -- 172.21.15.89:0/1650783411 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f91f80b85c0 con 0x5649d14e2e40 2024-09-10T08:47:29.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.950+0000 7f920f281740 1 -- 172.21.15.89:0/1650783411 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91f80c0250 con 0x5649d14e2e40 2024-09-10T08:47:29.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.950+0000 7f91e7400640 1 -- 172.21.15.89:0/1650783411 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f91f803b4f0 con 0x5649d14e2e40 2024-09-10T08:47:29.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.950+0000 7f920f281740 10 client.26798 fetch_fsmap learned FSMap version 31 2024-09-10T08:47:29.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.951+0000 7f920f281740 1 -- 172.21.15.89:0/1650783411 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5649d14f09b0 con 0x5649d14e2e40 2024-09-10T08:47:29.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.951+0000 7f91e7400640 1 -- 172.21.15.89:0/1650783411 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f80c0250 con 0x5649d14e2e40 2024-09-10T08:47:29.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.951+0000 7f91e7400640 1 -- 172.21.15.89:0/1650783411 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f91f80cbc20 con 0x5649d14e2e40 2024-09-10T08:47:29.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.951+0000 7f920f281740 10 client.26798 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:47:29.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.951+0000 7f920f281740 10 client.26798 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:47:29.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.951+0000 7f920f281740 20 client.26798 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:29.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.951+0000 7f920f281740 1 --2- 172.21.15.89:0/1650783411 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5649d14fdc40 0x5649d15000c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:29.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.951+0000 7f920f281740 4 client.26798 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:29.955 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.951+0000 7f920f281740 1 -- 172.21.15.89:0/1650783411 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649d1456950 con 0x5649d14fdc40 2024-09-10T08:47:29.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.951+0000 7f9209400640 1 --2- 172.21.15.89:0/1650783411 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5649d14fdc40 0x5649d15000c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:29.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.952+0000 7f9209400640 1 --2- 172.21.15.89:0/1650783411 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5649d14fdc40 0x5649d15000c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b79f1d5780050b55 server_cookie=d51662f80d2278b in_seq=0 out_seq=0 2024-09-10T08:47:29.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.952+0000 7f91e7400640 10 client.26798 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:47:29.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.953+0000 7f91e7400640 1 -- 172.21.15.89:0/1650783411 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649d1456950 con 0x5649d14fdc40 2024-09-10T08:47:29.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.953+0000 7f91e7400640 10 client.26798 handle_command_reply: tid=0 2024-09-10T08:47:29.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.953+0000 7f920f281740 1 client.26798 shutdown 2024-09-10T08:47:29.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.953+0000 7f920f281740 1 -- 172.21.15.89:0/1650783411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 msgr2=0x5649d14eaec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:29.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.953+0000 7f920f281740 1 --2- 172.21.15.89:0/1650783411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 0x5649d14eaec0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f91f80bbc50 tx=0x7f91f80c4c20 comp rx=0 tx=0).stop 2024-09-10T08:47:29.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.953+0000 7f920f281740 1 -- 172.21.15.89:0/1650783411 shutdown_connections 2024-09-10T08:47:29.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.954+0000 7f920f281740 1 --2- 172.21.15.89:0/1650783411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14ee020 0x5649d14eb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.954+0000 7f920f281740 1 --2- 172.21.15.89:0/1650783411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14e3760 0x5649d14eb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.954+0000 7f920f281740 1 --2- 172.21.15.89:0/1650783411 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5649d14fdc40 0x5649d15000c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.954+0000 7f920f281740 1 --2- 172.21.15.89:0/1650783411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 0x5649d14eaec0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.954+0000 7f920f281740 1 -- 172.21.15.89:0/1650783411 >> 172.21.15.89:0/1650783411 conn(0x5649d14df3d0 msgr2=0x5649d14e5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:29.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.954+0000 7f91e7400640 0 client.26798 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:47:29.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.954+0000 7f920f281740 1 -- 172.21.15.89:0/1650783411 shutdown_connections 2024-09-10T08:47:29.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.954+0000 7f920f281740 1 -- 172.21.15.89:0/1650783411 wait complete. 2024-09-10T08:47:29.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.954+0000 7f920f281740 20 client.26798 trim_cache size 0 max 16384 2024-09-10T08:47:29.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.978+0000 7f920f281740 1 Processor -- start 2024-09-10T08:47:29.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.978+0000 7f920f281740 1 -- start start 2024-09-10T08:47:29.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.978+0000 7f920f281740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 0x5649d1505990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:29.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.978+0000 7f920f281740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14e3760 0x5649d1505ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:29.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.978+0000 7f920f281740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14ee020 0x5649d14eae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:29.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.978+0000 7f920f281740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f92041b3740 con 0x5649d14e3760 2024-09-10T08:47:29.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.978+0000 7f920f281740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f92041b0fe0 con 0x5649d14e2e40 2024-09-10T08:47:29.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.978+0000 7f920f281740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f92041aee30 con 0x5649d14ee020 2024-09-10T08:47:29.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.979+0000 7f9209e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 0x5649d1505990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:29.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.979+0000 7f9209e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 0x5649d1505990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:34536/0 (socket says 172.21.15.89:34536) 2024-09-10T08:47:29.979 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.979+0000 7f9209e00640 1 -- 172.21.15.89:0/1356480201 learned_addr learned my addr 172.21.15.89:0/1356480201 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:29.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.979+0000 7f9209400640 1 --2- 172.21.15.89:0/1356480201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14e3760 0x5649d1505ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:29.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.979+0000 7f920a800640 1 --2- 172.21.15.89:0/1356480201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14ee020 0x5649d14eae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:29.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.979+0000 7f9209e00640 1 -- 172.21.15.89:0/1356480201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14ee020 msgr2=0x5649d14eae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:29.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.979+0000 7f9209e00640 1 --2- 172.21.15.89:0/1356480201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14ee020 0x5649d14eae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.979+0000 7f9209e00640 1 -- 172.21.15.89:0/1356480201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14e3760 msgr2=0x5649d1505ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:29.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.979+0000 7f9209e00640 1 --2- 172.21.15.89:0/1356480201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14e3760 0x5649d1505ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.979+0000 7f9209e00640 1 -- 172.21.15.89:0/1356480201 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649d14df200 con 0x5649d14e2e40 2024-09-10T08:47:29.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.979+0000 7f920a800640 1 --2- 172.21.15.89:0/1356480201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14ee020 0x5649d14eae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:29.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.979+0000 7f9209400640 1 --2- 172.21.15.89:0/1356480201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14e3760 0x5649d1505ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:29.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.979+0000 7f9209e00640 1 --2- 172.21.15.89:0/1356480201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 0x5649d1505990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f91f8031550 tx=0x7f91f802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e22a69251427498 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.979+0000 7f91f2600640 1 -- 172.21.15.89:0/1356480201 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f80bb3a0 con 0x5649d14e2e40 2024-09-10T08:47:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.979+0000 7f91f2600640 1 -- 172.21.15.89:0/1356480201 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f803b690 con 0x5649d14e2e40 2024-09-10T08:47:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.980+0000 7f91f2600640 1 -- 172.21.15.89:0/1356480201 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f80bc1b0 con 0x5649d14e2e40 2024-09-10T08:47:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.980+0000 7f920f281740 1 -- 172.21.15.89:0/1356480201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 msgr2=0x5649d1505990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.980+0000 7f920f281740 1 --2- 172.21.15.89:0/1356480201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 0x5649d1505990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f91f8031550 tx=0x7f91f802a410 comp rx=0 tx=0).stop 2024-09-10T08:47:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.980+0000 7f920f281740 1 -- 172.21.15.89:0/1356480201 shutdown_connections 2024-09-10T08:47:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.980+0000 7f920f281740 1 --2- 172.21.15.89:0/1356480201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14ee020 0x5649d14eae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.980+0000 7f920f281740 1 --2- 172.21.15.89:0/1356480201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14e3760 0x5649d1505ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.980+0000 7f920f281740 1 --2- 172.21.15.89:0/1356480201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 0x5649d1505990 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.980+0000 7f920f281740 1 -- 172.21.15.89:0/1356480201 >> 172.21.15.89:0/1356480201 conn(0x5649d14df3d0 msgr2=0x5649d14e9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:29.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.981+0000 7f920f281740 1 -- 172.21.15.89:0/1356480201 shutdown_connections 2024-09-10T08:47:29.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.981+0000 7f920f281740 1 -- 172.21.15.89:0/1356480201 wait complete. 2024-09-10T08:47:29.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.981+0000 7f920f281740 1 -- start start 2024-09-10T08:47:29.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.981+0000 7f920f281740 1 Processor -- start 2024-09-10T08:47:29.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.982+0000 7f920f281740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 0x5649d14f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:29.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.982+0000 7f920f281740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14e3760 0x5649d14f6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:29.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.982+0000 7f920f281740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14ee020 0x5649d14e7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:29.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.982+0000 7f920f281740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f92041af4a0 con 0x5649d14e3760 2024-09-10T08:47:29.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.982+0000 7f920f281740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f92041b0910 con 0x5649d14e2e40 2024-09-10T08:47:29.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.982+0000 7f920f281740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f92041afb70 con 0x5649d14ee020 2024-09-10T08:47:29.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.982+0000 7f920f281740 10 client.0 _ll_register_callbacks cb 0x7f91f80c4b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:29.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.982+0000 7f9209e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 0x5649d14f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:29.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.982+0000 7f9209e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 0x5649d14f5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:34544/0 (socket says 172.21.15.89:34544) 2024-09-10T08:47:29.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.982+0000 7f9209e00640 1 -- 172.21.15.89:0/2230271679 learned_addr learned my addr 172.21.15.89:0/2230271679 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:29.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.982+0000 7f9209400640 1 --2- 172.21.15.89:0/2230271679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14e3760 0x5649d14f6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:29.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.983+0000 7f920a800640 1 --2- 172.21.15.89:0/2230271679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14ee020 0x5649d14e7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:29.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.983+0000 7f9209e00640 1 -- 172.21.15.89:0/2230271679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14ee020 msgr2=0x5649d14e7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:29.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.983+0000 7f9209e00640 1 --2- 172.21.15.89:0/2230271679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14ee020 0x5649d14e7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.983+0000 7f9209e00640 1 -- 172.21.15.89:0/2230271679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14e3760 msgr2=0x5649d14f6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:29.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.983+0000 7f9209e00640 1 --2- 172.21.15.89:0/2230271679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14e3760 0x5649d14f6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.983+0000 7f9209e00640 1 -- 172.21.15.89:0/2230271679 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649d14e8180 con 0x5649d14e2e40 2024-09-10T08:47:29.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.983+0000 7f920a800640 1 --2- 172.21.15.89:0/2230271679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14ee020 0x5649d14e7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:29.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.983+0000 7f9209400640 1 --2- 172.21.15.89:0/2230271679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14e3760 0x5649d14f6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:29.988 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.983+0000 7f9209e00640 1 --2- 172.21.15.89:0/2230271679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 0x5649d14f5fe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f91f80bbc50 tx=0x7f91f803b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=991e17d5a8e7ce77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:29.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.983+0000 7f91e7400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:47:29.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.983+0000 7f91e7400640 1 -- 172.21.15.89:0/2230271679 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8049070 con 0x5649d14e2e40 2024-09-10T08:47:29.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.983+0000 7f91e7400640 1 -- 172.21.15.89:0/2230271679 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f91f80c48b0 con 0x5649d14e2e40 2024-09-10T08:47:29.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.983+0000 7f920f281740 1 -- 172.21.15.89:0/2230271679 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91f803b690 con 0x5649d14e2e40 2024-09-10T08:47:29.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.983+0000 7f91e7400640 1 -- 172.21.15.89:0/2230271679 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f91f80c4680 con 0x5649d14e2e40 2024-09-10T08:47:29.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.984+0000 7f920f281740 10 client.26804 fetch_fsmap learned FSMap version 31 2024-09-10T08:47:29.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.984+0000 7f920f281740 1 -- 172.21.15.89:0/2230271679 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5649d14e8180 con 0x5649d14e2e40 2024-09-10T08:47:29.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.984+0000 7f91e7400640 1 -- 172.21.15.89:0/2230271679 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f803b690 con 0x5649d14e2e40 2024-09-10T08:47:29.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.984+0000 7f91e7400640 1 -- 172.21.15.89:0/2230271679 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f91f80b6ac0 con 0x5649d14e2e40 2024-09-10T08:47:29.989 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.984+0000 7f920f281740 10 client.26804 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:47:29.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.984+0000 7f920f281740 10 client.26804 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:47:29.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.984+0000 7f920f281740 20 client.26804 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:29.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.984+0000 7f920f281740 1 --2- 172.21.15.89:0/2230271679 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5649d1568760 0x5649d156ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:29.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.984+0000 7f920f281740 4 client.26804 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:47:29.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.984+0000 7f920f281740 1 -- 172.21.15.89:0/2230271679 --> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649d1450e60 con 0x5649d1568760 2024-09-10T08:47:29.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.985+0000 7f9209400640 1 --2- 172.21.15.89:0/2230271679 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5649d1568760 0x5649d156ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:29.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.985+0000 7f9209400640 1 --2- 172.21.15.89:0/2230271679 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5649d1568760 0x5649d156ab50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=24c0b00539c61971 server_cookie=d10807a988415a66 in_seq=0 out_seq=0 2024-09-10T08:47:29.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.985+0000 7f91e7400640 10 client.26804 ms_handle_connect on v2:172.21.15.149:6838/2020874598 2024-09-10T08:47:29.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.985+0000 7f91e7400640 1 -- 172.21.15.89:0/2230271679 <== mds.2 v2:172.21.15.149:6838/2020874598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649d1450e60 con 0x5649d1568760 2024-09-10T08:47:29.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.985+0000 7f91e7400640 10 client.26804 handle_command_reply: tid=0 2024-09-10T08:47:29.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.985+0000 7f920f281740 1 client.26804 shutdown 2024-09-10T08:47:29.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.986+0000 7f920f281740 1 -- 172.21.15.89:0/2230271679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 msgr2=0x5649d14f5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:29.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.986+0000 7f920f281740 1 --2- 172.21.15.89:0/2230271679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 0x5649d14f5fe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f91f80bbc50 tx=0x7f91f803b890 comp rx=0 tx=0).stop 2024-09-10T08:47:29.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.986+0000 7f920f281740 1 -- 172.21.15.89:0/2230271679 shutdown_connections 2024-09-10T08:47:29.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.986+0000 7f920f281740 1 --2- 172.21.15.89:0/2230271679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649d14ee020 0x5649d14e7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.986+0000 7f920f281740 1 --2- 172.21.15.89:0/2230271679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5649d14e3760 0x5649d14f6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.986+0000 7f920f281740 1 --2- 172.21.15.89:0/2230271679 >> [v2:172.21.15.149:6838/2020874598,v1:172.21.15.149:6839/2020874598] conn(0x5649d1568760 0x5649d156ab50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.986+0000 7f920f281740 1 --2- 172.21.15.89:0/2230271679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649d14e2e40 0x5649d14f5fe0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.986+0000 7f920f281740 1 -- 172.21.15.89:0/2230271679 >> 172.21.15.89:0/2230271679 conn(0x5649d14df3d0 msgr2=0x5649d14ddbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:29.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.986+0000 7f91e7400640 0 client.26804 ms_handle_reset on v2:172.21.15.149:6838/2020874598 2024-09-10T08:47:29.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.986+0000 7f920f281740 1 -- 172.21.15.89:0/2230271679 shutdown_connections 2024-09-10T08:47:29.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.986+0000 7f920f281740 1 -- 172.21.15.89:0/2230271679 wait complete. 2024-09-10T08:47:29.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.986+0000 7f920f281740 20 client.26804 trim_cache size 0 max 16384 2024-09-10T08:47:29.993 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:47:29.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.987+0000 7f920b200640 1 -- 172.21.15.89:0/2322028588 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f91ec0a2c00 msgr2=0x7f91ec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:29.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.987+0000 7f920b200640 1 --2- 172.21.15.89:0/2322028588 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f91ec0a2c00 0x7f91ec0a50c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9200005fd0 tx=0x7f9200014040 comp rx=0 tx=0).stop 2024-09-10T08:47:29.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.987+0000 7f920b200640 1 -- 172.21.15.89:0/2322028588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9204103e30 msgr2=0x7f920419a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:29.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.987+0000 7f920b200640 1 --2- 172.21.15.89:0/2322028588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9204103e30 0x7f920419a430 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f91f8004770 tx=0x7f91f8003c10 comp rx=0 tx=0).stop 2024-09-10T08:47:29.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.987+0000 7f920b200640 1 -- 172.21.15.89:0/2322028588 shutdown_connections 2024-09-10T08:47:29.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.987+0000 7f920b200640 1 --2- 172.21.15.89:0/2322028588 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f91ec0a2c00 0x7f91ec0a50c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.987+0000 7f920b200640 1 --2- 172.21.15.89:0/2322028588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9204108f60 0x7f920419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.987+0000 7f920b200640 1 --2- 172.21.15.89:0/2322028588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92041047e0 0x7f920419a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.987+0000 7f920b200640 1 --2- 172.21.15.89:0/2322028588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9204103e30 0x7f920419a430 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:29.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.987+0000 7f920b200640 1 -- 172.21.15.89:0/2322028588 >> 172.21.15.89:0/2322028588 conn(0x7f92040ff8f0 msgr2=0x7f9204100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:29.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.987+0000 7f920b200640 1 -- 172.21.15.89:0/2322028588 shutdown_connections 2024-09-10T08:47:29.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:29.987+0000 7f920b200640 1 -- 172.21.15.89:0/2322028588 wait complete. 2024-09-10T08:47:30.008 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:30.164 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:29 smithi089 ceph-mon[30757]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:47:30.194 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:29 smithi111 ceph-mon[35220]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:47:30.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:29 smithi149 ceph-mon[36167]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:47:30.953 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:47:31.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:30 smithi149 ceph-mon[36167]: pgmap v620: 129 pgs: 129 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 1.5 MiB/s wr, 366 op/s 2024-09-10T08:47:31.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:30 smithi111 ceph-mon[35220]: pgmap v620: 129 pgs: 129 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 1.5 MiB/s wr, 366 op/s 2024-09-10T08:47:31.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:30 smithi089 ceph-mon[30757]: pgmap v620: 129 pgs: 129 active+clean; 450 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 1.5 MiB/s wr, 366 op/s 2024-09-10T08:47:31.610 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.610+0000 7f19f8a00640 1 -- 172.21.15.89:0/415807344 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19f4077040 msgr2=0x7f19f41124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:31.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.610+0000 7f19f8a00640 1 --2- 172.21.15.89:0/415807344 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19f4077040 0x7f19f41124f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009900 tx=0x7f19dc02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.610+0000 7f19f8a00640 1 -- 172.21.15.89:0/415807344 shutdown_connections 2024-09-10T08:47:31.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.610+0000 7f19f8a00640 1 --2- 172.21.15.89:0/415807344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19f4112a30 0x7f19f4114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.610+0000 7f19f8a00640 1 --2- 172.21.15.89:0/415807344 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19f4077040 0x7f19f41124f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.610+0000 7f19f8a00640 1 --2- 172.21.15.89:0/415807344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19f4076720 0x7f19f4076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.610+0000 7f19f8a00640 1 -- 172.21.15.89:0/415807344 >> 172.21.15.89:0/415807344 conn(0x7f19f40ffa90 msgr2=0x7f19f4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:31.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.610+0000 7f19f8a00640 1 -- 172.21.15.89:0/415807344 shutdown_connections 2024-09-10T08:47:31.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.610+0000 7f19f8a00640 1 -- 172.21.15.89:0/415807344 wait complete. 2024-09-10T08:47:31.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.611+0000 7f19f8a00640 1 Processor -- start 2024-09-10T08:47:31.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.612+0000 7f19f8a00640 1 -- start start 2024-09-10T08:47:31.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.612+0000 7f19f8a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19f4076720 0x7f19f41a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.612+0000 7f19f8a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19f4077040 0x7f19f41a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.612+0000 7f19f8a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19f4112a30 0x7f19f419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.612+0000 7f19f8a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f19f4118610 con 0x7f19f4076720 2024-09-10T08:47:31.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.612+0000 7f19f8a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f19f4118490 con 0x7f19f4077040 2024-09-10T08:47:31.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.612+0000 7f19f8a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f19f4118790 con 0x7f19f4112a30 2024-09-10T08:47:31.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.612+0000 7f19ebe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19f4077040 0x7f19f41a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.612+0000 7f19f1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19f4076720 0x7f19f41a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.612+0000 7f19ebe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19f4077040 0x7f19f41a4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:34576/0 (socket says 172.21.15.89:34576) 2024-09-10T08:47:31.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.612+0000 7f19ebe00640 1 -- 172.21.15.89:0/1354845597 learned_addr learned my addr 172.21.15.89:0/1354845597 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:31.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.613+0000 7f19f2800640 1 --2- 172.21.15.89:0/1354845597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19f4112a30 0x7f19f419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.613+0000 7f19ebe00640 1 -- 172.21.15.89:0/1354845597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19f4112a30 msgr2=0x7f19f419e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:31.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.613+0000 7f19ebe00640 1 --2- 172.21.15.89:0/1354845597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19f4112a30 0x7f19f419e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.613+0000 7f19ebe00640 1 -- 172.21.15.89:0/1354845597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19f4076720 msgr2=0x7f19f41a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:31.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.613+0000 7f19ebe00640 1 --2- 172.21.15.89:0/1354845597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19f4076720 0x7f19f41a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.613+0000 7f19ebe00640 1 -- 172.21.15.89:0/1354845597 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f419f020 con 0x7f19f4077040 2024-09-10T08:47:31.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.613+0000 7f19f2800640 1 --2- 172.21.15.89:0/1354845597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19f4112a30 0x7f19f419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:31.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.613+0000 7f19ebe00640 1 --2- 172.21.15.89:0/1354845597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19f4077040 0x7f19f41a4c70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f19dc004770 tx=0x7f19dc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:31.614 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.613+0000 7f19eaa00640 1 -- 172.21.15.89:0/1354845597 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc03f070 con 0x7f19f4077040 2024-09-10T08:47:31.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.613+0000 7f19eaa00640 1 -- 172.21.15.89:0/1354845597 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19dc031ca0 con 0x7f19f4077040 2024-09-10T08:47:31.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.613+0000 7f19f8a00640 1 -- 172.21.15.89:0/1354845597 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f419f2b0 con 0x7f19f4077040 2024-09-10T08:47:31.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.614+0000 7f19eaa00640 1 -- 172.21.15.89:0/1354845597 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc035030 con 0x7f19f4077040 2024-09-10T08:47:31.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.614+0000 7f19f8a00640 1 -- 172.21.15.89:0/1354845597 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f41ab600 con 0x7f19f4077040 2024-09-10T08:47:31.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.617+0000 7f19fb64c740 1 Processor -- start 2024-09-10T08:47:31.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.617+0000 7f19eaa00640 1 -- 172.21.15.89:0/1354845597 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f19dc042070 con 0x7f19f4077040 2024-09-10T08:47:31.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.617+0000 7f19fb64c740 1 -- start start 2024-09-10T08:47:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.617+0000 7f19fb64c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b93b0 0x5612e95b9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.617+0000 7f19fb64c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b9cd0 0x5612e95c4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.617+0000 7f19fb64c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95c4590 0x5612e95c6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.617+0000 7f19fb64c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5612e951f530 con 0x5612e95c4590 2024-09-10T08:47:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.617+0000 7f19fb64c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5612e94c6860 con 0x5612e95b93b0 2024-09-10T08:47:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.617+0000 7f19fb64c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5612e951b4f0 con 0x5612e95b9cd0 2024-09-10T08:47:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.618+0000 7f19f2800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b93b0 0x5612e95b9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.618+0000 7f19f2800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b93b0 0x5612e95b9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:34586/0 (socket says 172.21.15.89:34586) 2024-09-10T08:47:31.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.618+0000 7f19f2800640 1 -- 172.21.15.89:0/4081222336 learned_addr learned my addr 172.21.15.89:0/4081222336 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:31.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.618+0000 7f19f1e00640 1 --2- 172.21.15.89:0/4081222336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b9cd0 0x5612e95c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.618+0000 7f19ebe00640 1 --2- 172.21.15.89:0/4081222336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95c4590 0x5612e95c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.618+0000 7f19f2800640 1 -- 172.21.15.89:0/4081222336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b9cd0 msgr2=0x5612e95c4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.618+0000 7f19f2800640 1 --2- 172.21.15.89:0/4081222336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b9cd0 0x5612e95c4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.618+0000 7f19f2800640 1 -- 172.21.15.89:0/4081222336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95c4590 msgr2=0x5612e95c6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.618+0000 7f19f2800640 1 --2- 172.21.15.89:0/4081222336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95c4590 0x5612e95c6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.618+0000 7f19f2800640 1 -- 172.21.15.89:0/4081222336 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612e9290b90 con 0x5612e95b93b0 2024-09-10T08:47:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.618+0000 7f19f1e00640 1 --2- 172.21.15.89:0/4081222336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b9cd0 0x5612e95c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.619+0000 7f19ebe00640 1 --2- 172.21.15.89:0/4081222336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95c4590 0x5612e95c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:31.620 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.619+0000 7f19eaa00640 1 --2- 172.21.15.89:0/1354845597 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f19d80a2b80 0x7f19d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.619+0000 7f19f2800640 1 --2- 172.21.15.89:0/4081222336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b93b0 0x5612e95b9790 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009f80 tx=0x7f19ec010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29d422a6250da430 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:31.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.619+0000 7f19eaa00640 1 -- 172.21.15.89:0/1354845597 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f19dc0efaa0 con 0x7f19f4077040 2024-09-10T08:47:31.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.619+0000 7f19e3000640 1 -- 172.21.15.89:0/4081222336 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec011700 con 0x5612e95b93b0 2024-09-10T08:47:31.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.619+0000 7f19e3000640 1 -- 172.21.15.89:0/4081222336 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19ec011d80 con 0x5612e95b93b0 2024-09-10T08:47:31.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.619+0000 7f19e3000640 1 -- 172.21.15.89:0/4081222336 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec00e3c0 con 0x5612e95b93b0 2024-09-10T08:47:31.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.620+0000 7f19f1e00640 1 --2- 172.21.15.89:0/1354845597 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f19d80a2b80 0x7f19d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.620+0000 7f19fb64c740 1 -- 172.21.15.89:0/4081222336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b93b0 msgr2=0x5612e95b9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:31.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.620+0000 7f19fb64c740 1 --2- 172.21.15.89:0/4081222336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b93b0 0x5612e95b9790 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009f80 tx=0x7f19ec010860 comp rx=0 tx=0).stop 2024-09-10T08:47:31.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.620+0000 7f19fb64c740 1 -- 172.21.15.89:0/4081222336 shutdown_connections 2024-09-10T08:47:31.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.620+0000 7f19fb64c740 1 --2- 172.21.15.89:0/4081222336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95c4590 0x5612e95c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.620+0000 7f19fb64c740 1 --2- 172.21.15.89:0/4081222336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b9cd0 0x5612e95c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.620+0000 7f19fb64c740 1 --2- 172.21.15.89:0/4081222336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b93b0 0x5612e95b9790 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.620+0000 7f19fb64c740 1 -- 172.21.15.89:0/4081222336 >> 172.21.15.89:0/4081222336 conn(0x5612e95b5940 msgr2=0x5612e95b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:31.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.620+0000 7f19f1e00640 1 --2- 172.21.15.89:0/1354845597 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f19d80a2b80 0x7f19d80a5040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x5612e95b67a0 tx=0x7f19e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:31.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.620+0000 7f19fb64c740 1 -- 172.21.15.89:0/4081222336 shutdown_connections 2024-09-10T08:47:31.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.620+0000 7f19fb64c740 1 -- 172.21.15.89:0/4081222336 wait complete. 2024-09-10T08:47:31.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.621+0000 7f19fb64c740 1 -- start start 2024-09-10T08:47:31.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.621+0000 7f19fb64c740 1 Processor -- start 2024-09-10T08:47:31.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.622+0000 7f19fb64c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95b93b0 0x5612e95c1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.622+0000 7f19fb64c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b9cd0 0x5612e95c1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.622+0000 7f19fb64c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95c4590 0x5612e95c2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.622+0000 7f19fb64c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5612e95c6fb0 con 0x5612e95b93b0 2024-09-10T08:47:31.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.622+0000 7f19fb64c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5612e94d9e60 con 0x5612e95c4590 2024-09-10T08:47:31.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.622+0000 7f19fb64c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5612e95c7130 con 0x5612e95b9cd0 2024-09-10T08:47:31.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.622+0000 7f19fb64c740 10 client.0 _ll_register_callbacks cb 0x5612e94decc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:31.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.622+0000 7f19f2800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95c4590 0x5612e95c2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.622+0000 7f19f1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95b93b0 0x5612e95c1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.622+0000 7f19f2800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95c4590 0x5612e95c2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:34600/0 (socket says 172.21.15.89:34600) 2024-09-10T08:47:31.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.622+0000 7f19ebe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b9cd0 0x5612e95c1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.622+0000 7f19f2800640 1 -- 172.21.15.89:0/2910098578 learned_addr learned my addr 172.21.15.89:0/2910098578 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:31.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.623+0000 7f19f2800640 1 -- 172.21.15.89:0/2910098578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b9cd0 msgr2=0x5612e95c1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:31.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.623+0000 7f19f2800640 1 --2- 172.21.15.89:0/2910098578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b9cd0 0x5612e95c1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.623+0000 7f19f2800640 1 -- 172.21.15.89:0/2910098578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95b93b0 msgr2=0x5612e95c1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:31.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.623+0000 7f19f2800640 1 --2- 172.21.15.89:0/2910098578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95b93b0 0x5612e95c1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.623+0000 7f19f2800640 1 -- 172.21.15.89:0/2910098578 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612e94e5990 con 0x5612e95c4590 2024-09-10T08:47:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.623+0000 7f19f2800640 1 --2- 172.21.15.89:0/2910098578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95c4590 0x5612e95c2190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f19ec01bb20 tx=0x7f19ec004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b7aa0c151b6f147 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.623+0000 7f19d7400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:47:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.623+0000 7f19d7400640 1 -- 172.21.15.89:0/2910098578 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec02b060 con 0x5612e95c4590 2024-09-10T08:47:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.624+0000 7f19d7400640 1 -- 172.21.15.89:0/2910098578 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f19ec011500 con 0x5612e95c4590 2024-09-10T08:47:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.624+0000 7f19fb64c740 1 -- 172.21.15.89:0/2910098578 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19ec011d80 con 0x5612e95c4590 2024-09-10T08:47:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.624+0000 7f19d7400640 1 -- 172.21.15.89:0/2910098578 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f19ec032050 con 0x5612e95c4590 2024-09-10T08:47:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.624+0000 7f19fb64c740 10 client.26816 fetch_fsmap learned FSMap version 31 2024-09-10T08:47:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.624+0000 7f19fb64c740 1 -- 172.21.15.89:0/2910098578 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5612e95d2d90 con 0x5612e95c4590 2024-09-10T08:47:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.624+0000 7f19d7400640 1 -- 172.21.15.89:0/2910098578 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19ec011d80 con 0x5612e95c4590 2024-09-10T08:47:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.624+0000 7f19d7400640 1 -- 172.21.15.89:0/2910098578 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f19ec0057f0 con 0x5612e95c4590 2024-09-10T08:47:31.627 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.624+0000 7f19fb64c740 10 client.26816 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:47:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.624+0000 7f19fb64c740 10 client.26816 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:47:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.624+0000 7f19fb64c740 20 client.26816 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.624+0000 7f19fb64c740 1 --2- 172.21.15.89:0/2910098578 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5612e95d4250 0x5612e95d6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.625+0000 7f19fb64c740 4 client.26816 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.625+0000 7f19fb64c740 1 -- 172.21.15.89:0/2910098578 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612e94c2310 con 0x5612e95d4250 2024-09-10T08:47:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.625+0000 7f19f1e00640 1 --2- 172.21.15.89:0/2910098578 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5612e95d4250 0x5612e95d6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.625+0000 7f19f1e00640 1 --2- 172.21.15.89:0/2910098578 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5612e95d4250 0x5612e95d6640 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7bcad9bc9641ed1 server_cookie=9b507db2159917f5 in_seq=0 out_seq=0 2024-09-10T08:47:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.625+0000 7f19d7400640 10 client.26816 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.626+0000 7f19d7400640 1 -- 172.21.15.89:0/2910098578 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5612e94c2310 con 0x5612e95d4250 2024-09-10T08:47:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.626+0000 7f19d7400640 10 client.26816 handle_command_reply: tid=0 2024-09-10T08:47:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.626+0000 7f19fb64c740 1 client.26816 shutdown 2024-09-10T08:47:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.626+0000 7f19fb64c740 1 -- 172.21.15.89:0/2910098578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95c4590 msgr2=0x5612e95c2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:31.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.626+0000 7f19fb64c740 1 --2- 172.21.15.89:0/2910098578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95c4590 0x5612e95c2190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f19ec01bb20 tx=0x7f19ec004c70 comp rx=0 tx=0).stop 2024-09-10T08:47:31.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.626+0000 7f19fb64c740 1 -- 172.21.15.89:0/2910098578 shutdown_connections 2024-09-10T08:47:31.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.626+0000 7f19fb64c740 1 --2- 172.21.15.89:0/2910098578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95c4590 0x5612e95c2190 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.626+0000 7f19fb64c740 1 --2- 172.21.15.89:0/2910098578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b9cd0 0x5612e95c1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.626+0000 7f19fb64c740 1 --2- 172.21.15.89:0/2910098578 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5612e95d4250 0x5612e95d6640 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.627+0000 7f19fb64c740 1 --2- 172.21.15.89:0/2910098578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95b93b0 0x5612e95c1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.627+0000 7f19fb64c740 1 -- 172.21.15.89:0/2910098578 >> 172.21.15.89:0/2910098578 conn(0x5612e95b5940 msgr2=0x5612e95b4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:31.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.627+0000 7f19d7400640 0 client.26816 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:31.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.627+0000 7f19fb64c740 1 -- 172.21.15.89:0/2910098578 shutdown_connections 2024-09-10T08:47:31.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.627+0000 7f19fb64c740 1 -- 172.21.15.89:0/2910098578 wait complete. 2024-09-10T08:47:31.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.627+0000 7f19fb64c740 20 client.26816 trim_cache size 0 max 16384 2024-09-10T08:47:31.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.645+0000 7f19fb64c740 1 Processor -- start 2024-09-10T08:47:31.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.645+0000 7f19fb64c740 1 -- start start 2024-09-10T08:47:31.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.645+0000 7f19fb64c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b93b0 0x5612e95b5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.645+0000 7f19fb64c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b9cd0 0x5612e95c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.645+0000 7f19fb64c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95c4590 0x5612e95be9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.645+0000 7f19fb64c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f19f41b7d50 con 0x5612e95c4590 2024-09-10T08:47:31.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.645+0000 7f19fb64c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f19f41b55f0 con 0x5612e95b9cd0 2024-09-10T08:47:31.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.645+0000 7f19fb64c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f19f41b3440 con 0x5612e95b93b0 2024-09-10T08:47:31.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.646+0000 7f19ebe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b9cd0 0x5612e95c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.646+0000 7f19ebe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b9cd0 0x5612e95c3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:34602/0 (socket says 172.21.15.89:34602) 2024-09-10T08:47:31.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.646+0000 7f19ebe00640 1 -- 172.21.15.89:0/295896177 learned_addr learned my addr 172.21.15.89:0/295896177 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:31.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.646+0000 7f19f2800640 1 --2- 172.21.15.89:0/295896177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95c4590 0x5612e95be9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.646+0000 7f19f1e00640 1 --2- 172.21.15.89:0/295896177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b93b0 0x5612e95b5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.646+0000 7f19ebe00640 1 -- 172.21.15.89:0/295896177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b93b0 msgr2=0x5612e95b5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:31.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.646+0000 7f19ebe00640 1 --2- 172.21.15.89:0/295896177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b93b0 0x5612e95b5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.646+0000 7f19ebe00640 1 -- 172.21.15.89:0/295896177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95c4590 msgr2=0x5612e95be9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:31.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.646+0000 7f19ebe00640 1 --2- 172.21.15.89:0/295896177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95c4590 0x5612e95be9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.646+0000 7f19ebe00640 1 -- 172.21.15.89:0/295896177 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612e95c6cf0 con 0x5612e95b9cd0 2024-09-10T08:47:31.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.646+0000 7f19ebe00640 1 --2- 172.21.15.89:0/295896177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b9cd0 0x5612e95c3570 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0096c0 tx=0x7f19dc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b78894cef1a3ba49 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:31.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.647+0000 7f19e2600640 1 -- 172.21.15.89:0/295896177 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc049070 con 0x5612e95b9cd0 2024-09-10T08:47:31.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.647+0000 7f19e2600640 1 -- 172.21.15.89:0/295896177 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19dc0bbff0 con 0x5612e95b9cd0 2024-09-10T08:47:31.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.647+0000 7f19e2600640 1 -- 172.21.15.89:0/295896177 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc03b4f0 con 0x5612e95b9cd0 2024-09-10T08:47:31.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.648+0000 7f19fb64c740 1 -- 172.21.15.89:0/295896177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b9cd0 msgr2=0x5612e95c3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:31.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.648+0000 7f19fb64c740 1 --2- 172.21.15.89:0/295896177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b9cd0 0x5612e95c3570 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0096c0 tx=0x7f19dc009260 comp rx=0 tx=0).stop 2024-09-10T08:47:31.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.648+0000 7f19fb64c740 1 -- 172.21.15.89:0/295896177 shutdown_connections 2024-09-10T08:47:31.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.648+0000 7f19fb64c740 1 --2- 172.21.15.89:0/295896177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95c4590 0x5612e95be9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.648+0000 7f19fb64c740 1 --2- 172.21.15.89:0/295896177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b9cd0 0x5612e95c3570 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.648+0000 7f19fb64c740 1 --2- 172.21.15.89:0/295896177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b93b0 0x5612e95b5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.648+0000 7f19fb64c740 1 -- 172.21.15.89:0/295896177 >> 172.21.15.89:0/295896177 conn(0x5612e95b5940 msgr2=0x5612e95c2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:31.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.648+0000 7f19fb64c740 1 -- 172.21.15.89:0/295896177 shutdown_connections 2024-09-10T08:47:31.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.648+0000 7f19fb64c740 1 -- 172.21.15.89:0/295896177 wait complete. 2024-09-10T08:47:31.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.648+0000 7f19fb64c740 1 -- start start 2024-09-10T08:47:31.649 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.649+0000 7f19fb64c740 1 Processor -- start 2024-09-10T08:47:31.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.649+0000 7f19fb64c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b93b0 0x5612e95d9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.649+0000 7f19fb64c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b9cd0 0x5612e95d9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.649+0000 7f19fb64c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95c4590 0x5612e95d77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.649+0000 7f19fb64c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f19f41b3ab0 con 0x5612e95c4590 2024-09-10T08:47:31.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.649+0000 7f19fb64c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f19f41b4f20 con 0x5612e95b93b0 2024-09-10T08:47:31.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.649+0000 7f19fb64c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f19f41b4180 con 0x5612e95b9cd0 2024-09-10T08:47:31.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.649+0000 7f19fb64c740 10 client.0 _ll_register_callbacks cb 0x7f19ec009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:31.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.650+0000 7f19f1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b93b0 0x5612e95d9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.650+0000 7f19f1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b93b0 0x5612e95d9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:34610/0 (socket says 172.21.15.89:34610) 2024-09-10T08:47:31.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.650+0000 7f19f1e00640 1 -- 172.21.15.89:0/2867952381 learned_addr learned my addr 172.21.15.89:0/2867952381 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:31.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.650+0000 7f19ebe00640 1 --2- 172.21.15.89:0/2867952381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b9cd0 0x5612e95d9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.650+0000 7f19f2800640 1 --2- 172.21.15.89:0/2867952381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95c4590 0x5612e95d77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.650+0000 7f19ebe00640 1 -- 172.21.15.89:0/2867952381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b93b0 msgr2=0x5612e95d9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:31.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.650+0000 7f19ebe00640 1 --2- 172.21.15.89:0/2867952381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b93b0 0x5612e95d9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.650+0000 7f19ebe00640 1 -- 172.21.15.89:0/2867952381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95c4590 msgr2=0x5612e95d77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:31.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.650+0000 7f19ebe00640 1 --2- 172.21.15.89:0/2867952381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95c4590 0x5612e95d77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.650+0000 7f19ebe00640 1 -- 172.21.15.89:0/2867952381 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612e95d7fd0 con 0x5612e95b9cd0 2024-09-10T08:47:31.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.650+0000 7f19f1e00640 1 --2- 172.21.15.89:0/2867952381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b93b0 0x5612e95d9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:31.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.650+0000 7f19f2800640 1 --2- 172.21.15.89:0/2867952381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95c4590 0x5612e95d77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:31.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.651+0000 7f19ebe00640 1 --2- 172.21.15.89:0/2867952381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b9cd0 0x5612e95d9890 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5612e95c28d0 tx=0x7f19dc02a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26164105e5b54674 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:31.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.651+0000 7f19d7400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:47:31.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.651+0000 7f19d7400640 1 -- 172.21.15.89:0/2867952381 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc049070 con 0x5612e95b9cd0 2024-09-10T08:47:31.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.651+0000 7f19fb64c740 1 -- 172.21.15.89:0/2867952381 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19dc0bbff0 con 0x5612e95b9cd0 2024-09-10T08:47:31.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.651+0000 7f19d7400640 1 -- 172.21.15.89:0/2867952381 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f19dc0b8660 con 0x5612e95b9cd0 2024-09-10T08:47:31.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.651+0000 7f19d7400640 1 -- 172.21.15.89:0/2867952381 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f19dc0c03a0 con 0x5612e95b9cd0 2024-09-10T08:47:31.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.651+0000 7f19fb64c740 10 client.26764 fetch_fsmap learned FSMap version 31 2024-09-10T08:47:31.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.651+0000 7f19fb64c740 1 -- 172.21.15.89:0/2867952381 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5612e963f680 con 0x5612e95b9cd0 2024-09-10T08:47:31.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.651+0000 7f19d7400640 1 -- 172.21.15.89:0/2867952381 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19dc0bbff0 con 0x5612e95b9cd0 2024-09-10T08:47:31.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.652+0000 7f19d7400640 1 -- 172.21.15.89:0/2867952381 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f19dc0bb440 con 0x5612e95b9cd0 2024-09-10T08:47:31.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.652+0000 7f19fb64c740 10 client.26764 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:47:31.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.652+0000 7f19fb64c740 10 client.26764 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:47:31.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.652+0000 7f19fb64c740 20 client.26764 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:31.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.652+0000 7f19fb64c740 1 --2- 172.21.15.89:0/2867952381 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5612e963fd70 0x5612e9642160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.652+0000 7f19fb64c740 4 client.26764 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-10T08:47:31.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.652+0000 7f19fb64c740 1 -- 172.21.15.89:0/2867952381 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x5612e952cc60 con 0x5612e963fd70 2024-09-10T08:47:31.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.652+0000 7f19f1e00640 1 --2- 172.21.15.89:0/2867952381 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5612e963fd70 0x5612e9642160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.652+0000 7f19f1e00640 1 --2- 172.21.15.89:0/2867952381 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5612e963fd70 0x5612e9642160 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c49a7187dcd8de7 server_cookie=a35206ab2f222325 in_seq=0 out_seq=0 2024-09-10T08:47:31.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:31.652+0000 7f19d7400640 10 client.26764 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:31.736 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:32.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.411+0000 7fcad5e00640 1 -- 172.21.15.89:0/3195086254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcad01049c0 msgr2=0x7fcad0108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:32.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.411+0000 7fcad5e00640 1 --2- 172.21.15.89:0/3195086254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcad01049c0 0x7fcad0108a10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009900 tx=0x7fcab802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.411+0000 7fcad5e00640 1 -- 172.21.15.89:0/3195086254 shutdown_connections 2024-09-10T08:47:32.411 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.411+0000 7fcad5e00640 1 --2- 172.21.15.89:0/3195086254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcad0109140 0x7fcad0110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.411+0000 7fcad5e00640 1 --2- 172.21.15.89:0/3195086254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcad01049c0 0x7fcad0108a10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.411+0000 7fcad5e00640 1 --2- 172.21.15.89:0/3195086254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcad0104010 0x7fcad01043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.411+0000 7fcad5e00640 1 -- 172.21.15.89:0/3195086254 >> 172.21.15.89:0/3195086254 conn(0x7fcad00ffa50 msgr2=0x7fcad0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:32.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.411+0000 7fcad5e00640 1 -- 172.21.15.89:0/3195086254 shutdown_connections 2024-09-10T08:47:32.412 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.411+0000 7fcad5e00640 1 -- 172.21.15.89:0/3195086254 wait complete. 2024-09-10T08:47:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.412+0000 7fcad5e00640 1 Processor -- start 2024-09-10T08:47:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.412+0000 7fcad5e00640 1 -- start start 2024-09-10T08:47:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.412+0000 7fcad5e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcad0104010 0x7fcad019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.412+0000 7fcad5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcad01049c0 0x7fcad019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.412+0000 7fcad5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcad0109140 0x7fcad019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.412+0000 7fcad5e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcad01141b0 con 0x7fcad0109140 2024-09-10T08:47:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.412+0000 7fcad5e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcad0114030 con 0x7fcad01049c0 2024-09-10T08:47:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.412+0000 7fcad5e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcad0114330 con 0x7fcad0104010 2024-09-10T08:47:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.413+0000 7fcacea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcad0104010 0x7fcad019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.413+0000 7fcacea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcad0104010 0x7fcad019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:56746/0 (socket says 172.21.15.89:56746) 2024-09-10T08:47:32.414 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.413+0000 7fcacea00640 1 -- 172.21.15.89:0/2373730758 learned_addr learned my addr 172.21.15.89:0/2373730758 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.413+0000 7fcacf400640 1 --2- 172.21.15.89:0/2373730758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcad0109140 0x7fcad019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.413+0000 7fcac6000640 1 --2- 172.21.15.89:0/2373730758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcad01049c0 0x7fcad019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.413+0000 7fcacea00640 1 -- 172.21.15.89:0/2373730758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcad01049c0 msgr2=0x7fcad019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.413+0000 7fcacea00640 1 --2- 172.21.15.89:0/2373730758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcad01049c0 0x7fcad019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.413+0000 7fcacea00640 1 -- 172.21.15.89:0/2373730758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcad0109140 msgr2=0x7fcad019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.413+0000 7fcacea00640 1 --2- 172.21.15.89:0/2373730758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcad0109140 0x7fcad019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.413+0000 7fcacea00640 1 -- 172.21.15.89:0/2373730758 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad019f6c0 con 0x7fcad0104010 2024-09-10T08:47:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.413+0000 7fcacea00640 1 --2- 172.21.15.89:0/2373730758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcad0104010 0x7fcad019a710 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcac000e9f0 tx=0x7fcac000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.414+0000 7fcad4a00640 1 -- 172.21.15.89:0/2373730758 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac000cd80 con 0x7fcad0104010 2024-09-10T08:47:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.414+0000 7fcad5e00640 1 -- 172.21.15.89:0/2373730758 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad019f9b0 con 0x7fcad0104010 2024-09-10T08:47:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.414+0000 7fcad4a00640 1 -- 172.21.15.89:0/2373730758 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcac0005500 con 0x7fcad0104010 2024-09-10T08:47:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.414+0000 7fcad4a00640 1 -- 172.21.15.89:0/2373730758 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0010690 con 0x7fcad0104010 2024-09-10T08:47:32.415 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.414+0000 7fcad5e00640 1 -- 172.21.15.89:0/2373730758 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad01a7290 con 0x7fcad0104010 2024-09-10T08:47:32.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.417+0000 7fcad4a00640 1 -- 172.21.15.89:0/2373730758 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fcac00046c0 con 0x7fcad0104010 2024-09-10T08:47:32.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.417+0000 7fcad8a35740 1 Processor -- start 2024-09-10T08:47:32.417 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.417+0000 7fcad8a35740 1 -- start start 2024-09-10T08:47:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.418+0000 7fcad8a35740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540802c70 0x556540803050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.418+0000 7fcad8a35740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556540803590 0x55654080d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.418+0000 7fcad8a35740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55654080de50 0x556540810240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.418+0000 7fcad8a35740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5565407693e0 con 0x556540803590 2024-09-10T08:47:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.418+0000 7fcad8a35740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556540710d80 con 0x55654080de50 2024-09-10T08:47:32.418 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.418+0000 7fcad8a35740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5565407653a0 con 0x556540802c70 2024-09-10T08:47:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.418+0000 7fcacea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556540803590 0x55654080d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.418+0000 7fcacf400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540802c70 0x556540803050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.418+0000 7fcacea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556540803590 0x55654080d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52188/0 (socket says 172.21.15.89:52188) 2024-09-10T08:47:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.418+0000 7fcac6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55654080de50 0x556540810240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:32.419 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.418+0000 7fcacea00640 1 -- 172.21.15.89:0/794583751 learned_addr learned my addr 172.21.15.89:0/794583751 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.418+0000 7fcacf400640 1 -- 172.21.15.89:0/794583751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55654080de50 msgr2=0x556540810240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.418+0000 7fcacf400640 1 --2- 172.21.15.89:0/794583751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55654080de50 0x556540810240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.418+0000 7fcacf400640 1 -- 172.21.15.89:0/794583751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556540803590 msgr2=0x55654080d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.418+0000 7fcacf400640 1 --2- 172.21.15.89:0/794583751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556540803590 0x55654080d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.418+0000 7fcacf400640 1 -- 172.21.15.89:0/794583751 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55654072fae0 con 0x556540802c70 2024-09-10T08:47:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.419+0000 7fcacf400640 1 --2- 172.21.15.89:0/794583751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540802c70 0x556540803050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcac8009f00 tx=0x7fcac8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ac7cac97a88250e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.419+0000 7fcad4a00640 1 --2- 172.21.15.89:0/2373730758 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcab40a2b80 0x7fcab40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:32.421 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.419+0000 7fcabfe00640 1 -- 172.21.15.89:0/794583751 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac8004d20 con 0x556540802c70 2024-09-10T08:47:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.419+0000 7fcabfe00640 1 -- 172.21.15.89:0/794583751 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcac8004050 con 0x556540802c70 2024-09-10T08:47:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.419+0000 7fcabfe00640 1 -- 172.21.15.89:0/794583751 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac8005220 con 0x556540802c70 2024-09-10T08:47:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.419+0000 7fcad4a00640 1 -- 172.21.15.89:0/2373730758 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fcac00cf850 con 0x7fcad0104010 2024-09-10T08:47:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.419+0000 7fcacea00640 1 --2- 172.21.15.89:0/2373730758 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcab40a2b80 0x7fcab40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.420+0000 7fcad8a35740 1 -- 172.21.15.89:0/794583751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540802c70 msgr2=0x556540803050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.420+0000 7fcad8a35740 1 --2- 172.21.15.89:0/794583751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540802c70 0x556540803050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcac8009f00 tx=0x7fcac8011810 comp rx=0 tx=0).stop 2024-09-10T08:47:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.420+0000 7fcad8a35740 1 -- 172.21.15.89:0/794583751 shutdown_connections 2024-09-10T08:47:32.422 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.420+0000 7fcad8a35740 1 --2- 172.21.15.89:0/794583751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55654080de50 0x556540810240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.420+0000 7fcad8a35740 1 --2- 172.21.15.89:0/794583751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556540803590 0x55654080d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.420+0000 7fcad8a35740 1 --2- 172.21.15.89:0/794583751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540802c70 0x556540803050 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.420+0000 7fcad8a35740 1 -- 172.21.15.89:0/794583751 >> 172.21.15.89:0/794583751 conn(0x5565407ff200 msgr2=0x5565407ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.420+0000 7fcacea00640 1 --2- 172.21.15.89:0/2373730758 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcab40a2b80 0x7fcab40a5040 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcac00cfaf0 tx=0x7fcac00d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.420+0000 7fcad8a35740 1 -- 172.21.15.89:0/794583751 shutdown_connections 2024-09-10T08:47:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.420+0000 7fcad8a35740 1 -- 172.21.15.89:0/794583751 wait complete. 2024-09-10T08:47:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.421+0000 7fcad8a35740 1 -- start start 2024-09-10T08:47:32.423 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.421+0000 7fcad8a35740 1 Processor -- start 2024-09-10T08:47:32.426 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.421+0000 7fcad8a35740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540802c70 0x55654080acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:32.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.421+0000 7fcad8a35740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556540803590 0x55654080b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:32.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.421+0000 7fcad8a35740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55654080de50 0x55654080b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:32.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.421+0000 7fcad8a35740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556540810a20 con 0x55654080de50 2024-09-10T08:47:32.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.421+0000 7fcad8a35740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5565407240f0 con 0x556540803590 2024-09-10T08:47:32.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.421+0000 7fcad8a35740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556540810ba0 con 0x556540802c70 2024-09-10T08:47:32.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.421+0000 7fcad8a35740 10 client.0 _ll_register_callbacks cb 0x55654070bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:32.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.422+0000 7fcacf400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540802c70 0x55654080acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:32.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.422+0000 7fcacf400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540802c70 0x55654080acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:56774/0 (socket says 172.21.15.89:56774) 2024-09-10T08:47:32.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.422+0000 7fcacf400640 1 -- 172.21.15.89:0/3975450125 learned_addr learned my addr 172.21.15.89:0/3975450125 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:32.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.422+0000 7fcac6000640 1 --2- 172.21.15.89:0/3975450125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556540803590 0x55654080b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:32.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.422+0000 7fcacf400640 1 --2- 172.21.15.89:0/3975450125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55654080de50 0x55654080b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:32.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.422+0000 7fcacf400640 1 -- 172.21.15.89:0/3975450125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556540803590 msgr2=0x55654080b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:32.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.422+0000 7fcacf400640 1 --2- 172.21.15.89:0/3975450125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556540803590 0x55654080b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.422+0000 7fcacf400640 1 -- 172.21.15.89:0/3975450125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55654080de50 msgr2=0x55654080b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:32.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.422+0000 7fcacf400640 1 --2- 172.21.15.89:0/3975450125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55654080de50 0x55654080b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.422+0000 7fcacf400640 1 -- 172.21.15.89:0/3975450125 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565404da3c0 con 0x556540802c70 2024-09-10T08:47:32.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.422+0000 7fcac6000640 1 --2- 172.21.15.89:0/3975450125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556540803590 0x55654080b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:32.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.422+0000 7fcacf400640 1 --2- 172.21.15.89:0/3975450125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540802c70 0x55654080acf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcac8009d40 tx=0x7fcac80048d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=384103598782815f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:32.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.422+0000 7fcabcc00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:47:32.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.423+0000 7fcabcc00640 1 -- 172.21.15.89:0/3975450125 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac802c020 con 0x556540802c70 2024-09-10T08:47:32.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.423+0000 7fcad8a35740 1 -- 172.21.15.89:0/3975450125 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcac8004050 con 0x556540802c70 2024-09-10T08:47:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.423+0000 7fcabcc00640 1 -- 172.21.15.89:0/3975450125 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fcac8004610 con 0x556540802c70 2024-09-10T08:47:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.423+0000 7fcabcc00640 1 -- 172.21.15.89:0/3975450125 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcac8037050 con 0x556540802c70 2024-09-10T08:47:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.423+0000 7fcad8a35740 10 client.26776 fetch_fsmap learned FSMap version 31 2024-09-10T08:47:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.423+0000 7fcad8a35740 1 -- 172.21.15.89:0/3975450125 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55654081ca10 con 0x556540802c70 2024-09-10T08:47:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.423+0000 7fcabcc00640 1 -- 172.21.15.89:0/3975450125 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcac8004050 con 0x556540802c70 2024-09-10T08:47:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.424+0000 7fcabcc00640 1 -- 172.21.15.89:0/3975450125 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fcac8028900 con 0x556540802c70 2024-09-10T08:47:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.424+0000 7fcad8a35740 10 client.26776 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:47:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.424+0000 7fcad8a35740 10 client.26776 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:47:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.424+0000 7fcad8a35740 20 client.26776 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.424+0000 7fcad8a35740 1 --2- 172.21.15.89:0/3975450125 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55654081de20 0x556540820210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.424+0000 7fcad8a35740 4 client.26776 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.424+0000 7fcad8a35740 1 -- 172.21.15.89:0/3975450125 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556540776950 con 0x55654081de20 2024-09-10T08:47:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.424+0000 7fcac6000640 1 --2- 172.21.15.89:0/3975450125 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55654081de20 0x556540820210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:32.429 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.425+0000 7fcac6000640 1 --2- 172.21.15.89:0/3975450125 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55654081de20 0x556540820210 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=249379ea0a04a475 server_cookie=f93855cd5e34c7ad in_seq=0 out_seq=0 2024-09-10T08:47:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.425+0000 7fcabcc00640 10 client.26776 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:47:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.425+0000 7fcabcc00640 1 -- 172.21.15.89:0/3975450125 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556540776950 con 0x55654081de20 2024-09-10T08:47:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.425+0000 7fcabcc00640 10 client.26776 handle_command_reply: tid=0 2024-09-10T08:47:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.425+0000 7fcad8a35740 1 client.26776 shutdown 2024-09-10T08:47:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.425+0000 7fcad8a35740 1 -- 172.21.15.89:0/3975450125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540802c70 msgr2=0x55654080acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.425+0000 7fcad8a35740 1 --2- 172.21.15.89:0/3975450125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540802c70 0x55654080acf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcac8009d40 tx=0x7fcac80048d0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.426+0000 7fcad8a35740 1 -- 172.21.15.89:0/3975450125 shutdown_connections 2024-09-10T08:47:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.426+0000 7fcad8a35740 1 --2- 172.21.15.89:0/3975450125 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x55654081de20 0x556540820210 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.426+0000 7fcad8a35740 1 --2- 172.21.15.89:0/3975450125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55654080de50 0x55654080b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.426+0000 7fcad8a35740 1 --2- 172.21.15.89:0/3975450125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556540803590 0x55654080b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.426+0000 7fcabcc00640 0 client.26776 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:47:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.426+0000 7fcad8a35740 1 --2- 172.21.15.89:0/3975450125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540802c70 0x55654080acf0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.426+0000 7fcad8a35740 1 -- 172.21.15.89:0/3975450125 >> 172.21.15.89:0/3975450125 conn(0x5565407ff200 msgr2=0x5565407fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:32.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.426+0000 7fcad8a35740 1 -- 172.21.15.89:0/3975450125 shutdown_connections 2024-09-10T08:47:32.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.426+0000 7fcad8a35740 1 -- 172.21.15.89:0/3975450125 wait complete. 2024-09-10T08:47:32.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.426+0000 7fcad8a35740 20 client.26776 trim_cache size 0 max 16384 2024-09-10T08:47:32.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.439+0000 7fcad8a35740 1 Processor -- start 2024-09-10T08:47:32.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.439+0000 7fcad8a35740 1 -- start start 2024-09-10T08:47:32.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.440+0000 7fcad8a35740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540802c70 0x55654080c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:32.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.440+0000 7fcad8a35740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556540803590 0x5565408259a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:32.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.440+0000 7fcad8a35740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55654080de50 0x556540825ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:32.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.440+0000 7fcad8a35740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcad01b3930 con 0x55654080de50 2024-09-10T08:47:32.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.440+0000 7fcad8a35740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcad01b11d0 con 0x556540803590 2024-09-10T08:47:32.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.440+0000 7fcad8a35740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcad01af020 con 0x556540802c70 2024-09-10T08:47:32.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.440+0000 7fcacf400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540802c70 0x55654080c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:32.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.440+0000 7fcac6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556540803590 0x5565408259a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:32.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.440+0000 7fcacf400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540802c70 0x55654080c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:56790/0 (socket says 172.21.15.89:56790) 2024-09-10T08:47:32.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.440+0000 7fcacf400640 1 -- 172.21.15.89:0/2800520865 learned_addr learned my addr 172.21.15.89:0/2800520865 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:32.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.440+0000 7fcacf400640 1 --2- 172.21.15.89:0/2800520865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55654080de50 0x556540825ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:32.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.440+0000 7fcac6000640 1 -- 172.21.15.89:0/2800520865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540802c70 msgr2=0x55654080c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:32.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.440+0000 7fcac6000640 1 --2- 172.21.15.89:0/2800520865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540802c70 0x55654080c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.440+0000 7fcac6000640 1 -- 172.21.15.89:0/2800520865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55654080de50 msgr2=0x556540825ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:32.442 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.440+0000 7fcac6000640 1 --2- 172.21.15.89:0/2800520865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55654080de50 0x556540825ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.440+0000 7fcac6000640 1 -- 172.21.15.89:0/2800520865 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565407fa770 con 0x556540803590 2024-09-10T08:47:32.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.440+0000 7fcacf400640 1 --2- 172.21.15.89:0/2800520865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55654080de50 0x556540825ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:32.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.441+0000 7fcac6000640 1 --2- 172.21.15.89:0/2800520865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556540803590 0x5565408259a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcab802a9e0 tx=0x7fcab8005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53a7c2c4872a36b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:32.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.441+0000 7fcabf400640 1 -- 172.21.15.89:0/2800520865 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8035040 con 0x556540803590 2024-09-10T08:47:32.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.441+0000 7fcabf400640 1 -- 172.21.15.89:0/2800520865 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab804b070 con 0x556540803590 2024-09-10T08:47:32.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.441+0000 7fcabf400640 1 -- 172.21.15.89:0/2800520865 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8031810 con 0x556540803590 2024-09-10T08:47:32.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.442+0000 7fcad8a35740 1 -- 172.21.15.89:0/2800520865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556540803590 msgr2=0x5565408259a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:32.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.442+0000 7fcad8a35740 1 --2- 172.21.15.89:0/2800520865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556540803590 0x5565408259a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcab802a9e0 tx=0x7fcab8005b50 comp rx=0 tx=0).stop 2024-09-10T08:47:32.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.442+0000 7fcad8a35740 1 -- 172.21.15.89:0/2800520865 shutdown_connections 2024-09-10T08:47:32.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.442+0000 7fcad8a35740 1 --2- 172.21.15.89:0/2800520865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55654080de50 0x556540825ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.443 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.442+0000 7fcad8a35740 1 --2- 172.21.15.89:0/2800520865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556540803590 0x5565408259a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.442+0000 7fcad8a35740 1 --2- 172.21.15.89:0/2800520865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540802c70 0x55654080c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.442+0000 7fcad8a35740 1 -- 172.21.15.89:0/2800520865 >> 172.21.15.89:0/2800520865 conn(0x5565407ff200 msgr2=0x556540804ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:32.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.442+0000 7fcad8a35740 1 -- 172.21.15.89:0/2800520865 shutdown_connections 2024-09-10T08:47:32.444 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.442+0000 7fcad8a35740 1 -- 172.21.15.89:0/2800520865 wait complete. 2024-09-10T08:47:32.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.442+0000 7fcad8a35740 1 -- start start 2024-09-10T08:47:32.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.443+0000 7fcad8a35740 1 Processor -- start 2024-09-10T08:47:32.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.443+0000 7fcad8a35740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556540802c70 0x556540816620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:32.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.443+0000 7fcad8a35740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540803590 0x556540811950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:32.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.443+0000 7fcad8a35740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55654080de50 0x556540811f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:32.447 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.443+0000 7fcad8a35740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcad01af690 con 0x556540802c70 2024-09-10T08:47:32.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.443+0000 7fcad8a35740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcad01b0b00 con 0x55654080de50 2024-09-10T08:47:32.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.443+0000 7fcad8a35740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcad01afd60 con 0x556540803590 2024-09-10T08:47:32.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.443+0000 7fcad8a35740 10 client.0 _ll_register_callbacks cb 0x7fcac00d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:32.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.443+0000 7fcac6000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540803590 0x556540811950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:32.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.443+0000 7fcac6000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540803590 0x556540811950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:56798/0 (socket says 172.21.15.89:56798) 2024-09-10T08:47:32.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.443+0000 7fcac6000640 1 -- 172.21.15.89:0/2149690217 learned_addr learned my addr 172.21.15.89:0/2149690217 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:32.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.443+0000 7fcacf400640 1 --2- 172.21.15.89:0/2149690217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556540802c70 0x556540816620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:32.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.444+0000 7fcacf400640 1 --2- 172.21.15.89:0/2149690217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55654080de50 0x556540811f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:32.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.444+0000 7fcac6000640 1 -- 172.21.15.89:0/2149690217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55654080de50 msgr2=0x556540811f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:32.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.444+0000 7fcac6000640 1 --2- 172.21.15.89:0/2149690217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55654080de50 0x556540811f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.448 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.444+0000 7fcac6000640 1 -- 172.21.15.89:0/2149690217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556540802c70 msgr2=0x556540816620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:32.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.444+0000 7fcac6000640 1 --2- 172.21.15.89:0/2149690217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556540802c70 0x556540816620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.444+0000 7fcac6000640 1 -- 172.21.15.89:0/2149690217 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556540805ae0 con 0x556540803590 2024-09-10T08:47:32.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.444+0000 7fcacf400640 1 --2- 172.21.15.89:0/2149690217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55654080de50 0x556540811f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:32.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.444+0000 7fcac6000640 1 --2- 172.21.15.89:0/2149690217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540803590 0x556540811950 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcab8031550 tx=0x7fcab8005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fe5618679375cee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:32.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.444+0000 7fcabcc00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:47:32.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.444+0000 7fcabcc00640 1 -- 172.21.15.89:0/2149690217 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8053070 con 0x556540803590 2024-09-10T08:47:32.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.444+0000 7fcad8a35740 1 -- 172.21.15.89:0/2149690217 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab804b070 con 0x556540803590 2024-09-10T08:47:32.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.444+0000 7fcabcc00640 1 -- 172.21.15.89:0/2149690217 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fcab80317b0 con 0x556540803590 2024-09-10T08:47:32.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.444+0000 7fcabcc00640 1 -- 172.21.15.89:0/2149690217 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcab8035050 con 0x556540803590 2024-09-10T08:47:32.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.445+0000 7fcad8a35740 10 client.26782 fetch_fsmap learned FSMap version 31 2024-09-10T08:47:32.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.445+0000 7fcad8a35740 1 -- 172.21.15.89:0/2149690217 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556540891080 con 0x556540803590 2024-09-10T08:47:32.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.445+0000 7fcabcc00640 1 -- 172.21.15.89:0/2149690217 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab804b070 con 0x556540803590 2024-09-10T08:47:32.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.445+0000 7fcabcc00640 1 -- 172.21.15.89:0/2149690217 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fcab8003bd0 con 0x556540803590 2024-09-10T08:47:32.449 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.445+0000 7fcad8a35740 10 client.26782 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:47:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.445+0000 7fcad8a35740 10 client.26782 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:47:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.445+0000 7fcad8a35740 20 client.26782 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.445+0000 7fcad8a35740 1 --2- 172.21.15.89:0/2149690217 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x556540891740 0x556540893b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.445+0000 7fcad8a35740 4 client.26782 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:47:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.445+0000 7fcad8a35740 1 -- 172.21.15.89:0/2149690217 --> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556540770e60 con 0x556540891740 2024-09-10T08:47:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.445+0000 7fcacf400640 1 --2- 172.21.15.89:0/2149690217 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x556540891740 0x556540893b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.446+0000 7fcacf400640 1 --2- 172.21.15.89:0/2149690217 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x556540891740 0x556540893b60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4bf8c9b2455b294 server_cookie=d08f634929447485 in_seq=0 out_seq=0 2024-09-10T08:47:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.446+0000 7fcabcc00640 10 client.26782 ms_handle_connect on v2:172.21.15.111:6836/779590543 2024-09-10T08:47:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.446+0000 7fcabcc00640 1 -- 172.21.15.89:0/2149690217 <== mds.1 v2:172.21.15.111:6836/779590543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2674 (crc 0 0 0) 0x556540770e60 con 0x556540891740 2024-09-10T08:47:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.446+0000 7fcabcc00640 10 client.26782 handle_command_reply: tid=0 2024-09-10T08:47:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.446+0000 7fcad8a35740 1 client.26782 shutdown 2024-09-10T08:47:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.446+0000 7fcad8a35740 1 -- 172.21.15.89:0/2149690217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540803590 msgr2=0x556540811950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.446+0000 7fcad8a35740 1 --2- 172.21.15.89:0/2149690217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540803590 0x556540811950 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcab8031550 tx=0x7fcab8005fd0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.446+0000 7fcad8a35740 1 -- 172.21.15.89:0/2149690217 shutdown_connections 2024-09-10T08:47:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.446+0000 7fcad8a35740 1 --2- 172.21.15.89:0/2149690217 >> [v2:172.21.15.111:6836/779590543,v1:172.21.15.111:6837/779590543] conn(0x556540891740 0x556540893b60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.446+0000 7fcad8a35740 1 --2- 172.21.15.89:0/2149690217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55654080de50 0x556540811f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.446+0000 7fcad8a35740 1 --2- 172.21.15.89:0/2149690217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556540803590 0x556540811950 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.446+0000 7fcad8a35740 1 --2- 172.21.15.89:0/2149690217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556540802c70 0x556540816620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.446+0000 7fcabcc00640 0 client.26782 ms_handle_reset on v2:172.21.15.111:6836/779590543 2024-09-10T08:47:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.446+0000 7fcad8a35740 1 -- 172.21.15.89:0/2149690217 >> 172.21.15.89:0/2149690217 conn(0x5565407ff200 msgr2=0x5565407fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.446+0000 7fcad8a35740 1 -- 172.21.15.89:0/2149690217 shutdown_connections 2024-09-10T08:47:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.446+0000 7fcad8a35740 1 -- 172.21.15.89:0/2149690217 wait complete. 2024-09-10T08:47:32.451 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.447+0000 7fcad8a35740 20 client.26782 trim_cache size 0 max 16384 2024-09-10T08:47:32.451 INFO:teuthology.orchestra.run.smithi089.stdout:[ 2024-09-10T08:47:32.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:47:32.452 INFO:teuthology.orchestra.run.smithi089.stdout: "damage_type": "backtrace", 2024-09-10T08:47:32.452 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 715696985, 2024-09-10T08:47:32.452 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 1099511652059, 2024-09-10T08:47:32.452 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-4.xml" 2024-09-10T08:47:32.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:47:32.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:47:32.452 INFO:teuthology.orchestra.run.smithi089.stdout: "damage_type": "backtrace", 2024-09-10T08:47:32.452 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 822296160, 2024-09-10T08:47:32.452 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 1099511652084, 2024-09-10T08:47:32.452 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-5.jpg" 2024-09-10T08:47:32.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:47:32.452 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:47:32.453 INFO:teuthology.orchestra.run.smithi089.stdout: "damage_type": "backtrace", 2024-09-10T08:47:32.453 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 1114910407, 2024-09-10T08:47:32.453 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 1099511652067, 2024-09-10T08:47:32.453 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-2.xml" 2024-09-10T08:47:32.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:47:32.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:47:32.453 INFO:teuthology.orchestra.run.smithi089.stdout: "damage_type": "backtrace", 2024-09-10T08:47:32.453 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 1529065628, 2024-09-10T08:47:32.453 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 1099511652068, 2024-09-10T08:47:32.453 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-1.xml" 2024-09-10T08:47:32.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:47:32.453 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:47:32.453 INFO:teuthology.orchestra.run.smithi089.stdout: "damage_type": "backtrace", 2024-09-10T08:47:32.453 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 1552237096, 2024-09-10T08:47:32.454 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 1099511652077, 2024-09-10T08:47:32.454 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-7.jpg" 2024-09-10T08:47:32.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:47:32.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:47:32.454 INFO:teuthology.orchestra.run.smithi089.stdout: "damage_type": "backtrace", 2024-09-10T08:47:32.454 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 1681867536, 2024-09-10T08:47:32.454 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 1099511652088, 2024-09-10T08:47:32.454 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-4.jpg" 2024-09-10T08:47:32.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:47:32.454 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:47:32.454 INFO:teuthology.orchestra.run.smithi089.stdout: "damage_type": "backtrace", 2024-09-10T08:47:32.454 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 2612536088, 2024-09-10T08:47:32.454 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 1099511652055, 2024-09-10T08:47:32.454 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-5.xml" 2024-09-10T08:47:32.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:47:32.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:47:32.455 INFO:teuthology.orchestra.run.smithi089.stdout: "damage_type": "backtrace", 2024-09-10T08:47:32.455 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 2936482752, 2024-09-10T08:47:32.455 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 1099511652075, 2024-09-10T08:47:32.455 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-8.jpg" 2024-09-10T08:47:32.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:47:32.455 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:47:32.455 INFO:teuthology.orchestra.run.smithi089.stdout: "damage_type": "backtrace", 2024-09-10T08:47:32.455 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 3034715200, 2024-09-10T08:47:32.455 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 1099511652080, 2024-09-10T08:47:32.455 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-6.jpg" 2024-09-10T08:47:32.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:47:32.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:47:32.456 INFO:teuthology.orchestra.run.smithi089.stdout: "damage_type": "backtrace", 2024-09-10T08:47:32.456 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 3183236255, 2024-09-10T08:47:32.456 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 1099511652071, 2024-09-10T08:47:32.456 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-9.jpg" 2024-09-10T08:47:32.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:47:32.456 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:47:32.456 INFO:teuthology.orchestra.run.smithi089.stdout: "damage_type": "backtrace", 2024-09-10T08:47:32.456 INFO:teuthology.orchestra.run.smithi089.stdout: "id": 3249215083, 2024-09-10T08:47:32.456 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 1099511652064, 2024-09-10T08:47:32.456 INFO:teuthology.orchestra.run.smithi089.stdout: "path": "/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-3.xml" 2024-09-10T08:47:32.456 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:32.456 INFO:teuthology.orchestra.run.smithi089.stdout:] 2024-09-10T08:47:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.447+0000 7fcad5e00640 1 -- 172.21.15.89:0/2373730758 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcab40a2b80 msgr2=0x7fcab40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.447+0000 7fcad5e00640 1 --2- 172.21.15.89:0/2373730758 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcab40a2b80 0x7fcab40a5040 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcac00cfaf0 tx=0x7fcac00d5530 comp rx=0 tx=0).stop 2024-09-10T08:47:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.447+0000 7fcad5e00640 1 -- 172.21.15.89:0/2373730758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcad0104010 msgr2=0x7fcad019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.447+0000 7fcad5e00640 1 --2- 172.21.15.89:0/2373730758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcad0104010 0x7fcad019a710 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcac000e9f0 tx=0x7fcac000edf0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.447+0000 7fcad5e00640 1 -- 172.21.15.89:0/2373730758 shutdown_connections 2024-09-10T08:47:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.447+0000 7fcad5e00640 1 --2- 172.21.15.89:0/2373730758 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fcab40a2b80 0x7fcab40a5040 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.447+0000 7fcad5e00640 1 --2- 172.21.15.89:0/2373730758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcad0109140 0x7fcad019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.447+0000 7fcad5e00640 1 --2- 172.21.15.89:0/2373730758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcad01049c0 0x7fcad019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.447+0000 7fcad5e00640 1 --2- 172.21.15.89:0/2373730758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcad0104010 0x7fcad019a710 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:32.457 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.447+0000 7fcad5e00640 1 -- 172.21.15.89:0/2373730758 >> 172.21.15.89:0/2373730758 conn(0x7fcad00ffa50 msgr2=0x7fcad0100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:32.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.447+0000 7fcad5e00640 1 -- 172.21.15.89:0/2373730758 shutdown_connections 2024-09-10T08:47:32.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:32.447+0000 7fcad5e00640 1 -- 172.21.15.89:0/2373730758 wait complete. 2024-09-10T08:47:33.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:32 smithi089 ceph-mon[30757]: pgmap v621: 129 pgs: 129 active+clean; 435 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 514 op/s 2024-09-10T08:47:33.193 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:47:33.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:32 smithi149 ceph-mon[36167]: pgmap v621: 129 pgs: 129 active+clean; 435 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 514 op/s 2024-09-10T08:47:33.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:32 smithi111 ceph-mon[35220]: pgmap v621: 129 pgs: 129 active+clean; 435 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 514 op/s 2024-09-10T08:47:33.555 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.420+0000 7f3ca4200640 1 -- 172.21.15.89:0/3591414978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c9c105910 msgr2=0x7f3c9c10c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.420+0000 7f3ca4200640 1 --2- 172.21.15.89:0/3591414978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c9c105910 0x7f3c9c10c1a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009900 tx=0x7f3c9402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.420+0000 7f3ca4200640 1 -- 172.21.15.89:0/3591414978 shutdown_connections 2024-09-10T08:47:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.420+0000 7f3ca4200640 1 --2- 172.21.15.89:0/3591414978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c9c105910 0x7f3c9c10c1a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.420+0000 7f3ca4200640 1 --2- 172.21.15.89:0/3591414978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c9c104f50 0x7f3c9c1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.420+0000 7f3ca4200640 1 --2- 172.21.15.89:0/3591414978 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c9c103d60 0x7f3c9c104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.420+0000 7f3ca4200640 1 -- 172.21.15.89:0/3591414978 >> 172.21.15.89:0/3591414978 conn(0x7f3c9c0ff4e0 msgr2=0x7f3c9c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.420+0000 7f3ca4200640 1 -- 172.21.15.89:0/3591414978 shutdown_connections 2024-09-10T08:47:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.420+0000 7f3ca4200640 1 -- 172.21.15.89:0/3591414978 wait complete. 2024-09-10T08:47:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.421+0000 7f3ca4200640 1 Processor -- start 2024-09-10T08:47:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.421+0000 7f3ca4200640 1 -- start start 2024-09-10T08:47:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.421+0000 7f3ca4200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c9c103d60 0x7f3c9c199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.421+0000 7f3ca4200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c9c104f50 0x7f3c9c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.421+0000 7f3ca4200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c9c105910 0x7f3c9c1a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.421+0000 7f3ca4200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c10f9e0 con 0x7f3c9c103d60 2024-09-10T08:47:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.421+0000 7f3ca4200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c10f860 con 0x7f3c9c105910 2024-09-10T08:47:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.421+0000 7f3ca4200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c10fb60 con 0x7f3c9c104f50 2024-09-10T08:47:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.421+0000 7f3ca1800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c9c103d60 0x7f3c9c199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.421+0000 7f3ca1800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c9c103d60 0x7f3c9c199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55958/0 (socket says 172.21.15.89:55958) 2024-09-10T08:47:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.421+0000 7f3ca1800640 1 -- 172.21.15.89:0/1796455219 learned_addr learned my addr 172.21.15.89:0/1796455219 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.421+0000 7f3ca2200640 1 --2- 172.21.15.89:0/1796455219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c9c105910 0x7f3c9c1a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.422+0000 7f3ca0e00640 1 --2- 172.21.15.89:0/1796455219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c9c104f50 0x7f3c9c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.422+0000 7f3ca1800640 1 -- 172.21.15.89:0/1796455219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c9c104f50 msgr2=0x7f3c9c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.422+0000 7f3ca1800640 1 --2- 172.21.15.89:0/1796455219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c9c104f50 0x7f3c9c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.422+0000 7f3ca1800640 1 -- 172.21.15.89:0/1796455219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c9c105910 msgr2=0x7f3c9c1a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.422+0000 7f3ca1800640 1 --2- 172.21.15.89:0/1796455219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c9c105910 0x7f3c9c1a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.422+0000 7f3ca1800640 1 -- 172.21.15.89:0/1796455219 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9c1a1b70 con 0x7f3c9c103d60 2024-09-10T08:47:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.422+0000 7f3ca1800640 1 --2- 172.21.15.89:0/1796455219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c9c103d60 0x7f3c9c199f50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000ebc0 tx=0x7f3c9000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.422+0000 7f3c9a000640 1 -- 172.21.15.89:0/1796455219 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c90003b70 con 0x7f3c9c103d60 2024-09-10T08:47:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.422+0000 7f3c9a000640 1 -- 172.21.15.89:0/1796455219 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c90003d10 con 0x7f3c9c103d60 2024-09-10T08:47:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.422+0000 7f3ca4200640 1 -- 172.21.15.89:0/1796455219 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c9c1a1e60 con 0x7f3c9c103d60 2024-09-10T08:47:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.422+0000 7f3c9a000640 1 -- 172.21.15.89:0/1796455219 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c90019980 con 0x7f3c9c103d60 2024-09-10T08:47:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.423+0000 7f3ca4200640 1 -- 172.21.15.89:0/1796455219 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c9c1a2320 con 0x7f3c9c103d60 2024-09-10T08:47:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.425+0000 7f3ca6d22740 1 Processor -- start 2024-09-10T08:47:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.425+0000 7f3c9a000640 1 -- 172.21.15.89:0/1796455219 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3c90019b20 con 0x7f3c9c103d60 2024-09-10T08:47:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.426+0000 7f3ca6d22740 1 -- start start 2024-09-10T08:47:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.426+0000 7f3ca6d22740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc090 0x55c2dbabc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.426+0000 7f3ca6d22740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbabc9d0 0x55c2dbac1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.426+0000 7f3ca6d22740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbac1a60 0x55c2dbac3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.426+0000 7f3ca6d22740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c2dba21100 con 0x55c2dbabc090 2024-09-10T08:47:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.426+0000 7f3ca6d22740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c2db9c91b0 con 0x55c2dbabc9d0 2024-09-10T08:47:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.426+0000 7f3ca6d22740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c2dba1d0c0 con 0x55c2dbac1a60 2024-09-10T08:47:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.426+0000 7f3ca2200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc090 0x55c2dbabc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.426+0000 7f3ca1800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbabc9d0 0x55c2dbac1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.426+0000 7f3ca2200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc090 0x55c2dbabc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55960/0 (socket says 172.21.15.89:55960) 2024-09-10T08:47:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.426+0000 7f3ca1800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbabc9d0 0x55c2dbac1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:54964/0 (socket says 172.21.15.89:54964) 2024-09-10T08:47:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.426+0000 7f3ca2200640 1 -- 172.21.15.89:0/3747699157 learned_addr learned my addr 172.21.15.89:0/3747699157 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.426+0000 7f3ca0e00640 1 --2- 172.21.15.89:0/3747699157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbac1a60 0x55c2dbac3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.427+0000 7f3ca2200640 1 -- 172.21.15.89:0/3747699157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbac1a60 msgr2=0x55c2dbac3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.427+0000 7f3ca2200640 1 --2- 172.21.15.89:0/3747699157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbac1a60 0x55c2dbac3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.427+0000 7f3ca2200640 1 -- 172.21.15.89:0/3747699157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbabc9d0 msgr2=0x55c2dbac1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.427+0000 7f3ca2200640 1 --2- 172.21.15.89:0/3747699157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbabc9d0 0x55c2dbac1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.427+0000 7f3ca2200640 1 -- 172.21.15.89:0/3747699157 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2db734210 con 0x55c2dbabc090 2024-09-10T08:47:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.427+0000 7f3ca1800640 1 --2- 172.21.15.89:0/3747699157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbabc9d0 0x55c2dbac1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.427+0000 7f3c9a000640 1 --2- 172.21.15.89:0/1796455219 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3c840a2cb0 0x7f3c840a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.435 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.427+0000 7f3ca2200640 1 --2- 172.21.15.89:0/3747699157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc090 0x55c2dbabc490 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009900 tx=0x7f3c94005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f0f4551cb0d2df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:34.435 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.427+0000 7f3c9a000640 1 -- 172.21.15.89:0/1796455219 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f3c900cfba0 con 0x7f3c9c103d60 2024-09-10T08:47:34.435 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.427+0000 7f3ca1800640 1 --2- 172.21.15.89:0/1796455219 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3c840a2cb0 0x7f3c840a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.435 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.427+0000 7f3c8ea00640 1 -- 172.21.15.89:0/3747699157 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94040070 con 0x55c2dbabc090 2024-09-10T08:47:34.435 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.427+0000 7f3c8ea00640 1 -- 172.21.15.89:0/3747699157 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c940319f0 con 0x55c2dbabc090 2024-09-10T08:47:34.435 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.427+0000 7f3c8ea00640 1 -- 172.21.15.89:0/3747699157 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94035040 con 0x55c2dbabc090 2024-09-10T08:47:34.436 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.428+0000 7f3ca6d22740 1 -- 172.21.15.89:0/3747699157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc090 msgr2=0x55c2dbabc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.436 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.428+0000 7f3ca1800640 1 --2- 172.21.15.89:0/1796455219 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3c840a2cb0 0x7f3c840a5170 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3c900cd640 tx=0x7f3c900d3680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:34.436 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.428+0000 7f3ca6d22740 1 --2- 172.21.15.89:0/3747699157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc090 0x55c2dbabc490 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009900 tx=0x7f3c94005c00 comp rx=0 tx=0).stop 2024-09-10T08:47:34.436 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.428+0000 7f3ca6d22740 1 -- 172.21.15.89:0/3747699157 shutdown_connections 2024-09-10T08:47:34.436 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.428+0000 7f3ca6d22740 1 --2- 172.21.15.89:0/3747699157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbac1a60 0x55c2dbac3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.436 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.428+0000 7f3ca6d22740 1 --2- 172.21.15.89:0/3747699157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbabc9d0 0x55c2dbac1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.436 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.428+0000 7f3ca6d22740 1 --2- 172.21.15.89:0/3747699157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc090 0x55c2dbabc490 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.437 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.428+0000 7f3ca6d22740 1 -- 172.21.15.89:0/3747699157 >> 172.21.15.89:0/3747699157 conn(0x55c2dbab7700 msgr2=0x55c2dbab7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:34.437 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.428+0000 7f3ca6d22740 1 -- 172.21.15.89:0/3747699157 shutdown_connections 2024-09-10T08:47:34.437 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.428+0000 7f3ca6d22740 1 -- 172.21.15.89:0/3747699157 wait complete. 2024-09-10T08:47:34.437 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.429+0000 7f3ca6d22740 1 -- start start 2024-09-10T08:47:34.437 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.429+0000 7f3ca6d22740 1 Processor -- start 2024-09-10T08:47:34.437 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.429+0000 7f3ca6d22740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbabc090 0x55c2dbac6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.437 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.429+0000 7f3ca6d22740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc9d0 0x55c2dbac6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.437 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.429+0000 7f3ca6d22740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbac1a60 0x55c2dbacd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.438 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.429+0000 7f3ca6d22740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c2dbac49d0 con 0x55c2dbabc9d0 2024-09-10T08:47:34.438 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.429+0000 7f3ca6d22740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c2db9aa090 con 0x55c2dbac1a60 2024-09-10T08:47:34.438 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.429+0000 7f3ca6d22740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c2dbac4b50 con 0x55c2dbabc090 2024-09-10T08:47:34.438 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.430+0000 7f3ca6d22740 10 client.0 _ll_register_callbacks cb 0x55c2db9fef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:34.438 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.430+0000 7f3ca0e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc9d0 0x55c2dbac6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.438 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.430+0000 7f3ca0e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc9d0 0x55c2dbac6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55966/0 (socket says 172.21.15.89:55966) 2024-09-10T08:47:34.438 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.430+0000 7f3ca0e00640 1 -- 172.21.15.89:0/2083481849 learned_addr learned my addr 172.21.15.89:0/2083481849 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:34.438 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.430+0000 7f3ca2200640 1 --2- 172.21.15.89:0/2083481849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbac1a60 0x55c2dbacd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.438 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.430+0000 7f3ca2200640 1 --2- 172.21.15.89:0/2083481849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbabc090 0x55c2dbac6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.438 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.430+0000 7f3ca0e00640 1 -- 172.21.15.89:0/2083481849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbabc090 msgr2=0x55c2dbac6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.438 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.430+0000 7f3ca0e00640 1 --2- 172.21.15.89:0/2083481849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbabc090 0x55c2dbac6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.430+0000 7f3ca0e00640 1 -- 172.21.15.89:0/2083481849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbac1a60 msgr2=0x55c2dbacd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.430+0000 7f3ca0e00640 1 --2- 172.21.15.89:0/2083481849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbac1a60 0x55c2dbacd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.430+0000 7f3ca0e00640 1 -- 172.21.15.89:0/2083481849 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2db800620 con 0x55c2dbabc9d0 2024-09-10T08:47:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.430+0000 7f3ca2200640 1 --2- 172.21.15.89:0/2083481849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbac1a60 0x55c2dbacd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.430+0000 7f3ca0e00640 1 --2- 172.21.15.89:0/2083481849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc9d0 0x55c2dbac6750 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3c88012720 tx=0x7f3c88012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8fe4c399a8ca95 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.430+0000 7f3c83400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:47:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.431+0000 7f3c83400640 1 -- 172.21.15.89:0/2083481849 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8800f920 con 0x55c2dbabc9d0 2024-09-10T08:47:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.431+0000 7f3c83400640 1 -- 172.21.15.89:0/2083481849 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f3c8800fac0 con 0x55c2dbabc9d0 2024-09-10T08:47:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.431+0000 7f3ca6d22740 1 -- 172.21.15.89:0/2083481849 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c940319f0 con 0x55c2dbabc9d0 2024-09-10T08:47:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.431+0000 7f3c83400640 1 -- 172.21.15.89:0/2083481849 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3c8800fcc0 con 0x55c2dbabc9d0 2024-09-10T08:47:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.431+0000 7f3ca6d22740 10 client.16779 fetch_fsmap learned FSMap version 31 2024-09-10T08:47:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.431+0000 7f3ca6d22740 1 -- 172.21.15.89:0/2083481849 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c2dbad0650 con 0x55c2dbabc9d0 2024-09-10T08:47:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.431+0000 7f3c83400640 1 -- 172.21.15.89:0/2083481849 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c940319f0 con 0x55c2dbabc9d0 2024-09-10T08:47:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.431+0000 7f3c83400640 1 -- 172.21.15.89:0/2083481849 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f3c8801c540 con 0x55c2dbabc9d0 2024-09-10T08:47:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.431+0000 7f3ca6d22740 10 client.16779 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:47:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.431+0000 7f3ca6d22740 10 client.16779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.432+0000 7f3ca6d22740 20 client.16779 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.432+0000 7f3ca6d22740 1 --2- 172.21.15.89:0/2083481849 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c2dbad1d90 0x55c2dbad41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.432+0000 7f3ca6d22740 4 client.16779 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.432+0000 7f3ca6d22740 1 -- 172.21.15.89:0/2083481849 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2dba2e830 con 0x55c2dbad1d90 2024-09-10T08:47:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.432+0000 7f3ca2200640 1 --2- 172.21.15.89:0/2083481849 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c2dbad1d90 0x55c2dbad41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.432+0000 7f3ca2200640 1 --2- 172.21.15.89:0/2083481849 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c2dbad1d90 0x55c2dbad41d0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef6703d8eac2ef6e server_cookie=4df126fa6ea8ca75 in_seq=0 out_seq=0 2024-09-10T08:47:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.432+0000 7f3c83400640 10 client.16779 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.465+0000 7f3c83400640 1 -- 172.21.15.89:0/2083481849 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2dba2e830 con 0x55c2dbad1d90 2024-09-10T08:47:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.465+0000 7f3c83400640 10 client.16779 handle_command_reply: tid=0 2024-09-10T08:47:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.465+0000 7f3ca6d22740 1 client.16779 shutdown 2024-09-10T08:47:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.465+0000 7f3ca6d22740 1 -- 172.21.15.89:0/2083481849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc9d0 msgr2=0x55c2dbac6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.465+0000 7f3ca6d22740 1 --2- 172.21.15.89:0/2083481849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc9d0 0x55c2dbac6750 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3c88012720 tx=0x7f3c88012be0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.465+0000 7f3ca6d22740 1 -- 172.21.15.89:0/2083481849 shutdown_connections 2024-09-10T08:47:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.465+0000 7f3ca6d22740 1 --2- 172.21.15.89:0/2083481849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbac1a60 0x55c2dbacd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.465+0000 7f3ca6d22740 1 --2- 172.21.15.89:0/2083481849 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c2dbad1d90 0x55c2dbad41d0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.465+0000 7f3ca6d22740 1 --2- 172.21.15.89:0/2083481849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc9d0 0x55c2dbac6750 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.465+0000 7f3ca6d22740 1 --2- 172.21.15.89:0/2083481849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbabc090 0x55c2dbac6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.465+0000 7f3c83400640 0 client.16779 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.466+0000 7f3ca6d22740 1 -- 172.21.15.89:0/2083481849 >> 172.21.15.89:0/2083481849 conn(0x55c2dbab7700 msgr2=0x55c2dbab8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.466+0000 7f3ca6d22740 1 -- 172.21.15.89:0/2083481849 shutdown_connections 2024-09-10T08:47:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.466+0000 7f3ca6d22740 1 -- 172.21.15.89:0/2083481849 wait complete. 2024-09-10T08:47:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.466+0000 7f3ca6d22740 20 client.16779 trim_cache size 0 max 16384 2024-09-10T08:47:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.481+0000 7f3ca6d22740 1 Processor -- start 2024-09-10T08:47:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.481+0000 7f3ca6d22740 1 -- start start 2024-09-10T08:47:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.481+0000 7f3ca6d22740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbab7700 0x55c2dbab8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.481+0000 7f3ca6d22740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbabc090 0x55c2dbacd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.481+0000 7f3ca6d22740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc9d0 0x55c2dbab9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.481+0000 7f3ca6d22740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1af260 con 0x55c2dbabc9d0 2024-09-10T08:47:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.481+0000 7f3ca6d22740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1acb00 con 0x55c2dbab7700 2024-09-10T08:47:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.481+0000 7f3ca6d22740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1aa950 con 0x55c2dbabc090 2024-09-10T08:47:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.482+0000 7f3ca2200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbab7700 0x55c2dbab8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.482+0000 7f3ca2200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbab7700 0x55c2dbab8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:54982/0 (socket says 172.21.15.89:54982) 2024-09-10T08:47:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.482+0000 7f3ca2200640 1 -- 172.21.15.89:0/574487605 learned_addr learned my addr 172.21.15.89:0/574487605 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.482+0000 7f3ca2200640 1 --2- 172.21.15.89:0/574487605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc9d0 0x55c2dbab9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.482+0000 7f3ca0e00640 1 --2- 172.21.15.89:0/574487605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbabc090 0x55c2dbacd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.482+0000 7f3ca2200640 1 -- 172.21.15.89:0/574487605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbabc090 msgr2=0x55c2dbacd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.482+0000 7f3ca2200640 1 --2- 172.21.15.89:0/574487605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbabc090 0x55c2dbacd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.482+0000 7f3ca2200640 1 -- 172.21.15.89:0/574487605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc9d0 msgr2=0x55c2dbab9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.482+0000 7f3ca2200640 1 --2- 172.21.15.89:0/574487605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc9d0 0x55c2dbab9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.482+0000 7f3ca2200640 1 -- 172.21.15.89:0/574487605 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2dbab7520 con 0x55c2dbab7700 2024-09-10T08:47:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.482+0000 7f3ca2200640 1 --2- 172.21.15.89:0/574487605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbab7700 0x55c2dbab8af0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3c94000c00 tx=0x7f3c9402a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba5fa567514c92c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.483+0000 7f3c8e000640 1 -- 172.21.15.89:0/574487605 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94035040 con 0x55c2dbab7700 2024-09-10T08:47:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.483+0000 7f3c8e000640 1 -- 172.21.15.89:0/574487605 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c94005500 con 0x55c2dbab7700 2024-09-10T08:47:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.483+0000 7f3c8e000640 1 -- 172.21.15.89:0/574487605 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94040040 con 0x55c2dbab7700 2024-09-10T08:47:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.483+0000 7f3ca6d22740 1 -- 172.21.15.89:0/574487605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbab7700 msgr2=0x55c2dbab8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.483+0000 7f3ca6d22740 1 --2- 172.21.15.89:0/574487605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbab7700 0x55c2dbab8af0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3c94000c00 tx=0x7f3c9402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.483+0000 7f3ca6d22740 1 -- 172.21.15.89:0/574487605 shutdown_connections 2024-09-10T08:47:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.483+0000 7f3ca6d22740 1 --2- 172.21.15.89:0/574487605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc9d0 0x55c2dbab9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.483+0000 7f3ca6d22740 1 --2- 172.21.15.89:0/574487605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbabc090 0x55c2dbacd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.484+0000 7f3ca6d22740 1 --2- 172.21.15.89:0/574487605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbab7700 0x55c2dbab8af0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.484+0000 7f3ca6d22740 1 -- 172.21.15.89:0/574487605 >> 172.21.15.89:0/574487605 conn(0x55c2dbac15b0 msgr2=0x55c2dbab6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.484+0000 7f3ca6d22740 1 -- 172.21.15.89:0/574487605 shutdown_connections 2024-09-10T08:47:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.484+0000 7f3ca6d22740 1 -- 172.21.15.89:0/574487605 wait complete. 2024-09-10T08:47:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.484+0000 7f3ca6d22740 1 -- start start 2024-09-10T08:47:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.484+0000 7f3ca6d22740 1 Processor -- start 2024-09-10T08:47:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.485+0000 7f3ca6d22740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbab7700 0x55c2dbacb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.485+0000 7f3ca6d22740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbabc090 0x55c2dbacb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.485+0000 7f3ca6d22740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc9d0 0x55c2dbacc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.485+0000 7f3ca6d22740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1aafc0 con 0x55c2dbabc9d0 2024-09-10T08:47:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.485+0000 7f3ca6d22740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1ac430 con 0x55c2dbab7700 2024-09-10T08:47:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.485+0000 7f3ca6d22740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1ab690 con 0x55c2dbabc090 2024-09-10T08:47:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.485+0000 7f3ca6d22740 10 client.0 _ll_register_callbacks cb 0x7f3c88009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.485+0000 7f3ca2200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbab7700 0x55c2dbacb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.485+0000 7f3ca0e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbabc090 0x55c2dbacb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.485+0000 7f3ca2200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc9d0 0x55c2dbacc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.485+0000 7f3ca0e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbabc090 0x55c2dbacb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:55034/0 (socket says 172.21.15.89:55034) 2024-09-10T08:47:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.485+0000 7f3ca0e00640 1 -- 172.21.15.89:0/2303455562 learned_addr learned my addr 172.21.15.89:0/2303455562 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.485+0000 7f3ca0e00640 1 -- 172.21.15.89:0/2303455562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbab7700 msgr2=0x55c2dbacb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.485+0000 7f3ca0e00640 1 --2- 172.21.15.89:0/2303455562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbab7700 0x55c2dbacb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.485+0000 7f3ca0e00640 1 -- 172.21.15.89:0/2303455562 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc9d0 msgr2=0x55c2dbacc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.485+0000 7f3ca0e00640 1 --2- 172.21.15.89:0/2303455562 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc9d0 0x55c2dbacc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.485+0000 7f3ca0e00640 1 -- 172.21.15.89:0/2303455562 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2dbaccf20 con 0x55c2dbabc090 2024-09-10T08:47:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.486+0000 7f3ca2200640 1 --2- 172.21.15.89:0/2303455562 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc9d0 0x55c2dbacc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.486+0000 7f3ca0e00640 1 --2- 172.21.15.89:0/2303455562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbabc090 0x55c2dbacb900 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3c88000c00 tx=0x7f3c8801de40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9de77df61a61fa45 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.486+0000 7f3c83400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:47:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.486+0000 7f3c83400640 1 -- 172.21.15.89:0/2303455562 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8802f060 con 0x55c2dbabc090 2024-09-10T08:47:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.486+0000 7f3ca6d22740 1 -- 172.21.15.89:0/2303455562 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c94005500 con 0x55c2dbabc090 2024-09-10T08:47:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.486+0000 7f3c83400640 1 -- 172.21.15.89:0/2303455562 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f3c8801c630 con 0x55c2dbabc090 2024-09-10T08:47:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.486+0000 7f3c83400640 1 -- 172.21.15.89:0/2303455562 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3c88036050 con 0x55c2dbabc090 2024-09-10T08:47:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.486+0000 7f3ca6d22740 10 client.26800 fetch_fsmap learned FSMap version 31 2024-09-10T08:47:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.486+0000 7f3ca6d22740 1 -- 172.21.15.89:0/2303455562 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c2dbb3d050 con 0x55c2dbabc090 2024-09-10T08:47:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.487+0000 7f3c83400640 1 -- 172.21.15.89:0/2303455562 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c94005500 con 0x55c2dbabc090 2024-09-10T08:47:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.487+0000 7f3c83400640 1 -- 172.21.15.89:0/2303455562 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f3c88003710 con 0x55c2dbabc090 2024-09-10T08:47:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.487+0000 7f3ca6d22740 10 client.26800 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:47:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.487+0000 7f3ca6d22740 10 client.26800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.487+0000 7f3ca6d22740 20 client.26800 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.487+0000 7f3ca6d22740 1 --2- 172.21.15.89:0/2303455562 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c2dbac1a60 0x55c2dbb3fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.487+0000 7f3ca6d22740 4 client.26800 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:47:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.487+0000 7f3ca6d22740 1 -- 172.21.15.89:0/2303455562 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2dba28b80 con 0x55c2dbac1a60 2024-09-10T08:47:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.487+0000 7f3ca2200640 1 --2- 172.21.15.89:0/2303455562 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c2dbac1a60 0x55c2dbb3fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.488+0000 7f3c83400640 10 client.26800 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:34.488+0000 7f3ca2200640 1 --2- 172.21.15.89:0/2303455562 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c2dbac1a60 0x55c2dbb3fa70 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=421cee446a4af611 server_cookie=e0ffe744fd54e1e0 in_seq=0 out_seq=0 2024-09-10T08:47:34.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.577+0000 7fba8fa00640 1 -- 172.21.15.89:0/116870966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba88106f10 msgr2=0x7fba8810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.577+0000 7fba8fa00640 1 --2- 172.21.15.89:0/116870966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba88106f10 0x7fba8810ea90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fba80009ca0 tx=0x7fba8002a8b0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.577+0000 7fba8fa00640 1 -- 172.21.15.89:0/116870966 shutdown_connections 2024-09-10T08:47:34.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.577+0000 7fba8fa00640 1 --2- 172.21.15.89:0/116870966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba88106f10 0x7fba8810ea90 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.577+0000 7fba8fa00640 1 --2- 172.21.15.89:0/116870966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba88102790 0x7fba881067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.577+0000 7fba8fa00640 1 --2- 172.21.15.89:0/116870966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba88101de0 0x7fba881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.577+0000 7fba8fa00640 1 -- 172.21.15.89:0/116870966 >> 172.21.15.89:0/116870966 conn(0x7fba880fd8c0 msgr2=0x7fba880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:34.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.577+0000 7fba8fa00640 1 -- 172.21.15.89:0/116870966 shutdown_connections 2024-09-10T08:47:34.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.578+0000 7fba8fa00640 1 -- 172.21.15.89:0/116870966 wait complete. 2024-09-10T08:47:34.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.578+0000 7fba8fa00640 1 Processor -- start 2024-09-10T08:47:34.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.579+0000 7fba8fa00640 1 -- start start 2024-09-10T08:47:34.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.579+0000 7fba8fa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba88101de0 0x7fba8819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.579+0000 7fba8fa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba88102790 0x7fba8819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.579+0000 7fba8fa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba88106f10 0x7fba8819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.579+0000 7fba8fa00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fba88111fc0 con 0x7fba88101de0 2024-09-10T08:47:34.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.579+0000 7fba8fa00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fba88111e40 con 0x7fba88102790 2024-09-10T08:47:34.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.579+0000 7fba8fa00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fba88112140 con 0x7fba88106f10 2024-09-10T08:47:34.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.579+0000 7fba8d000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba88101de0 0x7fba8819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.579+0000 7fba8d000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba88101de0 0x7fba8819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55992/0 (socket says 172.21.15.89:55992) 2024-09-10T08:47:34.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.579+0000 7fba8d000640 1 -- 172.21.15.89:0/256037768 learned_addr learned my addr 172.21.15.89:0/256037768 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:34.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.579+0000 7fba7fe00640 1 --2- 172.21.15.89:0/256037768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba88102790 0x7fba8819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.579+0000 7fba8da00640 1 --2- 172.21.15.89:0/256037768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba88106f10 0x7fba8819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.580+0000 7fba8d000640 1 -- 172.21.15.89:0/256037768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba88106f10 msgr2=0x7fba8819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.580+0000 7fba8d000640 1 --2- 172.21.15.89:0/256037768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba88106f10 0x7fba8819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.580+0000 7fba8d000640 1 -- 172.21.15.89:0/256037768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba88102790 msgr2=0x7fba8819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.580+0000 7fba8d000640 1 --2- 172.21.15.89:0/256037768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba88102790 0x7fba8819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.580+0000 7fba8d000640 1 -- 172.21.15.89:0/256037768 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8819f5c0 con 0x7fba88101de0 2024-09-10T08:47:34.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.580+0000 7fba7fe00640 1 --2- 172.21.15.89:0/256037768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba88102790 0x7fba8819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:34.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.580+0000 7fba8d000640 1 --2- 172.21.15.89:0/256037768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba88101de0 0x7fba8819a610 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fba7800ea60 tx=0x7fba7800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:34.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.580+0000 7fba86000640 1 -- 172.21.15.89:0/256037768 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7800cde0 con 0x7fba88101de0 2024-09-10T08:47:34.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.580+0000 7fba86000640 1 -- 172.21.15.89:0/256037768 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba780035f0 con 0x7fba88101de0 2024-09-10T08:47:34.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.580+0000 7fba8fa00640 1 -- 172.21.15.89:0/256037768 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba8819f8b0 con 0x7fba88101de0 2024-09-10T08:47:34.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.580+0000 7fba86000640 1 -- 172.21.15.89:0/256037768 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba78010670 con 0x7fba88101de0 2024-09-10T08:47:34.582 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.580+0000 7fba8fa00640 1 -- 172.21.15.89:0/256037768 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba881a7190 con 0x7fba88101de0 2024-09-10T08:47:34.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.585+0000 7fba92434740 1 Processor -- start 2024-09-10T08:47:34.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.585+0000 7fba92434740 1 -- start start 2024-09-10T08:47:34.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.585+0000 7fba86000640 1 -- 172.21.15.89:0/256037768 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fba780037a0 con 0x7fba88101de0 2024-09-10T08:47:34.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.585+0000 7fba92434740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd767c70 0x55bfdd768050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.585+0000 7fba92434740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdd768590 0x55bfdd772910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.585+0000 7fba92434740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd772e50 0x55bfdd775240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.585+0000 7fba92434740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bfdd6ce3e0 con 0x55bfdd772e50 2024-09-10T08:47:34.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.585+0000 7fba92434740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55bfdd675d80 con 0x55bfdd767c70 2024-09-10T08:47:34.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.585+0000 7fba92434740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bfdd6ca3a0 con 0x55bfdd768590 2024-09-10T08:47:34.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.586+0000 7fba8da00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd767c70 0x55bfdd768050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.586+0000 7fba8da00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd767c70 0x55bfdd768050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:55014/0 (socket says 172.21.15.89:55014) 2024-09-10T08:47:34.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.586+0000 7fba8da00640 1 -- 172.21.15.89:0/536332639 learned_addr learned my addr 172.21.15.89:0/536332639 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:34.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.586+0000 7fba8d000640 1 --2- 172.21.15.89:0/536332639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdd768590 0x55bfdd772910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.586+0000 7fba7fe00640 1 --2- 172.21.15.89:0/536332639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd772e50 0x55bfdd775240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.586+0000 7fba8da00640 1 -- 172.21.15.89:0/536332639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdd768590 msgr2=0x55bfdd772910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.586+0000 7fba8da00640 1 --2- 172.21.15.89:0/536332639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdd768590 0x55bfdd772910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.586+0000 7fba8da00640 1 -- 172.21.15.89:0/536332639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd772e50 msgr2=0x55bfdd775240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.586+0000 7fba8da00640 1 --2- 172.21.15.89:0/536332639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd772e50 0x55bfdd775240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.587 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.586+0000 7fba8da00640 1 -- 172.21.15.89:0/536332639 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfdd694ae0 con 0x55bfdd767c70 2024-09-10T08:47:34.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.586+0000 7fba8da00640 1 --2- 172.21.15.89:0/536332639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd767c70 0x55bfdd768050 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fba80009ca0 tx=0x7fba800093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b62b02c2aadb5d6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:34.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.587+0000 7fba7e000640 1 -- 172.21.15.89:0/536332639 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80030760 con 0x55bfdd767c70 2024-09-10T08:47:34.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.587+0000 7fba7e000640 1 -- 172.21.15.89:0/536332639 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba80036030 con 0x55bfdd767c70 2024-09-10T08:47:34.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.587+0000 7fba7e000640 1 -- 172.21.15.89:0/536332639 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba800043c0 con 0x55bfdd767c70 2024-09-10T08:47:34.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.587+0000 7fba86000640 1 --2- 172.21.15.89:0/256037768 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fba6c0a2c00 0x7fba6c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.587+0000 7fba86000640 1 -- 172.21.15.89:0/256037768 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fba780cf6e0 con 0x7fba88101de0 2024-09-10T08:47:34.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.587+0000 7fba8d000640 1 --2- 172.21.15.89:0/256037768 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fba6c0a2c00 0x7fba6c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.587+0000 7fba92434740 1 -- 172.21.15.89:0/536332639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd767c70 msgr2=0x55bfdd768050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.587+0000 7fba92434740 1 --2- 172.21.15.89:0/536332639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd767c70 0x55bfdd768050 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fba80009ca0 tx=0x7fba800093e0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.588+0000 7fba92434740 1 -- 172.21.15.89:0/536332639 shutdown_connections 2024-09-10T08:47:34.589 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.588+0000 7fba92434740 1 --2- 172.21.15.89:0/536332639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd772e50 0x55bfdd775240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.588+0000 7fba92434740 1 --2- 172.21.15.89:0/536332639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdd768590 0x55bfdd772910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.588+0000 7fba92434740 1 --2- 172.21.15.89:0/536332639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd767c70 0x55bfdd768050 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.588+0000 7fba92434740 1 -- 172.21.15.89:0/536332639 >> 172.21.15.89:0/536332639 conn(0x55bfdd764200 msgr2=0x55bfdd7645e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:34.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.588+0000 7fba92434740 1 -- 172.21.15.89:0/536332639 shutdown_connections 2024-09-10T08:47:34.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.588+0000 7fba8d000640 1 --2- 172.21.15.89:0/256037768 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fba6c0a2c00 0x7fba6c0a50c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fba780cd180 tx=0x7fba780d5550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:34.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.588+0000 7fba92434740 1 -- 172.21.15.89:0/536332639 wait complete. 2024-09-10T08:47:34.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.589+0000 7fba92434740 1 -- start start 2024-09-10T08:47:34.590 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.589+0000 7fba92434740 1 Processor -- start 2024-09-10T08:47:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.589+0000 7fba92434740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdd767c70 0x55bfdd76fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.589+0000 7fba92434740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd768590 0x55bfdd770260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.589+0000 7fba92434740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd772e50 0x55bfdd7707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.589+0000 7fba92434740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bfdd775a20 con 0x55bfdd768590 2024-09-10T08:47:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.589+0000 7fba92434740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55bfdd6890f0 con 0x55bfdd772e50 2024-09-10T08:47:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.589+0000 7fba92434740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bfdd775ba0 con 0x55bfdd767c70 2024-09-10T08:47:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.589+0000 7fba92434740 10 client.0 _ll_register_callbacks cb 0x55bfdd670bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.590+0000 7fba7fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd768590 0x55bfdd770260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.590+0000 7fba8da00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd772e50 0x55bfdd7707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.590+0000 7fba8da00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd772e50 0x55bfdd7707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:55020/0 (socket says 172.21.15.89:55020) 2024-09-10T08:47:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.590+0000 7fba7fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd768590 0x55bfdd770260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:56008/0 (socket says 172.21.15.89:56008) 2024-09-10T08:47:34.593 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.590+0000 7fba8da00640 1 -- 172.21.15.89:0/2590016021 learned_addr learned my addr 172.21.15.89:0/2590016021 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.590+0000 7fba8da00640 1 --2- 172.21.15.89:0/2590016021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdd767c70 0x55bfdd76fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.590+0000 7fba8da00640 1 -- 172.21.15.89:0/2590016021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdd767c70 msgr2=0x55bfdd76fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.590+0000 7fba8da00640 1 --2- 172.21.15.89:0/2590016021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdd767c70 0x55bfdd76fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.590+0000 7fba8da00640 1 -- 172.21.15.89:0/2590016021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd768590 msgr2=0x55bfdd770260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.590+0000 7fba8da00640 1 --2- 172.21.15.89:0/2590016021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd768590 0x55bfdd770260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.590+0000 7fba8da00640 1 -- 172.21.15.89:0/2590016021 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfdd43f3c0 con 0x55bfdd772e50 2024-09-10T08:47:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.590+0000 7fba7fe00640 1 --2- 172.21.15.89:0/2590016021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd768590 0x55bfdd770260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.590+0000 7fba8da00640 1 --2- 172.21.15.89:0/2590016021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd772e50 0x55bfdd7707a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fba80030ec0 tx=0x7fba80030f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b6841dba5d7afe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.590+0000 7fba76a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:47:34.594 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.591+0000 7fba76a00640 1 -- 172.21.15.89:0/2590016021 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80059060 con 0x55bfdd772e50 2024-09-10T08:47:34.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.591+0000 7fba92434740 1 -- 172.21.15.89:0/2590016021 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba80036030 con 0x55bfdd772e50 2024-09-10T08:47:34.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.591+0000 7fba76a00640 1 -- 172.21.15.89:0/2590016021 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fba80004140 con 0x55bfdd772e50 2024-09-10T08:47:34.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.591+0000 7fba76a00640 1 -- 172.21.15.89:0/2590016021 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fba80004b40 con 0x55bfdd772e50 2024-09-10T08:47:34.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.591+0000 7fba92434740 10 client.26867 fetch_fsmap learned FSMap version 31 2024-09-10T08:47:34.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.591+0000 7fba92434740 1 -- 172.21.15.89:0/2590016021 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bfdd775a20 con 0x55bfdd772e50 2024-09-10T08:47:34.595 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.591+0000 7fba76a00640 1 -- 172.21.15.89:0/2590016021 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba80036030 con 0x55bfdd772e50 2024-09-10T08:47:34.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.591+0000 7fba76a00640 1 -- 172.21.15.89:0/2590016021 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fba80043070 con 0x55bfdd772e50 2024-09-10T08:47:34.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.591+0000 7fba92434740 10 client.26867 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:47:34.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.591+0000 7fba92434740 10 client.26867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:34.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.591+0000 7fba92434740 20 client.26867 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:34.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.591+0000 7fba92434740 1 --2- 172.21.15.89:0/2590016021 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bfdd782bc0 0x55bfdd784fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.591+0000 7fba92434740 4 client.26867 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:34.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.591+0000 7fba92434740 1 -- 172.21.15.89:0/2590016021 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfdd6db950 con 0x55bfdd782bc0 2024-09-10T08:47:34.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.592+0000 7fba7fe00640 1 --2- 172.21.15.89:0/2590016021 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bfdd782bc0 0x55bfdd784fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.592+0000 7fba7fe00640 1 --2- 172.21.15.89:0/2590016021 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bfdd782bc0 0x55bfdd784fb0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8a3dbd918e051e6 server_cookie=63419bf143c514b8 in_seq=0 out_seq=0 2024-09-10T08:47:34.597 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.592+0000 7fba76a00640 10 client.26867 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:34.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.606+0000 7fba76a00640 1 -- 172.21.15.89:0/2590016021 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bfdd6db950 con 0x55bfdd782bc0 2024-09-10T08:47:34.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.606+0000 7fba76a00640 10 client.26867 handle_command_reply: tid=0 2024-09-10T08:47:34.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.607+0000 7fba92434740 1 client.26867 shutdown 2024-09-10T08:47:34.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.607+0000 7fba92434740 1 -- 172.21.15.89:0/2590016021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd772e50 msgr2=0x55bfdd7707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.607+0000 7fba92434740 1 --2- 172.21.15.89:0/2590016021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd772e50 0x55bfdd7707a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fba80030ec0 tx=0x7fba80030f20 comp rx=0 tx=0).stop 2024-09-10T08:47:34.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.607+0000 7fba92434740 1 -- 172.21.15.89:0/2590016021 shutdown_connections 2024-09-10T08:47:34.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.607+0000 7fba92434740 1 --2- 172.21.15.89:0/2590016021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd772e50 0x55bfdd7707a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.607+0000 7fba92434740 1 --2- 172.21.15.89:0/2590016021 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bfdd782bc0 0x55bfdd784fb0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.607+0000 7fba92434740 1 --2- 172.21.15.89:0/2590016021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd768590 0x55bfdd770260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.607+0000 7fba92434740 1 --2- 172.21.15.89:0/2590016021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdd767c70 0x55bfdd76fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.607+0000 7fba76a00640 0 client.26867 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:34.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.607+0000 7fba92434740 1 -- 172.21.15.89:0/2590016021 >> 172.21.15.89:0/2590016021 conn(0x55bfdd764200 msgr2=0x55bfdd763380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:34.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.607+0000 7fba92434740 1 -- 172.21.15.89:0/2590016021 shutdown_connections 2024-09-10T08:47:34.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.607+0000 7fba92434740 1 -- 172.21.15.89:0/2590016021 wait complete. 2024-09-10T08:47:34.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.607+0000 7fba92434740 20 client.26867 trim_cache size 0 max 16384 2024-09-10T08:47:34.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.621+0000 7fba92434740 1 Processor -- start 2024-09-10T08:47:34.621 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.621+0000 7fba92434740 1 -- start start 2024-09-10T08:47:34.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.621+0000 7fba92434740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd767c70 0x55bfdd7705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.621+0000 7fba92434740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd768590 0x55bfdd78a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.621+0000 7fba92434740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdd772e50 0x55bfdd78ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.621+0000 7fba92434740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fba881b3830 con 0x55bfdd767c70 2024-09-10T08:47:34.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.621+0000 7fba92434740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fba881b10d0 con 0x55bfdd768590 2024-09-10T08:47:34.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.621+0000 7fba92434740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fba881aef20 con 0x55bfdd772e50 2024-09-10T08:47:34.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.622+0000 7fba7fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd768590 0x55bfdd78a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.622+0000 7fba7fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd768590 0x55bfdd78a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:55030/0 (socket says 172.21.15.89:55030) 2024-09-10T08:47:34.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.622+0000 7fba7fe00640 1 -- 172.21.15.89:0/3659147033 learned_addr learned my addr 172.21.15.89:0/3659147033 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:34.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.622+0000 7fba8da00640 1 --2- 172.21.15.89:0/3659147033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd767c70 0x55bfdd7705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.622+0000 7fba7fe00640 1 -- 172.21.15.89:0/3659147033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdd772e50 msgr2=0x55bfdd78ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.622+0000 7fba7fe00640 1 --2- 172.21.15.89:0/3659147033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdd772e50 0x55bfdd78ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.622+0000 7fba7fe00640 1 -- 172.21.15.89:0/3659147033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd767c70 msgr2=0x55bfdd7705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.622+0000 7fba7fe00640 1 --2- 172.21.15.89:0/3659147033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd767c70 0x55bfdd7705f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.622+0000 7fba7fe00640 1 -- 172.21.15.89:0/3659147033 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfdd7683c0 con 0x55bfdd768590 2024-09-10T08:47:34.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.622+0000 7fba7fe00640 1 --2- 172.21.15.89:0/3659147033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd768590 0x55bfdd78a6f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55bfdd765580 tx=0x7fba70005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c158d6a75988d816 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:34.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.623+0000 7fba7d600640 1 -- 172.21.15.89:0/3659147033 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba700120d0 con 0x55bfdd768590 2024-09-10T08:47:34.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.623+0000 7fba7d600640 1 -- 172.21.15.89:0/3659147033 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba70012270 con 0x55bfdd768590 2024-09-10T08:47:34.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.623+0000 7fba7d600640 1 -- 172.21.15.89:0/3659147033 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba70004e00 con 0x55bfdd768590 2024-09-10T08:47:34.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.623+0000 7fba92434740 1 -- 172.21.15.89:0/3659147033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd768590 msgr2=0x55bfdd78a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.623+0000 7fba92434740 1 --2- 172.21.15.89:0/3659147033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd768590 0x55bfdd78a6f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55bfdd765580 tx=0x7fba70005f90 comp rx=0 tx=0).stop 2024-09-10T08:47:34.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.623+0000 7fba92434740 1 -- 172.21.15.89:0/3659147033 shutdown_connections 2024-09-10T08:47:34.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.623+0000 7fba92434740 1 --2- 172.21.15.89:0/3659147033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdd772e50 0x55bfdd78ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.623+0000 7fba92434740 1 --2- 172.21.15.89:0/3659147033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd768590 0x55bfdd78a6f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.623+0000 7fba92434740 1 --2- 172.21.15.89:0/3659147033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd767c70 0x55bfdd7705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.623+0000 7fba92434740 1 -- 172.21.15.89:0/3659147033 >> 172.21.15.89:0/3659147033 conn(0x55bfdd764200 msgr2=0x55bfdd771da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:34.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.623+0000 7fba92434740 1 -- 172.21.15.89:0/3659147033 shutdown_connections 2024-09-10T08:47:34.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.624+0000 7fba92434740 1 -- 172.21.15.89:0/3659147033 wait complete. 2024-09-10T08:47:34.628 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.624+0000 7fba92434740 1 -- start start 2024-09-10T08:47:34.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.624+0000 7fba92434740 1 Processor -- start 2024-09-10T08:47:34.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.625+0000 7fba92434740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd767c70 0x55bfdd77b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.625+0000 7fba92434740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdd768590 0x55bfdd7768d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.625+0000 7fba92434740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd772e50 0x55bfdd776e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.625+0000 7fba92434740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fba881af590 con 0x55bfdd772e50 2024-09-10T08:47:34.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.625+0000 7fba92434740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fba881b0a00 con 0x55bfdd767c70 2024-09-10T08:47:34.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.625+0000 7fba92434740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fba881afc60 con 0x55bfdd768590 2024-09-10T08:47:34.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.625+0000 7fba92434740 10 client.0 _ll_register_callbacks cb 0x7fba70009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:34.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.625+0000 7fba8da00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd767c70 0x55bfdd77b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.629 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.625+0000 7fba8da00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd767c70 0x55bfdd77b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:55046/0 (socket says 172.21.15.89:55046) 2024-09-10T08:47:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.625+0000 7fba8da00640 1 -- 172.21.15.89:0/2685407628 learned_addr learned my addr 172.21.15.89:0/2685407628 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.625+0000 7fba8da00640 1 --2- 172.21.15.89:0/2685407628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd772e50 0x55bfdd776e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.625+0000 7fba7fe00640 1 --2- 172.21.15.89:0/2685407628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdd768590 0x55bfdd7768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.625+0000 7fba8da00640 1 -- 172.21.15.89:0/2685407628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdd768590 msgr2=0x55bfdd7768d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.625+0000 7fba8da00640 1 --2- 172.21.15.89:0/2685407628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdd768590 0x55bfdd7768d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.625+0000 7fba8da00640 1 -- 172.21.15.89:0/2685407628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd772e50 msgr2=0x55bfdd776e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.625+0000 7fba8da00640 1 --2- 172.21.15.89:0/2685407628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd772e50 0x55bfdd776e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.625+0000 7fba8da00640 1 -- 172.21.15.89:0/2685407628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfdd7775f0 con 0x55bfdd767c70 2024-09-10T08:47:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.626+0000 7fba8da00640 1 --2- 172.21.15.89:0/2685407628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd767c70 0x55bfdd77b550 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fba80000f80 tx=0x7fba8002a8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6282bb09df819a89 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.626+0000 7fba76a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:47:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.626+0000 7fba76a00640 1 -- 172.21.15.89:0/2685407628 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80059030 con 0x55bfdd767c70 2024-09-10T08:47:34.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.626+0000 7fba92434740 1 -- 172.21.15.89:0/2685407628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba70012270 con 0x55bfdd767c70 2024-09-10T08:47:34.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.626+0000 7fba76a00640 1 -- 172.21.15.89:0/2685407628 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fba80004ce0 con 0x55bfdd767c70 2024-09-10T08:47:34.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.626+0000 7fba76a00640 1 -- 172.21.15.89:0/2685407628 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fba80004ee0 con 0x55bfdd767c70 2024-09-10T08:47:34.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.626+0000 7fba92434740 10 client.26873 fetch_fsmap learned FSMap version 31 2024-09-10T08:47:34.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.626+0000 7fba92434740 1 -- 172.21.15.89:0/2685407628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bfdd7ed6d0 con 0x55bfdd767c70 2024-09-10T08:47:34.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.626+0000 7fba76a00640 1 -- 172.21.15.89:0/2685407628 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba70012270 con 0x55bfdd767c70 2024-09-10T08:47:34.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.627+0000 7fba76a00640 1 -- 172.21.15.89:0/2685407628 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fba80043040 con 0x55bfdd767c70 2024-09-10T08:47:34.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.627+0000 7fba92434740 10 client.26873 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:47:34.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.627+0000 7fba92434740 10 client.26873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:34.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.627+0000 7fba92434740 20 client.26873 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:34.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.627+0000 7fba92434740 1 --2- 172.21.15.89:0/2685407628 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bfdd7ede20 0x55bfdd7f0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.627+0000 7fba92434740 4 client.26873 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:47:34.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.627+0000 7fba92434740 1 -- 172.21.15.89:0/2685407628 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bfdd6d5e60 con 0x55bfdd7ede20 2024-09-10T08:47:34.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.627+0000 7fba7fe00640 1 --2- 172.21.15.89:0/2685407628 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bfdd7ede20 0x55bfdd7f0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.628+0000 7fba7fe00640 1 --2- 172.21.15.89:0/2685407628 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bfdd7ede20 0x55bfdd7f0240 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b693f7f3feaa5100 server_cookie=cd74ee2db9b184e3 in_seq=0 out_seq=0 2024-09-10T08:47:34.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.628+0000 7fba76a00640 10 client.26873 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:34.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.682+0000 7fba76a00640 1 -- 172.21.15.89:0/2685407628 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bfdd6d5e60 con 0x55bfdd7ede20 2024-09-10T08:47:34.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.683+0000 7fba76a00640 10 client.26873 handle_command_reply: tid=0 2024-09-10T08:47:34.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.683+0000 7fba92434740 1 client.26873 shutdown 2024-09-10T08:47:34.683 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.683+0000 7fba92434740 1 -- 172.21.15.89:0/2685407628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd767c70 msgr2=0x55bfdd77b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.683+0000 7fba92434740 1 --2- 172.21.15.89:0/2685407628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd767c70 0x55bfdd77b550 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fba80000f80 tx=0x7fba8002a8b0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.683+0000 7fba92434740 1 -- 172.21.15.89:0/2685407628 shutdown_connections 2024-09-10T08:47:34.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.683+0000 7fba92434740 1 --2- 172.21.15.89:0/2685407628 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55bfdd7ede20 0x55bfdd7f0240 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.683+0000 7fba92434740 1 --2- 172.21.15.89:0/2685407628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bfdd772e50 0x55bfdd776e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.683+0000 7fba92434740 1 --2- 172.21.15.89:0/2685407628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdd768590 0x55bfdd7768d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.683+0000 7fba76a00640 0 client.26873 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:34.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.683+0000 7fba92434740 1 --2- 172.21.15.89:0/2685407628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bfdd767c70 0x55bfdd77b550 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.683+0000 7fba92434740 1 -- 172.21.15.89:0/2685407628 >> 172.21.15.89:0/2685407628 conn(0x55bfdd764200 msgr2=0x55bfdd7696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:34.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.683+0000 7fba92434740 1 -- 172.21.15.89:0/2685407628 shutdown_connections 2024-09-10T08:47:34.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.683+0000 7fba92434740 1 -- 172.21.15.89:0/2685407628 wait complete. 2024-09-10T08:47:34.684 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.683+0000 7fba92434740 20 client.26873 trim_cache size 0 max 16384 2024-09-10T08:47:34.685 INFO:teuthology.orchestra.run.smithi089.stdout:[] 2024-09-10T08:47:34.685 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.684+0000 7fba8fa00640 1 -- 172.21.15.89:0/256037768 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fba6c0a2c00 msgr2=0x7fba6c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.684+0000 7fba8fa00640 1 --2- 172.21.15.89:0/256037768 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fba6c0a2c00 0x7fba6c0a50c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fba780cd180 tx=0x7fba780d5550 comp rx=0 tx=0).stop 2024-09-10T08:47:34.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.684+0000 7fba8fa00640 1 -- 172.21.15.89:0/256037768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba88101de0 msgr2=0x7fba8819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.684+0000 7fba8fa00640 1 --2- 172.21.15.89:0/256037768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba88101de0 0x7fba8819a610 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fba7800ea60 tx=0x7fba7800ef20 comp rx=0 tx=0).stop 2024-09-10T08:47:34.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.684+0000 7fba8fa00640 1 -- 172.21.15.89:0/256037768 shutdown_connections 2024-09-10T08:47:34.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.684+0000 7fba8fa00640 1 --2- 172.21.15.89:0/256037768 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fba6c0a2c00 0x7fba6c0a50c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.684+0000 7fba8fa00640 1 --2- 172.21.15.89:0/256037768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba88106f10 0x7fba8819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.684+0000 7fba8fa00640 1 --2- 172.21.15.89:0/256037768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba88102790 0x7fba8819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.684+0000 7fba8fa00640 1 --2- 172.21.15.89:0/256037768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba88101de0 0x7fba8819a610 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.684+0000 7fba8fa00640 1 -- 172.21.15.89:0/256037768 >> 172.21.15.89:0/256037768 conn(0x7fba880fd8c0 msgr2=0x7fba880ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:34.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.685+0000 7fba8fa00640 1 -- 172.21.15.89:0/256037768 shutdown_connections 2024-09-10T08:47:34.686 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:34.685+0000 7fba8fa00640 1 -- 172.21.15.89:0/256037768 wait complete. 2024-09-10T08:47:35.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:34 smithi149 ceph-mon[36167]: pgmap v622: 129 pgs: 129 active+clean; 426 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.0 MiB/s wr, 452 op/s 2024-09-10T08:47:35.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:34 smithi111 ceph-mon[35220]: pgmap v622: 129 pgs: 129 active+clean; 426 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.0 MiB/s wr, 452 op/s 2024-09-10T08:47:35.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:34 smithi089 ceph-mon[30757]: pgmap v622: 129 pgs: 129 active+clean; 426 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.0 MiB/s wr, 452 op/s 2024-09-10T08:47:35.758 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 715696985, 'ino': 1099511652059, 'path': '/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-4.xml'}, {'damage_type': 'backtrace', 'id': 822296160, 'ino': 1099511652084, 'path': '/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-5.jpg'}, {'damage_type': 'backtrace', 'id': 1114910407, 'ino': 1099511652067, 'path': '/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-2.xml'}, {'damage_type': 'backtrace', 'id': 1529065628, 'ino': 1099511652068, 'path': '/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-1.xml'}, {'damage_type': 'backtrace', 'id': 1552237096, 'ino': 1099511652077, 'path': '/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-7.jpg'}, {'damage_type': 'backtrace', 'id': 1681867536, 'ino': 1099511652088, 'path': '/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-4.jpg'}, {'damage_type': 'backtrace', 'id': 2612536088, 'ino': 1099511652055, 'path': '/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-5.xml'}, {'damage_type': 'backtrace', 'id': 2936482752, 'ino': 1099511652075, 'path': '/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-8.jpg'}, {'damage_type': 'backtrace', 'id': 3034715200, 'ino': 1099511652080, 'path': '/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-6.jpg'}, {'damage_type': 'backtrace', 'id': 3183236255, 'ino': 1099511652071, 'path': '/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/picture-9.jpg'}, {'damage_type': 'backtrace', 'id': 3249215083, 'ino': 1099511652064, 'path': '/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-102/article-3.xml'}] 2024-09-10T08:47:35.758 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/fwd_scrub.py", line 48, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/fwd_scrub.py", line 70, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/fwd_scrub.py", line 81, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-09-10T08:47:36.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:35 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:47:36.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:35 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:47:36.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:35 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:47:36.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.397+0000 7f19d7400640 1 -- 172.21.15.89:0/2867952381 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5612e952cc60 con 0x5612e963fd70 2024-09-10T08:47:36.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.397+0000 7f19d7400640 10 client.26764 handle_command_reply: tid=0 2024-09-10T08:47:36.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.397+0000 7f19fb64c740 1 client.26764 shutdown 2024-09-10T08:47:36.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.397+0000 7f19fb64c740 1 -- 172.21.15.89:0/2867952381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b9cd0 msgr2=0x5612e95d9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:36.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.397+0000 7f19fb64c740 1 --2- 172.21.15.89:0/2867952381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b9cd0 0x5612e95d9890 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5612e95c28d0 tx=0x7f19dc02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.397+0000 7f19fb64c740 1 -- 172.21.15.89:0/2867952381 shutdown_connections 2024-09-10T08:47:36.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.397+0000 7f19fb64c740 1 --2- 172.21.15.89:0/2867952381 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5612e963fd70 0x5612e9642160 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.397+0000 7f19fb64c740 1 --2- 172.21.15.89:0/2867952381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5612e95c4590 0x5612e95d77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.397+0000 7f19fb64c740 1 --2- 172.21.15.89:0/2867952381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612e95b9cd0 0x5612e95d9890 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.397+0000 7f19fb64c740 1 --2- 172.21.15.89:0/2867952381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612e95b93b0 0x5612e95d9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.397+0000 7f19d7400640 0 client.26764 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:36.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.397+0000 7f19fb64c740 1 -- 172.21.15.89:0/2867952381 >> 172.21.15.89:0/2867952381 conn(0x5612e95b5940 msgr2=0x5612e95cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:36.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.397+0000 7f19fb64c740 1 -- 172.21.15.89:0/2867952381 shutdown_connections 2024-09-10T08:47:36.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.398+0000 7f19fb64c740 1 -- 172.21.15.89:0/2867952381 wait complete. 2024-09-10T08:47:36.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.398+0000 7f19fb64c740 20 client.26764 trim_cache size 0 max 16384 2024-09-10T08:47:36.399 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:47:36.399 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 31, 2024-09-10T08:47:36.399 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24554, 2024-09-10T08:47:36.400 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 22, 2024-09-10T08:47:36.400 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": { 2024-09-10T08:47:36.400 INFO:teuthology.orchestra.run.smithi089.stdout: "a83c6b0f": { 2024-09-10T08:47:36.400 INFO:teuthology.orchestra.run.smithi089.stdout: "version": 22, 2024-09-10T08:47:36.400 INFO:teuthology.orchestra.run.smithi089.stdout: "age_ref": 0.0, 2024-09-10T08:47:36.400 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:47:36.400 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCED", 2024-09-10T08:47:36.400 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:47:36.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:47:36.400 INFO:teuthology.orchestra.run.smithi089.stdout: "timeout": 90.0, 2024-09-10T08:47:36.400 INFO:teuthology.orchestra.run.smithi089.stdout: "expiration": 140.1, 2024-09-10T08:47:36.400 INFO:teuthology.orchestra.run.smithi089.stdout: "members": { 2024-09-10T08:47:36.400 INFO:teuthology.orchestra.run.smithi089.stdout: "file:/": { 2024-09-10T08:47:36.400 INFO:teuthology.orchestra.run.smithi089.stdout: "excluded": false, 2024-09-10T08:47:36.401 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:47:36.401 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "QUIESCED", 2024-09-10T08:47:36.401 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:47:36.401 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:36.401 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:36.401 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:36.401 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:36.401 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:36.401 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:47:36.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.398+0000 7f19f8a00640 1 -- 172.21.15.89:0/1354845597 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f19d80a2b80 msgr2=0x7f19d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:36.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.398+0000 7f19f8a00640 1 --2- 172.21.15.89:0/1354845597 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f19d80a2b80 0x7f19d80a5040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x5612e95b67a0 tx=0x7f19e4005f90 comp rx=0 tx=0).stop 2024-09-10T08:47:36.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.398+0000 7f19f8a00640 1 -- 172.21.15.89:0/1354845597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19f4077040 msgr2=0x7f19f41a4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:36.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.398+0000 7f19f8a00640 1 --2- 172.21.15.89:0/1354845597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19f4077040 0x7f19f41a4c70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f19dc004770 tx=0x7f19dc003c10 comp rx=0 tx=0).stop 2024-09-10T08:47:36.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.398+0000 7f19f8a00640 1 -- 172.21.15.89:0/1354845597 shutdown_connections 2024-09-10T08:47:36.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.398+0000 7f19f8a00640 1 --2- 172.21.15.89:0/1354845597 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f19d80a2b80 0x7f19d80a5040 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.398+0000 7f19f8a00640 1 --2- 172.21.15.89:0/1354845597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19f4112a30 0x7f19f419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.399+0000 7f19f8a00640 1 --2- 172.21.15.89:0/1354845597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19f4077040 0x7f19f41a4c70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.399+0000 7f19f8a00640 1 --2- 172.21.15.89:0/1354845597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f19f4076720 0x7f19f41a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.399+0000 7f19f8a00640 1 -- 172.21.15.89:0/1354845597 >> 172.21.15.89:0/1354845597 conn(0x7f19f40ffa90 msgr2=0x7f19f4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:36.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.399+0000 7f19f8a00640 1 -- 172.21.15.89:0/1354845597 shutdown_connections 2024-09-10T08:47:36.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:36.399+0000 7f19f8a00640 1 -- 172.21.15.89:0/1354845597 wait complete. 2024-09-10T08:47:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:36 smithi089 ceph-mon[30757]: pgmap v623: 129 pgs: 129 active+clean; 425 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 936 KiB/s wr, 420 op/s 2024-09-10T08:47:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:47:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:47:37.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:47:37.171 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a83c6b0f', quiesce took 11.0 seconds. Will release after: 9.100000000000001 2024-09-10T08:47:37.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:36 smithi149 ceph-mon[36167]: pgmap v623: 129 pgs: 129 active+clean; 425 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 936 KiB/s wr, 420 op/s 2024-09-10T08:47:37.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:47:37.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:47:37.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:47:37.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:36 smithi111 ceph-mon[35220]: pgmap v623: 129 pgs: 129 active+clean; 425 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 936 KiB/s wr, 420 op/s 2024-09-10T08:47:37.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:47:37.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:47:37.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:47:39.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:38 smithi149 ceph-mon[36167]: pgmap v624: 129 pgs: 129 active+clean; 420 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.1 MiB/s wr, 471 op/s 2024-09-10T08:47:39.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:38 smithi111 ceph-mon[35220]: pgmap v624: 129 pgs: 129 active+clean; 420 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.1 MiB/s wr, 471 op/s 2024-09-10T08:47:39.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:38 smithi089 ceph-mon[30757]: pgmap v624: 129 pgs: 129 active+clean; 420 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.1 MiB/s wr, 471 op/s 2024-09-10T08:47:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.273+0000 7f3c83400640 1 -- 172.21.15.89:0/2303455562 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c2dba28b80 con 0x55c2dbac1a60 2024-09-10T08:47:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.273+0000 7f3c83400640 10 client.26800 handle_command_reply: tid=0 2024-09-10T08:47:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.273+0000 7f3ca6d22740 1 client.26800 shutdown 2024-09-10T08:47:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.273+0000 7f3ca6d22740 1 -- 172.21.15.89:0/2303455562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbabc090 msgr2=0x55c2dbacb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.273+0000 7f3ca6d22740 1 --2- 172.21.15.89:0/2303455562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbabc090 0x55c2dbacb900 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3c88000c00 tx=0x7f3c8801de40 comp rx=0 tx=0).stop 2024-09-10T08:47:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.273+0000 7f3ca6d22740 1 -- 172.21.15.89:0/2303455562 shutdown_connections 2024-09-10T08:47:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.273+0000 7f3ca6d22740 1 --2- 172.21.15.89:0/2303455562 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c2dbac1a60 0x55c2dbb3fa70 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.273+0000 7f3ca6d22740 1 --2- 172.21.15.89:0/2303455562 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2dbabc9d0 0x55c2dbacc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.273+0000 7f3ca6d22740 1 --2- 172.21.15.89:0/2303455562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2dbabc090 0x55c2dbacb900 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.273+0000 7f3c83400640 0 client.26800 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.273+0000 7f3ca6d22740 1 --2- 172.21.15.89:0/2303455562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c2dbab7700 0x55c2dbacb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.273+0000 7f3ca6d22740 1 -- 172.21.15.89:0/2303455562 >> 172.21.15.89:0/2303455562 conn(0x55c2dbac15b0 msgr2=0x55c2dbab7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.273+0000 7f3ca6d22740 1 -- 172.21.15.89:0/2303455562 shutdown_connections 2024-09-10T08:47:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.273+0000 7f3ca6d22740 1 -- 172.21.15.89:0/2303455562 wait complete. 2024-09-10T08:47:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.274+0000 7f3ca6d22740 20 client.26800 trim_cache size 0 max 16384 2024-09-10T08:47:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:47:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:47:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:47:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:47:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.274+0000 7f3ca4200640 1 -- 172.21.15.89:0/1796455219 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3c840a2cb0 msgr2=0x7f3c840a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.274+0000 7f3ca4200640 1 --2- 172.21.15.89:0/1796455219 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3c840a2cb0 0x7f3c840a5170 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3c900cd640 tx=0x7f3c900d3680 comp rx=0 tx=0).stop 2024-09-10T08:47:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.275+0000 7f3ca4200640 1 -- 172.21.15.89:0/1796455219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c9c103d60 msgr2=0x7f3c9c199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.275+0000 7f3ca4200640 1 --2- 172.21.15.89:0/1796455219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c9c103d60 0x7f3c9c199f50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000ebc0 tx=0x7f3c9000c6a0 comp rx=0 tx=0).stop 2024-09-10T08:47:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.275+0000 7f3ca4200640 1 -- 172.21.15.89:0/1796455219 shutdown_connections 2024-09-10T08:47:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.275+0000 7f3ca4200640 1 --2- 172.21.15.89:0/1796455219 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f3c840a2cb0 0x7f3c840a5170 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.275+0000 7f3ca4200640 1 --2- 172.21.15.89:0/1796455219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c9c105910 0x7f3c9c1a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.275+0000 7f3ca4200640 1 --2- 172.21.15.89:0/1796455219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c9c104f50 0x7f3c9c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.275+0000 7f3ca4200640 1 --2- 172.21.15.89:0/1796455219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c9c103d60 0x7f3c9c199f50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.275+0000 7f3ca4200640 1 -- 172.21.15.89:0/1796455219 >> 172.21.15.89:0/1796455219 conn(0x7f3c9c0ff4e0 msgr2=0x7f3c9c108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.275+0000 7f3ca4200640 1 -- 172.21.15.89:0/1796455219 shutdown_connections 2024-09-10T08:47:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:40.275+0000 7f3ca4200640 1 -- 172.21.15.89:0/1796455219 wait complete. 2024-09-10T08:47:41.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:40 smithi149 ceph-mon[36167]: pgmap v625: 129 pgs: 129 active+clean; 420 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 727 KiB/s wr, 362 op/s 2024-09-10T08:47:41.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:40 smithi111 ceph-mon[35220]: pgmap v625: 129 pgs: 129 active+clean; 420 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 727 KiB/s wr, 362 op/s 2024-09-10T08:47:41.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:40 smithi089 ceph-mon[30757]: pgmap v625: 129 pgs: 129 active+clean; 420 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 727 KiB/s wr, 362 op/s 2024-09-10T08:47:43.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:42 smithi149 ceph-mon[36167]: pgmap v626: 129 pgs: 129 active+clean; 414 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 699 KiB/s wr, 358 op/s 2024-09-10T08:47:43.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:42 smithi111 ceph-mon[35220]: pgmap v626: 129 pgs: 129 active+clean; 414 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 699 KiB/s wr, 358 op/s 2024-09-10T08:47:43.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:42 smithi089 ceph-mon[30757]: pgmap v626: 129 pgs: 129 active+clean; 414 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 699 KiB/s wr, 358 op/s 2024-09-10T08:47:45.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:44 smithi149 ceph-mon[36167]: pgmap v627: 129 pgs: 129 active+clean; 412 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 277 KiB/s wr, 220 op/s 2024-09-10T08:47:45.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:44 smithi111 ceph-mon[35220]: pgmap v627: 129 pgs: 129 active+clean; 412 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 277 KiB/s wr, 220 op/s 2024-09-10T08:47:45.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:44 smithi089 ceph-mon[30757]: pgmap v627: 129 pgs: 129 active+clean; 412 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 277 KiB/s wr, 220 op/s 2024-09-10T08:47:46.273 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:47:46.653 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:46 smithi089 ceph-mon[30757]: pgmap v628: 129 pgs: 129 active+clean; 408 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 259 KiB/s wr, 129 op/s 2024-09-10T08:47:47.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.212+0000 7f96e2200640 1 -- 172.21.15.89:0/1963113823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96dc103ff0 msgr2=0x7f96dc1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:47.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.212+0000 7f96e2200640 1 --2- 172.21.15.89:0/1963113823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96dc103ff0 0x7f96dc1043d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009900 tx=0x7f96cc02a410 comp rx=0 tx=0).stop 2024-09-10T08:47:47.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.212+0000 7f96e2200640 1 -- 172.21.15.89:0/1963113823 shutdown_connections 2024-09-10T08:47:47.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.212+0000 7f96e2200640 1 --2- 172.21.15.89:0/1963113823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96dc109120 0x7f96dc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:47.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.212+0000 7f96e2200640 1 --2- 172.21.15.89:0/1963113823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96dc1049a0 0x7f96dc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:47.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.212+0000 7f96e2200640 1 --2- 172.21.15.89:0/1963113823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96dc103ff0 0x7f96dc1043d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:47.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.212+0000 7f96e2200640 1 -- 172.21.15.89:0/1963113823 >> 172.21.15.89:0/1963113823 conn(0x7f96dc0ffa90 msgr2=0x7f96dc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:47.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.212+0000 7f96e2200640 1 -- 172.21.15.89:0/1963113823 shutdown_connections 2024-09-10T08:47:47.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.212+0000 7f96e2200640 1 -- 172.21.15.89:0/1963113823 wait complete. 2024-09-10T08:47:47.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.213+0000 7f96e2200640 1 Processor -- start 2024-09-10T08:47:47.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.213+0000 7f96e2200640 1 -- start start 2024-09-10T08:47:47.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.214+0000 7f96e2200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96dc103ff0 0x7f96dc19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:47.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.214+0000 7f96e2200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96dc1049a0 0x7f96dc19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:47.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.214+0000 7f96e2200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96dc109120 0x7f96dc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:47.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.214+0000 7f96e2200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc114170 con 0x7f96dc109120 2024-09-10T08:47:47.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.214+0000 7f96e2200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc113ff0 con 0x7f96dc103ff0 2024-09-10T08:47:47.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.214+0000 7f96e2200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc1142f0 con 0x7f96dc1049a0 2024-09-10T08:47:47.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.214+0000 7f96d3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96dc1049a0 0x7f96dc19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:47.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.214+0000 7f96dbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96dc109120 0x7f96dc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:47.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.214+0000 7f96d3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96dc1049a0 0x7f96dc19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:40232/0 (socket says 172.21.15.89:40232) 2024-09-10T08:47:47.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.214+0000 7f96db400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96dc103ff0 0x7f96dc19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:47.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.214+0000 7f96d3e00640 1 -- 172.21.15.89:0/857616063 learned_addr learned my addr 172.21.15.89:0/857616063 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:47.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.215+0000 7f96d3e00640 1 -- 172.21.15.89:0/857616063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96dc103ff0 msgr2=0x7f96dc19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:47.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.215+0000 7f96d3e00640 1 --2- 172.21.15.89:0/857616063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96dc103ff0 0x7f96dc19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:47.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.215+0000 7f96d3e00640 1 -- 172.21.15.89:0/857616063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96dc109120 msgr2=0x7f96dc19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:47.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.215+0000 7f96d3e00640 1 --2- 172.21.15.89:0/857616063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96dc109120 0x7f96dc19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:47.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.215+0000 7f96d3e00640 1 -- 172.21.15.89:0/857616063 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc19f620 con 0x7f96dc1049a0 2024-09-10T08:47:47.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.215+0000 7f96d3e00640 1 --2- 172.21.15.89:0/857616063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96dc1049a0 0x7f96dc19abb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f96c400d950 tx=0x7f96c400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:47.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.215+0000 7f96d8c00640 1 -- 172.21.15.89:0/857616063 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4017030 con 0x7f96dc1049a0 2024-09-10T08:47:47.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.215+0000 7f96d8c00640 1 -- 172.21.15.89:0/857616063 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96c4004020 con 0x7f96dc1049a0 2024-09-10T08:47:47.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.215+0000 7f96e2200640 1 -- 172.21.15.89:0/857616063 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96dc19f910 con 0x7f96dc1049a0 2024-09-10T08:47:47.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.215+0000 7f96d8c00640 1 -- 172.21.15.89:0/857616063 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c40047a0 con 0x7f96dc1049a0 2024-09-10T08:47:47.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.215+0000 7f96e2200640 1 -- 172.21.15.89:0/857616063 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96dc1a71f0 con 0x7f96dc1049a0 2024-09-10T08:47:47.223 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.216+0000 7f96e2200640 1 -- 172.21.15.89:0/857616063 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96dc1043d0 con 0x7f96dc1049a0 2024-09-10T08:47:47.224 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:46 smithi149 ceph-mon[36167]: pgmap v628: 129 pgs: 129 active+clean; 408 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 259 KiB/s wr, 129 op/s 2024-09-10T08:47:47.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.218+0000 7f96d8c00640 1 -- 172.21.15.89:0/857616063 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f96c40041d0 con 0x7f96dc1049a0 2024-09-10T08:47:47.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.220+0000 7f96d8c00640 1 --2- 172.21.15.89:0/857616063 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f96c00a2b80 0x7f96c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:47.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.220+0000 7f96d8c00640 1 -- 172.21.15.89:0/857616063 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f96c40cf070 con 0x7f96dc1049a0 2024-09-10T08:47:47.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.220+0000 7f96db400640 1 --2- 172.21.15.89:0/857616063 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f96c00a2b80 0x7f96c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:47.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.221+0000 7f96db400640 1 --2- 172.21.15.89:0/857616063 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f96c00a2b80 0x7f96c00a5040 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f96cc000c00 tx=0x7f96cc005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:47.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.222+0000 7f96d8c00640 1 -- 172.21.15.89:0/857616063 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f96c409bba0 con 0x7f96dc1049a0 2024-09-10T08:47:47.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:46 smithi111 ceph-mon[35220]: pgmap v628: 129 pgs: 129 active+clean; 408 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 259 KiB/s wr, 129 op/s 2024-09-10T08:47:47.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.429+0000 7f96e2200640 1 -- 172.21.15.89:0/857616063 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96dc19b9d0 con 0x7f96dc1049a0 2024-09-10T08:47:47.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.430+0000 7f96d8c00640 1 -- 172.21.15.89:0/857616063 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11921 (secure 0 0 0) 0x7f96c409b4a0 con 0x7f96dc1049a0 2024-09-10T08:47:47.432 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:47:47.432 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":31,"btime":"2024-09-10T08:47:28:912156+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:47:27.927133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":158,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24496,24518,24554]},"id":1}]} 2024-09-10T08:47:47.434 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 31 2024-09-10T08:47:47.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.433+0000 7f96e2200640 1 -- 172.21.15.89:0/857616063 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f96c00a2b80 msgr2=0x7f96c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:47.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.433+0000 7f96e2200640 1 --2- 172.21.15.89:0/857616063 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f96c00a2b80 0x7f96c00a5040 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f96cc000c00 tx=0x7f96cc005b50 comp rx=0 tx=0).stop 2024-09-10T08:47:47.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.433+0000 7f96e2200640 1 -- 172.21.15.89:0/857616063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96dc1049a0 msgr2=0x7f96dc19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:47.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.433+0000 7f96e2200640 1 --2- 172.21.15.89:0/857616063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96dc1049a0 0x7f96dc19abb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f96c400d950 tx=0x7f96c400de10 comp rx=0 tx=0).stop 2024-09-10T08:47:47.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.433+0000 7f96e2200640 1 -- 172.21.15.89:0/857616063 shutdown_connections 2024-09-10T08:47:47.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.433+0000 7f96e2200640 1 --2- 172.21.15.89:0/857616063 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f96c00a2b80 0x7f96c00a5040 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:47.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.433+0000 7f96e2200640 1 --2- 172.21.15.89:0/857616063 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96dc109120 0x7f96dc19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:47.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.433+0000 7f96e2200640 1 --2- 172.21.15.89:0/857616063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96dc1049a0 0x7f96dc19abb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:47.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.433+0000 7f96e2200640 1 --2- 172.21.15.89:0/857616063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96dc103ff0 0x7f96dc19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:47.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.433+0000 7f96e2200640 1 -- 172.21.15.89:0/857616063 >> 172.21.15.89:0/857616063 conn(0x7f96dc0ffa90 msgr2=0x7f96dc100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:47.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.433+0000 7f96e2200640 1 -- 172.21.15.89:0/857616063 shutdown_connections 2024-09-10T08:47:47.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:47.433+0000 7f96e2200640 1 -- 172.21.15.89:0/857616063 wait complete. 2024-09-10T08:47:48.267 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:47 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/857616063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:48.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:47 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/857616063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:48.306 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24554 quiesce db --set-id a83c6b0f --release --await' 2024-09-10T08:47:48.307 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24554 quiesce db --set-id a83c6b0f --release --await 2024-09-10T08:47:48.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:47 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/857616063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:48.671 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:49.105 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:48 smithi089 ceph-mon[30757]: pgmap v629: 129 pgs: 129 active+clean; 408 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 216 KiB/s wr, 110 op/s 2024-09-10T08:47:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.222+0000 7f6eafe00640 1 -- 172.21.15.89:0/3244876747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6eb0103e50 msgr2=0x7f6eb0104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.222+0000 7f6eafe00640 1 --2- 172.21.15.89:0/3244876747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6eb0103e50 0x7f6eb0104230 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009900 tx=0x7f6ea002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.222+0000 7f6eafe00640 1 -- 172.21.15.89:0/3244876747 shutdown_connections 2024-09-10T08:47:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.222+0000 7f6eafe00640 1 --2- 172.21.15.89:0/3244876747 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6eb0109090 0x7f6eb0110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.222+0000 7f6eafe00640 1 --2- 172.21.15.89:0/3244876747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6eb0104800 0x7f6eb0108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.222+0000 7f6eafe00640 1 --2- 172.21.15.89:0/3244876747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6eb0103e50 0x7f6eb0104230 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.222+0000 7f6eafe00640 1 -- 172.21.15.89:0/3244876747 >> 172.21.15.89:0/3244876747 conn(0x7f6eb00ff8b0 msgr2=0x7f6eb0101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.222+0000 7f6eafe00640 1 -- 172.21.15.89:0/3244876747 shutdown_connections 2024-09-10T08:47:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.222+0000 7f6eafe00640 1 -- 172.21.15.89:0/3244876747 wait complete. 2024-09-10T08:47:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.223+0000 7f6eafe00640 1 Processor -- start 2024-09-10T08:47:49.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.223+0000 7f6eafe00640 1 -- start start 2024-09-10T08:47:49.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.224+0000 7f6eafe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6eb0103e50 0x7f6eb019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:49.226 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.224+0000 7f6eafe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6eb0104800 0x7f6eb019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.224+0000 7f6eafe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6eb0109090 0x7f6eb019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.224+0000 7f6eafe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb0114110 con 0x7f6eb0104800 2024-09-10T08:47:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.224+0000 7f6eafe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb0113f90 con 0x7f6eb0109090 2024-09-10T08:47:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.224+0000 7f6eafe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb0114290 con 0x7f6eb0103e50 2024-09-10T08:47:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.224+0000 7f6ea7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6eb0104800 0x7f6eb019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.224+0000 7f6eaea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6eb0103e50 0x7f6eb019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.224+0000 7f6eaea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6eb0103e50 0x7f6eb019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:40252/0 (socket says 172.21.15.89:40252) 2024-09-10T08:47:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.224+0000 7f6eaea00640 1 -- 172.21.15.89:0/3935999198 learned_addr learned my addr 172.21.15.89:0/3935999198 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.224+0000 7f6eaf400640 1 --2- 172.21.15.89:0/3935999198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6eb0109090 0x7f6eb019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.224+0000 7f6ea7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6eb0104800 0x7f6eb019aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53744/0 (socket says 172.21.15.89:53744) 2024-09-10T08:47:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.224+0000 7f6eaea00640 1 -- 172.21.15.89:0/3935999198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6eb0109090 msgr2=0x7f6eb019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.224+0000 7f6eaea00640 1 --2- 172.21.15.89:0/3935999198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6eb0109090 0x7f6eb019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.224+0000 7f6eaea00640 1 -- 172.21.15.89:0/3935999198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6eb0104800 msgr2=0x7f6eb019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:49.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.224+0000 7f6eaea00640 1 --2- 172.21.15.89:0/3935999198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6eb0104800 0x7f6eb019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.224+0000 7f6eaea00640 1 -- 172.21.15.89:0/3935999198 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb019f540 con 0x7f6eb0103e50 2024-09-10T08:47:49.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.225+0000 7f6eaea00640 1 --2- 172.21.15.89:0/3935999198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6eb0103e50 0x7f6eb019a590 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0004770 tx=0x7f6ea0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:49.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.225+0000 7f6ea7400640 1 -- 172.21.15.89:0/3935999198 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea003f070 con 0x7f6eb0103e50 2024-09-10T08:47:49.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.225+0000 7f6eafe00640 1 -- 172.21.15.89:0/3935999198 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb019f7d0 con 0x7f6eb0103e50 2024-09-10T08:47:49.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.225+0000 7f6ea7400640 1 -- 172.21.15.89:0/3935999198 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ea0031ca0 con 0x7f6eb0103e50 2024-09-10T08:47:49.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.225+0000 7f6ea7400640 1 -- 172.21.15.89:0/3935999198 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0035030 con 0x7f6eb0103e50 2024-09-10T08:47:49.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.225+0000 7f6eafe00640 1 -- 172.21.15.89:0/3935999198 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb01a7110 con 0x7f6eb0103e50 2024-09-10T08:47:49.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.228+0000 7f6eb8455740 1 Processor -- start 2024-09-10T08:47:49.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.228+0000 7f6eb8455740 1 -- start start 2024-09-10T08:47:49.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.228+0000 7f6ea7400640 1 -- 172.21.15.89:0/3935999198 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6ea0042070 con 0x7f6eb0103e50 2024-09-10T08:47:49.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.229+0000 7f6eb8455740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70ad520 0x5564b70ad900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:49.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.229+0000 7f6eb8455740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70ade40 0x5564b70b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:49.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.229+0000 7f6eb8455740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70b8700 0x5564b70baaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:49.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.229+0000 7f6eb8455740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5564b7013650 con 0x5564b70ade40 2024-09-10T08:47:49.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.229+0000 7f6eb8455740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5564b6fbaf10 con 0x5564b70b8700 2024-09-10T08:47:49.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.229+0000 7f6eb8455740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5564b700f610 con 0x5564b70ad520 2024-09-10T08:47:49.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.229+0000 7f6eaea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70ade40 0x5564b70b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:49.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.229+0000 7f6eaea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70ade40 0x5564b70b81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53754/0 (socket says 172.21.15.89:53754) 2024-09-10T08:47:49.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.229+0000 7f6eaea00640 1 -- 172.21.15.89:0/3466817823 learned_addr learned my addr 172.21.15.89:0/3466817823 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:49.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.229+0000 7f6ea7e00640 1 --2- 172.21.15.89:0/3466817823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70b8700 0x5564b70baaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:49.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.229+0000 7f6eaf400640 1 --2- 172.21.15.89:0/3466817823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70ad520 0x5564b70ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:49.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.229+0000 7f6ea7e00640 1 -- 172.21.15.89:0/3466817823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70ad520 msgr2=0x5564b70ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:49.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.229+0000 7f6ea7e00640 1 --2- 172.21.15.89:0/3466817823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70ad520 0x5564b70ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.229+0000 7f6ea7e00640 1 -- 172.21.15.89:0/3466817823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70ade40 msgr2=0x5564b70b81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:49.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.229+0000 7f6ea7e00640 1 --2- 172.21.15.89:0/3466817823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70ade40 0x5564b70b81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.229+0000 7f6ea7e00640 1 -- 172.21.15.89:0/3466817823 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564b6d22580 con 0x5564b70b8700 2024-09-10T08:47:49.232 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.230+0000 7f6ea7e00640 1 --2- 172.21.15.89:0/3466817823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70b8700 0x5564b70baaf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009bf0 tx=0x7f6e98011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=917765f025afa821 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:49.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.230+0000 7f6e9fe00640 1 -- 172.21.15.89:0/3466817823 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9801e070 con 0x5564b70b8700 2024-09-10T08:47:49.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.230+0000 7f6e9fe00640 1 -- 172.21.15.89:0/3466817823 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e98009da0 con 0x5564b70b8700 2024-09-10T08:47:49.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.230+0000 7f6e9fe00640 1 -- 172.21.15.89:0/3466817823 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98018810 con 0x5564b70b8700 2024-09-10T08:47:49.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.230+0000 7f6ea7400640 1 --2- 172.21.15.89:0/3935999198 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6e940a2b80 0x7f6e940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:49.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.230+0000 7f6ea7400640 1 -- 172.21.15.89:0/3935999198 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f6ea00ef6d0 con 0x7f6eb0103e50 2024-09-10T08:47:49.236 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.230+0000 7f6eaf400640 1 --2- 172.21.15.89:0/3935999198 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6e940a2b80 0x7f6e940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:49.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.231+0000 7f6eb8455740 1 -- 172.21.15.89:0/3466817823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70b8700 msgr2=0x5564b70baaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:49.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.231+0000 7f6eb8455740 1 --2- 172.21.15.89:0/3466817823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70b8700 0x5564b70baaf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009bf0 tx=0x7f6e98011810 comp rx=0 tx=0).stop 2024-09-10T08:47:49.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.231+0000 7f6eb8455740 1 -- 172.21.15.89:0/3466817823 shutdown_connections 2024-09-10T08:47:49.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.231+0000 7f6eb8455740 1 --2- 172.21.15.89:0/3466817823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70b8700 0x5564b70baaf0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.231+0000 7f6eb8455740 1 --2- 172.21.15.89:0/3466817823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70ade40 0x5564b70b81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.231+0000 7f6eb8455740 1 --2- 172.21.15.89:0/3466817823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70ad520 0x5564b70ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.237 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.231+0000 7f6eb8455740 1 -- 172.21.15.89:0/3466817823 >> 172.21.15.89:0/3466817823 conn(0x5564b70a9ab0 msgr2=0x5564b70a9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:49.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.231+0000 7f6eaf400640 1 --2- 172.21.15.89:0/3935999198 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6e940a2b80 0x7f6e940a5040 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8005fd0 tx=0x7f6ea8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:49.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.231+0000 7f6eb8455740 1 -- 172.21.15.89:0/3466817823 shutdown_connections 2024-09-10T08:47:49.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.231+0000 7f6eb8455740 1 -- 172.21.15.89:0/3466817823 wait complete. 2024-09-10T08:47:49.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.232+0000 7f6eb8455740 1 -- start start 2024-09-10T08:47:49.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.232+0000 7f6eb8455740 1 Processor -- start 2024-09-10T08:47:49.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.232+0000 7f6eb8455740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70ad520 0x5564b70b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:49.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.232+0000 7f6eb8455740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70ade40 0x5564b70b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:49.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.232+0000 7f6eb8455740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70b8700 0x5564b70b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:49.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.232+0000 7f6eb8455740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5564b70bb2d0 con 0x5564b70ad520 2024-09-10T08:47:49.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.232+0000 7f6eb8455740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5564b6fce350 con 0x5564b70b8700 2024-09-10T08:47:49.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.232+0000 7f6eb8455740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5564b70bb450 con 0x5564b70ade40 2024-09-10T08:47:49.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.232+0000 7f6eb8455740 10 client.0 _ll_register_callbacks cb 0x5564b6ff1290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:49.238 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.233+0000 7f6eaea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70ad520 0x5564b70b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:49.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.233+0000 7f6eaea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70ad520 0x5564b70b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53766/0 (socket says 172.21.15.89:53766) 2024-09-10T08:47:49.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.233+0000 7f6eaea00640 1 -- 172.21.15.89:0/3973191407 learned_addr learned my addr 172.21.15.89:0/3973191407 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:49.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.233+0000 7f6ea7e00640 1 --2- 172.21.15.89:0/3973191407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70ade40 0x5564b70b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:49.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.233+0000 7f6eaf400640 1 --2- 172.21.15.89:0/3973191407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70b8700 0x5564b70b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:49.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.233+0000 7f6eaea00640 1 -- 172.21.15.89:0/3973191407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70ade40 msgr2=0x5564b70b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:49.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.233+0000 7f6eaea00640 1 --2- 172.21.15.89:0/3973191407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70ade40 0x5564b70b5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.233+0000 7f6eaea00640 1 -- 172.21.15.89:0/3973191407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70b8700 msgr2=0x5564b70b5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:49.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.233+0000 7f6eaea00640 1 --2- 172.21.15.89:0/3973191407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70b8700 0x5564b70b5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.233+0000 7f6eaea00640 1 -- 172.21.15.89:0/3973191407 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564b6dec960 con 0x5564b70ad520 2024-09-10T08:47:49.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.233+0000 7f6eaf400640 1 --2- 172.21.15.89:0/3973191407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70b8700 0x5564b70b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:49.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.233+0000 7f6eaea00640 1 --2- 172.21.15.89:0/3973191407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70ad520 0x5564b70b5560 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00ef970 tx=0x7f6ea0031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f285cc6a94c446 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:49.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.233+0000 7f6e9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:47:49.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.234+0000 7f6e9cc00640 1 -- 172.21.15.89:0/3973191407 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0049070 con 0x5564b70ad520 2024-09-10T08:47:49.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.234+0000 7f6e9cc00640 1 -- 172.21.15.89:0/3973191407 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f6ea00ef360 con 0x5564b70ad520 2024-09-10T08:47:49.239 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.234+0000 7f6eb8455740 1 -- 172.21.15.89:0/3973191407 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e98009da0 con 0x5564b70ad520 2024-09-10T08:47:49.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.234+0000 7f6e9cc00640 1 -- 172.21.15.89:0/3973191407 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6ea00d9070 con 0x5564b70ad520 2024-09-10T08:47:49.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.234+0000 7f6eb8455740 10 client.16821 fetch_fsmap learned FSMap version 31 2024-09-10T08:47:49.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.234+0000 7f6eb8455740 1 -- 172.21.15.89:0/3973191407 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5564b70c6ec0 con 0x5564b70ad520 2024-09-10T08:47:49.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.234+0000 7f6e9cc00640 1 -- 172.21.15.89:0/3973191407 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e98009da0 con 0x5564b70ad520 2024-09-10T08:47:49.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.234+0000 7f6e9cc00640 1 -- 172.21.15.89:0/3973191407 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f6ea004c060 con 0x5564b70ad520 2024-09-10T08:47:49.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.234+0000 7f6eb8455740 10 client.16821 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:47:49.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.234+0000 7f6eb8455740 10 client.16821 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:47:49.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.234+0000 7f6eb8455740 20 client.16821 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:49.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.234+0000 7f6eb8455740 1 --2- 172.21.15.89:0/3973191407 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5564b70c8320 0x5564b70ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:49.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.234+0000 7f6eb8455740 4 client.16821 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:49.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.234+0000 7f6eb8455740 1 -- 172.21.15.89:0/3973191407 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564b7020bc0 con 0x5564b70c8320 2024-09-10T08:47:49.240 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.235+0000 7f6ea7e00640 1 --2- 172.21.15.89:0/3973191407 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5564b70c8320 0x5564b70ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:49.241 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.235+0000 7f6ea7e00640 1 --2- 172.21.15.89:0/3973191407 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5564b70c8320 0x5564b70ca770 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e99eba89c15a0a04 server_cookie=ef861f245eb875e1 in_seq=0 out_seq=0 2024-09-10T08:47:49.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.235+0000 7f6e9cc00640 10 client.16821 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:49.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.236+0000 7f6e9cc00640 1 -- 172.21.15.89:0/3973191407 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5564b7020bc0 con 0x5564b70c8320 2024-09-10T08:47:49.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.236+0000 7f6e9cc00640 10 client.16821 handle_command_reply: tid=0 2024-09-10T08:47:49.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.236+0000 7f6eb8455740 1 client.16821 shutdown 2024-09-10T08:47:49.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.236+0000 7f6eb8455740 1 -- 172.21.15.89:0/3973191407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70ad520 msgr2=0x5564b70b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:49.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.236+0000 7f6eb8455740 1 --2- 172.21.15.89:0/3973191407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70ad520 0x5564b70b5560 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00ef970 tx=0x7f6ea0031f10 comp rx=0 tx=0).stop 2024-09-10T08:47:49.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.237+0000 7f6eb8455740 1 -- 172.21.15.89:0/3973191407 shutdown_connections 2024-09-10T08:47:49.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.237+0000 7f6eb8455740 1 --2- 172.21.15.89:0/3973191407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70b8700 0x5564b70b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.237+0000 7f6eb8455740 1 --2- 172.21.15.89:0/3973191407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70ade40 0x5564b70b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.237+0000 7f6eb8455740 1 --2- 172.21.15.89:0/3973191407 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5564b70c8320 0x5564b70ca770 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.237+0000 7f6eb8455740 1 --2- 172.21.15.89:0/3973191407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70ad520 0x5564b70b5560 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.237+0000 7f6eb8455740 1 -- 172.21.15.89:0/3973191407 >> 172.21.15.89:0/3973191407 conn(0x5564b70a9ab0 msgr2=0x5564b70a8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:49.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.237+0000 7f6e9cc00640 0 client.16821 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:49.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.237+0000 7f6eb8455740 1 -- 172.21.15.89:0/3973191407 shutdown_connections 2024-09-10T08:47:49.242 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.237+0000 7f6eb8455740 1 -- 172.21.15.89:0/3973191407 wait complete. 2024-09-10T08:47:49.243 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.237+0000 7f6eb8455740 20 client.16821 trim_cache size 0 max 16384 2024-09-10T08:47:49.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.254+0000 7f6eb8455740 1 Processor -- start 2024-09-10T08:47:49.255 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.254+0000 7f6eb8455740 1 -- start start 2024-09-10T08:47:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.255+0000 7f6eb8455740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70ad520 0x5564b70a8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.255+0000 7f6eb8455740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70ade40 0x5564b70d0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.255+0000 7f6eb8455740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70b8700 0x5564b70d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.255+0000 7f6eb8455740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb01b3720 con 0x5564b70ade40 2024-09-10T08:47:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.255+0000 7f6eb8455740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb01b0fc0 con 0x5564b70ad520 2024-09-10T08:47:49.257 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.255+0000 7f6eb8455740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb01aee10 con 0x5564b70b8700 2024-09-10T08:47:49.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.255+0000 7f6ea7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70ade40 0x5564b70d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:49.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.255+0000 7f6ea7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70ade40 0x5564b70d0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53772/0 (socket says 172.21.15.89:53772) 2024-09-10T08:47:49.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.255+0000 7f6ea7e00640 1 -- 172.21.15.89:0/298687581 learned_addr learned my addr 172.21.15.89:0/298687581 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:49.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.255+0000 7f6eaf400640 1 --2- 172.21.15.89:0/298687581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70b8700 0x5564b70d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:49.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.255+0000 7f6eaea00640 1 --2- 172.21.15.89:0/298687581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70ad520 0x5564b70a8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:49.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.255+0000 7f6ea7e00640 1 -- 172.21.15.89:0/298687581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70b8700 msgr2=0x5564b70d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:49.258 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.255+0000 7f6ea7e00640 1 --2- 172.21.15.89:0/298687581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70b8700 0x5564b70d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.255+0000 7f6ea7e00640 1 -- 172.21.15.89:0/298687581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70ad520 msgr2=0x5564b70a8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:49.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.255+0000 7f6ea7e00640 1 --2- 172.21.15.89:0/298687581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70ad520 0x5564b70a8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.255+0000 7f6ea7e00640 1 -- 172.21.15.89:0/298687581 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564b70a98e0 con 0x5564b70ade40 2024-09-10T08:47:49.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.256+0000 7f6ea7e00640 1 --2- 172.21.15.89:0/298687581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70ade40 0x5564b70d0050 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009bf0 tx=0x7f6e980183b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2560c146d0df9b84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:49.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.256+0000 7f6e9f400640 1 -- 172.21.15.89:0/298687581 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9801e040 con 0x5564b70ade40 2024-09-10T08:47:49.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.256+0000 7f6e9f400640 1 -- 172.21.15.89:0/298687581 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e9802a030 con 0x5564b70ade40 2024-09-10T08:47:49.259 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.256+0000 7f6e9f400640 1 -- 172.21.15.89:0/298687581 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98003b20 con 0x5564b70ade40 2024-09-10T08:47:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.257+0000 7f6eb8455740 1 -- 172.21.15.89:0/298687581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70ade40 msgr2=0x5564b70d0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.257+0000 7f6eb8455740 1 --2- 172.21.15.89:0/298687581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70ade40 0x5564b70d0050 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009bf0 tx=0x7f6e980183b0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.257+0000 7f6eb8455740 1 -- 172.21.15.89:0/298687581 shutdown_connections 2024-09-10T08:47:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.257+0000 7f6eb8455740 1 --2- 172.21.15.89:0/298687581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70b8700 0x5564b70d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.257+0000 7f6eb8455740 1 --2- 172.21.15.89:0/298687581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70ade40 0x5564b70d0050 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.257+0000 7f6eb8455740 1 --2- 172.21.15.89:0/298687581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70ad520 0x5564b70a8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.257+0000 7f6eb8455740 1 -- 172.21.15.89:0/298687581 >> 172.21.15.89:0/298687581 conn(0x5564b70a9ab0 msgr2=0x5564b70a84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.257+0000 7f6eb8455740 1 -- 172.21.15.89:0/298687581 shutdown_connections 2024-09-10T08:47:49.262 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.257+0000 7f6eb8455740 1 -- 172.21.15.89:0/298687581 wait complete. 2024-09-10T08:47:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.257+0000 7f6eb8455740 1 -- start start 2024-09-10T08:47:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.257+0000 7f6eb8455740 1 Processor -- start 2024-09-10T08:47:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.258+0000 7f6eb8455740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70ad520 0x5564b70c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.258+0000 7f6eb8455740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70ade40 0x5564b70bc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.258+0000 7f6eb8455740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70b8700 0x5564b70bc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.258+0000 7f6eb8455740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb01af480 con 0x5564b70b8700 2024-09-10T08:47:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.258+0000 7f6eb8455740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb01b08f0 con 0x5564b70ad520 2024-09-10T08:47:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.258+0000 7f6eb8455740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb01afb50 con 0x5564b70ade40 2024-09-10T08:47:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.258+0000 7f6eb8455740 10 client.0 _ll_register_callbacks cb 0x7f6ea00f5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.258+0000 7f6eaf400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70b8700 0x5564b70bc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.258+0000 7f6eaf400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70b8700 0x5564b70bc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:53774/0 (socket says 172.21.15.89:53774) 2024-09-10T08:47:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.258+0000 7f6eaea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70ad520 0x5564b70c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:49.263 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.258+0000 7f6eaf400640 1 -- 172.21.15.89:0/212887515 learned_addr learned my addr 172.21.15.89:0/212887515 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:49.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.258+0000 7f6ea7e00640 1 --2- 172.21.15.89:0/212887515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70ade40 0x5564b70bc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:49.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.259+0000 7f6eaf400640 1 -- 172.21.15.89:0/212887515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70ade40 msgr2=0x5564b70bc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:49.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.259+0000 7f6eaf400640 1 --2- 172.21.15.89:0/212887515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70ade40 0x5564b70bc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.259+0000 7f6eaf400640 1 -- 172.21.15.89:0/212887515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70ad520 msgr2=0x5564b70c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:49.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.259+0000 7f6eaf400640 1 --2- 172.21.15.89:0/212887515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70ad520 0x5564b70c0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:49.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.259+0000 7f6eaf400640 1 -- 172.21.15.89:0/212887515 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564b70bd100 con 0x5564b70b8700 2024-09-10T08:47:49.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.259+0000 7f6eaea00640 1 --2- 172.21.15.89:0/212887515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70ad520 0x5564b70c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:49.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.259+0000 7f6ea7e00640 1 --2- 172.21.15.89:0/212887515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70ade40 0x5564b70bc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:49.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.259+0000 7f6eaf400640 1 --2- 172.21.15.89:0/212887515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70b8700 0x5564b70bc8d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8004250 tx=0x7f6ea8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf1441f699b241a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:49.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.259+0000 7f6e9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:47:49.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.259+0000 7f6e9cc00640 1 -- 172.21.15.89:0/212887515 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea802e060 con 0x5564b70b8700 2024-09-10T08:47:49.264 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.259+0000 7f6e9cc00640 1 -- 172.21.15.89:0/212887515 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f6ea800f690 con 0x5564b70b8700 2024-09-10T08:47:49.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.259+0000 7f6eb8455740 1 -- 172.21.15.89:0/212887515 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e9802a030 con 0x5564b70b8700 2024-09-10T08:47:49.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.260+0000 7f6e9cc00640 1 -- 172.21.15.89:0/212887515 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6ea8035050 con 0x5564b70b8700 2024-09-10T08:47:49.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.260+0000 7f6eb8455740 10 client.16827 fetch_fsmap learned FSMap version 31 2024-09-10T08:47:49.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.260+0000 7f6eb8455740 1 -- 172.21.15.89:0/212887515 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5564b71331f0 con 0x5564b70b8700 2024-09-10T08:47:49.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.260+0000 7f6e9cc00640 1 -- 172.21.15.89:0/212887515 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e9802a030 con 0x5564b70b8700 2024-09-10T08:47:49.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.260+0000 7f6e9cc00640 1 -- 172.21.15.89:0/212887515 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f6ea8028070 con 0x5564b70b8700 2024-09-10T08:47:49.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.260+0000 7f6eb8455740 10 client.16827 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:47:49.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.260+0000 7f6eb8455740 10 client.16827 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:47:49.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.260+0000 7f6eb8455740 20 client.16827 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:49.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.260+0000 7f6eb8455740 1 --2- 172.21.15.89:0/212887515 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5564b71339a0 0x5564b7135dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:49.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.260+0000 7f6eb8455740 4 client.16827 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a83c6b0f", "release": true, "await": true}] 2024-09-10T08:47:49.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.260+0000 7f6eb8455740 1 -- 172.21.15.89:0/212887515 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a83c6b0f", "release": true, "await": true}) -- 0x5564b70cab00 con 0x5564b71339a0 2024-09-10T08:47:49.265 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.261+0000 7f6eaea00640 1 --2- 172.21.15.89:0/212887515 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5564b71339a0 0x5564b7135dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.261+0000 7f6eaea00640 1 --2- 172.21.15.89:0/212887515 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5564b71339a0 0x5564b7135dc0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=356e6177edc94ecb server_cookie=e1a2ef26997d8e36 in_seq=0 out_seq=0 2024-09-10T08:47:49.266 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:49.261+0000 7f6e9cc00640 10 client.16827 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:49.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:48 smithi111 ceph-mon[35220]: pgmap v629: 129 pgs: 129 active+clean; 408 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 216 KiB/s wr, 110 op/s 2024-09-10T08:47:49.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:48 smithi149 ceph-mon[36167]: pgmap v629: 129 pgs: 129 active+clean; 408 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 216 KiB/s wr, 110 op/s 2024-09-10T08:47:50.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.011+0000 7f6e9cc00640 1 -- 172.21.15.89:0/212887515 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5564b70cab00 con 0x5564b71339a0 2024-09-10T08:47:50.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.011+0000 7f6e9cc00640 10 client.16827 handle_command_reply: tid=0 2024-09-10T08:47:50.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.011+0000 7f6eb8455740 1 client.16827 shutdown 2024-09-10T08:47:50.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.011+0000 7f6eb8455740 1 -- 172.21.15.89:0/212887515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70b8700 msgr2=0x5564b70bc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:50.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.011+0000 7f6eb8455740 1 --2- 172.21.15.89:0/212887515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70b8700 0x5564b70bc8d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8004250 tx=0x7f6ea8004ab0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.011+0000 7f6eb8455740 1 -- 172.21.15.89:0/212887515 shutdown_connections 2024-09-10T08:47:50.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.011+0000 7f6eb8455740 1 --2- 172.21.15.89:0/212887515 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5564b71339a0 0x5564b7135dc0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.011+0000 7f6eb8455740 1 --2- 172.21.15.89:0/212887515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564b70b8700 0x5564b70bc8d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.011+0000 7f6eb8455740 1 --2- 172.21.15.89:0/212887515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564b70ade40 0x5564b70bc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.011+0000 7f6eb8455740 1 --2- 172.21.15.89:0/212887515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5564b70ad520 0x5564b70c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.011+0000 7f6eb8455740 1 -- 172.21.15.89:0/212887515 >> 172.21.15.89:0/212887515 conn(0x5564b70a9ab0 msgr2=0x5564b70a8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:50.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.011+0000 7f6e9cc00640 0 client.16827 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:50.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.011+0000 7f6eb8455740 1 -- 172.21.15.89:0/212887515 shutdown_connections 2024-09-10T08:47:50.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.011+0000 7f6eb8455740 1 -- 172.21.15.89:0/212887515 wait complete. 2024-09-10T08:47:50.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.011+0000 7f6eb8455740 20 client.16827 trim_cache size 0 max 16384 2024-09-10T08:47:50.014 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:47:50.014 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 31, 2024-09-10T08:47:50.014 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24554, 2024-09-10T08:47:50.014 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 24, 2024-09-10T08:47:50.015 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": { 2024-09-10T08:47:50.015 INFO:teuthology.orchestra.run.smithi089.stdout: "a83c6b0f": { 2024-09-10T08:47:50.015 INFO:teuthology.orchestra.run.smithi089.stdout: "version": 24, 2024-09-10T08:47:50.015 INFO:teuthology.orchestra.run.smithi089.stdout: "age_ref": 0.0, 2024-09-10T08:47:50.015 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:47:50.015 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RELEASED", 2024-09-10T08:47:50.015 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:47:50.015 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:47:50.015 INFO:teuthology.orchestra.run.smithi089.stdout: "timeout": 90.0, 2024-09-10T08:47:50.015 INFO:teuthology.orchestra.run.smithi089.stdout: "expiration": 140.1, 2024-09-10T08:47:50.015 INFO:teuthology.orchestra.run.smithi089.stdout: "members": { 2024-09-10T08:47:50.015 INFO:teuthology.orchestra.run.smithi089.stdout: "file:/": { 2024-09-10T08:47:50.015 INFO:teuthology.orchestra.run.smithi089.stdout: "excluded": false, 2024-09-10T08:47:50.016 INFO:teuthology.orchestra.run.smithi089.stdout: "state": { 2024-09-10T08:47:50.016 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RELEASED", 2024-09-10T08:47:50.016 INFO:teuthology.orchestra.run.smithi089.stdout: "age": 0.0 2024-09-10T08:47:50.016 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:50.016 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:50.016 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:50.016 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:50.016 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:47:50.016 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:47:50.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.012+0000 7f6eafe00640 1 -- 172.21.15.89:0/3935999198 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6e940a2b80 msgr2=0x7f6e940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:50.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.012+0000 7f6eafe00640 1 --2- 172.21.15.89:0/3935999198 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6e940a2b80 0x7f6e940a5040 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8005fd0 tx=0x7f6ea8009500 comp rx=0 tx=0).stop 2024-09-10T08:47:50.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.012+0000 7f6eafe00640 1 -- 172.21.15.89:0/3935999198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6eb0103e50 msgr2=0x7f6eb019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:50.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.012+0000 7f6eafe00640 1 --2- 172.21.15.89:0/3935999198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6eb0103e50 0x7f6eb019a590 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0004770 tx=0x7f6ea0003c10 comp rx=0 tx=0).stop 2024-09-10T08:47:50.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.012+0000 7f6eafe00640 1 -- 172.21.15.89:0/3935999198 shutdown_connections 2024-09-10T08:47:50.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.012+0000 7f6eafe00640 1 --2- 172.21.15.89:0/3935999198 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6e940a2b80 0x7f6e940a5040 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.012+0000 7f6eafe00640 1 --2- 172.21.15.89:0/3935999198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6eb0109090 0x7f6eb019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.012+0000 7f6eafe00640 1 --2- 172.21.15.89:0/3935999198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6eb0104800 0x7f6eb019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.012+0000 7f6eafe00640 1 --2- 172.21.15.89:0/3935999198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6eb0103e50 0x7f6eb019a590 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.012+0000 7f6eafe00640 1 -- 172.21.15.89:0/3935999198 >> 172.21.15.89:0/3935999198 conn(0x7f6eb00ff8b0 msgr2=0x7f6eb0100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:50.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.012+0000 7f6eafe00640 1 -- 172.21.15.89:0/3935999198 shutdown_connections 2024-09-10T08:47:50.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:47:50.013+0000 7f6eafe00640 1 -- 172.21.15.89:0/3935999198 wait complete. 2024-09-10T08:47:50.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:49 smithi111 ceph-mon[35220]: pgmap v630: 129 pgs: 129 active+clean; 408 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 42 op/s 2024-09-10T08:47:50.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:49 smithi089 ceph-mon[30757]: pgmap v630: 129 pgs: 129 active+clean; 408 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 42 op/s 2024-09-10T08:47:50.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:49 smithi149 ceph-mon[36167]: pgmap v630: 129 pgs: 129 active+clean; 408 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 42 op/s 2024-09-10T08:47:50.906 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a83c6b0f', total seconds elapsed: 24.7 2024-09-10T08:47:53.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:52 smithi149 ceph-mon[36167]: pgmap v631: 129 pgs: 129 active+clean; 409 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 239 KiB/s wr, 45 op/s 2024-09-10T08:47:53.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:52 smithi111 ceph-mon[35220]: pgmap v631: 129 pgs: 129 active+clean; 409 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 239 KiB/s wr, 45 op/s 2024-09-10T08:47:53.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:52 smithi089 ceph-mon[30757]: pgmap v631: 129 pgs: 129 active+clean; 409 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 239 KiB/s wr, 45 op/s 2024-09-10T08:47:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.466+0000 7f7450800640 1 -- 172.21.15.89:0/2665652100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7448072d20 msgr2=0x7f7448073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.466+0000 7f7450800640 1 --2- 172.21.15.89:0/2665652100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7448072d20 0x7f7448073180 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7438009900 tx=0x7f743802a400 comp rx=0 tx=0).stop 2024-09-10T08:47:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.466+0000 7f7450800640 1 -- 172.21.15.89:0/2665652100 shutdown_connections 2024-09-10T08:47:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.466+0000 7f7450800640 1 --2- 172.21.15.89:0/2665652100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74480737d0 0x7f7448112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.466+0000 7f7450800640 1 --2- 172.21.15.89:0/2665652100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7448072d20 0x7f7448073180 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.466+0000 7f7450800640 1 --2- 172.21.15.89:0/2665652100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7448109930 0x7f7448109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.466+0000 7f7450800640 1 -- 172.21.15.89:0/2665652100 >> 172.21.15.89:0/2665652100 conn(0x7f744806ece0 msgr2=0x7f7448071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.466+0000 7f7450800640 1 -- 172.21.15.89:0/2665652100 shutdown_connections 2024-09-10T08:47:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.466+0000 7f7450800640 1 -- 172.21.15.89:0/2665652100 wait complete. 2024-09-10T08:47:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.468+0000 7f7450800640 1 Processor -- start 2024-09-10T08:47:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.468+0000 7f7450800640 1 -- start start 2024-09-10T08:47:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.469+0000 7f7450800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7448072d20 0x7f74481a6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.470+0000 7f7450800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74480737d0 0x7f74481a70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.470+0000 7f7450800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7448109930 0x7f74481ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.470+0000 7f7450800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7448116020 con 0x7f7448109930 2024-09-10T08:47:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.470+0000 7f7450800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7448115ea0 con 0x7f74480737d0 2024-09-10T08:47:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.470+0000 7f7450800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f74481161a0 con 0x7f7448072d20 2024-09-10T08:47:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.470+0000 7f744de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7448072d20 0x7f74481a6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.471+0000 7f744de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7448072d20 0x7f74481a6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:47860/0 (socket says 172.21.15.89:47860) 2024-09-10T08:47:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.471+0000 7f744e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7448109930 0x7f74481ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.471+0000 7f744de00640 1 -- 172.21.15.89:0/1452829993 learned_addr learned my addr 172.21.15.89:0/1452829993 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.471+0000 7f744de00640 1 -- 172.21.15.89:0/1452829993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74480737d0 msgr2=0x7f74481a70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.471+0000 7f744de00640 1 --2- 172.21.15.89:0/1452829993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74480737d0 0x7f74481a70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.471+0000 7f744de00640 1 -- 172.21.15.89:0/1452829993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7448109930 msgr2=0x7f74481ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.471+0000 7f744de00640 1 --2- 172.21.15.89:0/1452829993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7448109930 0x7f74481ae120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.471+0000 7f744de00640 1 -- 172.21.15.89:0/1452829993 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74481ae790 con 0x7f7448072d20 2024-09-10T08:47:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.472+0000 7f744de00640 1 --2- 172.21.15.89:0/1452829993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7448072d20 0x7f74481a6b70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f743c00ebc0 tx=0x7f743c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.473+0000 7f7442a00640 1 -- 172.21.15.89:0/1452829993 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c010630 con 0x7f7448072d20 2024-09-10T08:47:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.473+0000 7f7442a00640 1 -- 172.21.15.89:0/1452829993 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f743c010cb0 con 0x7f7448072d20 2024-09-10T08:47:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.473+0000 7f7442a00640 1 -- 172.21.15.89:0/1452829993 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c018c40 con 0x7f7448072d20 2024-09-10T08:47:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.473+0000 7f7450800640 1 -- 172.21.15.89:0/1452829993 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74481aea80 con 0x7f7448072d20 2024-09-10T08:47:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.474+0000 7f7450800640 1 -- 172.21.15.89:0/1452829993 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74481aef80 con 0x7f7448072d20 2024-09-10T08:47:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.481+0000 7f7453642740 1 Processor -- start 2024-09-10T08:47:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.482+0000 7f7453642740 1 -- start start 2024-09-10T08:47:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.482+0000 7f7453642740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f5090 0x5615da2f5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.482+0000 7f7453642740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f59d0 0x5615da2fa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.482+0000 7f7453642740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2faa60 0x5615da2fcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.482+0000 7f7453642740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5615da25a100 con 0x5615da2f59d0 2024-09-10T08:47:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.482+0000 7f7453642740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5615da2021b0 con 0x5615da2faa60 2024-09-10T08:47:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.482+0000 7f7453642740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5615da2560c0 con 0x5615da2f5090 2024-09-10T08:47:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.483+0000 7f744e800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f5090 0x5615da2f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.483+0000 7f744d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2faa60 0x5615da2fcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.483+0000 7f744e800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f5090 0x5615da2f5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:47868/0 (socket says 172.21.15.89:47868) 2024-09-10T08:47:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.483+0000 7f744e800640 1 -- 172.21.15.89:0/3890643939 learned_addr learned my addr 172.21.15.89:0/3890643939 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.483+0000 7f744d400640 1 -- 172.21.15.89:0/3890643939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f5090 msgr2=0x5615da2f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.483+0000 7f744d400640 1 --2- 172.21.15.89:0/3890643939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f5090 0x5615da2f5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.483+0000 7f7442a00640 1 -- 172.21.15.89:0/1452829993 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f743c0107d0 con 0x7f7448072d20 2024-09-10T08:47:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.483+0000 7f744d400640 1 -- 172.21.15.89:0/3890643939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f59d0 msgr2=0x5615da2fa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.483+0000 7f744de00640 1 --2- 172.21.15.89:0/3890643939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f59d0 0x5615da2fa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.483+0000 7f744d400640 1 --2- 172.21.15.89:0/3890643939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f59d0 0x5615da2fa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.483+0000 7f744d400640 1 -- 172.21.15.89:0/3890643939 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615d9f6d210 con 0x5615da2faa60 2024-09-10T08:47:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.484+0000 7f744e800640 1 --2- 172.21.15.89:0/3890643939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f5090 0x5615da2f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.484+0000 7f744d400640 1 --2- 172.21.15.89:0/3890643939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2faa60 0x5615da2fcf20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f743802a920 tx=0x7f7438000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=987f26bd2596100 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.484+0000 7f744de00640 1 --2- 172.21.15.89:0/3890643939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f59d0 0x5615da2fa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.485+0000 7f7437000640 1 -- 172.21.15.89:0/3890643939 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743803e070 con 0x5615da2faa60 2024-09-10T08:47:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.485+0000 7f7437000640 1 -- 172.21.15.89:0/3890643939 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7438005530 con 0x5615da2faa60 2024-09-10T08:47:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.485+0000 7f7437000640 1 -- 172.21.15.89:0/3890643939 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438035040 con 0x5615da2faa60 2024-09-10T08:47:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.486+0000 7f7442a00640 1 --2- 172.21.15.89:0/1452829993 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f74300a2e20 0x7f74300a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.486+0000 7f7442a00640 1 -- 172.21.15.89:0/1452829993 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f743c0cfcc0 con 0x7f7448072d20 2024-09-10T08:47:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.487+0000 7f7453642740 1 -- 172.21.15.89:0/3890643939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2faa60 msgr2=0x5615da2fcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.487+0000 7f7453642740 1 --2- 172.21.15.89:0/3890643939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2faa60 0x5615da2fcf20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f743802a920 tx=0x7f7438000bc0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.487+0000 7f744e800640 1 --2- 172.21.15.89:0/1452829993 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f74300a2e20 0x7f74300a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.487+0000 7f7453642740 1 -- 172.21.15.89:0/3890643939 shutdown_connections 2024-09-10T08:47:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.487+0000 7f7453642740 1 --2- 172.21.15.89:0/3890643939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2faa60 0x5615da2fcf20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.487+0000 7f7453642740 1 --2- 172.21.15.89:0/3890643939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f59d0 0x5615da2fa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.487+0000 7f7453642740 1 --2- 172.21.15.89:0/3890643939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f5090 0x5615da2f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.487+0000 7f7453642740 1 -- 172.21.15.89:0/3890643939 >> 172.21.15.89:0/3890643939 conn(0x5615da2f0700 msgr2=0x5615da2f0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.487+0000 7f7453642740 1 -- 172.21.15.89:0/3890643939 shutdown_connections 2024-09-10T08:47:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.488+0000 7f7453642740 1 -- 172.21.15.89:0/3890643939 wait complete. 2024-09-10T08:47:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.488+0000 7f744e800640 1 --2- 172.21.15.89:0/1452829993 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f74300a2e20 0x7f74300a52e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7444005fd0 tx=0x7f744400b560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.489+0000 7f7453642740 1 -- start start 2024-09-10T08:47:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.489+0000 7f7453642740 1 Processor -- start 2024-09-10T08:47:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.490+0000 7f7453642740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f5090 0x5615da2ff210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.490+0000 7f7453642740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f59d0 0x5615da2ff750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.491+0000 7f7453642740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2faa60 0x5615da306790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.491+0000 7f7453642740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5615da2fd9d0 con 0x5615da2f59d0 2024-09-10T08:47:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.491+0000 7f7453642740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5615da1e3090 con 0x5615da2faa60 2024-09-10T08:47:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.491+0000 7f7453642740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5615da2fdb50 con 0x5615da2f5090 2024-09-10T08:47:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.491+0000 7f7453642740 10 client.0 _ll_register_callbacks cb 0x5615da237f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.491+0000 7f744e800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2faa60 0x5615da306790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.491+0000 7f744e800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2faa60 0x5615da306790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:52438/0 (socket says 172.21.15.89:52438) 2024-09-10T08:47:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.491+0000 7f744d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f59d0 0x5615da2ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.491+0000 7f744e800640 1 -- 172.21.15.89:0/4261816695 learned_addr learned my addr 172.21.15.89:0/4261816695 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.492+0000 7f744de00640 1 --2- 172.21.15.89:0/4261816695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f5090 0x5615da2ff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.492+0000 7f744e800640 1 -- 172.21.15.89:0/4261816695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f5090 msgr2=0x5615da2ff210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.492+0000 7f744e800640 1 --2- 172.21.15.89:0/4261816695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f5090 0x5615da2ff210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.492+0000 7f744e800640 1 -- 172.21.15.89:0/4261816695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f59d0 msgr2=0x5615da2ff750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.492+0000 7f744e800640 1 --2- 172.21.15.89:0/4261816695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f59d0 0x5615da2ff750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.492+0000 7f744e800640 1 -- 172.21.15.89:0/4261816695 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615da039620 con 0x5615da2faa60 2024-09-10T08:47:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.493+0000 7f744e800640 1 --2- 172.21.15.89:0/4261816695 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2faa60 0x5615da306790 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f744401d840 tx=0x7f74440046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e066b37d6b9fcd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.493+0000 7f742b400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:47:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.493+0000 7f742b400640 1 -- 172.21.15.89:0/4261816695 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744402a060 con 0x5615da2faa60 2024-09-10T08:47:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.493+0000 7f742b400640 1 -- 172.21.15.89:0/4261816695 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f7444010850 con 0x5615da2faa60 2024-09-10T08:47:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.494+0000 7f7453642740 1 -- 172.21.15.89:0/4261816695 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7438005530 con 0x5615da2faa60 2024-09-10T08:47:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.494+0000 7f742b400640 1 -- 172.21.15.89:0/4261816695 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7444010a50 con 0x5615da2faa60 2024-09-10T08:47:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.494+0000 7f7453642740 10 client.26906 fetch_fsmap learned FSMap version 31 2024-09-10T08:47:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.494+0000 7f7453642740 1 -- 172.21.15.89:0/4261816695 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5615da3095c0 con 0x5615da2faa60 2024-09-10T08:47:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.494+0000 7f742b400640 1 -- 172.21.15.89:0/4261816695 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7438005530 con 0x5615da2faa60 2024-09-10T08:47:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.495+0000 7f742b400640 1 -- 172.21.15.89:0/4261816695 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f7444019040 con 0x5615da2faa60 2024-09-10T08:47:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.495+0000 7f7453642740 10 client.26906 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:47:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.495+0000 7f7453642740 10 client.26906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.495+0000 7f7453642740 20 client.26906 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.495+0000 7f7453642740 1 --2- 172.21.15.89:0/4261816695 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5615da30ac40 0x5615da30d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.495+0000 7f7453642740 4 client.26906 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.495+0000 7f7453642740 1 -- 172.21.15.89:0/4261816695 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615da267830 con 0x5615da30ac40 2024-09-10T08:47:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.496+0000 7f744de00640 1 --2- 172.21.15.89:0/4261816695 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5615da30ac40 0x5615da30d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.498+0000 7f744de00640 1 --2- 172.21.15.89:0/4261816695 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5615da30ac40 0x5615da30d090 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5097c83cf75871e6 server_cookie=9ec0474cc81371a4 in_seq=0 out_seq=0 2024-09-10T08:47:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.498+0000 7f742b400640 10 client.26906 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.503+0000 7f742b400640 1 -- 172.21.15.89:0/4261816695 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5615da267830 con 0x5615da30ac40 2024-09-10T08:47:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.503+0000 7f742b400640 10 client.26906 handle_command_reply: tid=0 2024-09-10T08:47:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.503+0000 7f7453642740 1 client.26906 shutdown 2024-09-10T08:47:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.503+0000 7f7453642740 1 -- 172.21.15.89:0/4261816695 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2faa60 msgr2=0x5615da306790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.503+0000 7f7453642740 1 --2- 172.21.15.89:0/4261816695 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2faa60 0x5615da306790 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f744401d840 tx=0x7f74440046c0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.504+0000 7f7453642740 1 -- 172.21.15.89:0/4261816695 shutdown_connections 2024-09-10T08:47:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.504+0000 7f7453642740 1 --2- 172.21.15.89:0/4261816695 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2faa60 0x5615da306790 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.504+0000 7f7453642740 1 --2- 172.21.15.89:0/4261816695 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5615da30ac40 0x5615da30d090 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.504+0000 7f7453642740 1 --2- 172.21.15.89:0/4261816695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f59d0 0x5615da2ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.504+0000 7f7453642740 1 --2- 172.21.15.89:0/4261816695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f5090 0x5615da2ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.504+0000 7f7453642740 1 -- 172.21.15.89:0/4261816695 >> 172.21.15.89:0/4261816695 conn(0x5615da2f0700 msgr2=0x5615da2f1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.504+0000 7f742b400640 0 client.26906 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.504+0000 7f7453642740 1 -- 172.21.15.89:0/4261816695 shutdown_connections 2024-09-10T08:47:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.504+0000 7f7453642740 1 -- 172.21.15.89:0/4261816695 wait complete. 2024-09-10T08:47:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.504+0000 7f7453642740 20 client.26906 trim_cache size 0 max 16384 2024-09-10T08:47:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.530+0000 7f7453642740 1 Processor -- start 2024-09-10T08:47:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.530+0000 7f7453642740 1 -- start start 2024-09-10T08:47:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.531+0000 7f7453642740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f0700 0x5615da2f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.531+0000 7f7453642740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2f5090 0x5615da3066b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.531+0000 7f7453642740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f59d0 0x5615da2f1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.531+0000 7f7453642740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f74481bbe40 con 0x5615da2f59d0 2024-09-10T08:47:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.531+0000 7f7453642740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f74481b96e0 con 0x5615da2f5090 2024-09-10T08:47:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.531+0000 7f7453642740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f74481b7530 con 0x5615da2f0700 2024-09-10T08:47:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.531+0000 7f744e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f59d0 0x5615da2f1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.531+0000 7f744de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f0700 0x5615da2f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.532+0000 7f744de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f0700 0x5615da2f15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:47876/0 (socket says 172.21.15.89:47876) 2024-09-10T08:47:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.532+0000 7f744de00640 1 -- 172.21.15.89:0/3591471845 learned_addr learned my addr 172.21.15.89:0/3591471845 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.532+0000 7f744e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f59d0 0x5615da2f1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50400/0 (socket says 172.21.15.89:50400) 2024-09-10T08:47:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.532+0000 7f744d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2f5090 0x5615da3066b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.533+0000 7f744d400640 1 -- 172.21.15.89:0/3591471845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f0700 msgr2=0x5615da2f15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.533+0000 7f744d400640 1 --2- 172.21.15.89:0/3591471845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f0700 0x5615da2f15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.533+0000 7f744d400640 1 -- 172.21.15.89:0/3591471845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f59d0 msgr2=0x5615da2f1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.533+0000 7f744d400640 1 --2- 172.21.15.89:0/3591471845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f59d0 0x5615da2f1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.533+0000 7f744d400640 1 -- 172.21.15.89:0/3591471845 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615da2f0520 con 0x5615da2f5090 2024-09-10T08:47:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.534+0000 7f744d400640 1 --2- 172.21.15.89:0/3591471845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2f5090 0x5615da3066b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7438006010 tx=0x7f743802abb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f13b8ba933d85c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.534+0000 7f7436600640 1 -- 172.21.15.89:0/3591471845 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743803e050 con 0x5615da2f5090 2024-09-10T08:47:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.535+0000 7f7436600640 1 -- 172.21.15.89:0/3591471845 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7438005050 con 0x5615da2f5090 2024-09-10T08:47:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.535+0000 7f7436600640 1 -- 172.21.15.89:0/3591471845 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438035040 con 0x5615da2f5090 2024-09-10T08:47:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.536+0000 7f7453642740 1 -- 172.21.15.89:0/3591471845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2f5090 msgr2=0x5615da3066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.536+0000 7f7453642740 1 --2- 172.21.15.89:0/3591471845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2f5090 0x5615da3066b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7438006010 tx=0x7f743802abb0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.536+0000 7f7453642740 1 -- 172.21.15.89:0/3591471845 shutdown_connections 2024-09-10T08:47:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.536+0000 7f7453642740 1 --2- 172.21.15.89:0/3591471845 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f59d0 0x5615da2f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.536+0000 7f7453642740 1 --2- 172.21.15.89:0/3591471845 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2f5090 0x5615da3066b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.536+0000 7f7453642740 1 --2- 172.21.15.89:0/3591471845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f0700 0x5615da2f15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.536+0000 7f7453642740 1 -- 172.21.15.89:0/3591471845 >> 172.21.15.89:0/3591471845 conn(0x5615da2fa5f0 msgr2=0x5615da2ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.536+0000 7f7453642740 1 -- 172.21.15.89:0/3591471845 shutdown_connections 2024-09-10T08:47:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.536+0000 7f7453642740 1 -- 172.21.15.89:0/3591471845 wait complete. 2024-09-10T08:47:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.537+0000 7f7453642740 1 -- start start 2024-09-10T08:47:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.537+0000 7f7453642740 1 Processor -- start 2024-09-10T08:47:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.538+0000 7f7453642740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f0700 0x5615da3038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.538+0000 7f7453642740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2f5090 0x5615da303e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.539+0000 7f7453642740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f59d0 0x5615da304de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.539+0000 7f7453642740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f74481b7ba0 con 0x5615da2f0700 2024-09-10T08:47:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.539+0000 7f7453642740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f74481b9010 con 0x5615da2f5090 2024-09-10T08:47:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.539+0000 7f7453642740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f74481b8270 con 0x5615da2f59d0 2024-09-10T08:47:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.539+0000 7f7453642740 10 client.0 _ll_register_callbacks cb 0x7f744400bbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.539+0000 7f744d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2f5090 0x5615da303e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.539+0000 7f744d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2f5090 0x5615da303e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:52456/0 (socket says 172.21.15.89:52456) 2024-09-10T08:47:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.539+0000 7f744d400640 1 -- 172.21.15.89:0/2671033313 learned_addr learned my addr 172.21.15.89:0/2671033313 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:47:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.539+0000 7f744e800640 1 --2- 172.21.15.89:0/2671033313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f59d0 0x5615da304de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.540+0000 7f744de00640 1 --2- 172.21.15.89:0/2671033313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f0700 0x5615da3038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.540+0000 7f744d400640 1 -- 172.21.15.89:0/2671033313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f59d0 msgr2=0x5615da304de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.540+0000 7f744d400640 1 --2- 172.21.15.89:0/2671033313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f59d0 0x5615da304de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.540+0000 7f744d400640 1 -- 172.21.15.89:0/2671033313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f0700 msgr2=0x5615da3038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.540+0000 7f744d400640 1 --2- 172.21.15.89:0/2671033313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f0700 0x5615da3038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.540+0000 7f744d400640 1 -- 172.21.15.89:0/2671033313 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615da3053f0 con 0x5615da2f5090 2024-09-10T08:47:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.542+0000 7f744d400640 1 --2- 172.21.15.89:0/2671033313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2f5090 0x5615da303e20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7438000c00 tx=0x7f7438004370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0066309868a352d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.542+0000 7f742b400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:47:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.543+0000 7f742b400640 1 -- 172.21.15.89:0/2671033313 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438053060 con 0x5615da2f5090 2024-09-10T08:47:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.543+0000 7f7453642740 1 -- 172.21.15.89:0/2671033313 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7438005050 con 0x5615da2f5090 2024-09-10T08:47:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.543+0000 7f742b400640 1 -- 172.21.15.89:0/2671033313 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f7438005410 con 0x5615da2f5090 2024-09-10T08:47:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.543+0000 7f742b400640 1 -- 172.21.15.89:0/2671033313 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f743805a050 con 0x5615da2f5090 2024-09-10T08:47:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.543+0000 7f7453642740 10 client.26912 fetch_fsmap learned FSMap version 31 2024-09-10T08:47:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.543+0000 7f7453642740 1 -- 172.21.15.89:0/2671033313 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5615da306700 con 0x5615da2f5090 2024-09-10T08:47:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.544+0000 7f742b400640 1 -- 172.21.15.89:0/2671033313 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7438005050 con 0x5615da2f5090 2024-09-10T08:47:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.544+0000 7f742b400640 1 -- 172.21.15.89:0/2671033313 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f7438031bb0 con 0x5615da2f5090 2024-09-10T08:47:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.544+0000 7f7453642740 10 client.26912 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:47:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.544+0000 7f7453642740 10 client.26912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.544+0000 7f7453642740 20 client.26912 populate_metadata read hostname 'smithi089' 2024-09-10T08:47:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.544+0000 7f7453642740 1 --2- 172.21.15.89:0/2671033313 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5615da2faa60 0x5615da377a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.544+0000 7f7453642740 4 client.26912 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:47:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.544+0000 7f7453642740 1 -- 172.21.15.89:0/2671033313 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5615da261b80 con 0x5615da2faa60 2024-09-10T08:47:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.545+0000 7f744de00640 1 --2- 172.21.15.89:0/2671033313 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5615da2faa60 0x5615da377a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.545+0000 7f742b400640 10 client.26912 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.545+0000 7f744de00640 1 --2- 172.21.15.89:0/2671033313 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5615da2faa60 0x5615da377a20 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a0911eccacce8df server_cookie=2d9b117688c361d5 in_seq=0 out_seq=0 2024-09-10T08:47:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.848+0000 7f742b400640 1 -- 172.21.15.89:0/2671033313 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5615da261b80 con 0x5615da2faa60 2024-09-10T08:47:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.848+0000 7f742b400640 10 client.26912 handle_command_reply: tid=0 2024-09-10T08:47:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.848+0000 7f7453642740 1 client.26912 shutdown 2024-09-10T08:47:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.848+0000 7f7453642740 1 -- 172.21.15.89:0/2671033313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2f5090 msgr2=0x5615da303e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.848+0000 7f7453642740 1 --2- 172.21.15.89:0/2671033313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2f5090 0x5615da303e20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7438000c00 tx=0x7f7438004370 comp rx=0 tx=0).stop 2024-09-10T08:47:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.848+0000 7f7453642740 1 -- 172.21.15.89:0/2671033313 shutdown_connections 2024-09-10T08:47:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.848+0000 7f7453642740 1 --2- 172.21.15.89:0/2671033313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615da2f59d0 0x5615da304de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.848+0000 7f7453642740 1 --2- 172.21.15.89:0/2671033313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615da2f5090 0x5615da303e20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.848+0000 7f7453642740 1 --2- 172.21.15.89:0/2671033313 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5615da2faa60 0x5615da377a20 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.848+0000 7f7453642740 1 --2- 172.21.15.89:0/2671033313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5615da2f0700 0x5615da3038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.848+0000 7f7453642740 1 -- 172.21.15.89:0/2671033313 >> 172.21.15.89:0/2671033313 conn(0x5615da2fa5f0 msgr2=0x5615da2ef060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.848+0000 7f742b400640 0 client.26912 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:47:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.848+0000 7f7453642740 1 -- 172.21.15.89:0/2671033313 shutdown_connections 2024-09-10T08:47:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.848+0000 7f7453642740 1 -- 172.21.15.89:0/2671033313 wait complete. 2024-09-10T08:47:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.848+0000 7f7453642740 20 client.26912 trim_cache size 0 max 16384 2024-09-10T08:47:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:47:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:47:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:47:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:47:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.849+0000 7f7450800640 1 -- 172.21.15.89:0/1452829993 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f74300a2e20 msgr2=0x7f74300a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.849+0000 7f7450800640 1 --2- 172.21.15.89:0/1452829993 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f74300a2e20 0x7f74300a52e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7444005fd0 tx=0x7f744400b560 comp rx=0 tx=0).stop 2024-09-10T08:47:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.849+0000 7f7450800640 1 -- 172.21.15.89:0/1452829993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7448072d20 msgr2=0x7f74481a6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.849+0000 7f7450800640 1 --2- 172.21.15.89:0/1452829993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7448072d20 0x7f74481a6b70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f743c00ebc0 tx=0x7f743c00c6a0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.849+0000 7f7450800640 1 -- 172.21.15.89:0/1452829993 shutdown_connections 2024-09-10T08:47:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.849+0000 7f7450800640 1 --2- 172.21.15.89:0/1452829993 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f74300a2e20 0x7f74300a52e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.849+0000 7f7450800640 1 --2- 172.21.15.89:0/1452829993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7448109930 0x7f74481ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.849+0000 7f7450800640 1 --2- 172.21.15.89:0/1452829993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74480737d0 0x7f74481a70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.849+0000 7f7450800640 1 --2- 172.21.15.89:0/1452829993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7448072d20 0x7f74481a6b70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.849+0000 7f7450800640 1 -- 172.21.15.89:0/1452829993 >> 172.21.15.89:0/1452829993 conn(0x7f744806ece0 msgr2=0x7f744810c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.850+0000 7f7450800640 1 -- 172.21.15.89:0/1452829993 shutdown_connections 2024-09-10T08:47:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:47:53.850+0000 7f7450800640 1 -- 172.21.15.89:0/1452829993 wait complete. 2024-09-10T08:47:55.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:54 smithi149 ceph-mon[36167]: pgmap v632: 129 pgs: 129 active+clean; 406 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 234 KiB/s wr, 51 op/s 2024-09-10T08:47:55.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:54 smithi111 ceph-mon[35220]: pgmap v632: 129 pgs: 129 active+clean; 406 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 234 KiB/s wr, 51 op/s 2024-09-10T08:47:55.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:54 smithi089 ceph-mon[30757]: pgmap v632: 129 pgs: 129 active+clean; 406 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 234 KiB/s wr, 51 op/s 2024-09-10T08:47:57.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:56 smithi149 ceph-mon[36167]: pgmap v633: 129 pgs: 129 active+clean; 405 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 233 KiB/s wr, 74 op/s 2024-09-10T08:47:57.297 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:56 smithi111 ceph-mon[35220]: pgmap v633: 129 pgs: 129 active+clean; 405 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 233 KiB/s wr, 74 op/s 2024-09-10T08:47:57.305 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:56 smithi089 ceph-mon[30757]: pgmap v633: 129 pgs: 129 active+clean; 405 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 233 KiB/s wr, 74 op/s 2024-09-10T08:47:59.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:47:58 smithi149 ceph-mon[36167]: pgmap v634: 129 pgs: 129 active+clean; 369 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 733 KiB/s wr, 413 op/s 2024-09-10T08:47:59.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:47:58 smithi111 ceph-mon[35220]: pgmap v634: 129 pgs: 129 active+clean; 369 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 733 KiB/s wr, 413 op/s 2024-09-10T08:47:59.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:47:58 smithi089 ceph-mon[30757]: pgmap v634: 129 pgs: 129 active+clean; 369 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 733 KiB/s wr, 413 op/s 2024-09-10T08:48:01.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:00 smithi149 ceph-mon[36167]: pgmap v635: 129 pgs: 129 active+clean; 367 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 732 KiB/s wr, 441 op/s 2024-09-10T08:48:01.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:00 smithi111 ceph-mon[35220]: pgmap v635: 129 pgs: 129 active+clean; 367 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 732 KiB/s wr, 441 op/s 2024-09-10T08:48:01.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:00 smithi089 ceph-mon[30757]: pgmap v635: 129 pgs: 129 active+clean; 367 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 732 KiB/s wr, 441 op/s 2024-09-10T08:48:03.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:02 smithi149 ceph-mon[36167]: pgmap v636: 129 pgs: 129 active+clean; 355 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 583 op/s 2024-09-10T08:48:03.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:02 smithi111 ceph-mon[35220]: pgmap v636: 129 pgs: 129 active+clean; 355 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 583 op/s 2024-09-10T08:48:03.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:02 smithi089 ceph-mon[30757]: pgmap v636: 129 pgs: 129 active+clean; 355 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 583 op/s 2024-09-10T08:48:05.168 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:04 smithi089 ceph-mon[30757]: pgmap v637: 129 pgs: 129 active+clean; 349 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 675 op/s 2024-09-10T08:48:05.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:04 smithi149 ceph-mon[36167]: pgmap v637: 129 pgs: 129 active+clean; 349 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 675 op/s 2024-09-10T08:48:05.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:04 smithi111 ceph-mon[35220]: pgmap v637: 129 pgs: 129 active+clean; 349 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 675 op/s 2024-09-10T08:48:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.967+0000 7f7a2ba00640 1 -- 172.21.15.89:0/5470363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a24103710 msgr2=0x7f7a24109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.967+0000 7f7a2ba00640 1 --2- 172.21.15.89:0/5470363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a24103710 0x7f7a24109fa0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009be0 tx=0x7f7a1c02a6c0 comp rx=0 tx=0).stop 2024-09-10T08:48:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.967+0000 7f7a2ba00640 1 -- 172.21.15.89:0/5470363 shutdown_connections 2024-09-10T08:48:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.967+0000 7f7a2ba00640 1 --2- 172.21.15.89:0/5470363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a24103710 0x7f7a24109fa0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.967+0000 7f7a2ba00640 1 --2- 172.21.15.89:0/5470363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a24102d50 0x7f7a241031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.967+0000 7f7a2ba00640 1 --2- 172.21.15.89:0/5470363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a24101b60 0x7f7a24101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.967+0000 7f7a2ba00640 1 -- 172.21.15.89:0/5470363 >> 172.21.15.89:0/5470363 conn(0x7f7a240fd2e0 msgr2=0x7f7a240ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.967+0000 7f7a2ba00640 1 -- 172.21.15.89:0/5470363 shutdown_connections 2024-09-10T08:48:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.967+0000 7f7a2ba00640 1 -- 172.21.15.89:0/5470363 wait complete. 2024-09-10T08:48:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.968+0000 7f7a2ba00640 1 Processor -- start 2024-09-10T08:48:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.968+0000 7f7a2ba00640 1 -- start start 2024-09-10T08:48:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.969+0000 7f7a2ba00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a24101b60 0x7f7a24199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.969+0000 7f7a2ba00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a24102d50 0x7f7a2419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.969+0000 7f7a2ba00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a24103710 0x7f7a241a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.969+0000 7f7a2ba00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2410d7c0 con 0x7f7a24103710 2024-09-10T08:48:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.969+0000 7f7a2ba00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2410d640 con 0x7f7a24102d50 2024-09-10T08:48:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.969+0000 7f7a2ba00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2410d940 con 0x7f7a24101b60 2024-09-10T08:48:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.969+0000 7f7a23e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a24102d50 0x7f7a2419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.969+0000 7f7a23e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a24102d50 0x7f7a2419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:32972/0 (socket says 172.21.15.89:32972) 2024-09-10T08:48:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.969+0000 7f7a23e00640 1 -- 172.21.15.89:0/1233234585 learned_addr learned my addr 172.21.15.89:0/1233234585 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.969+0000 7f7a29a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a24103710 0x7f7a241a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.969+0000 7f7a29000640 1 --2- 172.21.15.89:0/1233234585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a24101b60 0x7f7a24199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.969+0000 7f7a23e00640 1 -- 172.21.15.89:0/1233234585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a24101b60 msgr2=0x7f7a24199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.969+0000 7f7a23e00640 1 --2- 172.21.15.89:0/1233234585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a24101b60 0x7f7a24199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.969+0000 7f7a23e00640 1 -- 172.21.15.89:0/1233234585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a24103710 msgr2=0x7f7a241a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.969+0000 7f7a23e00640 1 --2- 172.21.15.89:0/1233234585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a24103710 0x7f7a241a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.969+0000 7f7a23e00640 1 -- 172.21.15.89:0/1233234585 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a241a1ba0 con 0x7f7a24102d50 2024-09-10T08:48:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.970+0000 7f7a23e00640 1 --2- 172.21.15.89:0/1233234585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a24102d50 0x7f7a2419a4c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000d950 tx=0x7f7a1000dd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.970+0000 7f7a21600640 1 -- 172.21.15.89:0/1233234585 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10017030 con 0x7f7a24102d50 2024-09-10T08:48:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.970+0000 7f7a21600640 1 -- 172.21.15.89:0/1233234585 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a10004020 con 0x7f7a24102d50 2024-09-10T08:48:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.970+0000 7f7a2ba00640 1 -- 172.21.15.89:0/1233234585 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a241a1e90 con 0x7f7a24102d50 2024-09-10T08:48:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.970+0000 7f7a21600640 1 -- 172.21.15.89:0/1233234585 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10005650 con 0x7f7a24102d50 2024-09-10T08:48:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.970+0000 7f7a2ba00640 1 -- 172.21.15.89:0/1233234585 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a241a2350 con 0x7f7a24102d50 2024-09-10T08:48:06.974 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.973+0000 7f7a2e88d740 1 Processor -- start 2024-09-10T08:48:06.974 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.973+0000 7f7a2e88d740 1 -- start start 2024-09-10T08:48:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.973+0000 7f7a21600640 1 -- 172.21.15.89:0/1233234585 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7a100041d0 con 0x7f7a24102d50 2024-09-10T08:48:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.973+0000 7f7a2e88d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e3090 0x55aaf63e3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.973+0000 7f7a2e88d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e39d0 0x55aaf63e8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.973+0000 7f7a2e88d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e8a60 0x55aaf63eaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.973+0000 7f7a2e88d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55aaf6348100 con 0x55aaf63e3090 2024-09-10T08:48:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.973+0000 7f7a2e88d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55aaf62f01b0 con 0x55aaf63e39d0 2024-09-10T08:48:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.973+0000 7f7a2e88d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55aaf63440c0 con 0x55aaf63e8a60 2024-09-10T08:48:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.973+0000 7f7a29000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e39d0 0x55aaf63e8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.974+0000 7f7a29a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e3090 0x55aaf63e3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.974+0000 7f7a29000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e39d0 0x55aaf63e8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:32988/0 (socket says 172.21.15.89:32988) 2024-09-10T08:48:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.974+0000 7f7a29000640 1 -- 172.21.15.89:0/1147018235 learned_addr learned my addr 172.21.15.89:0/1147018235 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.974+0000 7f7a23e00640 1 --2- 172.21.15.89:0/1147018235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e8a60 0x55aaf63eaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.974+0000 7f7a29a00640 1 -- 172.21.15.89:0/1147018235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e8a60 msgr2=0x55aaf63eaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.974+0000 7f7a29a00640 1 --2- 172.21.15.89:0/1147018235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e8a60 0x55aaf63eaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.974+0000 7f7a29a00640 1 -- 172.21.15.89:0/1147018235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e39d0 msgr2=0x55aaf63e8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.974+0000 7f7a29a00640 1 --2- 172.21.15.89:0/1147018235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e39d0 0x55aaf63e8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.974+0000 7f7a29a00640 1 -- 172.21.15.89:0/1147018235 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaf605b210 con 0x55aaf63e3090 2024-09-10T08:48:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.974+0000 7f7a29000640 1 --2- 172.21.15.89:0/1147018235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e39d0 0x55aaf63e8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.974+0000 7f7a29a00640 1 --2- 172.21.15.89:0/1147018235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e3090 0x55aaf63e3490 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c034040 tx=0x7f7a1c005eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f3d846162b1d09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.975+0000 7f7a16000640 1 -- 172.21.15.89:0/1147018235 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c041030 con 0x55aaf63e3090 2024-09-10T08:48:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.975+0000 7f7a16000640 1 -- 172.21.15.89:0/1147018235 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a1c030c70 con 0x55aaf63e3090 2024-09-10T08:48:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.975+0000 7f7a21600640 1 --2- 172.21.15.89:0/1233234585 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7a0c0a2c30 0x7f7a0c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.975+0000 7f7a16000640 1 -- 172.21.15.89:0/1147018235 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c036040 con 0x55aaf63e3090 2024-09-10T08:48:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.975+0000 7f7a21600640 1 -- 172.21.15.89:0/1233234585 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f7a100cede0 con 0x7f7a24102d50 2024-09-10T08:48:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.975+0000 7f7a29000640 1 --2- 172.21.15.89:0/1233234585 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7a0c0a2c30 0x7f7a0c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.975+0000 7f7a2e88d740 1 -- 172.21.15.89:0/1147018235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e3090 msgr2=0x55aaf63e3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.975+0000 7f7a2e88d740 1 --2- 172.21.15.89:0/1147018235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e3090 0x55aaf63e3490 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c034040 tx=0x7f7a1c005eb0 comp rx=0 tx=0).stop 2024-09-10T08:48:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.975+0000 7f7a2e88d740 1 -- 172.21.15.89:0/1147018235 shutdown_connections 2024-09-10T08:48:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.975+0000 7f7a2e88d740 1 --2- 172.21.15.89:0/1147018235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e8a60 0x55aaf63eaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.975+0000 7f7a2e88d740 1 --2- 172.21.15.89:0/1147018235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e39d0 0x55aaf63e8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.975+0000 7f7a2e88d740 1 --2- 172.21.15.89:0/1147018235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e3090 0x55aaf63e3490 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.975+0000 7f7a2e88d740 1 -- 172.21.15.89:0/1147018235 >> 172.21.15.89:0/1147018235 conn(0x55aaf63de700 msgr2=0x55aaf63deb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.976+0000 7f7a2e88d740 1 -- 172.21.15.89:0/1147018235 shutdown_connections 2024-09-10T08:48:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.976+0000 7f7a2e88d740 1 -- 172.21.15.89:0/1147018235 wait complete. 2024-09-10T08:48:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.976+0000 7f7a29000640 1 --2- 172.21.15.89:0/1233234585 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7a0c0a2c30 0x7f7a0c0a50f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55aaf63e4400 tx=0x7f7a18009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.976+0000 7f7a2e88d740 1 -- start start 2024-09-10T08:48:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.976+0000 7f7a2e88d740 1 Processor -- start 2024-09-10T08:48:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.977+0000 7f7a2e88d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e3090 0x55aaf63ed280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.977+0000 7f7a2e88d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e39d0 0x55aaf63ed7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.977+0000 7f7a2e88d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e8a60 0x55aaf63f4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.977+0000 7f7a2e88d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55aaf63eb9d0 con 0x55aaf63e39d0 2024-09-10T08:48:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.977+0000 7f7a2e88d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55aaf62d1090 con 0x55aaf63e8a60 2024-09-10T08:48:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.977+0000 7f7a2e88d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55aaf63ebb50 con 0x55aaf63e3090 2024-09-10T08:48:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.977+0000 7f7a2e88d740 10 client.0 _ll_register_callbacks cb 0x55aaf6325f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.977+0000 7f7a29a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e8a60 0x55aaf63f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.977+0000 7f7a23e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e39d0 0x55aaf63ed7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.977+0000 7f7a29a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e8a60 0x55aaf63f4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:32994/0 (socket says 172.21.15.89:32994) 2024-09-10T08:48:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.977+0000 7f7a29a00640 1 -- 172.21.15.89:0/521251678 learned_addr learned my addr 172.21.15.89:0/521251678 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.977+0000 7f7a29000640 1 --2- 172.21.15.89:0/521251678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e3090 0x55aaf63ed280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.977+0000 7f7a29a00640 1 -- 172.21.15.89:0/521251678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e3090 msgr2=0x55aaf63ed280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.977+0000 7f7a29a00640 1 --2- 172.21.15.89:0/521251678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e3090 0x55aaf63ed280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.977+0000 7f7a29a00640 1 -- 172.21.15.89:0/521251678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e39d0 msgr2=0x55aaf63ed7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.977+0000 7f7a29a00640 1 --2- 172.21.15.89:0/521251678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e39d0 0x55aaf63ed7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.977+0000 7f7a29a00640 1 -- 172.21.15.89:0/521251678 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaf6127620 con 0x55aaf63e8a60 2024-09-10T08:48:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.977+0000 7f7a29000640 1 --2- 172.21.15.89:0/521251678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e3090 0x55aaf63ed280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.977+0000 7f7a23e00640 1 --2- 172.21.15.89:0/521251678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e39d0 0x55aaf63ed7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.978+0000 7f7a29a00640 1 --2- 172.21.15.89:0/521251678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e8a60 0x55aaf63f4800 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009d00 tx=0x7f7a1c030e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=960b05d4032d90bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.978+0000 7f7a0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:48:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.978+0000 7f7a0aa00640 1 -- 172.21.15.89:0/521251678 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c036050 con 0x55aaf63e8a60 2024-09-10T08:48:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.978+0000 7f7a2e88d740 1 -- 172.21.15.89:0/521251678 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a1c030c70 con 0x55aaf63e8a60 2024-09-10T08:48:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.978+0000 7f7a0aa00640 1 -- 172.21.15.89:0/521251678 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f7a1c0303a0 con 0x55aaf63e8a60 2024-09-10T08:48:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.978+0000 7f7a0aa00640 1 -- 172.21.15.89:0/521251678 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7a1c041040 con 0x55aaf63e8a60 2024-09-10T08:48:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.978+0000 7f7a2e88d740 10 client.26924 fetch_fsmap learned FSMap version 31 2024-09-10T08:48:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.978+0000 7f7a2e88d740 1 -- 172.21.15.89:0/521251678 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55aaf63f75f0 con 0x55aaf63e8a60 2024-09-10T08:48:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.978+0000 7f7a0aa00640 1 -- 172.21.15.89:0/521251678 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a1c030c70 con 0x55aaf63e8a60 2024-09-10T08:48:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.979+0000 7f7a0aa00640 1 -- 172.21.15.89:0/521251678 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f7a1c03d460 con 0x55aaf63e8a60 2024-09-10T08:48:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.979+0000 7f7a2e88d740 10 client.26924 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:48:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.979+0000 7f7a2e88d740 10 client.26924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.979+0000 7f7a2e88d740 20 client.26924 populate_metadata read hostname 'smithi089' 2024-09-10T08:48:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.979+0000 7f7a2e88d740 1 --2- 172.21.15.89:0/521251678 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55aaf63f8940 0x55aaf63fad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.979+0000 7f7a2e88d740 4 client.26924 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.979+0000 7f7a2e88d740 1 -- 172.21.15.89:0/521251678 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaf6355830 con 0x55aaf63f8940 2024-09-10T08:48:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.979+0000 7f7a29000640 1 --2- 172.21.15.89:0/521251678 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55aaf63f8940 0x55aaf63fad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.980+0000 7f7a29000640 1 --2- 172.21.15.89:0/521251678 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55aaf63f8940 0x55aaf63fad50 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa58f46262c8dee7 server_cookie=c71056a767141f99 in_seq=0 out_seq=0 2024-09-10T08:48:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.980+0000 7f7a0aa00640 10 client.26924 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:48:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.981+0000 7f7a0aa00640 1 -- 172.21.15.89:0/521251678 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aaf6355830 con 0x55aaf63f8940 2024-09-10T08:48:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.981+0000 7f7a0aa00640 10 client.26924 handle_command_reply: tid=0 2024-09-10T08:48:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.981+0000 7f7a2e88d740 1 client.26924 shutdown 2024-09-10T08:48:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.981+0000 7f7a2e88d740 1 -- 172.21.15.89:0/521251678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e8a60 msgr2=0x55aaf63f4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.981+0000 7f7a2e88d740 1 --2- 172.21.15.89:0/521251678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e8a60 0x55aaf63f4800 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009d00 tx=0x7f7a1c030e20 comp rx=0 tx=0).stop 2024-09-10T08:48:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.981+0000 7f7a2e88d740 1 -- 172.21.15.89:0/521251678 shutdown_connections 2024-09-10T08:48:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.981+0000 7f7a2e88d740 1 --2- 172.21.15.89:0/521251678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e8a60 0x55aaf63f4800 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.981+0000 7f7a2e88d740 1 --2- 172.21.15.89:0/521251678 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55aaf63f8940 0x55aaf63fad50 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.981+0000 7f7a2e88d740 1 --2- 172.21.15.89:0/521251678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e39d0 0x55aaf63ed7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.981+0000 7f7a2e88d740 1 --2- 172.21.15.89:0/521251678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e3090 0x55aaf63ed280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.981+0000 7f7a0aa00640 0 client.26924 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:48:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.981+0000 7f7a2e88d740 1 -- 172.21.15.89:0/521251678 >> 172.21.15.89:0/521251678 conn(0x55aaf63de700 msgr2=0x55aaf63dff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.981+0000 7f7a2e88d740 1 -- 172.21.15.89:0/521251678 shutdown_connections 2024-09-10T08:48:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.981+0000 7f7a2e88d740 1 -- 172.21.15.89:0/521251678 wait complete. 2024-09-10T08:48:06.987 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.981+0000 7f7a2e88d740 20 client.26924 trim_cache size 0 max 16384 2024-09-10T08:48:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.996+0000 7f7a2e88d740 1 Processor -- start 2024-09-10T08:48:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.997+0000 7f7a2e88d740 1 -- start start 2024-09-10T08:48:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.997+0000 7f7a2e88d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e3090 0x55aaf63df5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.997+0000 7f7a2e88d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e39d0 0x55aaf63f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.997+0000 7f7a2e88d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e8a60 0x55aaf63dfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.997+0000 7f7a2e88d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241af190 con 0x55aaf63e39d0 2024-09-10T08:48:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.997+0000 7f7a2e88d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241aca30 con 0x55aaf63e8a60 2024-09-10T08:48:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.997+0000 7f7a2e88d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241aa880 con 0x55aaf63e3090 2024-09-10T08:48:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.997+0000 7f7a29a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e8a60 0x55aaf63dfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.997+0000 7f7a23e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e39d0 0x55aaf63f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.997+0000 7f7a29000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e3090 0x55aaf63df5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.997+0000 7f7a29000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e3090 0x55aaf63df5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:35624/0 (socket says 172.21.15.89:35624) 2024-09-10T08:48:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.997+0000 7f7a29000640 1 -- 172.21.15.89:0/739349404 learned_addr learned my addr 172.21.15.89:0/739349404 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.997+0000 7f7a29000640 1 -- 172.21.15.89:0/739349404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e8a60 msgr2=0x55aaf63dfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.997+0000 7f7a29000640 1 --2- 172.21.15.89:0/739349404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e8a60 0x55aaf63dfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.997+0000 7f7a29000640 1 -- 172.21.15.89:0/739349404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e39d0 msgr2=0x55aaf63f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.997+0000 7f7a29000640 1 --2- 172.21.15.89:0/739349404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e39d0 0x55aaf63f47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.997+0000 7f7a29000640 1 -- 172.21.15.89:0/739349404 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaf63fb3d0 con 0x55aaf63e3090 2024-09-10T08:48:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.997+0000 7f7a29a00640 1 --2- 172.21.15.89:0/739349404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e8a60 0x55aaf63dfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.997+0000 7f7a23e00640 1 --2- 172.21.15.89:0/739349404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e39d0 0x55aaf63f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.998+0000 7f7a29000640 1 --2- 172.21.15.89:0/739349404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e3090 0x55aaf63df5b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009b50 tx=0x7f7a180046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6e05ec40d5c6905 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.998+0000 7f7a15600640 1 -- 172.21.15.89:0/739349404 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18004480 con 0x55aaf63e3090 2024-09-10T08:48:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.998+0000 7f7a15600640 1 -- 172.21.15.89:0/739349404 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a1802a3b0 con 0x55aaf63e3090 2024-09-10T08:48:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.998+0000 7f7a15600640 1 -- 172.21.15.89:0/739349404 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18003ea0 con 0x55aaf63e3090 2024-09-10T08:48:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.999+0000 7f7a2e88d740 1 -- 172.21.15.89:0/739349404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e3090 msgr2=0x55aaf63df5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.999+0000 7f7a2e88d740 1 --2- 172.21.15.89:0/739349404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e3090 0x55aaf63df5b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009b50 tx=0x7f7a180046c0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.999+0000 7f7a2e88d740 1 -- 172.21.15.89:0/739349404 shutdown_connections 2024-09-10T08:48:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.999+0000 7f7a2e88d740 1 --2- 172.21.15.89:0/739349404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e8a60 0x55aaf63dfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.999+0000 7f7a2e88d740 1 --2- 172.21.15.89:0/739349404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e39d0 0x55aaf63f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.999+0000 7f7a2e88d740 1 --2- 172.21.15.89:0/739349404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e3090 0x55aaf63df5b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.999+0000 7f7a2e88d740 1 -- 172.21.15.89:0/739349404 >> 172.21.15.89:0/739349404 conn(0x55aaf63de700 msgr2=0x55aaf63ed280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.999+0000 7f7a2e88d740 1 -- 172.21.15.89:0/739349404 shutdown_connections 2024-09-10T08:48:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.999+0000 7f7a2e88d740 1 -- 172.21.15.89:0/739349404 wait complete. 2024-09-10T08:48:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.999+0000 7f7a2e88d740 1 -- start start 2024-09-10T08:48:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:06.999+0000 7f7a2e88d740 1 Processor -- start 2024-09-10T08:48:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.000+0000 7f7a2e88d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e3090 0x55aaf63f1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.000+0000 7f7a2e88d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e39d0 0x55aaf63f1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.000+0000 7f7a2e88d740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e8a60 0x55aaf6461620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.000+0000 7f7a2e88d740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241aaef0 con 0x55aaf63e8a60 2024-09-10T08:48:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.000+0000 7f7a2e88d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241ac360 con 0x55aaf63e3090 2024-09-10T08:48:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.000+0000 7f7a2e88d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7a241ab5c0 con 0x55aaf63e39d0 2024-09-10T08:48:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.000+0000 7f7a2e88d740 10 client.0 _ll_register_callbacks cb 0x7f7a180097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.000+0000 7f7a29000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e3090 0x55aaf63f1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.000+0000 7f7a29a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e8a60 0x55aaf6461620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.000+0000 7f7a29a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e8a60 0x55aaf6461620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:33090/0 (socket says 172.21.15.89:33090) 2024-09-10T08:48:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.000+0000 7f7a29a00640 1 -- 172.21.15.89:0/146184733 learned_addr learned my addr 172.21.15.89:0/146184733 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.000+0000 7f7a29000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e3090 0x55aaf63f1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:33010/0 (socket says 172.21.15.89:33010) 2024-09-10T08:48:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.000+0000 7f7a23e00640 1 --2- 172.21.15.89:0/146184733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e39d0 0x55aaf63f1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.000+0000 7f7a29a00640 1 -- 172.21.15.89:0/146184733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e39d0 msgr2=0x55aaf63f1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.000+0000 7f7a29a00640 1 --2- 172.21.15.89:0/146184733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e39d0 0x55aaf63f1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.000+0000 7f7a29a00640 1 -- 172.21.15.89:0/146184733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e3090 msgr2=0x55aaf63f1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.000+0000 7f7a29a00640 1 --2- 172.21.15.89:0/146184733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e3090 0x55aaf63f1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.000+0000 7f7a29a00640 1 -- 172.21.15.89:0/146184733 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaf6461b60 con 0x55aaf63e8a60 2024-09-10T08:48:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.000+0000 7f7a23e00640 1 --2- 172.21.15.89:0/146184733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e39d0 0x55aaf63f1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.000+0000 7f7a29000640 1 --2- 172.21.15.89:0/146184733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e3090 0x55aaf63f1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.001+0000 7f7a29a00640 1 --2- 172.21.15.89:0/146184733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e8a60 0x55aaf6461620 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c02a700 tx=0x7f7a1c009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9320f44265021d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.001+0000 7f7a0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-10T08:48:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.001+0000 7f7a0aa00640 1 -- 172.21.15.89:0/146184733 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c036030 con 0x55aaf63e8a60 2024-09-10T08:48:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.001+0000 7f7a0aa00640 1 -- 172.21.15.89:0/146184733 <== mon.0 v2:172.21.15.89:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f7a1c003c30 con 0x55aaf63e8a60 2024-09-10T08:48:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.001+0000 7f7a2e88d740 1 -- 172.21.15.89:0/146184733 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a1802a3b0 con 0x55aaf63e8a60 2024-09-10T08:48:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.001+0000 7f7a0aa00640 1 -- 172.21.15.89:0/146184733 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7a1c009800 con 0x55aaf63e8a60 2024-09-10T08:48:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.001+0000 7f7a2e88d740 10 client.16860 fetch_fsmap learned FSMap version 31 2024-09-10T08:48:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.001+0000 7f7a2e88d740 1 -- 172.21.15.89:0/146184733 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55aaf6462ea0 con 0x55aaf63e8a60 2024-09-10T08:48:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.001+0000 7f7a0aa00640 1 -- 172.21.15.89:0/146184733 <== mon.0 v2:172.21.15.89:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a1802a3b0 con 0x55aaf63e8a60 2024-09-10T08:48:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.002+0000 7f7a0aa00640 1 -- 172.21.15.89:0/146184733 <== mon.0 v2:172.21.15.89:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f7a1c064030 con 0x55aaf63e8a60 2024-09-10T08:48:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.002+0000 7f7a2e88d740 10 client.16860 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:48:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.002+0000 7f7a2e88d740 10 client.16860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.002+0000 7f7a2e88d740 20 client.16860 populate_metadata read hostname 'smithi089' 2024-09-10T08:48:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.002+0000 7f7a2e88d740 1 --2- 172.21.15.89:0/146184733 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55aaf6463710 0x55aaf6465b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.002+0000 7f7a2e88d740 4 client.16860 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:48:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.002+0000 7f7a2e88d740 1 -- 172.21.15.89:0/146184733 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aaf634fb80 con 0x55aaf6463710 2024-09-10T08:48:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.002+0000 7f7a29000640 1 --2- 172.21.15.89:0/146184733 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55aaf6463710 0x55aaf6465b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.003+0000 7f7a29000640 1 --2- 172.21.15.89:0/146184733 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55aaf6463710 0x55aaf6465b30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60246c8cee14f0f9 server_cookie=f2cec55efec875a8 in_seq=0 out_seq=0 2024-09-10T08:48:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.003+0000 7f7a0aa00640 10 client.16860 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:48:07.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:06 smithi149 ceph-mon[36167]: pgmap v638: 129 pgs: 129 active+clean; 348 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 684 op/s 2024-09-10T08:48:07.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:06 smithi111 ceph-mon[35220]: pgmap v638: 129 pgs: 129 active+clean; 348 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 684 op/s 2024-09-10T08:48:07.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:06 smithi089 ceph-mon[30757]: pgmap v638: 129 pgs: 129 active+clean; 348 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 684 op/s 2024-09-10T08:48:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.660+0000 7f7a0aa00640 1 -- 172.21.15.89:0/146184733 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aaf634fb80 con 0x55aaf6463710 2024-09-10T08:48:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.660+0000 7f7a0aa00640 10 client.16860 handle_command_reply: tid=0 2024-09-10T08:48:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.660+0000 7f7a2e88d740 1 client.16860 shutdown 2024-09-10T08:48:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.660+0000 7f7a2e88d740 1 -- 172.21.15.89:0/146184733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e8a60 msgr2=0x55aaf6461620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.660+0000 7f7a2e88d740 1 --2- 172.21.15.89:0/146184733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e8a60 0x55aaf6461620 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c02a700 tx=0x7f7a1c009ba0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.661+0000 7f7a2e88d740 1 -- 172.21.15.89:0/146184733 shutdown_connections 2024-09-10T08:48:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.661+0000 7f7a2e88d740 1 --2- 172.21.15.89:0/146184733 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55aaf6463710 0x55aaf6465b30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.661+0000 7f7a2e88d740 1 --2- 172.21.15.89:0/146184733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aaf63e8a60 0x55aaf6461620 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.661+0000 7f7a2e88d740 1 --2- 172.21.15.89:0/146184733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aaf63e39d0 0x55aaf63f1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.661+0000 7f7a2e88d740 1 --2- 172.21.15.89:0/146184733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55aaf63e3090 0x55aaf63f1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.661+0000 7f7a2e88d740 1 -- 172.21.15.89:0/146184733 >> 172.21.15.89:0/146184733 conn(0x55aaf63de700 msgr2=0x55aaf63dd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.661+0000 7f7a0aa00640 0 client.16860 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:48:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.661+0000 7f7a2e88d740 1 -- 172.21.15.89:0/146184733 shutdown_connections 2024-09-10T08:48:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.661+0000 7f7a2e88d740 1 -- 172.21.15.89:0/146184733 wait complete. 2024-09-10T08:48:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.662+0000 7f7a2e88d740 20 client.16860 trim_cache size 0 max 16384 2024-09-10T08:48:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:48:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:48:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:48:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:48:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.663+0000 7f7a2ba00640 1 -- 172.21.15.89:0/1233234585 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7a0c0a2c30 msgr2=0x7f7a0c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.663+0000 7f7a2ba00640 1 --2- 172.21.15.89:0/1233234585 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7a0c0a2c30 0x7f7a0c0a50f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55aaf63e4400 tx=0x7f7a18009260 comp rx=0 tx=0).stop 2024-09-10T08:48:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.663+0000 7f7a2ba00640 1 -- 172.21.15.89:0/1233234585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a24102d50 msgr2=0x7f7a2419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.663+0000 7f7a2ba00640 1 --2- 172.21.15.89:0/1233234585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a24102d50 0x7f7a2419a4c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000d950 tx=0x7f7a1000dd50 comp rx=0 tx=0).stop 2024-09-10T08:48:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.665+0000 7f7a2ba00640 1 -- 172.21.15.89:0/1233234585 shutdown_connections 2024-09-10T08:48:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.665+0000 7f7a2ba00640 1 --2- 172.21.15.89:0/1233234585 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f7a0c0a2c30 0x7f7a0c0a50f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.665+0000 7f7a2ba00640 1 --2- 172.21.15.89:0/1233234585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7a24103710 0x7f7a241a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.665+0000 7f7a2ba00640 1 --2- 172.21.15.89:0/1233234585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a24102d50 0x7f7a2419a4c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.665+0000 7f7a2ba00640 1 --2- 172.21.15.89:0/1233234585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a24101b60 0x7f7a24199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.665+0000 7f7a2ba00640 1 -- 172.21.15.89:0/1233234585 >> 172.21.15.89:0/1233234585 conn(0x7f7a240fd2e0 msgr2=0x7f7a24074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.665+0000 7f7a2ba00640 1 -- 172.21.15.89:0/1233234585 shutdown_connections 2024-09-10T08:48:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:07.665+0000 7f7a2ba00640 1 -- 172.21.15.89:0/1233234585 wait complete. 2024-09-10T08:48:09.228 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:08 smithi149 ceph-mon[36167]: pgmap v639: 129 pgs: 129 active+clean; 322 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 870 op/s 2024-09-10T08:48:09.301 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:08 smithi111 ceph-mon[35220]: pgmap v639: 129 pgs: 129 active+clean; 322 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 870 op/s 2024-09-10T08:48:09.306 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:08 smithi089 ceph-mon[30757]: pgmap v639: 129 pgs: 129 active+clean; 322 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 870 op/s 2024-09-10T08:48:11.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:10 smithi149 ceph-mon[36167]: pgmap v640: 129 pgs: 129 active+clean; 324 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 562 op/s 2024-09-10T08:48:11.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:10 smithi111 ceph-mon[35220]: pgmap v640: 129 pgs: 129 active+clean; 324 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 562 op/s 2024-09-10T08:48:11.304 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:10 smithi089 ceph-mon[30757]: pgmap v640: 129 pgs: 129 active+clean; 324 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 562 op/s 2024-09-10T08:48:13.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:12 smithi149 ceph-mon[36167]: pgmap v641: 129 pgs: 129 active+clean; 297 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 710 op/s 2024-09-10T08:48:13.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:12 smithi111 ceph-mon[35220]: pgmap v641: 129 pgs: 129 active+clean; 297 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 710 op/s 2024-09-10T08:48:13.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:12 smithi089 ceph-mon[30757]: pgmap v641: 129 pgs: 129 active+clean; 297 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 710 op/s 2024-09-10T08:48:15.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:14 smithi149 ceph-mon[36167]: pgmap v642: 129 pgs: 129 active+clean; 287 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 686 op/s 2024-09-10T08:48:15.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:14 smithi111 ceph-mon[35220]: pgmap v642: 129 pgs: 129 active+clean; 287 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 686 op/s 2024-09-10T08:48:15.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:14 smithi089 ceph-mon[30757]: pgmap v642: 129 pgs: 129 active+clean; 287 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 686 op/s 2024-09-10T08:48:17.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:16 smithi149 ceph-mon[36167]: pgmap v643: 129 pgs: 129 active+clean; 290 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 621 op/s 2024-09-10T08:48:17.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:16 smithi111 ceph-mon[35220]: pgmap v643: 129 pgs: 129 active+clean; 290 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 621 op/s 2024-09-10T08:48:17.304 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:16 smithi089 ceph-mon[30757]: pgmap v643: 129 pgs: 129 active+clean; 290 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 621 op/s 2024-09-10T08:48:19.195 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:18 smithi089 ceph-mon[30757]: pgmap v644: 129 pgs: 129 active+clean; 265 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 797 op/s 2024-09-10T08:48:19.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:18 smithi149 ceph-mon[36167]: pgmap v644: 129 pgs: 129 active+clean; 265 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 797 op/s 2024-09-10T08:48:19.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:18 smithi111 ceph-mon[35220]: pgmap v644: 129 pgs: 129 active+clean; 265 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 797 op/s 2024-09-10T08:48:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.826+0000 7fc711200640 1 -- 172.21.15.89:0/1914594460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc70c1054b0 msgr2=0x7fc70c105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.826+0000 7fc711200640 1 --2- 172.21.15.89:0/1914594460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc70c1054b0 0x7fc70c105960 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc704009ca0 tx=0x7fc70402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.827+0000 7fc711200640 1 -- 172.21.15.89:0/1914594460 shutdown_connections 2024-09-10T08:48:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.827+0000 7fc711200640 1 --2- 172.21.15.89:0/1914594460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc70c1054b0 0x7fc70c105960 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.827+0000 7fc711200640 1 --2- 172.21.15.89:0/1914594460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc70c102b00 0x7fc70c104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.827+0000 7fc711200640 1 --2- 172.21.15.89:0/1914594460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc70c1001b0 0x7fc70c1025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.827+0000 7fc711200640 1 -- 172.21.15.89:0/1914594460 >> 172.21.15.89:0/1914594460 conn(0x7fc70c0fbfa0 msgr2=0x7fc70c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.827+0000 7fc711200640 1 -- 172.21.15.89:0/1914594460 shutdown_connections 2024-09-10T08:48:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.827+0000 7fc711200640 1 -- 172.21.15.89:0/1914594460 wait complete. 2024-09-10T08:48:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.827+0000 7fc711200640 1 Processor -- start 2024-09-10T08:48:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.828+0000 7fc711200640 1 -- start start 2024-09-10T08:48:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.828+0000 7fc711200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc70c1001b0 0x7fc70c19e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.828+0000 7fc711200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc70c102b00 0x7fc70c19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.828+0000 7fc711200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc70c1054b0 0x7fc70c1a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.828+0000 7fc711200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c113c10 con 0x7fc70c102b00 2024-09-10T08:48:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.828+0000 7fc711200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c113a90 con 0x7fc70c1001b0 2024-09-10T08:48:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.828+0000 7fc711200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c113d90 con 0x7fc70c1054b0 2024-09-10T08:48:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.828+0000 7fc70b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc70c1001b0 0x7fc70c19e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.828+0000 7fc70b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc70c1001b0 0x7fc70c19e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57660/0 (socket says 172.21.15.89:57660) 2024-09-10T08:48:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.828+0000 7fc70b400640 1 -- 172.21.15.89:0/2635682633 learned_addr learned my addr 172.21.15.89:0/2635682633 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.828+0000 7fc70aa00640 1 --2- 172.21.15.89:0/2635682633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc70c102b00 0x7fc70c19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.828+0000 7fc70be00640 1 --2- 172.21.15.89:0/2635682633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc70c1054b0 0x7fc70c1a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.828+0000 7fc70b400640 1 -- 172.21.15.89:0/2635682633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc70c1054b0 msgr2=0x7fc70c1a5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.828+0000 7fc70b400640 1 --2- 172.21.15.89:0/2635682633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc70c1054b0 0x7fc70c1a5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.828+0000 7fc70b400640 1 -- 172.21.15.89:0/2635682633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc70c102b00 msgr2=0x7fc70c19e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.829+0000 7fc70b400640 1 --2- 172.21.15.89:0/2635682633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc70c102b00 0x7fc70c19e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.829+0000 7fc70b400640 1 -- 172.21.15.89:0/2635682633 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c1a5e90 con 0x7fc70c1001b0 2024-09-10T08:48:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.829+0000 7fc70be00640 1 --2- 172.21.15.89:0/2635682633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc70c1054b0 0x7fc70c1a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.829+0000 7fc70b400640 1 --2- 172.21.15.89:0/2635682633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc70c1001b0 0x7fc70c19e270 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00e9f0 tx=0x7fc6fc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.829+0000 7fc703e00640 1 -- 172.21.15.89:0/2635682633 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc00cd80 con 0x7fc70c1001b0 2024-09-10T08:48:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.829+0000 7fc711200640 1 -- 172.21.15.89:0/2635682633 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70c1a6180 con 0x7fc70c1001b0 2024-09-10T08:48:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.829+0000 7fc703e00640 1 -- 172.21.15.89:0/2635682633 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6fc005500 con 0x7fc70c1001b0 2024-09-10T08:48:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.829+0000 7fc703e00640 1 -- 172.21.15.89:0/2635682633 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc010690 con 0x7fc70c1001b0 2024-09-10T08:48:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.829+0000 7fc711200640 1 -- 172.21.15.89:0/2635682633 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70c1a6680 con 0x7fc70c1001b0 2024-09-10T08:48:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.832+0000 7fc715680740 1 Processor -- start 2024-09-10T08:48:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.832+0000 7fc715680740 1 -- start start 2024-09-10T08:48:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.832+0000 7fc715680740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e774d260 0x5565e774d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.832+0000 7fc715680740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e774dba0 0x5565e7752660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.832+0000 7fc703e00640 1 -- 172.21.15.89:0/2635682633 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc6fc0046c0 con 0x7fc70c1001b0 2024-09-10T08:48:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.832+0000 7fc715680740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e7752c30 0x5565e77550f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.832+0000 7fc715680740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5565e76b2100 con 0x5565e774d260 2024-09-10T08:48:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.832+0000 7fc715680740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5565e765a1b0 con 0x5565e774dba0 2024-09-10T08:48:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.832+0000 7fc715680740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5565e76ae0c0 con 0x5565e7752c30 2024-09-10T08:48:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.833+0000 7fc70be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e774d260 0x5565e774d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.833+0000 7fc70b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e774dba0 0x5565e7752660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.833+0000 7fc70be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e774d260 0x5565e774d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60956/0 (socket says 172.21.15.89:60956) 2024-09-10T08:48:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.833+0000 7fc70be00640 1 -- 172.21.15.89:0/164305021 learned_addr learned my addr 172.21.15.89:0/164305021 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.833+0000 7fc70aa00640 1 --2- 172.21.15.89:0/164305021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e7752c30 0x5565e77550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.833+0000 7fc70be00640 1 -- 172.21.15.89:0/164305021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e7752c30 msgr2=0x5565e77550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.833+0000 7fc70be00640 1 --2- 172.21.15.89:0/164305021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e7752c30 0x5565e77550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.833+0000 7fc70be00640 1 -- 172.21.15.89:0/164305021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e774dba0 msgr2=0x5565e7752660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.833+0000 7fc70be00640 1 --2- 172.21.15.89:0/164305021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e774dba0 0x5565e7752660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.833+0000 7fc70be00640 1 -- 172.21.15.89:0/164305021 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565e73c5210 con 0x5565e774d260 2024-09-10T08:48:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.833+0000 7fc70aa00640 1 --2- 172.21.15.89:0/164305021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e7752c30 0x5565e77550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.833+0000 7fc70b400640 1 --2- 172.21.15.89:0/164305021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e774dba0 0x5565e7752660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.833+0000 7fc70be00640 1 --2- 172.21.15.89:0/164305021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e774d260 0x5565e774d660 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc704005e30 tx=0x7fc704005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369b47ff3ebb49ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.834+0000 7fc700c00640 1 -- 172.21.15.89:0/164305021 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704031650 con 0x5565e774d260 2024-09-10T08:48:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.834+0000 7fc700c00640 1 -- 172.21.15.89:0/164305021 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc704036040 con 0x5565e774d260 2024-09-10T08:48:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.834+0000 7fc700c00640 1 -- 172.21.15.89:0/164305021 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704004720 con 0x5565e774d260 2024-09-10T08:48:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.834+0000 7fc703e00640 1 --2- 172.21.15.89:0/2635682633 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc6f00a2b80 0x7fc6f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.834+0000 7fc703e00640 1 -- 172.21.15.89:0/2635682633 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fc6fc0cf900 con 0x7fc70c1001b0 2024-09-10T08:48:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.834+0000 7fc715680740 1 -- 172.21.15.89:0/164305021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e774d260 msgr2=0x5565e774d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.834+0000 7fc715680740 1 --2- 172.21.15.89:0/164305021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e774d260 0x5565e774d660 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc704005e30 tx=0x7fc704005f70 comp rx=0 tx=0).stop 2024-09-10T08:48:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.834+0000 7fc70b400640 1 --2- 172.21.15.89:0/2635682633 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc6f00a2b80 0x7fc6f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.834+0000 7fc715680740 1 -- 172.21.15.89:0/164305021 shutdown_connections 2024-09-10T08:48:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.834+0000 7fc715680740 1 --2- 172.21.15.89:0/164305021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e7752c30 0x5565e77550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.834+0000 7fc715680740 1 --2- 172.21.15.89:0/164305021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e774dba0 0x5565e7752660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.834+0000 7fc715680740 1 --2- 172.21.15.89:0/164305021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e774d260 0x5565e774d660 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.834+0000 7fc715680740 1 -- 172.21.15.89:0/164305021 >> 172.21.15.89:0/164305021 conn(0x5565e77488d0 msgr2=0x5565e7748cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.835+0000 7fc715680740 1 -- 172.21.15.89:0/164305021 shutdown_connections 2024-09-10T08:48:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.835+0000 7fc715680740 1 -- 172.21.15.89:0/164305021 wait complete. 2024-09-10T08:48:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.835+0000 7fc70b400640 1 --2- 172.21.15.89:0/2635682633 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc6f00a2b80 0x7fc6f00a5040 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc09bcc0 tx=0x7fc6fc0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.835+0000 7fc715680740 1 -- start start 2024-09-10T08:48:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.835+0000 7fc715680740 1 Processor -- start 2024-09-10T08:48:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.836+0000 7fc715680740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e774d260 0x5565e7757390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.836+0000 7fc715680740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e774dba0 0x5565e77578d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.836+0000 7fc715680740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e7752c30 0x5565e775e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.836+0000 7fc715680740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5565e7755ba0 con 0x5565e7752c30 2024-09-10T08:48:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.836+0000 7fc715680740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5565e763b090 con 0x5565e774d260 2024-09-10T08:48:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.836+0000 7fc715680740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5565e7755d20 con 0x5565e774dba0 2024-09-10T08:48:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.836+0000 7fc715680740 10 client.0 _ll_register_callbacks cb 0x5565e768ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.836+0000 7fc70be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e774d260 0x5565e7757390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.836+0000 7fc70be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e774d260 0x5565e7757390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57678/0 (socket says 172.21.15.89:57678) 2024-09-10T08:48:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.836+0000 7fc70be00640 1 -- 172.21.15.89:0/140840518 learned_addr learned my addr 172.21.15.89:0/140840518 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.836+0000 7fc70aa00640 1 --2- 172.21.15.89:0/140840518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e774dba0 0x5565e77578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.836+0000 7fc70be00640 1 --2- 172.21.15.89:0/140840518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e7752c30 0x5565e775e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.836+0000 7fc70be00640 1 -- 172.21.15.89:0/140840518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e774dba0 msgr2=0x5565e77578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.836+0000 7fc70be00640 1 --2- 172.21.15.89:0/140840518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e774dba0 0x5565e77578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.836+0000 7fc70be00640 1 -- 172.21.15.89:0/140840518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e7752c30 msgr2=0x5565e775e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.836+0000 7fc70be00640 1 --2- 172.21.15.89:0/140840518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e7752c30 0x5565e775e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.836+0000 7fc70be00640 1 -- 172.21.15.89:0/140840518 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565e7491620 con 0x5565e774d260 2024-09-10T08:48:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.837+0000 7fc70be00640 1 --2- 172.21.15.89:0/140840518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e774d260 0x5565e7757390 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc704031c10 tx=0x7fc70403e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b56d82f0fbabf3f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.837+0000 7fc6f5600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:48:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.837+0000 7fc6f5600640 1 -- 172.21.15.89:0/140840518 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704059060 con 0x5565e774d260 2024-09-10T08:48:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.837+0000 7fc6f5600640 1 -- 172.21.15.89:0/140840518 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fc704031650 con 0x5565e774d260 2024-09-10T08:48:20.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.837+0000 7fc715680740 1 -- 172.21.15.89:0/140840518 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc704036040 con 0x5565e774d260 2024-09-10T08:48:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.837+0000 7fc6f5600640 1 -- 172.21.15.89:0/140840518 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc704031850 con 0x5565e774d260 2024-09-10T08:48:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.837+0000 7fc715680740 10 client.26942 fetch_fsmap learned FSMap version 31 2024-09-10T08:48:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.837+0000 7fc715680740 1 -- 172.21.15.89:0/140840518 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5565e7755ba0 con 0x5565e774d260 2024-09-10T08:48:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.837+0000 7fc6f5600640 1 -- 172.21.15.89:0/140840518 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc704036040 con 0x5565e774d260 2024-09-10T08:48:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.837+0000 7fc6f5600640 1 -- 172.21.15.89:0/140840518 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fc70402f870 con 0x5565e774d260 2024-09-10T08:48:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.837+0000 7fc715680740 10 client.26942 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:48:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.838+0000 7fc715680740 10 client.26942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.838+0000 7fc715680740 20 client.26942 populate_metadata read hostname 'smithi089' 2024-09-10T08:48:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.838+0000 7fc715680740 1 --2- 172.21.15.89:0/140840518 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5565e7762940 0x5565e7764d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.838+0000 7fc715680740 4 client.26942 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.838+0000 7fc715680740 1 -- 172.21.15.89:0/140840518 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565e76bf830 con 0x5565e7762940 2024-09-10T08:48:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.838+0000 7fc70aa00640 1 --2- 172.21.15.89:0/140840518 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5565e7762940 0x5565e7764d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.838+0000 7fc6f5600640 10 client.26942 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:48:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.838+0000 7fc70aa00640 1 --2- 172.21.15.89:0/140840518 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5565e7762940 0x5565e7764d30 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea3af4115ff343f5 server_cookie=4033d7ba966213dd in_seq=0 out_seq=0 2024-09-10T08:48:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.839+0000 7fc6f5600640 1 -- 172.21.15.89:0/140840518 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565e76bf830 con 0x5565e7762940 2024-09-10T08:48:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.839+0000 7fc6f5600640 10 client.26942 handle_command_reply: tid=0 2024-09-10T08:48:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.839+0000 7fc715680740 1 client.26942 shutdown 2024-09-10T08:48:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.840+0000 7fc715680740 1 -- 172.21.15.89:0/140840518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e774d260 msgr2=0x5565e7757390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.840+0000 7fc715680740 1 --2- 172.21.15.89:0/140840518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e774d260 0x5565e7757390 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc704031c10 tx=0x7fc70403e040 comp rx=0 tx=0).stop 2024-09-10T08:48:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.840+0000 7fc715680740 1 -- 172.21.15.89:0/140840518 shutdown_connections 2024-09-10T08:48:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.840+0000 7fc715680740 1 --2- 172.21.15.89:0/140840518 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5565e7762940 0x5565e7764d30 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.840+0000 7fc715680740 1 --2- 172.21.15.89:0/140840518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e7752c30 0x5565e775e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.846 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.840+0000 7fc6f5600640 0 client.26942 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:48:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.840+0000 7fc715680740 1 --2- 172.21.15.89:0/140840518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e774dba0 0x5565e77578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.840+0000 7fc715680740 1 --2- 172.21.15.89:0/140840518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e774d260 0x5565e7757390 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.840+0000 7fc715680740 1 -- 172.21.15.89:0/140840518 >> 172.21.15.89:0/140840518 conn(0x5565e77488d0 msgr2=0x5565e774a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.840+0000 7fc715680740 1 -- 172.21.15.89:0/140840518 shutdown_connections 2024-09-10T08:48:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.840+0000 7fc715680740 1 -- 172.21.15.89:0/140840518 wait complete. 2024-09-10T08:48:20.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.840+0000 7fc715680740 20 client.26942 trim_cache size 0 max 16384 2024-09-10T08:48:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.855+0000 7fc715680740 1 Processor -- start 2024-09-10T08:48:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.855+0000 7fc715680740 1 -- start start 2024-09-10T08:48:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.855+0000 7fc715680740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e774d260 0x5565e77577b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.855+0000 7fc715680740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e774dba0 0x5565e775e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.855+0000 7fc715680740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e7752c30 0x5565e77695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.855+0000 7fc715680740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1b3570 con 0x5565e7752c30 2024-09-10T08:48:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.855+0000 7fc715680740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1b0e10 con 0x5565e774d260 2024-09-10T08:48:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.855+0000 7fc715680740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1aec60 con 0x5565e774dba0 2024-09-10T08:48:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.855+0000 7fc70be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e774d260 0x5565e77577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.855+0000 7fc70be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e7752c30 0x5565e77695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.855+0000 7fc70be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e774d260 0x5565e77577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57680/0 (socket says 172.21.15.89:57680) 2024-09-10T08:48:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.855+0000 7fc70be00640 1 -- 172.21.15.89:0/709179759 learned_addr learned my addr 172.21.15.89:0/709179759 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.855+0000 7fc70aa00640 1 --2- 172.21.15.89:0/709179759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e774dba0 0x5565e775e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.856+0000 7fc70be00640 1 -- 172.21.15.89:0/709179759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e774dba0 msgr2=0x5565e775e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.856+0000 7fc70be00640 1 --2- 172.21.15.89:0/709179759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e774dba0 0x5565e775e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.856+0000 7fc70be00640 1 -- 172.21.15.89:0/709179759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e7752c30 msgr2=0x5565e77695c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.856+0000 7fc70be00640 1 --2- 172.21.15.89:0/709179759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e7752c30 0x5565e77695c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.856+0000 7fc70be00640 1 -- 172.21.15.89:0/709179759 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565e774e080 con 0x5565e774d260 2024-09-10T08:48:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.856+0000 7fc70be00640 1 --2- 172.21.15.89:0/709179759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e774d260 0x5565e77577b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc704006010 tx=0x7fc70402a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86dc7446cb3c05c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.856+0000 7fc6f7e00640 1 -- 172.21.15.89:0/709179759 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704059040 con 0x5565e774d260 2024-09-10T08:48:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.856+0000 7fc6f7e00640 1 -- 172.21.15.89:0/709179759 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc704036020 con 0x5565e774d260 2024-09-10T08:48:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.856+0000 7fc6f7e00640 1 -- 172.21.15.89:0/709179759 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704004b40 con 0x5565e774d260 2024-09-10T08:48:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.857+0000 7fc715680740 1 -- 172.21.15.89:0/709179759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e774d260 msgr2=0x5565e77577b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.857+0000 7fc715680740 1 --2- 172.21.15.89:0/709179759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e774d260 0x5565e77577b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc704006010 tx=0x7fc70402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.857+0000 7fc715680740 1 -- 172.21.15.89:0/709179759 shutdown_connections 2024-09-10T08:48:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.857+0000 7fc715680740 1 --2- 172.21.15.89:0/709179759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e7752c30 0x5565e77695c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.857+0000 7fc715680740 1 --2- 172.21.15.89:0/709179759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e774dba0 0x5565e775e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.857+0000 7fc715680740 1 --2- 172.21.15.89:0/709179759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e774d260 0x5565e77577b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.857+0000 7fc715680740 1 -- 172.21.15.89:0/709179759 >> 172.21.15.89:0/709179759 conn(0x5565e77488d0 msgr2=0x5565e7747a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.857+0000 7fc715680740 1 -- 172.21.15.89:0/709179759 shutdown_connections 2024-09-10T08:48:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.857+0000 7fc715680740 1 -- 172.21.15.89:0/709179759 wait complete. 2024-09-10T08:48:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.857+0000 7fc715680740 1 -- start start 2024-09-10T08:48:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.857+0000 7fc715680740 1 Processor -- start 2024-09-10T08:48:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.858+0000 7fc715680740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e774d260 0x5565e775c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.858+0000 7fc715680740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e774dba0 0x5565e775c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.858+0000 7fc715680740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e7752c30 0x5565e775d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.858+0000 7fc715680740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1af2d0 con 0x5565e774dba0 2024-09-10T08:48:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.858+0000 7fc715680740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1b0740 con 0x5565e7752c30 2024-09-10T08:48:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.858+0000 7fc715680740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1af9a0 con 0x5565e774d260 2024-09-10T08:48:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.858+0000 7fc715680740 10 client.0 _ll_register_callbacks cb 0x7fc6fc0d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.858+0000 7fc70be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e7752c30 0x5565e775d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.858+0000 7fc70aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e774dba0 0x5565e775c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.858+0000 7fc70be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e7752c30 0x5565e775d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:57686/0 (socket says 172.21.15.89:57686) 2024-09-10T08:48:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.858+0000 7fc70be00640 1 -- 172.21.15.89:0/1679719565 learned_addr learned my addr 172.21.15.89:0/1679719565 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.858+0000 7fc70be00640 1 --2- 172.21.15.89:0/1679719565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e774d260 0x5565e775c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.858+0000 7fc70be00640 1 -- 172.21.15.89:0/1679719565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e774d260 msgr2=0x5565e775c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.858+0000 7fc70be00640 1 --2- 172.21.15.89:0/1679719565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e774d260 0x5565e775c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.858+0000 7fc70be00640 1 -- 172.21.15.89:0/1679719565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e774dba0 msgr2=0x5565e775c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.858+0000 7fc70be00640 1 --2- 172.21.15.89:0/1679719565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e774dba0 0x5565e775c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.858+0000 7fc70be00640 1 -- 172.21.15.89:0/1679719565 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565e775dcb0 con 0x5565e7752c30 2024-09-10T08:48:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.858+0000 7fc70be00640 1 --2- 172.21.15.89:0/1679719565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e7752c30 0x5565e775d6a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc704009bc0 tx=0x7fc704000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ee65bcfcf925b79 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.858+0000 7fc6f5600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:48:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.859+0000 7fc6f5600640 1 -- 172.21.15.89:0/1679719565 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704059030 con 0x5565e7752c30 2024-09-10T08:48:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.859+0000 7fc6f5600640 1 -- 172.21.15.89:0/1679719565 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fc70402f820 con 0x5565e7752c30 2024-09-10T08:48:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.859+0000 7fc715680740 1 -- 172.21.15.89:0/1679719565 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc704036020 con 0x5565e7752c30 2024-09-10T08:48:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.859+0000 7fc6f5600640 1 -- 172.21.15.89:0/1679719565 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc70402feb0 con 0x5565e7752c30 2024-09-10T08:48:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.859+0000 7fc715680740 10 client.26948 fetch_fsmap learned FSMap version 31 2024-09-10T08:48:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.859+0000 7fc715680740 1 -- 172.21.15.89:0/1679719565 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5565e77cd780 con 0x5565e7752c30 2024-09-10T08:48:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.859+0000 7fc6f5600640 1 -- 172.21.15.89:0/1679719565 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc704036020 con 0x5565e7752c30 2024-09-10T08:48:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.859+0000 7fc6f5600640 1 -- 172.21.15.89:0/1679719565 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fc704030650 con 0x5565e7752c30 2024-09-10T08:48:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.859+0000 7fc715680740 10 client.26948 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:48:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.859+0000 7fc715680740 10 client.26948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.859+0000 7fc715680740 20 client.26948 populate_metadata read hostname 'smithi089' 2024-09-10T08:48:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.859+0000 7fc715680740 1 --2- 172.21.15.89:0/1679719565 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5565e7762940 0x5565e77d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.859+0000 7fc715680740 4 client.26948 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:48:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.859+0000 7fc715680740 1 -- 172.21.15.89:0/1679719565 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565e76b9b80 con 0x5565e7762940 2024-09-10T08:48:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.860+0000 7fc70aa00640 1 --2- 172.21.15.89:0/1679719565 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5565e7762940 0x5565e77d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.860+0000 7fc70aa00640 1 --2- 172.21.15.89:0/1679719565 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5565e7762940 0x5565e77d00d0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebc8077383a582a server_cookie=592707643e11f299 in_seq=0 out_seq=0 2024-09-10T08:48:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:20.860+0000 7fc6f5600640 10 client.26948 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:48:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.079+0000 7fc6f5600640 1 -- 172.21.15.89:0/1679719565 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5565e76b9b80 con 0x5565e7762940 2024-09-10T08:48:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.079+0000 7fc6f5600640 10 client.26948 handle_command_reply: tid=0 2024-09-10T08:48:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.079+0000 7fc715680740 1 client.26948 shutdown 2024-09-10T08:48:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.079+0000 7fc715680740 1 -- 172.21.15.89:0/1679719565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e7752c30 msgr2=0x5565e775d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.079+0000 7fc715680740 1 --2- 172.21.15.89:0/1679719565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e7752c30 0x5565e775d6a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc704009bc0 tx=0x7fc704000f40 comp rx=0 tx=0).stop 2024-09-10T08:48:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.079+0000 7fc715680740 1 -- 172.21.15.89:0/1679719565 shutdown_connections 2024-09-10T08:48:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.079+0000 7fc715680740 1 --2- 172.21.15.89:0/1679719565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565e7752c30 0x5565e775d6a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.079+0000 7fc715680740 1 --2- 172.21.15.89:0/1679719565 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5565e7762940 0x5565e77d00d0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.079+0000 7fc715680740 1 --2- 172.21.15.89:0/1679719565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5565e774dba0 0x5565e775c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.079+0000 7fc715680740 1 --2- 172.21.15.89:0/1679719565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565e774d260 0x5565e775c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.079+0000 7fc6f5600640 0 client.26948 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:48:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.079+0000 7fc715680740 1 -- 172.21.15.89:0/1679719565 >> 172.21.15.89:0/1679719565 conn(0x5565e77488d0 msgr2=0x5565e7749780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.080+0000 7fc715680740 1 -- 172.21.15.89:0/1679719565 shutdown_connections 2024-09-10T08:48:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.081+0000 7fc715680740 1 -- 172.21.15.89:0/1679719565 wait complete. 2024-09-10T08:48:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.081+0000 7fc715680740 20 client.26948 trim_cache size 0 max 16384 2024-09-10T08:48:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:48:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:48:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:48:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:48:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.081+0000 7fc711200640 1 -- 172.21.15.89:0/2635682633 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc6f00a2b80 msgr2=0x7fc6f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.081+0000 7fc711200640 1 --2- 172.21.15.89:0/2635682633 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc6f00a2b80 0x7fc6f00a5040 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc09bcc0 tx=0x7fc6fc0d5530 comp rx=0 tx=0).stop 2024-09-10T08:48:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.081+0000 7fc711200640 1 -- 172.21.15.89:0/2635682633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc70c1001b0 msgr2=0x7fc70c19e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.081+0000 7fc711200640 1 --2- 172.21.15.89:0/2635682633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc70c1001b0 0x7fc70c19e270 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00e9f0 tx=0x7fc6fc00edf0 comp rx=0 tx=0).stop 2024-09-10T08:48:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.082+0000 7fc711200640 1 -- 172.21.15.89:0/2635682633 shutdown_connections 2024-09-10T08:48:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.082+0000 7fc711200640 1 --2- 172.21.15.89:0/2635682633 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fc6f00a2b80 0x7fc6f00a5040 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.082+0000 7fc711200640 1 --2- 172.21.15.89:0/2635682633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc70c1054b0 0x7fc70c1a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.082+0000 7fc711200640 1 --2- 172.21.15.89:0/2635682633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc70c102b00 0x7fc70c19e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.082+0000 7fc711200640 1 --2- 172.21.15.89:0/2635682633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc70c1001b0 0x7fc70c19e270 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.082+0000 7fc711200640 1 -- 172.21.15.89:0/2635682633 >> 172.21.15.89:0/2635682633 conn(0x7fc70c0fbfa0 msgr2=0x7fc70c100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.082+0000 7fc711200640 1 -- 172.21.15.89:0/2635682633 shutdown_connections 2024-09-10T08:48:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:21.082+0000 7fc711200640 1 -- 172.21.15.89:0/2635682633 wait complete. 2024-09-10T08:48:21.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:20 smithi149 ceph-mon[36167]: pgmap v645: 129 pgs: 129 active+clean; 265 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 608 op/s 2024-09-10T08:48:21.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:20 smithi111 ceph-mon[35220]: pgmap v645: 129 pgs: 129 active+clean; 265 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 608 op/s 2024-09-10T08:48:21.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:20 smithi089 ceph-mon[30757]: pgmap v645: 129 pgs: 129 active+clean; 265 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 608 op/s 2024-09-10T08:48:22.192 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:21 smithi089 ceph-mon[30757]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:48:22.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:21 smithi149 ceph-mon[36167]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:48:22.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:21 smithi111 ceph-mon[35220]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:48:23.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:22 smithi149 ceph-mon[36167]: pgmap v646: 129 pgs: 129 active+clean; 251 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 734 op/s 2024-09-10T08:48:23.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:22 smithi111 ceph-mon[35220]: pgmap v646: 129 pgs: 129 active+clean; 251 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 734 op/s 2024-09-10T08:48:23.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:22 smithi089 ceph-mon[30757]: pgmap v646: 129 pgs: 129 active+clean; 251 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 734 op/s 2024-09-10T08:48:25.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:24 smithi149 ceph-mon[36167]: pgmap v647: 129 pgs: 129 active+clean; 238 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 654 op/s 2024-09-10T08:48:25.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:24 smithi111 ceph-mon[35220]: pgmap v647: 129 pgs: 129 active+clean; 238 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 654 op/s 2024-09-10T08:48:25.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:24 smithi089 ceph-mon[30757]: pgmap v647: 129 pgs: 129 active+clean; 238 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 654 op/s 2024-09-10T08:48:27.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:26 smithi149 ceph-mon[36167]: pgmap v648: 129 pgs: 129 active+clean; 229 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 570 op/s 2024-09-10T08:48:27.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:26 smithi111 ceph-mon[35220]: pgmap v648: 129 pgs: 129 active+clean; 229 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 570 op/s 2024-09-10T08:48:27.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:26 smithi089 ceph-mon[30757]: pgmap v648: 129 pgs: 129 active+clean; 229 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 570 op/s 2024-09-10T08:48:29.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:28 smithi149 ceph-mon[36167]: pgmap v649: 129 pgs: 129 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 796 op/s 2024-09-10T08:48:29.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:28 smithi111 ceph-mon[35220]: pgmap v649: 129 pgs: 129 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 796 op/s 2024-09-10T08:48:29.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:28 smithi089 ceph-mon[30757]: pgmap v649: 129 pgs: 129 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 796 op/s 2024-09-10T08:48:31.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:30 smithi149 ceph-mon[36167]: pgmap v650: 129 pgs: 129 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 623 op/s 2024-09-10T08:48:31.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:30 smithi111 ceph-mon[35220]: pgmap v650: 129 pgs: 129 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 623 op/s 2024-09-10T08:48:31.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:30 smithi089 ceph-mon[30757]: pgmap v650: 129 pgs: 129 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 623 op/s 2024-09-10T08:48:33.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:32 smithi149 ceph-mon[36167]: pgmap v651: 129 pgs: 129 active+clean; 185 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 757 op/s 2024-09-10T08:48:33.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:32 smithi111 ceph-mon[35220]: pgmap v651: 129 pgs: 129 active+clean; 185 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 757 op/s 2024-09-10T08:48:33.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:32 smithi089 ceph-mon[30757]: pgmap v651: 129 pgs: 129 active+clean; 185 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 757 op/s 2024-09-10T08:48:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.198+0000 7ff1efe00640 1 -- 172.21.15.89:0/585066038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1e8105910 msgr2=0x7ff1e810c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.198+0000 7ff1efe00640 1 --2- 172.21.15.89:0/585066038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1e8105910 0x7ff1e810c1a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009ca0 tx=0x7ff1e002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.198+0000 7ff1efe00640 1 -- 172.21.15.89:0/585066038 shutdown_connections 2024-09-10T08:48:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.198+0000 7ff1efe00640 1 --2- 172.21.15.89:0/585066038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1e8105910 0x7ff1e810c1a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.198+0000 7ff1efe00640 1 --2- 172.21.15.89:0/585066038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1e8104f50 0x7ff1e81053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.198+0000 7ff1efe00640 1 --2- 172.21.15.89:0/585066038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1e8103d60 0x7ff1e8104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.198+0000 7ff1efe00640 1 -- 172.21.15.89:0/585066038 >> 172.21.15.89:0/585066038 conn(0x7ff1e80ff4e0 msgr2=0x7ff1e8101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.198+0000 7ff1efe00640 1 -- 172.21.15.89:0/585066038 shutdown_connections 2024-09-10T08:48:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.198+0000 7ff1efe00640 1 -- 172.21.15.89:0/585066038 wait complete. 2024-09-10T08:48:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.199+0000 7ff1efe00640 1 Processor -- start 2024-09-10T08:48:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.199+0000 7ff1efe00640 1 -- start start 2024-09-10T08:48:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.200+0000 7ff1efe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1e8103d60 0x7ff1e8199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.200+0000 7ff1efe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1e8104f50 0x7ff1e819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.200+0000 7ff1efe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1e8105910 0x7ff1e81a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.200+0000 7ff1efe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e810fa30 con 0x7ff1e8105910 2024-09-10T08:48:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.200+0000 7ff1efe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e810f8b0 con 0x7ff1e8103d60 2024-09-10T08:48:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.200+0000 7ff1efe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e810fbb0 con 0x7ff1e8104f50 2024-09-10T08:48:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.200+0000 7ff1ed400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1e8103d60 0x7ff1e8199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.200+0000 7ff1ed400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1e8103d60 0x7ff1e8199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35374/0 (socket says 172.21.15.89:35374) 2024-09-10T08:48:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.200+0000 7ff1ed400640 1 -- 172.21.15.89:0/3419456106 learned_addr learned my addr 172.21.15.89:0/3419456106 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.200+0000 7ff1ede00640 1 --2- 172.21.15.89:0/3419456106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1e8105910 0x7ff1e81a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.200+0000 7ff1eca00640 1 --2- 172.21.15.89:0/3419456106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1e8104f50 0x7ff1e819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.200+0000 7ff1ed400640 1 -- 172.21.15.89:0/3419456106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1e8104f50 msgr2=0x7ff1e819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.200+0000 7ff1ed400640 1 --2- 172.21.15.89:0/3419456106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1e8104f50 0x7ff1e819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.200+0000 7ff1ed400640 1 -- 172.21.15.89:0/3419456106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1e8105910 msgr2=0x7ff1e81a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.200+0000 7ff1ed400640 1 --2- 172.21.15.89:0/3419456106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1e8105910 0x7ff1e81a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.200+0000 7ff1ed400640 1 -- 172.21.15.89:0/3419456106 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e81a1bc0 con 0x7ff1e8103d60 2024-09-10T08:48:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.200+0000 7ff1eca00640 1 --2- 172.21.15.89:0/3419456106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1e8104f50 0x7ff1e819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.201+0000 7ff1ed400640 1 --2- 172.21.15.89:0/3419456106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1e8103d60 0x7ff1e8199fa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00e9c0 tx=0x7ff1dc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.201+0000 7ff1e6000640 1 -- 172.21.15.89:0/3419456106 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00cde0 con 0x7ff1e8103d60 2024-09-10T08:48:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.201+0000 7ff1efe00640 1 -- 172.21.15.89:0/3419456106 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e81a1eb0 con 0x7ff1e8103d60 2024-09-10T08:48:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.201+0000 7ff1e6000640 1 -- 172.21.15.89:0/3419456106 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1dc010910 con 0x7ff1e8103d60 2024-09-10T08:48:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.201+0000 7ff1e6000640 1 -- 172.21.15.89:0/3419456106 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc018a10 con 0x7ff1e8103d60 2024-09-10T08:48:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.201+0000 7ff1efe00640 1 -- 172.21.15.89:0/3419456106 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e81a2370 con 0x7ff1e8103d60 2024-09-10T08:48:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.204+0000 7ff1f2cc0740 1 Processor -- start 2024-09-10T08:48:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.204+0000 7ff1f2cc0740 1 -- start start 2024-09-10T08:48:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.204+0000 7ff1f2cc0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b0090 0x5582eb2b0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.204+0000 7ff1f2cc0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b09d0 0x5582eb2b5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.204+0000 7ff1f2cc0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b5a60 0x5582eb2b7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.204+0000 7ff1f2cc0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5582eb215100 con 0x5582eb2b5a60 2024-09-10T08:48:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.204+0000 7ff1f2cc0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5582eb1bd1b0 con 0x5582eb2b09d0 2024-09-10T08:48:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.204+0000 7ff1f2cc0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5582eb2110c0 con 0x5582eb2b0090 2024-09-10T08:48:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.205+0000 7ff1e6000640 1 -- 172.21.15.89:0/3419456106 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff1dc01f020 con 0x7ff1e8103d60 2024-09-10T08:48:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.205+0000 7ff1ede00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b0090 0x5582eb2b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.205+0000 7ff1ede00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b0090 0x5582eb2b0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:37282/0 (socket says 172.21.15.89:37282) 2024-09-10T08:48:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.205+0000 7ff1ede00640 1 -- 172.21.15.89:0/854365451 learned_addr learned my addr 172.21.15.89:0/854365451 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.205+0000 7ff1ed400640 1 --2- 172.21.15.89:0/854365451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b09d0 0x5582eb2b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.205+0000 7ff1eca00640 1 --2- 172.21.15.89:0/854365451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b5a60 0x5582eb2b7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.205+0000 7ff1ede00640 1 -- 172.21.15.89:0/854365451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b09d0 msgr2=0x5582eb2b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.205+0000 7ff1ede00640 1 --2- 172.21.15.89:0/854365451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b09d0 0x5582eb2b5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.205+0000 7ff1ede00640 1 -- 172.21.15.89:0/854365451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b5a60 msgr2=0x5582eb2b7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.205+0000 7ff1ede00640 1 --2- 172.21.15.89:0/854365451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b5a60 0x5582eb2b7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.205+0000 7ff1ede00640 1 -- 172.21.15.89:0/854365451 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582eaf28210 con 0x5582eb2b0090 2024-09-10T08:48:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.205+0000 7ff1ed400640 1 --2- 172.21.15.89:0/854365451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b09d0 0x5582eb2b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.205+0000 7ff1ede00640 1 --2- 172.21.15.89:0/854365451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b0090 0x5582eb2b0490 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0035000 tx=0x7ff1e00093c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=563bbe8bffc81785 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.206+0000 7ff1da600640 1 -- 172.21.15.89:0/854365451 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0031650 con 0x5582eb2b0090 2024-09-10T08:48:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.206+0000 7ff1da600640 1 -- 172.21.15.89:0/854365451 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1e0036030 con 0x5582eb2b0090 2024-09-10T08:48:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.206+0000 7ff1da600640 1 -- 172.21.15.89:0/854365451 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0004720 con 0x5582eb2b0090 2024-09-10T08:48:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.206+0000 7ff1e6000640 1 --2- 172.21.15.89:0/3419456106 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff1d00a2c30 0x7ff1d00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.206+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/854365451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b0090 msgr2=0x5582eb2b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.206+0000 7ff1f2cc0740 1 --2- 172.21.15.89:0/854365451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b0090 0x5582eb2b0490 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0035000 tx=0x7ff1e00093c0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.206+0000 7ff1e6000640 1 -- 172.21.15.89:0/3419456106 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff1dc0cf730 con 0x7ff1e8103d60 2024-09-10T08:48:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.206+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/854365451 shutdown_connections 2024-09-10T08:48:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.206+0000 7ff1f2cc0740 1 --2- 172.21.15.89:0/854365451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b5a60 0x5582eb2b7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.206+0000 7ff1f2cc0740 1 --2- 172.21.15.89:0/854365451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b09d0 0x5582eb2b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.206+0000 7ff1f2cc0740 1 --2- 172.21.15.89:0/854365451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b0090 0x5582eb2b0490 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.206+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/854365451 >> 172.21.15.89:0/854365451 conn(0x5582eb2ab700 msgr2=0x5582eb2abb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.206+0000 7ff1ed400640 1 --2- 172.21.15.89:0/3419456106 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff1d00a2c30 0x7ff1d00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.207+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/854365451 shutdown_connections 2024-09-10T08:48:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.207+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/854365451 wait complete. 2024-09-10T08:48:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.207+0000 7ff1ed400640 1 --2- 172.21.15.89:0/3419456106 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff1d00a2c30 0x7ff1d00a50f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc098530 tx=0x7ff1dc00efb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.207+0000 7ff1f2cc0740 1 -- start start 2024-09-10T08:48:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.207+0000 7ff1f2cc0740 1 Processor -- start 2024-09-10T08:48:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.208+0000 7ff1f2cc0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b0090 0x5582eb2c2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.208+0000 7ff1f2cc0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b09d0 0x5582eb2c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.208+0000 7ff1f2cc0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b5a60 0x5582eb2c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.208+0000 7ff1f2cc0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5582eb2b8850 con 0x5582eb2b09d0 2024-09-10T08:48:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.208+0000 7ff1f2cc0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5582eb19e090 con 0x5582eb2b5a60 2024-09-10T08:48:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.208+0000 7ff1f2cc0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5582eb2b89d0 con 0x5582eb2b0090 2024-09-10T08:48:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.208+0000 7ff1f2cc0740 10 client.0 _ll_register_callbacks cb 0x5582eb1f2f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.208+0000 7ff1ede00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b0090 0x5582eb2c2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.208+0000 7ff1ede00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b0090 0x5582eb2c2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:37284/0 (socket says 172.21.15.89:37284) 2024-09-10T08:48:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.208+0000 7ff1ede00640 1 -- 172.21.15.89:0/2356075749 learned_addr learned my addr 172.21.15.89:0/2356075749 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.208+0000 7ff1eca00640 1 --2- 172.21.15.89:0/2356075749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b09d0 0x5582eb2c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.208+0000 7ff1ede00640 1 --2- 172.21.15.89:0/2356075749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b5a60 0x5582eb2c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.208+0000 7ff1ede00640 1 -- 172.21.15.89:0/2356075749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b5a60 msgr2=0x5582eb2c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.208+0000 7ff1ede00640 1 --2- 172.21.15.89:0/2356075749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b5a60 0x5582eb2c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.208+0000 7ff1ede00640 1 -- 172.21.15.89:0/2356075749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b09d0 msgr2=0x5582eb2c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.208+0000 7ff1ede00640 1 --2- 172.21.15.89:0/2356075749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b09d0 0x5582eb2c2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.208+0000 7ff1ede00640 1 -- 172.21.15.89:0/2356075749 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582eaff4620 con 0x5582eb2b0090 2024-09-10T08:48:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.209+0000 7ff1ede00640 1 --2- 172.21.15.89:0/2356075749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b0090 0x5582eb2c2940 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009a60 tx=0x7ff1e0004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18fcea73a5157cd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.209+0000 7ff1cea00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:48:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.209+0000 7ff1cea00640 1 -- 172.21.15.89:0/2356075749 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0059060 con 0x5582eb2b0090 2024-09-10T08:48:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.209+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/2356075749 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1e0036030 con 0x5582eb2b0090 2024-09-10T08:48:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.209+0000 7ff1cea00640 1 -- 172.21.15.89:0/2356075749 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff1e0003d80 con 0x5582eb2b0090 2024-09-10T08:48:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.209+0000 7ff1cea00640 1 -- 172.21.15.89:0/2356075749 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff1e0004b40 con 0x5582eb2b0090 2024-09-10T08:48:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.209+0000 7ff1f2cc0740 10 client.26905 fetch_fsmap learned FSMap version 32 2024-09-10T08:48:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.209+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/2356075749 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5582eb2b8850 con 0x5582eb2b0090 2024-09-10T08:48:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.209+0000 7ff1cea00640 1 -- 172.21.15.89:0/2356075749 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1e0036030 con 0x5582eb2b0090 2024-09-10T08:48:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.210+0000 7ff1cea00640 1 -- 172.21.15.89:0/2356075749 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7ff1e0043070 con 0x5582eb2b0090 2024-09-10T08:48:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.210+0000 7ff1f2cc0740 10 client.26905 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T08:48:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.210+0000 7ff1f2cc0740 10 client.26905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.210+0000 7ff1f2cc0740 20 client.26905 populate_metadata read hostname 'smithi089' 2024-09-10T08:48:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.210+0000 7ff1f2cc0740 1 --2- 172.21.15.89:0/2356075749 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5582eb2ce020 0x5582eb2d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.210+0000 7ff1f2cc0740 4 client.26905 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.210+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/2356075749 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582eb222830 con 0x5582eb2ce020 2024-09-10T08:48:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.210+0000 7ff1eca00640 1 --2- 172.21.15.89:0/2356075749 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5582eb2ce020 0x5582eb2d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.211+0000 7ff1cea00640 10 client.26905 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:48:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.211+0000 7ff1eca00640 1 --2- 172.21.15.89:0/2356075749 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5582eb2ce020 0x5582eb2d0410 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f9dfdeedc9f12f2 server_cookie=c8b0d79981753d38 in_seq=0 out_seq=0 2024-09-10T08:48:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.212+0000 7ff1cea00640 1 -- 172.21.15.89:0/2356075749 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5582eb222830 con 0x5582eb2ce020 2024-09-10T08:48:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.212+0000 7ff1cea00640 10 client.26905 handle_command_reply: tid=0 2024-09-10T08:48:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.212+0000 7ff1f2cc0740 1 client.26905 shutdown 2024-09-10T08:48:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.212+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/2356075749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b0090 msgr2=0x5582eb2c2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.212+0000 7ff1f2cc0740 1 --2- 172.21.15.89:0/2356075749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b0090 0x5582eb2c2940 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009a60 tx=0x7ff1e0004040 comp rx=0 tx=0).stop 2024-09-10T08:48:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.212+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/2356075749 shutdown_connections 2024-09-10T08:48:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.212+0000 7ff1f2cc0740 1 --2- 172.21.15.89:0/2356075749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b5a60 0x5582eb2c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.212+0000 7ff1f2cc0740 1 --2- 172.21.15.89:0/2356075749 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5582eb2ce020 0x5582eb2d0410 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.212+0000 7ff1f2cc0740 1 --2- 172.21.15.89:0/2356075749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b09d0 0x5582eb2c2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.212+0000 7ff1f2cc0740 1 --2- 172.21.15.89:0/2356075749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b0090 0x5582eb2c2940 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.212+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/2356075749 >> 172.21.15.89:0/2356075749 conn(0x5582eb2ab700 msgr2=0x5582eb2acf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.212+0000 7ff1cea00640 0 client.26905 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:48:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.212+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/2356075749 shutdown_connections 2024-09-10T08:48:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.212+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/2356075749 wait complete. 2024-09-10T08:48:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.213+0000 7ff1f2cc0740 20 client.26905 trim_cache size 0 max 16384 2024-09-10T08:48:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.228+0000 7ff1f2cc0740 1 Processor -- start 2024-09-10T08:48:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.229+0000 7ff1f2cc0740 1 -- start start 2024-09-10T08:48:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.229+0000 7ff1f2cc0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b0090 0x5582eb2aa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.229+0000 7ff1f2cc0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b09d0 0x5582eb2c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.229+0000 7ff1f2cc0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b5a60 0x5582eb2aa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.229+0000 7ff1f2cc0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e81aae50 con 0x5582eb2b09d0 2024-09-10T08:48:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.229+0000 7ff1f2cc0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e81a86c0 con 0x5582eb2b0090 2024-09-10T08:48:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.229+0000 7ff1f2cc0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e81a6510 con 0x5582eb2b5a60 2024-09-10T08:48:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.229+0000 7ff1eca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b09d0 0x5582eb2c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.229+0000 7ff1eca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b09d0 0x5582eb2c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49314/0 (socket says 172.21.15.89:49314) 2024-09-10T08:48:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.229+0000 7ff1eca00640 1 -- 172.21.15.89:0/1290347861 learned_addr learned my addr 172.21.15.89:0/1290347861 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.229+0000 7ff1ede00640 1 --2- 172.21.15.89:0/1290347861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b0090 0x5582eb2aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.229+0000 7ff1ede00640 1 --2- 172.21.15.89:0/1290347861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b5a60 0x5582eb2aa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.229+0000 7ff1eca00640 1 -- 172.21.15.89:0/1290347861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b5a60 msgr2=0x5582eb2aa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.229+0000 7ff1eca00640 1 --2- 172.21.15.89:0/1290347861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b5a60 0x5582eb2aa5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.229+0000 7ff1eca00640 1 -- 172.21.15.89:0/1290347861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b0090 msgr2=0x5582eb2aa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.229+0000 7ff1eca00640 1 --2- 172.21.15.89:0/1290347861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b0090 0x5582eb2aa060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.229+0000 7ff1eca00640 1 -- 172.21.15.89:0/1290347861 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582eb2d0780 con 0x5582eb2b09d0 2024-09-10T08:48:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.229+0000 7ff1ede00640 1 --2- 172.21.15.89:0/1290347861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b5a60 0x5582eb2aa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.230+0000 7ff1eca00640 1 --2- 172.21.15.89:0/1290347861 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b09d0 0x5582eb2c9da0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x5582eb2ac580 tx=0x7ff1d4014900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d026dc1c113c6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.230+0000 7ff1d9c00640 1 -- 172.21.15.89:0/1290347861 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4005020 con 0x5582eb2b09d0 2024-09-10T08:48:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.230+0000 7ff1d9c00640 1 -- 172.21.15.89:0/1290347861 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1d40047a0 con 0x5582eb2b09d0 2024-09-10T08:48:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.230+0000 7ff1d9c00640 1 -- 172.21.15.89:0/1290347861 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4011760 con 0x5582eb2b09d0 2024-09-10T08:48:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.230+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/1290347861 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b09d0 msgr2=0x5582eb2c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.230+0000 7ff1f2cc0740 1 --2- 172.21.15.89:0/1290347861 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b09d0 0x5582eb2c9da0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x5582eb2ac580 tx=0x7ff1d4014900 comp rx=0 tx=0).stop 2024-09-10T08:48:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.230+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/1290347861 shutdown_connections 2024-09-10T08:48:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.230+0000 7ff1f2cc0740 1 --2- 172.21.15.89:0/1290347861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b5a60 0x5582eb2aa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.230+0000 7ff1f2cc0740 1 --2- 172.21.15.89:0/1290347861 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b09d0 0x5582eb2c9da0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.230+0000 7ff1f2cc0740 1 --2- 172.21.15.89:0/1290347861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b0090 0x5582eb2aa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.230+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/1290347861 >> 172.21.15.89:0/1290347861 conn(0x5582eb2ab700 msgr2=0x5582eb2c26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.231+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/1290347861 shutdown_connections 2024-09-10T08:48:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.231+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/1290347861 wait complete. 2024-09-10T08:48:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.231+0000 7ff1f2cc0740 1 -- start start 2024-09-10T08:48:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.231+0000 7ff1f2cc0740 1 Processor -- start 2024-09-10T08:48:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.231+0000 7ff1f2cc0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b0090 0x5582eb2c7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.231+0000 7ff1f2cc0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b09d0 0x5582eb2c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.231+0000 7ff1f2cc0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b5a60 0x5582eb2c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.231+0000 7ff1f2cc0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e81a6b80 con 0x5582eb2b09d0 2024-09-10T08:48:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.231+0000 7ff1f2cc0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e81a7ff0 con 0x5582eb2b0090 2024-09-10T08:48:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.231+0000 7ff1f2cc0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e81a7250 con 0x5582eb2b5a60 2024-09-10T08:48:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.231+0000 7ff1f2cc0740 10 client.0 _ll_register_callbacks cb 0x7ff1e0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.232+0000 7ff1ede00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b0090 0x5582eb2c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.232+0000 7ff1ede00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b0090 0x5582eb2c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:35428/0 (socket says 172.21.15.89:35428) 2024-09-10T08:48:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.232+0000 7ff1ede00640 1 -- 172.21.15.89:0/3170867716 learned_addr learned my addr 172.21.15.89:0/3170867716 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.232+0000 7ff1eca00640 1 --2- 172.21.15.89:0/3170867716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b09d0 0x5582eb2c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.232+0000 7ff1ede00640 1 --2- 172.21.15.89:0/3170867716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b5a60 0x5582eb2c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.232+0000 7ff1ede00640 1 -- 172.21.15.89:0/3170867716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b5a60 msgr2=0x5582eb2c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.232+0000 7ff1ede00640 1 --2- 172.21.15.89:0/3170867716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b5a60 0x5582eb2c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.232+0000 7ff1ede00640 1 -- 172.21.15.89:0/3170867716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b09d0 msgr2=0x5582eb2c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.232+0000 7ff1ede00640 1 --2- 172.21.15.89:0/3170867716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b09d0 0x5582eb2c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.232+0000 7ff1ede00640 1 -- 172.21.15.89:0/3170867716 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582eb2c8df0 con 0x5582eb2b0090 2024-09-10T08:48:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.232+0000 7ff1ede00640 1 --2- 172.21.15.89:0/3170867716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b0090 0x5582eb2c7290 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009ca0 tx=0x7ff1e002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9cba6dd6b44604c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.232+0000 7ff1cea00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:48:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.232+0000 7ff1cea00640 1 -- 172.21.15.89:0/3170867716 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0059070 con 0x5582eb2b0090 2024-09-10T08:48:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.233+0000 7ff1cea00640 1 -- 172.21.15.89:0/3170867716 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff1e00040b0 con 0x5582eb2b0090 2024-09-10T08:48:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.233+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/3170867716 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1d40047a0 con 0x5582eb2b0090 2024-09-10T08:48:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.233+0000 7ff1cea00640 1 -- 172.21.15.89:0/3170867716 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff1e006f050 con 0x5582eb2b0090 2024-09-10T08:48:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.233+0000 7ff1f2cc0740 10 client.26966 fetch_fsmap learned FSMap version 32 2024-09-10T08:48:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.233+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/3170867716 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5582eb2ca100 con 0x5582eb2b0090 2024-09-10T08:48:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.233+0000 7ff1cea00640 1 -- 172.21.15.89:0/3170867716 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1d40047a0 con 0x5582eb2b0090 2024-09-10T08:48:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.233+0000 7ff1cea00640 1 -- 172.21.15.89:0/3170867716 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7ff1e0064020 con 0x5582eb2b0090 2024-09-10T08:48:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.233+0000 7ff1f2cc0740 10 client.26966 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T08:48:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.233+0000 7ff1f2cc0740 10 client.26966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.233+0000 7ff1f2cc0740 20 client.26966 populate_metadata read hostname 'smithi089' 2024-09-10T08:48:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.233+0000 7ff1f2cc0740 1 --2- 172.21.15.89:0/3170867716 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5582eb339480 0x5582eb33b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.233+0000 7ff1f2cc0740 4 client.26966 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:48:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.233+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/3170867716 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5582eb21cb80 con 0x5582eb339480 2024-09-10T08:48:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.234+0000 7ff1eca00640 1 --2- 172.21.15.89:0/3170867716 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5582eb339480 0x5582eb33b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.234+0000 7ff1eca00640 1 --2- 172.21.15.89:0/3170867716 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5582eb339480 0x5582eb33b8a0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7db77d8abc4a3461 server_cookie=18b760e7fb991611 in_seq=0 out_seq=0 2024-09-10T08:48:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.234+0000 7ff1cea00640 10 client.26966 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:48:34.626 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.624+0000 7ff1cea00640 1 -- 172.21.15.89:0/3170867716 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5582eb21cb80 con 0x5582eb339480 2024-09-10T08:48:34.626 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.624+0000 7ff1cea00640 10 client.26966 handle_command_reply: tid=0 2024-09-10T08:48:34.626 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.624+0000 7ff1f2cc0740 1 client.26966 shutdown 2024-09-10T08:48:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.624+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/3170867716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b0090 msgr2=0x5582eb2c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.624+0000 7ff1f2cc0740 1 --2- 172.21.15.89:0/3170867716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b0090 0x5582eb2c7290 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009ca0 tx=0x7ff1e002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.624+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/3170867716 shutdown_connections 2024-09-10T08:48:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.624+0000 7ff1f2cc0740 1 --2- 172.21.15.89:0/3170867716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb2b5a60 0x5582eb2c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.624+0000 7ff1f2cc0740 1 --2- 172.21.15.89:0/3170867716 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x5582eb339480 0x5582eb33b8a0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.624+0000 7ff1f2cc0740 1 --2- 172.21.15.89:0/3170867716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5582eb2b09d0 0x5582eb2c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.624+0000 7ff1f2cc0740 1 --2- 172.21.15.89:0/3170867716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582eb2b0090 0x5582eb2c7290 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.624+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/3170867716 >> 172.21.15.89:0/3170867716 conn(0x5582eb2ab700 msgr2=0x5582eb2ac5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.625+0000 7ff1cea00640 0 client.26966 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:48:34.627 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.625+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/3170867716 shutdown_connections 2024-09-10T08:48:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.626+0000 7ff1f2cc0740 1 -- 172.21.15.89:0/3170867716 wait complete. 2024-09-10T08:48:34.628 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.626+0000 7ff1f2cc0740 20 client.26966 trim_cache size 0 max 16384 2024-09-10T08:48:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:48:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:48:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:48:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:48:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.627+0000 7ff1efe00640 1 -- 172.21.15.89:0/3419456106 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff1d00a2c30 msgr2=0x7ff1d00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.627+0000 7ff1efe00640 1 --2- 172.21.15.89:0/3419456106 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff1d00a2c30 0x7ff1d00a50f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc098530 tx=0x7ff1dc00efb0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.627+0000 7ff1efe00640 1 -- 172.21.15.89:0/3419456106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1e8103d60 msgr2=0x7ff1e8199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.627+0000 7ff1efe00640 1 --2- 172.21.15.89:0/3419456106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1e8103d60 0x7ff1e8199fa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00e9c0 tx=0x7ff1dc00ee80 comp rx=0 tx=0).stop 2024-09-10T08:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.627+0000 7ff1efe00640 1 -- 172.21.15.89:0/3419456106 shutdown_connections 2024-09-10T08:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.627+0000 7ff1efe00640 1 --2- 172.21.15.89:0/3419456106 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff1d00a2c30 0x7ff1d00a50f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.627+0000 7ff1efe00640 1 --2- 172.21.15.89:0/3419456106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1e8105910 0x7ff1e81a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.627+0000 7ff1efe00640 1 --2- 172.21.15.89:0/3419456106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1e8104f50 0x7ff1e819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.627+0000 7ff1efe00640 1 --2- 172.21.15.89:0/3419456106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1e8103d60 0x7ff1e8199fa0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.627+0000 7ff1efe00640 1 -- 172.21.15.89:0/3419456106 >> 172.21.15.89:0/3419456106 conn(0x7ff1e80ff4e0 msgr2=0x7ff1e81081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.627+0000 7ff1efe00640 1 -- 172.21.15.89:0/3419456106 shutdown_connections 2024-09-10T08:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:34.627+0000 7ff1efe00640 1 -- 172.21.15.89:0/3419456106 wait complete. 2024-09-10T08:48:35.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:34 smithi149 ceph-mon[36167]: pgmap v652: 129 pgs: 129 active+clean; 177 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 708 op/s 2024-09-10T08:48:35.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:34 smithi111 ceph-mon[35220]: pgmap v652: 129 pgs: 129 active+clean; 177 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 708 op/s 2024-09-10T08:48:35.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:34 smithi089 ceph-mon[30757]: pgmap v652: 129 pgs: 129 active+clean; 177 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 708 op/s 2024-09-10T08:48:37.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:36 smithi149 ceph-mon[36167]: pgmap v653: 129 pgs: 129 active+clean; 178 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 651 op/s 2024-09-10T08:48:37.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:48:37.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:36 smithi111 ceph-mon[35220]: pgmap v653: 129 pgs: 129 active+clean; 178 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 651 op/s 2024-09-10T08:48:37.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:48:37.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:36 smithi089 ceph-mon[30757]: pgmap v653: 129 pgs: 129 active+clean; 178 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 651 op/s 2024-09-10T08:48:37.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:48:38.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:37 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:48:38.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:37 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:48:38.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:37 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:48:38.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:37 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:48:38.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:37 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:48:38.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:37 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:48:38.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:37 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:48:38.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:37 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:48:38.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:37 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:48:39.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:38 smithi149 ceph-mon[36167]: pgmap v654: 129 pgs: 129 active+clean; 140 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 887 op/s 2024-09-10T08:48:39.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:38 smithi111 ceph-mon[35220]: pgmap v654: 129 pgs: 129 active+clean; 140 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 887 op/s 2024-09-10T08:48:39.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:38 smithi089 ceph-mon[30757]: pgmap v654: 129 pgs: 129 active+clean; 140 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 887 op/s 2024-09-10T08:48:41.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:40 smithi149 ceph-mon[36167]: pgmap v655: 129 pgs: 129 active+clean; 140 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 661 op/s 2024-09-10T08:48:41.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:40 smithi111 ceph-mon[35220]: pgmap v655: 129 pgs: 129 active+clean; 140 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 661 op/s 2024-09-10T08:48:41.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:40 smithi089 ceph-mon[30757]: pgmap v655: 129 pgs: 129 active+clean; 140 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 661 op/s 2024-09-10T08:48:43.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:42 smithi149 ceph-mon[36167]: pgmap v656: 129 pgs: 129 active+clean; 127 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 776 op/s 2024-09-10T08:48:43.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:42 smithi111 ceph-mon[35220]: pgmap v656: 129 pgs: 129 active+clean; 127 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 776 op/s 2024-09-10T08:48:43.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:42 smithi089 ceph-mon[30757]: pgmap v656: 129 pgs: 129 active+clean; 127 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 776 op/s 2024-09-10T08:48:45.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:44 smithi149 ceph-mon[36167]: pgmap v657: 129 pgs: 129 active+clean; 118 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 700 op/s 2024-09-10T08:48:45.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:44 smithi111 ceph-mon[35220]: pgmap v657: 129 pgs: 129 active+clean; 118 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 700 op/s 2024-09-10T08:48:45.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:44 smithi089 ceph-mon[30757]: pgmap v657: 129 pgs: 129 active+clean; 118 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 700 op/s 2024-09-10T08:48:47.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:46 smithi149 ceph-mon[36167]: pgmap v658: 129 pgs: 129 active+clean; 117 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 1006 KiB/s wr, 622 op/s 2024-09-10T08:48:47.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:46 smithi111 ceph-mon[35220]: pgmap v658: 129 pgs: 129 active+clean; 117 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 1006 KiB/s wr, 622 op/s 2024-09-10T08:48:47.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:46 smithi089 ceph-mon[30757]: pgmap v658: 129 pgs: 129 active+clean; 117 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 1006 KiB/s wr, 622 op/s 2024-09-10T08:48:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.828+0000 7ff953200640 1 -- 172.21.15.89:0/2898640721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff954100920 msgr2=0x7ff954100d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.828+0000 7ff953200640 1 --2- 172.21.15.89:0/2898640721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff954100920 0x7ff954100d80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009900 tx=0x7ff93c02a410 comp rx=0 tx=0).stop 2024-09-10T08:48:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.829+0000 7ff953200640 1 -- 172.21.15.89:0/2898640721 shutdown_connections 2024-09-10T08:48:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.829+0000 7ff953200640 1 --2- 172.21.15.89:0/2898640721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9541012c0 0x7ff954101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.829+0000 7ff953200640 1 --2- 172.21.15.89:0/2898640721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff954100920 0x7ff954100d80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.829+0000 7ff953200640 1 --2- 172.21.15.89:0/2898640721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9540ff8b0 0x7ff9540ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.829+0000 7ff953200640 1 -- 172.21.15.89:0/2898640721 >> 172.21.15.89:0/2898640721 conn(0x7ff9540fb2e0 msgr2=0x7ff9540fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.831+0000 7ff953200640 1 -- 172.21.15.89:0/2898640721 shutdown_connections 2024-09-10T08:48:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.831+0000 7ff953200640 1 -- 172.21.15.89:0/2898640721 wait complete. 2024-09-10T08:48:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.832+0000 7ff953200640 1 Processor -- start 2024-09-10T08:48:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.833+0000 7ff953200640 1 -- start start 2024-09-10T08:48:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.833+0000 7ff953200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9540ff8b0 0x7ff95419e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.833+0000 7ff953200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff954100920 0x7ff95419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.833+0000 7ff953200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9541012c0 0x7ff9541a5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.833+0000 7ff953200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff95410f7f0 con 0x7ff954100920 2024-09-10T08:48:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.833+0000 7ff953200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff95410f670 con 0x7ff9540ff8b0 2024-09-10T08:48:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.833+0000 7ff953200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff95410f970 con 0x7ff9541012c0 2024-09-10T08:48:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.833+0000 7ff951e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9540ff8b0 0x7ff95419e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.833+0000 7ff951e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9540ff8b0 0x7ff95419e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:49266/0 (socket says 172.21.15.89:49266) 2024-09-10T08:48:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.833+0000 7ff951e00640 1 -- 172.21.15.89:0/1303930334 learned_addr learned my addr 172.21.15.89:0/1303930334 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.834+0000 7ff951400640 1 --2- 172.21.15.89:0/1303930334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff954100920 0x7ff95419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.834+0000 7ff952800640 1 --2- 172.21.15.89:0/1303930334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9541012c0 0x7ff9541a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.834+0000 7ff952800640 1 -- 172.21.15.89:0/1303930334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9540ff8b0 msgr2=0x7ff95419e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.834+0000 7ff952800640 1 --2- 172.21.15.89:0/1303930334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9540ff8b0 0x7ff95419e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.834+0000 7ff952800640 1 -- 172.21.15.89:0/1303930334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff954100920 msgr2=0x7ff95419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.834+0000 7ff952800640 1 --2- 172.21.15.89:0/1303930334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff954100920 0x7ff95419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.834+0000 7ff952800640 1 -- 172.21.15.89:0/1303930334 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9541a5ed0 con 0x7ff9541012c0 2024-09-10T08:48:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.834+0000 7ff951400640 1 --2- 172.21.15.89:0/1303930334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff954100920 0x7ff95419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.834+0000 7ff951e00640 1 --2- 172.21.15.89:0/1303930334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9540ff8b0 0x7ff95419e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.834+0000 7ff952800640 1 --2- 172.21.15.89:0/1303930334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9541012c0 0x7ff9541a5860 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00e970 tx=0x7ff94c00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.835+0000 7ff94aa00640 1 -- 172.21.15.89:0/1303930334 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c00ccc0 con 0x7ff9541012c0 2024-09-10T08:48:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.835+0000 7ff953200640 1 -- 172.21.15.89:0/1303930334 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9541a61c0 con 0x7ff9541012c0 2024-09-10T08:48:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.835+0000 7ff94aa00640 1 -- 172.21.15.89:0/1303930334 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff94c005020 con 0x7ff9541012c0 2024-09-10T08:48:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.835+0000 7ff94aa00640 1 -- 172.21.15.89:0/1303930334 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c004a00 con 0x7ff9541012c0 2024-09-10T08:48:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.835+0000 7ff953200640 1 -- 172.21.15.89:0/1303930334 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9541a66e0 con 0x7ff9541012c0 2024-09-10T08:48:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.841+0000 7ff94aa00640 1 -- 172.21.15.89:0/1303930334 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff94c0051d0 con 0x7ff9541012c0 2024-09-10T08:48:47.844 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.843+0000 7ff95c083740 1 Processor -- start 2024-09-10T08:48:47.846 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.845+0000 7ff95c083740 1 -- start start 2024-09-10T08:48:47.846 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.845+0000 7ff95c083740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c5450f5260 0x55c5450f5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:47.846 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.845+0000 7ff95c083740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450f5ba0 0x55c5450fa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:47.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.845+0000 7ff95c083740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450fac30 0x55c5450fd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:47.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.845+0000 7ff95c083740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c54505a100 con 0x55c5450f5260 2024-09-10T08:48:47.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.845+0000 7ff95c083740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c5450021b0 con 0x55c5450fac30 2024-09-10T08:48:47.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.845+0000 7ff95c083740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c5450560c0 con 0x55c5450f5ba0 2024-09-10T08:48:47.847 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.846+0000 7ff951e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450f5ba0 0x55c5450fa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:47.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.846+0000 7ff951e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450f5ba0 0x55c5450fa660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:51124/0 (socket says 172.21.15.89:51124) 2024-09-10T08:48:47.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.846+0000 7ff951e00640 1 -- 172.21.15.89:0/1997307193 learned_addr learned my addr 172.21.15.89:0/1997307193 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:47.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.846+0000 7ff94aa00640 1 --2- 172.21.15.89:0/1303930334 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff9380a2c00 0x7ff9380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:47.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.847+0000 7ff951400640 1 --2- 172.21.15.89:0/1997307193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c5450f5260 0x55c5450f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:47.848 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.847+0000 7ff94aa00640 1 -- 172.21.15.89:0/1303930334 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff94c0cf470 con 0x7ff9541012c0 2024-09-10T08:48:47.849 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.847+0000 7ff951400640 1 --2- 172.21.15.89:0/1997307193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450fac30 0x55c5450fd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.847+0000 7ff952800640 1 --2- 172.21.15.89:0/1303930334 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff9380a2c00 0x7ff9380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.848+0000 7ff951400640 1 -- 172.21.15.89:0/1997307193 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450f5ba0 msgr2=0x55c5450fa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.848+0000 7ff951400640 1 --2- 172.21.15.89:0/1997307193 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450f5ba0 0x55c5450fa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.848+0000 7ff951400640 1 -- 172.21.15.89:0/1997307193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450fac30 msgr2=0x55c5450fd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.848+0000 7ff951400640 1 --2- 172.21.15.89:0/1997307193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450fac30 0x55c5450fd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.848+0000 7ff951400640 1 -- 172.21.15.89:0/1997307193 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c544d6d210 con 0x55c5450f5260 2024-09-10T08:48:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.849+0000 7ff951400640 1 --2- 172.21.15.89:0/1997307193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c5450f5260 0x55c5450f5660 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff93c02a930 tx=0x7ff93c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ecfdff067b93f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.849+0000 7ff952800640 1 --2- 172.21.15.89:0/1303930334 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff9380a2c00 0x7ff9380a50c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff94c01a040 tx=0x7ff94c005750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.849+0000 7ff943400640 1 -- 172.21.15.89:0/1997307193 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c041020 con 0x55c5450f5260 2024-09-10T08:48:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.849+0000 7ff943400640 1 -- 172.21.15.89:0/1997307193 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff93c0046c0 con 0x55c5450f5260 2024-09-10T08:48:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.849+0000 7ff943400640 1 -- 172.21.15.89:0/1997307193 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c041020 con 0x55c5450f5260 2024-09-10T08:48:47.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.851+0000 7ff95c083740 1 -- 172.21.15.89:0/1997307193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c5450f5260 msgr2=0x55c5450f5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:47.852 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.851+0000 7ff95c083740 1 --2- 172.21.15.89:0/1997307193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c5450f5260 0x55c5450f5660 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff93c02a930 tx=0x7ff93c000bc0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.851+0000 7ff95c083740 1 -- 172.21.15.89:0/1997307193 shutdown_connections 2024-09-10T08:48:47.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.851+0000 7ff95c083740 1 --2- 172.21.15.89:0/1997307193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450fac30 0x55c5450fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.851+0000 7ff95c083740 1 --2- 172.21.15.89:0/1997307193 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450f5ba0 0x55c5450fa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.851+0000 7ff95c083740 1 --2- 172.21.15.89:0/1997307193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c5450f5260 0x55c5450f5660 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.851+0000 7ff95c083740 1 -- 172.21.15.89:0/1997307193 >> 172.21.15.89:0/1997307193 conn(0x55c5450f08d0 msgr2=0x55c5450f0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:47.853 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.853+0000 7ff95c083740 1 -- 172.21.15.89:0/1997307193 shutdown_connections 2024-09-10T08:48:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.853+0000 7ff95c083740 1 -- 172.21.15.89:0/1997307193 wait complete. 2024-09-10T08:48:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.854+0000 7ff95c083740 1 -- start start 2024-09-10T08:48:47.855 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.855+0000 7ff95c083740 1 Processor -- start 2024-09-10T08:48:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.855+0000 7ff95c083740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450f5260 0x55c5450ff4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.855+0000 7ff95c083740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5ba0 0x55c5450ff9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.855+0000 7ff95c083740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c5450fac30 0x55c545106a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.855+0000 7ff95c083740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c5450fdba0 con 0x55c5450fac30 2024-09-10T08:48:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.855+0000 7ff95c083740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c544fe3090 con 0x55c5450f5ba0 2024-09-10T08:48:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.855+0000 7ff95c083740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c5450fdd20 con 0x55c5450f5260 2024-09-10T08:48:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.855+0000 7ff95c083740 10 client.0 _ll_register_callbacks cb 0x55c545037f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.856+0000 7ff951e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5ba0 0x55c5450ff9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.856+0000 7ff951e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5ba0 0x55c5450ff9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:49286/0 (socket says 172.21.15.89:49286) 2024-09-10T08:48:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.856+0000 7ff951e00640 1 -- 172.21.15.89:0/3340605129 learned_addr learned my addr 172.21.15.89:0/3340605129 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.856+0000 7ff951400640 1 --2- 172.21.15.89:0/3340605129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450f5260 0x55c5450ff4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.856+0000 7ff951e00640 1 -- 172.21.15.89:0/3340605129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450f5260 msgr2=0x55c5450ff4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.857+0000 7ff951e00640 1 --2- 172.21.15.89:0/3340605129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450f5260 0x55c5450ff4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.857+0000 7ff951e00640 1 -- 172.21.15.89:0/3340605129 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c5450fac30 msgr2=0x55c545106a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.857+0000 7ff951e00640 1 --2- 172.21.15.89:0/3340605129 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c5450fac30 0x55c545106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.857+0000 7ff951e00640 1 -- 172.21.15.89:0/3340605129 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c544e39620 con 0x55c5450f5ba0 2024-09-10T08:48:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.857+0000 7ff951400640 1 --2- 172.21.15.89:0/3340605129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450f5260 0x55c5450ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.858+0000 7ff937e00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:48:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.858+0000 7ff951e00640 1 --2- 172.21.15.89:0/3340605129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5ba0 0x55c5450ff9e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff944010ae0 tx=0x7ff94400e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54d953b73caa772d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.859+0000 7ff937e00640 1 -- 172.21.15.89:0/3340605129 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944004b20 con 0x55c5450f5ba0 2024-09-10T08:48:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.859+0000 7ff937e00640 1 -- 172.21.15.89:0/3340605129 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff944004cc0 con 0x55c5450f5ba0 2024-09-10T08:48:47.860 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.859+0000 7ff95c083740 1 -- 172.21.15.89:0/3340605129 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff93c0046c0 con 0x55c5450f5ba0 2024-09-10T08:48:47.861 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.860+0000 7ff937e00640 1 -- 172.21.15.89:0/3340605129 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff944003970 con 0x55c5450f5ba0 2024-09-10T08:48:47.861 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.860+0000 7ff95c083740 10 client.26975 fetch_fsmap learned FSMap version 32 2024-09-10T08:48:47.861 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.860+0000 7ff95c083740 1 -- 172.21.15.89:0/3340605129 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c54511a940 con 0x55c5450f5ba0 2024-09-10T08:48:47.861 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.861+0000 7ff937e00640 1 -- 172.21.15.89:0/3340605129 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff93c0046c0 con 0x55c5450f5ba0 2024-09-10T08:48:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.861+0000 7ff937e00640 1 -- 172.21.15.89:0/3340605129 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7ff944005020 con 0x55c5450f5ba0 2024-09-10T08:48:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.861+0000 7ff95c083740 10 client.26975 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T08:48:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.861+0000 7ff95c083740 10 client.26975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.861+0000 7ff95c083740 20 client.26975 populate_metadata read hostname 'smithi089' 2024-09-10T08:48:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.861+0000 7ff95c083740 1 --2- 172.21.15.89:0/3340605129 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c54511c090 0x55c54511e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.861+0000 7ff95c083740 4 client.26975 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.861+0000 7ff95c083740 1 -- 172.21.15.89:0/3340605129 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c545067830 con 0x55c54511c090 2024-09-10T08:48:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.863+0000 7ff951400640 1 --2- 172.21.15.89:0/3340605129 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c54511c090 0x55c54511e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:47.867 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.866+0000 7ff951400640 1 --2- 172.21.15.89:0/3340605129 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c54511c090 0x55c54511e4d0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8f4e7dca9749c54 server_cookie=b16ddb4a80fb4639 in_seq=0 out_seq=0 2024-09-10T08:48:47.867 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.866+0000 7ff937e00640 10 client.26975 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:48:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.869+0000 7ff937e00640 1 -- 172.21.15.89:0/3340605129 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c545067830 con 0x55c54511c090 2024-09-10T08:48:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.869+0000 7ff937e00640 10 client.26975 handle_command_reply: tid=0 2024-09-10T08:48:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.869+0000 7ff95c083740 1 client.26975 shutdown 2024-09-10T08:48:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.869+0000 7ff95c083740 1 -- 172.21.15.89:0/3340605129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5ba0 msgr2=0x55c5450ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.869+0000 7ff95c083740 1 --2- 172.21.15.89:0/3340605129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5ba0 0x55c5450ff9e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff944010ae0 tx=0x7ff94400e350 comp rx=0 tx=0).stop 2024-09-10T08:48:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.870+0000 7ff95c083740 1 -- 172.21.15.89:0/3340605129 shutdown_connections 2024-09-10T08:48:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.870+0000 7ff95c083740 1 --2- 172.21.15.89:0/3340605129 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c54511c090 0x55c54511e4d0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.870+0000 7ff95c083740 1 --2- 172.21.15.89:0/3340605129 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c5450fac30 0x55c545106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.870+0000 7ff95c083740 1 --2- 172.21.15.89:0/3340605129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5ba0 0x55c5450ff9e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.870+0000 7ff95c083740 1 --2- 172.21.15.89:0/3340605129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450f5260 0x55c5450ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.870+0000 7ff95c083740 1 -- 172.21.15.89:0/3340605129 >> 172.21.15.89:0/3340605129 conn(0x55c5450f08d0 msgr2=0x55c5450f20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.870+0000 7ff937e00640 0 client.26975 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:48:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.870+0000 7ff95c083740 1 -- 172.21.15.89:0/3340605129 shutdown_connections 2024-09-10T08:48:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.871+0000 7ff95c083740 1 -- 172.21.15.89:0/3340605129 wait complete. 2024-09-10T08:48:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.871+0000 7ff95c083740 20 client.26975 trim_cache size 0 max 16384 2024-09-10T08:48:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.903+0000 7ff95c083740 1 Processor -- start 2024-09-10T08:48:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.904+0000 7ff95c083740 1 -- start start 2024-09-10T08:48:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.905+0000 7ff95c083740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450f5260 0x55c5450ef900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.905+0000 7ff95c083740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5ba0 0x55c5451069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.905+0000 7ff95c083740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c5450fac30 0x55c5450f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.905+0000 7ff95c083740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541aaf20 con 0x55c5450fac30 2024-09-10T08:48:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.905+0000 7ff95c083740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541a8790 con 0x55c5450f5ba0 2024-09-10T08:48:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.905+0000 7ff95c083740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff95419d310 con 0x55c5450f5260 2024-09-10T08:48:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.905+0000 7ff951e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5ba0 0x55c5451069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.906+0000 7ff951e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5ba0 0x55c5451069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:49302/0 (socket says 172.21.15.89:49302) 2024-09-10T08:48:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.906+0000 7ff951e00640 1 -- 172.21.15.89:0/3248642394 learned_addr learned my addr 172.21.15.89:0/3248642394 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.906+0000 7ff951e00640 1 -- 172.21.15.89:0/3248642394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450f5260 msgr2=0x55c5450ef900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.906+0000 7ff951e00640 1 --2- 172.21.15.89:0/3248642394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450f5260 0x55c5450ef900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.906+0000 7ff951e00640 1 -- 172.21.15.89:0/3248642394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c5450fac30 msgr2=0x55c5450f1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.906+0000 7ff951e00640 1 --2- 172.21.15.89:0/3248642394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c5450fac30 0x55c5450f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.906+0000 7ff951e00640 1 -- 172.21.15.89:0/3248642394 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5450f06f0 con 0x55c5450f5ba0 2024-09-10T08:48:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.907+0000 7ff951e00640 1 --2- 172.21.15.89:0/3248642394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5ba0 0x55c5451069e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff9440054c0 tx=0x7ff944007f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9751d9eb202ea5b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.907+0000 7ff942a00640 1 -- 172.21.15.89:0/3248642394 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944034070 con 0x55c5450f5ba0 2024-09-10T08:48:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.908+0000 7ff942a00640 1 -- 172.21.15.89:0/3248642394 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff94402b040 con 0x55c5450f5ba0 2024-09-10T08:48:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.908+0000 7ff942a00640 1 -- 172.21.15.89:0/3248642394 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944011440 con 0x55c5450f5ba0 2024-09-10T08:48:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.909+0000 7ff95c083740 1 -- 172.21.15.89:0/3248642394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5ba0 msgr2=0x55c5451069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.909+0000 7ff95c083740 1 --2- 172.21.15.89:0/3248642394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5ba0 0x55c5451069e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff9440054c0 tx=0x7ff944007f80 comp rx=0 tx=0).stop 2024-09-10T08:48:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.909+0000 7ff95c083740 1 -- 172.21.15.89:0/3248642394 shutdown_connections 2024-09-10T08:48:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.909+0000 7ff95c083740 1 --2- 172.21.15.89:0/3248642394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c5450fac30 0x55c5450f1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.909+0000 7ff95c083740 1 --2- 172.21.15.89:0/3248642394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5ba0 0x55c5451069e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.909+0000 7ff95c083740 1 --2- 172.21.15.89:0/3248642394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450f5260 0x55c5450ef900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.910+0000 7ff95c083740 1 -- 172.21.15.89:0/3248642394 >> 172.21.15.89:0/3248642394 conn(0x55c5450f08d0 msgr2=0x55c5450ef1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.910+0000 7ff95c083740 1 -- 172.21.15.89:0/3248642394 shutdown_connections 2024-09-10T08:48:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.910+0000 7ff95c083740 1 -- 172.21.15.89:0/3248642394 wait complete. 2024-09-10T08:48:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.911+0000 7ff95c083740 1 -- start start 2024-09-10T08:48:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.911+0000 7ff95c083740 1 Processor -- start 2024-09-10T08:48:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.912+0000 7ff95c083740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5260 0x55c545104200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.913+0000 7ff95c083740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c5450f5ba0 0x55c545104740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.913+0000 7ff95c083740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450fac30 0x55c545105750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.913+0000 7ff95c083740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541a6c50 con 0x55c5450f5ba0 2024-09-10T08:48:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.913+0000 7ff95c083740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541a80c0 con 0x55c5450f5260 2024-09-10T08:48:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.913+0000 7ff95c083740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541a7320 con 0x55c5450fac30 2024-09-10T08:48:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.913+0000 7ff95c083740 10 client.0 _ll_register_callbacks cb 0x7ff944007720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.913+0000 7ff951e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c5450f5ba0 0x55c545104740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.913+0000 7ff951400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5260 0x55c545104200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.913+0000 7ff951400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5260 0x55c545104200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:49318/0 (socket says 172.21.15.89:49318) 2024-09-10T08:48:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.913+0000 7ff951400640 1 -- 172.21.15.89:0/1032368943 learned_addr learned my addr 172.21.15.89:0/1032368943 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.914+0000 7ff951400640 1 --2- 172.21.15.89:0/1032368943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450fac30 0x55c545105750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.915+0000 7ff951400640 1 -- 172.21.15.89:0/1032368943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450fac30 msgr2=0x55c545105750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.915+0000 7ff951400640 1 --2- 172.21.15.89:0/1032368943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450fac30 0x55c545105750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.915+0000 7ff951400640 1 -- 172.21.15.89:0/1032368943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c5450f5ba0 msgr2=0x55c545104740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.915+0000 7ff951400640 1 --2- 172.21.15.89:0/1032368943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c5450f5ba0 0x55c545104740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.915+0000 7ff951400640 1 -- 172.21.15.89:0/1032368943 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c545105d60 con 0x55c5450f5260 2024-09-10T08:48:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.915+0000 7ff951e00640 1 --2- 172.21.15.89:0/1032368943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c5450f5ba0 0x55c545104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.916+0000 7ff951400640 1 --2- 172.21.15.89:0/1032368943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5260 0x55c545104200 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff93c006010 tx=0x7ff93c031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2719a8f06ede926d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.916+0000 7ff937e00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:48:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.917+0000 7ff937e00640 1 -- 172.21.15.89:0/1032368943 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c041070 con 0x55c5450f5260 2024-09-10T08:48:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.917+0000 7ff937e00640 1 -- 172.21.15.89:0/1032368943 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff93c031bb0 con 0x55c5450f5260 2024-09-10T08:48:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.917+0000 7ff95c083740 1 -- 172.21.15.89:0/1032368943 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff94402b040 con 0x55c5450f5260 2024-09-10T08:48:47.918 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.918+0000 7ff937e00640 1 -- 172.21.15.89:0/1032368943 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff93c035050 con 0x55c5450f5260 2024-09-10T08:48:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.919+0000 7ff95c083740 10 client.26981 fetch_fsmap learned FSMap version 32 2024-09-10T08:48:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.919+0000 7ff95c083740 1 -- 172.21.15.89:0/1032368943 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c545186f30 con 0x55c5450f5260 2024-09-10T08:48:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.919+0000 7ff937e00640 1 -- 172.21.15.89:0/1032368943 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff94402b040 con 0x55c5450f5260 2024-09-10T08:48:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.920+0000 7ff937e00640 1 -- 172.21.15.89:0/1032368943 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7ff93c03d510 con 0x55c5450f5260 2024-09-10T08:48:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.921+0000 7ff95c083740 10 client.26981 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T08:48:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.921+0000 7ff95c083740 10 client.26981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.921+0000 7ff95c083740 20 client.26981 populate_metadata read hostname 'smithi089' 2024-09-10T08:48:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.921+0000 7ff95c083740 1 --2- 172.21.15.89:0/1032368943 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c545187480 0x55c5451898a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.921+0000 7ff95c083740 4 client.26981 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:48:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.921+0000 7ff95c083740 1 -- 172.21.15.89:0/1032368943 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c545061b80 con 0x55c545187480 2024-09-10T08:48:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.921+0000 7ff951e00640 1 --2- 172.21.15.89:0/1032368943 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c545187480 0x55c5451898a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.922+0000 7ff951e00640 1 --2- 172.21.15.89:0/1032368943 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c545187480 0x55c5451898a0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12c0454d4f0b05ca server_cookie=d41c4245c618644c in_seq=0 out_seq=0 2024-09-10T08:48:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:47.922+0000 7ff937e00640 10 client.26981 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:48:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.749+0000 7ff937e00640 1 -- 172.21.15.89:0/1032368943 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c545061b80 con 0x55c545187480 2024-09-10T08:48:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.749+0000 7ff937e00640 10 client.26981 handle_command_reply: tid=0 2024-09-10T08:48:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.749+0000 7ff95c083740 1 client.26981 shutdown 2024-09-10T08:48:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.750+0000 7ff95c083740 1 -- 172.21.15.89:0/1032368943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5260 msgr2=0x55c545104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.750+0000 7ff95c083740 1 --2- 172.21.15.89:0/1032368943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5260 0x55c545104200 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff93c006010 tx=0x7ff93c031400 comp rx=0 tx=0).stop 2024-09-10T08:48:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.750+0000 7ff95c083740 1 -- 172.21.15.89:0/1032368943 shutdown_connections 2024-09-10T08:48:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.750+0000 7ff95c083740 1 --2- 172.21.15.89:0/1032368943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5450fac30 0x55c545105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.750+0000 7ff95c083740 1 --2- 172.21.15.89:0/1032368943 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x55c545187480 0x55c5451898a0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.750+0000 7ff95c083740 1 --2- 172.21.15.89:0/1032368943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c5450f5ba0 0x55c545104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.750+0000 7ff95c083740 1 --2- 172.21.15.89:0/1032368943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5450f5260 0x55c545104200 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.750+0000 7ff95c083740 1 -- 172.21.15.89:0/1032368943 >> 172.21.15.89:0/1032368943 conn(0x55c5450f08d0 msgr2=0x55c5450efa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.750+0000 7ff937e00640 0 client.26981 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:48:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.751+0000 7ff95c083740 1 -- 172.21.15.89:0/1032368943 shutdown_connections 2024-09-10T08:48:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.751+0000 7ff95c083740 1 -- 172.21.15.89:0/1032368943 wait complete. 2024-09-10T08:48:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.751+0000 7ff95c083740 20 client.26981 trim_cache size 0 max 16384 2024-09-10T08:48:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:{ 2024-09-10T08:48:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "message": "", 2024-09-10T08:48:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout: "return_code": 0 2024-09-10T08:48:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stdout:} 2024-09-10T08:48:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.752+0000 7ff953200640 1 -- 172.21.15.89:0/1303930334 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff9380a2c00 msgr2=0x7ff9380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.752+0000 7ff953200640 1 --2- 172.21.15.89:0/1303930334 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff9380a2c00 0x7ff9380a50c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff94c01a040 tx=0x7ff94c005750 comp rx=0 tx=0).stop 2024-09-10T08:48:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.752+0000 7ff953200640 1 -- 172.21.15.89:0/1303930334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9541012c0 msgr2=0x7ff9541a5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.752+0000 7ff953200640 1 --2- 172.21.15.89:0/1303930334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9541012c0 0x7ff9541a5860 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00e970 tx=0x7ff94c00ee30 comp rx=0 tx=0).stop 2024-09-10T08:48:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.753+0000 7ff953200640 1 -- 172.21.15.89:0/1303930334 shutdown_connections 2024-09-10T08:48:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.753+0000 7ff953200640 1 --2- 172.21.15.89:0/1303930334 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff9380a2c00 0x7ff9380a50c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.753+0000 7ff953200640 1 --2- 172.21.15.89:0/1303930334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9541012c0 0x7ff9541a5860 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.753+0000 7ff953200640 1 --2- 172.21.15.89:0/1303930334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff954100920 0x7ff95419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.753+0000 7ff953200640 1 --2- 172.21.15.89:0/1303930334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff9540ff8b0 0x7ff95419e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.753+0000 7ff953200640 1 -- 172.21.15.89:0/1303930334 >> 172.21.15.89:0/1303930334 conn(0x7ff9540fb2e0 msgr2=0x7ff9540fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.753+0000 7ff953200640 1 -- 172.21.15.89:0/1303930334 shutdown_connections 2024-09-10T08:48:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi089.front.sepia.ceph.com.smithi089.stderr:2024-09-10T08:48:48.754+0000 7ff953200640 1 -- 172.21.15.89:0/1303930334 wait complete. 2024-09-10T08:48:49.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:48 smithi149 ceph-mon[36167]: pgmap v659: 129 pgs: 129 active+clean; 99 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 802 op/s 2024-09-10T08:48:49.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:48 smithi111 ceph-mon[35220]: pgmap v659: 129 pgs: 129 active+clean; 99 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 802 op/s 2024-09-10T08:48:49.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:48 smithi089 ceph-mon[30757]: pgmap v659: 129 pgs: 129 active+clean; 99 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 802 op/s 2024-09-10T08:48:51.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:50 smithi149 ceph-mon[36167]: pgmap v660: 129 pgs: 129 active+clean; 94 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 561 op/s 2024-09-10T08:48:51.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:50 smithi111 ceph-mon[35220]: pgmap v660: 129 pgs: 129 active+clean; 94 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 561 op/s 2024-09-10T08:48:51.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:50 smithi089 ceph-mon[30757]: pgmap v660: 129 pgs: 129 active+clean; 94 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 561 op/s 2024-09-10T08:48:52.739 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-09-10T08:48:52.739 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-10T08:48:53.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:52 smithi089 ceph-mon[30757]: pgmap v661: 129 pgs: 129 active+clean; 73 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 724 op/s 2024-09-10T08:48:53.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:52 smithi149 ceph-mon[36167]: pgmap v661: 129 pgs: 129 active+clean; 73 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 724 op/s 2024-09-10T08:48:53.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:52 smithi111 ceph-mon[35220]: pgmap v661: 129 pgs: 129 active+clean; 73 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 724 op/s 2024-09-10T08:48:53.932 DEBUG:teuthology.parallel:result is None 2024-09-10T08:48:53.933 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T08:48:54.001 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T08:48:54.001 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-10T08:48:54.014 INFO:tasks.quiescer:joining Quiescers 2024-09-10T08:48:54.014 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-10T08:48:54.014 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:48:54.382 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.893+0000 7f16a9800640 1 -- 172.21.15.89:0/622940836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16a4103ff0 msgr2=0x7f16a41043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.893+0000 7f16a9800640 1 --2- 172.21.15.89:0/622940836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16a4103ff0 0x7f16a41043d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1694009900 tx=0x7f169402a410 comp rx=0 tx=0).stop 2024-09-10T08:48:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.893+0000 7f16a9800640 1 -- 172.21.15.89:0/622940836 shutdown_connections 2024-09-10T08:48:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.893+0000 7f16a9800640 1 --2- 172.21.15.89:0/622940836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16a4109120 0x7f16a4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.893+0000 7f16a9800640 1 --2- 172.21.15.89:0/622940836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16a41049a0 0x7f16a41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:54.909 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.893+0000 7f16a9800640 1 --2- 172.21.15.89:0/622940836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16a4103ff0 0x7f16a41043d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.893+0000 7f16a9800640 1 -- 172.21.15.89:0/622940836 >> 172.21.15.89:0/622940836 conn(0x7f16a40ffa90 msgr2=0x7f16a4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.893+0000 7f16a9800640 1 -- 172.21.15.89:0/622940836 shutdown_connections 2024-09-10T08:48:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.893+0000 7f16a9800640 1 -- 172.21.15.89:0/622940836 wait complete. 2024-09-10T08:48:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.894+0000 7f16a9800640 1 Processor -- start 2024-09-10T08:48:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.895+0000 7f16a9800640 1 -- start start 2024-09-10T08:48:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.895+0000 7f16a9800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16a4103ff0 0x7f16a419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.895+0000 7f16a9800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16a41049a0 0x7f16a419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.895+0000 7f16a9800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16a4109120 0x7f16a419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.895+0000 7f16a9800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f16a4114120 con 0x7f16a4109120 2024-09-10T08:48:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.895+0000 7f16a9800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f16a4113fa0 con 0x7f16a41049a0 2024-09-10T08:48:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.895+0000 7f16a9800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f16a41142a0 con 0x7f16a4103ff0 2024-09-10T08:48:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.895+0000 7f16a3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16a4109120 0x7f16a419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:54.910 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.895+0000 7f16a3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16a4109120 0x7f16a419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42030/0 (socket says 172.21.15.89:42030) 2024-09-10T08:48:54.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.895+0000 7f16a3400640 1 -- 172.21.15.89:0/2730212286 learned_addr learned my addr 172.21.15.89:0/2730212286 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:54.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.895+0000 7f16a2a00640 1 --2- 172.21.15.89:0/2730212286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16a4103ff0 0x7f16a419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:54.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.895+0000 7f16a2000640 1 --2- 172.21.15.89:0/2730212286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16a41049a0 0x7f16a419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:54.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.896+0000 7f16a3400640 1 -- 172.21.15.89:0/2730212286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16a4103ff0 msgr2=0x7f16a419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:54.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.896+0000 7f16a3400640 1 --2- 172.21.15.89:0/2730212286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16a4103ff0 0x7f16a419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:54.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.896+0000 7f16a3400640 1 -- 172.21.15.89:0/2730212286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16a41049a0 msgr2=0x7f16a419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:54.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.896+0000 7f16a3400640 1 --2- 172.21.15.89:0/2730212286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16a41049a0 0x7f16a419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:54.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.896+0000 7f16a3400640 1 -- 172.21.15.89:0/2730212286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a419f590 con 0x7f16a4109120 2024-09-10T08:48:54.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.896+0000 7f16a2000640 1 --2- 172.21.15.89:0/2730212286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16a41049a0 0x7f16a419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:54.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.896+0000 7f16a3400640 1 --2- 172.21.15.89:0/2730212286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16a4109120 0x7f16a419eeb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f169c00eaa0 tx=0x7f169c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:54.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.896+0000 7f169b400640 1 -- 172.21.15.89:0/2730212286 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c00cdb0 con 0x7f16a4109120 2024-09-10T08:48:54.911 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.896+0000 7f169b400640 1 -- 172.21.15.89:0/2730212286 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f169c005020 con 0x7f16a4109120 2024-09-10T08:48:54.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.896+0000 7f169b400640 1 -- 172.21.15.89:0/2730212286 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c0116b0 con 0x7f16a4109120 2024-09-10T08:48:54.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.896+0000 7f16a9800640 1 -- 172.21.15.89:0/2730212286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a419f880 con 0x7f16a4109120 2024-09-10T08:48:54.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.897+0000 7f16a9800640 1 -- 172.21.15.89:0/2730212286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a41a7160 con 0x7f16a4109120 2024-09-10T08:48:54.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.897+0000 7f16a9800640 1 -- 172.21.15.89:0/2730212286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a41043d0 con 0x7f16a4109120 2024-09-10T08:48:54.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.899+0000 7f169b400640 1 -- 172.21.15.89:0/2730212286 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f169c005560 con 0x7f16a4109120 2024-09-10T08:48:54.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.901+0000 7f169b400640 1 --2- 172.21.15.89:0/2730212286 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f16880a2b80 0x7f16880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:54.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.901+0000 7f169b400640 1 -- 172.21.15.89:0/2730212286 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f169c0cf5b0 con 0x7f16a4109120 2024-09-10T08:48:54.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.901+0000 7f16a2a00640 1 --2- 172.21.15.89:0/2730212286 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f16880a2b80 0x7f16880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:54.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.902+0000 7f16a2a00640 1 --2- 172.21.15.89:0/2730212286 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f16880a2b80 0x7f16880a5040 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1694009230 tx=0x7f16940058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:54.912 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:54.905+0000 7f169b400640 1 -- 172.21.15.89:0/2730212286 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f169c016030 con 0x7f16a4109120 2024-09-10T08:48:55.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:54 smithi089 ceph-mon[30757]: pgmap v662: 129 pgs: 129 active+clean; 64 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.0 MiB/s wr, 655 op/s 2024-09-10T08:48:55.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:55.118+0000 7f16a9800640 1 -- 172.21.15.89:0/2730212286 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16a402d7c0 con 0x7f16a4109120 2024-09-10T08:48:55.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:55.119+0000 7f169b400640 1 -- 172.21.15.89:0/2730212286 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11922 (secure 0 0 0) 0x7f169c09ba90 con 0x7f16a4109120 2024-09-10T08:48:55.120 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:48:55.120 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":32,"btime":"2024-09-10T08:48:20:890997+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:48:20.485348+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":158,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24554,24518,24496]},"id":1}]} 2024-09-10T08:48:55.121 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 32 2024-09-10T08:48:55.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:55.121+0000 7f16a9800640 1 -- 172.21.15.89:0/2730212286 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f16880a2b80 msgr2=0x7f16880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:55.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:55.121+0000 7f16a9800640 1 --2- 172.21.15.89:0/2730212286 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f16880a2b80 0x7f16880a5040 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1694009230 tx=0x7f16940058d0 comp rx=0 tx=0).stop 2024-09-10T08:48:55.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:55.121+0000 7f16a9800640 1 -- 172.21.15.89:0/2730212286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16a4109120 msgr2=0x7f16a419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:55.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:55.121+0000 7f16a9800640 1 --2- 172.21.15.89:0/2730212286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16a4109120 0x7f16a419eeb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f169c00eaa0 tx=0x7f169c00ef60 comp rx=0 tx=0).stop 2024-09-10T08:48:55.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:55.121+0000 7f16a9800640 1 -- 172.21.15.89:0/2730212286 shutdown_connections 2024-09-10T08:48:55.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:55.121+0000 7f16a9800640 1 --2- 172.21.15.89:0/2730212286 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f16880a2b80 0x7f16880a5040 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:55.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:55.121+0000 7f16a9800640 1 --2- 172.21.15.89:0/2730212286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16a4109120 0x7f16a419eeb0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:55.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:55.121+0000 7f16a9800640 1 --2- 172.21.15.89:0/2730212286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16a41049a0 0x7f16a419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:55.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:55.121+0000 7f16a9800640 1 --2- 172.21.15.89:0/2730212286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16a4103ff0 0x7f16a419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:55.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:55.121+0000 7f16a9800640 1 -- 172.21.15.89:0/2730212286 >> 172.21.15.89:0/2730212286 conn(0x7f16a40ffa90 msgr2=0x7f16a4100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:55.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:55.122+0000 7f16a9800640 1 -- 172.21.15.89:0/2730212286 shutdown_connections 2024-09-10T08:48:55.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:55.122+0000 7f16a9800640 1 -- 172.21.15.89:0/2730212286 wait complete. 2024-09-10T08:48:55.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:54 smithi111 ceph-mon[35220]: pgmap v662: 129 pgs: 129 active+clean; 64 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.0 MiB/s wr, 655 op/s 2024-09-10T08:48:55.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:54 smithi149 ceph-mon[36167]: pgmap v662: 129 pgs: 129 active+clean; 64 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.0 MiB/s wr, 655 op/s 2024-09-10T08:48:56.020 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24554 quiesce db --cancel --all' 2024-09-10T08:48:56.021 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24554 quiesce db --cancel --all 2024-09-10T08:48:56.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:55 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2730212286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:48:56.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:55 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2730212286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:48:56.392 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:56.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:55 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2730212286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:48:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.968+0000 7f6c4b200640 1 -- 172.21.15.89:0/1338554313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c4410e310 msgr2=0x7f6c44110780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.968+0000 7f6c4b200640 1 --2- 172.21.15.89:0/1338554313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c4410e310 0x7f6c44110780 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009ca0 tx=0x7f6c4002a950 comp rx=0 tx=0).stop 2024-09-10T08:48:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.968+0000 7f6c4b200640 1 -- 172.21.15.89:0/1338554313 shutdown_connections 2024-09-10T08:48:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.968+0000 7f6c4b200640 1 --2- 172.21.15.89:0/1338554313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c4410e310 0x7f6c44110780 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.968+0000 7f6c4b200640 1 --2- 172.21.15.89:0/1338554313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c440750b0 0x7f6c4410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.968+0000 7f6c4b200640 1 --2- 172.21.15.89:0/1338554313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c44074790 0x7f6c44074b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.968+0000 7f6c4b200640 1 -- 172.21.15.89:0/1338554313 >> 172.21.15.89:0/1338554313 conn(0x7f6c440fd700 msgr2=0x7f6c440ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:56.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.969+0000 7f6c4b200640 1 -- 172.21.15.89:0/1338554313 shutdown_connections 2024-09-10T08:48:56.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.969+0000 7f6c4b200640 1 -- 172.21.15.89:0/1338554313 wait complete. 2024-09-10T08:48:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.969+0000 7f6c4b200640 1 Processor -- start 2024-09-10T08:48:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.970+0000 7f6c4b200640 1 -- start start 2024-09-10T08:48:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.970+0000 7f6c4b200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c44074790 0x7f6c4419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.970+0000 7f6c4b200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c440750b0 0x7f6c4419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.970+0000 7f6c4b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c4410e310 0x7f6c4419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.970+0000 7f6c4b200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44114100 con 0x7f6c44074790 2024-09-10T08:48:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.970+0000 7f6c4b200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44113f80 con 0x7f6c440750b0 2024-09-10T08:48:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.970+0000 7f6c4b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44114280 con 0x7f6c4410e310 2024-09-10T08:48:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.970+0000 7f6c49e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c44074790 0x7f6c4419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.970+0000 7f6c49400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c440750b0 0x7f6c4419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.970+0000 7f6c49400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c440750b0 0x7f6c4419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:36618/0 (socket says 172.21.15.89:36618) 2024-09-10T08:48:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.970+0000 7f6c49e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c44074790 0x7f6c4419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42052/0 (socket says 172.21.15.89:42052) 2024-09-10T08:48:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.970+0000 7f6c49400640 1 -- 172.21.15.89:0/2945148934 learned_addr learned my addr 172.21.15.89:0/2945148934 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.970+0000 7f6c4a800640 1 --2- 172.21.15.89:0/2945148934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c4410e310 0x7f6c4419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.971+0000 7f6c49400640 1 -- 172.21.15.89:0/2945148934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c4410e310 msgr2=0x7f6c4419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.971+0000 7f6c49400640 1 --2- 172.21.15.89:0/2945148934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c4410e310 0x7f6c4419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.971+0000 7f6c49400640 1 -- 172.21.15.89:0/2945148934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c44074790 msgr2=0x7f6c4419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.971+0000 7f6c49400640 1 --2- 172.21.15.89:0/2945148934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c44074790 0x7f6c4419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.971+0000 7f6c49400640 1 -- 172.21.15.89:0/2945148934 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4419f570 con 0x7f6c440750b0 2024-09-10T08:48:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.971+0000 7f6c49400640 1 --2- 172.21.15.89:0/2945148934 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c440750b0 0x7f6c4419ab00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400e9f0 tx=0x7f6c3400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.971+0000 7f6c3ea00640 1 -- 172.21.15.89:0/2945148934 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3400cd80 con 0x7f6c440750b0 2024-09-10T08:48:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.971+0000 7f6c3ea00640 1 -- 172.21.15.89:0/2945148934 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c34010910 con 0x7f6c440750b0 2024-09-10T08:48:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.971+0000 7f6c4b200640 1 -- 172.21.15.89:0/2945148934 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4419f860 con 0x7f6c440750b0 2024-09-10T08:48:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.971+0000 7f6c3ea00640 1 -- 172.21.15.89:0/2945148934 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34018990 con 0x7f6c440750b0 2024-09-10T08:48:56.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.971+0000 7f6c4b200640 1 -- 172.21.15.89:0/2945148934 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c441a7140 con 0x7f6c440750b0 2024-09-10T08:48:56.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.974+0000 7f6c4f2c3740 1 Processor -- start 2024-09-10T08:48:56.974 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.975+0000 7f6c3ea00640 1 -- 172.21.15.89:0/2945148934 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6c34010ac0 con 0x7f6c440750b0 2024-09-10T08:48:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.975+0000 7f6c4f2c3740 1 -- start start 2024-09-10T08:48:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.975+0000 7f6c4f2c3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b57513880 0x561b57513c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.975+0000 7f6c4f2c3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b575141a0 0x561b5751e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.975 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.975+0000 7f6c4f2c3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b5751ea60 0x561b57520e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.975+0000 7f6c4f2c3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561b5747a340 con 0x561b5751ea60 2024-09-10T08:48:56.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.975+0000 7f6c4f2c3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561b57421bf0 con 0x561b57513880 2024-09-10T08:48:56.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.975+0000 7f6c4f2c3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561b57476300 con 0x561b575141a0 2024-09-10T08:48:56.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.975+0000 7f6c4a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b57513880 0x561b57513c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.976 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.975+0000 7f6c49e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b575141a0 0x561b5751e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.975+0000 7f6c49400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b5751ea60 0x561b57520e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.975+0000 7f6c49e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b575141a0 0x561b5751e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:38112/0 (socket says 172.21.15.89:38112) 2024-09-10T08:48:56.978 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.975+0000 7f6c49e00640 1 -- 172.21.15.89:0/2485587714 learned_addr learned my addr 172.21.15.89:0/2485587714 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:56.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.976+0000 7f6c49e00640 1 -- 172.21.15.89:0/2485587714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b57513880 msgr2=0x561b57513c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:56.980 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.976+0000 7f6c49e00640 1 --2- 172.21.15.89:0/2485587714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b57513880 0x561b57513c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.976+0000 7f6c49e00640 1 -- 172.21.15.89:0/2485587714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b5751ea60 msgr2=0x561b57520e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:56.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.976+0000 7f6c49e00640 1 --2- 172.21.15.89:0/2485587714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b5751ea60 0x561b57520e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.976+0000 7f6c49e00640 1 -- 172.21.15.89:0/2485587714 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b571eb620 con 0x561b575141a0 2024-09-10T08:48:56.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.976+0000 7f6c49e00640 1 --2- 172.21.15.89:0/2485587714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b575141a0 0x561b5751e520 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800bd10 tx=0x7f6c380107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=698caf4a005f3de7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:56.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.976+0000 7f6c33000640 1 -- 172.21.15.89:0/2485587714 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38004030 con 0x561b575141a0 2024-09-10T08:48:56.981 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.976+0000 7f6c33000640 1 -- 172.21.15.89:0/2485587714 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c380041d0 con 0x561b575141a0 2024-09-10T08:48:56.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.976+0000 7f6c33000640 1 -- 172.21.15.89:0/2485587714 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38011820 con 0x561b575141a0 2024-09-10T08:48:56.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.976+0000 7f6c3ea00640 1 --2- 172.21.15.89:0/2945148934 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6c2c0a2b80 0x7f6c2c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.977+0000 7f6c3ea00640 1 -- 172.21.15.89:0/2945148934 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f6c340cf700 con 0x7f6c440750b0 2024-09-10T08:48:56.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.977+0000 7f6c4a800640 1 --2- 172.21.15.89:0/2945148934 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6c2c0a2b80 0x7f6c2c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.977+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/2485587714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b575141a0 msgr2=0x561b5751e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:56.982 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.977+0000 7f6c4f2c3740 1 --2- 172.21.15.89:0/2485587714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b575141a0 0x561b5751e520 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800bd10 tx=0x7f6c380107a0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.978+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/2485587714 shutdown_connections 2024-09-10T08:48:56.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.978+0000 7f6c4f2c3740 1 --2- 172.21.15.89:0/2485587714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b5751ea60 0x561b57520e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.978+0000 7f6c4f2c3740 1 --2- 172.21.15.89:0/2485587714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b575141a0 0x561b5751e520 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.978+0000 7f6c4f2c3740 1 --2- 172.21.15.89:0/2485587714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b57513880 0x561b57513c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.978+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/2485587714 >> 172.21.15.89:0/2485587714 conn(0x561b5750fe10 msgr2=0x561b575101f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:56.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.978+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/2485587714 shutdown_connections 2024-09-10T08:48:56.983 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.978+0000 7f6c4a800640 1 --2- 172.21.15.89:0/2945148934 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6c2c0a2b80 0x7f6c2c0a5040 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6c40006010 tx=0x7f6c40005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:56.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.978+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/2485587714 wait complete. 2024-09-10T08:48:56.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.978+0000 7f6c4f2c3740 1 -- start start 2024-09-10T08:48:56.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.979+0000 7f6c4f2c3740 1 Processor -- start 2024-09-10T08:48:56.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.979+0000 7f6c4f2c3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b57513880 0x561b5751b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.979+0000 7f6c4f2c3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b575141a0 0x561b5751be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.979+0000 7f6c4f2c3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b5751ea60 0x561b5751c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.984 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.979+0000 7f6c4f2c3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561b57521630 con 0x561b575141a0 2024-09-10T08:48:56.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.979+0000 7f6c4f2c3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561b57435060 con 0x561b5751ea60 2024-09-10T08:48:56.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.979+0000 7f6c4f2c3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561b575217b0 con 0x561b57513880 2024-09-10T08:48:56.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.979+0000 7f6c4f2c3740 10 client.0 _ll_register_callbacks cb 0x561b57423af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:56.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.979+0000 7f6c49e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b57513880 0x561b5751b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.979+0000 7f6c49400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b575141a0 0x561b5751be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.979+0000 7f6c49e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b57513880 0x561b5751b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:38124/0 (socket says 172.21.15.89:38124) 2024-09-10T08:48:56.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.979+0000 7f6c49e00640 1 -- 172.21.15.89:0/4181911247 learned_addr learned my addr 172.21.15.89:0/4181911247 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:56.985 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.979+0000 7f6c4a800640 1 --2- 172.21.15.89:0/4181911247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b5751ea60 0x561b5751c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.980+0000 7f6c49e00640 1 -- 172.21.15.89:0/4181911247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b5751ea60 msgr2=0x561b5751c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:56.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.980+0000 7f6c49e00640 1 --2- 172.21.15.89:0/4181911247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b5751ea60 0x561b5751c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.980+0000 7f6c49e00640 1 -- 172.21.15.89:0/4181911247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b575141a0 msgr2=0x561b5751be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:56.986 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.980+0000 7f6c49e00640 1 --2- 172.21.15.89:0/4181911247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b575141a0 0x561b5751be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.980+0000 7f6c49e00640 1 -- 172.21.15.89:0/4181911247 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b5752aad0 con 0x561b57513880 2024-09-10T08:48:56.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.980+0000 7f6c49e00640 1 --2- 172.21.15.89:0/4181911247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b57513880 0x561b5751b8f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6c38010ca0 tx=0x7f6c38003d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1417861cd86df1a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:56.987 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.980+0000 7f6c27400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-09-10T08:48:56.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.985+0000 7f6c27400640 1 -- 172.21.15.89:0/4181911247 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38029060 con 0x561b57513880 2024-09-10T08:48:56.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.985+0000 7f6c27400640 1 -- 172.21.15.89:0/4181911247 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f6c380035f0 con 0x561b57513880 2024-09-10T08:48:56.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.985+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/4181911247 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c380041d0 con 0x561b57513880 2024-09-10T08:48:56.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.985+0000 7f6c27400640 1 -- 172.21.15.89:0/4181911247 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6c38030050 con 0x561b57513880 2024-09-10T08:48:56.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.985+0000 7f6c4f2c3740 10 client.26944 fetch_fsmap learned FSMap version 32 2024-09-10T08:48:56.990 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.985+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/4181911247 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561b5752d440 con 0x561b57513880 2024-09-10T08:48:56.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.985+0000 7f6c27400640 1 -- 172.21.15.89:0/4181911247 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c380041d0 con 0x561b57513880 2024-09-10T08:48:56.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.985+0000 7f6c27400640 1 -- 172.21.15.89:0/4181911247 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f6c380046c0 con 0x561b57513880 2024-09-10T08:48:56.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.986+0000 7f6c4f2c3740 10 client.26944 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T08:48:56.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.986+0000 7f6c4f2c3740 10 client.26944 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:48:56.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.986+0000 7f6c4f2c3740 20 client.26944 populate_metadata read hostname 'smithi089' 2024-09-10T08:48:56.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.986+0000 7f6c4f2c3740 1 --2- 172.21.15.89:0/4181911247 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x561b5752e8a0 0x561b57530c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.986+0000 7f6c4f2c3740 4 client.26944 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:56.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.986+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/4181911247 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b574878b0 con 0x561b5752e8a0 2024-09-10T08:48:56.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.986+0000 7f6c49400640 1 --2- 172.21.15.89:0/4181911247 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x561b5752e8a0 0x561b57530c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.986+0000 7f6c49400640 1 --2- 172.21.15.89:0/4181911247 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x561b5752e8a0 0x561b57530c90 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b67a7bc76ce5cf0 server_cookie=2c61261f8a97d46d in_seq=0 out_seq=0 2024-09-10T08:48:56.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.986+0000 7f6c27400640 10 client.26944 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:48:56.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.987+0000 7f6c27400640 1 -- 172.21.15.89:0/4181911247 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b574878b0 con 0x561b5752e8a0 2024-09-10T08:48:56.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.987+0000 7f6c27400640 10 client.26944 handle_command_reply: tid=0 2024-09-10T08:48:56.991 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.987+0000 7f6c4f2c3740 1 client.26944 shutdown 2024-09-10T08:48:56.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.988+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/4181911247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b57513880 msgr2=0x561b5751b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:56.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.988+0000 7f6c4f2c3740 1 --2- 172.21.15.89:0/4181911247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b57513880 0x561b5751b8f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6c38010ca0 tx=0x7f6c38003d60 comp rx=0 tx=0).stop 2024-09-10T08:48:56.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.988+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/4181911247 shutdown_connections 2024-09-10T08:48:56.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.988+0000 7f6c4f2c3740 1 --2- 172.21.15.89:0/4181911247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b5751ea60 0x561b5751c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.988+0000 7f6c4f2c3740 1 --2- 172.21.15.89:0/4181911247 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x561b5752e8a0 0x561b57530c90 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.988+0000 7f6c4f2c3740 1 --2- 172.21.15.89:0/4181911247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b575141a0 0x561b5751be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.988+0000 7f6c4f2c3740 1 --2- 172.21.15.89:0/4181911247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b57513880 0x561b5751b8f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.988+0000 7f6c27400640 0 client.26944 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:48:56.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.988+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/4181911247 >> 172.21.15.89:0/4181911247 conn(0x561b5750fe10 msgr2=0x561b57515af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:56.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.988+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/4181911247 shutdown_connections 2024-09-10T08:48:56.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.988+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/4181911247 wait complete. 2024-09-10T08:48:56.992 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:56.988+0000 7f6c4f2c3740 20 client.26944 trim_cache size 0 max 16384 2024-09-10T08:48:57.006 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.005+0000 7f6c4f2c3740 1 Processor -- start 2024-09-10T08:48:57.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.005+0000 7f6c4f2c3740 1 -- start start 2024-09-10T08:48:57.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.006+0000 7f6c4f2c3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b57513880 0x561b5751b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:57.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.006+0000 7f6c4f2c3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b575141a0 0x561b57536290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:57.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.006+0000 7f6c4f2c3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b5751ea60 0x561b575367d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:57.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.006+0000 7f6c4f2c3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441b3810 con 0x561b57513880 2024-09-10T08:48:57.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.006+0000 7f6c4f2c3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441b1080 con 0x561b575141a0 2024-09-10T08:48:57.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.006+0000 7f6c4f2c3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441aeed0 con 0x561b5751ea60 2024-09-10T08:48:57.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.006+0000 7f6c49e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b57513880 0x561b5751b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:57.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.006+0000 7f6c49e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b57513880 0x561b5751b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42098/0 (socket says 172.21.15.89:42098) 2024-09-10T08:48:57.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.006+0000 7f6c49e00640 1 -- 172.21.15.89:0/194955067 learned_addr learned my addr 172.21.15.89:0/194955067 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:57.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.006+0000 7f6c49400640 1 --2- 172.21.15.89:0/194955067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b575141a0 0x561b57536290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:57.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.006+0000 7f6c4a800640 1 --2- 172.21.15.89:0/194955067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b5751ea60 0x561b575367d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:57.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.006+0000 7f6c49e00640 1 -- 172.21.15.89:0/194955067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b5751ea60 msgr2=0x561b575367d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:57.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.006+0000 7f6c49e00640 1 --2- 172.21.15.89:0/194955067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b5751ea60 0x561b575367d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:57.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.006+0000 7f6c49e00640 1 -- 172.21.15.89:0/194955067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b575141a0 msgr2=0x561b57536290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:57.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.006+0000 7f6c49e00640 1 --2- 172.21.15.89:0/194955067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b575141a0 0x561b57536290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:57.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.006+0000 7f6c49e00640 1 -- 172.21.15.89:0/194955067 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b575166f0 con 0x561b57513880 2024-09-10T08:48:57.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.006+0000 7f6c49400640 1 --2- 172.21.15.89:0/194955067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b575141a0 0x561b57536290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:57.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.006+0000 7f6c4a800640 1 --2- 172.21.15.89:0/194955067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b5751ea60 0x561b575367d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:57.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.007+0000 7f6c49e00640 1 --2- 172.21.15.89:0/194955067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b57513880 0x561b5751b1d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800b3d0 tx=0x7f6c38005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72155c84e8868ab6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:57.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.007+0000 7f6c32600640 1 -- 172.21.15.89:0/194955067 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38029040 con 0x561b57513880 2024-09-10T08:48:57.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.007+0000 7f6c32600640 1 -- 172.21.15.89:0/194955067 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c3800b480 con 0x561b57513880 2024-09-10T08:48:57.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.007+0000 7f6c32600640 1 -- 172.21.15.89:0/194955067 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38004fb0 con 0x561b57513880 2024-09-10T08:48:57.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.008+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/194955067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b57513880 msgr2=0x561b5751b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:57.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.008+0000 7f6c4f2c3740 1 --2- 172.21.15.89:0/194955067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b57513880 0x561b5751b1d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800b3d0 tx=0x7f6c38005cc0 comp rx=0 tx=0).stop 2024-09-10T08:48:57.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.008+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/194955067 shutdown_connections 2024-09-10T08:48:57.010 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.008+0000 7f6c4f2c3740 1 --2- 172.21.15.89:0/194955067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b5751ea60 0x561b575367d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:57.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.008+0000 7f6c4f2c3740 1 --2- 172.21.15.89:0/194955067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b575141a0 0x561b57536290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:57.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.008+0000 7f6c4f2c3740 1 --2- 172.21.15.89:0/194955067 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b57513880 0x561b5751b1d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:57.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.008+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/194955067 >> 172.21.15.89:0/194955067 conn(0x561b5750fe10 msgr2=0x561b5751d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:57.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.008+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/194955067 shutdown_connections 2024-09-10T08:48:57.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.008+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/194955067 wait complete. 2024-09-10T08:48:57.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.009+0000 7f6c4f2c3740 1 -- start start 2024-09-10T08:48:57.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.009+0000 7f6c4f2c3740 1 Processor -- start 2024-09-10T08:48:57.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.009+0000 7f6c4f2c3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b57513880 0x561b57522760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:57.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.009+0000 7f6c4f2c3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b575141a0 0x561b57522ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:57.011 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.009+0000 7f6c4f2c3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b5751ea60 0x561b575231e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:57.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.009+0000 7f6c4f2c3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441af540 con 0x561b575141a0 2024-09-10T08:48:57.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.009+0000 7f6c4f2c3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441b09b0 con 0x561b57513880 2024-09-10T08:48:57.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.009+0000 7f6c4f2c3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441afc10 con 0x561b5751ea60 2024-09-10T08:48:57.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.009+0000 7f6c4f2c3740 10 client.0 _ll_register_callbacks cb 0x7f6c340d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:57.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.009+0000 7f6c49400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b575141a0 0x561b57522ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:57.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.009+0000 7f6c49400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b575141a0 0x561b57522ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42100/0 (socket says 172.21.15.89:42100) 2024-09-10T08:48:57.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.010+0000 7f6c4a800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b5751ea60 0x561b575231e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:57.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.010+0000 7f6c49400640 1 -- 172.21.15.89:0/3083921135 learned_addr learned my addr 172.21.15.89:0/3083921135 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:57.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.010+0000 7f6c49e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b57513880 0x561b57522760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:57.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.010+0000 7f6c49e00640 1 -- 172.21.15.89:0/3083921135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b5751ea60 msgr2=0x561b575231e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:57.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.010+0000 7f6c49e00640 1 --2- 172.21.15.89:0/3083921135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b5751ea60 0x561b575231e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:57.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.010+0000 7f6c49e00640 1 -- 172.21.15.89:0/3083921135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b575141a0 msgr2=0x561b57522ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:57.012 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.010+0000 7f6c49e00640 1 --2- 172.21.15.89:0/3083921135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b575141a0 0x561b57522ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:57.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.010+0000 7f6c49e00640 1 -- 172.21.15.89:0/3083921135 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b57598260 con 0x561b57513880 2024-09-10T08:48:57.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.010+0000 7f6c49e00640 1 --2- 172.21.15.89:0/3083921135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b57513880 0x561b57522760 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6c38004e50 tx=0x7f6c38003f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7a375139a2b8b48 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:57.013 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.010+0000 7f6c27400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-10T08:48:57.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.010+0000 7f6c27400640 1 -- 172.21.15.89:0/3083921135 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3800b8e0 con 0x561b57513880 2024-09-10T08:48:57.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.010+0000 7f6c27400640 1 -- 172.21.15.89:0/3083921135 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f6c38039040 con 0x561b57513880 2024-09-10T08:48:57.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.011+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/3083921135 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c3800b480 con 0x561b57513880 2024-09-10T08:48:57.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.011+0000 7f6c27400640 1 -- 172.21.15.89:0/3083921135 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6c38004590 con 0x561b57513880 2024-09-10T08:48:57.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.011+0000 7f6c4f2c3740 10 client.27005 fetch_fsmap learned FSMap version 32 2024-09-10T08:48:57.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.011+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/3083921135 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561b57599520 con 0x561b57513880 2024-09-10T08:48:57.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.011+0000 7f6c27400640 1 -- 172.21.15.89:0/3083921135 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c3800b480 con 0x561b57513880 2024-09-10T08:48:57.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.011+0000 7f6c27400640 1 -- 172.21.15.89:0/3083921135 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f6c380053f0 con 0x561b57513880 2024-09-10T08:48:57.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.012+0000 7f6c4f2c3740 10 client.27005 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T08:48:57.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.012+0000 7f6c4f2c3740 10 client.27005 resolve_mds: validated gid 24554 aka daemon mds.b 2024-09-10T08:48:57.016 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.012+0000 7f6c4f2c3740 20 client.27005 populate_metadata read hostname 'smithi089' 2024-09-10T08:48:57.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.012+0000 7f6c4f2c3740 1 --2- 172.21.15.89:0/3083921135 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x561b57599b60 0x561b5759bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:57.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.012+0000 7f6c4f2c3740 4 client.27005 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-10T08:48:57.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.012+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/3083921135 --> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x561b57517510 con 0x561b57599b60 2024-09-10T08:48:57.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.012+0000 7f6c49400640 1 --2- 172.21.15.89:0/3083921135 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x561b57599b60 0x561b5759bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:57.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.012+0000 7f6c49400640 1 --2- 172.21.15.89:0/3083921135 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x561b57599b60 0x561b5759bf80 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58be0c1ce723b902 server_cookie=98d72d82ca7f9930 in_seq=0 out_seq=0 2024-09-10T08:48:57.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.012+0000 7f6c27400640 10 client.27005 ms_handle_connect on v2:172.21.15.111:6838/3640096926 2024-09-10T08:48:57.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.013+0000 7f6c27400640 1 -- 172.21.15.89:0/3083921135 <== mds.0 v2:172.21.15.111:6838/3640096926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x561b57517510 con 0x561b57599b60 2024-09-10T08:48:57.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.013+0000 7f6c27400640 10 client.27005 handle_command_reply: tid=0 2024-09-10T08:48:57.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.013+0000 7f6c4f2c3740 1 client.27005 shutdown 2024-09-10T08:48:57.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.013+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/3083921135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b57513880 msgr2=0x561b57522760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:57.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.013+0000 7f6c4f2c3740 1 --2- 172.21.15.89:0/3083921135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b57513880 0x561b57522760 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6c38004e50 tx=0x7f6c38003f00 comp rx=0 tx=0).stop 2024-09-10T08:48:57.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.013+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/3083921135 shutdown_connections 2024-09-10T08:48:57.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.013+0000 7f6c4f2c3740 1 --2- 172.21.15.89:0/3083921135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b5751ea60 0x561b575231e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:57.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.013+0000 7f6c4f2c3740 1 --2- 172.21.15.89:0/3083921135 >> [v2:172.21.15.111:6838/3640096926,v1:172.21.15.111:6839/3640096926] conn(0x561b57599b60 0x561b5759bf80 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:57.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.013+0000 7f6c4f2c3740 1 --2- 172.21.15.89:0/3083921135 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561b575141a0 0x561b57522ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:57.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.013+0000 7f6c4f2c3740 1 --2- 172.21.15.89:0/3083921135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b57513880 0x561b57522760 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:57.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.013+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/3083921135 >> 172.21.15.89:0/3083921135 conn(0x561b5750fe10 msgr2=0x561b575152d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:57.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.013+0000 7f6c27400640 0 client.27005 ms_handle_reset on v2:172.21.15.111:6838/3640096926 2024-09-10T08:48:57.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.013+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/3083921135 shutdown_connections 2024-09-10T08:48:57.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.013+0000 7f6c4f2c3740 1 -- 172.21.15.89:0/3083921135 wait complete. 2024-09-10T08:48:57.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.014+0000 7f6c4f2c3740 20 client.27005 trim_cache size 0 max 16384 2024-09-10T08:48:57.020 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:48:57.020 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 32, 2024-09-10T08:48:57.020 INFO:teuthology.orchestra.run.smithi089.stdout: "leader": 24554, 2024-09-10T08:48:57.020 INFO:teuthology.orchestra.run.smithi089.stdout: "set_version": 24, 2024-09-10T08:48:57.020 INFO:teuthology.orchestra.run.smithi089.stdout: "sets": {} 2024-09-10T08:48:57.020 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:48:57.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.014+0000 7f6c4b200640 1 -- 172.21.15.89:0/2945148934 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6c2c0a2b80 msgr2=0x7f6c2c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:57.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.014+0000 7f6c4b200640 1 --2- 172.21.15.89:0/2945148934 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6c2c0a2b80 0x7f6c2c0a5040 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6c40006010 tx=0x7f6c40005d80 comp rx=0 tx=0).stop 2024-09-10T08:48:57.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.014+0000 7f6c4b200640 1 -- 172.21.15.89:0/2945148934 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c440750b0 msgr2=0x7f6c4419ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:57.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.014+0000 7f6c4b200640 1 --2- 172.21.15.89:0/2945148934 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c440750b0 0x7f6c4419ab00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400e9f0 tx=0x7f6c3400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:48:57.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.014+0000 7f6c4b200640 1 -- 172.21.15.89:0/2945148934 shutdown_connections 2024-09-10T08:48:57.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.014+0000 7f6c4b200640 1 --2- 172.21.15.89:0/2945148934 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6c2c0a2b80 0x7f6c2c0a5040 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:57.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.014+0000 7f6c4b200640 1 --2- 172.21.15.89:0/2945148934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c4410e310 0x7f6c4419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:57.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.014+0000 7f6c4b200640 1 --2- 172.21.15.89:0/2945148934 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c440750b0 0x7f6c4419ab00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:57.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.014+0000 7f6c4b200640 1 --2- 172.21.15.89:0/2945148934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c44074790 0x7f6c4419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:57.022 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.014+0000 7f6c4b200640 1 -- 172.21.15.89:0/2945148934 >> 172.21.15.89:0/2945148934 conn(0x7f6c440fd700 msgr2=0x7f6c440fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:57.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.014+0000 7f6c4b200640 1 -- 172.21.15.89:0/2945148934 shutdown_connections 2024-09-10T08:48:57.023 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:57.014+0000 7f6c4b200640 1 -- 172.21.15.89:0/2945148934 wait complete. 2024-09-10T08:48:57.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:56 smithi111 ceph-mon[35220]: pgmap v663: 129 pgs: 129 active+clean; 60 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 1.0 MiB/s wr, 615 op/s 2024-09-10T08:48:57.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:56 smithi089 ceph-mon[30757]: pgmap v663: 129 pgs: 129 active+clean; 60 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 1.0 MiB/s wr, 615 op/s 2024-09-10T08:48:57.372 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:56 smithi149 ceph-mon[36167]: pgmap v663: 129 pgs: 129 active+clean; 60 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 1.0 MiB/s wr, 615 op/s 2024-09-10T08:48:57.773 INFO:tasks.quiescer:done joining Quiescers 2024-09-10T08:48:57.773 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-10T08:48:57.783 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi089.front.sepia.ceph.com 2024-09-10T08:48:57.796 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-10T08:48:57.805 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-10T08:48:57.805 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-09-10T08:48:57.806 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-10T08:48:57.855 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T08:48:58.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.014+0000 7ff4fd400640 1 -- 172.21.15.89:0/858700299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4f8078790 msgr2=0x7ff4f8079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:58.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.014+0000 7ff4fd400640 1 --2- 172.21.15.89:0/858700299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4f8078790 0x7ff4f8079030 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009ca0 tx=0x7ff4f002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.014+0000 7ff4fd400640 1 -- 172.21.15.89:0/858700299 shutdown_connections 2024-09-10T08:48:58.014 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.014+0000 7ff4fd400640 1 --2- 172.21.15.89:0/858700299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4f8078790 0x7ff4f8079030 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.014+0000 7ff4fd400640 1 --2- 172.21.15.89:0/858700299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f8077df0 0x7ff4f8078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.014+0000 7ff4fd400640 1 --2- 172.21.15.89:0/858700299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4f8076bb0 0x7ff4f8076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.014+0000 7ff4fd400640 1 -- 172.21.15.89:0/858700299 >> 172.21.15.89:0/858700299 conn(0x7ff4f80ff4b0 msgr2=0x7ff4f81018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:58.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.014+0000 7ff4fd400640 1 -- 172.21.15.89:0/858700299 shutdown_connections 2024-09-10T08:48:58.015 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.014+0000 7ff4fd400640 1 -- 172.21.15.89:0/858700299 wait complete. 2024-09-10T08:48:58.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.015+0000 7ff4fd400640 1 Processor -- start 2024-09-10T08:48:58.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.015+0000 7ff4fd400640 1 -- start start 2024-09-10T08:48:58.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.015+0000 7ff4fd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f8076bb0 0x7ff4f819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:58.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.015+0000 7ff4fd400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4f8077df0 0x7ff4f819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:58.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.015+0000 7ff4fd400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4f8078790 0x7ff4f81a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:58.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.015+0000 7ff4fd400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f8113e10 con 0x7ff4f8078790 2024-09-10T08:48:58.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.015+0000 7ff4fd400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f8113c90 con 0x7ff4f8076bb0 2024-09-10T08:48:58.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.015+0000 7ff4fd400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f8113f90 con 0x7ff4f8077df0 2024-09-10T08:48:58.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.015+0000 7ff4f7200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4f8078790 0x7ff4f81a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:58.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.016+0000 7ff4f6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f8076bb0 0x7ff4f819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:58.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.016+0000 7ff4f7200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4f8078790 0x7ff4f81a59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42118/0 (socket says 172.21.15.89:42118) 2024-09-10T08:48:58.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.016+0000 7ff4f7200640 1 -- 172.21.15.89:0/4290026420 learned_addr learned my addr 172.21.15.89:0/4290026420 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:58.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.016+0000 7ff4ede00640 1 --2- 172.21.15.89:0/4290026420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4f8077df0 0x7ff4f819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:58.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.016+0000 7ff4f6800640 1 -- 172.21.15.89:0/4290026420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4f8077df0 msgr2=0x7ff4f819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:58.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.016+0000 7ff4f6800640 1 --2- 172.21.15.89:0/4290026420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4f8077df0 0x7ff4f819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.018 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.016+0000 7ff4f6800640 1 -- 172.21.15.89:0/4290026420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4f8078790 msgr2=0x7ff4f81a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:58.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.016+0000 7ff4f6800640 1 --2- 172.21.15.89:0/4290026420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4f8078790 0x7ff4f81a59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.016+0000 7ff4f6800640 1 -- 172.21.15.89:0/4290026420 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f81a6030 con 0x7ff4f8076bb0 2024-09-10T08:48:58.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.016+0000 7ff4f7200640 1 --2- 172.21.15.89:0/4290026420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4f8078790 0x7ff4f81a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:58.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.016+0000 7ff4f6800640 1 --2- 172.21.15.89:0/4290026420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f8076bb0 0x7ff4f819e410 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800e9f0 tx=0x7ff4e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:58.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.017+0000 7ff4efe00640 1 -- 172.21.15.89:0/4290026420 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e800cd80 con 0x7ff4f8076bb0 2024-09-10T08:48:58.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.017+0000 7ff4efe00640 1 -- 172.21.15.89:0/4290026420 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4e80035f0 con 0x7ff4f8076bb0 2024-09-10T08:48:58.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.017+0000 7ff4fd400640 1 -- 172.21.15.89:0/4290026420 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f81a6320 con 0x7ff4f8076bb0 2024-09-10T08:48:58.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.017+0000 7ff4efe00640 1 -- 172.21.15.89:0/4290026420 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8010690 con 0x7ff4f8076bb0 2024-09-10T08:48:58.019 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.017+0000 7ff4fd400640 1 -- 172.21.15.89:0/4290026420 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f81a6870 con 0x7ff4f8076bb0 2024-09-10T08:48:58.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.018+0000 7ff4fd400640 1 -- 172.21.15.89:0/4290026420 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f8110840 con 0x7ff4f8076bb0 2024-09-10T08:48:58.020 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.019+0000 7ff4efe00640 1 -- 172.21.15.89:0/4290026420 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff4e8003b30 con 0x7ff4f8076bb0 2024-09-10T08:48:58.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.021+0000 7ff4efe00640 1 --2- 172.21.15.89:0/4290026420 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff4dc0a2b90 0x7ff4dc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:58.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.021+0000 7ff4efe00640 1 -- 172.21.15.89:0/4290026420 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff4e80cf750 con 0x7ff4f8076bb0 2024-09-10T08:48:58.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.021+0000 7ff4ede00640 1 --2- 172.21.15.89:0/4290026420 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff4dc0a2b90 0x7ff4dc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:58.021 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.022+0000 7ff4ede00640 1 --2- 172.21.15.89:0/4290026420 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff4dc0a2b90 0x7ff4dc0a5050 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff4f819f920 tx=0x7ff4e0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:58.025 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.025+0000 7ff4efe00640 1 -- 172.21.15.89:0/4290026420 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4e809c280 con 0x7ff4f8076bb0 2024-09-10T08:48:58.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.202+0000 7ff4fd400640 1 -- 172.21.15.89:0/4290026420 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4f802d7c0 con 0x7ff4f8076bb0 2024-09-10T08:48:58.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.204+0000 7ff4efe00640 1 -- 172.21.15.89:0/4290026420 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24894 (secure 0 0 0) 0x7ff4e809bb80 con 0x7ff4f8076bb0 2024-09-10T08:48:58.204 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:48:58.204 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":96,"fsid":"91f747bc-6f4e-11ef-bcea-c7b262605968","created":"2024-09-10T08:28:49.697035+0000","modified":"2024-09-10T08:38:43.618497+0000","last_up_change":"2024-09-10T08:36:00.535433+0000","last_in_change":"2024-09-10T08:35:37.131131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:32:02.060714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:36:52.462301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:36:55.119708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:37:03.729002+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d3b55192-bd95-4497-89ea-dc017a25c0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6803","nonce":1760523284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6805","nonce":1760523284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6809","nonce":1760523284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6807","nonce":1760523284}]},"public_addr":"172.21.15.89:6803/1760523284","cluster_addr":"172.21.15.89:6805/1760523284","heartbeat_back_addr":"172.21.15.89:6809/1760523284","heartbeat_front_addr":"172.21.15.89:6807/1760523284","state":["exists","up"]},{"osd":1,"uuid":"60aae117-cb52-4034-a700-11693171cd52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6801","nonce":2369291742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6803","nonce":2369291742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6807","nonce":2369291742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6805","nonce":2369291742}]},"public_addr":"172.21.15.111:6801/2369291742","cluster_addr":"172.21.15.111:6803/2369291742","heartbeat_back_addr":"172.21.15.111:6807/2369291742","heartbeat_front_addr":"172.21.15.111:6805/2369291742","state":["exists","up"]},{"osd":2,"uuid":"a305a019-0870-437c-a298-4f923ec54e77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6801","nonce":1092261395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6803","nonce":1092261395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6807","nonce":1092261395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6805","nonce":1092261395}]},"public_addr":"172.21.15.149:6801/1092261395","cluster_addr":"172.21.15.149:6803/1092261395","heartbeat_back_addr":"172.21.15.149:6807/1092261395","heartbeat_front_addr":"172.21.15.149:6805/1092261395","state":["exists","up"]},{"osd":3,"uuid":"d02f54c1-f883-40f6-9341-8d9fc4b712ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6811","nonce":1405773950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6813","nonce":1405773950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6817","nonce":1405773950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6815","nonce":1405773950}]},"public_addr":"172.21.15.89:6811/1405773950","cluster_addr":"172.21.15.89:6813/1405773950","heartbeat_back_addr":"172.21.15.89:6817/1405773950","heartbeat_front_addr":"172.21.15.89:6815/1405773950","state":["exists","up"]},{"osd":4,"uuid":"23cff46b-88dd-4344-be06-b35db6014f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6809","nonce":316717323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6811","nonce":316717323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6815","nonce":316717323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6813","nonce":316717323}]},"public_addr":"172.21.15.111:6809/316717323","cluster_addr":"172.21.15.111:6811/316717323","heartbeat_back_addr":"172.21.15.111:6815/316717323","heartbeat_front_addr":"172.21.15.111:6813/316717323","state":["exists","up"]},{"osd":5,"uuid":"8e516e8d-e7c6-4ebc-80a5-29932161df4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6809","nonce":1256148461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6811","nonce":1256148461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6815","nonce":1256148461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6813","nonce":1256148461}]},"public_addr":"172.21.15.149:6809/1256148461","cluster_addr":"172.21.15.149:6811/1256148461","heartbeat_back_addr":"172.21.15.149:6815/1256148461","heartbeat_front_addr":"172.21.15.149:6813/1256148461","state":["exists","up"]},{"osd":6,"uuid":"48f92472-f657-4c5a-adeb-20d4fea1ce8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6819","nonce":4190488024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6821","nonce":4190488024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6825","nonce":4190488024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6823","nonce":4190488024}]},"public_addr":"172.21.15.89:6819/4190488024","cluster_addr":"172.21.15.89:6821/4190488024","heartbeat_back_addr":"172.21.15.89:6825/4190488024","heartbeat_front_addr":"172.21.15.89:6823/4190488024","state":["exists","up"]},{"osd":7,"uuid":"14a2a6f0-c4cd-406b-b637-7527fd32babe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6817","nonce":652666376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6819","nonce":652666376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6823","nonce":652666376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6821","nonce":652666376}]},"public_addr":"172.21.15.111:6817/652666376","cluster_addr":"172.21.15.111:6819/652666376","heartbeat_back_addr":"172.21.15.111:6823/652666376","heartbeat_front_addr":"172.21.15.111:6821/652666376","state":["exists","up"]},{"osd":8,"uuid":"a54502e6-95f7-431f-8db1-54a6c7983b5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6817","nonce":2955172869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6819","nonce":2955172869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6823","nonce":2955172869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6821","nonce":2955172869}]},"public_addr":"172.21.15.149:6817/2955172869","cluster_addr":"172.21.15.149:6819/2955172869","heartbeat_back_addr":"172.21.15.149:6823/2955172869","heartbeat_front_addr":"172.21.15.149:6821/2955172869","state":["exists","up"]},{"osd":9,"uuid":"5fe17344-5b1c-4601-a633-639cc1691478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6827","nonce":2427467040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6829","nonce":2427467040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6833","nonce":2427467040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6831","nonce":2427467040}]},"public_addr":"172.21.15.89:6827/2427467040","cluster_addr":"172.21.15.89:6829/2427467040","heartbeat_back_addr":"172.21.15.89:6833/2427467040","heartbeat_front_addr":"172.21.15.89:6831/2427467040","state":["exists","up"]},{"osd":10,"uuid":"d8f29890-3064-4b67-b649-467a0ac86db7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6825","nonce":2328678744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6827","nonce":2328678744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6831","nonce":2328678744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6829","nonce":2328678744}]},"public_addr":"172.21.15.111:6825/2328678744","cluster_addr":"172.21.15.111:6827/2328678744","heartbeat_back_addr":"172.21.15.111:6831/2328678744","heartbeat_front_addr":"172.21.15.111:6829/2328678744","state":["exists","up"]},{"osd":11,"uuid":"92182c73-f139-47ca-a0ef-5046ae9b2310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6825","nonce":3130508827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6827","nonce":3130508827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6831","nonce":3130508827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6829","nonce":3130508827}]},"public_addr":"172.21.15.149:6825/3130508827","cluster_addr":"172.21.15.149:6827/3130508827","heartbeat_back_addr":"172.21.15.149:6831/3130508827","heartbeat_front_addr":"172.21.15.149:6829/3130508827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:09.031810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:33.695124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:58.065510+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:27.592838+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:53.275289+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:18.305677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:47.781535+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:12.889512+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:37.744193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:07.718408+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:33.391027+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:59.013250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/850669510":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/3541027428":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6801/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6800/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/2396468787":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/1609720980":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/995019144":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6801/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6800/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/2179977257":"2024-09-11T08:29:14.554869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:48:58.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.206+0000 7ff4fd400640 1 -- 172.21.15.89:0/4290026420 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff4dc0a2b90 msgr2=0x7ff4dc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:58.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.206+0000 7ff4fd400640 1 --2- 172.21.15.89:0/4290026420 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff4dc0a2b90 0x7ff4dc0a5050 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff4f819f920 tx=0x7ff4e0005f90 comp rx=0 tx=0).stop 2024-09-10T08:48:58.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.206+0000 7ff4fd400640 1 -- 172.21.15.89:0/4290026420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f8076bb0 msgr2=0x7ff4f819e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:58.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.206+0000 7ff4fd400640 1 --2- 172.21.15.89:0/4290026420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f8076bb0 0x7ff4f819e410 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800e9f0 tx=0x7ff4e800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.206+0000 7ff4fd400640 1 -- 172.21.15.89:0/4290026420 shutdown_connections 2024-09-10T08:48:58.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.206+0000 7ff4fd400640 1 --2- 172.21.15.89:0/4290026420 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff4dc0a2b90 0x7ff4dc0a5050 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.206+0000 7ff4fd400640 1 --2- 172.21.15.89:0/4290026420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4f8078790 0x7ff4f81a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.206+0000 7ff4fd400640 1 --2- 172.21.15.89:0/4290026420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4f8077df0 0x7ff4f819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.206+0000 7ff4fd400640 1 --2- 172.21.15.89:0/4290026420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f8076bb0 0x7ff4f819e410 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.206+0000 7ff4fd400640 1 -- 172.21.15.89:0/4290026420 >> 172.21.15.89:0/4290026420 conn(0x7ff4f80ff4b0 msgr2=0x7ff4f8100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:58.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.206+0000 7ff4fd400640 1 -- 172.21.15.89:0/4290026420 shutdown_connections 2024-09-10T08:48:58.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.206+0000 7ff4fd400640 1 -- 172.21.15.89:0/4290026420 wait complete. 2024-09-10T08:48:58.222 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T08:32:02.060714+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T08:36:52.462301+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T08:36:55.119708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T08:37:03.729002+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-10T08:48:58.223 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T08:48:58.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:58 smithi111 ceph-mon[35220]: pgmap v664: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 666 op/s 2024-09-10T08:48:58.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:58 smithi089 ceph-mon[30757]: pgmap v664: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 666 op/s 2024-09-10T08:48:58.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.383+0000 7fb92b200640 1 -- 172.21.15.89:0/3856347666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb92c104e00 msgr2=0x7fb92c105280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:58.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.383+0000 7fb92b200640 1 --2- 172.21.15.89:0/3856347666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb92c104e00 0x7fb92c105280 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb918009900 tx=0x7fb91802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.383+0000 7fb92b200640 1 -- 172.21.15.89:0/3856347666 shutdown_connections 2024-09-10T08:48:58.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.383+0000 7fb92b200640 1 --2- 172.21.15.89:0/3856347666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb92c1057c0 0x7fb92c10c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.383+0000 7fb92b200640 1 --2- 172.21.15.89:0/3856347666 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb92c104e00 0x7fb92c105280 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.383+0000 7fb92b200640 1 --2- 172.21.15.89:0/3856347666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb92c103c10 0x7fb92c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.383+0000 7fb92b200640 1 -- 172.21.15.89:0/3856347666 >> 172.21.15.89:0/3856347666 conn(0x7fb92c0ff390 msgr2=0x7fb92c1017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:58.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.383+0000 7fb92b200640 1 -- 172.21.15.89:0/3856347666 shutdown_connections 2024-09-10T08:48:58.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.383+0000 7fb92b200640 1 -- 172.21.15.89:0/3856347666 wait complete. 2024-09-10T08:48:58.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.384+0000 7fb92b200640 1 Processor -- start 2024-09-10T08:48:58.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.384+0000 7fb92b200640 1 -- start start 2024-09-10T08:48:58.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.385+0000 7fb92b200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb92c103c10 0x7fb92c199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:58.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.385+0000 7fb92b200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb92c104e00 0x7fb92c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:58.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.385+0000 7fb92b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb92c1057c0 0x7fb92c1a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:58.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.385+0000 7fb92b200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c10f890 con 0x7fb92c104e00 2024-09-10T08:48:58.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.385+0000 7fb92b200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c10f710 con 0x7fb92c103c10 2024-09-10T08:48:58.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.385+0000 7fb92b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c10fa10 con 0x7fb92c1057c0 2024-09-10T08:48:58.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.385+0000 7fb929400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb92c104e00 0x7fb92c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:58.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.385+0000 7fb929e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb92c103c10 0x7fb92c199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:58.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.385+0000 7fb929400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb92c104e00 0x7fb92c19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42128/0 (socket says 172.21.15.89:42128) 2024-09-10T08:48:58.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.385+0000 7fb929400640 1 -- 172.21.15.89:0/2904119100 learned_addr learned my addr 172.21.15.89:0/2904119100 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:58.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.385+0000 7fb92a800640 1 --2- 172.21.15.89:0/2904119100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb92c1057c0 0x7fb92c1a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:58.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.385+0000 7fb929e00640 1 -- 172.21.15.89:0/2904119100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb92c1057c0 msgr2=0x7fb92c1a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:58.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.385+0000 7fb929e00640 1 --2- 172.21.15.89:0/2904119100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb92c1057c0 0x7fb92c1a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.385+0000 7fb929e00640 1 -- 172.21.15.89:0/2904119100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb92c104e00 msgr2=0x7fb92c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:58.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.385+0000 7fb929e00640 1 --2- 172.21.15.89:0/2904119100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb92c104e00 0x7fb92c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.386+0000 7fb929e00640 1 -- 172.21.15.89:0/2904119100 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92c1a1b60 con 0x7fb92c103c10 2024-09-10T08:48:58.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.386+0000 7fb929400640 1 --2- 172.21.15.89:0/2904119100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb92c104e00 0x7fb92c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:58.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.386+0000 7fb929e00640 1 --2- 172.21.15.89:0/2904119100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb92c103c10 0x7fb92c199f40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00e9c0 tx=0x7fb91c00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:58.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.386+0000 7fb922a00640 1 -- 172.21.15.89:0/2904119100 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c00ccc0 con 0x7fb92c103c10 2024-09-10T08:48:58.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.386+0000 7fb92b200640 1 -- 172.21.15.89:0/2904119100 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92c1a1e50 con 0x7fb92c103c10 2024-09-10T08:48:58.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.386+0000 7fb922a00640 1 -- 172.21.15.89:0/2904119100 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb91c0035f0 con 0x7fb92c103c10 2024-09-10T08:48:58.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.386+0000 7fb922a00640 1 -- 172.21.15.89:0/2904119100 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c00ce60 con 0x7fb92c103c10 2024-09-10T08:48:58.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.386+0000 7fb92b200640 1 -- 172.21.15.89:0/2904119100 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb92c1a22b0 con 0x7fb92c103c10 2024-09-10T08:48:58.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.387+0000 7fb92b200640 1 -- 172.21.15.89:0/2904119100 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb92c10b050 con 0x7fb92c103c10 2024-09-10T08:48:58.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.389+0000 7fb922a00640 1 -- 172.21.15.89:0/2904119100 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb91c0037a0 con 0x7fb92c103c10 2024-09-10T08:48:58.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.391+0000 7fb922a00640 1 --2- 172.21.15.89:0/2904119100 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb9100a2c10 0x7fb9100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:58.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.391+0000 7fb922a00640 1 -- 172.21.15.89:0/2904119100 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb91c0cf770 con 0x7fb92c103c10 2024-09-10T08:48:58.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.391+0000 7fb929400640 1 --2- 172.21.15.89:0/2904119100 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb9100a2c10 0x7fb9100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:58.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.392+0000 7fb929400640 1 --2- 172.21.15.89:0/2904119100 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb9100a2c10 0x7fb9100a50d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb918031440 tx=0x7fb9180315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:58.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.393+0000 7fb922a00640 1 -- 172.21.15.89:0/2904119100 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb91c09c2a0 con 0x7fb92c103c10 2024-09-10T08:48:58.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:58 smithi149 ceph-mon[36167]: pgmap v664: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 666 op/s 2024-09-10T08:48:58.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.567+0000 7fb92b200640 1 -- 172.21.15.89:0/2904119100 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb92c02da50 con 0x7fb92c103c10 2024-09-10T08:48:58.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.568+0000 7fb922a00640 1 -- 172.21.15.89:0/2904119100 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7fb91c09bba0 con 0x7fb92c103c10 2024-09-10T08:48:58.568 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 1 2024-09-10T08:48:58.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.570+0000 7fb92b200640 1 -- 172.21.15.89:0/2904119100 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb9100a2c10 msgr2=0x7fb9100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:58.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.570+0000 7fb92b200640 1 --2- 172.21.15.89:0/2904119100 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb9100a2c10 0x7fb9100a50d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb918031440 tx=0x7fb9180315b0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.570+0000 7fb92b200640 1 -- 172.21.15.89:0/2904119100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb92c103c10 msgr2=0x7fb92c199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:58.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.570+0000 7fb92b200640 1 --2- 172.21.15.89:0/2904119100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb92c103c10 0x7fb92c199f40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00e9c0 tx=0x7fb91c00edc0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.570+0000 7fb92b200640 1 -- 172.21.15.89:0/2904119100 shutdown_connections 2024-09-10T08:48:58.570 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.570+0000 7fb92b200640 1 --2- 172.21.15.89:0/2904119100 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fb9100a2c10 0x7fb9100a50d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.570+0000 7fb92b200640 1 --2- 172.21.15.89:0/2904119100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb92c1057c0 0x7fb92c1a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.570+0000 7fb92b200640 1 --2- 172.21.15.89:0/2904119100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb92c104e00 0x7fb92c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.570+0000 7fb92b200640 1 --2- 172.21.15.89:0/2904119100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb92c103c10 0x7fb92c199f40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.570+0000 7fb92b200640 1 -- 172.21.15.89:0/2904119100 >> 172.21.15.89:0/2904119100 conn(0x7fb92c0ff390 msgr2=0x7fb92c108220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:58.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.570+0000 7fb92b200640 1 -- 172.21.15.89:0/2904119100 shutdown_connections 2024-09-10T08:48:58.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.570+0000 7fb92b200640 1 -- 172.21.15.89:0/2904119100 wait complete. 2024-09-10T08:48:58.586 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T08:48:58.746 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.743+0000 7fa2b3800640 1 -- 172.21.15.89:0/2233091451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2ac103d20 msgr2=0x7fa2ac104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:58.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.743+0000 7fa2b3800640 1 --2- 172.21.15.89:0/2233091451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2ac103d20 0x7fa2ac104120 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009900 tx=0x7fa29c02a410 comp rx=0 tx=0).stop 2024-09-10T08:48:58.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.743+0000 7fa2b3800640 1 -- 172.21.15.89:0/2233091451 shutdown_connections 2024-09-10T08:48:58.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.743+0000 7fa2b3800640 1 --2- 172.21.15.89:0/2233091451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2ac1058d0 0x7fa2ac10c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.743+0000 7fa2b3800640 1 --2- 172.21.15.89:0/2233091451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2ac104f10 0x7fa2ac105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.743+0000 7fa2b3800640 1 --2- 172.21.15.89:0/2233091451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2ac103d20 0x7fa2ac104120 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.743+0000 7fa2b3800640 1 -- 172.21.15.89:0/2233091451 >> 172.21.15.89:0/2233091451 conn(0x7fa2ac0ff4a0 msgr2=0x7fa2ac1018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:58.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.744+0000 7fa2b3800640 1 -- 172.21.15.89:0/2233091451 shutdown_connections 2024-09-10T08:48:58.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.744+0000 7fa2b3800640 1 -- 172.21.15.89:0/2233091451 wait complete. 2024-09-10T08:48:58.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.744+0000 7fa2b3800640 1 Processor -- start 2024-09-10T08:48:58.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.745+0000 7fa2b3800640 1 -- start start 2024-09-10T08:48:58.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.745+0000 7fa2b3800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2ac103d20 0x7fa2ac19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:58.747 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.745+0000 7fa2b3800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2ac104f10 0x7fa2ac19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:58.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.745+0000 7fa2b3800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2ac1058d0 0x7fa2ac1a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:58.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.745+0000 7fa2b3800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac10f980 con 0x7fa2ac1058d0 2024-09-10T08:48:58.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.745+0000 7fa2b3800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac10f800 con 0x7fa2ac104f10 2024-09-10T08:48:58.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.745+0000 7fa2b3800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac10fb00 con 0x7fa2ac103d20 2024-09-10T08:48:58.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.745+0000 7fa2b0e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2ac103d20 0x7fa2ac19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:58.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.745+0000 7fa2a3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2ac104f10 0x7fa2ac19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:58.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.745+0000 7fa2b1800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2ac1058d0 0x7fa2ac1a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:58.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.745+0000 7fa2b0e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2ac103d20 0x7fa2ac19a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:38200/0 (socket says 172.21.15.89:38200) 2024-09-10T08:48:58.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.745+0000 7fa2a3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2ac104f10 0x7fa2ac19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:36698/0 (socket says 172.21.15.89:36698) 2024-09-10T08:48:58.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.745+0000 7fa2b0e00640 1 -- 172.21.15.89:0/1954918953 learned_addr learned my addr 172.21.15.89:0/1954918953 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:58.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.746+0000 7fa2b0e00640 1 -- 172.21.15.89:0/1954918953 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2ac104f10 msgr2=0x7fa2ac19a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:58.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.746+0000 7fa2b0e00640 1 --2- 172.21.15.89:0/1954918953 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2ac104f10 0x7fa2ac19a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.746+0000 7fa2b0e00640 1 -- 172.21.15.89:0/1954918953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2ac1058d0 msgr2=0x7fa2ac1a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:58.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.746+0000 7fa2b0e00640 1 --2- 172.21.15.89:0/1954918953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2ac1058d0 0x7fa2ac1a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.746+0000 7fa2b0e00640 1 -- 172.21.15.89:0/1954918953 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2ac1a1cb0 con 0x7fa2ac103d20 2024-09-10T08:48:58.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.746+0000 7fa2a3e00640 1 --2- 172.21.15.89:0/1954918953 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2ac104f10 0x7fa2ac19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:58.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.746+0000 7fa2b0e00640 1 --2- 172.21.15.89:0/1954918953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2ac103d20 0x7fa2ac19a090 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa29c031610 tx=0x7fa29c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:58.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.746+0000 7fa2aa000640 1 -- 172.21.15.89:0/1954918953 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c03f030 con 0x7fa2ac103d20 2024-09-10T08:48:58.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.746+0000 7fa2aa000640 1 -- 172.21.15.89:0/1954918953 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa29c003ad0 con 0x7fa2ac103d20 2024-09-10T08:48:58.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.746+0000 7fa2b3800640 1 -- 172.21.15.89:0/1954918953 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2ac1a1f40 con 0x7fa2ac103d20 2024-09-10T08:48:58.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.746+0000 7fa2aa000640 1 -- 172.21.15.89:0/1954918953 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c035070 con 0x7fa2ac103d20 2024-09-10T08:48:58.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.746+0000 7fa2b3800640 1 -- 172.21.15.89:0/1954918953 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2ac1a2350 con 0x7fa2ac103d20 2024-09-10T08:48:58.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.747+0000 7fa2b3800640 1 -- 172.21.15.89:0/1954918953 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2ac10b150 con 0x7fa2ac103d20 2024-09-10T08:48:58.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.749+0000 7fa2aa000640 1 -- 172.21.15.89:0/1954918953 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa29c042030 con 0x7fa2ac103d20 2024-09-10T08:48:58.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.751+0000 7fa2aa000640 1 --2- 172.21.15.89:0/1954918953 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa2900a2b90 0x7fa2900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:58.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.751+0000 7fa2aa000640 1 -- 172.21.15.89:0/1954918953 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa29c0efd50 con 0x7fa2ac103d20 2024-09-10T08:48:58.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.751+0000 7fa2a3e00640 1 --2- 172.21.15.89:0/1954918953 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa2900a2b90 0x7fa2900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:58.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.752+0000 7fa2a3e00640 1 --2- 172.21.15.89:0/1954918953 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa2900a2b90 0x7fa2900a5050 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac19b5a0 tx=0x7fa29400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:58.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.753+0000 7fa2aa000640 1 -- 172.21.15.89:0/1954918953 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa29c0bc230 con 0x7fa2ac103d20 2024-09-10T08:48:58.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.927+0000 7fa2b3800640 1 -- 172.21.15.89:0/1954918953 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa2ac02da10 con 0x7fa2ac103d20 2024-09-10T08:48:58.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.928+0000 7fa2aa000640 1 -- 172.21.15.89:0/1954918953 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7fa29c0c0530 con 0x7fa2ac103d20 2024-09-10T08:48:58.928 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2024-09-10T08:48:58.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.930+0000 7fa2b3800640 1 -- 172.21.15.89:0/1954918953 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa2900a2b90 msgr2=0x7fa2900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:58.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.930+0000 7fa2b3800640 1 --2- 172.21.15.89:0/1954918953 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa2900a2b90 0x7fa2900a5050 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac19b5a0 tx=0x7fa29400b410 comp rx=0 tx=0).stop 2024-09-10T08:48:58.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.930+0000 7fa2b3800640 1 -- 172.21.15.89:0/1954918953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2ac103d20 msgr2=0x7fa2ac19a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:58.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.930+0000 7fa2b3800640 1 --2- 172.21.15.89:0/1954918953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2ac103d20 0x7fa2ac19a090 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa29c031610 tx=0x7fa29c004270 comp rx=0 tx=0).stop 2024-09-10T08:48:58.930 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.930+0000 7fa2b3800640 1 -- 172.21.15.89:0/1954918953 shutdown_connections 2024-09-10T08:48:58.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.930+0000 7fa2b3800640 1 --2- 172.21.15.89:0/1954918953 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa2900a2b90 0x7fa2900a5050 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.930+0000 7fa2b3800640 1 --2- 172.21.15.89:0/1954918953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2ac1058d0 0x7fa2ac1a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.930+0000 7fa2b3800640 1 --2- 172.21.15.89:0/1954918953 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2ac104f10 0x7fa2ac19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.930+0000 7fa2b3800640 1 --2- 172.21.15.89:0/1954918953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2ac103d20 0x7fa2ac19a090 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:58.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.930+0000 7fa2b3800640 1 -- 172.21.15.89:0/1954918953 >> 172.21.15.89:0/1954918953 conn(0x7fa2ac0ff4a0 msgr2=0x7fa2ac108320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:58.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.930+0000 7fa2b3800640 1 -- 172.21.15.89:0/1954918953 shutdown_connections 2024-09-10T08:48:58.931 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:58.930+0000 7fa2b3800640 1 -- 172.21.15.89:0/1954918953 wait complete. 2024-09-10T08:48:58.947 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T08:48:59.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.106+0000 7f553c600640 1 -- 172.21.15.89:0/3661676064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55341036b0 msgr2=0x7f5534109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.106+0000 7f553c600640 1 --2- 172.21.15.89:0/3661676064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55341036b0 0x7f5534109f40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f552c009ca0 tx=0x7f552c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.106+0000 7f553c600640 1 -- 172.21.15.89:0/3661676064 shutdown_connections 2024-09-10T08:48:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.106+0000 7f553c600640 1 --2- 172.21.15.89:0/3661676064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55341036b0 0x7f5534109f40 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.106+0000 7f553c600640 1 --2- 172.21.15.89:0/3661676064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5534102cf0 0x7f5534103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.106+0000 7f553c600640 1 --2- 172.21.15.89:0/3661676064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5534101b00 0x7f5534101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.106+0000 7f553c600640 1 -- 172.21.15.89:0/3661676064 >> 172.21.15.89:0/3661676064 conn(0x7f55340fd2a0 msgr2=0x7f55340ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.106+0000 7f553c600640 1 -- 172.21.15.89:0/3661676064 shutdown_connections 2024-09-10T08:48:59.107 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.106+0000 7f553c600640 1 -- 172.21.15.89:0/3661676064 wait complete. 2024-09-10T08:48:59.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.107+0000 7f553c600640 1 Processor -- start 2024-09-10T08:48:59.109 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.107+0000 7f553c600640 1 -- start start 2024-09-10T08:48:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.107+0000 7f553c600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5534101b00 0x7f553419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.107+0000 7f553c600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5534102cf0 0x7f553419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.107+0000 7f553c600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55341036b0 0x7f55341a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.107+0000 7f553c600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f553410d7d0 con 0x7f5534101b00 2024-09-10T08:48:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.107+0000 7f553c600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f553410d650 con 0x7f5534102cf0 2024-09-10T08:48:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.107+0000 7f553c600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f553410d950 con 0x7f55341036b0 2024-09-10T08:48:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.108+0000 7f5539c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5534101b00 0x7f553419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.108+0000 7f5539200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5534102cf0 0x7f553419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.108+0000 7f5539200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5534102cf0 0x7f553419a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:36718/0 (socket says 172.21.15.89:36718) 2024-09-10T08:48:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.108+0000 7f5539200640 1 -- 172.21.15.89:0/3832133869 learned_addr learned my addr 172.21.15.89:0/3832133869 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.108+0000 7f553a600640 1 --2- 172.21.15.89:0/3832133869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55341036b0 0x7f55341a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.108+0000 7f5539c00640 1 -- 172.21.15.89:0/3832133869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55341036b0 msgr2=0x7f55341a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:59.110 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.108+0000 7f5539c00640 1 --2- 172.21.15.89:0/3832133869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55341036b0 0x7f55341a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.108+0000 7f5539c00640 1 -- 172.21.15.89:0/3832133869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5534102cf0 msgr2=0x7f553419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.108+0000 7f5539c00640 1 --2- 172.21.15.89:0/3832133869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5534102cf0 0x7f553419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.108+0000 7f5539c00640 1 -- 172.21.15.89:0/3832133869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55341a1c30 con 0x7f5534101b00 2024-09-10T08:48:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.108+0000 7f5539200640 1 --2- 172.21.15.89:0/3832133869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5534102cf0 0x7f553419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.108+0000 7f5539c00640 1 --2- 172.21.15.89:0/3832133869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5534101b00 0x7f553419a010 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f552800e9c0 tx=0x7f552800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.109+0000 7f5532000640 1 -- 172.21.15.89:0/3832133869 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552800ccc0 con 0x7f5534101b00 2024-09-10T08:48:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.109+0000 7f5532000640 1 -- 172.21.15.89:0/3832133869 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55280035f0 con 0x7f5534101b00 2024-09-10T08:48:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.109+0000 7f5532000640 1 -- 172.21.15.89:0/3832133869 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552800ce60 con 0x7f5534101b00 2024-09-10T08:48:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.109+0000 7f553c600640 1 -- 172.21.15.89:0/3832133869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55341a1f20 con 0x7f5534101b00 2024-09-10T08:48:59.111 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.109+0000 7f553c600640 1 -- 172.21.15.89:0/3832133869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55341a2330 con 0x7f5534101b00 2024-09-10T08:48:59.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.110+0000 7f553c600640 1 -- 172.21.15.89:0/3832133869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55341a25f0 con 0x7f5534101b00 2024-09-10T08:48:59.112 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.112+0000 7f5532000640 1 -- 172.21.15.89:0/3832133869 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f55280037a0 con 0x7f5534101b00 2024-09-10T08:48:59.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.113+0000 7f5532000640 1 --2- 172.21.15.89:0/3832133869 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f551c0a2c10 0x7f551c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:59.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.113+0000 7f5532000640 1 -- 172.21.15.89:0/3832133869 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f55280cf930 con 0x7f5534101b00 2024-09-10T08:48:59.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.113+0000 7f5539200640 1 --2- 172.21.15.89:0/3832133869 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f551c0a2c10 0x7f551c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:59.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.114+0000 7f5539200640 1 --2- 172.21.15.89:0/3832133869 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f551c0a2c10 0x7f551c0a50d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f55200095f0 tx=0x7f5520005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:59.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.117+0000 7f5532000640 1 -- 172.21.15.89:0/3832133869 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f552809c460 con 0x7f5534101b00 2024-09-10T08:48:59.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.293+0000 7f553c600640 1 -- 172.21.15.89:0/3832133869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f55341a2eb0 con 0x7f5534101b00 2024-09-10T08:48:59.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.293+0000 7f5532000640 1 -- 172.21.15.89:0/3832133869 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7f552809bd60 con 0x7f5534101b00 2024-09-10T08:48:59.293 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 64 2024-09-10T08:48:59.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:59 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/4290026420' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:48:59.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:59 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2904119100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:48:59.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:48:59 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1954918953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T08:48:59.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.295+0000 7f553c600640 1 -- 172.21.15.89:0/3832133869 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f551c0a2c10 msgr2=0x7f551c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:59.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.295+0000 7f553c600640 1 --2- 172.21.15.89:0/3832133869 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f551c0a2c10 0x7f551c0a50d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f55200095f0 tx=0x7f5520005f90 comp rx=0 tx=0).stop 2024-09-10T08:48:59.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.295+0000 7f553c600640 1 -- 172.21.15.89:0/3832133869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5534101b00 msgr2=0x7f553419a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:59.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.295+0000 7f553c600640 1 --2- 172.21.15.89:0/3832133869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5534101b00 0x7f553419a010 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f552800e9c0 tx=0x7f552800ee80 comp rx=0 tx=0).stop 2024-09-10T08:48:59.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.295+0000 7f553c600640 1 -- 172.21.15.89:0/3832133869 shutdown_connections 2024-09-10T08:48:59.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.295+0000 7f553c600640 1 --2- 172.21.15.89:0/3832133869 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f551c0a2c10 0x7f551c0a50d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.295+0000 7f553c600640 1 --2- 172.21.15.89:0/3832133869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55341036b0 0x7f55341a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.295+0000 7f553c600640 1 --2- 172.21.15.89:0/3832133869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5534102cf0 0x7f553419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.295+0000 7f553c600640 1 --2- 172.21.15.89:0/3832133869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5534101b00 0x7f553419a010 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.295+0000 7f553c600640 1 -- 172.21.15.89:0/3832133869 >> 172.21.15.89:0/3832133869 conn(0x7f55340fd2a0 msgr2=0x7f5534074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:59.296 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.296+0000 7f553c600640 1 -- 172.21.15.89:0/3832133869 shutdown_connections 2024-09-10T08:48:59.297 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.296+0000 7f553c600640 1 -- 172.21.15.89:0/3832133869 wait complete. 2024-09-10T08:48:59.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:59 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/4290026420' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:48:59.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:59 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2904119100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:48:59.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:48:59 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1954918953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T08:48:59.313 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-10T08:48:59.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.469+0000 7f6e8a200640 1 -- 172.21.15.89:0/1981804726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e84077df0 msgr2=0x7f6e84078250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:59.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.469+0000 7f6e8a200640 1 --2- 172.21.15.89:0/1981804726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e84077df0 0x7f6e84078250 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6e70009900 tx=0x7f6e7002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.469+0000 7f6e8a200640 1 -- 172.21.15.89:0/1981804726 shutdown_connections 2024-09-10T08:48:59.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.469+0000 7f6e8a200640 1 --2- 172.21.15.89:0/1981804726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e84078790 0x7f6e84079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.469+0000 7f6e8a200640 1 --2- 172.21.15.89:0/1981804726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e84077df0 0x7f6e84078250 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.469+0000 7f6e8a200640 1 --2- 172.21.15.89:0/1981804726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e84076bb0 0x7f6e84076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.469+0000 7f6e8a200640 1 -- 172.21.15.89:0/1981804726 >> 172.21.15.89:0/1981804726 conn(0x7f6e840ff470 msgr2=0x7f6e841018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:59.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.469+0000 7f6e8a200640 1 -- 172.21.15.89:0/1981804726 shutdown_connections 2024-09-10T08:48:59.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.469+0000 7f6e8a200640 1 -- 172.21.15.89:0/1981804726 wait complete. 2024-09-10T08:48:59.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:59 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/4290026420' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:48:59.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:59 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2904119100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:48:59.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:48:59 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1954918953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T08:48:59.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.470+0000 7f6e8a200640 1 Processor -- start 2024-09-10T08:48:59.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.471+0000 7f6e8a200640 1 -- start start 2024-09-10T08:48:59.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.471+0000 7f6e8a200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e84076bb0 0x7f6e8419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:59.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.471+0000 7f6e8a200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e84077df0 0x7f6e8419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:59.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.471+0000 7f6e8a200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e84078790 0x7f6e841a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:59.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.471+0000 7f6e8a200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6e84113e10 con 0x7f6e84077df0 2024-09-10T08:48:59.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.471+0000 7f6e8a200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6e84113c90 con 0x7f6e84076bb0 2024-09-10T08:48:59.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.471+0000 7f6e8a200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6e84113f90 con 0x7f6e84078790 2024-09-10T08:48:59.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.471+0000 7f6e82a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e84077df0 0x7f6e8419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:59.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.471+0000 7f6e83400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e84076bb0 0x7f6e8419e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:59.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.471+0000 7f6e82a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e84077df0 0x7f6e8419e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42184/0 (socket says 172.21.15.89:42184) 2024-09-10T08:48:59.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.471+0000 7f6e82a00640 1 -- 172.21.15.89:0/440687271 learned_addr learned my addr 172.21.15.89:0/440687271 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:59.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.471+0000 7f6e83e00640 1 --2- 172.21.15.89:0/440687271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e84078790 0x7f6e841a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:59.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.472+0000 7f6e83400640 1 -- 172.21.15.89:0/440687271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e84078790 msgr2=0x7f6e841a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:59.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.472+0000 7f6e83400640 1 --2- 172.21.15.89:0/440687271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e84078790 0x7f6e841a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.472+0000 7f6e83400640 1 -- 172.21.15.89:0/440687271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e84077df0 msgr2=0x7f6e8419e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:59.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.472+0000 7f6e83400640 1 --2- 172.21.15.89:0/440687271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e84077df0 0x7f6e8419e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.472+0000 7f6e83400640 1 -- 172.21.15.89:0/440687271 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e841a6090 con 0x7f6e84076bb0 2024-09-10T08:48:59.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.472+0000 7f6e82a00640 1 --2- 172.21.15.89:0/440687271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e84077df0 0x7f6e8419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:59.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.472+0000 7f6e83400640 1 --2- 172.21.15.89:0/440687271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e84076bb0 0x7f6e8419e470 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6e74005fd0 tx=0x7f6e7400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:59.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.472+0000 7f6e7be00640 1 -- 172.21.15.89:0/440687271 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74009c30 con 0x7f6e84076bb0 2024-09-10T08:48:59.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.472+0000 7f6e8a200640 1 -- 172.21.15.89:0/440687271 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e841a6380 con 0x7f6e84076bb0 2024-09-10T08:48:59.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.472+0000 7f6e7be00640 1 -- 172.21.15.89:0/440687271 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e74009dd0 con 0x7f6e84076bb0 2024-09-10T08:48:59.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.472+0000 7f6e7be00640 1 -- 172.21.15.89:0/440687271 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74010460 con 0x7f6e84076bb0 2024-09-10T08:48:59.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.472+0000 7f6e8a200640 1 -- 172.21.15.89:0/440687271 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e841a68d0 con 0x7f6e84076bb0 2024-09-10T08:48:59.475 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.473+0000 7f6e8a200640 1 -- 172.21.15.89:0/440687271 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e84110830 con 0x7f6e84076bb0 2024-09-10T08:48:59.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.475+0000 7f6e7be00640 1 -- 172.21.15.89:0/440687271 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6e74003f50 con 0x7f6e84076bb0 2024-09-10T08:48:59.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.476+0000 7f6e7be00640 1 --2- 172.21.15.89:0/440687271 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6e680a2b90 0x7f6e680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:59.476 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.477+0000 7f6e7be00640 1 -- 172.21.15.89:0/440687271 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f6e740cf3e0 con 0x7f6e84076bb0 2024-09-10T08:48:59.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.477+0000 7f6e82a00640 1 --2- 172.21.15.89:0/440687271 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6e680a2b90 0x7f6e680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:59.477 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.477+0000 7f6e82a00640 1 --2- 172.21.15.89:0/440687271 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6e680a2b90 0x7f6e680a5050 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6e8419f980 tx=0x7f6e700315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:59.478 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.479+0000 7f6e7be00640 1 -- 172.21.15.89:0/440687271 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e74017020 con 0x7f6e84076bb0 2024-09-10T08:48:59.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.653+0000 7f6e8a200640 1 -- 172.21.15.89:0/440687271 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6e84062df0 con 0x7f6e84076bb0 2024-09-10T08:48:59.654 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.654+0000 7f6e7be00640 1 -- 172.21.15.89:0/440687271 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) ==== 103+0+11 (secure 0 0 0) 0x7f6e7409b8c0 con 0x7f6e84076bb0 2024-09-10T08:48:59.654 INFO:teuthology.orchestra.run.smithi089.stdout:pg_num: 32 2024-09-10T08:48:59.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.657+0000 7f6e8a200640 1 -- 172.21.15.89:0/440687271 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6e680a2b90 msgr2=0x7f6e680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:59.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.657+0000 7f6e8a200640 1 --2- 172.21.15.89:0/440687271 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6e680a2b90 0x7f6e680a5050 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6e8419f980 tx=0x7f6e700315b0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.657+0000 7f6e8a200640 1 -- 172.21.15.89:0/440687271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e84076bb0 msgr2=0x7f6e8419e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:59.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.657+0000 7f6e8a200640 1 --2- 172.21.15.89:0/440687271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e84076bb0 0x7f6e8419e470 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6e74005fd0 tx=0x7f6e7400dde0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.657+0000 7f6e8a200640 1 -- 172.21.15.89:0/440687271 shutdown_connections 2024-09-10T08:48:59.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.657+0000 7f6e8a200640 1 --2- 172.21.15.89:0/440687271 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f6e680a2b90 0x7f6e680a5050 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.657+0000 7f6e8a200640 1 --2- 172.21.15.89:0/440687271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e84078790 0x7f6e841a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.657+0000 7f6e8a200640 1 --2- 172.21.15.89:0/440687271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e84077df0 0x7f6e8419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.657+0000 7f6e8a200640 1 --2- 172.21.15.89:0/440687271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6e84076bb0 0x7f6e8419e470 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.657+0000 7f6e8a200640 1 -- 172.21.15.89:0/440687271 >> 172.21.15.89:0/440687271 conn(0x7f6e840ff470 msgr2=0x7f6e84100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:59.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.657+0000 7f6e8a200640 1 -- 172.21.15.89:0/440687271 shutdown_connections 2024-09-10T08:48:59.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.657+0000 7f6e8a200640 1 -- 172.21.15.89:0/440687271 wait complete. 2024-09-10T08:48:59.673 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-10T08:48:59.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.828+0000 7f7857e00640 1 -- 172.21.15.89:0/3344556273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7858103050 msgr2=0x7f7858103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:59.828 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.828+0000 7f7857e00640 1 --2- 172.21.15.89:0/3344556273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7858103050 0x7f7858103450 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7848009900 tx=0x7f784802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.828+0000 7f7857e00640 1 -- 172.21.15.89:0/3344556273 shutdown_connections 2024-09-10T08:48:59.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.828+0000 7f7857e00640 1 --2- 172.21.15.89:0/3344556273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7858101f40 0x7f7858109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.828+0000 7f7857e00640 1 --2- 172.21.15.89:0/3344556273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7858101580 0x7f7858101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.828+0000 7f7857e00640 1 --2- 172.21.15.89:0/3344556273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7858103050 0x7f7858103450 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.828+0000 7f7857e00640 1 -- 172.21.15.89:0/3344556273 >> 172.21.15.89:0/3344556273 conn(0x7f78580fd110 msgr2=0x7f78580ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:59.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.828+0000 7f7857e00640 1 -- 172.21.15.89:0/3344556273 shutdown_connections 2024-09-10T08:48:59.829 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.828+0000 7f7857e00640 1 -- 172.21.15.89:0/3344556273 wait complete. 2024-09-10T08:48:59.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.829+0000 7f7857e00640 1 Processor -- start 2024-09-10T08:48:59.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.829+0000 7f7857e00640 1 -- start start 2024-09-10T08:48:59.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.829+0000 7f7857e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7858101580 0x7f7858197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:59.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.829+0000 7f7857e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7858101f40 0x7f7858198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:59.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.829+0000 7f7857e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7858103050 0x7f785819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:59.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.829+0000 7f7857e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f785810d6f0 con 0x7f7858101580 2024-09-10T08:48:59.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.829+0000 7f7857e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f785810d570 con 0x7f7858101f40 2024-09-10T08:48:59.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.829+0000 7f7857e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f785810d870 con 0x7f7858103050 2024-09-10T08:48:59.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.829+0000 7f7856a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7858101580 0x7f7858197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:59.832 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.829+0000 7f7856a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7858101580 0x7f7858197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:42206/0 (socket says 172.21.15.89:42206) 2024-09-10T08:48:59.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.829+0000 7f7856a00640 1 -- 172.21.15.89:0/4279437873 learned_addr learned my addr 172.21.15.89:0/4279437873 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:48:59.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.830+0000 7f784fe00640 1 --2- 172.21.15.89:0/4279437873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7858101f40 0x7f7858198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:59.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.830+0000 7f7857400640 1 --2- 172.21.15.89:0/4279437873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7858103050 0x7f785819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:59.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.830+0000 7f7856a00640 1 -- 172.21.15.89:0/4279437873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7858103050 msgr2=0x7f785819f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:59.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.830+0000 7f7856a00640 1 --2- 172.21.15.89:0/4279437873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7858103050 0x7f785819f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.830+0000 7f7856a00640 1 -- 172.21.15.89:0/4279437873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7858101f40 msgr2=0x7f7858198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:59.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.830+0000 7f7856a00640 1 --2- 172.21.15.89:0/4279437873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7858101f40 0x7f7858198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:59.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.830+0000 7f7856a00640 1 -- 172.21.15.89:0/4279437873 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785819fa20 con 0x7f7858101580 2024-09-10T08:48:59.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.830+0000 7f7856a00640 1 --2- 172.21.15.89:0/4279437873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7858101580 0x7f7858197e00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7848031610 tx=0x7f7848004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:59.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.830+0000 7f785ca00640 1 -- 172.21.15.89:0/4279437873 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784803f030 con 0x7f7858101580 2024-09-10T08:48:59.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.830+0000 7f785ca00640 1 -- 172.21.15.89:0/4279437873 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7848003ad0 con 0x7f7858101580 2024-09-10T08:48:59.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.830+0000 7f785ca00640 1 -- 172.21.15.89:0/4279437873 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848035070 con 0x7f7858101580 2024-09-10T08:48:59.833 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.830+0000 7f7857e00640 1 -- 172.21.15.89:0/4279437873 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f785819fcb0 con 0x7f7858101580 2024-09-10T08:48:59.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.831+0000 7f7857e00640 1 -- 172.21.15.89:0/4279437873 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78581a0150 con 0x7f7858101580 2024-09-10T08:48:59.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.832+0000 7f7857e00640 1 -- 172.21.15.89:0/4279437873 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7858101a00 con 0x7f7858101580 2024-09-10T08:48:59.834 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.834+0000 7f785ca00640 1 -- 172.21.15.89:0/4279437873 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7848042030 con 0x7f7858101580 2024-09-10T08:48:59.835 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.836+0000 7f785ca00640 1 --2- 172.21.15.89:0/4279437873 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f783c0a2c10 0x7f783c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:59.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.836+0000 7f785ca00640 1 -- 172.21.15.89:0/4279437873 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f78480ef870 con 0x7f7858101580 2024-09-10T08:48:59.836 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.836+0000 7f784fe00640 1 --2- 172.21.15.89:0/4279437873 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f783c0a2c10 0x7f783c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:59.837 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.836+0000 7f784fe00640 1 --2- 172.21.15.89:0/4279437873 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f783c0a2c10 0x7f783c0a50d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7858199310 tx=0x7f7840005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:59.839 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:48:59.840+0000 7f785ca00640 1 -- 172.21.15.89:0/4279437873 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f78480f1e20 con 0x7f7858101580 2024-09-10T08:49:00.065 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:00.065+0000 7f7857e00640 1 -- 172.21.15.89:0/4279437873 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f7858062df0 con 0x7f7858101580 2024-09-10T08:49:00.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:00.075+0000 7f785ca00640 1 -- 172.21.15.89:0/4279437873 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740371 (secure 0 0 0) 0x7f78480bbd50 con 0x7f7858101580 2024-09-10T08:49:00.086 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:49:00.086 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:49:00.086 INFO:teuthology.orchestra.run.smithi089.stdout: "epoch": 20, 2024-09-10T08:49:00.086 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.086 INFO:teuthology.orchestra.run.smithi089.stdout: "active_gid": 14150, 2024-09-10T08:49:00.086 INFO:teuthology.orchestra.run.smithi089.stdout: "active_name": "x", 2024-09-10T08:49:00.086 INFO:teuthology.orchestra.run.smithi089.stdout: "active_addrs": { 2024-09-10T08:49:00.086 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-09-10T08:49:00.086 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.086 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-09-10T08:49:00.086 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6800", 2024-09-10T08:49:00.087 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 1633345099 2024-09-10T08:49:00.087 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.087 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.087 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v1", 2024-09-10T08:49:00.087 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:6801", 2024-09-10T08:49:00.087 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 1633345099 2024-09-10T08:49:00.087 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.087 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-10T08:49:00.087 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.087 INFO:teuthology.orchestra.run.smithi089.stdout: "active_addr": "172.21.15.89:6801/1633345099", 2024-09-10T08:49:00.087 INFO:teuthology.orchestra.run.smithi089.stdout: "active_change": "2024-09-10T08:29:41.472108+0000", 2024-09-10T08:49:00.087 INFO:teuthology.orchestra.run.smithi089.stdout: "active_mgr_features": 4540701547738038271, 2024-09-10T08:49:00.087 INFO:teuthology.orchestra.run.smithi089.stdout: "available": true, 2024-09-10T08:49:00.088 INFO:teuthology.orchestra.run.smithi089.stdout: "standbys": [ 2024-09-10T08:49:00.088 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.088 INFO:teuthology.orchestra.run.smithi089.stdout: "gid": 24109, 2024-09-10T08:49:00.088 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "z", 2024-09-10T08:49:00.088 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr_features": 4540701547738038271, 2024-09-10T08:49:00.088 INFO:teuthology.orchestra.run.smithi089.stdout: "available_modules": [ 2024-09-10T08:49:00.088 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.088 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "alerts", 2024-09-10T08:49:00.088 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.088 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.088 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.088 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-10T08:49:00.088 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-10T08:49:00.089 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.089 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.089 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.089 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-10T08:49:00.089 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.089 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.089 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.089 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T08:49:00.089 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.089 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.089 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.089 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.089 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.089 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.090 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.090 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.090 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.090 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.090 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.090 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.090 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.090 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.090 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.090 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.090 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.090 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.090 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.090 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.091 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.091 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.091 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.091 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.091 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.091 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.091 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.091 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.091 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.091 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.091 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.091 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.091 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.091 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.092 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.092 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.092 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.092 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.092 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.092 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.092 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.092 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.092 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.092 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.092 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.092 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.092 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.093 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.093 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.093 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.093 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.093 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.093 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.093 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.093 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.093 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.093 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.093 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.093 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.093 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.094 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.094 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.094 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.094 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.094 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.094 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.094 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.094 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.094 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.094 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.094 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.094 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.094 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.094 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_destination": { 2024-09-10T08:49:00.094 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_destination", 2024-09-10T08:49:00.095 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.095 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.095 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.095 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.095 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.095 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.095 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.095 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Email address to send alerts to", 2024-09-10T08:49:00.095 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.095 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.095 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.095 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.095 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_from_name": { 2024-09-10T08:49:00.095 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_from_name", 2024-09-10T08:49:00.096 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.096 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.096 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.096 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "Ceph", 2024-09-10T08:49:00.096 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.096 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.096 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.096 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Email From: name", 2024-09-10T08:49:00.096 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.097 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.097 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.097 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.097 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_host": { 2024-09-10T08:49:00.097 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_host", 2024-09-10T08:49:00.097 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.097 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.097 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.097 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.097 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.097 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.097 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.097 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP server", 2024-09-10T08:49:00.098 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.098 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.098 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.098 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.098 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_password": { 2024-09-10T08:49:00.098 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_password", 2024-09-10T08:49:00.098 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.098 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.098 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.098 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.098 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.098 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.098 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.098 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Password to authenticate with", 2024-09-10T08:49:00.099 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.099 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.099 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.099 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.099 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_port": { 2024-09-10T08:49:00.099 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_port", 2024-09-10T08:49:00.099 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.099 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.099 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.099 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "465", 2024-09-10T08:49:00.099 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.099 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.099 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.099 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP port", 2024-09-10T08:49:00.100 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.100 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.100 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.100 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.100 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_sender": { 2024-09-10T08:49:00.100 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_sender", 2024-09-10T08:49:00.100 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.100 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.100 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.100 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.100 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.100 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.100 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.101 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP envelope sender", 2024-09-10T08:49:00.101 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.101 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.101 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.101 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.101 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_ssl": { 2024-09-10T08:49:00.101 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_ssl", 2024-09-10T08:49:00.101 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.101 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.101 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.101 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.101 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.102 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.102 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.102 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T08:49:00.102 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.102 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.102 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.102 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.102 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_user": { 2024-09-10T08:49:00.102 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_user", 2024-09-10T08:49:00.102 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.102 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.102 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.102 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.102 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.103 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.103 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.103 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "User to authenticate as", 2024-09-10T08:49:00.103 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.103 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.103 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.103 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.103 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.103 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.103 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.103 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "balancer", 2024-09-10T08:49:00.103 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.103 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.103 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.104 INFO:teuthology.orchestra.run.smithi089.stdout: "active": { 2024-09-10T08:49:00.104 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "active", 2024-09-10T08:49:00.104 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.104 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.104 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.104 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.104 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.104 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.104 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.104 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T08:49:00.104 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.104 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.104 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.105 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.105 INFO:teuthology.orchestra.run.smithi089.stdout: "begin_time": { 2024-09-10T08:49:00.105 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "begin_time", 2024-09-10T08:49:00.105 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.105 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.105 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.105 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0000", 2024-09-10T08:49:00.105 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.105 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.105 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.105 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T08:49:00.105 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T08:49:00.105 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.105 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.106 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.106 INFO:teuthology.orchestra.run.smithi089.stdout: "begin_weekday": { 2024-09-10T08:49:00.106 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "begin_weekday", 2024-09-10T08:49:00.106 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-10T08:49:00.106 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.106 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.106 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-10T08:49:00.106 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0", 2024-09-10T08:49:00.106 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "6", 2024-09-10T08:49:00.106 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.106 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T08:49:00.106 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T08:49:00.106 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.107 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.107 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.107 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_max_iterations": { 2024-09-10T08:49:00.107 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_max_iterations", 2024-09-10T08:49:00.107 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-10T08:49:00.107 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.107 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.107 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "25", 2024-09-10T08:49:00.107 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-10T08:49:00.107 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "250", 2024-09-10T08:49:00.107 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.107 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T08:49:00.107 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.107 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.108 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.108 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.108 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_metrics": { 2024-09-10T08:49:00.108 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_metrics", 2024-09-10T08:49:00.108 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.108 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.108 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.108 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T08:49:00.108 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.108 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.108 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.108 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T08:49:00.109 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T08:49:00.109 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.109 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.109 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.109 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_step": { 2024-09-10T08:49:00.109 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_step", 2024-09-10T08:49:00.109 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:00.109 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.109 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.109 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0.5", 2024-09-10T08:49:00.109 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0.001", 2024-09-10T08:49:00.109 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "0.999", 2024-09-10T08:49:00.109 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.110 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "aggressiveness of optimization", 2024-09-10T08:49:00.110 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T08:49:00.110 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.110 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.110 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.110 INFO:teuthology.orchestra.run.smithi089.stdout: "end_time": { 2024-09-10T08:49:00.110 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "end_time", 2024-09-10T08:49:00.110 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.110 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.110 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.110 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "2359", 2024-09-10T08:49:00.110 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.111 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.111 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.111 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T08:49:00.111 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T08:49:00.111 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.111 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.111 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.111 INFO:teuthology.orchestra.run.smithi089.stdout: "end_weekday": { 2024-09-10T08:49:00.111 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "end_weekday", 2024-09-10T08:49:00.111 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-10T08:49:00.111 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.111 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.111 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-10T08:49:00.111 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0", 2024-09-10T08:49:00.112 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "6", 2024-09-10T08:49:00.112 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.112 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T08:49:00.112 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T08:49:00.112 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.112 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.112 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.112 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.112 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.112 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.112 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.112 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.112 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.113 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.113 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.113 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.113 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.113 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.113 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.113 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.113 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.113 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.113 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.113 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.113 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.113 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.114 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.114 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.114 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.114 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.114 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.114 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.114 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.114 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.114 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.114 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.114 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.114 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.114 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.114 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.115 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.115 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.115 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.115 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.115 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.115 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.115 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.115 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.115 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.115 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.115 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.115 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.115 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.116 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.116 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.116 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.116 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.116 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.116 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.116 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.116 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.116 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.116 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.116 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.116 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.116 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.116 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.117 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.117 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.117 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.117 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.117 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.117 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.117 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.117 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.117 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.117 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.117 INFO:teuthology.orchestra.run.smithi089.stdout: "min_score": { 2024-09-10T08:49:00.117 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "min_score", 2024-09-10T08:49:00.117 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:00.117 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.118 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.118 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-10T08:49:00.118 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.118 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.118 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.118 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T08:49:00.118 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.118 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.118 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.118 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.118 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": { 2024-09-10T08:49:00.118 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mode", 2024-09-10T08:49:00.118 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.119 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.119 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.119 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "upmap", 2024-09-10T08:49:00.119 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.119 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.119 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.119 INFO:teuthology.orchestra.run.smithi089.stdout: "crush-compat", 2024-09-10T08:49:00.119 INFO:teuthology.orchestra.run.smithi089.stdout: "none", 2024-09-10T08:49:00.119 INFO:teuthology.orchestra.run.smithi089.stdout: "read", 2024-09-10T08:49:00.119 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap", 2024-09-10T08:49:00.119 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap-read" 2024-09-10T08:49:00.119 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.119 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Balancer mode", 2024-09-10T08:49:00.120 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.120 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.120 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.120 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.120 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_ids": { 2024-09-10T08:49:00.120 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pool_ids", 2024-09-10T08:49:00.120 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.120 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.120 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.120 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.120 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.120 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.120 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.120 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T08:49:00.121 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.121 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.121 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.121 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.121 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-10T08:49:00.121 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-10T08:49:00.121 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.121 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.121 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.121 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-10T08:49:00.121 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.121 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.121 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.121 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T08:49:00.122 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.122 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.122 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.122 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.122 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap_max_deviation": { 2024-09-10T08:49:00.122 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "upmap_max_deviation", 2024-09-10T08:49:00.122 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.122 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.122 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.122 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-10T08:49:00.122 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-10T08:49:00.122 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.122 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.122 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T08:49:00.123 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T08:49:00.123 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.123 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.123 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.123 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap_max_optimizations": { 2024-09-10T08:49:00.123 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "upmap_max_optimizations", 2024-09-10T08:49:00.123 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-10T08:49:00.123 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.123 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.123 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:00.123 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.123 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.123 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.123 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T08:49:00.124 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.124 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.124 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.124 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.124 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.124 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.124 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.124 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephadm", 2024-09-10T08:49:00.124 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.124 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.124 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.124 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_down_multiplier": { 2024-09-10T08:49:00.124 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_down_multiplier", 2024-09-10T08:49:00.124 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:00.124 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.125 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.125 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3.0", 2024-09-10T08:49:00.125 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.125 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.125 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.125 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T08:49:00.125 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.125 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.125 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.125 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.125 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_refresh_rate": { 2024-09-10T08:49:00.125 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_refresh_rate", 2024-09-10T08:49:00.125 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.125 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.126 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.126 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "20", 2024-09-10T08:49:00.126 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.126 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.126 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.126 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T08:49:00.126 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.126 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.126 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.126 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.126 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_starting_port": { 2024-09-10T08:49:00.126 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_starting_port", 2024-09-10T08:49:00.126 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.126 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.127 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.127 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "4721", 2024-09-10T08:49:00.127 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.127 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.127 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.127 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T08:49:00.127 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.127 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.127 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.127 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.127 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_ptrace": { 2024-09-10T08:49:00.127 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_ptrace", 2024-09-10T08:49:00.128 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.128 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.128 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.128 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.128 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.128 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.128 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.128 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T08:49:00.128 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T08:49:00.128 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.128 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.128 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.128 INFO:teuthology.orchestra.run.smithi089.stdout: "autotune_interval": { 2024-09-10T08:49:00.128 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "autotune_interval", 2024-09-10T08:49:00.129 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.129 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.129 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.129 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-10T08:49:00.129 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.129 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.129 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.129 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T08:49:00.129 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.129 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.129 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.130 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.130 INFO:teuthology.orchestra.run.smithi089.stdout: "autotune_memory_target_ratio": { 2024-09-10T08:49:00.130 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T08:49:00.130 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:00.130 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.130 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.130 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0.7", 2024-09-10T08:49:00.130 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.130 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.130 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.130 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T08:49:00.130 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.130 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.130 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.131 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.131 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm_log_destination": { 2024-09-10T08:49:00.131 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephadm_log_destination", 2024-09-10T08:49:00.131 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.131 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.131 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.131 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.131 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.131 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.131 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.131 INFO:teuthology.orchestra.run.smithi089.stdout: "file", 2024-09-10T08:49:00.131 INFO:teuthology.orchestra.run.smithi089.stdout: "file,syslog", 2024-09-10T08:49:00.131 INFO:teuthology.orchestra.run.smithi089.stdout: "syslog" 2024-09-10T08:49:00.131 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.132 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T08:49:00.132 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.132 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.132 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.132 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.132 INFO:teuthology.orchestra.run.smithi089.stdout: "cgroups_split": { 2024-09-10T08:49:00.132 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cgroups_split", 2024-09-10T08:49:00.132 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.132 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.132 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.132 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.132 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.132 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.132 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.133 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T08:49:00.133 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.133 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.133 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.133 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.133 INFO:teuthology.orchestra.run.smithi089.stdout: "config_checks_enabled": { 2024-09-10T08:49:00.133 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_checks_enabled", 2024-09-10T08:49:00.133 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.133 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.133 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.133 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.133 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.133 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.133 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.134 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T08:49:00.134 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.134 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.134 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.134 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.134 INFO:teuthology.orchestra.run.smithi089.stdout: "config_dashboard": { 2024-09-10T08:49:00.134 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_dashboard", 2024-09-10T08:49:00.134 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.134 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.134 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.134 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.134 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.134 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.134 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.134 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T08:49:00.135 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.135 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.135 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.135 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.135 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_alertmanager": { 2024-09-10T08:49:00.135 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_alertmanager", 2024-09-10T08:49:00.135 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.135 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.135 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.135 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T08:49:00.135 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.135 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.135 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.135 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-10T08:49:00.136 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.136 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.136 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.136 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_base": { 2024-09-10T08:49:00.136 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_base", 2024-09-10T08:49:00.136 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.136 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.136 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.136 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T08:49:00.136 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.136 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.136 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.136 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Container image name, without the tag", 2024-09-10T08:49:00.137 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.137 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.137 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.137 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.137 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_elasticsearch": { 2024-09-10T08:49:00.137 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_elasticsearch", 2024-09-10T08:49:00.137 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.137 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.137 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.137 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T08:49:00.137 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.137 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.137 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.137 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "elasticsearch container image", 2024-09-10T08:49:00.138 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.138 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.138 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.138 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.138 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_grafana": { 2024-09-10T08:49:00.138 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_grafana", 2024-09-10T08:49:00.138 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.138 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.138 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.138 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T08:49:00.138 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.138 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.138 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.138 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-10T08:49:00.139 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.139 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.139 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.139 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_haproxy": { 2024-09-10T08:49:00.139 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_haproxy", 2024-09-10T08:49:00.139 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.139 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.139 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.139 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T08:49:00.139 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.139 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.139 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.140 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "HAproxy container image", 2024-09-10T08:49:00.140 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.140 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.140 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.140 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.140 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_agent": { 2024-09-10T08:49:00.140 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_agent", 2024-09-10T08:49:00.140 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.140 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.140 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.140 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T08:49:00.140 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.140 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.141 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.141 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger agent container image", 2024-09-10T08:49:00.141 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.141 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.141 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.141 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.141 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_collector": { 2024-09-10T08:49:00.141 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_collector", 2024-09-10T08:49:00.141 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.141 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.141 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.141 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T08:49:00.141 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.141 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.142 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.142 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger collector container image", 2024-09-10T08:49:00.142 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.142 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.142 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.142 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.142 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_query": { 2024-09-10T08:49:00.142 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_query", 2024-09-10T08:49:00.142 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.142 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.142 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.142 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T08:49:00.142 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.143 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.143 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.143 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger query container image", 2024-09-10T08:49:00.143 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.143 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.143 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.143 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_keepalived": { 2024-09-10T08:49:00.143 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_keepalived", 2024-09-10T08:49:00.143 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.144 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.144 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.144 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T08:49:00.144 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.144 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.144 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.144 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Keepalived container image", 2024-09-10T08:49:00.144 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.144 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.144 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.144 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.144 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_loki": { 2024-09-10T08:49:00.144 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_loki", 2024-09-10T08:49:00.145 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.145 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.145 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.145 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T08:49:00.145 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.145 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.145 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.145 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Loki container image", 2024-09-10T08:49:00.145 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.145 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.145 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.145 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.145 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_node_exporter": { 2024-09-10T08:49:00.145 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_node_exporter", 2024-09-10T08:49:00.146 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.146 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.146 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.146 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T08:49:00.146 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.146 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.146 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.146 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-10T08:49:00.146 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.146 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.146 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.146 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.146 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_nvmeof": { 2024-09-10T08:49:00.146 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_nvmeof", 2024-09-10T08:49:00.147 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.147 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.147 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.147 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T08:49:00.147 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.147 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.147 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.147 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Nvme-of container image", 2024-09-10T08:49:00.147 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.147 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.147 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.147 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.147 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_prometheus": { 2024-09-10T08:49:00.147 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_prometheus", 2024-09-10T08:49:00.148 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.148 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.148 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.148 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T08:49:00.148 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.148 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.148 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.149 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-10T08:49:00.149 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.149 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.149 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.149 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_promtail": { 2024-09-10T08:49:00.149 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_promtail", 2024-09-10T08:49:00.149 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.149 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.150 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.150 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T08:49:00.150 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.150 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.150 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.150 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Promtail container image", 2024-09-10T08:49:00.150 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.150 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.150 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.150 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_samba": { 2024-09-10T08:49:00.150 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_samba", 2024-09-10T08:49:00.150 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.150 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.150 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.151 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T08:49:00.151 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.151 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.151 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.151 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Samba/SMB container image", 2024-09-10T08:49:00.151 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.151 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.151 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.151 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.151 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_snmp_gateway": { 2024-09-10T08:49:00.151 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_snmp_gateway", 2024-09-10T08:49:00.151 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.151 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.152 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.152 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T08:49:00.152 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.152 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.152 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.152 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SNMP Gateway container image", 2024-09-10T08:49:00.152 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.152 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.152 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.152 INFO:teuthology.orchestra.run.smithi089.stdout: "container_init": { 2024-09-10T08:49:00.152 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_init", 2024-09-10T08:49:00.152 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.152 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.153 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.153 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.153 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.153 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.153 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.153 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T08:49:00.153 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.153 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.153 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.153 INFO:teuthology.orchestra.run.smithi089.stdout: "daemon_cache_timeout": { 2024-09-10T08:49:00.153 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "daemon_cache_timeout", 2024-09-10T08:49:00.154 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.154 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.154 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.154 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-10T08:49:00.154 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.154 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.154 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.154 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T08:49:00.154 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.154 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.154 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.154 INFO:teuthology.orchestra.run.smithi089.stdout: "default_cephadm_command_timeout": { 2024-09-10T08:49:00.155 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T08:49:00.155 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.155 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.155 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.155 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "900", 2024-09-10T08:49:00.155 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.155 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.155 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.155 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T08:49:00.155 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.155 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.155 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.155 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.155 INFO:teuthology.orchestra.run.smithi089.stdout: "default_registry": { 2024-09-10T08:49:00.156 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "default_registry", 2024-09-10T08:49:00.156 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.156 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.156 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.156 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io", 2024-09-10T08:49:00.156 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.156 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.156 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.156 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T08:49:00.156 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.156 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.156 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.156 INFO:teuthology.orchestra.run.smithi089.stdout: "device_cache_timeout": { 2024-09-10T08:49:00.157 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_cache_timeout", 2024-09-10T08:49:00.157 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.157 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.157 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.157 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1800", 2024-09-10T08:49:00.157 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.157 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.157 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.157 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache device inventory", 2024-09-10T08:49:00.157 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.157 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.157 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.157 INFO:teuthology.orchestra.run.smithi089.stdout: "device_enhanced_scan": { 2024-09-10T08:49:00.158 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_enhanced_scan", 2024-09-10T08:49:00.158 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.158 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.158 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.158 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.158 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.158 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.158 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.158 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T08:49:00.158 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.158 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.158 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.158 INFO:teuthology.orchestra.run.smithi089.stdout: "facts_cache_timeout": { 2024-09-10T08:49:00.159 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "facts_cache_timeout", 2024-09-10T08:49:00.159 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.159 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.159 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.159 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-10T08:49:00.159 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.159 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.159 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.159 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache host facts data", 2024-09-10T08:49:00.159 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.159 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.159 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.159 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.160 INFO:teuthology.orchestra.run.smithi089.stdout: "grafana_dashboards_path": { 2024-09-10T08:49:00.160 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "grafana_dashboards_path", 2024-09-10T08:49:00.160 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.160 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.160 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.160 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T08:49:00.160 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.160 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.160 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.160 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T08:49:00.160 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.160 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.160 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.161 INFO:teuthology.orchestra.run.smithi089.stdout: "host_check_interval": { 2024-09-10T08:49:00.161 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "host_check_interval", 2024-09-10T08:49:00.161 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.161 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.161 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.161 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-10T08:49:00.161 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.161 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.161 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.161 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to perform a host check", 2024-09-10T08:49:00.161 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.161 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.162 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.162 INFO:teuthology.orchestra.run.smithi089.stdout: "hw_monitoring": { 2024-09-10T08:49:00.162 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "hw_monitoring", 2024-09-10T08:49:00.162 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.162 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.162 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.162 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.162 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.162 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.162 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.162 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T08:49:00.162 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.162 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.163 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.163 INFO:teuthology.orchestra.run.smithi089.stdout: "inventory_list_all": { 2024-09-10T08:49:00.163 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "inventory_list_all", 2024-09-10T08:49:00.163 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.163 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.163 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.163 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.163 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.163 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.163 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.163 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T08:49:00.163 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.163 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.163 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.164 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.164 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.164 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.164 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.164 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.164 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.164 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.164 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.164 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.164 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.164 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.164 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.164 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.165 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.165 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.165 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.165 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.165 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.165 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.165 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.165 INFO:teuthology.orchestra.run.smithi089.stdout: "log_refresh_metadata": { 2024-09-10T08:49:00.165 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_refresh_metadata", 2024-09-10T08:49:00.165 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.165 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.165 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.165 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.166 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.166 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.166 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.166 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T08:49:00.166 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.166 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.166 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.166 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.166 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.166 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.166 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.166 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.166 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.167 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.167 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.167 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.167 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T08:49:00.167 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.167 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.167 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.167 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.167 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.167 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.167 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.167 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.167 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.168 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.168 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.168 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.168 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.168 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.168 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.168 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.168 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.168 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.168 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.168 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.168 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.168 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.169 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.169 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.169 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.169 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.169 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.169 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.169 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.169 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.169 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.169 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.169 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.169 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.169 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.170 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.170 INFO:teuthology.orchestra.run.smithi089.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T08:49:00.170 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T08:49:00.172 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.172 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.172 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.173 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.173 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.173 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.173 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.173 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T08:49:00.173 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.173 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.173 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.173 INFO:teuthology.orchestra.run.smithi089.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T08:49:00.173 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T08:49:00.173 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.173 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.173 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.174 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "*", 2024-09-10T08:49:00.174 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.174 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.174 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.174 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T08:49:00.174 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.174 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.174 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.174 INFO:teuthology.orchestra.run.smithi089.stdout: "max_count_per_host": { 2024-09-10T08:49:00.174 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_count_per_host", 2024-09-10T08:49:00.174 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.174 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.174 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.175 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:00.175 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.175 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.175 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.175 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "max number of daemons per service per host", 2024-09-10T08:49:00.175 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.175 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.175 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.175 INFO:teuthology.orchestra.run.smithi089.stdout: "max_osd_draining_count": { 2024-09-10T08:49:00.175 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_osd_draining_count", 2024-09-10T08:49:00.175 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.175 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.175 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.175 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:00.176 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.176 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.176 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.176 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T08:49:00.176 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.176 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.176 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.176 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.176 INFO:teuthology.orchestra.run.smithi089.stdout: "migration_current": { 2024-09-10T08:49:00.176 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "migration_current", 2024-09-10T08:49:00.176 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.176 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.176 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.176 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.177 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.177 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.177 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.177 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "internal - do not modify", 2024-09-10T08:49:00.177 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.177 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.177 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.177 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": { 2024-09-10T08:49:00.177 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mode", 2024-09-10T08:49:00.177 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.177 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.177 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.177 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "root", 2024-09-10T08:49:00.178 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.178 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.178 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.178 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm-package", 2024-09-10T08:49:00.178 INFO:teuthology.orchestra.run.smithi089.stdout: "root" 2024-09-10T08:49:00.178 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.178 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T08:49:00.178 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.178 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.178 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.178 INFO:teuthology.orchestra.run.smithi089.stdout: "oob_default_addr": { 2024-09-10T08:49:00.178 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "oob_default_addr", 2024-09-10T08:49:00.178 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.178 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.179 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.179 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "169.254.1.1", 2024-09-10T08:49:00.179 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.179 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.179 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.179 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T08:49:00.179 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.179 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.179 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.179 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.179 INFO:teuthology.orchestra.run.smithi089.stdout: "prometheus_alerts_path": { 2024-09-10T08:49:00.179 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prometheus_alerts_path", 2024-09-10T08:49:00.179 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.179 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.180 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.180 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T08:49:00.180 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.180 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.180 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.180 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T08:49:00.180 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.180 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.180 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.180 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.180 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_insecure": { 2024-09-10T08:49:00.180 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_insecure", 2024-09-10T08:49:00.181 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.181 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.181 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.181 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.181 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.181 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.181 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.181 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T08:49:00.181 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.181 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.181 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.181 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.181 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_password": { 2024-09-10T08:49:00.182 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_password", 2024-09-10T08:49:00.182 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.182 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.182 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.182 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.182 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.182 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.182 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.182 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T08:49:00.182 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.182 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.182 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.182 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.182 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_url": { 2024-09-10T08:49:00.183 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_url", 2024-09-10T08:49:00.183 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.183 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.183 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.183 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.183 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.183 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.183 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.183 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T08:49:00.183 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.183 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.183 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.183 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_username": { 2024-09-10T08:49:00.184 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_username", 2024-09-10T08:49:00.184 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.184 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.184 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.184 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.184 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.184 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.184 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.184 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T08:49:00.184 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.184 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.184 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.185 INFO:teuthology.orchestra.run.smithi089.stdout: "secure_monitoring_stack": { 2024-09-10T08:49:00.185 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "secure_monitoring_stack", 2024-09-10T08:49:00.185 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.185 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.185 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.185 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.185 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.185 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.185 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.185 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T08:49:00.185 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.185 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.185 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.186 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.186 INFO:teuthology.orchestra.run.smithi089.stdout: "service_discovery_port": { 2024-09-10T08:49:00.186 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "service_discovery_port", 2024-09-10T08:49:00.186 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.186 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.186 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.186 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8765", 2024-09-10T08:49:00.186 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.186 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.186 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.186 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "cephadm service discovery port", 2024-09-10T08:49:00.186 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.186 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.187 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.187 INFO:teuthology.orchestra.run.smithi089.stdout: "ssh_config_file": { 2024-09-10T08:49:00.187 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssh_config_file", 2024-09-10T08:49:00.187 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.187 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.187 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.187 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.187 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.187 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.187 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.187 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T08:49:00.187 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.188 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.188 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.188 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.188 INFO:teuthology.orchestra.run.smithi089.stdout: "use_agent": { 2024-09-10T08:49:00.188 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "use_agent", 2024-09-10T08:49:00.188 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.188 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.188 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.188 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.188 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.188 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.188 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.188 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T08:49:00.189 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.189 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.189 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.189 INFO:teuthology.orchestra.run.smithi089.stdout: "use_repo_digest": { 2024-09-10T08:49:00.189 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "use_repo_digest", 2024-09-10T08:49:00.189 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.189 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.189 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.189 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.189 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.189 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.189 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.189 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T08:49:00.190 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.190 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.190 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.190 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_failed_host_check": { 2024-09-10T08:49:00.190 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_failed_host_check", 2024-09-10T08:49:00.190 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.190 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.190 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.190 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.190 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.190 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.190 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.190 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T08:49:00.191 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.191 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.191 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.191 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_stray_daemons": { 2024-09-10T08:49:00.191 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_stray_daemons", 2024-09-10T08:49:00.191 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.191 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.191 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.191 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.191 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.191 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.191 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.191 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T08:49:00.192 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.192 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.192 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.192 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.192 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_stray_hosts": { 2024-09-10T08:49:00.192 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_stray_hosts", 2024-09-10T08:49:00.192 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.192 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.192 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.192 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.192 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.192 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.192 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.192 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T08:49:00.193 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.193 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.193 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.193 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.193 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.193 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.193 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crash", 2024-09-10T08:49:00.193 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.193 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.193 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.193 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.193 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.194 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.194 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.194 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.194 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.194 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.194 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.194 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.194 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.194 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.194 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.194 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.194 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.194 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.195 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.195 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.195 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.195 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.195 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.195 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.195 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.195 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.195 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.195 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.195 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.195 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.195 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.195 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.196 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.196 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.196 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.196 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.196 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.196 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.196 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.196 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.196 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.196 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.196 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.196 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.197 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.197 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.197 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.197 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.197 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.197 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.197 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.197 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.197 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.197 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.197 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.197 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.198 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.198 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.198 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.198 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.198 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.198 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.198 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.198 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.198 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.198 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.198 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.198 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.198 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.199 INFO:teuthology.orchestra.run.smithi089.stdout: "retain_interval": { 2024-09-10T08:49:00.199 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "retain_interval", 2024-09-10T08:49:00.199 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.199 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.199 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.199 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "31536000", 2024-09-10T08:49:00.199 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.199 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.199 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.199 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T08:49:00.199 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.199 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.199 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.200 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_recent_interval": { 2024-09-10T08:49:00.200 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_recent_interval", 2024-09-10T08:49:00.200 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.200 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.200 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.200 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1209600", 2024-09-10T08:49:00.200 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.200 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.200 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.200 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T08:49:00.200 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.200 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.200 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "dashboard", 2024-09-10T08:49:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T08:49:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T08:49:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.201 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.202 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:00.202 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.202 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.202 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.202 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.202 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.202 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.202 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.202 INFO:teuthology.orchestra.run.smithi089.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T08:49:00.202 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T08:49:00.202 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.202 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.202 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.203 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.203 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.203 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.203 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.203 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.203 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.203 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.203 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.203 INFO:teuthology.orchestra.run.smithi089.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T08:49:00.203 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T08:49:00.203 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.203 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.203 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.204 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.204 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.204 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.204 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.204 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.204 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.204 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.204 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.204 INFO:teuthology.orchestra.run.smithi089.stdout: "AUDIT_API_ENABLED": { 2024-09-10T08:49:00.204 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T08:49:00.204 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.204 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.205 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.205 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.205 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.205 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.205 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.205 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.205 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.205 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.205 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.205 INFO:teuthology.orchestra.run.smithi089.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T08:49:00.205 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T08:49:00.205 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.205 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.206 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.206 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.206 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.206 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.206 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.206 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.206 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.206 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.206 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.206 INFO:teuthology.orchestra.run.smithi089.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T08:49:00.206 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T08:49:00.206 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.206 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.207 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.207 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.207 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.207 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.207 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.207 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.207 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.207 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.207 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.207 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T08:49:00.207 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T08:49:00.208 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.208 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.208 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.208 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.208 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.208 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.208 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.208 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.208 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.208 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.208 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.208 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.208 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T08:49:00.208 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T08:49:00.209 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.209 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.209 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.209 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.209 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.209 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.209 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.209 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.209 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.209 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.209 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.209 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.209 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T08:49:00.209 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T08:49:00.210 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.210 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.210 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.210 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.210 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.210 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.210 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.210 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.210 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.210 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.210 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.210 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T08:49:00.211 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T08:49:00.211 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.211 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.211 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.211 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.211 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.211 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.211 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.211 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.211 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.211 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.211 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.211 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T08:49:00.211 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T08:49:00.212 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.212 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.212 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.212 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.212 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.212 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.212 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.212 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.212 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.212 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.212 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.212 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.212 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T08:49:00.213 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T08:49:00.213 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.213 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.213 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.213 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.213 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.213 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.213 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.213 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.213 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.213 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.213 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.214 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T08:49:00.214 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T08:49:00.214 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.214 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.214 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.214 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.214 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.214 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.214 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.214 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.214 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.214 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.214 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.215 INFO:teuthology.orchestra.run.smithi089.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T08:49:00.215 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T08:49:00.215 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.215 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.215 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.215 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.215 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.215 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.215 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.215 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.215 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.215 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.215 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.216 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T08:49:00.216 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T08:49:00.216 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.216 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.216 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.216 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2024-09-10T08:49:00.216 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.216 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.216 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.216 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.216 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.216 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.216 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.217 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T08:49:00.217 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T08:49:00.217 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.217 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.217 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.217 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.217 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.217 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.217 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.217 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.217 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.217 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.217 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.218 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_URL": { 2024-09-10T08:49:00.218 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_URL", 2024-09-10T08:49:00.218 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.218 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.218 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.218 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.218 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.218 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.218 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.218 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.218 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.218 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.218 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.219 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T08:49:00.219 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T08:49:00.219 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.219 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.219 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.219 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2024-09-10T08:49:00.219 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.219 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.219 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.219 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.219 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.219 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.219 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.220 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T08:49:00.220 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T08:49:00.220 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.220 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.220 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.220 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.220 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.220 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.220 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.220 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.220 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.220 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.220 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.221 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T08:49:00.221 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T08:49:00.221 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.221 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.221 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.221 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.221 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.221 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.221 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.221 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.221 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.221 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.222 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.222 INFO:teuthology.orchestra.run.smithi089.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T08:49:00.222 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T08:49:00.222 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.222 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.222 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.222 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.222 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.222 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.222 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.222 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.222 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.223 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.223 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.223 INFO:teuthology.orchestra.run.smithi089.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T08:49:00.223 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T08:49:00.223 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.223 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.223 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.223 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.223 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.223 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.223 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.223 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.224 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.224 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.224 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.224 INFO:teuthology.orchestra.run.smithi089.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T08:49:00.224 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T08:49:00.224 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.224 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.224 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.224 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.224 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.224 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.224 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.224 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.224 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.225 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.225 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.225 INFO:teuthology.orchestra.run.smithi089.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T08:49:00.225 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T08:49:00.225 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.225 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.225 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.225 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.225 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.225 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.225 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.225 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.225 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.226 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.226 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.226 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T08:49:00.226 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T08:49:00.226 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.226 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.226 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.226 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.226 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.226 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.226 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.226 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.226 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.227 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.227 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.227 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T08:49:00.227 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T08:49:00.227 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.227 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.227 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.227 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.227 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.227 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.227 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.227 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.228 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.228 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.228 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.228 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T08:49:00.228 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T08:49:00.228 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.228 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.228 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.228 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.228 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.228 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.229 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.229 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.229 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.229 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.229 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.229 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T08:49:00.229 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T08:49:00.229 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.229 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.229 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.229 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.229 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.230 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.230 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.230 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.230 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.230 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.230 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.230 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T08:49:00.230 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T08:49:00.230 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.230 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.230 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.230 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.230 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.231 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.231 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.231 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.231 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.231 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.231 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.231 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T08:49:00.231 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T08:49:00.231 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.231 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.231 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.231 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.231 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.231 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.232 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.232 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.232 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.232 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.232 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.232 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T08:49:00.232 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T08:49:00.232 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.232 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.232 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.232 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.232 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.232 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.233 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.233 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.233 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.233 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.233 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.233 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T08:49:00.233 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T08:49:00.233 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.233 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.233 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.233 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.233 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.234 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.234 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.234 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.234 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.234 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.234 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.234 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T08:49:00.234 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T08:49:00.234 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.234 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.234 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.234 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T08:49:00.234 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.235 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.235 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.235 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.235 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.235 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.235 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.235 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T08:49:00.235 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T08:49:00.235 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.235 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.235 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.235 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:00.236 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.236 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.236 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.236 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.236 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.236 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.236 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.236 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T08:49:00.236 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T08:49:00.236 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.236 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.236 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.237 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8", 2024-09-10T08:49:00.237 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.237 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.237 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.237 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.237 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.237 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.237 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.237 INFO:teuthology.orchestra.run.smithi089.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T08:49:00.237 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T08:49:00.237 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.237 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.237 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.238 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "45", 2024-09-10T08:49:00.238 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.238 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.238 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.238 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.238 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.238 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.238 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.238 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T08:49:00.238 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T08:49:00.238 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.238 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.238 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.239 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.239 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.239 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.239 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.239 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.239 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.239 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.239 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.239 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T08:49:00.239 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T08:49:00.239 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.239 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.239 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.240 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2024-09-10T08:49:00.240 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.240 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.240 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.240 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.240 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.240 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.240 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.240 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T08:49:00.240 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T08:49:00.240 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.240 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.240 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.241 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.241 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.241 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.241 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.241 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.241 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.241 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.241 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.241 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T08:49:00.241 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T08:49:00.241 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.241 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.241 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.242 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.242 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.242 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.242 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.242 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.242 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.242 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.242 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.242 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T08:49:00.242 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T08:49:00.242 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.242 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.242 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.242 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-10T08:49:00.243 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.243 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.243 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.243 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.243 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.243 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.243 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.243 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.243 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T08:49:00.243 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T08:49:00.243 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.243 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.244 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.244 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:00.244 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.244 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.244 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.244 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.244 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.244 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.244 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.244 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.245 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T08:49:00.245 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T08:49:00.245 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.245 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.245 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.245 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-10T08:49:00.245 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.245 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.245 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.245 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.245 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.245 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.245 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.245 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.245 INFO:teuthology.orchestra.run.smithi089.stdout: "cross_origin_url": { 2024-09-10T08:49:00.246 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cross_origin_url", 2024-09-10T08:49:00.246 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.246 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.246 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.246 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.246 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.246 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.246 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.246 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.247 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.247 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.247 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.247 INFO:teuthology.orchestra.run.smithi089.stdout: "crt_file": { 2024-09-10T08:49:00.247 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crt_file", 2024-09-10T08:49:00.247 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.247 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.247 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.247 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.247 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.247 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.247 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.248 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.248 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.248 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.248 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.248 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.248 INFO:teuthology.orchestra.run.smithi089.stdout: "debug": { 2024-09-10T08:49:00.248 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "debug", 2024-09-10T08:49:00.248 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.248 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.248 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.249 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.249 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.249 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.250 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.250 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable/disable debug options", 2024-09-10T08:49:00.250 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.250 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.250 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.250 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.250 INFO:teuthology.orchestra.run.smithi089.stdout: "jwt_token_ttl": { 2024-09-10T08:49:00.250 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "jwt_token_ttl", 2024-09-10T08:49:00.250 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.250 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.250 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.250 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "28800", 2024-09-10T08:49:00.250 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.250 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.251 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.251 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.251 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.251 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.251 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.251 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.251 INFO:teuthology.orchestra.run.smithi089.stdout: "key_file": { 2024-09-10T08:49:00.251 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "key_file", 2024-09-10T08:49:00.251 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.251 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.251 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.251 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.251 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.251 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.252 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.252 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.252 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.252 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.252 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.252 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.252 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.252 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.252 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.252 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.252 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.253 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.253 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.253 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.253 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.253 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.253 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.253 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.253 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.253 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.253 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.253 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.253 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.253 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.253 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.254 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.254 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.254 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.254 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.254 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.254 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.254 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.254 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.254 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.254 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.255 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.255 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.255 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.255 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.255 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.255 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.255 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.255 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.255 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.255 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.255 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.256 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.256 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.256 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.256 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.256 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.256 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.256 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.256 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.256 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.256 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.257 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.257 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.257 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.257 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.257 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.257 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.257 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.257 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.257 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.257 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.257 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.257 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.257 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.258 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.258 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.258 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.258 INFO:teuthology.orchestra.run.smithi089.stdout: "motd": { 2024-09-10T08:49:00.258 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "motd", 2024-09-10T08:49:00.258 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.258 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.258 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.258 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.258 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.258 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.258 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.259 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "The message of the day", 2024-09-10T08:49:00.259 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.259 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.259 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.259 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.259 INFO:teuthology.orchestra.run.smithi089.stdout: "redirect_resolve_ip_addr": { 2024-09-10T08:49:00.259 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T08:49:00.259 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.259 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.259 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.259 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.259 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.259 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.259 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.260 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.260 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.260 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.260 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.260 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2024-09-10T08:49:00.260 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2024-09-10T08:49:00.260 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.260 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.260 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.260 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "::", 2024-09-10T08:49:00.260 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.260 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.261 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.261 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.261 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.261 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.261 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.261 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.261 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2024-09-10T08:49:00.261 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2024-09-10T08:49:00.261 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.261 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.261 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.261 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8080", 2024-09-10T08:49:00.261 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.262 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.262 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.262 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.262 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.262 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.262 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.262 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl": { 2024-09-10T08:49:00.262 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl", 2024-09-10T08:49:00.262 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.262 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.262 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.262 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.262 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.262 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.263 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.263 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.263 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.263 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.263 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.263 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl_server_port": { 2024-09-10T08:49:00.263 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl_server_port", 2024-09-10T08:49:00.263 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.263 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.263 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.263 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8443", 2024-09-10T08:49:00.264 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.264 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.264 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.264 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.264 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.264 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.264 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.264 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_behaviour": { 2024-09-10T08:49:00.265 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_behaviour", 2024-09-10T08:49:00.265 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.265 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.265 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.265 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "redirect", 2024-09-10T08:49:00.265 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.265 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.265 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.265 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.265 INFO:teuthology.orchestra.run.smithi089.stdout: "redirect" 2024-09-10T08:49:00.265 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.265 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.266 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.266 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.266 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.266 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_error_status_code": { 2024-09-10T08:49:00.266 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_error_status_code", 2024-09-10T08:49:00.266 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.266 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.266 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.266 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2024-09-10T08:49:00.266 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "400", 2024-09-10T08:49:00.266 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "599", 2024-09-10T08:49:00.266 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.267 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.267 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.267 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.267 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.267 INFO:teuthology.orchestra.run.smithi089.stdout: "url_prefix": { 2024-09-10T08:49:00.267 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "url_prefix", 2024-09-10T08:49:00.267 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.267 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.267 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.267 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.267 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.268 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.268 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.268 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.268 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.268 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.268 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.268 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.268 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.268 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.268 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "devicehealth", 2024-09-10T08:49:00.268 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.268 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.268 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.269 INFO:teuthology.orchestra.run.smithi089.stdout: "enable_monitoring": { 2024-09-10T08:49:00.269 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enable_monitoring", 2024-09-10T08:49:00.269 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.269 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.269 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.269 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.269 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.269 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.269 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.269 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "monitor device health metrics", 2024-09-10T08:49:00.269 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.269 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.269 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.270 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.270 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.270 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.270 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.270 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.270 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.270 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.270 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.270 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.270 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.270 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.271 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.271 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.271 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.271 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.271 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.271 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.271 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.271 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.271 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.271 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.271 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.271 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.272 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.272 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.272 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.272 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.272 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.272 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.272 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.272 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.272 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.272 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.272 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.272 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.272 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.273 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.274 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "mark_out_threshold": { 2024-09-10T08:49:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mark_out_threshold", 2024-09-10T08:49:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "2419200", 2024-09-10T08:49:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.275 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T08:49:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_name": { 2024-09-10T08:49:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pool_name", 2024-09-10T08:49:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "device_health_metrics", 2024-09-10T08:49:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.276 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T08:49:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "retention_period": { 2024-09-10T08:49:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "retention_period", 2024-09-10T08:49:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15552000", 2024-09-10T08:49:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.277 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long to retain device health metrics", 2024-09-10T08:49:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "scrape_frequency": { 2024-09-10T08:49:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "scrape_frequency", 2024-09-10T08:49:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "86400", 2024-09-10T08:49:00.278 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T08:49:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "self_heal": { 2024-09-10T08:49:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "self_heal", 2024-09-10T08:49:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.279 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T08:49:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-10T08:49:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-10T08:49:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-10T08:49:00.280 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T08:49:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_threshold": { 2024-09-10T08:49:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_threshold", 2024-09-10T08:49:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.281 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7257600", 2024-09-10T08:49:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T08:49:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "diskprediction_local", 2024-09-10T08:49:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.282 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.283 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.284 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.285 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.286 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.287 INFO:teuthology.orchestra.run.smithi089.stdout: "predict_interval": { 2024-09-10T08:49:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "predict_interval", 2024-09-10T08:49:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "86400", 2024-09-10T08:49:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.288 INFO:teuthology.orchestra.run.smithi089.stdout: "predictor_model": { 2024-09-10T08:49:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "predictor_model", 2024-09-10T08:49:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "prophetstor", 2024-09-10T08:49:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.289 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-10T08:49:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-10T08:49:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-10T08:49:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.290 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "influx", 2024-09-10T08:49:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": false, 2024-09-10T08:49:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "influxdb python module not found", 2024-09-10T08:49:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "batch_size": { 2024-09-10T08:49:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "batch_size", 2024-09-10T08:49:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5000", 2024-09-10T08:49:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.291 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T08:49:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "database": { 2024-09-10T08:49:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "database", 2024-09-10T08:49:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "ceph", 2024-09-10T08:49:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.292 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T08:49:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "hostname": { 2024-09-10T08:49:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "hostname", 2024-09-10T08:49:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.293 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB server hostname", 2024-09-10T08:49:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-10T08:49:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-10T08:49:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "30", 2024-09-10T08:49:00.294 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "5", 2024-09-10T08:49:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T08:49:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.295 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.296 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.296 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.296 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.296 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.296 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.296 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.296 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.296 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.296 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.296 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.296 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.296 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.296 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.296 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.297 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.297 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.297 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.297 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.297 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.297 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.297 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.297 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.297 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.297 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.297 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.297 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.297 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.297 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.298 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.298 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.298 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.298 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.298 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.298 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.298 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.298 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.298 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.298 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.298 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.298 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.298 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.299 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.299 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.299 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.299 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.299 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.299 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.299 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.299 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.299 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.299 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.299 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.299 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.299 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.299 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.300 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.300 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.300 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.300 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.300 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.300 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.300 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.300 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.300 INFO:teuthology.orchestra.run.smithi089.stdout: "password": { 2024-09-10T08:49:00.300 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "password", 2024-09-10T08:49:00.300 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.300 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.300 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.300 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.301 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.301 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.301 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.301 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "password of InfluxDB server user", 2024-09-10T08:49:00.301 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.301 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.301 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.301 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.301 INFO:teuthology.orchestra.run.smithi089.stdout: "port": { 2024-09-10T08:49:00.301 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "port", 2024-09-10T08:49:00.301 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.301 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.301 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.301 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8086", 2024-09-10T08:49:00.302 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.302 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.302 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.302 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB server port", 2024-09-10T08:49:00.302 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.302 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.302 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.302 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.302 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl": { 2024-09-10T08:49:00.302 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl", 2024-09-10T08:49:00.302 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.302 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.302 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.302 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "false", 2024-09-10T08:49:00.302 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.303 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.303 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.303 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T08:49:00.303 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.303 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.303 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.303 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.303 INFO:teuthology.orchestra.run.smithi089.stdout: "threads": { 2024-09-10T08:49:00.303 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "threads", 2024-09-10T08:49:00.303 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.303 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.303 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.303 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-10T08:49:00.303 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-10T08:49:00.304 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "32", 2024-09-10T08:49:00.304 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.304 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T08:49:00.304 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.304 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.304 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.304 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.304 INFO:teuthology.orchestra.run.smithi089.stdout: "username": { 2024-09-10T08:49:00.304 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "username", 2024-09-10T08:49:00.304 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.304 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.304 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.304 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.305 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.305 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.305 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.305 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "username of InfluxDB server user", 2024-09-10T08:49:00.305 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.305 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.305 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.305 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.305 INFO:teuthology.orchestra.run.smithi089.stdout: "verify_ssl": { 2024-09-10T08:49:00.305 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "verify_ssl", 2024-09-10T08:49:00.305 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.305 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.305 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.306 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "true", 2024-09-10T08:49:00.306 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.306 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.306 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.306 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T08:49:00.306 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.306 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.306 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.306 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.306 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.306 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.306 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.306 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "insights", 2024-09-10T08:49:00.307 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.307 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.307 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.307 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.307 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.307 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.307 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.307 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.307 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.307 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.307 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.307 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.307 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.308 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.308 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.308 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.308 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.308 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.308 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.308 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.308 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.308 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.308 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.308 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.308 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.308 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.309 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.309 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.309 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.309 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.309 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.309 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.309 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.309 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.309 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.309 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.309 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.309 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.309 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.310 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.310 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.310 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.310 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.310 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.310 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.310 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.310 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.310 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.310 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.310 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.310 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.310 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.310 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.311 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.311 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.311 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.311 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.311 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.311 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.311 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.311 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.311 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.311 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.311 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.311 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.311 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.312 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.312 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.312 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.312 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.312 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.312 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.312 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.312 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.312 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.312 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.312 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "iostat", 2024-09-10T08:49:00.312 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.312 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.313 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.313 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.313 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.313 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.313 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.313 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.313 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.313 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.313 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.313 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.313 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.313 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.313 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.314 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.314 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.314 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.314 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.314 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.314 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.314 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.314 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.314 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.314 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.314 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.314 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.314 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.314 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.315 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.315 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.315 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.315 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.315 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.315 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.315 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.315 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.315 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.315 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.315 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.315 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.315 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.315 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.316 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.316 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.316 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.316 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.316 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.316 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.316 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.316 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.316 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.316 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.316 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.316 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.316 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.316 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.317 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.317 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.317 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.317 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.317 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.317 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.317 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.317 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.317 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.317 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.317 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.317 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.317 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.317 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.318 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.318 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.318 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.318 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.318 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.318 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "k8sevents", 2024-09-10T08:49:00.318 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.318 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.318 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.318 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_event_retention_days": { 2024-09-10T08:49:00.318 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ceph_event_retention_days", 2024-09-10T08:49:00.318 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.318 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.318 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.319 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7", 2024-09-10T08:49:00.319 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.319 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.319 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.319 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T08:49:00.319 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.319 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.319 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.319 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.319 INFO:teuthology.orchestra.run.smithi089.stdout: "config_check_secs": { 2024-09-10T08:49:00.319 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_check_secs", 2024-09-10T08:49:00.319 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.319 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.319 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.320 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:00.320 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "10", 2024-09-10T08:49:00.320 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.320 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.320 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T08:49:00.320 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.320 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.320 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.320 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.320 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.320 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.320 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.320 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.321 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.321 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.321 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.321 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.321 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.321 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.321 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.321 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.321 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.321 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.321 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.321 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.321 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.321 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.322 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.322 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.322 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.322 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.322 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.322 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.322 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.322 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.322 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.322 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.322 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.322 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.322 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.322 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.323 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.323 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.323 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.323 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.323 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.323 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.323 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.323 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.323 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.323 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.323 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.323 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.323 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.324 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.324 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.324 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.324 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.324 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.324 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.324 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.324 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.324 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.324 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.324 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.324 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.324 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.324 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.325 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.325 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.325 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.325 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.325 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.325 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.325 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.325 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.325 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.325 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.325 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.325 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.325 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.325 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "localpool", 2024-09-10T08:49:00.326 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.326 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.326 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.326 INFO:teuthology.orchestra.run.smithi089.stdout: "failure_domain": { 2024-09-10T08:49:00.326 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "failure_domain", 2024-09-10T08:49:00.326 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.326 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.326 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.326 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "host", 2024-09-10T08:49:00.326 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.326 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.326 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.326 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "failure domain for any created local pool", 2024-09-10T08:49:00.326 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T08:49:00.326 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.327 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.327 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.327 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.327 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.327 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.327 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.327 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.327 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.327 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.327 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.327 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.327 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.327 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.327 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.327 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.328 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.328 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.328 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.328 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.328 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.328 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.328 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.328 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.328 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.328 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.328 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.328 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.328 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.328 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.329 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.329 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.329 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.329 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.329 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.329 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.329 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.329 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.329 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.329 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.329 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.329 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.329 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.329 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.330 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.330 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.330 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.330 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.330 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.330 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.330 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.330 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.330 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.330 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.330 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.330 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.330 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.331 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.331 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.331 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.331 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.331 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.331 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.331 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.331 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.331 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.331 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.331 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.331 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.331 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.332 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.332 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.332 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.332 INFO:teuthology.orchestra.run.smithi089.stdout: "min_size": { 2024-09-10T08:49:00.332 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "min_size", 2024-09-10T08:49:00.332 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.332 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.332 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.332 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.332 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.332 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.332 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.332 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default min_size for any created local pool", 2024-09-10T08:49:00.333 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T08:49:00.333 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.333 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.333 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.333 INFO:teuthology.orchestra.run.smithi089.stdout: "num_rep": { 2024-09-10T08:49:00.333 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "num_rep", 2024-09-10T08:49:00.333 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.333 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.333 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.333 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3", 2024-09-10T08:49:00.333 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.334 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.334 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.334 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default replica count for any created local pool", 2024-09-10T08:49:00.334 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.334 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.334 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.334 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.334 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num": { 2024-09-10T08:49:00.334 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pg_num", 2024-09-10T08:49:00.334 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.334 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.334 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.335 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "128", 2024-09-10T08:49:00.335 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.335 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.335 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.335 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T08:49:00.335 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.335 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.335 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.335 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.335 INFO:teuthology.orchestra.run.smithi089.stdout: "prefix": { 2024-09-10T08:49:00.335 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prefix", 2024-09-10T08:49:00.335 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.335 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.336 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.336 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.336 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.336 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.336 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.336 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "name prefix for any created local pool", 2024-09-10T08:49:00.336 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.336 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.336 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.336 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.336 INFO:teuthology.orchestra.run.smithi089.stdout: "subtree": { 2024-09-10T08:49:00.336 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "subtree", 2024-09-10T08:49:00.336 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.337 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.337 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.337 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "rack", 2024-09-10T08:49:00.337 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.337 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.337 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.337 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T08:49:00.337 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T08:49:00.337 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.337 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.337 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.337 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.337 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.338 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.338 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mds_autoscaler", 2024-09-10T08:49:00.338 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.338 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.338 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.338 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.338 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.338 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.338 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.338 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.338 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.338 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.339 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.339 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.339 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.339 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.339 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.339 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.339 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.339 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.339 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.339 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.339 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.339 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.339 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.339 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.339 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.340 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.340 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.340 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.340 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.340 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.340 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.340 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.340 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.340 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.340 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.340 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.340 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.340 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.341 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.341 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.341 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.341 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.341 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.341 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.341 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.341 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.341 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.341 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.341 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.341 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.341 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.342 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.342 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.342 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.342 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.342 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.342 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.342 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.342 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.342 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.342 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.342 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.342 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.342 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.342 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.343 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.343 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.343 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.343 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.343 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.343 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.343 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.343 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.343 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.343 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.343 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.343 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mirroring", 2024-09-10T08:49:00.343 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.343 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.344 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.344 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.344 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.344 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.344 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.344 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.344 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.344 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.344 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.344 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.344 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.344 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.344 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.344 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.344 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.345 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.345 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.345 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.345 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.345 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.345 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.345 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.345 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.345 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.345 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.345 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.345 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.345 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.345 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.346 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.346 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.346 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.346 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.346 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.346 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.346 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.346 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.346 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.346 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.346 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.346 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.346 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.346 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.347 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.347 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.347 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.347 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.347 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.347 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.347 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.347 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.347 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.347 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.347 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.347 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.347 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.347 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.347 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.348 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.348 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.348 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.348 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.348 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.348 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.348 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.348 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.348 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.348 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.348 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.349 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.349 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.349 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.349 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.349 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.349 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "nfs", 2024-09-10T08:49:00.349 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.349 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.349 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.349 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.349 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.350 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.350 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.350 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.350 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.350 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.350 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.350 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.350 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.350 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.350 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.350 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.350 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.350 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.350 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.351 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.351 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.351 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.351 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.351 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.351 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.351 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.351 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.351 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.351 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.351 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.351 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.351 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.352 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.352 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.352 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.352 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.352 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.352 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.352 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.352 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.352 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.352 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.352 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.352 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.352 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.352 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.353 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.353 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.353 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.353 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.353 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.353 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.353 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.353 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.353 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.353 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.353 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.353 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.353 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.353 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.354 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.354 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.354 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.354 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.354 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.354 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.354 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.354 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.354 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.354 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.354 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.354 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.354 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.354 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.355 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.355 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.355 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "orchestrator", 2024-09-10T08:49:00.355 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.355 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.355 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.355 INFO:teuthology.orchestra.run.smithi089.stdout: "fail_fs": { 2024-09-10T08:49:00.355 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "fail_fs", 2024-09-10T08:49:00.355 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.355 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.355 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.355 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.355 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.355 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.356 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.356 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T08:49:00.356 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.356 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.356 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.356 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.356 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.356 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.356 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.356 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.356 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.356 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.356 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.356 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.357 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.357 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.357 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.357 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.357 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.357 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.357 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.357 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.357 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.357 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.357 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.357 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.357 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.358 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.358 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.358 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.358 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.358 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.358 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.358 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.358 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.358 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.358 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.358 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.358 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.358 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.358 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.359 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.359 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.359 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.359 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.359 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.359 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.359 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.359 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.359 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.359 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.359 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.359 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.359 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.359 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.359 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.360 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.360 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.360 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.360 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.360 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.360 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.360 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.360 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.360 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.360 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.360 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.360 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.360 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.360 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.361 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.361 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.361 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.361 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.361 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.361 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.361 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator": { 2024-09-10T08:49:00.361 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "orchestrator", 2024-09-10T08:49:00.361 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.361 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.361 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.361 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.362 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.362 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.362 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.362 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm", 2024-09-10T08:49:00.362 INFO:teuthology.orchestra.run.smithi089.stdout: "rook", 2024-09-10T08:49:00.362 INFO:teuthology.orchestra.run.smithi089.stdout: "test_orchestrator" 2024-09-10T08:49:00.362 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.362 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Orchestrator backend", 2024-09-10T08:49:00.362 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.362 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.362 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.362 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.362 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.363 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.363 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.363 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "osd_perf_query", 2024-09-10T08:49:00.363 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.363 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.363 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.363 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.363 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.363 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.363 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.363 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.363 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.363 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.364 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.364 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.364 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.364 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.364 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.364 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.364 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.364 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.364 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.364 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.364 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.364 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.364 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.364 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.365 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.365 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.365 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.365 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.365 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.365 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.365 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.365 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.365 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.365 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.365 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.365 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.365 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.365 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.366 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.366 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.366 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.366 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.366 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.366 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.366 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.366 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.366 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.366 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.366 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.366 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.366 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.366 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.367 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.367 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.367 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.367 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.367 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.367 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.367 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.367 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.367 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.367 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.367 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.367 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.367 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.368 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.368 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.368 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.368 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.368 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.368 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.368 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.368 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.368 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.368 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.368 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.368 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "osd_support", 2024-09-10T08:49:00.368 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.368 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.369 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.369 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.369 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.369 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.369 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.369 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.369 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.369 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.369 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.369 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.369 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.369 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.369 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.369 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.370 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.370 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.370 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.370 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.370 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.370 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.370 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.370 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.370 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.370 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.370 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.370 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.370 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.370 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.371 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.371 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.371 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.371 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.371 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.371 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.371 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.371 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.371 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.371 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.371 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.371 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.371 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.371 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.372 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.372 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.372 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.372 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.372 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.372 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.372 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.372 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.372 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.372 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.372 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.372 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.372 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.372 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.373 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.373 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.373 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.373 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.373 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.373 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.373 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.373 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.373 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.373 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.373 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.373 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.373 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.374 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.374 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.374 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.374 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pg_autoscaler", 2024-09-10T08:49:00.374 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.374 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.374 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.374 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.374 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.374 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.374 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.374 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.375 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.375 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.375 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.375 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.375 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.375 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.376 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.376 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.376 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.377 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.377 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.377 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.377 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.377 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.377 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.377 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.377 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.377 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.377 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.377 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.377 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.377 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.377 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.378 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.378 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.378 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.378 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.378 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.378 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.378 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.378 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.378 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.378 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.378 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.378 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.379 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.379 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.379 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.379 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.379 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.379 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.379 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.379 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.379 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.379 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.379 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.379 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.379 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.379 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.380 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.380 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.380 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.380 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.380 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.380 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.380 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.380 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.380 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.380 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.380 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.380 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.380 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.381 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-10T08:49:00.381 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-10T08:49:00.381 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.381 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.381 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.381 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-10T08:49:00.381 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.381 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.381 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.381 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.381 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.381 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.381 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.382 INFO:teuthology.orchestra.run.smithi089.stdout: "threshold": { 2024-09-10T08:49:00.382 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "threshold", 2024-09-10T08:49:00.382 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:00.382 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.382 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.382 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3.0", 2024-09-10T08:49:00.382 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1.0", 2024-09-10T08:49:00.382 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.382 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.382 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "scaling threshold", 2024-09-10T08:49:00.382 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T08:49:00.383 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.383 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.383 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.383 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.383 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.383 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.383 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "progress", 2024-09-10T08:49:00.383 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.383 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.383 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.383 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_pg_recovery_event": { 2024-09-10T08:49:00.383 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_pg_recovery_event", 2024-09-10T08:49:00.383 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.384 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.384 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.384 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.384 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.384 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.384 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.384 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T08:49:00.384 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.384 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.384 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.384 INFO:teuthology.orchestra.run.smithi089.stdout: "enabled": { 2024-09-10T08:49:00.384 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enabled", 2024-09-10T08:49:00.385 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.385 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.385 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.385 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.385 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.385 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.385 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.385 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.385 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.385 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.385 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.386 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.386 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.386 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.386 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.386 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.386 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.386 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.386 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.386 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.386 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.386 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.386 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.386 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.386 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.387 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.387 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.387 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.387 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.387 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.387 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.387 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.387 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.387 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.387 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.387 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.387 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.387 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.387 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.387 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.388 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.388 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.388 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.388 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.388 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.388 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.388 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.388 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.388 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.388 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.388 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.388 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.388 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.389 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.389 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.389 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.389 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.389 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.389 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.389 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.389 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.389 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.389 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.389 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.389 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.389 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.390 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.390 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.390 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.390 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.390 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.390 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.390 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.390 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.390 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.390 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.390 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.390 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.390 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.390 INFO:teuthology.orchestra.run.smithi089.stdout: "max_completed_events": { 2024-09-10T08:49:00.391 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_completed_events", 2024-09-10T08:49:00.391 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.391 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.391 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.391 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "50", 2024-09-10T08:49:00.391 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.391 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.391 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.391 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "number of past completed events to remember", 2024-09-10T08:49:00.391 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.391 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.391 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.391 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-10T08:49:00.392 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-10T08:49:00.392 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.392 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.392 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.392 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-10T08:49:00.392 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.392 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.392 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.392 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long the module is going to sleep", 2024-09-10T08:49:00.392 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.392 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.392 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.392 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.393 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.393 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.393 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prometheus", 2024-09-10T08:49:00.393 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.393 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.393 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.393 INFO:teuthology.orchestra.run.smithi089.stdout: "cache": { 2024-09-10T08:49:00.393 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cache", 2024-09-10T08:49:00.393 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.393 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.393 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.393 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.393 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.394 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.394 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.394 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.394 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.394 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.394 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.394 INFO:teuthology.orchestra.run.smithi089.stdout: "exclude_perf_counters": { 2024-09-10T08:49:00.394 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "exclude_perf_counters", 2024-09-10T08:49:00.394 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.394 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.394 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.394 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.394 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.395 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.395 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.395 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T08:49:00.395 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T08:49:00.395 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.395 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.395 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.395 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.395 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.395 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.395 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.395 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.395 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.396 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.396 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.396 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.396 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.396 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.396 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.396 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.396 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.396 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.396 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.396 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.396 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.396 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.397 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.397 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.397 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.397 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.397 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.397 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.397 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.397 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.397 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.397 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.397 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.397 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.398 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.398 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.398 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.398 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.398 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.398 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.398 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.398 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.398 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.398 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.398 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.398 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.399 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.399 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.399 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.399 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.399 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.399 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.399 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.399 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.399 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.399 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.399 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.399 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.399 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.399 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.400 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.400 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.400 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.400 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.400 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.400 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.400 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.400 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.400 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.400 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.400 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_stats_pools": { 2024-09-10T08:49:00.400 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_stats_pools", 2024-09-10T08:49:00.400 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.400 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.401 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.401 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.401 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.401 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.401 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.401 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.401 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.401 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.401 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.401 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T08:49:00.401 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T08:49:00.401 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.401 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.402 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.402 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "300", 2024-09-10T08:49:00.402 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.402 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.402 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.402 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.402 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.402 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.402 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.402 INFO:teuthology.orchestra.run.smithi089.stdout: "scrape_interval": { 2024-09-10T08:49:00.402 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "scrape_interval", 2024-09-10T08:49:00.402 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:00.402 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.403 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.403 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15.0", 2024-09-10T08:49:00.403 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.403 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.403 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.403 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.403 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.403 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.403 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.403 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2024-09-10T08:49:00.403 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2024-09-10T08:49:00.403 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.403 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.403 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.404 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "::", 2024-09-10T08:49:00.404 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.404 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.404 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.404 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T08:49:00.404 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.404 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.404 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.404 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2024-09-10T08:49:00.404 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2024-09-10T08:49:00.404 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.404 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.404 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.405 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "9283", 2024-09-10T08:49:00.405 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.405 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.405 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.405 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T08:49:00.405 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.405 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.405 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.405 INFO:teuthology.orchestra.run.smithi089.stdout: "stale_cache_strategy": { 2024-09-10T08:49:00.405 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "stale_cache_strategy", 2024-09-10T08:49:00.405 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.405 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.406 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.406 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "log", 2024-09-10T08:49:00.406 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.406 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.406 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.406 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.406 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.406 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.406 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.406 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_behaviour": { 2024-09-10T08:49:00.406 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_behaviour", 2024-09-10T08:49:00.406 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.407 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.407 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.407 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "default", 2024-09-10T08:49:00.407 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.407 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.407 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.407 INFO:teuthology.orchestra.run.smithi089.stdout: "default", 2024-09-10T08:49:00.407 INFO:teuthology.orchestra.run.smithi089.stdout: "error" 2024-09-10T08:49:00.407 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.407 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.407 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.407 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.407 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.408 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_error_status_code": { 2024-09-10T08:49:00.408 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_error_status_code", 2024-09-10T08:49:00.408 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.408 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.408 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.408 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2024-09-10T08:49:00.408 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "400", 2024-09-10T08:49:00.408 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "599", 2024-09-10T08:49:00.408 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.408 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.409 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.409 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.409 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.409 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.409 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.409 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.409 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_support", 2024-09-10T08:49:00.409 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.409 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.409 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.409 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.409 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.409 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.410 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.410 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.410 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.410 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.410 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.410 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.410 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.410 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.410 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.410 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.410 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.410 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.410 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.410 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.411 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.411 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.411 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.411 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.411 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.411 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.411 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.411 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.411 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.411 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.411 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.411 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.411 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.411 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.412 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.412 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.412 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.412 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.412 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.412 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.412 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.412 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.412 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.412 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.412 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.412 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.412 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.413 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.413 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.413 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.413 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.413 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.413 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.413 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.413 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.413 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.413 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.413 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.413 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.413 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.414 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.414 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.414 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.414 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.414 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.414 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.414 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.414 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.414 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.414 INFO:teuthology.orchestra.run.smithi089.stdout: "max_concurrent_snap_create": { 2024-09-10T08:49:00.414 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_concurrent_snap_create", 2024-09-10T08:49:00.414 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.414 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.415 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.415 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:00.415 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.415 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.415 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.415 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.415 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.415 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.415 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.415 INFO:teuthology.orchestra.run.smithi089.stdout: "mirror_snapshot_schedule": { 2024-09-10T08:49:00.415 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T08:49:00.415 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.415 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.416 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.416 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.416 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.416 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.416 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.416 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.416 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.416 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.416 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.416 INFO:teuthology.orchestra.run.smithi089.stdout: "trash_purge_schedule": { 2024-09-10T08:49:00.416 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "trash_purge_schedule", 2024-09-10T08:49:00.416 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.416 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.417 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.417 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.417 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.417 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.417 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.417 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.417 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.417 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.417 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.417 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.417 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.417 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.417 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "restful", 2024-09-10T08:49:00.418 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.418 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.418 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.418 INFO:teuthology.orchestra.run.smithi089.stdout: "enable_auth": { 2024-09-10T08:49:00.418 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enable_auth", 2024-09-10T08:49:00.418 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.418 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.418 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.418 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.418 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.418 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.418 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.418 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.418 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.419 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.419 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.419 INFO:teuthology.orchestra.run.smithi089.stdout: "key_file": { 2024-09-10T08:49:00.419 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "key_file", 2024-09-10T08:49:00.419 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.419 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.419 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.419 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.419 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.419 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.419 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.419 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.419 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.420 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.420 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.420 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.420 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.420 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.420 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.420 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.420 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.421 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.421 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.421 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.421 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.421 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.421 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.421 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.421 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.421 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.421 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.421 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.421 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.421 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.422 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.422 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.422 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.422 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.422 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.422 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.422 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.422 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.422 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.422 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.422 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.422 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.422 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.422 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.423 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.423 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.423 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.423 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.423 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.423 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.423 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.423 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.423 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.423 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.423 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.423 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.423 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.424 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.424 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.424 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.424 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.424 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.424 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.424 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.424 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.424 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.424 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.424 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.424 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.424 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.425 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.425 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.425 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.425 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.425 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.425 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.425 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.425 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2024-09-10T08:49:00.425 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2024-09-10T08:49:00.425 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.425 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.425 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.425 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.426 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.426 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.426 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.426 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.426 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.426 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.426 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.426 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2024-09-10T08:49:00.426 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2024-09-10T08:49:00.426 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.426 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.426 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.426 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.427 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.427 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.427 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.427 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.427 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.427 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.427 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.427 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.427 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.427 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.427 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rgw", 2024-09-10T08:49:00.427 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.428 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.428 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.428 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.428 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.428 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.428 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.428 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.428 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.428 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.428 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.428 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.428 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.428 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.429 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.429 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.429 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.429 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.429 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.429 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.429 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.429 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.429 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.429 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.429 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.429 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.429 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.430 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.430 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.430 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.430 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.430 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.430 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.430 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.430 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.430 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.430 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.430 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.430 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.430 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.431 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.431 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.431 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.431 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.431 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.431 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.431 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.431 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.431 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.431 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.431 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.431 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.431 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.431 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.431 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.432 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.432 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.432 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.432 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.432 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.432 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.432 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.432 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.432 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.432 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.432 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.432 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.432 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.432 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.433 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.433 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.433 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.433 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rook", 2024-09-10T08:49:00.433 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.433 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.433 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.433 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.433 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.433 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.433 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.433 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.433 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.434 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.434 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.434 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.434 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.434 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.434 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.434 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.434 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.434 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.434 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.434 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.434 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.434 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.434 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.435 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.435 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.435 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.435 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.435 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.435 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.435 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.435 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.435 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.435 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.435 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.435 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.435 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.436 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.436 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.436 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.436 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.436 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.436 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.436 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.436 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.436 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.436 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.436 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.436 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.436 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.437 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.437 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.437 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.437 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.437 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.437 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.437 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.437 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.437 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.437 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.437 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.437 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.437 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.438 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.438 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.438 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.438 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.438 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.438 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.438 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.438 INFO:teuthology.orchestra.run.smithi089.stdout: "storage_class": { 2024-09-10T08:49:00.438 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "storage_class", 2024-09-10T08:49:00.438 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.438 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.438 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.438 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "local", 2024-09-10T08:49:00.438 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.439 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.439 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.439 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T08:49:00.439 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.439 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.439 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.439 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.439 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.439 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.439 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "selftest", 2024-09-10T08:49:00.439 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.439 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.440 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.440 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.440 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.440 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.440 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.440 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.440 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.440 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.440 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.440 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.440 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.440 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.440 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.440 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.441 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.441 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.441 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.441 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.441 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.441 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.441 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.441 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.441 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.441 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.441 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.441 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.442 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.442 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.442 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.442 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.442 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.442 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.442 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.442 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.442 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.442 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.442 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.442 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.442 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.443 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.443 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.443 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.443 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.443 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.443 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.443 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.443 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.443 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.443 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.443 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.443 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.443 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.443 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.444 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.444 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.444 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.444 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.444 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.444 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.444 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.444 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.444 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.444 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.444 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.444 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.444 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.444 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.445 INFO:teuthology.orchestra.run.smithi089.stdout: "roption1": { 2024-09-10T08:49:00.445 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "roption1", 2024-09-10T08:49:00.445 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.445 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.445 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.445 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.445 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.445 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.445 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.445 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.445 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.445 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.445 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.446 INFO:teuthology.orchestra.run.smithi089.stdout: "roption2": { 2024-09-10T08:49:00.446 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "roption2", 2024-09-10T08:49:00.446 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.446 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.446 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.446 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "xyz", 2024-09-10T08:49:00.446 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.446 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.446 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.446 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.446 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.446 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.446 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.447 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption1": { 2024-09-10T08:49:00.447 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption1", 2024-09-10T08:49:00.447 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.447 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.447 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.447 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.447 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.447 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.447 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.447 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.447 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.447 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.447 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.448 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption2": { 2024-09-10T08:49:00.448 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption2", 2024-09-10T08:49:00.448 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.448 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.448 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.448 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.448 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.448 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.448 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.448 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.448 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.448 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.448 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.449 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption3": { 2024-09-10T08:49:00.449 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption3", 2024-09-10T08:49:00.449 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:00.449 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.449 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.449 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.449 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.449 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.449 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.449 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.449 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.449 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.449 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.450 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption4": { 2024-09-10T08:49:00.450 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption4", 2024-09-10T08:49:00.450 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.450 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.450 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.450 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.450 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.450 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.450 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.450 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.451 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.451 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.451 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.451 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption5": { 2024-09-10T08:49:00.451 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption5", 2024-09-10T08:49:00.451 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.451 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.451 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.451 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.451 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.451 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.451 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.451 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.452 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.452 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.452 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.452 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption6": { 2024-09-10T08:49:00.452 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption6", 2024-09-10T08:49:00.452 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.452 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.452 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.452 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.452 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.452 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.452 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.452 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.452 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.453 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.453 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.453 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption7": { 2024-09-10T08:49:00.453 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption7", 2024-09-10T08:49:00.453 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.453 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.453 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.453 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.453 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-10T08:49:00.453 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "42", 2024-09-10T08:49:00.453 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.453 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.454 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.454 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.454 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.454 INFO:teuthology.orchestra.run.smithi089.stdout: "testkey": { 2024-09-10T08:49:00.454 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testkey", 2024-09-10T08:49:00.454 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.454 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.454 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.454 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.454 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.454 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.454 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.454 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.455 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.455 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.455 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.455 INFO:teuthology.orchestra.run.smithi089.stdout: "testlkey": { 2024-09-10T08:49:00.455 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testlkey", 2024-09-10T08:49:00.455 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.455 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.455 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.455 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.455 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.455 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.455 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.455 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.456 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.456 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.456 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.456 INFO:teuthology.orchestra.run.smithi089.stdout: "testnewline": { 2024-09-10T08:49:00.456 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testnewline", 2024-09-10T08:49:00.456 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.456 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.456 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.456 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.456 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.456 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.456 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.456 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.457 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.457 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.457 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.457 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.457 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.457 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.457 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snap_schedule", 2024-09-10T08:49:00.457 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.457 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.457 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.457 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_m_granularity": { 2024-09-10T08:49:00.457 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_m_granularity", 2024-09-10T08:49:00.457 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.458 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.458 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.458 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.458 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.458 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.458 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.458 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T08:49:00.458 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.458 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.458 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.458 INFO:teuthology.orchestra.run.smithi089.stdout: "dump_on_update": { 2024-09-10T08:49:00.458 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "dump_on_update", 2024-09-10T08:49:00.458 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.459 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.459 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.459 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.459 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.459 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.459 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.459 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "dump database to debug log on update", 2024-09-10T08:49:00.459 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.459 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.459 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.459 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.459 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.459 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.459 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.460 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.460 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.460 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.460 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.460 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.460 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.460 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.460 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.460 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.460 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.460 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.460 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.460 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.460 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.461 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.461 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.461 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.461 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.461 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.461 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.461 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.461 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.461 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.461 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.461 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.461 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.461 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.461 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.462 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.462 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.462 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.462 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.462 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.462 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.462 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.462 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.462 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.462 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.462 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.462 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.463 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.463 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.463 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.463 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.463 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.463 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.463 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.463 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.463 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.463 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.463 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.463 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.463 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.464 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.464 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.464 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.464 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.464 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.464 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.464 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.464 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.464 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.464 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.464 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.464 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "stats", 2024-09-10T08:49:00.465 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.465 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.465 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.465 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.465 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.465 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.465 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.465 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.465 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.465 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.465 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.465 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.465 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.465 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.465 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.466 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.466 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.466 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.466 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.466 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.466 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.466 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.466 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.466 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.466 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.466 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.466 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.466 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.467 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.467 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.467 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.467 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.467 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.467 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.467 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.467 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.467 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.467 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.467 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.467 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.467 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.468 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.468 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.468 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.468 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.468 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.468 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.468 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.468 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.468 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.468 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.468 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.468 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.468 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.469 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.469 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.469 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.469 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.469 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.469 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.469 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.469 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.469 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.469 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.469 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.469 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.469 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.470 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.470 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.470 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.470 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.470 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.470 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.470 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "status", 2024-09-10T08:49:00.470 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.470 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.470 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.470 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.470 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.470 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.470 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.471 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.471 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.471 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.471 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.471 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.471 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.471 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.471 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.471 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.471 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.471 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.471 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.471 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.471 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.472 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.472 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.472 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.472 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.472 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.472 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.472 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.472 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.472 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.472 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.472 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.472 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.472 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.472 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.473 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.473 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.473 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.473 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.473 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.473 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.473 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.473 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.473 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.473 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.473 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.473 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.473 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.473 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.474 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.474 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.474 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.474 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.474 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.474 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.474 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.474 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.474 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.474 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.474 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.474 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.474 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.475 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.475 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.475 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.475 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.475 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.475 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.475 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.475 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.475 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.475 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.475 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.475 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "telegraf", 2024-09-10T08:49:00.475 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.475 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.476 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.476 INFO:teuthology.orchestra.run.smithi089.stdout: "address": { 2024-09-10T08:49:00.476 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "address", 2024-09-10T08:49:00.476 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.476 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.476 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.476 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T08:49:00.476 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.476 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.476 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.476 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.476 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.476 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.477 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.477 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-10T08:49:00.477 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-10T08:49:00.477 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.477 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.477 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.477 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15", 2024-09-10T08:49:00.477 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.477 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.477 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.477 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.477 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.477 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.478 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.478 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.478 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.478 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.478 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.478 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.478 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.478 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.478 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.478 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.478 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.478 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.478 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.479 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.479 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.479 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.479 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.479 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.479 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.479 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.479 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.479 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.479 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.479 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.479 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.479 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.480 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.480 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.480 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.480 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.480 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.480 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.480 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.480 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.480 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.480 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.480 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.480 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.480 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.481 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.481 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.481 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.481 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.481 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.481 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.481 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.481 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.481 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.481 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.481 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.481 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.481 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.481 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.481 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.482 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.482 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.482 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.482 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.482 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.482 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.482 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.482 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.482 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.482 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.482 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.482 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.483 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.483 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.483 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.483 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.483 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "telemetry", 2024-09-10T08:49:00.483 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.483 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.483 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.483 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_basic": { 2024-09-10T08:49:00.483 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_basic", 2024-09-10T08:49:00.483 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.483 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.483 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.484 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.484 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.484 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.484 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.484 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T08:49:00.484 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.484 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.484 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.484 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_crash": { 2024-09-10T08:49:00.484 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_crash", 2024-09-10T08:49:00.484 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.484 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.485 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.485 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.485 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.485 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.485 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.485 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T08:49:00.485 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.485 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.485 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.485 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_device": { 2024-09-10T08:49:00.485 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_device", 2024-09-10T08:49:00.485 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.485 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.486 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.486 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.486 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.486 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.486 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.486 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T08:49:00.486 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.486 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.486 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.486 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_ident": { 2024-09-10T08:49:00.486 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_ident", 2024-09-10T08:49:00.486 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.486 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.487 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.487 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.487 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.487 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.487 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.487 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T08:49:00.487 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.487 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.487 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.487 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_perf": { 2024-09-10T08:49:00.487 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_perf", 2024-09-10T08:49:00.487 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.487 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.488 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.488 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.488 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.488 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.488 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.488 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T08:49:00.488 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.488 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.488 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.488 INFO:teuthology.orchestra.run.smithi089.stdout: "contact": { 2024-09-10T08:49:00.488 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "contact", 2024-09-10T08:49:00.488 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.488 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.489 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.489 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.489 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.489 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.489 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.489 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.489 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.489 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.489 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.489 INFO:teuthology.orchestra.run.smithi089.stdout: "description": { 2024-09-10T08:49:00.489 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "description", 2024-09-10T08:49:00.489 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.489 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.490 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.490 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.490 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.490 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.490 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.490 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.490 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.490 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.490 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.490 INFO:teuthology.orchestra.run.smithi089.stdout: "device_url": { 2024-09-10T08:49:00.490 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_url", 2024-09-10T08:49:00.490 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.490 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.491 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.491 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T08:49:00.491 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.491 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.491 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.491 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.491 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.491 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.491 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.491 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.491 INFO:teuthology.orchestra.run.smithi089.stdout: "enabled": { 2024-09-10T08:49:00.491 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enabled", 2024-09-10T08:49:00.491 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.491 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.492 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.492 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.492 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.492 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.492 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.492 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.492 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.492 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.492 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.492 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-10T08:49:00.492 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-10T08:49:00.492 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.492 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.493 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.493 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "24", 2024-09-10T08:49:00.493 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "8", 2024-09-10T08:49:00.493 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.493 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.493 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.493 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.493 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.493 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.493 INFO:teuthology.orchestra.run.smithi089.stdout: "last_opt_revision": { 2024-09-10T08:49:00.493 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "last_opt_revision", 2024-09-10T08:49:00.493 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.494 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.494 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.494 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1", 2024-09-10T08:49:00.494 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.494 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.494 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.494 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.494 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.494 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.494 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.494 INFO:teuthology.orchestra.run.smithi089.stdout: "leaderboard": { 2024-09-10T08:49:00.494 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "leaderboard", 2024-09-10T08:49:00.494 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.495 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.495 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.495 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.495 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.495 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.495 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.495 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.495 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.495 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.495 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.495 INFO:teuthology.orchestra.run.smithi089.stdout: "leaderboard_description": { 2024-09-10T08:49:00.495 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "leaderboard_description", 2024-09-10T08:49:00.496 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.496 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.496 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.496 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.496 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.496 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.496 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.496 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.496 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.496 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.496 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.496 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.496 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.497 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.497 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.497 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.497 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.497 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.497 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.497 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.497 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.497 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.497 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.497 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.497 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.497 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.498 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.498 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.498 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.498 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.498 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.498 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.498 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.498 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.498 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.498 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.498 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.498 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.498 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.498 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.499 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.500 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "organization": { 2024-09-10T08:49:00.501 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "organization", 2024-09-10T08:49:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "proxy": { 2024-09-10T08:49:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "proxy", 2024-09-10T08:49:00.502 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "url": { 2024-09-10T08:49:00.503 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "url", 2024-09-10T08:49:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T08:49:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.504 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "test_orchestrator", 2024-09-10T08:49:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.505 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.506 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.507 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.508 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.509 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "volumes", 2024-09-10T08:49:00.510 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.511 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.512 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.513 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.514 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.515 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "max_concurrent_clones": { 2024-09-10T08:49:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_concurrent_clones", 2024-09-10T08:49:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "4", 2024-09-10T08:49:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.516 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T08:49:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "periodic_async_work": { 2024-09-10T08:49:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "periodic_async_work", 2024-09-10T08:49:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.517 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Periodically check for async work", 2024-09-10T08:49:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "snapshot_clone_delay": { 2024-09-10T08:49:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snapshot_clone_delay", 2024-09-10T08:49:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.518 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-10T08:49:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T08:49:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "snapshot_clone_no_wait": { 2024-09-10T08:49:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T08:49:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.519 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T08:49:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix", 2024-09-10T08:49:00.520 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: "discovery_interval": { 2024-09-10T08:49:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "discovery_interval", 2024-09-10T08:49:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-10T08:49:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "100", 2024-09-10T08:49:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.521 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "identifier": { 2024-09-10T08:49:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "identifier", 2024-09-10T08:49:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.522 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.523 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.523 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.523 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.523 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.523 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-10T08:49:00.523 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-10T08:49:00.523 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.523 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.523 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.523 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-10T08:49:00.523 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.523 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.523 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.523 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.524 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.524 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.524 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.524 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.524 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.524 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.524 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.524 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.524 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.524 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.524 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.524 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.525 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.525 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.525 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.525 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.525 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.525 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.525 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.525 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.525 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.525 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.525 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.525 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.525 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.525 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.526 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.526 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.526 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.526 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.526 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.526 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.526 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.526 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.526 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.526 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.526 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.526 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.526 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.526 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.527 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.527 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.527 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.527 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.527 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.527 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.527 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.527 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.527 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.527 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.527 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.527 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.527 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.528 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.528 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.528 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.528 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.528 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.528 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.528 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.528 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.528 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.528 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.528 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.528 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.528 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.528 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.529 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.529 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.529 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.529 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.529 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.529 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.529 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_host": { 2024-09-10T08:49:00.529 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_host", 2024-09-10T08:49:00.529 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.529 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.529 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.529 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.529 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.529 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.530 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.530 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.530 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.530 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.530 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.530 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.530 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_port": { 2024-09-10T08:49:00.530 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_port", 2024-09-10T08:49:00.530 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.530 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.530 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.530 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10051", 2024-09-10T08:49:00.530 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.530 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.530 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.531 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.531 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.531 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.531 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.531 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.531 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_sender": { 2024-09-10T08:49:00.531 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_sender", 2024-09-10T08:49:00.531 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.531 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.531 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.531 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T08:49:00.531 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.531 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.531 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.531 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.532 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.532 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.532 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.532 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.532 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.532 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.532 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-10T08:49:00.532 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.532 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.532 INFO:teuthology.orchestra.run.smithi089.stdout: "gid": 24113, 2024-09-10T08:49:00.532 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "y", 2024-09-10T08:49:00.532 INFO:teuthology.orchestra.run.smithi089.stdout: "mgr_features": 4540701547738038271, 2024-09-10T08:49:00.533 INFO:teuthology.orchestra.run.smithi089.stdout: "available_modules": [ 2024-09-10T08:49:00.533 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.533 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "alerts", 2024-09-10T08:49:00.533 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.533 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.533 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.533 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-10T08:49:00.533 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-10T08:49:00.533 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.533 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.533 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.533 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-10T08:49:00.533 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.533 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.533 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.534 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T08:49:00.534 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.534 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.534 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.534 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.534 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.534 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.534 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.534 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.534 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.534 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.534 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.534 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.534 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.535 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.535 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.535 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.535 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.535 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.535 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.535 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.535 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.535 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.535 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.535 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.535 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.535 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.535 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.536 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.536 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.536 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.536 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.536 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.536 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.536 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.536 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.536 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.536 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.536 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.536 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.536 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.536 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.536 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.537 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.537 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.537 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.537 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.537 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.537 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.537 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.537 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.537 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.537 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.537 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.537 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.537 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.538 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.538 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.538 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.538 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.538 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.538 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.538 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.538 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.538 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.538 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.538 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.538 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.538 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.538 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.539 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.539 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.539 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.539 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.539 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_destination": { 2024-09-10T08:49:00.539 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_destination", 2024-09-10T08:49:00.539 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.539 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.539 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.539 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.539 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.539 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.539 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.540 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Email address to send alerts to", 2024-09-10T08:49:00.540 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.540 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.540 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.540 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_from_name": { 2024-09-10T08:49:00.540 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_from_name", 2024-09-10T08:49:00.540 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.540 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.540 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.540 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "Ceph", 2024-09-10T08:49:00.540 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.540 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.540 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.541 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Email From: name", 2024-09-10T08:49:00.541 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.541 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.541 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.541 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_host": { 2024-09-10T08:49:00.541 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_host", 2024-09-10T08:49:00.541 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.541 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.541 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.541 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.541 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.541 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.541 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.542 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP server", 2024-09-10T08:49:00.542 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.542 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.542 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.542 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_password": { 2024-09-10T08:49:00.542 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_password", 2024-09-10T08:49:00.542 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.542 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.542 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.542 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.542 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.542 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.542 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.543 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Password to authenticate with", 2024-09-10T08:49:00.543 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.543 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.543 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.543 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.543 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_port": { 2024-09-10T08:49:00.543 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_port", 2024-09-10T08:49:00.543 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.543 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.543 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.543 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "465", 2024-09-10T08:49:00.543 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.543 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.544 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.544 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP port", 2024-09-10T08:49:00.544 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.544 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.544 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.544 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.544 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_sender": { 2024-09-10T08:49:00.544 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_sender", 2024-09-10T08:49:00.544 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.544 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.544 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.544 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.544 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.544 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.545 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.545 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP envelope sender", 2024-09-10T08:49:00.545 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.545 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.545 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.545 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.545 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_ssl": { 2024-09-10T08:49:00.545 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_ssl", 2024-09-10T08:49:00.545 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.545 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.545 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.545 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.545 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.545 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.546 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.546 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T08:49:00.546 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.546 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.546 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.546 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_user": { 2024-09-10T08:49:00.546 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_user", 2024-09-10T08:49:00.546 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.546 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.546 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.546 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.546 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.546 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.547 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.547 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "User to authenticate as", 2024-09-10T08:49:00.547 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.547 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.547 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.547 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.547 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.547 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.547 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.547 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "balancer", 2024-09-10T08:49:00.547 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.547 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.547 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.548 INFO:teuthology.orchestra.run.smithi089.stdout: "active": { 2024-09-10T08:49:00.548 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "active", 2024-09-10T08:49:00.548 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.548 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.548 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.548 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.548 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.548 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.548 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.548 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T08:49:00.548 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.548 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.548 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.548 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.549 INFO:teuthology.orchestra.run.smithi089.stdout: "begin_time": { 2024-09-10T08:49:00.549 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "begin_time", 2024-09-10T08:49:00.549 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.549 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.549 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.549 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0000", 2024-09-10T08:49:00.549 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.549 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.549 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.549 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T08:49:00.549 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T08:49:00.549 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.549 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.550 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.550 INFO:teuthology.orchestra.run.smithi089.stdout: "begin_weekday": { 2024-09-10T08:49:00.550 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "begin_weekday", 2024-09-10T08:49:00.550 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-10T08:49:00.550 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.550 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.550 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-10T08:49:00.550 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0", 2024-09-10T08:49:00.550 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "6", 2024-09-10T08:49:00.550 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.550 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T08:49:00.550 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T08:49:00.550 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.551 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.551 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.551 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_max_iterations": { 2024-09-10T08:49:00.551 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_max_iterations", 2024-09-10T08:49:00.551 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-10T08:49:00.551 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.551 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.551 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "25", 2024-09-10T08:49:00.551 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-10T08:49:00.551 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "250", 2024-09-10T08:49:00.551 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.551 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T08:49:00.551 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.551 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.552 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.552 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.552 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_metrics": { 2024-09-10T08:49:00.552 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_metrics", 2024-09-10T08:49:00.552 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.552 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.552 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.552 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T08:49:00.552 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.552 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.552 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.552 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T08:49:00.552 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T08:49:00.552 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.553 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.553 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_step": { 2024-09-10T08:49:00.553 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_step", 2024-09-10T08:49:00.553 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:00.553 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.553 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.553 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0.5", 2024-09-10T08:49:00.553 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0.001", 2024-09-10T08:49:00.553 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "0.999", 2024-09-10T08:49:00.553 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.553 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "aggressiveness of optimization", 2024-09-10T08:49:00.553 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T08:49:00.553 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.554 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.554 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.554 INFO:teuthology.orchestra.run.smithi089.stdout: "end_time": { 2024-09-10T08:49:00.554 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "end_time", 2024-09-10T08:49:00.554 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.554 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.554 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.554 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "2359", 2024-09-10T08:49:00.554 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.554 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.554 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.554 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T08:49:00.554 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T08:49:00.554 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.554 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.555 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.555 INFO:teuthology.orchestra.run.smithi089.stdout: "end_weekday": { 2024-09-10T08:49:00.555 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "end_weekday", 2024-09-10T08:49:00.555 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-10T08:49:00.555 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.555 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.555 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-10T08:49:00.555 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0", 2024-09-10T08:49:00.555 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "6", 2024-09-10T08:49:00.555 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.555 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T08:49:00.555 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T08:49:00.555 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.555 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.556 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.556 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.556 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.556 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.556 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.556 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.556 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.556 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.556 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.556 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.556 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.556 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.556 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.557 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.557 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.557 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.557 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.557 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.557 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.557 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.557 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.557 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.557 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.557 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.557 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.557 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.557 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.558 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.558 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.558 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.558 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.558 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.558 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.558 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.558 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.558 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.558 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.558 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.558 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.558 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.559 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.559 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.559 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.559 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.559 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.559 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.559 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.559 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.559 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.559 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.559 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.559 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.559 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.559 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.559 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.560 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.560 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.560 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.560 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.560 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.560 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.560 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.560 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.560 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.560 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.560 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.560 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.560 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.561 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.561 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.561 INFO:teuthology.orchestra.run.smithi089.stdout: "min_score": { 2024-09-10T08:49:00.561 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "min_score", 2024-09-10T08:49:00.561 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:00.561 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.561 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.561 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-10T08:49:00.561 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.561 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.561 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.561 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T08:49:00.561 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.561 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.562 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.562 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.562 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": { 2024-09-10T08:49:00.562 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mode", 2024-09-10T08:49:00.562 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.562 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.562 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.562 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "upmap", 2024-09-10T08:49:00.562 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.562 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.562 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.562 INFO:teuthology.orchestra.run.smithi089.stdout: "crush-compat", 2024-09-10T08:49:00.562 INFO:teuthology.orchestra.run.smithi089.stdout: "none", 2024-09-10T08:49:00.562 INFO:teuthology.orchestra.run.smithi089.stdout: "read", 2024-09-10T08:49:00.563 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap", 2024-09-10T08:49:00.563 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap-read" 2024-09-10T08:49:00.563 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.563 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Balancer mode", 2024-09-10T08:49:00.563 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.563 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.563 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.563 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.563 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_ids": { 2024-09-10T08:49:00.563 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pool_ids", 2024-09-10T08:49:00.563 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.563 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.563 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.563 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.564 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.564 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.564 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.564 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T08:49:00.564 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.564 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.564 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.564 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.564 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-10T08:49:00.564 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-10T08:49:00.564 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.564 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.564 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.564 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-10T08:49:00.565 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.565 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.565 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.565 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T08:49:00.565 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.565 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.565 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.565 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.565 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap_max_deviation": { 2024-09-10T08:49:00.565 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "upmap_max_deviation", 2024-09-10T08:49:00.565 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.565 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.565 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.565 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-10T08:49:00.566 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-10T08:49:00.566 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.566 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.566 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T08:49:00.566 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T08:49:00.566 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.566 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.566 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.566 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap_max_optimizations": { 2024-09-10T08:49:00.566 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "upmap_max_optimizations", 2024-09-10T08:49:00.566 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-10T08:49:00.566 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.566 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.567 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:00.567 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.567 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.567 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.567 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T08:49:00.567 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.567 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.567 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.567 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.567 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.567 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.567 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.567 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephadm", 2024-09-10T08:49:00.568 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.568 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.568 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.568 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_down_multiplier": { 2024-09-10T08:49:00.568 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_down_multiplier", 2024-09-10T08:49:00.568 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:00.568 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.568 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.568 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3.0", 2024-09-10T08:49:00.568 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.568 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.568 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.568 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T08:49:00.568 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.569 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.569 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.569 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.569 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_refresh_rate": { 2024-09-10T08:49:00.569 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_refresh_rate", 2024-09-10T08:49:00.569 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.569 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.569 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.569 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "20", 2024-09-10T08:49:00.569 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.569 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.569 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.569 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T08:49:00.569 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.570 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.570 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.570 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.570 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_starting_port": { 2024-09-10T08:49:00.570 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_starting_port", 2024-09-10T08:49:00.570 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.570 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.570 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.570 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "4721", 2024-09-10T08:49:00.570 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.570 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.570 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.570 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T08:49:00.570 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.571 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.571 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.571 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.571 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_ptrace": { 2024-09-10T08:49:00.571 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_ptrace", 2024-09-10T08:49:00.571 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.571 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.571 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.571 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.571 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.571 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.571 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.571 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T08:49:00.571 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T08:49:00.572 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.572 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.572 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.572 INFO:teuthology.orchestra.run.smithi089.stdout: "autotune_interval": { 2024-09-10T08:49:00.572 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "autotune_interval", 2024-09-10T08:49:00.572 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.572 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.572 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.572 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-10T08:49:00.572 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.572 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.572 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.572 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T08:49:00.573 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.573 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.573 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.573 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.573 INFO:teuthology.orchestra.run.smithi089.stdout: "autotune_memory_target_ratio": { 2024-09-10T08:49:00.573 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T08:49:00.573 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:00.573 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.573 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.573 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0.7", 2024-09-10T08:49:00.573 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.573 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.573 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.573 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T08:49:00.574 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.574 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.574 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.574 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.574 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm_log_destination": { 2024-09-10T08:49:00.574 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephadm_log_destination", 2024-09-10T08:49:00.574 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.574 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.574 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.574 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.574 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.574 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.574 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.574 INFO:teuthology.orchestra.run.smithi089.stdout: "file", 2024-09-10T08:49:00.575 INFO:teuthology.orchestra.run.smithi089.stdout: "file,syslog", 2024-09-10T08:49:00.575 INFO:teuthology.orchestra.run.smithi089.stdout: "syslog" 2024-09-10T08:49:00.575 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.575 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T08:49:00.575 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.575 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.575 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.575 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.575 INFO:teuthology.orchestra.run.smithi089.stdout: "cgroups_split": { 2024-09-10T08:49:00.575 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cgroups_split", 2024-09-10T08:49:00.575 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.575 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.575 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.575 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.576 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.576 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.576 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.576 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T08:49:00.576 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.576 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.576 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.576 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.576 INFO:teuthology.orchestra.run.smithi089.stdout: "config_checks_enabled": { 2024-09-10T08:49:00.576 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_checks_enabled", 2024-09-10T08:49:00.576 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.576 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.576 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.576 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.577 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.577 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.577 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.577 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T08:49:00.577 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.577 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.577 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.580 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.581 INFO:teuthology.orchestra.run.smithi089.stdout: "config_dashboard": { 2024-09-10T08:49:00.581 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_dashboard", 2024-09-10T08:49:00.581 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.581 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.581 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.581 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.581 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.581 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.581 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.581 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T08:49:00.581 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.581 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.582 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.582 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.582 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_alertmanager": { 2024-09-10T08:49:00.582 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_alertmanager", 2024-09-10T08:49:00.582 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.582 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.582 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.582 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T08:49:00.582 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.582 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.582 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.582 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-10T08:49:00.583 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.583 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.583 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.583 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.583 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_base": { 2024-09-10T08:49:00.583 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_base", 2024-09-10T08:49:00.583 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.583 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.583 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.583 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T08:49:00.583 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.583 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.583 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.583 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Container image name, without the tag", 2024-09-10T08:49:00.583 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.584 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.584 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.584 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.584 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_elasticsearch": { 2024-09-10T08:49:00.584 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_elasticsearch", 2024-09-10T08:49:00.584 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.584 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.584 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.584 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T08:49:00.584 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.584 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.585 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.585 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "elasticsearch container image", 2024-09-10T08:49:00.585 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.585 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.585 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.585 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.585 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_grafana": { 2024-09-10T08:49:00.585 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_grafana", 2024-09-10T08:49:00.585 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.585 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.585 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.585 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T08:49:00.586 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.586 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.586 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.586 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-10T08:49:00.586 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.586 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.586 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.586 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.586 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_haproxy": { 2024-09-10T08:49:00.587 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_haproxy", 2024-09-10T08:49:00.587 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.587 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.587 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.587 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T08:49:00.587 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.587 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.587 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.587 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "HAproxy container image", 2024-09-10T08:49:00.587 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.587 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.587 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.587 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.588 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_agent": { 2024-09-10T08:49:00.588 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_agent", 2024-09-10T08:49:00.588 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.588 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.588 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.588 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T08:49:00.588 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.588 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.588 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.588 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger agent container image", 2024-09-10T08:49:00.588 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.588 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.589 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.589 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.589 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_collector": { 2024-09-10T08:49:00.589 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_collector", 2024-09-10T08:49:00.589 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.589 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.589 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.589 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T08:49:00.589 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.589 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.589 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.589 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger collector container image", 2024-09-10T08:49:00.589 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.589 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.590 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.590 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.590 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_query": { 2024-09-10T08:49:00.590 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_query", 2024-09-10T08:49:00.590 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.590 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.590 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.590 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T08:49:00.590 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.590 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.590 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.590 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger query container image", 2024-09-10T08:49:00.590 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.591 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.591 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.591 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.591 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_keepalived": { 2024-09-10T08:49:00.591 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_keepalived", 2024-09-10T08:49:00.591 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.591 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.591 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.591 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T08:49:00.591 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.591 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.591 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.591 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Keepalived container image", 2024-09-10T08:49:00.591 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.591 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.592 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.592 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.592 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_loki": { 2024-09-10T08:49:00.592 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_loki", 2024-09-10T08:49:00.592 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.592 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.592 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.592 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T08:49:00.592 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.592 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.592 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.592 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Loki container image", 2024-09-10T08:49:00.592 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.593 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.593 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.593 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.593 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_node_exporter": { 2024-09-10T08:49:00.593 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_node_exporter", 2024-09-10T08:49:00.593 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.593 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.593 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.593 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T08:49:00.593 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.593 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.593 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.593 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-10T08:49:00.594 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.594 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.594 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.594 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.594 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_nvmeof": { 2024-09-10T08:49:00.594 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_nvmeof", 2024-09-10T08:49:00.594 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.594 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.594 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.594 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T08:49:00.594 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.594 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.594 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.594 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Nvme-of container image", 2024-09-10T08:49:00.595 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.595 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.595 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.595 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.595 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_prometheus": { 2024-09-10T08:49:00.595 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_prometheus", 2024-09-10T08:49:00.595 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.595 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.595 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.595 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T08:49:00.595 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.595 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.595 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.595 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-10T08:49:00.596 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.596 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.596 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.596 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.596 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_promtail": { 2024-09-10T08:49:00.596 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_promtail", 2024-09-10T08:49:00.596 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.596 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.596 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.596 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T08:49:00.596 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.596 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.596 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.596 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Promtail container image", 2024-09-10T08:49:00.597 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.597 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.597 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.597 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.597 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_samba": { 2024-09-10T08:49:00.597 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_samba", 2024-09-10T08:49:00.597 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.597 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.597 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.597 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T08:49:00.597 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.597 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.597 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.598 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Samba/SMB container image", 2024-09-10T08:49:00.598 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.598 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.598 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.598 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.598 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_snmp_gateway": { 2024-09-10T08:49:00.598 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_snmp_gateway", 2024-09-10T08:49:00.598 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.598 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.598 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.598 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T08:49:00.598 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.598 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.598 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.599 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SNMP Gateway container image", 2024-09-10T08:49:00.599 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.599 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.599 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.599 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.599 INFO:teuthology.orchestra.run.smithi089.stdout: "container_init": { 2024-09-10T08:49:00.599 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_init", 2024-09-10T08:49:00.599 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.599 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.599 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.599 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.599 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.599 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.600 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.600 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T08:49:00.600 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.600 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.600 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.600 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.600 INFO:teuthology.orchestra.run.smithi089.stdout: "daemon_cache_timeout": { 2024-09-10T08:49:00.600 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "daemon_cache_timeout", 2024-09-10T08:49:00.600 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.600 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.600 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.601 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-10T08:49:00.601 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.601 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.601 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.601 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T08:49:00.601 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.601 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.601 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.601 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.601 INFO:teuthology.orchestra.run.smithi089.stdout: "default_cephadm_command_timeout": { 2024-09-10T08:49:00.601 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T08:49:00.601 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.601 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.602 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.602 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "900", 2024-09-10T08:49:00.602 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.602 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.602 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.602 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T08:49:00.602 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.602 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.602 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.602 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.602 INFO:teuthology.orchestra.run.smithi089.stdout: "default_registry": { 2024-09-10T08:49:00.602 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "default_registry", 2024-09-10T08:49:00.602 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.603 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.603 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.603 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io", 2024-09-10T08:49:00.603 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.603 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.603 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.603 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T08:49:00.603 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.603 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.603 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.603 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.603 INFO:teuthology.orchestra.run.smithi089.stdout: "device_cache_timeout": { 2024-09-10T08:49:00.604 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_cache_timeout", 2024-09-10T08:49:00.604 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.604 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.604 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.604 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1800", 2024-09-10T08:49:00.604 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.604 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.604 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.604 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache device inventory", 2024-09-10T08:49:00.604 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.604 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.604 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.605 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.605 INFO:teuthology.orchestra.run.smithi089.stdout: "device_enhanced_scan": { 2024-09-10T08:49:00.605 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_enhanced_scan", 2024-09-10T08:49:00.605 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.605 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.605 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.605 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.605 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.605 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.605 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.605 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T08:49:00.605 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.605 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.606 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.606 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.606 INFO:teuthology.orchestra.run.smithi089.stdout: "facts_cache_timeout": { 2024-09-10T08:49:00.606 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "facts_cache_timeout", 2024-09-10T08:49:00.606 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.606 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.606 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.606 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-10T08:49:00.606 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.606 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.606 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.606 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache host facts data", 2024-09-10T08:49:00.606 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.607 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.607 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.607 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.607 INFO:teuthology.orchestra.run.smithi089.stdout: "grafana_dashboards_path": { 2024-09-10T08:49:00.607 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "grafana_dashboards_path", 2024-09-10T08:49:00.607 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.607 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.607 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.607 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T08:49:00.607 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.607 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.607 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.607 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T08:49:00.607 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.608 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.608 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.608 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.608 INFO:teuthology.orchestra.run.smithi089.stdout: "host_check_interval": { 2024-09-10T08:49:00.608 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "host_check_interval", 2024-09-10T08:49:00.608 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.608 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.608 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.608 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-10T08:49:00.608 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.608 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.608 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.608 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to perform a host check", 2024-09-10T08:49:00.608 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.609 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.609 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.609 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.609 INFO:teuthology.orchestra.run.smithi089.stdout: "hw_monitoring": { 2024-09-10T08:49:00.609 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "hw_monitoring", 2024-09-10T08:49:00.609 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.609 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.609 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.609 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.609 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.609 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.609 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.609 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T08:49:00.609 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.610 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.610 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.610 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.610 INFO:teuthology.orchestra.run.smithi089.stdout: "inventory_list_all": { 2024-09-10T08:49:00.610 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "inventory_list_all", 2024-09-10T08:49:00.610 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.610 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.610 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.610 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.610 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.610 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.610 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.610 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T08:49:00.610 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.611 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.611 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.611 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.611 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.611 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.611 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.611 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.611 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.611 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.611 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.611 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.611 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.611 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.611 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.612 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.612 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.612 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.612 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.612 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.612 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.612 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.612 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.612 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.612 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.612 INFO:teuthology.orchestra.run.smithi089.stdout: "log_refresh_metadata": { 2024-09-10T08:49:00.612 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_refresh_metadata", 2024-09-10T08:49:00.612 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.612 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.613 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.613 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.613 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.613 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.613 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.613 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T08:49:00.613 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.613 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.613 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.613 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.613 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.613 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.613 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.614 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.614 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.614 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.614 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.614 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.614 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.614 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T08:49:00.614 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.614 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.614 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.614 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.614 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.614 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.614 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.615 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.615 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.615 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.615 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.615 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.615 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.615 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.615 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.615 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.615 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.615 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.615 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.615 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.616 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.616 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.616 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.616 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.616 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.616 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.616 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.616 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.616 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.616 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.616 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.616 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.616 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.617 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.617 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.617 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.617 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.617 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.617 INFO:teuthology.orchestra.run.smithi089.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T08:49:00.617 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T08:49:00.617 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.617 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.617 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.617 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.617 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.617 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.618 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.618 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T08:49:00.618 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.618 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.618 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.618 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.618 INFO:teuthology.orchestra.run.smithi089.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T08:49:00.618 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T08:49:00.618 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.618 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.619 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.619 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "*", 2024-09-10T08:49:00.619 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.619 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.619 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.619 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T08:49:00.619 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.619 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.619 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.620 INFO:teuthology.orchestra.run.smithi089.stdout: "max_count_per_host": { 2024-09-10T08:49:00.620 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_count_per_host", 2024-09-10T08:49:00.620 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.620 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.620 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.620 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:00.620 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.620 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.620 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.620 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "max number of daemons per service per host", 2024-09-10T08:49:00.620 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.620 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.621 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.621 INFO:teuthology.orchestra.run.smithi089.stdout: "max_osd_draining_count": { 2024-09-10T08:49:00.621 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_osd_draining_count", 2024-09-10T08:49:00.621 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.621 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.621 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.621 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:00.621 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.621 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.621 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.621 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T08:49:00.621 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.621 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.622 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.622 INFO:teuthology.orchestra.run.smithi089.stdout: "migration_current": { 2024-09-10T08:49:00.622 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "migration_current", 2024-09-10T08:49:00.622 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.622 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.622 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.622 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.622 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.622 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.622 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.622 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "internal - do not modify", 2024-09-10T08:49:00.622 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.622 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.622 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.623 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": { 2024-09-10T08:49:00.623 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mode", 2024-09-10T08:49:00.623 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.623 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.623 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.623 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "root", 2024-09-10T08:49:00.623 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.623 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.623 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.623 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm-package", 2024-09-10T08:49:00.623 INFO:teuthology.orchestra.run.smithi089.stdout: "root" 2024-09-10T08:49:00.623 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.624 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T08:49:00.624 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.624 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.624 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.624 INFO:teuthology.orchestra.run.smithi089.stdout: "oob_default_addr": { 2024-09-10T08:49:00.624 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "oob_default_addr", 2024-09-10T08:49:00.624 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.624 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.624 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.624 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "169.254.1.1", 2024-09-10T08:49:00.624 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.624 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.624 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.625 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T08:49:00.625 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.625 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.625 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.625 INFO:teuthology.orchestra.run.smithi089.stdout: "prometheus_alerts_path": { 2024-09-10T08:49:00.625 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prometheus_alerts_path", 2024-09-10T08:49:00.625 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.625 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.625 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.625 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T08:49:00.625 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.625 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.625 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.626 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T08:49:00.626 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.626 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.626 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.626 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_insecure": { 2024-09-10T08:49:00.626 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_insecure", 2024-09-10T08:49:00.626 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.626 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.626 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.626 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.626 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.626 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.626 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.627 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T08:49:00.627 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.627 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.627 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.627 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_password": { 2024-09-10T08:49:00.627 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_password", 2024-09-10T08:49:00.627 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.627 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.627 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.627 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.627 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.628 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.628 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.628 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T08:49:00.628 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.628 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.628 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.628 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_url": { 2024-09-10T08:49:00.628 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_url", 2024-09-10T08:49:00.628 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.628 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.628 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.629 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.629 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.629 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.629 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.629 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T08:49:00.629 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.629 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.629 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.629 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_username": { 2024-09-10T08:49:00.629 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_username", 2024-09-10T08:49:00.629 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.630 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.630 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.630 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.630 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.630 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.630 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.630 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T08:49:00.630 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.630 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.630 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.630 INFO:teuthology.orchestra.run.smithi089.stdout: "secure_monitoring_stack": { 2024-09-10T08:49:00.631 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "secure_monitoring_stack", 2024-09-10T08:49:00.631 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.631 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.631 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.631 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.631 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.631 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.631 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.631 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T08:49:00.631 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.631 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.631 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.632 INFO:teuthology.orchestra.run.smithi089.stdout: "service_discovery_port": { 2024-09-10T08:49:00.632 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "service_discovery_port", 2024-09-10T08:49:00.632 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.632 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.632 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.632 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8765", 2024-09-10T08:49:00.632 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.632 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.632 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.632 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "cephadm service discovery port", 2024-09-10T08:49:00.632 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.632 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.632 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.633 INFO:teuthology.orchestra.run.smithi089.stdout: "ssh_config_file": { 2024-09-10T08:49:00.633 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssh_config_file", 2024-09-10T08:49:00.633 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.633 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.633 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.633 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.633 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.633 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.633 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.633 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T08:49:00.633 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.633 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.634 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.634 INFO:teuthology.orchestra.run.smithi089.stdout: "use_agent": { 2024-09-10T08:49:00.634 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "use_agent", 2024-09-10T08:49:00.634 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.634 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.634 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.634 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.634 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.634 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.634 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.634 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T08:49:00.634 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.635 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.635 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.635 INFO:teuthology.orchestra.run.smithi089.stdout: "use_repo_digest": { 2024-09-10T08:49:00.635 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "use_repo_digest", 2024-09-10T08:49:00.635 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.635 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.635 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.635 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.635 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.635 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.635 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.635 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T08:49:00.636 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.636 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.636 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.636 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_failed_host_check": { 2024-09-10T08:49:00.636 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_failed_host_check", 2024-09-10T08:49:00.636 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.636 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.636 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.636 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.636 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.636 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.636 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.636 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T08:49:00.637 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.637 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.637 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.637 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_stray_daemons": { 2024-09-10T08:49:00.637 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_stray_daemons", 2024-09-10T08:49:00.637 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.637 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.637 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.637 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.637 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.637 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.637 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.638 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T08:49:00.638 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.638 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.638 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.638 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_stray_hosts": { 2024-09-10T08:49:00.638 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_stray_hosts", 2024-09-10T08:49:00.638 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.638 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.638 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.638 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.638 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.638 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.639 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.639 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T08:49:00.639 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.639 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.639 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.639 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.639 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.639 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.639 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crash", 2024-09-10T08:49:00.639 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.639 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.639 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.639 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: "retain_interval": { 2024-09-10T08:49:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "retain_interval", 2024-09-10T08:49:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "31536000", 2024-09-10T08:49:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T08:49:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_recent_interval": { 2024-09-10T08:49:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_recent_interval", 2024-09-10T08:49:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1209600", 2024-09-10T08:49:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T08:49:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "dashboard", 2024-09-10T08:49:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T08:49:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T08:49:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T08:49:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T08:49:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T08:49:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T08:49:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: "AUDIT_API_ENABLED": { 2024-09-10T08:49:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T08:49:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T08:49:00.652 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T08:49:00.652 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.652 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.652 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.652 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.652 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.652 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.652 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.652 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.652 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.652 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.652 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.652 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.653 INFO:teuthology.orchestra.run.smithi089.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T08:49:00.653 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T08:49:00.653 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.653 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.653 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.653 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.653 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.653 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.653 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.653 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.653 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.653 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.653 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.653 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.654 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T08:49:00.654 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T08:49:00.654 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.654 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.654 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.654 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.654 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.654 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.654 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.654 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.654 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.654 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.654 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.655 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T08:49:00.655 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T08:49:00.655 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.655 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.655 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.655 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.655 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.655 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.655 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.655 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.655 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.655 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.655 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.656 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T08:49:00.656 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T08:49:00.656 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.656 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.656 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.656 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.656 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.656 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.656 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.656 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.656 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.656 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.657 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.657 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T08:49:00.657 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T08:49:00.657 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.657 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.657 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.657 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.657 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.657 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.657 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.657 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.657 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.657 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.658 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.658 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T08:49:00.658 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T08:49:00.658 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.658 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.658 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.658 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.658 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.658 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.658 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.658 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.658 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.658 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.659 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.659 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T08:49:00.659 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T08:49:00.659 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.659 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.659 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.659 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.659 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.659 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.659 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.659 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.659 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.659 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.660 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.660 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.660 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T08:49:00.660 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T08:49:00.660 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.660 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.660 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.660 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.660 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.660 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.660 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.660 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.661 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.661 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.661 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.661 INFO:teuthology.orchestra.run.smithi089.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T08:49:00.661 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T08:49:00.661 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.661 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.661 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.661 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.661 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.661 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.661 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.662 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.662 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.662 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.662 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.662 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.662 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T08:49:00.662 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T08:49:00.662 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.662 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.662 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.662 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2024-09-10T08:49:00.662 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.663 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.663 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.663 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.663 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.663 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.663 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.663 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.663 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T08:49:00.663 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T08:49:00.663 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.663 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.663 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.663 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.664 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.664 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.664 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.664 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.664 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.664 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.664 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.664 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.664 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_URL": { 2024-09-10T08:49:00.664 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_URL", 2024-09-10T08:49:00.664 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.664 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.664 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.665 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.665 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.665 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.665 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.665 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.665 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.665 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.665 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.665 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.665 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T08:49:00.665 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T08:49:00.665 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.665 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.665 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.665 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2024-09-10T08:49:00.666 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.666 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.666 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.666 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.666 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.666 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.666 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.666 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.666 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T08:49:00.666 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T08:49:00.666 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.666 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.666 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.666 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.667 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.667 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.667 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.667 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.667 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.667 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.667 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.667 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.667 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T08:49:00.667 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T08:49:00.667 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.667 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.667 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.667 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.667 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.668 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.668 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.668 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.668 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.668 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.668 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.668 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.668 INFO:teuthology.orchestra.run.smithi089.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T08:49:00.668 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T08:49:00.668 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.668 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.669 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.669 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.669 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.669 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.669 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.669 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.669 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.669 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.669 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.669 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.669 INFO:teuthology.orchestra.run.smithi089.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T08:49:00.669 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T08:49:00.669 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.669 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.670 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.670 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.670 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.670 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.670 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.670 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.670 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.670 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.670 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.670 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.670 INFO:teuthology.orchestra.run.smithi089.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T08:49:00.670 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T08:49:00.670 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.671 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.671 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.671 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.671 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.671 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.671 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.671 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.671 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.671 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.671 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.671 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.671 INFO:teuthology.orchestra.run.smithi089.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T08:49:00.671 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T08:49:00.672 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.672 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.672 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.672 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.672 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.672 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.672 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.672 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.672 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.672 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.672 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.672 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.673 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T08:49:00.673 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T08:49:00.673 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.673 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.673 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.673 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.673 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.673 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.673 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.673 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.673 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.673 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.673 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.674 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.674 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T08:49:00.674 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T08:49:00.674 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.674 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.674 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.674 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.674 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.674 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.674 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.674 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.674 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.674 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.674 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.675 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T08:49:00.675 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T08:49:00.675 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.675 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.675 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.675 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.675 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.675 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.675 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.675 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.675 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.675 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.675 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.675 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.676 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T08:49:00.676 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T08:49:00.676 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.676 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.676 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.676 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.676 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.676 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.676 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.676 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.676 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.676 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.676 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.676 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.677 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T08:49:00.677 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T08:49:00.677 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.677 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.677 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.677 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.677 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.677 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.677 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.677 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.677 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.677 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.677 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.678 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T08:49:00.678 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T08:49:00.678 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.678 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.678 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.678 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.678 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.678 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.678 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.678 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.678 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.678 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.678 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.678 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.679 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T08:49:00.679 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T08:49:00.679 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.679 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.679 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.679 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.679 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.679 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.679 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.679 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.679 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.679 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.679 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.679 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.679 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T08:49:00.680 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T08:49:00.680 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.680 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.680 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.680 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.680 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.680 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.680 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.680 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.680 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.680 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.680 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.680 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.681 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T08:49:00.681 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T08:49:00.681 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.681 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.681 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.681 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T08:49:00.681 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.681 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.681 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.681 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.681 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.681 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.682 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.682 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.682 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T08:49:00.682 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T08:49:00.682 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.682 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.682 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.682 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:00.682 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.682 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.682 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.682 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.682 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.682 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.683 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.683 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.683 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T08:49:00.683 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T08:49:00.683 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.683 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.683 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.683 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8", 2024-09-10T08:49:00.683 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.683 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.683 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.683 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.683 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.684 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.684 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.684 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.684 INFO:teuthology.orchestra.run.smithi089.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T08:49:00.684 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T08:49:00.684 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.684 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.684 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.684 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "45", 2024-09-10T08:49:00.684 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.684 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.685 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.685 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.685 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.685 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.685 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.685 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.685 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T08:49:00.685 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T08:49:00.685 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.685 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.685 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.685 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.686 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.686 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.686 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.686 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.686 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.686 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.686 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.686 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.686 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T08:49:00.686 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T08:49:00.686 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.687 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.687 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.687 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2024-09-10T08:49:00.687 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.687 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.687 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.687 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.687 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.687 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.687 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.687 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.687 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T08:49:00.687 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T08:49:00.688 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.688 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.688 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.688 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.688 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.688 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.688 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.688 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.688 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.688 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.689 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.689 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.689 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T08:49:00.689 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T08:49:00.689 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.689 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.689 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.689 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.689 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.689 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.689 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.689 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.690 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.690 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.690 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.690 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.690 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T08:49:00.690 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T08:49:00.690 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.690 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.690 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.690 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-10T08:49:00.691 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.691 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.691 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.691 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.691 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.691 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.691 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.691 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.691 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T08:49:00.691 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T08:49:00.691 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.691 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.691 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.692 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:00.692 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.692 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.692 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.692 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.692 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.692 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.692 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.692 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.692 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T08:49:00.692 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T08:49:00.693 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.693 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.693 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.693 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-10T08:49:00.693 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.693 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.693 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.693 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.693 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.693 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.693 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.694 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.694 INFO:teuthology.orchestra.run.smithi089.stdout: "cross_origin_url": { 2024-09-10T08:49:00.694 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cross_origin_url", 2024-09-10T08:49:00.694 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.694 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.694 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.694 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.694 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.694 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.694 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.694 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.694 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.695 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.695 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.695 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.695 INFO:teuthology.orchestra.run.smithi089.stdout: "crt_file": { 2024-09-10T08:49:00.695 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crt_file", 2024-09-10T08:49:00.695 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.695 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.695 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.695 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.695 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.695 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.695 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.695 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.696 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.696 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.696 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.696 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.696 INFO:teuthology.orchestra.run.smithi089.stdout: "debug": { 2024-09-10T08:49:00.696 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "debug", 2024-09-10T08:49:00.696 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.696 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.696 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.696 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.696 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.696 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.696 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.697 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable/disable debug options", 2024-09-10T08:49:00.697 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.697 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.697 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.697 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.697 INFO:teuthology.orchestra.run.smithi089.stdout: "jwt_token_ttl": { 2024-09-10T08:49:00.697 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "jwt_token_ttl", 2024-09-10T08:49:00.697 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.697 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.697 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.697 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "28800", 2024-09-10T08:49:00.698 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.698 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.698 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.698 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.698 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.698 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.698 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.698 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.698 INFO:teuthology.orchestra.run.smithi089.stdout: "key_file": { 2024-09-10T08:49:00.698 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "key_file", 2024-09-10T08:49:00.698 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.698 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.699 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.699 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.699 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.699 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.699 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.699 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.699 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.699 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.699 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.699 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.699 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.699 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.699 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.700 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.700 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.700 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.700 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.700 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.700 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.700 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.700 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.700 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.700 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.700 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.700 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.701 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.701 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.701 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.701 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.701 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.701 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.701 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.701 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.701 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.701 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.701 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.701 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.702 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.702 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.702 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.702 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.702 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.702 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.702 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.702 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.702 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.702 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.702 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.702 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.702 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.703 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.703 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.703 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.703 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.703 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.703 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.703 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.703 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.703 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.703 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.703 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.703 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.704 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.704 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.704 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.704 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.704 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.704 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.704 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.704 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.704 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.704 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.704 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.704 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.705 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.705 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.705 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.705 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.705 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.705 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.705 INFO:teuthology.orchestra.run.smithi089.stdout: "motd": { 2024-09-10T08:49:00.705 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "motd", 2024-09-10T08:49:00.705 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.705 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.705 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.705 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.705 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.705 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.706 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.706 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "The message of the day", 2024-09-10T08:49:00.706 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.706 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.706 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.706 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.706 INFO:teuthology.orchestra.run.smithi089.stdout: "redirect_resolve_ip_addr": { 2024-09-10T08:49:00.706 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T08:49:00.706 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.706 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.706 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.706 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.706 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.706 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.707 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.707 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.707 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.707 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.707 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.707 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.707 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2024-09-10T08:49:00.707 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2024-09-10T08:49:00.707 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.707 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.707 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.707 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "::", 2024-09-10T08:49:00.707 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.707 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.708 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.708 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.708 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.708 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.708 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.708 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.708 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2024-09-10T08:49:00.708 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2024-09-10T08:49:00.708 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.708 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.708 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.708 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8080", 2024-09-10T08:49:00.708 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.708 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.709 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.709 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.709 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.709 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.709 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.709 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.709 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl": { 2024-09-10T08:49:00.709 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl", 2024-09-10T08:49:00.709 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.709 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.709 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.709 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.709 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.709 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.710 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.710 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.710 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.710 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.710 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.710 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.710 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl_server_port": { 2024-09-10T08:49:00.710 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl_server_port", 2024-09-10T08:49:00.710 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.710 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.710 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.710 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8443", 2024-09-10T08:49:00.711 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.711 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.711 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.711 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.711 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.711 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.711 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.711 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.711 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_behaviour": { 2024-09-10T08:49:00.711 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_behaviour", 2024-09-10T08:49:00.711 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.711 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.711 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.711 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "redirect", 2024-09-10T08:49:00.712 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.712 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.712 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.712 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.712 INFO:teuthology.orchestra.run.smithi089.stdout: "redirect" 2024-09-10T08:49:00.712 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.712 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.712 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.712 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.712 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.712 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.712 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_error_status_code": { 2024-09-10T08:49:00.712 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_error_status_code", 2024-09-10T08:49:00.713 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.713 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.713 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.713 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2024-09-10T08:49:00.713 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "400", 2024-09-10T08:49:00.713 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "599", 2024-09-10T08:49:00.713 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.713 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.713 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.713 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.713 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.713 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.713 INFO:teuthology.orchestra.run.smithi089.stdout: "url_prefix": { 2024-09-10T08:49:00.714 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "url_prefix", 2024-09-10T08:49:00.714 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.714 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.714 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.714 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.714 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.714 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.714 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.714 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.714 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.714 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.714 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.714 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.714 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.715 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.715 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.715 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "devicehealth", 2024-09-10T08:49:00.715 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.715 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.715 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.715 INFO:teuthology.orchestra.run.smithi089.stdout: "enable_monitoring": { 2024-09-10T08:49:00.715 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enable_monitoring", 2024-09-10T08:49:00.715 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.715 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.715 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.715 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.715 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.715 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.716 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.716 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "monitor device health metrics", 2024-09-10T08:49:00.716 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.716 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.716 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.716 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.716 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.716 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.716 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.716 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.716 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.716 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.717 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.717 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.717 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.717 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.717 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.717 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.717 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.717 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.717 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.717 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.717 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.717 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.717 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.718 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.718 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.718 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.718 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.718 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.718 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.718 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.718 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.718 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.718 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.718 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.718 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.718 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.719 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.719 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.719 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.719 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.719 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.719 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.719 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.719 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.719 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.719 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.719 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.719 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.719 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.719 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.720 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.720 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.720 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.720 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.720 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.720 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.720 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.720 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.720 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.720 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.720 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.720 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.720 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.720 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.721 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.721 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.721 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.721 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.721 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.721 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.721 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.721 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.721 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.721 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.721 INFO:teuthology.orchestra.run.smithi089.stdout: "mark_out_threshold": { 2024-09-10T08:49:00.721 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mark_out_threshold", 2024-09-10T08:49:00.721 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.722 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.722 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.722 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "2419200", 2024-09-10T08:49:00.722 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.722 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.722 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.722 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T08:49:00.722 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.722 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.722 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.722 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.722 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_name": { 2024-09-10T08:49:00.722 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pool_name", 2024-09-10T08:49:00.722 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.723 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.723 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.723 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "device_health_metrics", 2024-09-10T08:49:00.723 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.723 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.723 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.723 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T08:49:00.723 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.723 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.723 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.723 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.723 INFO:teuthology.orchestra.run.smithi089.stdout: "retention_period": { 2024-09-10T08:49:00.723 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "retention_period", 2024-09-10T08:49:00.723 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.724 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.724 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.724 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15552000", 2024-09-10T08:49:00.724 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.724 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.724 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.724 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long to retain device health metrics", 2024-09-10T08:49:00.724 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.724 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.724 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.724 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.724 INFO:teuthology.orchestra.run.smithi089.stdout: "scrape_frequency": { 2024-09-10T08:49:00.724 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "scrape_frequency", 2024-09-10T08:49:00.725 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.725 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.725 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.725 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "86400", 2024-09-10T08:49:00.725 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.725 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.725 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.725 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T08:49:00.725 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.725 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.725 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.725 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.725 INFO:teuthology.orchestra.run.smithi089.stdout: "self_heal": { 2024-09-10T08:49:00.725 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "self_heal", 2024-09-10T08:49:00.726 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.726 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.726 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.726 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.726 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.726 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.726 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.726 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T08:49:00.726 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.727 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.727 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.727 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.727 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-10T08:49:00.727 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-10T08:49:00.727 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.727 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.727 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.727 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-10T08:49:00.727 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.727 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.727 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.727 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T08:49:00.728 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.728 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.728 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.728 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.728 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_threshold": { 2024-09-10T08:49:00.728 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_threshold", 2024-09-10T08:49:00.728 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.728 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.728 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.729 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7257600", 2024-09-10T08:49:00.729 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.729 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.729 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.729 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T08:49:00.729 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.729 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.729 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.729 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.729 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.729 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.729 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.730 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "diskprediction_local", 2024-09-10T08:49:00.730 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.730 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.730 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.730 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.730 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.730 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.730 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.730 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.731 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.731 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.731 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.731 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.731 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.731 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.731 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.731 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.731 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.731 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.731 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.731 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.731 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.732 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.732 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.732 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.732 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.732 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.732 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.732 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.732 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.732 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.732 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.732 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.732 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.733 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.733 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.733 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.733 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.733 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.733 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.733 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.733 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.733 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.733 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.733 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.733 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.733 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.733 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.734 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.734 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.734 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.734 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.734 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.734 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.734 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.734 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.734 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.734 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.734 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.734 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.734 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.735 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.735 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.735 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.735 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.735 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.735 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.735 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.735 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.735 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.735 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.735 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.735 INFO:teuthology.orchestra.run.smithi089.stdout: "predict_interval": { 2024-09-10T08:49:00.736 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "predict_interval", 2024-09-10T08:49:00.736 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.736 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.736 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.736 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "86400", 2024-09-10T08:49:00.736 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.736 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.736 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.736 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.736 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.736 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.737 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.737 INFO:teuthology.orchestra.run.smithi089.stdout: "predictor_model": { 2024-09-10T08:49:00.737 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "predictor_model", 2024-09-10T08:49:00.737 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.737 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.737 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.737 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "prophetstor", 2024-09-10T08:49:00.737 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.737 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.737 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.737 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.737 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.738 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.738 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.738 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-10T08:49:00.738 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-10T08:49:00.738 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.738 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.738 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.738 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-10T08:49:00.738 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.738 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.738 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.738 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.738 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.739 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.739 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.739 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.739 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.739 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.739 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "influx", 2024-09-10T08:49:00.739 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": false, 2024-09-10T08:49:00.739 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "influxdb python module not found", 2024-09-10T08:49:00.739 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.739 INFO:teuthology.orchestra.run.smithi089.stdout: "batch_size": { 2024-09-10T08:49:00.739 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "batch_size", 2024-09-10T08:49:00.739 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.739 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.740 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.740 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5000", 2024-09-10T08:49:00.740 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.740 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.740 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.740 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T08:49:00.740 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.740 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.740 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.740 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.740 INFO:teuthology.orchestra.run.smithi089.stdout: "database": { 2024-09-10T08:49:00.740 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "database", 2024-09-10T08:49:00.740 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.740 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.741 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.741 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "ceph", 2024-09-10T08:49:00.741 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.741 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.741 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.741 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T08:49:00.741 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.741 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.741 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.741 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.741 INFO:teuthology.orchestra.run.smithi089.stdout: "hostname": { 2024-09-10T08:49:00.741 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "hostname", 2024-09-10T08:49:00.742 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.742 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.742 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.742 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.742 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.742 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.742 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.742 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB server hostname", 2024-09-10T08:49:00.742 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.742 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.742 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.742 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.742 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-10T08:49:00.742 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-10T08:49:00.743 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.743 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.743 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.743 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "30", 2024-09-10T08:49:00.743 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "5", 2024-09-10T08:49:00.743 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.743 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.743 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T08:49:00.743 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.743 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.743 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.743 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.743 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.743 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.744 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.744 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.744 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.744 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.744 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.744 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.744 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.744 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.744 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.744 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.744 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.744 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.745 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.745 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.745 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.745 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.745 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.745 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.745 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.745 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.745 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.745 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.745 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.745 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.746 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.746 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.746 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.746 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.746 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.746 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.746 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.746 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.746 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.746 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.746 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.746 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.747 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.747 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.747 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.747 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.747 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.747 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.747 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.747 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.747 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.747 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.747 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.748 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.748 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.748 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.748 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.748 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.748 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.748 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.748 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.748 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.748 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.748 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.748 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.748 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.749 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.749 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.749 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.749 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.749 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.749 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.749 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.749 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.749 INFO:teuthology.orchestra.run.smithi089.stdout: "password": { 2024-09-10T08:49:00.749 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "password", 2024-09-10T08:49:00.749 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.749 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.749 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.750 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.750 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.750 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.750 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.750 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "password of InfluxDB server user", 2024-09-10T08:49:00.750 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.750 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.750 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.750 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.750 INFO:teuthology.orchestra.run.smithi089.stdout: "port": { 2024-09-10T08:49:00.750 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "port", 2024-09-10T08:49:00.750 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.750 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.751 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.751 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8086", 2024-09-10T08:49:00.751 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.751 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.751 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.751 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB server port", 2024-09-10T08:49:00.751 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.751 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.751 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.751 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.751 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl": { 2024-09-10T08:49:00.752 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl", 2024-09-10T08:49:00.752 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.752 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.752 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.752 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "false", 2024-09-10T08:49:00.752 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.752 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.752 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.753 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T08:49:00.753 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.753 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.753 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.753 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.753 INFO:teuthology.orchestra.run.smithi089.stdout: "threads": { 2024-09-10T08:49:00.753 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "threads", 2024-09-10T08:49:00.753 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.753 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.754 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.754 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-10T08:49:00.754 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-10T08:49:00.754 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "32", 2024-09-10T08:49:00.754 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.754 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T08:49:00.754 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.754 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.754 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.754 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.754 INFO:teuthology.orchestra.run.smithi089.stdout: "username": { 2024-09-10T08:49:00.754 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "username", 2024-09-10T08:49:00.754 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.754 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.755 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.755 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.755 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.755 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.755 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.755 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "username of InfluxDB server user", 2024-09-10T08:49:00.755 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.755 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.755 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.755 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.755 INFO:teuthology.orchestra.run.smithi089.stdout: "verify_ssl": { 2024-09-10T08:49:00.755 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "verify_ssl", 2024-09-10T08:49:00.755 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.755 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.756 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.756 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "true", 2024-09-10T08:49:00.756 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.756 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.756 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.756 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T08:49:00.756 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.756 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.757 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.757 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.757 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.757 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.757 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.757 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "insights", 2024-09-10T08:49:00.757 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.757 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.757 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.757 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.757 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.758 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.758 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.758 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.758 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.758 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.758 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.758 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.758 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.758 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.759 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.759 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.759 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.759 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.759 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.759 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.759 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.760 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.760 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.760 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.760 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.760 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.760 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.761 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.761 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.761 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.761 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.761 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.761 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.761 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.761 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.761 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.761 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.762 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.762 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.762 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.762 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.762 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.762 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.762 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.762 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.762 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.762 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.762 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.762 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.763 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.763 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.763 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.763 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.763 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.763 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.763 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.763 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.763 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.763 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.763 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.763 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.764 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.764 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.764 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.764 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.764 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.764 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.764 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.764 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.764 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.764 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.765 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.765 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.765 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.765 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.765 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.765 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "iostat", 2024-09-10T08:49:00.765 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.765 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.765 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.766 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.766 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.766 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.766 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.766 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.766 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.766 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.766 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.766 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.766 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.766 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.767 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.767 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.767 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.767 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.767 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.767 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.767 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.768 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.768 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.768 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.768 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.768 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.768 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.768 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.768 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.768 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.769 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.769 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.769 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.769 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.769 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.769 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.769 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.770 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.770 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.770 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.770 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.770 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.770 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.770 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.770 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.771 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.771 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.771 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.771 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.771 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.771 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.771 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.772 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.772 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.772 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.772 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.772 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.772 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.772 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.772 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.772 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.773 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.773 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.773 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.773 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.773 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.773 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.773 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.773 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.773 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.773 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.774 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.774 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.774 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.774 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.774 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.774 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "k8sevents", 2024-09-10T08:49:00.774 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.775 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.775 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.775 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_event_retention_days": { 2024-09-10T08:49:00.775 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ceph_event_retention_days", 2024-09-10T08:49:00.775 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.775 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.775 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.775 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7", 2024-09-10T08:49:00.775 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.775 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.775 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.776 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T08:49:00.776 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.776 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.776 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.776 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.776 INFO:teuthology.orchestra.run.smithi089.stdout: "config_check_secs": { 2024-09-10T08:49:00.776 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_check_secs", 2024-09-10T08:49:00.776 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.776 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.776 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.776 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:00.777 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "10", 2024-09-10T08:49:00.777 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.777 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.777 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T08:49:00.777 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.777 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.777 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.777 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.777 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.777 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.777 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.777 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.778 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.778 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.778 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.778 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.778 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.778 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.778 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.779 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.779 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.779 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.779 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.779 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.779 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.779 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.780 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.780 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.780 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.780 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.780 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.780 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.781 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.781 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.781 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.781 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.781 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.781 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.781 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.784 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.784 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.785 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.785 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.785 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.785 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.785 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.785 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.785 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.786 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.786 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.786 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.786 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.786 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.786 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.786 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.786 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.786 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.786 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.787 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.787 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.787 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.787 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.787 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.787 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.787 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.787 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.787 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.787 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.788 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.788 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.788 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.788 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.788 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.788 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.788 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.788 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.788 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.788 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.788 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.789 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.789 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.789 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "localpool", 2024-09-10T08:49:00.789 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.789 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.789 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.789 INFO:teuthology.orchestra.run.smithi089.stdout: "failure_domain": { 2024-09-10T08:49:00.789 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "failure_domain", 2024-09-10T08:49:00.789 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.789 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.790 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.790 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "host", 2024-09-10T08:49:00.790 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.790 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.790 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.790 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "failure domain for any created local pool", 2024-09-10T08:49:00.790 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T08:49:00.790 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.790 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.790 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.791 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.791 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.791 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.791 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.791 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.791 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.791 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.791 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.791 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.791 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.791 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.792 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.792 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.792 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.792 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.792 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.792 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.792 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.792 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.792 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.792 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.793 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.793 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.793 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.793 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.793 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.793 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.793 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.793 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.793 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.793 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.793 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.793 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.794 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.794 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.794 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.794 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.794 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.794 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.794 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.794 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.794 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.794 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.794 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.795 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.795 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.795 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.795 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.795 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.795 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.795 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.795 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.795 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.795 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.795 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.796 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.796 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.796 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.796 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.796 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.796 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.796 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.796 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.796 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.796 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.797 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.797 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.797 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.797 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.797 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.797 INFO:teuthology.orchestra.run.smithi089.stdout: "min_size": { 2024-09-10T08:49:00.797 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "min_size", 2024-09-10T08:49:00.797 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.797 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.797 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.797 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.797 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.798 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.798 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.798 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default min_size for any created local pool", 2024-09-10T08:49:00.798 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T08:49:00.798 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.798 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.798 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.798 INFO:teuthology.orchestra.run.smithi089.stdout: "num_rep": { 2024-09-10T08:49:00.798 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "num_rep", 2024-09-10T08:49:00.798 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.798 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.799 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.799 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3", 2024-09-10T08:49:00.799 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.799 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.799 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.799 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default replica count for any created local pool", 2024-09-10T08:49:00.799 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.799 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.799 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.799 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.800 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num": { 2024-09-10T08:49:00.800 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pg_num", 2024-09-10T08:49:00.800 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.800 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.800 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.801 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "128", 2024-09-10T08:49:00.801 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.801 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.801 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.801 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T08:49:00.801 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.802 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.802 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.802 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.802 INFO:teuthology.orchestra.run.smithi089.stdout: "prefix": { 2024-09-10T08:49:00.802 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prefix", 2024-09-10T08:49:00.802 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.802 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.802 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.803 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.803 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.803 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.803 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.803 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "name prefix for any created local pool", 2024-09-10T08:49:00.803 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.803 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.804 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.804 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.804 INFO:teuthology.orchestra.run.smithi089.stdout: "subtree": { 2024-09-10T08:49:00.804 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "subtree", 2024-09-10T08:49:00.804 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.804 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.804 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.805 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "rack", 2024-09-10T08:49:00.805 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.805 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.805 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.805 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T08:49:00.805 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T08:49:00.805 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.806 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.806 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.806 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.806 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.806 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.806 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mds_autoscaler", 2024-09-10T08:49:00.807 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.807 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.807 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.807 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.807 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.807 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.807 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.807 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.808 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.808 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.808 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.808 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.808 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.808 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.808 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.808 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.808 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.809 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.809 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.809 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.809 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.809 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.809 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.809 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.809 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.810 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.810 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.810 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.810 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.810 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.810 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.810 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.810 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.810 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.810 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.810 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.811 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.811 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.811 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.811 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.811 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.811 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.811 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.811 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.811 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.811 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.811 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.811 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.812 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.812 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.812 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.812 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.812 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.812 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.812 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.812 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.812 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.812 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.813 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.813 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.813 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.813 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.813 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.813 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.813 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.813 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.813 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.813 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.814 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.814 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.814 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.814 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.814 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.814 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.814 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.814 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.814 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mirroring", 2024-09-10T08:49:00.814 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.814 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.815 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.815 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.815 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.815 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.815 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.815 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.815 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.816 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.816 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.816 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.816 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.816 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.816 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.816 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.816 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.817 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.817 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.817 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.817 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.817 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.817 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.817 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.818 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.818 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.818 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.818 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.818 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.818 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.818 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.819 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.819 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.819 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.819 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.819 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.819 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.819 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.820 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.820 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.820 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.820 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.820 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.820 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.820 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.820 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.820 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.821 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.821 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.821 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.821 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.821 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.821 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.821 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.821 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.821 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.821 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.821 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.821 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.822 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.822 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.822 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.822 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.822 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.822 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.822 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.822 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.822 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.822 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.823 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.823 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.823 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.823 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.823 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.823 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "nfs", 2024-09-10T08:49:00.823 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.824 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.824 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.824 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.824 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.824 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.824 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.824 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.824 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.824 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.825 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.825 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.825 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.825 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.825 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.825 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.825 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.825 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.825 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.826 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.826 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.826 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.826 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.826 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.826 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.826 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.827 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.827 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.827 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.827 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.827 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.827 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.827 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.827 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.828 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.828 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.828 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.828 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.828 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.828 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.828 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.829 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.829 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.829 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.829 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.829 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.829 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.829 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.830 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.830 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.830 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.830 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.830 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.830 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.830 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.831 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.831 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.831 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.831 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.831 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.831 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.831 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.832 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.832 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.832 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.832 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.832 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.832 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.832 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.832 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.832 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.833 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.833 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.833 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.833 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "orchestrator", 2024-09-10T08:49:00.833 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.833 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.834 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.834 INFO:teuthology.orchestra.run.smithi089.stdout: "fail_fs": { 2024-09-10T08:49:00.834 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "fail_fs", 2024-09-10T08:49:00.834 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.834 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.834 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.834 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.834 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.834 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.834 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.834 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T08:49:00.834 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.835 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.835 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.835 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.835 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.835 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.835 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.835 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.835 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.835 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.836 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.836 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.836 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.836 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.836 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.836 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.836 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.837 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.837 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.837 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.837 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.837 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.837 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.837 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.838 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.838 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.838 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.838 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.838 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.838 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.838 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.838 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.838 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.839 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.839 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.839 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.839 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.839 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.839 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.839 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.840 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.840 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.840 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.840 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.840 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.840 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.840 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.840 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.840 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.840 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.841 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.841 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.841 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.841 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.841 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.841 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.841 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.842 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.842 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.842 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.842 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.842 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.842 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.842 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.842 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.842 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.843 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.843 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.843 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.843 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.843 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator": { 2024-09-10T08:49:00.843 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "orchestrator", 2024-09-10T08:49:00.843 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.843 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.843 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.844 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.844 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.844 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.844 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.844 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm", 2024-09-10T08:49:00.844 INFO:teuthology.orchestra.run.smithi089.stdout: "rook", 2024-09-10T08:49:00.844 INFO:teuthology.orchestra.run.smithi089.stdout: "test_orchestrator" 2024-09-10T08:49:00.844 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.845 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Orchestrator backend", 2024-09-10T08:49:00.845 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.845 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.845 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.845 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.845 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.845 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.846 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "osd_perf_query", 2024-09-10T08:49:00.846 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.846 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.846 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.846 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.846 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.846 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.847 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.847 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.847 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.847 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.847 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.847 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.847 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.847 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.847 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.848 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.848 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.848 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.848 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.848 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.848 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.848 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.849 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.849 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.849 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.849 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.849 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.849 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.849 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.849 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.849 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.849 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.850 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.850 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.850 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.850 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.850 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.850 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.850 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.850 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.850 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.851 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.851 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.851 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.851 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.851 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.851 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.851 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.851 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.852 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.852 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.852 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.852 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.852 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.852 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.852 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.853 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.853 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.853 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.853 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.853 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.853 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.854 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.854 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.854 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.854 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.854 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.854 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.854 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.855 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.855 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.855 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.855 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.855 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.855 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.855 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "osd_support", 2024-09-10T08:49:00.855 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.855 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.855 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.856 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.856 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.856 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.856 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.856 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.856 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.856 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.857 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.857 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.857 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.857 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.857 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.857 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.857 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.857 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.858 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.858 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.858 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.858 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.858 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.858 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.858 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.859 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.859 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.859 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.859 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.859 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.859 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.859 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.859 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.859 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.859 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.860 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.860 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.860 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.860 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.860 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.860 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.860 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.861 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.861 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.861 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.861 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.861 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.861 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.861 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.861 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.861 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.862 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.862 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.862 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.862 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.862 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.862 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.862 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.863 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.863 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.863 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.863 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.863 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.863 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.863 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.863 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.863 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.863 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.864 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.864 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.864 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.864 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.864 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.864 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pg_autoscaler", 2024-09-10T08:49:00.864 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.864 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.865 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.865 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.865 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.865 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.865 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.865 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.865 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.865 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.865 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.865 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.866 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.866 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.866 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.866 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.866 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.866 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.866 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.866 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.866 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.866 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.867 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.867 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.867 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.867 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.867 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.867 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.867 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.867 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.867 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.868 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.868 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.868 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.868 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.868 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.868 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.868 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.868 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.868 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.868 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.868 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.869 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.869 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.869 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.869 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.869 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.869 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.869 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.869 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.869 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.869 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.869 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.870 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.870 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.870 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.870 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.870 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.870 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.870 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.870 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.871 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.871 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.871 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.871 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.871 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.871 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.871 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.871 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.871 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.872 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.872 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.872 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-10T08:49:00.872 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-10T08:49:00.872 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.872 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.872 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.872 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-10T08:49:00.873 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.873 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.873 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.873 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.873 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.873 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.873 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.873 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.873 INFO:teuthology.orchestra.run.smithi089.stdout: "threshold": { 2024-09-10T08:49:00.873 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "threshold", 2024-09-10T08:49:00.873 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:00.874 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.874 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.874 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3.0", 2024-09-10T08:49:00.874 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1.0", 2024-09-10T08:49:00.874 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.874 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.874 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "scaling threshold", 2024-09-10T08:49:00.874 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T08:49:00.874 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.874 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.874 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.875 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.875 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.875 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.875 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "progress", 2024-09-10T08:49:00.875 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.875 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.875 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.875 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_pg_recovery_event": { 2024-09-10T08:49:00.875 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_pg_recovery_event", 2024-09-10T08:49:00.875 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.876 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.876 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.876 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.876 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.876 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.876 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.876 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T08:49:00.876 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.876 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.877 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.877 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.877 INFO:teuthology.orchestra.run.smithi089.stdout: "enabled": { 2024-09-10T08:49:00.877 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enabled", 2024-09-10T08:49:00.877 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.877 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.877 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.877 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.877 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.877 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.878 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.878 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.878 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.878 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.878 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.878 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.878 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.878 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.878 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.878 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.878 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.879 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.879 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.879 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.879 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.879 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.879 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.879 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.879 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.879 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.879 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.879 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.879 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.880 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.880 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.880 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.880 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.880 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.880 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.880 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.880 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.880 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.880 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.880 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.881 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.881 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.881 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.881 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.881 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.881 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.881 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.881 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.881 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.881 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.882 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.882 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.882 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.882 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.882 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.882 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.882 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.882 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.882 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.882 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.883 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.883 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.883 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.883 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.883 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.883 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.883 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.883 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.884 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.884 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.884 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.884 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.884 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.884 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.884 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.885 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.885 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.885 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.885 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.885 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.885 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.885 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.885 INFO:teuthology.orchestra.run.smithi089.stdout: "max_completed_events": { 2024-09-10T08:49:00.885 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_completed_events", 2024-09-10T08:49:00.885 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.886 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.886 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.886 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "50", 2024-09-10T08:49:00.886 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.886 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.886 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.886 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "number of past completed events to remember", 2024-09-10T08:49:00.886 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.887 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.887 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.887 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-10T08:49:00.887 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-10T08:49:00.887 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.887 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.887 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.888 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-10T08:49:00.888 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.888 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.888 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.888 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long the module is going to sleep", 2024-09-10T08:49:00.888 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.888 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.888 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.888 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.889 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.889 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.889 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.889 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prometheus", 2024-09-10T08:49:00.889 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.889 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.889 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.889 INFO:teuthology.orchestra.run.smithi089.stdout: "cache": { 2024-09-10T08:49:00.889 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cache", 2024-09-10T08:49:00.890 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.890 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.890 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.890 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.890 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.890 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.890 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.890 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.891 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.891 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.891 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.891 INFO:teuthology.orchestra.run.smithi089.stdout: "exclude_perf_counters": { 2024-09-10T08:49:00.891 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "exclude_perf_counters", 2024-09-10T08:49:00.891 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.891 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.891 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.891 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.891 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.892 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.892 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.892 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T08:49:00.892 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T08:49:00.892 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.892 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.892 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.892 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.892 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.892 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.893 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.893 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.893 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.893 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.893 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.893 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.893 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.893 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.893 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.893 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.893 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.893 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.893 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.894 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.894 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.894 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.894 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.894 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.894 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.894 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.894 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.894 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.895 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.895 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.895 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.895 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.895 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.895 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.895 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.895 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.895 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.895 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.896 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.896 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.896 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.896 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.896 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.896 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.896 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.896 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.896 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.896 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.896 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.896 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.896 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.897 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.897 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.897 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.897 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.897 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.897 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.897 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.897 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.897 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.897 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.897 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.897 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.897 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.898 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.898 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.898 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.898 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.898 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.898 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_stats_pools": { 2024-09-10T08:49:00.898 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_stats_pools", 2024-09-10T08:49:00.898 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.898 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.898 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.898 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.898 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.899 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.899 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.899 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.899 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.899 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.899 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.899 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T08:49:00.899 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T08:49:00.899 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.899 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.899 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.899 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "300", 2024-09-10T08:49:00.899 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.900 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.900 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.900 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.900 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.900 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.900 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.900 INFO:teuthology.orchestra.run.smithi089.stdout: "scrape_interval": { 2024-09-10T08:49:00.900 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "scrape_interval", 2024-09-10T08:49:00.900 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:00.900 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.900 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.901 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15.0", 2024-09-10T08:49:00.901 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.901 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.901 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.901 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.901 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.901 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.901 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.901 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2024-09-10T08:49:00.901 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2024-09-10T08:49:00.901 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.901 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.901 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.902 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "::", 2024-09-10T08:49:00.902 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.902 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.902 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.902 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T08:49:00.902 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.902 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.902 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.902 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2024-09-10T08:49:00.902 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2024-09-10T08:49:00.902 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.902 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.903 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.903 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "9283", 2024-09-10T08:49:00.903 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.903 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.903 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.903 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T08:49:00.903 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.903 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.903 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.903 INFO:teuthology.orchestra.run.smithi089.stdout: "stale_cache_strategy": { 2024-09-10T08:49:00.903 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "stale_cache_strategy", 2024-09-10T08:49:00.903 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.903 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.904 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.904 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "log", 2024-09-10T08:49:00.904 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.904 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.904 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.904 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.904 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.904 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.904 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.904 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_behaviour": { 2024-09-10T08:49:00.904 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_behaviour", 2024-09-10T08:49:00.904 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.905 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.905 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.905 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "default", 2024-09-10T08:49:00.905 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.905 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.905 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.905 INFO:teuthology.orchestra.run.smithi089.stdout: "default", 2024-09-10T08:49:00.905 INFO:teuthology.orchestra.run.smithi089.stdout: "error" 2024-09-10T08:49:00.905 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.905 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.905 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.905 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.905 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.906 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_error_status_code": { 2024-09-10T08:49:00.906 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_error_status_code", 2024-09-10T08:49:00.906 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.906 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.906 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.906 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2024-09-10T08:49:00.906 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "400", 2024-09-10T08:49:00.906 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "599", 2024-09-10T08:49:00.906 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.906 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.906 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.906 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.906 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.907 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.907 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.907 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.907 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_support", 2024-09-10T08:49:00.907 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.907 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.907 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.907 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.907 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.907 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.907 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.907 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.908 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.908 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.908 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.908 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.908 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.908 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.908 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.908 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.908 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.908 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.908 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.908 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.908 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.909 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.909 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.909 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.909 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.909 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.909 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.909 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.909 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.909 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.909 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.909 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.910 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.910 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.910 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.910 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.910 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.910 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.910 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.910 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.910 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.910 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.910 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.910 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.910 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.911 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.911 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.911 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.911 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.911 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.911 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.911 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.911 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.911 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.911 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.911 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.911 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.912 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.912 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.912 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.912 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.912 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.912 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.912 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.912 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.912 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.912 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.912 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.912 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.913 INFO:teuthology.orchestra.run.smithi089.stdout: "max_concurrent_snap_create": { 2024-09-10T08:49:00.913 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_concurrent_snap_create", 2024-09-10T08:49:00.913 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.913 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.913 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.913 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:00.913 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.913 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.913 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.913 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.913 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.913 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.913 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.914 INFO:teuthology.orchestra.run.smithi089.stdout: "mirror_snapshot_schedule": { 2024-09-10T08:49:00.914 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T08:49:00.914 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.914 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.914 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.914 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.914 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.914 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.914 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.914 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.914 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.915 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.915 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.915 INFO:teuthology.orchestra.run.smithi089.stdout: "trash_purge_schedule": { 2024-09-10T08:49:00.915 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "trash_purge_schedule", 2024-09-10T08:49:00.915 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.915 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.915 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.915 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.915 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.916 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.916 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.916 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.916 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.916 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.916 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.916 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.916 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.916 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.916 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.916 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "restful", 2024-09-10T08:49:00.916 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.916 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.916 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.917 INFO:teuthology.orchestra.run.smithi089.stdout: "enable_auth": { 2024-09-10T08:49:00.917 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enable_auth", 2024-09-10T08:49:00.917 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.917 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.917 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.917 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.917 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.917 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.917 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.917 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.917 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.917 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.917 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.918 INFO:teuthology.orchestra.run.smithi089.stdout: "key_file": { 2024-09-10T08:49:00.918 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "key_file", 2024-09-10T08:49:00.918 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.918 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.918 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.918 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.918 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.918 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.918 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.918 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.918 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.918 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.918 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.919 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.919 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.919 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.919 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.919 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.919 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.919 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.919 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.919 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.919 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.919 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.919 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.920 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.920 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.920 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.920 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.920 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.920 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.920 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.920 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.920 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.920 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.920 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.920 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.920 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.920 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.921 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.921 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.921 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.921 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.921 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.921 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.921 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.921 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.921 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.921 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.921 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.921 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.922 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.922 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.922 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.922 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.922 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.922 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.922 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.922 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.922 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.922 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.922 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.922 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.922 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.922 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.922 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.923 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.923 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.923 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.923 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.923 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.923 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.923 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.923 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.923 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.923 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.923 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.923 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.923 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.924 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.924 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.924 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.924 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2024-09-10T08:49:00.924 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2024-09-10T08:49:00.924 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.924 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.924 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.924 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.924 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.924 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.924 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.925 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.925 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.925 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.925 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.925 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2024-09-10T08:49:00.925 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2024-09-10T08:49:00.925 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.925 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.925 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.925 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.925 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.926 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.926 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.926 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.926 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.926 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.926 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.926 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.926 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.926 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.926 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rgw", 2024-09-10T08:49:00.926 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.926 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.926 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.927 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.927 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.927 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.927 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.927 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.927 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.927 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.927 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.927 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.927 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.927 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.927 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.927 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.927 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.928 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.928 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.928 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.928 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.928 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.928 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.928 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.928 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.928 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.928 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.928 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.928 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.928 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.928 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.929 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.929 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.929 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.929 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.929 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.929 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.929 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.929 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.929 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.929 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.929 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.929 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.929 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.930 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.930 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.930 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.930 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.930 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.930 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.930 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.930 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.930 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.930 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.930 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.930 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.930 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.930 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.931 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.931 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.931 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.931 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.931 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.931 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.931 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.931 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.931 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.931 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.931 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.931 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.931 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.932 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.932 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.932 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.932 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.932 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rook", 2024-09-10T08:49:00.932 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.932 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.932 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.932 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.932 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.932 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.932 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.932 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.933 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.933 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.933 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.933 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.933 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.933 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.933 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.933 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.933 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.933 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.933 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.933 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.933 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.933 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.934 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.934 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.934 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.934 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.934 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.934 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.934 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.934 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.934 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.934 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.934 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.934 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.934 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.935 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.935 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.935 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.935 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.935 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.935 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.935 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.935 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.935 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.935 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.935 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.935 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.936 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.936 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.936 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.936 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.936 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.936 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.936 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.936 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.936 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.936 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.936 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.937 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.937 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.937 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.937 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.937 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.937 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.937 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.937 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.937 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.937 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.937 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.937 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.937 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.937 INFO:teuthology.orchestra.run.smithi089.stdout: "storage_class": { 2024-09-10T08:49:00.938 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "storage_class", 2024-09-10T08:49:00.938 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.938 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.938 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.938 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "local", 2024-09-10T08:49:00.938 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.938 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.938 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.938 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T08:49:00.938 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.938 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.938 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.938 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.938 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.939 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.939 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "selftest", 2024-09-10T08:49:00.939 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.939 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.939 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.939 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.939 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.939 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.939 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.939 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.939 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.940 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.940 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.940 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.940 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.940 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.940 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.940 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.940 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.940 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.940 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.940 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.940 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.940 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.941 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.941 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.941 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.941 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.941 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.941 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.941 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.941 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.941 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.941 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.941 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.941 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.941 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.942 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.942 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.942 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.942 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.942 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.942 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.942 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.942 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.942 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.942 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.942 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.942 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.942 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.943 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.943 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.943 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.943 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.943 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.943 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.943 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.943 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.943 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.943 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.943 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.943 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.943 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.944 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.944 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.944 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.944 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.944 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.944 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.944 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.944 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.944 INFO:teuthology.orchestra.run.smithi089.stdout: "roption1": { 2024-09-10T08:49:00.944 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "roption1", 2024-09-10T08:49:00.944 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.945 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.945 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.945 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.945 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.945 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.945 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.945 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.945 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.945 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.945 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.945 INFO:teuthology.orchestra.run.smithi089.stdout: "roption2": { 2024-09-10T08:49:00.945 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "roption2", 2024-09-10T08:49:00.946 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.946 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.946 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.946 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "xyz", 2024-09-10T08:49:00.946 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.946 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.946 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.946 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.946 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.946 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.946 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.946 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption1": { 2024-09-10T08:49:00.946 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption1", 2024-09-10T08:49:00.947 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.947 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.947 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.947 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.947 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.947 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.947 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.947 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.947 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.947 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.947 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.947 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption2": { 2024-09-10T08:49:00.947 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption2", 2024-09-10T08:49:00.948 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.948 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.948 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.948 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.948 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.948 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.948 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.948 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.948 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.948 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.948 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.948 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption3": { 2024-09-10T08:49:00.949 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption3", 2024-09-10T08:49:00.949 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:00.949 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.949 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.949 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.949 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.949 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.949 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.949 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.949 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.949 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.949 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.950 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption4": { 2024-09-10T08:49:00.950 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption4", 2024-09-10T08:49:00.950 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.950 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.950 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.950 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.950 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.950 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.950 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.950 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.950 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.950 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.950 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.951 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption5": { 2024-09-10T08:49:00.951 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption5", 2024-09-10T08:49:00.951 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.951 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.951 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.951 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.951 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.951 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.951 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.951 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.951 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.951 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.951 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.952 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.952 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption6": { 2024-09-10T08:49:00.952 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption6", 2024-09-10T08:49:00.952 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.952 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.952 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.952 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.952 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.952 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.952 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.952 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.952 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.952 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.953 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.953 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption7": { 2024-09-10T08:49:00.953 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption7", 2024-09-10T08:49:00.953 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.953 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.953 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.953 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.953 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-10T08:49:00.953 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "42", 2024-09-10T08:49:00.953 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.953 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.953 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.954 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.954 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.954 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.954 INFO:teuthology.orchestra.run.smithi089.stdout: "testkey": { 2024-09-10T08:49:00.954 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testkey", 2024-09-10T08:49:00.954 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.954 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.954 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.954 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.954 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.954 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.954 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.954 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.955 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.955 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.955 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.955 INFO:teuthology.orchestra.run.smithi089.stdout: "testlkey": { 2024-09-10T08:49:00.955 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testlkey", 2024-09-10T08:49:00.955 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.955 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.955 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.955 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.955 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.955 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.955 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.955 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.956 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.956 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.956 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.956 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.956 INFO:teuthology.orchestra.run.smithi089.stdout: "testnewline": { 2024-09-10T08:49:00.956 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testnewline", 2024-09-10T08:49:00.956 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.956 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.956 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.956 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.956 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.956 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.957 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.957 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.957 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.957 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.957 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.957 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.957 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.957 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.957 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snap_schedule", 2024-09-10T08:49:00.957 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.957 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.957 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.957 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_m_granularity": { 2024-09-10T08:49:00.958 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_m_granularity", 2024-09-10T08:49:00.958 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.958 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.958 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.960 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.960 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.960 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.960 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.961 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T08:49:00.961 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.961 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.961 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.961 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.961 INFO:teuthology.orchestra.run.smithi089.stdout: "dump_on_update": { 2024-09-10T08:49:00.961 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "dump_on_update", 2024-09-10T08:49:00.961 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.961 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.961 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.961 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.961 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.961 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.962 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.962 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "dump database to debug log on update", 2024-09-10T08:49:00.962 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.962 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.962 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.962 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.962 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.962 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.962 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.962 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.962 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.962 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.962 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.962 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.963 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.963 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.963 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.963 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.963 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.963 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.963 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.963 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.963 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.963 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.963 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.963 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.963 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.963 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.964 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.964 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.964 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.964 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.964 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.964 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.964 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.964 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.964 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.964 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.964 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.964 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.964 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.964 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.965 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.965 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.965 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.965 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.965 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.965 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.965 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.965 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.965 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.965 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.965 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.965 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.966 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.966 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.966 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.966 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.966 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.966 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.966 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.966 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.966 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.966 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.966 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.966 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.967 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.967 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.967 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.967 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.967 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.967 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.967 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.967 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.967 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.967 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.967 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.967 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.967 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.968 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "stats", 2024-09-10T08:49:00.968 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.968 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.968 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.968 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.968 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.968 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.968 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.968 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.968 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.968 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.968 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.968 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.969 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.969 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.969 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.969 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.969 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.969 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.969 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.969 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.969 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.969 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.969 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.969 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.969 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.969 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.970 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.970 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.970 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.970 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.970 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.970 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.970 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.970 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.970 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.970 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.970 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.970 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.970 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.971 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.971 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.971 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.971 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.971 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.971 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.971 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.971 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.971 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.971 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.971 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.971 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.971 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.972 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.972 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.972 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.972 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.972 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.972 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.972 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.972 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.972 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.972 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.972 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.972 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.972 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.973 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.973 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.973 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.973 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.973 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.973 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.973 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.973 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.973 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.973 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.973 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.973 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "status", 2024-09-10T08:49:00.973 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.974 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.974 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.974 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.974 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.974 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.974 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.974 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.974 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.974 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.974 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.974 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.974 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.974 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.975 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.975 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.975 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.975 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.975 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.975 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.975 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.975 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.975 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.975 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.975 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.975 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.975 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.975 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.975 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.976 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.976 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.976 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.976 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.976 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.976 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.976 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.976 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.976 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.976 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.976 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.976 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.976 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.976 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.977 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.977 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.977 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.977 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.977 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.977 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.977 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.977 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.977 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.977 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.977 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.977 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.977 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.977 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.978 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.978 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.978 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.978 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.978 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.978 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.978 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.978 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.978 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.978 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.978 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.978 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.978 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.978 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.979 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.979 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.979 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.979 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.979 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.979 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "telegraf", 2024-09-10T08:49:00.979 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.979 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.979 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.979 INFO:teuthology.orchestra.run.smithi089.stdout: "address": { 2024-09-10T08:49:00.979 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "address", 2024-09-10T08:49:00.979 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.980 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.980 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.980 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T08:49:00.980 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.980 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.980 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.980 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.980 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.980 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.980 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.980 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.980 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-10T08:49:00.980 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-10T08:49:00.980 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:00.981 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.981 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.981 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15", 2024-09-10T08:49:00.981 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.981 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.981 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.981 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.981 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.981 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.981 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.981 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.981 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:00.981 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:00.981 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.982 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.982 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.982 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.982 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.982 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.982 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.982 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.982 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.982 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.982 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.982 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.982 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.982 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.982 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.983 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.983 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.983 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.983 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.983 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:00.983 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:00.983 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.983 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.983 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.983 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.983 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.983 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.984 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.984 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.984 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.984 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.984 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.984 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:00.984 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:00.984 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.984 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.984 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.984 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:00.984 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.985 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.985 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:00.985 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:00.985 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:00.985 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:00.985 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:00.985 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:00.985 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:00.985 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:00.985 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.985 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.985 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.986 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.986 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:00.986 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:00.986 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.986 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.986 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:00.986 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.986 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.986 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.986 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.986 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.986 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.987 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.987 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.987 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.987 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:00.987 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.987 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:00.987 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "telemetry", 2024-09-10T08:49:00.987 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:00.987 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:00.987 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:00.987 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_basic": { 2024-09-10T08:49:00.987 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_basic", 2024-09-10T08:49:00.987 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.988 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.988 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.988 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.988 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.988 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.988 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.988 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T08:49:00.988 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.988 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.988 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.988 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.988 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_crash": { 2024-09-10T08:49:00.989 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_crash", 2024-09-10T08:49:00.989 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.989 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.989 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.989 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.989 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.989 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.989 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.989 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T08:49:00.989 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.989 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.989 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.989 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.990 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_device": { 2024-09-10T08:49:00.990 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_device", 2024-09-10T08:49:00.990 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.990 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.990 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.990 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:00.990 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.990 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.990 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.990 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T08:49:00.990 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.990 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.990 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.991 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.991 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_ident": { 2024-09-10T08:49:00.991 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_ident", 2024-09-10T08:49:00.991 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.991 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.991 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.991 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.991 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.991 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.991 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.991 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T08:49:00.991 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.991 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.992 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.992 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_perf": { 2024-09-10T08:49:00.992 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_perf", 2024-09-10T08:49:00.992 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.992 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.992 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.992 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.992 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.992 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.992 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.992 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T08:49:00.993 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.993 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.993 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.993 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.993 INFO:teuthology.orchestra.run.smithi089.stdout: "contact": { 2024-09-10T08:49:00.993 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "contact", 2024-09-10T08:49:00.993 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.993 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.993 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.993 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.993 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.993 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.993 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.994 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.994 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.994 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.994 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.994 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.994 INFO:teuthology.orchestra.run.smithi089.stdout: "description": { 2024-09-10T08:49:00.994 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "description", 2024-09-10T08:49:00.994 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.994 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.994 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.994 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:00.994 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.994 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.994 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.995 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.995 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.995 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.995 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.995 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.995 INFO:teuthology.orchestra.run.smithi089.stdout: "device_url": { 2024-09-10T08:49:00.995 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_url", 2024-09-10T08:49:00.995 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:00.995 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.995 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.995 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T08:49:00.996 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.996 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.996 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.996 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.996 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.996 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.996 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.996 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.996 INFO:teuthology.orchestra.run.smithi089.stdout: "enabled": { 2024-09-10T08:49:00.996 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enabled", 2024-09-10T08:49:00.996 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:00.996 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.997 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.997 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:00.997 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.997 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.997 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.997 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.997 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.997 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.997 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.997 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.997 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-10T08:49:00.997 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-10T08:49:00.998 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.998 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.998 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.998 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "24", 2024-09-10T08:49:00.998 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "8", 2024-09-10T08:49:00.998 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.998 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.998 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.998 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.998 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:00.998 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:00.998 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:00.999 INFO:teuthology.orchestra.run.smithi089.stdout: "last_opt_revision": { 2024-09-10T08:49:00.999 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "last_opt_revision", 2024-09-10T08:49:00.999 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:00.999 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:00.999 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:00.999 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1", 2024-09-10T08:49:00.999 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:00.999 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:00.999 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:00.999 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:00.999 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:00.999 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.000 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.000 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.000 INFO:teuthology.orchestra.run.smithi089.stdout: "leaderboard": { 2024-09-10T08:49:01.000 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "leaderboard", 2024-09-10T08:49:01.000 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.000 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.000 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.000 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.000 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.000 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.000 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.000 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.001 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.001 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.001 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.001 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.001 INFO:teuthology.orchestra.run.smithi089.stdout: "leaderboard_description": { 2024-09-10T08:49:01.001 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "leaderboard_description", 2024-09-10T08:49:01.001 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.001 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.001 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.001 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.001 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.001 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.001 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.001 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.002 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.002 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.002 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.002 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.002 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.002 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.002 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.002 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.002 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.002 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.002 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.002 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.002 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.003 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.003 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.003 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.003 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.003 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.003 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.003 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.003 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.003 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.003 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.003 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.003 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.003 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.003 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.004 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.004 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.004 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.004 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.004 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.004 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.004 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.004 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.004 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.004 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.004 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.004 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.004 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.005 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.005 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.005 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.005 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.005 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.005 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.005 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.005 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.005 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.005 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.005 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.005 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.005 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.005 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.006 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.006 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.006 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.006 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.006 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.006 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.006 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.006 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.006 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.006 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.006 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.006 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.006 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.006 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.007 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.007 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.007 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.007 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.007 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.007 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.007 INFO:teuthology.orchestra.run.smithi089.stdout: "organization": { 2024-09-10T08:49:01.007 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "organization", 2024-09-10T08:49:01.007 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.007 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.007 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.007 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.007 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.007 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.008 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.008 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.008 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.008 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.008 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.008 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.008 INFO:teuthology.orchestra.run.smithi089.stdout: "proxy": { 2024-09-10T08:49:01.008 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "proxy", 2024-09-10T08:49:01.008 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.008 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.008 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.008 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.008 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.009 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.009 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.009 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.009 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.009 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.009 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.009 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.009 INFO:teuthology.orchestra.run.smithi089.stdout: "url": { 2024-09-10T08:49:01.009 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "url", 2024-09-10T08:49:01.009 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.009 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.009 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.009 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T08:49:01.009 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.010 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.010 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.010 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.010 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.010 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.010 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.010 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.010 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.010 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.010 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.010 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "test_orchestrator", 2024-09-10T08:49:01.010 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.010 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.011 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.011 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.011 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.011 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.011 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.011 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.011 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.011 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.011 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.011 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.011 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.011 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.011 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.011 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.012 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.012 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.012 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.012 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.012 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.012 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.012 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.012 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.012 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.012 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.012 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.012 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.012 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.012 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.013 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.013 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.013 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.013 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.013 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.013 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.013 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.013 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.013 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.013 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.013 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.013 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.014 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.014 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.014 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.014 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.014 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.014 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.014 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.014 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.014 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.014 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.014 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.014 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.014 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.014 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.015 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.015 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.015 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.015 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.015 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.015 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.015 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.015 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.015 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.015 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.015 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.015 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.015 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.016 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.016 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.016 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.016 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.016 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.016 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.016 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.016 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "volumes", 2024-09-10T08:49:01.016 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.016 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.016 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.016 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.016 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.016 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.017 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.017 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.017 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.017 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.017 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.017 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.017 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.017 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.017 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.017 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.017 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.017 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.017 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.017 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.018 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.018 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.018 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.018 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.018 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.018 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.018 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.018 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.018 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.018 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.018 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.018 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.018 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.019 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.019 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.019 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.019 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.019 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.019 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.019 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.019 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.019 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.019 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.019 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.019 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.019 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.019 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.019 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.020 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.020 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.020 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.020 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.020 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.020 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.020 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.020 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.020 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.020 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.020 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.020 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.020 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.020 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.021 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.021 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.021 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.021 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.021 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.021 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.021 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.021 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.021 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.021 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.021 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.021 INFO:teuthology.orchestra.run.smithi089.stdout: "max_concurrent_clones": { 2024-09-10T08:49:01.021 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_concurrent_clones", 2024-09-10T08:49:01.022 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.022 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.022 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.022 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "4", 2024-09-10T08:49:01.022 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.022 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.022 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.022 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T08:49:01.022 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.022 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.022 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.022 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.022 INFO:teuthology.orchestra.run.smithi089.stdout: "periodic_async_work": { 2024-09-10T08:49:01.023 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "periodic_async_work", 2024-09-10T08:49:01.023 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.023 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.023 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.023 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.023 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.023 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.023 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.023 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Periodically check for async work", 2024-09-10T08:49:01.023 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.023 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.023 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.023 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.023 INFO:teuthology.orchestra.run.smithi089.stdout: "snapshot_clone_delay": { 2024-09-10T08:49:01.023 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snapshot_clone_delay", 2024-09-10T08:49:01.024 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.024 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.024 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.024 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-10T08:49:01.024 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.024 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.024 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.024 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T08:49:01.024 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.024 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.024 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.024 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.025 INFO:teuthology.orchestra.run.smithi089.stdout: "snapshot_clone_no_wait": { 2024-09-10T08:49:01.025 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T08:49:01.025 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.025 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.025 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.025 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.025 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.025 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.025 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.025 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T08:49:01.025 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.025 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.025 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.026 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.026 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.026 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.026 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.026 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix", 2024-09-10T08:49:01.026 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.026 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.026 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.026 INFO:teuthology.orchestra.run.smithi089.stdout: "discovery_interval": { 2024-09-10T08:49:01.026 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "discovery_interval", 2024-09-10T08:49:01.026 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-10T08:49:01.026 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.026 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.026 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "100", 2024-09-10T08:49:01.027 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.027 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.027 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.027 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.027 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.027 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.027 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.027 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.027 INFO:teuthology.orchestra.run.smithi089.stdout: "identifier": { 2024-09-10T08:49:01.027 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "identifier", 2024-09-10T08:49:01.027 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.027 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.027 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.027 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.028 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.028 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.028 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.028 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.028 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.028 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.028 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.028 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.028 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-10T08:49:01.028 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-10T08:49:01.028 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.028 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.028 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.029 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-10T08:49:01.029 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.029 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.029 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.029 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.029 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.029 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.029 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.029 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.030 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.030 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.030 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.030 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.030 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.030 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.030 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.030 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.030 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.030 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.030 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.030 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.031 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.031 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.031 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.031 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.031 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.031 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.031 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.031 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.031 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.031 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.031 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.032 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.032 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.032 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.032 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.032 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.032 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.032 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.032 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.032 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.032 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.032 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.032 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.032 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.033 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.033 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.033 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.033 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.033 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.033 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.033 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.033 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.033 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.033 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.033 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.033 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.034 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.034 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.034 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.034 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.034 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.034 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.034 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.034 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.034 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.034 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.035 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.035 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.035 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.035 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.035 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.035 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.035 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.035 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.035 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.035 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.035 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.035 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.035 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_host": { 2024-09-10T08:49:01.035 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_host", 2024-09-10T08:49:01.036 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.036 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.036 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.036 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.036 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.036 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.036 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.036 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.036 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.036 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.036 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.036 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.036 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_port": { 2024-09-10T08:49:01.037 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_port", 2024-09-10T08:49:01.037 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.037 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.037 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.037 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10051", 2024-09-10T08:49:01.037 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.037 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.037 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.037 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.037 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.037 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.037 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.038 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_sender": { 2024-09-10T08:49:01.038 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_sender", 2024-09-10T08:49:01.038 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.038 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.038 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.038 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T08:49:01.038 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.038 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.038 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.038 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.038 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.038 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.038 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.038 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.039 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.039 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.039 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-10T08:49:01.039 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.039 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.039 INFO:teuthology.orchestra.run.smithi089.stdout: "modules": [ 2024-09-10T08:49:01.039 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm", 2024-09-10T08:49:01.039 INFO:teuthology.orchestra.run.smithi089.stdout: "dashboard", 2024-09-10T08:49:01.039 INFO:teuthology.orchestra.run.smithi089.stdout: "iostat", 2024-09-10T08:49:01.039 INFO:teuthology.orchestra.run.smithi089.stdout: "nfs", 2024-09-10T08:49:01.039 INFO:teuthology.orchestra.run.smithi089.stdout: "restful" 2024-09-10T08:49:01.039 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.039 INFO:teuthology.orchestra.run.smithi089.stdout: "available_modules": [ 2024-09-10T08:49:01.040 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.040 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "alerts", 2024-09-10T08:49:01.040 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.040 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.040 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.040 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-10T08:49:01.040 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-10T08:49:01.040 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.040 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.040 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.040 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-10T08:49:01.041 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.041 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.041 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.041 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T08:49:01.041 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.041 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.041 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.041 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.041 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.041 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.041 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.041 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.042 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.042 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.042 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.042 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.042 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.042 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.042 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.042 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.042 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.042 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.042 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.042 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.042 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.043 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.043 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.043 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.043 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.043 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.043 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.043 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.043 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.043 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.043 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.043 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.043 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.043 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.044 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.044 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.044 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.044 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.044 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.044 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.044 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.044 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.044 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.044 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.044 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.044 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.044 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.045 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.045 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.045 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.045 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.045 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.045 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.045 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.045 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.045 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.045 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.045 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.045 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.045 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.045 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.046 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.046 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.046 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.046 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.046 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.046 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.046 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.046 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.046 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.046 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.046 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.046 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.046 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.046 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_destination": { 2024-09-10T08:49:01.047 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_destination", 2024-09-10T08:49:01.047 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.047 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.047 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.047 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.047 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.047 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.047 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.047 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Email address to send alerts to", 2024-09-10T08:49:01.047 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.047 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.047 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.047 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.048 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_from_name": { 2024-09-10T08:49:01.048 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_from_name", 2024-09-10T08:49:01.048 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.048 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.048 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.048 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "Ceph", 2024-09-10T08:49:01.048 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.048 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.048 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.048 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Email From: name", 2024-09-10T08:49:01.048 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.048 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.048 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.048 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.049 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_host": { 2024-09-10T08:49:01.049 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_host", 2024-09-10T08:49:01.049 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.049 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.049 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.049 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.049 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.049 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.049 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.049 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP server", 2024-09-10T08:49:01.049 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.049 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.049 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.050 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.050 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_password": { 2024-09-10T08:49:01.050 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_password", 2024-09-10T08:49:01.050 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.050 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.050 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.050 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.050 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.050 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.050 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.050 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Password to authenticate with", 2024-09-10T08:49:01.050 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.050 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.050 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.050 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.051 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_port": { 2024-09-10T08:49:01.051 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_port", 2024-09-10T08:49:01.051 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.051 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.051 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.051 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "465", 2024-09-10T08:49:01.051 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.051 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.051 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.051 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP port", 2024-09-10T08:49:01.051 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.051 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.051 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.051 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.052 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_sender": { 2024-09-10T08:49:01.052 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_sender", 2024-09-10T08:49:01.052 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.052 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.052 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.052 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.052 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.052 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.052 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.052 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SMTP envelope sender", 2024-09-10T08:49:01.052 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.052 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.052 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.053 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.053 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_ssl": { 2024-09-10T08:49:01.053 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_ssl", 2024-09-10T08:49:01.053 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.053 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.053 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.053 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.053 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.053 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.053 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.053 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T08:49:01.053 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.053 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.053 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.054 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.054 INFO:teuthology.orchestra.run.smithi089.stdout: "smtp_user": { 2024-09-10T08:49:01.054 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "smtp_user", 2024-09-10T08:49:01.054 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.054 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.054 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.054 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.054 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.054 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.054 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.054 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "User to authenticate as", 2024-09-10T08:49:01.054 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.054 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.054 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.055 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.055 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.055 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.055 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.055 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "balancer", 2024-09-10T08:49:01.055 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.055 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.055 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.055 INFO:teuthology.orchestra.run.smithi089.stdout: "active": { 2024-09-10T08:49:01.055 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "active", 2024-09-10T08:49:01.055 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.055 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.055 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.055 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.056 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.056 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.056 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.056 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T08:49:01.056 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.056 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.056 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.056 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.056 INFO:teuthology.orchestra.run.smithi089.stdout: "begin_time": { 2024-09-10T08:49:01.056 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "begin_time", 2024-09-10T08:49:01.056 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.056 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.056 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.056 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0000", 2024-09-10T08:49:01.057 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.057 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.057 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.057 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T08:49:01.057 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T08:49:01.057 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.057 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.057 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.057 INFO:teuthology.orchestra.run.smithi089.stdout: "begin_weekday": { 2024-09-10T08:49:01.057 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "begin_weekday", 2024-09-10T08:49:01.057 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-10T08:49:01.057 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.057 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.058 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-10T08:49:01.058 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0", 2024-09-10T08:49:01.058 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "6", 2024-09-10T08:49:01.058 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.058 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T08:49:01.058 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T08:49:01.058 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.058 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.058 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.058 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_max_iterations": { 2024-09-10T08:49:01.058 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_max_iterations", 2024-09-10T08:49:01.058 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-10T08:49:01.058 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.059 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.059 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "25", 2024-09-10T08:49:01.059 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-10T08:49:01.059 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "250", 2024-09-10T08:49:01.059 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.059 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T08:49:01.059 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.059 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.059 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.059 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.059 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_metrics": { 2024-09-10T08:49:01.059 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_metrics", 2024-09-10T08:49:01.059 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.059 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.060 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.060 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T08:49:01.060 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.060 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.060 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.060 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T08:49:01.060 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T08:49:01.060 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.060 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.060 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.060 INFO:teuthology.orchestra.run.smithi089.stdout: "crush_compat_step": { 2024-09-10T08:49:01.060 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crush_compat_step", 2024-09-10T08:49:01.060 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:01.061 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.061 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.061 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0.5", 2024-09-10T08:49:01.061 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0.001", 2024-09-10T08:49:01.061 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "0.999", 2024-09-10T08:49:01.061 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.061 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "aggressiveness of optimization", 2024-09-10T08:49:01.061 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T08:49:01.061 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.061 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.061 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.061 INFO:teuthology.orchestra.run.smithi089.stdout: "end_time": { 2024-09-10T08:49:01.061 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "end_time", 2024-09-10T08:49:01.061 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.062 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.062 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.062 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "2359", 2024-09-10T08:49:01.062 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.062 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.062 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.062 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T08:49:01.062 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T08:49:01.062 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.062 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.062 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.062 INFO:teuthology.orchestra.run.smithi089.stdout: "end_weekday": { 2024-09-10T08:49:01.063 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "end_weekday", 2024-09-10T08:49:01.063 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-10T08:49:01.063 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.063 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.063 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-10T08:49:01.063 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "0", 2024-09-10T08:49:01.063 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "6", 2024-09-10T08:49:01.063 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.063 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T08:49:01.063 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T08:49:01.063 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.063 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.063 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.063 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.064 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.064 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.064 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.064 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.064 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.064 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.064 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.064 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.064 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.064 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.064 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.064 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.065 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.065 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.065 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.065 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.065 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.065 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.065 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.065 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.065 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.065 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.065 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.065 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.065 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.065 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.066 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.066 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.066 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.066 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.066 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.066 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.066 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.066 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.066 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.066 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.066 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.066 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.066 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.066 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.067 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.067 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.067 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.067 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.067 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.067 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.067 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.067 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.067 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.067 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.067 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.067 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.067 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.067 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.068 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.068 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.068 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.068 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.068 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.068 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.068 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.068 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.068 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.068 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.068 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.068 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.068 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.068 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.069 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.069 INFO:teuthology.orchestra.run.smithi089.stdout: "min_score": { 2024-09-10T08:49:01.069 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "min_score", 2024-09-10T08:49:01.069 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:01.069 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.069 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.069 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-10T08:49:01.069 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.069 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.069 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.069 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T08:49:01.069 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.069 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.069 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.070 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.070 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": { 2024-09-10T08:49:01.070 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mode", 2024-09-10T08:49:01.070 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.070 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.070 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.070 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "upmap", 2024-09-10T08:49:01.070 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.070 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.070 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.070 INFO:teuthology.orchestra.run.smithi089.stdout: "crush-compat", 2024-09-10T08:49:01.070 INFO:teuthology.orchestra.run.smithi089.stdout: "none", 2024-09-10T08:49:01.070 INFO:teuthology.orchestra.run.smithi089.stdout: "read", 2024-09-10T08:49:01.070 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap", 2024-09-10T08:49:01.070 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap-read" 2024-09-10T08:49:01.071 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.071 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Balancer mode", 2024-09-10T08:49:01.071 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.071 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.071 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.071 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.071 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_ids": { 2024-09-10T08:49:01.071 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pool_ids", 2024-09-10T08:49:01.071 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.071 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.071 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.071 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.071 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.071 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.072 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.072 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T08:49:01.072 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.072 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.072 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.072 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.072 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-10T08:49:01.072 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-10T08:49:01.072 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.072 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.072 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.072 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-10T08:49:01.072 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.072 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.073 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.073 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T08:49:01.073 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.073 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.073 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.073 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.073 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap_max_deviation": { 2024-09-10T08:49:01.073 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "upmap_max_deviation", 2024-09-10T08:49:01.073 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.073 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.073 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.073 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-10T08:49:01.074 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-10T08:49:01.074 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.074 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.074 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T08:49:01.074 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T08:49:01.074 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.074 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.074 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.074 INFO:teuthology.orchestra.run.smithi089.stdout: "upmap_max_optimizations": { 2024-09-10T08:49:01.074 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "upmap_max_optimizations", 2024-09-10T08:49:01.074 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-10T08:49:01.074 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.074 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.074 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:01.075 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.075 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.075 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.075 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T08:49:01.075 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.075 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.075 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.075 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.075 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.075 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.075 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.075 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephadm", 2024-09-10T08:49:01.076 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.076 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.076 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.076 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_down_multiplier": { 2024-09-10T08:49:01.076 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_down_multiplier", 2024-09-10T08:49:01.076 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:01.076 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.076 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.076 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3.0", 2024-09-10T08:49:01.076 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.076 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.076 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.076 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T08:49:01.076 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.077 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.077 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.077 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.077 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_refresh_rate": { 2024-09-10T08:49:01.077 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_refresh_rate", 2024-09-10T08:49:01.077 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.077 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.077 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.077 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "20", 2024-09-10T08:49:01.077 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.077 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.077 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.077 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T08:49:01.077 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.078 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.078 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.078 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.078 INFO:teuthology.orchestra.run.smithi089.stdout: "agent_starting_port": { 2024-09-10T08:49:01.078 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "agent_starting_port", 2024-09-10T08:49:01.078 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.078 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.078 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.078 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "4721", 2024-09-10T08:49:01.078 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.078 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.078 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.078 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T08:49:01.078 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.079 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.079 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.079 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.079 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_ptrace": { 2024-09-10T08:49:01.079 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_ptrace", 2024-09-10T08:49:01.079 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.079 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.079 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.079 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.079 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.079 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.079 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.080 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T08:49:01.080 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T08:49:01.080 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.080 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.080 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.080 INFO:teuthology.orchestra.run.smithi089.stdout: "autotune_interval": { 2024-09-10T08:49:01.080 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "autotune_interval", 2024-09-10T08:49:01.080 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.080 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.080 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.080 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-10T08:49:01.080 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.080 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.081 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.081 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T08:49:01.081 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.081 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.081 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.081 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.081 INFO:teuthology.orchestra.run.smithi089.stdout: "autotune_memory_target_ratio": { 2024-09-10T08:49:01.081 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T08:49:01.081 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:01.081 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.081 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.081 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0.7", 2024-09-10T08:49:01.081 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.081 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.082 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.082 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T08:49:01.082 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.082 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.082 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.082 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.082 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm_log_destination": { 2024-09-10T08:49:01.082 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cephadm_log_destination", 2024-09-10T08:49:01.082 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.082 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.082 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.082 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.082 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.082 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.083 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.083 INFO:teuthology.orchestra.run.smithi089.stdout: "file", 2024-09-10T08:49:01.083 INFO:teuthology.orchestra.run.smithi089.stdout: "file,syslog", 2024-09-10T08:49:01.083 INFO:teuthology.orchestra.run.smithi089.stdout: "syslog" 2024-09-10T08:49:01.083 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.083 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T08:49:01.083 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.083 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.083 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.083 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.083 INFO:teuthology.orchestra.run.smithi089.stdout: "cgroups_split": { 2024-09-10T08:49:01.083 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cgroups_split", 2024-09-10T08:49:01.083 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.083 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.084 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.084 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.084 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.084 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.084 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.084 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T08:49:01.084 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.084 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.084 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.084 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.084 INFO:teuthology.orchestra.run.smithi089.stdout: "config_checks_enabled": { 2024-09-10T08:49:01.084 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_checks_enabled", 2024-09-10T08:49:01.084 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.085 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.085 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.085 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.085 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.085 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.085 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.085 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T08:49:01.085 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.085 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.085 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.085 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.085 INFO:teuthology.orchestra.run.smithi089.stdout: "config_dashboard": { 2024-09-10T08:49:01.085 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_dashboard", 2024-09-10T08:49:01.086 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.086 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.086 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.086 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.086 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.086 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.086 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.086 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T08:49:01.086 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.086 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.086 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.086 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.086 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_alertmanager": { 2024-09-10T08:49:01.086 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_alertmanager", 2024-09-10T08:49:01.087 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.087 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.087 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.087 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T08:49:01.087 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.087 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.087 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.087 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-10T08:49:01.087 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.087 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.087 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.087 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.087 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_base": { 2024-09-10T08:49:01.087 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_base", 2024-09-10T08:49:01.088 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.088 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.088 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.088 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T08:49:01.088 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.088 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.088 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.088 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Container image name, without the tag", 2024-09-10T08:49:01.088 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.088 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.088 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.088 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.089 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_elasticsearch": { 2024-09-10T08:49:01.089 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_elasticsearch", 2024-09-10T08:49:01.089 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.089 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.089 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.089 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T08:49:01.089 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.089 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.089 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.089 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "elasticsearch container image", 2024-09-10T08:49:01.089 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.089 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.089 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.089 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.090 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_grafana": { 2024-09-10T08:49:01.090 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_grafana", 2024-09-10T08:49:01.090 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.090 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.090 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.090 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T08:49:01.090 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.090 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.090 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.090 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-10T08:49:01.090 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.090 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.090 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.091 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.091 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_haproxy": { 2024-09-10T08:49:01.091 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_haproxy", 2024-09-10T08:49:01.091 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.091 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.091 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.091 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T08:49:01.091 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.091 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.091 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.091 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "HAproxy container image", 2024-09-10T08:49:01.091 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.091 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.091 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.092 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.092 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_agent": { 2024-09-10T08:49:01.092 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_agent", 2024-09-10T08:49:01.092 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.092 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.092 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.092 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T08:49:01.092 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.092 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.092 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.092 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger agent container image", 2024-09-10T08:49:01.092 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.092 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.093 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.093 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.093 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_collector": { 2024-09-10T08:49:01.093 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_collector", 2024-09-10T08:49:01.093 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.093 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.093 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.093 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T08:49:01.093 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.093 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.093 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.093 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger collector container image", 2024-09-10T08:49:01.093 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.094 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.094 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.094 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.094 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_jaeger_query": { 2024-09-10T08:49:01.094 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_jaeger_query", 2024-09-10T08:49:01.094 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.094 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.094 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.094 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T08:49:01.094 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.094 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.094 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.094 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Jaeger query container image", 2024-09-10T08:49:01.095 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.095 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.095 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.095 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.095 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_keepalived": { 2024-09-10T08:49:01.095 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_keepalived", 2024-09-10T08:49:01.095 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.095 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.095 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.095 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T08:49:01.095 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.095 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.096 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.096 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Keepalived container image", 2024-09-10T08:49:01.096 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.096 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.096 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.096 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.096 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_loki": { 2024-09-10T08:49:01.096 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_loki", 2024-09-10T08:49:01.096 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.096 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.096 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.096 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T08:49:01.096 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.097 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.097 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.097 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Loki container image", 2024-09-10T08:49:01.097 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.097 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.097 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.097 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.097 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_node_exporter": { 2024-09-10T08:49:01.097 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_node_exporter", 2024-09-10T08:49:01.097 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.097 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.097 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.097 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T08:49:01.098 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.098 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.098 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.098 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-10T08:49:01.098 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.098 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.098 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.098 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.098 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_nvmeof": { 2024-09-10T08:49:01.098 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_nvmeof", 2024-09-10T08:49:01.098 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.098 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.098 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.098 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T08:49:01.099 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.099 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.099 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.099 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Nvme-of container image", 2024-09-10T08:49:01.099 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.099 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.099 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.099 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.099 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_prometheus": { 2024-09-10T08:49:01.099 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_prometheus", 2024-09-10T08:49:01.099 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.099 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.099 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.099 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T08:49:01.100 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.100 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.100 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.100 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Prometheus container image", 2024-09-10T08:49:01.100 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.100 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.100 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.100 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.100 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_promtail": { 2024-09-10T08:49:01.100 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_promtail", 2024-09-10T08:49:01.100 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.100 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.100 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.101 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T08:49:01.101 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.101 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.101 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.101 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Promtail container image", 2024-09-10T08:49:01.101 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.101 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.101 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.101 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.101 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_samba": { 2024-09-10T08:49:01.101 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_samba", 2024-09-10T08:49:01.101 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.101 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.102 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.102 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T08:49:01.102 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.102 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.102 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.102 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Samba/SMB container image", 2024-09-10T08:49:01.102 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.102 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.102 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.102 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.102 INFO:teuthology.orchestra.run.smithi089.stdout: "container_image_snmp_gateway": { 2024-09-10T08:49:01.102 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_image_snmp_gateway", 2024-09-10T08:49:01.102 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.102 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.103 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.103 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T08:49:01.103 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.103 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.103 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.103 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "SNMP Gateway container image", 2024-09-10T08:49:01.103 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.103 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.103 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.103 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.103 INFO:teuthology.orchestra.run.smithi089.stdout: "container_init": { 2024-09-10T08:49:01.103 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "container_init", 2024-09-10T08:49:01.104 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.104 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.104 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.104 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.104 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.104 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.104 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.104 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T08:49:01.104 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.104 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.104 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.104 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.105 INFO:teuthology.orchestra.run.smithi089.stdout: "daemon_cache_timeout": { 2024-09-10T08:49:01.105 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "daemon_cache_timeout", 2024-09-10T08:49:01.105 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.105 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.105 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.105 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-10T08:49:01.105 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.105 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.105 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.105 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T08:49:01.105 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.105 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.106 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.106 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.106 INFO:teuthology.orchestra.run.smithi089.stdout: "default_cephadm_command_timeout": { 2024-09-10T08:49:01.106 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T08:49:01.106 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.106 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.106 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.106 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "900", 2024-09-10T08:49:01.106 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.106 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.106 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.106 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T08:49:01.107 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.107 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.107 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.107 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.107 INFO:teuthology.orchestra.run.smithi089.stdout: "default_registry": { 2024-09-10T08:49:01.107 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "default_registry", 2024-09-10T08:49:01.107 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.107 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.107 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.107 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "docker.io", 2024-09-10T08:49:01.107 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.107 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.107 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.108 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T08:49:01.108 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.108 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.108 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.108 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.108 INFO:teuthology.orchestra.run.smithi089.stdout: "device_cache_timeout": { 2024-09-10T08:49:01.108 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_cache_timeout", 2024-09-10T08:49:01.108 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.108 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.108 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.108 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1800", 2024-09-10T08:49:01.108 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.108 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.109 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.109 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache device inventory", 2024-09-10T08:49:01.109 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.109 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.109 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.109 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.109 INFO:teuthology.orchestra.run.smithi089.stdout: "device_enhanced_scan": { 2024-09-10T08:49:01.109 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_enhanced_scan", 2024-09-10T08:49:01.109 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.109 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.109 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.109 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.109 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.110 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.110 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.110 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T08:49:01.110 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.110 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.110 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.110 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.110 INFO:teuthology.orchestra.run.smithi089.stdout: "facts_cache_timeout": { 2024-09-10T08:49:01.110 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "facts_cache_timeout", 2024-09-10T08:49:01.110 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.110 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.110 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.110 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-10T08:49:01.111 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.111 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.111 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.111 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "seconds to cache host facts data", 2024-09-10T08:49:01.111 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.111 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.111 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.111 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.111 INFO:teuthology.orchestra.run.smithi089.stdout: "grafana_dashboards_path": { 2024-09-10T08:49:01.111 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "grafana_dashboards_path", 2024-09-10T08:49:01.111 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.111 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.111 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.111 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T08:49:01.112 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.112 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.112 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.112 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T08:49:01.112 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.112 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.112 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.112 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.112 INFO:teuthology.orchestra.run.smithi089.stdout: "host_check_interval": { 2024-09-10T08:49:01.112 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "host_check_interval", 2024-09-10T08:49:01.112 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.112 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.112 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.113 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-10T08:49:01.113 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.113 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.113 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.113 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to perform a host check", 2024-09-10T08:49:01.113 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.113 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.113 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.113 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.113 INFO:teuthology.orchestra.run.smithi089.stdout: "hw_monitoring": { 2024-09-10T08:49:01.113 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "hw_monitoring", 2024-09-10T08:49:01.113 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.113 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.113 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.114 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.114 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.114 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.114 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.114 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T08:49:01.114 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.114 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.114 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.114 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.114 INFO:teuthology.orchestra.run.smithi089.stdout: "inventory_list_all": { 2024-09-10T08:49:01.114 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "inventory_list_all", 2024-09-10T08:49:01.114 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.114 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.114 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.115 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.115 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.115 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.115 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.115 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T08:49:01.115 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.115 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.115 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.115 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.115 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.115 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.115 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.115 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.115 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.115 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.116 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.116 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.116 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.116 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.116 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.116 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.116 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.116 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.116 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.116 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.116 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.116 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.116 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.117 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.117 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.117 INFO:teuthology.orchestra.run.smithi089.stdout: "log_refresh_metadata": { 2024-09-10T08:49:01.117 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_refresh_metadata", 2024-09-10T08:49:01.117 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.117 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.117 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.117 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.117 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.117 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.117 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.117 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T08:49:01.117 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.117 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.118 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.118 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.118 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.118 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.118 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.118 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.118 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.118 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.118 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.118 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.118 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.118 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T08:49:01.118 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.118 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.118 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.119 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.119 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.119 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.119 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.119 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.119 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.119 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.119 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.119 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.119 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.119 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.119 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.119 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.120 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.120 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.120 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.120 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.120 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.120 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.120 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.120 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.120 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.120 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.120 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.120 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.120 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.120 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.121 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.121 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.121 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.121 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.121 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.121 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.121 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.121 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.121 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.121 INFO:teuthology.orchestra.run.smithi089.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T08:49:01.121 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T08:49:01.121 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.121 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.122 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.122 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.122 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.122 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.122 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.122 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T08:49:01.122 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.122 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.122 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.122 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.122 INFO:teuthology.orchestra.run.smithi089.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T08:49:01.122 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T08:49:01.122 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.123 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.123 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.123 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "*", 2024-09-10T08:49:01.123 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.123 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.123 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.123 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T08:49:01.123 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.123 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.123 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.123 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.123 INFO:teuthology.orchestra.run.smithi089.stdout: "max_count_per_host": { 2024-09-10T08:49:01.123 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_count_per_host", 2024-09-10T08:49:01.123 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.124 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.124 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.124 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:01.124 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.124 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.124 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.124 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "max number of daemons per service per host", 2024-09-10T08:49:01.124 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.124 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.124 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.124 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.124 INFO:teuthology.orchestra.run.smithi089.stdout: "max_osd_draining_count": { 2024-09-10T08:49:01.124 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_osd_draining_count", 2024-09-10T08:49:01.125 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.125 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.125 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.125 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:01.125 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.125 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.125 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.125 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T08:49:01.125 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.125 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.125 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.125 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.125 INFO:teuthology.orchestra.run.smithi089.stdout: "migration_current": { 2024-09-10T08:49:01.125 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "migration_current", 2024-09-10T08:49:01.126 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.126 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.126 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.126 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.126 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.126 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.126 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.126 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "internal - do not modify", 2024-09-10T08:49:01.126 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.126 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.126 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.126 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.126 INFO:teuthology.orchestra.run.smithi089.stdout: "mode": { 2024-09-10T08:49:01.127 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mode", 2024-09-10T08:49:01.127 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.127 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.127 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.127 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "root", 2024-09-10T08:49:01.127 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.127 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.127 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.127 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm-package", 2024-09-10T08:49:01.127 INFO:teuthology.orchestra.run.smithi089.stdout: "root" 2024-09-10T08:49:01.127 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.127 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T08:49:01.127 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.127 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.128 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.128 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.128 INFO:teuthology.orchestra.run.smithi089.stdout: "oob_default_addr": { 2024-09-10T08:49:01.128 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "oob_default_addr", 2024-09-10T08:49:01.128 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.128 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.128 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.128 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "169.254.1.1", 2024-09-10T08:49:01.128 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.128 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.128 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.128 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T08:49:01.128 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.128 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.129 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.129 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.129 INFO:teuthology.orchestra.run.smithi089.stdout: "prometheus_alerts_path": { 2024-09-10T08:49:01.129 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prometheus_alerts_path", 2024-09-10T08:49:01.129 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.129 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.129 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.129 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T08:49:01.129 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.129 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.129 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.129 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T08:49:01.129 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.129 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.130 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.130 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.130 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_insecure": { 2024-09-10T08:49:01.130 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_insecure", 2024-09-10T08:49:01.130 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.130 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.130 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.130 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.130 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.130 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.130 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.130 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T08:49:01.130 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.131 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.131 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.131 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.131 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_password": { 2024-09-10T08:49:01.131 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_password", 2024-09-10T08:49:01.131 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.131 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.131 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.131 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.131 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.131 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.131 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.131 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T08:49:01.131 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.132 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.132 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.132 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.132 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_url": { 2024-09-10T08:49:01.132 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_url", 2024-09-10T08:49:01.132 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.132 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.132 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.132 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.132 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.132 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.132 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.132 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T08:49:01.132 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.133 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.133 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.133 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.133 INFO:teuthology.orchestra.run.smithi089.stdout: "registry_username": { 2024-09-10T08:49:01.133 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "registry_username", 2024-09-10T08:49:01.133 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.133 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.133 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.133 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.133 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.133 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.133 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.133 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T08:49:01.133 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.134 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.134 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.134 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.134 INFO:teuthology.orchestra.run.smithi089.stdout: "secure_monitoring_stack": { 2024-09-10T08:49:01.134 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "secure_monitoring_stack", 2024-09-10T08:49:01.134 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.134 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.134 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.134 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.134 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.134 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.134 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.134 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T08:49:01.135 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.135 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.135 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.135 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.135 INFO:teuthology.orchestra.run.smithi089.stdout: "service_discovery_port": { 2024-09-10T08:49:01.135 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "service_discovery_port", 2024-09-10T08:49:01.135 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.135 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.135 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.135 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8765", 2024-09-10T08:49:01.136 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.136 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.136 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.136 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "cephadm service discovery port", 2024-09-10T08:49:01.136 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.136 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.136 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.136 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.136 INFO:teuthology.orchestra.run.smithi089.stdout: "ssh_config_file": { 2024-09-10T08:49:01.136 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssh_config_file", 2024-09-10T08:49:01.136 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.136 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.137 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.137 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.137 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.137 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.137 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.137 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T08:49:01.137 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.137 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.137 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.137 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.137 INFO:teuthology.orchestra.run.smithi089.stdout: "use_agent": { 2024-09-10T08:49:01.137 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "use_agent", 2024-09-10T08:49:01.138 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.138 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.138 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.138 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.138 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.138 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.138 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.138 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T08:49:01.138 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.138 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.139 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.139 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.139 INFO:teuthology.orchestra.run.smithi089.stdout: "use_repo_digest": { 2024-09-10T08:49:01.139 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "use_repo_digest", 2024-09-10T08:49:01.139 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.139 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.139 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.139 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.139 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.139 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.139 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.139 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T08:49:01.139 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.140 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.140 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.140 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.140 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_failed_host_check": { 2024-09-10T08:49:01.140 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_failed_host_check", 2024-09-10T08:49:01.140 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.140 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.140 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.140 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.140 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.140 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.140 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.140 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T08:49:01.141 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.141 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.141 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.141 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.141 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_stray_daemons": { 2024-09-10T08:49:01.141 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_stray_daemons", 2024-09-10T08:49:01.141 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.141 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.141 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.141 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.141 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.141 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.141 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.141 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T08:49:01.142 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.142 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.142 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.142 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.142 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_on_stray_hosts": { 2024-09-10T08:49:01.142 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_on_stray_hosts", 2024-09-10T08:49:01.142 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.142 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.142 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.142 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.142 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.142 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.142 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.142 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T08:49:01.143 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.143 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.143 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.143 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.143 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.143 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.143 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.143 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crash", 2024-09-10T08:49:01.143 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.143 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.143 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.143 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.143 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.143 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.144 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.144 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.144 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.144 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.144 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.144 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.144 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.144 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.144 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.144 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.144 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.144 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.144 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.144 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.145 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.145 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.145 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.145 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.145 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.145 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.145 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.145 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.145 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.145 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.145 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.145 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.145 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.145 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.146 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.146 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.146 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.146 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.146 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.146 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.146 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.146 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.146 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.146 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.146 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.146 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.146 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.146 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.147 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.147 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.147 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.147 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.147 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.147 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.147 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.147 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.147 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.147 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.147 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.147 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.147 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.147 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.148 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.148 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.148 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.148 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.148 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.148 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.148 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.148 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.148 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.148 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.148 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.148 INFO:teuthology.orchestra.run.smithi089.stdout: "retain_interval": { 2024-09-10T08:49:01.148 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "retain_interval", 2024-09-10T08:49:01.148 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.149 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.149 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.149 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "31536000", 2024-09-10T08:49:01.149 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.149 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.149 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.149 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T08:49:01.149 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.149 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.149 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.149 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.149 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_recent_interval": { 2024-09-10T08:49:01.149 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_recent_interval", 2024-09-10T08:49:01.149 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.150 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.150 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.150 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1209600", 2024-09-10T08:49:01.150 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.150 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.150 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.150 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T08:49:01.150 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.150 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.150 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.150 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.150 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.150 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.150 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.151 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "dashboard", 2024-09-10T08:49:01.151 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.151 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.151 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.151 INFO:teuthology.orchestra.run.smithi089.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T08:49:01.151 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T08:49:01.151 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.151 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.151 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.151 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:01.151 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.151 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.151 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.152 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.152 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.152 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.152 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.152 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.152 INFO:teuthology.orchestra.run.smithi089.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T08:49:01.152 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T08:49:01.152 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.152 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.152 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.152 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.152 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.152 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.152 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.153 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.153 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.153 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.153 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.153 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.153 INFO:teuthology.orchestra.run.smithi089.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T08:49:01.153 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T08:49:01.153 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.153 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.153 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.153 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.153 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.153 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.154 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.154 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.154 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.154 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.154 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.154 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.154 INFO:teuthology.orchestra.run.smithi089.stdout: "AUDIT_API_ENABLED": { 2024-09-10T08:49:01.154 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T08:49:01.154 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.154 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.154 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.154 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.154 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.154 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.155 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.155 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.155 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.155 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.155 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.155 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.155 INFO:teuthology.orchestra.run.smithi089.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T08:49:01.155 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T08:49:01.155 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.155 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.155 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.155 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.155 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.155 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.155 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.156 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.156 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.156 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.156 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.156 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.156 INFO:teuthology.orchestra.run.smithi089.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T08:49:01.156 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T08:49:01.156 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.156 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.156 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.156 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.156 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.156 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.157 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.157 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.157 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.157 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.157 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.157 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.157 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T08:49:01.157 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T08:49:01.157 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.157 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.157 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.157 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.157 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.157 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.158 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.158 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.158 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.158 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.158 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.158 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.158 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T08:49:01.158 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T08:49:01.158 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.158 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.158 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.158 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.158 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.158 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.159 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.159 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.159 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.159 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.159 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.159 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.159 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T08:49:01.159 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T08:49:01.159 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.159 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.159 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.159 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.159 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.159 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.159 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.160 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.160 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.160 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.160 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.160 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.160 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T08:49:01.160 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T08:49:01.160 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.160 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.160 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.160 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.160 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.160 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.160 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.161 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.161 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.161 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.161 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.161 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.161 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T08:49:01.161 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T08:49:01.161 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.161 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.161 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.161 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.161 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.161 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.161 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.162 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.162 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.162 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.162 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.162 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.162 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T08:49:01.162 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T08:49:01.162 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.162 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.162 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.162 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.162 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.162 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.162 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.163 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.163 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.163 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.163 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.163 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.163 INFO:teuthology.orchestra.run.smithi089.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T08:49:01.163 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T08:49:01.163 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.163 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.163 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.163 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.163 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.163 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.163 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.164 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.164 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.164 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.164 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.164 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.164 INFO:teuthology.orchestra.run.smithi089.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T08:49:01.164 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T08:49:01.164 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.164 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.164 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.164 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.164 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.164 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.164 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.165 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.165 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.165 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.165 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.165 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.165 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T08:49:01.165 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T08:49:01.165 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.165 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.165 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.165 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2024-09-10T08:49:01.165 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.165 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.165 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.166 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.166 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.166 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.166 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.166 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.166 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T08:49:01.166 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T08:49:01.166 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.166 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.166 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.166 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.166 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.166 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.167 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.167 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.167 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.167 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.167 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.167 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.167 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_URL": { 2024-09-10T08:49:01.167 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_URL", 2024-09-10T08:49:01.167 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.167 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.167 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.167 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.167 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.168 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.168 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.168 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.168 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.168 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.168 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.168 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.168 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T08:49:01.168 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T08:49:01.168 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.168 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.168 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.168 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2024-09-10T08:49:01.169 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.169 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.169 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.169 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.169 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.169 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.169 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.169 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.169 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T08:49:01.169 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T08:49:01.169 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.169 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.169 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.169 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.170 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.170 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.170 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.170 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.170 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.170 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.170 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.170 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.170 INFO:teuthology.orchestra.run.smithi089.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T08:49:01.170 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T08:49:01.170 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.170 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.170 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.170 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.171 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.171 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.171 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.171 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.171 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.171 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.171 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.171 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.171 INFO:teuthology.orchestra.run.smithi089.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T08:49:01.171 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T08:49:01.171 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.171 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.171 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.172 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.172 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.172 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.172 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.172 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.172 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.172 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.172 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.172 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.172 INFO:teuthology.orchestra.run.smithi089.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T08:49:01.172 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T08:49:01.172 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.172 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.172 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.173 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.173 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.173 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.173 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.173 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.173 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.173 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.173 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.173 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.173 INFO:teuthology.orchestra.run.smithi089.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T08:49:01.173 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T08:49:01.173 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.173 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.174 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.174 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.174 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.174 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.174 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.174 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.174 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.174 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.174 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.174 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.174 INFO:teuthology.orchestra.run.smithi089.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T08:49:01.174 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T08:49:01.174 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.174 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.175 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.175 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.175 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.175 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.175 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.175 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.175 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.175 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.175 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.175 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.175 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T08:49:01.175 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T08:49:01.175 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.175 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.176 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.176 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.176 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.176 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.176 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.176 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.176 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.176 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.176 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.176 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.176 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T08:49:01.176 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T08:49:01.176 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.177 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.177 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.177 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.177 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.177 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.177 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.177 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.177 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.177 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.177 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.177 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.177 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T08:49:01.177 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T08:49:01.177 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.178 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.178 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.178 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.178 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.178 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.178 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.178 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.178 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.178 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.178 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.178 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.178 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T08:49:01.178 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T08:49:01.178 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.179 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.179 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.179 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.179 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.179 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.179 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.179 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.179 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.179 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.179 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.179 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.179 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T08:49:01.179 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T08:49:01.179 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.179 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.180 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.180 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.180 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.180 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.180 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.180 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.180 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.180 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.180 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.180 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.180 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T08:49:01.180 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T08:49:01.180 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.181 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.181 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.181 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.181 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.181 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.181 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.181 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.181 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.181 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.181 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.181 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.181 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T08:49:01.181 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T08:49:01.181 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.182 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.182 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.182 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.182 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.182 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.182 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.182 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.182 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.182 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.182 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.183 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.183 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T08:49:01.183 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T08:49:01.183 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.183 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.183 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.183 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.183 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.183 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.183 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.183 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.184 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.184 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.184 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.184 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.184 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T08:49:01.184 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T08:49:01.184 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.184 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.184 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.184 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T08:49:01.185 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.185 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.185 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.185 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.185 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.185 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.185 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.185 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.185 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T08:49:01.185 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T08:49:01.186 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.186 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.186 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.186 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:01.186 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.186 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.186 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.186 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.186 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.186 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.187 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.187 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.187 INFO:teuthology.orchestra.run.smithi089.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T08:49:01.187 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T08:49:01.187 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.187 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.187 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.187 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8", 2024-09-10T08:49:01.187 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.187 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.187 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.187 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.187 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.187 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.188 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.188 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.188 INFO:teuthology.orchestra.run.smithi089.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T08:49:01.188 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T08:49:01.188 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.188 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.188 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.188 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "45", 2024-09-10T08:49:01.188 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.188 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.188 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.188 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.188 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.188 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.189 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.189 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.189 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T08:49:01.189 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T08:49:01.189 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.189 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.189 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.189 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.189 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.189 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.189 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.189 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.189 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.190 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.190 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.190 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.190 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T08:49:01.190 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T08:49:01.190 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.190 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.190 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.190 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "admin", 2024-09-10T08:49:01.190 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.190 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.190 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.190 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.190 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.191 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.191 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.191 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.192 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T08:49:01.192 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T08:49:01.192 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.192 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.193 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.193 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.193 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.193 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.193 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.193 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.193 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.193 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.193 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.193 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.193 INFO:teuthology.orchestra.run.smithi089.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T08:49:01.193 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T08:49:01.193 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.193 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.194 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.194 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.194 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.194 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.194 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.194 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.194 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.194 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.194 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.194 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.194 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T08:49:01.194 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T08:49:01.194 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.194 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.195 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.195 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-10T08:49:01.195 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.195 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.195 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.195 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.195 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.195 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.195 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.195 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.195 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T08:49:01.195 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T08:49:01.195 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.195 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.196 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.196 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:01.196 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.196 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.196 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.196 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.196 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.196 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.196 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.196 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.196 INFO:teuthology.orchestra.run.smithi089.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T08:49:01.196 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T08:49:01.196 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.197 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.197 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.197 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-10T08:49:01.197 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.197 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.197 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.197 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.197 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.197 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.197 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.197 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.197 INFO:teuthology.orchestra.run.smithi089.stdout: "cross_origin_url": { 2024-09-10T08:49:01.197 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cross_origin_url", 2024-09-10T08:49:01.197 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: "crt_file": { 2024-09-10T08:49:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "crt_file", 2024-09-10T08:49:01.199 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.199 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.199 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.199 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.199 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.199 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.199 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.199 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.199 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.199 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.199 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.199 INFO:teuthology.orchestra.run.smithi089.stdout: "debug": { 2024-09-10T08:49:01.199 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "debug", 2024-09-10T08:49:01.199 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.200 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.200 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.200 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.200 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.200 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.200 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.200 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Enable/disable debug options", 2024-09-10T08:49:01.200 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.200 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.200 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.200 INFO:teuthology.orchestra.run.smithi089.stdout: "jwt_token_ttl": { 2024-09-10T08:49:01.200 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "jwt_token_ttl", 2024-09-10T08:49:01.201 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.201 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.201 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.201 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "28800", 2024-09-10T08:49:01.201 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.201 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.201 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.201 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.201 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.201 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.201 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.201 INFO:teuthology.orchestra.run.smithi089.stdout: "key_file": { 2024-09-10T08:49:01.201 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "key_file", 2024-09-10T08:49:01.202 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.202 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.202 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.202 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.202 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.202 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.202 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.202 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.202 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.202 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.202 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.202 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.202 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.202 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.203 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.203 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.203 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.203 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.203 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.203 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.203 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.203 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.203 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.203 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.203 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.203 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.203 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.203 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.204 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.204 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.204 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.204 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.204 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.204 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.204 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.204 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.204 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.204 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.204 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.204 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.205 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.205 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.205 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.205 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.205 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.205 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.205 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.205 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.205 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.205 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.205 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.205 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.205 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.206 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.206 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.206 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.206 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.206 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.206 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.206 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.206 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.206 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.206 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.206 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.206 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.206 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.207 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.207 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.207 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.207 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.207 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.207 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.207 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.207 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.207 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.207 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.207 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.207 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.207 INFO:teuthology.orchestra.run.smithi089.stdout: "motd": { 2024-09-10T08:49:01.207 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "motd", 2024-09-10T08:49:01.208 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.208 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.208 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.208 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.208 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.208 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.208 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.208 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "The message of the day", 2024-09-10T08:49:01.208 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.208 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.208 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.208 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.208 INFO:teuthology.orchestra.run.smithi089.stdout: "redirect_resolve_ip_addr": { 2024-09-10T08:49:01.208 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T08:49:01.209 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.209 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.209 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.209 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.209 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.209 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.209 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.209 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.209 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.209 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.209 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.209 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.209 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2024-09-10T08:49:01.209 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2024-09-10T08:49:01.209 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.210 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.210 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.210 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "::", 2024-09-10T08:49:01.210 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.210 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.210 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.210 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.210 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.210 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.210 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.210 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2024-09-10T08:49:01.210 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2024-09-10T08:49:01.211 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.211 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.211 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.211 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8080", 2024-09-10T08:49:01.211 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.211 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.211 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.211 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.211 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.211 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.211 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.211 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl": { 2024-09-10T08:49:01.212 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl", 2024-09-10T08:49:01.212 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.212 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.212 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.212 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.212 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.212 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.212 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.212 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.213 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.213 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.213 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.213 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl_server_port": { 2024-09-10T08:49:01.213 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl_server_port", 2024-09-10T08:49:01.213 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.213 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.213 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.213 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8443", 2024-09-10T08:49:01.213 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.213 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.213 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.213 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.214 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.214 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.214 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.214 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_behaviour": { 2024-09-10T08:49:01.214 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_behaviour", 2024-09-10T08:49:01.214 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.214 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.214 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.214 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "redirect", 2024-09-10T08:49:01.215 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.215 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.215 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.215 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.215 INFO:teuthology.orchestra.run.smithi089.stdout: "redirect" 2024-09-10T08:49:01.215 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.215 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.215 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.215 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.215 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.215 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_error_status_code": { 2024-09-10T08:49:01.215 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_error_status_code", 2024-09-10T08:49:01.216 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.216 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.216 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.216 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2024-09-10T08:49:01.216 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "400", 2024-09-10T08:49:01.216 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "599", 2024-09-10T08:49:01.216 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.216 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.216 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.216 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.216 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.216 INFO:teuthology.orchestra.run.smithi089.stdout: "url_prefix": { 2024-09-10T08:49:01.217 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "url_prefix", 2024-09-10T08:49:01.217 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.217 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.217 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.217 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.217 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.217 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.217 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.217 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.217 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.217 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.218 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.218 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.218 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.218 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.218 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "devicehealth", 2024-09-10T08:49:01.218 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.218 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.218 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.218 INFO:teuthology.orchestra.run.smithi089.stdout: "enable_monitoring": { 2024-09-10T08:49:01.218 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enable_monitoring", 2024-09-10T08:49:01.218 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.219 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.219 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.219 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.219 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.219 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.219 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.219 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "monitor device health metrics", 2024-09-10T08:49:01.219 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.219 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.219 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.219 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.220 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.220 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.220 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.220 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.220 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.220 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.220 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.220 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.220 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.220 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.221 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.221 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.221 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.221 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.221 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.221 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.221 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.221 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.221 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.221 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.222 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.222 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.222 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.222 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.222 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.222 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.222 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.222 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.222 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.222 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.222 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.223 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.223 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.223 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.223 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.223 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.223 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.223 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.223 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.223 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.224 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.224 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.224 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.224 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.224 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.224 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.224 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.224 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.224 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.224 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.224 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.224 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.225 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.225 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.225 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.225 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.225 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.225 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.225 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.225 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.225 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.225 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.225 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.225 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.226 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.226 INFO:teuthology.orchestra.run.smithi089.stdout: "mark_out_threshold": { 2024-09-10T08:49:01.226 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mark_out_threshold", 2024-09-10T08:49:01.226 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.226 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.226 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.226 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "2419200", 2024-09-10T08:49:01.226 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.226 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.226 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.226 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T08:49:01.226 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.227 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.227 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.227 INFO:teuthology.orchestra.run.smithi089.stdout: "pool_name": { 2024-09-10T08:49:01.227 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pool_name", 2024-09-10T08:49:01.227 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.227 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.227 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.227 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "device_health_metrics", 2024-09-10T08:49:01.227 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.227 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.227 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.227 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T08:49:01.228 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.228 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.228 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.228 INFO:teuthology.orchestra.run.smithi089.stdout: "retention_period": { 2024-09-10T08:49:01.228 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "retention_period", 2024-09-10T08:49:01.228 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.228 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.228 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.228 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15552000", 2024-09-10T08:49:01.228 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.228 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.229 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.229 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long to retain device health metrics", 2024-09-10T08:49:01.229 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.229 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.229 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.229 INFO:teuthology.orchestra.run.smithi089.stdout: "scrape_frequency": { 2024-09-10T08:49:01.229 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "scrape_frequency", 2024-09-10T08:49:01.229 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.229 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.229 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.229 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "86400", 2024-09-10T08:49:01.229 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.230 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.230 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.230 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T08:49:01.230 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.230 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.230 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.230 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.230 INFO:teuthology.orchestra.run.smithi089.stdout: "self_heal": { 2024-09-10T08:49:01.230 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "self_heal", 2024-09-10T08:49:01.230 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.230 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.230 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.230 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.231 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.231 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.231 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.231 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T08:49:01.231 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.231 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.231 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.231 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.231 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-10T08:49:01.231 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-10T08:49:01.231 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.231 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.231 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.231 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-10T08:49:01.232 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.232 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.232 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.232 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T08:49:01.232 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.232 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.232 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.232 INFO:teuthology.orchestra.run.smithi089.stdout: "warn_threshold": { 2024-09-10T08:49:01.232 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "warn_threshold", 2024-09-10T08:49:01.232 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.232 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.232 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.232 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7257600", 2024-09-10T08:49:01.232 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.233 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.233 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.233 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T08:49:01.233 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.233 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.233 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.233 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.233 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.233 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.233 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "diskprediction_local", 2024-09-10T08:49:01.233 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.233 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.233 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.234 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.234 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.234 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.234 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.234 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.234 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.234 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.234 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.234 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.234 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.234 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.234 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.234 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.235 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.235 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.235 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.235 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.235 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.235 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.235 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.235 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.235 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.235 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.235 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.235 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.235 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.236 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.236 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.236 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.236 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.236 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.236 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.236 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.236 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.236 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.236 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.236 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.236 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.237 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.237 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.237 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.237 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.237 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.237 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.237 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.237 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.237 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.237 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.237 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.237 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.237 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.238 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.238 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.238 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.238 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.238 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.238 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.238 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.238 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.238 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.238 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.238 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.238 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.238 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.239 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.239 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.239 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.239 INFO:teuthology.orchestra.run.smithi089.stdout: "predict_interval": { 2024-09-10T08:49:01.239 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "predict_interval", 2024-09-10T08:49:01.239 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.239 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.239 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.239 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "86400", 2024-09-10T08:49:01.240 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.240 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.240 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.240 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.240 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.240 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.240 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.240 INFO:teuthology.orchestra.run.smithi089.stdout: "predictor_model": { 2024-09-10T08:49:01.240 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "predictor_model", 2024-09-10T08:49:01.240 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.240 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.241 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.241 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "prophetstor", 2024-09-10T08:49:01.241 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.241 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.241 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.241 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.241 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.241 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.241 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.241 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-10T08:49:01.241 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-10T08:49:01.241 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.241 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.242 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.242 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "600", 2024-09-10T08:49:01.242 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.242 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.242 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.242 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.242 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.242 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.242 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.242 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.242 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.242 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.242 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.242 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "influx", 2024-09-10T08:49:01.243 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": false, 2024-09-10T08:49:01.243 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "influxdb python module not found", 2024-09-10T08:49:01.243 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.243 INFO:teuthology.orchestra.run.smithi089.stdout: "batch_size": { 2024-09-10T08:49:01.243 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "batch_size", 2024-09-10T08:49:01.243 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.243 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.243 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.243 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5000", 2024-09-10T08:49:01.243 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.243 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.243 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.243 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T08:49:01.243 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.244 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.244 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.244 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.244 INFO:teuthology.orchestra.run.smithi089.stdout: "database": { 2024-09-10T08:49:01.244 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "database", 2024-09-10T08:49:01.244 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.244 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.244 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.244 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "ceph", 2024-09-10T08:49:01.244 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.244 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.244 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.244 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T08:49:01.244 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.245 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.245 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.245 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.245 INFO:teuthology.orchestra.run.smithi089.stdout: "hostname": { 2024-09-10T08:49:01.245 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "hostname", 2024-09-10T08:49:01.245 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.245 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.245 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.245 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.245 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.245 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.245 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.245 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB server hostname", 2024-09-10T08:49:01.246 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.246 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.246 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.246 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.246 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-10T08:49:01.246 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-10T08:49:01.246 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.246 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.246 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.246 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "30", 2024-09-10T08:49:01.246 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "5", 2024-09-10T08:49:01.246 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.246 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.246 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T08:49:01.247 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.247 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.247 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.247 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.247 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.247 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.247 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.247 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.247 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.247 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.247 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.247 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.247 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.248 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.248 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.248 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.248 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.248 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.248 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.248 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.248 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.248 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.248 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.248 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.248 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.248 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.248 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.249 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.249 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.249 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.249 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.249 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.249 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.249 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.249 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.249 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.249 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.249 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.249 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.249 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.250 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.250 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.250 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.250 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.250 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.250 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.250 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.250 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.250 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.250 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.250 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.250 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.250 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.250 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.251 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.251 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.251 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.251 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.251 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.251 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.251 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.251 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.251 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.251 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.251 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.251 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.251 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.251 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.252 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.252 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.252 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.252 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.252 INFO:teuthology.orchestra.run.smithi089.stdout: "password": { 2024-09-10T08:49:01.252 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "password", 2024-09-10T08:49:01.252 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.252 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.252 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.252 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.252 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.252 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.252 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.252 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "password of InfluxDB server user", 2024-09-10T08:49:01.253 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.253 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.253 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.253 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.253 INFO:teuthology.orchestra.run.smithi089.stdout: "port": { 2024-09-10T08:49:01.253 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "port", 2024-09-10T08:49:01.253 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.253 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.253 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.253 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "8086", 2024-09-10T08:49:01.253 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.253 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.253 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.253 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "InfluxDB server port", 2024-09-10T08:49:01.253 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.254 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.254 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.254 INFO:teuthology.orchestra.run.smithi089.stdout: "ssl": { 2024-09-10T08:49:01.254 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ssl", 2024-09-10T08:49:01.254 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.254 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.254 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.254 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "false", 2024-09-10T08:49:01.254 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.254 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.254 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.254 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T08:49:01.254 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.255 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.255 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.255 INFO:teuthology.orchestra.run.smithi089.stdout: "threads": { 2024-09-10T08:49:01.255 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "threads", 2024-09-10T08:49:01.255 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.255 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.255 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.255 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-10T08:49:01.255 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-10T08:49:01.256 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "32", 2024-09-10T08:49:01.256 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.256 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T08:49:01.256 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.256 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.256 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.256 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.256 INFO:teuthology.orchestra.run.smithi089.stdout: "username": { 2024-09-10T08:49:01.256 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "username", 2024-09-10T08:49:01.256 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.256 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.256 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.256 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.257 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.257 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.257 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.257 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "username of InfluxDB server user", 2024-09-10T08:49:01.257 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.257 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.257 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.257 INFO:teuthology.orchestra.run.smithi089.stdout: "verify_ssl": { 2024-09-10T08:49:01.257 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "verify_ssl", 2024-09-10T08:49:01.258 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.258 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.258 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.258 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "true", 2024-09-10T08:49:01.258 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.258 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.258 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.258 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T08:49:01.258 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.258 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.258 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.258 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.258 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.259 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.259 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.259 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "insights", 2024-09-10T08:49:01.259 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.259 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.259 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.259 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.259 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.259 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.259 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.260 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.260 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.260 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.260 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.260 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.260 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.260 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.260 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.260 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.260 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.261 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.261 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.261 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.261 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.261 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.261 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.261 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.261 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.261 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.261 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.261 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.262 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.262 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.262 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.262 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.262 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.262 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.262 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.262 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.262 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.263 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.263 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.263 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.263 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.263 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.263 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.263 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.263 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.263 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.263 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.263 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.264 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.264 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.264 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.264 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.264 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.264 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.264 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.264 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.264 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.264 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.264 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.264 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.264 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.265 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.265 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.265 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.265 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.265 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.265 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.265 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.265 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.265 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.265 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.265 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.265 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.266 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "iostat", 2024-09-10T08:49:01.266 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.266 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.266 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.266 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.266 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.266 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.266 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.266 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.266 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.266 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.266 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.266 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.266 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.267 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.267 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.267 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.267 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.267 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.267 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.267 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.267 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.267 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.267 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.267 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.267 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.267 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.268 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.268 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.268 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.268 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.268 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.268 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.268 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.268 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.268 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.268 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.268 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.268 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.269 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.269 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.269 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.269 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.269 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.269 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.269 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.269 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.269 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.269 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.269 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.269 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.269 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.269 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.270 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.270 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.270 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.270 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.270 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.270 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.270 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.270 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.270 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.270 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.270 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.270 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.270 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.270 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.270 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.271 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.271 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.271 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.271 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.271 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.271 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.271 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "k8sevents", 2024-09-10T08:49:01.271 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.271 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.271 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.271 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_event_retention_days": { 2024-09-10T08:49:01.271 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "ceph_event_retention_days", 2024-09-10T08:49:01.271 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.272 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.272 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.272 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "7", 2024-09-10T08:49:01.272 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.272 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.272 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.272 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T08:49:01.272 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.272 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.272 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.272 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.272 INFO:teuthology.orchestra.run.smithi089.stdout: "config_check_secs": { 2024-09-10T08:49:01.272 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "config_check_secs", 2024-09-10T08:49:01.272 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.273 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.273 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.273 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:01.273 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "10", 2024-09-10T08:49:01.273 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.273 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.273 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T08:49:01.273 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.273 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.273 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.273 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.273 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.273 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.273 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.273 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.274 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.274 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.274 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.274 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.274 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.274 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.274 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.274 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.274 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.274 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.274 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.274 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.274 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.274 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.274 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.275 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.275 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.275 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.275 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.275 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.275 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.275 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.275 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.275 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.275 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.275 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.275 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.275 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.275 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.276 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.276 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.276 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.276 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.276 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.276 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.276 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.276 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.276 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.276 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.276 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.276 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.276 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.277 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.277 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.277 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.277 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.277 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.277 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.277 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.277 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.277 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.277 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.277 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.277 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.277 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.278 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.278 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.278 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.278 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.278 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.278 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.278 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.278 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.278 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.278 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.278 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.278 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.278 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.278 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.279 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "localpool", 2024-09-10T08:49:01.279 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.279 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.279 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.279 INFO:teuthology.orchestra.run.smithi089.stdout: "failure_domain": { 2024-09-10T08:49:01.279 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "failure_domain", 2024-09-10T08:49:01.279 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.279 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.279 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.279 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "host", 2024-09-10T08:49:01.279 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.279 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.279 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.280 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "failure domain for any created local pool", 2024-09-10T08:49:01.280 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T08:49:01.280 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.280 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.280 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.280 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.280 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.280 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.280 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.280 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.280 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.280 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.280 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.281 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.281 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.281 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.281 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.281 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.281 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.281 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.281 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.281 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.281 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.281 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.281 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.281 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.281 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.282 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.282 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.282 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.282 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.282 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.282 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.282 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.282 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.282 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.282 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.282 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.282 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.282 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.282 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.283 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.283 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.283 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.283 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.283 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.283 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.283 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.283 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.283 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.283 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.283 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.283 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.283 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.284 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.284 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.284 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.284 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.284 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.284 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.284 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.284 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.284 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.284 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.284 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.284 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.284 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.284 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.285 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.285 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.285 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.285 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.285 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.285 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.285 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.285 INFO:teuthology.orchestra.run.smithi089.stdout: "min_size": { 2024-09-10T08:49:01.285 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "min_size", 2024-09-10T08:49:01.285 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.286 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.286 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.286 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.286 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.286 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.286 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.286 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default min_size for any created local pool", 2024-09-10T08:49:01.286 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T08:49:01.286 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.286 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.286 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.286 INFO:teuthology.orchestra.run.smithi089.stdout: "num_rep": { 2024-09-10T08:49:01.287 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "num_rep", 2024-09-10T08:49:01.287 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.287 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.287 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.287 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3", 2024-09-10T08:49:01.287 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.287 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.287 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.287 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default replica count for any created local pool", 2024-09-10T08:49:01.287 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.287 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.287 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.287 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.288 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_num": { 2024-09-10T08:49:01.288 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pg_num", 2024-09-10T08:49:01.288 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.288 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.288 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.288 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "128", 2024-09-10T08:49:01.288 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.288 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.288 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.288 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T08:49:01.289 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.289 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.289 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.289 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.289 INFO:teuthology.orchestra.run.smithi089.stdout: "prefix": { 2024-09-10T08:49:01.289 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prefix", 2024-09-10T08:49:01.289 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.289 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.289 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.289 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.289 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.289 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.289 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.289 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "name prefix for any created local pool", 2024-09-10T08:49:01.290 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.290 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.290 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.290 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.290 INFO:teuthology.orchestra.run.smithi089.stdout: "subtree": { 2024-09-10T08:49:01.290 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "subtree", 2024-09-10T08:49:01.290 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.290 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.290 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.290 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "rack", 2024-09-10T08:49:01.291 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.291 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.291 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.291 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T08:49:01.291 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T08:49:01.291 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.291 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.291 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.291 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.291 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.291 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.291 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mds_autoscaler", 2024-09-10T08:49:01.291 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.292 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.292 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.292 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.292 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.292 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.292 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.292 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.292 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.292 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.292 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.292 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.293 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.293 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.293 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.293 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.293 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.293 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.293 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.293 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.293 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.293 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.293 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.294 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.294 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.294 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.294 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.294 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.294 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.294 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.294 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.294 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.294 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.294 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.294 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.295 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.295 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.295 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.295 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.295 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.295 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.295 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.295 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.295 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.295 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.296 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.296 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.296 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.296 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.296 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.296 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.296 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.296 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.296 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.296 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.297 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.297 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.297 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.297 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.297 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.297 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.297 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.297 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.297 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.297 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.298 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.298 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.298 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.298 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.298 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.298 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.298 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.298 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.298 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.298 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.298 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mirroring", 2024-09-10T08:49:01.299 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.299 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.299 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.299 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.299 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.299 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.299 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.299 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.299 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.299 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.299 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.299 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.299 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.300 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.300 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.300 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.300 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.300 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.300 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.300 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.300 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.300 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.300 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.300 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.300 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.300 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.301 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.301 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.301 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.301 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.301 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.301 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.301 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.301 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.301 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.301 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.301 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.301 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.301 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.301 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.301 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.302 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.302 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.302 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.302 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.302 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.302 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.302 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.302 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.302 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.302 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.302 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.303 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.303 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.303 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.303 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.303 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.303 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.303 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.303 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.303 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.303 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.303 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.303 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.303 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.304 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.304 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.304 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.304 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.304 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.304 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.304 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.304 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.304 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.304 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.304 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.304 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "nfs", 2024-09-10T08:49:01.304 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.305 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.305 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.305 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.305 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.305 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.305 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.305 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.305 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.305 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.305 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.305 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.305 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.305 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.305 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.306 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.306 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.306 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.306 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.306 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.306 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.306 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.306 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.306 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.306 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.306 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.306 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.306 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.306 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.307 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.307 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.307 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.307 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.307 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.307 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.307 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.307 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.307 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.307 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.307 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.307 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.307 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.307 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.308 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.308 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.308 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.308 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.308 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.308 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.308 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.308 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.308 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.308 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.308 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.308 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.308 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.309 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.309 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.309 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.309 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.309 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.309 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.309 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.309 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.309 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.309 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.309 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.309 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.310 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.310 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.310 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.310 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.310 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.310 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.310 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.310 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.310 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "orchestrator", 2024-09-10T08:49:01.310 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.310 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.310 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.310 INFO:teuthology.orchestra.run.smithi089.stdout: "fail_fs": { 2024-09-10T08:49:01.310 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "fail_fs", 2024-09-10T08:49:01.311 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.311 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.311 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.311 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.311 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.311 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.311 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.311 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T08:49:01.311 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.311 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.311 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.311 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.311 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.311 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.312 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.312 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.312 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.312 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.312 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.312 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.312 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.312 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.312 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.312 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.312 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.312 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.312 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.313 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.313 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.313 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.313 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.313 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.313 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.313 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.313 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.313 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.313 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.313 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.313 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.313 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.314 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.314 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.314 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.314 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.314 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.314 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.314 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.314 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.314 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.314 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.314 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.314 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.314 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.314 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.315 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.315 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.315 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.315 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.315 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.315 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.315 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.315 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.315 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.315 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.315 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.315 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.315 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.315 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.315 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.316 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.316 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.316 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.316 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.316 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.316 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.316 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.316 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.316 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.316 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.316 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.316 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.317 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.317 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator": { 2024-09-10T08:49:01.317 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "orchestrator", 2024-09-10T08:49:01.317 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.317 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.317 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.317 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.317 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.317 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.317 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.317 INFO:teuthology.orchestra.run.smithi089.stdout: "cephadm", 2024-09-10T08:49:01.317 INFO:teuthology.orchestra.run.smithi089.stdout: "rook", 2024-09-10T08:49:01.318 INFO:teuthology.orchestra.run.smithi089.stdout: "test_orchestrator" 2024-09-10T08:49:01.318 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.318 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Orchestrator backend", 2024-09-10T08:49:01.318 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.318 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.318 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.318 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.318 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.318 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.318 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.318 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "osd_perf_query", 2024-09-10T08:49:01.318 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.319 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.319 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.319 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.319 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.319 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.319 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.319 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.319 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.319 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.319 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.319 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.319 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.319 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.320 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.320 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.320 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.320 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.320 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.320 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.320 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.320 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.320 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.320 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.320 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.320 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.320 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.321 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.321 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.321 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.321 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.321 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.321 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.321 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.321 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.321 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.321 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.321 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.321 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.321 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.322 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.322 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.322 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.322 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.322 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.322 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.322 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.322 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.322 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.322 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.322 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.322 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.322 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.322 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.323 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.323 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.323 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.323 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.323 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.323 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.323 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.323 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.323 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.323 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.323 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.323 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.323 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.324 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.324 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.324 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.324 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.324 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.324 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.324 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.324 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.324 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.324 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "osd_support", 2024-09-10T08:49:01.324 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.324 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.324 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.324 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.325 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.325 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.325 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.325 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.325 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.325 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.325 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.325 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.325 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.325 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.326 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.326 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.326 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.326 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.326 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.326 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.326 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.326 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.326 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.326 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.326 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.326 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.327 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.327 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.327 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.327 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.327 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.327 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.327 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.327 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.327 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.327 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.327 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.328 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.328 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.328 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.328 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.328 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.328 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.328 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.328 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.328 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.328 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.328 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.328 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.329 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.329 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.329 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.329 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.329 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.329 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.329 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.329 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.329 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.329 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.330 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.330 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.330 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.330 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.330 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.330 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.330 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.330 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.330 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.330 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.330 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.330 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.331 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.331 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.331 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.331 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.331 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.331 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "pg_autoscaler", 2024-09-10T08:49:01.331 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.331 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.331 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.331 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.332 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.332 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.332 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.332 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.332 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.332 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.332 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.332 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.332 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.332 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.332 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.332 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.333 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.333 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.333 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.333 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.333 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.333 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.333 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.333 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.333 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.333 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.333 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.333 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.333 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.334 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.334 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.334 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.334 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.334 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.334 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.334 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.334 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.334 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.334 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.334 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.334 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.334 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.334 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.335 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.335 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.335 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.335 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.335 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.335 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.335 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.335 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.335 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.335 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.335 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.335 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.335 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.335 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.335 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.336 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.336 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.336 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.336 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.336 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.336 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.336 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.336 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.336 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.336 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.336 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.336 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.336 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.337 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.337 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.337 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-10T08:49:01.337 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-10T08:49:01.337 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.337 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.337 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.337 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-10T08:49:01.337 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.337 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.337 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.337 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.337 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.337 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.337 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.338 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.338 INFO:teuthology.orchestra.run.smithi089.stdout: "threshold": { 2024-09-10T08:49:01.338 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "threshold", 2024-09-10T08:49:01.338 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:01.338 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.338 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.338 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "3.0", 2024-09-10T08:49:01.338 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1.0", 2024-09-10T08:49:01.338 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.338 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.338 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "scaling threshold", 2024-09-10T08:49:01.338 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T08:49:01.338 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.338 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.339 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.339 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.339 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.339 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.339 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "progress", 2024-09-10T08:49:01.339 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.339 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.339 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.339 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_pg_recovery_event": { 2024-09-10T08:49:01.339 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_pg_recovery_event", 2024-09-10T08:49:01.339 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.339 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.339 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.340 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.340 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.340 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.340 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.340 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T08:49:01.340 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.340 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.340 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.340 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.340 INFO:teuthology.orchestra.run.smithi089.stdout: "enabled": { 2024-09-10T08:49:01.340 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enabled", 2024-09-10T08:49:01.340 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.340 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.340 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.341 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.341 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.341 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.341 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.341 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.341 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.341 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.341 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.341 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.341 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.341 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.341 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.341 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.341 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.342 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.342 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.342 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.342 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.342 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.342 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.342 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.342 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.342 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.342 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.342 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.342 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.342 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.342 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.343 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.343 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.343 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.343 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.343 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.343 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.343 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.343 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.343 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.343 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.343 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.343 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.343 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.343 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.343 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.344 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.344 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.344 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.344 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.344 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.344 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.344 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.344 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.344 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.344 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.344 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.344 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.344 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.344 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.345 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.345 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.345 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.345 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.345 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.345 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.345 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.345 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.345 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.345 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.345 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.345 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.345 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.345 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.346 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.346 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.346 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.346 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.346 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.346 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.346 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.346 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.346 INFO:teuthology.orchestra.run.smithi089.stdout: "max_completed_events": { 2024-09-10T08:49:01.346 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_completed_events", 2024-09-10T08:49:01.346 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.346 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.346 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.347 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "50", 2024-09-10T08:49:01.347 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.347 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.347 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.347 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "number of past completed events to remember", 2024-09-10T08:49:01.347 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.347 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.347 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.347 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.347 INFO:teuthology.orchestra.run.smithi089.stdout: "sleep_interval": { 2024-09-10T08:49:01.347 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "sleep_interval", 2024-09-10T08:49:01.347 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.347 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.347 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.347 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "5", 2024-09-10T08:49:01.348 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.348 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.348 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.348 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "how long the module is going to sleep", 2024-09-10T08:49:01.348 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.348 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.348 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.348 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.348 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.348 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.348 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.348 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "prometheus", 2024-09-10T08:49:01.348 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.349 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.349 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.349 INFO:teuthology.orchestra.run.smithi089.stdout: "cache": { 2024-09-10T08:49:01.349 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "cache", 2024-09-10T08:49:01.349 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.349 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.349 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.349 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.349 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.349 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.349 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.349 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.349 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.350 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.350 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.350 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.350 INFO:teuthology.orchestra.run.smithi089.stdout: "exclude_perf_counters": { 2024-09-10T08:49:01.350 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "exclude_perf_counters", 2024-09-10T08:49:01.350 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.350 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.350 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.350 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.350 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.350 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.350 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.350 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T08:49:01.350 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T08:49:01.351 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.351 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.351 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.351 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.351 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.351 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.351 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.351 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.351 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.351 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.351 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.351 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.351 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.351 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.352 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.352 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.352 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.352 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.352 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.352 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.352 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.352 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.352 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.352 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.352 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.352 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.352 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.352 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.353 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.353 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.353 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.353 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.353 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.353 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.353 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.353 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.353 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.353 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.353 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.353 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.353 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.353 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.354 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.354 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.354 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.354 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.354 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.354 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.354 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.354 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.354 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.354 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.354 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.354 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.354 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.354 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.355 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.355 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.355 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.355 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.355 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.355 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.355 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.355 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.355 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.355 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.355 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.355 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.355 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.355 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.356 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.356 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.356 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.356 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_stats_pools": { 2024-09-10T08:49:01.356 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_stats_pools", 2024-09-10T08:49:01.356 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.356 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.356 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.356 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.356 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.356 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.356 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.356 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.357 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.357 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.357 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.357 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.357 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T08:49:01.357 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T08:49:01.357 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.357 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.357 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.357 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "300", 2024-09-10T08:49:01.357 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.357 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.357 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.357 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.358 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.358 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.358 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.358 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.358 INFO:teuthology.orchestra.run.smithi089.stdout: "scrape_interval": { 2024-09-10T08:49:01.358 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "scrape_interval", 2024-09-10T08:49:01.358 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:01.358 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.358 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.358 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15.0", 2024-09-10T08:49:01.358 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.358 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.358 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.358 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.359 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.359 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.359 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.359 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.359 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2024-09-10T08:49:01.359 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2024-09-10T08:49:01.359 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.359 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.359 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.359 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "::", 2024-09-10T08:49:01.359 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.359 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.359 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.360 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T08:49:01.360 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.360 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.360 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.360 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.360 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2024-09-10T08:49:01.360 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2024-09-10T08:49:01.360 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.360 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.360 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.360 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "9283", 2024-09-10T08:49:01.360 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.361 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.361 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.361 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T08:49:01.361 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.361 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.361 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.361 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.361 INFO:teuthology.orchestra.run.smithi089.stdout: "stale_cache_strategy": { 2024-09-10T08:49:01.361 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "stale_cache_strategy", 2024-09-10T08:49:01.361 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.361 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.361 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.362 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "log", 2024-09-10T08:49:01.362 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.362 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.362 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.362 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.362 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.362 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.362 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.362 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.362 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_behaviour": { 2024-09-10T08:49:01.362 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_behaviour", 2024-09-10T08:49:01.363 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.363 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.363 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.363 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "default", 2024-09-10T08:49:01.363 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.363 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.363 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.363 INFO:teuthology.orchestra.run.smithi089.stdout: "default", 2024-09-10T08:49:01.363 INFO:teuthology.orchestra.run.smithi089.stdout: "error" 2024-09-10T08:49:01.363 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.363 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.363 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.363 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.363 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.364 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.364 INFO:teuthology.orchestra.run.smithi089.stdout: "standby_error_status_code": { 2024-09-10T08:49:01.364 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "standby_error_status_code", 2024-09-10T08:49:01.364 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.364 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.364 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.364 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "500", 2024-09-10T08:49:01.364 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "400", 2024-09-10T08:49:01.364 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "599", 2024-09-10T08:49:01.364 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.364 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.365 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.365 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.365 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.365 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.365 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.365 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.365 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.365 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_support", 2024-09-10T08:49:01.365 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.365 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.365 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.365 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.365 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.366 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.366 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.366 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.366 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.366 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.366 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.366 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.366 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.366 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.366 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.366 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.367 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.367 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.367 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.367 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.367 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.367 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.367 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.367 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.367 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.367 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.367 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.367 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.367 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.368 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.368 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.368 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.368 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.368 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.368 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.368 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.368 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.368 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.368 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.368 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.368 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.369 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.369 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.369 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.369 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.369 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.369 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.369 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.369 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.369 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.369 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.369 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.370 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.370 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.370 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.370 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.370 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.370 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.370 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.370 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.370 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.370 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.370 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.370 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.370 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.370 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.371 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.371 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.371 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.371 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.371 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.371 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.371 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.371 INFO:teuthology.orchestra.run.smithi089.stdout: "max_concurrent_snap_create": { 2024-09-10T08:49:01.371 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_concurrent_snap_create", 2024-09-10T08:49:01.371 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.371 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.371 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.371 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10", 2024-09-10T08:49:01.371 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.372 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.372 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.372 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.372 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.372 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.372 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.372 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.372 INFO:teuthology.orchestra.run.smithi089.stdout: "mirror_snapshot_schedule": { 2024-09-10T08:49:01.372 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T08:49:01.372 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.372 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.372 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.372 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.372 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.373 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.373 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.373 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.373 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.373 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.373 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.373 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.373 INFO:teuthology.orchestra.run.smithi089.stdout: "trash_purge_schedule": { 2024-09-10T08:49:01.373 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "trash_purge_schedule", 2024-09-10T08:49:01.373 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.373 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.373 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.373 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.373 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.374 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.374 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.374 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.374 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.374 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.374 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.374 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.374 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.374 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.374 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.374 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "restful", 2024-09-10T08:49:01.374 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.374 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.374 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.375 INFO:teuthology.orchestra.run.smithi089.stdout: "enable_auth": { 2024-09-10T08:49:01.375 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enable_auth", 2024-09-10T08:49:01.375 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.375 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.375 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.375 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.375 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.375 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.375 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.375 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.375 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.375 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.375 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.375 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.375 INFO:teuthology.orchestra.run.smithi089.stdout: "key_file": { 2024-09-10T08:49:01.376 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "key_file", 2024-09-10T08:49:01.376 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.376 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.376 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.376 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.376 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.376 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.376 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.376 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.376 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.376 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.376 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.376 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.376 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.377 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.377 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.377 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.377 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.377 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.377 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.377 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.377 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.377 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.377 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.377 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.377 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.377 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.377 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.378 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.378 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.378 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.378 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.378 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.378 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.378 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.378 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.378 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.378 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.378 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.378 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.378 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.378 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.378 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.379 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.379 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.379 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.379 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.379 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.379 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.379 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.379 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.379 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.379 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.379 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.379 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.379 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.379 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.379 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.380 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.380 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.380 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.380 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.380 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.380 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.380 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.380 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.380 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.380 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.380 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.380 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.380 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.381 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.381 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.381 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.381 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.381 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.381 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.381 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.381 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.381 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.381 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.381 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.381 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.381 INFO:teuthology.orchestra.run.smithi089.stdout: "server_addr": { 2024-09-10T08:49:01.381 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_addr", 2024-09-10T08:49:01.382 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.382 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.382 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.382 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.382 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.382 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.382 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.382 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.382 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.382 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.382 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.382 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.382 INFO:teuthology.orchestra.run.smithi089.stdout: "server_port": { 2024-09-10T08:49:01.383 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "server_port", 2024-09-10T08:49:01.383 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.383 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.383 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.383 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.383 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.383 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.383 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.383 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.383 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.383 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.383 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.383 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.383 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.384 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.384 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.384 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rgw", 2024-09-10T08:49:01.384 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.384 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.384 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.384 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.384 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.384 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.384 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.384 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.384 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.384 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.384 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.385 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.385 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.385 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.385 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.385 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.385 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.385 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.385 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.385 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.385 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.385 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.385 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.385 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.385 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.386 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.386 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.386 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.386 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.386 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.386 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.386 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.386 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.386 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.386 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.386 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.386 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.386 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.386 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.386 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.387 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.387 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.387 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.387 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.387 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.387 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.387 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.387 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.387 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.387 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.387 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.387 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.387 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.387 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.388 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.388 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.388 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.388 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.388 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.388 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.388 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.388 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.388 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.388 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.388 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.388 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.388 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.388 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.388 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.389 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.389 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.389 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.389 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.389 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.389 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.389 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.389 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rook", 2024-09-10T08:49:01.389 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.389 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.389 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.389 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.389 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.390 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.390 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.390 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.390 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.390 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.390 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.390 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.390 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.390 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.390 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.390 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.390 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.390 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.390 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.391 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.391 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.391 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.391 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.391 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.391 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.391 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.391 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.391 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.391 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.391 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.391 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.391 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.391 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.391 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.392 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.392 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.392 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.392 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.392 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.392 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.392 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.392 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.392 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.392 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.392 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.392 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.392 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.393 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.393 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.393 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.393 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.393 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.393 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.393 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.393 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.393 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.393 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.393 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.393 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.393 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.393 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.393 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.394 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.394 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.394 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.394 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.394 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.394 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.394 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.394 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.394 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.394 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.394 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.394 INFO:teuthology.orchestra.run.smithi089.stdout: "storage_class": { 2024-09-10T08:49:01.394 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "storage_class", 2024-09-10T08:49:01.395 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.395 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.395 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.395 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "local", 2024-09-10T08:49:01.395 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.395 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.395 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.395 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T08:49:01.395 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.395 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.395 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.395 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.395 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.395 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.395 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.396 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "selftest", 2024-09-10T08:49:01.396 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.396 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.396 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.396 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.396 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.397 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.397 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.397 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.397 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.397 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.397 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.397 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.397 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.398 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.398 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.398 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.398 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.398 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.398 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.398 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.398 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.398 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.398 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.398 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.398 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.398 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.398 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.399 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.399 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.399 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.399 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.399 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.399 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.399 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.399 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.399 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.399 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.399 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.399 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.399 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.400 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.400 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.400 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.400 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.400 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.400 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.400 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.400 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.400 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.400 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.400 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.400 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.400 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.400 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.401 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.401 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.401 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.401 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.401 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.401 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.401 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.401 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.401 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.401 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.401 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.401 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.401 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.401 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.401 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.402 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.402 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.402 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.402 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.402 INFO:teuthology.orchestra.run.smithi089.stdout: "roption1": { 2024-09-10T08:49:01.402 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "roption1", 2024-09-10T08:49:01.402 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.402 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.402 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.402 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.402 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.402 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.402 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.403 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.403 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.403 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.403 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.403 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.403 INFO:teuthology.orchestra.run.smithi089.stdout: "roption2": { 2024-09-10T08:49:01.403 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "roption2", 2024-09-10T08:49:01.403 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.403 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.403 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.403 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "xyz", 2024-09-10T08:49:01.403 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.403 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.403 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.404 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.404 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.404 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.404 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.404 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.404 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption1": { 2024-09-10T08:49:01.404 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption1", 2024-09-10T08:49:01.404 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.404 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.404 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.404 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.404 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.404 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.404 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.405 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.405 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.405 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.405 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.405 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.405 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption2": { 2024-09-10T08:49:01.405 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption2", 2024-09-10T08:49:01.405 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.405 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.405 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.405 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.405 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.405 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.405 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.406 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.406 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.406 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.406 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.406 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.406 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption3": { 2024-09-10T08:49:01.406 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption3", 2024-09-10T08:49:01.406 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "float", 2024-09-10T08:49:01.406 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.406 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.406 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.406 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.406 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.407 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.407 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.407 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.407 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.407 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.407 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.407 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption4": { 2024-09-10T08:49:01.407 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption4", 2024-09-10T08:49:01.407 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.407 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.407 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.407 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.408 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.408 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.408 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.408 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.408 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.408 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.408 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.408 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.408 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption5": { 2024-09-10T08:49:01.408 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption5", 2024-09-10T08:49:01.408 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.408 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.409 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.409 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.409 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.409 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.409 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.409 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.409 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.409 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.409 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.409 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.409 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption6": { 2024-09-10T08:49:01.409 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption6", 2024-09-10T08:49:01.409 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.410 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.410 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.410 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.410 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.410 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.410 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.410 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.410 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.410 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.410 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.410 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.410 INFO:teuthology.orchestra.run.smithi089.stdout: "rwoption7": { 2024-09-10T08:49:01.410 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rwoption7", 2024-09-10T08:49:01.410 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.411 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.411 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.411 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.411 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "1", 2024-09-10T08:49:01.411 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "42", 2024-09-10T08:49:01.411 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.411 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.411 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.411 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.411 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.411 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.411 INFO:teuthology.orchestra.run.smithi089.stdout: "testkey": { 2024-09-10T08:49:01.411 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testkey", 2024-09-10T08:49:01.411 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.412 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.412 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.412 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.412 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.412 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.412 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.412 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.412 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.412 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.412 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.412 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.412 INFO:teuthology.orchestra.run.smithi089.stdout: "testlkey": { 2024-09-10T08:49:01.412 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testlkey", 2024-09-10T08:49:01.412 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.413 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.413 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.413 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.413 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.413 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.413 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.413 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.413 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.413 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.413 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.413 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.413 INFO:teuthology.orchestra.run.smithi089.stdout: "testnewline": { 2024-09-10T08:49:01.413 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "testnewline", 2024-09-10T08:49:01.413 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.414 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.414 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.414 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.414 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.414 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.414 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.414 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.414 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.414 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.414 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.414 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.414 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.414 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.414 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snap_schedule", 2024-09-10T08:49:01.415 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.415 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.415 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.415 INFO:teuthology.orchestra.run.smithi089.stdout: "allow_m_granularity": { 2024-09-10T08:49:01.415 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "allow_m_granularity", 2024-09-10T08:49:01.415 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.415 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.415 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.415 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.415 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.415 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.415 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.415 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T08:49:01.415 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.416 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.416 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.416 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.416 INFO:teuthology.orchestra.run.smithi089.stdout: "dump_on_update": { 2024-09-10T08:49:01.416 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "dump_on_update", 2024-09-10T08:49:01.416 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.416 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.416 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.416 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.416 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.416 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.416 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.416 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "dump database to debug log on update", 2024-09-10T08:49:01.416 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.417 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.417 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.417 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.417 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.417 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.417 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.417 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.417 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.417 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.417 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.417 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.417 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.417 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.417 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.418 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.418 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.418 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.418 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.418 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.418 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.418 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.418 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.418 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.418 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.418 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.418 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.418 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.418 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.419 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.419 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.419 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.419 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.419 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.419 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.419 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.419 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.419 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.419 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.419 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.419 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.420 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.420 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.420 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.420 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.420 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.420 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.420 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.420 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.420 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.420 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.420 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.420 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.420 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.421 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.421 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.421 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.421 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.421 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.421 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.421 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.421 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.421 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.421 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.421 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.421 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.421 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.421 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.422 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.422 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.422 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.422 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.422 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "stats", 2024-09-10T08:49:01.422 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.422 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.422 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.422 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.422 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.422 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.422 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.423 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.423 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.423 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.423 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.423 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.423 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.423 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.423 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.423 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.423 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.423 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.423 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.423 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.424 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.424 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.424 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.424 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.424 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.424 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.424 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.424 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.424 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.424 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.424 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.424 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.424 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.425 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.425 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.425 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.425 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.425 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.425 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.425 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.425 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.425 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.425 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.425 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.425 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.425 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.425 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.426 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.426 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.426 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.426 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.426 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.426 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.426 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.426 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.426 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.426 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.426 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.426 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.426 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.427 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.427 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.427 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.427 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.427 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.427 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.427 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.427 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.427 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.427 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.427 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.427 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.428 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.428 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.428 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "status", 2024-09-10T08:49:01.428 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.428 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.428 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.428 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.428 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.428 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.428 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.428 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.428 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.428 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.429 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.429 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.429 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.429 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.429 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.429 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.429 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.429 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.429 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.429 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.429 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.429 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.429 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.430 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.430 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.430 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.430 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.430 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.430 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.430 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.430 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.430 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.430 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.430 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.430 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.431 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.431 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.431 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.431 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.431 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.431 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.431 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.431 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.431 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.431 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.431 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.431 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.431 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.432 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.432 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.432 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.432 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.432 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.432 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.432 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.432 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.432 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.432 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.433 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.433 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.433 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.433 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.433 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.433 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.433 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.433 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.433 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.433 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.433 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.433 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.433 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.434 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.434 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "telegraf", 2024-09-10T08:49:01.434 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.434 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.434 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.434 INFO:teuthology.orchestra.run.smithi089.stdout: "address": { 2024-09-10T08:49:01.434 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "address", 2024-09-10T08:49:01.434 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.434 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.434 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.434 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T08:49:01.434 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.434 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.434 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.435 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.435 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.435 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.435 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.435 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-10T08:49:01.435 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-10T08:49:01.435 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.435 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.435 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.435 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "15", 2024-09-10T08:49:01.435 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.435 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.435 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.436 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.436 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.436 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.436 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.436 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.436 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.436 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.436 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.436 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.436 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.436 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.436 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.436 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.437 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.437 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.437 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.437 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.437 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.437 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.437 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.437 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.437 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.437 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.437 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.437 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.437 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.438 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.438 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.438 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.438 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.438 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.438 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.438 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.438 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.438 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.438 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.438 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.438 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.439 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.439 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.439 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.439 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.439 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.439 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.439 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.439 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.439 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.439 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.439 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.439 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.439 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.439 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.440 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.440 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.440 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.440 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.440 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.440 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.440 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.440 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.440 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.440 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.440 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.440 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.441 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.441 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.441 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.441 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.441 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.441 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.441 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.441 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.441 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.441 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "telemetry", 2024-09-10T08:49:01.441 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.441 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.441 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.442 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_basic": { 2024-09-10T08:49:01.442 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_basic", 2024-09-10T08:49:01.442 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.442 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.442 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.442 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.442 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.442 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.442 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.442 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T08:49:01.442 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.442 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.442 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.443 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_crash": { 2024-09-10T08:49:01.443 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_crash", 2024-09-10T08:49:01.443 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.443 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.443 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.443 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.443 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.443 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.443 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.443 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T08:49:01.443 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.443 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.443 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.444 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_device": { 2024-09-10T08:49:01.444 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_device", 2024-09-10T08:49:01.444 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.444 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.444 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.444 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.444 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.444 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.444 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.444 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T08:49:01.444 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.444 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.444 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.445 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_ident": { 2024-09-10T08:49:01.445 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_ident", 2024-09-10T08:49:01.445 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.445 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.445 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.445 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.445 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.445 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.445 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.445 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T08:49:01.445 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.445 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.446 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.446 INFO:teuthology.orchestra.run.smithi089.stdout: "channel_perf": { 2024-09-10T08:49:01.446 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "channel_perf", 2024-09-10T08:49:01.446 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.446 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.446 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.446 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.446 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.446 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.446 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.446 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T08:49:01.446 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.447 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.447 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.447 INFO:teuthology.orchestra.run.smithi089.stdout: "contact": { 2024-09-10T08:49:01.447 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "contact", 2024-09-10T08:49:01.447 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.447 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.447 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.447 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.447 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.447 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.447 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.447 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.447 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.448 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.448 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.448 INFO:teuthology.orchestra.run.smithi089.stdout: "description": { 2024-09-10T08:49:01.448 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "description", 2024-09-10T08:49:01.448 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.448 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.448 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.448 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.448 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.448 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.448 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.448 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.449 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.449 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.449 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.449 INFO:teuthology.orchestra.run.smithi089.stdout: "device_url": { 2024-09-10T08:49:01.449 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "device_url", 2024-09-10T08:49:01.449 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.449 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.449 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.449 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T08:49:01.449 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.449 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.449 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.449 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.450 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.450 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.450 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.450 INFO:teuthology.orchestra.run.smithi089.stdout: "enabled": { 2024-09-10T08:49:01.450 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "enabled", 2024-09-10T08:49:01.450 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.450 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.450 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.450 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.450 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.450 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.450 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.450 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.451 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.451 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.451 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.451 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-10T08:49:01.451 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-10T08:49:01.451 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.451 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.451 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.451 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "24", 2024-09-10T08:49:01.451 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "8", 2024-09-10T08:49:01.451 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.451 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.452 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.452 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.452 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.452 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.452 INFO:teuthology.orchestra.run.smithi089.stdout: "last_opt_revision": { 2024-09-10T08:49:01.452 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "last_opt_revision", 2024-09-10T08:49:01.452 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.452 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.452 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.452 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "1", 2024-09-10T08:49:01.452 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.452 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.452 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.453 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.453 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.453 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.453 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.453 INFO:teuthology.orchestra.run.smithi089.stdout: "leaderboard": { 2024-09-10T08:49:01.453 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "leaderboard", 2024-09-10T08:49:01.453 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.453 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.453 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.453 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.453 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.453 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.453 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.453 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.454 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.454 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.454 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.454 INFO:teuthology.orchestra.run.smithi089.stdout: "leaderboard_description": { 2024-09-10T08:49:01.454 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "leaderboard_description", 2024-09-10T08:49:01.454 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.454 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.454 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.454 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.454 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.454 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.454 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.454 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.455 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.455 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.455 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.455 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.455 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.455 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.455 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.455 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.455 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.455 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.456 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.456 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.456 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.456 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.456 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.456 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.456 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.456 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.456 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.456 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.456 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.456 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.456 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.457 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.457 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.457 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.457 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.457 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.457 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.457 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.457 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.457 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.457 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.457 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.458 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.458 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.458 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.458 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.458 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.458 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.458 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.458 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.458 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.458 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.458 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.458 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.459 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.459 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.459 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.459 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.459 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.459 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.459 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.459 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.459 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.459 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.459 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.460 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.460 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.460 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.460 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.460 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.460 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.460 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.460 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.460 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.460 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.460 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.460 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.461 INFO:teuthology.orchestra.run.smithi089.stdout: "organization": { 2024-09-10T08:49:01.461 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "organization", 2024-09-10T08:49:01.461 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.461 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.461 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.461 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.461 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.461 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.461 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.461 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.461 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.461 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.461 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.461 INFO:teuthology.orchestra.run.smithi089.stdout: "proxy": { 2024-09-10T08:49:01.462 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "proxy", 2024-09-10T08:49:01.462 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.462 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.462 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.462 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.462 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.462 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.462 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.462 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.462 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.462 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.462 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.462 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.463 INFO:teuthology.orchestra.run.smithi089.stdout: "url": { 2024-09-10T08:49:01.463 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "url", 2024-09-10T08:49:01.463 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.463 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.463 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.463 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T08:49:01.463 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.463 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.463 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.463 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.463 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.463 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.464 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.464 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.464 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.464 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.464 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.464 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "test_orchestrator", 2024-09-10T08:49:01.464 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.464 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.464 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.464 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.464 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.464 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.464 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.465 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.465 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.465 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.465 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.465 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.465 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.465 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.465 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.465 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.465 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.465 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.465 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.465 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.466 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.466 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.466 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.466 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.466 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.466 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.466 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.466 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.466 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.466 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.466 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.466 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.466 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.466 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.466 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.467 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.467 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.467 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.467 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.467 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.467 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.467 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.467 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.467 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.467 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.467 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.467 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.467 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.467 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.468 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.468 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.468 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.468 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.468 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.468 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.468 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.468 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.468 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.468 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.468 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.468 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.469 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.469 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.469 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.469 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.469 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.469 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.469 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.469 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.469 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.469 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.469 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.469 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.469 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.469 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.470 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "volumes", 2024-09-10T08:49:01.470 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.470 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.470 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.470 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.470 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.470 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.470 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.470 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.470 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.470 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.470 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.470 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.470 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.471 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.471 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.471 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.471 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.471 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.471 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.471 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.471 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.471 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.471 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.471 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.471 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.471 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.471 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.472 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.472 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.472 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.472 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.472 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.472 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.472 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.472 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.472 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.472 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.472 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.472 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.472 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.473 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.473 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.473 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.473 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.473 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.473 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.473 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.473 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.473 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.473 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.473 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.473 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.474 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.474 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.474 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.474 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.474 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.474 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.474 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.474 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.474 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.474 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.474 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.474 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.475 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.475 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.475 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.475 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.475 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.475 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.475 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.475 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.475 INFO:teuthology.orchestra.run.smithi089.stdout: "max_concurrent_clones": { 2024-09-10T08:49:01.475 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "max_concurrent_clones", 2024-09-10T08:49:01.475 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.475 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.475 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.475 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "4", 2024-09-10T08:49:01.476 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.476 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.476 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.476 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T08:49:01.476 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.476 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.476 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.476 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.476 INFO:teuthology.orchestra.run.smithi089.stdout: "periodic_async_work": { 2024-09-10T08:49:01.476 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "periodic_async_work", 2024-09-10T08:49:01.476 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.476 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.476 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.477 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.477 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.477 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.477 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.477 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Periodically check for async work", 2024-09-10T08:49:01.477 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.477 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.477 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.477 INFO:teuthology.orchestra.run.smithi089.stdout: "snapshot_clone_delay": { 2024-09-10T08:49:01.477 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snapshot_clone_delay", 2024-09-10T08:49:01.477 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.477 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.478 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.478 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "0", 2024-09-10T08:49:01.478 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.478 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.478 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.478 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T08:49:01.478 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.478 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.478 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.478 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.478 INFO:teuthology.orchestra.run.smithi089.stdout: "snapshot_clone_no_wait": { 2024-09-10T08:49:01.478 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T08:49:01.478 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.479 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.479 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.479 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "True", 2024-09-10T08:49:01.479 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.479 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.479 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.479 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T08:49:01.479 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.479 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.479 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.479 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.479 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.479 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.480 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.480 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix", 2024-09-10T08:49:01.480 INFO:teuthology.orchestra.run.smithi089.stdout: "can_run": true, 2024-09-10T08:49:01.480 INFO:teuthology.orchestra.run.smithi089.stdout: "error_string": "", 2024-09-10T08:49:01.480 INFO:teuthology.orchestra.run.smithi089.stdout: "module_options": { 2024-09-10T08:49:01.480 INFO:teuthology.orchestra.run.smithi089.stdout: "discovery_interval": { 2024-09-10T08:49:01.480 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "discovery_interval", 2024-09-10T08:49:01.480 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "uint", 2024-09-10T08:49:01.480 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.480 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.480 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "100", 2024-09-10T08:49:01.480 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.480 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.481 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.481 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.481 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.481 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.481 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.481 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.481 INFO:teuthology.orchestra.run.smithi089.stdout: "identifier": { 2024-09-10T08:49:01.481 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "identifier", 2024-09-10T08:49:01.481 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.481 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.481 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.481 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.481 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.482 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.482 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.482 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.482 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.482 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.482 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.482 INFO:teuthology.orchestra.run.smithi089.stdout: "interval": { 2024-09-10T08:49:01.482 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "interval", 2024-09-10T08:49:01.482 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "secs", 2024-09-10T08:49:01.482 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.482 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.482 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "60", 2024-09-10T08:49:01.482 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.483 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.483 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.483 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.483 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.483 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.483 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.483 INFO:teuthology.orchestra.run.smithi089.stdout: "log_level": { 2024-09-10T08:49:01.483 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_level", 2024-09-10T08:49:01.483 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.483 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.483 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.483 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.483 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.484 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.484 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.484 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.484 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.484 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.484 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.484 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.484 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.484 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.484 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.484 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.484 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.484 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.485 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster": { 2024-09-10T08:49:01.485 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster", 2024-09-10T08:49:01.485 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.485 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.485 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.485 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.485 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.485 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.485 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.485 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.485 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.485 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.485 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.486 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_cluster_level": { 2024-09-10T08:49:01.486 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_cluster_level", 2024-09-10T08:49:01.486 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.486 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.486 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.486 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "info", 2024-09-10T08:49:01.486 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.486 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.486 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [ 2024-09-10T08:49:01.486 INFO:teuthology.orchestra.run.smithi089.stdout: "", 2024-09-10T08:49:01.486 INFO:teuthology.orchestra.run.smithi089.stdout: "critical", 2024-09-10T08:49:01.486 INFO:teuthology.orchestra.run.smithi089.stdout: "debug", 2024-09-10T08:49:01.486 INFO:teuthology.orchestra.run.smithi089.stdout: "error", 2024-09-10T08:49:01.486 INFO:teuthology.orchestra.run.smithi089.stdout: "info", 2024-09-10T08:49:01.487 INFO:teuthology.orchestra.run.smithi089.stdout: "warning" 2024-09-10T08:49:01.487 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.487 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.487 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.487 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.487 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.487 INFO:teuthology.orchestra.run.smithi089.stdout: "log_to_file": { 2024-09-10T08:49:01.487 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "log_to_file", 2024-09-10T08:49:01.487 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "bool", 2024-09-10T08:49:01.487 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.487 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 1, 2024-09-10T08:49:01.487 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "False", 2024-09-10T08:49:01.487 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.487 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.488 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.488 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.488 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.488 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.488 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.488 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_host": { 2024-09-10T08:49:01.488 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_host", 2024-09-10T08:49:01.488 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.488 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.488 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.488 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "", 2024-09-10T08:49:01.488 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.488 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.489 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.489 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.489 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.489 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.489 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.489 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_port": { 2024-09-10T08:49:01.489 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_port", 2024-09-10T08:49:01.489 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "int", 2024-09-10T08:49:01.489 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.489 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.489 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "10051", 2024-09-10T08:49:01.489 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.489 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.490 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.490 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.490 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.490 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.490 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.490 INFO:teuthology.orchestra.run.smithi089.stdout: "zabbix_sender": { 2024-09-10T08:49:01.490 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "zabbix_sender", 2024-09-10T08:49:01.490 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "str", 2024-09-10T08:49:01.490 INFO:teuthology.orchestra.run.smithi089.stdout: "level": "advanced", 2024-09-10T08:49:01.490 INFO:teuthology.orchestra.run.smithi089.stdout: "flags": 0, 2024-09-10T08:49:01.490 INFO:teuthology.orchestra.run.smithi089.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T08:49:01.490 INFO:teuthology.orchestra.run.smithi089.stdout: "min": "", 2024-09-10T08:49:01.490 INFO:teuthology.orchestra.run.smithi089.stdout: "max": "", 2024-09-10T08:49:01.491 INFO:teuthology.orchestra.run.smithi089.stdout: "enum_allowed": [], 2024-09-10T08:49:01.491 INFO:teuthology.orchestra.run.smithi089.stdout: "desc": "", 2024-09-10T08:49:01.491 INFO:teuthology.orchestra.run.smithi089.stdout: "long_desc": "", 2024-09-10T08:49:01.491 INFO:teuthology.orchestra.run.smithi089.stdout: "tags": [], 2024-09-10T08:49:01.491 INFO:teuthology.orchestra.run.smithi089.stdout: "see_also": [] 2024-09-10T08:49:01.491 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.491 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.491 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.491 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.491 INFO:teuthology.orchestra.run.smithi089.stdout: "services": { 2024-09-10T08:49:01.491 INFO:teuthology.orchestra.run.smithi089.stdout: "dashboard": "https://172.21.15.89:8443/" 2024-09-10T08:49:01.491 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.491 INFO:teuthology.orchestra.run.smithi089.stdout: "always_on_modules": { 2024-09-10T08:49:01.492 INFO:teuthology.orchestra.run.smithi089.stdout: "octopus": [ 2024-09-10T08:49:01.492 INFO:teuthology.orchestra.run.smithi089.stdout: "balancer", 2024-09-10T08:49:01.492 INFO:teuthology.orchestra.run.smithi089.stdout: "crash", 2024-09-10T08:49:01.492 INFO:teuthology.orchestra.run.smithi089.stdout: "devicehealth", 2024-09-10T08:49:01.492 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator", 2024-09-10T08:49:01.492 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_autoscaler", 2024-09-10T08:49:01.492 INFO:teuthology.orchestra.run.smithi089.stdout: "progress", 2024-09-10T08:49:01.492 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_support", 2024-09-10T08:49:01.492 INFO:teuthology.orchestra.run.smithi089.stdout: "status", 2024-09-10T08:49:01.492 INFO:teuthology.orchestra.run.smithi089.stdout: "telemetry", 2024-09-10T08:49:01.492 INFO:teuthology.orchestra.run.smithi089.stdout: "volumes" 2024-09-10T08:49:01.492 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.492 INFO:teuthology.orchestra.run.smithi089.stdout: "pacific": [ 2024-09-10T08:49:01.492 INFO:teuthology.orchestra.run.smithi089.stdout: "balancer", 2024-09-10T08:49:01.493 INFO:teuthology.orchestra.run.smithi089.stdout: "crash", 2024-09-10T08:49:01.493 INFO:teuthology.orchestra.run.smithi089.stdout: "devicehealth", 2024-09-10T08:49:01.493 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator", 2024-09-10T08:49:01.493 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_autoscaler", 2024-09-10T08:49:01.493 INFO:teuthology.orchestra.run.smithi089.stdout: "progress", 2024-09-10T08:49:01.493 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_support", 2024-09-10T08:49:01.493 INFO:teuthology.orchestra.run.smithi089.stdout: "status", 2024-09-10T08:49:01.493 INFO:teuthology.orchestra.run.smithi089.stdout: "telemetry", 2024-09-10T08:49:01.493 INFO:teuthology.orchestra.run.smithi089.stdout: "volumes" 2024-09-10T08:49:01.493 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.493 INFO:teuthology.orchestra.run.smithi089.stdout: "quincy": [ 2024-09-10T08:49:01.493 INFO:teuthology.orchestra.run.smithi089.stdout: "balancer", 2024-09-10T08:49:01.494 INFO:teuthology.orchestra.run.smithi089.stdout: "crash", 2024-09-10T08:49:01.494 INFO:teuthology.orchestra.run.smithi089.stdout: "devicehealth", 2024-09-10T08:49:01.494 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator", 2024-09-10T08:49:01.494 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_autoscaler", 2024-09-10T08:49:01.494 INFO:teuthology.orchestra.run.smithi089.stdout: "progress", 2024-09-10T08:49:01.494 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_support", 2024-09-10T08:49:01.494 INFO:teuthology.orchestra.run.smithi089.stdout: "status", 2024-09-10T08:49:01.494 INFO:teuthology.orchestra.run.smithi089.stdout: "telemetry", 2024-09-10T08:49:01.494 INFO:teuthology.orchestra.run.smithi089.stdout: "volumes" 2024-09-10T08:49:01.494 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.494 INFO:teuthology.orchestra.run.smithi089.stdout: "reef": [ 2024-09-10T08:49:01.494 INFO:teuthology.orchestra.run.smithi089.stdout: "balancer", 2024-09-10T08:49:01.494 INFO:teuthology.orchestra.run.smithi089.stdout: "crash", 2024-09-10T08:49:01.494 INFO:teuthology.orchestra.run.smithi089.stdout: "devicehealth", 2024-09-10T08:49:01.495 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator", 2024-09-10T08:49:01.495 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_autoscaler", 2024-09-10T08:49:01.495 INFO:teuthology.orchestra.run.smithi089.stdout: "progress", 2024-09-10T08:49:01.495 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_support", 2024-09-10T08:49:01.495 INFO:teuthology.orchestra.run.smithi089.stdout: "status", 2024-09-10T08:49:01.495 INFO:teuthology.orchestra.run.smithi089.stdout: "telemetry", 2024-09-10T08:49:01.495 INFO:teuthology.orchestra.run.smithi089.stdout: "volumes" 2024-09-10T08:49:01.495 INFO:teuthology.orchestra.run.smithi089.stdout: ], 2024-09-10T08:49:01.495 INFO:teuthology.orchestra.run.smithi089.stdout: "squid": [ 2024-09-10T08:49:01.495 INFO:teuthology.orchestra.run.smithi089.stdout: "balancer", 2024-09-10T08:49:01.495 INFO:teuthology.orchestra.run.smithi089.stdout: "crash", 2024-09-10T08:49:01.495 INFO:teuthology.orchestra.run.smithi089.stdout: "devicehealth", 2024-09-10T08:49:01.495 INFO:teuthology.orchestra.run.smithi089.stdout: "orchestrator", 2024-09-10T08:49:01.495 INFO:teuthology.orchestra.run.smithi089.stdout: "pg_autoscaler", 2024-09-10T08:49:01.496 INFO:teuthology.orchestra.run.smithi089.stdout: "progress", 2024-09-10T08:49:01.496 INFO:teuthology.orchestra.run.smithi089.stdout: "rbd_support", 2024-09-10T08:49:01.496 INFO:teuthology.orchestra.run.smithi089.stdout: "status", 2024-09-10T08:49:01.496 INFO:teuthology.orchestra.run.smithi089.stdout: "telemetry", 2024-09-10T08:49:01.496 INFO:teuthology.orchestra.run.smithi089.stdout: "volumes" 2024-09-10T08:49:01.496 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-10T08:49:01.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.496 INFO:teuthology.orchestra.run.smithi089.stdout: "last_failure_osd_epoch": 3, 2024-09-10T08:49:01.496 INFO:teuthology.orchestra.run.smithi089.stdout: "active_clients": [ 2024-09-10T08:49:01.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.496 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "devicehealth", 2024-09-10T08:49:01.496 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-09-10T08:49:01.496 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.496 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-09-10T08:49:01.497 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:0", 2024-09-10T08:49:01.497 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 3689147449 2024-09-10T08:49:01.497 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.497 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-10T08:49:01.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.497 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "libcephsqlite", 2024-09-10T08:49:01.497 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-09-10T08:49:01.497 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.497 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-09-10T08:49:01.498 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:0", 2024-09-10T08:49:01.498 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 3600869128 2024-09-10T08:49:01.498 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.498 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-10T08:49:01.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.498 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "rbd_support", 2024-09-10T08:49:01.498 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-09-10T08:49:01.498 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.498 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-09-10T08:49:01.498 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:0", 2024-09-10T08:49:01.498 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 2807915475 2024-09-10T08:49:01.498 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.499 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-10T08:49:01.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:01.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.499 INFO:teuthology.orchestra.run.smithi089.stdout: "name": "volumes", 2024-09-10T08:49:01.499 INFO:teuthology.orchestra.run.smithi089.stdout: "addrvec": [ 2024-09-10T08:49:01.499 INFO:teuthology.orchestra.run.smithi089.stdout: { 2024-09-10T08:49:01.499 INFO:teuthology.orchestra.run.smithi089.stdout: "type": "v2", 2024-09-10T08:49:01.499 INFO:teuthology.orchestra.run.smithi089.stdout: "addr": "172.21.15.89:0", 2024-09-10T08:49:01.499 INFO:teuthology.orchestra.run.smithi089.stdout: "nonce": 3367934879 2024-09-10T08:49:01.499 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.499 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-10T08:49:01.499 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:01.499 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-10T08:49:01.499 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:49:01.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:00.084+0000 7f7857e00640 1 -- 172.21.15.89:0/4279437873 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f783c0a2c10 msgr2=0x7f783c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:01.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:00.084+0000 7f7857e00640 1 --2- 172.21.15.89:0/4279437873 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f783c0a2c10 0x7f783c0a50d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7858199310 tx=0x7f7840005f90 comp rx=0 tx=0).stop 2024-09-10T08:49:01.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:00.084+0000 7f7857e00640 1 -- 172.21.15.89:0/4279437873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7858101580 msgr2=0x7f7858197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:01.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:00.084+0000 7f7857e00640 1 --2- 172.21.15.89:0/4279437873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7858101580 0x7f7858197e00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7848031610 tx=0x7f7848004270 comp rx=0 tx=0).stop 2024-09-10T08:49:01.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:00.084+0000 7f7857e00640 1 -- 172.21.15.89:0/4279437873 shutdown_connections 2024-09-10T08:49:01.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:00.084+0000 7f7857e00640 1 --2- 172.21.15.89:0/4279437873 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f783c0a2c10 0x7f783c0a50d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:01.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:00.084+0000 7f7857e00640 1 --2- 172.21.15.89:0/4279437873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7858103050 0x7f785819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:01.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:00.084+0000 7f7857e00640 1 --2- 172.21.15.89:0/4279437873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7858101f40 0x7f7858198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:01.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:00.084+0000 7f7857e00640 1 --2- 172.21.15.89:0/4279437873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7858101580 0x7f7858197e00 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:01.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:00.084+0000 7f7857e00640 1 -- 172.21.15.89:0/4279437873 >> 172.21.15.89:0/4279437873 conn(0x7f78580fd110 msgr2=0x7f7858105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:01.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:00.084+0000 7f7857e00640 1 -- 172.21.15.89:0/4279437873 shutdown_connections 2024-09-10T08:49:01.502 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:00.084+0000 7f7857e00640 1 -- 172.21.15.89:0/4279437873 wait complete. 2024-09-10T08:49:01.508 DEBUG:tasks.check_counter:Getting stats from a 2024-09-10T08:49:01.509 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-10T08:49:01.510 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:00 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3832133869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T08:49:01.510 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:00 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/440687271' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T08:49:01.510 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:00 smithi089 ceph-mon[30757]: pgmap v665: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 583 KiB/s wr, 446 op/s 2024-09-10T08:49:01.510 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:01 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/4279437873' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T08:49:01.511 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:00 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3832133869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T08:49:01.511 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:00 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/440687271' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T08:49:01.511 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:00 smithi111 ceph-mon[35220]: pgmap v665: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 583 KiB/s wr, 446 op/s 2024-09-10T08:49:01.511 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:00 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3832133869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T08:49:01.511 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:00 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/440687271' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T08:49:01.511 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:00 smithi149 ceph-mon[36167]: pgmap v665: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 583 KiB/s wr, 446 op/s 2024-09-10T08:49:01.511 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:01 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/4279437873' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T08:49:01.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:01 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/4279437873' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T08:49:01.891 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:02.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:02 smithi089 ceph-mon[30757]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:49:02.304 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:02 smithi089 ceph-mon[30757]: pgmap v666: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 583 KiB/s wr, 417 op/s 2024-09-10T08:49:02.470 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:02 smithi149 ceph-mon[36167]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:49:02.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:02 smithi149 ceph-mon[36167]: pgmap v666: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 583 KiB/s wr, 417 op/s 2024-09-10T08:49:02.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:02 smithi111 ceph-mon[35220]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:49:02.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:02 smithi111 ceph-mon[35220]: pgmap v666: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 583 KiB/s wr, 417 op/s 2024-09-10T08:49:02.622 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:49:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:49:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 0, 2024-09-10T08:49:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 2, 2024-09-10T08:49:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 0, 2024-09-10T08:49:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 0, 2024-09-10T08:49:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2024-09-10T08:49:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 0, 2024-09-10T08:49:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.003413521, 2024-09-10T08:49:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T08:49:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.000473987, 2024-09-10T08:49:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:49:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-10T08:49:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-10T08:49:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-10T08:49:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-10T08:49:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-10T08:49:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-10T08:49:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T08:49:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T08:49:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:49:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 205, 2024-09-10T08:49:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 200, 2024-09-10T08:49:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 749840, 2024-09-10T08:49:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 122784, 2024-09-10T08:49:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 2, 2024-09-10T08:49:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 1, 2024-09-10T08:49:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.047264554, 2024-09-10T08:49:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.024702105, 2024-09-10T08:49:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.028748979, 2024-09-10T08:49:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:49:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 199, 2024-09-10T08:49:02.625 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.022939794, 2024-09-10T08:49:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000115275 2024-09-10T08:49:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-10T08:49:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-10T08:49:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-10T08:49:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 749840, 2024-09-10T08:49:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 122784 2024-09-10T08:49:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:49:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 5, 2024-09-10T08:49:02.626 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 153, 2024-09-10T08:49:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 3184, 2024-09-10T08:49:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 105568, 2024-09-10T08:49:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 3, 2024-09-10T08:49:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 2, 2024-09-10T08:49:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.021560012, 2024-09-10T08:49:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.015284312, 2024-09-10T08:49:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.004511814, 2024-09-10T08:49:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:49:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 152, 2024-09-10T08:49:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.025835606, 2024-09-10T08:49:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000169971 2024-09-10T08:49:02.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-10T08:49:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-10T08:49:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-10T08:49:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-10T08:49:02.628 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 105568 2024-09-10T08:49:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2024-09-10T08:49:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 0, 2024-09-10T08:49:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2024-09-10T08:49:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2024-09-10T08:49:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:49:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2024-09-10T08:49:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:49:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:49:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:49:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:49:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:49:02.629 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:49:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:49:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:49:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:49:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:49:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:49:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:49:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:49:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2024-09-10T08:49:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:49:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2024-09-10T08:49:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:49:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:49:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:49:02.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:49:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:49:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:49:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:49:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2024-09-10T08:49:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:49:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:49:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:49:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2024-09-10T08:49:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2024-09-10T08:49:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2024-09-10T08:49:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:49:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:49:02.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:49:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:49:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 149998, 2024-09-10T08:49:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 30, 2024-09-10T08:49:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:49:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2024-09-10T08:49:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2024-09-10T08:49:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2024-09-10T08:49:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:49:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2024-09-10T08:49:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:49:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2024-09-10T08:49:02.632 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 0, 2024-09-10T08:49:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 0, 2024-09-10T08:49:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:49:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:49:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2024-09-10T08:49:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2024-09-10T08:49:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 928, 2024-09-10T08:49:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 2, 2024-09-10T08:49:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2024-09-10T08:49:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2024-09-10T08:49:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2024-09-10T08:49:02.633 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2024-09-10T08:49:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:49:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-10T08:49:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-10T08:49:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-10T08:49:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 206, 2024-09-10T08:49:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 720908, 2024-09-10T08:49:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 206, 2024-09-10T08:49:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-10T08:49:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-10T08:49:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 206, 2024-09-10T08:49:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 720908, 2024-09-10T08:49:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-10T08:49:02.634 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-10T08:49:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-10T08:49:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-10T08:49:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:02.635 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:02.635 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:49:03.652 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-10T08:49:03.653 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-10T08:49:03.653 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-09-10T08:49:03.653 DEBUG:tasks.check_counter:Getting stats from d 2024-09-10T08:49:03.653 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-10T08:49:04.026 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:05.059 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:49:05.059 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:49:05.059 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 214, 2024-09-10T08:49:05.059 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 207, 2024-09-10T08:49:05.059 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 754224, 2024-09-10T08:49:05.060 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 126432, 2024-09-10T08:49:05.060 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2024-09-10T08:49:05.060 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 0, 2024-09-10T08:49:05.060 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.054548132, 2024-09-10T08:49:05.060 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.024267977, 2024-09-10T08:49:05.060 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.033092352, 2024-09-10T08:49:05.060 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:49:05.060 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:05.060 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 207, 2024-09-10T08:49:05.060 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.029491416, 2024-09-10T08:49:05.060 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000142470 2024-09-10T08:49:05.060 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:05.060 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:05.061 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-10T08:49:05.061 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-10T08:49:05.061 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-10T08:49:05.061 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:05.061 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 754224, 2024-09-10T08:49:05.061 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 126432 2024-09-10T08:49:05.061 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:05.061 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:49:05.061 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 1, 2024-09-10T08:49:05.061 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 156, 2024-09-10T08:49:05.061 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 160, 2024-09-10T08:49:05.061 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 106000, 2024-09-10T08:49:05.061 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 3, 2024-09-10T08:49:05.061 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 2, 2024-09-10T08:49:05.062 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.017049435, 2024-09-10T08:49:05.062 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.013356411, 2024-09-10T08:49:05.062 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.001783695, 2024-09-10T08:49:05.062 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:49:05.062 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:05.062 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 154, 2024-09-10T08:49:05.062 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.018841921, 2024-09-10T08:49:05.062 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000122350 2024-09-10T08:49:05.062 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:05.062 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:05.062 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-10T08:49:05.062 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-10T08:49:05.062 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-10T08:49:05.062 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:05.062 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T08:49:05.063 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 106000 2024-09-10T08:49:05.063 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:05.063 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:49:05.063 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 0, 2024-09-10T08:49:05.063 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 2, 2024-09-10T08:49:05.063 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 0, 2024-09-10T08:49:05.063 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 0, 2024-09-10T08:49:05.063 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 2, 2024-09-10T08:49:05.063 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 1, 2024-09-10T08:49:05.063 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.000727651, 2024-09-10T08:49:05.063 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T08:49:05.063 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.000578321, 2024-09-10T08:49:05.063 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:49:05.063 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:05.064 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-10T08:49:05.064 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-10T08:49:05.064 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-10T08:49:05.064 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:05.064 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:05.064 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-10T08:49:05.064 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-10T08:49:05.064 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-10T08:49:05.064 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:05.064 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T08:49:05.064 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T08:49:05.064 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:05.064 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2024-09-10T08:49:05.065 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 0, 2024-09-10T08:49:05.065 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2024-09-10T08:49:05.065 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:05.065 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2024-09-10T08:49:05.065 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:49:05.065 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2024-09-10T08:49:05.065 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:49:05.065 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:49:05.065 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:49:05.065 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:49:05.065 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:49:05.065 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:49:05.065 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:49:05.065 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:49:05.066 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:49:05.066 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:49:05.066 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:49:05.066 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:49:05.066 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:49:05.066 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2024-09-10T08:49:05.066 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:49:05.066 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2024-09-10T08:49:05.066 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:49:05.066 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:49:05.066 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:49:05.066 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:49:05.066 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:49:05.066 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:49:05.067 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:49:05.067 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2024-09-10T08:49:05.067 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:49:05.067 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:49:05.067 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:49:05.067 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2024-09-10T08:49:05.067 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2024-09-10T08:49:05.067 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2024-09-10T08:49:05.067 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:49:05.067 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:49:05.067 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:49:05.067 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:49:05.067 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 148984, 2024-09-10T08:49:05.068 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 30, 2024-09-10T08:49:05.068 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:49:05.068 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2024-09-10T08:49:05.068 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2024-09-10T08:49:05.068 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2024-09-10T08:49:05.068 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:49:05.068 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2024-09-10T08:49:05.068 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:49:05.068 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2024-09-10T08:49:05.068 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 0, 2024-09-10T08:49:05.068 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 0, 2024-09-10T08:49:05.068 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:49:05.068 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:49:05.069 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2024-09-10T08:49:05.069 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2024-09-10T08:49:05.069 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 928, 2024-09-10T08:49:05.069 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 2, 2024-09-10T08:49:05.069 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2024-09-10T08:49:05.069 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2024-09-10T08:49:05.069 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2024-09-10T08:49:05.069 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2024-09-10T08:49:05.069 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:05.069 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:49:05.069 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-10T08:49:05.069 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-10T08:49:05.069 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-10T08:49:05.069 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 211, 2024-09-10T08:49:05.070 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 721558, 2024-09-10T08:49:05.070 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:05.070 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 211, 2024-09-10T08:49:05.070 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-10T08:49:05.070 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-10T08:49:05.070 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 211, 2024-09-10T08:49:05.070 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 721558, 2024-09-10T08:49:05.070 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-10T08:49:05.070 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-10T08:49:05.070 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-10T08:49:05.070 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-10T08:49:05.070 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:05.070 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:05.070 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:49:05.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:04 smithi149 ceph-mon[36167]: pgmap v667: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 207 KiB/s wr, 218 op/s 2024-09-10T08:49:05.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:04 smithi111 ceph-mon[35220]: pgmap v667: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 207 KiB/s wr, 218 op/s 2024-09-10T08:49:05.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:04 smithi089 ceph-mon[30757]: pgmap v667: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 207 KiB/s wr, 218 op/s 2024-09-10T08:49:05.858 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-10T08:49:05.858 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-10T08:49:05.858 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-10T08:49:05.858 DEBUG:tasks.check_counter:Getting stats from g 2024-09-10T08:49:05.858 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-10T08:49:06.225 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:06.840 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:49:06.841 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:49:06.841 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 0, 2024-09-10T08:49:06.841 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 2, 2024-09-10T08:49:06.841 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 0, 2024-09-10T08:49:06.841 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 0, 2024-09-10T08:49:06.841 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2024-09-10T08:49:06.841 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 0, 2024-09-10T08:49:06.842 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.003379622, 2024-09-10T08:49:06.842 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T08:49:06.842 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.000611014, 2024-09-10T08:49:06.842 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:49:06.842 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:06.842 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-10T08:49:06.842 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-10T08:49:06.842 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-10T08:49:06.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:06.842 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:06.842 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-10T08:49:06.842 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-10T08:49:06.842 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-10T08:49:06.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:06.843 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T08:49:06.843 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T08:49:06.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:06.843 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:49:06.843 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 5, 2024-09-10T08:49:06.843 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 155, 2024-09-10T08:49:06.843 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 3184, 2024-09-10T08:49:06.843 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 105920, 2024-09-10T08:49:06.844 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 3, 2024-09-10T08:49:06.844 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 2, 2024-09-10T08:49:06.844 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.020080612, 2024-09-10T08:49:06.844 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.014034562, 2024-09-10T08:49:06.844 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.004049077, 2024-09-10T08:49:06.844 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:49:06.844 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:06.844 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 154, 2024-09-10T08:49:06.844 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.019437786, 2024-09-10T08:49:06.845 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000126219 2024-09-10T08:49:06.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:06.845 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:06.845 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-10T08:49:06.845 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-10T08:49:06.845 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-10T08:49:06.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:06.845 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-10T08:49:06.845 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 105920 2024-09-10T08:49:06.845 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:06.845 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:49:06.846 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 207, 2024-09-10T08:49:06.846 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 202, 2024-09-10T08:49:06.846 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 750384, 2024-09-10T08:49:06.846 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 124064, 2024-09-10T08:49:06.846 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 2, 2024-09-10T08:49:06.846 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 1, 2024-09-10T08:49:06.846 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.047966935, 2024-09-10T08:49:06.846 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.020744925, 2024-09-10T08:49:06.846 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.033746154, 2024-09-10T08:49:06.847 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:49:06.847 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:06.847 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 201, 2024-09-10T08:49:06.847 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.026630374, 2024-09-10T08:49:06.847 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000132489 2024-09-10T08:49:06.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:06.847 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:06.847 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-10T08:49:06.847 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-10T08:49:06.847 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-10T08:49:06.847 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:06.847 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 750384, 2024-09-10T08:49:06.847 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 124064 2024-09-10T08:49:06.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:06.848 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2024-09-10T08:49:06.848 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 0, 2024-09-10T08:49:06.848 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2024-09-10T08:49:06.848 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:06.848 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2024-09-10T08:49:06.848 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:49:06.848 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2024-09-10T08:49:06.848 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:49:06.848 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:49:06.848 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:49:06.848 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:49:06.848 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:49:06.849 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:49:06.849 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:49:06.849 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:49:06.849 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:49:06.849 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:49:06.849 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:49:06.849 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:49:06.849 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:49:06.849 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2024-09-10T08:49:06.849 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:49:06.849 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2024-09-10T08:49:06.849 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:49:06.849 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:49:06.850 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:49:06.850 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:49:06.850 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:49:06.850 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:49:06.850 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:49:06.850 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2024-09-10T08:49:06.850 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:49:06.850 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:49:06.850 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:49:06.850 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2024-09-10T08:49:06.850 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2024-09-10T08:49:06.850 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2024-09-10T08:49:06.850 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:49:06.850 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:49:06.851 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:49:06.851 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:49:06.851 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 148967, 2024-09-10T08:49:06.851 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 29, 2024-09-10T08:49:06.851 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:49:06.851 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2024-09-10T08:49:06.851 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2024-09-10T08:49:06.851 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2024-09-10T08:49:06.851 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:49:06.851 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2024-09-10T08:49:06.851 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:49:06.851 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2024-09-10T08:49:06.851 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 0, 2024-09-10T08:49:06.852 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 0, 2024-09-10T08:49:06.852 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:49:06.852 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:49:06.852 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2024-09-10T08:49:06.852 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2024-09-10T08:49:06.852 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 928, 2024-09-10T08:49:06.852 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 2, 2024-09-10T08:49:06.852 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2024-09-10T08:49:06.852 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2024-09-10T08:49:06.852 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2024-09-10T08:49:06.852 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2024-09-10T08:49:06.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:06.852 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:49:06.852 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-10T08:49:06.853 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-10T08:49:06.853 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-10T08:49:06.853 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 208, 2024-09-10T08:49:06.853 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 721168, 2024-09-10T08:49:06.853 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:06.853 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 208, 2024-09-10T08:49:06.853 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-10T08:49:06.853 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-10T08:49:06.853 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 208, 2024-09-10T08:49:06.853 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 721168, 2024-09-10T08:49:06.853 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-10T08:49:06.853 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-10T08:49:06.853 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-10T08:49:06.853 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-10T08:49:06.854 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:06.854 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:06.854 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:49:07.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:06 smithi089 ceph-mon[30757]: pgmap v668: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 131 KiB/s wr, 137 op/s 2024-09-10T08:49:07.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:06 smithi149 ceph-mon[36167]: pgmap v668: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 131 KiB/s wr, 137 op/s 2024-09-10T08:49:07.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:06 smithi111 ceph-mon[35220]: pgmap v668: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 131 KiB/s wr, 137 op/s 2024-09-10T08:49:07.766 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-10T08:49:07.766 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-10T08:49:07.766 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-09-10T08:49:07.766 DEBUG:tasks.check_counter:Getting stats from j 2024-09-10T08:49:07.766 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-10T08:49:08.139 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:08.732 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-10T08:49:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:49:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 4, 2024-09-10T08:49:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 4, 2024-09-10T08:49:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 3024, 2024-09-10T08:49:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 448, 2024-09-10T08:49:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2024-09-10T08:49:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 0, 2024-09-10T08:49:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.005900875, 2024-09-10T08:49:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.000079487, 2024-09-10T08:49:08.733 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.003483236, 2024-09-10T08:49:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:49:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 3, 2024-09-10T08:49:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.006129371, 2024-09-10T08:49:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.002043123 2024-09-10T08:49:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-10T08:49:08.734 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-10T08:49:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-10T08:49:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-10T08:49:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T08:49:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:49:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 212, 2024-09-10T08:49:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 207, 2024-09-10T08:49:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 751744, 2024-09-10T08:49:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 127264, 2024-09-10T08:49:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 2, 2024-09-10T08:49:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 1, 2024-09-10T08:49:08.735 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.053755770, 2024-09-10T08:49:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.025663396, 2024-09-10T08:49:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.035073345, 2024-09-10T08:49:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:49:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 206, 2024-09-10T08:49:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.024257355, 2024-09-10T08:49:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000117754 2024-09-10T08:49:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:08.736 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-10T08:49:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-10T08:49:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-10T08:49:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 751744, 2024-09-10T08:49:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 127264 2024-09-10T08:49:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:49:08.737 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 1, 2024-09-10T08:49:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 158, 2024-09-10T08:49:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 160, 2024-09-10T08:49:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 106352, 2024-09-10T08:49:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 3, 2024-09-10T08:49:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 2, 2024-09-10T08:49:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.018742232, 2024-09-10T08:49:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.015249252, 2024-09-10T08:49:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.001502208, 2024-09-10T08:49:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:49:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 156, 2024-09-10T08:49:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.020519460, 2024-09-10T08:49:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000131535 2024-09-10T08:49:08.738 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-10T08:49:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-10T08:49:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-10T08:49:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T08:49:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 106352 2024-09-10T08:49:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2024-09-10T08:49:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 0, 2024-09-10T08:49:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2024-09-10T08:49:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2024-09-10T08:49:08.739 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:49:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2024-09-10T08:49:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:49:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:49:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:49:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:49:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:49:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:49:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:49:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:49:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:49:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:49:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:49:08.740 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:49:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:49:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2024-09-10T08:49:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:49:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2024-09-10T08:49:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:49:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:49:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:49:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:49:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:49:08.741 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:49:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:49:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2024-09-10T08:49:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:49:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:49:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:49:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2024-09-10T08:49:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2024-09-10T08:49:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2024-09-10T08:49:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:49:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:49:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:49:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:49:08.742 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 148967, 2024-09-10T08:49:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 29, 2024-09-10T08:49:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:49:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2024-09-10T08:49:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2024-09-10T08:49:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2024-09-10T08:49:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:49:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2024-09-10T08:49:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:49:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2024-09-10T08:49:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 0, 2024-09-10T08:49:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 0, 2024-09-10T08:49:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:49:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:49:08.743 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2024-09-10T08:49:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2024-09-10T08:49:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 928, 2024-09-10T08:49:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 2, 2024-09-10T08:49:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2024-09-10T08:49:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2024-09-10T08:49:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2024-09-10T08:49:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2024-09-10T08:49:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-10T08:49:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:49:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-10T08:49:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-10T08:49:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-10T08:49:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 213, 2024-09-10T08:49:08.744 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 721818, 2024-09-10T08:49:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 213, 2024-09-10T08:49:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-10T08:49:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-10T08:49:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 213, 2024-09-10T08:49:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 721818, 2024-09-10T08:49:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-10T08:49:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-10T08:49:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-10T08:49:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-10T08:49:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:08.745 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-10T08:49:08.745 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-10T08:49:09.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:08 smithi149 ceph-mon[36167]: pgmap v669: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 125 KiB/s wr, 89 op/s 2024-09-10T08:49:09.239 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:08 smithi089 ceph-mon[30757]: pgmap v669: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 125 KiB/s wr, 89 op/s 2024-09-10T08:49:09.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:08 smithi111 ceph-mon[35220]: pgmap v669: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 125 KiB/s wr, 89 op/s 2024-09-10T08:49:09.643 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-10T08:49:09.644 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-10T08:49:09.644 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-10T08:49:09.644 DEBUG:tasks.check_counter:Getting stats from b 2024-09-10T08:49:09.644 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-10T08:49:09.948 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:49:10.643 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-10T08:49:10.644 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:49:10.644 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 87766, 2024-09-10T08:49:10.644 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 79165, 2024-09-10T08:49:10.644 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 32332965, 2024-09-10T08:49:10.644 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 204995411, 2024-09-10T08:49:10.644 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 129, 2024-09-10T08:49:10.644 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 5, 2024-09-10T08:49:10.644 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 6.028909679, 2024-09-10T08:49:10.644 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 2.375650366, 2024-09-10T08:49:10.644 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 1.915257265, 2024-09-10T08:49:10.645 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 1.710667212, 2024-09-10T08:49:10.645 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:10.645 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 79167, 2024-09-10T08:49:10.645 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 36.975296104, 2024-09-10T08:49:10.645 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000467054 2024-09-10T08:49:10.645 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.645 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:10.645 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 39643, 2024-09-10T08:49:10.645 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.006324064, 2024-09-10T08:49:10.645 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000159 2024-09-10T08:49:10.645 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.646 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 864016, 2024-09-10T08:49:10.646 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 145664 2024-09-10T08:49:10.646 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.646 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:49:10.646 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 101224, 2024-09-10T08:49:10.646 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 96771, 2024-09-10T08:49:10.646 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 38664885, 2024-09-10T08:49:10.646 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 213229680, 2024-09-10T08:49:10.646 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 50, 2024-09-10T08:49:10.646 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 5, 2024-09-10T08:49:10.646 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 6.040468768, 2024-09-10T08:49:10.646 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 1.948066497, 2024-09-10T08:49:10.646 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 1.915063088, 2024-09-10T08:49:10.647 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 2.001711376, 2024-09-10T08:49:10.647 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:10.647 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 96772, 2024-09-10T08:49:10.647 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 46.119947026, 2024-09-10T08:49:10.647 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000476583 2024-09-10T08:49:10.647 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.647 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:10.647 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 53222, 2024-09-10T08:49:10.647 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.007434967, 2024-09-10T08:49:10.647 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000139 2024-09-10T08:49:10.647 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.647 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T08:49:10.647 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 353584 2024-09-10T08:49:10.648 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.648 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:49:10.648 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 426900, 2024-09-10T08:49:10.648 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 598624, 2024-09-10T08:49:10.648 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 145793143, 2024-09-10T08:49:10.648 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 465842506, 2024-09-10T08:49:10.648 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 20, 2024-09-10T08:49:10.648 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 8, 2024-09-10T08:49:10.648 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 19.764971110, 2024-09-10T08:49:10.648 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 9.280950424, 2024-09-10T08:49:10.649 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 13.730873827, 2024-09-10T08:49:10.649 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 2.648292255, 2024-09-10T08:49:10.649 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:10.649 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 598617, 2024-09-10T08:49:10.649 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 21394.731769493, 2024-09-10T08:49:10.649 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.035740267 2024-09-10T08:49:10.649 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.649 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:10.649 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 370334, 2024-09-10T08:49:10.649 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.092576448, 2024-09-10T08:49:10.649 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000249 2024-09-10T08:49:10.649 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.650 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 55968, 2024-09-10T08:49:10.650 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 30736 2024-09-10T08:49:10.650 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.650 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-09-10T08:49:10.650 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2024-09-10T08:49:10.650 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-09-10T08:49:10.650 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.650 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2024-09-10T08:49:10.650 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-10T08:49:10.650 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-10T08:49:10.650 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 56106, 2024-09-10T08:49:10.650 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 185.392786338, 2024-09-10T08:49:10.650 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.003304330 2024-09-10T08:49:10.651 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:10.651 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.651 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2024-09-10T08:49:10.651 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-10T08:49:10.651 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-10T08:49:10.651 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 59261, 2024-09-10T08:49:10.651 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 20.045666266, 2024-09-10T08:49:10.651 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000338260 2024-09-10T08:49:10.651 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:10.652 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.652 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2024-09-10T08:49:10.652 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 213470, 2024-09-10T08:49:10.652 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 209652, 2024-09-10T08:49:10.652 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2024-09-10T08:49:10.652 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 209652, 2024-09-10T08:49:10.652 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 6470.719169755, 2024-09-10T08:49:10.653 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.030864094 2024-09-10T08:49:10.653 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.653 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2024-09-10T08:49:10.653 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 3818, 2024-09-10T08:49:10.653 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 32, 2024-09-10T08:49:10.653 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 2, 2024-09-10T08:49:10.653 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 7435, 2024-09-10T08:49:10.654 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 115, 2024-09-10T08:49:10.654 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 11, 2024-09-10T08:49:10.654 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 608, 2024-09-10T08:49:10.654 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 184, 2024-09-10T08:49:10.654 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 410, 2024-09-10T08:49:10.654 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 14, 2024-09-10T08:49:10.655 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 345, 2024-09-10T08:49:10.655 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 5018, 2024-09-10T08:49:10.655 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 1, 2024-09-10T08:49:10.655 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 1, 2024-09-10T08:49:10.655 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 4, 2024-09-10T08:49:10.655 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 349101, 2024-09-10T08:49:10.655 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 296857, 2024-09-10T08:49:10.655 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 3799, 2024-09-10T08:49:10.656 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 403, 2024-09-10T08:49:10.656 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2024-09-10T08:49:10.656 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2024-09-10T08:49:10.656 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 582, 2024-09-10T08:49:10.656 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2024-09-10T08:49:10.656 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2024-09-10T08:49:10.656 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 22, 2024-09-10T08:49:10.657 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 2594, 2024-09-10T08:49:10.657 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 106, 2024-09-10T08:49:10.657 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 518, 2024-09-10T08:49:10.657 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2024-09-10T08:49:10.657 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 64106, 2024-09-10T08:49:10.657 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2024-09-10T08:49:10.657 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 4, 2024-09-10T08:49:10.657 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 234, 2024-09-10T08:49:10.658 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 0, 2024-09-10T08:49:10.658 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 64106, 2024-09-10T08:49:10.658 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2024-09-10T08:49:10.658 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T08:49:10.658 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2024-09-10T08:49:10.658 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 548, 2024-09-10T08:49:10.658 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 8, 2024-09-10T08:49:10.658 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 540, 2024-09-10T08:49:10.659 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 63558, 2024-09-10T08:49:10.659 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 42, 2024-09-10T08:49:10.659 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 74738, 2024-09-10T08:49:10.659 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 87773, 2024-09-10T08:49:10.659 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 7, 2024-09-10T08:49:10.659 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T08:49:10.659 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T08:49:10.659 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 108433, 2024-09-10T08:49:10.660 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 33694, 2024-09-10T08:49:10.660 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 6854, 2024-09-10T08:49:10.660 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 70759 2024-09-10T08:49:10.660 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.660 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2024-09-10T08:49:10.660 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 4, 2024-09-10T08:49:10.660 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 0, 2024-09-10T08:49:10.660 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 0, 2024-09-10T08:49:10.661 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 403, 2024-09-10T08:49:10.661 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 33, 2024-09-10T08:49:10.661 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 3, 2024-09-10T08:49:10.661 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2024-09-10T08:49:10.661 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2024-09-10T08:49:10.661 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 33587, 2024-09-10T08:49:10.661 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 33584, 2024-09-10T08:49:10.661 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2024-09-10T08:49:10.662 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2024-09-10T08:49:10.662 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2024-09-10T08:49:10.662 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2024-09-10T08:49:10.662 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2024-09-10T08:49:10.662 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2024-09-10T08:49:10.662 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2024-09-10T08:49:10.662 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_path": 6, 2024-09-10T08:49:10.662 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_inode": 65009, 2024-09-10T08:49:10.662 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 8, 2024-09-10T08:49:10.662 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 22, 2024-09-10T08:49:10.662 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2024-09-10T08:49:10.663 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 128, 2024-09-10T08:49:10.663 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2024-09-10T08:49:10.663 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2024-09-10T08:49:10.663 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.663 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2024-09-10T08:49:10.663 INFO:teuthology.orchestra.run.smithi111.stdout: "evlrg": 0, 2024-09-10T08:49:10.663 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 195903, 2024-09-10T08:49:10.663 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 192855, 2024-09-10T08:49:10.663 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 192855, 2024-09-10T08:49:10.663 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 3048, 2024-09-10T08:49:10.663 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2024-09-10T08:49:10.664 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 7925, 2024-09-10T08:49:10.664 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 267, 2024-09-10T08:49:10.664 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 263, 2024-09-10T08:49:10.664 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 263, 2024-09-10T08:49:10.664 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 4, 2024-09-10T08:49:10.664 INFO:teuthology.orchestra.run.smithi111.stdout: "segmjr": 1, 2024-09-10T08:49:10.664 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2024-09-10T08:49:10.664 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 10, 2024-09-10T08:49:10.664 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 441338486, 2024-09-10T08:49:10.664 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 447044553, 2024-09-10T08:49:10.664 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 0, 2024-09-10T08:49:10.664 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2024-09-10T08:49:10.664 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 19569, 2024-09-10T08:49:10.664 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 2575.017514155, 2024-09-10T08:49:10.665 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.131586566 2024-09-10T08:49:10.665 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.665 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 0 2024-09-10T08:49:10.665 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.665 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2024-09-10T08:49:10.665 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 42, 2024-09-10T08:49:10.665 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 35482, 2024-09-10T08:49:10.665 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 35440, 2024-09-10T08:49:10.665 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 58, 2024-09-10T08:49:10.665 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 1215, 2024-09-10T08:49:10.665 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 1157, 2024-09-10T08:49:10.665 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 608, 2024-09-10T08:49:10.665 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 81702, 2024-09-10T08:49:10.666 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 81094, 2024-09-10T08:49:10.666 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 1, 2024-09-10T08:49:10.666 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 66221, 2024-09-10T08:49:10.666 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 66220, 2024-09-10T08:49:10.666 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 173344, 2024-09-10T08:49:10.666 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 231712 2024-09-10T08:49:10.666 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.666 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2024-09-10T08:49:10.666 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 229474, 2024-09-10T08:49:10.666 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 10965, 2024-09-10T08:49:10.666 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 213470, 2024-09-10T08:49:10.666 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 44, 2024-09-10T08:49:10.666 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 24750, 2024-09-10T08:49:10.666 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2024-09-10T08:49:10.666 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 34621, 2024-09-10T08:49:10.667 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 909.548389404, 2024-09-10T08:49:10.667 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.026271580 2024-09-10T08:49:10.667 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.667 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2024-09-10T08:49:10.667 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 8750, 2024-09-10T08:49:10.667 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 465.610497873, 2024-09-10T08:49:10.667 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.053212628 2024-09-10T08:49:10.667 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.667 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2024-09-10T08:49:10.667 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.667 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.667 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.667 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.667 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2024-09-10T08:49:10.668 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.668 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.668 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.668 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.668 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2024-09-10T08:49:10.668 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 92097, 2024-09-10T08:49:10.668 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 3782.330614293, 2024-09-10T08:49:10.668 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.041068988 2024-09-10T08:49:10.668 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.668 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2024-09-10T08:49:10.668 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.668 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.669 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.669 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.669 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2024-09-10T08:49:10.669 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.669 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.669 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.669 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.669 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2024-09-10T08:49:10.669 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.669 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.669 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.669 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.669 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2024-09-10T08:49:10.669 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.670 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.670 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.670 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.670 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2024-09-10T08:49:10.670 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.671 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.671 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.671 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.671 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2024-09-10T08:49:10.671 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.671 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.671 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.671 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.671 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2024-09-10T08:49:10.671 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 158, 2024-09-10T08:49:10.671 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 22.422460502, 2024-09-10T08:49:10.671 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.141914306 2024-09-10T08:49:10.671 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.672 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2024-09-10T08:49:10.672 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.672 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.672 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.672 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.672 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2024-09-10T08:49:10.672 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.672 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.672 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.672 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.673 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2024-09-10T08:49:10.673 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 17666, 2024-09-10T08:49:10.673 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 527.280465068, 2024-09-10T08:49:10.673 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.029847190 2024-09-10T08:49:10.673 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.673 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2024-09-10T08:49:10.673 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 962, 2024-09-10T08:49:10.673 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 1.457505782, 2024-09-10T08:49:10.673 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001515078 2024-09-10T08:49:10.673 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.673 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2024-09-10T08:49:10.673 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 34239, 2024-09-10T08:49:10.673 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 696.540325161, 2024-09-10T08:49:10.674 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.020343477 2024-09-10T08:49:10.674 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.674 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2024-09-10T08:49:10.674 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.674 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.674 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.674 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.674 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2024-09-10T08:49:10.674 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.674 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.674 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.674 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.674 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2024-09-10T08:49:10.674 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 152, 2024-09-10T08:49:10.675 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 34.978228987, 2024-09-10T08:49:10.675 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.230119927 2024-09-10T08:49:10.675 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.675 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2024-09-10T08:49:10.675 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.675 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.675 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.675 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.675 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2024-09-10T08:49:10.675 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.675 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.675 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.675 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.675 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2024-09-10T08:49:10.675 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 9, 2024-09-10T08:49:10.676 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.626120667, 2024-09-10T08:49:10.676 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.069568963 2024-09-10T08:49:10.676 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.676 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2024-09-10T08:49:10.676 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.676 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.676 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.676 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.676 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2024-09-10T08:49:10.676 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.676 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.677 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.677 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.677 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2024-09-10T08:49:10.677 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.677 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.677 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.677 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.677 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2024-09-10T08:49:10.677 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 9, 2024-09-10T08:49:10.677 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.023780931, 2024-09-10T08:49:10.677 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.002642325 2024-09-10T08:49:10.677 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.677 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2024-09-10T08:49:10.677 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 3, 2024-09-10T08:49:10.678 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.002101383, 2024-09-10T08:49:10.678 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000700461 2024-09-10T08:49:10.678 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.678 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2024-09-10T08:49:10.678 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 20986, 2024-09-10T08:49:10.678 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 29.898679704, 2024-09-10T08:49:10.678 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001424696 2024-09-10T08:49:10.678 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.678 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2024-09-10T08:49:10.678 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2024-09-10T08:49:10.678 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2024-09-10T08:49:10.678 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.678 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.678 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.679 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:10.679 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.679 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2024-09-10T08:49:10.679 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 1, 2024-09-10T08:49:10.679 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 4, 2024-09-10T08:49:10.679 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 3, 2024-09-10T08:49:10.679 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 1, 2024-09-10T08:49:10.679 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2024-09-10T08:49:10.679 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 35054, 2024-09-10T08:49:10.679 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 35054, 2024-09-10T08:49:10.679 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 572, 2024-09-10T08:49:10.679 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2024-09-10T08:49:10.679 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.679 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-09-10T08:49:10.680 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:49:10.680 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-09-10T08:49:10.680 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:49:10.680 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:49:10.680 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:49:10.680 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:49:10.680 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:49:10.680 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:49:10.680 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:49:10.680 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:49:10.680 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:49:10.680 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:49:10.680 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:49:10.680 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:49:10.681 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:49:10.681 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-09-10T08:49:10.681 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:49:10.681 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-09-10T08:49:10.681 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:49:10.681 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:49:10.681 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:49:10.681 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:49:10.682 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:49:10.682 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:49:10.682 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:49:10.682 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-09-10T08:49:10.682 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:49:10.682 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:49:10.682 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:49:10.683 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-09-10T08:49:10.683 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-09-10T08:49:10.683 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-09-10T08:49:10.684 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:49:10.684 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:49:10.684 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:49:10.684 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:49:10.684 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 295871, 2024-09-10T08:49:10.685 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 108, 2024-09-10T08:49:10.685 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 352, 2024-09-10T08:49:10.685 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 4, 2024-09-10T08:49:10.685 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-09-10T08:49:10.685 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-09-10T08:49:10.685 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:49:10.686 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-09-10T08:49:10.686 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:49:10.686 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-09-10T08:49:10.686 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 9048, 2024-09-10T08:49:10.686 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 170, 2024-09-10T08:49:10.686 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:49:10.686 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:49:10.686 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-09-10T08:49:10.686 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-09-10T08:49:10.687 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 627728, 2024-09-10T08:49:10.687 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 6593, 2024-09-10T08:49:10.687 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-09-10T08:49:10.687 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-09-10T08:49:10.687 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-09-10T08:49:10.687 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-09-10T08:49:10.688 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.688 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2024-09-10T08:49:10.688 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2024-09-10T08:49:10.688 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2024-09-10T08:49:10.688 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 229099, 2024-09-10T08:49:10.688 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 539887431, 2024-09-10T08:49:10.689 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 31, 2024-09-10T08:49:10.689 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 229099, 2024-09-10T08:49:10.689 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2024-09-10T08:49:10.689 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 229099, 2024-09-10T08:49:10.689 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 105930.399664881, 2024-09-10T08:49:10.689 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.462378271 2024-09-10T08:49:10.689 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.689 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2024-09-10T08:49:10.690 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2024-09-10T08:49:10.690 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 229099, 2024-09-10T08:49:10.690 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 348390 2024-09-10T08:49:10.690 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.690 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 229099, 2024-09-10T08:49:10.690 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 64666, 2024-09-10T08:49:10.690 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 164433, 2024-09-10T08:49:10.691 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2024-09-10T08:49:10.691 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2024-09-10T08:49:10.691 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 6348, 2024-09-10T08:49:10.691 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 67307, 2024-09-10T08:49:10.691 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 518, 2024-09-10T08:49:10.691 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 20083, 2024-09-10T08:49:10.691 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 354, 2024-09-10T08:49:10.692 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2024-09-10T08:49:10.692 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2024-09-10T08:49:10.692 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2024-09-10T08:49:10.692 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 115, 2024-09-10T08:49:10.692 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 68735, 2024-09-10T08:49:10.692 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2024-09-10T08:49:10.692 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2024-09-10T08:49:10.692 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2024-09-10T08:49:10.692 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 64125, 2024-09-10T08:49:10.692 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 102225, 2024-09-10T08:49:10.692 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2024-09-10T08:49:10.692 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2024-09-10T08:49:10.692 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2024-09-10T08:49:10.692 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2024-09-10T08:49:10.693 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2024-09-10T08:49:10.693 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2024-09-10T08:49:10.693 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T08:49:10.693 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2024-09-10T08:49:10.693 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2024-09-10T08:49:10.693 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 7, 2024-09-10T08:49:10.693 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2024-09-10T08:49:10.693 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2024-09-10T08:49:10.693 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2024-09-10T08:49:10.693 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2024-09-10T08:49:10.693 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2024-09-10T08:49:10.693 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2024-09-10T08:49:10.694 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2024-09-10T08:49:10.694 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2024-09-10T08:49:10.694 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2024-09-10T08:49:10.694 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2024-09-10T08:49:10.694 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2024-09-10T08:49:10.694 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2024-09-10T08:49:10.694 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2024-09-10T08:49:10.694 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2024-09-10T08:49:10.694 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2024-09-10T08:49:10.694 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2024-09-10T08:49:10.694 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 96, 2024-09-10T08:49:10.694 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2024-09-10T08:49:10.694 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 10, 2024-09-10T08:49:10.694 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 12, 2024-09-10T08:49:10.695 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 12, 2024-09-10T08:49:10.695 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2024-09-10T08:49:10.695 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2024-09-10T08:49:10.695 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 15113, 2024-09-10T08:49:10.695 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 68, 2024-09-10T08:49:10.695 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 3392 2024-09-10T08:49:10.695 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.695 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2024-09-10T08:49:10.695 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 1, 2024-09-10T08:49:10.695 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2024-09-10T08:49:10.695 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2024-09-10T08:49:10.695 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2024-09-10T08:49:10.695 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.695 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2024-09-10T08:49:10.695 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 65035 2024-09-10T08:49:10.695 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.696 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2024-09-10T08:49:10.696 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2024-09-10T08:49:10.696 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 23, 2024-09-10T08:49:10.696 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2024-09-10T08:49:10.696 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 9, 2024-09-10T08:49:10.696 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed_ops": 67964, 2024-09-10T08:49:10.696 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 33584, 2024-09-10T08:49:10.696 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2024-09-10T08:49:10.696 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.696 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:49:10.696 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-10T08:49:10.696 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-10T08:49:10.696 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-10T08:49:10.697 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 615886, 2024-09-10T08:49:10.697 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 166649770, 2024-09-10T08:49:10.697 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:10.697 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 615886, 2024-09-10T08:49:10.697 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-10T08:49:10.697 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-10T08:49:10.697 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 615886, 2024-09-10T08:49:10.697 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 166649770, 2024-09-10T08:49:10.697 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-10T08:49:10.697 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.697 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.697 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.697 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:10.697 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.698 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2024-09-10T08:49:10.698 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-10T08:49:10.698 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-10T08:49:10.698 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-10T08:49:10.698 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-10T08:49:10.698 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-10T08:49:10.698 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:10.698 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-10T08:49:10.698 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 229099, 2024-09-10T08:49:10.698 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 543547319, 2024-09-10T08:49:10.698 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 229099, 2024-09-10T08:49:10.698 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 543547319, 2024-09-10T08:49:10.698 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-10T08:49:10.698 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.699 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.699 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.699 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:10.699 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.699 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2024-09-10T08:49:10.699 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-10T08:49:10.699 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2024-09-10T08:49:10.699 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-10T08:49:10.699 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-10T08:49:10.699 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-10T08:49:10.699 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:10.699 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-10T08:49:10.699 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 229099, 2024-09-10T08:49:10.699 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 229099, 2024-09-10T08:49:10.700 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 229099, 2024-09-10T08:49:10.700 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 229099, 2024-09-10T08:49:10.700 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-10T08:49:10.700 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.700 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.700 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.700 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:10.700 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.700 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2024-09-10T08:49:10.700 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-10T08:49:10.700 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-10T08:49:10.700 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-10T08:49:10.700 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 33584, 2024-09-10T08:49:10.701 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 3660408, 2024-09-10T08:49:10.701 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:10.701 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 33584, 2024-09-10T08:49:10.701 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-10T08:49:10.701 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-10T08:49:10.701 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 514, 2024-09-10T08:49:10.701 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 3660408, 2024-09-10T08:49:10.701 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-10T08:49:10.701 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.701 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.701 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.701 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:10.701 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:10.701 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x557faa848960": { 2024-09-10T08:49:10.702 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-10T08:49:10.702 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-10T08:49:10.702 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-10T08:49:10.702 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 195903, 2024-09-10T08:49:10.702 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 442850249, 2024-09-10T08:49:10.702 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:10.702 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 195903, 2024-09-10T08:49:10.702 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-10T08:49:10.702 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-10T08:49:10.702 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 19569, 2024-09-10T08:49:10.702 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 442850249, 2024-09-10T08:49:10.702 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-10T08:49:10.702 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:10.703 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:10.703 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:10.703 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:10.703 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:10.703 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-10T08:49:11.049 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:10 smithi111 ceph-mon[35220]: pgmap v670: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-10T08:49:11.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:10 smithi149 ceph-mon[36167]: pgmap v670: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-10T08:49:11.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:10 smithi089 ceph-mon[30757]: pgmap v670: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-10T08:49:11.723 INFO:tasks.check_counter:Daemon mds.b mds.exported=22 2024-09-10T08:49:11.723 INFO:tasks.check_counter:Daemon mds.b mds.imported=106 2024-09-10T08:49:11.723 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=115 2024-09-10T08:49:11.723 DEBUG:tasks.check_counter:Getting stats from e 2024-09-10T08:49:11.723 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-10T08:49:11.977 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:49:12.579 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-10T08:49:12.579 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:49:12.579 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 1395, 2024-09-10T08:49:12.579 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 1397, 2024-09-10T08:49:12.579 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 151421988, 2024-09-10T08:49:12.580 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 443620, 2024-09-10T08:49:12.580 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 8, 2024-09-10T08:49:12.580 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 4, 2024-09-10T08:49:12.580 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.181009791, 2024-09-10T08:49:12.580 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.041441753, 2024-09-10T08:49:12.580 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.095501780, 2024-09-10T08:49:12.581 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.033736905, 2024-09-10T08:49:12.581 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:12.581 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1395, 2024-09-10T08:49:12.581 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.048890101, 2024-09-10T08:49:12.581 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000035046 2024-09-10T08:49:12.581 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.581 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:12.581 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.581 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.581 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.582 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.582 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T08:49:12.582 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T08:49:12.582 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.582 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:49:12.582 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 560, 2024-09-10T08:49:12.582 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 716, 2024-09-10T08:49:12.582 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 145215849, 2024-09-10T08:49:12.582 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 514676, 2024-09-10T08:49:12.582 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 7, 2024-09-10T08:49:12.582 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 6, 2024-09-10T08:49:12.582 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.125116109, 2024-09-10T08:49:12.582 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.016002064, 2024-09-10T08:49:12.582 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.091981913, 2024-09-10T08:49:12.583 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.012407087, 2024-09-10T08:49:12.583 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:12.583 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 714, 2024-09-10T08:49:12.583 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.032072537, 2024-09-10T08:49:12.583 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000044919 2024-09-10T08:49:12.583 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.583 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:12.584 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.584 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.584 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.584 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.584 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T08:49:12.584 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 336848 2024-09-10T08:49:12.584 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.584 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:49:12.584 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 1049, 2024-09-10T08:49:12.584 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 1033, 2024-09-10T08:49:12.584 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 147833520, 2024-09-10T08:49:12.585 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 388223, 2024-09-10T08:49:12.585 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 5, 2024-09-10T08:49:12.585 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 4, 2024-09-10T08:49:12.585 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.168863469, 2024-09-10T08:49:12.585 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.027577973, 2024-09-10T08:49:12.585 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.123614879, 2024-09-10T08:49:12.585 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.019940220, 2024-09-10T08:49:12.585 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:12.585 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1033, 2024-09-10T08:49:12.585 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.057377705, 2024-09-10T08:49:12.586 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000055544 2024-09-10T08:49:12.586 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.586 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:12.586 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.586 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.586 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.586 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.586 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 808944, 2024-09-10T08:49:12.586 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 128416 2024-09-10T08:49:12.587 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.587 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-09-10T08:49:12.587 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2024-09-10T08:49:12.587 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-09-10T08:49:12.587 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.587 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2024-09-10T08:49:12.587 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-10T08:49:12.587 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-10T08:49:12.587 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 4394, 2024-09-10T08:49:12.587 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.773685209, 2024-09-10T08:49:12.587 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000176077 2024-09-10T08:49:12.587 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:12.588 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.588 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2024-09-10T08:49:12.588 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-10T08:49:12.589 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-10T08:49:12.589 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.589 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.589 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.589 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:12.589 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.589 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2024-09-10T08:49:12.589 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 0, 2024-09-10T08:49:12.589 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 0, 2024-09-10T08:49:12.589 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2024-09-10T08:49:12.590 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.590 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.590 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.590 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.590 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2024-09-10T08:49:12.590 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 0, 2024-09-10T08:49:12.590 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 0, 2024-09-10T08:49:12.590 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2024-09-10T08:49:12.590 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 0, 2024-09-10T08:49:12.590 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 0, 2024-09-10T08:49:12.590 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2024-09-10T08:49:12.590 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 34023, 2024-09-10T08:49:12.590 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 22633, 2024-09-10T08:49:12.590 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 11390, 2024-09-10T08:49:12.591 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 0, 2024-09-10T08:49:12.591 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 1690, 2024-09-10T08:49:12.591 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 0, 2024-09-10T08:49:12.591 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 0, 2024-09-10T08:49:12.591 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 0, 2024-09-10T08:49:12.591 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 2, 2024-09-10T08:49:12.591 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 0, 2024-09-10T08:49:12.591 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 0, 2024-09-10T08:49:12.591 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 0, 2024-09-10T08:49:12.591 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 0, 2024-09-10T08:49:12.592 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2024-09-10T08:49:12.592 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2024-09-10T08:49:12.592 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2024-09-10T08:49:12.592 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2024-09-10T08:49:12.592 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2024-09-10T08:49:12.592 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 0, 2024-09-10T08:49:12.592 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 0, 2024-09-10T08:49:12.592 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 0, 2024-09-10T08:49:12.592 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 0, 2024-09-10T08:49:12.592 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2024-09-10T08:49:12.592 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2024-09-10T08:49:12.592 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2024-09-10T08:49:12.593 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 4, 2024-09-10T08:49:12.593 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 234, 2024-09-10T08:49:12.593 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 0, 2024-09-10T08:49:12.593 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T08:49:12.593 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2024-09-10T08:49:12.593 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T08:49:12.593 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2024-09-10T08:49:12.593 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2024-09-10T08:49:12.593 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T08:49:12.593 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T08:49:12.593 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2024-09-10T08:49:12.593 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 0, 2024-09-10T08:49:12.593 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T08:49:12.594 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 0, 2024-09-10T08:49:12.594 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T08:49:12.594 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T08:49:12.594 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T08:49:12.594 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 0, 2024-09-10T08:49:12.595 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 0, 2024-09-10T08:49:12.595 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 0, 2024-09-10T08:49:12.595 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 0 2024-09-10T08:49:12.595 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.595 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2024-09-10T08:49:12.595 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 0, 2024-09-10T08:49:12.595 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 0, 2024-09-10T08:49:12.595 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 0, 2024-09-10T08:49:12.595 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 0, 2024-09-10T08:49:12.595 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 0, 2024-09-10T08:49:12.595 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 0, 2024-09-10T08:49:12.595 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2024-09-10T08:49:12.596 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2024-09-10T08:49:12.596 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 0, 2024-09-10T08:49:12.596 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 0, 2024-09-10T08:49:12.596 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2024-09-10T08:49:12.596 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2024-09-10T08:49:12.596 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2024-09-10T08:49:12.596 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2024-09-10T08:49:12.596 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2024-09-10T08:49:12.596 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2024-09-10T08:49:12.596 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2024-09-10T08:49:12.596 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_path": 0, 2024-09-10T08:49:12.596 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_inode": 0, 2024-09-10T08:49:12.596 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T08:49:12.597 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 0, 2024-09-10T08:49:12.597 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2024-09-10T08:49:12.597 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 0, 2024-09-10T08:49:12.597 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2024-09-10T08:49:12.597 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2024-09-10T08:49:12.597 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.597 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2024-09-10T08:49:12.597 INFO:teuthology.orchestra.run.smithi111.stdout: "evlrg": 0, 2024-09-10T08:49:12.597 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 0, 2024-09-10T08:49:12.597 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 0, 2024-09-10T08:49:12.597 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 0, 2024-09-10T08:49:12.597 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 195903, 2024-09-10T08:49:12.597 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2024-09-10T08:49:12.597 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 0, 2024-09-10T08:49:12.597 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 0, 2024-09-10T08:49:12.598 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 0, 2024-09-10T08:49:12.598 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 0, 2024-09-10T08:49:12.598 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 4, 2024-09-10T08:49:12.598 INFO:teuthology.orchestra.run.smithi111.stdout: "segmjr": 40, 2024-09-10T08:49:12.598 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2024-09-10T08:49:12.598 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 0, 2024-09-10T08:49:12.598 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 441338486, 2024-09-10T08:49:12.598 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 447044553, 2024-09-10T08:49:12.598 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 447041687, 2024-09-10T08:49:12.598 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2024-09-10T08:49:12.598 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.598 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.598 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.598 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.599 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 195903 2024-09-10T08:49:12.599 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.599 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2024-09-10T08:49:12.599 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 26, 2024-09-10T08:49:12.599 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 34795, 2024-09-10T08:49:12.599 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 34769, 2024-09-10T08:49:12.599 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 42, 2024-09-10T08:49:12.599 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 1197, 2024-09-10T08:49:12.599 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 1155, 2024-09-10T08:49:12.599 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 34023, 2024-09-10T08:49:12.599 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 77962, 2024-09-10T08:49:12.599 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 43939, 2024-09-10T08:49:12.599 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 0, 2024-09-10T08:49:12.599 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 0, 2024-09-10T08:49:12.600 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 0, 2024-09-10T08:49:12.600 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 121760, 2024-09-10T08:49:12.600 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 215328 2024-09-10T08:49:12.600 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.600 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2024-09-10T08:49:12.600 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 0, 2024-09-10T08:49:12.600 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 0, 2024-09-10T08:49:12.600 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 0, 2024-09-10T08:49:12.600 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 0, 2024-09-10T08:49:12.600 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 0, 2024-09-10T08:49:12.601 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2024-09-10T08:49:12.601 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.601 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.601 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.601 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.601 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2024-09-10T08:49:12.601 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.601 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.601 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.601 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.601 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2024-09-10T08:49:12.601 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.601 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.601 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.602 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.602 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2024-09-10T08:49:12.602 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.602 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.602 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.602 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.602 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2024-09-10T08:49:12.602 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.602 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.602 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.602 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.602 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2024-09-10T08:49:12.602 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.602 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.602 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.603 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.603 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2024-09-10T08:49:12.603 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.603 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.603 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.603 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.603 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2024-09-10T08:49:12.603 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.603 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.603 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.603 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.603 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2024-09-10T08:49:12.603 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.604 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.604 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.604 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.604 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2024-09-10T08:49:12.604 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.604 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.604 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.604 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.604 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2024-09-10T08:49:12.604 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.604 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.604 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.604 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.605 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2024-09-10T08:49:12.605 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.605 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.605 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.605 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.605 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2024-09-10T08:49:12.606 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.606 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.606 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.606 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.606 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2024-09-10T08:49:12.606 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.606 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.606 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.606 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.606 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2024-09-10T08:49:12.606 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.606 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.606 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.606 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.607 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2024-09-10T08:49:12.607 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.607 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.607 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.607 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.607 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2024-09-10T08:49:12.607 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.607 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.607 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.607 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.607 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2024-09-10T08:49:12.607 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.607 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.608 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.608 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.608 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2024-09-10T08:49:12.608 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.608 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.608 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.608 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.608 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2024-09-10T08:49:12.608 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.608 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.608 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.609 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.609 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2024-09-10T08:49:12.609 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.609 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.609 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.609 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.609 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2024-09-10T08:49:12.609 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.609 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.609 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.609 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.609 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2024-09-10T08:49:12.609 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.609 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.610 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.610 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.610 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2024-09-10T08:49:12.610 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.610 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.610 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.610 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.610 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2024-09-10T08:49:12.610 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.610 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.610 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.610 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.610 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2024-09-10T08:49:12.610 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.611 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.611 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.611 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.611 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2024-09-10T08:49:12.611 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.611 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.611 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.611 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.611 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2024-09-10T08:49:12.611 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.611 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.611 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.611 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.611 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2024-09-10T08:49:12.612 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.612 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.612 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.612 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.612 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2024-09-10T08:49:12.612 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2024-09-10T08:49:12.612 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2024-09-10T08:49:12.612 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.612 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.612 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.612 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:12.612 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.612 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2024-09-10T08:49:12.612 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 1, 2024-09-10T08:49:12.613 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 4, 2024-09-10T08:49:12.613 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 3, 2024-09-10T08:49:12.613 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 2, 2024-09-10T08:49:12.613 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2024-09-10T08:49:12.613 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 0, 2024-09-10T08:49:12.613 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 0, 2024-09-10T08:49:12.613 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 3283, 2024-09-10T08:49:12.613 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2024-09-10T08:49:12.613 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.613 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-09-10T08:49:12.613 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:49:12.613 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-09-10T08:49:12.613 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:49:12.613 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:49:12.614 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:49:12.614 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:49:12.614 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:49:12.614 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:49:12.614 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:49:12.614 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:49:12.614 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:49:12.614 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:49:12.614 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:49:12.614 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:49:12.614 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:49:12.614 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-09-10T08:49:12.614 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:49:12.614 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-09-10T08:49:12.614 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:49:12.615 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:49:12.615 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:49:12.615 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:49:12.615 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:49:12.615 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:49:12.615 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:49:12.615 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-09-10T08:49:12.615 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:49:12.615 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:49:12.615 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:49:12.615 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-09-10T08:49:12.615 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-09-10T08:49:12.615 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-09-10T08:49:12.615 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:49:12.616 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:49:12.616 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:49:12.616 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:49:12.616 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 239559, 2024-09-10T08:49:12.616 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 82, 2024-09-10T08:49:12.616 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:49:12.616 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-09-10T08:49:12.616 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-09-10T08:49:12.616 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-09-10T08:49:12.616 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:49:12.616 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-09-10T08:49:12.616 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:49:12.616 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-09-10T08:49:12.616 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 9048, 2024-09-10T08:49:12.616 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 170, 2024-09-10T08:49:12.617 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:49:12.617 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:49:12.617 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-09-10T08:49:12.617 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-09-10T08:49:12.617 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 24067220, 2024-09-10T08:49:12.617 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 138986, 2024-09-10T08:49:12.617 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-09-10T08:49:12.617 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-09-10T08:49:12.617 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-09-10T08:49:12.617 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-09-10T08:49:12.617 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.617 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2024-09-10T08:49:12.617 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2024-09-10T08:49:12.617 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2024-09-10T08:49:12.618 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 2771, 2024-09-10T08:49:12.618 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 26, 2024-09-10T08:49:12.618 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 4, 2024-09-10T08:49:12.618 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 2771, 2024-09-10T08:49:12.618 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2024-09-10T08:49:12.618 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2771, 2024-09-10T08:49:12.618 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 15.837704755, 2024-09-10T08:49:12.618 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.005715519 2024-09-10T08:49:12.618 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.618 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2024-09-10T08:49:12.618 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2024-09-10T08:49:12.618 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2771, 2024-09-10T08:49:12.618 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 2772 2024-09-10T08:49:12.618 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.619 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 2771, 2024-09-10T08:49:12.619 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 2771, 2024-09-10T08:49:12.619 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 0, 2024-09-10T08:49:12.619 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2024-09-10T08:49:12.619 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2024-09-10T08:49:12.619 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 1666, 2024-09-10T08:49:12.619 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 0, 2024-09-10T08:49:12.620 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 1102, 2024-09-10T08:49:12.620 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 0, 2024-09-10T08:49:12.620 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 0, 2024-09-10T08:49:12.620 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2024-09-10T08:49:12.620 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2024-09-10T08:49:12.620 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2024-09-10T08:49:12.620 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 0, 2024-09-10T08:49:12.620 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 0, 2024-09-10T08:49:12.620 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2024-09-10T08:49:12.620 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2024-09-10T08:49:12.620 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2024-09-10T08:49:12.620 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 2, 2024-09-10T08:49:12.620 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 0, 2024-09-10T08:49:12.621 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2024-09-10T08:49:12.621 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2024-09-10T08:49:12.621 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2024-09-10T08:49:12.621 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2024-09-10T08:49:12.621 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2024-09-10T08:49:12.621 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2024-09-10T08:49:12.621 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T08:49:12.621 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2024-09-10T08:49:12.621 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2024-09-10T08:49:12.621 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2024-09-10T08:49:12.621 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2024-09-10T08:49:12.621 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2024-09-10T08:49:12.621 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2024-09-10T08:49:12.621 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2024-09-10T08:49:12.622 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2024-09-10T08:49:12.622 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2024-09-10T08:49:12.622 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2024-09-10T08:49:12.622 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2024-09-10T08:49:12.622 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2024-09-10T08:49:12.622 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2024-09-10T08:49:12.622 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2024-09-10T08:49:12.622 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2024-09-10T08:49:12.622 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2024-09-10T08:49:12.622 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2024-09-10T08:49:12.622 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2024-09-10T08:49:12.622 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2024-09-10T08:49:12.622 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 96, 2024-09-10T08:49:12.622 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2024-09-10T08:49:12.623 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 2, 2024-09-10T08:49:12.623 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 11, 2024-09-10T08:49:12.623 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 11, 2024-09-10T08:49:12.623 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2024-09-10T08:49:12.623 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2024-09-10T08:49:12.623 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 0, 2024-09-10T08:49:12.623 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 2, 2024-09-10T08:49:12.623 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 0 2024-09-10T08:49:12.623 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.623 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2024-09-10T08:49:12.623 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 0, 2024-09-10T08:49:12.623 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2024-09-10T08:49:12.623 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2024-09-10T08:49:12.623 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2024-09-10T08:49:12.624 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.624 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2024-09-10T08:49:12.624 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 0 2024-09-10T08:49:12.624 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.624 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2024-09-10T08:49:12.624 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2024-09-10T08:49:12.624 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T08:49:12.624 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2024-09-10T08:49:12.624 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 0, 2024-09-10T08:49:12.624 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed_ops": 0, 2024-09-10T08:49:12.624 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 0, 2024-09-10T08:49:12.624 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2024-09-10T08:49:12.624 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.625 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:49:12.625 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-10T08:49:12.625 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-10T08:49:12.625 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-10T08:49:12.625 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 3000, 2024-09-10T08:49:12.625 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 444186673, 2024-09-10T08:49:12.625 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:12.625 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 3000, 2024-09-10T08:49:12.625 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-10T08:49:12.625 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-10T08:49:12.625 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 3000, 2024-09-10T08:49:12.625 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 444186673, 2024-09-10T08:49:12.625 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-10T08:49:12.626 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.626 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.626 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.626 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:12.626 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.626 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2024-09-10T08:49:12.626 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-10T08:49:12.626 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-10T08:49:12.626 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-10T08:49:12.626 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-10T08:49:12.626 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-10T08:49:12.626 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:12.626 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-10T08:49:12.626 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 2771, 2024-09-10T08:49:12.627 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 442900641, 2024-09-10T08:49:12.627 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 2771, 2024-09-10T08:49:12.627 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 442900641, 2024-09-10T08:49:12.627 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-10T08:49:12.627 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.627 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.627 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.627 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:12.628 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.628 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2024-09-10T08:49:12.628 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-10T08:49:12.628 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2024-09-10T08:49:12.628 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-10T08:49:12.628 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-10T08:49:12.628 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-10T08:49:12.628 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:12.628 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-10T08:49:12.628 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 2771, 2024-09-10T08:49:12.628 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 2771, 2024-09-10T08:49:12.628 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 2771, 2024-09-10T08:49:12.628 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 2771, 2024-09-10T08:49:12.629 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-10T08:49:12.629 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.629 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.629 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.629 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:12.629 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.629 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2024-09-10T08:49:12.629 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-10T08:49:12.629 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-10T08:49:12.629 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-10T08:49:12.629 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-10T08:49:12.629 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-10T08:49:12.629 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:12.630 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-10T08:49:12.630 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-10T08:49:12.630 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-10T08:49:12.630 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-09-10T08:49:12.630 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-09-10T08:49:12.630 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-10T08:49:12.630 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.630 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.630 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.630 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:12.630 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:12.630 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x55c2d4272b40": { 2024-09-10T08:49:12.630 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-10T08:49:12.630 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-10T08:49:12.630 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-10T08:49:12.631 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-10T08:49:12.631 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-10T08:49:12.631 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:12.631 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-10T08:49:12.631 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-10T08:49:12.631 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-10T08:49:12.631 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-09-10T08:49:12.631 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-09-10T08:49:12.631 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-10T08:49:12.631 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:12.631 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:12.631 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:12.631 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:12.631 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:12.632 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-10T08:49:13.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:12 smithi111 ceph-mon[35220]: pgmap v671: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-10T08:49:13.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:12 smithi149 ceph-mon[36167]: pgmap v671: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-10T08:49:13.288 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-10T08:49:13.288 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-10T08:49:13.288 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-09-10T08:49:13.288 DEBUG:tasks.check_counter:Getting stats from h 2024-09-10T08:49:13.288 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-10T08:49:13.304 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:12 smithi089 ceph-mon[30757]: pgmap v671: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-10T08:49:13.534 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:49:13.977 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-10T08:49:13.977 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:49:13.977 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 21494, 2024-09-10T08:49:13.977 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 30017, 2024-09-10T08:49:13.977 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 7085474, 2024-09-10T08:49:13.977 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 31503397, 2024-09-10T08:49:13.978 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 12, 2024-09-10T08:49:13.978 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 4, 2024-09-10T08:49:13.978 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 1.962316877, 2024-09-10T08:49:13.978 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.791274624, 2024-09-10T08:49:13.978 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.855829832, 2024-09-10T08:49:13.978 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.408123344, 2024-09-10T08:49:13.978 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:13.978 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 30015, 2024-09-10T08:49:13.978 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 1.373074500, 2024-09-10T08:49:13.978 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000045746 2024-09-10T08:49:13.978 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.978 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:13.979 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 39033, 2024-09-10T08:49:13.979 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.006991853, 2024-09-10T08:49:13.979 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000179 2024-09-10T08:49:13.979 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.979 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 3376, 2024-09-10T08:49:13.979 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 4528 2024-09-10T08:49:13.979 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.979 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:49:13.979 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 57659, 2024-09-10T08:49:13.979 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 77861, 2024-09-10T08:49:13.979 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 20204826, 2024-09-10T08:49:13.979 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 41750454, 2024-09-10T08:49:13.979 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 16, 2024-09-10T08:49:13.980 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 5, 2024-09-10T08:49:13.980 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 2.738596454, 2024-09-10T08:49:13.980 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 1.199261184, 2024-09-10T08:49:13.980 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 1.808197265, 2024-09-10T08:49:13.980 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.353794852, 2024-09-10T08:49:13.980 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:13.980 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 77860, 2024-09-10T08:49:13.980 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 4.872120710, 2024-09-10T08:49:13.980 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000062575 2024-09-10T08:49:13.980 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.980 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:13.980 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 51617, 2024-09-10T08:49:13.981 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.014753736, 2024-09-10T08:49:13.981 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000285 2024-09-10T08:49:13.981 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.981 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 820768, 2024-09-10T08:49:13.981 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 144960 2024-09-10T08:49:13.981 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.981 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:49:13.981 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 9427, 2024-09-10T08:49:13.981 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 9580, 2024-09-10T08:49:13.981 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 2948352, 2024-09-10T08:49:13.981 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 12758211, 2024-09-10T08:49:13.981 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 21, 2024-09-10T08:49:13.981 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 8, 2024-09-10T08:49:13.981 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.879288098, 2024-09-10T08:49:13.982 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.222171714, 2024-09-10T08:49:13.982 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.144237307, 2024-09-10T08:49:13.982 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.458204019, 2024-09-10T08:49:13.982 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:13.982 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 9579, 2024-09-10T08:49:13.982 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.734314049, 2024-09-10T08:49:13.982 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000076658 2024-09-10T08:49:13.982 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.982 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:13.982 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 14, 2024-09-10T08:49:13.982 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000001326, 2024-09-10T08:49:13.982 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000094 2024-09-10T08:49:13.983 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.983 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-10T08:49:13.983 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 339024 2024-09-10T08:49:13.983 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.983 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-09-10T08:49:13.983 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2024-09-10T08:49:13.983 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-09-10T08:49:13.983 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.983 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2024-09-10T08:49:13.983 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-10T08:49:13.983 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-10T08:49:13.983 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 7782, 2024-09-10T08:49:13.983 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 12.004526076, 2024-09-10T08:49:13.984 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001542601 2024-09-10T08:49:13.984 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:13.984 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.984 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2024-09-10T08:49:13.984 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-09-10T08:49:13.984 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-09-10T08:49:13.984 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 7663, 2024-09-10T08:49:13.984 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.730899338, 2024-09-10T08:49:13.984 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000095380 2024-09-10T08:49:13.984 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:13.984 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.984 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2024-09-10T08:49:13.985 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 25019, 2024-09-10T08:49:13.985 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 24985, 2024-09-10T08:49:13.985 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2024-09-10T08:49:13.985 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 24985, 2024-09-10T08:49:13.985 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 403.464788010, 2024-09-10T08:49:13.985 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.016148280 2024-09-10T08:49:13.985 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.985 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2024-09-10T08:49:13.985 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 34, 2024-09-10T08:49:13.985 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 7, 2024-09-10T08:49:13.985 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2024-09-10T08:49:13.985 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 99, 2024-09-10T08:49:13.985 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 10, 2024-09-10T08:49:13.986 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2024-09-10T08:49:13.986 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 20, 2024-09-10T08:49:13.986 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 3, 2024-09-10T08:49:13.986 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 3, 2024-09-10T08:49:13.986 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 14, 2024-09-10T08:49:13.986 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 14, 2024-09-10T08:49:13.986 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 948, 2024-09-10T08:49:13.986 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 3, 2024-09-10T08:49:13.986 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 3, 2024-09-10T08:49:13.986 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 3, 2024-09-10T08:49:13.986 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 53432, 2024-09-10T08:49:13.986 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 41647, 2024-09-10T08:49:13.986 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 32, 2024-09-10T08:49:13.987 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 23, 2024-09-10T08:49:13.987 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2024-09-10T08:49:13.987 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2024-09-10T08:49:13.987 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 592, 2024-09-10T08:49:13.987 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2024-09-10T08:49:13.987 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2024-09-10T08:49:13.987 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 90, 2024-09-10T08:49:13.987 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 460, 2024-09-10T08:49:13.987 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 20, 2024-09-10T08:49:13.987 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 2241, 2024-09-10T08:49:13.987 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2024-09-10T08:49:13.987 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 7644, 2024-09-10T08:49:13.987 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2024-09-10T08:49:13.987 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 2335, 2024-09-10T08:49:13.988 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 44884027, 2024-09-10T08:49:13.988 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 0, 2024-09-10T08:49:13.988 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 7644, 2024-09-10T08:49:13.988 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2024-09-10T08:49:13.988 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T08:49:13.988 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2024-09-10T08:49:13.988 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2024-09-10T08:49:13.988 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T08:49:13.988 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T08:49:13.989 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 7644, 2024-09-10T08:49:13.989 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 92, 2024-09-10T08:49:13.989 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 10052, 2024-09-10T08:49:13.989 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 12116, 2024-09-10T08:49:13.989 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T08:49:13.989 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T08:49:13.989 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T08:49:13.989 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 15031, 2024-09-10T08:49:13.989 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 4148, 2024-09-10T08:49:13.989 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 450, 2024-09-10T08:49:13.989 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 15750 2024-09-10T08:49:13.989 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.990 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2024-09-10T08:49:13.990 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 0, 2024-09-10T08:49:13.990 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 6, 2024-09-10T08:49:13.990 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 4, 2024-09-10T08:49:13.990 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 26, 2024-09-10T08:49:13.990 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 28, 2024-09-10T08:49:13.990 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 0, 2024-09-10T08:49:13.990 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2024-09-10T08:49:13.990 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2024-09-10T08:49:13.990 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 4153, 2024-09-10T08:49:13.990 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 4153, 2024-09-10T08:49:13.990 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2024-09-10T08:49:13.990 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2024-09-10T08:49:13.990 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2024-09-10T08:49:13.991 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2024-09-10T08:49:13.991 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2024-09-10T08:49:13.991 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2024-09-10T08:49:13.991 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2024-09-10T08:49:13.991 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_path": 6, 2024-09-10T08:49:13.991 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_inode": 6726, 2024-09-10T08:49:13.991 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T08:49:13.991 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 90, 2024-09-10T08:49:13.991 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2024-09-10T08:49:13.991 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 10, 2024-09-10T08:49:13.991 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2024-09-10T08:49:13.991 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2024-09-10T08:49:13.991 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.991 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2024-09-10T08:49:13.992 INFO:teuthology.orchestra.run.smithi111.stdout: "evlrg": 0, 2024-09-10T08:49:13.992 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 23562, 2024-09-10T08:49:13.992 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 0, 2024-09-10T08:49:13.992 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 0, 2024-09-10T08:49:13.992 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 23562, 2024-09-10T08:49:13.992 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2024-09-10T08:49:13.992 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 0, 2024-09-10T08:49:13.992 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 28, 2024-09-10T08:49:13.992 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 0, 2024-09-10T08:49:13.992 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 0, 2024-09-10T08:49:13.992 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 28, 2024-09-10T08:49:13.992 INFO:teuthology.orchestra.run.smithi111.stdout: "segmjr": 3, 2024-09-10T08:49:13.992 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2024-09-10T08:49:13.993 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 0, 2024-09-10T08:49:13.993 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 4194304, 2024-09-10T08:49:13.993 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 39174672, 2024-09-10T08:49:13.993 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 0, 2024-09-10T08:49:13.993 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2024-09-10T08:49:13.993 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2006, 2024-09-10T08:49:13.993 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 95.646771754, 2024-09-10T08:49:13.993 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.047680344 2024-09-10T08:49:13.993 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.993 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 0 2024-09-10T08:49:13.993 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.993 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2024-09-10T08:49:13.993 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 24, 2024-09-10T08:49:13.993 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 4212, 2024-09-10T08:49:13.994 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 4188, 2024-09-10T08:49:13.994 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 23, 2024-09-10T08:49:13.994 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 150, 2024-09-10T08:49:13.994 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 127, 2024-09-10T08:49:13.994 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 20, 2024-09-10T08:49:13.994 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 8594, 2024-09-10T08:49:13.994 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 8574, 2024-09-10T08:49:13.994 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 3, 2024-09-10T08:49:13.994 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 7219, 2024-09-10T08:49:13.994 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 7216, 2024-09-10T08:49:13.994 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 84136, 2024-09-10T08:49:13.994 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 223520 2024-09-10T08:49:13.994 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.995 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2024-09-10T08:49:13.995 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 35088, 2024-09-10T08:49:13.995 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 553, 2024-09-10T08:49:13.995 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 25019, 2024-09-10T08:49:13.995 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 12, 2024-09-10T08:49:13.995 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 9591, 2024-09-10T08:49:13.995 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2024-09-10T08:49:13.995 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 3125, 2024-09-10T08:49:13.995 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 119.033832111, 2024-09-10T08:49:13.995 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.038090826 2024-09-10T08:49:13.995 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.995 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2024-09-10T08:49:13.995 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-09-10T08:49:13.996 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.002258319, 2024-09-10T08:49:13.996 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001129159 2024-09-10T08:49:13.996 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.996 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2024-09-10T08:49:13.996 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:13.996 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:13.996 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:13.996 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.996 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2024-09-10T08:49:13.996 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:13.996 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:13.996 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:13.996 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.996 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2024-09-10T08:49:13.997 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 12667, 2024-09-10T08:49:13.997 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 146.470225742, 2024-09-10T08:49:13.997 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.011563134 2024-09-10T08:49:13.997 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.997 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2024-09-10T08:49:13.997 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:13.997 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:13.997 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:13.997 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.997 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2024-09-10T08:49:13.997 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-09-10T08:49:13.997 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000464974, 2024-09-10T08:49:13.997 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000232487 2024-09-10T08:49:13.997 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.998 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2024-09-10T08:49:13.998 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:13.998 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:13.998 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:13.998 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.998 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2024-09-10T08:49:13.998 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:13.998 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:13.998 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:13.998 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.999 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2024-09-10T08:49:13.999 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:13.999 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:13.999 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:13.999 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:13.999 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2024-09-10T08:49:14.000 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.000 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.000 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.000 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.000 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2024-09-10T08:49:14.000 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.000 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.000 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.000 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2024-09-10T08:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2024-09-10T08:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2024-09-10T08:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 3869, 2024-09-10T08:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 4.659691686, 2024-09-10T08:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001204365 2024-09-10T08:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2024-09-10T08:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 98, 2024-09-10T08:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 1.575121359, 2024-09-10T08:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.016072666 2024-09-10T08:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2024-09-10T08:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 3128, 2024-09-10T08:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 102.042475207, 2024-09-10T08:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.032622274 2024-09-10T08:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2024-09-10T08:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2024-09-10T08:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2024-09-10T08:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2024-09-10T08:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2024-09-10T08:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2024-09-10T08:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2024-09-10T08:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2024-09-10T08:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2024-09-10T08:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2024-09-10T08:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2024-09-10T08:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2024-09-10T08:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2094, 2024-09-10T08:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 29.680718612, 2024-09-10T08:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.014174173 2024-09-10T08:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2024-09-10T08:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2024-09-10T08:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2024-09-10T08:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2024-09-10T08:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 1, 2024-09-10T08:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 2, 2024-09-10T08:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 1, 2024-09-10T08:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 1, 2024-09-10T08:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2024-09-10T08:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 4270, 2024-09-10T08:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 4270, 2024-09-10T08:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 575, 2024-09-10T08:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2024-09-10T08:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-09-10T08:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-09-10T08:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-09-10T08:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-09-10T08:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-09-10T08:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-09-10T08:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-09-10T08:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-09-10T08:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 287953, 2024-09-10T08:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 101, 2024-09-10T08:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-09-10T08:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-09-10T08:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-09-10T08:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-09-10T08:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-09-10T08:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 9048, 2024-09-10T08:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 170, 2024-09-10T08:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-09-10T08:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-09-10T08:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 100805, 2024-09-10T08:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 515, 2024-09-10T08:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-09-10T08:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-09-10T08:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-09-10T08:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-09-10T08:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2024-09-10T08:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2024-09-10T08:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2024-09-10T08:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 18803, 2024-09-10T08:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 37670434, 2024-09-10T08:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 29, 2024-09-10T08:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 18803, 2024-09-10T08:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2024-09-10T08:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 18803, 2024-09-10T08:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 351.290477585, 2024-09-10T08:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.018682682 2024-09-10T08:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2024-09-10T08:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2024-09-10T08:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 18803, 2024-09-10T08:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 19244 2024-09-10T08:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 18803, 2024-09-10T08:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 7824, 2024-09-10T08:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 10979, 2024-09-10T08:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2024-09-10T08:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2024-09-10T08:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 0, 2024-09-10T08:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 10, 2024-09-10T08:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 173, 2024-09-10T08:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 2179, 2024-09-10T08:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 114, 2024-09-10T08:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2024-09-10T08:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2024-09-10T08:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2024-09-10T08:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 10, 2024-09-10T08:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 8326, 2024-09-10T08:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2024-09-10T08:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2024-09-10T08:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2024-09-10T08:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 7644, 2024-09-10T08:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 20, 2024-09-10T08:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2024-09-10T08:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2024-09-10T08:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2024-09-10T08:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2024-09-10T08:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2024-09-10T08:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2024-09-10T08:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T08:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2024-09-10T08:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2024-09-10T08:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2024-09-10T08:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2024-09-10T08:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2024-09-10T08:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2024-09-10T08:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2024-09-10T08:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2024-09-10T08:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2024-09-10T08:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2024-09-10T08:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2024-09-10T08:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2024-09-10T08:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2024-09-10T08:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2024-09-10T08:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2024-09-10T08:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2024-09-10T08:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2024-09-10T08:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2024-09-10T08:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2024-09-10T08:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 96, 2024-09-10T08:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2024-09-10T08:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 2, 2024-09-10T08:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 12, 2024-09-10T08:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 12, 2024-09-10T08:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2024-09-10T08:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2024-09-10T08:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 616, 2024-09-10T08:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 14, 2024-09-10T08:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 138 2024-09-10T08:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2024-09-10T08:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 1, 2024-09-10T08:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2024-09-10T08:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2024-09-10T08:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2024-09-10T08:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2024-09-10T08:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 6736 2024-09-10T08:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2024-09-10T08:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2024-09-10T08:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 16, 2024-09-10T08:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2024-09-10T08:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 8, 2024-09-10T08:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed_ops": 8306, 2024-09-10T08:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 4153, 2024-09-10T08:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2024-09-10T08:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-10T08:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-10T08:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-10T08:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 88576, 2024-09-10T08:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 23109211, 2024-09-10T08:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 88576, 2024-09-10T08:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-10T08:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-10T08:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 88576, 2024-09-10T08:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 23109211, 2024-09-10T08:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-10T08:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2024-09-10T08:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-10T08:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-10T08:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-10T08:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-10T08:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-10T08:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-10T08:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 18803, 2024-09-10T08:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 38122999, 2024-09-10T08:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 18803, 2024-09-10T08:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 38122999, 2024-09-10T08:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-10T08:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2024-09-10T08:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-10T08:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2024-09-10T08:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-10T08:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-09-10T08:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-09-10T08:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-09-10T08:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 18803, 2024-09-10T08:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 18803, 2024-09-10T08:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 18803, 2024-09-10T08:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 18803, 2024-09-10T08:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-10T08:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2024-09-10T08:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-10T08:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-10T08:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-10T08:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 4153, 2024-09-10T08:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 452677, 2024-09-10T08:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 4153, 2024-09-10T08:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-10T08:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-10T08:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 173, 2024-09-10T08:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 452677, 2024-09-10T08:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-10T08:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x55c6b0d4c960": { 2024-09-10T08:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-10T08:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-09-10T08:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-10T08:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 23562, 2024-09-10T08:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 34980368, 2024-09-10T08:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 23562, 2024-09-10T08:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-10T08:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-10T08:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 2006, 2024-09-10T08:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 34980368, 2024-09-10T08:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-10T08:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-10T08:49:14.861 INFO:tasks.check_counter:Daemon mds.h mds.exported=90 2024-09-10T08:49:14.861 INFO:tasks.check_counter:Daemon mds.h mds.imported=20 2024-09-10T08:49:14.861 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=10 2024-09-10T08:49:14.861 DEBUG:tasks.check_counter:Getting stats from k 2024-09-10T08:49:14.862 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-10T08:49:15.108 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:49:15.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:14 smithi149 ceph-mon[36167]: pgmap v672: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-10T08:49:15.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:14 smithi111 ceph-mon[35220]: pgmap v672: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-10T08:49:15.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:14 smithi089 ceph-mon[30757]: pgmap v672: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-10T08:49:15.635 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-10T08:49:15.635 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:49:15.635 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 0, 2024-09-10T08:49:15.635 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 2, 2024-09-10T08:49:15.635 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 0, 2024-09-10T08:49:15.635 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 0, 2024-09-10T08:49:15.635 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 4, 2024-09-10T08:49:15.636 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 0, 2024-09-10T08:49:15.636 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.000992441, 2024-09-10T08:49:15.636 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T08:49:15.636 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.000181184, 2024-09-10T08:49:15.636 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:49:15.636 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:15.636 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:15.636 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:15.636 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:15.636 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:15.636 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:15.636 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:15.636 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:15.637 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:15.637 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:15.637 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T08:49:15.637 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T08:49:15.637 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:15.637 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:49:15.637 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 219, 2024-09-10T08:49:15.637 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 212, 2024-09-10T08:49:15.637 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 755584, 2024-09-10T08:49:15.637 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 129632, 2024-09-10T08:49:15.638 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 2, 2024-09-10T08:49:15.638 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 1, 2024-09-10T08:49:15.638 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.031916617, 2024-09-10T08:49:15.638 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.012200713, 2024-09-10T08:49:15.638 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.026524956, 2024-09-10T08:49:15.638 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:49:15.638 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:15.638 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 212, 2024-09-10T08:49:15.638 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.014169836, 2024-09-10T08:49:15.638 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000066838 2024-09-10T08:49:15.638 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:15.638 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:15.638 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:15.639 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:15.639 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:15.639 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:15.639 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 755584, 2024-09-10T08:49:15.639 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 129632 2024-09-10T08:49:15.639 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:15.639 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:49:15.639 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 1, 2024-09-10T08:49:15.639 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 159, 2024-09-10T08:49:15.639 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 160, 2024-09-10T08:49:15.640 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 106528, 2024-09-10T08:49:15.640 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 3, 2024-09-10T08:49:15.640 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 2, 2024-09-10T08:49:15.640 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.009183118, 2024-09-10T08:49:15.640 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.006675042, 2024-09-10T08:49:15.640 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.001003610, 2024-09-10T08:49:15.640 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:49:15.640 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:15.640 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 157, 2024-09-10T08:49:15.640 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.006997792, 2024-09-10T08:49:15.640 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000044571 2024-09-10T08:49:15.640 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:15.640 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:15.640 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:15.641 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:15.641 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:15.641 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:15.641 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T08:49:15.641 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 106528 2024-09-10T08:49:15.641 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:15.641 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-09-10T08:49:15.641 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 0, 2024-09-10T08:49:15.641 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-09-10T08:49:15.641 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:15.641 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-09-10T08:49:15.641 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:49:15.641 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-09-10T08:49:15.642 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:49:15.642 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:49:15.642 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:49:15.642 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:49:15.642 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:49:15.642 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:49:15.642 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:49:15.642 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:49:15.642 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:49:15.642 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:49:15.642 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:49:15.642 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:49:15.642 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:49:15.642 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-09-10T08:49:15.643 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:49:15.643 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-09-10T08:49:15.643 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:49:15.643 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:49:15.643 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:49:15.643 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:49:15.643 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:49:15.643 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:49:15.643 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:49:15.643 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-09-10T08:49:15.643 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:49:15.644 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:49:15.644 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:49:15.644 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-09-10T08:49:15.644 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-09-10T08:49:15.644 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-09-10T08:49:15.644 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:49:15.644 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:49:15.644 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:49:15.644 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:49:15.644 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 148967, 2024-09-10T08:49:15.645 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 29, 2024-09-10T08:49:15.645 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:49:15.645 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-09-10T08:49:15.645 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-09-10T08:49:15.645 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-09-10T08:49:15.645 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:49:15.645 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-09-10T08:49:15.645 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:49:15.645 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-09-10T08:49:15.645 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 0, 2024-09-10T08:49:15.645 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 0, 2024-09-10T08:49:15.645 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:49:15.645 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:49:15.645 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-09-10T08:49:15.646 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-09-10T08:49:15.646 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 928, 2024-09-10T08:49:15.646 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 2, 2024-09-10T08:49:15.646 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-09-10T08:49:15.646 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-09-10T08:49:15.646 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-09-10T08:49:15.646 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-09-10T08:49:15.646 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-10T08:49:15.646 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:49:15.646 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-10T08:49:15.646 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-10T08:49:15.646 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-10T08:49:15.646 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 216, 2024-09-10T08:49:15.646 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 722208, 2024-09-10T08:49:15.647 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:15.647 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 216, 2024-09-10T08:49:15.647 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-10T08:49:15.647 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-10T08:49:15.647 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 216, 2024-09-10T08:49:15.647 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 722208, 2024-09-10T08:49:15.647 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-10T08:49:15.647 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-10T08:49:15.647 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-10T08:49:15.647 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-10T08:49:15.647 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:15.647 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-10T08:49:15.647 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-10T08:49:16.385 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-10T08:49:16.385 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-10T08:49:16.385 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-09-10T08:49:16.386 DEBUG:tasks.check_counter:Getting stats from c 2024-09-10T08:49:16.386 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-10T08:49:16.822 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:49:17.181 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:16 smithi149 ceph-mon[36167]: pgmap v673: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-10T08:49:17.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:16 smithi111 ceph-mon[35220]: pgmap v673: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-10T08:49:17.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:16 smithi089 ceph-mon[30757]: pgmap v673: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-10T08:49:17.493 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-09-10T08:49:17.493 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:49:17.493 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 1098, 2024-09-10T08:49:17.493 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 1084, 2024-09-10T08:49:17.493 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 6703180, 2024-09-10T08:49:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 404786, 2024-09-10T08:49:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 6, 2024-09-10T08:49:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 2, 2024-09-10T08:49:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.127755019, 2024-09-10T08:49:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.041226833, 2024-09-10T08:49:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.055095372, 2024-09-10T08:49:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.030678069, 2024-09-10T08:49:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1083, 2024-09-10T08:49:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.059309746, 2024-09-10T08:49:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000054764 2024-09-10T08:49:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 805760, 2024-09-10T08:49:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 127808 2024-09-10T08:49:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:49:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 282, 2024-09-10T08:49:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 438, 2024-09-10T08:49:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 8464584, 2024-09-10T08:49:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 427884, 2024-09-10T08:49:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 5, 2024-09-10T08:49:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 4, 2024-09-10T08:49:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.039829223, 2024-09-10T08:49:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.012828769, 2024-09-10T08:49:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.014397954, 2024-09-10T08:49:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.008816373, 2024-09-10T08:49:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 436, 2024-09-10T08:49:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.023048095, 2024-09-10T08:49:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000052862 2024-09-10T08:49:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:17.497 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.497 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.497 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.497 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.497 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T08:49:17.497 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 338464 2024-09-10T08:49:17.497 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.497 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:49:17.497 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 979, 2024-09-10T08:49:17.497 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 979, 2024-09-10T08:49:17.497 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 21223823, 2024-09-10T08:49:17.497 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 310505, 2024-09-10T08:49:17.497 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 5, 2024-09-10T08:49:17.498 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 4, 2024-09-10T08:49:17.498 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.082405270, 2024-09-10T08:49:17.498 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.023334824, 2024-09-10T08:49:17.498 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.031916822, 2024-09-10T08:49:17.498 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.021734764, 2024-09-10T08:49:17.498 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:17.498 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 978, 2024-09-10T08:49:17.498 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.040906839, 2024-09-10T08:49:17.498 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000041827 2024-09-10T08:49:17.498 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.498 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:17.498 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.499 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.499 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.499 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.499 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-10T08:49:17.499 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T08:49:17.499 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.499 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-09-10T08:49:17.499 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-09-10T08:49:17.499 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-09-10T08:49:17.499 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.499 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-09-10T08:49:17.499 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-09-10T08:49:17.499 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-09-10T08:49:17.499 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 3639, 2024-09-10T08:49:17.500 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.093801280, 2024-09-10T08:49:17.500 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000025776 2024-09-10T08:49:17.500 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:17.500 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.500 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-09-10T08:49:17.500 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-09-10T08:49:17.500 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-09-10T08:49:17.500 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.500 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.500 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.500 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:17.500 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.500 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-09-10T08:49:17.501 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 0, 2024-09-10T08:49:17.501 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 0, 2024-09-10T08:49:17.501 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-09-10T08:49:17.501 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.501 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.501 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.501 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.501 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-09-10T08:49:17.501 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2024-09-10T08:49:17.501 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-09-10T08:49:17.501 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-09-10T08:49:17.501 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 0, 2024-09-10T08:49:17.501 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-09-10T08:49:17.501 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-09-10T08:49:17.502 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 4163, 2024-09-10T08:49:17.502 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 0, 2024-09-10T08:49:17.502 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 4163, 2024-09-10T08:49:17.502 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-09-10T08:49:17.502 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 4163, 2024-09-10T08:49:17.502 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 0, 2024-09-10T08:49:17.502 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 0, 2024-09-10T08:49:17.502 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 0, 2024-09-10T08:49:17.502 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 1, 2024-09-10T08:49:17.502 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 0, 2024-09-10T08:49:17.502 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 0, 2024-09-10T08:49:17.502 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2024-09-10T08:49:17.502 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 0, 2024-09-10T08:49:17.503 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-09-10T08:49:17.503 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-09-10T08:49:17.503 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-09-10T08:49:17.503 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-09-10T08:49:17.503 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-09-10T08:49:17.503 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-09-10T08:49:17.503 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-09-10T08:49:17.503 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2024-09-10T08:49:17.503 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2024-09-10T08:49:17.503 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-09-10T08:49:17.503 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2024-09-10T08:49:17.503 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-09-10T08:49:17.503 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 0, 2024-09-10T08:49:17.503 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 0, 2024-09-10T08:49:17.504 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 0, 2024-09-10T08:49:17.504 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T08:49:17.504 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-09-10T08:49:17.504 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T08:49:17.504 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-09-10T08:49:17.504 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-09-10T08:49:17.504 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T08:49:17.504 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T08:49:17.504 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2024-09-10T08:49:17.504 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-09-10T08:49:17.504 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T08:49:17.504 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 0, 2024-09-10T08:49:17.504 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T08:49:17.505 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T08:49:17.505 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T08:49:17.505 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 0, 2024-09-10T08:49:17.505 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 0, 2024-09-10T08:49:17.505 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 0, 2024-09-10T08:49:17.505 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 0 2024-09-10T08:49:17.505 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.505 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-09-10T08:49:17.505 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-09-10T08:49:17.505 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2024-09-10T08:49:17.505 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2024-09-10T08:49:17.506 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 0, 2024-09-10T08:49:17.506 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2024-09-10T08:49:17.506 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-09-10T08:49:17.506 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-09-10T08:49:17.506 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-09-10T08:49:17.506 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 0, 2024-09-10T08:49:17.506 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 0, 2024-09-10T08:49:17.506 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-09-10T08:49:17.506 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-09-10T08:49:17.507 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-09-10T08:49:17.507 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-09-10T08:49:17.507 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-09-10T08:49:17.507 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-09-10T08:49:17.507 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-09-10T08:49:17.507 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_path": 0, 2024-09-10T08:49:17.507 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_inode": 0, 2024-09-10T08:49:17.507 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T08:49:17.507 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-09-10T08:49:17.507 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-09-10T08:49:17.507 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-09-10T08:49:17.508 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-09-10T08:49:17.508 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-09-10T08:49:17.508 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.508 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-09-10T08:49:17.508 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-09-10T08:49:17.508 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 0, 2024-09-10T08:49:17.508 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-09-10T08:49:17.508 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-09-10T08:49:17.508 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 23562, 2024-09-10T08:49:17.508 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-09-10T08:49:17.508 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-09-10T08:49:17.508 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 0, 2024-09-10T08:49:17.508 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-09-10T08:49:17.509 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-09-10T08:49:17.509 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 28, 2024-09-10T08:49:17.509 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 3, 2024-09-10T08:49:17.509 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-09-10T08:49:17.509 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-09-10T08:49:17.509 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194304, 2024-09-10T08:49:17.509 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 39174672, 2024-09-10T08:49:17.509 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 39173534, 2024-09-10T08:49:17.509 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-09-10T08:49:17.509 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.509 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.509 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.509 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.509 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 23562 2024-09-10T08:49:17.510 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.510 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-09-10T08:49:17.510 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 13, 2024-09-10T08:49:17.510 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 4211, 2024-09-10T08:49:17.510 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 4198, 2024-09-10T08:49:17.510 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 11, 2024-09-10T08:49:17.510 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 153, 2024-09-10T08:49:17.510 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 142, 2024-09-10T08:49:17.510 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 4163, 2024-09-10T08:49:17.510 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 8511, 2024-09-10T08:49:17.510 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 4348, 2024-09-10T08:49:17.510 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 0, 2024-09-10T08:49:17.510 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 0, 2024-09-10T08:49:17.510 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 0, 2024-09-10T08:49:17.511 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 56020, 2024-09-10T08:49:17.511 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 215328 2024-09-10T08:49:17.511 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.511 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-09-10T08:49:17.511 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 0, 2024-09-10T08:49:17.511 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2024-09-10T08:49:17.511 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 0, 2024-09-10T08:49:17.511 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 0, 2024-09-10T08:49:17.511 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 0, 2024-09-10T08:49:17.511 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-09-10T08:49:17.511 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.511 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.511 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.511 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.512 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-09-10T08:49:17.512 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.512 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.512 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.512 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.512 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-09-10T08:49:17.512 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.512 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.512 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.512 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.512 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-09-10T08:49:17.512 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.512 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.512 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.513 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.513 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-09-10T08:49:17.513 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.513 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.513 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.513 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.513 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-09-10T08:49:17.513 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.513 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.513 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.514 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.514 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-09-10T08:49:17.514 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.514 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.514 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.514 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.514 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-09-10T08:49:17.514 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.514 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.514 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.515 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.515 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-09-10T08:49:17.515 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.515 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.515 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.515 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.515 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-09-10T08:49:17.515 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.515 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.515 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.515 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.515 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-09-10T08:49:17.515 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.516 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.516 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.516 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.517 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-09-10T08:49:17.517 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.517 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.517 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.517 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.517 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-09-10T08:49:17.517 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.517 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.517 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.517 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.517 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-09-10T08:49:17.517 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.517 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.518 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.518 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.518 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-09-10T08:49:17.518 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.518 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.518 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.518 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.518 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-09-10T08:49:17.518 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.518 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.518 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.518 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.518 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-09-10T08:49:17.518 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.519 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.519 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.519 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-09-10T08:49:17.519 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.519 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.519 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.519 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-09-10T08:49:17.519 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.519 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.519 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.519 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-09-10T08:49:17.520 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.520 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.520 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.520 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.520 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-09-10T08:49:17.520 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.520 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.520 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.520 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.520 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-09-10T08:49:17.520 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.520 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.520 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.521 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-09-10T08:49:17.521 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.521 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.521 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.521 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-09-10T08:49:17.521 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.521 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.521 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.521 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-09-10T08:49:17.521 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.521 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.522 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.522 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-09-10T08:49:17.522 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.522 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.522 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.522 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-09-10T08:49:17.522 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.522 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.522 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.522 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-09-10T08:49:17.522 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.523 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.523 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.523 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-09-10T08:49:17.523 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.523 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.523 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.523 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-09-10T08:49:17.523 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-09-10T08:49:17.523 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-09-10T08:49:17.523 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.523 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.523 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.524 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:17.524 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.524 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-09-10T08:49:17.524 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2024-09-10T08:49:17.524 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 2, 2024-09-10T08:49:17.524 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 1, 2024-09-10T08:49:17.524 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 2, 2024-09-10T08:49:17.524 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-09-10T08:49:17.524 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 0, 2024-09-10T08:49:17.524 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 0, 2024-09-10T08:49:17.524 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 3325, 2024-09-10T08:49:17.524 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-09-10T08:49:17.524 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.524 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-09-10T08:49:17.524 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:49:17.525 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-09-10T08:49:17.525 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:49:17.525 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:49:17.525 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:49:17.525 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:49:17.525 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:49:17.525 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:49:17.525 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:49:17.525 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:49:17.525 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:49:17.525 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:49:17.525 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:49:17.525 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:49:17.525 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:49:17.526 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-09-10T08:49:17.526 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:49:17.526 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-09-10T08:49:17.526 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:49:17.526 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:49:17.526 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:49:17.526 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:49:17.526 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:49:17.526 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:49:17.526 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:49:17.526 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-09-10T08:49:17.527 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:49:17.527 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:49:17.527 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:49:17.527 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-09-10T08:49:17.527 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-09-10T08:49:17.527 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-09-10T08:49:17.527 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:49:17.527 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:49:17.527 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:49:17.527 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:49:17.527 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 207128, 2024-09-10T08:49:17.527 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 63, 2024-09-10T08:49:17.527 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:49:17.527 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-09-10T08:49:17.528 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-09-10T08:49:17.528 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-09-10T08:49:17.528 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:49:17.528 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-09-10T08:49:17.528 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:49:17.528 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-09-10T08:49:17.528 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 9048, 2024-09-10T08:49:17.528 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 170, 2024-09-10T08:49:17.528 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:49:17.528 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:49:17.528 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-09-10T08:49:17.528 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-09-10T08:49:17.528 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 2974344, 2024-09-10T08:49:17.528 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 16780, 2024-09-10T08:49:17.529 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-09-10T08:49:17.529 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-09-10T08:49:17.529 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-09-10T08:49:17.529 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-09-10T08:49:17.529 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.529 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-09-10T08:49:17.529 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-09-10T08:49:17.529 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-09-10T08:49:17.529 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 2127, 2024-09-10T08:49:17.529 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 21, 2024-09-10T08:49:17.529 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 3, 2024-09-10T08:49:17.529 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 2127, 2024-09-10T08:49:17.529 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-09-10T08:49:17.529 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2127, 2024-09-10T08:49:17.530 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 10.985760951, 2024-09-10T08:49:17.530 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.005164908 2024-09-10T08:49:17.530 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.530 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-09-10T08:49:17.530 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-09-10T08:49:17.530 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2127, 2024-09-10T08:49:17.530 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 2128 2024-09-10T08:49:17.530 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.530 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 2127, 2024-09-10T08:49:17.530 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 2127, 2024-09-10T08:49:17.530 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 0, 2024-09-10T08:49:17.530 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-09-10T08:49:17.530 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-09-10T08:49:17.530 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 1244, 2024-09-10T08:49:17.531 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 0, 2024-09-10T08:49:17.531 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 881, 2024-09-10T08:49:17.531 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 0, 2024-09-10T08:49:17.531 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 0, 2024-09-10T08:49:17.531 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-09-10T08:49:17.531 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-09-10T08:49:17.531 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-09-10T08:49:17.531 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-09-10T08:49:17.531 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 0, 2024-09-10T08:49:17.531 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-09-10T08:49:17.531 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-09-10T08:49:17.531 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-09-10T08:49:17.531 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 1, 2024-09-10T08:49:17.531 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 0, 2024-09-10T08:49:17.531 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-09-10T08:49:17.532 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-09-10T08:49:17.532 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-09-10T08:49:17.532 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-09-10T08:49:17.532 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-09-10T08:49:17.532 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-09-10T08:49:17.532 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T08:49:17.532 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-09-10T08:49:17.532 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-09-10T08:49:17.532 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-09-10T08:49:17.532 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-09-10T08:49:17.532 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-09-10T08:49:17.532 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-09-10T08:49:17.532 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-09-10T08:49:17.532 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-09-10T08:49:17.533 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-09-10T08:49:17.533 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-09-10T08:49:17.533 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-09-10T08:49:17.533 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-09-10T08:49:17.533 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-09-10T08:49:17.533 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-09-10T08:49:17.533 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-09-10T08:49:17.533 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-09-10T08:49:17.533 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-09-10T08:49:17.533 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-09-10T08:49:17.533 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-09-10T08:49:17.533 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 96, 2024-09-10T08:49:17.533 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-09-10T08:49:17.533 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 2, 2024-09-10T08:49:17.533 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 7, 2024-09-10T08:49:17.534 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 7, 2024-09-10T08:49:17.534 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-09-10T08:49:17.534 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-09-10T08:49:17.534 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 0, 2024-09-10T08:49:17.534 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 2, 2024-09-10T08:49:17.534 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 0 2024-09-10T08:49:17.534 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.534 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-09-10T08:49:17.534 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 0, 2024-09-10T08:49:17.534 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-09-10T08:49:17.534 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-09-10T08:49:17.534 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-09-10T08:49:17.534 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.534 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-09-10T08:49:17.535 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-09-10T08:49:17.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.535 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-09-10T08:49:17.535 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-09-10T08:49:17.535 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T08:49:17.535 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-09-10T08:49:17.535 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 0, 2024-09-10T08:49:17.535 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 0, 2024-09-10T08:49:17.535 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 0, 2024-09-10T08:49:17.535 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-09-10T08:49:17.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.535 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:49:17.535 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-09-10T08:49:17.535 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-09-10T08:49:17.535 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-09-10T08:49:17.536 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 2355, 2024-09-10T08:49:17.536 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 36164999, 2024-09-10T08:49:17.536 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:17.536 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 2355, 2024-09-10T08:49:17.536 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-09-10T08:49:17.536 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-09-10T08:49:17.536 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 2355, 2024-09-10T08:49:17.536 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 36164999, 2024-09-10T08:49:17.536 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-09-10T08:49:17.536 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.536 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.536 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.536 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:17.536 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.536 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-09-10T08:49:17.536 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-09-10T08:49:17.537 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-09-10T08:49:17.537 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-09-10T08:49:17.537 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-09-10T08:49:17.537 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-09-10T08:49:17.537 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:17.537 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-09-10T08:49:17.537 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 2127, 2024-09-10T08:49:17.537 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 34985047, 2024-09-10T08:49:17.537 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 2127, 2024-09-10T08:49:17.537 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 34985047, 2024-09-10T08:49:17.537 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-09-10T08:49:17.537 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.537 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.537 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.537 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:17.538 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.538 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-09-10T08:49:17.538 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-09-10T08:49:17.538 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-09-10T08:49:17.538 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-09-10T08:49:17.538 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-09-10T08:49:17.538 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-09-10T08:49:17.538 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:17.538 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-09-10T08:49:17.538 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 2127, 2024-09-10T08:49:17.538 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 2127, 2024-09-10T08:49:17.538 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 2127, 2024-09-10T08:49:17.538 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 2127, 2024-09-10T08:49:17.538 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-09-10T08:49:17.538 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.538 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.538 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.539 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:17.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.539 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-09-10T08:49:17.539 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-09-10T08:49:17.539 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-09-10T08:49:17.539 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-09-10T08:49:17.539 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-09-10T08:49:17.539 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-09-10T08:49:17.539 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:17.539 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-09-10T08:49:17.539 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-09-10T08:49:17.539 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-09-10T08:49:17.539 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-09-10T08:49:17.539 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-09-10T08:49:17.539 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-09-10T08:49:17.539 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.540 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.540 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.540 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:17.540 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:17.540 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x558dc86aab40": { 2024-09-10T08:49:17.540 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-09-10T08:49:17.540 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-09-10T08:49:17.540 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-09-10T08:49:17.540 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-09-10T08:49:17.540 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-09-10T08:49:17.540 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:17.540 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-09-10T08:49:17.540 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-09-10T08:49:17.540 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-09-10T08:49:17.540 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-09-10T08:49:17.541 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-09-10T08:49:17.541 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-09-10T08:49:17.541 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:17.541 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:17.541 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:17.541 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:17.541 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:17.541 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-09-10T08:49:18.525 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-10T08:49:18.525 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-10T08:49:18.525 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-09-10T08:49:18.525 DEBUG:tasks.check_counter:Getting stats from f 2024-09-10T08:49:18.526 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-10T08:49:18.788 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:49:19.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:18 smithi149 ceph-mon[36167]: pgmap v674: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-10T08:49:19.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:18 smithi111 ceph-mon[35220]: pgmap v674: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-10T08:49:19.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:18 smithi089 ceph-mon[30757]: pgmap v674: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-10T08:49:19.519 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-09-10T08:49:19.519 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:49:19.519 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 407, 2024-09-10T08:49:19.519 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 409, 2024-09-10T08:49:19.520 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 4300585, 2024-09-10T08:49:19.520 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 129490, 2024-09-10T08:49:19.520 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 6, 2024-09-10T08:49:19.520 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 2, 2024-09-10T08:49:19.520 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.040545761, 2024-09-10T08:49:19.520 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.010818982, 2024-09-10T08:49:19.521 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.014216024, 2024-09-10T08:49:19.521 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.010961524, 2024-09-10T08:49:19.521 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:19.521 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 407, 2024-09-10T08:49:19.521 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.015708505, 2024-09-10T08:49:19.521 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000038595 2024-09-10T08:49:19.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.522 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:19.522 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.522 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.522 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.523 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T08:49:19.523 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T08:49:19.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.523 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:49:19.523 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 891, 2024-09-10T08:49:19.523 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 875, 2024-09-10T08:49:19.524 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 3787695, 2024-09-10T08:49:19.524 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 338463, 2024-09-10T08:49:19.524 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 4, 2024-09-10T08:49:19.525 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 3, 2024-09-10T08:49:19.525 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.098306041, 2024-09-10T08:49:19.525 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.030087753, 2024-09-10T08:49:19.525 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.050509179, 2024-09-10T08:49:19.526 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.021473051, 2024-09-10T08:49:19.526 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:19.526 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 875, 2024-09-10T08:49:19.526 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.039807122, 2024-09-10T08:49:19.527 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000045493 2024-09-10T08:49:19.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.527 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:19.527 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.527 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.527 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.528 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 809056, 2024-09-10T08:49:19.528 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 128896 2024-09-10T08:49:19.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.528 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:49:19.529 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 1017, 2024-09-10T08:49:19.529 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 1174, 2024-09-10T08:49:19.529 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 311206, 2024-09-10T08:49:19.529 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 661728, 2024-09-10T08:49:19.529 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 5, 2024-09-10T08:49:19.530 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 4, 2024-09-10T08:49:19.530 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.110353440, 2024-09-10T08:49:19.530 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.035954979, 2024-09-10T08:49:19.530 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.027623757, 2024-09-10T08:49:19.530 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.036143089, 2024-09-10T08:49:19.530 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:19.531 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1172, 2024-09-10T08:49:19.531 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.052171097, 2024-09-10T08:49:19.531 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000044514 2024-09-10T08:49:19.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.531 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:19.531 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.531 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.532 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.532 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T08:49:19.532 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 338640 2024-09-10T08:49:19.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.532 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-09-10T08:49:19.533 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-09-10T08:49:19.533 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-09-10T08:49:19.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.533 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-09-10T08:49:19.533 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-09-10T08:49:19.533 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-09-10T08:49:19.533 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 3488, 2024-09-10T08:49:19.533 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.092365101, 2024-09-10T08:49:19.534 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000026480 2024-09-10T08:49:19.534 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:19.534 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.534 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-09-10T08:49:19.534 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-09-10T08:49:19.534 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-09-10T08:49:19.534 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.534 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.534 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.535 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:19.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.535 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-09-10T08:49:19.535 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 0, 2024-09-10T08:49:19.535 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 0, 2024-09-10T08:49:19.535 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-09-10T08:49:19.535 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.535 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.535 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.536 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-09-10T08:49:19.536 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2024-09-10T08:49:19.536 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-09-10T08:49:19.536 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-09-10T08:49:19.536 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 0, 2024-09-10T08:49:19.536 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-09-10T08:49:19.536 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-09-10T08:49:19.536 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 880, 2024-09-10T08:49:19.536 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 0, 2024-09-10T08:49:19.536 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 880, 2024-09-10T08:49:19.536 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-09-10T08:49:19.536 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 880, 2024-09-10T08:49:19.536 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 0, 2024-09-10T08:49:19.537 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 0, 2024-09-10T08:49:19.537 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 0, 2024-09-10T08:49:19.537 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 1, 2024-09-10T08:49:19.537 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 0, 2024-09-10T08:49:19.537 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 0, 2024-09-10T08:49:19.537 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2024-09-10T08:49:19.537 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 0, 2024-09-10T08:49:19.537 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-09-10T08:49:19.537 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-09-10T08:49:19.537 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-09-10T08:49:19.537 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-09-10T08:49:19.537 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-09-10T08:49:19.537 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-09-10T08:49:19.537 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-09-10T08:49:19.538 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2024-09-10T08:49:19.538 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2024-09-10T08:49:19.538 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-09-10T08:49:19.538 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2024-09-10T08:49:19.538 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-09-10T08:49:19.538 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 0, 2024-09-10T08:49:19.538 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 0, 2024-09-10T08:49:19.538 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 0, 2024-09-10T08:49:19.538 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T08:49:19.538 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-09-10T08:49:19.538 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T08:49:19.538 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-09-10T08:49:19.538 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-09-10T08:49:19.538 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T08:49:19.539 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T08:49:19.539 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2024-09-10T08:49:19.539 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-09-10T08:49:19.539 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T08:49:19.539 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 0, 2024-09-10T08:49:19.539 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T08:49:19.539 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T08:49:19.539 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T08:49:19.539 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 0, 2024-09-10T08:49:19.539 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 0, 2024-09-10T08:49:19.539 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 0, 2024-09-10T08:49:19.539 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 0 2024-09-10T08:49:19.540 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.540 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-09-10T08:49:19.540 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-09-10T08:49:19.540 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2024-09-10T08:49:19.540 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2024-09-10T08:49:19.540 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 0, 2024-09-10T08:49:19.540 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2024-09-10T08:49:19.540 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-09-10T08:49:19.540 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-09-10T08:49:19.540 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-09-10T08:49:19.540 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 0, 2024-09-10T08:49:19.540 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 0, 2024-09-10T08:49:19.541 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-09-10T08:49:19.541 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-09-10T08:49:19.541 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-09-10T08:49:19.541 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-09-10T08:49:19.541 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-09-10T08:49:19.541 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-09-10T08:49:19.541 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-09-10T08:49:19.541 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_path": 0, 2024-09-10T08:49:19.541 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_inode": 0, 2024-09-10T08:49:19.541 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T08:49:19.541 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-09-10T08:49:19.541 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-09-10T08:49:19.541 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-09-10T08:49:19.542 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-09-10T08:49:19.542 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-09-10T08:49:19.542 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.542 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-09-10T08:49:19.542 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-09-10T08:49:19.542 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 0, 2024-09-10T08:49:19.542 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-09-10T08:49:19.542 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-09-10T08:49:19.542 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 4930, 2024-09-10T08:49:19.542 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-09-10T08:49:19.542 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-09-10T08:49:19.542 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 0, 2024-09-10T08:49:19.542 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-09-10T08:49:19.543 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-09-10T08:49:19.543 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 7, 2024-09-10T08:49:19.543 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 2, 2024-09-10T08:49:19.543 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-09-10T08:49:19.543 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-09-10T08:49:19.543 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194304, 2024-09-10T08:49:19.543 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 11199463, 2024-09-10T08:49:19.543 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 11198325, 2024-09-10T08:49:19.543 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-09-10T08:49:19.543 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.543 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.543 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.543 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 4930 2024-09-10T08:49:19.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.544 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-09-10T08:49:19.544 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 8, 2024-09-10T08:49:19.544 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 898, 2024-09-10T08:49:19.544 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 890, 2024-09-10T08:49:19.544 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 6, 2024-09-10T08:49:19.544 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 38, 2024-09-10T08:49:19.544 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 32, 2024-09-10T08:49:19.544 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 880, 2024-09-10T08:49:19.544 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 1469, 2024-09-10T08:49:19.544 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 589, 2024-09-10T08:49:19.544 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 0, 2024-09-10T08:49:19.544 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 0, 2024-09-10T08:49:19.545 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 0, 2024-09-10T08:49:19.545 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 53032, 2024-09-10T08:49:19.545 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 215328 2024-09-10T08:49:19.545 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.545 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-09-10T08:49:19.545 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 0, 2024-09-10T08:49:19.545 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2024-09-10T08:49:19.545 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 0, 2024-09-10T08:49:19.545 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 0, 2024-09-10T08:49:19.545 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 0, 2024-09-10T08:49:19.545 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-09-10T08:49:19.545 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.545 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.546 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.546 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.546 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-09-10T08:49:19.546 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.546 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.546 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.546 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.546 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-09-10T08:49:19.546 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.546 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.546 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.546 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.546 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-09-10T08:49:19.546 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.547 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.547 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.547 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.547 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-09-10T08:49:19.547 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.547 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.547 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.547 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.547 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-09-10T08:49:19.547 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.547 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.547 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.547 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.547 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-09-10T08:49:19.547 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.548 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.548 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.548 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.548 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-09-10T08:49:19.548 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.548 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.548 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.548 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.548 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-09-10T08:49:19.548 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.548 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.548 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.548 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.549 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-09-10T08:49:19.549 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.549 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.549 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.549 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.549 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-09-10T08:49:19.549 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.549 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.550 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.550 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.550 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-09-10T08:49:19.550 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.550 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.550 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.550 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.550 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-09-10T08:49:19.550 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.551 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.551 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.551 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.551 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-09-10T08:49:19.551 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.551 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.551 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.551 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.551 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-09-10T08:49:19.551 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.551 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.551 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.552 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.552 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-09-10T08:49:19.552 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.552 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.552 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.552 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.552 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-09-10T08:49:19.552 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.552 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.552 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.552 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.552 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-09-10T08:49:19.553 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.553 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.553 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.553 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.553 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-09-10T08:49:19.553 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.553 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.553 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.553 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.553 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-09-10T08:49:19.553 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.554 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.554 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.554 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.554 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-09-10T08:49:19.554 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.554 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.554 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.554 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.554 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-09-10T08:49:19.554 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.554 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.554 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.555 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.555 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-09-10T08:49:19.555 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.555 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.555 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.555 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.555 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-09-10T08:49:19.555 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.555 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.555 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.555 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.555 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-09-10T08:49:19.555 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.555 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.556 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.556 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.556 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-09-10T08:49:19.556 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.556 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.556 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.556 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.556 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-09-10T08:49:19.556 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.556 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.556 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.557 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.557 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-09-10T08:49:19.557 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.557 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.557 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.557 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.557 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-09-10T08:49:19.557 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.557 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.557 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.558 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.558 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-09-10T08:49:19.558 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-09-10T08:49:19.558 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-09-10T08:49:19.558 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.558 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.558 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.558 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:19.558 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.558 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-09-10T08:49:19.558 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2024-09-10T08:49:19.558 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 3, 2024-09-10T08:49:19.558 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 2, 2024-09-10T08:49:19.559 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 1, 2024-09-10T08:49:19.559 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-09-10T08:49:19.559 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 0, 2024-09-10T08:49:19.559 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 0, 2024-09-10T08:49:19.559 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 1952, 2024-09-10T08:49:19.559 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-09-10T08:49:19.559 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.559 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-09-10T08:49:19.559 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:49:19.559 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-09-10T08:49:19.559 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:49:19.559 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:49:19.559 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:49:19.560 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:49:19.560 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:49:19.560 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:49:19.560 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:49:19.560 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:49:19.560 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:49:19.560 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:49:19.560 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:49:19.560 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:49:19.560 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:49:19.560 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-09-10T08:49:19.560 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:49:19.560 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-09-10T08:49:19.561 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:49:19.561 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:49:19.561 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:49:19.561 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:49:19.561 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:49:19.561 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:49:19.561 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:49:19.561 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-09-10T08:49:19.561 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:49:19.561 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:49:19.561 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:49:19.561 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-09-10T08:49:19.561 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-09-10T08:49:19.562 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-09-10T08:49:19.562 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:49:19.562 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:49:19.562 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:49:19.562 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:49:19.562 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 198264, 2024-09-10T08:49:19.563 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 57, 2024-09-10T08:49:19.563 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:49:19.563 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-09-10T08:49:19.563 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-09-10T08:49:19.563 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-09-10T08:49:19.563 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:49:19.563 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-09-10T08:49:19.564 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:49:19.564 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-09-10T08:49:19.564 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 9048, 2024-09-10T08:49:19.564 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 170, 2024-09-10T08:49:19.564 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:49:19.564 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:49:19.564 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-09-10T08:49:19.564 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-09-10T08:49:19.564 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 645592, 2024-09-10T08:49:19.564 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 3593, 2024-09-10T08:49:19.564 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-09-10T08:49:19.564 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-09-10T08:49:19.565 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-09-10T08:49:19.565 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-09-10T08:49:19.565 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.565 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-09-10T08:49:19.565 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-09-10T08:49:19.565 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-09-10T08:49:19.565 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 2082, 2024-09-10T08:49:19.566 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 21, 2024-09-10T08:49:19.566 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 3, 2024-09-10T08:49:19.566 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 2082, 2024-09-10T08:49:19.566 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-09-10T08:49:19.566 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2082, 2024-09-10T08:49:19.566 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 13.283711110, 2024-09-10T08:49:19.566 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.006380264 2024-09-10T08:49:19.566 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.566 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-09-10T08:49:19.567 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-09-10T08:49:19.567 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2082, 2024-09-10T08:49:19.567 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 2083 2024-09-10T08:49:19.567 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.567 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 2082, 2024-09-10T08:49:19.567 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 2082, 2024-09-10T08:49:19.567 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 0, 2024-09-10T08:49:19.567 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-09-10T08:49:19.567 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-09-10T08:49:19.567 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 1275, 2024-09-10T08:49:19.567 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 0, 2024-09-10T08:49:19.567 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 805, 2024-09-10T08:49:19.568 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 0, 2024-09-10T08:49:19.568 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 0, 2024-09-10T08:49:19.568 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-09-10T08:49:19.568 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-09-10T08:49:19.568 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-09-10T08:49:19.568 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-09-10T08:49:19.568 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 0, 2024-09-10T08:49:19.568 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-09-10T08:49:19.568 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-09-10T08:49:19.568 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-09-10T08:49:19.568 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 1, 2024-09-10T08:49:19.568 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 0, 2024-09-10T08:49:19.568 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-09-10T08:49:19.569 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-09-10T08:49:19.569 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-09-10T08:49:19.569 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-09-10T08:49:19.569 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-09-10T08:49:19.569 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-09-10T08:49:19.569 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T08:49:19.569 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-09-10T08:49:19.569 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-09-10T08:49:19.569 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-09-10T08:49:19.569 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-09-10T08:49:19.569 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-09-10T08:49:19.569 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-09-10T08:49:19.569 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-09-10T08:49:19.569 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-09-10T08:49:19.570 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-09-10T08:49:19.570 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-09-10T08:49:19.570 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-09-10T08:49:19.570 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-09-10T08:49:19.570 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-09-10T08:49:19.570 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-09-10T08:49:19.570 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-09-10T08:49:19.570 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-09-10T08:49:19.570 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-09-10T08:49:19.570 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-09-10T08:49:19.570 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-09-10T08:49:19.570 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 96, 2024-09-10T08:49:19.570 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-09-10T08:49:19.570 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 2, 2024-09-10T08:49:19.571 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 6, 2024-09-10T08:49:19.571 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 6, 2024-09-10T08:49:19.571 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-09-10T08:49:19.571 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-09-10T08:49:19.571 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 0, 2024-09-10T08:49:19.571 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 2, 2024-09-10T08:49:19.571 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 0 2024-09-10T08:49:19.571 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.571 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-09-10T08:49:19.571 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 0, 2024-09-10T08:49:19.571 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-09-10T08:49:19.571 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-09-10T08:49:19.572 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-09-10T08:49:19.572 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.572 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-09-10T08:49:19.572 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-09-10T08:49:19.572 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.572 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-09-10T08:49:19.572 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-09-10T08:49:19.572 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T08:49:19.572 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-09-10T08:49:19.572 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 0, 2024-09-10T08:49:19.572 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 0, 2024-09-10T08:49:19.572 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 0, 2024-09-10T08:49:19.572 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-09-10T08:49:19.572 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.573 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:49:19.573 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-09-10T08:49:19.573 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-09-10T08:49:19.573 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-09-10T08:49:19.573 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 2311, 2024-09-10T08:49:19.573 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 8176806, 2024-09-10T08:49:19.573 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:19.573 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 2311, 2024-09-10T08:49:19.573 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-09-10T08:49:19.573 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-09-10T08:49:19.573 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 2311, 2024-09-10T08:49:19.573 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 8176806, 2024-09-10T08:49:19.573 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-09-10T08:49:19.573 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.574 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.574 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.574 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:19.574 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.574 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-09-10T08:49:19.574 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-09-10T08:49:19.574 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-09-10T08:49:19.574 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-09-10T08:49:19.574 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-09-10T08:49:19.574 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-09-10T08:49:19.574 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:19.574 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-09-10T08:49:19.574 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 2082, 2024-09-10T08:49:19.574 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 7005164, 2024-09-10T08:49:19.575 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 2082, 2024-09-10T08:49:19.575 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 7005164, 2024-09-10T08:49:19.575 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-09-10T08:49:19.575 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.575 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.575 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.575 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:19.575 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.575 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-09-10T08:49:19.575 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-09-10T08:49:19.575 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-09-10T08:49:19.575 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-09-10T08:49:19.575 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-09-10T08:49:19.575 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-09-10T08:49:19.575 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:19.576 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-09-10T08:49:19.576 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 2082, 2024-09-10T08:49:19.576 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 2082, 2024-09-10T08:49:19.576 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 2082, 2024-09-10T08:49:19.576 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 2082, 2024-09-10T08:49:19.576 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-09-10T08:49:19.576 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.576 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.576 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.576 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:19.576 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.576 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-09-10T08:49:19.576 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-09-10T08:49:19.576 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-09-10T08:49:19.577 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-09-10T08:49:19.577 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-09-10T08:49:19.577 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-09-10T08:49:19.577 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:19.577 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-09-10T08:49:19.577 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-09-10T08:49:19.577 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-09-10T08:49:19.577 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-09-10T08:49:19.577 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-09-10T08:49:19.578 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-09-10T08:49:19.578 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.578 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.578 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.578 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:19.578 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:19.578 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x55ee23b58b40": { 2024-09-10T08:49:19.578 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-09-10T08:49:19.578 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-09-10T08:49:19.578 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-09-10T08:49:19.578 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-09-10T08:49:19.579 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-09-10T08:49:19.579 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:19.579 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-09-10T08:49:19.579 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-09-10T08:49:19.579 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-09-10T08:49:19.579 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-09-10T08:49:19.579 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-09-10T08:49:19.579 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-09-10T08:49:19.579 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:19.579 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:19.579 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:19.579 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:19.579 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:19.580 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-09-10T08:49:20.393 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-10T08:49:20.393 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-10T08:49:20.393 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-09-10T08:49:20.393 DEBUG:tasks.check_counter:Getting stats from i 2024-09-10T08:49:20.393 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-10T08:49:20.641 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:49:21.067 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:20 smithi149 ceph-mon[36167]: pgmap v675: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-10T08:49:21.176 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-09-10T08:49:21.176 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:49:21.176 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 18193, 2024-09-10T08:49:21.176 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 21965, 2024-09-10T08:49:21.176 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 6119265, 2024-09-10T08:49:21.176 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 13605940, 2024-09-10T08:49:21.176 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 16, 2024-09-10T08:49:21.176 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 3, 2024-09-10T08:49:21.176 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.837291124, 2024-09-10T08:49:21.177 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.388843374, 2024-09-10T08:49:21.177 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.531093377, 2024-09-10T08:49:21.177 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.050896920, 2024-09-10T08:49:21.177 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:21.177 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 21963, 2024-09-10T08:49:21.177 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.581139067, 2024-09-10T08:49:21.177 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000026459 2024-09-10T08:49:21.177 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.178 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:21.178 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 15884, 2024-09-10T08:49:21.178 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.004006869, 2024-09-10T08:49:21.178 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000252 2024-09-10T08:49:21.178 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.178 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1856, 2024-09-10T08:49:21.178 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 1472 2024-09-10T08:49:21.178 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.178 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:49:21.178 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 1748, 2024-09-10T08:49:21.179 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 1904, 2024-09-10T08:49:21.179 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 491372, 2024-09-10T08:49:21.179 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 3756065, 2024-09-10T08:49:21.179 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 14, 2024-09-10T08:49:21.179 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 6, 2024-09-10T08:49:21.179 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.135620868, 2024-09-10T08:49:21.179 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.050969856, 2024-09-10T08:49:21.179 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.032735774, 2024-09-10T08:49:21.179 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.041693783, 2024-09-10T08:49:21.179 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:21.179 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1902, 2024-09-10T08:49:21.180 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.156278965, 2024-09-10T08:49:21.180 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000082165 2024-09-10T08:49:21.180 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.180 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:21.180 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 6, 2024-09-10T08:49:21.180 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000503, 2024-09-10T08:49:21.180 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000083 2024-09-10T08:49:21.180 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.180 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T08:49:21.181 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 340432 2024-09-10T08:49:21.181 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.181 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:49:21.181 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 26065, 2024-09-10T08:49:21.181 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 30651, 2024-09-10T08:49:21.181 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 6713654, 2024-09-10T08:49:21.181 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 9424329, 2024-09-10T08:49:21.181 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 20, 2024-09-10T08:49:21.181 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 9, 2024-09-10T08:49:21.181 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 1.436189242, 2024-09-10T08:49:21.181 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.572461435, 2024-09-10T08:49:21.181 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 1.078289949, 2024-09-10T08:49:21.182 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.023552185, 2024-09-10T08:49:21.182 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:21.182 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 30651, 2024-09-10T08:49:21.182 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.652212127, 2024-09-10T08:49:21.182 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000021278 2024-09-10T08:49:21.182 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.183 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:21.183 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 52995, 2024-09-10T08:49:21.183 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.008525873, 2024-09-10T08:49:21.183 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000160 2024-09-10T08:49:21.183 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.183 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 820032, 2024-09-10T08:49:21.183 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 138992 2024-09-10T08:49:21.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.184 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-09-10T08:49:21.184 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-09-10T08:49:21.184 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-09-10T08:49:21.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.184 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-09-10T08:49:21.185 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-09-10T08:49:21.185 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-09-10T08:49:21.185 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2962, 2024-09-10T08:49:21.185 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 1.811972227, 2024-09-10T08:49:21.185 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000611739 2024-09-10T08:49:21.185 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:21.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.185 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-09-10T08:49:21.186 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-09-10T08:49:21.186 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-09-10T08:49:21.186 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1966, 2024-09-10T08:49:21.186 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.112361044, 2024-09-10T08:49:21.186 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000057152 2024-09-10T08:49:21.186 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:21.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.186 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-09-10T08:49:21.186 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 4509, 2024-09-10T08:49:21.186 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 4484, 2024-09-10T08:49:21.186 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-09-10T08:49:21.186 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4484, 2024-09-10T08:49:21.186 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 155.038800280, 2024-09-10T08:49:21.187 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.034576003 2024-09-10T08:49:21.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.187 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-09-10T08:49:21.187 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 25, 2024-09-10T08:49:21.187 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-09-10T08:49:21.187 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-09-10T08:49:21.187 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 28, 2024-09-10T08:49:21.187 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 2, 2024-09-10T08:49:21.187 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-09-10T08:49:21.187 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 18, 2024-09-10T08:49:21.187 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 3, 2024-09-10T08:49:21.188 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 2, 2024-09-10T08:49:21.188 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 13, 2024-09-10T08:49:21.188 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 13, 2024-09-10T08:49:21.188 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 126, 2024-09-10T08:49:21.188 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 0, 2024-09-10T08:49:21.188 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 0, 2024-09-10T08:49:21.188 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 3, 2024-09-10T08:49:21.188 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 10947, 2024-09-10T08:49:21.188 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 8945, 2024-09-10T08:49:21.188 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 25, 2024-09-10T08:49:21.188 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 4, 2024-09-10T08:49:21.188 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-09-10T08:49:21.188 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-09-10T08:49:21.189 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 216, 2024-09-10T08:49:21.189 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-09-10T08:49:21.189 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-09-10T08:49:21.189 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 16, 2024-09-10T08:49:21.189 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 58, 2024-09-10T08:49:21.189 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 2, 2024-09-10T08:49:21.189 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 353, 2024-09-10T08:49:21.189 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-09-10T08:49:21.189 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 1132, 2024-09-10T08:49:21.189 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-09-10T08:49:21.189 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 2335, 2024-09-10T08:49:21.190 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 44884027, 2024-09-10T08:49:21.190 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 0, 2024-09-10T08:49:21.191 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 1132, 2024-09-10T08:49:21.191 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-09-10T08:49:21.191 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T08:49:21.191 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-09-10T08:49:21.191 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-09-10T08:49:21.191 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T08:49:21.191 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T08:49:21.191 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 1132, 2024-09-10T08:49:21.191 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 747, 2024-09-10T08:49:21.191 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 5277, 2024-09-10T08:49:21.191 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 5676, 2024-09-10T08:49:21.191 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T08:49:21.191 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T08:49:21.192 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T08:49:21.192 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 9741, 2024-09-10T08:49:21.192 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 875, 2024-09-10T08:49:21.192 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 180, 2024-09-10T08:49:21.192 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 3141 2024-09-10T08:49:21.192 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.192 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-09-10T08:49:21.192 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-09-10T08:49:21.192 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 6, 2024-09-10T08:49:21.192 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 4, 2024-09-10T08:49:21.192 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 7, 2024-09-10T08:49:21.193 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 375, 2024-09-10T08:49:21.193 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-09-10T08:49:21.193 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-09-10T08:49:21.193 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-09-10T08:49:21.193 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 875, 2024-09-10T08:49:21.193 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 875, 2024-09-10T08:49:21.193 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-09-10T08:49:21.193 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-09-10T08:49:21.193 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-09-10T08:49:21.193 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-09-10T08:49:21.193 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-09-10T08:49:21.193 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-09-10T08:49:21.194 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-09-10T08:49:21.194 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_path": 6, 2024-09-10T08:49:21.194 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_inode": 1044, 2024-09-10T08:49:21.194 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T08:49:21.194 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 16, 2024-09-10T08:49:21.194 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-09-10T08:49:21.194 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 2, 2024-09-10T08:49:21.194 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-09-10T08:49:21.194 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-09-10T08:49:21.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.194 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-09-10T08:49:21.194 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-09-10T08:49:21.194 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 4930, 2024-09-10T08:49:21.194 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-09-10T08:49:21.195 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-09-10T08:49:21.195 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 4930, 2024-09-10T08:49:21.195 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-09-10T08:49:21.195 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-09-10T08:49:21.195 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 7, 2024-09-10T08:49:21.195 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-09-10T08:49:21.196 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-09-10T08:49:21.196 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 7, 2024-09-10T08:49:21.196 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 2, 2024-09-10T08:49:21.196 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-09-10T08:49:21.196 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-09-10T08:49:21.196 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194304, 2024-09-10T08:49:21.196 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 11199463, 2024-09-10T08:49:21.196 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 0, 2024-09-10T08:49:21.196 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-09-10T08:49:21.196 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 817, 2024-09-10T08:49:21.196 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 17.889885287, 2024-09-10T08:49:21.196 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.021897044 2024-09-10T08:49:21.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.196 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 0 2024-09-10T08:49:21.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.197 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-09-10T08:49:21.197 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 22, 2024-09-10T08:49:21.197 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 905, 2024-09-10T08:49:21.197 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 883, 2024-09-10T08:49:21.197 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 21, 2024-09-10T08:49:21.197 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 64, 2024-09-10T08:49:21.197 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 43, 2024-09-10T08:49:21.197 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 18, 2024-09-10T08:49:21.197 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 1484, 2024-09-10T08:49:21.197 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 1466, 2024-09-10T08:49:21.197 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 0, 2024-09-10T08:49:21.197 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 1319, 2024-09-10T08:49:21.198 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 1319, 2024-09-10T08:49:21.198 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 60832, 2024-09-10T08:49:21.198 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 223520 2024-09-10T08:49:21.198 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.198 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-09-10T08:49:21.198 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 8727, 2024-09-10T08:49:21.198 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 3469, 2024-09-10T08:49:21.198 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 4509, 2024-09-10T08:49:21.198 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 15, 2024-09-10T08:49:21.198 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 9403, 2024-09-10T08:49:21.198 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-09-10T08:49:21.198 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 718, 2024-09-10T08:49:21.199 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 82.624391638, 2024-09-10T08:49:21.199 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.115075754 2024-09-10T08:49:21.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.199 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-09-10T08:49:21.199 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4, 2024-09-10T08:49:21.199 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.001883046, 2024-09-10T08:49:21.199 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000470761 2024-09-10T08:49:21.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.199 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-09-10T08:49:21.199 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.199 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.200 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.200 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-09-10T08:49:21.200 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.200 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.200 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.201 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-09-10T08:49:21.201 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2142, 2024-09-10T08:49:21.201 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 38.364103795, 2024-09-10T08:49:21.201 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.017910412 2024-09-10T08:49:21.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.201 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-09-10T08:49:21.201 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.201 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.201 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.201 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-09-10T08:49:21.201 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.201 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.202 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.202 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-09-10T08:49:21.202 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.202 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.202 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.202 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-09-10T08:49:21.202 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.202 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.202 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.202 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-09-10T08:49:21.202 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.202 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.203 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.203 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-09-10T08:49:21.203 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.203 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.203 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.203 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-09-10T08:49:21.203 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.204 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.204 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.204 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-09-10T08:49:21.204 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.204 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.204 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.205 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-09-10T08:49:21.205 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.205 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.205 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.205 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-09-10T08:49:21.205 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 615, 2024-09-10T08:49:21.206 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.423500474, 2024-09-10T08:49:21.206 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000688618 2024-09-10T08:49:21.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.206 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-09-10T08:49:21.206 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 16, 2024-09-10T08:49:21.206 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.010399805, 2024-09-10T08:49:21.207 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000649987 2024-09-10T08:49:21.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.207 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-09-10T08:49:21.207 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 719, 2024-09-10T08:49:21.207 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 33.101107770, 2024-09-10T08:49:21.207 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.046037702 2024-09-10T08:49:21.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.208 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-09-10T08:49:21.208 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.208 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.208 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.208 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-09-10T08:49:21.208 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.208 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.209 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.209 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-09-10T08:49:21.209 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.209 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.209 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.209 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-09-10T08:49:21.209 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.209 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.209 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.210 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-09-10T08:49:21.210 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.210 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.210 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.210 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.210 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-09-10T08:49:21.210 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.210 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.210 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.210 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.210 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-09-10T08:49:21.210 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.210 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.210 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.210 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.211 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-09-10T08:49:21.211 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.211 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.211 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.211 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-09-10T08:49:21.211 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.211 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.211 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.211 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-09-10T08:49:21.211 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.211 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.211 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.212 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-09-10T08:49:21.212 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.212 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.212 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.212 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-09-10T08:49:21.212 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 270, 2024-09-10T08:49:21.212 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.513413752, 2024-09-10T08:49:21.212 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001901532 2024-09-10T08:49:21.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.212 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-09-10T08:49:21.212 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-09-10T08:49:21.212 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-09-10T08:49:21.212 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.213 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.213 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.213 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:21.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.213 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-09-10T08:49:21.213 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2024-09-10T08:49:21.213 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 3, 2024-09-10T08:49:21.213 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 2, 2024-09-10T08:49:21.213 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 1, 2024-09-10T08:49:21.213 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-09-10T08:49:21.214 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 673, 2024-09-10T08:49:21.214 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 673, 2024-09-10T08:49:21.214 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 454, 2024-09-10T08:49:21.214 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-09-10T08:49:21.214 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.214 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-09-10T08:49:21.214 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:49:21.214 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-09-10T08:49:21.214 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:49:21.214 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:49:21.214 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:49:21.214 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:49:21.214 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:49:21.214 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:49:21.215 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:49:21.215 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:49:21.215 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:49:21.215 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:49:21.215 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:49:21.215 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:49:21.215 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:49:21.215 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-09-10T08:49:21.215 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:49:21.215 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-09-10T08:49:21.215 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:49:21.215 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:49:21.215 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:49:21.215 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:49:21.216 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:49:21.216 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:49:21.216 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:49:21.216 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-09-10T08:49:21.216 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:49:21.216 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:49:21.216 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:49:21.216 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-09-10T08:49:21.216 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-09-10T08:49:21.216 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-09-10T08:49:21.216 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:49:21.217 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:49:21.217 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:49:21.217 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:49:21.217 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 287984, 2024-09-10T08:49:21.217 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 104, 2024-09-10T08:49:21.217 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 88, 2024-09-10T08:49:21.217 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 1, 2024-09-10T08:49:21.217 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-09-10T08:49:21.217 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-09-10T08:49:21.217 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:49:21.217 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-09-10T08:49:21.217 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:49:21.217 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-09-10T08:49:21.217 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 9048, 2024-09-10T08:49:21.218 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 170, 2024-09-10T08:49:21.218 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:49:21.218 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:49:21.218 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-09-10T08:49:21.218 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-09-10T08:49:21.218 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 90317, 2024-09-10T08:49:21.218 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 406, 2024-09-10T08:49:21.218 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-09-10T08:49:21.218 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-09-10T08:49:21.218 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-09-10T08:49:21.218 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-09-10T08:49:21.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.218 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-09-10T08:49:21.218 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-09-10T08:49:21.219 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-09-10T08:49:21.219 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 4190, 2024-09-10T08:49:21.219 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 7477081, 2024-09-10T08:49:21.219 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 29, 2024-09-10T08:49:21.219 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 4190, 2024-09-10T08:49:21.219 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-09-10T08:49:21.219 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4190, 2024-09-10T08:49:21.219 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 161.341498439, 2024-09-10T08:49:21.219 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.038506324 2024-09-10T08:49:21.219 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.219 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-09-10T08:49:21.219 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-09-10T08:49:21.220 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4190, 2024-09-10T08:49:21.220 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 4409 2024-09-10T08:49:21.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.220 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 4190, 2024-09-10T08:49:21.220 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 1242, 2024-09-10T08:49:21.220 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 2948, 2024-09-10T08:49:21.220 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-09-10T08:49:21.220 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-09-10T08:49:21.220 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 0, 2024-09-10T08:49:21.220 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 10, 2024-09-10T08:49:21.220 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 110, 2024-09-10T08:49:21.220 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 927, 2024-09-10T08:49:21.220 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 87, 2024-09-10T08:49:21.220 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-09-10T08:49:21.221 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-09-10T08:49:21.221 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-09-10T08:49:21.221 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 2, 2024-09-10T08:49:21.221 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 1763, 2024-09-10T08:49:21.221 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-09-10T08:49:21.221 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-09-10T08:49:21.221 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-09-10T08:49:21.221 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 1132, 2024-09-10T08:49:21.221 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 20, 2024-09-10T08:49:21.221 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-09-10T08:49:21.221 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-09-10T08:49:21.221 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-09-10T08:49:21.221 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-09-10T08:49:21.221 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-09-10T08:49:21.221 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-09-10T08:49:21.222 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T08:49:21.222 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-09-10T08:49:21.222 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-09-10T08:49:21.222 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-09-10T08:49:21.222 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-09-10T08:49:21.222 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-09-10T08:49:21.222 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-09-10T08:49:21.222 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-09-10T08:49:21.222 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-09-10T08:49:21.222 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-09-10T08:49:21.222 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-09-10T08:49:21.222 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-09-10T08:49:21.222 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-09-10T08:49:21.222 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-09-10T08:49:21.223 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-09-10T08:49:21.223 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-09-10T08:49:21.223 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-09-10T08:49:21.223 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-09-10T08:49:21.223 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-09-10T08:49:21.223 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-09-10T08:49:21.223 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 96, 2024-09-10T08:49:21.223 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-09-10T08:49:21.223 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 2, 2024-09-10T08:49:21.223 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 12, 2024-09-10T08:49:21.223 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 12, 2024-09-10T08:49:21.223 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-09-10T08:49:21.223 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-09-10T08:49:21.223 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 281, 2024-09-10T08:49:21.223 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 0, 2024-09-10T08:49:21.224 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 77 2024-09-10T08:49:21.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.224 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-09-10T08:49:21.224 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 1, 2024-09-10T08:49:21.224 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-09-10T08:49:21.224 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-09-10T08:49:21.224 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-09-10T08:49:21.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.224 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-09-10T08:49:21.224 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 1051 2024-09-10T08:49:21.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.225 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-09-10T08:49:21.225 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-09-10T08:49:21.225 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 16, 2024-09-10T08:49:21.225 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-09-10T08:49:21.225 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 8, 2024-09-10T08:49:21.225 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 1750, 2024-09-10T08:49:21.225 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 875, 2024-09-10T08:49:21.225 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-09-10T08:49:21.225 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.225 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:49:21.225 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-09-10T08:49:21.225 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-09-10T08:49:21.225 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-09-10T08:49:21.225 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 46002, 2024-09-10T08:49:21.226 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 9430665, 2024-09-10T08:49:21.226 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:21.226 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 46002, 2024-09-10T08:49:21.226 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-09-10T08:49:21.226 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-09-10T08:49:21.226 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 46002, 2024-09-10T08:49:21.226 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 9430665, 2024-09-10T08:49:21.226 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-09-10T08:49:21.226 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.226 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.226 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.226 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:21.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.226 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-09-10T08:49:21.226 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-09-10T08:49:21.227 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-09-10T08:49:21.227 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-09-10T08:49:21.227 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-09-10T08:49:21.228 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-09-10T08:49:21.229 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:21.229 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-09-10T08:49:21.229 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 4190, 2024-09-10T08:49:21.229 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 7572456, 2024-09-10T08:49:21.229 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 4190, 2024-09-10T08:49:21.229 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 7572456, 2024-09-10T08:49:21.229 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-09-10T08:49:21.229 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.229 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.229 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.230 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:21.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.230 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-09-10T08:49:21.230 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-09-10T08:49:21.230 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-09-10T08:49:21.230 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-09-10T08:49:21.230 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-09-10T08:49:21.230 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-09-10T08:49:21.230 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:21.230 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-09-10T08:49:21.230 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 4190, 2024-09-10T08:49:21.230 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 4190, 2024-09-10T08:49:21.231 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 4190, 2024-09-10T08:49:21.231 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 4190, 2024-09-10T08:49:21.231 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-09-10T08:49:21.231 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.231 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.231 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.231 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:21.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.232 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-09-10T08:49:21.232 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-09-10T08:49:21.232 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-09-10T08:49:21.232 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-09-10T08:49:21.232 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 875, 2024-09-10T08:49:21.232 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 95375, 2024-09-10T08:49:21.232 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:21.232 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 875, 2024-09-10T08:49:21.232 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-09-10T08:49:21.232 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-09-10T08:49:21.232 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 110, 2024-09-10T08:49:21.232 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 95375, 2024-09-10T08:49:21.232 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-09-10T08:49:21.233 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.233 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.233 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.233 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:21.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:21.233 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x55aea97f4960": { 2024-09-10T08:49:21.233 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-09-10T08:49:21.234 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-09-10T08:49:21.234 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-09-10T08:49:21.234 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 4930, 2024-09-10T08:49:21.234 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 7005159, 2024-09-10T08:49:21.234 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:21.234 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 4930, 2024-09-10T08:49:21.234 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-09-10T08:49:21.234 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-09-10T08:49:21.234 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 817, 2024-09-10T08:49:21.234 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 7005159, 2024-09-10T08:49:21.235 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-09-10T08:49:21.235 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:21.235 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:21.235 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:21.235 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:21.235 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:21.235 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-09-10T08:49:21.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:20 smithi111 ceph-mon[35220]: pgmap v675: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-10T08:49:21.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:20 smithi089 ceph-mon[30757]: pgmap v675: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-10T08:49:22.027 INFO:tasks.check_counter:Daemon mds.i mds.exported=16 2024-09-10T08:49:22.027 INFO:tasks.check_counter:Daemon mds.i mds.imported=2 2024-09-10T08:49:22.027 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=2 2024-09-10T08:49:22.027 DEBUG:tasks.check_counter:Getting stats from l 2024-09-10T08:49:22.028 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-10T08:49:22.274 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:49:22.786 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-09-10T08:49:22.787 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T08:49:22.787 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 222, 2024-09-10T08:49:22.787 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 215, 2024-09-10T08:49:22.787 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 756464, 2024-09-10T08:49:22.787 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 131072, 2024-09-10T08:49:22.788 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 4, 2024-09-10T08:49:22.788 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 0, 2024-09-10T08:49:22.788 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.034882381, 2024-09-10T08:49:22.788 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.013472989, 2024-09-10T08:49:22.788 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.027578659, 2024-09-10T08:49:22.789 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:49:22.789 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:22.789 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 215, 2024-09-10T08:49:22.789 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.016150205, 2024-09-10T08:49:22.789 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000075117 2024-09-10T08:49:22.789 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:22.790 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:22.790 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:22.790 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:22.790 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:22.790 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:22.790 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 756464, 2024-09-10T08:49:22.791 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 131072 2024-09-10T08:49:22.791 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:22.791 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T08:49:22.791 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 1, 2024-09-10T08:49:22.791 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 162, 2024-09-10T08:49:22.791 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 160, 2024-09-10T08:49:22.792 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 107056, 2024-09-10T08:49:22.792 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 3, 2024-09-10T08:49:22.792 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 2, 2024-09-10T08:49:22.792 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.009870526, 2024-09-10T08:49:22.792 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.007319593, 2024-09-10T08:49:22.793 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.000772033, 2024-09-10T08:49:22.793 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:49:22.793 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:22.793 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 160, 2024-09-10T08:49:22.793 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.010635173, 2024-09-10T08:49:22.793 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000066469 2024-09-10T08:49:22.793 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:22.793 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:22.793 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:22.793 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:22.794 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:22.794 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:22.794 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T08:49:22.794 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 107056 2024-09-10T08:49:22.794 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:22.794 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T08:49:22.794 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 0, 2024-09-10T08:49:22.794 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 2, 2024-09-10T08:49:22.794 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 0, 2024-09-10T08:49:22.794 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 0, 2024-09-10T08:49:22.794 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 2, 2024-09-10T08:49:22.795 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 1, 2024-09-10T08:49:22.795 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.000194873, 2024-09-10T08:49:22.795 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T08:49:22.795 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.000128488, 2024-09-10T08:49:22.795 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T08:49:22.795 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T08:49:22.795 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:22.795 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:22.795 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:22.795 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:22.795 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-09-10T08:49:22.795 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:22.795 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:22.795 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:22.796 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:22.796 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T08:49:22.796 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T08:49:22.796 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:22.796 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-09-10T08:49:22.796 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 0, 2024-09-10T08:49:22.796 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-09-10T08:49:22.796 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:22.796 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-09-10T08:49:22.796 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-09-10T08:49:22.796 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-09-10T08:49:22.796 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T08:49:22.796 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-09-10T08:49:22.797 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T08:49:22.797 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-09-10T08:49:22.797 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T08:49:22.797 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T08:49:22.797 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T08:49:22.797 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T08:49:22.797 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T08:49:22.797 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-09-10T08:49:22.797 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T08:49:22.797 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T08:49:22.797 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-09-10T08:49:22.797 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-09-10T08:49:22.797 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-09-10T08:49:22.797 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-09-10T08:49:22.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T08:49:22.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T08:49:22.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T08:49:22.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T08:49:22.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T08:49:22.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-09-10T08:49:22.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-09-10T08:49:22.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-09-10T08:49:22.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T08:49:22.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T08:49:22.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-09-10T08:49:22.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-09-10T08:49:22.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-09-10T08:49:22.798 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-09-10T08:49:22.799 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T08:49:22.799 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-09-10T08:49:22.799 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T08:49:22.799 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-09-10T08:49:22.799 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 148945, 2024-09-10T08:49:22.799 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 29, 2024-09-10T08:49:22.800 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-09-10T08:49:22.800 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-09-10T08:49:22.800 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-09-10T08:49:22.800 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-09-10T08:49:22.800 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-09-10T08:49:22.800 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-09-10T08:49:22.800 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-09-10T08:49:22.800 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-09-10T08:49:22.800 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 0, 2024-09-10T08:49:22.800 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 0, 2024-09-10T08:49:22.801 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T08:49:22.801 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-09-10T08:49:22.801 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-09-10T08:49:22.801 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-09-10T08:49:22.801 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 928, 2024-09-10T08:49:22.801 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 2, 2024-09-10T08:49:22.801 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-09-10T08:49:22.801 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-09-10T08:49:22.802 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-09-10T08:49:22.802 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-09-10T08:49:22.802 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-10T08:49:22.802 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T08:49:22.802 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-09-10T08:49:22.802 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-09-10T08:49:22.802 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-09-10T08:49:22.802 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 219, 2024-09-10T08:49:22.803 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 722662, 2024-09-10T08:49:22.803 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-09-10T08:49:22.803 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 219, 2024-09-10T08:49:22.803 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-09-10T08:49:22.803 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-09-10T08:49:22.804 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 219, 2024-09-10T08:49:22.804 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 722662, 2024-09-10T08:49:22.804 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-09-10T08:49:22.805 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-09-10T08:49:22.805 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-09-10T08:49:22.805 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-09-10T08:49:22.805 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:22.806 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-10T08:49:22.806 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-09-10T08:49:22.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:22 smithi149 ceph-mon[36167]: pgmap v676: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T08:49:23.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:22 smithi111 ceph-mon[35220]: pgmap v676: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T08:49:23.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:22 smithi089 ceph-mon[30757]: pgmap v676: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T08:49:23.437 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-10T08:49:23.437 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-10T08:49:23.437 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-09-10T08:49:23.437 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-10T08:49:23.447 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-10T08:49:23.448 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:49:23.448 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T08:49:23.489 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:49:23.489 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T08:49:23.569 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T08:49:23.983 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:24.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.534+0000 7f05dfc00640 1 -- 172.21.15.89:0/794841364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05d8102790 msgr2=0x7f05d81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:24.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.534+0000 7f05dfc00640 1 --2- 172.21.15.89:0/794841364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05d8102790 0x7f05d81067e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f05c4009900 tx=0x7f05c402a410 comp rx=0 tx=0).stop 2024-09-10T08:49:24.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.534+0000 7f05dfc00640 1 -- 172.21.15.89:0/794841364 shutdown_connections 2024-09-10T08:49:24.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.534+0000 7f05dfc00640 1 --2- 172.21.15.89:0/794841364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05d8106f10 0x7f05d810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:24.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.534+0000 7f05dfc00640 1 --2- 172.21.15.89:0/794841364 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05d8102790 0x7f05d81067e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:24.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.534+0000 7f05dfc00640 1 --2- 172.21.15.89:0/794841364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05d8101de0 0x7f05d81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:24.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.534+0000 7f05dfc00640 1 -- 172.21.15.89:0/794841364 >> 172.21.15.89:0/794841364 conn(0x7f05d80fd8c0 msgr2=0x7f05d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:24.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.534+0000 7f05dfc00640 1 -- 172.21.15.89:0/794841364 shutdown_connections 2024-09-10T08:49:24.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.534+0000 7f05dfc00640 1 -- 172.21.15.89:0/794841364 wait complete. 2024-09-10T08:49:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.535+0000 7f05dfc00640 1 Processor -- start 2024-09-10T08:49:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.536+0000 7f05dfc00640 1 -- start start 2024-09-10T08:49:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.536+0000 7f05dfc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05d8101de0 0x7f05d8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.536+0000 7f05dfc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05d8102790 0x7f05d8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.536+0000 7f05dfc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05d8106f10 0x7f05d819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:24.538 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.536+0000 7f05dfc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f05d8111f80 con 0x7f05d8102790 2024-09-10T08:49:24.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.536+0000 7f05dfc00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f05d8111e00 con 0x7f05d8101de0 2024-09-10T08:49:24.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.536+0000 7f05dfc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f05d8112100 con 0x7f05d8106f10 2024-09-10T08:49:24.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.536+0000 7f05d7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05d8102790 0x7f05d8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:24.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.536+0000 7f05d7e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05d8102790 0x7f05d8198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49028/0 (socket says 172.21.15.89:49028) 2024-09-10T08:49:24.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.536+0000 7f05d7e00640 1 -- 172.21.15.89:0/2493514290 learned_addr learned my addr 172.21.15.89:0/2493514290 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:49:24.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.536+0000 7f05dd200640 1 --2- 172.21.15.89:0/2493514290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05d8101de0 0x7f05d8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:24.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.536+0000 7f05ddc00640 1 --2- 172.21.15.89:0/2493514290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05d8106f10 0x7f05d819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:24.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.537+0000 7f05d7e00640 1 -- 172.21.15.89:0/2493514290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05d8106f10 msgr2=0x7f05d819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:24.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.537+0000 7f05d7e00640 1 --2- 172.21.15.89:0/2493514290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05d8106f10 0x7f05d819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:24.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.537+0000 7f05d7e00640 1 -- 172.21.15.89:0/2493514290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05d8101de0 msgr2=0x7f05d8198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:24.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.537+0000 7f05d7e00640 1 --2- 172.21.15.89:0/2493514290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05d8101de0 0x7f05d8198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:24.541 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.537+0000 7f05d7e00640 1 -- 172.21.15.89:0/2493514290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d819d3f0 con 0x7f05d8102790 2024-09-10T08:49:24.542 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.537+0000 7f05d7e00640 1 --2- 172.21.15.89:0/2493514290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05d8102790 0x7f05d8198980 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f05c4004770 tx=0x7f05c4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:24.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.537+0000 7f05d5600640 1 -- 172.21.15.89:0/2493514290 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c403f070 con 0x7f05d8102790 2024-09-10T08:49:24.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.537+0000 7f05d5600640 1 -- 172.21.15.89:0/2493514290 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05c4031ca0 con 0x7f05d8102790 2024-09-10T08:49:24.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.537+0000 7f05d5600640 1 -- 172.21.15.89:0/2493514290 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c4035030 con 0x7f05d8102790 2024-09-10T08:49:24.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.537+0000 7f05dfc00640 1 -- 172.21.15.89:0/2493514290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d819d680 con 0x7f05d8102790 2024-09-10T08:49:24.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.538+0000 7f05dfc00640 1 -- 172.21.15.89:0/2493514290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d819db10 con 0x7f05d8102790 2024-09-10T08:49:24.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.539+0000 7f05dfc00640 1 -- 172.21.15.89:0/2493514290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05d81021c0 con 0x7f05d8102790 2024-09-10T08:49:24.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.540+0000 7f05d5600640 1 -- 172.21.15.89:0/2493514290 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f05c4042070 con 0x7f05d8102790 2024-09-10T08:49:24.543 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.542+0000 7f05d5600640 1 --2- 172.21.15.89:0/2493514290 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f05c00a2c00 0x7f05c00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:24.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.542+0000 7f05d5600640 1 -- 172.21.15.89:0/2493514290 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f05c40efa70 con 0x7f05d8102790 2024-09-10T08:49:24.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.542+0000 7f05dd200640 1 --2- 172.21.15.89:0/2493514290 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f05c00a2c00 0x7f05c00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:24.544 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.543+0000 7f05dd200640 1 --2- 172.21.15.89:0/2493514290 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f05c00a2c00 0x7f05c00a50c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f05cc005d00 tx=0x7f05cc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:24.547 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.546+0000 7f05d5600640 1 -- 172.21.15.89:0/2493514290 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f05c40bbf50 con 0x7f05d8102790 2024-09-10T08:49:24.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.733+0000 7f05dfc00640 1 -- 172.21.15.89:0/2493514290 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f05d802d7c0 con 0x7f05d8102790 2024-09-10T08:49:24.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.734+0000 7f05d5600640 1 -- 172.21.15.89:0/2493514290 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v96) ==== 81+0+590 (secure 0 0 0) 0x7f05c40c0250 con 0x7f05d8102790 2024-09-10T08:49:24.735 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/850669510 2024-09-11T08:29:41.472002+0000 2024-09-10T08:49:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/3541027428 2024-09-11T08:29:41.472002+0000 2024-09-10T08:49:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6801/3794982914 2024-09-11T08:29:41.472002+0000 2024-09-10T08:49:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6800/3794982914 2024-09-11T08:29:41.472002+0000 2024-09-10T08:49:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2396468787 2024-09-11T08:29:41.472002+0000 2024-09-10T08:49:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/1609720980 2024-09-11T08:29:14.554869+0000 2024-09-10T08:49:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/995019144 2024-09-11T08:29:14.554869+0000 2024-09-10T08:49:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6801/4223112778 2024-09-11T08:29:14.554869+0000 2024-09-10T08:49:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:6800/4223112778 2024-09-11T08:29:14.554869+0000 2024-09-10T08:49:24.736 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89:0/2179977257 2024-09-11T08:29:14.554869+0000 2024-09-10T08:49:24.736 INFO:teuthology.orchestra.run.smithi089.stderr:listed 10 entries 2024-09-10T08:49:24.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.736+0000 7f05dfc00640 1 -- 172.21.15.89:0/2493514290 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f05c00a2c00 msgr2=0x7f05c00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:24.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.736+0000 7f05dfc00640 1 --2- 172.21.15.89:0/2493514290 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f05c00a2c00 0x7f05c00a50c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f05cc005d00 tx=0x7f05cc00b410 comp rx=0 tx=0).stop 2024-09-10T08:49:24.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.736+0000 7f05dfc00640 1 -- 172.21.15.89:0/2493514290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05d8102790 msgr2=0x7f05d8198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:24.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.736+0000 7f05dfc00640 1 --2- 172.21.15.89:0/2493514290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05d8102790 0x7f05d8198980 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f05c4004770 tx=0x7f05c4003c10 comp rx=0 tx=0).stop 2024-09-10T08:49:24.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.736+0000 7f05dfc00640 1 -- 172.21.15.89:0/2493514290 shutdown_connections 2024-09-10T08:49:24.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.736+0000 7f05dfc00640 1 --2- 172.21.15.89:0/2493514290 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f05c00a2c00 0x7f05c00a50c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:24.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.736+0000 7f05dfc00640 1 --2- 172.21.15.89:0/2493514290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05d8106f10 0x7f05d819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:24.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.736+0000 7f05dfc00640 1 --2- 172.21.15.89:0/2493514290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05d8102790 0x7f05d8198980 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:24.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.736+0000 7f05dfc00640 1 --2- 172.21.15.89:0/2493514290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05d8101de0 0x7f05d8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:24.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.737+0000 7f05dfc00640 1 -- 172.21.15.89:0/2493514290 >> 172.21.15.89:0/2493514290 conn(0x7f05d80fd8c0 msgr2=0x7f05d80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:24.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.737+0000 7f05dfc00640 1 -- 172.21.15.89:0/2493514290 shutdown_connections 2024-09-10T08:49:24.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:24.737+0000 7f05dfc00640 1 -- 172.21.15.89:0/2493514290 wait complete. 2024-09-10T08:49:25.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:24 smithi089 ceph-mon[30757]: pgmap v677: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T08:49:25.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:24 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2493514290' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:49:25.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:24 smithi149 ceph-mon[36167]: pgmap v677: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T08:49:25.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:24 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2493514290' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:49:25.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:24 smithi111 ceph-mon[35220]: pgmap v677: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T08:49:25.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:24 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2493514290' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:49:25.607 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T08:49:25.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:49:25.608 DEBUG:teuthology.orchestra.run.smithi089:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T08:49:25.758 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi089.front.sepia.ceph.com 2024-09-10T08:49:25.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:49:25.758 DEBUG:teuthology.orchestra.run.smithi089:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T08:49:25.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:49:25.836 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2024-09-10T08:49:25.902 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T08:49:25.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:49:25.902 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T08:49:25.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:49:25.961 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link delete ceph-brx 2024-09-10T08:49:26.285 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-10T08:49:26.285 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-10T08:49:26.285 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T08:49:26.322 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/464d23ed4592244eabfddda86578290c50b18381cd0cb45cf1e4dd757c52f12c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/AHOPTXHKD72IY4YDV3PGILT527:/var/lib/containers/storage/overlay/l/VM47NAAS5DSXPW5R3CYHPS6YEY,upperdir=/var/lib/containers/storage/overlay/464d23ed4592244eabfddda86578290c50b18381cd0cb45cf1e4dd757c52f12c/diff,workdir=/var/lib/containers/storage/overlay/464d23ed4592244eabfddda86578290c50b18381cd0cb45cf1e4dd757c52f12c/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/bbaa2b19f14ede464ff8db828d4003f1e815414f6abfaa6f717bad4e8ad8270a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/AHOPTXHKD72IY4YDV3PGILT527:/var/lib/containers/storage/overlay/l/VM47NAAS5DSXPW5R3CYHPS6YEY,upperdir=/var/lib/containers/storage/overlay/bbaa2b19f14ede464ff8db828d4003f1e815414f6abfaa6f717bad4e8ad8270a/diff,workdir=/var/lib/containers/storage/overlay/bbaa2b19f14ede464ff8db828d4003f1e815414f6abfaa6f717bad4e8ad8270a/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/72d346c3b4a424b3e242ff25039cd7e291d75976e2f165dc92403af3f7579412/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/AHOPTXHKD72IY4YDV3PGILT527:/var/lib/containers/storage/overlay/l/VM47NAAS5DSXPW5R3CYHPS6YEY,upperdir=/var/lib/containers/storage/overlay/72d346c3b4a424b3e242ff25039cd7e291d75976e2f165dc92403af3f7579412/diff,workdir=/var/lib/containers/storage/overlay/72d346c3b4a424b3e242ff25039cd7e291d75976e2f165dc92403af3f7579412/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/7ffeaf1e09c3ab61386ca16c7a5f623cc756860f136b7258fe877a099004e556/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/AHOPTXHKD72IY4YDV3PGILT527:/var/lib/containers/storage/overlay/l/VM47NAAS5DSXPW5R3CYHPS6YEY,upperdir=/var/lib/containers/storage/overlay/7ffeaf1e09c3ab61386ca16c7a5f623cc756860f136b7258fe877a099004e556/diff,workdir=/var/lib/containers/storage/overlay/7ffeaf1e09c3ab61386ca16c7a5f623cc756860f136b7258fe877a099004e556/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4efefdaf9407474ec9b1c6e4887ef2d5b2e0fc6e97288b72f9298c8ea4bc36ae/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/AHOPTXHKD72IY4YDV3PGILT527:/var/lib/containers/storage/overlay/l/VM47NAAS5DSXPW5R3CYHPS6YEY,upperdir=/var/lib/containers/storage/overlay/4efefdaf9407474ec9b1c6e4887ef2d5b2e0fc6e97288b72f9298c8ea4bc36ae/diff,workdir=/var/lib/containers/storage/overlay/4efefdaf9407474ec9b1c6e4887ef2d5b2e0fc6e97288b72f9298c8ea4bc36ae/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b2882445911fb66efb3ac24447b70137482a8705ee28adf3b3863dac3dadefbd/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/AHOPTXHKD72IY4YDV3PGILT527:/var/lib/containers/storage/overlay/l/VM47NAAS5DSXPW5R3CYHPS6YEY,upperdir=/var/lib/containers/storage/overlay/b2882445911fb66efb3ac24447b70137482a8705ee28adf3b3863dac3dadefbd/diff,workdir=/var/lib/containers/storage/overlay/b2882445911fb66efb3ac24447b70137482a8705ee28adf3b3863dac3dadefbd/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/12ce0649b002510fc4c6d7dfdde716190dc4ef70092e8e69d142bc16e44af6af/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/AHOPTXHKD72IY4YDV3PGILT527:/var/lib/containers/storage/overlay/l/VM47NAAS5DSXPW5R3CYHPS6YEY,upperdir=/var/lib/containers/storage/overlay/12ce0649b002510fc4c6d7dfdde716190dc4ef70092e8e69d142bc16e44af6af/diff,workdir=/var/lib/containers/storage/overlay/12ce0649b002510fc4c6d7dfdde716190dc4ef70092e8e69d142bc16e44af6af/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/5d17df5dbd72929c3b159ea0511a9d955baa44d0a646747cf5cded5118596a37/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/AHOPTXHKD72IY4YDV3PGILT527:/var/lib/containers/storage/overlay/l/VM47NAAS5DSXPW5R3CYHPS6YEY,upperdir=/var/lib/containers/storage/overlay/5d17df5dbd72929c3b159ea0511a9d955baa44d0a646747cf5cded5118596a37/diff,workdir=/var/lib/containers/storage/overlay/5d17df5dbd72929c3b159ea0511a9d955baa44d0a646747cf5cded5118596a37/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4eb5ed7db5a1053be0912a28ceb22facaeccfbad55567975c6ef122fc33d3413/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/AHOPTXHKD72IY4YDV3PGILT527:/var/lib/containers/storage/overlay/l/VM47NAAS5DSXPW5R3CYHPS6YEY,upperdir=/var/lib/containers/storage/overlay/4eb5ed7db5a1053be0912a28ceb22facaeccfbad55567975c6ef122fc33d3413/diff,workdir=/var/lib/containers/storage/overlay/4eb5ed7db5a1053be0912a28ceb22facaeccfbad55567975c6ef122fc33d3413/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4d89994cd12257862b7d66411bf9ef1698565d377ff446ba8766503e11c332c1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/AHOPTXHKD72IY4YDV3PGILT527:/var/lib/containers/storage/overlay/l/VM47NAAS5DSXPW5R3CYHPS6YEY,upperdir=/var/lib/containers/storage/overlay/4d89994cd12257862b7d66411bf9ef1698565d377ff446ba8766503e11c332c1/diff,workdir=/var/lib/containers/storage/overlay/4d89994cd12257862b7d66411bf9ef1698565d377ff446ba8766503e11c332c1/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 2024-09-10T08:49:26.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:49:26.323 DEBUG:teuthology.orchestra.run.smithi089:> ip netns list 2024-09-10T08:49:26.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:49:26.400 DEBUG:teuthology.orchestra.run.smithi089:> sudo ip link delete ceph-brx 2024-09-10T08:49:26.501 INFO:teuthology.orchestra.run.smithi089.stderr:Cannot find device "ceph-brx" 2024-09-10T08:49:26.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:49:26.505 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T08:49:27.158 INFO:tasks.cephadm:Teardown begin 2024-09-10T08:49:27.159 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T08:49:27.159 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:49:27.210 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:26 smithi089 ceph-mon[30757]: pgmap v678: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-10T08:49:27.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:26 smithi149 ceph-mon[36167]: pgmap v678: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-10T08:49:27.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:26 smithi111 ceph-mon[35220]: pgmap v678: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-10T08:49:27.530 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:28.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.121+0000 7f161e400640 1 -- 172.21.15.89:0/2087289790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1618102790 msgr2=0x7f16181067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:28.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.121+0000 7f161e400640 1 --2- 172.21.15.89:0/2087289790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1618102790 0x7f16181067e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1600009900 tx=0x7f160002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:49:28.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.121+0000 7f161e400640 1 -- 172.21.15.89:0/2087289790 shutdown_connections 2024-09-10T08:49:28.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.121+0000 7f161e400640 1 --2- 172.21.15.89:0/2087289790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1618106f10 0x7f161810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:28.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.121+0000 7f161e400640 1 --2- 172.21.15.89:0/2087289790 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1618102790 0x7f16181067e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:28.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.121+0000 7f161e400640 1 --2- 172.21.15.89:0/2087289790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1618101de0 0x7f16181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:28.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.121+0000 7f161e400640 1 -- 172.21.15.89:0/2087289790 >> 172.21.15.89:0/2087289790 conn(0x7f16180fd8c0 msgr2=0x7f16180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:28.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.121+0000 7f161e400640 1 -- 172.21.15.89:0/2087289790 shutdown_connections 2024-09-10T08:49:28.123 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.121+0000 7f161e400640 1 -- 172.21.15.89:0/2087289790 wait complete. 2024-09-10T08:49:28.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.122+0000 7f161e400640 1 Processor -- start 2024-09-10T08:49:28.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.122+0000 7f161e400640 1 -- start start 2024-09-10T08:49:28.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.122+0000 7f161e400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1618101de0 0x7f1618198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:28.125 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.122+0000 7f161e400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1618102790 0x7f1618198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.122+0000 7f161e400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1618106f10 0x7f161819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.123+0000 7f161e400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1618111f30 con 0x7f1618102790 2024-09-10T08:49:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.123+0000 7f161e400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1618111db0 con 0x7f1618101de0 2024-09-10T08:49:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.123+0000 7f161e400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f16181120b0 con 0x7f1618106f10 2024-09-10T08:49:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.123+0000 7f160ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1618102790 0x7f1618198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.123+0000 7f1617400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1618101de0 0x7f1618198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.123+0000 7f1617400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1618101de0 0x7f1618198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:44998/0 (socket says 172.21.15.89:44998) 2024-09-10T08:49:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.123+0000 7f1617400640 1 -- 172.21.15.89:0/44103472 learned_addr learned my addr 172.21.15.89:0/44103472 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:49:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.123+0000 7f1617e00640 1 --2- 172.21.15.89:0/44103472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1618106f10 0x7f161819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.123+0000 7f160ea00640 1 -- 172.21.15.89:0/44103472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1618106f10 msgr2=0x7f161819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.123+0000 7f160ea00640 1 --2- 172.21.15.89:0/44103472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1618106f10 0x7f161819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:28.126 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.123+0000 7f160ea00640 1 -- 172.21.15.89:0/44103472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1618101de0 msgr2=0x7f1618198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.123+0000 7f160ea00640 1 --2- 172.21.15.89:0/44103472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1618101de0 0x7f1618198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.123+0000 7f160ea00640 1 -- 172.21.15.89:0/44103472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161819d400 con 0x7f1618102790 2024-09-10T08:49:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.124+0000 7f160ea00640 1 --2- 172.21.15.89:0/44103472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1618102790 0x7f1618198990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1600005940 tx=0x7f1600004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.124+0000 7f1614c00640 1 -- 172.21.15.89:0/44103472 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160003f030 con 0x7f1618102790 2024-09-10T08:49:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.124+0000 7f1614c00640 1 -- 172.21.15.89:0/44103472 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1600003ad0 con 0x7f1618102790 2024-09-10T08:49:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.124+0000 7f1614c00640 1 -- 172.21.15.89:0/44103472 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600035070 con 0x7f1618102790 2024-09-10T08:49:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.124+0000 7f161e400640 1 -- 172.21.15.89:0/44103472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f161819d690 con 0x7f1618102790 2024-09-10T08:49:28.127 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.124+0000 7f161e400640 1 -- 172.21.15.89:0/44103472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16181a4fd0 con 0x7f1618102790 2024-09-10T08:49:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.126+0000 7f161e400640 1 -- 172.21.15.89:0/44103472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16181032c0 con 0x7f1618102790 2024-09-10T08:49:28.128 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.127+0000 7f1614c00640 1 -- 172.21.15.89:0/44103472 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1600042030 con 0x7f1618102790 2024-09-10T08:49:28.129 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.128+0000 7f1614c00640 1 --2- 172.21.15.89:0/44103472 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f15fc0a2b80 0x7f15fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.129+0000 7f1614c00640 1 -- 172.21.15.89:0/44103472 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f16000ef9d0 con 0x7f1618102790 2024-09-10T08:49:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.129+0000 7f1617400640 1 --2- 172.21.15.89:0/44103472 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f15fc0a2b80 0x7f15fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:28.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.129+0000 7f1617400640 1 --2- 172.21.15.89:0/44103472 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f15fc0a2b80 0x7f15fc0a5040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1608009fd0 tx=0x7f1608009720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:28.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.133+0000 7f1614c00640 1 -- 172.21.15.89:0/44103472 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16000bbeb0 con 0x7f1618102790 2024-09-10T08:49:28.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.348+0000 7f161e400640 1 -- 172.21.15.89:0/44103472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f161810a120 con 0x7f1618102790 2024-09-10T08:49:28.350 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.349+0000 7f1614c00640 1 -- 172.21.15.89:0/44103472 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11921 (secure 0 0 0) 0x7f16000c01b0 con 0x7f1618102790 2024-09-10T08:49:28.351 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:49:28.351 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":33,"btime":"2024-09-10T08:49:01:041663+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:49:00.247361+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":158,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24496,24518,24554]},"id":1}]} 2024-09-10T08:49:28.351 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 33 2024-09-10T08:49:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.352+0000 7f161e400640 1 -- 172.21.15.89:0/44103472 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f15fc0a2b80 msgr2=0x7f15fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.352+0000 7f161e400640 1 --2- 172.21.15.89:0/44103472 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f15fc0a2b80 0x7f15fc0a5040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1608009fd0 tx=0x7f1608009720 comp rx=0 tx=0).stop 2024-09-10T08:49:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.352+0000 7f161e400640 1 -- 172.21.15.89:0/44103472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1618102790 msgr2=0x7f1618198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.352+0000 7f161e400640 1 --2- 172.21.15.89:0/44103472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1618102790 0x7f1618198990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1600005940 tx=0x7f1600004270 comp rx=0 tx=0).stop 2024-09-10T08:49:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.352+0000 7f161e400640 1 -- 172.21.15.89:0/44103472 shutdown_connections 2024-09-10T08:49:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.352+0000 7f161e400640 1 --2- 172.21.15.89:0/44103472 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f15fc0a2b80 0x7f15fc0a5040 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.352+0000 7f161e400640 1 --2- 172.21.15.89:0/44103472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1618106f10 0x7f161819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:28.353 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.352+0000 7f161e400640 1 --2- 172.21.15.89:0/44103472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1618102790 0x7f1618198990 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:28.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.352+0000 7f161e400640 1 --2- 172.21.15.89:0/44103472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1618101de0 0x7f1618198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:28.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.352+0000 7f161e400640 1 -- 172.21.15.89:0/44103472 >> 172.21.15.89:0/44103472 conn(0x7f16180fd8c0 msgr2=0x7f16180fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:28.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.352+0000 7f161e400640 1 -- 172.21.15.89:0/44103472 shutdown_connections 2024-09-10T08:49:28.354 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:28.352+0000 7f161e400640 1 -- 172.21.15.89:0/44103472 wait complete. 2024-09-10T08:49:29.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:28 smithi089 ceph-mon[30757]: pgmap v679: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-10T08:49:29.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:28 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/44103472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:49:29.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:28 smithi149 ceph-mon[36167]: pgmap v679: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-10T08:49:29.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:28 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/44103472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:49:29.230 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:49:29.293 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:28 smithi111 ceph-mon[35220]: pgmap v679: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-10T08:49:29.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:28 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/44103472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:49:29.601 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:30.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.205+0000 7fbbc9a00640 1 -- 172.21.15.89:0/3559071176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbc4104010 msgr2=0x7fbbc41043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:30.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.205+0000 7fbbc9a00640 1 --2- 172.21.15.89:0/3559071176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbc4104010 0x7fbbc41043f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4009900 tx=0x7fbbb402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:49:30.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.205+0000 7fbbc9a00640 1 -- 172.21.15.89:0/3559071176 shutdown_connections 2024-09-10T08:49:30.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.205+0000 7fbbc9a00640 1 --2- 172.21.15.89:0/3559071176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc4109140 0x7fbbc4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:30.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.205+0000 7fbbc9a00640 1 --2- 172.21.15.89:0/3559071176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbc41049c0 0x7fbbc4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:30.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.205+0000 7fbbc9a00640 1 --2- 172.21.15.89:0/3559071176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbc4104010 0x7fbbc41043f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:30.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.205+0000 7fbbc9a00640 1 -- 172.21.15.89:0/3559071176 >> 172.21.15.89:0/3559071176 conn(0x7fbbc40ffa50 msgr2=0x7fbbc4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:30.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.205+0000 7fbbc9a00640 1 -- 172.21.15.89:0/3559071176 shutdown_connections 2024-09-10T08:49:30.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.205+0000 7fbbc9a00640 1 -- 172.21.15.89:0/3559071176 wait complete. 2024-09-10T08:49:30.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.206+0000 7fbbc9a00640 1 Processor -- start 2024-09-10T08:49:30.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.206+0000 7fbbc9a00640 1 -- start start 2024-09-10T08:49:30.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.206+0000 7fbbc9a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbc4104010 0x7fbbc419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:30.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.207+0000 7fbbc9a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc41049c0 0x7fbbc419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:30.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.207+0000 7fbbc9a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbc4109140 0x7fbbc419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:30.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.207+0000 7fbbc9a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41141c0 con 0x7fbbc4104010 2024-09-10T08:49:30.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.207+0000 7fbbc9a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4114040 con 0x7fbbc4109140 2024-09-10T08:49:30.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.207+0000 7fbbc9a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4114340 con 0x7fbbc41049c0 2024-09-10T08:49:30.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.207+0000 7fbbc2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbc4104010 0x7fbbc419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:30.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.207+0000 7fbbc2000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc41049c0 0x7fbbc419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:30.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.207+0000 7fbbc2000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc41049c0 0x7fbbc419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:47778/0 (socket says 172.21.15.89:47778) 2024-09-10T08:49:30.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.207+0000 7fbbc2a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbc4104010 0x7fbbc419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49068/0 (socket says 172.21.15.89:49068) 2024-09-10T08:49:30.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.207+0000 7fbbc2000640 1 -- 172.21.15.89:0/3509234225 learned_addr learned my addr 172.21.15.89:0/3509234225 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:49:30.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.207+0000 7fbbc3400640 1 --2- 172.21.15.89:0/3509234225 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbc4109140 0x7fbbc419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:30.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.207+0000 7fbbc2000640 1 -- 172.21.15.89:0/3509234225 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbc4109140 msgr2=0x7fbbc419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:30.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.207+0000 7fbbc2000640 1 --2- 172.21.15.89:0/3509234225 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbc4109140 0x7fbbc419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:30.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.207+0000 7fbbc2000640 1 -- 172.21.15.89:0/3509234225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbc4104010 msgr2=0x7fbbc419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:30.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.207+0000 7fbbc2000640 1 --2- 172.21.15.89:0/3509234225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbc4104010 0x7fbbc419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:30.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.207+0000 7fbbc2000640 1 -- 172.21.15.89:0/3509234225 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc419f5f0 con 0x7fbbc41049c0 2024-09-10T08:49:30.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.207+0000 7fbbc2a00640 1 --2- 172.21.15.89:0/3509234225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbc4104010 0x7fbbc419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:49:30.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.208+0000 7fbbc2000640 1 --2- 172.21.15.89:0/3509234225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc41049c0 0x7fbbc419ab80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000ea60 tx=0x7fbbb000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:30.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.208+0000 7fbbbb400640 1 -- 172.21.15.89:0/3509234225 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb000cde0 con 0x7fbbc41049c0 2024-09-10T08:49:30.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.208+0000 7fbbc9a00640 1 -- 172.21.15.89:0/3509234225 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc419f8e0 con 0x7fbbc41049c0 2024-09-10T08:49:30.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.208+0000 7fbbbb400640 1 -- 172.21.15.89:0/3509234225 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbb00035f0 con 0x7fbbc41049c0 2024-09-10T08:49:30.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.208+0000 7fbbbb400640 1 -- 172.21.15.89:0/3509234225 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0010670 con 0x7fbbc41049c0 2024-09-10T08:49:30.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.208+0000 7fbbc9a00640 1 -- 172.21.15.89:0/3509234225 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc41a71c0 con 0x7fbbc41049c0 2024-09-10T08:49:30.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.209+0000 7fbbc9a00640 1 -- 172.21.15.89:0/3509234225 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbc41043f0 con 0x7fbbc41049c0 2024-09-10T08:49:30.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.212+0000 7fbbbb400640 1 -- 172.21.15.89:0/3509234225 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbbb00037a0 con 0x7fbbc41049c0 2024-09-10T08:49:30.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.213+0000 7fbbbb400640 1 --2- 172.21.15.89:0/3509234225 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbba80a2b80 0x7fbba80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:30.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.213+0000 7fbbbb400640 1 -- 172.21.15.89:0/3509234225 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fbbb00cf700 con 0x7fbbc41049c0 2024-09-10T08:49:30.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.213+0000 7fbbc2a00640 1 --2- 172.21.15.89:0/3509234225 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbba80a2b80 0x7fbba80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:30.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.214+0000 7fbbc2a00640 1 --2- 172.21.15.89:0/3509234225 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbba80a2b80 0x7fbba80a5040 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4000c00 tx=0x7fbbb4031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:30.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.215+0000 7fbbbb400640 1 -- 172.21.15.89:0/3509234225 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbbb009c230 con 0x7fbbc41049c0 2024-09-10T08:49:30.427 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.425+0000 7fbbc9a00640 1 -- 172.21.15.89:0/3509234225 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbc419b9a0 con 0x7fbbc41049c0 2024-09-10T08:49:30.428 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.427+0000 7fbbbb400640 1 -- 172.21.15.89:0/3509234225 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11921 (secure 0 0 0) 0x7fbbb009bb30 con 0x7fbbc41049c0 2024-09-10T08:49:30.428 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:49:30.428 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":33,"btime":"2024-09-10T08:49:01:041663+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/2129325722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":2129325722},{"type":"v1","addr":"172.21.15.89:6835","nonce":2129325722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3715416661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3715416661},{"type":"v1","addr":"172.21.15.89:6837","nonce":3715416661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2573030321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2573030321},{"type":"v1","addr":"172.21.15.89:6839","nonce":2573030321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6841/3524299195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6840","nonce":3524299195},{"type":"v1","addr":"172.21.15.89:6841","nonce":3524299195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/210698608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":210698608},{"type":"v1","addr":"172.21.15.149:6833","nonce":210698608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2902311847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2902311847},{"type":"v1","addr":"172.21.15.111:6833","nonce":2902311847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:36:57.162188+0000","modified":"2024-09-10T08:49:00.247361+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24518,"mds_2":24496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/881263752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":881263752},{"type":"v1","addr":"172.21.15.149:6835","nonce":881263752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/2367754626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2367754626},{"type":"v1","addr":"172.21.15.149:6837","nonce":2367754626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/285258866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":285258866},{"type":"v1","addr":"172.21.15.111:6835","nonce":285258866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.149:6839/2020874598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2020874598},{"type":"v1","addr":"172.21.15.149:6839","nonce":2020874598}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":158,"addr":"172.21.15.111:6837/779590543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":779590543},{"type":"v1","addr":"172.21.15.111:6837","nonce":779590543}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/3640096926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":3640096926},{"type":"v1","addr":"172.21.15.111:6839","nonce":3640096926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24496,24518,24554]},"id":1}]} 2024-09-10T08:49:30.429 INFO:teuthology.orchestra.run.smithi089.stderr:dumped fsmap epoch 33 2024-09-10T08:49:30.430 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.429+0000 7fbbc9a00640 1 -- 172.21.15.89:0/3509234225 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbba80a2b80 msgr2=0x7fbba80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:30.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.429+0000 7fbbc9a00640 1 --2- 172.21.15.89:0/3509234225 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbba80a2b80 0x7fbba80a5040 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4000c00 tx=0x7fbbb4031510 comp rx=0 tx=0).stop 2024-09-10T08:49:30.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.429+0000 7fbbc9a00640 1 -- 172.21.15.89:0/3509234225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc41049c0 msgr2=0x7fbbc419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:30.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.429+0000 7fbbc9a00640 1 --2- 172.21.15.89:0/3509234225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc41049c0 0x7fbbc419ab80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000ea60 tx=0x7fbbb000ef20 comp rx=0 tx=0).stop 2024-09-10T08:49:30.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.429+0000 7fbbc9a00640 1 -- 172.21.15.89:0/3509234225 shutdown_connections 2024-09-10T08:49:30.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.429+0000 7fbbc9a00640 1 --2- 172.21.15.89:0/3509234225 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fbba80a2b80 0x7fbba80a5040 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:30.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.429+0000 7fbbc9a00640 1 --2- 172.21.15.89:0/3509234225 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbc4109140 0x7fbbc419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:30.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.429+0000 7fbbc9a00640 1 --2- 172.21.15.89:0/3509234225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc41049c0 0x7fbbc419ab80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:30.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.429+0000 7fbbc9a00640 1 --2- 172.21.15.89:0/3509234225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbc4104010 0x7fbbc419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:30.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.429+0000 7fbbc9a00640 1 -- 172.21.15.89:0/3509234225 >> 172.21.15.89:0/3509234225 conn(0x7fbbc40ffa50 msgr2=0x7fbbc4100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:30.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.429+0000 7fbbc9a00640 1 -- 172.21.15.89:0/3509234225 shutdown_connections 2024-09-10T08:49:30.431 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:30.429+0000 7fbbc9a00640 1 -- 172.21.15.89:0/3509234225 wait complete. 2024-09-10T08:49:31.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:30 smithi149 ceph-mon[36167]: pgmap v680: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-10T08:49:31.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:30 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3509234225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:49:31.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:30 smithi111 ceph-mon[35220]: pgmap v680: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-10T08:49:31.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:30 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3509234225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:49:31.302 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:30 smithi089 ceph-mon[30757]: pgmap v680: 129 pgs: 129 active+clean; 51 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-10T08:49:31.302 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:30 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3509234225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:49:31.341 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:49:31.708 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:32.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.213+0000 7f91bda00640 1 -- 172.21.15.89:0/653233731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91b8102790 msgr2=0x7f91b81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:32.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.213+0000 7f91bda00640 1 --2- 172.21.15.89:0/653233731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91b8102790 0x7f91b81067e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009900 tx=0x7f91a002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.214+0000 7f91bda00640 1 -- 172.21.15.89:0/653233731 shutdown_connections 2024-09-10T08:49:32.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.214+0000 7f91bda00640 1 --2- 172.21.15.89:0/653233731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91b8106f10 0x7f91b810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.214+0000 7f91bda00640 1 --2- 172.21.15.89:0/653233731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91b8102790 0x7f91b81067e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.214+0000 7f91bda00640 1 --2- 172.21.15.89:0/653233731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91b8101de0 0x7f91b81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.214+0000 7f91bda00640 1 -- 172.21.15.89:0/653233731 >> 172.21.15.89:0/653233731 conn(0x7f91b80fd8c0 msgr2=0x7f91b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:32.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.214+0000 7f91bda00640 1 -- 172.21.15.89:0/653233731 shutdown_connections 2024-09-10T08:49:32.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.214+0000 7f91bda00640 1 -- 172.21.15.89:0/653233731 wait complete. 2024-09-10T08:49:32.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.215+0000 7f91bda00640 1 Processor -- start 2024-09-10T08:49:32.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.215+0000 7f91bda00640 1 -- start start 2024-09-10T08:49:32.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.215+0000 7f91bda00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91b8101de0 0x7f91b819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.215+0000 7f91bda00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91b8102790 0x7f91b819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.215+0000 7f91bda00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91b8106f10 0x7f91b819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.215+0000 7f91bda00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f91b8111fc0 con 0x7f91b8102790 2024-09-10T08:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.215+0000 7f91bda00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f91b8111e40 con 0x7f91b8101de0 2024-09-10T08:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.215+0000 7f91bda00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f91b8112140 con 0x7f91b8106f10 2024-09-10T08:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.215+0000 7f91b6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91b8101de0 0x7f91b819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.215+0000 7f91ae000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91b8102790 0x7f91b819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.215+0000 7f91b6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91b8101de0 0x7f91b819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:45032/0 (socket says 172.21.15.89:45032) 2024-09-10T08:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.215+0000 7f91b6a00640 1 -- 172.21.15.89:0/54359999 learned_addr learned my addr 172.21.15.89:0/54359999 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.216+0000 7f91b7400640 1 --2- 172.21.15.89:0/54359999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91b8106f10 0x7f91b819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.216+0000 7f91b6a00640 1 -- 172.21.15.89:0/54359999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91b8106f10 msgr2=0x7f91b819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.216+0000 7f91b6a00640 1 --2- 172.21.15.89:0/54359999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91b8106f10 0x7f91b819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.216+0000 7f91b6a00640 1 -- 172.21.15.89:0/54359999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91b8102790 msgr2=0x7f91b819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.216+0000 7f91b6a00640 1 --2- 172.21.15.89:0/54359999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91b8102790 0x7f91b819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.216+0000 7f91b6a00640 1 -- 172.21.15.89:0/54359999 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b819f600 con 0x7f91b8101de0 2024-09-10T08:49:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.216+0000 7f91b6a00640 1 --2- 172.21.15.89:0/54359999 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91b8101de0 0x7f91b819a650 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f91a800e9f0 tx=0x7f91a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.216+0000 7f91afe00640 1 -- 172.21.15.89:0/54359999 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a800cd80 con 0x7f91b8101de0 2024-09-10T08:49:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.216+0000 7f91afe00640 1 -- 172.21.15.89:0/54359999 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91a8010910 con 0x7f91b8101de0 2024-09-10T08:49:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.216+0000 7f91bda00640 1 -- 172.21.15.89:0/54359999 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b819f8f0 con 0x7f91b8101de0 2024-09-10T08:49:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.216+0000 7f91afe00640 1 -- 172.21.15.89:0/54359999 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8018990 con 0x7f91b8101de0 2024-09-10T08:49:32.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.216+0000 7f91bda00640 1 -- 172.21.15.89:0/54359999 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b81a71d0 con 0x7f91b8101de0 2024-09-10T08:49:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.217+0000 7f91bda00640 1 -- 172.21.15.89:0/54359999 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b81021c0 con 0x7f91b8101de0 2024-09-10T08:49:32.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.219+0000 7f91afe00640 1 -- 172.21.15.89:0/54359999 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f91a8010ac0 con 0x7f91b8101de0 2024-09-10T08:49:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.220+0000 7f91afe00640 1 --2- 172.21.15.89:0/54359999 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f919c0a2c00 0x7f919c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.221+0000 7f91afe00640 1 -- 172.21.15.89:0/54359999 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f91a80cfd40 con 0x7f91b8101de0 2024-09-10T08:49:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.221+0000 7f91ae000640 1 --2- 172.21.15.89:0/54359999 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f919c0a2c00 0x7f919c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.221+0000 7f91ae000640 1 --2- 172.21.15.89:0/54359999 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f919c0a2c00 0x7f919c0a50c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f91a0000c00 tx=0x7f91a0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:32.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.223+0000 7f91afe00640 1 -- 172.21.15.89:0/54359999 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f91a809c870 con 0x7f91b8101de0 2024-09-10T08:49:32.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.397+0000 7f91bda00640 1 -- 172.21.15.89:0/54359999 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91b81a73d0 con 0x7f91b8101de0 2024-09-10T08:49:32.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.399+0000 7f91afe00640 1 -- 172.21.15.89:0/54359999 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24894 (secure 0 0 0) 0x7f91a809c170 con 0x7f91b8101de0 2024-09-10T08:49:32.401 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-10T08:49:32.401 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":96,"fsid":"91f747bc-6f4e-11ef-bcea-c7b262605968","created":"2024-09-10T08:28:49.697035+0000","modified":"2024-09-10T08:38:43.618497+0000","last_up_change":"2024-09-10T08:36:00.535433+0000","last_in_change":"2024-09-10T08:35:37.131131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:32:02.060714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:36:52.462301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:36:55.119708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:37:03.729002+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d3b55192-bd95-4497-89ea-dc017a25c0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6803","nonce":1760523284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6805","nonce":1760523284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6809","nonce":1760523284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1760523284},{"type":"v1","addr":"172.21.15.89:6807","nonce":1760523284}]},"public_addr":"172.21.15.89:6803/1760523284","cluster_addr":"172.21.15.89:6805/1760523284","heartbeat_back_addr":"172.21.15.89:6809/1760523284","heartbeat_front_addr":"172.21.15.89:6807/1760523284","state":["exists","up"]},{"osd":1,"uuid":"60aae117-cb52-4034-a700-11693171cd52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6801","nonce":2369291742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6803","nonce":2369291742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6807","nonce":2369291742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":2369291742},{"type":"v1","addr":"172.21.15.111:6805","nonce":2369291742}]},"public_addr":"172.21.15.111:6801/2369291742","cluster_addr":"172.21.15.111:6803/2369291742","heartbeat_back_addr":"172.21.15.111:6807/2369291742","heartbeat_front_addr":"172.21.15.111:6805/2369291742","state":["exists","up"]},{"osd":2,"uuid":"a305a019-0870-437c-a298-4f923ec54e77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6801","nonce":1092261395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6803","nonce":1092261395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6807","nonce":1092261395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1092261395},{"type":"v1","addr":"172.21.15.149:6805","nonce":1092261395}]},"public_addr":"172.21.15.149:6801/1092261395","cluster_addr":"172.21.15.149:6803/1092261395","heartbeat_back_addr":"172.21.15.149:6807/1092261395","heartbeat_front_addr":"172.21.15.149:6805/1092261395","state":["exists","up"]},{"osd":3,"uuid":"d02f54c1-f883-40f6-9341-8d9fc4b712ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6811","nonce":1405773950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6813","nonce":1405773950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6817","nonce":1405773950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1405773950},{"type":"v1","addr":"172.21.15.89:6815","nonce":1405773950}]},"public_addr":"172.21.15.89:6811/1405773950","cluster_addr":"172.21.15.89:6813/1405773950","heartbeat_back_addr":"172.21.15.89:6817/1405773950","heartbeat_front_addr":"172.21.15.89:6815/1405773950","state":["exists","up"]},{"osd":4,"uuid":"23cff46b-88dd-4344-be06-b35db6014f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6809","nonce":316717323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6811","nonce":316717323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6815","nonce":316717323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":316717323},{"type":"v1","addr":"172.21.15.111:6813","nonce":316717323}]},"public_addr":"172.21.15.111:6809/316717323","cluster_addr":"172.21.15.111:6811/316717323","heartbeat_back_addr":"172.21.15.111:6815/316717323","heartbeat_front_addr":"172.21.15.111:6813/316717323","state":["exists","up"]},{"osd":5,"uuid":"8e516e8d-e7c6-4ebc-80a5-29932161df4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6809","nonce":1256148461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6811","nonce":1256148461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6815","nonce":1256148461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1256148461},{"type":"v1","addr":"172.21.15.149:6813","nonce":1256148461}]},"public_addr":"172.21.15.149:6809/1256148461","cluster_addr":"172.21.15.149:6811/1256148461","heartbeat_back_addr":"172.21.15.149:6815/1256148461","heartbeat_front_addr":"172.21.15.149:6813/1256148461","state":["exists","up"]},{"osd":6,"uuid":"48f92472-f657-4c5a-adeb-20d4fea1ce8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6819","nonce":4190488024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6821","nonce":4190488024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6825","nonce":4190488024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4190488024},{"type":"v1","addr":"172.21.15.89:6823","nonce":4190488024}]},"public_addr":"172.21.15.89:6819/4190488024","cluster_addr":"172.21.15.89:6821/4190488024","heartbeat_back_addr":"172.21.15.89:6825/4190488024","heartbeat_front_addr":"172.21.15.89:6823/4190488024","state":["exists","up"]},{"osd":7,"uuid":"14a2a6f0-c4cd-406b-b637-7527fd32babe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6817","nonce":652666376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6819","nonce":652666376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6823","nonce":652666376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":652666376},{"type":"v1","addr":"172.21.15.111:6821","nonce":652666376}]},"public_addr":"172.21.15.111:6817/652666376","cluster_addr":"172.21.15.111:6819/652666376","heartbeat_back_addr":"172.21.15.111:6823/652666376","heartbeat_front_addr":"172.21.15.111:6821/652666376","state":["exists","up"]},{"osd":8,"uuid":"a54502e6-95f7-431f-8db1-54a6c7983b5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6817","nonce":2955172869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6819","nonce":2955172869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6823","nonce":2955172869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2955172869},{"type":"v1","addr":"172.21.15.149:6821","nonce":2955172869}]},"public_addr":"172.21.15.149:6817/2955172869","cluster_addr":"172.21.15.149:6819/2955172869","heartbeat_back_addr":"172.21.15.149:6823/2955172869","heartbeat_front_addr":"172.21.15.149:6821/2955172869","state":["exists","up"]},{"osd":9,"uuid":"5fe17344-5b1c-4601-a633-639cc1691478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6827","nonce":2427467040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6829","nonce":2427467040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6833","nonce":2427467040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2427467040},{"type":"v1","addr":"172.21.15.89:6831","nonce":2427467040}]},"public_addr":"172.21.15.89:6827/2427467040","cluster_addr":"172.21.15.89:6829/2427467040","heartbeat_back_addr":"172.21.15.89:6833/2427467040","heartbeat_front_addr":"172.21.15.89:6831/2427467040","state":["exists","up"]},{"osd":10,"uuid":"d8f29890-3064-4b67-b649-467a0ac86db7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6825","nonce":2328678744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6827","nonce":2328678744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6831","nonce":2328678744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":2328678744},{"type":"v1","addr":"172.21.15.111:6829","nonce":2328678744}]},"public_addr":"172.21.15.111:6825/2328678744","cluster_addr":"172.21.15.111:6827/2328678744","heartbeat_back_addr":"172.21.15.111:6831/2328678744","heartbeat_front_addr":"172.21.15.111:6829/2328678744","state":["exists","up"]},{"osd":11,"uuid":"92182c73-f139-47ca-a0ef-5046ae9b2310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6825","nonce":3130508827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6827","nonce":3130508827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6831","nonce":3130508827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3130508827},{"type":"v1","addr":"172.21.15.149:6829","nonce":3130508827}]},"public_addr":"172.21.15.149:6825/3130508827","cluster_addr":"172.21.15.149:6827/3130508827","heartbeat_back_addr":"172.21.15.149:6831/3130508827","heartbeat_front_addr":"172.21.15.149:6829/3130508827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:09.031810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:33.695124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:31:58.065510+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:27.592838+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:32:53.275289+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:18.305677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:33:47.781535+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:12.889512+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:34:37.744193+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:07.718408+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:33.391027+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:35:59.013250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.89:0/850669510":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/3541027428":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6801/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:6800/3794982914":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/2396468787":"2024-09-11T08:29:41.472002+0000","172.21.15.89:0/1609720980":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/995019144":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6801/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:6800/4223112778":"2024-09-11T08:29:14.554869+0000","172.21.15.89:0/2179977257":"2024-09-11T08:29:14.554869+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:49:32.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.401+0000 7f91bda00640 1 -- 172.21.15.89:0/54359999 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f919c0a2c00 msgr2=0x7f919c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.401+0000 7f91bda00640 1 --2- 172.21.15.89:0/54359999 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f919c0a2c00 0x7f919c0a50c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f91a0000c00 tx=0x7f91a0005b50 comp rx=0 tx=0).stop 2024-09-10T08:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.401+0000 7f91bda00640 1 -- 172.21.15.89:0/54359999 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91b8101de0 msgr2=0x7f91b819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.401+0000 7f91bda00640 1 --2- 172.21.15.89:0/54359999 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91b8101de0 0x7f91b819a650 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f91a800e9f0 tx=0x7f91a800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.401+0000 7f91bda00640 1 -- 172.21.15.89:0/54359999 shutdown_connections 2024-09-10T08:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.401+0000 7f91bda00640 1 --2- 172.21.15.89:0/54359999 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f919c0a2c00 0x7f919c0a50c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.401+0000 7f91bda00640 1 --2- 172.21.15.89:0/54359999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91b8106f10 0x7f91b819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.401+0000 7f91bda00640 1 --2- 172.21.15.89:0/54359999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91b8102790 0x7f91b819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.401+0000 7f91bda00640 1 --2- 172.21.15.89:0/54359999 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91b8101de0 0x7f91b819a650 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.401+0000 7f91bda00640 1 -- 172.21.15.89:0/54359999 >> 172.21.15.89:0/54359999 conn(0x7f91b80fd8c0 msgr2=0x7f91b80ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.401+0000 7f91bda00640 1 -- 172.21.15.89:0/54359999 shutdown_connections 2024-09-10T08:49:32.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:32.401+0000 7f91bda00640 1 -- 172.21.15.89:0/54359999 wait complete. 2024-09-10T08:49:33.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:32 smithi149 ceph-mon[36167]: pgmap v681: 129 pgs: 129 active+clean; 51 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T08:49:33.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:32 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/54359999' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:49:33.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:32 smithi111 ceph-mon[35220]: pgmap v681: 129 pgs: 129 active+clean; 51 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T08:49:33.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:32 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/54359999' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:49:33.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:32 smithi089 ceph-mon[30757]: pgmap v681: 129 pgs: 129 active+clean; 51 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T08:49:33.303 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:32 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/54359999' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:49:33.679 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-10T08:49:34.050 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:34.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.581+0000 7f3127e00640 1 -- 172.21.15.89:0/4130071562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3128103e30 msgr2=0x7f3128104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:34.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.581+0000 7f3127e00640 1 --2- 172.21.15.89:0/4130071562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3128103e30 0x7f3128104210 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3118009860 tx=0x7f311802a340 comp rx=0 tx=0).stop 2024-09-10T08:49:34.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.581+0000 7f3127e00640 1 -- 172.21.15.89:0/4130071562 shutdown_connections 2024-09-10T08:49:34.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.581+0000 7f3127e00640 1 --2- 172.21.15.89:0/4130071562 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3128108f60 0x7f3128110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.616 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.581+0000 7f3127e00640 1 --2- 172.21.15.89:0/4130071562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31281047e0 0x7f3128108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.581+0000 7f3127e00640 1 --2- 172.21.15.89:0/4130071562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3128103e30 0x7f3128104210 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.581+0000 7f3127e00640 1 -- 172.21.15.89:0/4130071562 >> 172.21.15.89:0/4130071562 conn(0x7f31280ff910 msgr2=0x7f3128101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:34.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.581+0000 7f3127e00640 1 -- 172.21.15.89:0/4130071562 shutdown_connections 2024-09-10T08:49:34.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.581+0000 7f3127e00640 1 -- 172.21.15.89:0/4130071562 wait complete. 2024-09-10T08:49:34.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.582+0000 7f3127e00640 1 Processor -- start 2024-09-10T08:49:34.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.582+0000 7f3127e00640 1 -- start start 2024-09-10T08:49:34.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.583+0000 7f3127e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3128103e30 0x7f312819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.583+0000 7f3127e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31281047e0 0x7f312819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.583+0000 7f3127e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3128108f60 0x7f312819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.583+0000 7f3127e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3128113fc0 con 0x7f31281047e0 2024-09-10T08:49:34.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.583+0000 7f3127e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3128113e40 con 0x7f3128103e30 2024-09-10T08:49:34.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.583+0000 7f3127e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3128114140 con 0x7f3128108f60 2024-09-10T08:49:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.583+0000 7f3126000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31281047e0 0x7f312819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.583+0000 7f3126000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31281047e0 0x7f312819a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57644/0 (socket says 172.21.15.89:57644) 2024-09-10T08:49:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.583+0000 7f3126000640 1 -- 172.21.15.89:0/2204903471 learned_addr learned my addr 172.21.15.89:0/2204903471 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:49:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.583+0000 7f3127400640 1 --2- 172.21.15.89:0/2204903471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3128108f60 0x7f312819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.583+0000 7f3126a00640 1 --2- 172.21.15.89:0/2204903471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3128103e30 0x7f312819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.583+0000 7f3126000640 1 -- 172.21.15.89:0/2204903471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3128108f60 msgr2=0x7f312819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.583+0000 7f3126000640 1 --2- 172.21.15.89:0/2204903471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3128108f60 0x7f312819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.583+0000 7f3126000640 1 -- 172.21.15.89:0/2204903471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3128103e30 msgr2=0x7f312819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.583+0000 7f3126000640 1 --2- 172.21.15.89:0/2204903471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3128103e30 0x7f312819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.583+0000 7f3126000640 1 -- 172.21.15.89:0/2204903471 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312819f430 con 0x7f31281047e0 2024-09-10T08:49:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.584+0000 7f3126000640 1 --2- 172.21.15.89:0/2204903471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31281047e0 0x7f312819a9c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f311400e9c0 tx=0x7f311400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.584+0000 7f311f400640 1 -- 172.21.15.89:0/2204903471 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311400cd80 con 0x7f31281047e0 2024-09-10T08:49:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.584+0000 7f311f400640 1 -- 172.21.15.89:0/2204903471 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31140035f0 con 0x7f31281047e0 2024-09-10T08:49:34.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.584+0000 7f3127e00640 1 -- 172.21.15.89:0/2204903471 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312819f720 con 0x7f31281047e0 2024-09-10T08:49:34.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.584+0000 7f311f400640 1 -- 172.21.15.89:0/2204903471 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114010670 con 0x7f31281047e0 2024-09-10T08:49:34.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.584+0000 7f3127e00640 1 -- 172.21.15.89:0/2204903471 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31281a7000 con 0x7f31281047e0 2024-09-10T08:49:34.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.585+0000 7f3127e00640 1 -- 172.21.15.89:0/2204903471 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3128105310 con 0x7f31281047e0 2024-09-10T08:49:34.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.587+0000 7f311f400640 1 -- 172.21.15.89:0/2204903471 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f31140037a0 con 0x7f31281047e0 2024-09-10T08:49:34.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.589+0000 7f311f400640 1 --2- 172.21.15.89:0/2204903471 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f310c0a2b80 0x7f310c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.589+0000 7f311f400640 1 -- 172.21.15.89:0/2204903471 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f31140cf690 con 0x7f31281047e0 2024-09-10T08:49:34.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.589+0000 7f3126a00640 1 --2- 172.21.15.89:0/2204903471 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f310c0a2b80 0x7f310c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:34.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.590+0000 7f3126a00640 1 --2- 172.21.15.89:0/2204903471 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f310c0a2b80 0x7f310c0a5040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f311802a930 tx=0x7f31180058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:34.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.593+0000 7f311f400640 1 -- 172.21.15.89:0/2204903471 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f311409c1c0 con 0x7f31281047e0 2024-09-10T08:49:34.805 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.803+0000 7f3127e00640 1 -- 172.21.15.89:0/2204903471 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f312810bbb0 con 0x7f31281047e0 2024-09-10T08:49:34.807 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.806+0000 7f311f400640 1 -- 172.21.15.89:0/2204903471 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) ==== 161+0+0 (secure 0 0 0) 0x7f311409bac0 con 0x7f31281047e0 2024-09-10T08:49:34.807 INFO:teuthology.orchestra.run.smithi089.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T08:49:34.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.809+0000 7f3127e00640 1 -- 172.21.15.89:0/2204903471 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f310c0a2b80 msgr2=0x7f310c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:34.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.809+0000 7f3127e00640 1 --2- 172.21.15.89:0/2204903471 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f310c0a2b80 0x7f310c0a5040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f311802a930 tx=0x7f31180058d0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.809+0000 7f3127e00640 1 -- 172.21.15.89:0/2204903471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31281047e0 msgr2=0x7f312819a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:34.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.809+0000 7f3127e00640 1 --2- 172.21.15.89:0/2204903471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31281047e0 0x7f312819a9c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f311400e9c0 tx=0x7f311400edc0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.809+0000 7f3127e00640 1 -- 172.21.15.89:0/2204903471 shutdown_connections 2024-09-10T08:49:34.812 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.809+0000 7f3127e00640 1 --2- 172.21.15.89:0/2204903471 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f310c0a2b80 0x7f310c0a5040 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.809+0000 7f3127e00640 1 --2- 172.21.15.89:0/2204903471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3128108f60 0x7f312819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.809+0000 7f3127e00640 1 --2- 172.21.15.89:0/2204903471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31281047e0 0x7f312819a9c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.809+0000 7f3127e00640 1 --2- 172.21.15.89:0/2204903471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3128103e30 0x7f312819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.809+0000 7f3127e00640 1 -- 172.21.15.89:0/2204903471 >> 172.21.15.89:0/2204903471 conn(0x7f31280ff910 msgr2=0x7f3128100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:34.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.810+0000 7f3127e00640 1 -- 172.21.15.89:0/2204903471 shutdown_connections 2024-09-10T08:49:34.813 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:34.810+0000 7f3127e00640 1 -- 172.21.15.89:0/2204903471 wait complete. 2024-09-10T08:49:35.176 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:34 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-a[30753]: 2024-09-10T08:49:34.804+0000 7f454ec00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T08:49:35.176 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:34 smithi089 ceph-mon[30757]: pgmap v682: 129 pgs: 129 active+clean; 51 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T08:49:35.176 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:34 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2204903471' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T08:49:35.177 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:34 smithi089 ceph-mon[30757]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T08:49:35.177 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:34 smithi089 ceph-mon[30757]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T08:49:35.177 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:34 smithi089 ceph-mon[30757]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-09-10T08:49:35.177 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:34 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/2204903471' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T08:49:35.177 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:34 smithi089 ceph-mon[30757]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T08:49:35.177 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:34 smithi089 ceph-mon[30757]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-10T08:49:35.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:34 smithi149 ceph-mon[36167]: pgmap v682: 129 pgs: 129 active+clean; 51 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T08:49:35.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:34 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2204903471' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T08:49:35.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:34 smithi149 ceph-mon[36167]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T08:49:35.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:34 smithi149 ceph-mon[36167]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T08:49:35.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:34 smithi149 ceph-mon[36167]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-09-10T08:49:35.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:34 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/2204903471' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T08:49:35.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:34 smithi149 ceph-mon[36167]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T08:49:35.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:34 smithi149 ceph-mon[36167]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-10T08:49:35.222 INFO:journalctl@ceph.mds.i.smithi149.stdout:Sep 10 08:49:34 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-i[66738]: ignoring --setuser ceph since I am not root 2024-09-10T08:49:35.222 INFO:journalctl@ceph.mds.i.smithi149.stdout:Sep 10 08:49:34 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-i[66738]: ignoring --setgroup ceph since I am not root 2024-09-10T08:49:35.222 INFO:journalctl@ceph.mds.i.smithi149.stdout:Sep 10 08:49:34 smithi149 ceph-mds[66742]: -- 172.21.15.149:0/2490256442 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564058865860 con 0x564058842800 2024-09-10T08:49:35.222 INFO:journalctl@ceph.mds.i.smithi149.stdout:Sep 10 08:49:34 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-i[66738]: starting mds.i at 2024-09-10T08:49:35.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:Sep 10 08:49:34 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-c[66435]: ignoring --setuser ceph since I am not root 2024-09-10T08:49:35.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:Sep 10 08:49:34 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-c[66435]: ignoring --setgroup ceph since I am not root 2024-09-10T08:49:35.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:Sep 10 08:49:34 smithi149 ceph-mds[66439]: -- 172.21.15.149:0/722897567 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56417b8f1860 con 0x56417b8ce800 2024-09-10T08:49:35.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:Sep 10 08:49:34 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-c[66435]: starting mds.c at 2024-09-10T08:49:35.223 INFO:journalctl@ceph.mds.f.smithi149.stdout:Sep 10 08:49:34 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-f[66131]: ignoring --setuser ceph since I am not root 2024-09-10T08:49:35.223 INFO:journalctl@ceph.mds.f.smithi149.stdout:Sep 10 08:49:34 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-f[66131]: ignoring --setgroup ceph since I am not root 2024-09-10T08:49:35.223 INFO:journalctl@ceph.mds.f.smithi149.stdout:Sep 10 08:49:34 smithi149 ceph-mds[66135]: -- 172.21.15.149:0/958879505 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eef2cdd860 con 0x55eef2cbb400 2024-09-10T08:49:35.223 INFO:journalctl@ceph.mds.f.smithi149.stdout:Sep 10 08:49:34 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-f[66131]: starting mds.f at 2024-09-10T08:49:35.294 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h[64759]: ignoring --setuser ceph since I am not root 2024-09-10T08:49:35.294 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h[64759]: ignoring --setgroup ceph since I am not root 2024-09-10T08:49:35.294 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-mds[64763]: -- 172.21.15.111:0/1790085353 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ff44ccc680 con 0x55ff44ca1000 2024-09-10T08:49:35.294 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-mds[64763]: -- 172.21.15.111:0/1790085353 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ff44cc3860 con 0x55ff44ca1000 2024-09-10T08:49:35.294 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h[64759]: starting mds.h at 2024-09-10T08:49:35.294 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-b[65063]: ignoring --setuser ceph since I am not root 2024-09-10T08:49:35.295 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-b[65063]: ignoring --setgroup ceph since I am not root 2024-09-10T08:49:35.295 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-mds[65067]: -- 172.21.15.111:0/3161409209 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e05c85c20 con 0x563e05c62800 2024-09-10T08:49:35.295 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-b[65063]: starting mds.b at 2024-09-10T08:49:35.295 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-e[64459]: ignoring --setuser ceph since I am not root 2024-09-10T08:49:35.295 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-e[64459]: ignoring --setgroup ceph since I am not root 2024-09-10T08:49:35.295 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-mds[64463]: -- 172.21.15.111:0/3887988399 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d218027860 con 0x55d218005000 2024-09-10T08:49:35.295 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-e[64459]: starting mds.e at 2024-09-10T08:49:35.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-mon[35220]: pgmap v682: 129 pgs: 129 active+clean; 51 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-10T08:49:35.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2204903471' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T08:49:35.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-mon[35220]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T08:49:35.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-mon[35220]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T08:49:35.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-mon[35220]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-09-10T08:49:35.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/2204903471' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T08:49:35.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-mon[35220]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T08:49:35.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:34 smithi111 ceph-mon[35220]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-10T08:49:35.679 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T08:49:36.051 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:36.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.628+0000 7fa1f8600640 1 -- 172.21.15.89:0/784884362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1f0112a30 msgr2=0x7fa1f0114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:36.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.628+0000 7fa1f8600640 1 --2- 172.21.15.89:0/784884362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1f0112a30 0x7fa1f0114e20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009ca0 tx=0x7fa1e802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.628+0000 7fa1f8600640 1 -- 172.21.15.89:0/784884362 shutdown_connections 2024-09-10T08:49:36.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.628+0000 7fa1f8600640 1 --2- 172.21.15.89:0/784884362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1f0112a30 0x7fa1f0114e20 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.628+0000 7fa1f8600640 1 --2- 172.21.15.89:0/784884362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1f0077040 0x7fa1f01124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.628+0000 7fa1f8600640 1 --2- 172.21.15.89:0/784884362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1f0076720 0x7fa1f0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.630 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.628+0000 7fa1f8600640 1 -- 172.21.15.89:0/784884362 >> 172.21.15.89:0/784884362 conn(0x7fa1f00ffa90 msgr2=0x7fa1f0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:36.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.628+0000 7fa1f8600640 1 -- 172.21.15.89:0/784884362 shutdown_connections 2024-09-10T08:49:36.631 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.629+0000 7fa1f8600640 1 -- 172.21.15.89:0/784884362 wait complete. 2024-09-10T08:49:36.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.629+0000 7fa1f8600640 1 Processor -- start 2024-09-10T08:49:36.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.630+0000 7fa1f8600640 1 -- start start 2024-09-10T08:49:36.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.630+0000 7fa1f8600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1f0076720 0x7fa1f01a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:36.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.630+0000 7fa1f8600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1f0077040 0x7fa1f01a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:36.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.630+0000 7fa1f8600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1f0112a30 0x7fa1f019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:36.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.630+0000 7fa1f8600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f0118570 con 0x7fa1f0077040 2024-09-10T08:49:36.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.630+0000 7fa1f8600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f01183f0 con 0x7fa1f0076720 2024-09-10T08:49:36.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.630+0000 7fa1f8600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f01186f0 con 0x7fa1f0112a30 2024-09-10T08:49:36.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.630+0000 7fa1f5200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1f0077040 0x7fa1f01a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:36.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.630+0000 7fa1f5200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1f0077040 0x7fa1f01a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57668/0 (socket says 172.21.15.89:57668) 2024-09-10T08:49:36.632 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.630+0000 7fa1f5c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1f0076720 0x7fa1f01a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:36.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.630+0000 7fa1f5200640 1 -- 172.21.15.89:0/1934395850 learned_addr learned my addr 172.21.15.89:0/1934395850 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:49:36.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.630+0000 7fa1f6600640 1 --2- 172.21.15.89:0/1934395850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1f0112a30 0x7fa1f019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:36.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.631+0000 7fa1f5c00640 1 -- 172.21.15.89:0/1934395850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1f0112a30 msgr2=0x7fa1f019e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:36.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.631+0000 7fa1f5c00640 1 --2- 172.21.15.89:0/1934395850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1f0112a30 0x7fa1f019e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.631+0000 7fa1f5c00640 1 -- 172.21.15.89:0/1934395850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1f0077040 msgr2=0x7fa1f01a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:36.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.631+0000 7fa1f5c00640 1 --2- 172.21.15.89:0/1934395850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1f0077040 0x7fa1f01a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.631+0000 7fa1f5c00640 1 -- 172.21.15.89:0/1934395850 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f019efc0 con 0x7fa1f0076720 2024-09-10T08:49:36.633 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.631+0000 7fa1f5200640 1 --2- 172.21.15.89:0/1934395850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1f0077040 0x7fa1f01a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:49:36.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.631+0000 7fa1f5c00640 1 --2- 172.21.15.89:0/1934395850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1f0076720 0x7fa1f01a46a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400d6d0 tx=0x7fa1e400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:36.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.631+0000 7fa1ee000640 1 -- 172.21.15.89:0/1934395850 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4017070 con 0x7fa1f0076720 2024-09-10T08:49:36.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.631+0000 7fa1f8600640 1 -- 172.21.15.89:0/1934395850 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f019f2b0 con 0x7fa1f0076720 2024-09-10T08:49:36.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.631+0000 7fa1ee000640 1 -- 172.21.15.89:0/1934395850 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e4009820 con 0x7fa1f0076720 2024-09-10T08:49:36.634 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.631+0000 7fa1ee000640 1 -- 172.21.15.89:0/1934395850 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4004cd0 con 0x7fa1f0076720 2024-09-10T08:49:36.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.632+0000 7fa1f8600640 1 -- 172.21.15.89:0/1934395850 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f01ab500 con 0x7fa1f0076720 2024-09-10T08:49:36.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.632+0000 7fa1f8600640 1 -- 172.21.15.89:0/1934395850 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f0077e60 con 0x7fa1f0076720 2024-09-10T08:49:36.636 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.635+0000 7fa1ee000640 1 -- 172.21.15.89:0/1934395850 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa1e4005430 con 0x7fa1f0076720 2024-09-10T08:49:36.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.636+0000 7fa1ee000640 1 --2- 172.21.15.89:0/1934395850 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa1d80a2b80 0x7fa1d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:36.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.636+0000 7fa1ee000640 1 -- 172.21.15.89:0/1934395850 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7fa1e40cf3e0 con 0x7fa1f0076720 2024-09-10T08:49:36.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.636+0000 7fa1f5200640 1 --2- 172.21.15.89:0/1934395850 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa1d80a2b80 0x7fa1d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:36.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.637+0000 7fa1f5200640 1 --2- 172.21.15.89:0/1934395850 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa1d80a2b80 0x7fa1d80a5040 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0079290 tx=0x7fa1dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:36.639 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.638+0000 7fa1ee000640 1 -- 172.21.15.89:0/1934395850 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa1e409b7c0 con 0x7fa1f0076720 2024-09-10T08:49:36.851 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:36.850+0000 7fa1f8600640 1 -- 172.21.15.89:0/1934395850 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa1f0076b00 con 0x7fa1f0076720 2024-09-10T08:49:37.074 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:36 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.149:6834/3590668135,v1:172.21.15.149:6835/3590668135] up:boot 2024-09-10T08:49:37.074 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:36 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.149:6836/3918006247,v1:172.21.15.149:6838/3918006247] up:boot 2024-09-10T08:49:37.074 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:36 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.149:6837/534753526,v1:172.21.15.149:6839/534753526] up:boot 2024-09-10T08:49:37.074 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:36 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.111:6834/3447329432,v1:172.21.15.111:6835/3447329432] up:boot 2024-09-10T08:49:37.074 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:36 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.111:6836/760470285,v1:172.21.15.111:6838/760470285] up:boot 2024-09-10T08:49:37.075 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:36 smithi149 ceph-mon[36167]: mds.? [v2:172.21.15.111:6837/557278496,v1:172.21.15.111:6839/557278496] up:boot 2024-09-10T08:49:37.075 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:36 smithi149 ceph-mon[36167]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-10T08:49:37.075 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T08:49:37.075 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T08:49:37.075 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T08:49:37.075 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T08:49:37.075 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T08:49:37.075 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:36 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T08:49:37.075 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:36 smithi149 ceph-mon[36167]: pgmap v684: 129 pgs: 129 active+clean; 51 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 818 B/s wr, 7 op/s 2024-09-10T08:49:37.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:36 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.149:6834/3590668135,v1:172.21.15.149:6835/3590668135] up:boot 2024-09-10T08:49:37.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:36 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.149:6836/3918006247,v1:172.21.15.149:6838/3918006247] up:boot 2024-09-10T08:49:37.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:36 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.149:6837/534753526,v1:172.21.15.149:6839/534753526] up:boot 2024-09-10T08:49:37.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:36 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.111:6834/3447329432,v1:172.21.15.111:6835/3447329432] up:boot 2024-09-10T08:49:37.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:36 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.111:6836/760470285,v1:172.21.15.111:6838/760470285] up:boot 2024-09-10T08:49:37.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:36 smithi111 ceph-mon[35220]: mds.? [v2:172.21.15.111:6837/557278496,v1:172.21.15.111:6839/557278496] up:boot 2024-09-10T08:49:37.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:36 smithi111 ceph-mon[35220]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-10T08:49:37.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T08:49:37.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T08:49:37.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T08:49:37.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T08:49:37.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T08:49:37.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:36 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T08:49:37.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:36 smithi111 ceph-mon[35220]: pgmap v684: 129 pgs: 129 active+clean; 51 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 818 B/s wr, 7 op/s 2024-09-10T08:49:37.304 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:36 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.149:6834/3590668135,v1:172.21.15.149:6835/3590668135] up:boot 2024-09-10T08:49:37.304 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:36 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.149:6836/3918006247,v1:172.21.15.149:6838/3918006247] up:boot 2024-09-10T08:49:37.304 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:36 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.149:6837/534753526,v1:172.21.15.149:6839/534753526] up:boot 2024-09-10T08:49:37.304 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:36 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.111:6834/3447329432,v1:172.21.15.111:6835/3447329432] up:boot 2024-09-10T08:49:37.304 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:36 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.111:6836/760470285,v1:172.21.15.111:6838/760470285] up:boot 2024-09-10T08:49:37.304 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:36 smithi089 ceph-mon[30757]: mds.? [v2:172.21.15.111:6837/557278496,v1:172.21.15.111:6839/557278496] up:boot 2024-09-10T08:49:37.305 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:36 smithi089 ceph-mon[30757]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-10T08:49:37.305 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T08:49:37.305 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T08:49:37.305 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T08:49:37.305 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T08:49:37.305 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T08:49:37.305 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:36 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T08:49:37.305 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:36 smithi089 ceph-mon[30757]: pgmap v684: 129 pgs: 129 active+clean; 51 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 818 B/s wr, 7 op/s 2024-09-10T08:49:37.823 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:37.822+0000 7fa1ee000640 1 -- 172.21.15.89:0/1934395850 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) ==== 104+0+0 (secure 0 0 0) 0x7fa1e409fac0 con 0x7fa1f0076720 2024-09-10T08:49:37.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:37.824+0000 7fa1f8600640 1 -- 172.21.15.89:0/1934395850 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa1d80a2b80 msgr2=0x7fa1d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:37.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:37.824+0000 7fa1f8600640 1 --2- 172.21.15.89:0/1934395850 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa1d80a2b80 0x7fa1d80a5040 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0079290 tx=0x7fa1dc005f90 comp rx=0 tx=0).stop 2024-09-10T08:49:37.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:37.824+0000 7fa1f8600640 1 -- 172.21.15.89:0/1934395850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1f0076720 msgr2=0x7fa1f01a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:37.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:37.824+0000 7fa1f8600640 1 --2- 172.21.15.89:0/1934395850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1f0076720 0x7fa1f01a46a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400d6d0 tx=0x7fa1e400db90 comp rx=0 tx=0).stop 2024-09-10T08:49:37.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:37.824+0000 7fa1f8600640 1 -- 172.21.15.89:0/1934395850 shutdown_connections 2024-09-10T08:49:37.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:37.824+0000 7fa1f8600640 1 --2- 172.21.15.89:0/1934395850 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fa1d80a2b80 0x7fa1d80a5040 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:37.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:37.824+0000 7fa1f8600640 1 --2- 172.21.15.89:0/1934395850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1f0112a30 0x7fa1f019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:37.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:37.824+0000 7fa1f8600640 1 --2- 172.21.15.89:0/1934395850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa1f0077040 0x7fa1f01a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:37.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:37.824+0000 7fa1f8600640 1 --2- 172.21.15.89:0/1934395850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1f0076720 0x7fa1f01a46a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:37.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:37.824+0000 7fa1f8600640 1 -- 172.21.15.89:0/1934395850 >> 172.21.15.89:0/1934395850 conn(0x7fa1f00ffa90 msgr2=0x7fa1f0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:37.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:37.825+0000 7fa1f8600640 1 -- 172.21.15.89:0/1934395850 shutdown_connections 2024-09-10T08:49:37.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:37.825+0000 7fa1f8600640 1 -- 172.21.15.89:0/1934395850 wait complete. 2024-09-10T08:49:38.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:37 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1934395850' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:49:38.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:37 smithi149 ceph-mon[36167]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:49:38.231 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:37 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1934395850' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:49:38.232 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:37 smithi089 ceph-mon[30757]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:49:38.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:37 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1934395850' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:49:38.294 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:37 smithi111 ceph-mon[35220]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:49:38.746 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T08:49:39.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:38 smithi089 ceph-mon[30757]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T08:49:39.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:38 smithi089 ceph-mon[30757]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T08:49:39.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:38 smithi089 ceph-mon[30757]: Cluster is now healthy 2024-09-10T08:49:39.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:38 smithi089 ceph-mon[30757]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T08:49:39.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:38 smithi089 ceph-mon[30757]: fsmap 12 up:standby 2024-09-10T08:49:39.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:38 smithi089 ceph-mon[30757]: pgmap v685: 129 pgs: 129 active+clean; 51 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 614 B/s wr, 6 op/s 2024-09-10T08:49:39.054 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:38 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:49:39.220 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:38 smithi149 ceph-mon[36167]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T08:49:39.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:38 smithi149 ceph-mon[36167]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T08:49:39.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:38 smithi149 ceph-mon[36167]: Cluster is now healthy 2024-09-10T08:49:39.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:38 smithi149 ceph-mon[36167]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T08:49:39.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:38 smithi149 ceph-mon[36167]: fsmap 12 up:standby 2024-09-10T08:49:39.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:38 smithi149 ceph-mon[36167]: pgmap v685: 129 pgs: 129 active+clean; 51 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 614 B/s wr, 6 op/s 2024-09-10T08:49:39.221 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:38 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:49:39.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:38 smithi111 ceph-mon[35220]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T08:49:39.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:38 smithi111 ceph-mon[35220]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T08:49:39.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:38 smithi111 ceph-mon[35220]: Cluster is now healthy 2024-09-10T08:49:39.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:38 smithi111 ceph-mon[35220]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T08:49:39.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:38 smithi111 ceph-mon[35220]: fsmap 12 up:standby 2024-09-10T08:49:39.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:38 smithi111 ceph-mon[35220]: pgmap v685: 129 pgs: 129 active+clean; 51 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 614 B/s wr, 6 op/s 2024-09-10T08:49:39.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:38 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:49:39.892 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:40.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.489+0000 7ff3b0a00640 1 -- 172.21.15.89:0/1239163436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3ac112870 msgr2=0x7ff3ac114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:40.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.489+0000 7ff3b0a00640 1 --2- 172.21.15.89:0/1239163436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3ac112870 0x7ff3ac114c60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009ca0 tx=0x7ff3a402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:49:40.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.490+0000 7ff3b0a00640 1 -- 172.21.15.89:0/1239163436 shutdown_connections 2024-09-10T08:49:40.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.490+0000 7ff3b0a00640 1 --2- 172.21.15.89:0/1239163436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3ac112870 0x7ff3ac114c60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:40.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.490+0000 7ff3b0a00640 1 --2- 172.21.15.89:0/1239163436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3ac077040 0x7ff3ac112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:40.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.490+0000 7ff3b0a00640 1 --2- 172.21.15.89:0/1239163436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ac076720 0x7ff3ac076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:40.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.490+0000 7ff3b0a00640 1 -- 172.21.15.89:0/1239163436 >> 172.21.15.89:0/1239163436 conn(0x7ff3ac0ff8d0 msgr2=0x7ff3ac101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:40.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.490+0000 7ff3b0a00640 1 -- 172.21.15.89:0/1239163436 shutdown_connections 2024-09-10T08:49:40.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.490+0000 7ff3b0a00640 1 -- 172.21.15.89:0/1239163436 wait complete. 2024-09-10T08:49:40.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.491+0000 7ff3b0a00640 1 Processor -- start 2024-09-10T08:49:40.495 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.491+0000 7ff3b0a00640 1 -- start start 2024-09-10T08:49:40.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.491+0000 7ff3b0a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3ac076720 0x7ff3ac1a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:40.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.491+0000 7ff3b0a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3ac077040 0x7ff3ac1a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:40.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.491+0000 7ff3b0a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ac112870 0x7ff3ac19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:40.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.492+0000 7ff3b0a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1183b0 con 0x7ff3ac112870 2024-09-10T08:49:40.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.492+0000 7ff3b0a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac118230 con 0x7ff3ac077040 2024-09-10T08:49:40.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.492+0000 7ff3b0a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac118530 con 0x7ff3ac076720 2024-09-10T08:49:40.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.492+0000 7ff3abe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ac112870 0x7ff3ac19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:40.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.492+0000 7ff3ab400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3ac076720 0x7ff3ac1a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:40.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.492+0000 7ff3ab400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3ac076720 0x7ff3ac1a45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.89:47910/0 (socket says 172.21.15.89:47910) 2024-09-10T08:49:40.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.492+0000 7ff3abe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ac112870 0x7ff3ac19e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:57686/0 (socket says 172.21.15.89:57686) 2024-09-10T08:49:40.496 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.492+0000 7ff3ab400640 1 -- 172.21.15.89:0/3244067261 learned_addr learned my addr 172.21.15.89:0/3244067261 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:49:40.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.492+0000 7ff3a2a00640 1 --2- 172.21.15.89:0/3244067261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3ac077040 0x7ff3ac1a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:40.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.492+0000 7ff3a2a00640 1 -- 172.21.15.89:0/3244067261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3ac076720 msgr2=0x7ff3ac1a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:40.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.492+0000 7ff3a2a00640 1 --2- 172.21.15.89:0/3244067261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3ac076720 0x7ff3ac1a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:40.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.492+0000 7ff3a2a00640 1 -- 172.21.15.89:0/3244067261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ac112870 msgr2=0x7ff3ac19e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:40.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.492+0000 7ff3a2a00640 1 --2- 172.21.15.89:0/3244067261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ac112870 0x7ff3ac19e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:40.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.492+0000 7ff3a2a00640 1 -- 172.21.15.89:0/3244067261 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ac19ef30 con 0x7ff3ac077040 2024-09-10T08:49:40.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.492+0000 7ff3ab400640 1 --2- 172.21.15.89:0/3244067261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3ac076720 0x7ff3ac1a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:49:40.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.493+0000 7ff3a2a00640 1 --2- 172.21.15.89:0/3244067261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3ac077040 0x7ff3ac1a4b30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff394005fd0 tx=0x7ff39400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:40.497 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.493+0000 7ff3a8c00640 1 -- 172.21.15.89:0/3244067261 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff394009c30 con 0x7ff3ac077040 2024-09-10T08:49:40.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.493+0000 7ff3a8c00640 1 -- 172.21.15.89:0/3244067261 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff394009dd0 con 0x7ff3ac077040 2024-09-10T08:49:40.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.493+0000 7ff3a8c00640 1 -- 172.21.15.89:0/3244067261 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff394010460 con 0x7ff3ac077040 2024-09-10T08:49:40.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.493+0000 7ff3b0a00640 1 -- 172.21.15.89:0/3244067261 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3ac19f220 con 0x7ff3ac077040 2024-09-10T08:49:40.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.493+0000 7ff3b0a00640 1 -- 172.21.15.89:0/3244067261 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3ac1ab3a0 con 0x7ff3ac077040 2024-09-10T08:49:40.498 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.494+0000 7ff3b0a00640 1 -- 172.21.15.89:0/3244067261 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3ac077e60 con 0x7ff3ac077040 2024-09-10T08:49:40.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.496+0000 7ff3a8c00640 1 -- 172.21.15.89:0/3244067261 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff394003f50 con 0x7ff3ac077040 2024-09-10T08:49:40.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.498+0000 7ff3a8c00640 1 --2- 172.21.15.89:0/3244067261 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3900a2c00 0x7ff3900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:40.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.498+0000 7ff3a8c00640 1 -- 172.21.15.89:0/3244067261 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7ff3940cf360 con 0x7ff3ac077040 2024-09-10T08:49:40.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.498+0000 7ff3ab400640 1 --2- 172.21.15.89:0/3244067261 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3900a2c00 0x7ff3900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:40.499 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.499+0000 7ff3ab400640 1 --2- 172.21.15.89:0/3244067261 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3900a2c00 0x7ff3900a50c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0095f0 tx=0x7ff39c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:40.501 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.501+0000 7ff3a8c00640 1 -- 172.21.15.89:0/3244067261 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff394017020 con 0x7ff3ac077040 2024-09-10T08:49:40.707 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:40.707+0000 7ff3b0a00640 1 -- 172.21.15.89:0/3244067261 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff3ac1ab5a0 con 0x7ff3ac077040 2024-09-10T08:49:40.708 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:40 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:40.708 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:40 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:40.708 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:40 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:40.708 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:40 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:40.708 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:40 smithi089 ceph-mon[30757]: pgmap v686: 129 pgs: 129 active+clean; 51 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-10T08:49:40.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:40 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:40.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:40 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:40.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:40 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:40.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:40 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:40.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:40 smithi149 ceph-mon[36167]: pgmap v686: 129 pgs: 129 active+clean; 51 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-10T08:49:40.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:40 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:40.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:40 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:40.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:40 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:40.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:40 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:40.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:40 smithi111 ceph-mon[35220]: pgmap v686: 129 pgs: 129 active+clean; 51 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-10T08:49:41.434 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:41.433+0000 7ff3a8c00640 1 -- 172.21.15.89:0/3244067261 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) ==== 181+0+0 (secure 0 0 0) 0x7ff39409b740 con 0x7ff3ac077040 2024-09-10T08:49:41.434 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_metadata' removed 2024-09-10T08:49:41.437 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:41.437+0000 7ff3b0a00640 1 -- 172.21.15.89:0/3244067261 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3900a2c00 msgr2=0x7ff3900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:41.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:41.437+0000 7ff3b0a00640 1 --2- 172.21.15.89:0/3244067261 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3900a2c00 0x7ff3900a50c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0095f0 tx=0x7ff39c005f90 comp rx=0 tx=0).stop 2024-09-10T08:49:41.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:41.437+0000 7ff3b0a00640 1 -- 172.21.15.89:0/3244067261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3ac077040 msgr2=0x7ff3ac1a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:41.438 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:41.437+0000 7ff3b0a00640 1 --2- 172.21.15.89:0/3244067261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3ac077040 0x7ff3ac1a4b30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff394005fd0 tx=0x7ff39400dde0 comp rx=0 tx=0).stop 2024-09-10T08:49:41.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:41.438+0000 7ff3b0a00640 1 -- 172.21.15.89:0/3244067261 shutdown_connections 2024-09-10T08:49:41.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:41.439+0000 7ff3b0a00640 1 --2- 172.21.15.89:0/3244067261 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7ff3900a2c00 0x7ff3900a50c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:41.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:41.439+0000 7ff3b0a00640 1 --2- 172.21.15.89:0/3244067261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ac112870 0x7ff3ac19e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:41.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:41.439+0000 7ff3b0a00640 1 --2- 172.21.15.89:0/3244067261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3ac077040 0x7ff3ac1a4b30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:41.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:41.439+0000 7ff3b0a00640 1 --2- 172.21.15.89:0/3244067261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff3ac076720 0x7ff3ac1a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:41.440 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:41.439+0000 7ff3b0a00640 1 -- 172.21.15.89:0/3244067261 >> 172.21.15.89:0/3244067261 conn(0x7ff3ac0ff8d0 msgr2=0x7ff3ac101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:41.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:41.440+0000 7ff3b0a00640 1 -- 172.21.15.89:0/3244067261 shutdown_connections 2024-09-10T08:49:41.441 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:41.440+0000 7ff3b0a00640 1 -- 172.21.15.89:0/3244067261 wait complete. 2024-09-10T08:49:41.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:41 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3244067261' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:49:41.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:41 smithi149 ceph-mon[36167]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:49:41.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:41 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:41.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:41 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:41.780 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:41 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3244067261' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:49:41.780 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:41 smithi089 ceph-mon[30757]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:49:41.780 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:41.780 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:41 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:41.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:41 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3244067261' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:49:41.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:41 smithi111 ceph-mon[35220]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:49:41.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:41 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:41.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:41 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:42.310 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T08:49:42.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:42 smithi089 ceph-mon[30757]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T08:49:42.554 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:42 smithi089 ceph-mon[30757]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T08:49:42.554 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:49:42.554 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:49:42.554 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:42 smithi089 ceph-mon[30757]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:42.554 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:42 smithi089 ceph-mon[30757]: pgmap v688: 97 pgs: 97 active+clean; 577 KiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:49:42.680 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:42.720 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:42 smithi149 ceph-mon[36167]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T08:49:42.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:42 smithi149 ceph-mon[36167]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T08:49:42.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:49:42.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:49:42.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:42 smithi149 ceph-mon[36167]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:42.721 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:42 smithi149 ceph-mon[36167]: pgmap v688: 97 pgs: 97 active+clean; 577 KiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:49:42.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:42 smithi111 ceph-mon[35220]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T08:49:42.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:42 smithi111 ceph-mon[35220]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T08:49:42.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:49:42.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:49:42.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:42 smithi111 ceph-mon[35220]: from='mgr.14150 172.21.15.89:0/1806428324' entity='mgr.x' 2024-09-10T08:49:42.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:42 smithi111 ceph-mon[35220]: pgmap v688: 97 pgs: 97 active+clean; 577 KiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:49:43.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.272+0000 7f0da7800640 1 -- 172.21.15.89:0/3948382309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da010c170 msgr2=0x7f0da010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:43.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.272+0000 7f0da7800640 1 --2- 172.21.15.89:0/3948382309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da010c170 0x7f0da010e610 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009ca0 tx=0x7f0d9802a950 comp rx=0 tx=0).stop 2024-09-10T08:49:43.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.273+0000 7f0da7800640 1 -- 172.21.15.89:0/3948382309 shutdown_connections 2024-09-10T08:49:43.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.273+0000 7f0da7800640 1 --2- 172.21.15.89:0/3948382309 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da010c170 0x7f0da010e610 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:43.273 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.273+0000 7f0da7800640 1 --2- 172.21.15.89:0/3948382309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0da0103660 0x7f0da010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:43.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.273+0000 7f0da7800640 1 --2- 172.21.15.89:0/3948382309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0da0102d40 0x7f0da0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:43.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.273+0000 7f0da7800640 1 -- 172.21.15.89:0/3948382309 >> 172.21.15.89:0/3948382309 conn(0x7f0da00fc7a0 msgr2=0x7f0da00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:43.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.273+0000 7f0da7800640 1 -- 172.21.15.89:0/3948382309 shutdown_connections 2024-09-10T08:49:43.274 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.273+0000 7f0da7800640 1 -- 172.21.15.89:0/3948382309 wait complete. 2024-09-10T08:49:43.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.274+0000 7f0da7800640 1 Processor -- start 2024-09-10T08:49:43.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.274+0000 7f0da7800640 1 -- start start 2024-09-10T08:49:43.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.274+0000 7f0da7800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0da0102d40 0x7f0da01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:43.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.274+0000 7f0da7800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da0103660 0x7f0da0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:43.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.274+0000 7f0da7800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0da010c170 0x7f0da0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:43.275 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.274+0000 7f0da7800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0da0111fc0 con 0x7f0da0103660 2024-09-10T08:49:43.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.274+0000 7f0da7800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0da0111e40 con 0x7f0da010c170 2024-09-10T08:49:43.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.274+0000 7f0da7800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0da0112140 con 0x7f0da0102d40 2024-09-10T08:49:43.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.275+0000 7f0d97e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da0103660 0x7f0da0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:43.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.275+0000 7f0d97e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da0103660 0x7f0da0100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:52016/0 (socket says 172.21.15.89:52016) 2024-09-10T08:49:43.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.275+0000 7f0d97e00640 1 -- 172.21.15.89:0/3690865026 learned_addr learned my addr 172.21.15.89:0/3690865026 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:49:43.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.275+0000 7f0da4e00640 1 --2- 172.21.15.89:0/3690865026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0da0102d40 0x7f0da01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:43.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.275+0000 7f0da5800640 1 --2- 172.21.15.89:0/3690865026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0da010c170 0x7f0da0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:43.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.275+0000 7f0d97e00640 1 -- 172.21.15.89:0/3690865026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0da0102d40 msgr2=0x7f0da01009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:43.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.275+0000 7f0d97e00640 1 --2- 172.21.15.89:0/3690865026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0da0102d40 0x7f0da01009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:43.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.275+0000 7f0d97e00640 1 -- 172.21.15.89:0/3690865026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0da010c170 msgr2=0x7f0da0101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:43.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.275+0000 7f0d97e00640 1 --2- 172.21.15.89:0/3690865026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0da010c170 0x7f0da0101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:43.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.275+0000 7f0d97e00640 1 -- 172.21.15.89:0/3690865026 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da01a6880 con 0x7f0da0103660 2024-09-10T08:49:43.276 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.275+0000 7f0d97e00640 1 --2- 172.21.15.89:0/3690865026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da0103660 0x7f0da0100ee0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800d950 tx=0x7f0d8800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:43.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.276+0000 7f0d9e000640 1 -- 172.21.15.89:0/3690865026 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88017030 con 0x7f0da0103660 2024-09-10T08:49:43.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.276+0000 7f0d9e000640 1 -- 172.21.15.89:0/3690865026 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d88004020 con 0x7f0da0103660 2024-09-10T08:49:43.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.276+0000 7f0d9e000640 1 -- 172.21.15.89:0/3690865026 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d880047a0 con 0x7f0da0103660 2024-09-10T08:49:43.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.276+0000 7f0da7800640 1 -- 172.21.15.89:0/3690865026 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da01a6b70 con 0x7f0da0103660 2024-09-10T08:49:43.277 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.276+0000 7f0da7800640 1 -- 172.21.15.89:0/3690865026 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da01a7060 con 0x7f0da0103660 2024-09-10T08:49:43.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.277+0000 7f0da7800640 1 -- 172.21.15.89:0/3690865026 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da0103120 con 0x7f0da0103660 2024-09-10T08:49:43.278 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.279+0000 7f0d9e000640 1 -- 172.21.15.89:0/3690865026 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0d880041d0 con 0x7f0da0103660 2024-09-10T08:49:43.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.280+0000 7f0d9e000640 1 --2- 172.21.15.89:0/3690865026 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0d840a2c00 0x7f0d840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:43.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.280+0000 7f0d9e000640 1 -- 172.21.15.89:0/3690865026 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8912+0+0 (secure 0 0 0) 0x7f0d880ceaa0 con 0x7f0da0103660 2024-09-10T08:49:43.280 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.280+0000 7f0da4e00640 1 --2- 172.21.15.89:0/3690865026 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0d840a2c00 0x7f0d840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:43.281 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.281+0000 7f0da4e00640 1 --2- 172.21.15.89:0/3690865026 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0d840a2c00 0x7f0d840a50c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009bd0 tx=0x7f0d90005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:43.284 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.284+0000 7f0d9e000640 1 -- 172.21.15.89:0/3690865026 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d8809b720 con 0x7f0da0103660 2024-09-10T08:49:43.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.459+0000 7f0da7800640 1 -- 172.21.15.89:0/3690865026 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0da0101ed0 con 0x7f0da0103660 2024-09-10T08:49:43.646 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.646+0000 7f0d9e000640 1 -- 172.21.15.89:0/3690865026 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) ==== 169+0+0 (secure 0 0 0) 0x7f0d8809b020 con 0x7f0da0103660 2024-09-10T08:49:43.647 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data' removed 2024-09-10T08:49:43.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.650+0000 7f0da7800640 1 -- 172.21.15.89:0/3690865026 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0d840a2c00 msgr2=0x7f0d840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:43.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.650+0000 7f0da7800640 1 --2- 172.21.15.89:0/3690865026 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0d840a2c00 0x7f0d840a50c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009bd0 tx=0x7f0d90005f90 comp rx=0 tx=0).stop 2024-09-10T08:49:43.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.650+0000 7f0da7800640 1 -- 172.21.15.89:0/3690865026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da0103660 msgr2=0x7f0da0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:43.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.650+0000 7f0da7800640 1 --2- 172.21.15.89:0/3690865026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da0103660 0x7f0da0100ee0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800d950 tx=0x7f0d8800de10 comp rx=0 tx=0).stop 2024-09-10T08:49:43.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.650+0000 7f0da7800640 1 -- 172.21.15.89:0/3690865026 shutdown_connections 2024-09-10T08:49:43.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.650+0000 7f0da7800640 1 --2- 172.21.15.89:0/3690865026 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7f0d840a2c00 0x7f0d840a50c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:43.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.650+0000 7f0da7800640 1 --2- 172.21.15.89:0/3690865026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0da010c170 0x7f0da0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:43.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.650+0000 7f0da7800640 1 --2- 172.21.15.89:0/3690865026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da0103660 0x7f0da0100ee0 secure :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800d950 tx=0x7f0d8800de10 comp rx=0 tx=0).stop 2024-09-10T08:49:43.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.650+0000 7f0da7800640 1 --2- 172.21.15.89:0/3690865026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0da0102d40 0x7f0da01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:43.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.650+0000 7f0da7800640 1 -- 172.21.15.89:0/3690865026 >> 172.21.15.89:0/3690865026 conn(0x7f0da00fc7a0 msgr2=0x7f0da00fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:43.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.652+0000 7f0da7800640 1 -- 172.21.15.89:0/3690865026 shutdown_connections 2024-09-10T08:49:43.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:43.652+0000 7f0da7800640 1 -- 172.21.15.89:0/3690865026 wait complete. 2024-09-10T08:49:43.803 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:43 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3690865026' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:49:43.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:43 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3690865026' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:49:44.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:43 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3690865026' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:49:44.435 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-10T08:49:44.801 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:44.939 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:44 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/3690865026' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T08:49:44.939 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:44 smithi089 ceph-mon[30757]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T08:49:44.939 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:44 smithi089 ceph-mon[30757]: pgmap v690: 33 pgs: 33 active+clean; 577 KiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:49:44.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:44 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/3690865026' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T08:49:44.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:44 smithi149 ceph-mon[36167]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T08:49:44.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:44 smithi149 ceph-mon[36167]: pgmap v690: 33 pgs: 33 active+clean; 577 KiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:49:45.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:44 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/3690865026' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T08:49:45.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:44 smithi111 ceph-mon[35220]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T08:49:45.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:44 smithi111 ceph-mon[35220]: pgmap v690: 33 pgs: 33 active+clean; 577 KiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:49:45.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.391+0000 7fd2b0e00640 1 -- 172.21.15.89:0/2465761719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2ac1034b0 msgr2=0x7fd2ac10b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:45.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.391+0000 7fd2b0e00640 1 --2- 172.21.15.89:0/2465761719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2ac1034b0 0x7fd2ac10b9a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd294009900 tx=0x7fd29402a480 comp rx=0 tx=0).stop 2024-09-10T08:49:45.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.391+0000 7fd2b0e00640 1 -- 172.21.15.89:0/2465761719 shutdown_connections 2024-09-10T08:49:45.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.391+0000 7fd2b0e00640 1 --2- 172.21.15.89:0/2465761719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2ac10bfc0 0x7fd2ac10e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:45.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.391+0000 7fd2b0e00640 1 --2- 172.21.15.89:0/2465761719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2ac1034b0 0x7fd2ac10b9a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:45.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.391+0000 7fd2b0e00640 1 --2- 172.21.15.89:0/2465761719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2ac102b90 0x7fd2ac102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:45.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.392+0000 7fd2b0e00640 1 -- 172.21.15.89:0/2465761719 >> 172.21.15.89:0/2465761719 conn(0x7fd2ac0fc610 msgr2=0x7fd2ac0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:45.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.392+0000 7fd2b0e00640 1 -- 172.21.15.89:0/2465761719 shutdown_connections 2024-09-10T08:49:45.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.392+0000 7fd2b0e00640 1 -- 172.21.15.89:0/2465761719 wait complete. 2024-09-10T08:49:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.393+0000 7fd2b0e00640 1 Processor -- start 2024-09-10T08:49:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.393+0000 7fd2b0e00640 1 -- start start 2024-09-10T08:49:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.394+0000 7fd2b0e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2ac102b90 0x7fd2ac1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.394+0000 7fd2b0e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2ac1034b0 0x7fd2ac1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.394+0000 7fd2b0e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2ac10bfc0 0x7fd2ac19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.394+0000 7fd2b0e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac111e10 con 0x7fd2ac10bfc0 2024-09-10T08:49:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.394+0000 7fd2b0e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac111c90 con 0x7fd2ac102b90 2024-09-10T08:49:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.394+0000 7fd2b0e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac111f90 con 0x7fd2ac1034b0 2024-09-10T08:49:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.394+0000 7fd2ab400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2ac102b90 0x7fd2ac1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.394+0000 7fd2ab400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2ac102b90 0x7fd2ac1982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.89:55198/0 (socket says 172.21.15.89:55198) 2024-09-10T08:49:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.394+0000 7fd2ab400640 1 -- 172.21.15.89:0/1733221505 learned_addr learned my addr 172.21.15.89:0/1733221505 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-10T08:49:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.394+0000 7fd2a3e00640 1 --2- 172.21.15.89:0/1733221505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2ac1034b0 0x7fd2ac1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:45.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.394+0000 7fd2abe00640 1 --2- 172.21.15.89:0/1733221505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2ac10bfc0 0x7fd2ac19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:45.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.394+0000 7fd2ab400640 1 -- 172.21.15.89:0/1733221505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2ac1034b0 msgr2=0x7fd2ac1987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:45.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.394+0000 7fd2ab400640 1 --2- 172.21.15.89:0/1733221505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2ac1034b0 0x7fd2ac1987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:45.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.394+0000 7fd2ab400640 1 -- 172.21.15.89:0/1733221505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2ac10bfc0 msgr2=0x7fd2ac19cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:45.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.394+0000 7fd2ab400640 1 --2- 172.21.15.89:0/1733221505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2ac10bfc0 0x7fd2ac19cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:45.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.394+0000 7fd2ab400640 1 -- 172.21.15.89:0/1733221505 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ac19d260 con 0x7fd2ac102b90 2024-09-10T08:49:45.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.395+0000 7fd2ab400640 1 --2- 172.21.15.89:0/1733221505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2ac102b90 0x7fd2ac1982b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00ebc0 tx=0x7fd29c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.395+0000 7fd2a8c00640 1 -- 172.21.15.89:0/1733221505 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c010630 con 0x7fd2ac102b90 2024-09-10T08:49:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.395+0000 7fd2b0e00640 1 -- 172.21.15.89:0/1733221505 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2ac19d550 con 0x7fd2ac102b90 2024-09-10T08:49:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.395+0000 7fd2a8c00640 1 -- 172.21.15.89:0/1733221505 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29c010cb0 con 0x7fd2ac102b90 2024-09-10T08:49:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.395+0000 7fd2a8c00640 1 -- 172.21.15.89:0/1733221505 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c018bd0 con 0x7fd2ac102b90 2024-09-10T08:49:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.395+0000 7fd2b0e00640 1 -- 172.21.15.89:0/1733221505 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2ac1a4e30 con 0x7fd2ac102b90 2024-09-10T08:49:45.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.396+0000 7fd2b0e00640 1 -- 172.21.15.89:0/1733221505 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2ac102f70 con 0x7fd2ac102b90 2024-09-10T08:49:45.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.398+0000 7fd2a8c00640 1 -- 172.21.15.89:0/1733221505 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd29c0107d0 con 0x7fd2ac102b90 2024-09-10T08:49:45.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.399+0000 7fd2a8c00640 1 --2- 172.21.15.89:0/1733221505 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd2900a2b80 0x7fd2900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:45.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.400+0000 7fd2a8c00640 1 -- 172.21.15.89:0/1733221505 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8324+0+0 (secure 0 0 0) 0x7fd29c0cfa00 con 0x7fd2ac102b90 2024-09-10T08:49:45.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.400+0000 7fd2a3e00640 1 --2- 172.21.15.89:0/1733221505 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd2900a2b80 0x7fd2900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:45.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.400+0000 7fd2a3e00640 1 --2- 172.21.15.89:0/1733221505 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd2900a2b80 0x7fd2900a5040 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd294000c00 tx=0x7fd294005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:45.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.402+0000 7fd2a8c00640 1 -- 172.21.15.89:0/1733221505 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd29c09c250 con 0x7fd2ac102b90 2024-09-10T08:49:45.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.581+0000 7fd2b0e00640 1 -- 172.21.15.89:0/1733221505 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd2ac199610 con 0x7fd2ac102b90 2024-09-10T08:49:45.666 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.666+0000 7fd2a8c00640 1 -- 172.21.15.89:0/1733221505 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v100) ==== 178+0+0 (secure 0 0 0) 0x7fd2ac199610 con 0x7fd2ac102b90 2024-09-10T08:49:45.667 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'cephfs_data_ec' removed 2024-09-10T08:49:45.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.670+0000 7fd2b0e00640 1 -- 172.21.15.89:0/1733221505 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd2900a2b80 msgr2=0x7fd2900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:45.670 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.670+0000 7fd2b0e00640 1 --2- 172.21.15.89:0/1733221505 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd2900a2b80 0x7fd2900a5040 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd294000c00 tx=0x7fd294005b50 comp rx=0 tx=0).stop 2024-09-10T08:49:45.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.670+0000 7fd2b0e00640 1 -- 172.21.15.89:0/1733221505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2ac102b90 msgr2=0x7fd2ac1982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:45.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.670+0000 7fd2b0e00640 1 --2- 172.21.15.89:0/1733221505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2ac102b90 0x7fd2ac1982b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00ebc0 tx=0x7fd29c00c6a0 comp rx=0 tx=0).stop 2024-09-10T08:49:45.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.670+0000 7fd2b0e00640 1 -- 172.21.15.89:0/1733221505 shutdown_connections 2024-09-10T08:49:45.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.670+0000 7fd2b0e00640 1 --2- 172.21.15.89:0/1733221505 >> [v2:172.21.15.89:6800/1633345099,v1:172.21.15.89:6801/1633345099] conn(0x7fd2900a2b80 0x7fd2900a5040 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:45.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.670+0000 7fd2b0e00640 1 --2- 172.21.15.89:0/1733221505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd2ac10bfc0 0x7fd2ac19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:45.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.670+0000 7fd2b0e00640 1 --2- 172.21.15.89:0/1733221505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2ac1034b0 0x7fd2ac1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:45.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.670+0000 7fd2b0e00640 1 --2- 172.21.15.89:0/1733221505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2ac102b90 0x7fd2ac1982b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:45.671 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.670+0000 7fd2b0e00640 1 -- 172.21.15.89:0/1733221505 >> 172.21.15.89:0/1733221505 conn(0x7fd2ac0fc610 msgr2=0x7fd2ac0fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:45.672 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.671+0000 7fd2b0e00640 1 -- 172.21.15.89:0/1733221505 shutdown_connections 2024-09-10T08:49:45.674 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-10T08:49:45.674+0000 7fd2b0e00640 1 -- 172.21.15.89:0/1733221505 wait complete. 2024-09-10T08:49:45.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:45 smithi149 ceph-mon[36167]: from='client.? 172.21.15.89:0/1733221505' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:49:46.163 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:45 smithi149 ceph-mon[36167]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:49:46.164 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:45 smithi089 ceph-mon[30757]: from='client.? 172.21.15.89:0/1733221505' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:49:46.164 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:45 smithi089 ceph-mon[30757]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:49:46.164 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:45 smithi111 ceph-mon[35220]: from='client.? 172.21.15.89:0/1733221505' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:49:46.165 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:45 smithi111 ceph-mon[35220]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T08:49:46.455 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T08:49:46.512 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T08:49:46.541 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T08:49:46.571 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T08:49:46.572 DEBUG:teuthology.orchestra.run.smithi089:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T08:49:46.607 DEBUG:teuthology.orchestra.run.smithi111:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T08:49:46.623 DEBUG:teuthology.orchestra.run.smithi149:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T08:49:46.640 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T08:49:46.640 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T08:49:46.640 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mon.a 2024-09-10T08:49:46.695 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:46 smithi089 ceph-mon[30757]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T08:49:46.696 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:46 smithi089 ceph-mon[30757]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T08:49:46.696 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:46 smithi089 ceph-mon[30757]: pgmap v692: 1 pgs: 1 active+clean; 577 KiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:49:46.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:46 smithi111 ceph-mon[35220]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T08:49:46.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:46 smithi111 ceph-mon[35220]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T08:49:46.794 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:46 smithi111 ceph-mon[35220]: pgmap v692: 1 pgs: 1 active+clean; 577 KiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:49:46.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:46 smithi149 ceph-mon[36167]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T08:49:46.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:46 smithi149 ceph-mon[36167]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T08:49:46.971 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:46 smithi149 ceph-mon[36167]: pgmap v692: 1 pgs: 1 active+clean; 577 KiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:49:47.053 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:46 smithi089 systemd[1]: Stopping Ceph mon.a for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:49:47.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:47 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-a[30753]: 2024-09-10T08:49:47.084+0000 7f4555a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:49:47.553 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:47 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-a[30753]: 2024-09-10T08:49:47.084+0000 7f4555a00640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T08:49:47.554 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:47 smithi089 podman[106021]: 2024-09-10 08:49:47.258197746 +0000 UTC m=+0.334135925 container died 975872fc80f0340371cdff7e9073f5d8c75ef9b60c8378582473de8625f73fc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T08:49:47.957 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:47 smithi089 podman[106021]: 2024-09-10 08:49:47.689071434 +0000 UTC m=+0.765009613 container cleanup 975872fc80f0340371cdff7e9073f5d8c75ef9b60c8378582473de8625f73fc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T08:49:47.958 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:47 smithi089 bash[106021]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-a 2024-09-10T08:49:48.210 INFO:journalctl@ceph.mon.a.smithi089.stdout:Sep 10 08:49:47 smithi089 podman[106033]: 2024-09-10 08:49:47.957749627 +0000 UTC m=+0.685718941 container remove 975872fc80f0340371cdff7e9073f5d8c75ef9b60c8378582473de8625f73fc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T08:49:48.217 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T08:49:48.305 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:49:48.305 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T08:49:48.305 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-10T08:49:48.305 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mon.b 2024-09-10T08:49:48.784 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:48 smithi111 systemd[1]: Stopping Ceph mon.b for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:49:49.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:48 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-b[35216]: 2024-09-10T08:49:48.782+0000 7f8b7b7ee640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:49:49.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:48 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-b[35216]: 2024-09-10T08:49:48.782+0000 7f8b7b7ee640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T08:49:49.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:48 smithi111 podman[69713]: 2024-09-10 08:49:48.854818396 +0000 UTC m=+0.220140629 container died 041932f325a20ba08b274792a283e682998dc246f9c4cd06cc9ad6a900ba823c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:49:49.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:49 smithi111 podman[69713]: 2024-09-10 08:49:49.383949156 +0000 UTC m=+0.749271377 container cleanup 041932f325a20ba08b274792a283e682998dc246f9c4cd06cc9ad6a900ba823c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-10T08:49:49.544 INFO:journalctl@ceph.mon.b.smithi111.stdout:Sep 10 08:49:49 smithi111 bash[69713]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-b 2024-09-10T08:49:49.817 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T08:49:49.861 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:49:49.862 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-10T08:49:49.862 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-10T08:49:49.862 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mon.c 2024-09-10T08:49:50.200 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:49 smithi149 systemd[1]: Stopping Ceph mon.c for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:49:50.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:50 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-c[36163]: 2024-09-10T08:49:50.198+0000 7f912e210640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:49:50.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:50 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-c[36163]: 2024-09-10T08:49:50.198+0000 7f912e210640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T08:49:50.471 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:50 smithi149 podman[71286]: 2024-09-10 08:49:50.230366828 +0000 UTC m=+0.152619749 container died 4f4d000116def9740424e7ec615117db366b7cefbd7b5e5f291220236d658b9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-c, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:49:50.894 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:50 smithi149 podman[71286]: 2024-09-10 08:49:50.591101111 +0000 UTC m=+0.513354030 container cleanup 4f4d000116def9740424e7ec615117db366b7cefbd7b5e5f291220236d658b9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T08:49:50.894 INFO:journalctl@ceph.mon.c.smithi149.stdout:Sep 10 08:49:50 smithi149 bash[71286]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mon-c 2024-09-10T08:49:51.031 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T08:49:51.081 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:49:51.082 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-10T08:49:51.082 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T08:49:51.082 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mgr.x 2024-09-10T08:49:51.490 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:49:51 smithi089 systemd[1]: Stopping Ceph mgr.x for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:49:51.803 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:49:51 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:49:51.489+0000 7f539f600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:49:51.804 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:49:51 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x[30979]: 2024-09-10T08:49:51.489+0000 7f539f600640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-10T08:49:51.804 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:49:51 smithi089 podman[106140]: 2024-09-10 08:49:51.571749383 +0000 UTC m=+0.205020911 container died 68b20d409ffe2c1c22c43cfc133613ccc063a40fc2d9afc32010d1bca28ff56a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-10T08:49:52.187 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:49:52 smithi089 podman[106140]: 2024-09-10 08:49:52.187519503 +0000 UTC m=+0.820791030 container cleanup 68b20d409ffe2c1c22c43cfc133613ccc063a40fc2d9afc32010d1bca28ff56a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-09-10T08:49:52.553 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:49:52 smithi089 bash[106140]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x 2024-09-10T08:49:52.553 INFO:journalctl@ceph.mgr.x.smithi089.stdout:Sep 10 08:49:52 smithi089 podman[106151]: 2024-09-10 08:49:52.414846145 +0000 UTC m=+0.834794113 container remove 68b20d409ffe2c1c22c43cfc133613ccc063a40fc2d9afc32010d1bca28ff56a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:49:52.673 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T08:49:52.747 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:49:52.748 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T08:49:52.748 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T08:49:52.748 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mgr.y 2024-09-10T08:49:53.044 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:49:52 smithi111 systemd[1]: Stopping Ceph mgr.y for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:49:53.365 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:49:53 smithi111 podman[69836]: 2024-09-10 08:49:53.076552391 +0000 UTC m=+0.138690413 container died 1e5793fe7519abbc91dc1e5f6d50e61424bc9dacc6ac5bf1fccdece156ed8844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:49:53.625 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:49:53 smithi111 podman[69836]: 2024-09-10 08:49:53.439886862 +0000 UTC m=+0.502024880 container cleanup 1e5793fe7519abbc91dc1e5f6d50e61424bc9dacc6ac5bf1fccdece156ed8844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:49:53.625 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:49:53 smithi111 bash[69836]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y 2024-09-10T08:49:53.853 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T08:49:53.876 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:49:53 smithi111 podman[69848]: 2024-09-10 08:49:53.625152184 +0000 UTC m=+0.543491046 container remove 1e5793fe7519abbc91dc1e5f6d50e61424bc9dacc6ac5bf1fccdece156ed8844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-10T08:49:53.876 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:49:53 smithi111 systemd[1]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-10T08:49:53.876 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:49:53 smithi111 systemd[1]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-09-10T08:49:53.876 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:49:53 smithi111 systemd[1]: Stopped Ceph mgr.y for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:49:53.876 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Sep 10 08:49:53 smithi111 systemd[1]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mgr.y.service: Consumed 6.556s CPU time. 2024-09-10T08:49:53.890 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:49:53.890 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T08:49:53.890 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-10T08:49:53.890 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mgr.z 2024-09-10T08:49:54.221 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:49:53 smithi149 systemd[1]: Stopping Ceph mgr.z for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:49:54.566 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:49:54 smithi149 podman[71410]: 2024-09-10 08:49:54.251176798 +0000 UTC m=+0.160933208 container died ee474161c8b215bccd2e3e1b2ec102c91dcfd6b7ce0ee4a88436cc86da5e245b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T08:49:54.916 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:49:54 smithi149 podman[71410]: 2024-09-10 08:49:54.672489169 +0000 UTC m=+0.582245579 container cleanup ee474161c8b215bccd2e3e1b2ec102c91dcfd6b7ce0ee4a88436cc86da5e245b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2) 2024-09-10T08:49:54.916 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:49:54 smithi149 bash[71410]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z 2024-09-10T08:49:55.150 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-10T08:49:55.172 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:49:54 smithi149 podman[71423]: 2024-09-10 08:49:54.915721667 +0000 UTC m=+0.656284311 container remove ee474161c8b215bccd2e3e1b2ec102c91dcfd6b7ce0ee4a88436cc86da5e245b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:49:55.172 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:49:54 smithi149 systemd[1]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-10T08:49:55.172 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:49:55 smithi149 systemd[1]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-09-10T08:49:55.172 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:49:55 smithi149 systemd[1]: Stopped Ceph mgr.z for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:49:55.172 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Sep 10 08:49:55 smithi149 systemd[1]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mgr.z.service: Consumed 6.645s CPU time. 2024-09-10T08:49:55.185 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:49:55.185 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-10T08:49:55.185 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T08:49:55.186 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.0 2024-09-10T08:49:55.553 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:49:55 smithi089 systemd[1]: Stopping Ceph osd.0 for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:49:56.053 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:49:55 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-0[44616]: 2024-09-10T08:49:55.603+0000 7f7b60800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:49:56.053 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:49:55 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-0[44616]: 2024-09-10T08:49:55.603+0000 7f7b60800640 -1 osd.0 100 *** Got signal Terminated *** 2024-09-10T08:49:56.053 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:49:55 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-0[44616]: 2024-09-10T08:49:55.603+0000 7f7b60800640 -1 osd.0 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:50:01.470 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:50:01 smithi089 podman[106261]: 2024-09-10 08:50:01.10391824 +0000 UTC m=+5.632257495 container died f3b9067605366edb25880617fea0acefd26fd3f468a4c0dcb00f77a60fd7528c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:50:01.721 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:50:01 smithi089 podman[106261]: 2024-09-10 08:50:01.684640616 +0000 UTC m=+6.212979868 container cleanup f3b9067605366edb25880617fea0acefd26fd3f468a4c0dcb00f77a60fd7528c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True) 2024-09-10T08:50:01.721 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:50:01 smithi089 bash[106261]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-0 2024-09-10T08:50:02.053 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:50:01 smithi089 podman[106278]: 2024-09-10 08:50:01.928512953 +0000 UTC m=+0.811248001 container remove f3b9067605366edb25880617fea0acefd26fd3f468a4c0dcb00f77a60fd7528c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:50:02.757 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:50:02 smithi089 podman[106371]: 2024-09-10 08:50:02.373324626 +0000 UTC m=+0.027053282 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:50:02.757 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:50:02 smithi089 podman[106371]: 2024-09-10 08:50:02.496256782 +0000 UTC m=+0.149985421 container create bfd010f21a0ba7490d7111f88fdfb07db9daa0092a17b253d28019b2d14504ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T08:50:03.020 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:50:02 smithi089 podman[106371]: 2024-09-10 08:50:02.757563274 +0000 UTC m=+0.411291942 container init bfd010f21a0ba7490d7111f88fdfb07db9daa0092a17b253d28019b2d14504ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T08:50:03.021 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:50:02 smithi089 podman[106371]: 2024-09-10 08:50:02.764300864 +0000 UTC m=+0.418029505 container start bfd010f21a0ba7490d7111f88fdfb07db9daa0092a17b253d28019b2d14504ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T08:50:03.021 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:50:02 smithi089 podman[106371]: 2024-09-10 08:50:02.841009583 +0000 UTC m=+0.494738238 container attach bfd010f21a0ba7490d7111f88fdfb07db9daa0092a17b253d28019b2d14504ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:50:03.303 INFO:journalctl@ceph.osd.0.smithi089.stdout:Sep 10 08:50:03 smithi089 podman[106371]: 2024-09-10 08:50:03.020995499 +0000 UTC m=+0.674724149 container died bfd010f21a0ba7490d7111f88fdfb07db9daa0092a17b253d28019b2d14504ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T08:50:03.776 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T08:50:03.855 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:50:03.856 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T08:50:03.856 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T08:50:03.856 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.1 2024-09-10T08:50:04.213 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:50:03 smithi111 systemd[1]: Stopping Ceph osd.1 for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:50:04.544 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:50:04 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-1[39687]: 2024-09-10T08:50:04.211+0000 7f349a253640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:50:04.544 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:50:04 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-1[39687]: 2024-09-10T08:50:04.211+0000 7f349a253640 -1 osd.1 100 *** Got signal Terminated *** 2024-09-10T08:50:04.544 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:50:04 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-1[39687]: 2024-09-10T08:50:04.211+0000 7f349a253640 -1 osd.1 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:50:10.044 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:50:09 smithi111 podman[69972]: 2024-09-10 08:50:09.623803103 +0000 UTC m=+5.561622438 container died e145fb0578cb884b0c8341283bf1c057352ec203ef595557de8fdb48d3b1b202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True) 2024-09-10T08:50:10.406 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:50:10 smithi111 podman[69972]: 2024-09-10 08:50:10.14572129 +0000 UTC m=+6.083540625 container cleanup e145fb0578cb884b0c8341283bf1c057352ec203ef595557de8fdb48d3b1b202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T08:50:10.406 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:50:10 smithi111 bash[69972]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-1 2024-09-10T08:50:10.706 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:50:10 smithi111 podman[69989]: 2024-09-10 08:50:10.405919306 +0000 UTC m=+0.764675879 container remove e145fb0578cb884b0c8341283bf1c057352ec203ef595557de8fdb48d3b1b202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-1, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:50:10.968 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:50:10 smithi111 podman[70076]: 2024-09-10 08:50:10.706811318 +0000 UTC m=+0.106601313 container create ec8ac8aa9f8e6c9a3f43aa3cf76bfee1e81f478107fff2030e4a53c0c76ee6ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-1-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-10T08:50:10.968 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:50:10 smithi111 podman[70076]: 2024-09-10 08:50:10.615874596 +0000 UTC m=+0.015664605 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:50:10.968 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:50:10 smithi111 podman[70076]: 2024-09-10 08:50:10.968033476 +0000 UTC m=+0.367823472 container init ec8ac8aa9f8e6c9a3f43aa3cf76bfee1e81f478107fff2030e4a53c0c76ee6ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T08:50:11.264 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:50:10 smithi111 podman[70076]: 2024-09-10 08:50:10.97158471 +0000 UTC m=+0.371374706 container start ec8ac8aa9f8e6c9a3f43aa3cf76bfee1e81f478107fff2030e4a53c0c76ee6ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T08:50:11.264 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:50:11 smithi111 podman[70076]: 2024-09-10 08:50:11.101808968 +0000 UTC m=+0.501598973 container attach ec8ac8aa9f8e6c9a3f43aa3cf76bfee1e81f478107fff2030e4a53c0c76ee6ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True) 2024-09-10T08:50:11.544 INFO:journalctl@ceph.osd.1.smithi111.stdout:Sep 10 08:50:11 smithi111 podman[70076]: 2024-09-10 08:50:11.264587238 +0000 UTC m=+0.664377241 container died ec8ac8aa9f8e6c9a3f43aa3cf76bfee1e81f478107fff2030e4a53c0c76ee6ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T08:50:11.871 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T08:50:11.910 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:50:11.910 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T08:50:11.911 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T08:50:11.911 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.2 2024-09-10T08:50:12.220 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:50:11 smithi149 systemd[1]: Stopping Ceph osd.2 for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:50:12.720 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:50:12 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-2[41115]: 2024-09-10T08:50:12.302+0000 7fedfd417640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:50:12.721 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:50:12 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-2[41115]: 2024-09-10T08:50:12.302+0000 7fedfd417640 -1 osd.2 100 *** Got signal Terminated *** 2024-09-10T08:50:12.721 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:50:12 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-2[41115]: 2024-09-10T08:50:12.302+0000 7fedfd417640 -1 osd.2 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:50:18.111 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:50:17 smithi149 podman[71550]: 2024-09-10 08:50:17.793907963 +0000 UTC m=+5.687650702 container died 2f97966fd9b0d838dfd67253d2f20531deec7d99a2e6502ea934e3ef65a1241c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:50:18.445 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:50:18 smithi149 podman[71550]: 2024-09-10 08:50:18.210521836 +0000 UTC m=+6.104264575 container cleanup 2f97966fd9b0d838dfd67253d2f20531deec7d99a2e6502ea934e3ef65a1241c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:50:18.445 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:50:18 smithi149 bash[71550]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-2 2024-09-10T08:50:18.720 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:50:18 smithi149 podman[71566]: 2024-09-10 08:50:18.445348417 +0000 UTC m=+0.646279800 container remove 2f97966fd9b0d838dfd67253d2f20531deec7d99a2e6502ea934e3ef65a1241c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:50:19.158 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:50:18 smithi149 podman[71653]: 2024-09-10 08:50:18.781167976 +0000 UTC m=+0.015601139 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:50:19.158 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:50:18 smithi149 podman[71653]: 2024-09-10 08:50:18.896602478 +0000 UTC m=+0.131035696 container create 6f4f7fd31ffbb4f001a9e1f75446d3c946901e37418654d91296a2065aa18fb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T08:50:19.158 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:50:19 smithi149 podman[71653]: 2024-09-10 08:50:19.157445762 +0000 UTC m=+0.391878918 container init 6f4f7fd31ffbb4f001a9e1f75446d3c946901e37418654d91296a2065aa18fb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T08:50:19.471 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:50:19 smithi149 podman[71653]: 2024-09-10 08:50:19.161203042 +0000 UTC m=+0.395636199 container start 6f4f7fd31ffbb4f001a9e1f75446d3c946901e37418654d91296a2065aa18fb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:50:19.471 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:50:19 smithi149 podman[71653]: 2024-09-10 08:50:19.257597723 +0000 UTC m=+0.492030880 container attach 6f4f7fd31ffbb4f001a9e1f75446d3c946901e37418654d91296a2065aa18fb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T08:50:19.471 INFO:journalctl@ceph.osd.2.smithi149.stdout:Sep 10 08:50:19 smithi149 podman[71653]: 2024-09-10 08:50:19.392862781 +0000 UTC m=+0.627295939 container died 6f4f7fd31ffbb4f001a9e1f75446d3c946901e37418654d91296a2065aa18fb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T08:50:20.068 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T08:50:20.111 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:50:20.111 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T08:50:20.111 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T08:50:20.112 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.3 2024-09-10T08:50:20.534 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 10 08:50:20 smithi089 systemd[1]: Stopping Ceph osd.3 for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:50:20.803 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 10 08:50:20 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-3[51082]: 2024-09-10T08:50:20.532+0000 7f6077200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:50:20.803 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 10 08:50:20 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-3[51082]: 2024-09-10T08:50:20.532+0000 7f6077200640 -1 osd.3 100 *** Got signal Terminated *** 2024-09-10T08:50:20.804 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 10 08:50:20 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-3[51082]: 2024-09-10T08:50:20.532+0000 7f6077200640 -1 osd.3 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:50:22.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:22 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:22.342+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:23.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:23 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:23.295+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:24.329 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:23 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:23.998+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:58.612764+0000 front 2024-09-10T08:49:58.612747+0000 (oldest deadline 2024-09-10T08:50:23.912615+0000) 2024-09-10T08:50:24.720 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:24 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:24.327+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:25.373 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:24 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:24.995+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:58.612764+0000 front 2024-09-10T08:49:58.612747+0000 (oldest deadline 2024-09-10T08:50:23.912615+0000) 2024-09-10T08:50:25.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:25 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:25.371+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:25.803 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 10 08:50:25 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-3[51082]: 2024-09-10T08:50:25.341+0000 7f606dc00640 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.384145+0000 front 2024-09-10T08:49:59.384177+0000 (oldest deadline 2024-09-10T08:50:24.684393+0000) 2024-09-10T08:50:26.303 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 10 08:50:26 smithi089 podman[106473]: 2024-09-10 08:50:26.018322095 +0000 UTC m=+5.612824876 container died d2172eaee4d2d2b799e39bf1ecac67d23bfb989ed94560ad657b45fe02be9ca4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:50:26.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:25 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:25.960+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:26.344 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:26 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:26.013+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:58.612764+0000 front 2024-09-10T08:49:58.612747+0000 (oldest deadline 2024-09-10T08:50:23.912615+0000) 2024-09-10T08:50:26.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:26 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:26.108+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:26.627 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 10 08:50:26 smithi089 podman[106473]: 2024-09-10 08:50:26.606106318 +0000 UTC m=+6.200609096 container cleanup d2172eaee4d2d2b799e39bf1ecac67d23bfb989ed94560ad657b45fe02be9ca4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T08:50:26.627 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 10 08:50:26 smithi089 bash[106473]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-3 2024-09-10T08:50:26.720 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:26 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:26.343+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:27.303 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 10 08:50:26 smithi089 podman[106491]: 2024-09-10 08:50:26.824420378 +0000 UTC m=+0.802496770 container remove d2172eaee4d2d2b799e39bf1ecac67d23bfb989ed94560ad657b45fe02be9ca4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T08:50:27.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:26 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:26.989+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:27.386 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:27 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:27.004+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:58.612764+0000 front 2024-09-10T08:49:58.612747+0000 (oldest deadline 2024-09-10T08:50:23.912615+0000) 2024-09-10T08:50:27.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:27 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:27.141+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:27.670 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 10 08:50:27 smithi089 podman[106576]: 2024-09-10 08:50:27.270921999 +0000 UTC m=+0.026759798 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:50:27.670 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 10 08:50:27 smithi089 podman[106576]: 2024-09-10 08:50:27.417309087 +0000 UTC m=+0.173146909 container create 972063de701a6429bb74a662f5aca87983d51a5b6713a53d44e4cf54b631b7a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-10T08:50:27.720 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:27 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:27.384+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:27.935 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 10 08:50:27 smithi089 podman[106576]: 2024-09-10 08:50:27.670092679 +0000 UTC m=+0.425930463 container init 972063de701a6429bb74a662f5aca87983d51a5b6713a53d44e4cf54b631b7a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-10T08:50:27.936 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 10 08:50:27 smithi089 podman[106576]: 2024-09-10 08:50:27.676929154 +0000 UTC m=+0.432766943 container start 972063de701a6429bb74a662f5aca87983d51a5b6713a53d44e4cf54b631b7a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T08:50:27.936 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 10 08:50:27 smithi089 podman[106576]: 2024-09-10 08:50:27.769732441 +0000 UTC m=+0.525570226 container attach 972063de701a6429bb74a662f5aca87983d51a5b6713a53d44e4cf54b631b7a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731) 2024-09-10T08:50:28.303 INFO:journalctl@ceph.osd.3.smithi089.stdout:Sep 10 08:50:27 smithi089 podman[106576]: 2024-09-10 08:50:27.935816317 +0000 UTC m=+0.691654112 container died 972063de701a6429bb74a662f5aca87983d51a5b6713a53d44e4cf54b631b7a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:50:28.303 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:28 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:28.026+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:28.368 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:27 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:27.994+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:58.612764+0000 front 2024-09-10T08:49:58.612747+0000 (oldest deadline 2024-09-10T08:50:23.912615+0000) 2024-09-10T08:50:28.543 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:28 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:28.141+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:28.720 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:28 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:28.367+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:28.768 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T08:50:28.845 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:50:28.845 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T08:50:28.845 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T08:50:28.846 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.4 2024-09-10T08:50:29.159 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:50:28 smithi111 systemd[1]: Stopping Ceph osd.4 for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:50:29.220 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:28 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:28.957+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:58.612764+0000 front 2024-09-10T08:49:58.612747+0000 (oldest deadline 2024-09-10T08:50:23.912615+0000) 2024-09-10T08:50:29.303 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:29 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:29.022+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:29.543 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:29 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:29.167+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:29.544 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:50:29 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-4[46171]: 2024-09-10T08:50:29.157+0000 7f7cfc0ef640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:50:29.544 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:50:29 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-4[46171]: 2024-09-10T08:50:29.157+0000 7f7cfc0ef640 -1 osd.4 100 *** Got signal Terminated *** 2024-09-10T08:50:29.544 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:50:29 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-4[46171]: 2024-09-10T08:50:29.157+0000 7f7cfc0ef640 -1 osd.4 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:50:29.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:29 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:29.339+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:30.044 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:50:29 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-4[46171]: 2024-09-10T08:50:29.765+0000 7f7cf46fd640 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.460357+0000 front 2024-09-10T08:50:04.460476+0000 (oldest deadline 2024-09-10T08:50:29.760489+0000) 2024-09-10T08:50:30.220 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:29 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:29.909+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:58.612764+0000 front 2024-09-10T08:49:58.612747+0000 (oldest deadline 2024-09-10T08:50:23.912615+0000) 2024-09-10T08:50:30.303 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:30 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:30.041+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:30.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:30 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:30.195+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:30.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:30 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:30.195+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:30.720 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:30 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:30.333+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:31.044 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:50:30 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-4[46171]: 2024-09-10T08:50:30.791+0000 7f7cf46fd640 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.460357+0000 front 2024-09-10T08:50:04.460476+0000 (oldest deadline 2024-09-10T08:50:29.760489+0000) 2024-09-10T08:50:31.220 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:30 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:30.921+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:58.612764+0000 front 2024-09-10T08:49:58.612747+0000 (oldest deadline 2024-09-10T08:50:23.912615+0000) 2024-09-10T08:50:31.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:30 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:30.921+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.613570+0000 front 2024-09-10T08:50:05.613653+0000 (oldest deadline 2024-09-10T08:50:30.313631+0000) 2024-09-10T08:50:31.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:31 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:31.217+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:31.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:31 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:31.217+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:31.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:31 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:31.068+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:31.608 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:31 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:31.362+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:32.044 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:50:31 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-4[46171]: 2024-09-10T08:50:31.763+0000 7f7cf46fd640 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.460357+0000 front 2024-09-10T08:50:04.460476+0000 (oldest deadline 2024-09-10T08:50:29.760489+0000) 2024-09-10T08:50:32.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:31 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:31.556+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:32.220 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:31 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:31.909+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:58.612764+0000 front 2024-09-10T08:49:58.612747+0000 (oldest deadline 2024-09-10T08:50:23.912615+0000) 2024-09-10T08:50:32.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:31 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:31.909+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.613570+0000 front 2024-09-10T08:50:05.613653+0000 (oldest deadline 2024-09-10T08:50:30.313631+0000) 2024-09-10T08:50:32.320 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:32 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:32.199+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:32.321 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:32 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:32.199+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:32.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:32 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:32.104+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:32.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:32 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:32.104+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:32.720 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:32 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:32.389+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:33.043 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:50:32 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-4[46171]: 2024-09-10T08:50:32.754+0000 7f7cf46fd640 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.460357+0000 front 2024-09-10T08:50:04.460476+0000 (oldest deadline 2024-09-10T08:50:29.760489+0000) 2024-09-10T08:50:33.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:32 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:32.584+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:33.220 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:32 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:32.893+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:33.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:32 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:32.931+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:58.612764+0000 front 2024-09-10T08:49:58.612747+0000 (oldest deadline 2024-09-10T08:50:23.912615+0000) 2024-09-10T08:50:33.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:32 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:32.931+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.613570+0000 front 2024-09-10T08:50:05.613653+0000 (oldest deadline 2024-09-10T08:50:30.313631+0000) 2024-09-10T08:50:33.543 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:33 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:33.166+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:33.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:33 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:33.166+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:33.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:33 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:33.109+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:33.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:33 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:33.109+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:33.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:33 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:33.413+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:34.044 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:50:33 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-4[46171]: 2024-09-10T08:50:33.777+0000 7f7cf46fd640 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.460357+0000 front 2024-09-10T08:50:04.460476+0000 (oldest deadline 2024-09-10T08:50:29.760489+0000) 2024-09-10T08:50:34.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:33 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:33.617+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:34.220 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:33 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:33.884+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:34.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:33 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:33.959+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:58.612764+0000 front 2024-09-10T08:49:58.612747+0000 (oldest deadline 2024-09-10T08:50:23.912615+0000) 2024-09-10T08:50:34.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:33 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:33.959+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.613570+0000 front 2024-09-10T08:50:05.613653+0000 (oldest deadline 2024-09-10T08:50:30.313631+0000) 2024-09-10T08:50:34.303 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:34 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:34.148+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:34.303 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:34 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:34.148+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:34.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:33 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:33.959+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.295502+0000 front 2024-09-10T08:50:04.295478+0000 (oldest deadline 2024-09-10T08:50:33.696113+0000) 2024-09-10T08:50:34.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:34 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:34.178+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:34.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:34 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:34.178+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:34.720 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:34 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:34.373+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:34.860 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:34 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:34.581+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:34.860 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:50:34 smithi111 podman[70179]: 2024-09-10 08:50:34.577394971 +0000 UTC m=+5.534776157 container died 75b1494d5a721055c33c8157df993d87e994c4b6cdec4cf1a00060cdaaeb1432 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-10T08:50:35.140 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:50:34 smithi111 podman[70179]: 2024-09-10 08:50:34.950822566 +0000 UTC m=+5.908203745 container cleanup 75b1494d5a721055c33c8157df993d87e994c4b6cdec4cf1a00060cdaaeb1432 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:50:35.141 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:50:34 smithi111 bash[70179]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-4 2024-09-10T08:50:35.220 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:34 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:34.864+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:35.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:35 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:35.005+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:58.612764+0000 front 2024-09-10T08:49:58.612747+0000 (oldest deadline 2024-09-10T08:50:23.912615+0000) 2024-09-10T08:50:35.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:35 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:35.005+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.613570+0000 front 2024-09-10T08:50:05.613653+0000 (oldest deadline 2024-09-10T08:50:30.313631+0000) 2024-09-10T08:50:35.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:34 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:34.995+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.295502+0000 front 2024-09-10T08:50:04.295478+0000 (oldest deadline 2024-09-10T08:50:33.696113+0000) 2024-09-10T08:50:35.303 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:35 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:35.103+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:35.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:35 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:35.103+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:35.544 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:50:35 smithi111 podman[70207]: 2024-09-10 08:50:35.219440971 +0000 UTC m=+0.637001997 container remove 75b1494d5a721055c33c8157df993d87e994c4b6cdec4cf1a00060cdaaeb1432 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:50:35.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:35 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:35.138+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:35.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:35 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:35.138+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:35.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:35 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:35.409+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:35.831 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:35 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:35.558+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:35.831 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:50:35 smithi111 podman[70299]: 2024-09-10 08:50:35.4682091 +0000 UTC m=+0.015598674 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:50:35.831 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:50:35 smithi111 podman[70299]: 2024-09-10 08:50:35.578699863 +0000 UTC m=+0.126089473 container create c9aff4b33ae57e453f974b64712497e5349cb7235281399e92683ec25d48443c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:50:35.831 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:50:35 smithi111 podman[70299]: 2024-09-10 08:50:35.8314374 +0000 UTC m=+0.378826972 container init c9aff4b33ae57e453f974b64712497e5349cb7235281399e92683ec25d48443c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T08:50:36.175 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:50:35 smithi111 podman[70299]: 2024-09-10 08:50:35.835005214 +0000 UTC m=+0.382394778 container start c9aff4b33ae57e453f974b64712497e5349cb7235281399e92683ec25d48443c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-10T08:50:36.175 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:50:35 smithi111 podman[70299]: 2024-09-10 08:50:35.897961756 +0000 UTC m=+0.445351322 container attach c9aff4b33ae57e453f974b64712497e5349cb7235281399e92683ec25d48443c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-10T08:50:36.175 INFO:journalctl@ceph.osd.4.smithi111.stdout:Sep 10 08:50:36 smithi111 podman[70299]: 2024-09-10 08:50:36.020648298 +0000 UTC m=+0.568037873 container died c9aff4b33ae57e453f974b64712497e5349cb7235281399e92683ec25d48443c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T08:50:36.220 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:35 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:35.827+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:36.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:35 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:35.956+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:58.612764+0000 front 2024-09-10T08:49:58.612747+0000 (oldest deadline 2024-09-10T08:50:23.912615+0000) 2024-09-10T08:50:36.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:35 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:35.956+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.613570+0000 front 2024-09-10T08:50:05.613653+0000 (oldest deadline 2024-09-10T08:50:30.313631+0000) 2024-09-10T08:50:36.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:36 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:36.017+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.295502+0000 front 2024-09-10T08:50:04.295478+0000 (oldest deadline 2024-09-10T08:50:33.696113+0000) 2024-09-10T08:50:36.303 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:36 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:36.142+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:36.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:36 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:36.142+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:36.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:36 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:36.172+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:36.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:36 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:36.172+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:36.634 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T08:50:36.673 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:50:36.674 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T08:50:36.674 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T08:50:36.674 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.5 2024-09-10T08:50:36.703 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:36 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:36.413+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:36.970 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:36 smithi149 systemd[1]: Stopping Ceph osd.5 for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:50:36.971 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:36 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:36.825+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:37.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:36 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:36.606+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:37.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:37 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:37.023+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.295502+0000 front 2024-09-10T08:50:04.295478+0000 (oldest deadline 2024-09-10T08:50:33.696113+0000) 2024-09-10T08:50:37.303 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:37 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:37.122+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:37.303 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:37 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:37.122+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:37.384 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:36 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:36.996+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:58.612764+0000 front 2024-09-10T08:49:58.612747+0000 (oldest deadline 2024-09-10T08:50:23.912615+0000) 2024-09-10T08:50:37.384 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:36 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:36.996+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.613570+0000 front 2024-09-10T08:50:05.613653+0000 (oldest deadline 2024-09-10T08:50:30.313631+0000) 2024-09-10T08:50:37.384 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:37 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:37.114+0000 7fd364cd5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:50:37.384 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:37 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:37.114+0000 7fd364cd5640 -1 osd.5 100 *** Got signal Terminated *** 2024-09-10T08:50:37.384 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:37 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:37.114+0000 7fd364cd5640 -1 osd.5 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:50:37.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:37 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:37.130+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:37.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:37 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:37.130+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:37.720 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:37 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:37.383+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:38.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:37 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:37.645+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:38.220 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:37 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:37.832+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:38.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:37 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:37.965+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:58.612764+0000 front 2024-09-10T08:49:58.612747+0000 (oldest deadline 2024-09-10T08:50:23.912615+0000) 2024-09-10T08:50:38.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:37 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:37.965+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.613570+0000 front 2024-09-10T08:50:05.613653+0000 (oldest deadline 2024-09-10T08:50:30.313631+0000) 2024-09-10T08:50:38.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:38 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:38.032+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.295502+0000 front 2024-09-10T08:50:04.295478+0000 (oldest deadline 2024-09-10T08:50:33.696113+0000) 2024-09-10T08:50:38.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:38 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:38.140+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:38.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:38 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:38.140+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:38.543 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:38 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:38.120+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:38.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:38 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:38.120+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:38.720 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:38 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:38.432+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:39.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:38 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:38.616+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:39.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:38 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:38.840+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:39.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:38 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:38.930+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:58.612764+0000 front 2024-09-10T08:49:58.612747+0000 (oldest deadline 2024-09-10T08:50:23.912615+0000) 2024-09-10T08:50:39.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:38 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:38.930+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.613570+0000 front 2024-09-10T08:50:05.613653+0000 (oldest deadline 2024-09-10T08:50:30.313631+0000) 2024-09-10T08:50:39.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:39 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:39.003+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.295502+0000 front 2024-09-10T08:50:04.295478+0000 (oldest deadline 2024-09-10T08:50:33.696113+0000) 2024-09-10T08:50:39.303 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:39 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:39.131+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:39.303 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:39 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:39.131+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:39.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:39 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:39.144+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:39.545 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:39 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:39.144+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:39.720 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:39 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:39.422+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:39.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:39 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:39.422+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:40.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:39 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:39.575+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:40.220 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:39 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:39.863+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:40.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:39 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:39.899+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:58.612764+0000 front 2024-09-10T08:49:58.612747+0000 (oldest deadline 2024-09-10T08:50:23.912615+0000) 2024-09-10T08:50:40.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:39 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:39.899+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.613570+0000 front 2024-09-10T08:50:05.613653+0000 (oldest deadline 2024-09-10T08:50:30.313631+0000) 2024-09-10T08:50:40.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:40 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:40.032+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.295502+0000 front 2024-09-10T08:50:04.295478+0000 (oldest deadline 2024-09-10T08:50:33.696113+0000) 2024-09-10T08:50:40.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:40 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:40.032+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.697001+0000 front 2024-09-10T08:50:13.696951+0000 (oldest deadline 2024-09-10T08:50:39.596652+0000) 2024-09-10T08:50:40.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:40 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:40.139+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:40.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:40 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:40.139+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:40.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:40 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:40.116+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:40.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:40 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:40.116+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:40.720 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:40 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:40.410+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:40.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:40 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:40.410+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:41.043 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:40 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:40.573+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:41.220 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:40 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:40.830+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:41.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:40 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:40.861+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:58.612764+0000 front 2024-09-10T08:49:58.612747+0000 (oldest deadline 2024-09-10T08:50:23.912615+0000) 2024-09-10T08:50:41.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:40 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:40.861+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.613570+0000 front 2024-09-10T08:50:05.613653+0000 (oldest deadline 2024-09-10T08:50:30.313631+0000) 2024-09-10T08:50:41.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:40 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:40.985+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.295502+0000 front 2024-09-10T08:50:04.295478+0000 (oldest deadline 2024-09-10T08:50:33.696113+0000) 2024-09-10T08:50:41.304 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:40 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:40.985+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.697001+0000 front 2024-09-10T08:50:13.696951+0000 (oldest deadline 2024-09-10T08:50:39.596652+0000) 2024-09-10T08:50:41.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:41 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:41.159+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:41.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:41 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:41.159+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:41.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:41 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:41.098+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:41.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:41 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:41.098+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:41.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:41 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:41.098+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:50:41.720 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:41 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:41.407+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:41.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:41 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:41.407+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:42.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:41 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:41.584+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:42.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:41 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:41.871+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:42.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:41 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:41.871+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:50:42.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:41 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:41.908+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:58.612764+0000 front 2024-09-10T08:49:58.612747+0000 (oldest deadline 2024-09-10T08:50:23.912615+0000) 2024-09-10T08:50:42.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:41 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:41.908+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.613570+0000 front 2024-09-10T08:50:05.613653+0000 (oldest deadline 2024-09-10T08:50:30.313631+0000) 2024-09-10T08:50:42.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:41 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5[47870]: 2024-09-10T08:50:41.908+0000 7fd3612eb640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:16.714828+0000 front 2024-09-10T08:50:16.714808+0000 (oldest deadline 2024-09-10T08:50:41.414972+0000) 2024-09-10T08:50:42.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:41 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:41.990+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.295502+0000 front 2024-09-10T08:50:04.295478+0000 (oldest deadline 2024-09-10T08:50:33.696113+0000) 2024-09-10T08:50:42.304 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:41 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:41.990+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.697001+0000 front 2024-09-10T08:50:13.696951+0000 (oldest deadline 2024-09-10T08:50:39.596652+0000) 2024-09-10T08:50:42.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:42.209+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:42.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:42 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:42.209+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:42.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:42 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:42.105+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:42.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:42 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:42.105+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:42.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:42 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:42.105+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:50:42.679 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:42 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:42.386+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:42.679 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:42 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:42.386+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:42.971 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:42 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:42.877+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:42.971 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:42 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:42.877+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:50:42.971 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:42 smithi149 podman[71779]: 2024-09-10 08:50:42.678778385 +0000 UTC m=+5.804572421 container died 0fdf25d0d4be99f1b3cf251cd496f530ffef2496ed4ce001d7383cb3d46fe7bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:50:43.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:42 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:42.558+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:43.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:42 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:42.559+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:16.115445+0000 front 2024-09-10T08:50:16.115398+0000 (oldest deadline 2024-09-10T08:50:42.015291+0000) 2024-09-10T08:50:43.194 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:43 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:43.015+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.295502+0000 front 2024-09-10T08:50:04.295478+0000 (oldest deadline 2024-09-10T08:50:33.696113+0000) 2024-09-10T08:50:43.194 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:43 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:43.015+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.697001+0000 front 2024-09-10T08:50:13.696951+0000 (oldest deadline 2024-09-10T08:50:39.596652+0000) 2024-09-10T08:50:43.203 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:43 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:43.192+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:43.203 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:43 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:43.192+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:43.316 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:43 smithi149 podman[71779]: 2024-09-10 08:50:43.081759331 +0000 UTC m=+6.207553357 container cleanup 0fdf25d0d4be99f1b3cf251cd496f530ffef2496ed4ce001d7383cb3d46fe7bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T08:50:43.316 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:43 smithi149 bash[71779]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5 2024-09-10T08:50:43.352 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:43 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:43.077+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:43.352 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:43 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:43.077+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:43.352 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:43 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:43.077+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:50:43.625 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:43 smithi149 podman[71796]: 2024-09-10 08:50:43.316433491 +0000 UTC m=+0.636549416 container remove 0fdf25d0d4be99f1b3cf251cd496f530ffef2496ed4ce001d7383cb3d46fe7bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:50:43.625 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:43 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:43.355+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:43.626 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:43 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:43.355+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:43.794 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:43 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:43.594+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:43.794 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:43 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:43.594+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:16.115445+0000 front 2024-09-10T08:50:16.115398+0000 (oldest deadline 2024-09-10T08:50:42.015291+0000) 2024-09-10T08:50:43.904 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:43 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:43.866+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:43.904 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:43 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:43.866+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:50:43.904 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:43 smithi149 podman[71883]: 2024-09-10 08:50:43.527841816 +0000 UTC m=+0.016066388 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:50:43.904 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:43 smithi149 podman[71883]: 2024-09-10 08:50:43.650721237 +0000 UTC m=+0.138945810 container create c6a46b10feb8a73b4f6d05f656e9cc18d53266ca8300a0e1674483dc1f2d6c2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-10T08:50:44.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:43 smithi149 podman[71883]: 2024-09-10 08:50:43.903856299 +0000 UTC m=+0.392080878 container init c6a46b10feb8a73b4f6d05f656e9cc18d53266ca8300a0e1674483dc1f2d6c2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:50:44.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:43 smithi149 podman[71883]: 2024-09-10 08:50:43.907587071 +0000 UTC m=+0.395811641 container start c6a46b10feb8a73b4f6d05f656e9cc18d53266ca8300a0e1674483dc1f2d6c2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:50:44.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:44 smithi149 podman[71883]: 2024-09-10 08:50:44.020233816 +0000 UTC m=+0.508458393 container attach c6a46b10feb8a73b4f6d05f656e9cc18d53266ca8300a0e1674483dc1f2d6c2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T08:50:44.221 INFO:journalctl@ceph.osd.5.smithi149.stdout:Sep 10 08:50:44 smithi149 podman[71883]: 2024-09-10 08:50:44.096486604 +0000 UTC m=+0.584711175 container died c6a46b10feb8a73b4f6d05f656e9cc18d53266ca8300a0e1674483dc1f2d6c2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-10T08:50:44.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:43 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:43.981+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.295502+0000 front 2024-09-10T08:50:04.295478+0000 (oldest deadline 2024-09-10T08:50:33.696113+0000) 2024-09-10T08:50:44.304 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:43 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:43.981+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.697001+0000 front 2024-09-10T08:50:13.696951+0000 (oldest deadline 2024-09-10T08:50:39.596652+0000) 2024-09-10T08:50:44.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:44 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:44.206+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:44.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:44 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:44.206+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:44.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:44 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:44.206+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:50:44.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:44 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:44.118+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:44.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:44 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:44.118+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:44.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:44 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:44.118+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:50:44.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:44 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:44.348+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:44.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:44 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:44.348+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:44.782 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T08:50:44.869 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:50:44.869 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T08:50:44.870 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T08:50:44.870 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.6 2024-09-10T08:50:45.045 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:44 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:44.562+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:45.045 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:44 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:44.562+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:16.115445+0000 front 2024-09-10T08:50:16.115398+0000 (oldest deadline 2024-09-10T08:50:42.015291+0000) 2024-09-10T08:50:45.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:44 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:44.867+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:45.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:44 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:44.867+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:50:45.288 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:45 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:45.163+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:45.288 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:45 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:45.163+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:45.289 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:45 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:45.163+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:50:45.289 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:44 smithi089 systemd[1]: Stopping Ceph osd.6 for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:50:45.289 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:44 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:44.974+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.295502+0000 front 2024-09-10T08:50:04.295478+0000 (oldest deadline 2024-09-10T08:50:33.696113+0000) 2024-09-10T08:50:45.289 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:44 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:44.974+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.697001+0000 front 2024-09-10T08:50:13.696951+0000 (oldest deadline 2024-09-10T08:50:39.596652+0000) 2024-09-10T08:50:45.289 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:45 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:45.287+0000 7eff4aa00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:50:45.289 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:45 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:45.287+0000 7eff4aa00640 -1 osd.6 100 *** Got signal Terminated *** 2024-09-10T08:50:45.289 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:45 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:45.287+0000 7eff4aa00640 -1 osd.6 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:50:45.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:45 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:45.076+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:45.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:45 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:45.076+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:45.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:45 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:45.076+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:50:45.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:45 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:45.341+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:45.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:45 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:45.341+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:46.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:45 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:45.544+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:46.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:45 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:45.544+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:16.115445+0000 front 2024-09-10T08:50:16.115398+0000 (oldest deadline 2024-09-10T08:50:42.015291+0000) 2024-09-10T08:50:46.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:45 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:45.860+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:46.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:45 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:45.860+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:50:46.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:46 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:46.007+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.295502+0000 front 2024-09-10T08:50:04.295478+0000 (oldest deadline 2024-09-10T08:50:33.696113+0000) 2024-09-10T08:50:46.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:46 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:46.007+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.697001+0000 front 2024-09-10T08:50:13.696951+0000 (oldest deadline 2024-09-10T08:50:39.596652+0000) 2024-09-10T08:50:46.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:46 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:46.149+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:46.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:46 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:46.149+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:46.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:46 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:46.149+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:50:46.526 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:46 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:46.065+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:46.527 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:46 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:46.065+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:46.527 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:46 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:46.065+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:50:46.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:46 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:46.371+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:46.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:46 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:46.371+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:46.794 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:46 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:46.525+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:46.794 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:46 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:46.525+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:16.115445+0000 front 2024-09-10T08:50:16.115398+0000 (oldest deadline 2024-09-10T08:50:42.015291+0000) 2024-09-10T08:50:47.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:46 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:46.872+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:47.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:46 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:46.872+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:50:47.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:46 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:46.971+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.295502+0000 front 2024-09-10T08:50:04.295478+0000 (oldest deadline 2024-09-10T08:50:33.696113+0000) 2024-09-10T08:50:47.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:46 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:46.971+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.697001+0000 front 2024-09-10T08:50:13.696951+0000 (oldest deadline 2024-09-10T08:50:39.596652+0000) 2024-09-10T08:50:47.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:47 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:47.172+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:47.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:47 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:47.172+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:47.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:47 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:47.172+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:50:47.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:47 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:47.069+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:47.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:47 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:47.069+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:47.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:47 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:47.069+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:50:47.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:47 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:47.069+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:50:47.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:47 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:47.402+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:47.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:47 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:47.402+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:48.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:47 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:47.559+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:48.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:47 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:47.559+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:16.115445+0000 front 2024-09-10T08:50:16.115398+0000 (oldest deadline 2024-09-10T08:50:42.015291+0000) 2024-09-10T08:50:48.220 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:47 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:47.893+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:48.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:47 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:47.893+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:50:48.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:47 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:47.951+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.295502+0000 front 2024-09-10T08:50:04.295478+0000 (oldest deadline 2024-09-10T08:50:33.696113+0000) 2024-09-10T08:50:48.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:47 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:47.951+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.697001+0000 front 2024-09-10T08:50:13.696951+0000 (oldest deadline 2024-09-10T08:50:39.596652+0000) 2024-09-10T08:50:48.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:48 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:48.169+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:48.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:48 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:48.169+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:48.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:48 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:48.169+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:50:48.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:48 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:48.052+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:48.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:48 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:48.052+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:48.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:48 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:48.052+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:50:48.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:48 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:48.052+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:50:48.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:48 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:48.412+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:48.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:48 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:48.412+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:49.043 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:48 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:48.571+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:49.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:48 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:48.571+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:16.115445+0000 front 2024-09-10T08:50:16.115398+0000 (oldest deadline 2024-09-10T08:50:42.015291+0000) 2024-09-10T08:50:49.220 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:48 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:48.920+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:49.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:48 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:48.920+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:50:49.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:48 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:48.920+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:22.540257+0000 front 2024-09-10T08:50:22.540140+0000 (oldest deadline 2024-09-10T08:50:48.439981+0000) 2024-09-10T08:50:49.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:48 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:48.938+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.295502+0000 front 2024-09-10T08:50:04.295478+0000 (oldest deadline 2024-09-10T08:50:33.696113+0000) 2024-09-10T08:50:49.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:48 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:48.938+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.697001+0000 front 2024-09-10T08:50:13.696951+0000 (oldest deadline 2024-09-10T08:50:39.596652+0000) 2024-09-10T08:50:49.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:49 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:49.135+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:49.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:49 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:49.135+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:49.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:49 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:49.135+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:50:49.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:49 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:49.135+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:50:49.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:49 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:49.068+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:49.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:49 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:49.068+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:49.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:49 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:49.068+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:50:49.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:49 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:49.068+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:50:49.720 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:49 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:49.372+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:49.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:49 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:49.372+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:49.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:49 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:49.372+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:50:50.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:49 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:49.571+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:50.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:49 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:49.571+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:16.115445+0000 front 2024-09-10T08:50:16.115398+0000 (oldest deadline 2024-09-10T08:50:42.015291+0000) 2024-09-10T08:50:50.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:49 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:49.875+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:50.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:49 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:49.875+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:50:50.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:49 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:49.875+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:22.540257+0000 front 2024-09-10T08:50:22.540140+0000 (oldest deadline 2024-09-10T08:50:48.439981+0000) 2024-09-10T08:50:50.305 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:49 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:49.966+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:04.295502+0000 front 2024-09-10T08:50:04.295478+0000 (oldest deadline 2024-09-10T08:50:33.696113+0000) 2024-09-10T08:50:50.305 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:49 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6[58018]: 2024-09-10T08:50:49.966+0000 7eff46400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.697001+0000 front 2024-09-10T08:50:13.696951+0000 (oldest deadline 2024-09-10T08:50:39.596652+0000) 2024-09-10T08:50:50.305 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:50 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:50.163+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:50.305 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:50 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:50.163+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:50.305 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:50 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:50.163+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:50:50.306 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:50 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:50.163+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:50:50.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:50 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:50.065+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:50.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:50 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:50.065+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:50.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:50 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:50.065+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:50:50.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:50 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:50.065+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:50:50.720 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:50 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:50.390+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:50.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:50 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:50.390+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:50.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:50 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:50.390+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:50:51.027 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:50 smithi089 podman[106678]: 2024-09-10 08:50:50.742981431 +0000 UTC m=+5.588835539 container died 31ff944bd93230394c022e39d6d9b9f0b3aaffba4a276811f02081b80774f64d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T08:50:51.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:50 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:50.599+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:51.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:50 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:50.599+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:16.115445+0000 front 2024-09-10T08:50:16.115398+0000 (oldest deadline 2024-09-10T08:50:42.015291+0000) 2024-09-10T08:50:51.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:50 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:50.923+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:51.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:50 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:50.923+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:50:51.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:50 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:50.923+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:22.540257+0000 front 2024-09-10T08:50:22.540140+0000 (oldest deadline 2024-09-10T08:50:48.439981+0000) 2024-09-10T08:50:51.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:51 smithi089 podman[106678]: 2024-09-10 08:50:51.143193118 +0000 UTC m=+5.989047224 container cleanup 31ff944bd93230394c022e39d6d9b9f0b3aaffba4a276811f02081b80774f64d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T08:50:51.303 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:51 smithi089 bash[106678]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6 2024-09-10T08:50:51.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:51 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:51.171+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:51.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:51 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:51.171+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:51.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:51 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:51.171+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:50:51.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:51 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:51.171+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:50:51.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:51 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:51.095+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:51.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:51 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:51.095+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:51.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:51 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:51.095+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:50:51.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:51 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:51.095+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:50:51.720 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:51 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:51.376+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:51.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:51 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:51.376+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:51.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:51 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:51.376+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:50:51.803 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:51 smithi089 podman[106695]: 2024-09-10 08:50:51.477851297 +0000 UTC m=+0.724160237 container remove 31ff944bd93230394c022e39d6d9b9f0b3aaffba4a276811f02081b80774f64d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T08:50:52.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:51 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:51.648+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:52.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:51 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:51.648+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:16.115445+0000 front 2024-09-10T08:50:16.115398+0000 (oldest deadline 2024-09-10T08:50:42.015291+0000) 2024-09-10T08:50:52.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:51 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:51.648+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.816909+0000 front 2024-09-10T08:50:24.816941+0000 (oldest deadline 2024-09-10T08:50:50.716725+0000) 2024-09-10T08:50:52.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:51 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:51.963+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:52.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:51 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:51.963+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:50:52.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:51 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:51.963+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:22.540257+0000 front 2024-09-10T08:50:22.540140+0000 (oldest deadline 2024-09-10T08:50:48.439981+0000) 2024-09-10T08:50:52.303 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:52 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:52.176+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:52.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:52 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:52.176+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:52.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:52 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:52.176+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:50:52.304 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:52 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:52.176+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:50:52.304 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:51 smithi089 podman[106779]: 2024-09-10 08:50:51.850744563 +0000 UTC m=+0.027228664 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:50:52.304 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:51 smithi089 podman[106779]: 2024-09-10 08:50:51.953814122 +0000 UTC m=+0.130298212 container create d3779f17d95695d998ddd1d61eb3b504d6bec2fa73c36e4176696d2762598008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-10T08:50:52.304 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:52 smithi089 podman[106779]: 2024-09-10 08:50:52.181124077 +0000 UTC m=+0.357608186 container init d3779f17d95695d998ddd1d61eb3b504d6bec2fa73c36e4176696d2762598008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T08:50:52.304 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:52 smithi089 podman[106779]: 2024-09-10 08:50:52.187984884 +0000 UTC m=+0.364468967 container start d3779f17d95695d998ddd1d61eb3b504d6bec2fa73c36e4176696d2762598008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-10T08:50:52.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:52 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:52.127+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:52.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:52 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:52.127+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:52.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:52 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:52.127+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:50:52.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:52 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:52.127+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:50:52.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:52 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:52.424+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:52.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:52 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:52.424+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:52.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:52 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:52.424+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:50:52.792 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:52 smithi089 podman[106779]: 2024-09-10 08:50:52.330989542 +0000 UTC m=+0.507473636 container attach d3779f17d95695d998ddd1d61eb3b504d6bec2fa73c36e4176696d2762598008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:50:52.792 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:52 smithi089 podman[106802]: 2024-09-10 08:50:52.489271627 +0000 UTC m=+0.026056744 container died d3779f17d95695d998ddd1d61eb3b504d6bec2fa73c36e4176696d2762598008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2) 2024-09-10T08:50:53.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:52 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:52.685+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:53.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:52 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:52.685+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:16.115445+0000 front 2024-09-10T08:50:16.115398+0000 (oldest deadline 2024-09-10T08:50:42.015291+0000) 2024-09-10T08:50:53.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:52 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:52.685+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.816909+0000 front 2024-09-10T08:50:24.816941+0000 (oldest deadline 2024-09-10T08:50:50.716725+0000) 2024-09-10T08:50:53.053 INFO:journalctl@ceph.osd.6.smithi089.stdout:Sep 10 08:50:53 smithi089 podman[106802]: 2024-09-10 08:50:53.042961083 +0000 UTC m=+0.579746201 container remove d3779f17d95695d998ddd1d61eb3b504d6bec2fa73c36e4176696d2762598008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:50:53.069 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T08:50:53.143 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:50:53.181 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T08:50:53.181 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T08:50:53.181 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.7 2024-09-10T08:50:53.449 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:52 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:52.994+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:53.449 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:52 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:52.994+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:50:53.449 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:52 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:52.994+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:22.540257+0000 front 2024-09-10T08:50:22.540140+0000 (oldest deadline 2024-09-10T08:50:48.439981+0000) 2024-09-10T08:50:53.481 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:53 smithi111 systemd[1]: Stopping Ceph osd.7 for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:50:53.481 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:53 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:53.176+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:53.481 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:53 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:53.176+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:53.481 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:53 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:53.176+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:50:53.482 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:53 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:53.176+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:50:53.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:53 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:53.147+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:53.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:53 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:53.147+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:53.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:53 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:53.147+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:50:53.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:53 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:53.147+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:50:53.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:53 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:53.447+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:53.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:53 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:53.447+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:53.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:53 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:53.447+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:50:53.794 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:53 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:53.480+0000 7f320f7b8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:50:53.794 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:53 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:53.480+0000 7f320f7b8640 -1 osd.7 100 *** Got signal Terminated *** 2024-09-10T08:50:53.794 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:53 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:53.480+0000 7f320f7b8640 -1 osd.7 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:50:53.794 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:53 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:53.669+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:53.794 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:53 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:53.669+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:16.115445+0000 front 2024-09-10T08:50:16.115398+0000 (oldest deadline 2024-09-10T08:50:42.015291+0000) 2024-09-10T08:50:53.794 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:53 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:53.669+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.816909+0000 front 2024-09-10T08:50:24.816941+0000 (oldest deadline 2024-09-10T08:50:50.716725+0000) 2024-09-10T08:50:54.470 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:54 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:54.008+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:54.471 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:54 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:54.008+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:50:54.471 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:54 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:54.008+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:22.540257+0000 front 2024-09-10T08:50:22.540140+0000 (oldest deadline 2024-09-10T08:50:48.439981+0000) 2024-09-10T08:50:54.543 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:54 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:54.181+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:54.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:54 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:54.181+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:54.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:54 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:54.181+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:50:54.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:54 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:54.181+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:50:54.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:54 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:54.136+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:54.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:54 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:54.136+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:54.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:54 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:54.136+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:50:54.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:54 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:54.136+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:50:54.970 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:54 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:54.493+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:54.971 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:54 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:54.493+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:54.971 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:54 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:54.493+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:50:55.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:54 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:54.645+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:55.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:54 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:54.645+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:16.115445+0000 front 2024-09-10T08:50:16.115398+0000 (oldest deadline 2024-09-10T08:50:42.015291+0000) 2024-09-10T08:50:55.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:54 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:54.645+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.816909+0000 front 2024-09-10T08:50:24.816941+0000 (oldest deadline 2024-09-10T08:50:50.716725+0000) 2024-09-10T08:50:55.470 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:54 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:54.986+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:55.471 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:54 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:54.986+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:50:55.471 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:54 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:54.986+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:22.540257+0000 front 2024-09-10T08:50:22.540140+0000 (oldest deadline 2024-09-10T08:50:48.439981+0000) 2024-09-10T08:50:55.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:55 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:55.182+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:55.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:55 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:55.182+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:55.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:55 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:55.182+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:50:55.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:55 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:55.182+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:50:55.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:55 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:55.182+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:50:55.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:55 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:55.138+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:55.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:55 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:55.138+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:55.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:55 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:55.138+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:50:55.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:55 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:55.138+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:50:55.971 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:55 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:55.481+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:55.971 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:55 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:55.481+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:55.971 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:55 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:55.481+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:50:56.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:55 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:55.671+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:56.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:55 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:55.671+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:16.115445+0000 front 2024-09-10T08:50:16.115398+0000 (oldest deadline 2024-09-10T08:50:42.015291+0000) 2024-09-10T08:50:56.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:55 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:55.671+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.816909+0000 front 2024-09-10T08:50:24.816941+0000 (oldest deadline 2024-09-10T08:50:50.716725+0000) 2024-09-10T08:50:56.471 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:56 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:56.024+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:56.471 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:56 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:56.024+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:50:56.471 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:56 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:56.024+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:22.540257+0000 front 2024-09-10T08:50:22.540140+0000 (oldest deadline 2024-09-10T08:50:48.439981+0000) 2024-09-10T08:50:56.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:56 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:56.160+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:56.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:56 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:56.160+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:56.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:56 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:56.160+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:50:56.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:56 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:56.160+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:50:56.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:56 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:56.160+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:50:56.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:56 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:56.169+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:56.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:56 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:56.169+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:56.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:56 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:56.169+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:50:56.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:56 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:56.169+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:50:56.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:56 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:56.169+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:30.261418+0000 front 2024-09-10T08:50:30.261334+0000 (oldest deadline 2024-09-10T08:50:55.561371+0000) 2024-09-10T08:50:56.970 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:56 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:56.514+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:56.971 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:56 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:56.514+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:56.971 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:56 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:56.514+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:50:57.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:56 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:56.655+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:57.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:56 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:56.655+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:16.115445+0000 front 2024-09-10T08:50:16.115398+0000 (oldest deadline 2024-09-10T08:50:42.015291+0000) 2024-09-10T08:50:57.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:56 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:56.655+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.816909+0000 front 2024-09-10T08:50:24.816941+0000 (oldest deadline 2024-09-10T08:50:50.716725+0000) 2024-09-10T08:50:57.470 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:57 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:57.004+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:57.471 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:57 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:57.005+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:50:57.471 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:57 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:57.005+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:22.540257+0000 front 2024-09-10T08:50:22.540140+0000 (oldest deadline 2024-09-10T08:50:48.439981+0000) 2024-09-10T08:50:57.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:57 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:57.177+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:57.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:57 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:57.177+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:57.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:57 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:57.177+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:50:57.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:57 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:57.177+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:50:57.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:57 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:57.177+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:50:57.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:57 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:57.150+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:57.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:57 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:57.150+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:57.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:57 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:57.150+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:50:57.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:57 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:57.150+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:50:57.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:57 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:57.150+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:30.261418+0000 front 2024-09-10T08:50:30.261334+0000 (oldest deadline 2024-09-10T08:50:55.561371+0000) 2024-09-10T08:50:57.970 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:57 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:57.513+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:57.971 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:57 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:57.513+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:57.971 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:57 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:57.513+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:50:57.971 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:57 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:57.513+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:50:58.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:57 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:57.667+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.213797+0000 front 2024-09-10T08:50:06.213889+0000 (oldest deadline 2024-09-10T08:50:30.913891+0000) 2024-09-10T08:50:58.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:57 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:57.667+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:16.115445+0000 front 2024-09-10T08:50:16.115398+0000 (oldest deadline 2024-09-10T08:50:42.015291+0000) 2024-09-10T08:50:58.044 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:57 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7[53122]: 2024-09-10T08:50:57.667+0000 7f320bdce640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.816909+0000 front 2024-09-10T08:50:24.816941+0000 (oldest deadline 2024-09-10T08:50:50.716725+0000) 2024-09-10T08:50:58.470 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:58 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:58.020+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:58.471 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:58 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:58.020+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:50:58.471 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:58 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:58.020+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:22.540257+0000 front 2024-09-10T08:50:22.540140+0000 (oldest deadline 2024-09-10T08:50:48.439981+0000) 2024-09-10T08:50:58.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:58 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:58.178+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:58.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:58 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:58.178+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:58.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:58 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:58.178+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:50:58.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:58 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:58.178+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:50:58.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:58 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:58.178+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:50:58.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:58 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:58.138+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:58.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:58 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:58.138+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:58.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:58 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:58.138+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:50:58.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:58 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:58.138+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:50:58.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:58 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:58.138+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:30.261418+0000 front 2024-09-10T08:50:30.261334+0000 (oldest deadline 2024-09-10T08:50:55.561371+0000) 2024-09-10T08:50:58.971 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:58 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:58.496+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:58.971 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:58 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:58.496+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:58.971 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:58 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:58.496+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:50:58.971 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:58 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:58.496+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:50:59.183 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:58 smithi111 podman[70400]: 2024-09-10 08:50:58.880030841 +0000 UTC m=+5.498073486 container died 922f7f563448a4a5fb050c55c4d8b02901c2b27576a4a17f0a700a4d7c68f939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-10T08:50:59.454 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:58 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:58.978+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:50:59.454 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:58 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:58.978+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:50:59.454 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:50:58 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:50:58.978+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:22.540257+0000 front 2024-09-10T08:50:22.540140+0000 (oldest deadline 2024-09-10T08:50:48.439981+0000) 2024-09-10T08:50:59.481 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:59 smithi111 podman[70400]: 2024-09-10 08:50:59.271817417 +0000 UTC m=+5.889860061 container cleanup 922f7f563448a4a5fb050c55c4d8b02901c2b27576a4a17f0a700a4d7c68f939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-10T08:50:59.481 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:59 smithi111 bash[70400]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7 2024-09-10T08:50:59.481 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:59 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:59.185+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:50:59.481 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:59 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:59.185+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:50:59.481 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:59 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:59.185+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:50:59.482 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:59 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:59.185+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:50:59.482 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:50:59 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:50:59.185+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:50:59.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:59 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:59.167+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:50:59.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:59 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:59.167+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:50:59.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:59 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:59.167+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:50:59.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:59 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:59.167+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:50:59.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:50:59 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:50:59.167+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:30.261418+0000 front 2024-09-10T08:50:30.261334+0000 (oldest deadline 2024-09-10T08:50:55.561371+0000) 2024-09-10T08:50:59.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:59 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:59.454+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:50:59.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:59 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:59.454+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:50:59.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:59 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:59.454+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:50:59.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:50:59 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:50:59.454+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:50:59.793 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:59 smithi111 podman[70417]: 2024-09-10 08:50:59.481130715 +0000 UTC m=+0.598175798 container remove 922f7f563448a4a5fb050c55c4d8b02901c2b27576a4a17f0a700a4d7c68f939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T08:51:00.101 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:59 smithi111 podman[70506]: 2024-09-10 08:50:59.714794364 +0000 UTC m=+0.015843028 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:51:00.101 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:50:59 smithi111 podman[70506]: 2024-09-10 08:50:59.865822484 +0000 UTC m=+0.166871173 container create da2351b97a4e21b17bdb1f24b1edbafdf7a3c01f52fb40924778afa8e00a0970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7-deactivate, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:51:00.101 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:51:00 smithi111 podman[70506]: 2024-09-10 08:51:00.101494496 +0000 UTC m=+0.402543143 container init da2351b97a4e21b17bdb1f24b1edbafdf7a3c01f52fb40924778afa8e00a0970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-10T08:51:00.416 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:00 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:00.012+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:51:00.416 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:00 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:00.012+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:51:00.416 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:00 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:00.012+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:22.540257+0000 front 2024-09-10T08:50:22.540140+0000 (oldest deadline 2024-09-10T08:50:48.439981+0000) 2024-09-10T08:51:00.417 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:00 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:00.012+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:33.741034+0000 front 2024-09-10T08:50:33.741050+0000 (oldest deadline 2024-09-10T08:50:59.041069+0000) 2024-09-10T08:51:00.544 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:51:00 smithi111 podman[70506]: 2024-09-10 08:51:00.104891727 +0000 UTC m=+0.405940373 container start da2351b97a4e21b17bdb1f24b1edbafdf7a3c01f52fb40924778afa8e00a0970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7-deactivate, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-10T08:51:00.544 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:51:00 smithi111 podman[70506]: 2024-09-10 08:51:00.18479786 +0000 UTC m=+0.485846520 container attach da2351b97a4e21b17bdb1f24b1edbafdf7a3c01f52fb40924778afa8e00a0970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:51:00.544 INFO:journalctl@ceph.osd.7.smithi111.stdout:Sep 10 08:51:00 smithi111 podman[70506]: 2024-09-10 08:51:00.294190272 +0000 UTC m=+0.595238918 container died da2351b97a4e21b17bdb1f24b1edbafdf7a3c01f52fb40924778afa8e00a0970 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-7-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T08:51:00.545 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:00 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:00.182+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:00.545 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:00 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:00.182+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:00.545 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:00 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:00.182+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:00.545 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:00 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:00.182+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:00.545 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:00 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:00.182+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:00.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:00 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:00.181+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:51:00.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:00 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:00.181+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:51:00.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:00 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:00.181+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:51:00.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:00 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:00.181+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:51:00.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:00 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:00.181+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:30.261418+0000 front 2024-09-10T08:50:30.261334+0000 (oldest deadline 2024-09-10T08:50:55.561371+0000) 2024-09-10T08:51:00.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:00 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:00.415+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:00.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:00 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:00.415+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:00.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:00 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:00.415+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:00.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:00 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:00.415+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:00.925 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T08:51:01.003 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:51:01.003 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T08:51:01.003 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-10T08:51:01.003 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.8 2024-09-10T08:51:01.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:00 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:00.968+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:51:01.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:00 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:00.968+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:51:01.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:00 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:00.968+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:22.540257+0000 front 2024-09-10T08:50:22.540140+0000 (oldest deadline 2024-09-10T08:50:48.439981+0000) 2024-09-10T08:51:01.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:00 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:00.968+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:33.741034+0000 front 2024-09-10T08:50:33.741050+0000 (oldest deadline 2024-09-10T08:50:59.041069+0000) 2024-09-10T08:51:01.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:01 smithi149 systemd[1]: Stopping Ceph osd.8 for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:51:01.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:01 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:01.212+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:01.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:01 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:01.212+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:01.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:01 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:01.212+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:01.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:01 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:01.212+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:01.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:01 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:01.212+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:01.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:01 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:01.166+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:51:01.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:01 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:01.166+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:51:01.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:01 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:01.166+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:51:01.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:01 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:01.166+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:51:01.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:01 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:01.166+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:30.261418+0000 front 2024-09-10T08:50:30.261334+0000 (oldest deadline 2024-09-10T08:50:55.561371+0000) 2024-09-10T08:51:01.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:01 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:01.383+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:01.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:01 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:01.383+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:01.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:01 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:01.383+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:01.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:01 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:01.383+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:01.722 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:01 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:01.317+0000 7fe0f258c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:51:01.722 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:01 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:01.318+0000 7fe0f258c640 -1 osd.8 100 *** Got signal Terminated *** 2024-09-10T08:51:01.722 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:01 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:01.318+0000 7fe0f258c640 -1 osd.8 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:51:02.398 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:02 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:02.009+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:51:02.398 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:02 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:02.009+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:51:02.398 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:02 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:02.009+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:22.540257+0000 front 2024-09-10T08:50:22.540140+0000 (oldest deadline 2024-09-10T08:50:48.439981+0000) 2024-09-10T08:51:02.398 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:02 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:02.009+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:33.741034+0000 front 2024-09-10T08:50:33.741050+0000 (oldest deadline 2024-09-10T08:50:59.041069+0000) 2024-09-10T08:51:02.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:02 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:02.166+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:02.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:02 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:02.166+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:02.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:02 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:02.166+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:02.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:02 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:02.166+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:02.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:02 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:02.166+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:02.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:02 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:02.179+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:51:02.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:02 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:02.179+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:51:02.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:02 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:02.179+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:51:02.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:02 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:02.179+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:51:02.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:02 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:02.179+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:30.261418+0000 front 2024-09-10T08:50:30.261334+0000 (oldest deadline 2024-09-10T08:50:55.561371+0000) 2024-09-10T08:51:02.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:02 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:02.398+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:02.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:02 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:02.398+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:02.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:02 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:02.398+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:02.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:02 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:02.398+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:03.361 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:03 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:03.000+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:51:03.362 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:03 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:03.000+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:51:03.362 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:03 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:03.000+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:22.540257+0000 front 2024-09-10T08:50:22.540140+0000 (oldest deadline 2024-09-10T08:50:48.439981+0000) 2024-09-10T08:51:03.362 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:03 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:03.000+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:33.741034+0000 front 2024-09-10T08:50:33.741050+0000 (oldest deadline 2024-09-10T08:50:59.041069+0000) 2024-09-10T08:51:03.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:03 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:03.158+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:03.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:03 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:03.158+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:03.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:03 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:03.158+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:03.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:03 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:03.158+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:03.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:03 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:03.158+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:03.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:03 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:03.135+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:51:03.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:03 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:03.135+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:51:03.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:03 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:03.135+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:51:03.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:03 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:03.135+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:51:03.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:03 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:03.135+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:30.261418+0000 front 2024-09-10T08:50:30.261334+0000 (oldest deadline 2024-09-10T08:50:55.561371+0000) 2024-09-10T08:51:03.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:03 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:03.361+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:03.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:03 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:03.361+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:03.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:03 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:03.361+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:03.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:03 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:03.361+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:04.381 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:04 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:04.030+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:51:04.381 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:04 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:04.030+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:51:04.381 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:04 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:04.030+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:22.540257+0000 front 2024-09-10T08:50:22.540140+0000 (oldest deadline 2024-09-10T08:50:48.439981+0000) 2024-09-10T08:51:04.381 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:04 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:04.030+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:33.741034+0000 front 2024-09-10T08:50:33.741050+0000 (oldest deadline 2024-09-10T08:50:59.041069+0000) 2024-09-10T08:51:04.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:04 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:04.185+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:04.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:04 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:04.185+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:04.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:04 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:04.185+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:04.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:04 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:04.185+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:04.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:04 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:04.185+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:04.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:04 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:04.176+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:51:04.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:04 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:04.176+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:51:04.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:04 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:04.176+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:51:04.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:04 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:04.176+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:51:04.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:04 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:04.176+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:30.261418+0000 front 2024-09-10T08:50:30.261334+0000 (oldest deadline 2024-09-10T08:50:55.561371+0000) 2024-09-10T08:51:04.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:04 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:04.176+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:37.862412+0000 front 2024-09-10T08:50:37.862655+0000 (oldest deadline 2024-09-10T08:51:03.162224+0000) 2024-09-10T08:51:04.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:04 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:04.381+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:04.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:04 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:04.381+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:04.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:04 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:04.381+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:04.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:04 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:04.381+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:05.379 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:05 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:05.004+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:51:05.380 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:05 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:05.004+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:51:05.380 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:05 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:05.004+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:22.540257+0000 front 2024-09-10T08:50:22.540140+0000 (oldest deadline 2024-09-10T08:50:48.439981+0000) 2024-09-10T08:51:05.380 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:05 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:05.004+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:33.741034+0000 front 2024-09-10T08:50:33.741050+0000 (oldest deadline 2024-09-10T08:50:59.041069+0000) 2024-09-10T08:51:05.380 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:05 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:05.004+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:40.642496+0000 front 2024-09-10T08:50:40.642549+0000 (oldest deadline 2024-09-10T08:51:04.742669+0000) 2024-09-10T08:51:05.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:05 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:05.222+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:05.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:05 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:05.222+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:05.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:05 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:05.222+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:05.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:05 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:05.222+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:05.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:05 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:05.222+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:05.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:05 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:05.190+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:51:05.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:05 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:05.190+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:51:05.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:05 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:05.190+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:51:05.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:05 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:05.190+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:51:05.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:05 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:05.190+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:30.261418+0000 front 2024-09-10T08:50:30.261334+0000 (oldest deadline 2024-09-10T08:50:55.561371+0000) 2024-09-10T08:51:05.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:05 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:05.190+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:37.862412+0000 front 2024-09-10T08:50:37.862655+0000 (oldest deadline 2024-09-10T08:51:03.162224+0000) 2024-09-10T08:51:05.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:05 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:05.378+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:05.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:05 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:05.378+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:05.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:05 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:05.378+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:05.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:05 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:05.378+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:06.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:05 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:05.958+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:06.936983+0000 front 2024-09-10T08:50:06.937046+0000 (oldest deadline 2024-09-10T08:50:32.836823+0000) 2024-09-10T08:51:06.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:05 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:05.958+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:15.638290+0000 front 2024-09-10T08:50:15.638362+0000 (oldest deadline 2024-09-10T08:50:41.538491+0000) 2024-09-10T08:51:06.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:05 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:05.958+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:22.540257+0000 front 2024-09-10T08:50:22.540140+0000 (oldest deadline 2024-09-10T08:50:48.439981+0000) 2024-09-10T08:51:06.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:05 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:05.958+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:33.741034+0000 front 2024-09-10T08:50:33.741050+0000 (oldest deadline 2024-09-10T08:50:59.041069+0000) 2024-09-10T08:51:06.221 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:05 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8[54336]: 2024-09-10T08:51:05.958+0000 7fe0eab9a640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:40.642496+0000 front 2024-09-10T08:50:40.642549+0000 (oldest deadline 2024-09-10T08:51:04.742669+0000) 2024-09-10T08:51:06.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:06 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:06.265+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:06.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:06 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:06.265+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:06.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:06 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:06.265+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:06.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:06 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:06.265+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:06.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:06 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:06.265+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:06.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:06 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:06.265+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:40.729879+0000 front 2024-09-10T08:50:40.729980+0000 (oldest deadline 2024-09-10T08:51:06.029805+0000) 2024-09-10T08:51:06.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:06 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:06.187+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:51:06.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:06 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:06.187+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:51:06.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:06 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:06.187+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:51:06.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:06 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:06.187+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:51:06.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:06 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:06.187+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:30.261418+0000 front 2024-09-10T08:50:30.261334+0000 (oldest deadline 2024-09-10T08:50:55.561371+0000) 2024-09-10T08:51:06.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:06 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:06.187+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:37.862412+0000 front 2024-09-10T08:50:37.862655+0000 (oldest deadline 2024-09-10T08:51:03.162224+0000) 2024-09-10T08:51:06.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:06 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:06.344+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:06.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:06 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:06.344+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:06.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:06 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:06.344+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:06.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:06 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:06.344+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:07.012 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:06 smithi149 podman[71987]: 2024-09-10 08:51:06.734802448 +0000 UTC m=+5.540116473 container died 000059b9f303aa8ef0d93f7b66a5896787113cec4d91bb38070733afab304add (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2) 2024-09-10T08:51:07.344 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:07 smithi149 podman[71987]: 2024-09-10 08:51:07.101326824 +0000 UTC m=+5.906640839 container cleanup 000059b9f303aa8ef0d93f7b66a5896787113cec4d91bb38070733afab304add (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:51:07.344 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:07 smithi149 bash[71987]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8 2024-09-10T08:51:07.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:07 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:07.236+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:51:07.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:07 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:07.236+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:51:07.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:07 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:07.236+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:51:07.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:07 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:07.236+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:51:07.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:07 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:07.236+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:30.261418+0000 front 2024-09-10T08:50:30.261334+0000 (oldest deadline 2024-09-10T08:50:55.561371+0000) 2024-09-10T08:51:07.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:07 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:07.236+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:37.862412+0000 front 2024-09-10T08:50:37.862655+0000 (oldest deadline 2024-09-10T08:51:03.162224+0000) 2024-09-10T08:51:07.686 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:07 smithi149 podman[72002]: 2024-09-10 08:51:07.377538795 +0000 UTC m=+0.643417236 container remove 000059b9f303aa8ef0d93f7b66a5896787113cec4d91bb38070733afab304add (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True) 2024-09-10T08:51:07.687 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:07 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:07.342+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:07.687 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:07 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:07.342+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:07.687 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:07 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:07.342+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:07.687 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:07 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:07.342+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:07.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:07 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:07.302+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:07.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:07 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:07.302+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:07.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:07 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:07.302+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:07.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:07 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:07.302+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:07.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:07 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:07.302+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:07.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:07 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:07.302+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:40.729879+0000 front 2024-09-10T08:50:40.729980+0000 (oldest deadline 2024-09-10T08:51:06.029805+0000) 2024-09-10T08:51:07.971 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:07 smithi149 podman[72091]: 2024-09-10 08:51:07.589180544 +0000 UTC m=+0.015958518 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:51:07.971 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:07 smithi149 podman[72091]: 2024-09-10 08:51:07.711600327 +0000 UTC m=+0.138378288 container create 02902132ec68312550776b0439c3bdce7aea59c56e411e7aaf13effdd152075b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-10T08:51:07.971 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:07 smithi149 podman[72091]: 2024-09-10 08:51:07.922490842 +0000 UTC m=+0.349268814 container init 02902132ec68312550776b0439c3bdce7aea59c56e411e7aaf13effdd152075b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:51:07.971 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:07 smithi149 podman[72091]: 2024-09-10 08:51:07.926174075 +0000 UTC m=+0.352952037 container start 02902132ec68312550776b0439c3bdce7aea59c56e411e7aaf13effdd152075b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:51:08.352 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:08 smithi149 podman[72091]: 2024-09-10 08:51:08.022342849 +0000 UTC m=+0.449120820 container attach 02902132ec68312550776b0439c3bdce7aea59c56e411e7aaf13effdd152075b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2) 2024-09-10T08:51:08.352 INFO:journalctl@ceph.osd.8.smithi149.stdout:Sep 10 08:51:08 smithi149 podman[72091]: 2024-09-10 08:51:08.117849044 +0000 UTC m=+0.544627016 container died 02902132ec68312550776b0439c3bdce7aea59c56e411e7aaf13effdd152075b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T08:51:08.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:08 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:08.285+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:08.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:08 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:08.285+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:08.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:08 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:08.285+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:08.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:08 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:08.285+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:08.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:08 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:08.285+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:08.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:08 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:08.285+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:40.729879+0000 front 2024-09-10T08:50:40.729980+0000 (oldest deadline 2024-09-10T08:51:06.029805+0000) 2024-09-10T08:51:08.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:08 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:08.276+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:51:08.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:08 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:08.276+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:51:08.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:08 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:08.276+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:51:08.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:08 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:08.276+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:51:08.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:08 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:08.276+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:30.261418+0000 front 2024-09-10T08:50:30.261334+0000 (oldest deadline 2024-09-10T08:50:55.561371+0000) 2024-09-10T08:51:08.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:08 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:08.276+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:37.862412+0000 front 2024-09-10T08:50:37.862655+0000 (oldest deadline 2024-09-10T08:51:03.162224+0000) 2024-09-10T08:51:08.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:08 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:08.350+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:08.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:08 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:08.350+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:08.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:08 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:08.350+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:08.722 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:08 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:08.350+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:08.722 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:08 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:08.350+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:08.736 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.8.service' 2024-09-10T08:51:08.776 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:51:08.777 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-10T08:51:08.777 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-10T08:51:08.778 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.9 2024-09-10T08:51:09.173 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:08 smithi089 systemd[1]: Stopping Ceph osd.9 for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:51:09.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:09 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:09.173+0000 7f3aa5400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:51:09.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:09 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:09.173+0000 7f3aa5400640 -1 osd.9 100 *** Got signal Terminated *** 2024-09-10T08:51:09.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:09 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:09.173+0000 7f3aa5400640 -1 osd.9 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:51:09.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:09 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:09.255+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:51:09.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:09 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:09.255+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:51:09.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:09 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:09.255+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:51:09.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:09 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:09.255+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:51:09.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:09 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:09.255+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:30.261418+0000 front 2024-09-10T08:50:30.261334+0000 (oldest deadline 2024-09-10T08:50:55.561371+0000) 2024-09-10T08:51:09.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:09 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:09.255+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:37.862412+0000 front 2024-09-10T08:50:37.862655+0000 (oldest deadline 2024-09-10T08:51:03.162224+0000) 2024-09-10T08:51:09.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:09 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:09.364+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:09.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:09 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:09.364+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:09.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:09 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:09.364+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:09.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:09 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:09.364+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:09.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:09 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:09.364+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:09.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:09 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:09.326+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:09.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:09 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:09.326+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:09.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:09 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:09.326+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:09.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:09 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:09.326+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:09.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:09 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:09.326+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:09.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:09 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:09.326+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:40.729879+0000 front 2024-09-10T08:50:40.729980+0000 (oldest deadline 2024-09-10T08:51:06.029805+0000) 2024-09-10T08:51:10.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:10 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:10.243+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:51:10.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:10 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:10.243+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:51:10.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:10 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:10.243+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:51:10.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:10 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:10.243+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:51:10.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:10 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:10.243+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:30.261418+0000 front 2024-09-10T08:50:30.261334+0000 (oldest deadline 2024-09-10T08:50:55.561371+0000) 2024-09-10T08:51:10.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:10 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:10.243+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:37.862412+0000 front 2024-09-10T08:50:37.862655+0000 (oldest deadline 2024-09-10T08:51:03.162224+0000) 2024-09-10T08:51:10.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:10 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:10.392+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:10.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:10 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:10.392+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:10.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:10 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:10.392+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:10.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:10 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:10.392+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:10.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:10 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:10.392+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:10.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:10 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:10.321+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:10.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:10 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:10.321+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:10.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:10 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:10.321+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:10.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:10 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:10.321+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:10.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:10 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:10.321+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:10.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:10 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:10.321+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:40.729879+0000 front 2024-09-10T08:50:40.729980+0000 (oldest deadline 2024-09-10T08:51:06.029805+0000) 2024-09-10T08:51:11.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:11 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:11.216+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:51:11.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:11 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:11.216+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:51:11.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:11 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:11.216+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:51:11.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:11 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:11.216+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:51:11.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:11 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:11.216+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:30.261418+0000 front 2024-09-10T08:50:30.261334+0000 (oldest deadline 2024-09-10T08:50:55.561371+0000) 2024-09-10T08:51:11.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:11 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:11.216+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:37.862412+0000 front 2024-09-10T08:50:37.862655+0000 (oldest deadline 2024-09-10T08:51:03.162224+0000) 2024-09-10T08:51:11.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:11 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:11.350+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:11.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:11 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:11.350+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:11.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:11 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:11.350+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:11.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:11 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:11.350+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:11.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:11 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:11.350+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:11.725 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:11 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:11.303+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:11.725 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:11 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:11.303+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:11.725 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:11 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:11.303+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:11.725 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:11 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:11.303+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:11.725 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:11 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:11.303+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:11.725 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:11 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:11.303+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:40.729879+0000 front 2024-09-10T08:50:40.729980+0000 (oldest deadline 2024-09-10T08:51:06.029805+0000) 2024-09-10T08:51:12.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:12 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:12.272+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:12.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:12 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:12.272+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:12.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:12 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:12.272+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:12.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:12 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:12.272+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:12.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:12 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:12.272+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:12.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:12 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:12.272+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:40.729879+0000 front 2024-09-10T08:50:40.729980+0000 (oldest deadline 2024-09-10T08:51:06.029805+0000) 2024-09-10T08:51:12.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:12 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:12.207+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:51:12.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:12 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:12.207+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:51:12.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:12 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:12.207+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:51:12.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:12 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:12.207+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:51:12.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:12 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:12.207+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:30.261418+0000 front 2024-09-10T08:50:30.261334+0000 (oldest deadline 2024-09-10T08:50:55.561371+0000) 2024-09-10T08:51:12.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:12 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:12.207+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:37.862412+0000 front 2024-09-10T08:50:37.862655+0000 (oldest deadline 2024-09-10T08:51:03.162224+0000) 2024-09-10T08:51:12.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:12 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:12.335+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:12.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:12 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:12.335+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:12.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:12 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:12.335+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:12.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:12 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:12.335+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:12.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:12 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:12.335+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:12.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:12 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:12.335+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:47.546151+0000 front 2024-09-10T08:50:47.546107+0000 (oldest deadline 2024-09-10T08:51:11.646023+0000) 2024-09-10T08:51:13.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:13 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:13.273+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:13.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:13 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:13.273+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:13.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:13 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:13.273+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:13.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:13 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:13.273+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:13.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:13 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:13.273+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:13.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:13 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:13.273+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:40.729879+0000 front 2024-09-10T08:50:40.729980+0000 (oldest deadline 2024-09-10T08:51:06.029805+0000) 2024-09-10T08:51:13.544 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:13 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:13.273+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:48.831656+0000 front 2024-09-10T08:50:48.831720+0000 (oldest deadline 2024-09-10T08:51:12.931391+0000) 2024-09-10T08:51:13.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:13 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:13.197+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:51:13.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:13 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:13.197+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:51:13.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:13 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:13.197+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:51:13.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:13 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:13.197+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:51:13.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:13 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:13.197+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:30.261418+0000 front 2024-09-10T08:50:30.261334+0000 (oldest deadline 2024-09-10T08:50:55.561371+0000) 2024-09-10T08:51:13.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:13 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:13.197+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:37.862412+0000 front 2024-09-10T08:50:37.862655+0000 (oldest deadline 2024-09-10T08:51:03.162224+0000) 2024-09-10T08:51:13.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:13 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:13.328+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:13.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:13 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:13.328+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:13.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:13 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:13.328+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:13.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:13 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:13.328+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:13.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:13 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:13.328+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:13.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:13 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:13.328+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:47.546151+0000 front 2024-09-10T08:50:47.546107+0000 (oldest deadline 2024-09-10T08:51:11.646023+0000) 2024-09-10T08:51:14.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:14 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:14.159+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:50:00.555973+0000 front 2024-09-10T08:50:00.556003+0000 (oldest deadline 2024-09-10T08:50:25.255954+0000) 2024-09-10T08:51:14.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:14 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:14.159+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:08.157334+0000 front 2024-09-10T08:50:08.157293+0000 (oldest deadline 2024-09-10T08:50:31.657091+0000) 2024-09-10T08:51:14.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:14 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:14.159+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.358269+0000 front 2024-09-10T08:50:13.358292+0000 (oldest deadline 2024-09-10T08:50:43.358924+0000) 2024-09-10T08:51:14.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:14 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:14.159+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.359472+0000 front 2024-09-10T08:50:23.359488+0000 (oldest deadline 2024-09-10T08:50:48.560070+0000) 2024-09-10T08:51:14.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:14 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:14.159+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:30.261418+0000 front 2024-09-10T08:50:30.261334+0000 (oldest deadline 2024-09-10T08:50:55.561371+0000) 2024-09-10T08:51:14.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:14 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9[65023]: 2024-09-10T08:51:14.159+0000 7f3aa0e00640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:37.862412+0000 front 2024-09-10T08:50:37.862655+0000 (oldest deadline 2024-09-10T08:51:03.162224+0000) 2024-09-10T08:51:14.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:14 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:14.322+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:14.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:14 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:14.322+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:14.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:14 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:14.322+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:14.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:14 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:14.322+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:14.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:14 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:14.322+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:14.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:14 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:14.322+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:47.546151+0000 front 2024-09-10T08:50:47.546107+0000 (oldest deadline 2024-09-10T08:51:11.646023+0000) 2024-09-10T08:51:14.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:14 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:14.300+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:14.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:14 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:14.300+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:14.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:14 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:14.300+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:14.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:14 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:14.300+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:14.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:14 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:14.300+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:14.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:14 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:14.300+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:40.729879+0000 front 2024-09-10T08:50:40.729980+0000 (oldest deadline 2024-09-10T08:51:06.029805+0000) 2024-09-10T08:51:14.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:14 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:14.300+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:48.831656+0000 front 2024-09-10T08:50:48.831720+0000 (oldest deadline 2024-09-10T08:51:12.931391+0000) 2024-09-10T08:51:15.112 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:14 smithi089 podman[106881]: 2024-09-10 08:51:14.837110196 +0000 UTC m=+5.780955098 container died 197f27152e486484e5c8a222e6a684598ac6a3711b830afeb7422f864659619f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T08:51:15.553 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:15 smithi089 podman[106881]: 2024-09-10 08:51:15.336020667 +0000 UTC m=+6.279865569 container cleanup 197f27152e486484e5c8a222e6a684598ac6a3711b830afeb7422f864659619f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:51:15.554 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:15 smithi089 bash[106881]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9 2024-09-10T08:51:15.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:15 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:15.292+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:15.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:15 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:15.292+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:15.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:15 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:15.292+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:15.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:15 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:15.292+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:15.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:15 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:15.292+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:15.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:15 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:15.292+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:47.546151+0000 front 2024-09-10T08:50:47.546107+0000 (oldest deadline 2024-09-10T08:51:11.646023+0000) 2024-09-10T08:51:15.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:15 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:15.335+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:15.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:15 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:15.335+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:15.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:15 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:15.335+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:15.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:15 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:15.335+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:15.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:15 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:15.335+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:15.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:15 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:15.335+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:40.729879+0000 front 2024-09-10T08:50:40.729980+0000 (oldest deadline 2024-09-10T08:51:06.029805+0000) 2024-09-10T08:51:15.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:15 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:15.335+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:48.831656+0000 front 2024-09-10T08:50:48.831720+0000 (oldest deadline 2024-09-10T08:51:12.931391+0000) 2024-09-10T08:51:16.035 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:15 smithi089 podman[106900]: 2024-09-10 08:51:15.587797621 +0000 UTC m=+0.738373846 container remove 197f27152e486484e5c8a222e6a684598ac6a3711b830afeb7422f864659619f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-10T08:51:16.291 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:16 smithi089 podman[106987]: 2024-09-10 08:51:15.939716368 +0000 UTC m=+0.026853651 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:51:16.291 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:16 smithi089 podman[106987]: 2024-09-10 08:51:16.055156847 +0000 UTC m=+0.142294108 container create e5249799b3545fe2c8beafa713f0988f4526726864f338226b4cbab9d3ed9976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T08:51:16.291 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:16 smithi089 podman[106987]: 2024-09-10 08:51:16.291241484 +0000 UTC m=+0.378378769 container init e5249799b3545fe2c8beafa713f0988f4526726864f338226b4cbab9d3ed9976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:51:16.548 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:16 smithi089 podman[106987]: 2024-09-10 08:51:16.298105347 +0000 UTC m=+0.385242611 container start e5249799b3545fe2c8beafa713f0988f4526726864f338226b4cbab9d3ed9976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:51:16.549 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:16 smithi089 podman[106987]: 2024-09-10 08:51:16.382621677 +0000 UTC m=+0.469758980 container attach e5249799b3545fe2c8beafa713f0988f4526726864f338226b4cbab9d3ed9976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T08:51:16.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:16 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:16.255+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:16.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:16 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:16.255+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:16.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:16 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:16.255+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:16.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:16 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:16.255+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:16.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:16 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:16.255+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:16.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:16 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:16.255+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:47.546151+0000 front 2024-09-10T08:50:47.546107+0000 (oldest deadline 2024-09-10T08:51:11.646023+0000) 2024-09-10T08:51:16.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:16 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:16.367+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:16.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:16 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:16.367+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:16.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:16 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:16.367+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:16.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:16 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:16.367+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:16.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:16 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:16.367+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:16.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:16 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:16.367+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:40.729879+0000 front 2024-09-10T08:50:40.729980+0000 (oldest deadline 2024-09-10T08:51:06.029805+0000) 2024-09-10T08:51:16.795 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:16 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:16.367+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:48.831656+0000 front 2024-09-10T08:50:48.831720+0000 (oldest deadline 2024-09-10T08:51:12.931391+0000) 2024-09-10T08:51:16.803 INFO:journalctl@ceph.osd.9.smithi089.stdout:Sep 10 08:51:16 smithi089 podman[106987]: 2024-09-10 08:51:16.548946882 +0000 UTC m=+0.636084157 container died e5249799b3545fe2c8beafa713f0988f4526726864f338226b4cbab9d3ed9976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:51:17.333 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.9.service' 2024-09-10T08:51:17.407 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:51:17.407 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-10T08:51:17.407 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-10T08:51:17.407 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.10 2024-09-10T08:51:17.700 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:17 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:17.385+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:17.700 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:17 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:17.385+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:17.701 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:17 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:17.385+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:17.701 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:17 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:17.385+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:17.701 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:17 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:17.385+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:17.701 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:17 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:17.385+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:40.729879+0000 front 2024-09-10T08:50:40.729980+0000 (oldest deadline 2024-09-10T08:51:06.029805+0000) 2024-09-10T08:51:17.701 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:17 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:17.385+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:48.831656+0000 front 2024-09-10T08:50:48.831720+0000 (oldest deadline 2024-09-10T08:51:12.931391+0000) 2024-09-10T08:51:17.701 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:17 smithi111 systemd[1]: Stopping Ceph osd.10 for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:51:17.701 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:17 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:17.699+0000 7ff7a3646640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:51:17.701 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:17 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:17.699+0000 7ff7a3646640 -1 osd.10 100 *** Got signal Terminated *** 2024-09-10T08:51:17.701 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:17 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:17.699+0000 7ff7a3646640 -1 osd.10 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:51:17.720 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:17 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:17.227+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:17.720 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:17 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:17.227+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:17.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:17 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:17.227+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:17.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:17 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:17.227+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:17.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:17 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:17.227+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:17.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:17 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:17.227+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:47.546151+0000 front 2024-09-10T08:50:47.546107+0000 (oldest deadline 2024-09-10T08:51:11.646023+0000) 2024-09-10T08:51:18.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:18 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:18.251+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:18.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:18 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:18.251+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:18.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:18 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:18.251+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:18.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:18 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:18.251+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:18.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:18 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:18.251+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:18.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:18 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:18.251+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:47.546151+0000 front 2024-09-10T08:50:47.546107+0000 (oldest deadline 2024-09-10T08:51:11.646023+0000) 2024-09-10T08:51:18.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:18 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:18.432+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:18.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:18 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:18.432+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:18.795 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:18 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:18.432+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:18.795 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:18 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:18.432+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:18.795 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:18 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:18.432+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:18.795 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:18 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:18.432+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:40.729879+0000 front 2024-09-10T08:50:40.729980+0000 (oldest deadline 2024-09-10T08:51:06.029805+0000) 2024-09-10T08:51:18.795 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:18 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:18.432+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:48.831656+0000 front 2024-09-10T08:50:48.831720+0000 (oldest deadline 2024-09-10T08:51:12.931391+0000) 2024-09-10T08:51:19.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:19 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:19.272+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:19.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:19 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:19.272+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:19.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:19 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:19.272+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:19.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:19 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:19.272+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:19.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:19 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:19.272+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:19.722 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:19 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:19.272+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:47.546151+0000 front 2024-09-10T08:50:47.546107+0000 (oldest deadline 2024-09-10T08:51:11.646023+0000) 2024-09-10T08:51:19.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:19 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:19.445+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:19.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:19 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:19.445+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:19.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:19 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:19.445+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:19.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:19 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:19.445+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:19.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:19 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:19.445+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:19.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:19 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:19.445+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:40.729879+0000 front 2024-09-10T08:50:40.729980+0000 (oldest deadline 2024-09-10T08:51:06.029805+0000) 2024-09-10T08:51:19.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:19 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:19.445+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:48.831656+0000 front 2024-09-10T08:50:48.831720+0000 (oldest deadline 2024-09-10T08:51:12.931391+0000) 2024-09-10T08:51:20.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:20 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:20.225+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:20.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:20 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:20.225+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:20.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:20 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:20.225+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:20.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:20 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:20.225+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:20.722 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:20 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:20.225+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:20.722 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:20 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:20.225+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:47.546151+0000 front 2024-09-10T08:50:47.546107+0000 (oldest deadline 2024-09-10T08:51:11.646023+0000) 2024-09-10T08:51:20.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:20 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:20.480+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:20.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:20 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:20.480+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:20.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:20 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:20.480+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:20.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:20 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:20.480+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:20.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:20 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:20.480+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:20.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:20 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:20.480+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:40.729879+0000 front 2024-09-10T08:50:40.729980+0000 (oldest deadline 2024-09-10T08:51:06.029805+0000) 2024-09-10T08:51:20.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:20 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:20.480+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:48.831656+0000 front 2024-09-10T08:50:48.831720+0000 (oldest deadline 2024-09-10T08:51:12.931391+0000) 2024-09-10T08:51:21.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:21 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:21.238+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:21.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:21 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:21.238+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:21.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:21 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:21.238+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:21.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:21 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:21.238+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:21.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:21 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:21.238+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:21.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:21 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:21.238+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:47.546151+0000 front 2024-09-10T08:50:47.546107+0000 (oldest deadline 2024-09-10T08:51:11.646023+0000) 2024-09-10T08:51:21.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:21 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:21.447+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:21.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:21 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:21.447+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:21.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:21 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:21.447+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:21.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:21 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:21.447+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:21.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:21 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:21.447+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:21.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:21 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:21.447+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:40.729879+0000 front 2024-09-10T08:50:40.729980+0000 (oldest deadline 2024-09-10T08:51:06.029805+0000) 2024-09-10T08:51:21.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:21 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:21.447+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:48.831656+0000 front 2024-09-10T08:50:48.831720+0000 (oldest deadline 2024-09-10T08:51:12.931391+0000) 2024-09-10T08:51:22.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:22 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:22.236+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:22.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:22 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:22.236+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:22.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:22 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:22.236+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:22.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:22 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:22.236+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:22.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:22 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:22.236+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:22.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:22 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:22.236+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:47.546151+0000 front 2024-09-10T08:50:47.546107+0000 (oldest deadline 2024-09-10T08:51:11.646023+0000) 2024-09-10T08:51:22.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:22 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:22.236+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-09-10T08:50:57.347642+0000 front 2024-09-10T08:50:57.347767+0000 (oldest deadline 2024-09-10T08:51:21.447640+0000) 2024-09-10T08:51:22.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:22 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:22.424+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:59.724969+0000 front 2024-09-10T08:49:59.725056+0000 (oldest deadline 2024-09-10T08:50:25.624931+0000) 2024-09-10T08:51:22.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:22 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:22.424+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-09-10T08:50:05.625255+0000 front 2024-09-10T08:50:05.625307+0000 (oldest deadline 2024-09-10T08:50:29.725392+0000) 2024-09-10T08:51:22.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:22 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:22.424+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:14.926787+0000 front 2024-09-10T08:50:14.926932+0000 (oldest deadline 2024-09-10T08:50:40.226691+0000) 2024-09-10T08:51:22.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:22 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:22.424+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:23.127769+0000 front 2024-09-10T08:50:23.127859+0000 (oldest deadline 2024-09-10T08:50:46.627813+0000) 2024-09-10T08:51:22.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:22 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:22.424+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:28.928664+0000 front 2024-09-10T08:50:28.928680+0000 (oldest deadline 2024-09-10T08:50:54.828775+0000) 2024-09-10T08:51:22.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:22 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:22.424+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:40.729879+0000 front 2024-09-10T08:50:40.729980+0000 (oldest deadline 2024-09-10T08:51:06.029805+0000) 2024-09-10T08:51:22.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:22 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10[59694]: 2024-09-10T08:51:22.424+0000 7ff79fc5c640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:48.831656+0000 front 2024-09-10T08:50:48.831720+0000 (oldest deadline 2024-09-10T08:51:12.931391+0000) 2024-09-10T08:51:23.401 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:23 smithi111 podman[70621]: 2024-09-10 08:51:23.144547046 +0000 UTC m=+5.536286294 container died 8dd49e38f9b2797d8fc8f7f2dbe1e67e33c7261a9daea7f9e6337cb294a730b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T08:51:23.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:23 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:23.253+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:23.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:23 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:23.253+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:23.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:23 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:23.253+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:23.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:23 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:23.253+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:23.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:23 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:23.253+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:23.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:23 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:23.253+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:47.546151+0000 front 2024-09-10T08:50:47.546107+0000 (oldest deadline 2024-09-10T08:51:11.646023+0000) 2024-09-10T08:51:23.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:23 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:23.253+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-09-10T08:50:57.347642+0000 front 2024-09-10T08:50:57.347767+0000 (oldest deadline 2024-09-10T08:51:21.447640+0000) 2024-09-10T08:51:23.776 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:23 smithi111 podman[70621]: 2024-09-10 08:51:23.491327992 +0000 UTC m=+5.883067228 container cleanup 8dd49e38f9b2797d8fc8f7f2dbe1e67e33c7261a9daea7f9e6337cb294a730b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T08:51:23.776 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:23 smithi111 bash[70621]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10 2024-09-10T08:51:24.043 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:23 smithi111 podman[70640]: 2024-09-10 08:51:23.775935791 +0000 UTC m=+0.631556875 container remove 8dd49e38f9b2797d8fc8f7f2dbe1e67e33c7261a9daea7f9e6337cb294a730b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T08:51:24.363 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:24 smithi111 podman[70726]: 2024-09-10 08:51:24.07672451 +0000 UTC m=+0.107269039 container create a48a0a1690d1b74af9f5730ba2063f068559a7bbfc8ea82749a9b4252865cd1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T08:51:24.363 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:24 smithi111 podman[70726]: 2024-09-10 08:51:23.985011229 +0000 UTC m=+0.015555758 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:51:24.363 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:24 smithi111 podman[70726]: 2024-09-10 08:51:24.287850443 +0000 UTC m=+0.318394966 container init a48a0a1690d1b74af9f5730ba2063f068559a7bbfc8ea82749a9b4252865cd1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:51:24.363 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:24 smithi111 podman[70726]: 2024-09-10 08:51:24.291428217 +0000 UTC m=+0.321972749 container start a48a0a1690d1b74af9f5730ba2063f068559a7bbfc8ea82749a9b4252865cd1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-09-10T08:51:24.363 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:24 smithi111 podman[70726]: 2024-09-10 08:51:24.362691458 +0000 UTC m=+0.393235991 container attach a48a0a1690d1b74af9f5730ba2063f068559a7bbfc8ea82749a9b4252865cd1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:51:24.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:24 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:24.296+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:24.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:24 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:24.296+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:24.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:24 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:24.296+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:24.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:24 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:24.296+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:24.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:24 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:24.296+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:24.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:24 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:24.296+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:47.546151+0000 front 2024-09-10T08:50:47.546107+0000 (oldest deadline 2024-09-10T08:51:11.646023+0000) 2024-09-10T08:51:24.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:24 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:24.296+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-09-10T08:50:57.347642+0000 front 2024-09-10T08:50:57.347767+0000 (oldest deadline 2024-09-10T08:51:21.447640+0000) 2024-09-10T08:51:24.794 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:24 smithi111 podman[70726]: 2024-09-10 08:51:24.486578522 +0000 UTC m=+0.517123056 container died a48a0a1690d1b74af9f5730ba2063f068559a7bbfc8ea82749a9b4252865cd1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T08:51:25.072 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.10.service' 2024-09-10T08:51:25.095 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:25 smithi111 podman[70750]: 2024-09-10 08:51:25.049387513 +0000 UTC m=+0.556061104 container remove a48a0a1690d1b74af9f5730ba2063f068559a7bbfc8ea82749a9b4252865cd1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-10T08:51:25.095 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:25 smithi111 systemd[1]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.10.service: Deactivated successfully. 2024-09-10T08:51:25.095 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:25 smithi111 systemd[1]: Stopped Ceph osd.10 for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:51:25.095 INFO:journalctl@ceph.osd.10.smithi111.stdout:Sep 10 08:51:25 smithi111 systemd[1]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.10.service: Consumed 1min 43.380s CPU time. 2024-09-10T08:51:25.110 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:51:25.110 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-10T08:51:25.111 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-10T08:51:25.111 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.11 2024-09-10T08:51:25.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:25 smithi149 systemd[1]: Stopping Ceph osd.11 for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:51:25.412 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:25 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:25.313+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:25.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:25 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:25.313+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:25.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:25 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:25.313+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:25.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:25 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:25.313+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:25.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:25 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:25.313+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:25.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:25 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:25.313+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:47.546151+0000 front 2024-09-10T08:50:47.546107+0000 (oldest deadline 2024-09-10T08:51:11.646023+0000) 2024-09-10T08:51:25.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:25 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:25.313+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-09-10T08:50:57.347642+0000 front 2024-09-10T08:50:57.347767+0000 (oldest deadline 2024-09-10T08:51:21.447640+0000) 2024-09-10T08:51:25.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:25 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:25.410+0000 7f825565c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:51:25.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:25 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:25.410+0000 7f825565c640 -1 osd.11 100 *** Got signal Terminated *** 2024-09-10T08:51:25.413 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:25 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:25.410+0000 7f825565c640 -1 osd.11 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T08:51:26.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:26 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:26.352+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:26.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:26 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:26.352+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:26.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:26 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:26.352+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:26.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:26 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:26.352+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:26.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:26 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:26.352+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:26.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:26 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:26.352+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:47.546151+0000 front 2024-09-10T08:50:47.546107+0000 (oldest deadline 2024-09-10T08:51:11.646023+0000) 2024-09-10T08:51:26.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:26 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:26.352+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-09-10T08:50:57.347642+0000 front 2024-09-10T08:50:57.347767+0000 (oldest deadline 2024-09-10T08:51:21.447640+0000) 2024-09-10T08:51:27.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:27 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:27.377+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:27.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:27 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:27.377+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:27.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:27 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:27.377+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:27.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:27 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:27.377+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:27.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:27 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:27.377+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:27.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:27 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:27.377+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:47.546151+0000 front 2024-09-10T08:50:47.546107+0000 (oldest deadline 2024-09-10T08:51:11.646023+0000) 2024-09-10T08:51:27.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:27 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:27.377+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-09-10T08:50:57.347642+0000 front 2024-09-10T08:50:57.347767+0000 (oldest deadline 2024-09-10T08:51:21.447640+0000) 2024-09-10T08:51:28.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:28 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:28.416+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:28.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:28 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:28.416+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:28.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:28 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:28.417+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:28.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:28 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:28.417+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:28.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:28 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:28.417+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:28.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:28 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:28.417+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:47.546151+0000 front 2024-09-10T08:50:47.546107+0000 (oldest deadline 2024-09-10T08:51:11.646023+0000) 2024-09-10T08:51:28.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:28 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:28.417+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-09-10T08:50:57.347642+0000 front 2024-09-10T08:50:57.347767+0000 (oldest deadline 2024-09-10T08:51:21.447640+0000) 2024-09-10T08:51:29.720 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:29 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:29.409+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:29.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:29 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:29.409+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:29.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:29 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:29.409+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:29.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:29 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:29.409+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:29.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:29 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:29.409+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:29.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:29 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:29.409+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:47.546151+0000 front 2024-09-10T08:50:47.546107+0000 (oldest deadline 2024-09-10T08:51:11.646023+0000) 2024-09-10T08:51:29.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:29 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:29.409+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-09-10T08:50:57.347642+0000 front 2024-09-10T08:50:57.347767+0000 (oldest deadline 2024-09-10T08:51:21.447640+0000) 2024-09-10T08:51:30.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:30 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:30.408+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6806 osd.0 since back 2024-09-10T08:49:55.836821+0000 front 2024-09-10T08:49:55.836939+0000 (oldest deadline 2024-09-10T08:50:21.736544+0000) 2024-09-10T08:51:30.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:30 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:30.408+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-09-10T08:50:13.140251+0000 front 2024-09-10T08:50:13.140138+0000 (oldest deadline 2024-09-10T08:50:39.040880+0000) 2024-09-10T08:51:30.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:30 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:30.408+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6814 osd.3 since back 2024-09-10T08:50:24.142968+0000 front 2024-09-10T08:50:24.142926+0000 (oldest deadline 2024-09-10T08:50:48.743375+0000) 2024-09-10T08:51:30.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:30 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:30.408+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-09-10T08:50:31.044375+0000 front 2024-09-10T08:50:31.044449+0000 (oldest deadline 2024-09-10T08:50:56.944417+0000) 2024-09-10T08:51:30.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:30 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:30.408+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-09-10T08:50:41.645430+0000 front 2024-09-10T08:50:41.645485+0000 (oldest deadline 2024-09-10T08:51:07.545612+0000) 2024-09-10T08:51:30.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:30 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:30.408+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.89:6822 osd.6 since back 2024-09-10T08:50:47.546151+0000 front 2024-09-10T08:50:47.546107+0000 (oldest deadline 2024-09-10T08:51:11.646023+0000) 2024-09-10T08:51:30.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:30 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11[61141]: 2024-09-10T08:51:30.408+0000 7f8251c72640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-09-10T08:50:57.347642+0000 front 2024-09-10T08:50:57.347767+0000 (oldest deadline 2024-09-10T08:51:21.447640+0000) 2024-09-10T08:51:31.221 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:30 smithi149 podman[72195]: 2024-09-10 08:51:30.950141797 +0000 UTC m=+5.638981429 container died 0b95f8eee16bd0f122d840243b62e498ea0950d7fd1e5ebae5f7fa97350b1236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:51:31.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:31 smithi149 podman[72195]: 2024-09-10 08:51:31.277685938 +0000 UTC m=+5.966525584 container cleanup 0b95f8eee16bd0f122d840243b62e498ea0950d7fd1e5ebae5f7fa97350b1236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T08:51:31.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:31 smithi149 bash[72195]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11 2024-09-10T08:51:31.721 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:31 smithi149 podman[72213]: 2024-09-10 08:51:31.462282833 +0000 UTC m=+0.513494828 container remove 0b95f8eee16bd0f122d840243b62e498ea0950d7fd1e5ebae5f7fa97350b1236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T08:51:32.132 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:31 smithi149 podman[72301]: 2024-09-10 08:51:31.863012656 +0000 UTC m=+0.102091191 container create 497b28291c3daaf64765685d239334cb77775c4426d4e7f3603af2aceb874eaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T08:51:32.132 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:31 smithi149 podman[72301]: 2024-09-10 08:51:31.776744973 +0000 UTC m=+0.015823476 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:51:32.132 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:32 smithi149 podman[72301]: 2024-09-10 08:51:32.056942594 +0000 UTC m=+0.296021093 container init 497b28291c3daaf64765685d239334cb77775c4426d4e7f3603af2aceb874eaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-10T08:51:32.132 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:32 smithi149 podman[72301]: 2024-09-10 08:51:32.060210021 +0000 UTC m=+0.299288514 container start 497b28291c3daaf64765685d239334cb77775c4426d4e7f3603af2aceb874eaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-10T08:51:32.471 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:32 smithi149 podman[72301]: 2024-09-10 08:51:32.131809224 +0000 UTC m=+0.370887722 container attach 497b28291c3daaf64765685d239334cb77775c4426d4e7f3603af2aceb874eaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-10T08:51:32.471 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:32 smithi149 podman[72301]: 2024-09-10 08:51:32.247347872 +0000 UTC m=+0.486426366 container died 497b28291c3daaf64765685d239334cb77775c4426d4e7f3603af2aceb874eaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T08:51:32.782 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.11.service' 2024-09-10T08:51:32.806 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:32 smithi149 podman[72324]: 2024-09-10 08:51:32.759975239 +0000 UTC m=+0.506772554 container remove 497b28291c3daaf64765685d239334cb77775c4426d4e7f3603af2aceb874eaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T08:51:32.806 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:32 smithi149 systemd[1]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.11.service: Deactivated successfully. 2024-09-10T08:51:32.806 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:32 smithi149 systemd[1]: Stopped Ceph osd.11 for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:51:32.806 INFO:journalctl@ceph.osd.11.smithi149.stdout:Sep 10 08:51:32 smithi149 systemd[1]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@osd.11.service: Consumed 1min 57.048s CPU time. 2024-09-10T08:51:32.821 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:51:32.821 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-10T08:51:32.821 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-10T08:51:32.821 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.a 2024-09-10T08:51:33.224 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 10 08:51:32 smithi089 systemd[1]: Stopping Ceph mds.a for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:51:33.548 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 10 08:51:33 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-a[71666]: 2024-09-10T08:51:33.223+0000 7f13a6a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:51:33.548 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 10 08:51:33 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-a[71666]: 2024-09-10T08:51:33.223+0000 7f13a6a00640 -1 mds.a *** got signal Terminated *** 2024-09-10T08:51:34.797 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 10 08:51:34 smithi089 podman[107091]: 2024-09-10 08:51:34.477729095 +0000 UTC m=+1.366748741 container died d384f926c0a908dc9441960f2359da2bb80852212035a745de514459990de481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T08:51:35.053 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 10 08:51:34 smithi089 podman[107091]: 2024-09-10 08:51:34.987393039 +0000 UTC m=+1.876412676 container cleanup d384f926c0a908dc9441960f2359da2bb80852212035a745de514459990de481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:51:35.053 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 10 08:51:34 smithi089 bash[107091]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-a 2024-09-10T08:51:35.404 INFO:journalctl@ceph.mds.a.smithi089.stdout:Sep 10 08:51:35 smithi089 podman[107114]: 2024-09-10 08:51:35.17210165 +0000 UTC m=+0.690270052 container remove d384f926c0a908dc9441960f2359da2bb80852212035a745de514459990de481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T08:51:35.426 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.a.service' 2024-09-10T08:51:35.500 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:51:35.500 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-10T08:51:35.500 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-10T08:51:35.501 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.d 2024-09-10T08:51:35.803 INFO:journalctl@ceph.mds.d.smithi089.stdout:Sep 10 08:51:35 smithi089 systemd[1]: Stopping Ceph mds.d for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:51:36.303 INFO:journalctl@ceph.mds.d.smithi089.stdout:Sep 10 08:51:35 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-d[70559]: 2024-09-10T08:51:35.910+0000 7f0ff3600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:51:36.303 INFO:journalctl@ceph.mds.d.smithi089.stdout:Sep 10 08:51:35 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-d[70559]: 2024-09-10T08:51:35.910+0000 7f0ff3600640 -1 mds.d *** got signal Terminated *** 2024-09-10T08:51:38.659 INFO:journalctl@ceph.mds.d.smithi089.stdout:Sep 10 08:51:38 smithi089 podman[107221]: 2024-09-10 08:51:38.335959336 +0000 UTC m=+2.567725703 container died 50ead59fd148974909ce07438264477e5b716981f062daf0d3a1350b6a3dbe58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T08:51:38.959 INFO:journalctl@ceph.mds.d.smithi089.stdout:Sep 10 08:51:38 smithi089 podman[107221]: 2024-09-10 08:51:38.766333895 +0000 UTC m=+2.998100244 container cleanup 50ead59fd148974909ce07438264477e5b716981f062daf0d3a1350b6a3dbe58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-d, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD) 2024-09-10T08:51:38.959 INFO:journalctl@ceph.mds.d.smithi089.stdout:Sep 10 08:51:38 smithi089 bash[107221]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-d 2024-09-10T08:51:39.230 INFO:journalctl@ceph.mds.d.smithi089.stdout:Sep 10 08:51:38 smithi089 podman[107232]: 2024-09-10 08:51:38.959350971 +0000 UTC m=+0.609409196 container remove 50ead59fd148974909ce07438264477e5b716981f062daf0d3a1350b6a3dbe58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-d, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T08:51:39.254 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.d.service' 2024-09-10T08:51:39.323 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:51:39.323 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-10T08:51:39.323 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-10T08:51:39.323 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.g 2024-09-10T08:51:39.553 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 10 08:51:39 smithi089 systemd[1]: Stopping Ceph mds.g for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:51:40.053 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 10 08:51:39 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-g[71319]: 2024-09-10T08:51:39.722+0000 7efd14800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:51:40.053 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 10 08:51:39 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-g[71319]: 2024-09-10T08:51:39.722+0000 7efd14800640 -1 mds.g *** got signal Terminated *** 2024-09-10T08:51:43.105 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 10 08:51:42 smithi089 podman[107339]: 2024-09-10 08:51:42.835130481 +0000 UTC m=+3.199412799 container died e499f39fca99fa88b5b91751749da0c787bf3067ffd04e65e5fe2dc7de93f4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-10T08:51:43.497 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 10 08:51:43 smithi089 podman[107339]: 2024-09-10 08:51:43.320164227 +0000 UTC m=+3.684446544 container cleanup e499f39fca99fa88b5b91751749da0c787bf3067ffd04e65e5fe2dc7de93f4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T08:51:43.497 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 10 08:51:43 smithi089 bash[107339]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-g 2024-09-10T08:51:43.755 INFO:journalctl@ceph.mds.g.smithi089.stdout:Sep 10 08:51:43 smithi089 podman[107352]: 2024-09-10 08:51:43.496750906 +0000 UTC m=+0.649080516 container remove e499f39fca99fa88b5b91751749da0c787bf3067ffd04e65e5fe2dc7de93f4fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0) 2024-09-10T08:51:43.763 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.g.service' 2024-09-10T08:51:43.832 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:51:43.832 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-10T08:51:43.832 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-10T08:51:43.832 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.j 2024-09-10T08:51:44.053 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 10 08:51:43 smithi089 systemd[1]: Stopping Ceph mds.j for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:51:44.553 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 10 08:51:44 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-j[70152]: 2024-09-10T08:51:44.209+0000 7f8d4c800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:51:44.554 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 10 08:51:44 smithi089 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-j[70152]: 2024-09-10T08:51:44.209+0000 7f8d4c800640 -1 mds.j *** got signal Terminated *** 2024-09-10T08:51:46.650 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 10 08:51:46 smithi089 podman[107458]: 2024-09-10 08:51:46.379391725 +0000 UTC m=+2.280825312 container died 2d77cfd89db9c09b152771e783836392b8edf12456e0e6f67d082aa320748d5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-j, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:51:47.016 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 10 08:51:46 smithi089 podman[107458]: 2024-09-10 08:51:46.748981922 +0000 UTC m=+2.650415508 container cleanup 2d77cfd89db9c09b152771e783836392b8edf12456e0e6f67d082aa320748d5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True) 2024-09-10T08:51:47.017 INFO:journalctl@ceph.mds.j.smithi089.stdout:Sep 10 08:51:46 smithi089 bash[107458]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-j 2024-09-10T08:51:47.175 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.j.service' 2024-09-10T08:51:47.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:51:47.243 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-10T08:51:47.243 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-10T08:51:47.243 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.b 2024-09-10T08:51:47.542 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:51:47 smithi111 systemd[1]: Stopping Ceph mds.b for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:51:47.793 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:51:47 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-b[65063]: 2024-09-10T08:51:47.541+0000 7f6989b00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:51:47.794 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:51:47 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-b[65063]: 2024-09-10T08:51:47.542+0000 7f6989b00640 -1 mds.b *** got signal Terminated *** 2024-09-10T08:51:51.159 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:51:50 smithi111 podman[70830]: 2024-09-10 08:51:50.898708665 +0000 UTC m=+3.462794105 container died d725d119605e2363ba9aa22261f90f90daa8b52afa2101b3f47d1f6213fe7e1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-b, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:51:51.434 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:51:51 smithi111 podman[70830]: 2024-09-10 08:51:51.241779868 +0000 UTC m=+3.805865318 container cleanup d725d119605e2363ba9aa22261f90f90daa8b52afa2101b3f47d1f6213fe7e1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-b, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T08:51:51.435 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:51:51 smithi111 bash[70830]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-b 2024-09-10T08:51:51.663 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.b.service' 2024-09-10T08:51:51.685 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:51:51 smithi111 podman[70854]: 2024-09-10 08:51:51.434562409 +0000 UTC m=+0.528892684 container remove d725d119605e2363ba9aa22261f90f90daa8b52afa2101b3f47d1f6213fe7e1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T08:51:51.685 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:51:51 smithi111 systemd[1]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.b.service: Deactivated successfully. 2024-09-10T08:51:51.686 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:51:51 smithi111 systemd[1]: Stopped Ceph mds.b for 91f747bc-6f4e-11ef-bcea-c7b262605968. 2024-09-10T08:51:51.686 INFO:journalctl@ceph.mds.b.smithi111.stdout:Sep 10 08:51:51 smithi111 systemd[1]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.b.service: Consumed 9min 19.447s CPU time. 2024-09-10T08:51:51.698 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:51:51.699 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-10T08:51:51.699 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-10T08:51:51.699 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.e 2024-09-10T08:51:52.030 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 10 08:51:51 smithi111 systemd[1]: Stopping Ceph mds.e for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:51:52.297 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 10 08:51:52 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-e[64459]: 2024-09-10T08:51:52.029+0000 7f9335456640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:51:52.297 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 10 08:51:52 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-e[64459]: 2024-09-10T08:51:52.029+0000 7f9335456640 -1 mds.e *** got signal Terminated *** 2024-09-10T08:51:55.196 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 10 08:51:54 smithi111 podman[70964]: 2024-09-10 08:51:54.907914664 +0000 UTC m=+2.992588101 container died c196a0b6f6fc1f368ed84eba8e1aeb85efaf874ecedca65be76b0e49754e7b3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T08:51:55.539 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 10 08:51:55 smithi111 podman[70964]: 2024-09-10 08:51:55.270698336 +0000 UTC m=+3.355371804 container cleanup c196a0b6f6fc1f368ed84eba8e1aeb85efaf874ecedca65be76b0e49754e7b3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-e, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T08:51:55.539 INFO:journalctl@ceph.mds.e.smithi111.stdout:Sep 10 08:51:55 smithi111 bash[70964]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-e 2024-09-10T08:51:55.703 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.e.service' 2024-09-10T08:51:55.736 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:51:55.736 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-10T08:51:55.736 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-10T08:51:55.737 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.h 2024-09-10T08:51:56.067 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:51:55 smithi111 systemd[1]: Stopping Ceph mds.h for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:51:56.543 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:51:56 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h[64759]: 2024-09-10T08:51:56.066+0000 7f5c26a02640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:51:56.544 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:51:56 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h[64759]: 2024-09-10T08:51:56.066+0000 7f5c26a02640 -1 mds.h *** got signal Terminated *** 2024-09-10T08:51:59.191 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:51:58 smithi111 podman[71087]: 2024-09-10 08:51:58.91286743 +0000 UTC m=+2.957650702 container died 34d380ce62f19dcd97633ac9482548a5946dfa4f9c1b6eef2dc97b1adfd3e8e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-10T08:51:59.191 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:51:59 smithi111 podman[71087]: 2024-09-10 08:51:59.191281807 +0000 UTC m=+3.236065079 container cleanup 34d380ce62f19dcd97633ac9482548a5946dfa4f9c1b6eef2dc97b1adfd3e8e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:51:59.543 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:51:59 smithi111 bash[71087]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h 2024-09-10T08:51:59.544 INFO:journalctl@ceph.mds.h.smithi111.stdout:Sep 10 08:51:59 smithi111 podman[71098]: 2024-09-10 08:51:59.434231714 +0000 UTC m=+0.521647683 container remove 34d380ce62f19dcd97633ac9482548a5946dfa4f9c1b6eef2dc97b1adfd3e8e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-h, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-10T08:51:59.581 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.h.service' 2024-09-10T08:51:59.614 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:51:59.615 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-10T08:51:59.615 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-10T08:51:59.615 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.k 2024-09-10T08:51:59.829 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 10 08:51:59 smithi111 systemd[1]: Stopping Ceph mds.k for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:52:00.294 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 10 08:51:59 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-k[64142]: 2024-09-10T08:51:59.936+0000 7f9ae6958640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:52:00.294 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 10 08:51:59 smithi111 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-k[64142]: 2024-09-10T08:51:59.936+0000 7f9ae6958640 -1 mds.k *** got signal Terminated *** 2024-09-10T08:52:03.196 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 10 08:52:02 smithi111 podman[71206]: 2024-09-10 08:52:02.835488422 +0000 UTC m=+2.992501175 container died 33aea65ed9d8d0d6bf6fc2b89d82242919909dfa693306e039402b3036e2eaf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-09-10T08:52:03.544 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 10 08:52:03 smithi111 podman[71206]: 2024-09-10 08:52:03.261829378 +0000 UTC m=+3.418842128 container cleanup 33aea65ed9d8d0d6bf6fc2b89d82242919909dfa693306e039402b3036e2eaf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-k, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:52:03.544 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 10 08:52:03 smithi111 bash[71206]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-k 2024-09-10T08:52:03.544 INFO:journalctl@ceph.mds.k.smithi111.stdout:Sep 10 08:52:03 smithi111 podman[71220]: 2024-09-10 08:52:03.429555585 +0000 UTC m=+0.504486440 container remove 33aea65ed9d8d0d6bf6fc2b89d82242919909dfa693306e039402b3036e2eaf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:52:03.638 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.k.service' 2024-09-10T08:52:03.671 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:52:03.671 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-10T08:52:03.671 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-10T08:52:03.671 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.c 2024-09-10T08:52:03.971 INFO:journalctl@ceph.mds.c.smithi149.stdout:Sep 10 08:52:03 smithi149 systemd[1]: Stopping Ceph mds.c for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:52:04.325 INFO:journalctl@ceph.mds.c.smithi149.stdout:Sep 10 08:52:03 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-c[66435]: 2024-09-10T08:52:03.975+0000 7f5d2bb59640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:52:04.325 INFO:journalctl@ceph.mds.c.smithi149.stdout:Sep 10 08:52:03 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-c[66435]: 2024-09-10T08:52:03.975+0000 7f5d2bb59640 -1 mds.c *** got signal Terminated *** 2024-09-10T08:52:07.221 INFO:journalctl@ceph.mds.c.smithi149.stdout:Sep 10 08:52:06 smithi149 podman[72405]: 2024-09-10 08:52:06.879297227 +0000 UTC m=+3.010811206 container died a437630f0b5577af4a324f6933d761e7d69de6cdcc83606d4ec810340463c0b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-c, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:52:07.636 INFO:journalctl@ceph.mds.c.smithi149.stdout:Sep 10 08:52:07 smithi149 podman[72405]: 2024-09-10 08:52:07.384632488 +0000 UTC m=+3.516146469 container cleanup a437630f0b5577af4a324f6933d761e7d69de6cdcc83606d4ec810340463c0b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True) 2024-09-10T08:52:07.636 INFO:journalctl@ceph.mds.c.smithi149.stdout:Sep 10 08:52:07 smithi149 bash[72405]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-c 2024-09-10T08:52:07.762 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.c.service' 2024-09-10T08:52:07.799 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:52:07.799 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-10T08:52:07.800 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-10T08:52:07.800 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.f 2024-09-10T08:52:08.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:Sep 10 08:52:07 smithi149 systemd[1]: Stopping Ceph mds.f for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:52:08.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:Sep 10 08:52:08 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-f[66131]: 2024-09-10T08:52:08.120+0000 7f7934eb2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:52:08.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:Sep 10 08:52:08 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-f[66131]: 2024-09-10T08:52:08.120+0000 7f7934eb2640 -1 mds.f *** got signal Terminated *** 2024-09-10T08:52:11.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:Sep 10 08:52:10 smithi149 podman[72537]: 2024-09-10 08:52:10.883510853 +0000 UTC m=+2.856936636 container died 67b01eafeae780b66b4392179247837df6c20b3954754ea045a894f5720d9e04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True) 2024-09-10T08:52:11.612 INFO:journalctl@ceph.mds.f.smithi149.stdout:Sep 10 08:52:11 smithi149 podman[72537]: 2024-09-10 08:52:11.229804051 +0000 UTC m=+3.203229826 container cleanup 67b01eafeae780b66b4392179247837df6c20b3954754ea045a894f5720d9e04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T08:52:11.612 INFO:journalctl@ceph.mds.f.smithi149.stdout:Sep 10 08:52:11 smithi149 bash[72537]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-f 2024-09-10T08:52:11.612 INFO:journalctl@ceph.mds.f.smithi149.stdout:Sep 10 08:52:11 smithi149 podman[72549]: 2024-09-10 08:52:11.42288338 +0000 UTC m=+0.533453910 container remove 67b01eafeae780b66b4392179247837df6c20b3954754ea045a894f5720d9e04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-10T08:52:11.627 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.f.service' 2024-09-10T08:52:11.661 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:52:11.661 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-10T08:52:11.661 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-10T08:52:11.662 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.i 2024-09-10T08:52:11.971 INFO:journalctl@ceph.mds.i.smithi149.stdout:Sep 10 08:52:11 smithi149 systemd[1]: Stopping Ceph mds.i for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:52:12.470 INFO:journalctl@ceph.mds.i.smithi149.stdout:Sep 10 08:52:11 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-i[66738]: 2024-09-10T08:52:11.991+0000 7f9cb43b1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:52:12.471 INFO:journalctl@ceph.mds.i.smithi149.stdout:Sep 10 08:52:11 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-i[66738]: 2024-09-10T08:52:11.991+0000 7f9cb43b1640 -1 mds.i *** got signal Terminated *** 2024-09-10T08:52:15.139 INFO:journalctl@ceph.mds.i.smithi149.stdout:Sep 10 08:52:14 smithi149 podman[72660]: 2024-09-10 08:52:14.871024734 +0000 UTC m=+2.991486537 container died f9e956b83d57675f6b352cab851977a2979ae18307985e21c5627d3a227d6890 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-i, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-10T08:52:15.470 INFO:journalctl@ceph.mds.i.smithi149.stdout:Sep 10 08:52:15 smithi149 podman[72660]: 2024-09-10 08:52:15.241723552 +0000 UTC m=+3.362185354 container cleanup f9e956b83d57675f6b352cab851977a2979ae18307985e21c5627d3a227d6890 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T08:52:15.471 INFO:journalctl@ceph.mds.i.smithi149.stdout:Sep 10 08:52:15 smithi149 bash[72660]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-i 2024-09-10T08:52:15.620 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.i.service' 2024-09-10T08:52:15.652 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:52:15.653 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-10T08:52:15.653 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-10T08:52:15.653 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.l 2024-09-10T08:52:15.962 INFO:journalctl@ceph.mds.l.smithi149.stdout:Sep 10 08:52:15 smithi149 systemd[1]: Stopping Ceph mds.l for 91f747bc-6f4e-11ef-bcea-c7b262605968... 2024-09-10T08:52:16.221 INFO:journalctl@ceph.mds.l.smithi149.stdout:Sep 10 08:52:15 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-l[65815]: 2024-09-10T08:52:15.962+0000 7fd18c8bf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:52:16.221 INFO:journalctl@ceph.mds.l.smithi149.stdout:Sep 10 08:52:15 smithi149 ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-l[65815]: 2024-09-10T08:52:15.962+0000 7fd18c8bf640 -1 mds.l *** got signal Terminated *** 2024-09-10T08:52:20.220 INFO:journalctl@ceph.mds.l.smithi149.stdout:Sep 10 08:52:19 smithi149 podman[72781]: 2024-09-10 08:52:19.969416032 +0000 UTC m=+4.102737298 container died 2d0dd4a58e261b911306a522e4bfaab20e7650a1c6913caf64e5ea67e4937aae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T08:52:20.622 INFO:journalctl@ceph.mds.l.smithi149.stdout:Sep 10 08:52:20 smithi149 podman[72781]: 2024-09-10 08:52:20.320245468 +0000 UTC m=+4.453566779 container cleanup 2d0dd4a58e261b911306a522e4bfaab20e7650a1c6913caf64e5ea67e4937aae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T08:52:20.622 INFO:journalctl@ceph.mds.l.smithi149.stdout:Sep 10 08:52:20 smithi149 bash[72781]: ceph-91f747bc-6f4e-11ef-bcea-c7b262605968-mds-l 2024-09-10T08:52:20.781 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f747bc-6f4e-11ef-bcea-c7b262605968@mds.l.service' 2024-09-10T08:52:20.813 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T08:52:20.813 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-10T08:52:20.814 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T08:52:21.084 INFO:teuthology.orchestra.run.smithi089.stdout:Deleting cluster with fsid: 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:52:25.138 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T08:52:25.339 INFO:teuthology.orchestra.run.smithi111.stdout:Deleting cluster with fsid: 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:52:27.817 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T08:52:28.018 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:52:30.510 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T08:52:30.574 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T08:52:30.608 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T08:52:30.635 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T08:52:30.636 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898917/remote/smithi089/crash 2024-09-10T08:52:30.637 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/crash -- . 2024-09-10T08:52:30.693 INFO:teuthology.orchestra.run.smithi089.stderr:tar: /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T08:52:30.693 INFO:teuthology.orchestra.run.smithi089.stderr:tar: Error is not recoverable: exiting now 2024-09-10T08:52:30.697 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898917/remote/smithi111/crash 2024-09-10T08:52:30.697 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/crash -- . 2024-09-10T08:52:30.735 INFO:teuthology.orchestra.run.smithi111.stderr:tar: /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T08:52:30.735 INFO:teuthology.orchestra.run.smithi111.stderr:tar: Error is not recoverable: exiting now 2024-09-10T08:52:30.737 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898917/remote/smithi149/crash 2024-09-10T08:52:30.738 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/crash -- . 2024-09-10T08:52:30.780 INFO:teuthology.orchestra.run.smithi149.stderr:tar: /var/lib/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T08:52:30.780 INFO:teuthology.orchestra.run.smithi149.stderr:tar: Error is not recoverable: exiting now 2024-09-10T08:52:30.781 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T08:52:30.781 DEBUG:teuthology.orchestra.run.smithi089:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-10T08:52:30.865 INFO:teuthology.orchestra.run.smithi089.stdout:2024-09-10T08:43:43.196033+0000 mds.b (mds.0) 24 : cluster [WRN] Scrub error on inode 0x10000001741 (/volumes/qa/sv_1/0b6bb52e-a4ec-44c5-8208-ac0bd9788e1e/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-33) see mds.b log and `damage ls` output for details 2024-09-10T08:52:30.866 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-10T08:52:30.866 INFO:tasks.cephadm:Compressing logs... 2024-09-10T08:52:30.867 DEBUG:teuthology.orchestra.run.smithi089:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T08:52:30.871 DEBUG:teuthology.orchestra.run.smithi111:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T08:52:30.873 DEBUG:teuthology.orchestra.run.smithi149:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T08:52:30.897 INFO:teuthology.orchestra.run.smithi149.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T08:52:30.897 INFO:teuthology.orchestra.run.smithi149.stderr:: No such file or directory 2024-09-10T08:52:30.901 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T08:52:30.901 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-volume.log: --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-10T08:52:30.901 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T08:52:30.901 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T08:52:30.902 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-10T08:52:30.902 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T08:52:30.902 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T08:52:30.902 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.log 2024-09-10T08:52:30.902 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-10T08:52:30.902 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-10T08:52:30.902 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-10T08:52:30.902 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-10T08:52:30.902 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-10T08:52:30.902 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T08:52:30.903 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T08:52:30.903 INFO:teuthology.orchestra.run.smithi149.stderr: 94.7% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-10T08:52:30.910 INFO:teuthology.orchestra.run.smithi111.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-10T08:52:30.910 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-10T08:52:30.911 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T08:52:30.911 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T08:52:30.912 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T08:52:30.912 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-10T08:52:30.912 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-10T08:52:30.915 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T08:52:30.915 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T08:52:30.915 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T08:52:30.915 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T08:52:30.915 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.log 2024-09-10T08:52:30.915 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-10T08:52:30.915 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T08:52:30.915 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-10T08:52:30.915 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T08:52:30.916 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-10T08:52:30.920 INFO:teuthology.orchestra.run.smithi089.stderr:find: ‘/var/log/rbd-target-api’: No such file or directorygzip 2024-09-10T08:52:30.921 INFO:teuthology.orchestra.run.smithi089.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-10T08:52:30.921 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T08:52:30.922 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-10T08:52:30.923 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T08:52:30.924 INFO:teuthology.orchestra.run.smithi089.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-10T08:52:30.924 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T08:52:30.925 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-10T08:52:30.926 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T08:52:30.927 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T08:52:30.929 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-10T08:52:30.929 INFO:teuthology.orchestra.run.smithi089.stderr: 82.4% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T08:52:30.930 INFO:teuthology.orchestra.run.smithi089.stderr: 90.2% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T08:52:30.930 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T08:52:30.931 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.log 2024-09-10T08:52:30.932 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.6.log: 94.5% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-10T08:52:30.935 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-10T08:52:30.936 INFO:teuthology.orchestra.run.smithi089.stderr: 94.6% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-10T08:52:30.936 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T08:52:30.938 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-10T08:52:30.939 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mgr.x.log: 90.9% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T08:52:30.941 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.j.log: 86.7% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T08:52:30.941 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T08:52:30.942 INFO:teuthology.orchestra.run.smithi089.stderr: 94.6% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-10T08:52:30.945 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T08:52:30.945 INFO:teuthology.orchestra.run.smithi089.stderr: 94.5% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-10T08:52:30.948 INFO:teuthology.orchestra.run.smithi149.stderr: 82.2% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T08:52:30.958 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/cephadm.log: 94.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T08:52:30.984 INFO:teuthology.orchestra.run.smithi149.stderr: 92.9% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-10T08:52:30.986 INFO:teuthology.orchestra.run.smithi149.stderr: 86.7% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T08:52:30.986 INFO:teuthology.orchestra.run.smithi149.stderr: 90.3% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T08:52:31.108 INFO:teuthology.orchestra.run.smithi149.stderr: 91.0% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T08:52:31.111 INFO:teuthology.orchestra.run.smithi111.stderr: 91.0% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T08:52:31.136 INFO:teuthology.orchestra.run.smithi111.stderr: 92.8% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T08:52:31.156 INFO:teuthology.orchestra.run.smithi111.stderr: 82.2% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T08:52:31.162 INFO:teuthology.orchestra.run.smithi111.stderr: 90.3% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T08:52:31.166 INFO:teuthology.orchestra.run.smithi111.stderr: 86.7% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T08:52:31.231 INFO:teuthology.orchestra.run.smithi089.stderr: 90.3% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T08:52:31.315 INFO:teuthology.orchestra.run.smithi111.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T08:52:31.325 INFO:teuthology.orchestra.run.smithi149.stderr: 91.5% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T08:52:31.349 INFO:teuthology.orchestra.run.smithi149.stderr: 94.9% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-10T08:52:32.168 INFO:teuthology.orchestra.run.smithi089.stderr: 90.3% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T08:52:32.536 INFO:teuthology.orchestra.run.smithi149.stderr: 95.3% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-10T08:52:33.918 INFO:teuthology.orchestra.run.smithi111.stderr: 91.6% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T08:52:38.278 INFO:teuthology.orchestra.run.smithi149.stderr: 94.3% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-10T08:52:50.280 INFO:teuthology.orchestra.run.smithi089.stderr: 91.9% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T08:52:51.980 INFO:teuthology.orchestra.run.smithi089.stderr: 91.7% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T08:52:52.208 INFO:teuthology.orchestra.run.smithi149.stderr: 91.8% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-10T08:52:52.510 INFO:teuthology.orchestra.run.smithi149.stderr: 91.8% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-10T08:52:52.567 INFO:teuthology.orchestra.run.smithi149.stderr: 91.9% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T08:52:53.641 INFO:teuthology.orchestra.run.smithi089.stderr: 91.8% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T08:52:54.260 INFO:teuthology.orchestra.run.smithi089.stderr: 91.8% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-10T08:52:54.263 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-09-10T08:52:54.263 INFO:teuthology.orchestra.run.smithi089.stderr:real 0m23.364s 2024-09-10T08:52:54.263 INFO:teuthology.orchestra.run.smithi089.stderr:user 1m21.516s 2024-09-10T08:52:54.263 INFO:teuthology.orchestra.run.smithi089.stderr:sys 0m6.584s 2024-09-10T08:53:01.565 INFO:teuthology.orchestra.run.smithi149.stderr: 91.7% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T08:53:01.566 INFO:teuthology.orchestra.run.smithi149.stderr: 2024-09-10T08:53:01.566 INFO:teuthology.orchestra.run.smithi149.stderr:real 0m30.678s 2024-09-10T08:53:01.566 INFO:teuthology.orchestra.run.smithi149.stderr:user 1m26.532s 2024-09-10T08:53:01.566 INFO:teuthology.orchestra.run.smithi149.stderr:sys 0m3.030s 2024-09-10T08:53:37.192 INFO:teuthology.orchestra.run.smithi111.stderr: 91.9% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T08:53:45.865 INFO:teuthology.orchestra.run.smithi111.stderr: 94.9% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-10T08:54:05.219 INFO:teuthology.orchestra.run.smithi111.stderr: 91.8% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-10T08:54:18.153 INFO:teuthology.orchestra.run.smithi111.stderr: 91.9% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T08:55:09.310 INFO:teuthology.orchestra.run.smithi111.stderr: 91.6% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T08:55:20.992 INFO:teuthology.orchestra.run.smithi111.stderr: 94.3% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-10T08:58:30.308 INFO:teuthology.orchestra.run.smithi111.stderr: 93.7% -- replaced with /var/log/ceph/91f747bc-6f4e-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-10T08:58:30.309 INFO:teuthology.orchestra.run.smithi111.stderr: 2024-09-10T08:58:30.310 INFO:teuthology.orchestra.run.smithi111.stderr:real 5m59.421s 2024-09-10T08:58:30.310 INFO:teuthology.orchestra.run.smithi111.stderr:user 5m3.983s 2024-09-10T08:58:30.310 INFO:teuthology.orchestra.run.smithi111.stderr:sys 0m11.677s 2024-09-10T08:58:30.310 INFO:tasks.cephadm:Archiving logs... 2024-09-10T08:58:30.311 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898917/remote/smithi089/log 2024-09-10T08:58:30.313 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T08:58:43.753 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898917/remote/smithi111/log 2024-09-10T08:58:43.754 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T08:59:31.391 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898917/remote/smithi149/log 2024-09-10T08:59:31.392 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T08:59:46.116 INFO:tasks.cephadm:Removing cluster... 2024-09-10T08:59:46.116 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 --force 2024-09-10T08:59:46.393 INFO:teuthology.orchestra.run.smithi089.stdout:Deleting cluster with fsid: 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:59:47.157 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 --force 2024-09-10T08:59:47.395 INFO:teuthology.orchestra.run.smithi111.stdout:Deleting cluster with fsid: 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:59:47.952 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 91f747bc-6f4e-11ef-bcea-c7b262605968 --force 2024-09-10T08:59:48.153 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: 91f747bc-6f4e-11ef-bcea-c7b262605968 2024-09-10T08:59:48.779 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T08:59:48.779 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T08:59:48.821 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T08:59:48.839 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T08:59:48.856 INFO:tasks.cephadm:Teardown complete 2024-09-10T08:59:48.856 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T08:59:48.869 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T08:59:48.869 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T08:59:48.873 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T08:59:48.882 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T08:59:49.004 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T08:59:49.015 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T08:59:49.015 DEBUG:teuthology.orchestra.run.smithi089:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T08:59:49.020 DEBUG:teuthology.orchestra.run.smithi111:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T08:59:49.026 DEBUG:teuthology.orchestra.run.smithi149:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T08:59:49.039 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-09-10T08:59:49.040 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-09-10T08:59:49.044 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T08:59:49.044 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-09-10T08:59:49.044 INFO:teuthology.orchestra.run.smithi149.stdout:^* hv01.front.sepia.ceph.com 3 6 377 27 +795us[ +771us] +/- 48ms 2024-09-10T08:59:49.044 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 27 +1670us[+1646us] +/- 70ms 2024-09-10T08:59:49.045 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 -3791us[-3791us] +/- 96ms 2024-09-10T08:59:49.045 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:59:49.050 INFO:teuthology.orchestra.run.smithi089.stderr:bash: line 1: ntpq: command not found 2024-09-10T08:59:49.055 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T08:59:49.055 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-09-10T08:59:49.055 INFO:teuthology.orchestra.run.smithi111.stdout:^* hv01.front.sepia.ceph.com 3 6 377 28 +734us[ +725us] +/- 48ms 2024-09-10T08:59:49.055 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 30 +1583us[+1574us] +/- 70ms 2024-09-10T08:59:49.055 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 -3857us[-3866us] +/- 96ms 2024-09-10T08:59:49.056 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:59:49.057 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T08:59:49.057 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2024-09-10T08:59:49.057 INFO:teuthology.orchestra.run.smithi089.stdout:^* hv01.front.sepia.ceph.com 3 6 377 28 +764us[ +773us] +/- 48ms 2024-09-10T08:59:49.057 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 26 +1615us[+1615us] +/- 70ms 2024-09-10T08:59:49.057 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 -3824us[-3816us] +/- 96ms 2024-09-10T08:59:49.058 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:59:49.059 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T08:59:49.099 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T08:59:49.099 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T08:59:49.137 DEBUG:teuthology.orchestra.run.smithi089:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T08:59:49.268 DEBUG:teuthology.orchestra.run.smithi111:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T08:59:49.365 DEBUG:teuthology.orchestra.run.smithi149:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T08:59:49.456 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T08:59:49.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:59:49.518 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T08:59:49.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:59:49.549 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T08:59:49.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:59:49.578 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T08:59:49.590 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T08:59:49.631 INFO:teuthology.task.internal:Duration was 2314.042719 seconds 2024-09-10T08:59:49.632 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T08:59:49.712 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T08:59:49.712 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T08:59:49.716 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T08:59:49.719 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T08:59:49.761 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T08:59:49.779 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T08:59:49.799 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T08:59:50.272 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T08:59:50.273 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi089.front.sepia.ceph.com 2024-09-10T08:59:50.273 DEBUG:teuthology.orchestra.run.smithi089:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T08:59:50.322 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi111.front.sepia.ceph.com 2024-09-10T08:59:50.322 DEBUG:teuthology.orchestra.run.smithi111:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T08:59:50.352 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi149.front.sepia.ceph.com 2024-09-10T08:59:50.352 DEBUG:teuthology.orchestra.run.smithi149:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T08:59:50.375 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T08:59:50.375 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T08:59:50.380 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T08:59:50.395 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T08:59:50.441 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T08:59:50.442 DEBUG:teuthology.orchestra.run.smithi089:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T08:59:50.446 DEBUG:teuthology.orchestra.run.smithi111:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T08:59:50.466 DEBUG:teuthology.orchestra.run.smithi149:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T08:59:51.501 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T08:59:51.512 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T08:59:51.512 DEBUG:teuthology.orchestra.run.smithi089:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T08:59:51.569 DEBUG:teuthology.orchestra.run.smithi111:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T08:59:51.612 DEBUG:teuthology.orchestra.run.smithi149:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T08:59:51.659 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T08:59:51.671 DEBUG:teuthology.orchestra.run.smithi089:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T08:59:51.676 DEBUG:teuthology.orchestra.run.smithi111:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T08:59:51.679 DEBUG:teuthology.orchestra.run.smithi149:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T08:59:51.705 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = core 2024-09-10T08:59:51.715 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = core 2024-09-10T08:59:51.742 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = core 2024-09-10T08:59:51.772 DEBUG:teuthology.orchestra.run.smithi089:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T08:59:51.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:59:51.805 DEBUG:teuthology.orchestra.run.smithi111:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T08:59:51.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:59:51.821 DEBUG:teuthology.orchestra.run.smithi149:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T08:59:51.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:59:51.837 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T08:59:51.849 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T08:59:51.850 DEBUG:teuthology.misc:Transferring archived files from smithi089:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898917/remote/smithi089 2024-09-10T08:59:51.850 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T08:59:51.937 DEBUG:teuthology.misc:Transferring archived files from smithi111:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898917/remote/smithi111 2024-09-10T08:59:51.937 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T08:59:51.976 DEBUG:teuthology.misc:Transferring archived files from smithi149:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898917/remote/smithi149 2024-09-10T08:59:51.976 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T08:59:52.023 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T08:59:52.023 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T08:59:52.027 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T08:59:52.029 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T08:59:52.073 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T08:59:52.089 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T08:59:52.089 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T08:59:52.124 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T08:59:52.124 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T08:59:52.129 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T08:59:52.132 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T08:59:52.146 INFO:teuthology.orchestra.run.smithi111.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 08:59 /home/ubuntu/cephtest 2024-09-10T08:59:52.149 INFO:teuthology.orchestra.run.smithi149.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 08:59 /home/ubuntu/cephtest 2024-09-10T08:59:52.160 INFO:teuthology.orchestra.run.smithi089.stdout: 393574 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 08:59 /home/ubuntu/cephtest 2024-09-10T08:59:52.162 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T08:59:52.174 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T08:59:52.250 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} duration: 2314.0427186489105 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_rishabh@teuthology status: fail success: false 2024-09-10T08:59:52.251 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T08:59:52.315 INFO:teuthology.run:FAIL